Nereus: Massively Parallel Computation in Java
 |
||
Google Tech TalksMay, 8 2008ABSTRACTWith 1 Billion idle PC's in the world, there is approximately $100 Billion's worth of CPU time going to waste every year. Given that insatiable global demand for computation is fuelling massive datacentre proliferation, with consequential environmental impact, surely there must be a way to leverage idle time on desktops to relieve some of this pressure?In the past, the term "Grid Computing" has been used to encompass the idea of using idle desktop capacity productively. Unfortunately "Grid Technology" has yet to provide a solution for this global problem, especially for the vast majority of machines which are outside the corporate firewall. Nevertheless, there is a class of problems which are ideally suited for idle desktops and can leverage millions of machines if managed effectively. We use the phrase "Massively Parallel Computation" to describe such tasks, by their nature beyond the scale of any datacentre, but also beyond the capabilities of virtually all grid software too!Nereus is a new, open source, pure Java MPC platform which finally offers an easy way to use the world's idle computers. Joining a computational project running on Nereus is as simple as clicking a link in a web page, and it provides a secure sandbox to protect the host computer from malicious software -- just like the Applet sandbox in a browser. Developing a Nereus application is also simple; it's virtually identical to building a servlet for a J2EE container, and the progamming concepts will be familiar to all Java developers (J2EE focussed or not). Nereus applications are as simple to write as any Java applet; one JAR on the classpath, one class to extend, and a small but powerful API.Perhaps most importantly, Nereus provides a simple way for owners to delegate control of their machine to resellers, who are then able to group machines together and offer attractive resources to customers. This feature also provides for the owner getting paid for this 3rd party use -- something drastically different from many MPC projects which require people to donate their spare computer time (e.g. SETI@Home).This talk will describe Nereus, its architecture and implementation details, as well as a step-by-step demonstration of building a "Hello Nereus" MPC application. Other demos include showing a MPC 3D rendering application -- animating a movie lightning fast. The audience will also be invited to join a Nereus MPC application created and started within the talk, using their laptops from where they sit. We can then all experience the CPU power unleashed by simply sitting in a talk!Nereus is simple x 3; simple to deploy, simple to join, simple to use. By demonstrating this during the talk, and by showing the successful work being done at Oxford University using it we hope to simulate interest in the more widespread use of this open source technology in the next generation web. Oh, and what about using "native" code on Nereus resources? We will show how, still within the full Java security sandbox, by using the JPC pure java x86 emulator (see JavaOne 2007/2008 and http://www-jpc.physics.ox.ac.u... )Further information about Nereus can be found at http://www-nereus.physics.ox.a... Speaker: Dr Rhys NewmanChief Architect IGD Group, Dept Physics Oxford University (JavaOne "Rock Star 2007")Rhys joined the IGD group in 2004 after spending over 20 years as a programmer in industry and academia. The first 10 were in C/C++ but since then he has focused exclusively on Java technology. Rhys got his DPhil from Oxford University in 1998 and since then has worked in several technology start-ups and academic positions.Speaker: Dr Jeff TsengGroup Lead, Grid Technology, Dept Physics Oxford University.Jeff formed the IGD group in 2003 when he arrived at Oxford to take up his post as University Lecturer in Physics. Before that, he was a research scientist at MIT and Fermilab working on the CDF particle physics experiment, where he built data acquisition systems out of off-the-shelf PC's and was in charge of the experiment's data handling systems and initial forays into grid com...
Canal: People & Blogs
Añadido: November 30, 1999 at 12:00 am
Autor: googletechtalks
Duración: 27:31
Puntuación: 4.30
Reproducciones: 3978
Etiquetas: education engedu google googletechtalks talk talks techtalk techtalks
Comentarios
|
rhysnewman (November 30, 1999 at 12:00 am)
Indeed there are many similar projects, many more recent than 2002 (when it seems activity on the Bayanihan project stopped). However there are critical differences which make Nereus+JPC unique and substantially more powerful; just that with the time allowed in this talk I could not devote time to contrasting directly with/to background/previous work. The manifest absence today of a global MPC grid demonstrates that previous work is lacking; Nereus+JPC addresses these gaps like nothing before it
kagemane10 (November 30, 1999 at 12:00 am)
This is an old topic...You should explore the BAYANIHAN PROJECT...With both Java and C# implementations...Nereus is just a big rip off version of the Philippines' BAYANIHAN Project.
debeastie (November 30, 1999 at 12:00 am)
Security to one side a moment. As nothing is ever perfect. Would be good to get it Pen tested though!I'm really impressed with the potential for this application. It's a bit like SETI DCN on amphetamines. thanks/debeastie
rhysnewman (November 30, 1999 at 12:00 am)
Our experience is that there is almost *no* amount of money which we could pay for someone to download and execute an arbitary executable. If I paid you $10 would you run a binary from me? Remember it could be a tojan, keylogger, virus.....would you do it for $100, $1000?However it is also true that many users care a lot about data privacy, and so they would not use Nereus for that reason. Its a balance of this privacy concern vs the *huge* processing power of the grid.
natlang1 (November 30, 1999 at 12:00 am)
Privacy is a subset of security. The public would be happy to run code with *no* sandbox if they get paid for it. However, the industries that can/would pay for such a service (medicine, finance, government, Hollywood, transportation, energy) do have security concerns. Without data security, the vast majority of problems can't be delegated to a public grid. Those are the profitable problems, and with out them, all grids will remain at least three orders of magnitude smaller than the potential.
rhysnewman (November 30, 1999 at 12:00 am)
We use "Security" in terms of the possibility guest code steals data from the host machine (or otherwise interferes with it). Your concern we label "Privacy"; and no, JPC does not provide a cast-iron guarantee on privacy (although it would be pretty difficult to pick apart the JVM, then JPC and then the x86 guest OS and then the memory of the guest application). Nevertheless we make no sweeping claims about Privacy - just that JPC and the JVM as a "Security" solution is really bullet-proof
rhysnewman (November 30, 1999 at 12:00 am)
JPC peaks currently at 20% of the speed of the underlying hardware. We know, from initial tests and some very helpful discussions with Sun engineers, that we can double this to over 40%. So your concern, while valid for most emulators, is not a major issue for JPC. In fact some code can actually run *faster* inside JPC as older x86 code is automatically re-compiled by JPC and Sun's Hotspot JVM
djdaedulus (November 30, 1999 at 12:00 am)
Screw that, that leaves people in the higher hierarchy of the IT departments who happen to be in the pockets of government agencies . The opportunity to seize control of PC's for the use of whatever they decide. The same goes along with having to put in your email address everywhere to register on the web and then they sell that info. But they always say yeah trust us. It will be ok. Also in most web contracts they leave open we can modify any part of the contract we want.
natlang1 (November 30, 1999 at 12:00 am)
If Java isn't secure enough, running an emulator written in Java will not be more secure. The real security problem is not to hide the hardware; it is to hide the data. For an anonymous grid, we need algorithms that can run in parallel without revealing any decipherable portions of data to the clients and without the clients being able to inject false results. The "sandbox" is mostly a side effect of the solution to those problems. We certainly don't need sandboxes within sandboxes within...
MiniBenoit (November 30, 1999 at 12:00 am)
I'm really skeptical about the jPC part.Even though it is certainly a nice way to learn things about JIT and x86, to me it really looks like another way to divide your processing power by 10...
|