> > since mapReduce assumes that all the code (and the data) is in place in the > respective nodes. >
As far as I can tell, the Hadoop, the open source implementation of map reduce, doesn't require your map reduce code to be in all nodes. It copies the jar files of the your application to the nodes that would execute the code. http://hadoop.apache.org/core/docs/current/mapred_tutorial.html#Job+Submission+and+Monitoring Data is also generally available via a network protocol such as http. I am wondering if distribution tools (such as Cabal) would be preferable to hadoop model. Instead of having the framework copy the code why not use the existing tools to install the code prior to running the map-reduce? daryoush 2009/2/25 Alberto G. Corona <agocor...@gmail.com> > Galchin, > > Maybe you are asking not only about remote execution, but also mobility of > code. This is a problem that is previous to mapReduce, since mapReduce > assumes that all the code (and the data) is in place in the respective > nodes. In fact, the distribution of resources in order to efficiently use > mapReduce is a design problem that the google people has done by hand. > But my intuition says that there are a general algorithm for distribution > of code, data, bandwidth and resources in general that moves around at > execution time to achieve better and better performance for a given grid of > nodes and for any task, for example, a mapReduce task. I would be very > interesting to read something about this. > > I know that some efforts have been carried out the past , for example > mobile haskell > <http://homepages.inf.ed.ac.uk/stg/workshops/TFP/book/DuBois/duboismhaskell/cameraready.pdf%C2%A0> > > > http://homepages.inf.ed.ac.uk/stg/workshops/TFP/book/DuBois/duboismhaskell/cameraready.pdf > > <http://homepages.inf.ed.ac.uk/stg/workshops/TFP/book/DuBois/duboismhaskell/cameraready.pdf%C2%A0> > > > which is a first step for this goal but I this has been discontinued and > the source code is not available. > > 2009/2/25 Galchin, Vasili <vigalc...@gmail.com> > > > > Hello, > > > > Here is an interesting paper of Google's MapReduce reverse > engineered into Haskell. I apologize if already posted ..... > http://www.cs.vu.nl/~ralf/MapReduce/<http://www.cs.vu.nl/%7Eralf/MapReduce/> > > > > Kind regards, Vasili > > > > _______________________________________________ > > Haskell-Cafe mailing list > > Haskell-Cafe@haskell.org > > http://www.haskell.org/mailman/listinfo/haskell-cafe > > > > > _______________________________________________ > Haskell-Cafe mailing list > Haskell-Cafe@haskell.org > http://www.haskell.org/mailman/listinfo/haskell-cafe > >
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe