How about writing the population to the file and using it as input to map-reduce directly? Evaluations that fit into a map can obviously handle that. Evaluations that need to be parallelized, can have the map emit multiple copies with sequential keys. This will put a copy of each input into multiple reduces that can do a piece of the evaluation. The pieces can be put back together in a second MR step.
I even think that you should be able to demonstrate that any MR program that evaluates a single population member can be applied to multiple population members with slight modifications. On Wed, May 28, 2008 at 11:44 AM, deneche abdelhakim <[EMAIL PROTECTED]> wrote: > > > I don't know if there are many ways to do it in Hadoop, but how about > writing the population into a file and pass it with the DistributedCache ? > > -- ted
