Yes, we've constructed a local version of a hadoop process, We needed 500 input files in hadoop to reach the speed of local process, total time was 82 seconds in a cluster of 6 machines. And I think it's a good performance among other distributed processing systems.
2009/4/2 jason hadoop <jason.had...@gmail.com> > 3) The framework is designed for working on large clusters of machines > where > there needs to be a little delay between operations to avoid massive > network > loading spikes, and the initial setup of the map task execution environment > on a machine, and the initial setup of the reduce task execution > environment > take a bit of time. > In production jobs, these delays and setup times are lost in the overall > task run time. > In the small test job case the delays and setup times will be the bulk of > the time spent executing the test. > > >
