On 9/30/07, Ger-Jan te Dorsthorst <[EMAIL PROTECTED]> wrote: > Not sure whether this is the right place to ask, but I'll give it a go: > > Is Hadoop able to distribute tasks to individual cores in multicore > nodes? I realise the framework is designed for running on a large number > of unreliable networked nodes, which I suppose is not so much of an > issue when it comes to using multiple CPU cores within one system. But > Hadoop nodes may already be dual or quad core machines. It would be > great if developers could harness this power transparently, I mean > without having to worry about the details of distribution.
In short, yes. Hadoop's code takes advantage of multiple native threads and you can tune the level of concurrency in the system by setting mapred.map.tasks and mapred.reduce.tasks to take advantage of multiple cores on the nodes which have them. -- Toby DiPasquale
