Elia, perhaps you can try changing "mapred.tasktracker.map.tasks.maximum" and "mapred.tasktracker.reduce.tasks.maximum" to "4" in hadoop-site.xml in hopes of getting better utilization. It's strange to me that having these both set to 2 only utilizes a single core, because I would imagine that any modern OS scheduler would do a good job of core utilization.
Just a thought. Alex On Wed, Oct 8, 2008 at 12:52 AM, Taeho Kang <[EMAIL PROTECTED]> wrote: > First of all, "mapred.tasktracker.map.tasks.maximum" and > "mapred.tasktracker.reduce.tasks.maximum" are both set to 2 in > hadoop-default.xml file; this file is read before hadoop-site.xml file so > any properties that aren't set in hadoop-site.xml will follow the values > set > in hadoop-default.xml. > As for the question on why only one core is utilized... > I think it really depends on the process scheduling of the underlying OS. > It's not like two tasks (two JVM subprocesses spawned by the tasktracker) > will always run on independent cores as there are other processes which > need > one or more cores to be run. > > By the way, what tools did you use to find out which tasks (or processes) > use which cores? > > /Taeho > > > On Wed, Oct 8, 2008 at 1:01 PM, Alex Loddengaard > <[EMAIL PROTECTED]>wrote: > > > Taeho, I was going to suggest this change as well, but it's documented > that > > "mapred.tasktracker.map.tasks.maximum" defaults to 2. Can you explain > why > > Elia is only having one core utilized when this config option is set to > 2? > > > > Here is the documentation I'm referring to: > > <http://hadoop.apache.org/core/docs/r0.18.1/cluster_setup.html> > > > > Alex > > > > On Tue, Oct 7, 2008 at 8:27 PM, Taeho Kang <[EMAIL PROTECTED]> wrote: > > > > > You can have your node (tasktracker) running more than 1 task > > > simultaneously. > > > You may set "mapred.tasktracker.map.tasks.maximum" and > > > "mapred.tasktracker.reduce.tasks.maximum" properties found in > > > hadoop-site.xml file. You should change hadoop-site.xml file on all > your > > > slave nodes depending on how many cores each slave has. For example, > you > > > don't really want to have 8 tasks running at once on a 2 core machine. > > > > > > /Taeho > > > > > > On Wed, Oct 8, 2008 at 5:53 AM, Elia Mazzawi > > > <[EMAIL PROTECTED]>wrote: > > > > > > > hello, > > > > > > > > I have some dual core nodes, and I've noticed hadoop is only running > 1 > > > > instance, and so is only using 1 on the CPU's on each node. > > > > is there a configuration to tell it to run more than once? > > > > or do i need to turn each machine into 2 nodes? > > > > > > > > Thanks. > > > > > > > > > >
