I'm surprised your use of an hdfs URL for job tracker works at all.
Usually you specify jobtracker as host:port. See hadoop doc. for examples.
St.Ack
Yair Even-Zohar wrote:
I actually have
<property>
<name>mapred.job.tracker</name>
<value>hdfs://sb-centercluster01:9101</value>
</property>
<property>
In my hadoop-site.xml. Is using hdfs:..... indentify it as local?
Thanks
-Yair
-----Original Message-----
From: Andrew Purtell [mailto:[EMAIL PROTECTED]
Sent: Thursday, July 10, 2008 8:21 PM
To: [email protected]
Subject: RE: Slow mapreduce using Hbase , regardless on number of
machines
You have Hadoop configured to run mapreduce in local mode.
In your hadoop-site.xml configuration, set mapred.job.tracker to the
network address of the jobtracker for your cluster. For example:
<property>
<name>mapred.job.tracker</name>
<value>192.168.1.1:50001</value>
</property>
- Andy
--- On Wed, 7/9/08, Yair Even-Zohar <[EMAIL PROTECTED]> wrote:
From: Yair Even-Zohar <[EMAIL PROTECTED]>
Subject: RE: Slow mapreduce using Hbase , regardless on number of
machines
To: "Andrew Purtell" <[EMAIL PROTECTED]>,
[email protected]
Date: Wednesday, July 9, 2008, 12:35 PM
This helps, thanks. I decreased hbase.hregion.max.filesize
to 67M and
increased the size of my table to around 500,000 so I
finally get
several tasks.
However, they don't seem to be parallel (see below) am
I doing anything wrong or is that the way it supposed to be?
Thanks
-Yair
08/07/08 22:27:37 INFO jvm.JvmMetrics: Initializing JVM
Metrics with processName=JobTracker, sessionId=
08/07/08 22:27:38 INFO mapred.JobClient: Running job: job_local_1
08/07/08 22:27:38 INFO mapred.MapTask: numReduceTasks: 1
08/07/08 22:27:38 INFO hbase.HTable: Creating scanner over ase
starting
at key
08/07/08 22:27:39 INFO mapred.JobClient: map 0% reduce 0%
08/07/08 22:27:44 INFO mapred.LocalJobRunner:
08/07/08 22:27:47 INFO mapred.LocalJobRunner:
08/07/08 22:27:50 INFO mapred.LocalJobRunner:
08/07/08 22:27:53 INFO mapred.LocalJobRunner:
08/07/08 22:27:56 INFO mapred.LocalJobRunner:
08/07/08 22:27:59 INFO mapred.LocalJobRunner:
08/07/08 22:28:02 INFO mapred.LocalJobRunner:
08/07/08 22:28:05 INFO mapred.LocalJobRunner:
08/07/08 22:28:08 INFO mapred.LocalJobRunner:
08/07/08 22:28:11 INFO mapred.LocalJobRunner:
08/07/08 22:28:14 INFO mapred.LocalJobRunner:
08/07/08 22:28:17 INFO mapred.LocalJobRunner:
08/07/08 22:28:20 INFO mapred.LocalJobRunner:
08/07/08 22:28:20 INFO mapred.TaskRunner: Task 'job_local_1_map_0000'
done.
