Thanks. That solved the problem :-) -Yair
-----Original Message----- From: stack [mailto:[EMAIL PROTECTED] Sent: Friday, July 11, 2008 11:38 AM To: [email protected] Subject: Re: Slow mapreduce using Hbase , regardless on number of machines I'm surprised your use of an hdfs URL for job tracker works at all. Usually you specify jobtracker as host:port. See hadoop doc. for examples. St.Ack Yair Even-Zohar wrote: > I actually have > <property> > <name>mapred.job.tracker</name> > <value>hdfs://sb-centercluster01:9101</value> > </property> > <property> > > In my hadoop-site.xml. Is using hdfs:..... indentify it as local? > > Thanks > -Yair > > -----Original Message----- > From: Andrew Purtell [mailto:[EMAIL PROTECTED] > Sent: Thursday, July 10, 2008 8:21 PM > To: [email protected] > Subject: RE: Slow mapreduce using Hbase , regardless on number of > machines > > You have Hadoop configured to run mapreduce in local mode. > > In your hadoop-site.xml configuration, set mapred.job.tracker to the > network address of the jobtracker for your cluster. For example: > > <property> > <name>mapred.job.tracker</name> > <value>192.168.1.1:50001</value> > </property> > > - Andy > > --- On Wed, 7/9/08, Yair Even-Zohar <[EMAIL PROTECTED]> wrote: > > >> From: Yair Even-Zohar <[EMAIL PROTECTED]> >> Subject: RE: Slow mapreduce using Hbase , regardless on number of >> machines >> To: "Andrew Purtell" <[EMAIL PROTECTED]>, >> > [email protected] > >> Date: Wednesday, July 9, 2008, 12:35 PM >> This helps, thanks. I decreased hbase.hregion.max.filesize >> to 67M and >> increased the size of my table to around 500,000 so I >> finally get >> several tasks. >> >> However, they don't seem to be parallel (see below) am >> I doing anything wrong or is that the way it supposed to be? >> >> Thanks >> -Yair >> >> >> 08/07/08 22:27:37 INFO jvm.JvmMetrics: Initializing JVM >> Metrics with processName=JobTracker, sessionId= >> 08/07/08 22:27:38 INFO mapred.JobClient: Running job: job_local_1 >> 08/07/08 22:27:38 INFO mapred.MapTask: numReduceTasks: 1 >> 08/07/08 22:27:38 INFO hbase.HTable: Creating scanner over ase >> > starting > >> at key >> 08/07/08 22:27:39 INFO mapred.JobClient: map 0% reduce 0% >> 08/07/08 22:27:44 INFO mapred.LocalJobRunner: >> 08/07/08 22:27:47 INFO mapred.LocalJobRunner: >> 08/07/08 22:27:50 INFO mapred.LocalJobRunner: >> 08/07/08 22:27:53 INFO mapred.LocalJobRunner: >> 08/07/08 22:27:56 INFO mapred.LocalJobRunner: >> 08/07/08 22:27:59 INFO mapred.LocalJobRunner: >> 08/07/08 22:28:02 INFO mapred.LocalJobRunner: >> 08/07/08 22:28:05 INFO mapred.LocalJobRunner: >> 08/07/08 22:28:08 INFO mapred.LocalJobRunner: >> 08/07/08 22:28:11 INFO mapred.LocalJobRunner: >> 08/07/08 22:28:14 INFO mapred.LocalJobRunner: >> 08/07/08 22:28:17 INFO mapred.LocalJobRunner: >> 08/07/08 22:28:20 INFO mapred.LocalJobRunner: >> 08/07/08 22:28:20 INFO mapred.TaskRunner: Task 'job_local_1_map_0000' >> done. >> > > > > >
