Yair Even-Zohar wrote: > I took the old hbase 0.2.0 EC2 scripts and modified them to work with > Hadoop 0.18.2 / hbase 0.18.1. > Things seemed to be working and I can see > the cluster with the different regions etc. of both the hadoop port > 50030 and hbase 60010. > > I will release these scripts once they are working properly. > Good stuff.
> My problem is that whenever I execute code on the master node it is > running as local. > > "09/01/13 09:22:40 INFO mapred.JobClient: Running job: job_local_0001" > > That is, the table is generated , the data gets in the table but I get a > "local" indication in the job name and I see no reflection of the actual > Map reduce via the web interface at port 50030. Looking at your output below, hbase looks like its set up right. You have more than just a single server participating in your hbase cluster. So issue is that when you launch a job, its not running on the distributed MR cluster? Instead its running all in a single process? When you launch the job, where you doing it from? The conf dir with the below hadoop-site.xml is not being overshadowed by another hadoop-site.xml? Is there a hadoop-site.xml in your job jar? St.Ack On Tue, Jan 13, 2009 at 10:58 AM, Yair Even-Zohar <[email protected]>wrote: > I took the old hbase 0.2.0 EC2 scripts and modified them to work with > Hadoop 0.18.2 / hbase 0.18.1. Things seemed to be working and I can see > the cluster with the different regions etc. of both the hadoop port > 50030 and hbase 60010. > > I will release these scripts once they are working properly. > > > > > > My problem is that whenever I execute code on the master node it is > running as local. > > "09/01/13 09:22:40 INFO mapred.JobClient: Running job: job_local_0001" > > That is, the table is generated , the data gets in the table but I get a > "local" indication in the job name and I see no reflection of the actual > Map reduce via the web interface at port 50030. > > > > I attach the hadoop-site and hbase site file here: > > > > HADOOP > > <?xml version="1.0"?> > > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> > > > > <configuration> > > > > <property> > > <name>hadoop.tmp.dir</name> > > <value>/mnt/hadoop</value> > > </property> > > > > <property> > > <name>fs.default.name</name> > > <value>hdfs://domU-12-31-39-00-13-A1.compute-1.internal:50001</value> > > </property> > > > > <property> > > <name>mapred.job.tracker</name> > > <value>domU-12-31-39-00-13-A1.compute-1.internal:50002</value> > > </property> > > > > <property> > > <name>tasktracker.http.threads</name> > > <value>80</value> > > </property> > > > > <property> > > <name>mapred.tasktracker.map.tasks.maximum</name> > > <value>4</value> > > </property> > > > > <property> > > <name>mapred.tasktracker.reduce.tasks.maximum</name> > > <value>4</value> > > </property> > > > > <property> > > <name>mapred.output.compress</name> > > <value>true</value> > > </property> > > > > <property> > > <name>mapred.output.compression.type</name> > > <value>BLOCK</value> > > </property> > > > > <property> > > <name>dfs.client.block.write.retries</name> > > <value>3</value> > > </property> > > > > > > > > > > HBASE > > <?xml version="1.0"?> > > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> > > > > <configuration> > > <property> > > <name>hbase.master</name> > > <value>domU-12-31-39-00-13-A1.compute-1.internal:60000</value> > > </property> > > > > <property> > > <name>hbase.rootdir</name> > > > <value>hdfs://domU-12-31-39-00-13-A1.compute-1.internal:50001/hbase</val > ue> > > </property> > > </configuration> > > > > > > > > > > > > Here is the Captured printscreen using Elasticfox > > > Master Attributes > > Attribute Name > > Value > > Description > > HBase Version > > 0.18.1, r707159 > > HBase version and svn revision > > HBase Compiled > > Wed Oct 22 12:43:06 PDT 2008, stack > > When HBase version was compiled and by whom > > Hadoop Version > > 0.18.1, r694836 > > Hadoop version and svn revision > > Hadoop Compiled > > Fri Sep 12 23:29:35 UTC 2008, hadoopqa > > When Hadoop version was compiled and by whom > > Filesystem > > hdfs://domU-12-31-39-00-13-A1.compute-1.internal:50001/hbase > > Filesystem HBase is running on > > HBase Root Directory > > hdfs://domU-12-31-39-00-13-A1.compute-1.internal:50001/hbase > > Location of HBase home directory > > Load average > > 1.0 > > Average load across all region servers. Naive computation. > > > Catalog Tables > > Table > > Description > > -ROOT- > <http://ec2-174-129-169-159.compute-1.amazonaws.com:60010/table.jsp?name > =-ROOT-<http://ec2-174-129-169-159.compute-1.amazonaws.com:60010/table.jsp?name=-ROOT-> > > > > The -ROOT- table holds references to all .META. regions. > > .META. > <http://ec2-174-129-169-159.compute-1.amazonaws.com:60010/table.jsp?name > =.META<http://ec2-174-129-169-159.compute-1.amazonaws.com:60010/table.jsp?name=.META> > .> > > The .META. table holds references to all User Table regions > > > User Tables > > > 1 table(s) in set. > > Table > > Description > > ase > <http://ec2-174-129-169-159.compute-1.amazonaws.com:60010/table.jsp?name > =ase<http://ec2-174-129-169-159.compute-1.amazonaws.com:60010/table.jsp?name=ase> > > > > {NAME => 'ase', IS_ROOT => 'false', IS_META => 'false', FAMILIES => > [{NAME => 'URL', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS > => '2147483647', LENGTH => '2147483647', TTL => '7257600', IN_MEMORY => > 'false', BLOCKCACHE => 'false'}, {NAME => 'CATEGORY', BLOOMFILTER => > 'false', COMPRESSION => 'NONE', VERSIONS => '2147483647', LENGTH => > '2147483647', TTL => '7257600', IN_MEMORY => 'false', BLOCKCACHE => > 'false'}, {NAME => 'SEARCHTERM', BLOOMFILTER => 'false', COMPRESSION => > 'NONE', VERSIONS => '2147483647', LENGTH => '2147483647', TTL => > '7257600', IN_MEMORY => 'false', BLOCKCACHE => 'false'}]} > > > Region Servers > > > > Address > > Start Code > > Load > > domU-12-31-39-02-CD-82.compute-1.internal:60020 > <http://domU-12-31-39-02-CD-82.compute-1.internal:60030/> > > 1231836487534 > > requests: 0 regions: 2 > > domU-12-31-39-02-10-91.compute-1.internal:60020 > <http://domU-12-31-39-02-10-91.compute-1.internal:60030/> > > 1231836848289 > > requests: 0 regions: 1 > > domU-12-31-39-00-13-A1.compute-1.internal:60020 > <http://domU-12-31-39-00-13-A1.compute-1.internal:60030/> > > 1231853256586 > > requests: 0 regions: 0 > > Total: > > servers: 3 > > > > requests: 0 regions: 3 > > Load is requests per second and count of regions loaded > > > >
