Yeah. Thats downside to using HADOOP_CLASSPATH. If its contents change, processes that depend on it need relaunch.
If you don't want to change per hbase version, bundle hbase into you job jar? From http://hbase.apache.org/docs/r0.20.6/api/org/apache/hadoop/hbase/mapreduce/package-summary.html#classpath "Another possibility, if for example you do not have access to hadoop-env.sh or are unable to restart the hadoop cluster, is bundling the hbase jars into a mapreduce job jar adding it and its dependencies under the job jar lib/ directory and the hbase conf into the job jars top-level directory." You do not need to restart hdfs. You'd just need to restart tasktrackers since these are the doohickeys running the MR jobs that depend on hbase. St.Ack On Fri, Aug 20, 2010 at 5:14 PM, Sharma, Avani <[email protected]> wrote: > Thanks. After I upgrade to 0.20.6, I have to go and change the > HADOOP_CLASSPATH in hadoop-env.sh which points to hbase jars, since the names > of the jars have changed : /usr/local/hbase-0.20.6/hbase-0.20.6.jar. > > Will I have to restart hdfs after the change - this would mean upgrade > required hbase restart and hdfs restart ? > I am guessing there might be a simpler way to do this. > > -Avani > > -----Original Message----- > From: [email protected] [mailto:[email protected]] On Behalf Of Stack > Sent: Thursday, August 19, 2010 8:53 PM > To: [email protected] > Subject: Re: improving random read performance > > On Thu, Aug 19, 2010 at 5:53 PM, Sharma, Avani <[email protected]> wrote: >> >> I am currently running under Hbasev0.20.3. I increased the block cache to .4 >> from .2. The heap size is 2GB. >> The default regionserver handler count is 25 in hbase-default.xml. Will try >> LZO compression. >> > > Up your handlers to 100. Up your block cache even more... .5 or .6 > even if you mostly random reading. Can you give it more RAM than 4G? > > >> What are other performance tunings I can do ? >> In particular, will applying HBASE-2180 or upgrading to a newer version help >> ? >> > > Yes, hbase-2180 will make a difference. You should upgrade to 0.20.6 > to catch 2180 and other fixes. > >> Another thing I notice is that the performance via stargate vs Java api is >> comparable for random reads.I thought stargate would have some latency. Is >> this expected? > > I'd expect come tax going via REST. > > St.Ack >
