Hello Michael , thank you for the hint
2012/5/22 Michael Segel <[email protected]> > Hi, > > Seems we just had someone talk about this just the other day... > > 1) 8GB of memory isn't enough to run both M/R and HBase. > Ok, yes you can run it, however don't expect it to perform well. > > 2) You never want a user to run their own code from the cluster itself. > Use an *edge* node. > > There's more, but you get the idea. > > On May 21, 2012, at 2:04 PM, Faruk Berksöz wrote: > > > Dear All, > > we have 4 node in our cluster (1nn+dn,3 dn). > > Hadoop dist. is cdh3u3. > > Every node has 2 tb disk , 8 gb memory. > > We are trying some insert performance test on hbase. > > I have tried to insert 250.000 records from hbase master (without > thread), > > that takes 5-7 sec. > > But when I try ro insert from any regionserver the same data (250.000) > , > > it takes longer 21 sec. > > Is that normal ? > > > > any response would be appreciated.. > > > > > > my java code looks like: > > ......... > > ......... > > ......... > > long start = System.currentTimeMillis(); > > //DateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd > > HH:mm:ss"); > > //Date sdate = new Date(); > > > > //LOG.info(" Start-Time :" + dateFormat.format(sdate)); > > for (int i = 0; i < eachsize; i++) { > > > > > > put = new Put(String.format("%016x", > > random.nextLong()).getBytes()); > > > > put.setWriteToWAL(false); > > //set column and their values > > addColumnAndValues(); > > table.put(put); > > > > > > } > > elapsedTimeMillis = System.currentTimeMillis() - start; > > elapsedTimeSec = elapsedTimeMillis / 1000F; > > elapsedTimeSecInMemory = elapsedTimeSec ; > > LOG.info(" Elapsed-Time (sec) inMemory :" + elapsedTimeSec); > > > > table.flushCommits(); > > //disk write elapsed time > > elapsedTimeMillis = System.currentTimeMillis() - start; > > elapsedTimeSec = elapsedTimeMillis / 1000F; > > elapsedTimeSecDiskWrite = elapsedTimeSec; > > LOG.info(" Elapsed-Time (sec) Disk Write:" + > > elapsedTimeSec); > > ......... > > ......... > > > > My hbase-site.xml is looks like : > > <configuration> > > <property> > > <name>hbase.rootdir</name> > > <value>hdfs://master.bigdata.com:54310/hbase</value> > > </property> > > <property> > > <name>hbase.cluster.distributed</name> > > <value>true</value> > > </property> > > <property> > > <name>hbase.zookeeper.quorum</name> > > <value>master.bigdata.com,slave1.bigdata.com,slave2.bigdata.com, > > slave3.bigdata.com</value> > > </property> > > <property> > > <name>hbase.zookeeper.dns.interface</name> > > <value>eth0</value> > > </property> > > <property> > > <name>hbase.zookeeper.dns.nameserver</name> > > <value>10.10.10.1</value> > > </property> > > > > <property> > > <name>hbase.regionserver.handler.count</name> > > <value>20</value> > > </property> > > > > <property> > > <name>hbase.client.write.buffer</name> > > <value>5097152</value> > > </property> > > </configuration> > >
