Hi,
Seems we just had someone talk about this just the other day...
1) 8GB of memory isn't enough to run both M/R and HBase.
Ok, yes you can run it, however don't expect it to perform well.
2) You never want a user to run their own code from the cluster itself. Use an
*edge* node.
There's more, but you get the idea.
On May 21, 2012, at 2:04 PM, Faruk Berksöz wrote:
> Dear All,
> we have 4 node in our cluster (1nn+dn,3 dn).
> Hadoop dist. is cdh3u3.
> Every node has 2 tb disk , 8 gb memory.
> We are trying some insert performance test on hbase.
> I have tried to insert 250.000 records from hbase master (without thread),
> that takes 5-7 sec.
> But when I try ro insert from any regionserver the same data (250.000) ,
> it takes longer 21 sec.
> Is that normal ?
>
> any response would be appreciated..
>
>
> my java code looks like:
> .........
> .........
> .........
> long start = System.currentTimeMillis();
> //DateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd
> HH:mm:ss");
> //Date sdate = new Date();
>
> //LOG.info(" Start-Time :" + dateFormat.format(sdate));
> for (int i = 0; i < eachsize; i++) {
>
>
> put = new Put(String.format("%016x",
> random.nextLong()).getBytes());
>
> put.setWriteToWAL(false);
> //set column and their values
> addColumnAndValues();
> table.put(put);
>
>
> }
> elapsedTimeMillis = System.currentTimeMillis() - start;
> elapsedTimeSec = elapsedTimeMillis / 1000F;
> elapsedTimeSecInMemory = elapsedTimeSec ;
> LOG.info(" Elapsed-Time (sec) inMemory :" + elapsedTimeSec);
>
> table.flushCommits();
> //disk write elapsed time
> elapsedTimeMillis = System.currentTimeMillis() - start;
> elapsedTimeSec = elapsedTimeMillis / 1000F;
> elapsedTimeSecDiskWrite = elapsedTimeSec;
> LOG.info(" Elapsed-Time (sec) Disk Write:" +
> elapsedTimeSec);
> .........
> .........
>
> My hbase-site.xml is looks like :
> <configuration>
> <property>
> <name>hbase.rootdir</name>
> <value>hdfs://master.bigdata.com:54310/hbase</value>
> </property>
> <property>
> <name>hbase.cluster.distributed</name>
> <value>true</value>
> </property>
> <property>
> <name>hbase.zookeeper.quorum</name>
> <value>master.bigdata.com,slave1.bigdata.com,slave2.bigdata.com,
> slave3.bigdata.com</value>
> </property>
> <property>
> <name>hbase.zookeeper.dns.interface</name>
> <value>eth0</value>
> </property>
> <property>
> <name>hbase.zookeeper.dns.nameserver</name>
> <value>10.10.10.1</value>
> </property>
>
> <property>
> <name>hbase.regionserver.handler.count</name>
> <value>20</value>
> </property>
>
> <property>
> <name>hbase.client.write.buffer</name>
> <value>5097152</value>
> </property>
> </configuration>
