We found that there are two issues about the PerformanceEvaluation class. - Is not match for hadoop-0.20.0. - The approach to split map is not strict. Need to provide correct InputSplit and InputFormat classes.
And we have just modified the org.apache.hadoop.hbase.PerformanceEvaluation for our evaluations. Please get our code at: http://dl.getdropbox.com/u/24074/code/PerformanceEvaluation.java Here is our evaluations of 0.20.0 RC1 http://docloud.blogspot.com/2009/08/hbase-0200-performance-evaluation.html Schubert On Tue, Aug 18, 2009 at 8:37 AM, Jeff Hammerbacher <[email protected]>wrote: > Thanks guys. For the lazy (e.g. me) and future searchers, here are some > links. The benchmark is meant to simulate the same performance tests quoted > in Google's BigTable paper. > > * PerformanceEvaluation wiki page: > http://wiki.apache.org/hadoop/Hbase/PerformanceEvaluation > * PerformanceEvaluation.java: > > http://svn.apache.org/viewvc/hadoop/hbase/trunk/src/test/org/apache/hadoop/hbase/PerformanceEvaluation.java?view=co > > Thanks, > Jeff > > On Mon, Aug 17, 2009 at 5:09 PM, stack <[email protected]> wrote: > > > On Mon, Aug 17, 2009 at 4:54 PM, Jeff Hammerbacher <[email protected] > > >wrote: > > > > > Hey Stack, > > > > > > I notice that the patch for this issue doesn't include any sort of > tests > > > that might have caught this regression. Do you guys have an HBaseBench, > > > HBaseMix, or similarly named tool for catching performance regressions? > > > > > > > Not as part of our build. The way its currently done is that near > release, > > we run our little PerformanceEvaluation doohickey. If its way off, crack > > the profiler. > > > > We have been trying to get some of the hadoop allotment of EC2 time so we > > could set up a regular run up on AWS but no luck so far. > > > > Good on your Jeff, > > St.Ack > > > > > > > > > > > > > > > > > > > > Thanks, > > > Jeff > > > > > > On Mon, Aug 17, 2009 at 4:51 PM, stack <[email protected]> wrote: > > > > > > > Our writes were off by a factor of 7 or 8. Writes should be better > now > > > > (HBASE-1771). > > > > Thanks, > > > > St.Ack > > > > > > > > > > > > On Thu, Aug 13, 2009 at 4:53 PM, stack <[email protected]> wrote: > > > > > > > > > I just tried it. It seems slow to me writing too. Let me take a > > > > look.... > > > > > St.Ack > > > > > > > > > > > > > > > On Thu, Aug 13, 2009 at 10:06 AM, llpind <[email protected]> > > > wrote: > > > > > > > > > >> > > > > >> Okay I changed replication to 2. and removed "-XX:NewSize=6m > > > > >> -XX:MaxNewSize=6m" > > > > >> > > > > >> here is results for randomWrite 3 clients: > > > > >> > > > > >> > > > > >> > > > > >> RandomWrite ================================================= > > > > >> > > > > >> hadoop-0.20.0/bin/hadoop jar hbase-0.20.0/hbase-0.20.0-test.jar > > > > >> --nomapred > > > > >> randomWrite 3 > > > > >> > > > > >> > > > > >> 09/08/13 09:51:15 INFO hbase.PerformanceEvaluation: client-0 Start > > > > >> randomWrite at offset 0 for 1048576 rows > > > > >> 09/08/13 09:51:15 INFO hbase.PerformanceEvaluation: client-1 Start > > > > >> randomWrite at offset 1048576 for 1048576 rows > > > > >> 09/08/13 09:51:15 INFO hbase.PerformanceEvaluation: client-2 Start > > > > >> randomWrite at offset 2097152 for 1048576 rows > > > > >> 09/08/13 09:51:47 INFO hbase.PerformanceEvaluation: client-0 > > > > >> 0/104857/1048576 > > > > >> 09/08/13 09:51:48 INFO hbase.PerformanceEvaluation: client-1 > > > > >> 1048576/1153427/2097152 > > > > >> 09/08/13 09:51:48 INFO hbase.PerformanceEvaluation: client-2 > > > > >> 2097152/2201997/3145728 > > > > >> 09/08/13 09:52:22 INFO hbase.PerformanceEvaluation: client-1 > > > > >> 1048576/1258284/2097152 > > > > >> 09/08/13 09:52:23 INFO hbase.PerformanceEvaluation: client-0 > > > > >> 0/209714/1048576 > > > > >> 09/08/13 09:52:24 INFO hbase.PerformanceEvaluation: client-2 > > > > >> 2097152/2306854/3145728 > > > > >> 09/08/13 09:52:47 INFO hbase.PerformanceEvaluation: client-1 > > > > >> 1048576/1363141/2097152 > > > > >> 09/08/13 09:52:58 INFO hbase.PerformanceEvaluation: client-0 > > > > >> 0/314571/1048576 > > > > >> 09/08/13 09:52:58 INFO hbase.PerformanceEvaluation: client-2 > > > > >> 2097152/2411711/3145728 > > > > >> 09/08/13 09:53:24 INFO hbase.PerformanceEvaluation: client-1 > > > > >> 1048576/1467998/2097152 > > > > >> 09/08/13 09:53:27 INFO hbase.PerformanceEvaluation: client-0 > > > > >> 0/419428/1048576 > > > > >> 09/08/13 09:53:27 INFO hbase.PerformanceEvaluation: client-2 > > > > >> 2097152/2516568/3145728 > > > > >> 09/08/13 09:53:48 INFO hbase.PerformanceEvaluation: client-1 > > > > >> 1048576/1572855/2097152 > > > > >> 09/08/13 09:54:08 INFO hbase.PerformanceEvaluation: client-2 > > > > >> 2097152/2621425/3145728 > > > > >> 09/08/13 09:54:10 INFO hbase.PerformanceEvaluation: client-0 > > > > >> 0/524285/1048576 > > > > >> 09/08/13 09:54:40 INFO hbase.PerformanceEvaluation: client-1 > > > > >> 1048576/1677712/2097152 > > > > >> 09/08/13 09:54:49 INFO hbase.PerformanceEvaluation: client-2 > > > > >> 2097152/2726282/3145728 > > > > >> 09/08/13 09:54:52 INFO hbase.PerformanceEvaluation: client-0 > > > > >> 0/629142/1048576 > > > > >> 09/08/13 09:55:57 INFO hbase.PerformanceEvaluation: client-1 > > > > >> 1048576/1782569/2097152 > > > > >> 09/08/13 09:56:21 INFO hbase.PerformanceEvaluation: client-2 > > > > >> 2097152/2831139/3145728 > > > > >> 09/08/13 09:56:41 INFO hbase.PerformanceEvaluation: client-0 > > > > >> 0/733999/1048576 > > > > >> 09/08/13 09:57:23 INFO hbase.PerformanceEvaluation: client-1 > > > > >> 1048576/1887426/2097152 > > > > >> 09/08/13 09:58:40 INFO hbase.PerformanceEvaluation: client-2 > > > > >> 2097152/2935996/3145728 > > > > >> 09/08/13 09:58:54 INFO hbase.PerformanceEvaluation: client-0 > > > > >> 0/838856/1048576 > > > > >> 09/08/13 10:00:29 INFO hbase.PerformanceEvaluation: client-1 > > > > >> 1048576/1992283/2097152 > > > > >> 09/08/13 10:01:01 INFO hbase.PerformanceEvaluation: client-2 > > > > >> 2097152/3040853/3145728 > > > > >> 09/08/13 10:01:24 INFO hbase.PerformanceEvaluation: client-0 > > > > >> 0/943713/1048576 > > > > >> 09/08/13 10:02:36 INFO hbase.PerformanceEvaluation: client-1 > > > > >> 1048576/2097140/2097152 > > > > >> 09/08/13 10:02:37 INFO hbase.PerformanceEvaluation: client-1 > > Finished > > > > >> randomWrite in 680674ms at offset 1048576 for 1048576 rows > > > > >> 09/08/13 10:02:37 INFO hbase.PerformanceEvaluation: Finished 1 in > > > > 680674ms > > > > >> writing 1048576 rows > > > > >> 09/08/13 10:03:19 INFO hbase.PerformanceEvaluation: client-2 > > > > >> 2097152/3145710/3145728 > > > > >> 09/08/13 10:03:20 INFO hbase.PerformanceEvaluation: client-2 > > Finished > > > > >> randomWrite in 723771ms at offset 2097152 for 1048576 rows > > > > >> 09/08/13 10:03:20 INFO hbase.PerformanceEvaluation: Finished 2 in > > > > 723771ms > > > > >> writing 1048576 rows > > > > >> 09/08/13 10:03:41 INFO hbase.PerformanceEvaluation: client-0 > > > > >> 0/1048570/1048576 > > > > >> 09/08/13 10:03:42 INFO hbase.PerformanceEvaluation: client-0 > > Finished > > > > >> randomWrite in 746054ms at offset 0 for 1048576 rows > > > > >> 09/08/13 10:03:42 INFO hbase.PerformanceEvaluation: Finished 0 in > > > > 746054ms > > > > >> writing 1048576 rows > > > > >> > > > > >> > > > > >> > > > > >> ============================================================ > > > > >> > > > > >> Still pretty slow. Any other ideas? I'm running the client from > > the > > > > >> master > > > > >> box, but its not running any regionServers or datanodes. > > > > >> > > > > >> stack-3 wrote: > > > > >> > > > > > >> > Your config. looks fine. > > > > >> > > > > > >> > Only think that gives me pause is: > > > > >> > > > > > >> > "-XX:NewSize=6m -XX:MaxNewSize=6m" > > > > >> > > > > > >> > Any reason for the above? > > > > >> > > > > > >> > If you study your GC logs, lots of pauses? > > > > >> > > > > > >> > Oh, and this: replication is set to 6. Why 6? Each write must > > > commit > > > > >> to > > > > >> > 6 > > > > >> > datanodes before complete. In the tests posted on wiki, we > > > replicate > > > > to > > > > >> 3 > > > > >> > nodes. > > > > >> > > > > > >> > In end of this message you say you are doing gets? Numbers you > > > posted > > > > >> > were > > > > >> > for writes? > > > > >> > > > > > >> > St.Ack > > > > >> > > > > > >> > > > > > >> > On Wed, Aug 12, 2009 at 1:15 PM, llpind <[email protected] > > > > > > wrote: > > > > >> > > > > > >> >> > > > > >> >> Not sure why my performance is so slow. Here is my > > configuration: > > > > >> >> > > > > >> >> box1: > > > > >> >> 10395 SecondaryNameNode > > > > >> >> 11628 Jps > > > > >> >> 10131 NameNode > > > > >> >> 10638 HQuorumPeer > > > > >> >> 10705 HMaster > > > > >> >> > > > > >> >> box 2-5: > > > > >> >> 6741 HQuorumPeer > > > > >> >> 6841 HRegionServer > > > > >> >> 7881 Jps > > > > >> >> 6610 DataNode > > > > >> >> > > > > >> >> > > > > >> >> hbase site: ======================= > > > > >> >> <?xml version="1.0"?> > > > > >> >> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> > > > > >> >> <!-- > > > > >> >> /** > > > > >> >> * Copyright 2007 The Apache Software Foundation > > > > >> >> * > > > > >> >> * Licensed to the Apache Software Foundation (ASF) under one > > > > >> >> * or more contributor license agreements. See the NOTICE file > > > > >> >> * distributed with this work for additional information > > > > >> >> * regarding copyright ownership. The ASF licenses this file > > > > >> >> * to you under the Apache License, Version 2.0 (the > > > > >> >> * "License"); you may not use this file except in compliance > > > > >> >> * with the License. You may obtain a copy of the License at > > > > >> >> * > > > > >> >> * http://www.apache.org/licenses/LICENSE-2.0 > > > > >> >> * > > > > >> >> * Unless required by applicable law or agreed to in writing, > > > > software > > > > >> >> * distributed under the License is distributed on an "AS IS" > > > BASIS, > > > > >> >> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express > > or > > > > >> >> implied. > > > > >> >> * See the License for the specific language governing > > permissions > > > > and > > > > >> >> * limitations under the License. > > > > >> >> */ > > > > >> >> --> > > > > >> >> <configuration> > > > > >> >> <property> > > > > >> >> <name>hbase.rootdir</name> > > > > >> >> <value>hdfs://box1:9000/hbase</value> > > > > >> >> <description>The directory shared by region servers. > > > > >> >> </description> > > > > >> >> </property> > > > > >> >> <property> > > > > >> >> <name>hbase.master.port</name> > > > > >> >> <value>60000</value> > > > > >> >> <description>The port that the HBase master runs at. > > > > >> >> </description> > > > > >> >> </property> > > > > >> >> <property> > > > > >> >> <name>hbase.cluster.distributed</name> > > > > >> >> <value>true</value> > > > > >> >> <description>The mode the cluster will be in. Possible > values > > > are > > > > >> >> false: standalone and pseudo-distributed setups with > managed > > > > >> >> Zookeeper > > > > >> >> true: fully-distributed with unmanaged Zookeeper Quorum > (see > > > > >> >> hbase-env.sh) > > > > >> >> </description> > > > > >> >> </property> > > > > >> >> <property> > > > > >> >> <name>hbase.regionserver.lease.period</name> > > > > >> >> <value>120000</value> > > > > >> >> <description>HRegion server lease period in milliseconds. > > > Default > > > > is > > > > >> >> 60 seconds. Clients must report in within this period else > > they > > > > are > > > > >> >> considered dead.</description> > > > > >> >> </property> > > > > >> >> > > > > >> >> <property> > > > > >> >> <name>hbase.zookeeper.property.clientPort</name> > > > > >> >> <value>2222</value> > > > > >> >> <description>Property from ZooKeeper's config zoo.cfg. > > > > >> >> The port at which the clients will connect. > > > > >> >> </description> > > > > >> >> </property> > > > > >> >> <property> > > > > >> >> <name>hbase.zookeeper.property.dataDir</name> > > > > >> >> <value>/home/hadoop/zookeeper</value> > > > > >> >> </property> > > > > >> >> <property> > > > > >> >> <name>hbase.zookeeper.property.syncLimit</name> > > > > >> >> <value>5</value> > > > > >> >> </property> > > > > >> >> <property> > > > > >> >> <name>hbase.zookeeper.property.tickTime</name> > > > > >> >> <value>2000</value> > > > > >> >> </property> > > > > >> >> <property> > > > > >> >> <name>hbase.zookeeper.property.initLimit</name> > > > > >> >> <value>10</value> > > > > >> >> </property> > > > > >> >> <property> > > > > >> >> <name>hbase.zookeeper.quorum</name> > > > > >> >> <value>box1,box2,box3,box4</value> > > > > >> >> <description>Comma separated list of servers in the > > ZooKeeper > > > > >> >> Quorum. > > > > >> >> For example, > > > > >> >> "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com". > > > > >> >> By default this is set to localhost for local and > > > > >> pseudo-distributed > > > > >> >> modes > > > > >> >> of operation. For a fully-distributed setup, this should > be > > > set > > > > to > > > > >> a > > > > >> >> full > > > > >> >> list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is > set > > > in > > > > >> >> hbase-env.sh > > > > >> >> this is the list of servers which we will start/stop > > ZooKeeper > > > > on. > > > > >> >> </description> > > > > >> >> </property> > > > > >> >> <property> > > > > >> >> <name>hfile.block.cache.size</name> > > > > >> >> <value>.5</value> > > > > >> >> <description>text</description> > > > > >> >> </property> > > > > >> >> > > > > >> >> </configuration> > > > > >> >> > > > > >> >> > > > > >> >> hbase env:==================================================== > > > > >> >> > > > > >> >> export HBASE_CLASSPATH=${HADOOP_CONF_DIR} > > > > >> >> > > > > >> >> export HBASE_HEAPSIZE=3000 > > > > >> >> > > > > >> >> export HBASE_OPTS="-XX:NewSize=6m -XX:MaxNewSize=6m > > > > >> >> -XX:+UseConcMarkSweepGC > > > > >> >> -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps > > > > >> >> -XX:+CMSIncrementalMode > > > > >> >> -Xloggc:/home/hadoop/hbase-0.20.0/logs/gc-hbase.log" > > > > >> >> > > > > >> >> export HBASE_MANAGES_ZK=true > > > > >> >> > > > > >> >> Hadoop core > > > > >> >> site=========================================================== > > > > >> >> > > > > >> >> <?xml version="1.0"?> > > > > >> >> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> > > > > >> >> > > > > >> >> <!-- Put site-specific property overrides in this file. --> > > > > >> >> > > > > >> >> <configuration> > > > > >> >> <property> > > > > >> >> <name>fs.default.name</name> > > > > >> >> <value>hdfs://box1:9000</value> > > > > >> >> <description>The name of the default file system. A URI > whose > > > > >> >> scheme and authority determine the FileSystem implementation. > > > The > > > > >> >> uri's scheme determines the config property (fs.SCHEME.impl) > > > naming > > > > >> >> the FileSystem implementation class. The uri's authority is > > used > > > > to > > > > >> >> determine the host, port, etc. for a > filesystem.</description> > > > > >> >> </property> > > > > >> >> <property> > > > > >> >> <name>hadoop.tmp.dir</name> > > > > >> >> <value>/data/hadoop-0.20.0-${user.name}</value> > > > > >> >> <description>A base for other temporary > > directories.</description> > > > > >> >> </property> > > > > >> >> </configuration> > > > > >> >> > > > > >> >> ============== > > > > >> >> > > > > >> >> replication is set to 6. > > > > >> >> > > > > >> >> hadoop env================= > > > > >> >> > > > > >> >> export HADOOP_HEAPSIZE=3000 > > > > >> >> export HADOOP_NAMENODE_OPTS="-Dcom.sun.management.jmxremote > > > > >> >> $HADOOP_NAMENODE_OPTS" > > > > >> >> export > > > HADOOP_SECONDARYNAMENODE_OPTS="-Dcom.sun.management.jmxremote > > > > >> >> $HADOOP_SECONDARYNAMENODE_OPTS" > > > > >> >> export HADOOP_DATANODE_OPTS="-Dcom.sun.management.jmxremote > > > > >> >> $HADOOP_DATANODE_OPTS" > > > > >> >> export HADOOP_BALANCER_OPTS="-Dcom.sun.management.jmxremote > > > > >> >> $HADOOP_BALANCER_OPTS" > > > > >> >> export HADOOP_JOBTRACKER_OPTS="-Dcom.sun.management.jmxremote > > > > >> >> $HADOOP_JOBTRACKER_OPTS" > > > > >> >> ================== > > > > >> >> > > > > >> >> > > > > >> >> Very basic setup. then i start the cluster do simple random > Get > > > > >> >> operations > > > > >> >> on a tall table (~60 M rows): > > > > >> >> > > > > >> >> {NAME => 'tallTable', FAMILIES => [{NAME => 'family1', > > COMPRESSION > > > => > > > > >> >> 'NONE', VERSIONS => '3', TTL => '2147483647', BLOCKSIZE => > > '65536', > > > > >> >> IN_MEMORY => 'false', BLOCKCACHE => 'true'}]} > > > > >> >> > > > > >> >> Is this fairly normal speeds? I'm unsure if this is a result > of > > > > having > > > > >> a > > > > >> >> small cluster? Please advise... > > > > >> >> > > > > >> >> stack-3 wrote: > > > > >> >> > > > > > >> >> > Yeah, seems slow. In old hbase, it could do 5-10k writes a > > > second > > > > >> >> going > > > > >> >> > by > > > > >> >> > performance eval page up on wiki. SequentialWrite was about > > same > > > > as > > > > >> >> > RandomWrite. Check out the stats on hw up on that page and > > > > >> description > > > > >> >> of > > > > >> >> > how test was set up. Can you figure where its slow? > > > > >> >> > > > > > >> >> > St.Ack > > > > >> >> > > > > > >> >> > On Wed, Aug 12, 2009 at 10:10 AM, llpind < > > [email protected] > > > > > > > > >> >> wrote: > > > > >> >> > > > > > >> >> >> > > > > >> >> >> Thanks Stack. > > > > >> >> >> > > > > >> >> >> I will try mapred with more clients. I tried it without > > mapred > > > > >> using > > > > >> >> 3 > > > > >> >> >> clients Random Write operations here was the output: > > > > >> >> >> > > > > >> >> >> 09/08/12 09:22:52 INFO hbase.PerformanceEvaluation: client-0 > > > Start > > > > >> >> >> randomWrite at offset 0 for 1048576 rows > > > > >> >> >> 09/08/12 09:22:52 INFO hbase.PerformanceEvaluation: client-1 > > > Start > > > > >> >> >> randomWrite at offset 1048576 for 1048576 rows > > > > >> >> >> 09/08/12 09:22:52 INFO hbase.PerformanceEvaluation: client-2 > > > Start > > > > >> >> >> randomWrite at offset 2097152 for 1048576 rows > > > > >> >> >> 09/08/12 09:24:23 INFO hbase.PerformanceEvaluation: client-1 > > > > >> >> >> 1048576/1153427/2097152 > > > > >> >> >> 09/08/12 09:24:23 INFO hbase.PerformanceEvaluation: client-2 > > > > >> >> >> 2097152/2201997/3145728 > > > > >> >> >> 09/08/12 09:24:25 INFO hbase.PerformanceEvaluation: client-0 > > > > >> >> >> 0/104857/1048576 > > > > >> >> >> 09/08/12 09:27:42 INFO hbase.PerformanceEvaluation: client-0 > > > > >> >> >> 0/209714/1048576 > > > > >> >> >> 09/08/12 09:27:46 INFO hbase.PerformanceEvaluation: client-1 > > > > >> >> >> 1048576/1258284/2097152 > > > > >> >> >> 09/08/12 09:27:46 INFO hbase.PerformanceEvaluation: client-2 > > > > >> >> >> 2097152/2306854/3145728 > > > > >> >> >> 09/08/12 09:32:32 INFO hbase.PerformanceEvaluation: client-1 > > > > >> >> >> 1048576/1363141/2097152 > > > > >> >> >> 09/08/12 09:32:33 INFO hbase.PerformanceEvaluation: client-0 > > > > >> >> >> 0/314571/1048576 > > > > >> >> >> 09/08/12 09:32:41 INFO hbase.PerformanceEvaluation: client-2 > > > > >> >> >> 2097152/2411711/3145728 > > > > >> >> >> 09/08/12 09:35:31 INFO hbase.PerformanceEvaluation: client-0 > > > > >> >> >> 0/419428/1048576 > > > > >> >> >> 09/08/12 09:35:34 INFO hbase.PerformanceEvaluation: client-1 > > > > >> >> >> 1048576/1467998/2097152 > > > > >> >> >> 09/08/12 09:35:53 INFO hbase.PerformanceEvaluation: client-2 > > > > >> >> >> 2097152/2516568/3145728 > > > > >> >> >> 09/08/12 09:39:02 INFO hbase.PerformanceEvaluation: client-0 > > > > >> >> >> 0/524285/1048576 > > > > >> >> >> 09/08/12 09:39:03 INFO hbase.PerformanceEvaluation: client-2 > > > > >> >> >> 2097152/2621425/3145728 > > > > >> >> >> 09/08/12 09:40:07 INFO hbase.PerformanceEvaluation: client-1 > > > > >> >> >> 1048576/1572855/2097152 > > > > >> >> >> 09/08/12 09:42:53 INFO hbase.PerformanceEvaluation: client-0 > > > > >> >> >> 0/629142/1048576 > > > > >> >> >> 09/08/12 09:44:25 INFO hbase.PerformanceEvaluation: client-2 > > > > >> >> >> 2097152/2726282/3145728 > > > > >> >> >> 09/08/12 09:44:44 INFO hbase.PerformanceEvaluation: client-1 > > > > >> >> >> 1048576/1677712/2097152 > > > > >> >> >> 09/08/12 09:46:43 INFO hbase.PerformanceEvaluation: client-0 > > > > >> >> >> 0/733999/1048576 > > > > >> >> >> 09/08/12 09:48:11 INFO hbase.PerformanceEvaluation: client-2 > > > > >> >> >> 2097152/2831139/3145728 > > > > >> >> >> 09/08/12 09:48:29 INFO hbase.PerformanceEvaluation: client-1 > > > > >> >> >> 1048576/1782569/2097152 > > > > >> >> >> 09/08/12 09:50:12 INFO hbase.PerformanceEvaluation: client-0 > > > > >> >> >> 0/838856/1048576 > > > > >> >> >> 09/08/12 09:52:47 INFO hbase.PerformanceEvaluation: client-2 > > > > >> >> >> 2097152/2935996/3145728 > > > > >> >> >> 09/08/12 09:53:51 INFO hbase.PerformanceEvaluation: client-1 > > > > >> >> >> 1048576/1887426/2097152 > > > > >> >> >> 09/08/12 09:56:32 INFO hbase.PerformanceEvaluation: client-0 > > > > >> >> >> 0/943713/1048576 > > > > >> >> >> 09/08/12 09:58:32 INFO hbase.PerformanceEvaluation: client-2 > > > > >> >> >> 2097152/3040853/3145728 > > > > >> >> >> 09/08/12 09:59:14 INFO hbase.PerformanceEvaluation: client-1 > > > > >> >> >> 1048576/1992283/2097152 > > > > >> >> >> 09/08/12 10:02:28 INFO hbase.PerformanceEvaluation: client-0 > > > > >> >> >> 0/1048570/1048576 > > > > >> >> >> 09/08/12 10:02:30 INFO hbase.PerformanceEvaluation: client-0 > > > > >> Finished > > > > >> >> >> randomWrite in 2376615ms at offset 0 for 1048576 rows > > > > >> >> >> 09/08/12 10:02:30 INFO hbase.PerformanceEvaluation: Finished > 0 > > > in > > > > >> >> >> 2376615ms > > > > >> >> >> writing 1048576 rows > > > > >> >> >> 09/08/12 10:06:35 INFO hbase.PerformanceEvaluation: client-2 > > > > >> >> >> 2097152/3145710/3145728 > > > > >> >> >> 09/08/12 10:06:38 INFO hbase.PerformanceEvaluation: client-2 > > > > >> Finished > > > > >> >> >> randomWrite in 2623395ms at offset 2097152 for 1048576 rows > > > > >> >> >> 09/08/12 10:06:38 INFO hbase.PerformanceEvaluation: Finished > 2 > > > in > > > > >> >> >> 2623395ms > > > > >> >> >> writing 1048576 rows > > > > >> >> >> 09/08/12 10:06:42 INFO hbase.PerformanceEvaluation: client-1 > > > > >> >> >> 1048576/2097140/2097152 > > > > >> >> >> 09/08/12 10:06:43 INFO hbase.PerformanceEvaluation: client-1 > > > > >> Finished > > > > >> >> >> randomWrite in 2630199ms at offset 1048576 for 1048576 rows > > > > >> >> >> 09/08/12 10:06:43 INFO hbase.PerformanceEvaluation: Finished > 1 > > > in > > > > >> >> >> 2630199ms > > > > >> >> >> writing 1048576 rows > > > > >> >> >> > > > > >> >> >> > > > > >> >> >> > > > > >> >> >> Seems kind of slow for ~3M records. I have a 4 node cluster > > up > > > at > > > > >> the > > > > >> >> >> moment. HMaster & Namenode running on same box. > > > > >> >> >> -- > > > > >> >> >> View this message in context: > > > > >> >> >> > > > > >> >> > > > > >> > > > > > > > > > > http://www.nabble.com/HBase-in-a-real-world-application-tp24920888p24940922.html > > > > >> >> >> Sent from the HBase User mailing list archive at Nabble.com. > > > > >> >> >> > > > > >> >> >> > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > >> >> -- > > > > >> >> View this message in context: > > > > >> >> > > > > >> > > > > > > > > > > http://www.nabble.com/HBase-in-a-real-world-application-tp24920888p24943406.html > > > > >> >> Sent from the HBase User mailing list archive at Nabble.com. > > > > >> >> > > > > >> >> > > > > >> > > > > > >> > > > > > >> > > > > >> -- > > > > >> View this message in context: > > > > >> > > > > > > > > > > http://www.nabble.com/HBase-in-a-real-world-application-tp24920888p24955595.html > > > > >> Sent from the HBase User mailing list archive at Nabble.com. > > > > >> > > > > >> > > > > > > > > > > > > > > >
