Yeah, seems slow. In old hbase, it could do 5-10k writes a second going by performance eval page up on wiki. SequentialWrite was about same as RandomWrite. Check out the stats on hw up on that page and description of how test was set up. Can you figure where its slow?
St.Ack On Wed, Aug 12, 2009 at 10:10 AM, llpind <[email protected]> wrote: > > Thanks Stack. > > I will try mapred with more clients. I tried it without mapred using 3 > clients Random Write operations here was the output: > > 09/08/12 09:22:52 INFO hbase.PerformanceEvaluation: client-0 Start > randomWrite at offset 0 for 1048576 rows > 09/08/12 09:22:52 INFO hbase.PerformanceEvaluation: client-1 Start > randomWrite at offset 1048576 for 1048576 rows > 09/08/12 09:22:52 INFO hbase.PerformanceEvaluation: client-2 Start > randomWrite at offset 2097152 for 1048576 rows > 09/08/12 09:24:23 INFO hbase.PerformanceEvaluation: client-1 > 1048576/1153427/2097152 > 09/08/12 09:24:23 INFO hbase.PerformanceEvaluation: client-2 > 2097152/2201997/3145728 > 09/08/12 09:24:25 INFO hbase.PerformanceEvaluation: client-0 > 0/104857/1048576 > 09/08/12 09:27:42 INFO hbase.PerformanceEvaluation: client-0 > 0/209714/1048576 > 09/08/12 09:27:46 INFO hbase.PerformanceEvaluation: client-1 > 1048576/1258284/2097152 > 09/08/12 09:27:46 INFO hbase.PerformanceEvaluation: client-2 > 2097152/2306854/3145728 > 09/08/12 09:32:32 INFO hbase.PerformanceEvaluation: client-1 > 1048576/1363141/2097152 > 09/08/12 09:32:33 INFO hbase.PerformanceEvaluation: client-0 > 0/314571/1048576 > 09/08/12 09:32:41 INFO hbase.PerformanceEvaluation: client-2 > 2097152/2411711/3145728 > 09/08/12 09:35:31 INFO hbase.PerformanceEvaluation: client-0 > 0/419428/1048576 > 09/08/12 09:35:34 INFO hbase.PerformanceEvaluation: client-1 > 1048576/1467998/2097152 > 09/08/12 09:35:53 INFO hbase.PerformanceEvaluation: client-2 > 2097152/2516568/3145728 > 09/08/12 09:39:02 INFO hbase.PerformanceEvaluation: client-0 > 0/524285/1048576 > 09/08/12 09:39:03 INFO hbase.PerformanceEvaluation: client-2 > 2097152/2621425/3145728 > 09/08/12 09:40:07 INFO hbase.PerformanceEvaluation: client-1 > 1048576/1572855/2097152 > 09/08/12 09:42:53 INFO hbase.PerformanceEvaluation: client-0 > 0/629142/1048576 > 09/08/12 09:44:25 INFO hbase.PerformanceEvaluation: client-2 > 2097152/2726282/3145728 > 09/08/12 09:44:44 INFO hbase.PerformanceEvaluation: client-1 > 1048576/1677712/2097152 > 09/08/12 09:46:43 INFO hbase.PerformanceEvaluation: client-0 > 0/733999/1048576 > 09/08/12 09:48:11 INFO hbase.PerformanceEvaluation: client-2 > 2097152/2831139/3145728 > 09/08/12 09:48:29 INFO hbase.PerformanceEvaluation: client-1 > 1048576/1782569/2097152 > 09/08/12 09:50:12 INFO hbase.PerformanceEvaluation: client-0 > 0/838856/1048576 > 09/08/12 09:52:47 INFO hbase.PerformanceEvaluation: client-2 > 2097152/2935996/3145728 > 09/08/12 09:53:51 INFO hbase.PerformanceEvaluation: client-1 > 1048576/1887426/2097152 > 09/08/12 09:56:32 INFO hbase.PerformanceEvaluation: client-0 > 0/943713/1048576 > 09/08/12 09:58:32 INFO hbase.PerformanceEvaluation: client-2 > 2097152/3040853/3145728 > 09/08/12 09:59:14 INFO hbase.PerformanceEvaluation: client-1 > 1048576/1992283/2097152 > 09/08/12 10:02:28 INFO hbase.PerformanceEvaluation: client-0 > 0/1048570/1048576 > 09/08/12 10:02:30 INFO hbase.PerformanceEvaluation: client-0 Finished > randomWrite in 2376615ms at offset 0 for 1048576 rows > 09/08/12 10:02:30 INFO hbase.PerformanceEvaluation: Finished 0 in 2376615ms > writing 1048576 rows > 09/08/12 10:06:35 INFO hbase.PerformanceEvaluation: client-2 > 2097152/3145710/3145728 > 09/08/12 10:06:38 INFO hbase.PerformanceEvaluation: client-2 Finished > randomWrite in 2623395ms at offset 2097152 for 1048576 rows > 09/08/12 10:06:38 INFO hbase.PerformanceEvaluation: Finished 2 in 2623395ms > writing 1048576 rows > 09/08/12 10:06:42 INFO hbase.PerformanceEvaluation: client-1 > 1048576/2097140/2097152 > 09/08/12 10:06:43 INFO hbase.PerformanceEvaluation: client-1 Finished > randomWrite in 2630199ms at offset 1048576 for 1048576 rows > 09/08/12 10:06:43 INFO hbase.PerformanceEvaluation: Finished 1 in 2630199ms > writing 1048576 rows > > > > Seems kind of slow for ~3M records. I have a 4 node cluster up at the > moment. HMaster & Namenode running on same box. > -- > View this message in context: > http://www.nabble.com/HBase-in-a-real-world-application-tp24920888p24940922.html > Sent from the HBase User mailing list archive at Nabble.com. > >
