[jira] [Commented] (HBASE-15403) Performance Evaluation tool isn't working as expected
[ https://issues.apache.org/jira/browse/HBASE-15403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285273#comment-15285273 ] Jerry He commented on HBASE-15403: -- The total row count is not being updated correctly. Will open another JIRA to address it. > Performance Evaluation tool isn't working as expected > - > > Key: HBASE-15403 > URL: https://issues.apache.org/jira/browse/HBASE-15403 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.0 >Reporter: Appy >Priority: Critical > > hbase pe --nomapred --rows=100 --table='t4' randomWrite 10 > # count on t4 gives 620 rows > hbase pe --nomapred --rows=200 --table='t5' randomWrite 10 > # count on t5 gives 1257 rows > hbase pe --nomapred --table='t6' --rows=200 randomWrite 1 > # count on t6 gives 126 rows > I was working with 1.2.0, but it's likely that it'll also be affecting master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15403) Performance Evaluation tool isn't working as expected
[ https://issues.apache.org/jira/browse/HBASE-15403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15283041#comment-15283041 ] Jerry He commented on HBASE-15403: -- Let me dig a little more. > Performance Evaluation tool isn't working as expected > - > > Key: HBASE-15403 > URL: https://issues.apache.org/jira/browse/HBASE-15403 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.0 >Reporter: Appy >Priority: Critical > > hbase pe --nomapred --rows=100 --table='t4' randomWrite 10 > # count on t4 gives 620 rows > hbase pe --nomapred --rows=200 --table='t5' randomWrite 10 > # count on t5 gives 1257 rows > hbase pe --nomapred --table='t6' --rows=200 randomWrite 1 > # count on t6 gives 126 rows > I was working with 1.2.0, but it's likely that it'll also be affecting master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15403) Performance Evaluation tool isn't working as expected
[ https://issues.apache.org/jira/browse/HBASE-15403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15283038#comment-15283038 ] Jerry He commented on HBASE-15403: -- Recently when I ran PerformanceEvaluation, I did notice some problem with the number of rows. It was not exactly the same problem described in this JIRA. hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=TestTable1 randomWrite 1 hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=TestTable5 randomWrite 5 hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=TestTable10 randomWrite 10 hbase org.apache.hadoop.hbase.PerformanceEvaluation --table=TestTable10 randomWrite 20 All produced the same number of rows, and on the file system, they look like in similar size as well: hadoop fs -du -h /apps/hbase/data/data/default 786.5 M /apps/hbase/data/data/default/TestTable1 786.0 M /apps/hbase/data/data/default/TestTable10 782.0 M /apps/hbase/data/data/default/TestTable20 713.4 M /apps/hbase/data/data/default/TestTable5 HBase is 1.2.0. Looks like a regression somewhere. > Performance Evaluation tool isn't working as expected > - > > Key: HBASE-15403 > URL: https://issues.apache.org/jira/browse/HBASE-15403 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.0 >Reporter: Appy >Priority: Critical > > hbase pe --nomapred --rows=100 --table='t4' randomWrite 10 > # count on t4 gives 620 rows > hbase pe --nomapred --rows=200 --table='t5' randomWrite 10 > # count on t5 gives 1257 rows > hbase pe --nomapred --table='t6' --rows=200 randomWrite 1 > # count on t6 gives 126 rows > I was working with 1.2.0, but it's likely that it'll also be affecting master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15403) Performance Evaluation tool isn't working as expected
[ https://issues.apache.org/jira/browse/HBASE-15403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15282365#comment-15282365 ] Appy commented on HBASE-15403: -- Thanks for the explanation. I should have researched more before filing the jira. Closing it out as 'Invalid'. > Performance Evaluation tool isn't working as expected > - > > Key: HBASE-15403 > URL: https://issues.apache.org/jira/browse/HBASE-15403 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.0 >Reporter: Appy >Priority: Critical > > hbase pe --nomapred --rows=100 --table='t4' randomWrite 10 > # count on t4 gives 620 rows > hbase pe --nomapred --rows=200 --table='t5' randomWrite 10 > # count on t5 gives 1257 rows > hbase pe --nomapred --table='t6' --rows=200 randomWrite 1 > # count on t6 gives 126 rows > I was working with 1.2.0, but it's likely that it'll also be affecting master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15403) Performance Evaluation tool isn't working as expected
[ https://issues.apache.org/jira/browse/HBASE-15403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15181985#comment-15181985 ] Anoop Sam John commented on HBASE-15403: --rows dont mean total rows which will be written.. I see, this is the #times one client thread loop runs. The write tests allows for multi? If so this will be #multi requests. So then that will come out like #threads * --rows * puts per multi... For scan --rows also means same way. If we test with ScanRange1000 and --rows = 1000 and 10 threads. total 1000* 1000 * 10 rows read overall.. Yes there may be duplicate rks coming out across diff threads. Need to check more on code. > Performance Evaluation tool isn't working as expected > - > > Key: HBASE-15403 > URL: https://issues.apache.org/jira/browse/HBASE-15403 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.0 >Reporter: Appy >Priority: Critical > > hbase pe --nomapred --rows=100 --table='t4' randomWrite 10 > # count on t4 gives 620 rows > hbase pe --nomapred --rows=200 --table='t5' randomWrite 10 > # count on t5 gives 1257 rows > hbase pe --nomapred --table='t6' --rows=200 randomWrite 1 > # count on t6 gives 126 rows > I was working with 1.2.0, but it's likely that it'll also be affecting master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15403) Performance Evaluation tool isn't working as expected
[ https://issues.apache.org/jira/browse/HBASE-15403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15181976#comment-15181976 ] Jean-Marc Spaggiari commented on HBASE-15403: - Is that not "normal"? Random can mean duplicates, no? Is you keep all versions and count the versions too, don't you get the expected number of cells? 100 rows 10 clients means 1 000 rows so you have about 380 duplicates. Sound realistic to me? > Performance Evaluation tool isn't working as expected > - > > Key: HBASE-15403 > URL: https://issues.apache.org/jira/browse/HBASE-15403 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 1.2.0 >Reporter: Appy >Priority: Critical > > hbase pe --nomapred --rows=100 --table='t4' randomWrite 10 > # count on t4 gives 620 rows > hbase pe --nomapred --rows=200 --table='t5' randomWrite 10 > # count on t5 gives 1257 rows > hbase pe --nomapred --table='t6' --rows=200 randomWrite 1 > # count on t6 gives 126 rows > I was working with 1.2.0, but it's likely that it'll also be affecting master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
