[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
[
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16131105#comment-16131105
]
Hudson commented on HBASE-18485:
FAILURE: Integrated in Jenkins build HBASE-14070.HLC #233 (See
[https://builds.apache.org/job/HBASE-14070.HLC/233/])
HBASE-18485 Performance issue: ClientAsyncPrefetchScanner is slower than
(zghao: rev 5915d73a70eb69adc639062f372559c9fc5130be)
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestScannersFromClientSide.java
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientAsyncPrefetchScanner.java
> Performance issue: ClientAsyncPrefetchScanner is slower than
> ClientSimpleScanner
>
>
> Key: HBASE-18485
> URL: https://issues.apache.org/jira/browse/HBASE-18485
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 2.0.0-alpha-2
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18485-v1.patch, HBASE-18485-v2.patch,
> HBASE-18485-v3.patch, HBASE-18485-v4.patch, HBASE-18485-v4.patch,
> HBASE-18485-v4.patch, HBASE-18485-v5.patch, HBASE-18485-v5.patch,
> HBASE-18485-v6.patch, HBASE-18485-v6.patch
>
>
> Copied the test result from HBASE-17994.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred scan 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred --asyncPrefetch=True scan 1
> {code}
> Mean latency.
> || ||Test1|| Test2 || Test3 || Test4|| Test5||
> |scan| 12.21 | 14.32 | 13.25 | 13.07 | 11.83 |
> |scan with prefetch=True | 37.36 | 37.88 | 37.56 | 37.66 | 38.28 |
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
[
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115835#comment-16115835
]
Chia-Ping Tsai commented on HBASE-18485:
+1
> Performance issue: ClientAsyncPrefetchScanner is slower than
> ClientSimpleScanner
>
>
> Key: HBASE-18485
> URL: https://issues.apache.org/jira/browse/HBASE-18485
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 2.0.0-alpha-2
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18485-v1.patch, HBASE-18485-v2.patch,
> HBASE-18485-v3.patch, HBASE-18485-v4.patch, HBASE-18485-v4.patch,
> HBASE-18485-v4.patch, HBASE-18485-v5.patch, HBASE-18485-v5.patch,
> HBASE-18485-v6.patch, HBASE-18485-v6.patch
>
>
> Copied the test result from HBASE-17994.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred scan 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred --asyncPrefetch=True scan 1
> {code}
> Mean latency.
> || ||Test1|| Test2 || Test3 || Test4|| Test5||
> |scan| 12.21 | 14.32 | 13.25 | 13.07 | 11.83 |
> |scan with prefetch=True | 37.36 | 37.88 | 37.56 | 37.66 | 38.28 |
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
[
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115816#comment-16115816
]
Hadoop QA commented on HBASE-18485:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 2s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
42s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}111m
59s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}164m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18485 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12880560/HBASE-18485-v6.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 8514bee0aa0c 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 637f7ab |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/7952/testReport/ |
| modules | C: hbase-client hbase-server U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7952/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Performance issue: ClientAsyncPrefetchScanner
[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
[
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114951#comment-16114951
]
Hadoop QA commented on HBASE-18485:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
39m 9s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
51s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}112m 56s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
42s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}183m 26s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.TestMasterFailover |
| Timed out junit tests |
org.apache.hadoop.hbase.master.TestGetLastFlushedSequenceId |
| | org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2 |
| |
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint
|
| | org.apache.hadoop.hbase.replication.regionserver.TestWALEntryStream |
| | org.apache.hadoop.hbase.master.TestAssignmentListener |
| | org.apache.hadoop.hbase.filter.TestScanRowPrefix |
| | org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster |
| | org.apache.hadoop.hbase.io.asyncfs.TestSaslFanOutOneBlockAsyncDFSOutput |
| | org.apache.hadoop.hbase.master.TestMasterMetrics |
| | org.apache.hadoop.hbase.master.TestTableStateManager |
| | org.apache.hadoop.hbase.master.cleaner.TestReplicationZKNodeCleaner |
| | org.apache.hadoop.hbase.io.asyncfs.TestFanOutOneBlockAsyncDFSOutput |
| | org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable |
| | org.apache.hadoop.hbase.master.TestSplitLogManager
[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
[
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114556#comment-16114556
]
Hadoop QA commented on HBASE-18485:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 17s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
36s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}108m 33s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}161m 33s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.TestMasterFailover |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18485 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12880370/HBASE-18485-v5.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux c9a5eb08c3f4 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 6266bb3 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/7918/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/7918/testReport/ |
| modules | C: hbase-client hbase-server U: . |
| Console output |
[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
[
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114496#comment-16114496
]
Guanghao Zhang commented on HBASE-18485:
bq. That is a potential deadlock.
Nice catch.
bq. Is signal() more suitable than signalAll()?
Yes, there only one thread to call next and another thread to load cache. But I
thought it still should use signalAll() to avoid misuse? And for most cases, we
should use singalAll() instead of signal(). Thanks.
> Performance issue: ClientAsyncPrefetchScanner is slower than
> ClientSimpleScanner
>
>
> Key: HBASE-18485
> URL: https://issues.apache.org/jira/browse/HBASE-18485
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 2.0.0-alpha-2
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18485-v1.patch, HBASE-18485-v2.patch,
> HBASE-18485-v3.patch, HBASE-18485-v4.patch, HBASE-18485-v4.patch,
> HBASE-18485-v4.patch, HBASE-18485-v5.patch, HBASE-18485-v5.patch,
> HBASE-18485-v6.patch
>
>
> Copied the test result from HBASE-17994.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred scan 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred --asyncPrefetch=True scan 1
> {code}
> Mean latency.
> || ||Test1|| Test2 || Test3 || Test4|| Test5||
> |scan| 12.21 | 14.32 | 13.25 | 13.07 | 11.83 |
> |scan with prefetch=True | 37.36 | 37.88 | 37.56 | 37.66 | 38.28 |
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
[
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114374#comment-16114374
]
Chia-Ping Tsai commented on HBASE-18485:
{code}
+boolean succeed = false;
+try {
+ lock.lock();
+ while (!prefetchCondition()) {
+notFull.await();
+ }
+ loadCache();
+ succeed = true;
+ notEmpty.signalAll();
+} catch (Exception e) {
+ exceptionsQueue.add(e);
+} finally {
+ lock.unlock();
+ if (prefetchListener != null) {
+prefetchListener.accept(succeed);
{code}
# If exception occurs on loadCache(), the notEmpty.signalAll() won't be
executed. That is a potential deadlock.
# The client scanner is not thread-safe and we run a single PrefetchRunnable
for prefetch. Is signal() more suitable than signalAll()?
> Performance issue: ClientAsyncPrefetchScanner is slower than
> ClientSimpleScanner
>
>
> Key: HBASE-18485
> URL: https://issues.apache.org/jira/browse/HBASE-18485
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 2.0.0-alpha-2
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18485-v1.patch, HBASE-18485-v2.patch,
> HBASE-18485-v3.patch, HBASE-18485-v4.patch, HBASE-18485-v4.patch,
> HBASE-18485-v4.patch, HBASE-18485-v5.patch, HBASE-18485-v5.patch
>
>
> Copied the test result from HBASE-17994.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred scan 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred --asyncPrefetch=True scan 1
> {code}
> Mean latency.
> || ||Test1|| Test2 || Test3 || Test4|| Test5||
> |scan| 12.21 | 14.32 | 13.25 | 13.07 | 11.83 |
> |scan with prefetch=True | 37.36 | 37.88 | 37.56 | 37.66 | 38.28 |
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
[
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114069#comment-16114069
]
Guanghao Zhang commented on HBASE-18485:
[~chia7712] Any more concerns?
> Performance issue: ClientAsyncPrefetchScanner is slower than
> ClientSimpleScanner
>
>
> Key: HBASE-18485
> URL: https://issues.apache.org/jira/browse/HBASE-18485
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 2.0.0-alpha-2
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18485-v1.patch, HBASE-18485-v2.patch,
> HBASE-18485-v3.patch, HBASE-18485-v4.patch, HBASE-18485-v4.patch,
> HBASE-18485-v4.patch, HBASE-18485-v5.patch
>
>
> Copied the test result from HBASE-17994.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred scan 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred --asyncPrefetch=True scan 1
> {code}
> Mean latency.
> || ||Test1|| Test2 || Test3 || Test4|| Test5||
> |scan| 12.21 | 14.32 | 13.25 | 13.07 | 11.83 |
> |scan with prefetch=True | 37.36 | 37.88 | 37.56 | 37.66 | 38.28 |
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
[
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113957#comment-16113957
]
Guanghao Zhang commented on HBASE-18485:
Ok. Changed type to improvement.
bq. Should we enable the prefetch as default?
Checked HBASE-13071, it was merged since hbase 2.0. So it should be default in
3.0? But we will remove the blocking code and use the async code base. The
async client scanner will do the prefetch job by default. So I thought we don't
need enable it as default in this patch.
> Performance issue: ClientAsyncPrefetchScanner is slower than
> ClientSimpleScanner
>
>
> Key: HBASE-18485
> URL: https://issues.apache.org/jira/browse/HBASE-18485
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-18485-v1.patch, HBASE-18485-v2.patch,
> HBASE-18485-v3.patch, HBASE-18485-v4.patch, HBASE-18485-v4.patch,
> HBASE-18485-v4.patch
>
>
> Copied the test result from HBASE-17994.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred scan 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred --asyncPrefetch=True scan 1
> {code}
> Mean latency.
> || ||Test1|| Test2 || Test3 || Test4|| Test5||
> |scan| 12.21 | 14.32 | 13.25 | 13.07 | 11.83 |
> |scan with prefetch=True | 37.36 | 37.88 | 37.56 | 37.66 | 38.28 |
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
[
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113927#comment-16113927
]
Chia-Ping Tsai commented on HBASE-18485:
Have any bugs been fixed? It seems to me this issue is related to performance.
Maybe we should change the type to improvement. Should we enable the prefetch
as default?
> Performance issue: ClientAsyncPrefetchScanner is slower than
> ClientSimpleScanner
>
>
> Key: HBASE-18485
> URL: https://issues.apache.org/jira/browse/HBASE-18485
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-18485-v1.patch, HBASE-18485-v2.patch,
> HBASE-18485-v3.patch, HBASE-18485-v4.patch, HBASE-18485-v4.patch,
> HBASE-18485-v4.patch
>
>
> Copied the test result from HBASE-17994.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred scan 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred --asyncPrefetch=True scan 1
> {code}
> Mean latency.
> || ||Test1|| Test2 || Test3 || Test4|| Test5||
> |scan| 12.21 | 14.32 | 13.25 | 13.07 | 11.83 |
> |scan with prefetch=True | 37.36 | 37.88 | 37.56 | 37.66 | 38.28 |
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
[
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113790#comment-16113790
]
Guanghao Zhang commented on HBASE-18485:
Retry for Hadoop QA. And TestMasterFailover was tracked by HBASE-18425.
[~tedyu] Thanks for your review.
> Performance issue: ClientAsyncPrefetchScanner is slower than
> ClientSimpleScanner
>
>
> Key: HBASE-18485
> URL: https://issues.apache.org/jira/browse/HBASE-18485
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-18485-v1.patch, HBASE-18485-v2.patch,
> HBASE-18485-v3.patch, HBASE-18485-v4.patch, HBASE-18485-v4.patch
>
>
> Copied the test result from HBASE-17994.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred scan 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred --asyncPrefetch=True scan 1
> {code}
> Mean latency.
> || ||Test1|| Test2 || Test3 || Test4|| Test5||
> |scan| 12.21 | 14.32 | 13.25 | 13.07 | 11.83 |
> |scan with prefetch=True | 37.36 | 37.88 | 37.56 | 37.66 | 38.28 |
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
[
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113054#comment-16113054
]
Ted Yu commented on HBASE-18485:
lgtm
Better check failed tests locally before committing.
> Performance issue: ClientAsyncPrefetchScanner is slower than
> ClientSimpleScanner
>
>
> Key: HBASE-18485
> URL: https://issues.apache.org/jira/browse/HBASE-18485
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-18485-v1.patch, HBASE-18485-v2.patch,
> HBASE-18485-v3.patch, HBASE-18485-v4.patch
>
>
> Copied the test result from HBASE-17994.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred scan 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred --asyncPrefetch=True scan 1
> {code}
> Mean latency.
> || ||Test1|| Test2 || Test3 || Test4|| Test5||
> |scan| 12.21 | 14.32 | 13.25 | 13.07 | 11.83 |
> |scan with prefetch=True | 37.36 | 37.88 | 37.56 | 37.66 | 38.28 |
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
[
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112261#comment-16112261
]
Guanghao Zhang commented on HBASE-18485:
TestMasterFailover failed on master branch too. And TestHRegion is not related.
> Performance issue: ClientAsyncPrefetchScanner is slower than
> ClientSimpleScanner
>
>
> Key: HBASE-18485
> URL: https://issues.apache.org/jira/browse/HBASE-18485
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-18485-v1.patch, HBASE-18485-v2.patch,
> HBASE-18485-v3.patch, HBASE-18485-v4.patch
>
>
> Copied the test result from HBASE-17994.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred scan 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred --asyncPrefetch=True scan 1
> {code}
> Mean latency.
> || ||Test1|| Test2 || Test3 || Test4|| Test5||
> |scan| 12.21 | 14.32 | 13.25 | 13.07 | 11.83 |
> |scan with prefetch=True | 37.36 | 37.88 | 37.56 | 37.66 | 38.28 |
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
[
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112175#comment-16112175
]
Hadoop QA commented on HBASE-18485:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 3s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
38s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}129m 34s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}182m 38s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestHRegion |
| | hadoop.hbase.master.TestMasterFailover |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18485 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12880143/HBASE-18485-v4.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 4b649f6502dc 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 504a1f1 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/7893/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/7893/testReport/ |
| modules | C: hbase-client hbase-server
[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
[
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111859#comment-16111859
]
Guanghao Zhang commented on HBASE-18485:
The prefether thread will be terminated when the closed is true. But user may
forget to close scanner? In this case, daemon thread maybe better. Thanks. Let
me prepare a new patch.
> Performance issue: ClientAsyncPrefetchScanner is slower than
> ClientSimpleScanner
>
>
> Key: HBASE-18485
> URL: https://issues.apache.org/jira/browse/HBASE-18485
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-18485-v1.patch, HBASE-18485-v2.patch,
> HBASE-18485-v3.patch
>
>
> Copied the test result from HBASE-17994.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred scan 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred --asyncPrefetch=True scan 1
> {code}
> Mean latency.
> || ||Test1|| Test2 || Test3 || Test4|| Test5||
> |scan| 12.21 | 14.32 | 13.25 | 13.07 | 11.83 |
> |scan with prefetch=True | 37.36 | 37.88 | 37.56 | 37.66 | 38.28 |
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
[
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111761#comment-16111761
]
Ted Yu commented on HBASE-18485:
See
http://docs.oracle.com/javase/7/docs/api/java/lang/Thread.html#setDaemon(boolean)
> Performance issue: ClientAsyncPrefetchScanner is slower than
> ClientSimpleScanner
>
>
> Key: HBASE-18485
> URL: https://issues.apache.org/jira/browse/HBASE-18485
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-18485-v1.patch, HBASE-18485-v2.patch,
> HBASE-18485-v3.patch
>
>
> Copied the test result from HBASE-17994.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred scan 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred --asyncPrefetch=True scan 1
> {code}
> Mean latency.
> || ||Test1|| Test2 || Test3 || Test4|| Test5||
> |scan| 12.21 | 14.32 | 13.25 | 13.07 | 11.83 |
> |scan with prefetch=True | 37.36 | 37.88 | 37.56 | 37.66 | 38.28 |
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
[
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111324#comment-16111324
]
Hadoop QA commented on HBASE-18485:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
34m 22s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
33s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}127m
4s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}187m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18485 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12880043/HBASE-18485-v3.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 1134f05fb502 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / f260f09 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/7889/testReport/ |
| modules | C: hbase-client hbase-server U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7889/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Performance issue:
[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
[
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110925#comment-16110925
]
Guanghao Zhang commented on HBASE-18485:
bq. prefetcher should be background thread.
You mean?
bq. What's the rationale behind the doubling
This is same with AsyncTableResultScanner. I thought doubling is good, too.
> Performance issue: ClientAsyncPrefetchScanner is slower than
> ClientSimpleScanner
>
>
> Key: HBASE-18485
> URL: https://issues.apache.org/jira/browse/HBASE-18485
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-18485-v1.patch, HBASE-18485-v2.patch
>
>
> Copied the test result from HBASE-17994.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred scan 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred --asyncPrefetch=True scan 1
> {code}
> Mean latency.
> || ||Test1|| Test2 || Test3 || Test4|| Test5||
> |scan| 12.21 | 14.32 | 13.25 | 13.07 | 11.83 |
> |scan with prefetch=True | 37.36 | 37.88 | 37.56 | 37.66 | 38.28 |
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
[
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110670#comment-16110670
]
Ted Yu commented on HBASE-18485:
{code}
+Thread prefetcher = new Thread(prefetchRunnable);
{code}
prefetcher should be background thread.
{code}
+ private long resultSize2CacheSize(long maxResultSize) {
+// * 2 if possible
{code}
What's the rationale behind the doubling ?
{code}
+ throw new InterruptedIOException();
{code}
Better add some message to the exception.
> Performance issue: ClientAsyncPrefetchScanner is slower than
> ClientSimpleScanner
>
>
> Key: HBASE-18485
> URL: https://issues.apache.org/jira/browse/HBASE-18485
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-18485-v1.patch, HBASE-18485-v2.patch
>
>
> Copied the test result from HBASE-17994.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred scan 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred --asyncPrefetch=True scan 1
> {code}
> Mean latency.
> || ||Test1|| Test2 || Test3 || Test4|| Test5||
> |scan| 12.21 | 14.32 | 13.25 | 13.07 | 11.83 |
> |scan with prefetch=True | 37.36 | 37.88 | 37.56 | 37.66 | 38.28 |
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
[
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110626#comment-16110626
]
Hadoop QA commented on HBASE-18485:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 32s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
1s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}122m
57s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}180m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18485 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879967/HBASE-18485-v2.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux d764ff245cdf 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / f260f09 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/7885/testReport/ |
| modules | C: hbase-client hbase-server U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7885/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Performance issue:
[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
[
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110258#comment-16110258
]
Hadoop QA commented on HBASE-18485:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
35m 34s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
54s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 48m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18485 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879949/HBASE-18485-v1.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 6cb70ac6d3ea 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / f260f09 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/7882/testReport/ |
| modules | C: hbase-client U: hbase-client |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/7882/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Performance issue: ClientAsyncPrefetchScanner is slower than
> ClientSimpleScanner
>
>
> Key: HBASE-18485
> URL: https://issues.apache.org/jira/browse/HBASE-18485
> Project: HBase
> Issue Type: Bug
>Reporter:
[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
[
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110207#comment-16110207
]
Guanghao Zhang commented on HBASE-18485:
Test with the v1 patch.
Mean latency.
|| ||Test1|| Test2 || Test3 || Test4|| Test5||
|scan with prefetch=True | 8.81 | 9.26 | 9.15 | 9.52 | 8.99 |
And I run mvn clean test -Dtest=Test*Scan*. All unit tests passed.
> Performance issue: ClientAsyncPrefetchScanner is slower than
> ClientSimpleScanner
>
>
> Key: HBASE-18485
> URL: https://issues.apache.org/jira/browse/HBASE-18485
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-18485-v1.patch
>
>
> Copied the test result from HBASE-17994.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred scan 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10
> --nomapred --asyncPrefetch=True scan 1
> {code}
> Mean latency.
> || ||Test1|| Test2 || Test3 || Test4|| Test5||
> |scan| 12.21 | 14.32 | 13.25 | 13.07 | 11.83 |
> |scan with prefetch=True | 37.36 | 37.88 | 37.56 | 37.66 | 38.28 |
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
