[jira] [Commented] (HBASE-24831) Avoid invoke Counter using reflection in SnapshotInputFormat
[
https://issues.apache.org/jira/browse/HBASE-24831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17197050#comment-17197050
]
Hudson commented on HBASE-24831:
Results for branch branch-2
[build #42 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/42/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/42/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/42/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/42/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/42/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/42//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Avoid invoke Counter using reflection in SnapshotInputFormat
> -
>
> Key: HBASE-24831
> URL: https://issues.apache.org/jira/browse/HBASE-24831
> Project: HBase
> Issue Type: Improvement
>Reporter: Yechao Chen
>Assignee: Yechao Chen
>Priority: Major
> Labels: Performance, mapreduce, snapshot
> Fix For: 3.0.0-alpha-1, 2.4.0, 2.3.2
>
>
> In TableRecordReaderImpl we invoke Counter increment by reflection
> This will be called nextKeyValue() in TableSnapshotInputFormat
> reflection invoke is very slower than normal method call
> we can avoid these to improve the read performance
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24831) Avoid invoke Counter using reflection in SnapshotInputFormat
[
https://issues.apache.org/jira/browse/HBASE-24831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17196609#comment-17196609
]
Hudson commented on HBASE-24831:
Results for branch master
[build #48 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/48/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/48/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/48/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/48/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/48//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Avoid invoke Counter using reflection in SnapshotInputFormat
> -
>
> Key: HBASE-24831
> URL: https://issues.apache.org/jira/browse/HBASE-24831
> Project: HBase
> Issue Type: Improvement
>Reporter: Yechao Chen
>Assignee: Yechao Chen
>Priority: Major
> Labels: Performance, mapreduce, snapshot
> Fix For: 3.0.0-alpha-1, 2.4.0, 2.3.2
>
>
> In TableRecordReaderImpl we invoke Counter increment by reflection
> This will be called nextKeyValue() in TableSnapshotInputFormat
> reflection invoke is very slower than normal method call
> we can avoid these to improve the read performance
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24831) Avoid invoke Counter using reflection in SnapshotInputFormat
[
https://issues.apache.org/jira/browse/HBASE-24831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17196571#comment-17196571
]
Hudson commented on HBASE-24831:
Results for branch branch-2.3
[build #44 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/44/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/44/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/44/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/44/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/44/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Avoid invoke Counter using reflection in SnapshotInputFormat
> -
>
> Key: HBASE-24831
> URL: https://issues.apache.org/jira/browse/HBASE-24831
> Project: HBase
> Issue Type: Improvement
>Reporter: Yechao Chen
>Assignee: Yechao Chen
>Priority: Major
> Labels: Performance, mapreduce, snapshot
> Fix For: 3.0.0-alpha-1, 2.4.0, 2.3.2
>
>
> In TableRecordReaderImpl we invoke Counter increment by reflection
> This will be called nextKeyValue() in TableSnapshotInputFormat
> reflection invoke is very slower than normal method call
> we can avoid these to improve the read performance
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24831) Avoid invoke Counter using reflection in SnapshotInputFormat
[
https://issues.apache.org/jira/browse/HBASE-24831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190231#comment-17190231
]
Hudson commented on HBASE-24831:
Results for branch branch-2
[build #31 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/31/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/31//console].
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/31//console].
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/31//console].
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/31//console].
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/31//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Avoid invoke Counter using reflection in SnapshotInputFormat
> -
>
> Key: HBASE-24831
> URL: https://issues.apache.org/jira/browse/HBASE-24831
> Project: HBase
> Issue Type: Improvement
>Reporter: Yechao Chen
>Assignee: Yechao Chen
>Priority: Major
> Labels: Performance, mapreduce, snapshot
> Fix For: 3.0.0-alpha-1, 2.4.0, 2.3.2
>
>
> In TableRecordReaderImpl we invoke Counter increment by reflection
> This will be called nextKeyValue() in TableSnapshotInputFormat
> reflection invoke is very slower than normal method call
> we can avoid these to improve the read performance
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24831) Avoid invoke Counter using reflection in SnapshotInputFormat
[
https://issues.apache.org/jira/browse/HBASE-24831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189772#comment-17189772
]
Hudson commented on HBASE-24831:
Results for branch master
[build #33 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/33/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/33/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/33//console].
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/33/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/33//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Avoid invoke Counter using reflection in SnapshotInputFormat
> -
>
> Key: HBASE-24831
> URL: https://issues.apache.org/jira/browse/HBASE-24831
> Project: HBase
> Issue Type: Improvement
>Reporter: Yechao Chen
>Assignee: Yechao Chen
>Priority: Major
> Labels: Performance, mapreduce, snapshot
> Fix For: 3.0.0-alpha-1, 2.4.0, 2.3.2
>
>
> In TableRecordReaderImpl we invoke Counter increment by reflection
> This will be called nextKeyValue() in TableSnapshotInputFormat
> reflection invoke is very slower than normal method call
> we can avoid these to improve the read performance
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24831) Avoid invoke Counter using reflection in SnapshotInputFormat
[
https://issues.apache.org/jira/browse/HBASE-24831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189630#comment-17189630
]
Hudson commented on HBASE-24831:
Results for branch branch-2.3
[build #27 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/27/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/27/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/27//console].
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/27//console].
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/27//console].
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Avoid invoke Counter using reflection in SnapshotInputFormat
> -
>
> Key: HBASE-24831
> URL: https://issues.apache.org/jira/browse/HBASE-24831
> Project: HBase
> Issue Type: Improvement
>Reporter: Yechao Chen
>Assignee: Yechao Chen
>Priority: Major
> Labels: Performance, mapreduce, snapshot
> Fix For: 3.0.0-alpha-1, 2.4.0, 2.3.2
>
>
> In TableRecordReaderImpl we invoke Counter increment by reflection
> This will be called nextKeyValue() in TableSnapshotInputFormat
> reflection invoke is very slower than normal method call
> we can avoid these to improve the read performance
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24831) Avoid invoke Counter using reflection in SnapshotInputFormat
[
https://issues.apache.org/jira/browse/HBASE-24831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17172940#comment-17172940
]
Yechao Chen commented on HBASE-24831:
-
with the pr we get about 7.6%/9.1% performance improve
in my test case with ScanPerformanceEvaluation tools
like that
{{hbase org.apache.hadoop.hbase.ScanPerformanceEvaluation -t
snapshotscanmapreduce -rs /tmp/hbase/rs1 -sn sp_test}}
*with pr*
1#
total time to scan: 143943 ms
total bytes: 16357785600 bytes (15.2 G)
throughput : 109.1 MB/s
total rows : 10485760
throughput : 71.6 K rows/s
total cells : 104857600
throughput : 716.1 K cells/s
2#
total time to scan: 145948 ms
total bytes: 16357785600 bytes (15.2 G)
throughput : 107.6 MB/s
total rows : 10485760
throughput : 70.6 K rows/s
total cells : 104857600
throughput : 706.2 K cells/s
total time to scan: 149962 ms
total bytes: 16357785600 bytes (15.2 G)
throughput : 104.7 MB/s
total rows : 10485760
throughput : 68.7 K rows/s
total cells : 104857600
throughput : 687.2 K cells/s
*without pr*
1th:
total time to scan: 156929 ms
total bytes: 16357785600 bytes (15.2 G)
throughput : 100 MB/s
total rows : 10485760
throughput : 65.6 K rows/s
total cells : 104857600
throughput : 656.4 K cells/s
2th
total time to scan: 156078 ms
total bytes: 16357785600 bytes (15.2 G)
throughput : 100 MB/s
total rows : 10485760
throughput : 65.6 K rows/s
total cells : 104857600
throughput : 656.4 K cells/s
> Avoid invoke Counter using reflection in SnapshotInputFormat
> -
>
> Key: HBASE-24831
> URL: https://issues.apache.org/jira/browse/HBASE-24831
> Project: HBase
> Issue Type: Improvement
>Reporter: Yechao Chen
>Assignee: Yechao Chen
>Priority: Major
>
> In TableRecordReaderImpl we invoke Counter increment by reflection
> This will be called nextKeyValue() in TableSnapshotInputFormat
> reflection invoke is very slower than normal method call
> we can avoid these to improve the read performance
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24831) Avoid invoke Counter using reflection in SnapshotInputFormat
[ https://issues.apache.org/jira/browse/HBASE-24831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17172944#comment-17172944 ] Yechao Chen commented on HBASE-24831: - more details please see [GitHub Pull Request #2209|https://github.com/apache/hbase/pull/2209] > Avoid invoke Counter using reflection in SnapshotInputFormat > - > > Key: HBASE-24831 > URL: https://issues.apache.org/jira/browse/HBASE-24831 > Project: HBase > Issue Type: Improvement >Reporter: Yechao Chen >Assignee: Yechao Chen >Priority: Major > > In TableRecordReaderImpl we invoke Counter increment by reflection > This will be called nextKeyValue() in TableSnapshotInputFormat > reflection invoke is very slower than normal method call > we can avoid these to improve the read performance -- This message was sent by Atlassian Jira (v8.3.4#803005)
