[jira] [Commented] (HBASE-21926) Profiler servlet
[
https://issues.apache.org/jira/browse/HBASE-21926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794740#comment-16794740
]
Hudson commented on HBASE-21926:
Results for branch branch-1.4
[build #703 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/703/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/703//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/703//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/703//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Profiler servlet
>
>
> Key: HBASE-21926
> URL: https://issues.apache.org/jira/browse/HBASE-21926
> Project: HBase
> Issue Type: New Feature
> Components: master, Operability, regionserver
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.4.10, 1.3.4, 2.3.0, 2.1.4, 1.2.12, 2.2.1
>
> Attachments: 1.png, 2.png, 3.png, 4.png, HBASE-21926-branch-1.patch,
> HBASE-21926-branch-1.patch, HBASE-21926-branch-1.patch,
> HBASE-21926-branch-1.patch, HBASE-21926.patch, HBASE-21926.patch,
> HBASE-21926.patch, HBASE-21926.patch
>
>
> HIVE-20202 describes how Hive added a web endpoint for online in production
> profiling based on async-profiler. The endpoint was added as a servlet to
> httpserver and supports retrieval of flamegraphs compiled from the profiler
> trace. Async profiler
> ([https://github.com/jvm-profiling-tools/async-profiler] ) can also profile
> heap allocations, lock contention, and HW performance counters in addition to
> CPU.
> The profiling overhead is pretty low and is safe to run in production. The
> async-profiler project measured and describes CPU and memory overheads on
> these issues:
> [https://github.com/jvm-profiling-tools/async-profiler/issues/14] and
> [https://github.com/jvm-profiling-tools/async-profiler/issues/131]
> We have an httpserver based servlet stack so we can use HIVE-20202 as an
> implementation template for a similar feature for HBase daemons. Ideally we
> achieve these requirements:
> * Retrieve flamegraph SVG generated from latest profile trace.
> * Online enable and disable of profiling activity. (async-profiler does not
> do instrumentation based profiling so this should not cause the code gen
> related perf problems of that other approach and can be safely toggled on and
> off while under production load.)
> * CPU profiling.
> * ALLOCATION profiling.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21926) Profiler servlet
[ https://issues.apache.org/jira/browse/HBASE-21926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794721#comment-16794721 ] Hudson commented on HBASE-21926: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1211 (See [https://builds.apache.org/job/HBase-1.2-IT/1211/]) HBASE-21926 Profiler servlet (apurtell: [https://github.com/apache/hbase/commit/5dfe1ccbabe3c4c677033fbd025544d03c6ed674]) * (edit) hbase-server/src/main/resources/hbase-webapps/master/table.jsp * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/http/ProfileServlet.java * (edit) hbase-rest/src/main/resources/hbase-webapps/rest/rest.jsp * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/util/ProcessUtils.java * (edit) hbase-server/src/main/resources/hbase-webapps/master/snapshot.jsp * (edit) hbase-thrift/src/main/resources/hbase-webapps/thrift/thrift.jsp * (edit) src/main/asciidoc/book.adoc * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java * (edit) hbase-server/src/main/resources/hbase-webapps/master/zk.jsp * (edit) hbase-server/src/main/resources/hbase-webapps/master/tablesDetailed.jsp * (edit) hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RSStatusTmpl.jamon * (edit) bin/hbase * (edit) hbase-server/src/main/resources/hbase-webapps/thrift/thrift.jsp * (edit) hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/http/ProfileOutputServlet.java * (edit) hbase-server/src/main/resources/hbase-webapps/regionserver/region.jsp * (edit) hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon * (add) src/main/asciidoc/_chapters/profiler.adoc > Profiler servlet > > > Key: HBASE-21926 > URL: https://issues.apache.org/jira/browse/HBASE-21926 > Project: HBase > Issue Type: New Feature > Components: master, Operability, regionserver >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 3.0.0, 1.5.0, 1.4.10, 1.3.4, 2.3.0, 2.1.4, 1.2.12, 2.2.1 > > Attachments: 1.png, 2.png, 3.png, 4.png, HBASE-21926-branch-1.patch, > HBASE-21926-branch-1.patch, HBASE-21926-branch-1.patch, > HBASE-21926-branch-1.patch, HBASE-21926.patch, HBASE-21926.patch, > HBASE-21926.patch, HBASE-21926.patch > > > HIVE-20202 describes how Hive added a web endpoint for online in production > profiling based on async-profiler. The endpoint was added as a servlet to > httpserver and supports retrieval of flamegraphs compiled from the profiler > trace. Async profiler > ([https://github.com/jvm-profiling-tools/async-profiler] ) can also profile > heap allocations, lock contention, and HW performance counters in addition to > CPU. > The profiling overhead is pretty low and is safe to run in production. The > async-profiler project measured and describes CPU and memory overheads on > these issues: > [https://github.com/jvm-profiling-tools/async-profiler/issues/14] and > [https://github.com/jvm-profiling-tools/async-profiler/issues/131] > We have an httpserver based servlet stack so we can use HIVE-20202 as an > implementation template for a similar feature for HBase daemons. Ideally we > achieve these requirements: > * Retrieve flamegraph SVG generated from latest profile trace. > * Online enable and disable of profiling activity. (async-profiler does not > do instrumentation based profiling so this should not cause the code gen > related perf problems of that other approach and can be safely toggled on and > off while under production load.) > * CPU profiling. > * ALLOCATION profiling. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21926) Profiler servlet
[
https://issues.apache.org/jira/browse/HBASE-21926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794712#comment-16794712
]
Hudson commented on HBASE-21926:
Results for branch branch-1.2
[build #698 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/698/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/698//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/698//JDK7_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/698//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Profiler servlet
>
>
> Key: HBASE-21926
> URL: https://issues.apache.org/jira/browse/HBASE-21926
> Project: HBase
> Issue Type: New Feature
> Components: master, Operability, regionserver
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.4.10, 1.3.4, 2.3.0, 2.1.4, 1.2.12, 2.2.1
>
> Attachments: 1.png, 2.png, 3.png, 4.png, HBASE-21926-branch-1.patch,
> HBASE-21926-branch-1.patch, HBASE-21926-branch-1.patch,
> HBASE-21926-branch-1.patch, HBASE-21926.patch, HBASE-21926.patch,
> HBASE-21926.patch, HBASE-21926.patch
>
>
> HIVE-20202 describes how Hive added a web endpoint for online in production
> profiling based on async-profiler. The endpoint was added as a servlet to
> httpserver and supports retrieval of flamegraphs compiled from the profiler
> trace. Async profiler
> ([https://github.com/jvm-profiling-tools/async-profiler] ) can also profile
> heap allocations, lock contention, and HW performance counters in addition to
> CPU.
> The profiling overhead is pretty low and is safe to run in production. The
> async-profiler project measured and describes CPU and memory overheads on
> these issues:
> [https://github.com/jvm-profiling-tools/async-profiler/issues/14] and
> [https://github.com/jvm-profiling-tools/async-profiler/issues/131]
> We have an httpserver based servlet stack so we can use HIVE-20202 as an
> implementation template for a similar feature for HBase daemons. Ideally we
> achieve these requirements:
> * Retrieve flamegraph SVG generated from latest profile trace.
> * Online enable and disable of profiling activity. (async-profiler does not
> do instrumentation based profiling so this should not cause the code gen
> related perf problems of that other approach and can be safely toggled on and
> off while under production load.)
> * CPU profiling.
> * ALLOCATION profiling.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21926) Profiler servlet
[
https://issues.apache.org/jira/browse/HBASE-21926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794714#comment-16794714
]
Hudson commented on HBASE-21926:
Results for branch branch-1.3
[build #686 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/686/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/686//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/686//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/686//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Profiler servlet
>
>
> Key: HBASE-21926
> URL: https://issues.apache.org/jira/browse/HBASE-21926
> Project: HBase
> Issue Type: New Feature
> Components: master, Operability, regionserver
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.4.10, 1.3.4, 2.3.0, 2.1.4, 1.2.12, 2.2.1
>
> Attachments: 1.png, 2.png, 3.png, 4.png, HBASE-21926-branch-1.patch,
> HBASE-21926-branch-1.patch, HBASE-21926-branch-1.patch,
> HBASE-21926-branch-1.patch, HBASE-21926.patch, HBASE-21926.patch,
> HBASE-21926.patch, HBASE-21926.patch
>
>
> HIVE-20202 describes how Hive added a web endpoint for online in production
> profiling based on async-profiler. The endpoint was added as a servlet to
> httpserver and supports retrieval of flamegraphs compiled from the profiler
> trace. Async profiler
> ([https://github.com/jvm-profiling-tools/async-profiler] ) can also profile
> heap allocations, lock contention, and HW performance counters in addition to
> CPU.
> The profiling overhead is pretty low and is safe to run in production. The
> async-profiler project measured and describes CPU and memory overheads on
> these issues:
> [https://github.com/jvm-profiling-tools/async-profiler/issues/14] and
> [https://github.com/jvm-profiling-tools/async-profiler/issues/131]
> We have an httpserver based servlet stack so we can use HIVE-20202 as an
> implementation template for a similar feature for HBase daemons. Ideally we
> achieve these requirements:
> * Retrieve flamegraph SVG generated from latest profile trace.
> * Online enable and disable of profiling activity. (async-profiler does not
> do instrumentation based profiling so this should not cause the code gen
> related perf problems of that other approach and can be safely toggled on and
> off while under production load.)
> * CPU profiling.
> * ALLOCATION profiling.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22045) Mutable range histogram reports incorrect outliers
[
https://issues.apache.org/jira/browse/HBASE-22045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794713#comment-16794713
]
Hudson commented on HBASE-22045:
Results for branch branch-1.3
[build #686 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/686/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/686//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/686//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/686//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Mutable range histogram reports incorrect outliers
> --
>
> Key: HBASE-22045
> URL: https://issues.apache.org/jira/browse/HBASE-22045
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 1.5.0, 1.3.3, 2.0.0, 1.4.9, 2.1.3, 2.2.1
>Reporter: Abhishek Singh Chouhan
>Assignee: Abhishek Singh Chouhan
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.4.10, 1.3.4, 2.3.0, 2.1.5, 2.2.1
>
> Attachments: HBASE-22045.master.001.patch
>
>
> MutableRangeHistogram during the snapshot calculates the outliers (eg.
> mutate_TimeRange_60-inf) and adds the counter with incorrect calculation
> by using the overall count of event and not the number of events in the
> snapshot.
> {code:java}
> long val = histogram.getCount();
> if (val - cumNum > 0) {
> metricsRecordBuilder.addCounter(
> Interns.info(name + "_" + rangeType + "_" + ranges[ranges.length -
> 1] + "-inf", desc),
> val - cumNum);
> }{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21964) unset Quota by Throttle Type
[
https://issues.apache.org/jira/browse/HBASE-21964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yaojingyi updated HBASE-21964:
--
Attachment: HBASE-21964.master.006.patch
> unset Quota by Throttle Type
>
>
> Key: HBASE-21964
> URL: https://issues.apache.org/jira/browse/HBASE-21964
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 1.4.8
>Reporter: yaojingyi
>Assignee: yaojingyi
>Priority: Major
> Attachments: HBASE-21964.branch-1.4.001.patch,
> HBASE-21964.master.001.patch, HBASE-21964.master.002.patch,
> HBASE-21964.master.003.patch, HBASE-21964.master.004.patch,
> HBASE-21964.master.005.patch, HBASE-21964.master.006.patch,
> unthrottleByType.patch
>
>
>
> {code:java}
> //first set_quota to USER=> 'u1'
> set_quota TYPE => THROTTLE, USER => 'u1', THROTTLE_TYPE => WRITE, LIMIT =>
> '1000req/sec'
> //then
> hbase(main):004:0> set_quota TYPE => THROTTLE, THROTTLE_TYPE => WRITE, USER
> => 'u1', LIMIT => NONE
> ERROR: Unexpected arguments: {"THROTTLE_TYPE"=>"WRITE"}
> // or try "THROTTLE_TYPE"=>"WRITE_NUMBER"
> hbase(main):012:0* set_quota TYPE => THROTTLE, THROTTLE_TYPE => WRITE_NUMBER,
> USER => 'u1', LIMIT => NONE
> NameError: uninitialized constant WRITE_NUMBER
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22015) UserPermission should be annotated as InterfaceAudience.Public
[
https://issues.apache.org/jira/browse/HBASE-22015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Mei updated HBASE-22015:
---
Attachment: HBASE-22015.master.003.patch
> UserPermission should be annotated as InterfaceAudience.Public
> --
>
> Key: HBASE-22015
> URL: https://issues.apache.org/jira/browse/HBASE-22015
> Project: HBase
> Issue Type: Sub-task
>Reporter: Yi Mei
>Assignee: Yi Mei
>Priority: Blocker
> Fix For: 3.0.0, 2.2.0, 2.3.0
>
> Attachments: HBASE-22015.master.001.patch,
> HBASE-22015.master.002.patch, HBASE-22015.master.003.patch
>
>
> HBASE-11318 mark UserPermission as InterfaceAudience.Private.
> HBASE-11452 instroduce AccessControlClient#getUserPermissions and return
> UserPermission list but the UserPermission class is Private.
> I also encounter the same problem when I want to move getUserPermissions
> method as a admin api in HBASE-21911, otherwise the api of getUserPermissions
> may be
> {code:java}
> Map> getUserPermissions{code}
> So shall we mark the UserPermission as Public? discussions are welcomed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22045) Mutable range histogram reports incorrect outliers
[
https://issues.apache.org/jira/browse/HBASE-22045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794660#comment-16794660
]
Hudson commented on HBASE-22045:
SUCCESS: Integrated in Jenkins build HBase-1.3-IT #530 (See
[https://builds.apache.org/job/HBase-1.3-IT/530/])
Revert "HBASE-22045 Mutable range histogram reports incorrect outliers"
(apurtell:
[https://github.com/apache/hbase/commit/2fc27892fac883abba4e041335d8422d79b0a262])
* (edit)
hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MutableRangeHistogram.java
> Mutable range histogram reports incorrect outliers
> --
>
> Key: HBASE-22045
> URL: https://issues.apache.org/jira/browse/HBASE-22045
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 1.5.0, 1.3.3, 2.0.0, 1.4.9, 2.1.3, 2.2.1
>Reporter: Abhishek Singh Chouhan
>Assignee: Abhishek Singh Chouhan
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.4.10, 1.3.4, 2.3.0, 2.1.5, 2.2.1
>
> Attachments: HBASE-22045.master.001.patch
>
>
> MutableRangeHistogram during the snapshot calculates the outliers (eg.
> mutate_TimeRange_60-inf) and adds the counter with incorrect calculation
> by using the overall count of event and not the number of events in the
> snapshot.
> {code:java}
> long val = histogram.getCount();
> if (val - cumNum > 0) {
> metricsRecordBuilder.addCounter(
> Interns.info(name + "_" + rangeType + "_" + ranges[ranges.length -
> 1] + "-inf", desc),
> val - cumNum);
> }{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21926) Profiler servlet
[ https://issues.apache.org/jira/browse/HBASE-21926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794661#comment-16794661 ] Hudson commented on HBASE-21926: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #530 (See [https://builds.apache.org/job/HBase-1.3-IT/530/]) HBASE-21926 Profiler servlet (apurtell: [https://github.com/apache/hbase/commit/db8a24797e5db17804fb722498f2d9c49d979e5a]) * (edit) hbase-server/src/main/resources/hbase-webapps/master/snapshotsStats.jsp * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/util/ProcessUtils.java * (edit) hbase-thrift/src/main/resources/hbase-webapps/thrift/thrift.jsp * (edit) hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon * (add) src/main/asciidoc/_chapters/profiler.adoc * (edit) src/main/asciidoc/book.adoc * (edit) hbase-server/src/main/resources/hbase-webapps/master/tablesDetailed.jsp * (edit) bin/hbase * (edit) hbase-server/src/main/resources/hbase-webapps/master/zk.jsp * (edit) hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RSStatusTmpl.jamon * (edit) hbase-rest/src/main/resources/hbase-webapps/rest/rest.jsp * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/http/ProfileOutputServlet.java * (edit) hbase-server/src/main/resources/hbase-webapps/regionserver/region.jsp * (edit) hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp * (edit) hbase-server/src/main/resources/hbase-webapps/master/procedures.jsp * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java * (edit) hbase-server/src/main/resources/hbase-webapps/master/table.jsp * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/http/ProfileServlet.java * (edit) hbase-server/src/main/resources/hbase-webapps/master/snapshot.jsp * (edit) hbase-server/src/main/resources/hbase-webapps/thrift/thrift.jsp > Profiler servlet > > > Key: HBASE-21926 > URL: https://issues.apache.org/jira/browse/HBASE-21926 > Project: HBase > Issue Type: New Feature > Components: master, Operability, regionserver >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 3.0.0, 1.5.0, 1.4.10, 1.3.4, 2.3.0, 2.1.4, 1.2.12, 2.2.1 > > Attachments: 1.png, 2.png, 3.png, 4.png, HBASE-21926-branch-1.patch, > HBASE-21926-branch-1.patch, HBASE-21926-branch-1.patch, > HBASE-21926-branch-1.patch, HBASE-21926.patch, HBASE-21926.patch, > HBASE-21926.patch, HBASE-21926.patch > > > HIVE-20202 describes how Hive added a web endpoint for online in production > profiling based on async-profiler. The endpoint was added as a servlet to > httpserver and supports retrieval of flamegraphs compiled from the profiler > trace. Async profiler > ([https://github.com/jvm-profiling-tools/async-profiler] ) can also profile > heap allocations, lock contention, and HW performance counters in addition to > CPU. > The profiling overhead is pretty low and is safe to run in production. The > async-profiler project measured and describes CPU and memory overheads on > these issues: > [https://github.com/jvm-profiling-tools/async-profiler/issues/14] and > [https://github.com/jvm-profiling-tools/async-profiler/issues/131] > We have an httpserver based servlet stack so we can use HIVE-20202 as an > implementation template for a similar feature for HBase daemons. Ideally we > achieve these requirements: > * Retrieve flamegraph SVG generated from latest profile trace. > * Online enable and disable of profiling activity. (async-profiler does not > do instrumentation based profiling so this should not cause the code gen > related perf problems of that other approach and can be safely toggled on and > off while under production load.) > * CPU profiling. > * ALLOCATION profiling. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22049) getReopenStatus() didn't skip counting split parent region
[
https://issues.apache.org/jira/browse/HBASE-22049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingyun Tian updated HBASE-22049:
-
Attachment: HBASE-22049.master.002.patch
> getReopenStatus() didn't skip counting split parent region
> --
>
> Key: HBASE-22049
> URL: https://issues.apache.org/jira/browse/HBASE-22049
> Project: HBase
> Issue Type: Bug
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
> Attachments: HBASE-22049.master.001.patch,
> HBASE-22049.master.002.patch
>
>
> After we modify some attributes of table, hbaseAdmin will getAlterStatus to
> check if all region's attributes updated. It will skip opened region and
> split region as the following code shows.
> {code}
> for (RegionState regionState: states) {
> if (!regionState.isOpened() && !regionState.isSplit()) {
> ritCount++;
> }
> }
> {code}
> But since now the split procedure is to unassign the split parent region,
> thus the state is CLOSED， and the check will hang there until timeout.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Reopened] (HBASE-22045) Mutable range histogram reports incorrect outliers
[
https://issues.apache.org/jira/browse/HBASE-22045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell reopened HBASE-22045:
I reverted this from branch-1.3 because the cherry pick of this commit broke
the build. Please reapply and re-resolve.
> Mutable range histogram reports incorrect outliers
> --
>
> Key: HBASE-22045
> URL: https://issues.apache.org/jira/browse/HBASE-22045
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 1.5.0, 1.3.3, 2.0.0, 1.4.9, 2.1.3, 2.2.1
>Reporter: Abhishek Singh Chouhan
>Assignee: Abhishek Singh Chouhan
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.4.10, 1.3.4, 2.3.0, 2.1.5, 2.2.1
>
> Attachments: HBASE-22045.master.001.patch
>
>
> MutableRangeHistogram during the snapshot calculates the outliers (eg.
> mutate_TimeRange_60-inf) and adds the counter with incorrect calculation
> by using the overall count of event and not the number of events in the
> snapshot.
> {code:java}
> long val = histogram.getCount();
> if (val - cumNum > 0) {
> metricsRecordBuilder.addCounter(
> Interns.info(name + "_" + rangeType + "_" + ranges[ranges.length -
> 1] + "-inf", desc),
> val - cumNum);
> }{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21926) Profiler servlet
[ https://issues.apache.org/jira/browse/HBASE-21926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21926: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.2.1 1.2.12 2.1.4 1.3.4 1.4.10 Status: Resolved (was: Patch Available) Pushed to all active branches. > Profiler servlet > > > Key: HBASE-21926 > URL: https://issues.apache.org/jira/browse/HBASE-21926 > Project: HBase > Issue Type: New Feature > Components: master, Operability, regionserver >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 3.0.0, 1.5.0, 1.4.10, 1.3.4, 2.3.0, 2.1.4, 1.2.12, 2.2.1 > > Attachments: 1.png, 2.png, 3.png, 4.png, HBASE-21926-branch-1.patch, > HBASE-21926-branch-1.patch, HBASE-21926-branch-1.patch, > HBASE-21926-branch-1.patch, HBASE-21926.patch, HBASE-21926.patch, > HBASE-21926.patch, HBASE-21926.patch > > > HIVE-20202 describes how Hive added a web endpoint for online in production > profiling based on async-profiler. The endpoint was added as a servlet to > httpserver and supports retrieval of flamegraphs compiled from the profiler > trace. Async profiler > ([https://github.com/jvm-profiling-tools/async-profiler] ) can also profile > heap allocations, lock contention, and HW performance counters in addition to > CPU. > The profiling overhead is pretty low and is safe to run in production. The > async-profiler project measured and describes CPU and memory overheads on > these issues: > [https://github.com/jvm-profiling-tools/async-profiler/issues/14] and > [https://github.com/jvm-profiling-tools/async-profiler/issues/131] > We have an httpserver based servlet stack so we can use HIVE-20202 as an > implementation template for a similar feature for HBase daemons. Ideally we > achieve these requirements: > * Retrieve flamegraph SVG generated from latest profile trace. > * Online enable and disable of profiling activity. (async-profiler does not > do instrumentation based profiling so this should not cause the code gen > related perf problems of that other approach and can be safely toggled on and > off while under production load.) > * CPU profiling. > * ALLOCATION profiling. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22055) Support to compare different CF names between source and peer clusters in VerifyReplication
Toshihiro Suzuki created HBASE-22055:
Summary: Support to compare different CF names between source and
peer clusters in VerifyReplication
Key: HBASE-22055
URL: https://issues.apache.org/jira/browse/HBASE-22055
Project: HBase
Issue Type: Improvement
Reporter: Toshihiro Suzuki
Assignee: Toshihiro Suzuki
After HBASE-21201 and HBASE-21871, we can compare any 2 tables in any remote
clusters with specifying both peerId and --peerTableName in VerifyReplication.
However, we can compare only the same CF names between source and peer clusters
in the tool. So I would like to propose to allow the tool to be able to compare
different CF names between source and peer clusters.
Like CopyTable, I think we can make the tool to be able to specify column
family mapping with --families like the following:
{code}
--families=sourceCf:peerCf,cf2,cf3
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22034) Backport HBASE-21401 and HBASE-22032 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-22034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794636#comment-16794636 ] Andrew Purtell commented on HBASE-22034: Unit test failure is not related as far as I can tell. Does not reproduce locally. It looks like a flake in the precommit env. I can fix the one remaining checkstyle nit upon commit. (Indentation at TestKeyValue.java:573) > Backport HBASE-21401 and HBASE-22032 to branch-1 > > > Key: HBASE-22034 > URL: https://issues.apache.org/jira/browse/HBASE-22034 > Project: HBase > Issue Type: Improvement >Reporter: Geoffrey Jacoby >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0, 1.4.10, 1.3.4, 1.2.12 > > Attachments: HBASE-22034-branch-1.patch, HBASE-22034-branch-1.patch > > > Branch-2 and master have good validation checks when constructing KeyValues. > We should also have them on branch-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22031) Provide RSGroupInfo with a new constructor using shallow copy
[
https://issues.apache.org/jira/browse/HBASE-22031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794648#comment-16794648
]
Xiang Li commented on HBASE-22031:
--
[~xucang], would you please help to review patch v002 at your convenience?
> Provide RSGroupInfo with a new constructor using shallow copy
> -
>
> Key: HBASE-22031
> URL: https://issues.apache.org/jira/browse/HBASE-22031
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22031.master.000.patch,
> HBASE-22031.master.001.patch, HBASE-22031.master.002.patch
>
>
> As for org.apache.hadoop.hbase.rsgroup.RSGroupInfo, the following constructor
> performs deep copies on both servers and tables inputed.
> {code:title=hbase-common/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfo.java.java|borderStyle=solid}
> RSGroupInfo(String name, SortedSet servers, SortedSet
> tables) {
> this.name = name;
> this.servers = (servers == null) ? new TreeSet<>() : new TreeSet<>(servers);
> this.tables = (tables == null) ? new TreeSet<>() : new TreeSet<>(tables);
> }
> {code}
> The constructor of TreeSet is heavy and I think it is better to have a new
> constructor with shallow copy and it could be used at least in
> {code:title=hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManagerImpl.java|borderStyle=solid}
> private synchronized void refresh(boolean forceOnline) throws IOException {
> ...
> groupList.add(new RSGroupInfo(RSGroupInfo.DEFAULT_GROUP,
> getDefaultServers(), orphanTables));
> ...
> {code}
> It is not needed to allocate new TreeSet to deep copy the output of
> getDefaultServers() and orphanTables, both of which are allocated in the near
> context and not updated in the code followed. So it is safe to make a shadow
> copy here.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21926) Profiler servlet
[ https://issues.apache.org/jira/browse/HBASE-21926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794645#comment-16794645 ] Andrew Purtell commented on HBASE-21926: Committing. Attaching what I committed, with a bit more detail in the adoc for the two other significant parameters 'output' and 'duration' > Profiler servlet > > > Key: HBASE-21926 > URL: https://issues.apache.org/jira/browse/HBASE-21926 > Project: HBase > Issue Type: New Feature > Components: master, Operability, regionserver >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0 > > Attachments: 1.png, 2.png, 3.png, 4.png, HBASE-21926-branch-1.patch, > HBASE-21926-branch-1.patch, HBASE-21926-branch-1.patch, > HBASE-21926-branch-1.patch, HBASE-21926.patch, HBASE-21926.patch, > HBASE-21926.patch, HBASE-21926.patch > > > HIVE-20202 describes how Hive added a web endpoint for online in production > profiling based on async-profiler. The endpoint was added as a servlet to > httpserver and supports retrieval of flamegraphs compiled from the profiler > trace. Async profiler > ([https://github.com/jvm-profiling-tools/async-profiler] ) can also profile > heap allocations, lock contention, and HW performance counters in addition to > CPU. > The profiling overhead is pretty low and is safe to run in production. The > async-profiler project measured and describes CPU and memory overheads on > these issues: > [https://github.com/jvm-profiling-tools/async-profiler/issues/14] and > [https://github.com/jvm-profiling-tools/async-profiler/issues/131] > We have an httpserver based servlet stack so we can use HIVE-20202 as an > implementation template for a similar feature for HBase daemons. Ideally we > achieve these requirements: > * Retrieve flamegraph SVG generated from latest profile trace. > * Online enable and disable of profiling activity. (async-profiler does not > do instrumentation based profiling so this should not cause the code gen > related perf problems of that other approach and can be safely toggled on and > off while under production load.) > * CPU profiling. > * ALLOCATION profiling. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21926) Profiler servlet
[ https://issues.apache.org/jira/browse/HBASE-21926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21926: --- Attachment: HBASE-21926.patch HBASE-21926-branch-1.patch > Profiler servlet > > > Key: HBASE-21926 > URL: https://issues.apache.org/jira/browse/HBASE-21926 > Project: HBase > Issue Type: New Feature > Components: master, Operability, regionserver >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0 > > Attachments: 1.png, 2.png, 3.png, 4.png, HBASE-21926-branch-1.patch, > HBASE-21926-branch-1.patch, HBASE-21926-branch-1.patch, > HBASE-21926-branch-1.patch, HBASE-21926.patch, HBASE-21926.patch, > HBASE-21926.patch, HBASE-21926.patch > > > HIVE-20202 describes how Hive added a web endpoint for online in production > profiling based on async-profiler. The endpoint was added as a servlet to > httpserver and supports retrieval of flamegraphs compiled from the profiler > trace. Async profiler > ([https://github.com/jvm-profiling-tools/async-profiler] ) can also profile > heap allocations, lock contention, and HW performance counters in addition to > CPU. > The profiling overhead is pretty low and is safe to run in production. The > async-profiler project measured and describes CPU and memory overheads on > these issues: > [https://github.com/jvm-profiling-tools/async-profiler/issues/14] and > [https://github.com/jvm-profiling-tools/async-profiler/issues/131] > We have an httpserver based servlet stack so we can use HIVE-20202 as an > implementation template for a similar feature for HBase daemons. Ideally we > achieve these requirements: > * Retrieve flamegraph SVG generated from latest profile trace. > * Online enable and disable of profiling activity. (async-profiler does not > do instrumentation based profiling so this should not cause the code gen > related perf problems of that other approach and can be safely toggled on and > off while under production load.) > * CPU profiling. > * ALLOCATION profiling. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21871) Support to specify a peer table name in VerifyReplication tool
[ https://issues.apache.org/jira/browse/HBASE-21871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HBASE-21871: - Release Note: After HBASE-21871, we can specify a peer table name with --peerTableName in VerifyReplication tool like the following: hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication --peerTableName=peerTable 5 TestTable In addition, we can compare any 2 tables in any remote clusters with specifying both peerId and --peerTableName. For example: hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication --peerTableName=peerTable zk1,zk2,zk3:2181/hbase TestTable was: After HBASE-21871, we can specify a peer table name in VerifyReplication tool like the following: hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication --peerTableName=peerTable 5 TestTable In addition, we can compare any 2 tables in any remote clusters with specifying both peerId and --peerTableName. For example: hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication --peerTableName=peerTable zk1,zk2,zk3:2181/hbase TestTable > Support to specify a peer table name in VerifyReplication tool > -- > > Key: HBASE-21871 > URL: https://issues.apache.org/jira/browse/HBASE-21871 > Project: HBase > Issue Type: Improvement >Reporter: Toshihiro Suzuki >Assignee: Subrat Mishra >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.1.4, 2.2.1 > > Attachments: HBASE-21871.master.001.patch, > HBASE-21871.master.002.patch, HBASE-21871.master.002.patch > > > After HBASE-21201, we can specify peerQuorumAddress instead of peerId in > VerifyReplication tool. So it no longer requires peerId to be setup when > using this tool. However, we don't have a way to specify a peer table name in > VerifyReplication for now. > So I would like to propose to update the tool to pass as a peer table name as > an argument (ex. --peerTableName=). > After resolving this Jira, we will be able to compare any 2 tables in any > remote clusters. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21871) Support to specify a peer table name in VerifyReplication tool
[ https://issues.apache.org/jira/browse/HBASE-21871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HBASE-21871: - Release Note: After HBASE-21871, we can specify a peer table name in VerifyReplication tool like the following: hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication --peerTableName=peerTable 5 TestTable In addition, we can compare any 2 tables in any remote clusters with specifying both peerId and --peerTableName. For example: hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication --peerTableName=peerTable zk1,zk2,zk3:2181/hbase TestTable > Support to specify a peer table name in VerifyReplication tool > -- > > Key: HBASE-21871 > URL: https://issues.apache.org/jira/browse/HBASE-21871 > Project: HBase > Issue Type: Improvement >Reporter: Toshihiro Suzuki >Assignee: Subrat Mishra >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.1.4, 2.2.1 > > Attachments: HBASE-21871.master.001.patch, > HBASE-21871.master.002.patch, HBASE-21871.master.002.patch > > > After HBASE-21201, we can specify peerQuorumAddress instead of peerId in > VerifyReplication tool. So it no longer requires peerId to be setup when > using this tool. However, we don't have a way to specify a peer table name in > VerifyReplication for now. > So I would like to propose to update the tool to pass as a peer table name as > an argument (ex. --peerTableName=). > After resolving this Jira, we will be able to compare any 2 tables in any > remote clusters. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22034) Backport HBASE-21401 and HBASE-22032 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-22034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794639#comment-16794639 ] Andrew Purtell commented on HBASE-22034: Mind taking a look if you have a sec [~abhishek.chouhan]? This is something [~gjacoby] worked on for later branches, now we are backporting it and the base commit to branch-1. Not essential but a nice to have sanity check. > Backport HBASE-21401 and HBASE-22032 to branch-1 > > > Key: HBASE-22034 > URL: https://issues.apache.org/jira/browse/HBASE-22034 > Project: HBase > Issue Type: Improvement >Reporter: Geoffrey Jacoby >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0, 1.4.10, 1.3.4, 1.2.12 > > Attachments: HBASE-22034-branch-1.patch, HBASE-22034-branch-1.patch > > > Branch-2 and master have good validation checks when constructing KeyValues. > We should also have them on branch-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21935) Replace make_rc.sh with customized spark/dev/create-release
[ https://issues.apache.org/jira/browse/HBASE-21935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21935: -- Attachment: HBASE-21935.branch-2.0.010.patch > Replace make_rc.sh with customized spark/dev/create-release > --- > > Key: HBASE-21935 > URL: https://issues.apache.org/jira/browse/HBASE-21935 > Project: HBase > Issue Type: Task > Components: rm >Reporter: stack >Assignee: stack >Priority: Minor > Labels: rm > Attachments: HBASE-21935.branch-2.0.001.patch, > HBASE-21935.branch-2.0.002.patch, HBASE-21935.branch-2.0.003.patch, > HBASE-21935.branch-2.0.004.patch, HBASE-21935.branch-2.0.005.patch, > HBASE-21935.branch-2.0.006.patch, HBASE-21935.branch-2.0.007.patch, > HBASE-21935.branch-2.0.008.patch, HBASE-21935.branch-2.0.009.patch, > HBASE-21935.branch-2.0.010.patch, HBASE-21935.branch-2.1.001.patch, > HBASE-21935.branch-2.1.002.patch, HBASE-21935.branch-2.1.003.patch, > HBASE-21935.branch-2.1.004.patch, HBASE-21935.branch-2.1.005.patch, > HBASE-21935.branch-2.1.006.patch, HBASE-21935.branch-2.1.007.patch > > > The spark/dev/create-release is more comprehensive than our hokey make_rc.sh > script. It codifies the bulk of the RM process from tagging, version-setting, > building, signing, and pushing. It does it in a container so environment is > same each time. It has a bunch of spark-specifics as is -- naturally -- but > should be possible to pull it around to suit hbase RM'ing. It'd save a bunch > of time and would allow us to get to a place where RM'ing is canned, > evolvable, and consistent. > I've been hacking on the tooling before the filing of this JIRA and was > polluting branch-2.0 w/ tagging and reverts. Let me make a branch named for > this JIRA to play with (There is a dry-run flag but it too needs work...). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21963) Add a script for building and verifying release candidate
[
https://issues.apache.org/jira/browse/HBASE-21963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794504#comment-16794504
]
Peter Somogyi commented on HBASE-21963:
---
[~taklwu], sorry for the late reply.
I tested the patch with 2.2.0RC0 and it went well, 2 nits:
- The script is not executable by default, it has 644 permission.
- In case the output directory does not exist the script fails with an error
like this: {{./hbase-vote.sh: line 130:
/home/psomogyi/asdasdasd/2.2.0RC0_rat_test: No such file or directory}}
This is also the case when you don't specify a full path, e.g. ~/tmp also
failed on my machine.
> Add a script for building and verifying release candidate
> -
>
> Key: HBASE-21963
> URL: https://issues.apache.org/jira/browse/HBASE-21963
> Project: HBase
> Issue Type: Test
> Components: release, scripts
>Affects Versions: 3.0.0, 2.1.3
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
> Attachments: HBASE-21963.master.001.patch,
> HBASE-21963.master.002.patch, HBASE-21963.master.003.patch,
> HBASE-21963.master.004.patch, HBASE-21963.master.005.patch
>
>
> During the release vote for HBase 2.1.3RC1, a driver/helper script was
> mentioned and can potentially help contributors prepare to vote for a release
> candidate. As recommended, we decided to move toward this tool to under
> {{dev-support/}}
> Here the driver script provides the following automation:
> 1. Import and check publisher key(s)
> 2. Checksum of sources and binaries
> 3. Signature of sources and binaries
> 4. Rat check
> 5. Built from source
> 6. Verify unit tests
> {code}
> # example usage
> $ bash dev-support/hbase-vote.sh -s
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.5.0RC2/
> $ bash dev-support/hbase-vote.sh -h
> hbase-vote. A script for standard vote which verifies the following items
> 1. Checksum of sources and binaries
> 2. Signature of sources and binaries
> 3. Rat check
> 4. Built from source
> 5. Unit tests
> Usage: hbase-vote.sh -s | --source [-k | --key ] [-f |
> --keys-file-url ]
>hbase-vote.sh -h | --help
> -h | --help Show this screen.
> -s | --source '' A URL pointing to the release candidate
> sources and binaries
> e.g.
> https://dist.apache.org/repos/dist/dev/hbase/hbase-RC0/
> -k | --key '' A signature of the public key, e.g. 9AD2AE49
> -f | --keys-file-url '' the URL of the key file, default is
> http://www.apache.org/dist/hbase/KEYS
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21512) Introduce an AsyncClusterConnection and replace the usage of ClusterConnection
[
https://issues.apache.org/jira/browse/HBASE-21512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794497#comment-16794497
]
Hudson commented on HBASE-21512:
Results for branch HBASE-21512
[build #138 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/138/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/138//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/138//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/138//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Introduce an AsyncClusterConnection and replace the usage of ClusterConnection
> --
>
> Key: HBASE-21512
> URL: https://issues.apache.org/jira/browse/HBASE-21512
> Project: HBase
> Issue Type: Umbrella
>Reporter: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
>
> At least for the RSProcedureDispatcher, with CompletableFuture we do not need
> to set a delay and use a thread pool any more, which could reduce the
> resource usage and also the latency.
> Once this is done, I think we can remove the ClusterConnection completely,
> and start to rewrite the old sync client based on the async client, which
> could reduce the code base a lot for our client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22041) The crashed node exists in onlineServer forever, and if it holds the meta data, master will start up hang.
[
https://issues.apache.org/jira/browse/HBASE-22041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lujie updated HBASE-22041:
--
Summary: The crashed node exists in onlineServer forever, and if it holds
the meta data, master will start up hang. (was: Master stuck in startup and
print "FailedServerException" forever)
> The crashed node exists in onlineServer forever, and if it holds the meta
> data, master will start up hang.
> --
>
> Key: HBASE-22041
> URL: https://issues.apache.org/jira/browse/HBASE-22041
> Project: HBase
> Issue Type: Bug
>Reporter: lujie
>Priority: Critical
> Attachments: bug.zip, normal.zip
>
>
> while master fresh boot, we crash (kill- 9) the RS who hold meta. we find
> that the master startup fails and print thounds of logs like:
> {code:java}
> 2019-03-13 01:09:54,896 WARN [RSProcedureDispatcher-pool4-t1]
> procedure.RSProcedureDispatcher: request to server
> hadoop14,16020,1552410583724 failed due to java.net.ConnectException: Call to
> hadoop14/172.16.1.131:16020 failed on connection exception:
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException:
> syscall:getsockopt(..) failed: Connection refused:
> hadoop14/172.16.1.131:16020, try=0, retrying...
> 2019-03-13 01:09:55,004 WARN [RSProcedureDispatcher-pool4-t2]
> procedure.RSProcedureDispatcher: request to server
> hadoop14,16020,1552410583724 failed due to
> org.apache.hadoop.hbase.ipc.FailedServerException: Call to
> hadoop14/172.16.1.131:16020 failed on local exception:
> org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the
> failed servers list: hadoop14/172.16.1.131:16020, try=1, retrying...
> 2019-03-13 01:09:55,114 WARN [RSProcedureDispatcher-pool4-t3]
> procedure.RSProcedureDispatcher: request to server
> hadoop14,16020,1552410583724 failed due to
> org.apache.hadoop.hbase.ipc.FailedServerException: Call to
> hadoop14/172.16.1.131:16020 failed on local exception:
> org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the
> failed servers list: hadoop14/172.16.1.131:16020, try=2, retrying...
> 2019-03-13 01:09:55,219 WARN [RSProcedureDispatcher-pool4-t4]
> procedure.RSProcedureDispatcher: request to server
> hadoop14,16020,1552410583724 failed due to
> org.apache.hadoop.hbase.ipc.FailedServerException: Call to
> hadoop14/172.16.1.131:16020 failed on local exception:
> org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the
> failed servers list: hadoop14/172.16.1.131:16020, try=3, retrying...
> 2019-03-13 01:09:55,324 WARN [RSProcedureDispatcher-pool4-t5]
> procedure.RSProcedureDispatcher: request to server
> hadoop14,16020,1552410583724 failed due to
> org.apache.hadoop.hbase.ipc.FailedServerException: Call to
> hadoop14/172.16.1.131:16020 failed on local exception:
> org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the
> failed servers list: hadoop14/172.16.1.131:16020, try=4, retrying...
> 2019-03-13 01:09:55,428 WARN [RSProcedureDispatcher-pool4-t6]
> procedure.RSProcedureDispatcher: request to server
> hadoop14,16020,1552410583724 failed due to
> org.apache.hadoop.hbase.ipc.FailedServerException: Call to
> hadoop14/172.16.1.131:16020 failed on local exception:
> org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the
> failed servers list: hadoop14/172.16.1.131:16020, try=5, retrying...
> 2019-03-13 01:09:55,533 WARN [RSProcedureDispatcher-pool4-t7]
> procedure.RSProcedureDispatcher: request to server
> hadoop14,16020,1552410583724 failed due to
> org.apache.hadoop.hbase.ipc.FailedServerException: Call to
> hadoop14/172.16.1.131:16020 failed on local exception:
> org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the
> failed servers list: hadoop14/172.16.1.131:16020, try=6, retrying...
> 2019-03-13 01:09:55,638 WARN [RSProcedureDispatcher-pool4-t8]
> procedure.RSProcedureDispatcher: request to server
> hadoop14,16020,1552410583724 failed due to
> org.apache.hadoop.hbase.ipc.FailedServerException: Call to
> hadoop14/172.16.1.131:16020 failed on local exception:
> org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the
> failed servers list: hadoop14/172.16.1.131:16020, try=7, retrying...
> 2019-03-13 01:09:55,755 WARN [RSProcedureDispatcher-pool4-t9]
> procedure.RSProcedureDispatcher: request to server
> hadoop14,16020,1552410583724 failed due to
> org.apache.hadoop.hbase.ipc.FailedServerException: Call to
> hadoop14/172.16.1.131:16020 failed on local exception:
> org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the
> failed servers list: hadoop14/172.16.1.131:16020, try=8, retrying...
> {code}
--
This message was sent by Atlassian
[jira] [Commented] (HBASE-22041) The crashed node exists in onlineServer forever, and if it holds the meta data, master will start up hang.
[
https://issues.apache.org/jira/browse/HBASE-22041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794481#comment-16794481
]
lujie commented on HBASE-22041:
---
[~Apache9] and [~allan163]
Could please see this issue and give some suggestion to fix it?
My personal option is give a threshold for numberOfAttemptsSoFar(like the TODO
in above code).
Thanks.
> The crashed node exists in onlineServer forever, and if it holds the meta
> data, master will start up hang.
> --
>
> Key: HBASE-22041
> URL: https://issues.apache.org/jira/browse/HBASE-22041
> Project: HBase
> Issue Type: Bug
>Reporter: lujie
>Priority: Critical
> Attachments: bug.zip, normal.zip
>
>
> while master fresh boot, we crash (kill- 9) the RS who hold meta. we find
> that the master startup fails and print thounds of logs like:
> {code:java}
> 2019-03-13 01:09:54,896 WARN [RSProcedureDispatcher-pool4-t1]
> procedure.RSProcedureDispatcher: request to server
> hadoop14,16020,1552410583724 failed due to java.net.ConnectException: Call to
> hadoop14/172.16.1.131:16020 failed on connection exception:
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException:
> syscall:getsockopt(..) failed: Connection refused:
> hadoop14/172.16.1.131:16020, try=0, retrying...
> 2019-03-13 01:09:55,004 WARN [RSProcedureDispatcher-pool4-t2]
> procedure.RSProcedureDispatcher: request to server
> hadoop14,16020,1552410583724 failed due to
> org.apache.hadoop.hbase.ipc.FailedServerException: Call to
> hadoop14/172.16.1.131:16020 failed on local exception:
> org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the
> failed servers list: hadoop14/172.16.1.131:16020, try=1, retrying...
> 2019-03-13 01:09:55,114 WARN [RSProcedureDispatcher-pool4-t3]
> procedure.RSProcedureDispatcher: request to server
> hadoop14,16020,1552410583724 failed due to
> org.apache.hadoop.hbase.ipc.FailedServerException: Call to
> hadoop14/172.16.1.131:16020 failed on local exception:
> org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the
> failed servers list: hadoop14/172.16.1.131:16020, try=2, retrying...
> 2019-03-13 01:09:55,219 WARN [RSProcedureDispatcher-pool4-t4]
> procedure.RSProcedureDispatcher: request to server
> hadoop14,16020,1552410583724 failed due to
> org.apache.hadoop.hbase.ipc.FailedServerException: Call to
> hadoop14/172.16.1.131:16020 failed on local exception:
> org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the
> failed servers list: hadoop14/172.16.1.131:16020, try=3, retrying...
> 2019-03-13 01:09:55,324 WARN [RSProcedureDispatcher-pool4-t5]
> procedure.RSProcedureDispatcher: request to server
> hadoop14,16020,1552410583724 failed due to
> org.apache.hadoop.hbase.ipc.FailedServerException: Call to
> hadoop14/172.16.1.131:16020 failed on local exception:
> org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the
> failed servers list: hadoop14/172.16.1.131:16020, try=4, retrying...
> 2019-03-13 01:09:55,428 WARN [RSProcedureDispatcher-pool4-t6]
> procedure.RSProcedureDispatcher: request to server
> hadoop14,16020,1552410583724 failed due to
> org.apache.hadoop.hbase.ipc.FailedServerException: Call to
> hadoop14/172.16.1.131:16020 failed on local exception:
> org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the
> failed servers list: hadoop14/172.16.1.131:16020, try=5, retrying...
> 2019-03-13 01:09:55,533 WARN [RSProcedureDispatcher-pool4-t7]
> procedure.RSProcedureDispatcher: request to server
> hadoop14,16020,1552410583724 failed due to
> org.apache.hadoop.hbase.ipc.FailedServerException: Call to
> hadoop14/172.16.1.131:16020 failed on local exception:
> org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the
> failed servers list: hadoop14/172.16.1.131:16020, try=6, retrying...
> 2019-03-13 01:09:55,638 WARN [RSProcedureDispatcher-pool4-t8]
> procedure.RSProcedureDispatcher: request to server
> hadoop14,16020,1552410583724 failed due to
> org.apache.hadoop.hbase.ipc.FailedServerException: Call to
> hadoop14/172.16.1.131:16020 failed on local exception:
> org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the
> failed servers list: hadoop14/172.16.1.131:16020, try=7, retrying...
> 2019-03-13 01:09:55,755 WARN [RSProcedureDispatcher-pool4-t9]
> procedure.RSProcedureDispatcher: request to server
> hadoop14,16020,1552410583724 failed due to
> org.apache.hadoop.hbase.ipc.FailedServerException: Call to
> hadoop14/172.16.1.131:16020 failed on local exception:
> org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the
> failed servers list: hadoop14/172.16.1.131:16020, try=8, retrying...
> {code}
[jira] [Commented] (HBASE-21879) Read HFile's block to ByteBuffer directly instead of to byte for reducing young gc purpose
[
https://issues.apache.org/jira/browse/HBASE-21879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794480#comment-16794480
]
Hudson commented on HBASE-21879:
Results for branch HBASE-21879
[build #29 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/29/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/29//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/29//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/29//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Read HFile's block to ByteBuffer directly instead of to byte for reducing
> young gc purpose
> --
>
> Key: HBASE-21879
> URL: https://issues.apache.org/jira/browse/HBASE-21879
> Project: HBase
> Issue Type: Improvement
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-21879.v1.patch, HBASE-21879.v1.patch,
> QPS-latencies-before-HBASE-21879.png, gc-data-before-HBASE-21879.png
>
>
> In HFileBlock#readBlockDataInternal, we have the following:
> {code}
> @VisibleForTesting
> protected HFileBlock readBlockDataInternal(FSDataInputStream is, long offset,
> long onDiskSizeWithHeaderL, boolean pread, boolean verifyChecksum,
> boolean updateMetrics)
> throws IOException {
> // .
> // TODO: Make this ByteBuffer-based. Will make it easier to go to HDFS with
> BBPool (offheap).
> byte [] onDiskBlock = new byte[onDiskSizeWithHeader + hdrSize];
> int nextBlockOnDiskSize = readAtOffset(is, onDiskBlock, preReadHeaderSize,
> onDiskSizeWithHeader - preReadHeaderSize, true, offset +
> preReadHeaderSize, pread);
> if (headerBuf != null) {
> // ...
> }
> // ...
> }
> {code}
> In the read path, we still read the block from hfile to on-heap byte[], then
> copy the on-heap byte[] to offheap bucket cache asynchronously, and in my
> 100% get performance test, I also observed some frequent young gc, The
> largest memory footprint in the young gen should be the on-heap block byte[].
> In fact, we can read HFile's block to ByteBuffer directly instead of to
> byte[] for reducing young gc purpose. we did not implement this before,
> because no ByteBuffer reading interface in the older HDFS client, but 2.7+
> has supported this now, so we can fix this now. I think.
> Will provide an patch and some perf-comparison for this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22041) Master stuck in startup and print "FailedServerException" forever
[
https://issues.apache.org/jira/browse/HBASE-22041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790411#comment-16790411
]
lujie edited comment on HBASE-22041 at 3/17/19 1:20 PM:
We have four machine, HMaster is hadoop11, the RegionServers are
:hadoop12,hadoop13,hadoop14.
When we start master, who is on a busy machine. So the the
RegionServerTracker#refresh become slow. Before RegionServerTracker#refresh
detect the hadoop14 join the cluster, hadoop14 crash, but hadoop14 is aleardy
added to ServerManager#onlineSever(see below log#3). So even hadoop14 crashes,
it exist in onlineServer for ever.
{code:java}
1 master.ServerManager: Registering regionserver=hadoop14,16020,1552410583724
2 master.ServerManager: Registering regionserver=hadoop12,16020,1552410578454
3 master.ServerManager: Registering regionserver=hadoop13,16020,1552410566504
4 zookeeper.MetaTableLocator: Setting hbase:meta (replicaId=0) location in
ZooKeeper as hadoop14,16020,1552410583724
5 master.RegionServerTracker: RegionServer ephemeral node created, adding
[hadoop12,16020,1552410578454]
6 master.RegionServerTracker: RegionServer ephemeral node created, adding
[hadoop13,16020,1552410566504]
7 procedure.RSProcedureDispatcher: request to server
hadoop14,16020,1552410583724 failed due to java.net.ConnectException: Call to
hadoop14/172.16.1.131:16020 failed on connection exception:
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException:
syscall:getsockopt(..) failed: Connection refused:
hadoop14/172.16.1.131:16020, try=0, retrying...
{code}
This can cause the master hangs and print log forever, if hadoop14 hold the
meta table.
See
org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher.ExecuteProceduresRemoteCall.run
{code:java}
1 try {
2 sendRequest(getServerName(), request.build());
3 } catch (IOException e) {
4 e = unwrapException(e);
5 // TODO: In the future some operation may want to bail out early.
6 // TODO: How many times should we retry (use numberOfAttemptsSoFar)
7 if (!scheduleForRetry(e)) {
8 remoteCallFailed(procedureEnv, e);
9 }
10 }
{code}
master will sendReust to hadoop14 and fails, so it will call scheduleForRetry
to retry , and in scheduleForRetry , master will check whether hadoop14 is in
onlineServers , if is, retry, hence master will retry forever and print
thousands of logs like log #7.
was (Author: xiaoheipangzi):
Attaching the bug execution logs and normal execution log for comparison. The
RegionServer are :hadoop12,hadoop13,hadoop14
h4. 1 In the normal execution, we crash the hadoop13 who holds the meta data,
everything is ok, the log can be like:
{code:java}
1 master.ServerManager: Registering regionserver=hadoop12,16020,1552412058473
2 master.ServerManager: Registering regionserver=hadoop13,16020,1552412046289
3 master.ServerManager: Registering regionserver=hadoop14,16020,1552412063546
4 zookeeper.MetaTableLocator: Setting hbase:meta (replicaId=0) location in
ZooKeeper as hadoop13,16020,1552412046289
5 master.RegionServerTracker: RegionServer ephemeral node created, adding
[hadoop13,16020,1552412046289]
6 master.RegionServerTracker: RegionServer ephemeral node created, adding
[hadoop12,16020,1552412058473]
7 master.RegionServerTracker: RegionServer ephemeral node created, adding
[hadoop14,16020,1552412063546]
8 master.RegionServerTracker: RegionServer ephemeral node deleted, processing
expiration [hadoop13,16020,1552412046289]
9 master.ServerManager: Processing expiration of hadoop13,16020,1552412046289
on hadoop11,16000,1552412053502
{code}
log#1,2,3 show that hadoop12,13,14 are added to "onlineServers" (in
ServerManager).
log#8,9 shows that master detect hadoop13 crash and will remove it from the the
field "onlineServers" of ServerManager.
h4. 2 In the bug execution, we crash the hadoop14 and the
RegionServerTracker#refresh slow down(we inject sleep to simulate), the log
becomes:
{code:java}
1 master.ServerManager: Registering regionserver=hadoop14,16020,1552410583724
2 master.ServerManager: Registering regionserver=hadoop12,16020,1552410578454
3 master.ServerManager: Registering regionserver=hadoop13,16020,1552410566504
4 zookeeper.MetaTableLocator: Setting hbase:meta (replicaId=0) location in
ZooKeeper as hadoop14,16020,1552410583724
5 master.RegionServerTracker: RegionServer ephemeral node created, adding
[hadoop12,16020,1552410578454]
6 master.RegionServerTracker: RegionServer ephemeral node created, adding
[hadoop13,16020,1552410566504]
7 procedure.RSProcedureDispatcher: request to server
hadoop14,16020,1552410583724 failed due to java.net.ConnectException: Call to
hadoop14/172.16.1.131:16020 failed on connection exception:
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException:
syscall:getsockopt(..) failed: Connection refused:
hadoop14/172.16.1.131:16020,
[jira] [Commented] (HBASE-21964) unset Quota by Throttle Type
[
https://issues.apache.org/jira/browse/HBASE-21964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794477#comment-16794477
]
Hadoop QA commented on HBASE-21964:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
14s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 2m
35s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 1m
41s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 41s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} rubocop {color} | {color:green} 0m
4s{color} | {color:green} The patch generated 0 new + 69 unchanged - 20 fixed =
69 total (was 89) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 3s{color} | {color:orange} The patch generated 37 new + 124 unchanged - 15
fixed = 161 total (was 139) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 3m
18s{color} | {color:red} patch has 16 errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
53s{color} | {color:red} The patch causes 16 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m
50s{color} | {color:red} The patch causes 16 errors with Hadoop v3.0.0. {color}
|
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
28s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
14s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 48s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m
52s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21964 |
|
[jira] [Updated] (HBASE-21964) unset Quota by Throttle Type
[
https://issues.apache.org/jira/browse/HBASE-21964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yaojingyi updated HBASE-21964:
--
Attachment: HBASE-21964.master.005.patch
> unset Quota by Throttle Type
>
>
> Key: HBASE-21964
> URL: https://issues.apache.org/jira/browse/HBASE-21964
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 1.4.8
>Reporter: yaojingyi
>Assignee: yaojingyi
>Priority: Major
> Attachments: HBASE-21964.branch-1.4.001.patch,
> HBASE-21964.master.001.patch, HBASE-21964.master.002.patch,
> HBASE-21964.master.003.patch, HBASE-21964.master.004.patch,
> HBASE-21964.master.005.patch, unthrottleByType.patch
>
>
>
> {code:java}
> //first set_quota to USER=> 'u1'
> set_quota TYPE => THROTTLE, USER => 'u1', THROTTLE_TYPE => WRITE, LIMIT =>
> '1000req/sec'
> //then
> hbase(main):004:0> set_quota TYPE => THROTTLE, THROTTLE_TYPE => WRITE, USER
> => 'u1', LIMIT => NONE
> ERROR: Unexpected arguments: {"THROTTLE_TYPE"=>"WRITE"}
> // or try "THROTTLE_TYPE"=>"WRITE_NUMBER"
> hbase(main):012:0* set_quota TYPE => THROTTLE, THROTTLE_TYPE => WRITE_NUMBER,
> USER => 'u1', LIMIT => NONE
> NameError: uninitialized constant WRITE_NUMBER
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21895) Take more care on the error prone plugin and warnings
[ https://issues.apache.org/jira/browse/HBASE-21895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794414#comment-16794414 ] Duo Zhang commented on HBASE-21895: --- +1. Great that we could remove a module... And do not need to wait for my +1 if we change the patch again in the future. My second child was born so not always online these days... Thanks. > Take more care on the error prone plugin and warnings > - > > Key: HBASE-21895 > URL: https://issues.apache.org/jira/browse/HBASE-21895 > Project: HBase > Issue Type: Umbrella >Reporter: Duo Zhang >Priority: Major > Attachments: HBASE-21895.master.001.patch, > HBASE-21895.master.002.patch > > > As shown in HBASE-21890 , the error prone warnings are truly useful. > But now, the javac warnings section in the pre commit result is messed up, it > always reports unrelated warnings. Need to take a look at it. > And also, we should try out best to clean up the warnings. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
