[jira] [Commented] (HBASE-21657) PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.
[
https://issues.apache.org/jira/browse/HBASE-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743726#comment-16743726
]
Hudson commented on HBASE-21657:
Results for branch master
[build #722 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/722/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/722//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/722//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/722//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100%
> scan case.
>
>
> Key: HBASE-21657
> URL: https://issues.apache.org/jira/browse/HBASE-21657
> Project: HBase
> Issue Type: Bug
> Components: Performance
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21657.v1.patch, HBASE-21657.v2.patch,
> HBASE-21657.v3.patch, HBASE-21657.v3.patch, HBASE-21657.v4.patch,
> HBASE-21657.v5.patch, HBASE-21657.v5.patch, HBASE-21657.v5.patch,
> HBASE-21657.v6.patch, HBASE-21657.v7.patch,
> HBase1.4.9-ssd-1000-rows-flamegraph.svg,
> HBase1.4.9-ssd-1000-rows-qps-latency.png,
> HBase2.0.4-patch-v2-ssd-1000-rows-qps-and-latency.png,
> HBase2.0.4-patch-v2-ssd-1000-rows.svg,
> HBase2.0.4-patch-v3-ssd-1000-rows-flamegraph.svg,
> HBase2.0.4-patch-v3-ssd-1000-rows-qps-and-latency.png,
> HBase2.0.4-patch-v4-ssd-1000-rows-flamegraph.svg,
> HBase2.0.4-ssd-1000-rows-flamegraph.svg,
> HBase2.0.4-ssd-1000-rows-qps-latency.png, HBase2.0.4-with-patch.v2.png,
> HBase2.0.4-without-patch-v2.png, debug-the-ByteBufferKeyValue.diff,
> hbase2.0.4-ssd-scan-traces.2.svg, hbase2.0.4-ssd-scan-traces.svg,
> hbase20-ssd-100-scan-traces.svg, image-2019-01-07-19-03-37-930.png,
> image-2019-01-07-19-03-55-577.png, overview-statstics-1.png, run.log
>
>
> We are evaluating the performance of branch-2, and find that the throughput
> of scan in SSD cluster is almost the same as HDD cluster. so I made a
> FlameGraph on RS, and found that the
> PrivateCellUtil#estimatedSerializedSizeOf cost about 29% cpu, Obviously, it
> has been the bottleneck in 100% scan case.
> See the [^hbase20-ssd-100-scan-traces.svg]
> BTW, in our XiaoMi branch, we introduce a
> HRegion#updateReadRequestsByCapacityUnitPerSecond to sum up the size of cells
> (for metric monitor), so it seems the performance loss was amplified.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21639) maxHeapUsage value not read properly from config during EntryBuffers initialization
[
https://issues.apache.org/jira/browse/HBASE-21639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743723#comment-16743723
]
Hudson commented on HBASE-21639:
Results for branch master
[build #722 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/722/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/722//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/722//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/722//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> maxHeapUsage value not read properly from config during EntryBuffers
> initialization
> ---
>
> Key: HBASE-21639
> URL: https://issues.apache.org/jira/browse/HBASE-21639
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1, 2.1.0
>Reporter: Bo Cui
>Assignee: Pankaj Kumar
>Priority: Minor
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21639.001.patch
>
>
>
> {code:java|title=WALSplitter.java|borderStyle=solid}
> entryBuffers = new EntryBuffers(controller,
> this.conf.getInt("hbase.regionserver.hlog.splitlog.buffersize", 128 * 1024 *
> 1024),
> splitWriterCreationBounded);
> {code}
> In above case, EntryBuffers can't support maxHeapUsage in GB size.
> The parameter type of the new EntryBuffers() is long, but the conf max value
> is INT.MAX.
> this is wrong？it should be getLong？
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21647) Add status track for splitting WAL tasks
[
https://issues.apache.org/jira/browse/HBASE-21647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743724#comment-16743724
]
Hudson commented on HBASE-21647:
Results for branch master
[build #722 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/722/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/722//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/722//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/722//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add status track for splitting WAL tasks
>
>
> Key: HBASE-21647
> URL: https://issues.apache.org/jira/browse/HBASE-21647
> Project: HBase
> Issue Type: Sub-task
> Components: Operability
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21647.master.001.patch,
> HBASE-21647.master.002.patch, HBASE-21647.master.003.patch,
> HBASE-21647.master.003.patch, Screenshot from 2019-01-03 15-31-26.png,
> Screenshot from 2019-01-03 15-31-36.png
>
>
> add status track to help operator check the status of splitting WAL
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21595) Print thread's information and stack traces when RS is aborting forcibly
[
https://issues.apache.org/jira/browse/HBASE-21595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743722#comment-16743722
]
Hudson commented on HBASE-21595:
Results for branch master
[build #722 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/722/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/722//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/722//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/722//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Print thread's information and stack traces when RS is aborting forcibly
>
>
> Key: HBASE-21595
> URL: https://issues.apache.org/jira/browse/HBASE-21595
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Minor
> Labels: operability
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21595.patch
>
>
> After HBASE-21325 RS terminate forcibly on abort timeout.
> We should print the thread info before terminating, will be useful to analyze
> the RS abort timeout problem.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21731) Do not need to use ClusterConnection in IntegrationTestBigLinkedListWithVisibility
[
https://issues.apache.org/jira/browse/HBASE-21731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743721#comment-16743721
]
Hadoop QA commented on HBASE-21731:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
9s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
12s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 32s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
45s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21731 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12955047/HBASE-21731-v1.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux b18d64bb800d 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 5ca1b64be5 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15595/testReport/ |
| Max. process+thread count | 403 (vs. ulimit of 1) |
| modules | C: hbase-it U: hbase-it |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15595/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Do not need to use ClusterConnection in
>
[jira] [Updated] (HBASE-21731) Do not need to use ClusterConnection in IntegrationTestBigLinkedListWithVisibility
[ https://issues.apache.org/jira/browse/HBASE-21731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21731: -- Attachment: HBASE-21731-v1.patch > Do not need to use ClusterConnection in > IntegrationTestBigLinkedListWithVisibility > -- > > Key: HBASE-21731 > URL: https://issues.apache.org/jira/browse/HBASE-21731 > Project: HBase > Issue Type: Task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-21731-v1.patch, HBASE-21731.patch > > > We could just use the RegionLocator instead of > ClusterConnection.relocateRegion -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21731) Do not need to use ClusterConnection in IntegrationTestBigLinkedListWithVisibility
[
https://issues.apache.org/jira/browse/HBASE-21731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743700#comment-16743700
]
Hadoop QA commented on HBASE-21731:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
8s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} hbase-it: The patch generated 1 new + 6 unchanged - 0
fixed = 7 total (was 6) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
9s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 41s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
46s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 55s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21731 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12955043/HBASE-21731.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 2b4f51aceadd 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 5ca1b64be5 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/15594/artifact/patchprocess/diff-checkstyle-hbase-it.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15594/testReport/ |
| Max. process+thread count | 400 (vs. ulimit of 1) |
| modules | C: hbase-it U: hbase-it |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15594/console |
|
[jira] [Commented] (HBASE-21595) Print thread's information and stack traces when RS is aborting forcibly
[ https://issues.apache.org/jira/browse/HBASE-21595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743694#comment-16743694 ] Guanghao Zhang commented on HBASE-21595: [~pankaj2461] I am not working on it. Thanks for take it. :) > Print thread's information and stack traces when RS is aborting forcibly > > > Key: HBASE-21595 > URL: https://issues.apache.org/jira/browse/HBASE-21595 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 3.0.0, 2.2.0 >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Minor > Labels: operability > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21595.patch > > > After HBASE-21325 RS terminate forcibly on abort timeout. > We should print the thread info before terminating, will be useful to analyze > the RS abort timeout problem. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21675) Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a lot of time) to branch-1
[
https://issues.apache.org/jira/browse/HBASE-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743685#comment-16743685
]
Hadoop QA commented on HBASE-21675:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 21m
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
6s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} branch-1 passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
34s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
24s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} branch-1 passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} the patch passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} the patch passed with JDK v1.7.0_201 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
34s{color} | {color:red} hbase-server: The patch generated 4 new + 20 unchanged
- 2 fixed = 24 total (was 22) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
25s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
2m 15s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} the patch passed with JDK v1.7.0_201 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}148m 47s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
22s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}193m 8s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.coprocessor.TestMetaTableMetrics |
| | hadoop.hbase.mapreduce.TestLoadIncrementalHFiles |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:61288f8 |
| JIRA Issue | HBASE-21675 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12955031/HBASE-21675.branch-1.v3.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti
[jira] [Commented] (HBASE-21710) Add quota related methods to the Admin interface
[
https://issues.apache.org/jira/browse/HBASE-21710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743689#comment-16743689
]
Hadoop QA commented on HBASE-21710:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
35s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
33s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
10s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} hbase-client: The patch generated 2 new + 234
unchanged - 9 fixed = 236 total (was 243) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} hbase-server: The patch generated 0 new + 8
unchanged - 2 fixed = 8 total (was 10) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} The patch passed checkstyle in hbase-thrift {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch passed checkstyle in hbase-shell {color} |
| {color:green}+1{color} | {color:green} rubocop {color} | {color:green} 0m
7s{color} | {color:green} There were no new rubocop issues. {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 3s{color} | {color:orange} The patch generated 1 new + 125 unchanged - 3
fixed = 126 total (was 128) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
46s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 58s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
10s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}156m 16s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
41s{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} |
[jira] [Commented] (HBASE-19695) Handle disabled table for async client
[ https://issues.apache.org/jira/browse/HBASE-19695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743681#comment-16743681 ] Duo Zhang commented on HBASE-19695: --- Ping [~tianjingyun]. PTAL. Thanks. > Handle disabled table for async client > -- > > Key: HBASE-19695 > URL: https://issues.apache.org/jira/browse/HBASE-19695 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5 > > Attachments: HBASE-19695-v1.patch, HBASE-19695-v2.patch, > HBASE-19695.master.001.patch, HBASE-19695.master.002.patch, HBASE-19695.patch > > > Now for async client we will not check if a table is disabled when retrying, > so we will retry until the time or count limit is reached, and will not throw > a TableNotEnabledException. > We should have the same behavior as sync client, but the implementation > should be carefully designed. For sync client, we will also check if a table > is disabled if it is a retry, no matter what the exception is. This will > double the pressure on meta table. We should try our best to eliminate > unnecessary access to meta table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21731) Do not need to use ClusterConnection in IntegrationTestBigLinkedListWithVisibility
[ https://issues.apache.org/jira/browse/HBASE-21731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21731: -- Attachment: HBASE-21731.patch > Do not need to use ClusterConnection in > IntegrationTestBigLinkedListWithVisibility > -- > > Key: HBASE-21731 > URL: https://issues.apache.org/jira/browse/HBASE-21731 > Project: HBase > Issue Type: Task >Reporter: Duo Zhang >Priority: Major > Attachments: HBASE-21731.patch > > > We could just use the RegionLocator instead of > ClusterConnection.relocateRegion -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21731) Do not need to use ClusterConnection in IntegrationTestBigLinkedListWithVisibility
[ https://issues.apache.org/jira/browse/HBASE-21731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21731: -- Assignee: Duo Zhang Status: Patch Available (was: Open) > Do not need to use ClusterConnection in > IntegrationTestBigLinkedListWithVisibility > -- > > Key: HBASE-21731 > URL: https://issues.apache.org/jira/browse/HBASE-21731 > Project: HBase > Issue Type: Task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-21731.patch > > > We could just use the RegionLocator instead of > ClusterConnection.relocateRegion -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21685) Change repository urls to Gitbox
[ https://issues.apache.org/jira/browse/HBASE-21685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743677#comment-16743677 ] Duo Zhang commented on HBASE-21685: --- It seems that the hadoop project has already started to use PR? I've received lots of emails which is sent to had...@noreply.github.com. > Change repository urls to Gitbox > > > Key: HBASE-21685 > URL: https://issues.apache.org/jira/browse/HBASE-21685 > Project: HBase > Issue Type: Task >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Critical > Fix For: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.1.3, 2.0.5, 1.3.4, 1.2.11 > > Attachments: HBASE-21685.master.001.patch, > HBASE-21685.master.001.patch > > > Moving to Gitbox is approaching and references to git-wip-us need to be > changed to gitbox. > Some of the Jenkins jobs are referring to git-wip-us which if going to be > locked after the migration. We could move them to github so the build flow > will remain intact. > Previous discussion on dev@: > [https://lists.apache.org/thread.html/3496568d6cc002f74f5c3bcce46ed44b7ee9e90d7d53af2c65b6f785@%3Cdev.hbase.apache.org%3E] > After this notify INFRA to make the change -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21711) Remove references to git.apache.org/hbase.git
[ https://issues.apache.org/jira/browse/HBASE-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743676#comment-16743676 ] Duo Zhang commented on HBASE-21711: --- +1. > Remove references to git.apache.org/hbase.git > - > > Key: HBASE-21711 > URL: https://issues.apache.org/jira/browse/HBASE-21711 > Project: HBase > Issue Type: Sub-task >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Critical > Attachments: HBASE-21711.branch-1.001.patch, > HBASE-21711.master.001.patch > > > With the GitBox migration not only git-wip-us was removed but also > git.apache.org/hbase.git is not available anymore. (INFRA-17640) > We need to remove all references to this url. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21657) PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.
[
https://issues.apache.org/jira/browse/HBASE-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743675#comment-16743675
]
Hudson commented on HBASE-21657:
Results for branch branch-2
[build #1614 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1614/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1614//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1614//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1614//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100%
> scan case.
>
>
> Key: HBASE-21657
> URL: https://issues.apache.org/jira/browse/HBASE-21657
> Project: HBase
> Issue Type: Bug
> Components: Performance
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21657.v1.patch, HBASE-21657.v2.patch,
> HBASE-21657.v3.patch, HBASE-21657.v3.patch, HBASE-21657.v4.patch,
> HBASE-21657.v5.patch, HBASE-21657.v5.patch, HBASE-21657.v5.patch,
> HBASE-21657.v6.patch, HBASE-21657.v7.patch,
> HBase1.4.9-ssd-1000-rows-flamegraph.svg,
> HBase1.4.9-ssd-1000-rows-qps-latency.png,
> HBase2.0.4-patch-v2-ssd-1000-rows-qps-and-latency.png,
> HBase2.0.4-patch-v2-ssd-1000-rows.svg,
> HBase2.0.4-patch-v3-ssd-1000-rows-flamegraph.svg,
> HBase2.0.4-patch-v3-ssd-1000-rows-qps-and-latency.png,
> HBase2.0.4-patch-v4-ssd-1000-rows-flamegraph.svg,
> HBase2.0.4-ssd-1000-rows-flamegraph.svg,
> HBase2.0.4-ssd-1000-rows-qps-latency.png, HBase2.0.4-with-patch.v2.png,
> HBase2.0.4-without-patch-v2.png, debug-the-ByteBufferKeyValue.diff,
> hbase2.0.4-ssd-scan-traces.2.svg, hbase2.0.4-ssd-scan-traces.svg,
> hbase20-ssd-100-scan-traces.svg, image-2019-01-07-19-03-37-930.png,
> image-2019-01-07-19-03-55-577.png, overview-statstics-1.png, run.log
>
>
> We are evaluating the performance of branch-2, and find that the throughput
> of scan in SSD cluster is almost the same as HDD cluster. so I made a
> FlameGraph on RS, and found that the
> PrivateCellUtil#estimatedSerializedSizeOf cost about 29% cpu, Obviously, it
> has been the bottleneck in 100% scan case.
> See the [^hbase20-ssd-100-scan-traces.svg]
> BTW, in our XiaoMi branch, we introduce a
> HRegion#updateReadRequestsByCapacityUnitPerSecond to sum up the size of cells
> (for metric monitor), so it seems the performance loss was amplified.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21647) Add status track for splitting WAL tasks
[
https://issues.apache.org/jira/browse/HBASE-21647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743674#comment-16743674
]
Hudson commented on HBASE-21647:
Results for branch branch-2
[build #1614 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1614/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1614//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1614//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1614//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add status track for splitting WAL tasks
>
>
> Key: HBASE-21647
> URL: https://issues.apache.org/jira/browse/HBASE-21647
> Project: HBase
> Issue Type: Sub-task
> Components: Operability
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21647.master.001.patch,
> HBASE-21647.master.002.patch, HBASE-21647.master.003.patch,
> HBASE-21647.master.003.patch, Screenshot from 2019-01-03 15-31-26.png,
> Screenshot from 2019-01-03 15-31-36.png
>
>
> add status track to help operator check the status of splitting WAL
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21595) Print thread's information and stack traces when RS is aborting forcibly
[
https://issues.apache.org/jira/browse/HBASE-21595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743672#comment-16743672
]
Hudson commented on HBASE-21595:
Results for branch branch-2
[build #1614 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1614/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1614//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1614//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1614//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Print thread's information and stack traces when RS is aborting forcibly
>
>
> Key: HBASE-21595
> URL: https://issues.apache.org/jira/browse/HBASE-21595
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Minor
> Labels: operability
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21595.patch
>
>
> After HBASE-21325 RS terminate forcibly on abort timeout.
> We should print the thread info before terminating, will be useful to analyze
> the RS abort timeout problem.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21639) maxHeapUsage value not read properly from config during EntryBuffers initialization
[
https://issues.apache.org/jira/browse/HBASE-21639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743673#comment-16743673
]
Hudson commented on HBASE-21639:
Results for branch branch-2
[build #1614 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1614/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1614//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1614//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1614//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> maxHeapUsage value not read properly from config during EntryBuffers
> initialization
> ---
>
> Key: HBASE-21639
> URL: https://issues.apache.org/jira/browse/HBASE-21639
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1, 2.1.0
>Reporter: Bo Cui
>Assignee: Pankaj Kumar
>Priority: Minor
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21639.001.patch
>
>
>
> {code:java|title=WALSplitter.java|borderStyle=solid}
> entryBuffers = new EntryBuffers(controller,
> this.conf.getInt("hbase.regionserver.hlog.splitlog.buffersize", 128 * 1024 *
> 1024),
> splitWriterCreationBounded);
> {code}
> In above case, EntryBuffers can't support maxHeapUsage in GB size.
> The parameter type of the new EntryBuffers() is long, but the conf max value
> is INT.MAX.
> this is wrong？it should be getLong？
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21672) Allow skipping HDFS block distribution computation
[
https://issues.apache.org/jira/browse/HBASE-21672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743646#comment-16743646
]
Nihal Jain commented on HBASE-21672:
bq. Can we do that here?
Sounds good. I will soon submit a patch with the suggested change.
> Allow skipping HDFS block distribution computation
> --
>
> Key: HBASE-21672
> URL: https://issues.apache.org/jira/browse/HBASE-21672
> Project: HBase
> Issue Type: Improvement
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Labels: S3
>
> We should have a configuration to skip HDFS block distribution calculation in
> HBase. For example on file systems that do not surface locality such as S3,
> calculating block distribution would not be any useful.
> Currentlly, we do not have a way to skip hdfs block distribution computation.
> For this, we can provide a new configuration key, say
> {{hbase.block.distribution.skip.computation}} (which would be {{false}} by
> default).
> Users using filesystems such as s3 may choose to make this {{true}}, thus
> skipping block distribution computation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21672) Allow skipping HDFS block distribution computation
[
https://issues.apache.org/jira/browse/HBASE-21672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743646#comment-16743646
]
Nihal Jain edited comment on HBASE-21672 at 1/16/19 5:50 AM:
-
bq. Ideally we can have one change that automatically disables locality
calculation after determining what filesystem implementation is in use. Can we
do that here?
Sounds good. I will soon submit a patch with the suggested change.
was (Author: nihaljain.cs):
bq. Can we do that here?
Sounds good. I will soon submit a patch with the suggested change.
> Allow skipping HDFS block distribution computation
> --
>
> Key: HBASE-21672
> URL: https://issues.apache.org/jira/browse/HBASE-21672
> Project: HBase
> Issue Type: Improvement
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Labels: S3
>
> We should have a configuration to skip HDFS block distribution calculation in
> HBase. For example on file systems that do not surface locality such as S3,
> calculating block distribution would not be any useful.
> Currentlly, we do not have a way to skip hdfs block distribution computation.
> For this, we can provide a new configuration key, say
> {{hbase.block.distribution.skip.computation}} (which would be {{false}} by
> default).
> Users using filesystems such as s3 may choose to make this {{true}}, thus
> skipping block distribution computation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21639) maxHeapUsage value not read properly from config during EntryBuffers initialization
[
https://issues.apache.org/jira/browse/HBASE-21639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743635#comment-16743635
]
Hudson commented on HBASE-21639:
Results for branch branch-2.1
[build #774 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/774/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/774//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/774//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/774//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> maxHeapUsage value not read properly from config during EntryBuffers
> initialization
> ---
>
> Key: HBASE-21639
> URL: https://issues.apache.org/jira/browse/HBASE-21639
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1, 2.1.0
>Reporter: Bo Cui
>Assignee: Pankaj Kumar
>Priority: Minor
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21639.001.patch
>
>
>
> {code:java|title=WALSplitter.java|borderStyle=solid}
> entryBuffers = new EntryBuffers(controller,
> this.conf.getInt("hbase.regionserver.hlog.splitlog.buffersize", 128 * 1024 *
> 1024),
> splitWriterCreationBounded);
> {code}
> In above case, EntryBuffers can't support maxHeapUsage in GB size.
> The parameter type of the new EntryBuffers() is long, but the conf max value
> is INT.MAX.
> this is wrong？it should be getLong？
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19722) Meta query statistics metrics source
[
https://issues.apache.org/jira/browse/HBASE-19722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743637#comment-16743637
]
Hudson commented on HBASE-19722:
Results for branch branch-2.1
[build #774 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/774/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/774//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/774//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/774//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Meta query statistics metrics source
>
>
> Key: HBASE-19722
> URL: https://issues.apache.org/jira/browse/HBASE-19722
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors, meta, metrics, Operability
>Reporter: Andrew Purtell
>Assignee: Xu Cang
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 1.4.6, 2.2.0, 2.0.2, 2.1.3
>
> Attachments: HBASE-19722-branch-2.1.v1.patch,
> HBASE-19722.branch-1.v001.patch, HBASE-19722.branch-1.v002.patch,
> HBASE-19722.master.010.patch, HBASE-19722.master.011.patch,
> HBASE-19722.master.012.patch, HBASE-19722.master.013.patch,
> HBASE-19722.master.014.patch, HBASE-19722.master.015.patch,
> HBASE-19722.master.016.patch
>
>
> Implement a meta query statistics metrics source, created whenever a
> regionserver starts hosting meta, removed when meta hosting moves. Provide
> views on top tables by request counts, top meta rowkeys by request count, top
> clients making requests by their hostname.
> Can be implemented as a coprocessor.
>
>
>
>
> ===
> *Release Note* (WIP)
> *1. Usage:*
> Use this coprocessor by adding below section to hbase-site.xml
> {{}}
> {{ hbase.coprocessor.region.classes}}
> {{ org.apache.hadoop.hbase.coprocessor.MetaTableMetrics}}
> {{}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20917) MetaTableMetrics#stop references uninitialized requestsMap for non-meta region
[
https://issues.apache.org/jira/browse/HBASE-20917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743638#comment-16743638
]
Hudson commented on HBASE-20917:
Results for branch branch-2.1
[build #774 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/774/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/774//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/774//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/774//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> MetaTableMetrics#stop references uninitialized requestsMap for non-meta region
> --
>
> Key: HBASE-20917
> URL: https://issues.apache.org/jira/browse/HBASE-20917
> Project: HBase
> Issue Type: Bug
> Components: meta, metrics
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.4.6, 2.2.0, 2.0.4, 2.1.3
>
> Attachments: 20917.addendum, 20917.v1.txt, 20917.v2.txt
>
>
> I noticed the following in test output:
> {code}
> 2018-07-21 15:54:43,181 ERROR [RS_CLOSE_REGION-regionserver/172.17.5.4:0-1]
> executor.EventHandler(186): Caught throwable while processing event
> M_RS_CLOSE_REGION
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics.stop(MetaTableMetrics.java:329)
> at
> org.apache.hadoop.hbase.coprocessor.BaseEnvironment.shutdown(BaseEnvironment.java:91)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionEnvironment.shutdown(RegionCoprocessorHost.java:165)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.shutdown(CoprocessorHost.java:290)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$4.postEnvCall(RegionCoprocessorHost.java:559)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:622)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postClose(RegionCoprocessorHost.java:551)
> at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1678)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1484)
> at
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:104)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> {code}
> {{requestsMap}} is only initialized for the meta region.
> However, check for meta region is absent in the stop method:
> {code}
> public void stop(CoprocessorEnvironment e) throws IOException {
> // since meta region can move around, clear stale metrics when stop.
> for (String meterName : requestsMap.keySet()) {
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21590) Optimize trySkipToNextColumn in StoreScanner a bit
[
https://issues.apache.org/jira/browse/HBASE-21590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743636#comment-16743636
]
Hudson commented on HBASE-21590:
Results for branch branch-2.1
[build #774 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/774/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/774//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/774//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/774//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Optimize trySkipToNextColumn in StoreScanner a bit
> --
>
> Key: HBASE-21590
> URL: https://issues.apache.org/jira/browse/HBASE-21590
> Project: HBase
> Issue Type: Improvement
> Components: Performance, Scanners
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 2.2.0, 2.0.4, 1.4.10, 2.1.3
>
> Attachments: 21590-1.5.txt, HBASE-21590-master.txt,
> HBASE-21590.addendum.v1.patch
>
>
> See latest comment on HBASE-17958
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20209) Do Not Use Both Map containsKey and get Methods
[
https://issues.apache.org/jira/browse/HBASE-20209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743625#comment-16743625
]
Sean Busbey commented on HBASE-20209:
-
I'm going to push this in the morning MST. I see a +1 from Duo and no
actionable objection from Ted. If either of you feel otherwise please ping here.
> Do Not Use Both Map containsKey and get Methods
> ---
>
> Key: HBASE-20209
> URL: https://issues.apache.org/jira/browse/HBASE-20209
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HBASE-20209.1.patch, HBASE-20209.2.patch,
> HBASE-20209.3.patch
>
>
> {code:title=ReplicationSink.java}
> String tableName = table.getNameWithNamespaceInclAsString();
> if (bulkLoadHFileMap.containsKey(tableName)) {
> List>> familyHFilePathsList =
> bulkLoadHFileMap.get(tableName);
> boolean foundFamily = false;
> for (int i = 0; i < familyHFilePathsList.size(); i++) {
> Pair> familyHFilePathsPair =
> familyHFilePathsList.get(i);
> if (Bytes.equals(familyHFilePathsPair.getFirst(), family)) {
> // Found family already present, just add the path to the
> existing list
> familyHFilePathsPair.getSecond().add(pathToHfileFromNS);
> foundFamily = true;
> break;
> }
> }
> {code}
> I propose that this code does not use the Map methods _containsKey_ *and*
> _get_. Simply use the _get_ method once and check a _null_ return value to
> check for existence. Saves a trip to the Map data structure for each call.
> Also, use enhanced for loop.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21711) Remove references to git.apache.org/hbase.git
[ https://issues.apache.org/jira/browse/HBASE-21711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743629#comment-16743629 ] Sean Busbey commented on HBASE-21711: - +1 > Remove references to git.apache.org/hbase.git > - > > Key: HBASE-21711 > URL: https://issues.apache.org/jira/browse/HBASE-21711 > Project: HBase > Issue Type: Sub-task >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Critical > Attachments: HBASE-21711.branch-1.001.patch, > HBASE-21711.master.001.patch > > > With the GitBox migration not only git-wip-us was removed but also > git.apache.org/hbase.git is not available anymore. (INFRA-17640) > We need to remove all references to this url. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20209) Do Not Use Both Map containsKey and get Methods in Replication Sink
[
https://issues.apache.org/jira/browse/HBASE-20209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-20209:
Component/s: Replication
> Do Not Use Both Map containsKey and get Methods in Replication Sink
> ---
>
> Key: HBASE-20209
> URL: https://issues.apache.org/jira/browse/HBASE-20209
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 2.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HBASE-20209.1.patch, HBASE-20209.2.patch,
> HBASE-20209.3.patch
>
>
> {code:title=ReplicationSink.java}
> String tableName = table.getNameWithNamespaceInclAsString();
> if (bulkLoadHFileMap.containsKey(tableName)) {
> List>> familyHFilePathsList =
> bulkLoadHFileMap.get(tableName);
> boolean foundFamily = false;
> for (int i = 0; i < familyHFilePathsList.size(); i++) {
> Pair> familyHFilePathsPair =
> familyHFilePathsList.get(i);
> if (Bytes.equals(familyHFilePathsPair.getFirst(), family)) {
> // Found family already present, just add the path to the
> existing list
> familyHFilePathsPair.getSecond().add(pathToHfileFromNS);
> foundFamily = true;
> break;
> }
> }
> {code}
> I propose that this code does not use the Map methods _containsKey_ *and*
> _get_. Simply use the _get_ method once and check a _null_ return value to
> check for existence. Saves a trip to the Map data structure for each call.
> Also, use enhanced for loop.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20209) Do Not Use Both Map containsKey and get Methods in Replication Sink
[
https://issues.apache.org/jira/browse/HBASE-20209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743627#comment-16743627
]
Sean Busbey commented on HBASE-20209:
-
[~belugabehr] please give me a patch made with {{git format-patch}} or the git
authorship information you'd prefer be used for yourself.
> Do Not Use Both Map containsKey and get Methods in Replication Sink
> ---
>
> Key: HBASE-20209
> URL: https://issues.apache.org/jira/browse/HBASE-20209
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HBASE-20209.1.patch, HBASE-20209.2.patch,
> HBASE-20209.3.patch
>
>
> {code:title=ReplicationSink.java}
> String tableName = table.getNameWithNamespaceInclAsString();
> if (bulkLoadHFileMap.containsKey(tableName)) {
> List>> familyHFilePathsList =
> bulkLoadHFileMap.get(tableName);
> boolean foundFamily = false;
> for (int i = 0; i < familyHFilePathsList.size(); i++) {
> Pair> familyHFilePathsPair =
> familyHFilePathsList.get(i);
> if (Bytes.equals(familyHFilePathsPair.getFirst(), family)) {
> // Found family already present, just add the path to the
> existing list
> familyHFilePathsPair.getSecond().add(pathToHfileFromNS);
> foundFamily = true;
> break;
> }
> }
> {code}
> I propose that this code does not use the Map methods _containsKey_ *and*
> _get_. Simply use the _get_ method once and check a _null_ return value to
> check for existence. Saves a trip to the Map data structure for each call.
> Also, use enhanced for loop.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20209) Do Not Use Both Map containsKey and get Methods in Replication Sink
[
https://issues.apache.org/jira/browse/HBASE-20209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-20209:
Summary: Do Not Use Both Map containsKey and get Methods in Replication
Sink (was: Do Not Use Both Map containsKey and get Methods)
> Do Not Use Both Map containsKey and get Methods in Replication Sink
> ---
>
> Key: HBASE-20209
> URL: https://issues.apache.org/jira/browse/HBASE-20209
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HBASE-20209.1.patch, HBASE-20209.2.patch,
> HBASE-20209.3.patch
>
>
> {code:title=ReplicationSink.java}
> String tableName = table.getNameWithNamespaceInclAsString();
> if (bulkLoadHFileMap.containsKey(tableName)) {
> List>> familyHFilePathsList =
> bulkLoadHFileMap.get(tableName);
> boolean foundFamily = false;
> for (int i = 0; i < familyHFilePathsList.size(); i++) {
> Pair> familyHFilePathsPair =
> familyHFilePathsList.get(i);
> if (Bytes.equals(familyHFilePathsPair.getFirst(), family)) {
> // Found family already present, just add the path to the
> existing list
> familyHFilePathsPair.getSecond().add(pathToHfileFromNS);
> foundFamily = true;
> break;
> }
> }
> {code}
> I propose that this code does not use the Map methods _containsKey_ *and*
> _get_. Simply use the _get_ method once and check a _null_ return value to
> check for existence. Saves a trip to the Map data structure for each call.
> Also, use enhanced for loop.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21612) Add developer debug options in HBase Config for REST server
[
https://issues.apache.org/jira/browse/HBASE-21612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743614#comment-16743614
]
Pankaj Kumar commented on HBASE-21612:
--
[~stack] sir, pls review this.
> Add developer debug options in HBase Config for REST server
>
>
> Key: HBASE-21612
> URL: https://issues.apache.org/jira/browse/HBASE-21612
> Project: HBase
> Issue Type: Wish
> Components: REST, scripts
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-21612.patch
>
>
> Add developer debug options in HBase Config for REST server.
> Currently we have,
> {noformat}
> # export HBASE_MASTER_OPTS="$HBASE_MASTER_OPTS -Xdebug
> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8070"
> # export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -Xdebug
> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8071"
> # export HBASE_THRIFT_OPTS="$HBASE_THRIFT_OPTS -Xdebug
> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8072"
> # export HBASE_ZOOKEEPER_OPTS="$HBASE_ZOOKEEPER_OPTS -Xdebug
> -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8073"
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21402) Backport parent "HBASE-21325 Force to terminate regionserver when abort hang in somewhere"
[ https://issues.apache.org/jira/browse/HBASE-21402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743609#comment-16743609 ] Pankaj Kumar commented on HBASE-21402: -- [~zghaobac] are you working this backport Jira? > Backport parent "HBASE-21325 Force to terminate regionserver when abort hang > in somewhere" > -- > > Key: HBASE-21402 > URL: https://issues.apache.org/jira/browse/HBASE-21402 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.3, 2.1.2 >Reporter: stack >Priority: Major > > Backport parent. Looks like good idea for fixing long-standing problem. I can > do the backport [~zghaobac]. Just FYI. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21595) Print thread's information and stack traces when RS is aborting forcibly
[ https://issues.apache.org/jira/browse/HBASE-21595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743608#comment-16743608 ] Pankaj Kumar commented on HBASE-21595: -- Thank you [~stack] for commiting the patch. HBASE-21325 is not commited in 2.0/2.1 branch. Backport HBASE-21402 is still open, will take that if [~zghaobac] is not working. > Print thread's information and stack traces when RS is aborting forcibly > > > Key: HBASE-21595 > URL: https://issues.apache.org/jira/browse/HBASE-21595 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 3.0.0, 2.2.0 >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Minor > Labels: operability > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21595.patch > > > After HBASE-21325 RS terminate forcibly on abort timeout. > We should print the thread info before terminating, will be useful to analyze > the RS abort timeout problem. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21639) maxHeapUsage value not read properly from config during EntryBuffers initialization
[
https://issues.apache.org/jira/browse/HBASE-21639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743597#comment-16743597
]
Hudson commented on HBASE-21639:
Results for branch branch-2.0
[build #1258 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1258/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1258//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1258//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1258//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> maxHeapUsage value not read properly from config during EntryBuffers
> initialization
> ---
>
> Key: HBASE-21639
> URL: https://issues.apache.org/jira/browse/HBASE-21639
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1, 2.1.0
>Reporter: Bo Cui
>Assignee: Pankaj Kumar
>Priority: Minor
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21639.001.patch
>
>
>
> {code:java|title=WALSplitter.java|borderStyle=solid}
> entryBuffers = new EntryBuffers(controller,
> this.conf.getInt("hbase.regionserver.hlog.splitlog.buffersize", 128 * 1024 *
> 1024),
> splitWriterCreationBounded);
> {code}
> In above case, EntryBuffers can't support maxHeapUsage in GB size.
> The parameter type of the new EntryBuffers() is long, but the conf max value
> is INT.MAX.
> this is wrong？it should be getLong？
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21675) Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a lot of time) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21675: - Attachment: HBASE-21675.branch-1.v3.patch > Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a > lot of time) to branch-1 > > > Key: HBASE-21675 > URL: https://issues.apache.org/jira/browse/HBASE-21675 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Priority: Major > Attachments: HBASE-21675-branch-1-WIP.patch, > HBASE-21675.branch-1.v2.patch, HBASE-21675.branch-1.v3.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21675) Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a lot of time) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21675: --- Fix Version/s: 1.5.0 > Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a > lot of time) to branch-1 > > > Key: HBASE-21675 > URL: https://issues.apache.org/jira/browse/HBASE-21675 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Zheng Hu >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21675-branch-1-WIP.patch, > HBASE-21675.branch-1.v2.patch, HBASE-21675.branch-1.v3.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21675) Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a lot of time) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743583#comment-16743583 ] Andrew Purtell commented on HBASE-21675: Good, glad it was something simple. +1 for commit, assuming test passes. Thanks! > Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a > lot of time) to branch-1 > > > Key: HBASE-21675 > URL: https://issues.apache.org/jira/browse/HBASE-21675 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-21675-branch-1-WIP.patch, > HBASE-21675.branch-1.v2.patch, HBASE-21675.branch-1.v3.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-21675) Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a lot of time) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell reassigned HBASE-21675: -- Assignee: Zheng Hu > Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a > lot of time) to branch-1 > > > Key: HBASE-21675 > URL: https://issues.apache.org/jira/browse/HBASE-21675 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-21675-branch-1-WIP.patch, > HBASE-21675.branch-1.v2.patch, HBASE-21675.branch-1.v3.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21675) Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a lot of time) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21675: - Attachment: (was: HBASE-21675.v3.patch) > Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a > lot of time) to branch-1 > > > Key: HBASE-21675 > URL: https://issues.apache.org/jira/browse/HBASE-21675 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Priority: Major > Attachments: HBASE-21675-branch-1-WIP.patch, > HBASE-21675.branch-1.v2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21675) Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a lot of time) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21675: - Attachment: HBASE-21675.v3.patch > Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a > lot of time) to branch-1 > > > Key: HBASE-21675 > URL: https://issues.apache.org/jira/browse/HBASE-21675 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Priority: Major > Attachments: HBASE-21675-branch-1-WIP.patch, > HBASE-21675.branch-1.v2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21675) Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a lot of time) to branch-1
[
https://issues.apache.org/jira/browse/HBASE-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743580#comment-16743580
]
Hadoop QA commented on HBASE-21675:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} HBASE-21675 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.8.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-21675 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12955030/HBASE-21675.v3.patch |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15592/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a
> lot of time) to branch-1
>
>
> Key: HBASE-21675
> URL: https://issues.apache.org/jira/browse/HBASE-21675
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Purtell
>Priority: Major
> Attachments: HBASE-21675-branch-1-WIP.patch,
> HBASE-21675.branch-1.v2.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21675) Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a lot of time) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21675: - Status: Patch Available (was: Reopened) > Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a > lot of time) to branch-1 > > > Key: HBASE-21675 > URL: https://issues.apache.org/jira/browse/HBASE-21675 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Priority: Major > Attachments: HBASE-21675-branch-1-WIP.patch, > HBASE-21675.branch-1.v2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21675) Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a lot of time) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21675: - Attachment: HBASE-21675.branch-1.v2.patch > Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a > lot of time) to branch-1 > > > Key: HBASE-21675 > URL: https://issues.apache.org/jira/browse/HBASE-21675 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Priority: Major > Attachments: HBASE-21675-branch-1-WIP.patch, > HBASE-21675.branch-1.v2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21675) Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a lot of time) to branch-1
[
https://issues.apache.org/jira/browse/HBASE-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743576#comment-16743576
]
Zheng Hu commented on HBASE-21675:
--
Not a big problem. the runCopy skip the bulk load step before, fix this then
all UT pass:
{code}
@@ -258,7 +251,64 @@ public class TestCopyTable {
Configuration configuration = opts.getConfiguration();
args = opts.getRemainingArgs();
Job job = new CopyTable(configuration).createSubmittableJob(args);
-job.waitForCompletion(false);
-return job.isSuccessful();
+if (job != null) {
+ job.waitForCompletion(false);
+ return job.isSuccessful();
+} else {
+ return false;
+}
+ }
{code}
We can see that:
{code}
[INFO] ---
[INFO] T E S T S
[INFO] ---
[INFO] Running org.apache.hadoop.hbase.mapreduce.TestCopyTable
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 351.217
s - in org.apache.hadoop.hbase.mapreduce.TestCopyTable
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0
{code}
> Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a
> lot of time) to branch-1
>
>
> Key: HBASE-21675
> URL: https://issues.apache.org/jira/browse/HBASE-21675
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Purtell
>Priority: Major
> Attachments: HBASE-21675-branch-1-WIP.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21710) Add quota related methods to the Admin interface
[ https://issues.apache.org/jira/browse/HBASE-21710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21710: -- Attachment: HBASE-21710-v2.patch > Add quota related methods to the Admin interface > > > Key: HBASE-21710 > URL: https://issues.apache.org/jira/browse/HBASE-21710 > Project: HBase > Issue Type: Task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21710-v1.patch, HBASE-21710-v1.patch, > HBASE-21710-v2.patch, HBASE-21710.patch > > > So we can limit the ClusterConnection usage inside the sync client > implementation. Now we will use ClusterConnection in QuotaTableUtil, which > will be a pain when we want to replace the implement of the old sync client > with the new async client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21731) Do not need to use ClusterConnection in IntegrationTestBigLinkedListWithVisibility
Duo Zhang created HBASE-21731: - Summary: Do not need to use ClusterConnection in IntegrationTestBigLinkedListWithVisibility Key: HBASE-21731 URL: https://issues.apache.org/jira/browse/HBASE-21731 Project: HBase Issue Type: Task Reporter: Duo Zhang We could just use the RegionLocator instead of ClusterConnection.relocateRegion -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21710) Add quota related methods to the Admin interface
[ https://issues.apache.org/jira/browse/HBASE-21710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21710: -- Attachment: (was: HBASE-21710-v2.patch) > Add quota related methods to the Admin interface > > > Key: HBASE-21710 > URL: https://issues.apache.org/jira/browse/HBASE-21710 > Project: HBase > Issue Type: Task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21710-v1.patch, HBASE-21710-v1.patch, > HBASE-21710.patch > > > So we can limit the ClusterConnection usage inside the sync client > implementation. Now we will use ClusterConnection in QuotaTableUtil, which > will be a pain when we want to replace the implement of the old sync client > with the new async client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21710) Add quota related methods to the Admin interface
[ https://issues.apache.org/jira/browse/HBASE-21710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21710: -- Attachment: HBASE-21710-v2.patch > Add quota related methods to the Admin interface > > > Key: HBASE-21710 > URL: https://issues.apache.org/jira/browse/HBASE-21710 > Project: HBase > Issue Type: Task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21710-v1.patch, HBASE-21710-v1.patch, > HBASE-21710-v2.patch, HBASE-21710.patch > > > So we can limit the ClusterConnection usage inside the sync client > implementation. Now we will use ClusterConnection in QuotaTableUtil, which > will be a pain when we want to replace the implement of the old sync client > with the new async client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21730) Update hbase-book with the procedure based WAL splitting
Jingyun Tian created HBASE-21730: Summary: Update hbase-book with the procedure based WAL splitting Key: HBASE-21730 URL: https://issues.apache.org/jira/browse/HBASE-21730 Project: HBase Issue Type: Sub-task Reporter: Jingyun Tian -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21710) Add quota related methods to the Admin interface
[ https://issues.apache.org/jira/browse/HBASE-21710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743563#comment-16743563 ] Duo Zhang commented on HBASE-21710: --- Oh, there is a problem on the method definition in the AsyncAdmin. Let me upload a new patch first... > Add quota related methods to the Admin interface > > > Key: HBASE-21710 > URL: https://issues.apache.org/jira/browse/HBASE-21710 > Project: HBase > Issue Type: Task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21710-v1.patch, HBASE-21710-v1.patch, > HBASE-21710.patch > > > So we can limit the ClusterConnection usage inside the sync client > implementation. Now we will use ClusterConnection in QuotaTableUtil, which > will be a pain when we want to replace the implement of the old sync client > with the new async client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21710) Add quota related methods to the Admin interface
[ https://issues.apache.org/jira/browse/HBASE-21710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743561#comment-16743561 ] Duo Zhang commented on HBASE-21710: --- Let me commit. > Add quota related methods to the Admin interface > > > Key: HBASE-21710 > URL: https://issues.apache.org/jira/browse/HBASE-21710 > Project: HBase > Issue Type: Task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21710-v1.patch, HBASE-21710-v1.patch, > HBASE-21710.patch > > > So we can limit the ClusterConnection usage inside the sync client > implementation. Now we will use ClusterConnection in QuotaTableUtil, which > will be a pain when we want to replace the implement of the old sync client > with the new async client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21730) Update HBase-book with the procedure based WAL splitting
[ https://issues.apache.org/jira/browse/HBASE-21730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingyun Tian updated HBASE-21730: - Summary: Update HBase-book with the procedure based WAL splitting (was: Update hbase-book with the procedure based WAL splitting) > Update HBase-book with the procedure based WAL splitting > > > Key: HBASE-21730 > URL: https://issues.apache.org/jira/browse/HBASE-21730 > Project: HBase > Issue Type: Sub-task >Reporter: Jingyun Tian >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21034) Add new throttle type: read/write capacity unit
[ https://issues.apache.org/jira/browse/HBASE-21034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743556#comment-16743556 ] Andrew Purtell commented on HBASE-21034: Yeah branch-1 backport is being considered on HBASE-21616. Patch there is ready for review (or discussion) > Add new throttle type: read/write capacity unit > --- > > Key: HBASE-21034 > URL: https://issues.apache.org/jira/browse/HBASE-21034 > Project: HBase > Issue Type: Sub-task >Affects Versions: 3.0.0, 2.2.0 >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21034.master.001.patch, > HBASE-21034.master.002.patch, HBASE-21034.master.003.patch, > HBASE-21034.master.004.patch, HBASE-21034.master.005.patch, > HBASE-21034.master.006.patch, HBASE-21034.master.006.patch, > HBASE-21034.master.007.patch, HBASE-21034.master.007.patch > > > Add new throttle type: read/write capacity unit like DynamoDB. > One read capacity unit represents that read up to 1K data per time unit. If > data size is more than 1K, then consume additional read capacity units. > One write capacity unit represents that one write for an item up to 1 KB in > size per time unit. If data size is more than 1K, then consume additional > write capacity units. > For example, 100 read capacity units per second means that, HBase user can > read 100 times for 1K data in every second, or 50 times for 2K data in every > second and so on. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21729) Extract ProcedureCoordinatorRpcs and ProcedureMemberRpcs from CoordinatedStateManager
Jingyun Tian created HBASE-21729: Summary: Extract ProcedureCoordinatorRpcs and ProcedureMemberRpcs from CoordinatedStateManager Key: HBASE-21729 URL: https://issues.apache.org/jira/browse/HBASE-21729 Project: HBase Issue Type: Sub-task Reporter: Jingyun Tian Assignee: Jingyun Tian If procedureV2 based WAL splitting is enabled, CoordinatedStateManager will not be initialized. Then ProcedureCoordinatorRpcs and ProcedureMemberRpcs will make backup not work. Let me extract these two method to another class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21675) Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a lot of time) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743555#comment-16743555 ] Zheng Hu commented on HBASE-21675: -- What's you patch ? [~apurtell], I can take a look for the ut. > Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a > lot of time) to branch-1 > > > Key: HBASE-21675 > URL: https://issues.apache.org/jira/browse/HBASE-21675 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21675) Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a lot of time) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743559#comment-16743559 ] Zheng Hu commented on HBASE-21675: -- Well, let me dig this. > Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a > lot of time) to branch-1 > > > Key: HBASE-21675 > URL: https://issues.apache.org/jira/browse/HBASE-21675 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Priority: Major > Attachments: HBASE-21675-branch-1-WIP.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21675) Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a lot of time) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743558#comment-16743558 ] Andrew Purtell commented on HBASE-21675: Thanks [~openinx]. Attached the WIP patch. Tests pass except unit for snapshot+bulkload. Feel free to take this if you like! > Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a > lot of time) to branch-1 > > > Key: HBASE-21675 > URL: https://issues.apache.org/jira/browse/HBASE-21675 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Priority: Major > Attachments: HBASE-21675-branch-1-WIP.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21675) Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a lot of time) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21675: --- Attachment: HBASE-21675-branch-1-WIP.patch > Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a > lot of time) to branch-1 > > > Key: HBASE-21675 > URL: https://issues.apache.org/jira/browse/HBASE-21675 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Priority: Major > Attachments: HBASE-21675-branch-1-WIP.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21657) PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.
[ https://issues.apache.org/jira/browse/HBASE-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21657: - Resolution: Fixed Hadoop Flags: Incompatible change,Reviewed (was: Incompatible change) Status: Resolved (was: Patch Available) > PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% > scan case. > > > Key: HBASE-21657 > URL: https://issues.apache.org/jira/browse/HBASE-21657 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21657.v1.patch, HBASE-21657.v2.patch, > HBASE-21657.v3.patch, HBASE-21657.v3.patch, HBASE-21657.v4.patch, > HBASE-21657.v5.patch, HBASE-21657.v5.patch, HBASE-21657.v5.patch, > HBASE-21657.v6.patch, HBASE-21657.v7.patch, > HBase1.4.9-ssd-1000-rows-flamegraph.svg, > HBase1.4.9-ssd-1000-rows-qps-latency.png, > HBase2.0.4-patch-v2-ssd-1000-rows-qps-and-latency.png, > HBase2.0.4-patch-v2-ssd-1000-rows.svg, > HBase2.0.4-patch-v3-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-patch-v3-ssd-1000-rows-qps-and-latency.png, > HBase2.0.4-patch-v4-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-ssd-1000-rows-qps-latency.png, HBase2.0.4-with-patch.v2.png, > HBase2.0.4-without-patch-v2.png, debug-the-ByteBufferKeyValue.diff, > hbase2.0.4-ssd-scan-traces.2.svg, hbase2.0.4-ssd-scan-traces.svg, > hbase20-ssd-100-scan-traces.svg, image-2019-01-07-19-03-37-930.png, > image-2019-01-07-19-03-55-577.png, overview-statstics-1.png, run.log > > > We are evaluating the performance of branch-2, and find that the throughput > of scan in SSD cluster is almost the same as HDD cluster. so I made a > FlameGraph on RS, and found that the > PrivateCellUtil#estimatedSerializedSizeOf cost about 29% cpu, Obviously, it > has been the bottleneck in 100% scan case. > See the [^hbase20-ssd-100-scan-traces.svg] > BTW, in our XiaoMi branch, we introduce a > HRegion#updateReadRequestsByCapacityUnitPerSecond to sum up the size of cells > (for metric monitor), so it seems the performance loss was amplified. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HBASE-21675) Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a lot of time) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell reopened HBASE-21675: > Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a > lot of time) to branch-1 > > > Key: HBASE-21675 > URL: https://issues.apache.org/jira/browse/HBASE-21675 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21657) PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.
[ https://issues.apache.org/jira/browse/HBASE-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743557#comment-16743557 ] Zheng Hu commented on HBASE-21657: -- Pushed to branch-2 & master, thanks for your careful feedback, [~stack]. > PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% > scan case. > > > Key: HBASE-21657 > URL: https://issues.apache.org/jira/browse/HBASE-21657 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21657.v1.patch, HBASE-21657.v2.patch, > HBASE-21657.v3.patch, HBASE-21657.v3.patch, HBASE-21657.v4.patch, > HBASE-21657.v5.patch, HBASE-21657.v5.patch, HBASE-21657.v5.patch, > HBASE-21657.v6.patch, HBASE-21657.v7.patch, > HBase1.4.9-ssd-1000-rows-flamegraph.svg, > HBase1.4.9-ssd-1000-rows-qps-latency.png, > HBase2.0.4-patch-v2-ssd-1000-rows-qps-and-latency.png, > HBase2.0.4-patch-v2-ssd-1000-rows.svg, > HBase2.0.4-patch-v3-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-patch-v3-ssd-1000-rows-qps-and-latency.png, > HBase2.0.4-patch-v4-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-ssd-1000-rows-qps-latency.png, HBase2.0.4-with-patch.v2.png, > HBase2.0.4-without-patch-v2.png, debug-the-ByteBufferKeyValue.diff, > hbase2.0.4-ssd-scan-traces.2.svg, hbase2.0.4-ssd-scan-traces.svg, > hbase20-ssd-100-scan-traces.svg, image-2019-01-07-19-03-37-930.png, > image-2019-01-07-19-03-55-577.png, overview-statstics-1.png, run.log > > > We are evaluating the performance of branch-2, and find that the throughput > of scan in SSD cluster is almost the same as HDD cluster. so I made a > FlameGraph on RS, and found that the > PrivateCellUtil#estimatedSerializedSizeOf cost about 29% cpu, Obviously, it > has been the bottleneck in 100% scan case. > See the [^hbase20-ssd-100-scan-traces.svg] > BTW, in our XiaoMi branch, we introduce a > HRegion#updateReadRequestsByCapacityUnitPerSecond to sum up the size of cells > (for metric monitor), so it seems the performance loss was amplified. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21657) PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.
[ https://issues.apache.org/jira/browse/HBASE-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21657: - Fix Version/s: (was: 2.0.5) (was: 2.1.3) > PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% > scan case. > > > Key: HBASE-21657 > URL: https://issues.apache.org/jira/browse/HBASE-21657 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21657.v1.patch, HBASE-21657.v2.patch, > HBASE-21657.v3.patch, HBASE-21657.v3.patch, HBASE-21657.v4.patch, > HBASE-21657.v5.patch, HBASE-21657.v5.patch, HBASE-21657.v5.patch, > HBASE-21657.v6.patch, HBASE-21657.v7.patch, > HBase1.4.9-ssd-1000-rows-flamegraph.svg, > HBase1.4.9-ssd-1000-rows-qps-latency.png, > HBase2.0.4-patch-v2-ssd-1000-rows-qps-and-latency.png, > HBase2.0.4-patch-v2-ssd-1000-rows.svg, > HBase2.0.4-patch-v3-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-patch-v3-ssd-1000-rows-qps-and-latency.png, > HBase2.0.4-patch-v4-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-ssd-1000-rows-qps-latency.png, HBase2.0.4-with-patch.v2.png, > HBase2.0.4-without-patch-v2.png, debug-the-ByteBufferKeyValue.diff, > hbase2.0.4-ssd-scan-traces.2.svg, hbase2.0.4-ssd-scan-traces.svg, > hbase20-ssd-100-scan-traces.svg, image-2019-01-07-19-03-37-930.png, > image-2019-01-07-19-03-55-577.png, overview-statstics-1.png, run.log > > > We are evaluating the performance of branch-2, and find that the throughput > of scan in SSD cluster is almost the same as HDD cluster. so I made a > FlameGraph on RS, and found that the > PrivateCellUtil#estimatedSerializedSizeOf cost about 29% cpu, Obviously, it > has been the bottleneck in 100% scan case. > See the [^hbase20-ssd-100-scan-traces.svg] > BTW, in our XiaoMi branch, we introduce a > HRegion#updateReadRequestsByCapacityUnitPerSecond to sum up the size of cells > (for metric monitor), so it seems the performance loss was amplified. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-21674) Port HBASE-21652 (Refactor ThriftServer making thrift2 server inherited from thrift1 server) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-21674. Resolution: Later Fix Version/s: (was: 1.5.0) Not going to have time for this. At best it is a nice-to-have. Resolving as Later > Port HBASE-21652 (Refactor ThriftServer making thrift2 server inherited from > thrift1 server) to branch-1 > > > Key: HBASE-21674 > URL: https://issues.apache.org/jira/browse/HBASE-21674 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21710) Add quota related methods to the Admin interface
[ https://issues.apache.org/jira/browse/HBASE-21710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743552#comment-16743552 ] Duo Zhang commented on HBASE-21710: --- Thanks sir. I've replied on HBASE-21724. And the final replacement will be done on branch HBASE-21512. And it will have a fat release note when we want to merge it back to master:) > Add quota related methods to the Admin interface > > > Key: HBASE-21710 > URL: https://issues.apache.org/jira/browse/HBASE-21710 > Project: HBase > Issue Type: Task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21710-v1.patch, HBASE-21710-v1.patch, > HBASE-21710.patch > > > So we can limit the ClusterConnection usage inside the sync client > implementation. Now we will use ClusterConnection in QuotaTableUtil, which > will be a pain when we want to replace the implement of the old sync client > with the new async client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21728) Backport to branch-2 (and maybe branch-1) HBASE-21684 Throw DNRIOE when connection or rpc client is closed
[
https://issues.apache.org/jira/browse/HBASE-21728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743550#comment-16743550
]
Hadoop QA commented on HBASE-21728:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
47s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} branch-1 passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
47s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} branch-1 passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed with JDK v1.7.0_201 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} hbase-client: The patch generated 0 new + 0
unchanged - 1 fixed = 0 total (was 1) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
12s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
2m 4s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed with JDK v1.7.0_201 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
34s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:61288f8 |
| JIRA Issue | HBASE-21728 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12955025/HBASE-21728-branch-1.001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars
[jira] [Assigned] (HBASE-21674) Port HBASE-21652 (Refactor ThriftServer making thrift2 server inherited from thrift1 server) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell reassigned HBASE-21674: -- Assignee: (was: Andrew Purtell) > Port HBASE-21652 (Refactor ThriftServer making thrift2 server inherited from > thrift1 server) to branch-1 > > > Key: HBASE-21674 > URL: https://issues.apache.org/jira/browse/HBASE-21674 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Priority: Major > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21728) Backport to branch-2 (and maybe branch-1) HBASE-21684 Throw DNRIOE when connection or rpc client is closed
[ https://issues.apache.org/jira/browse/HBASE-21728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743527#comment-16743527 ] Xu Cang commented on HBASE-21728: - the original patch can be applied to branch-1 without any change. Tested locally. for branch-1, since there is no ReadOnlyZKClient class, only need to change the StoppedRpcClientException class to throw DNRIOE. > Backport to branch-2 (and maybe branch-1) HBASE-21684 Throw DNRIOE when > connection or rpc client is closed > -- > > Key: HBASE-21728 > URL: https://issues.apache.org/jira/browse/HBASE-21728 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Priority: Major > Fix For: 2.2.0, 2.1.3, 2.0.5 > > > Backport the parent. May need a few changes. See suggestions in parent. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-21675) Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a lot of time) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-21675. Resolution: Later Fix Version/s: (was: 1.5.0) The backport was simple enough but deceptively so. New test for snapshot+bulkload did not pass. Spent some time looking at it but didn't make progress. This is only a nice to have at best so not worth more time. Resolving as Later. > Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a > lot of time) to branch-1 > > > Key: HBASE-21675 > URL: https://issues.apache.org/jira/browse/HBASE-21675 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-21675) Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a lot of time) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell reassigned HBASE-21675: -- Assignee: (was: Andrew Purtell) > Port HBASE-21642 (CopyTable by reading snapshot and bulkloading will save a > lot of time) to branch-1 > > > Key: HBASE-21675 > URL: https://issues.apache.org/jira/browse/HBASE-21675 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Priority: Major > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21713) Support set region server throttle quota
[ https://issues.apache.org/jira/browse/HBASE-21713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743545#comment-16743545 ] Xu Cang commented on HBASE-21713: - "+hbase> set_quota TYPE => THROTTLE, REGIONSERVER => 'all', THROTTLE_TYPE => WRITE, LIMIT => '2CU/sec' " Is there a typo here? I suppose you wanted to say 2req/sec. > Support set region server throttle quota > > > Key: HBASE-21713 > URL: https://issues.apache.org/jira/browse/HBASE-21713 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-21713.master.001.patch > > > Support set region server throttle quota which represents the read/write > ability of region servers. > Use the following command to set RS quota: > set_quota TYPE => THROTTLE, REGIONSERVER => 'all', THROTTLE_TYPE => WRITE, > LIMIT => '2req/sec' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21659) Avoid to load duplicate coprocessors in system config and table descriptor
[ https://issues.apache.org/jira/browse/HBASE-21659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743532#comment-16743532 ] Guanghao Zhang commented on HBASE-21659: [~stack] If enable it default, this behavior will not same with the old impl. Mark this as incompatiable change to enable this? And this should be apply to branch-2.0 and branch-2.1, too? > Avoid to load duplicate coprocessors in system config and table descriptor > -- > > Key: HBASE-21659 > URL: https://issues.apache.org/jira/browse/HBASE-21659 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Minor > Fix For: 3.0.0, 1.5.0, 2.2.0 > > Attachments: HBASE-21659.master.001.patch, > HBASE-21659.master.002.patch, HBASE-21659.master.003.patch, > HBASE-21659.master.004.patch > > > When there are same coprocessor in system coprocessor config and table > descriptor. It will load the coprocessor twice and the coprocessor method may > run twice, too. It should avoid to load duplicate coprocessors. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21034) Add new throttle type: read/write capacity unit
[ https://issues.apache.org/jira/browse/HBASE-21034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743531#comment-16743531 ] Guanghao Zhang commented on HBASE-21034: There already is a issue to backport to branch-1. So I thought we should backport it to branch-2.0 and branch-2.1, too? [~Yi Mei] Can you help to add a patch for branch-2.0 and branch-2.1? Thanks. > Add new throttle type: read/write capacity unit > --- > > Key: HBASE-21034 > URL: https://issues.apache.org/jira/browse/HBASE-21034 > Project: HBase > Issue Type: Sub-task >Affects Versions: 3.0.0, 2.2.0 >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21034.master.001.patch, > HBASE-21034.master.002.patch, HBASE-21034.master.003.patch, > HBASE-21034.master.004.patch, HBASE-21034.master.005.patch, > HBASE-21034.master.006.patch, HBASE-21034.master.006.patch, > HBASE-21034.master.007.patch, HBASE-21034.master.007.patch > > > Add new throttle type: read/write capacity unit like DynamoDB. > One read capacity unit represents that read up to 1K data per time unit. If > data size is more than 1K, then consume additional read capacity units. > One write capacity unit represents that one write for an item up to 1 KB in > size per time unit. If data size is more than 1K, then consume additional > write capacity units. > For example, 100 read capacity units per second means that, HBase user can > read 100 times for 1K data in every second, or 50 times for 2K data in every > second and so on. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21724) Introduce a createClusterConnection method in ClusterConnectionFactory
[ https://issues.apache.org/jira/browse/HBASE-21724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743534#comment-16743534 ] Duo Zhang commented on HBASE-21724: --- https://github.com/Apache9/hbase/blob/HBASE-21585/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClusterConnection.java This is what I have done so far to clean up the ClusterConnection interface, most methods can be removed. And the intention here is that, use ConnectionImplementation directly in the client library instead of ClusterConnection. This is what we have done in the async client implementation. The AsyncConnection interface is very clean, just like the Connection interface, and we do not have an AsyncClusterConnection in the client module, instead, all the related classes will use AsyncConnectionImpl directly, so you can get the protobuf stubs, the retrying callers, and everything you want. I think this is a common philosophy in software design, high cohesion and low coupling. The AsyncConnectionImpl is package private, so in other module you are not allowed to access it directly, you must use the methods in AsyncConnection to do what you want. So in the future, if we find another good framework and want to reimplement the client, we can just reimplement the AsyncConnection interface and remove the old implementation completely, as we do not leak any internal things out. So I want to apply the same pattern to the current sync client implementation. The on-going replacement work of the sync client is really a pain, as we leak everything out, the protobuf stubs and retrying callers are used everywhere in different modules and different packages. Hope we do not need to do the clean up work again in the future when we want to introduce a new client implementation... > Introduce a createClusterConnection method in ClusterConnectionFactory > -- > > Key: HBASE-21724 > URL: https://issues.apache.org/jira/browse/HBASE-21724 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Priority: Major > > When we want a cluster connection in the code, we should use this method to > create one directly, instead of casting the one created by ConnectionFactory. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21728) Backport to branch-2 (and maybe branch-1) HBASE-21684 Throw DNRIOE when connection or rpc client is closed
[ https://issues.apache.org/jira/browse/HBASE-21728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Cang updated HBASE-21728: Attachment: HBASE-21728-branch-1.001.patch Status: Patch Available (was: Open) > Backport to branch-2 (and maybe branch-1) HBASE-21684 Throw DNRIOE when > connection or rpc client is closed > -- > > Key: HBASE-21728 > URL: https://issues.apache.org/jira/browse/HBASE-21728 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Priority: Major > Fix For: 2.2.0, 2.1.3, 2.0.5 > > Attachments: HBASE-21728-branch-1.001.patch > > > Backport the parent. May need a few changes. See suggestions in parent. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21684) Throw DNRIOE when connection or rpc client is closed
[ https://issues.apache.org/jira/browse/HBASE-21684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743516#comment-16743516 ] Duo Zhang commented on HBASE-21684: --- The exception is marked as IA.Public so theoretically it should only be pushed to master? > Throw DNRIOE when connection or rpc client is closed > > > Key: HBASE-21684 > URL: https://issues.apache.org/jira/browse/HBASE-21684 > Project: HBase > Issue Type: Improvement > Components: asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21684-for-ut.patch, HBASE-21684.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20209) Do Not Use Both Map containsKey and get Methods
[
https://issues.apache.org/jira/browse/HBASE-20209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743493#comment-16743493
]
Hadoop QA commented on HBASE-20209:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
21s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
11s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 55s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}136m
40s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}184m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20209 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12955013/HBASE-20209.3.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux d278bd48d3d7 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 6da0b4ec34 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15589/testReport/ |
| Max. process+thread count | 4980 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
[jira] [Updated] (HBASE-21647) Add status track for splitting WAL tasks
[ https://issues.apache.org/jira/browse/HBASE-21647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21647: -- Component/s: Operability > Add status track for splitting WAL tasks > > > Key: HBASE-21647 > URL: https://issues.apache.org/jira/browse/HBASE-21647 > Project: HBase > Issue Type: Sub-task > Components: Operability >Reporter: Jingyun Tian >Assignee: Jingyun Tian >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21647.master.001.patch, > HBASE-21647.master.002.patch, HBASE-21647.master.003.patch, > HBASE-21647.master.003.patch, Screenshot from 2019-01-03 15-31-26.png, > Screenshot from 2019-01-03 15-31-36.png > > > add status track to help operator check the status of splitting WAL -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21647) Add status track for splitting WAL tasks
[ https://issues.apache.org/jira/browse/HBASE-21647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21647: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.2.0 3.0.0 Release Note: Adds task monitor that shows ServerCrashProcedure progress in UI. Status: Resolved (was: Patch Available) Pushed to branch-2 and master. Thanks for the patch [~tianjingyun]. > Add status track for splitting WAL tasks > > > Key: HBASE-21647 > URL: https://issues.apache.org/jira/browse/HBASE-21647 > Project: HBase > Issue Type: Sub-task >Reporter: Jingyun Tian >Assignee: Jingyun Tian >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21647.master.001.patch, > HBASE-21647.master.002.patch, HBASE-21647.master.003.patch, > HBASE-21647.master.003.patch, Screenshot from 2019-01-03 15-31-26.png, > Screenshot from 2019-01-03 15-31-36.png > > > add status track to help operator check the status of splitting WAL -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21034) Add new throttle type: read/write capacity unit
[ https://issues.apache.org/jira/browse/HBASE-21034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743452#comment-16743452 ] stack commented on HBASE-21034: --- [~jatsakthi] looks like new feature rather than bug fix? You need it in branch-2.1 or branch-2.0 [~zghaobac]/[~allan163] ? > Add new throttle type: read/write capacity unit > --- > > Key: HBASE-21034 > URL: https://issues.apache.org/jira/browse/HBASE-21034 > Project: HBase > Issue Type: Sub-task >Affects Versions: 3.0.0, 2.2.0 >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21034.master.001.patch, > HBASE-21034.master.002.patch, HBASE-21034.master.003.patch, > HBASE-21034.master.004.patch, HBASE-21034.master.005.patch, > HBASE-21034.master.006.patch, HBASE-21034.master.006.patch, > HBASE-21034.master.007.patch, HBASE-21034.master.007.patch > > > Add new throttle type: read/write capacity unit like DynamoDB. > One read capacity unit represents that read up to 1K data per time unit. If > data size is more than 1K, then consume additional read capacity units. > One write capacity unit represents that one write for an item up to 1 KB in > size per time unit. If data size is more than 1K, then consume additional > write capacity units. > For example, 100 read capacity units per second means that, HBase user can > read 100 times for 1K data in every second, or 50 times for 2K data in every > second and so on. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21647) Add status track for splitting WAL tasks
[
https://issues.apache.org/jira/browse/HBASE-21647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743438#comment-16743438
]
Hadoop QA commented on HBASE-21647:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
13s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
12s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 33s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}130m
48s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}168m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21647 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12955004/HBASE-21647.master.003.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 7bc6d66cb269 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 0fd4243fcd |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15587/testReport/ |
| Max. process+thread count | 5005 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
[jira] [Commented] (HBASE-21627) race condition between a recovered RIT for meta replica, and master startup
[
https://issues.apache.org/jira/browse/HBASE-21627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743424#comment-16743424
]
Sergey Shelukhin commented on HBASE-21627:
--
Update: we've seen another instance of this or similar race where the meta
replica actually gets assigned by one of the procedures, but then the other
messes something up and while RS thinks the replica is opened, master thinks
it's in opening state, and it's stuck forever like that, including for some
reason across master restarts (may be a persistent bad state, or it's racing
repeatedly).
We kinda gave up on meta replicas at this point, so I'm not investigating
further. But I think it needs to be redone to solve all the races and crashes
(also HBASE-21624 :) Right now it seems uncoordinated with anything.
> race condition between a recovered RIT for meta replica, and master startup
> ---
>
> Key: HBASE-21627
> URL: https://issues.apache.org/jira/browse/HBASE-21627
> Project: HBase
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Priority: Major
>
> Master recovers RIT for a meta replica
> {noformat}
> 2018-12-14 23:16:12,008 INFO [master/...:17000:becomeActiveMaster]
> assignment.AssignmentManager: Attach pid=83796, ppid=83788,
> state=RUNNABLE:REGION_STATE_TRANSITION_OPEN, hasLock=false;
> TransitRegionStateProcedure table=hbase:meta, region=(region), ASSIGN to
> rit=OFFLINE, location=null, table=hbase:meta, region=(region) to restore RIT
> 2018-12-14 23:16:16,475 WARN [PEWorker-8]
> assignment.TransitRegionStateProcedure: No location specified for {ENCODED =>
> (region), NAME => 'hbase:meta,,1_0001', STARTKEY => '', ENDKEY => '',
> REPLICA_ID => 1}, jump back to state
> REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE to get one
> ...
> 2018-12-14 23:16:30,010 INFO [PEWorker-16] procedure2.ProcedureExecutor:
> Finished pid=83796, ppid=83788, state=SUCCESS, hasLock=false;
> TransitRegionStateProcedure table=hbase:meta, region=(region), ASSIGN in
> 8mins, 23.39sec
> {noformat}
> Then tries to assign replicas..
> {noformat}
> 2018-12-14 23:16:36,091 ERROR [master/...:17000:becomeActiveMaster]
> master.HMaster: Failed to become active master
> org.apache.hadoop.hbase.client.DoNotRetryRegionException: Unexpected state
> for rit=OPEN, location=server,17020,1544858156805, table=hbase:meta,
> region=(region)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.preTransitCheck(AssignmentManager.java:548)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.assign(AssignmentManager.java:563)
> at
> org.apache.hadoop.hbase.master.MasterMetaBootstrap.assignMetaReplicas(MasterMetaBootstrap.java:84)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1146)
> {noformat}
> Unfortunately I misplaced the log from this after copy-pasting a grep result
> so that's all I have for this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21712) Make submit-patch.py python3 compatible
[ https://issues.apache.org/jira/browse/HBASE-21712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743410#comment-16743410 ] Sergey Shelukhin commented on HBASE-21712: -- +1 on the addendum > Make submit-patch.py python3 compatible > --- > > Key: HBASE-21712 > URL: https://issues.apache.org/jira/browse/HBASE-21712 > Project: HBase > Issue Type: Improvement > Components: tooling >Reporter: Tommy Li >Assignee: Tommy Li >Priority: Minor > Fix For: 3.0.0 > > Attachments: HBASE-21712.master.001.patch, > HBASE-21712.master.002.patch, HBASE-21722.master.001.patch.txt > > > Attached patch was submitted with `python3 dev-support/submit-patch.py -b > master -srb -jid HBASE-21712` -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20209) Do Not Use Both Map containsKey and get Methods
[
https://issues.apache.org/jira/browse/HBASE-20209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HBASE-20209:
Status: Open (was: Patch Available)
> Do Not Use Both Map containsKey and get Methods
> ---
>
> Key: HBASE-20209
> URL: https://issues.apache.org/jira/browse/HBASE-20209
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HBASE-20209.1.patch, HBASE-20209.2.patch,
> HBASE-20209.3.patch
>
>
> {code:title=ReplicationSink.java}
> String tableName = table.getNameWithNamespaceInclAsString();
> if (bulkLoadHFileMap.containsKey(tableName)) {
> List>> familyHFilePathsList =
> bulkLoadHFileMap.get(tableName);
> boolean foundFamily = false;
> for (int i = 0; i < familyHFilePathsList.size(); i++) {
> Pair> familyHFilePathsPair =
> familyHFilePathsList.get(i);
> if (Bytes.equals(familyHFilePathsPair.getFirst(), family)) {
> // Found family already present, just add the path to the
> existing list
> familyHFilePathsPair.getSecond().add(pathToHfileFromNS);
> foundFamily = true;
> break;
> }
> }
> {code}
> I propose that this code does not use the Map methods _containsKey_ *and*
> _get_. Simply use the _get_ method once and check a _null_ return value to
> check for existence. Saves a trip to the Map data structure for each call.
> Also, use enhanced for loop.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20209) Do Not Use Both Map containsKey and get Methods
[
https://issues.apache.org/jira/browse/HBASE-20209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HBASE-20209:
Attachment: HBASE-20209.3.patch
> Do Not Use Both Map containsKey and get Methods
> ---
>
> Key: HBASE-20209
> URL: https://issues.apache.org/jira/browse/HBASE-20209
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HBASE-20209.1.patch, HBASE-20209.2.patch,
> HBASE-20209.3.patch
>
>
> {code:title=ReplicationSink.java}
> String tableName = table.getNameWithNamespaceInclAsString();
> if (bulkLoadHFileMap.containsKey(tableName)) {
> List>> familyHFilePathsList =
> bulkLoadHFileMap.get(tableName);
> boolean foundFamily = false;
> for (int i = 0; i < familyHFilePathsList.size(); i++) {
> Pair> familyHFilePathsPair =
> familyHFilePathsList.get(i);
> if (Bytes.equals(familyHFilePathsPair.getFirst(), family)) {
> // Found family already present, just add the path to the
> existing list
> familyHFilePathsPair.getSecond().add(pathToHfileFromNS);
> foundFamily = true;
> break;
> }
> }
> {code}
> I propose that this code does not use the Map methods _containsKey_ *and*
> _get_. Simply use the _get_ method once and check a _null_ return value to
> check for existence. Saves a trip to the Map data structure for each call.
> Also, use enhanced for loop.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20209) Do Not Use Both Map containsKey and get Methods
[
https://issues.apache.org/jira/browse/HBASE-20209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HBASE-20209:
Status: Patch Available (was: Open)
Attaching same page again. Making sure it's still good.
> Do Not Use Both Map containsKey and get Methods
> ---
>
> Key: HBASE-20209
> URL: https://issues.apache.org/jira/browse/HBASE-20209
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HBASE-20209.1.patch, HBASE-20209.2.patch,
> HBASE-20209.3.patch
>
>
> {code:title=ReplicationSink.java}
> String tableName = table.getNameWithNamespaceInclAsString();
> if (bulkLoadHFileMap.containsKey(tableName)) {
> List>> familyHFilePathsList =
> bulkLoadHFileMap.get(tableName);
> boolean foundFamily = false;
> for (int i = 0; i < familyHFilePathsList.size(); i++) {
> Pair> familyHFilePathsPair =
> familyHFilePathsList.get(i);
> if (Bytes.equals(familyHFilePathsPair.getFirst(), family)) {
> // Found family already present, just add the path to the
> existing list
> familyHFilePathsPair.getSecond().add(pathToHfileFromNS);
> foundFamily = true;
> break;
> }
> }
> {code}
> I propose that this code does not use the Map methods _containsKey_ *and*
> _get_. Simply use the _get_ method once and check a _null_ return value to
> check for existence. Saves a trip to the Map data structure for each call.
> Also, use enhanced for loop.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21596) HBase Shell "delete" command can cause older versions to be shown even if VERSIONS is configured as 1
[
https://issues.apache.org/jira/browse/HBASE-21596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743407#comment-16743407
]
Wellington Chevreuil commented on HBASE-21596:
--
{quote}This would seem to argue that we should not allow explicit delete of a
single Cell version – we should always delete all behind the tombstone?
{quote}
Good point, I guess that would then require an API change, since
*Delete.addColumn(family,qualifier)/Delete.addColumn(family,qualifier,timestamp)*
wouldn't make sense. Ain't sure how important this feature is, since it seems
broken for a while, so maybe minimal impact to have this API change.
{quote}A point delete could always revive the Cell that was just outside the
version window – at least until the major compaction runs?
{quote}
So this latest submitted patch fixes that by also adding delete markers to any
existing extra versions once a delete for an specific TS is ran. The way shell
delete was behaving prior to HBASE-18142 was that it was always deleting all
versions (calling *Delete.addColumns*, instead of *Delete.addColumn* with the
latest TS).
> HBase Shell "delete" command can cause older versions to be shown even if
> VERSIONS is configured as 1
> -
>
> Key: HBASE-21596
> URL: https://issues.apache.org/jira/browse/HBASE-21596
> Project: HBase
> Issue Type: Bug
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Minor
> Attachments: HBASE-21596-master.001.patch,
> HBASE-21596-master.002.patch, HBASE-21596-master.003.patch, initial-patch.txt
>
>
> HBase Shell delete command is supposed to operate over an specific TS. If no
> TS is informed, it will assume the latest TS for the cell and put delete
> marker for it.
> However, for a CF with VERSIONS => 1, if multiple puts were performed for
> same cell, there may be multiple cell versions on the memstore, so delete
> would only be "delete marking" one of those, and causing the most recent no
> marked one to be shown on gets/scans, which then contradicts the CF "VERSIONS
> => 1" configuration.
> This issue is not seen with deleteall command or using Delete operation from
> Java API.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21712) Make submit-patch.py python3 compatible
[
https://issues.apache.org/jira/browse/HBASE-21712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743398#comment-16743398
]
Hadoop QA commented on HBASE-21712:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
3s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.8.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 0m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21712 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12955009/HBASE-21722.master.001.patch.txt
|
| Optional Tests | dupname asflicense |
| uname | Linux 9a7ce315477f 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 6da0b4ec34 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Max. process+thread count | 48 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15588/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Make submit-patch.py python3 compatible
> ---
>
> Key: HBASE-21712
> URL: https://issues.apache.org/jira/browse/HBASE-21712
> Project: HBase
> Issue Type: Improvement
> Components: tooling
>Reporter: Tommy Li
>Assignee: Tommy Li
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-21712.master.001.patch,
> HBASE-21712.master.002.patch, HBASE-21722.master.001.patch.txt
>
>
> Attached patch was submitted with `python3 dev-support/submit-patch.py -b
> master -srb -jid HBASE-21712`
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21712) Make submit-patch.py python3 compatible
[ https://issues.apache.org/jira/browse/HBASE-21712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743395#comment-16743395 ] Tommy Li commented on HBASE-21712: -- [~psomogyi] I've attached HBASE-21722.master.001.patch > Make submit-patch.py python3 compatible > --- > > Key: HBASE-21712 > URL: https://issues.apache.org/jira/browse/HBASE-21712 > Project: HBase > Issue Type: Improvement > Components: tooling >Reporter: Tommy Li >Assignee: Tommy Li >Priority: Minor > Fix For: 3.0.0 > > Attachments: HBASE-21712.master.001.patch, > HBASE-21712.master.002.patch, HBASE-21722.master.001.patch.txt > > > Attached patch was submitted with `python3 dev-support/submit-patch.py -b > master -srb -jid HBASE-21712` -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21712) Make submit-patch.py python3 compatible
[ https://issues.apache.org/jira/browse/HBASE-21712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tommy Li updated HBASE-21712: - Attachment: HBASE-21722.master.001.patch.txt Status: Patch Available (was: Reopened) > Make submit-patch.py python3 compatible > --- > > Key: HBASE-21712 > URL: https://issues.apache.org/jira/browse/HBASE-21712 > Project: HBase > Issue Type: Improvement > Components: tooling >Reporter: Tommy Li >Assignee: Tommy Li >Priority: Minor > Fix For: 3.0.0 > > Attachments: HBASE-21712.master.001.patch, > HBASE-21712.master.002.patch, HBASE-21722.master.001.patch.txt > > > Attached patch was submitted with `python3 dev-support/submit-patch.py -b > master -srb -jid HBASE-21712` -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743392#comment-16743392
]
Sakthi commented on HBASE-21225:
[~elserj], I think this backport can be done after HBASE-21034 is backported to
branch-2.0 & branch-2.1.
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch,
> hbase-21225.master.004.patch, hbase-21225.master.005.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21034) Add new throttle type: read/write capacity unit
[ https://issues.apache.org/jira/browse/HBASE-21034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743388#comment-16743388 ] Sakthi commented on HBASE-21034: [~stack] & [~zghaobac], we might want to push this to branch-2.0 & branch-2.1 as well? > Add new throttle type: read/write capacity unit > --- > > Key: HBASE-21034 > URL: https://issues.apache.org/jira/browse/HBASE-21034 > Project: HBase > Issue Type: Sub-task >Affects Versions: 3.0.0, 2.2.0 >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21034.master.001.patch, > HBASE-21034.master.002.patch, HBASE-21034.master.003.patch, > HBASE-21034.master.004.patch, HBASE-21034.master.005.patch, > HBASE-21034.master.006.patch, HBASE-21034.master.006.patch, > HBASE-21034.master.007.patch, HBASE-21034.master.007.patch > > > Add new throttle type: read/write capacity unit like DynamoDB. > One read capacity unit represents that read up to 1K data per time unit. If > data size is more than 1K, then consume additional read capacity units. > One write capacity unit represents that one write for an item up to 1 KB in > size per time unit. If data size is more than 1K, then consume additional > write capacity units. > For example, 100 read capacity units per second means that, HBase user can > read 100 times for 1K data in every second, or 50 times for 2K data in every > second and so on. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-20917) MetaTableMetrics#stop references uninitialized requestsMap for non-meta region
[
https://issues.apache.org/jira/browse/HBASE-20917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey resolved HBASE-20917.
-
Resolution: Fixed
Fix Version/s: 2.1.3
> MetaTableMetrics#stop references uninitialized requestsMap for non-meta region
> --
>
> Key: HBASE-20917
> URL: https://issues.apache.org/jira/browse/HBASE-20917
> Project: HBase
> Issue Type: Bug
> Components: meta, metrics
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 2.2.0, 2.1.3, 2.0.4, 1.4.6
>
> Attachments: 20917.addendum, 20917.v1.txt, 20917.v2.txt
>
>
> I noticed the following in test output:
> {code}
> 2018-07-21 15:54:43,181 ERROR [RS_CLOSE_REGION-regionserver/172.17.5.4:0-1]
> executor.EventHandler(186): Caught throwable while processing event
> M_RS_CLOSE_REGION
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics.stop(MetaTableMetrics.java:329)
> at
> org.apache.hadoop.hbase.coprocessor.BaseEnvironment.shutdown(BaseEnvironment.java:91)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionEnvironment.shutdown(RegionCoprocessorHost.java:165)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.shutdown(CoprocessorHost.java:290)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$4.postEnvCall(RegionCoprocessorHost.java:559)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:622)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postClose(RegionCoprocessorHost.java:551)
> at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1678)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1484)
> at
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:104)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> {code}
> {{requestsMap}} is only initialized for the meta region.
> However, check for meta region is absent in the stop method:
> {code}
> public void stop(CoprocessorEnvironment e) throws IOException {
> // since meta region can move around, clear stale metrics when stop.
> for (String meterName : requestsMap.keySet()) {
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19722) Meta query statistics metrics source
[
https://issues.apache.org/jira/browse/HBASE-19722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-19722:
Resolution: Fixed
Fix Version/s: 2.1.3
Status: Resolved (was: Patch Available)
> Meta query statistics metrics source
>
>
> Key: HBASE-19722
> URL: https://issues.apache.org/jira/browse/HBASE-19722
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors, meta, metrics, Operability
>Reporter: Andrew Purtell
>Assignee: Xu Cang
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 2.2.0, 2.1.3, 2.0.2, 1.4.6
>
> Attachments: HBASE-19722-branch-2.1.v1.patch,
> HBASE-19722.branch-1.v001.patch, HBASE-19722.branch-1.v002.patch,
> HBASE-19722.master.010.patch, HBASE-19722.master.011.patch,
> HBASE-19722.master.012.patch, HBASE-19722.master.013.patch,
> HBASE-19722.master.014.patch, HBASE-19722.master.015.patch,
> HBASE-19722.master.016.patch
>
>
> Implement a meta query statistics metrics source, created whenever a
> regionserver starts hosting meta, removed when meta hosting moves. Provide
> views on top tables by request counts, top meta rowkeys by request count, top
> clients making requests by their hostname.
> Can be implemented as a coprocessor.
>
>
>
>
> ===
> *Release Note* (WIP)
> *1. Usage:*
> Use this coprocessor by adding below section to hbase-site.xml
> {{}}
> {{ hbase.coprocessor.region.classes}}
> {{ org.apache.hadoop.hbase.coprocessor.MetaTableMetrics}}
> {{}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-21590) Optimize trySkipToNextColumn in StoreScanner a bit
[ https://issues.apache.org/jira/browse/HBASE-21590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey resolved HBASE-21590. - Resolution: Fixed Fix Version/s: 2.1.3 > Optimize trySkipToNextColumn in StoreScanner a bit > -- > > Key: HBASE-21590 > URL: https://issues.apache.org/jira/browse/HBASE-21590 > Project: HBase > Issue Type: Improvement > Components: Performance, Scanners >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Critical > Fix For: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.1.3, 2.0.4 > > Attachments: 21590-1.5.txt, HBASE-21590-master.txt, > HBASE-21590.addendum.v1.patch > > > See latest comment on HBASE-17958 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21634) Print error message when user uses unacceptable values for LIMIT while setting quotas.
[
https://issues.apache.org/jira/browse/HBASE-21634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743378#comment-16743378
]
Sakthi commented on HBASE-21634:
Thanks for pushing [~zghaobac]. I'll work on patches for branch-2.1 & branch-2.0
> Print error message when user uses unacceptable values for LIMIT while
> setting quotas.
> --
>
> Key: HBASE-21634
> URL: https://issues.apache.org/jira/browse/HBASE-21634
> Project: HBase
> Issue Type: Improvement
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: hbase-21634.master.001.patch,
> hbase-21634.master.002.patch, hbase-21634.master.003.patch,
> hbase-21634.master.004.patch
>
>
> When unacceptable value(like 2.3G or 70H) to LIMIT are passed while setting
> quotas, we currently do not print any error message (informing the user about
> the erroneous input). Like below:
> {noformat}
> hbase(main):002:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '2.3G',
> POLICY => NO_WRITES
> Took 0.0792 seconds
> hbase(main):003:0> list_quotas
> OWNERQUOTAS
> TABLE => t2 TYPE => SPACE,
> TABLE => t2, LIMIT => 2B, VIOLATION_POLICY => NO_WRITES
> 1 row(s)
> Took 0.0512 seconds
> hbase(main):006:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '70H',
> POLICY => NO_WRITES
> Took 0.0088 seconds
> hbase(main):007:0> list_quotas
> OWNERQUOTAS
> TABLE => t2 TYPE => SPACE,
> TABLE => t2, LIMIT => 70B, VIOLATION_POLICY => NO_WRITES
> 1 row(s)
> Took 0.0448 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21590) Optimize trySkipToNextColumn in StoreScanner a bit
[ https://issues.apache.org/jira/browse/HBASE-21590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-21590: Fix Version/s: (was: 2.1.2) > Optimize trySkipToNextColumn in StoreScanner a bit > -- > > Key: HBASE-21590 > URL: https://issues.apache.org/jira/browse/HBASE-21590 > Project: HBase > Issue Type: Improvement > Components: Performance, Scanners >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Critical > Fix For: 3.0.0, 1.5.0, 2.2.0, 2.0.4, 1.4.10 > > Attachments: 21590-1.5.txt, HBASE-21590-master.txt, > HBASE-21590.addendum.v1.patch > > > See latest comment on HBASE-17958 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21595) Print thread's information and stack traces when RS is aborting forcibly
[ https://issues.apache.org/jira/browse/HBASE-21595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21595: -- Labels: operability (was: ) > Print thread's information and stack traces when RS is aborting forcibly > > > Key: HBASE-21595 > URL: https://issues.apache.org/jira/browse/HBASE-21595 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 3.0.0, 2.2.0 >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Minor > Labels: operability > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21595.patch > > > After HBASE-21325 RS terminate forcibly on abort timeout. > We should print the thread info before terminating, will be useful to analyze > the RS abort timeout problem. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HBASE-21590) Optimize trySkipToNextColumn in StoreScanner a bit
[ https://issues.apache.org/jira/browse/HBASE-21590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reopened HBASE-21590: - looks like I missed branch-2.1 after all. reopening while I push it. > Optimize trySkipToNextColumn in StoreScanner a bit > -- > > Key: HBASE-21590 > URL: https://issues.apache.org/jira/browse/HBASE-21590 > Project: HBase > Issue Type: Improvement > Components: Performance, Scanners >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Critical > Fix For: 3.0.0, 1.5.0, 2.2.0, 2.1.2, 2.0.4, 1.4.10 > > Attachments: 21590-1.5.txt, HBASE-21590-master.txt, > HBASE-21590.addendum.v1.patch > > > See latest comment on HBASE-17958 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21728) Backport to branch-2 (and maybe branch-1) HBASE-21684 Throw DNRIOE when connection or rpc client is closed
[ https://issues.apache.org/jira/browse/HBASE-21728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21728: -- Fix Version/s: 2.0.5 2.1.3 2.2.0 > Backport to branch-2 (and maybe branch-1) HBASE-21684 Throw DNRIOE when > connection or rpc client is closed > -- > > Key: HBASE-21728 > URL: https://issues.apache.org/jira/browse/HBASE-21728 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Priority: Major > Fix For: 2.2.0, 2.1.3, 2.0.5 > > > Backport the parent. May need a few changes. See suggestions in parent. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21595) Print thread's information and stack traces when RS is aborting forcibly
[ https://issues.apache.org/jira/browse/HBASE-21595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21595: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.2.0 Release Note: Does thread dump on stdout on abort. Status: Resolved (was: Patch Available) Pushed to branch-2 and master branch. It didn't go back to 2.1. If you put up a patch for branch-2.1 I'll push it [~pankaj2461]. Thanks. > Print thread's information and stack traces when RS is aborting forcibly > > > Key: HBASE-21595 > URL: https://issues.apache.org/jira/browse/HBASE-21595 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 3.0.0, 2.2.0 >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Minor > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21595.patch > > > After HBASE-21325 RS terminate forcibly on abort timeout. > We should print the thread info before terminating, will be useful to analyze > the RS abort timeout problem. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
