[jira] [Commented] (HBASE-21671) Rewrite RegionReplicaReplicationEndpoint to use AsyncClusterConnection
[ https://issues.apache.org/jira/browse/HBASE-21671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740134#comment-16740134 ] Jingyun Tian commented on HBASE-21671: -- Yes.. It's ok now > Rewrite RegionReplicaReplicationEndpoint to use AsyncClusterConnection > -- > > Key: HBASE-21671 > URL: https://issues.apache.org/jira/browse/HBASE-21671 > Project: HBase > Issue Type: Sub-task > Components: read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-21671-HBASE-21512-v1.patch, > HBASE-21671-HBASE-21512-v2.patch, HBASE-21671-HBASE-21512.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sakthi updated HBASE-21225:
---
Attachment: hbase-21225.master.004.patch
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch,
> hbase-21225.master.004.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21671) Rewrite RegionReplicaReplicationEndpoint to use AsyncClusterConnection
[ https://issues.apache.org/jira/browse/HBASE-21671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740129#comment-16740129 ] Duo Zhang commented on HBASE-21671: --- You forgot to push the publish button? [~tianjingyun] > Rewrite RegionReplicaReplicationEndpoint to use AsyncClusterConnection > -- > > Key: HBASE-21671 > URL: https://issues.apache.org/jira/browse/HBASE-21671 > Project: HBase > Issue Type: Sub-task > Components: read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-21671-HBASE-21512-v1.patch, > HBASE-21671-HBASE-21512-v2.patch, HBASE-21671-HBASE-21512.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19695) Handle disabled table for async client
[ https://issues.apache.org/jira/browse/HBASE-19695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19695: -- Assignee: Duo Zhang (was: Jingyun Tian) Fix Version/s: 2.0.5 2.1.3 2.2.0 3.0.0 Status: Patch Available (was: Open) > Handle disabled table for async client > -- > > Key: HBASE-19695 > URL: https://issues.apache.org/jira/browse/HBASE-19695 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5 > > Attachments: HBASE-19695-v1.patch, HBASE-19695.master.001.patch, > HBASE-19695.master.002.patch, HBASE-19695.patch > > > Now for async client we will not check if a table is disabled when retrying, > so we will retry until the time or count limit is reached, and will not throw > a TableNotEnabledException. > We should have the same behavior as sync client, but the implementation > should be carefully designed. For sync client, we will also check if a table > is disabled if it is a retry, no matter what the exception is. This will > double the pressure on meta table. We should try our best to eliminate > unnecessary access to meta table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21679) Port HBASE-6028 (Start/Stop compactions at region server level) to branch-1
[
https://issues.apache.org/jira/browse/HBASE-21679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740120#comment-16740120
]
Hadoop QA commented on HBASE-21679:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 20m
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 5m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
28s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
49s{color} | {color:green} branch-1 passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
39s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 10m
33s{color} | {color:green} branch-1 passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 3m
31s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
3s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
50s{color} | {color:green} branch-1 passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
7s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
44s{color} | {color:green} the patch passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} the patch passed with JDK v1.7.0_201 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 45s{color}
| {color:red} root-jdk1.7.0_201 with JDK v1.7.0_201 generated 2 new + 23
unchanged - 2 fixed = 25 total (was 25) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
37s{color} | {color:red} hbase-server: The patch generated 3 new + 563
unchanged - 9 fixed = 566 total (was 572) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 5m
4s{color} | {color:red} root: The patch generated 3 new + 775 unchanged - 9
fixed = 778 total (was 784) {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 7 new + 921 unchanged - 3 fixed =
928 total (was 924) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 4s{color} | {color:orange} The patch generated 8 new + 667 unchanged - 0
fixed = 675 total (was 667) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} xml {color} | {color:red} 0m 0s{color} |
{color:red} The patch has 1
[jira] [Commented] (HBASE-21620) Problem in scan query when using more than one column prefix filter in some cases.
[
https://issues.apache.org/jira/browse/HBASE-21620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740116#comment-16740116
]
Karthick commented on HBASE-21620:
--
[~openinx] we removed the *(comparator.compare(nextKV, cell) > 0)* condition
and tested, but still the scan is slow.
> Problem in scan query when using more than one column prefix filter in some
> cases.
> --
>
> Key: HBASE-21620
> URL: https://issues.apache.org/jira/browse/HBASE-21620
> Project: HBase
> Issue Type: Bug
> Components: scan
>Affects Versions: 3.0.0, 1.5.0, 2.2.0, 1.4.8, 2.1.2, 2.0.4
> Environment: hbase-1.4.8, hbase-1.4.9
> hadoop-2.7.3
>Reporter: Mohamed Mohideen Meeran
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.2, 2.0.4
>
> Attachments: HBASE-21620.branch-1.patch, HBASE-21620.v1.patch,
> HBASE-21620.v2.patch, HBASE-21620.v3.patch, HBASE-21620.v3.patch,
> HBaseImportData.java, columnkey.txt, file.txt, test.patch
>
>
> In some cases, unable to get the scan results when using more than one column
> prefix filter.
> Attached a java file to import the data which we used and a text file
> containing the values..
> While executing the following query (hbase shell as well as java program) it
> is waiting indefinitely and after RPC timeout we got the following error..
> Also we noticed high cpu, high load average and very frequent young gc in
> the region server containing this row...
> scan 'namespace:tablename',\{STARTROW => 'test',ENDROW => 'test', FILTER =>
> "ColumnPrefixFilter('1544770422942010001_') OR
> ColumnPrefixFilter('1544769883529010001_')"}
> ROW COLUMN+CELL
> ERROR: Call id=18,
> waitTime=60005, rpcTimetout=6
>
> Note: Table scan operation and scan with a single column prefix filter works
> fine in this case.
> When we check the same query in hbase-1.2.5 it is working fine.
> Can you please help me on this..
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21671) Rewrite RegionReplicaReplicationEndpoint to use AsyncClusterConnection
[
https://issues.apache.org/jira/browse/HBASE-21671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740115#comment-16740115
]
Hadoop QA commented on HBASE-21671:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} HBASE-21512 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
7s{color} | {color:green} HBASE-21512 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
59s{color} | {color:green} HBASE-21512 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
54s{color} | {color:green} HBASE-21512 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
42s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
34s{color} | {color:green} HBASE-21512 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} HBASE-21512 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
43s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 40s{color}
| {color:red} hbase-client generated 1 new + 99 unchanged - 1 fixed = 100 total
(was 100) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} The patch passed checkstyle in hbase-client {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
13s{color} | {color:green} hbase-server: The patch generated 0 new + 7
unchanged - 6 fixed = 7 total (was 13) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
34s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 46s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
4s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}138m 56s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
45s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}193m 10s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.replication.TestSyncReplicationActive |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21671 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954523/HBASE-21671-HBASE-21512-v2.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 42b5f4020fcb
[jira] [Commented] (HBASE-21671) Rewrite RegionReplicaReplicationEndpoint to use AsyncClusterConnection
[ https://issues.apache.org/jira/browse/HBASE-21671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740113#comment-16740113 ] Jingyun Tian commented on HBASE-21671: -- +1 except a remark on RB. > Rewrite RegionReplicaReplicationEndpoint to use AsyncClusterConnection > -- > > Key: HBASE-21671 > URL: https://issues.apache.org/jira/browse/HBASE-21671 > Project: HBase > Issue Type: Sub-task > Components: read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-21671-HBASE-21512-v1.patch, > HBASE-21671-HBASE-21512-v2.patch, HBASE-21671-HBASE-21512.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19695) Handle disabled table for async client
[ https://issues.apache.org/jira/browse/HBASE-19695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740095#comment-16740095 ] Duo Zhang commented on HBASE-19695: --- Let's split this to two issues, as the one for batch operation will be much complicated. > Handle disabled table for async client > -- > > Key: HBASE-19695 > URL: https://issues.apache.org/jira/browse/HBASE-19695 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Jingyun Tian >Priority: Major > Attachments: HBASE-19695.master.001.patch, > HBASE-19695.master.002.patch, HBASE-19695.patch > > > Now for async client we will not check if a table is disabled when retrying, > so we will retry until the time or count limit is reached, and will not throw > a TableNotEnabledException. > We should have the same behavior as sync client, but the implementation > should be carefully designed. For sync client, we will also check if a table > is disabled if it is a retry, no matter what the exception is. This will > double the pressure on meta table. We should try our best to eliminate > unnecessary access to meta table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21663) Add replica scan support
[ https://issues.apache.org/jira/browse/HBASE-21663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21663: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to branch-2.0+. Thanks [~openinx] for reviewing. > Add replica scan support > > > Key: HBASE-21663 > URL: https://issues.apache.org/jira/browse/HBASE-21663 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client, read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5 > > Attachments: HBASE-21663-v1.patch, HBASE-21663.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740079#comment-16740079
]
Sakthi commented on HBASE-21225:
Thanks for your detailed review [~elserj]. One, I hadn't thought of backward
compatibility issues. Two, the reason I was trying to remove the "remove"
attribute was, I saw that throttle quota's implementation didn't seem to have a
"remove" attribute and wanted to keep both the implementations as similar as
possible. But like you mentioned, the removal would require too much of work
(for little to no gain).
{quote}I'm pretty sure this is what the intention was. If REMOVE was set, the
merged quota object should rip it out when constructing the "merged" view of
all quotas for the entity (table or ns).
{quote}
Will update the patch for this.
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21282) Upgrade Jetty dependencies to latest in major-line
[
https://issues.apache.org/jira/browse/HBASE-21282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740076#comment-16740076
]
Hudson commented on HBASE-21282:
Results for branch branch-2.1
[build #760 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/760/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/760//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/760//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/760//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Upgrade Jetty dependencies to latest in major-line
> --
>
> Key: HBASE-21282
> URL: https://issues.apache.org/jira/browse/HBASE-21282
> Project: HBase
> Issue Type: Task
> Components: dependencies
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21282.001.branch-2.0.patch
>
>
> Looks like we have dependencies on both jetty 9.2 and 9.3, but we're lagging
> pretty far behind in both. We can upgrade both of these to the latest (august
> 2018).
>
> I'll also have to take a look at why we're using two separate versions (maybe
> we didn't want to switch from jetty-jsp to apache-jsp on 9.2->9.3?). Not sure
> if there's a good reason for this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21663) Add replica scan support
[ https://issues.apache.org/jira/browse/HBASE-21663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740075#comment-16740075 ] Duo Zhang commented on HBASE-21663: --- Will commit after fixing the checkstyle issues. > Add replica scan support > > > Key: HBASE-21663 > URL: https://issues.apache.org/jira/browse/HBASE-21663 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client, read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5 > > Attachments: HBASE-21663-v1.patch, HBASE-21663.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21663) Add replica scan support
[
https://issues.apache.org/jira/browse/HBASE-21663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740065#comment-16740065
]
Hadoop QA commented on HBASE-21663:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
8s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
31s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 38s{color}
| {color:red} hbase-client generated 3 new + 97 unchanged - 3 fixed = 100 total
(was 100) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
30s{color} | {color:red} hbase-client: The patch generated 2 new + 10 unchanged
- 16 fixed = 12 total (was 26) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
6s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 39s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
20s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}129m
48s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
52s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}177m 9s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21663 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954514/HBASE-21663-v1.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux d648889161f8 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 5d32e80f9e |
| maven | version: Apache Maven 3.5.4
[jira] [Commented] (HBASE-21374) Backport HBASE-21342 to branch-1
[
https://issues.apache.org/jira/browse/HBASE-21374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740059#comment-16740059
]
Hudson commented on HBASE-21374:
Results for branch branch-1
[build #627 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/627/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/627//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/627//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/627//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Backport HBASE-21342 to branch-1
>
>
> Key: HBASE-21374
> URL: https://issues.apache.org/jira/browse/HBASE-21374
> Project: HBase
> Issue Type: Sub-task
>Reporter: Mike Drob
>Assignee: mazhenlin
>Priority: Major
> Fix For: 1.5.0, 1.4.10, 1.3.4
>
> Attachments: HBASE-21374.branch-1.001.patch,
> HBASE-21374.branch-1.002.patch, HBASE-21374.branch-1.003.patch,
> HBASE-21374.branch-1.003.patch, HBASE-21374.branch-1.003.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21342) FileSystem in use may get closed by other bulk load call in secure bulkLoad
[
https://issues.apache.org/jira/browse/HBASE-21342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740060#comment-16740060
]
Hudson commented on HBASE-21342:
Results for branch branch-1
[build #627 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/627/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/627//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/627//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/627//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> FileSystem in use may get closed by other bulk load call in secure bulkLoad
>
>
> Key: HBASE-21342
> URL: https://issues.apache.org/jira/browse/HBASE-21342
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 1.4.4, 2.0.1, 1.2.7
>Reporter: mazhenlin
>Assignee: mazhenlin
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: 21342.v1.txt, HBASE-21342.002.patch,
> HBASE-21342.003.patch, HBASE-21342.004.patch, HBASE-21342.005.patch,
> HBASE-21342.006.patch, HBASE-21342.007.patch, race.patch
>
>
> As mentioned in [HBASE-15291|#HBASE-15291], there is a race condition. If
> Two secure bulkload calls from the same UGI into two different regions and
> one region finishes earlier, it will close the bulk load fs, and the other
> region will fail.
>
> Another case would be more serious. The FileSystem.close() function needs two
> synchronized variables : CACHE and deleteOnExit. If one region calls
> FileSystem.closeAllForUGI ( in SecureBulkLoadManager.cleanupBulkLoad) while
> another region is trying to close srcFS ( in
> SecureBulkLoadListener.closeSrcFs) , can cause deadlock here.
>
> I have wrote a UT for this and fixed it using reference counter.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21374) Backport HBASE-21342 to branch-1
[
https://issues.apache.org/jira/browse/HBASE-21374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740050#comment-16740050
]
Hudson commented on HBASE-21374:
Results for branch branch-1.4
[build #623 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/623/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/623//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/623//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/623//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Backport HBASE-21342 to branch-1
>
>
> Key: HBASE-21374
> URL: https://issues.apache.org/jira/browse/HBASE-21374
> Project: HBase
> Issue Type: Sub-task
>Reporter: Mike Drob
>Assignee: mazhenlin
>Priority: Major
> Fix For: 1.5.0, 1.4.10, 1.3.4
>
> Attachments: HBASE-21374.branch-1.001.patch,
> HBASE-21374.branch-1.002.patch, HBASE-21374.branch-1.003.patch,
> HBASE-21374.branch-1.003.patch, HBASE-21374.branch-1.003.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21342) FileSystem in use may get closed by other bulk load call in secure bulkLoad
[
https://issues.apache.org/jira/browse/HBASE-21342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740051#comment-16740051
]
Hudson commented on HBASE-21342:
Results for branch branch-1.4
[build #623 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/623/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/623//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/623//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/623//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> FileSystem in use may get closed by other bulk load call in secure bulkLoad
>
>
> Key: HBASE-21342
> URL: https://issues.apache.org/jira/browse/HBASE-21342
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 1.4.4, 2.0.1, 1.2.7
>Reporter: mazhenlin
>Assignee: mazhenlin
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: 21342.v1.txt, HBASE-21342.002.patch,
> HBASE-21342.003.patch, HBASE-21342.004.patch, HBASE-21342.005.patch,
> HBASE-21342.006.patch, HBASE-21342.007.patch, race.patch
>
>
> As mentioned in [HBASE-15291|#HBASE-15291], there is a race condition. If
> Two secure bulkload calls from the same UGI into two different regions and
> one region finishes earlier, it will close the bulk load fs, and the other
> region will fail.
>
> Another case would be more serious. The FileSystem.close() function needs two
> synchronized variables : CACHE and deleteOnExit. If one region calls
> FileSystem.closeAllForUGI ( in SecureBulkLoadManager.cleanupBulkLoad) while
> another region is trying to close srcFS ( in
> SecureBulkLoadListener.closeSrcFs) , can cause deadlock here.
>
> I have wrote a UT for this and fixed it using reference counter.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21374) Backport HBASE-21342 to branch-1
[
https://issues.apache.org/jira/browse/HBASE-21374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740035#comment-16740035
]
Hudson commented on HBASE-21374:
Results for branch branch-1.3
[build #606 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/606/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/606//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/606//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/606//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Backport HBASE-21342 to branch-1
>
>
> Key: HBASE-21374
> URL: https://issues.apache.org/jira/browse/HBASE-21374
> Project: HBase
> Issue Type: Sub-task
>Reporter: Mike Drob
>Assignee: mazhenlin
>Priority: Major
> Fix For: 1.5.0, 1.4.10, 1.3.4
>
> Attachments: HBASE-21374.branch-1.001.patch,
> HBASE-21374.branch-1.002.patch, HBASE-21374.branch-1.003.patch,
> HBASE-21374.branch-1.003.patch, HBASE-21374.branch-1.003.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21342) FileSystem in use may get closed by other bulk load call in secure bulkLoad
[
https://issues.apache.org/jira/browse/HBASE-21342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740036#comment-16740036
]
Hudson commented on HBASE-21342:
Results for branch branch-1.3
[build #606 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/606/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/606//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/606//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/606//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> FileSystem in use may get closed by other bulk load call in secure bulkLoad
>
>
> Key: HBASE-21342
> URL: https://issues.apache.org/jira/browse/HBASE-21342
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 1.4.4, 2.0.1, 1.2.7
>Reporter: mazhenlin
>Assignee: mazhenlin
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: 21342.v1.txt, HBASE-21342.002.patch,
> HBASE-21342.003.patch, HBASE-21342.004.patch, HBASE-21342.005.patch,
> HBASE-21342.006.patch, HBASE-21342.007.patch, race.patch
>
>
> As mentioned in [HBASE-15291|#HBASE-15291], there is a race condition. If
> Two secure bulkload calls from the same UGI into two different regions and
> one region finishes earlier, it will close the bulk load fs, and the other
> region will fail.
>
> Another case would be more serious. The FileSystem.close() function needs two
> synchronized variables : CACHE and deleteOnExit. If one region calls
> FileSystem.closeAllForUGI ( in SecureBulkLoadManager.cleanupBulkLoad) while
> another region is trying to close srcFS ( in
> SecureBulkLoadListener.closeSrcFs) , can cause deadlock here.
>
> I have wrote a UT for this and fixed it using reference counter.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21663) Add replica scan support
[ https://issues.apache.org/jira/browse/HBASE-21663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740017#comment-16740017 ] Zheng Hu commented on HBASE-21663: -- +1 for the latest patch in rb. > Add replica scan support > > > Key: HBASE-21663 > URL: https://issues.apache.org/jira/browse/HBASE-21663 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client, read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5 > > Attachments: HBASE-21663-v1.patch, HBASE-21663.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21671) Rewrite RegionReplicaReplicationEndpoint to use AsyncClusterConnection
[ https://issues.apache.org/jira/browse/HBASE-21671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21671: -- Attachment: HBASE-21671-HBASE-21512-v2.patch > Rewrite RegionReplicaReplicationEndpoint to use AsyncClusterConnection > -- > > Key: HBASE-21671 > URL: https://issues.apache.org/jira/browse/HBASE-21671 > Project: HBase > Issue Type: Sub-task > Components: read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-21671-HBASE-21512-v1.patch, > HBASE-21671-HBASE-21512-v2.patch, HBASE-21671-HBASE-21512.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21702) Transcribe "what is HBaseCon" into book
[ https://issues.apache.org/jira/browse/HBASE-21702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740005#comment-16740005 ] Sean Busbey commented on HBASE-21702: - I know you just wanted to transcribe the existing thing here, but I think it would really benefit from adding a sentence to the intro that essentially says "folks who are interested in hosting a future HBaseCon event should keep the following guidelines in mind and email the PMC via the mailing list priv...@hbase.apache.org, which allows anyone to send messages, but restricts who can read them" > Transcribe "what is HBaseCon" into book > --- > > Key: HBASE-21702 > URL: https://issues.apache.org/jira/browse/HBASE-21702 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Attachments: HBASE-21702.001.patch > > > Take the document I wrote on google docs > (https://docs.google.com/document/d/15ZmgNKSzyR_UCzOmHEACnGZp-vuh-QlCYfr6bCmQ-4w/edit) > and get it into the HBase book. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21580) Support getting Hbck instance from AsyncConnection
[ https://issues.apache.org/jira/browse/HBASE-21580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21580: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to branch-2.0+. Thanks [~tianjingyun] for reviewing. > Support getting Hbck instance from AsyncConnection > -- > > Key: HBASE-21580 > URL: https://issues.apache.org/jira/browse/HBASE-21580 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client, hbck2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5 > > Attachments: HBASE-21580.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21702) Transcribe "what is HBaseCon" into book
[
https://issues.apache.org/jira/browse/HBASE-21702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739996#comment-16739996
]
Hadoop QA commented on HBASE-21702:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
52s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 5m
41s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 5m
44s{color} | {color:blue} patch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 49s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21702 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954519/HBASE-21702.001.patch
|
| Optional Tests | dupname asflicense refguide |
| uname | Linux aec9f5310774 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 5d32e80f9e |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| refguide |
https://builds.apache.org/job/PreCommit-HBASE-Build/15537/artifact/patchprocess/branch-site/book.html
|
| refguide |
https://builds.apache.org/job/PreCommit-HBASE-Build/15537/artifact/patchprocess/patch-site/book.html
|
| Max. process+thread count | 87 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15537/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Transcribe "what is HBaseCon" into book
> ---
>
> Key: HBASE-21702
> URL: https://issues.apache.org/jira/browse/HBASE-21702
> Project: HBase
> Issue Type: Task
> Components: documentation
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Attachments: HBASE-21702.001.patch
>
>
> Take the document I wrote on google docs
> (https://docs.google.com/document/d/15ZmgNKSzyR_UCzOmHEACnGZp-vuh-QlCYfr6bCmQ-4w/edit)
> and get it into the HBase book.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21580) Support getting Hbck instance from AsyncConnection
[ https://issues.apache.org/jira/browse/HBASE-21580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739988#comment-16739988 ] Jingyun Tian commented on HBASE-21580: -- +1 for this patch. > Support getting Hbck instance from AsyncConnection > -- > > Key: HBASE-21580 > URL: https://issues.apache.org/jira/browse/HBASE-21580 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client, hbck2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5 > > Attachments: HBASE-21580.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21702) Transcribe "what is HBaseCon" into book
[ https://issues.apache.org/jira/browse/HBASE-21702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-21702: --- Attachment: HBASE-21702.001.patch > Transcribe "what is HBaseCon" into book > --- > > Key: HBASE-21702 > URL: https://issues.apache.org/jira/browse/HBASE-21702 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Attachments: HBASE-21702.001.patch > > > Take the document I wrote on google docs > (https://docs.google.com/document/d/15ZmgNKSzyR_UCzOmHEACnGZp-vuh-QlCYfr6bCmQ-4w/edit) > and get it into the HBase book. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21702) Transcribe "what is HBaseCon" into book
[ https://issues.apache.org/jira/browse/HBASE-21702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-21702: --- Status: Patch Available (was: Open) > Transcribe "what is HBaseCon" into book > --- > > Key: HBASE-21702 > URL: https://issues.apache.org/jira/browse/HBASE-21702 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Attachments: HBASE-21702.001.patch > > > Take the document I wrote on google docs > (https://docs.google.com/document/d/15ZmgNKSzyR_UCzOmHEACnGZp-vuh-QlCYfr6bCmQ-4w/edit) > and get it into the HBase book. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21374) Backport HBASE-21342 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739985#comment-16739985 ] Pankaj Kumar commented on HBASE-21374: -- Thank you [~apurtell]... > Backport HBASE-21342 to branch-1 > > > Key: HBASE-21374 > URL: https://issues.apache.org/jira/browse/HBASE-21374 > Project: HBase > Issue Type: Sub-task >Reporter: Mike Drob >Assignee: mazhenlin >Priority: Major > Fix For: 1.5.0, 1.4.10, 1.3.4 > > Attachments: HBASE-21374.branch-1.001.patch, > HBASE-21374.branch-1.002.patch, HBASE-21374.branch-1.003.patch, > HBASE-21374.branch-1.003.patch, HBASE-21374.branch-1.003.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21679) Port HBASE-6028 (Start/Stop compactions at region server level) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21679: --- Status: Patch Available (was: Open) > Port HBASE-6028 (Start/Stop compactions at region server level) to branch-1 > --- > > Key: HBASE-21679 > URL: https://issues.apache.org/jira/browse/HBASE-21679 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21679-branch-1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21679) Port HBASE-6028 (Start/Stop compactions at region server level) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739978#comment-16739978 ] Andrew Purtell commented on HBASE-21679: [~lhofhansl] Request for review > Port HBASE-6028 (Start/Stop compactions at region server level) to branch-1 > --- > > Key: HBASE-21679 > URL: https://issues.apache.org/jira/browse/HBASE-21679 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21679-branch-1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21679) Port HBASE-6028 (Start/Stop compactions at region server level) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21679: --- Attachment: HBASE-21679-branch-1.patch > Port HBASE-6028 (Start/Stop compactions at region server level) to branch-1 > --- > > Key: HBASE-21679 > URL: https://issues.apache.org/jira/browse/HBASE-21679 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21679-branch-1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739976#comment-16739976
]
Josh Elser commented on HBASE-21225:
bq. Have uploaded a patch with "remove" attribute from SpaceQuota protobuf
removed
Let me also clarify: you can try to do this, but it's going to be tricky to
unwind correctly.
* You must _never_ remove fields from a protobuf message. This causes things to
really break down
https://developers.google.com/protocol-buffers/docs/proto#reserved
* You would need to add something to MasterQuotaManager to "fix" the
hbase:quota table that might have a REMOVE in it already
* You would still need special handling in the Master to handle older clients
that do not have the change that you're proposing in this Jira issue.
Perhaps I'm missing some fundamental reason why you want to remove this
attribute instead of make it work, but, for now, I'll assume it's just that
hadn't thought about the backwards compatibility issues yet :)
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21663) Add replica scan support
[ https://issues.apache.org/jira/browse/HBASE-21663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21663: -- Attachment: HBASE-21663-v1.patch > Add replica scan support > > > Key: HBASE-21663 > URL: https://issues.apache.org/jira/browse/HBASE-21663 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client, read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5 > > Attachments: HBASE-21663-v1.patch, HBASE-21663.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21342) FileSystem in use may get closed by other bulk load call in secure bulkLoad
[ https://issues.apache.org/jira/browse/HBASE-21342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739973#comment-16739973 ] Hudson commented on HBASE-21342: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #516 (See [https://builds.apache.org/job/HBase-1.3-IT/516/]) HBASE-21374 Backport HBASE-21342 to branch-1 (apurtell: [https://github.com/apache/hbase/commit/25d0c3ec7f80f51e814ed53caa9509fe60a4a8cf]) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestSecureBulkLoadEndpoint.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java > FileSystem in use may get closed by other bulk load call in secure bulkLoad > > > Key: HBASE-21342 > URL: https://issues.apache.org/jira/browse/HBASE-21342 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 1.4.4, 2.0.1, 1.2.7 >Reporter: mazhenlin >Assignee: mazhenlin >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3 > > Attachments: 21342.v1.txt, HBASE-21342.002.patch, > HBASE-21342.003.patch, HBASE-21342.004.patch, HBASE-21342.005.patch, > HBASE-21342.006.patch, HBASE-21342.007.patch, race.patch > > > As mentioned in [HBASE-15291|#HBASE-15291], there is a race condition. If > Two secure bulkload calls from the same UGI into two different regions and > one region finishes earlier, it will close the bulk load fs, and the other > region will fail. > > Another case would be more serious. The FileSystem.close() function needs two > synchronized variables : CACHE and deleteOnExit. If one region calls > FileSystem.closeAllForUGI ( in SecureBulkLoadManager.cleanupBulkLoad) while > another region is trying to close srcFS ( in > SecureBulkLoadListener.closeSrcFs) , can cause deadlock here. > > I have wrote a UT for this and fixed it using reference counter. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21374) Backport HBASE-21342 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739972#comment-16739972 ] Hudson commented on HBASE-21374: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #516 (See [https://builds.apache.org/job/HBase-1.3-IT/516/]) HBASE-21374 Backport HBASE-21342 to branch-1 (apurtell: [https://github.com/apache/hbase/commit/25d0c3ec7f80f51e814ed53caa9509fe60a4a8cf]) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestSecureBulkLoadEndpoint.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java > Backport HBASE-21342 to branch-1 > > > Key: HBASE-21374 > URL: https://issues.apache.org/jira/browse/HBASE-21374 > Project: HBase > Issue Type: Sub-task >Reporter: Mike Drob >Assignee: mazhenlin >Priority: Major > Fix For: 1.5.0, 1.4.10, 1.3.4 > > Attachments: HBASE-21374.branch-1.001.patch, > HBASE-21374.branch-1.002.patch, HBASE-21374.branch-1.003.patch, > HBASE-21374.branch-1.003.patch, HBASE-21374.branch-1.003.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21225) Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
[
https://issues.apache.org/jira/browse/HBASE-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739967#comment-16739967
]
Josh Elser commented on HBASE-21225:
bq. Have uploaded a patch with "remove" attribute from SpaceQuota protobuf
removed. Waiting for the unit tests reports from 'Hadoop QA'.
I still believe this is the wrong approach. If you start removing things from
the protobuf, you're going to break backwards compatibility and have to write
an upgrade step for serialized data that might have this attribute (albeit,
unwanted).
bq. As the "mergedQuota" is of type GlobalQuotaSettingsImpl and it's
constructor takes in Throttle.Builder & SpaceQuota.Builder as parameter, we can
pass in null values to both these parameters as and when required (whenever
"remove" is set?)
I'm pretty sure this is what the intention was. If {{REMOVE}} was set, the
merged quota object should rip it out when constructing the "merged" view of
all quotas for the entity (table or ns).
> Having RPC & Space quota on a table/Namespace doesn't allow space quota to be
> removed using 'NONE'
> --
>
> Key: HBASE-21225
> URL: https://issues.apache.org/jira/browse/HBASE-21225
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21225.master.001.patch,
> hbase-21225.master.002.patch, hbase-21225.master.003.patch
>
>
> A part of HBASE-20705 is still unresolved. In that Jira it was assumed that
> problem is: when table having both rpc & space quotas is dropped (with
> hbase.quota.remove.on.table.delete set as true), the rpc quota is not set to
> be dropped along with table-drops, and space quota was not being able to be
> removed completely because of the "EMPTY" row that rpc quota left even after
> removing.
> The proposed solution for that was to make sure that rpc quota didn't leave
> empty rows after removal of quota. And setting automatic removal of rpc quota
> with table drops. That made sure that space quotas can be recreated/removed.
> But all this was under the assumption that hbase.quota.remove.on.table.delete
> is set as true. When it is set as false, the same issue can reproduced. Also
> the below shown steps can used to reproduce the issue without table-drops.
> {noformat}
> hbase(main):005:0> create 't2','cf'
> Created table t2
> Took 0.7619 seconds
> => Hbase::Table - t2
> hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT =>
> '10M/sec'
> Took 0.0514 seconds
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0162 seconds
> hbase(main):008:0> list_quotas
> OWNER QUOTAS
> TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE,
> LIMIT => 10M/sec, SCOPE =>
>MACHINE
> TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824,
> VIOLATION_POLICY => NO_WRIT
>ES
> 2 row(s)
> Took 0.0716 seconds
> hbase(main):009:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => NONE
> Took 0.0082 seconds
> hbase(main):010:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0254 seconds
> hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G',
> POLICY => NO_WRITES
> Took 0.0082 seconds
> hbase(main):012:0> list_quotas
> OWNER QUOTAS
> TABLE => t2TYPE => THROTTLE, THROTTLE_TYPE =>
> REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE
> TABLE => t2TYPE => SPACE, TABLE => t2, REMOVE => true
> 2 row(s)
> Took 0.0411 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21296) [2.1] Upgrade Jetty dependencies to latest in major-line
[ https://issues.apache.org/jira/browse/HBASE-21296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-21296: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) > [2.1] Upgrade Jetty dependencies to latest in major-line > > > Key: HBASE-21296 > URL: https://issues.apache.org/jira/browse/HBASE-21296 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 2.1.3 > > Attachments: HBASE-21282.001.branch-2.0.patch, > HBASE-21296.branch-2.1.patch > > > Looks like we have dependencies on both jetty 9.2 and 9.3, but we're lagging > pretty far behind in both. We can upgrade both of these to the latest (august > 2018). > > I'll also have to take a look at why we're using two separate versions (maybe > we didn't want to switch from jetty-jsp to apache-jsp on 9.2->9.3?). Not sure > if there's a good reason for this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739895#comment-16739895
]
Hadoop QA commented on HBASE-20662:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
34s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} hbase-client: The patch generated 0 new + 5
unchanged - 1 fixed = 5 total (was 6) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
13s{color} | {color:green} hbase-server: The patch generated 0 new + 150
unchanged - 4 fixed = 150 total (was 154) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
29s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 21s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
22s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}137m
24s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
45s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}190m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20662 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954485/HBASE-20662.master.006.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 163c575a1150 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Updated] (HBASE-21374) Backport HBASE-21342 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21374: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 1.3.4 1.4.10 Status: Resolved (was: Patch Available) > Backport HBASE-21342 to branch-1 > > > Key: HBASE-21374 > URL: https://issues.apache.org/jira/browse/HBASE-21374 > Project: HBase > Issue Type: Sub-task >Reporter: Mike Drob >Assignee: mazhenlin >Priority: Major > Fix For: 1.5.0, 1.4.10, 1.3.4 > > Attachments: HBASE-21374.branch-1.001.patch, > HBASE-21374.branch-1.002.patch, HBASE-21374.branch-1.003.patch, > HBASE-21374.branch-1.003.patch, HBASE-21374.branch-1.003.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21374) Backport HBASE-21342 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739873#comment-16739873 ] Andrew Purtell commented on HBASE-21374: Looks good, committing after some local checks > Backport HBASE-21342 to branch-1 > > > Key: HBASE-21374 > URL: https://issues.apache.org/jira/browse/HBASE-21374 > Project: HBase > Issue Type: Sub-task >Reporter: Mike Drob >Assignee: mazhenlin >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21374.branch-1.001.patch, > HBASE-21374.branch-1.002.patch, HBASE-21374.branch-1.003.patch, > HBASE-21374.branch-1.003.patch, HBASE-21374.branch-1.003.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21689) Make table/namespace specific current quota info available in shell(describe_namespace & describe)
[
https://issues.apache.org/jira/browse/HBASE-21689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sakthi updated HBASE-21689:
---
Description:
describe_namespace & describe command in shell should show current quota(if
present) on the particular table/namespace.
{noformat}
// Namespace
hbase(main):002:0> create_namespace 'ns1',
{'hbase.namespace.quota.maxtables'=>'5'}
Took 0.1703 seconds
hbase(main):004:0> set_quota TYPE => SPACE, NAMESPACE => 'ns1', LIMIT => '50T',
POLICY => NO_WRITES
Took 0.0644 seconds
hbase(main):007:0> set_quota TYPE => THROTTLE, NAMESPACE => 'ns1', LIMIT =>
'10m/sec'
Took 0.0271 seconds
hbase(main):005:0> describe_namespace 'ns1'
DESCRIPTION
{NAME => 'ns1', hbase.namespace.quota.maxtables => '5'}
// Table
hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't1', LIMIT => '1G',
POLICY => NO_INSERTS
Took 0.0155 seconds
hbase(main):009:0> set_quota TYPE => THROTTLE, TABLE => 't1', LIMIT =>
'10T/hour'
Took 0.0309 seconds
hbase(main):008:0> describe 't1'
Table t1 is ENABLED
t1
COLUMN FAMILIES DESCRIPTION
{NAME => 'cf', VERSIONS => '1', EVICT_BLOCKS_ON_CLOSE => 'false',
NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS
=> 'FALSE', CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL
=> 'FOREVER', MIN_VERSIONS => '0', REPL
ICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE => 'false',
IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRI
TE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE',
BLOCKCACHE => 'true', BLOCKSIZE => '65536'}
1 row(s)
Took 0.0341 seconds
{noformat}
was:
describe_namespace & describe command in shell should show current quota(if
present) on the particular table/namespace.
{noformat}
// Namespace
hbase(main):002:0> create_namespace 'ns1',
{'hbase.namespace.quota.maxtables'=>'5'}
Took 0.1703 seconds
hbase(main):004:0> set_quota TYPE => SPACE, NAMESPACE => 'ns1', LIMIT => '50T',
POLICY => NO_WRITES
Took 0.0644 seconds
hbase(main):005:0> describe_namespace 'ns1'
DESCRIPTION
{NAME => 'ns1', hbase.namespace.quota.maxtables => '5'}
// Table
hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't1', LIMIT => '1G',
POLICY => NO_INSERTS
Took 0.0155 seconds
hbase(main):009:0> set_quota TYPE => THROTTLE, TABLE => 't1', LIMIT =>
'10T/hour'
Took 0.0309 seconds
hbase(main):008:0> describe 't1'
Table t1 is ENABLED
t1
COLUMN FAMILIES DESCRIPTION
{NAME => 'cf', VERSIONS => '1', EVICT_BLOCKS_ON_CLOSE => 'false',
NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS
=> 'FALSE', CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL
=> 'FOREVER', MIN_VERSIONS => '0', REPL
ICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE => 'false',
IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRI
TE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE',
BLOCKCACHE => 'true', BLOCKSIZE => '65536'}
1 row(s)
Took 0.0341 seconds
{noformat}
> Make table/namespace specific current quota info available in
> shell(describe_namespace & describe)
> --
>
> Key: HBASE-21689
> URL: https://issues.apache.org/jira/browse/HBASE-21689
> Project: HBase
> Issue Type: Improvement
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Minor
>
> describe_namespace & describe command in shell should show current quota(if
> present) on the particular table/namespace.
> {noformat}
> // Namespace
> hbase(main):002:0> create_namespace 'ns1',
> {'hbase.namespace.quota.maxtables'=>'5'}
> Took 0.1703 seconds
> hbase(main):004:0> set_quota TYPE => SPACE, NAMESPACE => 'ns1', LIMIT =>
> '50T', POLICY => NO_WRITES
> Took 0.0644 seconds
> hbase(main):007:0> set_quota TYPE => THROTTLE, NAMESPACE => 'ns1', LIMIT =>
> '10m/sec'
> Took 0.0271 seconds
> hbase(main):005:0> describe_namespace 'ns1'
> DESCRIPTION
> {NAME => 'ns1', hbase.namespace.quota.maxtables => '5'}
> // Table
> hbase(main):007:0> set_quota TYPE => SPACE, TABLE => 't1', LIMIT => '1G',
> POLICY => NO_INSERTS
> Took 0.0155 seconds
> hbase(main):009:0> set_quota TYPE => THROTTLE, TABLE => 't1', LIMIT =>
> '10T/hour'
> Took 0.0309 seconds
> hbase(main):008:0> describe 't1'
> Table t1 is ENABLED
> t1
> COLUMN FAMILIES DESCRIPTION
> {NAME => 'cf', VERSIONS => '1', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS
> => 'FALSE', CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE',
> TTL => 'FOREVER', MIN_VERSIONS => '0', REPL
> ICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE => 'false',
> IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRI
> TE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE',
> BLOCKCACHE => 'true', BLOCKSIZE => '65536'}
> 1 row(s)
> Took 0.0341 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21500) Jetty aliases parameter need to change as per jetty 9.x version
[ https://issues.apache.org/jira/browse/HBASE-21500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739800#comment-16739800 ] Nihal Jain commented on HBASE-21500: Ping [~elserj], [~stack] > Jetty aliases parameter need to change as per jetty 9.x version > --- > > Key: HBASE-21500 > URL: https://issues.apache.org/jira/browse/HBASE-21500 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 2.1.0, 2.0.0, 2.0.1, 2.1.1 >Reporter: Bhupendra Kumar Jain >Assignee: Nihal Jain >Priority: Minor > Fix For: 3.0.0 > > Attachments: HBASE-21500.master.001.patch > > > Noticed that Jetty aliases parameter in HttpServer.java > "*org.mortbay.jetty.servlet.Default.aliases*" is as per old jetty version and > need to change as per jetty 9.x new version after the HBASE-12894 > Refer > https://github.com/apache/hbase/blob/405bf5e6383a09f435baadbac6c389e9f6c43ac6/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java#L647 > It should be *"org.eclipse.jetty.servlet.Default.aliases"* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739779#comment-16739779
]
Nihal Jain edited comment on HBASE-20662 at 1/10/19 9:24 PM:
-
{quote}This makes me think of one case: what happens if a table is in violation
of a quota with an SVP of NO_WRITES. Then, an administrator changes the SVP
from NO_WRITES to DISABLE. Similarly, what about the case where an admin moves
from DISABLE to something else?
{quote}
Both scenarios fail with current logic with/without patch.
I have provided a fix and test for these scenarios too. See
[^HBASE-20662.master.006.patch]
- Fixes bug: Upon switching from SpaceViolationPolicy.DISABLE to any other
SVP, system does not enable table and enforce the new policy
** {{testSetQuotaFirstWithWithDisableNextNoWritesCompaction()}}
** {{testSetQuotaFirstWithWithDisableNextNoWrites()}}
** {{testSetQuotaFirstWithWithDisableNextNoInserts()}}
* Fixes bug: Switching from policy 1 to policy 2, where policy 1 != policy 2
** {{testSetQuotaFirstWithNoWritesNextWithDisable()}}
was (Author: nihaljain.cs):
{quote}This makes me think of one case: what happens if a table is in violation
of a quota with an SVP of NO_WRITES. Then, an administrator changes the SVP
from NO_WRITES to DISABLE. Similarly, what about the case where an admin moves
from DISABLE to something else?
{quote}
Both scenarios fail with current logic with/without patch. i have provided a
fix and test for these scenarios too. See [^HBASE-20662.master.006.patch]
- So this patch make Disable SVP different from others as now it won't take
action (i.e. enable/disable table) local to the RegionServer. In case of
violation, the appropriate action will be initiated by the master.
- Fixes bug: Increasing space quota on a violated table does not remove
SpaceViolationPolicy.DISABLE enforcement
- Fixes bug: Upon switching from SpaceViolationPolicy.DISABLE to any other
SVP, system does not enable table and enforce the new policy
Without the patch following new tests will fail:
- {{testSetQuotaAndThenIncreaseQuotaWithDisable()}}
- {{testSetQuotaAndThenDisableIncrEnableWithDisable()}}
- {{testSetQuotaFirstWithNoWritesNextWithDisable()}}
- {{testSetQuotaFirstWithWithDisableNextNoWritesCompaction()}}
- {{testSetQuotaFirstWithWithDisableNextNoWrites()}}
- {{testSetQuotaFirstWithWithDisableNextNoInserts()}}
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20662.master.001.patch,
> HBASE-20662.master.002.patch, HBASE-20662.master.003.patch,
> HBASE-20662.master.004.patch, HBASE-20662.master.004.patch,
> HBASE-20662.master.005.patch, HBASE-20662.master.006.patch
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService
[jira] [Commented] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739779#comment-16739779
]
Nihal Jain commented on HBASE-20662:
{quote}This makes me think of one case: what happens if a table is in violation
of a quota with an SVP of NO_WRITES. Then, an administrator changes the SVP
from NO_WRITES to DISABLE. Similarly, what about the case where an admin moves
from DISABLE to something else?
{quote}
Both scenarios fail with current logic with/without patch. i have provided a
fix and test for these scenarios too. See [^HBASE-20662.master.006.patch]
- So this patch make Disable SVP different from others as now it won't take
action (i.e. enable/disable table) local to the RegionServer. In case of
violation, the appropriate action will be initiated by the master.
- Fixes bug: Increasing space quota on a violated table does not remove
SpaceViolationPolicy.DISABLE enforcement
- Fixes bug: Upon switching from SpaceViolationPolicy.DISABLE to any other
SVP, system does not enable table and enforce the new policy
Without the patch following new tests will fail:
- {{testSetQuotaAndThenIncreaseQuotaWithDisable()}}
- {{testSetQuotaAndThenDisableIncrEnableWithDisable()}}
- {{testSetQuotaFirstWithNoWritesNextWithDisable()}}
- {{testSetQuotaFirstWithWithDisableNextNoWritesCompaction()}}
- {{testSetQuotaFirstWithWithDisableNextNoWrites()}}
- {{testSetQuotaFirstWithWithDisableNextNoInserts()}}
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20662.master.001.patch,
> HBASE-20662.master.002.patch, HBASE-20662.master.003.patch,
> HBASE-20662.master.004.patch, HBASE-20662.master.004.patch,
> HBASE-20662.master.005.patch, HBASE-20662.master.006.patch
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService methodName:
> EnableTable size: 104 connection: 127.0.0.1:38030 deadline: 1527707127568,
> exception=org.apache.hadoop.hbase.security.AccessDeniedException: Enabling
> the table 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to
> a violated space quota.
> 2018-05-31 00:34:27,571 ERROR [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> quotas.RegionServerSpaceQuotaManager(210): Failed to disable space violation
> policy for testSetQuotaAndThenIncreaseQuotaWithDisable0. This table will
> remain in violation.
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Enabling the table
> 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to a
> violated space quota.
> at org.apache.hadoop.hbase.master.HMaster$6.run(HMaster.java:2275)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:131)
>
[jira] [Updated] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain updated HBASE-20662:
---
Attachment: HBASE-20662.master.006.patch
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20662.master.001.patch,
> HBASE-20662.master.002.patch, HBASE-20662.master.003.patch,
> HBASE-20662.master.004.patch, HBASE-20662.master.004.patch,
> HBASE-20662.master.005.patch, HBASE-20662.master.006.patch
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService methodName:
> EnableTable size: 104 connection: 127.0.0.1:38030 deadline: 1527707127568,
> exception=org.apache.hadoop.hbase.security.AccessDeniedException: Enabling
> the table 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to
> a violated space quota.
> 2018-05-31 00:34:27,571 ERROR [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> quotas.RegionServerSpaceQuotaManager(210): Failed to disable space violation
> policy for testSetQuotaAndThenIncreaseQuotaWithDisable0. This table will
> remain in violation.
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Enabling the table
> 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to a
> violated space quota.
> at org.apache.hadoop.hbase.master.HMaster$6.run(HMaster.java:2275)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:131)
> at org.apache.hadoop.hbase.master.HMaster.enableTable(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.enableTable(MasterRpcServices.java:725)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:100)
> at
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:90)
> at
>
[jira] [Commented] (HBASE-21296) [2.1] Upgrade Jetty dependencies to latest in major-line
[
https://issues.apache.org/jira/browse/HBASE-21296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739766#comment-16739766
]
Hadoop QA commented on HBASE-21296:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2.1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
48s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
46s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
45s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
22s{color} | {color:green} branch-2.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
38s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 2s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}173m
25s{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}236m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:42ca976 |
| JIRA Issue | HBASE-21296 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954448/HBASE-21296.branch-2.1.patch
|
| Optional Tests | dupname asflicense javac javadoc unit shadedjars
hadoopcheck xml compile |
| uname | Linux df9b741fc31a 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2.1 / f357412941 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15533/testReport/ |
| Max. process+thread count | 5257 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15533/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> [2.1] Upgrade Jetty dependencies to latest in major-line
>
>
> Key: HBASE-21296
> URL: https://issues.apache.org/jira/browse/HBASE-21296
> Project: HBase
> Issue Type: Task
> Components: dependencies
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 2.1.3
>
> Attachments:
[jira] [Commented] (HBASE-21639) maxHeapUsage value not read properly from config during EntryBuffers initialization
[
https://issues.apache.org/jira/browse/HBASE-21639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739742#comment-16739742
]
Pankaj Kumar commented on HBASE-21639:
--
Thanks [~brfrn169] for looking into this issue.
> maxHeapUsage value not read properly from config during EntryBuffers
> initialization
> ---
>
> Key: HBASE-21639
> URL: https://issues.apache.org/jira/browse/HBASE-21639
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1, 2.1.0
>Reporter: Bo Cui
>Assignee: Pankaj Kumar
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-21639.001.patch
>
>
>
> {code:java|title=WALSplitter.java|borderStyle=solid}
> entryBuffers = new EntryBuffers(controller,
> this.conf.getInt("hbase.regionserver.hlog.splitlog.buffersize", 128 * 1024 *
> 1024),
> splitWriterCreationBounded);
> {code}
> In above case, EntryBuffers can't support maxHeapUsage in GB size.
> The parameter type of the new EntryBuffers() is long, but the conf max value
> is INT.MAX.
> this is wrong？it should be getLong？
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21519) Namespace region is never assigned in a HM failover scenario when multiwal is enabled
[
https://issues.apache.org/jira/browse/HBASE-21519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pankaj Kumar updated HBASE-21519:
-
Summary: Namespace region is never assigned in a HM failover scenario when
multiwal is enabled (was: Namespace region is never assigned in a HM failover
scenario and HM abort always due to init timeout)
> Namespace region is never assigned in a HM failover scenario when multiwal is
> enabled
> -
>
> Key: HBASE-21519
> URL: https://issues.apache.org/jira/browse/HBASE-21519
> Project: HBase
> Issue Type: Bug
> Components: master, wal
>Affects Versions: 2.1.1
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.2.0
>
> Attachments: HBASE-21519.branch-2.patch
>
>
> In our test env we found that namespace region is never be assigned on HM
> failover scenario when multiwal feature is enabled,
> {noformat}
> 2018-11-28 01:38:28,085 WARN [master/HM-1:16000:becomeActiveMaster]
> master.HMaster:
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15. is NOT
> online; state=\{31f6d3383af09e18e1e81ca02a93de15 state=OPEN,
> ts=1543340156928, server=RS-2,16020,1543339824397};
> ServerCrashProcedures=false. Master startup cannot progress, in
> holding-pattern until region onlined.
> {noformat}
> And finally HM abort with following error,
> {noformat}
> 2018-11-28 01:39:16,858 ERROR
> [ActiveMasterInitializationMonitor-1543338648565] master.HMaster: Master
> failed to complete initialization after 24ms. Please consider submitting
> a bug report including a thread dump of this process.
> 2018-11-28 01:39:18,980 ERROR
> [ActiveMasterInitializationMonitor-1543338648565] master.HMaster: Zombie
> Master exiting. Thread dump to stdout
> {noformat}
> Stack trace:
> {noformat}
> Thread 102 (master/HM-1:16000:becomeActiveMaster):
> State: TIMED_WAITING
> Blocked count: 100
> Waited count: 246
> Stack:
> java.lang.Thread.sleep(Native Method)
> org.apache.hadoop.hbase.util.Threads.sleep(Threads.java:148)
> org.apache.hadoop.hbase.master.HMaster.isRegionOnline(HMaster.java:1166)
>
> org.apache.hadoop.hbase.master.HMaster.waitForNamespaceOnline(HMaster.java:1187)
>
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1044)
>
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2285)
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:590)
> org.apache.hadoop.hbase.master.HMaster$$Lambda$40/1078246575.run(Unknown
> Source)
> java.lang.Thread.run(Thread.java:745)
> {noformat}
>
> Step to reproduce:
> 1) Setup a HBase cluster with 1/2 HM (say HM-1) and 2 RS(say RS-1 & RS-2)
> 2) Enable multiwal feature with following configuration setting and start
> the cluster,
> {noformat}
>
> hbase.wal.provider
> multiwal
>
>
> hbase.wal.regiongrouping.strategy
> identity
>
> {noformat}
> 3) Make sure meta and namespace regions are assigned on different RS, suppose
> RS-1 & RS-2 respectively.
> 4) Create table 't1'
> 5) Flush the meta table explicitly
> 6) Kill the RS-2, so during RS-2 SCP all regions including namespace region
> will be assigned to RS-1.
> 7) Now Kill RS-1 before meta flush happen. Here both RS-2 & RS-1 are
> shutdown now.
> 8) Stop the HM and start RS-1 & RS-2.
> 9) Now start the HM.
> Meta region is assigned successfully but HM is keep waiting for the namespace
> region onlline (Master startup cannot progress, in holding-pattern until
> region onlined) and abort with timeout.
> Observation:
> 1) After step-3 namespace region was assigned to RS-2 and meta entry was as
> follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server, timestamp=1543339860920, value=RS-2:16020
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:serverstartcode, timestamp=1543339860920, value=1543339824397
> {noformat}
> 2) After step-6 namespace region was assigned to RS-1 and meta entry was as
> follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server, timestamp=1543339880920, value=RS-1:16020
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:serverstartcode, timestamp=1543339880920, value=1543339829288
> {noformat}
> 3) After Step-9, meta entry for namespace region was as follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server, timestamp=1543339860920, value=RS-2:16020
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:serverstartcode, timestamp=1543339860920, value=1543339824397
> {noformat}
> During SCP we do meta
[jira] [Commented] (HBASE-21595) Print thread's information and stack traces when RS is aborting forcibly
[ https://issues.apache.org/jira/browse/HBASE-21595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739743#comment-16739743 ] Pankaj Kumar commented on HBASE-21595: -- [~stack] sir, kindly review this. > Print thread's information and stack traces when RS is aborting forcibly > > > Key: HBASE-21595 > URL: https://issues.apache.org/jira/browse/HBASE-21595 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 3.0.0, 2.2.0 >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Minor > Fix For: 3.0.0 > > Attachments: HBASE-21595.patch > > > After HBASE-21325 RS terminate forcibly on abort timeout. > We should print the thread info before terminating, will be useful to analyze > the RS abort timeout problem. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21519) Namespace region is never assigned in a HM failover scenario and HM abort always due to init timeout
[
https://issues.apache.org/jira/browse/HBASE-21519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739738#comment-16739738
]
Pankaj Kumar commented on HBASE-21519:
--
Thank you [~stack] sir for looking into the issue.
We are already sending meta provider id as ".meta" in the following trace,
{noformat}
RegionGroupingProvider.init(WALFactory, Configuration, String)
WALFactory.createProvider(Class, String)
WALFactory.getMetaProvider()
WALFactory.getWAL(RegionInfo)
{noformat}
But in RegionGroupingProvider.init() providerId is set along with the factoryId
(factoryId is the region sever name),
{code:java}
this.factory = factory;
StringBuilder sb = new StringBuilder().append(factory.factoryId);
if (providerId != null) {
if (providerId.startsWith(WAL_FILE_NAME_DELIMITER)) {
sb.append(providerId);
} else {
sb.append(WAL_FILE_NAME_DELIMITER).append(providerId);
}
}
this.providerId = sb.toString();
{code}
So finally providerId will be *.meta* and META_WAL_GROUP_NAME will
never be set for meta region as per below condition
{code:java}
public WAL getWAL(RegionInfo region) throws IOException {
String group;
if (META_WAL_PROVIDER_ID.equals(this.providerId)) {
// if (region != null && region.isMetaRegion()) {
group = META_WAL_GROUP_NAME;
}
{code}
Since meta region encoded name is 1588230740, so group will be *1588230740* and
provider will not be created as expected.
{code:java}
else {
byte[] id;
byte[] namespace;
if (region != null) {
id = region.getEncodedNameAsBytes();
namespace = region.getTable().getNamespace();
} else {
id = HConstants.EMPTY_BYTE_ARRAY;
namespace = null;
}
group = strategy.group(id, namespace);
}
return getWAL(group);
}
{code}
> Namespace region is never assigned in a HM failover scenario and HM abort
> always due to init timeout
>
>
> Key: HBASE-21519
> URL: https://issues.apache.org/jira/browse/HBASE-21519
> Project: HBase
> Issue Type: Bug
> Components: master, wal
>Affects Versions: 2.1.1
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.2.0
>
> Attachments: HBASE-21519.branch-2.patch
>
>
> In our test env we found that namespace region is never be assigned on HM
> failover scenario when multiwal feature is enabled,
> {noformat}
> 2018-11-28 01:38:28,085 WARN [master/HM-1:16000:becomeActiveMaster]
> master.HMaster:
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15. is NOT
> online; state=\{31f6d3383af09e18e1e81ca02a93de15 state=OPEN,
> ts=1543340156928, server=RS-2,16020,1543339824397};
> ServerCrashProcedures=false. Master startup cannot progress, in
> holding-pattern until region onlined.
> {noformat}
> And finally HM abort with following error,
> {noformat}
> 2018-11-28 01:39:16,858 ERROR
> [ActiveMasterInitializationMonitor-1543338648565] master.HMaster: Master
> failed to complete initialization after 24ms. Please consider submitting
> a bug report including a thread dump of this process.
> 2018-11-28 01:39:18,980 ERROR
> [ActiveMasterInitializationMonitor-1543338648565] master.HMaster: Zombie
> Master exiting. Thread dump to stdout
> {noformat}
> Stack trace:
> {noformat}
> Thread 102 (master/HM-1:16000:becomeActiveMaster):
> State: TIMED_WAITING
> Blocked count: 100
> Waited count: 246
> Stack:
> java.lang.Thread.sleep(Native Method)
> org.apache.hadoop.hbase.util.Threads.sleep(Threads.java:148)
> org.apache.hadoop.hbase.master.HMaster.isRegionOnline(HMaster.java:1166)
>
> org.apache.hadoop.hbase.master.HMaster.waitForNamespaceOnline(HMaster.java:1187)
>
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1044)
>
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2285)
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:590)
> org.apache.hadoop.hbase.master.HMaster$$Lambda$40/1078246575.run(Unknown
> Source)
> java.lang.Thread.run(Thread.java:745)
> {noformat}
>
> Step to reproduce:
> 1) Setup a HBase cluster with 1/2 HM (say HM-1) and 2 RS(say RS-1 & RS-2)
> 2) Enable multiwal feature with following configuration setting and start
> the cluster,
> {noformat}
>
> hbase.wal.provider
> multiwal
>
>
> hbase.wal.regiongrouping.strategy
> identity
>
> {noformat}
> 3) Make sure meta and namespace regions are assigned on different RS, suppose
> RS-1 & RS-2 respectively.
> 4) Create table 't1'
> 5) Flush the meta table explicitly
> 6) Kill the RS-2, so during RS-2 SCP all regions including namespace region
> will be assigned to RS-1.
> 7) Now Kill RS-1 before meta flush happen. Here both RS-2 & RS-1 are
> shutdown now.
> 8) Stop the HM and start RS-1 & RS-2.
> 9)
[jira] [Commented] (HBASE-21297) ModifyTableProcedure can throw TNDE instead of IOE in case of REGION_REPLICATION change
[
https://issues.apache.org/jira/browse/HBASE-21297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739709#comment-16739709
]
Hudson commented on HBASE-21297:
Results for branch branch-2.1
[build #758 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/758/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/758//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/758//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/758//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> ModifyTableProcedure can throw TNDE instead of IOE in case of
> REGION_REPLICATION change
> ---
>
> Key: HBASE-21297
> URL: https://issues.apache.org/jira/browse/HBASE-21297
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 2.0.0, 2.1.2
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21297.master.001.patch,
> HBASE-21297.master.001.patch
>
>
> Currently {{ModifyTableProcedure}} throws an {{IOException}} (See
> [ModifyTableProcedure.java#L252|https://github.com/apache/hbase/blob/924d183ba0e67b975e998f6006c993f457e03c20/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java#L252])
> when a user tries to modify REGION_REPLICATION for an enabled table.
> Instead, it can throw a more specific {{TableNotDisabledException}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21297) ModifyTableProcedure can throw TNDE instead of IOE in case of REGION_REPLICATION change
[
https://issues.apache.org/jira/browse/HBASE-21297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739691#comment-16739691
]
Hudson commented on HBASE-21297:
Results for branch branch-2.0
[build #1243 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1243/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1243//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1243//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1243//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> ModifyTableProcedure can throw TNDE instead of IOE in case of
> REGION_REPLICATION change
> ---
>
> Key: HBASE-21297
> URL: https://issues.apache.org/jira/browse/HBASE-21297
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 2.0.0, 2.1.2
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21297.master.001.patch,
> HBASE-21297.master.001.patch
>
>
> Currently {{ModifyTableProcedure}} throws an {{IOException}} (See
> [ModifyTableProcedure.java#L252|https://github.com/apache/hbase/blob/924d183ba0e67b975e998f6006c993f457e03c20/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java#L252])
> when a user tries to modify REGION_REPLICATION for an enabled table.
> Instead, it can throw a more specific {{TableNotDisabledException}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21700) Simplify the implementation of RSGroupInfoManagerImpl
[
https://issues.apache.org/jira/browse/HBASE-21700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739666#comment-16739666
]
Hudson commented on HBASE-21700:
Results for branch branch-2
[build #1602 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1602/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1602//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1602//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1602//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Simplify the implementation of RSGroupInfoManagerImpl
> -
>
> Key: HBASE-21700
> URL: https://issues.apache.org/jira/browse/HBASE-21700
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21700.patch
>
>
> Or maybe even do not use ClusterConnection related stuffs. The code is super
> complicated and seems over kill, as we only need to create the system table
> if it does not exist...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21297) ModifyTableProcedure can throw TNDE instead of IOE in case of REGION_REPLICATION change
[
https://issues.apache.org/jira/browse/HBASE-21297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739668#comment-16739668
]
Hudson commented on HBASE-21297:
Results for branch branch-2
[build #1602 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1602/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1602//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1602//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1602//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> ModifyTableProcedure can throw TNDE instead of IOE in case of
> REGION_REPLICATION change
> ---
>
> Key: HBASE-21297
> URL: https://issues.apache.org/jira/browse/HBASE-21297
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 2.0.0, 2.1.2
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21297.master.001.patch,
> HBASE-21297.master.001.patch
>
>
> Currently {{ModifyTableProcedure}} throws an {{IOException}} (See
> [ModifyTableProcedure.java#L252|https://github.com/apache/hbase/blob/924d183ba0e67b975e998f6006c993f457e03c20/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java#L252])
> when a user tries to modify REGION_REPLICATION for an enabled table.
> Instead, it can throw a more specific {{TableNotDisabledException}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21652) Refactor ThriftServer making thrift2 server inherited from thrift1 server
[
https://issues.apache.org/jira/browse/HBASE-21652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739667#comment-16739667
]
Hudson commented on HBASE-21652:
Results for branch branch-2
[build #1602 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1602/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1602//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1602//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1602//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Refactor ThriftServer making thrift2 server inherited from thrift1 server
> -
>
> Key: HBASE-21652
> URL: https://issues.apache.org/jira/browse/HBASE-21652
> Project: HBase
> Issue Type: Sub-task
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21652.addendum.patch, HBASE-21652.branch-2.patch,
> HBASE-21652.patch, HBASE-21652.v2.patch, HBASE-21652.v3.patch,
> HBASE-21652.v4.patch, HBASE-21652.v5.patch, HBASE-21652.v6.patch,
> HBASE-21652.v7.patch
>
>
> Except the different protocol, thrift2 Server should have no much difference
> from thrift1 Server. So refactoring the thrift server, making thrift2 server
> inherit from thrift1 server. Getting rid of many duplicated code.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21297) ModifyTableProcedure can throw TNDE instead of IOE in case of REGION_REPLICATION change
[
https://issues.apache.org/jira/browse/HBASE-21297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739664#comment-16739664
]
Hudson commented on HBASE-21297:
Results for branch master
[build #712 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/712/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/712//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/712//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/712//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> ModifyTableProcedure can throw TNDE instead of IOE in case of
> REGION_REPLICATION change
> ---
>
> Key: HBASE-21297
> URL: https://issues.apache.org/jira/browse/HBASE-21297
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 2.0.0, 2.1.2
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21297.master.001.patch,
> HBASE-21297.master.001.patch
>
>
> Currently {{ModifyTableProcedure}} throws an {{IOException}} (See
> [ModifyTableProcedure.java#L252|https://github.com/apache/hbase/blob/924d183ba0e67b975e998f6006c993f457e03c20/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java#L252])
> when a user tries to modify REGION_REPLICATION for an enabled table.
> Instead, it can throw a more specific {{TableNotDisabledException}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21628) memory leak risk when ClientAsyncPrefetchScanner not close
[
https://issues.apache.org/jira/browse/HBASE-21628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739652#comment-16739652
]
stack commented on HBASE-21628:
---
Took a quick look. A lot of new moving parts. Do we have to add prefetch
condition to Scan?
Do we have to pass in a new executorservice just for prefetch? Can't we use the
one that is passed in already?
The below is trying to override the existing prefetchCondidtion method?
86 if (prefetchCondition == null) {
87this.prefetchCondition = (maxCacheSize,currentCacheSize) ->
this.prefetchCondition();
So, the different thread possibility is new to this patch?
118 if (!scanThread.compareAndSet(null,Thread.currentThread())
119 && scanThread.get() != Thread.currentThread()) {
Here on each next we are doing an atomic op. Do we have to? This is because we
are using executorpool and can't be sure same thread passed? The old manner of
starting a dedicated thread didn't necessitate this because we knew there one
dedicated thread only?
> memory leak risk when ClientAsyncPrefetchScanner not close
> --
>
> Key: HBASE-21628
> URL: https://issues.apache.org/jira/browse/HBASE-21628
> Project: HBase
> Issue Type: Bug
>Reporter: cong.han
>Assignee: cong.han
>Priority: Major
> Attachments: HBASE-21328-v1.patch, HBASE-21628-v2.patch,
> HBASE-21628-v3.patch, HBASE-21628-v4.patch, HBASE-21628-v5.patch,
> HBASE-21628-v5.patch, HBASE-21628-v5.patch
>
>
> When we use ClientAsyncPrefetchScanner and we do two things below
> 1 Forgot to call close() method
> 2 The result are not full fetched
> The prefetch thread will not exit and leave a memory leak risk.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21691) Fix flaky test TestRecoveredEdits
[
https://issues.apache.org/jira/browse/HBASE-21691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739620#comment-16739620
]
Hudson commented on HBASE-21691:
Results for branch branch-2.0
[build #1242 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1242/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1242//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1242//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1242//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Fix flaky test TestRecoveredEdits
> -
>
> Key: HBASE-21691
> URL: https://issues.apache.org/jira/browse/HBASE-21691
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21691.branch-2.1.001.patch,
> HBASE-21691.master.001.patch, HBASE-21691.master.001.patch,
> HBASE-21691.master.001.patch, HBASE-21691.master.001.patch
>
>
> TestRecoveredEdits failed a lot times in precommit jobs.
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/master/Flaky_20Test_20Report/
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21702) Transcribe "what is HBaseCon" into book
Josh Elser created HBASE-21702: -- Summary: Transcribe "what is HBaseCon" into book Key: HBASE-21702 URL: https://issues.apache.org/jira/browse/HBASE-21702 Project: HBase Issue Type: Task Components: documentation Reporter: Josh Elser Assignee: Josh Elser Take the document I wrote on google docs (https://docs.google.com/document/d/15ZmgNKSzyR_UCzOmHEACnGZp-vuh-QlCYfr6bCmQ-4w/edit) and get it into the HBase book. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21296) [2.1] Upgrade Jetty dependencies to latest in major-line
[ https://issues.apache.org/jira/browse/HBASE-21296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-21296: --- Attachment: HBASE-21599.HBASE-20952.002.patch > [2.1] Upgrade Jetty dependencies to latest in major-line > > > Key: HBASE-21296 > URL: https://issues.apache.org/jira/browse/HBASE-21296 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 2.1.3 > > Attachments: HBASE-21282.001.branch-2.0.patch > > > Looks like we have dependencies on both jetty 9.2 and 9.3, but we're lagging > pretty far behind in both. We can upgrade both of these to the latest (august > 2018). > > I'll also have to take a look at why we're using two separate versions (maybe > we didn't want to switch from jetty-jsp to apache-jsp on 9.2->9.3?). Not sure > if there's a good reason for this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21296) [2.1] Upgrade Jetty dependencies to latest in major-line
[ https://issues.apache.org/jira/browse/HBASE-21296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-21296: --- Status: Patch Available (was: Open) Let me trigger a QA run first. That sounds like a good idea. > [2.1] Upgrade Jetty dependencies to latest in major-line > > > Key: HBASE-21296 > URL: https://issues.apache.org/jira/browse/HBASE-21296 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 2.1.3 > > Attachments: HBASE-21282.001.branch-2.0.patch, > HBASE-21296.branch-2.1.patch > > > Looks like we have dependencies on both jetty 9.2 and 9.3, but we're lagging > pretty far behind in both. We can upgrade both of these to the latest (august > 2018). > > I'll also have to take a look at why we're using two separate versions (maybe > we didn't want to switch from jetty-jsp to apache-jsp on 9.2->9.3?). Not sure > if there's a good reason for this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21296) [2.1] Upgrade Jetty dependencies to latest in major-line
[ https://issues.apache.org/jira/browse/HBASE-21296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-21296: --- Attachment: HBASE-21296.branch-2.1.patch > [2.1] Upgrade Jetty dependencies to latest in major-line > > > Key: HBASE-21296 > URL: https://issues.apache.org/jira/browse/HBASE-21296 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 2.1.3 > > Attachments: HBASE-21282.001.branch-2.0.patch, > HBASE-21296.branch-2.1.patch > > > Looks like we have dependencies on both jetty 9.2 and 9.3, but we're lagging > pretty far behind in both. We can upgrade both of these to the latest (august > 2018). > > I'll also have to take a look at why we're using two separate versions (maybe > we didn't want to switch from jetty-jsp to apache-jsp on 9.2->9.3?). Not sure > if there's a good reason for this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21296) [2.1] Upgrade Jetty dependencies to latest in major-line
[ https://issues.apache.org/jira/browse/HBASE-21296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-21296: --- Attachment: (was: HBASE-21599.HBASE-20952.002.patch) > [2.1] Upgrade Jetty dependencies to latest in major-line > > > Key: HBASE-21296 > URL: https://issues.apache.org/jira/browse/HBASE-21296 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 2.1.3 > > Attachments: HBASE-21282.001.branch-2.0.patch > > > Looks like we have dependencies on both jetty 9.2 and 9.3, but we're lagging > pretty far behind in both. We can upgrade both of these to the latest (august > 2018). > > I'll also have to take a look at why we're using two separate versions (maybe > we didn't want to switch from jetty-jsp to apache-jsp on 9.2->9.3?). Not sure > if there's a good reason for this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21296) [2.1] Upgrade Jetty dependencies to latest in major-line
[ https://issues.apache.org/jira/browse/HBASE-21296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739602#comment-16739602 ] Josh Elser commented on HBASE-21296: Thanks, sir. Will commit now before I forget. > [2.1] Upgrade Jetty dependencies to latest in major-line > > > Key: HBASE-21296 > URL: https://issues.apache.org/jira/browse/HBASE-21296 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 2.1.3 > > Attachments: HBASE-21282.001.branch-2.0.patch > > > Looks like we have dependencies on both jetty 9.2 and 9.3, but we're lagging > pretty far behind in both. We can upgrade both of these to the latest (august > 2018). > > I'll also have to take a look at why we're using two separate versions (maybe > we didn't want to switch from jetty-jsp to apache-jsp on 9.2->9.3?). Not sure > if there's a good reason for this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21691) Fix flaky test TestRecoveredEdits
[
https://issues.apache.org/jira/browse/HBASE-21691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739584#comment-16739584
]
Hudson commented on HBASE-21691:
Results for branch branch-2.1
[build #757 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/757/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/757//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/757//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/757//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix flaky test TestRecoveredEdits
> -
>
> Key: HBASE-21691
> URL: https://issues.apache.org/jira/browse/HBASE-21691
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21691.branch-2.1.001.patch,
> HBASE-21691.master.001.patch, HBASE-21691.master.001.patch,
> HBASE-21691.master.001.patch, HBASE-21691.master.001.patch
>
>
> TestRecoveredEdits failed a lot times in precommit jobs.
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/master/Flaky_20Test_20Report/
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-17942) Disable region splits and merges per table
[ https://issues.apache.org/jira/browse/HBASE-17942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739543#comment-16739543 ] Vladimir Rodionov commented on HBASE-17942: --- Sure, [~nihaljain.cs], go ahead. > Disable region splits and merges per table > -- > > Key: HBASE-17942 > URL: https://issues.apache.org/jira/browse/HBASE-17942 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > > Snapshot of a large tables usually fails when region split/merge happens > during snapshot. HBASE-15128 has introduced enable/disable switch for split > /merges, but only cluster - wide which is not always good in a large > deployments. The new feature will improve snapshot stability ands performance > for a single table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-17370) Fix or provide shell scripts to drain and decommission region server
[ https://issues.apache.org/jira/browse/HBASE-17370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739447#comment-16739447 ] Guanghao Zhang commented on HBASE-17370: Add a ut for these cmd? > Fix or provide shell scripts to drain and decommission region server > > > Key: HBASE-17370 > URL: https://issues.apache.org/jira/browse/HBASE-17370 > Project: HBase > Issue Type: Sub-task >Reporter: Jerry He >Assignee: Nihal Jain >Priority: Major > Labels: operability > Attachments: HBASE-17370.master.001.patch > > > 1. Update the existing shell scripts to use the new drain related API. > 2 Or provide new shell scripts. > 3. Provide a 'decommission' shell tool that puts the server in drain mode and > offload the server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21297) ModifyTableProcedure can throw TNDE instead of IOE in case of REGION_REPLICATION change
[
https://issues.apache.org/jira/browse/HBASE-21297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-21297:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to branch-2.0+. Thanks [~nihaljain.cs] for contributing.
> ModifyTableProcedure can throw TNDE instead of IOE in case of
> REGION_REPLICATION change
> ---
>
> Key: HBASE-21297
> URL: https://issues.apache.org/jira/browse/HBASE-21297
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 2.0.0, 2.1.2
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21297.master.001.patch,
> HBASE-21297.master.001.patch
>
>
> Currently {{ModifyTableProcedure}} throws an {{IOException}} (See
> [ModifyTableProcedure.java#L252|https://github.com/apache/hbase/blob/924d183ba0e67b975e998f6006c993f457e03c20/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java#L252])
> when a user tries to modify REGION_REPLICATION for an enabled table.
> Instead, it can throw a more specific {{TableNotDisabledException}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21657) PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.
[
https://issues.apache.org/jira/browse/HBASE-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739451#comment-16739451
]
Hadoop QA commented on HBASE-21657:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
5s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
47s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
16s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 34s{color}
| {color:red} hbase-common generated 1 new + 42 unchanged - 0 fixed = 43 total
(was 42) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} hbase-common: The patch generated 0 new + 278
unchanged - 3 fixed = 278 total (was 281) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} The patch passed checkstyle in hbase-client {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
27s{color} | {color:red} hbase-server: The patch generated 2 new + 410
unchanged - 0 fixed = 412 total (was 410) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} The patch passed checkstyle in hbase-mapreduce
{color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
57s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 22s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
30s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
0s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}191m 57s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 26m 27s{color}
| {color:red} hbase-mapreduce in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
46s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
|
[jira] [Commented] (HBASE-21628) memory leak risk when ClientAsyncPrefetchScanner not close
[
https://issues.apache.org/jira/browse/HBASE-21628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739450#comment-16739450
]
Hadoop QA commented on HBASE-21628:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
56s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} hbase-client: The patch generated 0 new + 61
unchanged - 1 fixed = 61 total (was 62) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
16s{color} | {color:green} The patch passed checkstyle in hbase-server {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
57s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 57s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
34s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}148m 1s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
50s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}204m 43s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestRestoreSnapshotFromClientClone |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21628 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954412/HBASE-21628-v5.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 111c936e 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 GNU/Linux
[jira] [Commented] (HBASE-21512) Introduce an AsyncClusterConnection and replace the usage of ClusterConnection
[
https://issues.apache.org/jira/browse/HBASE-21512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739438#comment-16739438
]
Hudson commented on HBASE-21512:
Results for branch HBASE-21512
[build #48 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/48/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/48//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/48//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/48//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Introduce an AsyncClusterConnection and replace the usage of ClusterConnection
> --
>
> Key: HBASE-21512
> URL: https://issues.apache.org/jira/browse/HBASE-21512
> Project: HBase
> Issue Type: Umbrella
>Reporter: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
>
> At least for the RSProcedureDispatcher, with CompletableFuture we do not need
> to set a delay and use a thread pool any more, which could reduce the
> resource usage and also the latency.
> Once this is done, I think we can remove the ClusterConnection completely,
> and start to rewrite the old sync client based on the async client, which
> could reduce the code base a lot for our client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21652) Refactor ThriftServer making thrift2 server inherited from thrift1 server
[
https://issues.apache.org/jira/browse/HBASE-21652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739400#comment-16739400
]
Hudson commented on HBASE-21652:
Results for branch master
[build #711 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/711/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/711//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/711//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/711//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Refactor ThriftServer making thrift2 server inherited from thrift1 server
> -
>
> Key: HBASE-21652
> URL: https://issues.apache.org/jira/browse/HBASE-21652
> Project: HBase
> Issue Type: Sub-task
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21652.addendum.patch, HBASE-21652.branch-2.patch,
> HBASE-21652.patch, HBASE-21652.v2.patch, HBASE-21652.v3.patch,
> HBASE-21652.v4.patch, HBASE-21652.v5.patch, HBASE-21652.v6.patch,
> HBASE-21652.v7.patch
>
>
> Except the different protocol, thrift2 Server should have no much difference
> from thrift1 Server. So refactoring the thrift server, making thrift2 server
> inherit from thrift1 server. Getting rid of many duplicated code.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21700) Simplify the implementation of RSGroupInfoManagerImpl
[
https://issues.apache.org/jira/browse/HBASE-21700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739398#comment-16739398
]
Hudson commented on HBASE-21700:
Results for branch master
[build #711 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/711/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/711//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/711//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/711//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Simplify the implementation of RSGroupInfoManagerImpl
> -
>
> Key: HBASE-21700
> URL: https://issues.apache.org/jira/browse/HBASE-21700
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21700.patch
>
>
> Or maybe even do not use ClusterConnection related stuffs. The code is super
> complicated and seems over kill, as we only need to create the system table
> if it does not exist...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-17942) Disable region splits and merges per table
[ https://issues.apache.org/jira/browse/HBASE-17942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739366#comment-16739366 ] Nihal Jain commented on HBASE-17942: [~vrodionov], [~stack ] This seems like a useful feature. Can I take this up? No activity since a long time. > Disable region splits and merges per table > -- > > Key: HBASE-17942 > URL: https://issues.apache.org/jira/browse/HBASE-17942 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > > Snapshot of a large tables usually fails when region split/merge happens > during snapshot. HBASE-15128 has introduced enable/disable switch for split > /merges, but only cluster - wide which is not always good in a large > deployments. The new feature will improve snapshot stability ands performance > for a single table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-17370) Fix or provide shell scripts to drain and decommission region server
[ https://issues.apache.org/jira/browse/HBASE-17370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739301#comment-16739301 ] Nihal Jain edited comment on HBASE-17370 at 1/10/19 11:05 AM: -- I tested the above commands on a 3 node cluster. Works as expected. (Earlier had tested on a standalone node) Could you please review this? [~stack], [~zghaobac] was (Author: nihaljain.cs): I tested the above scripts on a 3 node cluster. Works as expected. (Earlier had tested on a standalone node) Could you please review this? [~stack], [~zghaobac] > Fix or provide shell scripts to drain and decommission region server > > > Key: HBASE-17370 > URL: https://issues.apache.org/jira/browse/HBASE-17370 > Project: HBase > Issue Type: Sub-task >Reporter: Jerry He >Assignee: Nihal Jain >Priority: Major > Labels: operability > Attachments: HBASE-17370.master.001.patch > > > 1. Update the existing shell scripts to use the new drain related API. > 2 Or provide new shell scripts. > 3. Provide a 'decommission' shell tool that puts the server in drain mode and > offload the server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-17370) Fix or provide shell scripts to drain and decommission region server
[ https://issues.apache.org/jira/browse/HBASE-17370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739301#comment-16739301 ] Nihal Jain commented on HBASE-17370: I tested the above scripts on a 3 node cluster. Works as expected. (Earlier had tested on a standalone node) Could you please review this? [~stack], [~zghaobac] > Fix or provide shell scripts to drain and decommission region server > > > Key: HBASE-17370 > URL: https://issues.apache.org/jira/browse/HBASE-17370 > Project: HBase > Issue Type: Sub-task >Reporter: Jerry He >Assignee: Nihal Jain >Priority: Major > Labels: operability > Attachments: HBASE-17370.master.001.patch > > > 1. Update the existing shell scripts to use the new drain related API. > 2 Or provide new shell scripts. > 3. Provide a 'decommission' shell tool that puts the server in drain mode and > offload the server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21628) memory leak risk when ClientAsyncPrefetchScanner not close
[ https://issues.apache.org/jira/browse/HBASE-21628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cong.han updated HBASE-21628: - Attachment: HBASE-21628-v5.patch > memory leak risk when ClientAsyncPrefetchScanner not close > -- > > Key: HBASE-21628 > URL: https://issues.apache.org/jira/browse/HBASE-21628 > Project: HBase > Issue Type: Bug >Reporter: cong.han >Assignee: cong.han >Priority: Major > Attachments: HBASE-21328-v1.patch, HBASE-21628-v2.patch, > HBASE-21628-v3.patch, HBASE-21628-v4.patch, HBASE-21628-v5.patch, > HBASE-21628-v5.patch, HBASE-21628-v5.patch > > > When we use ClientAsyncPrefetchScanner and we do two things below > 1 Forgot to call close() method > 2 The result are not full fetched > The prefetch thread will not exit and leave a memory leak risk. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21064) Support set region server quota and allow exceed user/table/ns quota when rs has available quota
[
https://issues.apache.org/jira/browse/HBASE-21064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739286#comment-16739286
]
Hadoop QA commented on HBASE-21064:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 7 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
36s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 4m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
5s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
35s{color} | {color:red} hbase-client: The patch generated 3 new + 194
unchanged - 7 fixed = 197 total (was 201) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
15s{color} | {color:red} hbase-server: The patch generated 7 new + 113
unchanged - 7 fixed = 120 total (was 120) {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 22 new + 133 unchanged - 12 fixed
= 155 total (was 145) {color} |
| {color:orange}-0{color} | {color:orange} ruby-lint {color} | {color:orange}
0m 8s{color} | {color:orange} The patch generated 30 new + 394 unchanged - 0
fixed = 424 total (was 394) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
35s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 31s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
36s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
7s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}137m
20s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} |
[jira] [Commented] (HBASE-21297) ModifyTableProcedure can throw TNDE instead of IOE in case of REGION_REPLICATION change
[
https://issues.apache.org/jira/browse/HBASE-21297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739250#comment-16739250
]
Hadoop QA commented on HBASE-21297:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
7s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
7s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 35s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}130m
45s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}168m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21297 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954395/HBASE-21297.master.001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 8832b6b83d37 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 52bc6db050 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15528/testReport/ |
| Max. process+thread count | 4959 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15528/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> ModifyTableProcedure
[jira] [Commented] (HBASE-20345) HBase shell crash after backspace
[
https://issues.apache.org/jira/browse/HBASE-20345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739229#comment-16739229
]
Hari Krishna Dara commented on HBASE-20345:
---
For the sake of Googlers who come here searching for this hbase shell issue,
this is actually a JRuby (rather JLine) issue as described in this SO thread:
[https://stackoverflow.com/questions/49649449/hbase-shell-crash-after-backspace|https://stackoverflow.com/questions/49649449/hbase-shell-crash-after-backspace]
A quick workaround is listed in the above thread, set column count by running
something like {{stty columns 100}}.
There is a docker bug that is apparently resolved, though I am not sure if it
was released:
[https://github.com/docker/for-linux/issues/314|https://github.com/docker/for-linux/issues/314]
> HBase shell crash after backspace
> -
>
> Key: HBASE-20345
> URL: https://issues.apache.org/jira/browse/HBASE-20345
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.6
>Reporter: saffi
>Priority: Blocker
>
> I am running HBase in a docker container. Version is 1.2.4 (Also tried 1.2.6)
> Basically based on
> {quote}[https://github.com/dajobe/hbase-docker]
> {quote}
> When i do the following:
> 1) Build the image: *docker build -t hbase-docker .*
> 2) Start the container: *./start-hbase.sh*
> 3) Go in the container: *docker exec -it hbase bash*
> 4) Open HBase shell: *hbase shell*
> 5) and then if i type something and press backspace, it crashes with
> following:
> {code:java}
> hbase(main):001:0> li ConsoleReader.java:1414:in `backspace':
> java.lang.ArithmeticException: / by zero
> from ConsoleReader.java:1436:in `backspace'
> from ConsoleReader.java:628:in `readLine'
> from ConsoleReader.java:457:in `readLine'
> from Readline.java:237:in `s_readline'
> from Readline$s$s_readline.gen:65535:in `call'
> from CachingCallSite.java:332:in `cacheAndCall'
> from CachingCallSite.java:203:in `call'
> from FCallTwoArgNode.java:38:in `interpret'
> from LocalAsgnNode.java:123:in `interpret'
> from IfNode.java:111:in `interpret'
> from NewlineNode.java:104:in `interpret'
> from ASTInterpreter.java:74:in `INTERPRET_METHOD'
> from InterpretedMethod.java:147:in `call'
> from DefaultMethod.java:183:in `call'
> ...
> ...{code}
>
> Any idea how to make backspace work and prevent this from happening?! Thank
> you
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21657) PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.
[ https://issues.apache.org/jira/browse/HBASE-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21657: - Attachment: HBASE-21657.v5.patch > PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% > scan case. > > > Key: HBASE-21657 > URL: https://issues.apache.org/jira/browse/HBASE-21657 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5 > > Attachments: HBASE-21657.v1.patch, HBASE-21657.v2.patch, > HBASE-21657.v3.patch, HBASE-21657.v3.patch, HBASE-21657.v4.patch, > HBASE-21657.v5.patch, HBASE-21657.v5.patch, > HBase1.4.9-ssd-1000-rows-flamegraph.svg, > HBase1.4.9-ssd-1000-rows-qps-latency.png, > HBase2.0.4-patch-v2-ssd-1000-rows-qps-and-latency.png, > HBase2.0.4-patch-v2-ssd-1000-rows.svg, > HBase2.0.4-patch-v3-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-patch-v3-ssd-1000-rows-qps-and-latency.png, > HBase2.0.4-patch-v4-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-ssd-1000-rows-qps-latency.png, HBase2.0.4-with-patch.v2.png, > HBase2.0.4-without-patch-v2.png, debug-the-ByteBufferKeyValue.diff, > hbase2.0.4-ssd-scan-traces.2.svg, hbase2.0.4-ssd-scan-traces.svg, > hbase20-ssd-100-scan-traces.svg, image-2019-01-07-19-03-37-930.png, > image-2019-01-07-19-03-55-577.png, overview-statstics-1.png, run.log > > > We are evaluating the performance of branch-2, and find that the throughput > of scan in SSD cluster is almost the same as HDD cluster. so I made a > FlameGraph on RS, and found that the > PrivateCellUtil#estimatedSerializedSizeOf cost about 29% cpu, Obviously, it > has been the bottleneck in 100% scan case. > See the [^hbase20-ssd-100-scan-traces.svg] > BTW, in our XiaoMi branch, we introduce a > HRegion#updateReadRequestsByCapacityUnitPerSecond to sum up the size of cells > (for metric monitor), so it seems the performance loss was amplified. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21657) PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.
[ https://issues.apache.org/jira/browse/HBASE-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21657: - Attachment: (was: HBASE-21657.branch-2.0.patch) > PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% > scan case. > > > Key: HBASE-21657 > URL: https://issues.apache.org/jira/browse/HBASE-21657 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5 > > Attachments: HBASE-21657.v1.patch, HBASE-21657.v2.patch, > HBASE-21657.v3.patch, HBASE-21657.v3.patch, HBASE-21657.v4.patch, > HBASE-21657.v5.patch, HBase1.4.9-ssd-1000-rows-flamegraph.svg, > HBase1.4.9-ssd-1000-rows-qps-latency.png, > HBase2.0.4-patch-v2-ssd-1000-rows-qps-and-latency.png, > HBase2.0.4-patch-v2-ssd-1000-rows.svg, > HBase2.0.4-patch-v3-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-patch-v3-ssd-1000-rows-qps-and-latency.png, > HBase2.0.4-patch-v4-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-ssd-1000-rows-qps-latency.png, HBase2.0.4-with-patch.v2.png, > HBase2.0.4-without-patch-v2.png, debug-the-ByteBufferKeyValue.diff, > hbase2.0.4-ssd-scan-traces.2.svg, hbase2.0.4-ssd-scan-traces.svg, > hbase20-ssd-100-scan-traces.svg, image-2019-01-07-19-03-37-930.png, > image-2019-01-07-19-03-55-577.png, overview-statstics-1.png, run.log > > > We are evaluating the performance of branch-2, and find that the throughput > of scan in SSD cluster is almost the same as HDD cluster. so I made a > FlameGraph on RS, and found that the > PrivateCellUtil#estimatedSerializedSizeOf cost about 29% cpu, Obviously, it > has been the bottleneck in 100% scan case. > See the [^hbase20-ssd-100-scan-traces.svg] > BTW, in our XiaoMi branch, we introduce a > HRegion#updateReadRequestsByCapacityUnitPerSecond to sum up the size of cells > (for metric monitor), so it seems the performance loss was amplified. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21657) PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.
[ https://issues.apache.org/jira/browse/HBASE-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21657: - Attachment: HBASE-21657.branch-2.0.patch > PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% > scan case. > > > Key: HBASE-21657 > URL: https://issues.apache.org/jira/browse/HBASE-21657 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5 > > Attachments: HBASE-21657.v1.patch, HBASE-21657.v2.patch, > HBASE-21657.v3.patch, HBASE-21657.v3.patch, HBASE-21657.v4.patch, > HBASE-21657.v5.patch, HBase1.4.9-ssd-1000-rows-flamegraph.svg, > HBase1.4.9-ssd-1000-rows-qps-latency.png, > HBase2.0.4-patch-v2-ssd-1000-rows-qps-and-latency.png, > HBase2.0.4-patch-v2-ssd-1000-rows.svg, > HBase2.0.4-patch-v3-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-patch-v3-ssd-1000-rows-qps-and-latency.png, > HBase2.0.4-patch-v4-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-ssd-1000-rows-qps-latency.png, HBase2.0.4-with-patch.v2.png, > HBase2.0.4-without-patch-v2.png, debug-the-ByteBufferKeyValue.diff, > hbase2.0.4-ssd-scan-traces.2.svg, hbase2.0.4-ssd-scan-traces.svg, > hbase20-ssd-100-scan-traces.svg, image-2019-01-07-19-03-37-930.png, > image-2019-01-07-19-03-55-577.png, overview-statstics-1.png, run.log > > > We are evaluating the performance of branch-2, and find that the throughput > of scan in SSD cluster is almost the same as HDD cluster. so I made a > FlameGraph on RS, and found that the > PrivateCellUtil#estimatedSerializedSizeOf cost about 29% cpu, Obviously, it > has been the bottleneck in 100% scan case. > See the [^hbase20-ssd-100-scan-traces.svg] > BTW, in our XiaoMi branch, we introduce a > HRegion#updateReadRequestsByCapacityUnitPerSecond to sum up the size of cells > (for metric monitor), so it seems the performance loss was amplified. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21657) PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.
[
https://issues.apache.org/jira/browse/HBASE-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739199#comment-16739199
]
Hadoop QA commented on HBASE-21657:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
36s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 1m
5s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
24s{color} | {color:red} hbase-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
19s{color} | {color:red} hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
33s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
22s{color} | {color:red} hbase-mapreduce in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 24s{color}
| {color:red} hbase-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 19s{color}
| {color:red} hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 33s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 22s{color}
| {color:red} hbase-mapreduce in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} hbase-common: The patch generated 0 new + 211
unchanged - 3 fixed = 211 total (was 214) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} The patch passed checkstyle in hbase-client {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
20s{color} | {color:red} hbase-server: The patch generated 2 new + 410
unchanged - 0 fixed = 412 total (was 410) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} The patch passed checkstyle in hbase-mapreduce
{color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 1m
10s{color} | {color:red} patch has 16 errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 0m
53s{color} | {color:red} The patch causes 16 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
45s{color} | {color:red} The patch causes 16 errors with Hadoop v3.0.0. {color}
|
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
15s{color} | {color:red} hbase-common
[jira] [Commented] (HBASE-21620) Problem in scan query when using more than one column prefix filter in some cases.
[
https://issues.apache.org/jira/browse/HBASE-21620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739209#comment-16739209
]
Zheng Hu commented on HBASE-21620:
--
Not yet, you can try to have a simple test after remove the &&
comparator.compare(nextKV, cell) > 0 , and compare the performance.
> Problem in scan query when using more than one column prefix filter in some
> cases.
> --
>
> Key: HBASE-21620
> URL: https://issues.apache.org/jira/browse/HBASE-21620
> Project: HBase
> Issue Type: Bug
> Components: scan
>Affects Versions: 3.0.0, 1.5.0, 2.2.0, 1.4.8, 2.1.2, 2.0.4
> Environment: hbase-1.4.8, hbase-1.4.9
> hadoop-2.7.3
>Reporter: Mohamed Mohideen Meeran
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.2, 2.0.4
>
> Attachments: HBASE-21620.branch-1.patch, HBASE-21620.v1.patch,
> HBASE-21620.v2.patch, HBASE-21620.v3.patch, HBASE-21620.v3.patch,
> HBaseImportData.java, columnkey.txt, file.txt, test.patch
>
>
> In some cases, unable to get the scan results when using more than one column
> prefix filter.
> Attached a java file to import the data which we used and a text file
> containing the values..
> While executing the following query (hbase shell as well as java program) it
> is waiting indefinitely and after RPC timeout we got the following error..
> Also we noticed high cpu, high load average and very frequent young gc in
> the region server containing this row...
> scan 'namespace:tablename',\{STARTROW => 'test',ENDROW => 'test', FILTER =>
> "ColumnPrefixFilter('1544770422942010001_') OR
> ColumnPrefixFilter('1544769883529010001_')"}
> ROW COLUMN+CELL
> ERROR: Call id=18,
> waitTime=60005, rpcTimetout=6
>
> Note: Table scan operation and scan with a single column prefix filter works
> fine in this case.
> When we check the same query in hbase-1.2.5 it is working fine.
> Can you please help me on this..
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id
[ https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739204#comment-16739204 ] xufeng commented on HBASE-7191: --- ok,I will try to do it :) > HBCK - Add offline create/fix hbase.version and hbase.id > - > > Key: HBASE-7191 > URL: https://issues.apache.org/jira/browse/HBASE-7191 > Project: HBase > Issue Type: Improvement > Components: hbck >Affects Versions: 0.94.1 >Reporter: Enis Soztutar >Priority: Major > Attachments: 7191-2.patch > > > One of our clients run into a problem, in which they have the hbase.root.dir, > and cluster data, but their hbase.id and hbase.version files are corrupted. > HMaster creates those on start, but not if there is already existing data. > We can add smt like --fixIdFile, and ability for HBCK to do some offline > repairs for the version file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-21603) Move access control service from regionserver to master
[ https://issues.apache.org/jira/browse/HBASE-21603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei reassigned HBASE-21603: -- Assignee: Yi Mei > Move access control service from regionserver to master > --- > > Key: HBASE-21603 > URL: https://issues.apache.org/jira/browse/HBASE-21603 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > > Create a sub task to move access control service from regionserver to master. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
