[jira] [Commented] (HBASE-22058) upgrade thrift dependency to 0.9.3.1 on branches 1.4, 1.3 and 1.2
[
https://issues.apache.org/jira/browse/HBASE-22058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801387#comment-16801387
]
Hudson commented on HBASE-22058:
Results for branch branch-1.2
[build #708 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/708/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/708//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/708//JDK7_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/708//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> upgrade thrift dependency to 0.9.3.1 on branches 1.4, 1.3 and 1.2
> --
>
> Key: HBASE-22058
> URL: https://issues.apache.org/jira/browse/HBASE-22058
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Reporter: Francis Liu
>Assignee: Francis Liu
>Priority: Major
> Fix For: 1.4.10, 1.3.4, 1.2.12
>
> Attachments: HBASE-22058-branch-1.4.patch,
> HBASE-22058.branch-1.4.001.patch, HBASE-22058.branch-1.4.002.patch
>
>
> Creating a separate Jira to do the backport since the .thrift files differ
> between branch-1 and 1.4, 1.3. I backported the change in the pom.xml from
> branch-1 and regenerated the thrift configs.
> cc [~apurtell]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21619) Fix warning message caused by incorrect ternary operator evaluation
[
https://issues.apache.org/jira/browse/HBASE-21619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801382#comment-16801382
]
Hudson commented on HBASE-21619:
Results for branch branch-2.1
[build #994 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/994/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/994//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/994//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/994//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix warning message caused by incorrect ternary operator evaluation
> ---
>
> Key: HBASE-21619
> URL: https://issues.apache.org/jira/browse/HBASE-21619
> Project: HBase
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.1.5, 2.2.1
>
> Attachments: HBASE-21619.master.001.patch
>
>
> {code:title=LoadIncrementalHFiles#doBulkLoad}
> LOG.warn(
> "Bulk load operation did not find any files to load in " +
> "directory " + hfofDir != null
> ? hfofDir.toUri().toString()
> : "" + ". Does it contain files in " +
> "subdirectories that correspond to column family names?");
> {code}
> JDK complains {{"Bulk load operation did not find any files to load in " +
> "directory " + hfofDir != null}} is always true, which is not what is
> intended, and that produces a wrong message.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-15560) TinyLFU-based BlockCache
[
https://issues.apache.org/jira/browse/HBASE-15560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801363#comment-16801363
]
Hadoop QA commented on HBASE-15560:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
29s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 6m
46s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
28s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-resource-bundle hbase-shaded . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 11m 1s{color}
| {color:red} root generated 50 new + 1327 unchanged - 50 fixed = 1377 total
(was 1377) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
22s{color} | {color:red} root: The patch generated 3 new + 55 unchanged - 1
fixed = 58 total (was 56) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
7s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 6m
5s{color} | {color:blue} patch has no errors when building the reference guide.
See footer for rendered docs, which you should manually inspect. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
50s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 9s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-resource-bundle hbase-shaded . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}195m 16s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 2m
7s{color} | {color:green} The patch does not generate ASF License
[jira] [Updated] (HBASE-21455) Update filesystem-space quota fail if there is a space quota for non-existing namespace
[
https://issues.apache.org/jira/browse/HBASE-21455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pankaj Kumar updated HBASE-21455:
-
Fix Version/s: 3.0.0
Status: Patch Available (was: Open)
> Update filesystem-space quota fail if there is a space quota for non-existing
> namespace
> ---
>
> Key: HBASE-21455
> URL: https://issues.apache.org/jira/browse/HBASE-21455
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.2
>Reporter: wenbang
>Assignee: wenbang
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-21455.master.001.patch,
> HBASE-21455.master.002.patch
>
>
> QuotaObserverChore#fetchAllTablesWithQuotasDefined may fail and throw a
> NamespaceNotFoundException because of namespace does not exist
> {code:java}
> // Collect all of the tables in the namespace
> TableName[] tablesInNS =
> conn.getAdmin().listTableNamesByNamespace(namespace);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21619) Fix warning message caused by incorrect ternary operator evaluation
[
https://issues.apache.org/jira/browse/HBASE-21619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801358#comment-16801358
]
Hudson commented on HBASE-21619:
Results for branch branch-2.0
[build #1465 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1465/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1465//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1465//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1465//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Fix warning message caused by incorrect ternary operator evaluation
> ---
>
> Key: HBASE-21619
> URL: https://issues.apache.org/jira/browse/HBASE-21619
> Project: HBase
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.1.5, 2.2.1
>
> Attachments: HBASE-21619.master.001.patch
>
>
> {code:title=LoadIncrementalHFiles#doBulkLoad}
> LOG.warn(
> "Bulk load operation did not find any files to load in " +
> "directory " + hfofDir != null
> ? hfofDir.toUri().toString()
> : "" + ". Does it contain files in " +
> "subdirectories that correspond to column family names?");
> {code}
> JDK complains {{"Bulk load operation did not find any files to load in " +
> "directory " + hfofDir != null}} is always true, which is not what is
> intended, and that produces a wrong message.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21455) Update filesystem-space quota fail if there is a space quota for non-existing namespace
[
https://issues.apache.org/jira/browse/HBASE-21455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801347#comment-16801347
]
wenbang commented on HBASE-21455:
-
[~pankaj2461] Would you please take a look at v2 ? Thanks.
> Update filesystem-space quota fail if there is a space quota for non-existing
> namespace
> ---
>
> Key: HBASE-21455
> URL: https://issues.apache.org/jira/browse/HBASE-21455
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.2
>Reporter: wenbang
>Assignee: wenbang
>Priority: Major
> Attachments: HBASE-21455.master.001.patch,
> HBASE-21455.master.002.patch
>
>
> QuotaObserverChore#fetchAllTablesWithQuotasDefined may fail and throw a
> NamespaceNotFoundException because of namespace does not exist
> {code:java}
> // Collect all of the tables in the namespace
> TableName[] tablesInNS =
> conn.getAdmin().listTableNamesByNamespace(namespace);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20952) Re-visit the WAL API
[
https://issues.apache.org/jira/browse/HBASE-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801343#comment-16801343
]
Hudson commented on HBASE-20952:
Results for branch HBASE-20952
[build #75 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20952/75/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20952/75//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20952/75//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20952/75//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Re-visit the WAL API
>
>
> Key: HBASE-20952
> URL: https://issues.apache.org/jira/browse/HBASE-20952
> Project: HBase
> Issue Type: Improvement
> Components: wal
>Reporter: Josh Elser
>Priority: Major
> Attachments: 20952.v1.txt
>
>
> Take a step back from the current WAL implementations and think about what an
> HBase WAL API should look like. What are the primitive calls that we require
> to guarantee durability of writes with a high degree of performance?
> The API needs to take the current implementations into consideration. We
> should also have a mind for what is happening in the Ratis LogService (but
> the LogService should not dictate what HBase's WAL API looks like RATIS-272).
> Other "systems" inside of HBase that use WALs are replication and
> backup Replication has the use-case for "tail"'ing the WAL which we
> should provide via our new API. B doesn't do anything fancy (IIRC). We
> should make sure all consumers are generally going to be OK with the API we
> create.
> The API may be "OK" (or OK in a part). We need to also consider other methods
> which were "bolted" on such as {{AbstractFSWAL}} and
> {{WALFileLengthProvider}}. Other corners of "WAL use" (like the
> {{WALSplitter}} should also be looked at to use WAL-APIs only).
> We also need to make sure that adequate interface audience and stability
> annotations are chosen.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22074) Should use procedure store to persist the state in reportRegionStateTransition
[ https://issues.apache.org/jira/browse/HBASE-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801342#comment-16801342 ] Guanghao Zhang commented on HBASE-22074: +1 for v7 patch after run HADOOP QA. > Should use procedure store to persist the state in reportRegionStateTransition > -- > > Key: HBASE-22074 > URL: https://issues.apache.org/jira/browse/HBASE-22074 > Project: HBase > Issue Type: Bug > Components: amv2, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: HBASE-22074-v1.patch, HBASE-22074-v2.patch, > HBASE-22074-v3.patch, HBASE-22074-v3.patch, HBASE-22074-v4.patch, > HBASE-22074-v5.patch, HBASE-22074-v6.patch, HBASE-22074-v7.patch, > HBASE-22074.patch > > > For now we will update the meta region directly. This may cause lots of > problems and after a bunch of fixes, we still can not solve the problem in > HBASE-22060. > So maybe the approach itself is not a good choice, let's try another way to > see if it could work better. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21455) Update filesystem-space quota fail if there is a space quota for non-existing namespace
[
https://issues.apache.org/jira/browse/HBASE-21455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wenbang updated HBASE-21455:
Attachment: HBASE-21455.master.002.patch
> Update filesystem-space quota fail if there is a space quota for non-existing
> namespace
> ---
>
> Key: HBASE-21455
> URL: https://issues.apache.org/jira/browse/HBASE-21455
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.2
>Reporter: wenbang
>Assignee: wenbang
>Priority: Major
> Attachments: HBASE-21455.master.001.patch,
> HBASE-21455.master.002.patch
>
>
> QuotaObserverChore#fetchAllTablesWithQuotasDefined may fail and throw a
> NamespaceNotFoundException because of namespace does not exist
> {code:java}
> // Collect all of the tables in the namespace
> TableName[] tablesInNS =
> conn.getAdmin().listTableNamesByNamespace(namespace);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21911) Move getUserPermissions from regionserver to master
[ https://issues.apache.org/jira/browse/HBASE-21911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801339#comment-16801339 ] Guanghao Zhang commented on HBASE-21911: [~Yi Mei] The patch can't applied to branch-2 and branch-2.2 directly by cmd "git am". > Move getUserPermissions from regionserver to master > --- > > Key: HBASE-21911 > URL: https://issues.apache.org/jira/browse/HBASE-21911 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-21911.master.003.patch, > HBASE-21911.master.004.patch, HBASE-21911.master.005.patch, > HBASE-21911.master.006.patch > > > Create a sub-task to move acl methods: getUserPermissions from regionserver > to master. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22074) Should use procedure store to persist the state in reportRegionStateTransition
[ https://issues.apache.org/jira/browse/HBASE-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-22074: -- Attachment: HBASE-22074-v7.patch > Should use procedure store to persist the state in reportRegionStateTransition > -- > > Key: HBASE-22074 > URL: https://issues.apache.org/jira/browse/HBASE-22074 > Project: HBase > Issue Type: Bug > Components: amv2, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: HBASE-22074-v1.patch, HBASE-22074-v2.patch, > HBASE-22074-v3.patch, HBASE-22074-v3.patch, HBASE-22074-v4.patch, > HBASE-22074-v5.patch, HBASE-22074-v6.patch, HBASE-22074-v7.patch, > HBASE-22074.patch > > > For now we will update the meta region directly. This may cause lots of > problems and after a bunch of fixes, we still can not solve the problem in > HBASE-22060. > So maybe the approach itself is not a good choice, let's try another way to > see if it could work better. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21964) unset Quota by Throttle Type
[
https://issues.apache.org/jira/browse/HBASE-21964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-21964:
---
Resolution: Fixed
Fix Version/s: 2.3.0
2.2.0
3.0.0
Status: Resolved (was: Patch Available)
Pushed to branch-2.2+. Thanks [~y_static_y] for contributing.
> unset Quota by Throttle Type
>
>
> Key: HBASE-21964
> URL: https://issues.apache.org/jira/browse/HBASE-21964
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 1.4.8
>Reporter: yaojingyi
>Assignee: yaojingyi
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.3.0
>
> Attachments: HBASE-21964.branch-1.4.001.patch,
> HBASE-21964.master.006.patch, HBASE-21964.master.007.patch,
> HBASE-21964.master.008.patch, HBASE-21964.master.009.patch,
> image-2019-03-20-11-30-37-350.png, unthrottleByType.patch
>
>
>
> {code:java}
> //first set_quota to USER=> 'u1'
> set_quota TYPE => THROTTLE, USER => 'u1', THROTTLE_TYPE => WRITE, LIMIT =>
> '1000req/sec'
> //then
> hbase(main):004:0> set_quota TYPE => THROTTLE, THROTTLE_TYPE => WRITE, USER
> => 'u1', LIMIT => NONE
> ERROR: Unexpected arguments: {"THROTTLE_TYPE"=>"WRITE"}
> // or try "THROTTLE_TYPE"=>"WRITE_NUMBER"
> hbase(main):012:0* set_quota TYPE => THROTTLE, THROTTLE_TYPE => WRITE_NUMBER,
> USER => 'u1', LIMIT => NONE
> NameError: uninitialized constant WRITE_NUMBER
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20159) Support using separate ZK quorums for client
[
https://issues.apache.org/jira/browse/HBASE-20159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801323#comment-16801323
]
Yu Li commented on HBASE-20159:
---
Users using this feature please note the newly found issue in HBASE-22079 which
will cause ZK leak.
> Support using separate ZK quorums for client
>
>
> Key: HBASE-20159
> URL: https://issues.apache.org/jira/browse/HBASE-20159
> Project: HBase
> Issue Type: New Feature
> Components: Client, Operability, Zookeeper
>Reporter: Yu Li
>Assignee: Yu Li
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: 20159.addendum, 20159.addendum2.patch,
> HBASE-20159.branch-2.addendum.v2.patch, HBASE-20159.branch-2.patch,
> HBASE-20159.patch, HBASE-20159.v2.patch, HBASE-20159.v3.patch
>
>
> Currently we are using the same zookeeper quorums for client and server,
> which makes us under risk that if some client connection boost exhausted
> zookeeper, RegionServer might abort due to zookeeper session loss. Actually
> we have suffered from this many times in production.
> Here we propose to allow client to use different ZK quorums, through below
> settings:
> {noformat}
> hbase.client.zookeeper.quorum
> hbase.client.zookeeper.property.clientPort
> hbase.client.zookeeper.observer.mode
> {noformat}
> The first two are for specifying client zookeeper properties, and the third
> one indicating whether the client ZK nodes are in observer mode. If the
> client ZK are not observer nodes, HMaster will take responsibility to
> synchronize necessary meta information (such as meta location and master
> address, etc.) from server to client ZK
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22108) Avoid passing null in Admin methods
[ https://issues.apache.org/jira/browse/HBASE-22108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-22108: -- Component/s: Admin > Avoid passing null in Admin methods > --- > > Key: HBASE-22108 > URL: https://issues.apache.org/jira/browse/HBASE-22108 > Project: HBase > Issue Type: Task > Components: Admin >Reporter: Duo Zhang >Priority: Major > > For some methods we must pass null if we want specific behaviors, for > example, move region to a random server, split region without providing an > explicit split point, etc. > We should provide methods without some parameters so user do not need to pass > null. > So in the future users do not need to guess whether a method accept null > parameters, the answer is always no. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22108) Avoid passing null in Admin methods
Duo Zhang created HBASE-22108: - Summary: Avoid passing null in Admin methods Key: HBASE-22108 URL: https://issues.apache.org/jira/browse/HBASE-22108 Project: HBase Issue Type: Task Reporter: Duo Zhang For some methods we must pass null if we want specific behaviors, for example, move region to a random server, split region without providing an explicit split point, etc. We should provide methods without some parameters so user do not need to pass null. So in the future users do not need to guess whether a method accept null parameters, the answer is always no. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22094) Throw TableNotFoundException if table not exists in AsyncAdmin.compact
[ https://issues.apache.org/jira/browse/HBASE-22094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801314#comment-16801314 ] Duo Zhang commented on HBASE-22094: --- The patch is fine, but maybe it can be a bit simpler? Just check whether the returned region locations by calling getTableHRegionLocations is empty? > Throw TableNotFoundException if table not exists in AsyncAdmin.compact > -- > > Key: HBASE-22094 > URL: https://issues.apache.org/jira/browse/HBASE-22094 > Project: HBase > Issue Type: Sub-task > Components: Admin >Reporter: Duo Zhang >Assignee: Sakthi >Priority: Major > Labels: beginner > Attachments: hbase-22094.master.001.patch, > hbase-22094.master.002.patch, hbase-22094.master.003.patch, > hbase-22094.master.004.patch > > > Now we will return successfully, which is not the same with the HBaseAdmin. > For NORMAL compaction, we should throw TableNotFoundException if the > locations is empty. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21911) Move getUserPermissions from regionserver to master
[ https://issues.apache.org/jira/browse/HBASE-21911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801304#comment-16801304 ] Yi Mei commented on HBASE-21911: Yes, the javac warnings are not related, the class with warnings are not modified in this patch. > Move getUserPermissions from regionserver to master > --- > > Key: HBASE-21911 > URL: https://issues.apache.org/jira/browse/HBASE-21911 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-21911.master.003.patch, > HBASE-21911.master.004.patch, HBASE-21911.master.005.patch, > HBASE-21911.master.006.patch > > > Create a sub-task to move acl methods: getUserPermissions from regionserver > to master. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21964) unset Quota by Throttle Type
[
https://issues.apache.org/jira/browse/HBASE-21964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801301#comment-16801301
]
Guanghao Zhang commented on HBASE-21964:
Ping [~apurtell] for branch-1.
> unset Quota by Throttle Type
>
>
> Key: HBASE-21964
> URL: https://issues.apache.org/jira/browse/HBASE-21964
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 1.4.8
>Reporter: yaojingyi
>Assignee: yaojingyi
>Priority: Major
> Attachments: HBASE-21964.branch-1.4.001.patch,
> HBASE-21964.master.006.patch, HBASE-21964.master.007.patch,
> HBASE-21964.master.008.patch, HBASE-21964.master.009.patch,
> image-2019-03-20-11-30-37-350.png, unthrottleByType.patch
>
>
>
> {code:java}
> //first set_quota to USER=> 'u1'
> set_quota TYPE => THROTTLE, USER => 'u1', THROTTLE_TYPE => WRITE, LIMIT =>
> '1000req/sec'
> //then
> hbase(main):004:0> set_quota TYPE => THROTTLE, THROTTLE_TYPE => WRITE, USER
> => 'u1', LIMIT => NONE
> ERROR: Unexpected arguments: {"THROTTLE_TYPE"=>"WRITE"}
> // or try "THROTTLE_TYPE"=>"WRITE_NUMBER"
> hbase(main):012:0* set_quota TYPE => THROTTLE, THROTTLE_TYPE => WRITE_NUMBER,
> USER => 'u1', LIMIT => NONE
> NameError: uninitialized constant WRITE_NUMBER
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21911) Move getUserPermissions from regionserver to master
[ https://issues.apache.org/jira/browse/HBASE-21911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801299#comment-16801299 ] Guanghao Zhang commented on HBASE-21911: [~Yi Mei] The javac warnings is not related? > Move getUserPermissions from regionserver to master > --- > > Key: HBASE-21911 > URL: https://issues.apache.org/jira/browse/HBASE-21911 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-21911.master.003.patch, > HBASE-21911.master.004.patch, HBASE-21911.master.005.patch, > HBASE-21911.master.006.patch > > > Create a sub-task to move acl methods: getUserPermissions from regionserver > to master. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22100) False positive for error prone warnings in pre commit job
[
https://issues.apache.org/jira/browse/HBASE-22100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801297#comment-16801297
]
Duo Zhang commented on HBASE-22100:
---
{quote}
If you think this is a thing we can fix by updating the default javac output
diff set up in yetus, then yeah a yetus jira makes sense.
{quote}
No sure how yetus works but I found something maybe related in yetus
{code:maven.sh}
## @description process javac output from maven
## @audience private
## @stabilityevolving
function maven_javac_logfilter
{
declare input=$1
declare output=$2
${GREP} -E '\[(ERROR|WARNING)\] /.*\.java:' "${input}" > "${output}"
}
## @description handle diffing maven javac errors
## @audience private
## @stabilityevolving
## @replaceable no
## @parambranchlog
## @parampatchlog
## @return differences
function maven_javac_calcdiffs
{
declare orig=$1
declare new=$2
declare tmp=${PATCH_DIR}/pl.$$.${RANDOM}
# first, strip :[line
# this keeps file,column in an attempt to increase
# accuracy in case of multiple, repeated errors
# since the column number shouldn't change
# if the line of code hasn't been touched
"${SED}" -e 's#:\[[0-9]*,#:#' "${orig}" > "${tmp}.branch"
"${SED}" -e 's#:\[[0-9]*,#:#' "${new}" > "${tmp}.patch"
# compare the errors, generating a string of line
# numbers. Sorry portability: GNU diff makes this too easy
${DIFF} --unchanged-line-format="" \
--old-line-format="" \
--new-line-format="%dn " \
"${tmp}.branch" \
"${tmp}.patch" > "${tmp}.lined"
# now, pull out those lines of the raw output
# shellcheck disable=SC2013
for j in $(cat "${tmp}.lined"); do
head -"${j}" "${new}" | tail -1
done
rm "${tmp}.branch" "${tmp}.patch" "${tmp}.lined" 2>/dev/null
}
{code}
Is it possible to customize these two functions? Maybe just add a sort on the
output...
> False positive for error prone warnings in pre commit job
> -
>
> Key: HBASE-22100
> URL: https://issues.apache.org/jira/browse/HBASE-22100
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Duo Zhang
>Priority: Major
>
> https://builds.apache.org/job/PreCommit-HBASE-Build/16516/artifact/patchprocess/branch-compile-javac-hbase-client.txt
> {noformat}
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,69]
> [UnusedVariable] The parameter 'updateCachedLocation' is never read.
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,42]
> [UnusedVariable] The parameter 'error' is never read.
> {noformat}
> https://builds.apache.org/job/PreCommit-HBASE-Build/16516/artifact/patchprocess/patch-compile-javac-hbase-client.txt
> {noformat}
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,42]
> [UnusedVariable] The parameter 'error' is never read.
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,69]
> [UnusedVariable] The parameter 'updateCachedLocation' is never read.
> {noformat}
> And the output is 1 new and 1 fixed, the new one is
> {noformat}
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,69]
> [UnusedVariable] The parameter 'updateCachedLocation' is never read.
> {noformat}
> I think here we should report nothing, as it is just an order change...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22079) master leaks ZK on shutdown and gets stuck because of netty threads if netty socket is used
[
https://issues.apache.org/jira/browse/HBASE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HBASE-22079:
-
Attachment: HBASE-22079.01.patch
> master leaks ZK on shutdown and gets stuck because of netty threads if netty
> socket is used
> ---
>
> Key: HBASE-22079
> URL: https://issues.apache.org/jira/browse/HBASE-22079
> Project: HBase
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HBASE-22079.01.patch, HBASE-22079.patch
>
>
> {noformat}
> "master/...:17000:becomeActiveMaster-SendThread(...1)" #311 daemon prio=5
> os_prio=0 tid=0x58c61800 nid=0x2dd0 waiting on condition
> [0x000c477fe000]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc4a5b3c0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at
> java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
> at
> java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
> at
> org.apache.zookeeper.ClientCnxnSocketNetty.doTransport(ClientCnxnSocketNetty.java:232)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1146)
> {noformat}
> This causes a bunch of netty threads to also leak it looks like, and these
> are not daemon (by design, apparently)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22107) make dead server metric work for HBase in a compute fabric (e.g. YARN) use case
[ https://issues.apache.org/jira/browse/HBASE-22107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801263#comment-16801263 ] Sean Busbey commented on HBASE-22107: - How about a TTL for how long to keep the info around? Default it to -1 for current behavior and when set to something else we effectively have a "recently dead server" list > make dead server metric work for HBase in a compute fabric (e.g. YARN) use > case > --- > > Key: HBASE-22107 > URL: https://issues.apache.org/jira/browse/HBASE-22107 > Project: HBase > Issue Type: Improvement > Components: Operability >Reporter: Sergey Shelukhin >Priority: Minor > > dead servers appear to only be cleaned up when a server comes up on the same > host and port; however, if HBase is running on smth like YARN with many more > hosts than RSes, RS may come up on a different server and the dead one will > never be cleaned. > The metric should be improved to account for that... it will potentially > require configuring master with expected number of region servers, so that > the metric could be output based on that. > Dead server list should also be expired based on timestamp in such cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22107) make dead server metric work for HBase in a compute fabric (e.g. YARN) use case
[ https://issues.apache.org/jira/browse/HBASE-22107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-22107: Priority: Minor (was: Major) Component/s: Operability > make dead server metric work for HBase in a compute fabric (e.g. YARN) use > case > --- > > Key: HBASE-22107 > URL: https://issues.apache.org/jira/browse/HBASE-22107 > Project: HBase > Issue Type: Bug > Components: Operability >Reporter: Sergey Shelukhin >Priority: Minor > > dead servers appear to only be cleaned up when a server comes up on the same > host and port; however, if HBase is running on smth like YARN with many more > hosts than RSes, RS may come up on a different server and the dead one will > never be cleaned. > The metric should be improved to account for that... it will potentially > require configuring master with expected number of region servers, so that > the metric could be output based on that. > Dead server list should also be expired based on timestamp in such cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22107) make dead server metric work for HBase in a compute fabric (e.g. YARN) use case
[ https://issues.apache.org/jira/browse/HBASE-22107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-22107: Issue Type: Improvement (was: Bug) > make dead server metric work for HBase in a compute fabric (e.g. YARN) use > case > --- > > Key: HBASE-22107 > URL: https://issues.apache.org/jira/browse/HBASE-22107 > Project: HBase > Issue Type: Improvement > Components: Operability >Reporter: Sergey Shelukhin >Priority: Minor > > dead servers appear to only be cleaned up when a server comes up on the same > host and port; however, if HBase is running on smth like YARN with many more > hosts than RSes, RS may come up on a different server and the dead one will > never be cleaned. > The metric should be improved to account for that... it will potentially > require configuring master with expected number of region servers, so that > the metric could be output based on that. > Dead server list should also be expired based on timestamp in such cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19222) update jruby to 9.1.17.0
[ https://issues.apache.org/jira/browse/HBASE-19222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801260#comment-16801260 ] Clément Guillaume commented on HBASE-19222: --- FYI me and my team are interested in Java 9 (and so a recent jruby version) for the 1.x branch (1.2 for now) > update jruby to 9.1.17.0 > > > Key: HBASE-19222 > URL: https://issues.apache.org/jira/browse/HBASE-19222 > Project: HBase > Issue Type: Sub-task > Components: shell >Reporter: Sean Busbey >Assignee: Peter Somogyi >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: HBASE-19222.master.001.patch > > > JRuby's 9.1.14.0 release is described as "things generally work in jdk9" > https://twitter.com/headius/status/928405094407827457 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22107) make dead server metric work for HBase in a compute fabric (e.g. YARN) use case
Sergey Shelukhin created HBASE-22107: Summary: make dead server metric work for HBase in a compute fabric (e.g. YARN) use case Key: HBASE-22107 URL: https://issues.apache.org/jira/browse/HBASE-22107 Project: HBase Issue Type: Bug Reporter: Sergey Shelukhin dead servers appear to only be cleaned up when a server comes up on the same host and port; however, if HBase is running on smth like YARN with many more hosts than RSes, RS may come up on a different server and the dead one will never be cleaned. The metric should be improved to account for that... it will potentially require configuring master with expected number of region servers, so that the metric could be output based on that. Dead server list should also be expired based on timestamp in such cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19222) update jruby to 9.1.17.0
[ https://issues.apache.org/jira/browse/HBASE-19222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801249#comment-16801249 ] stack commented on HBASE-19222: --- [~psomogyi] Can it be done in branch-2.2 only? Work on jdk9+ in branch-2.2+? > update jruby to 9.1.17.0 > > > Key: HBASE-19222 > URL: https://issues.apache.org/jira/browse/HBASE-19222 > Project: HBase > Issue Type: Sub-task > Components: shell >Reporter: Sean Busbey >Assignee: Peter Somogyi >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: HBASE-19222.master.001.patch > > > JRuby's 9.1.14.0 release is described as "things generally work in jdk9" > https://twitter.com/headius/status/928405094407827457 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22058) upgrade thrift dependency to 0.9.3.1 on branches 1.4, 1.3 and 1.2
[ https://issues.apache.org/jira/browse/HBASE-22058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801246#comment-16801246 ] Hudson commented on HBASE-22058: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1213 (See [https://builds.apache.org/job/HBase-1.2-IT/1213/]) HBASE-22058 upgrade thrift dependency to 0.9.3.1 (toffer: [https://github.com/apache/hbase/commit/e650ade86ccbf683b5d34a22069b977ba6b2b5e5]) * (edit) hbase-thrift/pom.xml * (edit) pom.xml > upgrade thrift dependency to 0.9.3.1 on branches 1.4, 1.3 and 1.2 > -- > > Key: HBASE-22058 > URL: https://issues.apache.org/jira/browse/HBASE-22058 > Project: HBase > Issue Type: Bug > Components: Thrift >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Major > Fix For: 1.4.10, 1.3.4, 1.2.12 > > Attachments: HBASE-22058-branch-1.4.patch, > HBASE-22058.branch-1.4.001.patch, HBASE-22058.branch-1.4.002.patch > > > Creating a separate Jira to do the backport since the .thrift files differ > between branch-1 and 1.4, 1.3. I backported the change in the pom.xml from > branch-1 and regenerated the thrift configs. > cc [~apurtell] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-15560) TinyLFU-based BlockCache
[
https://issues.apache.org/jira/browse/HBASE-15560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-15560:
---
Status: Patch Available (was: Open)
Rebased patch for master. Passes unit tests.
Also passes hbase-server unit tests with policy set to non-default "TinyLFU" in
test-resources hbase-site.xml, although that change is not included:
{code:java}
diff --git a/hbase-server/src/test/resources/hbase-site.xml
b/hbase-server/src/test/resources/hbase-site.xml
index 64a1964435..6b332603e1 100644
--- a/hbase-server/src/test/resources/hbase-site.xml
+++ b/hbase-server/src/test/resources/hbase-site.xml
@@ -158,4 +158,8 @@
hbase.hconnection.threads.keepalivetime
3
+
+ hfile.block.cache.policy
+ TinyLFU
+
{code}
> TinyLFU-based BlockCache
>
>
> Key: HBASE-15560
> URL: https://issues.apache.org/jira/browse/HBASE-15560
> Project: HBase
> Issue Type: Improvement
> Components: BlockCache
>Affects Versions: 2.0.0
>Reporter: Ben Manes
>Assignee: Ben Manes
>Priority: Major
> Fix For: 3.0.0, 1.6.0, 2.3.0
>
> Attachments: HBASE-15560.patch, HBASE-15560.patch, HBASE-15560.patch,
> HBASE-15560.patch, HBASE-15560.patch, HBASE-15560.patch, HBASE-15560.patch,
> HBASE-15560.patch, bc.hit.count, bc.miss.count, branch-1.tinylfu.txt, gets,
> run_ycsb_c.sh, run_ycsb_loading.sh, tinylfu.patch
>
>
> LruBlockCache uses the Segmented LRU (SLRU) policy to capture frequency and
> recency of the working set. It achieves concurrency by using an O( n )
> background thread to prioritize the entries and evict. Accessing an entry is
> O(1) by a hash table lookup, recording its logical access time, and setting a
> frequency flag. A write is performed in O(1) time by updating the hash table
> and triggering an async eviction thread. This provides ideal concurrency and
> minimizes the latencies by penalizing the thread instead of the caller.
> However the policy does not age the frequencies and may not be resilient to
> various workload patterns.
> W-TinyLFU ([research paper|http://arxiv.org/pdf/1512.00727.pdf]) records the
> frequency in a counting sketch, ages periodically by halving the counters,
> and orders entries by SLRU. An entry is discarded by comparing the frequency
> of the new arrival (candidate) to the SLRU's victim, and keeping the one with
> the highest frequency. This allows the operations to be performed in O(1)
> time and, though the use of a compact sketch, a much larger history is
> retained beyond the current working set. In a variety of real world traces
> the policy had [near optimal hit
> rates|https://github.com/ben-manes/caffeine/wiki/Efficiency].
> Concurrency is achieved by buffering and replaying the operations, similar to
> a write-ahead log. A read is recorded into a striped ring buffer and writes
> to a queue. The operations are applied in batches under a try-lock by an
> asynchronous thread, thereby track the usage pattern without incurring high
> latencies
> ([benchmarks|https://github.com/ben-manes/caffeine/wiki/Benchmarks#server-class]).
> In YCSB benchmarks the results were inconclusive. For a large cache (99% hit
> rates) the two caches have near identical throughput and latencies with
> LruBlockCache narrowly winning. At medium and small caches, TinyLFU had a
> 1-4% hit rate improvement and therefore lower latencies. The lack luster
> result is because a synthetic Zipfian distribution is used, which SLRU
> performs optimally. In a more varied, real-world workload we'd expect to see
> improvements by being able to make smarter predictions.
> The provided patch implements BlockCache using the
> [Caffeine|https://github.com/ben-manes/caffeine] caching library (see
> HighScalability
> [article|http://highscalability.com/blog/2016/1/25/design-of-a-modern-cache.html]).
> Edward Bortnikov and Eshcar Hillel have graciously provided guidance for
> evaluating this patch ([github
> branch|https://github.com/ben-manes/hbase/tree/tinylfu]).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-15560) TinyLFU-based BlockCache
[ https://issues.apache.org/jira/browse/HBASE-15560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15560: --- Attachment: HBASE-15560.patch > TinyLFU-based BlockCache > > > Key: HBASE-15560 > URL: https://issues.apache.org/jira/browse/HBASE-15560 > Project: HBase > Issue Type: Improvement > Components: BlockCache >Affects Versions: 2.0.0 >Reporter: Ben Manes >Assignee: Ben Manes >Priority: Major > Fix For: 3.0.0, 1.6.0, 2.3.0 > > Attachments: HBASE-15560.patch, HBASE-15560.patch, HBASE-15560.patch, > HBASE-15560.patch, HBASE-15560.patch, HBASE-15560.patch, HBASE-15560.patch, > HBASE-15560.patch, bc.hit.count, bc.miss.count, branch-1.tinylfu.txt, gets, > run_ycsb_c.sh, run_ycsb_loading.sh, tinylfu.patch > > > LruBlockCache uses the Segmented LRU (SLRU) policy to capture frequency and > recency of the working set. It achieves concurrency by using an O( n ) > background thread to prioritize the entries and evict. Accessing an entry is > O(1) by a hash table lookup, recording its logical access time, and setting a > frequency flag. A write is performed in O(1) time by updating the hash table > and triggering an async eviction thread. This provides ideal concurrency and > minimizes the latencies by penalizing the thread instead of the caller. > However the policy does not age the frequencies and may not be resilient to > various workload patterns. > W-TinyLFU ([research paper|http://arxiv.org/pdf/1512.00727.pdf]) records the > frequency in a counting sketch, ages periodically by halving the counters, > and orders entries by SLRU. An entry is discarded by comparing the frequency > of the new arrival (candidate) to the SLRU's victim, and keeping the one with > the highest frequency. This allows the operations to be performed in O(1) > time and, though the use of a compact sketch, a much larger history is > retained beyond the current working set. In a variety of real world traces > the policy had [near optimal hit > rates|https://github.com/ben-manes/caffeine/wiki/Efficiency]. > Concurrency is achieved by buffering and replaying the operations, similar to > a write-ahead log. A read is recorded into a striped ring buffer and writes > to a queue. The operations are applied in batches under a try-lock by an > asynchronous thread, thereby track the usage pattern without incurring high > latencies > ([benchmarks|https://github.com/ben-manes/caffeine/wiki/Benchmarks#server-class]). > In YCSB benchmarks the results were inconclusive. For a large cache (99% hit > rates) the two caches have near identical throughput and latencies with > LruBlockCache narrowly winning. At medium and small caches, TinyLFU had a > 1-4% hit rate improvement and therefore lower latencies. The lack luster > result is because a synthetic Zipfian distribution is used, which SLRU > performs optimally. In a more varied, real-world workload we'd expect to see > improvements by being able to make smarter predictions. > The provided patch implements BlockCache using the > [Caffeine|https://github.com/ben-manes/caffeine] caching library (see > HighScalability > [article|http://highscalability.com/blog/2016/1/25/design-of-a-modern-cache.html]). > Edward Bortnikov and Eshcar Hillel have graciously provided guidance for > evaluating this patch ([github > branch|https://github.com/ben-manes/hbase/tree/tinylfu]). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21619) Fix warning message caused by incorrect ternary operator evaluation
[
https://issues.apache.org/jira/browse/HBASE-21619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801237#comment-16801237
]
Hudson commented on HBASE-21619:
Results for branch branch-2
[build #1774 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1774/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1774//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1774//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1774//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix warning message caused by incorrect ternary operator evaluation
> ---
>
> Key: HBASE-21619
> URL: https://issues.apache.org/jira/browse/HBASE-21619
> Project: HBase
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.1.5, 2.2.1
>
> Attachments: HBASE-21619.master.001.patch
>
>
> {code:title=LoadIncrementalHFiles#doBulkLoad}
> LOG.warn(
> "Bulk load operation did not find any files to load in " +
> "directory " + hfofDir != null
> ? hfofDir.toUri().toString()
> : "" + ". Does it contain files in " +
> "subdirectories that correspond to column family names?");
> {code}
> JDK complains {{"Bulk load operation did not find any files to load in " +
> "directory " + hfofDir != null}} is always true, which is not what is
> intended, and that produces a wrong message.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22059) 2.1 addendum; (check-jar-contents) @ hbase-shaded-check-invariants fails
[ https://issues.apache.org/jira/browse/HBASE-22059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801235#comment-16801235 ] stack commented on HBASE-22059: --- bq. which shaded client stuff? the test that fails? or the thing that broke? branch-2.0 doesn't produce a shaded hbase client. It shows up first in branch-2.1. Whats in here was applied to branch-2.1 only. It was subsequently reverted and another approach applied up in the parent. I tried to do a narrative on my travails around failing RC building and nightlies timing out in the shaded client it test. Shout if not clear. > 2.1 addendum; (check-jar-contents) @ hbase-shaded-check-invariants fails > > > Key: HBASE-22059 > URL: https://issues.apache.org/jira/browse/HBASE-22059 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.1.4 > > Attachments: HBASE-22059.branch-2.1.001.patch > > > Exclusions in parent made for an inclusion in the shaded mapreduce client. > While I was playing around trying to figure which exclusion responsible, our > [~psomogyi] came up w/ attached addendum. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22058) upgrade thrift dependency to 0.9.3.1 on branches 1.4, 1.3 and 1.2
[
https://issues.apache.org/jira/browse/HBASE-22058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801195#comment-16801195
]
Hudson commented on HBASE-22058:
Results for branch branch-1.3
[build #695 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/695/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/695//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/695//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/695//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> upgrade thrift dependency to 0.9.3.1 on branches 1.4, 1.3 and 1.2
> --
>
> Key: HBASE-22058
> URL: https://issues.apache.org/jira/browse/HBASE-22058
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Reporter: Francis Liu
>Assignee: Francis Liu
>Priority: Major
> Fix For: 1.4.10, 1.3.4, 1.2.12
>
> Attachments: HBASE-22058-branch-1.4.patch,
> HBASE-22058.branch-1.4.001.patch, HBASE-22058.branch-1.4.002.patch
>
>
> Creating a separate Jira to do the backport since the .thrift files differ
> between branch-1 and 1.4, 1.3. I backported the change in the pom.xml from
> branch-1 and regenerated the thrift configs.
> cc [~apurtell]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22105) Snapshot of the table fails when region is transitioning
[
https://issues.apache.org/jira/browse/HBASE-22105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801165#comment-16801165
]
Xu Cang commented on HBASE-22105:
-
Nice finding. Agreed we should check both opening and closing.
"Closed' should be fine IMO.
> Snapshot of the table fails when region is transitioning
> -
>
> Key: HBASE-22105
> URL: https://issues.apache.org/jira/browse/HBASE-22105
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 2.1
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
>
> Region is moving(MoveRegionProcedure)
> Unassign Procedure completed on source node:-
> {code:java}
> 2019-03-18 06:56:12,027 INFO org.apache.hadoop.hbase.master.HMaster: balance
> hri=819809c43ee3f276be024a9e0a1bcce5, source=,22101,1552916141124,
> destination=,22101,1552916140986
> 2019-03-18 06:56:14,334 INFO
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler: Took xlock
> for pid=44, state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure
> hri=819809c43ee3f276be024a9e0a1bcce5, source=,22101,1552916141124,
> destination=,22101,1552916140986
> 2019-03-18 06:56:14,338 INFO
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: Initialized
> subprocedures=[{pid=51, ppid=44, state=RUNNABLE:REGION_TRANSITION_DISPATCH;
> UnassignProcedure table=t11, region=819809c43ee3f276be024a9e0a1bcce5,
> override=true, server=,22101,1552916141124}]
> 2019-03-18 06:56:14,344 INFO
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler: Took xlock
> for pid=51, ppid=44, state=RUNNABLE:REGION_TRANSITION_DISPATCH;
> UnassignProcedure table=t11, region=819809c43ee3f276be024a9e0a1bcce5,
> override=true, server=,22101,1552916141124
> 2019-03-18 06:56:14,349 INFO
> org.apache.hadoop.hbase.master.assignment.RegionStateStore: pid=51 updating
> hbase:meta row=819809c43ee3f276be024a9e0a1bcce5, regionState=CLOSING,
> regionLocation=,22101,1552916141124
> 2019-03-18 06:56:14,354 INFO
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure: Dispatch
> pid=51, ppid=44, state=RUNNABLE:REGION_TRANSITION_DISPATCH, locked=true;
> UnassignProcedure table=t11, region=819809c43ee3f276be024a9e0a1bcce5,
> override=true, server=,22101,1552916141124
> 2019-03-18 06:56:14,529 INFO
> org.apache.hadoop.hbase.master.assignment.RegionStateStore: pid=51 updating
> hbase:meta row=819809c43ee3f276be024a9e0a1bcce5, regionState=CLOSED
> {code}
> Assign procedure on destination started:-
> {code:java}
> 2019-03-18 06:56:14,545 INFO
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: Finished subprocedure
> pid=51, resume processing parent pid=44, state=RUNNABLE:MOVE_REGION_ASSIGN,
> locked=true; MoveRegionProcedure hri=819809c43ee3f276be024a9e0a1bcce5,
> source=,22101,1552916141124, destination=,22101,1552916140986
> 2019-03-18 06:56:14,545 INFO
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: Initialized
> subprocedures=[{pid=53, ppid=44, state=RUNNABLE:REGION_TRANSITION_QUEUE;
> AssignProcedure table=t11, region=819809c43ee3f276be024a9e0a1bcce5,
> target=,22101,1552916140986}]
> 2019-03-18 06:56:14,545 INFO
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: Finished pid=51,
> ppid=44, state=SUCCESS; UnassignProcedure table=t11,
> region=819809c43ee3f276be024a9e0a1bcce5, override=true,
> server=,22101,1552916141124 in 195msec, unfinishedSiblingCount=1
> 2019-03-18 06:56:14,550 INFO
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler: Took xlock
> for pid=53, ppid=44, state=RUNNABLE:REGION_TRANSITION_QUEUE; AssignProcedure
> table=t11, region=819809c43ee3f276be024a9e0a1bcce5,
> target=,22101,1552916140986
> 2019-03-18 06:56:14,555 INFO
> org.apache.hadoop.hbase.master.assignment.AssignProcedure: Starting pid=53,
> ppid=44, state=RUNNABLE:REGION_TRANSITION_QUEUE, locked=true; AssignProcedure
> table=t11, region=819809c43ee3f276be024a9e0a1bcce5,
> target=,22101,1552916140986; rit=OFFLINE,
> location=,22101,1552916140986; forceNewPlan=false, retain=false
> 2019-03-18 06:56:14,706 INFO
> org.apache.hadoop.hbase.master.assignment.RegionStateStore: pid=53 updating
> hbase:meta row=819809c43ee3f276be024a9e0a1bcce5, regionState=OPENING,
> regionLocation=,22101,1552916140986
> {code}
> Before Assign completes, snapshot procedure started and failed:-
> {code:java}
> 2019-03-18 06:56:14,788 INFO org.apache.hadoop.hbase.procedure.Procedure:
> Starting procedure
> 'cm-auto-6d40d404-5b14-4bd5-be8b-e67b374d6c22_HOURLY_2019-03-18_06-56-08_t11'
> 2019-03-18 06:56:14,830 INFO org.apache.hadoop.hbase.procedure.Procedure:
> Procedure
> 'cm-auto-6d40d404-5b14-4bd5-be8b-e67b374d6c22_HOURLY_2019-03-18_06-56-08_t11'
> execution completed
> 2019-03-18 06:56:14,830 INFO
>
[jira] [Created] (HBASE-22106) Log cause of the failure when coprocessor specification parsing fails.
Ankit Singhal created HBASE-22106:
-
Summary: Log cause of the failure when coprocessor specification
parsing fails.
Key: HBASE-22106
URL: https://issues.apache.org/jira/browse/HBASE-22106
Project: HBase
Issue Type: Bug
Components: logging
Affects Versions: 1.4.9
Reporter: Ankit Singhal
Assignee: Ankit Singhal
{code}
2019-02-15 22:56:33,008 ERROR [RS_OPEN_REGION-n79:16020-16]
regionserver.RegionCoprocessorHost: Malformed table coprocessor specification:
key=coprocessor$2, spec:
|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|805306366|
2019-02-16 00:39:33,651 ERROR [RS_OPEN_REGION-n76:16020-14]
regionserver.RegionCoprocessorHost: Malformed table coprocessor specification:
key=coprocessor$1, spec:
|org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|
2019-02-16 00:39:33,651 ERROR [RS_OPEN_REGION-n76:16020-19]
regionserver.RegionCoprocessorHost: Malformed table coprocessor specification:
key=coprocessor$1, spec:
|org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|
{code}
I checked the same coprocessor specification(logged in exception) with the code
and it is parsed successfully. And even after the restart , regionserver also
didn't complain about it.
I think we should be logging the cause of the exception while parsing table
descriptor for the coprocessor for better debugging.
https://github.com/apache/hbase/blob/branch-1.4/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java#L318
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-22105) Snapshot of the table fails when region is transitioning
Ankit Singhal created HBASE-22105:
-
Summary: Snapshot of the table fails when region is transitioning
Key: HBASE-22105
URL: https://issues.apache.org/jira/browse/HBASE-22105
Project: HBase
Issue Type: Bug
Components: snapshots
Affects Versions: 2.1
Reporter: Ankit Singhal
Assignee: Ankit Singhal
Region is moving(MoveRegionProcedure)
Unassign Procedure completed on source node:-
{code:java}
2019-03-18 06:56:12,027 INFO org.apache.hadoop.hbase.master.HMaster: balance
hri=819809c43ee3f276be024a9e0a1bcce5, source=,22101,1552916141124,
destination=,22101,1552916140986
2019-03-18 06:56:14,334 INFO
org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler: Took xlock
for pid=44, state=RUNNABLE:MOVE_REGION_UNASSIGN; MoveRegionProcedure
hri=819809c43ee3f276be024a9e0a1bcce5, source=,22101,1552916141124,
destination=,22101,1552916140986
2019-03-18 06:56:14,338 INFO
org.apache.hadoop.hbase.procedure2.ProcedureExecutor: Initialized
subprocedures=[{pid=51, ppid=44, state=RUNNABLE:REGION_TRANSITION_DISPATCH;
UnassignProcedure table=t11, region=819809c43ee3f276be024a9e0a1bcce5,
override=true, server=,22101,1552916141124}]
2019-03-18 06:56:14,344 INFO
org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler: Took xlock
for pid=51, ppid=44, state=RUNNABLE:REGION_TRANSITION_DISPATCH;
UnassignProcedure table=t11, region=819809c43ee3f276be024a9e0a1bcce5,
override=true, server=,22101,1552916141124
2019-03-18 06:56:14,349 INFO
org.apache.hadoop.hbase.master.assignment.RegionStateStore: pid=51 updating
hbase:meta row=819809c43ee3f276be024a9e0a1bcce5, regionState=CLOSING,
regionLocation=,22101,1552916141124
2019-03-18 06:56:14,354 INFO
org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure: Dispatch
pid=51, ppid=44, state=RUNNABLE:REGION_TRANSITION_DISPATCH, locked=true;
UnassignProcedure table=t11, region=819809c43ee3f276be024a9e0a1bcce5,
override=true, server=,22101,1552916141124
2019-03-18 06:56:14,529 INFO
org.apache.hadoop.hbase.master.assignment.RegionStateStore: pid=51 updating
hbase:meta row=819809c43ee3f276be024a9e0a1bcce5, regionState=CLOSED
{code}
Assign procedure on destination started:-
{code:java}
2019-03-18 06:56:14,545 INFO
org.apache.hadoop.hbase.procedure2.ProcedureExecutor: Finished subprocedure
pid=51, resume processing parent pid=44, state=RUNNABLE:MOVE_REGION_ASSIGN,
locked=true; MoveRegionProcedure hri=819809c43ee3f276be024a9e0a1bcce5,
source=,22101,1552916141124, destination=,22101,1552916140986
2019-03-18 06:56:14,545 INFO
org.apache.hadoop.hbase.procedure2.ProcedureExecutor: Initialized
subprocedures=[{pid=53, ppid=44, state=RUNNABLE:REGION_TRANSITION_QUEUE;
AssignProcedure table=t11, region=819809c43ee3f276be024a9e0a1bcce5,
target=,22101,1552916140986}]
2019-03-18 06:56:14,545 INFO
org.apache.hadoop.hbase.procedure2.ProcedureExecutor: Finished pid=51, ppid=44,
state=SUCCESS; UnassignProcedure table=t11,
region=819809c43ee3f276be024a9e0a1bcce5, override=true,
server=,22101,1552916141124 in 195msec, unfinishedSiblingCount=1
2019-03-18 06:56:14,550 INFO
org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler: Took xlock
for pid=53, ppid=44, state=RUNNABLE:REGION_TRANSITION_QUEUE; AssignProcedure
table=t11, region=819809c43ee3f276be024a9e0a1bcce5,
target=,22101,1552916140986
2019-03-18 06:56:14,555 INFO
org.apache.hadoop.hbase.master.assignment.AssignProcedure: Starting pid=53,
ppid=44, state=RUNNABLE:REGION_TRANSITION_QUEUE, locked=true; AssignProcedure
table=t11, region=819809c43ee3f276be024a9e0a1bcce5,
target=,22101,1552916140986; rit=OFFLINE,
location=,22101,1552916140986; forceNewPlan=false, retain=false
2019-03-18 06:56:14,706 INFO
org.apache.hadoop.hbase.master.assignment.RegionStateStore: pid=53 updating
hbase:meta row=819809c43ee3f276be024a9e0a1bcce5, regionState=OPENING,
regionLocation=,22101,1552916140986
{code}
Before Assign completes, snapshot procedure started and failed:-
{code:java}
2019-03-18 06:56:14,788 INFO org.apache.hadoop.hbase.procedure.Procedure:
Starting procedure
'cm-auto-6d40d404-5b14-4bd5-be8b-e67b374d6c22_HOURLY_2019-03-18_06-56-08_t11'
2019-03-18 06:56:14,830 INFO org.apache.hadoop.hbase.procedure.Procedure:
Procedure
'cm-auto-6d40d404-5b14-4bd5-be8b-e67b374d6c22_HOURLY_2019-03-18_06-56-08_t11'
execution completed
2019-03-18 06:56:14,830 INFO org.apache.hadoop.hbase.procedure.ZKProcedureUtil:
Clearing all znodes for procedure
cm-auto-6d40d404-5b14-4bd5-be8b-e67b374d6c22_HOURLY_2019-03-18_06-56-08_t11including
nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached
/hbase/online-snapshot/abort
2019-03-18 06:56:14,849 INFO
org.apache.hadoop.hbase.master.snapshot.EnabledTableSnapshotHandler: Done
waiting - online snapshot for
cm-auto-6d40d404-5b14-4bd5-be8b-e67b374d6c22_HOURLY_2019-03-18_06-56-08_t11
2019-03-18 06:56:14,962
[jira] [Commented] (HBASE-22097) Modify the description of split command in shell
[ https://issues.apache.org/jira/browse/HBASE-22097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801149#comment-16801149 ] Xu Cang commented on HBASE-22097: - +1 > Modify the description of split command in shell > > > Key: HBASE-22097 > URL: https://issues.apache.org/jira/browse/HBASE-22097 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Trivial > Attachments: HBASE-22097.master.001.patch > > > We can make the description of split command in shell better. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19222) update jruby to 9.1.17.0
[ https://issues.apache.org/jira/browse/HBASE-19222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801063#comment-16801063 ] Peter Somogyi commented on HBASE-19222: --- [~zghaobac], [~stack]: what are your opinions for 2.2 and 2.1? > update jruby to 9.1.17.0 > > > Key: HBASE-19222 > URL: https://issues.apache.org/jira/browse/HBASE-19222 > Project: HBase > Issue Type: Sub-task > Components: shell >Reporter: Sean Busbey >Assignee: Peter Somogyi >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: HBASE-19222.master.001.patch > > > JRuby's 9.1.14.0 release is described as "things generally work in jdk9" > https://twitter.com/headius/status/928405094407827457 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22094) Throw TableNotFoundException if table not exists in AsyncAdmin.compact
[ https://issues.apache.org/jira/browse/HBASE-22094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801052#comment-16801052 ] Sakthi commented on HBASE-22094: [~Apache9] how does this patch look? > Throw TableNotFoundException if table not exists in AsyncAdmin.compact > -- > > Key: HBASE-22094 > URL: https://issues.apache.org/jira/browse/HBASE-22094 > Project: HBase > Issue Type: Sub-task > Components: Admin >Reporter: Duo Zhang >Assignee: Sakthi >Priority: Major > Labels: beginner > Attachments: hbase-22094.master.001.patch, > hbase-22094.master.002.patch, hbase-22094.master.003.patch, > hbase-22094.master.004.patch > > > Now we will return successfully, which is not the same with the HBaseAdmin. > For NORMAL compaction, we should throw TableNotFoundException if the > locations is empty. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19222) update jruby to 9.1.17.0
[
https://issues.apache.org/jira/browse/HBASE-19222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801048#comment-16801048
]
Hadoop QA commented on HBASE-19222:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
35s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
33s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 38s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}205m
30s{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}264m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-19222 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12963638/HBASE-19222.master.001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit shadedjars
hadoopcheck xml compile |
| uname | Linux a8f95ab30a19 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 6de8a37b63 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/16531/testReport/ |
| Max. process+thread count | 5017 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/16531/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> update jruby to 9.1.17.0
>
>
> Key: HBASE-19222
> URL: https://issues.apache.org/jira/browse/HBASE-19222
> Project: HBase
> Issue Type: Sub-task
> Components: shell
>Reporter: Sean Busbey
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-19222.master.001.patch
>
>
> JRuby's 9.1.14.0 release is described as
[jira] [Updated] (HBASE-22058) upgrade thrift dependency to 0.9.3.1 on branches 1.4, 1.3 and 1.2
[ https://issues.apache.org/jira/browse/HBASE-22058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-22058: Summary: upgrade thrift dependency to 0.9.3.1 on branches 1.4, 1.3 and 1.2 (was: upgrade thrift dependency to 0.9.3.1 on branch-1.4 and branch-1.3) > upgrade thrift dependency to 0.9.3.1 on branches 1.4, 1.3 and 1.2 > -- > > Key: HBASE-22058 > URL: https://issues.apache.org/jira/browse/HBASE-22058 > Project: HBase > Issue Type: Bug > Components: Thrift >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Major > Fix For: 1.4.10, 1.3.4, 1.2.12 > > Attachments: HBASE-22058-branch-1.4.patch, > HBASE-22058.branch-1.4.001.patch, HBASE-22058.branch-1.4.002.patch > > > Creating a separate Jira to do the backport since the .thrift files differ > between branch-1 and 1.4, 1.3. I backported the change in the pom.xml from > branch-1 and regenerated the thrift configs. > cc [~apurtell] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22058) upgrade thrift dependency to 0.9.3.1 on branch-1.4 and branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-22058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801031#comment-16801031 ] Francis Liu commented on HBASE-22058: - Pushed to branch-1.2 as well. > upgrade thrift dependency to 0.9.3.1 on branch-1.4 and branch-1.3 > -- > > Key: HBASE-22058 > URL: https://issues.apache.org/jira/browse/HBASE-22058 > Project: HBase > Issue Type: Bug > Components: Thrift >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Major > Fix For: 1.4.10, 1.3.4 > > Attachments: HBASE-22058-branch-1.4.patch, > HBASE-22058.branch-1.4.001.patch, HBASE-22058.branch-1.4.002.patch > > > Creating a separate Jira to do the backport since the .thrift files differ > between branch-1 and 1.4, 1.3. I backported the change in the pom.xml from > branch-1 and regenerated the thrift configs. > cc [~apurtell] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22058) upgrade thrift dependency to 0.9.3.1 on branches 1.4, 1.3 and 1.2
[ https://issues.apache.org/jira/browse/HBASE-22058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801031#comment-16801031 ] Francis Liu edited comment on HBASE-22058 at 3/25/19 8:26 PM: -- Cherry-picked to branch-1.2 as well. was (Author: toffer): Pushed to branch-1.2 as well. > upgrade thrift dependency to 0.9.3.1 on branches 1.4, 1.3 and 1.2 > -- > > Key: HBASE-22058 > URL: https://issues.apache.org/jira/browse/HBASE-22058 > Project: HBase > Issue Type: Bug > Components: Thrift >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Major > Fix For: 1.4.10, 1.3.4, 1.2.12 > > Attachments: HBASE-22058-branch-1.4.patch, > HBASE-22058.branch-1.4.001.patch, HBASE-22058.branch-1.4.002.patch > > > Creating a separate Jira to do the backport since the .thrift files differ > between branch-1 and 1.4, 1.3. I backported the change in the pom.xml from > branch-1 and regenerated the thrift configs. > cc [~apurtell] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22058) upgrade thrift dependency to 0.9.3.1 on branches 1.4, 1.3 and 1.2
[ https://issues.apache.org/jira/browse/HBASE-22058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-22058: Resolution: Fixed Status: Resolved (was: Patch Available) > upgrade thrift dependency to 0.9.3.1 on branches 1.4, 1.3 and 1.2 > -- > > Key: HBASE-22058 > URL: https://issues.apache.org/jira/browse/HBASE-22058 > Project: HBase > Issue Type: Bug > Components: Thrift >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Major > Fix For: 1.4.10, 1.3.4, 1.2.12 > > Attachments: HBASE-22058-branch-1.4.patch, > HBASE-22058.branch-1.4.001.patch, HBASE-22058.branch-1.4.002.patch > > > Creating a separate Jira to do the backport since the .thrift files differ > between branch-1 and 1.4, 1.3. I backported the change in the pom.xml from > branch-1 and regenerated the thrift configs. > cc [~apurtell] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22058) upgrade thrift dependency to 0.9.3.1 on branch-1.4 and branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-22058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-22058: Fix Version/s: 1.2.12 > upgrade thrift dependency to 0.9.3.1 on branch-1.4 and branch-1.3 > -- > > Key: HBASE-22058 > URL: https://issues.apache.org/jira/browse/HBASE-22058 > Project: HBase > Issue Type: Bug > Components: Thrift >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Major > Fix For: 1.4.10, 1.3.4, 1.2.12 > > Attachments: HBASE-22058-branch-1.4.patch, > HBASE-22058.branch-1.4.001.patch, HBASE-22058.branch-1.4.002.patch > > > Creating a separate Jira to do the backport since the .thrift files differ > between branch-1 and 1.4, 1.3. I backported the change in the pom.xml from > branch-1 and regenerated the thrift configs. > cc [~apurtell] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22052) pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove redunant version specifications
[
https://issues.apache.org/jira/browse/HBASE-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801009#comment-16801009
]
Hudson commented on HBASE-22052:
Results for branch branch-2.2
[build #130 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/130/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/130//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/130//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/130//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove
> redunant version specifications
> ---
>
> Key: HBASE-22052
> URL: https://issues.apache.org/jira/browse/HBASE-22052
> Project: HBase
> Issue Type: Task
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.0.5, 2.3.0, 2.1.4
>
> Attachments: HBASE-22052-branch-2.1.003.addendum2.txt,
> HBASE-22052.2.1.003.addendum1.txt, HBASE-22052.branch-2.0.001.patch,
> HBASE-22052.branch-2.0.002.patch, HBASE-22052.branch-2.0.002.patch,
> HBASE-22052.branch-2.0.003.patch, HBASE-22052.branch-2.0.004.patch,
> HBASE-22052.branch-2.1.001.patch, HBASE-22052.branch-2.1.002.patch,
> HBASE-22052.branch-2.1.003.patch, HBASE-22052.branch-2.2.001.patch,
> HBASE-22052.branch-2.2.002.patch, nothing_change.patch, nothing_change.patch
>
>
> Working on HBASE-22029, where we fail compile of hbase-it module four hours
> into an RC build, it looks like transitive include of jersey-core is the
> culprit. hadoop3 profile does a bunch of jersey-core exclusions. This issue
> is about having hadoop2 profile and hadoop3 profiles match around jersey-core
> treatment. Some miscellaneous cleanups are also done.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21619) Fix warning message caused by incorrect ternary operator evaluation
[
https://issues.apache.org/jira/browse/HBASE-21619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi updated HBASE-21619:
--
Resolution: Fixed
Fix Version/s: 2.2.1
2.1.5
2.0.6
2.3.0
3.0.0
Status: Resolved (was: Patch Available)
> Fix warning message caused by incorrect ternary operator evaluation
> ---
>
> Key: HBASE-21619
> URL: https://issues.apache.org/jira/browse/HBASE-21619
> Project: HBase
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.1.5, 2.2.1
>
> Attachments: HBASE-21619.master.001.patch
>
>
> {code:title=LoadIncrementalHFiles#doBulkLoad}
> LOG.warn(
> "Bulk load operation did not find any files to load in " +
> "directory " + hfofDir != null
> ? hfofDir.toUri().toString()
> : "" + ". Does it contain files in " +
> "subdirectories that correspond to column family names?");
> {code}
> JDK complains {{"Bulk load operation did not find any files to load in " +
> "directory " + hfofDir != null}} is always true, which is not what is
> intended, and that produces a wrong message.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21619) Fix warning message caused by incorrect ternary operator evaluation
[
https://issues.apache.org/jira/browse/HBASE-21619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800990#comment-16800990
]
Hadoop QA commented on HBASE-21619:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} HBASE-21619 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.8.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-21619 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12952363/HBASE-21619.master.001.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/16532/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Fix warning message caused by incorrect ternary operator evaluation
> ---
>
> Key: HBASE-21619
> URL: https://issues.apache.org/jira/browse/HBASE-21619
> Project: HBase
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Trivial
> Attachments: HBASE-21619.master.001.patch
>
>
> {code:title=LoadIncrementalHFiles#doBulkLoad}
> LOG.warn(
> "Bulk load operation did not find any files to load in " +
> "directory " + hfofDir != null
> ? hfofDir.toUri().toString()
> : "" + ". Does it contain files in " +
> "subdirectories that correspond to column family names?");
> {code}
> JDK complains {{"Bulk load operation did not find any files to load in " +
> "directory " + hfofDir != null}} is always true, which is not what is
> intended, and that produces a wrong message.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21619) Fix warning message caused by incorrect ternary operator evaluation
[
https://issues.apache.org/jira/browse/HBASE-21619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800985#comment-16800985
]
Peter Somogyi commented on HBASE-21619:
---
+1, let me commit this patch.
> Fix warning message caused by incorrect ternary operator evaluation
> ---
>
> Key: HBASE-21619
> URL: https://issues.apache.org/jira/browse/HBASE-21619
> Project: HBase
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Trivial
> Attachments: HBASE-21619.master.001.patch
>
>
> {code:title=LoadIncrementalHFiles#doBulkLoad}
> LOG.warn(
> "Bulk load operation did not find any files to load in " +
> "directory " + hfofDir != null
> ? hfofDir.toUri().toString()
> : "" + ". Does it contain files in " +
> "subdirectories that correspond to column family names?");
> {code}
> JDK complains {{"Bulk load operation did not find any files to load in " +
> "directory " + hfofDir != null}} is always true, which is not what is
> intended, and that produces a wrong message.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22058) upgrade thrift dependency to 0.9.3.1 on branch-1.4 and branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-22058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800980#comment-16800980 ] Hudson commented on HBASE-22058: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #532 (See [https://builds.apache.org/job/HBase-1.3-IT/532/]) HBASE-22058 upgrade thrift dependency to 0.9.3.1 (toffer: [https://github.com/apache/hbase/commit/7a49a7f97d1f3c16266c18b5ba7cd80ff1b33ed9]) * (edit) hbase-thrift/pom.xml * (edit) pom.xml > upgrade thrift dependency to 0.9.3.1 on branch-1.4 and branch-1.3 > -- > > Key: HBASE-22058 > URL: https://issues.apache.org/jira/browse/HBASE-22058 > Project: HBase > Issue Type: Bug > Components: Thrift >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Major > Fix For: 1.4.10, 1.3.4 > > Attachments: HBASE-22058-branch-1.4.patch, > HBASE-22058.branch-1.4.001.patch, HBASE-22058.branch-1.4.002.patch > > > Creating a separate Jira to do the backport since the .thrift files differ > between branch-1 and 1.4, 1.3. I backported the change in the pom.xml from > branch-1 and regenerated the thrift configs. > cc [~apurtell] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22052) pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove redunant version specifications
[
https://issues.apache.org/jira/browse/HBASE-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800976#comment-16800976
]
Hudson commented on HBASE-22052:
Results for branch master
[build #884 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/884/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/884//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/884//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/884//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove
> redunant version specifications
> ---
>
> Key: HBASE-22052
> URL: https://issues.apache.org/jira/browse/HBASE-22052
> Project: HBase
> Issue Type: Task
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.0.5, 2.3.0, 2.1.4
>
> Attachments: HBASE-22052-branch-2.1.003.addendum2.txt,
> HBASE-22052.2.1.003.addendum1.txt, HBASE-22052.branch-2.0.001.patch,
> HBASE-22052.branch-2.0.002.patch, HBASE-22052.branch-2.0.002.patch,
> HBASE-22052.branch-2.0.003.patch, HBASE-22052.branch-2.0.004.patch,
> HBASE-22052.branch-2.1.001.patch, HBASE-22052.branch-2.1.002.patch,
> HBASE-22052.branch-2.1.003.patch, HBASE-22052.branch-2.2.001.patch,
> HBASE-22052.branch-2.2.002.patch, nothing_change.patch, nothing_change.patch
>
>
> Working on HBASE-22029, where we fail compile of hbase-it module four hours
> into an RC build, it looks like transitive include of jersey-core is the
> culprit. hadoop3 profile does a bunch of jersey-core exclusions. This issue
> is about having hadoop2 profile and hadoop3 profiles match around jersey-core
> treatment. Some miscellaneous cleanups are also done.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22058) upgrade thrift dependency to 0.9.3.1 on branch-1.4 and branch-1.3
[
https://issues.apache.org/jira/browse/HBASE-22058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800969#comment-16800969
]
Hudson commented on HBASE-22058:
Results for branch branch-1.4
[build #714 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/714/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/714//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/714//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/714//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> upgrade thrift dependency to 0.9.3.1 on branch-1.4 and branch-1.3
> --
>
> Key: HBASE-22058
> URL: https://issues.apache.org/jira/browse/HBASE-22058
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Reporter: Francis Liu
>Assignee: Francis Liu
>Priority: Major
> Fix For: 1.4.10, 1.3.4
>
> Attachments: HBASE-22058-branch-1.4.patch,
> HBASE-22058.branch-1.4.001.patch, HBASE-22058.branch-1.4.002.patch
>
>
> Creating a separate Jira to do the backport since the .thrift files differ
> between branch-1 and 1.4, 1.3. I backported the change in the pom.xml from
> branch-1 and regenerated the thrift configs.
> cc [~apurtell]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22058) upgrade thrift dependency to 0.9.3.1 on branch-1.4 and branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-22058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800960#comment-16800960 ] Sean Busbey commented on HBASE-22058: - yeah, pom-only update to 0.9.3.1 would be great for branch-1.2 thanks! > upgrade thrift dependency to 0.9.3.1 on branch-1.4 and branch-1.3 > -- > > Key: HBASE-22058 > URL: https://issues.apache.org/jira/browse/HBASE-22058 > Project: HBase > Issue Type: Bug > Components: Thrift >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Major > Fix For: 1.4.10, 1.3.4 > > Attachments: HBASE-22058-branch-1.4.patch, > HBASE-22058.branch-1.4.001.patch, HBASE-22058.branch-1.4.002.patch > > > Creating a separate Jira to do the backport since the .thrift files differ > between branch-1 and 1.4, 1.3. I backported the change in the pom.xml from > branch-1 and regenerated the thrift configs. > cc [~apurtell] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22052) pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove redunant version specifications
[
https://issues.apache.org/jira/browse/HBASE-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800958#comment-16800958
]
Hudson commented on HBASE-22052:
Results for branch branch-2
[build #1773 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1773/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1773//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1773//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1773//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove
> redunant version specifications
> ---
>
> Key: HBASE-22052
> URL: https://issues.apache.org/jira/browse/HBASE-22052
> Project: HBase
> Issue Type: Task
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.0.5, 2.3.0, 2.1.4
>
> Attachments: HBASE-22052-branch-2.1.003.addendum2.txt,
> HBASE-22052.2.1.003.addendum1.txt, HBASE-22052.branch-2.0.001.patch,
> HBASE-22052.branch-2.0.002.patch, HBASE-22052.branch-2.0.002.patch,
> HBASE-22052.branch-2.0.003.patch, HBASE-22052.branch-2.0.004.patch,
> HBASE-22052.branch-2.1.001.patch, HBASE-22052.branch-2.1.002.patch,
> HBASE-22052.branch-2.1.003.patch, HBASE-22052.branch-2.2.001.patch,
> HBASE-22052.branch-2.2.002.patch, nothing_change.patch, nothing_change.patch
>
>
> Working on HBASE-22029, where we fail compile of hbase-it module four hours
> into an RC build, it looks like transitive include of jersey-core is the
> culprit. hadoop3 profile does a bunch of jersey-core exclusions. This issue
> is about having hadoop2 profile and hadoop3 profiles match around jersey-core
> treatment. Some miscellaneous cleanups are also done.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22059) 2.1 addendum; (check-jar-contents) @ hbase-shaded-check-invariants fails
[
https://issues.apache.org/jira/browse/HBASE-22059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800946#comment-16800946
]
Sean Busbey commented on HBASE-22059:
-
{quote} I'd pushed on branch-2.0 but it doesn't have the shaded client stuff.
{quote}
which shaded client stuff? the test that fails? or the thing that broke?
> 2.1 addendum; (check-jar-contents) @ hbase-shaded-check-invariants fails
>
>
> Key: HBASE-22059
> URL: https://issues.apache.org/jira/browse/HBASE-22059
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.1.4
>
> Attachments: HBASE-22059.branch-2.1.001.patch
>
>
> Exclusions in parent made for an inclusion in the shaded mapreduce client.
> While I was playing around trying to figure which exclusion responsible, our
> [~psomogyi] came up w/ attached addendum.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-22104) Remove Hadoop 2.7 from next minor releases
Sean Busbey created HBASE-22104: --- Summary: Remove Hadoop 2.7 from next minor releases Key: HBASE-22104 URL: https://issues.apache.org/jira/browse/HBASE-22104 Project: HBase Issue Type: Task Components: hadoop2 Affects Versions: 3.0.0, 1.6.0, 2.3.0 Reporter: Sean Busbey Hadoop 2.7 is now EOM ([common-dev@hadoop "\[DISCUSS\] branch 2.7 EoL"|https://lists.apache.org/thread.html/d1f98c2c386f2f4b980489b543db3d0bb7bdb94ea12f8fc5a90f527b@%3Ccommon-dev.hadoop.apache.org%3E]) and has an active licensing issue (HADOOP-13794) Let's go ahead and axe it from the next minor releases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22058) upgrade thrift dependency to 0.9.3.1 on branch-1.4 and branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-22058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800900#comment-16800900 ] Francis Liu commented on HBASE-22058: - Pushed to 1.4 and cherry-picked to 1.3. I went with the pom-only change since that was the intent of the thrift release. Thanks for the review guys! [~busbey] since the change is fairly small do you want this backported to 1.2 as well? > upgrade thrift dependency to 0.9.3.1 on branch-1.4 and branch-1.3 > -- > > Key: HBASE-22058 > URL: https://issues.apache.org/jira/browse/HBASE-22058 > Project: HBase > Issue Type: Bug > Components: Thrift >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Major > Fix For: 1.4.10, 1.3.4 > > Attachments: HBASE-22058-branch-1.4.patch, > HBASE-22058.branch-1.4.001.patch, HBASE-22058.branch-1.4.002.patch > > > Creating a separate Jira to do the backport since the .thrift files differ > between branch-1 and 1.4, 1.3. I backported the change in the pom.xml from > branch-1 and regenerated the thrift configs. > cc [~apurtell] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22058) upgrade thrift dependency to 0.9.3.1 on branch-1.4 and branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-22058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-22058: Summary: upgrade thrift dependency to 0.9.3.1 on branch-1.4 and branch-1.3 (was: backport HBASE-HBASE-21791 (Upgrade thrift dependency to 0.12.0) to 1.4 and 1.3) > upgrade thrift dependency to 0.9.3.1 on branch-1.4 and branch-1.3 > -- > > Key: HBASE-22058 > URL: https://issues.apache.org/jira/browse/HBASE-22058 > Project: HBase > Issue Type: Bug > Components: Thrift >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Major > Fix For: 1.4.10, 1.3.4 > > Attachments: HBASE-22058-branch-1.4.patch, > HBASE-22058.branch-1.4.001.patch, HBASE-22058.branch-1.4.002.patch > > > Creating a separate Jira to do the backport since the .thrift files differ > between branch-1 and 1.4, 1.3. I backported the change in the pom.xml from > branch-1 and regenerated the thrift configs. > cc [~apurtell] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22074) Should use procedure store to persist the state in reportRegionStateTransition
[
https://issues.apache.org/jira/browse/HBASE-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800898#comment-16800898
]
Hadoop QA commented on HBASE-22074:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
32s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 4m
44s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 52s{color}
| {color:red} hbase-client generated 2 new + 127 unchanged - 2 fixed = 129
total (was 129) {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 2m 43s{color}
| {color:red} hbase-server generated 17 new + 177 unchanged - 17 fixed = 194
total (was 194) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch passed checkstyle in hbase-protocol-shaded
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} hbase-client: The patch generated 0 new + 295
unchanged - 3 fixed = 295 total (was 298) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
16s{color} | {color:green} The patch passed checkstyle in hbase-server {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
30s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 45s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
35s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
8s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}139m
5s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} |
[jira] [Commented] (HBASE-22100) False positive for error prone warnings in pre commit job
[
https://issues.apache.org/jira/browse/HBASE-22100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800894#comment-16800894
]
Sean Busbey commented on HBASE-22100:
-
fyi, I think I went over how to change the diff parsing in HBASE-21895.
> False positive for error prone warnings in pre commit job
> -
>
> Key: HBASE-22100
> URL: https://issues.apache.org/jira/browse/HBASE-22100
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Duo Zhang
>Priority: Major
>
> https://builds.apache.org/job/PreCommit-HBASE-Build/16516/artifact/patchprocess/branch-compile-javac-hbase-client.txt
> {noformat}
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,69]
> [UnusedVariable] The parameter 'updateCachedLocation' is never read.
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,42]
> [UnusedVariable] The parameter 'error' is never read.
> {noformat}
> https://builds.apache.org/job/PreCommit-HBASE-Build/16516/artifact/patchprocess/patch-compile-javac-hbase-client.txt
> {noformat}
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,42]
> [UnusedVariable] The parameter 'error' is never read.
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,69]
> [UnusedVariable] The parameter 'updateCachedLocation' is never read.
> {noformat}
> And the output is 1 new and 1 fixed, the new one is
> {noformat}
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,69]
> [UnusedVariable] The parameter 'updateCachedLocation' is never read.
> {noformat}
> I think here we should report nothing, as it is just an order change...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22100) False positive for error prone warnings in pre commit job
[
https://issues.apache.org/jira/browse/HBASE-22100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800892#comment-16800892
]
Sean Busbey commented on HBASE-22100:
-
If you think this is a thing we can fix by updating the default javac output
diff set up in yetus, then yeah a yetus jira makes sense.
Personally, I'd try to fix this in our personality first by customizing how the
javac output is diffed specifically for our build. Knowing what the fix looks
like should make it more obvious if this is a problem with how we do things or
a general weakness in yetus's default plugin.
> False positive for error prone warnings in pre commit job
> -
>
> Key: HBASE-22100
> URL: https://issues.apache.org/jira/browse/HBASE-22100
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Duo Zhang
>Priority: Major
>
> https://builds.apache.org/job/PreCommit-HBASE-Build/16516/artifact/patchprocess/branch-compile-javac-hbase-client.txt
> {noformat}
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,69]
> [UnusedVariable] The parameter 'updateCachedLocation' is never read.
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,42]
> [UnusedVariable] The parameter 'error' is never read.
> {noformat}
> https://builds.apache.org/job/PreCommit-HBASE-Build/16516/artifact/patchprocess/patch-compile-javac-hbase-client.txt
> {noformat}
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,42]
> [UnusedVariable] The parameter 'error' is never read.
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,69]
> [UnusedVariable] The parameter 'updateCachedLocation' is never read.
> {noformat}
> And the output is 1 new and 1 fixed, the new one is
> {noformat}
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,69]
> [UnusedVariable] The parameter 'updateCachedLocation' is never read.
> {noformat}
> I think here we should report nothing, as it is just an order change...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19222) update jruby to 9.1.17.0
[ https://issues.apache.org/jira/browse/HBASE-19222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800884#comment-16800884 ] Sean Busbey commented on HBASE-19222: - I personally don't like changing dependency versions in maintenance releases, but I'd defer to RMs for those lines. > update jruby to 9.1.17.0 > > > Key: HBASE-19222 > URL: https://issues.apache.org/jira/browse/HBASE-19222 > Project: HBase > Issue Type: Sub-task > Components: shell >Reporter: Sean Busbey >Assignee: Peter Somogyi >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: HBASE-19222.master.001.patch > > > JRuby's 9.1.14.0 release is described as "things generally work in jdk9" > https://twitter.com/headius/status/928405094407827457 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21911) Move getUserPermissions from regionserver to master
[
https://issues.apache.org/jira/browse/HBASE-21911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800868#comment-16800868
]
Hadoop QA commented on HBASE-21911:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
31s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 5m
55s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 2m 53s{color}
| {color:red} hbase-server generated 15 new + 179 unchanged - 15 fixed = 194
total (was 194) {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 56s{color}
| {color:red} hbase-thrift generated 3 new + 24 unchanged - 3 fixed = 27 total
(was 27) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch passed checkstyle in hbase-protocol-shaded
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} The patch passed checkstyle in hbase-client {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} hbase-server: The patch generated 0 new + 106
unchanged - 3 fixed = 106 total (was 109) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} The patch passed checkstyle in hbase-thrift {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
40s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 29s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
2m 0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
35s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
15s{color} | {color:green} hbase-client in the patch passed. {color} |
|
[jira] [Updated] (HBASE-19222) update jruby to 9.1.17.0
[ https://issues.apache.org/jira/browse/HBASE-19222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-19222: -- Status: Patch Available (was: Open) > update jruby to 9.1.17.0 > > > Key: HBASE-19222 > URL: https://issues.apache.org/jira/browse/HBASE-19222 > Project: HBase > Issue Type: Sub-task > Components: shell >Reporter: Sean Busbey >Assignee: Peter Somogyi >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: HBASE-19222.master.001.patch > > > JRuby's 9.1.14.0 release is described as "things generally work in jdk9" > https://twitter.com/headius/status/928405094407827457 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19222) update jruby to 9.1.17.0
[ https://issues.apache.org/jira/browse/HBASE-19222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800846#comment-16800846 ] Peter Somogyi commented on HBASE-19222: --- Roger! One liner patch attached. Since this dependency upgrade does not require any code modification I think we can target branch-2.0+. WDYT? > update jruby to 9.1.17.0 > > > Key: HBASE-19222 > URL: https://issues.apache.org/jira/browse/HBASE-19222 > Project: HBase > Issue Type: Sub-task > Components: shell >Reporter: Sean Busbey >Assignee: Peter Somogyi >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: HBASE-19222.master.001.patch > > > JRuby's 9.1.14.0 release is described as "things generally work in jdk9" > https://twitter.com/headius/status/928405094407827457 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19222) update jruby to 9.1.17.0
[ https://issues.apache.org/jira/browse/HBASE-19222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-19222: -- Attachment: HBASE-19222.master.001.patch > update jruby to 9.1.17.0 > > > Key: HBASE-19222 > URL: https://issues.apache.org/jira/browse/HBASE-19222 > Project: HBase > Issue Type: Sub-task > Components: shell >Reporter: Sean Busbey >Assignee: Peter Somogyi >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: HBASE-19222.master.001.patch > > > JRuby's 9.1.14.0 release is described as "things generally work in jdk9" > https://twitter.com/headius/status/928405094407827457 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22053) zookeeper URL links in documentation are failing with 404
[
https://issues.apache.org/jira/browse/HBASE-22053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800841#comment-16800841
]
Hadoop QA commented on HBASE-22053:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 5m
29s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 5m
1s{color} | {color:blue} patch has no errors when building the reference guide.
See footer for rendered docs, which you should manually inspect. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}200m
14s{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}226m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-22053 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12963620/HBASE-22053.master.002.patch
|
| Optional Tests | dupname asflicense javac javadoc unit refguide xml |
| uname | Linux 2e09f7dc5861 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 44f8abd5c6 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| refguide |
https://builds.apache.org/job/PreCommit-HBASE-Build/16527/artifact/patchprocess/branch-site/book.html
|
| refguide |
https://builds.apache.org/job/PreCommit-HBASE-Build/16527/artifact/patchprocess/patch-site/book.html
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/16527/testReport/ |
| Max. process+thread count | 5408 (vs. ulimit of 1) |
| modules | C: hbase-common . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/16527/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> zookeeper URL links in documentation are failing with 404
> -
>
> Key: HBASE-22053
> URL: https://issues.apache.org/jira/browse/HBASE-22053
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Subrat Mishra
>Assignee: Subrat Mishra
>Priority: Minor
> Attachments:
[jira] [Updated] (HBASE-19222) update jruby to 9.1.17.0
[ https://issues.apache.org/jira/browse/HBASE-19222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-19222: -- Fix Version/s: (was: 1.5.1) > update jruby to 9.1.17.0 > > > Key: HBASE-19222 > URL: https://issues.apache.org/jira/browse/HBASE-19222 > Project: HBase > Issue Type: Sub-task > Components: shell >Reporter: Sean Busbey >Assignee: Peter Somogyi >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > JRuby's 9.1.14.0 release is described as "things generally work in jdk9" > https://twitter.com/headius/status/928405094407827457 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19222) update jruby to 9.1.17.0
[ https://issues.apache.org/jira/browse/HBASE-19222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-19222: -- Summary: update jruby to 9.1.17.0 (was: update jruby to 9.1.14.0+) > update jruby to 9.1.17.0 > > > Key: HBASE-19222 > URL: https://issues.apache.org/jira/browse/HBASE-19222 > Project: HBase > Issue Type: Sub-task > Components: shell >Reporter: Sean Busbey >Assignee: Peter Somogyi >Priority: Major > Fix For: 3.0.0, 2.3.0, 1.5.1 > > > JRuby's 9.1.14.0 release is described as "things generally work in jdk9" > https://twitter.com/headius/status/928405094407827457 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19222) update jruby to 9.1.14.0+
[ https://issues.apache.org/jira/browse/HBASE-19222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800805#comment-16800805 ] Sean Busbey commented on HBASE-19222: - I'd recommend breaking off any branch-1 stuff to another jira. the Ruby version on branch-1 is currently Ruby 1.8 (per defaults in that version of JRuby) and that version doesn't work at all in JRuby 9k. The Ruby version in JRuby 9k is a couple of major incompatible Ruby versions newer. The jiras for the branch-2 move to JRuby 9k should explain the problem. > update jruby to 9.1.14.0+ > - > > Key: HBASE-19222 > URL: https://issues.apache.org/jira/browse/HBASE-19222 > Project: HBase > Issue Type: Sub-task > Components: shell >Reporter: Sean Busbey >Assignee: Peter Somogyi >Priority: Major > Fix For: 3.0.0, 2.3.0, 1.5.1 > > > JRuby's 9.1.14.0 release is described as "things generally work in jdk9" > https://twitter.com/headius/status/928405094407827457 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22057) Impose upper-bound on size of ZK ops sent in a single multi()
[
https://issues.apache.org/jira/browse/HBASE-22057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800799#comment-16800799
]
Hadoop QA commented on HBASE-22057:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
6s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 22s{color}
| {color:red} hbase-zookeeper generated 4 new + 46 unchanged - 4 fixed = 50
total (was 50) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} hbase-zookeeper: The patch generated 2 new + 0
unchanged - 0 fixed = 2 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
8s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 57s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
46s{color} | {color:green} hbase-zookeeper in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-22057 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12963633/HBASE-22057.004.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux ed4094f69996 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 6de8a37b63 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| javac |
https://builds.apache.org/job/PreCommit-HBASE-Build/16530/artifact/patchprocess/diff-compile-javac-hbase-zookeeper.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/16530/artifact/patchprocess/diff-checkstyle-hbase-zookeeper.txt
|
| Test Results |
[jira] [Assigned] (HBASE-19222) update jruby to 9.1.14.0+
[ https://issues.apache.org/jira/browse/HBASE-19222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi reassigned HBASE-19222: - Assignee: Peter Somogyi > update jruby to 9.1.14.0+ > - > > Key: HBASE-19222 > URL: https://issues.apache.org/jira/browse/HBASE-19222 > Project: HBase > Issue Type: Sub-task > Components: shell >Reporter: Sean Busbey >Assignee: Peter Somogyi >Priority: Major > Fix For: 3.0.0, 2.3.0, 1.5.1 > > > JRuby's 9.1.14.0 release is described as "things generally work in jdk9" > https://twitter.com/headius/status/928405094407827457 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19222) update jruby to 9.1.14.0+
[ https://issues.apache.org/jira/browse/HBASE-19222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800785#comment-16800785 ] Peter Somogyi commented on HBASE-19222: --- Yes, on branch-1 it could be problematic. I successfully ran hbase-shell tests on master with JRuby 9.1.17.0 without any problem. Let me take a look to branch-1 just in case the upgrade is simple there. > update jruby to 9.1.14.0+ > - > > Key: HBASE-19222 > URL: https://issues.apache.org/jira/browse/HBASE-19222 > Project: HBase > Issue Type: Sub-task > Components: shell >Reporter: Sean Busbey >Priority: Major > Fix For: 3.0.0, 2.3.0, 1.5.1 > > > JRuby's 9.1.14.0 release is described as "things generally work in jdk9" > https://twitter.com/headius/status/928405094407827457 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22057) Impose upper-bound on size of ZK ops sent in a single multi()
[
https://issues.apache.org/jira/browse/HBASE-22057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800730#comment-16800730
]
Josh Elser commented on HBASE-22057:
{quote}Javadoc of getMaxMultiSizeLimit and submitBatchedMultiOrSequential refer
to the number of operations instead of size
{quote}
Oops! Thanks for the catch.
> Impose upper-bound on size of ZK ops sent in a single multi()
> -
>
> Key: HBASE-22057
> URL: https://issues.apache.org/jira/browse/HBASE-22057
> Project: HBase
> Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 1.6.0, 2.2.0
>
> Attachments: HBASE-22057.001.patch, HBASE-22057.002.patch,
> HBASE-22057.003.patch
>
>
> In {{ZKUtil#multiOrSequential}}, we accept a list of {{ZKUtilOp}}'s to pass
> down to the {{ZooKeeper#multi(Iterable)}} method.
> One problem with this approach is that we may generate a large list of ZNodes
> to mutate in one batch which exceeds the allowable client package length,
> specified by {{jute.maxbuffer}}.
> This problem can manifest when we have a large number of WALs to replicate,
> queued in ZooKeeper, from a disabled peer. When that peer is dropped, the RS
> would submit deletes of those queued WALs. The RS will see ConnectionLoss for
> the resulting {{multi()}} calls it tries to make, because we are sending too
> large of a client message (because we're trying to delete too many WALs at
> once). The result (at least in branch-1 ish versions) is that the RS aborts
> after exceeding the ZK retries (as this operation will never succeed).
> A simple fix would be to impose a maximum number of Ops to run in a single
> batch inside ZKUtil, and split apart the caller-submitted batch into smaller
> chunks. Before we make such a change, I do need to make sure that we don't
> have any expectations on atomicity of the operations. I'm not sure what ZK
> provides here -- for the above example, splitting up batches of deletes is
> not an issue, but there could be issues with batches of creates where we only
> apply some.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22057) Impose upper-bound on size of ZK ops sent in a single multi()
[
https://issues.apache.org/jira/browse/HBASE-22057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800780#comment-16800780
]
Peter Somogyi commented on HBASE-22057:
---
+1
Thanks for fixes!
> Impose upper-bound on size of ZK ops sent in a single multi()
> -
>
> Key: HBASE-22057
> URL: https://issues.apache.org/jira/browse/HBASE-22057
> Project: HBase
> Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 1.6.0, 2.2.0
>
> Attachments: HBASE-22057.001.patch, HBASE-22057.002.patch,
> HBASE-22057.003.patch, HBASE-22057.004.patch
>
>
> In {{ZKUtil#multiOrSequential}}, we accept a list of {{ZKUtilOp}}'s to pass
> down to the {{ZooKeeper#multi(Iterable)}} method.
> One problem with this approach is that we may generate a large list of ZNodes
> to mutate in one batch which exceeds the allowable client package length,
> specified by {{jute.maxbuffer}}.
> This problem can manifest when we have a large number of WALs to replicate,
> queued in ZooKeeper, from a disabled peer. When that peer is dropped, the RS
> would submit deletes of those queued WALs. The RS will see ConnectionLoss for
> the resulting {{multi()}} calls it tries to make, because we are sending too
> large of a client message (because we're trying to delete too many WALs at
> once). The result (at least in branch-1 ish versions) is that the RS aborts
> after exceeding the ZK retries (as this operation will never succeed).
> A simple fix would be to impose a maximum number of Ops to run in a single
> batch inside ZKUtil, and split apart the caller-submitted batch into smaller
> chunks. Before we make such a change, I do need to make sure that we don't
> have any expectations on atomicity of the operations. I'm not sure what ZK
> provides here -- for the above example, splitting up batches of deletes is
> not an issue, but there could be issues with batches of creates where we only
> apply some.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22058) backport HBASE-HBASE-21791 (Upgrade thrift dependency to 0.12.0) to 1.4 and 1.3
[ https://issues.apache.org/jira/browse/HBASE-22058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800770#comment-16800770 ] Sean Busbey commented on HBASE-22058: - +1, also on either approach (though I personally prefer the pom-only change) you're correct the 0.9.3.1 release only changed the java library impacted by the CVE; it didn't even update the thrift version. no code gen should be needed. > backport HBASE-HBASE-21791 (Upgrade thrift dependency to 0.12.0) to 1.4 and > 1.3 > --- > > Key: HBASE-22058 > URL: https://issues.apache.org/jira/browse/HBASE-22058 > Project: HBase > Issue Type: Bug > Components: Thrift >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Major > Fix For: 1.4.10, 1.3.4 > > Attachments: HBASE-22058-branch-1.4.patch, > HBASE-22058.branch-1.4.001.patch, HBASE-22058.branch-1.4.002.patch > > > Creating a separate Jira to do the backport since the .thrift files differ > between branch-1 and 1.4, 1.3. I backported the change in the pom.xml from > branch-1 and regenerated the thrift configs. > cc [~apurtell] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22057) Impose upper-bound on size of ZK ops sent in a single multi()
[
https://issues.apache.org/jira/browse/HBASE-22057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800764#comment-16800764
]
Josh Elser commented on HBASE-22057:
.004 is a javadoc-only change to address the feedback from Peter (thanks!)
> Impose upper-bound on size of ZK ops sent in a single multi()
> -
>
> Key: HBASE-22057
> URL: https://issues.apache.org/jira/browse/HBASE-22057
> Project: HBase
> Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 1.6.0, 2.2.0
>
> Attachments: HBASE-22057.001.patch, HBASE-22057.002.patch,
> HBASE-22057.003.patch, HBASE-22057.004.patch
>
>
> In {{ZKUtil#multiOrSequential}}, we accept a list of {{ZKUtilOp}}'s to pass
> down to the {{ZooKeeper#multi(Iterable)}} method.
> One problem with this approach is that we may generate a large list of ZNodes
> to mutate in one batch which exceeds the allowable client package length,
> specified by {{jute.maxbuffer}}.
> This problem can manifest when we have a large number of WALs to replicate,
> queued in ZooKeeper, from a disabled peer. When that peer is dropped, the RS
> would submit deletes of those queued WALs. The RS will see ConnectionLoss for
> the resulting {{multi()}} calls it tries to make, because we are sending too
> large of a client message (because we're trying to delete too many WALs at
> once). The result (at least in branch-1 ish versions) is that the RS aborts
> after exceeding the ZK retries (as this operation will never succeed).
> A simple fix would be to impose a maximum number of Ops to run in a single
> batch inside ZKUtil, and split apart the caller-submitted batch into smaller
> chunks. Before we make such a change, I do need to make sure that we don't
> have any expectations on atomicity of the operations. I'm not sure what ZK
> provides here -- for the above example, splitting up batches of deletes is
> not an issue, but there could be issues with batches of creates where we only
> apply some.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22057) Impose upper-bound on size of ZK ops sent in a single multi()
[
https://issues.apache.org/jira/browse/HBASE-22057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-22057:
---
Attachment: HBASE-22057.004.patch
> Impose upper-bound on size of ZK ops sent in a single multi()
> -
>
> Key: HBASE-22057
> URL: https://issues.apache.org/jira/browse/HBASE-22057
> Project: HBase
> Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 1.6.0, 2.2.0
>
> Attachments: HBASE-22057.001.patch, HBASE-22057.002.patch,
> HBASE-22057.003.patch, HBASE-22057.004.patch
>
>
> In {{ZKUtil#multiOrSequential}}, we accept a list of {{ZKUtilOp}}'s to pass
> down to the {{ZooKeeper#multi(Iterable)}} method.
> One problem with this approach is that we may generate a large list of ZNodes
> to mutate in one batch which exceeds the allowable client package length,
> specified by {{jute.maxbuffer}}.
> This problem can manifest when we have a large number of WALs to replicate,
> queued in ZooKeeper, from a disabled peer. When that peer is dropped, the RS
> would submit deletes of those queued WALs. The RS will see ConnectionLoss for
> the resulting {{multi()}} calls it tries to make, because we are sending too
> large of a client message (because we're trying to delete too many WALs at
> once). The result (at least in branch-1 ish versions) is that the RS aborts
> after exceeding the ZK retries (as this operation will never succeed).
> A simple fix would be to impose a maximum number of Ops to run in a single
> batch inside ZKUtil, and split apart the caller-submitted batch into smaller
> chunks. Before we make such a change, I do need to make sure that we don't
> have any expectations on atomicity of the operations. I'm not sure what ZK
> provides here -- for the above example, splitting up batches of deletes is
> not an issue, but there could be issues with batches of creates where we only
> apply some.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21802) Release 2.0.5
[ https://issues.apache.org/jira/browse/HBASE-21802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800759#comment-16800759 ] stack edited comment on HBASE-21802 at 3/25/19 2:49 PM: + Made RC using script in HBASE-21935 + Put up a VOTE. It lasted a week and passed w/ 6 votes. + Closed the vote. + Tagged rel/2.0.5 and pushed tag upstream. + Released the maven repo staged artifact. + Removed release/hbase/2.0.4 + svn mv dev/hbase/hbase-2.0.5RC1 release/hbase/2.0.5 under URL: https://dist.apache.org/repos/dist + svn commit. + Update the download page pointing to 2.0.5 instead of 2.0.4. + Release 2.0.5 in JIRA. Waiting till morning to send announcement while artifacts propagate and until show on download page. was (Author: stack): + Closed the vote. + Tagged rel/2.0.5 and pushed tag upstream. + Released the maven repo staged artifact. + Removed release/hbase/2.0.4 + svn mv dev/hbase/hbase-2.0.5RC1 release/hbase/2.0.5 under URL: https://dist.apache.org/repos/dist + svn commit. + Update the download page pointing to 2.0.5 instead of 2.0.4. + Release 2.0.5 in JIRA. Waiting till morning to send announcement while artifacts propagate and until show on download page. > Release 2.0.5 > - > > Key: HBASE-21802 > URL: https://issues.apache.org/jira/browse/HBASE-21802 > Project: HBase > Issue Type: Umbrella >Reporter: Duo Zhang >Assignee: stack >Priority: Major > Attachments: Screen Shot 2019-02-05 at 8.11.33 PM.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21802) Release 2.0.5
[ https://issues.apache.org/jira/browse/HBASE-21802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800759#comment-16800759 ] stack commented on HBASE-21802: --- + Closed the vote. + Tagged rel/2.0.5 and pushed tag upstream. + Released the maven repo staged artifact. + Removed release/hbase/2.0.4 + svn mv dev/hbase/hbase-2.0.5RC1 release/hbase/2.0.5 under URL: https://dist.apache.org/repos/dist + svn commit. + Update the download page pointing to 2.0.5 instead of 2.0.4. + Release 2.0.5 in JIRA. Waiting till morning to send announcement while artifacts propagate and until show on download page. > Release 2.0.5 > - > > Key: HBASE-21802 > URL: https://issues.apache.org/jira/browse/HBASE-21802 > Project: HBase > Issue Type: Umbrella >Reporter: Duo Zhang >Assignee: stack >Priority: Major > Attachments: Screen Shot 2019-02-05 at 8.11.33 PM.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21935) Replace make_rc.sh with customized spark/dev/create-release
[
https://issues.apache.org/jira/browse/HBASE-21935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800748#comment-16800748
]
stack commented on HBASE-21935:
---
This ran end-to-end yesterday successfully but exited with a +1 so said it
FAILED. Trying to figure where the errcode comes from Pasting here in
meantime in case others run this script in meantime.
{code}
...
Command: /opt/hbase-rm/release-build.sh build
[52/4964]
+ echo 'Log file: build.log'
Log file: build.log
+ /opt/hbase-rm/release-build.sh build
+ local EC=0
+ '[' 0 '!=' 0 ']'
+ should_build publish
+ local WHAT=publish
+ '[' -z '' ']'
+ run_silent 'Publishing release' publish.log /opt/hbase-rm/release-build.sh
publish-release
+ local 'BANNER=Publishing release'
+ local LOG_FILE=publish.log
+ shift 2
+ echo
+ echo '= Publishing release'
= Publishing release
+ echo 'Command: /opt/hbase-rm/release-build.sh' publish-release
Command: /opt/hbase-rm/release-build.sh publish-release
+ echo 'Log file: publish.log'
Log file: publish.log
+ /opt/hbase-rm/release-build.sh publish-release
+ local EC=1
+ '[' 1 '!=' 0 ']'
+ echo 'Command FAILED. Check full logs for details.'
Command FAILED. Check full logs for details.
+ tail publish.log
...
{code}
> Replace make_rc.sh with customized spark/dev/create-release
> ---
>
> Key: HBASE-21935
> URL: https://issues.apache.org/jira/browse/HBASE-21935
> Project: HBase
> Issue Type: Task
> Components: rm
>Reporter: stack
>Assignee: stack
>Priority: Minor
> Labels: rm
> Attachments: HBASE-21935.branch-2.0.001.patch,
> HBASE-21935.branch-2.0.002.patch, HBASE-21935.branch-2.0.003.patch,
> HBASE-21935.branch-2.0.004.patch, HBASE-21935.branch-2.0.005.patch,
> HBASE-21935.branch-2.0.006.patch, HBASE-21935.branch-2.0.007.patch,
> HBASE-21935.branch-2.0.008.patch, HBASE-21935.branch-2.0.009.patch,
> HBASE-21935.branch-2.0.010.patch, HBASE-21935.branch-2.0.011.patch,
> HBASE-21935.branch-2.0.012.patch, HBASE-21935.branch-2.0.013.patch,
> HBASE-21935.branch-2.0.014.patch, HBASE-21935.branch-2.0.015.patch,
> HBASE-21935.branch-2.0.016.patch, HBASE-21935.branch-2.0.017.patch,
> HBASE-21935.branch-2.0.018.patch, HBASE-21935.branch-2.1.001.patch,
> HBASE-21935.branch-2.1.002.patch, HBASE-21935.branch-2.1.003.patch,
> HBASE-21935.branch-2.1.004.patch, HBASE-21935.branch-2.1.005.patch,
> HBASE-21935.branch-2.1.006.patch, HBASE-21935.branch-2.1.007.patch
>
>
> The spark/dev/create-release is more comprehensive than our hokey make_rc.sh
> script. It codifies the bulk of the RM process from tagging, version-setting,
> building, signing, and pushing. It does it in a container so environment is
> same each time. It has a bunch of spark-specifics as is -- naturally -- but
> should be possible to pull it around to suit hbase RM'ing. It'd save a bunch
> of time and would allow us to get to a place where RM'ing is canned,
> evolvable, and consistent.
> I've been hacking on the tooling before the filing of this JIRA and was
> polluting branch-2.0 w/ tagging and reverts. Let me make a branch named for
> this JIRA to play with (There is a dry-run flag but it too needs work...).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22075) Potential data loss when MOB compaction fails
[ https://issues.apache.org/jira/browse/HBASE-22075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-22075: -- Fix Version/s: (was: 2.0.5) 2.0.6 > Potential data loss when MOB compaction fails > - > > Key: HBASE-22075 > URL: https://issues.apache.org/jira/browse/HBASE-22075 > Project: HBase > Issue Type: Bug > Components: mob >Affects Versions: 2.1.0, 2.0.0, 2.0.1, 2.1.1, 2.0.2, 2.0.3, 2.1.2, 2.0.4, > 2.1.3 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Critical > Labels: mob > Fix For: 2.2.0, 2.1.4, 2.0.6 > > Attachments: HBASE-22075-v1.patch > > > When MOB compaction fails during last step (bulk load of a newly created > reference file) there is a high chance of a data loss due to partially loaded > reference file, cells of which refer to (now) non-existent MOB file. The > newly created MOB file is deleted automatically in case of a MOB compaction > failure, but some cells with the references to this file might be loaded to > HBase. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22013) SpaceQuota utilization issue with region replicas enabled
[
https://issues.apache.org/jira/browse/HBASE-22013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800705#comment-16800705
]
Josh Elser commented on HBASE-22013:
{quote}Can you provide permission to work on the issue,so that I can assign
this to me?
{quote}
Done.
> SpaceQuota utilization issue with region replicas enabled
> -
>
> Key: HBASE-22013
> URL: https://issues.apache.org/jira/browse/HBASE-22013
> Project: HBase
> Issue Type: Bug
>Reporter: Ajeet Rai
>Assignee: Uma Maheswari
>Priority: Major
>
> Space Quota: Space Quota Issue: If a table is created with region replica
> then quota calculation is not happening
> Steps:
> 1: Create a table with 100 regions with region replica 3
> 2: Observe that 'hbase:quota' table doesn't have entry of usage for this
> table So In UI only policy Limit and Policy is shown but not Usage and State.
> Reason:
> It looks like File system utilization core is sending data of 100 reasons
> but not the size of region replicas.
> But in quota observer chore, it is considering total region(actual regions+
> replica reasons)
> So the ratio of reported regions is less then configured
> percentRegionsReportedThreshold.
> SO quota calculation is not happening
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22103) HDFS-13209 in Hadoop 3.3.0 breaks asyncwal
[
https://issues.apache.org/jira/browse/HBASE-22103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800701#comment-16800701
]
Gabor Bota commented on HBASE-22103:
My plan is to backport my change from trunk (3.3) to 3.0.3 and run the tests
with HBase. The guava update is HADOOP-15960 if anyone interested.
> HDFS-13209 in Hadoop 3.3.0 breaks asyncwal
> --
>
> Key: HBASE-22103
> URL: https://issues.apache.org/jira/browse/HBASE-22103
> Project: HBase
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HBASE-22103.master.001.patch
>
>
> HDFS-13209 added an additional parameter to {{DistributedFileSystem.create}}
> and broke asyncwal.
> {noformat}
> 2019-03-25 12:19:21,061 ERROR [Listener at localhost/54758]
> asyncfs.FanOutOneBlockAsyncDFSOutputHelper(562): Couldn't properly initialize
> access to HDFS internals. Please update your WAL Provider to not make use of
> the 'asyncfs' provider. See HBASE-16110 for more information.
> java.lang.NoSuchMethodException:
> org.apache.hadoop.hdfs.protocol.ClientProtocol.create(java.lang.String,
> org.apache.hadoop.fs.permission.FsPermission, java.lang.String,
> org.apache.hadoop.io.EnumSetWritable, boolean, short, long,
> [Lorg.apache.hadoop.crypto.CryptoProtocolVersion;)
> at java.lang.Class.getMethod(Class.java:1786)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createFileCreator2(FanOutOneBlockAsyncDFSOutputHelper.java:513)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createFileCreator(FanOutOneBlockAsyncDFSOutputHelper.java:530)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:557)
> at
> org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:51)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:169)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166)
> at
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:105)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createAsyncWriter(AsyncFSWAL.java:663)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:669)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:126)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:813)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:519)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:460)
> at
> org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL.newWAL(TestAsyncFSWAL.java:72)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractTestFSWAL.testWALComparator(AbstractTestFSWAL.java:194)
> {noformat}
> Credit: this bug was found by [~gabor.bota]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-22013) SpaceQuota utilization issue with region replicas enabled
[ https://issues.apache.org/jira/browse/HBASE-22013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser reassigned HBASE-22013: -- Assignee: Uma Maheswari > SpaceQuota utilization issue with region replicas enabled > - > > Key: HBASE-22013 > URL: https://issues.apache.org/jira/browse/HBASE-22013 > Project: HBase > Issue Type: Bug >Reporter: Ajeet Rai >Assignee: Uma Maheswari >Priority: Major > > Space Quota: Space Quota Issue: If a table is created with region replica > then quota calculation is not happening > Steps: > 1: Create a table with 100 regions with region replica 3 > 2: Observe that 'hbase:quota' table doesn't have entry of usage for this > table So In UI only policy Limit and Policy is shown but not Usage and State. > Reason: > It looks like File system utilization core is sending data of 100 reasons > but not the size of region replicas. > But in quota observer chore, it is considering total region(actual regions+ > replica reasons) > So the ratio of reported regions is less then configured > percentRegionsReportedThreshold. > SO quota calculation is not happening -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22054) Space Quota: Compaction is not working for super user in case of NO_WRITES_COMPACTIONS
[
https://issues.apache.org/jira/browse/HBASE-22054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800703#comment-16800703
]
Josh Elser commented on HBASE-22054:
Sorry, early post:
{code:java}
this.compactionThroughputController =
CompactionThroughputControllerFactory.create(server, newConf);
// We change this atomically here instead of reloading the config in order
that upstream
// would be the only one with the flexibility to reload the config.
this.conf.reloadConfiguration();{code}
In CompactSplit.java in master, we have this. Reinitializing the Superusers
after reloading the configuration would make sense, but not so much down here
in CompactSplit. We should be doing that in the RegionServer (for all
RS-internal services).
{quote}+ // if regionserver's constructor wasn't called for some reason, then
Superusers{quote}
What are the circumstances which create this? Seems like we should fix the
situations where the superuser code is not initialized, not work around it like
this.
{code:java}
-if (spaceQuotaManager != null &&
-
spaceQuotaManager.areCompactionsDisabled(region.getTableDescriptor().getTableName()))
{
+if (!Superusers.isSuperUser(RpcServer.getRequestUser().orElse(null))
+&& spaceQuotaManager != null && spaceQuotaManager
+.areCompactionsDisabled(region.getTableDescriptor().getTableName()))
{{code}
I'd like to see a DEBUG message here explaining that we are still running a
compaction because a superuser requested it.
On that line of thinking, I'm not sure if we've fully thought through what
"NO_WRITES_COMPACTIONS" should mean. Do we want to disallow the system from
running compactions over this data or are we just preventing users from
submitting compactions? Looking back at HBASE-17978, I think the original
intent was for this policy was to prevent user-submitted compactions. We should
clarify the hbase book to make that more clear.
> Space Quota: Compaction is not working for super user in case of
> NO_WRITES_COMPACTIONS
> --
>
> Key: HBASE-22054
> URL: https://issues.apache.org/jira/browse/HBASE-22054
> Project: HBase
> Issue Type: Bug
>Reporter: Ajeet Rai
>Assignee: Sakthi
>Priority: Minor
> Attachments: hbase-22054.master.001.patch,
> hbase-22054.master.002.patch, hbase-22054.master.003.patch
>
>
> Space Quota: Compaction is not working for super user. Compaction command is
> issued successfully at client but actually compaction is not happening.
> In debug log below message is printed:
> as an active space quota violation policy disallows compaction.
> Reference:
>
> [https://lists.apache.org/thread.html/d09aa7abaacf1f0be9d59fa9260515ddc0c17ac0aba9cc0f2ac569bf@%3Cuser.hbase.apache.org%3E]
> Actually in requestCompactionInternal method of CompactSplit class ,there is
> no check for super user and compcations are disallowed
> {noformat}
> RegionServerSpaceQuotaManager spaceQuotaManager =
> this.server.getRegionServerSpaceQuotaManager();
> if (spaceQuotaManager != null &&
>
> spaceQuotaManager.areCompactionsDisabled(region.getTableDescriptor().getTableName()))
> {
> String reason = "Ignoring compaction request for " + region +
> " as an active space quota violation " + " policy disallows
> compactions.";
> tracker.notExecuted(store, reason);
> completeTracker.completed(store);
> LOG.debug(reason);
> return;
> }
> {noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22074) Should use procedure store to persist the state in reportRegionStateTransition
[ https://issues.apache.org/jira/browse/HBASE-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-22074: -- Attachment: HBASE-22074-v6.patch > Should use procedure store to persist the state in reportRegionStateTransition > -- > > Key: HBASE-22074 > URL: https://issues.apache.org/jira/browse/HBASE-22074 > Project: HBase > Issue Type: Bug > Components: amv2, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: HBASE-22074-v1.patch, HBASE-22074-v2.patch, > HBASE-22074-v3.patch, HBASE-22074-v3.patch, HBASE-22074-v4.patch, > HBASE-22074-v5.patch, HBASE-22074-v6.patch, HBASE-22074.patch > > > For now we will update the meta region directly. This may cause lots of > problems and after a bunch of fixes, we still can not solve the problem in > HBASE-22060. > So maybe the approach itself is not a good choice, let's try another way to > see if it could work better. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22103) HDFS-13209 in Hadoop 3.3.0 breaks asyncwal
[
https://issues.apache.org/jira/browse/HBASE-22103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800697#comment-16800697
]
Wei-Chiu Chuang commented on HBASE-22103:
-
[~Apache9] certainly. Supporting Hadoop 3.3 is not my priority now. Gabor is
updating Guava in Hadoop and found this issue.
> HDFS-13209 in Hadoop 3.3.0 breaks asyncwal
> --
>
> Key: HBASE-22103
> URL: https://issues.apache.org/jira/browse/HBASE-22103
> Project: HBase
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HBASE-22103.master.001.patch
>
>
> HDFS-13209 added an additional parameter to {{DistributedFileSystem.create}}
> and broke asyncwal.
> {noformat}
> 2019-03-25 12:19:21,061 ERROR [Listener at localhost/54758]
> asyncfs.FanOutOneBlockAsyncDFSOutputHelper(562): Couldn't properly initialize
> access to HDFS internals. Please update your WAL Provider to not make use of
> the 'asyncfs' provider. See HBASE-16110 for more information.
> java.lang.NoSuchMethodException:
> org.apache.hadoop.hdfs.protocol.ClientProtocol.create(java.lang.String,
> org.apache.hadoop.fs.permission.FsPermission, java.lang.String,
> org.apache.hadoop.io.EnumSetWritable, boolean, short, long,
> [Lorg.apache.hadoop.crypto.CryptoProtocolVersion;)
> at java.lang.Class.getMethod(Class.java:1786)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createFileCreator2(FanOutOneBlockAsyncDFSOutputHelper.java:513)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createFileCreator(FanOutOneBlockAsyncDFSOutputHelper.java:530)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:557)
> at
> org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:51)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:169)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166)
> at
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:105)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createAsyncWriter(AsyncFSWAL.java:663)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:669)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:126)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:813)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:519)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:460)
> at
> org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL.newWAL(TestAsyncFSWAL.java:72)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractTestFSWAL.testWALComparator(AbstractTestFSWAL.java:194)
> {noformat}
> Credit: this bug was found by [~gabor.bota]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-19222) update jruby to 9.1.14.0+
[ https://issues.apache.org/jira/browse/HBASE-19222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800693#comment-16800693 ] Sean Busbey edited comment on HBASE-19222 at 3/25/19 1:48 PM: -- a fix version of 1.5.1+ for branch-1s is a problem, since upgrading JRuby from branch-1's super old JRuby 1.6 to the 9k line is going to be very disruptive. I personally would like to see us avoid that for any branch-1s, but that may not be possible if we really need JDKs after JDK8 to work on a future branch-1 release. was (Author: busbey): a fix version of 1.5.1+ is a problem, since upgrading JRuby from branch-1's super old JRuby 1.6 to the 9k line is going to be very disruptive. I personally would like to see us avoid that for any branch-1s, but that may not be possible if we really need JDKs after JDK8 to work on a future branch-1 release. > update jruby to 9.1.14.0+ > - > > Key: HBASE-19222 > URL: https://issues.apache.org/jira/browse/HBASE-19222 > Project: HBase > Issue Type: Sub-task > Components: shell >Reporter: Sean Busbey >Priority: Major > Fix For: 3.0.0, 2.3.0, 1.5.1 > > > JRuby's 9.1.14.0 release is described as "things generally work in jdk9" > https://twitter.com/headius/status/928405094407827457 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19222) update jruby to 9.1.14.0+
[ https://issues.apache.org/jira/browse/HBASE-19222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800693#comment-16800693 ] Sean Busbey commented on HBASE-19222: - a fix version of 1.5.1+ is a problem, since upgrading JRuby from branch-1's super old JRuby 1.6 to the 9k line is going to be very disruptive. I personally would like to see us avoid that for any branch-1s, but that may not be possible if we really need JDKs after JDK8 to work on a future branch-1 release. > update jruby to 9.1.14.0+ > - > > Key: HBASE-19222 > URL: https://issues.apache.org/jira/browse/HBASE-19222 > Project: HBase > Issue Type: Sub-task > Components: shell >Reporter: Sean Busbey >Priority: Major > Fix For: 3.0.0, 2.3.0, 1.5.1 > > > JRuby's 9.1.14.0 release is described as "things generally work in jdk9" > https://twitter.com/headius/status/928405094407827457 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21512) Introduce an AsyncClusterConnection and replace the usage of ClusterConnection
[
https://issues.apache.org/jira/browse/HBASE-21512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800694#comment-16800694
]
Hudson commented on HBASE-21512:
Results for branch HBASE-21512
[build #149 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/149/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/149//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/149//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/149//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Introduce an AsyncClusterConnection and replace the usage of ClusterConnection
> --
>
> Key: HBASE-21512
> URL: https://issues.apache.org/jira/browse/HBASE-21512
> Project: HBase
> Issue Type: Umbrella
>Reporter: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
>
> At least for the RSProcedureDispatcher, with CompletableFuture we do not need
> to set a delay and use a thread pool any more, which could reduce the
> resource usage and also the latency.
> Once this is done, I think we can remove the ClusterConnection completely,
> and start to rewrite the old sync client based on the async client, which
> could reduce the code base a lot for our client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22054) Space Quota: Compaction is not working for super user in case of NO_WRITES_COMPACTIONS
[
https://issues.apache.org/jira/browse/HBASE-22054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800690#comment-16800690
]
Josh Elser commented on HBASE-22054:
{code:java}
diff --git
a/hbase-common/src/main/java/org/apache/hadoop/hbase/security/Superusers.java
b/hbase-common/src/main/java/org/apache/hadoop/hbase/security/Superusers.java
index
b5566e65be77706b6cf39873ac0ca6fffcec7d0d..8135205110c3da4ae481f3650e0e8eafeac782c2
100644
---
a/hbase-common/src/main/java/org/apache/hadoop/hbase/security/Superusers.java
+++
b/hbase-common/src/main/java/org/apache/hadoop/hbase/security/Superusers.java
@@ -91,6 +91,10 @@ public final class Superusers {
throw new IllegalStateException("Super users/super groups lists"
+ " have not been initialized properly.");
}
+if (user == null) {
+ LOG.trace("isSuperUser check received for a null user. Returned false");
+ return false;
+}{code}
I don't think we should allow callers to provide a null user. Is there a reason
we have to support this? Should be an IllegalArgumentException.
> Space Quota: Compaction is not working for super user in case of
> NO_WRITES_COMPACTIONS
> --
>
> Key: HBASE-22054
> URL: https://issues.apache.org/jira/browse/HBASE-22054
> Project: HBase
> Issue Type: Bug
>Reporter: Ajeet Rai
>Assignee: Sakthi
>Priority: Minor
> Attachments: hbase-22054.master.001.patch,
> hbase-22054.master.002.patch, hbase-22054.master.003.patch
>
>
> Space Quota: Compaction is not working for super user. Compaction command is
> issued successfully at client but actually compaction is not happening.
> In debug log below message is printed:
> as an active space quota violation policy disallows compaction.
> Reference:
>
> [https://lists.apache.org/thread.html/d09aa7abaacf1f0be9d59fa9260515ddc0c17ac0aba9cc0f2ac569bf@%3Cuser.hbase.apache.org%3E]
> Actually in requestCompactionInternal method of CompactSplit class ,there is
> no check for super user and compcations are disallowed
> {noformat}
> RegionServerSpaceQuotaManager spaceQuotaManager =
> this.server.getRegionServerSpaceQuotaManager();
> if (spaceQuotaManager != null &&
>
> spaceQuotaManager.areCompactionsDisabled(region.getTableDescriptor().getTableName()))
> {
> String reason = "Ignoring compaction request for " + region +
> " as an active space quota violation " + " policy disallows
> compactions.";
> tracker.notExecuted(store, reason);
> completeTracker.completed(store);
> LOG.debug(reason);
> return;
> }
> {noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22052) pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove redunant version specifications
[ https://issues.apache.org/jira/browse/HBASE-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-22052: -- Release Note: Fixed awkward dependency issue that prevented site building. HBase 2.1.4 shipped with the first version of this patch. The patch was subsequently reopened when it was found that it failed a special IT test that was using the shaded hbase mapreduce client. The IT test was failing because an old htrace lib was excluded in the original committed patch. > pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove > redunant version specifications > --- > > Key: HBASE-22052 > URL: https://issues.apache.org/jira/browse/HBASE-22052 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.3.0, 2.1.4 > > Attachments: HBASE-22052-branch-2.1.003.addendum2.txt, > HBASE-22052.2.1.003.addendum1.txt, HBASE-22052.branch-2.0.001.patch, > HBASE-22052.branch-2.0.002.patch, HBASE-22052.branch-2.0.002.patch, > HBASE-22052.branch-2.0.003.patch, HBASE-22052.branch-2.0.004.patch, > HBASE-22052.branch-2.1.001.patch, HBASE-22052.branch-2.1.002.patch, > HBASE-22052.branch-2.1.003.patch, HBASE-22052.branch-2.2.001.patch, > HBASE-22052.branch-2.2.002.patch, nothing_change.patch, nothing_change.patch > > > Working on HBASE-22029, where we fail compile of hbase-it module four hours > into an RC build, it looks like transitive include of jersey-core is the > culprit. hadoop3 profile does a bunch of jersey-core exclusions. This issue > is about having hadoop2 profile and hadoop3 profiles match around jersey-core > treatment. Some miscellaneous cleanups are also done. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19222) update jruby to 9.1.14.0+
[ https://issues.apache.org/jira/browse/HBASE-19222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800688#comment-16800688 ] Sean Busbey commented on HBASE-19222: - none that I know of for branch-2+ > update jruby to 9.1.14.0+ > - > > Key: HBASE-19222 > URL: https://issues.apache.org/jira/browse/HBASE-19222 > Project: HBase > Issue Type: Sub-task > Components: shell >Reporter: Sean Busbey >Priority: Major > Fix For: 3.0.0, 2.3.0, 1.5.1 > > > JRuby's 9.1.14.0 release is described as "things generally work in jdk9" > https://twitter.com/headius/status/928405094407827457 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22103) HDFS-13209 in Hadoop 3.3.0 breaks asyncwal
[
https://issues.apache.org/jira/browse/HBASE-22103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800684#comment-16800684
]
Gabor Bota commented on HBASE-22103:
so right now hbase only supports hadoop-3.0.3?
> HDFS-13209 in Hadoop 3.3.0 breaks asyncwal
> --
>
> Key: HBASE-22103
> URL: https://issues.apache.org/jira/browse/HBASE-22103
> Project: HBase
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HBASE-22103.master.001.patch
>
>
> HDFS-13209 added an additional parameter to {{DistributedFileSystem.create}}
> and broke asyncwal.
> {noformat}
> 2019-03-25 12:19:21,061 ERROR [Listener at localhost/54758]
> asyncfs.FanOutOneBlockAsyncDFSOutputHelper(562): Couldn't properly initialize
> access to HDFS internals. Please update your WAL Provider to not make use of
> the 'asyncfs' provider. See HBASE-16110 for more information.
> java.lang.NoSuchMethodException:
> org.apache.hadoop.hdfs.protocol.ClientProtocol.create(java.lang.String,
> org.apache.hadoop.fs.permission.FsPermission, java.lang.String,
> org.apache.hadoop.io.EnumSetWritable, boolean, short, long,
> [Lorg.apache.hadoop.crypto.CryptoProtocolVersion;)
> at java.lang.Class.getMethod(Class.java:1786)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createFileCreator2(FanOutOneBlockAsyncDFSOutputHelper.java:513)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createFileCreator(FanOutOneBlockAsyncDFSOutputHelper.java:530)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:557)
> at
> org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:51)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:169)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166)
> at
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:105)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createAsyncWriter(AsyncFSWAL.java:663)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:669)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:126)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:813)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:519)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:460)
> at
> org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL.newWAL(TestAsyncFSWAL.java:72)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractTestFSWAL.testWALComparator(AbstractTestFSWAL.java:194)
> {noformat}
> Credit: this bug was found by [~gabor.bota]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22086) space quota issue: deleting snapshot doesn't update the usage of table
[
https://issues.apache.org/jira/browse/HBASE-22086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800679#comment-16800679
]
Josh Elser commented on HBASE-22086:
{quote}we rely on computing the size of the list of snapshots we list out
everytime and then re-writing it in the quota table, so it would be wise to
remove any existing snapshots entry from the quota table and have our updated
puts do the job for us.
{quote}
Good catch. Similar to another edge case like this you've fixed, I think,
Sakthi.
A tricky situation. Even after you delete a snapshot, it's not necessarily
"gone" from the filesystem yet. It would be trivial to just add some logic to
remove the size impact if we have the right MasterCP hook. But, that wouldn't
be completely accurate, and folks could delete a snapshot from HDFS by hand.
Feel free to bounce ideas. I can pull up the relevant code too.
> space quota issue: deleting snapshot doesn't update the usage of table
> --
>
> Key: HBASE-22086
> URL: https://issues.apache.org/jira/browse/HBASE-22086
> Project: HBase
> Issue Type: Bug
>Reporter: Ajeet Rai
>Assignee: Sakthi
>Priority: Minor
>
> space quota issue: deleting snapshot doesn't update the usage of table
> Steps: 1:
> set_quota TYPE => SPACE, TABLE => 'bugatti', LIMIT => '7M', POLICY =>
> NO_WRITES_COMPACTIONS
> 2: ./hbase pe --table="bugatti" --nomapred --rows=200 sequentialWrite 10
> 3: ./hbase pe --table="bugatti" --nomapred --rows=200 sequentialWrite 10
> 4: snapshot 'bugatti','bugatti_snapshot'
> 5: ./hbase pe --table="bugatti" --nomapred --rows=200 sequentialWrite 10
> 6: major_compact 'bugatti'
> 7: delete_snapshot 'bugatti_snapshot'
> now check the usage and observe that it is not getting updated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22052) pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove redunant version specifications
[ https://issues.apache.org/jira/browse/HBASE-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-22052: -- Resolution: Fixed Fix Version/s: 2.3.0 3.0.0 Status: Resolved (was: Patch Available) Reverted from 2.2 and then applied a compound of the mess I'd worked out on branch-2.1 as a single commit on branch-2.2. Cherry-picked from 2.2 to branch-2 and master. > pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove > redunant version specifications > --- > > Key: HBASE-22052 > URL: https://issues.apache.org/jira/browse/HBASE-22052 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.3.0, 2.1.4 > > Attachments: HBASE-22052-branch-2.1.003.addendum2.txt, > HBASE-22052.2.1.003.addendum1.txt, HBASE-22052.branch-2.0.001.patch, > HBASE-22052.branch-2.0.002.patch, HBASE-22052.branch-2.0.002.patch, > HBASE-22052.branch-2.0.003.patch, HBASE-22052.branch-2.0.004.patch, > HBASE-22052.branch-2.1.001.patch, HBASE-22052.branch-2.1.002.patch, > HBASE-22052.branch-2.1.003.patch, HBASE-22052.branch-2.2.001.patch, > HBASE-22052.branch-2.2.002.patch, nothing_change.patch, nothing_change.patch > > > Working on HBASE-22029, where we fail compile of hbase-it module four hours > into an RC build, it looks like transitive include of jersey-core is the > culprit. hadoop3 profile does a bunch of jersey-core exclusions. This issue > is about having hadoop2 profile and hadoop3 profiles match around jersey-core > treatment. Some miscellaneous cleanups are also done. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-22088) Jenkins post-build step times out after HBASE-20522
[ https://issues.apache.org/jira/browse/HBASE-22088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-22088. --- Resolution: Invalid Resolving as invalid. The actual issue was HBASE-22059 fixed by addendums in HBASE-22052. > Jenkins post-build step times out after HBASE-20522 > --- > > Key: HBASE-22088 > URL: https://issues.apache.org/jira/browse/HBASE-22088 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Priority: Major > > The post Jenkins step timesout (takes 9 hours) after HBASE-22052 (the parent) > was committed. Its hard to see why since don't have access to jenkins build > box logs. I asked for access. Until it arrives going to mess around here. > * What about HBASE-20522 made it so post step takes forever? > * The post step is archiving and publishing an html report. At first I > blamed the publishHMTL plugin but I think instead it the archiving step just > before. HBASE-20522 made the build output way bigger? Let me get some > signal -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22103) HDFS-13209 in Hadoop 3.3.0 breaks asyncwal
[
https://issues.apache.org/jira/browse/HBASE-22103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HBASE-22103:
Description:
HDFS-13209 added an additional parameter to {{DistributedFileSystem.create}}
and broke asyncwal.
{noformat}
2019-03-25 12:19:21,061 ERROR [Listener at localhost/54758]
asyncfs.FanOutOneBlockAsyncDFSOutputHelper(562): Couldn't properly initialize
access to HDFS internals. Please update your WAL Provider to not make use of
the 'asyncfs' provider. See HBASE-16110 for more information.
java.lang.NoSuchMethodException:
org.apache.hadoop.hdfs.protocol.ClientProtocol.create(java.lang.String,
org.apache.hadoop.fs.permission.FsPermission, java.lang.String,
org.apache.hadoop.io.EnumSetWritable, boolean, short, long,
[Lorg.apache.hadoop.crypto.CryptoProtocolVersion;)
at java.lang.Class.getMethod(Class.java:1786)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createFileCreator2(FanOutOneBlockAsyncDFSOutputHelper.java:513)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createFileCreator(FanOutOneBlockAsyncDFSOutputHelper.java:530)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:557)
at
org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:51)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:169)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166)
at
org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:105)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createAsyncWriter(AsyncFSWAL.java:663)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:669)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:126)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:813)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:519)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:460)
at
org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL.newWAL(TestAsyncFSWAL.java:72)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractTestFSWAL.testWALComparator(AbstractTestFSWAL.java:194)
{noformat}
Credit: this bug was found by [~gabor.bota]
was:
HDFS-13209 added an additional parameter to {{DistributedFileSystem.create}}
and broke asyncfs.
{noformat}
2019-03-25 12:19:21,061 ERROR [Listener at localhost/54758]
asyncfs.FanOutOneBlockAsyncDFSOutputHelper(562): Couldn't properly initialize
access to HDFS internals. Please update your WAL Provider to not make use of
the 'asyncfs' provider. See HBASE-16110 for more information.
java.lang.NoSuchMethodException:
org.apache.hadoop.hdfs.protocol.ClientProtocol.create(java.lang.String,
org.apache.hadoop.fs.permission.FsPermission, java.lang.String,
org.apache.hadoop.io.EnumSetWritable, boolean, short, long,
[Lorg.apache.hadoop.crypto.CryptoProtocolVersion;)
at java.lang.Class.getMethod(Class.java:1786)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createFileCreator2(FanOutOneBlockAsyncDFSOutputHelper.java:513)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createFileCreator(FanOutOneBlockAsyncDFSOutputHelper.java:530)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:557)
at
org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:51)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:169)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166)
at
org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:105)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createAsyncWriter(AsyncFSWAL.java:663)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:669)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:126)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:813)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:519)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:460)
at
[jira] [Updated] (HBASE-22103) HDFS-13209 in Hadoop 3.3.0 breaks asyncwal
[
https://issues.apache.org/jira/browse/HBASE-22103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HBASE-22103:
Description:
HDFS-13209 added an additional parameter to {{DistributedFileSystem.create}}
and broke asyncfs.
{noformat}
2019-03-25 12:19:21,061 ERROR [Listener at localhost/54758]
asyncfs.FanOutOneBlockAsyncDFSOutputHelper(562): Couldn't properly initialize
access to HDFS internals. Please update your WAL Provider to not make use of
the 'asyncfs' provider. See HBASE-16110 for more information.
java.lang.NoSuchMethodException:
org.apache.hadoop.hdfs.protocol.ClientProtocol.create(java.lang.String,
org.apache.hadoop.fs.permission.FsPermission, java.lang.String,
org.apache.hadoop.io.EnumSetWritable, boolean, short, long,
[Lorg.apache.hadoop.crypto.CryptoProtocolVersion;)
at java.lang.Class.getMethod(Class.java:1786)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createFileCreator2(FanOutOneBlockAsyncDFSOutputHelper.java:513)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createFileCreator(FanOutOneBlockAsyncDFSOutputHelper.java:530)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:557)
at
org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:51)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:169)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166)
at
org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:105)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createAsyncWriter(AsyncFSWAL.java:663)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:669)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:126)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:813)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:519)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:460)
at
org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL.newWAL(TestAsyncFSWAL.java:72)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractTestFSWAL.testWALComparator(AbstractTestFSWAL.java:194)
{noformat}
Credit: this bug was found by [~gabor.bota]
was:
HDFS-13209 added an additional parameter to {{DistributedFileSystem.create}}
and broke asyncfs.
{noformat}
2019-03-25 12:19:21,061 ERROR [Listener at localhost/54758]
asyncfs.FanOutOneBlockAsyncDFSOutputHelper(562): Couldn't properly initialize
access to HDFS internals. Please update your WAL Provider to not make use of
the 'asyncfs' provider. See HBASE-16110 for more information.
java.lang.NoSuchMethodException:
org.apache.hadoop.hdfs.protocol.ClientProtocol.create(java.lang.String,
org.apache.hadoop.fs.permission.FsPermission, java.lang.String,
org.apache.hadoop.io.EnumSetWritable, boolean, short, long,
[Lorg.apache.hadoop.crypto.CryptoProtocolVersion;)
at java.lang.Class.getMethod(Class.java:1786)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createFileCreator2(FanOutOneBlockAsyncDFSOutputHelper.java:513)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createFileCreator(FanOutOneBlockAsyncDFSOutputHelper.java:530)
at
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:557)
at
org.apache.hadoop.hbase.io.asyncfs.AsyncFSOutputHelper.createOutput(AsyncFSOutputHelper.java:51)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.initOutput(AsyncProtobufLogWriter.java:169)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:166)
at
org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:105)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createAsyncWriter(AsyncFSWAL.java:663)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:669)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:126)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:813)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:519)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:460)
at
