[jira] [Updated] (HBASE-22099) Backport HBASE-21895 "Error prone upgrade" to branch-2
[ https://issues.apache.org/jira/browse/HBASE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-22099: -- Assignee: Duo Zhang Status: Patch Available (was: Open) > Backport HBASE-21895 "Error prone upgrade" to branch-2 > -- > > Key: HBASE-22099 > URL: https://issues.apache.org/jira/browse/HBASE-22099 > Project: HBase > Issue Type: Task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-22099-branch-2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22099) Backport HBASE-21895 "Error prone upgrade" to branch-2
[ https://issues.apache.org/jira/browse/HBASE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-22099: -- Attachment: HBASE-22099-branch-2.patch > Backport HBASE-21895 "Error prone upgrade" to branch-2 > -- > > Key: HBASE-22099 > URL: https://issues.apache.org/jira/browse/HBASE-22099 > Project: HBase > Issue Type: Task >Reporter: Duo Zhang >Priority: Major > Attachments: HBASE-22099-branch-2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22099) Backport HBASE-21895 "Error prone upgrade" to branch-2
[ https://issues.apache.org/jira/browse/HBASE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-22099: -- Component/s: build > Backport HBASE-21895 "Error prone upgrade" to branch-2 > -- > > Key: HBASE-22099 > URL: https://issues.apache.org/jira/browse/HBASE-22099 > Project: HBase > Issue Type: Task > Components: build >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-22099-branch-2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22060) postOpenDeployTasks could send OPENED region transition state to the wrong master
[
https://issues.apache.org/jira/browse/HBASE-22060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803601#comment-16803601
]
Bahram Chehrazy commented on HBASE-22060:
-
[~Apache9], thank you for quick fix. I tried to apply your patch to our branch,
but wasn't straight forward, as our branches have diverged. I'll try to cherry
pick some missing patches tomorrow. But if too much, we may have to try a
simpler fix.
> postOpenDeployTasks could send OPENED region transition state to the wrong
> master
> -
>
> Key: HBASE-22060
> URL: https://issues.apache.org/jira/browse/HBASE-22060
> Project: HBase
> Issue Type: Bug
> Components: amv2, proc-v2
>Affects Versions: 3.0.0
>Reporter: Bahram Chehrazy
>Assignee: Duo Zhang
>Priority: Critical
>
> As was reported in HBASE-21788, we have repeatedly seen regions getting stuck
> in OPENING after master restarts. Here is one scenario that I've observed
> recently:
>
> 1) There is a region is transit (RIT).
> 2) The active master aborts and begins shutting down.
> 3) The backup master becomes active quickly, finds the RIT, creates
> OpenRegionProcedure and send request to some server.
> 4) The server quickly opens the region and posts OPENED state transition, but
> it uses its cached master instead of the new one.
> 5) The old active master which had not completely shutdown its assignment
> manager yet, notes the OPENED state report and ignores it. Because no
> corresponding procedure can be found.
> 6) The new master waits forever for a response to its OPEN region request.
>
> This happens more often with the meta region because it's small and takes a
> few seconds to open. Below are some related logs:
> *Previous HMaster:*
> 2019-03-14 13:19:16,310 ERROR [PEWorker-1] master.HMaster: * ABORTING
> master ,17000,1552438242232: Shutting down HBase cluster: file
> system not available *
> 2019-03-14 13:19:16,310 INFO [PEWorker-1] regionserver.HRegionServer: *
> STOPPING region server ',17000,1552438242232' *
> 2019-03-14 13:20:54,358 WARN
> [RpcServer.priority.FPBQ.Fifo.handler=11,queue=1,port=17000]
> assignment.AssignmentManager: No matching procedure found for rit=OPEN,
> location=*,17020,1552561955412, table=hbase:meta,
> region=1588230740 transition to OPENED
> 2019-03-14 13:20:55,707 INFO [master/:17000]
> assignment.AssignmentManager: Stopping assignment manager
> *New HMaster logs:*
> 2019-03-14 13:19:16,907 INFO [master/:17000:becomeActiveMaster]
> master.ActiveMasterManager: Deleting ZNode for
> /HBaseServerZnodeCommonDir/**/backup-masters/,17000,1552438259871
> from backup master directory
> 2019-03-14 13:19:17,031 INFO [master/:17000:becomeActiveMaster]
> master.ActiveMasterManager: Registered as active
> master=,17000,1552438259871
> 2019-03-14 13:20:52,017 INFO [PEWorker-12] zookeeper.MetaTableLocator:
> Setting hbase:meta (replicaId=0) location in ZooKeeper as
> ,17020,1552536956826
> 2019-03-14 13:20:52,105 INFO [PEWorker-12] procedure2.ProcedureExecutor:
> Initialized subprocedures=[\{pid=178230, ppid=178229, state=RUNNABLE,
> hasLock=false; org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure}]
>
> *HServer logs:*
> 2019-03-14 13:20:52,708 INFO [RS_CLOSE_META-regionserver/:17020-0]
> handler.AssignRegionHandler: Open hbase:meta,,1.1588230740
> 2019-03-14 13:20:54,353 INFO [RS_CLOSE_META-regionserver/:17020-0]
> regionserver.HRegion: Opened 1588230740; next sequenceid=229166
> 2019-03-14 13:20:54,356 INFO [RS_CLOSE_META-regionserver/:17020-0]
> regionserver.HRegionServer: Post open deploy tasks for
> hbase:meta,,1.1588230740
> 2019-03-14 13:20:54,358 INFO [RS_CLOSE_META-regionserver/:17020-0]
> handler.AssignRegionHandler: Opened hbase:meta,,1.1588230740
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22099) Backport HBASE-21895 "Error prone upgrade" to branch-2
[ https://issues.apache.org/jira/browse/HBASE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-22099: -- Fix Version/s: 2.1.5 2.0.6 2.3.0 2.2.0 > Backport HBASE-21895 "Error prone upgrade" to branch-2 > -- > > Key: HBASE-22099 > URL: https://issues.apache.org/jira/browse/HBASE-22099 > Project: HBase > Issue Type: Task > Components: build >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.2.0, 2.3.0, 2.0.6, 2.1.5 > > Attachments: HBASE-22099-branch-2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22100) False positive for error prone warnings in pre commit job
[
https://issues.apache.org/jira/browse/HBASE-22100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-22100:
--
Release Note: Now we will sort the javac WARNING/ERROR before generating
diff in pre-commit so we can get a stable output for the error prone. The
downside is that we just sort the output lexicographically so the line number
will also be sorted lexicographically, which is a bit strange to human.
> False positive for error prone warnings in pre commit job
> -
>
> Key: HBASE-22100
> URL: https://issues.apache.org/jira/browse/HBASE-22100
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Minor
> Fix For: 3.0.0, 2.2.0, 2.3.0, 2.0.6, 2.1.5
>
> Attachments: HBASE-22100-test.patch, HBASE-22100.patch
>
>
> https://builds.apache.org/job/PreCommit-HBASE-Build/16516/artifact/patchprocess/branch-compile-javac-hbase-client.txt
> {noformat}
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,69]
> [UnusedVariable] The parameter 'updateCachedLocation' is never read.
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,42]
> [UnusedVariable] The parameter 'error' is never read.
> {noformat}
> https://builds.apache.org/job/PreCommit-HBASE-Build/16516/artifact/patchprocess/patch-compile-javac-hbase-client.txt
> {noformat}
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,42]
> [UnusedVariable] The parameter 'error' is never read.
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,69]
> [UnusedVariable] The parameter 'updateCachedLocation' is never read.
> {noformat}
> And the output is 1 new and 1 fixed, the new one is
> {noformat}
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,69]
> [UnusedVariable] The parameter 'updateCachedLocation' is never read.
> {noformat}
> I think here we should report nothing, as it is just an order change...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22100) False positive for error prone warnings in pre commit job
[
https://issues.apache.org/jira/browse/HBASE-22100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-22100:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Pushed to branch-2.0+.
> False positive for error prone warnings in pre commit job
> -
>
> Key: HBASE-22100
> URL: https://issues.apache.org/jira/browse/HBASE-22100
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Minor
> Fix For: 3.0.0, 2.2.0, 2.3.0, 2.0.6, 2.1.5
>
> Attachments: HBASE-22100-test.patch, HBASE-22100.patch
>
>
> https://builds.apache.org/job/PreCommit-HBASE-Build/16516/artifact/patchprocess/branch-compile-javac-hbase-client.txt
> {noformat}
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,69]
> [UnusedVariable] The parameter 'updateCachedLocation' is never read.
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,42]
> [UnusedVariable] The parameter 'error' is never read.
> {noformat}
> https://builds.apache.org/job/PreCommit-HBASE-Build/16516/artifact/patchprocess/patch-compile-javac-hbase-client.txt
> {noformat}
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,42]
> [UnusedVariable] The parameter 'error' is never read.
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,69]
> [UnusedVariable] The parameter 'updateCachedLocation' is never read.
> {noformat}
> And the output is 1 new and 1 fixed, the new one is
> {noformat}
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,69]
> [UnusedVariable] The parameter 'updateCachedLocation' is never read.
> {noformat}
> I think here we should report nothing, as it is just an order change...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22100) False positive for error prone warnings in pre commit job
[
https://issues.apache.org/jira/browse/HBASE-22100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-22100:
--
Fix Version/s: 2.1.5
2.0.6
2.3.0
2.2.0
> False positive for error prone warnings in pre commit job
> -
>
> Key: HBASE-22100
> URL: https://issues.apache.org/jira/browse/HBASE-22100
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Minor
> Fix For: 3.0.0, 2.2.0, 2.3.0, 2.0.6, 2.1.5
>
> Attachments: HBASE-22100-test.patch, HBASE-22100.patch
>
>
> https://builds.apache.org/job/PreCommit-HBASE-Build/16516/artifact/patchprocess/branch-compile-javac-hbase-client.txt
> {noformat}
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,69]
> [UnusedVariable] The parameter 'updateCachedLocation' is never read.
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,42]
> [UnusedVariable] The parameter 'error' is never read.
> {noformat}
> https://builds.apache.org/job/PreCommit-HBASE-Build/16516/artifact/patchprocess/patch-compile-javac-hbase-client.txt
> {noformat}
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,42]
> [UnusedVariable] The parameter 'error' is never read.
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,69]
> [UnusedVariable] The parameter 'updateCachedLocation' is never read.
> {noformat}
> And the output is 1 new and 1 fixed, the new one is
> {noformat}
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,69]
> [UnusedVariable] The parameter 'updateCachedLocation' is never read.
> {noformat}
> I think here we should report nothing, as it is just an order change...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22100) False positive for error prone warnings in pre commit job
[
https://issues.apache.org/jira/browse/HBASE-22100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803588#comment-16803588
]
Duo Zhang commented on HBASE-22100:
---
Seems worked.
https://builds.apache.org/job/PreCommit-HBASE-Build/16557/artifact/patchprocess/patch-compile-javac-root.txt
The file is sorted, as you can see that less leading digit will placed first.
Let me commit, Thanks [~busbey] for reivewing.
> False positive for error prone warnings in pre commit job
> -
>
> Key: HBASE-22100
> URL: https://issues.apache.org/jira/browse/HBASE-22100
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-22100-test.patch, HBASE-22100.patch
>
>
> https://builds.apache.org/job/PreCommit-HBASE-Build/16516/artifact/patchprocess/branch-compile-javac-hbase-client.txt
> {noformat}
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,69]
> [UnusedVariable] The parameter 'updateCachedLocation' is never read.
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,42]
> [UnusedVariable] The parameter 'error' is never read.
> {noformat}
> https://builds.apache.org/job/PreCommit-HBASE-Build/16516/artifact/patchprocess/patch-compile-javac-hbase-client.txt
> {noformat}
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,42]
> [UnusedVariable] The parameter 'error' is never read.
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,69]
> [UnusedVariable] The parameter 'updateCachedLocation' is never read.
> {noformat}
> And the output is 1 new and 1 fixed, the new one is
> {noformat}
> [WARNING]
> /testptch/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRpcRetryingCaller.java:[125,69]
> [UnusedVariable] The parameter 'updateCachedLocation' is never read.
> {noformat}
> I think here we should report nothing, as it is just an order change...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22053) zookeeper URL links in documentation are failing with 404
[
https://issues.apache.org/jira/browse/HBASE-22053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803578#comment-16803578
]
Hudson commented on HBASE-22053:
Results for branch branch-2.1
[build #1000 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1000/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1000//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1000//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1000//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> zookeeper URL links in documentation are failing with 404
> -
>
> Key: HBASE-22053
> URL: https://issues.apache.org/jira/browse/HBASE-22053
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Subrat Mishra
>Assignee: Subrat Mishra
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.1.5, 2.2.1
>
> Attachments: HBASE-22053.master.001.patch,
> HBASE-22053.master.002.patch
>
>
> zookeeper URL changed from hadoop.apache.org to zookeeper.apache.org
> E.g: Below URL failing with 404
> http://hadoop.apache.org/zookeeper/docs/current/zookeeperProgrammers.html#ch_zkSessions
> should be changed to:
> https://zookeeper.apache.org/doc/current/zookeeperProgrammers.html#ch_zkSessions
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22097) Modify the description of split command in shell
[
https://issues.apache.org/jira/browse/HBASE-22097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803577#comment-16803577
]
Hudson commented on HBASE-22097:
Results for branch branch-2.1
[build #1000 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1000/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1000//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1000//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1000//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Modify the description of split command in shell
>
>
> Key: HBASE-22097
> URL: https://issues.apache.org/jira/browse/HBASE-22097
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0, 2.1.5, 2.2.1
>
> Attachments: HBASE-22097.master.001.patch
>
>
> We can make the description of split command in shell better.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22073) /rits.jsp throws an exception if no procedure
[
https://issues.apache.org/jira/browse/HBASE-22073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803579#comment-16803579
]
Hudson commented on HBASE-22073:
Results for branch branch-2.1
[build #1000 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1000/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1000//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1000//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1000//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> /rits.jsp throws an exception if no procedure
> -
>
> Key: HBASE-22073
> URL: https://issues.apache.org/jira/browse/HBASE-22073
> Project: HBase
> Issue Type: Bug
> Components: UI
>Affects Versions: 2.1.0
>Reporter: Balazs Meszaros
>Assignee: Balazs Meszaros
>Priority: Major
> Fix For: 2.1.5
>
> Attachments: HBASE-22073.branch-2.1.patch
>
>
> I got the following exception in our test environment:
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.generated.master.rits_jsp._jspService(rits_jsp.java:101)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> ...
> {noformat}
> Because {{regionStateNode.getProcedure()}} returns {{null}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803580#comment-16803580
]
Hudson commented on HBASE-20662:
Results for branch branch-2.1
[build #1000 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1000/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1000//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1000//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1000//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.0.6, 2.1.5
>
> Attachments: HBASE-20662.branch-2.1.001.patch,
> HBASE-20662.master.001.patch, HBASE-20662.master.002.patch,
> HBASE-20662.master.003.patch, HBASE-20662.master.004.patch,
> HBASE-20662.master.004.patch, HBASE-20662.master.005.patch,
> HBASE-20662.master.006.patch, HBASE-20662.master.007.patch,
> HBASE-20662.master.008.patch, HBASE-20662.master.008.patch,
> HBASE-20662.master.009.patch, HBASE-20662.master.009.patch,
> HBASE-20662.master.010.patch, screenshot.png
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService methodName:
> EnableTable size: 104 connection: 127.0.0.1:38030 deadline: 1527707127568,
> exception=org.apache.hadoop.hbase.security.AccessDeniedException: Enabling
> the table 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to
> a violated space quota.
> 2018-05-31 00:34:27,571 ERROR [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> quotas.RegionServerSpaceQuotaManager(210): Failed to disable space violation
> policy for testSetQuotaAndThenIncreaseQuotaWithDisable0. This table will
> remain in violation.
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Enabling the table
> 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to a
> violated space quota.
> at org.apache.hadoop.hbase.master.HMaster$6.run(HMaster.java:2275)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:131)
> at org.apache.hadoop.hbase.master.HMaster.enableTable(HMaster.java:2258)
> at
>
[jira] [Updated] (HBASE-22098) Backport HBASE-18667 "Disable error-prone for hbase-protocol-shaded" to branch-2
[ https://issues.apache.org/jira/browse/HBASE-22098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-22098: -- Component/s: build > Backport HBASE-18667 "Disable error-prone for hbase-protocol-shaded" to > branch-2 > > > Key: HBASE-22098 > URL: https://issues.apache.org/jira/browse/HBASE-22098 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.2.0, 2.3.0, 2.0.6, 2.1.5 > > Attachments: HBASE-22098-branch-2.patch > > > Otherwise the error prone compile will cost a very long time when processing > the hbase-shaded-protocol module... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22098) Backport HBASE-18667 "Disable error-prone for hbase-protocol-shaded" to branch-2
[ https://issues.apache.org/jira/browse/HBASE-22098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-22098: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to branch-2.0+. Thanks [~stack] for reviewing. > Backport HBASE-18667 "Disable error-prone for hbase-protocol-shaded" to > branch-2 > > > Key: HBASE-22098 > URL: https://issues.apache.org/jira/browse/HBASE-22098 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.2.0, 2.3.0, 2.0.6, 2.1.5 > > Attachments: HBASE-22098-branch-2.patch > > > Otherwise the error prone compile will cost a very long time when processing > the hbase-shaded-protocol module... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803572#comment-16803572
]
Hudson commented on HBASE-20662:
Results for branch branch-2.0
[build #1470 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1470/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1470//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1470//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1470//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.0.6, 2.1.5
>
> Attachments: HBASE-20662.branch-2.1.001.patch,
> HBASE-20662.master.001.patch, HBASE-20662.master.002.patch,
> HBASE-20662.master.003.patch, HBASE-20662.master.004.patch,
> HBASE-20662.master.004.patch, HBASE-20662.master.005.patch,
> HBASE-20662.master.006.patch, HBASE-20662.master.007.patch,
> HBASE-20662.master.008.patch, HBASE-20662.master.008.patch,
> HBASE-20662.master.009.patch, HBASE-20662.master.009.patch,
> HBASE-20662.master.010.patch, screenshot.png
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService methodName:
> EnableTable size: 104 connection: 127.0.0.1:38030 deadline: 1527707127568,
> exception=org.apache.hadoop.hbase.security.AccessDeniedException: Enabling
> the table 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to
> a violated space quota.
> 2018-05-31 00:34:27,571 ERROR [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> quotas.RegionServerSpaceQuotaManager(210): Failed to disable space violation
> policy for testSetQuotaAndThenIncreaseQuotaWithDisable0. This table will
> remain in violation.
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Enabling the table
> 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to a
> violated space quota.
> at org.apache.hadoop.hbase.master.HMaster$6.run(HMaster.java:2275)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:131)
> at org.apache.hadoop.hbase.master.HMaster.enableTable(HMaster.java:2258)
> at
>
[jira] [Commented] (HBASE-22098) Backport HBASE-18667 "Disable error-prone for hbase-protocol-shaded" to branch-2
[
https://issues.apache.org/jira/browse/HBASE-22098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803566#comment-16803566
]
Duo Zhang commented on HBASE-22098:
---
Just cherry picked the original patch in HBASE-18667.
{quote}
So, you are adding generated annotation on all generated classes so they'll be
ignored by errorprone?
{quote}
There is a '-XepDisableWarningsInGeneratedCode' option in error prone to
disable checks on generated code. And this is really necessary, if you do not
have this, it may spend hours on compiling the generated protobuf code when
error prone is enabled...
> Backport HBASE-18667 "Disable error-prone for hbase-protocol-shaded" to
> branch-2
>
>
> Key: HBASE-22098
> URL: https://issues.apache.org/jira/browse/HBASE-22098
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-22098-branch-2.patch
>
>
> Otherwise the error prone compile will cost a very long time when processing
> the hbase-shaded-protocol module...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22098) Backport HBASE-18667 "Disable error-prone for hbase-protocol-shaded" to branch-2
[ https://issues.apache.org/jira/browse/HBASE-22098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-22098: -- Fix Version/s: 2.1.5 2.0.6 2.3.0 2.2.0 > Backport HBASE-18667 "Disable error-prone for hbase-protocol-shaded" to > branch-2 > > > Key: HBASE-22098 > URL: https://issues.apache.org/jira/browse/HBASE-22098 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.2.0, 2.3.0, 2.0.6, 2.1.5 > > Attachments: HBASE-22098-branch-2.patch > > > Otherwise the error prone compile will cost a very long time when processing > the hbase-shaded-protocol module... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22075) Potential data loss when MOB compaction fails
[
https://issues.apache.org/jira/browse/HBASE-22075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803562#comment-16803562
]
Hadoop QA commented on HBASE-22075:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
7s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
29s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 3m 29s{color}
| {color:red} hbase-server generated 3 new + 191 unchanged - 3 fixed = 194
total (was 194) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
12s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 3s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}262m 29s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}311m 39s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.quotas.TestSpaceQuotas |
| | hadoop.hbase.client.TestFromClientSideWithCoprocessor |
| | hadoop.hbase.master.procedure.TestSCPWithReplicas |
| |
hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-22075 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12963957/HBASE-22075-v2.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux f3692fd68972 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 59406c44e3 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
|
[jira] [Commented] (HBASE-22082) Should not use an instance to access static members, which will increases compilation costs.
[ https://issues.apache.org/jira/browse/HBASE-22082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803557#comment-16803557 ] lixiaobao commented on HBASE-22082: --- Ok,Thanks,I will pay attention next time. > Should not use an instance to access static members, which will increases > compilation costs. > - > > Key: HBASE-22082 > URL: https://issues.apache.org/jira/browse/HBASE-22082 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 1.4.5, 2.1.1, 2.0.4 >Reporter: lixiaobao >Assignee: lixiaobao >Priority: Minor > Fix For: 3.0.0 > > Attachments: HBASE-22082.patch > > > A little modification: > In class MutationBatchOperation, the method prepareMiniBatchOperations has a > small change,we should not use an instance to access static members, which > will increases compilation costs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21688) Address WAL filesystem issues
[
https://issues.apache.org/jira/browse/HBASE-21688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803555#comment-16803555
]
Hadoop QA commented on HBASE-21688:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.0 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
17s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
55s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
8s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
32s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
9s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} branch-2.0 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
16s{color} | {color:red} hbase-server: The patch generated 6 new + 121
unchanged - 1 fixed = 127 total (was 122) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
24s{color} | {color:red} hbase-it: The patch generated 3 new + 108 unchanged -
0 fixed = 111 total (was 108) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 3 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
20s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 19s{color} | {color:green} Patch does not cause any errors with Hadoop 2.6.5
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
38s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}122m
28s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
56s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
0s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}171m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:6f01af0 |
| JIRA Issue | HBASE-21688 |
| JIRA Patch URL |
[jira] [Commented] (HBASE-22005) Use ByteBuff's refcnt to track the life cycle of data block
[ https://issues.apache.org/jira/browse/HBASE-22005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803550#comment-16803550 ] Zheng Hu commented on HBASE-22005: -- I wrote an method to dump the bytes in cellBlockStream at server side. each line represent an KV, the first four byte 00 00 00 2c represent length of kv (=44)... seems start from 31 cell, the bytes are rotated... > Use ByteBuff's refcnt to track the life cycle of data block > --- > > Key: HBASE-22005 > URL: https://issues.apache.org/jira/browse/HBASE-22005 > Project: HBase > Issue Type: Sub-task >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-22005.HBASE-21879.v1.patch, > HBASE-22005.HBASE-21879.v2.patch, HBASE-22005.HBASE-21879.v3.patch, > HBASE-22005.HBASE-21879.v4.patch, HBASE-22005.HBASE-21879.v5.patch, > HBASE-22005.HBASE-21879.v6.patch, HBASE-22005.HBASE-21879.v7.patch, > HBASE-22005.HBASE-21879.v8.patch, cell-encoding.jpeg > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22005) Use ByteBuff's refcnt to track the life cycle of data block
[
https://issues.apache.org/jira/browse/HBASE-22005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803548#comment-16803548
]
Hadoop QA commented on HBASE-22005:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s{color}
| {color:red} HBASE-22005 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.8.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-22005 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/16571/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Use ByteBuff's refcnt to track the life cycle of data block
> ---
>
> Key: HBASE-22005
> URL: https://issues.apache.org/jira/browse/HBASE-22005
> Project: HBase
> Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-22005.HBASE-21879.v1.patch,
> HBASE-22005.HBASE-21879.v2.patch, HBASE-22005.HBASE-21879.v3.patch,
> HBASE-22005.HBASE-21879.v4.patch, HBASE-22005.HBASE-21879.v5.patch,
> HBASE-22005.HBASE-21879.v6.patch, HBASE-22005.HBASE-21879.v7.patch,
> HBASE-22005.HBASE-21879.v8.patch, cell-encoding.jpeg
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22005) Use ByteBuff's refcnt to track the life cycle of data block
[ https://issues.apache.org/jira/browse/HBASE-22005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-22005: - Attachment: cell-encoding.jpeg > Use ByteBuff's refcnt to track the life cycle of data block > --- > > Key: HBASE-22005 > URL: https://issues.apache.org/jira/browse/HBASE-22005 > Project: HBase > Issue Type: Sub-task >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-22005.HBASE-21879.v1.patch, > HBASE-22005.HBASE-21879.v2.patch, HBASE-22005.HBASE-21879.v3.patch, > HBASE-22005.HBASE-21879.v4.patch, HBASE-22005.HBASE-21879.v5.patch, > HBASE-22005.HBASE-21879.v6.patch, HBASE-22005.HBASE-21879.v7.patch, > HBASE-22005.HBASE-21879.v8.patch, cell-encoding.jpeg > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22089) HMaster#getClusterSchema() could return null and cause NPE
[
https://issues.apache.org/jira/browse/HBASE-22089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kevin su updated HBASE-22089:
-
Attachment: HBASE-22089.v0.patch
> HMaster#getClusterSchema() could return null and cause NPE
> --
>
> Key: HBASE-22089
> URL: https://issues.apache.org/jira/browse/HBASE-22089
> Project: HBase
> Issue Type: Improvement
> Components: master
>Reporter: Xiang Li
>Assignee: kevin su
>Priority: Minor
> Attachments: HBASE-22089.v0.patch
>
>
> HMaster#getClusterSchema() has a lot of usage, for example:
> {code:title=hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java|borderStyle=solid}
> void assignTableToGroup(TableDescriptor desc) throws IOException {
> String groupName =
>
> master.getClusterSchema().getNamespace(desc.getTableName().getNamespaceAsString())
> .getConfigurationValue(RSGroupInfo.NAMESPACE_DESC_PROP_GROUP);
> ...
> }
> {code}
> getClusterSchema() might return null when cluster schema service is not ready
> yet. Actually, it won't be ready until HMaster#initClusterSchemaService() is
> finished. See [this
> comment|https://issues.apache.org/jira/browse/HBASE-20690?focusedCommentId=16784549=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16784549]
> for details.
> We need to add protections, like adding null check before calling
> getClusterSchema().getXXX(), or in HMaster#getClusterSchema(), add a loop to
> wait until this.clusterSchemaService is not null.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22089) HMaster#getClusterSchema() could return null and cause NPE
[
https://issues.apache.org/jira/browse/HBASE-22089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16799584#comment-16799584
]
kevin su edited comment on HBASE-22089 at 3/28/19 12:42 AM:
[~water] I found that there is a function *_isInitialized_* in Hmaster, It will
set to true after initClusterSchemaService was completed.
So,I will use *_this.master.isInitialized()_* to check if this is completed, if
no,It will give a log
{code}LOG.debug("Master has not been initialized, don't
assignTableToGroup.");{code}
And then return NULL.
What do you think?
was (Author: pingsutw):
[~water] I found that there is a function *_isInitialized_* in Hmaster, It will
set to true after initClusterSchemaService was completed.
So,I will use *_this.master.isInitialized()_* to check if this is completed, if
no,It will give a log
{code}LOG.debug("Master has not been initialized, don't get
NamespaceDescriptor.");{code}
And then return NULL.
What do you think?
> HMaster#getClusterSchema() could return null and cause NPE
> --
>
> Key: HBASE-22089
> URL: https://issues.apache.org/jira/browse/HBASE-22089
> Project: HBase
> Issue Type: Improvement
> Components: master
>Reporter: Xiang Li
>Assignee: kevin su
>Priority: Minor
>
> HMaster#getClusterSchema() has a lot of usage, for example:
> {code:title=hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java|borderStyle=solid}
> void assignTableToGroup(TableDescriptor desc) throws IOException {
> String groupName =
>
> master.getClusterSchema().getNamespace(desc.getTableName().getNamespaceAsString())
> .getConfigurationValue(RSGroupInfo.NAMESPACE_DESC_PROP_GROUP);
> ...
> }
> {code}
> getClusterSchema() might return null when cluster schema service is not ready
> yet. Actually, it won't be ready until HMaster#initClusterSchemaService() is
> finished. See [this
> comment|https://issues.apache.org/jira/browse/HBASE-20690?focusedCommentId=16784549=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16784549]
> for details.
> We need to add protections, like adding null check before calling
> getClusterSchema().getXXX(), or in HMaster#getClusterSchema(), add a loop to
> wait until this.clusterSchemaService is not null.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22115) HBase RPC aspires to grow an infinite tree of trace scopes; some other places are also unsafe
[ https://issues.apache.org/jira/browse/HBASE-22115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803491#comment-16803491 ] Sergey Shelukhin commented on HBASE-22115: -- Does that constitute a +1? :) > HBase RPC aspires to grow an infinite tree of trace scopes; some other places > are also unsafe > - > > Key: HBASE-22115 > URL: https://issues.apache.org/jira/browse/HBASE-22115 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Critical > Attachments: Clipboard01.jpg, HBASE-22115.01.patch, HBASE-22115.patch > > > !Clipboard01.jpg! > All of those are ClientServices.Multi in this case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21883) Enhancements to Major Compaction tool from HBASE-19528
[ https://issues.apache.org/jira/browse/HBASE-21883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803492#comment-16803492 ] stack commented on HBASE-21883: --- No need of apology sir [~thiruvel]. > Enhancements to Major Compaction tool from HBASE-19528 > -- > > Key: HBASE-21883 > URL: https://issues.apache.org/jira/browse/HBASE-21883 > Project: HBase > Issue Type: Improvement > Components: Client, Compaction, tooling >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Minor > Fix For: 3.0.0, 2.3.0, 1.5.1 > > Attachments: HBASE-21883.branch-1.001.patch, > HBASE-21883.branch-1.002.patch, HBASE-21883.master.001.patch, > HBASE-21883.master.002.patch > > > I would like to add new compaction tools based on [~churromorales]'s tool at > HBASE-19528. > We internally have tools that pick and compact regions based on multiple > criteria. Since Rahul already has a version in community, we would like to > build on top of it instead of pushing yet another tool. > With this jira, I would like to add a tool which looks at regions beyond TTL > and compacts them in a rsgroup. We have time series data and those regions > will become dead after a while, so we compact those regions to save disk > space. We also merge those empty regions to reduce load, but that tool comes > later. > Will prep a patch for 2.x once 1.5 gets in. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21688) Address WAL filesystem issues
[ https://issues.apache.org/jira/browse/HBASE-21688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803487#comment-16803487 ] Vladimir Rodionov commented on HBASE-21688: --- All patches for 2.x are ready. For master branch there is a small addendum patch, which has to be applied. > Address WAL filesystem issues > - > > Key: HBASE-21688 > URL: https://issues.apache.org/jira/browse/HBASE-21688 > Project: HBase > Issue Type: Bug > Components: Filesystem Integration, wal >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Labels: s3 > Fix For: 3.0.0 > > Attachments: HBASE-21688-branch-2.0-v1.patch, > HBASE-21688-branch-2.1-v2.patch, HBASE-21688-branch-2.2-v1.patch, > HBASE-21688-master-addendum.patch, HBASE-21688-v1.patch > > > Scan and fix code base to use new way of instantiating WAL File System. > https://issues.apache.org/jira/browse/HBASE-21457?focusedCommentId=16734688=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16734688 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21688) Address WAL filesystem issues
[ https://issues.apache.org/jira/browse/HBASE-21688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-21688: -- Attachment: (was: HBASE-21688-branch-2.1-v1.patch) > Address WAL filesystem issues > - > > Key: HBASE-21688 > URL: https://issues.apache.org/jira/browse/HBASE-21688 > Project: HBase > Issue Type: Bug > Components: Filesystem Integration, wal >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Labels: s3 > Fix For: 3.0.0 > > Attachments: HBASE-21688-branch-2.0-v1.patch, > HBASE-21688-branch-2.1-v2.patch, HBASE-21688-branch-2.2-v1.patch, > HBASE-21688-master-addendum.patch, HBASE-21688-v1.patch > > > Scan and fix code base to use new way of instantiating WAL File System. > https://issues.apache.org/jira/browse/HBASE-21457?focusedCommentId=16734688=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16734688 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21688) Address WAL filesystem issues
[ https://issues.apache.org/jira/browse/HBASE-21688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-21688: -- Attachment: HBASE-21688-branch-2.0-v1.patch > Address WAL filesystem issues > - > > Key: HBASE-21688 > URL: https://issues.apache.org/jira/browse/HBASE-21688 > Project: HBase > Issue Type: Bug > Components: Filesystem Integration, wal >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Labels: s3 > Fix For: 3.0.0 > > Attachments: HBASE-21688-branch-2.0-v1.patch, > HBASE-21688-branch-2.1-v1.patch, HBASE-21688-branch-2.1-v2.patch, > HBASE-21688-branch-2.2-v1.patch, HBASE-21688-master-addendum.patch, > HBASE-21688-v1.patch > > > Scan and fix code base to use new way of instantiating WAL File System. > https://issues.apache.org/jira/browse/HBASE-21457?focusedCommentId=16734688=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16734688 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22097) Modify the description of split command in shell
[
https://issues.apache.org/jira/browse/HBASE-22097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803475#comment-16803475
]
Hudson commented on HBASE-22097:
Results for branch branch-2
[build #1780 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1780/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1780//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1780//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1780//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Modify the description of split command in shell
>
>
> Key: HBASE-22097
> URL: https://issues.apache.org/jira/browse/HBASE-22097
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0, 2.1.5, 2.2.1
>
> Attachments: HBASE-22097.master.001.patch
>
>
> We can make the description of split command in shell better.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22053) zookeeper URL links in documentation are failing with 404
[
https://issues.apache.org/jira/browse/HBASE-22053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803476#comment-16803476
]
Hudson commented on HBASE-22053:
Results for branch branch-2
[build #1780 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1780/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1780//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1780//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1780//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> zookeeper URL links in documentation are failing with 404
> -
>
> Key: HBASE-22053
> URL: https://issues.apache.org/jira/browse/HBASE-22053
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Subrat Mishra
>Assignee: Subrat Mishra
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.1.5, 2.2.1
>
> Attachments: HBASE-22053.master.001.patch,
> HBASE-22053.master.002.patch
>
>
> zookeeper URL changed from hadoop.apache.org to zookeeper.apache.org
> E.g: Below URL failing with 404
> http://hadoop.apache.org/zookeeper/docs/current/zookeeperProgrammers.html#ch_zkSessions
> should be changed to:
> https://zookeeper.apache.org/doc/current/zookeeperProgrammers.html#ch_zkSessions
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22053) zookeeper URL links in documentation are failing with 404
[
https://issues.apache.org/jira/browse/HBASE-22053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803464#comment-16803464
]
Hudson commented on HBASE-22053:
Results for branch branch-2.2
[build #137 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/137/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/137//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/137//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/137//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> zookeeper URL links in documentation are failing with 404
> -
>
> Key: HBASE-22053
> URL: https://issues.apache.org/jira/browse/HBASE-22053
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Subrat Mishra
>Assignee: Subrat Mishra
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.1.5, 2.2.1
>
> Attachments: HBASE-22053.master.001.patch,
> HBASE-22053.master.002.patch
>
>
> zookeeper URL changed from hadoop.apache.org to zookeeper.apache.org
> E.g: Below URL failing with 404
> http://hadoop.apache.org/zookeeper/docs/current/zookeeperProgrammers.html#ch_zkSessions
> should be changed to:
> https://zookeeper.apache.org/doc/current/zookeeperProgrammers.html#ch_zkSessions
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22097) Modify the description of split command in shell
[
https://issues.apache.org/jira/browse/HBASE-22097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803463#comment-16803463
]
Hudson commented on HBASE-22097:
Results for branch branch-2.2
[build #137 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/137/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/137//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/137//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/137//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Modify the description of split command in shell
>
>
> Key: HBASE-22097
> URL: https://issues.apache.org/jira/browse/HBASE-22097
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0, 2.1.5, 2.2.1
>
> Attachments: HBASE-22097.master.001.patch
>
>
> We can make the description of split command in shell better.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22094) Throw TableNotFoundException if table not exists in AsyncAdmin.compact
[ https://issues.apache.org/jira/browse/HBASE-22094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803434#comment-16803434 ] Sakthi commented on HBASE-22094: Thanks for pushing this in [~Apache9]. > Throw TableNotFoundException if table not exists in AsyncAdmin.compact > -- > > Key: HBASE-22094 > URL: https://issues.apache.org/jira/browse/HBASE-22094 > Project: HBase > Issue Type: Sub-task > Components: Admin >Reporter: Duo Zhang >Assignee: Sakthi >Priority: Major > Labels: beginner > Fix For: 3.0.0, 2.2.0, 2.3.0, 2.1.5 > > Attachments: hbase-22094.master.001.patch, > hbase-22094.master.002.patch, hbase-22094.master.003.patch, > hbase-22094.master.004.patch, hbase-22094.master.005.patch > > > Now we will return successfully, which is not the same with the HBaseAdmin. > For NORMAL compaction, we should throw TableNotFoundException if the > locations is empty. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22070) Checking restoreDir in RestoreSnapshotHelper
[
https://issues.apache.org/jira/browse/HBASE-22070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803426#comment-16803426
]
Hadoop QA commented on HBASE-22070:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
14s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
40s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 2m 40s{color}
| {color:red} hbase-server generated 4 new + 190 unchanged - 4 fixed = 194
total (was 194) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
4s{color} | {color:red} hbase-server: The patch generated 1 new + 17 unchanged
- 0 fixed = 18 total (was 17) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
13s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 1s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}131m
23s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}169m 55s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-22070 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12963056/HBASE-22070.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux e05b923af753 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 59406c44e3 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| javac |
https://builds.apache.org/job/PreCommit-HBASE-Build/16568/artifact/patchprocess/diff-compile-javac-hbase-server.txt
|
| checkstyle |
[jira] [Updated] (HBASE-22075) Potential data loss when MOB compaction fails
[ https://issues.apache.org/jira/browse/HBASE-22075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-22075: -- Status: Patch Available (was: Open) > Potential data loss when MOB compaction fails > - > > Key: HBASE-22075 > URL: https://issues.apache.org/jira/browse/HBASE-22075 > Project: HBase > Issue Type: Bug > Components: mob >Affects Versions: 2.1.3, 2.0.4, 2.1.2, 2.0.3, 2.0.2, 2.1.1, 2.0.1, 2.0.0, > 2.1.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Critical > Labels: mob > Fix For: 2.2.0, 2.0.6, 2.1.5 > > Attachments: HBASE-22075-v1.patch, HBASE-22075-v2.patch > > > When MOB compaction fails during last step (bulk load of a newly created > reference file) there is a high chance of a data loss due to partially loaded > reference file, cells of which refer to (now) non-existent MOB file. The > newly created MOB file is deleted automatically in case of a MOB compaction > failure, but some cells with the references to this file might be loaded to > HBase. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22075) Potential data loss when MOB compaction fails
[ https://issues.apache.org/jira/browse/HBASE-22075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803402#comment-16803402 ] Vladimir Rodionov commented on HBASE-22075: --- Patch v2, simplified. The only change in the code now: we do not delete compacted MOB file due to bulk load phase failure anymore. The reason is explained in previous comments. There is no need anymore to artificially inflate number of retries, as since post 2.0 LoadIncrementalHFiles tool calculates number of attempts, based on current number of regions in a table. > Potential data loss when MOB compaction fails > - > > Key: HBASE-22075 > URL: https://issues.apache.org/jira/browse/HBASE-22075 > Project: HBase > Issue Type: Bug > Components: mob >Affects Versions: 2.1.0, 2.0.0, 2.0.1, 2.1.1, 2.0.2, 2.0.3, 2.1.2, 2.0.4, > 2.1.3 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Critical > Labels: mob > Fix For: 2.2.0, 2.0.6, 2.1.5 > > Attachments: HBASE-22075-v1.patch, HBASE-22075-v2.patch > > > When MOB compaction fails during last step (bulk load of a newly created > reference file) there is a high chance of a data loss due to partially loaded > reference file, cells of which refer to (now) non-existent MOB file. The > newly created MOB file is deleted automatically in case of a MOB compaction > failure, but some cells with the references to this file might be loaded to > HBase. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22075) Potential data loss when MOB compaction fails
[ https://issues.apache.org/jira/browse/HBASE-22075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-22075: -- Status: Open (was: Patch Available) > Potential data loss when MOB compaction fails > - > > Key: HBASE-22075 > URL: https://issues.apache.org/jira/browse/HBASE-22075 > Project: HBase > Issue Type: Bug > Components: mob >Affects Versions: 2.1.3, 2.0.4, 2.1.2, 2.0.3, 2.0.2, 2.1.1, 2.0.1, 2.0.0, > 2.1.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Critical > Labels: mob > Fix For: 2.2.0, 2.0.6, 2.1.5 > > Attachments: HBASE-22075-v1.patch, HBASE-22075-v2.patch > > > When MOB compaction fails during last step (bulk load of a newly created > reference file) there is a high chance of a data loss due to partially loaded > reference file, cells of which refer to (now) non-existent MOB file. The > newly created MOB file is deleted automatically in case of a MOB compaction > failure, but some cells with the references to this file might be loaded to > HBase. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22075) Potential data loss when MOB compaction fails
[ https://issues.apache.org/jira/browse/HBASE-22075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-22075: -- Attachment: HBASE-22075-v2.patch > Potential data loss when MOB compaction fails > - > > Key: HBASE-22075 > URL: https://issues.apache.org/jira/browse/HBASE-22075 > Project: HBase > Issue Type: Bug > Components: mob >Affects Versions: 2.1.0, 2.0.0, 2.0.1, 2.1.1, 2.0.2, 2.0.3, 2.1.2, 2.0.4, > 2.1.3 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Critical > Labels: mob > Fix For: 2.2.0, 2.0.6, 2.1.5 > > Attachments: HBASE-22075-v1.patch, HBASE-22075-v2.patch > > > When MOB compaction fails during last step (bulk load of a newly created > reference file) there is a high chance of a data loss due to partially loaded > reference file, cells of which refer to (now) non-existent MOB file. The > newly created MOB file is deleted automatically in case of a MOB compaction > failure, but some cells with the references to this file might be loaded to > HBase. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21883) Enhancements to Major Compaction tool from HBASE-19528
[ https://issues.apache.org/jira/browse/HBASE-21883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803390#comment-16803390 ] Thiruvel Thirumoolan commented on HBASE-21883: -- [~stack] - Apologies, updated release notes. > Enhancements to Major Compaction tool from HBASE-19528 > -- > > Key: HBASE-21883 > URL: https://issues.apache.org/jira/browse/HBASE-21883 > Project: HBase > Issue Type: Improvement > Components: Client, Compaction, tooling >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Minor > Fix For: 3.0.0, 2.3.0, 1.5.1 > > Attachments: HBASE-21883.branch-1.001.patch, > HBASE-21883.branch-1.002.patch, HBASE-21883.master.001.patch, > HBASE-21883.master.002.patch > > > I would like to add new compaction tools based on [~churromorales]'s tool at > HBASE-19528. > We internally have tools that pick and compact regions based on multiple > criteria. Since Rahul already has a version in community, we would like to > build on top of it instead of pushing yet another tool. > With this jira, I would like to add a tool which looks at regions beyond TTL > and compacts them in a rsgroup. We have time series data and those regions > will become dead after a while, so we compact those regions to save disk > space. We also merge those empty regions to reduce load, but that tool comes > later. > Will prep a patch for 2.x once 1.5 gets in. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21883) Enhancements to Major Compaction tool from HBASE-19528
[ https://issues.apache.org/jira/browse/HBASE-21883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvel Thirumoolan updated HBASE-21883: - Release Note: MajorCompactorTTL Tool allows to compact all regions in a table that have been TTLed out. This saves space on DFS and is useful for tables which are similar to time series data. This is typically scheduled to run frequently (say via cron) to cleanup old data on an ongoing basis. RSGroupMajorCompactionTTL tool is similar to MajorCompactorTTL but runs at a region server group level. If multiple tables in an rsgroup are similar to time-series data, then it runs a single command to clean them up. As more tables are added/removed from rsgroup, it's easy to have a single command to take care of all of them. > Enhancements to Major Compaction tool from HBASE-19528 > -- > > Key: HBASE-21883 > URL: https://issues.apache.org/jira/browse/HBASE-21883 > Project: HBase > Issue Type: Improvement > Components: Client, Compaction, tooling >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Minor > Fix For: 3.0.0, 2.3.0, 1.5.1 > > Attachments: HBASE-21883.branch-1.001.patch, > HBASE-21883.branch-1.002.patch, HBASE-21883.master.001.patch, > HBASE-21883.master.002.patch > > > I would like to add new compaction tools based on [~churromorales]'s tool at > HBASE-19528. > We internally have tools that pick and compact regions based on multiple > criteria. Since Rahul already has a version in community, we would like to > build on top of it instead of pushing yet another tool. > With this jira, I would like to add a tool which looks at regions beyond TTL > and compacts them in a rsgroup. We have time series data and those regions > will become dead after a while, so we compact those regions to save disk > space. We also merge those empty regions to reduce load, but that tool comes > later. > Will prep a patch for 2.x once 1.5 gets in. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-19995) Current Jetty 9 version in HBase master branch can memory leak under high traffic
[ https://issues.apache.org/jira/browse/HBASE-19995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Lau resolved HBASE-19995. - Resolution: Fixed Closing the ticket as Jetty was later updated in HBASE-21282 so the memory leak is gone now. > Current Jetty 9 version in HBase master branch can memory leak under high > traffic > - > > Key: HBASE-19995 > URL: https://issues.apache.org/jira/browse/HBASE-19995 > Project: HBase > Issue Type: Bug > Components: REST >Affects Versions: 2.0.0 > Environment: Closing this ticket, Jetty was later upgraded in > HBASE-21282 >Reporter: Ben Lau >Priority: Major > Attachments: HBASE-19995-master.patch > > > There is a memory-leak in Jetty 9 that manifests whenever you hit the call > queue limit in HBase REST. The memory-leak leaks both on-heap and off-heap > objects permanently. It happens because whenever the call queue for Jetty > server overflows, the task that is rejected runs a 'reject' method if it is a > Rejectable to do any cleanup. This clean up is necessary to for example close > the connection, deallocate any buffers, etc. Unfortunately, in Jetty 9, they > implemented the 'reject' / cleanup method of the SelectChannelEndpoint as a > non-blocking call that is not guaranteed to run. This was later fixed in > Jetty 9.4 and later backported however the version of Jetty 9 pulled in HBase > for REST comes before this fix. See > [https://github.com/eclipse/jetty.project/issues/1804] and > [https://github.com/apache/hbase/blob/master/pom.xml#L1416.] > If we want to stay on 9.3.X we could update to > [9.3.22.v20171030|https://mvnrepository.com/artifact/org.eclipse.jetty/jetty-server/9.3.22.v20171030] > which is the latest version of 9.3. Thoughts? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19995) Current Jetty 9 version in HBase master branch can memory leak under high traffic
[ https://issues.apache.org/jira/browse/HBASE-19995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Lau updated HBASE-19995: Affects Version/s: 2.0.0 Environment: Closing this ticket, Jetty was later upgraded in HBASE-21282 > Current Jetty 9 version in HBase master branch can memory leak under high > traffic > - > > Key: HBASE-19995 > URL: https://issues.apache.org/jira/browse/HBASE-19995 > Project: HBase > Issue Type: Bug > Components: REST >Affects Versions: 2.0.0 > Environment: Closing this ticket, Jetty was later upgraded in > HBASE-21282 >Reporter: Ben Lau >Priority: Major > Attachments: HBASE-19995-master.patch > > > There is a memory-leak in Jetty 9 that manifests whenever you hit the call > queue limit in HBase REST. The memory-leak leaks both on-heap and off-heap > objects permanently. It happens because whenever the call queue for Jetty > server overflows, the task that is rejected runs a 'reject' method if it is a > Rejectable to do any cleanup. This clean up is necessary to for example close > the connection, deallocate any buffers, etc. Unfortunately, in Jetty 9, they > implemented the 'reject' / cleanup method of the SelectChannelEndpoint as a > non-blocking call that is not guaranteed to run. This was later fixed in > Jetty 9.4 and later backported however the version of Jetty 9 pulled in HBase > for REST comes before this fix. See > [https://github.com/eclipse/jetty.project/issues/1804] and > [https://github.com/apache/hbase/blob/master/pom.xml#L1416.] > If we want to stay on 9.3.X we could update to > [9.3.22.v20171030|https://mvnrepository.com/artifact/org.eclipse.jetty/jetty-server/9.3.22.v20171030] > which is the latest version of 9.3. Thoughts? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21718) Implement Admin based on AsyncAdmin
[
https://issues.apache.org/jira/browse/HBASE-21718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803354#comment-16803354
]
Hadoop QA commented on HBASE-21718:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 38 new or modified test
files. {color} |
|| || || || {color:brown} HBASE-21512 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
21s{color} | {color:green} HBASE-21512 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
23s{color} | {color:green} HBASE-21512 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
22s{color} | {color:green} HBASE-21512 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
26s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
24s{color} | {color:green} HBASE-21512 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
12s{color} | {color:green} HBASE-21512 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 40s{color}
| {color:red} hbase-client generated 2 new + 127 unchanged - 2 fixed = 129
total (was 129) {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 3m 40s{color}
| {color:red} hbase-server generated 12 new + 182 unchanged - 12 fixed = 194
total (was 194) {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 52s{color}
| {color:red} hbase-mapreduce generated 1 new + 156 unchanged - 1 fixed = 157
total (was 157) {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 7s{color}
| {color:red} hbase-thrift generated 3 new + 24 unchanged - 3 fixed = 27 total
(was 27) {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 43s{color}
| {color:red} hbase-backup generated 2 new + 50 unchanged - 2 fixed = 52 total
(was 52) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} hbase-client: The patch generated 0 new + 119
unchanged - 4 fixed = 119 total (was 123) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
36s{color} | {color:green} hbase-server: The patch generated 0 new + 387
unchanged - 1 fixed = 387 total (was 388) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} The patch passed checkstyle in hbase-mapreduce
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} The patch passed checkstyle in hbase-thrift {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} The patch passed checkstyle in hbase-backup {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
23s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 37s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 8m
30s{color} | {color:green} the patch
[jira] [Comment Edited] (HBASE-22119) Provide functions in RSGroupInfo to check if a group is the default group
[
https://issues.apache.org/jira/browse/HBASE-22119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803305#comment-16803305
]
Xu Cang edited comment on HBASE-22119 at 3/27/19 8:08 PM:
--
LGTM +1
(let's have a branch-1 patch if you don't mind, then I will push) thanks
was (Author: xucang):
LGTM +1
> Provide functions in RSGroupInfo to check if a group is the default group
> -
>
> Key: HBASE-22119
> URL: https://issues.apache.org/jira/browse/HBASE-22119
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22119.master.000.patch
>
>
> There are several places to check if a group is the "default" group, where
> the input could be a String or a RSGroupInfo.
> It is better to provide official functions in RSGroupInfo to tell if a group
> is the default group, so as to
> * Simply the code
> * Make it more safe. It is not safe as there is no null check like
> {code}
> if (!group.getName().equals(RSGroupInfo.DEFAULT_GROUP))
> {code}
> It is more safe to check like:
> {code}
> RSGroupInfo.DEFAULT_GROUP.equals(group.getName())
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22119) Provide functions in RSGroupInfo to check if a group is the default group
[
https://issues.apache.org/jira/browse/HBASE-22119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803305#comment-16803305
]
Xu Cang commented on HBASE-22119:
-
LGTM +1
> Provide functions in RSGroupInfo to check if a group is the default group
> -
>
> Key: HBASE-22119
> URL: https://issues.apache.org/jira/browse/HBASE-22119
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22119.master.000.patch
>
>
> There are several places to check if a group is the "default" group, where
> the input could be a String or a RSGroupInfo.
> It is better to provide official functions in RSGroupInfo to tell if a group
> is the default group, so as to
> * Simply the code
> * Make it more safe. It is not safe as there is no null check like
> {code}
> if (!group.getName().equals(RSGroupInfo.DEFAULT_GROUP))
> {code}
> It is more safe to check like:
> {code}
> RSGroupInfo.DEFAULT_GROUP.equals(group.getName())
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22031) Provide RSGroupInfo with a new constructor using shallow copy
[
https://issues.apache.org/jira/browse/HBASE-22031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803304#comment-16803304
]
Xu Cang commented on HBASE-22031:
-
There are 2 ways can possibly cause issues.
Original holder of this object modifies the 'tables' or new object who is
pointing to the shared object can modify it.
I am still open to discuss this with you, waiting for your new review as you
mentioned above. Thanks.
> Provide RSGroupInfo with a new constructor using shallow copy
> -
>
> Key: HBASE-22031
> URL: https://issues.apache.org/jira/browse/HBASE-22031
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22031.master.000.patch,
> HBASE-22031.master.001.patch, HBASE-22031.master.002.patch
>
>
> As for org.apache.hadoop.hbase.rsgroup.RSGroupInfo, the following constructor
> performs deep copies on both servers and tables inputed.
> {code:title=hbase-common/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfo.java.java|borderStyle=solid}
> RSGroupInfo(String name, SortedSet servers, SortedSet
> tables) {
> this.name = name;
> this.servers = (servers == null) ? new TreeSet<>() : new TreeSet<>(servers);
> this.tables = (tables == null) ? new TreeSet<>() : new TreeSet<>(tables);
> }
> {code}
> The constructor of TreeSet is heavy and I think it is better to have a new
> constructor with shallow copy and it could be used at least in
> {code:title=hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManagerImpl.java|borderStyle=solid}
> private synchronized void refresh(boolean forceOnline) throws IOException {
> ...
> groupList.add(new RSGroupInfo(RSGroupInfo.DEFAULT_GROUP,
> getDefaultServers(), orphanTables));
> ...
> {code}
> It is not needed to allocate new TreeSet to deep copy the output of
> getDefaultServers() and orphanTables, both of which are allocated in the near
> context and not updated in the code followed. So it is safe to make a shadow
> copy here.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-20662:
---
Fix Version/s: 2.1.5
2.0.6
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.0.6, 2.1.5
>
> Attachments: HBASE-20662.branch-2.1.001.patch,
> HBASE-20662.master.001.patch, HBASE-20662.master.002.patch,
> HBASE-20662.master.003.patch, HBASE-20662.master.004.patch,
> HBASE-20662.master.004.patch, HBASE-20662.master.005.patch,
> HBASE-20662.master.006.patch, HBASE-20662.master.007.patch,
> HBASE-20662.master.008.patch, HBASE-20662.master.008.patch,
> HBASE-20662.master.009.patch, HBASE-20662.master.009.patch,
> HBASE-20662.master.010.patch, screenshot.png
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService methodName:
> EnableTable size: 104 connection: 127.0.0.1:38030 deadline: 1527707127568,
> exception=org.apache.hadoop.hbase.security.AccessDeniedException: Enabling
> the table 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to
> a violated space quota.
> 2018-05-31 00:34:27,571 ERROR [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> quotas.RegionServerSpaceQuotaManager(210): Failed to disable space violation
> policy for testSetQuotaAndThenIncreaseQuotaWithDisable0. This table will
> remain in violation.
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Enabling the table
> 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to a
> violated space quota.
> at org.apache.hadoop.hbase.master.HMaster$6.run(HMaster.java:2275)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:131)
> at org.apache.hadoop.hbase.master.HMaster.enableTable(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.enableTable(MasterRpcServices.java:725)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>
[jira] [Updated] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-20662:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.0.6, 2.1.5
>
> Attachments: HBASE-20662.branch-2.1.001.patch,
> HBASE-20662.master.001.patch, HBASE-20662.master.002.patch,
> HBASE-20662.master.003.patch, HBASE-20662.master.004.patch,
> HBASE-20662.master.004.patch, HBASE-20662.master.005.patch,
> HBASE-20662.master.006.patch, HBASE-20662.master.007.patch,
> HBASE-20662.master.008.patch, HBASE-20662.master.008.patch,
> HBASE-20662.master.009.patch, HBASE-20662.master.009.patch,
> HBASE-20662.master.010.patch, screenshot.png
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService methodName:
> EnableTable size: 104 connection: 127.0.0.1:38030 deadline: 1527707127568,
> exception=org.apache.hadoop.hbase.security.AccessDeniedException: Enabling
> the table 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to
> a violated space quota.
> 2018-05-31 00:34:27,571 ERROR [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> quotas.RegionServerSpaceQuotaManager(210): Failed to disable space violation
> policy for testSetQuotaAndThenIncreaseQuotaWithDisable0. This table will
> remain in violation.
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Enabling the table
> 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to a
> violated space quota.
> at org.apache.hadoop.hbase.master.HMaster$6.run(HMaster.java:2275)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:131)
> at org.apache.hadoop.hbase.master.HMaster.enableTable(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.enableTable(MasterRpcServices.java:725)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at
[jira] [Updated] (HBASE-22057) Impose upper-bound on size of ZK ops sent in a single multi()
[
https://issues.apache.org/jira/browse/HBASE-22057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-22057:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Release Note: Exposes a new configuration property
"zookeeper.multi.max.size" which dictates the maximum size of deletes that
HBase will make to ZooKeeper in a single RPC. This property defaults to 1MB,
which should fall beneath the default ZooKeeper limit of 2MB, controlled by
"jute.maxbuffer".
Status: Resolved (was: Patch Available)
> Impose upper-bound on size of ZK ops sent in a single multi()
> -
>
> Key: HBASE-22057
> URL: https://issues.apache.org/jira/browse/HBASE-22057
> Project: HBase
> Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-22057.001.patch, HBASE-22057.002.patch,
> HBASE-22057.003.patch, HBASE-22057.004.patch
>
>
> In {{ZKUtil#multiOrSequential}}, we accept a list of {{ZKUtilOp}}'s to pass
> down to the {{ZooKeeper#multi(Iterable)}} method.
> One problem with this approach is that we may generate a large list of ZNodes
> to mutate in one batch which exceeds the allowable client package length,
> specified by {{jute.maxbuffer}}.
> This problem can manifest when we have a large number of WALs to replicate,
> queued in ZooKeeper, from a disabled peer. When that peer is dropped, the RS
> would submit deletes of those queued WALs. The RS will see ConnectionLoss for
> the resulting {{multi()}} calls it tries to make, because we are sending too
> large of a client message (because we're trying to delete too many WALs at
> once). The result (at least in branch-1 ish versions) is that the RS aborts
> after exceeding the ZK retries (as this operation will never succeed).
> A simple fix would be to impose a maximum number of Ops to run in a single
> batch inside ZKUtil, and split apart the caller-submitted batch into smaller
> chunks. Before we make such a change, I do need to make sure that we don't
> have any expectations on atomicity of the operations. I'm not sure what ZK
> provides here -- for the above example, splitting up batches of deletes is
> not an issue, but there could be issues with batches of creates where we only
> apply some.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22057) Impose upper-bound on size of ZK ops sent in a single multi()
[
https://issues.apache.org/jira/browse/HBASE-22057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-22057:
---
Fix Version/s: (was: 1.6.0)
> Impose upper-bound on size of ZK ops sent in a single multi()
> -
>
> Key: HBASE-22057
> URL: https://issues.apache.org/jira/browse/HBASE-22057
> Project: HBase
> Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-22057.001.patch, HBASE-22057.002.patch,
> HBASE-22057.003.patch, HBASE-22057.004.patch
>
>
> In {{ZKUtil#multiOrSequential}}, we accept a list of {{ZKUtilOp}}'s to pass
> down to the {{ZooKeeper#multi(Iterable)}} method.
> One problem with this approach is that we may generate a large list of ZNodes
> to mutate in one batch which exceeds the allowable client package length,
> specified by {{jute.maxbuffer}}.
> This problem can manifest when we have a large number of WALs to replicate,
> queued in ZooKeeper, from a disabled peer. When that peer is dropped, the RS
> would submit deletes of those queued WALs. The RS will see ConnectionLoss for
> the resulting {{multi()}} calls it tries to make, because we are sending too
> large of a client message (because we're trying to delete too many WALs at
> once). The result (at least in branch-1 ish versions) is that the RS aborts
> after exceeding the ZK retries (as this operation will never succeed).
> A simple fix would be to impose a maximum number of Ops to run in a single
> batch inside ZKUtil, and split apart the caller-submitted batch into smaller
> chunks. Before we make such a change, I do need to make sure that we don't
> have any expectations on atomicity of the operations. I'm not sure what ZK
> provides here -- for the above example, splitting up batches of deletes is
> not an issue, but there could be issues with batches of creates where we only
> apply some.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21688) Address WAL filesystem issues
[ https://issues.apache.org/jira/browse/HBASE-21688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803297#comment-16803297 ] Josh Elser commented on HBASE-21688: Sounds good. Thanks Vlad. > Address WAL filesystem issues > - > > Key: HBASE-21688 > URL: https://issues.apache.org/jira/browse/HBASE-21688 > Project: HBase > Issue Type: Bug > Components: Filesystem Integration, wal >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Labels: s3 > Fix For: 3.0.0 > > Attachments: HBASE-21688-branch-2.1-v1.patch, > HBASE-21688-branch-2.1-v2.patch, HBASE-21688-branch-2.2-v1.patch, > HBASE-21688-master-addendum.patch, HBASE-21688-v1.patch > > > Scan and fix code base to use new way of instantiating WAL File System. > https://issues.apache.org/jira/browse/HBASE-21457?focusedCommentId=16734688=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16734688 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22070) Checking restoreDir in RestoreSnapshotHelper
[ https://issues.apache.org/jira/browse/HBASE-22070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-22070: -- Status: Patch Available (was: Open) > Checking restoreDir in RestoreSnapshotHelper > - > > Key: HBASE-22070 > URL: https://issues.apache.org/jira/browse/HBASE-22070 > Project: HBase > Issue Type: Bug > Components: snapshots >Affects Versions: 2.0.0 >Reporter: Vincent Choi >Assignee: Vincent Choi >Priority: Major > Attachments: HBASE-22070.patch > > > The restoreDir shouldn't be a sub directory of rootDir. The code check it > with a prefix check > "restoreDir.toUri().getPath().startsWith(rootDir.toUri().getPath())". But it > goes error in some reasonable cases. > eg: rootDir = hdfs://user/hbase restoreDir = hdfs://user/hbase_restore. So I > think it's more reasonable to chang the code to > "restoreDir.toUri().getPath().startsWith(rootDir.toUri().getPath() + "/" )". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21456) Make WALFactory only used for creating WALProviders
[
https://issues.apache.org/jira/browse/HBASE-21456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803280#comment-16803280
]
Josh Elser commented on HBASE-21456:
{noformat}
- reader = WALFactory.createReader(fs, edits, conf);
+ reader = WALProviderFactory.getInstance(conf).createReader(fs, edits,
null, true);{noformat}
I see a lot of changes where we just pass down {{null, true}} as the third and
fourth arguments to {{createReader}}. Would it make sense to overload the
{{createReader}} method, creating a 2-arg and a 4-arg version?
Overall, I think this looks good. I think it helps clear up some of the
ambiguity we had before. Have you investigated if the test values are related
to your patch?
> Make WALFactory only used for creating WALProviders
> ---
>
> Key: HBASE-21456
> URL: https://issues.apache.org/jira/browse/HBASE-21456
> Project: HBase
> Issue Type: Sub-task
> Components: wal
>Affects Versions: HBASE-20952
>Reporter: Josh Elser
>Assignee: Ankit Singhal
>Priority: Major
> Fix For: HBASE-20952
>
> Attachments: HBASE-21456.HBASE-20952.001.patch,
> HBASE-21456.HBASE-20952.002.patch, HBASE-21456.HBASE-20952.003.patch,
> HBASE-21456.HBASE-20952.wip.patch
>
>
> As a Factory, WALFactory should only have one job: creating instances of
> WALProvider.
> However, over the years, it has been a landing place for lots of wal-related
> methods (e.g. constructing readers, WALEntryStream, and more). We want all of
> this to live in the WALProvider.
> We can do this in two steps: have the WALFactory methods invoke the method on
> the WALProvider (handled elsewhere), then here we can replace usage of the
> WALFactory "wrapper methods" with the WALProvider itself.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21688) Address WAL filesystem issues
[ https://issues.apache.org/jira/browse/HBASE-21688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803279#comment-16803279 ] Vladimir Rodionov commented on HBASE-21688: --- [~elserj], This patch will be ready for commit soon. I will update master and will add 2.0 branch version today. For 1.x branches back ports I will open separate JIRA. > Address WAL filesystem issues > - > > Key: HBASE-21688 > URL: https://issues.apache.org/jira/browse/HBASE-21688 > Project: HBase > Issue Type: Bug > Components: Filesystem Integration, wal >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Labels: s3 > Fix For: 3.0.0 > > Attachments: HBASE-21688-branch-2.1-v1.patch, > HBASE-21688-branch-2.1-v2.patch, HBASE-21688-branch-2.2-v1.patch, > HBASE-21688-master-addendum.patch, HBASE-21688-v1.patch > > > Scan and fix code base to use new way of instantiating WAL File System. > https://issues.apache.org/jira/browse/HBASE-21457?focusedCommentId=16734688=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16734688 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22094) Throw TableNotFoundException if table not exists in AsyncAdmin.compact
[
https://issues.apache.org/jira/browse/HBASE-22094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803278#comment-16803278
]
Hudson commented on HBASE-22094:
Results for branch master
[build #890 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/890/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/890//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/890//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/890//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Throw TableNotFoundException if table not exists in AsyncAdmin.compact
> --
>
> Key: HBASE-22094
> URL: https://issues.apache.org/jira/browse/HBASE-22094
> Project: HBase
> Issue Type: Sub-task
> Components: Admin
>Reporter: Duo Zhang
>Assignee: Sakthi
>Priority: Major
> Labels: beginner
> Fix For: 3.0.0, 2.2.0, 2.3.0, 2.1.5
>
> Attachments: hbase-22094.master.001.patch,
> hbase-22094.master.002.patch, hbase-22094.master.003.patch,
> hbase-22094.master.004.patch, hbase-22094.master.005.patch
>
>
> Now we will return successfully, which is not the same with the HBaseAdmin.
> For NORMAL compaction, we should throw TableNotFoundException if the
> locations is empty.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22102) Remove AsyncAdmin.isTableAvailable(TableName, byte[][])
[
https://issues.apache.org/jira/browse/HBASE-22102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803277#comment-16803277
]
Hudson commented on HBASE-22102:
Results for branch master
[build #890 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/890/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/890//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/890//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/890//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove AsyncAdmin.isTableAvailable(TableName, byte[][])
> ---
>
> Key: HBASE-22102
> URL: https://issues.apache.org/jira/browse/HBASE-22102
> Project: HBase
> Issue Type: Task
> Components: Admin, asyncclient, Client
>Reporter: Duo Zhang
>Assignee: kevin su
>Priority: Major
> Labels: beginner
> Fix For: 3.0.0
>
> Attachments: HBASE-22102.v0.patch, HBASE-22102.v1.patch
>
>
> It has been deprecated so remove it in 3.0.0, to align with the Admin
> interface.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21911) Move getUserPermissions from regionserver to master
[
https://issues.apache.org/jira/browse/HBASE-21911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803266#comment-16803266
]
Hudson commented on HBASE-21911:
Results for branch branch-2.2
[build #136 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/136/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/136//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/136//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/136//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move getUserPermissions from regionserver to master
> ---
>
> Key: HBASE-21911
> URL: https://issues.apache.org/jira/browse/HBASE-21911
> Project: HBase
> Issue Type: Sub-task
>Reporter: Yi Mei
>Assignee: Yi Mei
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.3.0
>
> Attachments: HBASE-21911.branch-2.001.patch,
> HBASE-21911.branch-2.2.001.patch, HBASE-21911.master.003.patch,
> HBASE-21911.master.004.patch, HBASE-21911.master.005.patch,
> HBASE-21911.master.006.patch
>
>
> Create a sub-task to move acl methods: getUserPermissions from regionserver
> to master.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22094) Throw TableNotFoundException if table not exists in AsyncAdmin.compact
[
https://issues.apache.org/jira/browse/HBASE-22094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803267#comment-16803267
]
Hudson commented on HBASE-22094:
Results for branch branch-2.2
[build #136 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/136/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/136//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/136//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/136//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Throw TableNotFoundException if table not exists in AsyncAdmin.compact
> --
>
> Key: HBASE-22094
> URL: https://issues.apache.org/jira/browse/HBASE-22094
> Project: HBase
> Issue Type: Sub-task
> Components: Admin
>Reporter: Duo Zhang
>Assignee: Sakthi
>Priority: Major
> Labels: beginner
> Fix For: 3.0.0, 2.2.0, 2.3.0, 2.1.5
>
> Attachments: hbase-22094.master.001.patch,
> hbase-22094.master.002.patch, hbase-22094.master.003.patch,
> hbase-22094.master.004.patch, hbase-22094.master.005.patch
>
>
> Now we will return successfully, which is not the same with the HBaseAdmin.
> For NORMAL compaction, we should throw TableNotFoundException if the
> locations is empty.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-22070) Checking restoreDir in RestoreSnapshotHelper
[ https://issues.apache.org/jira/browse/HBASE-22070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi reassigned HBASE-22070: - Assignee: Vincent Choi > Checking restoreDir in RestoreSnapshotHelper > - > > Key: HBASE-22070 > URL: https://issues.apache.org/jira/browse/HBASE-22070 > Project: HBase > Issue Type: Bug > Components: snapshots >Affects Versions: 2.0.0 >Reporter: Vincent Choi >Assignee: Vincent Choi >Priority: Major > Attachments: HBASE-22070.patch > > > The restoreDir shouldn't be a sub directory of rootDir. The code check it > with a prefix check > "restoreDir.toUri().getPath().startsWith(rootDir.toUri().getPath())". But it > goes error in some reasonable cases. > eg: rootDir = hdfs://user/hbase restoreDir = hdfs://user/hbase_restore. So I > think it's more reasonable to chang the code to > "restoreDir.toUri().getPath().startsWith(rootDir.toUri().getPath() + "/" )". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22070) Checking restoreDir in RestoreSnapshotHelper
[ https://issues.apache.org/jira/browse/HBASE-22070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803263#comment-16803263 ] Peter Somogyi commented on HBASE-22070: --- [~VincentChoiCN], I added you as a contributor and assigned this issue to you. > Checking restoreDir in RestoreSnapshotHelper > - > > Key: HBASE-22070 > URL: https://issues.apache.org/jira/browse/HBASE-22070 > Project: HBase > Issue Type: Bug > Components: snapshots >Affects Versions: 2.0.0 >Reporter: Vincent Choi >Assignee: Vincent Choi >Priority: Major > Attachments: HBASE-22070.patch > > > The restoreDir shouldn't be a sub directory of rootDir. The code check it > with a prefix check > "restoreDir.toUri().getPath().startsWith(rootDir.toUri().getPath())". But it > goes error in some reasonable cases. > eg: rootDir = hdfs://user/hbase restoreDir = hdfs://user/hbase_restore. So I > think it's more reasonable to chang the code to > "restoreDir.toUri().getPath().startsWith(rootDir.toUri().getPath() + "/" )". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22094) Throw TableNotFoundException if table not exists in AsyncAdmin.compact
[
https://issues.apache.org/jira/browse/HBASE-22094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803254#comment-16803254
]
Hudson commented on HBASE-22094:
Results for branch branch-2
[build #1779 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1779/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1779//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1779//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1779//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Throw TableNotFoundException if table not exists in AsyncAdmin.compact
> --
>
> Key: HBASE-22094
> URL: https://issues.apache.org/jira/browse/HBASE-22094
> Project: HBase
> Issue Type: Sub-task
> Components: Admin
>Reporter: Duo Zhang
>Assignee: Sakthi
>Priority: Major
> Labels: beginner
> Fix For: 3.0.0, 2.2.0, 2.3.0, 2.1.5
>
> Attachments: hbase-22094.master.001.patch,
> hbase-22094.master.002.patch, hbase-22094.master.003.patch,
> hbase-22094.master.004.patch, hbase-22094.master.005.patch
>
>
> Now we will return successfully, which is not the same with the HBaseAdmin.
> For NORMAL compaction, we should throw TableNotFoundException if the
> locations is empty.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21911) Move getUserPermissions from regionserver to master
[
https://issues.apache.org/jira/browse/HBASE-21911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803253#comment-16803253
]
Hudson commented on HBASE-21911:
Results for branch branch-2
[build #1779 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1779/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1779//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1779//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1779//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move getUserPermissions from regionserver to master
> ---
>
> Key: HBASE-21911
> URL: https://issues.apache.org/jira/browse/HBASE-21911
> Project: HBase
> Issue Type: Sub-task
>Reporter: Yi Mei
>Assignee: Yi Mei
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.3.0
>
> Attachments: HBASE-21911.branch-2.001.patch,
> HBASE-21911.branch-2.2.001.patch, HBASE-21911.master.003.patch,
> HBASE-21911.master.004.patch, HBASE-21911.master.005.patch,
> HBASE-21911.master.006.patch
>
>
> Create a sub-task to move acl methods: getUserPermissions from regionserver
> to master.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21512) Introduce an AsyncClusterConnection and replace the usage of ClusterConnection
[
https://issues.apache.org/jira/browse/HBASE-21512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803223#comment-16803223
]
Hudson commented on HBASE-21512:
Results for branch HBASE-21512
[build #152 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/152/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/152//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/152//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/152//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Introduce an AsyncClusterConnection and replace the usage of ClusterConnection
> --
>
> Key: HBASE-21512
> URL: https://issues.apache.org/jira/browse/HBASE-21512
> Project: HBase
> Issue Type: Umbrella
>Reporter: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
>
> At least for the RSProcedureDispatcher, with CompletableFuture we do not need
> to set a delay and use a thread pool any more, which could reduce the
> resource usage and also the latency.
> Once this is done, I think we can remove the ClusterConnection completely,
> and start to rewrite the old sync client based on the async client, which
> could reduce the code base a lot for our client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22120) remove HTrace
[ https://issues.apache.org/jira/browse/HBASE-22120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803220#comment-16803220 ] Andrew Purtell commented on HBASE-22120: We need a replacement concurrent with removal, please. One that aligns with Hadoop so we have whole stack tracing capability. > remove HTrace > - > > Key: HBASE-22120 > URL: https://issues.apache.org/jira/browse/HBASE-22120 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Sergey Shelukhin >Priority: Major > > Suggested in HBASE-22115 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22115) HBase RPC aspires to grow an infinite tree of trace scopes; some other places are also unsafe
[ https://issues.apache.org/jira/browse/HBASE-22115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803219#comment-16803219 ] Andrew Purtell commented on HBASE-22115: Until we have a workable story for HTrace replacement, we need to fix it the best we can. This change lgtm (although I only skimmed it) > HBase RPC aspires to grow an infinite tree of trace scopes; some other places > are also unsafe > - > > Key: HBASE-22115 > URL: https://issues.apache.org/jira/browse/HBASE-22115 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Critical > Attachments: Clipboard01.jpg, HBASE-22115.01.patch, HBASE-22115.patch > > > !Clipboard01.jpg! > All of those are ClientServices.Multi in this case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22118) Backport HBASE-22095 "Taking a snapshot fails in local mode"
[ https://issues.apache.org/jira/browse/HBASE-22118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803217#comment-16803217 ] Andrew Purtell commented on HBASE-22118: Ok, great, thank you for checking > Backport HBASE-22095 "Taking a snapshot fails in local mode" > > > Key: HBASE-22118 > URL: https://issues.apache.org/jira/browse/HBASE-22118 > Project: HBase > Issue Type: Sub-task >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > > Backport the parent issue to branch-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22095) Taking a snapshot fails in local mode
[ https://issues.apache.org/jira/browse/HBASE-22095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803215#comment-16803215 ] Andrew Purtell commented on HBASE-22095: Much appreciated [~brfrn169]! > Taking a snapshot fails in local mode > - > > Key: HBASE-22095 > URL: https://issues.apache.org/jira/browse/HBASE-22095 > Project: HBase > Issue Type: Bug >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.1 > > Attachments: HBASE-22095.master.001.patch > > > It looks like after HBASE-21098, taking a snapshot fails in local mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22120) remove HTrace
[ https://issues.apache.org/jira/browse/HBASE-22120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803206#comment-16803206 ] Sergey Shelukhin commented on HBASE-22120: -- That is an option... I think the original proposal is to just nuke it. We can replace in the same change cc [~stack] [~Apache9] > remove HTrace > - > > Key: HBASE-22120 > URL: https://issues.apache.org/jira/browse/HBASE-22120 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Sergey Shelukhin >Priority: Major > > Suggested in HBASE-22115 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22120) remove HTrace
[ https://issues.apache.org/jira/browse/HBASE-22120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803201#comment-16803201 ] Wei-Chiu Chuang edited comment on HBASE-22120 at 3/27/19 6:38 PM: -- In Hadoop, we're discussing to replace HTrace with OpenTracing, and ozone has already started with OpenTracing. If that's the goal here I'd be happy to participate. HADOOP-15566 is the jira. was (Author: jojochuang): In Hadoop, we're discussing to replace HTrace with OpenTracing, and ozone is already started with OpenTracing. If that's the goal here I'd be happy to participate. HADOOP-15566 is the jira. > remove HTrace > - > > Key: HBASE-22120 > URL: https://issues.apache.org/jira/browse/HBASE-22120 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Sergey Shelukhin >Priority: Major > > Suggested in HBASE-22115 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22120) remove HTrace
[ https://issues.apache.org/jira/browse/HBASE-22120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803201#comment-16803201 ] Wei-Chiu Chuang commented on HBASE-22120: - In Hadoop, we're discussing to replace HTrace with OpenTracing, and ozone is already started with OpenTracing. If that's the goal here I'd be happy to participate. > remove HTrace > - > > Key: HBASE-22120 > URL: https://issues.apache.org/jira/browse/HBASE-22120 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Sergey Shelukhin >Priority: Major > > Suggested in HBASE-22115 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22120) remove HTrace
[ https://issues.apache.org/jira/browse/HBASE-22120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803201#comment-16803201 ] Wei-Chiu Chuang edited comment on HBASE-22120 at 3/27/19 6:37 PM: -- In Hadoop, we're discussing to replace HTrace with OpenTracing, and ozone is already started with OpenTracing. If that's the goal here I'd be happy to participate. HADOOP-15566 is the jira. was (Author: jojochuang): In Hadoop, we're discussing to replace HTrace with OpenTracing, and ozone is already started with OpenTracing. If that's the goal here I'd be happy to participate. > remove HTrace > - > > Key: HBASE-22120 > URL: https://issues.apache.org/jira/browse/HBASE-22120 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Sergey Shelukhin >Priority: Major > > Suggested in HBASE-22115 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22117) Move hasPermission/checkPermissions from region server to master
[
https://issues.apache.org/jira/browse/HBASE-22117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803184#comment-16803184
]
Hadoop QA commented on HBASE-22117:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
3s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 5m
25s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 51s{color}
| {color:red} hbase-client generated 2 new + 127 unchanged - 2 fixed = 129
total (was 129) {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 2m 38s{color}
| {color:red} hbase-server generated 12 new + 182 unchanged - 12 fixed = 194
total (was 194) {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 49s{color}
| {color:red} hbase-thrift generated 2 new + 25 unchanged - 2 fixed = 27 total
(was 27) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch passed checkstyle in hbase-protocol-shaded
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} The patch passed checkstyle in hbase-client {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} hbase-server: The patch generated 0 new + 154
unchanged - 4 fixed = 154 total (was 158) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} The patch passed checkstyle in hbase-thrift {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
59s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 39s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
35s{color} | {color:green} hbase-protocol-shaded in the patch
[jira] [Commented] (HBASE-22115) HBase RPC aspires to grow an infinite tree of trace scopes; some other places are also unsafe
[ https://issues.apache.org/jira/browse/HBASE-22115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803170#comment-16803170 ] Sergey Shelukhin commented on HBASE-22115: -- Fixed whitespace; filed HBASE-22120 for HTrace removal > HBase RPC aspires to grow an infinite tree of trace scopes; some other places > are also unsafe > - > > Key: HBASE-22115 > URL: https://issues.apache.org/jira/browse/HBASE-22115 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Critical > Attachments: Clipboard01.jpg, HBASE-22115.01.patch, HBASE-22115.patch > > > !Clipboard01.jpg! > All of those are ClientServices.Multi in this case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22120) remove HTrace
Sergey Shelukhin created HBASE-22120: Summary: remove HTrace Key: HBASE-22120 URL: https://issues.apache.org/jira/browse/HBASE-22120 Project: HBase Issue Type: Bug Affects Versions: 3.0.0 Reporter: Sergey Shelukhin Suggested in HBASE-22115 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22115) HBase RPC aspires to grow an infinite tree of trace scopes; some other places are also unsafe
[ https://issues.apache.org/jira/browse/HBASE-22115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-22115: - Attachment: HBASE-22115.01.patch > HBase RPC aspires to grow an infinite tree of trace scopes; some other places > are also unsafe > - > > Key: HBASE-22115 > URL: https://issues.apache.org/jira/browse/HBASE-22115 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Critical > Attachments: Clipboard01.jpg, HBASE-22115.01.patch, HBASE-22115.patch > > > !Clipboard01.jpg! > All of those are ClientServices.Multi in this case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22052) pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove redunant version specifications
[ https://issues.apache.org/jira/browse/HBASE-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803167#comment-16803167 ] stack commented on HBASE-22052: --- [~busbey] Thanks. Nice release note. The notice on downloads page shows now http://hbase.apache.org/downloads.html > pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove > redunant version specifications > --- > > Key: HBASE-22052 > URL: https://issues.apache.org/jira/browse/HBASE-22052 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.3.0, 2.1.4 > > Attachments: HBASE-22052-branch-2.1.003.addendum2.txt, > HBASE-22052.2.1.003.addendum1.txt, HBASE-22052.branch-2.0.001.patch, > HBASE-22052.branch-2.0.002.patch, HBASE-22052.branch-2.0.002.patch, > HBASE-22052.branch-2.0.003.patch, HBASE-22052.branch-2.0.004.patch, > HBASE-22052.branch-2.1.001.patch, HBASE-22052.branch-2.1.002.patch, > HBASE-22052.branch-2.1.003.patch, HBASE-22052.branch-2.2.001.patch, > HBASE-22052.branch-2.2.002.patch, nothing_change.patch, nothing_change.patch > > > Working on HBASE-22029, where we fail compile of hbase-it module four hours > into an RC build, it looks like transitive include of jersey-core is the > culprit. hadoop3 profile does a bunch of jersey-core exclusions. This issue > is about having hadoop2 profile and hadoop3 profiles match around jersey-core > treatment. Some miscellaneous cleanups are also done. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20662) Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
[
https://issues.apache.org/jira/browse/HBASE-20662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803157#comment-16803157
]
Josh Elser commented on HBASE-20662:
Thanks, Nihal. Happy to commit this to 2.1.
> Increasing space quota on a violated table does not remove
> SpaceViolationPolicy.DISABLE enforcement
> ---
>
> Key: HBASE-20662
> URL: https://issues.apache.org/jira/browse/HBASE-20662
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-20662.branch-2.1.001.patch,
> HBASE-20662.master.001.patch, HBASE-20662.master.002.patch,
> HBASE-20662.master.003.patch, HBASE-20662.master.004.patch,
> HBASE-20662.master.004.patch, HBASE-20662.master.005.patch,
> HBASE-20662.master.006.patch, HBASE-20662.master.007.patch,
> HBASE-20662.master.008.patch, HBASE-20662.master.008.patch,
> HBASE-20662.master.009.patch, HBASE-20662.master.009.patch,
> HBASE-20662.master.010.patch, screenshot.png
>
>
> *Steps to reproduce*
> * Create a table and set quota with {{SpaceViolationPolicy.DISABLE}} having
> limit say 2MB
> * Now put rows until space quota is violated and table gets disabled
> * Next, increase space quota with limit say 4MB on the table
> * Now try putting a row into the table
> {code:java}
> private void testSetQuotaThenViolateAndFinallyIncreaseQuota() throws
> Exception {
> SpaceViolationPolicy policy = SpaceViolationPolicy.DISABLE;
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> // Do puts until we violate space policy
> final TableName tn = writeUntilViolationAndVerifyViolation(policy, put);
> // Now, increase limit
> setQuotaLimit(tn, policy, 4L);
> // Put some row now: should not violate as quota limit increased
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*
> We should be able to put data as long as newly set quota limit is not reached
> *Actual*
> We fail to put any new row even after increasing limit
> *Root cause*
> Increasing quota on a violated table triggers the table to be enabled, but
> since the table is already in violation, the system does not allow it to be
> enabled (may be thinking that a user is trying to enable it)
> *Relevant exception trace*
> {noformat}
> 2018-05-31 00:34:27,563 INFO [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> client.HBaseAdmin$14(844): Started enable of
> testSetQuotaAndThenIncreaseQuotaWithDisable0
> 2018-05-31 00:34:27,571 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=42525]
> ipc.CallRunner(142): callId: 11 service: MasterService methodName:
> EnableTable size: 104 connection: 127.0.0.1:38030 deadline: 1527707127568,
> exception=org.apache.hadoop.hbase.security.AccessDeniedException: Enabling
> the table 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to
> a violated space quota.
> 2018-05-31 00:34:27,571 ERROR [regionserver/root1-ThinkPad-T440p:0.Chore.1]
> quotas.RegionServerSpaceQuotaManager(210): Failed to disable space violation
> policy for testSetQuotaAndThenIncreaseQuotaWithDisable0. This table will
> remain in violation.
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Enabling the table
> 'testSetQuotaAndThenIncreaseQuotaWithDisable0' is disallowed due to a
> violated space quota.
> at org.apache.hadoop.hbase.master.HMaster$6.run(HMaster.java:2275)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:131)
> at org.apache.hadoop.hbase.master.HMaster.enableTable(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.enableTable(MasterRpcServices.java:725)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at
[jira] [Commented] (HBASE-22057) Impose upper-bound on size of ZK ops sent in a single multi()
[
https://issues.apache.org/jira/browse/HBASE-22057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803138#comment-16803138
]
Josh Elser commented on HBASE-22057:
Thanks for the review, Peter. Fixed the checkstyle locally and then will commit.
> Impose upper-bound on size of ZK ops sent in a single multi()
> -
>
> Key: HBASE-22057
> URL: https://issues.apache.org/jira/browse/HBASE-22057
> Project: HBase
> Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 1.6.0, 2.2.0
>
> Attachments: HBASE-22057.001.patch, HBASE-22057.002.patch,
> HBASE-22057.003.patch, HBASE-22057.004.patch
>
>
> In {{ZKUtil#multiOrSequential}}, we accept a list of {{ZKUtilOp}}'s to pass
> down to the {{ZooKeeper#multi(Iterable)}} method.
> One problem with this approach is that we may generate a large list of ZNodes
> to mutate in one batch which exceeds the allowable client package length,
> specified by {{jute.maxbuffer}}.
> This problem can manifest when we have a large number of WALs to replicate,
> queued in ZooKeeper, from a disabled peer. When that peer is dropped, the RS
> would submit deletes of those queued WALs. The RS will see ConnectionLoss for
> the resulting {{multi()}} calls it tries to make, because we are sending too
> large of a client message (because we're trying to delete too many WALs at
> once). The result (at least in branch-1 ish versions) is that the RS aborts
> after exceeding the ZK retries (as this operation will never succeed).
> A simple fix would be to impose a maximum number of Ops to run in a single
> batch inside ZKUtil, and split apart the caller-submitted batch into smaller
> chunks. Before we make such a change, I do need to make sure that we don't
> have any expectations on atomicity of the operations. I'm not sure what ZK
> provides here -- for the above example, splitting up batches of deletes is
> not an issue, but there could be issues with batches of creates where we only
> apply some.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22087) Update LICENSE/shading for the dependencies from the latest Hadoop trunk
[
https://issues.apache.org/jira/browse/HBASE-22087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803137#comment-16803137
]
Sean Busbey commented on HBASE-22087:
-
{quote}
bq. This is concerning, as relocating and including javax.activation is
probably not correct behavior.
This was explicitly included in Hadoop because of JDK9 support.
I excluded javax.activation from HBase. This should be consistent with the
handling of javax.annotation.
{quote}
I get that. I'm just saying that it's not clear to me that it should be done
this broadly. e.g. JVM for JDK8 already provides the {{javax.annotation}}
classes. Should these versions of Hadoop not be used with JDK8? JDK9 and JDK10
will provide the {{javax.annotation}} module so long as you start with
{{--add-modules javax.annotation}}. The only place you *need* to have it
brought with you is JDK11+.
> Update LICENSE/shading for the dependencies from the latest Hadoop trunk
>
>
> Key: HBASE-22087
> URL: https://issues.apache.org/jira/browse/HBASE-22087
> Project: HBase
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HBASE-22087.master.001.patch, depcheck_hadoop33.log
>
>
> The following list of dependencies were added in Hadoop trunk (3.3.0) and
> HBase does not compile successfully:
> YARN-8778 added jline 3.9.0
> HADOOP-15775 added javax.activation
> HADOOP-15531 added org.apache.common.text (commons-text)
> HADOOP-15764 added dnsjava (org.xbill)
> Some of these are needed to support JDK9/10/11 in Hadoop.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22052) pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove redunant version specifications
[ https://issues.apache.org/jira/browse/HBASE-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803130#comment-16803130 ] Sean Busbey commented on HBASE-22052: - Okay, I finished working through using the HBase 2.1.4 release with a bunch of Hadoop versions we claim as supported in the ref guide for 2.1.z and updated the release note with what is broken and what it takes to make things work. > pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove > redunant version specifications > --- > > Key: HBASE-22052 > URL: https://issues.apache.org/jira/browse/HBASE-22052 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.0.5, 2.3.0, 2.1.4 > > Attachments: HBASE-22052-branch-2.1.003.addendum2.txt, > HBASE-22052.2.1.003.addendum1.txt, HBASE-22052.branch-2.0.001.patch, > HBASE-22052.branch-2.0.002.patch, HBASE-22052.branch-2.0.002.patch, > HBASE-22052.branch-2.0.003.patch, HBASE-22052.branch-2.0.004.patch, > HBASE-22052.branch-2.1.001.patch, HBASE-22052.branch-2.1.002.patch, > HBASE-22052.branch-2.1.003.patch, HBASE-22052.branch-2.2.001.patch, > HBASE-22052.branch-2.2.002.patch, nothing_change.patch, nothing_change.patch > > > Working on HBASE-22029, where we fail compile of hbase-it module four hours > into an RC build, it looks like transitive include of jersey-core is the > culprit. hadoop3 profile does a bunch of jersey-core exclusions. This issue > is about having hadoop2 profile and hadoop3 profiles match around jersey-core > treatment. Some miscellaneous cleanups are also done. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22052) pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove redunant version specifications
[
https://issues.apache.org/jira/browse/HBASE-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-22052:
Release Note:
Fixed awkward dependency issue that prevented site building.
note specific to HBase 2.1.4
HBase 2.1.4 shipped with an early version of this fix that incorrectly altered
the libraries included in our binary assembly for using Apache Hadoop 2.7 (the
current build default Hadoop version for 2.1.z). For folks running out of the
box against a Hadoop 2.7 cluster (or folks who skip the installation step of
[replacing the bundled Hadoop
libraries](http://hbase.apache.org/book.html#hadoop)) this will result in a
failure at Region Server startup due to a missing class definition. e.g.:
```
2019-03-27 09:02:05,779 ERROR [main] regionserver.HRegionServer: Failed
construction RegionServer
java.lang.NoClassDefFoundError: org/apache/htrace/SamplerBuilder
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:644)
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:628)
at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:93)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2701)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2683)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:372)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:171)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:356)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
at
org.apache.hadoop.hbase.util.CommonFSUtils.getRootDir(CommonFSUtils.java:362)
at
org.apache.hadoop.hbase.util.CommonFSUtils.isValidWALRootDir(CommonFSUtils.java:411)
at
org.apache.hadoop.hbase.util.CommonFSUtils.getWALRootDir(CommonFSUtils.java:387)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.initializeFileSystem(HRegionServer.java:704)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:613)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:3029)
at
org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:63)
at
org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:149)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3047)
Caused by: java.lang.ClassNotFoundException: org.apache.htrace.SamplerBuilder
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 26 more
```
Workaround via any _one_ of the following:
* If you are running against a Hadoop cluster that is 2.8+, ensure you replace
the Hadoop libaries in the default binary assembly with those for your version.
* If you are running against a Hadoop cluster that is 2.8+, build the binary
assembly from the source release while specifying your Hadoop version.
* If you are running against a Hadoop cluster that is a supported 2.7 release,
ensure the `hadoop` executable is in the `PATH` seen at Region Server startup
and that you are not using the `HBASE_DISABLE_HADOOP_CLASSPATH_LOOKUP` bypass.
* For any supported Hadoop version, manually make the Apache HTrace artifact
`htrace-core-3.1.0-incubating.jar` available to all Region Servers via the
HBASE_CLASSPATH environment variable.
* For any supported Hadoop version, manually make the Apache HTrace artifact
`htrace-core-3.1.0-incubating.jar` available to all Region Servers by copying
it into the directory `${HBASE_HOME}/lib/client-facing-thirdparty/`.
was:
Fixed awkward dependency issue that prevented site building.
*note*
The HBase Shaded Client is likely 'busted' in 2.1.4. HBase 2.1.4 shipped with
the first version of this patch. The patch was subsequently reopened when it
was found that it failed a special IT test that was using the shaded hbase
mapreduce client. The IT test was failing because an old htrace
[jira] [Commented] (HBASE-22119) Provide functions in RSGroupInfo to check if a group is the default group
[
https://issues.apache.org/jira/browse/HBASE-22119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803118#comment-16803118
]
Hadoop QA commented on HBASE-22119:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
5s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
21s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 0s{color}
| {color:red} hbase-rsgroup generated 22 new + 86 unchanged - 22 fixed = 108
total (was 108) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
41s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 44s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
49s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 5m
22s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 59m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-22119 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12963904/HBASE-22119.master.000.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 9d814d4c7f53 4.4.0-131-generic #157-Ubuntu SMP Thu Jul 12
15:51:36 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / e46dc61569 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe;
[jira] [Commented] (HBASE-22054) Space Quota: Compaction is not working for super user in case of NO_WRITES_COMPACTIONS
[
https://issues.apache.org/jira/browse/HBASE-22054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803108#comment-16803108
]
Josh Elser commented on HBASE-22054:
{quote}I am seeing cases where RpcServer.getCurrentUser might return null. I am
not sure why. IllegalArgumentException makes sense
{quote}
One case is when we are calling the method from within the RegionServer. For
example, if the RegionServer itself initiates a compaction, there would be no
RPC user.
{quote}Most of the reasons being uninitialized superusers/null user. Digging
into it.
{quote}
Fantastic. Could be that making the change once in HBaseTestingUtility would be
better? Hard to say when we have mocks in play :)
> Space Quota: Compaction is not working for super user in case of
> NO_WRITES_COMPACTIONS
> --
>
> Key: HBASE-22054
> URL: https://issues.apache.org/jira/browse/HBASE-22054
> Project: HBase
> Issue Type: Bug
>Reporter: Ajeet Rai
>Assignee: Sakthi
>Priority: Minor
> Attachments: hbase-22054.master.001.patch,
> hbase-22054.master.002.patch, hbase-22054.master.003.patch
>
>
> Space Quota: Compaction is not working for super user. Compaction command is
> issued successfully at client but actually compaction is not happening.
> In debug log below message is printed:
> as an active space quota violation policy disallows compaction.
> Reference:
>
> [https://lists.apache.org/thread.html/d09aa7abaacf1f0be9d59fa9260515ddc0c17ac0aba9cc0f2ac569bf@%3Cuser.hbase.apache.org%3E]
> Actually in requestCompactionInternal method of CompactSplit class ,there is
> no check for super user and compcations are disallowed
> {noformat}
> RegionServerSpaceQuotaManager spaceQuotaManager =
> this.server.getRegionServerSpaceQuotaManager();
> if (spaceQuotaManager != null &&
>
> spaceQuotaManager.areCompactionsDisabled(region.getTableDescriptor().getTableName()))
> {
> String reason = "Ignoring compaction request for " + region +
> " as an active space quota violation " + " policy disallows
> compactions.";
> tracker.notExecuted(store, reason);
> completeTracker.completed(store);
> LOG.debug(reason);
> return;
> }
> {noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21455) Update filesystem-space quota fail if there is a space quota for non-existing namespace
[
https://issues.apache.org/jira/browse/HBASE-21455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803098#comment-16803098
]
Josh Elser commented on HBASE-21455:
{code:java}
+//Set a space quota on non-exist namespace
+
admin.setQuota(QuotaSettingsFactory.limitNamespaceSpace(testName.getMethodName(),
+SpaceQuotaHelperForTests.ONE_GIGABYTE,
SpaceViolationPolicy.NO_INSERTS));
{code}
Open-ended question: are we sure we want to allow configuring quotas on
entities that don't exist? A quick glacne doesn't indicate that we require the
entity we're setting a quota on (ns, user, or table) to exist prior to
configuring this quota. I wonder if this issue exists beyond just the
namespace. You look into that at all [~wenbang]?
{code:java}
+ public static TableName[] listTableNamesByNamespace(Connection conn, String
namespace)
+ throws IOException {
+TableName[] tablesInNS = null;
+try {
+ tablesInNS = conn.getAdmin().listTableNamesByNamespace(namespace);
+} catch (NamespaceNotFoundException e) {
+ LOG.warn("NameSpace " + namespace + " doesn't exist，skip it");
+}
+return tablesInNS;
+ }
{code}
Could we change this method to return {{Optional}} instead of
relying on this null-check? I think that would clean up the current patch and
help prevent future bugs around this.
> Update filesystem-space quota fail if there is a space quota for non-existing
> namespace
> ---
>
> Key: HBASE-21455
> URL: https://issues.apache.org/jira/browse/HBASE-21455
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.2
>Reporter: wenbang
>Assignee: wenbang
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-21455.master.001.patch,
> HBASE-21455.master.002.patch, HBASE-21455.master.003.patch,
> HBASE-21455.master.004.patch
>
>
> QuotaObserverChore#fetchAllTablesWithQuotasDefined may fail and throw a
> NamespaceNotFoundException because of namespace does not exist
> {code:java}
> // Collect all of the tables in the namespace
> TableName[] tablesInNS =
> conn.getAdmin().listTableNamesByNamespace(namespace);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22116) HttpDoAsClient to support keytab and principal in command line argument.
[ https://issues.apache.org/jira/browse/HBASE-22116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803095#comment-16803095 ] Subrat Mishra commented on HBASE-22116: --- Thanks [~esteban] for the review. The javac errors are not related to this change. Shall I still fix those? > HttpDoAsClient to support keytab and principal in command line argument. > > > Key: HBASE-22116 > URL: https://issues.apache.org/jira/browse/HBASE-22116 > Project: HBase > Issue Type: Improvement >Reporter: Subrat Mishra >Assignee: Subrat Mishra >Priority: Major > Attachments: HBASE-22116.master.001.patch, > HBASE-22116.master.002.patch > > > Currently, HttpDoAsClient relies only on kinit. It's good to add support for > keytab and principal in command line argument. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22087) Update LICENSE/shading for the dependencies from the latest Hadoop trunk
[
https://issues.apache.org/jira/browse/HBASE-22087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803068#comment-16803068
]
Hadoop QA commented on HBASE-22087:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
29s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
16s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 21s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
8s{color} | {color:green} hbase-resource-bundle in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
23s{color} | {color:green} hbase-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
12s{color} | {color:green} hbase-shaded-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-22087 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12963902/HBASE-22087.master.001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit xml shadedjars
hadoopcheck compile |
| uname | Linux 252df6f31cb1 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / e46dc61569 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/16565/testReport/ |
| Max. process+thread count | 97 (vs. ulimit of 1) |
| modules | C: hbase-resource-bundle hbase-shaded
hbase-shaded/hbase-shaded-client U: . |
| Console output |
[jira] [Commented] (HBASE-22116) HttpDoAsClient to support keytab and principal in command line argument.
[ https://issues.apache.org/jira/browse/HBASE-22116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803034#comment-16803034 ] Esteban Gutierrez commented on HBASE-22116: --- Thanks for this improvement [~subrat.mishra]. Could you please address the errors reported by Hadoop QA by javac? > HttpDoAsClient to support keytab and principal in command line argument. > > > Key: HBASE-22116 > URL: https://issues.apache.org/jira/browse/HBASE-22116 > Project: HBase > Issue Type: Improvement >Reporter: Subrat Mishra >Assignee: Subrat Mishra >Priority: Major > Attachments: HBASE-22116.master.001.patch, > HBASE-22116.master.002.patch > > > Currently, HttpDoAsClient relies only on kinit. It's good to add support for > keytab and principal in command line argument. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22094) Throw TableNotFoundException if table not exists in AsyncAdmin.compact
[
https://issues.apache.org/jira/browse/HBASE-22094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803023#comment-16803023
]
Hudson commented on HBASE-22094:
Results for branch branch-2.1
[build #998 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/998/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/998//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/998//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/998//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Throw TableNotFoundException if table not exists in AsyncAdmin.compact
> --
>
> Key: HBASE-22094
> URL: https://issues.apache.org/jira/browse/HBASE-22094
> Project: HBase
> Issue Type: Sub-task
> Components: Admin
>Reporter: Duo Zhang
>Assignee: Sakthi
>Priority: Major
> Labels: beginner
> Fix For: 3.0.0, 2.2.0, 2.3.0, 2.1.5
>
> Attachments: hbase-22094.master.001.patch,
> hbase-22094.master.002.patch, hbase-22094.master.003.patch,
> hbase-22094.master.004.patch, hbase-22094.master.005.patch
>
>
> Now we will return successfully, which is not the same with the HBaseAdmin.
> For NORMAL compaction, we should throw TableNotFoundException if the
> locations is empty.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-22031) Provide RSGroupInfo with a new constructor using shallow copy
[
https://issues.apache.org/jira/browse/HBASE-22031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803006#comment-16803006
]
Xiang Li edited comment on HBASE-22031 at 3/27/19 4:22 PM:
---
Hi [~xucang] yes, it is a good question (for me) that if it worths here, or
necessary.
The shallow copy trades safety for a better performance: It is safe if we deep
copy (allocate new TreeSets and then make a copy) for tables and servers
inputted. The update on the inputs will not affect RSGroupInfo instance. But as
a trace-off, the allocation of TreeSet is heavy.
By using shallow copy, RSGroupInfo instance points the the input ones, rather
than making a copy within itself. It is fast but the subsequent update on the
inputs will also affect that RSGroupInfo instance.
If the code followed does not change the input servers and tables, it is safe
to do the shallow copy. I will review the code in the context to make sure it
is safe to do that.
was (Author: water):
Hi [~xucang] yes, it is a good question (for me) that if it worths here.
The shallow copy trades safety for a better performance: It is safe if we deep
copy (allocate new TreeSets and then make a copy) for tables and servers
inputted. The update on the inputs will not affect RSGroupInfo instance. But as
a trace-off, the allocation of TreeSet is heavy.
By using shallow copy, RSGroupInfo instance points the the input ones, rather
than making a copy within itself. It is fast but the subsequent update on the
inputs will also affect that RSGroupInfo instance.
If the code followed does not change the input servers and tables, it is safe
to do the shallow copy. I will review the code in the context to make sure it
is safe to do that.
> Provide RSGroupInfo with a new constructor using shallow copy
> -
>
> Key: HBASE-22031
> URL: https://issues.apache.org/jira/browse/HBASE-22031
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22031.master.000.patch,
> HBASE-22031.master.001.patch, HBASE-22031.master.002.patch
>
>
> As for org.apache.hadoop.hbase.rsgroup.RSGroupInfo, the following constructor
> performs deep copies on both servers and tables inputed.
> {code:title=hbase-common/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfo.java.java|borderStyle=solid}
> RSGroupInfo(String name, SortedSet servers, SortedSet
> tables) {
> this.name = name;
> this.servers = (servers == null) ? new TreeSet<>() : new TreeSet<>(servers);
> this.tables = (tables == null) ? new TreeSet<>() : new TreeSet<>(tables);
> }
> {code}
> The constructor of TreeSet is heavy and I think it is better to have a new
> constructor with shallow copy and it could be used at least in
> {code:title=hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManagerImpl.java|borderStyle=solid}
> private synchronized void refresh(boolean forceOnline) throws IOException {
> ...
> groupList.add(new RSGroupInfo(RSGroupInfo.DEFAULT_GROUP,
> getDefaultServers(), orphanTables));
> ...
> {code}
> It is not needed to allocate new TreeSet to deep copy the output of
> getDefaultServers() and orphanTables, both of which are allocated in the near
> context and not updated in the code followed. So it is safe to make a shadow
> copy here.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22031) Provide RSGroupInfo with a new constructor using shallow copy
[
https://issues.apache.org/jira/browse/HBASE-22031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803006#comment-16803006
]
Xiang Li commented on HBASE-22031:
--
Hi [~xucang] yes, it is a good question (for me) that if it worths here.
The shallow copy trades safety for a better performance: It is safe if we deep
copy (allocate new TreeSets and then make a copy) for tables and servers
inputted. The update on the inputs will not affect RSGroupInfo instance. But as
a trace-off, the allocation of TreeSet is heavy.
By using shallow copy, RSGroupInfo instance points the the input ones, rather
than making a copy within itself. It is fast but the subsequent update on the
inputs will also affect that RSGroupInfo instance.
If the code followed does not change the input servers and tables, it is safe
to do the shallow copy. I will review the code in the context to make sure it
is safe to do that.
> Provide RSGroupInfo with a new constructor using shallow copy
> -
>
> Key: HBASE-22031
> URL: https://issues.apache.org/jira/browse/HBASE-22031
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22031.master.000.patch,
> HBASE-22031.master.001.patch, HBASE-22031.master.002.patch
>
>
> As for org.apache.hadoop.hbase.rsgroup.RSGroupInfo, the following constructor
> performs deep copies on both servers and tables inputed.
> {code:title=hbase-common/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfo.java.java|borderStyle=solid}
> RSGroupInfo(String name, SortedSet servers, SortedSet
> tables) {
> this.name = name;
> this.servers = (servers == null) ? new TreeSet<>() : new TreeSet<>(servers);
> this.tables = (tables == null) ? new TreeSet<>() : new TreeSet<>(tables);
> }
> {code}
> The constructor of TreeSet is heavy and I think it is better to have a new
> constructor with shallow copy and it could be used at least in
> {code:title=hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManagerImpl.java|borderStyle=solid}
> private synchronized void refresh(boolean forceOnline) throws IOException {
> ...
> groupList.add(new RSGroupInfo(RSGroupInfo.DEFAULT_GROUP,
> getDefaultServers(), orphanTables));
> ...
> {code}
> It is not needed to allocate new TreeSet to deep copy the output of
> getDefaultServers() and orphanTables, both of which are allocated in the near
> context and not updated in the code followed. So it is safe to make a shadow
> copy here.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-17094) Add a sitemap for hbase.apache.org
[ https://issues.apache.org/jira/browse/HBASE-17094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803001#comment-16803001 ] kevin su commented on HBASE-17094: -- There are some error in my this two patch I am fixing this. If it works, I will tell u. > Add a sitemap for hbase.apache.org > -- > > Key: HBASE-17094 > URL: https://issues.apache.org/jira/browse/HBASE-17094 > Project: HBase > Issue Type: Task >Affects Versions: 3.0.0 >Reporter: stack >Assignee: kevin su >Priority: Major > Labels: beginner > Fix For: 3.0.0 > > Attachments: HBASE-17094.v0.patch, HBASE-17094.v0.patch, > HBASE-17094.v1.patch, HBASE-17094.v2.patch > > > We don't have a sitemap. It was pointed out by [~mbrukman]. Lets add one. > Add tooling under dev-support so it gets autogenerated as part of site build. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22119) Provide functions in RSGroupInfo to check if a group is the default group
[
https://issues.apache.org/jira/browse/HBASE-22119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-22119:
-
Description:
There are several places to check if a group is the "default" group, where the
input could be a String or a RSGroupInfo.
It is better to provide official functions in RSGroupInfo to tell if a group is
the default group, so as to
* Simply the code
* Make it more safe. It is not safe as there is no null check like
{code}
if (!group.getName().equals(RSGroupInfo.DEFAULT_GROUP))
{code}
It is more safe to check like:
{code}
RSGroupInfo.DEFAULT_GROUP.equals(group.getName())
{code}
was:
There are several places to check if a group is the "default" group, where the
input could be a String or a RSGroupInfo.
It is better to provide official functions in RSGroupInfo to tell if a group is
the default group, so as to
* Simply the code
* Make it more safe. It is not safe as there is no null check like
{code}
if (!group.getName().equals(RSGroupInfo.DEFAULT_GROUP))
{code}
It is more safe to check like:
{code}
RSGroupInfo.DEFAULT_GROUP.equals(group.getName)
{code}
> Provide functions in RSGroupInfo to check if a group is the default group
> -
>
> Key: HBASE-22119
> URL: https://issues.apache.org/jira/browse/HBASE-22119
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22119.master.000.patch
>
>
> There are several places to check if a group is the "default" group, where
> the input could be a String or a RSGroupInfo.
> It is better to provide official functions in RSGroupInfo to tell if a group
> is the default group, so as to
> * Simply the code
> * Make it more safe. It is not safe as there is no null check like
> {code}
> if (!group.getName().equals(RSGroupInfo.DEFAULT_GROUP))
> {code}
> It is more safe to check like:
> {code}
> RSGroupInfo.DEFAULT_GROUP.equals(group.getName())
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22119) Provide functions in RSGroupInfo to check if a group is the default group
[
https://issues.apache.org/jira/browse/HBASE-22119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16802974#comment-16802974
]
Xiang Li commented on HBASE-22119:
--
[~xucang] Would you please review this simple improvements at your convenience?
> Provide functions in RSGroupInfo to check if a group is the default group
> -
>
> Key: HBASE-22119
> URL: https://issues.apache.org/jira/browse/HBASE-22119
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22119.master.000.patch
>
>
> There are several places to check if a group is the "default" group, where
> the input could be a String or a RSGroupInfo.
> It is better to provide official functions in RSGroupInfo to tell if a group
> is the default group, so as to
> * Simply the code
> * Make it more safe. It is not safe as there is no null check like
> {code}
> if (!group.getName().equals(RSGroupInfo.DEFAULT_GROUP))
> {code}
> It is more safe to check like:
> {code}
> RSGroupInfo.DEFAULT_GROUP.equals(group.getName())
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22119) Provide functions in RSGroupInfo to check if a group is the default group
[
https://issues.apache.org/jira/browse/HBASE-22119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-22119:
-
Description:
There are several places to check if a group is the "default" group, where the
input could be a String or a RSGroupInfo.
It is better to provide official functions in RSGroupInfo to tell if a group is
the default group, so as to
* Simply the code
* Make it more safe. It is not safe as there is no null check like
{code}
if (!group.getName().equals(RSGroupInfo.DEFAULT_GROUP))
{code}
It is more safe to check like:
{code}
RSGroupInfo.DEFAULT_GROUP.equals(group.getName)
{code}
was:
There are several places to check if a group is the "default" group, where the
input could be a String or a RSGroupInfo.
It is better to provide official functions in RSGroupInfo to tell if a group is
the default group, so as to
* Simply the code
* Make it safe. There is no null check like
{code}
if (!group.getName().equals(RSGroupInfo.DEFAULT_GROUP))
{code}
> Provide functions in RSGroupInfo to check if a group is the default group
> -
>
> Key: HBASE-22119
> URL: https://issues.apache.org/jira/browse/HBASE-22119
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22119.master.000.patch
>
>
> There are several places to check if a group is the "default" group, where
> the input could be a String or a RSGroupInfo.
> It is better to provide official functions in RSGroupInfo to tell if a group
> is the default group, so as to
> * Simply the code
> * Make it more safe. It is not safe as there is no null check like
> {code}
> if (!group.getName().equals(RSGroupInfo.DEFAULT_GROUP))
> {code}
> It is more safe to check like:
> {code}
> RSGroupInfo.DEFAULT_GROUP.equals(group.getName)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22119) Provide functions in RSGroupInfo to check if a group is the default group
[
https://issues.apache.org/jira/browse/HBASE-22119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-22119:
-
Status: Patch Available (was: Open)
> Provide functions in RSGroupInfo to check if a group is the default group
> -
>
> Key: HBASE-22119
> URL: https://issues.apache.org/jira/browse/HBASE-22119
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22119.master.000.patch
>
>
> There are several places to check if a group is the "default" group, where
> the input could be a String or a RSGroupInfo.
> It is better to provide official functions in RSGroupInfo to tell if a group
> is the default group, so as to
> * Simply the code
> * Make it safe. There is no null check like
> {code}
> if (!group.getName().equals(RSGroupInfo.DEFAULT_GROUP))
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22119) Provide functions in RSGroupInfo to check if a group is the default group
[
https://issues.apache.org/jira/browse/HBASE-22119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-22119:
-
Attachment: HBASE-22119.master.000.patch
> Provide functions in RSGroupInfo to check if a group is the default group
> -
>
> Key: HBASE-22119
> URL: https://issues.apache.org/jira/browse/HBASE-22119
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22119.master.000.patch
>
>
> There are several places to check if a group is the "default" group, where
> the input could be a String or a RSGroupInfo.
> It is better to provide official functions in RSGroupInfo to tell if a group
> is the default group, so as to
> * Simply the code
> * Make it safe. There is no null check like
> {code}
> if (!group.getName().equals(RSGroupInfo.DEFAULT_GROUP))
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22119) Provide functions in RSGroupInfo to check if a group is the default group
[
https://issues.apache.org/jira/browse/HBASE-22119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-22119:
-
Description:
There are several places to check if a group is the "default" group, where the
input could be a String or a RSGroupInfo.
It is better to provide official functions in RSGroupInfo to tell if a group is
the default group, so as to
* Simply the code
* Make it safe. There is no null check like
{code}
if (!group.getName().equals(RSGroupInfo.DEFAULT_GROUP))
{code}
was:
There are several places to check if a group is the default group, where the
input could be a string or a RSGroupInfo.
It is better to provide official functions in RSGroupInfo to tell if a group is
the default group, so as to
* Simply the code
* Make it safe. There is no null check like
{code}
if (!group.getName().equals(RSGroupInfo.DEFAULT_GROUP))
{code}
> Provide functions in RSGroupInfo to check if a group is the default group
> -
>
> Key: HBASE-22119
> URL: https://issues.apache.org/jira/browse/HBASE-22119
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> There are several places to check if a group is the "default" group, where
> the input could be a String or a RSGroupInfo.
> It is better to provide official functions in RSGroupInfo to tell if a group
> is the default group, so as to
> * Simply the code
> * Make it safe. There is no null check like
> {code}
> if (!group.getName().equals(RSGroupInfo.DEFAULT_GROUP))
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
