[jira] [Commented] (HBASE-18141) Regionserver fails to shutdown when abort triggered in RegionScannerImpl during RPC call
[ https://issues.apache.org/jira/browse/HBASE-18141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043945#comment-16043945 ] Hudson commented on HBASE-18141: SUCCESS: Integrated in Jenkins build HBase-1.3-JDK8 #193 (See [https://builds.apache.org/job/HBase-1.3-JDK8/193/]) HBASE-18141 Regionserver fails to shutdown when abort triggered during (garyh: rev 92f3db6cb99da542059560c04746c192930e8646) * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerAbort.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > Regionserver fails to shutdown when abort triggered in RegionScannerImpl > during RPC call > > > Key: HBASE-18141 > URL: https://issues.apache.org/jira/browse/HBASE-18141 > Project: HBase > Issue Type: Bug > Components: regionserver, security >Affects Versions: 1.3.1 >Reporter: Gary Helmling >Assignee: Gary Helmling >Priority: Critical > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.2.7 > > Attachments: HBASE-18141.001.patch, HBASE-18141.branch-1.3.001.patch, > HBASE-18141.branch-1.3.002.patch > > > When an abort is triggered within the RPC call path by > HRegion.RegionScannerImpl, AccessController is incorrectly apply the RPC > caller identity in the RegionServerObserver.preStopRegionServer() hook. This > leaves the regionserver in a non-responsive state, where its regions are not > reassigned and it returns exceptions for all requests. > When an abort is triggered on the server side, we should not allow a > coprocessor to reject the abort at all. > Here is a sample stack trace: > {noformat} > 17/05/25 06:10:29 FATAL regionserver.HRegionServer: RegionServer abort: > loaded coprocessors are: > [org.apache.hadoop.hbase.security.access.AccessController, > org.apache.hadoop.hbase.security.token.TokenProvider] > 17/05/25 06:10:29 WARN regionserver.HRegionServer: The region server did not > stop > org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient > permissions for user 'rpcuser' (global, action=ADMIN) > at > org.apache.hadoop.hbase.security.access.AccessController.requireGlobalPermission(AccessController.java:548) > at > org.apache.hadoop.hbase.security.access.AccessController.requirePermission(AccessController.java:522) > at > org.apache.hadoop.hbase.security.access.AccessController.preStopRegionServer(AccessController.java:2501) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$1.call(RegionServerCoprocessorHost.java:86) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.execShutdown(RegionServerCoprocessorHost.java:300) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.preStop(RegionServerCoprocessorHost.java:82) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1905) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2118) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2125) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.abortRegionServer(HRegion.java:6326) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.handleFileNotFound(HRegion.java:6319) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5941) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6084) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5858) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2649) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34950) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2320) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168) > {noformat} > I haven't yet evaluated which other release branches this might apply to. > I have a patch currently in progress, which I will post as soon as I complete > a test case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18141) Regionserver fails to shutdown when abort triggered in RegionScannerImpl during RPC call
[ https://issues.apache.org/jira/browse/HBASE-18141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043943#comment-16043943 ] Hudson commented on HBASE-18141: SUCCESS: Integrated in Jenkins build HBase-1.3-JDK7 #179 (See [https://builds.apache.org/job/HBase-1.3-JDK7/179/]) HBASE-18141 Regionserver fails to shutdown when abort triggered during (garyh: rev 92f3db6cb99da542059560c04746c192930e8646) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerAbort.java > Regionserver fails to shutdown when abort triggered in RegionScannerImpl > during RPC call > > > Key: HBASE-18141 > URL: https://issues.apache.org/jira/browse/HBASE-18141 > Project: HBase > Issue Type: Bug > Components: regionserver, security >Affects Versions: 1.3.1 >Reporter: Gary Helmling >Assignee: Gary Helmling >Priority: Critical > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.2.7 > > Attachments: HBASE-18141.001.patch, HBASE-18141.branch-1.3.001.patch, > HBASE-18141.branch-1.3.002.patch > > > When an abort is triggered within the RPC call path by > HRegion.RegionScannerImpl, AccessController is incorrectly apply the RPC > caller identity in the RegionServerObserver.preStopRegionServer() hook. This > leaves the regionserver in a non-responsive state, where its regions are not > reassigned and it returns exceptions for all requests. > When an abort is triggered on the server side, we should not allow a > coprocessor to reject the abort at all. > Here is a sample stack trace: > {noformat} > 17/05/25 06:10:29 FATAL regionserver.HRegionServer: RegionServer abort: > loaded coprocessors are: > [org.apache.hadoop.hbase.security.access.AccessController, > org.apache.hadoop.hbase.security.token.TokenProvider] > 17/05/25 06:10:29 WARN regionserver.HRegionServer: The region server did not > stop > org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient > permissions for user 'rpcuser' (global, action=ADMIN) > at > org.apache.hadoop.hbase.security.access.AccessController.requireGlobalPermission(AccessController.java:548) > at > org.apache.hadoop.hbase.security.access.AccessController.requirePermission(AccessController.java:522) > at > org.apache.hadoop.hbase.security.access.AccessController.preStopRegionServer(AccessController.java:2501) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$1.call(RegionServerCoprocessorHost.java:86) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.execShutdown(RegionServerCoprocessorHost.java:300) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.preStop(RegionServerCoprocessorHost.java:82) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1905) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2118) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2125) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.abortRegionServer(HRegion.java:6326) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.handleFileNotFound(HRegion.java:6319) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5941) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6084) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5858) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2649) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34950) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2320) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168) > {noformat} > I haven't yet evaluated which other release branches this might apply to. > I have a patch currently in progress, which I will post as soon as I complete > a test case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18141) Regionserver fails to shutdown when abort triggered in RegionScannerImpl during RPC call
[ https://issues.apache.org/jira/browse/HBASE-18141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043922#comment-16043922 ] Hudson commented on HBASE-18141: SUCCESS: Integrated in Jenkins build HBase-1.2-JDK7 #150 (See [https://builds.apache.org/job/HBase-1.2-JDK7/150/]) HBASE-18141 Regionserver fails to shutdown when abort triggered during (garyh: rev a2617b00c0fd330d5b779c4aedc65e9dcc08cb5c) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerAbort.java > Regionserver fails to shutdown when abort triggered in RegionScannerImpl > during RPC call > > > Key: HBASE-18141 > URL: https://issues.apache.org/jira/browse/HBASE-18141 > Project: HBase > Issue Type: Bug > Components: regionserver, security >Affects Versions: 1.3.1 >Reporter: Gary Helmling >Assignee: Gary Helmling >Priority: Critical > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.2.7 > > Attachments: HBASE-18141.001.patch, HBASE-18141.branch-1.3.001.patch, > HBASE-18141.branch-1.3.002.patch > > > When an abort is triggered within the RPC call path by > HRegion.RegionScannerImpl, AccessController is incorrectly apply the RPC > caller identity in the RegionServerObserver.preStopRegionServer() hook. This > leaves the regionserver in a non-responsive state, where its regions are not > reassigned and it returns exceptions for all requests. > When an abort is triggered on the server side, we should not allow a > coprocessor to reject the abort at all. > Here is a sample stack trace: > {noformat} > 17/05/25 06:10:29 FATAL regionserver.HRegionServer: RegionServer abort: > loaded coprocessors are: > [org.apache.hadoop.hbase.security.access.AccessController, > org.apache.hadoop.hbase.security.token.TokenProvider] > 17/05/25 06:10:29 WARN regionserver.HRegionServer: The region server did not > stop > org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient > permissions for user 'rpcuser' (global, action=ADMIN) > at > org.apache.hadoop.hbase.security.access.AccessController.requireGlobalPermission(AccessController.java:548) > at > org.apache.hadoop.hbase.security.access.AccessController.requirePermission(AccessController.java:522) > at > org.apache.hadoop.hbase.security.access.AccessController.preStopRegionServer(AccessController.java:2501) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$1.call(RegionServerCoprocessorHost.java:86) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.execShutdown(RegionServerCoprocessorHost.java:300) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.preStop(RegionServerCoprocessorHost.java:82) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1905) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2118) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2125) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.abortRegionServer(HRegion.java:6326) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.handleFileNotFound(HRegion.java:6319) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5941) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6084) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5858) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2649) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34950) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2320) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168) > {noformat} > I haven't yet evaluated which other release branches this might apply to. > I have a patch currently in progress, which I will post as soon as I complete > a test case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18141) Regionserver fails to shutdown when abort triggered in RegionScannerImpl during RPC call
[ https://issues.apache.org/jira/browse/HBASE-18141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043899#comment-16043899 ] Hudson commented on HBASE-18141: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #3161 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3161/]) HBASE-18141 Regionserver fails to shutdown when abort triggered during (garyh: rev a558d6c57a02219cb429b311bad13199042ad720) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerAbort.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerCoprocessorHost.java > Regionserver fails to shutdown when abort triggered in RegionScannerImpl > during RPC call > > > Key: HBASE-18141 > URL: https://issues.apache.org/jira/browse/HBASE-18141 > Project: HBase > Issue Type: Bug > Components: regionserver, security >Affects Versions: 1.3.1 >Reporter: Gary Helmling >Assignee: Gary Helmling >Priority: Critical > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.2.7 > > Attachments: HBASE-18141.001.patch, HBASE-18141.branch-1.3.001.patch, > HBASE-18141.branch-1.3.002.patch > > > When an abort is triggered within the RPC call path by > HRegion.RegionScannerImpl, AccessController is incorrectly apply the RPC > caller identity in the RegionServerObserver.preStopRegionServer() hook. This > leaves the regionserver in a non-responsive state, where its regions are not > reassigned and it returns exceptions for all requests. > When an abort is triggered on the server side, we should not allow a > coprocessor to reject the abort at all. > Here is a sample stack trace: > {noformat} > 17/05/25 06:10:29 FATAL regionserver.HRegionServer: RegionServer abort: > loaded coprocessors are: > [org.apache.hadoop.hbase.security.access.AccessController, > org.apache.hadoop.hbase.security.token.TokenProvider] > 17/05/25 06:10:29 WARN regionserver.HRegionServer: The region server did not > stop > org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient > permissions for user 'rpcuser' (global, action=ADMIN) > at > org.apache.hadoop.hbase.security.access.AccessController.requireGlobalPermission(AccessController.java:548) > at > org.apache.hadoop.hbase.security.access.AccessController.requirePermission(AccessController.java:522) > at > org.apache.hadoop.hbase.security.access.AccessController.preStopRegionServer(AccessController.java:2501) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$1.call(RegionServerCoprocessorHost.java:86) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.execShutdown(RegionServerCoprocessorHost.java:300) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.preStop(RegionServerCoprocessorHost.java:82) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1905) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2118) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2125) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.abortRegionServer(HRegion.java:6326) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.handleFileNotFound(HRegion.java:6319) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5941) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6084) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5858) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2649) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34950) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2320) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168) > {noformat} > I haven't yet evaluated which other release branches this might apply to. > I have a patch currently in progress, which I will post as soon as I complete > a test case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18141) Regionserver fails to shutdown when abort triggered in RegionScannerImpl during RPC call
[ https://issues.apache.org/jira/browse/HBASE-18141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043898#comment-16043898 ] Hudson commented on HBASE-18141: SUCCESS: Integrated in Jenkins build HBase-1.2-JDK8 #146 (See [https://builds.apache.org/job/HBase-1.2-JDK8/146/]) HBASE-18141 Regionserver fails to shutdown when abort triggered during (garyh: rev a2617b00c0fd330d5b779c4aedc65e9dcc08cb5c) * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerAbort.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > Regionserver fails to shutdown when abort triggered in RegionScannerImpl > during RPC call > > > Key: HBASE-18141 > URL: https://issues.apache.org/jira/browse/HBASE-18141 > Project: HBase > Issue Type: Bug > Components: regionserver, security >Affects Versions: 1.3.1 >Reporter: Gary Helmling >Assignee: Gary Helmling >Priority: Critical > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.2.7 > > Attachments: HBASE-18141.001.patch, HBASE-18141.branch-1.3.001.patch, > HBASE-18141.branch-1.3.002.patch > > > When an abort is triggered within the RPC call path by > HRegion.RegionScannerImpl, AccessController is incorrectly apply the RPC > caller identity in the RegionServerObserver.preStopRegionServer() hook. This > leaves the regionserver in a non-responsive state, where its regions are not > reassigned and it returns exceptions for all requests. > When an abort is triggered on the server side, we should not allow a > coprocessor to reject the abort at all. > Here is a sample stack trace: > {noformat} > 17/05/25 06:10:29 FATAL regionserver.HRegionServer: RegionServer abort: > loaded coprocessors are: > [org.apache.hadoop.hbase.security.access.AccessController, > org.apache.hadoop.hbase.security.token.TokenProvider] > 17/05/25 06:10:29 WARN regionserver.HRegionServer: The region server did not > stop > org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient > permissions for user 'rpcuser' (global, action=ADMIN) > at > org.apache.hadoop.hbase.security.access.AccessController.requireGlobalPermission(AccessController.java:548) > at > org.apache.hadoop.hbase.security.access.AccessController.requirePermission(AccessController.java:522) > at > org.apache.hadoop.hbase.security.access.AccessController.preStopRegionServer(AccessController.java:2501) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$1.call(RegionServerCoprocessorHost.java:86) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.execShutdown(RegionServerCoprocessorHost.java:300) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.preStop(RegionServerCoprocessorHost.java:82) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1905) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2118) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2125) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.abortRegionServer(HRegion.java:6326) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.handleFileNotFound(HRegion.java:6319) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5941) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6084) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5858) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2649) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34950) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2320) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168) > {noformat} > I haven't yet evaluated which other release branches this might apply to. > I have a patch currently in progress, which I will post as soon as I complete > a test case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18141) Regionserver fails to shutdown when abort triggered in RegionScannerImpl during RPC call
[ https://issues.apache.org/jira/browse/HBASE-18141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043890#comment-16043890 ] Hudson commented on HBASE-18141: SUCCESS: Integrated in Jenkins build HBase-1.4 #768 (See [https://builds.apache.org/job/HBase-1.4/768/]) HBASE-18141 Regionserver fails to shutdown when abort triggered during (garyh: rev 6c4980161b736878d8c1f83bd9b65a591d5bae9c) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerAbort.java > Regionserver fails to shutdown when abort triggered in RegionScannerImpl > during RPC call > > > Key: HBASE-18141 > URL: https://issues.apache.org/jira/browse/HBASE-18141 > Project: HBase > Issue Type: Bug > Components: regionserver, security >Affects Versions: 1.3.1 >Reporter: Gary Helmling >Assignee: Gary Helmling >Priority: Critical > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.2.7 > > Attachments: HBASE-18141.001.patch, HBASE-18141.branch-1.3.001.patch, > HBASE-18141.branch-1.3.002.patch > > > When an abort is triggered within the RPC call path by > HRegion.RegionScannerImpl, AccessController is incorrectly apply the RPC > caller identity in the RegionServerObserver.preStopRegionServer() hook. This > leaves the regionserver in a non-responsive state, where its regions are not > reassigned and it returns exceptions for all requests. > When an abort is triggered on the server side, we should not allow a > coprocessor to reject the abort at all. > Here is a sample stack trace: > {noformat} > 17/05/25 06:10:29 FATAL regionserver.HRegionServer: RegionServer abort: > loaded coprocessors are: > [org.apache.hadoop.hbase.security.access.AccessController, > org.apache.hadoop.hbase.security.token.TokenProvider] > 17/05/25 06:10:29 WARN regionserver.HRegionServer: The region server did not > stop > org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient > permissions for user 'rpcuser' (global, action=ADMIN) > at > org.apache.hadoop.hbase.security.access.AccessController.requireGlobalPermission(AccessController.java:548) > at > org.apache.hadoop.hbase.security.access.AccessController.requirePermission(AccessController.java:522) > at > org.apache.hadoop.hbase.security.access.AccessController.preStopRegionServer(AccessController.java:2501) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$1.call(RegionServerCoprocessorHost.java:86) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.execShutdown(RegionServerCoprocessorHost.java:300) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.preStop(RegionServerCoprocessorHost.java:82) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1905) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2118) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2125) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.abortRegionServer(HRegion.java:6326) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.handleFileNotFound(HRegion.java:6319) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5941) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6084) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5858) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2649) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34950) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2320) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168) > {noformat} > I haven't yet evaluated which other release branches this might apply to. > I have a patch currently in progress, which I will post as soon as I complete > a test case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18141) Regionserver fails to shutdown when abort triggered in RegionScannerImpl during RPC call
[ https://issues.apache.org/jira/browse/HBASE-18141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043820#comment-16043820 ] Hudson commented on HBASE-18141: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #883 (See [https://builds.apache.org/job/HBase-1.2-IT/883/]) HBASE-18141 Regionserver fails to shutdown when abort triggered during (garyh: rev a2617b00c0fd330d5b779c4aedc65e9dcc08cb5c) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerAbort.java > Regionserver fails to shutdown when abort triggered in RegionScannerImpl > during RPC call > > > Key: HBASE-18141 > URL: https://issues.apache.org/jira/browse/HBASE-18141 > Project: HBase > Issue Type: Bug > Components: regionserver, security >Affects Versions: 1.3.1 >Reporter: Gary Helmling >Assignee: Gary Helmling >Priority: Critical > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.2.7 > > Attachments: HBASE-18141.001.patch, HBASE-18141.branch-1.3.001.patch, > HBASE-18141.branch-1.3.002.patch > > > When an abort is triggered within the RPC call path by > HRegion.RegionScannerImpl, AccessController is incorrectly apply the RPC > caller identity in the RegionServerObserver.preStopRegionServer() hook. This > leaves the regionserver in a non-responsive state, where its regions are not > reassigned and it returns exceptions for all requests. > When an abort is triggered on the server side, we should not allow a > coprocessor to reject the abort at all. > Here is a sample stack trace: > {noformat} > 17/05/25 06:10:29 FATAL regionserver.HRegionServer: RegionServer abort: > loaded coprocessors are: > [org.apache.hadoop.hbase.security.access.AccessController, > org.apache.hadoop.hbase.security.token.TokenProvider] > 17/05/25 06:10:29 WARN regionserver.HRegionServer: The region server did not > stop > org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient > permissions for user 'rpcuser' (global, action=ADMIN) > at > org.apache.hadoop.hbase.security.access.AccessController.requireGlobalPermission(AccessController.java:548) > at > org.apache.hadoop.hbase.security.access.AccessController.requirePermission(AccessController.java:522) > at > org.apache.hadoop.hbase.security.access.AccessController.preStopRegionServer(AccessController.java:2501) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$1.call(RegionServerCoprocessorHost.java:86) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.execShutdown(RegionServerCoprocessorHost.java:300) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.preStop(RegionServerCoprocessorHost.java:82) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1905) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2118) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2125) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.abortRegionServer(HRegion.java:6326) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.handleFileNotFound(HRegion.java:6319) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5941) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6084) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5858) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2649) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34950) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2320) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168) > {noformat} > I haven't yet evaluated which other release branches this might apply to. > I have a patch currently in progress, which I will post as soon as I complete > a test case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18141) Regionserver fails to shutdown when abort triggered in RegionScannerImpl during RPC call
[ https://issues.apache.org/jira/browse/HBASE-18141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043808#comment-16043808 ] Hudson commented on HBASE-18141: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #61 (See [https://builds.apache.org/job/HBase-1.3-IT/61/]) HBASE-18141 Regionserver fails to shutdown when abort triggered during (garyh: rev 92f3db6cb99da542059560c04746c192930e8646) * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerAbort.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > Regionserver fails to shutdown when abort triggered in RegionScannerImpl > during RPC call > > > Key: HBASE-18141 > URL: https://issues.apache.org/jira/browse/HBASE-18141 > Project: HBase > Issue Type: Bug > Components: regionserver, security >Affects Versions: 1.3.1 >Reporter: Gary Helmling >Assignee: Gary Helmling >Priority: Critical > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.2.7 > > Attachments: HBASE-18141.001.patch, HBASE-18141.branch-1.3.001.patch, > HBASE-18141.branch-1.3.002.patch > > > When an abort is triggered within the RPC call path by > HRegion.RegionScannerImpl, AccessController is incorrectly apply the RPC > caller identity in the RegionServerObserver.preStopRegionServer() hook. This > leaves the regionserver in a non-responsive state, where its regions are not > reassigned and it returns exceptions for all requests. > When an abort is triggered on the server side, we should not allow a > coprocessor to reject the abort at all. > Here is a sample stack trace: > {noformat} > 17/05/25 06:10:29 FATAL regionserver.HRegionServer: RegionServer abort: > loaded coprocessors are: > [org.apache.hadoop.hbase.security.access.AccessController, > org.apache.hadoop.hbase.security.token.TokenProvider] > 17/05/25 06:10:29 WARN regionserver.HRegionServer: The region server did not > stop > org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient > permissions for user 'rpcuser' (global, action=ADMIN) > at > org.apache.hadoop.hbase.security.access.AccessController.requireGlobalPermission(AccessController.java:548) > at > org.apache.hadoop.hbase.security.access.AccessController.requirePermission(AccessController.java:522) > at > org.apache.hadoop.hbase.security.access.AccessController.preStopRegionServer(AccessController.java:2501) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$1.call(RegionServerCoprocessorHost.java:86) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.execShutdown(RegionServerCoprocessorHost.java:300) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.preStop(RegionServerCoprocessorHost.java:82) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1905) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2118) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2125) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.abortRegionServer(HRegion.java:6326) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.handleFileNotFound(HRegion.java:6319) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5941) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6084) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5858) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2649) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34950) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2320) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168) > {noformat} > I haven't yet evaluated which other release branches this might apply to. > I have a patch currently in progress, which I will post as soon as I complete > a test case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18141) Regionserver fails to shutdown when abort triggered in RegionScannerImpl during RPC call
[ https://issues.apache.org/jira/browse/HBASE-18141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043787#comment-16043787 ] Hudson commented on HBASE-18141: FAILURE: Integrated in Jenkins build HBase-2.0 #12 (See [https://builds.apache.org/job/HBase-2.0/12/]) HBASE-18141 Regionserver fails to shutdown when abort triggered during (garyh: rev 17966525e94524f7fc4a72aeeb6804b05020c97c) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerCoprocessorHost.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerAbort.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > Regionserver fails to shutdown when abort triggered in RegionScannerImpl > during RPC call > > > Key: HBASE-18141 > URL: https://issues.apache.org/jira/browse/HBASE-18141 > Project: HBase > Issue Type: Bug > Components: regionserver, security >Affects Versions: 1.3.1 >Reporter: Gary Helmling >Assignee: Gary Helmling >Priority: Critical > Fix For: 1.3.2 > > Attachments: HBASE-18141.001.patch, HBASE-18141.branch-1.3.001.patch, > HBASE-18141.branch-1.3.002.patch > > > When an abort is triggered within the RPC call path by > HRegion.RegionScannerImpl, AccessController is incorrectly apply the RPC > caller identity in the RegionServerObserver.preStopRegionServer() hook. This > leaves the regionserver in a non-responsive state, where its regions are not > reassigned and it returns exceptions for all requests. > When an abort is triggered on the server side, we should not allow a > coprocessor to reject the abort at all. > Here is a sample stack trace: > {noformat} > 17/05/25 06:10:29 FATAL regionserver.HRegionServer: RegionServer abort: > loaded coprocessors are: > [org.apache.hadoop.hbase.security.access.AccessController, > org.apache.hadoop.hbase.security.token.TokenProvider] > 17/05/25 06:10:29 WARN regionserver.HRegionServer: The region server did not > stop > org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient > permissions for user 'rpcuser' (global, action=ADMIN) > at > org.apache.hadoop.hbase.security.access.AccessController.requireGlobalPermission(AccessController.java:548) > at > org.apache.hadoop.hbase.security.access.AccessController.requirePermission(AccessController.java:522) > at > org.apache.hadoop.hbase.security.access.AccessController.preStopRegionServer(AccessController.java:2501) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$1.call(RegionServerCoprocessorHost.java:86) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.execShutdown(RegionServerCoprocessorHost.java:300) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.preStop(RegionServerCoprocessorHost.java:82) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1905) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2118) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2125) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.abortRegionServer(HRegion.java:6326) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.handleFileNotFound(HRegion.java:6319) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5941) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6084) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5858) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2649) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34950) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2320) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168) > {noformat} > I haven't yet evaluated which other release branches this might apply to. > I have a patch currently in progress, which I will post as soon as I complete > a test case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18141) Regionserver fails to shutdown when abort triggered in RegionScannerImpl during RPC call
[ https://issues.apache.org/jira/browse/HBASE-18141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043411#comment-16043411 ] Ted Yu commented on HBASE-18141: lgtm > Regionserver fails to shutdown when abort triggered in RegionScannerImpl > during RPC call > > > Key: HBASE-18141 > URL: https://issues.apache.org/jira/browse/HBASE-18141 > Project: HBase > Issue Type: Bug > Components: regionserver, security >Affects Versions: 1.3.1 >Reporter: Gary Helmling >Assignee: Gary Helmling >Priority: Critical > Fix For: 1.3.2 > > Attachments: HBASE-18141.001.patch, HBASE-18141.branch-1.3.001.patch, > HBASE-18141.branch-1.3.002.patch > > > When an abort is triggered within the RPC call path by > HRegion.RegionScannerImpl, AccessController is incorrectly apply the RPC > caller identity in the RegionServerObserver.preStopRegionServer() hook. This > leaves the regionserver in a non-responsive state, where its regions are not > reassigned and it returns exceptions for all requests. > When an abort is triggered on the server side, we should not allow a > coprocessor to reject the abort at all. > Here is a sample stack trace: > {noformat} > 17/05/25 06:10:29 FATAL regionserver.HRegionServer: RegionServer abort: > loaded coprocessors are: > [org.apache.hadoop.hbase.security.access.AccessController, > org.apache.hadoop.hbase.security.token.TokenProvider] > 17/05/25 06:10:29 WARN regionserver.HRegionServer: The region server did not > stop > org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient > permissions for user 'rpcuser' (global, action=ADMIN) > at > org.apache.hadoop.hbase.security.access.AccessController.requireGlobalPermission(AccessController.java:548) > at > org.apache.hadoop.hbase.security.access.AccessController.requirePermission(AccessController.java:522) > at > org.apache.hadoop.hbase.security.access.AccessController.preStopRegionServer(AccessController.java:2501) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$1.call(RegionServerCoprocessorHost.java:86) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.execShutdown(RegionServerCoprocessorHost.java:300) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.preStop(RegionServerCoprocessorHost.java:82) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1905) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2118) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2125) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.abortRegionServer(HRegion.java:6326) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.handleFileNotFound(HRegion.java:6319) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5941) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6084) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5858) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2649) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34950) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2320) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168) > {noformat} > I haven't yet evaluated which other release branches this might apply to. > I have a patch currently in progress, which I will post as soon as I complete > a test case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18141) Regionserver fails to shutdown when abort triggered in RegionScannerImpl during RPC call
[ https://issues.apache.org/jira/browse/HBASE-18141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043380#comment-16043380 ] Andrew Purtell commented on HBASE-18141: +1 > Regionserver fails to shutdown when abort triggered in RegionScannerImpl > during RPC call > > > Key: HBASE-18141 > URL: https://issues.apache.org/jira/browse/HBASE-18141 > Project: HBase > Issue Type: Bug > Components: regionserver, security >Affects Versions: 1.3.1 >Reporter: Gary Helmling >Assignee: Gary Helmling >Priority: Critical > Fix For: 1.3.2 > > Attachments: HBASE-18141.001.patch, HBASE-18141.branch-1.3.001.patch, > HBASE-18141.branch-1.3.002.patch > > > When an abort is triggered within the RPC call path by > HRegion.RegionScannerImpl, AccessController is incorrectly apply the RPC > caller identity in the RegionServerObserver.preStopRegionServer() hook. This > leaves the regionserver in a non-responsive state, where its regions are not > reassigned and it returns exceptions for all requests. > When an abort is triggered on the server side, we should not allow a > coprocessor to reject the abort at all. > Here is a sample stack trace: > {noformat} > 17/05/25 06:10:29 FATAL regionserver.HRegionServer: RegionServer abort: > loaded coprocessors are: > [org.apache.hadoop.hbase.security.access.AccessController, > org.apache.hadoop.hbase.security.token.TokenProvider] > 17/05/25 06:10:29 WARN regionserver.HRegionServer: The region server did not > stop > org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient > permissions for user 'rpcuser' (global, action=ADMIN) > at > org.apache.hadoop.hbase.security.access.AccessController.requireGlobalPermission(AccessController.java:548) > at > org.apache.hadoop.hbase.security.access.AccessController.requirePermission(AccessController.java:522) > at > org.apache.hadoop.hbase.security.access.AccessController.preStopRegionServer(AccessController.java:2501) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$1.call(RegionServerCoprocessorHost.java:86) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.execShutdown(RegionServerCoprocessorHost.java:300) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.preStop(RegionServerCoprocessorHost.java:82) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1905) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2118) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2125) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.abortRegionServer(HRegion.java:6326) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.handleFileNotFound(HRegion.java:6319) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5941) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6084) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5858) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2649) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34950) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2320) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168) > {noformat} > I haven't yet evaluated which other release branches this might apply to. > I have a patch currently in progress, which I will post as soon as I complete > a test case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18141) Regionserver fails to shutdown when abort triggered in RegionScannerImpl during RPC call
[ https://issues.apache.org/jira/browse/HBASE-18141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16042229#comment-16042229 ] Hadoop QA commented on HBASE-18141: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 59s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 1 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 29s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 20s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 29s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 26s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 16s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 39s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 25s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 25s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 29s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 25s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 57m 37s {color} | {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 20s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 194m 16s {color} | {color:red} hbase-server in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 26s {color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 278m 56s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | Timed out junit tests | org.apache.hadoop.hbase.coprocessor.TestCoprocessorMetrics | | | org.apache.hadoop.hbase.filter.TestFuzzyRowFilterEndToEnd | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12871966/HBASE-18141.001.patch | | JIRA Issue | HBASE-18141 | | Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 39a4b880c53b 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh | | git revision | master / 3f891a6 | | Default Java | 1.8.0_131 | | findbugs | v3.0.0 | | unit | https://builds.apache.org/job/PreCommit-HBASE-Build/7140/artifact/patchprocess/patch-unit-hbase-server.txt | | unit test logs | https://builds.apache.org/job/PreCommit-HBASE-Build/7140/artifact/patchprocess/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/7140/testReport/ | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/7140/console | | Powered by | Apache Yetus 0.3.0 http://yetus.apache.org | This message was automatically generated. > Regionserver fails to shutdown when abort triggered in
[jira] [Commented] (HBASE-18141) Regionserver fails to shutdown when abort triggered in RegionScannerImpl during RPC call
[ https://issues.apache.org/jira/browse/HBASE-18141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16042164#comment-16042164 ] Hadoop QA commented on HBASE-18141: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 22s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 1 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 6s {color} | {color:green} branch-1.3 passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s {color} | {color:green} branch-1.3 passed with JDK v1.8.0_131 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s {color} | {color:green} branch-1.3 passed with JDK v1.7.0_131 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 6s {color} | {color:green} branch-1.3 passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 21s {color} | {color:green} branch-1.3 passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 57s {color} | {color:red} hbase-server in branch-1.3 has 1 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s {color} | {color:green} branch-1.3 passed with JDK v1.8.0_131 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s {color} | {color:green} branch-1.3 passed with JDK v1.7.0_131 {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 44s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s {color} | {color:green} the patch passed with JDK v1.8.0_131 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 31s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s {color} | {color:green} the patch passed with JDK v1.7.0_131 {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 33s {color} | {color:red} hbase-server-jdk1.7.0_131 with JDK v1.7.0_131 generated 1 new + 4 unchanged - 1 fixed = 5 total (was 5) {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 56s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 16s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 16m 42s {color} | {color:green} The patch does not cause any errors with Hadoop 2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} | | {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m 15s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 15s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s {color} | {color:green} the patch passed with JDK v1.8.0_131 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s {color} | {color:green} the patch passed with JDK v1.7.0_131 {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 93m 1s {color} | {color:green} hbase-server in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 31s {color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 130m 29s {color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:9ba21e3 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12871970/HBASE-18141.branch-1.3.002.patch | | JIRA Issue | HBASE-18141 | | Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 8fde312fec2f 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/hbase.sh | | git revision |
[jira] [Commented] (HBASE-18141) Regionserver fails to shutdown when abort triggered in RegionScannerImpl during RPC call
[ https://issues.apache.org/jira/browse/HBASE-18141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16035265#comment-16035265 ] Andrew Purtell commented on HBASE-18141: bq. I think the changes to HRegionServer are still relevant, as I do not think a coprocessor should ever be able to reject an internally triggered abort. +1 Also, we still need the more tightly scoped fix for branch-1.3, right? > Regionserver fails to shutdown when abort triggered in RegionScannerImpl > during RPC call > > > Key: HBASE-18141 > URL: https://issues.apache.org/jira/browse/HBASE-18141 > Project: HBase > Issue Type: Bug > Components: regionserver, security >Affects Versions: 1.3.1 >Reporter: Gary Helmling >Assignee: Gary Helmling >Priority: Critical > Fix For: 1.3.2 > > Attachments: HBASE-18141.branch-1.3.001.patch > > > When an abort is triggered within the RPC call path by > HRegion.RegionScannerImpl, AccessController is incorrectly apply the RPC > caller identity in the RegionServerObserver.preStopRegionServer() hook. This > leaves the regionserver in a non-responsive state, where its regions are not > reassigned and it returns exceptions for all requests. > When an abort is triggered on the server side, we should not allow a > coprocessor to reject the abort at all. > Here is a sample stack trace: > {noformat} > 17/05/25 06:10:29 FATAL regionserver.HRegionServer: RegionServer abort: > loaded coprocessors are: > [org.apache.hadoop.hbase.security.access.AccessController, > org.apache.hadoop.hbase.security.token.TokenProvider] > 17/05/25 06:10:29 WARN regionserver.HRegionServer: The region server did not > stop > org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient > permissions for user 'rpcuser' (global, action=ADMIN) > at > org.apache.hadoop.hbase.security.access.AccessController.requireGlobalPermission(AccessController.java:548) > at > org.apache.hadoop.hbase.security.access.AccessController.requirePermission(AccessController.java:522) > at > org.apache.hadoop.hbase.security.access.AccessController.preStopRegionServer(AccessController.java:2501) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$1.call(RegionServerCoprocessorHost.java:86) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.execShutdown(RegionServerCoprocessorHost.java:300) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.preStop(RegionServerCoprocessorHost.java:82) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1905) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2118) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2125) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.abortRegionServer(HRegion.java:6326) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.handleFileNotFound(HRegion.java:6319) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5941) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6084) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5858) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2649) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34950) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2320) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168) > {noformat} > I haven't yet evaluated which other release branches this might apply to. > I have a patch currently in progress, which I will post as soon as I complete > a test case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18141) Regionserver fails to shutdown when abort triggered in RegionScannerImpl during RPC call
[ https://issues.apache.org/jira/browse/HBASE-18141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16034244#comment-16034244 ] Gary Helmling commented on HBASE-18141: --- The code path where I originally observed this and that TestRegionServerAbort exercises has been changed in master and branch-1 by HBASE-17712, so that the regionserver no longer aborts. I think the changes to HRegionServer are still relevant, as I do not think a coprocessor should ever be able to reject an internally triggered abort. However, I will need to rework the test for master. > Regionserver fails to shutdown when abort triggered in RegionScannerImpl > during RPC call > > > Key: HBASE-18141 > URL: https://issues.apache.org/jira/browse/HBASE-18141 > Project: HBase > Issue Type: Bug > Components: regionserver, security >Affects Versions: 1.3.1 >Reporter: Gary Helmling >Assignee: Gary Helmling >Priority: Critical > Fix For: 1.3.2 > > Attachments: HBASE-18141.branch-1.3.001.patch > > > When an abort is triggered within the RPC call path by > HRegion.RegionScannerImpl, AccessController is incorrectly apply the RPC > caller identity in the RegionServerObserver.preStopRegionServer() hook. This > leaves the regionserver in a non-responsive state, where its regions are not > reassigned and it returns exceptions for all requests. > When an abort is triggered on the server side, we should not allow a > coprocessor to reject the abort at all. > Here is a sample stack trace: > {noformat} > 17/05/25 06:10:29 FATAL regionserver.HRegionServer: RegionServer abort: > loaded coprocessors are: > [org.apache.hadoop.hbase.security.access.AccessController, > org.apache.hadoop.hbase.security.token.TokenProvider] > 17/05/25 06:10:29 WARN regionserver.HRegionServer: The region server did not > stop > org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient > permissions for user 'rpcuser' (global, action=ADMIN) > at > org.apache.hadoop.hbase.security.access.AccessController.requireGlobalPermission(AccessController.java:548) > at > org.apache.hadoop.hbase.security.access.AccessController.requirePermission(AccessController.java:522) > at > org.apache.hadoop.hbase.security.access.AccessController.preStopRegionServer(AccessController.java:2501) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$1.call(RegionServerCoprocessorHost.java:86) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.execShutdown(RegionServerCoprocessorHost.java:300) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.preStop(RegionServerCoprocessorHost.java:82) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1905) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2118) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2125) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.abortRegionServer(HRegion.java:6326) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.handleFileNotFound(HRegion.java:6319) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5941) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6084) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5858) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2649) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34950) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2320) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168) > {noformat} > I haven't yet evaluated which other release branches this might apply to. > I have a patch currently in progress, which I will post as soon as I complete > a test case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18141) Regionserver fails to shutdown when abort triggered in RegionScannerImpl during RPC call
[ https://issues.apache.org/jira/browse/HBASE-18141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16034029#comment-16034029 ] Hadoop QA commented on HBASE-18141: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 33s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 1 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 43s {color} | {color:green} branch-1.3 passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 5s {color} | {color:green} branch-1.3 passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 28s {color} | {color:green} branch-1.3 passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 23s {color} | {color:green} branch-1.3 passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 43s {color} | {color:red} hbase-server in branch-1.3 has 1 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s {color} | {color:green} branch-1.3 passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 30s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 24s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 24s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 44s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 24s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 34m 53s {color} | {color:green} The patch does not cause any errors with Hadoop 2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} | | {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m 21s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 42s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 30m 54s {color} | {color:red} hbase-server in the patch failed. {color} | | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 24s {color} | {color:red} The patch generated 1 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 89m 30s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hbase.TestCheckTestClasses | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:e1e11ad | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12870877/HBASE-18141.branch-1.3.001.patch | | JIRA Issue | HBASE-18141 | | Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 103bea27b1e1 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/hbase.sh | | git revision | branch-1.3 / ae7b631 | | Default Java | 1.8.0_131 | | findbugs | v3.0.0 | | findbugs | https://builds.apache.org/job/PreCommit-HBASE-Build/7043/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html | | unit | https://builds.apache.org/job/PreCommit-HBASE-Build/7043/artifact/patchprocess/patch-unit-hbase-server.txt | | unit test logs | https://builds.apache.org/job/PreCommit-HBASE-Build/7043/artifact/patchprocess/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/7043/testReport/ | | asflicense | https://builds.apache.org/job/PreCommit-HBASE-Build/7043/artifact/patchprocess/patch-asflicense-problems.txt | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/7043/console | | Powered by | Apache Yetus
[jira] [Commented] (HBASE-18141) Regionserver fails to shutdown when abort triggered in RegionScannerImpl during RPC call
[ https://issues.apache.org/jira/browse/HBASE-18141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16033973#comment-16033973 ] Ted Yu commented on HBASE-18141: lgtm Please add test category to TestRegionServerAbort. I assume patch for master branch is coming. > Regionserver fails to shutdown when abort triggered in RegionScannerImpl > during RPC call > > > Key: HBASE-18141 > URL: https://issues.apache.org/jira/browse/HBASE-18141 > Project: HBase > Issue Type: Bug > Components: regionserver, security >Affects Versions: 1.3.1 >Reporter: Gary Helmling >Assignee: Gary Helmling >Priority: Critical > Fix For: 1.3.2 > > Attachments: HBASE-18141.branch-1.3.001.patch > > > When an abort is triggered within the RPC call path by > HRegion.RegionScannerImpl, AccessController is incorrectly apply the RPC > caller identity in the RegionServerObserver.preStopRegionServer() hook. This > leaves the regionserver in a non-responsive state, where its regions are not > reassigned and it returns exceptions for all requests. > When an abort is triggered on the server side, we should not allow a > coprocessor to reject the abort at all. > Here is a sample stack trace: > {noformat} > 17/05/25 06:10:29 FATAL regionserver.HRegionServer: RegionServer abort: > loaded coprocessors are: > [org.apache.hadoop.hbase.security.access.AccessController, > org.apache.hadoop.hbase.security.token.TokenProvider] > 17/05/25 06:10:29 WARN regionserver.HRegionServer: The region server did not > stop > org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient > permissions for user 'rpcuser' (global, action=ADMIN) > at > org.apache.hadoop.hbase.security.access.AccessController.requireGlobalPermission(AccessController.java:548) > at > org.apache.hadoop.hbase.security.access.AccessController.requirePermission(AccessController.java:522) > at > org.apache.hadoop.hbase.security.access.AccessController.preStopRegionServer(AccessController.java:2501) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost$1.call(RegionServerCoprocessorHost.java:86) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.execShutdown(RegionServerCoprocessorHost.java:300) > at > org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.preStop(RegionServerCoprocessorHost.java:82) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1905) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2118) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2125) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.abortRegionServer(HRegion.java:6326) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.handleFileNotFound(HRegion.java:6319) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5941) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6084) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5858) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2649) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34950) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2320) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168) > {noformat} > I haven't yet evaluated which other release branches this might apply to. > I have a patch currently in progress, which I will post as soon as I complete > a test case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)