[
https://issues.apache.org/jira/browse/HBASE-28106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang resolved HBASE-28106.
-------------------------------
Hadoop Flags: Reviewed
Assignee: Duo Zhang
Resolution: Fixed
Pushed to branch-2.x.
Thanks [~nihaljain.cs] for reviewing!
> TestShadeSaslAuthenticationProvider fails for branch-2.x
> --------------------------------------------------------
>
> Key: HBASE-28106
> URL: https://issues.apache.org/jira/browse/HBASE-28106
> Project: HBase
> Issue Type: Bug
> Components: test
> Reporter: Nihal Jain
> Assignee: Duo Zhang
> Priority: Blocker
> Fix For: 2.6.0, 2.4.18, 2.5.6
>
>
> {{org.apache.hadoop.hbase.security.provider.example.TestShadeSaslAuthenticationProvider}}
> is failing consistently in branch-2.x
> This could be a code bug as well!
>
> For branch-2:
> * Nightly failing:
> [https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/885/testReport/]
> For branch-2.5
> * PR failing:
> [https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5422/3/testReport/]
> * Nightly failing:
> [https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/405/testReport/]
> For branch-2.4
> * PR failing:[
> [https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5423/3/testReport/]
> * Nightly failing:
> [https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/619/testReport/]
>
> StackTrace copied from failed test case:
> {code:java}
> java.lang.Exception: Unexpected exception,
> expected<org.apache.hadoop.hbase.DoNotRetryIOException> but
> was<org.apache.hadoop.hbase.client.RetriesExhaustedException>
> at
> org.junit.internal.runners.statements.ExpectException.evaluate(ExpectException.java:30)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at java.base/java.lang.Thread.run(Thread.java:829)
> Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed
> after attempts=4, exceptions:
> 2023-09-21T17:53:07.490Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:52:04.943Z, pause=100,
> maxAttempts=4}, org.apache.hadoop.hbase.client.RetriesExhaustedException:
> Failed after attempts=4, exceptions:
> 2023-09-21T17:53:06.869Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:53:06.862Z, pause=100,
> maxAttempts=4}, org.apache.hadoop.hbase.exceptions.ConnectionClosedException:
> Call to address=6a87b11f0f42:38383 failed on local exception:
> org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Connection
> closed
> 2023-09-21T17:53:06.975Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:53:06.862Z, pause=100,
> maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383
> failed on local exception: java.io.IOException: Connection reset by peer
> 2023-09-21T17:53:07.182Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:53:06.862Z, pause=100,
> maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383
> failed on local exception: java.io.IOException: Connection reset by peer
> 2023-09-21T17:53:07.489Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:53:06.862Z, pause=100,
> maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383
> failed on local exception: java.io.IOException: Connection reset by peer
> 2023-09-21T17:53:08.215Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:52:04.943Z, pause=100,
> maxAttempts=4}, org.apache.hadoop.hbase.client.RetriesExhaustedException:
> Failed after attempts=4, exceptions:
> 2023-09-21T17:53:07.597Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:53:07.591Z, pause=100,
> maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383
> failed on local exception: java.io.IOException: Connection reset by peer
> 2023-09-21T17:53:07.703Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:53:07.591Z, pause=100,
> maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383
> failed on local exception: java.io.IOException: Connection reset by peer
> 2023-09-21T17:53:07.909Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:53:07.591Z, pause=100,
> maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383
> failed on local exception: java.io.IOException: Connection reset by peer
> 2023-09-21T17:53:08.215Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:53:07.591Z, pause=100,
> maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383
> failed on local exception: java.io.IOException: Connection reset by peer
> 2023-09-21T17:53:09.057Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:52:04.943Z, pause=100,
> maxAttempts=4}, org.apache.hadoop.hbase.client.RetriesExhaustedException:
> Failed after attempts=4, exceptions:
> 2023-09-21T17:53:08.422Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:53:08.416Z, pause=100,
> maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383
> failed on local exception: java.io.IOException: Connection reset by peer
> 2023-09-21T17:53:08.530Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:53:08.416Z, pause=100,
> maxAttempts=4}, org.apache.hadoop.hbase.exceptions.ConnectionClosedException:
> Call to address=6a87b11f0f42:38383 failed on local exception:
> org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Connection
> closed
> 2023-09-21T17:53:08.748Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:53:08.416Z, pause=100,
> maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383
> failed on local exception: java.io.IOException: Connection reset by peer
> 2023-09-21T17:53:09.056Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:53:08.416Z, pause=100,
> maxAttempts=4}, org.apache.hadoop.hbase.exceptions.ConnectionClosedException:
> Call to address=6a87b11f0f42:38383 failed on local exception:
> org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Connection
> closed
> 2023-09-21T17:53:09.990Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:52:04.943Z, pause=100,
> maxAttempts=4}, org.apache.hadoop.hbase.client.RetriesExhaustedException:
> Failed after attempts=4, exceptions:
> 2023-09-21T17:53:09.362Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:53:09.357Z, pause=100,
> maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383
> failed on local exception: java.io.IOException: Connection reset by peer
> 2023-09-21T17:53:09.469Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:53:09.357Z, pause=100,
> maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383
> failed on local exception: java.io.IOException: Connection reset by peer
> 2023-09-21T17:53:09.683Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:53:09.357Z, pause=100,
> maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383
> failed on local exception: java.io.IOException: Connection reset by peer
> 2023-09-21T17:53:09.990Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:53:09.357Z, pause=100,
> maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383
> failed on local exception: java.io.IOException: Connection reset by peer
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:141)
> at org.apache.hadoop.hbase.client.HTable.get(HTable.java:390)
> at org.apache.hadoop.hbase.client.HTable.lambda$get$0(HTable.java:363)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216)
> at org.apache.hadoop.hbase.client.HTable.get(HTable.java:363)
> at
> org.apache.hadoop.hbase.security.provider.example.TestShadeSaslAuthenticationProvider$3.run(TestShadeSaslAuthenticationProvider.java:235)
> at
> org.apache.hadoop.hbase.security.provider.example.TestShadeSaslAuthenticationProvider$3.run(TestShadeSaslAuthenticationProvider.java:230)
> at java.base/java.security.AccessController.doPrivileged(Native Method)
> at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
> at
> org.apache.hadoop.hbase.security.provider.example.TestShadeSaslAuthenticationProvider.testNegativeAuthentication(TestShadeSaslAuthenticationProvider.java:230)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.ExpectException.evaluate(ExpectException.java:19)
> ... 18 more
> Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed
> after attempts=4, exceptions:
> 2023-09-21T17:53:09.362Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:53:09.357Z, pause=100,
> maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383
> failed on local exception: java.io.IOException: Connection reset by peer
> 2023-09-21T17:53:09.469Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:53:09.357Z, pause=100,
> maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383
> failed on local exception: java.io.IOException: Connection reset by peer
> 2023-09-21T17:53:09.683Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:53:09.357Z, pause=100,
> maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383
> failed on local exception: java.io.IOException: Connection reset by peer
> 2023-09-21T17:53:09.990Z,
> RpcRetryingCaller{globalStartTime=2023-09-21T17:53:09.357Z, pause=100,
> maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383
> failed on local exception: java.io.IOException: Connection reset by peer
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:141)
> at org.apache.hadoop.hbase.client.HTable.get(HTable.java:390)
> at org.apache.hadoop.hbase.client.HTable.lambda$get$0(HTable.java:363)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216)
> at org.apache.hadoop.hbase.client.HTable.get(HTable.java:363)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.getTableState(MetaTableAccessor.java:1141)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.getTableState(ConnectionImplementation.java:2235)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.isTableDisabled(ConnectionImplementation.java:742)
> at
> org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:214)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:101)
> ... 37 more
> Caused by: java.io.IOException: Call to address=6a87b11f0f42:38383 failed on
> local exception: java.io.IOException: Connection reset by peer
> at
> java.base/jdk.internal.reflect.GeneratedConstructorAccessor69.newInstance(Unknown
> Source)
> at
> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at
> java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
> at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:239)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:391)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:92)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:425)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:420)
> at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:114)
> at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:129)
> at
> org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.cleanupCalls(NettyRpcDuplexHandler.java:212)
> at
> org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.exceptionCaught(NettyRpcDuplexHandler.java:229)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:325)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:317)
> at
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:325)
> at
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.handleReadException(AbstractNioByteChannel.java:125)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:177)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
> at
> org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> ... 1 more
> Caused by: java.io.IOException: Connection reset by peer
> at java.base/sun.nio.ch.FileDispatcherImpl.read0(Native Method)
> at java.base/sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
> at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:276)
> at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:233)
> at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:223)
> at
> java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:356)
> at
> org.apache.hbase.thirdparty.io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:259)
> at
> org.apache.hbase.thirdparty.io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1132)
> at
> org.apache.hbase.thirdparty.io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:357)
> at
> org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:151)
> ... 8 more
> {code}
>
> Also note the test class code is not same between master and branch-2.x
> *
> [https://github.com/apache/hbase/blob/master/hbase-examples/src/test/java/org/apache/hadoop/hbase/security/provider/example/TestShadeSaslAuthenticationProvider.java]
> *
> [https://github.com/apache/hbase/blob/branch-2/hbase-examples/src/test/java/org/apache/hadoop/hbase/security/provider/example/TestShadeSaslAuthenticationProvider.java]
>
> This issue could be related to HBASE-28101
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
