[ 
https://issues.apache.org/jira/browse/HDDS-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hanisha Koneru reassigned HDDS-1414:
------------------------------------

    Assignee: Hanisha Koneru

> TestOzoneManagerHA.testMultipartUploadWithOneOmNodeDown is flaky
> ----------------------------------------------------------------
>
>                 Key: HDDS-1414
>                 URL: https://issues.apache.org/jira/browse/HDDS-1414
>             Project: Hadoop Distributed Data Store
>          Issue Type: Bug
>            Reporter: Elek, Marton
>            Assignee: Hanisha Koneru
>            Priority: Major
>              Labels: ozone-flaky-test
>         Attachments: ci2.log
>
>
> TestOzoneManagerHA.testMultipartUploadWithOneOmNodeDown is flaky, we get the 
> below exception when it fails.
> {code}
> org.apache.ratis.protocol.AlreadyClosedException: SlidingWindow$Client 
> client-04649B8D5AF3->RAFT is closed.
>  at 
> java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
>  at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
>  at 
> org.apache.hadoop.ozone.om.ratis.OzoneManagerRatisClient.sendCommand(OzoneManagerRatisClient.java:133)
>  at 
> org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitRequestToRatis(OzoneManagerProtocolServerSideTranslatorPB.java:97)
>  at 
> org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitRequest(OzoneManagerProtocolServerSideTranslatorPB.java:83)
>  at 
> org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OzoneManagerService$2.callBlockingMethod(OzoneManagerProtocolProtos.java)
>  at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
>  at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1025)
>  at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:876)
>  at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:822)
>  at java.security.AccessController.doPrivileged(Native Method)
>  at javax.security.auth.Subject.doAs(Subject.java:422)
>  at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
>  at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2682)
> Caused by: org.apache.ratis.protocol.AlreadyClosedException: 
> SlidingWindow$Client client-04649B8D5AF3->RAFT is closed.
>  at 
> org.apache.ratis.util.SlidingWindow$Client.alreadyClosed(SlidingWindow.java:350)
>  at 
> org.apache.ratis.util.SlidingWindow$Client.submitNewRequest(SlidingWindow.java:224)
>  at 
> org.apache.ratis.client.impl.RaftClientImpl.sendAsync(RaftClientImpl.java:207)
>  at 
> org.apache.ratis.client.impl.RaftClientImpl.sendAsync(RaftClientImpl.java:174)
>  at 
> org.apache.hadoop.ozone.om.ratis.OzoneManagerRatisClient.sendRequestAsync(OzoneManagerRatisClient.java:208)
>  at 
> org.apache.hadoop.ozone.om.ratis.OzoneManagerRatisClient.sendCommandAsync(OzoneManagerRatisClient.java:168)
>  at 
> org.apache.hadoop.ozone.om.ratis.OzoneManagerRatisClient.sendCommand(OzoneManagerRatisClient.java:132)
>  ... 11 more
> Caused by: org.apache.ratis.protocol.RaftRetryFailureException: Failed 
> RaftClientRequest:client-04649B8D5AF3->omNode-1@group-523986131536, cid=71, 
> seq=1*, RW, 
> org.apache.hadoop.ozone.om.ratis.OzoneManagerRatisClient$$Lambda$396/1529424209@7ae5da75
>  for 10 attempts with RetryLimited(maxAttempts=10, sleepTime=100ms)
>  at 
> org.apache.ratis.client.impl.RaftClientImpl.newRaftRetryFailureException(RaftClientImpl.java:383)
>  at 
> org.apache.ratis.client.impl.RaftClientImpl.handleAsyncRetryFailure(RaftClientImpl.java:388)
>  at 
> org.apache.ratis.client.impl.RaftClientImpl.lambda$sendRequestAsync$14(RaftClientImpl.java:370)
>  at 
> java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:870)
>  at 
> java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:852)
>  at 
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
>  at 
> java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
>  at 
> org.apache.ratis.grpc.client.GrpcClientProtocolClient$AsyncStreamObservers.completeReplyExceptionally(GrpcClientProtocolClient.java:329)
>  at 
> org.apache.ratis.grpc.client.GrpcClientProtocolClient$AsyncStreamObservers.access$000(GrpcClientProtocolClient.java:245)
>  at 
> org.apache.ratis.grpc.client.GrpcClientProtocolClient$AsyncStreamObservers$1.onNext(GrpcClientProtocolClient.java:257)
>  at 
> org.apache.ratis.grpc.client.GrpcClientProtocolClient$AsyncStreamObservers$1.onNext(GrpcClientProtocolClient.java:248)
>  at 
> org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onMessage(ClientCalls.java:421)
>  at 
> org.apache.ratis.thirdparty.io.grpc.ForwardingClientCallListener.onMessage(ForwardingClientCallListener.java:33)
>  at 
> org.apache.ratis.thirdparty.io.grpc.ForwardingClientCallListener.onMessage(ForwardingClientCallListener.java:33)
>  at 
> org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1MessagesAvailable.runInContext(ClientCallImpl.java:519)
>  at 
> org.apache.ratis.thirdparty.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
>  at 
> org.apache.ratis.thirdparty.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

Reply via email to