[
https://issues.apache.org/jira/browse/RATIS-724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16956115#comment-16956115
]
Tsz-wo Sze commented on RATIS-724:
----------------------------------
[~shashikant], the stack trace is pasted as a single (very) long line so that
it is hard to read. Could you reformat it?
BTW, what is the exception? Leader election could timeout if the other servers
do not response.
> Leader election seem to timeout in Ratis
> ----------------------------------------
>
> Key: RATIS-724
> URL: https://issues.apache.org/jira/browse/RATIS-724
> Project: Ratis
> Issue Type: Bug
> Reporter: Shashikant Banerjee
> Priority: Major
> Fix For: 0.5.0
>
>
> {code:java}
> java.util.concurrent.ExecutionException:
> org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: INTERNAL:
> e9477db3-627c-447b-9726-7a0202331e44@group-284D2F681BFD is not in [RUNNING]:
> current state is NEWjava.util.concurrent.ExecutionException:
> org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: INTERNAL:
> e9477db3-627c-447b-9726-7a0202331e44@group-284D2F681BFD is not in [RUNNING]:
> current state is NEW at
> java.util.concurrent.FutureTask.report(FutureTask.java:122) at
> java.util.concurrent.FutureTask.get(FutureTask.java:192) at
> org.apache.ratis.server.impl.LeaderElection.waitForResults(LeaderElection.java:259)
> at
> org.apache.ratis.server.impl.LeaderElection.askForVotes(LeaderElection.java:197)
> at org.apache.ratis.server.impl.LeaderElection.run(LeaderElection.java:133)
> at java.lang.Thread.run(Thread.java:748)Caused by:
> org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: INTERNAL:
> e9477db3-627c-447b-9726-7a0202331e44@group-284D2F681BFD is not in [RUNNING]:
> current state is NEW at
> org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:233)
> at
> org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:214)
> at
> org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:139)
> at
> org.apache.ratis.proto.grpc.RaftServerProtocolServiceGrpc$RaftServerProtocolServiceBlockingStub.requestVote(RaftServerProtocolServiceGrpc.java:265)
> at
> org.apache.ratis.grpc.server.GrpcServerProtocolClient.requestVote(GrpcServerProtocolClient.java:99)
> at
> org.apache.ratis.grpc.server.GrpcService.requestVote(GrpcService.java:204) at
> org.apache.ratis.server.impl.LeaderElection.lambda$submitRequests$1(LeaderElection.java:234)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at
> java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ... 1 more2019-10-21 17:40:18,702
> [4ed74939-427d-455d-852a-df3499c9dbb2@group-284D2F681BFD-LeaderElection1]
> INFO impl.LeaderElection (LogUtils.java:infoOrTrace(149)) -
> 4ed74939-427d-455d-852a-df3499c9dbb2@group-284D2F681BFD-LeaderElection1 got
> exception when requesting votes: {}java.util.concurrent.ExecutionException:
> org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException:
> DEADLINE_EXCEEDED: deadline exceeded after 2998885738ns at
> java.util.concurrent.FutureTask.report(FutureTask.java:122) at
> java.util.concurrent.FutureTask.get(FutureTask.java:192) at
> org.apache.ratis.server.impl.LeaderElection.waitForResults(LeaderElection.java:259)
> at
> org.apache.ratis.server.impl.LeaderElection.askForVotes(LeaderElection.java:197)
> at org.apache.ratis.server.impl.LeaderElection.run(LeaderElection.java:133)
> at java.lang.Thread.run(Thread.java:748)Caused by:
> org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException:
> DEADLINE_EXCEEDED: deadline exceeded after 2998885738ns at
> org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:233)
> at
> org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:214)
> at
> org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:139)
> at
> org.apache.ratis.proto.grpc.RaftServerProtocolServiceGrpc$RaftServerProtocolServiceBlockingStub.requestVote(RaftServerProtocolServiceGrpc.java:265)
> at
> org.apache.ratis.grpc.server.GrpcServerProtocolClient.requestVote(GrpcServerProtocolClient.java:99)
> at
> org.apache.ratis.grpc.server.GrpcService.requestVote(GrpcService.java:204) at
> org.apache.ratis.server.impl.LeaderElection.lambda$submitRequests$1(LeaderElection.java:234)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at
> java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ... 1 more2019-10-21 17:40:18,705
> [4ed74939-427d-455d-852a-df3499c9dbb2@group-284D2F681BFD-LeaderElection1]
> INFO impl.LeaderElection (LeaderElection.java:logAndReturn(56)) -
> 4ed74939-427d-455d-852a-df3499c9dbb2@group-284D2F681BFD-LeaderElection1:
> Election REJECTED; received 0 response(s) [] and 2 exception(s);
> 4ed74939-427d-455d-852a-df3499c9dbb2@group-284D2F681BFD:t1, leader=null,
> voted=4ed74939-427d-455d-852a-df3499c9dbb2,
> raftlog=4ed74939-427d-455d-852a-df3499c9dbb2@group-284D2F681BFD-SegmentedRaftLog:OPENED:c-1,f-1,i0,
> conf=-1: [4ed74939-427d-455d-852a-df3499c9dbb2:192.168.29.38:50945,
> 31cc45f0-16ea-48bc-b870-9e866f150589:192.168.29.38:50943,
> e9477db3-627c-447b-9726-7a0202331e44:192.168.29.38:50944], old=null2019-10-21
> 17:40:18,705
> [4ed74939-427d-455d-852a-df3499c9dbb2@group-284D2F681BFD-LeaderElection1]
> INFO impl.LeaderElection (LogUtils.java:infoOrTrace(149)) - Exception 0:
> {}java.util.concurrent.ExecutionException:
> org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: INTERNAL:
> e9477db3-627c-447b-9726-7a0202331e44@group-284D2F681BFD is not in [RUNNING]:
> current state is NEW at
> java.util.concurrent.FutureTask.report(FutureTask.java:122) at
> java.util.concurrent.FutureTask.get(FutureTask.java:192) at
> org.apache.ratis.server.impl.LeaderElection.waitForResults(LeaderElection.java:259)
> at
> org.apache.ratis.server.impl.LeaderElection.askForVotes(LeaderElection.java:197)
> at org.apache.ratis.server.impl.LeaderElection.run(LeaderElection.java:133)
> at java.lang.Thread.run(Thread.java:748)Caused by:
> org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: INTERNAL:
> e9477db3-627c-447b-9726-7a0202331e44@group-284D2F681BFD is not in [RUNNING]:
> current state is NEW at
> org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:233)
> at
> org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:214)
> at
> org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:139)
> at
> org.apache.ratis.proto.grpc.RaftServerProtocolServiceGrpc$RaftServerProtocolServiceBlockingStub.requestVote(RaftServerProtocolServiceGrpc.java:265)
> at
> org.apache.ratis.grpc.server.GrpcServerProtocolClient.requestVote(GrpcServerProtocolClient.java:99)
> at
> org.apache.ratis.grpc.server.GrpcService.requestVote(GrpcService.java:204) at
> org.apache.ratis.server.impl.LeaderElection.lambda$submitRequests$1(LeaderElection.java:234)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at
> java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ... 1 more2019-10-21 17:40:18,705
> [4ed74939-427d-455d-852a-df3499c9dbb2@group-284D2F681BFD-LeaderElection1]
> INFO impl.LeaderElection (LogUtils.java:infoOrTrace(149)) - Exception 1:
> {}java.util.concurrent.ExecutionException:
> org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException:
> DEADLINE_EXCEEDED: deadline exceeded after 2998885738ns at
> java.util.concurrent.FutureTask.report(FutureTask.java:122) at
> java.util.concurrent.FutureTask.get(FutureTask.java:192) at
> org.apache.ratis.server.impl.LeaderElection.waitForResults(LeaderElection.java:259)
> at
> org.apache.ratis.server.impl.LeaderElection.askForVotes(LeaderElection.java:197)
> at org.apache.ratis.server.impl.LeaderElection.run(LeaderElection.java:133)
> at java.lang.Thread.run(Thread.java:748)Caused by:
> org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException:
> DEADLINE_EXCEEDED: deadline exceeded after 2998885738ns at
> org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:233)
> at
> org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:214)
> at
> org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:139)
> at
> org.apache.ratis.proto.grpc.RaftServerProtocolServiceGrpc$RaftServerProtocolServiceBlockingStub.requestVote(RaftServerProtocolServiceGrpc.java:265)
> at
> org.apache.ratis.grpc.server.GrpcServerProtocolClient.requestVote(GrpcServerProtocolClient.java:99)
> at
> org.apache.ratis.grpc.server.GrpcService.requestVote(GrpcService.java:204) at
> org.apache.ratis.server.impl.LeaderElection.lambda$submitRequests$1(LeaderElection.java:234)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at
> java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ... 1 more2019-10-21 17:40:18,708
> [4ed74939-427d-455d-852a-df3499c9dbb2@group-284D2F681BFD-LeaderElection1]
> INFO impl.RaftServerImpl (RaftServerImpl.java:setRole(173)) -
> 4ed74939-427d-455d-85
> {code}
> This was observed while running ozone with ratis snapshot
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
