runzhiwang opened a new pull request #127: URL: https://github.com/apache/incubator-ratis/pull/127
## What changes were proposed in this pull request? Fix thread unsafe reply. What's the problem ? 1. when s3 ask s0 to append entry, then s0 finish append entry, and reply to s3. `2020-06-15T02:09:05.5568987Z 2020-06-15 02:09:05,554 [Thread-82] DEBUG impl.RaftServerImpl (RaftServerImpl.java:logAppendEntries(916)) - s0@group-E95D39D4BA60: succeeded to handle AppendEntries. Reply: s3<-s0#9:OK,SUCCESS,nextIndex:2,term:2,followerCommit:0,matchIndex:1` 2. s3 failed to read message `2020-06-15T02:09:05.6204785Z 2020-06-15 02:09:05,605 [grpc-default-executor-1] WARN server.GrpcLogAppender (LogUtils.java:warn(122)) - s3@group-E95D39D4BA60->s0-AppendLogResponseHandler: Failed appendEntries: org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: CANCELLED: Failed to read message.` 3. The stack trace is: ``` 2020-06-15T02:09:05.6092588Z org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: CANCELLED: Failed to read message. 2020-06-15T02:09:05.6092751Z at org.apache.ratis.thirdparty.io.grpc.Status.asRuntimeException(Status.java:533) 2020-06-15T02:09:05.6092920Z at org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:449) 2020-06-15T02:09:05.6093083Z at org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:426) 2020-06-15T02:09:05.6093245Z at org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl.access$500(ClientCallImpl.java:66) 2020-06-15T02:09:05.6093420Z at org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:689) 2020-06-15T02:09:05.6093787Z at org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$900(ClientCallImpl.java:577) 2020-06-15T02:09:05.6093967Z at org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1MessagesAvailable.runInternal(ClientCallImpl.java:670) 2020-06-15T02:09:05.6094134Z at org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1MessagesAvailable.runInContext(ClientCallImpl.java:643) 2020-06-15T02:09:05.6094300Z at org.apache.ratis.thirdparty.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) 2020-06-15T02:09:05.6094460Z at org.apache.ratis.thirdparty.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) 2020-06-15T02:09:05.6094615Z at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 2020-06-15T02:09:05.6094846Z at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 2020-06-15T02:09:05.6094997Z at java.lang.Thread.run(Thread.java:748) 2020-06-15T02:09:05.6095159Z Caused by: org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: INTERNAL: Invalid protobuf byte sequence 2020-06-15T02:09:05.6095316Z at org.apache.ratis.thirdparty.io.grpc.Status.asRuntimeException(Status.java:524) 2020-06-15T02:09:05.6095480Z at org.apache.ratis.thirdparty.io.grpc.protobuf.lite.ProtoLiteUtils$MessageMarshaller.parse(ProtoLiteUtils.java:218) 2020-06-15T02:09:05.6095649Z at org.apache.ratis.thirdparty.io.grpc.protobuf.lite.ProtoLiteUtils$MessageMarshaller.parse(ProtoLiteUtils.java:118) 2020-06-15T02:09:05.6095816Z at org.apache.ratis.thirdparty.io.grpc.MethodDescriptor.parseResponse(MethodDescriptor.java:275) 2020-06-15T02:09:05.6095972Z at org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1MessagesAvailable.runInternal(ClientCallImpl.java:658) 2020-06-15T02:09:05.6096126Z ... 6 more 2020-06-15T02:09:05.6096283Z Caused by: org.apache.ratis.thirdparty.com.google.protobuf.InvalidProtocolBufferException: Protocol message contained an invalid tag (zero). 2020-06-15T02:09:05.6096473Z at org.apache.ratis.thirdparty.com.google.protobuf.InvalidProtocolBufferException.invalidTag(InvalidProtocolBufferException.java:102) 2020-06-15T02:09:05.6096697Z at org.apache.ratis.thirdparty.com.google.protobuf.CodedInputStream$ArrayDecoder.readTag(CodedInputStream.java:627) 2020-06-15T02:09:05.6096864Z at org.apache.ratis.proto.RaftProtos$AppendEntriesReplyProto.<init>(RaftProtos.java:16335) ``` ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/RATIS-977 ## How was this patch tested? Existed tests. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
