[
https://issues.apache.org/jira/browse/HDDS-1591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16849969#comment-16849969
]
Eric Yang commented on HDDS-1591:
---------------------------------
The root cause of this problem is the protobuf is not version controlled
properly.
{code}
message OMResponse {
required Type cmdType = 1; // Type of the command
...
}
enum Status {
...
}
{code}
The first required field is Type. This is a enum that is changing. If the
protobuf format changes over time, it is best to define a required String tag
that represent the version number as the first field. This will provide hints
to which version of Java class the protobuf message should serialize into.
Without the proper version control of protobuf format, we may get random error
messages when passing message does not match codegen Java class.
> Protocol buffer classes are incompatible between builds
> -------------------------------------------------------
>
> Key: HDDS-1591
> URL: https://issues.apache.org/jira/browse/HDDS-1591
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Reporter: Eric Yang
> Priority: Major
>
> The generated classes are incompatible between builds. If developer run mvn
> clean package to build the entire Ozone project and did not preserve the
> generated jar files in maven cache. When he rebuild certain subproject, he
> might end up using snapshot jar files from nightly build. However, the
> protobuff message generated locally is not compatible with newly downloaded
> protobuff classes from nightly build. This result in strange RPC behavior
> such as:
> {code:java}
> 2019-05-22 17:57:14,650 INFO retry.RetryInvocationHandler
> (RetryInvocationHandler.java:log(411)) -
> com.google.protobuf.ServiceException:
> org.apache.hadoop.ipc.RemoteException(com.google.protobuf.InvalidProtocolBufferException):
> Message missing required fields:
> createVolumeRequest.volumeInfo.volumeAcls[0].rights,
> createVolumeRequest.volumeInfo.volumeAcls[1].rights,
> createVolumeRequest.volumeInfo.volumeAcls[2].rights,
> createVolumeRequest.volumeInfo.volumeAcls[3].rights
> at
> com.google.protobuf.UninitializedMessageException.asInvalidProtocolBufferException(UninitializedMessageException.java:81)
> at
> com.google.protobuf.AbstractParser.checkMessageInitialized(AbstractParser.java:71)
> at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:89)
> at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:95)
> at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:49)
> at
> org.apache.hadoop.ipc.RpcWritable$ProtobufWrapper.readFrom(RpcWritable.java:125)
> at org.apache.hadoop.ipc.RpcWritable$Buffer.getValue(RpcWritable.java:187)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:514)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1025)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:876)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:822)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2682)
> , while invoking $Proxy19.submitRequest over null(localhost:9862). Trying to
> failover immediately.{code}
> This is a codegen issue that version control of the generated code is not
> done correctly, and result in incompatible RPC calls. There are two
> solutions, using proto3 to remove mandatory fields, or get better at version
> control the codegen process.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
