HolyLow commented on PR #3177: URL: https://github.com/apache/celeborn/pull/3177#issuecomment-2769602423
@pan3793 The "extra byte" is to ensure that the new cpp serialization method (or what you called language-agnostic serialization method) would not collide with the existing java serialization. The java serialization has its specific header started by 0xAC, and if we use the new serialization method without a new header, there is possibility that the serialized data's prefix is exactly the same as existing java's serialization header, and we fail to recognize the new serialization method. Therefore, the language header or marker is necessary and couldn't be eliminated without breaking existing java serialization stack. Besides, the serialization method used by cpp is actually language-agnostic already as it doesn't introduce any language-dependent component. The new serialization method is mostly based on the protobuf but with some adaptation to java's existing TransportMessage's handling stack. I think reorganizing/refactoring java's message decode/encode framework is the java engine's issue, not the cppClient's issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
