HolyLow commented on PR #3177: URL: https://github.com/apache/celeborn/pull/3177#issuecomment-2769233862
@pan3793 We choose to use different serialisation method for different language because of following reasons: 1. Backward compatibility: we want to keep complete backward compatibility and ensure that old-versioned existing java clients could still communicate with new-versioned servers. In production environment, upgrading all the clients and server nodes to use the new version might not be viable, and backward compatibility is top priority for the new serialisation design. 2. Difficulty in being compatible with java serialisation: the existing java serialisation invokes java-related serialisation stack, and the java messeges including class hierarchy, magic number, versioning are deeply involved. These behaviours make it almost impossible for cpp to adapt to java serialisation. For these reasons, we choose to use different serialisation for different language, marked by the first byte. Actually, only the java-serialization-involved messages would require the special handling. These messages are used by cppClient/javaClient to communicate with lifecycleManager. For other messages, the serialisation doesn't involve java serialisation and would be handled uniformly. You could refer to design doc for more design comparisons, see https://docs.google.com/document/d/1EMx1fryZx0gKd4Q32uQEi1835kNhB92yRjteILQpijg/edit?usp=sharing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
