chaokunyang commented on PR #1413: URL: https://github.com/apache/incubator-fury/pull/1413#issuecomment-2015069093
The Rust and C++ need more consideration, since they don't support dynamic codegen. We need to genertate all code are compilet-time use meta programing. The generate code may not be the best code for execution. For type evolution, the serializer will encode the type meta into the serialized data. The deserializer can compare this meta with class in current process, and use the diff to generate the serializer for deserializaiton. - For java/javascript/python, we can use the diff to generate serializer code at runtime and load it as `class/function` for deserialization. In this way, the type evolution will be as fast as type consist mode. - For C++/Rust, we can't generate the serializer code at runtime. So we need to generate the code at compile-time. But at that time, we don't know the type schema in other processes. So we can't generate the serializer code for such inconsistent types. We may need to generate the code which has a loop and compare field name one by one to decide whether deserialize and assign the field or deserialize and skip the field. One lucky thing is that we can cache the type meta, and genetate a 64-bit id for every for field name, and genetate the id for field names for current type at build time, so we can convert the field name comparasion into long comparasion. And it will be fast too. - - - use field pointer to -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
