stayrascal commented on pull request #4724: URL: https://github.com/apache/hudi/pull/4724#issuecomment-1041115256
> CC @xushiyan @alexeykudinkin : who might be working on refactoring payload interfaces. and @xiarixiaoyao who might be working on schema evolution story. can you folks take a look at the patch. Thanks @nsivabalan to follow up. Take a summary, the current implementation has two drawbacks: - embedding schema in payload might hard to do schema evolution later. - using flink state to store full HoodieRecord instead of location, will increase the state size. And there are two challenges need to be solved for supporting partial update. - how combine/merge two payload in `#preCombine` if without schema. - how to lookup the record from base/log file during the record partition patch changed, and then combine/merge with the incoming record later. Any thoughts on these two challenges? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
