vinothchandar commented on pull request #1819: URL: https://github.com/apache/hudi/pull/1819#issuecomment-668284682
Folks, I see a performance/efficiency problem with the approach here. We added a string field to every payload object, which will get increase the shuffle size in the write path. We would be just sending the same string with each and every object. Can we rework this so that we introduce a new `PayloadConfig` object which we can hand to the payload methods at runtime, instead of adding this to constructor? We need to introduce new methods without breaking existing Custom payload implementations. cc @nsivabalan @shenh062326 @xushiyan ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
