yihua commented on issue #9430: URL: https://github.com/apache/hudi/issues/9430#issuecomment-1693807306
@beyond1920 IIUC the problem you have at high-level, if the record of a record key does not exist in the current data based on the input (e.g., `id=2` record does not exist in the current data, and `id=2` exists after legacy precombine logic), the new merger API is not called and no customization can be made. If this is the case, we can adjust the merge API in `HoodieRecordMerger ` to: ``` Option<HoodieRecord> merge(Option<HoodieRecord> older, Option<HoodieRecord> newer, Schema schema, Properties props) throws IOException; ``` The changes are to make `older` and `newer` records optional. So that besides calling this for `combineAndGetUpdateValue` logic, when only `newer` HoodieRecord exists (i.e., `older` is empty), we also call this API to allow custom logic, which solves your problem. This requires changes to the new merging logic but it opens up the same functionality and flexibility as before. Then you only need to implement this new record merger API without having to change other classes. wdyt? Does this work for you? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
