n3nash commented on issue #2975: URL: https://github.com/apache/hudi/issues/2975#issuecomment-847407792
@calleo Hudi allows you to write custom merge logic at a record level so you don't have to read the target table. Instead, you can just provide the input from the source non-hudi table, define your merge logic and let Hudi ensure to merge the incoming and on-disk data using the logic you have defined. One way to do this is to implement the [HoodieRecordPayload](https://github.com/apache/hudi/blob/master/hudi-common/src/main/java/org/apache/hudi/common/model/HoodieRecordPayload.java) in scala or java and combine it with the bundle hudi jars or drop this in with your spark classpath. You will also need to set this as the custom merge logic class here -> https://github.com/apache/hudi/blob/master/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieCompactionConfig.java#L100 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
