xushiyan commented on PR #5436:
URL: https://github.com/apache/hudi/pull/5436#issuecomment-1156384006
Clarified with @YannByron and @danny0405 and we agreed that for
`supplemental.logging=true` use case:
- users can set `cdc.supplemental.logging=true` to log extra info
- the extra info contains record key, operation type (I/U/D), "before"
image, "after" image
- a config can be used to skip logging the "after" image
- the extra info should be encoded in a new cdc log block
Clarified with @danny0405 that when `supplemental.logging=false`, we only
log record keys and operation type, which are used to infer records' changes
during cdc/incremental queries. The record keys are important to reducing the
compute cost during the inference, as explained in the comment
https://github.com/apache/hudi/pull/5436#issuecomment-1151213942
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
