nsivabalan edited a comment on pull request #2486: URL: https://github.com/apache/hudi/pull/2486#issuecomment-773336965
if I am not wrong, preCombine is an optional field. don't think its a mandatory one. So, if not set, we might choose some default value. Also, since this is a behavior change and not everyone prefers to filter out but rather fail, and this incurs additional processing as well, can we add a new config and guard it against it. If the config is not set, old behavior should be retained. only those users who are interested can enable it. @vinothchandar @n3nash : Any opinions on this PR is appreciated. on a high level, this patch filters out those records considered to be invalid (i.e. w/ record key field or preCombine field) and proceeds w/ rest. if not for this patch, we will fail the operation. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
