bvaradar commented on issue #2186: URL: https://github.com/apache/hudi/issues/2186#issuecomment-713305298
cc @n3nash who is rethinking schema management in Hudi. 1. Regarding schema evolution, Hudi uses avro schema compatibility rules. For example, new fields must be nullable, columns cannot be deleted... 2. https://github.com/apache/hudi/issues/1977 should give context around AWS Glue support (cc @umehrot2 ) 3. Yeah, I am assuming you are referring to cleaner retentions. Commits happen before cleaning. Any failed commits will not cause any side-effects. 4. For Incremental pull to work, you need to employ soft-deletes only with tombstones general pattern. You can retain hudi key but null out other columns (nullable) or obfuscate. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
