codope commented on issue #5880: URL: https://github.com/apache/hudi/issues/5880#issuecomment-1158935896
@gtwuser You can setup cleaning to clear old data either using hudi-cli or along with the writes. Check [this doc](https://hudi.apache.org/docs/hoodie_cleaner) for more details. But, it should not result in duplicate (as you see in the case of Athena). Any query engine that understands the Hudi metadata should be able to filter out older data in the latest snapshot. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
