parisni commented on issue #8824: URL: https://github.com/apache/hudi/issues/8824#issuecomment-1590137768
hi @machadoluiz . [Was reading that blog post](https://www.onehouse.ai/blog/hudi-metafields-demystified?830fc854_page=2) from @bhasudha > Near Infinite time-travel history >Another powerful feature that Hudi unlocks with this meta field is the ability to retain near infinite history for records. One of the Hudi community users - a large bank, is able to successfully leverage this feature to support time travel queries on historical data - even upto 5 or 6 years back. This can be realized in practice by merely managing file sizing configs, enabling scalable metadata and disabling cleaner. Without persisting the commit time along with the record, it is impossible to see a history of the record right from when it was created. This feature comes in handy when you want to tap the time travelling abilities in a historical table with so many years of data. > Combined with Hudi’s scalable table metadata, this can unlock near infinite history retention, that has enabled some Hudi users time travel even few years back. Then if I understand correctly they disable the cleaning, and also archive their commits. Then in oder to time travel they rely on spark filters on _hoodie_commit_time instead of relying on the timeline. As a result their timeline keep having a managable size, while being able to travel (with a compute cost) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
