swuferhong commented on PR #3623: URL: https://github.com/apache/fluss/pull/3623#issuecomment-4988737514
> Thanks for working on this. After reviewing the changes, I have a couple of questions: > > 1. Without this PR, if we set table.log.tiered.local-segments = 1, would that achieve the same goal as this PR? > 2. It looks like this PR does not roll the active segment for historical/inactive partitions. Hi, @zuston. Based on the offline discussion with @gyang94 , implementing TTL for the active segment in the short term is fairly tricky and requires a detailed design—but that doesn't mean it shouldn't be done. You can continue working on this in a follow-up PR. Regarding whether it can be set to `table.log.tiered.local-segments = 1`: these are two completely different concepts. One is about how much local data to reserve as a local read cache, and the other is about how long data should be retained before TTL kicks in. Consider the following scenario: the user has an hourly-partitioned table where partitions are retained for 24 hours, the log TTL is set to 1 hour, and the number of local segments is 5 (to ensure that the most recent hour of appended data stays on the server). In this case, the historical 23 hours would incur a large amount of redundant storage—even though the user has configured the log TTL to 1 hour. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
