jihoonson commented on issue #9755: URL: https://github.com/apache/druid/issues/9755#issuecomment-664823080
Ah sorry for missing details. For the generation of segments, it's similar to what you understand. It means how many overwrites have done before the most recent segments. So, the first generation of segments can be the very initial set of segments in a time chunk. The second generation can be another set of segments which were created by an overwrite task and have a higher version than the first generation segments. But, I didn't think storing the generation information anywhere since it would be only used for auto cleanup. Instead, you can check how many generations are in a time chunk when you retrieve segments from metadata store. So, given `numGenerationsToKeep = 3`, the auto cleanup task can remove the segments older than 3 generations if there are any. In that way, we can safely keep the most recent couple of generations of segments. You can keep only the latest generation if you want. Does it make sense? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
