JRobTS commented on issue #15912: URL: https://github.com/apache/druid/issues/15912#issuecomment-1954275964
I switched to 10 minute intervals and it is now working; the kill task runs every hour (since I have hourly segments) and picks up both the unused segments from compaction and unused segments from drop rules. Using the default indexing and kill periods would allow deep storage to grow forever but using a configuration like the following works: ``` # Enable auto-delete from deep storage druid.coordinator.kill.on=true druid.coordinator.kill.period=PT10M druid.coordinator.kill.durationToRetain=P1D druid.coordinator.kill.bufferPeriod=PT6H druid.coordinator.kill.maxSegments=10000 druid.coordinator.period.indexingPeriod=PT10M ``` I will stress that this works because the kill task runs more frequently than compaction does, which allows the kill job to find nothing and clear the memory map so that subsequent kill jobs can run with full context (include drop rules). The kill job has to find nothing before it will look for everything. :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
