maytasm commented on a change in pull request #12190: URL: https://github.com/apache/druid/pull/12190#discussion_r793047456
########## File path: docs/operations/clean-metadata-store.md ########## @@ -117,12 +115,19 @@ Rule cleanup uses the following configuration: ### Compaction configuration records +Druid retains all compaction configuration records by default, which should be suitable for most use cases. +If you have a high datasource churn rate, that is, you create and delete short-lived datasources with high frequency, consider turning on automated cleanup of compaction configuration records. +With this feature turned on, when you create a compaction configuration for some datasource before the datasource exists, for example if ingestion is ongoing, Druid may remove the compaction configuration. +To prevent the configuration from being prematurely removed, wait to set `druid.coordinator.kill.compaction.on=true` until after the datasource is created. + Compaction configuration records in the `druid_config` table become eligible for deletion after all segments for the datasource have been killed by the kill task. Automated cleanup for compaction configuration requires a [kill task](#kill-task). Compaction configuration cleanup uses the following configuration: - - `druid.coordinator.kill.compaction.on`: When `true`, enables cleanup for compaction configuration records. + - `druid.coordinator.kill.compaction.on`: When `true`, enables cleanup for compaction configuration records. - `druid.coordinator.kill.compaction.period`: Defines the frequency in [ISO 8601 format](https://en.wikipedia.org/wiki/ISO_8601#Durations) for the cleanup job to check for and delete eligible compaction configuration records. Defaults to `P1D`. +Compaction configuration records do not have a retention period. Druid deletes compaction configuration records at every cleanup cycle for inactive datasources, which do not have segments either used or unused. Review comment: Feel like this might fit better in the earlier paragraph. Maybe between line 119 and 120? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
