zachjsh commented on code in PR #14831:
URL: https://github.com/apache/druid/pull/14831#discussion_r1298796406
##########
docs/configuration/index.md:
##########
@@ -853,7 +853,7 @@ These Coordinator static configurations can be defined in
the `coordinator/runti
|`druid.coordinator.load.timeout`|The timeout duration for when the
Coordinator assigns a segment to a Historical process.|PT15M|
|`druid.coordinator.kill.pendingSegments.on`|Boolean flag for whether or not
the Coordinator clean up old entries in the `pendingSegments` table of metadata
store. If set to true, Coordinator will check the created time of most recently
complete task. If it doesn't exist, it finds the created time of the earliest
running/pending/waiting tasks. Once the created time is found, then for all
dataSources not in the `killPendingSegmentsSkipList` (see [Dynamic
configuration](#dynamic-configuration)), Coordinator will ask the Overlord to
clean up the entries 1 day or more older than the found created time in the
`pendingSegments` table. This will be done periodically based on
`druid.coordinator.period.indexingPeriod` specified.|true|
|`druid.coordinator.kill.on`|Boolean flag for whether or not the Coordinator
should submit kill task for unused segments, that is, permanently delete them
from metadata store and deep storage. If set to true, then for all whitelisted
dataSources (or optionally all), Coordinator will submit tasks periodically
based on `period` specified. A whitelist can be set via dynamic configuration
`killDataSourceWhitelist` described later.<br /><br />When
`druid.coordinator.kill.on` is true, segments are eligible for permanent
deletion once their data intervals are older than
`druid.coordinator.kill.durationToRetain` relative to the current time. If a
segment's data interval is older than this threshold at the time it is marked
unused, it is eligible for permanent deletion immediately after being marked
unused.|false|
-|`druid.coordinator.kill.period`|How often to send kill tasks to the indexing
service. Value must be greater than `druid.coordinator.period.indexingPeriod`.
Only applies if kill is turned on.|P1D (1 Day)|
+|`druid.coordinator.kill.period`| How often to send kill tasks to the indexing
service. Value must be greater `druid.coordinator.period.indexingPeriod` if
set. Only applies if kill is turned on.
| NULL (indexing period is used) |
Review Comment:
thanks done.
##########
server/src/main/java/org/apache/druid/server/coordinator/DruidCoordinator.java:
##########
@@ -624,6 +630,27 @@ CompactSegments
initializeCompactSegmentsDuty(CompactionSegmentSearchPolicy comp
}
}
+ @Nullable
+ @VisibleForTesting
+ KillUnusedSegments initializeKillUnusedSegmentsDuty()
+ {
+ List<KillUnusedSegments> killUnusedSegmentsDutyFromCustomGroups =
getKillUnusedSegmentsDutyFromCustomGroups();
+ if (!config.getAutoKillEnabled()) {
+ return null;
+ }
+ if (killUnusedSegmentsDutyFromCustomGroups.isEmpty()) {
+ return new KillUnusedSegments(segmentsMetadataManager, overlordClient,
config);
+ } else {
+ if (killUnusedSegmentsDutyFromCustomGroups.size() > 1) {
+ log.warn(
Review Comment:
done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
