This is an automated email from the ASF dual-hosted git repository.
gian pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new d29343cbe3 Disable autokill of segments by default. (#12693)
d29343cbe3 is described below
commit d29343cbe36357356ec65a2f56087d5b39e92148
Author: Gian Merlino <[email protected]>
AuthorDate: Thu Jun 23 17:17:11 2022 -0700
Disable autokill of segments by default. (#12693)
Also add clarifying commentary to the documentation about how
durationToRetain works.
---
docs/configuration/index.md | 4 ++--
services/src/main/java/org/apache/druid/cli/CliCoordinator.java | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/docs/configuration/index.md b/docs/configuration/index.md
index 82d6cdf4eb..a27e390a1a 100644
--- a/docs/configuration/index.md
+++ b/docs/configuration/index.md
@@ -810,9 +810,9 @@ These Coordinator static configurations can be defined in
the `coordinator/runti
|`druid.coordinator.startDelay`|The operation of the Coordinator works on the
assumption that it has an up-to-date view of the state of the world when it
runs, the current ZK interaction code, however, is written in a way that
doesn’t allow the Coordinator to know for a fact that it’s done loading the
current state of the world. This delay is a hack to give it enough time to
believe that it has all the data.|PT300S|
|`druid.coordinator.load.timeout`|The timeout duration for when the
Coordinator assigns a segment to a Historical process.|PT15M|
|`druid.coordinator.kill.pendingSegments.on`|Boolean flag for whether or not
the Coordinator clean up old entries in the `pendingSegments` table of metadata
store. If set to true, Coordinator will check the created time of most recently
complete task. If it doesn't exist, it finds the created time of the earliest
running/pending/waiting tasks. Once the created time is found, then for all
dataSources not in the `killPendingSegmentsSkipList` (see [Dynamic
configuration](#dynamic-configurat [...]
-|`druid.coordinator.kill.on`|Boolean flag for whether or not the Coordinator
should submit kill task for unused segments, that is, hard delete them from
metadata store and deep storage. If set to true, then for all whitelisted
dataSources (or optionally all), Coordinator will submit tasks periodically
based on `period` specified. These kill tasks will delete all unused segments
except for the last `durationToRetain` period. A whitelist can be set via
dynamic configuration `killDataSource [...]
+|`druid.coordinator.kill.on`|Boolean flag for whether or not the Coordinator
should submit kill task for unused segments, that is, permanently delete them
from metadata store and deep storage. If set to true, then for all whitelisted
dataSources (or optionally all), Coordinator will submit tasks periodically
based on `period` specified. A whitelist can be set via dynamic configuration
`killDataSourceWhitelist` described later.<br /><br />When
`druid.coordinator.kill.on` is true, segments [...]
|`druid.coordinator.kill.period`|How often to send kill tasks to the indexing
service. Value must be greater than `druid.coordinator.period.indexingPeriod`.
Only applies if kill is turned on.|P1D (1 Day)|
-|`druid.coordinator.kill.durationToRetain`|Only applies if you set
`druid.coordinator.kill.on` to `true`. This value is ignored if
`druid.coordinator.kill.ignoreDurationToRetain` is `true`. Valid configurations
must be a ISO8601 period. Druid will not kill unused segments whose interval
end date is beyond `now - durationToRetain`. `durationToRetain` can be a
negative ISO8601 period, which would result in `now - durationToRetain` to be
in the future.|`P90D`|
+|`druid.coordinator.kill.durationToRetain`|Only applies if you set
`druid.coordinator.kill.on` to `true`. This value is ignored if
`druid.coordinator.kill.ignoreDurationToRetain` is `true`. Valid configurations
must be a ISO8601 period. Druid will not kill unused segments whose interval
end date is beyond `now - durationToRetain`. `durationToRetain` can be a
negative ISO8601 period, which would result in `now - durationToRetain` to be
in the future.<br /><br />Note that the `durationToRe [...]
|`druid.coordinator.kill.ignoreDurationToRetain`|A way to override
`druid.coordinator.kill.durationToRetain` and tell the coordinator that you do
not care about the end date of unused segment intervals when it comes to
killing them. If true, the coordinator considers all unused segments as
eligible to be killed.|false|
|`druid.coordinator.kill.maxSegments`|Kill at most n unused segments per kill
task submission, must be greater than 0. Only applies and MUST be specified if
kill is turned on.|100|
|`druid.coordinator.balancer.strategy`|Specify the type of balancing strategy
for the coordinator to use to distribute segments among the historicals.
`cachingCost` is logically equivalent to `cost` but is more CPU-efficient on
large clusters. `diskNormalized` weights the costs according to the servers'
disk usage ratios - there are known issues with this strategy distributing
segments unevenly across the cluster. `random` distributes segments among
services randomly.|`cost`|
diff --git a/services/src/main/java/org/apache/druid/cli/CliCoordinator.java
b/services/src/main/java/org/apache/druid/cli/CliCoordinator.java
index ea363696b4..841034e2be 100644
--- a/services/src/main/java/org/apache/druid/cli/CliCoordinator.java
+++ b/services/src/main/java/org/apache/druid/cli/CliCoordinator.java
@@ -264,7 +264,7 @@ public class CliCoordinator extends ServerRunnable
}
conditionalIndexingServiceDutyMultibind.addConditionBinding(
"druid.coordinator.kill.on",
- "true",
+ "false",
Predicates.equalTo("true"),
KillUnusedSegments.class
);
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
