This is an automated email from the ASF dual-hosted git repository.
techdocsmith pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new b86ea4d5c40 [Docs] Improve druid.coordinator.kill.on description
(#17538)
b86ea4d5c40 is described below
commit b86ea4d5c40af56aff5f90cf9816f38d734696fc
Author: Katya Macedo <[email protected]>
AuthorDate: Thu Dec 12 18:58:38 2024 -0600
[Docs] Improve druid.coordinator.kill.on description (#17538)
* Docs: improve druid.coordinator.kill.on description
* Update docs/configuration/index.md
Co-authored-by: Charles Smith <[email protected]>
* Update description for durationToRetain
* Update docs/configuration/index.md
* Update after review
---------
Co-authored-by: Charles Smith <[email protected]>
---
docs/configuration/index.md | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/docs/configuration/index.md b/docs/configuration/index.md
index df14614d95e..75ffab25a60 100644
--- a/docs/configuration/index.md
+++ b/docs/configuration/index.md
@@ -890,9 +890,9 @@ These Coordinator static configurations can be defined in
the `coordinator/runti
|`druid.coordinator.startDelay`|The operation of the Coordinator works on the
assumption that it has an up-to-date view of the state of the world when it
runs, the current ZooKeeper interaction code, however, is written in a way that
doesn’t allow the Coordinator to know for a fact that it’s done loading the
current state of the world. This delay is a hack to give it enough time to
believe that it has all the data.|`PT300S`|
|`druid.coordinator.load.timeout`|The timeout duration for when the
Coordinator assigns a segment to a Historical service.|`PT15M`|
|`druid.coordinator.kill.pendingSegments.on`|Boolean flag for whether or not
the Coordinator clean up old entries in the `pendingSegments` table of metadata
store. If set to true, Coordinator will check the created time of most recently
complete task. If it doesn't exist, it finds the created time of the earliest
running/pending/waiting tasks. Once the created time is found, then for all
datasources not in the `killPendingSegmentsSkipList` (see [Dynamic
configuration](#dynamic-configurat [...]
-|`druid.coordinator.kill.on`|Boolean flag for whether or not the Coordinator
should submit kill task for unused segments, that is, permanently delete them
from metadata store and deep storage. If set to true, then for all whitelisted
datasources (or optionally all), Coordinator will submit tasks periodically
based on `period` specified. A whitelist can be set via dynamic configuration
`killDataSourceWhitelist` described later.<br /><br />When
`druid.coordinator.kill.on` is true, segments [...]
+|`druid.coordinator.kill.on`|Boolean flag to enable the Coordinator to submit
a kill task for unused segments and delete them permanently from the metadata
store and deep storage.|false|
|`druid.coordinator.kill.period`| The frequency of sending kill tasks to the
indexing service. The value must be greater than or equal to
`druid.coordinator.period.indexingPeriod`. Only applies if kill is turned
on.|Same as `druid.coordinator.period.indexingPeriod`|
-|`druid.coordinator.kill.durationToRetain`|Only applies if you set
`druid.coordinator.kill.on` to `true`. This value is ignored if
`druid.coordinator.kill.ignoreDurationToRetain` is `true`. Valid configurations
must be a ISO8601 period. Druid will not kill unused segments whose interval
end date is beyond `now - durationToRetain`. `durationToRetain` can be a
negative ISO8601 period, which would result in `now - durationToRetain` to be
in the future.<br /><br />Note that the `durationToRe [...]
+|`druid.coordinator.kill.durationToRetain`|Duration, in ISO 8601 format,
relative to the current time that identifies the data interval of segments to
retain. When `druid.coordinator.kill.on` is true, any segment with a data
interval ending before `now - durationToRetain` is eligible for permanent
deletion. For example, if `durationToRetain` is set to `P90D`, unused segments
with time intervals ending 90 days in the past are eligible for deletion. If
`durationToRetain` is set to a negati [...]
|`druid.coordinator.kill.ignoreDurationToRetain`|A way to override
`druid.coordinator.kill.durationToRetain` and tell the coordinator that you do
not care about the end date of unused segment intervals when it comes to
killing them. If true, the coordinator considers all unused segments as
eligible to be killed.|false|
|`druid.coordinator.kill.bufferPeriod`|The amount of time that a segment must
be unused before it is able to be permanently removed from metadata and deep
storage. This can serve as a buffer period to prevent data loss if data ends up
being needed after being marked unused.|`P30D`|
|`druid.coordinator.kill.maxSegments`|The number of unused segments to kill
per kill task. This number must be greater than 0. This only applies when
`druid.coordinator.kill.on=true`.|100|
@@ -957,7 +957,7 @@ The following table shows the dynamic configuration
properties for the Coordinat
|`replicantLifetime`|The maximum number of Coordinator runs for which a
segment can wait in the load queue of a Historical before Druid raises an
alert.|15|
|`replicationThrottleLimit`|The maximum number of segment replicas that can be
assigned to a historical tier in a single Coordinator run. This property
prevents Historical services from becoming overwhelmed when loading extra
replicas of segments that are already available in the cluster.|500|
|`balancerComputeThreads`|Thread pool size for computing moving cost of
segments during segment balancing. Consider increasing this if you have a lot
of segments and moving segments begins to stall.|`num_cores` / 2|
-|`killDataSourceWhitelist`|List of specific data sources for which kill tasks
are sent if property `druid.coordinator.kill.on` is true. This can be a list of
comma-separated data source names or a JSON array.|none|
+|`killDataSourceWhitelist`|List of specific data sources for which kill tasks
can be issued if `druid.coordinator.kill.on` is true. It can be a
comma-separated list of data source names or a JSON array. If
`killDataSourceWhitelist` is empty, the Coordinator issues kill tasks for all
data sources.|none|
|`killTaskSlotRatio`|Ratio of total available task slots, including
autoscaling if applicable that will be allowed for kill tasks. This value must
be between 0 and 1. Only applicable for kill tasks that are spawned
automatically by the coordinator's auto kill duty, which is enabled when
`druid.coordinator.kill.on` is true.|0.1|
|`maxKillTaskSlots`|Maximum number of tasks that will be allowed for kill
tasks. This limit only applies for kill tasks that are spawned automatically by
the coordinator's auto kill duty, which is enabled when
`druid.coordinator.kill.on` is true.|`Integer.MAX_VALUE` - no limit|
|`killPendingSegmentsSkipList`|List of data sources for which pendingSegments
are _NOT_ cleaned up if property `druid.coordinator.kill.pendingSegments.on` is
true. This can be a list of comma-separated data sources or a JSON array.|none|
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
