This is an automated email from the ASF dual-hosted git repository.
surekha pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-druid.git
The following commit(s) were added to refs/heads/master by this push:
new eeae5d9 Add a warning about experimental segment locking (#8301)
eeae5d9 is described below
commit eeae5d936501d5a9f5f7d4a555f363f97c002119
Author: Jihoon Son <jihoon...@apache.org>
AuthorDate: Thu Aug 15 16:07:59 2019 -0700
Add a warning about experimental segment locking (#8301)
* Add a warning about experimental segment locking
* fix typo
---
docs/content/configuration/index.md | 2 +-
docs/content/ingestion/locking-and-priority.md | 8 ++++++--
2 files changed, 7 insertions(+), 3 deletions(-)
diff --git a/docs/content/configuration/index.md
b/docs/content/configuration/index.md
index 134ff18..4812d66 100644
--- a/docs/content/configuration/index.md
+++ b/docs/content/configuration/index.md
@@ -907,7 +907,7 @@ These Overlord static configurations can be defined in the
`overlord/runtime.pro
|`druid.indexer.runner.type`|Choices "local" or "remote". Indicates whether
tasks should be run locally or in a distributed environment. Experimental task
runner "httpRemote" is also available which is same as "remote" but uses HTTP
to interact with Middle Manaters instead of Zookeeper.|local|
|`druid.indexer.storage.type`|Choices are "local" or "metadata". Indicates
whether incoming tasks should be stored locally (in heap) or in metadata
storage. Storing incoming tasks in metadata storage allows for tasks to be
resumed if the Overlord should fail.|local|
|`druid.indexer.storage.recentlyFinishedThreshold`|A duration of time to store
task results.|PT24H|
-|`druid.indexer.tasklock.forceTimeChunkLock`|If set, all tasks are enforced to
use time chunk lock. If not set, each task automatically chooses a lock type to
use. This configuration can be overwritten by setting `forceTimeChunkLock` in
the [task context](../ingestion/locking-and-priority.html#task-context). See
[Task Locking & Priority](../ingestion/locking-and-priority.html) for more
details about locking in tasks.|true|
+|`druid.indexer.tasklock.forceTimeChunkLock`|_**Setting this to false is still
experimental**_<br/> If set, all tasks are enforced to use time chunk lock. If
not set, each task automatically chooses a lock type to use. This configuration
can be overwritten by setting `forceTimeChunkLock` in the [task
context](../ingestion/locking-and-priority.html#task-context). See [Task
Locking & Priority](../ingestion/locking-and-priority.html) for more details
about locking in tasks.|true|
|`druid.indexer.queue.maxSize`|Maximum number of active tasks at one
time.|Integer.MAX_VALUE|
|`druid.indexer.queue.startDelay`|Sleep this long before starting Overlord
queue management. This can be useful to give a cluster time to re-orient itself
after e.g. a widespread network issue.|PT1M|
|`druid.indexer.queue.restartDelay`|Sleep this long when Overlord queue
management throws an exception before trying again.|PT30S|
diff --git a/docs/content/ingestion/locking-and-priority.md
b/docs/content/ingestion/locking-and-priority.md
index f0ba3e1..eaeab99 100644
--- a/docs/content/ingestion/locking-and-priority.md
+++ b/docs/content/ingestion/locking-and-priority.md
@@ -40,7 +40,7 @@ are used to determine the overshadow relation between
segments as seen below.
A segment `s1` overshadows another `s2` if
-- `s1` has a higher major version than `s2`.
+- `s1` has a higher major version than `s2`, or
- `s1` has the same major version and a higher minor version than `s2`.
Here are some examples.
@@ -71,6 +71,10 @@ For example, a Kafka indexing task and a compaction task can
always write segmen
The reason for this is because a Kafka indexing task always appends new
segments, while a compaction task always overwrites existing segments.
The segments created with the segment locking have the _same_ major version
and a _higher_ minor version.
+<div class="note caution">
+The segment locking is still experimental. It could have unknown bugs which
potentially lead to incorrect query results.
+</div>
+
To enable segment locking, you may need to set `forceTimeChunkLock` to `false`
in the [task context](#task-context).
Once `forceTimeChunkLock` is unset, the task will choose a proper lock type to
use automatically.
Please note that segment lock is not always available. The most common use
case where time chunk lock is enforced is
@@ -126,7 +130,7 @@ The task context is used for various individual task
configuration. The followin
|property|default|description|
|--------|-------|-----------|
|taskLockTimeout|300000|task lock timeout in millisecond. For more details,
see [Locking](#locking).|
-|forceTimeChunkLock|true|Force to always use time chunk lock. If not set, each
task automatically chooses a lock type to use. If this set, it will overwrite
the `druid.indexer.tasklock.forceTimeChunkLock` [configuration for the
overlord](../configuration/index.html#overlord-operations). See
[Locking](#locking) for more details.|
+|forceTimeChunkLock|true|_**Setting this to false is still
experimental**_<br/> Force to always use time chunk lock. If not set, each task
automatically chooses a lock type to use. If this is set, it will overwrite the
`druid.indexer.tasklock.forceTimeChunkLock` [configuration for the
overlord](../configuration/index.html#overlord-operations). See
[Locking](#locking) for more details.|
|priority|Different based on task types. See [Priority](#priority).|Task
priority|
<div class="note caution">
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
