This is an automated email from the ASF dual-hosted git repository.
gian pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new 99065445f79 Docs: Mark use concurrent locks as GA (#17612)
99065445f79 is described below
commit 99065445f796b0f83afce466f9e8a3bf307a4b04
Author: Kashif Faraz <[email protected]>
AuthorDate: Thu Jan 9 21:08:14 2025 +0530
Docs: Mark use concurrent locks as GA (#17612)
* Mark use concurrent locks as GA
* Revert changes to web-console snapshot
* Update snapshots
---
docs/data-management/automatic-compaction.md | 2 +-
docs/ingestion/concurrent-append-replace.md | 29 ++++++++++++++--------
.../compaction-config-dialog.spec.tsx.snap | 8 +++---
.../compaction-config-dialog.tsx | 2 +-
.../src/views/load-data-view/load-data-view.tsx | 2 +-
5 files changed, 25 insertions(+), 18 deletions(-)
diff --git a/docs/data-management/automatic-compaction.md
b/docs/data-management/automatic-compaction.md
index cf129ea1ee2..62e209317f8 100644
--- a/docs/data-management/automatic-compaction.md
+++ b/docs/data-management/automatic-compaction.md
@@ -173,7 +173,7 @@ You can use concurrent append and replace to safely replace
the existing data in
To do this, you need to update your datasource to allow concurrent append and
replace tasks:
* If you're using the API, include the following `taskContext` property in
your API call: `"useConcurrentLocks": true`
-* If you're using the UI, enable **Use concurrent locks (experimental)** in
the **Compaction config** for your datasource.
+* If you're using the UI, enable **Use concurrent locks** in the **Compaction
config** for your datasource.
You'll also need to update your ingestion jobs for the datasource to include
the task context `"useConcurrentLocks": true`.
diff --git a/docs/ingestion/concurrent-append-replace.md
b/docs/ingestion/concurrent-append-replace.md
index 5468bc28c5c..607fbedfc48 100644
--- a/docs/ingestion/concurrent-append-replace.md
+++ b/docs/ingestion/concurrent-append-replace.md
@@ -22,21 +22,21 @@ title: Concurrent append and replace
~ under the License.
-->
-Concurrent append and replace safely replaces the existing data in an interval
of a datasource while new data is being appended to that interval. One of the
most common applications of this feature is appending new data (such as with
streaming ingestion) to an interval while compaction of that interval is
already in progress. Druid segments the data ingested during this time
dynamically. The subsequent compaction run segments the data into the
granularity you specified.
+Concurrent append and replace safely replaces the existing data in an interval
of a datasource while new data is being appended to that interval. One of the
most common applications of this feature is appending new data (such as with
streaming ingestion) to an interval while compaction of that interval is
already in progress. Druid partitions the data ingested during this time using
`dynamic` partitioning. The subsequent compaction run would partition the data
into the granularity you sp [...]
To set up concurrent append and replace, use the context flag
`useConcurrentLocks`. Druid will then determine the correct lock type for you,
either append or replace. Although you can set the type of lock manually, we
don't recommend it.
-## Update the compaction settings
+## Update compaction config to use concurrent locks
If you want to append data to a datasource while compaction is running, you
need to enable concurrent append and replace for the datasource by updating the
compaction settings.
-### Update the compaction settings with the UI
+### Update compaction config from the Druid web-console
-In the **Compaction config** for a datasource, enable **Use concurrent locks
(experimental)**.
+In the **Compaction config** for a datasource, enable **Use concurrent
locks**.
For details on accessing the compaction config in the UI, see [Enable
automatic compaction with the web
console](../data-management/automatic-compaction.md#manage-auto-compaction-using-the-web-console).
-### Update the compaction settings with the API
+### Update compaction config using REST API
Add the `taskContext` like you would any other automatic compaction setting
through the API:
@@ -51,17 +51,17 @@ curl --location --request POST
'http://localhost:8081/druid/coordinator/v1/confi
}'
```
-## Configure a task lock type for your ingestion job
+## Use concurrent locks in ingestion jobs
-You also need to configure the ingestion job to allow concurrent tasks.
+You also need to configure the ingestion job to allow concurrent locks.
You can provide the context parameter like any other parameter for ingestion
jobs through the API or the UI.
-### Add a task lock using the Druid console
+### Use concurrent locks in the Druid web-console
-As part of the **Load data** wizard for classic batch (JSON-based ingestion)
and streaming ingestion, enable the following config on the **Publish** step:
**Use concurrent locks (experimental)**.
+As part of the **Load data** wizard for classic batch (JSON-based) ingestion
and streaming ingestion, enable the following config on the **Publish** step:
**Use concurrent locks**.
-### Add the task lock through the API
+### Use concurrent locks in the REST APIs
Add the following JSON snippet to your supervisor or ingestion spec if you're
using the API:
@@ -70,7 +70,14 @@ Add the following JSON snippet to your supervisor or
ingestion spec if you're us
"useConcurrentLocks": true
}
```
-
+
+## Update Overlord properties to use concurrent locks for all ingestion and
compaction jobs
+
+Updating the compaction config and ingestion job for each data source can be
cumbersome if you have several data sources in your cluster. You can instead
set the following config in the `runtime.properties` of the Overlord service to
use concurrent locks across all ingestion and compaction jobs.
+
+```bash
+druid.indexer.task.default.context={"useConcurrentLocks":true}
+```
## Task lock types
diff --git
a/web-console/src/dialogs/compaction-config-dialog/__snapshots__/compaction-config-dialog.spec.tsx.snap
b/web-console/src/dialogs/compaction-config-dialog/__snapshots__/compaction-config-dialog.spec.tsx.snap
index 372153bfde0..b9b48cf6e58 100644
---
a/web-console/src/dialogs/compaction-config-dialog/__snapshots__/compaction-config-dialog.spec.tsx.snap
+++
b/web-console/src/dialogs/compaction-config-dialog/__snapshots__/compaction-config-dialog.spec.tsx.snap
@@ -372,7 +372,7 @@ exports[`CompactionConfigDialog matches snapshot with
compactionConfig (dynamic
>
<Blueprint5.Switch
checked={false}
- label="Use concurrent locks (experimental)"
+ label="Use concurrent locks"
onChange={[Function]}
/>
</Memo(FormGroupWithInfo)>
@@ -782,7 +782,7 @@ exports[`CompactionConfigDialog matches snapshot with
compactionConfig (hashed p
>
<Blueprint5.Switch
checked={false}
- label="Use concurrent locks (experimental)"
+ label="Use concurrent locks"
onChange={[Function]}
/>
</Memo(FormGroupWithInfo)>
@@ -1192,7 +1192,7 @@ exports[`CompactionConfigDialog matches snapshot with
compactionConfig (range pa
>
<Blueprint5.Switch
checked={false}
- label="Use concurrent locks (experimental)"
+ label="Use concurrent locks"
onChange={[Function]}
/>
</Memo(FormGroupWithInfo)>
@@ -1602,7 +1602,7 @@ exports[`CompactionConfigDialog matches snapshot without
compactionConfig 1`] =
>
<Blueprint5.Switch
checked={false}
- label="Use concurrent locks (experimental)"
+ label="Use concurrent locks"
onChange={[Function]}
/>
</Memo(FormGroupWithInfo)>
diff --git
a/web-console/src/dialogs/compaction-config-dialog/compaction-config-dialog.tsx
b/web-console/src/dialogs/compaction-config-dialog/compaction-config-dialog.tsx
index 800d88f78b2..887acb7a280 100644
---
a/web-console/src/dialogs/compaction-config-dialog/compaction-config-dialog.tsx
+++
b/web-console/src/dialogs/compaction-config-dialog/compaction-config-dialog.tsx
@@ -130,7 +130,7 @@ export const CompactionConfigDialog = React.memo(function
CompactionConfigDialog
}
>
<Switch
- label="Use concurrent locks (experimental)"
+ label="Use concurrent locks"
checked={Boolean(deepGet(currentConfig,
'taskContext.useConcurrentLocks'))}
onChange={() => {
setCurrentConfig(
diff --git a/web-console/src/views/load-data-view/load-data-view.tsx
b/web-console/src/views/load-data-view/load-data-view.tsx
index 3767a9852e9..25943929a18 100644
--- a/web-console/src/views/load-data-view/load-data-view.tsx
+++ b/web-console/src/views/load-data-view/load-data-view.tsx
@@ -3443,7 +3443,7 @@ export class LoadDataView extends
React.PureComponent<LoadDataViewProps, LoadDat
}
>
<Switch
- label="Use concurrent locks (experimental)"
+ label="Use concurrent locks"
checked={Boolean(deepGet(spec, 'context.useConcurrentLocks'))}
onChange={() => {
this.updateSpec(
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
