This is an automated email from the ASF dual-hosted git repository.
yihua pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 0c77d045da [HUDI-4010] Improve docs on DynamoDB-based lock provider
(#6752)
0c77d045da is described below
commit 0c77d045dabea5d6f821cb3f91d60877d7574fe3
Author: Y Ethan Guo <[email protected]>
AuthorDate: Fri Sep 23 18:14:40 2022 -0700
[HUDI-4010] Improve docs on DynamoDB-based lock provider (#6752)
---
website/docs/concurrency_control.md | 25 +++++++++++++++------
.../version-0.10.0/concurrency_control.md | 25 +++++++++++++++------
.../version-0.10.1/concurrency_control.md | 26 ++++++++++++++++------
.../version-0.11.0/concurrency_control.md | 26 ++++++++++++++++------
.../version-0.11.1/concurrency_control.md | 26 ++++++++++++++++------
.../version-0.12.0/concurrency_control.md | 25 +++++++++++++++------
6 files changed, 111 insertions(+), 42 deletions(-)
diff --git a/website/docs/concurrency_control.md
b/website/docs/concurrency_control.md
index 25a523ee7c..a784b705f9 100644
--- a/website/docs/concurrency_control.md
+++ b/website/docs/concurrency_control.md
@@ -71,16 +71,27 @@ hoodie.write.lock.hivemetastore.table
**`Amazon DynamoDB`** based lock provider
-Amazon DynamoDB based lock provides a simple way to support multi writing
across different clusters
+Amazon DynamoDB based lock provides a simple way to support multi writing
across different clusters. You can refer to the
+[DynamoDB based Locks
Configurations](https://hudi.apache.org/docs/configurations#DynamoDB-based-Locks-Configurations)
+section for the details of each related configuration knob.
```
hoodie.write.lock.provider=org.apache.hudi.aws.transaction.lock.DynamoDBBasedLockProvider
-hoodie.write.lock.dynamodb.table
-hoodie.write.lock.dynamodb.partition_key
-hoodie.write.lock.dynamodb.region
-hoodie.write.lock.dynamodb.endpoint_url
-hoodie.write.lock.dynamodb.billing_mode
-```
+hoodie.write.lock.dynamodb.table (required)
+hoodie.write.lock.dynamodb.partition_key (optional)
+hoodie.write.lock.dynamodb.region (optional)
+hoodie.write.lock.dynamodb.endpoint_url (optional)
+hoodie.write.lock.dynamodb.billing_mode (optional)
+```
+
+When using the DynamoDB-based lock provider, the name of the DynamoDB table
acting as the lock table for Hudi is
+specified by the config `hoodie.write.lock.dynamodb.table`. This DynamoDB
table is automatically created by Hudi, so you
+don't have to create the table yourself. If you want to use an existing
DynamoDB table, make sure that an attribute with
+the name `key` is present in the table. The `key` attribute should be the
partition key of the DynamoDB table. The
+config `hoodie.write.lock.dynamodb.partition_key` specifies the value to put
for the `key` attribute (not the attribute
+name), which is used for the lock on the same table. By default,
`hoodie.write.lock.dynamodb.partition_key` is set to
+the table name, so that multiple writers writing to the same table share the
same lock. If you customize the name, make
+sure it's the same across multiple writers.
Also, to set up the credentials for accessing AWS resources, customers can
pass the following props to Hudi jobs:
```
diff --git a/website/versioned_docs/version-0.10.0/concurrency_control.md
b/website/versioned_docs/version-0.10.0/concurrency_control.md
index fe38f102cd..4602ea5df4 100644
--- a/website/versioned_docs/version-0.10.0/concurrency_control.md
+++ b/website/versioned_docs/version-0.10.0/concurrency_control.md
@@ -71,16 +71,27 @@ hoodie.write.lock.hivemetastore.table
**`Amazon DynamoDB`** based lock provider
-Amazon DynamoDB based lock provides a simple way to support multi writing
across different clusters
+Amazon DynamoDB based lock provides a simple way to support multi writing
across different clusters. You can refer to the
+[DynamoDB based Locks
Configurations](https://hudi.apache.org/docs/configurations#DynamoDB-based-Locks-Configurations)
+section for the details of each related configuration knob.
```
hoodie.write.lock.provider=org.apache.hudi.aws.transaction.lock.DynamoDBBasedLockProvider
-hoodie.write.lock.dynamodb.table
-hoodie.write.lock.dynamodb.partition_key
-hoodie.write.lock.dynamodb.region
-hoodie.write.lock.dynamodb.endpoint_url
-hoodie.write.lock.dynamodb.billing_mode
-```
+hoodie.write.lock.dynamodb.table (required)
+hoodie.write.lock.dynamodb.partition_key (optional)
+hoodie.write.lock.dynamodb.region (optional)
+hoodie.write.lock.dynamodb.endpoint_url (optional)
+hoodie.write.lock.dynamodb.billing_mode (optional)
+```
+
+When using the DynamoDB-based lock provider, the name of the DynamoDB table
acting as the lock table for Hudi is
+specified by the config `hoodie.write.lock.dynamodb.table`. This DynamoDB
table is automatically created by Hudi, so you
+don't have to create the table yourself. If you want to use an existing
DynamoDB table, make sure that an attribute with
+the name `key` is present in the table. The `key` attribute should be the
partition key of the DynamoDB table. The
+config `hoodie.write.lock.dynamodb.partition_key` specifies the value to put
for the `key` attribute (not the attribute
+name), which is used for the lock on the same table. By default,
`hoodie.write.lock.dynamodb.partition_key` is set to
+the table name, so that multiple writers writing to the same table share the
same lock. If you customize the name, make
+sure it's the same across multiple writers.
Also, to set up the credentials for accessing AWS resources, customers can
pass the following props to Hudi jobs:
```
diff --git a/website/versioned_docs/version-0.10.1/concurrency_control.md
b/website/versioned_docs/version-0.10.1/concurrency_control.md
index 6377c762bd..6ea34baa9a 100644
--- a/website/versioned_docs/version-0.10.1/concurrency_control.md
+++ b/website/versioned_docs/version-0.10.1/concurrency_control.md
@@ -70,16 +70,28 @@ hoodie.write.lock.hivemetastore.table
`The HiveMetastore URI's are picked up from the hadoop configuration file
loaded during runtime.`
**`Amazon DynamoDB`** based lock provider
-Amazon DynamoDB based lock provides a simple way to support multi writing
across different clusters
+
+Amazon DynamoDB based lock provides a simple way to support multi writing
across different clusters. You can refer to the
+[DynamoDB based Locks
Configurations](https://hudi.apache.org/docs/configurations#DynamoDB-based-Locks-Configurations)
+section for the details of each related configuration knob.
```
hoodie.write.lock.provider=org.apache.hudi.aws.transaction.lock.DynamoDBBasedLockProvider
-hoodie.write.lock.dynamodb.table
-hoodie.write.lock.dynamodb.partition_key
-hoodie.write.lock.dynamodb.region
-hoodie.write.lock.dynamodb.endpoint_url
-hoodie.write.lock.dynamodb.billing_mode
-```
+hoodie.write.lock.dynamodb.table (required)
+hoodie.write.lock.dynamodb.partition_key (optional)
+hoodie.write.lock.dynamodb.region (optional)
+hoodie.write.lock.dynamodb.endpoint_url (optional)
+hoodie.write.lock.dynamodb.billing_mode (optional)
+```
+
+When using the DynamoDB-based lock provider, the name of the DynamoDB table
acting as the lock table for Hudi is
+specified by the config `hoodie.write.lock.dynamodb.table`. This DynamoDB
table is automatically created by Hudi, so you
+don't have to create the table yourself. If you want to use an existing
DynamoDB table, make sure that an attribute with
+the name `key` is present in the table. The `key` attribute should be the
partition key of the DynamoDB table. The
+config `hoodie.write.lock.dynamodb.partition_key` specifies the value to put
for the `key` attribute (not the attribute
+name), which is used for the lock on the same table. By default,
`hoodie.write.lock.dynamodb.partition_key` is set to
+the table name, so that multiple writers writing to the same table share the
same lock. If you customize the name, make
+sure it's the same across multiple writers.
Also, to set up the credentials for accessing AWS resources, customers can
pass the following props to Hudi jobs:
```
diff --git a/website/versioned_docs/version-0.11.0/concurrency_control.md
b/website/versioned_docs/version-0.11.0/concurrency_control.md
index aec626f9d2..daef609a72 100644
--- a/website/versioned_docs/version-0.11.0/concurrency_control.md
+++ b/website/versioned_docs/version-0.11.0/concurrency_control.md
@@ -70,16 +70,28 @@ hoodie.write.lock.hivemetastore.table
`The HiveMetastore URI's are picked up from the hadoop configuration file
loaded during runtime.`
**`Amazon DynamoDB`** based lock provider
-Amazon DynamoDB based lock provides a simple way to support multi writing
across different clusters
+
+Amazon DynamoDB based lock provides a simple way to support multi writing
across different clusters. You can refer to the
+[DynamoDB based Locks
Configurations](https://hudi.apache.org/docs/configurations#DynamoDB-based-Locks-Configurations)
+section for the details of each related configuration knob.
```
hoodie.write.lock.provider=org.apache.hudi.aws.transaction.lock.DynamoDBBasedLockProvider
-hoodie.write.lock.dynamodb.table
-hoodie.write.lock.dynamodb.partition_key
-hoodie.write.lock.dynamodb.region
-hoodie.write.lock.dynamodb.endpoint_url
-hoodie.write.lock.dynamodb.billing_mode
-```
+hoodie.write.lock.dynamodb.table (required)
+hoodie.write.lock.dynamodb.partition_key (optional)
+hoodie.write.lock.dynamodb.region (optional)
+hoodie.write.lock.dynamodb.endpoint_url (optional)
+hoodie.write.lock.dynamodb.billing_mode (optional)
+```
+
+When using the DynamoDB-based lock provider, the name of the DynamoDB table
acting as the lock table for Hudi is
+specified by the config `hoodie.write.lock.dynamodb.table`. This DynamoDB
table is automatically created by Hudi, so you
+don't have to create the table yourself. If you want to use an existing
DynamoDB table, make sure that an attribute with
+the name `key` is present in the table. The `key` attribute should be the
partition key of the DynamoDB table. The
+config `hoodie.write.lock.dynamodb.partition_key` specifies the value to put
for the `key` attribute (not the attribute
+name), which is used for the lock on the same table. By default,
`hoodie.write.lock.dynamodb.partition_key` is set to
+the table name, so that multiple writers writing to the same table share the
same lock. If you customize the name, make
+sure it's the same across multiple writers.
Also, to set up the credentials for accessing AWS resources, customers can
pass the following props to Hudi jobs:
```
diff --git a/website/versioned_docs/version-0.11.1/concurrency_control.md
b/website/versioned_docs/version-0.11.1/concurrency_control.md
index aec626f9d2..daef609a72 100644
--- a/website/versioned_docs/version-0.11.1/concurrency_control.md
+++ b/website/versioned_docs/version-0.11.1/concurrency_control.md
@@ -70,16 +70,28 @@ hoodie.write.lock.hivemetastore.table
`The HiveMetastore URI's are picked up from the hadoop configuration file
loaded during runtime.`
**`Amazon DynamoDB`** based lock provider
-Amazon DynamoDB based lock provides a simple way to support multi writing
across different clusters
+
+Amazon DynamoDB based lock provides a simple way to support multi writing
across different clusters. You can refer to the
+[DynamoDB based Locks
Configurations](https://hudi.apache.org/docs/configurations#DynamoDB-based-Locks-Configurations)
+section for the details of each related configuration knob.
```
hoodie.write.lock.provider=org.apache.hudi.aws.transaction.lock.DynamoDBBasedLockProvider
-hoodie.write.lock.dynamodb.table
-hoodie.write.lock.dynamodb.partition_key
-hoodie.write.lock.dynamodb.region
-hoodie.write.lock.dynamodb.endpoint_url
-hoodie.write.lock.dynamodb.billing_mode
-```
+hoodie.write.lock.dynamodb.table (required)
+hoodie.write.lock.dynamodb.partition_key (optional)
+hoodie.write.lock.dynamodb.region (optional)
+hoodie.write.lock.dynamodb.endpoint_url (optional)
+hoodie.write.lock.dynamodb.billing_mode (optional)
+```
+
+When using the DynamoDB-based lock provider, the name of the DynamoDB table
acting as the lock table for Hudi is
+specified by the config `hoodie.write.lock.dynamodb.table`. This DynamoDB
table is automatically created by Hudi, so you
+don't have to create the table yourself. If you want to use an existing
DynamoDB table, make sure that an attribute with
+the name `key` is present in the table. The `key` attribute should be the
partition key of the DynamoDB table. The
+config `hoodie.write.lock.dynamodb.partition_key` specifies the value to put
for the `key` attribute (not the attribute
+name), which is used for the lock on the same table. By default,
`hoodie.write.lock.dynamodb.partition_key` is set to
+the table name, so that multiple writers writing to the same table share the
same lock. If you customize the name, make
+sure it's the same across multiple writers.
Also, to set up the credentials for accessing AWS resources, customers can
pass the following props to Hudi jobs:
```
diff --git a/website/versioned_docs/version-0.12.0/concurrency_control.md
b/website/versioned_docs/version-0.12.0/concurrency_control.md
index 25a523ee7c..a784b705f9 100644
--- a/website/versioned_docs/version-0.12.0/concurrency_control.md
+++ b/website/versioned_docs/version-0.12.0/concurrency_control.md
@@ -71,16 +71,27 @@ hoodie.write.lock.hivemetastore.table
**`Amazon DynamoDB`** based lock provider
-Amazon DynamoDB based lock provides a simple way to support multi writing
across different clusters
+Amazon DynamoDB based lock provides a simple way to support multi writing
across different clusters. You can refer to the
+[DynamoDB based Locks
Configurations](https://hudi.apache.org/docs/configurations#DynamoDB-based-Locks-Configurations)
+section for the details of each related configuration knob.
```
hoodie.write.lock.provider=org.apache.hudi.aws.transaction.lock.DynamoDBBasedLockProvider
-hoodie.write.lock.dynamodb.table
-hoodie.write.lock.dynamodb.partition_key
-hoodie.write.lock.dynamodb.region
-hoodie.write.lock.dynamodb.endpoint_url
-hoodie.write.lock.dynamodb.billing_mode
-```
+hoodie.write.lock.dynamodb.table (required)
+hoodie.write.lock.dynamodb.partition_key (optional)
+hoodie.write.lock.dynamodb.region (optional)
+hoodie.write.lock.dynamodb.endpoint_url (optional)
+hoodie.write.lock.dynamodb.billing_mode (optional)
+```
+
+When using the DynamoDB-based lock provider, the name of the DynamoDB table
acting as the lock table for Hudi is
+specified by the config `hoodie.write.lock.dynamodb.table`. This DynamoDB
table is automatically created by Hudi, so you
+don't have to create the table yourself. If you want to use an existing
DynamoDB table, make sure that an attribute with
+the name `key` is present in the table. The `key` attribute should be the
partition key of the DynamoDB table. The
+config `hoodie.write.lock.dynamodb.partition_key` specifies the value to put
for the `key` attribute (not the attribute
+name), which is used for the lock on the same table. By default,
`hoodie.write.lock.dynamodb.partition_key` is set to
+the table name, so that multiple writers writing to the same table share the
same lock. If you customize the name, make
+sure it's the same across multiple writers.
Also, to set up the credentials for accessing AWS resources, customers can
pass the following props to Hudi jobs:
```
