This is an automated email from the ASF dual-hosted git repository. wenningd pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git
commit c47f323765b1abb5196b16d0224b5baa0940bc7e Author: atharvai <[email protected]> AuthorDate: Thu Jul 21 11:11:09 2022 +0100 update dynamodb lockk provider docs to include iam and additional dependencies --- website/docs/concurrency_control.md | 33 +++++++++++++++++++++++++++++++++ website/docs/configurations.md | 2 +- 2 files changed, 34 insertions(+), 1 deletion(-) diff --git a/website/docs/concurrency_control.md b/website/docs/concurrency_control.md index e71cb4a8f2..689b7632f7 100644 --- a/website/docs/concurrency_control.md +++ b/website/docs/concurrency_control.md @@ -78,7 +78,10 @@ hoodie.write.lock.provider=org.apache.hudi.aws.transaction.lock.DynamoDBBasedLoc hoodie.write.lock.dynamodb.table hoodie.write.lock.dynamodb.partition_key hoodie.write.lock.dynamodb.region +hoodie.write.lock.dynamodb.endpoint_url +hoodie.write.lock.dynamodb.billing_mode ``` + Also, to set up the credentials for accessing AWS resources, customers can pass the following props to Hudi jobs: ``` hoodie.aws.access.key @@ -87,6 +90,36 @@ hoodie.aws.session.token ``` If not configured, Hudi falls back to use [DefaultAWSCredentialsProviderChain](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/DefaultAWSCredentialsProviderChain.html). + +IAM policy for your service instance will need to add the following permissions: + +```json +{ + "Sid":"DynamoDBLocksTable", + "Effect": "Allow", + "Action": [ + "dynamodb:CreateTable", + "dynamodb:DeleteItem", + "dynamodb:DescribeTable", + "dynamodb:GetItem", + "dynamodb:PutItem", + "dynamodb:Scan", + "dynamodb:UpdateItem" + ], + "Resource": "arn:${Partition}:dynamodb:${Region}:${Account}:table/${TableName}" +} +``` +- `TableName` : same as `hoodie.write.lock.dynamodb.partition_key` +- `Region`: same as `hoodie.write.lock.dynamodb.region` + +AWS SDK dependencies are not bundled with Hudi from v0.10.x and will need to be added to your classpath. +Add the following Maven packages (check the latest versions at time of install): +``` +com.amazonaws:dynamodb-lock-client +com.amazonaws:aws-java-sdk-dynamodb +com.amazonaws:aws-java-sdk-core +``` + ## Datasource Writer The `hudi-spark` module offers the DataSource API to write (and read) a Spark DataFrame into a Hudi table. diff --git a/website/docs/configurations.md b/website/docs/configurations.md index b92e40f06c..6dcb76179b 100644 --- a/website/docs/configurations.md +++ b/website/docs/configurations.md @@ -1696,7 +1696,7 @@ Configs that control DynamoDB based locking mechanisms required for concurrency `Config Class`: org.apache.hudi.config.DynamoDbBasedLockConfig<br></br> > #### hoodie.write.lock.dynamodb.billing_mode -> For DynamoDB based lock provider, by default it is PAY_PER_REQUEST mode<br></br> +> For DynamoDB based lock provider, by default it is PAY_PER_REQUEST mode. Alternative is PROVISIONED<br></br> > **Default Value**: PAY_PER_REQUEST (Optional)<br></br> > `Config Param: DYNAMODB_LOCK_BILLING_MODE`<br></br> > `Since Version: 0.10.0`<br></br>
