gtwuser commented on issue #6925: URL: https://github.com/apache/hudi/issues/6925#issuecomment-1303947778
> oh btw, one issue I see w/ you configs is, you are not setting the configs properly for multi-writer. https://hudi.apache.org/docs/concurrency_control#enabling-multi-writing > > ``` > hoodie.write.concurrency.mode=optimistic_concurrency_control > hoodie.cleaner.policy.failed.writes=LAZY > hoodie.write.lock.provider=<lock-provider-classname> > ``` > > you were setting the last one, but not the first one. > > Also, can you first try out using just 1 thread along w/ setting above configs. we can go from there if basic set up is not working. Sorry for the delay. But now after adding above changes we are getting this error now : `Caused by: java.lang.ClassNotFoundException: org.apache.hudi.aws.transaction.lock.DynamoDBBasedLockProvider` [link to full stack of error ](https://gist.github.com/gtwuser/cd23f4851aab035774717f08ce1fbfc2) I feel we are close to fix it, so please let me know whats that im missing now. I checked the classpath of the `DynamoDBBasedLockProvider` it seems correct. **config used for locking**: ```bash 'className': 'org.apache.hudi', 'hoodie.datasource.hive_sync.use_jdbc': 'false', 'hoodie.datasource.write.precombine.field': 'payload.recordedAt', 'hoodie.datasource.write.recordkey.field': 'metadata.msgID,metadata.topic', 'hoodie.write.lock.provider': 'org.apache.hudi.aws.transaction.lock.DynamoDBBasedLockProvider', 'hoodie.write.lock.dynamodb.table': 'hudi_db_lock', 'hoodie.write.lock.dynamodb.partition_key':'hudi_db_lock', 'hoodie.write.concurrency.mode': 'optimistic_concurrency_control', 'hoodie.write.lock.dynamodb.endpoint_url':'dynamodb.us-east-1.amazonaws.com', 'hoodie.cleaner.policy.failed.writes': 'LAZY', -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
