nochimow opened a new issue #4456: URL: https://github.com/apache/hudi/issues/4456
Hello, I'm currently trying the multiwriter feature using the dynamoDB lock. I followed all the steps documented on the https://hudi.apache.org/docs/concurrency_control/ and also the hoodie.write.lock.dynamodb.billing_mode=PAY_PER_REQUEST config, thanks to @bhasudha advice on slack. After that i ended with the following error: org.apache.hudi.exception.HoodieLockException: Unable to acquire lock, lock object null This error happens when trying to write in a existing Hudi table. Since there is no details on how the DynamoDB table must me created on the documentation, i created one simple DynamoDB with a String field like partition. On slack, there is also other users with the same problem, also AWS Glue users **Environment Description** * Hudi version : 0.10 * Spark version : AWS Glue 2.0 * Storage (HDFS/S3/GCS..) : S3 **Stacktrace** Caused by: org.apache.hudi.exception.HoodieException: Unable to acquire lock, lock object null at org.apache.hudi.internal.DataSourceInternalWriterHelper.commit(DataSourceInternalWriterHelper.java:86) at org.apache.hudi.spark3.internal.HoodieDataSourceInternalBatchWrite.commit(HoodieDataSourceInternalBatchWrite.java:93) at org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.writeWithV2(WriteToDataSourceV2Exec.scala:371) ... 69 more Caused by: org.apache.hudi.exception.HoodieLockException: Unable to acquire lock, lock object null at org.apache.hudi.client.transaction.lock.LockManager.lock(LockManager.java:82) at org.apache.hudi.client.transaction.TransactionManager.beginTransaction(TransactionManager.java:64) at org.apache.hudi.client.AbstractHoodieWriteClient.commitStats(AbstractHoodieWriteClient.java:186) at org.apache.hudi.client.AbstractHoodieWriteClient.commitStats(AbstractHoodieWriteClient.java:171) at org.apache.hudi.internal.DataSourceInternalWriterHelper.commit(DataSourceInternalWriterHelper.java:83) ... 71 more -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
