megha2411 edited a comment on issue #4814:
URL: https://github.com/apache/hudi/issues/4814#issuecomment-1039112003
**To Reproduce**
Steps to reproduce the behavior:
This is coming up intermittently. I tried processing 30 files and 1 failed
with this error. It did not fail in 2nd run. But again failed in 5th run. So
not sure on the steps to reproduce it.
**Expected behavior:**
IMO it should not fail with any NULL pointer exception
**Environment Description**
Hudi version : 0.10.1
Spark version : 3.1.2
Hive version : Hive 3.1.2
Hadoop version :
Storage (HDFS/S3/GCS..) : s3
Running on Docker? (yes/no) : no
Additional context
We run our application on EMR 6.4.0
Concurrency options
"hoodie.write.lock.provider" ->
"org.apache.hudi.client.transaction.lock.ZookeeperBasedLockProvider",
"hoodie.write.lock.zookeeper.url" -> Conf.hiveConf.host,
"hoodie.write.lock.zookeeper.port" ->
Conf.hudiConf.zookeeperConf.port,
"hoodie.write.lock.zookeeper.base_path" ->
Conf.hudiConf.zookeeperConf.basePath,
"hoodie.write.concurrency.mode" -> "optimistic_concurrency_control",
"hoodie.cleaner.policy.failed.writes" -> "LAZY",
"hoodie.write.lock.zookeeper.lock_key" -> options.tableName,
"hoodie.write.lock.wait_time_ms" -> 5000,
"hoodie.write.lock.num_retries" -> 5,
"hoodie.write.lock.client.wait_time_ms" ->5000,
"hoodie.write.lock.client.num_retries" -> 5s,
"hoodie.write.lock.wait_time_ms_between_retry" -> 12000,
"hoodie.metadata.enable" -> true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
