megha2411 edited a comment on issue #4814:
URL: https://github.com/apache/hudi/issues/4814#issuecomment-1039112003
**To Reproduce**
Steps to reproduce the behavior:
This is coming up intermittently. I tried processing 30 files and 1 failed
with this error. It did not fail in 2ns run. But again failed in 5th run. So
not sure on the steps to reproduce it.
**Expected behavior:**
IMO it should not fail with any NULL pointer exception
**Environment Description**
Hudi version : 0.10.1
Spark version : 3.1.2
Hive version : Hive 3.1.2
Hadoop version :
Storage (HDFS/S3/GCS..) : s3
Running on Docker? (yes/no) : no
Additional context
Attaching sbt build context for additional context. We run our application
on EMR 6.4.0
Concurrency options
"hoodie.write.lock.provider" ->
"org.apache.hudi.client.transaction.lock.ZookeeperBasedLockProvider",
"hoodie.write.lock.zookeeper.url" -> Conf.hiveConf.host,
"hoodie.write.lock.zookeeper.port" ->
Conf.hudiConf.zookeeperConf.port,
"hoodie.write.lock.zookeeper.base_path" ->
Conf.hudiConf.zookeeperConf.basePath,
"hoodie.write.concurrency.mode" -> "optimistic_concurrency_control",
"hoodie.cleaner.policy.failed.writes" -> "LAZY",
"hoodie.write.lock.zookeeper.lock_key" -> options.tableName,
"hoodie.write.lock.wait_time_ms" -> Conf.hudiConf.lockWaitTimeMs,
"hoodie.write.lock.num_retries" -> Conf.hudiConf.lockNumRetries,
"hoodie.write.lock.client.wait_time_ms" ->
Conf.hudiConf.lockClientWaitTimeMs,
"hoodie.write.lock.client.num_retries" ->
Conf.hudiConf.lockClientNumRetries,
"hoodie.write.lock.wait_time_ms_between_retry" ->
Conf.hudiConf.lockWaitTimeMsBetweenRetry,
"hoodie.metadata.enable" -> Conf.hudiConf.enableMetadata.toString
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
