fengjian428 edited a comment on issue #3755:
URL: https://github.com/apache/hudi/issues/3755#issuecomment-940179806
> It seems that the file left in reconcile stage is different with commit
meta. Could you kindly share relevant logs and file status about marker file?
before I got error above, this rpc timeout error happen, and when I restart
delta streamer, above error happen, should I change unpersist method
non-blocking and try again?
`Caused by: java.util.concurrent.ExecutionException:
org.apache.hudi.exception.HoodieException: Futures timed out after [600
seconds]. This timeout is controlled by spark.rpc.askTimeout
at
java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
at
org.apache.hudi.async.HoodieAsyncService.waitForShutdown(HoodieAsyncService.java:90)
at
org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.lambda$sync$1(HoodieDeltaStreamer.java:171)
... 8 more
Caused by: org.apache.hudi.exception.HoodieException: Futures timed out
after [600 seconds]. This timeout is controlled by spark.rpc.askTimeout
at
org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:657)
at
java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.spark.rpc.RpcTimeoutException: Futures timed out after
[600 seconds]. This timeout is controlled by spark.rpc.askTimeout
at
org.apache.spark.rpc.RpcTimeout.org$apache$spark$rpc$RpcTimeout$$createRpcTimeoutException(RpcTimeout.scala:47)
at
org.apache.spark.rpc.RpcTimeout$$anonfun$addMessageIfTimeout$1.applyOrElse(RpcTimeout.scala:62)
at
org.apache.spark.rpc.RpcTimeout$$anonfun$addMessageIfTimeout$1.applyOrElse(RpcTimeout.scala:58)
at
scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36)
at org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:76)
at
org.apache.spark.storage.BlockManagerMaster.removeRdd(BlockManagerMaster.scala:131)
at
org.apache.spark.SparkContext.unpersistRDD(SparkContext.scala:1821)
at org.apache.spark.rdd.RDD.unpersist(RDD.scala:217)
at org.apache.spark.api.java.JavaRDD.unpersist(JavaRDD.scala:53)
at
org.apache.hudi.client.SparkRDDWriteClient.lambda$releaseResources$5(SparkRDDWriteClient.java:499)
at java.lang.Iterable.forEach(Iterable.java:75)
at
org.apache.hudi.client.SparkRDDWriteClient.releaseResources(SparkRDDWriteClient.java:499)
at
org.apache.hudi.client.AbstractHoodieWriteClient.commitStats(AbstractHoodieWriteClient.java:193)
at
org.apache.hudi.client.SparkRDDWriteClient.commit(SparkRDDWriteClient.java:124)
at
org.apache.hudi.utilities.deltastreamer.DeltaSync.writeToSink(DeltaSync.java:525)
at
org.apache.hudi.utilities.deltastreamer.DeltaSync.syncOnce(DeltaSync.java:304)
at
org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:633)
... 4 more
Caused by: java.util.concurrent.TimeoutException: Futures timed out after
[600 seconds]
at
scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:223)
at
scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:227)
at
org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:220)
at org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:75)
... 16 more`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
