pengxianzi commented on issue #12589:
URL: https://github.com/apache/hudi/issues/12589#issuecomment-2574869799
@danny0405 I couldn't find the reason. The error log indicates that the
checkpoint failed
org.apache.flink.streaming.runtime.tasks.AsynccheckpointRunnable [].
bucket_write default_databases.loan_withdraw_order_new_cow ardemmewlcos part of
checkpoint 1 could not be completed.
java.util.concurrent.cancellationException:null
at java.util.concurrent.FutureTask.report(FutureTask.java:122) [? :
1.8.0_232]
at java.util.concurrent.FutureTask.get(FutureTask.java:192) [? :
1.8.0_232]
at
org.apache.flink.runtime.concurrent.FutureUtils.runIfNotDoneAndGet(FutureUtils.java:636)
[flink-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.flink.streaming.api.operators.OperatorSnapshotFinalizer.<init>(OperatorSnapshotFinalizer.java:60)
[flink-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointRunnable.run(SubtaskCheckpointRunnable.java:128)
[flink-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[? : 1.8.0_232]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[? : 1.8.0_232]
at java.lang.Thread.run(Thread.java:748) [? : 1.8.0_232]
org.apache.flink.runtime.checkpoint.CheckpointException: Could not complete
snapshot 2 for operator bucket_write:
default_database.lom_withdraw_order_new_con (3/4). failure reason: Checkpoint
was declined.
at
org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:264)
[flink-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:169)
[flink-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:371)
[flink-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointStreamOperator(SubtaskCheckpointCoordinatorImpl.java:706)
[flink-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.buildOperatorSnapshotFutures(SubtaskCheckpointCoordinatorImpl.java:627)
[flink-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.takeSnapshotSync(SubtaskCheckpointCoordinatorImpl.java:590)
[flink-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointState(SubtaskCheckpointCoordinatorImpl.java:312)
[flink-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$performCheckpoint$13(StreamTask.java:1089)
[flink-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)
[flink-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:1084)
[flink-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointOnBarrier(StreamTask.java:903)
[flink-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.flink.streaming.runtime.io.checkpointing.CheckpointBarrierHandler.notifyCheckpoint(CheckpointBarrierHandler.java:135)
[flink-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.flink.streaming.runtime.io.checkpointing.CheckpointBarrierHandler.processBarrier(CheckpointBarrierHandler.java:250)
[flink-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.processBarrier(SingleCheckpointBarrierHandler.java:61)
[flink-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.triggerGlobalCheckpoint(SingleCheckpointBarrierHandler.java:231)
[flink-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
Caused by :org.apache.hudi.exception.HoodieException:Timeout(1201000ms)
while waiting for instant initialize
at org.apache.hudi.sink.utils.TimeWait.waitFor(TimeWait.java:57)
~[hudi-flink1.13-bundle-0.14.0.jar:0.14.0]
at
org.apache.hudi.sink.common.AbstractStreamWriteFunction.instantToWrite(AbstractStreamWriteFunction.java:269)
~[hudi-flink1.13-bundle-0.14.0.jar:0.14.0]
at
org.apache.hudi.sink.StreamWriteFunction.flushRemaining(StreamWriteFunction.java:452)
~[hudi-flink1.13-bundle-0.14.0.jar:0.14.0]
at
org.apache.hudi.sink.StreamWriteFunction.snapshotState(StreamWriteFunction.java:137)
~[hudi-flink1.13-bundle-0.14.0.jar:0.14.0]
at
org.apache.hudi.sink.bucket.BucketStreamWriteFunction.snapshotState(BucketStreamWriteFunction.java:101)
~[hudi-flink1.13-bundle-0.14.0.jar:0.14.0]
at
org.apache.hudi.sink.common.AbstractStreamWriteFunction.snapshotState(AbstractStreamWriteFunction.java:167)
~[hudi-flink1.13-bundle-0.14.0.jar:0.14.0]
at
org.apache.flink.streaming.util.functions.StreamingFunctionUtils.trySnapshotFunctionState(StreamingFunctionUtils.java:118)
~[flink-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.flink.streaming.util.functions.StreamingFunctionUtils.snapshotFunctionState(StreamingFunctionUtils.java:99)
~[flink-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.snapshotState(AbstractUdfStreamOperator.java:89)
~[flink-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:218)
~[flink-1.0-SNAPSHOT-jar-with-dependencies.jar:?]... 29 more
switched from RUNNING to FAILED with failure cause: jave.io.I0Exception:
could not perforn checkpoint 2 for operator bucket_write:
default_database.loan_withdraw_order_new_cow(3/4)#0.
at
org.apache.flink.streaming.runtime.tasks.streamTask.triggerCheckpointOnBarrier(StreamTask.java:1045)
at
org.apache.flink.streaming.runtime.io.checkpointing.CheckpointBarrierHandler.notifyCheckpoint(CheckpointBarrierHandler.java:135)
at
org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.triggerCheckpoint(SingleCheckpointBarrierHandler.java:298)
at
org.apache.flink.streaming.runtime.io.checkpointing.singlecheckpointbarrierhandler.acces...@stringlcheckpointbarrierhandler.java:61)
at
org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandlerControllerImpl.triggerGlobalCheckpoint(SingleCheckpointBarrierHandler.java:43)
at
org.apache.flink.streaming.runtime.io.checkpointing.AbstrackAlignmentBarrierHandlerState.DarrierReceivedAbstractsLineOfBarrierHandlerState.java:61)
at
org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.processBarrier(SingleCheckpointBarrierHandler.java:227)
at
org.apache.flink.streaming.runtime.io.checkpointing.CheckpointedInputGate.handleEvent(CheckpointedInputGate.java:180)
at
org.apache.flink.streaming.runtime.io.AbstrackStreamTaskMethodFunction.emitNext(AbstrackStreamTaskMethodFunction.java:158)
at
org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:66)
at
org.apache.flink.streaming.runtime.tasks.streamTask.processInput(StreamTask.java:423)
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailLoopProcessor.runMailBoxLoop(AllDownProcessor.java:204)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.runMailBoxLoop(StreamTask.java:681)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.executeInvoke(StreamTask.java:616)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.runMitChCleanuponMail(StreamTask.java:647)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
