maheshguptags commented on issue #12738:
URL: https://github.com/apache/hudi/issues/12738#issuecomment-2678313469
Sure @cshuo
I manually deleted the TM with MDT enabled(4th checkpointing in UI and 5 is
completing within ms). although I am not seeing any message unusual
info(regarding discarding the data if chkpnt fails), I am attaching both JM
stack trace and a Screenshot for checkpointing UI.
```025-02-24 12:36:23,154 INFO
org.apache.hudi.sink.StreamWriteOperatorCoordinator [] - Executor
executes action [handle write metadata event for instant 20250224123109474]
success!
2025-02-24 12:36:23,155 INFO
org.apache.hudi.sink.StreamWriteOperatorCoordinator [] - Executor
executes action [handle write metadata event for instant 20250224123109474]
success!
2025-02-24 12:36:23,155 INFO
org.apache.hudi.sink.StreamWriteOperatorCoordinator [] - Executor
executes action [handle write metadata event for instant 20250224123109474]
success!
2025-02-24 12:36:41,196 INFO
org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] -
Closing TaskExecutor connection
cdp-customer-profile-flink-hudi-temp-taskmanager-1-8 because: The TaskExecutor
is shutting down.
2025-02-24 12:36:41,196 INFO
org.apache.flink.runtime.resourcemanager.slotmanager.FineGrainedSlotManager []
- Unregistering task executor 0ab3cda26ee84c4d17a07170cc3d6512 from the slot
manager.
2025-02-24 12:36:41,196 INFO
org.apache.flink.runtime.resourcemanager.slotmanager.DefaultSlotStatusSyncer []
- Freeing slot 1b034d142f1d190f7a90a2b6d1ed559e.
2025-02-24 12:36:41,196 INFO
org.apache.flink.runtime.resourcemanager.slotmanager.DefaultSlotStatusSyncer []
- Freeing slot 2750e82c8c6933e02f6430036ad374fb.
2025-02-24 12:36:41,196 INFO
org.apache.flink.runtime.resourcemanager.slotmanager.DefaultSlotStatusSyncer []
- Freeing slot 1b6d0825dc15af289115be621bfc4d24.
2025-02-24 12:36:41,196 INFO
org.apache.flink.runtime.resourcemanager.slotmanager.DefaultSlotStatusSyncer []
- Freeing slot daac4949d81d51b41651522e10ad4382.
2025-02-24 12:36:41,196 INFO
org.apache.flink.runtime.resourcemanager.slotmanager.DefaultSlotStatusSyncer []
- Freeing slot 65daa7e56ca17a98127fee67481debcf.
2025-02-24 12:36:41,196 INFO
org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] -
Stopping worker cdp-customer-profile-flink-hudi-temp-taskmanager-1-8.
2025-02-24 12:36:41,196 INFO
org.apache.flink.kubernetes.KubernetesResourceManagerDriver [] - Stopping
TaskManager pod cdp-customer-profile-flink-hudi-temp-taskmanager-1-8.
2025-02-24 12:36:41,261 INFO
org.apache.flink.runtime.resourcemanager.slotmanager.FineGrainedSlotManager []
- Matching resource requirements against available resources.
Missing resources:
Job 5c60fca92dab0e8ab0795dd9e706a4c2
ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN},
numberOfRequiredSlots=5}
Current resources:
TaskManager cdp-customer-profile-flink-hudi-temp-taskmanager-1-6
Available: ResourceProfile{cpuCores=0, taskHeapMemory=0 bytes,
taskOffHeapMemory=0 bytes, managedMemory=3 bytes, networkMemory=2 bytes}
Total: ResourceProfile{cpuCores=2.5,
taskHeapMemory=741.000mb (776994800 bytes), taskOffHeapMemory=0 bytes,
managedMemory=797.600mb (836344228 bytes), networkMemory=199.400mb (209086057
bytes)}
TaskManager cdp-customer-profile-flink-hudi-temp-taskmanager-1-7
Available: ResourceProfile{cpuCores=0, taskHeapMemory=0 bytes,
taskOffHeapMemory=0 bytes, managedMemory=3 bytes, networkMemory=2 bytes}
Total: ResourceProfile{cpuCores=2.5,
taskHeapMemory=741.000mb (776994800 bytes), taskOffHeapMemory=0 bytes,
managedMemory=797.600mb (836344228 bytes), networkMemory=199.400mb (209086057
bytes)}
TaskManager cdp-customer-profile-flink-hudi-temp-taskmanager-1-4
Available: ResourceProfile{cpuCores=0, taskHeapMemory=0 bytes,
taskOffHeapMemory=0 bytes, managedMemory=3 bytes, networkMemory=2 bytes}
Total: ResourceProfile{cpuCores=2.5,
taskHeapMemory=741.000mb (776994800 bytes), taskOffHeapMemory=0 bytes,
managedMemory=797.600mb (836344228 bytes), networkMemory=199.400mb (209086057
bytes)}
TaskManager cdp-customer-profile-flink-hudi-temp-taskmanager-1-5
Available: ResourceProfile{cpuCores=0, taskHeapMemory=0 bytes,
taskOffHeapMemory=0 bytes, managedMemory=3 bytes, networkMemory=2 bytes}
Total: ResourceProfile{cpuCores=2.5,
taskHeapMemory=741.000mb (776994800 bytes), taskOffHeapMemory=0 bytes,
managedMemory=797.600mb (836344228 bytes), networkMemory=199.400mb (209086057
bytes)}
2025-02-24 12:36:41,331 INFO
org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] - need
request 1 new workers, current worker number 4, declared worker number 5
2025-02-24 12:36:41,331 INFO
org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] -
Requesting new worker with resource spec WorkerResourceSpec {cpuCores=2.5,
taskHeapSize=741.000mb (776994800 bytes), taskOffHeapSize=0 bytes,
networkMemSize=199.400mb (209086057 bytes), managedMemSize=797.600mb (836344228
bytes), numSlots=5}, current pending count: 1.
2025-02-24 12:36:41,332 INFO
org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Enabled
external resources: []
2025-02-24 12:36:41,332 INFO org.apache.flink.configuration.Configuration
[] - Config uses fallback configuration key
'kubernetes.service-account' instead of key
'kubernetes.taskmanager.service-account'
2025-02-24 12:36:41,332 INFO org.apache.flink.configuration.Configuration
[] - Config uses fallback configuration key
'kubernetes.service-account' instead of key
'kubernetes.taskmanager.service-account'
2025-02-24 12:36:41,332 INFO
org.apache.flink.kubernetes.utils.KubernetesUtils [] - The service
account configured in pod template will be overwritten to
'qbm-cdp-ingestion-flink-service-account' because of explicitly configured
options.
2025-02-24 12:36:41,334 INFO
org.apache.flink.kubernetes.KubernetesResourceManagerDriver [] - Creating new
TaskManager pod with name cdp-customer-profile-flink-hudi-temp-taskmanager-1-9
and resource <2500,2.5>.
2025-02-24 12:36:41,377 INFO
org.apache.flink.kubernetes.KubernetesResourceManagerDriver [] - Pod
cdp-customer-profile-flink-hudi-temp-taskmanager-1-9 is created.
2025-02-24 12:36:41,538 INFO org.apache.flink.runtime.jobmaster.JobMaster
[] - Disconnect TaskExecutor
cdp-customer-profile-flink-hudi-temp-taskmanager-1-8 because: The TaskExecutor
is shutting down.
2025-02-24 12:36:41,539 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - bucket_write:
default_database.customer_profile (21/25)
(3af50fb5a53008686151e6064afe51a7_3f6cfb4bf36a2f36a133726411a32a74_20_0)
switched from RUNNING to FAILED on
cdp-customer-profile-flink-hudi-temp-taskmanager-1-8 @
ip-10-232-117-102.ap-south-1.compute.internal (dataPort=39121).
org.apache.flink.util.FlinkExpectedException: The TaskExecutor is shutting
down.
at
org.apache.flink.runtime.taskexecutor.TaskExecutor.onStop(TaskExecutor.java:481)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:239)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor$StartedState.lambda$terminate$0(PekkoRpcActor.java:574)
~[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at
org.apache.flink.runtime.concurrent.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor$StartedState.terminate(PekkoRpcActor.java:573)
~[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleControlMessage(PekkoRpcActor.java:196)
~[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at
org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33)
~[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at
org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29)
~[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at scala.PartialFunction.applyOrElse(PartialFunction.scala:127)
~[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:126)
~[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at
org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29)
~[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:175)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at
org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) [?:?]
at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown
Source) [?:?]
at java.util.concurrent.ForkJoinPool.scan(Unknown Source) [?:?]
at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source) [?:?]
at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source) [?:?]
2025-02-24 12:36:41,547 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
3af50fb5a53008686151e6064afe51a7_3f6cfb4bf36a2f36a133726411a32a74_20_0.
2025-02-24 12:36:41,547 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
3af50fb5a53008686151e6064afe51a7_3f6cfb4bf36a2f36a133726411a32a74_20_0.
2025-02-24 12:36:41,547 INFO
org.apache.flink.runtime.scheduler.adaptive.AdaptiveScheduler [] - Restarting
job.
org.apache.flink.util.FlinkExpectedException: The TaskExecutor is shutting
down.
at
org.apache.flink.runtime.taskexecutor.TaskExecutor.onStop(TaskExecutor.java:481)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:239)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor$StartedState.lambda$terminate$0(PekkoRpcActor.java:574)
~[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at
org.apache.flink.runtime.concurrent.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor$StartedState.terminate(PekkoRpcActor.java:573)
~[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleControlMessage(PekkoRpcActor.java:196)
~[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at
org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33)
~[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at
org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29)
~[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at scala.PartialFunction.applyOrElse(PartialFunction.scala:127)
~[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:126)
~[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at
org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29)
~[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:175)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at
org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) [?:?]
at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown
Source) [?:?]
at java.util.concurrent.ForkJoinPool.scan(Unknown Source) [?:?]
at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source) [?:?]
at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source) [?:?]
2025-02-24 12:36:41,548 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job
cdp-customer-profile-flink-hudi-temp (5c60fca92dab0e8ab0795dd9e706a4c2)
switched from state RUNNING to CANCELLING.
2025-02-24 12:36:41,548 WARN
org.apache.flink.runtime.checkpoint.CheckpointFailureManager [] - Failed to
trigger or complete checkpoint 5 for job 5c60fca92dab0e8ab0795dd9e706a4c2. (0
consecutive failed attempts so far)
org.apache.flink.runtime.checkpoint.CheckpointException: Checkpoint
Coordinator is suspending.
at
org.apache.flink.runtime.checkpoint.CheckpointCoordinator.stopCheckpointScheduler(CheckpointCoordinator.java:1976)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.checkpoint.CheckpointCoordinatorDeActivator.jobStatusChanges(CheckpointCoordinatorDeActivator.java:46)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.executiongraph.DefaultExecutionGraph.notifyJobStatusChange(DefaultExecutionGraph.java:1589)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.executiongraph.DefaultExecutionGraph.transitionState(DefaultExecutionGraph.java:1155)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.executiongraph.DefaultExecutionGraph.transitionState(DefaultExecutionGraph.java:1127)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.executiongraph.DefaultExecutionGraph.cancel(DefaultExecutionGraph.java:955)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.scheduler.adaptive.Restarting.<init>(Restarting.java:65)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.scheduler.adaptive.Restarting$Factory.getState(Restarting.java:160)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.scheduler.adaptive.Restarting$Factory.getState(Restarting.java:125)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.scheduler.adaptive.AdaptiveScheduler.transitionToState(AdaptiveScheduler.java:1295)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.scheduler.adaptive.AdaptiveScheduler.goToRestarting(AdaptiveScheduler.java:977)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.scheduler.adaptive.FailureResultUtil.restartOrFail(FailureResultUtil.java:28)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.scheduler.adaptive.Executing.onFailure(Executing.java:93)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.scheduler.adaptive.StateWithExecutionGraph.updateTaskExecutionState(StateWithExecutionGraph.java:387)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.scheduler.adaptive.AdaptiveScheduler.lambda$updateTaskExecutionState$4(AdaptiveScheduler.java:560)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.scheduler.adaptive.State.tryCall(State.java:136)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.scheduler.adaptive.AdaptiveScheduler.updateTaskExecutionState(AdaptiveScheduler.java:557)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.scheduler.UpdateSchedulerNgOnInternalFailuresListener.notifyTaskFailure(UpdateSchedulerNgOnInternalFailuresListener.java:51)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.executiongraph.DefaultExecutionGraph.notifySchedulerNgAboutInternalTaskFailure(DefaultExecutionGraph.java:1645)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.executiongraph.Execution.processFail(Execution.java:1144)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.executiongraph.Execution.processFail(Execution.java:1084)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.executiongraph.Execution.fail(Execution.java:785)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.jobmaster.slotpool.SingleLogicalSlot.signalPayloadRelease(SingleLogicalSlot.java:195)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.jobmaster.slotpool.SingleLogicalSlot.releaseSlot(SingleLogicalSlot.java:129)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.scheduler.adaptive.allocator.SharedSlot.release(SharedSlot.java:144)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.jobmaster.slotpool.AllocatedSlot.releasePayload(AllocatedSlot.java:152)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool.releasePayload(DefaultDeclarativeSlotPool.java:482)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool.freeAndReleaseSlots(DefaultDeclarativeSlotPool.java:474)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool.releaseSlots(DefaultDeclarativeSlotPool.java:445)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.jobmaster.slotpool.DeclarativeSlotPoolService.internalReleaseTaskManager(DeclarativeSlotPoolService.java:275)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.jobmaster.slotpool.DeclarativeSlotPoolService.releaseTaskManager(DeclarativeSlotPoolService.java:231)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.jobmaster.JobMaster.disconnectTaskManager(JobMaster.java:549)
~[flink-dist-1.18.1.jar:1.18.1]
at jdk.internal.reflect.GeneratedMethodAccessor404.invoke(Unknown
Source) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
Source) ~[?:?]
at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]
at
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.lambda$handleRpcInvocation$1(PekkoRpcActor.java:309)
~[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at
org.apache.flink.runtime.concurrent.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83)
~[flink-dist-1.18.1.jar:1.18.1]
at
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleRpcInvocation(PekkoRpcActor.java:307)
~[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleRpcMessage(PekkoRpcActor.java:222)
~[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at
org.apache.flink.runtime.rpc.pekko.FencedPekkoRpcActor.handleRpcMessage(FencedPekkoRpcActor.java:85)
~[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleMessage(PekkoRpcActor.java:168)
~[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at
org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at
org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at scala.PartialFunction.applyOrElse(PartialFunction.scala:127)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:126)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at
org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:175)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at
org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253)
[flink-rpc-akkab03b76dc-1f97-48c6-9a76-a2d0a4b7bea8.jar:1.18.1]
at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) [?:?]
at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown
Source) [?:?]
at java.util.concurrent.ForkJoinPool.scan(Unknown Source) [?:?]
at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source) [?:?]
at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source) [?:?]
2025-02-24 12:36:41,550 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
cdp_identify_temp_perf -> Map -> *anonymous_datastream_source$1*[1] -> Calc[2]
-> row_data_to_hoodie_record (1/20)
(3af50fb5a53008686151e6064afe51a7_cbc357ccb763df2852fee8c4fc7d55f2_0_4)
switched from RUNNING to CANCELING.
2025-02-24 12:36:41,550 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
cdp_identify_temp_
```
<img width="1175" alt="Image"
src="https://github.com/user-attachments/assets/d905cdd7-7777-4d6c-9f75-cf58fc3fc488";
/>
Thanks
Mahesh
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
