[
https://issues.apache.org/jira/browse/FLINK-28024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17553590#comment-17553590
]
Matthias Pohl commented on FLINK-28024:
---------------------------------------
The build in question seem to have stalled in
{{{}org.apache.flink.test.checkpointing.KeyedStateCheckpointingITCase.testWithRocksDbBackendIncremental{}}}.
The following fragment gets repeated over and over again:
{code:java}
16:30:01,430 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - No master
state to restore
16:30:01,431 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (1/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_bc764cd8ddf7a0cff126f51c16239658_0_2)
switched from CREATED to SCHEDULED.
16:30:01,431 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (2/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_bc764cd8ddf7a0cff126f51c16239658_1_2)
switched from CREATED to SCHEDULED.
16:30:01,431 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (3/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_bc764cd8ddf7a0cff126f51c16239658_2_2)
switched from CREATED to SCHEDULED.
16:30:01,431 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (4/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_bc764cd8ddf7a0cff126f51c16239658_3_2)
switched from CREATED to SCHEDULED.
16:30:01,431 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (1/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_feca28aff5a3958840bee985ee7de4d3_0_2)
switched from CREATED to SCHEDULED.
16:30:01,431 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (2/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_feca28aff5a3958840bee985ee7de4d3_1_2)
switched from CREATED to SCHEDULED.
16:30:01,431 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (3/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_feca28aff5a3958840bee985ee7de4d3_2_2)
switched from CREATED to SCHEDULED.
16:30:01,431 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (4/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_feca28aff5a3958840bee985ee7de4d3_3_2)
switched from CREATED to SCHEDULED.
16:30:01,431 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Map (1/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_b27f31f3e3a199a9981d185a455185be_0_2)
switched from CREATED to SCHEDULED.
16:30:01,431 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Map (2/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_b27f31f3e3a199a9981d185a455185be_1_2)
switched from CREATED to SCHEDULED.
16:30:01,431 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Map (3/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_b27f31f3e3a199a9981d185a455185be_2_2)
switched from CREATED to SCHEDULED.
16:30:01,431 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Map (4/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_b27f31f3e3a199a9981d185a455185be_3_2)
switched from CREATED to SCHEDULED.
16:30:01,431 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Sink: Unnamed
(1/4) (b8f79df70c20ee5cc51bd99bd6852bb8_353a6b34b8b7f1c1d0fb4616d911049c_0_2)
switched from CREATED to SCHEDULED.
16:30:01,431 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Sink: Unnamed
(2/4) (b8f79df70c20ee5cc51bd99bd6852bb8_353a6b34b8b7f1c1d0fb4616d911049c_1_2)
switched from CREATED to SCHEDULED.
16:30:01,431 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Sink: Unnamed
(3/4) (b8f79df70c20ee5cc51bd99bd6852bb8_353a6b34b8b7f1c1d0fb4616d911049c_2_2)
switched from CREATED to SCHEDULED.
16:30:01,431 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Sink: Unnamed
(4/4) (b8f79df70c20ee5cc51bd99bd6852bb8_353a6b34b8b7f1c1d0fb4616d911049c_3_2)
switched from CREATED to SCHEDULED.
16:30:01,431 [flink-akka.actor.default-dispatcher-45] INFO
org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager []
- Received resource requirements from job 25a853c8210c1d08c0560a1e512bb0c7:
[ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN},
numberOfRequiredSlots=1}]
16:30:01,432 [flink-akka.actor.default-dispatcher-45] INFO
org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager []
- Received resource requirements from job 25a853c8210c1d08c0560a1e512bb0c7:
[ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN},
numberOfRequiredSlots=2}]
16:30:01,432 [flink-akka.actor.default-dispatcher-45] INFO
org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager []
- Received resource requirements from job 25a853c8210c1d08c0560a1e512bb0c7:
[ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN},
numberOfRequiredSlots=3}]
16:30:01,432 [flink-akka.actor.default-dispatcher-45] WARN
org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager []
- Could not fulfill resource requirements of job
25a853c8210c1d08c0560a1e512bb0c7. Free slots: 0
16:30:01,432 [flink-akka.actor.default-dispatcher-45] INFO
org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager []
- Received resource requirements from job 25a853c8210c1d08c0560a1e512bb0c7:
[ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN},
numberOfRequiredSlots=4}]
16:30:01,433 [flink-akka.actor.default-dispatcher-45] WARN
org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager []
- Could not fulfill resource requirements of job
25a853c8210c1d08c0560a1e512bb0c7. Free slots: 0
16:30:01,433 [flink-akka.actor.default-dispatcher-38] WARN
org.apache.flink.runtime.jobmaster.slotpool.DeclarativeSlotPoolBridge [] -
Could not acquire the minimum required resources, failing slot requests.
Acquired:
[ResourceRequirement{resourceProfile=ResourceProfile{taskHeapMemory=512.000gb
(549755813888 bytes), taskOffHeapMemory=512.000gb (549755813888 byte
s), managedMemory=6.000mb (6291456 bytes), networkMemory=32.000mb (33554432
bytes)}, numberOfRequiredSlots=2}]. Current slot pool status: Registered TMs:
1, registered slots: 2 free slots: 0
16:30:01,434 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (3/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_bc764cd8ddf7a0cff126f51c16239658_2_2)
switched from SCHEDULED to FAILED on [unassigned resource].
org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException:
Could not acquire the minimum required resources.
16:30:01,435 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
b8f79df70c20ee5cc51bd99bd6852bb8_bc764cd8ddf7a0cff126f51c16239658_2_2.
16:30:01,435 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy
[] - Calculating tasks to restart to recover the failed task
bc764cd8ddf7a0cff126f51c16239658_2.
16:30:01,436 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy
[] - 16 tasks should be restarted to recover the failed task
bc764cd8ddf7a0cff126f51c16239658_2.
16:30:01,436 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job Flink
Streaming Job (25a853c8210c1d08c0560a1e512bb0c7) switched from state RUNNING to
RESTARTING.
16:30:01,437 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (1/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_bc764cd8ddf7a0cff126f51c16239658_0_2)
switched from SCHEDULED to CANCELING.
16:30:01,437 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (1/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_bc764cd8ddf7a0cff126f51c16239658_0_2)
switched from CANCELING to CANCELED.
16:30:01,437 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
b8f79df70c20ee5cc51bd99bd6852bb8_bc764cd8ddf7a0cff126f51c16239658_0_2.
16:30:01,437 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (2/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_bc764cd8ddf7a0cff126f51c16239658_1_2)
switched from SCHEDULED to CANCELING.
16:30:01,438 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (2/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_bc764cd8ddf7a0cff126f51c16239658_1_2)
switched from CANCELING to CANCELED.
16:30:01,438 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
b8f79df70c20ee5cc51bd99bd6852bb8_bc764cd8ddf7a0cff126f51c16239658_1_2.
16:30:01,438 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
b8f79df70c20ee5cc51bd99bd6852bb8_bc764cd8ddf7a0cff126f51c16239658_1_2.
16:30:01,438 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (1/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_feca28aff5a3958840bee985ee7de4d3_0_2)
switched from SCHEDULED to CANCELING.
16:30:01,438 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (1/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_feca28aff5a3958840bee985ee7de4d3_0_2)
switched from CANCELING to CANCELED.
16:30:01,438 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
b8f79df70c20ee5cc51bd99bd6852bb8_feca28aff5a3958840bee985ee7de4d3_0_2.
16:30:01,438 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Map (3/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_b27f31f3e3a199a9981d185a455185be_2_2)
switched from SCHEDULED to CANCELING.
16:30:01,438 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Map (3/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_b27f31f3e3a199a9981d185a455185be_2_2)
switched from CANCELING to CANCELED.
16:30:01,438 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
b8f79df70c20ee5cc51bd99bd6852bb8_b27f31f3e3a199a9981d185a455185be_2_2.
16:30:01,438 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Sink: Unnamed
(3/4) (b8f79df70c20ee5cc51bd99bd6852bb8_353a6b34b8b7f1c1d0fb4616d911049c_2_2)
switched from SCHEDULED to CANCELING.
16:30:01,438 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Sink: Unnamed
(3/4) (b8f79df70c20ee5cc51bd99bd6852bb8_353a6b34b8b7f1c1d0fb4616d911049c_2_2)
switched from CANCELING to CANCELED.
16:30:01,438 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
b8f79df70c20ee5cc51bd99bd6852bb8_353a6b34b8b7f1c1d0fb4616d911049c_2_2.
16:30:01,438 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Map (4/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_b27f31f3e3a199a9981d185a455185be_3_2)
switched from SCHEDULED to CANCELING.
16:30:01,438 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Map (4/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_b27f31f3e3a199a9981d185a455185be_3_2)
switched from CANCELING to CANCELED.
16:30:01,438 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
b8f79df70c20ee5cc51bd99bd6852bb8_b27f31f3e3a199a9981d185a455185be_3_2.
16:30:01,438 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
b8f79df70c20ee5cc51bd99bd6852bb8_b27f31f3e3a199a9981d185a455185be_3_2.
16:30:01,438 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Sink: Unnamed
(2/4) (b8f79df70c20ee5cc51bd99bd6852bb8_353a6b34b8b7f1c1d0fb4616d911049c_1_2)
switched from SCHEDULED to CANCELING.
16:30:01,439 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Sink: Unnamed
(2/4) (b8f79df70c20ee5cc51bd99bd6852bb8_353a6b34b8b7f1c1d0fb4616d911049c_1_2)
switched from CANCELING to CANCELED.
16:30:01,439 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
b8f79df70c20ee5cc51bd99bd6852bb8_353a6b34b8b7f1c1d0fb4616d911049c_1_2.
16:30:01,439 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Sink: Unnamed
(1/4) (b8f79df70c20ee5cc51bd99bd6852bb8_353a6b34b8b7f1c1d0fb4616d911049c_0_2)
switched from SCHEDULED to CANCELING.
16:30:01,439 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Sink: Unnamed
(1/4) (b8f79df70c20ee5cc51bd99bd6852bb8_353a6b34b8b7f1c1d0fb4616d911049c_0_2)
switched from CANCELING to CANCELED.
16:30:01,439 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
b8f79df70c20ee5cc51bd99bd6852bb8_353a6b34b8b7f1c1d0fb4616d911049c_0_2.
16:30:01,439 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Map (1/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_b27f31f3e3a199a9981d185a455185be_0_2)
switched from SCHEDULED to CANCELING.
16:30:01,439 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Map (1/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_b27f31f3e3a199a9981d185a455185be_0_2)
switched from CANCELING to CANCELED.
16:30:01,439 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
b8f79df70c20ee5cc51bd99bd6852bb8_b27f31f3e3a199a9981d185a455185be_0_2.
16:30:01,439 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Map (2/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_b27f31f3e3a199a9981d185a455185be_1_2)
switched from SCHEDULED to CANCELING.
16:30:01,439 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Map (2/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_b27f31f3e3a199a9981d185a455185be_1_2)
switched from CANCELING to CANCELED.
16:30:01,439 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
b8f79df70c20ee5cc51bd99bd6852bb8_b27f31f3e3a199a9981d185a455185be_1_2.
16:30:01,439 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
b8f79df70c20ee5cc51bd99bd6852bb8_b27f31f3e3a199a9981d185a455185be_1_2.
16:30:01,439 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Sink: Unnamed
(4/4) (b8f79df70c20ee5cc51bd99bd6852bb8_353a6b34b8b7f1c1d0fb4616d911049c_3_2)
switched from SCHEDULED to CANCELING.
16:30:01,439 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Sink: Unnamed
(4/4) (b8f79df70c20ee5cc51bd99bd6852bb8_353a6b34b8b7f1c1d0fb4616d911049c_3_2)
switched from CANCELING to CANCELED.
16:30:01,439 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
b8f79df70c20ee5cc51bd99bd6852bb8_353a6b34b8b7f1c1d0fb4616d911049c_3_2.
16:30:01,439 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (4/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_feca28aff5a3958840bee985ee7de4d3_3_2)
switched from SCHEDULED to CANCELING.
16:30:01,440 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (4/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_feca28aff5a3958840bee985ee7de4d3_3_2)
switched from CANCELING to CANCELED.
16:30:01,440 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
b8f79df70c20ee5cc51bd99bd6852bb8_feca28aff5a3958840bee985ee7de4d3_3_2.
16:30:01,440 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
b8f79df70c20ee5cc51bd99bd6852bb8_feca28aff5a3958840bee985ee7de4d3_3_2.
16:30:01,440 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (3/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_feca28aff5a3958840bee985ee7de4d3_2_2)
switched from SCHEDULED to CANCELING.
16:30:01,440 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (3/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_feca28aff5a3958840bee985ee7de4d3_2_2)
switched from CANCELING to CANCELED.
16:30:01,440 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
b8f79df70c20ee5cc51bd99bd6852bb8_feca28aff5a3958840bee985ee7de4d3_2_2.
16:30:01,440 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (2/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_feca28aff5a3958840bee985ee7de4d3_1_2)
switched from SCHEDULED to CANCELING.
16:30:01,440 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (2/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_feca28aff5a3958840bee985ee7de4d3_1_2)
switched from CANCELING to CANCELED.
16:30:01,440 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
b8f79df70c20ee5cc51bd99bd6852bb8_feca28aff5a3958840bee985ee7de4d3_1_2.
16:30:01,440 [flink-akka.actor.default-dispatcher-45] INFO
org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager []
- Received resource requirements from job 25a853c8210c1d08c0560a1e512bb0c7:
[ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN},
numberOfRequiredSlots=3}]
16:30:01,440 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
b8f79df70c20ee5cc51bd99bd6852bb8_feca28aff5a3958840bee985ee7de4d3_1_2.
16:30:01,440 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (4/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_bc764cd8ddf7a0cff126f51c16239658_3_2)
switched from SCHEDULED to CANCELING.
16:30:01,441 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (4/4)
(b8f79df70c20ee5cc51bd99bd6852bb8_bc764cd8ddf7a0cff126f51c16239658_3_2)
switched from CANCELING to CANCELED.
16:30:01,441 [flink-akka.actor.default-dispatcher-45] WARN
org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager []
- Could not fulfill resource requirements of job
25a853c8210c1d08c0560a1e512bb0c7. Free slots: 0
16:30:01,441 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
b8f79df70c20ee5cc51bd99bd6852bb8_bc764cd8ddf7a0cff126f51c16239658_3_2.
16:30:01,441 [flink-akka.actor.default-dispatcher-43] INFO
org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager []
- Received resource requirements from job 25a853c8210c1d08c0560a1e512bb0c7:
[ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN},
numberOfRequiredSlots=2}]
16:30:01,441 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding
the results produced by task execution
b8f79df70c20ee5cc51bd99bd6852bb8_bc764cd8ddf7a0cff126f51c16239658_3_2.
16:30:01,441 [flink-akka.actor.default-dispatcher-43] INFO
org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager []
- Clearing resource requirements of job 25a853c8210c1d08c0560a1e512bb0c7
16:30:01,442 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job Flink
Streaming Job (25a853c8210c1d08c0560a1e512bb0c7) switched from state RESTARTING
to RUNNING.
16:30:01,442 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Restoring job
25a853c8210c1d08c0560a1e512bb0c7 from Checkpoint 1 @ 1654705506274 for
25a853c8210c1d08c0560a1e512bb0c7 located at
<checkpoint-not-externally-addressable>.
16:30:01,444 [flink-akka.actor.default-dispatcher-38] INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - No master
state to restore {code}
> Azure worker stops operating due to log files becoming too big
> --------------------------------------------------------------
>
> Key: FLINK-28024
> URL: https://issues.apache.org/jira/browse/FLINK-28024
> Project: Flink
> Issue Type: Bug
> Components: Build System / Azure Pipelines
> Affects Versions: 1.16.0
> Reporter: Matthias Pohl
> Priority: Major
>
> We observed several situations already where log files reached a file size of
> over 120G. This caused the worker's disk usage to reach 100% resulting in the
> worker machine to go "offline", i.e. not being available to pick up new tasks.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
