[
https://issues.apache.org/jira/browse/FLINK-36302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18123261#comment-18123261
]
Martijn Visser commented on FLINK-36302:
----------------------------------------
Three more on master, all in the JobManager failover variant, in the GHA
nightlies of 2026-10-04 and 2026-10-05:
https://github.com/apache/flink/actions/runs/37171533299/job/111347595030
(AdaptiveScheduler, JDK 17)
https://github.com/apache/flink/actions/runs/37254546337/job/111591580690 (JDK
21)
https://github.com/apache/flink/actions/runs/37254546337/job/111592643564
(Hadoop 3.4.3, JDK 17)
{code}
03:08:52.183 [INFO] Running
org.apache.flink.connector.file.src.FileSourceTextLinesITCase
=== WARNING: This task took already 95% of the available time budget of 239
minutes ===
"ForkJoinPool-1-worker-1" #20 daemon prio=5 os_prio=0 cpu=15129.99ms
elapsed=12663.79s
java.lang.Thread.State: TIMED_WAITING (sleeping)
at
org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.sleepBeforeRetry(CollectResultFetcher.java:249)
at
org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.next(CollectResultFetcher.java:115)
at
org.apache.flink.streaming.api.operators.collect.CollectResultIterator.hasNext(CollectResultIterator.java:98)
at
org.apache.flink.connector.file.src.FileSourceTextLinesITCase.testContinuousTextFileSource(FileSourceTextLinesITCase.java:270)
at
org.apache.flink.connector.file.src.FileSourceTextLinesITCase.testContinuousTextFileSourceWithJobManagerFailover(FileSourceTextLinesITCase.java:216)
{code}
The job keeps running with idle tasks and completes checkpoints until the leg
is killed. In all three fork logs the coordinator snapshot of the checkpoint
that the new JobMaster restores was still awaiting a split assignment event,
that file is not assigned again after the restore, and the collect sink comes
back with offset 5 and no buffered results, while the test waits for 30 more
lines:
{code}
03:08:57,474 [SourceCoordinator-Source: file-source] INFO
LocalityAwareSplitAssigner [] - Assigning remote split to requesting host
'localhost': Optional[FileSourceSplit:
file:/tmp/junit-12434794939581622029/nested1/text.1 [0, 225) (no host info)
ID=0000000104 position=null]
03:08:57,475 [flink-pekko.actor.default-dispatcher-5] INFO
OperatorCoordinatorHolder [] - Coordinator checkpoint 8 for coordinator
bc764cd8ddf7a0cff126f51c16239658 is awaiting 1 pending events
03:08:57,486 [Source: file-source (1/4)#0] INFO SourceReaderBase [] - Finished
reading split(s) [0000000104]
03:08:57,496 [ForkJoinPool-1-worker-1] INFO EmbeddedLeaderService [] -
Revoking leadership of
org.apache.flink.runtime.jobmaster.JobMasterServiceLeadershipRunner@6997c35.
03:08:57,498 [jobmanager-io-thread-2] INFO CheckpointCoordinator [] -
Completed checkpoint 8 for job a8ade8b8bae1d0c72951cfe1931df82e (1741 bytes,
checkpointDuration=23 ms, finalizationTime=4 ms).
03:08:57,657 [jobmanager-io-thread-1] INFO CheckpointCoordinator [] -
Restoring job a8ade8b8bae1d0c72951cfe1931df82e from Checkpoint 8 @
1791083337471 ...
03:08:57,743 [Sink: Data stream collect sink (1/1)#0] INFO CollectSinkFunction
[] - Initializing collect sink state with offset = 5, buffered results bytes = 0
{code}
In the two passing runs I have fork logs for (release-2.3, Azure 79631 and
79742) the restored checkpoint had no pending events and all six files were
assigned again. All three hung runs had unaligned checkpoints and
recover-output-on-downstream randomized on. The class passed in the other
connect legs of both nightlies and in the master push runs of both commits;
none of the 98 GHA master nightlies between 2026-06-26 and 2026-10-03 hung here.
> FileSourceTextLinesITCase timed out
> -----------------------------------
>
> Key: FLINK-36302
> URL: https://issues.apache.org/jira/browse/FLINK-36302
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Common
> Affects Versions: 2.0-preview
> Reporter: Matthias Pohl
> Priority: Critical
> Labels: test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=62064&view=logs&j=1c002d28-a73d-5309-26ee-10036d8476b4&t=d1c117a6-8f13-5466-55f0-d48dbb767fcd&l=12386
> {code}
> "ForkJoinPool-1-worker-1" #15 daemon prio=5 os_prio=0 tid=0x00007f6c0c8b5800
> nid=0xda34 waiting on condition [0x00007f6bf0dfc000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000ff6d4038> (a
> java.util.concurrent.CompletableFuture$Signaller)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1707)
> at
> java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3313)
> at
> java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1742)
> at
> java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
> at
> org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.isJobTerminated(CollectResultFetcher.java:213)
> at
> org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.next(CollectResultFetcher.java:120)
> at
> org.apache.flink.streaming.api.operators.collect.CollectResultIterator.nextResultFromFetcher(CollectResultIterator.java:126)
> at
> org.apache.flink.streaming.api.operators.collect.CollectResultIterator.hasNext(CollectResultIterator.java:100)
> at
> org.apache.flink.streaming.api.datastream.DataStreamUtils.collectRecordsFromUnboundedStream(DataStreamUtils.java:142)
> at
> org.apache.flink.connector.file.src.FileSourceTextLinesITCase.testContinuousTextFileSource(FileSourceTextLinesITCase.java:252)
> at
> org.apache.flink.connector.file.src.FileSourceTextLinesITCase.testContinuousTextFileSource(FileSourceTextLinesITCase.java:192)
> [...]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
