damondouglas commented on issue #31338:
URL: https://github.com/apache/beam/issues/31338#issuecomment-2146300399
@lostluck so far I found that job being stuck, reported by the panic, may be
due to the upstream watermark. In the following logs, I see a reporting of
"insufficient upstream watermark" i.e. `inputW == upstreamW` in the bundleReady
method. One hypothesis generating idea would be to collect which Java validates
runner tests are leading to the panic to detect a potential pattern.
```
{"time":"2024-06-03T23:16:17.984115241Z","level":"DEBUG","msg":"bundleReady:
insufficient upstream
watermark","stage":"stage-001","watermark":{"upstream":9223372036854775,"input":9223372036854775}}
{"time":"2024-06-03T23:16:17.984145011Z","level":"DEBUG","msg":"Bundles:
nothing in progress after
advance","advanced":{"stage-001":{}},"refreshCount":1,"pendingElementCount":2}
{"time":"2024-06-03T23:16:17.984161971Z","level":"DEBUG","msg":"bundleReady:
insufficient upstream
watermark","stage":"stage-006","watermark":{"upstream":-9223372036854775,"input":-9223372036854775}}
{"time":"2024-06-03T23:16:17.984172671Z","level":"DEBUG","msg":"Bundles:
nothing in progress and no refreshes","pendingElementCount":2}
panic: nothing in progress and no refreshes with non zero pending elements: 2
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
