Abacn commented on code in PR #36750:
URL: https://github.com/apache/beam/pull/36750#discussion_r2518785151
##########
sdks/java/core/src/main/java/org/apache/beam/sdk/fn/splittabledofn/RestrictionTrackers.java:
##########
@@ -101,8 +141,16 @@ protected RestrictionTrackerObserverWithProgress(
}
@Override
- public synchronized Progress getProgress() {
- return ((HasProgress) delegate).getProgress();
+ public Progress getProgress() {
Review Comment:
There were cases of inaccurate progress confused runner: prior to #26352
boundedsource as restriction tracker (the scenario of Dataflow runner v2
evaluating boundedsource) always reports zero progress, and the effect was
slowness and runner not splitting sources, from the PR description.
At least if the stale progress is zero it should be minimum concern
(happened before). If the stale progress is a positive number, this is similar
to long running DoFn.process we often observes (in case of Dataflow, there is a
straggler alert) but the pipeline can still work as intended. Let me write a
pipeline and test this scenario, would a stale positive prgress make runner
eagerly issue trySplit, or other unexpected behavior
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
