boyuanzz edited a comment on pull request #14013: URL: https://github.com/apache/beam/pull/14013#issuecomment-782309427
> Hi Boyuan, > I think that it is incorrect to call `tracker.currentRestriction().getWatermark()` when tryClaim returns `false`. The returned value of `BoundedWindow.TIMESTAMP_MAX_VALUE` seems not to be the actual watermark. Why would we want to feed this value into the watermark estimator? > Regarding the hold - if we set the hold to TIMESTAMP_MAX_VALUE after restriction split, that could cause the output watermark to skip past the watermark of the residual restriction (watermark of the remaining data in the reader), which seems incorrect. That could cause emitting late data even though the reader's watermark is correct. Am I missing something? The reason why we want to call `watermarkEstimator.setWatermark(currentRestriction.getWatermark());` when `tryClaim()` returns `false` is for tracking watermark when returning ProcessContinuation.resume(). It could happen when there is no output records from reader and we want to read again later. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
