[
https://issues.apache.org/jira/browse/BEAM-11833?focusedWorklogId=554897&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-554897
]
ASF GitHub Bot logged work on BEAM-11833:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 19/Feb/21 20:03
Start Date: 19/Feb/21 20:03
Worklog Time Spent: 10m
Work Description: boyuanzz commented on pull request #14013:
URL: https://github.com/apache/beam/pull/14013#issuecomment-782309427
> Hi Boyuan,
> I think that it is incorrect to call
`tracker.currentRestriction().getWatermark()` when tryClaim returns `false`.
The returned value of `BoundedWindow.TIMESTAMP_MAX_VALUE` seems not to be the
actual watermark. Why would we want to feed this value into the watermark
estimator?
> Regarding the hold - if we set the hold to TIMESTAMP_MAX_VALUE after
restriction split, that could cause the output watermark to skip past the
watermark of the residual restriction (watermark of the remaining data in the
reader), which seems incorrect. That could cause emitting late data even though
the reader's watermark is correct. Am I missing something?
The reason why we want call
`watermarkEstimator.setWatermark(currentRestriction.getWatermark());` when
`tryClaim()` returns `false` is for tracking watermark when returning
ProcessContinuation.resume(). It could happen when there is no output records
from reader and we want to read again later.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 554897)
Time Spent: 1h 10m (was: 1h)
> UnboundedSourceAsSDFRestrictionTracker reports incorrect watermark after
> failed claim
> -------------------------------------------------------------------------------------
>
> Key: BEAM-11833
> URL: https://issues.apache.org/jira/browse/BEAM-11833
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
> Affects Versions: 2.28.0
> Reporter: Jan Lukavský
> Assignee: Jan Lukavský
> Priority: P1
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> After being split by call to {{trySplit}}, the watermark reported by
> {{UnboundedSourceAsSDFRestrictionTracker.currentRestriction().getWatermark()}}
> is BoundedWindow.TIMESTAMP_MAX_VALUE, which is incorrect.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
