Mani Kolbe created BEAM-10493:
---------------------------------
Summary: Unbounded sources unable to recover from checkpointMark
when withMaxReadTime() is used
Key: BEAM-10493
URL: https://issues.apache.org/jira/browse/BEAM-10493
Project: Beam
Issue Type: Bug
Components: sdk-java-core
Affects Versions: 2.22.0
Reporter: Mani Kolbe
Observed on v2.22.0
When withMaxReadTime() is used, Beam creates an BoundedReadFromUnboundedSource
[1]. The ReadFn<T> class in BoundedReadFromUnboundedSource which is
responsible for reading records from source. You can see this class doesnt
verify if there is a recoverable checkpoint exist. Instead it always creates
Reader with checkpointMark set as null [2].
Reference:
[1]
[https://github.com/apache/beam/blob/v2.22.0/sdks/java/core/src/main/java/org/apache/beam/sdk/io/Read.java#L205]
[2][https://github.com/apache/beam/blob/v2.22.0/sdks/java/core/src/main/java/org/apache/beam/sdk/io/BoundedReadFromUnboundedSource.java#L193]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
