[
https://issues.apache.org/jira/browse/BEAM-10493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mani Kolbe updated BEAM-10493:
------------------------------
Description:
Observed on v2.22.0
When withMaxReadTime() is used, Beam creates an BoundedReadFromUnboundedSource
[1]. The ReadFn<T> class in BoundedReadFromUnboundedSource which is
responsible for reading records from source. You can see this class doesnt
verify if there is a recoverable checkpoint exist. Instead it always creates
Reader with checkpointMark set as null [2].
Reference:
[1]
[https://github.com/apache/beam/blob/v2.22.0/sdks/java/core/src/main/java/org/apache/beam/sdk/io/Read.java#L205]
[2][https://github.com/apache/beam/blob/v2.22.0/sdks/java/core/src/main/java/org/apache/beam/sdk/io/BoundedReadFromUnboundedSource.java#L193]
!image-2020-07-15-15-15-12-937.png|width=836,height=346!
was:
Observed on v2.22.0
When withMaxReadTime() is used, Beam creates an BoundedReadFromUnboundedSource
[1]. The ReadFn<T> class in BoundedReadFromUnboundedSource which is
responsible for reading records from source. You can see this class doesnt
verify if there is a recoverable checkpoint exist. Instead it always creates
Reader with checkpointMark set as null [2].
Reference:
[1]
[https://github.com/apache/beam/blob/v2.22.0/sdks/java/core/src/main/java/org/apache/beam/sdk/io/Read.java#L205]
[2][https://github.com/apache/beam/blob/v2.22.0/sdks/java/core/src/main/java/org/apache/beam/sdk/io/BoundedReadFromUnboundedSource.java#L193]
> Unbounded sources unable to recover from checkpointMark when
> withMaxReadTime() is used
> --------------------------------------------------------------------------------------
>
> Key: BEAM-10493
> URL: https://issues.apache.org/jira/browse/BEAM-10493
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
> Affects Versions: 2.22.0
> Reporter: Mani Kolbe
> Priority: P2
> Attachments: image-2020-07-15-15-15-01-705.png,
> image-2020-07-15-15-15-12-937.png
>
>
> Observed on v2.22.0
> When withMaxReadTime() is used, Beam creates an
> BoundedReadFromUnboundedSource [1]. The ReadFn<T> class in
> BoundedReadFromUnboundedSource which is responsible for reading records from
> source. You can see this class doesnt verify if there is a recoverable
> checkpoint exist. Instead it always creates Reader with checkpointMark set as
> null [2].
>
> Reference:
> [1]
> [https://github.com/apache/beam/blob/v2.22.0/sdks/java/core/src/main/java/org/apache/beam/sdk/io/Read.java#L205]
> [2][https://github.com/apache/beam/blob/v2.22.0/sdks/java/core/src/main/java/org/apache/beam/sdk/io/BoundedReadFromUnboundedSource.java#L193]
>
> !image-2020-07-15-15-15-12-937.png|width=836,height=346!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
