[
https://issues.apache.org/jira/browse/BEAM-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990731#comment-15990731
]
ASF GitHub Bot commented on BEAM-2072:
--------------------------------------
Github user asfgit closed the pull request at:
https://github.com/apache/beam/pull/2698
> MicrobatchSource.reader stops reading after reaching maxNumRecords for the
> first time
> -------------------------------------------------------------------------------------
>
> Key: BEAM-2072
> URL: https://issues.apache.org/jira/browse/BEAM-2072
> Project: Beam
> Issue Type: Bug
> Components: runner-spark
> Affects Versions: Not applicable
> Reporter: Stas Levin
> Assignee: Stas Levin
>
> {{MicrobatchSource.Reader}} stops reading further data after the first batch
> in which it has reached {{maxNumRecords}}. Subsequent read request will check
> the {{recordsRead}} and conclude that no data can be read.
> The root cause being that {{recordsRead}} is maintained throughout the
> reader's lifecycle, and can span multiple (spark) batches.
> We should reset {{recordsRead}} every time the reader starts reading a fresh
> batch.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
