[
https://issues.apache.org/jira/browse/BEAM-7442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851183#comment-16851183
]
Akshay Iyangar edited comment on BEAM-7442 at 5/29/19 6:47 PM:
---------------------------------------------------------------
Yes .. I have taken those comments into consideration and should have the PR
out . shortly.. like i was not able to assign the ticket to myself.. will
appreciate if someone can help me with that.
Cool !!! you have already added the changes.. sweet !!! didn't see that..
was (Author: aiyangar):
Yes .. I have taken those comments into consideration and should have the PR
out . shortly.. like i was not able to assign the ticket to myself.. will
appreciate if someone can help me with that.
> Bounded Reads for Flink Runner fails with OOM
> ---------------------------------------------
>
> Key: BEAM-7442
> URL: https://issues.apache.org/jira/browse/BEAM-7442
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Reporter: Akshay Iyangar
> Assignee: Maximilian Michels
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When Flink runner is reading from a bounded source and if the total number of
> files are huge and the count is more. FlinkRunner throws an OOM error. This
> is happening because the current implementation doesn't read them
> sequentially but simultaneously thus causing all of the files to be in memory
> which quickly breaks the cluster.
> Solution : To wrap `UnboundedReadFromBoundedSource` class by a wrapper to see
> that when the stream is a bounded source we make it read it sequentially
> using a queue.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
