[
https://issues.apache.org/jira/browse/BEAM-10503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17232350#comment-17232350
]
Beam JIRA Bot commented on BEAM-10503:
--------------------------------------
This issue is P2 but has been unassigned without any comment for 60 days so it
has been labeled "stale-P2". If this issue is still affecting you, we care!
Please comment and remove the label. Otherwise, in 14 days the issue will be
moved to P3.
Please see https://beam.apache.org/contribute/jira-priorities/ for a detailed
explanation of what these priorities mean.
> Document expectations around UnboundedReader advance interface
> --------------------------------------------------------------
>
> Key: BEAM-10503
> URL: https://issues.apache.org/jira/browse/BEAM-10503
> Project: Beam
> Issue Type: Task
> Components: sdk-java-core
> Reporter: Aaron Meihm
> Priority: P2
> Labels: Clarified, P2, stale-P2
>
> We have implemented some custom IO classes based on
> UnboundedReader/UnboundedSource. These work as expected, but while doing this
> I noticed a few things that didn't seem to be well documented and I'm not
> sure if they behave as would be anticipated.
> With the direct runner, when advance returns false repeatedly it appears as
> though direct runner will apply an increasing backoff to repeated calls to
> advance until it returns true, at which point the backoff is reset. This
> seems to be what I'd expect.
> However when the same code is used with Dataflow, advance will be called
> multiple times a second for a single given UnboundedSource instance with no
> backoff continuously. With more then one instance/worker this can start to
> produce additional CPU load.
> I'm a bit unclear what the right way to do this is, for example should you
> sleep in advance? I assume not, but it would be great if there was
> documentation around this interface, especially around the differing behavior
> of the various runners here and what the right way to implement this is to
> ensure efficient resource usage when no events are available from the
> underlying source.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
