[
https://issues.apache.org/jira/browse/BEAM-12118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Whittle updated BEAM-12118:
-------------------------------
Fix Version/s: 2.30.0
Resolution: Fixed
Status: Resolved (was: Open)
> QueuingBeamFnDataClient adds polling latency to completing bundle processing
> ----------------------------------------------------------------------------
>
> Key: BEAM-12118
> URL: https://issues.apache.org/jira/browse/BEAM-12118
> Project: Beam
> Issue Type: Bug
> Components: java-fn-execution
> Reporter: Sam Whittle
> Assignee: Sam Whittle
> Priority: P2
> Fix For: 2.30.0
>
> Time Spent: 7h 50m
> Remaining Estimate: 0h
>
> Currently the inboundDataClients are registered with recieve, and they add
> data to a queue. There is no explicit indication from the clients that they
> are no longer going to add values to the queue.
> Within QueueingBeamFnDataClient.drainAndBlock the queue is therefore polled
> and if nothing is present all clients are polled to see if they are complete.
> This design makes for unfortunate tradeoffs on poll timeout:
> - cpu wasted with small timeout
> - additional latency in noticing we have completed with larger timeout
> With the existing InboundDataClient interface, we could have a separate
> thread call awaitCompletion on all of the clients and then shutdown the queue
> (adding a poison pill perhaps)
> Or we could modify InboundDataClient interface to allow registering iterest
> in when the client is done producing elements. The existing clients all seem
> based upon futures which allow that.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
