[
https://issues.apache.org/jira/browse/STORM-4016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Abreu resolved STORM-4016.
------------------------------
Resolution: Fixed
> Kafka spout: start using poll(Duration)
> ---------------------------------------
>
> Key: STORM-4016
> URL: https://issues.apache.org/jira/browse/STORM-4016
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-kafka
> Affects Versions: 2.6.0
> Reporter: Rui Abreu
> Assignee: Rui Abreu
> Priority: Minor
> Fix For: 2.7.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Kafka has deprecated poll(long) in favour of poll(Duration): [KIP-266: Fix
> consumer indefinite blocking
> behavior|https://cwiki.apache.org/confluence/display/KAFKA/KIP-266%3A+Fix+consumer+indefinite+blocking+behavior]
> There is also an interesting report about the behaviour of it poll:
> _The pre-existing variant {{poll(long timeout)}} would block indefinitely for
> metadata updates if they were needed, then it would issue a fetch and poll
> for {{timeout}} ms for new records. The initial indefinite metadata block
> caused applications to become stuck when the brokers became unavailable. The
> existence of the timeout parameter made the indefinite block especially
> unintuitive._
> _We will add a new method {{poll(Duration timeout)}} with the semantics:_
> # _iff a metadata update is needed:_
> ## _send (asynchronous) metadata requests_
> ## _poll for metadata responses (counts against timeout)_
> *** _if no response within timeout, return an empty collection immediately_
> # _if there is fetch data available, return it immediately_
> # _if there is no fetch request in flight, send fetch requests_
> # _poll for fetch responses (counts against timeout)_
> ** _if no response within timeout, return an empty collection (leaving async
> fetch request for the next poll)_
> ** _if we get a response, return the response_
> _We will deprecate the original method, {{{}poll(long timeout){}}}, and we
> will not change its semantics, so it remains:_
> # _iff a metadata update is needed:_
> ## _send (asynchronous) metadata requests_
> ## _poll for metadata responses indefinitely until we get it_
> # _if there is fetch data available, return it immediately_
> # _if there is no fetch request in flight, send fetch requests_
> # _poll for fetch responses (counts against timeout)_
> ** _if no response within timeout, return an empty collection (leaving async
> fetch request for the next poll)_
> ** _if we get a response, return the response_
> _One notable usage is prohibited by the new {{{}poll{}}}: previously, you
> could call {{poll(0)}} to block for metadata updates, for example to
> initialize the client, supposedly without fetching records. Note, though,
> that this behavior is not according to any contract, and there is no
> guarantee that {{poll(0)}} won't return records the first time it's called.
> Therefore, it has always been unsafe to ignore the response._
>
>
> [https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75974886|http://example.com]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
