[ https://issues.apache.org/jira/browse/STORM-4016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17799099#comment-17799099 ]
Rui Abreu commented on STORM-4016: ---------------------------------- Pull request: https://github.com/apache/storm/pull/3612/commits > Kafka spout: start using poll(Duration) > --------------------------------------- > > Key: STORM-4016 > URL: https://issues.apache.org/jira/browse/STORM-4016 > Project: Apache Storm > Issue Type: Improvement > Affects Versions: 2.6.0 > Reporter: Rui Abreu > Assignee: Rui Abreu > Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > Kafka has deprecated poll(long) in favour of poll(Duration): [KIP-266: Fix > consumer indefinite blocking > behavior|https://cwiki.apache.org/confluence/display/KAFKA/KIP-266%3A+Fix+consumer+indefinite+blocking+behavior] > There is also an interesting report about the behaviour of it poll: > _The pre-existing variant {{poll(long timeout)}} would block indefinitely for > metadata updates if they were needed, then it would issue a fetch and poll > for {{timeout}} ms for new records. The initial indefinite metadata block > caused applications to become stuck when the brokers became unavailable. The > existence of the timeout parameter made the indefinite block especially > unintuitive._ > _We will add a new method {{poll(Duration timeout)}} with the semantics:_ > # _iff a metadata update is needed:_ > ## _send (asynchronous) metadata requests_ > ## _poll for metadata responses (counts against timeout)_ > *** _if no response within timeout, return an empty collection immediately_ > # _if there is fetch data available, return it immediately_ > # _if there is no fetch request in flight, send fetch requests_ > # _poll for fetch responses (counts against timeout)_ > ** _if no response within timeout, return an empty collection (leaving async > fetch request for the next poll)_ > ** _if we get a response, return the response_ > _We will deprecate the original method, {{{}poll(long timeout){}}}, and we > will not change its semantics, so it remains:_ > # _iff a metadata update is needed:_ > ## _send (asynchronous) metadata requests_ > ## _poll for metadata responses indefinitely until we get it_ > # _if there is fetch data available, return it immediately_ > # _if there is no fetch request in flight, send fetch requests_ > # _poll for fetch responses (counts against timeout)_ > ** _if no response within timeout, return an empty collection (leaving async > fetch request for the next poll)_ > ** _if we get a response, return the response_ > _One notable usage is prohibited by the new {{{}poll{}}}: previously, you > could call {{poll(0)}} to block for metadata updates, for example to > initialize the client, supposedly without fetching records. Note, though, > that this behavior is not according to any contract, and there is no > guarantee that {{poll(0)}} won't return records the first time it's called. > Therefore, it has always been unsafe to ignore the response._ > > > [https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75974886|http://example.com] -- This message was sent by Atlassian Jira (v8.20.10#820010)