[
https://issues.apache.org/jira/browse/KAFKA-48?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13140231#comment-13140231
]
Taylor Gautier commented on KAFKA-48:
-------------------------------------
Hi - please keep in mind the use case where a consumer is interested in more
than one topic.
This feature if implemented only for one topic will not be useful for this use
case - assuming it's infeasible to open multiple tcp connections.
The first proposal I have is to allow the request to contain a list of topics.
However, upon consideration, this would require the response to also be
adjusted such that it would contain the name of the topic, otherwise it would
be next to impossible to ascertain which topic the response corresponds to -
well it could be done such that the response is returned in the same way as the
request was requested, and for topics with no messages, an empty response is
given, but this seems pretty bad from a network bandwidth standpoint.
So my final proposal would be to introduce an epoll like request/response. The
consumer would submit a request with a list of interested topics, and the
response would be a topic and # of messages available on that topic when the
topic(s) have messages.
The advantage to this solution is that it would be entirely backward
compatible, since you would simply introduce a new request/response pair and it
would also allow the consumer to decide which topics to poll (or pull) from
first, so that it could prioritize, if it wanted.
Finally, I like the idea of allowing the consumer to specify a min # of
messages required to trigger the poll, you might want to copy the pattern you
already setup for log flushing, e.g. max time and/or min # of messages. So the
request might look like:
list-of : topic-name:min msgs:max time
and the response might be:
list-of : topic-name:# msgs available
> Implement optional "long poll" support in fetch request
> -------------------------------------------------------
>
> Key: KAFKA-48
> URL: https://issues.apache.org/jira/browse/KAFKA-48
> Project: Kafka
> Issue Type: Bug
> Reporter: Alan Cabrera
> Assignee: Jay Kreps
> Attachments: KAFKA-48-socket-server-refactor-draft.patch
>
>
> Currently, the fetch request is non-blocking. If there is nothing on the
> broker for the consumer to retrieve, the broker simply returns an empty set
> to the consumer. This can be inefficient, if you want to ensure low-latency
> because you keep polling over and over. We should make a blocking version of
> the fetch request so that the fetch request is not returned until the broker
> has at least one message for the fetcher or some timeout passes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
