[
https://issues.apache.org/jira/browse/KAFKA-659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sönke Liebau resolved KAFKA-659.
--------------------------------
Resolution: Fixed
I'll close this for now since no-one objected here or on the mailing list, so
I'll assume my understanding of this being fixed by now is correct.
> Support request pipelining in the network server
> ------------------------------------------------
>
> Key: KAFKA-659
> URL: https://issues.apache.org/jira/browse/KAFKA-659
> Project: Kafka
> Issue Type: New Feature
> Reporter: Jay Kreps
> Priority: Major
>
> Currently the network layer in kafka will only process a single request at a
> time from a given connection. The protocol is designed to allow pipelining of
> requests which would improve latency.
> There are two changes that would have to made for this to work, in my
> understanding:
> 1. Currently once a completed request is read from a socket the server does
> not register for "read interest" again until a response is sent. The server
> would have to register for read interest immediately to allow reading more
> requests.
> 2. Currently the socket server adds all requests to a single "request
> channel" that serves as a work queue for all the background i/o threads. One
> requirement for Kafka is to do in order processing of requests from a given
> socket. This is currently achieved by not reading any new requests from a
> socket until the currently outstanding request is processed. To maintain this
> guarantee we would have to guarantee that all requests from a particular
> socket went to the same I/O thread. A simple way to do this would be to have
> work queue per I/O thread. One downside of this is that pinning requests to
> I/O threads will add latency variance--if that thread stalls due to a slow
> I/O no other thread can pick up the slack. So perhaps there is a better way
> that isn't overly complex?
> Would be good to nail down the design for this as a first step.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
