Hi, On Mon, Oct 3, 2022 at 9:52 PM Tomas Fernandez Lobbe <tflo...@gmail.com> wrote: > > Hello, > I was looking at the way SolrJ (Solr's client library) is using Jetty to > issue requests to Solr server (using Jetty server). As of now, Solr is using > Jetty 9.4.48 in main (don't know if that's relevant to my question, unless > behavior in this area recently changed). > From what I could see in the code and docs, the way Jetty client handles a > new request to a particular destination is: > 1. Add request to a queue > 2. Attempt to get a connection from the pool > 3. If successful getting a connection, use that to send the request, > otherwise, just exit, something else will grab the request from the queue and > send it when connections are available. > > Caller is expected to provide listener(s) that will receive a callback for > events happening in the request/response. Solr is using an > "InputStreamResponseListener" like[1]: > > InputStreamResponseListener listener = new > InputStreamResponseListener(); > req.send(listener); > Response response = listener.get(idleTimeout, TimeUnit.MILLISECONDS); > InputStream is = listener.getInputStream(); > return processErrorsAndResponse(solrRequest, parser, response, is); > > This pattern looks similar to what's in the Jetty docs, and even similar to > what the blocking APIs in Jetty client itself are using. > > The thread that sends the request (because it was successful acquiring the > connection from the pool) will continue fetching requests from the queue and > sending those for as long as there are requests in the queue. My question is, > can't this be problematic as the queue grows? Can the thread that sends a > request "FOO" be stuck sending other requests from the queue for longer than > the request "FOO" takes to be processed on the server side, and a response is > available on the client (at which point the listener.get(...) would return > immediately). >
When thread T1 sends a request, it acquires connection C1 and sends the request bytes. In order for T1 to come back and find another request queued, another thread T2 must have queued it, found no connection available, and initiated the opening of a second connection C2. T1 would try to acquire a connection but it cannot because C2 is not opened yet, so it returns. When C2 is opened, another thread, say T3, runs the after-opening code, which would poll a request from the queue and try to send it. T1 may steal C2 from T3, and do additional work that is not pertinent to the first request, but it's typically a quite rare case. For T1 to find a large queue of requests, it would have to compete with a large number of connection-opening threads, and the chances that T1 steals from all of them are minimal. In case of a single connection per destination, T1 can only send a second request if the first request/response cycle is completed (for HTTP1). It's typical that it is not due to network latency, so T1 would return after sending the first request. For multiplexed protocols such as HTTP2, sender thread T2 would find a connection, so it would typically send the request. Again, it may be possible that T1 steals the request from T2, but that again should be a rare case. I guess there is a degenerate case in HTTP1 where, with 1 connection per destination, the responses arrive so fast that T1 is always busy sending, but things must be aligned pretty precisely for that to happen. > Would it make more sense for the `send` to happen on the client's executor, > something like? > > httpClient.getExecutor().execute(() -> req.send(listener)); This will always pay the cost of a CPU context switch also for the non-degenerate cases, so it's not particularly desirable. Calling execute() for request.send(listener) would not be the right place to cope with the degenerate case -- the execute() should be called from much deeper in the implementation when T1 loops to try to send another request and it has found another connection. Feels like quite some work for little return to cover a rare case. Do you have evidence that you are hitting the degenerate case often? -- Simone Bordet ---- http://cometd.org http://webtide.com Developer advice, training, services and support from the Jetty & CometD experts. _______________________________________________ jetty-users mailing list jetty-users@eclipse.org To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jetty-users
