magibney commented on code in PR #2313:
URL: https://github.com/apache/solr/pull/2313#discussion_r1571200400
##########
solr/solrj/src/java/org/apache/solr/client/solrj/impl/Http2SolrClient.java:
##########
@@ -1017,19 +1023,21 @@ public String getBaseURL() {
}
private static class AsyncTracker {
- private static final int MAX_OUTSTANDING_REQUESTS = 1000;
+ private static final int DEFAULT_MAX_OUTSTANDING_REQUESTS = 1000;
// wait for async requests
private final Phaser phaser;
+ private final int maxOutstandingRequests;
// maximum outstanding requests left
private final Semaphore available;
private final Request.QueuedListener queuedListener;
private final Response.CompleteListener completeListener;
- AsyncTracker() {
+ AsyncTracker(int maxOutstandingRequests) {
// TODO: what about shared instances?
+ this.maxOutstandingRequests = maxOutstandingRequests;
phaser = new Phaser(1);
- available = new Semaphore(MAX_OUTSTANDING_REQUESTS, false);
+ available = new Semaphore(maxOutstandingRequests, false);
Review Comment:
It's been a while since I was in this code, but iirc (and thinking back to
the "relevant discussion" comment linked above), there are _2_ queues: 1 is a
per-destination buffer (before send), the other is the actual "request enqueued
to be sent" queue, which is the one to which the semaphore applies, and covers
_all_ destinations.
So I suspect that what happened was that the second queue (to which the
semaphore applies) is actually throttling the requests, which then causes the
buffers to back up to the point where the requests get proactively rejected,
which is what you would have been seeing in production.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
