Hello! Our reverse proxy uses the Async Client pool to handle connections to
backend servers. We've been tracking a problem for a while where we observe
the initial TCP connection is made, but no thread is available to handle the
SSL setup before a 10 second timeout expires. We get into trouble because some
of our backend servers are very slow, and some of our clients download very
slowly.
I'm experimenting with a patch to AbstractMultiworkerIOReactor.addChannel() to
determine whether the next dispatcher thread is "busy." My first try was to
look at bufferedSessions from the BaseIOReactor, and go through the list of
dispatchers one time to see if I can find a free one.
int i = Math.abs(this.currentWorker++ % this.workerCount);
for (int j = 0; j < this.workerCount; j++) {
if (this.dispatchers[i].getSessionCount() == 0) {
break;
}
i = Math.abs(this.currentWorker++ % this.workerCount);
}
this.dispatchers[i].addChannel(entry);
This seems to help us in MOST of the cases we see this issue in production, but
there still seem to be a small number of threads which collide. I'm testing a
different version which looks at AbstractIOReactor "sessions" to determine
thread busy state, but it never seems to show more than "1" session if I look
at the size after piling up slow connections on top of each other.
I have two questions:
Is there a better way to determine whether a thread is busy?
Would you be willing to accept a patch to make the dispatchers array in
AbstractMultiworkerIOReactor "protected" so I can implement my own
ConnectingIOReactor that overrides addChannel() with my own thread selection
model?
Thanks a lot for your help and for providing such a great library to the
community!
- edan
