Hi,
i found after an high number of CLOSE_WAIT's in an high load production
environment that there is and problem with
sun.net.www.http.KeepAliveCache. There are two conditions when an
sun.net.www.http.HttpClient will be closed:
1) If the connection is detected to be closed by usage.
2) If the idleTimeout is expired.
We had an server that send an incorrect Header "Keep-Alive:
timeout=1200000, max=1000" and close the connection
much before the timeout.
This can cause an problem because there seems to be no limit for timeout
and the "run()" method in KeepAliveCache
does not detect if the client connection is in state CLOSE_WAIT.
I solved this by an "dirty" trick with " socket.sendUrgentData(0);".
Sadly that there is not option to use getsockopt - SO_ERROR to check if
the socket is closed.
Is there any clean way to find out if an socket is in state close_wait?
- Maybe check if we can read one byte (nonBlocking) because we are not
expecting data on idle connection.
Gruß Thomas Lußnig
p.s. My workaround
public void run() {
do {
try { Thread.sleep(LIFETIME); } catch (final
InterruptedException e) {}
synchronized (this) {
/* Remove all unused HttpClients. Starting from the
bottom of the stack (the least-recently used first).
* REMIND: It'd be nice to not remove *all* connections
that aren't presently in use.
* One could have been added a second ago that's still
perfectly valid, and we're needlessly axing it.
* But it's not clear how to do this cleanly, and doing
it right may be more trouble than it's worth.
*/
final long currentTime = System.currentTimeMillis();
final ArrayList<KeepAliveKey> keysToRemove = new
ArrayList<>();
for (final KeepAliveKey key : keySet()) {
final ClientVector v = get(key);
synchronized (v) {
for (int i = 0; i < v.size(); i++) {
final KeepAliveEntry e = v.elementAt(i);
if ((currentTime - e.idleStartTime) > v.nap) {
e.hc.closeServer();
} else {
if(i>0) v.subList(0, i).clear();
break;
}
}
if(serverSocket != null) {
final ArrayList<KeepAliveEntry>
clientsToRemove = new ArrayList<>();
for (final KeepAliveEntry e : v) {
try {
@SuppressWarnings("resource") final
Socket socket = (Socket)serverSocket.get(e.hc);
try { socket.sendUrgentData(0); }
catch(final Throwable t) {
e.hc.closeServer();
clientsToRemove.add(e);
System.out.println("ClosedHttp:
"+socket.getLocalPort()+">"+socket.getRemoteSocketAddress()+"
Timeout:"+v.nap+"/" + (currentTime-e.idleStartTime)+" => "+t.getMessage());
}
} catch(final Throwable t) {
t.printStackTrace(System.out); }
}
for(final KeepAliveEntry e :
clientsToRemove) { v.remove(e); }
} // FOR
if (v.size() == 0) { keysToRemove.add(key); }
}
}
for(final KeepAliveKey key : keysToRemove) {
removeVector(key); }
}
} while (size() > 0);
return;
}
