Found the problem. Code was leaking connections when server returned 400 with some content in it. It was not consuming the content from stream.
Thanks for your help. jaikit On Tuesday, July 9, 2013 6:29 PM, Jaikit Savla <[email protected]> wrote: I tried to reproduce CLOSE_WAIT scenario on client side - but was not able to reproduce using HttpClient. However I was able to reproduce using basic sockets. Basic socket: SERVER: Socket socket = new ServerSocket(12000).accept(); OutputStream out = socket.getOutputStream(); out.write("HTTP/1.1 200 OK\n\nWorkerRunnable\n".getBytes()); out.flush(); out.close(); CLIENT: Socket socket = new Socket(InetAddress.getByName("localhost"), 12000); InputStream in = socket.getInputStream(); int c; while ((c = in.read()) != -1) { System.out.write(c); } System.out.flush(); // should now be in CLOSE_WAIT Thread.sleep(Integer.MAX_VALUE); // After this sleep - client goes into CLOSE_WAIT state But when I replace the client with HTTP Post request HTTPCLIENT: HttpClientFactory factory = new HttpClientFactory(); HttpClient httpclient = factory.getHttpClient(config); final String url = "http://localhost:12000";; HttpPost postMethod = new HttpPost(url); postMethod.addHeader("Content-Type", "application/json"); postMethod.setEntity(new ByteArrayEntity("{\"name\" : \"jaikit\"".getBytes())); HttpResponse response = httpclient.execute(postMethod); InputStream lsgInputStream = response.getEntity().getContent(); int c; while ((c = lsgInputStream.read()) != -1) { System.out.write(c); } Here - as soon as I close the socket on server - the entire connection gets closed and netstat does not show any trace of it. Any suggestions ? On Tuesday, July 9, 2013 11:13 AM, Jaikit Savla <[email protected]> wrote: Hello Users, I am using PoolingClientConnectionManager for my HttpClient instance and have set 512 connections per route. On one of my production box, all 512 connections are in CLOSE_WAIT state. I also have IdleConnectionMonitorThread running which claims Idle connections (30s) and expired connections. I verified that the thread is running on my jstack. However it is not reclaiming the 512 connections for last 5 days. Has anyone encountered similar issue ? have any clue on how to debug ? I have simulating the CLOSE_WAIT scenario in my unit test and verifying if IdleConnectionMonitorThread is reclaiming the idle connections. Ref: http://stackoverflow.com/questions/4724193/how-can-i-ensure-that-my-httpclient-4-1-does-not-leak-sockets Any pointers on how to debug this is appreciated. Thanks, Jaikit
