Oleg, Thanks for the response.
I agree that it doesn't necessarily appear to be HttpClient code itself.. because you are just calling to the jdk inputstream.read. I did some research and saw that we are sending back Transfer-Encoding: chunked and am able to spot (in a network capture) on all the good calls the following: 30 0D 0A 0D 0A (which is 0 and the CRLF). We haven't reproduced the error yet while doing a network capture so I'll respond with an update as soon as we do. In the mean time, am I correct in removing the ois.close() from my code and letting httpclient close it with the AutoCloseInputStream when I call releaseConnection? Thanks, Ken -----Original Message----- From: Oleg Kalnichevski [mailto:[email protected]] Sent: Thursday, November 19, 2009 2:50 PM To: HttpClient User Discussion Subject: Re: Infinite loop in ChunkedInputStream.close [email protected] wrote: > We are using httpclient 3.1 to send a post and get back serialized java > objects (think of something analogous to Spring Remoting). > We deploy this client code to many client machines talking thru > loadbalancers communicating with Weblogic 10MP1 servers. > The clients in this test are on windows, running jdk5 thru mercury > loadrunner (performance testing software). > > We've been testing anywhere between 45 and 90tps and the vast majority > of the transactions are successful. > > About 1 transaction every half hour ends up with a > java.net.SocketException: Connection reset. > The loadbalancer will close the socket after 60 seconds. > The reason we get that exception (and not a different one) is that we > have the client timeout set up higher than 60 seconds (just for trying > to expose this issue) and the loadbalancer has a 60 second timeout. > > Here is what we have learned so far. > > We get the response body as stream: > InputStream responseBody = > postMethod.getResponseBodyAsStream(); > > We then send it thru a custom deserializer, we know that the object in > the response is being read because the log right after it is being > printed out: > However we have a close on that input stream in the finally. I'm > probably going to remove that close (since I read elsewhere that its > taken care of by the postMethod.releaseConnection(). > try { > Object object = ois.readObject(); > LOG.debug(METHOD_NAME + "ois.readObject completed"); > return object; > } finally { > ois.close(); > LOG.debug(METHOD_NAME + "ois.close completed"); > } > > The ois.close() is calling AutoCloseInputStream.close, calling > ChunkedInputStream.close which is trying to exhaust the stream by > calling exhaustInputStream which eventually calls SocketInputStream.read > and just sits there until the connection reset happens. > > Any clue as to what would cause it to hang there? > I seriously doubt this has anything to do with a tight loop in HttpClient code. Most likely the server fails to send the closing chunk, which causes the input stream to get blocked in a read operation. > The confusing factors are that the object seems to be fully read (so I > would assume there is nothing left in the stream) and we average a > couple thousand good transactions for every 1 occurrence of a bad one. > > Let me know if I need the full headers and wire. We have only the > header, not the content turned on because the volume of logging just > kills the clients when turned up to trace. I am afraid the only reliable way of establishing the cause would be getting a complete wire log. > Also, we will be looking to migrate eventually to httpclient 4, but that > requires jdk5 and right now we have clients that are still > (unfortunately) jdk1.4 so we can't quite make that leap yet. > I have network captures as well, if you think you know what I should > look for please advise and I will do so. > If you can find out where the session gets stuck see if the server sends the closing chunk back to the client. The chunk should look like <CR><LF>0<CR><LF> Oleg > Thanks, > Ken > > Here is the pertinent httpclient log. > Notice the 1 minute gap in log statements between 15:54:25:723 EST and > 15:55:25:756 EST > Also be aware that the reason you see logs in between the issue and the > time you see the stacktrace is that the postMethod.releaseConnection() > is in a finally, so of course it will complete before the exception is > propogated. > > System.err: 2009/11/17 15:54:25:707 EST [TRACE] PostMethod - enter > PostMethod.clearRequestBody() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] EntityEnclosingMethod - > enter EntityEnclosingMethod.clearRequestBody() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [DEBUG] > RemoteServiceExecutorImpl - executeRequest calling > httpClient.ExecuteMethod > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpClient - enter > HttpClient.executeMethod(HttpMethod) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpClient - enter > HttpClient.executeMethod(HostConfiguration,HttpMethod,HttpState) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] > MultiThreadedHttpConnectionManager - enter > HttpConnectionManager.getConnectionWithTimeout(HostConfiguration, long) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [DEBUG] > MultiThreadedHttpConnectionManager - > HttpConnectionManager.getConnection: config = > HostConfiguration[host=http://my-loadbalancer:9810], timeout = 0 > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] > MultiThreadedHttpConnectionManager - enter > HttpConnectionManager.ConnectionPool.getHostPool(HostConfiguration) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] > MultiThreadedHttpConnectionManager - enter > HttpConnectionManager.ConnectionPool.getHostPool(HostConfiguration) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [DEBUG] > MultiThreadedHttpConnectionManager - Allocating new connection, > hostConfig=HostConfiguration[host=http://my-loadbalancer:9810] > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodDirector - > Attempt number 1 to process request > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.open() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [DEBUG] HttpConnection - Open > connection to my-loadbalancer:9810 > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodBase - enter > HttpMethodBase.execute(HttpState, HttpConnection) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodBase - enter > HttpMethodBase.writeRequest(HttpState, HttpConnection) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodBase - enter > HttpMethodBase.writeRequestLine(HttpState, HttpConnection) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodBase - enter > HttpMethodBase.generateRequestLine(HttpConnection, String, String, > String, String) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [DEBUG] header - >> "POST > /cms.rcts/2009.9.1.006/ccs/prf.lr/2008.8.1/CDC/essd-remote/ > HTTP/1.1[\r][\n]" > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.print(String) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.write(byte[]) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.write(byte[], int, int) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodBase - enter > HttpMethodBase.writeRequestHeaders(HttpState,HttpConnection) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] EntityEnclosingMethod - > enter EntityEnclosingMethod.addRequestHeaders(HttpState, HttpConnection) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] ExpectContinueMethod - > enter ExpectContinueMethod.addRequestHeaders(HttpState, HttpConnection) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodBase - enter > HttpMethodBase.addRequestHeaders(HttpState, HttpConnection) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodBase - enter > HttpMethodBase.addUserAgentRequestHeaders(HttpState, HttpConnection) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodBase - enter > HttpMethodBase.addHostRequestHeader(HttpState, HttpConnection) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [DEBUG] HttpMethodBase - Adding > Host request header > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodBase - enter > HttpMethodBase.addCookieRequestHeader(HttpState, HttpConnection) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpState - enter > HttpState.getCookies() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodBase - enter > HttpMethodBase.addProxyConnectionHeader(HttpState, HttpConnection) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] EntityEnclosingMethod - > enter EntityEnclosingMethod.addContentLengthRequestHeader(HttpState, > HttpConnection) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] EntityEnclosingMethod - > enter EntityEnclosingMethod.getRequestContentLength() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] PostMethod - enter > PostMethod.hasRequestContent() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] EntityEnclosingMethod - > enter EntityEnclosingMethod.hasRequestContent() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpMethodBase - > HttpMethodBase.addRequestHeader(Header) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [DEBUG] header - >> > "Content-Type: application/x-java-serialized-object[\r][\n]" > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.print(String) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.write(byte[]) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.write(byte[], int, int) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [DEBUG] header - >> > "Content-Encoding: none[\r][\n]" > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.print(String) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.write(byte[]) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.write(byte[], int, int) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [DEBUG] header - >> > "Serialization-Type: java[\r][\n]" > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.print(String) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.write(byte[]) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.write(byte[], int, int) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [DEBUG] header - >> "User-Agent: > Jakarta Commons-HttpClient/3.1[\r][\n]" > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.print(String) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.write(byte[]) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.write(byte[], int, int) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [DEBUG] header - >> "Host: > my-loadbalancer:9810[\r][\n]" > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.print(String) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.write(byte[]) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.write(byte[], int, int) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [DEBUG] header - >> > "Content-Length: 3051[\r][\n]" > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.print(String) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.write(byte[]) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.write(byte[], int, int) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.writeLine() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.write(byte[]) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] HttpConnection - enter > HttpConnection.write(byte[], int, int) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [DEBUG] header - >> "[\r][\n]" > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] EntityEnclosingMethod - > enter EntityEnclosingMethod.writeRequestBody(HttpState, HttpConnection) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] PostMethod - enter > PostMethod.hasRequestContent() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] EntityEnclosingMethod - > enter EntityEnclosingMethod.hasRequestContent() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] EntityEnclosingMethod - > enter EntityEnclosingMethod.getRequestContentLength() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] PostMethod - enter > PostMethod.hasRequestContent() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:707 EST [TRACE] EntityEnclosingMethod - > enter EntityEnclosingMethod.hasRequestContent() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpConnection - enter > HttpConnection.getRequestOutputStream() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [DEBUG] EntityEnclosingMethod - > Request body sent > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpConnection - enter > HttpConnection.flushRequestOutputStream() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpMethodBase - enter > HttpMethodBase.readResponse(HttpState, HttpConnection) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpMethodBase - enter > HttpMethodBase.readStatusLine(HttpState, HttpConnection) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpConnection - enter > HttpConnection.readLine() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter > HttpParser.readLine(InputStream, String) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter > HttpParser.readRawLine() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [DEBUG] header - << "HTTP/1.1 > 200 OK[\r][\n]" > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [DEBUG] header - << "HTTP/1.1 > 200 OK[\r][\n]" > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpMethodBase - enter > HttpMethodBase.readResponseHeaders(HttpState,HttpConnection) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpConnection - enter > HttpConnection.getResponseInputStream() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter > HeaderParser.parseHeaders(InputStream, String) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter > HttpParser.readLine(InputStream, String) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter > HttpParser.readRawLine() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [DEBUG] header - << "Date: Tue, > 17 Nov 2009 20:54:25 GMT[\r][\n]" > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter > HttpParser.readLine(InputStream, String) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter > HttpParser.readRawLine() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [DEBUG] header - << > "Transfer-Encoding: chunked[\r][\n]" > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter > HttpParser.readLine(InputStream, String) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter > HttpParser.readRawLine() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [DEBUG] header - << > "Content-Type: application/x-java-serialized-object[\r][\n]" > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter > HttpParser.readLine(InputStream, String) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter > HttpParser.readRawLine() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [DEBUG] header - << > "Content-Encoding: none[\r][\n]" > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter > HttpParser.readLine(InputStream, String) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter > HttpParser.readRawLine() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [DEBUG] header - << > "Serialization-Type: java[\r][\n]" > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter > HttpParser.readLine(InputStream, String) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpParser - enter > HttpParser.readRawLine() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [DEBUG] header - << "[\r][\n]" > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpMethodBase - enter > HttpMethodBase.processResponseHeaders(HttpState, HttpConnection) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpMethodBase - enter > HttpMethodBase.processCookieHeaders(Header[], HttpState, HttpConnection) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpMethodBase - enter > HttpMethodBase.readResponseBody(HttpState, HttpConnection) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpMethodBase - enter > HttpMethodBase.readResponseBody(HttpConnection) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpConnection - enter > HttpConnection.getResponseInputStream() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpMethodBase - enter > HttpMethodBase.canResponseHaveBody(int) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HeaderElement - enter > HeaderElement.parseElements(String) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HeaderElement - enter > HeaderElement.parseElements(char[]) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [TRACE] HttpConnection - enter > HttpConnection.isResponseAvailable(int) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [DEBUG] > RemoteServiceExecutorImpl - executeRequesthttpClient.ExecuteMethod > completed > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:54:25:723 EST [DEBUG] > RemoteServiceExecutorImpl - > executeRequestpostMethod.getResponseBodyAsStream completed > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:55:25:756 EST [TRACE] HttpConnection - enter > HttpConnection.releaseConnection() > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:55:25:756 EST [DEBUG] HttpConnection - > Releasing connection back to connection manager. > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:55:25:756 EST [TRACE] > MultiThreadedHttpConnectionManager - enter > HttpConnectionManager.releaseConnection(HttpConnection) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:55:25:756 EST [DEBUG] > MultiThreadedHttpConnectionManager - Freeing connection, > hostConfig=HostConfiguration[host=http://my-loadbalancer:9810] > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:55:25:756 EST [TRACE] > MultiThreadedHttpConnectionManager - enter > HttpConnectionManager.ConnectionPool.getHostPool(HostConfiguration) > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:55:25:756 EST [DEBUG] IdleConnectionHandler - > Adding connection at: 1258491325756 > Error [MsgId: MMSG-17999] > System.err: 2009/11/17 15:55:25:756 EST [DEBUG] > MultiThreadedHttpConnectionManager - Notifying no-one, there are no > waiting threads > Error [MsgId: MMSG-17999] > Error: Failed to end Transaction "callTrackingService_insertWorkFlow" > (by name). Please check that you made a call to start transaction > operation. [MsgId: MERR-16985] > Error: System.err: com.fusa.ssg.SystemException: Error making call[Error > executing Remote Method=[getCallSeqId] for Service > Url=[http://my-loadbalancer:9810/cms.rcts/2009.9.1.006/ccs/prf.lr/2008.8 > .1/CDC] Exception=[java.net.SocketException: Connection reset > at java.net.SocketInputStream.read(SocketInputStream.java:168) > at > java.io.BufferedInputStream.fill(BufferedInputStream.java:218) > at > java.io.BufferedInputStream.read(BufferedInputStream.java:235) > at > org.apache.commons.httpclient.ChunkedInputStream.getChunkSizeFromInputSt > ream(ChunkedInputStream.java:250) > at > org.apache.commons.httpclient.ChunkedInputStream.nextChunk(ChunkedInputS > tream.java:221) > at > org.apache.commons.httpclient.ChunkedInputStream.read(ChunkedInputStream > .java:176) > at > org.apache.commons.httpclient.ChunkedInputStream.read(ChunkedInputStream > .java:196) > at > org.apache.commons.httpclient.ChunkedInputStream.exhaustInputStream(Chun > kedInputStream.java:369) > at > org.apache.commons.httpclient.ChunkedInputStream.close(ChunkedInputStrea > m.java:346) > at java.io.FilterInputStream.close(FilterInputStream.java:159) > at > org.apache.commons.httpclient.AutoCloseInputStream.notifyWatcher(AutoClo > seInputStream.java:194) > at > org.apache.commons.httpclient.AutoCloseInputStream.close(AutoCloseInputS > tream.java:158) > at > java.io.ObjectInputStream$PeekInputStream.close(ObjectInputStream.java:2 > 252) > at > java.io.ObjectInputStream$BlockDataInputStream.close(ObjectInputStream.j > ava:2587) > at java.io.ObjectInputStream.close(ObjectInputStream.java:853) > at > com.fusa.ssg.infra.remoting.common.io.StandardJavaSerializer.readObject( > StandardJavaSerializer.java:41) > at > com.fusa.ssg.infra.remoting.client.service.RemoteServiceExecutorImpl.des > erializeServiceResponse(RemoteServiceExecutorImpl.java:153) > at > com.fusa.ssg.infra.remoting.client.service.RemoteServiceExecutorImpl.exe > cuteRequest(RemoteServiceExecutorImpl.java:111) > at > com.fusa.ssg.infra.remoting.client.service.RemoteServiceProxy.invoke(Rem > oteServiceProxy.java:54) > at $Proxy5.getCallSeqId(Unknown Source) > at > com.ccs.cco.cms.rcts.client.calltracking.CallTrackingRemotingDelegate.ge > tCallSeqId(CallTrackingRemotingDelegate.java:66) > at Actions.action(Actions.java:2327) > ]] > Error [MsgId: MERR-17999] > System.err: at > com.ccs.cco.cms.rcts.client.calltracking.CallTrackingRemotingDelegate.ge > tCallSeqId(CallTrackingRemotingDelegate.java:81) > Error [MsgId: MMSG-17999] > System.err: at Actions.action(Actions.java:2327) > Error [MsgId: MMSG-17999] > Error: System.err: ------------Wrapped Exception----------- > Error [MsgId: MERR-17999] > Error: System.err: > com.fusa.ssg.infra.remoting.common.RemoteServiceException: Error > executing Remote Method=[getCallSeqId] for Service > Url=[http://my-loadbalancer:9810/cms.rcts/2009.9.1.006/ccs/prf.lr/2008.8 > .1/CDC] Exception=[java.net.SocketException: Connection reset > at java.net.SocketInputStream.read(SocketInputStream.java:168) > at > java.io.BufferedInputStream.fill(BufferedInputStream.java:218) > at > java.io.BufferedInputStream.read(BufferedInputStream.java:235) > at > org.apache.commons.httpclient.ChunkedInputStream.getChunkSizeFromInputSt > ream(ChunkedInputStream.java:250) > at > org.apache.commons.httpclient.ChunkedInputStream.nextChunk(ChunkedInputS > tream.java:221) > at > org.apache.commons.httpclient.ChunkedInputStream.read(ChunkedInputStream > .java:176) > at > org.apache.commons.httpclient.ChunkedInputStream.read(ChunkedInputStream > .java:196) > at > org.apache.commons.httpclient.ChunkedInputStream.exhaustInputStream(Chun > kedInputStream.java:369) > at > org.apache.commons.httpclient.ChunkedInputStream.close(ChunkedInputStrea > m.java:346) > at java.io.FilterInputStream.close(FilterInputStream.java:159) > at > org.apache.commons.httpclient.AutoCloseInputStream.notifyWatcher(AutoClo > seInputStream.java:194) > at > org.apache.commons.httpclient.AutoCloseInputStream.close(AutoCloseInputS > tream.java:158) > at > java.io.ObjectInputStream$PeekInputStream.close(ObjectInputStream.java:2 > 252) > at > java.io.ObjectInputStream$BlockDataInputStream.close(ObjectInputStream.j > ava:2587) > at java.io.ObjectInputStream.close(ObjectInputStream.java:853) > at > com.fusa.ssg.infra.remoting.common.io.StandardJavaSerializer.readObject( > StandardJavaSerializer.java:41) > at > com.fusa.ssg.infra.remoting.client.service.RemoteServiceExecutorImpl.des > erializeServiceResponse(RemoteServiceExecutorImpl.java:153) > at > com.fusa.ssg.infra.remoting.client.service.RemoteServiceExecutorImpl.exe > cuteRequest(RemoteServiceExecutorImpl.java:111) > at > com.fusa.ssg.infra.remoting.client.service.RemoteServiceProxy.invoke(Rem > oteServiceProxy.java:54) > at $Proxy5.getCallSeqId(Unknown Source) > at > com.ccs.cco.cms.rcts.client.calltracking.CallTrackingRemotingDelegate.ge > tCallSeqId(CallTrackingRemotingDelegate.java:66) > at Actions.action(Actions.java:2327) > ] > Error [MsgId: MERR-17999] > System.err: at > com.fusa.ssg.infra.remoting.client.service.RemoteServiceProxy.invoke(Rem > oteServiceProxy.java:58) > Error [MsgId: MMSG-17999] > System.err: at $Proxy5.getCallSeqId(Unknown Source) > Error [MsgId: MMSG-17999] > System.err: at > com.ccs.cco.cms.rcts.client.calltracking.CallTrackingRemotingDelegate.ge > tCallSeqId(CallTrackingRemotingDelegate.java:66) > Error [MsgId: MMSG-17999] > System.err: at Actions.action(Actions.java:2327) > Error [MsgId: MMSG-17999] > Error: System.err: -----------Wrapped Exception----------- > Error [MsgId: MERR-17999] > Error: System.err: > com.fusa.ssg.infra.remoting.common.RemoteServiceException: Error > executing Remote Method=[getCallSeqId] for Service > Url=[http://my-loadbalancer:9810/cms.rcts/2009.9.1.006/ccs/prf.lr/2008.8 > .1/CDC] Exception=[java.net.SocketException: Connection reset > at java.net.SocketInputStream.read(SocketInputStream.java:168) > at > java.io.BufferedInputStream.fill(BufferedInputStream.java:218) > at > java.io.BufferedInputStream.read(BufferedInputStream.java:235) > at > org.apache.commons.httpclient.ChunkedInputStream.getChunkSizeFromInputSt > ream(ChunkedInputStream.java:250) > at > org.apache.commons.httpclient.ChunkedInputStream.nextChunk(ChunkedInputS > tream.java:221) > at > org.apache.commons.httpclient.ChunkedInputStream.read(ChunkedInputStream > .java:176) > at > org.apache.commons.httpclient.ChunkedInputStream.read(ChunkedInputStream > .java:196) > at > org.apache.commons.httpclient.ChunkedInputStream.exhaustInputStream(Chun > kedInputStream.java:369) > at > org.apache.commons.httpclient.ChunkedInputStream.close(ChunkedInputStrea > m.java:346) > at java.io.FilterInputStream.close(FilterInputStream.java:159) > at > org.apache.commons.httpclient.AutoCloseInputStream.notifyWatcher(AutoClo > seInputStream.java:194) > at > org.apache.commons.httpclient.AutoCloseInputStream.close(AutoCloseInputS > tream.java:158) > at > java.io.ObjectInputStream$PeekInputStream.close(ObjectInputStream.java:2 > 252) > at > java.io.ObjectInputStream$BlockDataInputStream.close(ObjectInputStream.j > ava:2587) > at java.io.ObjectInputStream.close(ObjectInputStream.java:853) > at > com.fusa.ssg.infra.remoting.common.io.StandardJavaSerializer.readObject( > StandardJavaSerializer.java:41) > at > com.fusa.ssg.infra.remoting.client.service.RemoteServiceExecutorImpl.des > erializeServiceResponse(RemoteServiceExecutorImpl.java:153) > at > com.fusa.ssg.infra.remoting.client.service.RemoteServiceExecutorImpl.exe > cuteRequest(RemoteServiceExecutorImpl.java:111) > at > com.fusa.ssg.infra.remoting.client.service.RemoteServiceProxy.invoke(Rem > oteServiceProxy.java:54) > at $Proxy5.getCallSeqId(Unknown Source) > at > com.ccs.cco.cms.rcts.client.calltracking.CallTrackingRemotingDelegate.ge > tCallSeqId(CallTrackingRemotingDelegate.java:66) > at Actions.action(Actions.java:2327) > ] > Error [MsgId: MERR-17999] > System.err: at > com.fusa.ssg.infra.remoting.client.service.RemoteServiceProxy.invoke(Rem > oteServiceProxy.java:58) > Error [MsgId: MMSG-17999] > System.err: at $Proxy5.getCallSeqId(Unknown Source) > Error [MsgId: MMSG-17999] > System.err: at > com.ccs.cco.cms.rcts.client.calltracking.CallTrackingRemotingDelegate.ge > tCallSeqId(CallTrackingRemotingDelegate.java:66) > Error [MsgId: MMSG-17999] > System.err: at Actions.action(Actions.java:2327) > Error [MsgId: MMSG-17999] > > > > > > > > > > > This transmission may contain information that is privileged, > confidential, legally privileged, and/or exempt from disclosure > under applicable law. If you are not the intended recipient, you > are hereby notified that any disclosure, copying, distribution, or > use of the information contained herein (including any reliance > thereon) is STRICTLY PROHIBITED. Although this transmission and > any attachments are believed to be free of any virus or other > defect that might affect any computer system into which it is > received and opened, it is the responsibility of the recipient to > ensure that it is virus free and no responsibility is accepted by > JPMorgan Chase & Co., its subsidiaries and affiliates, as > applicable, for any loss or damage arising in any way from its use. > If you received this transmission in error, please immediately > contact the sender and destroy the material in its entirety, > whether in electronic or hard copy format. Thank you. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
