[
https://issues.apache.org/jira/browse/SYNAPSE-225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12589287#action_12589287
]
Jef Waltman commented on SYNAPSE-225:
-------------------------------------
DISCLAIMER - I am not now, nor have I ever been, a Java developer.
>From what I can tell there are two issues, both with
>org.apache.synapse.transport.nhttp.ServerHandler:
Problem #1: sink() and source() channels opened in requestReceived() only get
closed in inputReady() and outputReady() respectively.
If the request is null (for instance, you hit http://localhost:8080 with a
browser) inputReady does not get called and the sink() channel stays open.
Likewise, if there is no response (say, you send the request straight to a drop
mediator) outputReady does not get run and the source() channel stays open.
Solution #1: Check to make sure the source() and sink() channels are closed
when closed() is called. This should clean up anything that wasn't closed by
inputReady() or outputReady():
public void closed(final NHttpServerConnection conn) {
// Check sink and source channels and close them if they aren't closed
already.
// Normally these should be closed by inputReady() and outputReady(). A
null request
// or response will not hit inputReady and outputReady however.
HttpContext context = conn.getContext();
WritableByteChannel sink = (WritableByteChannel)
context.getAttribute(REQUEST_SINK_CHANNEL);
ReadableByteChannel source = (ReadableByteChannel)
context.getAttribute(RESPONSE_SOURCE_CHANNEL);
try {
if (sink.isOpen()) {
sink.close();
if (log.isTraceEnabled()) {
log.trace("Closed stale sink channel");
}
}
if (source.isOpen()) {
source.close();
if (log.isTraceEnabled()) {
log.trace("Closed stale source channel");
}
}
} catch (IOException ignore) {}
if (log.isTraceEnabled()) {
log.trace("Connection closed");
}
}
I can't think of any reason not to close those channels on connection close,
but I didn't audit the code exhaustively. Using the browser connection to
http://localhost:8080 to reproduce, this gets me from 4 leaked FIFO pipes to 2.
Problem #2: Each pipe has two channels, sink() and source().
requestReceived() opens two pipes, requestPipe and responsePipe. The sink()
channel on requestPipe is assigned and passed to the ServerWorker process. The
source() channel on the responsePipe is assigned and passed to the ServerWorker
process. This leaves the source() channel on requestPipe and the sink()
channel on responsePipe unassigned to anything, but still consuming FIFO
handles as reported by lsof. Also they never get closed, as they are neither
assigned nor can they be referrenced.
Solution #2: Open an single pipe and assign the sink() channel to the input
stream and the source() channel to the output stream. Not only does it take
less resource, it avoids creating orphans of the unused channels:
public void requestReceived(final NHttpServerConnection conn) {
HttpContext context = conn.getContext();
HttpRequest request = conn.getHttpRequest();
context.setAttribute(ExecutionContext.HTTP_REQUEST, request);
// allocate temporary buffers to process this request
context.setAttribute(REQUEST_BUFFER,
ByteBuffer.allocate(cfg.getBufferZise()));
context.setAttribute(RESPONSE_BUFFER,
ByteBuffer.allocate(cfg.getBufferZise()));
try {
PipeImpl handlerPipe = new PipeImpl(); // the pipe for request and
response
context.setAttribute(REQUEST_SINK_CHANNEL, handlerPipe.sink());
context.setAttribute(RESPONSE_SOURCE_CHANNEL, handlerPipe.source());
// create the default response to this request
ProtocolVersion httpVersion =
request.getRequestLine().getProtocolVersion();
HttpResponse response = responseFactory.newHttpResponse(
httpVersion, HttpStatus.SC_OK, context);
response.setParams(this.params);
// create a basic HttpEntity using the source channel of the
response pipe
BasicHttpEntity entity = new BasicHttpEntity();
entity.setContent(Channels.newInputStream(handlerPipe.source()));
if (httpVersion.greaterEquals(HttpVersion.HTTP_1_1)) {
entity.setChunked(true);
}
response.setEntity(entity);
// hand off processing of the request to a thread off the pool
workerPool.execute(
new ServerWorker(cfgCtx, conn, isHttps, metrics, this,
request, Channels.newInputStream(handlerPipe.source()),
response, Channels.newOutputStream(handlerPipe.sink())));
} catch (IOException e) {
handleException("Error processing request received for : " +
request.getRequestLine().getUri(), e, conn);
if (metrics != null) {
metrics.incrementFaultsReceiving();
}
} catch (Exception e) {
handleException("Error processing request received for : " +
request.getRequestLine().getUri(), e, conn);
if (metrics != null) {
metrics.incrementFaultsReceiving();
}
}
}
These changes appear to solve the problem for me, though I haven't tested
exhaustively. My test box for this:
webdev:~# cat /etc/redhat-release
CentOS release 5 (Final)
webdev:~# uname -a
Linux webdev.siriuscom.com 2.6.18-53.el5xen #1 SMP Mon Nov 12 03:26:12 EST 2007
i686 i686 i386 GNU/Linux
webdev:~# java -version
java version "1.5.0_14"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_14-b03)
Java HotSpot(TM) Client VM (build 1.5.0_14-b03, mixed mode, sharing)
webdev:~# ls -l synapse-1.1.1-src.tar.gz
-rw-r--r-- 1 root root 6167030 Jan 28 08:04 synapse-1.1.1-src.tar.gz
--Jef.
> Leaking resources when the connection times out.
> ------------------------------------------------
>
> Key: SYNAPSE-225
> URL: https://issues.apache.org/jira/browse/SYNAPSE-225
> Project: Synapse
> Issue Type: Bug
> Components: Transports
> Affects Versions: 1.1.1
> Environment: [EMAIL PROTECTED] ~]# cat /etc/redhat-release
> Red Hat Enterprise Linux Server release 5.1 (Tikanga)
> [EMAIL PROTECTED] ~]# uname -a
> Linux lou-lxapachetest 2.6.18-53.el5 #1 SMP Wed Oct 10 16:34:02 EDT 2007 i686
> i686 i386 GNU/Linux
> [EMAIL PROTECTED] ~]# java -version
> java version "1.5.0_13"
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_13-b05)
> Java HotSpot(TM) Client VM (build 1.5.0_13-b05, mixed mode, sharing)
> [EMAIL PROTECTED] ~]#
> Reporter: SteveKrall
> Assignee: Asankha C. Perera
> Fix For: 1.2
>
>
> I'm using 1.1.1-RC1. When a connection times out, I can see using lsof pipes
> being left open. This does not happen when the connection works normally.
> Steps to reproduce:
> 1. Check the open files
> 2. Run the SimpleStockQuoteApplication successfully
> 3. Check the open files (open files will go up on the first successful
> request, but not subsequent successful request
> 4. Run the SimpleStockQuoteApplication this time with a -Daddurl that will
> generate a timeout.
> 5. Check the open files. Repeat steps 4 and 5. The number will continue to
> grow.
> My output from a test:
> [EMAIL PROTECTED] ~/files/downloads/synapse/synapse-1.1.1/samples/axis2Client
> $ lsof -p 27788 | wc -l
> 245
> [EMAIL PROTECTED] ~/files/downloads/synapse/synapse-1.1.1/samples/axis2Client
> $ lsof -p 27788 | grep pipe | wc -l
> 24
> [EMAIL PROTECTED] ~/files/downloads/synapse/synapse-1.1.1/samples/axis2Client
> $ ant stockquote -Daddurl=http://localhost:9000/soap/SimpleStockQuoteService
> -Dtrpurl=http://localhost:8080
> Buildfile: build.xml
> init:
> compile:
> stockquote:
> [java] Standard :: Stock price = $99.43283516858692
> BUILD SUCCESSFUL
> Total time: 2 seconds
> [EMAIL PROTECTED] ~/files/downloads/synapse/synapse-1.1.1/samples/axis2Client
> $ lsof -p 27788 | wc -l
> 245
> [EMAIL PROTECTED] ~/files/downloads/synapse/synapse-1.1.1/samples/axis2Client
> $ lsof -p 27788 | grep pipe | wc -l
> 24
> [EMAIL PROTECTED] ~/files/downloads/synapse/synapse-1.1.1/samples/axis2Client
> $ ant stockquote
> -Daddurl=http://<HostThatWillCauseTimeout>:9000/soap/SimpleStockQuoteService
> -Dtrpurl=http://localhost:8080
> Buildfile: build.xml
> init:
> compile:
> stockquote:
> [java] 2008-01-15 16:05:40,829 [-] [main] INFO HTTPSender Unable to
> sendViaPost to url[http://localhost:8080]
> [java] java.net.SocketTimeoutException: Read timed out
> [java] at java.net.SocketInputStream.socketRead0(Native Method)
> [java] at java.net.SocketInputStream.read(SocketInputStream.java:129)
> [java] at
> java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
> [java] at
> java.io.BufferedInputStream.read(BufferedInputStream.java:237)
> [java] at
> org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:77)
> [java] at
> org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:105)
> [java] at
> org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1115)
> [java] at
> org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.readLine(MultiThreadedHttpConnectionManager.java:1373)
> [java] at
> org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1832)
> [java] at
> org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1590)
> [java] at
> org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:995)
> [java] at
> org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:397)
> [java] at
> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)
> [java] at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
> [java] at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
> [java] at
> org.apache.axis2.transport.http.AbstractHTTPSender.executeMethod(AbstractHTTPSender.java:520)
> [java] at
> org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:191)
> [java] at
> org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:77)
> [java] at
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:327)
> [java] at
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:206)
> [java] at
> org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:396)
> [java] at
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:374)
> [java] at
> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:211)
> [java] at
> org.apache.axis2.client.OperationClient.execute(OperationClient.java:163)
> [java] at
> org.apache.axis2.client.ServiceClient.sendReceive(ServiceClient.java:528)
> [java] at
> org.apache.axis2.client.ServiceClient.sendReceive(ServiceClient.java:508)
> [java] at samples.userguide.StockQuoteClient.executeClient(Unknown
> Source)
> [java] at samples.userguide.StockQuoteClient.main(Unknown Source)
> [java] org.apache.axis2.AxisFault: Read timed out
> [java] at org.apache.axis2.AxisFault.makeFault(AxisFault.java:417)
> [java] at
> org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:195)
> [java] at
> org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:77)
> [java] at
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:327)
> [java] at
> org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:206)
> [java] at
> org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:396)
> [java] at
> org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:374)
> [java] at
> org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:211)
> [java] at
> org.apache.axis2.client.OperationClient.execute(OperationClient.java:163)
> [java] at
> org.apache.axis2.client.ServiceClient.sendReceive(ServiceClient.java:528)
> [java] at
> org.apache.axis2.client.ServiceClient.sendReceive(ServiceClient.java:508)
> [java] at samples.userguide.StockQuoteClient.executeClient(Unknown
> Source)
> [java] at samples.userguide.StockQuoteClient.main(Unknown Source)
> [java] Caused by: java.net.SocketTimeoutException: Read timed out
> [java] at java.net.SocketInputStream.socketRead0(Native Method)
> [java] at java.net.SocketInputStream.read(SocketInputStream.java:129)
> [java] at
> java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
> [java] at
> java.io.BufferedInputStream.read(BufferedInputStream.java:237)
> [java] at
> org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:77)
> [java] at
> org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:105)
> [java] at
> org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1115)
> [java] at
> org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.readLine(MultiThreadedHttpConnectionManager.java:1373)
> [java] at
> org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1832)
> [java] at
> org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1590)
> [java] at
> org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:995)
> [java] at
> org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:397)
> [java] at
> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)
> [java] at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
> [java] at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
> [java] at
> org.apache.axis2.transport.http.AbstractHTTPSender.executeMethod(AbstractHTTPSender.java:520)
> [java] at
> org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:191)
> [java] ... 11 more
> BUILD SUCCESSFUL
> Total time: 32 seconds
> [EMAIL PROTECTED] ~/files/downloads/synapse/synapse-1.1.1/samples/axis2Client
> $ lsof -p 27788 | wc -l
> 250
> [EMAIL PROTECTED] ~/files/downloads/synapse/synapse-1.1.1/samples/axis2Client
> $ lsof -p 27788 | grep pipe | wc -l
> 28
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
