I added some Javadoc explaining the role of cleanupTransport (see revision 748730).
Andreas On Fri, Feb 27, 2009 at 14:05, Andreas Veithen <andreas.veit...@gmail.com> wrote: > I think that callTransportCleanup should never be turned on by default > because it would disable deferred parsing of the response. What needs > to be done urgently is to improve the documentation of the > ServiceClient class to make it clear that it is mandatory to either > call cleanupTransport explicitly or to set callTransportCleanup to > true. Also the cleanupTransport method itself doesn't have any > Javadoc. > > Andreas > > On Fri, Feb 27, 2009 at 12:19, Dobri Kitipov > <kdobrik.ax...@googlemail.com> wrote: >> Hi Andreas, >> thank you for the comment. I think you get the question. >> Quick test shows that setting the following line of code into the client: >> >> options.setCallTransportCleanup(true); >> >> forces the closure of the http connection. It seems it is not the default >> behavior. This is a good and fast solution. I was a little bit more focused >> on wondering why I have such a difference b/n using SOAP and MIME builder. >> >> I need to think about some use cases when we need to have >> options.setCallTransportCleanup(false). Can we have this by default in some >> cases? >> >> Anyway, it will be worth having a further analysis of the issue we have with >> SOAPBuilder behavior. >> >> Thank you, >> Dobri >> >> >> On Fri, Feb 27, 2009 at 12:46 PM, Andreas Veithen >> <andreas.veit...@gmail.com> wrote: >>> >>> If I understand correctly, Dobri's findings can be summarized as follows: >>> 1. Once the InputStream is consumed, commons-httpclient automatically >>> releases the connection. >>> 2. SOAPBuilder never completely consumes the InputStream. >>> >>> The SOAPBuilder behavior is indeed somewhat questionable, but it is >>> important to understand that because of the deferred parsing model >>> used by Axiom, there is never a guarantee that the InputStream will be >>> completely consumed. Normally releasing the connection is the >>> responsibility of the CommonsHTTPTransportSender#cleanup method which >>> should be called by ServiceClient#cleanupTransport. It would be >>> interesting to know if that method is called, and if it is, why it >>> fails to release the connection. >>> >>> Andreas >>> >>> On Fri, Feb 27, 2009 at 10:10, Dobri Kitipov >>> <kdobrik.ax...@googlemail.com> wrote: >>> > Hi all, >>> > I have observed absolutely the same thing these days. I need some more >>> > time >>> > to analyze the whole picture, but here is my current synthesis of the >>> > issue. >>> > >>> > It seems that http connection release is tightly coupled with the Axis2 >>> > builder used to handle and process the response body and the >>> > corresponding >>> > input streams used. The builder and the InputStream used are based on >>> > the >>> > HTTP headers fields like "Content-Type" and "Transfer-Encoding" (e.g. >>> > Content-Type: text/xml; charset=UTF-8 Transfer-Encoding: chunked). So >>> > if we >>> > have Content-Type: text/xml; then SOAPBuilder class will be used. If we >>> > have type="application/xop+xml" then MIMEBuilder will be used. >>> > >>> > The successfull story when we have MIMIBuilder: >>> > >>> > When MIMEBuilder is used then the response Buffered InputStream (IS) is >>> > wrrapped (I will use "->" sign as substitution for wrrapped) ChunkedIS >>> > -> >>> > AutoCloseIS (this has a responseConsumed() method notified when >>> > IS.read() >>> > returns -1, which means that the response IS has been completely read) >>> > -> >>> > PushBackIS ->BounderyPushBackIS. The BounderyPushBackIS reads the >>> > response >>> > stream (see readFromStream(....)) in a cycle till it reaches its end. At >>> > every iteration of this cycle a AutoCloseIS checkClose(l) is invoked. So >>> > when the end is reached (-1 is returned) then this check causes the >>> > invokation of the AutoCloseIS checkClose(...) method. This method >>> > invokes >>> > notifyWatcher() that in turn invokes responseBodyConsumed() method of >>> > the >>> > HttpMethodBase class. This causes the release of the http connection >>> > which >>> > is returned back to the connection pool. So here we have no problem with >>> > connection reuse. >>> > >>> > The bad story we have with SOAPBuilder: >>> > >>> > When SOAPBuilder is used then the response Buffered InputStream is >>> > wrrapped >>> > in a ChunkedIS -> AutoCloseIS -> PushBackIS. May be you has noticed that >>> > BounderyPushBackIS is not used. As a result the respose IS is not >>> > completely >>> > read (in fact this is not really correct, it could be read but the >>> > invokation of the PushBackIS unread(...) method causes the AutoCloseIS >>> > checkClose() method never to return -1). As a result the http connection >>> > is >>> > not released. And since there is a limit to have 2 connection per host >>> > then after the second invokation of the WS client the thread hangs >>> > waiting >>> > for a free connection. >>> > >>> > Please, provide us with your comments on this issue. >>> > >>> > Thank you in advance. >>> > >>> > Regards, >>> > Dobri >>> > >>> > On Fri, Feb 27, 2009 at 3:54 AM, Alexis Midon <mi...@intalio.com> wrote: >>> >> >>> >> no taker for an easy patch? >>> >> >>> >> Alexis >>> >> >>> >> >>> >> On Wed, Feb 25, 2009 at 6:45 PM, Alexis Midon <mi...@intalio.com> >>> >> wrote: >>> >>> >>> >>> Hi everyone, >>> >>> >>> >>> All the issues relatives to AXIS2-935 are really messy, some of them >>> >>> are >>> >>> closed but their clones are not. Some are flagged as fixed but are >>> >>> obviously >>> >>> not. All these issues are really old, so I'd like to take a chance to >>> >>> bring >>> >>> them back to your attention, especially before releasing 1.5. >>> >>> >>> >>> I'll post a description of the issue in this email as a summary all >>> >>> the >>> >>> jiras. >>> >>> >>> >>> By default, ServiceClient uses one HttpConnectionManager per >>> >>> invocation >>> >>> [2]. This connection manager will create and provide one connection to >>> >>> HTTPSender. The first issue is that by default this connection is >>> >>> never >>> >>> released to the pool [3]. if you do zillions of invocations, this leak >>> >>> will >>> >>> max out your number of file descriptors. >>> >>> >>> >>> Your investigations in Axis2 options quickly lead you to the >>> >>> REUSE_HTTP_CLIENT option. But this first issue has some unfortunate >>> >>> consequences if you activate it. Actually if you do so, a single >>> >>> connection >>> >>> manager is shared across all invocations. But because connections are >>> >>> not >>> >>> release, the pool is starved after two invocations, and the third >>> >>> invocation >>> >>> hangs out indefinitely. :( >>> >>> >>> >>> If you keep digging you will find the AUTO_RELEASE_CONNECTION option. >>> >>> Its >>> >>> sounds like a good lead! Let's try it. If you activate this option the >>> >>> connection is properly released -Yahoooo! the leak is fixed - but >>> >>> unfortunately a new issue shows up (issue #2, aka AXIS2-3478). >>> >>> AbstractHTTPSender passes the stream of the connection to the message >>> >>> context [4] , but that the connection is now properly released, so >>> >>> this >>> >>> stream is closed before the SOAPBuilder gets a chance to read the >>> >>> response >>> >>> body. >>> >>> Boom! "IOException: Attempted read on closed stream" >>> >>> >>> >>> These issues are easily reproducible in versions 1.3, 1.4, 1.5. >>> >>> >>> >>> I submitted and documented a fix in AXIS2-2931 [5], if you had a >>> >>> chance >>> >>> to look at it that would be much appreciate. >>> >>> >>> >>> >>> >>> Alexis >>> >>> >>> >>> >>> >>> [1] >>> >>> >>> >>> https://issues.apache.org/jira/browse/AXIS2-935?focusedCommentId=12513543#action_12513543 >>> >>> [2] see method getHttpClient in >>> >>> >>> >>> https://svn.apache.org/repos/asf/webservices/commons/trunk/modules/transport/modules/http/src/org/apache/axis2/transport/http/AbstractHTTPSender.java >>> >>> [3] see method cleanup in >>> >>> >>> >>> https://svn.apache.org/repos/asf/webservices/commons/trunk/modules/transport/modules/http/src/org/apache/axis2/transport/http/HTTPSender.java >>> >>> [4] see method processResponse in AbstractHTTPSender.java >>> >>> [5] >>> >>> >>> >>> https://issues.apache.org/jira/browse/AXIS2-2931?focusedCommentId=12676837#action_12676837 >>> >> >>> > >>> > >> >> >
