Ok, I hope you aren't getting sick of this problem. :) HttpMethod.abort does solve the problem of sites that send an infinite response. However, it seems that by calling abort we cannot properly release the connection. I've tried calling method.releaseConnection right after abort.
My usage for HttpClient is a multi-threaded crawler so I've followed the suggestions on the threading page http://jakarta.apache.org/commons/httpclient/threading.html (nice documentation by the way). So I use the MultiThreadedHttpConnectionManager as suggested and reuse the same HttpClient over and over as suggested. After a certain number of calls to HttpMethod.abort my HttpClient goes bad (hangs). So it appears that abort is too harsh and doesn't allow clean return of the client to the pool. Any more suggestions? On 7/21/05, Tony Spencer <[EMAIL PROTECTED]> wrote: > Disregard my last message. Your suggestion did work Oleg. Originally > I put the abort statement after attempted to close the input stream. > Once I moved it in front of the stream close statement it worked fine. > Thank you very much. > > On 7/21/05, Oleg Kalnichevski <[EMAIL PROTECTED]> wrote: > > Just call HttpMethod#abort to close the underlying connection > > > > Oleg > > > > > > On Thu, 2005-07-21 at 16:34 -0400, Tony Spencer wrote: > > > Ok, I managed to limit the the response to 8k in the following code > > > but it doesn't help with what I'm really trying to accomplish. > > > Sometimes there is a site that will spew a neverending response. This > > > causes HttpClient to hang indefinitely. My code below does not solve > > > the problem. Here is an example of a nasty site that never stops > > > sending response: http://www.tfc-charts.w2d.com/chart/dw/w (beware. > > > it may crash your browser if you browse it) > > > > > > InputStream is = method.getResponseBodyAsStream(); > > > BufferedInputStream bis = new BufferedInputStream(is); > > > byte[] bytes = new byte[ 8192 ]; > > > bis.read(bytes); > > > bis.close(); > > > is.close(); > > > ret = new String(bytes); > > > > > > > > > On 7/21/05, Tony Spencer <[EMAIL PROTECTED]> wrote: > > > > I'd like to limit the size of the response but don't know how. For > > > > instance, if the response body is greater than 100k I would like to > > > > close the connection to the site. How can I go about doing this? I > > > > see the available method param : BUFFER_WARN_TRIGGER_LIMIT but it only > > > > seems to control warning logging. > > > > > > > > Currently I receive the response body like so: > > > > byte[] bytes = method.getResponseBody(); > > > > > > > > Any help greatly appreciated. > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
