On 5 October 2011 07:38, Konstantin Boudnik <[email protected]> wrote: > On Wed, Oct 05, 2011 at 01:46AM, sebb wrote: > > On 5 October 2011 00:25, Oleg Kalnichevski <[email protected]> wrote: > > > On Tue, 2011-10-04 at 15:52 -0700, Konstantin Boudnik wrote: > > >> Hello > > >> > > >> Firstly, I'd like to thank this group of people for outstanding job in > > >> development Java layer for generic HTTP: very nicely and conveniently > done! > > >> > > >> Now, once I am done with being nice I have a question ;) Here's the > use case > > >> where I don't see a good workaround for. > > >> > > >> We have that piece of functionality in our app. where we need to grab > a little > > >> sample of a web.object (e.g. a file somewhere on a web-server side). > Once the > > >> sample is obtained the software isn't interested in the rest of the > content > > >> and would like to close the InputStream (actually it happens to be > > >> ContentLengthInputStream) from that object. The situation we are in is > that we > > >> can't set ContentLength on the client side and the connection is > forcefully > > >> kept alive from the server side. > > >> > > >> The objects (files) are pretty large (10s or 100s of Mb) and we don't > want to > > >> keep downloading the rest of the content every time we need to sample. > By > > >> looking into the code of that class above I see that close() method > keeps > > >> pumping the data until ContentLength is reached. This seems to be > troublesome > > >> in cases when the app's code is rapidly opens a whole bunch of > connections to > > >> different remote objects for the sampling purposes and then trying to > close > > >> those. However, close() doesn't do any real closing of the socket > input so > > >> data transfer continues and I might end up with OOME ;( > > >> > > >> Considering above restrictions is their any advisable solution for the > problem > > >> I am facing? I believe patching ContentLengthInputStream might be > pretty > > >> tricky because ContentLengthInputStream can't actually close > SessionInputBuffer. > > >> > > >> Any hints would be highly appreciated! > > > > > > Konstantin > > > > > > Per default HttpClient always makes an attempt to keep a persistent > > > connection alive. This is the reason for #close() method of > > > ContentLengthInputStream always reading from the underlying connection > > > until the end of the message. One can use HttpUriRequest#abort() to > > > immediately shut down the underlying connection (if allocated) and > > > remove it from the connection pool. > > > > It sounds as if the application will always want to abort the transfer > > for such requests, so would it be better not to use a persistent > > connection? > > I am not sure this is possible: we are wrapping AWS Java API and this seems > to > be the only kind of connections they are provide. Also, transfer > termination > is just a single use case. > > Thanks, > Cos > > > And would that affect how close works? > > > > Just curious. > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > > iF4EAREIAAYFAk6L+10ACgkQenyFlstYjhIBGgD/VtCFcXAAyiBeI2fVSEUyL4ZB > qnPFShJum7wwZAY7mO0BAKZoBUcX96LL8jG3WnluVkxUS13h7C8FFUw5454FJelg > =92AX > -----END PGP SIGNATURE----- > > Would it help if you were more explicit about the sampling at an HTTP level, and make a Range request [1]?
Range: bytes=0-2048 for example Cheers, James [1] http://tools.ietf.org/html/rfc2616#section-14.35
