John Just for the record: HttpClient 2.0 design is completely broken in many, many wonderful ways, and we are perfectly aware of that. Excuse my lack of understanding, however, I do think that applies to the current implementation of the content streaming.
Allow me to reiterate that HttpClient does provide enough flexibility to avoid content buffering. Albeit, instead of exposing the OutputStream interface and letting the caller handle the content writing, HttpClient accepts InputStream as input and does all the content reading from the stream and writing it into output stream of the underlying socket. The idea here is to avoid exposure of underlying socket in order to prevent possible misuse/abuse. Think of maintaining integrity of persistent connection, for instance. One situation where to content buffering seems unavoidable is object serialization. I believe this is what Eric was referring to. There's currently a request pending for better OutputSteam based serialization. See Mohammad's post for more details Hope this helps Oleg -----Original Message----- From: John Keyes [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 24, 2004 15:19 To: Commons HttpClient Project Subject: Re: streaming request body > For (a), Oleg's response is correct. You might easily be confused, in > the sense that HttpClient's API inverts the control. It is not that > you write to an "OutputStream" to send your data, it is that you > provide HttpClient with an "InputStream", and it reads that stream and > sends the data. HttpClient is designed to accomodate your concern, and > if your configuration is correct (as per the examples), it will not > buffer the entire contents of your InputStream, but rather read it and > send it in small chunks. As another post points you, you may still > have to buffer what you're sending to *disk*, but not to memory. So you think buffering all requests to disk to support streaming is an acceptable solution? If I am dealing with XXX,000 of requests that sure as hell would suck with all the disk I/O going on. Does this not suggest that there is a problem with the architecture? > As for (b), this is again under your control via > HttpMethod.getResponseBodyAsStream(). As with (a), you can also invoke > HttpClient such that it does cache the entire contents > (HttpMethod.getResponseBodyAsString() ). > > In both cases, it is possible to get the behavior that you desire. Not it is not. Again think of XXX,000 of requests. > Connection pooling is only part of the concern. HttpClient supports > HTTP 1.1 persistent connections. It doesn't expose the underlying > socket's InputStream and OutputStream. If it did, it cound not ensure > that persistent connections work properly. I still don't see the problem. The OutputStream and InputStream can be wrapped so there is no loss of control. Why do you think control would be lost? -John K > > -Eric. > > John Keyes wrote: > >> Guys, >> >> A colleague pointed out to me that this does not in fact resolve the >> situation. The solutions pointed out allow me to read the attachment >> as a stream. The contents are still held in memory prior to writing >> it on the wire. To fully support this you would need access to the >> OutputStream. >> >> If we could pass a HttpClient to the HttpMethod then we could get >> access to the output stream via the getRequestOutputStream method. >> >> I don't understand the connection pooling argument. I thought it >> should be a user preference whether to have connection pooling. >> >> Any ideas on this? >> -John K >> >> On 23 Feb 2004, at 13:02, Kalnichevski, Oleg wrote: >> >>> >>> John, >>> >>> HttpClient's entity enclosing methods (POST, PUT) do support content >>> streaming when (1) the content length does not need to be >>> automatically calculated or (2) chunk-encoding is used >>> >>> Please refer to the following sample applications for details >>> >>> Unbuffered post: >>> >>> http://cvs.apache.org/viewcvs.cgi/*checkout*/jakarta-commons/ >>> httpclient/src/examples/UnbufferedPost.java?content- >>> type=text%2Fplain&rev=1.2.2.1 >>> >>> Chunk-encoded post: >>> >>> http://cvs.apache.org/viewcvs.cgi/*checkout*/jakarta-commons/ >>> httpclient/src/examples/ChunkEncodedPost.java?content- >>> type=text%2Fplain&rev=1.4.2.1 >>> >>> Hope this helps >>> >>> Oleg >>> >>> >>> -----Original Message----- >>> From: John Keyes [mailto:[EMAIL PROTECTED] >>> Sent: Monday, February 23, 2004 13:54 >>> To: [EMAIL PROTECTED] >>> Subject: streaming request body >>> >>> >>> Hi, >>> >>> I notice you have separated out the functions of the connection and >>> the >>> content creation. So the code must be something like >>> >>> HttpClient client = new HttpClient( url ); >>> HttpMethod method = new GetMethod(); >>> method.setRequestHeader( ... ); ... >>> method.setRequestBody( ... ); >>> client.execute( method ); >>> >>> If I want to send a large attachment and I don't want it all to be in >>> memory then I can't do it. The issue is that you have to write your >>> data to the HttpMethod. The HttpMethod doesn't know where to then >>> write >>> this data until you call execute and pass the client which has the >>> connection to write to. So there isn't really a way around this >>> because >>> of the separation of the connection from the HttpMethod. >>> >>> So my question is, is there a way to stream the request body rather >>> than having to store the request in memory prior to writing it on the >>> wire. >>> >>> Thanks, >>> -John K >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: >>> [EMAIL PROTECTED] >>> For additional commands, e-mail: >>> [EMAIL PROTECTED] >>> >>> >>> ********************************************************************* >>> ** **************************** >>> The information in this email is confidential and may be legally >>> privileged. Access to this email by anyone other than the intended >>> addressee is unauthorized. If you are not the intended recipient of >>> this message, any review, disclosure, copying, distribution, >>> retention, or any action taken or omitted to be taken in reliance on >>> it is prohibited and may be unlawful. If you are not the intended >>> recipient, please reply to or forward a copy of this message to the >>> sender and delete the message, any attachments, and any copies >>> thereof from your system. >>> ********************************************************************* >>> ** **************************** >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: >>> [EMAIL PROTECTED] >>> For additional commands, e-mail: >>> [EMAIL PROTECTED] >>> >>> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: >> [EMAIL PROTECTED] >> For additional commands, e-mail: >> [EMAIL PROTECTED] >> >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: > [EMAIL PROTECTED] > For additional commands, e-mail: > [EMAIL PROTECTED] > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] *************************************************************************************************** The information in this email is confidential and may be legally privileged. Access to this email by anyone other than the intended addressee is unauthorized. If you are not the intended recipient of this message, any review, disclosure, copying, distribution, retention, or any action taken or omitted to be taken in reliance on it is prohibited and may be unlawful. If you are not the intended recipient, please reply to or forward a copy of this message to the sender and delete the message, any attachments, and any copies thereof from your system. *************************************************************************************************** --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
