Yeah, I already tried httpurlconnnection and got road blocked by the int field input in httpConnection.setFixedLengthStreamingMode() - max file size can only be 2GB until 7.0.
So still looking for some real options. Thanks Sachin -----Original Message----- From: Oleg Kalnichevski [mailto:[email protected]] Sent: Wednesday, July 20, 2011 7:42 PM To: HttpClient User Discussion Subject: Re: How to stream a file via PUT to multiple HttpClients On Wed, 2011-07-20 at 09:37 -0400, Jon Moore wrote: > Hi Sachin, > > In this case, I would suggest running the .execute() calls in separate > threads, or perhaps using the asynchronous HttpClient: > http://hc.apache.org/httpcomponents-asyncclient-dev/index.html > > I think those are your only options given the existing APIs for HttpClient. > > Jon > I agree with Jon. Zero copy support in HttpAsyncClient might be the best option available to you. Otherwise you will have to use HttpUrlConnection or some other HTTP client. Oleg > On Wed, Jul 20, 2011 at 9:20 AM, Sachin Shetty <[email protected]> wrote: > > But since I would execute client.execute(put1) and client.execute(put2) > > serially in a single thread, ForkedInputStream will end up buffering almost > > the entire file for put2 which I want to avoid at any cost. > > > > In short, I want to spawn two puts and write to them in blocks one after > > another. > > > > -----Original Message----- > > From: Jon Moore [mailto:[email protected]] > > Sent: Wednesday, July 20, 2011 6:42 PM > > To: HttpClient User Discussion > > Subject: Re: How to stream a file via PUT to multiple HttpClients > > > > Hi Sachin, > > > > I think what you want to do is wrap your FileInputStream in another > > object, let's call it a ForkedInputStream: > > > > FileInputStream fis = ...; > > ForkedInputStream forked = new ForkedInputStream(fis); > > HttpPut put1 = new HttpPut(...); > > HttpPut put2 = new HttpPut(...); > > put1.setEntity(new InputStreamEntity(forked.getLeft())); > > put2.setEntity(new InputStreamEntity(forked.getRight())); > > > > etc. > > > > Then your ForkedInputStream takes care of reading from the underlying > > File (perhaps on demand) and keeping the multiple writers in as much > > lockstep or with as much buffering as desired. > > > > Jon > > > > On Wed, Jul 20, 2011 at 6:02 AM, Sachin Shetty <[email protected]> wrote: > >> Hi, > >> > >> > >> > >> We have a unique case where we want to put a file via HTTP PUT to more > > than > >> one http servers but we want to read the file only once which means we > > want > >> to read a block of data from the file and write to two entities > >> simultaneously. Streaming the PUT and reading the file only once without > >> having to hold it in memory are the two things we cannot avoid. > >> > >> > >> > >> These things are pretty easy when we have a direct outputstream to write > > to, > >> but I am not able to figure out how this can be achieved using the > > entities. > >> > >> > >> > >> basically we want an httpclient equivalent of: > >> > >> > >> > >> OutputStream stream1 = ... > >> > >> OutputStream stream2 = ... > >> > >> > >> > >> while (input.read(byte[1024]) { > >> > >> stream1.write(bytes); > >> > >> stream2.write(bytes); > >> > >> } > >> > >> > >> > >> Please let me know if there is a way out? > >> > >> > >> > >> Thanks > >> > >> Sachin > >> > >> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
