reading entity bytes from a HttpEntityEnclosingRequest in case of large content size

[Brijesh Deo]

Hi,

 

I am implementing an http reverse proxy, somewhat on the lines of the
sample implementation - ElementalReverseProxy with some modifications. I
am using HttpCore classes to implement the http server (using
org.apache.http.protocol.HttpService) and the HttpClient api (instead of
the HttpRequestExecutor) to implement the client side for sending the
Http requests to the target servers and receiving Http Responses from
them. I am using the latest HttpClient 4.0 binaries bundle (with
dependencies).

 

I have a requirement to store Http Post data from some webpages
(accessed through my reverse proxy) when they are submitted and later
Post the stored data to a different target server or the same target
server. For this, I read the entity data into a byte array whenever a
HttpPost request comes into my http server. I can store the byte array
into a file/database table. Then I recreate a ByteArrayEntity from the
saved byte array and re-populate the HttpPost request object and send it
to the intended target server using the HttpClient instance. This works
fine when the content type is "application/x-www-form-urlencoded" or
even "multipart/form-data". 

 

However, in case of "multipart/form-data", if the content size is huge
(in cases of larger file uploads > 50MB) I have some doubts/concerns
whether I will be able to read all the entity data completely. I am
reading the entity data in the way shown below: My questions are
mentioned below.

 

if (request instanceof HttpEntityEnclosingRequest) {

      HttpEntity entity =
((HttpEntityEnclosingRequest)request).getEntity();

      byte entityBytes[] = null;          

      if (entity != null) {

ByteArrayOutputStream baos = new ByteArrayOutputStream() ;

            entity.writeTo(baos);

            entityBytes = baos.toByteArray() ;

            baos.flush();

            baos.close();

            entity.consumeContent();

            Question 1: Does calling consumeContent() here close the
underlying socket connection with the client(browser)? In that case, how
would the HttpResponse be sent back to the client(browser) using this
connection?

      }

if (entityBytes != null) {

            ByteArrayEntity byteArrayEntity = new
ByteArrayEntity(entityBytes);

 
((HttpEntityEnclosingRequest)request).setEntity(byteArrayEntity);

      }

}

 

Question 2: As mentioned above, if it is a large file upload request
(multipart/form-data), is it guaranteed that I will be able to read all
the bytes of the file being uploaded with the above approach? 

 

Question 3: Is there any possibility that I could receive more than one
HttpRequest object(s) for such a (multipart/form-data) post with the
bytes of the uploaded file contained partially in each of those request
object entities? If that is a possibility, how can I associate/append
all such request entities to form a single piece of byte array?

 

Question 4: Is there a more efficient method to achieve this using some
other classes/methods of the HttpCore and HttpClient apis?

 

I would be thankful for any guidance or help here.

 

Thanks,

Brijesh