>On Tue, 2020-10-06 at 02:20 +0000, Cesar, Scott wrote:
>> Hi,
>> I'm working to build a streaming HTTP library for Apache Beam. As
>> part of providing a low entry effort
>> interface we want to provide asynchronous and synchronous options
>> which rely on
>> SimpleHttpRequest / SimpleHttpResponse.
>>
>> As the current synchronous client only handles ClassicHttpRequests,
>> I'm currently using:
>> public static ClassicHttpRequest convert(SimpleHttpRequest original){
>> BasicClassicHttpRequest copy = new
>> BasicClassicHttpRequest(original.getMethod(),
>> original.getRequestUri());
>> copy.setVersion(original.getVersion());
>> for (final Iterator<Header> it = original.headerIterator();
>> it.hasNext(); ) {
>> copy.addHeader(it.next());
>> }
>> copy.setScheme(original.getScheme());
>> copy.setAuthority(original.getAuthority());
>>
>> HttpEntity entity;
>> SimpleBody body = original.getBody();
>> if(body != null){
>> if (body.isBytes()){
>> entity = new ByteArrayEntity(body.getBodyBytes(),
>> body.getContentType());
>> } else{
>> entity = new StringEntity(body.getBodyText(),
>> body.getContentType());
>> }
>> copy.setEntity(entity);
>> }
>> return copy;
>> }
>>
>> To convert simple requests into classic ones. Is this the best way to
>> go about it (and if so, would this be
>> welcome as a pull request on SimpleHttpRequest), or is there a better
>> way to provide this support (e.g.
>> extending the classic client to be able to handle SimpleRequests
>> directly)
>>
>> Similarly, I'm using a relatively involved method of greedily
>> consuming a ClassicHttpResponse to produce a
>> SimpleHttpResponse from the classic client (code included at the
>> end), and am curious if there's a better way
>> of doing this conversion or, if not, if providing a
>> SimpleHttpResponseHandler and making it available from the client
>> would be a good way of providing a unified interface going forewards.
>>
>> public class SimpleHttpResponseHandler implements
>> HttpClientResponseHandler<SimpleHttpResponse> {
>>
>> @Override
>> public SimpleHttpResponse handleResponse(ClassicHttpResponse
>> response) throws HttpException, IOException {
>> SimpleHttpResponse simple =
>> SimpleHttpResponse.copy(response);
>> HttpEntity ent = response.getEntity();
>> if(ent != null){
>> byte[] data;
>> if (ent.getContentLength() >0) {
>> if(ent.getContentLength() > Integer.MAX_VALUE){
>> throw new RuntimeException("Received to much data
>> to buffer in memory, try using a streaming consumer");
>> }
>> data = new byte[(int) ent.getContentLength()];
>> int read = ent.getContent().read(data);
>> if(read != ent.getContentLength())
>> throw new RuntimeException("Received "+read+"
>> bytes instead of "+ent.getContentLength());
>> } else{
>> final int bufferSize = 8192;
>> InputStream is = ent.getContent();
>> ArrayList<byte[]> buffers = new ArrayList<>();
>> byte[] swapBuf = new byte[bufferSize];
>> int read = is.read(swapBuf);
>> while(read != -1 && read == bufferSize){
>> buffers.add(swapBuf);
>> swapBuf = new byte[bufferSize];
>> read = is.read(swapBuf);
>> }
>>
>> int content = bufferSize*buffers.size();
>> if(read != -1)
>> content += read;
>>
>> if(content <= 0)
>> throw new RuntimeException("Received to many
>> bytes to buffer in memory");
>> data = new byte[content];
>>
>> int index = 0;
>> for (final byte[] toCpy : buffers){
>> System.arraycopy(toCpy, 0, data,
>> bufferSize*index, bufferSize);
>> index +=1;
>> }
>> if(read != -1)
>> System.arraycopy(swapBuf, 0, data,
>> bufferSize*index, bufferSize);
>>
>> swapBuf = null;
>> buffers.clear();
>> }
>>
>> simple.setBody(
>> data,
>> ContentType.create(ent.getContentType())
>> );
>> }
>> return simple;
>> }
>> }
>>
>>
>> Thank you for your time,
>> Scott
>
>Hi Scott
>
>Your code looks correct. However I am wondering why one would want to
>convert streaming messages into simple ones in the first place.
>
>Oleg
Hi Oleg,
There are two reasons I'm interested in turning the streaming responses from
the classic client into simple messages:
1) Allowing users to provide very simple in memory parsing solutions which will
run against both synchronous and asynchronous modules.
2) Beam has very restrictive serializability requirements. This means not all
out of the box streaming consumption tools available can be put to work, at
least not without some syntactic tradeoffs, so providing users simple and
involved platforms to build off of seems to make the most sense.
Scott
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org
