Yes, you are right - the 'Accept-Encoding: gzip' header is the cause.
Adding that to the curl request causes the response to block, while
removing it from Apache HTTP client request prevents the response blocking.
But I'm not sure why this would be the case. Gzip compression is
something that can be done of a streaming basis so there should be no
need to read the entire dataset.
Turning off compression will have a big performance impact!
Tim
On 16/01/2019 18:48, Luiz Eduardo Valmont wrote:
Hi Tim!
Sorry about the delay. I was summoned into a meeting that took a while.
Plus my PC is in the process of upgrading from Debian 9 to Debian
10/testing.
On the important matter.
There are certainly differences when it comes to headers. But I suspect the
culprit is the accept-encoding. I don't think it's the Camel headers.
I believe because it has to compress the content, it has to read everything
beforehand. Try adding "-H 'Accept-Encoding: gzip'" (double quotes
excluded) to your curl command and see if it changes the actual behaviour.
If it does, then you'll have to configure your client accordingly.
In any case, you can emulate the Async request in full by configuring the
headers with curl's "-H" parameter.
HTH
On Wed, 16 Jan 2019, 15:17 Tim Dudgeon <tdudgeon...@gmail.com wrote:
Hi Luiz,
here are the request headers. For Apache HTTPClient:
accept -> chemical/x-mdl-sdfile
accept-encoding -> gzip
breadcrumbId -> ID-6667e3dab68a-38877-1547658276076-0-15
CamelHttpMethod -> POST
CamelHttpPath ->
CamelHttpQuery -> null
CamelHttpServletRequest ->
org.apache.catalina.connector.RequestFacade@7551baa3
CamelHttpServletResponse ->
org.apache.catalina.connector.ResponseFacade@60e05546
CamelHttpUri ->
/chem-services-cdk-basic/rest/v1/converters/dataset_to_sdf
CamelHttpUrl ->
http://localhost:8092/chem-services-cdk-basic/rest/v1/converters/dataset_to_sdf
CamelServletContextPath -> /v1/converters/dataset_to_sdf
connection -> Keep-Alive
content-encoding -> gzip
Content-Type -> application/x-squonk-dataset-molecule+json
host -> localhost:8092
transfer-encoding -> chunked
user-agent -> Apache-HttpAsyncClient/4.1.4 (Java/1.8.0_191)
For curl:
accept -> chemical/x-mdl-sdfile
breadcrumbId -> ID-09b7601451a2-34027-1547658868812-0-15
CamelHttpMethod -> POST
CamelHttpPath ->
CamelHttpQuery -> null
CamelHttpServletRequest ->
org.apache.catalina.connector.RequestFacade@3af9ee39
CamelHttpServletResponse ->
org.apache.catalina.connector.ResponseFacade@1f33d20b
CamelHttpUri ->
/chem-services-cdk-basic/rest/v1/converters/dataset_to_sdf
CamelHttpUrl ->
http://localhost:8092/chem-services-cdk-basic/rest/v1/converters/dataset_to_sdf
CamelServletContextPath -> /v1/converters/dataset_to_sdf
content-encoding -> gzip
content-length -> 19397182
Content-Type -> application/x-squonk-dataset-molecule+json
expect -> 100-continue
host -> localhost:8092
user-agent -> curl/7.55.1
On 16/01/2019 16:55, Luiz Eduardo Valmont wrote:
Hi, Tim!
I'd suggest trying to intercept / dump the HttpAsyncClient request as
well
as the curl one. A simple "nc -l -p 8888" in an *NIX OS will do.
My guess is that the perceived difference lies in the request headers.
Might be in the Accept.
Can you post both requests in full, body excluded?
Thanks!
On Wed, 16 Jan 2019, 14:46 Tim Dudgeon <tdudgeon...@gmail.com wrote:
I'm hitting a strange problem with a route that is using the servlet
component to send a large stream of data in its response.
This is part of a route that uses the REST DSL which is set up like
this:
restConfiguration().component("servlet").host("0.0.0.0");
The route sets the response body as an PipedInputStream and then (in a
separate thread) writes to that InputStream and eventually closes it.
The expectation is that as the response size is unknown the data will be
chunked and response returned to the client immediately and the client
will start to consume the data immediately.
When using curl as the client it seems to be working correctly, and I
see a 'Transfer-Encoding: chunked' header in the response.
However my real client is Apache HttpAsyncClient and with that what I'm
seeing is that the response is not returned until all the data is
written, and instead of a 'Transfer-Encoding: chunked' header I see a
'Content-Length: 19196336' header suggesting that Camel has waited for
the entire content to be written so that it can set the Content-Length.
I tried manually setting the 'Transfer-Encoding: chunked' in the
response message but it did not appear and instead I saw the
Content-Length header.
I've got HttpAsyncClient working in a test example to prove that it can
received the response asynchronously, so I think (but can't be sure)
that the problem is on the server (Camel) side.
What can explain this different behaviour from the same service when
using different clients?
