On Fri, 2021-06-25 at 10:39 +0800, 雷咩咩 wrote:
> Hi Developers,
> 
> 
>    This is my first time to write to apache mailing list.
>     Recently I(and we, our company team) met some issue
> when reading chunked transfer encoding response. 
> 
> 
>    Goal: use the library's built in functions(not manully
> bother with http protocal internals such as chunk endings) to get the
> chunks, of the exactly same chunks count, and each chunk
> with exactly same bytes as the server sends.
> 
> 
> 
>    Problem: often(if not always) got chunks that are
> splitted from one to two or more. The result varies on different
> rounds of tests(somehow random).
> 
> 
> 
> 
>    I've post the question to stackoverflow(
> https://stackoverflow.com/questions/68077410/java-apache-http-components-messes-up-http-chunked-transfer-encoding-response
> ), but it is closed now(I'm angry with that!)
> 
>    Also the demo code to reproduce:  
> https://github.com/LeiYangGH/http-chunk-tests
> 
> 


Yang

I usually do not respond to messages that require moderation. Please
subscribe prior to posting to any of the ASF mailing lists.

As to your question

1. HttpClient makes no guarantees it returns the exact same size of
chunks for chunk coded message bodies mainly for performance
optimization reasons but also because transfer encoding is a transport
aspect and it should be completely transparent to the application
layer. HTTP messages can be delivered on a hop by hop basis and various
intermediate proxies are permitted to change transfer encoding of
messages as they see fit.  

So, your reliance on specific composition of transfer chunks is
conceptually wrong. If your application transfers sequences of
structured data it should be using an appropriate content format for
that end, such as JSON, instead of abusing HTTP transfer coding of the
transport library. 


2. I am quite certain if you configure HttpClient to use larger
connection buffers (100K) you are likely to get the same result as with
your Python code.

Oleg


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Reply via email to