[
https://issues.apache.org/jira/browse/HTTPCLIENT-1992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Serkan Turgut updated HTTPCLIENT-1992:
--------------------------------------
Description:
Message trailers in chunked messaging are being
[parsed|http://hc.apache.org/httpcomponents-core-5.0.x/httpcore5/xref/org/apache/hc/core5/http/impl/io/ChunkedInputStream.html#L284]
and [kept in
ChunkedInputStream|http://hc.apache.org/httpcomponents-core-5.0.x/httpcore5/xref/org/apache/hc/core5/http/impl/io/ChunkedInputStream.html#L88].
The parsed trailer headers are exposed with
[getFooters()|http://hc.apache.org/httpcomponents-core-5.0.x/httpcore5/xref/org/apache/hc/core5/http/impl/io/ChunkedInputStream.html#L321]
method. However, there is no caller of this method in the entire call stack
during a chunked messaging (both in version 4 and 5).
HttpClient 5 brings the support for message trailers by creating an API in
HttpEntity interface named as
[getTrailers().|https://hc.apache.org/httpcomponents-core-5.0.x/httpcore5/apidocs/org/apache/hc/core5/http/HttpEntity.html#getTrailers(]).
Even though there are multiple implementations of these HttpEntity, none of
them reach to ChunkedInputStream.getFooters() to get the trailers and expose
them to caller of HttpEntity.getTrailers().
* AbstractHttpEntity
*
** Following implementations return hardcoded 'null' for getTrailers() because
the default implementation in AbstractHttpEntity returns null
** BasicHttpEntity
** BufferedHttpEntity
** ByteArrayEntity
** ByteBufferEntity
** EntityTemplate
** FileEntity
** InputStreamEntity
** SerializableEntity
** StringEntity
** ResponseEntityProxy
** HttpEntityWithTrailers
*** Only returns the trailer list given its constructor instead of getting
them from ChunkedInputStream.getFooters()
** HttpEntityWrapper
*** Call wrappterEntity.getTrailers(), if the underlying entity is one of them
in this list, this class also returns null
Currently the call chain is following (omitted irrelevant calls);
#1 MainClientExec.execute() -> creates ResponseEntityProxy regardless the
response is chunked or not.
#2 ResponseEntityProxy.enhance() -> wraps the underlying stream with
EofSensorInputStream which hides ChunkedInputStream.getFooters() method from
outside. Since EofSensorInputStream is unaware of chunked messaging and
trailers, there is no way to reach ChuckedInputStream.getFooters() from outside
after this point.
#3 EofSensorInputStream -> This is to watch the stream and free it if it's EOF.
#4 ChunkedInputStream.read()
#5 ChunkedInputStream.nextChunk()
#6 ChunkedInputStream.parseTrailerHeaders() -> Parses the trailer headers and
puts into ChunkedInputSteam.footers field, exposed by getFooters() method but
no one is calling this method.
Code snip to observe the behavior;
{code:java}
CloseableHttpClient httpClient =
HttpClients.custom().addResponseInterceptorLast(new HttpResponseInterceptor() {
@Override
public void process(HttpResponse response, EntityDetails entity,
HttpContext context) throws HttpException, IOException {
if (response instanceof CloseableHttpResponse) {
CloseableHttpResponse closeableHttpResponse =
(CloseableHttpResponse) response;
HttpEntity httpEntity = closeableHttpResponse.getEntity();
Supplier<List<? extends Header>> trailers =
httpEntity.getTrailers();
// --> 'trailers' is always NULL
}
}
}).build();
// Make Http requests to any server which can responds in 'Transfer-Encoding:
chunked' and send trailers, observe the trailer supplier above is always null.
{code}
*Proposed solution (the patch is also ready, I will submit a pull request if
agree on the problem and proposal)*
There needs to be a way to propagate what has been parsed in
ChunkedInputStream.parseTrailerHeaders() by calling
ChunkedInputStream.getFooters() and pass those headers to callers of
HttpEntity.getTrailers(). To overcome this issue, the proposal is the following;
* Implement a version of ResponseEntityProxy named as
ResponseChunkedEntityProxy for chunked messaging so it recognizes the
underlying stream is ChunkedInputStream and It can return a Supplier which
delegates the call to ChunkedInputStream.getFooters() when
ResponseChunkedEntityProxy.getTrailers() called and return the supplier for
trailers to be used by users of HttpClient at the end of the communication.
* Change
[MainClientExec.execute|https://hc.apache.org/httpcomponents-client-5.0.x/httpclient5/xref/org/apache/hc/client5/http/impl/classic/MainClientExec.html#L138]
in a way to use ResponseChunkedEntityProxy when the response header
'Transfer-Encoding: chunked' presents, so the HttpEntity.getTrailers() API can
return trailers parsed by ChunkedInputStream.
was:
Message trailers in chunked messaging are being
[parsed|http://hc.apache.org/httpcomponents-core-5.0.x/httpcore5/xref/org/apache/hc/core5/http/impl/io/ChunkedInputStream.html#L284]
and [kept in
ChunkedInputStream|http://hc.apache.org/httpcomponents-core-5.0.x/httpcore5/xref/org/apache/hc/core5/http/impl/io/ChunkedInputStream.html#L88].
The parsed trailer headers are exposed with
[getFooters()|http://hc.apache.org/httpcomponents-core-5.0.x/httpcore5/xref/org/apache/hc/core5/http/impl/io/ChunkedInputStream.html#L321]
method. However, there is no caller of this method in the entire call stack
during a chunked messaging (both in version 4 and 5).
HttpClient 5 brings the support for message trailers by creating an API in
HttpEntity interface named as
[getTrailers().|https://hc.apache.org/httpcomponents-core-5.0.x/httpcore5/apidocs/org/apache/hc/core5/http/HttpEntity.html#getTrailers(]).
Even though there are multiple implementations of these HttpEntity, none of
them reach to ChunkedInputStream.getFooters() to get the trailers and expose
them to caller of HttpEntity.getTrailers().
* AbstractHttpEntity
** Following implementations return hardcoded 'null' for getTrailers() because
the default implementation in AbstractHttpEntity returns null
** BasicHttpEntity
** BufferedHttpEntity
** ByteArrayEntity
** ByteBufferEntity
** EntityTemplate
** FileEntity
** InputStreamEntity
** SerializableEntity
** StringEntity
** ResponseEntityProxy
** HttpEntityWithTrailers
*** Only returns the trailer list given its constructor instead of getting
them from ChunkedInputStream.getFooters()
** HttpEntityWrapper
*** Call wrappterEntity.getTrailers(), if the underlying entity is one of them
in this list, this class also returns null
Currently the call chain is following (omitted irrelevant calls);
-1 MainClientExec.execute() -> creates ResponseEntityProxy regardless the
response is chunked or not.
-2 ResponseEntityProxy.enhance() -> wraps the underlying stream with
EofSensorInputStream which hides ChunkedInputStream.getFooters() method from
outside. Since EofSensorInputStream is unaware of chunked messaging and
trailers, there is no way to reach ChuckedInputStream.getFooters() from outside
after this point.
-3 EofSensorInputStream -> This is to watch the stream and free it if it's EOF.
-4 ChunkedInputStream.read()
-5 ChunkedInputStream.nextChunk()
-6 ChunkedInputStream.parseTrailerHeaders() -> Parses the trailer headers and
puts into ChunkedInputSteam.footers field, exposed by getFooters() method but
no one is calling this method.
Code snip to observe the behavior;
{code:java}
CloseableHttpClient httpClient =
HttpClients.custom().addResponseInterceptorLast(new HttpResponseInterceptor() {
@Override
public void process(HttpResponse response, EntityDetails entity,
HttpContext context) throws HttpException, IOException {
if (response instanceof CloseableHttpResponse) {
CloseableHttpResponse closeableHttpResponse =
(CloseableHttpResponse) response;
HttpEntity httpEntity = closeableHttpResponse.getEntity();
Supplier<List<? extends Header>> trailers =
httpEntity.getTrailers();
// --> 'trailers' is always NULL
}
}
}).build();
// Make Http requests to any server which can responds in 'Transfer-Encoding:
chunked' and send trailers, observe the trailer supplier above is always null.
{code}
*Proposed solution (the patch is also ready, I will submit a pull request if
agree on the problem and proposal)*
There needs to be a way to propagate what has been parsed in
ChunkedInputStream.parseTrailerHeaders() by calling
ChunkedInputStream.getFooters() and pass those headers to callers of
HttpEntity.getTrailers(). To overcome this issue, the proposal is the following;
* Implement a version of ResponseEntityProxy named as
ResponseChunkedEntityProxy for chunked messaging so it recognizes the
underlying stream is ChunkedInputStream and It can return a Supplier which
delegates the call to ChunkedInputStream.getFooters() when
ResponseChunkedEntityProxy.getTrailers() called and return the supplier for
trailers to be used by users of HttpClient at the end of the communication.
* Change
[MainClientExec.execute|https://hc.apache.org/httpcomponents-client-5.0.x/httpclient5/xref/org/apache/hc/client5/http/impl/classic/MainClientExec.html#L138]
in a way to use ResponseChunkedEntityProxy when the response header
'Transfer-Encoding: chunked' presents, so the HttpEntity.getTrailers() API can
return trailers parsed by ChunkedInputStream.
> Impossible to access trailer-headers available in chunked transfer-encoding
> ---------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1992
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1992
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
> Affects Versions: 5.0 Beta4, 5.0 Beta5
> Reporter: Serkan Turgut
> Priority: Major
>
> Message trailers in chunked messaging are being
> [parsed|http://hc.apache.org/httpcomponents-core-5.0.x/httpcore5/xref/org/apache/hc/core5/http/impl/io/ChunkedInputStream.html#L284]
> and [kept in
> ChunkedInputStream|http://hc.apache.org/httpcomponents-core-5.0.x/httpcore5/xref/org/apache/hc/core5/http/impl/io/ChunkedInputStream.html#L88].
> The parsed trailer headers are exposed with
> [getFooters()|http://hc.apache.org/httpcomponents-core-5.0.x/httpcore5/xref/org/apache/hc/core5/http/impl/io/ChunkedInputStream.html#L321]
> method. However, there is no caller of this method in the entire call stack
> during a chunked messaging (both in version 4 and 5).
> HttpClient 5 brings the support for message trailers by creating an API in
> HttpEntity interface named as
> [getTrailers().|https://hc.apache.org/httpcomponents-core-5.0.x/httpcore5/apidocs/org/apache/hc/core5/http/HttpEntity.html#getTrailers(]).
> Even though there are multiple implementations of these HttpEntity, none of
> them reach to ChunkedInputStream.getFooters() to get the trailers and expose
> them to caller of HttpEntity.getTrailers().
> * AbstractHttpEntity
> *
> ** Following implementations return hardcoded 'null' for getTrailers()
> because the default implementation in AbstractHttpEntity returns null
> ** BasicHttpEntity
> ** BufferedHttpEntity
> ** ByteArrayEntity
> ** ByteBufferEntity
> ** EntityTemplate
> ** FileEntity
> ** InputStreamEntity
> ** SerializableEntity
> ** StringEntity
> ** ResponseEntityProxy
> ** HttpEntityWithTrailers
> *** Only returns the trailer list given its constructor instead of getting
> them from ChunkedInputStream.getFooters()
> ** HttpEntityWrapper
> *** Call wrappterEntity.getTrailers(), if the underlying entity is one of
> them in this list, this class also returns null
>
> Currently the call chain is following (omitted irrelevant calls);
> #1 MainClientExec.execute() -> creates ResponseEntityProxy regardless the
> response is chunked or not.
> #2 ResponseEntityProxy.enhance() -> wraps the underlying stream with
> EofSensorInputStream which hides ChunkedInputStream.getFooters() method from
> outside. Since EofSensorInputStream is unaware of chunked messaging and
> trailers, there is no way to reach ChuckedInputStream.getFooters() from
> outside after this point.
> #3 EofSensorInputStream -> This is to watch the stream and free it if it's
> EOF.
> #4 ChunkedInputStream.read()
> #5 ChunkedInputStream.nextChunk()
> #6 ChunkedInputStream.parseTrailerHeaders() -> Parses the trailer headers
> and puts into ChunkedInputSteam.footers field, exposed by getFooters() method
> but no one is calling this method.
> Code snip to observe the behavior;
> {code:java}
> CloseableHttpClient httpClient =
> HttpClients.custom().addResponseInterceptorLast(new HttpResponseInterceptor()
> {
> @Override
> public void process(HttpResponse response, EntityDetails entity,
> HttpContext context) throws HttpException, IOException {
> if (response instanceof CloseableHttpResponse) {
> CloseableHttpResponse closeableHttpResponse =
> (CloseableHttpResponse) response;
> HttpEntity httpEntity = closeableHttpResponse.getEntity();
> Supplier<List<? extends Header>> trailers =
> httpEntity.getTrailers();
> // --> 'trailers' is always NULL
> }
> }
> }).build();
> // Make Http requests to any server which can responds in 'Transfer-Encoding:
> chunked' and send trailers, observe the trailer supplier above is always null.
> {code}
> *Proposed solution (the patch is also ready, I will submit a pull request if
> agree on the problem and proposal)*
> There needs to be a way to propagate what has been parsed in
> ChunkedInputStream.parseTrailerHeaders() by calling
> ChunkedInputStream.getFooters() and pass those headers to callers of
> HttpEntity.getTrailers(). To overcome this issue, the proposal is the
> following;
> * Implement a version of ResponseEntityProxy named as
> ResponseChunkedEntityProxy for chunked messaging so it recognizes the
> underlying stream is ChunkedInputStream and It can return a Supplier which
> delegates the call to ChunkedInputStream.getFooters() when
> ResponseChunkedEntityProxy.getTrailers() called and return the supplier for
> trailers to be used by users of HttpClient at the end of the communication.
> * Change
> [MainClientExec.execute|https://hc.apache.org/httpcomponents-client-5.0.x/httpclient5/xref/org/apache/hc/client5/http/impl/classic/MainClientExec.html#L138]
> in a way to use ResponseChunkedEntityProxy when the response header
> 'Transfer-Encoding: chunked' presents, so the HttpEntity.getTrailers() API
> can return trailers parsed by ChunkedInputStream.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
