pjfanning opened a new pull request, #1233:
URL: https://github.com/apache/pekko-http/pull/1233
### Motivation
Client responses are always dispatched with streamed entities, so
applications have to consume or discard each response entity before the
connection can be reused. Applications that only ever work with fully buffered
responses end up calling `toStrict` on every response themselves.
Note that the client already produces `HttpEntity.Strict` opportunistically
(when the whole body arrives together with the headers in one buffer), so
"sometimes strict" is already the observable behaviour — this just makes it
deterministic when asked for.
### Modification
Two new settings in `pekko.http.client`:
```hocon
# `off` by default - response entities stay streamed
strict-response-entity-timeout = off
strict-response-entity-max-bytes = ${pekko.http.parsing.max-to-strict-bytes}
```
When a timeout is configured, `OutgoingConnectionBlueprint` collects every
response entity into an `HttpEntity.Strict` (reusing the existing
`impl.util.ToStrict` stage) before the response leaves the connection layer.
Entities that are already strict pass through untouched.
Because `PoolInterface` builds its connections through
`Http().outgoingConnectionUsingContext`, this covers the connection-level API,
the host-level API and `singleRequest` in one place. Responses on an HTTP/1.1
connection are strictly sequential, so collecting one entity never holds up
another response.
Caveats, documented in `reference.conf` and in the client configuration docs:
- the complete response body is buffered in memory;
- a response that is not fully received in time fails with a
`TimeoutException`, one that exceeds `strict-response-entity-max-bytes` with an
`EntityStreamException`, and in both cases the connection is failed too, since
the rest of the body cannot be skipped safely;
- trailing headers of chunked responses are dropped, as they are with
`HttpEntity.toStrict`;
- HTTP/2 is not covered. The HTTP/2 client is a separate stack and doing the
same there would serialize response emission across concurrent streams
(head-of-line blocking), so it is deliberately left out.
Draft: opening for feedback on the setting names/shape and on whether the
HTTP/2 side should be tackled in the same PR before this is marked ready.
### Result
The HTTP/1.1 client can be configured to hand out strict response entities.
Default behaviour is unchanged.
### Tests
- `sbt "http-core/testOnly
org.apache.pekko.http.impl.engine.client.LowLevelOutgoingConnectionSpec"` -
pass (6 new cases: chunked, default, close-delimited, already-strict,
over-limit, and streamed-by-default)
- `sbt "http-core/testOnly
org.apache.pekko.http.impl.engine.client.HttpConfigurationSpec"` - pass (2 new
cases for defaults and pool propagation)
- `sbt http-core/mimaReportBinaryIssues` - pass (no new filters needed)
- `sbt ++3.3.8 http-core/Test/compile` - pass
- `sbt docs/paradox` - pass
- `scalafmt --list --mode diff-ref=origin/main` - no changes
### References
None - makes client response entity handling configurable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
