We are now running up against this issue in practice, so we need a
workaround.
To recap: if a client makes a large upload request to an Akka HTTP
("akka-stream-and-http-experimental-1.0-M2") and the server responds (i.e.
returns an HttpResponse to the "handlerFlow" function) before reading the
full request body, then the TCP stream stalls.
The client receives the response, but thinks that the stream is still
valid. It will then send the next HTTP request on the same TCP stream, but
Akka will never read the request, as it’s still waiting for the (now
aborted) request stream to finish reading in the responded-to request.
So, for our Akka HTTP server to be well behaved, we must either always read
in the full request and/or half-close the TCP upload stream.
How do I do those two things?
I'd like to do this in my my routing layer, in case there is an exception
in the processing layer.
How can I detect whether the request stream needs cancelling (i.e. whether
it has not been fully read)? Or is it safe to always cancel?
How can I "cancel" the stream? Given an instance of HttpRequest, the entity
has “dataBytes”, but that’s not a materialized stream, but a Source. How do
I get a reference to the materialized stream to cancel it? I tried
"request.entity.dataBytes.runWith(Sink.cancelled)", but that seems wrong.
Does cancelling the stream half-close the TCP upload stream? If not, how do
I do that? (Send Akka a "connection: close" header on the response? Again
-- how do I detect when that's needed, i.e. when the request has not been
fully read?)
(see also https://github.com/akka/akka/issues/16510 )
Many thanks,
Rich
On Friday, January 30, 2015 at 10:15:10 AM UTC, Björn Antonsson wrote:
>
> Hi Joe,
>
> If you properly cancel the request stream I don't think that you need to
> consume it with a black hole sink.
>
> B/
>
> On 29 January 2015 at 11:30:32, Joe Edwards ([email protected]
> <javascript:>) wrote:
>
> Thanks for the explanation.
>
> Unfortunately I don't control the clients so it sounds like we have to
> send a 'Connection: Close' header and hope the client aborts the request
> for us, otherwise we have to read (and discard) the whole request to ensure
> the client will read the response. That's a bit of a pain, but I suppose
> it's really the client's problem if they insist on ignoring the response
> until they've uploaded their whole 2GB message.
>
> Re: akka-http Is it true once the stream processing the request is
> cancelled we still need to attach a (blackhole) sink to the request stream,
> otherwise the back-pressure may simply stall the client indefinitely?
>
> On Wednesday, 28 January 2015 19:33:58 UTC, Daniel Armak wrote:
>>
>> I can't comment on akka-http, but here's my understanding of the HTTP
>> spec.
>>
>> It's certainly legal to send a response before receiving the request
>> entity, but the client isn't obligated to start reading the response before
>> it finishes sending the request. And if the server closes the connection
>> without reading the whole request, the client may not see the response,
>> depending on whether it fit into its receive buffer and on how it's
>> written. If you just close the connection, the client might think it a
>> network error and retry the request. So if you want to guarantee the client
>> sees the rejection response, you have to read and discard the whole
>> request.
>>
>> If you control the client as well as the server, and the server can
>> reject the request based on its headers, the standard HTTP solution is to
>> send Expect: 100-continue in the request, and then either 100 Continue or a
>> 4xx rejection from the server. But if the server can only reject the
>> request based on some prefix of the entity, HTTP provides no way to abort
>> without closing the connection. You could cheat though: use chunked
>> encoding for the request entity, and send an empty chunk ending the request
>> entity as soon as you read the server's rejection headers.
>>
>> On Wednesday, January 28, 2015 at 6:33:47 PM UTC+2, Joe Edwards wrote:
>>>
>>> I have an application in which clients can upload large request entities
>>> to the server. In some situations the request will be rejected early
>>> (before the request body is totally consumed, as this may be hundreds of
>>> MB).
>>>
>>> Is there a standard way of handling this situation? Since the rejections
>>> may happen asynchronously, I can't guarantee how much (if any) of the
>>> request body has been read.
>>>
>>> As far as I can tell from RFC 7230 section 6.5
>>>
>>> - If the request is still in flight, we should respond early with a
>>> 'Connection: Close' (and the request stream will die as the connection
>>> closes).
>>> - If the request has been totally received, we can respond normally
>>> and reuse the connection.
>>>
>>> Is there any easy way to determine when a response is being sent early?
>>>
>>> More generally, is it even *legal* to send a response before the entire
>>> request is received *without* closing the connection? I can't really
>>> tell from the spec, but if not *it would make much more sense for
>>> akka-http to close the connection for us*.
>>>
>>> Thanks, Joe
>>>
>> --
> >>>>>>>>>> Read the docs: http://akka.io/docs/
> >>>>>>>>>> Check the FAQ:
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected] <javascript:>.
> To post to this group, send email to [email protected]
> <javascript:>.
> Visit this group at http://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> Björn Antonsson
> Typesafe <http://typesafe.com/> – Reactive Apps on the JVM
> twitter: @bantonsson <http://twitter.com/#!/bantonsson>
>
>
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
