Eric Rescorla wrote:
>
> I'm not quite following who's who in this scenario, so some potentially
> stupid
> questions below.
>
> As I understand it, you have the following situation:
>
> - A Web application server
> - Some middleware, which comes in two pieces
> - A crypto-unaware network component
> - The TLS stack (you control this piece as well, right?)
> - The client
This is about any conceivable scenario involving at least one of our
components, just client, just server, or at both peers.
The "middleware" is part of our clients and servers. The middleware
performs all the network I/O and necessary calls into the TLS stack,
offers an appdata streaming convenience option for reading,
variable blocking I/O (non-blocking (0ms) up to infinite timeout).
TLS records of type Handshake, Alert and CCS are always processed
in batch, as many as the network buffers have already received.
TLS records of type AppData are processed based on read strategy
desired by the application caller. The desired reading strategy
(trickling, efficient, streaming) is a parameter of the middleware
read API call, so the app could change it for every call.
- Trickling means the traditional TLS record reading, i.e. two network
read calls for every TLS record.
- Improved means on average one network read call per TLS record.
- Streaming means reading and decoding as many TLS appdata record
as there are present in the network read buffers and can be
received non-blocking. Only TLS AppData records will be passed
to the TLS Stack for decoding, Alerts and Handshake records
will be left in the middlewares network read buffers until all
appdata as been properly received by the application caller.
Only upon the next read call from the application, the alert
or handshake records will be passed to the TLS stack.
Example: GET https://www.google.de/ HTTP/1.0
(where my call to www.google.com is redirected to...)
currently returns a HTTP-response with a 657 byte Header and 44879 byte Body
The response would easily fit into 3 TLS appdata records
(4 Records if the Header is sent in its own TLS record), however
in reality, the Google servers perform pathological fragmentation
of the AppData and use a whooping 36 TLS records for the response
(curiously no TLS AppData record split between header and body).
Processing overhead for _receiving_ such a badly fragmented response:
Trickling:
34 Read calls into the middleware
73 recv() calls to read the TLS AppData records from the socket
34 calls into the TLS stack
Improved:
34 Read calls into the middleware
42 recv() calls to read the TLS AppData records from the socket
34 calls into the TLS stack
Streaming:
5 Read calls into the middleware
14 recv() calls to read the TLS AppData records from the socket
34 calls into the TLS stack
>
> When do you deliver the close_notify and how do you know how?
When the application calls for read, and has already seen all
prior AppData, then the close_notify will be processed and the
result (connection closure, no more data) reported to the calling app.
There is no magic involved here.
Apps *know* when to perform reads, and when not to perform reads.
The middleware doesn't because it is protocol ignorant (it is used
by SMTP, ldaps, HTTPS, HTTP/2.0, websockets, etc).
Think of a simple HTTP-based server app receiving a HTTP/1.0 request
from a TLS-stack on top of a blocking network socket. If the App would
keep calling SSL_read() (for an OpenSSL-style API), it would get stuck
(blocked on read), and at some point the client would give up and
close the connection (with close_notify or TCP RST), and the server
waking from either the processing of the close_notify or the TCP RST
would be unable to deliver a response (which the client would not read
anyway).
Whether or not the calling App wants to shutdown a communication
at different times in both directions depends on the existing semantics
of that application (which has just added TLS protection around its
communication). Reading and processing a close_notify in the TLS stack
(e.g. OpenSSL) will tear down *BOTH* directions immediately, and preclude
any further of sending of responses by the application, so the middleware
really will want to hold of processing of close_notify alerts unless
_explicitly_ asked to read further AppData by the application.
With TLS up to TLSv1.2 streaming is no problem, the middleware can
easily recognize non-AppData records and avoid passing them to
the TLS stack for processing unless the application explicitly asks
the middleware to do so. When TLSv1.3 hides the ContentType,
the fact that a close_notify was received & processed can only be
determined after the fact, when the shattered pieces are on the floor.
Communication in the other direction will be impossible, and it will
not be possible to prevent this from happening.
While it is conceivable to jump hoops and implement new APIs and
callback for the TLS stack
