Hey folks, For those not following along, we're very close to the point where we'll be enabling TLS for all wire communication done by a Kudu cluster (at least when security features are enabled). One thing we've decided is important is to preserve good performance for applications like Spark and Impala which typically schedule tasks local to the data on the tablet servers, and we think that enabling TLS for these localhost connections will have an unacceptable performance hit.
Our thinking was to continue to use TLS *authentication* to prevent MITM attacks (possible because we typically don't bind to low ports). But, we don't need TLS *encryption*. This is possible using the various TLS "NULL" ciphers -- we can have both the client and server notice that the remote peer is local and enable the NULL cipher suite. However, I did some research this evening and it looks like the NULL ciphers disable encryption but don't disable the MAC integrity portion of TLS. Best I can tell, there is no API to do so. I did some brief checks using openssl s_client and s_server on my laptop (openssl 1.0.2g, haswell), and got the following numbers for transferring 5GB: ADH-AES128-SHA Client: 42.2M cycles Server: 35.3M cycles AECDH-NULL-SHA: (closest NULL I could find to the above) Client: 36.2M cycles Server: 28.6M cycles no TLS at all (using netcat to a local TCP port): Client: 20.8M cycles Server: 10.0M cycles baseline: iperf -n 5000M localhost Client: 2.3M cycles Server: 1.8M cycles [not sure why this is so much faster than netcat - I guess because with netcat I was piping to /dev/null which still requires more syscalls?] (note that the client in all of these cases includes the 'dd' command to generate the data, which probably explains why it's 7-10M cycles more than the server in every case) To summarize, just disabling encryption has not much improvement, given that Intel chips now optimize AES. The checksumming itself adds more significant overhead than the encryption. This agrees with numbers I've seen around the web that crypto-strength checksums only go 1GB/sec or so max, typically much slower. Thinking about the best solution here, I think we should consider using TLS during negotiation, and then just completely dropping the TLS (i.e not wrapping the sockets in TlsSockets). I think this still gives us the protection against the localhost MITM (because the handshake would fail) and be trivially zero-overhead. Am I missing any big issues with this idea? Anyone got a better one? -Todd -- Todd Lipcon Software Engineer, Cloudera
