We've been using SSL for produce traffic (mirror makers only right now, but that's a very large percentage of traffic for us), and we're in the process of turning it on for inter broker traffic as well. Our experience is that this does not present a significant amount of overhead to the brokers. Specifically with switching over the IBP, we were expecting a lot more of a hit, and it really only ended up being something like a 5% increase in system load, and no reduction in the cluster capacity, in our test cluster. Note that this relies on the fix in KAFKA-4050 and switching the PRNG to SHA1PRNG.
Right now, we're specifically avoiding moving consume traffic to SSL, due to the zero copy send issue. Now I've been told (but I have not investigated) that OpenSSL can solve this. It would probably be a good use of time to look into that further. That said, switching the message format to the newer option (KIP-31 I believe?) will result in the brokers not needing to recompress message batches that are produced. This should result in a significant reduction in CPU usage, which may offset the cost of SSL. We haven't had a chance to fully investigate this, however, as changing that config depends on the clients being updated to support the new format. -Todd On Sunday, September 4, 2016, Jaikiran Pai <jai.forums2...@gmail.com> wrote: > We are using 0.10.0.1 of Kafka and (Java) client libraries. We recently > decided to start using SSL for Kafka communication between broker and > clients. Right now, we have a pretty basic setup with just 1 broker with > SSL keystore setup and the Java client(s) communicate using the > Producer/Consumer APIs against this single broker. There's no client auth > (intentionally) right now. We also have plain text enabled for the initial > testing. > > What we have noticed is that the consumer/producer performance when SSL is > enabled is noticeably poor when compared to plain text. I understand that > there are expected to be performance impacts when SSL is enabled but the > order of magnitude is too high and in fact it shows up in a very noticeable > fashion in our product. I do have the numbers, but I haven't been able to > narrow it down yet (because there's nothing that stands out, except that > it's slow). Our application code is exactly the same between non-SSL and > SSL usage. > > Furthermore, I'm aware of this specific JIRA in Kafka > https://issues.apache.org/jira/browse/KAFKA-2561 which acknowledges a > similar issue. So what I would like to know is, in context of Kafka 0.10.x > releases and Java 8 support, are there any timelines that the dev team is > looking for in terms of improving this performance issue (which I believe > requires usage of OpenSSL or other non-JDK implementations of SSLEngine)? > We would like to go for GA of our product in the next couple of months and > in order to do that, we do plan to have Kafka over SSL working with > reasonably good performance, but the current performance isn't promising. > Expecting this to be fixed in the next couple of months and have it > available in 0.10.x is probably too much to expect, but if we know the > plans around this, we should be able to come up with a plan of our own for > our product. > > > -Jaikiran > -- *Todd Palino* Staff Site Reliability Engineer Data Infrastructure Streaming linkedin.com/in/toddpalino
