If you are doing batch encryption then you are more similar to a scenario of file encryption. The more frequent the messages are you are closer to the ssl/https scenarios. You may learn from those protocols on how they handle keys, how long they keep them etc. to implement your E2e solution .
> Am 08.08.2019 um 08:11 schrieb Maulin Vasavada <maulin.vasav...@gmail.com>: > > Hi Sönke Liebau > <https://www.mail-archive.com/search?l=dev@kafka.apache.org&q=from:%22S%C3%B6nke+Liebau%22> > > Thanks for the great detailed documentation. However, I feel by leaving the > KMS outside of Kafka might simplify the whole thing to a great extent. If > the broker is not going to touch the encrypted messages, why would we put > any dependency of KMS interfaces on the Broker. We have experimented doing > end-to-end message encryption and we used topic level keys and message > encryption with serializer wrapper which encrypts each message before > serializing. The serializer wrapper have to integrate with required KMS we > use internally and that was all. > > However one key observation we had was - if we could do encryption at > 'batch' level instead of 'per-message' it can perform much better > (depending upon batch sizing). We didn't experiment with that though. > > Thanks > Maulin
