Hi, Based on the security wiki page <https://cwiki.apache.org/confluence/display/KAFKA/Security> encryption of data at rest is out of scope for the time being. However, we are implementing encryption in Kafka and would like to see if there is interest in submitting a patch got it.
I suppose that one way to implement encryption would be to add an 'encrypted key' field to the Message/MessageSet structures in the wire protocole - however, this is a very big and fundamental change. A simpler way to add encryption support would be: 1) Custom Serializer, but it wouldn't be compatible with other custom serializers (Avro, etc. ) 2) Add a step in KafkaProducer after serialization to encrypt the data before it's being submitted to the accumulator (encryption is done in the submitting thread, not in the producer io thread) Is there interest in adding #2 to Kafka? Cheers, Eugene
