Hi Sijie, May I please know is this already part of backlog or something community need to analyze and add it to the backlog? Because we are planning to take Pulsar 2.5.1 for implementation and targeting to release it as part of our product by September 2020.
If this can’t be adapted in Kafka-Adaptor before we start our implementation (next month), we can look for alternate solutions. Regards, Subash Kunjupillai From: Subash K Sent: Friday, April 24, 2020 8:51 AM To: [email protected] Subject: RE: Data Encryption Yes, this approach looks promising to me as of now. Regards, Subash Kunjupillai From: Sijie Guo <[email protected]<mailto:[email protected]>> Sent: Thursday, April 23, 2020 10:21 PM To: [email protected]<mailto:[email protected]> Subject: Re: Data Encryption I see. There is one approach we can explore - add a CryptoKeyReader implementation into the Kafka Adaptor and let user only can configure the key files. If you are loading the properties from a properties file, this approach might work. - Sijie On Thu, Apr 23, 2020 at 3:05 AM Subash K <[email protected]<mailto:[email protected]>> wrote: Hi Sijie, We see only Pulsar URL and Topic to be changed to run our application AS-IS in Pulsar with Kafka Adaptor. Already we had these (URL and Topic) as a configuration so we were able to change the configuration and run the application AS-IS on top of Pulsar by adding pulsar-client-kafka to the classpath. I’m not really sure on how to do this without modifying our application code to achieve End-to-End encryption. Is there any example code that you can point us to where this was achieved? Regards, Subash Kunjupillai From: Sijie Guo <[email protected]<mailto:[email protected]>> Sent: Thursday, April 23, 2020 12:24 PM To: [email protected]<mailto:[email protected]> Subject: Re: Data Encryption Subash, I think End-to-End Encryption is the only feasible solution for your requirement at this moment. Out of curiosity, if you are using Kafka Adaptor, you anyway need to re-compile your consumer and producer with the Kafka adaptor. Are you able to specify additional settings in the properties used for constructing Kafka producer and consumer? If you can do that, it should be easy to inject the CryptoKeyReader that. - Sijie On Wed, Apr 22, 2020 at 7:39 AM Subash K <[email protected]<mailto:[email protected]>> wrote: Hi, I’m evaluating on the encryption feature provided by Pulsar. We need to encrypt data at wire-level and at rest, also we are having an existing Kafka application which we are planning to port to Pulsar using Kafka adaptor without any code change. Now I understand that Encryption of data is possible pulsar in below ways: 1. End-to-End Encryption: From my understanding, this method covers both transport and at rest encryption of data and looks a viable option. But this needs adaptation to our producer and consumer to implement CryptoKeyReader, which is not possible for us as we are planning to port our Kafka producer and consumer as is. 2. Encryption using TLS: In this option, I see only the transport layer is encrypted but the data stored by Bookkeeper will be in plain text. Can someone let me know is there any possible way to encrypt data at both transport and at rest if our applications are using Kafka Adaptor? Regards, Subash Kunjupillai
