Re: PGP Encrypt using spark Scala

I want to add that the major hadoop distributions also offer additional encryption possibilities (for example Ranger from Hortonworks) Roland Johann Software Developer/Data Engineer phenetic GmbH Lütticher Straße 10, 50674 Köln, Germany Mobil: +49 172 365 26 46 Mail: roland.joh...@phenetic.io W

Re: PGP Encrypt using spark Scala

Hi all, instead of handling encryption explicit at application level, I suggest that you investigate into the topic „encryption at rest“, for example encryption at HDFS level https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/TransparentEncryption.html

Re: PGP Encrypt using spark Scala

Hi Deepak, Thanks for reply. Yes. That is the option I am considering now because even apache camel needs data in local. I might need to copy data from hdfs to local if I want apache camel ( to get rid of shell). Thanks Sachit On Mon, 26 Aug 2019, 21:11 Deepak Sharma, wrote: > Hi Schit > PGP

Re: PGP Encrypt using spark Scala

Hi Schit PGP Encrypt is something that is not inbuilt with spark. I would suggest writing a shell script that would do pgp encrypt and use it in spark scala program , which would run from driver. Thanks Deepak On Mon, Aug 26, 2019 at 8:10 PM Sachit Murarka wrote: > Hi All, > > I want to encrypt

PGP Encrypt using spark Scala

Hi All, I want to encrypt my files available at HDFS location using PGP Encryption How can I do it in spark. I saw Apache Camel but it seems camel is used when source files are in Local location rather than HDFS. Kind Regards, Sachit Murarka