And to circle back to Raman’s original question, I did find this example script [1] I wrote to help someone with a similar problem (encrypting flowfile attributes rather than JSON fields, but it can easily be modified).
[1] https://stackoverflow.com/a/40295725/70465 <https://stackoverflow.com/a/40295725/70465> Andy LoPresto [email protected] [email protected] PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jun 27, 2017, at 10:28 AM, Andy LoPresto <[email protected]> wrote: > > James, > > The Developer Guide [1] is not a step-by-step tutorial but does include some > excellent information. Here are some other resources that may be helpful but > are not official Apache NiFi documents: > > * Step-by-step guide to building custom processor (1.1.0) - > http://www.nifi.rocks/developing-a-custom-apache-nifi-processor-json/ > <http://www.nifi.rocks/developing-a-custom-apache-nifi-processor-json/> > * Bryan Bende’s guide (0.0.2-incubating) - > http://bryanbende.com/development/2015/02/04/custom-processors-for-apache-nifi > > <http://bryanbende.com/development/2015/02/04/custom-processors-for-apache-nifi> > * HCC article on building custom processors - > https://community.hortonworks.com/articles/4318/build-custom-nifi-processor.html > > <https://community.hortonworks.com/articles/4318/build-custom-nifi-processor.html> > > Hopefully those are sufficient to get started/familiar with the process. > > > [1] https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html > <https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html> > > Andy LoPresto > [email protected] <mailto:[email protected]> > [email protected] <mailto:[email protected]> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > >> On Jun 27, 2017, at 9:52 AM, James McMahon <[email protected] >> <mailto:[email protected]>> wrote: >> >> Andy and Joe, would you remind those of us who have not done it before with >> a link to an example showing how to build a Java implementation of a >> processor? I recall looking into that at one time and discovering that it >> can be a nontrivial effort. >> >> On Tue, Jun 27, 2017 at 12:42 PM, Andy LoPresto <[email protected] >> <mailto:[email protected]>> wrote: >> Hi Raman, >> >> I agree with Joe that you can use the ExecuteScript processor to prototype >> what you are looking for (and if you need any help with that, let me know). >> I also wanted to note that there are some outstanding Jiras which will >> hopefully make this type of activity easier in the near future, specifically >> NIFI-4132: >> >> * NIFI-3890 - Create Key Management Controller Service [1] >> * NIFI-3889 - EncryptContent processor should add encryption metadata as >> attributes [2] >> * NIFI-3834 - Implement encrypted content repository [3] >> * NIFI-1885 - Extend HashAttribute to provide multiple algorithms [4] >> * NIFI-3929 - Allow external key management for EncryptContent processor [5] >> * NIFI-2961 - Create EncryptAttribute processor [6] >> * NIFI-4132 - Create EncryptRecord controller service and processor [7] >> >> >> [1] https://issues.apache.org/jira/browse/NIFI-3890 >> <https://issues.apache.org/jira/browse/NIFI-3890> >> [2] https://issues.apache.org/jira/browse/NIFI-3889 >> <https://issues.apache.org/jira/browse/NIFI-3889> >> [3] https://issues.apache.org/jira/browse/NIFI-3834 >> <https://issues.apache.org/jira/browse/NIFI-3834> >> [4] https://issues.apache.org/jira/browse/NIFI-1885 >> <https://issues.apache.org/jira/browse/NIFI-1885> >> [5] https://issues.apache.org/jira/browse/NIFI-3929 >> <https://issues.apache.org/jira/browse/NIFI-3929> >> [6] https://issues.apache.org/jira/browse/NIFI-2961 >> <https://issues.apache.org/jira/browse/NIFI-2961> >> [7] https://issues.apache.org/jira/browse/NIFI-4132 >> <https://issues.apache.org/jira/browse/NIFI-4132> >> >> >> >> Andy LoPresto >> [email protected] <mailto:[email protected]> >> [email protected] <mailto:[email protected]> >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >> >>> On Jun 27, 2017, at 4:45 AM, Ramaninder Singh Jhajj >>> <[email protected] <mailto:[email protected]>> wrote: >>> >>> Thanks a lot Joe for the quick response. >>> >>> That seems like a good solution. Will dig deep into it. >>> >>> Thanks a lot. >>> >>> On Tue, Jun 27, 2017 at 1:40 PM, Joe Witt <[email protected] >>> <mailto:[email protected]>> wrote: >>> Raman, >>> >>> There are no out of the box apache nifi processors to encrypt specific >>> fields of a given record. With the record reader/writer capabilities >>> that came out in 1.3.0 we can probably much more easily provide one >>> but in the mean time it is a pretty straight forward extension to >>> build. You could use execute script to nail down requirements and >>> understand throughput needs then build a Java implementation of the >>> processor to optimize performance. >>> >>> Thanks >>> Joe >>> >>> On Tue, Jun 27, 2017 at 7:33 AM, Ramaninder Singh Jhajj >>> <[email protected] <mailto:[email protected]>> wrote: >>> > Hello Everyone, >>> > >>> > I need some help with Nifi. I have a requirement where we need to encrypt >>> > and hash some of the data in a flowfile instead of the whole flowfile. The >>> > situation is, we receive customer information in a flowfile in XML >>> > format, I >>> > do some cleanup and tranform the flowfile content in JSON content. Now the >>> > requirement if we need to encrypt some of the sensitive data of >>> > customer(only some attributes of the JSON flowfile). So if customer JSON >>> > contains name, phoneno, email, SSN etc. we just want to encrypt email and >>> > SSN and not the whole file. >>> > >>> > I know there is a processor for encryption but that encrypts the whole >>> > flowfile. Is there a processor or a way to encrypt some of the attributes >>> > of >>> > JSON file. We do not want to use "ExecuteScript" processor as it can not >>> > be >>> > parallelised and will run the script in linear fashion (let me know if I >>> > am >>> > wrong here). >>> > >>> > >>> > Kind Regards, >>> > Raman >>> >> >> >
signature.asc
Description: Message signed with OpenPGP using GPGMail
