Re: Data anonymization in Nifi

2017-10-31 Thread Matt Burgess
Vyshali, I would love to help, but I've never used ARX so I'm not at all familiar with their APIs. They do have an examples page though [1]. Regards, Matt [1] http://arx.deidentifier.org/overview/#a3 On Tue, Oct 31, 2017 at 1:11 PM, Vyshali wrote: > Hi Matt, > >

Re: Data anonymization in Nifi

2017-10-31 Thread Vyshali
Hi Matt, Thanks for your valuable comment. Is it possible to anonymize data without specifying generalization hierarchies in ARX.? Also,can you please help me with some basic examples using ARX APIs. Regards, Vyshali -- Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: Data anonymization in Nifi

2017-10-24 Thread Mike Thomsen
Groovy is very close to being a superset of Java 7 in terms of syntax, so in most cases you can copy and paste Java code directly into a Groovy script without modification. On Tue, Oct 24, 2017 at 8:52 AM, Vyshali wrote: > Matt, > > Thanks for your valuable suggestion.

Re: Data anonymization in Nifi

2017-10-24 Thread Vyshali
Matt, Thanks for your valuable suggestion. ARX supports JAVA and only languages like Groovy,Python,Jython,Python are available in executescript processor.Have you tried using ARX functionalities in any of these languages ? If so, please send some references. Thanks, Vyshali -- Sent from:

Re: Data anonymization in Nifi

2017-10-23 Thread Matt Burgess
Vyshali, The AnonymizeRecord processor does not yet exist, I just wrote up a Jira to track the addition of it possibly sometime in the future. For the scripted solution, you can add the location of the ARX JARs to the Module Directory property of ExecuteScript. If it is a flat directory of JARs

Re: Data anonymization in Nifi

2017-10-23 Thread Vyshali
Hi Matt, Thanks for the suggestion. It would be very much helpful if you can give the instruction on how to use the AnonymizeRecord processor. Please give some clarity on how to setup processor after downloading ARX jars I downloaded the jar from http://arx.deidentifier.org/downloads/

Re: Data anonymization in Nifi

2017-10-22 Thread Chris Herssens
Hello Vyshali below you can find python code example for hashing the fourth column of a CSV file using the ExecuteScript processor If you hash a field using SHA256 then the length of the field is changed. A sha256 is 256 bits long import hashlib import java.io from org.apache.commons.io import

Re: Data anonymization in Nifi

2017-10-20 Thread Vyshali
Hi Chris, Thanks for the suggestion.Should I have code in python or some languagues for hashing the data using exectescript processor ? If so,will the format of the data be detained after hashing. Please provide some clarity on that. Thanks, Vyshali -- Sent from:

Re: Data anonymization in Nifi

2017-10-17 Thread Andy LoPresto
Vyshali, You may be interested in format preserving encryption (FPE) [1] if you need to maintain format while performing data masking. There are also methods to derive a cryptographically secure hash function from encryption [2] so that you can have “one way” data transformation and maintain a

Re: Data anonymization in Nifi

2017-10-17 Thread Mike Thomsen
Not if you use hashing. You'll get a field value like this (sha1 algorithm): c3499c2729730a7f807efb8676a92dcb6f8a3f8f For getting closer to the original data in the sort of values present, you'll need to try something like ARX. On Tue, Oct 17, 2017 at 11:53 AM, Vyshali

Re: Data anonymization in Nifi

2017-10-17 Thread Vyshali
Hi Chris, Hashing using executescript processor means that I should write some coding logic to do that.If so,will the format of the field will remain the same ? Please explain me with examples. Regards, Vyshali -- Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: Data anonymization in Nifi

2017-10-17 Thread Matt Burgess
processor for hashing some fields is for > instance CSV data > > Regards, > > Chris > > On Tue, Oct 17, 2017 at 8:41 AM, Vyshali <vyshal...@honeywell.com> wrote: > >> Hi, >> >> Please suggest possible ways to do data anonymization in Nifi such that P

Re: Data anonymization in Nifi

2017-10-17 Thread Chris Herssens
You can use the ExecuteScript processor for hashing some fields is for instance CSV data Regards, Chris On Tue, Oct 17, 2017 at 8:41 AM, Vyshali <vyshal...@honeywell.com> wrote: > Hi, > > Please suggest possible ways to do data anonymization in Nifi such that PII > d

Data anonymization in Nifi

2017-10-17 Thread Vyshali
Hi, Please suggest possible ways to do data anonymization in Nifi such that PII data is not exposed. Suggest suitable processors for the same. Thanks in advance. Regards, Vyshali -- Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/