Andy,
I and several of my co-workers are very interested in seeing this feature added to NIFI. I have been trying to write an EncryptedFileSystemRepository myself but unfortunately did not get very far before running into issues. So far I just tried a simple solution: I essentially extended FileSystemRepository, and overrode the "read" and "write" methods to wrap them with java's CipherInputStream or CipherOutputStream. I also added ways to pass in a secret key. If that had worked, I would have updated it to use AWS's KMS service to provide secret keys. I do think we would need to roll keys periodically, but still haven't figured out how that would work. I ran into a lot of classpath issues. First I tried making a thin jar and putting it on NIFI's lib, but unfortunately NIFI's classloader ensured that my jar did not see any of nifi's framework classes. I tried making an uber jar but encountered a lot of other classpath issues. I tried making a nar, but NIFI still could not find my EncryptedRepository when spring was wiring up the application. It seems you can only extend repositories from within the source code. So I tried putting my code into the source and re-building. Unfortunately my work's proxy prevented me from getting some essential apache artifacts so I am unable to build NIFI from source. I might try this again from a personal account. So it seems that with NIFI it is not easy to implement things that are not processors or controller services, since you have to add that to the source and re-build it. Perhaps NIFI needs an easier way to add these implementations without re-building from source. Since I don't want to put the entire AWS core artifact and its transitive dependencies inside NIFI, I was thinking of using java's SPI framework to provide encryption services to the EncryptedRepository. I'm not sure I really answered any of your questions, but hopefully I gave you an idea of how we might use this. I might be available to help develop and/or test the solution. Michael Knapp Capital One ________________________________ From: Andy LoPresto <[email protected]> Sent: Tuesday, January 24, 2017 12:13:07 AM To: [email protected] Subject: [DISCUSS] Encrypted repositories (content, flowfile, provenance) I am working on building drop-in encrypted repositories for NiFi [1]. I am currently in the planning stages and have written up a fairly extensive Jira ticket documenting my plans (high-level) and some concerns (much bigger section). I would welcome community feedback in order to capture expectations, concerns (for security, performance, and usability), and any other valuable contributions. All of us are smarter than one of us, and I am not that smart. Thanks. [1] https://issues.apache.org/jira/browse/NIFI-3388 Andy LoPresto [email protected]<mailto:[email protected]> [email protected]<mailto:[email protected]> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 ________________________________________________________ The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.
