You can use the two permissions detailed here https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.2/org.apache.nifi.processors.standard.GetFile/index.html
On Fri, 4 Jun 2021, 21:11 Ruth, Thomas, <[email protected]> wrote: > No matter how secure I make the cluster, what’s to prevent a user from: > > - Using GetFile to copy the NiFi configuration files and keystores > into flows. > - Creating flows which contain python scripts that utilize the > keys/secrets in the nifi configuration files to decrypt the contents of the > keystores. > - Executing those flows to gain access to the private keys used to > encrypt data in NiFi > - Using that data to gain unauthorized access to the cluster to > further compromise it, possibly by generating their own admin-level user > certificates. > > > > I’m genuinely stumped here as to how to create a solution that is > production ready and that prevents a complete system failure by a user > accidentally supplying the wrong directory to a processor component or > generally creating python scripts that execute as the nifi user, being able > to basically use a NiFi server as their own script-running playground. > > > > Tom > > > > *From:* Mike Sofen <[email protected]> > *Sent:* Friday, June 4, 2021 2:02 PM > *To:* [email protected] > *Subject:* RE: Problem with the GetFile processor deleting my entire > installation > > > > Setting aside the user auth/permissions issue (which sounds like, from > your initial email, that you nailed perfectly with a secured cluster), I’ll > address this accidental deletion issue using what I consider a general > design pattern for Nifi (or any process that write-touches files): > > - I don’t allow physical file deletes (which happen with the default > GetFiles settings) > - I create an archive folder for files I’ve processed if I want/need > to move them out of the processing folder > - I always set the “leave files” flag > - I use either the timestamp or filename tracker to eliminate repeated > processing if I’m not moving the files > - OR – I use Move File flag to the archive folder destination. > - And I advise new users to always spec a small test folder to start, > and to double-check that the remove files is turned off. > > > > In some cases, like when I’m processing text files in place in secure > repo, I can’t move the files so I use the timestamp eval method. > > > > In others, like log files where I definitely want to archive them out of > that processing folder, I use the move model. > > > > Don’t get discouraged about this twist in the road – you’ll find Nifi to > be a truly exceptional product for pipeline automation, wildly more > powerful/flexible/stable than any other product out there (I’ve used > most). I’ve used it IoT to DB to ML processing, document processing, > metrology data processing, you name it. In particular, the built data > provenance and auditability will be valuable for your situation at Optum. > All the best, > > > > Mike > > > > *From:* Ruth, Thomas <[email protected]> > *Sent:* Friday, June 04, 2021 12:26 PM > *To:* [email protected] > *Subject:* RE: Problem with the GetFile processor deleting my entire > installation > > > > Is it recommended that after installing NiFi, that I then proceed to > remove read permissions from all installation files and directories in > order to protect them from removal by users? Will this present a problem > running the software if it’s unable to read any files? > > > > So for fun, I loaded up a quick nifi container: > docker run –name nifi -p 8080:8080 -d apache/nifi:latest > > > > Connected to localhost:8080/nifi > > > > Created a GetFile processor, with the directory /opt/nifi. Connected the > success to a LogAttributes. Hit Run…. > > > > Now I get this: > *HTTP ERROR 404 /nifi/canvas.jsp* > > *URI:* > > /nifi/ > > *STATUS:* > > 404 > > *MESSAGE:* > > /nifi/canvas.jsp > > *SERVLET:* > > jsp > > > > This behavior shouldn’t be possible, in my opinion. It’s putting security > in the hands of my developers. I really am looking for a solution to this > issue. > > > > The help text says this: > Creates FlowFiles from files in a directory. NiFi will ignore files it > doesn't have at least read permissions for. > > > > So as you suggested, I removed the read permission recursively from all > files in /opt/nifi. After doing this, nifi no longer starts. > > > > find /opt/nifi -print > /tmp/files.txt > > for I in `cat /tmp/files.txt`; do chmod a-r $i; done > > > > I also went to https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator and > attempted to calculate a CVSS score for this vulnerability. I ended up > calculating a score of 8.0. > > > > Tom > > > > *From:* Russell Bateman <[email protected]> > *Sent:* Friday, June 4, 2021 11:20 AM > *To:* [email protected] > *Subject:* Re: Problem with the GetFile processor deleting my entire > installation > > > > Oh, sorry, to finish the answer, yes, you do need to be *very* careful > how you specify the Input Directory and File Filter properties; the last > one is a regular expression. It's true that the documentation is less than > flag-waving or hair-lighting-on-fire as it presents its help on filling > those in. > > Russ > > On 6/4/21 11:16 AM, Russell Bateman wrote: > > Sorry for this behavior of *GetFile* which is purposeful. If you > configure to keep the files instead of removing them, you'll keep getting > the same files ingested over and over again as flow files. It's just how it > is. > > The secret was to read the help blurb when configuring this processor. > > Hope this helps, > > Russ > > On 6/4/21 10:44 AM, Ruth, Thomas wrote: > > Hello, > > > > I recently built a 3-node NiFi cluster in my organization as a > proof-of-concept for some work we are doing. I used version 1.13.2 and > installed it onto 3 CentOS 7.9 systems. In my organization, I don’t have > root access to the system, so I used a different user called “nfadm” to > install and run the product. I don’t remember seeing anything in the > documentation that stated that this would be an issue. > > > > I am also new to NiFi, and was relying heavily on the Admin documentation > on the web site for instructions to set up the OS and NiFi installations. I > configured certificate-based security and distributed them to my users. I > also configured policies for groups that I thought were OK for them from a > development standpoint. > > > > I had an incident occur yesterday in which a user, who is also new to > NiFi, ran a component called “GetFile” for the filesystem “/” with the > default settings (Recurse=true, KeepSourceFile=false). Well, this > essentially ran “rm -rf /” as the user that owns all the installation files > and files in the various repositories, nfadm, the same user running the > NiFi processes. This deleted all the installation and configuration files > for the entire cluster, making it completely useless now. > > > > I am surprised to find out that NiFi allowed a user to basically wipe out > all the files the user running the NiFi server had access to. I would > expect much higher security to be present in a default system. I have some > questions that hopefully you can help me with: > > > > Is this a known issue in NiFi? > > > > Am I doing something wrong when configuring or installing NiFi? > > > > Is there a section in the documentation that warns me of this type of > scenario? > > > > Thanks in advance for your help with this, > > > > Tom Ruth > > Sr. Data Engineer > > Optum, Inc. > > E: [email protected] > > > > > This e-mail, including attachments, may include confidential and/or > proprietary information, and may be used only by the person or entity > to which it is addressed. If the reader of this e-mail is not the intended > recipient or his or her authorized agent, the reader is hereby notified > that any dissemination, distribution or copying of this e-mail is > prohibited. If you have received this e-mail in error, please notify the > sender by replying to this message and delete this e-mail immediately. > > > > > > > This e-mail, including attachments, may include confidential and/or > proprietary information, and may be used only by the person or entity > to which it is addressed. If the reader of this e-mail is not the intended > recipient or his or her authorized agent, the reader is hereby notified > that any dissemination, distribution or copying of this e-mail is > prohibited. If you have received this e-mail in error, please notify the > sender by replying to this message and delete this e-mail immediately. > > > This e-mail, including attachments, may include confidential and/or > proprietary information, and may be used only by the person or entity > to which it is addressed. If the reader of this e-mail is not the intended > recipient or his or her authorized agent, the reader is hereby notified > that any dissemination, distribution or copying of this e-mail is > prohibited. If you have received this e-mail in error, please notify the > sender by replying to this message and delete this e-mail immediately. >
