The reason for requiring the FS permissions on the HDFS processors is because you can provide a core-site.xml with file:/// as the default FS and then use ListHDFS/FetchHDFS/PutHDFS to essentially do the same thing as ListFile/FetchFile/PutFile.
A possible consideration would be to remove the requirement for the FS permissions, and then add validation logic that prevents HDFS processors from being used with a file:/// filesystem. On Fri, Aug 28, 2020 at 11:14 AM oliver twix <[email protected]> wrote: > Hello, thank you for your quick answers and sorry for my late reply. > > In the context of preparing platform security audit, I am trying to > identify possible threats over our NIFI clusters. > > @Mark, I didn't know about NIFI_ALLOW_EXPLICIT_KEYTAB parameter. Good to > know; it will limit a semi-friendly user to re-use easily a keytab from the > filesystem. > > My main concern now is the following : In my context, HDFS related > processors are required for our standard users. It means that any standard > user will require Filesystem access. It leads to the fact that if a > "standard" account is corrupted (common security audit use case) the whole > cluster can be easily corrupted gaining for instance admin privileges and > obviously keytab leaks and associated sensitive data leaks. > > Coming more on a design question, which constraint led to add a local FS > access permission to HDFS related processors? I would expect similar > requirements than for other equivalent distant storage processors (S3, > etc.). Is it linked to explicit keytab configuration, HDFS cluster > configuration files ? other deeper reason? Is there a hope at some point > that a change could get rid of this specific permission requirement? > > So far, if I understand well, giving HDFS related process access means > security threats on the cluster itself (reason why processors are > restricted). A workaround I could identify is to spawn on behalf of users > HDFS related processors but I am not sure it will be possible keeping a > good UX. > > > Many thanks for your help, > Olivier > > > > Le jeu. 30 juil. 2020 à 18:17, Joe Witt <[email protected]> a écrit : > >> ....in short this case has been really deeply considered and it is a very >> common usage pattern. We offer a set of policies/controls that let it be >> well restricted and locked down. But if a user is given too many accesses >> then yes they can be malicious. >> >> Thanks >> >> On Thu, Jul 30, 2020 at 9:13 AM Andy LoPresto <[email protected]> >> wrote: >> >>> If your concern is the malicious insider using FetchHDFS to read the >>> keytab as data from the filesystem, the *HDFS processors are marked as >>> Restricted and require an additional explicit permission to be granted for >>> users to configure them. At a file system interaction level, the NiFi Java >>> processes are running as a single OS user, so all of the keytabs will need >>> to be readable by that OS user, and the OS can’t detect which Java process >>> is acting as which application user. >>> >>> >>> Andy LoPresto >>> [email protected] >>> *[email protected] <[email protected]>* >>> He/Him >>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >>> >>> On Jul 30, 2020, at 8:10 AM, Mark Payne <[email protected]> wrote: >>> >>> Olivier, >>> >>> As Joe mentioned, it may help to further explain the exact scenario that >>> you are concerned about. >>> >>> But what I *think* you are concerned about is the following scenario: >>> - You have several different users developing flows in NiFi. >>> - You want the ability to give User A (and only User A) access to Keytab >>> A by creating a KeytabCredentialsService and giving User A READ Access to >>> it. >>> - You want the ability to give User B (and only User B) access to Keytab >>> B by creating a KeytabCredentialsService and giving User B READ Access to >>> it. >>> - If you do the above, but User A happens to know that Keytab B is >>> stored at /etc/keytabs/keytab-b, then all User A has to do is configure >>> PutHDFS’s Keytab property to “/etc/keytabs/keytab-b” instead of using the >>> KeytabCredentialsService. Then User A has access to User B’s keytab. >>> >>> Is that the scenario that you are concerned about? >>> >>> If yes, then the answer is to set the NIFI_ALLOW_EXPLICIT_KEYTAB >>> Environment Variable to a value of “false”. If you do that, then PutHDFS >>> and related processors that allow for either a KeytabCredentialsService or >>> an explicit keytab will become invalid (and therefore not runnable) if an >>> explicit keytab is used. This prevents User A from using Keytab A or Keytab >>> B directly and instead forces them to use no Keytab (which presumably will >>> result in authorization failure) or using the KeytabCrdentialsService, >>> which you can control READ access to. >>> >>> Does this help? >>> >>> Thanks >>> -Mark >>> >>> On Jul 30, 2020, at 10:09 AM, Joe Witt <[email protected]> wrote: >>> >>> Hello >>> >>> Can you more fully explain the scenario you have in mind and what an >>> intentionally malicious user might do? >>> >>> Thanks >>> >>> On Thu, Jul 30, 2020 at 6:54 AM oliver twix <[email protected]> >>> wrote: >>> >>>> Hello, >>>> Getting deeper on using nifi in multitenant use cases, I am facing a >>>> security question: our nifi users must be able to interact with hdfs not >>>> sharing their credentials (keytabs). >>>> >>>> From what understood, keytabCredentialsService enable a way to give a >>>> policy based control over keytabs access. >>>> Where I miss something is that for a user to use an hdfs processor, it >>>> requires read/write filesystem permissions. In this context, any hdfs user >>>> is able to read the keytabs of any other users. So in my understanding, it >>>> breaks the initial objective of keytabCredentialsService to control keytabs >>>> accesses. >>>> >>>> Am I missing something ? Do you have a mean to avoid giving access to >>>> all keytabs stored on local filesystem? >>>> >>>> Olivier >>>> >>>> >>>> >>> >>>
