How are you doing OpenID authentication to nifi-registry? We are using Keycloak OpenID auth to nifi itself, but couldn't configure it for nifi-registry. Everything I've read says its not currently possible.
https://community.cloudera.com/t5/Support-Questions/Nifi-Registry-openid/m-p/290935 https://issues.apache.org/jira/browse/NIFIREG-313 On Fri, Mar 20, 2020 at 12:09 PM Chris Sampson <[email protected]> wrote: > We've got a separate NiFi and NiFi Registry ingresses and OIDC for both > (different secure cookies used to differentiate the logins in the user's > browser). Users are able to login to both within the same browser session > without issue. > > OIDC AuthN in Registry is achieved using keycloak-gatekeeper in front of > the Registry instance with an nginx-proxy in between the two to add the > required X-ProxiedEntity header for Registry to be able to identify the > user and then apply AuthZ. > > Connection from NiFi to NiFi Registry doesn't go via the ingress, rather > they're direct between "internal" services within the k8s namespace. > AuthN/Z for the NiFi instances to Registry is via PKI certificates issued > (currently) by NiFi Toolkit (may look to use k8s cert-manager in future > instead). > > You're right that it's frustrating the two products don't use the same > AuthN/Z connectors, maybe there's a ticket on the Registry backlog for such > an update. The NiFi OIDC connector also doesn't use Refresh Token, which is > unfortunate as it means the lifetime of the ID/Account Token has to be > extended to prevent the user continually being logged out of the UI (which > is less secure than having regular refreshes) - think I've seen something > on the NiFi backlog to address that, but no work on it last time I checked. > > > *Chris Sampson* > IT Consultant > *Tel:* 07867 843 675 > [email protected] > > > > On Fri, 20 Mar 2020 at 15:33, Wyllys Ingersoll < > [email protected]> wrote: > >> >> Yup, I came up with a similar configuration for my NiFi cluster - fully >> secured using OIDC directly from NIFI. The issue I had with the >> nifi-registry was that it had to use different authentication (Kerberos vs >> OIDC) (aside: why aren't the 2 packages sharing the same authentication >> modules and configuration models??) but because they were both behind the >> same k8s ingress hostname, whenever one would authenticate (nifi vs >> nifi-registry) the browser would overwrite the authorization header so we >> could only be logged into one at a time. The solution was to add a DNS >> alias so we could still share the same k8s ingress but using different >> hostnames so that the browser would keep the authorization headers separate >> and we could stay authenticated to both within the same browser. >> >> Wyllys Ingersoll >> >> On Fri, Mar 20, 2020 at 10:15 AM Chris Sampson <[email protected]> >> wrote: >> >>> We've found a clustered k8s NiFi with OIDC auth works OK once: >>> >>> - Session Affinity annotations added to the nginx Ingress (change >>> cookie name appropriately): >>> >>> ingress.kubernetes.io/affinity: cookie >>>> ingress.kubernetes.io/session-cookie-name: "my-nifi" >>>> >>> >>> - Ingress Service (used by the Ingress to talk to NiFi) is >>> configured with: >>> >>> sessionAffinity: ClientIP >>>> >>> >>> We originally had a separate nginx reverse proxy (configured to the NiFi >>> instance root "/"), but removed that once we fully configured security in >>> the NiFi cluster instances directly and end-to-end through our >>> ingress/service chain (i.e. all https, OIDC done directly by NiFi). >>> >>> We do still have a reverse proxy in front of our NiFi Registry instance >>> though so it can do OIDC auth for us (as Registry doesn't have that >>> capability baked in) - this, again, is configured to go directly to the >>> Registry webroot "/". Bit different for Registry of course as it's not as >>> complex as NiFi and isn't clustered. >>> >>> >>> *Chris Sampson* >>> IT Consultant >>> *Tel:* 07867 843 675 >>> [email protected] >>> >>> >>> >>> On Fri, 20 Mar 2020 at 13:44, Wyllys Ingersoll < >>> [email protected]> wrote: >>> >>>> >>>> Yes, it would be easier to configure behind a single root context path, >>>> but since nifi uses hardcoded root paths in the responses, it makes things >>>> very challenging. I managed to get things working *mostly* with the >>>> existing paths and some regular expression based path mapping in the k8s >>>> ingress (i.e. nginx proxy), but I maintain that things would be MUCH easier >>>> if NiFi wasn't so dependent on hardcoded path names and could unify things >>>> behind a single root path. >>>> >>>> As I said before, if authentication is not involved, it's not a >>>> problem, but once authentication and tokens are involved, it becomes more >>>> challenging. Cookie and session affinity settings in the proxy are helpful >>>> but in the most common k8s cluster configuration, the cluster of NiFi nodes >>>> (i.e. stateful set of pods) is behind a single service name so the >>>> proxy/ingress in front of everything doesn't actually know which one it's >>>> talking to which make affinity harder to establish and maintain. >>>> >>>> >>>> On Fri, Mar 20, 2020 at 9:31 AM Matt Gilman <[email protected]> >>>> wrote: >>>> >>>>> Wyllys, >>>>> >>>>> The guidance for the proxy configuration is to map it behind a >>>>> single path. In order to work with any number of context paths that NiFi >>>>> supports the proxying needs to be based on the root path. Let me know if >>>>> I'm misunderstanding something. Unfortunately, I'm not super familiar with >>>>> nginx ingress in k8s environments but it sounds like that session affinity >>>>> may not working right. >>>>> >>>>> On Wed, Mar 18, 2020 at 5:37 PM Wyllys Ingersoll < >>>>> [email protected]> wrote: >>>>> >>>>>> Yeah, I've read all of that and I have a semi-working configuration. >>>>>> The problem is that when using OpenID tokens (oidc) in a clustered >>>>>> configuration, the node that requests the authentication is the only one >>>>>> that can validate it. If a user authenticates to node-1, but then later >>>>>> node-2 gets a request with the token (because its clustered and the user >>>>>> has no control over which node handles the request), node-2 cannot verify >>>>>> the token and rejects it. Even configuring sticky-sessions and cookie >>>>>> affinity in the nginx ingress proxy don't solve the problem as far as I >>>>>> can >>>>>> tell. >>>>>> >>>>>> I don't even know if having it all behind a single root path would >>>>>> make a difference for the authentication issue, it just makes setting up >>>>>> the reverse proxy configuration easier since you only need to worry >>>>>> about 1 >>>>>> path instead of multiple. >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Mar 18, 2020 at 2:46 PM Matt Gilman <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Wyllys, >>>>>>> >>>>>>> NiFi is comprised of any number of web applications. NiFi offers >>>>>>> extension points for custom processor configuration UIs and data type >>>>>>> viewers. These UIs can be bundled and discovered at runtime. These docs >>>>>>> [1] >>>>>>> detail the steps necessary for proxying a NiFi instance. >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> Matt >>>>>>> >>>>>>> [1] >>>>>>> https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#proxy_configuration >>>>>>> >>>>>>> On Wed, Mar 18, 2020 at 1:29 PM Wyllys Ingersoll < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> >>>>>>>> I'm surprised you haven't had lots of requests for this already. >>>>>>>> As it stands now, I cannot figure out how to configure a secure cluster >>>>>>>> behind a reverse proxy (for example, in a kubernetes environment >>>>>>>> behind an >>>>>>>> nginx ingress) that also incorporates OpenID authentication from an >>>>>>>> external service. I was thinking that if the NiFi nodes were able to >>>>>>>> operate under a single root path, it might make it easier to reverse >>>>>>>> proxy >>>>>>>> all of the different paths that Nifi uses (/nifi, /nifi-api, for >>>>>>>> example) >>>>>>>> behind a single ingress. I think having multiple ingress paths for the >>>>>>>> nifi service makes the reverse proxy configuration very complex when >>>>>>>> authentication tokens are involved. Without authentication, it works >>>>>>>> fine. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Wyllys Ingersoll >>>>>>>> >>>>>>>> On Wed, Mar 18, 2020 at 12:56 PM Andy LoPresto < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Wyllys, >>>>>>>>> >>>>>>>>> As I started reading, I was going to suggest the proxy approach. >>>>>>>>> Unfortunately, at this time I am unaware of any way to change the >>>>>>>>> paths >>>>>>>>> within NiFi itself - there would be substantial refactoring required >>>>>>>>> to >>>>>>>>> make that an option. You can open a feature request Jira for that, or >>>>>>>>> perhaps the ability to inject a path prefix, but I expect it to be a >>>>>>>>> high >>>>>>>>> level of effort to implement. >>>>>>>>> >>>>>>>>> >>>>>>>>> Andy LoPresto >>>>>>>>> [email protected] >>>>>>>>> *[email protected] <[email protected]>* >>>>>>>>> He/Him >>>>>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >>>>>>>>> >>>>>>>>> On Mar 18, 2020, at 9:25 AM, Wyllys Ingersoll < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> Is there a way to configure nifi to use a different root directory >>>>>>>>> for web requests? >>>>>>>>> >>>>>>>>> We would like everything to be under a common root such as: >>>>>>>>> /XXX/nifi/... >>>>>>>>> /XXX/nifi-api/... >>>>>>>>> >>>>>>>>> Having to proxy 2 (/nifi and /nifi-api) paths makes it very >>>>>>>>> difficult to setup a reverse proxy that also can incorporate OpenID >>>>>>>>> authentication tokens to a secure backend cluster of nodes. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>
