Keycloak Gatekeeper - https://github.com/keycloak/keycloak-gatekeeper
Then nginx-proxy to set the required X-ProxiedEntity header. *Chris Sampson* IT Consultant *Tel:* 07867 843 675 [email protected] On Fri, 20 Mar 2020 at 17:15, Wyllys Ingersoll < [email protected]> wrote: > > How are you doing OpenID authentication to nifi-registry? We are using > Keycloak OpenID auth to nifi itself, but couldn't configure it for > nifi-registry. Everything I've read says its not currently possible. > > > https://community.cloudera.com/t5/Support-Questions/Nifi-Registry-openid/m-p/290935 > https://issues.apache.org/jira/browse/NIFIREG-313 > > > > On Fri, Mar 20, 2020 at 12:09 PM Chris Sampson <[email protected]> > wrote: > >> We've got a separate NiFi and NiFi Registry ingresses and OIDC for both >> (different secure cookies used to differentiate the logins in the user's >> browser). Users are able to login to both within the same browser session >> without issue. >> >> OIDC AuthN in Registry is achieved using keycloak-gatekeeper in front of >> the Registry instance with an nginx-proxy in between the two to add the >> required X-ProxiedEntity header for Registry to be able to identify the >> user and then apply AuthZ. >> >> Connection from NiFi to NiFi Registry doesn't go via the ingress, rather >> they're direct between "internal" services within the k8s namespace. >> AuthN/Z for the NiFi instances to Registry is via PKI certificates issued >> (currently) by NiFi Toolkit (may look to use k8s cert-manager in future >> instead). >> >> You're right that it's frustrating the two products don't use the same >> AuthN/Z connectors, maybe there's a ticket on the Registry backlog for such >> an update. The NiFi OIDC connector also doesn't use Refresh Token, which is >> unfortunate as it means the lifetime of the ID/Account Token has to be >> extended to prevent the user continually being logged out of the UI (which >> is less secure than having regular refreshes) - think I've seen something >> on the NiFi backlog to address that, but no work on it last time I checked. >> >> >> *Chris Sampson* >> IT Consultant >> *Tel:* 07867 843 675 >> [email protected] >> >> >> >> On Fri, 20 Mar 2020 at 15:33, Wyllys Ingersoll < >> [email protected]> wrote: >> >>> >>> Yup, I came up with a similar configuration for my NiFi cluster - fully >>> secured using OIDC directly from NIFI. The issue I had with the >>> nifi-registry was that it had to use different authentication (Kerberos vs >>> OIDC) (aside: why aren't the 2 packages sharing the same authentication >>> modules and configuration models??) but because they were both behind the >>> same k8s ingress hostname, whenever one would authenticate (nifi vs >>> nifi-registry) the browser would overwrite the authorization header so we >>> could only be logged into one at a time. The solution was to add a DNS >>> alias so we could still share the same k8s ingress but using different >>> hostnames so that the browser would keep the authorization headers separate >>> and we could stay authenticated to both within the same browser. >>> >>> Wyllys Ingersoll >>> >>> On Fri, Mar 20, 2020 at 10:15 AM Chris Sampson < >>> [email protected]> wrote: >>> >>>> We've found a clustered k8s NiFi with OIDC auth works OK once: >>>> >>>> - Session Affinity annotations added to the nginx Ingress (change >>>> cookie name appropriately): >>>> >>>> ingress.kubernetes.io/affinity: cookie >>>>> ingress.kubernetes.io/session-cookie-name: "my-nifi" >>>>> >>>> >>>> - Ingress Service (used by the Ingress to talk to NiFi) is >>>> configured with: >>>> >>>> sessionAffinity: ClientIP >>>>> >>>> >>>> We originally had a separate nginx reverse proxy (configured to the >>>> NiFi instance root "/"), but removed that once we fully configured security >>>> in the NiFi cluster instances directly and end-to-end through our >>>> ingress/service chain (i.e. all https, OIDC done directly by NiFi). >>>> >>>> We do still have a reverse proxy in front of our NiFi Registry instance >>>> though so it can do OIDC auth for us (as Registry doesn't have that >>>> capability baked in) - this, again, is configured to go directly to the >>>> Registry webroot "/". Bit different for Registry of course as it's not as >>>> complex as NiFi and isn't clustered. >>>> >>>> >>>> *Chris Sampson* >>>> IT Consultant >>>> *Tel:* 07867 843 675 >>>> [email protected] >>>> >>>> >>>> >>>> On Fri, 20 Mar 2020 at 13:44, Wyllys Ingersoll < >>>> [email protected]> wrote: >>>> >>>>> >>>>> Yes, it would be easier to configure behind a single root context >>>>> path, but since nifi uses hardcoded root paths in the responses, it makes >>>>> things very challenging. I managed to get things working *mostly* with >>>>> the >>>>> existing paths and some regular expression based path mapping in the k8s >>>>> ingress (i.e. nginx proxy), but I maintain that things would be MUCH >>>>> easier >>>>> if NiFi wasn't so dependent on hardcoded path names and could unify things >>>>> behind a single root path. >>>>> >>>>> As I said before, if authentication is not involved, it's not a >>>>> problem, but once authentication and tokens are involved, it becomes more >>>>> challenging. Cookie and session affinity settings in the proxy are helpful >>>>> but in the most common k8s cluster configuration, the cluster of NiFi >>>>> nodes >>>>> (i.e. stateful set of pods) is behind a single service name so the >>>>> proxy/ingress in front of everything doesn't actually know which one it's >>>>> talking to which make affinity harder to establish and maintain. >>>>> >>>>> >>>>> On Fri, Mar 20, 2020 at 9:31 AM Matt Gilman <[email protected]> >>>>> wrote: >>>>> >>>>>> Wyllys, >>>>>> >>>>>> The guidance for the proxy configuration is to map it behind a >>>>>> single path. In order to work with any number of context paths that NiFi >>>>>> supports the proxying needs to be based on the root path. Let me know if >>>>>> I'm misunderstanding something. Unfortunately, I'm not super familiar >>>>>> with >>>>>> nginx ingress in k8s environments but it sounds like that session >>>>>> affinity >>>>>> may not working right. >>>>>> >>>>>> On Wed, Mar 18, 2020 at 5:37 PM Wyllys Ingersoll < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Yeah, I've read all of that and I have a semi-working >>>>>>> configuration. The problem is that when using OpenID tokens (oidc) in a >>>>>>> clustered configuration, the node that requests the authentication is >>>>>>> the >>>>>>> only one that can validate it. If a user authenticates to node-1, but >>>>>>> then >>>>>>> later node-2 gets a request with the token (because its clustered and >>>>>>> the >>>>>>> user has no control over which node handles the request), node-2 cannot >>>>>>> verify the token and rejects it. Even configuring sticky-sessions and >>>>>>> cookie affinity in the nginx ingress proxy don't solve the problem as >>>>>>> far >>>>>>> as I can tell. >>>>>>> >>>>>>> I don't even know if having it all behind a single root path would >>>>>>> make a difference for the authentication issue, it just makes setting up >>>>>>> the reverse proxy configuration easier since you only need to worry >>>>>>> about 1 >>>>>>> path instead of multiple. >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Wed, Mar 18, 2020 at 2:46 PM Matt Gilman <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Wyllys, >>>>>>>> >>>>>>>> NiFi is comprised of any number of web applications. NiFi offers >>>>>>>> extension points for custom processor configuration UIs and data type >>>>>>>> viewers. These UIs can be bundled and discovered at runtime. These >>>>>>>> docs [1] >>>>>>>> detail the steps necessary for proxying a NiFi instance. >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> Matt >>>>>>>> >>>>>>>> [1] >>>>>>>> https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#proxy_configuration >>>>>>>> >>>>>>>> On Wed, Mar 18, 2020 at 1:29 PM Wyllys Ingersoll < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> I'm surprised you haven't had lots of requests for this already. >>>>>>>>> As it stands now, I cannot figure out how to configure a secure >>>>>>>>> cluster >>>>>>>>> behind a reverse proxy (for example, in a kubernetes environment >>>>>>>>> behind an >>>>>>>>> nginx ingress) that also incorporates OpenID authentication from an >>>>>>>>> external service. I was thinking that if the NiFi nodes were able to >>>>>>>>> operate under a single root path, it might make it easier to reverse >>>>>>>>> proxy >>>>>>>>> all of the different paths that Nifi uses (/nifi, /nifi-api, for >>>>>>>>> example) >>>>>>>>> behind a single ingress. I think having multiple ingress paths for >>>>>>>>> the >>>>>>>>> nifi service makes the reverse proxy configuration very complex when >>>>>>>>> authentication tokens are involved. Without authentication, it works >>>>>>>>> fine. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Wyllys Ingersoll >>>>>>>>> >>>>>>>>> On Wed, Mar 18, 2020 at 12:56 PM Andy LoPresto < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi Wyllys, >>>>>>>>>> >>>>>>>>>> As I started reading, I was going to suggest the proxy approach. >>>>>>>>>> Unfortunately, at this time I am unaware of any way to change the >>>>>>>>>> paths >>>>>>>>>> within NiFi itself - there would be substantial refactoring required >>>>>>>>>> to >>>>>>>>>> make that an option. You can open a feature request Jira for that, or >>>>>>>>>> perhaps the ability to inject a path prefix, but I expect it to be a >>>>>>>>>> high >>>>>>>>>> level of effort to implement. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Andy LoPresto >>>>>>>>>> [email protected] >>>>>>>>>> *[email protected] <[email protected]>* >>>>>>>>>> He/Him >>>>>>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D >>>>>>>>>> EF69 >>>>>>>>>> >>>>>>>>>> On Mar 18, 2020, at 9:25 AM, Wyllys Ingersoll < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Is there a way to configure nifi to use a different root >>>>>>>>>> directory for web requests? >>>>>>>>>> >>>>>>>>>> We would like everything to be under a common root such as: >>>>>>>>>> /XXX/nifi/... >>>>>>>>>> /XXX/nifi-api/... >>>>>>>>>> >>>>>>>>>> Having to proxy 2 (/nifi and /nifi-api) paths makes it very >>>>>>>>>> difficult to setup a reverse proxy that also can incorporate OpenID >>>>>>>>>> authentication tokens to a secure backend cluster of nodes. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>
