Cool, thanks. I wasn't aware of that project. I'll check it out. On Fri, Mar 20, 2020 at 1:21 PM Chris Sampson <[email protected]> wrote:
> Keycloak Gatekeeper - https://github.com/keycloak/keycloak-gatekeeper > > Then nginx-proxy to set the required X-ProxiedEntity header. > > > *Chris Sampson* > IT Consultant > *Tel:* 07867 843 675 > [email protected] > > > > On Fri, 20 Mar 2020 at 17:15, Wyllys Ingersoll < > [email protected]> wrote: > >> >> How are you doing OpenID authentication to nifi-registry? We are using >> Keycloak OpenID auth to nifi itself, but couldn't configure it for >> nifi-registry. Everything I've read says its not currently possible. >> >> >> https://community.cloudera.com/t5/Support-Questions/Nifi-Registry-openid/m-p/290935 >> https://issues.apache.org/jira/browse/NIFIREG-313 >> >> >> >> On Fri, Mar 20, 2020 at 12:09 PM Chris Sampson <[email protected]> >> wrote: >> >>> We've got a separate NiFi and NiFi Registry ingresses and OIDC for both >>> (different secure cookies used to differentiate the logins in the user's >>> browser). Users are able to login to both within the same browser session >>> without issue. >>> >>> OIDC AuthN in Registry is achieved using keycloak-gatekeeper in front of >>> the Registry instance with an nginx-proxy in between the two to add the >>> required X-ProxiedEntity header for Registry to be able to identify the >>> user and then apply AuthZ. >>> >>> Connection from NiFi to NiFi Registry doesn't go via the ingress, rather >>> they're direct between "internal" services within the k8s namespace. >>> AuthN/Z for the NiFi instances to Registry is via PKI certificates issued >>> (currently) by NiFi Toolkit (may look to use k8s cert-manager in future >>> instead). >>> >>> You're right that it's frustrating the two products don't use the same >>> AuthN/Z connectors, maybe there's a ticket on the Registry backlog for such >>> an update. The NiFi OIDC connector also doesn't use Refresh Token, which is >>> unfortunate as it means the lifetime of the ID/Account Token has to be >>> extended to prevent the user continually being logged out of the UI (which >>> is less secure than having regular refreshes) - think I've seen something >>> on the NiFi backlog to address that, but no work on it last time I checked. >>> >>> >>> *Chris Sampson* >>> IT Consultant >>> *Tel:* 07867 843 675 >>> [email protected] >>> >>> >>> >>> On Fri, 20 Mar 2020 at 15:33, Wyllys Ingersoll < >>> [email protected]> wrote: >>> >>>> >>>> Yup, I came up with a similar configuration for my NiFi cluster - fully >>>> secured using OIDC directly from NIFI. The issue I had with the >>>> nifi-registry was that it had to use different authentication (Kerberos vs >>>> OIDC) (aside: why aren't the 2 packages sharing the same authentication >>>> modules and configuration models??) but because they were both behind the >>>> same k8s ingress hostname, whenever one would authenticate (nifi vs >>>> nifi-registry) the browser would overwrite the authorization header so we >>>> could only be logged into one at a time. The solution was to add a DNS >>>> alias so we could still share the same k8s ingress but using different >>>> hostnames so that the browser would keep the authorization headers separate >>>> and we could stay authenticated to both within the same browser. >>>> >>>> Wyllys Ingersoll >>>> >>>> On Fri, Mar 20, 2020 at 10:15 AM Chris Sampson < >>>> [email protected]> wrote: >>>> >>>>> We've found a clustered k8s NiFi with OIDC auth works OK once: >>>>> >>>>> - Session Affinity annotations added to the nginx Ingress (change >>>>> cookie name appropriately): >>>>> >>>>> ingress.kubernetes.io/affinity: cookie >>>>>> ingress.kubernetes.io/session-cookie-name: "my-nifi" >>>>>> >>>>> >>>>> - Ingress Service (used by the Ingress to talk to NiFi) is >>>>> configured with: >>>>> >>>>> sessionAffinity: ClientIP >>>>>> >>>>> >>>>> We originally had a separate nginx reverse proxy (configured to the >>>>> NiFi instance root "/"), but removed that once we fully configured >>>>> security >>>>> in the NiFi cluster instances directly and end-to-end through our >>>>> ingress/service chain (i.e. all https, OIDC done directly by NiFi). >>>>> >>>>> We do still have a reverse proxy in front of our NiFi Registry >>>>> instance though so it can do OIDC auth for us (as Registry doesn't have >>>>> that capability baked in) - this, again, is configured to go directly to >>>>> the Registry webroot "/". Bit different for Registry of course as it's not >>>>> as complex as NiFi and isn't clustered. >>>>> >>>>> >>>>> *Chris Sampson* >>>>> IT Consultant >>>>> *Tel:* 07867 843 675 >>>>> [email protected] >>>>> >>>>> >>>>> >>>>> On Fri, 20 Mar 2020 at 13:44, Wyllys Ingersoll < >>>>> [email protected]> wrote: >>>>> >>>>>> >>>>>> Yes, it would be easier to configure behind a single root context >>>>>> path, but since nifi uses hardcoded root paths in the responses, it makes >>>>>> things very challenging. I managed to get things working *mostly* with >>>>>> the >>>>>> existing paths and some regular expression based path mapping in the k8s >>>>>> ingress (i.e. nginx proxy), but I maintain that things would be MUCH >>>>>> easier >>>>>> if NiFi wasn't so dependent on hardcoded path names and could unify >>>>>> things >>>>>> behind a single root path. >>>>>> >>>>>> As I said before, if authentication is not involved, it's not a >>>>>> problem, but once authentication and tokens are involved, it becomes more >>>>>> challenging. Cookie and session affinity settings in the proxy are >>>>>> helpful >>>>>> but in the most common k8s cluster configuration, the cluster of NiFi >>>>>> nodes >>>>>> (i.e. stateful set of pods) is behind a single service name so the >>>>>> proxy/ingress in front of everything doesn't actually know which one it's >>>>>> talking to which make affinity harder to establish and maintain. >>>>>> >>>>>> >>>>>> On Fri, Mar 20, 2020 at 9:31 AM Matt Gilman <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Wyllys, >>>>>>> >>>>>>> The guidance for the proxy configuration is to map it behind a >>>>>>> single path. In order to work with any number of context paths that NiFi >>>>>>> supports the proxying needs to be based on the root path. Let me know if >>>>>>> I'm misunderstanding something. Unfortunately, I'm not super familiar >>>>>>> with >>>>>>> nginx ingress in k8s environments but it sounds like that session >>>>>>> affinity >>>>>>> may not working right. >>>>>>> >>>>>>> On Wed, Mar 18, 2020 at 5:37 PM Wyllys Ingersoll < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Yeah, I've read all of that and I have a semi-working >>>>>>>> configuration. The problem is that when using OpenID tokens (oidc) in >>>>>>>> a >>>>>>>> clustered configuration, the node that requests the authentication is >>>>>>>> the >>>>>>>> only one that can validate it. If a user authenticates to node-1, but >>>>>>>> then >>>>>>>> later node-2 gets a request with the token (because its clustered and >>>>>>>> the >>>>>>>> user has no control over which node handles the request), node-2 cannot >>>>>>>> verify the token and rejects it. Even configuring sticky-sessions and >>>>>>>> cookie affinity in the nginx ingress proxy don't solve the problem as >>>>>>>> far >>>>>>>> as I can tell. >>>>>>>> >>>>>>>> I don't even know if having it all behind a single root path would >>>>>>>> make a difference for the authentication issue, it just makes setting >>>>>>>> up >>>>>>>> the reverse proxy configuration easier since you only need to worry >>>>>>>> about 1 >>>>>>>> path instead of multiple. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Mar 18, 2020 at 2:46 PM Matt Gilman < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Wyllys, >>>>>>>>> >>>>>>>>> NiFi is comprised of any number of web applications. NiFi offers >>>>>>>>> extension points for custom processor configuration UIs and data type >>>>>>>>> viewers. These UIs can be bundled and discovered at runtime. These >>>>>>>>> docs [1] >>>>>>>>> detail the steps necessary for proxying a NiFi instance. >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> >>>>>>>>> Matt >>>>>>>>> >>>>>>>>> [1] >>>>>>>>> https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#proxy_configuration >>>>>>>>> >>>>>>>>> On Wed, Mar 18, 2020 at 1:29 PM Wyllys Ingersoll < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> I'm surprised you haven't had lots of requests for this already. >>>>>>>>>> As it stands now, I cannot figure out how to configure a secure >>>>>>>>>> cluster >>>>>>>>>> behind a reverse proxy (for example, in a kubernetes environment >>>>>>>>>> behind an >>>>>>>>>> nginx ingress) that also incorporates OpenID authentication from an >>>>>>>>>> external service. I was thinking that if the NiFi nodes were able to >>>>>>>>>> operate under a single root path, it might make it easier to reverse >>>>>>>>>> proxy >>>>>>>>>> all of the different paths that Nifi uses (/nifi, /nifi-api, for >>>>>>>>>> example) >>>>>>>>>> behind a single ingress. I think having multiple ingress paths for >>>>>>>>>> the >>>>>>>>>> nifi service makes the reverse proxy configuration very complex when >>>>>>>>>> authentication tokens are involved. Without authentication, it >>>>>>>>>> works fine. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Wyllys Ingersoll >>>>>>>>>> >>>>>>>>>> On Wed, Mar 18, 2020 at 12:56 PM Andy LoPresto < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Wyllys, >>>>>>>>>>> >>>>>>>>>>> As I started reading, I was going to suggest the proxy approach. >>>>>>>>>>> Unfortunately, at this time I am unaware of any way to change the >>>>>>>>>>> paths >>>>>>>>>>> within NiFi itself - there would be substantial refactoring >>>>>>>>>>> required to >>>>>>>>>>> make that an option. You can open a feature request Jira for that, >>>>>>>>>>> or >>>>>>>>>>> perhaps the ability to inject a path prefix, but I expect it to be >>>>>>>>>>> a high >>>>>>>>>>> level of effort to implement. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Andy LoPresto >>>>>>>>>>> [email protected] >>>>>>>>>>> *[email protected] <[email protected]>* >>>>>>>>>>> He/Him >>>>>>>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D >>>>>>>>>>> EF69 >>>>>>>>>>> >>>>>>>>>>> On Mar 18, 2020, at 9:25 AM, Wyllys Ingersoll < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Is there a way to configure nifi to use a different root >>>>>>>>>>> directory for web requests? >>>>>>>>>>> >>>>>>>>>>> We would like everything to be under a common root such as: >>>>>>>>>>> /XXX/nifi/... >>>>>>>>>>> /XXX/nifi-api/... >>>>>>>>>>> >>>>>>>>>>> Having to proxy 2 (/nifi and /nifi-api) paths makes it very >>>>>>>>>>> difficult to setup a reverse proxy that also can incorporate OpenID >>>>>>>>>>> authentication tokens to a secure backend cluster of nodes. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>
