Jon, Sorry to hear that securing NiFi is proving difficult for you.
The reason this feature was introduced is because prior to this check being enforced, a malicious Host header in requests could poison the server and return untrusted resources. Please see the Admin Guide [1] section on Proxy Configuration for more details. When not running behind a proxy, this feature is usually very simple to configure — the hostname of the NiFi service is automatically included in the whitelist, so requests are handled successfully. The original feature request (for proxy whitelisting) is in NIFI-4761 [2] (unit tests [3]) if you want more detail on why it was implemented, or see this article [4] for more details on host header attacks. If your dynamic hostnames fall in a list that is assigned on demand, you could provide the full list (comma-separated) as the whitelisted host value in all nodes, and this would allow each to be verified. If not, you could request a feature to allow regex-parsing of a pattern for that value as well, but I am opposed to this as it usually introduces more problems than it solves. I guess you could request an explicit disabling of this header check via nifi.properties, but I would certainly encourage you to find a solution that maintains the header check if possible. [1] https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#proxy_configuration <https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#proxy_configuration> [2] https://issues.apache.org/jira/browse/NIFI-4761 <https://issues.apache.org/jira/browse/NIFI-4761> [3] https://github.com/apache/nifi/pull/2279/files <https://github.com/apache/nifi/pull/2279/files> [4] https://dzone.com/articles/what-is-a-host-header-attack <https://dzone.com/articles/what-is-a-host-header-attack> Andy LoPresto [email protected] [email protected] PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Oct 15, 2018, at 11:06 PM, Peter Wilcsinszky <[email protected]> > wrote: > > Jon, > > as for the port-forward you can bind NiFi to lo(required by the > port-forward)+eth0 (I beleive eth0 is not provider dependent): > nifi.web.https.network.interface.lo=lo > nifi.web.https.network.interface.eth0=eth0 > > Peter > > On Mon, Oct 15, 2018 at 3:46 PM Jon Logan <[email protected] > <mailto:[email protected]>> wrote: > Thanks Peter. I briefly was playing around with connecting to localhost via > the port-forwarding late last week but was having issues where Nifi was > internally trying to connect to 0.0.0.0 and failing...I'll toy with it some > more this morning (I had set the https listen property as 0.0.0.0 so that > it'd bind to localhost). Or I think the NodePort option will work -- but will > require mucking with the yaml file every deployment to avoid port clashing > between deployments. Mildly inconvenient, but certainly is doable. > > If this feature were disabled, certificates would still work -- in this case, > we know our entrypoint, but do not know the port. We could have certs > dynamically issued including the entrypoint hostname into the cert as an > alternate name. Our issue hasn't been knowing the hostname in advance, but > knowing the port, which isn't part of the certificate entry. We haven't made > it to that point yet, but it should work, and is on our roadmap. > > Again, thanks a lot -- securing this is turning out to be a lot more > complicated than originally anticipated. > > > > > On Mon, Oct 15, 2018 at 3:57 AM Peter Wilcsinszky <[email protected] > <mailto:[email protected]>> wrote: > Hey, > > I can't tell about the original intent and motivations, but this is the Jira > that introduced this check [1]. > > What I can tell is mutual TLS is not the only option to authenticate against > NiFi. You can set up LDAP for example to authenticate the client and in that > case MITM is possible I beleive. > > You will need a stable network identity that you can use to configure as your > "proxy" in advance. For example in a testing scenario where you have access > to the kubernetes cluster you can simply use "localhost" as the name of the > proxy and use kubernetes port forward to tunnel requests from the localhost > to your individual nodes (only one node at a time). > > Another option that could better work for non-local use cases is to use a > LoadBalancer service in front of the nodes and configure DNS to point to your > LoadBalancer IP. If you want to do this in advance it is possible to create > floating IPs and preconfigure DNS for it at almost any cloud provider. Then > add the configured DNS to nifi.web.proxy.host property when starting your > cluster. If setting up DNS is not an option you can use the IP directly. If > setting up the IP in advance is not an option you may use an arbitrary > hostname as the proxy host and add that hostname to your hosts file (or > dnsmasq or company dns) to point to the dynamically generated LoadBalancer IP > after NiFi started up. > > Yet another option could be using Knox proxy that can act as an > authentication gateway to your NiFi cluster. Knox can use a fully independent > authentication scheme in front of the clients and at the same time > authenticate against NiFi using NiFi's supported mechanisms. To get a grasp > of how that works here is a detailed post on that matter [2]. > > Maybe you try to connect to you NiFi nodes through Kubernetes provided > NodePorts and want to access them directly. You can set up dedicated > Kubernetes services if your node list is static but then you have to > configure all the services with IP/DNS individually. Maybe you can leverage > Knox here as well to route your request to the individual nodes. > > As for disabling this feature I'm not sure how that would work. You would > still need to add the host as a SAN entry to your node certificates and that > is something that must be set in advance. Of course you can decide to avoid > TLS verification of the server, but then you cannot really trust the > connection anymore. (You may think wildcard cert could be used there, but it > is discouraged for several reasons) > > It would be good to know more about your general and most specifically about > your security requirements to know what would be the easiest way to setup > your cluster. > > [1] https://issues.apache.org/jira/browse/NIFI-4501 > <https://issues.apache.org/jira/browse/NIFI-4501> > [2] > https://risdenk.github.io/2018/03/18/apache-knox-proxying-apache-nifi.html > <https://risdenk.github.io/2018/03/18/apache-knox-proxying-apache-nifi.html> > > Cheers, > Peter > > On Sun, Oct 14, 2018 at 11:24 PM Jon Logan <[email protected] > <mailto:[email protected]>> wrote: > Thanks Jeff. We're not using Helm (I don't think at least!) and are using > kubectl directly with yaml files. We are trying to avoid binding to explicit > external ports so that we can deploy multiple times, and I am not sure how to > determine the arbitrary external port assigned to be able to do the property > substitution. Is there a way to do this or am I approaching this wrong? > > As an aside, I'm not really sure what this is accomplishing, as it seems like > the 2-way SSL is preventing any sort of MITM attack. Is there a way to just > turn off or bypass this check? > > On Sun, Oct 14, 2018 at 4:33 PM Jeff <[email protected] > <mailto:[email protected]>> wrote: > Jon, > > I'll start off by saying that I'm still new to k8s, and don't know the > specifics of most of this. You should be able to inject the host and port of > the proxy into the NiFi configuration before starting the NiFi instances, > with a Helm chart, for instance. I've done similar things with docker > compose. > > You are correct, though, that nifi.web.proxy.host and > nifi.web.proxy.context.path need to be set in nifi.properties before starting > NiFi. This is required for security purposes, and allows NiFi return > responses which are based on the proxy host and context path values so that > references to resources hosted by NiFi are made through the proxy, instead of > exposing the internal URI to a resource. > > - Jeff > > On Fri, Oct 12, 2018 at 12:42 PM Jon Logan <[email protected] > <mailto:[email protected]>> wrote: > We are running into issues with NiFi not allowing secure connections in a > container due to the proxy...the only documentation we've found on this > involves whitelisting specific proxy addresses. Is this the only solution? > Specifically, we're concerned about the fact that we don't know the proxy > address ahead of time to whitelist -- the port is an arbitrary assigned at > runtime port, and the proxy name could be any of the nodes of our Kubernetes > cluster. > > Are we missing something? > > > Thanks!
signature.asc
Description: Message signed with OpenPGP using GPGMail
