That means the user representing host-1 does not have permissions to proxy.
You can look in authorizations.xml on nifi-1 for a policy like:
<policy identifier="287edf48-da72-359b-8f61-da5d4c45a270"
resource="/proxy" action="W">
<user identifier="c22273fa-7ed3-38a9-8994-3ed5fea5d234"/>
</policy>
That user identifier should point to a user in users.xml like:
<user identifier="c22273fa-7ed3-38a9-8994-3ed5fea5d234"
identity="CN=<host-1, redacted>, OU=Devices, OU=NIH, OU=HHS, O=U.S.
Government, C=US"/>
All of the user identities are case sensitive and white space
sensitive so make sure whatever is in users.xml is exactly what is
shown in the logs.
On Tue, Oct 23, 2018 at 12:28 PM Saip, Alexander (NIH/CC/BTRIS) [C]
<alexander.s...@nih.gov> wrote:
>
> Hi Bryan,
>
>
>
> Yes, converting two standalone NiFi instances into a cluster is exactly what
> we are trying to do. Here are the steps I went through in this round:
>
> · restored the original configuration files (nifi.properties,
> users.xml, authorizers.xml and authorizations.xml)
>
> · restarted one instance in the standalone mode
>
> · added two new node users in the NiFi web UI (CN=<host-1, redacted>,
> OU=Devices, OU=NIH, OU=HHS, O=U.S. Government, C=US and CN=<host-2,
> redacted>, OU=Devices, OU=NIH, OU=HHS, O=U.S. Government, C=US)
>
> · granted them the “proxy user requests” privileges
>
> · edited the nifi.properties file
> (nifi.state.management.embedded.zookeeper.start=true,
> nifi.cluster.is.node=true, nifi.zookeeper.connect.string=<host-1,
> redacted>:2181)
>
> · restarted the node on host-1
>
>
>
> On logging in, I see the cluster section of the dashboard showing 1/1 as
> expected, although I’m unable to do anything there due to errors like this:
>
>
>
> Insufficient Permissions
>
> Node <host-1, redacted>:8008 is unable to fulfill this request due to:
> Untrusted proxy CN=<host-1, redacted>, OU=Devices, OU=NIH, OU=HHS, O=U.S.
> Government, C=US Contact the system administrator.
>
>
>
> The nifi-user.log also contains
>
>
>
> 2018-10-23 12:17:01,916 WARN [NiFi Web Server-224]
> o.a.n.w.s.NiFiAuthenticationFilter Rejecting access to web api: Untrusted
> proxy CN=<host-1, redacted>, OU=Devices, OU=NIH, OU=HHS, O=U.S. Government,
> C=US
>
>
>
> From your experience, what the most likely causes for this exception?
>
>
>
> Thank you,
>
>
>
> Alexander
>
>
>
> -----Original Message-----
> From: Bryan Bende <bbe...@gmail.com>
> Sent: Monday, October 22, 2018 1:25 PM
> To: users@nifi.apache.org
> Subject: Re: NiFi fails on cluster nodes
>
>
>
> Yes, to further clarify what I meant...
>
>
>
> If you are trying to change the Initial Admin or Node Identities in
> authorizers.xml, these will only be used when there are no other
> users/group/policies present. People frequently make a mistake during initial
> config and then try to edit authorizers.xml and try again, but it won't
> actually do anything unless you remove the users.xml and authorizations.xml
> to start over.
>
>
>
> In your case it sounds like you are trying to convert and existing standalone
> node to a cluster, given that I would do the following...
>
>
>
> - In standalone mode, use the UI to add users for the DN's of the server
> certificates (CN=nifi-node-1, OU=NIFI, CN=nifi-node-2, OU=NIFI)
>
> - In the UI, grant those users Write access to "Proxy"
>
> - Convert to a cluster and keep your same authorizers.xml, users.xml, and
> authorizations.xml when you setup your cluster, this way all your users and
> policies are already setup and the Initial Admin and Node Identities are not
> needed
>
>
>
>
>
> On Mon, Oct 22, 2018 at 1:06 PM Saip, Alexander (NIH/CC/BTRIS) [C]
> <alexander.s...@nih.gov> wrote:
>
> >
>
> > Thanks again, Bryan. Just a quick follow-up question: does removing
> > users.xml and authorizations.xml mean that we will need to re-create all
> > users and groups that we had in the original standalone NiFi instance?
>
> >
>
> > -----Original Message-----
>
> > From: Bryan Bende <bbe...@gmail.com>
>
> > Sent: Monday, October 22, 2018 12:48 PM
>
> > To: users@nifi.apache.org
>
> > Subject: Re: NiFi fails on cluster nodes
>
> >
>
> > Sorry I was confused when you said two 1 node clusters and I assumed they
> > each had their own ZooKeeper.
>
> >
>
> > You don't need to run ZK on both nodes, you can create a 2 node cluster
> > using the embedded ZK on the first node.
>
> >
>
> > This blog post shows how to setup a secure 2 node cluster:
>
> >
>
> > https://bryanbende.com/development/2016/08/17/apache-nifi-1-0-0-author
>
> > ization-and-multi-tenancy
>
> >
>
> > The only difference is that the authorizers.xml has changed slightly, so
> > instead of:
>
> >
>
> > <authorizer>
>
> > <identifier>file-provider</identifier>
>
> > <class>org.apache.nifi.authorization.FileAuthorizer</class>
>
> > <property name="Authorizations
> > File">./conf/authorizations.xml</property>
>
> > <property name="Users File">./conf/users.xml</property>
>
> > <property name="Initial Admin Identity">CN=bbende,
> > OU=ApacheNiFi</property>
>
> > <property name="Legacy Authorized Users File"></property>
>
> > <property name="Node Identity 1">CN=localhost, OU=NIFI</property>
>
> > </authorizer>
>
> >
>
> > You need to add the the users to the user-group-provider and then to the
> > access-policy-provider...
>
> >
>
> > <userGroupProvider>
>
> > <identifier>file-user-group-provider</identifier>
>
> > <class>org.apache.nifi.authorization.FileUserGroupProvider</class>
>
> > <property name="Users File">./conf/users.xml</property>
>
> > <property name="Legacy Authorized Users File"></property>
>
> > <property name="Initial User Identity 1">CN=bbende, OU=Apache
> > NiFI</property>
>
> > <property name="Initial User Identity 2">CN=nifi-host-1,
> > OU=NIFI</property>
>
> > <property name="Initial User Identity 2">CN=nifi-host-2,
> > OU=NIFI</property>
>
> > </userGroupProvider>
>
> >
>
> > <accessPolicyProvider>
>
> > <identifier>file-access-policy-provider</identifier>
>
> >
> > <class>org.apache.nifi.authorization.FileAccessPolicyProvider</class>
>
> > <property name="User Group
>
> > Provider">composite-configurable-user-group-provider</property>
>
> > <property name="Authorizations
>
> > File">./conf/authorizations.xml</property>
>
> > <property name="Initial Admin Identity">CN=bbende, OU=Apache
> > NiFI</property>
>
> > <property name="Legacy Authorized Users File"></property>
>
> > <property name="Node Identity 1">CN=nifi-host-1, OU=NIFI</property>
>
> > <property name="Node Identity 1">CN=nifi-host-2, OU=NIFI</property>
>
> > </accessPolicyProvider>
>
> >
>
> >
>
> > Also, whenever you change any config in the authorizers.xml related to the
> > file-based providers, then you will need to remove users.xml and
> > authorizations.xml On Mon, Oct 22, 2018 at 12:20 PM Saip, Alexander
> > (NIH/CC/BTRIS) [C] <alexander.s...@nih.gov> wrote:
>
> > >
>
> > > Hi Bryan,
>
> > >
>
> > >
>
> > >
>
> > > At this point, we don't want to run ZooKeeper on both nodes (as far as I
> > > understand, it prefers an odd number of members in the ensemble).
> > > Actually, the ZooKeeper running on one of them, sees both NiFi instances,
> > > but they don't talk to each other. When we try to make them do so by
> > > using a different authorizers.xml file, which is very much just a
> > > customized version of the “composite” example from the NiFi Admin Guide,
> > > then none of the nodes is able to start at all, throwing the error I
> > > mentioned in my previous post.
>
> > >
>
> > >
>
> > >
>
> > > Are you saying that we have to run ZooKeeper on both nodes? BTW, do
>
> > > we still need
>
> > >
>
> > >
>
> > >
>
> > > nifi.login.identity.provider.configuration.file=./conf/login-identit
>
> > > y-
>
> > > providers.xml
>
> > >
>
> > >
>
> > >
>
> > > in the nifi.properties file when we use that new authorizers.xml? I’m
> > > asking since we have the same LDAP authentication/authorization settings
> > > in the latter.
>
> > >
>
> > >
>
> > >
>
> > > Thank you,
>
> > >
>
> > >
>
> > >
>
> > > Alexander
>
> > >
>
> > >
>
> > >
>
> > > -----Original Message-----
>
> > > From: Bryan Bende <bbe...@gmail.com>
>
> > > Sent: Monday, October 22, 2018 11:55 AM
>
> > > To: users@nifi.apache.org
>
> > > Subject: Re: NiFi fails on cluster nodes
>
> > >
>
> > >
>
> > >
>
> > > If you are getting separate clusters then each node is likely only using
> > > it's own ZooKeeper and therefore doesn't know about the other node.
>
> > >
>
> > >
>
> > >
>
> > > In nifi.properties the ZK connect string would need to be something like
> > > nifi-node1-hostname:2181,nifi-node2-hostname:2181 and in zoo.properties
> > > you would need entries for both ZooKeepers:
>
> > >
>
> > >
>
> > >
>
> > > server.1=nifi-node1-hostname:2888:3888
>
> > >
>
> > > server.2=nifi-node2-hostname:2888:3888
>
> > >
>
> > > On Mon, Oct 22, 2018 at 11:28 AM Saip, Alexander (NIH/CC/BTRIS) [C]
> > > <alexander.s...@nih.gov> wrote:
>
> > >
>
> > > >
>
> > >
>
> > > > I wonder if anyone has run into the same problem when trying to
>
> > >
>
> > > > configure composite authentication/authorization (LDAP and local
>
> > >
>
> > > > file)? When we use the “stand-alone” authorizers.xml file with the
>
> > >
>
> > > > addition of two extra properties
>
> > >
>
> > > >
>
> > >
>
> > > >
>
> > >
>
> > > >
>
> > >
>
> > > > <property name="Node Identity 1">…
>
> > >
>
> > > >
>
> > >
>
> > > > <property name="Node Identity 2">…
>
> > >
>
> > > >
>
> > >
>
> > > >
>
> > >
>
> > > >
>
> > >
>
> > > > and let ZooKeeper start on one on the nodes, we end up with two
>
> > >
>
> > > > one-node clusters, since apparently, the NiFi instances don’t talk
>
> > > > to
>
> > >
>
> > > > each other, but at least, they come alive…
>
> > >
>
> > > >
>
> > >
>
> > > >
>
> > >
>
> > > >
>
> > >
>
> > > > From: Saip, Alexander (NIH/CC/BTRIS) [C] <alexander.s...@nih.gov>
>
> > >
>
> > > > Sent: Friday, October 19, 2018 11:18 AM
>
> > >
>
> > > > To: users@nifi.apache.org
>
> > >
>
> > > > Subject: RE: NiFi fails on cluster nodes
>
> > >
>
> > > >
>
> > >
>
> > > >
>
> > >
>
> > > >
>
> > >
>
> > > > We have managed to get past that error by installing the CA cert in the
> > > > truststore. So, we can get a one-node cluster up and running. In order
> > > > to add another node, I edited the authorizers.xml file, basically,
> > > > using the “example composite implementation loading users and groups
> > > > from LDAP and a local file” from the Admin guide as a template. When I
> > > > re-started the node running ZooKeeper, though, it crashed with the
> > > > following error written into the nifi-app.log file:
>
> > >
>
> > > >
>
> > >
>
> > > >
>
> > >
>
> > > >
>
> > >
>
> > > > 2018-10-19 08:09:26,992 ERROR [main] o.s.web.context.ContextLoader
>
> > >
>
> > > > Context initialization failed
>
> > >
>
> > > >
>
> > >
>
> > > > org.springframework.beans.factory.UnsatisfiedDependencyException:
>
> > >
>
> > > > Error creating bean with name
>
> > >
>
> > > > 'org.springframework.security.config.annotation.web.configuration.
>
> > > > We
>
> > > > bS
>
> > >
>
> > > > ecurityConfiguration': Unsatisfied dependency expressed through
>
> > > > method
>
> > >
>
> > > > 'setFilterChainProxySecurityConfigurer' parameter 1; nested
>
> > > > exception
>
> > >
>
> > > > is org.springframework.beans.factory.BeanExpressionException:
>
> > >
>
> > > > Expression parsing failed; nested exception is
>
> > >
>
> > > > org.springframework.beans.factory.UnsatisfiedDependencyException:
>
> > >
>
> > > > Error creating bean with name
>
> > >
>
> > > > 'org.apache.nifi.web.NiFiWebApiSecurityConfiguration': Unsatisfied
>
> > >
>
> > > > dependency expressed through method 'setJwtAuthenticationProvider'
>
> > >
>
> > > > parameter 0; nested exception is
>
> > >
>
> > > > org.springframework.beans.factory.BeanCreationException: Error
>
> > >
>
> > > > creating bean with name 'jwtAuthenticationProvider' defined in
>
> > > > class
>
> > >
>
> > > > path resource [nifi-web-security-context.xml]: Cannot resolve
>
> > >
>
> > > > reference to bean 'authorizer' while setting constructor argument;
>
> > >
>
> > > > nested exception is
>
> > >
>
> > > > org.springframework.beans.factory.BeanCreationException: Error
>
> > >
>
> > > > creating bean with name 'authorizer': FactoryBean threw exception
>
> > > > on
>
> > >
>
> > > > object creation; nested exception is java.lang.NullPointerException:
>
> > >
>
> > > > Name is null
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.annotation.AutowiredAnnotationBe
>
> > > > an
>
> > > > Po
>
> > >
>
> > > > stProcessor$AutowiredMethodElement.inject(AutowiredAnnotationBeanP
>
> > > > os
>
> > > > tP
>
> > >
>
> > > > rocessor.java:667)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.annotation.InjectionMetadata.inj
>
> > > > ec
>
> > > > t(
>
> > >
>
> > > > InjectionMetadata.java:88)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.annotation.AutowiredAnnotationBe
>
> > > > an
>
> > > > Po
>
> > >
>
> > > > stProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostP
>
> > > > ro
>
> > > > ce
>
> > >
>
> > > > ssor.java:366)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.support.AbstractAutowireCapableB
>
> > > > ea
>
> > > > nF
>
> > >
>
> > > > actory.populateBean(AbstractAutowireCapableBeanFactory.java:1264)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.support.AbstractAutowireCapableB
>
> > > > ea
>
> > > > nF
>
> > >
>
> > > > actory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.support.AbstractAutowireCapableB
>
> > > > ea
>
> > > > nF
>
> > >
>
> > > > actory.createBean(AbstractAutowireCapableBeanFactory.java:483)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.support.AbstractBeanFactory$1.ge
>
> > > > tO
>
> > > > bj
>
> > >
>
> > > > ect(AbstractBeanFactory.java:306)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.support.DefaultSingletonBeanRegi
>
> > > > st
>
> > > > ry
>
> > >
>
> > > > .getSingleton(DefaultSingletonBeanRegistry.java:230)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.support.AbstractBeanFactory.doGe
>
> > > > tB
>
> > > > ea
>
> > >
>
> > > > n(AbstractBeanFactory.java:302)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.support.AbstractBeanFactory.getB
>
> > > > ea
>
> > > > n(
>
> > >
>
> > > > AbstractBeanFactory.java:197)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.support.DefaultListableBeanFacto
>
> > > > ry
>
> > > > .p
>
> > >
>
> > > > reInstantiateSingletons(DefaultListableBeanFactory.java:761)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.context.support.AbstractApplicationContext.fin
>
> > > > is
>
> > > > hB
>
> > >
>
> > > > eanFactoryInitialization(AbstractApplicationContext.java:867)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.context.support.AbstractApplicationContext.ref
>
> > > > re
>
> > > > sh
>
> > >
>
> > > > (AbstractApplicationContext.java:543)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.web.context.ContextLoader.configureAndRefreshW
>
> > > > eb
>
> > > > Ap
>
> > >
>
> > > > plicationContext(ContextLoader.java:443)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.web.context.ContextLoader.initWebApplicationCo
>
> > > > nt
>
> > > > ex
>
> > >
>
> > > > t(ContextLoader.java:325)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.web.context.ContextLoaderListener.contextIniti
>
> > > > al
>
> > > > iz
>
> > >
>
> > > > ed(ContextLoaderListener.java:107)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.server.handler.ContextHandler.callContextInitial
>
> > > > iz
>
> > > > ed
>
> > >
>
> > > > (ContextHandler.java:876)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.servlet.ServletContextHandler.callContextInitial
>
> > > > iz
>
> > > > ed
>
> > >
>
> > > > (ServletContextHandler.java:532)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.server.handler.ContextHandler.startContext(Conte
>
> > > > xt
>
> > > > Ha
>
> > >
>
> > > > ndler.java:839)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.servlet.ServletContextHandler.startContext(Servl
>
> > > > et
>
> > > > Co
>
> > >
>
> > > > ntextHandler.java:344)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:
>
> > >
>
> > > > 1480)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.
>
> > > > ja
>
> > > > va
>
> > >
>
> > > > :1442)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHan
>
> > > > dl
>
> > > > er
>
> > >
>
> > > > .java:799)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletCon
>
> > > > te
>
> > > > xt
>
> > >
>
> > > > Handler.java:261)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:
>
> > > > 54
>
> > > > 0)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractL
>
> > > > if
>
> > > > eC
>
> > >
>
> > > > ycle.java:68)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.util.component.ContainerLifeCycle.start(Containe
>
> > > > rL
>
> > > > if
>
> > >
>
> > > > eCycle.java:131)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(Contai
>
> > > > ne
>
> > > > rL
>
> > >
>
> > > > ifeCycle.java:113)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractH
>
> > > > an
>
> > > > dl
>
> > >
>
> > > > er.java:113)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractL
>
> > > > if
>
> > > > eC
>
> > >
>
> > > > ycle.java:68)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.util.component.ContainerLifeCycle.start(Containe
>
> > > > rL
>
> > > > if
>
> > >
>
> > > > eCycle.java:131)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(Contai
>
> > > > ne
>
> > > > rL
>
> > >
>
> > > > ifeCycle.java:105)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractH
>
> > > > an
>
> > > > dl
>
> > >
>
> > > > er.java:113)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.
>
> > >
>
> > > > java:290)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractL
>
> > > > if
>
> > > > eC
>
> > >
>
> > > > ycle.java:68)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.util.component.ContainerLifeCycle.start(Containe
>
> > > > rL
>
> > > > if
>
> > >
>
> > > > eCycle.java:131)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(Contai
>
> > > > ne
>
> > > > rL
>
> > >
>
> > > > ifeCycle.java:113)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractH
>
> > > > an
>
> > > > dl
>
> > >
>
> > > > er.java:113)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractL
>
> > > > if
>
> > > > eC
>
> > >
>
> > > > ycle.java:68)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.util.component.ContainerLifeCycle.start(Containe
>
> > > > rL
>
> > > > if
>
> > >
>
> > > > eCycle.java:131)
>
> > >
>
> > > >
>
> > >
>
> > > > at org.eclipse.jetty.server.Server.start(Server.java:452)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(Contai
>
> > > > ne
>
> > > > rL
>
> > >
>
> > > > ifeCycle.java:105)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractH
>
> > > > an
>
> > > > dl
>
> > >
>
> > > > er.java:113)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > > > org.eclipse.jetty.server.Server.doStart(Server.java:419)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractL
>
> > > > if
>
> > > > eC
>
> > >
>
> > > > ycle.java:68)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.apache.nifi.web.server.JettyServer.start(JettyServer.java:838)
>
> > >
>
> > > >
>
> > >
>
> > > > at org.apache.nifi.NiFi.<init>(NiFi.java:157)
>
> > >
>
> > > >
>
> > >
>
> > > > at org.apache.nifi.NiFi.<init>(NiFi.java:71)
>
> > >
>
> > > >
>
> > >
>
> > > > at org.apache.nifi.NiFi.main(NiFi.java:292)
>
> > >
>
> > > >
>
> > >
>
> > > > Caused by: org.springframework.beans.factory.BeanExpressionException:
>
> > >
>
> > > > Expression parsing failed; nested exception is
>
> > >
>
> > > > org.springframework.beans.factory.UnsatisfiedDependencyException:
>
> > >
>
> > > > Error creating bean with name
>
> > >
>
> > > > 'org.apache.nifi.web.NiFiWebApiSecurityConfiguration': Unsatisfied
>
> > >
>
> > > > dependency expressed through method 'setJwtAuthenticationProvider'
>
> > >
>
> > > > parameter 0; nested exception is
>
> > >
>
> > > > org.springframework.beans.factory.BeanCreationException: Error
>
> > >
>
> > > > creating bean with name 'jwtAuthenticationProvider' defined in
>
> > > > class
>
> > >
>
> > > > path resource [nifi-web-security-context.xml]: Cannot resolve
>
> > >
>
> > > > reference to bean 'authorizer' while setting constructor argument;
>
> > >
>
> > > > nested exception is
>
> > >
>
> > > > org.springframework.beans.factory.BeanCreationException: Error
>
> > >
>
> > > > creating bean with name 'authorizer': FactoryBean threw exception
>
> > > > on
>
> > >
>
> > > > object creation; nested exception is java.lang.NullPointerException:
>
> > >
>
> > > > Name is null
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.context.expression.StandardBeanExpressionResolver.
>
> > >
>
> > > > evaluate(StandardBeanExpressionResolver.java:164)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.support.AbstractBeanFactory.eval
>
> > > > ua
>
> > > > te
>
> > >
>
> > > > BeanDefinitionString(AbstractBeanFactory.java:1448)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.support.DefaultListableBeanFacto
>
> > > > ry
>
> > > > .d
>
> > >
>
> > > > oResolveDependency(DefaultListableBeanFactory.java:1088)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.support.DefaultListableBeanFacto
>
> > > > ry
>
> > > > .r
>
> > >
>
> > > > esolveDependency(DefaultListableBeanFactory.java:1066)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.annotation.AutowiredAnnotationBe
>
> > > > an
>
> > > > Po
>
> > >
>
> > > > stProcessor$AutowiredMethodElement.inject(AutowiredAnnotationBeanP
>
> > > > os
>
> > > > tP
>
> > >
>
> > > > rocessor.java:659)
>
> > >
>
> > > >
>
> > >
>
> > > > ... 48 common frames omitted
>
> > >
>
> > > >
>
> > >
>
> > > > Caused by:
>
> > >
>
> > > > org.springframework.beans.factory.UnsatisfiedDependencyException:
>
> > >
>
> > > > Error creating bean with name
>
> > >
>
> > > > 'org.apache.nifi.web.NiFiWebApiSecurityConfiguration': Unsatisfied
>
> > >
>
> > > > dependency expressed through method 'setJwtAuthenticationProvider'
>
> > >
>
> > > > parameter 0; nested exception is
>
> > >
>
> > > > org.springframework.beans.factory.BeanCreationException: Error
>
> > >
>
> > > > creating bean with name 'jwtAuthenticationProvider' defined in
>
> > > > class
>
> > >
>
> > > > path resource [nifi-web-security-context.xml]: Cannot resolve
>
> > >
>
> > > > reference to bean 'authorizer' while setting constructor argument;
>
> > >
>
> > > > nested exception is
>
> > >
>
> > > > org.springframework.beans.factory.BeanCreationException: Error
>
> > >
>
> > > > creating bean with name 'authorizer': FactoryBean threw exception
>
> > > > on
>
> > >
>
> > > > object creation; nested exception is java.lang.NullPointerException:
>
> > >
>
> > > > Name is null
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.annotation.AutowiredAnnotationBe
>
> > > > an
>
> > > > Po
>
> > >
>
> > > > stProcessor$AutowiredMethodElement.inject(AutowiredAnnotationBeanP
>
> > > > os
>
> > > > tP
>
> > >
>
> > > > rocessor.java:667)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.annotation.InjectionMetadata.inj
>
> > > > ec
>
> > > > t(
>
> > >
>
> > > > InjectionMetadata.java:88)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.annotation.AutowiredAnnotationBe
>
> > > > an
>
> > > > Po
>
> > >
>
> > > > stProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostP
>
> > > > ro
>
> > > > ce
>
> > >
>
> > > > ssor.java:366)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.support.AbstractAutowireCapableB
>
> > > > ea
>
> > > > nF
>
> > >
>
> > > > actory.populateBean(AbstractAutowireCapableBeanFactory.java:1264)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.support.AbstractAutowireCapableB
>
> > > > ea
>
> > > > nF
>
> > >
>
> > > > actory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.support.AbstractAutowireCapableB
>
> > > > ea
>
> > > > nF
>
> > >
>
> > > > actory.createBean(AbstractAutowireCapableBeanFactory.java:483)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.support.AbstractBeanFactory$1.ge
>
> > > > tO
>
> > > > bj
>
> > >
>
> > > > ect(AbstractBeanFactory.java:306)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.support.DefaultSingletonBeanRegi
>
> > > > st
>
> > > > ry
>
> > >
>
> > > > .getSingleton(DefaultSingletonBeanRegistry.java:230)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.support.AbstractBeanFactory.doGe
>
> > > > tB
>
> > > > ea
>
> > >
>
> > > > n(AbstractBeanFactory.java:302)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.support.AbstractBeanFactory.getB
>
> > > > ea
>
> > > > n(
>
> > >
>
> > > > AbstractBeanFactory.java:202)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.support.DefaultListableBeanFacto
>
> > > > ry
>
> > > > .g
>
> > >
>
> > > > etBeansOfType(DefaultListableBeanFactory.java:519)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.beans.factory.support.DefaultListableBeanFacto
>
> > > > ry
>
> > > > .g
>
> > >
>
> > > > etBeansOfType(DefaultListableBeanFactory.java:508)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.security.config.annotation.web.configuration.A
>
> > > > ut
>
> > > > ow
>
> > >
>
> > > > iredWebSecurityConfigurersIgnoreParents.getWebSecurityConfigurers(
>
> > > > Au
>
> > > > to
>
> > >
>
> > > > wiredWebSecurityConfigurersIgnoreParents.java:53)
>
> > >
>
> > > >
>
> > >
>
> > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>
> > > > Method)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorIm
>
> > > > pl
>
> > > > .j
>
> > >
>
> > > > ava:62)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAc
>
> > > > ce
>
> > > > ss
>
> > >
>
> > > > orImpl.java:43)
>
> > >
>
> > > >
>
> > >
>
> > > > at java.lang.reflect.Method.invoke(Method.java:498)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.expression.spel.support.ReflectiveMethodExecut
>
> > > > or
>
> > > > .e
>
> > >
>
> > > > xecute(ReflectiveMethodExecutor.java:113)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.expression.spel.ast.MethodReference.getValueIn
>
> > > > te
>
> > > > rn
>
> > >
>
> > > > al(MethodReference.java:129)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.expression.spel.ast.MethodReference.access$000
>
> > > > (M
>
> > > > et
>
> > >
>
> > > > hodReference.java:49)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.expression.spel.ast.MethodReference$MethodValu
>
> > > > eR
>
> > > > ef
>
> > >
>
> > > > .getValue(MethodReference.java:347)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.expression.spel.ast.CompoundExpression.getValu
>
> > > > eI
>
> > > > nt
>
> > >
>
> > > > ernal(CompoundExpression.java:88)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.expression.spel.ast.SpelNodeImpl.getValue(Spel
>
> > > > No
>
> > > > de
>
> > >
>
> > > > Impl.java:120)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.expression.spel.standard.SpelExpression.getVal
>
> > > > ue
>
> > > > (S
>
> > >
>
> > > > pelExpression.java:262)
>
> > >
>
> > > >
>
> > >
>
> > > > at
>
> > >
>
> > > > org.springframework.context.expression.StandardBeanExpressionResolver.
>
> > >
>
> > > > evaluate(StandardBeanExpressionResolver.java:161)
>
> > >
>
> > > >
>
> > >
>
> > > > ... 52 common frames omitted
>
> > >
>
> > > >
>
> > >
>
> > > >
>
> > >
>
> > > >
>
> > >
>
> > > > I tried to Google for possible clues, but so far, there hasn’t
>
> > > > been
>
> > >
>
> > > > any luck…
>
> > >
>
> > > >
>
> > >
>
> > > >
>
> > >
>
> > > >
>
> > >
>
> > > > -----Original Message-----
>
> > >
>
> > > > From: Bryan Bende <bbe...@gmail.com>
>
> > >
>
> > > > Sent: Monday, October 15, 2018 10:27 AM
>
> > >
>
> > > > To: users@nifi.apache.org
>
> > >
>
> > > > Subject: Re: NiFi fails on cluster nodes
>
> > >
>
> > > >
>
> > >
>
> > > >
>
> > >
>
> > > >
>
> > >
>
> > > > I'm not really sure, the error message is indicating that either a
> > > > certificate was not sent during cluster communications, or possibly the
> > > > cert was not valid/trusted.
>
> > >
>
> > > >
>
> > >
>
> > > >
>
> > >
>
> > > >
>
> > >
>
> > > > In this case since it is only 1 node, it is the same node talking back
> > > > to itself, so the only parts involved here are the keystore and
> > > > truststore of that node, and the config in nifi.properties.
>
> > >
>
> > > >
>
> > >
>
> > > >
>
> > >
>
> > > >
>
> > >
>
> > > > Maybe your truststore is not setup correctly to trust certs signed by
> > > > the CA that created the server cert?
>
> > >
>
> > > >
>
> > >
>
> > > > On Mon, Oct 15, 2018 at 9:53 AM Saip, Alexander (NIH/CC/BTRIS) [C]
> > > > <alexander.s...@nih.gov> wrote:
>
> > >
>
> > > >
>
> > >
>
> > > > >
>
> > >
>
> > > >
>
> > >
>
> > > > > Yes, 'nifi.cluster.protocol.is.secure' is set to 'true', since
> > > > > otherwise, NiFi would require values for 'nifi.web.http.host' and
> > > > > 'nifi.web.http.port'. We have a cert that is used to serve HTTPS
> > > > > requests to the NiFi web UI, and it works just fine.
>
> > >
>
> > > >
>
> > >
>
> > > > >
>
> > >
>
> > > >
>
> > >
>
> > > > > -----Original Message-----
>
> > >
>
> > > >
>
> > >
>
> > > > > From: Bryan Bende <bbe...@gmail.com>
>
> > >
>
> > > >
>
> > >
>
> > > > > Sent: Monday, October 15, 2018 9:43 AM
>
> > >
>
> > > >
>
> > >
>
> > > > > To: users@nifi.apache.org
>
> > >
>
> > > >
>
> > >
>
> > > > > Subject: Re: NiFi fails on cluster nodes
>
> > >
>
> > > >
>
> > >
>
> > > > >
>
> > >
>
> > > >
>
> > >
>
> > > > > This is not related to ZooKeeper... I think you are missing something
> > > > > related to TLS/SSL configuration, maybe you set cluster protocol to
> > > > > be secure, but then you didn't configure NiFi with a
> > > > > keystore/truststore?
>
> > >
>
> > > >
>
> > >
>
> > > > >
>
> > >
>
> > > >
>
> > >
>
> > > > > On Mon, Oct 15, 2018 at 9:41 AM Mike Thomsen <mikerthom...@gmail.com>
> > > > > wrote:
>
> > >
>
> > > >
>
> > >
>
> > > > > >
>
> > >
>
> > > >
>
> > >
>
> > > > > > Not sure what's going on here, but NiFi does not require a cert to
> > > > > > setup ZooKeeper.
>
> > >
>
> > > >
>
> > >
>
> > > > > >
>
> > >
>
> > > >
>
> > >
>
> > > > > > Mike
>
> > >
>
> > > >
>
> > >
>
> > > > > >
>
> > >
>
> > > >
>
> > >
>
> > > > > > On Mon, Oct 15, 2018 at 9:39 AM Saip, Alexander (NIH/CC/BTRIS) [C]
> > > > > > <alexander.s...@nih.gov> wrote:
>
> > >
>
> > > >
>
> > >
>
> > > > > >>
>
> > >
>
> > > >
>
> > >
>
> > > > > >> Hi Mike and Bryan,
>
> > >
>
> > > >
>
> > >
>
> > > > > >>
>
> > >
>
> > > >
>
> > >
>
> > > > > >>
>
> > >
>
> > > >
>
> > >
>
> > > > > >>
>
> > >
>
> > > >
>
> > >
>
> > > > > >> I’ve installed and started ZooKeeper 3.4.13 and re-started a
> > > > > >> single NiFi node so far. Here is the error from the NiFi log:
>
> > >
>
> > > >
>
> > >
>
> > > > > >>
>
> > >
>
> > > >
>
> > >
>
> > > > > >>
>
> > >
>
> > > >
>
> > >
>
> > > > > >>
>
> > >
>
> > > >
>
> > >
>
> > > > > >> 2018-10-15 09:19:48,371 ERROR [Process Cluster Protocol
>
> > >
>
> > > > > >> Request-1]
>
> > >
>
> > > >
>
> > >
>
> > > > > >> o.a.nifi.security.util.CertificateUtils The incoming request
>
> > > > > >> did
>
> > >
>
> > > >
>
> > >
>
> > > > > >> not contain client certificates and thus the DN cannot be
> > > > > >> extracted.
>
> > >
>
> > > >
>
> > >
>
> > > > > >> Check that the other endp
