Hi,
I have finally taken the plunge to upgrade my cluster from 0.6.1 to 1.0.0.
6 nodes with a NCM.
With the removal of NCM in 1.0.0 I believe I now have an issue where none of my
Remote Process Groups work as they previously did because they were configured
to connect to the NCM (as the RPG url) which now doesn’t exist.
I have tried converting my NCM to a node but whilst I can get it running (sort
of) when I try and connect to the cluster I get something like this in my logs…
2016-10-19 13:14:44,109 ERROR [main] o.a.nifi.controller.StandardFlowService
Failed to load flow from cluster due to:
org.apache.nifi.controller.UninheritableFlowException: Failed to connect node
to cluster because local flow is different than cluster flow.
org.apache.nifi.controller.UninheritableFlowException: Failed to connect node
to cluster because local flow is different than cluster flow.
at
org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:879)
~[nifi-framework-core-1.0.0.jar:1.0.0]
at
org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:493)
~[nifi-framework-core-1.0.0.jar:1.0.0]
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:746)
[nifi-jetty-1.0.0.jar:1.0.0]
at org.apache.nifi.NiFi.<init>(NiFi.java:152)
[nifi-runtime-1.0.0.jar:1.0.0]
at org.apache.nifi.NiFi.main(NiFi.java:243)
[nifi-runtime-1.0.0.jar:1.0.0]
Caused by: org.apache.nifi.controller.UninheritableFlowException: Proposed
Authorizer is not inheritable by the flow controller because of Authorizer
differences: Proposed Authorizations do not match current Authorizations
at
org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:252)
~[nifi-framework-core-1.0.0.jar:1.0.0]
at
org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1435)
~[nifi-framework-core-1.0.0.jar:1.0.0]
at
org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:83)
~[nifi-framework-core-1.0.0.jar:1.0.0]
at
org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:671)
~[nifi-framework-core-1.0.0.jar:1.0.0]
at
org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:857)
~[nifi-framework-core-1.0.0.jar:1.0.0]
... 4 common frames omitted
2016-10-19 13:14:44,414 ERROR [main] o.a.n.c.c.node.NodeClusterCoordinator
Event Reported for ncm-cm1.mis-cds.local:9090 -- Node disconnected from cluster
due to org.apache.nifi.controller.UninheritableFlowException: Failed to connect
node to cluster because local flow is different than cluster flow.
2016-10-19 13:14:44,420 ERROR [Shutdown Cluster Coordinator]
org.apache.nifi.NiFi An Unknown Error Occurred in Thread Thread[Shutdown
Cluster Coordinator,5,main]: java.lang.NullPointerException
2016-10-19 13:14:44,423 ERROR [Shutdown Cluster Coordinator]
org.apache.nifi.NiFi
java.lang.NullPointerException: null
at
java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
~[na:1.8.0_51]
at
java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
~[na:1.8.0_51]
at
org.apache.nifi.cluster.coordination.node.NodeClusterCoordinator.updateNodeStatus(NodeClusterCoordinator.java:570)
~[nifi-framework-cluster-1.0.0.jar:1.0.0]
at
org.apache.nifi.cluster.coordination.node.NodeClusterCoordinator.shutdown(NodeClusterCoordinator.java:119)
~[nifi-framework-cluster-1.0.0.jar:1.0.0]
at
org.apache.nifi.controller.StandardFlowService$1.run(StandardFlowService.java:330)
~[nifi-framework-core-1.0.0.jar:1.0.0]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_51]
2016-10-19 13:14:44,448 WARN [main] o.a.n.c.l.e.CuratorLeaderElectionManager
Failed to close Leader Selector for Cluster Coordinator
java.lang.IllegalStateException: Already closed or has not been started
at
com.google.common.base.Preconditions.checkState(Preconditions.java:173)
~[guava-18.0.jar:na]
at
org.apache.curator.framework.recipes.leader.LeaderSelector.close(LeaderSelector.java:270)
~[curator-recipes-2.11.0.jar:na]
at
org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.stop(CuratorLeaderElectionManager.java:159)
~[nifi-framework-core-1.0.0.jar:1.0.0]
at
org.apache.nifi.controller.FlowController.shutdown(FlowController.java:1303)
[nifi-framework-core-1.0.0.jar:1.0.0]
at
org.apache.nifi.controller.StandardFlowService.stop(StandardFlowService.java:339)
[nifi-framework-core-1.0.0.jar:1.0.0]
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:753)
[nifi-jetty-1.0.0.jar:1.0.0]
at org.apache.nifi.NiFi.<init>(NiFi.java:152)
[nifi-runtime-1.0.0.jar:1.0.0]
at org.apache.nifi.NiFi.main(NiFi.java:243)
[nifi-runtime-1.0.0.jar:1.0.0]
2016-10-19 13:14:45,062 WARN [Cluster Socket Listener]
org.apache.nifi.io.socket.SocketListener Failed to communicate with Unknown
Host due to java.net.SocketException: Socket closed
java.net.SocketException: Socket closed
at java.net.PlainSocketImpl.socketAccept(Native Method) ~[na:1.8.0_51]
at
java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:404)
~[na:1.8.0_51]
at java.net.ServerSocket.implAccept(ServerSocket.java:545)
~[na:1.8.0_51]
at
sun.security.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:348)
~[na:1.8.0_51]
at
org.apache.nifi.io.socket.SocketListener$2.run(SocketListener.java:112)
~[nifi-socket-utils-1.0.0.jar:1.0.0]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
2016-10-19 13:14:45,064 WARN [main] org.apache.nifi.web.server.JettyServer
Failed to start web server... shutting down.
java.lang.Exception: Unable to load flow due to: java.io.IOException:
org.apache.nifi.controller.UninheritableFlowException: Failed to connect node
to cluster because local flow is different than cluster flow.
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:755)
~[nifi-jetty-1.0.0.jar:1.0.0]
at org.apache.nifi.NiFi.<init>(NiFi.java:152)
[nifi-runtime-1.0.0.jar:1.0.0]
at org.apache.nifi.NiFi.main(NiFi.java:243)
[nifi-runtime-1.0.0.jar:1.0.0]
Caused by: java.io.IOException:
org.apache.nifi.controller.UninheritableFlowException: Failed to connect node
to cluster because local flow is different than cluster flow.
at
org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:497)
~[nifi-framework-core-1.0.0.jar:1.0.0]
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:746)
~[nifi-jetty-1.0.0.jar:1.0.0]
... 2 common frames omitted
Caused by: org.apache.nifi.controller.UninheritableFlowException: Failed to
connect node to cluster because local flow is different than cluster flow.
at
org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:879)
~[nifi-framework-core-1.0.0.jar:1.0.0]
at
org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:493)
~[nifi-framework-core-1.0.0.jar:1.0.0]
... 3 common frames omitted
Caused by: org.apache.nifi.controller.UninheritableFlowException: Proposed
Authorizer is not inheritable by the flow controller because of Authorizer
differences: Proposed Authorizations do not match current Authorizations
at
org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:252)
~[nifi-framework-core-1.0.0.jar:1.0.0]
at
org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1435)
~[nifi-framework-core-1.0.0.jar:1.0.0]
at
org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:83)
~[nifi-framework-core-1.0.0.jar:1.0.0]
at
org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:671)
~[nifi-framework-core-1.0.0.jar:1.0.0]
at
org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:857)
~[nifi-framework-core-1.0.0.jar:1.0.0]
... 4 common frames omitted
[root@ncm-cm1 logs]#
I don’t know if the ‘Proposed Authorizer is not inheritable…’ exception is part
of the problem too.
The docs weren’t very clear on whether (when upgrading and using the legacy
support of the authorized-user.xml path required the nodes to be also added to
the authorizers.xml.
I did add them in the end as various attempts to get the cluster up and running
without them failed (as each server didn’t seem to have rights to do anything.
I have a lot of RPG in my work flows as I am ingesting many syslog data sources
and this was the recommended pattern to distribute the data (listensyslog…run
on primary, output to port (RPG), pick up in rest of data flow),
Any suggestions on where to start trying to get this working?
I’ve tried creating a new RPG on one on the datanodes and connecting the syslog
to that which sort of worked but then I have a bunch of other errors when
trying to enable the ports to do with not being able to connect to (what was)
the NCM.
Thanks
Conrad
SecureData, combating cyber threats
______________________________________________________________________
The information contained in this message or any of its attachments may be
privileged and confidential and intended for the exclusive use of the intended
recipient. If you are not the intended recipient any disclosure, reproduction,
distribution or other dissemination or use of this communications is strictly
prohibited. The views expressed in this email are those of the individual and
not necessarily of SecureData Europe Ltd. Any prices quoted are only valid if
followed up by a formal written quote.
SecureData Europe Limited. Registered in England & Wales 04365896. Registered
Address: SecureData House, Hermitage Court, Hermitage Lane, Maidstone, Kent,
ME16 9NT
