Joseph Niemiec created METRON-841:
-------------------------------------
Summary: Failed to connect node to cluster due to:
java.lang.NullPointerException
Key: METRON-841
URL: https://issues.apache.org/jira/browse/METRON-841
Project: Metron
Issue Type: Bug
Environment: SUSE 11 SP3, NIFI 1.1.0.2.1.2.0-10, Java jdk1.8.0_60, 2
node clustered Nifi
Reporter: Joseph Niemiec
We had a server fill its root directory, we stopped NiFi, moved the
repositories to mounts with space, updated the configs and now we have a single
node that will not join the cluster. There are no major errors but we do run
into some WARNS talking about an NPE before Jetty explodes.
2017-04-10 14:55:57,293 ERROR [main] o.a.nifi.controller.StandardFlowService
Failed to load flow from cluster due to:
org.apache.nifi.cluster.ConnectionException: Failed to connect node to cluster
due to: java.lang.NullPointerException
org.apache.nifi.cluster.ConnectionException: Failed to connect node to cluster
due to: java.lang.NullPointerException
at
org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:901)
~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at
org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:493)
~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:831)
[nifi-jetty-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at org.apache.nifi.NiFi.<init>(NiFi.java:156)
[nifi-runtime-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at org.apache.nifi.NiFi.main(NiFi.java:262)
[nifi-runtime-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
Caused by: java.lang.NullPointerException: null
at
org.apache.nifi.controller.repository.SchemaRepositoryRecordSerde.deserializeRecord(SchemaRepositoryRecordSerde.java:119)
~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at
org.apache.nifi.controller.repository.SchemaRepositoryRecordSerde.deserializeEdit(SchemaRepositoryRecordSerde.java:109)
~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at
org.apache.nifi.controller.repository.SchemaRepositoryRecordSerde.deserializeEdit(SchemaRepositoryRecordSerde.java:46)
~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at
org.wali.MinimalLockingWriteAheadLog$Partition.recoverNextTransaction(MinimalLockingWriteAheadLog.java:1072)
~[nifi-write-ahead-log-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at
org.wali.MinimalLockingWriteAheadLog.recoverFromEdits(MinimalLockingWriteAheadLog.java:459)
~[nifi-write-ahead-log-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at
org.wali.MinimalLockingWriteAheadLog.recoverRecords(MinimalLockingWriteAheadLog.java:301)
~[nifi-write-ahead-log-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at
org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.loadFlowFiles(WriteAheadFlowFileRepository.java:346)
~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at
org.apache.nifi.controller.FlowController.initializeFlow(FlowController.java:699)
~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at
org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:701)
~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at
org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:872)
~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
... 4 common frames omitted
2017-04-10 14:55:57,293 INFO [main] o.a.n.c.c.node.NodeClusterCoordinator
hpce2r01n11.abc.com:9090 requested disconnection from cluster due to
org.apache.nifi.cluster.ConnectionException: Failed to connect node to cluster
due to: java.lang.NullPointerException
2017-04-10 14:55:57,293 INFO [main] o.a.n.c.c.node.NodeClusterCoordinator
Status of hpce2r01n11.abc.com:9090 changed from
NodeConnectionStatus[nodeId=hpce2r01n11.abc.com:9090, state=CONNECTING,
updateId=60] to NodeConnectionStatus[nodeId=hpce2r01n11.abc.com:9090,
state=DISCONNECTED, Disconnect Code=Node Failed to Startup Properly, Disconnect
Reason=org.apache.nifi.cluster.ConnectionException: Failed to connect node to
cluster due to: java.lang.NullPointerException, updateId=60]
2017-04-10 14:55:57,293 DEBUG [main] o.a.n.c.c.node.NodeClusterCoordinator
State of cluster nodes is now
{hpce2r01n12.abc.com:9090=NodeConnectionStatus[nodeId=hpce2r01n12.abc.com:9090,
state=CONNECTED, updateId=54],
hpce2r01n11.abc.com:9090=NodeConnectionStatus[nodeId=hpce2r01n11.abc.com:9090,
state=DISCONNECTED, Disconnect Code=Node Failed to Startup Properly, Disconnect
Reason=org.apache.nifi.cluster.ConnectionException: Failed to connect node to
cluster due to: java.lang.NullPointerException, updateId=60]}
2017-04-10 14:55:57,294 DEBUG [main] o.a.n.c.c.node.NodeClusterCoordinator
Notifying cluster coordinator that node status changed from
NodeConnectionStatus[nodeId=hpce2r01n11.abc.com:9090, state=CONNECTING,
updateId=60] to NodeConnectionStatus[nodeId=hpce2r01n11.abc.com:9090,
state=DISCONNECTED, Disconnect Code=Node Failed to Startup Properly, Disconnect
Reason=org.apache.nifi.cluster.ConnectionException: Failed to connect node to
cluster due to: java.lang.NullPointerException, updateId=60]
2017-04-10 14:55:57,295 TRACE [main] o.a.curator.utils.DefaultTracerDriver
Trace: GetChildrenBuilderImpl-Foreground - 0 ms
2017-04-10 14:55:57,295 TRACE [main] o.a.curator.utils.DefaultTracerDriver
Trace: GetDataBuilderImpl-Foreground - 0 ms
2017-04-10 14:55:57,404 DEBUG [Notify Cluster of Node Status Change-1]
o.a.n.c.p.i.StandardClusterCoordinationProtocolSender Notified
hpce2r01n12.abc.com:9090 of status change
NodeStatusChangeMessage[nodeId=hpce2r01n11.abc.com:9090,
status=NodeConnectionStatus[nodeId=hpce2r01n11.abc.com:9090,
state=DISCONNECTED, Disconnect Code=Node Failed to Startup Properly, Disconnect
Reason=org.apache.nifi.cluster.ConnectionException: Failed to connect node to
cluster due to: java.lang.NullPointerException, updateId=60]]
2017-04-10 14:55:57,405 ERROR [main] o.a.n.c.c.node.NodeClusterCoordinator
Event Reported for hpce2r01n11.abc.com:9090 -- Node disconnected from cluster
due to org.apache.nifi.cluster.ConnectionException: Failed to connect node to
cluster due to: java.lang.NullPointerException
2017-04-10 14:55:57,405 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager
Cannot unregister Leader Election Role 'Primary Node' becuase that role is not
registered
2017-04-10 14:55:57,407 WARN [main] org.apache.nifi.web.server.JettyServer
Failed to start web server... shutting down.
java.lang.IllegalStateException: Already closed or has not been started
at
com.google.common.base.Preconditions.checkState(Preconditions.java:173)
~[guava-18.0.jar:na]
at
org.apache.curator.framework.recipes.leader.LeaderSelector.close(LeaderSelector.java:270)
~[curator-recipes-2.11.0.jar:na]
at
org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.unregister(CuratorLeaderElectionManager.java:151)
~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at
org.apache.nifi.controller.FlowController.setClustered(FlowController.java:3469)
~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at
org.apache.nifi.controller.StandardFlowService.handleConnectionFailure(StandardFlowService.java:525)
~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at
org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:497)
~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:831)
~[nifi-jetty-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at org.apache.nifi.NiFi.<init>(NiFi.java:156)
[nifi-runtime-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at org.apache.nifi.NiFi.main(NiFi.java:262)
[nifi-runtime-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
2017-04-10 14:55:57,407 INFO [Thread-1] org.apache.nifi.NiFi Initiating
shutdown of Jetty web server...
2017-04-10 14:55:57,408 DEBUG [Thread-1] o.e.j.util.component.AbstractLifeCycle
stopping org.eclipse.jetty.server.Server@16736040
2017-04-10 14:55:57,408 DEBUG [Thread-1] org.eclipse.jetty.server.Server doStop
org.eclipse.jetty.server.Server@16736040
2017-04-10 14:55:57,409 DEBUG [Thread-1] org.eclipse.jetty.server.Server
Graceful shutdown org.eclipse.jetty.server.Server@16736040 by
2017-04-10 14:55:57,410 DEBUG [Thread-1] o.e.j.util.component.AbstractLifeCycle
stopping ServerConnector@546394ed{SSL,[ssl, http/1.1]}{hpce2r01n11.abc.com:9090}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
