Janaki, In addition to looking at the nifi-bootstrap.log file, I would check the $NIFI_HOME directory and see if there are any files that have a name like hs_err_pid_<pid>.log. These indicate that the entire JVM suddenly crashed, generally due to a bug in Java. We have seen this sometimes when using the G1 Garbage Collector, which is currently the default Garbage Collector. If you are seeing these types of files, I would recommend commenting out the following line in $NIFI_HOME/conf/bootstrap.conf:
java.arg.13=-XX:+UseG1GC (This is around line 49 or so in the file). I hope this helps! Thanks -Mark On May 5, 2017, at 7:41 AM, Joe Witt <[email protected]<mailto:[email protected]>> wrote: Is there anything interesting in the nifi-bootstrap.log? On May 5, 2017 6:52 AM, "Janaki Joshi" <[email protected]<mailto:[email protected]>> wrote: Hello all, I've been using NiFi for a while now. Since last evening I've started facing an issue that I am unable to get to the root of. My NiFi (Deployed on an EC2 instance running Ubuntu) keeps relaunching every 3-4 minutes. Below you can find a snippet of my app.log file. I have highlighted the area where NiFi suddenly restarts without warning (the portion between the dashed lines). One change which could be a possible reason is the JVM heap size. I increased it from 4g to 5g yesterday (I have an 8GB RAM.) Any help would be appreciated. Thanks in advance. Regards, Janaki nifi-app.log: 2017-05-05 07:01:47,537 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@28ffa527 checkpointed with 17 Records and 0 Swap Files in 5 milliseconds (Stop-the-world time = 1 milliseconds, Clear Edit Logs time = 1 millis), max Transaction ID 50 2017-05-05 07:01:51,028 INFO [pool-8-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository 2017-05-05 07:01:51,093 INFO [pool-8-thread-1] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@77614e7c checkpointed with 0 Records and 0 Swap Files in 65 milliseconds (Stop-the-world time = 31 milliseconds, Clear Edit Logs time = 26 millis), max Transaction ID 7732406 2017-05-05 07:01:51,094 INFO [pool-8-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 0 records in 65 milliseconds 2017-05-05 07:02:02,743 INFO [Flow Service Tasks Thread-1] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@64c45102 // Another save pending = false 2017-05-05 07:02:49,899 INFO [Flow Service Tasks Thread-1] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@64c45102 // Another save pending = false 2017-05-05 07:03:48,302 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@28ffa527 checkpointed with 17 Records and 0 Swap Files in 20 milliseconds (Stop-the-world time = 0 milliseconds, Clear Edit Logs time = 0 millis), max Transaction ID 50 2017-05-05 07:03:51,094 INFO [pool-8-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository 2017-05-05 07:03:51,748 INFO [pool-8-thread-1] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@77614e7c checkpointed with 0 Records and 0 Swap Files in 653 milliseconds (Stop-the-world time = 108 milliseconds, Clear Edit Logs time = 536 millis), max Transaction ID 7732406 ----------------------------------- 2017-05-05 07:03:51,748 INFO [pool-8-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 0 records in 653 milliseconds 2017-05-05 07:04:14,113 INFO [main] org.apache.nifi.NiFi Launching NiFi... ----------------------------------- 2017-05-05 07:04:14,587 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Determined default nifi.properties path to be '/home/ubuntu/nifi-1.1.2/./conf/nifi.properties' 2017-05-05 07:04:14,592 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Loaded 121 properties from /home/ubuntu/nifi-1.1.2/./conf/nifi.properties 2017-05-05 07:04:14,602 INFO [main] org.apache.nifi.NiFi Loaded 121 properties 2017-05-05 07:04:14,610 INFO [main] org.apache.nifi.BootstrapListener Started Bootstrap Listener, Listening for incoming requests on port 42755 2017-05-05 07:04:14,628 INFO [main] org.apache.nifi.BootstrapListener Successfully initiated communication with Bootstrap 2017-05-05 07:05:04,366 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ubuntu/nifi-1.1.2/./work/nar/extensions/nifi-jetty-bundle-1.1.2.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-jetty-bundle-1.1.2.nar-unpacked] 2017-05-05 07:05:04,375 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ubuntu/nifi-1.1.2/./work/nar/framework/nifi-framework-nar-1.1.2.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/framework/nifi-framework-nar-1.1.2.nar-unpacked] 2017-05-05 07:05:04,376 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ubuntu/nifi-1.1.2/./work/nar/extensions/nifi-slack-nar-1.1.2.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-slack-nar-1.1.2.nar-unpacked] 2017-05-05 07:05:04,377 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ubuntu/nifi-1.1.2/./work/nar/extensions/nifi-ldap-iaa-providers-nar-1.1.2.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-ldap-iaa-providers-nar-1.1.2.nar-unpacked] 2017-05-05 07:05:04,378 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ubuntu/nifi-1.1.2/./work/nar/extensions/nifi-provenance-repository-nar-1.1.2.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-provenance-repository-nar-1.1.2.nar-unpacked] 2017-05-05 07:05:04,382 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ubuntu/nifi-1.1.2/./work/nar/extensions/nifi-azure-nar-1.1.2.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-azure-nar-1.1.2.nar-unpacked] 2017-05-05 07:05:04,383 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /home/ubuntu/nifi-1.1.2/./work/nar/extensions/nifi-update-attribute-nar-1.1.2.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-update-attribute-nar-1.1.2.nar-unpacked]
