...ok so we definitely want to figure out why there was another nifi process still lingering around in the background. Do you have any idea why that might have been? Perhaps the startup script has some corner cases not well covered just yet.
On Fri, Jan 27, 2017 at 12:39 PM, Peter Wicks (pwicks) <[email protected]> wrote: > Restarting the box fixed it. > > > > From: Bryan Rosander [mailto:[email protected]] > Sent: Friday, January 27, 2017 8:51 AM > To: [email protected] > Subject: Re: NiFi 1.1.0 stuck starting, no errors > > > > Hey Peter, > > > > The relevant section of the thread dump is below. It looks like startup is > blocking while trying to get a file lock for the flowfile repository. Are > you positive that there is only one instance of NiFi running with the > repository directory you've configured? If so, maybe something failed to > release the lock in which case a reboot might be worth a try. > > > > Thanks, > > Bryan > > > > "main" #1 prio=5 os_prio=0 tid=0x00007f8da000d000 nid=0xcd68 runnable > [0x00007f8da6daa000] > > java.lang.Thread.State: RUNNABLE > > at sun.nio.ch.FileDispatcherImpl.lock0(Native Method) > > at sun.nio.ch.FileDispatcherImpl.lock(FileDispatcherImpl.java:90) > > at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1073) > > at java.nio.channels.FileChannel.lock(FileChannel.java:1053) > > at > org.wali.MinimalLockingWriteAheadLog.<init>(MinimalLockingWriteAheadLog.java:187) > > at > org.wali.MinimalLockingWriteAheadLog.<init>(MinimalLockingWriteAheadLog.java:112) > > at > org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.initialize(WriteAheadFlowFileRepository.java:148) > > at > org.apache.nifi.controller.FlowController.createFlowFileRepository(FlowController.java:638) > > - locked <0x00000006bfc17cc0> (a > org.apache.nifi.controller.repository.WriteAheadFlowFileRepository) > > at > org.apache.nifi.controller.FlowController.<init>(FlowController.java:447) > > at > org.apache.nifi.controller.FlowController.createStandaloneInstance(FlowController.java:375) > > at > org.apache.nifi.spring.FlowControllerFactoryBean.getObject(FlowControllerFactoryBean.java:74) > > at > org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168) > > at > org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103) > > - locked <0x000000067a34fbf8> (a > java.util.concurrent.ConcurrentHashMap) > > at > org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1585) > > at > org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:317) > > at > org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) > > at > org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1060) > > at > org.apache.nifi.spring.StandardFlowServiceFactoryBean.getObject(StandardFlowServiceFactoryBean.java:48) > > at > org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168) > > at > org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103) > > - locked <0x000000067a34fbf8> (a > java.util.concurrent.ConcurrentHashMap) > > at > org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1585) > > at > org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:317) > > at > org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) > > at > org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1060) > > at > org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:53) > > at > org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:837) > > at > org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:533) > > at > org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:810) > > at > org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:345) > > at > org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1404) > > at > org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1366) > > at > org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:772) > > at > org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:262) > > at > org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:520) > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) > > - locked <0x000000067a34ffb8> (a java.lang.Object) > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) > > - locked <0x000000067a350048> (a java.lang.Object) > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106) > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) > > at > org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:231) > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) > > - locked <0x000000067a3500f8> (a java.lang.Object) > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) > > at org.eclipse.jetty.server.Server.start(Server.java:411) > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106) > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) > > at org.eclipse.jetty.server.Server.doStart(Server.java:378) > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) > > - locked <0x000000067a3501c8> (a java.lang.Object) > > at > org.apache.nifi.web.server.JettyServer.start(JettyServer.java:675) > > at org.apache.nifi.NiFi.<init>(NiFi.java:156) > > at org.apache.nifi.NiFi.main(NiFi.java:262) > > > > On Fri, Jan 27, 2017 at 10:42 AM, Peter Wicks (pwicks) <[email protected]> > wrote: > > Thanks Bryan for the pointer. > > > > Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.101-b13 mixed mode): > > > > "H2 Log Writer NIFI-FLOW-AUDIT" #37 daemon prio=5 os_prio=0 > tid=0x00007f8da3216000 nid=0xd154 in Object.wait() [0x00007f8cf27a1000] > > java.lang.Thread.State: TIMED_WAITING (on object monitor) > > at java.lang.Object.wait(Native Method) > > at org.h2.store.WriterThread.run(WriterThread.java:103) > > - locked <0x000000067a4e0850> (a org.h2.store.WriterThread) > > at java.lang.Thread.run(Thread.java:745) > > > > "H2 File Lock Watchdog > /data/nifi/repositories/database_repository/nifi-flow-audit.lock.db" #36 > daemon prio=9 os_prio=0 tid=0x00007f8da3215000 nid=0xd153 waiting on > condition [0x00007f8cf28a2000] > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > at java.lang.Thread.sleep(Native Method) > > at org.h2.store.FileLock.run(FileLock.java:517) > > at java.lang.Thread.run(Thread.java:745) > > > > "H2 Log Writer NIFI-USER-KEYS" #35 daemon prio=5 os_prio=0 > tid=0x00007f8da2e6c000 nid=0xd135 in Object.wait() [0x00007f8cf2da3000] > > java.lang.Thread.State: TIMED_WAITING (on object monitor) > > at java.lang.Object.wait(Native Method) > > at org.h2.store.WriterThread.run(WriterThread.java:103) > > - locked <0x000000067a4e0d08> (a org.h2.store.WriterThread) > > at java.lang.Thread.run(Thread.java:745) > > > > "H2 File Lock Watchdog > /data/nifi/repositories/database_repository/nifi-user-keys.lock.db" #34 > daemon prio=9 os_prio=0 tid=0x00007f8da3b87800 nid=0xd132 waiting on > condition [0x00007f8cf30a4000] > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > at java.lang.Thread.sleep(Native Method) > > at org.h2.store.FileLock.run(FileLock.java:517) > > at java.lang.Thread.run(Thread.java:745) > > > > "org.eclipse.jetty.server.session.HashSessionManager@4ca970d5Timer" #32 > daemon prio=5 os_prio=0 tid=0x00007f8da18fc000 nid=0xd0d6 waiting on > condition [0x00007f8cf3ba6000] > > java.lang.Thread.State: TIMED_WAITING (parking) > > at sun.misc.Unsafe.park(Native Method) > > - parking to wait for <0x000000067a4e11c0> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > > at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) > > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > > at java.lang.Thread.run(Thread.java:745) > > > > "org.eclipse.jetty.server.session.HashSessionManager@26865b57Timer" #31 > daemon prio=5 os_prio=0 tid=0x00007f8da1871800 nid=0xd08f waiting on > condition [0x00007f8cf3ea7000] > > java.lang.Thread.State: TIMED_WAITING (parking) > > at sun.misc.Unsafe.park(Native Method) > > - parking to wait for <0x000000067a4e1420> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > > at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) > > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > > at java.lang.Thread.run(Thread.java:745) > > > > "org.eclipse.jetty.server.session.HashSessionManager@38929daTimer" #30 > daemon prio=5 os_prio=0 tid=0x00007f8da190b800 nid=0xd02f waiting on > condition [0x00007f8cf3fa8000] > > java.lang.Thread.State: TIMED_WAITING (parking) > > at sun.misc.Unsafe.park(Native Method) > > - parking to wait for <0x000000067a4e1680> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > > at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) > > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > > at java.lang.Thread.run(Thread.java:745) > > > > "org.eclipse.jetty.server.session.HashSessionManager@5f6a8efeTimer" #29 > daemon prio=5 os_prio=0 tid=0x00007f8da3b56800 nid=0xd020 waiting on > condition [0x00007f8d04193000] > > java.lang.Thread.State: TIMED_WAITING (parking) > > at sun.misc.Unsafe.park(Native Method) > > - parking to wait for <0x000000067a4e18e0> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > > at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) > > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > > at java.lang.Thread.run(Thread.java:745) > > > > "org.eclipse.jetty.server.session.HashSessionManager@344f9467Timer" #28 > daemon prio=5 os_prio=0 tid=0x00007f8da3b40800 nid=0xd01f waiting on > condition [0x00007f8d042ac000] > > java.lang.Thread.State: TIMED_WAITING (parking) > > at sun.misc.Unsafe.park(Native Method) > > - parking to wait for <0x000000067a4e1b40> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > > at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) > > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > > at java.lang.Thread.run(Thread.java:745) > > > > "org.eclipse.jetty.server.session.HashSessionManager@4afcae7Timer" #27 > daemon prio=5 os_prio=0 tid=0x00007f8da39ff800 nid=0xd016 waiting on > condition [0x00007f8d04435000] > > java.lang.Thread.State: TIMED_WAITING (parking) > > at sun.misc.Unsafe.park(Native Method) > > - parking to wait for <0x000000067a4e1da0> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > > at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) > > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > > at java.lang.Thread.run(Thread.java:745) > > > > "NiFi Web Server-26" #26 prio=5 os_prio=0 tid=0x00007f8da24e0000 nid=0xd00c > waiting on condition [0x00007f8d04736000] > > java.lang.Thread.State: TIMED_WAITING (parking) > > at sun.misc.Unsafe.park(Native Method) > > - parking to wait for <0x000000067a4e2000> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > > at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > > at > org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) > > at java.lang.Thread.run(Thread.java:745) > > > > "NiFi Web Server-25" #25 prio=5 os_prio=0 tid=0x00007f8da24de000 nid=0xd00b > waiting on condition [0x00007f8d04837000] > > java.lang.Thread.State: TIMED_WAITING (parking) > > at sun.misc.Unsafe.park(Native Method) > > - parking to wait for <0x000000067a4e2000> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > > at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > > at > org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) > > at java.lang.Thread.run(Thread.java:745) > > > > "NiFi Web Server-24" #24 prio=5 os_prio=0 tid=0x00007f8da24dc800 nid=0xd00a > waiting on condition [0x00007f8d04938000] > > java.lang.Thread.State: TIMED_WAITING (parking) > > at sun.misc.Unsafe.park(Native Method) > > - parking to wait for <0x000000067a4e2000> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > > at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > > at > org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) > > at java.lang.Thread.run(Thread.java:745) > > > > "NiFi Web Server-23" #23 prio=5 os_prio=0 tid=0x00007f8da18b2800 nid=0xd009 > waiting on condition [0x00007f8d04a39000] > > java.lang.Thread.State: TIMED_WAITING (parking) > > at sun.misc.Unsafe.park(Native Method) > > - parking to wait for <0x000000067a4e2000> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > > at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > > at > org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) > > at java.lang.Thread.run(Thread.java:745) > > > > "NiFi Web Server-22" #22 prio=5 os_prio=0 tid=0x00007f8da18b0800 nid=0xd008 > waiting on condition [0x00007f8d04b3a000] > > java.lang.Thread.State: TIMED_WAITING (parking) > > at sun.misc.Unsafe.park(Native Method) > > - parking to wait for <0x000000067a4e2000> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > > at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > > at > org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) > > at java.lang.Thread.run(Thread.java:745) > > > > "NiFi Web Server-21" #21 prio=5 os_prio=0 tid=0x00007f8da18ae800 nid=0xd007 > waiting on condition [0x00007f8d04c3b000] > > java.lang.Thread.State: TIMED_WAITING (parking) > > at sun.misc.Unsafe.park(Native Method) > > - parking to wait for <0x000000067a4e2000> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > > at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > > at > org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) > > at java.lang.Thread.run(Thread.java:745) > > > > "NiFi Web Server-20" #20 prio=5 os_prio=0 tid=0x00007f8da1a1c800 nid=0xd004 > waiting on condition [0x00007f8d4a2c5000] > > java.lang.Thread.State: TIMED_WAITING (parking) > > at sun.misc.Unsafe.park(Native Method) > > - parking to wait for <0x000000067a4e2000> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > > at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > > at > org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) > > at java.lang.Thread.run(Thread.java:745) > > > > "NiFi Web Server-19" #19 prio=5 os_prio=0 tid=0x00007f8da1a1b800 nid=0xd003 > waiting on condition [0x00007f8d4a3c6000] > > java.lang.Thread.State: TIMED_WAITING (parking) > > at sun.misc.Unsafe.park(Native Method) > > - parking to wait for <0x000000067a4e2000> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > > at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > > at > org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) > > at java.lang.Thread.run(Thread.java:745) > > > > "Attach Listener" #17 daemon prio=9 os_prio=0 tid=0x00007f8d34001000 > nid=0xcdff waiting on condition [0x0000000000000000] > > java.lang.Thread.State: RUNNABLE > > > > "Listen to Bootstrap" #14 daemon prio=5 os_prio=0 tid=0x00007f8da134c000 > nid=0xcd87 runnable [0x00007f8d4a4c7000] > > java.lang.Thread.State: RUNNABLE > > at java.net.PlainSocketImpl.socketAccept(Native Method) > > at > java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409) > > at java.net.ServerSocket.implAccept(ServerSocket.java:545) > > at java.net.ServerSocket.accept(ServerSocket.java:513) > > at > org.apache.nifi.BootstrapListener$Listener.run(BootstrapListener.java:158) > > at java.lang.Thread.run(Thread.java:745) > > > > "Service Thread" #11 daemon prio=9 os_prio=0 tid=0x00007f8da0d78800 > nid=0xcd80 runnable [0x0000000000000000] > > java.lang.Thread.State: RUNNABLE > > > > "C1 CompilerThread2" #10 daemon prio=9 os_prio=0 tid=0x00007f8da0d53000 > nid=0xcd7f waiting on condition [0x0000000000000000] > > java.lang.Thread.State: RUNNABLE > > > > "C2 CompilerThread1" #9 daemon prio=9 os_prio=0 tid=0x00007f8da0d51000 > nid=0xcd7e waiting on condition [0x0000000000000000] > > java.lang.Thread.State: RUNNABLE > > > > "C2 CompilerThread0" #8 daemon prio=9 os_prio=0 tid=0x00007f8da0d4e800 > nid=0xcd7d waiting on condition [0x0000000000000000] > > java.lang.Thread.State: RUNNABLE > > > > "JDWP Event Helper Thread" #7 daemon prio=10 os_prio=0 > tid=0x00007f8da0d4c800 nid=0xcd7c runnable [0x0000000000000000] > > java.lang.Thread.State: RUNNABLE > > > > "JDWP Transport Listener: dt_socket" #6 daemon prio=10 os_prio=0 > tid=0x00007f8da0d49000 nid=0xcd7b runnable [0x0000000000000000] > > java.lang.Thread.State: RUNNABLE > > > > "Signal Dispatcher" #5 daemon prio=9 os_prio=0 tid=0x00007f8da0d3c000 > nid=0xcd7a runnable [0x0000000000000000] > > java.lang.Thread.State: RUNNABLE > > > > "Surrogate Locker Thread (Concurrent GC)" #4 daemon prio=9 os_prio=0 > tid=0x00007f8da0d3a800 nid=0xcd79 waiting on condition [0x0000000000000000] > > java.lang.Thread.State: RUNNABLE > > > > "Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007f8da0d09000 nid=0xcd78 in > Object.wait() [0x00007f8d4b6fd000] > > java.lang.Thread.State: WAITING (on object monitor) > > at java.lang.Object.wait(Native Method) > > at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143) > > - locked <0x00000005c006beb0> (a java.lang.ref.ReferenceQueue$Lock) > > at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164) > > at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) > > > > "Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007f8da0d04800 > nid=0xcd77 in Object.wait() [0x00007f8d4b7fe000] > > java.lang.Thread.State: WAITING (on object monitor) > > at java.lang.Object.wait(Native Method) > > at java.lang.Object.wait(Object.java:502) > > at java.lang.ref.Reference.tryHandlePending(Reference.java:191) > > - locked <0x00000005c006bef0> (a java.lang.ref.Reference$Lock) > > at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153) > > > > "main" #1 prio=5 os_prio=0 tid=0x00007f8da000d000 nid=0xcd68 runnable > [0x00007f8da6daa000] > > java.lang.Thread.State: RUNNABLE > > at sun.nio.ch.FileDispatcherImpl.lock0(Native Method) > > at sun.nio.ch.FileDispatcherImpl.lock(FileDispatcherImpl.java:90) > > at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1073) > > at java.nio.channels.FileChannel.lock(FileChannel.java:1053) > > at > org.wali.MinimalLockingWriteAheadLog.<init>(MinimalLockingWriteAheadLog.java:187) > > at > org.wali.MinimalLockingWriteAheadLog.<init>(MinimalLockingWriteAheadLog.java:112) > > at > org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.initialize(WriteAheadFlowFileRepository.java:148) > > at > org.apache.nifi.controller.FlowController.createFlowFileRepository(FlowController.java:638) > > - locked <0x00000006bfc17cc0> (a > org.apache.nifi.controller.repository.WriteAheadFlowFileRepository) > > at > org.apache.nifi.controller.FlowController.<init>(FlowController.java:447) > > at > org.apache.nifi.controller.FlowController.createStandaloneInstance(FlowController.java:375) > > at > org.apache.nifi.spring.FlowControllerFactoryBean.getObject(FlowControllerFactoryBean.java:74) > > at > org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168) > > at > org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103) > > - locked <0x000000067a34fbf8> (a > java.util.concurrent.ConcurrentHashMap) > > at > org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1585) > > at > org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:317) > > at > org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) > > at > org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1060) > > at > org.apache.nifi.spring.StandardFlowServiceFactoryBean.getObject(StandardFlowServiceFactoryBean.java:48) > > at > org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168) > > at > org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103) > > - locked <0x000000067a34fbf8> (a > java.util.concurrent.ConcurrentHashMap) > > at > org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1585) > > at > org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:317) > > at > org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) > > at > org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1060) > > at > org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:53) > > at > org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:837) > > at > org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:533) > > at > org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:810) > > at > org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:345) > > at > org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1404) > > at > org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1366) > > at > org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:772) > > at > org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:262) > > at > org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:520) > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) > > - locked <0x000000067a34ffb8> (a java.lang.Object) > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) > > - locked <0x000000067a350048> (a java.lang.Object) > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106) > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) > > at > org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:231) > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) > > - locked <0x000000067a3500f8> (a java.lang.Object) > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) > > at org.eclipse.jetty.server.Server.start(Server.java:411) > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106) > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) > > at org.eclipse.jetty.server.Server.doStart(Server.java:378) > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) > > - locked <0x000000067a3501c8> (a java.lang.Object) > > at > org.apache.nifi.web.server.JettyServer.start(JettyServer.java:675) > > at org.apache.nifi.NiFi.<init>(NiFi.java:156) > > at org.apache.nifi.NiFi.main(NiFi.java:262) > > > > "VM Thread" os_prio=0 tid=0x00007f8da0cfc800 nid=0xcd76 runnable > > > > "Gang worker#0 (Parallel GC Threads)" os_prio=0 tid=0x00007f8da0024000 > nid=0xcd6b runnable > > > > "Gang worker#1 (Parallel GC Threads)" os_prio=0 tid=0x00007f8da0026000 > nid=0xcd6c runnable > > > > "Gang worker#2 (Parallel GC Threads)" os_prio=0 tid=0x00007f8da0027800 > nid=0xcd6d runnable > > > > "Gang worker#3 (Parallel GC Threads)" os_prio=0 tid=0x00007f8da0029800 > nid=0xcd6e runnable > > > > "G1 Main Concurrent Mark GC Thread" os_prio=0 tid=0x00007f8da0045000 > nid=0xcd74 runnable > > > > "Gang worker#0 (G1 Parallel Marking Threads)" os_prio=0 > tid=0x00007f8da0047000 nid=0xcd75 runnable > > > > "G1 Concurrent Refinement Thread#0" os_prio=0 tid=0x00007f8da0033000 > nid=0xcd73 runnable > > > > "G1 Concurrent Refinement Thread#1" os_prio=0 tid=0x00007f8da0031800 > nid=0xcd72 runnable > > > > "G1 Concurrent Refinement Thread#2" os_prio=0 tid=0x00007f8da002f800 > nid=0xcd71 runnable > > > > "G1 Concurrent Refinement Thread#3" os_prio=0 tid=0x00007f8da002d800 > nid=0xcd70 runnable > > > > "G1 Concurrent Refinement Thread#4" os_prio=0 tid=0x00007f8da002c000 > nid=0xcd6f runnable > > > > "VM Periodic Task Thread" os_prio=0 tid=0x00007f8da0d7c800 nid=0xcd81 > waiting on condition > > > > JNI global references: 8070 > > > > From: Bryan Rosander [mailto:[email protected]] > Sent: Friday, January 27, 2017 7:32 AM > To: [email protected] > Subject: Re: NiFi 1.1.0 stuck starting, no errors > > > > Hey Peter, > > > > Weighing in at 200MB means it is likely a heap dump (all working memory). A > thread dump should be much smaller. If using jcmd, please try Thread.print > as the command. Another alternative would be to use jstack [PID]. > > > > Thanks, > > Bryan > > > > On Fri, Jan 27, 2017 at 9:24 AM, Peter Wicks (pwicks) <[email protected]> > wrote: > > It clocked in at around 200MB’s… > https://drive.google.com/file/d/0B4yjbe5sOeAuRXAxT3Z6anRRN1k/view?usp=sharing > > > > > > From: Andy LoPresto [mailto:[email protected]] > Sent: Thursday, January 26, 2017 3:06 PM > To: [email protected] > Subject: Re: NiFi 1.1.0 stuck starting, no errors > > > > Hi Peter, > > Can you provide a thread dump of the process? You should be able to do this > via the jcmd tool [1]. > > [1] > https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/tooldescr006.html#BABEHABG > <https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/tooldescr006.html#BABEHABG> > > Andy LoPresto > [email protected] > [email protected] > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > >> On Jan 26, 2017, at 12:42 PM, Peter Wicks (pwicks) <[email protected]> >> wrote: >> >> I’m looking for help in troubleshooting my NiFi 1.1.0 install. It’s been >> running stably for some time, but I restarted it this morning when I >> deployed an updated custom NAR. Now it gets stuck at startup, see logs at >> the end. > >> There are no error messages, and the processes don’t die. The process just >> seems to be hanging waiting for something. >> >> · My first thought was to try rolling back the modified nar, and >> even just removing the nar all together since it was custom. Neither of >> these made any difference. > >> · I also tried deleting the “work” folder, which has fixed nar >> versioning issues for me in the past (not really related, but was worth a >> shot). This made no difference. > >> · NiFi is set to start with java.arg.2=-Xms4G and >> java.arg.3=-Xmx8G, 22GB’s of free RAM are available on the system (out of >> some 60GB’s total). > >> · I’ve checked running processes, and when I stop NiFi no rouge >> instances are left running. >> · Since NiFi gets stuck right around the JettyServer step I >> checked to see if any processes were using port 8443. No other processes are >> using this port. > >> · I thought maybe a key file was being locked, but with NiFi off >> `lsof | grep nifi` returns no locked files. >> >> Nifi-app Log: >> 2017-01-26 20:23:43,359 INFO [main] org.eclipse.jetty.util.log Logging >> initialized @90357ms >> 2017-01-26 20:23:43,418 INFO [main] org.apache.nifi.web.server.JettyServer >> Configuring Jetty for HTTPs on port: 8443 >> 2017-01-26 20:23:43,691 INFO [main] org.apache.nifi.web.server.JettyServer >> Loading WAR: >> /data/nifi/nifi-1.1.0/./work/nar/extensions/nifi-media-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-image-viewer-1.1.0.war >> with context path set to /nifi-image-viewer-1.1.0 > >> 2017-01-26 20:23:43,702 INFO [main] org.apache.nifi.web.server.JettyServer >> Loading WAR: >> /data/nifi/nifi-1.1.0/./work/nar/extensions/nifi-update-attribute-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-update-attribute-ui-1.1.0.war >> with context path set to /nifi-update-attribute-ui-1.1.0 > >> 2017-01-26 20:23:43,703 INFO [main] org.apache.nifi.web.server.JettyServer >> Loading UI extension [ProcessorConfiguration, >> /nifi-update-attribute-ui-1.1.0] for >> [org.apache.nifi.processors.attributes.UpdateAttribute] > >> 2017-01-26 20:23:43,713 INFO [main] org.apache.nifi.web.server.JettyServer >> Loading WAR: >> /data/nifi/nifi-1.1.0/./work/nar/extensions/nifi-standard-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-standard-content-viewer-1.1.0.war >> with context path set to /nifi-standard-content-viewer-1.1.0 > >> 2017-01-26 20:23:43,723 INFO [main] org.apache.nifi.web.server.JettyServer >> Loading WAR: >> /data/nifi/nifi-1.1.0/./work/nar/extensions/nifi-standard-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-jolt-transform-json-ui-1.1.0.war >> with context path set to /nifi-jolt-transform-json-ui-1.1.0 > >> 2017-01-26 20:23:43,724 INFO [main] org.apache.nifi.web.server.JettyServer >> Loading UI extension [ProcessorConfiguration, >> /nifi-jolt-transform-json-ui-1.1.0] for >> [org.apache.nifi.processors.standard.JoltTransformJSON] > >> 2017-01-26 20:23:43,729 INFO [main] org.apache.nifi.web.server.JettyServer >> Loading WAR: >> /data/nifi/nifi-1.1.0/./work/nar/framework/nifi-framework-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-web-ui-1.1.0.war >> with context path set to /nifi > >> 2017-01-26 20:23:43,733 INFO [main] org.apache.nifi.web.server.JettyServer >> Loading WAR: >> /data/nifi/nifi-1.1.0/./work/nar/framework/nifi-framework-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-web-api-1.1.0.war >> with context path set to /nifi-api > >> 2017-01-26 20:23:43,735 INFO [main] org.apache.nifi.web.server.JettyServer >> Loading WAR: >> /data/nifi/nifi-1.1.0/./work/nar/framework/nifi-framework-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-web-content-viewer-1.1.0.war >> with context path set to /nifi-content-viewer > >> 2017-01-26 20:23:43,738 INFO [main] org.apache.nifi.web.server.JettyServer >> Loading WAR: >> /data/nifi/nifi-1.1.0/./work/nar/framework/nifi-framework-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-web-docs-1.1.0.war >> with context path set to /nifi-docs > >> 2017-01-26 20:23:43,753 INFO [main] org.apache.nifi.web.server.JettyServer >> Loading documents web app with context path set to /nifi-docs > >> 2017-01-26 20:23:43,761 INFO [main] org.apache.nifi.web.server.JettyServer >> Loading WAR: >> /data/nifi/nifi-1.1.0/./work/nar/framework/nifi-framework-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-web-error-1.1.0.war >> with context path set to / > >> 2017-01-26 20:23:43,804 INFO [main] org.eclipse.jetty.server.Server >> jetty-9.3.9.v20160517 > >> 2017-01-26 20:23:44,748 INFO [main] >> o.e.jetty.server.handler.ContextHandler Started >> o.e.j.w.WebAppContext@4b511e61{/nifi-image-viewer-1.1.0,file:///data/nifi/nifi-1.1.0/work/jetty/nifi-image-viewer-1.1.0.war/webapp/,AVAILABLE >> <file:///data/nifi/nifi-1.1.0/work/jetty/nifi-image-viewer-1.1.0.war/webapp/,AVAILABLE>}{./work/nar/extensions/nifi-media-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-image-viewer-1.1.0.war} > >> 2017-01-26 20:23:46,566 INFO [main] >> o.e.jetty.server.handler.ContextHandler Started >> o.e.j.w.WebAppContext@26888c31{/nifi-update-attribute-ui-1.1.0,file:///data/nifi/nifi-1.1.0/work/jetty/nifi-update-attribute-ui-1.1.0.war/webapp/,AVAILABLE >> <file:///data/nifi/nifi-1.1.0/work/jetty/nifi-update-attribute-ui-1.1.0.war/webapp/,AVAILABLE>}{./work/nar/extensions/nifi-update-attribute-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-update-attribute-ui-1.1.0.war} > >> 2017-01-26 20:23:47,365 INFO [main] >> o.e.jetty.server.handler.ContextHandler Started >> o.e.j.w.WebAppContext@7c840fe3{/nifi-standard-content-viewer-1.1.0,file:///data/nifi/nifi-1.1.0/work/jetty/nifi-standard-content-viewer-1.1.0.war/webapp/,AVAILABLE >> <file:///data/nifi/nifi-1.1.0/work/jetty/nifi-standard-content-viewer-1.1.0.war/webapp/,AVAILABLE>}{./work/nar/extensions/nifi-standard-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-standard-content-viewer-1.1.0.war} > >> 2017-01-26 20:23:50,583 INFO [main] >> o.e.jetty.server.handler.ContextHandler Started >> o.e.j.w.WebAppContext@58b03029{/nifi-jolt-transform-json-ui-1.1.0,file:///data/nifi/nifi-1.1.0/work/jetty/nifi-jolt-transform-json-ui-1.1.0.war/webapp/,AVAILABLE >> <file:///data/nifi/nifi-1.1.0/work/jetty/nifi-jolt-transform-json-ui-1.1.0.war/webapp/,AVAILABLE>}{./work/nar/extensions/nifi-standard-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-jolt-transform-json-ui-1.1.0.war} > >> 2017-01-26 20:23:54,945 INFO [main] /nifi No Spring >> WebApplicationInitializer types detected on classpath >> 2017-01-26 20:23:55,068 INFO [main] >> o.e.jetty.server.handler.ContextHandler Started >> o.e.j.w.WebAppContext@2358443e{/nifi,file:///data/nifi/nifi-1.1.0/work/jetty/nifi-web-ui-1.1.0.war/webapp/,AVAILABLE >> <file:///data/nifi/nifi-1.1.0/work/jetty/nifi-web-ui-1.1.0.war/webapp/,AVAILABLE>}{./work/nar/framework/nifi-framework-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-web-ui-1.1.0.war} > >> 2017-01-26 20:23:56,715 INFO [main] /nifi-api No Spring >> WebApplicationInitializer types detected on classpath >> 2017-01-26 20:23:56,727 INFO [main] /nifi-api Initializing Spring root >> WebApplicationContext >> 2017-01-26 20:24:04,029 INFO [main] >> o.a.nifi.properties.NiFiPropertiesLoader Determined default nifi.properties >> path to be '/data/nifi/nifi-1.1.0/./conf/nifi.properties' > >> 2017-01-26 20:24:04,049 INFO [main] >> o.a.nifi.properties.NiFiPropertiesLoader Determined default nifi.properties >> path to be '/data/nifi/nifi-1.1.0/./conf/nifi.properties' > >> 2017-01-26 20:24:04,064 INFO [main] >> o.a.nifi.properties.NiFiPropertiesLoader Loaded 126 properties from >> /data/nifi/nifi-1.1.0/./conf/nifi.properties > >> 2017-01-26 20:24:30,415 INFO [main] >> o.a.nifi.util.FileBasedVariableRegistry Loaded 120 properties from system >> properties and environment variables > >> 2017-01-26 20:24:30,415 INFO [main] >> o.a.nifi.util.FileBasedVariableRegistry Loaded a total of 120 properties. >> Including precedence overrides effective accessible registry key size is 120 > >> >> NiFi-bootstrap Log (including when I killed it after about 10 minutes): >> 2017-01-26 20:22:12,986 INFO [main] org.apache.nifi.bootstrap.Command >> Starting Apache NiFi... >> 2017-01-26 20:22:12,986 INFO [main] org.apache.nifi.bootstrap.Command >> Working Directory: /data/nifi/nifi-1.1.0 >> 2017-01-26 20:22:12,987 INFO [main] org.apache.nifi.bootstrap.Command >> Command: java -classpath >> /data/nifi/nifi-1.1.0/./conf:/data/nifi/nifi-1.1.0/./lib/logback-classic-1.1.3.jar:/data/nifi/nifi-1.1.0/./lib/logback-core-1.1.3.jar:/data/nifi/nifi-1.1.0/./lib/slf4j-api-1.7.12.jar:/data/nifi/nifi-1.1.0/./lib/jcl-over-slf4j-1.7.12.jar:/data/nifi/nifi-1.1.0/./lib/jul-to-slf4j-1.7.12.jar:/data/nifi/nifi-1.1.0/./lib/log4j-over-slf4j-1.7.12.jar:/data/nifi/nifi-1.1.0/./lib/nifi-api-1.1.0.jar:/data/nifi/nifi-1.1.0/./lib/nifi-framework-api-1.1.0.jar:/data/nifi/nifi-1.1.0/./lib/nifi-runtime-1.1.0.jar:/data/nifi/nifi-1.1.0/./lib/nifi-nar-utils-1.1.0.jar:/data/nifi/nifi-1.1.0/./lib/nifi-properties-1.1.0.jar:/data/nifi/nifi-1.1.0/./lib/nifi-documentation-1.1.0.jar:/data/nifi/nifi-1.1.0/./lib/bcprov-jdk15on-1.54.jar:/data/nifi/nifi-1.1.0/./lib/commons-lang3-3.4.jar:/data/nifi/nifi-1.1.0/./lib/ngdbc.jar:/data/nifi/nifi-1.1.0/./lib/ojdbc6.jar:/data/nifi/nifi-1.1.0/./lib/sqljdbc42.jar:/data/nifi/nifi-1.1.0/./lib/tdgssconfig-15.10.00.09.jar:/data/nifi/nifi-1.1.0/./lib/terajdbc4-15.10.00.09.jar >> -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx8G -Xms4G >> -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=10000 >> -Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true >> -Djava.awt.headless=true -XX:+UseG1GC >> -Djava.protocol.handler.pkgs=sun.net.www.protocol >> -Dnifi.properties.file.path=/data/nifi/nifi-1.1.0/./conf/nifi.properties >> -Dnifi.bootstrap.listen.port=57107 -Dapp=NiFi >> -Dorg.apache.nifi.bootstrap.config.log.dir=/data/nifi/nifi-logs >> org.apache.nifi.NiFi > >> 2017-01-26 20:22:13,140 INFO [NiFi logging handler] org.apache.nifi.StdOut >> Listening for transport dt_socket at address: 10000 > >> 2017-01-26 20:22:14,340 INFO [NiFi Bootstrap Command Listener] >> org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for >> Bootstrap requests on port 38045 > >> 2017-01-26 20:38:22,626 INFO [main] o.a.n.b.NotificationServiceManager >> Successfully loaded the following 0 services: [] > >> 2017-01-26 20:38:22,633 INFO [main] org.apache.nifi.bootstrap.RunNiFi >> Registered no Notification Services for Notification Type NIFI_STARTED > >> 2017-01-26 20:38:22,633 INFO [main] org.apache.nifi.bootstrap.RunNiFi >> Registered no Notification Services for Notification Type NIFI_STOPPED > >> 2017-01-26 20:38:22,633 INFO [main] org.apache.nifi.bootstrap.RunNiFi >> Registered no Notification Services for Notification Type NIFI_DIED > >> 2017-01-26 20:38:32,663 ERROR [main] org.apache.nifi.bootstrap.Command >> Failed to send shutdown command to port 38045 due to >> java.net.SocketTimeoutException: Read timed out. Will kill the NiFi Process >> with PID 19744. > >> 2017-01-26 20:38:33,197 INFO [main] org.apache.nifi.bootstrap.RunNiFi >> Status File no longer exists. Will not restart NiFi > > > > <html><head><meta http-equiv="Content-Type" content="text/html > charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: > space; -webkit-line-break: after-white-space;" class="">Hi Peter,<div > class=""><br class=""></div><div class="">Can you provide a thread dump of > the process? You should be able to do this via the jcmd tool > [1]. </div><div class=""><br class=""></div><div class="">[1] <a > href="https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/tooldescr006.html#BABEHABG"; > class="">https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/tooldescr006.html#BABEHABG</a></div><div > class=""><br class=""><div class=""> > > <div style="letter-spacing: normal; orphans: auto; text-align: start; > text-indent: 0px; text-transform: none; white-space: normal; widows: auto; > word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; > -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" > class=""><div style="letter-spacing: normal; orphans: auto; text-align: > start; text-indent: 0px; text-transform: none; white-space: normal; widows: > auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: > break-word; -webkit-nbsp-mode: space; -webkit-line-break: > after-white-space;" class=""><div style="color: rgb(0, 0, 0);" class="">Andy > LoPresto</div><div style="color: rgb(0, 0, 0);" class=""><a > href="mailto:[email protected]"; > class="">[email protected]</a></div><div class=""><i class=""><font > color="#c0c0c0" class=""><a href="mailto:[email protected]"; > class="">[email protected]</a></font></i></div><div style="color: > rgb(0, 0, 0);" class="">PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 > BACE 3C6E F65B 2F7D EF69</div></div></div> > > </div> > <br class=""><div><blockquote type="cite" class=""><div class="">On Jan 26, > 2017, at 12:42 PM, Peter Wicks (pwicks) <<a > href="mailto:[email protected]"; class="">[email protected]</a>> > wrote:</div><br class="Apple-interchange-newline"><div class=""><div > class="WordSection1" style="page: WordSection1; font-family: Helvetica; > font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: > normal; letter-spacing: normal; orphans: auto; text-align: start; > text-indent: 0px; text-transform: none; white-space: normal; widows: auto; > word-spacing: 0px; -webkit-text-stroke-width: 0px;"><div style="margin: 0in > 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" > class="">I’m looking for help in troubleshooting my NiFi 1.1.0 > install. It’s been running stably for some time, but I restarted it > this morning when I deployed an updated custom NAR. Now it gets stuck at > startup, see logs at the end.<o:p class=""></o:p></div><div style="margin: > 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" > class="">There are no error messages, and the processes don’t die. The > process just seems to be hanging waiting for something.<o:p > class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; > font-family: Calibri, sans-serif;" class=""><o:p > class=""> </o:p></div><div style="margin: 0in 0in 0.0001pt 0.5in; > font-size: 11pt; font-family: Calibri, sans-serif; text-indent: -0.25in;" > class=""><span style="font-family: Symbol;" class=""><span class="">·<span > style="font-style: normal; font-variant-caps: normal; font-weight: normal; > font-size: 7pt; line-height: normal; font-family: 'Times New Roman';" > class=""> <span > class="Apple-converted-space"> </span></span></span></span>My first > thought was to try rolling back the modified nar, and even just removing the > nar all together since it was custom. Neither of these made any > difference.<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt > 0.5in; font-size: 11pt; font-family: Calibri, sans-serif; text-indent: > -0.25in;" class=""><span style="font-family: Symbol;" class=""><span > class="">·<span style="font-style: normal; font-variant-caps: normal; > font-weight: normal; font-size: 7pt; line-height: normal; font-family: > 'Times New Roman';" > class=""> <span > class="Apple-converted-space"> </span></span></span></span>I also tried > deleting the “work” folder, which has fixed nar versioning issues for me in > the past (not really related, but was worth a shot). This made no > difference.<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt > 0.5in; font-size: 11pt; font-family: Calibri, sans-serif; text-indent: > -0.25in;" class=""><span style="font-family: Symbol;" class=""><span > class="">·<span style="font-style: normal; font-variant-caps: normal; > font-weight: normal; font-size: 7pt; line-height: normal; font-family: > 'Times New Roman';" > class=""> <span > class="Apple-converted-space"> </span></span></span></span>NiFi is set > to start with java.arg.2=-Xms4G and java.arg.3=-Xmx8G, 22GB’s of free RAM > are available on the system (out of some 60GB’s total).<o:p > class=""></o:p></div><div style="margin: 0in 0in 0.0001pt 0.5in; font-size: > 11pt; font-family: Calibri, sans-serif; text-indent: -0.25in;" > class=""><span style="font-family: Symbol;" class=""><span class="">·<span > style="font-style: normal; font-variant-caps: normal; font-weight: normal; > font-size: 7pt; line-height: normal; font-family: 'Times New Roman';" > class=""> <span > class="Apple-converted-space"> </span></span></span></span>I’ve checked > running processes, and when I stop NiFi no rouge instances are left > running.<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt > 0.5in; font-size: 11pt; font-family: Calibri, sans-serif; text-indent: > -0.25in;" class=""><span style="font-family: Symbol;" class=""><span > class="">·<span style="font-style: normal; font-variant-caps: normal; > font-weight: normal; font-size: 7pt; line-height: normal; font-family: > 'Times New Roman';" > class=""> <span > class="Apple-converted-space"> </span></span></span></span>Since NiFi > gets stuck right around the JettyServer step I checked to see if any > processes were using port 8443. No other processes are using this port.<o:p > class=""></o:p></div><div style="margin: 0in 0in 0.0001pt 0.5in; font-size: > 11pt; font-family: Calibri, sans-serif; text-indent: -0.25in;" > class=""><span style="font-family: Symbol;" class=""><span class="">·<span > style="font-style: normal; font-variant-caps: normal; font-weight: normal; > font-size: 7pt; line-height: normal; font-family: 'Times New Roman';" > class=""> <span > class="Apple-converted-space"> </span></span></span></span>I thought > maybe a key file was being locked, but with NiFi off `lsof | grep nifi` > returns no locked files.<o:p class=""></o:p></div><div style="margin: 0in > 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" > class=""><o:p class=""> </o:p></div><div style="margin: 0in 0in > 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><b > class="">Nifi-app Log:<o:p class=""></o:p></b></div><div style="margin: 0in > 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" > class="">2017-01-26 20:23:43,359 INFO [main] org.eclipse.jetty.util.log > Logging initialized @90357ms<o:p class=""></o:p></div><div style="margin: > 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" > class="">2017-01-26 20:23:43,418 INFO [main] > org.apache.nifi.web.server.JettyServer Configuring Jetty for HTTPs on port: > 8443<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; > font-size: 11pt; font-family: Calibri, sans-serif;" class="">2017-01-26 > 20:23:43,691 INFO [main] org.apache.nifi.web.server.JettyServer Loading WAR: > /data/nifi/nifi-1.1.0/./work/nar/extensions/nifi-media-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-image-viewer-1.1.0.war > with context path set to /nifi-image-viewer-1.1.0<o:p > class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; > font-family: Calibri, sans-serif;" class="">2017-01-26 20:23:43,702 INFO > [main] org.apache.nifi.web.server.JettyServer Loading WAR: > /data/nifi/nifi-1.1.0/./work/nar/extensions/nifi-update-attribute-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-update-attribute-ui-1.1.0.war > with context path set to /nifi-update-attribute-ui-1.1.0<o:p > class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; > font-family: Calibri, sans-serif;" class="">2017-01-26 20:23:43,703 INFO > [main] org.apache.nifi.web.server.JettyServer Loading UI extension > [ProcessorConfiguration, /nifi-update-attribute-ui-1.1.0] for > [org.apache.nifi.processors.attributes.UpdateAttribute]<o:p > class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; > font-family: Calibri, sans-serif;" class="">2017-01-26 20:23:43,713 INFO > [main] org.apache.nifi.web.server.JettyServer Loading WAR: > /data/nifi/nifi-1.1.0/./work/nar/extensions/nifi-standard-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-standard-content-viewer-1.1.0.war > with context path set to /nifi-standard-content-viewer-1.1.0<o:p > class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; > font-family: Calibri, sans-serif;" class="">2017-01-26 20:23:43,723 INFO > [main] org.apache.nifi.web.server.JettyServer Loading WAR: > /data/nifi/nifi-1.1.0/./work/nar/extensions/nifi-standard-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-jolt-transform-json-ui-1.1.0.war > with context path set to /nifi-jolt-transform-json-ui-1.1.0<o:p > class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; > font-family: Calibri, sans-serif;" class="">2017-01-26 20:23:43,724 INFO > [main] org.apache.nifi.web.server.JettyServer Loading UI extension > [ProcessorConfiguration, /nifi-jolt-transform-json-ui-1.1.0] for > [org.apache.nifi.processors.standard.JoltTransformJSON]<o:p > class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; > font-family: Calibri, sans-serif;" class="">2017-01-26 20:23:43,729 INFO > [main] org.apache.nifi.web.server.JettyServer Loading WAR: > /data/nifi/nifi-1.1.0/./work/nar/framework/nifi-framework-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-web-ui-1.1.0.war > with context path set to /nifi<o:p class=""></o:p></div><div style="margin: > 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" > class="">2017-01-26 20:23:43,733 INFO [main] > org.apache.nifi.web.server.JettyServer Loading WAR: > /data/nifi/nifi-1.1.0/./work/nar/framework/nifi-framework-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-web-api-1.1.0.war > with context path set to /nifi-api<o:p class=""></o:p></div><div > style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, > sans-serif;" class="">2017-01-26 20:23:43,735 INFO [main] > org.apache.nifi.web.server.JettyServer Loading WAR: > /data/nifi/nifi-1.1.0/./work/nar/framework/nifi-framework-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-web-content-viewer-1.1.0.war > with context path set to /nifi-content-viewer<o:p class=""></o:p></div><div > style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, > sans-serif;" class="">2017-01-26 20:23:43,738 INFO [main] > org.apache.nifi.web.server.JettyServer Loading WAR: > /data/nifi/nifi-1.1.0/./work/nar/framework/nifi-framework-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-web-docs-1.1.0.war > with context path set to /nifi-docs<o:p class=""></o:p></div><div > style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, > sans-serif;" class="">2017-01-26 20:23:43,753 INFO [main] > org.apache.nifi.web.server.JettyServer Loading documents web app with > context path set to /nifi-docs<o:p class=""></o:p></div><div style="margin: > 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" > class="">2017-01-26 20:23:43,761 INFO [main] > org.apache.nifi.web.server.JettyServer Loading WAR: > /data/nifi/nifi-1.1.0/./work/nar/framework/nifi-framework-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-web-error-1.1.0.war > with context path set to /<o:p class=""></o:p></div><div style="margin: 0in > 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" > class="">2017-01-26 20:23:43,804 INFO [main] org.eclipse.jetty.server.Server > jetty-9.3.9.v20160517<o:p class=""></o:p></div><div style="margin: 0in 0in > 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" > class="">2017-01-26 20:23:44,748 INFO [main] > o.e.jetty.server.handler.ContextHandler Started > o.e.j.w.WebAppContext@4b511e61{/nifi-image-viewer-1.1.0,<a > href="file:///data/nifi/nifi-1.1.0/work/jetty/nifi-image-viewer-1.1.0.war/webapp/,AVAILABLE" > style="color: rgb(149, 79, 114); text-decoration: underline;" > class="">file:///data/nifi/nifi-1.1.0/work/jetty/nifi-image-viewer-1.1.0.war/webapp/,AVAILABLE</a>}{./work/nar/extensions/nifi-media-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-image-viewer-1.1.0.war}<o:p > class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; > font-family: Calibri, sans-serif;" class="">2017-01-26 20:23:46,566 INFO > [main] o.e.jetty.server.handler.ContextHandler Started > o.e.j.w.WebAppContext@26888c31{/nifi-update-attribute-ui-1.1.0,<a > href="file:///data/nifi/nifi-1.1.0/work/jetty/nifi-update-attribute-ui-1.1.0.war/webapp/,AVAILABLE" > style="color: rgb(149, 79, 114); text-decoration: underline;" > class="">file:///data/nifi/nifi-1.1.0/work/jetty/nifi-update-attribute-ui-1.1.0.war/webapp/,AVAILABLE</a>}{./work/nar/extensions/nifi-update-attribute-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-update-attribute-ui-1.1.0.war}<o:p > class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; > font-family: Calibri, sans-serif;" class="">2017-01-26 20:23:47,365 INFO > [main] o.e.jetty.server.handler.ContextHandler Started > o.e.j.w.WebAppContext@7c840fe3{/nifi-standard-content-viewer-1.1.0,<a > href="file:///data/nifi/nifi-1.1.0/work/jetty/nifi-standard-content-viewer-1.1.0.war/webapp/,AVAILABLE" > style="color: rgb(149, 79, 114); text-decoration: underline;" > class="">file:///data/nifi/nifi-1.1.0/work/jetty/nifi-standard-content-viewer-1.1.0.war/webapp/,AVAILABLE</a>}{./work/nar/extensions/nifi-standard-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-standard-content-viewer-1.1.0.war}<o:p > class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; > font-family: Calibri, sans-serif;" class="">2017-01-26 20:23:50,583 INFO > [main] o.e.jetty.server.handler.ContextHandler Started > o.e.j.w.WebAppContext@58b03029{/nifi-jolt-transform-json-ui-1.1.0,<a > href="file:///data/nifi/nifi-1.1.0/work/jetty/nifi-jolt-transform-json-ui-1.1.0.war/webapp/,AVAILABLE" > style="color: rgb(149, 79, 114); text-decoration: underline;" > class="">file:///data/nifi/nifi-1.1.0/work/jetty/nifi-jolt-transform-json-ui-1.1.0.war/webapp/,AVAILABLE</a>}{./work/nar/extensions/nifi-standard-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-jolt-transform-json-ui-1.1.0.war}<o:p > class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; > font-family: Calibri, sans-serif;" class="">2017-01-26 20:23:54,945 INFO > [main] /nifi No Spring WebApplicationInitializer types detected on > classpath<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; > font-size: 11pt; font-family: Calibri, sans-serif;" class="">2017-01-26 > 20:23:55,068 INFO [main] o.e.jetty.server.handler.ContextHandler Started > o.e.j.w.WebAppContext@2358443e{/nifi,<a > href="file:///data/nifi/nifi-1.1.0/work/jetty/nifi-web-ui-1.1.0.war/webapp/,AVAILABLE" > style="color: rgb(149, 79, 114); text-decoration: underline;" > class="">file:///data/nifi/nifi-1.1.0/work/jetty/nifi-web-ui-1.1.0.war/webapp/,AVAILABLE</a>}{./work/nar/framework/nifi-framework-nar-1.1.0.nar-unpacked/META-INF/bundled-dependencies/nifi-web-ui-1.1.0.war}<o:p > class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; > font-family: Calibri, sans-serif;" class="">2017-01-26 20:23:56,715 INFO > [main] /nifi-api No Spring WebApplicationInitializer types detected on > classpath<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; > font-size: 11pt; font-family: Calibri, sans-serif;" class="">2017-01-26 > 20:23:56,727 INFO [main] /nifi-api Initializing Spring root > WebApplicationContext<o:p class=""></o:p></div><div style="margin: 0in 0in > 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" > class="">2017-01-26 20:24:04,029 INFO [main] > o.a.nifi.properties.NiFiPropertiesLoader Determined default nifi.properties > path to be '/data/nifi/nifi-1.1.0/./conf/nifi.properties'<o:p > class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; > font-family: Calibri, sans-serif;" class="">2017-01-26 20:24:04,049 INFO > [main] o.a.nifi.properties.NiFiPropertiesLoader Determined default > nifi.properties path to be > '/data/nifi/nifi-1.1.0/./conf/nifi.properties'<o:p class=""></o:p></div><div > style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, > sans-serif;" class="">2017-01-26 20:24:04,064 INFO [main] > o.a.nifi.properties.NiFiPropertiesLoader Loaded 126 properties from > /data/nifi/nifi-1.1.0/./conf/nifi.properties<o:p class=""></o:p></div><div > style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, > sans-serif;" class="">2017-01-26 20:24:30,415 INFO [main] > o.a.nifi.util.FileBasedVariableRegistry Loaded 120 properties from system > properties and environment variables<o:p class=""></o:p></div><div > style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, > sans-serif;" class="">2017-01-26 20:24:30,415 INFO [main] > o.a.nifi.util.FileBasedVariableRegistry Loaded a total of 120 > properties. Including precedence overrides effective accessible > registry key size is 120<o:p class=""></o:p></div><div style="margin: 0in > 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" > class=""><o:p class=""> </o:p></div><div style="margin: 0in 0in > 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><b > class="">NiFi-bootstrap Log (including when I killed it after about 10 > minutes):<o:p class=""></o:p></b></div><div style="margin: 0in 0in 0.0001pt; > font-size: 11pt; font-family: Calibri, sans-serif;" class="">2017-01-26 > 20:22:12,986 INFO [main] org.apache.nifi.bootstrap.Command Starting Apache > NiFi...<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; > font-size: 11pt; font-family: Calibri, sans-serif;" class="">2017-01-26 > 20:22:12,986 INFO [main] org.apache.nifi.bootstrap.Command Working > Directory: /data/nifi/nifi-1.1.0<o:p class=""></o:p></div><div > style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, > sans-serif;" class="">2017-01-26 20:22:12,987 INFO [main] > org.apache.nifi.bootstrap.Command Command: java -classpath > /data/nifi/nifi-1.1.0/./conf:/data/nifi/nifi-1.1.0/./lib/logback-classic-1.1.3.jar:/data/nifi/nifi-1.1.0/./lib/logback-core-1.1.3.jar:/data/nifi/nifi-1.1.0/./lib/slf4j-api-1.7.12.jar:/data/nifi/nifi-1.1.0/./lib/jcl-over-slf4j-1.7.12.jar:/data/nifi/nifi-1.1.0/./lib/jul-to-slf4j-1.7.12.jar:/data/nifi/nifi-1.1.0/./lib/log4j-over-slf4j-1.7.12.jar:/data/nifi/nifi-1.1.0/./lib/nifi-api-1.1.0.jar:/data/nifi/nifi-1.1.0/./lib/nifi-framework-api-1.1.0.jar:/data/nifi/nifi-1.1.0/./lib/nifi-runtime-1.1.0.jar:/data/nifi/nifi-1.1.0/./lib/nifi-nar-utils-1.1.0.jar:/data/nifi/nifi-1.1.0/./lib/nifi-properties-1.1.0.jar:/data/nifi/nifi-1.1.0/./lib/nifi-documentation-1.1.0.jar:/data/nifi/nifi-1.1.0/./lib/bcprov-jdk15on-1.54.jar:/data/nifi/nifi-1.1.0/./lib/commons-lang3-3.4.jar:/data/nifi/nifi-1.1.0/./lib/ngdbc.jar:/data/nifi/nifi-1.1.0/./lib/ojdbc6.jar:/data/nifi/nifi-1.1.0/./lib/sqljdbc42.jar:/data/nifi/nifi-1.1.0/./lib/tdgssconfig-15.10.00.09.jar:/data/nifi/nifi-1.1.0/./lib/terajdbc4-15.10.00.09.jar > -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx8G -Xms4G > -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=10000 > -Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true > -Djava.awt.headless=true -XX:+UseG1GC > -Djava.protocol.handler.pkgs=sun.net.www.protocol > -Dnifi.properties.file.path=/data/nifi/nifi-1.1.0/./conf/nifi.properties > -Dnifi.bootstrap.listen.port=57107 -Dapp=NiFi > -Dorg.apache.nifi.bootstrap.config.log.dir=/data/nifi/nifi-logs > org.apache.nifi.NiFi<o:p class=""></o:p></div><div style="margin: 0in 0in > 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" > class="">2017-01-26 20:22:13,140 INFO [NiFi logging handler] > org.apache.nifi.StdOut Listening for transport dt_socket at address: > 10000<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; > font-size: 11pt; font-family: Calibri, sans-serif;" class="">2017-01-26 > 20:22:14,340 INFO [NiFi Bootstrap Command Listener] > org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for > Bootstrap requests on port 38045<o:p class=""></o:p></div><div > style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, > sans-serif;" class="">2017-01-26 20:38:22,626 INFO [main] > o.a.n.b.NotificationServiceManager Successfully loaded the following 0 > services: []<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; > font-size: 11pt; font-family: Calibri, sans-serif;" class="">2017-01-26 > 20:38:22,633 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no > Notification Services for Notification Type NIFI_STARTED<o:p > class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; > font-family: Calibri, sans-serif;" class="">2017-01-26 20:38:22,633 INFO > [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services > for Notification Type NIFI_STOPPED<o:p class=""></o:p></div><div > style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, > sans-serif;" class="">2017-01-26 20:38:22,633 INFO [main] > org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for > Notification Type NIFI_DIED<o:p class=""></o:p></div><div style="margin: 0in > 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" > class="">2017-01-26 20:38:32,663 ERROR [main] > org.apache.nifi.bootstrap.Command Failed to send shutdown command to port > 38045 due to java.net.SocketTimeoutException: Read timed out. Will kill the > NiFi Process with PID 19744.<o:p class=""></o:p></div><div style="margin: > 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" > class="">2017-01-26 20:38:33,197 INFO [main] > org.apache.nifi.bootstrap.RunNiFi Status File no longer exists. Will not > restart NiFi</div></div></div></blockquote></div><br > class=""></div></body></html> > > > >
