You’ll want to get thread dumps during the life of nifi to figure out the pattern of what is leaking threads. Often it will be around some tcp socket handling thread in something like sftp for example. Can be a config issue or a bug.
Thanks Joe On Tue, Nov 14, 2017 at 8:32 PM Peter Wicks (pwicks) <pwi...@micron.com> wrote: > I’ve been getting the error: > > > > 2017-11-15 09:22:24,959 ERROR [NiFi Web Server-566674] org.apache.nifi.NiFi > > java.lang.OutOfMemoryError: unable to create new native thread > > at java.lang.Thread.start0(Native Method) > > at java.lang.Thread.start(Thread.java:714) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool.startThreads(QueuedThreadPool.java:476) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool.access$200(QueuedThreadPool.java:49) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:657) > > at java.lang.Thread.run(Thread.java:745) > > > > My research online says this isn’t an out of memory error, but an out of > resources error. The system can’t support making new threads. > > > > This really sucks, because I can’t *su* to my service account, because > that requires a new thread… and if I have a bash session still open I can’t > kill NiFi because that requires creating a new thread… last time this > happened my Unix admin had to restart my server… > > > > The first time this happened we changed the limits for our service > account, here is my ulimit statement. > > > > # ulimit -a > > core file size (blocks, -c) 0 > > data seg size (kbytes, -d) unlimited > > scheduling priority (-e) 0 > > file size (blocks, -f) unlimited > > pending signals (-i) 96297 > > max locked memory (kbytes, -l) 64 > > max memory size (kbytes, -m) unlimited > > open files (-n) 50000 > > pipe size (512 bytes, -p) 8 > > POSIX message queues (bytes, -q) 819200 > > real-time priority (-r) 0 > > stack size (kbytes, -s) 8192 > > cpu time (seconds, -t) unlimited > > max user processes (-u) 50000 > > virtual memory (kbytes, -v) unlimited > > file locks (-x) unlimited > > > > It’s been about two weeks, and now the issue has come up again. Is this an > actual hardware limitation? > > > > If I run “*ps huH p <PID>| wc -l”* I can get a thread count, right now > it’s about 9900 threads. > > > > Thanks, > > Peter > > >
