On 19/09/17 09:51, Peddi, Radhika (Radhika) wrote: > Hi, > > We have upgraded RHEL 7.2 to RHEL 7.4. After upgrade when we are running > performance testing of our application we are seeing below error in > catalina.out. > > java.lang.OutOfMemoryError: unable to create new native thread Exception in > Catalina.out after upgrading to RHEL 7.4 > > Attached is the thread dump.
Attachments get dropped by the list. Best to post the thread dump somewhere and link to it. > We are using Tomcat 7.0.69. That is quite old. There have been a few security vulnerabilities disclosed since then. If you haven't already, make sure none of them are a risk for you. > As per REDHAT we have increased tomcat5 user which used to run Tomcat process > in /etc/security/limits.d/20-nproc.conf like below tomcat5 ? That looks odd. I don't know anything about how RedHat packages Tomcat but I'd expect that to be tomcat7. It makes me wonder if there is some old Tomcat 5 configuration hanging around causing problems. > * soft nproc 4096 > root soft nproc unlimited > tomcat5 soft nproc 8192 > > All the processes are used by Tomcat. If we increase the value to 16K or 32K > all threads are consumed and they are in Waiting(Parking) state. > > "http-bio-8443-exec-56" #973 daemon prio=5 os_prio=0 tid=0x00007efc8c029800 > nid=0x10e7b waiting on condition [0x00007ef9bc7ee000] > java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x00000003c454c008> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:104) > at > org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:32) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) > at java.lang.Thread.run(Thread.java:745) That is perfectly normal. It is an idle thread waiting for a request to process. > "http-bio-8443-exec-55" #972 daemon prio=5 os_prio=0 tid=0x00007efc0401e000 > nid=0x10e7a waiting on condition [0x00007ef9bc82f000] > java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x00000003c454c008> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:104) > at > org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:32) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) > at java.lang.Thread.run(Thread.java:745) Likewise. > They are not getting release. As expected. > After some time Tomcat getting stopped automatically. By what? Or do you mean the OOME? > You may need the output of below from our setup. > > [root@iqsvtk05<mailto:root@iqsvtk05> ~]# rpm -qif > /opt/coreservices/tomcat-7.0.69/logs/ > Name : tomcat > Version : 7.0.69 > Release : AV10 > Architecture: x86_64 > Install Date: Fri 18 Aug 2017 01:41:54 AM MDT > Group : System Environment/Daemons > Size : 8468465 > License : Apache License, Version 2.0 > Signature : (none) > Source RPM : tomcat-7.0.69-AV10.src.rpm > Build Date : Fri 24 Jun 2016 01:59:37 AM MDT > Build Host : puiqx3650dev03.apac.avaya.com > Relocations : /usr/share > Packager : Avaya, Inc> Vendor : Apache Software Foundation > URL : http://tomcat.apache.org/ > Summary : Core Services Apache Jakarta Servlet/JSP server > Description : Servlet container for developing Web applications in Java. Used > as basis for the > Reference Implementation of the Servlet and JSP specificiations. This RPM > mirrors the layout used by official Tomcat distributions on all other OSes > and linux variants. If you want a FHS-based package - be warned that tomcat > may not work very well. (TODO: add more warnings !). If you still want > FHS-based package - please use jpackage.org or an RPM that is using the same > exact layout. > > Can you please help on this. When does the exception occur? Please provide the full thread dump. What start-up options have you configured? How much memory does the host have? Mark --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
