On Dec 10, 2008, at 2:19 PM, Vinny wrote: > Running Resin 3.1.6 on centos linux 5, 2.0ghz proc , 2G ram. > I have a box that is running a bunch of virtual hosts configured in > resin and also connects to Apache 2.2.3 via mod_caucho. > > I am getting these in the logs and I'm wondering how I can read this > in order to figure out why my cpu load is being pegged so high
Can you do a 'ps' or 'top' to make sure that Resin is the overloading process? Since that thread dump doesn't show any active threads, if the Resin JVM has a high CPU, then something odd is happening. With the exception of the "CMS" thread, those are all idle Resin threads. So the load due to Resin should be zero based on that thread dump. -- Scott > > Thanks in advance: > > ==> /usr/local/resin/log/jvm-default.log <== > [21:36:00.000] StatService[] cpu-load=11.57 > [21:36:00.001] StatService[] dumping threads due to cpu load=11.57 > [21:36:00.003] Thread Dump: > [21:36:00.003] > [21:36:00.003] "Signal Dispatcher" id=5 RUNNABLE > [21:36:00.003] > [21:36:00.003] "Surrogate Locker Thread (CMS)" id=4 RUNNABLE > [21:36:00.003] > [21:36:00.003] "resin-pro-statistics" id=27 RUNNABLE > [21:36:00.003] at sun.management.ThreadImpl.getThreadInfo0 > (ThreadImpl.java) (native) > [21:36:00.003] at sun.management.ThreadImpl.getThreadInfo > (ThreadImpl.java:147) > [21:36:00.003] at > com.caucho.server.admin.ThreadDump.threadDumpImpl (ThreadDump.java:46) > [21:36:00.003] at com.caucho.server.admin.ThreadDump.dumpThreads > (ThreadDump.java:38) > [21:36:00.003] at com.caucho.server.admin.StatService.run > (StatService.java:135) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "RMI TCP Accept-0" id=10 RUNNABLE (in native) > [21:36:00.003] at java.net.PlainSocketImpl.socketAccept > (PlainSocketImpl.java) (native) > [21:36:00.003] at java.net.PlainSocketImpl.accept > (PlainSocketImpl.java:384) > [21:36:00.003] at java.net.ServerSocket.implAccept > (ServerSocket.java:453) > [21:36:00.003] at java.net.ServerSocket.accept (ServerSocket.java: > 421) > [21:36:00.003] at sun.rmi.transport.tcp.TCPTransport > $AcceptLoop.executeAcceptLoop (TCPTransport.java:369) > [21:36:00.003] at sun.rmi.transport.tcp.TCPTransport > $AcceptLoop.run (TCPTransport.java:341) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "hmux-127.0.0.1:6802-105" id=1859 RUNNABLE (in native) > [21:36:00.003] at com.caucho.vfs.JniSocketImpl.readNative > (JniSocketImpl.java) (native) > [21:36:00.003] at com.caucho.vfs.JniSocketImpl.read > (JniSocketImpl.java:279) > [21:36:00.003] at com.caucho.vfs.JniStream.readTimeout > (JniStream.java:87) > [21:36:00.003] at com.caucho.vfs.ReadStream.fillWithTimeout > (ReadStream.java:982) > [21:36:00.003] at > com.caucho.server.port.TcpConnection.waitForKeepalive > (TcpConnection.java:208) > [21:36:00.003] at com.caucho.server.port.TcpConnection.run > (TcpConnection.java:691) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.runTasks > (ThreadPool.java:721) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.run > (ThreadPool.java:643) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "hmux-127.0.0.1:6802-107" id=1881 RUNNABLE (in native) > [21:36:00.003] at com.caucho.vfs.JniServerSocketImpl.nativeAccept > (JniServerSocketImpl.java) (native) > [21:36:00.003] at com.caucho.vfs.JniServerSocketImpl.accept > (JniServerSocketImpl.java:199) > [21:36:00.003] at com.caucho.server.port.Port.accept (Port.java: > 1176) > [21:36:00.003] at com.caucho.server.port.TcpConnection.run > (TcpConnection.java:655) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.runTasks > (ThreadPool.java:721) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.run > (ThreadPool.java:643) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "hmux-127.0.0.1:6802-108" id=1871 RUNNABLE (in native) > [21:36:00.003] at com.caucho.vfs.JniServerSocketImpl.nativeAccept > (JniServerSocketImpl.java) (native) > [21:36:00.003] at com.caucho.vfs.JniServerSocketImpl.accept > (JniServerSocketImpl.java:199) > [21:36:00.003] at com.caucho.server.port.Port.accept (Port.java: > 1176) > [21:36:00.003] at com.caucho.server.port.TcpConnection.run > (TcpConnection.java:655) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.runTasks > (ThreadPool.java:721) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.run > (ThreadPool.java:643) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "hmux-127.0.0.1:6802-57" id=1897 RUNNABLE (in native) > [21:36:00.003] at com.caucho.vfs.JniServerSocketImpl.nativeAccept > (JniServerSocketImpl.java) (native) > [21:36:00.003] at com.caucho.vfs.JniServerSocketImpl.accept > (JniServerSocketImpl.java:199) > [21:36:00.003] at com.caucho.server.port.Port.accept (Port.java: > 1176) > [21:36:00.003] at com.caucho.server.port.TcpConnection.run > (TcpConnection.java:655) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.runTasks > (ThreadPool.java:721) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.run > (ThreadPool.java:643) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "hmux-127.0.0.1:6802-79" id=1899 RUNNABLE (in native) > [21:36:00.003] at com.caucho.vfs.JniServerSocketImpl.nativeAccept > (JniServerSocketImpl.java) (native) > [21:36:00.003] at com.caucho.vfs.JniServerSocketImpl.accept > (JniServerSocketImpl.java:199) > [21:36:00.003] at com.caucho.server.port.Port.accept (Port.java: > 1176) > [21:36:00.003] at com.caucho.server.port.TcpConnection.run > (TcpConnection.java:655) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.runTasks > (ThreadPool.java:721) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.run > (ThreadPool.java:643) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "hmux-127.0.0.1:6802-87" id=1898 RUNNABLE (in native) > [21:36:00.003] at com.caucho.vfs.JniServerSocketImpl.nativeAccept > (JniServerSocketImpl.java) (native) > [21:36:00.003] at com.caucho.vfs.JniServerSocketImpl.accept > (JniServerSocketImpl.java:199) > [21:36:00.003] at com.caucho.server.port.Port.accept (Port.java: > 1176) > [21:36:00.003] at com.caucho.server.port.TcpConnection.run > (TcpConnection.java:655) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.runTasks > (ThreadPool.java:721) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.run > (ThreadPool.java:643) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "http--8080-1" id=24 RUNNABLE (in native) > [21:36:00.003] at com.caucho.vfs.JniServerSocketImpl.nativeAccept > (JniServerSocketImpl.java) (native) > [21:36:00.003] at com.caucho.vfs.JniServerSocketImpl.accept > (JniServerSocketImpl.java:199) > [21:36:00.003] at com.caucho.server.port.Port.accept (Port.java: > 1176) > [21:36:00.003] at com.caucho.server.port.TcpConnection.run > (TcpConnection.java:655) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.runTasks > (ThreadPool.java:721) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.run > (ThreadPool.java:643) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "http--8080-3" id=22 RUNNABLE (in native) > [21:36:00.003] at com.caucho.vfs.JniServerSocketImpl.nativeAccept > (JniServerSocketImpl.java) (native) > [21:36:00.003] at com.caucho.vfs.JniServerSocketImpl.accept > (JniServerSocketImpl.java:199) > [21:36:00.003] at com.caucho.server.port.Port.accept (Port.java: > 1176) > [21:36:00.003] at com.caucho.server.port.TcpConnection.run > (TcpConnection.java:655) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.runTasks > (ThreadPool.java:721) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.run > (ThreadPool.java:643) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "http--8080-5" id=20 RUNNABLE (in native) > [21:36:00.003] at com.caucho.vfs.JniServerSocketImpl.nativeAccept > (JniServerSocketImpl.java) (native) > [21:36:00.003] at com.caucho.vfs.JniServerSocketImpl.accept > (JniServerSocketImpl.java:199) > [21:36:00.003] at com.caucho.server.port.Port.accept (Port.java: > 1176) > [21:36:00.003] at com.caucho.server.port.TcpConnection.run > (TcpConnection.java:655) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.runTasks > (ThreadPool.java:721) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.run > (ThreadPool.java:643) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "http--8080-7" id=18 RUNNABLE (in native) > [21:36:00.003] at com.caucho.vfs.JniServerSocketImpl.nativeAccept > (JniServerSocketImpl.java) (native) > [21:36:00.003] at com.caucho.vfs.JniServerSocketImpl.accept > (JniServerSocketImpl.java:199) > [21:36:00.003] at com.caucho.server.port.Port.accept (Port.java: > 1176) > [21:36:00.003] at com.caucho.server.port.TcpConnection.run > (TcpConnection.java:655) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.runTasks > (ThreadPool.java:721) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.run > (ThreadPool.java:643) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "http--8080-9" id=33 RUNNABLE (in native) > [21:36:00.003] at com.caucho.vfs.JniServerSocketImpl.nativeAccept > (JniServerSocketImpl.java) (native) > [21:36:00.003] at com.caucho.vfs.JniServerSocketImpl.accept > (JniServerSocketImpl.java:199) > [21:36:00.003] at com.caucho.server.port.Port.accept (Port.java: > 1176) > [21:36:00.003] at com.caucho.server.port.TcpConnection.run > (TcpConnection.java:655) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.runTasks > (ThreadPool.java:721) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.run > (ThreadPool.java:643) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "main" id=1 RUNNABLE (in native) > [21:36:00.003] at java.net.SocketInputStream.socketRead0 > (SocketInputStream.java) (native) > [21:36:00.003] at java.net.SocketInputStream.read > (SocketInputStream.java:129) > [21:36:00.003] at java.net.SocketInputStream.read > (SocketInputStream.java:182) > [21:36:00.003] at com.caucho.server.resin.Resin.waitForExit > (Resin.java:1265) > [21:36:00.003] at com.caucho.server.resin.Resin.main (Resin.java: > 1367) > [21:36:00.003] > [21:36:00.003] "resin-select-manager" id=28 RUNNABLE (in native) > [21:36:00.003] at > com.caucho.server.port.JniSelectManager.selectNative > (JniSelectManager.java) (native) > [21:36:00.003] at com.caucho.server.port.JniSelectManager.run > (JniSelectManager.java:289) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "Finalizer" id=3 WAITING > [21:36:00.003] waiting on java.lang.ref.ReferenceQueue > [EMAIL PROTECTED] > [21:36:00.003] at java.lang.Object.wait (Object.java) (native) > [21:36:00.003] at java.lang.ref.ReferenceQueue.remove > (ReferenceQueue.java:116) > [21:36:00.003] at java.lang.ref.ReferenceQueue.remove > (ReferenceQueue.java:132) > [21:36:00.003] at java.lang.ref.Finalizer$FinalizerThread.run > (Finalizer.java:159) > [21:36:00.003] > [21:36:00.003] "Reference Handler" id=2 WAITING > [21:36:00.003] waiting on [EMAIL PROTECTED] > [21:36:00.003] at java.lang.Object.wait (Object.java) (native) > [21:36:00.003] at java.lang.Object.wait (Object.java:485) > [21:36:00.003] at java.lang.ref.Reference$ReferenceHandler.run > (Reference.java:116) > [21:36:00.003] > [21:36:00.003] "resin-block-manager-writer" id=13 WAITING > [21:36:00.003] waiting on [EMAIL PROTECTED] > [21:36:00.003] at java.lang.Object.wait (Object.java) (native) > [21:36:00.003] at java.lang.Object.wait (Object.java:485) > [21:36:00.003] at com.caucho.db.store.BlockManager > $BlockManagerWriter.run (BlockManager.java:354) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "resin-destroy" id=31 WAITING > [21:36:00.003] waiting on com.caucho.server.resin.Resin > [EMAIL PROTECTED] > [21:36:00.003] at java.lang.Object.wait (Object.java) (native) > [21:36:00.003] at java.lang.Object.wait (Object.java:485) > [21:36:00.003] at com.caucho.server.resin.Resin$DestroyThread.run > (Resin.java:1774) > [21:36:00.003] > [21:36:00.003] "resin-1843" id=1868 TIMED_WAITING > [21:36:00.003] waiting on [EMAIL PROTECTED] > [21:36:00.003] at java.lang.Object.wait (Object.java) (native) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.runTasks > (ThreadPool.java:705) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.run > (ThreadPool.java:643) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "resin-1853" id=1878 TIMED_WAITING > [21:36:00.003] waiting on [EMAIL PROTECTED] > [21:36:00.003] at java.lang.Object.wait (Object.java) (native) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.runTasks > (ThreadPool.java:705) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.run > (ThreadPool.java:643) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "resin-1854" id=1879 TIMED_WAITING > [21:36:00.003] waiting on [EMAIL PROTECTED] > [21:36:00.003] at java.lang.Object.wait (Object.java) (native) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.runTasks > (ThreadPool.java:705) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.run > (ThreadPool.java:643) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "resin-1855" id=1880 TIMED_WAITING > [21:36:00.003] waiting on [EMAIL PROTECTED] > [21:36:00.003] at java.lang.Object.wait (Object.java) (native) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.runTasks > (ThreadPool.java:705) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.run > (ThreadPool.java:643) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "resin-1858" id=1883 TIMED_WAITING > [21:36:00.003] waiting on [EMAIL PROTECTED] > [21:36:00.003] at java.lang.Object.wait (Object.java) (native) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.runTasks > (ThreadPool.java:705) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.run > (ThreadPool.java:643) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "resin-1859" id=1884 TIMED_WAITING > [21:36:00.003] waiting on [EMAIL PROTECTED] > [21:36:00.003] at java.lang.Object.wait (Object.java) (native) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.runTasks > (ThreadPool.java:705) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.run > (ThreadPool.java:643) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "resin-1863" id=1888 TIMED_WAITING > [21:36:00.003] waiting on [EMAIL PROTECTED] > [21:36:00.003] at java.lang.Object.wait (Object.java) (native) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.runTasks > (ThreadPool.java:705) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.run > (ThreadPool.java:643) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "resin-1867" id=1892 TIMED_WAITING > [21:36:00.003] waiting on [EMAIL PROTECTED] > [21:36:00.003] at java.lang.Object.wait (Object.java) (native) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.runTasks > (ThreadPool.java:705) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.run > (ThreadPool.java:643) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "resin-1868" id=1893 TIMED_WAITING > [21:36:00.003] waiting on [EMAIL PROTECTED] > [21:36:00.003] at java.lang.Object.wait (Object.java) (native) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.runTasks > (ThreadPool.java:705) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.run > (ThreadPool.java:643) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "resin-1870" id=1895 TIMED_WAITING > [21:36:00.003] waiting on [EMAIL PROTECTED] > [21:36:00.003] at java.lang.Object.wait (Object.java) (native) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.runTasks > (ThreadPool.java:705) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.run > (ThreadPool.java:643) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "resin-1875" id=1900 TIMED_WAITING > [21:36:00.003] waiting on [EMAIL PROTECTED] > [21:36:00.003] at java.lang.Object.wait (Object.java) (native) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.runTasks > (ThreadPool.java:705) > [21:36:00.003] at com.caucho.util.ThreadPool$Item.run > (ThreadPool.java:643) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "resin-alarm" id=14 TIMED_WAITING > [21:36:00.003] at java.lang.Thread.sleep (Thread.java) (native) > [21:36:00.003] at com.caucho.util.Alarm$AlarmThread.run > (Alarm.java:586) > [21:36:00.003] > [21:36:00.003] "resin-ping" id=26 TIMED_WAITING > [21:36:00.003] at java.lang.Thread.sleep (Thread.java) (native) > [21:36:00.003] at com.caucho.server.admin.PingThread.run > (PingThread.java:334) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "resin-port-6802" id=29 TIMED_WAITING > [21:36:00.003] waiting on > [EMAIL PROTECTED] > [21:36:00.003] at java.lang.Object.wait (Object.java) (native) > [21:36:00.003] at com.caucho.server.port.Port.run (Port.java:1513) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "resin-port-8080" id=30 TIMED_WAITING > [21:36:00.003] waiting on [EMAIL PROTECTED] > [21:36:00.003] at java.lang.Object.wait (Object.java) (native) > [21:36:00.003] at com.caucho.server.port.Port.run (Port.java:1513) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "resin-thread-launcher" id=11 TIMED_WAITING > [21:36:00.003] waiting on com.caucho.util.ThreadPool > [EMAIL PROTECTED] > [21:36:00.003] at java.lang.Object.wait (Object.java) (native) > [21:36:00.003] at com.caucho.util.ThreadPool > $ThreadLauncher.startConnection (ThreadPool.java:828) > [21:36:00.003] at com.caucho.util.ThreadPool$ThreadLauncher.run > (ThreadPool.java:851) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > [21:36:00.003] "resin-thread-scheduler" id=12 TIMED_WAITING > [21:36:00.003] waiting on [EMAIL PROTECTED] > [21:36:00.003] at java.lang.Object.wait (Object.java) (native) > [21:36:00.003] at com.caucho.util.ThreadPool$ScheduleThread.run > (ThreadPool.java:895) > [21:36:00.003] at java.lang.Thread.run (Thread.java:619) > [21:36:00.003] > > <!-- the end --> > > > > > > The future is here. It's just not widely distributed yet. > -William Gibson > _______________________________________________ > resin-interest mailing list > resin-interest@caucho.com > http://maillist.caucho.com/mailman/listinfo/resin-interest _______________________________________________ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
