Re: Service killed by signal 9
Coty, I will try and set it back to 256k, I am using centos 7 on the latest kernel. It is an older PC and has 4gb of memory but only reports 3gb. I can see from top that the memory is not good (gkreallm reports 800mb free) top - 08:35:40 up 2:25, 1 user, load average: 0.50, 0.18, 0.14 Tasks: 258 total, 1 running, 257 sleeping, 0 stopped, 0 zombie %Cpu(s): 13.4 us, 2.2 sy, 0.0 ni, 84.1 id, 0.2 wa, 0.0 hi, 0.2 si, 0.0 st KiB Mem : 3036956 total, 236104 free, 2216104 used, 584748 buff/cache KiB Swap: 2113532 total, 1912644 free, 200888 used. 516460 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1292 user20 03435632 476088 6560 S 0.3 15.7 4:43.96 jsvc Cheers Greg On 10 November 2017 at 15:06, Coty Sutherland wrote: > On Fri, Nov 10, 2017 at 4:31 AM, Greg Huber wrote: > > Since switching to jsvc, randomly I am getting tomcat restarting, looking > > at the logs I see that the jsvc is using alot of memory total-vm: and > > being killed by the system. > > > > Nov 9 13:11:11 prodbox kernel: Out of memory: Kill process 1287 (jsvc) > > score 121 or sacrifice child > > Nov 9 13:11:11 prodbox kernel: Killed process 1287 (jsvc) > > total-vm:3453120kB, anon-rss:378280kB, file-rss:0kB, shmem-rss:0kB > > > > I am using the below on properties on the startup (from previous version > > statup scripts): > > > > -Xms256M -Xmx768m -Xss1280k -XX:+UseParallelGC -XX:MaxGCPauseMillis=1500 > > -XX:GCTimeRatio=9 -server -XX:+DisableExplicitGC > > > > (I replaced -Xss256k with -Xss1280k to stop crashing see > > https://issues.apache.org/jira/browse/DAEMON-365) > > The root cause for that broke a lot of things, so I imagine your OS > has fixed the kernel problem by now. If you tell me which OS I can try > and see if it's fixed for you. I'd suggest that you try reducing your > Xss back to 256k and see if that shrinks your process size enough to > keep it off of oom-killer's radar. > > > Is there a way to limit the total-vm or find out why its got so big? > > -XX:+DisableExplicitGC ?? > > > > Standard tomcat install with apache/modjk. > > > > Cheers Greg > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: Service killed by signal 9
On Fri, Nov 10, 2017 at 4:31 AM, Greg Huber wrote: > Since switching to jsvc, randomly I am getting tomcat restarting, looking > at the logs I see that the jsvc is using alot of memory total-vm: and > being killed by the system. > > Nov 9 13:11:11 prodbox kernel: Out of memory: Kill process 1287 (jsvc) > score 121 or sacrifice child > Nov 9 13:11:11 prodbox kernel: Killed process 1287 (jsvc) > total-vm:3453120kB, anon-rss:378280kB, file-rss:0kB, shmem-rss:0kB > > I am using the below on properties on the startup (from previous version > statup scripts): > > -Xms256M -Xmx768m -Xss1280k -XX:+UseParallelGC -XX:MaxGCPauseMillis=1500 > -XX:GCTimeRatio=9 -server -XX:+DisableExplicitGC > > (I replaced -Xss256k with -Xss1280k to stop crashing see > https://issues.apache.org/jira/browse/DAEMON-365) The root cause for that broke a lot of things, so I imagine your OS has fixed the kernel problem by now. If you tell me which OS I can try and see if it's fixed for you. I'd suggest that you try reducing your Xss back to 256k and see if that shrinks your process size enough to keep it off of oom-killer's radar. > Is there a way to limit the total-vm or find out why its got so big? > -XX:+DisableExplicitGC ?? > > Standard tomcat install with apache/modjk. > > Cheers Greg - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Service killed by signal 9
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Konstantin, On 11/10/17 8:46 AM, Konstantin Kolinko wrote: > 2017-11-10 12:31 GMT+03:00 Greg Huber : >> Since switching to jsvc, randomly I am getting tomcat restarting, >> looking at the logs I see that the jsvc is using alot of memory >> total-vm: and being killed by the system. >> >> Nov 9 13:11:11 prodbox kernel: Out of memory: Kill process 1287 >> (jsvc) score 121 or sacrifice child Nov 9 13:11:11 prodbox >> kernel: Killed process 1287 (jsvc) total-vm:3453120kB, >> anon-rss:378280kB, file-rss:0kB, shmem-rss:0kB >> >> I am using the below on properties on the startup (from previous >> version statup scripts): >> >> -Xms256M -Xmx768m -Xss1280k -XX:+UseParallelGC >> -XX:MaxGCPauseMillis=1500 -XX:GCTimeRatio=9 -server >> -XX:+DisableExplicitGC >> >> (I replaced -Xss256k with -Xss1280k to stop crashing see >> https://issues.apache.org/jira/browse/DAEMON-365) > > The links in that issue go to a thread that explains that the issue > is actually caused a kernel bug. It was reported & diagnosed in > June 2017, thus should it have already been fixed? > > 1,2 Mb of stack for each thread mean that for 200 threads you will > need 240 Mb of memory just for thread stacks. Most of that is > wasted memory: You do not need to increase -Xss unless you are > hitting StackOverflowError. > >> Is there a way to limit the total-vm or find out why its got so >> big? -XX:+DisableExplicitGC ?? >> >> Standard tomcat install with apache/modjk. > > https://wiki.apache.org/tomcat/FAQ/Linux_Unix#Q5 > > In Tomcat Manager web application there is "Server Status" page. > It has a table that shows current memory consumption, as reported > by Java. The values can also be queried via JMX (e.g. with > jconsole application). Linux oom will use the actual process's used memory to compute its killability, so the Java-reported memory usage won't tell the whole stor y. Greg, how much total memory does the server actually have? Do you have other services running on it as well? If you run low on memory, Linux will simply kill your process. There are ways to minimize the likelihood that your process will be killed, but if it's the only way to free-up memory that the kernel is convinced it needs, well, then your process is simply going to die. One way to reduce your memory usage would be to reduce the heap size you are using. Do you need that much heap? It's also possible that you have a native memory leak somewhere, possibly caused by code outside of your control. Do you have any native components involved that don't ship with the JVM (or jsvc)? I'm unaware of any memory issues with jsvc, but I'm not super familiar with it. - -chris -BEGIN PGP SIGNATURE- Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAloFvOIACgkQHPApP6U8 pFhPdBAAyvgvwEovYQXGpDDVZ2c3JwkzyKjQrWboW9wYjKsPTjy+xGiDqVfhxCLC ukbQL6spVYlqc1y5N+5i2iyqX+JRKa7OA3unvFRDrpxJDOfJmDqwBavJ8xlpl9mX v/E/WMPXAgZLYK05aPCsul7/ewavahADi2vwt2SQgUE9ARrV/qN7QXYPaa6XwNjM SSb35dWK0k9pDCl2OrKfFOEgm3nsY+P6FKx29k1z/8sAYBYQJRC/slBboqsj7uzj X4DFThtKvkiPGRSjzEeDmkthMN7CFz1QFHa3NaPDkj5Zjv7mUfLZRdHB0ETlQdvq Zx2md/OZBw/EoE5j+K/C9yfK6qwcdgHX2zsHdwbweC/QZJrFVaf9VxfaXmeZavLX KXYRhk3W1R93/QigXVjPxMod1ToiLjSzGPG34F1jkC/FBj7X359dZokL6pLl6hDw h+6M5rUxWevLVdKg9gWX8peT/jxk4P7KInWjdH2oI4Rl40vMbSPA23qWkaVTbHfG WWsdDMWrmxNW2e+zzNRCikjMT6ksWas18tzgn9KVWy8cau8yy208d0opSUloUZht 2LwUhrEKmbZjQywyyq/7MyXTIwMeAg6y3xJ04al6ZBp7YnrmW/zhay4yaT2ZBJ8R E1PnVf1MPnOAIHIev562CadG/FiveQ0n94/7KMiQf81YgIiFQeo= =AXyl -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Service killed by signal 9
2017-11-10 12:31 GMT+03:00 Greg Huber : > Since switching to jsvc, randomly I am getting tomcat restarting, looking > at the logs I see that the jsvc is using alot of memory total-vm: and > being killed by the system. > > Nov 9 13:11:11 prodbox kernel: Out of memory: Kill process 1287 (jsvc) > score 121 or sacrifice child > Nov 9 13:11:11 prodbox kernel: Killed process 1287 (jsvc) > total-vm:3453120kB, anon-rss:378280kB, file-rss:0kB, shmem-rss:0kB > > I am using the below on properties on the startup (from previous version > statup scripts): > > -Xms256M -Xmx768m -Xss1280k -XX:+UseParallelGC -XX:MaxGCPauseMillis=1500 > -XX:GCTimeRatio=9 -server -XX:+DisableExplicitGC > > (I replaced -Xss256k with -Xss1280k to stop crashing see > https://issues.apache.org/jira/browse/DAEMON-365) The links in that issue go to a thread that explains that the issue is actually caused a kernel bug. It was reported & diagnosed in June 2017, thus should it have already been fixed? 1,2 Mb of stack for each thread mean that for 200 threads you will need 240 Mb of memory just for thread stacks. Most of that is wasted memory: You do not need to increase -Xss unless you are hitting StackOverflowError. > Is there a way to limit the total-vm or find out why its got so big? > -XX:+DisableExplicitGC ?? > > Standard tomcat install with apache/modjk. https://wiki.apache.org/tomcat/FAQ/Linux_Unix#Q5 In Tomcat Manager web application there is "Server Status" page. It has a table that shows current memory consumption, as reported by Java. The values can also be queried via JMX (e.g. with jconsole application). Best regards, Konstantin Kolinko - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Service killed by signal 9
Since switching to jsvc, randomly I am getting tomcat restarting, looking at the logs I see that the jsvc is using alot of memory total-vm: and being killed by the system. Nov 9 13:11:11 prodbox kernel: Out of memory: Kill process 1287 (jsvc) score 121 or sacrifice child Nov 9 13:11:11 prodbox kernel: Killed process 1287 (jsvc) total-vm:3453120kB, anon-rss:378280kB, file-rss:0kB, shmem-rss:0kB I am using the below on properties on the startup (from previous version statup scripts): -Xms256M -Xmx768m -Xss1280k -XX:+UseParallelGC -XX:MaxGCPauseMillis=1500 -XX:GCTimeRatio=9 -server -XX:+DisableExplicitGC (I replaced -Xss256k with -Xss1280k to stop crashing see https://issues.apache.org/jira/browse/DAEMON-365) Is there a way to limit the total-vm or find out why its got so big? -XX:+DisableExplicitGC ?? Standard tomcat install with apache/modjk. Cheers Greg