Kapil, if your server is running out of memory, then it is most likely the application rather than jetty itself. I would advise: + increasing the memory allocated to your production JVMs + running a test system with realistic load and profiling the applications memory usage both over time (for leaks) or just for memory hungry operations.
regards On 12 January 2018 at 05:38, kapil gupta <[email protected]> wrote: > Thanks everyone for your replies. > > Some more points into it: > > 1. @Silvio - As you mentioned, yes sometimes OOM killer invokes jetty > process to stop, but many of times it doesnt. > 2. @Greg - We are using cloud, and running on CentOS 7. We have > stopped updates due to some reasons. > 3. @Joakim - Yes, checked the thread dump and some times OOM killer > invokes jetty crash, sometimes I see below properties in dump file: > 1. > 2. Current thread (0x00007f18e01b9800): VMThread [stack: > 0x00007f18d33f4000,0x00007f18d34f5000] [id=11584] > 3. > 4. Stack: [0x00007f18d33f4000,0x00007f18d34f5000], > sp=0x00007f18d34f3140, free space=1020k > 5. Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, > C=native code) > 6. V [libjvm.so+0x9a320a] VMError::report_and_die()+0x2ea > 7. V [libjvm.so+0x498d3b] report_vm_out_of_memory(char const*, > int, unsigned long, char const*)+0x9b > 8. V [libjvm.so+0x82191e] os::Linux::commit_memory_impl(char*, > unsigned long, bool)+0xfe > 9. V [libjvm.so+0x821e69] os::pd_commit_memory(char*, unsigned > long, unsigned long, bool)+0x29 > 10. V [libjvm.so+0x81bb6a] os::commit_memory(char*, unsigned > long, unsigned long, bool)+0x2a > 11. V [libjvm.so+0x88d623] PSVirtualSpace::expand_by(unsigned > long)+0x53 > 12. V [libjvm.so+0x88e9f8] PSYoungGen::resize_generation(unsigned > long, unsigned long)+0xf8 > 13. V [libjvm.so+0x88db62] PSYoungGen::resize(unsigned long, > unsigned long)+0x22 > 14. V [libjvm.so+0x88af1d] PSScavenge::invoke_no_policy()+0xf3d > 15. V [libjvm.so+0x88b761] PSScavenge::invoke()+0x41 > 16. V [libjvm.so+0x843f40] > ParallelScavengeHeap::failed_mem_allocate(unsigned > long)+0x70 > 17. V [libjvm.so+0x9a4a97] VM_ParallelGCFailedAllocation:: > doit()+0x97 > 18. V [libjvm.so+0x9abf35] VM_Operation::evaluate()+0x55 > 19. V [libjvm.so+0x9aa2fa] VMThread::evaluate_operation( > VM_Operation*)+0xba > 20. V [libjvm.so+0x9aa67e] VMThread::loop()+0x1ce > 21. V [libjvm.so+0x9aaaf0] VMThread::run()+0x70 > 22. V [libjvm.so+0x8238c8] java_start(Thread*)+0x108 > > I do not have much of knowledge in to these configuration, need you guys > help. > Is this because of my server goes OutOfMemory, if yes then why it doesnt > show the memory occupied more in VisualVM when we check heap size > > > On Thu, Jan 11, 2018 at 3:00 PM, Silvio Bierman < > [email protected]> wrote: > >> Kapil, >> >> If you are running on Linux check the system log for kernel panics due to >> lack of memory. Linux will ruthlessly kill the process with highest memory >> load in such a situation. That has happened to me plenty of times and also >> left me wondering where my JVM process had gone. >> >> And to add to what Greg said: the System.exit thing can also happen in a >> library. I used a library once that would do a System.exit in some error >> situations killing my server now and then. That was a real puzzler. >> >> Cheers, >> >> Silvio >> >> >> >> On 10-01-18 12:50, kapil gupta wrote: >> >> Greg, >> >> Actually below things are observed: >> >> 1. No process of jetty server running >> 2. MySQL still runs >> 3. CPU usage shows data fine >> 4. RAM is also fine >> >> Not sure how can we isolate if issue could be because of JVM, can you >> please guide on setting some parameters or moniotring them. >> >> Thanks >> >> On Wed, Jan 10, 2018 at 1:40 PM, Greg Wilkins <[email protected]> wrote: >> >>> >>> Kapil, >>> >>> I'm sorry but you've not given us enough information to say anything. >>> What do you mean by "crashed"? If it is the JVM that stops, then that is >>> not a Jetty problem. Is there a stack trace? Is there any logging? Is the >>> process still running? Can any requests be served? Is the connector still >>> listening? Can you run with debug? Why do you think it is jetty and not >>> some other component? >>> >>> regards >>> >>> >>> >>> On 10 January 2018 at 05:37, kapil gupta <[email protected]> wrote: >>> >>>> We are using jetty server and when it is on load then the server is >>>> crashed without showing much of information. >>>> We have also put it on JMX. The CPU usage and memory looks fine, but >>>> still Jetty server crashes. Please let me know what parameters we should >>>> look for and how can we isolate issue which is causing jetty server crash. >>>> >>>> _______________________________________________ >>>> jetty-users mailing list >>>> [email protected] >>>> To change your delivery options, retrieve your password, or unsubscribe >>>> from this list, visit >>>> https://dev.eclipse.org/mailman/listinfo/jetty-users >>>> >>> >>> >>> >>> -- >>> Greg Wilkins <[email protected]> CTO http://webtide.com >>> >>> _______________________________________________ >>> jetty-users mailing list >>> [email protected] >>> To change your delivery options, retrieve your password, or unsubscribe >>> from this list, visit >>> https://dev.eclipse.org/mailman/listinfo/jetty-users >>> >> >> >> >> _______________________________________________ >> jetty-users mailing [email protected] >> To change your delivery options, retrieve your password, or unsubscribe from >> this list, visithttps://dev.eclipse.org/mailman/listinfo/jetty-users >> >> >> >> _______________________________________________ >> jetty-users mailing list >> [email protected] >> To change your delivery options, retrieve your password, or unsubscribe >> from this list, visit >> https://dev.eclipse.org/mailman/listinfo/jetty-users >> > > > _______________________________________________ > jetty-users mailing list > [email protected] > To change your delivery options, retrieve your password, or unsubscribe > from this list, visit > https://dev.eclipse.org/mailman/listinfo/jetty-users > -- Greg Wilkins <[email protected]> CTO http://webtide.com
_______________________________________________ jetty-users mailing list [email protected] To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.eclipse.org/mailman/listinfo/jetty-users
