Sure Greg, I will check out for the points suggested. But one thing, why my VisualVM doesn show heap size hike, it hardly consumes 50-60% of allocated memory.
On Fri, Jan 12, 2018 at 3:34 PM, Greg Wilkins <[email protected]> wrote: > Kapil, > > if your server is running out of memory, then it is most likely the > application rather than jetty itself. > I would advise: > + increasing the memory allocated to your production JVMs > + running a test system with realistic load and profiling the > applications memory usage both over time (for leaks) or just for memory > hungry operations. > > regards > > > On 12 January 2018 at 05:38, kapil gupta <[email protected]> wrote: > >> Thanks everyone for your replies. >> >> Some more points into it: >> >> 1. @Silvio - As you mentioned, yes sometimes OOM killer invokes jetty >> process to stop, but many of times it doesnt. >> 2. @Greg - We are using cloud, and running on CentOS 7. We have >> stopped updates due to some reasons. >> 3. @Joakim - Yes, checked the thread dump and some times OOM killer >> invokes jetty crash, sometimes I see below properties in dump file: >> 1. >> 2. Current thread (0x00007f18e01b9800): VMThread [stack: >> 0x00007f18d33f4000,0x00007f18d34f5000] [id=11584] >> 3. >> 4. Stack: [0x00007f18d33f4000,0x00007f18d34f5000], >> sp=0x00007f18d34f3140, free space=1020k >> 5. Native frames: (J=compiled Java code, j=interpreted, Vv=VM >> code, C=native code) >> 6. V [libjvm.so+0x9a320a] VMError::report_and_die()+0x2ea >> 7. V [libjvm.so+0x498d3b] report_vm_out_of_memory(char const*, >> int, unsigned long, char const*)+0x9b >> 8. V [libjvm.so+0x82191e] os::Linux::commit_memory_impl(char*, >> unsigned long, bool)+0xfe >> 9. V [libjvm.so+0x821e69] os::pd_commit_memory(char*, unsigned >> long, unsigned long, bool)+0x29 >> 10. V [libjvm.so+0x81bb6a] os::commit_memory(char*, unsigned >> long, unsigned long, bool)+0x2a >> 11. V [libjvm.so+0x88d623] PSVirtualSpace::expand_by(unsigned >> long)+0x53 >> 12. V [libjvm.so+0x88e9f8] PSYoungGen::resize_generation(unsigned >> long, unsigned long)+0xf8 >> 13. V [libjvm.so+0x88db62] PSYoungGen::resize(unsigned long, >> unsigned long)+0x22 >> 14. V [libjvm.so+0x88af1d] PSScavenge::invoke_no_policy()+0xf3d >> 15. V [libjvm.so+0x88b761] PSScavenge::invoke()+0x41 >> 16. V [libjvm.so+0x843f40] >> ParallelScavengeHeap::failed_mem_allocate(unsigned >> long)+0x70 >> 17. V [libjvm.so+0x9a4a97] VM_ParallelGCFailedAllocation >> ::doit()+0x97 >> 18. V [libjvm.so+0x9abf35] VM_Operation::evaluate()+0x55 >> 19. V [libjvm.so+0x9aa2fa] VMThread::evaluate_operation( >> VM_Operation*)+0xba >> 20. V [libjvm.so+0x9aa67e] VMThread::loop()+0x1ce >> 21. V [libjvm.so+0x9aaaf0] VMThread::run()+0x70 >> 22. V [libjvm.so+0x8238c8] java_start(Thread*)+0x108 >> >> I do not have much of knowledge in to these configuration, need you guys >> help. >> Is this because of my server goes OutOfMemory, if yes then why it doesnt >> show the memory occupied more in VisualVM when we check heap size >> >> >> On Thu, Jan 11, 2018 at 3:00 PM, Silvio Bierman < >> [email protected]> wrote: >> >>> Kapil, >>> >>> If you are running on Linux check the system log for kernel panics due >>> to lack of memory. Linux will ruthlessly kill the process with highest >>> memory load in such a situation. That has happened to me plenty of times >>> and also left me wondering where my JVM process had gone. >>> >>> And to add to what Greg said: the System.exit thing can also happen in a >>> library. I used a library once that would do a System.exit in some error >>> situations killing my server now and then. That was a real puzzler. >>> >>> Cheers, >>> >>> Silvio >>> >>> >>> >>> On 10-01-18 12:50, kapil gupta wrote: >>> >>> Greg, >>> >>> Actually below things are observed: >>> >>> 1. No process of jetty server running >>> 2. MySQL still runs >>> 3. CPU usage shows data fine >>> 4. RAM is also fine >>> >>> Not sure how can we isolate if issue could be because of JVM, can you >>> please guide on setting some parameters or moniotring them. >>> >>> Thanks >>> >>> On Wed, Jan 10, 2018 at 1:40 PM, Greg Wilkins <[email protected]> wrote: >>> >>>> >>>> Kapil, >>>> >>>> I'm sorry but you've not given us enough information to say anything. >>>> What do you mean by "crashed"? If it is the JVM that stops, then that is >>>> not a Jetty problem. Is there a stack trace? Is there any logging? Is the >>>> process still running? Can any requests be served? Is the connector still >>>> listening? Can you run with debug? Why do you think it is jetty and not >>>> some other component? >>>> >>>> regards >>>> >>>> >>>> >>>> On 10 January 2018 at 05:37, kapil gupta <[email protected]> >>>> wrote: >>>> >>>>> We are using jetty server and when it is on load then the server is >>>>> crashed without showing much of information. >>>>> We have also put it on JMX. The CPU usage and memory looks fine, but >>>>> still Jetty server crashes. Please let me know what parameters we should >>>>> look for and how can we isolate issue which is causing jetty server crash. >>>>> >>>>> _______________________________________________ >>>>> jetty-users mailing list >>>>> [email protected] >>>>> To change your delivery options, retrieve your password, or >>>>> unsubscribe from this list, visit >>>>> https://dev.eclipse.org/mailman/listinfo/jetty-users >>>>> >>>> >>>> >>>> >>>> -- >>>> Greg Wilkins <[email protected]> CTO http://webtide.com >>>> >>>> _______________________________________________ >>>> jetty-users mailing list >>>> [email protected] >>>> To change your delivery options, retrieve your password, or unsubscribe >>>> from this list, visit >>>> https://dev.eclipse.org/mailman/listinfo/jetty-users >>>> >>> >>> >>> >>> _______________________________________________ >>> jetty-users mailing [email protected] >>> To change your delivery options, retrieve your password, or unsubscribe >>> from this list, visithttps://dev.eclipse.org/mailman/listinfo/jetty-users >>> >>> >>> >>> _______________________________________________ >>> jetty-users mailing list >>> [email protected] >>> To change your delivery options, retrieve your password, or unsubscribe >>> from this list, visit >>> https://dev.eclipse.org/mailman/listinfo/jetty-users >>> >> >> >> _______________________________________________ >> jetty-users mailing list >> [email protected] >> To change your delivery options, retrieve your password, or unsubscribe >> from this list, visit >> https://dev.eclipse.org/mailman/listinfo/jetty-users >> > > > > -- > Greg Wilkins <[email protected]> CTO http://webtide.com > > _______________________________________________ > jetty-users mailing list > [email protected] > To change your delivery options, retrieve your password, or unsubscribe > from this list, visit > https://dev.eclipse.org/mailman/listinfo/jetty-users >
_______________________________________________ jetty-users mailing list [email protected] To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.eclipse.org/mailman/listinfo/jetty-users
