Okay, have you tried enabling detailed report?
"*Get detail data:* To get a more detailed view of native memory usage,
start the JVM with command line option:
|-XX:NativeMemoryTracking=detail|. This will track exactly what methods
allocate the most memory. Enabling NMT will result in 5-10 percent JVM
performance drop and memory usage for NMT adds 2 words to all malloc
memory as malloc header. NMT memory usage is also tracked by NMT."
how long are you running the process and have you tried to force a
Full-GC (e.g. via JMX console).
Am 28.08.2015 10:50, schrieb Ciprian Hacman:
Out Java team tried to debug this issue using Java tools. The problem
we encountered is that this extra memory is not tracked by Java at all
(off heap memory). We realised this using NMT -
https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/tooldescr007.html.
There was not change in NMT tracked memory with and without
annotations enabled. Please see the output of "jcmd <PID>
VM.native_memory summary scale=MB" below:
Glassfish + annotations disabled (*551* MB RES on startup):
Total: reserved=2397MB, committed=*1159MB*
- Java Heap (reserved=1024MB, committed=1024MB)
(mmap: reserved=1024MB, committed=1024MB)
- Class (reserved=1077MB, committed=60MB)
(classes #10470)
(malloc=1MB #15133)
(mmap: reserved=1076MB, committed=59MB)
- Thread (reserved=23MB, committed=23MB)
(thread #80)
(stack: reserved=23MB, committed=23MB)
- Code (reserved=249MB, committed=28MB)
(malloc=5MB #8188)
(mmap: reserved=244MB, committed=23MB)
- GC (reserved=3MB, committed=3MB)
(mmap: reserved=3MB, committed=3MB)
- Internal (reserved=3MB, committed=3MB)
(malloc=3MB #13793)
- Symbol (reserved=14MB, committed=14MB)
(malloc=10MB #111702)
(arena=4MB #1)
- Native Memory Tracking (reserved=2MB, committed=2MB)
(tracking overhead=2MB)
Glassfish + annotations (*1909* MB RES on startup):
Total: reserved=2400MB, committed=*1162MB*
- Java Heap (reserved=1024MB, committed=1024MB)
(mmap: reserved=1024MB, committed=1024MB)
- Class (reserved=1079MB, committed=61MB)
(classes #10552)
(malloc=1MB #16204)
(mmap: reserved=1078MB, committed=59MB)
- Thread (reserved=22MB, committed=22MB)
(thread #74)
(stack: reserved=22MB, committed=22MB)
- Code (reserved=249MB, committed=30MB)
(malloc=6MB #8349)
(mmap: reserved=244MB, committed=25MB)
- GC (reserved=3MB, committed=3MB)
(mmap: reserved=3MB, committed=3MB)
- Internal (reserved=3MB, committed=3MB)
(malloc=3MB #13773)
- Symbol (reserved=14MB, committed=14MB)
(malloc=11MB #112818)
(arena=4MB #1)
- Native Memory Tracking (reserved=3MB, committed=3MB)
(tracking overhead=2MB)
- Arena Chunk (reserved=1MB, committed=1MB)
(malloc=1MB)
Apache + annotations (*1900* MB RES on startup):
Total: reserved=2400MB, committed=*1163MB*
- Java Heap (reserved=1024MB, committed=1024MB)
(mmap: reserved=1024MB, committed=1024MB)
- Class (reserved=1079MB, committed=61MB)
(classes #10679)
(malloc=1MB #16766)
(mmap: reserved=1078MB, committed=60MB)
- Thread (reserved=23MB, committed=23MB)
(thread #80)
(stack: reserved=23MB, committed=23MB)
- Code (reserved=249MB, committed=30MB)
(malloc=6MB #8492)
(mmap: reserved=244MB, committed=25MB)
- GC (reserved=3MB, committed=3MB)
(mmap: reserved=3MB, committed=3MB)
- Internal (reserved=3MB, committed=3MB)
(malloc=3MB #14043)
- Symbol (reserved=14MB, committed=14MB)
(malloc=11MB #113587)
(arena=4MB #1)
- Native Memory Tracking (reserved=3MB, committed=3MB)
(tracking overhead=2MB)
Thanks,
Ciprian
On Fri, Aug 28, 2015 at 11:20 AM, Christoph Läubrich
<[email protected] <mailto:[email protected]>> wrote:
In general you should use a proper tool (Eclipse MAT or JProfiler
for example) to investigate where how much memory is used.
Depending on your application, annotation parsing might involve a
lot of (extra) class-loading and information caching... but this
is just a wild guess!
Am 28.08.2015 10:03, schrieb Ciprian Hacman:
Hi,
Just wanted to make things a little more clear.
We changed from Apache to Glassfish, but we disabled the
"annotations" module to reduce the memory usage. We had to move
to Glassfish because the Apache engine did not work without
"annotations".
To make things easier to follow, here are some numbers for our
webapp (numbers represent RES memory):
1. apache + annotations - 2.5GB
2. apache + annotations disabled - not working
3. glassfish + annotations - 2.5GB
4. glassfish + annotations disabled - 750MB
Not sure why annotations make this huge difference for us and how
to debug further.
I see some other people bumped into similar issues (which gave us
this idea):
https://www.eclipse.org/forums/index.php/t/1067502/
If anyone has any idea here, we would be happy to give it a try.
Thanks,
Ciprian
_______________________________________________
jetty-users mailing list
[email protected] <mailto:[email protected]>
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users
_______________________________________________
jetty-users mailing list
[email protected]
To change your delivery options, retrieve your password, or unsubscribe from
this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users
_______________________________________________
jetty-users mailing list
[email protected]
To change your delivery options, retrieve your password, or unsubscribe from
this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users
