On 2/5/2016 12:24 AM, Thomas Stüfe wrote:
Hi Timur,
On Fri, Feb 5, 2016 at 7:53 AM, Timur Akhmadeev
<timur.akhmad...@gmail.com <mailto:timur.akhmad...@gmail.com>> wrote:
Thanks Thomas and Jon!
I think I've opened the page Thomas has referenced more than once
before asking my question but somehow I missed this paragraph
which is the answer :)
If |UseCompressedOops| is turned on and
|UseCompressedClassesPointers| is used, then two logically
different areas of native memory are used for class metadata.
|UseCompressedClassPointers| uses a 32-bit offset to represent
the class pointer in a 64-bit process as does
|UseCompressedOops| for Java object references. A region is
allocated for these compressed class pointers (the 32-bit
offsets). The size of the region can be set with
|CompressedClassSpaceSize| and is 1 gigabyte (GB) by default.
The space for the compressed class pointers is reserved as
space allocated by |mmap| at initialization and committed as
needed. The|MaxMetaspaceSize| applies to the sum of the
committed compressed class space and the space for the other
class metadata.
It's a bit unnatural to call Compressed Class pointers part of the
Metaspace and at the same time allow its part to be set bigger
than the whole space while not allowing Compressed Class pointers
(hopefully) to grow beyond MaxMetaspaceSize.
I agree. Maybe it is just me, but I also have a hard time
understanding the relationship between those two parameters.
See also the mail thread here:
http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2015-June/013797.html
.
I do not know the history, so I only can guess that the permgen
removal came first, and that at first it was just a "simple"
metaspace. But with moving out class data from the heap we lost the
ability to address them with compressed oops (32bit pointers) and this
feature was reintroduced in the form of compressed class pointers,
which made the implementation more complicated and now we have those
two switches.
When there was a perm gen, the perm gen and the rest of the heap were a
contiguous
block of addressed with one base address (heap-base). With compressed
oops (COOPS)
object pointers (including pointers to metadata in the perm gen) were
32bit offsets from
heap-base. When metadata was moved to native memory, the address space
for the
metadata was reserved and committed as it was needed (so there are
multiple chunks
of addresses for metadata and so not a single metadata base address
(meta-base)).
For compressed class pointers we needed a single base address
(class-base) again
so we could use offsets from class-base for the class pointers. So we
reserve a
single contiguous address space for the compressed class pointers (separate
from the other metadata) and commit from that reserved space as it is
needed.
But this is only a guess from my side. Whatever the reasons, I think
the parameters are confusing, especially as the sizes have two
different meanings - reserved size in case of
CompressedClassSpaceSize, committed size in case of MaxMetaSpaceSize.
We're trying to manage data that can be accessed differently (full 64bit
address and 32bit offset)
so need to have different spaces. CompressedClassSpaceSize is the size
we reserve at
initialization. The part of that which is committed plus the amount of
committed space for
the other metadata is limited by MaxMetaSpaceSize.
I also do not know if CompressedClassSpaceSize is a hard limit - what
happens when we fill compressed class space with more than that? Will
it expand or throw an OOM?
CompressedClassSpaceSize is a hard limit and will limit the number of
classes that can be
loaded. There are plans to fix that.
Jon
Kind Regards, Thomas
On Thu, Feb 4, 2016 at 10:02 PM, Jon Masamitsu
<jon.masami...@oracle.com <mailto:jon.masami...@oracle.com>> wrote:
On 02/04/2016 06:41 AM, Thomas Stüfe wrote:
Hi Timur,
I believe you see the space for the compressed class data
(Compressed class space), which is by default 1GB and
controlled by -XX:CompressedClassSpaceSize . I think this
area is semantically one subarea of the metaspace, the other
being non-class data.
I am unsure about the difference between MaxMetaSpaceSize and
CompressedClassSpaceSize, but what I believe is that:
- MaxMetaSpaceSize limits the amount of total committed (not
reserved) space
- CompressedClassSpaceSize sets the amount of space to be
reserved (not committed) for compressed classes only.
This would fit your NMT output, which says:
"Class (reserved=1142808KB, committed=106520KB)"
so, ~100M are committed. With a MaxMetaSpaceSize=256M - what
you specified - you would not be able to commit the whole
1GB, but hit the limit far earlier.
See also:
https://docs.oracle.com/javase/8/docs/technotes/guides/vm/gctuning/considerations.htm
(the section about UseCompressedClassesPointers)
Please take this with a grain of salt. I am not sure I
understand this correctly and would value input from other
developers.
Thomas,
No disagreement from me. I think you described it
correctly. Note that if you turn off UseCompressedClassPointers,
there will not be a separate space for the class metadata.
Jon
Kind Regards, Thomas
On Thu, Feb 4, 2016 at 2:13 PM, Timur Akhmadeev
<timur.akhmad...@gmail.com
<mailto:timur.akhmad...@gmail.com>> wrote:
Hi,
I'm trying to figure out which parts of memory Hotspot
can allocate in HugePages on Linux (appears to be heap
and code cache by default, plus Metaspace with
-XX:+UseLargePagesInMetaspace) and noticed something I
can't explain.
There's 1G allocation on the OS level in 4K pages that is
apparently attributed to Class metadata, although
Metaspace is maxed at a value much lower than 1G.
Question: why?
[oracle@oel6u7 bin]$ grep -B 11 'KernelPageSize: 4 kB'
/proc/18906/smaps | grep '^Size:' | head
Size: 4 kB
Size: 4 kB
Size: 2764 kB
Size: 12288 kB
Size: 1036288 kB
Size: 128 kB
Size: 4 kB
Size: 4 kB
Size: 4 kB
Size: 8 kB
[oracle@oel6u7 bin]$ /home/oracle/jdk1.8.0_72/bin/jcmd
18906 VM.native_memory summary
18906:
Native Memory Tracking:
Total: reserved=2069703KB, committed=1032559KB
- Java Heap (reserved=786432KB, committed=786432KB)
(mmap: reserved=786432KB,
committed=786432KB)
- Class (reserved=1142808KB, committed=106520KB)
(classes #13624)
(malloc=2072KB #22966)
(mmap: reserved=1140736KB,
committed=104448KB)
- Thread (reserved=56009KB, committed=56009KB)
(thread #55)
(stack: reserved=55512KB, committed=55512KB)
(malloc=178KB #272)
(arena=319KB #108)
- Code (reserved=55407KB, committed=55407KB)
(malloc=5487KB #7167)
(mmap: reserved=49920KB, committed=49920KB)
- GC (reserved=2600KB, committed=1748KB)
(malloc=32KB #355)
(mmap: reserved=2568KB, committed=1716KB)
- Compiler (reserved=213KB, committed=213KB)
(malloc=82KB #331)
(arena=131KB #3)
- Internal (reserved=4921KB, committed=4917KB)
(malloc=4885KB #16137)
(mmap: reserved=36KB, committed=32KB)
- Symbol (reserved=17518KB, committed=17518KB)
(malloc=14377KB #172159)
(arena=3141KB #1)
- Native Memory Tracking (reserved=3617KB,
committed=3617KB)
(malloc=149KB #2329)
(tracking overhead=3468KB)
- Arena Chunk (reserved=178KB, committed=178KB)
(malloc=178KB)
[oracle@oel6u7 bin]$ /home/oracle/jdk1.8.0_72/bin/jcmd
18906 VM.native_memory detail | grep -A 10 'for Class'
[0x0000000100000000 - 0x0000000140000000] reserved
1048576KB for Class from
[0x00007f71950e2ad2] ReservedSpace::initialize(unsigned
long, unsigned long, bool, char*, unsigned long, bool)+0xc2
[0x00007f71950e2d0b]
ReservedSpace::ReservedSpace(unsigned long, unsigned
long, bool, char*, unsigned long)+0x1b
[0x00007f7194ec8300]
Metaspace::allocate_metaspace_compressed_klass_ptrs(char*, unsigned
char*)+0x40
[0x00007f7194eca7df] Metaspace::global_initialize()+0x4cf
[0x0000000100000000 - 0x0000000100c00000] committed
12288KB from
[0x00007f71950e2549] VirtualSpace::expand_by(unsigned
long, bool)+0x199
[0x00007f7194ec6766]
VirtualSpaceList::expand_node_by(VirtualSpaceNode*,
unsigned long, unsigned long)+0x76
[0x00007f7194ec9580] VirtualSpaceList::expand_by(unsigned
long, unsigned long)+0xf0
[0x00007f7194ec9713]
VirtualSpaceList::get_new_chunk(unsigned long, unsigned
long, unsigned long)+0xb3
...
[oracle@oel6u7 bin]$ head ../logs/gc.log
Java HotSpot(TM) 64-Bit Server VM (25.72-b15) for
linux-amd64 JRE (1.8.0_72-b15), built on Dec 22 2015
22:00:07 by "java_re" with gcc 4.3.0 20080428 (Red Hat
4.3.0-8)
Memory: 4k page, physical 3082784k(506980k free), swap
1257468k(1148276k free)
CommandLine flags: -XX:InitialCodeCacheSize=50331648
-XX:InitialHeapSize=49324544 -XX:MaxHeapSize=805306368
-XX:MaxMetaspaceSize=268435456
-XX:MetaspaceSize=268435456
-XX:NativeMemoryTracking=detail -XX:+PrintGC
-XX:+PrintGCTimeStamps -XX:ReservedCodeCacheSize=50331648
-XX:+UseCompressedClassPointers -XX:+UseCompressedOops
-XX:+UseLargePages -XX:+UseLargePagesInMetaspace
--
Regards
Timur Akhmadeev
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use@openjdk.java.net
<mailto:hotspot-gc-use@openjdk.java.net>
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use@openjdk.java.net
<mailto:hotspot-gc-use@openjdk.java.net>
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use@openjdk.java.net
<mailto:hotspot-gc-use@openjdk.java.net>
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
--
Regards
Timur Akhmadeev
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use@openjdk.java.net
<mailto:hotspot-gc-use@openjdk.java.net>
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
