Hi Jon, thanks again for the clarification!
Kind Regards, Thomas On Fri, Feb 5, 2016 at 5:42 PM, Jon Masamitsu <jon.masami...@oracle.com> wrote: > > > On 2/5/2016 12:24 AM, Thomas Stüfe wrote: > > Hi Timur, > > On Fri, Feb 5, 2016 at 7:53 AM, Timur Akhmadeev <timur.akhmad...@gmail.com > > wrote: > >> Thanks Thomas and Jon! >> I think I've opened the page Thomas has referenced more than once before >> asking my question but somehow I missed this paragraph which is the answer >> :) >> >> If UseCompressedOops is turned on and UseCompressedClassesPointers is >>> used, then two logically different areas of native memory are used for >>> class metadata. UseCompressedClassPointers uses a 32-bit offset to >>> represent the class pointer in a 64-bit process as does >>> UseCompressedOops for Java object references. A region is allocated for >>> these compressed class pointers (the 32-bit offsets). The size of the >>> region can be set with CompressedClassSpaceSize and is 1 gigabyte (GB) >>> by default. The space for the compressed class pointers is reserved as >>> space allocated by mmap at initialization and committed as needed. The >>> MaxMetaspaceSize applies to the sum of the committed compressed class >>> space and the space for the other class metadata. >> >> >> It's a bit unnatural to call Compressed Class pointers part of the >> Metaspace and at the same time allow its part to be set bigger than the >> whole space while not allowing Compressed Class pointers (hopefully) to >> grow beyond MaxMetaspaceSize. >> >> > I agree. Maybe it is just me, but I also have a hard time understanding > the relationship between those two parameters. > > > > See also the mail thread here: > http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2015-June/013797.html > . > > I do not know the history, so I only can guess that the permgen removal > came first, and that at first it was just a "simple" metaspace. But with > moving out class data from the heap we lost the ability to address them > with compressed oops (32bit pointers) and this feature was reintroduced in > the form of compressed class pointers, which made the implementation more > complicated and now we have those two switches. > > > > When there was a perm gen, the perm gen and the rest of the heap were a > contiguous > block of addressed with one base address (heap-base). With compressed > oops (COOPS) > object pointers (including pointers to metadata in the perm gen) were > 32bit offsets from > heap-base. When metadata was moved to native memory, the address space > for the > metadata was reserved and committed as it was needed (so there are > multiple chunks > of addresses for metadata and so not a single metadata base address > (meta-base)). > For compressed class pointers we needed a single base address (class-base) > again > so we could use offsets from class-base for the class pointers. So we > reserve a > single contiguous address space for the compressed class pointers (separate > from the other metadata) and commit from that reserved space as it is > needed. > > > But this is only a guess from my side. Whatever the reasons, I think the > parameters are confusing, especially as the sizes have two different > meanings - reserved size in case of CompressedClassSpaceSize, committed > size in case of MaxMetaSpaceSize. > > > We're trying to manage data that can be accessed differently (full 64bit > address and 32bit offset) > so need to have different spaces. CompressedClassSpaceSize is the size we > reserve at > initialization. The part of that which is committed plus the amount of > committed space for > the other metadata is limited by MaxMetaSpaceSize. > > I also do not know if CompressedClassSpaceSize is a hard limit - what > happens when we fill compressed class space with more than that? Will it > expand or throw an OOM? > > > CompressedClassSpaceSize is a hard limit and will limit the number of > classes that can be > loaded. There are plans to fix that. > > Jon > > > Kind Regards, Thomas > > > >> On Thu, Feb 4, 2016 at 10:02 PM, Jon Masamitsu < >> <jon.masami...@oracle.com>jon.masami...@oracle.com> wrote: >> >>> >>> >>> On 02/04/2016 06:41 AM, Thomas Stüfe wrote: >>> >>> Hi Timur, >>> >>> I believe you see the space for the compressed class data (Compressed >>> class space), which is by default 1GB and controlled by >>> -XX:CompressedClassSpaceSize . I think this area is semantically one >>> subarea of the metaspace, the other being non-class data. >>> >>> I am unsure about the difference between MaxMetaSpaceSize and >>> CompressedClassSpaceSize, but what I believe is that: >>> >>> - MaxMetaSpaceSize limits the amount of total committed (not reserved) >>> space >>> - CompressedClassSpaceSize sets the amount of space to be reserved (not >>> committed) for compressed classes only. >>> >>> This would fit your NMT output, which says: >>> >>> "Class (reserved=1142808KB, committed=106520KB)" >>> >>> so, ~100M are committed. With a MaxMetaSpaceSize=256M - what you >>> specified - you would not be able to commit the whole 1GB, but hit the >>> limit far earlier. >>> >>> See also: >>> https://docs.oracle.com/javase/8/docs/technotes/guides/vm/gctuning/considerations.htm >>> (the section about UseCompressedClassesPointers) >>> >>> Please take this with a grain of salt. I am not sure I understand this >>> correctly and would value input from other developers. >>> >>> >>> Thomas, >>> >>> No disagreement from me. I think you described it >>> correctly. Note that if you turn off UseCompressedClassPointers, >>> there will not be a separate space for the class metadata. >>> >>> Jon >>> >>> >>> >>> Kind Regards, Thomas >>> >>> On Thu, Feb 4, 2016 at 2:13 PM, Timur Akhmadeev < >>> <timur.akhmad...@gmail.com>timur.akhmad...@gmail.com> wrote: >>> >>>> Hi, >>>> >>>> I'm trying to figure out which parts of memory Hotspot can allocate in >>>> HugePages on Linux (appears to be heap and code cache by default, plus >>>> Metaspace with -XX:+UseLargePagesInMetaspace) and noticed something I can't >>>> explain. >>>> >>>> There's 1G allocation on the OS level in 4K pages that is apparently >>>> attributed to Class metadata, although Metaspace is maxed at a value much >>>> lower than 1G. Question: why? >>>> >>>> [oracle@oel6u7 bin]$ grep -B 11 'KernelPageSize: 4 kB' >>>> /proc/18906/smaps | grep '^Size:' | head >>>> Size: 4 kB >>>> Size: 4 kB >>>> Size: 2764 kB >>>> Size: 12288 kB >>>> Size: 1036288 kB >>>> Size: 128 kB >>>> Size: 4 kB >>>> Size: 4 kB >>>> Size: 4 kB >>>> Size: 8 kB >>>> >>>> [oracle@oel6u7 bin]$ /home/oracle/jdk1.8.0_72/bin/jcmd 18906 >>>> VM.native_memory summary >>>> 18906: >>>> >>>> Native Memory Tracking: >>>> >>>> Total: reserved=2069703KB, committed=1032559KB >>>> - Java Heap (reserved=786432KB, committed=786432KB) >>>> (mmap: reserved=786432KB, >>>> committed=786432KB) >>>> >>>> - Class (reserved=1142808KB, committed=106520KB) >>>> (classes #13624) >>>> (malloc=2072KB #22966) >>>> (mmap: reserved=1140736KB, >>>> committed=104448KB) >>>> >>>> - Thread (reserved=56009KB, committed=56009KB) >>>> (thread #55) >>>> (stack: reserved=55512KB, committed=55512KB) >>>> (malloc=178KB #272) >>>> (arena=319KB #108) >>>> >>>> - Code (reserved=55407KB, committed=55407KB) >>>> (malloc=5487KB #7167) >>>> (mmap: reserved=49920KB, committed=49920KB) >>>> >>>> - GC (reserved=2600KB, committed=1748KB) >>>> (malloc=32KB #355) >>>> (mmap: reserved=2568KB, committed=1716KB) >>>> >>>> - Compiler (reserved=213KB, committed=213KB) >>>> (malloc=82KB #331) >>>> (arena=131KB #3) >>>> >>>> - Internal (reserved=4921KB, committed=4917KB) >>>> (malloc=4885KB #16137) >>>> (mmap: reserved=36KB, committed=32KB) >>>> >>>> - Symbol (reserved=17518KB, committed=17518KB) >>>> (malloc=14377KB #172159) >>>> (arena=3141KB #1) >>>> >>>> - Native Memory Tracking (reserved=3617KB, committed=3617KB) >>>> (malloc=149KB #2329) >>>> (tracking overhead=3468KB) >>>> >>>> - Arena Chunk (reserved=178KB, committed=178KB) >>>> (malloc=178KB) >>>> >>>> [oracle@oel6u7 bin]$ /home/oracle/jdk1.8.0_72/bin/jcmd 18906 >>>> VM.native_memory detail | grep -A 10 'for Class' >>>> [0x0000000100000000 - 0x0000000140000000] reserved 1048576KB for Class >>>> from >>>> [0x00007f71950e2ad2] ReservedSpace::initialize(unsigned long, >>>> unsigned long, bool, char*, unsigned long, bool)+0xc2 >>>> [0x00007f71950e2d0b] ReservedSpace::ReservedSpace(unsigned long, >>>> unsigned long, bool, char*, unsigned long)+0x1b >>>> [0x00007f7194ec8300] >>>> Metaspace::allocate_metaspace_compressed_klass_ptrs(char*, unsigned >>>> char*)+0x40 >>>> [0x00007f7194eca7df] Metaspace::global_initialize()+0x4cf >>>> >>>> [0x0000000100000000 - 0x0000000100c00000] committed 12288KB from >>>> [0x00007f71950e2549] VirtualSpace::expand_by(unsigned long, >>>> bool)+0x199 >>>> [0x00007f7194ec6766] >>>> VirtualSpaceList::expand_node_by(VirtualSpaceNode*, unsigned long, unsigned >>>> long)+0x76 >>>> [0x00007f7194ec9580] VirtualSpaceList::expand_by(unsigned >>>> long, unsigned long)+0xf0 >>>> [0x00007f7194ec9713] >>>> VirtualSpaceList::get_new_chunk(unsigned long, unsigned long, unsigned >>>> long)+0xb3 >>>> ... >>>> >>>> [oracle@oel6u7 bin]$ head ../logs/gc.log >>>> Java HotSpot(TM) 64-Bit Server VM (25.72-b15) for linux-amd64 JRE >>>> (1.8.0_72-b15), built on Dec 22 2015 22:00:07 by "java_re" with gcc 4.3.0 >>>> 20080428 (Red Hat 4.3.0-8) >>>> Memory: 4k page, physical 3082784k(506980k free), swap >>>> 1257468k(1148276k free) >>>> CommandLine flags: -XX:InitialCodeCacheSize=50331648 >>>> -XX:InitialHeapSize=49324544 -XX:MaxHeapSize=805306368 >>>> -XX:MaxMetaspaceSize=268435456 -XX:MetaspaceSize=268435456 >>>> -XX:NativeMemoryTracking=detail -XX:+PrintGC -XX:+PrintGCTimeStamps >>>> -XX:ReservedCodeCacheSize=50331648 -XX:+UseCompressedClassPointers >>>> -XX:+UseCompressedOops -XX:+UseLargePages -XX:+UseLargePagesInMetaspace >>>> >>>> -- >>>> Regards >>>> Timur Akhmadeev >>>> >>>> _______________________________________________ >>>> hotspot-gc-use mailing list >>>> hotspot-gc-use@openjdk.java.net >>>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use >>>> >>>> >>> >>> >>> _______________________________________________ >>> hotspot-gc-use mailing >>> listhotspot-gc-...@openjdk.java.nethttp://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use >>> >>> >>> >>> _______________________________________________ >>> hotspot-gc-use mailing list >>> hotspot-gc-use@openjdk.java.net >>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use >>> >>> >> >> >> -- >> Regards >> Timur Akhmadeev >> >> _______________________________________________ >> hotspot-gc-use mailing list >> hotspot-gc-use@openjdk.java.net >> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use >> >> > >
_______________________________________________ hotspot-gc-use mailing list hotspot-gc-use@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
