Hi Ioi, Please don't do this :)
First off, how would this work when dumping with UseCompressedClassPointers off? In that case allocation would be relegated to non-class metaspace which cannot guarantee that kind of address stability. Even in class space, I do not think you can guarantee addresses growing monotonously. Class unloading could happens during dump time, so space may be returned to class freelist and later reused. Metadata can be prematurely deallocated, e.g. if a class load error occurs or byte code is rewritten by some agent. Remainder of Metachunks are used up in a delayed fashion. All these cases will present you with pointers which are not growing monotonously. I also believe this problem of non-deterministic placement is not limited to Symbols, but that you should see it for Klass structures too, albeit rarely. I believe the fact that you do not see this is an accident, or we are not looking that closely. E.g. if we were to change the frequency at which we retrieve MetaBlocks from the freeblocklist (see http://hg.openjdk.java.net/jdk/jdk/file/534855a30ef5/src/hotspot/share/memory/metaspace/spaceManager.cpp#l414), you would get more reuse of deallocated blocks and would certainly see more volatility in the addresses. This is all true with the current implementation; the upcoming new one uses a buddy style allocator behind the scenes where it is by no means guaranteed that the first chunks get used first. I think this is what still happens, by sheer accident, but I am hesitant to promise such a behavior in the future. It removes freedom from the implementation in a lot of ways. Small examples, we might want to shepherd certain allocations into separate parts of class space (e.g. Klass structures from hidden classes) to minimize fragmentation. Or add a mode, for testing, where we would allocate Klass at the very end of ccs, or at certain "round" addresses, to shake loose errors in the calling layers which rely too much on how Klass pointers look like. Bottomline I think the assumption that ccs allocates in monotonously ascending order is not even correct today, and may break very easily, and we should not rely on this. I think instead of misusing ccs for this, it would be cleaner to just allocate a large C heap area as backing storage for the symbols? How much space are we talking about? If memory is a concern, we could just reserve a range and commit it manually as we go. Or could we not order the placement of Klass and Symbol at dump time? Dump time is not that time critical, no? Thanks & Sorry, Thomas On Mon, Apr 27, 2020 at 7:31 AM Ioi Lam <ioi....@oracle.com> wrote: > https://bugs.openjdk.java.net/browse/JDK-8241071 > > http://cr.openjdk.java.net/~iklam/jdk15/8241071-deterministic-cds-archive.v02/ > > The goal is to for "java -Xshare:dump" to produce deterministic contents in > the CDS archive that depend only on the contents of $JAVA_HOME/lib/modules. > > Problems: > [1] Symbols in the CDS archive may have non-deterministic order because > Arena allocation is non-deterministic. > [2] The contents of the CDS shared heap region may be randomized due to > ImmutableCollections.SALT32L. > > Fixes: > [1] With -Xshare:dump, allocate Symbols from the class space (64-bit only). > See changes in symbol.cpp for details. > [2] When running the VM with -Xshare:dump, ImmutableCollections.SALT32L is > initialized with a deterministic seed. NOTE: this affects ONLY when > the > VM is running with the special flag -Xshare:dump to dump the CDS > archive. > It does NOT affect normal execution of Java programs. > > --- > I also cleaned up the -Xlog:cds output and print out the CRC of each > CDS region, to help diagnose why two CDS archives may have different > contents. > > Thanks > - Ioi >
