Re: RFR: 8253495: CDS generates non-deterministic output [v8]

2022-03-15 Thread Calvin Cheung
On Tue, 15 Mar 2022 17:08:27 GMT, Ioi Lam wrote: >> This patch makes the result of "java -Xshare:dump" deterministic: >> - Disabled new Java threads from launching. This is harmless. See comments >> in jvm.cpp >> - Fixed a problem in hashtable ordering in heapShared.cpp >> - BasicHashtableEntry

Re: RFR: 8253495: CDS generates non-deterministic output [v8]

2022-03-15 Thread Ioi Lam
> This patch makes the result of "java -Xshare:dump" deterministic: > - Disabled new Java threads from launching. This is harmless. See comments in > jvm.cpp > - Fixed a problem in hashtable ordering in heapShared.cpp > - BasicHashtableEntry has a gap on 64-bit platforms that may contain random

Re: RFR: 8253495: CDS generates non-deterministic output [v7]

2022-03-15 Thread Magnus Ihse Bursie
On Tue, 15 Mar 2022 08:17:24 GMT, Ioi Lam wrote: >> This patch makes the result of "java -Xshare:dump" deterministic: >> - Disabled new Java threads from launching. This is harmless. See comments >> in jvm.cpp >> - Fixed a problem in hashtable ordering in heapShared.cpp >> - BasicHashtableEntry

Re: RFR: 8253495: CDS generates non-deterministic output [v6]

2022-03-15 Thread Ioi Lam
On Mon, 14 Mar 2022 22:07:24 GMT, Calvin Cheung wrote: >> Ioi Lam has updated the pull request incrementally with one additional >> commit since the last revision: >> >> Added helper function CollectedHeap::zap_filler_array_with > >

Re: RFR: 8253495: CDS generates non-deterministic output [v7]

2022-03-15 Thread Ioi Lam
> This patch makes the result of "java -Xshare:dump" deterministic: > - Disabled new Java threads from launching. This is harmless. See comments in > jvm.cpp > - Fixed a problem in hashtable ordering in heapShared.cpp > - BasicHashtableEntry has a gap on 64-bit platforms that may contain random

Re: RFR: 8253495: CDS generates non-deterministic output [v6]

2022-03-14 Thread Calvin Cheung
On Fri, 11 Mar 2022 06:55:23 GMT, Ioi Lam wrote: >> This patch makes the result of "java -Xshare:dump" deterministic: >> - Disabled new Java threads from launching. This is harmless. See comments >> in jvm.cpp >> - Fixed a problem in hashtable ordering in heapShared.cpp >> - BasicHashtableEntry

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-11 Thread Thomas Stuefe
On Fri, 11 Mar 2022 08:28:32 GMT, Ioi Lam wrote: >> Is reproducibility also a topic for users calling -Xdump with custom JNI >> coding? Or maybe having the VM instrumented somehow? Since it seems such an >> easy fix, I would prevent attaching too. At least the user would get a clear >> error

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-11 Thread Ioi Lam
On Fri, 11 Mar 2022 07:13:35 GMT, Thomas Stuefe wrote: > Is reproducibility also a topic for users calling -Xdump with custom JNI > coding? Or maybe having the VM instrumented somehow? Since it seems such an > easy fix, I would prevent attaching too. At least the user would get a clear >

Re: RFR: 8253495: CDS generates non-deterministic output

2022-03-10 Thread Thomas Stuefe
On Fri, 11 Mar 2022 07:03:00 GMT, David Holmes wrote: > > Well, he does it for `DumpSharedSpaces` only. Are you really worried about > > that one load+conditional jump? > > As I said (and I'm not the only one who says this :) ) "death by a thousand > cuts". Thank you, that is a good

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-10 Thread Thomas Stuefe
On Thu, 10 Mar 2022 19:34:29 GMT, Ioi Lam wrote: >> src/hotspot/share/prims/jvm.cpp line 2887: >> >>> 2885: return; >>> 2886: } >>> 2887: #endif >> >> Should we do this for jni_AttachCurrentThread too? > > This hasn't been necessary for me because jni_AttachCurrentThread is not > called

Re: RFR: 8253495: CDS generates non-deterministic output [v6]

2022-03-10 Thread Kim Barrett
On Fri, 11 Mar 2022 06:55:23 GMT, Ioi Lam wrote: >> This patch makes the result of "java -Xshare:dump" deterministic: >> - Disabled new Java threads from launching. This is harmless. See comments >> in jvm.cpp >> - Fixed a problem in hashtable ordering in heapShared.cpp >> - BasicHashtableEntry

Re: RFR: 8253495: CDS generates non-deterministic output

2022-03-10 Thread David Holmes
On 11/03/2022 4:50 pm, Thomas Stuefe wrote: On Fri, 11 Mar 2022 05:59:00 GMT, David Holmes wrote: Thanks for pointing this out. I ran more tests and found that on certain platforms, there are other structures that have problems with uninitialized gaps. I ended up changing `os::malloc()` to

Re: RFR: 8253495: CDS generates non-deterministic output

2022-03-10 Thread Thomas Stuefe
On Fri, 11 Mar 2022 06:50:00 GMT, David Holmes wrote: > I can combine the tests for `MemTracker::tracking_level()` and > `DumpSharedSpaces` into a single test and do more work only when the uncommon > path is taken. This would require some refactoring of the > MemTracker/MallocTracker code.

Re: RFR: 8253495: CDS generates non-deterministic output [v6]

2022-03-10 Thread Ioi Lam
> This patch makes the result of "java -Xshare:dump" deterministic: > - Disabled new Java threads from launching. This is harmless. See comments in > jvm.cpp > - Fixed a problem in hashtable ordering in heapShared.cpp > - BasicHashtableEntry has a gap on 64-bit platforms that may contain random

Re: RFR: 8253495: CDS generates non-deterministic output

2022-03-10 Thread Thomas Stuefe
On Fri, 11 Mar 2022 05:59:00 GMT, David Holmes wrote: > > Thanks for pointing this out. I ran more tests and found that on certain > > platforms, there are other structures that have problems with uninitialized > > gaps. I ended up changing `os::malloc()` to zero the buffer when running > >

Re: RFR: 8253495: CDS generates non-deterministic output

2022-03-10 Thread David Holmes
On 11/03/2022 4:40 pm, Ioi Lam wrote: On Fri, 11 Mar 2022 05:59:00 GMT, David Holmes wrote: I ended up changing `os::malloc()` to zero the buffer when running with -Xshare:dump. Hopefully one extra check of `if (DumpSharedSpaces)` doesn't matter too much for regular VM executions because

Re: RFR: 8253495: CDS generates non-deterministic output

2022-03-10 Thread Ioi Lam
On Fri, 11 Mar 2022 05:59:00 GMT, David Holmes wrote: > > I ended up changing `os::malloc()` to zero the buffer when running with > > -Xshare:dump. Hopefully one extra check of `if (DumpSharedSpaces)` doesn't > > matter too much for regular VM executions because `os::malloc()` already > > has

Re: RFR: 8253495: CDS generates non-deterministic output [v5]

2022-03-10 Thread Ioi Lam
> This patch makes the result of "java -Xshare:dump" deterministic: > - Disabled new Java threads from launching. This is harmless. See comments in > jvm.cpp > - Fixed a problem in hashtable ordering in heapShared.cpp > - BasicHashtableEntry has a gap on 64-bit platforms that may contain random

Re: RFR: 8253495: CDS generates non-deterministic output [v4]

2022-03-10 Thread Ioi Lam
On Fri, 11 Mar 2022 05:55:20 GMT, Kim Barrett wrote: >> Ioi Lam has updated the pull request incrementally with one additional >> commit since the last revision: >> >> zero GC heap filler arrays > > src/hotspot/share/gc/shared/collectedHeap.cpp line 449: > >> 447:

Re: RFR: 8253495: CDS generates non-deterministic output [v4]

2022-03-10 Thread Kim Barrett
On Fri, 11 Mar 2022 04:56:23 GMT, Ioi Lam wrote: >> This patch makes the result of "java -Xshare:dump" deterministic: >> - Disabled new Java threads from launching. This is harmless. See comments >> in jvm.cpp >> - Fixed a problem in hashtable ordering in heapShared.cpp >> - BasicHashtableEntry

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-10 Thread David Holmes
I can't find this comment in the PR so replying via email ... On 11/03/2022 9:24 am, Ioi Lam wrote: On Wed, 9 Mar 2022 07:47:19 GMT, Thomas Stuefe wrote: Ioi Lam has updated the pull request incrementally with one additional commit since the last revision: Fixed zero build

Re: RFR: 8253495: CDS generates non-deterministic output [v4]

2022-03-10 Thread Ioi Lam
> This patch makes the result of "java -Xshare:dump" deterministic: > - Disabled new Java threads from launching. This is harmless. See comments in > jvm.cpp > - Fixed a problem in hashtable ordering in heapShared.cpp > - BasicHashtableEntry has a gap on 64-bit platforms that may contain random

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-10 Thread Ioi Lam
On Wed, 9 Mar 2022 07:47:19 GMT, Thomas Stuefe wrote: >> Ioi Lam has updated the pull request incrementally with one additional >> commit since the last revision: >> >> Fixed zero build > > src/hotspot/share/utilities/hashtable.hpp line 42: > >> 40: >> 41: LP64_ONLY(unsigned int _gap;)

Re: RFR: 8253495: CDS generates non-deterministic output [v3]

2022-03-10 Thread Ioi Lam
> This patch makes the result of "java -Xshare:dump" deterministic: > - Disabled new Java threads from launching. This is harmless. See comments in > jvm.cpp > - Fixed a problem in hashtable ordering in heapShared.cpp > - BasicHashtableEntry has a gap on 64-bit platforms that may contain random

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-10 Thread David Holmes
On Thu, 10 Mar 2022 19:41:03 GMT, Ioi Lam wrote: >> I think he already did. I'm quoting: >> >>> However, the CDS archive also contains a heap dump, which includes Java >>> HashMaps. If I allow those 3 Java threads to start, some HashMaps in the >>> module graph will have unstable ordering. I

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-10 Thread Ioi Lam
On Thu, 10 Mar 2022 13:51:56 GMT, Magnus Ihse Bursie wrote: >> Ioi Lam has updated the pull request incrementally with one additional >> commit since the last revision: >> >> Fixed zero build > > I think he already did. I'm quoting: > >> However, the CDS archive also contains a heap dump,

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-10 Thread Ioi Lam
On Wed, 9 Mar 2022 07:51:46 GMT, Thomas Stuefe wrote: >> Ioi Lam has updated the pull request incrementally with one additional >> commit since the last revision: >> >> Fixed zero build > > src/hotspot/share/prims/jvm.cpp line 2887: > >> 2885: return; >> 2886: } >> 2887: #endif > >

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-10 Thread Ioi Lam
On Thu, 10 Mar 2022 13:51:56 GMT, Magnus Ihse Bursie wrote: >> Ioi Lam has updated the pull request incrementally with one additional >> commit since the last revision: >> >> Fixed zero build > > I think he already did. I'm quoting: > >> However, the CDS archive also contains a heap dump,

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-10 Thread Magnus Ihse Bursie
On Wed, 9 Mar 2022 05:10:44 GMT, Ioi Lam wrote: >> This patch makes the result of "java -Xshare:dump" deterministic: >> - Disabled new Java threads from launching. This is harmless. See comments >> in jvm.cpp >> - Fixed a problem in hashtable ordering in heapShared.cpp >> - BasicHashtableEntry

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-10 Thread David Holmes
On Thu, 10 Mar 2022 12:50:58 GMT, Magnus Ihse Bursie wrote: >> Ioi Lam has updated the pull request incrementally with one additional >> commit since the last revision: >> >> Fixed zero build > > Well, previously we'd get different dumps on different runs. If that was an > issue, surely it

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-10 Thread Magnus Ihse Bursie
The Skara bots messed up this one badly. It was a reply to David's comment, not Ioi's latest push. /Magnus On 2022-03-10 13:56, Magnus Ihse Bursie wrote: On Wed, 9 Mar 2022 05:10:44 GMT, Ioi Lam wrote: This patch makes the result of "java -Xshare:dump" deterministic: - Disabled new Java

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-10 Thread Thomas Stuefe
On Wed, 9 Mar 2022 07:58:51 GMT, Thomas Stuefe wrote: >> Ioi Lam has updated the pull request incrementally with one additional >> commit since the last revision: >> >> Fixed zero build > > Hi Ioi, > > some questions, comments inline. > > Like David in the comments, I am also a bit vague

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-10 Thread Magnus Ihse Bursie
On Wed, 9 Mar 2022 05:10:44 GMT, Ioi Lam wrote: >> This patch makes the result of "java -Xshare:dump" deterministic: >> - Disabled new Java threads from launching. This is harmless. See comments >> in jvm.cpp >> - Fixed a problem in hashtable ordering in heapShared.cpp >> - BasicHashtableEntry

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-10 Thread David Holmes
On Thu, 10 Mar 2022 12:11:06 GMT, Magnus Ihse Bursie wrote: >> The "heap dump" aspect of this is not something I'm familiar with, but if >> the threads don't affect the list of classes dumped, they surely must affect >> what is in the heap dump otherwise their execution would not be an issue.

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-10 Thread Magnus Ihse Bursie
On Wed, 9 Mar 2022 11:45:59 GMT, David Holmes wrote: >> Ioi Lam has updated the pull request incrementally with one additional >> commit since the last revision: >> >> Fixed zero build > > The "heap dump" aspect of this is not something I'm familiar with, but if the > threads don't affect

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-10 Thread Magnus Ihse Bursie
On Wed, 9 Mar 2022 07:58:51 GMT, Thomas Stuefe wrote: >> Ioi Lam has updated the pull request incrementally with one additional >> commit since the last revision: >> >> Fixed zero build > > Hi Ioi, > > some questions, comments inline. > > Like David in the comments, I am also a bit vague

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-09 Thread David Holmes
On Wed, 9 Mar 2022 05:10:44 GMT, Ioi Lam wrote: >> This patch makes the result of "java -Xshare:dump" deterministic: >> - Disabled new Java threads from launching. This is harmless. See comments >> in jvm.cpp >> - Fixed a problem in hashtable ordering in heapShared.cpp >> - BasicHashtableEntry

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-09 Thread Thomas Stuefe
On Wed, 9 Mar 2022 05:10:44 GMT, Ioi Lam wrote: >> This patch makes the result of "java -Xshare:dump" deterministic: >> - Disabled new Java threads from launching. This is harmless. See comments >> in jvm.cpp >> - Fixed a problem in hashtable ordering in heapShared.cpp >> - BasicHashtableEntry

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-08 Thread Ioi Lam
On Wed, 9 Mar 2022 07:04:56 GMT, David Holmes wrote: > I have reservations about contorting things this way just to get > "deterministic output". > > The VM needs to fully initialize and then become quiescent before the dump > occurs, and as I say below if you don't start other threads then

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-08 Thread Ioi Lam
On Wed, 9 Mar 2022 06:49:02 GMT, David Holmes wrote: >> Ioi Lam has updated the pull request incrementally with one additional >> commit since the last revision: >> >> Fixed zero build > > src/hotspot/share/prims/jvm.cpp line 2873: > >> 2871: // execute in parallel, symbols and classes

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-08 Thread David Holmes
On Wed, 9 Mar 2022 05:10:44 GMT, Ioi Lam wrote: >> This patch makes the result of "java -Xshare:dump" deterministic: >> - Disabled new Java threads from launching. This is harmless. See comments >> in jvm.cpp >> - Fixed a problem in hashtable ordering in heapShared.cpp >> - BasicHashtableEntry

Re: RFR: 8253495: CDS generates non-deterministic output [v2]

2022-03-08 Thread Ioi Lam
> This patch makes the result of "java -Xshare:dump" deterministic: > - Disabled new Java threads from launching. This is harmless. See comments in > jvm.cpp > - Fixed a problem in hashtable ordering in heapShared.cpp > - BasicHashtableEntry has a gap on 64-bit platforms that may contain random

Re: RFR: 8253495: CDS generates non-deterministic output

2022-03-08 Thread Erik Joelsson
On Tue, 8 Mar 2022 19:11:02 GMT, Ioi Lam wrote: > This patch makes the result of "java -Xshare:dump" deterministic: > - Disabled new Java threads from launching. This is harmless. See comments in > jvm.cpp > - Fixed a problem in hashtable ordering in heapShared.cpp > - BasicHashtableEntry has a

RFR: 8253495: CDS generates non-deterministic output

2022-03-08 Thread Ioi Lam
This patch makes the result of "java -Xshare:dump" deterministic: - Disabled new Java threads from launching. This is harmless. See comments in jvm.cpp - Fixed a problem in hashtable ordering in heapShared.cpp - BasicHashtableEntry has a gap on 64-bit platforms that may contain random bits.