Re: RFR: JDK-8256844: Make NMT late-initializable [v2]
On Mon, 2 Aug 2021 14:24:54 GMT, Coleen Phillimore wrote: >>> This is an interesting and it seems a better way to solve this problem. >>> Where were you all those years ago?? I hope @zhengyu123 has a chance to >>> review it. >> >> Thank you! I was here, but we were not yet doing much upstream :) To be >> fair, this problem got quite involved and did cost me some cycles and false >> starts. I fully understand that the first solution uses the environment >> variable approach. >> >> I spend some time investigating different ideas with this one; at first I >> did not use a hash-table but a static pre-allocated buffer from which I fed >> early allocations. But the code got too complex, and Kim's suggestion with >> the side table turned out just to be a lot simpler. >> >>> Also interesting is that we were wondering how we could return malloc'd >>> memory on JVM creation failure, and this might partially help with that >>> larger problem. >> >> Yes, this would be trivial now, to return that memory. Though I am afraid it >> would be a small part only. But NMT may be instrumental in releasing all >> memory, since it knows everything. Only, its not always enabled. >> >> Is that a real-life problem? Are there cases where the launcher would want >> to live on if the JVM failed to load? > >> Is that a real-life problem? Are there cases where the launcher would want >> to live on if the JVM failed to load? > > There are a lot of other reasons why the launcher couldn't live on if the JVM > fails to load. This was only one of them. We used to think about this > problem once but don't really think it's solveable. Thanks @coleenp and @zhengyu123 ! - PR: https://git.openjdk.java.net/jdk/pull/4874
Re: RFR: JDK-8256844: Make NMT late-initializable [v2]
On Wed, 4 Aug 2021 03:36:11 GMT, Thomas Stuefe wrote: > Nightlies at SAP showed no problems for several runs now. The failed GHA test > (StringDeduplication) seems to have nothing to do with my patch. > > @zhengyu123 are you fine with the latest version of this patch? Still good. - PR: https://git.openjdk.java.net/jdk/pull/4874
Re: RFR: JDK-8256844: Make NMT late-initializable [v2]
On Sun, 1 Aug 2021 08:17:08 GMT, Thomas Stuefe wrote: >> Short: this patch makes NMT available in custom-launcher scenarios and >> during gtests. It simplifies NMT initialization. It adds a lot of >> NMT-specific testing, cleans them up and makes them sideeffect-free. >> >> - >> >> NMT continues to be an extremely useful tool for SAP to tackle memory >> problems in the JVM. >> >> However, NMT is of limited use due to the following restrictions: >> >> - NMT cannot be used if the hotspot is embedded into a custom launcher >> unless the launcher actively cooperates. Just creating and invoking the JVM >> is not enough, it needs to do some steps prior to loading the hotspot. This >> limitation is not well known (nor, do I believe, documented). Many products >> don't do this, e.g., you cannot use NMT with IntelliJ. For us at SAP this >> problem limits NMT usefulness greatly since our VMs are often embedded into >> custom launchers and modifying every launcher is impossible. >> - Worse, if that custom launcher links the libjvm *statically* there is just >> no way to activate NMT at all. This is the reason NMT cannot be used in the >> `gtestlauncher`. >> - Related to that is that we cannot pass NMT options via `JAVA_TOOL_OPTIONS` >> and `-XX:Flags=`. >> - The fact that NMT cannot be used in gtests is really a pity since it would >> allow us to both test NMT itself more rigorously and check for memory leaks >> while testing other stuff. >> >> The reason for all this is that NMT initialization happens very early, on >> the first call to `os::malloc()`. And those calls happen already during >> dynamic C++ initialization - a long time before the VM gets around parsing >> arguments. So, regular VM argument parsing is too late to parse NMT >> arguments. >> >> The current solution is to pass NMT arguments via a specially prepared >> environment variable: `NMT_LEVEL_=`. That environment >> variable has to be set by the embedding launcher, before it loads the >> libjvm. Since its name contains the PID, we cannot even set that variable in >> the shell before starting the launcher. >> >> All that means that every launcher needs to especially parse and process the >> NMT arguments given at the command line (or via whatever method) and prepare >> the environment variable. `java` itself does this. This only works before >> the libjvm.so is loaded, before its dynamic C++ initialization. For that >> reason, it does not work if the launcher links statically against the >> hotspot, since in that case C++ initialization of the launcher and hotspot >> are folded into one phase with no possibility of executing code beforehand. >> >> And since it bypasses argument handling in the VM, it bypasses a number of >> argument processing ways, e.g., `JAVA_TOOL_OPTIONS`. >> >> -- >> >> This patch fixes these shortcomings by making NMT late-initializable: it can >> now be initialized after normal VM argument parsing, like all other parts of >> the VM. This greatly simplifies NMT initialization and makes it work >> automagically for every third party launcher, as well as within our gtests. >> >> The glaring problem with late-initializing NMT is the NMT malloc headers. If >> we rule out just always having them (unacceptable in terms of memory >> overhead), there is no safe way to determine, in os::free(), if an >> allocation came from before or after NMT initialization ran, and therefore >> what to do with its malloc headers. For a more extensive explanation, please >> see the comment block `nmtPreInit.hpp` and the discussion with @kimbarrett >> and @zhengyu123 in the JBS comment section. >> >> The heart of this patch is a new way to track early, pre-NMT-init >> allocations. These are tracked via a lookup table. This was a suggestion by >> Kim and it worked out well. >> >> Changes in detail: >> >> - pre-NMT-init handling: >> - the new files `nmtPreInit.hpp/cpp` take case of NMT pre-init >> handling. They contain a small global lookup table managing C-heap blocks >> allocated in the pre-NMT-init phase. >> - `os::malloc()/os::realloc()/os::free()` defer to this code before >> doing anything else. >> - Please see the extensive comment block at the start of >> `nmtPreinit.hpp` explaining the details. >> >> - Changes to NMT: >> - Before, NMT initialization was spread over two phases, `initialize()` >> and `late_initialize()`. Those were merged into one and simplified - there >> is only one initialization now which happens after argument parsing. >> - Minor changes were needed for the `NMT_TrackingLevel` enum - to >> simplify code, I changed NMT_unknown to be numerically 0. A new comment >> block in `nmtCommon.hpp` now clearly specifies what's what, including >> allowed level state transitions. >> - New utility functions to translate tracking level from/to strings >> added to `NMTUtil` >> - NMT has never been able to handle virtual memory
Re: RFR: JDK-8256844: Make NMT late-initializable [v2]
On Mon, 2 Aug 2021 14:38:23 GMT, Coleen Phillimore wrote: > This looks good. Thanks for fixing the mysterious (to me) cast. Thank you, Coleen! - PR: https://git.openjdk.java.net/jdk/pull/4874
Re: RFR: JDK-8256844: Make NMT late-initializable [v2]
On Sun, 1 Aug 2021 08:17:08 GMT, Thomas Stuefe wrote: >> Short: this patch makes NMT available in custom-launcher scenarios and >> during gtests. It simplifies NMT initialization. It adds a lot of >> NMT-specific testing, cleans them up and makes them sideeffect-free. >> >> - >> >> NMT continues to be an extremely useful tool for SAP to tackle memory >> problems in the JVM. >> >> However, NMT is of limited use due to the following restrictions: >> >> - NMT cannot be used if the hotspot is embedded into a custom launcher >> unless the launcher actively cooperates. Just creating and invoking the JVM >> is not enough, it needs to do some steps prior to loading the hotspot. This >> limitation is not well known (nor, do I believe, documented). Many products >> don't do this, e.g., you cannot use NMT with IntelliJ. For us at SAP this >> problem limits NMT usefulness greatly since our VMs are often embedded into >> custom launchers and modifying every launcher is impossible. >> - Worse, if that custom launcher links the libjvm *statically* there is just >> no way to activate NMT at all. This is the reason NMT cannot be used in the >> `gtestlauncher`. >> - Related to that is that we cannot pass NMT options via `JAVA_TOOL_OPTIONS` >> and `-XX:Flags=`. >> - The fact that NMT cannot be used in gtests is really a pity since it would >> allow us to both test NMT itself more rigorously and check for memory leaks >> while testing other stuff. >> >> The reason for all this is that NMT initialization happens very early, on >> the first call to `os::malloc()`. And those calls happen already during >> dynamic C++ initialization - a long time before the VM gets around parsing >> arguments. So, regular VM argument parsing is too late to parse NMT >> arguments. >> >> The current solution is to pass NMT arguments via a specially prepared >> environment variable: `NMT_LEVEL_=`. That environment >> variable has to be set by the embedding launcher, before it loads the >> libjvm. Since its name contains the PID, we cannot even set that variable in >> the shell before starting the launcher. >> >> All that means that every launcher needs to especially parse and process the >> NMT arguments given at the command line (or via whatever method) and prepare >> the environment variable. `java` itself does this. This only works before >> the libjvm.so is loaded, before its dynamic C++ initialization. For that >> reason, it does not work if the launcher links statically against the >> hotspot, since in that case C++ initialization of the launcher and hotspot >> are folded into one phase with no possibility of executing code beforehand. >> >> And since it bypasses argument handling in the VM, it bypasses a number of >> argument processing ways, e.g., `JAVA_TOOL_OPTIONS`. >> >> -- >> >> This patch fixes these shortcomings by making NMT late-initializable: it can >> now be initialized after normal VM argument parsing, like all other parts of >> the VM. This greatly simplifies NMT initialization and makes it work >> automagically for every third party launcher, as well as within our gtests. >> >> The glaring problem with late-initializing NMT is the NMT malloc headers. If >> we rule out just always having them (unacceptable in terms of memory >> overhead), there is no safe way to determine, in os::free(), if an >> allocation came from before or after NMT initialization ran, and therefore >> what to do with its malloc headers. For a more extensive explanation, please >> see the comment block `nmtPreInit.hpp` and the discussion with @kimbarrett >> and @zhengyu123 in the JBS comment section. >> >> The heart of this patch is a new way to track early, pre-NMT-init >> allocations. These are tracked via a lookup table. This was a suggestion by >> Kim and it worked out well. >> >> Changes in detail: >> >> - pre-NMT-init handling: >> - the new files `nmtPreInit.hpp/cpp` take case of NMT pre-init >> handling. They contain a small global lookup table managing C-heap blocks >> allocated in the pre-NMT-init phase. >> - `os::malloc()/os::realloc()/os::free()` defer to this code before >> doing anything else. >> - Please see the extensive comment block at the start of >> `nmtPreinit.hpp` explaining the details. >> >> - Changes to NMT: >> - Before, NMT initialization was spread over two phases, `initialize()` >> and `late_initialize()`. Those were merged into one and simplified - there >> is only one initialization now which happens after argument parsing. >> - Minor changes were needed for the `NMT_TrackingLevel` enum - to >> simplify code, I changed NMT_unknown to be numerically 0. A new comment >> block in `nmtCommon.hpp` now clearly specifies what's what, including >> allowed level state transitions. >> - New utility functions to translate tracking level from/to strings >> added to `NMTUtil` >> - NMT has never been able to handle virtual memory
Re: RFR: JDK-8256844: Make NMT late-initializable [v2]
On Fri, 30 Jul 2021 09:32:22 GMT, Thomas Stuefe wrote: >> [Not a review, just a drive-by comment.] This is a normal and idiomatic >> overload on the const-ness of `this`. > > To expand on Kim's answer. This is a way to solve const/nonconst problems > like https://github.com/openjdk/jdk/pull/4938/files#r679639391. > > You get a const version (suitably returning a write-protected pointer) which > the compiler chooses in const code, and a non-const version for non-const > code, and no awkward const-casts are needed from the outside. > > In this case the implementation is simple enough that I just duplicated it; > were it more complex, I'd call one in terms of the other. We do this in other > places too, see e.g. `ResourceHashTable::lookup_node`. This is ok. This was just new to me. - PR: https://git.openjdk.java.net/jdk/pull/4874
Re: RFR: JDK-8256844: Make NMT late-initializable [v2]
On Fri, 30 Jul 2021 09:44:57 GMT, Thomas Stuefe wrote: > Is that a real-life problem? Are there cases where the launcher would want to > live on if the JVM failed to load? There are a lot of other reasons why the launcher couldn't live on if the JVM fails to load. This was only one of them. We used to think about this problem once but don't really think it's solveable. - PR: https://git.openjdk.java.net/jdk/pull/4874
Re: RFR: JDK-8256844: Make NMT late-initializable [v2]
On Sun, 1 Aug 2021 08:17:08 GMT, Thomas Stuefe wrote: >> Short: this patch makes NMT available in custom-launcher scenarios and >> during gtests. It simplifies NMT initialization. It adds a lot of >> NMT-specific testing, cleans them up and makes them sideeffect-free. >> >> - >> >> NMT continues to be an extremely useful tool for SAP to tackle memory >> problems in the JVM. >> >> However, NMT is of limited use due to the following restrictions: >> >> - NMT cannot be used if the hotspot is embedded into a custom launcher >> unless the launcher actively cooperates. Just creating and invoking the JVM >> is not enough, it needs to do some steps prior to loading the hotspot. This >> limitation is not well known (nor, do I believe, documented). Many products >> don't do this, e.g., you cannot use NMT with IntelliJ. For us at SAP this >> problem limits NMT usefulness greatly since our VMs are often embedded into >> custom launchers and modifying every launcher is impossible. >> - Worse, if that custom launcher links the libjvm *statically* there is just >> no way to activate NMT at all. This is the reason NMT cannot be used in the >> `gtestlauncher`. >> - Related to that is that we cannot pass NMT options via `JAVA_TOOL_OPTIONS` >> and `-XX:Flags=`. >> - The fact that NMT cannot be used in gtests is really a pity since it would >> allow us to both test NMT itself more rigorously and check for memory leaks >> while testing other stuff. >> >> The reason for all this is that NMT initialization happens very early, on >> the first call to `os::malloc()`. And those calls happen already during >> dynamic C++ initialization - a long time before the VM gets around parsing >> arguments. So, regular VM argument parsing is too late to parse NMT >> arguments. >> >> The current solution is to pass NMT arguments via a specially prepared >> environment variable: `NMT_LEVEL_=`. That environment >> variable has to be set by the embedding launcher, before it loads the >> libjvm. Since its name contains the PID, we cannot even set that variable in >> the shell before starting the launcher. >> >> All that means that every launcher needs to especially parse and process the >> NMT arguments given at the command line (or via whatever method) and prepare >> the environment variable. `java` itself does this. This only works before >> the libjvm.so is loaded, before its dynamic C++ initialization. For that >> reason, it does not work if the launcher links statically against the >> hotspot, since in that case C++ initialization of the launcher and hotspot >> are folded into one phase with no possibility of executing code beforehand. >> >> And since it bypasses argument handling in the VM, it bypasses a number of >> argument processing ways, e.g., `JAVA_TOOL_OPTIONS`. >> >> -- >> >> This patch fixes these shortcomings by making NMT late-initializable: it can >> now be initialized after normal VM argument parsing, like all other parts of >> the VM. This greatly simplifies NMT initialization and makes it work >> automagically for every third party launcher, as well as within our gtests. >> >> The glaring problem with late-initializing NMT is the NMT malloc headers. If >> we rule out just always having them (unacceptable in terms of memory >> overhead), there is no safe way to determine, in os::free(), if an >> allocation came from before or after NMT initialization ran, and therefore >> what to do with its malloc headers. For a more extensive explanation, please >> see the comment block `nmtPreInit.hpp` and the discussion with @kimbarrett >> and @zhengyu123 in the JBS comment section. >> >> The heart of this patch is a new way to track early, pre-NMT-init >> allocations. These are tracked via a lookup table. This was a suggestion by >> Kim and it worked out well. >> >> Changes in detail: >> >> - pre-NMT-init handling: >> - the new files `nmtPreInit.hpp/cpp` take case of NMT pre-init >> handling. They contain a small global lookup table managing C-heap blocks >> allocated in the pre-NMT-init phase. >> - `os::malloc()/os::realloc()/os::free()` defer to this code before >> doing anything else. >> - Please see the extensive comment block at the start of >> `nmtPreinit.hpp` explaining the details. >> >> - Changes to NMT: >> - Before, NMT initialization was spread over two phases, `initialize()` >> and `late_initialize()`. Those were merged into one and simplified - there >> is only one initialization now which happens after argument parsing. >> - Minor changes were needed for the `NMT_TrackingLevel` enum - to >> simplify code, I changed NMT_unknown to be numerically 0. A new comment >> block in `nmtCommon.hpp` now clearly specifies what's what, including >> allowed level state transitions. >> - New utility functions to translate tracking level from/to strings >> added to `NMTUtil` >> - NMT has never been able to handle virtual memory
Re: RFR: JDK-8256844: Make NMT late-initializable [v2]
On Fri, 30 Jul 2021 20:14:04 GMT, Zhengyu Gu wrote:
>> Thomas Stuefe has updated the pull request incrementally with six additional
>> commits since the last revision:
>>
>> - feedback zhengyu
>> - feeback Coleen/Kim (constness fix)
>> - Feedback David
>> - Add test to test for non-java launcher support of NMT
>> - move all test code to gtest
>> - actually free blocks freed in pre-init phase
>
> src/hotspot/share/services/nmtPreInit.hpp line 153:
>
>> 151:
>> 152: static unsigned calculate_hash(const void* p) {
>> 153: uintptr_t tmp = p2i(p);
>
> malloc memory usually is 2-machine word aligned, maybe tmp = tmp >>
> LP64_ONLY(4) NOT_LP64(3) can result better hash distribution?
I thought so too at first, but it is not necessary. The hash function uses all
input bits:
uintptr_t tmp = p2i(p);
unsigned hash = (unsigned)tmp
LP64_ONLY( ^ (unsigned)(tmp >> 32));
p2i is lossless since input and result have the same size.
Then,
- for 32-bit, it does not matter since unsigned is the same size as uintptr_t
and we lose no bits
- for 64-bit, we xor the upper half of the 64-bit input value into the lower
half; again, all input bits count toward the result. We don't gain anything by
shifting, we would just exchange the lower 2 bits always being zero against the
upper 2 bits always being zero.
---
BTW, I experimented with different hash functions, e.g.
http://www.concentric.net/~Ttwang/tech/inthash.htm, but did not really get a
better distribution. This somewhat mirrors my experiences when I tried to
optimize your hash function in the MallocSiteTable. Seems malloc return value
is already "random enough". I coupled it with a crooked table size though, made
it a prime.
-
PR: https://git.openjdk.java.net/jdk/pull/4874
Re: RFR: JDK-8256844: Make NMT late-initializable [v2]
> Short: this patch makes NMT available in custom-launcher scenarios and during > gtests. It simplifies NMT initialization. It adds a lot of NMT-specific > testing, cleans them up and makes them sideeffect-free. > > - > > NMT continues to be an extremely useful tool for SAP to tackle memory > problems in the JVM. > > However, NMT is of limited use due to the following restrictions: > > - NMT cannot be used if the hotspot is embedded into a custom launcher unless > the launcher actively cooperates. Just creating and invoking the JVM is not > enough, it needs to do some steps prior to loading the hotspot. This > limitation is not well known (nor, do I believe, documented). Many products > don't do this, e.g., you cannot use NMT with IntelliJ. For us at SAP this > problem limits NMT usefulness greatly since our VMs are often embedded into > custom launchers and modifying every launcher is impossible. > - Worse, if that custom launcher links the libjvm *statically* there is just > no way to activate NMT at all. This is the reason NMT cannot be used in the > `gtestlauncher`. > - Related to that is that we cannot pass NMT options via `JAVA_TOOL_OPTIONS` > and `-XX:Flags=`. > - The fact that NMT cannot be used in gtests is really a pity since it would > allow us to both test NMT itself more rigorously and check for memory leaks > while testing other stuff. > > The reason for all this is that NMT initialization happens very early, on the > first call to `os::malloc()`. And those calls happen already during dynamic > C++ initialization - a long time before the VM gets around parsing arguments. > So, regular VM argument parsing is too late to parse NMT arguments. > > The current solution is to pass NMT arguments via a specially prepared > environment variable: `NMT_LEVEL_=`. That environment > variable has to be set by the embedding launcher, before it loads the libjvm. > Since its name contains the PID, we cannot even set that variable in the > shell before starting the launcher. > > All that means that every launcher needs to especially parse and process the > NMT arguments given at the command line (or via whatever method) and prepare > the environment variable. `java` itself does this. This only works before the > libjvm.so is loaded, before its dynamic C++ initialization. For that reason, > it does not work if the launcher links statically against the hotspot, since > in that case C++ initialization of the launcher and hotspot are folded into > one phase with no possibility of executing code beforehand. > > And since it bypasses argument handling in the VM, it bypasses a number of > argument processing ways, e.g., `JAVA_TOOL_OPTIONS`. > > -- > > This patch fixes these shortcomings by making NMT late-initializable: it can > now be initialized after normal VM argument parsing, like all other parts of > the VM. This greatly simplifies NMT initialization and makes it work > automagically for every third party launcher, as well as within our gtests. > > The glaring problem with late-initializing NMT is the NMT malloc headers. If > we rule out just always having them (unacceptable in terms of memory > overhead), there is no safe way to determine, in os::free(), if an allocation > came from before or after NMT initialization ran, and therefore what to do > with its malloc headers. For a more extensive explanation, please see the > comment block `nmtPreInit.hpp` and the discussion with @kimbarrett and > @zhengyu123 in the JBS comment section. > > The heart of this patch is a new way to track early, pre-NMT-init > allocations. These are tracked via a lookup table. This was a suggestion by > Kim and it worked out well. > > Changes in detail: > > - pre-NMT-init handling: > - the new files `nmtPreInit.hpp/cpp` take case of NMT pre-init > handling. They contain a small global lookup table managing C-heap blocks > allocated in the pre-NMT-init phase. > - `os::malloc()/os::realloc()/os::free()` defer to this code before > doing anything else. > - Please see the extensive comment block at the start of > `nmtPreinit.hpp` explaining the details. > > - Changes to NMT: > - Before, NMT initialization was spread over two phases, `initialize()` > and `late_initialize()`. Those were merged into one and simplified - there is > only one initialization now which happens after argument parsing. > - Minor changes were needed for the `NMT_TrackingLevel` enum - to > simplify code, I changed NMT_unknown to be numerically 0. A new comment block > in `nmtCommon.hpp` now clearly specifies what's what, including allowed level > state transitions. > - New utility functions to translate tracking level from/to strings > added to `NMTUtil` > - NMT has never been able to handle virtual memory allocations before > initialization, which is fine since os::reserve_memory() is not called before > VM parses arguments. We now assert that.
Re: RFR: JDK-8256844: Make NMT late-initializable
On Thu, 29 Jul 2021 06:33:38 GMT, David Holmes wrote: >> Short: this patch makes NMT available in custom-launcher scenarios and >> during gtests. It simplifies NMT initialization. It adds a lot of >> NMT-specific testing, cleans them up and makes them sideeffect-free. >> >> - >> >> NMT continues to be an extremely useful tool for SAP to tackle memory >> problems in the JVM. >> >> However, NMT is of limited use due to the following restrictions: >> >> - NMT cannot be used if the hotspot is embedded into a custom launcher >> unless the launcher actively cooperates. Just creating and invoking the JVM >> is not enough, it needs to do some steps prior to loading the hotspot. This >> limitation is not well known (nor, do I believe, documented). Many products >> don't do this, e.g., you cannot use NMT with IntelliJ. For us at SAP this >> problem limits NMT usefulness greatly since our VMs are often embedded into >> custom launchers and modifying every launcher is impossible. >> - Worse, if that custom launcher links the libjvm *statically* there is just >> no way to activate NMT at all. This is the reason NMT cannot be used in the >> `gtestlauncher`. >> - Related to that is that we cannot pass NMT options via `JAVA_TOOL_OPTIONS` >> and `-XX:Flags=`. >> - The fact that NMT cannot be used in gtests is really a pity since it would >> allow us to both test NMT itself more rigorously and check for memory leaks >> while testing other stuff. >> >> The reason for all this is that NMT initialization happens very early, on >> the first call to `os::malloc()`. And those calls happen already during >> dynamic C++ initialization - a long time before the VM gets around parsing >> arguments. So, regular VM argument parsing is too late to parse NMT >> arguments. >> >> The current solution is to pass NMT arguments via a specially prepared >> environment variable: `NMT_LEVEL_=`. That environment >> variable has to be set by the embedding launcher, before it loads the >> libjvm. Since its name contains the PID, we cannot even set that variable in >> the shell before starting the launcher. >> >> All that means that every launcher needs to especially parse and process the >> NMT arguments given at the command line (or via whatever method) and prepare >> the environment variable. `java` itself does this. This only works before >> the libjvm.so is loaded, before its dynamic C++ initialization. For that >> reason, it does not work if the launcher links statically against the >> hotspot, since in that case C++ initialization of the launcher and hotspot >> are folded into one phase with no possibility of executing code beforehand. >> >> And since it bypasses argument handling in the VM, it bypasses a number of >> argument processing ways, e.g., `JAVA_TOOL_OPTIONS`. >> >> -- >> >> This patch fixes these shortcomings by making NMT late-initializable: it can >> now be initialized after normal VM argument parsing, like all other parts of >> the VM. This greatly simplifies NMT initialization and makes it work >> automagically for every third party launcher, as well as within our gtests. >> >> The glaring problem with late-initializing NMT is the NMT malloc headers. If >> we rule out just always having them (unacceptable in terms of memory >> overhead), there is no safe way to determine, in os::free(), if an >> allocation came from before or after NMT initialization ran, and therefore >> what to do with its malloc headers. For a more extensive explanation, please >> see the comment block `nmtPreInit.hpp` and the discussion with @kimbarrett >> and @zhengyu123 in the JBS comment section. >> >> The heart of this patch is a new way to track early, pre-NMT-init >> allocations. These are tracked via a lookup table. This was a suggestion by >> Kim and it worked out well. >> >> Changes in detail: >> >> - pre-NMT-init handling: >> - the new files `nmtPreInit.hpp/cpp` take case of NMT pre-init >> handling. They contain a small global lookup table managing C-heap blocks >> allocated in the pre-NMT-init phase. >> - `os::malloc()/os::realloc()/os::free()` defer to this code before >> doing anything else. >> - Please see the extensive comment block at the start of >> `nmtPreinit.hpp` explaining the details. >> >> - Changes to NMT: >> - Before, NMT initialization was spread over two phases, `initialize()` >> and `late_initialize()`. Those were merged into one and simplified - there >> is only one initialization now which happens after argument parsing. >> - Minor changes were needed for the `NMT_TrackingLevel` enum - to >> simplify code, I changed NMT_unknown to be numerically 0. A new comment >> block in `nmtCommon.hpp` now clearly specifies what's what, including >> allowed level state transitions. >> - New utility functions to translate tracking level from/to strings >> added to `NMTUtil` >> - NMT has never been able to handle virtual memory
Re: RFR: JDK-8256844: Make NMT late-initializable
On Thu, 22 Jul 2021 14:58:47 GMT, Thomas Stuefe wrote: > Short: this patch makes NMT available in custom-launcher scenarios and during > gtests. It simplifies NMT initialization. It adds a lot of NMT-specific > testing, cleans them up and makes them sideeffect-free. > > - > > NMT continues to be an extremely useful tool for SAP to tackle memory > problems in the JVM. > > However, NMT is of limited use due to the following restrictions: > > - NMT cannot be used if the hotspot is embedded into a custom launcher unless > the launcher actively cooperates. Just creating and invoking the JVM is not > enough, it needs to do some steps prior to loading the hotspot. This > limitation is not well known (nor, do I believe, documented). Many products > don't do this, e.g., you cannot use NMT with IntelliJ. For us at SAP this > problem limits NMT usefulness greatly since our VMs are often embedded into > custom launchers and modifying every launcher is impossible. > - Worse, if that custom launcher links the libjvm *statically* there is just > no way to activate NMT at all. This is the reason NMT cannot be used in the > `gtestlauncher`. > - Related to that is that we cannot pass NMT options via `JAVA_TOOL_OPTIONS` > and `-XX:Flags=`. > - The fact that NMT cannot be used in gtests is really a pity since it would > allow us to both test NMT itself more rigorously and check for memory leaks > while testing other stuff. > > The reason for all this is that NMT initialization happens very early, on the > first call to `os::malloc()`. And those calls happen already during dynamic > C++ initialization - a long time before the VM gets around parsing arguments. > So, regular VM argument parsing is too late to parse NMT arguments. > > The current solution is to pass NMT arguments via a specially prepared > environment variable: `NMT_LEVEL_=`. That environment > variable has to be set by the embedding launcher, before it loads the libjvm. > Since its name contains the PID, we cannot even set that variable in the > shell before starting the launcher. > > All that means that every launcher needs to especially parse and process the > NMT arguments given at the command line (or via whatever method) and prepare > the environment variable. `java` itself does this. This only works before the > libjvm.so is loaded, before its dynamic C++ initialization. For that reason, > it does not work if the launcher links statically against the hotspot, since > in that case C++ initialization of the launcher and hotspot are folded into > one phase with no possibility of executing code beforehand. > > And since it bypasses argument handling in the VM, it bypasses a number of > argument processing ways, e.g., `JAVA_TOOL_OPTIONS`. > > -- > > This patch fixes these shortcomings by making NMT late-initializable: it can > now be initialized after normal VM argument parsing, like all other parts of > the VM. This greatly simplifies NMT initialization and makes it work > automagically for every third party launcher, as well as within our gtests. > > The glaring problem with late-initializing NMT is the NMT malloc headers. If > we rule out just always having them (unacceptable in terms of memory > overhead), there is no safe way to determine, in os::free(), if an allocation > came from before or after NMT initialization ran, and therefore what to do > with its malloc headers. For a more extensive explanation, please see the > comment block `nmtPreInit.hpp` and the discussion with @kimbarrett and > @zhengyu123 in the JBS comment section. > > The heart of this patch is a new way to track early, pre-NMT-init > allocations. These are tracked via a lookup table. This was a suggestion by > Kim and it worked out well. > > Changes in detail: > > - pre-NMT-init handling: > - the new files `nmtPreInit.hpp/cpp` take case of NMT pre-init > handling. They contain a small global lookup table managing C-heap blocks > allocated in the pre-NMT-init phase. > - `os::malloc()/os::realloc()/os::free()` defer to this code before > doing anything else. > - Please see the extensive comment block at the start of > `nmtPreinit.hpp` explaining the details. > > - Changes to NMT: > - Before, NMT initialization was spread over two phases, `initialize()` > and `late_initialize()`. Those were merged into one and simplified - there is > only one initialization now which happens after argument parsing. > - Minor changes were needed for the `NMT_TrackingLevel` enum - to > simplify code, I changed NMT_unknown to be numerically 0. A new comment block > in `nmtCommon.hpp` now clearly specifies what's what, including allowed level > state transitions. > - New utility functions to translate tracking level from/to strings > added to `NMTUtil` > - NMT has never been able to handle virtual memory allocations before > initialization, which is fine since os::reserve_memory() is not
Re: RFR: JDK-8256844: Make NMT late-initializable
On Fri, 30 Jul 2021 18:28:44 GMT, Zhengyu Gu wrote: >> So, you are saying that there is no memory that is malloc'd pre-NMT-init >> phase and freed post-NMT-init phase? > > Okay, I see. It just leaks those memory, so the table can be read-only. Exactly. There are a few allocs that either get free'd or realloc'ed post-init, but not enough to make freeing them worth it if it means having to serialize access to the lookup table. - PR: https://git.openjdk.java.net/jdk/pull/4874
Re: RFR: JDK-8256844: Make NMT late-initializable
On Fri, 30 Jul 2021 16:36:41 GMT, Thomas Stuefe wrote: >> src/hotspot/share/services/nmtPreInit.hpp line 309: >> >>> 307: ::memcpy(p_new, a->payload(), MIN2(a->size, new_size)); >>> 308: (*rc) = p_new; >>> 309: _num_reallocs_pre_to_post++; >> >> post-NMT-init counter updates are racy. > > True, this is racy. It's just for diagnostics though - I rather remove them > than make them atomic since we would pay for this with every malloc. Or, > maybe atomic + debug only? Either one is fine. - PR: https://git.openjdk.java.net/jdk/pull/4874
Re: RFR: JDK-8256844: Make NMT late-initializable
On Fri, 30 Jul 2021 16:33:17 GMT, Zhengyu Gu wrote: >> The code is implicitly thread-safe because the hashmap is only modified in >> the pre-NMT-init phase. After NMT initialization, the table is read-only. >> During pre-NMT-init we are effectively single-threaded - at most two threads >> access the map, the thread loading the libjvm, and the thread calling >> CreateJavaVM, but not at the same time. >> >> See also the asserts in the AllStatic class `NMTPreInit`. >> >> (I should have described it more clearly, will add a comment.) > > So, you are saying that there is no memory that is malloc'd pre-NMT-init > phase and freed post-NMT-init phase? Okay, I see. It just leaks those memory, so the table can be read-only. - PR: https://git.openjdk.java.net/jdk/pull/4874
Re: RFR: JDK-8256844: Make NMT late-initializable
On Fri, 30 Jul 2021 16:28:54 GMT, Thomas Stuefe wrote: >> src/hotspot/share/services/nmtPreInit.hpp line 202: >> >>> 200: assert((*aa) != NULL, "Entry not found: " PTR_FORMAT, p2i(p)); >>> 201: NMTPreInitAllocation* a = (*aa); >>> 202: (*aa) = (*aa)->next; // remove from its list >> >> Could this be a race? There is no synchronization, read/write result could >> be arbitrary. > > The code is implicitly thread-safe because the hashmap is only modified in > the pre-NMT-init phase. After NMT initialization, the table is read-only. > During pre-NMT-init we are effectively single-threaded - at most two threads > access the map, the thread loading the libjvm, and the thread calling > CreateJavaVM, but not at the same time. > > See also the asserts in the AllStatic class `NMTPreInit`. > > (I should have described it more clearly, will add a comment.) So, you are saying that there is no memory that is malloc'd pre-NMT-init phase and freed post-NMT-init phase? - PR: https://git.openjdk.java.net/jdk/pull/4874
Re: RFR: JDK-8256844: Make NMT late-initializable
On Fri, 30 Jul 2021 13:03:32 GMT, Zhengyu Gu wrote: > Sorry for late review. > > Did a quick scan and have a few questions, will do more detail reading later. Thanks a lot, I appreciate your feedback! > src/hotspot/share/services/nmtPreInit.hpp line 108: > >> 106: // - lookup speed is paramount since lookup is done for every >> os::free() call. >> 107: // - insert/delete speed only matters for VM startup - after NMT >> initialization the lookup >> 108: // table is readonly > > This comment does not seem to be true, you have find_and_remove() that alters > table. The point is *after NMT initialization* - `find_and_remove` only gets called before NMT initialization; after that, we only do non-modifying lookup. You'll find the logic in `NMTPreInit::handle_realloc()` and `NMTPreInit::handle_free()`, respectively. The basic idea behind this is that we remove pointers from the map as long as we can without risking concurrency issues, which is until NMT initialization. After that, we leave the map alone. It was either that or protect the map with a lock, and this is the lesser of two evils since the map is usually sparsely populated. > src/hotspot/share/services/nmtPreInit.hpp line 309: > >> 307: ::memcpy(p_new, a->payload(), MIN2(a->size, new_size)); >> 308: (*rc) = p_new; >> 309: _num_reallocs_pre_to_post++; > > post-NMT-init counter updates are racy. True, this is racy. It's just for diagnostics though - I rather remove them than make them atomic since we would pay for this with every malloc. Or, maybe atomic + debug only? - PR: https://git.openjdk.java.net/jdk/pull/4874
Re: RFR: JDK-8256844: Make NMT late-initializable
On Fri, 30 Jul 2021 12:56:59 GMT, Zhengyu Gu wrote: >> Short: this patch makes NMT available in custom-launcher scenarios and >> during gtests. It simplifies NMT initialization. It adds a lot of >> NMT-specific testing, cleans them up and makes them sideeffect-free. >> >> - >> >> NMT continues to be an extremely useful tool for SAP to tackle memory >> problems in the JVM. >> >> However, NMT is of limited use due to the following restrictions: >> >> - NMT cannot be used if the hotspot is embedded into a custom launcher >> unless the launcher actively cooperates. Just creating and invoking the JVM >> is not enough, it needs to do some steps prior to loading the hotspot. This >> limitation is not well known (nor, do I believe, documented). Many products >> don't do this, e.g., you cannot use NMT with IntelliJ. For us at SAP this >> problem limits NMT usefulness greatly since our VMs are often embedded into >> custom launchers and modifying every launcher is impossible. >> - Worse, if that custom launcher links the libjvm *statically* there is just >> no way to activate NMT at all. This is the reason NMT cannot be used in the >> `gtestlauncher`. >> - Related to that is that we cannot pass NMT options via `JAVA_TOOL_OPTIONS` >> and `-XX:Flags=`. >> - The fact that NMT cannot be used in gtests is really a pity since it would >> allow us to both test NMT itself more rigorously and check for memory leaks >> while testing other stuff. >> >> The reason for all this is that NMT initialization happens very early, on >> the first call to `os::malloc()`. And those calls happen already during >> dynamic C++ initialization - a long time before the VM gets around parsing >> arguments. So, regular VM argument parsing is too late to parse NMT >> arguments. >> >> The current solution is to pass NMT arguments via a specially prepared >> environment variable: `NMT_LEVEL_=`. That environment >> variable has to be set by the embedding launcher, before it loads the >> libjvm. Since its name contains the PID, we cannot even set that variable in >> the shell before starting the launcher. >> >> All that means that every launcher needs to especially parse and process the >> NMT arguments given at the command line (or via whatever method) and prepare >> the environment variable. `java` itself does this. This only works before >> the libjvm.so is loaded, before its dynamic C++ initialization. For that >> reason, it does not work if the launcher links statically against the >> hotspot, since in that case C++ initialization of the launcher and hotspot >> are folded into one phase with no possibility of executing code beforehand. >> >> And since it bypasses argument handling in the VM, it bypasses a number of >> argument processing ways, e.g., `JAVA_TOOL_OPTIONS`. >> >> -- >> >> This patch fixes these shortcomings by making NMT late-initializable: it can >> now be initialized after normal VM argument parsing, like all other parts of >> the VM. This greatly simplifies NMT initialization and makes it work >> automagically for every third party launcher, as well as within our gtests. >> >> The glaring problem with late-initializing NMT is the NMT malloc headers. If >> we rule out just always having them (unacceptable in terms of memory >> overhead), there is no safe way to determine, in os::free(), if an >> allocation came from before or after NMT initialization ran, and therefore >> what to do with its malloc headers. For a more extensive explanation, please >> see the comment block `nmtPreInit.hpp` and the discussion with @kimbarrett >> and @zhengyu123 in the JBS comment section. >> >> The heart of this patch is a new way to track early, pre-NMT-init >> allocations. These are tracked via a lookup table. This was a suggestion by >> Kim and it worked out well. >> >> Changes in detail: >> >> - pre-NMT-init handling: >> - the new files `nmtPreInit.hpp/cpp` take case of NMT pre-init >> handling. They contain a small global lookup table managing C-heap blocks >> allocated in the pre-NMT-init phase. >> - `os::malloc()/os::realloc()/os::free()` defer to this code before >> doing anything else. >> - Please see the extensive comment block at the start of >> `nmtPreinit.hpp` explaining the details. >> >> - Changes to NMT: >> - Before, NMT initialization was spread over two phases, `initialize()` >> and `late_initialize()`. Those were merged into one and simplified - there >> is only one initialization now which happens after argument parsing. >> - Minor changes were needed for the `NMT_TrackingLevel` enum - to >> simplify code, I changed NMT_unknown to be numerically 0. A new comment >> block in `nmtCommon.hpp` now clearly specifies what's what, including >> allowed level state transitions. >> - New utility functions to translate tracking level from/to strings >> added to `NMTUtil` >> - NMT has never been able to handle virtual memory
Re: RFR: JDK-8256844: Make NMT late-initializable
On Thu, 22 Jul 2021 14:58:47 GMT, Thomas Stuefe wrote: > Short: this patch makes NMT available in custom-launcher scenarios and during > gtests. It simplifies NMT initialization. It adds a lot of NMT-specific > testing, cleans them up and makes them sideeffect-free. > > - > > NMT continues to be an extremely useful tool for SAP to tackle memory > problems in the JVM. > > However, NMT is of limited use due to the following restrictions: > > - NMT cannot be used if the hotspot is embedded into a custom launcher unless > the launcher actively cooperates. Just creating and invoking the JVM is not > enough, it needs to do some steps prior to loading the hotspot. This > limitation is not well known (nor, do I believe, documented). Many products > don't do this, e.g., you cannot use NMT with IntelliJ. For us at SAP this > problem limits NMT usefulness greatly since our VMs are often embedded into > custom launchers and modifying every launcher is impossible. > - Worse, if that custom launcher links the libjvm *statically* there is just > no way to activate NMT at all. This is the reason NMT cannot be used in the > `gtestlauncher`. > - Related to that is that we cannot pass NMT options via `JAVA_TOOL_OPTIONS` > and `-XX:Flags=`. > - The fact that NMT cannot be used in gtests is really a pity since it would > allow us to both test NMT itself more rigorously and check for memory leaks > while testing other stuff. > > The reason for all this is that NMT initialization happens very early, on the > first call to `os::malloc()`. And those calls happen already during dynamic > C++ initialization - a long time before the VM gets around parsing arguments. > So, regular VM argument parsing is too late to parse NMT arguments. > > The current solution is to pass NMT arguments via a specially prepared > environment variable: `NMT_LEVEL_=`. That environment > variable has to be set by the embedding launcher, before it loads the libjvm. > Since its name contains the PID, we cannot even set that variable in the > shell before starting the launcher. > > All that means that every launcher needs to especially parse and process the > NMT arguments given at the command line (or via whatever method) and prepare > the environment variable. `java` itself does this. This only works before the > libjvm.so is loaded, before its dynamic C++ initialization. For that reason, > it does not work if the launcher links statically against the hotspot, since > in that case C++ initialization of the launcher and hotspot are folded into > one phase with no possibility of executing code beforehand. > > And since it bypasses argument handling in the VM, it bypasses a number of > argument processing ways, e.g., `JAVA_TOOL_OPTIONS`. > > -- > > This patch fixes these shortcomings by making NMT late-initializable: it can > now be initialized after normal VM argument parsing, like all other parts of > the VM. This greatly simplifies NMT initialization and makes it work > automagically for every third party launcher, as well as within our gtests. > > The glaring problem with late-initializing NMT is the NMT malloc headers. If > we rule out just always having them (unacceptable in terms of memory > overhead), there is no safe way to determine, in os::free(), if an allocation > came from before or after NMT initialization ran, and therefore what to do > with its malloc headers. For a more extensive explanation, please see the > comment block `nmtPreInit.hpp` and the discussion with @kimbarrett and > @zhengyu123 in the JBS comment section. > > The heart of this patch is a new way to track early, pre-NMT-init > allocations. These are tracked via a lookup table. This was a suggestion by > Kim and it worked out well. > > Changes in detail: > > - pre-NMT-init handling: > - the new files `nmtPreInit.hpp/cpp` take case of NMT pre-init > handling. They contain a small global lookup table managing C-heap blocks > allocated in the pre-NMT-init phase. > - `os::malloc()/os::realloc()/os::free()` defer to this code before > doing anything else. > - Please see the extensive comment block at the start of > `nmtPreinit.hpp` explaining the details. > > - Changes to NMT: > - Before, NMT initialization was spread over two phases, `initialize()` > and `late_initialize()`. Those were merged into one and simplified - there is > only one initialization now which happens after argument parsing. > - Minor changes were needed for the `NMT_TrackingLevel` enum - to > simplify code, I changed NMT_unknown to be numerically 0. A new comment block > in `nmtCommon.hpp` now clearly specifies what's what, including allowed level > state transitions. > - New utility functions to translate tracking level from/to strings > added to `NMTUtil` > - NMT has never been able to handle virtual memory allocations before > initialization, which is fine since os::reserve_memory() is not
Re: RFR: JDK-8256844: Make NMT late-initializable
On Fri, 30 Jul 2021 04:09:32 GMT, Kim Barrett wrote:
>> src/hotspot/share/services/nmtPreInit.hpp line 166:
>>
>>> 164: NMTPreInitAllocation** find_entry(const void* p) const {
>>> 165: const unsigned index = index_for_key(p);
>>> 166: NMTPreInitAllocation** aa = (NMTPreInitAllocation**)
>>> (&(_entries[index]));
>>
>> Why is this cast needed?
>
> [Not a review, just a drive-by comment.] It's casting away const. Better
> would be a const_cast. And probably moved to the final result, with the body
> keeping things const-qualified. And maybe const and non-const overloads of
> this function. Or maybe this function shouldn't be const-qualified if a
> non-const result is always needed, but that doesn't seem likely.
I'll rethink this.
-
PR: https://git.openjdk.java.net/jdk/pull/4874
Re: RFR: JDK-8256844: Make NMT late-initializable
On Thu, 29 Jul 2021 23:03:46 GMT, Coleen Phillimore wrote: > This is an interesting and it seems a better way to solve this problem. Where > were you all those years ago?? I hope @zhengyu123 has a chance to review it. Thank you! I was here, but we were not yet doing much upstream :) To be fair, this problem got quite involved and did cost me some cycles and false starts. I fully understand that the first solution uses the environment variable approach. I spend some time investigating different ideas with this one; at first I did not use a hash-table but a static pre-allocated buffer from which I fed early allocations. But the code got too complex, and Kim's suggestion with the side table turned out just to be a lot simpler. > Also interesting is that we were wondering how we could return malloc'd > memory on JVM creation failure, and this might partially help with that > larger problem. Yes, this would be trivial now, to return that memory. Though I am afraid it would be a small part only. But NMT may be instrumental in releasing all memory, since it knows everything. Only, its not always enabled. Is that a real-life problem? Are there cases where the launcher would want to live on if the JVM failed to load? - PR: https://git.openjdk.java.net/jdk/pull/4874
Re: RFR: JDK-8256844: Make NMT late-initializable
On Thu, 29 Jul 2021 06:37:36 GMT, David Holmes wrote: > Hi Thomas, > > I had a look through this and it seems reasonable, but I'm not familiar > enough with the details to approve at this stage. > > A few nits below. > > Thanks, > David I did not expect a quick review for this one, so thanks for looking at this! All your suggestions make sense, I'll incorporate them. ..Thomas - PR: https://git.openjdk.java.net/jdk/pull/4874
Re: RFR: JDK-8256844: Make NMT late-initializable
On Fri, 30 Jul 2021 04:03:57 GMT, Kim Barrett wrote:
>> src/hotspot/share/services/nmtPreInit.hpp line 128:
>>
>>> 126: // Returns start of the user data area
>>> 127: void* payload() { return this + 1; }
>>> 128: const void* payload() const { return this + 1; }
>>
>> This is an odd looking overload (that I wouldn't have thought possible).
>> Maybe one of these payloads can be renamed to why it's const.
>
> [Not a review, just a drive-by comment.] This is a normal and idiomatic
> overload on the const-ness of `this`.
To expand on Kim's answer. This is a way to solve const/nonconst problems like
https://github.com/openjdk/jdk/pull/4938/files#r679639391.
You get a const version (suitably returning a write-protected pointer) which
the compiler chooses in const code, and a non-const version for non-const code,
and no awkward const-casts are needed from the outside.
In this case the implementation is simple enough that I just duplicated it;
were it more complex, I'd call one in terms of the other. We do this in other
places too, see e.g. `ResourceHashTable::lookup_node`.
-
PR: https://git.openjdk.java.net/jdk/pull/4874
Re: RFR: JDK-8256844: Make NMT late-initializable
On Thu, 29 Jul 2021 22:53:42 GMT, Coleen Phillimore wrote: >> Short: this patch makes NMT available in custom-launcher scenarios and >> during gtests. It simplifies NMT initialization. It adds a lot of >> NMT-specific testing, cleans them up and makes them sideeffect-free. >> >> - >> >> NMT continues to be an extremely useful tool for SAP to tackle memory >> problems in the JVM. >> >> However, NMT is of limited use due to the following restrictions: >> >> - NMT cannot be used if the hotspot is embedded into a custom launcher >> unless the launcher actively cooperates. Just creating and invoking the JVM >> is not enough, it needs to do some steps prior to loading the hotspot. This >> limitation is not well known (nor, do I believe, documented). Many products >> don't do this, e.g., you cannot use NMT with IntelliJ. For us at SAP this >> problem limits NMT usefulness greatly since our VMs are often embedded into >> custom launchers and modifying every launcher is impossible. >> - Worse, if that custom launcher links the libjvm *statically* there is just >> no way to activate NMT at all. This is the reason NMT cannot be used in the >> `gtestlauncher`. >> - Related to that is that we cannot pass NMT options via `JAVA_TOOL_OPTIONS` >> and `-XX:Flags=`. >> - The fact that NMT cannot be used in gtests is really a pity since it would >> allow us to both test NMT itself more rigorously and check for memory leaks >> while testing other stuff. >> >> The reason for all this is that NMT initialization happens very early, on >> the first call to `os::malloc()`. And those calls happen already during >> dynamic C++ initialization - a long time before the VM gets around parsing >> arguments. So, regular VM argument parsing is too late to parse NMT >> arguments. >> >> The current solution is to pass NMT arguments via a specially prepared >> environment variable: `NMT_LEVEL_=`. That environment >> variable has to be set by the embedding launcher, before it loads the >> libjvm. Since its name contains the PID, we cannot even set that variable in >> the shell before starting the launcher. >> >> All that means that every launcher needs to especially parse and process the >> NMT arguments given at the command line (or via whatever method) and prepare >> the environment variable. `java` itself does this. This only works before >> the libjvm.so is loaded, before its dynamic C++ initialization. For that >> reason, it does not work if the launcher links statically against the >> hotspot, since in that case C++ initialization of the launcher and hotspot >> are folded into one phase with no possibility of executing code beforehand. >> >> And since it bypasses argument handling in the VM, it bypasses a number of >> argument processing ways, e.g., `JAVA_TOOL_OPTIONS`. >> >> -- >> >> This patch fixes these shortcomings by making NMT late-initializable: it can >> now be initialized after normal VM argument parsing, like all other parts of >> the VM. This greatly simplifies NMT initialization and makes it work >> automagically for every third party launcher, as well as within our gtests. >> >> The glaring problem with late-initializing NMT is the NMT malloc headers. If >> we rule out just always having them (unacceptable in terms of memory >> overhead), there is no safe way to determine, in os::free(), if an >> allocation came from before or after NMT initialization ran, and therefore >> what to do with its malloc headers. For a more extensive explanation, please >> see the comment block `nmtPreInit.hpp` and the discussion with @kimbarrett >> and @zhengyu123 in the JBS comment section. >> >> The heart of this patch is a new way to track early, pre-NMT-init >> allocations. These are tracked via a lookup table. This was a suggestion by >> Kim and it worked out well. >> >> Changes in detail: >> >> - pre-NMT-init handling: >> - the new files `nmtPreInit.hpp/cpp` take case of NMT pre-init >> handling. They contain a small global lookup table managing C-heap blocks >> allocated in the pre-NMT-init phase. >> - `os::malloc()/os::realloc()/os::free()` defer to this code before >> doing anything else. >> - Please see the extensive comment block at the start of >> `nmtPreinit.hpp` explaining the details. >> >> - Changes to NMT: >> - Before, NMT initialization was spread over two phases, `initialize()` >> and `late_initialize()`. Those were merged into one and simplified - there >> is only one initialization now which happens after argument parsing. >> - Minor changes were needed for the `NMT_TrackingLevel` enum - to >> simplify code, I changed NMT_unknown to be numerically 0. A new comment >> block in `nmtCommon.hpp` now clearly specifies what's what, including >> allowed level state transitions. >> - New utility functions to translate tracking level from/to strings >> added to `NMTUtil` >> - NMT has never been able to handle virtual
Re: RFR: JDK-8256844: Make NMT late-initializable
On Thu, 29 Jul 2021 22:52:07 GMT, Coleen Phillimore wrote: >> Short: this patch makes NMT available in custom-launcher scenarios and >> during gtests. It simplifies NMT initialization. It adds a lot of >> NMT-specific testing, cleans them up and makes them sideeffect-free. >> >> - >> >> NMT continues to be an extremely useful tool for SAP to tackle memory >> problems in the JVM. >> >> However, NMT is of limited use due to the following restrictions: >> >> - NMT cannot be used if the hotspot is embedded into a custom launcher >> unless the launcher actively cooperates. Just creating and invoking the JVM >> is not enough, it needs to do some steps prior to loading the hotspot. This >> limitation is not well known (nor, do I believe, documented). Many products >> don't do this, e.g., you cannot use NMT with IntelliJ. For us at SAP this >> problem limits NMT usefulness greatly since our VMs are often embedded into >> custom launchers and modifying every launcher is impossible. >> - Worse, if that custom launcher links the libjvm *statically* there is just >> no way to activate NMT at all. This is the reason NMT cannot be used in the >> `gtestlauncher`. >> - Related to that is that we cannot pass NMT options via `JAVA_TOOL_OPTIONS` >> and `-XX:Flags=`. >> - The fact that NMT cannot be used in gtests is really a pity since it would >> allow us to both test NMT itself more rigorously and check for memory leaks >> while testing other stuff. >> >> The reason for all this is that NMT initialization happens very early, on >> the first call to `os::malloc()`. And those calls happen already during >> dynamic C++ initialization - a long time before the VM gets around parsing >> arguments. So, regular VM argument parsing is too late to parse NMT >> arguments. >> >> The current solution is to pass NMT arguments via a specially prepared >> environment variable: `NMT_LEVEL_=`. That environment >> variable has to be set by the embedding launcher, before it loads the >> libjvm. Since its name contains the PID, we cannot even set that variable in >> the shell before starting the launcher. >> >> All that means that every launcher needs to especially parse and process the >> NMT arguments given at the command line (or via whatever method) and prepare >> the environment variable. `java` itself does this. This only works before >> the libjvm.so is loaded, before its dynamic C++ initialization. For that >> reason, it does not work if the launcher links statically against the >> hotspot, since in that case C++ initialization of the launcher and hotspot >> are folded into one phase with no possibility of executing code beforehand. >> >> And since it bypasses argument handling in the VM, it bypasses a number of >> argument processing ways, e.g., `JAVA_TOOL_OPTIONS`. >> >> -- >> >> This patch fixes these shortcomings by making NMT late-initializable: it can >> now be initialized after normal VM argument parsing, like all other parts of >> the VM. This greatly simplifies NMT initialization and makes it work >> automagically for every third party launcher, as well as within our gtests. >> >> The glaring problem with late-initializing NMT is the NMT malloc headers. If >> we rule out just always having them (unacceptable in terms of memory >> overhead), there is no safe way to determine, in os::free(), if an >> allocation came from before or after NMT initialization ran, and therefore >> what to do with its malloc headers. For a more extensive explanation, please >> see the comment block `nmtPreInit.hpp` and the discussion with @kimbarrett >> and @zhengyu123 in the JBS comment section. >> >> The heart of this patch is a new way to track early, pre-NMT-init >> allocations. These are tracked via a lookup table. This was a suggestion by >> Kim and it worked out well. >> >> Changes in detail: >> >> - pre-NMT-init handling: >> - the new files `nmtPreInit.hpp/cpp` take case of NMT pre-init >> handling. They contain a small global lookup table managing C-heap blocks >> allocated in the pre-NMT-init phase. >> - `os::malloc()/os::realloc()/os::free()` defer to this code before >> doing anything else. >> - Please see the extensive comment block at the start of >> `nmtPreinit.hpp` explaining the details. >> >> - Changes to NMT: >> - Before, NMT initialization was spread over two phases, `initialize()` >> and `late_initialize()`. Those were merged into one and simplified - there >> is only one initialization now which happens after argument parsing. >> - Minor changes were needed for the `NMT_TrackingLevel` enum - to >> simplify code, I changed NMT_unknown to be numerically 0. A new comment >> block in `nmtCommon.hpp` now clearly specifies what's what, including >> allowed level state transitions. >> - New utility functions to translate tracking level from/to strings >> added to `NMTUtil` >> - NMT has never been able to handle virtual
Re: RFR: JDK-8256844: Make NMT late-initializable
On Thu, 22 Jul 2021 14:58:47 GMT, Thomas Stuefe wrote: > Short: this patch makes NMT available in custom-launcher scenarios and during > gtests. It simplifies NMT initialization. It adds a lot of NMT-specific > testing, cleans them up and makes them sideeffect-free. > > - > > NMT continues to be an extremely useful tool for SAP to tackle memory > problems in the JVM. > > However, NMT is of limited use due to the following restrictions: > > - NMT cannot be used if the hotspot is embedded into a custom launcher unless > the launcher actively cooperates. Just creating and invoking the JVM is not > enough, it needs to do some steps prior to loading the hotspot. This > limitation is not well known (nor, do I believe, documented). Many products > don't do this, e.g., you cannot use NMT with IntelliJ. For us at SAP this > problem limits NMT usefulness greatly since our VMs are often embedded into > custom launchers and modifying every launcher is impossible. > - Worse, if that custom launcher links the libjvm *statically* there is just > no way to activate NMT at all. This is the reason NMT cannot be used in the > `gtestlauncher`. > - Related to that is that we cannot pass NMT options via `JAVA_TOOL_OPTIONS` > and `-XX:Flags=`. > - The fact that NMT cannot be used in gtests is really a pity since it would > allow us to both test NMT itself more rigorously and check for memory leaks > while testing other stuff. > > The reason for all this is that NMT initialization happens very early, on the > first call to `os::malloc()`. And those calls happen already during dynamic > C++ initialization - a long time before the VM gets around parsing arguments. > So, regular VM argument parsing is too late to parse NMT arguments. > > The current solution is to pass NMT arguments via a specially prepared > environment variable: `NMT_LEVEL_=`. That environment > variable has to be set by the embedding launcher, before it loads the libjvm. > Since its name contains the PID, we cannot even set that variable in the > shell before starting the launcher. > > All that means that every launcher needs to especially parse and process the > NMT arguments given at the command line (or via whatever method) and prepare > the environment variable. `java` itself does this. This only works before the > libjvm.so is loaded, before its dynamic C++ initialization. For that reason, > it does not work if the launcher links statically against the hotspot, since > in that case C++ initialization of the launcher and hotspot are folded into > one phase with no possibility of executing code beforehand. > > And since it bypasses argument handling in the VM, it bypasses a number of > argument processing ways, e.g., `JAVA_TOOL_OPTIONS`. > > -- > > This patch fixes these shortcomings by making NMT late-initializable: it can > now be initialized after normal VM argument parsing, like all other parts of > the VM. This greatly simplifies NMT initialization and makes it work > automagically for every third party launcher, as well as within our gtests. > > The glaring problem with late-initializing NMT is the NMT malloc headers. If > we rule out just always having them (unacceptable in terms of memory > overhead), there is no safe way to determine, in os::free(), if an allocation > came from before or after NMT initialization ran, and therefore what to do > with its malloc headers. For a more extensive explanation, please see the > comment block `nmtPreInit.hpp` and the discussion with @kimbarrett and > @zhengyu123 in the JBS comment section. > > The heart of this patch is a new way to track early, pre-NMT-init > allocations. These are tracked via a lookup table. This was a suggestion by > Kim and it worked out well. > > Changes in detail: > > - pre-NMT-init handling: > - the new files `nmtPreInit.hpp/cpp` take case of NMT pre-init > handling. They contain a small global lookup table managing C-heap blocks > allocated in the pre-NMT-init phase. > - `os::malloc()/os::realloc()/os::free()` defer to this code before > doing anything else. > - Please see the extensive comment block at the start of > `nmtPreinit.hpp` explaining the details. > > - Changes to NMT: > - Before, NMT initialization was spread over two phases, `initialize()` > and `late_initialize()`. Those were merged into one and simplified - there is > only one initialization now which happens after argument parsing. > - Minor changes were needed for the `NMT_TrackingLevel` enum - to > simplify code, I changed NMT_unknown to be numerically 0. A new comment block > in `nmtCommon.hpp` now clearly specifies what's what, including allowed level > state transitions. > - New utility functions to translate tracking level from/to strings > added to `NMTUtil` > - NMT has never been able to handle virtual memory allocations before > initialization, which is fine since os::reserve_memory() is not
Re: RFR: JDK-8256844: Make NMT late-initializable
On Thu, 22 Jul 2021 14:58:47 GMT, Thomas Stuefe wrote: > Short: this patch makes NMT available in custom-launcher scenarios and during > gtests. It simplifies NMT initialization. It adds a lot of NMT-specific > testing, cleans them up and makes them sideeffect-free. > > - > > NMT continues to be an extremely useful tool for SAP to tackle memory > problems in the JVM. > > However, NMT is of limited use due to the following restrictions: > > - NMT cannot be used if the hotspot is embedded into a custom launcher unless > the launcher actively cooperates. Just creating and invoking the JVM is not > enough, it needs to do some steps prior to loading the hotspot. This > limitation is not well known (nor, do I believe, documented). Many products > don't do this, e.g., you cannot use NMT with IntelliJ. For us at SAP this > problem limits NMT usefulness greatly since our VMs are often embedded into > custom launchers and modifying every launcher is impossible. > - Worse, if that custom launcher links the libjvm *statically* there is just > no way to activate NMT at all. This is the reason NMT cannot be used in the > `gtestlauncher`. > - Related to that is that we cannot pass NMT options via `JAVA_TOOL_OPTIONS` > and `-XX:Flags=`. > - The fact that NMT cannot be used in gtests is really a pity since it would > allow us to both test NMT itself more rigorously and check for memory leaks > while testing other stuff. > > The reason for all this is that NMT initialization happens very early, on the > first call to `os::malloc()`. And those calls happen already during dynamic > C++ initialization - a long time before the VM gets around parsing arguments. > So, regular VM argument parsing is too late to parse NMT arguments. > > The current solution is to pass NMT arguments via a specially prepared > environment variable: `NMT_LEVEL_=`. That environment > variable has to be set by the embedding launcher, before it loads the libjvm. > Since its name contains the PID, we cannot even set that variable in the > shell before starting the launcher. > > All that means that every launcher needs to especially parse and process the > NMT arguments given at the command line (or via whatever method) and prepare > the environment variable. `java` itself does this. This only works before the > libjvm.so is loaded, before its dynamic C++ initialization. For that reason, > it does not work if the launcher links statically against the hotspot, since > in that case C++ initialization of the launcher and hotspot are folded into > one phase with no possibility of executing code beforehand. > > And since it bypasses argument handling in the VM, it bypasses a number of > argument processing ways, e.g., `JAVA_TOOL_OPTIONS`. > > -- > > This patch fixes these shortcomings by making NMT late-initializable: it can > now be initialized after normal VM argument parsing, like all other parts of > the VM. This greatly simplifies NMT initialization and makes it work > automagically for every third party launcher, as well as within our gtests. > > The glaring problem with late-initializing NMT is the NMT malloc headers. If > we rule out just always having them (unacceptable in terms of memory > overhead), there is no safe way to determine, in os::free(), if an allocation > came from before or after NMT initialization ran, and therefore what to do > with its malloc headers. For a more extensive explanation, please see the > comment block `nmtPreInit.hpp` and the discussion with @kimbarrett and > @zhengyu123 in the JBS comment section. > > The heart of this patch is a new way to track early, pre-NMT-init > allocations. These are tracked via a lookup table. This was a suggestion by > Kim and it worked out well. > > Changes in detail: > > - pre-NMT-init handling: > - the new files `nmtPreInit.hpp/cpp` take case of NMT pre-init > handling. They contain a small global lookup table managing C-heap blocks > allocated in the pre-NMT-init phase. > - `os::malloc()/os::realloc()/os::free()` defer to this code before > doing anything else. > - Please see the extensive comment block at the start of > `nmtPreinit.hpp` explaining the details. > > - Changes to NMT: > - Before, NMT initialization was spread over two phases, `initialize()` > and `late_initialize()`. Those were merged into one and simplified - there is > only one initialization now which happens after argument parsing. > - Minor changes were needed for the `NMT_TrackingLevel` enum - to > simplify code, I changed NMT_unknown to be numerically 0. A new comment block > in `nmtCommon.hpp` now clearly specifies what's what, including allowed level > state transitions. > - New utility functions to translate tracking level from/to strings > added to `NMTUtil` > - NMT has never been able to handle virtual memory allocations before > initialization, which is fine since os::reserve_memory() is not
Re: RFR: JDK-8256844: Make NMT late-initializable
On Thu, 22 Jul 2021 14:58:47 GMT, Thomas Stuefe wrote: > Short: this patch makes NMT available in custom-launcher scenarios and during > gtests. It simplifies NMT initialization. It adds a lot of NMT-specific > testing, cleans them up and makes them sideeffect-free. > > - > > NMT continues to be an extremely useful tool for SAP to tackle memory > problems in the JVM. > > However, NMT is of limited use due to the following restrictions: > > - NMT cannot be used if the hotspot is embedded into a custom launcher unless > the launcher actively cooperates. Just creating and invoking the JVM is not > enough, it needs to do some steps prior to loading the hotspot. This > limitation is not well known (nor, do I believe, documented). Many products > don't do this, e.g., you cannot use NMT with IntelliJ. For us at SAP this > problem limits NMT usefulness greatly since our VMs are often embedded into > custom launchers and modifying every launcher is impossible. > - Worse, if that custom launcher links the libjvm *statically* there is just > no way to activate NMT at all. This is the reason NMT cannot be used in the > `gtestlauncher`. > - Related to that is that we cannot pass NMT options via `JAVA_TOOL_OPTIONS` > and `-XX:Flags=`. > - The fact that NMT cannot be used in gtests is really a pity since it would > allow us to both test NMT itself more rigorously and check for memory leaks > while testing other stuff. > > The reason for all this is that NMT initialization happens very early, on the > first call to `os::malloc()`. And those calls happen already during dynamic > C++ initialization - a long time before the VM gets around parsing arguments. > So, regular VM argument parsing is too late to parse NMT arguments. > > The current solution is to pass NMT arguments via a specially prepared > environment variable: `NMT_LEVEL_=`. That environment > variable has to be set by the embedding launcher, before it loads the libjvm. > Since its name contains the PID, we cannot even set that variable in the > shell before starting the launcher. > > All that means that every launcher needs to especially parse and process the > NMT arguments given at the command line (or via whatever method) and prepare > the environment variable. `java` itself does this. This only works before the > libjvm.so is loaded, before its dynamic C++ initialization. For that reason, > it does not work if the launcher links statically against the hotspot, since > in that case C++ initialization of the launcher and hotspot are folded into > one phase with no possibility of executing code beforehand. > > And since it bypasses argument handling in the VM, it bypasses a number of > argument processing ways, e.g., `JAVA_TOOL_OPTIONS`. > > -- > > This patch fixes these shortcomings by making NMT late-initializable: it can > now be initialized after normal VM argument parsing, like all other parts of > the VM. This greatly simplifies NMT initialization and makes it work > automagically for every third party launcher, as well as within our gtests. > > The glaring problem with late-initializing NMT is the NMT malloc headers. If > we rule out just always having them (unacceptable in terms of memory > overhead), there is no safe way to determine, in os::free(), if an allocation > came from before or after NMT initialization ran, and therefore what to do > with its malloc headers. For a more extensive explanation, please see the > comment block `nmtPreInit.hpp` and the discussion with @kimbarrett and > @zhengyu123 in the JBS comment section. > > The heart of this patch is a new way to track early, pre-NMT-init > allocations. These are tracked via a lookup table. This was a suggestion by > Kim and it worked out well. > > Changes in detail: > > - pre-NMT-init handling: > - the new files `nmtPreInit.hpp/cpp` take case of NMT pre-init > handling. They contain a small global lookup table managing C-heap blocks > allocated in the pre-NMT-init phase. > - `os::malloc()/os::realloc()/os::free()` defer to this code before > doing anything else. > - Please see the extensive comment block at the start of > `nmtPreinit.hpp` explaining the details. > > - Changes to NMT: > - Before, NMT initialization was spread over two phases, `initialize()` > and `late_initialize()`. Those were merged into one and simplified - there is > only one initialization now which happens after argument parsing. > - Minor changes were needed for the `NMT_TrackingLevel` enum - to > simplify code, I changed NMT_unknown to be numerically 0. A new comment block > in `nmtCommon.hpp` now clearly specifies what's what, including allowed level > state transitions. > - New utility functions to translate tracking level from/to strings > added to `NMTUtil` > - NMT has never been able to handle virtual memory allocations before > initialization, which is fine since os::reserve_memory() is not
Re: RFR: JDK-8256844: Make NMT late-initializable
On 28/07/2021 12:17 am, Thomas Stuefe wrote: On Mon, 26 Jul 2021 21:08:04 GMT, David Holmes wrote: Before looking at this, have you checked the startup performance impact? Thanks, David - Hi David, performance should not be a problem. The potentially costly thing is the underlying hashmap. But we keep it operating with a very small load factor. More details: Adding entries is O(1). Since during pre-init phase almost only adds happen, startup time is not affected. Still, to make sure this is true, I did a bunch of tests: - tested WCT of a helloworld, no differences with and without patch - tested startup time in various of ways, no differences - repeated those tests with 25000 (!) VM arguments, the only way to influence the number of pre-init allocations. No differences (VM gets slower with and without patch). The expensive thing is lookup since we potentially need to walk a very full hashmap. Lookup affects post-init more than pre-init. To get an idea of the cost of a too-full preinit lookup table, I modified the VM to do a configurable number of pre-init test-allocations, with the intent of artificially inflating the lookup table. Then, after NMT initialization, I measured the cost of lookup. The short story, I was not able to measure anything, even with a million pre-init allocations. Of course, with more allocations lookup table got fuller and the VM got slower, but the time increase was caused by the cost of the malloc calls themselves, not the table lookup. Finally, I did an isolated test for the lookup table, testing pure adding and retrieval cost with artificial values. There, I could see costs for add were static (as expected), and lookup cost increased with table population. On my machine: | lu table entries| time per lookup | | -- |:-:| | 1000| 3 ns | | 1 mio | 240 ns | As you can see, if lookup table population goes beyond 1 mio entries, lookup time starts being noticeable over background noise. But with these numbers, I am not worried. Standard lookup population should be around *300-500*, with very long command lines resulting in table populations of *~1000*. We should never seen 1 entries, let alone millions of them. Still, I added a jtreg test to verify the expected hash table population. To catch errors like an unforeseen mass of pre-init allocations (lets say a leak or badly written code sneaked in), or if the hash algorithm suddenly is not good anymore. Two more points 1) I kept this coding deliberately simple. If we are really worried about a degenerated lookup table, we can do things to fix that: - we could automatically resize and rehash - we could, if we sense something wrong, just stop filling it and disable NMT, stopping NMT init phase prematurely at the cost of not being able to use NMT. The latter I had implemented already but removed it again to keep complexity down, and because I saw no need. 2) In our propietary production VM we have a system similar to NMT, but predating it. In that system we don't use malloc headers but store all (millions of) malloc'ed pointers in a big hash map. It performs excellent on *all our libc variants*. It is so fast that we just leave it always switched on. This solution has been productive since >10 years, and therefore I am confident that this is viable. This proposed hashmap with a planned population of 300-1000 is really not much :) Thanks Thomas! I appreciate the detailed investigation. Cheers, David - PR: https://git.openjdk.java.net/jdk/pull/4874
Re: RFR: JDK-8256844: Make NMT late-initializable
On Mon, 26 Jul 2021 21:08:04 GMT, David Holmes wrote: > Before looking at this, have you checked the startup performance impact? > > Thanks, > David > - Hi David, performance should not be a problem. The potentially costly thing is the underlying hashmap. But we keep it operating with a very small load factor. More details: Adding entries is O(1). Since during pre-init phase almost only adds happen, startup time is not affected. Still, to make sure this is true, I did a bunch of tests: - tested WCT of a helloworld, no differences with and without patch - tested startup time in various of ways, no differences - repeated those tests with 25000 (!) VM arguments, the only way to influence the number of pre-init allocations. No differences (VM gets slower with and without patch). The expensive thing is lookup since we potentially need to walk a very full hashmap. Lookup affects post-init more than pre-init. To get an idea of the cost of a too-full preinit lookup table, I modified the VM to do a configurable number of pre-init test-allocations, with the intent of artificially inflating the lookup table. Then, after NMT initialization, I measured the cost of lookup. The short story, I was not able to measure anything, even with a million pre-init allocations. Of course, with more allocations lookup table got fuller and the VM got slower, but the time increase was caused by the cost of the malloc calls themselves, not the table lookup. Finally, I did an isolated test for the lookup table, testing pure adding and retrieval cost with artificial values. There, I could see costs for add were static (as expected), and lookup cost increased with table population. On my machine: | lu table entries| time per lookup | | -- |:-:| | 1000| 3 ns | | 1 mio | 240 ns | As you can see, if lookup table population goes beyond 1 mio entries, lookup time starts being noticeable over background noise. But with these numbers, I am not worried. Standard lookup population should be around *300-500*, with very long command lines resulting in table populations of *~1000*. We should never seen 1 entries, let alone millions of them. Still, I added a jtreg test to verify the expected hash table population. To catch errors like an unforeseen mass of pre-init allocations (lets say a leak or badly written code sneaked in), or if the hash algorithm suddenly is not good anymore. Two more points 1) I kept this coding deliberately simple. If we are really worried about a degenerated lookup table, we can do things to fix that: - we could automatically resize and rehash - we could, if we sense something wrong, just stop filling it and disable NMT, stopping NMT init phase prematurely at the cost of not being able to use NMT. The latter I had implemented already but removed it again to keep complexity down, and because I saw no need. 2) In our propietary production VM we have a system similar to NMT, but predating it. In that system we don't use malloc headers but store all (millions of) malloc'ed pointers in a big hash map. It performs excellent on *all our libc variants*. It is so fast that we just leave it always switched on. This solution has been productive since >10 years, and therefore I am confident that this is viable. This proposed hashmap with a planned population of 300-1000 is really not much :) - PR: https://git.openjdk.java.net/jdk/pull/4874
Re: RFR: JDK-8256844: Make NMT late-initializable
Hi Thomas, On 26/07/2021 10:15 pm, Thomas Stuefe wrote: Short: this patch makes NMT available in custom-launcher scenarios and during gtests. It simplifies NMT initialization. It adds a lot of NMT-specific testing. Before looking at this, have you checked the startup performance impact? Thanks, David - - NMT continues to be an extremely useful tool for SAP to tackle memory problems in the JVM. However, NMT is of limited use due to the following restrictions: - NMT cannot be used if the hotspot is embedded into a custom launcher unless the launcher actively cooperates. Just creating and invoking the JVM is not enough, it needs to do some steps prior to loading the hotspot. This limitation is not well known (nor, do I believe, documented). Many products don't do this, e.g., you cannot use NMT with IntelliJ. For us at SAP this problem limits NMT usefulness greatly since our VMs are often embedded into custom launchers and modifying every launcher is impossible. - Worse, if that custom launcher links the libjvm *statically* there is just no way to activate NMT at all. This is the reason NMT cannot be used in the `gtestlauncher`. - Related to that is that we cannot pass NMT options via `JAVA_TOOL_OPTIONS` and `-XX:Flags=`. - The fact that NMT cannot be used in gtests is really a pity since it would allow us to both test NMT itself more rigorously and check for memory leaks while testing other stuff. The reason for all this is that NMT initialization happens very early, on the first call to `os::malloc()`. And those calls happen already during dynamic C++ initialization - a long time before the VM gets around parsing arguments. So, regular VM argument parsing is too late to parse NMT arguments. The current solution is to pass NMT arguments via a specially prepared environment variable: `NMT_LEVEL_=`. That environment variable has to be set by the embedding launcher, before it loads the libjvm. Since its name contains the PID, we cannot even set that variable in the shell before starting the launcher. All that means that every launcher needs to especially parse and process the NMT arguments given at the command line (or via whatever method) and prepare the environment variable. `java` itself does this. This only works before the libjvm.so is loaded, before its dynamic C++ initialization. For that reason, it does not work if the launcher links statically against the hotspot, since in that case C++ initialization of the launcher and hotspot are folded into one phase with no possibility of executing code beforehand. And since it bypasses argument handling in the VM, it bypasses a number of argument processing ways, e.g., `JAVA_TOOL_OPTIONS`. -- This patch fixes these shortcomings by making NMT late-initializable: it can now be initialized after normal VM argument parsing, like all other parts of the VM. This greatly simplifies NMT initialization and makes it work automagically for every third party launcher, as well as within our gtests. The glaring problem with late-initializing NMT is the NMT malloc headers. If we rule out just always having them (unacceptable in terms of memory overhead), there is no safe way to determine, in os::free(), if an allocation came from before or after NMT initialization ran, and therefore what to do with its malloc headers. For a more extensive explanation, please see the comment block `nmtPreInit.hpp` and the discussion with @kimbarrett and @zhengyu123 in the JBS comment section. The heart of this patch is a new way to track early, pre-NMT-init allocations. These are tracked via a lookup table. This was a suggestion by Kim and it worked out well. Changes in detail: - pre-NMT-init handling: - the new files `nmtPreInit.hpp/cpp` take case of NMT pre-init handling. They contain a small global lookup table managing C-heap blocks allocated in the pre-NMT-init phase. - `os::malloc()/os::realloc()/os::free()` defer to this code before doing anything else. - Please see the extensive comment block at the start of `nmtPreinit.hpp` explaining the details. - Changes to NMT: - Before, NMT initialization was spread over two phases, `initialize()` and `late_initialize()`. Those were merged into one and simplified - there is only one initialization now which happens after argument parsing. - Minor changes were needed for the `NMT_TrackingLevel` enum - to simplify code, I changed NMT_unknown to be numerically 0. A new comment block in `nmtCommon.hpp` now clearly specifies what's what, including allowed level state transitions. - New utility functions to translate tracking level from/to strings added to `NMTUtil` - NMT has never been able to handle virtual memory allocations before initialization, which is fine since os::reserve_memory() is not called before VM parses arguments. We now assert that. - All code outside the VM handling NMT initialization
