Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers

[Adam Farley8]

> On 13/04/2018 15:14, Adam Farley8 wrote:
>> Hi Alan, Peter, 
>> 
>> I see that native memory is tracked in java.nio.Bits, but that only 
includes what the user thinks they are allocating. 
>> 
>> When the VM adds extra memory to the allocation amount this extra bit 
is not represented in the Bits total. 
>> A cursory glance shows, minimum, that we round the requested memory 
quantity up to the heap word size in 
>> the Unsafe.allocateMemory code, and something to do with 
nmt_header_size in os:malloc() (os.cpp) too.
> Is the align_up(sz, HeapWordSize) really causing an issue?

Coupled with the nmt_header_size business, it makes the Bits values wrong. 
The more DBB allocations, the more 
inaccurate those numbers will be.

> 
> We could change Bits to align with HotSpot. The BufferPoolMXBean API 
allows the capacity and memory usage to differ (when we originally added 
this, direct buffers were page aligned) so doing this would mean it more 
accurately reflects the memory allocated to direct buffers.
> 
> -Alan

I agree with you that the "+x" information should be added to only one of 
the two atomic longs.

To get "X", it seems to me that the best option would be to introduce an 
native method in Bits that fetches
"X" directly from Hotspot, using the same code that Hotspot uses (so we'd 
have to abstract-out the Hotspot logic
that adds X to the memory quantity).

This way, anyone modifying the Hotspot logic won't risk rendering the Bits 
logic wrong again.

Best Regards

Adam Farley

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers

[Adam Farley8]

> On 13/04/2018 15:14, Adam Farley8 wrote:
>> Hi Alan, Peter, 
>> 
>> I see that native memory is tracked in java.nio.Bits, but that only 
includes what the user thinks they are allocating. 
>> 
>> When the VM adds extra memory to the allocation amount this extra bit 
is not represented in the Bits total. 
>> A cursory glance shows, minimum, that we round the requested memory 
quantity up to the heap word size in 
>> the Unsafe.allocateMemory code, and something to do with 
nmt_header_size in os:malloc() (os.cpp) too.
> Is the align_up(sz, HeapWordSize) really causing an issue?

Coupled with the nmt_header_size business, it makes the Bits values wrong. 
The more DBB allocations, the more 
inaccurate those numbers will be.

> 
> We could change Bits to align with HotSpot. The BufferPoolMXBean API 
allows the capacity and memory usage to differ (when we originally added 
this, direct buffers were page aligned) so doing this would mean it more 
accurately reflects the memory allocated to direct buffers.
> 
> -Alan

I agree with you that the "+x" information should be added to only one of 
the two atomic longs.

To get "X", it seems to me that the best option would be to introduce an 
native method in Bits that fetches
"X" directly from Hotspot, using the same code that Hotspot uses (so we'd 
have to abstract-out the Hotspot logic
that adds X to the memory quantity).

This way, anyone modifying the Hotspot logic won't risk rendering the Bits 
logic wrong again.

Best Regards

Adam Farley

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers

[Alan Bateman]

On 13/04/2018 15:14, Adam Farley8 wrote:

Hi Alan, Peter,

I see that native memory is tracked in java.nio.Bits, but that only 
includes what the user thinks they are allocating.


When the VM adds extra memory to the allocation amount this extra bit 
is not represented in the Bits total.
A cursory glance shows, minimum, that we round the requested memory 
quantity up to the heap word size in
the Unsafe.allocateMemory code, and something to do with 
nmt_header_size in os:malloc() (os.cpp) too.

Is the align_up(sz, HeapWordSize) really causing an issue?

We could change Bits to align with HotSpot. The BufferPoolMXBean API 
allows the capacity and memory usage to differ (when we originally added 
this, direct buffers were page aligned) so doing this would mean it more 
accurately reflects the memory allocated to direct buffers.


-Alan

Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers

[Adam Farley8]

Hi Alan, Peter,

I see that native memory is tracked in java.nio.Bits, but that only 
includes what the user thinks they are allocating.

When the VM adds extra memory to the allocation amount this extra bit is 
not represented in the Bits total. 
A cursory glance shows, minimum, that we round the requested memory 
quantity up to the heap word size in 
the Unsafe.allocateMemory code, and something to do with nmt_header_size 
in os:malloc() (os.cpp) too.

Here's a set of solutions for the problem of "what you see isn't what you 
get" for DBBs (3 is my favourite, but I 
suspect 2 is the most likely to be accepted):

1) Use the code below to call DBB-specific Unsafe methods, allowing us to 
track the quantity of native
memory reserved for DBBs at the point of allocation.

Pros: 
- Doesn't require changing the Bits class.
- Allows us to easily devise a list of DBB native memory allocation 
locations.
- Fits with existing OpenJ9 code, which tracks DBB memory usage *after* 
native code has added to the total 
memory requested, resulting in more accurate totals than Bits.

Cons: 
- Requires some work on the Hotspot side to make it happen if Hotspot 
users want the true benefit.
OR
- Requires a commit that only benefits Java if you're running on the 
OpenJ9 VM.

2) Modify the Bits code to interrogate the VM via an extra method in 
Unsafe, in order to determine the
true quantity of native memory that is being allocated. 

E.g. User requests 10 bits, VM says it needs +2, add 12 to cumulative 
total, return "true". 
User later says to free 10 bits, VM says it needs +2, so we subtract 12 
from the total.

Note: For consistency, the VM and Bits should use the same code when 
figuring out how much space 
will be added to the request by the VM.

Pros: 
- Much smaller change than (1)
- Makes Bits much more accurate.
- Retains the Bits interface, and doesn't require a change to Unsafe.

Cons: 
- Requires us to add a native method to Bits, or somewhere visible to 
Bits.

3) Modify the Bits code to use an internal array for memory reservations, 
returns an index. The user
can use this index to:
- Return the amount of memory they requested.
- Return the amount of memory the VM will actually reserve (Bits will 
retrieve this from the VM via a 
native method).
- Set the address the VM gave them for this reservation. 
- Free the memory reserved.

Note: The existing internal totals, along with Shared Secrets, remain.

Pros: 
- *Much* more accurate memory tracking for DBBs.
- Easier debugging for DBB overruns, as we know where all the DBBs are.
- Prevents the user trying to free too much or too little memory for an 
allocated DBB.

Cons: 
- Many changes to Bits.
- Native code changes.
- Probably best to restructure the memory allocation code to share the 
same logic
(abstracting out the size+x gubbins so Bits' new native code logic remains 
up to date).


Any thoughts?

- Adam




>> Hi Adam,
>>
>> Did you know that native memory is already tracked on the Java side for
>> direct ByteBuffers? See class java.nio.Bits. Could you make use of it?
>>
> Right, these are the fields that are exposed at runtime via 
> BufferPoolMXBean. A SA based tool could read from a core file. I can't 
> tell if this is enough for Adam, it may be that the his tool reveals 
> more details on the buffers in the pools.
> 
> -Alan


P.S. Code:

diff --git 
a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template 
b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
--- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
+++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
@@ -85,7 +85,7 @@
 // Paranoia
 return;
 }
-UNSAFE.freeMemory(address);
+UNSAFE.freeDBBMemory(address);
 address = 0;
 Bits.unreserveMemory(size, capacity);
 }
@@ -118,7 +118,7 @@
 
 long base = 0;
 try {
-base = UNSAFE.allocateMemory(size);
+base = UNSAFE.allocateDBBMemory(size);
 } catch (OutOfMemoryError x) {
 Bits.unreserveMemory(size, cap);
 throw x;
diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java 
b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
--- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
+++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
@@ -632,6 +632,26 @@
 }
 
 /**
+ * Allocates a new block of native memory for DirectByteBuffers, of 
the
+ * given size in bytes.  The contents of the memory are 
uninitialized;
+ * they will generally be garbage.  The resulting native pointer will
+ * never be zero, and will be aligned for all value types.  Dispose 
of
+ * this memory by calling {@link #freeDBBMemory} or resize it with
+ * {@link #reallocateDBBMemory}.
+ *
+ * @throws RuntimeException if the size is negative or too large 
+ *

Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers

[Adam Farley8]

Hi Alan, Peter,

The main bit of data I wanted out of this was the sum total of native 
memory 
being used to store DirectByteBuffers, and to have that information 
printed
in diagnostic cores.

Thanks for pointing out Bits. I will investigate if there is a way to make 
that
data available to the VM when a diagnostic core is generated (I'm poking
through SharedSecrets and JavaNioAccess now) without running java code.

Worst case scenario, we don't get this feature, and at least we can
retrieve this information from the core by using, as Alan suggests, an SA-
based tool to retrieve the state of the Bits variables at crash-time.

Best Regards

Adam Farley



From:   Alan Bateman <alan.bate...@oracle.com>
To: Peter Levart <peter.lev...@gmail.com>, Adam Farley8 
<adam.far...@uk.ibm.com>
Cc: "hotspot-...@openjdk.java.net developers" 
<hotspot-...@openjdk.java.net>, core-libs-dev 
<core-libs-dev@openjdk.java.net>
Date:   23/02/2018 17:52
Subject:    Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track 
Native Memory Usage for Direct Byte Buffers





On 23/02/2018 15:28, Peter Levart wrote:
> Hi Adam,
>
> Did you know that native memory is already tracked on the Java side for
> direct ByteBuffers? See class java.nio.Bits. Could you make use of it?
>
Right, these are the fields that are exposed at runtime via 
BufferPoolMXBean. A SA based tool could read from a core file. I can't 
tell if this is enough for Adam, it may be that the his tool reveals 
more details on the buffers in the pools.

-Alan





Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers

[Alan Bateman]

On 23/02/2018 15:28, Peter Levart wrote:

Hi Adam,

Did you know that native memory is already tracked on the Java side for
direct ByteBuffers? See class java.nio.Bits. Could you make use of it?

Right, these are the fields that are exposed at runtime via 
BufferPoolMXBean. A SA based tool could read from a core file. I can't 
tell if this is enough for Adam, it may be that the his tool reveals 
more details on the buffers in the pools.


-Alan

Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers

[Peter Levart]

Hi Adam,

Did you know that native memory is already tracked on the Java side for
direct ByteBuffers? See class java.nio.Bits. Could you make use of it?

Regards, Peter

On 23 Feb 2018 1:09 pm, "Adam Farley8" <adam.far...@uk.ibm.com> wrote:

> Hi Paul,
>
> The larger picture for (read: effect of) these changes is best explained
> in my email here:
>
> http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-
> February/051441.html
>
> See the hyperlink I posted, and the few lines before it.
>
> Unfortunately the only understanding I have regarding the workings of the
> native code
> is would be derived from the OpenJ9 implimentation. I figured I wouldn't
> be thanked for
> posting that code here, so I posted what code I could share, with the
> additional note
> that the Hotspot native side of this should be implimented by:
>
> 1) Turning those Unsafe.java methods into native methods, and make them
> abstract
> (descriptor only, for the uninitiated).
>
> 2) Find the Hotspot native code for native memory allocation,
> reallocation, and
> freeing. Basically create a method that stores the sum total amount of
> native memory
> used by the DBBs, and then calls the regular allocate/reallocate/free
> methods.
>
> E.g.
>
> NM Allocate (Java) - NM Allocate (Native)
>
> DBB NM Allocate (Java) - DBB NM Allocate (Native) - NM allocate (Native)
>
> 3) Find the code that prints the current native memory usage in core
> files, and
> add a similar bit to show the native memory usage for DBBs as a subset
> (see the
> aforementioned linked link for an example).
>
> This seems like a straightforward task, though that's easy for me to say.
> :)
>
> Does that answer your question?
>
> Also, I'm unfamiliar with Java Flight Recorder. Are other developers on
> the list
> familiar with JFR that can snwer this? I'll put the message in IRC as
> well, and
> update here if I get any answers.
>
> Best Regards
>
> Adam Farley
>
>
>
> From:   Paul Sandoz <paul.san...@oracle.com>
> To: Adam Farley8 <adam.far...@uk.ibm.com>
> Cc: core-libs-dev <core-libs-dev@openjdk.java.net>, hotspot-dev
> developers <hotspot-...@openjdk.java.net>
> Date:   22/02/2018 02:20
> Subject:Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track
> Native Memory Usage for Direct Byte Buffers
>
>
>
> Hi Adam,
>
> While the burden is minimal there is a principle here that i think we
> should adhere to regarding additions to the code base: additions should
> have value within OpenJDK itself otherwise it can become a thin end of the
> wedge to more stuff (“well you added these things, why not just add these
> too?”).
>
> So i would still be reluctant to add such methods without understanding
> the larger picture and what you have in mind.
>
> Can you send a pointer to your email referring in more detail to the
> larger change sets?
>
> This use-case might also apply in other related areas too with regards to
> logging/monitoring. I would be interested to understand what Java Flight
> Recorder (JFR) does in this regard (it being open sourced soon i believe)
> and how JFR might relate to what you are doing. Should we be adding JFR
> events to unsafe memory allocation? Can JFR efficiently access part of the
> Java call stack to determine the origin?
>
> Thanks,
> Paul.
>
> On Feb 19, 2018, at 5:08 AM, Adam Farley8 <adam.far...@uk.ibm.com> wrote:
>
> Hi Paul,
>
> > Hi Adam,
> >
> > From reading the thread i cannot tell if this is part of a wider
> solution including some yet to be proposed HotSpot changes.
>
> The wider solution would need to include some Hotspot changes, yes.
> I'm proposing raising a bug, committing the code we have here to
> "set the stage", and then we can invest more time later
> if the concept goes down well and the community agrees to pursue
> the full solution.
>
> As an aside, I tried submitting a big code set (including hotspot
> changes) months ago, and I'm *still* struggling to find someone to
> commit the thing, so I figured I'd try a more gradual, staged approach
> this time.
>
> >
> > As is i would be resistant to adding such standalone internal wrapper
> methods to Unsafe that have no apparent benefit within the OpenJDK itself
> since it's a maintenance burden.
>
> I'm hoping the fact that the methods are a single line (sans
> comments, descriptors and curly braces) will minimise this burden.
>
> >
> > Can you determine if the calls to UNSAFE.freeMemory/allocateMemory come
> from a DBB by looking at the call stack frame above the unsafe call?
> >
> > Thanks,
> >

Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers

[Adam Farley8]

Hi Paul,

The larger picture for (read: effect of) these changes is best explained 
in my email here:

http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-February/051441.html

See the hyperlink I posted, and the few lines before it.

Unfortunately the only understanding I have regarding the workings of the 
native code
is would be derived from the OpenJ9 implimentation. I figured I wouldn't 
be thanked for 
posting that code here, so I posted what code I could share, with the 
additional note 
that the Hotspot native side of this should be implimented by:

1) Turning those Unsafe.java methods into native methods, and make them 
abstract
(descriptor only, for the uninitiated).

2) Find the Hotspot native code for native memory allocation, 
reallocation, and 
freeing. Basically create a method that stores the sum total amount of 
native memory
used by the DBBs, and then calls the regular allocate/reallocate/free 
methods.

E.g.

NM Allocate (Java) - NM Allocate (Native)

DBB NM Allocate (Java) - DBB NM Allocate (Native) - NM allocate (Native)

3) Find the code that prints the current native memory usage in core 
files, and 
add a similar bit to show the native memory usage for DBBs as a subset 
(see the
aforementioned linked link for an example).

This seems like a straightforward task, though that's easy for me to say. 
:)

Does that answer your question?

Also, I'm unfamiliar with Java Flight Recorder. Are other developers on 
the list
familiar with JFR that can snwer this? I'll put the message in IRC as 
well, and
update here if I get any answers.

Best Regards

Adam Farley



From:   Paul Sandoz <paul.san...@oracle.com>
To: Adam Farley8 <adam.far...@uk.ibm.com>
Cc: core-libs-dev <core-libs-dev@openjdk.java.net>, hotspot-dev 
developers <hotspot-...@openjdk.java.net>
Date:   22/02/2018 02:20
Subject:    Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track 
Native Memory Usage for Direct Byte Buffers



Hi Adam,

While the burden is minimal there is a principle here that i think we 
should adhere to regarding additions to the code base: additions should 
have value within OpenJDK itself otherwise it can become a thin end of the 
wedge to more stuff (“well you added these things, why not just add these 
too?”).

So i would still be reluctant to add such methods without understanding 
the larger picture and what you have in mind.

Can you send a pointer to your email referring in more detail to the 
larger change sets?

This use-case might also apply in other related areas too with regards to 
logging/monitoring. I would be interested to understand what Java Flight 
Recorder (JFR) does in this regard (it being open sourced soon i believe) 
and how JFR might relate to what you are doing. Should we be adding JFR 
events to unsafe memory allocation? Can JFR efficiently access part of the 
Java call stack to determine the origin?

Thanks,
Paul.

On Feb 19, 2018, at 5:08 AM, Adam Farley8 <adam.far...@uk.ibm.com> wrote:

Hi Paul, 

> Hi Adam, 
> 
> From reading the thread i cannot tell if this is part of a wider 
solution including some yet to be proposed HotSpot changes. 

The wider solution would need to include some Hotspot changes, yes. 
I'm proposing raising a bug, committing the code we have here to 
"set the stage", and then we can invest more time later 
if the concept goes down well and the community agrees to pursue 
the full solution. 

As an aside, I tried submitting a big code set (including hotspot 
changes) months ago, and I'm *still* struggling to find someone to 
commit the thing, so I figured I'd try a more gradual, staged approach 
this time. 

> 
> As is i would be resistant to adding such standalone internal wrapper 
methods to Unsafe that have no apparent benefit within the OpenJDK itself 
since it's a maintenance burden. 

I'm hoping the fact that the methods are a single line (sans 
comments, descriptors and curly braces) will minimise this burden. 

> 
> Can you determine if the calls to UNSAFE.freeMemory/allocateMemory come 
from a DBB by looking at the call stack frame above the unsafe call? 
> 
> Thanks, 
> Paul. 

Yes that is possible, though I would advise against this because: 

 A) Checking the call stack is expensive, and doing this every time we 
allocate native memory is an easy way to slow down a program, 
or rack up mips. 
and 
 B) deciding which code path we're using based on the stack 
means the DBB class+method (and anything the parsing code 
mistakes for that class+method) can only ever allocate native 
memory for DBBs. 

What do you think? 

Best Regards 

Adam Farley 

> 
>> On Feb 14, 2018, at 3:32 AM, Adam Farley8 <adam.far...@uk.ibm.com> 
wrote: 
>> 
>> Hi All, 
>> 
>> Currently, diagnostic core files generated from OpenJDK seem to lump 
all 
>> of the 
>> native memory usages together, making it near-impossibl

Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers

[Paul Sandoz]

Hi Adam,

While the burden is minimal there is a principle here that i think we should 
adhere to regarding additions to the code base: additions should have value 
within OpenJDK itself otherwise it can become a thin end of the wedge to more 
stuff (“well you added these things, why not just add these too?”).

So i would still be reluctant to add such methods without understanding the 
larger picture and what you have in mind.

Can you send a pointer to your email referring in more detail to the larger 
change sets?

This use-case might also apply in other related areas too with regards to 
logging/monitoring. I would be interested to understand what Java Flight 
Recorder (JFR) does in this regard (it being open sourced soon i believe) and 
how JFR might relate to what you are doing. Should we be adding JFR events to 
unsafe memory allocation? Can JFR efficiently access part of the Java call 
stack to determine the origin?

Thanks,
Paul.

> On Feb 19, 2018, at 5:08 AM, Adam Farley8  wrote:
> 
> Hi Paul, 
> 
> > Hi Adam, 
> > 
> > From reading the thread i cannot tell if this is part of a wider solution 
> > including some yet to be proposed HotSpot changes. 
> 
> The wider solution would need to include some Hotspot changes, yes. 
> I'm proposing raising a bug, committing the code we have here to 
> "set the stage", and then we can invest more time later 
> if the concept goes down well and the community agrees to pursue 
> the full solution. 
> 
> As an aside, I tried submitting a big code set (including hotspot 
> changes) months ago, and I'm *still* struggling to find someone to 
> commit the thing, so I figured I'd try a more gradual, staged approach 
> this time. 
> 
> > 
> > As is i would be resistant to adding such standalone internal wrapper 
> > methods to Unsafe that have no apparent benefit within the OpenJDK itself 
> > since it's a maintenance burden. 
> 
> I'm hoping the fact that the methods are a single line (sans 
> comments, descriptors and curly braces) will minimise this burden. 
> 
> > 
> > Can you determine if the calls to UNSAFE.freeMemory/allocateMemory come 
> > from a DBB by looking at the call stack frame above the unsafe call? 
> > 
> > Thanks, 
> > Paul. 
> 
> Yes that is possible, though I would advise against this because: 
> 
>  A) Checking the call stack is expensive, and doing this every time we 
> allocate native memory is an easy way to slow down a program, 
> or rack up mips. 
> and 
>  B) deciding which code path we're using based on the stack 
> means the DBB class+method (and anything the parsing code 
> mistakes for that class+method) can only ever allocate native 
> memory for DBBs. 
> 
> What do you think? 
> 
> Best Regards 
> 
> Adam Farley 
> 
> > 
> >> On Feb 14, 2018, at 3:32 AM, Adam Farley8  wrote: 
> >> 
> >> Hi All, 
> >> 
> >> Currently, diagnostic core files generated from OpenJDK seem to lump all 
> >> of the 
> >> native memory usages together, making it near-impossible for someone to 
> >> figure 
> >> out *what* is using all that memory in the event of a memory leak. 
> >> 
> >> The OpenJ9 VM has a feature which allows it to track the allocation of 
> >> native 
> >> memory for Direct Byte Buffers (DBBs), and to supply that information into 
> >> the 
> >> cores when they are generated. This makes it a *lot* easier to find out 
> >> what is using 
> >> all that native memory, making memory leak resolution less like some dark 
> >> art, and 
> >> more like logical debugging. 
> >> 
> >> To use this feature, there is a native method referenced in Unsafe.java. 
> >> To open 
> >> up this feature so that any VM can make use of it, the java code below 
> >> sets the 
> >> stage for it. This change starts letting people call DBB-specific methods 
> >> when 
> >> allocating native memory, and getting into the habit of using it. 
> >> 
> >> Thoughts? 
> >> 
> >> Best Regards 
> >> 
> >> Adam Farley 
> >> 
> >> P.S. Code: 
> >> 
> >> diff --git 
> >> a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template 
> >> b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template 
> >> --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template 
> >> +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template 
> >> @@ -85,7 +85,7 @@ 
> >> // Paranoia 
> >> return; 
> >> } 
> >> -UNSAFE.freeMemory(address); 
> >> +UNSAFE.freeDBBMemory(address); 
> >> address = 0; 
> >> Bits.unreserveMemory(size, capacity); 
> >> } 
> >> @@ -118,7 +118,7 @@ 
> >> 
> >> long base = 0; 
> >> try { 
> >> -base = UNSAFE.allocateMemory(size); 
> >> +base = UNSAFE.allocateDBBMemory(size); 
> >> } catch (OutOfMemoryError x) { 
> >> Bits.unreserveMemory(size, cap); 
> >> throw x; 
> >> diff --git 

Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers

[Adam Farley8]

Hi Paul,

> Hi Adam,
>
> From reading the thread i cannot tell if this is part of a wider 
solution including some yet to be proposed HotSpot changes.

The wider solution would need to include some Hotspot changes, yes.
I'm proposing raising a bug, committing the code we have here to
"set the stage", and then we can invest more time later
if the concept goes down well and the community agrees to pursue
the full solution.

As an aside, I tried submitting a big code set (including hotspot
changes) months ago, and I'm *still* struggling to find someone to
commit the thing, so I figured I'd try a more gradual, staged approach
this time.

>
> As is i would be resistant to adding such standalone internal wrapper 
methods to Unsafe that have no apparent benefit within the OpenJDK itself 
since it's a maintenance burden.

I'm hoping the fact that the methods are a single line (sans 
comments, descriptors and curly braces) will minimise this burden.

>
> Can you determine if the calls to UNSAFE.freeMemory/allocateMemory come 
from a DBB by looking at the call stack frame above the unsafe call?
>
> Thanks,
> Paul.

Yes that is possible, though I would advise against this because:

 A) Checking the call stack is expensive, and doing this every time we 
allocate native memory is an easy way to slow down a program,
or rack up mips.
and 
 B) deciding which code path we're using based on the stack 
means the DBB class+method (and anything the parsing code 
mistakes for that class+method) can only ever allocate native 
memory for DBBs.

What do you think?

Best Regards

Adam Farley

>
>> On Feb 14, 2018, at 3:32 AM, Adam Farley8  
wrote:
>> 
>> Hi All,
>> 
>> Currently, diagnostic core files generated from OpenJDK seem to lump 
all 
>> of the 
>> native memory usages together, making it near-impossible for someone to 

>> figure 
>> out *what* is using all that memory in the event of a memory leak.
>> 
>> The OpenJ9 VM has a feature which allows it to track the allocation of 
>> native
>> memory for Direct Byte Buffers (DBBs), and to supply that information 
into 
>> the
>> cores when they are generated. This makes it a *lot* easier to find out 

>> what is using
>> all that native memory, making memory leak resolution less like some 
dark 
>> art, and
>> more like logical debugging.
>> 
>> To use this feature, there is a native method referenced in 
Unsafe.java. 
>> To open
>> up this feature so that any VM can make use of it, the java code below 
>> sets the 
>> stage for it. This change starts letting people call DBB-specific 
methods 
>> when 
>> allocating native memory, and getting into the habit of using it.
>> 
>> Thoughts?
>> 
>> Best Regards
>> 
>> Adam Farley
>> 
>> P.S. Code:
>> 
>> diff --git 
>> a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template 
>> b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
>> --- 
a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
>> +++ 
b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
>> @@ -85,7 +85,7 @@
>> // Paranoia
>> return;
>> }
>> -UNSAFE.freeMemory(address);
>> +UNSAFE.freeDBBMemory(address);
>> address = 0;
>> Bits.unreserveMemory(size, capacity);
>> }
>> @@ -118,7 +118,7 @@
>> 
>> long base = 0;
>> try {
>> -base = UNSAFE.allocateMemory(size);
>> +base = UNSAFE.allocateDBBMemory(size);
>> } catch (OutOfMemoryError x) {
>> Bits.unreserveMemory(size, cap);
>> throw x;
>> diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java 
>> b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
>> --- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
>> +++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
>> @@ -632,6 +632,26 @@
>> }
>> 
>> /**
>> + * Allocates a new block of native memory for DirectByteBuffers, 
of 
>> the
>> + * given size in bytes.  The contents of the memory are 
>> uninitialized;
>> + * they will generally be garbage.  The resulting native pointer 
will
>> + * never be zero, and will be aligned for all value types. Dispose 

>> of
>> + * this memory by calling {@link #freeDBBMemory} or resize it with
>> + * {@link #reallocateDBBMemory}.
>> + *
>> + * @throws RuntimeException if the size is negative or too large 
>> + *  for the native size_t type
>> + *
>> + * @throws OutOfMemoryError if the allocation is refused by the 
>> system
>> + *
>> + * @see #getByte(long)
>> + * @see #putByte(long, byte)
>> + */
>> +public long allocateDBBMemory(long bytes) {
>> +return allocateMemory(bytes);
>> +}
>> + 
>> +/**
>>  * Resizes a new block of native memory, to the given size in 
bytes. 
>> The
>>  * contents of the new block 

Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers

[Paul Sandoz]

Hi Adam,

From reading the thread i cannot tell if this is part of a wider solution 
including some yet to be proposed HotSpot changes.

As is i would be resistant to adding such standalone internal wrapper methods 
to Unsafe that have no apparent benefit within the OpenJDK itself since it's a 
maintenance burden.

Can you determine if the calls to UNSAFE.freeMemory/allocateMemory come from a 
DBB by looking at the call stack frame above the unsafe call?

Thanks,
Paul.

> On Feb 14, 2018, at 3:32 AM, Adam Farley8  wrote:
> 
> Hi All,
> 
> Currently, diagnostic core files generated from OpenJDK seem to lump all 
> of the 
> native memory usages together, making it near-impossible for someone to 
> figure 
> out *what* is using all that memory in the event of a memory leak.
> 
> The OpenJ9 VM has a feature which allows it to track the allocation of 
> native
> memory for Direct Byte Buffers (DBBs), and to supply that information into 
> the
> cores when they are generated. This makes it a *lot* easier to find out 
> what is using
> all that native memory, making memory leak resolution less like some dark 
> art, and
> more like logical debugging.
> 
> To use this feature, there is a native method referenced in Unsafe.java. 
> To open
> up this feature so that any VM can make use of it, the java code below 
> sets the 
> stage for it. This change starts letting people call DBB-specific methods 
> when 
> allocating native memory, and getting into the habit of using it.
> 
> Thoughts?
> 
> Best Regards
> 
> Adam Farley
> 
> P.S. Code:
> 
> diff --git 
> a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template 
> b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
> --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
> +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
> @@ -85,7 +85,7 @@
> // Paranoia
> return;
> }
> -UNSAFE.freeMemory(address);
> +UNSAFE.freeDBBMemory(address);
> address = 0;
> Bits.unreserveMemory(size, capacity);
> }
> @@ -118,7 +118,7 @@
> 
> long base = 0;
> try {
> -base = UNSAFE.allocateMemory(size);
> +base = UNSAFE.allocateDBBMemory(size);
> } catch (OutOfMemoryError x) {
> Bits.unreserveMemory(size, cap);
> throw x;
> diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java 
> b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
> --- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
> +++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
> @@ -632,6 +632,26 @@
> }
> 
> /**
> + * Allocates a new block of native memory for DirectByteBuffers, of 
> the
> + * given size in bytes.  The contents of the memory are 
> uninitialized;
> + * they will generally be garbage.  The resulting native pointer will
> + * never be zero, and will be aligned for all value types.  Dispose 
> of
> + * this memory by calling {@link #freeDBBMemory} or resize it with
> + * {@link #reallocateDBBMemory}.
> + *
> + * @throws RuntimeException if the size is negative or too large 
> + *  for the native size_t type
> + *
> + * @throws OutOfMemoryError if the allocation is refused by the 
> system
> + *
> + * @see #getByte(long)
> + * @see #putByte(long, byte)
> + */
> +public long allocateDBBMemory(long bytes) {
> +return allocateMemory(bytes);
> +}
> + 
> +/**
>  * Resizes a new block of native memory, to the given size in bytes. 
> The
>  * contents of the new block past the size of the old block are
>  * uninitialized; they will generally be garbage.  The resulting 
> native
> @@ -687,6 +707,27 @@
> }
> 
> /**
> + * Resizes a new block of native memory for DirectByteBuffers, to the
> + * given size in bytes.  The contents of the new block past the size 
> of
> + * the old block are uninitialized; they will generally be garbage. 
> The
> + * resulting native pointer will be zero if and only if the requested 
> size
> + * is zero.  The resulting native pointer will be aligned for all 
> value
> + * types.  Dispose of this memory by calling {@link #freeDBBMemory}, 
> or
> + * resize it with {@link #reallocateDBBMemory}.  The address passed 
> to
> + * this method may be null, in which case an allocation will be 
> performed.
> + *
> + * @throws RuntimeException if the size is negative or too large 
> + *  for the native size_t type
> + *
> + * @throws OutOfMemoryError if the allocation is refused by the 
> system
> + *
> + * @see #allocateDBBMemory
> + */
> +public long reallocateDBBMemory(long address, long bytes) {
> +return reallocateMemory(address, bytes);
> +}
> +
> +/**

Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers

[Zhengyu Gu]

Think of it as an NMT upgrade.

Here's an example of what the output should look like:

https://developer.ibm.com/answers/questions/288697/why-
does-nativememinfo-in-javacore-show-incorrect.html?sort=oldest

- Adam



I think NMT walks the stack, so we should get allocation points grouped by
call stacks. Provided we have symbols loaded for the native library using
Unsafe.allocateMemory(), this should give us too a fine granularity. But I
have not yet tested this in practice. Maybe Zhengyu knows more.


Quick test shows this call site:

[0x7f8558b26243] Unsafe_AllocateMemory0+0x93
[0x7f8537b085cb]
 (malloc=2KB type=Internal #1)

I will take a look why there is a frame not decoded.

Thanks,

-Zhengyu





..Thomas





David

On 14/02/2018 9:32 PM, Adam Farley8 wrote:

Hi All,

Currently, diagnostic core files generated from OpenJDK seem to lump

all

of the
native memory usages together, making it near-impossible for someone

to

figure
out *what* is using all that memory in the event of a memory leak.

The OpenJ9 VM has a feature which allows it to track the allocation of
native
memory for Direct Byte Buffers (DBBs), and to supply that information
into
the
cores when they are generated. This makes it a *lot* easier to find

out

what is using
all that native memory, making memory leak resolution less like some

dark

art, and
more like logical debugging.

To use this feature, there is a native method referenced in

Unsafe.java.

To open
up this feature so that any VM can make use of it, the java code below
sets the
stage for it. This change starts letting people call DBB-specific

methods

when
allocating native memory, and getting into the habit of using it.

Thoughts?

Best Regards

Adam Farley

P.S. Code:

diff --git
a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
---

a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template

+++

b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template

@@ -85,7 +85,7 @@
   // Paranoia
   return;
   }
-UNSAFE.freeMemory(address);
+UNSAFE.freeDBBMemory(address);
   address = 0;
   Bits.unreserveMemory(size, capacity);
   }
@@ -118,7 +118,7 @@
   long base = 0;
   try {
-base = UNSAFE.allocateMemory(size);
+base = UNSAFE.allocateDBBMemory(size);
   } catch (OutOfMemoryError x) {
   Bits.unreserveMemory(size, cap);
   throw x;
diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
--- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
+++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
@@ -632,6 +632,26 @@
   }
   /**
+ * Allocates a new block of native memory for DirectByteBuffers,

of

the
+ * given size in bytes.  The contents of the memory are
uninitialized;
+ * they will generally be garbage.  The resulting native pointer
will
+ * never be zero, and will be aligned for all value types.

Dispose

of
+ * this memory by calling {@link #freeDBBMemory} or resize it

with

+ * {@link #reallocateDBBMemory}.
+ *
+ * @throws RuntimeException if the size is negative or too large
+ *  for the native size_t type
+ *
+ * @throws OutOfMemoryError if the allocation is refused by the
system
+ *
+ * @see #getByte(long)
+ * @see #putByte(long, byte)
+ */
+public long allocateDBBMemory(long bytes) {
+return allocateMemory(bytes);
+}
+
+/**
* Resizes a new block of native memory, to the given size in
bytes.
The
* contents of the new block past the size of the old block are
* uninitialized; they will generally be garbage.  The resulting
native
@@ -687,6 +707,27 @@
   }
   /**
+ * Resizes a new block of native memory for DirectByteBuffers, to



the
+ * given size in bytes.  The contents of the new block past the

size

of
+ * the old block are uninitialized; they will generally be

garbage.

The
+ * resulting native pointer will be zero if and only if the
requested
size
+ * is zero.  The resulting native pointer will be aligned for all
value
+ * types.  Dispose of this memory by calling {@link

#freeDBBMemory},

or
+ * resize it with {@link #reallocateDBBMemory}.  The address

passed

to
+ * this method may be null, in which case an allocation will be
performed.
+ *
+ * @throws RuntimeException if the size is negative or too large
+ *  for the native size_t type
+ *
+ * @throws OutOfMemoryError if the allocation is refused by the
system
+ *
+ * @see #allocateDBBMemory
+ */
+public long reallocateDBBMemory(long address, long bytes) {
+

Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers

[Zhengyu Gu]

On 02/14/2018 08:16 AM, Thomas Stüfe wrote:

On Wed, Feb 14, 2018 at 1:53 PM, David Holmes 
wrote:


On 14/02/2018 10:43 PM, David Holmes wrote:


Adding in core-libs-dev as there's nothing related to hotspot directly
here.



Correction, this is of course leading to a proposed change in hotspot to
implement the new Unsafe methods and perform the native memory tracking. Of
course we already have NMT so the obvious question is how this will fit in
with NMT?



I thought Unsafe.allocateMemory is served by hotspot os::malloc(), is it
not? So, allocations should show up in NMT with "Unsafe_AllocateMemory0".


Could use another category? to make it earlier to identify.

Thanks,

-Zhengyu



..Thomas




David


David


On 14/02/2018 9:32 PM, Adam Farley8 wrote:


Hi All,

Currently, diagnostic core files generated from OpenJDK seem to lump all
of the
native memory usages together, making it near-impossible for someone to
figure
out *what* is using all that memory in the event of a memory leak.

The OpenJ9 VM has a feature which allows it to track the allocation of
native
memory for Direct Byte Buffers (DBBs), and to supply that information
into
the
cores when they are generated. This makes it a *lot* easier to find out
what is using
all that native memory, making memory leak resolution less like some dark
art, and
more like logical debugging.

To use this feature, there is a native method referenced in Unsafe.java.
To open
up this feature so that any VM can make use of it, the java code below
sets the
stage for it. This change starts letting people call DBB-specific methods
when
allocating native memory, and getting into the habit of using it.

Thoughts?

Best Regards

Adam Farley

P.S. Code:

diff --git
a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
--- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
+++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
@@ -85,7 +85,7 @@
   // Paranoia
   return;
   }
-UNSAFE.freeMemory(address);
+UNSAFE.freeDBBMemory(address);
   address = 0;
   Bits.unreserveMemory(size, capacity);
   }
@@ -118,7 +118,7 @@
   long base = 0;
   try {
-base = UNSAFE.allocateMemory(size);
+base = UNSAFE.allocateDBBMemory(size);
   } catch (OutOfMemoryError x) {
   Bits.unreserveMemory(size, cap);
   throw x;
diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
--- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
+++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
@@ -632,6 +632,26 @@
   }
   /**
+ * Allocates a new block of native memory for DirectByteBuffers, of
the
+ * given size in bytes.  The contents of the memory are
uninitialized;
+ * they will generally be garbage.  The resulting native pointer
will
+ * never be zero, and will be aligned for all value types.  Dispose
of
+ * this memory by calling {@link #freeDBBMemory} or resize it with
+ * {@link #reallocateDBBMemory}.
+ *
+ * @throws RuntimeException if the size is negative or too large
+ *  for the native size_t type
+ *
+ * @throws OutOfMemoryError if the allocation is refused by the
system
+ *
+ * @see #getByte(long)
+ * @see #putByte(long, byte)
+ */
+public long allocateDBBMemory(long bytes) {
+return allocateMemory(bytes);
+}
+
+/**
* Resizes a new block of native memory, to the given size in
bytes.
The
* contents of the new block past the size of the old block are
* uninitialized; they will generally be garbage.  The resulting
native
@@ -687,6 +707,27 @@
   }
   /**
+ * Resizes a new block of native memory for DirectByteBuffers, to
the
+ * given size in bytes.  The contents of the new block past the size
of
+ * the old block are uninitialized; they will generally be garbage.
The
+ * resulting native pointer will be zero if and only if the
requested
size
+ * is zero.  The resulting native pointer will be aligned for all
value
+ * types.  Dispose of this memory by calling {@link #freeDBBMemory},
or
+ * resize it with {@link #reallocateDBBMemory}.  The address passed
to
+ * this method may be null, in which case an allocation will be
performed.
+ *
+ * @throws RuntimeException if the size is negative or too large
+ *  for the native size_t type
+ *
+ * @throws OutOfMemoryError if the allocation is refused by the
system
+ *
+ * @see #allocateDBBMemory
+ */
+public long reallocateDBBMemory(long address, long bytes) {
+return reallocateMemory(address, bytes);
+}
+
+ 

Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers

[Thomas Stüfe]

On Wed, Feb 14, 2018 at 2:32 PM, Adam Farley8 
wrote:

> >> Adding in core-libs-dev as there's nothing related to hotspot directly
> >> here.
> >
> > Correction, this is of course leading to a proposed change in hotspot to
>
> > implement the new Unsafe methods and perform the native memory tracking.
>
>
> Hah, I wrote the same thing in a parallel reply. Jinx. :)
>
> - Adam
>
> > Of course we already have NMT so the obvious question is how this will
> > fit in with NMT?
> >
> > David
>
> It will add granularity to Native Memory Tracking, allowing people
> to tell, at a glance, how much of the allocated native memory has been
> used for Direct Byte Buffers. This makes native memory OOM
> debugging easier.
>
> Think of it as an NMT upgrade.
>
> Here's an example of what the output should look like:
>
> https://developer.ibm.com/answers/questions/288697/why-
> does-nativememinfo-in-javacore-show-incorrect.html?sort=oldest
>
> - Adam
>
>
I think NMT walks the stack, so we should get allocation points grouped by
call stacks. Provided we have symbols loaded for the native library using
Unsafe.allocateMemory(), this should give us too a fine granularity. But I
have not yet tested this in practice. Maybe Zhengyu knows more.

..Thomas


> >
> >> David
> >>
> >> On 14/02/2018 9:32 PM, Adam Farley8 wrote:
> >>> Hi All,
> >>>
> >>> Currently, diagnostic core files generated from OpenJDK seem to lump
> all
> >>> of the
> >>> native memory usages together, making it near-impossible for someone
> to
> >>> figure
> >>> out *what* is using all that memory in the event of a memory leak.
> >>>
> >>> The OpenJ9 VM has a feature which allows it to track the allocation of
> >>> native
> >>> memory for Direct Byte Buffers (DBBs), and to supply that information
> >>> into
> >>> the
> >>> cores when they are generated. This makes it a *lot* easier to find
> out
> >>> what is using
> >>> all that native memory, making memory leak resolution less like some
> dark
> >>> art, and
> >>> more like logical debugging.
> >>>
> >>> To use this feature, there is a native method referenced in
> Unsafe.java.
> >>> To open
> >>> up this feature so that any VM can make use of it, the java code below
> >>> sets the
> >>> stage for it. This change starts letting people call DBB-specific
> methods
> >>> when
> >>> allocating native memory, and getting into the habit of using it.
> >>>
> >>> Thoughts?
> >>>
> >>> Best Regards
> >>>
> >>> Adam Farley
> >>>
> >>> P.S. Code:
> >>>
> >>> diff --git
> >>> a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
> >>> b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
> >>> ---
> a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
> >>> +++
> b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
> >>> @@ -85,7 +85,7 @@
> >>>   // Paranoia
> >>>   return;
> >>>   }
> >>> -UNSAFE.freeMemory(address);
> >>> +UNSAFE.freeDBBMemory(address);
> >>>   address = 0;
> >>>   Bits.unreserveMemory(size, capacity);
> >>>   }
> >>> @@ -118,7 +118,7 @@
> >>>   long base = 0;
> >>>   try {
> >>> -base = UNSAFE.allocateMemory(size);
> >>> +base = UNSAFE.allocateDBBMemory(size);
> >>>   } catch (OutOfMemoryError x) {
> >>>   Bits.unreserveMemory(size, cap);
> >>>   throw x;
> >>> diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
> >>> b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
> >>> --- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
> >>> +++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
> >>> @@ -632,6 +632,26 @@
> >>>   }
> >>>   /**
> >>> + * Allocates a new block of native memory for DirectByteBuffers,
> of
> >>> the
> >>> + * given size in bytes.  The contents of the memory are
> >>> uninitialized;
> >>> + * they will generally be garbage.  The resulting native pointer
> >>> will
> >>> + * never be zero, and will be aligned for all value types.
> Dispose
> >>> of
> >>> + * this memory by calling {@link #freeDBBMemory} or resize it
> with
> >>> + * {@link #reallocateDBBMemory}.
> >>> + *
> >>> + * @throws RuntimeException if the size is negative or too large
> >>> + *  for the native size_t type
> >>> + *
> >>> + * @throws OutOfMemoryError if the allocation is refused by the
> >>> system
> >>> + *
> >>> + * @see #getByte(long)
> >>> + * @see #putByte(long, byte)
> >>> + */
> >>> +public long allocateDBBMemory(long bytes) {
> >>> +return allocateMemory(bytes);
> >>> +}
> >>> +
> >>> +/**
> >>>* Resizes a new block of native memory, to the given size in
> >>> bytes.
> >>> The
> >>>* contents of the new block past the size of the old block are
> >>>* uninitialized; they 

Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers

[Adam Farley8]

>> Adding in core-libs-dev as there's nothing related to hotspot directly 
>> here.
>
> Correction, this is of course leading to a proposed change in hotspot to 

> implement the new Unsafe methods and perform the native memory tracking. 


Hah, I wrote the same thing in a parallel reply. Jinx. :)

- Adam

> Of course we already have NMT so the obvious question is how this will 
> fit in with NMT?
>
> David

It will add granularity to Native Memory Tracking, allowing people
to tell, at a glance, how much of the allocated native memory has been
used for Direct Byte Buffers. This makes native memory OOM
debugging easier.

Think of it as an NMT upgrade.

Here's an example of what the output should look like:

https://developer.ibm.com/answers/questions/288697/why-does-nativememinfo-in-javacore-show-incorrect.html?sort=oldest

- Adam

>
>> David
>> 
>> On 14/02/2018 9:32 PM, Adam Farley8 wrote:
>>> Hi All,
>>>
>>> Currently, diagnostic core files generated from OpenJDK seem to lump 
all
>>> of the
>>> native memory usages together, making it near-impossible for someone 
to
>>> figure
>>> out *what* is using all that memory in the event of a memory leak.
>>>
>>> The OpenJ9 VM has a feature which allows it to track the allocation of
>>> native
>>> memory for Direct Byte Buffers (DBBs), and to supply that information 
>>> into
>>> the
>>> cores when they are generated. This makes it a *lot* easier to find 
out
>>> what is using
>>> all that native memory, making memory leak resolution less like some 
dark
>>> art, and
>>> more like logical debugging.
>>>
>>> To use this feature, there is a native method referenced in 
Unsafe.java.
>>> To open
>>> up this feature so that any VM can make use of it, the java code below
>>> sets the
>>> stage for it. This change starts letting people call DBB-specific 
methods
>>> when
>>> allocating native memory, and getting into the habit of using it.
>>>
>>> Thoughts?
>>>
>>> Best Regards
>>>
>>> Adam Farley
>>>
>>> P.S. Code:
>>>
>>> diff --git
>>> a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
>>> b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
>>> --- 
a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
>>> +++ 
b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
>>> @@ -85,7 +85,7 @@
>>>   // Paranoia
>>>   return;
>>>   }
>>> -UNSAFE.freeMemory(address);
>>> +UNSAFE.freeDBBMemory(address);
>>>   address = 0;
>>>   Bits.unreserveMemory(size, capacity);
>>>   }
>>> @@ -118,7 +118,7 @@
>>>   long base = 0;
>>>   try {
>>> -base = UNSAFE.allocateMemory(size);
>>> +base = UNSAFE.allocateDBBMemory(size);
>>>   } catch (OutOfMemoryError x) {
>>>   Bits.unreserveMemory(size, cap);
>>>   throw x;
>>> diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
>>> b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
>>> --- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
>>> +++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
>>> @@ -632,6 +632,26 @@
>>>   }
>>>   /**
>>> + * Allocates a new block of native memory for DirectByteBuffers, 
of
>>> the
>>> + * given size in bytes.  The contents of the memory are
>>> uninitialized;
>>> + * they will generally be garbage.  The resulting native pointer 
>>> will
>>> + * never be zero, and will be aligned for all value types.  
Dispose
>>> of
>>> + * this memory by calling {@link #freeDBBMemory} or resize it 
with
>>> + * {@link #reallocateDBBMemory}.
>>> + *
>>> + * @throws RuntimeException if the size is negative or too large
>>> + *  for the native size_t type
>>> + *
>>> + * @throws OutOfMemoryError if the allocation is refused by the
>>> system
>>> + *
>>> + * @see #getByte(long)
>>> + * @see #putByte(long, byte)
>>> + */
>>> +public long allocateDBBMemory(long bytes) {
>>> +return allocateMemory(bytes);
>>> +}
>>> +
>>> +/**
>>>* Resizes a new block of native memory, to the given size in 
>>> bytes.
>>> The
>>>* contents of the new block past the size of the old block are
>>>* uninitialized; they will generally be garbage.  The resulting
>>> native
>>> @@ -687,6 +707,27 @@
>>>   }
>>>   /**
>>> + * Resizes a new block of native memory for DirectByteBuffers, to 

>>> the
>>> + * given size in bytes.  The contents of the new block past the 
size
>>> of
>>> + * the old block are uninitialized; they will generally be 
garbage.
>>> The
>>> + * resulting native pointer will be zero if and only if the 
>>> requested
>>> size
>>> + * is zero.  The resulting native pointer will be aligned for all
>>> value
>>> + * types.  Dispose of this memory by calling {@link 
#freeDBBMemory},
>>> or
>>> +   

Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers

[Thomas Stüfe]

On Wed, Feb 14, 2018 at 1:53 PM, David Holmes 
wrote:

> On 14/02/2018 10:43 PM, David Holmes wrote:
>
>> Adding in core-libs-dev as there's nothing related to hotspot directly
>> here.
>>
>
> Correction, this is of course leading to a proposed change in hotspot to
> implement the new Unsafe methods and perform the native memory tracking. Of
> course we already have NMT so the obvious question is how this will fit in
> with NMT?
>
>
I thought Unsafe.allocateMemory is served by hotspot os::malloc(), is it
not? So, allocations should show up in NMT with "Unsafe_AllocateMemory0".

..Thomas



> David
>
>
> David
>>
>> On 14/02/2018 9:32 PM, Adam Farley8 wrote:
>>
>>> Hi All,
>>>
>>> Currently, diagnostic core files generated from OpenJDK seem to lump all
>>> of the
>>> native memory usages together, making it near-impossible for someone to
>>> figure
>>> out *what* is using all that memory in the event of a memory leak.
>>>
>>> The OpenJ9 VM has a feature which allows it to track the allocation of
>>> native
>>> memory for Direct Byte Buffers (DBBs), and to supply that information
>>> into
>>> the
>>> cores when they are generated. This makes it a *lot* easier to find out
>>> what is using
>>> all that native memory, making memory leak resolution less like some dark
>>> art, and
>>> more like logical debugging.
>>>
>>> To use this feature, there is a native method referenced in Unsafe.java.
>>> To open
>>> up this feature so that any VM can make use of it, the java code below
>>> sets the
>>> stage for it. This change starts letting people call DBB-specific methods
>>> when
>>> allocating native memory, and getting into the habit of using it.
>>>
>>> Thoughts?
>>>
>>> Best Regards
>>>
>>> Adam Farley
>>>
>>> P.S. Code:
>>>
>>> diff --git
>>> a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
>>> b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
>>> --- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
>>> +++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
>>> @@ -85,7 +85,7 @@
>>>   // Paranoia
>>>   return;
>>>   }
>>> -UNSAFE.freeMemory(address);
>>> +UNSAFE.freeDBBMemory(address);
>>>   address = 0;
>>>   Bits.unreserveMemory(size, capacity);
>>>   }
>>> @@ -118,7 +118,7 @@
>>>   long base = 0;
>>>   try {
>>> -base = UNSAFE.allocateMemory(size);
>>> +base = UNSAFE.allocateDBBMemory(size);
>>>   } catch (OutOfMemoryError x) {
>>>   Bits.unreserveMemory(size, cap);
>>>   throw x;
>>> diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
>>> b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
>>> --- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
>>> +++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
>>> @@ -632,6 +632,26 @@
>>>   }
>>>   /**
>>> + * Allocates a new block of native memory for DirectByteBuffers, of
>>> the
>>> + * given size in bytes.  The contents of the memory are
>>> uninitialized;
>>> + * they will generally be garbage.  The resulting native pointer
>>> will
>>> + * never be zero, and will be aligned for all value types.  Dispose
>>> of
>>> + * this memory by calling {@link #freeDBBMemory} or resize it with
>>> + * {@link #reallocateDBBMemory}.
>>> + *
>>> + * @throws RuntimeException if the size is negative or too large
>>> + *  for the native size_t type
>>> + *
>>> + * @throws OutOfMemoryError if the allocation is refused by the
>>> system
>>> + *
>>> + * @see #getByte(long)
>>> + * @see #putByte(long, byte)
>>> + */
>>> +public long allocateDBBMemory(long bytes) {
>>> +return allocateMemory(bytes);
>>> +}
>>> +
>>> +/**
>>>* Resizes a new block of native memory, to the given size in
>>> bytes.
>>> The
>>>* contents of the new block past the size of the old block are
>>>* uninitialized; they will generally be garbage.  The resulting
>>> native
>>> @@ -687,6 +707,27 @@
>>>   }
>>>   /**
>>> + * Resizes a new block of native memory for DirectByteBuffers, to
>>> the
>>> + * given size in bytes.  The contents of the new block past the size
>>> of
>>> + * the old block are uninitialized; they will generally be garbage.
>>> The
>>> + * resulting native pointer will be zero if and only if the
>>> requested
>>> size
>>> + * is zero.  The resulting native pointer will be aligned for all
>>> value
>>> + * types.  Dispose of this memory by calling {@link #freeDBBMemory},
>>> or
>>> + * resize it with {@link #reallocateDBBMemory}.  The address passed
>>> to
>>> + * this method may be null, in which case an allocation will be
>>> performed.
>>> + *
>>> + * @throws RuntimeException if the size is 

Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers

[Adam Farley8]

> Adding in core-libs-dev as there's nothing related to hotspot directly 
here.
>
> David

Thought it was best to pass this through hotspot lists first, as full 
completion
of the native side of things will probably require hotspot changes.

You're quite right though, I should have cc'd hotspot *and* core libs.

- Adam

>
> On 14/02/2018 9:32 PM, Adam Farley8 wrote:
>> Hi All,
>> 
>> Currently, diagnostic core files generated from OpenJDK seem to lump 
all
>> of the
>> native memory usages together, making it near-impossible for someone to
>> figure
>> out *what* is using all that memory in the event of a memory leak.
>> 
>> The OpenJ9 VM has a feature which allows it to track the allocation of
>> native
>> memory for Direct Byte Buffers (DBBs), and to supply that information 
into
>> the
>> cores when they are generated. This makes it a *lot* easier to find out
>> what is using
>> all that native memory, making memory leak resolution less like some 
dark
>> art, and
>> more like logical debugging.
>> 
>> To use this feature, there is a native method referenced in 
Unsafe.java.
>> To open
>> up this feature so that any VM can make use of it, the java code below
>> sets the
>> stage for it. This change starts letting people call DBB-specific 
methods
>> when
>> allocating native memory, and getting into the habit of using it.
>> 
>> Thoughts?
>> 
>> Best Regards
>> 
>> Adam Farley
>> 
>> P.S. Code:
>> 
>> diff --git
>> a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
>> b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
>> --- 
a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
>> +++ 
b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
>> @@ -85,7 +85,7 @@
>>   // Paranoia
>>   return;
>>   }
>> -UNSAFE.freeMemory(address);
>> +UNSAFE.freeDBBMemory(address);
>>   address = 0;
>>   Bits.unreserveMemory(size, capacity);
>>   }
>> @@ -118,7 +118,7 @@
>> 
>>   long base = 0;
>>   try {
>> -base = UNSAFE.allocateMemory(size);
>> +base = UNSAFE.allocateDBBMemory(size);
>>   } catch (OutOfMemoryError x) {
>>   Bits.unreserveMemory(size, cap);
>>   throw x;
>> diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
>> b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
>> --- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
>> +++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
>> @@ -632,6 +632,26 @@
>>   }
>> 
>>   /**
>> + * Allocates a new block of native memory for DirectByteBuffers, 
of
>> the
>> + * given size in bytes.  The contents of the memory are
>> uninitialized;
>> + * they will generally be garbage.  The resulting native pointer 
will
>> + * never be zero, and will be aligned for all value types. Dispose
>> of
>> + * this memory by calling {@link #freeDBBMemory} or resize it with
>> + * {@link #reallocateDBBMemory}.
>> + *
>> + * @throws RuntimeException if the size is negative or too large
>> + *  for the native size_t type
>> + *
>> + * @throws OutOfMemoryError if the allocation is refused by the
>> system
>> + *
>> + * @see #getByte(long)
>> + * @see #putByte(long, byte)
>> + */
>> +public long allocateDBBMemory(long bytes) {
>> +return allocateMemory(bytes);
>> +}
>> +
>> +/**
>>* Resizes a new block of native memory, to the given size in 
bytes.
>> The
>>* contents of the new block past the size of the old block are
>>* uninitialized; they will generally be garbage.  The resulting
>> native
>> @@ -687,6 +707,27 @@
>>   }
>> 
>>   /**
>> + * Resizes a new block of native memory for DirectByteBuffers, to 
the
>> + * given size in bytes.  The contents of the new block past the 
size
>> of
>> + * the old block are uninitialized; they will generally be 
garbage.
>> The
>> + * resulting native pointer will be zero if and only if the 
requested
>> size
>> + * is zero.  The resulting native pointer will be aligned for all
>> value
>> + * types.  Dispose of this memory by calling {@link 
#freeDBBMemory},
>> or
>> + * resize it with {@link #reallocateDBBMemory}.  The address 
passed
>> to
>> + * this method may be null, in which case an allocation will be
>> performed.
>> + *
>> + * @throws RuntimeException if the size is negative or too large
>> + *  for the native size_t type
>> + *
>> + * @throws OutOfMemoryError if the allocation is refused by the
>> system
>> + *
>> + * @see #allocateDBBMemory
>> + */
>> +public long reallocateDBBMemory(long address, long bytes) {
>> +return reallocateMemory(address, bytes);
>> +}
>> +
>> +/**
>>* Sets all bytes in a given 

Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers

[David Holmes]

On 14/02/2018 10:43 PM, David Holmes wrote:
Adding in core-libs-dev as there's nothing related to hotspot directly 
here.


Correction, this is of course leading to a proposed change in hotspot to 
implement the new Unsafe methods and perform the native memory tracking. 
Of course we already have NMT so the obvious question is how this will 
fit in with NMT?


David


David

On 14/02/2018 9:32 PM, Adam Farley8 wrote:

Hi All,

Currently, diagnostic core files generated from OpenJDK seem to lump all
of the
native memory usages together, making it near-impossible for someone to
figure
out *what* is using all that memory in the event of a memory leak.

The OpenJ9 VM has a feature which allows it to track the allocation of
native
memory for Direct Byte Buffers (DBBs), and to supply that information 
into

the
cores when they are generated. This makes it a *lot* easier to find out
what is using
all that native memory, making memory leak resolution less like some dark
art, and
more like logical debugging.

To use this feature, there is a native method referenced in Unsafe.java.
To open
up this feature so that any VM can make use of it, the java code below
sets the
stage for it. This change starts letting people call DBB-specific methods
when
allocating native memory, and getting into the habit of using it.

Thoughts?

Best Regards

Adam Farley

P.S. Code:

diff --git
a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
--- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
+++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
@@ -85,7 +85,7 @@
  // Paranoia
  return;
  }
-    UNSAFE.freeMemory(address);
+    UNSAFE.freeDBBMemory(address);
  address = 0;
  Bits.unreserveMemory(size, capacity);
  }
@@ -118,7 +118,7 @@
  long base = 0;
  try {
-    base = UNSAFE.allocateMemory(size);
+    base = UNSAFE.allocateDBBMemory(size);
  } catch (OutOfMemoryError x) {
  Bits.unreserveMemory(size, cap);
  throw x;
diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
--- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
+++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
@@ -632,6 +632,26 @@
  }
  /**
+ * Allocates a new block of native memory for DirectByteBuffers, of
the
+ * given size in bytes.  The contents of the memory are
uninitialized;
+ * they will generally be garbage.  The resulting native pointer 
will

+ * never be zero, and will be aligned for all value types.  Dispose
of
+ * this memory by calling {@link #freeDBBMemory} or resize it with
+ * {@link #reallocateDBBMemory}.
+ *
+ * @throws RuntimeException if the size is negative or too large
+ *  for the native size_t type
+ *
+ * @throws OutOfMemoryError if the allocation is refused by the
system
+ *
+ * @see #getByte(long)
+ * @see #putByte(long, byte)
+ */
+    public long allocateDBBMemory(long bytes) {
+    return allocateMemory(bytes);
+    }
+
+    /**
   * Resizes a new block of native memory, to the given size in 
bytes.

The
   * contents of the new block past the size of the old block are
   * uninitialized; they will generally be garbage.  The resulting
native
@@ -687,6 +707,27 @@
  }
  /**
+ * Resizes a new block of native memory for DirectByteBuffers, to 
the

+ * given size in bytes.  The contents of the new block past the size
of
+ * the old block are uninitialized; they will generally be garbage.
The
+ * resulting native pointer will be zero if and only if the 
requested

size
+ * is zero.  The resulting native pointer will be aligned for all
value
+ * types.  Dispose of this memory by calling {@link #freeDBBMemory},
or
+ * resize it with {@link #reallocateDBBMemory}.  The address passed
to
+ * this method may be null, in which case an allocation will be
performed.
+ *
+ * @throws RuntimeException if the size is negative or too large
+ *  for the native size_t type
+ *
+ * @throws OutOfMemoryError if the allocation is refused by the
system
+ *
+ * @see #allocateDBBMemory
+ */
+    public long reallocateDBBMemory(long address, long bytes) {
+    return reallocateMemory(address, bytes);
+    }
+
+    /**
   * Sets all bytes in a given block of memory to a fixed value
   * (usually zero).
   *
@@ -918,6 +959,17 @@
  checkPointer(null, address);
  }
+    /**
+ * Disposes of a block of native memory, as obtained from {@link
+ * #allocateDBBMemory} or {@link #reallocateDBBMemory}.  The address
passed
+ * to this method may be null, in which 

Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers

[David Holmes]

Adding in core-libs-dev as there's nothing related to hotspot directly here.

David

On 14/02/2018 9:32 PM, Adam Farley8 wrote:

Hi All,

Currently, diagnostic core files generated from OpenJDK seem to lump all
of the
native memory usages together, making it near-impossible for someone to
figure
out *what* is using all that memory in the event of a memory leak.

The OpenJ9 VM has a feature which allows it to track the allocation of
native
memory for Direct Byte Buffers (DBBs), and to supply that information into
the
cores when they are generated. This makes it a *lot* easier to find out
what is using
all that native memory, making memory leak resolution less like some dark
art, and
more like logical debugging.

To use this feature, there is a native method referenced in Unsafe.java.
To open
up this feature so that any VM can make use of it, the java code below
sets the
stage for it. This change starts letting people call DBB-specific methods
when
allocating native memory, and getting into the habit of using it.

Thoughts?

Best Regards

Adam Farley

P.S. Code:

diff --git
a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
--- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
+++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
@@ -85,7 +85,7 @@
  // Paranoia
  return;
  }
-UNSAFE.freeMemory(address);
+UNSAFE.freeDBBMemory(address);
  address = 0;
  Bits.unreserveMemory(size, capacity);
  }
@@ -118,7 +118,7 @@
  
  long base = 0;

  try {
-base = UNSAFE.allocateMemory(size);
+base = UNSAFE.allocateDBBMemory(size);
  } catch (OutOfMemoryError x) {
  Bits.unreserveMemory(size, cap);
  throw x;
diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
--- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
+++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
@@ -632,6 +632,26 @@
  }
  
  /**

+ * Allocates a new block of native memory for DirectByteBuffers, of
the
+ * given size in bytes.  The contents of the memory are
uninitialized;
+ * they will generally be garbage.  The resulting native pointer will
+ * never be zero, and will be aligned for all value types.  Dispose
of
+ * this memory by calling {@link #freeDBBMemory} or resize it with
+ * {@link #reallocateDBBMemory}.
+ *
+ * @throws RuntimeException if the size is negative or too large
+ *  for the native size_t type
+ *
+ * @throws OutOfMemoryError if the allocation is refused by the
system
+ *
+ * @see #getByte(long)
+ * @see #putByte(long, byte)
+ */
+public long allocateDBBMemory(long bytes) {
+return allocateMemory(bytes);
+}
+
+/**
   * Resizes a new block of native memory, to the given size in bytes.
The
   * contents of the new block past the size of the old block are
   * uninitialized; they will generally be garbage.  The resulting
native
@@ -687,6 +707,27 @@
  }
  
  /**

+ * Resizes a new block of native memory for DirectByteBuffers, to the
+ * given size in bytes.  The contents of the new block past the size
of
+ * the old block are uninitialized; they will generally be garbage.
The
+ * resulting native pointer will be zero if and only if the requested
size
+ * is zero.  The resulting native pointer will be aligned for all
value
+ * types.  Dispose of this memory by calling {@link #freeDBBMemory},
or
+ * resize it with {@link #reallocateDBBMemory}.  The address passed
to
+ * this method may be null, in which case an allocation will be
performed.
+ *
+ * @throws RuntimeException if the size is negative or too large
+ *  for the native size_t type
+ *
+ * @throws OutOfMemoryError if the allocation is refused by the
system
+ *
+ * @see #allocateDBBMemory
+ */
+public long reallocateDBBMemory(long address, long bytes) {
+return reallocateMemory(address, bytes);
+}
+
+/**
   * Sets all bytes in a given block of memory to a fixed value
   * (usually zero).
   *
@@ -918,6 +959,17 @@
  checkPointer(null, address);
  }
  
+/**

+ * Disposes of a block of native memory, as obtained from {@link
+ * #allocateDBBMemory} or {@link #reallocateDBBMemory}.  The address
passed
+ * to this method may be null, in which case no action is taken.
+ *
+ * @see #allocateDBBMemory
+ */
+public void freeDBBMemory(long address) {
+freeMemory(address);
+}
+
  /// random queries
  
  /**


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and