On Wed, Sep 9, 2015 at 11:46 AM, Peter Levart wrote:
>
> By wanting to truly release the resources you allocated, you are essentially
> wanting to manage the resources yourself. If you are willing to track the
> active mapped byte buffers manually yourself, then what about
It seems less than ideal to count on System.gc to do this as a library
though.
Now the user has to worry about what affect System.gc has on what JVM with
what Garbage Collector and whether or not ExplicitGCInvokesConcurrent was
turned on for the JVM, or...
- Mark
On Wed, Sep 9, 2015 at 11:46 AM
On 11/09/2015 12:56 AM, Vitaly Davidovich wrote:
Sure, if you like, but that's a detail. We are arguing about what
colour to paint the bike shed. :)
The reason it may be more than just a bikeshed is because if this requires
more cleverness in the segfault handler, then it's yet more work
On 09/11/2015 07:15 AM, David Holmes wrote:
> On 11/09/2015 12:56 AM, Vitaly Davidovich wrote:
>>>
>>> Sure, if you like, but that's a detail. We are arguing about what
>>> colour to paint the bike shed. :)
>>
>>
>> The reason it may be more than just a bikeshed is because if this requires
>>
On 09/11/2015 12:42 AM, Vitaly Davidovich wrote:
There's still a race condition since someone may have passed the
guard, entered a safepoint, and then proceeded to access the memory.
Yeah, and I realized yesterday after posting that this wouldn't work for
views of the buffer
On 11/09/2015 6:37 PM, Andrew Haley wrote:
On 09/11/2015 07:15 AM, David Holmes wrote:
On 11/09/2015 12:56 AM, Vitaly Davidovich wrote:
Sure, if you like, but that's a detail. We are arguing about what
colour to paint the bike shed. :)
The reason it may be more than just a bikeshed is
Those are precisely the ones I was referring to that we'd want to avoid
when requesting a safepoint for this scenario. That ought to be a matter
of simply passing a flag to safepoint synchronization telling it to skip
cleanup tasks.
sent from my phone
On Sep 11, 2015 6:01 AM, "David Holmes"
Ha
On 09/10/2015 02:17 AM, David Holmes wrote:
> On 10/09/2015 12:01 AM, Andrew Haley wrote:
>> On 09/09/2015 02:55 PM, David M. Lloyd wrote:
>>> On 09/09/2015 08:17 AM, Andrew Haley wrote:
On 09/09/2015 02:13 PM, David M. Lloyd wrote:
> How would you hook into the safepoint to perform
The safepoint happiness is unfortunately a separate issue in Hotspot, and
it's definitely not happy times :). Part of the problem is the
piggybacking of various operations on a safepoint - the safepoint time
alone (not counting GC, say) keeps growing. You probably could piggyback
this on
On 09/10/2015 12:25 PM, Vitaly Davidovich wrote:
> The safepoint happiness is unfortunately a separate issue in
> Hotspot, and it's definitely not happy times :). Part of the
> problem is the piggybacking of various operations on a safepoint -
> the safepoint time alone (not counting GC, say)
Or, the Java methods which wrap this access can just catch NPE and throw
the new exception type.
On 09/10/2015 09:35 AM, Vitaly Davidovich wrote:
Well, you'd probably want something other than NPE here -- perhaps a new
dedicated exception to signal this condition. And this means the segfault
On 09/10/2015 02:48 PM, David M. Lloyd wrote:
> On 09/10/2015 08:22 AM, Andrew Haley wrote:
>> On 09/10/2015 12:25 PM, Vitaly Davidovich wrote:
>>
>>> The safepoint happiness is unfortunately a separate issue in
>>> Hotspot, and it's definitely not happy times :). Part of the
>>> problem is the
On 09/10/2015 08:22 AM, Andrew Haley wrote:
On 09/10/2015 12:25 PM, Vitaly Davidovich wrote:
The safepoint happiness is unfortunately a separate issue in
Hotspot, and it's definitely not happy times :). Part of the
problem is the piggybacking of various operations on a safepoint -
the
>
> Sure, if you like, but that's a detail. We are arguing about what
> colour to paint the bike shed. :)
The reason it may be more than just a bikeshed is because if this requires
more cleverness in the segfault handler, then it's yet more work to make
this happen. I agree it's an
On 09/10/2015 03:13 PM, Vitaly Davidovich wrote:
> So what happens after safepoint is done and thread accesses the
> mapping (it would need protection NONE, not just write)? JVM is then
> going to trap the signal and convert to an exception?
One of us is missing something, and I don't think it's
On 09/10/2015 03:26 PM, Vitaly Davidovich wrote:
> Yes, so what happens when that guard page is accessed by a thread after
> safepoint?
A segfault and a null pointer exception.
Andrew.
On 09/10/2015 03:56 PM, Vitaly Davidovich wrote:
> Sure, if you like, but that's a detail. We are arguing about what
> colour to paint the bike shed. :)
>
> The reason it may be more than just a bikeshed is because if this requires
> more cleverness in the segfault handler, then it's
Well, you'd probably want something other than NPE here -- perhaps a new
dedicated exception to signal this condition. And this means the segfault
handling now needs to know about this type of situation as well, rather
than just NPEs.
On Thu, Sep 10, 2015 at 10:32 AM, Andrew Haley
On 09/10/2015 03:35 PM, Vitaly Davidovich wrote:
> Well, you'd probably want something other than NPE here -- perhaps a
> new dedicated exception to signal this condition.
Sure, if you like, but that's a detail. We are arguing about what
colour to paint the bike shed. :)
> And this means the
So what happens after safepoint is done and thread accesses the mapping (it
would need protection NONE, not just write)? JVM is then going to trap the
signal and convert to an exception?
sent from my phone
On Sep 10, 2015 10:06 AM, "Andrew Haley" wrote:
> On 09/10/2015 02:48
Whether biased locking being enabled by default is up for debate; my
opinion is it should be off on modern CPUs. If you're not contending, CAS
isn't much of an issue. If you are contending, then you'll get revocations
which is more expensive than the CAS. Biased locking would be profitable
on
Yes, so what happens when that guard page is accessed by a thread after
safepoint? The safepoint does nothing, as you said, it just ensures no
thread is actively working against the mapping but it may still be
reachable.
sent from my phone
On Sep 10, 2015 10:22 AM, "Andrew Haley"
Sure, if we're comfortable saying that the only cause of NPE there is due
to this and not something else.
On Thu, Sep 10, 2015 at 10:37 AM, David M. Lloyd
wrote:
> Or, the Java methods which wrap this access can just catch NPE and throw
> the new exception type.
>
>
> On
> there're already ways to bring down the JVM (e.g. JNI, existing Unsafe usage,
> etc),
If you can somehow pass switches to the JVM, this one wins my personal
beauty contest (from globals.hpp):
product(intx, SelfDestructTimer, 0, \
"Will cause VM
To be fair, this one sounds like it would initiate an orderly/graceful
shutdown whereas a bug in JNI/Unsafe kills it almost instantly (modulo
error reporting/core dumping) :)
On Thu, Sep 10, 2015 at 2:47 PM, Dawid Weiss wrote:
> > there're already ways to bring down the
Hi guys,
Perhaps there's no need for this protection/trap dance. If the situation
is never tripped in correct programs (that unmap only after noone is
using the buffers any more), then checking for address and throwing in
case it is equal to some guard value is a never taken branch that is
There's still a race condition since someone may have passed the guard,
entered a safepoint, and then proceeded to access the memory.
sent from my phone
On Sep 10, 2015 6:29 PM, "Peter Levart" wrote:
> Hi guys,
>
> Perhaps there's no need for this protection/trap dance.
On 09/08/2015 08:15 PM, Remi Forax wrote:
> On 09/08/2015 03:29 PM, Andrew Haley wrote:
>> On 09/08/2015 02:05 PM, Andrew Haley wrote:
>>> I don't think you'd actually need to unmap anything until a safepoint.
>>> I don't think that the speed of unmapping is critical as long as it
>>> happens
On 09/09/2015 01:41 PM, David M. Lloyd wrote:
> If the answer to both of these can be shown to be affirmative, then I
> think there is a real viable solution which allows immediate release of
> backing resources, with the address space being reclaimed by GC.
GC delays can be days if an object
On 09/09/2015 02:13 PM, David M. Lloyd wrote:
> How would you hook into the safepoint to perform the unmapping? You'd
> have to wait for all threads to arrive at safepoints, wouldn't you?
Yes. That happens in the VM already, e.g. when we need to revoke the
bias of a lock. It's a
On 09/09/2015 01:46 PM, Mike Hearn wrote:
> I guess, although it's ugly, a system property could control whether the
> NIO implementation returns an ordinary MappedByteBuffer or a new subclass,
> the UnmappableMappedByteBuffer. HotSpot would then be responsible for
> removing the overhead of the
On 09/09/2015 01:58 PM, David M. Lloyd wrote:
> On 09/09/2015 07:49 AM, Andrew Haley wrote:
>> On 09/09/2015 01:41 PM, David M. Lloyd wrote:
>>> If the answer to both of these can be shown to be affirmative, then I
>>> think there is a real viable solution which allows immediate release of
>>>
On 09/09/2015 08:17 AM, Andrew Haley wrote:
On 09/09/2015 02:13 PM, David M. Lloyd wrote:
How would you hook into the safepoint to perform the unmapping? You'd
have to wait for all threads to arrive at safepoints, wouldn't you?
Yes. That happens in the VM already, e.g. when we need to
On 09/09/2015 13:41, David M. Lloyd wrote:
If you have access to a Windows development environment, it seems to
me that you could help directly with the testing required to determine
a solution.
I think my proposed remapping approach will work, but it is contingent
on testing the following
On 09/09/2015 07:49 AM, Andrew Haley wrote:
On 09/09/2015 01:41 PM, David M. Lloyd wrote:
If the answer to both of these can be shown to be affirmative, then I
think there is a real viable solution which allows immediate release of
backing resources, with the address space being reclaimed by
On 09/09/2015 02:55 PM, David M. Lloyd wrote:
> On 09/09/2015 08:17 AM, Andrew Haley wrote:
>> On 09/09/2015 02:13 PM, David M. Lloyd wrote:
>>> How would you hook into the safepoint to perform the unmapping? You'd
>>> have to wait for all threads to arrive at safepoints, wouldn't you?
>>
>> Yes.
If you have access to a Windows development environment, it seems to me
that you could help directly with the testing required to determine a
solution.
I think my proposed remapping approach will work, but it is contingent
on testing the following on Windows:
* Determine if it is possible
Hi Uwe,
As I thought, the problem for some seems to be non-prompt unmapping of
mapped address space held by otherwise unreachable mapped byte buffers.
The mapped address space doesn't live in the Java heap and doesn't
represent a heap memory pressure, so GC doesn't kick-in automatically
when
On 09/09/2015 04:21 PM, Peter Levart wrote:
Hi Uwe,
As I thought, the problem for some seems to be non-prompt unmapping of
mapped address space held by otherwise unreachable mapped byte
buffers. The mapped address space doesn't live in the Java heap and
doesn't represent a heap memory
Hi,
> As I thought, the problem for some seems to be non-prompt unmapping of
> mapped address space held by otherwise unreachable mapped byte buffers.
> The mapped address space doesn't live in the Java heap and doesn't
> represent a heap memory pressure, so GC doesn't kick-in automatically
>
Hi,
> As I thought, the problem for some seems to be non-prompt unmapping of
> mapped address space held by otherwise unreachable mapped byte buffers.
> The mapped address space doesn't live in the Java heap and doesn't
> represent a heap memory pressure, so GC doesn't kick-in automatically
>
> - you track the number of mapped buffers (or mapped address space) that you
> "know" is active in the application manually.
The problem is you really can't do it on a global, JVM-scale, Peter.
It's enough that the same JVM process starts two isolated class
loaders with Lucene in each and such
On 09/09/2015 04:56 PM, Dawid Weiss wrote:
I think it would be best to leave to the application to decide and
implement the tracking and also triggering GC at times when it approaches
the limit.
I disagree. The GC -- when and how it is triggered -- should be
transparent to the application. We
> I think it would be best to leave to the application to decide and
> implement the tracking and also triggering GC at times when it approaches
> the limit.
I disagree. The GC -- when and how it is triggered -- should be
transparent to the application. We don't want to manage GC, we want to
On 10/09/2015 12:01 AM, Andrew Haley wrote:
On 09/09/2015 02:55 PM, David M. Lloyd wrote:
On 09/09/2015 08:17 AM, Andrew Haley wrote:
On 09/09/2015 02:13 PM, David M. Lloyd wrote:
How would you hook into the safepoint to perform the unmapping? You'd
have to wait for all threads to arrive at
I think I've managed to reasonably firmly establish that atomically
remapping memory is impossible on Windows based on various anecdotal
evidence found from searching around (particularly on Cygwin mailing
list archives).
Another possible option is to use other Windows memory management APIs
> À: core-libs-dev@openjdk.java.net
> Envoyé: Mercredi 9 Septembre 2015 15:17:31
> Objet: Re: Suggested fix for JDK-4724038 (Add unmap method to
> MappedByteBuffer)
>
> On 09/09/2015 02:13 PM, David M. Lloyd wrote:
> > How would you hook into the safepoint to perform the unmappi
s how one could delete the file after it has been unmapped (if
> the byte buffer was the only mapping).
> Would such notification be enough to cover the use cases? What are the use
> cases one would like to cover with hypothetical unmap() was it available?
> Regards, Peter
HI Mike,
This is fundamentally about *integrity* of the runtime. It follows there are
security implications, but it’s still fundamentally an integrity issue and
guarding an unsafe operation with a Security Manager is unfortunately an
insufficient solution.
Paul.
On 7 Sep 2015, at 18:41, Mike
On 09/08/2015 09:58 AM, Paul Sandoz wrote:
> This is fundamentally about *integrity* of the runtime. It follows
> there are security implications, but it’s still fundamentally an
> integrity issue and guarding an unsafe operation with a Security
> Manager is unfortunately an insufficient
On 8 Sep 2015, at 11:30, Andrew Haley wrote:
> On 09/08/2015 09:58 AM, Paul Sandoz wrote:
>
>> This is fundamentally about *integrity* of the runtime. It follows
>> there are security implications, but it’s still fundamentally an
>> integrity issue and guarding an unsafe
On 08/09/2015 09:58, Paul Sandoz wrote:
HI Mike,
This is fundamentally about *integrity* of the runtime. It follows there are
security implications, but it’s still fundamentally an integrity issue and
guarding an unsafe operation with a Security Manager is unfortunately an
insufficient
Sorry, seems I must have hit send without noticing.
Windows has this:
https://msdn.microsoft.com/en-us/library/windows/desktop/aa366890(v=vs.85).aspx
However the question is, what happens if you remap a range and then unmap
the file from that same range?
I'm assuming the file unmap operation
On 09/08/2015 06:54 PM, David M. Lloyd wrote:
> I think the only questionable platform at this point is Windows (to me
> at least, given that I do not have much experience on it).
That's right, and it's Windows which has the worst problem: on UNIX
you can delete a file which is mapped, on
On 09/08/2015 01:37 PM, Andrew Haley wrote:
On 09/08/2015 06:54 PM, David M. Lloyd wrote:
I think the only questionable platform at this point is Windows (to me
at least, given that I do not have much experience on it).
That's right, and it's Windows which has the worst problem: on UNIX
you
You'd want to then MapViewOfFileEx later using VirtualAllocEx reservation
(to change the mapping), but I don't think MapViewOfFileEx allows
specifying a base address that is reserved via VirtualAllocEx.
On Tue, Sep 8, 2015 at 3:41 PM, Mike Hearn wrote:
> Sorry, seems I must
>
I think a simple way to solve that is to ask for a safepoint explicitly,
buffer.unmap();
waitUntilUnmapped();
file.delete();
Rémi
- Mail original -
> De: "Andrew Haley" <a...@redhat.com>
> À: core-libs-dev@openjdk.java.net
> Envoyé: Mardi 8 Septembre 2015
>
> I think that MR is referring to Windows when he talks about
> race conditions.
The race can happen on any platform if you can't do an atomic unmap of a
file and remap to some kind of guard page. I guess on Linux you can, maybe?
I've never tried it.
It may be that on Windows if you remap a
>
> So Windows has no operation which can remap a previously mapped section of
> memory to something else? (say, just a block of non-access memory)
>
> Note that such an operation has to be atomic with respect to all other
> operations which affect the address space of the process (or at least,
>
Am 08.09.2015 um 20:37 schrieb Andrew Haley:
I think that MR is referring to Windows when he talks about race conditions. Andrew.
Couldn't we introduce a unmap() method which throws an UnsupportedOperationException if the
underlying OS isn't able to unmap the the buffer safely in the mean
edhat.com>
À: core-libs-dev@openjdk.java.net
Envoyé: Mardi 8 Septembre 2015 15:29:25
Objet: Re: Suggested fix for JDK-4724038 (Add unmap method to
MappedByteBuffer)
On 09/08/2015 02:05 PM, Andrew Haley wrote:
I don't think you'd actually need to unmap anything until a safepoint.
I
On 09/08/2015 12:37 PM, David M. Lloyd wrote:
> On 09/08/2015 04:30 AM, Andrew Haley wrote:
>> On 09/08/2015 09:58 AM, Paul Sandoz wrote:
>
>> However, I think that some cleverness in HotSpot could make that cost
>> go away. For example, we could associate with every
>>
>
> If you're already doing this, why not skip the level of indirection and
> mprotect the entire mapped region to PROT_NONE when the user unmaps?
> ... Then you do the real unmap when the buffer is GC'd (maybe via
> Cleaner).
Because the reason users are working around this is Windows,
> Because the reason users are working around this is Windows, where they
> need the ability to do:
>
> buffer.unmap();
> file.delete();
This is exactly the case in Lucene, for example. Which currently
resorts to ugly hacks (the entire documentation of this class is
interesting -- concerns memory
On 09/08/2015 02:05 PM, Andrew Haley wrote:
> I don't think you'd actually need to unmap anything until a safepoint.
> I don't think that the speed of unmapping is critical as long as it
> happens "soon".
Although given the desire to do
buffer.unmap();
file.delete();
that belief may be
On 09/08/2015 11:42 AM, Paul Sandoz wrote:
>
> On 8 Sep 2015, at 11:30, Andrew Haley wrote:
>> But never mind that; how about this idea? Create a
>> MappedByteBufferForwardingObject whose only job is to forward requests
>> to a MappedByteBuffer. That MappedByteBuffer does not
On 09/08/2015 12:37 PM, David M. Lloyd wrote:
> Then you do the real unmap when
> the buffer is GC'd (maybe via Cleaner). This is very akin to how file
> descriptors are cleaned up, AFAICT.
Indeed it is, and it's no better than the status quo. This is an attempt
to do better.
Andrew.
On 09/08/2015 06:03 PM, David M. Lloyd wrote:
> On 09/08/2015 08:11 AM, Andrew Haley wrote:
>> On 09/08/2015 12:37 PM, David M. Lloyd wrote:
>>> Then you do the real unmap when
>>> the buffer is GC'd (maybe via Cleaner). This is very akin to how file
>>> descriptors are cleaned up, AFAICT.
>>
>>
On 09/08/2015 12:40 PM, Andrew Haley wrote:
On 09/08/2015 06:03 PM, David M. Lloyd wrote:
On 09/08/2015 08:11 AM, Andrew Haley wrote:
On 09/08/2015 12:37 PM, David M. Lloyd wrote:
Then you do the real unmap when
the buffer is GC'd (maybe via Cleaner). This is very akin to how file
On 09/08/2015 08:03 AM, Mike Hearn wrote:
If you're already doing this, why not skip the level of indirection
and mprotect the entire mapped region to PROT_NONE when the user
unmaps? ... Then you do the real unmap when the buffer is GC'd
(maybe via Cleaner).
Because the
On 09/08/2015 08:11 AM, Andrew Haley wrote:
On 09/08/2015 12:37 PM, David M. Lloyd wrote:
Then you do the real unmap when
the buffer is GC'd (maybe via Cleaner). This is very akin to how file
descriptors are cleaned up, AFAICT.
Indeed it is, and it's no better than the status quo. This is
71 matches
Mail list logo