Re: [racket-users] Safely allocating memory in places enabled world

[Matthew Flatt]

At Mon, 2 Oct 2017 16:48:40 -0400, George Neuner wrote:
> 
> On 10/2/2017 2:52 PM, Matthew Flatt wrote:
> > Meanwhile, there's no similar way to order
> > callbacks that are registered with a custodian.
> 
> IIRC, the JVM executes finalizers in reverse order of registration - 
> expecting that, in your example, A would have been allocated before B.  
> Is that something that would work here? [He asks naively.]

That's an interesting idea. Adding an order on finalization in general
is not easy, but it might be easy enough for custodian-shutdown actions
to be ordered that way, and it does seem like a helpful guarantee.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Safely allocating memory in places enabled world

[George Neuner]

Hi Eric,

On 10/2/2017 9:40 PM, Eric Dobson wrote:
George: I don't see that invariant for java in anything that I search 
for on Java finalizers. Do you have a reference? In particular java 
only has one finalizer per object (the finalize method) and across 
objects the references I found seem to imply that there is no 
guaranteed order?


https://en.wikipedia.org/wiki/Finalization#Connection_with_finally 
seems to imply no ordering across finalizers.


What I recall was that members were guaranteed to be finalized (if 
needed) before their parent object - the reverse of the initialization 
process.  This did not necessary conflict with "objects may be finalized 
in any order" ... it simply meant that when the parent was to be 
finalized, some or all of the members also needing finalization might 
have already been done.


Unfortunately I don't recall exactly where I saw it ... I'm pretty sure 
it was in a paper on JVM internals and so was maybe about what was 
actually done vs what was guaranteed by specification.  And it was quite 
a while ago that I was studying virtual machines, so it may no longer be 
true of current implementations.  I'll have to look around and see if I 
can find it again.


Even so, my question to Matthew was - would something like this be 
workable for Racket?  I don't know how much additional bookkeeping would 
be required.



Matthew: Ok, I'll just file a bug to track the issue. I'm trying to do 
the best available solution following standard practices, and not 
actually that worried about the memory leak for my program.


On Mon, Oct 2, 2017 at 1:48 PM, George Neuner > wrote:




IIRC, the JVM executes finalizers in reverse order of registration
- expecting that, in your example, A would have been allocated
before B.  Is that something that would work here?  [He asks naively.]




George

--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Safely allocating memory in places enabled world

[Eric Dobson]

George: I don't see that invariant for java in anything that I search for
on Java finalizers. Do you have a reference? In particular java only has
one finalizer per object (the finalize method) and across objects the
references I found seem to imply that there is no guaranteed order?

https://en.wikipedia.org/wiki/Finalization#Connection_with_finally seems to
imply no ordering across finalizers.


Matthew: Ok, I'll just file a bug to track the issue. I'm trying to do the
best available solution following standard practices, and not actually that
worried about the memory leak for my program.

On Mon, Oct 2, 2017 at 1:48 PM, George Neuner  wrote:

>
> On 10/2/2017 2:52 PM, Matthew Flatt wrote:
>
> If we change `ffi/unsafe/alloc` so that a custodian shutdown runs all
> deallocators, that would work in many cases. It would create a problem,
> however, if there are objects to deallocate where the deallocation
> function references other objects that themselves must be deallocated.
> That is, an allocated value A might depend on an allocated value B up
> until the point that A has been deallocated. That dependency can be
> expressed to `ffi/unsafe/alloc` (via the garbage collector) by
> including a reference to B in the deallocation closure for A. Setting
> up dependencies that way is not common, but I'm pretty sure I've used
> that pattern once or twice. Meanwhile, there's no similar way to order
> callbacks that are registered with a custodian.
>
>
> IIRC, the JVM executes finalizers in reverse order of registration -
> expecting that, in your example, A would have been allocated before B.  Is
> that something that would work here?  [He asks naively.]
>
> I'm tempted to say that `ffi/unsafe/alloc` should only be used for
> objects where the deallocation order doesn't matter, so that
> deallocators can be triggered by a custodian shutdown. But an
> additional constraint is needed: values to be deallocated cannot be
> used by any other custodian-shutdown callbacks. That additional
> constraint seems too limiting.
>
> Maybe we need to add some sort of ordering constraints to custodians,
> but I'm not immediately sure of the right way to do that.
>
>
> George
>
> --
> You received this message because you are subscribed to the Google Groups
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to racket-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Safely allocating memory in places enabled world

[George Neuner]

On 10/2/2017 2:52 PM, Matthew Flatt wrote:

If we change `ffi/unsafe/alloc` so that a custodian shutdown runs all
deallocators, that would work in many cases. It would create a problem,
however, if there are objects to deallocate where the deallocation
function references other objects that themselves must be deallocated.
That is, an allocated value A might depend on an allocated value B up
until the point that A has been deallocated. That dependency can be
expressed to `ffi/unsafe/alloc` (via the garbage collector) by
including a reference to B in the deallocation closure for A. Setting
up dependencies that way is not common, but I'm pretty sure I've used
that pattern once or twice. Meanwhile, there's no similar way to order
callbacks that are registered with a custodian.


IIRC, the JVM executes finalizers in reverse order of registration - 
expecting that, in your example, A would have been allocated before B.  
Is that something that would work here? [He asks naively.]



I'm tempted to say that `ffi/unsafe/alloc` should only be used for
objects where the deallocation order doesn't matter, so that
deallocators can be triggered by a custodian shutdown. But an
additional constraint is needed: values to be deallocated cannot be
used by any other custodian-shutdown callbacks. That additional
constraint seems too limiting.

Maybe we need to add some sort of ordering constraints to custodians,
but I'm not immediately sure of the right way to do that.


George

--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Safely allocating memory in places enabled world

[Matthew Flatt]

At Sat, 30 Sep 2017 09:45:02 -0700, Eric Dobson wrote:
> I'm trying to write some racket code which interfaces with a foreign
> library and provides a safe interface. Some of the functions I'm calling
> allocate memory and need to have this explicitly freed by the caller. The
> 'allocator' binding from ffi/unsafe/alloc seems to solve this problem, but
> I'm running into issues using it from places other than the main one. The
> issue is that if a place gets shut down then the finalizer that was
> registered don't seem to run, and I can observe the leaked memory at the
> process level.
> 
> I took a look at ffi/unsafe/custodian and it seems that
> register-custodian-shutdown would allow this to be implemented. Is there a
> reason that such allocator doesn't do this by default?

The simple answer is that I didn't think about places originally, but
there are some challenges beyond just wiring in a custodian hook.

If we change `ffi/unsafe/alloc` so that a custodian shutdown runs all
deallocators, that would work in many cases. It would create a problem,
however, if there are objects to deallocate where the deallocation
function references other objects that themselves must be deallocated.
That is, an allocated value A might depend on an allocated value B up
until the point that A has been deallocated. That dependency can be
expressed to `ffi/unsafe/alloc` (via the garbage collector) by
including a reference to B in the deallocation closure for A. Setting
up dependencies that way is not common, but I'm pretty sure I've used
that pattern once or twice. Meanwhile, there's no similar way to order
callbacks that are registered with a custodian.

I'm tempted to say that `ffi/unsafe/alloc` should only be used for
objects where the deallocation order doesn't matter, so that
deallocators can be triggered by a custodian shutdown. But an
additional constraint is needed: values to be deallocated cannot be
used by any other custodian-shutdown callbacks. That additional
constraint seems too limiting.

Maybe we need to add some sort of ordering constraints to custodians,
but I'm not immediately sure of the right way to do that.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.