Re: Don't expect class destructors to be called at all by the GC
On Wed, Jan 31, 2018 at 01:38:07PM +, Bienlein via Digitalmars-d-learn wrote: > On Thursday, 21 December 2017 at 18:45:27 UTC, Adam D. Ruppe wrote: > > On Thursday, 21 December 2017 at 18:20:19 UTC, H. S. Teoh wrote: > > > When the scoped destruction of structs isn't an option, > > > RefCounted!T seems to be a less evil alternative than an > > > unreliable class dtor. :-/ > > > > Alas, RefCounted doesn't work well with inheritance... > > > > Though, what you could do is make the refcounted owners and borrow > > the actual reference later. > > Is there some summary of the things you have to be aware of when using > the GC in D and not using the GC? I feel this would be very useful > especially for people that are new to D or are not used to that kind > of issues (because coming from a GCed language). D was originally designed with a GC in mind, so if you're using the GC and writing typical GC'd code, there's really not much to be aware of, esp. if you're coming from a GC'd language. The only time you have to be actively concerned about the GC is if: 1) You're interacting with C code and passing pointers around. You may need to hold on to a copy of pointers you pass to C, because the GC does not know any pointer roots in C land, so it may wrongly collect something that the C code still holds a reference to. 2) You're writing performance-sensitive code and profiling shows that the GC is becoming a bottleneck. The usual easy solution is to import core.memory and make use of GC.disable, GC.collect, GC.enable at strategic points in your code. 3) You need to do useful work in dtors, and/or need deterministic destruction, which is fundamentally incompatible with the GC. In this case, probably your best bet is RefCounted or some other kind of non-GC memory management scheme. Lately I'm beginning more and more to just avoid class dtors in general, due to their inherent unreliability, and just use scoped destruction (structs with dtors), or RefCounted, or some other memory management scheme (perhaps raw malloc/free if I need tight control over things). T -- VI = Visual Irritation
Re: Don't expect class destructors to be called at all by the GC
On Wednesday, January 31, 2018 10:51:10 DanielG via Digitalmars-d-learn wrote: > On Wednesday, 31 January 2018 at 10:34:53 UTC, Mike Parker wrote: > > delete is deprecated: > > > > https://dlang.org/deprecate.html#delete > > Ah, thanks! Actually double-thanks, because my progress through > your book is what prompted me to search for threads about class > destructors. The existence of .destroy answers my question > (namely, "should I just use 'delete', or my own .dispose method, > for deterministic resource freeing?") The main problem with delete is that it's inherently unsafe. GC-managed memory is supposed to be @safe (it's one of the main reasons that D has a GC in the first place), but having the programmer go and delete a GC-managed object rather than waiting for the GC to do it makes it trivial to do wrong stuff like free an object's memory while it's still referenced by something else (the sort of thing that the GC is supposed to avoid). It's far better to either explicitly destroy the object without freeing its memory or to use memory that is not managed by the GC if you want deterministic destruction of an object on the heap. - Jonathan M Davis
Re: Don't expect class destructors to be called at all by the GC
On Thursday, 21 December 2017 at 18:45:27 UTC, Adam D. Ruppe wrote: On Thursday, 21 December 2017 at 18:20:19 UTC, H. S. Teoh wrote: When the scoped destruction of structs isn't an option, RefCounted!T seems to be a less evil alternative than an unreliable class dtor. :-/ Alas, RefCounted doesn't work well with inheritance... Though, what you could do is make the refcounted owners and borrow the actual reference later. Is there some summary of the things you have to be aware of when using the GC in D and not using the GC? I feel this would be very useful especially for people that are new to D or are not used to that kind of issues (because coming from a GCed language).
Re: Don't expect class destructors to be called at all by the GC
On Wednesday, 31 January 2018 at 10:34:53 UTC, Mike Parker wrote: delete is deprecated: https://dlang.org/deprecate.html#delete Ah, thanks! Actually double-thanks, because my progress through your book is what prompted me to search for threads about class destructors. The existence of .destroy answers my question (namely, "should I just use 'delete', or my own .dispose method, for deterministic resource freeing?")
Re: Don't expect class destructors to be called at all by the GC
On Wednesday, 31 January 2018 at 10:14:53 UTC, DanielG wrote: Pardon my probable ignorance (D newbie and all), but why wouldn't a 'delete' work for this? https://dlang.org/spec/expression.html#delete_expressions delete is deprecated: https://dlang.org/deprecate.html#delete
Re: Don't expect class destructors to be called at all by the GC
On Thursday, 21 December 2017 at 18:20:19 UTC, H. S. Teoh wrote: I ended up calling .destroy on the class instance explicitly just so the destructor would run at the right time, right before nulling the reference so that the GC would collect the memory. Pardon my probable ignorance (D newbie and all), but why wouldn't a 'delete' work for this? https://dlang.org/spec/expression.html#delete_expressions
Re: Don't expect class destructors to be called at all by the GC
On Thursday, December 28, 2017 10:37:09 H. S. Teoh via Digitalmars-d-learn wrote: > On Sun, Dec 24, 2017 at 02:07:26PM -0700, Jonathan M Davis via > Digitalmars-d-learn wrote: [...] > > > Regardless, even if it were the case that it were guaranteed that all > > finalizers were run when the program exited, it would still be > > terrible practice to rely on it. It's trivial to end up in a situation > > where no collection is run for quite some time (e.g. just don't do > > much memory allocation for a while), which would leave any resources > > that needed to be freed by a finalizer unfreed for quite a while even > > though they weren't needed anymore. So practically speaking, it > > doesn't really matter where the finalizers are guaranteed to run. > > Relying on them to be run rather than forcing them to be run via > > destroy or using some other helper function is just going to cause > > problems, so it's just plain bad practice to rely on finalizers to be > > run to release resources. That's just life with GC's in general, not > > just D's. It's why C# has the while dispose/IDisposable thing, and why > > D code should either be using destroy to deal with freeing resources > > for a class or using structs on the stack for resources that need to > > be freed. Or alternate memory strategies can be used via > > std.experimental.allocator. The GC works great for lots of stuff but > > not for system resources. Honestly, in some ways, we'd be better off > > if D didn't even have finalizers. > > [...] > > This makes me wonder if a better approach to memory management is to use > refcounting by default, and fallback to the GC to collect cycles. > > In my current project, I'll probably end up having to use > RefCounted!Database just so I can have deterministic releasing of > database handles without needing to worry about dangling references that > may still be lingering around. (Currently, I'm just calling .destroy > directly on the handle when I'm done with it, but there's a slim > possibility that there might be dangling references left somewhere. So > that needs to be addressed at some point.) The GC works perfectly well for most things. You just have to be aware of its downsides - and that includes being aware of what's going to happen if you try and use it to manage system resources. For that, ref-counting is almost certainly going to be better (though there are probably cases where it doesn't matter, because it's not really a problem if the resources aren't freed before the program exits). In any case, for a lot of the cases where dispose/IDisposable would be used in C#, ref-counting is almost certainly what the code should be doing (either that or scope statements if what's being done is localized and not common enough to create a type to use with RAII). That's probably on the list of things that needs to be written up somewhere as being among best practices for D. There are probably too many things like that that are sitting in the heads of a lot of folks but not necessarily well communicated to others - though this particular issue may have been discussed in the recent GC articles. I don't know. I still need to read them. - Jonathan M Davis
Re: Don't expect class destructors to be called at all by the GC
On Sun, Dec 24, 2017 at 02:07:26PM -0700, Jonathan M Davis via Digitalmars-d-learn wrote: [...] > Regardless, even if it were the case that it were guaranteed that all > finalizers were run when the program exited, it would still be > terrible practice to rely on it. It's trivial to end up in a situation > where no collection is run for quite some time (e.g. just don't do > much memory allocation for a while), which would leave any resources > that needed to be freed by a finalizer unfreed for quite a while even > though they weren't needed anymore. So practically speaking, it > doesn't really matter where the finalizers are guaranteed to run. > Relying on them to be run rather than forcing them to be run via > destroy or using some other helper function is just going to cause > problems, so it's just plain bad practice to rely on finalizers to be > run to release resources. That's just life with GC's in general, not > just D's. It's why C# has the while dispose/IDisposable thing, and why > D code should either be using destroy to deal with freeing resources > for a class or using structs on the stack for resources that need to > be freed. Or alternate memory strategies can be used via > std.experimental.allocator. The GC works great for lots of stuff but > not for system resources. Honestly, in some ways, we'd be better off > if D didn't even have finalizers. [...] This makes me wonder if a better approach to memory management is to use refcounting by default, and fallback to the GC to collect cycles. In my current project, I'll probably end up having to use RefCounted!Database just so I can have deterministic releasing of database handles without needing to worry about dangling references that may still be lingering around. (Currently, I'm just calling .destroy directly on the handle when I'm done with it, but there's a slim possibility that there might be dangling references left somewhere. So that needs to be addressed at some point.) T -- "I suspect the best way to deal with procrastination is to put off the procrastination itself until later. I've been meaning to try this, but haven't gotten around to it yet. " -- swr
Re: Don't expect class destructors to be called at all by the GC
On Friday, December 22, 2017 01:23:26 Mike Parker via Digitalmars-d-learn wrote: > The class destructor is not run during the lifetime of the > program. The fact that it's run during runtime termination is an > implementation detail. Another implementation might not run a > finalization at termination. > > So the destructors (finalizers) are only run when an object is > collected. No collection, no destructor call. IIRC, there were reasons why running the destructors/finalizers when unloading dynamically loaded libraries was a problem too and one that we were stuck with, but I could be remembering incorrectly. Regardless, even if it were the case that it were guaranteed that all finalizers were run when the program exited, it would still be terrible practice to rely on it. It's trivial to end up in a situation where no collection is run for quite some time (e.g. just don't do much memory allocation for a while), which would leave any resources that needed to be freed by a finalizer unfreed for quite a while even though they weren't needed anymore. So practically speaking, it doesn't really matter where the finalizers are guaranteed to run. Relying on them to be run rather than forcing them to be run via destroy or using some other helper function is just going to cause problems, so it's just plain bad practice to rely on finalizers to be run to release resources. That's just life with GC's in general, not just D's. It's why C# has the while dispose/IDisposable thing, and why D code should either be using destroy to deal with freeing resources for a class or using structs on the stack for resources that need to be freed. Or alternate memory strategies can be used via std.experimental.allocator. The GC works great for lots of stuff but not for system resources. Honestly, in some ways, we'd be better off if D didn't even have finalizers. - Jonathan M Davis
Re: Don't expect class destructors to be called at all by the GC
On Friday, 22 December 2017 at 00:09:31 UTC, Mike Franklin wrote: What condition(s) would cause a destructor for an object that is managed by the GC to potentially not be called? Good question. It's true that barring an Error, they should be called by the GC at runtime termination.
Re: Don't expect class destructors to be called at all by the GC
On Friday, 22 December 2017 at 23:34:55 UTC, Mengu wrote: i really wonder how Objective-C and Swift is pulling this off. It isn't a fundamental problem, D just can't express it in the existing language (heck, even D, as defined, could do it, the implementation just isn't there.)
Re: Don't expect class destructors to be called at all by the GC
On Thursday, 21 December 2017 at 18:45:27 UTC, Adam D. Ruppe wrote: On Thursday, 21 December 2017 at 18:20:19 UTC, H. S. Teoh wrote: When the scoped destruction of structs isn't an option, RefCounted!T seems to be a less evil alternative than an unreliable class dtor. :-/ Alas, RefCounted doesn't work well with inheritance... Though, what you could do is make the refcounted owners and borrow the actual reference later. i really wonder how Objective-C and Swift is pulling this off.
Re: Don't expect class destructors to be called at all by the GC
On Thursday, 21 December 2017 at 18:20:19 UTC, H. S. Teoh wrote: Even calling GC.collect directly did not guarantee the DB handle was closed at the right time. This may have been a bug in my code that left dangling references to it, or perhaps the array of Database handles was still scanned through by the GC even though the only remaining array slice has a shorter length. Whatever the reason was, it left me with the very unpleasant prospect of silently accumulating file descriptor leaks. Last I checked, the GC doesn't understand arrays. It only understands "segment of memory that might contain pointers" and "segment of memory that doesn't contain pointers". You might have gotten better results if you had nulled out the reference in the array. Of course, that relies on not having any remaining references on the stack or in registers, neither of which is easy to guarantee.
Re: Don't expect class destructors to be called at all by the GC
On Thursday, 21 December 2017 at 19:43:16 UTC, Steven Schveighoffer wrote: On 12/20/17 9:57 PM, Mike Franklin wrote: [...] It's implementation defined :) The gist is, you cannot expect that destructors will be run in a timely manner, or at all. They may be called, and most of the time they are. But the language nor the current implementation makes a guarantee that they will be called. For this reason, any classes that use non-memory resources should clean up those resources before becoming garbage. This is why most of the time, such items are managed by structs. Note that the same non-guarantee exists in other GC'd languages, such as Java or C#. -Steve Except for that in C# you have the IDisposable interface, which can actually be used to prevent this kind of stuff and generally used to clean up non-GC memory.
Re: Don't expect class destructors to be called at all by the GC
On Friday, 22 December 2017 at 00:09:31 UTC, Mike Franklin wrote: What condition(s) would cause a destructor for an object that is managed by the GC to potentially not be called? Here: === import std.stdio; class Clazz { ~this() { writeln("Class dest"); } } void makeClazz() { auto clazz = new Clazz; } void main() { makeClazz(); } static ~this() { writeln("Static dest"); } This outputs: Static dest Class dest The class destructor is not run during the lifetime of the program. The fact that it's run during runtime termination is an implementation detail. Another implementation might not run a finalization at termination. So the destructors (finalizers) are only run when an object is collected. No collection, no destructor call.
Re: Don't expect class destructors to be called at all by the GC
On Thursday, 21 December 2017 at 19:43:16 UTC, Steven Schveighoffer wrote: The gist is, you cannot expect that destructors will be run in a timely manner, or at all. They may be called, and most of the time they are. But the language nor the current implementation makes a guarantee that they will be called. I understand that we can't deterministically predict when a destructor will be called, but if we can't deterministically predict if a destructor will be called, that seems asinine. What condition(s) would cause a destructor for an object that is managed by the GC to potentially not be called? Thanks, Mike
Re: Don't expect class destructors to be called at all by the GC
On Thursday, 21 December 2017 at 18:48:38 UTC, H. S. Teoh wrote: Alas, RefCounted doesn't work well with inheritance... Oh? What's the issue? Implicit casts don't work so you can't pass a RefCounted!Class as RefCounted!Interface except in simple cases using alias this tricks.
Re: Don't expect class destructors to be called at all by the GC
On 12/20/17 9:57 PM, Mike Franklin wrote: "Don't expect class destructors to be called at all by the GC" I was a bit shocked to read that here: https://p0nce.github.io/d-idioms/#The-trouble-with-class-destructors The document tries to clarify with: "The garbage collector is not guaranteed to run the destructors for all unreferenced objects." Unfortunately, that doesn't really shed much light on this oddity. So, specifically, under what circumstances are destructors not called? It's implementation defined :) The gist is, you cannot expect that destructors will be run in a timely manner, or at all. They may be called, and most of the time they are. But the language nor the current implementation makes a guarantee that they will be called. For this reason, any classes that use non-memory resources should clean up those resources before becoming garbage. This is why most of the time, such items are managed by structs. Note that the same non-guarantee exists in other GC'd languages, such as Java or C#. -Steve
Re: Don't expect class destructors to be called at all by the GC
On Thu, Dec 21, 2017 at 06:45:27PM +, Adam D. Ruppe via Digitalmars-d-learn wrote: > On Thursday, 21 December 2017 at 18:20:19 UTC, H. S. Teoh wrote: > > When the scoped destruction of structs isn't an option, RefCounted!T > > seems to be a less evil alternative than an unreliable class dtor. > > :-/ > > Alas, RefCounted doesn't work well with inheritance... Oh? What's the issue? > Though, what you could do is make the refcounted owners and borrow the > actual reference later. Yeah, I figured that I pretty much have to use a proxy struct with RefCounted, and have the struct dtor do the actual cleanup of the class reference, something like: class Resource { void cleanup(); // inheritable } struct Proxy { private Resource res; this(Resource _res) { res = _res; } ~this() { res.cleanup(); } } ... auto obj = RefCounted!Proxy(allocateResource()); T -- The problem with the world is that everybody else is stupid.
Re: Don't expect class destructors to be called at all by the GC
On Thursday, 21 December 2017 at 18:20:19 UTC, H. S. Teoh wrote: When the scoped destruction of structs isn't an option, RefCounted!T seems to be a less evil alternative than an unreliable class dtor. :-/ Alas, RefCounted doesn't work well with inheritance... Though, what you could do is make the refcounted owners and borrow the actual reference later.
Re: Don't expect class destructors to be called at all by the GC
On Thu, Dec 21, 2017 at 06:50:44AM +, Mike Parker via Digitalmars-d-learn wrote: [...] > I just don't even bother with class destructors. Without a guarantee > that they can run and without any sort of deterministic behavior, it's > really not appropriate to refer to them as destructors and they're > about as useful as Java finalizers, which means not at all. In order > to make them less error prone, we need to separate the concept of > destruction from finalization and allow both destructors and > finalizers. That's what I've taken to doing manually, by implementing > a `terminate` function in my classes that I either call directly or > via a ref-counted templated struct called Terminator. I recently ran into this problem while using Adam Ruppe's lightweight SQLite binding (arsd/sqlite.d). Originally, I kept an open database handle (which is a class instance) throughout the lifetime of the program; in this case, I could just use a scoped reference and it would ensure the DB is closed when the handle went out of scope, just what I want. But as my code developed, I began to need to cache multiple DB handles for performance, and scope no longer helps me there. At first I thought, no problem, the GC would handle this for me. Right? Wrong. Even calling GC.collect directly did not guarantee the DB handle was closed at the right time. This may have been a bug in my code that left dangling references to it, or perhaps the array of Database handles was still scanned through by the GC even though the only remaining array slice has a shorter length. Whatever the reason was, it left me with the very unpleasant prospect of silently accumulating file descriptor leaks. I ended up calling .destroy on the class instance explicitly just so the destructor would run at the right time, right before nulling the reference so that the GC would collect the memory. This makes using classes in D an even dimmer prospect than it already generally is (nowadays, and I don't seem to be the only one, I prefer to just use structs and templates instead of runtime polymorphism, where possible). When the scoped destruction of structs isn't an option, RefCounted!T seems to be a less evil alternative than an unreliable class dtor. :-/ T -- MACINTOSH: Most Applications Crash, If Not, The Operating System Hangs
Re: Don't expect class destructors to be called at all by the GC
On Thursday, 21 December 2017 at 06:50:44 UTC, Mike Parker wrote: On Thursday, 21 December 2017 at 04:10:56 UTC, user1234 wrote: [...] [...] [...] The root of the problem is that in D, class destruction and finalization are conflated. It would be much more accurate to refer to ~this in classes as a finalizer. Then this sort of confusion wouldn't be so widespread. [...] Have you considered writing a DIP on this?
Re: Don't expect class destructors to be called at all by the GC
On Thursday, 21 December 2017 at 14:26:55 UTC, Guillaume Piolat wrote: On Thursday, 21 December 2017 at 06:50:44 UTC, Mike Parker wrote: That's what I've taken to doing manually, by implementing a `terminate` function in my classes that I either call directly or via a ref-counted templated struct called Terminator. I feel like I'm rambling but.. The problem with that approach is that you can't reuse Unique, RefCounted, scoped!T because they rely on .destroy I'm not proposing it as a general solution. It's easy to implement and it works for my use case, so it's one possible solution.
Re: Don't expect class destructors to be called at all by the GC
On Thursday, 21 December 2017 at 06:50:44 UTC, Mike Parker wrote: That's what I've taken to doing manually, by implementing a `terminate` function in my classes that I either call directly or via a ref-counted templated struct called Terminator. I feel like I'm rambling but.. The problem with that approach is that you can't reuse Unique, RefCounted, scoped!T because they rely on .destroy
Re: Don't expect class destructors to be called at all by the GC
On Thursday, 21 December 2017 at 04:10:56 UTC, user1234 wrote: On Thursday, 21 December 2017 at 02:57:00 UTC, Mike Franklin Unfortunately, that doesn't really shed much light on this oddity. So, specifically, under what circumstances are destructors not called? When the GC is unaware of a class instance (an "unreferenced object") it won't call its destructor. This happens when you use alternative memory management strategy, for example using Mallocator and make you get an unreferenced object that you have to manage yourself. The root of the problem is that in D, class destruction and finalization are conflated. It would be much more accurate to refer to ~this in classes as a finalizer. Then this sort of confusion wouldn't be so widespread. Also, consider the current GC implementation finalizes any objects that haven't yet been finalized when it terminates. It terminates during runtime termination, but *after* static destructors are executed (which is how it should be). We already know that you can't rely on any GC memory references to be valid in a class destructor, but because of this cleanup phase, you also can't rely on any program state still being valid. As an example, each of the Derelict packages used to (pointlessly) unload its shared library in a static destructor, but people repeatedly had segfaults at app exit because their class destructors were calling into the loaded libraries to release resources. The solution there was easy -- stop manually unloading the libraries and let the OS do it at process termination. But anyone who wants to unload any resources in a class destructor needs to be aware of this issue in case a static destructor somewhere is getting to it first. And I still see code using Derelict where people unload the library themselves in a static destructor. I just don't even bother with class destructors. Without a guarantee that they can run and without any sort of deterministic behavior, it's really not appropriate to refer to them as destructors and they're about as useful as Java finalizers, which means not at all. In order to make them less error prone, we need to separate the concept of destruction from finalization and allow both destructors and finalizers. That's what I've taken to doing manually, by implementing a `terminate` function in my classes that I either call directly or via a ref-counted templated struct called Terminator.
Re: Don't expect class destructors to be called at all by the GC
On Thursday, 21 December 2017 at 02:57:00 UTC, Mike Franklin wrote: "Don't expect class destructors to be called at all by the GC" I was a bit shocked to read that here: https://p0nce.github.io/d-idioms/#The-trouble-with-class-destructors The document tries to clarify with: "The garbage collector is not guaranteed to run the destructors for all unreferenced objects." Unfortunately, that doesn't really shed much light on this oddity. So, specifically, under what circumstances are destructors not called? Thanks, Mike When the GC is unaware of a class instance (an "unreferenced object") it won't call its destructor. This happens when you use alternative memory management strategy, for example using Mallocator and make you get an unreferenced object that you have to manage yourself.
Don't expect class destructors to be called at all by the GC
"Don't expect class destructors to be called at all by the GC" I was a bit shocked to read that here: https://p0nce.github.io/d-idioms/#The-trouble-with-class-destructors The document tries to clarify with: "The garbage collector is not guaranteed to run the destructors for all unreferenced objects." Unfortunately, that doesn't really shed much light on this oddity. So, specifically, under what circumstances are destructors not called? Thanks, Mike