Re: Garbage Collectors
On 7/19/23 10:46 AM, Johan wrote: On Wednesday, 19 July 2023 at 11:27:14 UTC, Richard (Rikki) Andrew Cattermole wrote: [...] you would have to do a new build of druntime/phobos special which isn't the easiest thing to do. Side remark: LDC ships with the ldc-build-runtime tool which should help the user a lot in building druntime/phobos. (downloads the correct source belonging to the specific compiler version, and then builds the libraries with user-specified flags, compilers, etc.) This is very cool to know, I wasn't aware! -Steve
Re: Garbage Collectors
On Wednesday, 19 July 2023 at 11:27:14 UTC, Richard (Rikki) Andrew Cattermole wrote: [...] you would have to do a new build of druntime/phobos special which isn't the easiest thing to do. Side remark: LDC ships with the ldc-build-runtime tool which should help the user a lot in building druntime/phobos. (downloads the correct source belonging to the specific compiler version, and then builds the libraries with user-specified flags, compilers, etc.) -Johan
Re: Garbage Collectors
On 7/19/23 3:24 AM, IchorDev wrote: So, D’s default garbage collector is the one named “conservative” in DRuntime… I see there’s also “manual” which doesn’t actually function as a GC, which is interesting. Nothing says what ProtoGC is… so I guess it’s useless. Has anyone ever published any custom GCs? A search through the dub package registry yielded nothing. How hard would it be to port (for instance) a Java GC to D’s interface? To answer the question about ProtoGC, it is a stub GC which will create the appropriate GC (based on runtime flags) when the first function that requires actually using the GC is called. It is actually the default GC. This was introduced to implement more pay-as-you-go runtime features. If you don't use the GC, no GC is created. If I recall correctly, you must attempt to allocate using the GC for the ProtoGC to do its thing. Everything else is just simple stubs (e.g. ProtoGC.free does nothing, since it can never allocate memory) -Steve
Re: Garbage Collectors
On Wednesday, 19 July 2023 at 11:27:14 UTC, Richard (Rikki) Andrew Cattermole wrote: druntime supports registering of GC's not compiled with druntime. But because some of the machinery isn't available to you, you would have to recreate it. Oh right, so it's more of a matter of making GC-related code part of the GC implementation so that it's more convenient for others to build their own GC on top of it. So in practice you cannot find a GC on the dub-registry to try it out. Even if one were to exist, you would have to do a new build of druntime/phobos special which isn't the easiest thing to do. Just to confirm: would none of this would require *me* recompiling DRuntime at all, or is it that DRuntime would have to be modified to the point where anyone else registering a unique GC from user code wouldn't have to recompile DRuntime?
Re: Garbage Collectors
On 19/07/2023 11:13 PM, IchorDev wrote: On Wednesday, 19 July 2023 at 10:50:07 UTC, Richard (Rikki) Andrew Cattermole wrote: Copying out the conservative GC, register it under a different name and getting that to compile and link without recompiling druntime would be a good place to begin without having to understand how GC's work. That sounds pretty easy, but I don't really see how would that be particularly helpful to allowing a different style of GC to be registered? druntime supports registering of GC's not compiled with druntime. But because some of the machinery isn't available to you, you would have to recreate it. So in practice you cannot find a GC on the dub-registry to try it out. Even if one were to exist, you would have to do a new build of druntime/phobos special which isn't the easiest thing to do. https://github.com/dlang/dmd/blob/master/druntime/src/core/gc/registry.d#L39 Regardless, its a barrier to anyone wanting to try writing a GC to experiment with (or port one).
Re: Garbage Collectors
On Wednesday, 19 July 2023 at 10:50:07 UTC, Richard (Rikki) Andrew Cattermole wrote: Copying out the conservative GC, register it under a different name and getting that to compile and link without recompiling druntime would be a good place to begin without having to understand how GC's work. That sounds pretty easy, but I don't really see how would that be particularly helpful to allowing a different style of GC to be registered?
Re: Garbage Collectors
On 19/07/2023 9:02 PM, IchorDev wrote: On Wednesday, 19 July 2023 at 08:27:18 UTC, Richard (Rikki) Andrew Cattermole wrote: Its not as simple as porting to the API unfortunately. We don't have barriers of any kind, so that removes most GC designs you would want to use today. We are very close to maxing out what we can do as a result. A whole pile of logic is hidden in rt, so you have no choice but to either do the work to fix that, or recompile druntime with your GC. I'm mostly a frontend person, but is what would need doing there? Is there any existing discussion about that? Not really, last time I tried, I didn't document what was hidden. Copying out the conservative GC, register it under a different name and getting that to compile and link without recompiling druntime would be a good place to begin without having to understand how GC's work.
Re: Garbage Collectors
On Wednesday, 19 July 2023 at 08:27:18 UTC, Richard (Rikki) Andrew Cattermole wrote: Its not as simple as porting to the API unfortunately. We don't have barriers of any kind, so that removes most GC designs you would want to use today. We are very close to maxing out what we can do as a result. A whole pile of logic is hidden in rt, so you have no choice but to either do the work to fix that, or recompile druntime with your GC. I'm mostly a frontend person, but is what would need doing there? Is there any existing discussion about that?
Re: Garbage Collectors
Its not as simple as porting to the API unfortunately. We don't have barriers of any kind, so that removes most GC designs you would want to use today. We are very close to maxing out what we can do as a result. A whole pile of logic is hidden in rt, so you have no choice but to either do the work to fix that, or recompile druntime with your GC.
Re: Garbage Collectors
On 19/07/2023 7:44 PM, Sergey wrote: Forking GC was introduced some time ago. But I don't think it is quite different from regular (https://forum.dlang.org/post/tf8mbo$1jvp$1...@digitalmars.com) It is a modification of the conservative GC. Rather than a unique GC implementation and is upstreamed. There are some posts about it GCs, but I don't think anything was released. https://forum.dlang.org/thread/jquklsqxtfgvsezrc...@forum.dlang.org?page=1 https://forum.dlang.org/post/yonbmhifafbyjhwpc...@forum.dlang.org The only GC design I know of that is semi-interesting is Rainer's concurrent GC for Windows. http://rainers.github.io/visuald/druntime/concurrentgc.html This has not been upstreamed.
Re: Garbage Collectors
On Wednesday, 19 July 2023 at 07:24:06 UTC, IchorDev wrote: So, D’s default garbage collector is the one named “conservative” in DRuntime… I see there’s also “manual” which doesn’t actually function as a GC, which is interesting. Nothing says what ProtoGC is… so I guess it’s useless. Has anyone ever published any custom GCs? A search through the dub package registry yielded nothing. How hard would it be to port (for instance) a Java GC to D’s interface? Forking GC was introduced some time ago. But I don't think it is quite different from regular (https://forum.dlang.org/post/tf8mbo$1jvp$1...@digitalmars.com) There are some posts about it GCs, but I don't think anything was released. https://forum.dlang.org/thread/jquklsqxtfgvsezrc...@forum.dlang.org?page=1 https://forum.dlang.org/post/yonbmhifafbyjhwpc...@forum.dlang.org
Re: Memory management and garbage collectors
Thank you for your reply. It was helpful.
Re: Memory management and garbage collectors
JMRyan: > In theory, garbage collectors make memory leaks a thing of the past. Even with a perfect GC you may leave around references that keep alive some data that you will never need to use. This is a kind of memory leak. And the current D GC is not fully precise, this means that sometimes it sees as possible pointers even stuff that is not a pointer, so noise may keep alive memory blocks that are not really referenced. > So take a for- > instance. Consider a circular linked list that we are done with. Each > node has a reference to it (from another node in the list). But, being > done with the list, we imagine that there are no references from outside > the list to any node. Is D's garbage collector smart enough to deal > recognize the circle of nodes as garbage? It's not a matter of being smart, it's a matter of how its algorithm works. In a GC based on reference counts, you keep an object alive if its count is nonzero, this means if one or more references point to it. In a GC like the D one (well, in a basic implementation of its idea), you have some starting pointers, the roots, and starting form them you explore the graph of the references, and along the way you mark the visited objects. At the end you scan the list of all the objects and delete the ones with no mark. So the presence of a cycle is not a problem. A cycle is recycled if no external pointers point to it. CPython GC is a reference counter, but they have added extra logic to recognize and recycle cycles too. I leave other of your questions to other people. Bye, bearophile
Memory management and garbage collectors
In theory, garbage collectors make memory leaks a thing of the past. In practice, garbage collectors don't always work according to theory. This makes me curious: how does one test for memory leaks in a D program? I also don't know how smart or dumb garbage collectors are. How much help does it need? Okay, that is too general a question. So take a for- instance. Consider a circular linked list that we are done with. Each node has a reference to it (from another node in the list). But, being done with the list, we imagine that there are no references from outside the list to any node. Is D's garbage collector smart enough to deal recognize the circle of nodes as garbage? Is it necessary or at least helpful to set all the circle's links to null before being done with it? I assume that reference counting collectors are not smart enough to handle an abandoned circular list. But D's GC is not a reference counter. I don't know how smart GC's get. Is it okay to leave a single node linking to itself? Another for-instance. Consider this time a linear linked list. This time, the list is finite: it terminates with a node pointing to null. Are the issues here any different? Thanks in advance for any insights.
