Re: GC memory fragmentation

2021-04-17 Thread kdevel via Digitalmars-d-learn
On Sunday, 11 April 2021 at 09:10:22 UTC, tchaloupka wrote: we're using vibe-d (on Linux) for a long running REST API server [...] [...] lot of small allocations (postgresql query, result processing and REST API json serialization). I am wondering about your overall system design. Are there

Re: GC memory fragmentation

2021-04-16 Thread sarn via Digitalmars-d-learn
On Tuesday, 13 April 2021 at 12:30:13 UTC, tchaloupka wrote: Some kind of GC memory dump and analyzer tool as mentioned `Diamond` would be of tremendous help to diagnose this.. I've used bpftrace to do some of that stuff: https://theartofmachinery.com/2019/04/26/bpftrace_d_gc.html

Re: GC memory fragmentation

2021-04-14 Thread Imperatorn via Digitalmars-d-learn
Framework is also suffering from this. We are trying to make a simple example to illustrate it. I think this is a rather serious problem for a language saying "GC is fine" . Is there an issue for this?

Re: GC memory fragmentation

2021-04-14 Thread Heromyth via Digitalmars-d-learn
On Sunday, 11 April 2021 at 09:10:22 UTC, tchaloupka wrote: Hi, we're using vibe-d (on Linux) for a long running REST API server and have problem with constantly growing memory until system kills it with OOM killer. The Hunt Framework is also suffering from this. We are trying to make a

Re: GC memory fragmentation

2021-04-14 Thread tchaloupka via Digitalmars-d-learn
On Tuesday, 13 April 2021 at 12:30:13 UTC, tchaloupka wrote: I'm not so sure if pages of small objects (or large) that are not completely empty can be reused as a new bucket or only free pages can be reused. Does anyone has some insight of this? Some kind of GC memory dump and analyzer tool

Re: GC memory fragmentation

2021-04-13 Thread Tobias Pankrath via Digitalmars-d-learn
On Tuesday, 13 April 2021 at 12:30:13 UTC, tchaloupka wrote: Some kind of GC memory dump and analyzer tool as mentioned `Diamond` would be of tremendous help to diagnose this.. You could try to get the stack traces of the allocating calls via eBPF. Maybe that leads to some new insights.

Re: GC memory fragmentation

2021-04-13 Thread tchaloupka via Digitalmars-d-learn
good failover). I have reduced the problem by refactoring some of our gc usage, but the problem still persists. On side-note, it would also be good if the GC can be aware of the max memory it is allotted so that it knows it needs to do more aggressive collections when nearing it. I knew

Re: GC memory fragmentation

2021-04-12 Thread Per Nordlöw via Digitalmars-d-learn
On Monday, 12 April 2021 at 20:50:49 UTC, Per Nordlöw wrote: more aggressive collections when nearing it. What is a more aggressive collection compared to a normal collection? Unfortunately we still have no move support in D's GC because of its impreciseness.

Re: GC memory fragmentation

2021-04-12 Thread Per Nordlöw via Digitalmars-d-learn
. I wonder how other GC-backed languages handle this. [1] https://dlang.org/phobos/core_memory.html#.GC.Stats.usedSize

Re: GC memory fragmentation

2021-04-12 Thread Per Nordlöw via Digitalmars-d-learn
On Monday, 12 April 2021 at 07:03:02 UTC, Sebastiaan Koppe wrote: On side-note, it would also be good if the GC can be aware of the max memory it is allotted so that it knows it needs to do more aggressive collections when nearing it. I'm surprised there is no such functionality available

Re: GC memory fragmentation

2021-04-12 Thread Imperatorn via Digitalmars-d-learn
On Sunday, 11 April 2021 at 09:10:22 UTC, tchaloupka wrote: Hi, we're using vibe-d (on Linux) for a long running REST API server and have problem with constantly growing memory until system kills it with OOM killer. [...] Looks like the GC needs some love

Re: GC memory fragmentation

2021-04-12 Thread Sebastiaan Koppe via Digitalmars-d-learn
). I have reduced the problem by refactoring some of our gc usage, but the problem still persists. On side-note, it would also be good if the GC can be aware of the max memory it is allotted so that it knows it needs to do more aggressive collections when nearing it.

Re: GC memory fragmentation

2021-04-11 Thread frame via Digitalmars-d-learn
holds the 'live' data. But that should be freed too at some point and GC should minimize (if another request doesn'cause allocation in the 'wrong' page again). If so, setting minPoolSize:1 could help you to control the memory usage, and if this memory is kept, you could inspect whats inside

Re: GC memory fragmentation

2021-04-11 Thread ryuukk_ via Digitalmars-d-learn
I should have added https://dub.pm/package-format-json#build-types profileGC as build option in your dub file That will generate a profile_gc file once the program exit, a table that lists all the allocations

Re: GC memory fragmentation

2021-04-11 Thread ryuukk_ via Digitalmars-d-learn
likely for a random int to appear to be a pointer to the interior of a block of GC-allocated memory. Nope it's 64bit build. I've also tried to switch to precise GC with same result. Tom Try to tweak the GC, first could try with the precise one? Also try to enable GC profiling with profile:1

Re: GC memory fragmentation

2021-04-11 Thread tchaloupka via Digitalmars-d-learn
to the interior of a block of GC-allocated memory. Nope it's 64bit build. I've also tried to switch to precise GC with same result. Tom

Re: GC memory fragmentation

2021-04-11 Thread Nathan S. via Digitalmars-d-learn
? The garbage collector is much more likely to leak memory with a 32-bit address space since it much more likely for a random int to appear to be a pointer to the interior of a block of GC-allocated memory.

GC memory fragmentation

2021-04-11 Thread tchaloupka via Digitalmars-d-learn
and now memory still goes up (not that dramatically but still). Stats of the GC when it growed up between 30s are these (logged after `GC.collect()` and `GC.minimize()`: ``` GC.stats: usedSize=9994864, freeSize=92765584, total=102760448 GC.stats: usedSize=11571456, freeSize=251621120, total

Re: Minimize GC memory footprint

2021-02-13 Thread Siemargl via Digitalmars-d-learn
that edgy example to find the problem with the GC. As someone mentioned before, in a real application you would choose some pre-allocation like reserve() on Appender instead which performs better. LDC 1.24 is unaffected of this bug and x64 target consume less memory.

Re: Minimize GC memory footprint

2021-02-13 Thread frame via Digitalmars-d-learn
On Saturday, 13 February 2021 at 17:54:53 UTC, Siemargl wrote: And it works too, for 32-bit also =) Consuming about 100MB RAM. Yes, Appender is nice but I had no control about .data since the real property is private so I chose that edgy example to find the problem with the GC. As someone

Re: Minimize GC memory footprint

2021-02-13 Thread Siemargl via Digitalmars-d-learn
MB. Best is 100MB. But it doesn't leak endlessly like the 32bit variant. Hmm, I try to rewrite example in C# and it just hangs in GC, when str += "1" added 5 million times. Then i fix this using StringBuilder, as documented. It works fine. Next i searched flang forums for D's Str

Re: Minimize GC memory footprint

2021-02-08 Thread frame via Digitalmars-d-learn
// GC.free(s.ptr); GC.free(GC.addrOf(s.ptr)); } I think the automatic GC is also affected by this issue.

Re: Minimize GC memory footprint

2021-02-06 Thread frame via Digitalmars-d-learn
On Saturday, 6 February 2021 at 19:13:33 UTC, Mike Parker wrote: On Saturday, 6 February 2021 at 17:50:18 UTC, frame wrote:> But .length = 0 should. What do you expect it to do in this case? Don't know - some compiler optimization? :D On Saturday, 6 February 2021 at 19:31:39 UTC,

Re: Minimize GC memory footprint

2021-02-06 Thread Siemargl via Digitalmars-d-learn
On Saturday, 6 February 2021 at 19:10:14 UTC, Mike Parker wrote: On Saturday, 6 February 2021 at 17:50:18 UTC, frame wrote: Sorry, i forgot mem leak. Or maybe i incorrect understand Gc counters So log Usage: 698.46 MiB (free 187.42 MiB) / collected: 14

Re: Minimize GC memory footprint

2021-02-06 Thread Siemargl via Digitalmars-d-learn
On Saturday, 6 February 2021 at 19:10:14 UTC, Mike Parker wrote: On Saturday, 6 February 2021 at 17:50:18 UTC, frame wrote: On Saturday, 6 February 2021 at 15:45:47 UTC, rikki cattermole wrote: Default settings should work out of the box. If not - it's bad for reputation of the language.

Re: Minimize GC memory footprint

2021-02-06 Thread Mike Parker via Digitalmars-d-learn
On Saturday, 6 February 2021 at 17:50:18 UTC, frame wrote:> But .length = 0 should. What do you expect it to do in this case?

Re: Minimize GC memory footprint

2021-02-06 Thread Mike Parker via Digitalmars-d-learn
On Saturday, 6 February 2021 at 17:50:18 UTC, frame wrote: On Saturday, 6 February 2021 at 15:45:47 UTC, rikki cattermole wrote: Default settings should work out of the box. If not - it's bad for reputation of the language. Given that 32-bit has been the default on Windows for D's entire

Re: Minimize GC memory footprint

2021-02-06 Thread frame via Digitalmars-d-learn
On Saturday, 6 February 2021 at 15:45:47 UTC, rikki cattermole wrote: The GC shouldn't be aware of the scope guard. It expands out into a try finally block. But .length = 0 should. Nah, this is old. It is also bad D code. Allocate up front and then set. I agree but it has to work anyway

Re: Minimize GC memory footprint

2021-02-06 Thread rikki cattermole via Digitalmars-d-learn
On 07/02/2021 4:22 AM, frame wrote: On Saturday, 6 February 2021 at 13:30:03 UTC, rikki cattermole wrote: Okay, its still seeing something is alive then. That's why I used the scope guard. I know it shouldn't have any effect but I want to give the GC an extra hint ;) The GC shouldn't

Re: Minimize GC memory footprint

2021-02-06 Thread frame via Digitalmars-d-learn
On Saturday, 6 February 2021 at 13:30:03 UTC, rikki cattermole wrote: Okay, its still seeing something is alive then. That's why I used the scope guard. I know it shouldn't have any effect but I want to give the GC an extra hint ;) I've compiled and ran it under ldc. Dmd in 32bit mode

Re: Minimize GC memory footprint

2021-02-06 Thread rikki cattermole via Digitalmars-d-learn
stuff can get caught in the buffer before erroring out. [...] Turn on the precise GC, 32bit is a bit too small of a range and you can get false positives like in this case (at least looks like it). For reference, how does one turn on precise GC? https://dlang.org/spec/garbage.html

Re: Minimize GC memory footprint

2021-02-06 Thread Siemargl via Digitalmars-d-learn
before erroring out. [...] Turn on the precise GC, 32bit is a bit too small of a range and you can get false positives like in this case (at least looks like it). For reference, how does one turn on precise GC? https://dlang.org/spec/garbage.html#gc_config Strange things happens: - precise

Re: Minimize GC memory footprint

2021-02-06 Thread Imperatorn via Digitalmars-d-learn
On Saturday, 6 February 2021 at 09:42:38 UTC, rikki cattermole wrote: On 06/02/2021 3:32 PM, frame wrote: [...] This won't do anything. [...] Don't forget to stdout.flush; Otherwise stuff can get caught in the buffer before erroring out. [...] Turn on the precise GC, 32bit

Re: Minimize GC memory footprint

2021-02-06 Thread rikki cattermole via Digitalmars-d-learn
On 06/02/2021 3:32 PM, frame wrote: On Friday, 5 February 2021 at 22:46:05 UTC, Bastiaan Veelo wrote: ?? Do you mean no collections happen? 32bit GC should just work. No, it doesn't - this code fails on memory allocation and works fine with -m64 switch: import std.stdio; import

Re: Minimize GC memory footprint

2021-02-05 Thread frame via Digitalmars-d-learn
On Friday, 5 February 2021 at 22:46:05 UTC, Bastiaan Veelo wrote: ?? Do you mean no collections happen? 32bit GC should just work. No, it doesn't - this code fails on memory allocation and works fine with -m64 switch: import std.stdio; import core.memory : GC; void main() { void

Re: Minimize GC memory footprint

2021-02-05 Thread frame via Digitalmars-d-learn
On Friday, 5 February 2021 at 22:46:05 UTC, Bastiaan Veelo wrote: I think it means that you need to make sure that enable() is called as many times as disable() is called before collection can happen automatically. — Bastiaan. Thanks, in the meanwhile I looked into the source: struct Gcx

Re: Minimize GC memory footprint

2021-02-05 Thread Bastiaan Veelo via Digitalmars-d-learn
On Wednesday, 3 February 2021 at 13:37:42 UTC, frame wrote: I have to deal with GC as long I want to use other libraries that are relying on it or even just phobos. Conclusion so far (for Windows): 32bit: - GC just doesn't work at all ?? Do you mean no collections happen? 32bit GC should

Re: Minimize GC memory footprint

2021-02-03 Thread frame via Digitalmars-d-learn
On Sunday, 31 January 2021 at 12:14:53 UTC, Imperatorn wrote: It says experimental, but it's fine: https://dlang.org/phobos/std_experimental_allocator.html Well, this looks very nice but I have to deal with GC as long I want to use other libraries that are relying on it or even just phobos

Re: Minimize GC memory footprint

2021-01-31 Thread Imperatorn via Digitalmars-d-learn
On Sunday, 31 January 2021 at 04:12:14 UTC, frame wrote: On Saturday, 30 January 2021 at 22:57:41 UTC, Imperatorn wrote: On Saturday, 30 January 2021 at 16:42:35 UTC, frame wrote: Is there a way to force the GC to re-use memory in already existing pools? I set maxPoolSize:1 to gain pools

Re: Minimize GC memory footprint

2021-01-31 Thread Max Haughton via Digitalmars-d-learn
On Saturday, 30 January 2021 at 16:42:35 UTC, frame wrote: Is there a way to force the GC to re-use memory in already existing pools? I set maxPoolSize:1 to gain pools that can be quicker released after there no longer in use. This already reduces memory usage to 1:3. Sadly the application

Re: Minimize GC memory footprint

2021-01-30 Thread frame via Digitalmars-d-learn
On Saturday, 30 January 2021 at 22:57:41 UTC, Imperatorn wrote: On Saturday, 30 January 2021 at 16:42:35 UTC, frame wrote: Is there a way to force the GC to re-use memory in already existing pools? I set maxPoolSize:1 to gain pools that can be quicker released after there no longer in use

Re: Minimize GC memory footprint

2021-01-30 Thread Imperatorn via Digitalmars-d-learn
On Saturday, 30 January 2021 at 16:42:35 UTC, frame wrote: Is there a way to force the GC to re-use memory in already existing pools? I set maxPoolSize:1 to gain pools that can be quicker released after there no longer in use. This already reduces memory usage to 1:3. Sadly the application

Minimize GC memory footprint

2021-01-30 Thread frame via Digitalmars-d-learn
Is there a way to force the GC to re-use memory in already existing pools? I set maxPoolSize:1 to gain pools that can be quicker released after there no longer in use. This already reduces memory usage to 1:3. Sadly the application creates multiple pools that are not necessary in my POV

Re: Compile time check for GC?

2021-01-28 Thread Kagamin via Digitalmars-d-learn
You can make it opt in, it's insurance.

Re: Compile time check for GC?

2021-01-27 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
On Wednesday, 27 January 2021 at 20:21:02 UTC, Kagamin wrote: You can define a symbol that will conflict with GC and prevent linking with it. That was an interesting thought! Maybe that could be a first step. It didn't occur to me that I could sabotage the usage of GC. :-D

Re: Compile time check for GC?

2021-01-27 Thread Kagamin via Digitalmars-d-learn
You can define a symbol that will conflict with GC and prevent linking with it.

Re: Compile time check for GC?

2021-01-26 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
On Tuesday, 26 January 2021 at 16:08:15 UTC, Steven Schveighoffer wrote: std.traits.hasAliasing? I guess, but it will limit me too much. I should accept Element types that manage their own memory and has pointers to sub-ojects that don't point to GC memory. But I guess that would require

Re: Compile time check for GC?

2021-01-26 Thread Steven Schveighoffer via Digitalmars-d-learn
On 1/26/21 11:00 AM, Ola Fosheim Grøstad wrote: On Tuesday, 26 January 2021 at 15:30:09 UTC, Steven Schveighoffer wrote: The only way to ensure the GC isn't used is with betterC. Even with @nogc tag on main, the GC could be used in static ctors, and casting function pointers/etc. Yes, @nogc

Re: Compile time check for GC?

2021-01-26 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
On Tuesday, 26 January 2021 at 15:30:09 UTC, Steven Schveighoffer wrote: The only way to ensure the GC isn't used is with betterC. Even with @nogc tag on main, the GC could be used in static ctors, and casting function pointers/etc. Yes, @nogc is not strong enough... It is for a container

Re: Compile time check for GC?

2021-01-26 Thread Steven Schveighoffer via Digitalmars-d-learn
On 1/26/21 8:10 AM, Ola Fosheim Grøstad wrote: Is there some way for library authors to test whether a GC is present at compile time? @nogc just means that my code does not depend on a GC, but it doesn't tell the compiler that my code is incompatible with GC. I want to compute pointers

Compile time check for GC?

2021-01-26 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
Is there some way for library authors to test whether a GC is present at compile time? @nogc just means that my code does not depend on a GC, but it doesn't tell the compiler that my code is incompatible with GC. I want to compute pointers in a way that is not scannable by a conservative

Re: Why many programmers don't like GC?

2021-01-19 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
On Tuesday, 19 January 2021 at 13:41:33 UTC, ddcovery wrote: (so I really don't understand your somewhat over-acted answer... maybe I need to read all the threads to understand your discomfort. In any case, accept my forgiveness if I have been able to bother you). Forgot to answer, maybe I

Re: Why many programmers don't like GC?

2021-01-19 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
On Tuesday, 19 January 2021 at 13:41:33 UTC, ddcovery wrote: That you want GC to work efficiently seems great to me... but at least we agree that D memory management is (and must be) GC based (so I really don't understand your somewhat over-acted answer... maybe I need to read all the threads

Re: Why many programmers don't like GC?

2021-01-19 Thread ddcovery via Digitalmars-d-learn
On Tuesday, 19 January 2021 at 11:25:13 UTC, Ola Fosheim Grøstad wrote: On Tuesday, 19 January 2021 at 10:43:45 UTC, Ola Fosheim Grøstad wrote: On Tuesday, 19 January 2021 at 10:36:13 UTC, ddcovery wrote: GC if D is not enough for you), but think about the thousands of experienced developers

Re: Why many programmers don't like GC?

2021-01-19 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
On Tuesday, 19 January 2021 at 10:43:45 UTC, Ola Fosheim Grøstad wrote: On Tuesday, 19 January 2021 at 10:36:13 UTC, ddcovery wrote: GC if D is not enough for you), but think about the thousands of experienced developers that where looking for something mature to work with and found that D

Re: Why many programmers don't like GC?

2021-01-19 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
On Tuesday, 19 January 2021 at 10:36:13 UTC, ddcovery wrote: GC if D is not enough for you), but think about the thousands of experienced developers that where looking for something mature to work with and found that D was not an option. And that's the point. The vast majority

Re: Why many programmers don't like GC?

2021-01-19 Thread ddcovery via Digitalmars-d-learn
are really clever about its paradigms decisions and no one (as far as I perceive) is discussing if GC must be removed from Go or added to Rust: developers see what language offers them and they decide. D toke it's key decisions in the past: of course it is a "generalist" langu

Re: Why many programmers don't like GC?

2021-01-19 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
is needed): Precise tracing: - The compiler currently cannot know what a tracing pointer points to: unions, casts. - The GC traces many paths that never leads to ownership. Non-tracing pointer needed. ARC: - There is no knowledge of ownership, so ARC cannot be added. Compiler needs to know

Re: Why many programmers don't like GC?

2021-01-18 Thread Imperatorn via Digitalmars-d-learn
On Monday, 18 January 2021 at 12:41:31 UTC, Ola Fosheim Grøstad wrote: On Monday, 18 January 2021 at 12:17:24 UTC, aberba wrote: [...] Not fighting the GC, but the whole argument about improving it, or mix or match, does not work for most developers looking for a new language. So either

Re: Why many programmers don't like GC?

2021-01-18 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
On Monday, 18 January 2021 at 15:18:40 UTC, aberba wrote: 1). You're not a minority at all. System programming is also vast so having a GC (especially D's special kind of GC) is nothing alien in System programming. If you look out there, This is not true, and you know it. There is nothing

Re: Why many programmers don't like GC?

2021-01-18 Thread aberba via Digitalmars-d-learn
On Monday, 18 January 2021 at 13:14:16 UTC, Arafel wrote: On 18/1/21 13:41, Ola Fosheim Grøstad wrote: Yes, it is natural that the current D population don't mind the current GC. Otherwise they would be gone... but then you have to factor in all the people that go through the revolving door

Re: Why many programmers don't like GC?

2021-01-18 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
On Monday, 18 January 2021 at 13:14:16 UTC, Arafel wrote: I must be in the minority here because one of the reasons why I started using D was precisely because it HAS a GC with full support. I wouldn't even have considered it if it hadn't. You are probably not in a minority among those

Re: Why many programmers don't like GC?

2021-01-18 Thread Arafel via Digitalmars-d-learn
On 18/1/21 13:41, Ola Fosheim Grøstad wrote: Yes, it is natural that the current D population don't mind the current GC. Otherwise they would be gone... but then you have to factor in all the people that go through the revolving door and does not stay. If they stayed the eco system would

Re: Why many programmers don't like GC?

2021-01-18 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
On Monday, 18 January 2021 at 12:17:24 UTC, aberba wrote: If you read the Origin of D book, you would see that the GC was a desire thing when D was designed probably due to how useful it is for ... as said, 90% or so of software development. So at this point, fighting the GC isn't (in my

Re: Why many programmers don't like GC?

2021-01-18 Thread Imperatorn via Digitalmars-d-learn
you write in, your program will still suck. No amount of compiler magic will be able to help you. The solution is not to blame this or that, it's to learn how to use what the language offers you effectively. [...] +1 for this. We should/could improve the GC instead of fighting it 

Re: Why many programmers don't like GC?

2021-01-18 Thread H. S. Teoh via Digitalmars-d-learn
On Mon, Jan 18, 2021 at 11:43:20AM +, aberba via Digitalmars-d-learn wrote: [...] > It talks how the use of GC is desired even in a game engine like > Unreal. Several AAA title's have been built on Unreal. > > Apparently you can't convince people who have made up their mind abou

Re: Why many programmers don't like GC?

2021-01-18 Thread aberba via Digitalmars-d-learn
On Monday, 18 January 2021 at 11:55:46 UTC, Ola Fosheim Grøstad wrote: On Monday, 18 January 2021 at 11:43:20 UTC, aberba wrote: Nevertheless, GC in D isn't going anywhere. And if the approach for writing nogc code in D doesn't cut it, then I'm not what else will. As long as that attitude

Re: Why many programmers don't like GC?

2021-01-18 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
On Monday, 18 January 2021 at 11:43:20 UTC, aberba wrote: Nevertheless, GC in D isn't going anywhere. And if the approach for writing nogc code in D doesn't cut it, then I'm not what else will. As long as that attitude prevails, D will be going nowhere as well.

Re: Why many programmers don't like GC?

2021-01-18 Thread aberba via Digitalmars-d-learn
On Monday, 18 January 2021 at 07:11:20 UTC, Ola Fosheim Grostad wrote: On Monday, 18 January 2021 at 01:41:35 UTC, James Blachly wrote: Those were not aberba's words, but the author of the first link, in which one does find a conceptual, high level description of GC. I read it, it said

Re: Why many programmers don't like GC?

2021-01-17 Thread Ola Fosheim Grostad via Digitalmars-d-learn
On Monday, 18 January 2021 at 01:41:35 UTC, James Blachly wrote: Those were not aberba's words, but the author of the first link, in which one does find a conceptual, high level description of GC. I read it, it said nothing of relevance to the D collector. That is not TLDR informative.

Re: Why many programmers don't like GC?

2021-01-17 Thread James Blachly via Digitalmars-d-learn
r not) has to hide internal structures and intricacies and provide some convenience. Those were not aberba's words, but the author of the first link, in which one does find a conceptual, high level description of GC.

Re: Why many programmers don't like GC?

2021-01-17 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
On Saturday, 16 January 2021 at 00:20:16 UTC, Guillaume Piolat wrote: It's certainly true that in team dynamics, without any reward, efficiency can be victim to a tragedy of commons. Well, any software invariant is harder to hold if the shareholders don't care. (be it "being fast", or "being

Re: Why many programmers don't like GC?

2021-01-15 Thread Guillaume Piolat via Digitalmars-d-learn
On Friday, 15 January 2021 at 19:49:34 UTC, Ola Fosheim Grøstad wrote: Many open source projects (and also some commercial ones) work ok for small datasets, but tank when you increase the dataset. So "match and mix" basically means use it for prototyping, but

Re: Why many programmers don't like GC?

2021-01-15 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
allocated then you just put the reference count at a negative offset (common strategy). You need pointer types for it, but that is not a big issue if the strategy is to support both the old GC and ARC. You basically just need to get library authors that support ARC to mark their library code in some

Re: Why many programmers don't like GC?

2021-01-15 Thread Max Haughton via Digitalmars-d-learn
ike slices and by-value structs. Having slices backed by the GC is actually a very powerful combination that people seem to overlook: it means you can freely refer to data by slicing the buffer. Strings being slices, as opposed to null-terminated, is a big part of this. In C, you cannot assum

Re: Why many programmers don't like GC?

2021-01-15 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
On Friday, 15 January 2021 at 21:18:55 UTC, aberba wrote: TL;DR: In summation, the garbage collection system is a robust part of Unreal Engine that affords C++ programmers a lot of safety from memory leaks, as well as convenience. With this high-level discussion, I was aiming to introduce

Re: Why many programmers don't like GC?

2021-01-15 Thread H. S. Teoh via Digitalmars-d-learn
ing a > > large amount of garbage from small allocations; > > <...> > > The result was about 40-50% reduction in runtime, which is close to > > about a 2x speedup. > > I think this message needs to be signal boosted. Most of the time GC > is not the problem.

Re: Why many programmers don't like GC?

2021-01-15 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
acquire resources in a somewhat chaotic manner, but they have fine tuned it and only call it when the call stack is short. High quality game engines have similarly fine tuned collection, and not really a big sweeping conservative scan that lock down threads. (BTW ARC is also another form of GC

Re: Why many programmers don't like GC?

2021-01-15 Thread aberba via Digitalmars-d-learn
On Friday, 15 January 2021 at 19:49:34 UTC, Ola Fosheim Grøstad wrote: On Friday, 15 January 2021 at 19:37:12 UTC, Guillaume Piolat wrote: A small GC heap is sufficient. There is this blog post where there was a quantitative measure of the sub-1ms D GC heap size. That's ok for a small game

Re: Why many programmers don't like GC?

2021-01-15 Thread aberba via Digitalmars-d-learn
used mix and match. (BTW ARC is also another form of GC) Unreal game engine https://mikelis.net/garbage-collection-in-ue4-a-high-level-overview/ Unity (of course) https://docs.unity3d.com/Manual/UnderstandingAutomaticMemoryManagement.html [...] TL;DR: In summation, the garbage collection

Re: Why many programmers don't like GC?

2021-01-15 Thread welkam via Digitalmars-d-learn
uction in runtime, which is close to about a 2x speedup. I think this message needs to be signal boosted. Most of the time GC is not the problem. The problem is sloppy memory usage. If you allocate a lot of temporary objects your performance will suffer even if you use malloc and free. If you writ

Re: To switch GC from FIFO to LIFO paradigm.

2021-01-15 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Jan 15, 2021 at 08:19:18PM +, tsbockman via Digitalmars-d-learn wrote: [...] > However, generational GCs are somewhat closer to LIFO than what we > have now, which does provide some performance gains under common usage > patterns. People have discussed adding a generationa

Re: To switch GC from FIFO to LIFO paradigm.

2021-01-15 Thread tsbockman via Digitalmars-d-learn
On Friday, 15 January 2021 at 12:39:30 UTC, MGW wrote: GC cleans memory using the FIFO paradigm. Is it possible to switch GC to work using the LIFO paradigm? As others already said, the current GC isn't FIFO; it just scans everything once in a while a frees whatever it can, new or old

Re: Why many programmers don't like GC?

2021-01-15 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
On Friday, 15 January 2021 at 19:37:12 UTC, Guillaume Piolat wrote: A small GC heap is sufficient. There is this blog post where there was a quantitative measure of the sub-1ms D GC heap size. That's ok for a small game, but not for applications that grow over time or projects where

Re: Why many programmers don't like GC?

2021-01-15 Thread Guillaume Piolat via Digitalmars-d-learn
On Friday, 15 January 2021 at 18:55:27 UTC, Ola Fosheim Grøstad wrote: On Friday, 15 January 2021 at 18:43:44 UTC, Guillaume Piolat wrote: Calling collect() isn't very good, it's way better to ensure the GC heap is relatively small, hence easy to traverse. You can use -gc=profile

Re: Why many programmers don't like GC?

2021-01-15 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
On Friday, 15 January 2021 at 18:43:44 UTC, Guillaume Piolat wrote: Calling collect() isn't very good, it's way better to ensure the GC heap is relatively small, hence easy to traverse. You can use -gc=profile for this (noting that things that can't contain pointer, such as ubyte[], scan way

Re: Why many programmers don't like GC?

2021-01-15 Thread Guillaume Piolat via Digitalmars-d-learn
On Friday, 15 January 2021 at 16:37:46 UTC, Ola Fosheim Grøstad wrote: But when do you call collect? Do you not create more and more long-lived objects? Calling collect() isn't very good, it's way better to ensure the GC heap is relatively small, hence easy to traverse. You can use -gc

Re: Why many programmers don't like GC?

2021-01-15 Thread jmh530 via Digitalmars-d-learn
. DMD, as a program, uses the bump the pointer allocation strategy. If you compile a D program with DMD that uses new or appends to a dynamic array (or whenver else), then it is using the GC to do that. You can also use malloc or your own custom strategy. The GC will reclaim memory

Re: Why many programmers don't like GC?

2021-01-15 Thread welkam via Digitalmars-d-learn
On Wednesday, 13 January 2021 at 18:58:56 UTC, Marcone wrote: I've always heard programmers complain about Garbage Collector GC. But I never understood why they complain. What's bad about GC? Most people get to know GC trough Java or C#. Those languages promote the use of OOP and they say

Re: Why many programmers don't like GC?

2021-01-15 Thread H. S. Teoh via Digitalmars-d-learn
no memory reclamation? We're apparently cross-talking here. A default D program uses the GC, as should be obvious by now. DMD itself, however, uses bump-the-pointer (*not* programs it compiles, though!). The two are completely unrelated. T -- Let X be the set not defined by this sentence...

Re: Why many programmers don't like GC?

2021-01-15 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
GC for some long-lived objects. Memory pools for most game entities. Audio thread has disabled GC. But when do you call collect? Do you not create more and more long-lived objects? - Dplug plugins before runtime removal used GC in the UI, but no GC in whatever was called repeatedly

Re: Why many programmers don't like GC?

2021-01-15 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
On Friday, 15 January 2021 at 16:21:43 UTC, jmh530 wrote: On Friday, 15 January 2021 at 15:36:37 UTC, Ola Fosheim Grøstad wrote: The library smart pointers would make it difficult to interact with existing D GC code though. Yes. So it would be better to do it automatically in the compiler

Re: Why many programmers don't like GC?

2021-01-15 Thread Guillaume Piolat via Digitalmars-d-learn
On Friday, 15 January 2021 at 16:21:18 UTC, Ola Fosheim Grøstad wrote: What do you mean by "mix and match"? If it means shutting down the GC after initialization then it can easily backfire for more complicated software that accidentally calls code that relies on the GC. I mean:

Re: Why many programmers don't like GC?

2021-01-15 Thread IGotD- via Digitalmars-d-learn
On Friday, 15 January 2021 at 15:50:50 UTC, H. S. Teoh wrote: DMD *never* frees anything. *That's* part of why it's so fast; it completely drops the complexity of tracking free lists and all of that jazz. That's also why it's a gigantic memory hog that can be a big embarrassment when run

Re: Why many programmers don't like GC?

2021-01-15 Thread jmh530 via Digitalmars-d-learn
. It uses library smart pointers for dirty-marking. But it requires you to write a virtual function that points out what should be traced (actually does the tracing for the outgoing pointers from that object): The library smart pointers would make it difficult to interact with existing D GC

Re: Why many programmers don't like GC?

2021-01-15 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
On Friday, 15 January 2021 at 15:50:59 UTC, Guillaume Piolat wrote: On Friday, 15 January 2021 at 11:11:14 UTC, Mike Parker wrote: That's the whole point of being able to mix and match. Anyone avoiding the GC completely is missing it (unless they really, really, must be GC-less). +1 mix

Re: To switch GC from FIFO to LIFO paradigm.

2021-01-15 Thread Imperatorn via Digitalmars-d-learn
On Friday, 15 January 2021 at 12:39:30 UTC, MGW wrote: GC cleans memory using the FIFO paradigm. Is it possible to switch GC to work using the LIFO paradigm? AFAIK the GC just sweeps, and the only queue is for destructors (unreachable memory) iirc

Re: Why many programmers don't like GC?

2021-01-15 Thread Ola Fosheim Grøstad via Digitalmars-d-learn
On Friday, 15 January 2021 at 15:48:07 UTC, welkam wrote: On Friday, 15 January 2021 at 14:35:55 UTC, Ola Fosheim Grøstad wrote: improved with precise collection Precise GC is slower than default GC. D does not have a fully precise GC. The "precise" collector still sc

Re: Why many programmers don't like GC?

2021-01-15 Thread welkam via Digitalmars-d-learn
On Friday, 15 January 2021 at 15:18:31 UTC, IGotD- wrote: I have a feeling that bump the pointer is not the complete algorithm that D uses because of that was the only one, D would waste a lot of memory. Freeing memory is for loosers :D https://issues.dlang.org/show_bug.cgi?id=21248 DMD

Re: Why many programmers don't like GC?

2021-01-15 Thread Guillaume Piolat via Digitalmars-d-learn
On Friday, 15 January 2021 at 11:11:14 UTC, Mike Parker wrote: That's the whole point of being able to mix and match. Anyone avoiding the GC completely is missing it (unless they really, really, must be GC-less). +1 mix and match is a different style versus only having a GC, or only having

  1   2   3   4   5   6   7   8   9   10   >