Re: GDC - program runs in one thread, DMD - in 4 threads, why?
On Friday, 10 September 2021 at 12:59:08 UTC, bauss wrote: It's just used to speed-up the GC. Yeah, I got the point, but to be absolutely honest, I (>20 years of C coding) do not like GC as such. I believe manual free() is not that 'hard'. And one must still release other resources. (in C I write just close(fd), in D I write client.destroy() - in both cases I have to do this manually, if that fd was obtained in absolutely another scope) Yes, I know, I can do without GC in D, just did not tried yet (started learning D two months ago or so) And again - I've chosen D (not Rust), because 'borrow checker' and especially 'explicit lifetimes' is a nightmare :) - i would rather accept GC.
Re: GDC - program runs in one thread, DMD - in 4 threads, why?
On Friday, 10 September 2021 at 12:46:36 UTC, eugene wrote: On Friday, 10 September 2021 at 12:10:58 UTC, Adam D Ruppe wrote: btw why do the threads cause you trouble? Well... probably it is subjective thing - just do not 'like' when a program is doing something that is not explicitly in it's source (I am C coder, you guessed). More specifically - I have event driven state machine engine (reactor pattern, written in C), which I have been using over recent 5 years in various data acquisition and network client/servers programs. All these programs by their nature are single-threaded, since 'event/message driven' + 'state machines' combination gives me very high-grained concurrency within single thread (and I don't like fibers/greenlets/coroutines). Recently I tried to re-implement that engine in D (I also tried Rust and C#, but D appeared to be less 'problematic'). I can give a link to sources of those echo-client/server pair if someone is interested (but there is a 'little' problem remained - when compliled with dmd, it sometimes crashes upon receiving SIGINT). Well the program you're writing is still single-threaded. The GC only uses the other threads when sweeping so the program flow will still be single-threaded and the exact same. It's just used to speed-up the GC.
Re: GDC - program runs in one thread, DMD - in 4 threads, why?
On Friday, 10 September 2021 at 12:10:58 UTC, Adam D Ruppe wrote: btw why do the threads cause you trouble? Well... probably it is subjective thing - just do not 'like' when a program is doing something that is not explicitly in it's source (I am C coder, you guessed). More specifically - I have event driven state machine engine (reactor pattern, written in C), which I have been using over recent 5 years in various data acquisition and network client/servers programs. All these programs by their nature are single-threaded, since 'event/message driven' + 'state machines' combination gives me very high-grained concurrency within single thread (and I don't like fibers/greenlets/coroutines). Recently I tried to re-implement that engine in D (I also tried Rust and C#, but D appeared to be less 'problematic'). I can give a link to sources of those echo-client/server pair if someone is interested (but there is a 'little' problem remained - when compliled with dmd, it sometimes crashes upon receiving SIGINT).
Re: GDC - program runs in one thread, DMD - in 4 threads, why?
btw why do the threads cause you trouble?
Re: GDC - program runs in one thread, DMD - in 4 threads, why?
On Friday, 10 September 2021 at 11:53:04 UTC, eugene wrote: On Friday, 10 September 2021 at 11:32:02 UTC, Adam D Ruppe wrote: You either pass as an argument *to your application* --DRT-gcopt=parallel:0 oops... :) ps xH | grep [e]cho 5727 pts/14 S+ 0:13 ./echo-server --DRT-gcopt=parallel:0 5762 pts/15 S+ 0:12 ./echo-client --DRT-gcopt=parallel:0 thanks everybody for answers!
Re: GDC - program runs in one thread, DMD - in 4 threads, why?
On 9/10/21 7:47 AM, eugene wrote: On Friday, 10 September 2021 at 11:09:10 UTC, bauss wrote: --DRT-gcopt=parallel:2 on the command line. A value of 0 disables parallel marking completely. but it does not: make -f Makefile-dmd dmd --DRT-gcopt=parallel:0 engine/*.d common-sm/*.d server-sm/*.d pool.d echo_server.d -ofecho-server dmd --DRT-gcopt=parallel:0 engine/*.d common-sm/*.d client-sm/*.d pool.d echo_client.d -ofecho-client ps xH | grep [e]cho 5460 pts/14 Sl+ 0:00 ./echo-server 5460 pts/14 Sl+ 0:00 ./echo-server 5460 pts/14 Sl+ 0:00 ./echo-server 5460 pts/14 Sl+ 0:00 ./echo-server 5466 pts/15 Sl+ 0:00 ./echo-client 5466 pts/15 Sl+ 0:00 ./echo-client 5466 pts/15 Sl+ 0:00 ./echo-client 5466 pts/15 Sl+ 0:00 ./echo-client `--DRT...` is a d runtime switch, which is processed while running your program, not by the compiler. Try `./echo-client --DRT-gcopt=parallel:0` There is also a way to add this to your program so it's not needed on the command line. -Steve
Re: GDC - program runs in one thread, DMD - in 4 threads, why?
On Friday, 10 September 2021 at 11:32:02 UTC, Adam D Ruppe wrote: You either pass as an argument *to your application* --DRT-gcopt=parallel:0 oops... :)
Re: GDC - program runs in one thread, DMD - in 4 threads, why?
On Friday, 10 September 2021 at 11:09:10 UTC, bauss wrote: --DRT-gcopt=parallel:2 on the command line. A value of 0 disables parallel marking completely. but it does not: make -f Makefile-dmd dmd --DRT-gcopt=parallel:0 engine/*.d common-sm/*.d server-sm/*.d pool.d echo_server.d -ofecho-server dmd --DRT-gcopt=parallel:0 engine/*.d common-sm/*.d client-sm/*.d pool.d echo_client.d -ofecho-client ps xH | grep [e]cho 5460 pts/14 Sl+0:00 ./echo-server 5460 pts/14 Sl+0:00 ./echo-server 5460 pts/14 Sl+0:00 ./echo-server 5460 pts/14 Sl+0:00 ./echo-server 5466 pts/15 Sl+0:00 ./echo-client 5466 pts/15 Sl+0:00 ./echo-client 5466 pts/15 Sl+0:00 ./echo-client 5466 pts/15 Sl+0:00 ./echo-client
Re: GDC - program runs in one thread, DMD - in 4 threads, why?
On Friday, 10 September 2021 at 09:27:49 UTC, eugene wrote: What are these extra threads for? GC? So I think it is **very aggressive usage** of DList that causes this. Yeah, in newer versions, when the garbage collector does its first collect, it spawns some helper threads to speed up its mark process. When the GC is not actively marking, those extra threads are 100% idle, so they don't affect the rest of your program. Can this (really unwanted) behavior be disabled in DMD? I do not want to have multiple threads, here's the page on it https://dlang.org/spec/garbage.html#gc_config You either pass as an argument *to your application* --DRT-gcopt=parallel:0 or to make it permanent, in your source file with the main() function, add this: extern(C) __gshared string[] rt_options = [ "gcopt=parallel:0" ]; and it will stop doing that.
Re: GDC - program runs in one thread, DMD - in 4 threads, why?
On Friday, 10 September 2021 at 11:09:10 UTC, bauss wrote: Here's the specific change: https://dlang.org/changelog/2.087.0.html#gc_parallel thanx a lot!
Re: GDC - program runs in one thread, DMD - in 4 threads, why?
On Friday, 10 September 2021 at 10:20:52 UTC, drug wrote: It is off-topic a bit I am newbie - have been learning D for about 2 months or so. I understand that my question is not about the language itself, just picked forum for new users. but I think none can compare gdc 4.9.2 to same picture with gdc 8.4.0 - one thread, no pthread_create() behind the scenes.
Re: GDC - program runs in one thread, DMD - in 4 threads, why?
On Friday, 10 September 2021 at 11:20:10 UTC, eugene wrote: same picture with gdc 8.4.0 - one thread, no pthread_create() behind the scenes. GDC is stuck on a much older version of D. Iain has backported some bugfixes and optimizations, but featurewise it's mostly D 2.076. This is because that's the last version of the D frontend implemented in C++, and he needed that to get gdc into gcc. As bauss pointed out, the library feature you're talking about was implemented in 2.087. Iain is working on porting gdc over to the D version of the D frontend, at which point it will be much more up-to-date.
Re: GDC - program runs in one thread, DMD - in 4 threads, why?
On Friday, 10 September 2021 at 10:39:48 UTC, Basile B. wrote: On Friday, 10 September 2021 at 09:27:49 UTC, eugene wrote: Here is test program (which is using DList aggressively) [...] Can this (really unwanted) behavior be disabled in DMD? I do not want to have multiple threads, a program (real program, not the test above) has to be single-threaded. system used: @dexp ~ $ cat /etc/debian_version 8.11 most recent dmd has a GC that collect in threads. it can be tuned but I dont find the documentation for this right now. Here's the specific change: https://dlang.org/changelog/2.087.0.html#gc_parallel ``` GC now marks the heap with multiple threads The garbage collector now uses available CPU cores to mark the heap faster. This reduces pause times for a collection considerably. By default, the GC uses all available logical cores of your CPU. This might affect your application if it has threads that are not suspended during the mark phase of the collection. You can configure the number of additional threads used for marking by DRT option parallel to the GC configuration, e.g. by passing --DRT-gcopt=parallel:2 on the command line. A value of 0 disables parallel marking completely. As usual, you can also embed the configuration into the application by redefining rt_options, e.g. extern(C) __gshared string[] rt_options = [ "gcopt=parallel:0" ]; ``` __ So for OP what you want to use is: ```d --DRT-gcopt=parallel:0 ```
Re: GDC - program runs in one thread, DMD - in 4 threads, why?
On Friday, 10 September 2021 at 09:27:49 UTC, eugene wrote: Here is test program (which is using DList aggressively) [...] Can this (really unwanted) behavior be disabled in DMD? I do not want to have multiple threads, a program (real program, not the test above) has to be single-threaded. system used: @dexp ~ $ cat /etc/debian_version 8.11 most recent dmd has a GC that collect in threads. it can be tuned but I dont find the documentation for this right now.
Re: GDC - program runs in one thread, DMD - in 4 threads, why?
10.09.2021 12:27, eugene пишет: //import std.container.dlist; // dmd (v2.097.2) import std.container: DList; // gdc (4.9.2) It is off-topic a bit but I think none can compare gdc 4.9.2 to dmd 2.097.2 because gdc has older version than dmd. I would compare gdc to appropriate dmd version, it's about 2.078. But note that gdc backported some code from more recent dmd-fe versions so it has 2.078-dirty version in fact.
GDC - program runs in one thread, DMD - in 4 threads, why?
Here is test program (which is using DList aggressively)
```d
import std.stdio : writeln;
import core.sys.posix.unistd : sleep;
//import std.container.dlist; // dmd (v2.097.2)
import std.container: DList; // gdc (4.9.2)
void main() {
auto list = DList!(int)();
int k;
while (true) {
// 1.sleep;
k.writeln;
list.insertBack(k);
list.insertBack(k);
list.removeFront();
list.removeFront();
k++;
}
}
```
When compiled with gdc, it runs in one thread:
@dexp ~ $ ps xH | grep [t]est
3823 pts/12 R+ 0:07 ./test
But when with dmd, program creates 3 additional threads:
@dexp ~ $ ps xH | grep [t]est
3839 pts/12 Rl+0:00 ./test
3839 pts/12 Sl+0:00 ./test
3839 pts/12 Sl+0:00 ./test
3839 pts/12 Sl+0:00 ./test
What are these extra threads for?
To work with list (which is strange)???
GC?
Something else?
Observations:
- without DList there is no extra-threads
- with sleep() there is also no extra-threads
So I think it is **very aggressive usage** of DList that causes
this.
Can this (really unwanted) behavior be disabled in DMD?
I do not want to have multiple threads,
a program (real program, not the test above) has to be
single-threaded.
system used:
@dexp ~ $ cat /etc/debian_version
8.11
