Re: Practical parallelization of D compilation

2020-01-15 Thread Guillaume Lathoud via Digitalmars-d-learn
Concerning the first (fresh) compilation: On Wednesday, 8 January 2020 at 13:14:38 UTC, kinke wrote: [...] you can also try to compile all modules at once, but output separate object files - `ldc2 -c a.d b.d c.d`. I just had another try at `ldc2 -c ...`. It does work when grouping the files i

Re: Practical parallelization of D compilation

2020-01-09 Thread Guillaume Lathoud via Digitalmars-d-learn
On Wednesday, 8 January 2020 at 19:31:13 UTC, H. S. Teoh wrote: Personally I prefer using SCons (https://scons.org/), but there are plenty of similar build systems out there, like tup, Meson, CMake, etc.. There are also fancier offerings that double as package managers like Gradle, but from th

Re: Practical parallelization of D compilation

2020-01-09 Thread Guillaume Lathoud via Digitalmars-d-learn
On Wednesday, 8 January 2020 at 15:44:24 UTC, user1234 wrote: yeah there's one. DUB does the same as you script with the following options: dub build --parallel --build-mode=singleFile I just gave it a try, and it stopped with a warning on some file. After fixing the file, I relaunched d

Re: Practical parallelization of D compilation

2020-01-08 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Jan 08, 2020 at 09:13:18AM +, Chris Katko via Digitalmars-d-learn wrote: > On Wednesday, 8 January 2020 at 06:51:57 UTC, H. S. Teoh wrote: [...] > > Generally, the recommendation is to separately compile each package. [...] > What's the downsides / difficulties / "hoops to jump through

Re: Practical parallelization of D compilation

2020-01-08 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Jan 08, 2020 at 06:56:20PM +, Guillaume Lathoud via Digitalmars-d-learn wrote: > Thanks to all for the answers. > > The package direction is precisely what I am trying to avoid. It is > still not obvious to me how much work (how many trials) would be > needed to decide on granularity,

Re: Practical parallelization of D compilation

2020-01-08 Thread Guillaume Lathoud via Digitalmars-d-learn
Thanks to all for the answers. The package direction is precisely what I am trying to avoid. It is still not obvious to me how much work (how many trials) would be needed to decide on granularity, as well as how much work to automatize the decision to recompile a package or not ; and finally,

Re: Practical parallelization of D compilation

2020-01-08 Thread user1234 via Digitalmars-d-learn
On Wednesday, 8 January 2020 at 04:40:02 UTC, Guillaume Lathoud wrote: Hello, One of my D applications grew from a simple main and a few source files to more than 200 files. Although I minimized usage of templating and CTFE, the compiling time is now about a minute. I did not find any solutio

Re: Practical parallelization of D compilation

2020-01-08 Thread kinke via Digitalmars-d-learn
On Wednesday, 8 January 2020 at 04:40:02 UTC, Guillaume Lathoud wrote: * first run (compiling everything): 50% to 100% slower than classical compilation, depending on the hardware, resp. on an old 4-core or a more recent 8-core. If parallel compiler invocations for each source file are

Re: Practical parallelization of D compilation

2020-01-08 Thread Chris Katko via Digitalmars-d-learn
On Wednesday, 8 January 2020 at 06:51:57 UTC, H. S. Teoh wrote: On Wed, Jan 08, 2020 at 04:40:02AM +, Guillaume Lathoud via Digitalmars-d-learn wrote: [...] [...] Generally, the recommendation is to separately compile each package. E.g., if you have a source tree of the form: sr

Re: Practical parallelization of D compilation

2020-01-07 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Jan 08, 2020 at 04:40:02AM +, Guillaume Lathoud via Digitalmars-d-learn wrote: > Hello, > > One of my D applications grew from a simple main and a few source > files to more than 200 files. Although I minimized usage of > templating and CTFE, the compiling time is now about a minute.

Practical parallelization of D compilation

2020-01-07 Thread Guillaume Lathoud via Digitalmars-d-learn
Hello, One of my D applications grew from a simple main and a few source files to more than 200 files. Although I minimized usage of templating and CTFE, the compiling time is now about a minute. I did not find any solution to take advantage of having multiple cores during compilation, lest I wo