Hi all,
Kenji and Walter have been working on improving the template
emission strategy during the last couple of releases, i.e.
whether a template instance is emitted to a given object file or
not. Nevertheless, I've been continually hearing complaints from
various people with large D code bases (our commercial users and
some of the more complex open source projects) that they have
problems compiling their code doing anything else than an
all-at-once build.
This of course detracts from what we like to cite as one of D's
key advantages, compilation speed, because you cannot exploit the
many cores of your build machine anymore, and the
edit-compile-debug cycle is slowed down because you have to build
the full application every time. But this turns into a severe
problem as soon as your compilation does not fit into RAM
anymore, which happens quite quickly when using D's advanced
features – see for example Liran Zvibel's DConf talk, where he
describes that they needed machines with more than 100 GB of RAM
in order to build the Weka code base.
In any case, I hope you agree that fixing these kinds of issues
that prevent D code from being compiled all is strategically
important for us, since they are a deciding factor in driving
widespread adoption. Sadly, the template problems didn't seem to
receive a lot of attention beyond quick fixes recently, possibly
because they don't occur as readily for smaller projects and if
so, are easier to work around.
With all that out of my system, I'd like to point your attention
to a particular template instantiation issue I managed to reduce
recently when working with the folks at Weka. It seems like there
is a fundamental oversight in the way the code tries to elide
repeat code generation of instances – or, of course, I'm just
missing something:
https://issues.dlang.org/show_bug.cgi?id=15318
This is one particular issue that makes writing "idiomatic D"
(which is rather template-heavy) in large code bases hard, but
likely not the only one. In fact, I already know about another
issue where function-local imports cause semantic analysis not to
be properly run on template instances, but I'm still struggling
to find a minimal example for the problem. My hope would be that
we can clean up this mess soon and replace the current ad-hoc
patchwork with a more principled approach. In the process, we
should also make sure that all of our assumptions [1] about the
compilation process are clearly stated in the documentation, as
that frequently leads to confusion among newcomers.
Best,
David
[1] That is: All imported modules must also be compiled into the
executable; incremental compilation is only guaranteed to work if
precisely the same subsets of modules are compiled every time.
