Re: Templating everything? One module per function/struct/class/etc, grouped by package?
On 12/05/14 20:58, Francesco Cattoglio wrote: Error messages! If your code is not compiled, you can't know whether it is valid or not. I must say that since we have unittests, this is somewhat less relevant, but still... One nice thing would be stripping the executable of unneeded code. One trick I've seen done in a program which compiled some scripts to an intermediate language was zeroing the parts which are unused, then use some executable compressor. I had an issue with a module in Tango back in the D1 days. It was a templated class that worked with char, but when I instantiated it with wchar it didn't compile. -- /Jacob Carlborg
Templating everything? One module per function/struct/class/etc, grouped by package?
Given that...
1. importing a module makes it compile the entirety of it, as
well as whatever it may be importing in turn
2. templates are only compiled if instantiated
3. the new package.d functionality
...is there a reason *not* to make every single
function/struct/class separate submodules in a package, and make
*all* of those templates? Unnused functionality would never be
imported nor instantiated, and as such never be compiled, so my
binary would only include what it actually uses.
std/stdio/package.d:
module std.stdio;
// still allows for importing the entirety of std.stdio
public import std.stdio.foo;
public import std.stdio.writefln;
__EOF__
std/stdio/foo.d:
module std.stdio.foo;
void fooify(Args...)(Args args)
if (Args.length 0)
{
// ...
}
void fooify()()
{
// don't need this, won't compile this
}
__EOF__
std/stdio/writefln.d;
module std.stdio.writefln;
// nevermind the incompatible signature
auto writefln(string pattern, Args...)(Args args)
if (!pattern.canFind(PatternIdentifier.json))
{
// code that doesn't need std.json -- it is never
imported
}
__EOF__
What am I missing?
Re: Templating everything? One module per function/struct/class/etc, grouped by package?
On Mon, 12 May 2014 08:37:42 +
JR via Digitalmars-d-learn digitalmars-d-learn@puremagic.com wrote:
Given that...
1. importing a module makes it compile the entirety of it, as
well as whatever it may be importing in turn
2. templates are only compiled if instantiated
3. the new package.d functionality
...is there a reason *not* to make every single
function/struct/class separate submodules in a package, and make
*all* of those templates? Unnused functionality would never be
imported nor instantiated, and as such never be compiled, so my
binary would only include what it actually uses.
std/stdio/package.d:
module std.stdio;
// still allows for importing the entirety of std.stdio
public import std.stdio.foo;
public import std.stdio.writefln;
__EOF__
std/stdio/foo.d:
module std.stdio.foo;
void fooify(Args...)(Args args)
if (Args.length 0)
{
// ...
}
void fooify()()
{
// don't need this, won't compile this
}
__EOF__
std/stdio/writefln.d;
module std.stdio.writefln;
// nevermind the incompatible signature
auto writefln(string pattern, Args...)(Args args)
if (!pattern.canFind(PatternIdentifier.json))
{
// code that doesn't need std.json -- it is never
imported
}
__EOF__
What am I missing?
Well, that would be a lot of extraneous files, which would be very messy IMHO.
It also makes it much harder to share private functionality, because
everything is scattered across modules - you'd be force to use the package for
that. It also wouldn't surprise me if it cost more to compile the code that
way if you were actually using most of it (though it may very well save
compilation time if you're using a relatively small number of the functions
and types). So, from a purely organization perspective, I think that it's a
very bad idea, though others may think that it's a good one. And since
package.d imports all of those modules anyway, separating them out into
separate files didn't even help you any.
Also, templates cost more to compile, so while you may avoid having to compile
some functions, becasue you're not using them, everything that _does_ get
compiled will take longer to compile. And if you templatize them in a way that
would result in more template instantiations (e.g. you actually templatize the
parameters rather than just giving the function an empty template parameter
list), then not only will the functions have to be compiled more frequently
(due to multiple instantiations), but they'll take up more space in the final
binary. Also, while D does a _much_ better job with template errors than C++
does, template-related errors in D are still far worse than with functions
that aren't templated, so you're likely going to cost yourself more time
debugging template-related compilation errors than you ever would gain in
reduced compilation times.
In addition, if a function is templatized, it's harder to use it with function
prototypes, which can be a definite problem for some code. It's also a
horrible idea for libraries to have functions templatized just to be
templatized, because that means that a function has to be compiled _every_
time that a program uses it rather than having it compiled once when the
library is compiled (the function would still have to be parsed unless it just
had its signature in a .di file, but that's still faster than full
compilation - and if a .di file is used, then all that has to be parsed is the
signature). So, while it's often valuable to templatize functions,
templatizing them to save compilation times is questionable at best.
D already does a _very_ good job at compiling quickly. Often, the linking step
costs more than the actualy compilation does (though obviosuly, as programs
grow larger, the compilation time does definitely exceed the link time).
Unless you're running into problems with compilation speed, I'd strongly
advise against trying to work around the compiler to speed up code
compilation. Templatize functions when it makes sense to do so, but don't
templatize them just in an attempt to avoid having them be compiled.
If you're looking to speed up compilation times, it makes far more sense to
look at doing things like reducing how much CTFE you use and how many
templates you use. CTFE in particular is ridiculously expensive thanks to it
effectively just being hacked into the compiler originally. Don has been doing
work to improve that, and I expect it to improve over time, but I don't know
how far along he is, and I don't know that it'll ever be exactly cheap to use
CTFE.
Keep in mind that lexing and parsing are the _cheap_ part of the compiler. So,
importing stuff really doesn't cost you much. Already, the compiler won't
fully compile all of the symbols within a module except when it's compiling
that module. Simply importing it just causes it to process the module as much
as required to
Re: Templating everything? One module per function/struct/class/etc, grouped by package?
On 5/12/14, 5:37 AM, JR wrote: Given that... 1. importing a module makes it compile the entirety of it, as well as whatever it may be importing in turn 2. templates are only compiled if instantiated 3. the new package.d functionality ...is there a reason *not* to make every single function/struct/class separate submodules in a package, and make *all* of those templates? Unnused functionality would never be imported nor instantiated, and as such never be compiled, so my binary would only include what it actually uses. Welcome to Crystal :-) In Crystal, every function and method is templated. When you compile your program, only what you use gets compiled. A simple hello world program is just 16KB. And contrary to what Jonathan M. Davis says, compiling programs is very fast. In fact, compilation times might indeed be faster, because you don't need to compile unused code. And the resulting binary is as small as possible. And the error messages are pretty good, also. But D has an entirely different philosophy, so I don't think they will like it. I also once suggested auto for parameters, but they didn't like it. I'm just saying this to say that yes, it's possible, and no, it doesn't hurt compilation times or error messages.
Re: Templating everything? One module per function/struct/class/etc, grouped by package?
On Monday, 12 May 2014 at 09:16:53 UTC, Jonathan M Davis via Digitalmars-d-learn wrote: Well, that would be a lot of extraneous files, which would be very messy IMHO. It also makes it much harder to share private functionality, because everything is scattered across modules - you'd be force to use the package for that. It also wouldn't surprise me if it cost more to compile the code that way if you were actually using most of it (though it may very well save compilation time if you're using a relatively small number of the functions and types). So, from a purely organization perspective, I think that it's a very bad idea, though others may think that it's a good one. And since package.d imports all of those modules anyway, separating them out into separate files didn't even help you any. Thank you for answering. The package.d example was mostly to highlight that the current syntax and practice of importing everything in a package wouldn't change. To be a bit more specific I'm new to everything beyond advanced scripting, so my code has grown organically. Partly due to shotgun programming, and partly due to exploratory programming. In the process of figuring out what code depends on what, to allow for making pieces of it more self-contained, I realized that with my current practice of gathering everything related to the same topic into one module made it all unneccesarily tangled. (--gc-sections seems to cause segfaults.) As an example, both the homebrew string builder 'Yarn' and the simple 'string plurality(ptrdiff_t, string, string)' belong in mylib.string. But if I want to reuse only plurality there in other projects I would end up pulling in all the ancillary stuff Yarn needs to validate input, and it only segways from there. Correct? The logical step would be to split the all-encompassing mylib.string module into submodules, and the logical followup question would be why I shouldn't do that everywhere. Knowing that templates aren't built unless concretely instantiated, the idea of templatizing everything fell into the same basket. I hope you understand my concerns that I was falling into an anti-pattern.
Re: Templating everything? One module per function/struct/class/etc, grouped by package?
On Monday, 12 May 2014 at 08:37:43 UTC, JR wrote: What am I missing? Error messages! If your code is not compiled, you can't know whether it is valid or not. I must say that since we have unittests, this is somewhat less relevant, but still... One nice thing would be stripping the executable of unneeded code. One trick I've seen done in a program which compiled some scripts to an intermediate language was zeroing the parts which are unused, then use some executable compressor.
Re: Templating everything? One module per function/struct/class/etc, grouped by package?
You can write a tool, which will construct an amalgamation build of your code.
