Re: D compiles fast, right? Right??

[Stefan Koch via Digitalmars-d]

On Wednesday, 4 April 2018 at 20:02:56 UTC, Dmitry Olshansky 
wrote:

On Wednesday, 4 April 2018 at 19:25:43 UTC, Stefan Koch wrote:
On Wednesday, 4 April 2018 at 01:08:48 UTC, Andrei 
Alexandrescu wrote:

[ ... ]

Exactly, which is why I'm insisting this - and not compiler 
benchmarking, let alone idle chattaroo in the forums - is 
where we need to hit. What we have here, ladies and 
gentlemen, is a high-impact preapproved item of great general 
interest. Shall we start the auction?


I have analyzed the problem;
And in this case it points to _std.uni_ as the main time-taker.


Could be because of huge tables. Does it point specifically to 
std.uni or its dependencies might count as well?


Anyway to import a binary file as an array? Turbo Pascal had a 
feature like that, it was awesome ;)


On other thought - maybe just .di + separate compilation is it.


There is the string import feature, which can give you a char 
array which you can then cast to ubyte slice and from there to a 
ubyte pointer you cast that to a pointer of the target type and 
slice it. This requires the target type to have no pointers 
inside of it.

Re: D compiles fast, right? Right??

[Jonathan Marler via Digitalmars-d]

On Wednesday, 4 April 2018 at 20:29:19 UTC, Stefan Koch wrote:

On Wednesday, 4 April 2018 at 20:04:04 UTC, Jack Stouffer wrote:
On Wednesday, 4 April 2018 at 01:08:48 UTC, Andrei 
Alexandrescu wrote:
Exactly, which is why I'm insisting this - and not compiler 
benchmarking, let alone idle chattaroo in the forums - is 
where we need to hit. What we have here, ladies and 
gentlemen, is a high-impact preapproved item of great general 
interest. Shall we start the auction?


Are you aware of this PR? 
https://github.com/dlang/dmd/pull/8124


This is but a layer of paint over the real problem.
Unneeded Dependencies. Programming should not be a game of 
jenga.

Piling things on top of other things rarely works out.


Having unittests included from precompiled libraries is a problem 
in and of itself.  This is causing many templates to be 
instantiated that will never be used by the application, killing 
compilation time.


There are also other problems...here's a link to my description 
of "Lazy Imports" that I think would help other issues we 
currently have.


https://github.com/marler8997/dlangfeatures#lazy-imports

Re: D compiles fast, right? Right??

[H. S. Teoh via Digitalmars-d]

On Wed, Apr 04, 2018 at 08:29:19PM +, Stefan Koch via Digitalmars-d wrote:
> On Wednesday, 4 April 2018 at 20:04:04 UTC, Jack Stouffer wrote:
> > On Wednesday, 4 April 2018 at 01:08:48 UTC, Andrei Alexandrescu wrote:
> > > Exactly, which is why I'm insisting this - and not compiler
> > > benchmarking, let alone idle chattaroo in the forums - is where we
> > > need to hit. What we have here, ladies and gentlemen, is a
> > > high-impact preapproved item of great general interest. Shall we
> > > start the auction?
> > 
> > Are you aware of this PR? https://github.com/dlang/dmd/pull/8124
> 
> This is but a layer of paint over the real problem.  Unneeded
> Dependencies. Programming should not be a game of jenga.  Piling
> things on top of other things rarely works out.

Unneeded dependencies is (partly) addressed by this PR, by isolating
user code from the unittests (and by extension, their dependencies) of
external libraries that said user code has no interest in.

True, this does not solve the entire problem, which is that too much of
Phobos is involved in a hairball of inextricable dependencies.  The
situation actually has already improved since 3-4 years ago, when it was
much, much worse.  It's just that we have still some ways to go.

On the flip side, though, why *shouldn't* unittests in a Phobos module
make use of other Phobos modules?  It's one thing to say that Phobos
code proper (i.e., outside of unittests) should be as independent as
possible, and that I agree with, strongly.  But unittests should be free
to import other stuff in order to make testing easier, and also ddoc'd
unittest examples more relevant to the reader.  By separating the import
of a module from the importing of its unittests (which doesn't even make
sense from an API perspective -- why should downstream user code need to
pull in upstream unittests, which are supposed to be used only for
upstream development?), we provide freedom for upstream library authors
to make use of external components in order to test their code, without
penalizing downstream user code with dependencies they don't care about.

I'd say this PR strikes at the heart of this problem, it's not just
papering over the issue.  It's not a full solution, but it's an
important step towards a full solution.


T

-- 
English is useful because it is a mess. Since English is a mess, it maps well 
onto the problem space, which is also a mess, which we call reality. Similarly, 
Perl was designed to be a mess, though in the nicest of all possible ways. -- 
Larry Wall

Re: D compiles fast, right? Right??

[Jack Stouffer via Digitalmars-d]

On Wednesday, 4 April 2018 at 20:29:19 UTC, Stefan Koch wrote:

On Wednesday, 4 April 2018 at 20:04:04 UTC, Jack Stouffer wrote:
On Wednesday, 4 April 2018 at 01:08:48 UTC, Andrei 
Alexandrescu wrote:
Exactly, which is why I'm insisting this - and not compiler 
benchmarking, let alone idle chattaroo in the forums - is 
where we need to hit. What we have here, ladies and 
gentlemen, is a high-impact preapproved item of great general 
interest. Shall we start the auction?


Are you aware of this PR? 
https://github.com/dlang/dmd/pull/8124


This is but a layer of paint over the real problem.
Unneeded Dependencies. Programming should not be a game of 
jenga.

Piling things on top of other things rarely works out.


I encourage you to leave a review on the PR then with corrective 
measures.

Re: D compiles fast, right? Right??

[Stefan Koch via Digitalmars-d]

On Wednesday, 4 April 2018 at 20:04:04 UTC, Jack Stouffer wrote:
On Wednesday, 4 April 2018 at 01:08:48 UTC, Andrei Alexandrescu 
wrote:
Exactly, which is why I'm insisting this - and not compiler 
benchmarking, let alone idle chattaroo in the forums - is 
where we need to hit. What we have here, ladies and gentlemen, 
is a high-impact preapproved item of great general interest. 
Shall we start the auction?


Are you aware of this PR? https://github.com/dlang/dmd/pull/8124


This is but a layer of paint over the real problem.
Unneeded Dependencies. Programming should not be a game of jenga.
Piling things on top of other things rarely works out.

Re: D compiles fast, right? Right??

[Dmitry Olshansky via Digitalmars-d]

On Wednesday, 4 April 2018 at 19:25:43 UTC, Stefan Koch wrote:
On Wednesday, 4 April 2018 at 01:08:48 UTC, Andrei Alexandrescu 
wrote:

[ ... ]

Exactly, which is why I'm insisting this - and not compiler 
benchmarking, let alone idle chattaroo in the forums - is 
where we need to hit. What we have here, ladies and gentlemen, 
is a high-impact preapproved item of great general interest. 
Shall we start the auction?


I have analyzed the problem;
And in this case it points to _std.uni_ as the main time-taker.


Could be because of huge tables. Does it point specifically to 
std.uni or its dependencies might count as well?


Anyway to import a binary file as an array? Turbo Pascal had a 
feature like that, it was awesome ;)


On other thought - maybe just .di + separate compilation is it.

Re: D compiles fast, right? Right??

[Jack Stouffer via Digitalmars-d]

On Wednesday, 4 April 2018 at 01:08:48 UTC, Andrei Alexandrescu 
wrote:
Exactly, which is why I'm insisting this - and not compiler 
benchmarking, let alone idle chattaroo in the forums - is where 
we need to hit. What we have here, ladies and gentlemen, is a 
high-impact preapproved item of great general interest. Shall 
we start the auction?


Are you aware of this PR? https://github.com/dlang/dmd/pull/8124

Re: D compiles fast, right? Right??

[Stefan Koch via Digitalmars-d]

On Wednesday, 4 April 2018 at 01:08:48 UTC, Andrei Alexandrescu 
wrote:

[ ... ]

Exactly, which is why I'm insisting this - and not compiler 
benchmarking, let alone idle chattaroo in the forums - is where 
we need to hit. What we have here, ladies and gentlemen, is a 
high-impact preapproved item of great general interest. Shall 
we start the auction?


I have analyzed the problem;
And in this case it points to _std.uni_ as the main time-taker.

I have also created and uploaded a video where I show how to do 
this kind of profiling.
It got a bit carried away at the beginning so you can skip to 
10:45 :)

https://www.youtube.com/watch?v=28bOtu64CCM

For those are not up to watching me talk slowly:
Phobos is in dire need of having dependencies broken.

And having dedicated compile-time profiling tools is pretty 
useful.
Even if they are very rudimentary it's better then semi-randomly 
commenting out code.


--
Stefan

Re: D compiles fast, right? Right??

[Jonathan M Davis via Digitalmars-d]

On Tuesday, April 03, 2018 21:08:48 Andrei Alexandrescu via Digitalmars-d 
wrote:
> On 04/03/2018 05:53 PM, bachmeier wrote:
> > On Tuesday, 3 April 2018 at 21:17:35 UTC, Rubn wrote:
> >> I feel that's probably the case for any comparisons across two
> >> languages, you are going to have a person that is more knowledgeable
> >> in one language than another. Mistakes are going to be made, but I
> >> think it should be blatantly obvious that one language is going to
> >> compiler slower if it is compiling all the unittests for a library
> >> compared to one that isn't. That's just blatant bias against D, not a
> >> mistake from misunderstanding Go.
> >
> > Yeah, I don't understand that either. Unit tests can be arbitrarily
> > large, so no matter how fast the compiler, it would always be possible
> > to make it take longer than any other language.
>
> Exactly, which is why I'm insisting this - and not compiler
> benchmarking, let alone idle chattaroo in the forums - is where we need
> to hit. What we have here, ladies and gentlemen, is a high-impact
> preapproved item of great general interest. Shall we start the auction?

Steven already said somewhere in here that he's planning to work on this
during the dconf hackathon. So, at least one person has stepped forward and
said that they'll look into it.

- Jonathan M Davis

Re: D compiles fast, right? Right??

[Andrei Alexandrescu via Digitalmars-d]

On 04/03/2018 05:53 PM, bachmeier wrote:

On Tuesday, 3 April 2018 at 21:17:35 UTC, Rubn wrote:
I feel that's probably the case for any comparisons across two 
languages, you are going to have a person that is more knowledgeable 
in one language than another. Mistakes are going to be made, but I 
think it should be blatantly obvious that one language is going to 
compiler slower if it is compiling all the unittests for a library 
compared to one that isn't. That's just blatant bias against D, not a 
mistake from misunderstanding Go.


Yeah, I don't understand that either. Unit tests can be arbitrarily 
large, so no matter how fast the compiler, it would always be possible 
to make it take longer than any other language.


Exactly, which is why I'm insisting this - and not compiler 
benchmarking, let alone idle chattaroo in the forums - is where we need 
to hit. What we have here, ladies and gentlemen, is a high-impact 
preapproved item of great general interest. Shall we start the auction?

Re: D compiles fast, right? Right??

[Jonathan Marler via Digitalmars-d]

On Tuesday, 3 April 2018 at 23:29:34 UTC, Atila Neves wrote:

On Tuesday, 3 April 2018 at 19:07:54 UTC, Jonathan Marler wrote:

On Tuesday, 3 April 2018 at 10:24:15 UTC, Atila Neves wrote:
On Monday, 2 April 2018 at 18:52:14 UTC, Jonathan Marler 
wrote:






You still missed my point.


I got your point. I'm disagreeing.



I don't know why you keep saying you are "disagreeing" with me.  
It looks like we agree.  You're example is showing that DLANG's 
std.path compiles slower than GO's path library.  I agree.  All I 
was saying is that this example doesn't show that GO code 
compiles faster than D.  That was my one and only point.

Re: D compiles fast, right? Right??

[Jonathan M Davis via Digitalmars-d]

On Tuesday, April 03, 2018 16:15:35 H. S. Teoh via Digitalmars-d wrote:
> On Tue, Apr 03, 2018 at 10:59:13PM +, Atila Neves via Digitalmars-d 
wrote:
> > On Tuesday, 3 April 2018 at 20:47:48 UTC, Kagamin wrote:
> > > On Friday, 30 March 2018 at 16:12:44 UTC, Atila Neves wrote:
> > > > * Building a whole project in C++ still takes a lot longer since D
> > > > scales much better, but that's not my typical worflow, nor should
> > > > it be anyone else's.
> > >
> > > I can write code for days without even saving :)
> >
> > That sentence might as well be fingernails on a blackboard for me! I
> > save compulsively. Whenever I stop typing, C-x C-s it is for me.
>
> [...]
>
> Yeah, after having lost valuable work more often than I'd care to admit
> due to unexpected power outages and other hardware problems, I've
> developed a compulsive twitch in my fingers that automatically saves my
> file every other minute.  It has become so automatic that it's basically
> unconscious now.  I can't imagine not saving for long periods of time.

LOL. I save probably closer to every ten seconds, though I'm not sure
exactly how often it is because of how automatic it is. It's highly
dependent on when I finish a set of commands in vim, so with code, I tend to
save _very_ frequently, whereas with text, it's a lot less frequent, because
there, I often stay in insert mode for while. In addition to just generally
being a good idea, it eliminates the need for vim's swap files, and I hate
it when programs create files like that (be it to restore what you didn't
save when the program crashes or as a backup file every time you save). The
file is what is on disk, and I don't want the program trying to keep track
of unsaved data or old data and make it easy for me to restore it when my
computer crashes. That's what saving is for, and if I want older versions,
I'll use source control. But I guess that there are folks who do a lot of
typing without saving. I sure couldn't though.

Now, it is true that I frequently write a lot of code before bothering to
compile anything - not days worth at a time though.

- Jonathan M Davis

Re: D compiles fast, right? Right??

[Marco Leise via Digitalmars-d]

Am Tue, 3 Apr 2018 16:15:35 -0700
schrieb "H. S. Teoh" :

> On Tue, Apr 03, 2018 at 10:59:13PM +, Atila Neves via Digitalmars-d wrote:
> > That sentence might as well be fingernails on a blackboard for me! I
> > save compulsively. Whenever I stop typing, C-x C-s it is for me.  
> 
> […] I can't imagine not saving for long periods of time.

Me too, but for me it was Delphi 2005/2006 with frequent IDE
crashes that had me start doing this and dmd eating into swap
memory when I write messy CTFE code that keeps me doing
Ctrl+S before before any action that has any likelihood to
stall the system. I also don't trust my own code.

-- 
Marco

Re: D compiles fast, right? Right??

[H. S. Teoh via Digitalmars-d]

On Tue, Apr 03, 2018 at 11:09:10PM +, burjui via Digitalmars-d wrote:
[...]
> H. S. Teoh is not the only one here cringing at "fast code fast" on
> the main page. I use D from time to time for over 10 years now, and
> even used it at work and it was a relatively positive experience,
> thanks to vibe.d. But compilation times are just horrible - minimum 3
> seconds for a 1500 lines project (on a 8-core 4GHz CPU with 16 GB
> RAM), and that's after I ditched std.regex, made all imports qualified
> (didn't help that much, though) and switched to ld.gold.

3 seconds for 1500 lines?  Is that because you're using dub with its
unbearably slow compulsive network lookups?  Or because you imported
monsters like std.format?  Or because you have heavily-templated code?
Vibe.d is pretty heavy on templates (Diet templates, while pretty cool
from a geekiness POV, also slow things down like a hog because of heavy
template + CTFE usage).

Also, which compiler version did you use?  If you use nested templates
heavily (like UFCS chains of ranges), in older releases you may have run
into the exponential symbol size problem, where most of the compilation
time is spent generating, looking up, and reading symbols that are tens
of megabytes long.  After Rainers' symbol compression PR was merged,
compilation times on such code was hugely improved. As of a release or
two ago, this is no longer a problem.

I've noticed that if I don't use certain slow things, dmd is actually
lightning fast at compiling up to several thousand LOCs.  But as soon as
std.format enters the picture, or if you have recursive templates or
heavy CTFE, it rapidly deteriorates.  (Note, though, that relatively
simple template code doesn't significantly slow things down; it's when
you start doing things like manipulating type tuples AKA AliasSeq's,
recursive templates, compile-time loops like static foreach over
introspection, heavy CTFE, that things start grinding. Simple
parametrized types are still lightning fast, and last time I checked a
lot faster than, say, equivalent C++ code compiled with g++.)


> And I would be ok with slow compilation if DMD was smart enough, doing
> some graph magic, like extensive control flow analysis, and insane
> optimizations, but it doesn't. For example, Rust compilation times are
> no picnic either, but it's obvious why - you get nice good-looking
> error messages, tons of useful warnings and very fast programs free of
> memory corruption bugs. It's not the case with DMD, though. The
> language may be better than C++, but it's fastest compiler is slower
> and produces worse code? I'd rather not boast about speed at the main
> page in this situation.  And god save us from ridicule by Goers.

These days, I don't even look at benchmarks of dmd-compiled code
anymore.  For anything even remotely performance-related, I look at
gdc/ldc.  They do compile noticeably slower than dmd, but with the huge
benefit of far superior backends that gives me top runtime performance.

If Andrei & Walter are serious about this "fast code fast" thing, then
we'd better get our act together and do some serious optimization work,
both on the compiler itself, and on the quality of its codegen.


T

-- 
Bomb technician: If I'm running, try to keep up.

Re: D compiles fast, right? Right??

[Atila Neves via Digitalmars-d]

On Tuesday, 3 April 2018 at 19:07:54 UTC, Jonathan Marler wrote:

On Tuesday, 3 April 2018 at 10:24:15 UTC, Atila Neves wrote:

On Monday, 2 April 2018 at 18:52:14 UTC, Jonathan Marler wrote:





You still missed my point.


I got your point. I'm disagreeing.


You're post was saying that "D does not compile as fast as GO".


Please show me where in my post where you think I said that.


But the libraries you're comparing are vastly different.


Their sizes are different. I disagree that they're vastly 
different.


If you're post was saying, "dlang's std.path compiles much 
slower than GO's" then you would be fine.


That is exactly what I said.

However, you're post was misleading saying the Go compile's 
faster than D in general,


I never said that.

and I was pointing out that the use case you provided doesn't 
apply in the general case,


Maybe it applies in the general case, maybe it doesn't. I have no 
idea.


it only applies to a library with the same name/type of 
functionality.


I don't know about “only".


You're totally misunderstanding me.  I was just saying that if 
you want to compare the compile speed of D vs GO (IN THE 
GENERAL CASE), you should not include the unittests in D's 
performance because you weren't including them in your GO 
example.


Include what? The Go standard library's own tests? libstdc++'s?

All the code I compiled was in that post. The only reason the Go 
file isn't just a one liner is because the silly opinionated 
language won't let me.


I showed how long it takes to compile the minimum amount of code 
necessary to import the part of the standard library responsible 
for paths in 3 languages. Then I showed how much slower it got in 
D with -unittest on the exact same one liner.


There isn't an equivalent in Go or C++. And yet one can write 
tests in them. And when one does, the compile-time penalty is 0.


What I am arguing against is that your example is not evidence 
that GO compiles faster than D in general.


I have no idea why you're arguing against something I never 
stated.


You're example is comparing 2 different libraries in 2 
different languages, not about the languages themselves.


No, I compared importing path functionality in files that did 
nothing else (except for some dummy code in Go) in *3* different 
languages. Then I showed that compiling the one liner in D with 
-unittest was slower than C++ by just a bit and nearly 50 slower 
than Go. With no actual tests in sight.

Re: D compiles fast, right? Right??

[H. S. Teoh via Digitalmars-d]

On Tue, Apr 03, 2018 at 10:59:13PM +, Atila Neves via Digitalmars-d wrote:
> On Tuesday, 3 April 2018 at 20:47:48 UTC, Kagamin wrote:
> > On Friday, 30 March 2018 at 16:12:44 UTC, Atila Neves wrote:
> > > * Building a whole project in C++ still takes a lot longer since D
> > > scales much better, but that's not my typical worflow, nor should
> > > it be anyone else's.
> > 
> > I can write code for days without even saving :)
> 
> That sentence might as well be fingernails on a blackboard for me! I
> save compulsively. Whenever I stop typing, C-x C-s it is for me.
[...]

Yeah, after having lost valuable work more often than I'd care to admit
due to unexpected power outages and other hardware problems, I've
developed a compulsive twitch in my fingers that automatically saves my
file every other minute.  It has become so automatic that it's basically
unconscious now.  I can't imagine not saving for long periods of time.


T

-- 
All men are mortal. Socrates is mortal. Therefore all men are Socrates.

Re: D compiles fast, right? Right??

[burjui via Digitalmars-d]

Atila laid it out pretty clear: he doesn't care about the 
differences, he wants the work to be done. And I'm with him on 
that. Go and it's standard library may be way simpler, but it 
get's the job done (which is trivial in both cases, by the way) 
almost instantaneously, which is a much bigger deal than it seems 
to be. When your edit-compile cycle is that fast, it changes the 
way you write code, you develop a habit of writing smaller pieces 
of code and testing them more frequently. Remember that Linus 
Torvalds' talk about Git at Google?

https://www.youtube.com/watch?v=4XpnKHJAok8=3025

H. S. Teoh is not the only one here cringing at "fast code fast" 
on the main page. I use D from time to time for over 10 years 
now, and even used it at work and it was a relatively positive 
experience, thanks to vibe.d. But compilation times are just 
horrible - minimum 3 seconds for a 1500 lines project (on a 
8-core 4GHz CPU with 16 GB RAM), and that's after I ditched 
std.regex, made all imports qualified (didn't help that much, 
though) and switched to ld.gold. And I would be ok with slow 
compilation if DMD was smart enough, doing some graph magic, like 
extensive control flow analysis, and insane optimizations, but it 
doesn't. For example, Rust compilation times are no picnic 
either, but it's obvious why - you get nice good-looking error 
messages, tons of useful warnings and very fast programs free of 
memory corruption bugs. It's not the case with DMD, though. The 
language may be better than C++, but it's fastest compiler is 
slower and produces worse code? I'd rather not boast about speed 
at the main page in this situation. And god save us from ridicule 
by Goers.

Re: D compiles fast, right? Right??

[Atila Neves via Digitalmars-d]

On Tuesday, 3 April 2018 at 21:53:35 UTC, bachmeier wrote:

On Tuesday, 3 April 2018 at 21:17:35 UTC, Rubn wrote:
I feel that's probably the case for any comparisons across two 
languages, you are going to have a person that is more 
knowledgeable in one language than another. Mistakes are going 
to be made, but I think it should be blatantly obvious that 
one language is going to compiler slower if it is compiling 
all the unittests for a library compared to one that isn't. 
That's just blatant bias against D, not a mistake from 
misunderstanding Go.


Yeah, I don't understand that either. Unit tests can be 
arbitrarily large, so no matter how fast the compiler, it would 
always be possible to make it take longer than any other 
language.


There were 0 tests in my example.

Re: D compiles fast, right? Right??

[Atila Neves via Digitalmars-d]

On Tuesday, 3 April 2018 at 20:47:48 UTC, Kagamin wrote:

On Friday, 30 March 2018 at 16:12:44 UTC, Atila Neves wrote:
* Building a whole project in C++ still takes a lot longer 
since D scales much better, but that's not my typical worflow, 
nor should it be anyone else's.


I can write code for days without even saving :)


That sentence might as well be fingernails on a blackboard for 
me! I save compulsively. Whenever I stop typing, C-x C-s it is 
for me.



What's the point to compile code that's not supposed to work?


TDD.

Re: D compiles fast, right? Right??

[bachmeier via Digitalmars-d]

On Tuesday, 3 April 2018 at 21:17:35 UTC, Rubn wrote:
I feel that's probably the case for any comparisons across two 
languages, you are going to have a person that is more 
knowledgeable in one language than another. Mistakes are going 
to be made, but I think it should be blatantly obvious that one 
language is going to compiler slower if it is compiling all the 
unittests for a library compared to one that isn't. That's just 
blatant bias against D, not a mistake from misunderstanding Go.


Yeah, I don't understand that either. Unit tests can be 
arbitrarily large, so no matter how fast the compiler, it would 
always be possible to make it take longer than any other language.

Re: D compiles fast, right? Right??

[Rubn via Digitalmars-d]

On Tuesday, 3 April 2018 at 19:07:54 UTC, Jonathan Marler wrote:

On Tuesday, 3 April 2018 at 10:24:15 UTC, Atila Neves wrote:

On Monday, 2 April 2018 at 18:52:14 UTC, Jonathan Marler wrote:
My point was that if you want to compare "compile-time" 
performance, you should not include the unittests in D's time 
since Go does not include unittests.


"Go does not include unittests"? Under some interpretations I 
guess that could be viewed as correct, but in practical terms 
I can write Go tests without an external library 
(https://golang.org/pkg/testing/)/ Whether it's a language 
keyword or not is irrelevant.


What _is_ relevant (to me) is that I can write Go code that 
manipulates paths and test it with everything building in less 
time that it takes to render a frame in a videogame, whereas 
in D...


You're totally misunderstanding me.  I was just saying that if 
you want to compare the compile speed of D vs GO (IN THE 
GENERAL CASE), you should not include the unittests in D's 
performance because you weren't including them in your GO 
example.


I feel that's probably the case for any comparisons across two 
languages, you are going to have a person that is more 
knowledgeable in one language than another. Mistakes are going to 
be made, but I think it should be blatantly obvious that one 
language is going to compiler slower if it is compiling all the 
unittests for a library compared to one that isn't. That's just 
blatant bias against D, not a mistake from misunderstanding Go.

Re: D compiles fast, right? Right??

[Kagamin via Digitalmars-d]

On Friday, 30 March 2018 at 16:12:44 UTC, Atila Neves wrote:
* Building a whole project in C++ still takes a lot longer 
since D scales much better, but that's not my typical worflow, 
nor should it be anyone else's.


I can write code for days without even saving :)
What's the point to compile code that's not supposed to work?

Re: D compiles fast, right? Right??

[Jonathan Marler via Digitalmars-d]

On Tuesday, 3 April 2018 at 10:24:15 UTC, Atila Neves wrote:

On Monday, 2 April 2018 at 18:52:14 UTC, Jonathan Marler wrote:

My point was that GO's path library is very different from 
dlang's std.path library.  It has an order of magnitude less 
code so the point was that you're comparing a very small 
library with much less functionality to a very large one.


I understood your point. I'm not sure you understood mine, 
which is: I don't care. I want to get work done, and I don't 
want to wait for the computer.


You still missed my point.  You're post was saying that "D does 
not compile as fast as GO". But the libraries you're comparing 
are vastly different.  If you're post was saying, "dlang's 
std.path compiles much slower than GO's" then you would be fine.  
However, you're post was misleading saying the Go compile's 
faster than D in general, and I was pointing out that the use 
case you provided doesn't apply in the general case, it only 
applies to a library with the same name/type of functionality.





I didn't say anything about whether it was advantageous, the 
point was that it's more code so you should take that into 
account when you evaluate performance.  You're post was 
misleading because it was assuming that both libraries were 
comparable when in reality they appear to be very different.


I disagree. They're very similar in the sense that, if I want 
to build a path and want to rely on the standard library, it 
takes vastly different amounts of time to compile my code in 
one situation vs the other.




I refer to my previous answer.  Your example shows that dlang's 
std.path compiles slower than GO's, but that doesn't say anything 
about the compile performance for both languages in the general 
case.  To make such a claim you should compare the exact same 
"functionality" implemented in both languages.





My point was that if you want to compare "compile-time" 
performance, you should not include the unittests in D's time 
since Go does not include unittests.


"Go does not include unittests"? Under some interpretations I 
guess that could be viewed as correct, but in practical terms I 
can write Go tests without an external library 
(https://golang.org/pkg/testing/)/ Whether it's a language 
keyword or not is irrelevant.


What _is_ relevant (to me) is that I can write Go code that 
manipulates paths and test it with everything building in less 
time that it takes to render a frame in a videogame, whereas in 
D...


You're totally misunderstanding me.  I was just saying that if 
you want to compare the compile speed of D vs GO (IN THE GENERAL 
CASE), you should not include the unittests in D's performance 
because you weren't including them in your GO example.


This is a problem that should be fixed but still doesn't 
change the fact that not taking this into consideration would 
be an unfair comparison.


No, no, no, a thousand times more no.

We can't make a marketing point of D compiling so fast it might 
as well be a scripting language when it's not even true. I get 
a better edit-compile-test cycle in *C++*, which is embarassing.


Atila


You totally misunderstood what I was saying once again.  I agree 
with what you said here, but it has nothing to do with what I was 
saying.


If your point is that it takes too long to access std.path's 
functionality then I completely agree. What I am arguing against 
is that your example is not evidence that GO compiles faster than 
D in general. You're example is comparing 2 different libraries 
in 2 different languages, not about the languages themselves.

Re: D compiles fast, right? Right??

[H. S. Teoh via Digitalmars-d]

On Tue, Apr 03, 2018 at 10:24:15AM +, Atila Neves via Digitalmars-d wrote:
[...]
> We can't make a marketing point of D compiling so fast it might as
> well be a scripting language when it's not even true. I get a better
> edit-compile-test cycle in *C++*, which is embarassing.
[...]

+1.  And this is why our "fast code fast" slogan makes me cringe.  We
need to do some serious work on delivering that "fast" promise.


T

-- 
Caffeine underflow. Brain dumped.

Re: D compiles fast, right? Right??

[Stefan Koch via Digitalmars-d]

On Sunday, 1 April 2018 at 02:40:26 UTC, Walter Bright wrote:

On 3/30/2018 1:17 PM, Andrei Alexandrescu wrote:

Could be faster.


It's been a fair amount of time since somebody has done 
profiling of dmd. It needs to be done. There's probably plenty 
of low hanging fruit. Speculating about why it is slow is 
pointless without data.


Well Surprisingly enough the biggest function in a profile for 
sane code, (no templates) is called spellerY.

Re: D compiles fast, right? Right??

[Atila Neves via Digitalmars-d]

On Monday, 2 April 2018 at 18:52:14 UTC, Jonathan Marler wrote:

My point was that GO's path library is very different from 
dlang's std.path library.  It has an order of magnitude less 
code so the point was that you're comparing a very small 
library with much less functionality to a very large one.


I understood your point. I'm not sure you understood mine, which 
is: I don't care. I want to get work done, and I don't want to 
wait for the computer.


I didn't say anything about whether it was advantageous, the 
point was that it's more code so you should take that into 
account when you evaluate performance.  You're post was 
misleading because it was assuming that both libraries were 
comparable when in reality they appear to be very different.


I disagree. They're very similar in the sense that, if I want to 
build a path and want to rely on the standard library, it takes 
vastly different amounts of time to compile my code in one 
situation vs the other.



My point was that if you want to compare "compile-time" 
performance, you should not include the unittests in D's time 
since Go does not include unittests.


"Go does not include unittests"? Under some interpretations I 
guess that could be viewed as correct, but in practical terms I 
can write Go tests without an external library 
(https://golang.org/pkg/testing/)/ Whether it's a language 
keyword or not is irrelevant.


What _is_ relevant (to me) is that I can write Go code that 
manipulates paths and test it with everything building in less 
time that it takes to render a frame in a videogame, whereas in 
D...


In practicality, D should not be compiling in the standard 
library unittest by default.


I think everyone is in agreement here.

This is a problem that should be fixed but still doesn't change 
the fact that not taking this into consideration would be an 
unfair comparison.


No, no, no, a thousand times more no.

We can't make a marketing point of D compiling so fast it might 
as well be a scripting language when it's not even true. I get a 
better edit-compile-test cycle in *C++*, which is embarassing.


Atila

Re: D compiles fast, right? Right??

[Jonathan M Davis via Digitalmars-d]

On Monday, April 02, 2018 17:15:40 Steven Schveighoffer via Digitalmars-d 
wrote:
> On 4/2/18 4:05 PM, Jonathan M Davis wrote:
> > One concern I have is version(unittest) blocks. In order to avoid code
> > breakage, those would need to still be compiled in. I know that I've
> > personally used version(unittest) blocks that had package access level
> > and were then imported in the unit tests within that package in order
> > to avoid declaring helper types or functions in each module. So,
> > treating version(unittest) blocks like they're not there when importing
> > a module would definitely break som existing code. It could be argued
> > that such uses should be deprecated in some manner, but simply not
> > compiling in anything related to -unittest in imported modules would be
> > a problem. However, aside from maybe __traits(getUnitTests), I don't
> > see why it would be a problem to ignore code inside a unittest block
> > except for when that module is being compiled (or when that unittest
> > block is inside a template that's being compiled).
>
> version(unittest) blocks would have to be rethought in your code at some
> point. I think they have to go through the same rules -- only if
> __traits(getUnitTests) is used would it be included.
>
> I have run into horrible problems with version(unittest), where I
> actually defined a virtual function (by accident, of course) for
> unittests only, and then when you compile/link with different modes of
> -unittest you have spectacular failure. There have been similar problems
> in phobos: https://github.com/dlang/phobos/pull/5932
>
> There are other ways to achieve the same things, and while not as
> convenient, they would be safer and less troublesome for imports.

I do think that there's some value in being able to do version(unittest) to
code and import it, but I can certainly live with losing that ability if
it's really needed to fix the situation with unit tests being compiled in
when they shouldn't. Regardless, my main point was that there is existing
code which relies on the current behavior. So, if we want to change that
behavior without just pulling the rug out from under existing code, we're
going to need some sort of deprecation process for it, whereas I would think
that it would be possible to stop compiling unittest blocks when importing
modules without breaking any code.

- Jonathan M Davis

Re: D compiles fast, right? Right??

[Seb via Digitalmars-d]

On Monday, 2 April 2018 at 19:32:33 UTC, Steven Schveighoffer 
wrote:

On 4/2/18 12:28 PM, Andrei Alexandrescu wrote:

On 04/02/2018 12:22 PM, H. S. Teoh wrote:
Lately this has been mentioned more and more frequently.  So 
what's the
status on this?  Are we going to move forward with making it 
so that
-unittest only applies to modules supplied on the 
command-line?


Has there been an investigation into how 
__traits(getUnitTests) could be

made to work in spite of this change?


Walter and I are willing to go with the change assuming no 
showstopper presents itself, but we don't have time budgeted 
for it. So it needs a strong champion.


If nobody else volunteers, I might give it a shot at the 
hackathon. I'd need help figuring out where the important bits 
are, so I may be bugging compiler devs that day :)


-Steve


Wow. I think we better get an extra hackathon day then :)
(there's a lot more work just "like this" lying around)

Re: D compiles fast, right? Right??

[Steven Schveighoffer via Digitalmars-d]

On 4/2/18 4:05 PM, Jonathan M Davis wrote:


One concern I have is version(unittest) blocks. In order to avoid code
breakage, those would need to still be compiled in. I know that I've
personally used version(unittest) blocks that had package access level and
were then imported in the unit tests within that package in order to avoid
declaring helper types or functions in each module. So, treating
version(unittest) blocks like they're not there when importing a module
would definitely break som existing code. It could be argued that such uses
should be deprecated in some manner, but simply not compiling in anything
related to -unittest in imported modules would be a problem. However, aside
from maybe __traits(getUnitTests), I don't see why it would be a problem to
ignore code inside a unittest block except for when that module is being
compiled (or when that unittest block is inside a template that's being
compiled).


version(unittest) blocks would have to be rethought in your code at some 
point. I think they have to go through the same rules -- only if 
__traits(getUnitTests) is used would it be included.


I have run into horrible problems with version(unittest), where I 
actually defined a virtual function (by accident, of course) for 
unittests only, and then when you compile/link with different modes of 
-unittest you have spectacular failure. There have been similar problems 
in phobos: https://github.com/dlang/phobos/pull/5932


There are other ways to achieve the same things, and while not as 
convenient, they would be safer and less troublesome for imports.


-Steve

Re: D compiles fast, right? Right??

[Steven Schveighoffer via Digitalmars-d]

On 4/2/18 3:55 PM, H. S. Teoh wrote:

On Mon, Apr 02, 2018 at 03:28:02PM -0400, Steven Schveighoffer via 
Digitalmars-d wrote:
[...]

We really need to change the unittest import strategy.

[...]

I think this has been established beyond reasonable doubt for the last
little while.  What about we start hashing out a solution?

AFAIK, the current proposal is to make it so that `-unittest` only takes
effect on modules that are being compiled (i.e., specified on the
command-line).  Imported modules will *not* have unittests compiled,
unless they have also been specified on the command-line.

The only thing blocking this proposal that I'm aware of, is that it will
break __traits(getUnitTests).  But I'm not sure if that's actually a
serious problem at all.  What if we make it so that a unittest block in
an imported module that isn't specified on the command-line is basically
treated as a template? I.e., parse the AST, but don't do anything with
it.  Don't bother running semantic, don't bother resolving identifiers
(in particular, import statements inside the unittest), etc.. Just leave
it in the AST as essentially translated syntax.  Then if
__traits(getUnitTests) is ever invoked, run semantic on the unittests in
the targeted module.

I'm not 100% certain, but I have a suspicion that this will mitigate the
breakage to __traits(getUnitTests) without compromising on the "don't
compile unittests outside the current list of modules to compile" fix.


Yes, I think this approach (at least, parsing unittests but deferring 
semantic until invoked) is the correct way to avoid breakage. It may be 
easy enough to do this without "treating it as a template", but I'm not 
sure.


-Steve

Re: D compiles fast, right? Right??

[Jonathan M Davis via Digitalmars-d]

On Monday, April 02, 2018 12:55:05 H. S. Teoh via Digitalmars-d wrote:
> On Mon, Apr 02, 2018 at 03:28:02PM -0400, Steven Schveighoffer via
> Digitalmars-d wrote: [...]
>
> > We really need to change the unittest import strategy.
>
> [...]
>
> I think this has been established beyond reasonable doubt for the last
> little while.  What about we start hashing out a solution?
>
> AFAIK, the current proposal is to make it so that `-unittest` only takes
> effect on modules that are being compiled (i.e., specified on the
> command-line).  Imported modules will *not* have unittests compiled,
> unless they have also been specified on the command-line.
>
> The only thing blocking this proposal that I'm aware of, is that it will
> break __traits(getUnitTests).  But I'm not sure if that's actually a
> serious problem at all.  What if we make it so that a unittest block in
> an imported module that isn't specified on the command-line is basically
> treated as a template? I.e., parse the AST, but don't do anything with
> it.  Don't bother running semantic, don't bother resolving identifiers
> (in particular, import statements inside the unittest), etc.. Just leave
> it in the AST as essentially translated syntax.  Then if
> __traits(getUnitTests) is ever invoked, run semantic on the unittests in
> the targeted module.
>
> I'm not 100% certain, but I have a suspicion that this will mitigate the
> breakage to __traits(getUnitTests) without compromising on the "don't
> compile unittests outside the current list of modules to compile" fix.

Having never used __traits(getUnitTest), I'm not very familiar with it, but
depending on what it does, it might be enough to register the fact that a
unittest block exists in the file, and then the unittest block itself only
needs to be analyzed enough to parse passed it. But I don't know. I'll have
to study up on what it does exactly to say much intelligent about it.

One concern I have is version(unittest) blocks. In order to avoid code
breakage, those would need to still be compiled in. I know that I've
personally used version(unittest) blocks that had package access level and
were then imported in the unit tests within that package in order to avoid
declaring helper types or functions in each module. So, treating
version(unittest) blocks like they're not there when importing a module
would definitely break som existing code. It could be argued that such uses
should be deprecated in some manner, but simply not compiling in anything
related to -unittest in imported modules would be a problem. However, aside
from maybe __traits(getUnitTests), I don't see why it would be a problem to
ignore code inside a unittest block except for when that module is being
compiled (or when that unittest block is inside a template that's being
compiled).

- Jonathan M Davis

Re: D compiles fast, right? Right??

[H. S. Teoh via Digitalmars-d]

On Mon, Apr 02, 2018 at 03:28:02PM -0400, Steven Schveighoffer via 
Digitalmars-d wrote:
[...]
> We really need to change the unittest import strategy.
[...]

I think this has been established beyond reasonable doubt for the last
little while.  What about we start hashing out a solution?

AFAIK, the current proposal is to make it so that `-unittest` only takes
effect on modules that are being compiled (i.e., specified on the
command-line).  Imported modules will *not* have unittests compiled,
unless they have also been specified on the command-line.

The only thing blocking this proposal that I'm aware of, is that it will
break __traits(getUnitTests).  But I'm not sure if that's actually a
serious problem at all.  What if we make it so that a unittest block in
an imported module that isn't specified on the command-line is basically
treated as a template? I.e., parse the AST, but don't do anything with
it.  Don't bother running semantic, don't bother resolving identifiers
(in particular, import statements inside the unittest), etc.. Just leave
it in the AST as essentially translated syntax.  Then if
__traits(getUnitTests) is ever invoked, run semantic on the unittests in
the targeted module.

I'm not 100% certain, but I have a suspicion that this will mitigate the
breakage to __traits(getUnitTests) without compromising on the "don't
compile unittests outside the current list of modules to compile" fix.


T

-- 
Questions are the beginning of intelligence, but the fear of God is the 
beginning of wisdom.

Re: D compiles fast, right? Right??

[Steven Schveighoffer via Digitalmars-d]

On 4/2/18 12:28 PM, Andrei Alexandrescu wrote:

On 04/02/2018 12:22 PM, H. S. Teoh wrote:

Lately this has been mentioned more and more frequently.  So what's the
status on this?  Are we going to move forward with making it so that
-unittest only applies to modules supplied on the command-line?

Has there been an investigation into how __traits(getUnitTests) could be
made to work in spite of this change?


Walter and I are willing to go with the change assuming no showstopper 
presents itself, but we don't have time budgeted for it. So it needs a 
strong champion.


If nobody else volunteers, I might give it a shot at the hackathon. I'd 
need help figuring out where the important bits are, so I may be bugging 
compiler devs that day :)


-Steve

Re: D compiles fast, right? Right??

[Steven Schveighoffer via Digitalmars-d]

On 4/2/18 8:33 AM, Atila Neves wrote:

No... that's ~11.583x with no unittests and ~43.75x with. The former 
number not being interesting to me in the slightest.


Here is an interesting tidbit as well -- importing with -unittest takes 
4x as long AND NO CODE IS GENERATED OR WILL EVER BE GENERATED. 
Literally, the compiler is parsing, semantically analyzing, consuming 
300% more cycles just to "compile" code it will never actually use or 
generate files for, and as far as I know, it has no way of doing so.


I did a simple test:

dmd -c -v testpath.d | grep import | wc -l
60
dmd -c -v -unittest testpath.d | grep import | wc -l
87

So there are 27 more modules processed by DMD that make this 300% 
increase happen (Not to mention, ~60 imports for building paths...). 
Doing some bash-script work, I get the following differences


> importcore.atomic
> importcore.bitop
> importcore.checkedint
> importcore.math
> importcore.stdc.fenv
> importcore.stdc.math
> importcore.sync.exception
> importcore.sync.mutex
> importcore.sys.darwin.mach.port
> importcore.sys.darwin.mach.thread_act
> importcore.sys.darwin.pthread
> importcore.sys.posix.pthread
> importcore.sys.posix.sched
> importcore.sys.posix.semaphore
> importcore.sys.posix.stdlib
> importcore.sys.posix.sys.wait
> importcore.thread
> importstd.algorithm
> importstd.algorithm.internal
> importstd.algorithm.setops
> importstd.algorithm.sorting
> importstd.bitmanip
> importstd.math
> importstd.process
> importstd.random
> importstd.system
> importstd.utf

One disconcerting thing here is, not all of these imports come from 
std.path. Some of them come from unit tests in modules imported by 
std.path unittests. Because inside a unit test you want to have free 
access to all of phobos to do whatever the hell you want for testing, 
it's quite easy to import ALL of phobos when you do unit tests.


We really need to change the unittest import strategy.

-Steve

Re: D compiles fast, right? Right??

[Jonathan M Davis via Digitalmars-d]

On Monday, April 02, 2018 18:52:14 Jonathan Marler via Digitalmars-d wrote:
> On Monday, 2 April 2018 at 12:33:37 UTC, Atila Neves wrote:
> > On Friday, 30 March 2018 at 16:41:42 UTC, Jonathan Marler wrote:
> >> Seems like you're comparing apples to oranges.
> >
> > No, I'm comparing one type of apple to another with regards to
> > weight in my shopping bag before I've even taken a bite.
>
> My point was that GO's path library is very different from
> dlang's std.path library.  It has an order of magnitude less code
> so the point was that you're comparing a very small library with
> much less functionality to a very large one.
>
> >> It's over an order of magnitude more code
> >
> > More lines of code is a liability, not an advantage.
>
> I didn't say anything about whether it was advantageous, the
> point was that it's more code so you should take that into
> account when you evaluate performance.  You're post was
> misleading because it was assuming that both libraries were
> comparable when in reality they appear to be very different.
>
> >> and it's only fair to compare the "non-unittest" version of
> >> std.path with Go, since Go does not include unittests.
> >
> > Absolutely not.
> >
> > There is *0* compile-time penalty on Go programmers when they
> > test their programs, whereas my compile times go up by a factor
> > of 3 on a one-line program. And that's >3 multiplied by
> > "already slow to begin with".
>
> My point was that if you want to compare "compile-time"
> performance, you should not include the unittests in D's time
> since Go does not include unittests.  In practicality, D should
> not be compiling in the standard library unittest by default.
> This is a problem that should be fixed but still doesn't change
> the fact that not taking this into consideration would be an
> unfair comparison.

You both have good points. On the one hand, yes, std.path is doing more, so
it's not surprising that it takes longer to compile, and in that sense, it's
comparing apples and oranges. However, from the standpoint of the user,
they're just calling these functions to get something done, and the
implementation details don't really matter. So, from the user's standpoint,
as far as compilation time goes, std.path is just worse. The reasons why are
kind of irrelevant from that perspective. So, in a way, you're both right.

Now, ultimately, given how D works and the functionality in std.path, I
don't know that it really makes sense for it to compile as fast as Go's
solution if Go's solution is doing so much less, but regardless, we should
be trying to at least eliminate the unnecessary slowdowns for things like
compiling unit tests, and we should be looking into how to speed up the
stuff that's slow to compile (e.g. improving the compilation speed of
templates in general would be a huge boon). std.path doesn't need to be as
slow to compile as it is to do what it does.

- Jonathan M Davis

Re: D compiles fast, right? Right??

[Jonathan Marler via Digitalmars-d]

On Monday, 2 April 2018 at 12:33:37 UTC, Atila Neves wrote:

On Friday, 30 March 2018 at 16:41:42 UTC, Jonathan Marler wrote:


Seems like you're comparing apples to oranges.


No, I'm comparing one type of apple to another with regards to 
weight in my shopping bag before I've even taken a bite.


My point was that GO's path library is very different from 
dlang's std.path library.  It has an order of magnitude less code 
so the point was that you're comparing a very small library with 
much less functionality to a very large one.



It's over an order of magnitude more code


More lines of code is a liability, not an advantage.


I didn't say anything about whether it was advantageous, the 
point was that it's more code so you should take that into 
account when you evaluate performance.  You're post was 
misleading because it was assuming that both libraries were 
comparable when in reality they appear to be very different.



and it's only fair to compare the "non-unittest" version of 
std.path with Go, since Go does not include unittests.


Absolutely not.

There is *0* compile-time penalty on Go programmers when they 
test their programs, whereas my compile times go up by a factor 
of 3 on a one-line program. And that's >3 multiplied by 
"already slow to begin with".


My point was that if you want to compare "compile-time" 
performance, you should not include the unittests in D's time 
since Go does not include unittests.  In practicality, D should 
not be compiling in the standard library unittest by default.  
This is a problem that should be fixed but still doesn't change 
the fact that not taking this into consideration would be an 
unfair comparison.

Re: D compiles fast, right? Right??

[Andrei Alexandrescu via Digitalmars-d]

On 04/02/2018 12:22 PM, H. S. Teoh wrote:

Lately this has been mentioned more and more frequently.  So what's the
status on this?  Are we going to move forward with making it so that
-unittest only applies to modules supplied on the command-line?

Has there been an investigation into how __traits(getUnitTests) could be
made to work in spite of this change?


Walter and I are willing to go with the change assuming no showstopper 
presents itself, but we don't have time budgeted for it. So it needs a 
strong champion.

Re: D compiles fast, right? Right??

[H. S. Teoh via Digitalmars-d]

On Mon, Apr 02, 2018 at 12:09:01PM -0400, Andrei Alexandrescu via Digitalmars-d 
wrote:
> On 04/02/2018 08:35 AM, Atila Neves wrote:
> > On Sunday, 1 April 2018 at 02:40:26 UTC, Walter Bright wrote:
> > > On 3/30/2018 1:17 PM, Andrei Alexandrescu wrote:
> > > > Could be faster.
> > > 
> > > It's been a fair amount of time since somebody has done profiling
> > > of dmd. It needs to be done. There's probably plenty of low
> > > hanging fruit. Speculating about why it is slow is pointless
> > > without data.
> > 
> > I profiled it as soon as I had the numbers. I didn't bother to
> > mention it because I thought I'd just work on making dmd faster
> > instead.
> > 
> > I seem to be the one who feels the most pain by this, it'd be silly
> > of me to expect anyone else to work on it.
> 
> A large and obvious time sink is that unittests in library code are
> built when user code is to be unittested. I'd recommend doing this
> before any other optimization.

Lately this has been mentioned more and more frequently.  So what's the
status on this?  Are we going to move forward with making it so that
-unittest only applies to modules supplied on the command-line?

Has there been an investigation into how __traits(getUnitTests) could be
made to work in spite of this change?


T

-- 
Indifference will certainly be the downfall of mankind, but who cares? -- 
Miquel van Smoorenburg

Re: D compiles fast, right? Right??

[Andrei Alexandrescu via Digitalmars-d]

On 04/02/2018 08:35 AM, Atila Neves wrote:

On Sunday, 1 April 2018 at 02:40:26 UTC, Walter Bright wrote:

On 3/30/2018 1:17 PM, Andrei Alexandrescu wrote:

Could be faster.


It's been a fair amount of time since somebody has done profiling of 
dmd. It needs to be done. There's probably plenty of low hanging 
fruit. Speculating about why it is slow is pointless without data.


I profiled it as soon as I had the numbers. I didn't bother to mention 
it because I thought I'd just work on making dmd faster instead.


I seem to be the one who feels the most pain by this, it'd be silly of 
me to expect anyone else to work on it.


A large and obvious time sink is that unittests in library code are 
built when user code is to be unittested. I'd recommend doing this 
before any other optimization.

Re: D compiles fast, right? Right??

[H. S. Teoh via Digitalmars-d]

On Mon, Apr 02, 2018 at 12:35:03PM +, Atila Neves via Digitalmars-d wrote:
> On Sunday, 1 April 2018 at 02:40:26 UTC, Walter Bright wrote:
> > On 3/30/2018 1:17 PM, Andrei Alexandrescu wrote:
> > > Could be faster.
> > 
> > It's been a fair amount of time since somebody has done profiling of
> > dmd. It needs to be done. There's probably plenty of low hanging
> > fruit.  Speculating about why it is slow is pointless without data.
> 
> I profiled it as soon as I had the numbers. I didn't bother to mention
> it because I thought I'd just work on making dmd faster instead.
> 
> I seem to be the one who feels the most pain by this, it'd be silly of
> me to expect anyone else to work on it.

Recently I've also started feeling the sting of slow compile times.  I
don't know why I didn't notice it before... either I was jaded by C++
compile times in the past and even D's "slower" compile times are fast
by comparison, or maybe dmd performance has degraded over time? It's a
possibility we should investigate.  Or most likely, the recent
templatization of certain parts of druntime and Phobos has exacerbated
the problem to the point that it's now very noticeable.  The recent
fiasco with __switch and `import std.format` come to mind.

While currently all the fingers seem to be pointing at templates, I have
to say that I'm a big fan of templated code, and would rather see an
improvement in how the compiler deals with templates, than a reduction
in the usage of templates. Some of D's key selling features being
templates and meta-programming, how they are implemented is pretty
important.


T

-- 
Береги платье снову, а здоровье смолоду. 

Re: D compiles fast, right? Right??

[Stefan Koch via Digitalmars-d]

On Monday, 2 April 2018 at 12:35:03 UTC, Atila Neves wrote:

On Sunday, 1 April 2018 at 02:40:26 UTC, Walter Bright wrote:

On 3/30/2018 1:17 PM, Andrei Alexandrescu wrote:

Could be faster.


It's been a fair amount of time since somebody has done 
profiling of dmd. It needs to be done. There's probably plenty 
of low hanging fruit. Speculating about why it is slow is 
pointless without data.


I profiled it as soon as I had the numbers. I didn't bother to 
mention it because I thought I'd just work on making dmd faster 
instead.


I seem to be the one who feels the most pain by this, it'd be 
silly of me to expect anyone else to work on it.


Making certain parts of the compiler faster is also on my agenda.
Feel free to talk to me anytime :)

Re: D compiles fast, right? Right??

[Seb via Digitalmars-d]

On Monday, 2 April 2018 at 12:33:37 UTC, Atila Neves wrote:
I don't think how big the files are is revelant for me, a user 
of the standard library. If I want to do something with paths 
and don't want to roll my own code, I pay a price for it in D, 
whereas it's relatively free with Go. It makes me want to 
substitute every usage of std.path.buildPath in my code with 
just `foo ~ "/" ~ bar ~ ...`.


That will also be _faster_ at runtime ;-)
std.path is a performance nightmare.
My favorite example is buildNormalizedPath:

https://github.com/dlang/phobos/blob/2e105c72a9e5fa028f31f1898ec8d479a9bae4a1/std/path.d#L1738

We ought to start to separate: the good, the bad and the ugly 
modules.

Re: D compiles fast, right? Right??

[Atila Neves via Digitalmars-d]

On Sunday, 1 April 2018 at 02:40:26 UTC, Walter Bright wrote:

On 3/30/2018 1:17 PM, Andrei Alexandrescu wrote:

Could be faster.


It's been a fair amount of time since somebody has done 
profiling of dmd. It needs to be done. There's probably plenty 
of low hanging fruit. Speculating about why it is slow is 
pointless without data.


I profiled it as soon as I had the numbers. I didn't bother to 
mention it because I thought I'd just work on making dmd faster 
instead.


I seem to be the one who feels the most pain by this, it'd be 
silly of me to expect anyone else to work on it.

Re: D compiles fast, right? Right??

[Atila Neves via Digitalmars-d]

On Friday, 30 March 2018 at 16:41:42 UTC, Jonathan Marler wrote:


Seems like you're comparing apples to oranges.


No, I'm comparing one type of apple to another with regards to 
weight in my shopping bag before I've even taken a bite.



Go's path.go is very small, a 215 line file:
https://github.com/golang/go/blob/master/src/path/path.go
Documentation: https://golang.org/pkg/path/


gocloc has it at 123 SLOC.

Dlang's std.path is much more comprehensive with 4181 lines: 
https://github.com/dlang/phobos/blob/master/std/path.d

Documentation: https://dlang.org/phobos/std_path.html


dscanner says 1857 SLOC. Also, that includes unit tests, of which 
there are 72 so probably some 700SLOC there.


I don't think how big the files are is revelant for me, a user of 
the standard library. If I want to do something with paths and 
don't want to roll my own code, I pay a price for it in D, 
whereas it's relatively free with Go. It makes me want to 
substitute every usage of std.path.buildPath in my code with just 
`foo ~ "/" ~ bar ~ ...`.



It's over an order of magnitude more code


More lines of code is a liability, not an advantage.


and only takes twice as long to compile without unittests,


No... that's ~11.583x with no unittests and ~43.75x with. The 
former number not being interesting to me in the slightest.


and it's only fair to compare the "non-unittest" version of 
std.path with Go, since Go does not include unittests.


Absolutely not.

There is *0* compile-time penalty on Go programmers when they 
test their programs, whereas my compile times go up by a factor 
of 3 on a one-line program. And that's >3 multiplied by "already 
slow to begin with".


Atila

Re: D compiles fast, right? Right??

[Stefan Koch via Digitalmars-d]

On Sunday, 1 April 2018 at 02:40:26 UTC, Walter Bright wrote:

On 3/30/2018 1:17 PM, Andrei Alexandrescu wrote:

Could be faster.


It's been a fair amount of time since somebody has done 
profiling of dmd. It needs to be done. There's probably plenty 
of low hanging fruit. Speculating about why it is slow is 
pointless without data.


have a look at https://github.com/dlang/dmd/pull/7792 for a 
little profiling utility which tells you about which parts of a 
programm draw compiletime.


Whenever I see long compile times 90% of it is due to templates.

Re: D compiles fast, right? Right??

[Walter Bright via Digitalmars-d]

On 3/30/2018 1:17 PM, Andrei Alexandrescu wrote:

Could be faster.


It's been a fair amount of time since somebody has done profiling of dmd. It 
needs to be done. There's probably plenty of low hanging fruit. Speculating 
about why it is slow is pointless without data.

Re: D compiles fast, right? Right??

[Jonathan M Davis via Digitalmars-d]

On Sunday, April 01, 2018 01:25:41 Jonathan Marler via Digitalmars-d wrote:
> On Saturday, 31 March 2018 at 21:37:13 UTC, Jonathan M Davis
>
> wrote:
> > On Saturday, March 31, 2018 08:28:31 Jonathan Marler via
> >
> > Digitalmars-d wrote:
> >> On Friday, 30 March 2018 at 20:17:39 UTC, Andrei Alexandrescu
> >>
> >> wrote:
> >> > On 3/30/18 12:12 PM, Atila Neves wrote:
> >> >> Fast code fast, they said. It'll be fun, they said. Here's
> >> >> a D
> >> >>
> >> >> file:
> >> >>  import std.path;
> >> >>
> >> >> Yep, that's all there is to it. Let's compile it on my
> >> >>
> >> >> laptop:
> >> >>  /tmp % time dmd -c  foo.d
> >> >>  dmd -c foo.d  0.12s user 0.02s system 98% cpu 0.139
> >> >>
> >> >> total
> >> >
> >> > Could be faster.
> >> >
> >> >> That... doesn't seem too fast to me. But wait, there's more:
> >> >>  /tmp % time dmd -c -unittest foo.d
> >> >>  dmd -c -unittest foo.d  0.46s user 0.06s system 99% cpu
> >> >>
> >> >> 0.525 total
> >> >
> >> > Not fast. We need to make -unittest only affect the built
> >> > module. Even though it breaks certain uses of
> >> > __traits(getUnittests). No two ways about it. Who can work
> >> > on that?
> >> >
> >> > Andrei
> >>
> >> If you approve of the -unittest= approach then
> >> timotheecour has already offered to implement this.  It's
> >> pattern matching would work the same as -i and would also use
> >> the "implied standard exclusions" that -i uses, namely,
> >>
> >> -unittest=-std -unittest=-etc -unittest=-core
> >>
> >> This would mean that by default, just passing "-unittest"
> >> would exclude druntime/phobos just like "-i" by itself also
> >> does.
> >
> > And every time you used another library, you'd have the same
> > problem and have to add -unittest=- whatever for each and every
> > one of them, or you would have to use -unittest= with
> > everything from your application or library rather than using
> > -unittest. I really don't see how that scales well, and it's
> > way too manual and too easy to screw up. It might be a decent
> > idea for a workaround, but it's not a great solution.
> >
> > IMHO, this is really something that should be handled by the
> > compiler. It simply shouldn't be compiling in the unittest
> > blocks for modules that you're not compiling directly. And if
> > that's done right, then this whole problem goes away without
> > having to make sure that every project you work on is
> > configured correctly to avoid pulling in the unit tests from
> > everything that it depends on.
> >
> > And maybe figuring out what to do about __traits(getUnittests)
> > complicates things, but it seems like the fact that we're even
> > having this problem is due to a flaw in the design of D's unit
> > tests and that that should be fixed, not worked around.
> >
> > - Jonathan M Davis
>
> Let's make this conversation a bit more concrete, I'm not sure we
> are discussing the exact same thing.
>
> The proposed solution is to have -unittest mean "compile
> unittests for all 'compiled modules' according to the pattern
> rules".  The default pattern rule is to include all modules
> except druntime/phobos.
>
> Say you have two "packages" foo and bar that contain modules
> inside their respective directories.  With the proposed
> -unittest= this is the semantics you would get.
>
> dmd -unittest   foo/*.d bar/*.d # compiles unittests for
> foo/*.d and bar/*.d
> dmd -unittest=foo   foo/*.d bar/*.d # compiles unittests for
> foo/*.d
> dmd -unittest=bar   foo/*.d bar/*.d # compiles unittests for
> bar/*.d
> dmd -unittest=foo.x foo/*.d bar/*.d # compiles unittests for
> foo/x.d
> dmd -unittest=-bar  foo/*.d bar/*.d # compiles unittests for
> foo/*.d
>
> Note that the default behavior makes sense, but this mechanism
> also allows you more fine-graned control to limit unittesting to
> certain packages or modules.  This degree of control would be
> quite helpful to me, any of the previously listed use cases
> represent valid scenarios that I would like to be able to do.  Do
> you have another solution that would provide this functionality?
> I don't see any reason not to support these use cases.

My point is that this is not a good solution for the problem of dependencies
having their unit tests included in your project. That should simply not be
happening.

Discussions of ways to tell the compiler to enable unit tests for some
modules and not others within your own project are IMHO therefore a separate
issue. Now, as for the specific proposal trying to solve the issue of
running only certain unit tests in your project, I think that it's overly
complicated once you start having to deal with stuff like exclusions. I'd
argue for simply compiling the module or modules that you care about with
-unittest and that you not use -unittest when compiling the other modules.
All of this is going to have to be controlled by whatever build system
you're using anyway, since it really doesn't scale to keep calling the
compiler 

Re: D compiles fast, right? Right??

[Jonathan Marler via Digitalmars-d]

On Saturday, 31 March 2018 at 21:37:13 UTC, Jonathan M Davis 
wrote:
On Saturday, March 31, 2018 08:28:31 Jonathan Marler via 
Digitalmars-d wrote:

On Friday, 30 March 2018 at 20:17:39 UTC, Andrei Alexandrescu

wrote:
> On 3/30/18 12:12 PM, Atila Neves wrote:
>> Fast code fast, they said. It'll be fun, they said. Here's 
>> a D

>>
>> file:
>>  import std.path;
>>
>> Yep, that's all there is to it. Let's compile it on my 
>> laptop:

>>  /tmp % time dmd -c  foo.d
>>  dmd -c foo.d  0.12s user 0.02s system 98% cpu 0.139 
>> total

>
> Could be faster.
>
>> That... doesn't seem too fast to me. But wait, there's more:
>>  /tmp % time dmd -c -unittest foo.d
>>  dmd -c -unittest foo.d  0.46s user 0.06s system 99% cpu
>>
>> 0.525 total
>
> Not fast. We need to make -unittest only affect the built 
> module. Even though it breaks certain uses of 
> __traits(getUnittests). No two ways about it. Who can work 
> on that?

>
> Andrei

If you approve of the -unittest= approach then 
timotheecour has already offered to implement this.  It's 
pattern matching would work the same as -i and would also use 
the "implied standard exclusions" that -i uses, namely,


-unittest=-std -unittest=-etc -unittest=-core

This would mean that by default, just passing "-unittest" 
would exclude druntime/phobos just like "-i" by itself also 
does.


And every time you used another library, you'd have the same 
problem and have to add -unittest=- whatever for each and every 
one of them, or you would have to use -unittest= with 
everything from your application or library rather than using 
-unittest. I really don't see how that scales well, and it's 
way too manual and too easy to screw up. It might be a decent 
idea for a workaround, but it's not a great solution.


IMHO, this is really something that should be handled by the 
compiler. It simply shouldn't be compiling in the unittest 
blocks for modules that you're not compiling directly. And if 
that's done right, then this whole problem goes away without 
having to make sure that every project you work on is 
configured correctly to avoid pulling in the unit tests from 
everything that it depends on.


And maybe figuring out what to do about __traits(getUnittests) 
complicates things, but it seems like the fact that we're even 
having this problem is due to a flaw in the design of D's unit 
tests and that that should be fixed, not worked around.


- Jonathan M Davis


Let's make this conversation a bit more concrete, I'm not sure we 
are discussing the exact same thing.


The proposed solution is to have -unittest mean "compile 
unittests for all 'compiled modules' according to the pattern 
rules".  The default pattern rule is to include all modules 
except druntime/phobos.


Say you have two "packages" foo and bar that contain modules 
inside their respective directories.  With the proposed 
-unittest= this is the semantics you would get.


dmd -unittest   foo/*.d bar/*.d # compiles unittests for 
foo/*.d and bar/*.d
dmd -unittest=foo   foo/*.d bar/*.d # compiles unittests for 
foo/*.d
dmd -unittest=bar   foo/*.d bar/*.d # compiles unittests for 
bar/*.d
dmd -unittest=foo.x foo/*.d bar/*.d # compiles unittests for 
foo/x.d
dmd -unittest=-bar  foo/*.d bar/*.d # compiles unittests for 
foo/*.d


Note that the default behavior makes sense, but this mechanism 
also allows you more fine-graned control to limit unittesting to 
certain packages or modules.  This degree of control would be 
quite helpful to me, any of the previously listed use cases 
represent valid scenarios that I would like to be able to do.  Do 
you have another solution that would provide this functionality?  
I don't see any reason not to support these use cases.

Re: D compiles fast, right? Right??

[Andrei Alexandrescu via Digitalmars-d]

On 3/31/18 5:37 PM, Jonathan M Davis wrote:

And every time you used another library, you'd have the same problem and
have to add -unittest=- whatever for each and every one of them, or you
would have to use -unittest= with everything from your application or
library rather than using -unittest. I really don't see how that scales
well, and it's way too manual and too easy to screw up. It might be a decent
idea for a workaround, but it's not a great solution.


Nod. We really need to get to the point where the application of 
unittest is modular.

Re: D compiles fast, right? Right??

[Jonathan M Davis via Digitalmars-d]

On Saturday, March 31, 2018 08:28:31 Jonathan Marler via Digitalmars-d 
wrote:
> On Friday, 30 March 2018 at 20:17:39 UTC, Andrei Alexandrescu
>
> wrote:
> > On 3/30/18 12:12 PM, Atila Neves wrote:
> >> Fast code fast, they said. It'll be fun, they said. Here's a D
> >>
> >> file:
> >>  import std.path;
> >>
> >> Yep, that's all there is to it. Let's compile it on my laptop:
> >>  /tmp % time dmd -c  foo.d
> >>  dmd -c foo.d  0.12s user 0.02s system 98% cpu 0.139 total
> >
> > Could be faster.
> >
> >> That... doesn't seem too fast to me. But wait, there's more:
> >>  /tmp % time dmd -c -unittest foo.d
> >>  dmd -c -unittest foo.d  0.46s user 0.06s system 99% cpu
> >>
> >> 0.525 total
> >
> > Not fast. We need to make -unittest only affect the built
> > module. Even though it breaks certain uses of
> > __traits(getUnittests). No two ways about it. Who can work on
> > that?
> >
> > Andrei
>
> If you approve of the -unittest= approach then
> timotheecour has already offered to implement this.  It's pattern
> matching would work the same as -i and would also use the
> "implied standard exclusions" that -i uses, namely,
>
> -unittest=-std -unittest=-etc -unittest=-core
>
> This would mean that by default, just passing "-unittest" would
> exclude druntime/phobos just like "-i" by itself also does.

And every time you used another library, you'd have the same problem and
have to add -unittest=- whatever for each and every one of them, or you
would have to use -unittest= with everything from your application or
library rather than using -unittest. I really don't see how that scales
well, and it's way too manual and too easy to screw up. It might be a decent
idea for a workaround, but it's not a great solution.

IMHO, this is really something that should be handled by the compiler. It
simply shouldn't be compiling in the unittest blocks for modules that you're
not compiling directly. And if that's done right, then this whole problem
goes away without having to make sure that every project you work on is
configured correctly to avoid pulling in the unit tests from everything that
it depends on.

And maybe figuring out what to do about __traits(getUnittests) complicates
things, but it seems like the fact that we're even having this problem is
due to a flaw in the design of D's unit tests and that that should be fixed,
not worked around.

- Jonathan M Davis

Re: D compiles fast, right? Right??

[Jonathan Marler via Digitalmars-d]

On Friday, 30 March 2018 at 20:17:39 UTC, Andrei Alexandrescu 
wrote:

On 3/30/18 12:12 PM, Atila Neves wrote:
Fast code fast, they said. It'll be fun, they said. Here's a D 
file:


     import std.path;


Yep, that's all there is to it. Let's compile it on my laptop:

     /tmp % time dmd -c  foo.d
     dmd -c foo.d  0.12s user 0.02s system 98% cpu 0.139 total


Could be faster.


That... doesn't seem too fast to me. But wait, there's more:

     /tmp % time dmd -c -unittest foo.d
     dmd -c -unittest foo.d  0.46s user 0.06s system 99% cpu 
0.525 total


Not fast. We need to make -unittest only affect the built 
module. Even though it breaks certain uses of 
__traits(getUnittests). No two ways about it. Who can work on 
that?


Andrei


If you approve of the -unittest= approach then 
timotheecour has already offered to implement this.  It's pattern 
matching would work the same as -i and would also use the 
"implied standard exclusions" that -i uses, namely,


-unittest=-std -unittest=-etc -unittest=-core

This would mean that by default, just passing "-unittest" would 
exclude druntime/phobos just like "-i" by itself also does.

Re: D compiles fast, right? Right??

[Seb via Digitalmars-d]

On Friday, 30 March 2018 at 20:40:16 UTC, Stefan Koch wrote:
On Friday, 30 March 2018 at 20:17:39 UTC, Andrei Alexandrescu 
wrote:

On 3/30/18 12:12 PM, Atila Neves wrote:
Fast code fast, they said. It'll be fun, they said. Here's a 
D file:


     import std.path;


Yep, that's all there is to it. Let's compile it on my laptop:

     /tmp % time dmd -c  foo.d
     dmd -c foo.d  0.12s user 0.02s system 98% cpu 0.139 total


Could be faster.


That... doesn't seem too fast to me. But wait, there's more:

     /tmp % time dmd -c -unittest foo.d
     dmd -c -unittest foo.d  0.46s user 0.06s system 99% cpu 
0.525 total


Not fast. We need to make -unittest only affect the built 
module. Even though it breaks certain uses of 
__traits(getUnittests). No two ways about it. Who can work on 
that?


Andrei


unittests by nature usually have a short list of dependencies 
and therefore their compilation can be parallelized.
There are many performance leaks we can fix, before we need to 
think about breaking useful features!


No one wants to run the std.regex's testsuite when they pass 
-unittest or instantiate and run all tests for all Tuple 
combinations present.
If you can make dmd so fast that this happens unnoticed, that 
would be amazing, otherwise I suggest we don't run Phobos's 
templated unittests in user-code.

Re: D compiles fast, right? Right??

[Stefan Koch via Digitalmars-d]

On Friday, 30 March 2018 at 20:17:39 UTC, Andrei Alexandrescu 
wrote:

On 3/30/18 12:12 PM, Atila Neves wrote:
Fast code fast, they said. It'll be fun, they said. Here's a D 
file:


     import std.path;


Yep, that's all there is to it. Let's compile it on my laptop:

     /tmp % time dmd -c  foo.d
     dmd -c foo.d  0.12s user 0.02s system 98% cpu 0.139 total


Could be faster.


That... doesn't seem too fast to me. But wait, there's more:

     /tmp % time dmd -c -unittest foo.d
     dmd -c -unittest foo.d  0.46s user 0.06s system 99% cpu 
0.525 total


Not fast. We need to make -unittest only affect the built 
module. Even though it breaks certain uses of 
__traits(getUnittests). No two ways about it. Who can work on 
that?


Andrei


unittests by nature usually have a short list of dependencies and 
therefore their compilation can be parallelized.
There are many performance leaks we can fix, before we need to 
think about breaking useful features!

Re: D compiles fast, right? Right??

[Seb via Digitalmars-d]

On Friday, 30 March 2018 at 16:12:44 UTC, Atila Neves wrote:
Fast code fast, they said. It'll be fun, they said. Here's a D 
file:


import std.path;


Yep, that's all there is to it. Let's compile it on my laptop:

/tmp % time dmd -c  foo.d
dmd -c foo.d  0.12s user 0.02s system 98% cpu 0.139 total


We had a recent discussion about this:

https://github.com/dlang/phobos/pull/5916

tl;dr:
- e.g. all unicode tables are probably imported and CTFE-ed 
(that's a huge chunk)
- Vladimir has written an excellent tool (-> 
https://github.com/CyberShadow/DBuildStat)

Re: D compiles fast, right? Right??

[Andrei Alexandrescu via Digitalmars-d]

On 3/30/18 12:12 PM, Atila Neves wrote:

Fast code fast, they said. It'll be fun, they said. Here's a D file:

     import std.path;


Yep, that's all there is to it. Let's compile it on my laptop:

     /tmp % time dmd -c  foo.d
     dmd -c foo.d  0.12s user 0.02s system 98% cpu 0.139 total


Could be faster.


That... doesn't seem too fast to me. But wait, there's more:

     /tmp % time dmd -c -unittest foo.d
     dmd -c -unittest foo.d  0.46s user 0.06s system 99% cpu 0.525 total


Not fast. We need to make -unittest only affect the built module. Even 
though it breaks certain uses of __traits(getUnittests). No two ways 
about it. Who can work on that?


Andrei

Re: D compiles fast, right? Right??

[bachmeier via Digitalmars-d]

On Friday, 30 March 2018 at 16:12:44 UTC, Atila Neves wrote:


/tmp % time dmd -c  foo.d
dmd -c foo.d  0.12s user 0.02s system 98% cpu 0.139 total


That... doesn't seem too fast to me. But wait, there's more:

/tmp % time dmd -c -unittest foo.d
dmd -c -unittest foo.d  0.46s user 0.06s system 99% cpu 
0.525 total




I did the same timings on my computer, and I got 0.128s and 
0.272s. I'm on DMD 2.075.1, so something must have changed in 
std.path since then.

Re: D compiles fast, right? Right??

[Jonathan Marler via Digitalmars-d]

On Friday, 30 March 2018 at 16:12:44 UTC, Atila Neves wrote:
Fast code fast, they said. It'll be fun, they said. Here's a D 
file:


import std.path;


Yep, that's all there is to it. Let's compile it on my laptop:

/tmp % time dmd -c  foo.d
dmd -c foo.d  0.12s user 0.02s system 98% cpu 0.139 total


That... doesn't seem too fast to me. But wait, there's more:

/tmp % time dmd -c -unittest foo.d
dmd -c -unittest foo.d  0.46s user 0.06s system 99% cpu 
0.525 total



Half. A. Second. AKA "an eternity" in dog years, err, CPU time. 
I know this has been brought up before, and recently even, but, 
just... just... sigh.


So I wondered how fast it'd be in Go, since it's got a 
reputation for speedy compilation:


package foo
import "path"
func Foo() string {
return path.Base("foo")
}

/tmp % time go tool compile foo.go
go tool compile foo.go  0.01s user 0.01s system 117% cpu 
0.012 total



See, now that's what I'd consider fast. It has actual code in 
the file because otherwise it complains the file isn't using 
the imported package, because, Go things. It compiled so fast I 
had to check I'd generated an object file, and then I learned 
you can't use objdump on Go .o files, because... more Go things 
(go tool objdump for the curious).


Ok, so how about C++, surely that will make D look good?

#include   // yes, also a one-liner

/tmp % time /usr/bin/clang++ -std=c++17 -c foo.cpp
/usr/bin/clang++ -std=c++17 -c foo.cpp  0.45s user 0.03s 
system 96% cpu 0.494 total


/tmp % time /usr/bin/g++ -std=c++17 -c foo.cpp
/usr/bin/g++ -std=c++17 -c foo.cpp  0.39s user 0.04s system 
99% cpu 0.429 total



So yeeah. If one is compiling unit tests, which I 
happen to pretty much only exclusively do, then trying to do 
anything with paths in D is


1. Comparable to C++ in build times
2. Actually _slower_ than C++ (who'd've thunk it?) *
3. Gets lapped around Captain America vs The Falcon style about 
50 times by Go.


And that's assuming there's a crazy D programmer out there 
(hint: me) that actually tries to compile minimal units at a 
time (with actual dependency tracking!) instead of the whole 
project at once, otherwise it'll take even longer. And this to 
just import `std.path`, then there's the actual work you were 
trying to get to.


Today actually made me want to write Go. I'm going to take a 
shower now.


Atila

* Building a whole project in C++ still takes a lot longer 
since D scales much better, but that's not my typical worflow, 
nor should it be anyone else's.


Seems like you're comparing apples to oranges.

Go's path.go is very small, a 215 line file:
https://github.com/golang/go/blob/master/src/path/path.go
Documentation: https://golang.org/pkg/path/


Dlang's std.path is much more comprehensive with 4181 lines: 
https://github.com/dlang/phobos/blob/master/std/path.d

Documentation: https://dlang.org/phobos/std_path.html

It's over an order of magnitude more code and only takes twice as 
long to compile without unittests, and it's only fair to compare 
the "non-unittest" version of std.path with Go, since Go does not 
include unittests.


I'm not sure why you would compile the standard library unittests 
every time you compile anything. Probably a consequence of not 
having `-unittest=`.  timotheecour suggested we add 
support for this and I agree for cases like this, where druntime 
and phobos would be exclude by default (just like we do with -i), 
meaning that your compilation example would not have compiled 
phobos unittests.

Re: D compiles fast, right? Right??

[Meta via Digitalmars-d]

On Friday, 30 March 2018 at 16:12:44 UTC, Atila Neves wrote:
Fast code fast, they said. It'll be fun, they said. Here's a D 
file:


import std.path;


Yep, that's all there is to it. Let's compile it on my laptop:

/tmp % time dmd -c  foo.d
dmd -c foo.d  0.12s user 0.02s system 98% cpu 0.139 total


That... doesn't seem too fast to me. But wait, there's more:

/tmp % time dmd -c -unittest foo.d
dmd -c -unittest foo.d  0.46s user 0.06s system 99% cpu 
0.525 total



Half. A. Second. AKA "an eternity" in dog years, err, CPU time. 
I know this has been brought up before, and recently even, but, 
just... just... sigh.


So I wondered how fast it'd be in Go, since it's got a 
reputation for speedy compilation:


package foo
import "path"
func Foo() string {
return path.Base("foo")
}

/tmp % time go tool compile foo.go
go tool compile foo.go  0.01s user 0.01s system 117% cpu 
0.012 total



See, now that's what I'd consider fast. It has actual code in 
the file because otherwise it complains the file isn't using 
the imported package, because, Go things. It compiled so fast I 
had to check I'd generated an object file, and then I learned 
you can't use objdump on Go .o files, because... more Go things 
(go tool objdump for the curious).


Ok, so how about C++, surely that will make D look good?

#include   // yes, also a one-liner

/tmp % time /usr/bin/clang++ -std=c++17 -c foo.cpp
/usr/bin/clang++ -std=c++17 -c foo.cpp  0.45s user 0.03s 
system 96% cpu 0.494 total


/tmp % time /usr/bin/g++ -std=c++17 -c foo.cpp
/usr/bin/g++ -std=c++17 -c foo.cpp  0.39s user 0.04s system 
99% cpu 0.429 total



So yeeah. If one is compiling unit tests, which I 
happen to pretty much only exclusively do, then trying to do 
anything with paths in D is


1. Comparable to C++ in build times
2. Actually _slower_ than C++ (who'd've thunk it?) *
3. Gets lapped around Captain America vs The Falcon style about 
50 times by Go.


And that's assuming there's a crazy D programmer out there 
(hint: me) that actually tries to compile minimal units at a 
time (with actual dependency tracking!) instead of the whole 
project at once, otherwise it'll take even longer. And this to 
just import `std.path`, then there's the actual work you were 
trying to get to.


Today actually made me want to write Go. I'm going to take a 
shower now.


Atila

* Building a whole project in C++ still takes a lot longer 
since D scales much better, but that's not my typical worflow, 
nor should it be anyone else's.


Yeah, that's pretty bad, relatively, for a no-op build. Probably 
some CTFE or template stuff that gets pulled in by one of the 
imports.