On Tuesday, 5 January 2016 at 14:18:32 UTC, Andrei Alexandrescu
There's been a recent discussion with Walter and Martin about
using wildcards in makefiles (which would obviate the necessity
of being explicit about files).
I actually do NOT use wildcards in most of my own makefiles for
exactly these reasons, I tend to keep dead files around.
But I also list the file once and only once. There's no separate
SRC, MANIFEST, DOCS, whatever. I don't even know what the
difference is between each of those.
However, that's for building code which is separate from building
documentation. Building code needs a bit more care because it has
to actually compile. Docs don't, it can make a few extra html
files without lots of worry.
This is one of ddoc's fundamental technical flaws, by the way,
being integrated with the compiler. The compiler *needs*
everything set up right to function properly. You can't compile
code without the proper conditional compilation settings, for
example. You can't compile a program with two main()'s.
But docs are different. You don't want a function not to appear
just because it is under a version(cool_feature) block.
Notice I said "fundamental" rather than "fatal" because we have a
workaround: the version(D_DDOC) thing. But that sucks. You now
have to write the whole function prototype again to show up in
the docs and if it happens to be different on the other
platforms, well, you might as well just write the docs by hand.
You've lost all benefits of auto-generation now.
Oh yeah, and the website also has a navbar that is manually
maintained, and an index page which is often neglected, probably
because people don't expect to have SIX different places to add a
new file! (The makefile parts, these macros, and, of course, the
git add to the filesystem.)
My nav bar is automatic. Click around std.socket to see what I
did with it today:
to produce incomplete builds if some files are missing by
That would surely be caught by the compiler itself (module file
not found) or the test runner. In the case of documentation, a
bunch of broken links would be detected. For druntime, the Phobos
build would fail, etc.
Would you recommend switching to wildcards in our makefiles and
assume people use git to keep their directories in good shape?
It'd probably work fine in git; worth an exploration. Especially
for doc building which wants everything in /import, without a
doubt. Phobos isn't as clear cut because it builds right from
/src and has some internal files in there, but still, a wildcard
that excludes the internals beats one that is wrong in the common
* Each library version must be built with the same compiler
version. The appropriate compiler version might be missing, so
it needs to be downloaded, installed, and run on the side
without affecting any existing installation.
And this is again because everything is so strongly coupled to
dmd. I'm not even convinced Phobos ought to be this closely
coupled (my libs outside Phobos are compatible with at least a
year of previous versions, sometimes more, and my users find this
very valuable for stability against the fear of regressions), but
the documentation certainly shouldn't be.
With an external doc generator, you can get much broader version
compatibility since it isn't responsible for actually compiling
the entire language like dmd. It just needs to parse enough of it
to get useful information to attach to comments, and understand
just enough to attach some useful text.
For example, my doc generator (built on Brian Schott's libdparse
btw) knows that "@safe" is worth pointing out to the user. But it
doesn't need to know what it actually means. It doesn't run that
level of semantic analysis and is thus isolated by changes in the
safed spec. dmd doesn't have that luxury.
My understanding is that your system does not do any of these
I ask you to consider the importance of compromising website
quality in favor of those things from a strategic viewpoint.
If the pdf is really popular enough that its continuance is
warranted (tbh, I can't understand why it would be, it is
equivalent to or inferior to even the current website in almost
every way), you should find a way to do it that doesn't hurt the
I certainly won't be concerning myself with it. I do care about
offline viewing, and have a simple solution: I can offer a
zip/tarball of the website html files. They will just work
without a network connection as well.
BTW: generating those other formats from html is also pretty easy
and can be done after the fact.
https://github.com/D-Programming-Language/dlang.org/blob/master/posix.mak stands at 498 lines. What ways there are to make it considerably simpler and smaller?
I've looked at this before and didn't even understand what it is
trying to do, much less why it is doing that. I don't feel
comfortable modifying code that I don't understand.
And frankly, I have zero interest in dealing with complicated
build systems. My view is if they aren't nearly invisible, they
have failed me.
And I deal with failing subordinates in a similar manner to Darth
Indeed the web forms sound like a great idea. Would you want to
work on such for the mainline website as well? It's not a
contest; the better everything is, the better off everybody is.
It's not the web form that's difficult though, it is getting the
new file integrated with the build and actually running it. The
existing site makes that exceedingly difficult.
Nice. What could we do in ddoc and ddox to have a similar
I implemented it as a magic macro, they could do the same.
This kind of expansion cannot be implemented as a ddoc macro
itself because it needs more info than text expansion (and mine
has an if statement if name lookup fails), but it is fairly easy
to express in D.
How do you envision scaling it up as the system becomes more
successful and you get more requests than you can handle
I'll cross that bridge when I get to it. Let's not let tomorrow's
potential hurdle be today's showstopper.
Would you be willing to set up such a system for dlang.org as
well? It would be absolutely fantastic.
People can (and often do) email me all kinds of things. But I'm
not the god-emperor of dlang.org so what I can easily do there is
Anyone could do the same today for all pull requests - just
fetch them, fix them, and resubmit with credit. Additionally,
team members could just pull the PR, fix it, and make an
Indeed, that's not a technical barrier. But it is a cultural
barrier - the D PR process (which btw is strongly encouraged by
the GitHub UI, it isn't just us) has more emphasis on leaving
comments than on fixing it yourself.
I'll note that you are already a member of that team so you
already had the rights to exact quite a few of the improvements
you intend to make going forward. Would you please join me in
taking a look at older PRs (oldest is from 2014) and pull them
The technical access and the feeling of ownership are two
If you gave me a key to your house, would it be fine for me to do
in some day when you're not home and start remodeling it?
That's the problem with the "lieutenant" thing that's been
discussed before. Lieutenants are junior officers who just keep
things running according to regulation while the captain is busy
elsewhere. If anything extraordinary happens, they report it up
the chain of command.
What we need is a greater amount of delegation.
Lieutenant-generals perhaps: individuals who have full
authorization and broad autonomy to run an entire sector of
operations as they see fit.
Overall my understanding of your message is "My system would be
better not for a fundamental technical reason, but because I am
willing to pour in it time and talent." This is the kind of
argument I have a lot of respect for.
Well, there are technical reasons too. Tying ddoc to dmd was a
short-term win, it got to reuse the compiler to do hard work at a
time when there were no realistic alternatives.
But, today, it is limiting. Sharing code with the compiler means
ddoc is subject to the compiler's trade-offs.... which, given a
choice that strengthens code or strengths docs, code is going to
win every time. As it should, that's the compiler's job. Moving
ddoc to a separate path will result in code duplication at best,
so at that point, you don't gain much from staying in dmd anymore.
Ddoc's text macro system, while an elegant implementation and
good enough for many things, is also a fundamentally limiting
factor (and now tied up a bit due to backward-compatibility with
arbitrary user-defined extensions), and prone to complications.
It puts a limit on how much analysis of the source text we can
realistically do because anything could expand to anything else.
It is a mistake. (The missing word there btw is $1, disappeared
because ddoc mistakes it for syntax and just erases it.
Seriously, try it.)
Ddoc's total failure to understand what it is generating means it
cannot automatically link. And since it lacks conditionals, you
can't even program it to link to user-defined things. (And no, I
am not suggesting we add $(IF). We don't need yet another
programming language... nobody wants to write their documentation
in uncommon Lisp. We want it to just work.)
Fixing these will mean breaking backward compatibility. If you're
willing to do that, great! I've been writing a ddoc 2.0 generator