On Tuesday, 5 January 2016 at 14:18:32 UTC, Andrei Alexandrescu wrote:
There's been a recent discussion with Walter and Martin about using wildcards in makefiles (which would obviate the necessity of being explicit about files).

I actually do NOT use wildcards in most of my own makefiles for exactly these reasons, I tend to keep dead files around.

But I also list the file once and only once. There's no separate SRC, MANIFEST, DOCS, whatever. I don't even know what the difference is between each of those.

However, that's for building code which is separate from building documentation. Building code needs a bit more care because it has to actually compile. Docs don't, it can make a few extra html files without lots of worry.

This is one of ddoc's fundamental technical flaws, by the way, being integrated with the compiler. The compiler *needs* everything set up right to function properly. You can't compile code without the proper conditional compilation settings, for example. You can't compile a program with two main()'s.

But docs are different. You don't want a function not to appear just because it is under a version(cool_feature) block.


Notice I said "fundamental" rather than "fatal" because we have a workaround: the version(D_DDOC) thing. But that sucks. You now have to write the whole function prototype again to show up in the docs and if it happens to be different on the other platforms, well, you might as well just write the docs by hand. You've lost all benefits of auto-generation now.


Oh yeah, and the website also has a navbar that is manually maintained, and an index page which is often neglected, probably because people don't expect to have SIX different places to add a new file! (The makefile parts, these macros, and, of course, the git add to the filesystem.)

My nav bar is automatic. Click around std.socket to see what I did with it today: http://dpldocs.info/experimental-docs-2/std.socket.Socket.setOption.4.html


to produce incomplete builds if some files are missing by mistake.

That would surely be caught by the compiler itself (module file not found) or the test runner. In the case of documentation, a bunch of broken links would be detected. For druntime, the Phobos build would fail, etc.



Would you recommend switching to wildcards in our makefiles and assume people use git to keep their directories in good shape?

It'd probably work fine in git; worth an exploration. Especially for doc building which wants everything in /import, without a doubt. Phobos isn't as clear cut because it builds right from /src and has some internal files in there, but still, a wildcard that excludes the internals beats one that is wrong in the common case.

* Each library version must be built with the same compiler version. The appropriate compiler version might be missing, so it needs to be downloaded, installed, and run on the side without affecting any existing installation.

And this is again because everything is so strongly coupled to dmd. I'm not even convinced Phobos ought to be this closely coupled (my libs outside Phobos are compatible with at least a year of previous versions, sometimes more, and my users find this very valuable for stability against the fear of regressions), but the documentation certainly shouldn't be.

With an external doc generator, you can get much broader version compatibility since it isn't responsible for actually compiling the entire language like dmd. It just needs to parse enough of it to get useful information to attach to comments, and understand just enough to attach some useful text.

For example, my doc generator (built on Brian Schott's libdparse btw) knows that "@safe" is worth pointing out to the user. But it doesn't need to know what it actually means. It doesn't run that level of semantic analysis and is thus isolated by changes in the safed spec. dmd doesn't have that luxury.


My understanding is that your system does not do any of these things.

I ask you to consider the importance of compromising website quality in favor of those things from a strategic viewpoint.

If the pdf is really popular enough that its continuance is warranted (tbh, I can't understand why it would be, it is equivalent to or inferior to even the current website in almost every way), you should find a way to do it that doesn't hurt the main site.

I certainly won't be concerning myself with it. I do care about offline viewing, and have a simple solution: I can offer a zip/tarball of the website html files. They will just work without a network connection as well.

BTW: generating those other formats from html is also pretty easy and can be done after the fact.

The script https://github.com/D-Programming-Language/dlang.org/blob/master/posix.mak stands at 498 lines. What ways there are to make it considerably simpler and smaller?

I've looked at this before and didn't even understand what it is trying to do, much less why it is doing that. I don't feel comfortable modifying code that I don't understand.

And frankly, I have zero interest in dealing with complicated build systems. My view is if they aren't nearly invisible, they have failed me.

And I deal with failing subordinates in a similar manner to Darth Vader:

https://www.youtube.com/watch?v=aV2DLkDPwM8&feature=youtu.be&t=33s

Indeed the web forms sound like a great idea. Would you want to work on such for the mainline website as well? It's not a contest; the better everything is, the better off everybody is.

It's not the web form that's difficult though, it is getting the new file integrated with the build and actually running it. The existing site makes that exceedingly difficult.

Nice. What could we do in ddoc and ddox to have a similar improvement?

I implemented it as a magic macro, they could do the same.

This kind of expansion cannot be implemented as a ddoc macro itself because it needs more info than text expansion (and mine has an if statement if name lookup fails), but it is fairly easy to express in D.

How do you envision scaling it up as the system becomes more successful and you get more requests than you can handle reasonably quick?

I'll cross that bridge when I get to it. Let's not let tomorrow's potential hurdle be today's showstopper.


Would you be willing to set up such a system for dlang.org as well? It would be absolutely fantastic.

People can (and often do) email me all kinds of things. But I'm not the god-emperor of dlang.org so what I can easily do there is limited.

Anyone could do the same today for all pull requests - just fetch them, fix them, and resubmit with credit. Additionally, team members could just pull the PR, fix it, and make an additional commit.

Indeed, that's not a technical barrier. But it is a cultural barrier - the D PR process (which btw is strongly encouraged by the GitHub UI, it isn't just us) has more emphasis on leaving comments than on fixing it yourself.

I'll note that you are already a member of that team so you already had the rights to exact quite a few of the improvements you intend to make going forward. Would you please join me in taking a look at older PRs (oldest is from 2014) and pull them in?

The technical access and the feeling of ownership are two different things.

If you gave me a key to your house, would it be fine for me to do in some day when you're not home and start remodeling it?

That's the problem with the "lieutenant" thing that's been discussed before. Lieutenants are junior officers who just keep things running according to regulation while the captain is busy elsewhere. If anything extraordinary happens, they report it up the chain of command.

What we need is a greater amount of delegation. Lieutenant-generals perhaps: individuals who have full authorization and broad autonomy to run an entire sector of operations as they see fit.

Overall my understanding of your message is "My system would be better not for a fundamental technical reason, but because I am willing to pour in it time and talent." This is the kind of argument I have a lot of respect for.

Well, there are technical reasons too. Tying ddoc to dmd was a short-term win, it got to reuse the compiler to do hard work at a time when there were no realistic alternatives.

But, today, it is limiting. Sharing code with the compiler means ddoc is subject to the compiler's trade-offs.... which, given a choice that strengthens code or strengths docs, code is going to win every time. As it should, that's the compiler's job. Moving ddoc to a separate path will result in code duplication at best, so at that point, you don't gain much from staying in dmd anymore.

Ddoc's text macro system, while an elegant implementation and good enough for many things, is also a fundamentally limiting factor (and now tied up a bit due to backward-compatibility with arbitrary user-defined extensions), and prone to complications. It puts a limit on how much analysis of the source text we can realistically do because anything could expand to anything else. It is a mistake. (The missing word there btw is $1, disappeared because ddoc mistakes it for syntax and just erases it. Seriously, try it.)

Ddoc's total failure to understand what it is generating means it cannot automatically link. And since it lacks conditionals, you can't even program it to link to user-defined things. (And no, I am not suggesting we add $(IF). We don't need yet another programming language... nobody wants to write their documentation in uncommon Lisp. We want it to just work.)


Fixing these will mean breaking backward compatibility. If you're willing to do that, great! I've been writing a ddoc 2.0 generator :P
              • ... default0 via Digitalmars-d-announce
              • ... Andrei Alexandrescu via Digitalmars-d-announce
              • ... JohnCK via Digitalmars-d-announce
              • ... John Colvin via Digitalmars-d-announce
              • ... Ola Fosheim Grøstad via Digitalmars-d-announce
              • ... bachmeier via Digitalmars-d-announce
              • ... default0 via Digitalmars-d-announce
              • ... Adam D. Ruppe via Digitalmars-d-announce
              • ... bachmeier via Digitalmars-d-announce
              • ... Jacob Carlborg via Digitalmars-d-announce
              • ... Adam D. Ruppe via Digitalmars-d-announce
              • ... Jacob Carlborg via Digitalmars-d-announce
              • ... Andrei Alexandrescu via Digitalmars-d-announce
              • ... Jacob Carlborg via Digitalmars-d-announce
              • ... Adam D. Ruppe via Digitalmars-d-announce
              • ... Vladimir Panteleev via Digitalmars-d-announce
              • ... Jacob Carlborg via Digitalmars-d-announce
              • ... Andrei Alexandrescu via Digitalmars-d-announce
  • Re: Better docs for D (WIP... Nordlöw via Digitalmars-d-announce

Reply via email to