On Thursday, 31 December 2015 at 00:04:03 UTC, bachmeier wrote:
It's the process that requires so much overhead that nobody
wants to contribute. I really tried to do so myself, but I'm busy, and it is senseless that 95% (or more) of the time I spend on it is wasted due to a system that is flawed from top to bottom. The only thing that surprises me is that there are any contributions.

Exactly. I've put my time into ddoc. Let me give you a brief history of my efforts:

~2010: I had just written this awesome dom.d library and wanted to document it and release it to the world. I write stuff like:

/// Returns the text in the element. For example, innerText of <span>foo<span> is "foo" (without quotes)
string innerText();

And it came out mangled because of ddocs embedded HTML "feature" which doesn't encode the output. I proposed a fix, using ddoc's existing ESCAPES macro, and wrote the patch for dmd. Embedded html would be automatically encoded, but you can still define it in macro definitions, which can be inline with the comment, so it isn't hard to use when you want it.

It was rejected. Walter didn't see what the problem was and I was told to just write $(LT)span$(GT)foo$(LT)/span$(GT). Seriously.

Well, that was unacceptable so I decided to take matters into my own hands. On Feb 20, 2010, I registered dpldocs.info and started writing my own web service to delver dom.d's docs, encoding the output correctly, getting the docs out of dmd's JSON output, using -D -X.

Well, it worked, but it sucked. The JSON output didn't give me enough information to do cross referencing, so I wrote a little search engine to paper over it... and that didn't help much.

I moved on to other things, every so often still advocating to fix ddoc's escaping, but mostly just writing my docs such that they were readable in the source code and not bothering with generating them at all. (I'm told my source tends to be fairly readable anyway and I like to narrate my thoughts a bit in the comments including of implementation details so they understand the pros and cons of my decisions.)

At some point around here, I wrote a crappy Javascript table of contents for the D website (then hosted at digitalmars.com still) - the infamous "Jump list". This was meant to be a temporary measure until ddoc got proper table of contents support.

That sucky Javascript hack is still live today.

2011: We had a proposal for a website redesign. Remember this one? http://arsdnet.net/d-web-site/index.html

The jump list hack garnered a lot of hate. It was a blob of text that we couldn't read, but ddoc's design made a user-defined table of contents extremely difficult to implement. A few attempts were started, but none finished.

I wrote a program, called improveddoc.d, still on the 'net: http://arsdnet.net/d-web-site/improveddoc.d that post-processed the generated HTML to create a cheat-sheet table.

This is what it made of std.algorithm at the time:


I offered to integrate it into the website tools... and got my first look at the horror that is that build process. Nevertheless, running a little post process script to run that on the html was doable. It also would read tag macros and use them to organize the content.

The idea (and working program) was rejected because the team felt a post-processor was the wrong way to do it.

Instead, Andrei manually wrote a std.algorithm cheat sheet and we added a bullet between the links in the jump list hack. Both of these have remained basically unchanged to this day and dmd still cannot generate a table of contents in ddoc.

We did get a website redesign after that, d-programming-language.org was launched, and Vladimir's forum got integrated (I wrote one too but his was so much better than mine that I gladly yielded). ddoc, however, barely changed.

For the next couple years, despite ddoc being unusable for half my libraries because they are web libs and showing HTML examples was practically impossible, I still was basically on its side, while still working on improving the json support. I had tried the json thing and found the compiler approach to be better, but the json output is good for a few things and Ddoc's syntax isn't all that bad if you aim for source readability and clever use of recursive macros can do some pretty cool things.

I put my support behind dmd. With the vibe.d team also pushing for improved json support, a few things happened there... though little of consequence. My opinion of the json output hasn't significantly changed since 2010: cool for a few things, I like that we have it, but it is not a substitute for compiler-integrated ddoc.

We got another website redesign in these years, and moved to dlang.org. The build process finally got documented after I and others complained for a while, but it was never actually cleaned up.

At the end of 2014, yea, even a year before this writing, people were complaining about ddoc's syntax again and asking for some Markdown variant to be introduced. Seeing an opportunity to revive my old encoding problem and finally write some ddoc for myself, I jumped on the `code` syntax and implemented it, including proper encoding of the output. Finally, I can write `<span>foo</span>` and have it legible to the user!

And with it also alleviating the pressure for syntax sugar from the community... this patch was actually accepted this time! For the first time in five years, I was excited about ddoc again.

I started using it to write This Week in D (and noticing the limits the macro syntax place on legible prose source...) and wrote at some length about my simpledisplay.d, cgi.d, and others (though not really much on dom.d, ironically. I still haven't gotten around to updating that.)

Just a few months ago, I got the simpledisplay docs looking pretty good with ddoc: http://arsdnet.net/arsd/simpledisplay.html

2015 was a good year for ddoc and I, we basically got along, despite its warts.

Then, October rolls around. A friend of a friend is looking for work and I say maybe you can help me with some D stuff. Don't know D? Not a problem, it is easy!

It wasn't easy. From installation, problems arose. I'd then show how to fix it and explain why.

She'd say "I understand now, but why don't the docs just say what you just said?"

Good question. I decide to start editing it in... and quickly hit a brick wall that kills my momentum. I couldn't get through the nonsense before having to get back to work.

I handle hundreds of D support requests in a year. Sometimes, I spend so much time on the chat, the learn forum, or Stack Overflow that I feel like I'm a full-time D support engineer. Time and time again, people ask things that ought to be pretty easy to find, yet they ask us anyway.

Maybe they just don't like searching and ask first. I actually don't mind that - I tell my coworkers that if their attempts at Google don't yield results in one minute, send me an IM or email (and continue looking). There's a good chance I'll have an answer in my brain somewhere, or at least a helpful pointer, and I don't mind the interruptions at all (if I didn't want them, I'd just close the chat client!). Better to ping me than waste hours on a fruitless search.

But still, some people say "I tried searching and couldn't find it"... and I totally believe that with D. I used it think it was just because we were young, but it has now been almost SIX YEARS since my first dive into the website issues and it has barely changed. There must be something systemic.

Fast-forward to December. We have that base64 question. I go to edit the site and get stopped by the process again. Nevertheless, I blamed the makefile and saw new documentation Andrei wrote on this. Maybe there's still hope.

Then another friend of mine tells me he finally started looking at the D Cookbook I gave him last year and is writing some code. I watch him work through a couple problems... and notice that the website was only useful to him insofar as he could copy/paste examples. As soon as he modified them and got an error message or tried to put functions together without pre-written examples, he'd slow way down. He'd get through it but not easily.

The D leaders know how important examples are. We are often told adding more is low hanging fruit. I completely agree. But that's not ALL we need. He wants examples to get started, yes, but he also wants understanding to go beyond examples. That's where text helps. That's where understanding the function signatures help. Eventually, when people go to write their own libraries, they might want to do Phobos style genericity. You can't do that without understanding the function signatures... and those blobs of text are not understandable.

Shift to the forum. We got talking about website redesign on the forum again a couple weeks ago and I remark how I think just whitespace formatting of the constraints would help a lot and almost went in dmd to implement it.

I just got sidetracked doing a quick paragraph fix for Andrei... and then wanted to see how far I could go. I got it working, despite the obstacles that ddoc's freeform macros bring, and surprise, it was rejected. Even the simpler patch that just collapses blank lines sits unanswered. A tool to look for broken links sits unanswered. There's always a call to contributions, but when you do push through the painful process to get the code up (and the tester, the f$^%$^ing tester), nobody seems to care.

The combined experience over these last six years tells me that the website sucks and it keeps sucking because changing it sucks even more than using it. A new strategy was required.

And here we are.

I still need to figure out how I want to add new docs and link them all in, but I guarantee you it won't consist of editing SRCS and MANIFEST and posix.mak and win32.mak and std.ddoc and latex.ddoc and <voice trails off>...

Reply via email to