The order Josh mentions seems correct, but usability and ease of consumption is just as important if not more as correctness and coverage.
In technical writing, the key elements to usability and ease of consumption are findability and searchability. Findability means finding information for something you want to do without knowing what it is beforehand. Searchability is finding information you know about using the terms that you know. The key to effective documentation is that information is both findable and searchable in "terms that the users know". A simple example is gossip. If you know nothing about Cassandra, you probably understand that nodes talk to each other, which you might search for using "internode communication" or "network communication". Without the ability to locate information you want, correctness and coverage are meaningless. Another principle that makes good documentation is that they are solution-based. Two examples are replacing a node and adding a node. Another important feature of being able to produce accurate and complete docs is the ability to reuse information. Using the previous examples, replacing a node and adding a node, may have some of the same steps. Reusing information is not saving time by writing only once, it's about making sure that when information is updated, it's updated everywhere it needs to be (especially in places you don't know about). Having a single source for reusing information is essential to making this happen. Also, related to reusing information, the ability to pull from a central location using includes/shortcodes/etc. can ease the testability and findability of code examples used in documentation. Rather than scattering code throughout the docs, you can store the code snippets in their own repo. For instance, asciidoc has such a capability (amongst others): [source,ruby] ---- include::example.rb[] ---- The last feature I want to mention that contributes to good documentation is semantic structure. The idea of semantic structure is similar to object-oriented programming, where objects contain data. So instead of manually defining all the attributes of the warning, you can just declare the warning and add the data. For example, suppose you want a warning that says "Don't do this, it will kill your system!" In a non-semantics authoring, such as Markdown (designed as format for writing for the web), you'd have to define each element: **Warning** Don't do this, it will kill your system! The problem here is not so much having to define each element but that a different writer can do something different, such as vary the marking from ** to *, as there is no enforced structure: *Warning* Don't do this, it will kill your system! Although this is a very simple example, not being able to enforce a standard can be confusing to the user because clues to using the documentation lack consistency and refinement. In semantics-based documentation, such in reStructuredText, you can just write . warning:: Don't do this, it will kill your system! and every instance will be consistent. I realize that everyone wants something simple that they don't have to learn, but doing so will: 1) Decrease the efficiency of writing docs, which reduces the likelihood of complete coverage. 2) Reduce correctness, because the writer does not necessarily know everywhere information needs to be updated. 3) Diminish the usability and ease of consumption. For example, a lack of consistency reduces the ability of the user to quickly scan a document for the information they need and appears amateurish. On 2020/05/04 15:13:49, Joshua McKenzie <jmcken...@apache.org> wrote: > I've been mulling over this topic the past few days as we often seem to get > mired in debates over technical details of offerings without a clear value > system to weigh them against one another. In the case of documentation, I'd > propose that we think about this from the perspective of the users of the > documentation. I suspect (and would love to hear points of view for or > against this claim as I do not have first-hand knowledge) that doc users > would care about the following in this order: > > 1) Correctness > 2) Coverage > 3) Usability and ease of consumption > > Assuming we can get a simple list of traits to optimize for, it may be > helpful to weigh the pros and cons of various documentation frameworks > against how they facilitate or deliver against those metrics. For example: > ease of developer ramp and contribution to docs would increase Coverage, > where more robust tooling and inter-linkage could contribute to ease of > consumption. > > > > On Fri, May 1, 2020 at 1:52 PM Jon Haddad <j...@jonhaddad.com> wrote: > > > We've already got Sphinx set up, so you can contribute today. There's a > > docker container in the `docs` directory and a readme that can help you get > > started. > > > > On Fri, May 1, 2020 at 10:46 AM Chen-Becker, Derek > > <dchen...@amazon.com.invalid> wrote: > > > > > From the peanut gallery, my main concern is less with the features of a > > > given markup and more with ensuring that whatever markup/doc system is > > > used stays mostly out of the way of people who want to contribute to the > > > docs. I don't want to have to learn a whole publishing system just to be > > > able to contribute, whereas minor differences in markup syntax seem > > > reasonable. Whatever system ends up getting chosen, is there additional > > > work that can be done to simplify work for writers? I've used all three > > > (albeit not in-depth), so I'm willing to help. > > > > > > Derek > > > > > > On 5/1/20 11:08 AM, Jon Haddad wrote: > > > > > > > CAUTION: This email originated from outside of the organization. Do not > > > click links or open attachments unless you can confirm the sender and > > know > > > the content is safe. > > > > > > > > > > > > > > > > Apologies, I didn't mean to upset or insult you. My intent was to > > > > demonstrate that my opinion is based on experience and I wasn't > > > suggesting > > > > we switch tooling based on a whim. I also wasn't trying to imply you > > > > aren't knowledgeable about writing documentation. > > > > > > > > Apart from this misunderstanding, I think we mostly agree. I'm not > > > looking > > > > to perform a migration that removes functionality, and the features > > > you've > > > > listed are all important to keep. Thanks for listing out the bits that > > > are > > > > more complex that I glossed over with my "We write basic text with > > links > > > > and a menu" comment, that was clearly wrong and I appreciate the > > > correction. > > > > > > > > Regarding the functionality you listed: > > > > > > > > * Note and warning are both useful features and come built into > > > > asciidoctor. I made use of them in the TLP documentation for > > tlp-cluster > > > > [1] > > > > * I believe the extlinks feature can be replicated easily using a > > macro. > > > > Here's an example [2]. > > > > * The grammar is a bit more difficult and I don't think there's a drop > > in > > > > replacement. Writing a block processor [3] would be the right way to > > > > approach it, I think. > > > > * We'd probably want to set up a configuration for syntax highlighting > > > via > > > > highlight.js (or one of the other supported libs). We can use the SQL > > > one > > > > [4] as a guide since it's going to be similar to what we need, and > > > ideally > > > > we would contribute it back for CQL support. > > > > > > > > I agree with you that any migration would at a *minimum* need the above > > > > functionality to be on par with what we already have. A POC in a > > branch > > > > displaying a handful of pages (that work with the site theme, etc), one > > > of > > > > which is a converted DDL page [5] would suffice, I think, to assess > > > whether > > > > or not it's worth the effort. > > > > > > > > No matter which direction we end up going, we still need to get > > > > documentation improvements in for 4.0, so it's probably worth focusing > > on > > > > that now, and convert to adoc later. I'm happy to get on a call soon > > > with > > > > folks who are new to the project documentation to answer any questions > > > you > > > > all may have. I'm also available to review patches to the docs, just > > set > > > > me as the reviewer and ping me on Slack. I try to get to them within > > > 24h. > > > > > > > > Jon > > > > > > > > [1] http://thelastpickle.com/tlp-cluster/#_setup > > > > [2] > > https://markhneedham.com/blog/2018/02/19/asciidoctor-creating-macro/ > > > > [3] > > > > > > > > > https://github.com/asciidoctor/asciidoctorj/blob/v2.1.0/docs/integrator-guide.adoc#blockprocessor > > > > [4] > > > > > > > > > https://github.com/highlightjs/highlight.js/blob/master/src/languages/sql.js > > > > [5] https://cassandra.apache.org/doc/latest/cql/ddl.html > > > > > > > > On Thu, Apr 30, 2020 at 2:21 PM Sylvain Lebresne <lebre...@gmail.com> > > > wrote: > > > > > > > >> As I mentioned, I really have nothing against asciidoc. In fact, I > > think > > > >> it's > > > >> great. > > > >> > > > >> Let's just say that I think rst/sphinx is also pretty capable, and > > that > > > I > > > >> consider > > > >> your characterization of the syntax difference (one "awful", the other > > > "a > > > >> dream") a tad over-the-top. I do agree on the point on documentation > > > >> though, > > > >> the asciidoc one is better (not that I find the rst one completely > > > >> inadequate > > > >> either), and I reckon it's a good argument. > > > >> > > > >> So to be clear, if someone makes the change to asciidoc and it's not > > > >> botched, I > > > >> won't personally stand in the way. > > > >> > > > >> I'll note however that asking we analyze the pros and cons of a change > > > >> should not be seen as suspicious. And we should imo strive to justify > > > any > > > >> change with objective arguments. One's experience certainly increases > > > the > > > >> believability of one's arguments, but it doesn't dispense from > > > presenting > > > >> arguments in the first place. > > > >> > > > >> And I wish the substance of your previous email wasn't: I know, you > > > don't, > > > >> and > > > >> the project don't have time to wait on you learning, so just trust me. > > > >> > > > >>> You're right about markdown being a little limited, but we're not > > > really > > > >>> using anything advanced in sphinx. We write basic text with links > > and a > > > >> menu. > > > >> > > > >> Not really true of at least the CQL section. It makes somewhat > > extensive > > > >> use > > > >> of the 'productionlist::' feature. Which gives us decent formatting of > > > the > > > >> CQL > > > >> grammar elements "for free", automatic cross-referencing within said > > > >> grammar > > > >> and easy cross-referencing to said grammar from the rest of the text. > > I > > > >> think > > > >> that's kind of nice? I could be wrong, but getting the same even with > > > >> asciidoc > > > >> is going to be much more manual, and definitively would with markdown. > > > >> > > > >> We also use 'note::' and 'warning::' boxes in a few places, and those > > > are > > > >> also > > > >> nice to have imo, and I don't think mardown would give us that easily. > > > >> > > > >> We also define a jira "extlinks" (so that anywhere in the doc, > > > ":jira:`42`" > > > >> is > > > >> replaced by a proper link named CASSANDRA-42 and pointing to that > > > ticket) > > > >> and > > > >> it's used in a few places. > > > >> > > > >> Fwiw, it's this kind of things (and any future similar use we may > > want) > > > I > > > >> had > > > >> in mind when discussing markdown being limited, and we can debate > > their > > > >> importance, but we do use them. > > > >> > > > >> But maybe those don't qualify as "really" using advanced stuffs. How > > > would > > > >> I > > > >> know, I'm the guy that needs to learn, you're the expert. > > > >> > > > >> -- > > > >> Sylvain > > > >> > > > >> > > > >> On Thu, Apr 30, 2020 at 4:11 PM Jon Haddad <j...@jonhaddad.com> wrote: > > > >> > > > >>> I've got a bit of experience here using all three systems we're > > > >> discussing > > > >>> here. > > > >>> > > > >>> * rst & sphinx: I've handled most of the doc reviews for Cassandra, > > > >> written > > > >>> quite a bit of them as well, and I authored most of the documentation > > > for > > > >>> cqlengine [1] > > > >>> * markdown: all over the place, let's be honest, it's ubiquitous. > > > Within > > > >>> the community I built the Reaper website [2], the docs are all > > markdown > > > >> and > > > >>> work fine. Source [3] if you're interested. > > > >>> * asciidoctor: tlp-stress [3] (src [4]) and tlp-cluster [5] (src > > [6]) > > > >>> were *extremely* nice to use. My favorite part here was the Gradle > > > >> plugin > > > >>> to generate documentation making it a breeze to integrate into my > > build > > > >>> system. > > > >>> > > > >>> You're right about markdown being a little limited, but we're not > > > really > > > >>> using anything advanced in sphinx. We write basic text with links > > and > > > a > > > >>> menu. I don't know if that will ever change, but given my experience > > > >> with > > > >>> Hugo + markdown on the reaper website, I'd say it's perfectly fine > > for > > > >>> writing technical documentation. I also write a *lot* of > > documentation > > > >> for > > > >>> clients at TLP, which was all technical writing. I would regularly > > > >> deliver > > > >>> 30-60 page cluster analysis documents written in markdown with > > tables, > > > >> CQL, > > > >>> and code. > > > >>> > > > >>> I absolutely love asciidoc. Moving from markdown was really, really > > > >> easy. > > > >>> In fact, most markdown will render properly in asciidoctor. The > > > >>> documentation is excellent and it's designed for writing. I even > > have > > > a > > > >>> (private) repo where I'm writing a cookbook, something that requires > > > just > > > >>> as much attention to detail and flexibility as technical writing. > > > Take a > > > >>> look at the quick reference [7] to see some examples (this is my go > > to > > > >>> document if I forget the syntax). The quick ref here is *so good* > > that > > > >> it > > > >>> only takes a second if you can't remember what you need. > > > >>> > > > >>> rst & sphinx have poor documentation (imo) in comparison. I find the > > > >>> syntax to be difficult to get right at times. Tables [8], for > > > instance, > > > >>> are particularly awful, whereas in markdown or asciidoc they're a > > dream > > > >> in > > > >>> comparison [9]. I recently wrote the production recommendations page > > > [10] > > > >>> for C* and was *struggling* with the tables. It's like trying to > > write > > > >>> code with a stick in the ground after using IDEA. > > > >>> > > > >>> I'm not sure how you're calculating ROI, or why. There are people > > > >> willing > > > >>> to do the work, and nobody is asking you to. I'm willing to lead the > > > >>> effort and work with the technical writers at datastax to make this > > > >>> happen. The investment cost is irrelevant to the project because > > time > > > is > > > >>> donated, and unless you're someone's manager it shouldn't matter how > > > much > > > >>> time they invest. Even if you are, that's a private matter not > > > relevant > > > >> to > > > >>> the list. If the writers are willing to help migrate documentation, > > > I'm > > > >>> willing to shepherd the process, and I absolutely love that they're > > > >> willing > > > >>> to help in this area. > > > >>> > > > >>> From a technical angle, I ask that you trust my experience and > > > judgement > > > >>> using all three tools extensively over a long period of time (3 years > > > >>> minimum, 10 years of rst). I have written thousands of pages of > > > >> technical > > > >>> documentation in my life and I understand the pros and cons of all > > > three > > > >>> systems and have weighed the costs of each of them for the last > > several > > > >>> months. Otherwise, you're asking for the rest of the project to wait > > > >> while > > > >>> you become an expert in the remaining tooling. I doubt you have the > > > time > > > >>> (or interest) in doing that. > > > >>> > > > >>> I know, without question, asciidoctor will give us the richest > > > >>> documentation with a very quick learning curve, so it's my personal > > > >>> preference. I'm 100% sure someone someone that is already familiar > > > with > > > >>> markdown will find it easy to move to asciidoc since they're so > > similar > > > >> in > > > >>> structure and the syntax is mostly compatible. > > > >>> > > > >>> I understand you don't want to see the project docs fall into a state > > > of > > > >>> disrepair and as the person maintaining most of the docs today, I > > > >> agree! I > > > >>> remember you did the initial work because I created the JIRA to do > > so. > > > >>> We've both invested a lot of time in the docs, so hopefully you trust > > > >> that > > > >>> I don't take this lightly and wouldn't want to make the change > > without > > > >>> expecting to see a big payoff in the end. > > > >>> > > > >>> Jon > > > >>> > > > >>> [1] https://cqlengine.readthedocs.io/en/latest/ > > > >>> [2] http://cassandra-reaper.io > > > >>> [3] http://thelastpickle.com/tlp-stress/ > > > >>> [4] > > > >>> > > > >> > > > > > https://github.com/thelastpickle/tlp-stress/blob/master/manual/MANUAL.adoc > > > >>> [5] > > > >>> > > > >>> > > > >> > > > > > https://github.com/thelastpickle/tlp-cluster/blob/master/manual/src/index.adoc > > > >>> [6] http://github.com/thelastpickle/tlp-cluster > > > >>> [7] https://asciidoctor.org/docs/asciidoc-syntax-quick-reference/ > > > >>> [8] > > https://docutils.sourceforge.io/docs/user/rst/quickref.html#tables > > > >>> [9] > > > https://asciidoctor.org/docs/asciidoc-syntax-quick-reference/#tables > > > >>> [10] https://issues.apache.org/jira/browse/CASSANDRA-8700 > > > >>> > > > >>> > > > >>> On Thu, Apr 30, 2020 at 6:05 AM Sylvain Lebresne <lebre...@gmail.com > > > > > > >>> wrote: > > > >>> > > > >>>> I do worry Markdown might not be a great choice. > > > >>>> > > > >>>> It's definitively most well know by a large margin, and that's a > > good, > > > >>> but > > > >>>> it's also a bit limited (even with common extensions). It's perfect > > > for > > > >>>> comments, README or even somewhat informal docs, but we're talking > > the > > > >>>> fairly > > > >>>> large (and meant to grow) user facing documentation of a large and > > > >>> somewhat > > > >>>> complex software, and a bit more flexibility is of definite use > > imo. I > > > >>>> truly > > > >>>> worry Markdown will effectively yield a lesser experience for user > > of > > > >> the > > > >>>> doc. > > > >>>> > > > >>>> By how much, I'm not sure, but insofar that the documentation is > > read > > > >>> order > > > >>>> of > > > >>>> magnitudes more (and by order of magnitudes more people) than > > written, > > > >>> I'm > > > >>>> not > > > >>>> a fan of shrugging this off too quickly. > > > >>>> > > > >>>> Regarding asciidoc, it looks most likely capable enough, and I have > > no > > > >>>> technical > > > >>>> objection to its use on principle. But I'm also unconvinced it's a > > > >>>> significantly better > > > >>>> choice than restructuredText (currently used). Both syntax are > > > >> different > > > >>>> enough from Markdown that there is a bit of muscle memory to > > retrain, > > > >> but > > > >>>> both are also easy enough in general (it's all human readable > > markup) > > > >>> that > > > >>>> it > > > >>>> doesn't feel like a huge deal either. And while it's hard to get > > > >> perfect > > > >>>> data > > > >>>> on this, a simple Google trends search > > > >>>> ( > > > >>>> > > > >>>> > > > >> > > > > > https://trends.google.fr/trends/explore?date=today%205-y&q=markdown,asciidoc,restructuredText > > > >>>> ) > > > >>>> suggests that while asciidoc is a tad more popular (than rst), > > neither > > > >>> are > > > >>>> ubiquitous enough for me to imagine that it'd make a meaningful > > > >>> difference. > > > >>>> I'm really not against asciidoc, but also keep in mind the current > > doc > > > >>> has > > > >>>> had > > > >>>> some amount of setup: it's somewhat integrated to the website > > (though > > > >>> I'll > > > >>>> admit it's debatable whether that's important to preserve or not), > > > >>>> automatic > > > >>>> syntax highlighting for CQL is already setup, etc. Switching to > > > >> asciidoc > > > >>> is > > > >>>> not "no work". Are we sufficiently certain it is worth it? > > > >>>> > > > >>>> Tl;dr, my current position is: > > > >>>> 1. I'm rather cold on using markdown. I would insist on making a > > good > > > >>> case > > > >>>> this won't meaningfully degrade the output quality before > > jumping > > > >>> ship. > > > >>>> 2. I see the ROI of switching to asciidoc as rather small (the > > > >> investment > > > >>>> is > > > >>>> non null, and the return not that clear to me, though I > > obviously > > > >> may > > > >>> be > > > >>>> missing some of the advantages of asciidoc over reStructuredText > > > and > > > >>>> will, > > > >>>> as always, happily re-evaluate on new information). It won't > > > oppose > > > >> it > > > >>>> if > > > >>>> someone makes the work (and it's not botched), but I think the > > > >> effort > > > >>>> would > > > >>>> be better spent elsewhere. > > > >>>> -- > > > >>>> Sylvain > > > >>>> > > > >>>> > > > >>>> On Thu, Apr 30, 2020 at 5:02 AM John Sanda <john.sa...@gmail.com> > > > >> wrote: > > > >>>>> +1 to asciidoc. My guess would be that more people are familiar > > with > > > >>>>> markdown, but asciidoc definitely has more to offer and is easy > > > >> enough > > > >>> to > > > >>>>> use if you are familiar with markdown. > > > >>>>> > > > >>>>> On Fri, Apr 24, 2020 at 11:24 AM Jon Haddad <j...@jonhaddad.com> > > > >> wrote: > > > >>>>>> I'd like to get the docs out of Sphinx. I hate it. The syntax is > > > >>> crap > > > >>>>> and > > > >>>>>> almost nobody knows it. > > > >>>>>> > > > >>>>>> I'm fine with markdown (it makes it easy for most people) and have > > > >> a > > > >>>>>> personal preference for asciidoc, since it makes it easier to > > > >>> generate > > > >>>>> PDFs > > > >>>>>> and is a bit richer / better for documentation. I'd go with > > > >> markdown > > > >>>> if > > > >>>>> it > > > >>>>>> means more contributions though. > > > >>>>>> > > > >>>>>> I'd love to see the site maintained with Hugo. It's a really nice > > > >>>> tool, > > > >>>>> I > > > >>>>>> used it to build the reaper website [1] and the docs [2]. Source > > > >>>> example > > > >>>>>> for documentation here [3]. > > > >>>>>> > > > >>>>>> I won't have time for the conversion anytime soon, but if > > someone's > > > >>>>> willing > > > >>>>>> to take it on I think it would really help with both writing and > > > >>>>> reviewing > > > >>>>>> docs. > > > >>>>>> > > > >>>>>> [1] http://cassandra-reaper.io > > > >>>>>> [2] http://cassandra-reaper.io/docs/ > > > >>>>>> [3] > > > >>>>>> > > > >>>>>> > > > >> > > > > > https://github.com/thelastpickle/cassandra-reaper/blob/master/src/docs/content/docs/development.md > > > >>>>>> > > > >>>>>> > > > >>>>>> > > > >>>>>> > > > >>>>>> > > > >>>>>> > > > >>>>>> On Fri, Apr 24, 2020 at 8:11 AM Joshua McKenzie < > > > >>> jmcken...@apache.org> > > > >>>>>> wrote: > > > >>>>>> > > > >>>>>>> All, > > > >>>>>>> > > > >>>>>>> A few of us have the opportunity to offer a large portion of > > > >>>>>> documentation > > > >>>>>>> to the apache foundation and specifically the Apache Cassandra > > > >>>> project > > > >>>>> as > > > >>>>>>> well as dedicate a good portion of time to maintaining this going > > > >>>>>> forward. > > > >>>>>>> For those of you familiar, this is the DataStax sponsored / > > > >>> authored > > > >>>>>>> Cassandra documentation people often refer to in the community. > > > >>> Links > > > >>>>> can > > > >>>>>>> be found here > > > >>>>>>> < > > > >> > > > > > https://docs.datastax.com/en/landing_page/doc/landing_page/cassandra.html > > > >>>>>>>> . > > > >>>>>>> I've spoken with some of the doc writers and there's going to be > > > >>>>>>> significant work involved to go from the doc writing system these > > > >>> are > > > >>>>>>> authored in to Sphinx, or some other doc authoring system if we > > > >> as > > > >>> a > > > >>>>>>> project decide to switch things. I know Jon Haddad has some > > > >>> opinions > > > >>>>> here > > > >>>>>>> and I think that'd be a great conversation to have on this thread > > > >>> for > > > >>>>>> those > > > >>>>>>> interested. A couple of people in the near future are going to > > > >> have > > > >>>> the > > > >>>>>>> opportunity to continue working on these docs full-time in the > > > >>>> in-tree > > > >>>>>>> docs, so maintenance going forward should represent little > > > >>> disruption > > > >>>>> to > > > >>>>>>> the project's workings day-to-day. > > > >>>>>>> > > > >>>>>>> Looking for feedback on: > > > >>>>>>> > > > >>>>>>> 1. > > > >>>>>>> > > > >>>>>>> Are there any questions or concerns about this donation? > > > >>>>>>> 2. > > > >>>>>>> > > > >>>>>>> Any thoughts on documentation system to use long-term, since > > a > > > >>>>>> donation > > > >>>>>>> of this size would be a reasonable time to consider switching > > > >> to > > > >>>>>>> something > > > >>>>>>> more preferable (not advocating for the system these current > > > >>> docs > > > >>>>> are > > > >>>>>>> in to > > > >>>>>>> be clear - poking Haddad to speak up since he has a strong > > PoV > > > >>>> here > > > >>>>>> ;) ) > > > >>>>>>> 3. > > > >>>>>>> > > > >>>>>>> What are next steps? > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> I'm genuinely excited about this; here's to hoping everyone else > > > >> is > > > >>>>> too! > > > >>>>>>> > > > >>>>>>> ~Josh > > > >>>>>>> > > > >>>>> > > > >>>>> -- > > > >>>>> > > > >>>>> - John > > > >>>>> > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > > > > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
