Am 27.09.21 um 13:46 schrieb Andrew David Wong:
## Summary
Currently, the Qubes documentation is written in plain text Markdown
files, version controlled with Git, rendered using Jekyll, and hosted
from GitHub Pages. This setup has served us well over the years, but as
the demands on the documentation have grown, it has become increasingly
difficult for the current setup to meet our needs. Therefore, it may be
time to consider migrating to a dedicated platform, such as Read the
Docs or a standard wiki. The purpose of this thread is to help us make
the best decision for our users and for the project as a whole.
## Pros and cons of the status quo
There are good reasons for our current setup:
1. History of changes is transparent and immutable thanks to Git.
2. Authenticity can be verified thanks to PGP-signed commits and tags.
3. Unauthorized changes are prohibited thanks to signature-checking.
4. Plain text files can be read directly, offline, even in a terminal.
5. Full control over web-view styling with custom CSS.
And more...
However, there are also challenges:
6. Difficult for many users to contribute. Requires use of GitHub's web
editor at the very least or local command-line usage for full control.
No WYSIWYG editor. (GitHub offers a Markdown preview, but you have to
serve the site locally to see how your changes will really look on the
live website.)
7. Serving the website on one's local machine is cumbersome and
error-prone. Upstream changes often break the workflow. Even users who
have all the required software installed and follow the exact same steps
we do sometimes can't get it to work.
We used to instruct users to install Jekyll and run it directly to serve
the website, but they often couldn't get it to work. We thought that
switching to Docker/Podman would solve this "works on my machine but
doesn't work on yours" problem, but it still often doesn't work out in
practice. (Currently, even I'm having trouble getting it to work
consistently on my own development machine.)
8. Requires a complex backend to achieve basic functionality that's
standard elsewhere. Since we're hosted from GitHub Pages, we can't use
most Jekyll plugins, which means we have to write and maintain custom
code for basic things like per-page tables of contents, language
switching, and version switching.
To support translations, we've had to drastically increase the
complexity of all the layouts, includes, and data files that constitute
the guts of a Jekyll website. We've had to add a lot of Liquid code in
these files to do the kinds of processing we need, and we have to hook
it all up to complicated series of Git repos, subrepos, and branches.
This makes it very difficult to understand and maintain.
Anyone who wants to contribute or help out with maintenance faces a
steep learning curve, which discourages contributions and changes. Even
an existing expert in Git, Jekyll, Liquid, and Markdown would have to
become acquainted with all of our custom code. And most of it remains
undocumented, as many of the people who built it had to rush off to
other tasks due to lack of time.
9. Related to the previous point, adding new functionality becomes
increasingly difficult as the system grows more complex. The latest
example is release-specific documentation. There are many small design
decisions that have to be made, and it all has to be done in a way
that's compatible with translations. We can't simply create a new branch
and call it a day, because all the customized guts of the Jekyll system
have to be changed to account for multiple versions of documentation,
and the documentation files themselves are a mix of release-specific and
non-release-specific files which have to be reorganized and classified
with new metadata.
Jekyll doesn't support variables in YAML, so we can't just add a release
variable to our existing YAML files and headers. We have to either
duplicate them for every release or fundamentally redesign large swaths
of the system with layers of indirection to work around such
limitations. Even bulk-manipulation of the YAML headers in all the doc
files requires us to write our own special tools, which themselves then
have to be maintained.
And all of this only gets us to the point where different doc sets can
simply *exist* without everything breaking. Exposing them to users in a
friendly way would be an entire further stage of the process, in which
we'd probably have to write a lot more custom code just to get a simple
dropdown menu that allows users to switch between versions. All of this
work would probably result in something that's still not as good as
existing off-the-shelf solutions backed by dedicated teams who
specialize in developing documentation management systems.
## Why now?
We've had the current setup for a long time. Why should we consider
migrating now? We've considered it in the past and decided against it at
the time, so what's changed? The answer is that there are new and
increasing demands on the documentation, the main two being localization
(i.e., translations) and release-specific docs.
Back before we tried to address these kinds of needs, our documentation
setup was simpler and served us well. The backend was smaller, easier to
understand, and easier to maintain, since it didn't need to do much.
There was only a single, canonical version of every documentation page.
The pros mentioned above easily outweighed the cons, since the cons list
was much shorter back then.
However, times change. As our project has matured, we've decided it was
time to acknowledge and take seriously the reality that not everyone
reads English and not everyone uses the same version of Qubes OS at the
same time. We have users and contributors from around the world, and we
have multiple versions of Qubes.
There are times when an older Qubes release is supported concurrently
with a newer release to give users time to upgrade. Even when there's
only one currently-supported release, there's usually a new one in
development and testing that folks want to start documenting. This means
that we have to support a separate version of each appropriate
documentation page for every translated language *and* for every
supported and in-development Qubes version, not to mention historical
documentation for EOL releases.
## What are our options?
Looking around at other projects similar to ours, it appears that two of
the most popular options are Read the Docs (RTD) and standard wiki
software. Here are what I currently understand to be some of the pros
and cons of each. (I haven't personally used RTD very much, so some of
this is based on hearsay. I'm sure these lists will change as we learn
more.)
RTD:
+ Built-in release-specific doc support.
+ Built-in localization support.
+ Supports Git for version control and history. (Presumably, we could
continue to use signed commits and tags for user and content
authentication.)
+ GitHub integration.
+ Supports Markdown (natively, AFAIK, though many seem to use RST). We
currently use GitHub-flavored Markdown, so some conversion may be
necessary but shouldn't be too bad.
+ Built-in support for downloading/exporting docs in multiple formats
for offline reading. Direct access to human-readable Markdown source
files in Git repos should also continue to be possible.
+ Includes search functionality.
+ Widespread use among open-source projects. Established history.
Probably not going away anytime soon (though it could eventually, e.g.,
if the project behind it ever shuts down).
+ Can be self-hosted.
? Unclear exactly how easy it'll be for users to contribute. (Almost
certainly no more difficult than it is now.)
? [Subjective] Some people think RTD looks better than a wiki.
? Might look different from the rest of our website. Might require a
subdomain.
Wiki:
+ Familiar to almost everyone thanks to Wikipedia and specialized wikis.
+ Easy for users to contribute.
+ Built-in version control and history.
+ Includes search functionality.
+ Even more widespread use throughout the web. Even longer established
history. Practically ubiquitous. Definitely not going away anytime soon.
+ Can be self-hosted.
? Unclear how good the security would be with respect to user and
content authentication.
? Unclear to what extent localization support is built-in. There are
probably plugins. Unclear how easy Transifex integration would be.
? Would probably look different from the rest of our website. Might
require a subdomain.
? [Subjective] Some people think wikis look bad.
- No built-in support for release-specific docs (AFAIK).
- No (common, standard, native) Git support.
- Native wiki syntax is different from Markdown. Markdown plugins are
available but can be problematic. Markdown can be converted into wiki
syntax using, e.g., pandoc, but will probably still require some manual
cleanup. However, once this conversion is done, contributions could be
written natively in wiki syntax with no ongoing conversions required.
- Probably wouldn't easily support direct access to human-readable
source files. Offline docs might require a local wiki instance.
## Conclusion
Overall, I'm leaning toward RTD, but I don't yet know what it'll be like
to maintain the docs there, so I'd like to experiment with it a bit
before forming an opinion. Of course, we're also open to considering
other options (including the option of not migrating at all). Finally,
although I've been saying "documentation" throughout this post, we may
also want to consider migrating the entire website if it turns out that
keeping everything together makes more sense.
So, what do you all think? Again, the purpose of this thread is to help
us make the best decision for our users and for the project as a whole.
In addition to the user-facing aspects, I'd also like us to consider
long-term maintenance requirements and the experience for contributors.
Hello,
This discussion here contains a lot of Pros and Cons but no decision. Is
there already a silent decision towards "no"? At least, I cannot see any
action towards "yes".
Best regards,
Tobias Killer
--
You received this message because you are subscribed to the Google Groups
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to qubes-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/qubes-devel/5a5a95a7-a2df-4769-ec02-25451866f9d4%40posteo.de.
