Re: Bug#932957: Please migrate Release Notes to reStructuredText
Hi, Stéphane Blondon wrote (Sun, 28 May 2023 13:16:24 +0200): > > > Richard Lewis wrote (Fri, 19 May > > 2023 00:58:26 +0100): > > > > > - are the red hyphens in eg the 'deb...' line near the top of > > > > > > https://people.debian.org/~holgerw/release-notes_sphinx/en/html/issues.html > > > > meant to be red? (maybe it is a syntax error?) > > > > > > > Sphinx uses Pygments to highlight source code. I guess no language is > defined so Sphinx uses a wrong lexer. We should force Sphinx to use > 'SourceListLexer' with: > .. code-block :: sources.list The highlighting feature is great, thanks for pointing this out! However, we cannot use it here in most cases for sources.list entries, since resolving substitutions does not work within such lines :-( like in deb https://deb.debian.org/debian |RELEASENAME| main contrib But in many cases the highlighting gives us a great benefit, so I will merge your MR and adapt the rare cases, where it does not work. Thanks again! Holger -- Holger Wansing PGP-Fingerprint: 496A C6E8 1442 4B34 8508 3529 59F1 87CA 156E B076
Re: Bug#932957: Please migrate Release Notes to reStructuredText
> > Richard Lewis wrote (Fri, 19 May > 2023 00:58:26 +0100): > > > - are the red hyphens in eg the 'deb...' line near the top of > > > > https://people.debian.org/~holgerw/release-notes_sphinx/en/html/issues.html > > > meant to be red? (maybe it is a syntax error?) > > Sphinx uses Pygments to highlight source code. I guess no language is defined so Sphinx uses a wrong lexer. We should force Sphinx to use 'SourceListLexer' with: .. code-block :: sources.list
Re: #932957 Please migrate Release Notes to reStructuredText
[ Sending this to #932957 as well, as it contains valueable info on that topic ]
Jeremy Stanley wrote (Wed, 24 May 2023 18:22:09 +):
> On 2023-05-24 19:40:56 +0200 (+0200), Agathe Porte wrote:
> [...]
> > Maybe the idea was to introduce %OLD_RELEASE_NAME% and to call sed to
> > replace this kind of strings in a build step, and not rely on sphinx
> > substitution at all.
> >
> > I know that I have done this a few times and it works fine as a very
> > simple preprocessor.
>
> Similar things can be done at sphinx-build time with a custom Sphinx
> extension (can be a trivial Python module). We do that for the
> published security advisories list upstream in OpenStack:
>
> https://opendev.org/openstack/ossa/src/commit/136b24c/doc/source/conf.py#L31
>
> https://opendev.org/openstack/ossa/src/commit/136b24c/doc/source/_exts/vmt.py
>
> That's a more complex example because we generate a ton of content
> out of structured data (YAML) files by splatting the relevant fields
> into substitutions in a Jinja2 template, but for basic string
> substitution you could get away with something far simpler, or even
> use a canned one like (this was simply my first web search result):
>
> https://pypi.org/project/Sphinx-Substitution-Extensions/
>
> The examples in its readme look to be along the lines of the Debian
> Release Notes use case anyway. There's also basic substitutions
> support in the reStructuredText specification, which might be useful
> to reduce the amount of actual content you need to swap at build
> time:
>
> https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html#substitution-definitions
The above seem related to the issue in question, but the solution pointed
out by James Addison in
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=932957#90
(use parsed-literal ('.. parsed-literal::') blocks)
seems to be the easier one (at least for this document) and it works fine
here. So I would go for it.
Thanks anyway
Holger
--
Holger Wansing
PGP-Fingerprint: 496A C6E8 1442 4B34 8508 3529 59F1 87CA 156E B076
Re: #932957 Please migrate Release Notes to reStructuredText
On 2023-05-24 19:40:56 +0200 (+0200), Agathe Porte wrote: [...] > Maybe the idea was to introduce %OLD_RELEASE_NAME% and to call sed to > replace this kind of strings in a build step, and not rely on sphinx > substitution at all. > > I know that I have done this a few times and it works fine as a very > simple preprocessor. Similar things can be done at sphinx-build time with a custom Sphinx extension (can be a trivial Python module). We do that for the published security advisories list upstream in OpenStack: https://opendev.org/openstack/ossa/src/commit/136b24c/doc/source/conf.py#L31 https://opendev.org/openstack/ossa/src/commit/136b24c/doc/source/_exts/vmt.py That's a more complex example because we generate a ton of content out of structured data (YAML) files by splatting the relevant fields into substitutions in a Jinja2 template, but for basic string substitution you could get away with something far simpler, or even use a canned one like (this was simply my first web search result): https://pypi.org/project/Sphinx-Substitution-Extensions/ The examples in its readme look to be along the lines of the Debian Release Notes use case anyway. There's also basic substitutions support in the reStructuredText specification, which might be useful to reduce the amount of actual content you need to swap at build time: https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html#substitution-definitions -- Jeremy Stanley signature.asc Description: PGP signature
Re: #932957 Please migrate Release Notes to reStructuredText
Hi, 2023-05-20 23:36 CEST, Holger Wansing: > Hi, > > RL wrote (Sat, 20 May 2023 12:52:25 > +0100): > > Holger Wansing writes: > > > > > However, I may have some objections against the migration at all: > > > as far as I know, sphinx/reStructuredText is still lacking some > > > functionality, > > > which is heavily used in the release-notes. > > > That is the use of substitutions within URLs. > > > In docbook speach these were entities, and you could use them in URLs > > > like this: > > > > > > Please follow the instructions in the > > > > > url="https://www.debian.org/releases/&oldreleasename;/releasenotes";>Release > > > Notes for &debian; &oldrelease; to upgrade to &debian; > > > &oldrelease; first if needed. > > > > > > Please note the &oldreleasename; in the URL! > > > I could not get this working with sphinx (if someone knows better, please > > > contact me!) > > > > No idea about sphinx, but we could we just run a simple sed script to update > > That will not work. > You cannot replace all 'bullseye' by 'bookworm' for example. > There are places, where the 'bullseye' needs to stay. > So that needs to be done selective one-by-one. Maybe the idea was to introduce %OLD_RELEASE_NAME% and to call sed to replace this kind of strings in a build step, and not rely on sphinx substitution at all. I know that I have done this a few times and it works fine as a very simple preprocessor.
Re: Bug#932957: Please migrate Release Notes to reStructuredText
[[ Replying to two mails in one ]] Hi, Holger Wansing wrote (Sat, 20 May 2023 23:26:47 +0200): > James Addison wrote (Fri, 19 May 2023 23:28:55 +0100): > > > Please note the &oldreleasename; in the URL! > > > I could not get this working with sphinx (if someone knows better, please > > > contact me!) > > > > Could the 'extlinks' feature[1] of Sphinx be helpful to migrate those? > > Yes, thanks for the pointer! I used that now to get the manpage links fixed! > > > (it allows defining URL pattern aliases, so for example :oldreleasenotes: > > could > > map to https://www.debian.org/releases/bullseye/releasenotes and > > :releasenotes: > > could map to https://www.debian.org/releases/bookworm/releasenotes for D12) > > > > Parameters are supported too, and that could be useful for > > package-or-filepath > > refs (re: grep -r '`.*<' source |grep -o '' | sort | uniq -c |sort > > -n) > > The drawback is, it is not as flexible as the entities in docbook. > For example, there is the following URL in this manual in docbook: > > &url-debian-mirror-eg;/debian/dists/&releasename;/main/binary-&architecture;/... > > You see, there are three entites in this URL! > This seems to be not supported by extlinks :-(( I focused a bit too much on the URL front here. There is also a problem using entities (or now called substitutions) in quoted lines like deb https://deb.debian.org/debian RELEASENAME main contrib or # script -t 2>~/upgrade-RELEASENAMEstep.time -a ~/upgrade-RELEASENAMEstep.script These also do not work. That's the biggest blocker in the upgrading chapter IMHO. > > > Beside this, I need help to adapt the buildchain, to get the possibility > > > of > > > building the release-notes for the different architectures. > > > I have no python knowledge, so I will most likely not get this running > > > myself. > > > > I'll try to take a look into that soon to see what I can find out. Do you > > know how the current docbook-based build varies by architecture? > > There are some chapters, which are only visible for specific architectures, > like > in installing.dbk: > > > Cloud installations > > The cloud team publishes > Debian &releasename; for several popular cloud computing services > including: > > > > Perhaps we can borrow some build scripting from developers-reference and/or > > debian-policy.. but I suppose those don't have per-architecture output. > > I think so, yes. That will be of no help, I fear... > > > > And the last point is the integration into the debhelper tools: I don't > > > know > > > if it is required, to have the release-notes fit for building as a whole > > > package with sbuild or debuild or similar. Salsa tries to build it via CI > > > at every push, but currently fails. > > > > It's possible I've misunderstood, but would be the goals here to be to get > > the release-notes documentation built by CI, and also for it to be > > distributed > > as a Debian package itself? > > > > (someone who knows more may correct me, but I think it would be great to > > have > > the package available for install using apt in addition to the website) > > That was my first thought as well, yes. Hi, Holger Wansing wrote (Sat, 20 May 2023 23:10:59 +0200): > Hi, > > Richard Lewis wrote (Fri, 19 May 2023 > 00:58:26 +0100): > > On Thu, 18 May 2023 22:39:11 +0200 Holger Wansing > > wrote: > > Unfortunately, my first impression is that it the output has quite a > > few issues which make it a lot harder to read than > > the docbook version - which im sure is because it's still only a > > prototype, but thought it might helpful to list the things that jumped > > out at me: > > - It is a lot more cluttered than the docbook version - it feels > > off-putting and dense to read > > - it's all a bit 'blue' - i'd suggest red is more on-brand for debian > > - the "next"/"prev" links at the bottom-right are white on green --- > > I totally missed at first, and found hard to read > > I copied style and font settings from developers-reference, another > document, which has been migrated recently from docbook to sphinx. > So I consider this as a quasi-standard at the moment. > I copied it by intent, to get a consistent look for all manuals generated > with sphinx. > We can work on that later on, as Laura already pointed out there is even > a bugreport for this. > So will ignore this for now here. > > > - i was a bit confused by the "12.1" version number at the bottom of > > every page, and having 'sphinx' reminded me of websites with "hosted > > by geocities" > > The first version I built with sphinx had version displayed > "release-notes 5.0.1" which confused me even more. > Therefore I changed that in the sense of "release-notes version 12 for > Debian release 12". Can still be changed in any way... > > > - are the red hyphens in eg the 'deb...' line near the top of > > https://people.debian.org/~holgerw/releas
Re: #932957 Please migrate Release Notes to reStructuredText
Hi, RL wrote (Sat, 20 May 2023 12:52:25 +0100): > Holger Wansing writes: > > > However, I may have some objections against the migration at all: > > as far as I know, sphinx/reStructuredText is still lacking some > > functionality, > > which is heavily used in the release-notes. > > That is the use of substitutions within URLs. > > In docbook speach these were entities, and you could use them in URLs like > > this: > > > > Please follow the instructions in the > > > url="https://www.debian.org/releases/&oldreleasename;/releasenotes";>Release > > Notes for &debian; &oldrelease; to upgrade to &debian; > > &oldrelease; first if needed. > > > > Please note the &oldreleasename; in the URL! > > I could not get this working with sphinx (if someone knows better, please > > contact me!) > > No idea about sphinx, but we could we just run a simple sed script to update That will not work. You cannot replace all 'bullseye' by 'bookworm' for example. There are places, where the 'bullseye' needs to stay. So that needs to be done selective one-by-one. > i've submitted several patches to release notes, and found the use of > entities and XML more confusing than helpful (especially '&debian;' for > 'Debian') - this is meant to be a document for people, and needs a > decent proof-read anyway. I dropped that &debian; entity already in Sphinx for the release-notes :-) Holger -- Holger Wansing PGP-Fingerprint: 496A C6E8 1442 4B34 8508 3529 59F1 87CA 156E B076
Re: #932957 Please migrate Release Notes to reStructuredText
Holger Wansing writes: > However, I may have some objections against the migration at all: > as far as I know, sphinx/reStructuredText is still lacking some functionality, > which is heavily used in the release-notes. > That is the use of substitutions within URLs. > In docbook speach these were entities, and you could use them in URLs like > this: > > Please follow the instructions in the > url="https://www.debian.org/releases/&oldreleasename;/releasenotes";>Release > Notes for &debian; &oldrelease; to upgrade to &debian; > &oldrelease; first if needed. > > Please note the &oldreleasename; in the URL! > I could not get this working with sphinx (if someone knows better, please > contact me!) No idea about sphinx, but we could we just run a simple sed script to update i've submitted several patches to release notes, and found the use of entities and XML more confusing than helpful (especially '&debian;' for 'Debian') - this is meant to be a document for people, and needs a decent proof-read anyway.
Re: #932957 Please migrate Release Notes to reStructuredText
[[ debian-devel in CC, to get a wider audience regarding reStructuredText ]] Hi, I worked on this recently, and I have something like a prototype ready. It can be found (as html) at https://people.debian.org/~holgerw/release-notes_sphinx/ while the git repo containing the migration is at https://salsa.debian.org/holgerw/release-notes However, I may have some objections against the migration at all: as far as I know, sphinx/reStructuredText is still lacking some functionality, which is heavily used in the release-notes. That is the use of substitutions within URLs. In docbook speach these were entities, and you could use them in URLs like this: Please follow the instructions in the https://www.debian.org/releases/&oldreleasename;/releasenotes";>Release Notes for &debian; &oldrelease; to upgrade to &debian; &oldrelease; first if needed. Please note the &oldreleasename; in the URL! I could not get this working with sphinx (if someone knows better, please contact me!) In sphinx, I used hardcoded codenames like https://www.debian.org/releases/bullseye/releasenotes instead, which means, that there is much work to do to make the release-notes fit for the next release, while with docbook you only need to change the entity in one place !!! In sphinx you need to change every single occurence, and don't forget the translations !!! Beside this, I need help to adapt the buildchain, to get the possibility of building the release-notes for the different architectures. I have no python knowledge, so I will most likely not get this running myself. And the last point is the integration into the debhelper tools: I don't know if it is required, to have the release-notes fit for building as a whole package with sbuild or debuild or similar. Salsa tries to build it via CI at every push, but currently fails. However, there is no package "release-notes" in the archive, so currently it is only a matter of building it on wolkenstein for www.debian.org, right? Regards Holger -- Holger Wansing PGP-Fingerprint: 496A C6E8 1442 4B34 8508 3529 59F1 87CA 156E B076
