A follow-up: after doing more work to try to confirm the behaviour of the fix -- something I should have done before even starting development! -- I was confused that I couldn't replicate the original problem when using a version of the codebase _before_ my proposed fix pr#10949 was applied.
I now believe that the issue had, in fact, already been fixed between versions v4.5.0 and v5.0.0 of Sphinx - so I've offered a revert of my changes. Details about tracing the location of the existing fix (using 'git bisect') can be found here: https://github.com/sphinx-doc/sphinx/issues/9778#issuecomment-1501172176 Attempting to learn from this: I find the experience interesting because it seems that I deluded myself into believing that I'd resolved a bug -- and unfortunately drew in a bunch of other people's time doing that -- when, in fact, following good time-and-vesion-aware engineering practices (not always easy when keen to contribute to a project, but as demonstrated here, potentially very important) could have avoided much of that work in the first place. The original bugreport was detailed and quite clear, so I think that much of the fault here was from me not carefully reading and considering how to proceed. There could be other learnings / recommendations (I'm mulling over some ideas related to continuous-bug-checking), but I'm unable to make any clear recommendations there yet, partly because I think that bug-checking-and-verification, while valuable, is not always considered desirable or economically-beneficial work -- and partly because writing test cases to reproduce bugs (which could make continuous evaluation of stale bugs easier) often performs 80%+ (guestimate, in my opinion) of the work to find the cause of the bug -- meaning that it's frequently worthwhile to combine with fixing the bug (in other words: there's often a significant overlap between developing a bug reproduction test case and fixing the bug). Probably nothing new to many of the folks on this mailing list and/or seasoned software engineers generally, but I figured I'd try to document my findings :) On Sat, 8 Apr 2023 at 11:10, James Addison <j...@jp-hosting.net> wrote: > > Hi folks, > > A set of reproducible-build-related changes[1] that I've developed for > sphinx (a documentation project generator) have been accepted for > inclusion in v6.2.0 of sphinx. > > I'm optimistic that those changes can address a sizable category[2] of > reproducible build failures related to translation of documentation > during software builds (reproducible build testing intentionally > varies the host LANGUAGE setting to shake out unintended sources of > build variation, and some sphinx projects fail rb-tests due to that). > > However.. with the changes merged (although not yet released) I'm > beginning to have some doubts about them. > > The positive effect of the changes is that I expect they'll help to > confirm and achieve reproducibility for a good chunk of remaining > non-reproducible software. > > The downside is: disabling localization -- or perhaps more accurately: > emitting all documentation for each project using 'null'[3] > translation -- seems like a fairly blunt, and perhaps unwelcome (for > consumers) way to achieve reproducibility. > > > A longer, better path to achieve reproducibility would be to support > building documentation in _all_ available translated locales during > Sphinx project builds (something that is not yet supported -- and with > at least one component that I'm aware of (objects.inv) that doesn't > seem to support multi-language content). Doing that should produce > output artifacts that users of any supported locale can access in a > relevant localised way, and that can be made bit-for-bit consistent. > > In summary: I'm writing partly optimistically because I think the > merged changes could improve and help to confirm reproducibility of > software during testing. But I also feel a bit conflicted about the > way I've approached the changes and their implications, so I'm also > keen to gather feedback and thoughts. > > Thank you, > James > > [1] - https://github.com/sphinx-doc/sphinx/pull/10949 > > [2] - > https://tests.reproducible-builds.org/debian/issues/unstable/sphinxdoc_translations_issue.html > > [3] - > https://docs.python.org/3/library/gettext.html#the-nulltranslations-class