Hi Luca, On Fri, Aug 02, 2024 at 01:43:04AM +0100, Luca Boccassi wrote: > 1) The os-release specification must be adhered to, and it must be > possible to tell the difference between testing vs unstable, and each > must be correctly identified by the respective metadata
Given the state of discussion, I think it is fairly evident that we do not have consensus for the need to distinguish testing and unstable. We have people arguing in favour and we have people arguing against that. This is a point of disagreement. > 2) Testing and unstable can continue to remain indistinguishable, and > both be erroneously identified as trixie Isn't there the third option of adhering to the os-release specification without making testing and unstable distinguishable? I did not see this ranked in your preference. Do you see it as even worse than the status quo? > Again you'll know better than me, but it seems to me rulings were made > in the past that were along similar lines (eg: usrmerge) - there > wasn't reviewable code there, no? I think /usr-merge was quite different. The usrmerge package existed for a long time before things to to the CTTE. The changes to debootstrap were uploaded before things escalated to the CTTE another time and what remained was basically a question of what defaults debootstrap should have. In the case of os-release, the disagreement reached CTTE before actual changes to the disputed status quo have been uploaded to unstable and the available solution space is wider. Also the transition cost is lower. Keep in mind that our constitution says that you cannot be asked to work (2.1) and in particular the CTTE cannot request work to be done either. Instead, it can authorize a solution and require that developers do not work against the chosen solution. Furthermore, the CTTE should select among designs that have been proposed and "reasonably thoroughly discussed" (6.3.5). My perception is that your initial filing did not meet this bar, but you later clarified your design and discussion is ongoing. > Sorry but I do not think that is an accurate representation. First of > all, the implementation of the spec is bugged, period - it's not about > being pedantic about it, it's about being completely incompatible: sid > is identified as trixie, and that is just plain wrong, and there's no > room for interpretation, no matter how one might try to bend it. You > might say that you don't _care_ that the implementation is buggy, you > might even say that it is worth, nay _good_ it to leave it buggy - but > buggy it is, if I create a sid image, it will never in a million years > be trixie, and yet it says it's trixie. I think it has become abundantly clear that your view on this does not represent consensus of discussion participants. It is not as black and white as you paint it. Simon compared unstable to Ubuntu's proposed pocket. It also happens that testing and unstable share the same pool hierarchy and the vast majority of packages. On the other hand, Devuan operates as an overlay repository to be added on top of a Debian repository. I don't think it matters that Ubuntu's proposed is implemented as a partial distribution overlaid onto another as that's what other derivatives (that do update their os-release file) do as well. It would help if you could stop dismissing other people's views and respectfully disagree with their views instead. Refer to Simon's reply https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1077764#118 for more details on this aspect. > Secondly, I am not even sure what these conflicting requirements > actually are? Could you please spell them out? If trixie was > identified as trixie, and sid was identified as unstable, what > compromise would be, er, compromised, precisely? Because the only > practical objections I could find were based on the idea that > implementations would be ugly or hard to maintain or so - as already > mentioned, I am happy to relieve anybody else and take that hard and > ugly maintenance burden for myself. What other actual problem would > suddenly appear? What feature or advantage do we leave behind? How are > things made worse for our users? The devil is in the detail. Debian currently does not provide a simple way to distinguish testing and unstable. As a result, some users have opted to not rely on such differentiation and instead inspect particular package versions for the features they are interested in. Others have implemented heuristics to somehow tell testing and unstable apart as they saw a need for doing so (that I still fail to see at this time). Whilst you and others have presented code for telling testing and unstable apart in multiple places, I am still having a hard time figuring out why we had to tell them apart and in what way code would behave differently for one or the other. If Debian were to differentiate testing and unstable in a simple way, people would start relying on that. That in turn (as laid out by Russ and Simon) would degrade our distribution testing as code that was working in unstable may stop working once it migrates to testing for the sole reason of inspecting /etc/os-release. To put this another way, enabling users to differentiate between testing and unstable would encourage developers to write code that some of us see as broken by design. > in the weirdest places. Here's one I am well familiar with as the git > history will show: > > https://github.com/openSUSE/obs-build/blob/master/build-recipe-livebuild#L138 This one vastly predates the introduction of os-release and the actual problem being solved there is to not distinguish testing from unstable and rather treat unstable in the same way as testing even though lsb_release would differentiate them. > build live-build based images in the open build service, because of > this base-files bug. Fortunately at the time we didn't care about > that, for external reasons, so it didn't cause troubles, besides > needing to hack it. > > Here's a random one, first result found searching on Github: > > https://github.com/lpereira/hardinfo/blob/4c97625c5666fa5fc353e7cab322b09159e54ed4/modules/computer/os.c#L486 > > /* HACK: Some Debian systems doesn't include the distribuition > * name in /etc/debian_release, so add them here. */ Again, this is not concerned with os-release, but with lsb_release --codename attempting to differentiate between testing and unstable. The actual result here is that DIST would contain the codename of testing when run in an unstable environment. Both of these are likely no longer needed. > Note how it starts with HACK, all caps? This is what we subject our > users to. This is what we are known for. What do we even gain in > exchange? I hear you and agree that we should get rid of this code once we no longer need to support old releases. On the flip side, my reading of the examples you presented strengthens the position that testing and unstable should indeed be indistinguishable, which likely is not what you intended. > "Ah it's Debian Trix..." > > $ sudo build/systemd-dissect --copy-from image.raw > /etc/apt/sources.list /tmp/sources > $ grep Suites /tmp/sources > Suites: unstable > Suites: unstable-debug > > "Oh wait no, it's Debian Si..." > > $ sudo build/systemd-dissect --copy-from image.raw /etc/debian_version > /tmp/debian_version > $ cat /tmp/debian_version > trixie/sid > > "Oh no, wait, it's... both?!" You successfully detail how testing and unstable are difficult to distinguish. What I still fail to see is what negative consequence there is as a result. > To be pedantic, please note that /etc/os-release is a symlink, not a > conffile. The real content is in /usr/lib/os-release. Right. To be pedantic, a symlink can be a conffile. Either way, that only changes the location of the dispute, but little beyond. > I do not find using t-p-u to update the content ugly at all to be > honest, it's there to be used as a piece of infrastructure. I do not > see anything wrong with using it? What would be the actual issue? and > anyway as already mentioned, I am fine with taking the burden of such > ugliness upon myself, so that nobody else has to be subjected to it. I'm not sure this is worth debating. Whether I or Santiago would describe it as ugly likely does not matter much. What really matters is whether the release team agrees with such use of t-p-u and that's a question you can relatively easily settle. > > Consider adding a new Debian binary package containing /etc/os-release > > with contents suitable for unstable. This package would use the > > dpkg-divert mechanism to avoid a conflict with base-files which would > > continue to install /etc/os-release with contents suitable for testing > > (even in unstable). We would mark this new package "Essential: yes", > > but have no other package depend on it. We would also file a permanent > > rc-bug to prevent it from migrating to testing. As a result, the set of > > essential packages as discovered by bootstrapping tools would differ > > between testing and unstable and the /etc/os-release would be diverted > > for unstable only. A significant downside of this approach is that all > > existing unstable systems would never install this package (and thus > > continue to be seen as testing installations) and all systems that were > > bootstrapped as unstable would be keep this package even if they were to > > be downgraded to testing at a later time (unless the administrator > > actively removes the new package). > > The downsides you have mentioned are significant already, and also as > you know I very strongly dislike dpkg-diverts, but that's my problem. > However, having complex maintainer script machinery in the essential > set is everyone's problem, as it can and will break, and as above, I > am sure it is best avoided. This still requires blocking the migration > of a package, so it does not seem to me it provides any advantages > over simply having an os-release binary package pinned to unstable, > with no maintainer scripts, and just the content appropriate for > unstable inside it. It still requires overruling the base-files > maintainer. What would be the difference/advantage over the other > approach? Unless base-files gains a dependency on the proposed os-release package, your t-p-u approach shares the downside of existing unstable systems never getting classified as unstable. Your approach also shares the downside of systems that are being bootstrapped as unstable keep being classified as unstable even if they become testing. If you see these as significant, you should go back to the drawing board. I sympathise with your dislike of diversions in principle, but not at any cost. I've removed diversions of /bin/sh from unstable when I saw their cost exceeding their benefit and added a lot of diversions or finalizing the /usr-move. To me they're a tool whose cost should be evaluated for each use. What makes them less attractive here is that this diversion would be permanent. On the flip side, it would only ever be present in unstable and never reach any other release (and thus the majority of users would never see it). Indeed, you may pin a package to unstable by other means than adding an rc-bug. Adding a britney hint works as well (both for the t-p-u approach and for the diversion approach). One advantage of the diversion mechanism I see is that its effects are limited to unstable. We don't need to add any new package to any other release nor do we need to move files between essential packages or pull a new package into the essential set. Please do not read this as me trying to convince you to use diversions. I've given some arguments, but even now, I am not convinced myself that I would prefer this approach. I merely try to widen the solution space and allow us to better understand their advantages and disadvantages. > > We have a disagreement about whether the information you intend to > > convey actually exists beyond the point in time where you deboostrap. If > > I debootstrap unstable and look at it a week later, it more closely > > represents testing than unstable. > > I don't think so. It is unstable, one week out of date. It's not any > different than running "debootstrap bookworm" and checking back X > weeks later, after a point release is available - it will be outdated, > but still bookworm, and still one apt update; apt upgrade away from > being up to date again. Same for unstable: if it's a week old, after > you apt upgrade it, it will be on par again. It will not magically > become testing by itself. If you debootstrap unstable today and check > back again in a year, it will not be stable trixie, it will be > outdated unstable, and an apt upgrade will make it an up to date > unstable. At that point when you have an outdated unstable and a current testing, differentiating them becomes meaningless as your current testing will have a lot of more recent packages. What do you do with this knowledge that one is unstable and the other is testing? > > Thank you for referencing a practical effect. It is not clear how mkosi > > uses the Release field and why it is important to differentiate testing > > from unstable though. > > It's an image builder tool - it builds images, so it needs to know > _which_ image to build and in this case, I need it to build a guest > that matches the host - do I pull from testing or unstable? Which one > is this? Ah it says trixie, so I can use the trixie repository and... > oh wait now everything is broken because it was actually sid and a > transition was not finished so I got a broken image. Try again next > week? Both Johannes and me have fallen into this trap. You have create a flaky autopkgtest that is not actually testing what it is supposed to test. Rather than inspecting the release name, your test should copy the hosts' sources and apt pins into the guest. Your typical autopkgtest runs with both testing and unstable enabled with some packages pinned from unstable and you need to reproduce this package set inside your guest. This now is an example for why distinguishing testing and unstable bears a risk of abuse. Thus far, every single use of the testing vs unstable difference that I've seen in this discussion is better solved in another way. > Because it's not just standalone images, but images based on other > images (ie, sysext: > https://uapi-group.org/specifications/specs/extension_image/ ). If you > cannot identify an image, what the heck did you just build? And > against what? And what do you base this other one on? Nobody knows. > And then the hacks begin. If you knew your image was unstable, you'd gain nothing in terms figuring out what is inside without knowing when it was produced. You gain something once you add a BUILD_ID. As you rewrite /etc/os-release anyway, you may update VERSION_CODENAME as well at little extra effort. > Yes, and that solves the "wrong information given" problem (which is > new, VERSION_CODENAME was introduced somewhat recently in base-files), > but does not solve the "cannot identify image without horrendous > hacks" problem, so the implementation is still buggy - less so, but > still buggy. The point of the spec is to uniquely identify an image, > and if you are Arch and don't do releases then of course you don't > specify the release codename (there ain't one), so the field is > optional to cater to those use cases. It's not optional to cater to > the "distribution is held together with duct tape and prayers" cases, > I'm afraid. And if you don't implement the spec to enable its purpose > of identifying vendor trees, why implement it at all? More and more I am reaching the conclusion that not being able to distinguish testing from unstable is a feature, not a bug. I'm using the information from /etc/os-release (e.g. via ansible or puppet). For that use case, Simon gave a recipe that works well with the VERSION_ID as it is today. I'm trying to better understand other uses of the information contained. > Sorry, I do not follow - how would this impact debootstrap negatively? > It's just a package with a single file? The only part to decide is > what pulls it in and yeah that should be compatible with debootstrap > of course. This matter is discussed in Russ' subthread. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1077764#93 > The best and easiest solution is what I shared in a previous mail: a Thanks for summarizing and ranking the solutions you considered. That helped better understand your point of view. > Besides what I provided above, is there anything else required to make > it actionable? We've made significant progress in understanding your request and desired solution. I expect that we will take at least a week to gather views and use cases on distinguishing testing from unstable before we can draw any conclusions or ballots, but then this doesn't look super urgent to me. Thanks for your patience. Helmut
