Bug#971515: Status as of last tech-ctte meeting
On Friday, 20 November 2020 4:35:23 PM AEDT Elana Hashman wrote: > Kubernetes is a very large and active project. It has about 600 > members,[0] 1000 voters,[1] 100 committers (which I define as members of > the milestone-maintainers team),[2] and over 50,000 contributors.[3] The > project has its own security[4] and release teams,[5] and the release > team includes a software licensing team. > [...] > As such, the scale of Kubernetes is similar to that of Debian itself. I was too slow to reply to this email but I'll leave some comments for the record. Comparison of Kubernetes' team size to Debian is misleading and the comparison is not in favour of Kubernetes. Debian is compartmentalised into relatively small (mostly) well coordinated teams. Kubernetes - the monolithic project - suffers from well known problem with coordinating large teams. Frederick Brooks described the problem well in his "The Mythical Man-Month" book. Regarding number of contributors, what is strength for Debian is a weakness for Kubernetes. Kubernetes bug tracker shows the scale of the problem and I don't have impression that the team is big enough to handle bug reports effectively or have them under control. It was my observation that vendoring problems in Kubernetes are worse than everything I've seen in other projects. Is that despite the size of the team or because of it? The particular example that I did not mention enough [1] (and it is not the only one) show how little that remarkably large team cares about dependency hygiene: a trivial library update with a patch was not done in a few years time and all that was produced by the large team are excuses for not doing the update, only to finally perform it in the end as advised. Who can have confidence in the Kubernetes dependency management after that? [1]: https://github.com/kubernetes/kubernetes/issues/27543 Of course the problem have little to do with Kubernetes. The "use world" situation with over-dependency on large number of 3rd party libraries is a challenging one especially with historically poor Golang approach to dependency management a.k.a. "vendoring". I'm just saying that Kubernetes is not all that special in regards to vendoring because of the team size. > Kubernetes as an upstream project is much better resourced than the > single downstream maintainer in Debian. Nobody argues with that. But it is only "single maintainer" with monolitic packaging - a case that I'm arguing against. Golang team is strong and its capacity is impressive. Packages that use dependency libraries are always team-maintained. It is the same situation as with Kubernetes upstream where much of the 3rd party code is not maintained by Kubernetes team. > The resourcing and scale of the Kubernetes project gives us better > assurances that upstream has done due diligence for dependency > management. IMHO upstream demonstrated bad and terrible attitude with dependency management. But apparently nobody here considered that argument... :( -- Kind regards, Dmitry Smirnov GPG key : 4096R/52B6BBD953968D1B --- We often refuse to accept an idea merely because the way in which it has been expressed is unsympathetic to us. -- Friedrich Nietzsche --- COVID-19: The Trouble With PCR Tests https://swprs.org/the-trouble-with-pcr-tests/ signature.asc Description: This is a digitally signed message part.
Bug#971515: Status as of last tech-ctte meeting
]] Philip Hands > Tollef Fog Heen writes: > > > ]] Shengjing Zhu > > > >> Firefox is special, since for Debian desktop users, they need a browser. Is > >> kubernetes same here? > > > > FWIW, the lack of Kubernetes or a similar orchestration platform (mesos, > > nomad, docker swarm) in stable has been keeping back development of the > ^^ > > Do you really mean stable here? I had gained the impression that there > was no prospect of Kubernetes getting into stable, regardless of the > details of how it ends up being packaged. Have I misunderstood? I do mean stable. I was making a narrow comment about how special or not Kubernetes is, I don't know if the goal is for it to reach stable or not. (My comment above was also slightly imprecise, I've since learned that Docker Swarm is in Debian, in the docker.io package, thanks to Shengjing Zhu who pointed it out in private email.) -- Tollef Fog Heen
Bug#971515: Status as of last tech-ctte meeting
Tollef Fog Heen writes: > ]] Shengjing Zhu > >> Firefox is special, since for Debian desktop users, they need a browser. Is >> kubernetes same here? > > FWIW, the lack of Kubernetes or a similar orchestration platform (mesos, > nomad, docker swarm) in stable has been keeping back development of the ^^ Do you really mean stable here? I had gained the impression that there was no prospect of Kubernetes getting into stable, regardless of the details of how it ends up being packaged. Have I misunderstood? Cheers, Phil. -- |)| Philip Hands [+44 (0)20 8530 9560] HANDS.COM Ltd. |-| http://www.hands.com/http://ftp.uk.debian.org/ |(| Hugo-Klemm-Strasse 34, 21075 Hamburg,GERMANY signature.asc Description: PGP signature
Bug#971515: Status as of last tech-ctte meeting
]] Shengjing Zhu > Firefox is special, since for Debian desktop users, they need a browser. Is > kubernetes same here? FWIW, the lack of Kubernetes or a similar orchestration platform (mesos, nomad, docker swarm) in stable has been keeping back development of the next generation way of handling Debian services, since that will need a reasonable container orchestation platform to build on. The lack of a platform is not the only reason for the delay, but it surely hasn't helped either. -- Tollef Fog Heen, speaking for himself, but as a DSA member
Bug#971515: Status as of last tech-ctte meeting
Hi Josh On 2020/11/20 01:30, Josh Triplett wrote: > In particular, with my upstream Rust/Cargo hats on, I would love to work > with you and others on questions of what software packaging could look > like, and how to maintain the quality and curation *and* package > availability of Debian in collaboration with other ecosystems of package > and dependency management. > > Other potentially interesting questions: what are the assumptions that > go into our current tradeoffs about shared libraries vs static > libraries, and are those still the correct tradeoffs in all cases? There is already a lot of text to read on this bug report, please try to refrain from adding more off-topic content to it. thanks, -Jonathan
Bug#971515: Status as of last tech-ctte meeting
On Wed, Nov 18, 2020 at 10:26:21AM -0800, Russ Allbery wrote: > There are a lot of fascinating edge cases and precedents and philosophical > questions about the function of a distribution here, and I think they > rightfully attract a lot of energy, but I'm worried this is at the cost of > losing sight of the core functionality provided by Debian. Because that > functionality is currently working, the people who are benefiting from it > don't know to weigh in. [...] > As a Debian developer, I appreciate the arguments about vendoring and the > desire to maintain Go libraries and the pride that we take in our work of > really understanding software and packaging it properly. > > However, as a Debian *user* of the kubernetes-client, I have to say that I > do not care in the slightest. The older, unvendored kubernetes-client > package worked great. The new, heavily vendored kubernetes-client package > works great. Both do exactly what I want, and I don't care at all, as a > user, which is in Debian. But if the package were removed from Debian, I > would be really unhappy. And if Helm and Argo CD were packaged for > Debian, I would be much happier, so if unvendoring them is the obstacle, > as a user I'm opposed to unvendoring! > > I want to push back a bit against the feeling that some things are > ill-suited for how Debian has historically packaged software and therefore > maybe Debian isn't the place for them, and we should instead ask people to > manage them outside of Debian (but somehow make this easy). > > First, while I appreciate and cheer Phil and Sean's optimism, there have > been a lot of plans in Debian historically to make something that isn't a > package easy to build and install, and they have basically never worked. > The way Debian makes something easy to build and install is by making it a > package. That's what our entire project is designed around, and I'm > dubious that we're going to be able to reproduce those properties with > something that isn't a package. > > Second, the point of Debian at the end of the day is that I can install it > on a computer and use it to get things done. The details we're discussing > here are important and work towards making Debian maintainable and > sustainable and to ensure that a quality bar has been met in terms of > security and legality and licensing, but I think it's important that they > are also means to an end and the end is not security and licensing per se. > We're not making a random collection of relatively secure software; we're > giving people programs that they can run while keeping them secure. We're > not just a classification system for what software is free; we're giving > people software they can use while ensuring that all of it is free. I > think it's worth occasionally stepping back and dwelling on that thought > for a moment and making sure we're picking the right strategy for meeting > our quality bar that allows us to still achieve the mission. > > This is particularly true for something like vendoring or the avoidance of > vendoring, which is not a core mission. It's not in the social contract, > and it's not a DFSG principle. It's a hard-won and very valuable piece of > experience with how best to sustainably make a distribution... but one > that was hard-won in the era of C programs and shared libraries and that > has generalized admirably to Perl and Python. It may generalize to other > languages and other mechanisms for distributing software. It may not! If > it doesn't, that's significant because it's such a deeply-engrained part > of our culture, but it's *not* a breach of our fundamental project goals. > We can consider new approaches here without becoming untethered, and > indeed may be able to achieve our primary goal *better* by expanding the > scope of software that we can include in the distribution and that can > therefore just work. > > I think there's a bit of a crisis of confidence in Debian because of how > much larger the free software ecosystem is now than it was when the > project started, how far away from doing everything through a distribution > a lot of developers have moved, and how many resources are flooding into > other areas that we have difficulty keeping pace with. One of the natural > reactions to that crisis of confidence is to pull back from these new and > difficult and unfamiliar areas and decrease scope to the things we know > we're really good at: packaging primarily C, C++, Perl, and Python code. > And that is a valid strategy; it's okay to just keep being very good at > something one is already very good at. But I think in the long term that > means Debian becomes something much different than it historically has > been. It means Debian would become a base on which other people would > build things, rather than a comprehensive collection of tools that covers > all the incidental things you need from a computer, and where people need > only reach outside of Debian for
Bug#971515: Status as of last tech-ctte meeting
On Wed, Nov 18, 2020 at 10:26:21AM -0800, Russ Allbery wrote: > > I maintain a bunch of Kubernetes clusters as part of my job. Those > clusters are run by other people (cloud providers, data centers, etc.). I > need clients to talk to those clusters. When I first started working with > Kubernetes, I realized I needed a client, worried about how much of a pain > that would be, did an apt-cache search for kubernetes, found > kubernetes-client, breathed a sigh of relief, and ran apt install > kubernetes-client. I have subsequently not given Kubernetes clients a > second thought. > In priciple, the kubectl compatibility matrix says it only supports one minor version (older or newer). When you working with many clusters, the cluster version may not be compatible with the kubectl version in Debian stable. PS, in practice, the basic function of kubectl doesn't change much, and new kubectl with old cluster seems to work. As a user of upstream kubectl package user, I'm quite happy with it. I can always install the latest, or any old version. They keep all old versions in their apt repo. And with the nature of static link, their repo works well other parts of the system, regardless I'm running Debian oldstable, stable, or unstable. IMO, we should admit that current Debian way doesn't fit well for such softwares. signature.asc Description: PGP signature
Bug#971515: Status as of last tech-ctte meeting
Russ Allbery writes ("Bug#971515: Status as of last tech-ctte meeting"):
> [much stuff]
Oh, wow, Russ. Thank you very much. What an excellent piece of
writing. I agree entirely with all of it.
Ian.
(And I speak as someone who thinks that this newfangled "vendor
everything" and "just download that shit from the internet" approach
is hideously bad engineering practice.)
--
Ian JacksonThese opinions are my own.
Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
Bug#971515: Status as of last tech-ctte meeting
Hello all, On Wed 18 Nov 2020 at 11:36AM -06, Gunnar Wolf wrote: > So... It's not like we reached a conclusion, but I do feel the meeting > was interesting and the discussion very much worthy. Yes, this will > surely end up in reinforcing the notion that the Technical Committee > is a slow and bureaucratic beast :-) But... well, I'll stop > apologizing. Only some minutes to go before the meeting, and I want to > give the rest of the Committee the opportunity to check if I didn't > misrepresent them or skip too much of their opinion. Thank you for this summary. At today's meeting one point which we thought was missing from this summary was that no-one on the committee has any appetite for overruling the package maintainer, so it is very unlikely that will be the outcome of this bug. -- Sean Whitton signature.asc Description: PGP signature
Bug#971515: Status as of last tech-ctte meeting
Hello world, When we had our last tech-ctte meeting, 2020.10.21¹, I volunteered to write up a summary of our positions regarding this bug. Then... Well, life happened, and I have not had the time to sit down and write until today -- A couple of hours before our next meeting. Several other discussions have happened as well, most notably, the article by Jonathan Corbet on this issue in LWN². ¹ http://meetbot.debian.net/debian-ctte/2020/debian-ctte.2020-10-21-17.58.html http://meetbot.debian.net/debian-ctte/2020/debian-ctte.2020-10-21-17.58.log.html ² https://lwn.net/Articles/835599/ # Vendoring: Impedance mismatch with our long-established # practices/traditions ## I believe the issue of vendoring to be central to the discussion of what Debian is, and what its role should be. We are very lucky to have proponents of both sides of this issue in the Committee, and I'll try to keep my ideas as centered as possible (of course, disclaimer: I do hold a position, I really do not like vendoring; we have the luck of having Elana in the ctte, as she is a developer of the affected package, and Sean, who is a Debian Policy editor). Elana started off with a very important and true point: Elana> I think this bug was sort of inevitable. Debian policy cruft is clashing with how people actually build and distribute software these days. we have an appeal to override a maintainer on the basis of "policy" and the "Debian way being superior" without any real technical examination of the merits of "the Debian way" We are quite far from 1996, yes, and many languages have for a long time delivered their own packaging systems, "freeing" the users from the need of installing a myriad of little packages, and "freeing" the distribution caregivers (and I don't only mean the developers, but also the ftp-masters) and infrastructure from carrying this myriad of little packages. Elana and Sean seem to share the thought that each language ecosystem could work with somewhat different rules: Sean> It might be reasonable to vendor like mad for something written in Go but not for something written in Haskell, say Elana> Debian policy is specifically built around the distribution and execution of dynamically linked C/C++ applications and libraries. distros in general were. but modern software above that base does not necessarily operate under the same assumptions and it is silly to apply policy that was designed for applications that are dynamically linked against programs that are statically linked and are impossible to dynamically link Simon mentioned that "our identity should be about shipping high-quality packages, whatever that means", and mentioning that "our packaging policy is designed for medium-sized packages", refereed back to the discussions had long time ago over tiny Javascript snippets packaged as packages, back in the starting days of the nodejs growth. # DFSG-freeness checking ## Sean and I shared the concern that excessive vendoring (say, having tens to hundreds of libraries shipped as part of a single package) place a very heavy burden on our ftp-masters when checking DFSG-freeness; coupling this with the rapid change rate that "new-wave" software development usually has, if adding / dropping dependencies from already packaged software is usual practice, DFSG-checks would not just have to be made in NEW, but as a continuous process. Far from sustainable, and impossible to do by our ftp-master team practices. # Security issues ### The issue of security support was also mentioned: If many packages start shipping tens or hundreds of vendored libraries, how can we ensure security support? This has long been a pride point for Debian and, in general, for the Linux distributions model. We package independent libraries, dynamically link them, and security supports "just happens" for the users. Now, what happens in languages as Go or Rust, where linking is done statically? They would have to be at least recompiled when any of their constitutive libraries is updated. But what if the libraries are vendored-in? Security issues are prone to be hidden from our sight. I'm pasting here a bit of the discussion that happened later during the meeting: having this discussion K8S-specific, Elana mentioned that "that is a big part of the tension. sometimes, you have an upstream where the authors are less resourced and responsive than the downstream. this case is almost certainly the opposite". At this point, we found some friction as to _what_ we are discussing on: Is this bug specifically on Kubernetes, which should be taken as a special case? Or would we try to rule as the Technical Committee on vendoring in general in the Debian ecosystem? Elana repeated her assurance that the Kubernetes team is thorough in their freeness-checking and security practices; I insisted on "not discussing about K8S, but about a bundling practice to which K8S
