Bug#971515: kubernetes: excessive vendoring (private libraries)

2020-10-22 Thread Dmitry Smirnov
On Friday, 23 October 2020 4:20:37 AM AEDT Shengjing Zhu wrote:
> However kubernetes is also a library, and the maintainer doesn't provide
> it. It becomes headache for other maintainers.

Yes we have to vendor "k8s.io/" all the time in multiple packages but I doubt 
it would be of help if Kubernetes provide a "-dev" library because
it is notoriously unstable with frequent breaking changes.

Ideally that's how it should be done but, pragmatically speaking, Kubernetes 
can never migrate to "testing" hence everything that depends on it will be 
unsuitable for release.


> The way the kubernetes maintainer chooses, is making others difficult.
> Every package that uses kubernetes as a library, has to keep a local copy
> of the source.

The partial solution might be to introduce another source package only with 
portion of Kubernetes that provides a shared library but that will require 
proper dependency management which is being discussed here.


> But I hope Janos can cooperate with the Go packaging team to make Debian,
> especially the Go packages in Debian more maintainable.

Yes indeed. We should not need to argue that cooperation is important.
Cooperation should always be a default mode of operation.

-- 
Kind regards,
 Dmitry Smirnov
 GPG key : 4096R/52B6BBD953968D1B

---

Hay smells different to lovers and horses.
-- Stanisław Jerzy Lec


signature.asc
Description: This is a digitally signed message part.


Bug#971515: Request for security team input on kubernetes TC bug

2020-10-22 Thread Dmitry Smirnov
On Thursday, 22 October 2020 2:22:11 AM AEDT Sean Whitton wrote:
> The TC are being asked about src:kubernetes, and it would be good to
> hear from you about whether and how security support is a relevant
> consideration in determining whether the level of vendoring in that
> package is acceptable.  Could you take a look at #971515 and perhaps
> give us some input, please?

I'm not a member of security team but I want to share my prospective on this.

To us what matters is what we can do about vulnerability in situation where
the problem has been identified and fixed upstream, in the particular 
library.

When a library is packaged separately, this is a matter of patching/updating 
a particular library then re-building depending packages.

With vendored libraries, we have no control as patching arbitrary versions of 
all instances of vendored library is a very bad option that is practically 
unsustainable. Even tracking of vendored libraries is difficult.

So in case of vendored libraries we can rely on "Oracle model" when upstream 
releases an update of a software where vendored library is patched so we ship 
the whole bundle. This is the worst option because we have no control and 
because we have to rely on faith in good upstream maintenance.

But here lies the problem. Kubernetes have bad dependency hygiene and poor 
abolity to control their enormous vendor mess (burden). With hundreds of 
vendored libraries they have large surface for problems and limited ability 
to track and respond. Just like we know from experience, an attempt to update 
a vendored library may lead to FTBFS and that's why upstream is reluctant to 
make changes to vendored libraries, unless when they absolutely have to.
Kubernetes can not effectively address issues in "vendor/" space.

So my conclusion about security support of upstream Kubernetes is that it is 
can not be done effectively in either case. There are greater chances for 
meaningful security maintenance when most dependency libraries are un-
vendored. With most libraries vendored, there is less security maintenance 
burden to us, but the outcome is no better.

-- 
All the best,
 Dmitry Smirnov
 GPG key : 4096R/52B6BBD953968D1B

---

No snowflake in an avalanche ever feels responsible.
-- Stanisław Jerzy Lec


signature.asc
Description: This is a digitally signed message part.


Bug#971515: kubernetes: excessive vendoring (private libraries)

2020-10-22 Thread Shengjing Zhu
On Tue, Oct 20, 2020 at 12:16:03PM -0700, Sean Whitton wrote:
> 
> - are people trying to do cross-archive work going to find the packaging
>   of kubernetes getting in their way?  (e.g. other Go team members
>   trying to update things, improve our binNMU techniques and machinery,
>   etc.)
> 

The kubernetes maintainer is not in the Go packaging team, so doesn't follow
the team practice. But as kubernetes vendor everything, it doesn't need to
work with all other Go packages.

However kubernetes is also a library, and the maintainer doesn't provide it.
It becomes headache for other maintainers.
We (the Go packaging team) need to vendor the kubernetes in our packages,
for example:

+ 
https://sources.debian.org/src/golang-github-openshift-api/4.0+git20190508.81d064c-4/vendor/k8s.io/
+ 
https://sources.debian.org/src/gitlab-ci-multi-runner/13.3.1+dfsg-1/vendor/k8s.io/
+ https://sources.debian.org/src/flannel/0.9.1~ds1-1/vendor/k8s.io/
+ https://sources.debian.org/src/consul/1.7.4+dfsg1-1/vendor/k8s.io/

And many, they can be found with:

https://codesearch.debian.net/search?q=%22k8s.io%2F+filetype%3Ago=1

It has been asked at #970331, without response from maintainer.

If kubernetes package want to make other package which Build-Depends on it
to build successfully, it needs to decouple the dependencies from vendor dir.
When building other packages, the Go toolchain can't use the vendor in
kubernetes, since the Go toolchain only supports the top level vendor dir.

The way the kubernetes maintainer chooses, is making others difficult.
Every package that uses kubernetes as a library, has to keep a local copy of
the source.

It's not Janos fault, as he doesn't make things worse. Kubernetes package is
in a bad state for a long time.
But I hope Janos can cooperate with the Go packaging team to make Debian,
especially the Go packages in Debian more maintainable.

-- 
Shengjing Zhu


signature.asc
Description: PGP signature