Bug#901030: some context
Antonio Terceiro writes ("Bug#901030: some context"):
> the problem here is that by definition, some types of testbed don't
> allow you to test some packages.
Right, but in principle these testbeds could. All that is needed is
to install binfmt-misc in the host. I am suggesting that this should
be fixed by having the container virt server packages depend on
binfmt-misc.
> in this specific case, there is just no
> way you can reliably test a feature that depends on a kernel feature in
> a container.
As a general rule, what you say is clearly not true. "Being able to
run amd64 binaries" is a kernel feature which is generally available
in amd64 containers :-). And many more sophisticated kernel features
are containerisable (in the sense that they can be used in the
container, even if the host doesn't want to know about it, and even if
they require root).
But, our whole dependency scheme does not really handle kernel
features. binfmt-misc is a helper for accessing a kernel feature -
one which is available in containers but not configurable separately
from the host.
Stepping back a bit: if we don't wnat to make adt-virt-lxc depend on
binfmt-misc, we could do something more sophisticated:
Define two new DEP-8 test fieldc:
Virt-Server-Capability-Depends
Virt-Server-Capability-Conflicts
These are in Depends field syntax but the referred-to names are not
packages but capabilities from the virt server protocol.
We provide both, so that test authors have the choice between
(i) using Depends, and experiencing unnecessary test skips;
(ii) using Conflicts, and expericing spurious test failures.
We also define new conventional capability syntaxes:
bug-*
bug-debianN
for use in Virt-Server-Capability-Conflicts. (-Conflicts should
usually be used rather than -Depends, because other virt servers
who do not have the bug should not be required to advertise it.)
We only need to teach autopkgtest about the field names, and then the
affected packages can conspire as follows.
Package: something-needing-binfmt
Virt-Server-Capability-Conflicts: bug-debian901030
And then adt-virt-lxc can advertise a `capability' bug-debian901030.
Indeed, if we want to get clever, adt-virt-lxc can do so if:
* binfmt-misc is not installed on the host
and
* the kernel has not been fixed to support per-container binfmt handling
Also, virt servers should gain a runtime option for the administrator
to have them advertise capabilities. That way it is not necessary to
update the software when a new bug is discovered: the admin can update
the config.
What do people think ?
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Bug#901030: some context
On Tue, Jun 12, 2018 at 02:34:57PM +0200, Hans-Christoph Steiner wrote: > > I agree that there can't be a Restriction for every config under the > sun. I'm thinking more like Ian, where there can be some useful chunks > with a finer grain than "needs isolation". Allowing a test definition to cause any side effect to the host that is running the test opens the doors to a security nightmare. > I think that autopkgtest will never work well for a lot of packages with > such a inflexible system for specifying requirements. What if > autopkgtest took a more "user generated" approach like gitlab CI > runners? That means that anyone can create tags/labels for runners, and > jobs can specify any label that they require. Then it is up to the > people implementing the jobs/runners to make sure that they work. > > We can learn a lot from the various CI services out there (GitLab, > Travis, Circle, etc). They are all based on containers, and they > generally put few restrictions on what can be done in the containers. > Containers are really well suited for CI builds since they provide an > easy system for building up throwaway environments. note that no configuration option in gitlab ci, travis etc allows you to make changes to the *host* where the tests run. you can specify several aspects of the *testbed* (container) where the tests run, and autopkgtest already allows you to do whatever you need inside the testbeds. the problem here is that by definition, some types of testbed don't allow you to test some packages. in this specific case, there is just no way you can reliably test a feature that depends on a kernel feature in a container. signature.asc Description: PGP signature
Bug#901030: some context
Hans-Christoph Steiner writes ("Bug#901030: some context"):
> I agree that there can't be a Restriction for every config under the
> sun. I'm thinking more like Ian, where there can be some useful chunks
> with a finer grain than "needs isolation".
>
> I think that autopkgtest will never work well for a lot of packages with
> such a inflexible system for specifying requirements. What if
> autopkgtest took a more "user generated" approach like gitlab CI
> runners? That means that anyone can create tags/labels for runners, and
> jobs can specify any label that they require. Then it is up to the
> people implementing the jobs/runners to make sure that they work.
I'm not sure what you mean.
There is nothing stopping anyone inventing whatever test restriction
labels they like right now. Except, of course, that no test runner
will provide support for it.
I don't see how this problem can be solved without coordination
between test authors and (as applicable) autopkgtest developers and
virt server developers.
> As for the binfmt_misc module discussion, I know nothing about how that
> works. If you can fix this issue in the kernel, fine by me.
ISTM that the bug is that the binfmt thing is not properly virtualised
by containers. We don't have to fix that in the kernel; we could
bodge it by installing binfmt-misc in the host.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Bug#901030: some context
I agree that there can't be a Restriction for every config under the sun. I'm thinking more like Ian, where there can be some useful chunks with a finer grain than "needs isolation". I think that autopkgtest will never work well for a lot of packages with such a inflexible system for specifying requirements. What if autopkgtest took a more "user generated" approach like gitlab CI runners? That means that anyone can create tags/labels for runners, and jobs can specify any label that they require. Then it is up to the people implementing the jobs/runners to make sure that they work. We can learn a lot from the various CI services out there (GitLab, Travis, Circle, etc). They are all based on containers, and they generally put few restrictions on what can be done in the containers. Containers are really well suited for CI builds since they provide an easy system for building up throwaway environments. As for the binfmt_misc module discussion, I know nothing about how that works. If you can fix this issue in the kernel, fine by me.
Bug#901030: some context
Antonio Terceiro writes ("Bug#901030: some context"):
> Control: severity -1 wishlist
> Control: tag -1 + wontfix
..
> By the same argument we would need one restriction for every
> kernel-related feature out there. Implementing every type of special
> snow flake configuration in autopkgtest is unsustainable, and will be a
> maintainance nightmare.
>
> Instead, we have isolation-machine that says the test needs to be run on
> a VM, and then one can make the tests depend on the needed packages and
> make any other configuration required in the test scripts
> themselves.
I agree that reproducing every package that might need to be installed
in a container host, as a possible test restriction, is not really
workable.
But OTOH these tests _can_ be run in a container, provided that the
container host has some appropriate support. It would be nice to have
a way to declare that.
One way might be to maybe invent new test header
Container-Depends:
which is to be ignored by non-container runners ? The container
runners could have a passlist of packages they were willing to
install.
Alternatively, another completely different approach: One might view
it as a bug that installing binfmt-misc *inside* the container does
not work. After all, each container might reasonably want a different
configuration, and the paths etc. configured in the kernel via the
kernel binfmt API ought not to need to refer to textually-identical
paths inside and outside the container which must therefore refer to
similar programs.
Of course fixing that in the kernel might be too awkward. But it
would justify taking the following view: we should work around this in
test runners that use containers. Most straightforwardly, add a
dependency on binfmt-misc to the package containing adt-virt-lxc.
Does that make some kind of sense ?
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Bug#901030: some context
Control: severity -1 wishlist Control: tag -1 + wontfix On Fri, Jun 08, 2018 at 12:35:13PM +0200, Hans-Christoph Steiner wrote: > > This issue was discovered while working on getting autopkgtest running > smoothly in Docker containers running in Gitlab CI on salsa.debian.org: > https://salsa.debian.org/salsa/support/issues/27 > > When running autopkgtest in a gitlab-ci docker container, it can run > directly in it since it has root access. But since the Docker host > provides the kernel, the Docker host must have binfmt-support installed > and the binfmt_misc kernel module loaded in order for Docker containers > to use binfmt. By the same argument we would need one restriction for every kernel-related feature out there. Implementing every type of special snow flake configuration in autopkgtest is unsustainable, and will be a maintainance nightmare. Instead, we have isolation-machine that says the test needs to be run on a VM, and then one can make the tests depend on the needed packages and make any other configuration required in the test scripts themselves. signature.asc Description: PGP signature
Bug#901030: some context
This issue was discovered while working on getting autopkgtest running smoothly in Docker containers running in Gitlab CI on salsa.debian.org: https://salsa.debian.org/salsa/support/issues/27 When running autopkgtest in a gitlab-ci docker container, it can run directly in it since it has root access. But since the Docker host provides the kernel, the Docker host must have binfmt-support installed and the binfmt_misc kernel module loaded in order for Docker containers to use binfmt.
