Re: Relationship between Docker and Guix
Hellp zimoun, my two cents on Docker as container images builder (not as "container instantiation toolbox") zimoun writes: [...] > The relationship between Docker and GNU Guix is container and the LXC > [1] technology. They use both but differently: > > - Docker is rooted in mutable/imperative and tries to go to more > functional; Docker images are immutable, they are usually (but not exclusively) generated via `docker build` using an imperative "dockerfile language" (so immutable/imperative :-) ); images are used to instantiate Linux containers and there are tools (bind mounts, environment) to separate state (data/config) from "system" to obtain a stateless container: the "stateless" status of the container however is strictly tied to the stateless properties of the underlying ditribution used to build the container I'm not sure what you mean with "tries to go more funtional" since AFAIU dockerfile "language" is not going to be funcional anytime soon, probably it does not even need to. The main issue with the use of dockerfiles is that unfortunately the FROM layering option (usually *heavily* used by docker images packagers) often makes keeping control of what is actually distributed with the image [1], cryptominers included, simply an... illusion... ehrm hard work :-). I've done it for work and that is the main reason I *avoid* any docker image not built by me as soon as I can. [...] > Everything starts with a configuration file: Dockerfile versus manifest.scm. > > - Dockerfile depends on the state of the distribution that one will > use -- say Debian -- and each time "RUN apt-get update" and/or "RUN > apt-get install" is called then no one can know in advance what the > resulting disk image will *exactly* contain; The non reproducible (not stateful) nature of the resulting docker images directly depends on the distribution used to build it: Debian have no means to "pin" a specific version in time (tag or commit, ala Guix) to use; indeed if I use Guix as initial system image (FROM guix-base) and a combination of manifests and channels definitions I can get a reproducible image (never tried this, just theory :-) ) Anyway, using `guix pack -f docker...` is much much better than the above dockerfile example :-O This is just to say that it is not `docker build` fault not creating reproducible images :-) [...] Ciao. Gio' [1] even from a legal POV -- Giovanni Biscuolo Xelera IT Infrastructures signature.asc Description: PGP signature
Re: Relationship between Docker and Guix
Hi, Thank you Ricardo for the detailed explanations. I do not know if my analogy below is correct and/or useful. The relationship between Docker and GNU Guix is container and the LXC [1] technology. They use both but differently: - Docker is rooted in mutable/imperative and tries to go to more functional; - Guix is rooted in immutable/functional. Everything starts with a configuration file: Dockerfile versus manifest.scm. - Dockerfile depends on the state of the distribution that one will use -- say Debian -- and each time "RUN apt-get update" and/or "RUN apt-get install" is called then no one can know in advance what the resulting disk image will *exactly* contain; - the manifest.scm depends on the state of the channel trees, other said, on commit hashes -- manifest.scm acts as a pure function: same inputs, same outputs -- so one obtain exactly the same container. We cannot guarantee that the manifest.scm file which one runs today will generate the same bit-to-bit disk image in the future. Mainly because it has not been tested yet on the long run. :-) However, Guix provides the tools to detect that something is not as expected. For example, we can imagine that today one says: this is the manifest to build the disk image and the hashes of the store are that (the same way one provides the MD5 of files when downloading); then in the future, building again the disk image, we can compare. Currently, it is impossible with Docker because all the distro are doomed (mutable). ;-) So what people are currently doing is to store all the Docker disk images. Docker motto: build once and run anywhere. Guix motto(*): build anytime and run everywhere. To me the relationship(**) is: guix pack -f docker -m stuff.scm docker load < /gnu/store/-fancy-name.tar docker push Hope that helps. All the best, simon [1] https://en.wikipedia.org/wiki/LXC (*) it is not official and my personal view ;-) (**) again my personal view.
Re: Relationship between Docker and Guix
Hi Ricardo! Thank you sooo much for taking your time for the explanation :) I will probably come back to this when I prepare that part of the presentation about it. Regards :) Laura
Re: Relationship between Docker and Guix
Hi Laura, > I am preparing a talk to introduce the community in a meetup, and one of > the topics the people want to know about is its relationship with > Docker. There is no direct relationship between the two. They serve different purposes and there’s little overlap, but there’s confusion about what Docker can do ad that has often lead to people assuming that it does way more for reproducibility than it actually does. Docker and systems like it are a combination of a “disk image” (i.e. an archive containing files and directories) and a tool to bind together Linux kernel features that can be used for virtualizing parts of the system. There are different levels of virtualization. One can emulate a CPU and all hardware in software and run a full operating system on that fully virtualized computer. Or one can use special hardware support to avoid having to implement the CPU in software. Similar features exist to avoid having to implement other hardware in software (“paravirtualization”). But traditionally with Linux that’s as far as things would go: you’d still virtualize the whole computer, the whole disk, and run a full operating system on it. With the Hurd, on the other hand, virtualization was always fine grain. You can, for example, redefine the directory tree as seen by a single process (i.e. file system virtualization). Or you could virtualize the network interface and share everything else. With Linux full-machine virtualization turned out to be a little too much for many applications. So eventually Linux gained features to virtualize the process namespace (a process can think it runs alone on the machine), the user account namespace (a process can think it runs as root), cgroups to virtualize e.g. memory (a process can use *all* the available memory, but the system pretends only a fraction of the memory is “all the available memory”), etc. With bind mounts and chroot (and others) Linux also has a way to virtualize the file system (and e.g. pretend that a certain directory is the root directory). Docker coordinates all of these features and thus enables fine grain virtualization for individual applications: applications can be delivered as big disk archives that are unpacked, mounted, and then declared to be the root file system from the point of view of the application. This makes application deployment really easy: just take a big disk image that contains the application and *all* its dependencies, and share the host system’s kernel, CPU, memory, and other file systems. Some people thought that this ease of deployment also translates to increased reproducibility: given the same disk image one could run an application off that image without having any of the host system’s libraries affect its operation. This is hardly different from old-school full system virtualization – it’s just more convenient and a tad lighter. Docker does not care about reproducibility of the disk images it uses. Generating these images at different points in time usually yields different disk images. The images can also contain way more stuff than needed. There is no direct correspondence between the contents of the disk image and the commands in a Dockerfile that may be used to generate the disk image. This can even be a security problem as the disk archive cannot be verified and hence cannot be trusted. Guix is not a container system, but when used on Linux it does support the same virtualization features (they are also exposed by “guix environment”, for example). Guix is concerned with providing a path from the declaration of a full environment to a reproducible binary. Guix can thus be used to generate Docker disk images in a reproducible fashion. Guix does without disk images (it just installs things in finer grain resolution to individual directories under /gnu/store) but still achieves isolation and separation of packages. Together with Linux container/virtualization features it can be used for pretty much the same things that Docker is often used for. -- Ricardo
Relationship between Docker and Guix
Hi Guix! I am preparing a talk to introduce the community in a meetup, and one of the topics the people want to know about is its relationship with Docker. I've read the manual, and have Ludo's talk from FOSDEM 2019, but if someone more experienced than me can help me (maybe there is a blog post, another talk, or through here) I would really appreciate it, because I am pretty new to Docker. Regards :) Laura
