On Thu, Mar 10, 2022 at 9:51 AM Peter Bergin <[email protected]> wrote: > > Hi Bruce, > > following up an old thread with some more toughts... > > On 2022-02-09 19:51, Bruce Ashfield wrote: > > On Wed, Feb 9, 2022 at 1:29 PM Peter Bergin <[email protected]> wrote: > >> Hi, > >> > >> I'm exploring the world of containers combined with Yocto. I can build a > >> container image and bundle that one with my rootfs image. The container > >> image is stored in the rootfs as a tar-file of a OCI image spec with the > >> content blobs, index.json and oci-layout. As per the description in > >> classes/image-oci.bbclass the way to run the container is to unpack the > >> file and the create the OCI runtime bundle and start it with runc. > >> > >> I have played around with docker and tried to import OCI image directly > >> in to docker store but have not succeeded. Anyone that knows if it is > >> possible? 'docker image import <oci-image>.tar does not give any errors > >> and the image shows up in 'docker images' but does not import the > >> correct rootfs. > > There's no viable way to do this on the build side, and then have it appear > > in the image. I've experimented several times with this, and haven't found > > a decent solution. Running docker on the build host is a non-starter, which > > rules out many options. > > > > There's more options if you use podman versus docker for the container > > runtime, but I also haven't had time to finish anything there yet. > > > > To get the OCI images into docker, I bounce them through a registry and > > use docker pull. You can see the logs of that process in several of my > > yocto summit presentations. > > > I have found another solution that I would like to have your input on. > I'm installing the OCI-image output from Yocto in my image for the host. > Then running `docker import` on it the first boot, then it is possible > to start. I don't know what I tried the last time but this seems to work > now: > > root@qemuarm64:~# ls -la > /var/lib/machines/container-hello-image-qemuarm64.rootfs.tar.bz2 > -rw-r--r-- 1 root root 1981366 Mar 9 2018 > /var/lib/machines/container-hello-image-qemuarm64.rootfs.tar.bz2 > root@qemuarm64:~# docker import > /var/lib/machines/container-hello-image-qemuarm64.rootfs.tar.bz2 hello > [ 35.155151] Initializing XFRM netlink socket > sha256:9db6cb5ce8f772caebf6ca430dd8de452e36320af861635f81f75cd4b9089f33 > root@qemuarm64:~# docker run -it --rm hello /bin/sh > [ 54.109667] docker0: port 1(vethe5c8c2b) entered blocking state > [ 54.110127] docker0: port 1(vethe5c8c2b) entered disabled state > [ 54.112606] device vethe5c8c2b entered promiscuous mode > [ 54.218708] IPv6: ADDRCONF(NETDEV_CHANGE): vethd1bfa4c: link becomes > ready > [ 54.219547] IPv6: ADDRCONF(NETDEV_CHANGE): vethe5c8c2b: link becomes > ready > [ 54.220098] docker0: port 1(vethe5c8c2b) entered blocking state > [ 54.220464] docker0: port 1(vethe5c8c2b) entered forwarding state > [ 54.221866] IPv6: ADDRCONF(NETDEV_CHANGE): docker0: link becomes ready > [ 54.349014] device vethe5c8c2b left promiscuous mode > [ 54.349629] docker0: port 1(vethe5c8c2b) entered disabled state > [ 56.939373] eth0: renamed from vethd1bfa4c > / # > > I see it could be hard to have ´docker import` in the Yocto build flow > but I think it is okay to have one step on first boot to import the > image to docker on target. With this it doesn't seems necessary to > push/pull through a registry. What do you think about this process? Any > drawback to have in mind?
It is something I've used in the past as well (import on first boot), but due to the nature of how the docker daemon, and tools run, it isn't feasible to do on the build side (which is what you mention as well). I've also arranged to have a local registry (docker-distribution) available, and the containers can be pulled from that on first boot, so no network access isn't required, but the registry is there, so it enables updates, etc, that are perhaps harder to do with just a docker import. No real downsides of that first boot import, outside of not having logs on the build side of the image construction, and you may have to go out of your way a bit to get licensing, sbom information for the containers (depending on how you are getting them into the image for first boot import). And to ramble on the docker import on the build side: I've spent some time trying to figure out how to easily drop permissions on docker, redirect where it stores the containers, and also reverse engineering the storage format so that we could manipulate it directly. None of the above proved to be viable about a year ago, but I tend to revisit it on a yearly cadence. podman is easier to use in this sort of mode (build side, with the storage redirected, etc), and is more viable .. but to keep all of the runtimes on a relatively even footing, I'd like to figure it out for them all at the same time. (sort of how we don't really want package manager specific features in OEcore). There have been presentations in conferences, and other folks doing docker imports on the build side of things, so it is doable. But it opens the door to licensing, sbom, reproducibility and other issues (since there's no guarantee that the containers are yocto built, hence could be from anywhere). As such, it wouldn't be an approach to bless/support in meta-virt itself (at least not until the above issues I mention have been sorted out), but it is feasible for people to do in their own layers. Cheers, Bruce > > Thanks, > /Peter > -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#7083): https://lists.yoctoproject.org/g/meta-virtualization/message/7083 Mute This Topic: https://lists.yoctoproject.org/mt/89028209/21656 Group Owner: [email protected] Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
