Hi all, in may 2020 we started a discussion about raw cloud images without tar, and without compression.... https://lists.debian.org/debian-cloud/2020/05/msg00057.html esp. mails referencing <[email protected]>
I want to advance this topic, because I think it's very confusing for our user that we have two directories providing different images for Openstack. Our goal is to have one tool to make cloud images and provide them in one directory. IMO we can easily provide raw uncompressed Openstack images for our users right now. Can't we just run a script on petterson after each release (not on the dailys) to untar the disk image, rename it (because it's always disk.raw) and generate a sha512sums file? Yes, the raw will be of 2G size, but this is much less than the 310 subdirectories in our daily builds. We also can have the sha512sum of the raw in the json file and check this. But this should only be a first step. Currently the images sizes are hardcoded in the python build scripts and fai-diskimage needs to know the size before partitioning the virtual disk and creating all files inside it. To get smaller raw images for Openstack (and maybe others cloud type) a better solution would be to create the filesystem for a cloud image (using fai dirinstall), and after that copy the files into a raw image that perfectly matches the size needed. growfs will then resize the partition during the first boot. Bastian already had some ideas in https://lists.debian.org/debian-cloud/2020/05/msg00075.html without using fai-diskimage. We also had some irc communication about this: --- Day changed Wed Jul 22 2020 13:44:49< Mrfai> zigo: I'm looking at how to make cloud images smaller using fai-diskimage. resize2fs does not seem to help much, because it does not shrink the FS as much as (theoretical) possible. 13:45:19< Mrfai> Do you build the openstack images (using your build script) with the autimatic-resize option? 13:45:35< waldi> Mrfai: that will not really help, i'm currently dropping fai-diskimage and fix a lot of other problems at the same time 13:47:24< Mrfai> waldi: it would be nice to communicate this making big changes. 13:48:33< waldi> Mrfai: i think i did already 13:49:38< Mrfai> The cloud team should have the opportunities to discuss bigger changes. You;ve mentioned this once, but IMO we should discuss the pros can cons in our irc meeting. 13:51:30< waldi> see /usr/sbin/freqanl-inject-data 13:51:32< waldi> err 13:51:37< waldi> <[email protected]> 13:58:51< waldi> and there are much more problems. stray mounts, like /run/udev, if files exist in the host system. which makes it almost impossible to cleanup files that should not be there. stray processes, which could be fixed by using pid namespaces in fai-diskimage 14:16:08< Mrfai> I know that fai-diskimage needs pid namespaces. I already have a patch for this. Still need to test it. --- Day changed Mon Jul 27 2020 21:30:30< kuLa[m]> <waldi "and there are much more problems"> waldi do you have anything written down what you suggest we should do about it? I'd suggest opening a bug against cloud.d.o to keep track of it Here's my list of things that are needed: create directory tree using fai dirinstall (using unshare) du -s create raw image (dd) of size XXX Handle MBR/EFI partition create partition and fs with minimal size (shell hooks?) copy directory tree adjust fstab? create grub.cfg write MBR/EFI info?? Anything missing? I like to discuss some technical details, for e.g. when to run our tests. What are pros and cons of implementing this in fai-diskimage or in our build script. Can we start this discuss during our next IRC meeting? -- regards Thomas
