Installing via iso.xz, really best idea? -- was Re: Gnome Boxes
Hi, On +2021-02-22 00:39:36 -0500, Evan Rowley wrote: > FreeBSD also provides .iso.xz. Some examples here: > http://ftp-archive.freebsd.org/pub/FreeBSD/releases/amd64/amd64/ISO-IMAGES/13.0/ > Looking there for an example: --8<---cut here---start->8--- FreeBSD-13.0-BETA2-amd64-bootonly.iso 363927552 2021-Feb-12 07:47 FreeBSD-13.0-BETA2-amd64-bootonly.iso.xz 82351604 2021-Feb-12 07:47 --8<---cut here---end--->8--- That seems like a really worthwhile compression! (ratio 4.4:1) However, I wonder if big .iso's are really the best way to take advantage of the xz (or any) compression for installation purposes. If the point of having the information in an .iso is to have it be bootable with everything (drivers, UI, etc) needed to install to any raw disk anywhere, ISTM that is wastefully general for my usual case: I have several laptops all able to dd to a USB-attached SSD or thumbdrive. I trust their impure apps to do the xfr and dd, if I can verify result hashes independently. I don't need to boot anything but my laptop for these apps: they're on my laptops. I don't intend to use my foreign laptop to synthesize content of the files to xfr by dd, so there is only dependency on the pure guix versions of fdisk, mkfs, etc used on the machine that executed guix system -make-custom-bootstrap-scripted-tarball my-neat-system.scm (in my dreams :) (maybe a modded version of guix pack could do it, but sizes?) ┌─┐ │ I don't want to modify the laptop I'm using (in fact, I want a guarantee │ │ that it WON'T be modified as I use it to dd-write to the attached USB storage). │ └─┘ (boxed for emphasis :) All I really need is a tarball with a bootstrap script that essentially does a series of dd file transfers to selected block sequences on the target disk(s). I am thinking this could be smaller and easier to use than using guix pack to concoct the same functionality. As the script starts, IWBN to be able to select target disk(s) interactively, probably using lsblk to probe and display, but I want to avoid partitioning target disk(s) with foreign tools (i.e. writing to the medium with other than dd). I think read-only access to get existing (if any) partition locations and sizes and types is ok, so the script can potentially choose between sparse files to dd into common sizes of disk and partition spaces where e.g. just padding with zeroes is not workable. IWBN to be able to target a toy 32-gb USB as well as the front end of a 500GB SSD with the same script/tarball. Those files for dd can contain ready-made images of GPT pieces -- superblock, VFAT fs, legacy-bootable grub blocks, /boot and / and maybe other specialized block-sequence images (with mkfs and transfers into the populated fs images already done on a pure guix system using loopback-mounted files for fs images), or whatever else may be needed. (caveat hunch: subject to some init-hook binary patching on first boot (hopefully no grub-launched or BIOS-validated thing like firmware update necessary if kernel init can do it ;/ ). For the case where I want to modify my own laptop, e.g. to boot from the USB-attached storage I just prepared, that depends on what its BIOS does for booting, but I guess that is commonly just a matter of updating grub's .cfg. For the case where I want to install to my own laptop's disk(s), I still don't like booting a monster iso. If my laptop's existing OS can't go into a safe maintenace mode, where the target disk is synced and unmounted, then I will have to reboot. If that is from a bootable installer USB, I want a rescue-size bootable, not the monster iso including everything (nor the job and wait of writing the iso image to a USB just so I can boot it once). ISTM a manifest for the installation job could be a file of symbolic by-hash links or equivalent sha256sum listing, and a list of mirrors, specifying directories where the by-hash links are findable, e.g. like https://example.com/pub/filedir/by-hash/ and links in that directory looking like e.g., e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 -> ../zerolengthfile (which are easy to generate selectively or wholesale for a filedir/ directory just creating the ./by-hash/ subdirectory and making the links)) ISTM like a custom bootstrap.sh could safely wget each file, piping directly to dd writing these chunks to raw disk slots, afterwards checking the sha256 of every block sequence it wrote. Done simply -- and irrespective of whether the by-hash links and files were stored in a git repo or plain file directories. I think "guix system" could automate the production of customized my-neat-system-boostrap.sh scripts containing the right wget and dd commands and sha256
Re: Gnome Boxes
FreeBSD also provides .iso.xz. Some examples here: http://ftp-archive.freebsd.org/pub/FreeBSD/releases/amd64/amd64/ISO-IMAGES/13.0/ On Thu, Feb 18, 2021 at 4:23 PM Julien Lepiller wrote: > Sorry, a compressed .iso is probably common, a .iso.xz is very uncommon > :). We even have had some reports of people trying to copy that directly to > an installation media. > > Le 18 février 2021 14:56:44 GMT-05:00, Tobias Geerinckx-Rice > a écrit : >> >> Julien Lepiller 写道： >> >>> Usually compression is provided by the webserver, but maybe ours >>> is not configured to do that. I think we're the only distro to >>> provide compressed isos. >>> >> >> Really? Most distribution ISOs use squashfs or similar with >> XZ/LZMA compression. It doesn't make sense to compress that over >> the wire. >> >> That said: XZ compression currently saves 27% (559M -> 405M). >> Transparently serving pre-compressed ISOs with nginx (gzip level >> 9) would save about 25% (559M -> 415M), which is surprisingly >> similar. >> >> Kind regards, >> >> T G-R >> >> -- - EJR
Re: Gnome Boxes
Sorry, a compressed .iso is probably common, a .iso.xz is very uncommon :). We even have had some reports of people trying to copy that directly to an installation media. Le 18 février 2021 14:56:44 GMT-05:00, Tobias Geerinckx-Rice a écrit : >Julien Lepiller 写道： >> Usually compression is provided by the webserver, but maybe ours >> is not configured to do that. I think we're the only distro to >> provide compressed isos. > >Really? Most distribution ISOs use squashfs or similar with >XZ/LZMA compression. It doesn't make sense to compress that over >the wire. > >That said: XZ compression currently saves 27% (559M -> 405M). >Transparently serving pre-compressed ISOs with nginx (gzip level >9) would save about 25% (559M -> 415M), which is surprisingly >similar. > >Kind regards, > >T G-R
Re: Gnome Boxes
Julien Lepiller 写道： Usually compression is provided by the webserver, but maybe ours is not configured to do that. I think we're the only distro to provide compressed isos. Really? Most distribution ISOs use squashfs or similar with XZ/LZMA compression. It doesn't make sense to compress that over the wire. That said: XZ compression currently saves 27% (559M -> 405M). Transparently serving pre-compressed ISOs with nginx (gzip level 9) would save about 25% (559M -> 415M), which is surprisingly similar. Kind regards, T G-R signature.asc Description: PGP signature
Re: Gnome Boxes
Usually compression is provided by the webserver, but maybe ours is not configured to do that. I think we're the only distro to provide compressed isos. Le 18 février 2021 12:18:44 GMT-05:00, "Ludovic Courtès" a écrit : >Hi, > >Julien Lepiller skribis: > >> When submitting the MR to libosinfo, there were concerns that the .xz >extension would cause problems with other software, so we left it out. >> >> I think we should provide a .iso download directly, if we want gnome >boxes to propose the guix system. > >We could do that but… isn’t it a bit ridiculous? I mean, compression >is >a well-known technique to reduce bandwidth usage, right? :-) > >That said, we now produce compressed ISOs IIRC, so it might be that the >extra compression layer doesn’t buy us much. Would be worth checking! > >Thanks, >Ludo’.
Re: Gnome Boxes
Hi, Julien Lepiller skribis: > When submitting the MR to libosinfo, there were concerns that the .xz > extension would cause problems with other software, so we left it out. > > I think we should provide a .iso download directly, if we want gnome boxes to > propose the guix system. We could do that but… isn’t it a bit ridiculous? I mean, compression is a well-known technique to reduce bandwidth usage, right? :-) That said, we now produce compressed ISOs IIRC, so it might be that the extra compression layer doesn’t buy us much. Would be worth checking! Thanks, Ludo’.
Re: Gnome Boxes
Hi! When submitting the MR to libosinfo, there were concerns that the .xz extension would cause problems with other software, so we left it out. I think we should provide a .iso download directly, if we want gnome boxes to propose the guix system. Le 9 février 2021 08:58:35 GMT-05:00, Adriano Peluso a écrit : >Gnome Boxes is GUI app that manages virtual machines and OSs images to >run in them > >It's a sort of Qemu for the rest of us. > >It offers a menu with images of OSs to download and run > >There are several versions of NixOs, among many others > >There's a recipe for Guix System by Julien Lepiller, it's here >https://gitlab.com/libosinfo/osinfo-db/-/blob/master/data/os/guix.gnu.org/guix-1.1.xml.in > >In their irc channel, the Boxes people infornmed me that they filter >out images that have no download url, as it seems to be the case with >this recipe > >In fact, as you can see, the download url is commented out (on line 15) > >Why is this ? > >Was there a specific problem with the download url ? > >It's be nice if I could pach this and have Guix System in the Gnome >Boxes menu > >Thanks
Gnome Boxes
Gnome Boxes is GUI app that manages virtual machines and OSs images to run in them It's a sort of Qemu for the rest of us. It offers a menu with images of OSs to download and run There are several versions of NixOs, among many others There's a recipe for Guix System by Julien Lepiller, it's here https://gitlab.com/libosinfo/osinfo-db/-/blob/master/data/os/guix.gnu.org/guix-1.1.xml.in In their irc channel, the Boxes people infornmed me that they filter out images that have no download url, as it seems to be the case with this recipe In fact, as you can see, the download url is commented out (on line 15) Why is this ? Was there a specific problem with the download url ? It's be nice if I could pach this and have Guix System in the Gnome Boxes menu Thanks
