Re: Progress on t64 transition -> building the installer in sid
Hello Cyril, list, On 21/03/2024 15:58, Cyril Brulebois wrote: https://lists.debian.org/debian-boot/2024/03/msg00102.html The diagram shows nicely that the t64-transition is affecting the installer, with currently 1 major bottleneck, libpng16-16t64-udeb: https://d-i.debian.org/dose/graph-unstable-amd64.png On openQA I've additionally seen that for Debian Edu, the installer fails with the message that libaio.1.so is missing, so some udeb is probably also requiring an update. Do you have more details? That thing doesn't exist, but libaio.so.1 does (different suffix order). My bad, I reversed the order when typing. I've done some basic triaging in the openQA comment: https://openqa.debian.net/tests/244163#comments The installer fails here: https://openqa.debian.net/tests/244163#step/grub/3 Some details are here (/var/log/syslog): https://openqa.debian.net/tests/244163#step/grub/35 In any case, there are no reasons to complicate the t64 transition with transitioning udebs, so I wouldn't be surprised if “images” (whatever they are) built against old udebs would break if newer udebs are pulled from the network. The images I've spoken of are the daily-built Debian live ISO-images based on sid. They are built by Jenkins https://jenkins.debian.net/view/live/ Seeing the breakage on sid before the weekly-build installer breaks on trixie would be nice :-) With kind regards, Roland OpenPGP_signature.asc Description: OpenPGP digital signature
Progress on t64 transition -> building the installer in sid
Hello list, Since two days the live images based on sid can be generated again (hurray!), now that debootstrap is able to generate a bootstrap again. The smallest image is generated properly now, because it does not have an installer. For the other images, the installer is currently failing to build from source, as some dependencies (in the udebs) are still missing (due to the t64-transition). The latest message (from my local build_cdrom_gtk.log) is: The following packages have unmet dependencies: libcairo2-udeb : Depends: libpng16-16t64-udeb (>= 1.6.2) but it is not installable libfreetype6-udeb : Depends: libpng16-16t64-udeb (>= 1.6.2) but it is not installable libgdk-pixbuf-2.0-0-udeb : Depends: libpng16-16t64-udeb (>= 1.6.2) but it is not installable libinput10-udeb : Depends: libmtdev1t64 but it is not installable On openQA I've additionally seen that for Debian Edu, the installer fails with the message that libaio.1.so is missing, so some udeb is probably also requiring an update. With kind regards, Roland Clobus OpenPGP_signature.asc Description: OpenPGP digital signature
Re: Weekly builds of installer/live images failing
Hello Daniel, On 16/01/2024 09:52, Daniel Lewart wrote: However, on Jan 8 and 15, the weekly builds of installer/live images have been failing. I'll guess that these are part of the fallout from "the big linux changes": https://lists.debian.org/debian-kernel/2023/12/msg00421.html https://salsa.debian.org/kernel-team/linux/-/merge_requests/851 Indeed. The kernel changes and usrmerge issue in the installer have caused build failures for the weekly builds. They appear to have been resolved now, and Jenkins shows that the daily builds for the live images on sid are building (reproducibly) again. The tests in OpenQA also look good. With kind regards, Roland Clobus OpenPGP_signature.asc Description: OpenPGP digital signature
Bug#1057844: bookworm live images: d-i speech installation not loading sof firmware
On 09/12/2023 15:48, Steve McIntyre wrote: Package: cdimage.debian.org Severity: important Tags: a11y Testing the gnome live image for the 12.3 release... Running d-i from that image complained early on about missing Intel SOF firmware. Later on, the same image finds and loads intel wifi firmware just fine. Checking on the image, all the firmware debs are sym-linked in /firmware just fine but we don't have the extra metadata file (Contents file and dep11 stuff). Early on, AFAICS this means that the sof firmware isn't detected and loaded. Indeed, checking a d-i speech installation from this image, I don't get audio. Argh. Checking the same machine (Thinkpad T14s) with the amd64 netinst, it successfully loads firmware and starts the speech installer. This isn't new for Bookworm 12.3, is it? There has been no change in the images generated by live-build regarding the /firmware folder, they have always contained just the symlinked .deb files. I only recently started to look deeper into the special folders for the installer and noticed that the content of /firmware differs between the netinst and the live images. Let's discuss this under less time pressure (today is the 12.3 release day). I think that this ticket could be re-assigned (or cloned) to live-build for the implementation of the fix. If needed, could a 12.3.1 live image be released, with just this fix, instead of waiting for 12.4? With kind regards, Roland Clobus OpenPGP_signature.asc Description: OpenPGP digital signature
Automating testing for the netinst and live images
Hello Debian-cd Team and Phil, Now that the busy period for releasing Debian 12.1 is over and some of the live images have been verified by openQA (via me), would it make sense to think about automating the tests for the officially released Debian images, and the weekly builds as well? My thoughts/ramblings: * As soon as an image has been generated, and it has been made accessible via an URL, openQA will be invoked and starts to download and test the image (i.e. the generator will trigger openQA instead of openQA polling) ** Phil will be able to generate API keys for openQA ** I've already implemented a similar setup on Jenkins [1] for the live images [6] ** Phil has already implemented a similar setup for the netinst images, using polling [2] * By testing on virtualised hardware, at least many of the manual, tediously repeating tests can be verified to work correctly, which could make the tests on real hardware faster, because less needs to be tested * Automated tests would automatically see e.g. kernel mismatches in the installer [3] ** However, for the live images (based on testing and unstable) I've implemented an automatic kernel selection, which saves additional maintenance [4] * The automated tests will show issues earlier, but that would require regular monitoring/dashboarding ** I've tried to tags the issues that I've reported [5] * For the medium to long term, would it make sense to shift these test from debian.net machines to debian.org machines? ** The workload on osuosl3-amd64.d.n is already rather high That's already a lot for a single mail, with kind regards, Roland Clobus [1] https://jenkins.debian.net/view/live/ [2] https://openqa.debian.net/group_overview/10 [3] https://openqa.debian.net/tests/overview?distri=debian=testing=20230724_1119-testing-amd64=10 [4] https://salsa.debian.org/live-team/live-build/-/blob/master/scripts/build/installer_debian-installer#L309= [5] https://udd.debian.org/cgi-bin/bts-usertags.cgi?user=debian-qa%40lists.debian.org=openqa=html#results [6] https://openqa.debian.net/group_overview/14 OpenPGP_signature Description: OpenPGP digital signature
Push in git repo missing?
Hello Debian-cd Team, now that 12.1 has been released, is a push to the git repository at Salsa missing? The file `available/CONF.sh.bookworm_release` still mentions 12.0.0 With kind regards, Roland Clobus OpenPGP_signature Description: OpenPGP digital signature
Bug#1040899: debian-cd: "Failed to mount /target/proc" during GRUB installation
Control: affects 1039710 debian-cd On 12/07/2023 09:26, Gioele Barabucci wrote: installing Debian 13 testing using today's image [1] always fails at the boot loader installation stage with the error "Failed to mount /target/proc". After this message is shown, GRUB is left uninstalled and it is not possible the boot the system. This was also reported in #1039710, but diagnosis is hindered by an empty syslog which needs to be resolved first. With kind regards, Roland Clobus OpenPGP_signature Description: OpenPGP digital signature
Re: .zsync files for the weekly live images
Hello Debian-Images team, (and reply to self for the second time) :-) On 12/04/2023 17:03, Roland Clobus wrote: On 08/04/2023 10:48, Roland Clobus wrote: I've tested the .zsync files on [1]. They currently don't work, because the original filename (live-image-amd64.hybrid.iso) is embedded in those files. I've chosen to turn off zsync: https://salsa.debian.org/images-team/live-setup/-/merge_requests/3 Reasons: * The package zsync was orphaned for 1.5 years ago * The package zsync does not support https for the URL with the .zsync file * Minor reason: The live-build-binary script could generate the images with the correct filename part, but the extension (.hybrid.iso) is fixed by the live-build package. With kind regards, Roland Clobus [1] https://get.debian.org/images/weekly-live-builds/amd64/iso-hybrid/ OpenPGP_signature Description: OpenPGP digital signature
Re: .zsync files for the weekly live images
On 08/04/2023 10:48, Roland Clobus wrote:
Hello Debian-Images team,
I've tested the .zsync files on [1].
They currently don't work, because the original filename
(live-image-amd64.hybrid.iso) is embedded in those files.
As a fix, I see several possibilities, which do you prefer?
1) Turn off zsync
2) Keep zsync, and generate unique filenames
Option 1 would possibly increase the amount of traffic on
get.debian.org. Are metrics available whether the .zsync files are used
at all?
I haven't tested whether the zsync method will work properly for the
live images, as they contain a huge squashfs file embedded in the iso
file (so a compressed file inside another compressed file), so I don't
know whether it will help in saving bandwidth.
Option 2 would mean that (with the current code) filenames can be
generated with the structure
`{something-we-decide}-{architecture}.hybrid.iso`, e.g.
`debian-live-testing-gnome-amd64.hybrid.iso`. This is different from the
current names on [1].
If we go for option 2, it will need (small) changes in the `live-build`,
`live-setup` and `jenkins.debian.net` git repos.
With kind regards,
Roland Clobus
[1] https://get.debian.org/images/weekly-live-builds/amd64/iso-hybrid/
In my previous mail I've expressed my doubts that zsync would be able do
reduce the amount of bytes that is to be downloaded. I was able to do an
experiment now, and it looks good.
With the GNOME images from 2023-04-03 and 2023-04-10 I got the following
output:
```
verifying download...checksum matches OK
used 3612643328 local, fetched 84375881
```
In order to do so, I had to convince zsync to use different filename
(which I managed by splitting the zsync file into its text and binary
part, adjusting the text part and combining both parts again).
Of the 3.6GB only 84MB needed to be downloaded, which is quite
impressive, given that the compressed file `filesystem.squashfs` also
has many differences.
This looks promising enough to prepare for option 2.
With kind regards,
Roland Clobus
PS: I'm not the only one reporting the issue with zsync:
https://lists.debian.org/debian-cd/2023/04/msg00016.html
OpenPGP_signature
Description: OpenPGP digital signature
Re: live-installer update for Bookworm?
Hello Cyril, list, On 10/04/2023 10:14, Cyril Brulebois wrote: Roland Clobus (2023-04-10): It turns out that the package 'live-installer' (which is a udeb-only package) in Bookworm still is at version 57 [2], while the fix was released at version 58. Could the package be unblocked? ... I asked Jonathan on IRC while preparing RC1 (slightly edited): [ kibi] highvoltage: I don't think you answered my live-installer question? [ highvoltage] kibi: I didn't think it warranted an unblock [ highvoltage] kibi: but if it did for roland's changes then he could file one [ kibi] highvoltage: ok, ta Besides the obviously missing `dch -r` call (“Mon, 15 Apr 2019” was quite surprising for something uploaded in March 2023), there are lots of changes that are nowhere suitable at this stage of the release process. I would guess that the timestamp from 2019 was the moment 'Fix·typo·in·previous·changelog·entry' was written. Aside from Janitor, modernisation and lintian corrections, my change was the only functionality change in four years (as seen by diffoscope, see below). There is no autopkgtest, but the openQA run for sid shows that the change has its intended effect. If I had known that I would need to file an unblock request, I would have done so, but the timing (patch, merge, release) was rather unfortunate. That plus Jonathan's answer triggered my deciding against unblocking the package on my own (with my d-i release manager hat). That being said, if the release team is willing to unblock the package as is, that'd be fine with me. I suppose it'll be suggested to cherry-pick the desired change(s) and to upload 57+deb12u1 via tpu, or to back out the undesired changes and proceed with 59 via unstable. (Both are fine from a d-i point of view, as long as the package reaches testing in the end.) It is certainly not a release critical issue, but I personally find it quite annoying to have to wait about 30 minutes for the installation, and then to read 'Installation is complete, so it is time to boot into your new system', press a key and then wait another 2-3 minutes before the reboot is actually performed, while the additional waiting time could have been incorporated into the longer non-interactive phase. With kind regards, Roland --- dget https://deb.debian.org/debian/pool/main/l/live-installer/live-installer_57.dsc dget https://deb.debian.org/debian/pool/main/l/live-installer/live-installer_58.dsc diffoscope live-installer-57 live-installer-58 --html live-installer-57vs58.html PS: No need to CC me, I'm subscribe to the mailing list OpenPGP_signature Description: OpenPGP digital signature
live-installer update for Bookworm?
Hello images-team, I've wondered why the installation still takes a long time after it has finished installing from a live image, as seen on openQA [1]. It turns out that the package 'live-installer' (which is a udeb-only package) in Bookworm still is at version 57 [2], while the fix was released at version 58. Could the package be unblocked? (And/Or will there be an installer RC2?) The new version of live-installer is working correctly, as can be seen by a sid build of the live image on openQA [4]. With kind regards, Roland Clobus [1] https://openqa.debian.net/tests/139702#step/complete_install/44 [2] https://packages.debian.org/search?keywords=live-installer [3] https://salsa.debian.org/installer-team/live-installer/-/commit/ccda07e77f6f2071156941a3c53fd100dcbf5440 [4] https://openqa.debian.net/tests/138478/video?filename=video.ogv See the movie at 0:50-0:56 OpenPGP_signature Description: OpenPGP digital signature
.zsync files for the weekly live images
Hello Debian-Images team,
I've tested the .zsync files on [1].
They currently don't work, because the original filename
(live-image-amd64.hybrid.iso) is embedded in those files.
As a fix, I see several possibilities, which do you prefer?
1) Turn off zsync
2) Keep zsync, and generate unique filenames
Option 1 would possibly increase the amount of traffic on
get.debian.org. Are metrics available whether the .zsync files are used
at all?
I haven't tested whether the zsync method will work properly for the
live images, as they contain a huge squashfs file embedded in the iso
file (so a compressed file inside another compressed file), so I don't
know whether it will help in saving bandwidth.
Option 2 would mean that (with the current code) filenames can be
generated with the structure
`{something-we-decide}-{architecture}.hybrid.iso`, e.g.
`debian-live-testing-gnome-amd64.hybrid.iso`. This is different from the
current names on [1].
If we go for option 2, it will need (small) changes in the `live-build`,
`live-setup` and `jenkins.debian.net` git repos.
With kind regards,
Roland Clobus
[1] https://get.debian.org/images/weekly-live-builds/amd64/iso-hybrid/
OpenPGP_signature
Description: OpenPGP digital signature
Re: Starting the weekly live images for Bookworm building again
My last cross-post. Follow-up please on debian-live Hello Steve and lists, On 19/03/2023 16:13, Steve McIntyre wrote: So, after some delay from me and some further delays from various Debian machines committing suicide, I've got bookworm live builds running again. \o/ Thanks for merging the changes. I've taken Roland's updated patches and tweaked a little more in the setup.git and live-setup.git repos, and we now have live builds integrated. Changes I've added: * turned on source tarball generation using LB_SOURCE=true, and disable the external source build that we did for the older live-wrapper builds That's a good way to enable the source tarballs. I haven't looked at that part of live-build for a while, so the source tarball might need some love. * when building on casulana, warn about archive updates rather than restarting builds * don't attempt to build i386 live images any more, they're not useful * tweaked logging And an additional change to use the installer images from the repository instead of rebuilding them from git [1]. That change is now causing the missing kernel modules for d-i. I've rebuilding the installer images from git, after the discussion in #1006800 [2], to save the d-i team from doing an upload for every single kernel bump, while bookworm was not yet in freeze. (That is a recurring issue for every release [3]) So, *builds* work fine but I've not *yet* tested actually booting/using one of these images in any way. I've just triggered a full build of "testing" live images now, please help test if you can once they're in place at [2] in a couple of hours from now. > [2] https://get.debian.org/images/weekly-live-builds/ In openQA [4] the live images that were generated by Jenkins [5] have been tested for a while now. Now we can switch and use these new images instead of the images from Jenkins. Since both images are generated by the same script, I wouldn't expect many issues. I don't yet know how close we are to having full non-free-firmware integration with the live images; I expect there might be some more work needed there yet, but I'd love to be proven wrong. :-) This weekend I've written the missing parts, non-free-firmware images are now generated by default by live-build, and the ISO images are still bit-for-bit reproducible. With kind regards, Roland Clobus [1] 3cef309a5cfa4758ba33480b170734133b7104b5 [2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1006800 [3] #986506, https://lists.debian.org/debian-boot/2021/03/msg00166.html [4] https://openqa.debian.net/group_overview/14 [5] https://jenkins.debian.net/view/live/ OpenPGP_signature Description: OpenPGP digital signature
Re: Starting the weekly live images for Bookworm building again
Hello lists (sorry for cross-posting to so many lists), This is a follow-up of my mail from 2022-11-21 [A1]. I've made progress in the last two months, but would like to have some merge requests approved, to get more traction and to allow others to jump aboard and make the live images for Bookworm possible. As you can see, this affects many teams: * live-setup: a MR to generate all live images for Bookworm [A2] * localechooser: A minor fix [A3] * live-installer: A better user experience after the installer is finished [A4] * live-build: Various installer improvements, including off-line installation [A5] With kind regards, Roland Clobus [A1] https://lists.debian.org/debian-devel/2022/11/msg00221.html [A2] https://salsa.debian.org/images-team/live-setup/-/merge_requests/2 [A3] https://salsa.debian.org/installer-team/localechooser/-/merge_requests/7 [A4] https://salsa.debian.org/installer-team/live-installer/-/merge_requests/3 [A5] https://salsa.debian.org/live-team/live-build/-/merge_requests/297 OpenPGP_signature Description: OpenPGP digital signature
Bug#1017555: Let's drop win32-loader from amd64/i386 and multiarch CDs
On 17/08/2022 21:08, Didier 'OdyX' Raboud wrote: Following the recent discussion on d-boot [1], it seems we agreed to drop win32-loader from the Debian CDs, as it's not likely to be very useful these days. win32-loader seems also present on debian-installer [2], I'll therefore clone the bug. FYI: win32-loader has also been dropped from live-build images. https://salsa.debian.org/live-team/live-build/-/commit/6908dfd6df69c9e9cd2458d9987b40b06bf1cfd3 With kind regards, Roland Clobus OpenPGP_signature Description: OpenPGP digital signature
Re: Live System images and Pure Blends
Hello Stefan,
On 24/07/2022 13:02, Stefan Kropp wrote:
I have been started to look into live system generation
(live-build), Be honest, I have never been used a Debian Live CD.
Few days/weeks back I looked into the live CD which are provided
by Debian (live-wrapper).
Your timing is rather unfortunate. At the moment there are no recent
(i.e. bookworm/sid) live images being generated and the snapshot server
that I use to generate them was shut down during DebConf22.
However, yesterday I updated the script to generate live-build-based
images such that can be run without a snapshot server.
See the recently updated Wiki page
(https://wiki.debian.org/ReproducibleInstalls/LiveImages) in the section
'Building'
I also think, it is very important to provide live-systems also
for the next releases.
I'm working on that, but it needs more time.
I have been looked into:
* debian-live-11.4.0-amd64-standard.iso (1,1G / 800 packages)
* debian-live-11.4.0-amd64-xfce.iso (2,5G / 2270 packages)
* debian-live-11.4.0-amd64-gnome.iso (2,7G / 2572 packages)
The live-build sid-based GNOME image is now 3.2GB / 2540 packages
You could download a recent GNOME live image from openQA:
https://openqa.debian.net/tests/65162/asset/iso/gnome_sid_20220711T17Zdeb.iso
The tests that running are at:
https://openqa.debian.net/tests/overview?distri=debian=sid=20220711T17Z_sid=14
I'm wondering if we should refine those images. Maybe the Debian
Project and the users can benefit from it.
These images use the meta-package 'live-task-' to select which
packages belong to each desktop. E.g live-task-gnome depends on
task-gnome-desktop. Any adjustment can take place in these meta-packages.
One of the live images is called "standard". What does standard
means? What can the user do with the "standard" image?
The standard image does not contain a graphical desktop. It could have
been called 'text' as well.
I looked
into xfce and gnome. The gnome image also includes few games,
evolution and thunderbird. xfce doesn't have a mailclient at all.
You might want to report a bug against live-task-xfce or
task-xfce-desktop to add a mail client.
I think one person at the BoF said, it's not necessary to build a
image for every desktop manager. Personally, I agree.
For the record, that was Mrfai.
While my time is still limited, I will also (have to) focus on a few
images at first.
I asked myself:
* Who will use the live system?
* Why will somebody use the live system?
* When should we provide a Debian Pure Blend?
Who will use the live system? Everybody!
Why will somebody use the live system? Everything!
It may helpful we the project defines some images with a scope.
Maybe something like this,..
# Rescue Disk Image
An image for a live system to rescues a broken system. It may
have a set of useful tool, but may not have a X-Server. Powerful
editor, maybe also gnu compilers and manpages and screen / tmux.
This is not just a 'small' Rescue Disk Image this will be a real
powerful Debian system (without X).
I don't want to discourage you, but there are already several
Debian-derivatives that do a good job. Also the Debian-installer (from
the boot menu) has some rescue capability (not tried by me yet)
# Calamares-Installer
Live System to provide the Calamares-Installer. The live system
can be used to install Debian with Calamares.
(I haven't used it - I can not give much feedback, yet)
The Calamares installer is targeted at a broader audience.
At the moment there is no automated test yet in openQA.
The use case is: "Let's try Debian and see what it is". The
result: "It's awesome!" To achieve this result, we need to have a
good set of pre-installed applications.
Now, pure blends come in. I think the goal of the pure blends is
exactly what we need for Desktop / Office / Junior / Med / ...
The 'experts' (blends team) knows which packages are required /
helpful / working.
Agreed.
One idea would be to provide packages like
debian-{desktop,office,med,junior}-live-system
Those packages will include a well defined structure to build the
live-system images via live-build.
You can use a 'Tasks' meta package for that.
Personally, I think it would be much better to provide "soon" the
possible to access to the pure blends, instead to "wait" a long
time until we are able to find a nice solution for a hierarchical
selection. Sure, if somebody is able to provide a solution, it's
fine.
You can file an ITP for the 'task-debian-junior' metapackage.
We should also work on a README.html / pdf file as a first
"Welcome" and guide new users where they can get more help. Links
to pre-installed documentation, Debian Wiki pages, Debian Local
Groups.
I have created a small example of a Debian Desktop live system.
https://salsa.debian.org/StefanKropp/debian-desktop-live-system/
The 'Welcome to Debian' is a
Bug#1013432: debian-cd: UEFI boot uses black text on dark blue background
Package: debian-cd Version: 3.1.35 Severity: normal Hello, since grub version 2.06-3 entered sid, the UEFI menu of the daily netinst image shows a black text on the dark blue background for the selected boot item. https://cdimage.debian.org/cdimage/daily-builds/sid_d-i/arch-latest/amd64/iso- cd/ debian-testing-amd64-netinst.iso This can be seen e.g. at https://openqa.debian.net/tests/61507#step/bootwalk_0/2 It appears that the grub theme is active now (which was not the case for e.g. the netinst-11 images). As a local hack, I've replaced: selected_item_color = "black" with selected_item_color = "white" in boot/grub/theme/1 inside my netinst image. The original file that is used is: https://sources.debian.org/src/debian-cd/3.1.35/data/bullseye/grub- theme.in/?hl=58#L58 With kind regards, Roland Clobus -- System Information: Debian Release: bookworm/sid APT prefers testing APT policy: (990, 'testing'), (500, 'testing-debug'), (50, 'unstable') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 5.18.0-1-amd64 (SMP w/8 CPU threads; PREEMPT) Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8), LANGUAGE=en_GB:en Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages debian-cd depends on: ii apt2.5.0 ii bc 1.07.1-3+b1 ii bzip2 1.0.8-5 ii cpp4:11.2.0-2 ii curl 7.83.1-2 ii dctrl-tools [grep-dctrl] 2.24-3+b1 ii dpkg-dev 1.21.8 ii genisoimage9:1.1.11-3.2 pn libcompress-zlib-perl pn libdigest-md5-perl ii libdpkg-perl 1.21.8 ii libfile-slurp-perl .32-1 ii libyaml-libyaml-perl 0.83+ds-1+b1 ii lynx 2.9.0dev.10-1 ii make 4.3-4.1 ii perl [libdigest-sha-perl] 5.34.0-4 ii tofrodos 1.7.13+ds-5 ii wget 1.21.3-1+b2 ii xorriso1.5.4-2 Versions of packages debian-cd recommends: ii dosfstools 4.2-1 ii hfsutils 3.2.6-15 ii isolinux 3:6.04~git20190206.bf6db5b4+dfsg1-3 ii mtools 4.0.33-1+really4.0.32-1 ii syslinux-common 3:6.04~git20190206.bf6db5b4+dfsg1-3 debian-cd suggests no packages. -- no debconf information
Start building live images again for bookworm and sid
Hello Debian-cd list, I've collected my thoughts and have published a summary about live CD generation for Bookworm (and sid), updated to the state of today. https://rclobus.nl/blog/?p=286 It is now about 2 years ago that I wrote about the live image generation, perhaps it will be in time to have support in Bookworm. At this moment they are written down on my personal blog, but if you would like to add your own thoughts, perhaps I should convert it to a Wiki page somewhere within the Debian wiki structure. I've had some private discussions and I know that my proposal to use live-build will not be popular (understatement). Please comment and share your thoughts. I'll be at the Hamburg reunion to talk about this in person. See you soon, With kind regards, Roland OpenPGP_signature Description: OpenPGP digital signature
Re: Custom Buster live-build
+debian-live Hello Nathanael, On 29/06/2021 06:14, Nathanael Higley wrote: > Hi my name is Nathanael and I was wondering how I could build an > unofficial/ for personal use Debian Buster livecd with updates and > security patches included on the disc? > I've had previously built a custom live-cd with live-build on Debian > stretch with live-images or more precisely the gnome-desktop live-image > configuration. I had modified the configuration to include the full > gnome-desktop development stack with gnome-boxes emulation, however It's > been a good while since I've built a live-cd, what tools or process > would you recommend? The official live images for Debian Buster are created by the tool live-wrapper, before that the tool live-build was used (which you already used for Stretch). Image for Bullseye can be built by either tool. I'm biased, I've worked in live-build in the last couple of years, I would recommend to use live-build. You can take a look at the Wiki [1] page that details some command line examples for e.g. building a GNOME live image, and the documentation [2]. With kind regards, Roland Clobus [1] https://wiki.debian.org/ReproducibleInstalls/LiveImages [2] https://live-team.pages.debian.net/live-manual/ OpenPGP_signature Description: OpenPGP digital signature
Replacing live-wrapper for live images by live-build?
Hello Debian-Live list, Debian-Images list, debian-boot list, About a year ago I wrote [2] about my steps to reproduce the command line that is currently used to generate the live images. By then it was already clear that live-wrapper needs a replacement. Steve McIntyre wrote at that time: "The current live-wrapper code, and vmdebootstrap, are both basically dead IMHO. I've suggested moving to something else supported like FAI instead, like the Debian cloud images. highvoltage has other ideas. I'm not working on anything live-related at all any more." In November [1] I wrote to the live list about 'Porting the standard image from live-wrapper to live-build'. Since then I've continued working on live-build, which is now IMHO in a good shape (it even creates reproducible images [3]), but live-build would need some final features to be a proper replacement. These final features can be subdivided in a few categories: * Accessibility: better support for speech-synthesis and adding beeps to isolinx/grub * Localisation: support for running the live image in another language starting from the boot * Cosmetic: using the official Debian 11 artwork during the boot * Branding: the live image might need to differ between 'official' and 'unofficial' images Each of these categories can be tackled with reasonable effort, but some coordination might be required. My questions: * Has it already been decided what the replacement for live-wrapper will be? * If no, would you consider using live-build again? With kind regards, Roland Clobus [1] https://lists.debian.org/debian-live/2020/11/msg1.html [2] https://lists.debian.org/debian-live/2020/03/msg00225.html [3] https://wiki.debian.org/ReproducibleInstalls/LiveImages OpenPGP_signature Description: OpenPGP digital signature
Re: Do MacBook support/need EFI secure boot (Was: Porting the standard image from live-wrapper to live-build)
Hello Jeroen and list, Now that I'm able to reproducibly build images with live-build, I'm looking at missing features in live-build. On 17/11/2020 11:03, Jeroen Diederen wrote: > On 17/11/2020 10:44 Luca Boccassi wrote: >> On 11/11/202 11:54 Roland Clobus wrote: >>> live-build:>>> * /EFI/boot contains a 32-bit EFI image on the amd64 iso.>>> >>> ** AP: Is this needed/correct? >> IIRC yes - it's rare, but I think there is hardware out there with>> 32bit >> EFI and 64bit CPUs. > Correct, early MacBooks are of this type. I own one, a MacBook 2,1. Do you know whether your MacBook 2.1 supports/needs EFI secure boot? Currently the live-build generated image does not contain a signed EFI, but it could get one with a reasonable amount of effort. With kind regards, Roland Clobus OpenPGP_signature Description: OpenPGP digital signature
Porting the standard image from live-wrapper to live-build
Hello Debian-Live list and Debian-Images list, On 2020-03-21T17:27+0200 [1] I wrote these mailing lists about the generation of the official live images for Debian. Since then I have announced my effort to make the generation of live images reproducible [2] and have come really far (for the standard image). As detailed in my blog [3] I can re-generate the official Debian 10.6 Live Standard image now, using live-build. Since vmdebootstrap isn't available in the current Debian Testing, and appears not to be planned to be included, I've prepared an image that is as similar as feasible/meaningful using the latest git version of live-build (with my own patches for adding reproducibilty and a few hacks on top) [4]. With Debian point release 10.7 announced [5], now would be perhaps an ideal moment to make the switch to live build. This is a long mail with many questions, but please take the time to answer some... First, my restrictions/goals: * While testing, use as little bandwidth as possible (therefore using apt-cacher-ng) * Re-create the live-wrapper image for Debian 10.6 in the standard configuration as close as possible * Try to make a reasonable well reproducible live image * Have reasonable short test cycles (therefore using /dev/shm) Below follows a list of differences between the live-wrapper and live-build images, and my personal opinion about them. Some features are not present in live build and needs further action (marked with AP). Please answer to this mail to confirm or reject my proposals: live-wrapper: * ISO volume ID contains '10.6.0' ** AP: live build mentions 'buster', which contains less details * Makes a beep on boot ** AP: Should this be added to live build as well? * The grub configuration contains 'SAYS ...' ** AP: Should this be added to live build as well? * /pool contains some packages that are not in the squashfs image ** live build has a more minimal list * /pool contains exim4, mailutils, mokutil, python2.7, but it is not in the squashfs ** live build has a more minimal list * /pool contains multiple versions of udeb files ** live build has only the active version * The grub and isolinux menus contain all available languages ** AP: It would be nice to reproduce this in live build * Contains /etc/hosts, /etc/machine-id, several logs in /var/log, references to contrib and non-free in /var/lib/apt/lists ** live build cleans up these files better * Does not contain apparmor (recommends of linux-image-4.19.0-11-amd64) ** live build contains all recommends packages * Does not contain acpi-support-base (recommends of acpid) ** live build contains all recommends packages * Contains /etc/modprobe.d/qemu-blacklist.conf (for bochs-drm) ** AP: Is this also needed in live build? * Encoding is us-ascii, live build uses utf-8 ** I think utf-8 would be the best * The boot splash screen uses the Debian theme ** AP: live build shows a helmet and the versions of the live packages. We need the Debian-themed splash screen, combined with the version numbers live-build: * /EFI/boot contains a 32-bit EFI image on the amd64 iso. ** AP: Is this needed/correct? * /pool contains firmware-free ** This is missing in the live-wrapper image * boot/grub/grub.cfg: No findiso= line in the fail-safe mode ** AP: Untested by me, is it a bug? * xorriso complains about issues with Joliet (symlinks not supported, volid too long) ** AP: Do we need support for Joliet? Untested: does Windows XP/7/10 support RockRidge sufficiently well? And is it needed? * The packages lists are available both uncompressed and as gzip file ** AP: Isn't just one variant suffient? live-wrapper only has the uncompressed file My comments to the command line options to lb config (as shown below): * --security false ** AP: Shouldn't this be true per default for Debian Stable? * --updates false ** AP: Shouldn't this be true per default for Debian Stable? * --loadlin false ** Is this still tested? (I don't have a computer which can run 16-bit executables at the moment) * acpid ** If this is needed, shouldn't if be in the live-task-standard package? * Rename install to d-i ** Only needed to make the image more similar to the live-wrapper image -> should not be merged to the git repository of live-build * The git repo [4] has 2 commits which start with 'HACK' ** Only needed to make the image more similar to the live-wrapper image -> should not be merged to the git repository of live-build With kind regards, Roland Clobus --- Appendix: Additional comparison on top of the result of diffoscope --- # Compare the squashfs images # 1. Align the timestamps to the official image. Everything after 10:00 will get the SOURCE_DATE_EPOCH timestamp # 2. Remove the directories from the diff -> they might have different 'sizes', which are not of interest TZ=UTC unsquashfs -lls live-wrapper-mounted/live/filesystem.squashfs | sed -e "s/2020-09-26 10:[0-9][0-9]/2020-09-26 10:42/" | sed -e "/^d/d" > live-wrapper.squash TZ=UTC un
Re: Patching apt-cacher-ng (Was: Documenting the generation of the live images in Debian)
On 21/03/2020 18:53, Eduard Bloch wrote: >> In reply to the patch at https://bugs.debian.org/954437 > I cannot accept your patch as-is. Because it's brute-force, a directory > is not neccessarily a non-volatile file. Can you give me any hints how > to distinguish between volatile and non-volatile directories? Maybe the > same as it's done for for other d-i files, by regex, so > > "|/dists/.*/installer-[^/]+/[0-9][^/]+/images/.*" // d-i stuff > with revision > > means frozen contents and > > "|/dists/.*/installer-[^/]+/[^0-9][^/]+/images/.*" // d-i stuff > but not containing a date (year number) in the revision directory (like > "current", "beta", ...) > > is volatile? (those regexps are already defined, I just need to change > your patch to make them applicable to directories) Agreed. The patch is a brute-force hack, but at least it allowed me to build my first CD unofficial images. After I've written my summary to the list, I dug deeper into live-wrapper. It uses either the released installer (using the apt proxy), which doesn't contain versioning information in the path, but will most probably be rather non-volatile, from https://deb.debian.org/debian/dists/buster/main/installer-amd64/current/images/cdrom/ or the daily build directly (without apt proxy) from https://d-i.debian.org/daily-images/%s/daily/cdrom/ However, I currently think that my issue is located within the live-wrapper package. lwr performs a download which isn't used. In lwr/utils.py the line check_url(base_url) should be removed. Then I wouldn't need a patched version of apt-cacher-ng at all... > Also, what's your deadline? I was planing to release a new version of > ACNG in a couple of days anyway. Do you also need it in backports? If you still would like to include the patch, I think that the content listing of a directory should always be considered volatile. But, as said, I think that the bug should be reassigned to the live-wrapper package. With kind regards, Roland Clobus signature.asc Description: OpenPGP digital signature
Re: Historical images
On 21/03/2020 18:01, PICCORO McKAY Lenz wrote: > this mail are very important.. but i cannot find a archive link to > distribute it! how can i get a link of this archive mail? https://lists.debian.org/debian-live/2020/03/msg00225.html https://lists.debian.org/debian-cd/2020/03/msg5.html > ALSO: >> From what I've seen in the live-setup repository [5], there are two >> strategies for generating Debian CD images. >> >> A) Use lb config (run-30live-build) >> B) Use lwr (run-30live-wrapper) >> > this only generates lasted today images? how to generates for debian 9 > and debian 8 live? and cd/dvd 1 set? You are perhaps looking for the archive? https://cdimage.debian.org/cdimage/archive/ With kind regards, Roland Clobus signature.asc Description: OpenPGP digital signature
Documenting the generation of the live images in Debian
Hello Steve McIntyre, Eduard Bloch and lists, On 11/07/2019 12:21, Steve McIntyre wrote on debian-live: > On Thu, Jul 11, 2019 at 03:13:48PM +0700, Azure Zanculmarktum wrote: >> Where can I find the source to build debian live? ... the live-build config. > > The setup we use is in git: > https://salsa.debian.org/images-team/live-setup/ ... > https://salsa.debian.org/images-team/live-setup/blob/master/available/run-30live-wrapper As part of my documentation effort for live-manual, I would like to include the procedure for the official Debian CD image generation (which will effectively replace chapter 16) in https://live-team.pages.debian.net/live-manual/html/live-manual/procedures.en.html This mail is rather technical, but I want to document the steps that I've taken. From what I've seen in the live-setup repository [5], there are two strategies for generating Debian CD images. A) Use lb config (run-30live-build) B) Use lwr (run-30live-wrapper) I downloaded 'debian-live-10.3.0-amd64-standard.iso' from [1], which contains the string 'Official Debian GNU/Linux Live 10.3.0 standard 2020-02-08T12:27'. This string is generated by the run-30live-wrapper script, so I assume that the official images are generated by method B, lwr. I've read through run-30live-wrapper, and I think by now I have correctly extracted the commands for the generation of the Debian Stable CDs, but I'm unsure how to proceed. I have the following questions/requests: * lwr depends on vmdebootstrap, which is currently only in Buster (stable). According to its author vmdebootstrap is to be replaced by vmdb2 or something else [2]. Is someone working on this? * I've seen a huge speed-up when I'm using a ramdisk, but that needs a merge request to be accepted [3]. When invoking 'chroot' in live-setup/available/live-customise.sh the variable TMPDIR must not be set, otherwise it is not possible to generate temporary files within the chroot. * To reduce the amount of downloads, I am using a slightly patched version of apt-cacher-ng, in order to be able to use the installer. See my bug report [4] The live-wrapper script downloads the installer, but apt-cacher-ng rejects a path ending in a slash. The following command simulates this download: wget -S "http://localhost:3142/deb.debian.org/debian/dists/buster/main/installer-amd64/current/images/cdrom/; * The run-30live-wrapper mentions wheezy and jessie, so it is rather old I would like to add to the live-manual the steps to build an image from the current stable and testing versions of Debian. Do you have hints/ideas how I can proceed? With kind regards, Roland Clobus --- Commands This is the list of commands that I use (all as root): mkdir -p /tmp/mytmp mount -t tmpfs tmpfs /tmp/mytmp export TMPDIR=/tmp/mytmp lwr -m http://localhost:3142/deb.debian.org/debian \ --apt-mirror http://deb.debian.org/debian \ --customise /home/roland/git/live-setup/available/live-customise.sh \ --architecture amd64 \ -d buster \ --isolinux \ --grub \ --log stderr \ --installer \ -t live-task-standard \ -f "" \ --base_debs "eject pciutils usbutils keyutils keyboard-configuration console-setup lvm2 mdadm dmsetup cryptsetup dmraid e2fsprogs btrfs-progs xfsprogs jfsutils grub-efi-amd64 grub-efi-amd64-bin grub-pc grub-efi-amd64-signed shim-signed" \ --description "Unofficial Debian GNU/Linux Live 10.3 standard" \ --volume_id "d-live 10.3 st amd64" \ --image_output debian-live-10.3.iso --- Links [1] https://www.debian.org/CD/live/ [2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=910201 [3] https://salsa.debian.org/images-team/live-setup/-/merge_requests/1 [4] https://bugs.debian.org/954437 [5] https://salsa.debian.org/images-team/live-setup signature.asc Description: OpenPGP digital signature
