Re: Starting the weekly live images for Bookworm building again

[Yashraj Moghe]

Apologies I forgot to link the post, my bad
https://salsa.debian.org/Disaster2life/bits/-/blob/master/content/2023/debian-bookworm-live-images.md

apologies,

Yashraj

On 4/6/23 16:59, Yashraj Moghe wrote:


Greetings Live team,

I have been working on a draft for an announcement regarding the live 
images being up and building again, I hope everyone can look over the 
draft currently and offer feedback if any, apologies on any errors in 
the post, feel free to correct via a pull request, if everything seems 
fine, I will then merge it into the main repository and publish it.


Thanks for your time,

Yashraj

Re: Starting the weekly live images for Bookworm building again

[Yashraj Moghe]

Greetings Live team,

I have been working on a draft for an announcement regarding the live 
images being up and building again, I hope everyone can look over the 
draft currently and offer feedback if any, apologies on any errors in 
the post, feel free to correct via a pull request, if everything seems 
fine, I will then merge it into the main repository and publish it.


Thanks for your time,

Yashraj

Fwd: Starting the weekly live images for Bookworm building again

[Luna Jernberg]

-- Forwarded message --
From: Yashraj Moghe 
Date: Mon, 3 Apr 2023 13:47:48 +0530
Subject: Re: Starting the weekly live images for Bookworm building again
To: debian-public...@lists.debian.org

Hello Everyone,

I have finished a draft of the blog for the announcement of the Live
images (|https://deb.li/publipad|), apologies for taking long, I
messaged in the IRC, and I am now putting this on the mailing list once,
I hope everyone can look over and offer words and pitch improvements,
hopefully by today I would be able to share it also with the debian-live
team, so we can publish this soon, I am unsure to what edit permissions
are available, but feel free to let me know regarding anything you would
like edited.

Regards,

Yashraj

Re: Starting the weekly live images for Bookworm building again

[Laura Arjona Reina]

Hello Debian Live and Debian Publicity teams

Note: not subscribed to debian-live, subscribed to debian-publicity

It's great news that the weekly live images for Bookworm are available 
again! Thanks everybody for the work (I know it's going on but this is a 
nice milestone).


I have no spoons to do it myself, but if anybody could write up a 
summary of the thread/work, with links for people to download and test, 
and pointers to the Debian Live team on how to help/submit feedback, 
that would be great.


If there is a volunteer to do it, please reply to this mail in 
-publicity mailing list, and go ahead: start a pad in storm.debian.net 
or directly commit a draft in 
https://salsa.debian.org/publicity-team/bits/-/tree/master/content/2023 
- I'm happy to review text, just not able right now to start from scratch.


If nobody can do this (let's say, no replies to this mail in one week), 
I can micronews the message 
https://lists.debian.org/debian-live/2023/03/msg00026.html next Friday, 
so at least our wide audience gets the news and the pointer to the 
debian-list mailing list to follow-up.


Kind regards,
--
Laura Arjona Reina
https://wiki.debian.org/LauraArjona

Re: Source of firmware packages (Was: Starting the weekly live images for Bookworm building again)

[Jonathan Carter]

Hi Roland

On 2023/03/20 19:10, Jonathan Carter wrote:

On 2023/03/20 18:36, Roland Clobus wrote:
FWIW, I've uploaded live-tasks-non-free-firmware that could be used 
to provide firmware packages for either desktops or servers. Control 
file here with more details: 
https://salsa.debian.org/live-team/live-tasks-non-free-firmware/-/blob/main/debian/control


I'm filing an unblock for it too, so that it can migrate to testing.


Should the live images use those 2 meta-packages instead of the 
heuristic?


I'm fine either way as long as the key packages makes its way on to the 
iso. Will install those metapackages on the latest iso and see what (if 
anything) extra gets installed.


Looks like your heuristic got it all! I don't think that this is at all 
bad, but I do think there might be an excessive amount of firmware by 
default now. For example, the dahdi firmware used by asterisk cards 
(probably unlikely most people will use them on Debian live media), 56k 
modem drivers, opsis firmware (although maybe video team might like that 
;) ), etc. I was also thinking that if you're doing a desktop install, 
you could do an 'apt remove live-task-non-free-firmware-server ; apt 
autoremove' to remove a bunch of firmware that is unlikely to ever be 
used on a desktop/laptop. Of course, I think in the future this 
ultimately needs to be handled better in the installer.


For now this /does/ work. But we do end up with 740M of uncompressed 
firmware on the image, so at the very least I think we should circle 
back to this for Debian 13.


BTW, are you planning to change the syslinux config? I notice it still 
shows the construction cap.


-Jonathan

Re: Source of firmware packages (Was: Starting the weekly live images for Bookworm building again)

[Jonathan Carter]

Hi Roland

On 2023/03/20 18:36, Roland Clobus wrote:
FWIW, I've uploaded live-tasks-non-free-firmware that could be used to 
provide firmware packages for either desktops or servers. Control file 
here with more details: 
https://salsa.debian.org/live-team/live-tasks-non-free-firmware/-/blob/main/debian/control


I'm filing an unblock for it too, so that it can migrate to testing.


Should the live images use those 2 meta-packages instead of the heuristic?


I'm fine either way as long as the key packages makes its way on to the 
iso. Will install those metapackages on the latest iso and see what (if 
anything) extra gets installed.


-Jonathan

Re: Source of firmware packages (Was: Starting the weekly live images for Bookworm building again)

[Roland Clobus]

Hello Jonathan,

On 20/03/2023 12:59, Jonathan Carter wrote:

On 2023/03/20 09:11, Roland Clobus wrote:
This weekend I've written the missing parts, non-free-firmware images 
are now generated by default by live-build, and the ISO images are 
still bit-for-bit reproducible.


How do you currently install the non-free firmware packages?


Depending on the availability of the section 'non-free-firmware', the 
package 'firmware-linux' pulls in the firmware-related packages.


Also the script [2] looks for packages that contain files in the folder 
'/lib/firmware' and pulls in all those packages as well.


FWIW, I've uploaded live-tasks-non-free-firmware that could be used to 
provide firmware packages for either desktops or servers. Control file 
here with more details: 
https://salsa.debian.org/live-team/live-tasks-non-free-firmware/-/blob/main/debian/control


I'm filing an unblock for it too, so that it can migrate to testing.


Should the live images use those 2 meta-packages instead of the heuristic?

With kind regards,
Roland

[1] 
https://salsa.debian.org/live-team/live-build/-/commit/50c7e1a8b7b5a966cce0bd432f0c5f02793330a9
[2] 
https://sources.debian.org/src/live-build/1%3A20230131/functions/firmwarelists.sh/


OpenPGP_signature
Description: OpenPGP digital signature

Re: Starting the weekly live images for Bookworm building again

[Jonathan Carter]

Hey Roland

On 2023/03/20 09:11, Roland Clobus wrote:
This weekend I've written the missing parts, non-free-firmware images 
are now generated by default by live-build, and the ISO images are still 
bit-for-bit reproducible.


How do you currently install the non-free firmware packages?

FWIW, I've uploaded live-tasks-non-free-firmware that could be used to 
provide firmware packages for either desktops or servers. Control file 
here with more details: 
https://salsa.debian.org/live-team/live-tasks-non-free-firmware/-/blob/main/debian/control


I'm filing an unblock for it too, so that it can migrate to testing.

-Jonathan

Re: Starting the weekly live images for Bookworm building again

[Holger Levsen]

On Sun, Mar 19, 2023 at 03:13:47PM +, Steve McIntyre wrote:
> So, after some delay from me and some further delays from various
> Debian machines committing suicide [1], I've got bookworm live builds
> running again. \o/

this is great news! thanks and kudos to everyone involved!


-- 
cheers,
Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁  holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ⠈⠳⣄

40% of homeless people in the United States have full-time jobs.


signature.asc
Description: PGP signature

Re: Starting the weekly live images for Bookworm building again

[Roland Clobus]

My last cross-post. Follow-up please on debian-live

Hello Steve and lists,

On 19/03/2023 16:13, Steve McIntyre wrote:

So, after some delay from me and some further delays from various
Debian machines committing suicide, I've got bookworm live builds
running again. \o/


Thanks for merging the changes.


I've taken Roland's updated patches and tweaked a little more in the
setup.git and live-setup.git repos, and we now have live builds
integrated. Changes I've added:

  * turned on source tarball generation using LB_SOURCE=true, and
disable the external source build that we did for the older
live-wrapper builds


That's a good way to enable the source tarballs. I haven't looked at 
that part of live-build for a while, so the source tarball might need 
some love.



  * when building on casulana, warn about archive updates rather than
restarting builds
  * don't attempt to build i386 live images any more, they're not useful
  * tweaked logging


And an additional change to use the installer images from the repository 
instead of rebuilding them from git [1].

That change is now causing the missing kernel modules for d-i.

I've rebuilding the installer images from git, after the discussion in 
#1006800 [2], to save the d-i team from doing an upload for every single 
kernel bump, while bookworm was not yet in freeze. (That is a recurring 
issue for every release [3])



So, *builds* work fine but I've not *yet* tested actually
booting/using one of these images in any way. I've just triggered a
full build of "testing" live images now, please help test if you can
once they're in place at [2] in a couple of hours from now.

> [2] https://get.debian.org/images/weekly-live-builds/

In openQA [4] the live images that were generated by Jenkins [5]
have been tested for a while now. Now we can switch and use these new 
images instead of the images from Jenkins. Since both images are 
generated by the same script, I wouldn't expect many issues.



I don't yet know how close we are to having full non-free-firmware
integration with the live images; I expect there might be some more
work needed there yet, but I'd love to be proven wrong. :-)


This weekend I've written the missing parts, non-free-firmware images 
are now generated by default by live-build, and the ISO images are still 
bit-for-bit reproducible.


With kind regards,
Roland Clobus

[1] 3cef309a5cfa4758ba33480b170734133b7104b5
[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1006800
[3] #986506, https://lists.debian.org/debian-boot/2021/03/msg00166.html
[4] https://openqa.debian.net/group_overview/14
[5] https://jenkins.debian.net/view/live/


OpenPGP_signature
Description: OpenPGP digital signature

Re: Starting the weekly live images for Bookworm building again

[Cyril Brulebois]

Hi,

Steve McIntyre  (2023-03-19):
> So, after some delay from me and some further delays from various
> Debian machines committing suicide [1], I've got bookworm live builds
> running again. \o/

Great news.

> I don't yet know how close we are to having full non-free-firmware
> integration with the live images; I expect there might be some more
> work needed there yet, but I'd love to be proven wrong. :-)

This shouldn't be a surprise as I've been focussing on debian-installer
topics rather than trying to also do debian-live stuff… but just for the
avoidance of doubt: While I have been behind most of non-free-firmware
related work[1], I don't plan on touching anything on the debian-live
side. I'm happy to try and answer any n-f-f related questions though.

 1. https://debamax.com/blog/2023/02/27/debian-versus-non-free-firmware/


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: Starting the weekly live images for Bookworm building again

[Steve McIntyre]

Hey again,

On Tue, Jan 31, 2023 at 03:36:53PM +, Steve McIntyre wrote:
>>
>>As you can see, this affects many teams:
>>* live-setup: a MR to generate all live images for Bookworm [A2]

So, after some delay from me and some further delays from various
Debian machines committing suicide [1], I've got bookworm live builds
running again. \o/

I've taken Roland's updated patches and tweaked a little more in the
setup.git and live-setup.git repos, and we now have live builds
integrated. Changes I've added:

 * turned on source tarball generation using LB_SOURCE=true, and
   disable the external source build that we dod for the older
   live-wrapper builds
 * when building on casulana, warn about archive updates rather than
   restarting builds
 * don't attempt to build i386 live images any more, they're not useful
 * tweaked logging

So, *builds* work fine but I've not *yet* tested actually
booting/using one of these images in any way. I've just triggered a
full build of "testing" live images now, please help test if you can
once they're in place at [2] in a couple of hours from now.

I don't yet know how close we are to having full non-free-firmware
integration with the live images; I expect there might be some more
work needed there yet, but I'd love to be proven wrong. :-)

[1] "yay" for the long-standing tradition of services failing as we
get close to a release: this time it was casulana and salsa...
[2] https://get.debian.org/images/weekly-live-builds/

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
"...In the UNIX world, people tend to interpret `non-technical user'
 as meaning someone who's only ever written one device driver." -- Daniel Pead

Re: Starting the weekly live images for Bookworm building again

[Cyril Brulebois]

Luna Jernberg  (2023-03-15):
> And now they have not built for 2 days any specific reason?

casulana broke; please refrain from hijacking threads and cross-posting
to so many lists. debian-cd would have done just fine…


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: Starting the weekly live images for Bookworm building again

[Luna Jernberg]

And now they have not built for 2 days any specific reason?

On 3/15/23, Holger Wansing  wrote:
> Hi,
>
> Steve McIntyre  wrote (Tue, 31 Jan 2023 15:36:53 +):
>> On Mon, Jan 16, 2023 at 05:35:50PM +0100, Roland Clobus wrote:
>> >* localechooser: A minor fix [A3]
>>
>> No idea about that, leaving for somebody else.
>
> A (small) change, that we want, I guess, to add Curaçao to Carribean
> region,
> where it belongs, and skip no-longer existing country 'Netherlands Antilles'
>
> from the list.
>
> Tested on a local build; works as it should.
> Thus merged.
>
>
> Holger
>
>
> --
> Holger Wansing 
> PGP-Fingerprint: 496A C6E8 1442 4B34 8508  3529 59F1 87CA 156E B076
>
>

Re: Starting the weekly live images for Bookworm building again

[Holger Wansing]

Hi,

Steve McIntyre  wrote (Tue, 31 Jan 2023 15:36:53 +):
> On Mon, Jan 16, 2023 at 05:35:50PM +0100, Roland Clobus wrote:
> >* localechooser: A minor fix [A3]
> 
> No idea about that, leaving for somebody else.

A (small) change, that we want, I guess, to add Curaçao to Carribean region,
where it belongs, and skip no-longer existing country 'Netherlands Antilles' 
from the list.

Tested on a local build; works as it should.
Thus merged.


Holger


-- 
Holger Wansing 
PGP-Fingerprint: 496A C6E8 1442 4B34 8508  3529 59F1 87CA 156E B076

Re: Starting the weekly live images for Bookworm building again

[Steve McIntyre]

Hey Roland,

Apologies for leaving you waiting a while :-/

On Mon, Jan 16, 2023 at 05:35:50PM +0100, Roland Clobus wrote:
>
>This is a follow-up of my mail from 2022-11-21 [A1].
>
>I've made progress in the last two months, but would like to have some merge
>requests approved, to get more traction and to allow others to jump aboard
>and make the live images for Bookworm possible.
>
>As you can see, this affects many teams:
>* live-setup: a MR to generate all live images for Bookworm [A2]

ACK, I'll take a look at this again shortly.

>* localechooser: A minor fix [A3]

No idea about that, leaving for somebody else.

>* live-installer: A better user experience after the installer is finished
>[A4]

Merred just now.

>* live-build: Various installer improvements, including off-line installation
>[A5]

Not sure who might review that, let's see

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
< Aardvark> I dislike C++ to start with. C++11 just seems to be
handing rope-creating factories for users to hang multiple
instances of themselves.

Re: Starting the weekly live images for Bookworm building again

[Roland Clobus]

Hello lists (sorry for cross-posting to so many lists),

This is a follow-up of my mail from 2022-11-21 [A1].

I've made progress in the last two months, but would like to have some 
merge requests approved, to get more traction and to allow others to 
jump aboard and make the live images for Bookworm possible.


As you can see, this affects many teams:
* live-setup: a MR to generate all live images for Bookworm [A2]
* localechooser: A minor fix [A3]
* live-installer: A better user experience after the installer is 
finished [A4]
* live-build: Various installer improvements, including off-line 
installation [A5]


With kind regards,
Roland Clobus

[A1] https://lists.debian.org/debian-devel/2022/11/msg00221.html
[A2] https://salsa.debian.org/images-team/live-setup/-/merge_requests/2
[A3] 
https://salsa.debian.org/installer-team/localechooser/-/merge_requests/7
[A4] 
https://salsa.debian.org/installer-team/live-installer/-/merge_requests/3

[A5] https://salsa.debian.org/live-team/live-build/-/merge_requests/297


OpenPGP_signature
Description: OpenPGP digital signature