Re: amd64 daily installer images not updated since 2026-07-05

2026-07-19 Thread Steve McIntyre 
On Sun, Jul 19, 2026 at 06:29:15PM +0100, Steve McIntyre wrote:
>On Sun, Jul 19, 2026 at 01:56:00AM +0200, Cyril Brulebois wrote:

...

>>Please do consider merging this branch. :-)
>>
>>Interestingly, now the failure hits another hunch of mine (see earlier
>>answers of mine):
>>
>>CD 1: GUESS_TOTAL is 377703 after adding 
>> amd64:non-free-firmware:firmware-qcom-dsp:110853068:
>>CD 1: Real current size is 377856 blocks after adding 
>> amd64:non-free-firmware:firmware-qcom-dsp:110853068:
>>CD 1 over-full (377856 > 332031). Rollback!
>>CD 1: Real current size is 323584 blocks after rolling back 
>> amd64:non-free-firmware:firmware-qcom-dsp:110853068:
>>No linux-image-* package(s) found
>>
>>Hopefully regular builds on casulana will succeed with either just the
>>branch or with both the branch and that firmware's exclusion (if doing
>>so indeed makes sense)…
>
>Testing now.

So, your change matches *exactly* the change I'd already come up
with. Great minds, or something. :-) That now lets us build an amd64
netinst, but armhf is still failing - there are more things we'll need
to strip out there.

-- 
Steve McIntyre, Cambridge, [email protected]
"I can't ever sleep on planes ... call it irrational if you like, but I'm
 afraid I'll miss my stop" -- Vivek Das Mohapatra

Re: amd64 daily installer images not updated since 2026-07-05

2026-07-19 Thread Steve McIntyre 
On Sun, Jul 19, 2026 at 01:56:00AM +0200, Cyril Brulebois wrote:
>[ Please ignore previous mail, I got confused by all the 7.1.3 removals,
>but that was just a side effect of looking at a before/after diff where
>all the sizes changed, which means one HUGE list of deletions (including
>those 7.1.3 which worried me), and way later, another HUGE list of
>additions. Let's get back to the one I had written waiting for the
>confirmation.

I had a similar issue in testing my (very similar) change - the
archive updated and dropped all the older ABI versions which really
confused me!

:-)

>Better living through chemistry / Better patching through coffee… ]

*grin*

>Cyril Brulebois  (2026-07-19):
>> That'd be tools/generate_di_list that works on -modules- (per code,
>> and as confirmed above) but not on -base-, -binary- (as seen above).
>> Looks to me replacing -modules- by -(?:base|binary|modules)- in both
>> regexes should help.
>> 
>> I hadn't realized the split across several packages also happened on
>> the udeb side (see other mails regarding my inability to keep track of
>> stuff). Not sure whether we might have other similar packages…
>
>Please check the kibi/extend-kernel-abi-filtering branch, a quick
>easy-build.sh run locally seems to confirm it helps but isn't sufficient
>(but you might remember this isn't 1:1 with the regular build so this is
>not a definitive answer regarding amd64 netinst buildability on
>casulana).
>
>At the very least, those lines go away between libzstd1-udeb and
>load-media:
>
>-CD 1: GUESS_TOTAL is 126362 after adding 
> amd64:main:linux-base-7.0.13+deb14-amd64-di:44800:
>-CD 1: GUESS_TOTAL is 126385 after adding 
> amd64:main:linux-base-7.0.14+deb14-amd64-di:44764:
>-CD 1: GUESS_TOTAL is 126408 after adding 
> amd64:main:linux-base-7.1.3+deb14-amd64-di:44060:
>-CD 1: GUESS_TOTAL is 133146 after adding 
> amd64:main:linux-binary-7.0.13+deb14-amd64-di:13792008:
>-CD 1: GUESS_TOTAL is 139860 after adding 
> amd64:main:linux-binary-7.0.14+deb14-amd64-di:13748212:
>-CD 1: GUESS_TOTAL is 146602 after adding 
> amd64:main:linux-binary-7.1.3+deb14-amd64-di:13803224:
>
>And make_disc_tree.log doesn't contain any other hit for the 7.0.13 or
>7.0.14 versions.

Cool.

>This is consistent with the end of tmp/sid/tasks/debian-installer:
>
>/* Next: 32 udebs for kernel/ABI version 7.1.3-0 */
>base-modules-7.1.3+deb14-amd64-di
>btrfs-modules-7.1.3+deb14-amd64-di
>crypto-dm-modules-7.1.3+deb14-amd64-di
>crypto-modules-7.1.3+deb14-amd64-di
>drm-core-modules-7.1.3+deb14-amd64-di
>ext4-modules-7.1.3+deb14-amd64-di
>f2fs-modules-7.1.3+deb14-amd64-di
>firewire-core-modules-7.1.3+deb14-amd64-di
>jfs-modules-7.1.3+deb14-amd64-di
>linux-base-7.1.3+deb14-amd64-di
>linux-binary-7.1.3+deb14-amd64-di
>loop-modules-7.1.3+deb14-amd64-di
>md-modules-7.1.3+deb14-amd64-di
>mmc-core-modules-7.1.3+deb14-amd64-di
>mtd-core-modules-7.1.3+deb14-amd64-di
>multipath-modules-7.1.3+deb14-amd64-di
>nbd-modules-7.1.3+deb14-amd64-di
>nic-modules-7.1.3+deb14-amd64-di
>nic-pcmcia-modules-7.1.3+deb14-amd64-di
>nic-shared-modules-7.1.3+deb14-amd64-di
>nic-usb-modules-7.1.3+deb14-amd64-di
>nic-wireless-modules-7.1.3+deb14-amd64-di
>pata-modules-7.1.3+deb14-amd64-di
>pcmcia-storage-modules-7.1.3+deb14-amd64-di
>ppp-modules-7.1.3+deb14-amd64-di
>rfkill-modules-7.1.3+deb14-amd64-di
>scsi-nic-modules-7.1.3+deb14-amd64-di
>sound-modules-7.1.3+deb14-amd64-di
>squashfs-modules-7.1.3+deb14-amd64-di
>udf-modules-7.1.3+deb14-amd64-di
>uinput-modules-7.1.3+deb14-amd64-di
>xfs-modules-7.1.3+deb14-amd64-di
>/* IGNORING 32 udebs for kernel/ABI version 7.0.13-0 */
>/* IGNORING 32 udebs for kernel/ABI version 7.0.14-0 */
>
>Please do consider merging this branch. :-)
>
>Interestingly, now the failure hits another hunch of mine (see earlier
>answers of mine):
>
>CD 1: GUESS_TOTAL is 377703 after adding 
> amd64:non-free-firmware:firmware-qcom-dsp:110853068:
>CD 1: Real current size is 377856 blocks after adding 
> amd64:non-free-firmware:firmware-qcom-dsp:110853068:
>CD 1 over-full (377856 > 332031). Rollback!
>CD 1: Real current size is 323584 blocks after rolling back 
> amd64:non-free-firmware:firmware-qcom-dsp:110853068:
>No linux-image-* package(s) found
>
>Hopefully regular builds on casulana will succeed with either just the
>branch or with both the branch and that firmware's exclusion (if doing
>so indeed makes sense)…

Testing now.

-- 
Steve McIntyre, Cambridge, [email protected]
"I suspect most samba developers are already technically insane... Of
 course, since many of them are Australians, you can't tell." -- Linus Torvalds

Re: amd64 daily installer images not updated since 2026-07-05

2026-07-19 Thread Steve McIntyre 
Hey Kibi!

On Sun, Jul 19, 2026 at 01:07:54AM +0200, Cyril Brulebois wrote:
>Steve McIntyre  (2026-07-18):
>> I can see that we're including bits of multiple kernels in the amd64
>> netinst at the moment, which is not going to help:
>> 
>> .../2sidamd64$ grep :linux forky/make_disc_tree.log 
>> CD 1: GUESS_TOTAL is 111963 after adding 
>> amd64:main:linux-sysctl-defaults:5852:
>> CD 1: GUESS_TOTAL is 154674 after adding 
>> amd64:main:linux-base-7.0.13+deb14-amd64-di:44800:
>> CD 1: GUESS_TOTAL is 154697 after adding 
>> amd64:main:linux-base-7.0.14+deb14-amd64-di:44764:
>> CD 1: GUESS_TOTAL is 154720 after adding 
>> amd64:main:linux-base-7.1.3+deb14-amd64-di:44060:
>> CD 1: GUESS_TOTAL is 161459 after adding 
>> amd64:main:linux-binary-7.0.13+deb14-amd64-di:13792008:
>> CD 1: GUESS_TOTAL is 168173 after adding 
>> amd64:main:linux-binary-7.0.14+deb14-amd64-di:13748212:
>> CD 1: GUESS_TOTAL is 174915 after adding 
>> amd64:main:linux-binary-7.1.3+deb14-amd64-di:13803224:
>> CD 1: GUESS_TOTAL is 217814 after adding 
>> amd64:main:linux-base:26312:
>> CD 1: GUESS_TOTAL is 433073 after adding 
>> amd64:main:linux-base-7.1.3+deb14-amd64:406916:
>> CD 1: GUESS_TOTAL is 433076 after adding 
>> amd64:main:linux-base-amd64:1204:
>> CD 1: GUESS_TOTAL is 439818 after adding 
>> amd64:main:linux-binary-7.1.3+deb14-amd64:13802032:
>> CD 1: GUESS_TOTAL is 492701 after adding 
>> amd64:main:linux-modules-7.1.3+deb14-amd64:108302028:
>> CD 1: Real current size is 492032 blocks after adding 
>> amd64:main:linux-modules-7.1.3+deb14-amd64:108302028:
>> 
>> which leads to:
>> 
>> CD 1 over-full (492032 > 488281). Rollback!
>> CD 1: Real current size is 439296 blocks after rolling back 
>> amd64:main:linux-modules-7.1.3+deb14-amd64:108302028:
>> No linux-image-* package(s) found
>> 
>> so this isn't going to be helping. We already have code in debian-cd
>> to limit the kernel version we're wanting to the latest, but that
>> doesn't seem to be working here.
>
>That'd be tools/generate_di_list that works on -modules- (per code,
>and as confirmed above) but not on -base-, -binary- (as seen above).
>Looks to me replacing -modules- by -(?:base|binary|modules)- in both
>regexes should help.

Nod, that's exactly what I was looking at (and hacking on last night
before I ran out of brain).

>I hadn't realized the split across several packages also happened on
>the udeb side (see other mails regarding my inability to keep track of
>stuff). Not sure whether we might have other similar packages…

ACK.

-- 
Steve McIntyre, Cambridge, [email protected]
"Every time you use Tcl, God kills a kitten." -- Malcolm Ray

Re: amd64 daily installer images not updated since 2026-07-05

2026-07-18 Thread Cyril Brulebois 
[ Please ignore previous mail, I got confused by all the 7.1.3 removals,
but that was just a side effect of looking at a before/after diff where
all the sizes changed, which means one HUGE list of deletions (including
those 7.1.3 which worried me), and way later, another HUGE list of
additions. Let's get back to the one I had written waiting for the
confirmation.

Better living through chemistry / Better patching through coffee… ]

Cyril Brulebois  (2026-07-19):
> That'd be tools/generate_di_list that works on -modules- (per code,
> and as confirmed above) but not on -base-, -binary- (as seen above).
> Looks to me replacing -modules- by -(?:base|binary|modules)- in both
> regexes should help.
> 
> I hadn't realized the split across several packages also happened on
> the udeb side (see other mails regarding my inability to keep track of
> stuff). Not sure whether we might have other similar packages…

Please check the kibi/extend-kernel-abi-filtering branch, a quick
easy-build.sh run locally seems to confirm it helps but isn't sufficient
(but you might remember this isn't 1:1 with the regular build so this is
not a definitive answer regarding amd64 netinst buildability on
casulana).

At the very least, those lines go away between libzstd1-udeb and
load-media:

-CD 1: GUESS_TOTAL is 126362 after adding 
amd64:main:linux-base-7.0.13+deb14-amd64-di:44800:
-CD 1: GUESS_TOTAL is 126385 after adding 
amd64:main:linux-base-7.0.14+deb14-amd64-di:44764:
-CD 1: GUESS_TOTAL is 126408 after adding 
amd64:main:linux-base-7.1.3+deb14-amd64-di:44060:
-CD 1: GUESS_TOTAL is 133146 after adding 
amd64:main:linux-binary-7.0.13+deb14-amd64-di:13792008:
-CD 1: GUESS_TOTAL is 139860 after adding 
amd64:main:linux-binary-7.0.14+deb14-amd64-di:13748212:
-CD 1: GUESS_TOTAL is 146602 after adding 
amd64:main:linux-binary-7.1.3+deb14-amd64-di:13803224:

And make_disc_tree.log doesn't contain any other hit for the 7.0.13 or
7.0.14 versions.

This is consistent with the end of tmp/sid/tasks/debian-installer:

/* Next: 32 udebs for kernel/ABI version 7.1.3-0 */
base-modules-7.1.3+deb14-amd64-di
btrfs-modules-7.1.3+deb14-amd64-di
crypto-dm-modules-7.1.3+deb14-amd64-di
crypto-modules-7.1.3+deb14-amd64-di
drm-core-modules-7.1.3+deb14-amd64-di
ext4-modules-7.1.3+deb14-amd64-di
f2fs-modules-7.1.3+deb14-amd64-di
firewire-core-modules-7.1.3+deb14-amd64-di
jfs-modules-7.1.3+deb14-amd64-di
linux-base-7.1.3+deb14-amd64-di
linux-binary-7.1.3+deb14-amd64-di
loop-modules-7.1.3+deb14-amd64-di
md-modules-7.1.3+deb14-amd64-di
mmc-core-modules-7.1.3+deb14-amd64-di
mtd-core-modules-7.1.3+deb14-amd64-di
multipath-modules-7.1.3+deb14-amd64-di
nbd-modules-7.1.3+deb14-amd64-di
nic-modules-7.1.3+deb14-amd64-di
nic-pcmcia-modules-7.1.3+deb14-amd64-di
nic-shared-modules-7.1.3+deb14-amd64-di
nic-usb-modules-7.1.3+deb14-amd64-di
nic-wireless-modules-7.1.3+deb14-amd64-di
pata-modules-7.1.3+deb14-amd64-di
pcmcia-storage-modules-7.1.3+deb14-amd64-di
ppp-modules-7.1.3+deb14-amd64-di
rfkill-modules-7.1.3+deb14-amd64-di
scsi-nic-modules-7.1.3+deb14-amd64-di
sound-modules-7.1.3+deb14-amd64-di
squashfs-modules-7.1.3+deb14-amd64-di
udf-modules-7.1.3+deb14-amd64-di
uinput-modules-7.1.3+deb14-amd64-di
xfs-modules-7.1.3+deb14-amd64-di
/* IGNORING 32 udebs for kernel/ABI version 7.0.13-0 */
/* IGNORING 32 udebs for kernel/ABI version 7.0.14-0 */

Please do consider merging this branch. :-)

Interestingly, now the failure hits another hunch of mine (see earlier
answers of mine):

CD 1: GUESS_TOTAL is 377703 after adding 
amd64:non-free-firmware:firmware-qcom-dsp:110853068:
CD 1: Real current size is 377856 blocks after adding 
amd64:non-free-firmware:firmware-qcom-dsp:110853068:
CD 1 over-full (377856 > 332031). Rollback!
CD 1: Real current size is 323584 blocks after rolling back 
amd64:non-free-firmware:firmware-qcom-dsp:110853068:
No linux-image-* package(s) found

Hopefully regular builds on casulana will succeed with either just the
branch or with both the branch and that firmware's exclusion (if doing
so indeed makes sense)…


Cheers,
-- 
Cyril Brulebois ([email protected])
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: amd64 daily installer images not updated since 2026-07-05

2026-07-18 Thread Cyril Brulebois 
Cyril Brulebois  (2026-07-19):
> That'd be tools/generate_di_list that works on -modules- (per code,
> and as confirmed above) but not on -base-, -binary- (as seen above).
> Looks to me replacing -modules- by -(?:base|binary|modules)- in both
> regexes should help.

That hunch doesn't work, we would get rid of everything related to the
latest ABI as well. Please don't merge kibi/extend-kernel-abi-filtering
blindly…


Cheers,
-- 
Cyril Brulebois ([email protected])
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: amd64 daily installer images not updated since 2026-07-05

2026-07-18 Thread Cyril Brulebois 
Hi,

Steve McIntyre  (2026-07-18):
> I can see that we're including bits of multiple kernels in the amd64
> netinst at the moment, which is not going to help:
> 
> .../2sidamd64$ grep :linux forky/make_disc_tree.log 
> CD 1: GUESS_TOTAL is 111963 after adding 
> amd64:main:linux-sysctl-defaults:5852:
> CD 1: GUESS_TOTAL is 154674 after adding 
> amd64:main:linux-base-7.0.13+deb14-amd64-di:44800:
> CD 1: GUESS_TOTAL is 154697 after adding 
> amd64:main:linux-base-7.0.14+deb14-amd64-di:44764:
> CD 1: GUESS_TOTAL is 154720 after adding 
> amd64:main:linux-base-7.1.3+deb14-amd64-di:44060:
> CD 1: GUESS_TOTAL is 161459 after adding 
> amd64:main:linux-binary-7.0.13+deb14-amd64-di:13792008:
> CD 1: GUESS_TOTAL is 168173 after adding 
> amd64:main:linux-binary-7.0.14+deb14-amd64-di:13748212:
> CD 1: GUESS_TOTAL is 174915 after adding 
> amd64:main:linux-binary-7.1.3+deb14-amd64-di:13803224:
> CD 1: GUESS_TOTAL is 217814 after adding amd64:main:linux-base:26312:
> CD 1: GUESS_TOTAL is 433073 after adding 
> amd64:main:linux-base-7.1.3+deb14-amd64:406916:
> CD 1: GUESS_TOTAL is 433076 after adding 
> amd64:main:linux-base-amd64:1204:
> CD 1: GUESS_TOTAL is 439818 after adding 
> amd64:main:linux-binary-7.1.3+deb14-amd64:13802032:
> CD 1: GUESS_TOTAL is 492701 after adding 
> amd64:main:linux-modules-7.1.3+deb14-amd64:108302028:
> CD 1: Real current size is 492032 blocks after adding 
> amd64:main:linux-modules-7.1.3+deb14-amd64:108302028:
> 
> which leads to:
> 
> CD 1 over-full (492032 > 488281). Rollback!
> CD 1: Real current size is 439296 blocks after rolling back 
> amd64:main:linux-modules-7.1.3+deb14-amd64:108302028:
> No linux-image-* package(s) found
> 
> so this isn't going to be helping. We already have code in debian-cd
> to limit the kernel version we're wanting to the latest, but that
> doesn't seem to be working here.

That'd be tools/generate_di_list that works on -modules- (per code,
and as confirmed above) but not on -base-, -binary- (as seen above).
Looks to me replacing -modules- by -(?:base|binary|modules)- in both
regexes should help.

I hadn't realized the split across several packages also happened on
the udeb side (see other mails regarding my inability to keep track of
stuff). Not sure whether we might have other similar packages…


Cheers,
-- 
Cyril Brulebois ([email protected])
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: amd64 daily installer images not updated since 2026-07-05

2026-07-18 Thread Steve McIntyre 
Sorry, I've been busy travelling and dealing with other stuff in the
last few days. :-/

So, checking...

I can see that we're including bits of multiple kernels in the amd64
netinst at the moment, which is not going to help:

.../2sidamd64$ grep :linux forky/make_disc_tree.log 
CD 1: GUESS_TOTAL is 111963 after adding 
amd64:main:linux-sysctl-defaults:5852:
CD 1: GUESS_TOTAL is 154674 after adding 
amd64:main:linux-base-7.0.13+deb14-amd64-di:44800:
CD 1: GUESS_TOTAL is 154697 after adding 
amd64:main:linux-base-7.0.14+deb14-amd64-di:44764:
CD 1: GUESS_TOTAL is 154720 after adding 
amd64:main:linux-base-7.1.3+deb14-amd64-di:44060:
CD 1: GUESS_TOTAL is 161459 after adding 
amd64:main:linux-binary-7.0.13+deb14-amd64-di:13792008:
CD 1: GUESS_TOTAL is 168173 after adding 
amd64:main:linux-binary-7.0.14+deb14-amd64-di:13748212:
CD 1: GUESS_TOTAL is 174915 after adding 
amd64:main:linux-binary-7.1.3+deb14-amd64-di:13803224:
CD 1: GUESS_TOTAL is 217814 after adding amd64:main:linux-base:26312:
CD 1: GUESS_TOTAL is 433073 after adding 
amd64:main:linux-base-7.1.3+deb14-amd64:406916:
CD 1: GUESS_TOTAL is 433076 after adding 
amd64:main:linux-base-amd64:1204:
CD 1: GUESS_TOTAL is 439818 after adding 
amd64:main:linux-binary-7.1.3+deb14-amd64:13802032:
CD 1: GUESS_TOTAL is 492701 after adding 
amd64:main:linux-modules-7.1.3+deb14-amd64:108302028:
CD 1: Real current size is 492032 blocks after adding 
amd64:main:linux-modules-7.1.3+deb14-amd64:108302028:

which leads to:

CD 1 over-full (492032 > 488281). Rollback!
CD 1: Real current size is 439296 blocks after rolling back 
amd64:main:linux-modules-7.1.3+deb14-amd64:108302028:
No linux-image-* package(s) found

so this isn't going to be helping. We already have code in debian-cd
to limit the kernel version we're wanting to the latest, but that
doesn't seem to be working here. The same pattern is showing up on
other arches too, e.g. on arm64:

.../2sidarm64$ grep :linux forky/make_disc_tree.log 
CD 1: GUESS_TOTAL is 89457 after adding 
arm64:main:linux-sysctl-defaults:5852:
CD 1: GUESS_TOTAL is 131849 after adding 
arm64:main:linux-base-7.0.13+deb14-arm64-di:2937232:
CD 1: GUESS_TOTAL is 133285 after adding 
arm64:main:linux-base-7.0.14+deb14-arm64-di:2937792:
CD 1: GUESS_TOTAL is 134876 after adding 
arm64:main:linux-base-7.1.3+deb14-arm64-di:3253452:
CD 1: GUESS_TOTAL is 140440 after adding 
arm64:main:linux-binary-7.0.13+deb14-arm64-di:11388528:
CD 1: GUESS_TOTAL is 145996 after adding 
arm64:main:linux-binary-7.0.14+deb14-arm64-di:11376132:
CD 1: GUESS_TOTAL is 151679 after adding 
arm64:main:linux-binary-7.1.3+deb14-arm64-di:11633408:
CD 1: GUESS_TOTAL is 197529 after adding arm64:main:linux-base:26312:
CD 1: GUESS_TOTAL is 419674 after adding 
arm64:main:linux-base-7.1.3+deb14-arm64:3809288:
CD 1: GUESS_TOTAL is 419676 after adding 
arm64:main:linux-base-arm64:1224:
CD 1: GUESS_TOTAL is 425358 after adding 
arm64:main:linux-binary-7.1.3+deb14-arm64:11628868:
CD 1: GUESS_TOTAL is 470580 after adding 
arm64:main:linux-modules-7.1.3+deb14-arm64:92612356:
CD 1: Real current size is 469654 blocks after adding 
arm64:main:linux-modules-7.1.3+deb14-arm64:92612356:
CD 1: GUESS_TOTAL is 469977 after adding 
arm64:main:linux-image-7.1.3+deb14-arm64:1884:
CD 1: GUESS_TOTAL is 469980 after adding 
arm64:main:linux-image-arm64:1652:

but amd64 is just that little bit larger so it's failing while arm64
is not.

I'm digging further.

-- 
Steve McIntyre, Cambridge, [email protected]
< liw> everything I know about UK hotels I learned from "Fawlty Towers"

Re: amd64 daily installer images not updated since 2026-07-05

2026-07-17 Thread Thomas Schmitt 
Hi,

Cyril Brulebois wrote:
>  - [x] Trick myself into actually looking instead of just giving a few
>examples.

Roughly comparing 13.5.0-amd64-netinst to testing-amd64-netinst of
2026-06-29 by executing

  old_iso=/dvdbuffer/debian-13.5.0-amd64-netinst.iso
  new_iso=/dvdbuffer/debian-testing-amd64-netinst.iso
  old_du=/tmp/old_du
  new_du=/tmp/new_du

  xorriso -indev "$old_iso" -dus "/*" "/pool/*" "/pool/main/*" -- >"$old_du"
  xorriso -indev "$new_iso" -dus "/*" "/pool/*" "/pool/main/*" -- >"$new_du"

and viewing the resulting /tmp/*_du files side-by-side, i see inflation
mostly at firmware and somewhat with /install.amd.
 Old:New:
 167592 '/firmware'  361697 '/firmware'
 198849 '/install.amd'   219552 '/install.amd'

/pool does not look promising for size reduction. All files in
/pool/non-free-firmware share their content with files in /firmware.
So the size increase of /pool
 584572 '/pool'  768112 '/pool'
is only virtual.
/pool/main even shrunk:
 417892 '/pool/main' 407515 '/pool/main'

The ten fattest firmware packages in testing-amd64-netinst are
 103221 'firmware-qcom-dsp_0~20260519-1_all.deb'
 101724 'firmware-nvidia-graphics_20260519-1_all.deb'
  37053 'firmware-atheros_20260519-1_all.deb'
  25307 'firmware-mediatek_20260519-1_all.deb'
  23169 'firmware-iwlwifi_20260519-1_all.deb'
  15177 'firmware-amd-graphics_20260519-1_all.deb'
  13289 'intel-microcode_3.20260227.1_amd64.deb'
   7791 'firmware-intel-graphics_20260519-1_all.deb'
   5260 'firmware-brcm80211_20260519-1_all.deb'
   4986 'firmware-misc-nonfree_20260519-1_all.deb'
summing up to 336977 of 361697 KiB.


>  - [ ] Bump the size from 1G to 1.1G or even 1.2G, effectively punting
>the investigation to “later”. This is likely easier and quicker
>than what I mentioned, but I can't help but worry this could be a
>one-way street, and I'd rather try and avoid bumping the limit if
>some reasonable amount of work lets us stay below the existing
>limit.

I don't see much chance to curb the growth of firmware blobs.
Even if a big one is thrown out, the others will eat up the free space
quickly.

Consider to expand generously without giving up the goal to make netinst
as small as possible.
Some data points from my desk:
My smallest USB stick from around 2005 offers 1.9 GiB.
The next larger one has ~ 4 GiB.
The size of a DVD is 2295104 * 2048 = 4,700,372,992 bytes.
(DVD-RAM are smaller. Their size depends on formatting.)


Have a nice day :)

Thomas

Re: amd64 daily installer images not updated since 2026-07-05

2026-07-17 Thread Cyril Brulebois 
Hi,

Roland Clobus  (2026-07-17):
> It's a good thing we have such safety mechanism. I've now reverted the
> change, so there will be no new images until the core problem (the
> size of the netinst-iso) is solved.

Thanks for doing so.

> I saw these files as well (I looked at 2sidamd64) and the 'No
> linux-image-*' line triggered me. I've seen some other changes in the
> git repos in the past that replaced 'linux-image' with 'linux-binary',
> so I (wrongly) concluded that it might be needed here as well. I
> missed the 'over-full, Rollback!' line.

Just for the avoidance of doubt, looking at all our repositories, I'm
only finding one occurrence for such a switch, and that one is indeed
correct:

  
https://salsa.debian.org/installer-team/debian-installer/-/commit/105ceb2ca1b8abc874dad53e2be1dcf456e8c0dd

Backstory: we stayed with the kernel-image name on the udeb side because
the kernel-image → linux-image rename only happened on the deb side (a
long time ago), where the kernel-image udeb shipped the actual kernel
binary, plus some modules depending on the architecture (mostly arm* a
while ago but even amd64 got some more in trixie — see
debian/installer/modules/*/kernel-image in the linux.git repository),
as opposed to the linux-image deb which used to ship the kernel binary
plus many modules (which is now getting shipped into different
packages).

In the “building the installer” context, pulling the linux-image udeb is
the right thing to do (provided the required modules are still shipped
alongside the actual kernel binary).

> Which turned out to be incorrect. I've reverted the local change in
> casulana (from
> https://salsa.debian.org/images-team/debian-cd/-/merge_requests/59)

Thanks. I'm wondering whether it would make sense to also unpublish the
few images built yesterday/today as well. People wondering about the
missing images might search other directories and find/download/test
broken images?

> So the answer to my own question is: 'No, it needs a different
> solution to make everything important fit again'.

In a nutshell: Yes, please. :-)

(I decided to send the previous mail as soon as I felt like it had
enough details to give you a clear picture, but I wasn't sure if and
when I would have more time to share more hints. Devil, details, …
is the reason why I didn't write that particular answer. See the
alternative plan below.)

> It turns out that the current limit of 1GB (10^9 bytes in 488281
> blocks of 2048 bytes) is the reason for the missing amd64 netinst
> image, not anything related to the kernel.

I'm not going to hunt for numbers here, but it could be that one of the
recent version bump pushed us over the edge. It also happens that one or
a few other packages needed for debootstrap get built with debug info,
or “stuff” happens that make them really bigger all of a sudden. It
could be that we end up pulling several gcc or llvm runtimes at the same
time, which makes the footprint larger (both on the installation images
and in the installed systems), and/or new/bigger firmware packages.

My preferred action, if I had time, would be to try and understand
whether there's an “obvious” package/set of packages that's responsible
for hitting the limit, or whether it's just some kind of slow/steady
growth that ended up triggering the overflow.

Here are a few examples from the past (debian-cd.git):
 - 18b865eca709bcc21880d7db4d3fae8693575f37
 - 213c680c07fee552a28b8fef5bdae166b2eb1ff2

Looking at one of the logs mentioned earlier, you can spot all .deb
files being considered but also some getting ignored. Seeing
firmware-qcom-soc ignored but firmware-qcom-dsp included kind of
triggers some itching sensation, and indeed, that's not a small package:

106Mfirmware-qcom-dsp_0~20260622-1_all.deb

This is a package only found in testing/unstable, introduced early 2026.
The following changelog entry reinforces my initial impression: if it's
OK to ignore firmware-qcom-soc (from src:firmware-nonfree), it might be
reasonable to also ignore firmware-qcom-dsp as well…

   [ Robie Basak ]
   * Adjust versioned Breaks/Replaces from firmware-qcom-dsp to
 firmware-qcom-soc now that we know the exact version the files were dropped
 from the src:firmware-nonfree side.
   * Upload to unstable since there's no longer any benefit to holding this
 package out of unstable.

Source: 
https://tracker.debian.org/news/1718101/accepted-hexagon-dsp-binaries-020251021-1-source-all-into-unstable/

Possible plan:

 - [x] Trick myself into actually looking instead of just giving a few
   examples.
 - [ ] Exclude this package as well and see if it builds again, putting
   the maintainers in the loop at some point for input/sanity check.
 - [ ] Actually do a survey of all firmware getting included now,
   comparing to what's available/included/excluded in trixie, and
   investigate new packages to see if (1) they actually take
   significant space and (2) they are or might be

Re: amd64 daily installer images not updated since 2026-07-05

2026-07-17 Thread Roland Clobus 

Hello Cyril, lists,

On 17/07/2026 12:14, Cyril Brulebois wrote:

Roland Clobus  (2026-07-17):



I'm wondering at 2 things:
* Why is only amd64 affected, whereas the other architectures are still 
building?


Because things are getting too big, on this architecture, and because
we're making extra sure we're not missing something as critical as the
kernel, ensuring we don't build and publish clearly-broken images.


It's a good thing we have such safety mechanism. I've now reverted the change, 
so there will be no new images until the core problem (the size of the 
netinst-iso) is solved.


Let's look at the first one from 2026-07-13 (Fsidamd64):

 kibi@casulana:/home/debian-cd/build/log$ tail -5 Fsidamd64
   Checking base is installable for amd64
   1 files missing for debootstrap and kernel, aborting!
 amd64 is missing 1 files needed for debootstrap, look in 
/srv/cdbuilder.debian.org/src/deb-cd/tmp/Fsidamd64/forky/make_disc_tree.log for 
the list
 make[1]: *** [Makefile:483: image-trees] Error 2
 make[1]: Leaving directory 
'/srv/cdbuilder.debian.org/git/setup/forky/debian-cd'

Deeper down the rabbit hole:

 kibi@casulana:/home/debian-cd/build/log$ tail 
/srv/cdbuilder.debian.org/src/deb-cd/tmp/Fsidamd64/forky/make_disc_tree.log
 CD 1: GUESS_TOTAL is 429722 after adding 
amd64:main:systemd-boot:82200:
 CD 1: GUESS_TOTAL is 429793 after adding 
amd64:main:systemd-boot-efi:135492:
 CD 1: GUESS_TOTAL is 429993 after adding 
amd64:main:linux-base-7.1.3+deb14-amd64:406916:
 CD 1: GUESS_TOTAL is 429995 after adding 
amd64:main:linux-base-amd64:1204:
 CD 1: GUESS_TOTAL is 436737 after adding 
amd64:main:linux-binary-7.1.3+deb14-amd64:13802032:
 CD 1: GUESS_TOTAL is 489620 after adding 
amd64:main:linux-modules-7.1.3+deb14-amd64:108302028:
 CD 1: Real current size is 489984 blocks after adding 
amd64:main:linux-modules-7.1.3+deb14-amd64:108302028:
 CD 1 over-full (489984 > 488281). Rollback!
 CD 1: Real current size is 436736 blocks after rolling back 
amd64:main:linux-modules-7.1.3+deb14-amd64:108302028:
 No linux-image-* package(s) found


I saw these files as well (I looked at 2sidamd64) and the 'No linux-image-*' 
line triggered me. I've seen some other changes in the git repos in the past 
that replaced 'linux-image' with 'linux-binary', so I (wrongly) concluded that 
it might be needed here as well. I missed the 'over-full, Rollback!' line.


Your MR effectively turns the “I tried to fit linux-image in, adding its
dependencies one after another, until there wasn't enough space, so we
only have some of them” situation into an acceptable one.


Which turned out to be incorrect. I've reverted the local change in casulana 
(from https://salsa.debian.org/images-team/debian-cd/-/merge_requests/59)


* Is the switch from 'linux-image-*' to 'linux-binary-*' the correct way to 
proceed?


I don't recall seeing requests/suggestions to move away from installing
linux-image- (this is a simplification, there are variations). It
doesn't look like d-i should focus on installing the binary instead of
pulling the image, as we wouldn't have any modules…


So the answer to my own question is: 'No, it needs a different solution to make 
everything important fit again'.
It turns out that the current limit of 1GB (10^9 bytes in 488281 blocks of 2048 
bytes) is the reason for the missing amd64 netinst image, not anything related 
to the kernel.

With kind regards,
Roland


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: amd64 daily installer images not updated since 2026-07-05

2026-07-17 Thread Cyril Brulebois 
Hi Roland,

Thanks for looking into this.

Roland Clobus  (2026-07-17):
> +CC: debian-boot
> 
> Hi,
> 
> On 13/07/2026 00:01, Besmir Hasanaj wrote:
> > The amd64 directory under the daily installer builds appears not to
> > have been updated since July 5, while the other architecture
> > directories are updating normally.

I don't think any code changes are needed in the first place,
linux-image-* packages are still there, including for amd64.

For example:

Package: linux-image-amd64
Source: linux-signed-amd64 (7.1.3+1)
Provides: linux-image-generic, linux-latest-modules-7.1.3+deb14-amd64, 
virtualbox-guest-modules, wireguard-modules (= 1.0.0)
Depends: linux-base-amd64 (= 7.1.3-1), linux-image-7.1.3+deb14-amd64 (= 
7.1.3-1)

FWIW I knew linux and linux-signed-* were in the process of getting
updated last week so I decided not to jump the gun at the time:
  https://buildd.debian.org/status/package.php?p=linux&suite=sid
  https://buildd.debian.org/status/package.php?p=linux-signed-amd64&suite=sid

> I'm wondering at 2 things:
> * Why is only amd64 affected, whereas the other architectures are still 
> building?

Because things are getting too big, on this architecture, and because
we're making extra sure we're not missing something as critical as the
kernel, ensuring we don't build and publish clearly-broken images.

Let's start with:

kibi@casulana:/home/debian-cd/build/log$ ls *sid*amd64* -ltr|grep 'Jul 1'
-rw-r--r-- 1 debian-cd debian-cd  38654 Jul 13 03:38 Fsidamd64
-rw-r--r-- 1 debian-cd debian-cd 108922 Jul 13 03:58 Msidamd64
-rw-r--r-- 1 debian-cd debian-cd 113747 Jul 13 03:59 Ksidamd64
-rw-r--r-- 1 debian-cd debian-cd 385272 Jul 13 05:57 Dsidamd64
-rw-r--r-- 1 debian-cd debian-cd 395446 Jul 13 06:11 9sidamd64
-rw-r--r-- 1 debian-cd debian-cd 450269 Jul 13 06:30 3sidamd64
-rw-r--r-- 1 debian-cd debian-cd  56640 Jul 17 09:02 2sidamd64
-rw-r--r-- 1 debian-cd debian-cd  75701 Jul 17 09:03 Gsidamd64

Let's look at the first one from 2026-07-13 (Fsidamd64):

kibi@casulana:/home/debian-cd/build/log$ tail -5 Fsidamd64
  Checking base is installable for amd64
  1 files missing for debootstrap and kernel, aborting!
amd64 is missing 1 files needed for debootstrap, look in 
/srv/cdbuilder.debian.org/src/deb-cd/tmp/Fsidamd64/forky/make_disc_tree.log for 
the list
make[1]: *** [Makefile:483: image-trees] Error 2
make[1]: Leaving directory 
'/srv/cdbuilder.debian.org/git/setup/forky/debian-cd'

Deeper down the rabbit hole:

kibi@casulana:/home/debian-cd/build/log$ tail 
/srv/cdbuilder.debian.org/src/deb-cd/tmp/Fsidamd64/forky/make_disc_tree.log
CD 1: GUESS_TOTAL is 429722 after adding 
amd64:main:systemd-boot:82200:
CD 1: GUESS_TOTAL is 429793 after adding 
amd64:main:systemd-boot-efi:135492:
CD 1: GUESS_TOTAL is 429993 after adding 
amd64:main:linux-base-7.1.3+deb14-amd64:406916:
CD 1: GUESS_TOTAL is 429995 after adding 
amd64:main:linux-base-amd64:1204:
CD 1: GUESS_TOTAL is 436737 after adding 
amd64:main:linux-binary-7.1.3+deb14-amd64:13802032:
CD 1: GUESS_TOTAL is 489620 after adding 
amd64:main:linux-modules-7.1.3+deb14-amd64:108302028:
CD 1: Real current size is 489984 blocks after adding 
amd64:main:linux-modules-7.1.3+deb14-amd64:108302028:
CD 1 over-full (489984 > 488281). Rollback!
CD 1: Real current size is 436736 blocks after rolling back 
amd64:main:linux-modules-7.1.3+deb14-amd64:108302028:
No linux-image-* package(s) found

Your MR effectively turns the “I tried to fit linux-image in, adding its
dependencies one after another, until there wasn't enough space, so we
only have some of them” situation into an acceptable one.

> * Is the switch from 'linux-image-*' to 'linux-binary-*' the correct way to 
> proceed?

I don't recall seeing requests/suggestions to move away from installing
linux-image- (this is a simplification, there are variations). It
doesn't look like d-i should focus on installing the binary instead of
pulling the image, as we wouldn't have any modules…

(unstable-amd64-devel)kibi@tokyo:~$ apt-file list 
linux-binary-7.1.3+deb14-amd64
linux-binary-7.1.3+deb14-amd64: /boot/vmlinuz-7.1.3+deb14-amd64
linux-binary-7.1.3+deb14-amd64: 
/usr/share/bug/linux-binary-7.1.3+deb14-amd64
linux-binary-7.1.3+deb14-amd64: 
/usr/share/doc/linux-binary-7.1.3+deb14-amd64
linux-binary-7.1.3+deb14-amd64: 
/usr/share/lintian/overrides/linux-binary-7.1.3+deb14-amd64

The only dependency of that package is linux-base-7.1.3+deb14-amd64,
which doesn't pull anything else, and installs a limited number of
support files:

(unstable-amd64-devel)kibi@tokyo:~$ apt-file show 
linux-base-7.1.3+deb14-amd64
linux-base-7.1.3+deb14-amd64: /boot/System.map-7.1.3+deb14-amd64
linux-base-7.1.3+deb14-amd64: /boot/config-7.1.3+deb14-amd64
linux-base-7.1.3+deb14-amd64: 
/usr/lib/modules/7.1.3+deb14-amd64/modules.builtin
linux-base-7.1.3+deb14-amd64:

Re: amd64 daily installer images not updated since 2026-07-05

2026-07-17 Thread Roland Clobus 

+CC: debian-boot

Hi,

On 13/07/2026 00:01, Besmir Hasanaj wrote:

The amd64 directory under the daily installer builds appears not to have been 
updated since July 5, while the other architecture directories are updating 
normally.


I've looked at the issue.
The log files mentioned that the kernel package with the naming 'linux-image-*' 
was not found. I prepared a MR [1] to make it accept also the 'linux-binary-*' 
package name, but today I found out that the base-installer module would need 
to adapt as well [2], see the openQA tests [3].

I'm wondering at 2 things:
* Why is only amd64 affected, whereas the other architectures are still 
building?
* Is the switch from 'linux-image-*' to 'linux-binary-*' the correct way to 
proceed?

With kind regards,
Roland Clobus

[1] https://salsa.debian.org/images-team/debian-cd/-/merge_requests/59
[2] https://sources.debian.org/src/base-installer/1.229/library.sh?hl=348#L348
[3] https://openqa.debian.net/group_overview/10


OpenPGP_signature.asc
Description: OpenPGP digital signature