[Kernel-packages] [Bug 1942260] Re: compress firmware in /lib/firmware
Have you tried environment variables?
From the xz manpage:
ENVIRONMENT
xz parses space-separated lists of options from the environment vari‐
ables XZ_DEFAULTS and XZ_OPT, in this order, before parsing the options
from the command line. Note that only options are parsed from the en‐
vironment variables; all non-options are silently ignored. Parsing is
done with getopt_long(3) which is used also for the command line argu‐
ments.
XZ_DEFAULTS
User-specific or system-wide default options. Typically this is
set in a shell initialization script to enable xz's memory usage
limiter by default. Excluding shell initialization scripts and
similar special cases, scripts must never set or unset XZ_DE‐
FAULTS.
XZ_OPT
This is for passing options to xz when it is not possible to set
the options directly on the xz command line. This is the case
when xz is run by a script or tool, for example, GNU tar(1):
XZ_OPT=-2v tar caf foo.tar.xz foo
Scripts may use XZ_OPT, for example, to set script-specific de‐
fault compression options. It is still recommended to allow
users to override XZ_OPT if that is reasonable. For example, in
sh(1) scripts one may use something like this:
XZ_OPT=${XZ_OPT-"-7e"}
export XZ_OPT
From the zstd manpage:
Environment Variables
Employing environment variables to set parameters has security implica‐
tions. Therefore, this avenue is intentionally limited. Only
ZSTD_CLEVEL and ZSTD_NBTHREADS are currently supported. They set the
compression level and number of threads to use during compression, re‐
spectively.
ZSTD_CLEVEL can be used to set the level between 1 and 19 (the "normal"
range). If the value of ZSTD_CLEVEL is not a valid integer, it will be
ignored with a warning message. ZSTD_CLEVEL just replaces the default
compression level (3).
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux-firmware in Ubuntu.
https://bugs.launchpad.net/bugs/1942260
Title:
compress firmware in /lib/firmware
Status in initramfs-tools package in Ubuntu:
Fix Released
Status in linux-firmware package in Ubuntu:
Confirmed
Status in linux-firmware-raspi2 package in Ubuntu:
Confirmed
Bug description:
-- initramfs-tools
[Impact]
* linux supports xz compressed linux-firmware which saves disk space.
In focal, initramfs-tools only knows how to included uncompressed
firmware files (even when kernel supports loading compressed ones).
Newer releases of linux-firmware may use compressed firmware files
only, in such cases it would be nice for focal's initramfs-tools to
support compressed firmware files in case of partial or incomplete
upgrades (i.e. linux-firmware force installed or upgraded, without
newer initramfs-tools). The proposed changes to initramfs-tools are
backwards and forwards compatible, they prefer to include uncompressed
firmware files; and if missing, include compressed firmware files in
their uncompressed form. Thus maintaining compatibility with any
kernels, irrespective of compressed/uncompressed firmware inputs.
[Test Plan]
* Compress all files shipped by linux-firmware with xz
* Rebuild initrd
* Check that all the same firmware files are still included in the
initramfs, in their uncompressed form as before
[Where problems could occur]
* This SRU is precautionary to prevent accidental installation of
compressed linux-firmware from generating incorrect initramfs. It
should be noted that whilst initramfs-tools would create a compatible
initramfs with any kernels, pre-v5.3 kernels do not support xz
compressed firmware files at runtime. Mixing this new initramfs with
compressed firmwares and pre 5.3 kernels may lead to expectations of
supporting compressed firmware files with them only working at initrd
stage and not at runtime.
[Other Info]
Original bug report
Some facts:
- The linux kernel has supported loading xz compressed firmware since 5.3
- The size of /lib/firmware in impish is ~650Mb (and growing)
- The compressed size of firmware could be ~230Mb
It would be nice to install compressed firmware to save space.
Here are the plans from the Fedora project:
https://fedoraproject.org/wiki/Changes/CompressKernelFirmware
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/initramfs-tools/+bug/1942260/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHel
[Kernel-packages] [Bug 1990964] Re: FTBFS on kinetic
While kinetic is still supported for a few more days, I think it is obvious now that we won't release a fix for this for kinetic. I'm therefore going to mark this as won't fix for kinetic. ** Changed in: strace (Ubuntu Kinetic) Status: New => Won't Fix ** Changed in: linux (Ubuntu Kinetic) Status: New => Fix Released -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1990964 Title: FTBFS on kinetic Status in linux package in Ubuntu: Fix Released Status in strace package in Ubuntu: Fix Released Status in linux source package in Kinetic: Fix Released Status in strace source package in Kinetic: Won't Fix Bug description: As can be seen in [1], strace FTBFS in kinetic: this is caused by the kernel headers (linux-libc-dev) which do not define F_GETLK64 and other on 64b builds because the kernel contains a bogus commit (306f7cc1e906 "uapi: always define F_GETLK64/F_SETLK64/F_SETLKW64 in fcntl.h"). This commit actually did the opposite of what it was supposed to do, namely defining those macros even on 64b builds. There is an attempt here to fix this which was not merged yet: https://lore.kernel.org/lkml/cajf2gtqtnmoeb62-63ou8y4dbrdym7iztdtfluxx9u0ltwu...@mail.gmail.com/T/ [1]: https://launchpadlibrarian.net/625441996/buildlog_ubuntu-kinetic- amd64.strace_5.16-0ubuntu4_BUILDING.txt.gz To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1990964/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1990964] Re: FTBFS on kinetic
I was too quick to mark it Fix Released in Kinetic. The issue is probably still there. It seems pretty unlikely it's fixed for Kinetic though. ** Changed in: strace (Ubuntu Kinetic) Status: Fix Released => New -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1990964 Title: FTBFS on kinetic Status in linux package in Ubuntu: Fix Released Status in strace package in Ubuntu: Fix Released Status in linux source package in Kinetic: New Status in strace source package in Kinetic: New Bug description: As can be seen in [1], strace FTBFS in kinetic: this is caused by the kernel headers (linux-libc-dev) which do not define F_GETLK64 and other on 64b builds because the kernel contains a bogus commit (306f7cc1e906 "uapi: always define F_GETLK64/F_SETLK64/F_SETLKW64 in fcntl.h"). This commit actually did the opposite of what it was supposed to do, namely defining those macros even on 64b builds. There is an attempt here to fix this which was not merged yet: https://lore.kernel.org/lkml/cajf2gtqtnmoeb62-63ou8y4dbrdym7iztdtfluxx9u0ltwu...@mail.gmail.com/T/ [1]: https://launchpadlibrarian.net/625441996/buildlog_ubuntu-kinetic- amd64.strace_5.16-0ubuntu4_BUILDING.txt.gz To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1990964/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1990964] Re: FTBFS on kinetic
** Changed in: strace (Ubuntu Kinetic) Status: New => Fix Released -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1990964 Title: FTBFS on kinetic Status in linux package in Ubuntu: Fix Released Status in strace package in Ubuntu: Fix Released Status in linux source package in Kinetic: New Status in strace source package in Kinetic: Fix Released Bug description: As can be seen in [1], strace FTBFS in kinetic: this is caused by the kernel headers (linux-libc-dev) which do not define F_GETLK64 and other on 64b builds because the kernel contains a bogus commit (306f7cc1e906 "uapi: always define F_GETLK64/F_SETLK64/F_SETLKW64 in fcntl.h"). This commit actually did the opposite of what it was supposed to do, namely defining those macros even on 64b builds. There is an attempt here to fix this which was not merged yet: https://lore.kernel.org/lkml/cajf2gtqtnmoeb62-63ou8y4dbrdym7iztdtfluxx9u0ltwu...@mail.gmail.com/T/ [1]: https://launchpadlibrarian.net/625441996/buildlog_ubuntu-kinetic- amd64.strace_5.16-0ubuntu4_BUILDING.txt.gz To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1990964/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1842320] Re: Can't boot: "error: out of memory." immediately after the grub menu
The terrible thing with compression is how we know of no universal rule. I'm sure you can even find non-pathological cases where lz4 compresses better than zpaq (and does so 100 times faster). And that's without taking I/O into account (or filters). An important thing to keep in mind here is that zstd currently uses its -1 preset while xz uses its default of -6. That means that these two tools which are often in the same ballpark are configured at vastly different compression/speed setting. What's important is that you get something that works for your setup and as I said above: we don't have a rule for optimal settings (and it's probably impossible to find one). One last note: xz decompression speed depends on the compressed size, not uncompressed size. This means that a more-compressed file decompresses faster. This might be the same with zstd since they're related compressors but I'm not 100% sure. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1842320 Title: Can't boot: "error: out of memory." immediately after the grub menu Status in grub: Unknown Status in OEM Priority Project: Triaged Status in grub2-signed package in Ubuntu: Fix Committed Status in grub2-unsigned package in Ubuntu: Fix Released Status in initramfs-tools package in Ubuntu: Won't Fix Status in linux package in Ubuntu: Confirmed Bug description: [Workaround] Some workarounds have been suggested in https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/comments/125 [Impact] * In some cases, if the users’ initramfs grow bigger, then it’ll likely not be able to be loaded by grub2. * Some real cases from OEM projects: In many built-in 4k monitor laptops with nvidia drivers, the u-d-c puts the nvidia*.ko to initramfs which grows the initramfs to ~120M. Also the gfxpayload=auto will remain to use 4K resolution since it’s what EFI POST passed. In this case, the grub isn't able to load initramfs because the grub_memalign() won't be able to get suitable memory for the larger file: ``` #0 grub_memalign (align=1, size=592214020) at ../../../grub-core/kern/mm.c:376 #1 0x7dd7b074 in grub_malloc (size=592214020) at ../../../grub-core/kern/mm.c:408 #2 0x7dd7a2c8 in grub_verifiers_open (io=0x7bc02d80, type=131076) at ../../../grub-core/kern/verifiers.c:150 #3 0x7dd801d4 in grub_file_open (name=0x7bc02f00 "/boot/initrd.img-5.17.0-1011-oem", type=131076) at ../../../grub-core/kern/file.c:121 #4 0x7bcd5a30 in ?? () #5 0x7fe21247 in ?? () #6 0x7bc030c8 in ?? () #7 0x00017fe21238 in ?? () #8 0x7bcd5320 in ?? () #9 0x7fe21250 in ?? () #10 0x in ?? () ``` Based on grub_mm_dump, we can see the memory region starvation in <1G addresses: Type StartEnd # Pages Attributes Available -00086FFF 0087 000F BS_Data00087000-00087FFF 0001 000F Available 00088000-0009EFFF 0017 000F Reserved 0009F000-0009 0001 000F Available 0010-00FF 0F00 000F LoaderCode 0100-01021FFF 0022 000F Available 01022000-238A7FFF 00022886 000F BS_Data238A8000-23927FFF 0080 000F Available 23928000-28860FFF 4F39 000F BS_Data28861000-2AB09FFF 22A9 000F LoaderCode 2AB0A000-2ACF8FFF 01EF 000F BS_Data2ACF9000-2B2FAFFF 0602 000F Available 2B2FB000-2B611FFF 0317 000F BS_Data2B612000-2B630FFF 001F 000F Available 2B631000-2B632FFF 0002 000F BS_Data2B633000-2B63CFFF 000A 000F Available 2B63D000-2B649FFF 000D 000F BS_Data2B64A000-2B64EFFF 0005 000F Available 2B64F000-2B666FFF 0018 000F BS_Data2B667000-2D8D5FFF 226F 000F LoaderCode 2D8D6000-2D8E9FFF 0014 000F BS_Data2D8EA000-2D925FFF 003C 000F LoaderCode 2D926000-2D932FFF 000D 000F BS_Data2D933000-2D969FFF 0037 000F BS_Code2D96A000-2D97
[Kernel-packages] [Bug 1842320] Re: Can't boot: "error: out of memory." immediately after the grub menu
Jeremy, there are duplicate firmware files. Replacing duplicates with
symlinks is probably the easiest and most efficient way to improve the
situation.
I get the following:
> % jdupes -mrS /lib/firmware
> Scanning: 2830 files, 286 items (in 1 specified)
> 405 duplicate files (in 212 sets), occupying 37 MB
With "jdupes -rl", "tar c /lib/firmware | zstd -1" drops from 428MB to
411MB. The effect on initrds is maybe slightly lower iirc but it's still
noticeable and it should be very easy and safe for us.
I can't tell exactly how many more people will be able to boot with that
change but I think the cost is small enough that it's worth trying.
Worst case, every kernel update is faster.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1842320
Title:
Can't boot: "error: out of memory." immediately after the grub menu
Status in grub:
Unknown
Status in OEM Priority Project:
Triaged
Status in grub2-signed package in Ubuntu:
Triaged
Status in grub2-unsigned package in Ubuntu:
Triaged
Status in initramfs-tools package in Ubuntu:
Won't Fix
Status in linux package in Ubuntu:
Confirmed
Bug description:
[Workaround]
Some workarounds have been suggested in
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1842320/comments/125
[Impact]
* In some cases, if the users’ initramfs grow bigger, then it’ll
likely not be able to be loaded by grub2.
* Some real cases from OEM projects:
In many built-in 4k monitor laptops with nvidia drivers, the u-d-c
puts the nvidia*.ko to initramfs which grows the initramfs to ~120M.
Also the gfxpayload=auto will remain to use 4K resolution since it’s
what EFI POST passed.
In this case, the grub isn't able to load initramfs because the
grub_memalign() won't be able to get suitable memory for the larger
file:
```
#0 grub_memalign (align=1, size=592214020) at ../../../grub-core/kern/mm.c:376
#1 0x7dd7b074 in grub_malloc (size=592214020) at
../../../grub-core/kern/mm.c:408
#2 0x7dd7a2c8 in grub_verifiers_open (io=0x7bc02d80, type=131076)
at ../../../grub-core/kern/verifiers.c:150
#3 0x7dd801d4 in grub_file_open (name=0x7bc02f00
"/boot/initrd.img-5.17.0-1011-oem",
type=131076) at ../../../grub-core/kern/file.c:121
#4 0x7bcd5a30 in ?? ()
#5 0x7fe21247 in ?? ()
#6 0x7bc030c8 in ?? ()
#7 0x00017fe21238 in ?? ()
#8 0x7bcd5320 in ?? ()
#9 0x7fe21250 in ?? ()
#10 0x in ?? ()
```
Based on grub_mm_dump, we can see the memory fragment (some parts seem
likely be used because of 4K resolution?) and doesn’t have available
contiguous memory for larger file as:
```
grub_real_malloc(...)
...
if (cur->size >= n + extra)
```
Based on UEFI Specification Section 7.2[1] and UEFI driver writers’
guide 4.2.3[2], we can ask 32bits+ on AllocatePages().
As most X86_64 platforms should support 64 bits addressing, we should
extend GRUB_EFI_MAX_USABLE_ADDRESS to 64 bits to get more available
memory.
* When users grown the initramfs, then probably will get initramfs
not found which really annoyed and impact the user experience (system
not able to boot).
[Test Plan]
* detailed instructions how to reproduce the bug:
1. Any method to grow the initramfs, such as install nvidia-driver.
2. If developers would like to reproduce, then could dd if=/dev/random
of=... bs=1M count=500, something like:
```
$ cat /usr/share/initramfs-tools/hooks/zzz-touch-a-file
#!/bin/sh
PREREQ=""
prereqs()
{
echo "$PREREQ"
}
case $1 in
# get pre-requisites
prereqs)
prereqs
exit 0
;;
esac
. /usr/share/initramfs-tools/hook-functions
dd if=/dev/random of=${DESTDIR}/test-500M bs=1M count=500
```
And then update-initramfs
* After applying my patches, the issue is gone.
* I did also test my test grubx64.efi in:
1. X86_64 qemu with
1.1. 60M initramfs + 5.15.0-37-generic kernel
1.2. 565M initramfs + 5.17.0-1011-oem kernel
2. Amd64 HP mobile workstation with
2.1. 65M initramfs + 5.15.0-39-generic kernel
2.2. 771M initramfs + 5.17.0-1011-oem kernel
All working well.
[Where problems could occur]
* The changes almost in i386/efi, thus the impact will be in the i386 /
x86_64 EFI system.
The other change is to modify the “grub-core/kern/efi/mm.c” but I use the
original addressing for “arm/arm64/ia64/riscv32/riscv64”.
Thus it should not impact them.
* There is a “#if defined(__x86_64__)” which intent to limit the >
32bits code in i386 system and also
```
#if defined (__code_model_large__)
-#define GRUB_EFI_MAX_USABLE_ADDRESS 0x
+#define GRUB_EFI_MAX_USABLE_ADDRESS __UINTPTR_MAX__
+#define GRUB_EFI_MAX_ALLOCATION_ADDRESS 0x7fff
#else
#define GRUB_EFI_MAX_USABLE_ADDRESS 0x7fff
+#defin
[Kernel-packages] [Bug 1842320] Re: Can't boot: "error: out of memory." immediately after the grub menu
I put together some notes and work-arounds in order to provide a simpler reference for people hitting this issue. I didn't test everything below but nothing should be risky. # Summary Grub attempts to read the initrd into a memory location that is too small. This issue is caused by a combination of several factors: - Grub not setting aside enough space for the initrd in memory and not at the right location, - Initrds having grown due to holding more modules and more firmware. This is especially the case with the nvidia proprietary driver. - Typical screen resolutions and their associated buffers in memory having increased. A related issue is not having enough space in /boot to hold enough initrds. # Solutions There are four ways to tackle these issues: - Patch grub, - Have fewer modules, - Compress initrds more, - Lower screen resolution at boot. ## Patch grub This is being worked on. It’s the trickier option so I’m not going to provide details but you can find that in the discussion above. ## Have fewer modules The default configuration creates initrds that are meant to be universal in order to accommodate as many hardware variants as possible. Unsurprisingly this makes initrds much bigger. It is possible to change the value of MODULES to ‘dep’ in /etc/initramfs-tools/initramfs.conf . This will drastically reduce the initrd size in virtually every configuration at the cost of not supporting during boot hardware that is not plugged when the initrd generation takes place. ## Compress initrds more Change the value of COMPRESS in /etc/initramfs-tools/initramfs.conf . In terms of compression level, you will have something along the lines of the following: lz4: 74066711 zstd: 54260379 gzip: 53829310 xz: 29665544 NB: these values are only valid on one specific machine and configuration; they are only meant to give an idea of compression ratio that can be obtained but the initrd uses MODULES=dep as outlined above. With a small hack, it is also possible to make xz compression much faster at a small cost in compression by adding the following at the end of initramfs.conf: export XZ_OPT='--lzma2=preset=0,dict=8M' In the future, it will be possible to directly set compression levels for every compression method. ## Lower screen resolution at boot You can use a lower resolution screen when booting. You can also edit /etc/default/grub and use the following: GRUB_GFXMODE=640x480 Remember to run update-grub afterwards. If you need to do it at boot-time, you will use GFXMODE instead (no ‘GRUB_’ prefix). -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1842320 Title: Can't boot: "error: out of memory." immediately after the grub menu Status in grub: Unknown Status in OEM Priority Project: Triaged Status in grub2-signed package in Ubuntu: Triaged Status in grub2-unsigned package in Ubuntu: Triaged Status in initramfs-tools package in Ubuntu: Won't Fix Status in linux package in Ubuntu: Confirmed Bug description: [Impact] * In some cases, if the users’ initramfs grow bigger, then it’ll likely not be able to be loaded by grub2. * Some real cases from OEM projects: In many built-in 4k monitor laptops with nvidia drivers, the u-d-c puts the nvidia*.ko to initramfs which grows the initramfs to ~120M. Also the gfxpayload=auto will remain to use 4K resolution since it’s what EFI POST passed. In this case, the grub isn't able to load initramfs because the grub_memalign() won't be able to get suitable memory for the larger file: ``` #0 grub_memalign (align=1, size=592214020) at ../../../grub-core/kern/mm.c:376 #1 0x7dd7b074 in grub_malloc (size=592214020) at ../../../grub-core/kern/mm.c:408 #2 0x7dd7a2c8 in grub_verifiers_open (io=0x7bc02d80, type=131076) at ../../../grub-core/kern/verifiers.c:150 #3 0x7dd801d4 in grub_file_open (name=0x7bc02f00 "/boot/initrd.img-5.17.0-1011-oem", type=131076) at ../../../grub-core/kern/file.c:121 #4 0x7bcd5a30 in ?? () #5 0x7fe21247 in ?? () #6 0x7bc030c8 in ?? () #7 0x00017fe21238 in ?? () #8 0x7bcd5320 in ?? () #9 0x7fe21250 in ?? () #10 0x in ?? () ``` Based on grub_mm_dump, we can see the memory fragment (some parts seem likely be used because of 4K resolution?) and doesn’t have available contiguous memory for larger file as: ``` grub_real_malloc(...) ... if (cur->size >= n + extra) ``` Based on UEFI Specification Section 7.2[1] and UEFI driver writers’ guide 4.2.3[2], we can ask 32bits+ on AllocatePages(). As most X86_64 platforms should support 64 bits addressing, we should extend GRUB_EFI_MAX_USABLE_ADDRESS to 64 bits to get more available memory. * When users grown the initramfs, then probably will get initramfs not found which reall
[Kernel-packages] [Bug 1990964] Re: FTBFS on kinetic
Attached is a patch to use strace 5.19 in order to match kernel 5.19. It also reduces delta with upstream sources for packaging and in particular it fixes debian/copyright (strace is now LGPL >= 2.1). ** Patch added: "new upstream release 5.19" https://bugs.launchpad.net/ubuntu/+source/strace/+bug/1990964/+attachment/5632653/+files/strace_5.19-0ubuntu1.debdiff -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1990964 Title: FTBFS on kinetic Status in linux package in Ubuntu: Fix Released Status in strace package in Ubuntu: Triaged Bug description: As can be seen in [1], strace FTBFS in kinetic: this is caused by the kernel headers (linux-libc-dev) which do not define F_GETLK64 and other on 64b builds because the kernel contains a bogus commit (306f7cc1e906 "uapi: always define F_GETLK64/F_SETLK64/F_SETLKW64 in fcntl.h"). This commit actually did the opposite of what it was supposed to do, namely defining those macros even on 64b builds. There is an attempt here to fix this which was not merged yet: https://lore.kernel.org/lkml/cajf2gtqtnmoeb62-63ou8y4dbrdym7iztdtfluxx9u0ltwu...@mail.gmail.com/T/ [1]: https://launchpadlibrarian.net/625441996/buildlog_ubuntu-kinetic- amd64.strace_5.16-0ubuntu4_BUILDING.txt.gz To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1990964/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1990964] Re: FTBFS on kinetic
** Changed in: strace (Ubuntu) Assignee: (unassigned) => Adrien Nader (adrien-n) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1990964 Title: FTBFS on kinetic Status in linux package in Ubuntu: Fix Released Status in strace package in Ubuntu: Triaged Bug description: As can be seen in [1], strace FTBFS in kinetic: this is caused by the kernel headers (linux-libc-dev) which do not define F_GETLK64 and other on 64b builds because the kernel contains a bogus commit (306f7cc1e906 "uapi: always define F_GETLK64/F_SETLK64/F_SETLKW64 in fcntl.h"). This commit actually did the opposite of what it was supposed to do, namely defining those macros even on 64b builds. There is an attempt here to fix this which was not merged yet: https://lore.kernel.org/lkml/cajf2gtqtnmoeb62-63ou8y4dbrdym7iztdtfluxx9u0ltwu...@mail.gmail.com/T/ [1]: https://launchpadlibrarian.net/625441996/buildlog_ubuntu-kinetic- amd64.strace_5.16-0ubuntu4_BUILDING.txt.gz To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1990964/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1842320] Re: Can't boot: "error: out of memory." immediately after the grub menu
As I've said elsewhere, if we dedup firmware files through symlinks, we
can save 10MB in initrds. Compression does not help because the
compressors have very small compression windows and cannot see
redundancy in practice (this applies to xz to a lesser extent but even
for xz there is an improvement to be gained).
I don't know if it will be enough but it could tip us just below some
threshold and it's easy to implement. It should also save memory at
runtime.
And for reference, there's a working but weird work-around that only
touches initramfs.conf, keeps MODULES=most and is reasonably fast. Use
'xz' as the compress program and then add the following on a new line:
export XZ_OPT=--lzma2=preset=0,dict=16M
(yes, it relies on implementation details of mkinitramfs)
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1842320
Title:
Can't boot: "error: out of memory." immediately after the grub menu
Status in grub:
Unknown
Status in OEM Priority Project:
Triaged
Status in grub2-signed package in Ubuntu:
Triaged
Status in grub2-unsigned package in Ubuntu:
Triaged
Status in initramfs-tools package in Ubuntu:
Won't Fix
Status in linux package in Ubuntu:
Confirmed
Bug description:
[Impact]
* In some cases, if the users’ initramfs grow bigger, then it’ll
likely not be able to be loaded by grub2.
* Some real cases from OEM projects:
In many built-in 4k monitor laptops with nvidia drivers, the u-d-c
puts the nvidia*.ko to initramfs which grows the initramfs to ~120M.
Also the gfxpayload=auto will remain to use 4K resolution since it’s
what EFI POST passed.
In this case, the grub isn't able to load initramfs because the
grub_memalign() won't be able to get suitable memory for the larger
file:
```
#0 grub_memalign (align=1, size=592214020) at ../../../grub-core/kern/mm.c:376
#1 0x7dd7b074 in grub_malloc (size=592214020) at
../../../grub-core/kern/mm.c:408
#2 0x7dd7a2c8 in grub_verifiers_open (io=0x7bc02d80, type=131076)
at ../../../grub-core/kern/verifiers.c:150
#3 0x7dd801d4 in grub_file_open (name=0x7bc02f00
"/boot/initrd.img-5.17.0-1011-oem",
type=131076) at ../../../grub-core/kern/file.c:121
#4 0x7bcd5a30 in ?? ()
#5 0x7fe21247 in ?? ()
#6 0x7bc030c8 in ?? ()
#7 0x00017fe21238 in ?? ()
#8 0x7bcd5320 in ?? ()
#9 0x7fe21250 in ?? ()
#10 0x in ?? ()
```
Based on grub_mm_dump, we can see the memory fragment (some parts seem
likely be used because of 4K resolution?) and doesn’t have available
contiguous memory for larger file as:
```
grub_real_malloc(...)
...
if (cur->size >= n + extra)
```
Based on UEFI Specification Section 7.2[1] and UEFI driver writers’
guide 4.2.3[2], we can ask 32bits+ on AllocatePages().
As most X86_64 platforms should support 64 bits addressing, we should
extend GRUB_EFI_MAX_USABLE_ADDRESS to 64 bits to get more available
memory.
* When users grown the initramfs, then probably will get initramfs
not found which really annoyed and impact the user experience (system
not able to boot).
[Test Plan]
* detailed instructions how to reproduce the bug:
1. Any method to grow the initramfs, such as install nvidia-driver.
2. If developers would like to reproduce, then could dd if=/dev/random
of=... bs=1M count=500, something like:
```
$ cat /usr/share/initramfs-tools/hooks/zzz-touch-a-file
#!/bin/sh
PREREQ=""
prereqs()
{
echo "$PREREQ"
}
case $1 in
# get pre-requisites
prereqs)
prereqs
exit 0
;;
esac
. /usr/share/initramfs-tools/hook-functions
dd if=/dev/random of=${DESTDIR}/test-500M bs=1M count=500
```
And then update-initramfs
* After applying my patches, the issue is gone.
* I did also test my test grubx64.efi in:
1. X86_64 qemu with
1.1. 60M initramfs + 5.15.0-37-generic kernel
1.2. 565M initramfs + 5.17.0-1011-oem kernel
2. Amd64 HP mobile workstation with
2.1. 65M initramfs + 5.15.0-39-generic kernel
2.2. 771M initramfs + 5.17.0-1011-oem kernel
All working well.
[Where problems could occur]
* The changes almost in i386/efi, thus the impact will be in the i386 /
x86_64 EFI system.
The other change is to modify the “grub-core/kern/efi/mm.c” but I use the
original addressing for “arm/arm64/ia64/riscv32/riscv64”.
Thus it should not impact them.
* There is a “#if defined(__x86_64__)” which intent to limit the >
32bits code in i386 system and also
```
#if defined (__code_model_large__)
-#define GRUB_EFI_MAX_USABLE_ADDRESS 0x
+#define GRUB_EFI_MAX_USABLE_ADDRESS __UINTPTR_MAX__
+#define GRUB_EFI_MAX_ALLOCATION_ADDRESS 0x7fff
#else
#define GRUB_EFI_MAX_USABLE_ADDRESS 0x7fff
+#define GRUB_EFI_MAX_ALLOCATION_ADDRESS 0x3fff
#e
