AW: Debugging initramfs, server hangs during boot process
I have booted on 6.3 and 6.4, both with the same result. If I boot with 6.1 the prompt for entering the luks passphrase is there and I can use the machine. Getting 6.3 using snapshot.debian.org is possible, but also needs some changes on the packages config. I managed, but this needs additional work to get it right. I have filed a bug against udev and later vs. nvidia-driver. Purging nvidia stuff and using nouveau I could also run with 6.3 o 6.4. After 6.5 came into debian I have tried again with nvidia and it works again. So for 6.3 and 6.4 I basically couldn't use nvidia. Not sure about 6.2 as this is not available on debian. To diagnose the issue I always booted into debian live. Mounted the encrypted luks partition and modified the initramfs-tools, generated new initrd-Images and try again and again. Getting early netconsole to work and fiddling with the initramfs stuff was not so easy for me. I know that there can be issues running sid, but this was a lot harder than expected. You have a working system, do an upgrade and boom this system cannot even boot up. Using debug=x (or whatever value, it doesn't matter) and netconsole=... helped to remedy this problem. Sigh, Thomas -Original-Nachricht- Betreff: Re: AW: Debugging initramfs, server hangs during boot process Datum: 2023-08-30T12:00:56+0200 Von: "Michel Verdier" An: "debian-user@lists.debian.org" On 2023-08-30, thah...@t-online.de wrote: > The last USB device in this list is a bluetooth card. I have blacklisted > btusb, > but this didn't help, still hangs. > At 182 seconds I pressed CRTL_ALT_DEL and the times without sleep statements > come down to like 11 seconds when it hangs. You should blacklist usb devices not found during boot and still listed with udevadm. You can also try to unplug all usb devices and plug them one by one to find which hangs. Do you try to boot on another kernel or a debian live ?
Re: AW: Debugging initramfs, server hangs during boot process
On 2023-08-30, thah...@t-online.de wrote: > The last USB device in this list is a bluetooth card. I have blacklisted > btusb, > but this didn't help, still hangs. > At 182 seconds I pressed CRTL_ALT_DEL and the times without sleep statements > come down to like 11 seconds when it hangs. You should blacklist usb devices not found during boot and still listed with udevadm. You can also try to unplug all usb devices and plug them one by one to find which hangs. Do you try to boot on another kernel or a debian live ?
AW: Debugging initramfs, server hangs during boot process
3] input: SINOWEALTH GXT 144 Gaming Mouse Keyboard as /devices/pci:00/:00:14.0/usb1/1-4/1-4:1.1/0003:145F:026D.0004/input/input7 [ 68.977606] hid-generic 0003:145F:026D.0004: input,hiddev0,hidraw3: USB HID v1.11 Keyboard [SINOWEALTH GXT 144 Gaming Mouse] on usb-:00:14.0-4/input1 [ 69.105124] usb 1-9: new high-speed USB device number 4 using xhci_hcd [ 69.254887] usb 1-9: New USB device found, idVendor=05e3, idProduct=0608, bcdDevice=60.90 [ 69.254915] usb 1-9: New USB device strings: Mfr=0, Product=1, SerialNumber=0 [ 69.254929] usb 1-9: Product: USB2.0 Hub [ 69.256630] hub 1-9:1.0: USB hub found [ 69.256981] hub 1-9:1.0: 4 ports detected [ 69.258124] usb: port power management may be unreliable [ 69.389128] usb 1-11: new full-speed USB device number 5 using xhci_hcd [ 69.538356] usb 1-11: New USB device found, idVendor=1462, idProduct=7d18, bcdDevice= 0.01 [ 69.538385] usb 1-11: New USB device strings: Mfr=1, Product=2, SerialNumber=3 [ 69.538400] usb 1-11: Product: MYSTIC LIGHT [ 69.538410] usb 1-11: Manufacturer: MSI [ 69.538420] usb 1-11: SerialNumber: A02021081805 [ 69.541808] input: MSI MYSTIC LIGHT as /devices/pci:00/:00:14.0/usb1/1-11/1-11:1.0/0003:1462:7D18.0005/input/input9 [ 69.542258] hid-generic 0003:1462:7D18.0005: input,hiddev1,hidraw4: USB HID v1.10 Device [MSI MYSTIC LIGHT ] on usb-:00:14.0-11/input0 [ 69.669124] usb 1-14: new full-speed USB device number 6 using xhci_hcd [ 69.819394] usb 1-14: New USB device found, idVendor=8087, idProduct=0026, bcdDevice= 0.02 [ 69.819423] usb 1-14: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [ 182.141933] sd 5:0:0:0: [sda] Synchronizing SCSI cache [ 182.142749] sd 4:0:0:0: [sdb] Synchronizing SCSI cache The last USB device in this list is a bluetooth card. I have blacklisted btusb, but this didn't help, still hangs. At 182 seconds I pressed CRTL_ALT_DEL and the times without sleep statements come down to like 11 seconds when it hangs. -Original-Nachricht----- Betreff: Re: AW: Debugging initramfs, server hangs during boot process Datum: 2023-08-28T11:42:19+0200 Von: "Michel Verdier" An: "debian-user@lists.debian.org" On 2023-08-28, thah...@t-online.de wrote: > It hangs in /usr/share/initramfs-tools/scripts/init-top/udev > > The 2nd last udev call hangs my box > udevadm trigger --type=devices --action=add Perhaps add before udevadm trigger --verbose --dry-run --type=devices --action=add
Re: AW: Debugging initramfs, server hangs during boot process
On 2023-08-28, thah...@t-online.de wrote: > It hangs in /usr/share/initramfs-tools/scripts/init-top/udev > > The 2nd last udev call hangs my box > udevadm trigger --type=devices --action=add Perhaps add before udevadm trigger --verbose --dry-run --type=devices --action=add
Re: AW: Debugging initramfs, server hangs during boot process
On 28/08/2023 05:19, thah...@t-online.de wrote: It hangs in /usr/share/initramfs-tools/scripts/init-top/udev Unsure if it is related or not (I have not tried to debug it), but I have noticed some issues with laptop boot when a USB hub with a keyboard and a mouse is connected. It might be an unreliable connector inside the hub, however symptoms with another laptop are more mild.
AW: Debugging initramfs, server hangs during boot process
The udev script is from the udev package -Original-Nachricht- Betreff: AW: Debugging initramfs, server hangs during boot process Datum: 2023-08-28T00:20:33+0200 Von: "thah...@t-online.de" An: "debian user" It hangs in /usr/share/initramfs-tools/scripts/init-top/udev The 2nd last udev call hangs my box udevadm trigger --type=devices --action=add This initramfs stuff hasn't changed since July 2022, so the real problem must be inside udev To narrow it down I have added echo and sleep statements If I print out the contents of the initramfs-tools-core package, than the udev script is not present But it is just a first step, as the commented "udevadm trigger --type=devices --action=add" does not hang the system, but without it I cannot use the keyboard to unlock the crypted root. Sigh! -Original-Nachricht----- Betreff: Re: AW: Debugging initramfs, server hangs during boot process Datum: 2023-08-26T13:45:19+0200 Von: "Michel Verdier" An: "debian-user@lists.debian.org" On 2023-08-26, thah...@t-online.de wrote: > Tried with clocksourche=hpet > Now the Switched to clocksouce tsc is missing and the last line is > clocksource: tsc: mask . > like before the 2nd last line (as to be expected) On my kernel I always have those 3 lines during boot : clocksource: tsc: mask: 0x max_cycles: 0x255cb6cc5db, max_idle_ns: 440795203504 ns clocksource: Switched to clocksource tsc Freeing initrd memory: 26804K So you appears to freeze when freeing initrd memory
AW: Debugging initramfs, server hangs during boot process
It hangs in /usr/share/initramfs-tools/scripts/init-top/udev The 2nd last udev call hangs my box udevadm trigger --type=devices --action=add This initramfs stuff hasn't changed since July 2022, so the real problem must be inside udev To narrow it down I have added echo and sleep statements If I print out the contents of the initramfs-tools-core package, than the udev script is not present But it is just a first step, as the commented "udevadm trigger --type=devices --action=add" does not hang the system, but without it I cannot use the keyboard to unlock the crypted root. Sigh! -Original-Nachricht- Betreff: Re: AW: Debugging initramfs, server hangs during boot process Datum: 2023-08-26T13:45:19+0200 Von: "Michel Verdier" An: "debian-user@lists.debian.org" On 2023-08-26, thah...@t-online.de wrote: > Tried with clocksourche=hpet > Now the Switched to clocksouce tsc is missing and the last line is > clocksource: tsc: mask . > like before the 2nd last line (as to be expected) On my kernel I always have those 3 lines during boot : clocksource: tsc: mask: 0x max_cycles: 0x255cb6cc5db, max_idle_ns: 440795203504 ns clocksource: Switched to clocksource tsc Freeing initrd memory: 26804K So you appears to freeze when freeing initrd memory
Re: AW: Debugging initramfs, server hangs during boot process
On 2023-08-26, thah...@t-online.de wrote: > Tried with clocksourche=hpet > Now the Switched to clocksouce tsc is missing and the last line is > clocksource: tsc: mask . > like before the 2nd last line (as to be expected) On my kernel I always have those 3 lines during boot : clocksource: tsc: mask: 0x max_cycles: 0x255cb6cc5db, max_idle_ns: 440795203504 ns clocksource: Switched to clocksource tsc Freeing initrd memory: 26804K So you appears to freeze when freeing initrd memory
AW: Debugging initramfs, server hangs during boot process
Tried with clocksourche=hpet Now the Switched to clocksouce tsc is missing and the last line is clocksource: tsc: mask . like before the 2nd last line (as to be expected) It still hangs. It is not hard locked, can do CTRL-ALT-DEL to boot again, booting Windows works. -Original-Nachricht- Betreff: Re: AW: Debugging initramfs, server hangs during boot process Datum: 2023-08-26T12:04:46+0200 Von: "Tixy" An: "debian-user@lists.debian.org" On Sat, 2023-08-26 at 11:07 +0200, thah...@t-online.de wrote: > I had debug on the command line before, just didn't know of debug=vc > However the output on the screen is the same with either one. > > Last line is clocksource: Switched to clocksource tsc I notice two 1 second delays during boot on my new PC which occurred when TSC related stuff was the last line on the screen. I did fiddle around with some settings (can't remember what) in order to try and remove this delay, but decided to leave things at default in the end. Now googling for "clocksource TSC" gets me a Redhat page about hardware clocks. [1] Maybe you could try changing the clock with "clocksource=hpet" or something else, in case the kernel is having problems with TSC? It's ominous that some search results reveal the commandline option "tsc=reliable" implying the opposite may true sometimes. [1] https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux_for_real_time/7/html/reference_guide/chap-timestamping -- Tixy
Re: AW: Debugging initramfs, server hangs during boot process
On Sat, 2023-08-26 at 11:07 +0200, thah...@t-online.de wrote: > I had debug on the command line before, just didn't know of debug=vc > However the output on the screen is the same with either one. > > Last line is clocksource: Switched to clocksource tsc I notice two 1 second delays during boot on my new PC which occurred when TSC related stuff was the last line on the screen. I did fiddle around with some settings (can't remember what) in order to try and remove this delay, but decided to leave things at default in the end. Now googling for "clocksource TSC" gets me a Redhat page about hardware clocks. [1] Maybe you could try changing the clock with "clocksource=hpet" or something else, in case the kernel is having problems with TSC? It's ominous that some search results reveal the commandline option "tsc=reliable" implying the opposite may true sometimes. [1] https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux_for_real_time/7/html/reference_guide/chap-timestamping -- Tixy
AW: Debugging initramfs, server hangs during boot process
I had debug on the command line before, just didn't know of debug=vc However the output on the screen is the same with either one. Last line is clocksource: Switched to clocksource tsc Looking at the stuff in /usr/share/initramfs-tools/scripts I think I would need to mess the shell scripts with debug prints just to find where it is stuck uaaah! -Original-Nachricht- Betreff: Re: Debugging initramfs, server hangs during boot process Datum: 2023-08-26T10:20:54+0200 Von: "Michel Verdier" An: "debian-user@lists.debian.org" On 2023-08-26, Tixy wrote: >> I thought of that, too, but debug writes to a tmpfs (it has to, at this >> point). If the machine locks up, the log is lost... > > Logs will appear on the screen so long as you don't have the 'quiet' > parameter on the Linux commandline. I think it always writes to tmpfs which is moved to /run/initramfs, but only if the boot succeded. So thahn01 should use debug=vc to get debug logs on screen during boot.
Re: Debugging initramfs, server hangs during boot process
On 2023-08-26, Tixy wrote: >> I thought of that, too, but debug writes to a tmpfs (it has to, at this >> point). If the machine locks up, the log is lost... > > Logs will appear on the screen so long as you don't have the 'quiet' > parameter on the Linux commandline. I think it always writes to tmpfs which is moved to /run/initramfs, but only if the boot succeded. So thahn01 should use debug=vc to get debug logs on screen during boot.
Re: Debugging initramfs, server hangs during boot process
On Sat, 2023-08-26 at 07:59 +0200, to...@tuxteam.de wrote: > On Sat, Aug 26, 2023 at 07:40:21AM +0200, Michel Verdier wrote: [...] > > > > Did you try with "debug" on the linux command line to get more logs > > ? > > I thought of that, too, but debug writes to a tmpfs (it has to, at this > point). If the machine locks up, the log is lost... Logs will appear on the screen so long as you don't have the 'quiet' parameter on the Linux commandline. -- Tixy
Re: Debugging initramfs, server hangs during boot process
On Sat, Aug 26, 2023 at 07:40:21AM +0200, Michel Verdier wrote: > On 2023-08-25, thah...@t-online.de wrote: > > > Looking at the initramfs manpages i found that I can get into the busybox > > during > > the boot process with break=... on the linux command line. > > break=top > > works, but > > break=modules > > is not reached. It hangs before that. > > Did you try with "debug" on the linux command line to get more logs ? I thought of that, too, but debug writes to a tmpfs (it has to, at this point). If the machine locks up, the log is lost... Does the machine lock up hard? Perhaps it's one module being loaded. May be trying to blacklist as much as possible and going from there is a possibility? Cheers -- t signature.asc Description: PGP signature
Re: Debugging initramfs, server hangs during boot process
On 2023-08-25, thah...@t-online.de wrote: > Looking at the initramfs manpages i found that I can get into the busybox > during > the boot process with break=... on the linux command line. > break=top > works, but > break=modules > is not reached. It hangs before that. Did you try with "debug" on the linux command line to get more logs ?
Debugging initramfs, server hangs during boot process
Hi all, being on sid, I know that there can be problems. However, this seems to be a bit hard for me to solve without proper help. The machine hangs during boot. Looking at the initramfs manpages i found that I can get into the busybox during the boot process with break=... on the linux command line. break=top works, but break=modules is not reached. It hangs before that. I have compared the initrd image of that broken box with the image of this box I am writing this mail on. There are smaller differences, but it doesn't ring a bell with me I have used debian live to chroot into the system and did updates twice. However that didn't help, boot process still hangs. Any advice? Tnx Thomas
