[Bug 1185394] Re: systemd-udev fails when processing many logical volumes on boot

2013-06-03 Thread Martin Pitt
Could you please try 202-0ubuntu11pitti1 from https://launchpad.net/~pitti/+archive/ppa ? (It's the only package for saucy, thus safe for dist-upgrading). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1185394] Re: systemd-udev fails when processing many logical volumes on boot

2013-06-03 Thread Stefan Bader
I have done about 5 reboots with the updated packages, and I can confirm the crash on a Xen boot to be gone. Interestingly now the init-bottom is taking a while as expected (maybe 5s) and all of the LVs on the VG with the large number of LVs are correctly set up (having the additional links). Just

[Bug 1185394] Re: systemd-udev fails when processing many logical volumes on boot

2013-06-03 Thread Stefan Bader
Now done another 5 reboots with the udev script in init-bottom doing a udevadm settle --timeout=121 before the exit. The delay before init- bottom finishes seem the same (or even a little bit quicker) than before, a ps after the exit has no udevd or vgscan/vgchange processes running (with only

[Bug 1185394] Re: systemd-udev fails when processing many logical volumes on boot

2013-06-03 Thread Martin Pitt
Stefan and I debugged and discussed that. The new udev has a reduced number of parallel workers now (performance optimization, see Harald Hoyer's measurements), which potentially leads to more serialization for a huge number of events (as we get in initramfs by calling udevadm trigger). This is

[Bug 1185394] Re: systemd-udev fails when processing many logical volumes on boot

2013-06-03 Thread Martin Pitt
** Changed in: systemd (Ubuntu) Status: In Progress = Fix Committed ** Changed in: systemd (Ubuntu) Assignee: Stefan Bader (stefan-bader-canonical) = Martin Pitt (pitti) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1185394] Re: systemd-udev fails when processing many logical volumes on boot

2013-06-03 Thread Stefan Bader
Updated to 202-0ubuntu11pitti2 (which dropped the dm cookie patch but added a settle before exit in init-bottom/udev) and re-ran the reboot tests. 5-6 times to Xen and once to non-Xen. All runs did set up all LVs. Time waiting in settle/exit seems to be not longer (but maybe even quicker) than

[Bug 1185394] Re: systemd-udev fails when processing many logical volumes on boot

2013-06-03 Thread Launchpad Bug Tracker
** Branch linked: lp:ubuntu/saucy-proposed/systemd -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1185394 Title: systemd-udev fails when processing many logical volumes on boot To manage

[Bug 1185394] Re: systemd-udev fails when processing many logical volumes on boot

2013-06-03 Thread Launchpad Bug Tracker
This bug was fixed in the package systemd - 202-0ubuntu12 --- systemd (202-0ubuntu12) saucy; urgency=low * Rename debian/tests/localed to locale-locale, as we are going to add more tests for localed. * Add debian/tests/localed-x11-keymap: Test localectl set-x11-keymap. *

Re: [Bug 1185394] Re: systemd-udev fails when processing many logical volumes on boot

2013-06-02 Thread Martin Pitt
Stefan Bader [2013-05-29 17:32 -]: There is also a related change missing from us which was handling events with timeout as urgently as dm cookie ones. Right, this was our old avoid-exit-deadlock-for-timely-events patch: That was applied for http://pad.lv/842560. Andy discussed that with

Re: [Bug 1185394] Re: systemd-udev fails when processing many logical volumes on boot

2013-06-02 Thread Martin Pitt
Thanks for your investigations! Stefan Bader [2013-05-29 13:42 -]: Booting in Xen mode there is one systemd-udevd crashing immediatly after calling udevadm control --exit and that ends the udevadm command (so it does not wait). That indeed sounds like the most plausible cause for now, and

[Bug 1185394] Re: systemd-udev fails when processing many logical volumes on boot

2013-05-29 Thread Stefan Bader
Debugging ideas (from discussion with pitti on IRC): (1) add a ps | grep ... wait loop into the hook until really all udev processes are gone that would confirm that the child processes actually run into the /dev issue (2) some ulimit -c unlimited and copying core dumps to /run/udev/ (3) move

[Bug 1185394] Re: systemd-udev fails when processing many logical volumes on boot

2013-05-29 Thread Stefan Bader
(1) Struggeling a bit with weirdly different behaviour between boots as Xen host and normal boots. First there was debug added for the Xen boot. This should only cause kernel messages of lower priority to be sent to the console, but it seems to have the effect of not showing any output of the

[Bug 1185394] Re: systemd-udev fails when processing many logical volumes on boot

2013-05-29 Thread Stefan Bader
I think the long report for (1) also covers question (4). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1185394 Title: systemd-udev fails when processing many logical volumes on boot To manage

[Bug 1185394] Re: systemd-udev fails when processing many logical volumes on boot

2013-05-29 Thread Stefan Bader
(1) Addition, seems I managed to see one rare-time segfault on a normal boot (but only one so far on many reboots). (3) Ok, finally I got it! Updated udev/libudev to 202-0ubuntu10(amd64) to be able to have matching ddebs and disabled mounting /home and /home/isos as otherwise the plymouth bug

[Bug 1185394] Re: systemd-udev fails when processing many logical volumes on boot

2013-05-29 Thread Stefan Bader
** Attachment added: corefile of /lib/systemd/systemd-udevd https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1185394/+attachment/3689950/+files/boot-core -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1185394] Re: systemd-udev fails when processing many logical volumes on boot

2013-05-29 Thread Stefan Bader
Also a braindump it gets late. Tried to figure out what changed between the udev-175 udevd and the new one. Taking away the whitespace changes. Most things are log function renames. Thought there is a hunk missing that would copy dev entries but that might have been old support code for tmpfs

[Bug 1185394] Re: systemd-udev fails when processing many logical volumes on boot

2013-05-29 Thread Stefan Bader
So summarizing: Maybe two problems: 1. Boot as Xen host, higher chance of having the segfault resulting in udevadm returning before workers have ended and that yields /dev/null not found. 2. Some problem in the sequence leading to missed events. Could be that vgchange running after udevd