[Touch-packages] [Bug 1623383] Re: Some restarts fail due to missing base devices
** Changed in: systemd (Ubuntu) Status: Confirmed => Won't Fix -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1623383 Title: Some restarts fail due to missing base devices Status in systemd package in Ubuntu: Won't Fix Bug description: Arch: s390x Release: Yakkety / 16.10 This happens on some (but not all) system starts with Yakkety. In Xenial (which is using the same 4.4 kernel version the Yakkety systems were using when the problem was first observed) this did not happen. The system (LPAR) this was seen first was an upgrade from Xenial but since then has been freshly installed with Yakkety. The same behaviour is seen on a zVM guest running Yakkety. The attached syslog shows a failed boot, followed by one that did work. Note the "Found device .*(sclp|encc00).*" messages in the good boot. Those are missing in the bad attempt and as a result networking and console fail to be usable. Also note, those boots were 4.8 kernels but we saw this with 4.4 kernels, too. This might be a systemd problem / race, I just filed it into udev for now as that better matches the not finding basic devices symptom. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1623383/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1623383] Re: Some restarts fail due to missing base devices
The system LPAR this was seen first was an upgrade from Xenial but since then has been freshly installed with Yakkety. The same behaviour is seen on a zVM guest running Yakkety. If anyone face printer issue click here http://canonprintersupport247.com -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1623383 Title: Some restarts fail due to missing base devices Status in systemd package in Ubuntu: Confirmed Bug description: Arch: s390x Release: Yakkety / 16.10 This happens on some (but not all) system starts with Yakkety. In Xenial (which is using the same 4.4 kernel version the Yakkety systems were using when the problem was first observed) this did not happen. The system (LPAR) this was seen first was an upgrade from Xenial but since then has been freshly installed with Yakkety. The same behaviour is seen on a zVM guest running Yakkety. The attached syslog shows a failed boot, followed by one that did work. Note the "Found device .*(sclp|encc00).*" messages in the good boot. Those are missing in the bad attempt and as a result networking and console fail to be usable. Also note, those boots were 4.8 kernels but we saw this with 4.4 kernels, too. This might be a systemd problem / race, I just filed it into udev for now as that better matches the not finding basic devices symptom. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1623383/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1623383] Re: Some restarts fail due to missing base devices
Okay suggestion is to: - Update zKVM (which is based on IBM KVM on z) to the latest and last release 1.1.2 FP4. 1.1.2 FP4 is the only one that is still supported. I can help on that. - It might also be worth trying to limit the devices on the LPAR with the help of cio_ignore and see if that make a difference - in case of a potential race-conditions. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1623383 Title: Some restarts fail due to missing base devices Status in systemd package in Ubuntu: Confirmed Bug description: Arch: s390x Release: Yakkety / 16.10 This happens on some (but not all) system starts with Yakkety. In Xenial (which is using the same 4.4 kernel version the Yakkety systems were using when the problem was first observed) this did not happen. The system (LPAR) this was seen first was an upgrade from Xenial but since then has been freshly installed with Yakkety. The same behaviour is seen on a zVM guest running Yakkety. The attached syslog shows a failed boot, followed by one that did work. Note the "Found device .*(sclp|encc00).*" messages in the good boot. Those are missing in the bad attempt and as a result networking and console fail to be usable. Also note, those boots were 4.8 kernels but we saw this with 4.4 kernels, too. This might be a systemd problem / race, I just filed it into udev for now as that better matches the not finding basic devices symptom. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1623383/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1623383] Re: Some restarts fail due to missing base devices
Here is the number from the beginning of this SRU cycle till today (since Jul. 3) We have 6 jobs for each node, which requires at least 6 re-deployments (note that it's just a reboot for s390x, since we don't re-deploy them). The failures here means how many times we need to reboot it manually. The total jobs count on jenkins is larger than the sum of failures + 6, as one failure may cause all the other job to fail and need to be restarted. s2lp4 (Ubuntu on LPAR) - 1 failures, jobs count: 17 kernel02 (zVM) - 2 failures, jobs count: 16 s2lp6g003 (zKVM) - 6 failures, jobs count: 9 kernel03 (zKVM) - 7 failures, jobs count: 21 kernel03 is still running. The number of failures and jobs count will change. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1623383 Title: Some restarts fail due to missing base devices Status in systemd package in Ubuntu: Confirmed Bug description: Arch: s390x Release: Yakkety / 16.10 This happens on some (but not all) system starts with Yakkety. In Xenial (which is using the same 4.4 kernel version the Yakkety systems were using when the problem was first observed) this did not happen. The system (LPAR) this was seen first was an upgrade from Xenial but since then has been freshly installed with Yakkety. The same behaviour is seen on a zVM guest running Yakkety. The attached syslog shows a failed boot, followed by one that did work. Note the "Found device .*(sclp|encc00).*" messages in the good boot. Those are missing in the bad attempt and as a result networking and console fail to be usable. Also note, those boots were 4.8 kernels but we saw this with 4.4 kernels, too. This might be a systemd problem / race, I just filed it into udev for now as that better matches the not finding basic devices symptom. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1623383/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1623383] Re: Some restarts fail due to missing base devices
Hi, we’re trying to explore different approaches to investigate, resolve and/or work around this issue, but regrettably there do not appear to be any simple solutions. Would it be possible to record the failure rate during the testing for the next SRU cycle? Perhaps something simple like “5 failures in 50 redeployments over a 2 week period”? Many thanks. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1623383 Title: Some restarts fail due to missing base devices Status in systemd package in Ubuntu: Confirmed Bug description: Arch: s390x Release: Yakkety / 16.10 This happens on some (but not all) system starts with Yakkety. In Xenial (which is using the same 4.4 kernel version the Yakkety systems were using when the problem was first observed) this did not happen. The system (LPAR) this was seen first was an upgrade from Xenial but since then has been freshly installed with Yakkety. The same behaviour is seen on a zVM guest running Yakkety. The attached syslog shows a failed boot, followed by one that did work. Note the "Found device .*(sclp|encc00).*" messages in the good boot. Those are missing in the bad attempt and as a result networking and console fail to be usable. Also note, those boots were 4.8 kernels but we saw this with 4.4 kernels, too. This might be a systemd problem / race, I just filed it into udev for now as that better matches the not finding basic devices symptom. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1623383/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1623383] Re: Some restarts fail due to missing base devices
journalctl log for kernel02 (s390x.zVM), which drops to emergency mode as well ** Attachment added: "kernel02.log" https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1623383/+attachment/4841328/+files/kernel02.log -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1623383 Title: Some restarts fail due to missing base devices Status in systemd package in Ubuntu: Confirmed Bug description: Arch: s390x Release: Yakkety / 16.10 This happens on some (but not all) system starts with Yakkety. In Xenial (which is using the same 4.4 kernel version the Yakkety systems were using when the problem was first observed) this did not happen. The system (LPAR) this was seen first was an upgrade from Xenial but since then has been freshly installed with Yakkety. The same behaviour is seen on a zVM guest running Yakkety. The attached syslog shows a failed boot, followed by one that did work. Note the "Found device .*(sclp|encc00).*" messages in the good boot. Those are missing in the bad attempt and as a result networking and console fail to be usable. Also note, those boots were 4.8 kernels but we saw this with 4.4 kernels, too. This might be a systemd problem / race, I just filed it into udev for now as that better matches the not finding basic devices symptom. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1623383/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1623383] Re: Some restarts fail due to missing base devices
Node s2lp6g003 (s390x.zKVM) boot to emergency mode. Please find attachment for the journalctl log. ** Attachment added: "s2lp6g003.log" https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1623383/+attachment/4840829/+files/s2lp6g003.log -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1623383 Title: Some restarts fail due to missing base devices Status in systemd package in Ubuntu: Confirmed Bug description: Arch: s390x Release: Yakkety / 16.10 This happens on some (but not all) system starts with Yakkety. In Xenial (which is using the same 4.4 kernel version the Yakkety systems were using when the problem was first observed) this did not happen. The system (LPAR) this was seen first was an upgrade from Xenial but since then has been freshly installed with Yakkety. The same behaviour is seen on a zVM guest running Yakkety. The attached syslog shows a failed boot, followed by one that did work. Note the "Found device .*(sclp|encc00).*" messages in the good boot. Those are missing in the bad attempt and as a result networking and console fail to be usable. Also note, those boots were 4.8 kernels but we saw this with 4.4 kernels, too. This might be a systemd problem / race, I just filed it into udev for now as that better matches the not finding basic devices symptom. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1623383/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
Re: [Touch-packages] [Bug 1623383] Re: Some restarts fail due to missing base devices
On 13 March 2017 at 16:58, Ian Thompson <1623...@bugs.launchpad.net> wrote: > I have the same problem with two Dell PowerEdge FC630. > Ubuntu 16.04.2 LTS/ 4.4.0-66-generic > > Shutdown/reboot stalls at different points each attempt. > > An IBM System x3650 M4 in the same rack, at the same software level, > reboots cleanly every time. > Could you please open a new bug report using ubuntu-bug tool and include systemd journal which includes failed boots? Just in case if this is not the same issue. As so far we have only reproduced this on s390x architecture. Please enable persistent journal first by doing $ sudo mkdir -p /var/log/journal -- Regards, Dimitri. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1623383 Title: Some restarts fail due to missing base devices Status in systemd package in Ubuntu: Confirmed Bug description: Arch: s390x Release: Yakkety / 16.10 This happens on some (but not all) system starts with Yakkety. In Xenial (which is using the same 4.4 kernel version the Yakkety systems were using when the problem was first observed) this did not happen. The system (LPAR) this was seen first was an upgrade from Xenial but since then has been freshly installed with Yakkety. The same behaviour is seen on a zVM guest running Yakkety. The attached syslog shows a failed boot, followed by one that did work. Note the "Found device .*(sclp|encc00).*" messages in the good boot. Those are missing in the bad attempt and as a result networking and console fail to be usable. Also note, those boots were 4.8 kernels but we saw this with 4.4 kernels, too. This might be a systemd problem / race, I just filed it into udev for now as that better matches the not finding basic devices symptom. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1623383/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1623383] Re: Some restarts fail due to missing base devices
I have the same problem with two Dell PowerEdge FC630. Ubuntu 16.04.2 LTS/ 4.4.0-66-generic Shutdown/reboot stalls at different points each attempt. An IBM System x3650 M4 in the same rack, at the same software level, reboots cleanly every time. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1623383 Title: Some restarts fail due to missing base devices Status in systemd package in Ubuntu: Confirmed Bug description: Arch: s390x Release: Yakkety / 16.10 This happens on some (but not all) system starts with Yakkety. In Xenial (which is using the same 4.4 kernel version the Yakkety systems were using when the problem was first observed) this did not happen. The system (LPAR) this was seen first was an upgrade from Xenial but since then has been freshly installed with Yakkety. The same behaviour is seen on a zVM guest running Yakkety. The attached syslog shows a failed boot, followed by one that did work. Note the "Found device .*(sclp|encc00).*" messages in the good boot. Those are missing in the bad attempt and as a result networking and console fail to be usable. Also note, those boots were 4.8 kernels but we saw this with 4.4 kernels, too. This might be a systemd problem / race, I just filed it into udev for now as that better matches the not finding basic devices symptom. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1623383/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1623383] Re: Some restarts fail due to missing base devices
Hello Frank, I have set udev.log_priority and rd.udev.log_priority to debug on kernel03 with smb's help. Let's wait for this happens. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1623383 Title: Some restarts fail due to missing base devices Status in systemd package in Ubuntu: Confirmed Bug description: Arch: s390x Release: Yakkety / 16.10 This happens on some (but not all) system starts with Yakkety. In Xenial (which is using the same 4.4 kernel version the Yakkety systems were using when the problem was first observed) this did not happen. The system (LPAR) this was seen first was an upgrade from Xenial but since then has been freshly installed with Yakkety. The same behaviour is seen on a zVM guest running Yakkety. The attached syslog shows a failed boot, followed by one that did work. Note the "Found device .*(sclp|encc00).*" messages in the good boot. Those are missing in the bad attempt and as a result networking and console fail to be usable. Also note, those boots were 4.8 kernels but we saw this with 4.4 kernels, too. This might be a systemd problem / race, I just filed it into udev for now as that better matches the not finding basic devices symptom. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1623383/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1623383] Re: Some restarts fail due to missing base devices
After further investigation the issue could be related to udev. Would it be possible to start the systems with kernel parameters udev.log_priority and rd.udev.log_priority set to debug and share again the logs from a failed system? -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1623383 Title: Some restarts fail due to missing base devices Status in systemd package in Ubuntu: Confirmed Bug description: Arch: s390x Release: Yakkety / 16.10 This happens on some (but not all) system starts with Yakkety. In Xenial (which is using the same 4.4 kernel version the Yakkety systems were using when the problem was first observed) this did not happen. The system (LPAR) this was seen first was an upgrade from Xenial but since then has been freshly installed with Yakkety. The same behaviour is seen on a zVM guest running Yakkety. The attached syslog shows a failed boot, followed by one that did work. Note the "Found device .*(sclp|encc00).*" messages in the good boot. Those are missing in the bad attempt and as a result networking and console fail to be usable. Also note, those boots were 4.8 kernels but we saw this with 4.4 kernels, too. This might be a systemd problem / race, I just filed it into udev for now as that better matches the not finding basic devices symptom. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1623383/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1623383] Re: Some restarts fail due to missing base devices
Looks like this can happen on VM guests and KVM vms, too. (could be somehow related to zfs ...) -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1623383 Title: Some restarts fail due to missing base devices Status in systemd package in Ubuntu: Confirmed Bug description: Arch: s390x Release: Yakkety / 16.10 This happens on some (but not all) system starts with Yakkety. In Xenial (which is using the same 4.4 kernel version the Yakkety systems were using when the problem was first observed) this did not happen. The system (LPAR) this was seen first was an upgrade from Xenial but since then has been freshly installed with Yakkety. The same behaviour is seen on a zVM guest running Yakkety. The attached syslog shows a failed boot, followed by one that did work. Note the "Found device .*(sclp|encc00).*" messages in the good boot. Those are missing in the bad attempt and as a result networking and console fail to be usable. Also note, those boots were 4.8 kernels but we saw this with 4.4 kernels, too. This might be a systemd problem / race, I just filed it into udev for now as that better matches the not finding basic devices symptom. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1623383/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1623383] Re: Some restarts fail due to missing base devices
** Package changed: udev (Ubuntu) => systemd (Ubuntu) -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to systemd in Ubuntu. https://bugs.launchpad.net/bugs/1623383 Title: Some restarts fail due to missing base devices Status in systemd package in Ubuntu: Confirmed Bug description: Arch: s390x Release: Yakkety / 16.10 This happens on some (but not all) system starts with Yakkety. In Xenial (which is using the same 4.4 kernel version the Yakkety systems were using when the problem was first observed) this did not happen. The system (LPAR) this was seen first was an upgrade from Xenial but since then has been freshly installed with Yakkety. The same behaviour is seen on a zVM guest running Yakkety. The attached syslog shows a failed boot, followed by one that did work. Note the "Found device .*(sclp|encc00).*" messages in the good boot. Those are missing in the bad attempt and as a result networking and console fail to be usable. Also note, those boots were 4.8 kernels but we saw this with 4.4 kernels, too. This might be a systemd problem / race, I just filed it into udev for now as that better matches the not finding basic devices symptom. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1623383/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
