Well, some significant progress! I think there are two problems. This dracut fix is definitely needed because udev rules are named differently once you start adding disk under SP4.
But I still had the problem last night on a dozen servers that hadn't received any new disks. The /run/initramfs/rdsosreport.txt showed me that the disks were indeed visible but LVM didn't find 3 of 6 of them. It seems to be related to LVM. I discovered in Dracut emergency shell that it forces you into, I could simply run 3 things rm /etc/lvm/lvm.conf lvm vgscan #cp ipl 101 clear And server happily proceeds on its merry way to boot grub and the rest of linux. Yes, the lvm.conf on disk is still the same and was not removed, but apparently the vgscan makes LVM happy again. Whatever this is about will probably come back.. Unless... Coincidentally, we have another ticket open with SUSE about LVM in SP4. The vgextend command wouldn't take /dev/disk/by-path/ccw-0.0.xxxx-part1 names. They said that was a filtering problem and so produced a PTF that doesn't take them for the other vg* commands as well (not the direction I wanted). Well, I took and applied that to five that looked just like these twelve that hadn't been patched yet and voila! No dracut timeout problem! -----Original Message----- From: Linux on 390 Port <[email protected]> On Behalf Of Juha Vuori Sent: Friday, July 26, 2019 2:48 PM To: [email protected] Subject: Re: [LINUX-390] Dracut fix is available now from SUSE for problem with SP4 dasd disks Not here, sorry. We've had the incident only on one south pole bird, whose life cycle is characterized by: - / on ckd mdisk, /opt /var /home on LVs of a VG (ckd PVs) - migration from sles11sp4 to sles12sp4; boots correctly at this stage - another PV added to the VG - /usr migrated to a new LV (due to space shortage in /) -> next boot hangs in the beginning, dracut warning about "/dev/system/usr does not exist" Udev rule for the new ckd was named "51-dasd-0.0.0206.rules". Still, SUSE workaroud fix was to edit parm of inst_rules_wildcard in module-setup.sh: 41-s390x-dasd-*.rules -> 41-dasd-*.rules. And that corrected the problem. Now we've applied the public patch on top, and everything is still fine. Honestly, I don't understand how the workaround could fix the problem, and I have no idea if the public patch really now prevents it. And I am not sure which preconditions caused the problem to pop up in the first place, but probably some of the above.. For us that's not too relevant because all out other servers have their system disk space on EDEV mdisks so that's why we haven't tried to investigate more.. -- Best regards, Juha Vuori On 26.7.2019 21.41, Ted Rodriguez-Bell wrote: > Has anyone been able to make this problem happen on a system of their choice? > We've seen it pop up in in our dev environment, but neither Marcy nor I has > been able to cause the problem when we've tried. Without that it's hard to > have complete confidence that this has the fix for our problem. > > Ted Rodriguez-Bell > Enterprise Virtualization, z/VM and z/Linux, Wells Fargo > > Company policy requires: This message may contain confidential and/or > privileged information. If you are not the addressee or authorized to > receive this for the addressee, you must not use, copy, disclose, or take any > action based on this message or any information herein. If you have received > this message in error, please advise the sender immediately by reply e-mail > and delete this message. Thank you for your cooperation. > > -----Original Message----- > From: Marcy Cortes <[email protected]> > Sent: Thursday, July 25, 2019 8:58 AM > Subject: Dracut fix is available now from SUSE for problem with SP4 dasd disks > > Bug Fix Advisory - SUSE-12-SP4-2019-1969 > > ------------------------------------------------------------------------------ > > Summary: > > Recommended update for dracut > > > > This update for dracut fixes the following issues: > > > > - 95dasd-rules 95zfcp-rules: was not correctly looking for rule names > (bsc#1137784) > > - Ensure early microcode gets added from files with .early postfix > > (bsc#1098915, bsc#1125393) > > - Ensure GPIO modules get included on ARM (bsc#1133819) > > - Fix Routes are not properly added due to spelling error (bsc#1134347) > > - Decouple iscsi from sysinit.target (bsc#1134472) > > - dracut-lib.sh:dev_unit_name() guard against $dev beginning with "-" > (bsc#1132448) > > - 95iscsi: avoid error messages when building initrd, multipath timeouts > > (bsc#1130114, bsc#1130107, bsc#1121238) > > > Marcy > > This message may contain confidential and/or privileged information. If you > are not the addressee or authorized to receive this for the addressee, you > must not use, copy, disclose, or take any action based on this message or any > information herein. If you have received this message in error, please advise > the sender immediately by reply e-mail and delete this message. Thank you for > your cooperation. > > > ---------------------------------------------------------------------- > For LINUX-390 subscribe / signoff / archive access instructions, > send email to [email protected] with the message: INFO LINUX-390 or visit > http://www2.marist.edu/htbin/wlvindex?LINUX-390 > > ---------------------------------------------------------------------- > For LINUX-390 subscribe / signoff / archive access instructions, > send email to [email protected] with the message: INFO LINUX-390 or visit > http://www2.marist.edu/htbin/wlvindex?LINUX-390 > ---------------------------------------------------------------------- For LINUX-390 subscribe / signoff / archive access instructions, send email to [email protected] with the message: INFO LINUX-390 or visit http://www2.marist.edu/htbin/wlvindex?LINUX-390 ---------------------------------------------------------------------- For LINUX-390 subscribe / signoff / archive access instructions, send email to [email protected] with the message: INFO LINUX-390 or visit http://www2.marist.edu/htbin/wlvindex?LINUX-390
