Re: [CentOS-virt] Xen CentOS 7.3 server + CentOS 7.3 VM fails to boot after CR updates (applied to VM)!

[Anderson, Dave]

I've been pretty happy using 7.3+CR/7.4 as PVHVM for the last week or two now, 
no complaints other than the sudden problem with PV taking me by surprise.

Has anyone tried with 7.4 as the dom0 yet? I saw the point in the 7.4 release 
notes about enough rebasing taking place that I probably shouldn't update my 
xen hypervisor machines yet...but wondering what the status is on that.

Thanks,
-Dave


> On Sep 14, 2017, at 4:00 AM, Adi Pircalabu  wrote:
> 
> On 14-09-2017 20:57, Adi Pircalabu wrote:
>> On 08-09-2017 6:17, Kevin Stange wrote:
>>> On 09/06/2017 05:21 PM, Kevin Stange wrote:
 On 09/06/2017 08:40 AM, Johnny Hughes wrote:
> On 09/05/2017 02:26 PM, Kevin Stange wrote:
>> On 09/04/2017 05:27 PM, Johnny Hughes wrote:
>>> On 09/04/2017 03:59 PM, Kevin Stange wrote:
 On 09/02/2017 08:11 AM, Johnny Hughes wrote:
> On 09/01/2017 02:41 PM, Kevin Stange wrote:
>> On 08/31/2017 07:50 AM, PJ Welsh wrote:
>>> A recently created and fully functional CentOS 7.3 VM fails to boot
>>> after applying CR updates:
>> 
>>> Server OS is CentOS 7.3 using Xen (no CR updates):
>>> rpm -qa xen\*
>>> xen-hypervisor-4.6.3-15.el7.x86_64
>>> xen-4.6.3-15.el7.x86_64
>>> xen-licenses-4.6.3-15.el7.x86_64
>>> xen-libs-4.6.3-15.el7.x86_64
>>> xen-runtime-4.6.3-15.el7.x86_64
>>> uname -a
>>> Linux tsxen2.xx.com 
>>> >>  > 4.9.39-29.el7.x86_64 #1 SMP
>>> Fri Jul 21 15:09:00 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
>>> Sadly, the other issue is that the grub menu will not display for 
>>> me to
>>> select another kernel to see if it is just a kernel issue.
>>> The dracut prompt does not show any /dev/disk folder either.
>> I'm seeing this as well.  My host is 4.9.44-29 and Xen 4.4.4-26 from
>> testing repo, my guest is 3.10.0-693.1.1.  Guest boots fine with
>> 514.26.2.  The kernel messages that appear to kick off the failure 
>> for
>> me start with a page allocation failure.  It eventually reaches 
>> dracut
>> failures due to systemd/udev not setting up properly, but I think the
>> root is this:
 
> Do any of you guys have access to RHEL to try the RHEL 7.4 Kernel?
 I think I may.  I haven't tried yet, but I'll see if I can get my hands
 on one and test it tomorrow when I'm back at the office tomorrow.
 RH closed my bug as "WONTFIX" so far, saying Red Hat Quality 
 Engineering
 Management declined the request.  I started to look at the Red Hat
 source browser to see the list of patches from 693 to 514, but getting
 the full list seems impossible because the change log only goes back to
 644 and there doesn't seem to be a way to obtain full builds of
 unreleased kernels.  Unless I'm mistaken.
 I will also do some digging via RH support if I can.
>>> I would think that RH would want AWS support for RHEL 7.4 and I thought
>>> AWS was run on Xen // Note:  I could be wrong about that.
>>> In any event, at the very least, we can make a kernel that boots PV for
>>> 7.4 at some point.
>> AWS does run on Xen, but the modifications they make to Xen are not
>> known to me nor which version of Xen they use.  They may also run the
>> domains as HVM, which seems to mitigate the issue here.
>> I just verified this kernel issue exists on a RHEL 7.3 system image
>> under the same conditions, when it's updated to RHEL 7.4 and kernel
>> 3.10.0-693.2.1.el7.x86_64.
> One other option is to run the DomU's as PVHVM:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.xen.org_wiki_Xen-5FLinux-5FPV-5Fon-5FHVM-5Fdrivers=DwICAg=C3yme8gMkxg_ihJNXS06ZyWk4EJm8LdrrvxQb-Je7sw=YXRZwMwEbTMJ1t85qnlbGKfbihJPB3W_h8JMF05uQhA=nL_mZmY1UFFXIyjRjNDnOYF72oaFqTb61_8qV-7trBA=O2qKd2kfvOIH5E9Ndt3WhhHlOdvQQseXJtTNtriIftg=
>  That should be much better performance than HVM and may be a workable
> solution for people who don't want to modify their VM kernel.
> Here is more info on PVHVM:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.xen.org_wiki_PV-5Fon-5FHVM=DwICAg=C3yme8gMkxg_ihJNXS06ZyWk4EJm8LdrrvxQb-Je7sw=YXRZwMwEbTMJ1t85qnlbGKfbihJPB3W_h8JMF05uQhA=nL_mZmY1UFFXIyjRjNDnOYF72oaFqTb61_8qV-7trBA=KEvkgBy5Xk4kxaQvwzaOy78t7rk2YrRT0Amziht84lc=
>  
> Also heard from someone to try this Config file change to the base
> kernel and rebuild:
> CONFIG_RANDOMIZE_BASE=n
 This suggestion was mirrored in the RH bugzilla as well, it worked, but
 the same issue does 

Re: [CentOS-virt] Xen CentOS 7.3 server + CentOS 7.3 VM fails to boot after CR updates (applied to VM)!

[PJ Welsh]

On Wed, Sep 6, 2017 at 8:40 AM, Johnny Hughes  wrote:

> On 09/05/2017 02:26 PM, Kevin Stange wrote:
> > On 09/04/2017 05:27 PM, Johnny Hughes wrote:
> >> On 09/04/2017 03:59 PM, Kevin Stange wrote:
> >>> On 09/02/2017 08:11 AM, Johnny Hughes wrote:
>  On 09/01/2017 02:41 PM, Kevin Stange wrote:
> > On 08/31/2017 07:50 AM, PJ Welsh wrote:
> >> A recently created and fully functional CentOS 7.3 VM fails to boot
> >> after applying CR updates:
> ...
> One other option is to run the DomU's as PVHVM:
> https://wiki.xen.org/wiki/Xen_Linux_PV_on_HVM_drivers
>
> That should be much better performance than HVM and may be a workable
> solution for people who don't want to modify their VM kernel.
>
> Here is more info on PVHVM:
> https://wiki.xen.org/wiki/PV_on_HVM
>
> 
> Also heard from someone to try this Config file change to the base
> kernel and rebuild:
>
> CONFIG_RANDOMIZE_BASE=n
> ...


The referenced link (https://wiki.xen.org/wiki/Xen_Linux_PV_on_HVM_drivers)
looks to be about adding to the older configuration style file. I do not
see the proper incantation for the " xen_platform_pci=1" in the domain xml
file syntax. I'm not even sure if I attempted it correctly. Is there a
portion of that page that pertains explicitly to booting? I think I missed
something.

Thanks
PJ
___
CentOS-virt mailing list
CentOS-virt@centos.org
https://lists.centos.org/mailman/listinfo/centos-virt

Re: [CentOS-virt] Xen CentOS 7.3 server + CentOS 7.3 VM fails to boot after CR updates (applied to VM)!

[Adi Pircalabu]

On 14-09-2017 20:57, Adi Pircalabu wrote:

On 08-09-2017 6:17, Kevin Stange wrote:

On 09/06/2017 05:21 PM, Kevin Stange wrote:

On 09/06/2017 08:40 AM, Johnny Hughes wrote:

On 09/05/2017 02:26 PM, Kevin Stange wrote:

On 09/04/2017 05:27 PM, Johnny Hughes wrote:

On 09/04/2017 03:59 PM, Kevin Stange wrote:

On 09/02/2017 08:11 AM, Johnny Hughes wrote:

On 09/01/2017 02:41 PM, Kevin Stange wrote:

On 08/31/2017 07:50 AM, PJ Welsh wrote:
A recently created and fully functional CentOS 7.3 VM fails to 
boot

after applying CR updates:



Server OS is CentOS 7.3 using Xen (no CR updates):
rpm -qa xen\*
xen-hypervisor-4.6.3-15.el7.x86_64
xen-4.6.3-15.el7.x86_64
xen-licenses-4.6.3-15.el7.x86_64
xen-libs-4.6.3-15.el7.x86_64
xen-runtime-4.6.3-15.el7.x86_64

uname -a
Linux tsxen2.xx.com  
4.9.39-29.el7.x86_64 #1 SMP

Fri Jul 21 15:09:00 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Sadly, the other issue is that the grub menu will not display 
for me to

select another kernel to see if it is just a kernel issue.

The dracut prompt does not show any /dev/disk folder either.



I'm seeing this as well.  My host is 4.9.44-29 and Xen 4.4.4-26 
from
testing repo, my guest is 3.10.0-693.1.1.  Guest boots fine 
with
514.26.2.  The kernel messages that appear to kick off the 
failure for
me start with a page allocation failure.  It eventually reaches 
dracut
failures due to systemd/udev not setting up properly, but I 
think the

root is this:





Do any of you guys have access to RHEL to try the RHEL 7.4 
Kernel?


I think I may.  I haven't tried yet, but I'll see if I can get my 
hands

on one and test it tomorrow when I'm back at the office tomorrow.

RH closed my bug as "WONTFIX" so far, saying Red Hat Quality 
Engineering
Management declined the request.  I started to look at the Red 
Hat
source browser to see the list of patches from 693 to 514, but 
getting
the full list seems impossible because the change log only goes 
back to

644 and there doesn't seem to be a way to obtain full builds of
unreleased kernels.  Unless I'm mistaken.

I will also do some digging via RH support if I can.

I would think that RH would want AWS support for RHEL 7.4 and I 
thought

AWS was run on Xen // Note:  I could be wrong about that.

In any event, at the very least, we can make a kernel that boots 
PV for

7.4 at some point.


AWS does run on Xen, but the modifications they make to Xen are not
known to me nor which version of Xen they use.  They may also run 
the

domains as HVM, which seems to mitigate the issue here.

I just verified this kernel issue exists on a RHEL 7.3 system image
under the same conditions, when it's updated to RHEL 7.4 and kernel
3.10.0-693.2.1.el7.x86_64.



One other option is to run the DomU's as PVHVM:
https://wiki.xen.org/wiki/Xen_Linux_PV_on_HVM_drivers

That should be much better performance than HVM and may be a 
workable

solution for people who don't want to modify their VM kernel.

Here is more info on PVHVM:
https://wiki.xen.org/wiki/PV_on_HVM


Also heard from someone to try this Config file change to the base
kernel and rebuild:

CONFIG_RANDOMIZE_BASE=n


This suggestion was mirrored in the RH bugzilla as well, it worked, 
but
the same issue does not exist in newer kernels which have the option 
on.
 I've posted updated findings in the CentOS bug, which includes a 
patch

that I found which seems to fix the issue:

https://bugs.centos.org/view.php?id=13763#c30014


With many thanks to hughesjr and toracat, I was able to find a patch
that seems to resolve this issue and get it into CentOS Plus
3.10.0-693.2.1.  I've asked Red Hat to apply it to some future kernel
update, but that is only a dream for now.

In the meantime, if anyone who has been experiencing the issue with PV
domains can try out the CentOS Plus kernel here and provide feedback,
I'd appreciate it!

https://buildlogs.centos.org/c7-plus/kernel-plus/20170907163005/3.10.0-693.2.1.el7.centos.plus.x86_64/


Loaded 3.10.0-693.2.2.el7.centos.plus.x86_64 successfully on two
CentOS 7.4 PV domUs which failed previously on
kernel-3.10.0-693.2.2.el7.x86_64, the 2 hypervisors tested are:
1. CentOS 6.9, kernel 4.9.13-22.el6.x86_64, Xen 4.6.3-8.el6
2. CentOS 7.3, kernel 4.9.31-27.el7.x86_64, Xen 4.9.31-27.el7.x86_64


Should read:
1. CentOS 6.9, kernel 4.9.13-22.el6.x86_64, Xen 4.6.3-8.el6
2. CentOS 7.3, kernel 4.9.31-27.el7.x86_64, Xen 4.6.3-15.el7

---
Adi Pircalabu, System Administrator
___
CentOS-virt mailing list
CentOS-virt@centos.org
https://lists.centos.org/mailman/listinfo/centos-virt

Re: [CentOS-virt] Xen CentOS 7.3 server + CentOS 7.3 VM fails to boot after CR updates (applied to VM)!

[Johnny Hughes]

On 09/07/2017 03:17 PM, Kevin Stange wrote:
> On 09/06/2017 05:21 PM, Kevin Stange wrote:
>> On 09/06/2017 08:40 AM, Johnny Hughes wrote:
>>> On 09/05/2017 02:26 PM, Kevin Stange wrote:
 On 09/04/2017 05:27 PM, Johnny Hughes wrote:
> On 09/04/2017 03:59 PM, Kevin Stange wrote:
>> On 09/02/2017 08:11 AM, Johnny Hughes wrote:
>>> On 09/01/2017 02:41 PM, Kevin Stange wrote:
 On 08/31/2017 07:50 AM, PJ Welsh wrote:
> A recently created and fully functional CentOS 7.3 VM fails to boot
> after applying CR updates:
 
> Server OS is CentOS 7.3 using Xen (no CR updates):
> rpm -qa xen\*
> xen-hypervisor-4.6.3-15.el7.x86_64
> xen-4.6.3-15.el7.x86_64
> xen-licenses-4.6.3-15.el7.x86_64
> xen-libs-4.6.3-15.el7.x86_64
> xen-runtime-4.6.3-15.el7.x86_64
>
> uname -a
> Linux tsxen2.xx.com  4.9.39-29.el7.x86_64 #1 SMP
> Fri Jul 21 15:09:00 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
>
> Sadly, the other issue is that the grub menu will not display for me 
> to
> select another kernel to see if it is just a kernel issue.
>
> The dracut prompt does not show any /dev/disk folder either.
>

 I'm seeing this as well.  My host is 4.9.44-29 and Xen 4.4.4-26 from
 testing repo, my guest is 3.10.0-693.1.1.  Guest boots fine with
 514.26.2.  The kernel messages that appear to kick off the failure for
 me start with a page allocation failure.  It eventually reaches dracut
 failures due to systemd/udev not setting up properly, but I think the
 root is this:

>> 
>>>
>>> Do any of you guys have access to RHEL to try the RHEL 7.4 Kernel?
>>
>> I think I may.  I haven't tried yet, but I'll see if I can get my hands
>> on one and test it tomorrow when I'm back at the office tomorrow.
>>
>> RH closed my bug as "WONTFIX" so far, saying Red Hat Quality Engineering
>> Management declined the request.  I started to look at the Red Hat
>> source browser to see the list of patches from 693 to 514, but getting
>> the full list seems impossible because the change log only goes back to
>> 644 and there doesn't seem to be a way to obtain full builds of
>> unreleased kernels.  Unless I'm mistaken.
>>
>> I will also do some digging via RH support if I can.
>>
> I would think that RH would want AWS support for RHEL 7.4 and I thought
> AWS was run on Xen // Note:  I could be wrong about that.
>
> In any event, at the very least, we can make a kernel that boots PV for
> 7.4 at some point.

 AWS does run on Xen, but the modifications they make to Xen are not
 known to me nor which version of Xen they use.  They may also run the
 domains as HVM, which seems to mitigate the issue here.

 I just verified this kernel issue exists on a RHEL 7.3 system image
 under the same conditions, when it's updated to RHEL 7.4 and kernel
 3.10.0-693.2.1.el7.x86_64.

>>>
>>> One other option is to run the DomU's as PVHVM:
>>> https://wiki.xen.org/wiki/Xen_Linux_PV_on_HVM_drivers
>>>
>>> That should be much better performance than HVM and may be a workable
>>> solution for people who don't want to modify their VM kernel.
>>>
>>> Here is more info on PVHVM:
>>> https://wiki.xen.org/wiki/PV_on_HVM
>>>
>>> 
>>> Also heard from someone to try this Config file change to the base
>>> kernel and rebuild:
>>>
>>> CONFIG_RANDOMIZE_BASE=n
>>
>> This suggestion was mirrored in the RH bugzilla as well, it worked, but
>> the same issue does not exist in newer kernels which have the option on.
>>  I've posted updated findings in the CentOS bug, which includes a patch
>> that I found which seems to fix the issue:
>>
>> https://bugs.centos.org/view.php?id=13763#c30014
> 
> With many thanks to hughesjr and toracat, I was able to find a patch
> that seems to resolve this issue and get it into CentOS Plus
> 3.10.0-693.2.1.  I've asked Red Hat to apply it to some future kernel
> update, but that is only a dream for now.
> 
> In the meantime, if anyone who has been experiencing the issue with PV
> domains can try out the CentOS Plus kernel here and provide feedback,
> I'd appreciate it!
> 
> https://buildlogs.centos.org/c7-plus/kernel-plus/20170907163005/3.10.0-693.2.1.el7.centos.plus.x86_64/
> 

I can verify that PV-on-HVM mode works with the regular kernel as well
if that is an option for you.  Many public clouds (like AWS) require HVM
or PV-on-HVM.

See these for more information on PV-on-HVM:

https://wiki.xen.org/wiki/PV_on_HVM

https://wiki.xen.org/wiki/Xen_Linux_PV_on_HVM_drivers



signature.asc
Description: OpenPGP digital signature
___
CentOS-virt mailing list
CentOS-virt@centos.org
https://lists.centos.org/mailman/listinfo/centos-virt

Re: [CentOS-virt] Xen CentOS 7.3 server + CentOS 7.3 VM fails to boot after CR updates (applied to VM)!

[Kevin Stange]

On 09/06/2017 05:21 PM, Kevin Stange wrote:
> On 09/06/2017 08:40 AM, Johnny Hughes wrote:
>> On 09/05/2017 02:26 PM, Kevin Stange wrote:
>>> On 09/04/2017 05:27 PM, Johnny Hughes wrote:
 On 09/04/2017 03:59 PM, Kevin Stange wrote:
> On 09/02/2017 08:11 AM, Johnny Hughes wrote:
>> On 09/01/2017 02:41 PM, Kevin Stange wrote:
>>> On 08/31/2017 07:50 AM, PJ Welsh wrote:
 A recently created and fully functional CentOS 7.3 VM fails to boot
 after applying CR updates:
>>> 
 Server OS is CentOS 7.3 using Xen (no CR updates):
 rpm -qa xen\*
 xen-hypervisor-4.6.3-15.el7.x86_64
 xen-4.6.3-15.el7.x86_64
 xen-licenses-4.6.3-15.el7.x86_64
 xen-libs-4.6.3-15.el7.x86_64
 xen-runtime-4.6.3-15.el7.x86_64

 uname -a
 Linux tsxen2.xx.com  4.9.39-29.el7.x86_64 #1 SMP
 Fri Jul 21 15:09:00 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

 Sadly, the other issue is that the grub menu will not display for me to
 select another kernel to see if it is just a kernel issue.

 The dracut prompt does not show any /dev/disk folder either.

>>>
>>> I'm seeing this as well.  My host is 4.9.44-29 and Xen 4.4.4-26 from
>>> testing repo, my guest is 3.10.0-693.1.1.  Guest boots fine with
>>> 514.26.2.  The kernel messages that appear to kick off the failure for
>>> me start with a page allocation failure.  It eventually reaches dracut
>>> failures due to systemd/udev not setting up properly, but I think the
>>> root is this:
>>>
> 
>>
>> Do any of you guys have access to RHEL to try the RHEL 7.4 Kernel?
>
> I think I may.  I haven't tried yet, but I'll see if I can get my hands
> on one and test it tomorrow when I'm back at the office tomorrow.
>
> RH closed my bug as "WONTFIX" so far, saying Red Hat Quality Engineering
> Management declined the request.  I started to look at the Red Hat
> source browser to see the list of patches from 693 to 514, but getting
> the full list seems impossible because the change log only goes back to
> 644 and there doesn't seem to be a way to obtain full builds of
> unreleased kernels.  Unless I'm mistaken.
>
> I will also do some digging via RH support if I can.
>
 I would think that RH would want AWS support for RHEL 7.4 and I thought
 AWS was run on Xen // Note:  I could be wrong about that.

 In any event, at the very least, we can make a kernel that boots PV for
 7.4 at some point.
>>>
>>> AWS does run on Xen, but the modifications they make to Xen are not
>>> known to me nor which version of Xen they use.  They may also run the
>>> domains as HVM, which seems to mitigate the issue here.
>>>
>>> I just verified this kernel issue exists on a RHEL 7.3 system image
>>> under the same conditions, when it's updated to RHEL 7.4 and kernel
>>> 3.10.0-693.2.1.el7.x86_64.
>>>
>>
>> One other option is to run the DomU's as PVHVM:
>> https://wiki.xen.org/wiki/Xen_Linux_PV_on_HVM_drivers
>>
>> That should be much better performance than HVM and may be a workable
>> solution for people who don't want to modify their VM kernel.
>>
>> Here is more info on PVHVM:
>> https://wiki.xen.org/wiki/PV_on_HVM
>>
>> 
>> Also heard from someone to try this Config file change to the base
>> kernel and rebuild:
>>
>> CONFIG_RANDOMIZE_BASE=n
> 
> This suggestion was mirrored in the RH bugzilla as well, it worked, but
> the same issue does not exist in newer kernels which have the option on.
>  I've posted updated findings in the CentOS bug, which includes a patch
> that I found which seems to fix the issue:
> 
> https://bugs.centos.org/view.php?id=13763#c30014

With many thanks to hughesjr and toracat, I was able to find a patch
that seems to resolve this issue and get it into CentOS Plus
3.10.0-693.2.1.  I've asked Red Hat to apply it to some future kernel
update, but that is only a dream for now.

In the meantime, if anyone who has been experiencing the issue with PV
domains can try out the CentOS Plus kernel here and provide feedback,
I'd appreciate it!

https://buildlogs.centos.org/c7-plus/kernel-plus/20170907163005/3.10.0-693.2.1.el7.centos.plus.x86_64/

-- 
Kevin Stange
Chief Technology Officer
Steadfast | Managed Infrastructure, Datacenter and Cloud Services
800 S Wells, Suite 190 | Chicago, IL 60607
312.602.2689 X203 | Fax: 312.602.2688
ke...@steadfast.net | www.steadfast.net
___
CentOS-virt mailing list
CentOS-virt@centos.org
https://lists.centos.org/mailman/listinfo/centos-virt

Re: [CentOS-virt] Xen CentOS 7.3 server + CentOS 7.3 VM fails to boot after CR updates (applied to VM)!

[Kevin Stange]

On 09/06/2017 08:40 AM, Johnny Hughes wrote:
> On 09/05/2017 02:26 PM, Kevin Stange wrote:
>> On 09/04/2017 05:27 PM, Johnny Hughes wrote:
>>> On 09/04/2017 03:59 PM, Kevin Stange wrote:
 On 09/02/2017 08:11 AM, Johnny Hughes wrote:
> On 09/01/2017 02:41 PM, Kevin Stange wrote:
>> On 08/31/2017 07:50 AM, PJ Welsh wrote:
>>> A recently created and fully functional CentOS 7.3 VM fails to boot
>>> after applying CR updates:
>> 
>>> Server OS is CentOS 7.3 using Xen (no CR updates):
>>> rpm -qa xen\*
>>> xen-hypervisor-4.6.3-15.el7.x86_64
>>> xen-4.6.3-15.el7.x86_64
>>> xen-licenses-4.6.3-15.el7.x86_64
>>> xen-libs-4.6.3-15.el7.x86_64
>>> xen-runtime-4.6.3-15.el7.x86_64
>>>
>>> uname -a
>>> Linux tsxen2.xx.com  4.9.39-29.el7.x86_64 #1 SMP
>>> Fri Jul 21 15:09:00 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
>>>
>>> Sadly, the other issue is that the grub menu will not display for me to
>>> select another kernel to see if it is just a kernel issue.
>>>
>>> The dracut prompt does not show any /dev/disk folder either.
>>>
>>
>> I'm seeing this as well.  My host is 4.9.44-29 and Xen 4.4.4-26 from
>> testing repo, my guest is 3.10.0-693.1.1.  Guest boots fine with
>> 514.26.2.  The kernel messages that appear to kick off the failure for
>> me start with a page allocation failure.  It eventually reaches dracut
>> failures due to systemd/udev not setting up properly, but I think the
>> root is this:
>>

>
> Do any of you guys have access to RHEL to try the RHEL 7.4 Kernel?

 I think I may.  I haven't tried yet, but I'll see if I can get my hands
 on one and test it tomorrow when I'm back at the office tomorrow.

 RH closed my bug as "WONTFIX" so far, saying Red Hat Quality Engineering
 Management declined the request.  I started to look at the Red Hat
 source browser to see the list of patches from 693 to 514, but getting
 the full list seems impossible because the change log only goes back to
 644 and there doesn't seem to be a way to obtain full builds of
 unreleased kernels.  Unless I'm mistaken.

 I will also do some digging via RH support if I can.

>>> I would think that RH would want AWS support for RHEL 7.4 and I thought
>>> AWS was run on Xen // Note:  I could be wrong about that.
>>>
>>> In any event, at the very least, we can make a kernel that boots PV for
>>> 7.4 at some point.
>>
>> AWS does run on Xen, but the modifications they make to Xen are not
>> known to me nor which version of Xen they use.  They may also run the
>> domains as HVM, which seems to mitigate the issue here.
>>
>> I just verified this kernel issue exists on a RHEL 7.3 system image
>> under the same conditions, when it's updated to RHEL 7.4 and kernel
>> 3.10.0-693.2.1.el7.x86_64.
>>
> 
> One other option is to run the DomU's as PVHVM:
> https://wiki.xen.org/wiki/Xen_Linux_PV_on_HVM_drivers
> 
> That should be much better performance than HVM and may be a workable
> solution for people who don't want to modify their VM kernel.
> 
> Here is more info on PVHVM:
> https://wiki.xen.org/wiki/PV_on_HVM
> 
> 
> Also heard from someone to try this Config file change to the base
> kernel and rebuild:
> 
> CONFIG_RANDOMIZE_BASE=n

This suggestion was mirrored in the RH bugzilla as well, it worked, but
the same issue does not exist in newer kernels which have the option on.
 I've posted updated findings in the CentOS bug, which includes a patch
that I found which seems to fix the issue:

https://bugs.centos.org/view.php?id=13763#c30014

-- 
Kevin Stange
Chief Technology Officer
Steadfast | Managed Infrastructure, Datacenter and Cloud Services
800 S Wells, Suite 190 | Chicago, IL 60607
312.602.2689 X203 | Fax: 312.602.2688
ke...@steadfast.net | www.steadfast.net
___
CentOS-virt mailing list
CentOS-virt@centos.org
https://lists.centos.org/mailman/listinfo/centos-virt

Re: [CentOS-virt] Xen CentOS 7.3 server + CentOS 7.3 VM fails to boot after CR updates (applied to VM)!

[Johnny Hughes]

On 09/05/2017 02:26 PM, Kevin Stange wrote:
> On 09/04/2017 05:27 PM, Johnny Hughes wrote:
>> On 09/04/2017 03:59 PM, Kevin Stange wrote:
>>> On 09/02/2017 08:11 AM, Johnny Hughes wrote:
 On 09/01/2017 02:41 PM, Kevin Stange wrote:
> On 08/31/2017 07:50 AM, PJ Welsh wrote:
>> A recently created and fully functional CentOS 7.3 VM fails to boot
>> after applying CR updates:
> 
>> Server OS is CentOS 7.3 using Xen (no CR updates):
>> rpm -qa xen\*
>> xen-hypervisor-4.6.3-15.el7.x86_64
>> xen-4.6.3-15.el7.x86_64
>> xen-licenses-4.6.3-15.el7.x86_64
>> xen-libs-4.6.3-15.el7.x86_64
>> xen-runtime-4.6.3-15.el7.x86_64
>>
>> uname -a
>> Linux tsxen2.xx.com  4.9.39-29.el7.x86_64 #1 SMP
>> Fri Jul 21 15:09:00 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
>>
>> Sadly, the other issue is that the grub menu will not display for me to
>> select another kernel to see if it is just a kernel issue.
>>
>> The dracut prompt does not show any /dev/disk folder either.
>>
>
> I'm seeing this as well.  My host is 4.9.44-29 and Xen 4.4.4-26 from
> testing repo, my guest is 3.10.0-693.1.1.  Guest boots fine with
> 514.26.2.  The kernel messages that appear to kick off the failure for
> me start with a page allocation failure.  It eventually reaches dracut
> failures due to systemd/udev not setting up properly, but I think the
> root is this:
>
> [1.970630] [ cut here ]
> [1.970651] WARNING: CPU: 2 PID: 225 at mm/vmalloc.c:131
> vmap_page_range_noflush+0x2c1/0x350
> [1.970660] Modules linked in:
> [1.970668] CPU: 2 PID: 225 Comm: systemd-udevd Not tainted
> 3.10.0-693.1.1.el7.x86_64 #1
> [1.970677]   8cddc75d 8803e8587bd8
> 816a3d91
> [1.970688]  8803e8587c18 810879c8 0083811c14e8
> 8800066eb000
> [1.970698]  0001 8803e86d6940 c000
> 
> [1.970708] Call Trace:
> [1.970725]  [] dump_stack+0x19/0x1b
> [1.970736]  [] __warn+0xd8/0x100
> [1.970742]  [] warn_slowpath_null+0x1d/0x20
> [1.970748]  [] vmap_page_range_noflush+0x2c1/0x350
> [1.970758]  [] map_vm_area+0x2e/0x40
> [1.970765]  [] __vmalloc_node_range+0x170/0x270
> [1.970774]  [] ? 
> module_alloc_update_bounds+0x14/0x70
> [1.970781]  [] ? 
> module_alloc_update_bounds+0x14/0x70
> [1.970792]  [] module_alloc+0x73/0xd0
> [1.970798]  [] ? 
> module_alloc_update_bounds+0x14/0x70
> [1.970804]  [] module_alloc_update_bounds+0x14/0x70
> [1.970811]  [] load_module+0xb02/0x29e0
> [1.970817]  [] ? vmap_page_range_noflush+0x257/0x350
> [1.970823]  [] ? map_vm_area+0x2e/0x40
> [1.970829]  [] ? __vmalloc_node_range+0x170/0x270
> [1.970838]  [] ? SyS_init_module+0x99/0x110
> [1.970846]  [] SyS_init_module+0xc5/0x110
> [1.970856]  [] system_call_fastpath+0x16/0x1b
> [1.970862] ---[ end trace 2117480876ed90d2 ]---
> [1.970869] vmalloc: allocation failure, allocated 24576 of 28672 bytes
> [1.970874] systemd-udevd: page allocation failure: order:0, mode:0xd2
> [1.970883] CPU: 2 PID: 225 Comm: systemd-udevd Tainted: GW
>   3.10.0-693.1.1.el7.x86_64 #1
> [1.970894]  00d2 8cddc75d 8803e8587c48
> 816a3d91
> [1.970910]  8803e8587cd8 81188810 8190ea38
> 8803e8587c68
> [1.970923]  0018 8803e8587ce8 8803e8587c88
> 8cddc75d
> [1.970939] Call Trace:
> [1.970946]  [] dump_stack+0x19/0x1b
> [1.970961]  [] warn_alloc_failed+0x110/0x180
> [1.970971]  [] __vmalloc_node_range+0x234/0x270
> [1.970981]  [] ? 
> module_alloc_update_bounds+0x14/0x70
> [1.970989]  [] ? 
> module_alloc_update_bounds+0x14/0x70
> [1.970999]  [] module_alloc+0x73/0xd0
> [1.971031]  [] ? 
> module_alloc_update_bounds+0x14/0x70
> [1.971038]  [] module_alloc_update_bounds+0x14/0x70
> [1.971046]  [] load_module+0xb02/0x29e0
> [1.971052]  [] ? vmap_page_range_noflush+0x257/0x350
> [1.971061]  [] ? map_vm_area+0x2e/0x40
> [1.971067]  [] ? __vmalloc_node_range+0x170/0x270
> [1.971075]  [] ? SyS_init_module+0x99/0x110
> [1.971081]  [] SyS_init_module+0xc5/0x110
> [1.971088]  [] system_call_fastpath+0x16/0x1b
> [1.971094] Mem-Info:
> [1.971103] active_anon:875 inactive_anon:2049 isolated_anon:0
> [1.971103]  active_file:791 inactive_file:8841 isolated_file:0
> [1.971103]  unevictable:0 dirty:0 writeback:0 unstable:0
> [1.971103]  slab_reclaimable:1732 slab_unreclaimable:1629
> [1.971103]  

Re: [CentOS-virt] Xen CentOS 7.3 server + CentOS 7.3 VM fails to boot after CR updates (applied to VM)!

[Kevin Stange]

On 09/04/2017 05:27 PM, Johnny Hughes wrote:
> On 09/04/2017 03:59 PM, Kevin Stange wrote:
>> On 09/02/2017 08:11 AM, Johnny Hughes wrote:
>>> On 09/01/2017 02:41 PM, Kevin Stange wrote:
 On 08/31/2017 07:50 AM, PJ Welsh wrote:
> A recently created and fully functional CentOS 7.3 VM fails to boot
> after applying CR updates:
 
> Server OS is CentOS 7.3 using Xen (no CR updates):
> rpm -qa xen\*
> xen-hypervisor-4.6.3-15.el7.x86_64
> xen-4.6.3-15.el7.x86_64
> xen-licenses-4.6.3-15.el7.x86_64
> xen-libs-4.6.3-15.el7.x86_64
> xen-runtime-4.6.3-15.el7.x86_64
>
> uname -a
> Linux tsxen2.xx.com  4.9.39-29.el7.x86_64 #1 SMP
> Fri Jul 21 15:09:00 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
>
> Sadly, the other issue is that the grub menu will not display for me to
> select another kernel to see if it is just a kernel issue.
>
> The dracut prompt does not show any /dev/disk folder either.
>

 I'm seeing this as well.  My host is 4.9.44-29 and Xen 4.4.4-26 from
 testing repo, my guest is 3.10.0-693.1.1.  Guest boots fine with
 514.26.2.  The kernel messages that appear to kick off the failure for
 me start with a page allocation failure.  It eventually reaches dracut
 failures due to systemd/udev not setting up properly, but I think the
 root is this:

 [1.970630] [ cut here ]
 [1.970651] WARNING: CPU: 2 PID: 225 at mm/vmalloc.c:131
 vmap_page_range_noflush+0x2c1/0x350
 [1.970660] Modules linked in:
 [1.970668] CPU: 2 PID: 225 Comm: systemd-udevd Not tainted
 3.10.0-693.1.1.el7.x86_64 #1
 [1.970677]   8cddc75d 8803e8587bd8
 816a3d91
 [1.970688]  8803e8587c18 810879c8 0083811c14e8
 8800066eb000
 [1.970698]  0001 8803e86d6940 c000
 
 [1.970708] Call Trace:
 [1.970725]  [] dump_stack+0x19/0x1b
 [1.970736]  [] __warn+0xd8/0x100
 [1.970742]  [] warn_slowpath_null+0x1d/0x20
 [1.970748]  [] vmap_page_range_noflush+0x2c1/0x350
 [1.970758]  [] map_vm_area+0x2e/0x40
 [1.970765]  [] __vmalloc_node_range+0x170/0x270
 [1.970774]  [] ? module_alloc_update_bounds+0x14/0x70
 [1.970781]  [] ? module_alloc_update_bounds+0x14/0x70
 [1.970792]  [] module_alloc+0x73/0xd0
 [1.970798]  [] ? module_alloc_update_bounds+0x14/0x70
 [1.970804]  [] module_alloc_update_bounds+0x14/0x70
 [1.970811]  [] load_module+0xb02/0x29e0
 [1.970817]  [] ? vmap_page_range_noflush+0x257/0x350
 [1.970823]  [] ? map_vm_area+0x2e/0x40
 [1.970829]  [] ? __vmalloc_node_range+0x170/0x270
 [1.970838]  [] ? SyS_init_module+0x99/0x110
 [1.970846]  [] SyS_init_module+0xc5/0x110
 [1.970856]  [] system_call_fastpath+0x16/0x1b
 [1.970862] ---[ end trace 2117480876ed90d2 ]---
 [1.970869] vmalloc: allocation failure, allocated 24576 of 28672 bytes
 [1.970874] systemd-udevd: page allocation failure: order:0, mode:0xd2
 [1.970883] CPU: 2 PID: 225 Comm: systemd-udevd Tainted: GW
   3.10.0-693.1.1.el7.x86_64 #1
 [1.970894]  00d2 8cddc75d 8803e8587c48
 816a3d91
 [1.970910]  8803e8587cd8 81188810 8190ea38
 8803e8587c68
 [1.970923]  0018 8803e8587ce8 8803e8587c88
 8cddc75d
 [1.970939] Call Trace:
 [1.970946]  [] dump_stack+0x19/0x1b
 [1.970961]  [] warn_alloc_failed+0x110/0x180
 [1.970971]  [] __vmalloc_node_range+0x234/0x270
 [1.970981]  [] ? module_alloc_update_bounds+0x14/0x70
 [1.970989]  [] ? module_alloc_update_bounds+0x14/0x70
 [1.970999]  [] module_alloc+0x73/0xd0
 [1.971031]  [] ? module_alloc_update_bounds+0x14/0x70
 [1.971038]  [] module_alloc_update_bounds+0x14/0x70
 [1.971046]  [] load_module+0xb02/0x29e0
 [1.971052]  [] ? vmap_page_range_noflush+0x257/0x350
 [1.971061]  [] ? map_vm_area+0x2e/0x40
 [1.971067]  [] ? __vmalloc_node_range+0x170/0x270
 [1.971075]  [] ? SyS_init_module+0x99/0x110
 [1.971081]  [] SyS_init_module+0xc5/0x110
 [1.971088]  [] system_call_fastpath+0x16/0x1b
 [1.971094] Mem-Info:
 [1.971103] active_anon:875 inactive_anon:2049 isolated_anon:0
 [1.971103]  active_file:791 inactive_file:8841 isolated_file:0
 [1.971103]  unevictable:0 dirty:0 writeback:0 unstable:0
 [1.971103]  slab_reclaimable:1732 slab_unreclaimable:1629
 [1.971103]  mapped:1464 shmem:2053 pagetables:480 bounce:0
 [1.971103]  free:4065966 free_pcp:763 free_cma:0
 [1.971131] Node 0 DMA free:15912kB min:12kB low:12kB high:16kB
 

Re: [CentOS-virt] Xen CentOS 7.3 server + CentOS 7.3 VM fails to boot after CR updates (applied to VM)!

[Johnny Hughes]

On 09/04/2017 03:59 PM, Kevin Stange wrote:
> On 09/02/2017 08:11 AM, Johnny Hughes wrote:
>> On 09/01/2017 02:41 PM, Kevin Stange wrote:
>>> On 08/31/2017 07:50 AM, PJ Welsh wrote:
 A recently created and fully functional CentOS 7.3 VM fails to boot
 after applying CR updates:
>>> 
 Server OS is CentOS 7.3 using Xen (no CR updates):
 rpm -qa xen\*
 xen-hypervisor-4.6.3-15.el7.x86_64
 xen-4.6.3-15.el7.x86_64
 xen-licenses-4.6.3-15.el7.x86_64
 xen-libs-4.6.3-15.el7.x86_64
 xen-runtime-4.6.3-15.el7.x86_64

 uname -a
 Linux tsxen2.xx.com  4.9.39-29.el7.x86_64 #1 SMP
 Fri Jul 21 15:09:00 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

 Sadly, the other issue is that the grub menu will not display for me to
 select another kernel to see if it is just a kernel issue.

 The dracut prompt does not show any /dev/disk folder either.

>>>
>>> I'm seeing this as well.  My host is 4.9.44-29 and Xen 4.4.4-26 from
>>> testing repo, my guest is 3.10.0-693.1.1.  Guest boots fine with
>>> 514.26.2.  The kernel messages that appear to kick off the failure for
>>> me start with a page allocation failure.  It eventually reaches dracut
>>> failures due to systemd/udev not setting up properly, but I think the
>>> root is this:
>>>
>>> [1.970630] [ cut here ]
>>> [1.970651] WARNING: CPU: 2 PID: 225 at mm/vmalloc.c:131
>>> vmap_page_range_noflush+0x2c1/0x350
>>> [1.970660] Modules linked in:
>>> [1.970668] CPU: 2 PID: 225 Comm: systemd-udevd Not tainted
>>> 3.10.0-693.1.1.el7.x86_64 #1
>>> [1.970677]   8cddc75d 8803e8587bd8
>>> 816a3d91
>>> [1.970688]  8803e8587c18 810879c8 0083811c14e8
>>> 8800066eb000
>>> [1.970698]  0001 8803e86d6940 c000
>>> 
>>> [1.970708] Call Trace:
>>> [1.970725]  [] dump_stack+0x19/0x1b
>>> [1.970736]  [] __warn+0xd8/0x100
>>> [1.970742]  [] warn_slowpath_null+0x1d/0x20
>>> [1.970748]  [] vmap_page_range_noflush+0x2c1/0x350
>>> [1.970758]  [] map_vm_area+0x2e/0x40
>>> [1.970765]  [] __vmalloc_node_range+0x170/0x270
>>> [1.970774]  [] ? module_alloc_update_bounds+0x14/0x70
>>> [1.970781]  [] ? module_alloc_update_bounds+0x14/0x70
>>> [1.970792]  [] module_alloc+0x73/0xd0
>>> [1.970798]  [] ? module_alloc_update_bounds+0x14/0x70
>>> [1.970804]  [] module_alloc_update_bounds+0x14/0x70
>>> [1.970811]  [] load_module+0xb02/0x29e0
>>> [1.970817]  [] ? vmap_page_range_noflush+0x257/0x350
>>> [1.970823]  [] ? map_vm_area+0x2e/0x40
>>> [1.970829]  [] ? __vmalloc_node_range+0x170/0x270
>>> [1.970838]  [] ? SyS_init_module+0x99/0x110
>>> [1.970846]  [] SyS_init_module+0xc5/0x110
>>> [1.970856]  [] system_call_fastpath+0x16/0x1b
>>> [1.970862] ---[ end trace 2117480876ed90d2 ]---
>>> [1.970869] vmalloc: allocation failure, allocated 24576 of 28672 bytes
>>> [1.970874] systemd-udevd: page allocation failure: order:0, mode:0xd2
>>> [1.970883] CPU: 2 PID: 225 Comm: systemd-udevd Tainted: GW
>>>   3.10.0-693.1.1.el7.x86_64 #1
>>> [1.970894]  00d2 8cddc75d 8803e8587c48
>>> 816a3d91
>>> [1.970910]  8803e8587cd8 81188810 8190ea38
>>> 8803e8587c68
>>> [1.970923]  0018 8803e8587ce8 8803e8587c88
>>> 8cddc75d
>>> [1.970939] Call Trace:
>>> [1.970946]  [] dump_stack+0x19/0x1b
>>> [1.970961]  [] warn_alloc_failed+0x110/0x180
>>> [1.970971]  [] __vmalloc_node_range+0x234/0x270
>>> [1.970981]  [] ? module_alloc_update_bounds+0x14/0x70
>>> [1.970989]  [] ? module_alloc_update_bounds+0x14/0x70
>>> [1.970999]  [] module_alloc+0x73/0xd0
>>> [1.971031]  [] ? module_alloc_update_bounds+0x14/0x70
>>> [1.971038]  [] module_alloc_update_bounds+0x14/0x70
>>> [1.971046]  [] load_module+0xb02/0x29e0
>>> [1.971052]  [] ? vmap_page_range_noflush+0x257/0x350
>>> [1.971061]  [] ? map_vm_area+0x2e/0x40
>>> [1.971067]  [] ? __vmalloc_node_range+0x170/0x270
>>> [1.971075]  [] ? SyS_init_module+0x99/0x110
>>> [1.971081]  [] SyS_init_module+0xc5/0x110
>>> [1.971088]  [] system_call_fastpath+0x16/0x1b
>>> [1.971094] Mem-Info:
>>> [1.971103] active_anon:875 inactive_anon:2049 isolated_anon:0
>>> [1.971103]  active_file:791 inactive_file:8841 isolated_file:0
>>> [1.971103]  unevictable:0 dirty:0 writeback:0 unstable:0
>>> [1.971103]  slab_reclaimable:1732 slab_unreclaimable:1629
>>> [1.971103]  mapped:1464 shmem:2053 pagetables:480 bounce:0
>>> [1.971103]  free:4065966 free_pcp:763 free_cma:0
>>> [1.971131] Node 0 DMA free:15912kB min:12kB low:12kB high:16kB
>>> active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
>>> unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15996kB
>>> 

Re: [CentOS-virt] Xen CentOS 7.3 server + CentOS 7.3 VM fails to boot after CR updates (applied to VM)!

[Kevin Stange]

On 09/02/2017 08:11 AM, Johnny Hughes wrote:
> On 09/01/2017 02:41 PM, Kevin Stange wrote:
>> On 08/31/2017 07:50 AM, PJ Welsh wrote:
>>> A recently created and fully functional CentOS 7.3 VM fails to boot
>>> after applying CR updates:
>> 
>>> Server OS is CentOS 7.3 using Xen (no CR updates):
>>> rpm -qa xen\*
>>> xen-hypervisor-4.6.3-15.el7.x86_64
>>> xen-4.6.3-15.el7.x86_64
>>> xen-licenses-4.6.3-15.el7.x86_64
>>> xen-libs-4.6.3-15.el7.x86_64
>>> xen-runtime-4.6.3-15.el7.x86_64
>>>
>>> uname -a
>>> Linux tsxen2.xx.com  4.9.39-29.el7.x86_64 #1 SMP
>>> Fri Jul 21 15:09:00 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
>>>
>>> Sadly, the other issue is that the grub menu will not display for me to
>>> select another kernel to see if it is just a kernel issue.
>>>
>>> The dracut prompt does not show any /dev/disk folder either.
>>>
>>
>> I'm seeing this as well.  My host is 4.9.44-29 and Xen 4.4.4-26 from
>> testing repo, my guest is 3.10.0-693.1.1.  Guest boots fine with
>> 514.26.2.  The kernel messages that appear to kick off the failure for
>> me start with a page allocation failure.  It eventually reaches dracut
>> failures due to systemd/udev not setting up properly, but I think the
>> root is this:
>>
>> [1.970630] [ cut here ]
>> [1.970651] WARNING: CPU: 2 PID: 225 at mm/vmalloc.c:131
>> vmap_page_range_noflush+0x2c1/0x350
>> [1.970660] Modules linked in:
>> [1.970668] CPU: 2 PID: 225 Comm: systemd-udevd Not tainted
>> 3.10.0-693.1.1.el7.x86_64 #1
>> [1.970677]   8cddc75d 8803e8587bd8
>> 816a3d91
>> [1.970688]  8803e8587c18 810879c8 0083811c14e8
>> 8800066eb000
>> [1.970698]  0001 8803e86d6940 c000
>> 
>> [1.970708] Call Trace:
>> [1.970725]  [] dump_stack+0x19/0x1b
>> [1.970736]  [] __warn+0xd8/0x100
>> [1.970742]  [] warn_slowpath_null+0x1d/0x20
>> [1.970748]  [] vmap_page_range_noflush+0x2c1/0x350
>> [1.970758]  [] map_vm_area+0x2e/0x40
>> [1.970765]  [] __vmalloc_node_range+0x170/0x270
>> [1.970774]  [] ? module_alloc_update_bounds+0x14/0x70
>> [1.970781]  [] ? module_alloc_update_bounds+0x14/0x70
>> [1.970792]  [] module_alloc+0x73/0xd0
>> [1.970798]  [] ? module_alloc_update_bounds+0x14/0x70
>> [1.970804]  [] module_alloc_update_bounds+0x14/0x70
>> [1.970811]  [] load_module+0xb02/0x29e0
>> [1.970817]  [] ? vmap_page_range_noflush+0x257/0x350
>> [1.970823]  [] ? map_vm_area+0x2e/0x40
>> [1.970829]  [] ? __vmalloc_node_range+0x170/0x270
>> [1.970838]  [] ? SyS_init_module+0x99/0x110
>> [1.970846]  [] SyS_init_module+0xc5/0x110
>> [1.970856]  [] system_call_fastpath+0x16/0x1b
>> [1.970862] ---[ end trace 2117480876ed90d2 ]---
>> [1.970869] vmalloc: allocation failure, allocated 24576 of 28672 bytes
>> [1.970874] systemd-udevd: page allocation failure: order:0, mode:0xd2
>> [1.970883] CPU: 2 PID: 225 Comm: systemd-udevd Tainted: GW
>>   3.10.0-693.1.1.el7.x86_64 #1
>> [1.970894]  00d2 8cddc75d 8803e8587c48
>> 816a3d91
>> [1.970910]  8803e8587cd8 81188810 8190ea38
>> 8803e8587c68
>> [1.970923]  0018 8803e8587ce8 8803e8587c88
>> 8cddc75d
>> [1.970939] Call Trace:
>> [1.970946]  [] dump_stack+0x19/0x1b
>> [1.970961]  [] warn_alloc_failed+0x110/0x180
>> [1.970971]  [] __vmalloc_node_range+0x234/0x270
>> [1.970981]  [] ? module_alloc_update_bounds+0x14/0x70
>> [1.970989]  [] ? module_alloc_update_bounds+0x14/0x70
>> [1.970999]  [] module_alloc+0x73/0xd0
>> [1.971031]  [] ? module_alloc_update_bounds+0x14/0x70
>> [1.971038]  [] module_alloc_update_bounds+0x14/0x70
>> [1.971046]  [] load_module+0xb02/0x29e0
>> [1.971052]  [] ? vmap_page_range_noflush+0x257/0x350
>> [1.971061]  [] ? map_vm_area+0x2e/0x40
>> [1.971067]  [] ? __vmalloc_node_range+0x170/0x270
>> [1.971075]  [] ? SyS_init_module+0x99/0x110
>> [1.971081]  [] SyS_init_module+0xc5/0x110
>> [1.971088]  [] system_call_fastpath+0x16/0x1b
>> [1.971094] Mem-Info:
>> [1.971103] active_anon:875 inactive_anon:2049 isolated_anon:0
>> [1.971103]  active_file:791 inactive_file:8841 isolated_file:0
>> [1.971103]  unevictable:0 dirty:0 writeback:0 unstable:0
>> [1.971103]  slab_reclaimable:1732 slab_unreclaimable:1629
>> [1.971103]  mapped:1464 shmem:2053 pagetables:480 bounce:0
>> [1.971103]  free:4065966 free_pcp:763 free_cma:0
>> [1.971131] Node 0 DMA free:15912kB min:12kB low:12kB high:16kB
>> active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
>> unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15996kB
>> managed:15912kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB
>> slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB
>> 

Re: [CentOS-virt] Xen CentOS 7.3 server + CentOS 7.3 VM fails to boot after CR updates (applied to VM)!

[Johnny Hughes]

On 09/01/2017 02:41 PM, Kevin Stange wrote:
> On 08/31/2017 07:50 AM, PJ Welsh wrote:
>> A recently created and fully functional CentOS 7.3 VM fails to boot
>> after applying CR updates:
> 
>> Server OS is CentOS 7.3 using Xen (no CR updates):
>> rpm -qa xen\*
>> xen-hypervisor-4.6.3-15.el7.x86_64
>> xen-4.6.3-15.el7.x86_64
>> xen-licenses-4.6.3-15.el7.x86_64
>> xen-libs-4.6.3-15.el7.x86_64
>> xen-runtime-4.6.3-15.el7.x86_64
>>
>> uname -a
>> Linux tsxen2.xx.com  4.9.39-29.el7.x86_64 #1 SMP
>> Fri Jul 21 15:09:00 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
>>
>> Sadly, the other issue is that the grub menu will not display for me to
>> select another kernel to see if it is just a kernel issue.
>>
>> The dracut prompt does not show any /dev/disk folder either.
>>
> 
> I'm seeing this as well.  My host is 4.9.44-29 and Xen 4.4.4-26 from
> testing repo, my guest is 3.10.0-693.1.1.  Guest boots fine with
> 514.26.2.  The kernel messages that appear to kick off the failure for
> me start with a page allocation failure.  It eventually reaches dracut
> failures due to systemd/udev not setting up properly, but I think the
> root is this:
> 
> [1.970630] [ cut here ]
> [1.970651] WARNING: CPU: 2 PID: 225 at mm/vmalloc.c:131
> vmap_page_range_noflush+0x2c1/0x350
> [1.970660] Modules linked in:
> [1.970668] CPU: 2 PID: 225 Comm: systemd-udevd Not tainted
> 3.10.0-693.1.1.el7.x86_64 #1
> [1.970677]   8cddc75d 8803e8587bd8
> 816a3d91
> [1.970688]  8803e8587c18 810879c8 0083811c14e8
> 8800066eb000
> [1.970698]  0001 8803e86d6940 c000
> 
> [1.970708] Call Trace:
> [1.970725]  [] dump_stack+0x19/0x1b
> [1.970736]  [] __warn+0xd8/0x100
> [1.970742]  [] warn_slowpath_null+0x1d/0x20
> [1.970748]  [] vmap_page_range_noflush+0x2c1/0x350
> [1.970758]  [] map_vm_area+0x2e/0x40
> [1.970765]  [] __vmalloc_node_range+0x170/0x270
> [1.970774]  [] ? module_alloc_update_bounds+0x14/0x70
> [1.970781]  [] ? module_alloc_update_bounds+0x14/0x70
> [1.970792]  [] module_alloc+0x73/0xd0
> [1.970798]  [] ? module_alloc_update_bounds+0x14/0x70
> [1.970804]  [] module_alloc_update_bounds+0x14/0x70
> [1.970811]  [] load_module+0xb02/0x29e0
> [1.970817]  [] ? vmap_page_range_noflush+0x257/0x350
> [1.970823]  [] ? map_vm_area+0x2e/0x40
> [1.970829]  [] ? __vmalloc_node_range+0x170/0x270
> [1.970838]  [] ? SyS_init_module+0x99/0x110
> [1.970846]  [] SyS_init_module+0xc5/0x110
> [1.970856]  [] system_call_fastpath+0x16/0x1b
> [1.970862] ---[ end trace 2117480876ed90d2 ]---
> [1.970869] vmalloc: allocation failure, allocated 24576 of 28672 bytes
> [1.970874] systemd-udevd: page allocation failure: order:0, mode:0xd2
> [1.970883] CPU: 2 PID: 225 Comm: systemd-udevd Tainted: GW
>   3.10.0-693.1.1.el7.x86_64 #1
> [1.970894]  00d2 8cddc75d 8803e8587c48
> 816a3d91
> [1.970910]  8803e8587cd8 81188810 8190ea38
> 8803e8587c68
> [1.970923]  0018 8803e8587ce8 8803e8587c88
> 8cddc75d
> [1.970939] Call Trace:
> [1.970946]  [] dump_stack+0x19/0x1b
> [1.970961]  [] warn_alloc_failed+0x110/0x180
> [1.970971]  [] __vmalloc_node_range+0x234/0x270
> [1.970981]  [] ? module_alloc_update_bounds+0x14/0x70
> [1.970989]  [] ? module_alloc_update_bounds+0x14/0x70
> [1.970999]  [] module_alloc+0x73/0xd0
> [1.971031]  [] ? module_alloc_update_bounds+0x14/0x70
> [1.971038]  [] module_alloc_update_bounds+0x14/0x70
> [1.971046]  [] load_module+0xb02/0x29e0
> [1.971052]  [] ? vmap_page_range_noflush+0x257/0x350
> [1.971061]  [] ? map_vm_area+0x2e/0x40
> [1.971067]  [] ? __vmalloc_node_range+0x170/0x270
> [1.971075]  [] ? SyS_init_module+0x99/0x110
> [1.971081]  [] SyS_init_module+0xc5/0x110
> [1.971088]  [] system_call_fastpath+0x16/0x1b
> [1.971094] Mem-Info:
> [1.971103] active_anon:875 inactive_anon:2049 isolated_anon:0
> [1.971103]  active_file:791 inactive_file:8841 isolated_file:0
> [1.971103]  unevictable:0 dirty:0 writeback:0 unstable:0
> [1.971103]  slab_reclaimable:1732 slab_unreclaimable:1629
> [1.971103]  mapped:1464 shmem:2053 pagetables:480 bounce:0
> [1.971103]  free:4065966 free_pcp:763 free_cma:0
> [1.971131] Node 0 DMA free:15912kB min:12kB low:12kB high:16kB
> active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
> unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15996kB
> managed:15912kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB
> slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB
> pagetables:0kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB
> free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
> [1.971217] 

Re: [CentOS-virt] Xen CentOS 7.3 server + CentOS 7.3 VM fails to boot after CR updates (applied to VM)!

[Fabian Arrotin]

On 01/09/17 21:41, Kevin Stange wrote:
> On 08/31/2017 07:50 AM, PJ Welsh wrote:
>> A recently created and fully functional CentOS 7.3 VM fails to boot
>> after applying CR updates:
> 
>> Server OS is CentOS 7.3 using Xen (no CR updates):
>> rpm -qa xen\*
>> xen-hypervisor-4.6.3-15.el7.x86_64
>> xen-4.6.3-15.el7.x86_64
>> xen-licenses-4.6.3-15.el7.x86_64
>> xen-libs-4.6.3-15.el7.x86_64
>> xen-runtime-4.6.3-15.el7.x86_64
>>
>> uname -a
>> Linux tsxen2.xx.com  4.9.39-29.el7.x86_64 #1 SMP
>> Fri Jul 21 15:09:00 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
>>
>> Sadly, the other issue is that the grub menu will not display for me to
>> select another kernel to see if it is just a kernel issue.
>>
>> The dracut prompt does not show any /dev/disk folder either.
>>
> 
> I'm seeing this as well.  My host is 4.9.44-29 and Xen 4.4.4-26 from
> testing repo, my guest is 3.10.0-693.1.1.  Guest boots fine with
> 514.26.2.  The kernel messages that appear to kick off the failure for
> me start with a page allocation failure.  It eventually reaches dracut
> failures due to systemd/udev not setting up properly, but I think the
> root is this:
> 
> [1.970630] [ cut here ]
> [1.970651] WARNING: CPU: 2 PID: 225 at mm/vmalloc.c:131
> vmap_page_range_noflush+0x2c1/0x350
> [1.970660] Modules linked in:
> [1.970668] CPU: 2 PID: 225 Comm: systemd-udevd Not tainted
> 3.10.0-693.1.1.el7.x86_64 #1
> [1.970677]   8cddc75d 8803e8587bd8
> 816a3d91
> [1.970688]  8803e8587c18 810879c8 0083811c14e8
> 8800066eb000
> [1.970698]  0001 8803e86d6940 c000
> 
> [1.970708] Call Trace:
> [1.970725]  [] dump_stack+0x19/0x1b
> [1.970736]  [] __warn+0xd8/0x100
> [1.970742]  [] warn_slowpath_null+0x1d/0x20
> [1.970748]  [] vmap_page_range_noflush+0x2c1/0x350
> [1.970758]  [] map_vm_area+0x2e/0x40
> [1.970765]  [] __vmalloc_node_range+0x170/0x270
> [1.970774]  [] ? module_alloc_update_bounds+0x14/0x70
> [1.970781]  [] ? module_alloc_update_bounds+0x14/0x70
> [1.970792]  [] module_alloc+0x73/0xd0
> [1.970798]  [] ? module_alloc_update_bounds+0x14/0x70
> [1.970804]  [] module_alloc_update_bounds+0x14/0x70
> [1.970811]  [] load_module+0xb02/0x29e0
> [1.970817]  [] ? vmap_page_range_noflush+0x257/0x350
> [1.970823]  [] ? map_vm_area+0x2e/0x40
> [1.970829]  [] ? __vmalloc_node_range+0x170/0x270
> [1.970838]  [] ? SyS_init_module+0x99/0x110
> [1.970846]  [] SyS_init_module+0xc5/0x110
> [1.970856]  [] system_call_fastpath+0x16/0x1b
> [1.970862] ---[ end trace 2117480876ed90d2 ]---
> [1.970869] vmalloc: allocation failure, allocated 24576 of 28672 bytes
> [1.970874] systemd-udevd: page allocation failure: order:0, mode:0xd2
> [1.970883] CPU: 2 PID: 225 Comm: systemd-udevd Tainted: GW
>   3.10.0-693.1.1.el7.x86_64 #1
> [1.970894]  00d2 8cddc75d 8803e8587c48
> 816a3d91
> [1.970910]  8803e8587cd8 81188810 8190ea38
> 8803e8587c68
> [1.970923]  0018 8803e8587ce8 8803e8587c88
> 8cddc75d
> [1.970939] Call Trace:
> [1.970946]  [] dump_stack+0x19/0x1b
> [1.970961]  [] warn_alloc_failed+0x110/0x180
> [1.970971]  [] __vmalloc_node_range+0x234/0x270
> [1.970981]  [] ? module_alloc_update_bounds+0x14/0x70
> [1.970989]  [] ? module_alloc_update_bounds+0x14/0x70
> [1.970999]  [] module_alloc+0x73/0xd0
> [1.971031]  [] ? module_alloc_update_bounds+0x14/0x70
> [1.971038]  [] module_alloc_update_bounds+0x14/0x70
> [1.971046]  [] load_module+0xb02/0x29e0
> [1.971052]  [] ? vmap_page_range_noflush+0x257/0x350
> [1.971061]  [] ? map_vm_area+0x2e/0x40
> [1.971067]  [] ? __vmalloc_node_range+0x170/0x270
> [1.971075]  [] ? SyS_init_module+0x99/0x110
> [1.971081]  [] SyS_init_module+0xc5/0x110
> [1.971088]  [] system_call_fastpath+0x16/0x1b
> [1.971094] Mem-Info:
> [1.971103] active_anon:875 inactive_anon:2049 isolated_anon:0
> [1.971103]  active_file:791 inactive_file:8841 isolated_file:0
> [1.971103]  unevictable:0 dirty:0 writeback:0 unstable:0
> [1.971103]  slab_reclaimable:1732 slab_unreclaimable:1629
> [1.971103]  mapped:1464 shmem:2053 pagetables:480 bounce:0
> [1.971103]  free:4065966 free_pcp:763 free_cma:0
> [1.971131] Node 0 DMA free:15912kB min:12kB low:12kB high:16kB
> active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
> unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15996kB
> managed:15912kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB
> slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB
> pagetables:0kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB
> free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
> [1.971217] 

Re: [CentOS-virt] Xen CentOS 7.3 server + CentOS 7.3 VM fails to boot after CR updates (applied to VM)!

[Kevin Stange]

On 09/01/2017 02:41 PM, Kevin Stange wrote:
> On 08/31/2017 07:50 AM, PJ Welsh wrote:
>> A recently created and fully functional CentOS 7.3 VM fails to boot
>> after applying CR updates:
> 
>> Server OS is CentOS 7.3 using Xen (no CR updates):
>> rpm -qa xen\*
>> xen-hypervisor-4.6.3-15.el7.x86_64
>> xen-4.6.3-15.el7.x86_64
>> xen-licenses-4.6.3-15.el7.x86_64
>> xen-libs-4.6.3-15.el7.x86_64
>> xen-runtime-4.6.3-15.el7.x86_64
>>
>> uname -a
>> Linux tsxen2.xx.com  4.9.39-29.el7.x86_64 #1 SMP
>> Fri Jul 21 15:09:00 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
>>
>> Sadly, the other issue is that the grub menu will not display for me to
>> select another kernel to see if it is just a kernel issue.
>>
>> The dracut prompt does not show any /dev/disk folder either.
>>
> 
> I'm seeing this as well.  My host is 4.9.44-29 and Xen 4.4.4-26 from
> testing repo, my guest is 3.10.0-693.1.1.  Guest boots fine with
> 514.26.2.  The kernel messages that appear to kick off the failure for
> me start with a page allocation failure.  It eventually reaches dracut
> failures due to systemd/udev not setting up properly, but I think the
> root is this:
> 

I created bugs for this issue (at least as I'm able to reproduce it):

https://bugs.centos.org/view.php?id=0013763
https://bugzilla.redhat.com/show_bug.cgi?id=1487754

Please add any extra information you might have to hopefully increase
the chance the problem gets fixed.

-- 
Kevin Stange
Chief Technology Officer
Steadfast | Managed Infrastructure, Datacenter and Cloud Services
800 S Wells, Suite 190 | Chicago, IL 60607
312.602.2689 X203 | Fax: 312.602.2688
ke...@steadfast.net | www.steadfast.net
___
CentOS-virt mailing list
CentOS-virt@centos.org
https://lists.centos.org/mailman/listinfo/centos-virt

Re: [CentOS-virt] Xen CentOS 7.3 server + CentOS 7.3 VM fails to boot after CR updates (applied to VM)!

[Kevin Stange]

On 08/31/2017 07:50 AM, PJ Welsh wrote:
> A recently created and fully functional CentOS 7.3 VM fails to boot
> after applying CR updates:

> Server OS is CentOS 7.3 using Xen (no CR updates):
> rpm -qa xen\*
> xen-hypervisor-4.6.3-15.el7.x86_64
> xen-4.6.3-15.el7.x86_64
> xen-licenses-4.6.3-15.el7.x86_64
> xen-libs-4.6.3-15.el7.x86_64
> xen-runtime-4.6.3-15.el7.x86_64
> 
> uname -a
> Linux tsxen2.xx.com  4.9.39-29.el7.x86_64 #1 SMP
> Fri Jul 21 15:09:00 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
> 
> Sadly, the other issue is that the grub menu will not display for me to
> select another kernel to see if it is just a kernel issue.
> 
> The dracut prompt does not show any /dev/disk folder either.
> 

I'm seeing this as well.  My host is 4.9.44-29 and Xen 4.4.4-26 from
testing repo, my guest is 3.10.0-693.1.1.  Guest boots fine with
514.26.2.  The kernel messages that appear to kick off the failure for
me start with a page allocation failure.  It eventually reaches dracut
failures due to systemd/udev not setting up properly, but I think the
root is this:

[1.970630] [ cut here ]
[1.970651] WARNING: CPU: 2 PID: 225 at mm/vmalloc.c:131
vmap_page_range_noflush+0x2c1/0x350
[1.970660] Modules linked in:
[1.970668] CPU: 2 PID: 225 Comm: systemd-udevd Not tainted
3.10.0-693.1.1.el7.x86_64 #1
[1.970677]   8cddc75d 8803e8587bd8
816a3d91
[1.970688]  8803e8587c18 810879c8 0083811c14e8
8800066eb000
[1.970698]  0001 8803e86d6940 c000

[1.970708] Call Trace:
[1.970725]  [] dump_stack+0x19/0x1b
[1.970736]  [] __warn+0xd8/0x100
[1.970742]  [] warn_slowpath_null+0x1d/0x20
[1.970748]  [] vmap_page_range_noflush+0x2c1/0x350
[1.970758]  [] map_vm_area+0x2e/0x40
[1.970765]  [] __vmalloc_node_range+0x170/0x270
[1.970774]  [] ? module_alloc_update_bounds+0x14/0x70
[1.970781]  [] ? module_alloc_update_bounds+0x14/0x70
[1.970792]  [] module_alloc+0x73/0xd0
[1.970798]  [] ? module_alloc_update_bounds+0x14/0x70
[1.970804]  [] module_alloc_update_bounds+0x14/0x70
[1.970811]  [] load_module+0xb02/0x29e0
[1.970817]  [] ? vmap_page_range_noflush+0x257/0x350
[1.970823]  [] ? map_vm_area+0x2e/0x40
[1.970829]  [] ? __vmalloc_node_range+0x170/0x270
[1.970838]  [] ? SyS_init_module+0x99/0x110
[1.970846]  [] SyS_init_module+0xc5/0x110
[1.970856]  [] system_call_fastpath+0x16/0x1b
[1.970862] ---[ end trace 2117480876ed90d2 ]---
[1.970869] vmalloc: allocation failure, allocated 24576 of 28672 bytes
[1.970874] systemd-udevd: page allocation failure: order:0, mode:0xd2
[1.970883] CPU: 2 PID: 225 Comm: systemd-udevd Tainted: GW
      3.10.0-693.1.1.el7.x86_64 #1
[1.970894]  00d2 8cddc75d 8803e8587c48
816a3d91
[1.970910]  8803e8587cd8 81188810 8190ea38
8803e8587c68
[1.970923]  0018 8803e8587ce8 8803e8587c88
8cddc75d
[1.970939] Call Trace:
[1.970946]  [] dump_stack+0x19/0x1b
[1.970961]  [] warn_alloc_failed+0x110/0x180
[1.970971]  [] __vmalloc_node_range+0x234/0x270
[1.970981]  [] ? module_alloc_update_bounds+0x14/0x70
[1.970989]  [] ? module_alloc_update_bounds+0x14/0x70
[1.970999]  [] module_alloc+0x73/0xd0
[1.971031]  [] ? module_alloc_update_bounds+0x14/0x70
[1.971038]  [] module_alloc_update_bounds+0x14/0x70
[1.971046]  [] load_module+0xb02/0x29e0
[1.971052]  [] ? vmap_page_range_noflush+0x257/0x350
[1.971061]  [] ? map_vm_area+0x2e/0x40
[1.971067]  [] ? __vmalloc_node_range+0x170/0x270
[1.971075]  [] ? SyS_init_module+0x99/0x110
[1.971081]  [] SyS_init_module+0xc5/0x110
[1.971088]  [] system_call_fastpath+0x16/0x1b
[1.971094] Mem-Info:
[1.971103] active_anon:875 inactive_anon:2049 isolated_anon:0
[1.971103]  active_file:791 inactive_file:8841 isolated_file:0
[1.971103]  unevictable:0 dirty:0 writeback:0 unstable:0
[1.971103]  slab_reclaimable:1732 slab_unreclaimable:1629
[1.971103]  mapped:1464 shmem:2053 pagetables:480 bounce:0
[1.971103]  free:4065966 free_pcp:763 free_cma:0
[1.971131] Node 0 DMA free:15912kB min:12kB low:12kB high:16kB
active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15996kB
managed:15912kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB
slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB
pagetables:0kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB
free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
[1.971217] lowmem_reserve[]: 0 4063 16028 16028
[1.971226] Node 0 DMA32 free:4156584kB min:4104kB low:5128kB
high:6156kB active_anon:952kB inactive_anon:1924kB active_file:0kB
inactive_file:0kB unevictable:0kB isolated(anon):0kB 

[CentOS-virt] Xen CentOS 7.3 server + CentOS 7.3 VM fails to boot after CR updates (applied to VM)!

[PJ Welsh]

A recently created and fully functional CentOS 7.3 VM fails to boot after
applying CR updates:
...
;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f
CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS
Linux 7 (Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f CentOS Linux 7 (Core)1;-1f1;-1f CentOS Linux 7
(Core)1;-1f1;-1f
 ...
[  194.011062] dracut-initqueue[227]: Warning: dracut-initqueue timeout -
starting timeout scripts
[  194.546622] dracut-initqueue[227]: Warning: dracut-initqueue timeout -
starting timeout scripts
[  194.547515] dracut-initqueue[227]: Warning: Could not boot.
[  194.557324] dracut-initqueue[227]: Warning:
/dev/disk/by-uuid/bbafa3a5-0a24-40d2-a362-03cb33d5d76b does not exist
 Starting Dracut Emergency Shell...
Warning: /dev/disk/by-uuid/bbafa3a5-0a24-40d2-a362-03cb33d5d76b does not
exist

Generating "/run/initramfs/rdsosreport.txt"

Entering emergency mode. Exit the shell to continue.
Type "journalctl" to view system logs.
You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or
/boot
after mounting them and attach it to a bug report.

dracut:/#

Server OS is CentOS 7.3 using Xen (no CR updates):
rpm -qa xen\*
xen-hypervisor-4.6.3-15.el7.x86_64
xen-4.6.3-15.el7.x86_64
xen-licenses-4.6.3-15.el7.x86_64
xen-libs-4.6.3-15.el7.x86_64
xen-runtime-4.6.3-15.el7.x86_64

uname -a
Linux tsxen2.xx.com 4.9.39-29.el7.x86_64 #1 SMP Fri Jul 21 15:09:00 UTC
2017 x86_64 x86_64 x86_64 GNU/Linux

Sadly, the other issue is that the grub menu will not display for me to
select another kernel to see if it is just a kernel issue.

The dracut prompt does not show any /dev/disk folder either.

Any info?

Thanks
PJWelsh
___
CentOS-virt mailing list
CentOS-virt@centos.org
https://lists.centos.org/mailman/listinfo/centos-virt