Re: [CentOS-virt] Are xen and centos incompatible?
Konrad Rzeszutek Wilk konrad.w...@oracle.com writes: On Sat, Jun 14, 2014 at 09:07:51AM +0200, lee wrote: Konrad Rzeszutek Wilk konrad.w...@oracle.com writes: I am wondering if you are using an older kernel. The xen-acpi-processor driver should be loaded which would give the C and P states to the hypervisor. Which in turn would result in those above commands providing the right data. Linux heimdall 3.2.0-4-amd64 #1 SMP Debian 3.2.57-3+deb7u2 x86_64 GNU/Linux This is what comes in Debian. Unfortunately, this kernel crashes when I'm copying data to a domU NFS server over the network :(( I need to find out how to get some useful information out of it to make a bug report. How do I know whether the xen-acpi-processor driver is loaded or not? lsmod Well, yes, I didn't know what the name of this module exactly is. There's also the processor module. But it looks like v3.4 and later were the kernels that started having this driver. That would explain why it does not exist as you are using 3.2. Because of the crashes, I've upgraded to a 3.14.5-1~bpo70+1 kernel from Debian backports. That one does have the xen-acpi-processor module, and it's working. I don't know yet if it'll crash, though. -- Knowledge is volatile and fluid. Software is power. ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] Are xen and centos incompatible?
On Sat, Jun 14, 2014 at 09:07:51AM +0200, lee wrote: Konrad Rzeszutek Wilk konrad.w...@oracle.com writes: Hm, xen kinda makes the cpus and their power management invisible, too: root@heimdall:~# xenpm get-cpufreq-para [CPU0] failed to get cpufreq parameter [...] root@heimdall:~# xenpm get-cpufreq-states root@heimdall:~# So I guess it could as well make it so that lspci doesn't show passed-out devices. I am wondering if you are using an older kernel. The xen-acpi-processor driver should be loaded which would give the C and P states to the hypervisor. Which in turn would result in those above commands providing the right data. Linux heimdall 3.2.0-4-amd64 #1 SMP Debian 3.2.57-3+deb7u2 x86_64 GNU/Linux This is what comes in Debian. Unfortunately, this kernel crashes when I'm copying data to a domU NFS server over the network :(( I need to find out how to get some useful information out of it to make a bug report. How do I know whether the xen-acpi-processor driver is loaded or not? lsmod But it looks like v3.4 and later were the kernels that started having this driver. That would explain why it does not exist as you are using 3.2. BTW, getting some info in dmesg might be nice, like a message saying xen-pciback: device 06:00.0 can be passed through to guests. We could You just need to boot with 'debug' - and it should tell you that a device is being assigned to another guest (when assigning). Also at bootup it will tell you that it is seizinging. Just do 'dmesg | grep pciback' and you will get it. Ok, I enabled debugging. Maybe that also helps to get some more info about the crashes. actually see right away if it did work or not. That a device disappears isn't too great as indication, especially not when lspci still lists it. Of course, you could use the command (which I don't remember) to show devices that can be passed through. But that may just work as well as Such as xl or xm pci-list-assignable? yes 'xenpm get-cpufreq-states': Apparently, there aren't any CPUs ... See if xen-acpi-processor is loaded or built in. Unless it's called processor, it doesn't seem to exist: root@heimdall:~# lsmod |grep proc processor 28149 1 acpi_cpufreq thermal_sys18040 1 processor root@heimdall:~# grep -i proces /boot/config-3.2.0-4-amd64 |less root@heimdall:~# find /lib/modules/3.2.0-4-amd64/ -name *proces* /lib/modules/3.2.0-4-amd64/kernel/drivers/acpi/processor.ko root@heimdall:~# find /lib/modules/3.2.0-4-amd64/ -name *xen* /lib/modules/3.2.0-4-amd64/kernel/drivers/net/xen-netfront.ko /lib/modules/3.2.0-4-amd64/kernel/drivers/net/xen-netback /lib/modules/3.2.0-4-amd64/kernel/drivers/net/xen-netback/xen-netback.ko /lib/modules/3.2.0-4-amd64/kernel/drivers/net/ethernet/qlogic/netxen /lib/modules/3.2.0-4-amd64/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko /lib/modules/3.2.0-4-amd64/kernel/drivers/watchdog/xen_wdt.ko /lib/modules/3.2.0-4-amd64/kernel/drivers/xen /lib/modules/3.2.0-4-amd64/kernel/drivers/xen/xen-gntalloc.ko /lib/modules/3.2.0-4-amd64/kernel/drivers/xen/xen-gntdev.ko /lib/modules/3.2.0-4-amd64/kernel/drivers/xen/xen-evtchn.ko /lib/modules/3.2.0-4-amd64/kernel/drivers/xen/xen-pciback /lib/modules/3.2.0-4-amd64/kernel/drivers/xen/xen-pciback/xen-pciback.ko /lib/modules/3.2.0-4-amd64/kernel/drivers/xen/xenfs /lib/modules/3.2.0-4-amd64/kernel/drivers/xen/xenfs/xenfs.ko /lib/modules/3.2.0-4-amd64/kernel/drivers/block/xen-blkfront.ko /lib/modules/3.2.0-4-amd64/kernel/drivers/block/xen-blkback /lib/modules/3.2.0-4-amd64/kernel/drivers/block/xen-blkback/xen-blkback.ko /lib/modules/3.2.0-4-amd64/kernel/drivers/pci/xen-pcifront.ko root@heimdall:~# grep -i proces /boot/config-3.2.0-4-amd64 CONFIG_BSD_PROCESS_ACCT=y CONFIG_BSD_PROCESS_ACCT_V3=y # Processor type and features CONFIG_ACPI_PROCESSOR=m CONFIG_ACPI_PROCESSOR_AGGREGATOR=m # Audio decoders, processors and mixers root@heimdall:~# grep -i xen /boot/config-3.2.0-4-amd64 CONFIG_XEN=y CONFIG_XEN_DOM0=y CONFIG_XEN_PRIVILEGED_GUEST=y CONFIG_XEN_PVHVM=y CONFIG_XEN_MAX_DOMAIN_MEMORY=128 CONFIG_XEN_SAVE_RESTORE=y # CONFIG_XEN_DEBUG_FS is not set CONFIG_MICROCODE_XEN=y CONFIG_PCI_XEN=y CONFIG_XEN_PCIDEV_FRONTEND=m CONFIG_XEN_BLKDEV_FRONTEND=m CONFIG_XEN_BLKDEV_BACKEND=m CONFIG_NETXEN_NIC=m CONFIG_XEN_NETDEV_FRONTEND=m CONFIG_XEN_NETDEV_BACKEND=m CONFIG_INPUT_XEN_KBDDEV_FRONTEND=y CONFIG_HVC_XEN=y CONFIG_XEN_WDT=m CONFIG_XEN_FBDEV_FRONTEND=y # Xen driver support CONFIG_XEN_BALLOON=y #
Re: [CentOS-virt] Are xen and centos incompatible?
Konrad Rzeszutek Wilk konrad.w...@oracle.com writes: Hm, xen kinda makes the cpus and their power management invisible, too: root@heimdall:~# xenpm get-cpufreq-para [CPU0] failed to get cpufreq parameter [...] root@heimdall:~# xenpm get-cpufreq-states root@heimdall:~# So I guess it could as well make it so that lspci doesn't show passed-out devices. I am wondering if you are using an older kernel. The xen-acpi-processor driver should be loaded which would give the C and P states to the hypervisor. Which in turn would result in those above commands providing the right data. Linux heimdall 3.2.0-4-amd64 #1 SMP Debian 3.2.57-3+deb7u2 x86_64 GNU/Linux This is what comes in Debian. Unfortunately, this kernel crashes when I'm copying data to a domU NFS server over the network :(( I need to find out how to get some useful information out of it to make a bug report. How do I know whether the xen-acpi-processor driver is loaded or not? BTW, getting some info in dmesg might be nice, like a message saying xen-pciback: device 06:00.0 can be passed through to guests. We could You just need to boot with 'debug' - and it should tell you that a device is being assigned to another guest (when assigning). Also at bootup it will tell you that it is seizinging. Just do 'dmesg | grep pciback' and you will get it. Ok, I enabled debugging. Maybe that also helps to get some more info about the crashes. actually see right away if it did work or not. That a device disappears isn't too great as indication, especially not when lspci still lists it. Of course, you could use the command (which I don't remember) to show devices that can be passed through. But that may just work as well as Such as xl or xm pci-list-assignable? yes 'xenpm get-cpufreq-states': Apparently, there aren't any CPUs ... See if xen-acpi-processor is loaded or built in. Unless it's called processor, it doesn't seem to exist: root@heimdall:~# lsmod |grep proc processor 28149 1 acpi_cpufreq thermal_sys18040 1 processor root@heimdall:~# grep -i proces /boot/config-3.2.0-4-amd64 |less root@heimdall:~# find /lib/modules/3.2.0-4-amd64/ -name *proces* /lib/modules/3.2.0-4-amd64/kernel/drivers/acpi/processor.ko root@heimdall:~# find /lib/modules/3.2.0-4-amd64/ -name *xen* /lib/modules/3.2.0-4-amd64/kernel/drivers/net/xen-netfront.ko /lib/modules/3.2.0-4-amd64/kernel/drivers/net/xen-netback /lib/modules/3.2.0-4-amd64/kernel/drivers/net/xen-netback/xen-netback.ko /lib/modules/3.2.0-4-amd64/kernel/drivers/net/ethernet/qlogic/netxen /lib/modules/3.2.0-4-amd64/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko /lib/modules/3.2.0-4-amd64/kernel/drivers/watchdog/xen_wdt.ko /lib/modules/3.2.0-4-amd64/kernel/drivers/xen /lib/modules/3.2.0-4-amd64/kernel/drivers/xen/xen-gntalloc.ko /lib/modules/3.2.0-4-amd64/kernel/drivers/xen/xen-gntdev.ko /lib/modules/3.2.0-4-amd64/kernel/drivers/xen/xen-evtchn.ko /lib/modules/3.2.0-4-amd64/kernel/drivers/xen/xen-pciback /lib/modules/3.2.0-4-amd64/kernel/drivers/xen/xen-pciback/xen-pciback.ko /lib/modules/3.2.0-4-amd64/kernel/drivers/xen/xenfs /lib/modules/3.2.0-4-amd64/kernel/drivers/xen/xenfs/xenfs.ko /lib/modules/3.2.0-4-amd64/kernel/drivers/block/xen-blkfront.ko /lib/modules/3.2.0-4-amd64/kernel/drivers/block/xen-blkback /lib/modules/3.2.0-4-amd64/kernel/drivers/block/xen-blkback/xen-blkback.ko /lib/modules/3.2.0-4-amd64/kernel/drivers/pci/xen-pcifront.ko root@heimdall:~# grep -i proces /boot/config-3.2.0-4-amd64 CONFIG_BSD_PROCESS_ACCT=y CONFIG_BSD_PROCESS_ACCT_V3=y # Processor type and features CONFIG_ACPI_PROCESSOR=m CONFIG_ACPI_PROCESSOR_AGGREGATOR=m # Audio decoders, processors and mixers root@heimdall:~# grep -i xen /boot/config-3.2.0-4-amd64 CONFIG_XEN=y CONFIG_XEN_DOM0=y CONFIG_XEN_PRIVILEGED_GUEST=y CONFIG_XEN_PVHVM=y CONFIG_XEN_MAX_DOMAIN_MEMORY=128 CONFIG_XEN_SAVE_RESTORE=y # CONFIG_XEN_DEBUG_FS is not set CONFIG_MICROCODE_XEN=y CONFIG_PCI_XEN=y CONFIG_XEN_PCIDEV_FRONTEND=m CONFIG_XEN_BLKDEV_FRONTEND=m CONFIG_XEN_BLKDEV_BACKEND=m CONFIG_NETXEN_NIC=m CONFIG_XEN_NETDEV_FRONTEND=m CONFIG_XEN_NETDEV_BACKEND=m CONFIG_INPUT_XEN_KBDDEV_FRONTEND=y CONFIG_HVC_XEN=y CONFIG_XEN_WDT=m CONFIG_XEN_FBDEV_FRONTEND=y # Xen driver support CONFIG_XEN_BALLOON=y # CONFIG_XEN_BALLOON_MEMORY_HOTPLUG is not set CONFIG_XEN_SCRUB_PAGES=y CONFIG_XEN_DEV_EVTCHN=m CONFIG_XEN_BACKEND=y CONFIG_XENFS=m CONFIG_XEN_COMPAT_XENFS=y CONFIG_XEN_SYS_HYPERVISOR=y CONFIG_XEN_XENBUS_FRONTEND=y CONFIG_XEN_GNTDEV=m CONFIG_XEN_GRANT_DEV_ALLOC=m CONFIG_SWIOTLB_XEN=y CONFIG_XEN_PCIDEV_BACKEND=m root@heimdall:~# -- Knowledge is
Re: [CentOS-virt] Are xen and centos incompatible?
Hm, xen kinda makes the cpus and their power management invisible, too: root@heimdall:~# xenpm get-cpufreq-para [CPU0] failed to get cpufreq parameter [...] root@heimdall:~# xenpm get-cpufreq-states root@heimdall:~# So I guess it could as well make it so that lspci doesn't show passed-out devices. I am wondering if you are using an older kernel. The xen-acpi-processor driver should be loaded which would give the C and P states to the hypervisor. Which in turn would result in those above commands providing the right data. BTW, getting some info in dmesg might be nice, like a message saying xen-pciback: device 06:00.0 can be passed through to guests. We could You just need to boot with 'debug' - and it should tell you that a device is being assigned to another guest (when assigning). Also at bootup it will tell you that it is seizinging. Just do 'dmesg | grep pciback' and you will get it. actually see right away if it did work or not. That a device disappears isn't too great as indication, especially not when lspci still lists it. Of course, you could use the command (which I don't remember) to show devices that can be passed through. But that may just work as well as Such as xl or xm pci-list-assignable? 'xenpm get-cpufreq-states': Apparently, there aren't any CPUs ... See if xen-acpi-processor is loaded or built in. -- Knowledge is volatile and fluid. Software is power. ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] Are xen and centos incompatible?
Konrad Rzeszutek Wilk konrad.w...@oracle.com writes: On Tue, Jun 10, 2014 at 06:47:20PM +0200, lee wrote: Konrad Rzeszutek Wilk konrad.w...@oracle.com writes: On Tue, Jun 10, 2014 at 04:44:23AM +0200, lee wrote: Konrad Rzeszutek Wilk konrad.w...@oracle.com writes: The device should be visible in the dom0 - even when it is for passthrough. Why should it be visible when it's hidden? The 'hide' part is to hide it from the device drivers in the initial domain - dom0. That is so that they will not try to use it - as we plan to pass them to a guest. We need it in the dom0 to do admin type work - FLR it, etc. With Debian, it's not visible in dom0 when the passthrough works. That's how I found out that it does work to begin with, and it makes sense to me. That is a surprise. If you do 'lspci' in your dom0 do you see the device (06:00.0)? Lspci still shows it, and the device (eth1) is invisible. What does FLR mean? And how do you do something with a device for which no drivers are loaded? I'd find it rather unusual to have a device without drivers and even be able to use it; such devices usually don't show up. Function Level Reset. You pass the device to a guest so it can load the drivers and the initial domain (dom0) won't use it. Hm, xen kinda makes the cpus and their power management invisible, too: root@heimdall:~# xenpm get-cpufreq-para [CPU0] failed to get cpufreq parameter [...] root@heimdall:~# xenpm get-cpufreq-states root@heimdall:~# So I guess it could as well make it so that lspci doesn't show passed-out devices. BTW, getting some info in dmesg might be nice, like a message saying xen-pciback: device 06:00.0 can be passed through to guests. We could actually see right away if it did work or not. That a device disappears isn't too great as indication, especially not when lspci still lists it. Of course, you could use the command (which I don't remember) to show devices that can be passed through. But that may just work as well as 'xenpm get-cpufreq-states': Apparently, there aren't any CPUs ... -- Knowledge is volatile and fluid. Software is power. ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] Are xen and centos incompatible?
On 06/10/2014 05:44 AM, lee wrote: [...] and xen just don't go along with each other. The server is all on debian now. The problem with incompetence is its inability to recognize itself. -- Orrin Woodward https://www.goodreads.com/author/show/249881.Orrin_Woodward, /L.I.F.E. Living Intentionally For Excellence https://www.goodreads.com/work/quotes/16132299 / ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] Are xen and centos incompatible?
Konrad Rzeszutek Wilk konrad.w...@oracle.com writes: On Tue, Jun 10, 2014 at 04:44:23AM +0200, lee wrote: Konrad Rzeszutek Wilk konrad.w...@oracle.com writes: The device should be visible in the dom0 - even when it is for passthrough. Why should it be visible when it's hidden? The 'hide' part is to hide it from the device drivers in the initial domain - dom0. That is so that they will not try to use it - as we plan to pass them to a guest. We need it in the dom0 to do admin type work - FLR it, etc. With Debian, it's not visible in dom0 when the passthrough works. That's how I found out that it does work to begin with, and it makes sense to me. What does FLR mean? And how do you do something with a device for which no drivers are loaded? I'd find it rather unusual to have a device without drivers and even be able to use it; such devices usually don't show up. But irrespective of that - the steps mentioned there are out of date. The correct option should be 'xen-pciback.hide=(06:00.0) xen-pciback.permissive=1' That's one of the problems: Xen is very poorly documented. Any help in improving the documentations would be appreciated. Every month we run 'Documentation days' and any work - either on Wiki, manuals, docs, etc would be quite appreciated. If I have some time, I might make a writeup about how to set up what I did. But it seems I'm using an outdated version of xen, which is what comes with Debian, so by the time I'd finish the writeup, it would be outdated and contribute to confusion more than do any good. And considering xen, I don't really know anything. I figured out that passthrough doesn't work out of the box on Debian because the module for the device was loaded from the initrd.img before the xen-pciback module and made a bug report because you're supposed to be able to use files in /etc/modprobe.d which can specify dependencies and when you do that, you can't have that just overridden or there's no point in doing that --- and there doesn't seem to be any other way to specify the order in which modules are loaded, and long ago, Debian came up with a policy that things should work out of the box whenever possible (which they might have forgotten by now ...). So maybe they'll fix this problem. Anyway, it probably goes for other distributions as well, and a hint in the xen docs probably won't hurt. Please see http://wiki.xenproject.org/wiki/Xen_Project_Document_Days I tried to make a request to become a wiki editor. There might be some places in the docs I might be able to make clearer. I don't know if that was successful, though. It seemed to want to redirect me to some google website ... -- Knowledge is volatile and fluid. Software is power. ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] Are xen and centos incompatible?
On Sat, Jun 07, 2014 at 02:44:54AM +0200, lee wrote: Hi, I'm trying to pass a physical network interface through to a domU. This seems to be impossible because the way xen wants to do it is incompatible with the way centos wants to do it. Huh? I followed documentation on http://www.xen-support.com/?p=151 and tried booting with 'pciback.permissive pciback.hide=(06:00.0)'. This gives a hint in dmesg kernel: xen-pciback: backend is vpci, and the device is still visible in dom0. So this obviously doesn't work. The device should be visible in the dom0 - even when it is for passthrough. But irrespective of that - the steps mentioned there are out of date. The correct option should be 'xen-pciback.hide=(06:00.0) xen-pciback.permissive=1' Following http://wiki.xen.org/wiki/Xen_PCI_Passthrough: which is mentioned in that link. [root@heimdall ~]# xl pci-assignable-add 06:00.0 xend is running, which may cause unpredictable results when using this xl command. Please shut down xend before continuing. (This check can be overridden with the -f option.) [root@heimdall ~]# There doesn't seem to be any documentation about what xend does or is for and how to pass a physical network interface through with it. Did you just try using the same parameter but replace 'xl' with 'xm'? I'm starting to think that using centos for a server os is an extremely bad choice because nothing works. It's not like a Linux distribution but like a mess of pieces from several unrelated puzzles thrown together randomly. -- Knowledge is volatile and fluid. Software is power. ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] Are xen and centos incompatible?
Konrad Rzeszutek Wilk konrad.w...@oracle.com writes: On Sat, Jun 07, 2014 at 02:44:54AM +0200, lee wrote: Hi, I'm trying to pass a physical network interface through to a domU. This seems to be impossible because the way xen wants to do it is incompatible with the way centos wants to do it. Huh? I followed documentation on http://www.xen-support.com/?p=151 and tried booting with 'pciback.permissive pciback.hide=(06:00.0)'. This gives a hint in dmesg kernel: xen-pciback: backend is vpci, and the device is still visible in dom0. So this obviously doesn't work. The device should be visible in the dom0 - even when it is for passthrough. Why should it be visible when it's hidden? But irrespective of that - the steps mentioned there are out of date. The correct option should be 'xen-pciback.hide=(06:00.0) xen-pciback.permissive=1' That's one of the problems: Xen is very poorly documented. I replaced centos with debian and finally got it to work. Things are much easier with debian. Following http://wiki.xen.org/wiki/Xen_PCI_Passthrough: which is mentioned in that link. [root@heimdall ~]# xl pci-assignable-add 06:00.0 xend is running, which may cause unpredictable results when using this xl command. Please shut down xend before continuing. (This check can be overridden with the -f option.) [root@heimdall ~]# There doesn't seem to be any documentation about what xend does or is for and how to pass a physical network interface through with it. Did you just try using the same parameter but replace 'xl' with 'xm'? I don't remember ... It seemed I had to use virsh since nothing else worked on centos, and virsh doesn't let you seem to get networking stuff to work. Centos and xen just don't go along with each other. The server is all on debian now. -- Knowledge is volatile and fluid. Software is power. ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
[CentOS-virt] Are xen and centos incompatible?
Hi, I'm trying to pass a physical network interface through to a domU. This seems to be impossible because the way xen wants to do it is incompatible with the way centos wants to do it. I followed documentation on http://www.xen-support.com/?p=151 and tried booting with 'pciback.permissive pciback.hide=(06:00.0)'. This gives a hint in dmesg kernel: xen-pciback: backend is vpci, and the device is still visible in dom0. So this obviously doesn't work. Following http://wiki.xen.org/wiki/Xen_PCI_Passthrough: [root@heimdall ~]# xl pci-assignable-add 06:00.0 xend is running, which may cause unpredictable results when using this xl command. Please shut down xend before continuing. (This check can be overridden with the -f option.) [root@heimdall ~]# There doesn't seem to be any documentation about what xend does or is for and how to pass a physical network interface through with it. I'm starting to think that using centos for a server os is an extremely bad choice because nothing works. It's not like a Linux distribution but like a mess of pieces from several unrelated puzzles thrown together randomly. -- Knowledge is volatile and fluid. Software is power. ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
