Re: [Xen-devel] Clarifying PVH mode requirements

[PGNet Dev]

On Mon, Feb 1, 2016 at 11:58 AM, Boris Ostrovsky
Current PVH implementation has never been described as
production-ready. What is happening now with HVMlite is
essentially bringing PVH to production-quality level.


So should I s/PVH/HVMlite/g?


 From user perspective that will be almost true. I am not sure it should
be classified as PV mode anymore since it's really an HVM guest without
any devices. But it's not there yet so it's too early to point your
editor there.

BTW, I don't think the flowchart in the wiki is correct as far as PVH is
concerned --- you can't use PVH unless HVM (and, in fact, PVHVM) is
supported.


Noting the very recent flurry of HVMLite activity, so where does PVH sit?

As it's not production-ready (and, atm, unusable here), is it planned to 
be?  Or is it being simply leap-frogged by HVMLite?



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[Boris Ostrovsky]

On 02/01/2016 06:49 PM, Brendan Gregg wrote:



On Mon, Feb 1, 2016 at 11:58 AM, Boris Ostrovsky 
> wrote:



Current PVH implementation has never been described as
production-ready. What is happening now with HVMlite is
essentially bringing PVH to production-quality level.


So should I s/PVH/HVMlite/g?


From user perspective that will be almost true. I am not sure it should 
be classified as PV mode anymore since it's really an HVM guest without 
any devices. But it's not there yet so it's too early to point your 
editor there.


BTW, I don't think the flowchart in the wiki is correct as far as PVH is 
concerned --- you can't use PVH unless HVM (and, in fact, PVHVM) is 
supported.



-boris



Or too much of a simlification? thanks,





___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[Jan Beulich]

>>> On 01.02.16 at 20:17,  wrote:
> On 02/01/2016 11:56 AM, Jan Beulich wrote:
> On 01.02.16 at 15:54,  wrote:
>
>
> This looks very much like it needs backport of 33c19df9a ("x86/PCI:
> intercept accesses to RO MMIO from dom0s in HVM containers") from
> unstable, which fixes PVH regression introduced by 9256f66c1606
> ("x86/PCI: intercept all PV Dom0 MMCFG writes")
>> I don't really understand: The former was needed to fix an issue
>> introduced by the latter, but the latter isn't in 4.6 iirc.
>>
> 
> I see it in 4.6, for example
> 
> http://xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=refs/heads/stable-4.6;p 
> g=2
> 
> and search for "intercept all PV Dom0 MMCFG writes".

Oh, I'm sorry, I had only looked at what has gone in after 4.6.0.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[Jan Beulich]

>>> On 01.02.16 at 15:54,  wrote:
> This looks very much like it needs backport of 33c19df9a ("x86/PCI: 
> intercept accesses to RO MMIO from dom0s in HVM containers") from 
> unstable, which fixes PVH regression introduced by 9256f66c1606 
> ("x86/PCI: intercept all PV Dom0 MMCFG writes")

So would you please confirm that this indeed fixes your issue?
I'm hesitant to put it in without confirmation, and it's likely too
late for 4.6.1 now anyway (so would then only appear in 4.6.2).

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[PGNet Dev]

On 02/02/2016 07:51 AM, Jan Beulich wrote:

So would you please confirm that this indeed fixes your issue?
I'm hesitant to put it in without confirmation, and it's likely too
late for 4.6.1 now anyway (so would then only appear in 4.6.2).


I don't build Xen.  I use packages provided by Opensuse distro @


http://download.opensuse.org/repositories/Virtualization/openSUSE_Leap_42.1

Happy to test any built/installable packages.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[PGNet Dev]

On 02/01/2016 11:14 AM, Boris Ostrovsky wrote:

Is 'HVMLite' replacing 'PVH'? Or are they separate modes?


Yes, HVMlite is replacing PVH. Probably once we get dom0 support.


If that's a 'done deal', and it sounds like it is, it'd be useful to 
have it integrated into:


  http://wiki.xen.org/wiki/Understanding_the_Virtualization_Spectrum

particularly as there's no mention of HVMlite on the wiki, at all.

It's unclear whether PVH, in its current dev state (at least here), is 
worth-the-visit -- especially if HVMlite is "ComingSoon(tm)".


I suppose I'm looking for some guidance as to which to invest time in 
while on Xen 4.6.0, ack'ing that neither PVH nor HVMlite are 
production-ready.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[Boris Ostrovsky]

On 02/01/2016 10:49 AM, PGNet Dev wrote:

On 02/01/2016 06:11 AM, Boris Ostrovsky wrote:

This actually never happened for Linux: HVMlite showed up fast enough
that it didn't make sense anymore to add 32-bit support to Linux
(especially given that AMD was still not supported).


Is 'HVMLite' replacing 'PVH'? Or are they separate modes?


Yes, HVMlite is replacing PVH. Probably once we get dom0 support.

-boris

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[Boris Ostrovsky]

On 02/01/2016 11:56 AM, Jan Beulich wrote:

On 01.02.16 at 15:54,  wrote:


This looks very much like it needs backport of 33c19df9a ("x86/PCI:
intercept accesses to RO MMIO from dom0s in HVM containers") from
unstable, which fixes PVH regression introduced by 9256f66c1606
("x86/PCI: intercept all PV Dom0 MMCFG writes")

I don't really understand: The former was needed to fix an issue
introduced by the latter, but the latter isn't in 4.6 iirc.



I see it in 4.6, for example

http://xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=refs/heads/stable-4.6;pg=2

and search for "intercept all PV Dom0 MMCFG writes".

-boris

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[Brendan Gregg]

On Mon, Feb 1, 2016 at 11:58 AM, Boris Ostrovsky  wrote:

> On 02/01/2016 02:27 PM, PGNet Dev wrote:
>
>> On 02/01/2016 11:14 AM, Boris Ostrovsky wrote:
>>
>>> Is 'HVMLite' replacing 'PVH'? Or are they separate modes?

>>>
>>> Yes, HVMlite is replacing PVH. Probably once we get dom0 support.
>>>
>>
>> If that's a 'done deal', and it sounds like it is, it'd be useful to have
>> it integrated into:
>>
>> http://wiki.xen.org/wiki/Understanding_the_Virtualization_Spectrum
>>
>> particularly as there's no mention of HVMlite on the wiki, at all.
>>
>
Thanks, HVMlite is new to me. I created the Xen modes diagram on this page
(based on the older modes diagram; if anyone wants the new omnigraffle
source, email me), and I just created a Xen wiki account so I can update
this page. I've been meaning to update this modes diagram anyway, and
improve the columns.


> HVMlite is very new: domU hypervisor support has been added less than two
> months ago and Linux patches are being reviewed as we speak (FreeBSD, I
> believe, is supported).
>
>
>> It's unclear whether PVH, in its current dev state (at least here), is
>> worth-the-visit -- especially if HVMlite is "ComingSoon(tm)".
>>
>> I suppose I'm looking for some guidance as to which to invest time in
>> while on Xen 4.6.0, ack'ing that neither PVH nor HVMlite are
>> production-ready.
>>
>
> Current PVH implementation has never been described as production-ready.
> What is happening now with HVMlite is essentially bringing PVH to
> production-quality level.


So should I s/PVH/HVMlite/g? Or too much of a simlification? thanks,

Brendan

-- 
Brendan Gregg, Senior Performance Architect, Netflix
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[Wei Liu]

(Cc Roger)

On Sun, Jan 31, 2016 at 01:27:23PM -0800, PGNet Dev wrote:
> I run Xen 4.6 Dom0
> 
>   rpm -qa | egrep -i "kernel-default-4|xen-4"
>   kernel-default-devel-4.4.0-8.1.g9f68b90.x86_64
>   xen-4.6.0_08-405.1.x86_64
> 
> My guests are currently HVM in PVHVM mode; I'm exploring PVH.
> 
> IIUC, for 4.6, this doc
> 
>   http://xenbits.xen.org/docs/4.6-testing/misc/pvh-readme.txt
> 

AIUI that document is not very up to date. 

In the mean time. There is another document in the same directory called
pvh.markdown that you can have a look.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[Roger Pau Monné]

El 31/01/16 a les 22.27, PGNet Dev ha escrit:
> I run Xen 4.6 Dom0
> 
> rpm -qa | egrep -i "kernel-default-4|xen-4"
> kernel-default-devel-4.4.0-8.1.g9f68b90.x86_64
> xen-4.6.0_08-405.1.x86_64

Are your kernels compiled with CONFIG_PVH enabled?

> My guests are currently HVM in PVHVM mode; I'm exploring PVH.
> 
> IIUC, for 4.6, this doc
> 
> http://xenbits.xen.org/docs/4.6-testing/misc/pvh-readme.txt
> 
> instructs the following necessary changes:
> 
> @ GRUBG cfg
> 
> -GRUB_CMDLINE_XEN=" ..."
> +GRUB_CMDLINE_XEN=" dom0pvh ..."
> 
> &, @ guest.cfg
> 
> +pvh = 1
> 
> For my guest.cfg, currently in PVHVM mode, I have
> 
> builder = 'hvm'
> xen_platform_pci = 1
> device_model_version="qemu-xen"
> hap = 1
> ...
> 
> Q:
> Do any of these^^ params need to also change with the addition of
> 
> pvh = 1

Yes, you need to remove builder, xen_platform_pci and
device_model_version, and add a kernel and ramdisk parameters that point
to the actual kernel and ramdisk that you want to use. The file should
look like:

kernel = "/path/to/kernel"
ramdisk = "/path/to/ramdisk"
pvh=1
hap=1

[... other options, memory, vcpus ...]

The paths in the kernel and ramdisk options are relative to Dom0, not
DomU. You can also use pygrub if you prefer, by removing the
kernel/ramdisk options and setting the bootloader one:

bootloader="pygrub"

> 
>> At the moment HAP is required for PVH.
> 
> As above, I've 'hap = 1' enabled.
> 
> But checking cpu,
> 
> hwinfo --cpu | egrep "Arch|Model"
>   Arch: X86-64
>   Model: 6.60.3 "Intel(R) Xeon(R) CPU E3-1220 v3 @ 3.10GHz"

You CPU is perfectly capable of running both a PVH Dom0 or DomU, check:

http://ark.intel.com/products/52269/Intel-Xeon-Processor-E3-1220-8M-Cache-3_10-GHz

Look for EPT and VT-d which are the main requirements for PVH.

> Q:
> Am I out of luck re: PVH with more modern Haswell? Or is there a
> different check I should be running ?
> 
>> At present the only PVH guest is an x86 64bit PV linux.
> 
> Is this still current/true info?

IIRC Boris (CCed) added support for 32bit PVH to Linux, so you should be
able to use either 32 or 64 kernels.

Roger.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[Roger Pau Monné]

El 01/02/16 a les 4.47, PGNet Dev ha escrit:
> In any case, the !st issue, prior to any guest being launched, simply
> adding
> 
>> @ GRUBG cfg
>>
>> -GRUB_CMDLINE_XEN=" ..."
>> +GRUB_CMDLINE_XEN=" dom0pvh ..."

Does your kernel support PVH mode (ie: CONFIG_PVH enabled?)

> causes boot fail,
> 
> ...
> (XEN) [2016-01-31 19:28:09] d0v0 EPT violation 0x1aa (-w-/r-x) gpa
> 0x00f100054c mfn 0xf15
> (XEN) [2016-01-31 19:28:09] d0v0 Walking EPT tables for GFN f1000:
> (XEN) [2016-01-31 19:28:09] d0v0  epte 800845108107
> (XEN) [2016-01-31 19:28:09] d0v0  epte 80085b680107
> (XEN) [2016-01-31 19:28:09] d0v0  epte 800844af7107
> (XEN) [2016-01-31 19:28:09] d0v0  epte 8050f1000905
> (XEN) [2016-01-31 19:28:09] d0v0  --- GLA 0xc96a254c
> (XEN) [2016-01-31 19:28:09] domain_crash called from vmx.c:2685
> (XEN) [2016-01-31 19:28:09] Domain 0 (vcpu#0) crashed on cpu#0:
> (XEN) [2016-01-31 19:28:09] [ Xen-4.6.0_08-405  x86_64  debug=n
> Tainted:C ]
> (XEN) [2016-01-31 19:28:09] CPU:0
> (XEN) [2016-01-31 19:28:09] RIP:0010:[]
> (XEN) [2016-01-31 19:28:09] RFLAGS: 00010246   CONTEXT: hvm
> guest (d0v0)
> (XEN) [2016-01-31 19:28:09] rax: 000d   rbx:
> f100054c   rcx: 9e9f
> (XEN) [2016-01-31 19:28:09] rdx:    rsi:
> 0100   rdi: 81e0
> (XEN) [2016-01-31 19:28:09] rbp: 880164b57908   rsp:
> 880164b578d8   r8:  88016d88
> (XEN) [2016-01-31 19:28:09] r9:  0241   r10:
>    r11: 0001
> (XEN) [2016-01-31 19:28:09] r12: 0020   r13:
> 88016453aec0   r14: c96c
> (XEN) [2016-01-31 19:28:09] r15: 880164b57a20   cr0:
> 80050033   cr4: 
> (XEN) [2016-01-31 19:28:09] cr3: 01e0b000   cr2: 
> (XEN) [2016-01-31 19:28:09] ds:    es:    fs:    gs: 
> ss:    cs: 0010
> (XEN) [2016-01-31 19:28:09] Guest stack trace from rsp=880164b578d8:
> (XEN) [2016-01-31 19:28:09]   Fault while accessing guest memory.
> (XEN) [2016-01-31 19:28:09] Hardware Dom0 crashed: rebooting machine in
> 5 seconds.
> ...
> 
> Removing the dom0pvh gets me back up & running.

You will have to post the full boot log (Xen + Linux), there's not
enough information here to diagnose the issue.

Roger.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[Boris Ostrovsky]

On 02/01/2016 05:28 AM, Roger Pau Monné wrote:
IIRC Boris (CCed) added support for 32bit PVH to Linux, so you should 
be able to use either 32 or 64 kernels. Roger. 


This actually never happened for Linux: HVMlite showed up fast enough 
that it didn't make sense anymore to add 32-bit support to Linux 
(especially given that AMD was still not supported).


-boris

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[PGNet Dev]

I'll get the dom0pvh issue logs posted.


http://xenbits.xen.org/docs/unstable/misc/pvh-readme.txt

" ...
To boot 64bit dom0 in PVH mode, add dom0pvh to grub xen command line.
..."

http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=docs/misc/pvh.markdown

no mention of dom0pvh ...

http://wiki.xenproject.org/wiki/Linux_PVH

"...
and use on the Xen command line the dom0pvh=1 bootup parameter.
..."

As per my OP, and the 1st document above, I'd added

dom0pvh

to the grub config.

After re-grub2-mkconfig, and reboot -> crash

Changing, per the wiki, to

dom0pvh=1

unfortunately makes no difference. (It'll be useful to get those 3 docs 
consistent in usage).


In any case, here's the serial output with debug on


Loading Xen 4.6.0_08-405 with Linux 4.4.0-8.g9f68b90-default ...Loading 
Xen 4.6.0_08-405 with Linux 4.4.0-8.g9f68b90-default ...


/EndEntire
/EndEntire
file path: file path: 
/ACPI(a0341d0,0)/ACPI(a0341d0,0)/PCI(1,1c)/PCI(1,1c)/PCI(0,0)/PCI(0,0)/PCI(0,1)/PCI(0,1)/PCI(0,0)/PCI(0,0)/HardwareVendor

(cf31fac5-c24e-11d2-85f3-00a0c93ec93b)[1:
/HardwareVendor(cf31fac5-c24e-11d2-85f3-00a0c93ec93b)[1: 88 88 
]]/HD(2,1000,96000,c5cc9661271ee648

,2,2)/HD(2,1000,96000,c5cc9661271ee648,2,2)/File(\EFI\OPENSUSE)
/File(\EFI\OPENSUSE)/File(xen-4.6.0_08-405.efi)/File(xen-4.6.0_08-405.efi)/EndEntire
/EndEntire
Xen 4.6.0_08-405 (c/s ) EFI loader
Using configuration file 'xen-4.6.0_08-405.cfg'
vmlinuz-4.4.0-8.g9f68b90-default: 0x8bf22000-0x8c507000
initrd-4.4.0-8.g9f68b90-default: 0x8afbc000-0x8bf21da8
0x:0x00:0x19.0x0: ROM: 0x1 bytes at 0x92a26018
0x:0x03:0x00.0x0: ROM: 0x8000 bytes at 0x92a1d018
0x:0x0f:0x00.0x0: ROM: 0x10800 bytes at 0x929fc018
 __  ___  ______ ___   ___ _  ____  
 \ \/ /___ _ __   | || |  / /_  / _ \   / _ \ ( _ )   | || |  / _ \| ___|
  \  // _ \ '_ \  | || |_| '_ \| | | | | | | |/ _ \ __| || |_| | | |___ \
  /  \  __/ | | | |__   _| (_) | |_| | | |_| | (_) |__|__   _| |_| |___) |
 /_/\_\___|_| |_||_|(_)___(_)___/___\___/ \___/  |_|  \___/|/
   |_|
(XEN) Xen version 4.6.0_08-405 (abu...@suse.de) (gcc (SUSE Linux) 4.8.5) 
debug=n Wed Jan 27 15:23:26 UTC 2016

(XEN) Latest ChangeSet:
(XEN) Console output is synchronous.
(XEN) Bootloader: EFI
(XEN) Command line: dom0pvh=1 dom0_mem=4096M,max:4096M dom0_max_vcpus=1 
dom0_vcpus_pin=true cpuidle=1 cpufreq=xen clocksource=hpet iommu=verbose 
sched=credit vga=gfx-1920x1080x16 com1=1152

(XEN) Video information:
(XEN)  VGA is graphics mode 800x600, 32 bpp
(XEN) Disc information:
(XEN)  Found 0 MBR signatures
(XEN)  Found 6 EDD information structures
(XEN) EFI RAM map:
(XEN)   - 8000 (reserved)
(XEN)  8000 - 00048000 (usable)
(XEN)  00048000 - 00059000 (reserved)
(XEN)  00059000 - 0005f000 (usable)
(XEN)  0005f000 - 000a (reserved)
(XEN)  0010 - 8d93e000 (usable)
(XEN)  8d93e000 - 8d945000 (ACPI NVS)
(XEN)  8d945000 - 8e25a000 (reserved)
(XEN)  8e25a000 - 8e26 (usable)
(XEN)  8e26 - 8e6a1000 (reserved)
(XEN)  8e6a1000 - 91783000 (usable)
(XEN)  91783000 - 919eb000 (reserved)
(XEN)  919eb000 - 91a1e000 (usable)
(XEN)  91a1e000 - 91a7b000 (reserved)
(XEN)  91a7b000 - 91ae1000 (usable)
(XEN)  91ae1000 - 91b87000 (reserved)
(XEN)  91b87000 - 91bbb000 (usable)
(XEN)  91bbb000 - 91bbc000 (reserved)
(XEN)  91bbc000 - 91bbe000 (usable)
(XEN)  91bbe000 - 91bbf000 (reserved)
(XEN)  91bbf000 - 91bc7000 (usable)
(XEN)  91bc7000 - 91bca000 (reserved)
(XEN)  91bca000 - 91bdd000 (usable)
(XEN)  91bdd000 - 91be8000 (reserved)
(XEN)  91be8000 - 91c69000 (usable)
(XEN)  91c69000 - 91ce6000 (reserved)
(XEN)  91ce6000 - 91d3 (usable)
(XEN)  91d3 - 9208e000 (reserved)
(XEN)  9208e000 - 920d5000 (usable)
(XEN)  920d5000 - 9214b000 (reserved)
(XEN)  9214b000 - 9217e000 (usable)
(XEN)  9217e000 - 92296000 (reserved)
(XEN)  92296000 - 922b5000 (usable)
(XEN)  922b5000 - 92359000 (reserved)
(XEN)  92359000 - 92363000 (usable)
(XEN)  92363000 - 92462000 (reserved)
(XEN)  92462000 - 92467000 (usable)
(XEN)  92467000 - 925b6000 (reserved)
(XEN)  925b6000 - 925b8000 (usable)
(XEN)  925b8000 - 925be000 (reserved)
(XEN)  925be000 - 925c (usable)
(XEN)  925c - 925c6000 (reserved)

Re: [Xen-devel] Clarifying PVH mode requirements

[Jan Beulich]

>>> On 01.02.16 at 15:54,  wrote:
> On 02/01/2016 08:38 AM, PGNet Dev wrote:
>>
>> Loading Xen 4.6.0_08-405 with Linux 4.4.0-8.g9f68b90-default 
>> ...Loading Xen 4.6.0_08-405 with Linux 4.4.0-8.g9f68b90-default ...
>>
>> /EndEntire
>> /EndEntire
>> file path: file path: 
>> 
> /ACPI(a0341d0,0)/ACPI(a0341d0,0)/PCI(1,1c)/PCI(1,1c)/PCI(0,0)/PCI(0,0)/PCI(0,
> 1)/PCI(0,1)/PCI(0,0)/PCI(0,0)/HardwareVendor
>> (cf31fac5-c24e-11d2-85f3-00a0c93ec93b)[1:
>> /HardwareVendor(cf31fac5-c24e-11d2-85f3-00a0c93ec93b)[1: 88 88 
>> ]]/HD(2,1000,96000,c5cc9661271ee648
>> ,2,2)/HD(2,1000,96000,c5cc9661271ee648,2,2)/File(\EFI\OPENSUSE)
>> 
> /File(\EFI\OPENSUSE)/File(xen-4.6.0_08-405.efi)/File(xen-4.6.0_08-405.efi)/EndEnt
> ire 
>>
>> /EndEntire
>> Xen 4.6.0_08-405 (c/s ) EFI loader
>> Using configuration file 'xen-4.6.0_08-405.cfg'
>> vmlinuz-4.4.0-8.g9f68b90-default: 0x8bf22000-0x8c507000
>> initrd-4.4.0-8.g9f68b90-default: 0x8afbc000-0x8bf21da8
>> 0x:0x00:0x19.0x0: ROM: 0x1 bytes at 0x92a26018
>> 0x:0x03:0x00.0x0: ROM: 0x8000 bytes at 0x92a1d018
>> 0x:0x0f:0x00.0x0: ROM: 0x10800 bytes at 0x929fc018
>>  __  ___  ______ ___   ___ _  _ ___  
>>  \ \/ /___ _ __   | || |  / /_  / _ \   / _ \ ( _ )   | || |  / _ \| ___|
>>   \  // _ \ '_ \  | || |_| '_ \| | | | | | | |/ _ \ __| || |_| | | |___ \
>>   /  \  __/ | | | |__   _| (_) | |_| | | |_| | (_) |__|__   _| |_| 
>> |___) |
>>  /_/\_\___|_| |_||_|(_)___(_)___/___\___/ \___/  |_| \___/|/
>>|_|
>> (XEN) Xen version 4.6.0_08-405 (abu...@suse.de) (gcc (SUSE Linux) 
>> 4.8.5) debug=n Wed Jan 27 15:23:26 UTC 2016
>>
> 
> 
> 
>> (XEN) [2016-02-01 05:28:29] d0v0 EPT violation 0x1aa (-w-/r-x) gpa 
>> 0x00f100054c mfn 0xf1000 type 5
>> (XEN) [2016-02-01 05:28:29] d0v0 Walking EPT tables for GFN f1000:
>> (XEN) [2016-02-01 05:28:29] d0v0  epte 80084510c107
>> (XEN) [2016-02-01 05:28:29] d0v0  epte 80085b684107
>> (XEN) [2016-02-01 05:28:29] d0v0  epte 800844afb107
>> (XEN) [2016-02-01 05:28:29] d0v0  epte 8050f1000905
>> (XEN) [2016-02-01 05:28:29] d0v0  --- GLA 0xc96a254c
>> (XEN) [2016-02-01 05:28:29] domain_crash called from vmx.c:2685
>> (XEN) [2016-02-01 05:28:29] Domain 0 (vcpu#0) crashed on cpu#0:
>> (XEN) [2016-02-01 05:28:29] [ Xen-4.6.0_08-405  x86_64 debug=n 
>> Tainted:C ]
>> (XEN) [2016-02-01 05:28:29] CPU:0
>> (XEN) [2016-02-01 05:28:29] RIP: 0010:[]
>> (XEN) [2016-02-01 05:28:29] RFLAGS: 00010246   CONTEXT: hvm 
>> guest (d0v0)
>> (XEN) [2016-02-01 05:28:29] rax: 000d   rbx: 
>> f100054c   rcx: 9e9c4fff
>> (XEN) [2016-02-01 05:28:29] rdx:    rsi: 
>> 0100   rdi: 81eaedc0
>> (XEN) [2016-02-01 05:28:29] rbp: 880164b57908   rsp: 
>> 880164b578d8   r8:  88016d8c7458
>> (XEN) [2016-02-01 05:28:29] r9:  0241   r10: 
>>    r11: 2001
>> (XEN) [2016-02-01 05:28:29] r12: 0020   r13: 
>> 88016453aec0   r14: c96a254c
>> (XEN) [2016-02-01 05:28:29] r15: 880164b57a20   cr0: 
>> 80050033   cr4: 000406b0
>> (XEN) [2016-02-01 05:28:29] cr3: 01e0b000   cr2: 
>> (XEN) [2016-02-01 05:28:29] ds:    es:    fs:    gs:  
>> ss:    cs: 0010
>> (XEN) [2016-02-01 05:28:29] Guest stack trace from rsp=880164b578d8:
>> (XEN) [2016-02-01 05:28:29]   Fault while accessing guest memory.
>> (XEN) [2016-02-01 05:28:29] Hardware Dom0 crashed: rebooting machine 
>> in 5 seconds.
> 
> (+Jan)
> 
> This looks very much like it needs backport of 33c19df9a ("x86/PCI: 
> intercept accesses to RO MMIO from dom0s in HVM containers") from 
> unstable, which fixes PVH regression introduced by 9256f66c1606 
> ("x86/PCI: intercept all PV Dom0 MMCFG writes")

I don't really understand: The former was needed to fix an issue
introduced by the latter, but the latter isn't in 4.6 iirc.

Jan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[PGNet Dev]

On 02/01/2016 02:30 AM, Roger Pau Monné wrote:

Does your kernel support PVH mode (ie: CONFIG_PVH enabled?)


not CONFIG_PVH, but per http://wiki.xenproject.org/wiki/Linux_PVH

egrep \

"CONFIG_HYPERVISOR_GUEST=|CONFIG_PARAVIRT=|CONFIG_PARAVIRT_GUEST=|CONFIG_PARAVIRT_SPINLOCKS=|CONFIG_XEN=|CONFIG_XEN_PVH=" 
\

 /boot/config-4.4.0-8.g9f68b90-default
CONFIG_HYPERVISOR_GUEST=y
CONFIG_PARAVIRT=y
CONFIG_PARAVIRT_SPINLOCKS=y
CONFIG_XEN=y
CONFIG_XEN_PVH=y

note, there's no 'CONFIG_PARAVIRT_GUEST=', which appears an outdated reqt,

  http://marc.info/?l=linux-pm=136862383726154=2

as CONFIG_PARAVIRT_GUEST was replaced with CONFIG_HYPERVISOR_GUEST


You will have to post the full boot log (Xen + Linux), there's not
enough information here to diagnose the issue.


I'll get more detail posted.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[PGNet Dev]

On 02/01/2016 01:59 AM, Wei Liu wrote:

(Cc Roger)

On Sun, Jan 31, 2016 at 01:27:23PM -0800, PGNet Dev wrote:

I run Xen 4.6 Dom0

rpm -qa | egrep -i "kernel-default-4|xen-4"
kernel-default-devel-4.4.0-8.1.g9f68b90.x86_64
xen-4.6.0_08-405.1.x86_64

My guests are currently HVM in PVHVM mode; I'm exploring PVH.

IIUC, for 4.6, this doc

http://xenbits.xen.org/docs/4.6-testing/misc/pvh-readme.txt



AIUI that document is not very up to date.

In the mean time. There is another document in the same directory called
pvh.markdown that you can have a look.


There's no

http://xenbits.xen.org/docs/4.6-testing/misc/pvh.markdown

but there is,

(1) http://xenbits.xen.org/docs/unstable/misc/pvh.html

which also looks 'dusty'

This looks more promising,

(2) https://github.com/mirage/xen/blob/master/docs/misc/pvh.markdown

That the one?

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[Roger Pau Monné]

El 01/02/16 a les 13.23, PGNet Dev ha escrit:
> On 02/01/2016 01:59 AM, Wei Liu wrote:
> 
> (1) http://xenbits.xen.org/docs/unstable/misc/pvh.html
> 
> which also looks 'dusty'
> 
> This looks more promising,
> 
> (2) https://github.com/mirage/xen/blob/master/docs/misc/pvh.markdown
> 
> That the one?

Hm, maybe you haven't realized, but the content in (1) and (2) it's
exactly the same. They are just different renderings of the same
original document, which resides at:

http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=docs/misc/pvh.markdown

And is the document that Wei was referring to.

Roger.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[Wei Liu]

On Mon, Feb 01, 2016 at 04:23:46AM -0800, PGNet Dev wrote:
> On 02/01/2016 01:59 AM, Wei Liu wrote:
> >(Cc Roger)
> >
> >On Sun, Jan 31, 2016 at 01:27:23PM -0800, PGNet Dev wrote:
> >>I run Xen 4.6 Dom0
> >>
> >>rpm -qa | egrep -i "kernel-default-4|xen-4"
> >>kernel-default-devel-4.4.0-8.1.g9f68b90.x86_64
> >>xen-4.6.0_08-405.1.x86_64
> >>
> >>My guests are currently HVM in PVHVM mode; I'm exploring PVH.
> >>
> >>IIUC, for 4.6, this doc
> >>
> >>http://xenbits.xen.org/docs/4.6-testing/misc/pvh-readme.txt
> >>
> >
> >AIUI that document is not very up to date.
> >
> >In the mean time. There is another document in the same directory called
> >pvh.markdown that you can have a look.
> 
> There's no
> 
> http://xenbits.xen.org/docs/4.6-testing/misc/pvh.markdown
> 
> but there is,
> 
> (1) http://xenbits.xen.org/docs/unstable/misc/pvh.html
> 
> which also looks 'dusty'
> 
> This looks more promising,
> 
> (2) https://github.com/mirage/xen/blob/master/docs/misc/pvh.markdown
> 

I think

 (1) http://xenbits.xen.org/docs/unstable/misc/pvh.html

is generate from

 (2) https://github.com/mirage/xen/blob/master/docs/misc/pvh.markdown
  
In any case, yes, I was referring to (2).

Wei.

> That the one?

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[PGNet Dev]

On 02/01/2016 04:29 AM, Roger Pau Monné wrote:

http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=docs/misc/pvh.markdown


That's all sorted now, thanks.

I'll get the dom0pvh issue logs posted.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[PGNet Dev]

On 02/01/2016 02:28 AM, Roger Pau Monné wrote:

 Do any of these^^ params need to also change with the addition of

 pvh = 1


Yes, you need to remove builder, xen_platform_pci and
device_model_version, and add a kernel and ramdisk parameters that point
to the actual kernel and ramdisk that you want to use. The file should
look like:


Got it now.  It's a PVH config to start, not an HVM -- different than 
PVHVM ...



But checking cpu,

 hwinfo --cpu | egrep "Arch|Model"
   Arch: X86-64
   Model: 6.60.3 "Intel(R) Xeon(R) CPU E3-1220 v3 @ 3.10GHz"


You CPU is perfectly capable of running both a PVH Dom0 or DomU, check:

http://ark.intel.com/products/52269/Intel-Xeon-Processor-E3-1220-8M-Cache-3_10-GHz

Look for EPT and VT-d which are the main requirements for PVH.


Clear enough.  Guess it's presumed in 'modern' VT-x; odd that cat 
/proc/cpuinfo provides no indication


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[Boris Ostrovsky]

On 02/01/2016 02:27 PM, PGNet Dev wrote:

On 02/01/2016 11:14 AM, Boris Ostrovsky wrote:

Is 'HVMLite' replacing 'PVH'? Or are they separate modes?


Yes, HVMlite is replacing PVH. Probably once we get dom0 support.


If that's a 'done deal', and it sounds like it is, it'd be useful to 
have it integrated into:


http://wiki.xen.org/wiki/Understanding_the_Virtualization_Spectrum

particularly as there's no mention of HVMlite on the wiki, at all.


HVMlite is very new: domU hypervisor support has been added less than 
two months ago and Linux patches are being reviewed as we speak 
(FreeBSD, I believe, is supported).




It's unclear whether PVH, in its current dev state (at least here), is 
worth-the-visit -- especially if HVMlite is "ComingSoon(tm)".


I suppose I'm looking for some guidance as to which to invest time in 
while on Xen 4.6.0, ack'ing that neither PVH nor HVMlite are 
production-ready.


Current PVH implementation has never been described as production-ready. 
What is happening now with HVMlite is essentially bringing PVH to 
production-quality level.


-boris

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[Boris Ostrovsky]

On 02/01/2016 08:38 AM, PGNet Dev wrote:


Loading Xen 4.6.0_08-405 with Linux 4.4.0-8.g9f68b90-default 
...Loading Xen 4.6.0_08-405 with Linux 4.4.0-8.g9f68b90-default ...


/EndEntire
/EndEntire
file path: file path: 
/ACPI(a0341d0,0)/ACPI(a0341d0,0)/PCI(1,1c)/PCI(1,1c)/PCI(0,0)/PCI(0,0)/PCI(0,1)/PCI(0,1)/PCI(0,0)/PCI(0,0)/HardwareVendor

(cf31fac5-c24e-11d2-85f3-00a0c93ec93b)[1:
/HardwareVendor(cf31fac5-c24e-11d2-85f3-00a0c93ec93b)[1: 88 88 
]]/HD(2,1000,96000,c5cc9661271ee648

,2,2)/HD(2,1000,96000,c5cc9661271ee648,2,2)/File(\EFI\OPENSUSE)
/File(\EFI\OPENSUSE)/File(xen-4.6.0_08-405.efi)/File(xen-4.6.0_08-405.efi)/EndEntire 


/EndEntire
Xen 4.6.0_08-405 (c/s ) EFI loader
Using configuration file 'xen-4.6.0_08-405.cfg'
vmlinuz-4.4.0-8.g9f68b90-default: 0x8bf22000-0x8c507000
initrd-4.4.0-8.g9f68b90-default: 0x8afbc000-0x8bf21da8
0x:0x00:0x19.0x0: ROM: 0x1 bytes at 0x92a26018
0x:0x03:0x00.0x0: ROM: 0x8000 bytes at 0x92a1d018
0x:0x0f:0x00.0x0: ROM: 0x10800 bytes at 0x929fc018
 __  ___  ______ ___   ___ _  _ ___  
 \ \/ /___ _ __   | || |  / /_  / _ \   / _ \ ( _ )   | || |  / _ \| ___|
  \  // _ \ '_ \  | || |_| '_ \| | | | | | | |/ _ \ __| || |_| | | |___ \
  /  \  __/ | | | |__   _| (_) | |_| | | |_| | (_) |__|__   _| |_| 
|___) |

 /_/\_\___|_| |_||_|(_)___(_)___/___\___/ \___/  |_| \___/|/
   |_|
(XEN) Xen version 4.6.0_08-405 (abu...@suse.de) (gcc (SUSE Linux) 
4.8.5) debug=n Wed Jan 27 15:23:26 UTC 2016






(XEN) [2016-02-01 05:28:29] d0v0 EPT violation 0x1aa (-w-/r-x) gpa 
0x00f100054c mfn 0xf1000 type 5

(XEN) [2016-02-01 05:28:29] d0v0 Walking EPT tables for GFN f1000:
(XEN) [2016-02-01 05:28:29] d0v0  epte 80084510c107
(XEN) [2016-02-01 05:28:29] d0v0  epte 80085b684107
(XEN) [2016-02-01 05:28:29] d0v0  epte 800844afb107
(XEN) [2016-02-01 05:28:29] d0v0  epte 8050f1000905
(XEN) [2016-02-01 05:28:29] d0v0  --- GLA 0xc96a254c
(XEN) [2016-02-01 05:28:29] domain_crash called from vmx.c:2685
(XEN) [2016-02-01 05:28:29] Domain 0 (vcpu#0) crashed on cpu#0:
(XEN) [2016-02-01 05:28:29] [ Xen-4.6.0_08-405  x86_64 debug=n 
Tainted:C ]

(XEN) [2016-02-01 05:28:29] CPU:0
(XEN) [2016-02-01 05:28:29] RIP: 0010:[]
(XEN) [2016-02-01 05:28:29] RFLAGS: 00010246   CONTEXT: hvm 
guest (d0v0)
(XEN) [2016-02-01 05:28:29] rax: 000d   rbx: 
f100054c   rcx: 9e9c4fff
(XEN) [2016-02-01 05:28:29] rdx:    rsi: 
0100   rdi: 81eaedc0
(XEN) [2016-02-01 05:28:29] rbp: 880164b57908   rsp: 
880164b578d8   r8:  88016d8c7458
(XEN) [2016-02-01 05:28:29] r9:  0241   r10: 
   r11: 2001
(XEN) [2016-02-01 05:28:29] r12: 0020   r13: 
88016453aec0   r14: c96a254c
(XEN) [2016-02-01 05:28:29] r15: 880164b57a20   cr0: 
80050033   cr4: 000406b0

(XEN) [2016-02-01 05:28:29] cr3: 01e0b000   cr2: 
(XEN) [2016-02-01 05:28:29] ds:    es:    fs:    gs:  
ss:    cs: 0010

(XEN) [2016-02-01 05:28:29] Guest stack trace from rsp=880164b578d8:
(XEN) [2016-02-01 05:28:29]   Fault while accessing guest memory.
(XEN) [2016-02-01 05:28:29] Hardware Dom0 crashed: rebooting machine 
in 5 seconds.


(+Jan)

This looks very much like it needs backport of 33c19df9a ("x86/PCI: 
intercept accesses to RO MMIO from dom0s in HVM containers") from 
unstable, which fixes PVH regression introduced by 9256f66c1606 
("x86/PCI: intercept all PV Dom0 MMCFG writes")


-boris

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[PGNet Dev]

On 02/01/2016 06:11 AM, Boris Ostrovsky wrote:

This actually never happened for Linux: HVMlite showed up fast enough
that it didn't make sense anymore to add 32-bit support to Linux
(especially given that AMD was still not supported).


Is 'HVMLite' replacing 'PVH'? Or are they separate modes?

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] Clarifying PVH mode requirements

[PGNet Dev]

I run Xen 4.6 Dom0

rpm -qa | egrep -i "kernel-default-4|xen-4"
kernel-default-devel-4.4.0-8.1.g9f68b90.x86_64
xen-4.6.0_08-405.1.x86_64

My guests are currently HVM in PVHVM mode; I'm exploring PVH.

IIUC, for 4.6, this doc

http://xenbits.xen.org/docs/4.6-testing/misc/pvh-readme.txt

instructs the following necessary changes:

@ GRUBG cfg

-   GRUB_CMDLINE_XEN=" ..."
+   GRUB_CMDLINE_XEN=" dom0pvh ..."

&, @ guest.cfg

+   pvh = 1

For my guest.cfg, currently in PVHVM mode, I have

builder = 'hvm'
xen_platform_pci = 1
device_model_version="qemu-xen"
hap = 1
...

Q:
Do any of these^^ params need to also change with the addition of

pvh = 1



At the moment HAP is required for PVH.


As above, I've 'hap = 1' enabled.

But checking cpu,

hwinfo --cpu | egrep "Arch|Model"
  Arch: X86-64
  Model: 6.60.3 "Intel(R) Xeon(R) CPU E3-1220 v3 @ 3.10GHz"

neither 'hap' nor 'emt' are specifically called out,

egrep -wo 'vmx|lm|aes' /proc/cpuinfo  | sort | uniq \
 | sed -e 's/aes/Hardware encryption=Yes (&)/g' \
	 -e 's/lm/64 bit cpu=Yes (&)/g' -e 's/vmx/Intel hardware 
virtualization=Yes (&)/g'


Hardware encryption=Yes (aes)
64 bit cpu=Yes (lm)
	egrep -wo 'hap|vmx|ept|vpid|npt|tpr_shadow|flexpriority|vnmi|lm|aes' 
/proc/cpuinfo  | sort | uniq

aes
lm

Iiuc, Intel introduced EPT with Nehalem arch, which preceds Haswell by ~ 
5 years.


Q:
	Am I out of luck re: PVH with more modern Haswell? Or is there a 
different check I should be running ?



At present the only PVH guest is an x86 64bit PV linux.


Is this still current/true info?

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Clarifying PVH mode requirements

[PGNet Dev]

In any case, the !st issue, prior to any guest being launched, simply adding


@ GRUBG cfg

-GRUB_CMDLINE_XEN=" ..."
+GRUB_CMDLINE_XEN=" dom0pvh ..."


causes boot fail,

...
(XEN) [2016-01-31 19:28:09] d0v0 EPT violation 0x1aa (-w-/r-x) gpa 
0x00f100054c mfn 0xf15

(XEN) [2016-01-31 19:28:09] d0v0 Walking EPT tables for GFN f1000:
(XEN) [2016-01-31 19:28:09] d0v0  epte 800845108107
(XEN) [2016-01-31 19:28:09] d0v0  epte 80085b680107
(XEN) [2016-01-31 19:28:09] d0v0  epte 800844af7107
(XEN) [2016-01-31 19:28:09] d0v0  epte 8050f1000905
(XEN) [2016-01-31 19:28:09] d0v0  --- GLA 0xc96a254c
(XEN) [2016-01-31 19:28:09] domain_crash called from vmx.c:2685
(XEN) [2016-01-31 19:28:09] Domain 0 (vcpu#0) crashed on cpu#0:
(XEN) [2016-01-31 19:28:09] [ Xen-4.6.0_08-405  x86_64  debug=n 
Tainted:C ]

(XEN) [2016-01-31 19:28:09] CPU:0
(XEN) [2016-01-31 19:28:09] RIP:0010:[]
(XEN) [2016-01-31 19:28:09] RFLAGS: 00010246   CONTEXT: hvm 
guest (d0v0)
(XEN) [2016-01-31 19:28:09] rax: 000d   rbx: 
f100054c   rcx: 9e9f
(XEN) [2016-01-31 19:28:09] rdx:    rsi: 
0100   rdi: 81e0
(XEN) [2016-01-31 19:28:09] rbp: 880164b57908   rsp: 
880164b578d8   r8:  88016d88
(XEN) [2016-01-31 19:28:09] r9:  0241   r10: 
   r11: 0001
(XEN) [2016-01-31 19:28:09] r12: 0020   r13: 
88016453aec0   r14: c96c
(XEN) [2016-01-31 19:28:09] r15: 880164b57a20   cr0: 
80050033   cr4: 

(XEN) [2016-01-31 19:28:09] cr3: 01e0b000   cr2: 
(XEN) [2016-01-31 19:28:09] ds:    es:    fs:    gs:  
ss:    cs: 0010

(XEN) [2016-01-31 19:28:09] Guest stack trace from rsp=880164b578d8:
(XEN) [2016-01-31 19:28:09]   Fault while accessing guest memory.
(XEN) [2016-01-31 19:28:09] Hardware Dom0 crashed: rebooting machine in 
5 seconds.

...

Removing the dom0pvh gets me back up & running.



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel