Re: [Xen-devel] [PATCH 2/2] x86/vm_event: Allow returning i-cache for emulation

[Jan Beulich]

>>> On 14.09.16 at 18:20,  wrote:
> On Wed, Sep 14, 2016 at 9:55 AM, Jan Beulich  wrote:
> On 13.09.16 at 20:12,  wrote:
>>> When emulating instructions the emulator maintains a small i-cache fetched
>>> from the guest memory. This patch extends the vm_event interface to allow
>>> returning this i-cache via the vm_event response instead.
>>
>> I guess I'm sightly confused: Isn't the purpose to have the monitoring
>> app _write_ to the cache maintained in Xen? Or else, who is
>> "emulator" here? If that's the app, then I think subject and description
>> for hypervisor patches would better be written taking the perspective
>> of the hypervisor, especially when using potentially ambiguous terms.
> 
> Well, I thought it was obvious that the built-in emulator in Xen is
> what this patch is referring to. Otherwise none of this makes sense.

It would have been if the sentence didn't say "returning". The
internal emulator's cache gets effectively overwritten, not
returned to anything (unless I'm still misunderstanding something).

>>> @@ -1944,11 +1951,19 @@ void hvm_mem_access_emulate_one(enum emul_kind 
>>> kind, unsigned int trapnr,
>>>  case EMUL_KIND_NOWRITE:
>>>  rc = hvm_emulate_one_no_write();
>>>  break;
>>> -case EMUL_KIND_SET_CONTEXT:
>>> -ctx.set_context = 1;
>>> -/* Intentional fall-through. */
>>> -default:
>>> +case EMUL_KIND_SET_CONTEXT_DATA:
>>> +ctx.set_context_data = 1;
>>> +rc = hvm_emulate_one();
>>> +break;
>>> +case EMUL_KIND_SET_CONTEXT_INSN:
>>> +ctx.set_context_insn = 1;
>>>  rc = hvm_emulate_one();
>>> +break;
>>> +case EMUL_KIND_NORMAL:
>>> +rc = hvm_emulate_one();
>>> +break;
>>
>> One of the former two surely can be made fall through into the latter?
> 
> That's what I did before by turning this into if-else's and you
> requested to go back to a switch. With a switch though I'll rather
> make the cases explicit as a fall-through just makes it harder to read
> for no real benefit.

I disagree.

>>> +default:
>>> +return;
>>
>> Why don't you retain the previous default handling?
> 
> The default label is unused and this makes it more apparent when
> reading the code.

Just like before, imo there shouldn't be any EMUL_KIND_NORMAL
case; that should be the default label instead.

>>> @@ -265,6 +272,10 @@ struct vm_event_emul_read_data {
>>>  uint8_t  data[sizeof(struct vm_event_regs_x86) - sizeof(uint32_t)];
>>>  };
>>>
>>> +struct vm_event_emul_insn_data {
>>> +uint8_t data[16]; /* Has to be completely filled */
>>> +};
>>
>> Any reason for the 16 (rather than the architectural 15) here?
> 
> Yes, see the definition in include/asm-x86/hvm/emulate.h:
> 
> struct hvm_emulate_ctxt {
> struct x86_emulate_ctxt ctxt;
> 
> /* Cache of 16 bytes of instruction. */
> uint8_t insn_buf[16];

Ah, I didn't recall we have an oversized cache there too. But such a
connection would better be documented by a BUILD_BUG_ON()
comparing the two sizeof()s.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 2/3] x86 Test and expose CPUID faulting capabilities in /proc/cpuinfo

[Kyle Huey]

On Wed, Sep 14, 2016 at 6:17 PM, Andy Lutomirski  wrote:
> On Wed, Sep 14, 2016 at 3:03 PM, Kyle Huey  wrote:
>> On Wed, Sep 14, 2016 at 2:35 PM, Dave Hansen
>>  wrote:
>>> On 09/14/2016 02:01 PM, Kyle Huey wrote:
>
>>> Is any of this useful to optimize away at compile-time?  We have config
>>> options for when we're running as a guest, and this seems like a feature
>>> that isn't available when running on bare metal.
>>
>> On the contrary, this is only available when we're on bare metal.
>> Neither Xen nor KVM virtualize CPUID faulting (although KVM correctly
>> suppresses MSR_PLATFORM_INFO's report of support for it).
>
> KVM could easily support this.  If rr starts using it, I think KVM
> *should* add support, possibly even for older CPUs that don't support
> the feature in hardware.
>
> It's too bad that x86 doesn't give us the instruction bytes on a
> fault.  Otherwise we could lazily switch this feature.

We are *very* interested in having KVM and Xen support virtualization
of this feature.  I am planning to work on KVM after I get this series
of patches in :)

- Kyle

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 2/3] x86 Test and expose CPUID faulting capabilities in /proc/cpuinfo

[Kyle Huey]

On Wed, Sep 14, 2016 at 2:35 PM, Dave Hansen
 wrote:
> On 09/14/2016 02:01 PM, Kyle Huey wrote:
>> Xen advertises the underlying support for CPUID faulting but not does pass
>> through writes to the relevant MSR, nor does it virtualize it, so it does
>> not actually work. For now mask off the relevant bit on MSR_PLATFORM_INFO.
>
> That needs to make it into a comment, please.
>
> That *is* a Xen bug, right?

Yes.  Xen needs to either not advertise the feature or actually
support it.  This came up in the prior thread ("[PATCH] prctl,x86 Add
PR_[GET|SET]_CPUID for controlling the CPUID instruction.").

>> Signed-off-by: Kyle Huey 
>> ---
>>  arch/x86/include/asm/cpufeatures.h |  1 +
>>  arch/x86/include/asm/msr-index.h   |  1 +
>>  arch/x86/kernel/cpu/scattered.c| 14 ++
>>  arch/x86/xen/enlighten.c   |  3 +++
>>  4 files changed, 19 insertions(+)
>>
>> diff --git a/arch/x86/include/asm/cpufeatures.h 
>> b/arch/x86/include/asm/cpufeatures.h
>> index 92a8308..78b9d06 100644
>> --- a/arch/x86/include/asm/cpufeatures.h
>> +++ b/arch/x86/include/asm/cpufeatures.h
>> @@ -190,6 +190,7 @@
>>
>>  #define X86_FEATURE_CPB  ( 7*32+ 2) /* AMD Core Performance 
>> Boost */
>>  #define X86_FEATURE_EPB  ( 7*32+ 3) /* IA32_ENERGY_PERF_BIAS 
>> support */
>> +#define X86_FEATURE_CPUID_FAULT ( 7*32+ 4) /* Intel CPUID faulting */
>>
>>  #define X86_FEATURE_HW_PSTATE( 7*32+ 8) /* AMD HW-PState */
>>  #define X86_FEATURE_PROC_FEEDBACK ( 7*32+ 9) /* AMD ProcFeedbackInterface */
>> diff --git a/arch/x86/include/asm/msr-index.h 
>> b/arch/x86/include/asm/msr-index.h
>> index 56f4c66..83908d5 100644
>> --- a/arch/x86/include/asm/msr-index.h
>> +++ b/arch/x86/include/asm/msr-index.h
>> @@ -41,6 +41,7 @@
>>  #define MSR_IA32_PERFCTR10x00c2
>>  #define MSR_FSB_FREQ 0x00cd
>>  #define MSR_PLATFORM_INFO0x00ce
>> +#define CPUID_FAULTING_SUPPORT   (1UL << 31)
>>
>>  #define MSR_NHM_SNB_PKG_CST_CFG_CTL  0x00e2
>>  #define NHM_C3_AUTO_DEMOTE   (1UL << 25)
>> diff --git a/arch/x86/kernel/cpu/scattered.c 
>> b/arch/x86/kernel/cpu/scattered.c
>> index 8cb57df..d502da1 100644
>> --- a/arch/x86/kernel/cpu/scattered.c
>> +++ b/arch/x86/kernel/cpu/scattered.c
>> @@ -24,6 +24,17 @@ enum cpuid_regs {
>>   CR_EBX
>>  };
>>
>> +static int supports_cpuid_faulting(void)
>> +{
>> + unsigned int lo, hi;
>> +
>> + if (rdmsr_safe(MSR_PLATFORM_INFO, , ) == 0 &&
>> + (lo & CPUID_FAULTING_SUPPORT))
>> + return 1;
>> + else
>> + return 0;
>> +}
>
> Is any of this useful to optimize away at compile-time?  We have config
> options for when we're running as a guest, and this seems like a feature
> that isn't available when running on bare metal.

On the contrary, this is only available when we're on bare metal.
Neither Xen nor KVM virtualize CPUID faulting (although KVM correctly
suppresses MSR_PLATFORM_INFO's report of support for it).

>> diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
>> index b86ebb1..2c47f0c 100644
>> --- a/arch/x86/xen/enlighten.c
>> +++ b/arch/x86/xen/enlighten.c
>> @@ -1050,6 +1050,9 @@ static u64 xen_read_msr_safe(unsigned int msr, int 
>> *err)
>>  #endif
>>   val &= ~X2APIC_ENABLE;
>>   break;
>> + case MSR_PLATFORM_INFO:
>> + val &= ~CPUID_FAULTING_SUPPORT;
>> + break;
>>   }
>>   return val;
>>  }
>
> Does this mean that Xen guests effectively can't take advantage of this
> feature?

Yes.

- Kyle

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v5 00/16] Xen ARM DomU ACPI support

[Boris Ostrovsky]

- julien.gr...@arm.com wrote:

> Hi Stefano,
> 
> On 14/09/2016 21:48, Stefano Stabellini wrote:
> > On Wed, 14 Sep 2016, Julien Grall wrote:
> >> On 14/09/2016 02:06, Stefano Stabellini wrote:
> >>> On Wed, 14 Sep 2016, Shannon Zhao wrote:
>  On 2016/9/13 23:17, Julien Grall wrote:
> >
> >
> > On 13/09/16 14:06, Shannon Zhao wrote:
> >> Hi Julien,
> >
> > Hello Shannon,
> >
> >> On 2016/9/13 19:56, Julien Grall wrote:
> >>> Hi Shannon,
> >>>
> >>> On 02/09/16 03:55, Shannon Zhao wrote:
>  From: Shannon Zhao 
> 
>  The design of this feature is described as below.
>  Firstly, the toolstack (libxl) generates the ACPI tables
> according
>  the
>  number of vcpus and gic controller.
> 
>  Then, it copies these ACPI tables to DomU non-RAM memory map
> space
>  and
>  passes them to UEFI firmware through the "ARM multiboot"
> protocol.
> 
>  At last, UEFI gets the ACPI tables through the "ARM
> multiboot"
>  protocol
>  and installs these tables like the usual way and passes both
> ACPI
>  and DT
>  information to the Xen DomU.
> 
>  Currently libxl only generates RSDP, XSDT, GTDT, MADT, FADT,
> DSDT
>  tables
>  since it's enough now.
> 
>  This has been tested using guest kernel with the Dom0 ACPI
> support
>  patches which could be fetched from linux master or:
> 
> https://git.kernel.org/cgit/linux/kernel/git/mfleming/efi.git/log/?h=efi/arm-xen
> 
> 
> 
>  The UEFI binary could be fetched from or built from edk2
> master
>  branch:
>  http://people.linaro.org/~shannon.zhao/DomU_ACPI/XEN_EFI.fd
> >>>
> >>> On which commit this EFI binary is based? I am trying to
> rebuild
> >>> myself,
> >>> and go no luck to boot it so far.
> >>>
> >> I forgot the exact commit. But I just tried below commit which
> adds
> >> the
> >> support to edk2 and the guest can boot up successfully with
> ACPI.
> >>
> >> 402dde6 ArmVirtPkg/ArmVirtXen: Add ACPI support for Virt Xen
> ARM
> >
> > Thanks, the commit does not build on my platform. After some
> help for
> > Ard I managed to boot UEFI with the patch [1] applied.
> >
> > However Linux does not boot when passing acpi=on and abort with
> the
> > following message:
> >
> > (d86) 6RCU: Adjusting geometry for rcu_fanout_leaf=64,
> nr_cpu_ids=1
> > (d86) 6NR_IRQS:64 nr_irqs:64 0
> > (d86) 3No valid GICC entries exist
> > (d86) 0Kernel panic - not syncing: No interrupt controller
> found.
> > (d86) dCPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.8.0-rc6+ #420
> > (d86) dHardware name: XENVM-4.8 (DT)
> > (d86) Call trace:
> > (d86) [] dump_backtrace+0x0/0x1a8
> > (d86) [] show_stack+0x14/0x20
> > (d86) [] dump_stack+0x94/0xb8
> > (d86) [] panic+0x10c/0x250
> > (d86) [] init_IRQ+0x24/0x2c
> > (d86) [] start_kernel+0x238/0x394
> > (d86) [] __primary_switched+0x30/0x74
> > (d86) 0---[ end Kernel panic - not syncing: No interrupt
> controller
> > found.
> >
> > This is because the header.length for GICC is not valid for ACPI
> 5.1
> > (see BAD_MADT_GICC_ENTRY). So please check all the size of each
> table
> > against ACPI 5.1.
> >
>  Oops. The reason is that acpi_madt_generic_interrupt in Xen is
> already
>  updated to ACPI 6.0 and the length is 80 not 76 of ACPI 5.1.
>  One solution is that we still use ACPI 5.1 and make
> gicc->header.length
>  76. Other one is that we update to ACPI 6.0 since the Xen ARM
> ACPI
>  support in Linux was introduced after ACPI 6.0.
> 
>  Which one do you prefer?
> >>>
> >>> Certainly the versions of all tables need to be consistent. I
> would
> >>> prefer to have ACPI 6.0 but 5.1 is acceptable too (especially if
> >>> upgrading to 6.0 causes a large amount of changes to your
> patches).
> >>
> >> I disagree on this, we should use the first version of ACPI that is
> fully
> >> supporting ARM because a guest operating system may choose to
> support the
> >> first one (there is a lot hardware platform out which only provides
> ACPI 5.1).
> >
> > And I thought that compatibility was supposed to be ACPI's strong
> suit.
> > I mistakenly had the impression that new ACPI releases weren't
> suppose
> > to break old OSes. I take back my comment, you are right that we
> should
> > stay on 5.1 (including all the erratas, many are important for ARM).
> >
> 
> IIRC, early version of ACPI used to have some incompatibility. I will
> have to go through the ACPI spec to find the main differences between
> 5.1 and 6.0 for ARM.


Transition from 1.x to 2.0 introduced incompatibilities (I believe in RSDP
structure definition) but I thought that since then they kept everything back

Re: [Xen-devel] [PATCH v2 2/2] xen_platform: SUSE xenlinux unplug for emulated PCI

[Stefano Stabellini]

On Fri, 2 Sep 2016, Olaf Hering wrote:
> Implement SUSE specific unplug protocol for emulated PCI devices
> in PVonHVM guests. Its a simple 'outl(1, (ioaddr + 4));'.
> This protocol was implemented and used since Xen 3.0.4.
> It is used in all SUSE/SLES/openSUSE releases up to SLES11SP3 and
> openSUSE 12.3.
> 
> Signed-off-by: Olaf Hering 
> ---
>  hw/i386/xen/xen_platform.c | 31 ++-
>  1 file changed, 30 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/i386/xen/xen_platform.c b/hw/i386/xen/xen_platform.c
> index 53be3c7..6faee4c 100644
> --- a/hw/i386/xen/xen_platform.c
> +++ b/hw/i386/xen/xen_platform.c
> @@ -313,13 +313,42 @@ static void xen_platform_ioport_writeb(void *opaque, 
> hwaddr addr,
> uint64_t val, unsigned int size)
>  {
>  PCIXenPlatformState *s = opaque;
> +PCIDevice *pci_dev = PCI_DEVICE(s);
>  
>  switch (addr) {
>  case 0: /* Platform flags */
>  platform_fixed_ioport_writeb(opaque, 0, (uint32_t)val);
>  break;
> +case 4:
> +if (val == 1) {
> +/*
> + * SUSE unplug for Xenlinux
> + * xen-kmp used this since xen-3.0.4, instead the official 
> protocol
> + * from xen-3.3+ It did an unconditional "outl(1, (ioaddr + 4));"
> + * Pre VMDP 1.7 used 4 and 8 depending on how VMDP was 
> configured.
> + * If VMDP was to control both disk and LAN it would use 4.
> + * If it controlled just disk or just LAN, it would use 8 below.
> + */
> +blk_drain_all();
> +blk_flush_all();
> +pci_unplug_disks(pci_dev->bus);
> +pci_unplug_nics(pci_dev->bus);
> +}
> +break;
>  case 8:
> -log_writeb(s, (uint32_t)val);
> +switch (val) {
> +case 1:
> +blk_drain_all();
> +blk_flush_all();
> +pci_unplug_disks(pci_dev->bus);
> +break;
> +case 2:
> +pci_unplug_nics(pci_dev->bus);
> +break;
> +default:
> +log_writeb(s, (uint32_t)val);
> +break;

The doc says:

"If VMDP was configured to control just NIC devices it would write the
value 0x1 to offset 0x8. If VMDP was configured to control just storage
devices it would write the value 0x2 to offset 0x8."

So 0x1 to 0x8 to unplug NICs, otherwise 0x2 to 0x8 to unplug storage.
The switch above does the opposite. What am I missing? Am I misreading
the document?

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [qemu-mainline baseline-only test] 67712: tolerable FAIL

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 67712 qemu-mainline real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/67712/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail like 67708
 test-amd64-amd64-amd64-pvgrub 10 guest-start  fail  like 67708

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-midway   12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-midway   13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop  fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail never pass

version targeted for testing:
 qemuu507e4ddc3abf67391bcbc9624fd60b969c159b78
baseline version:
 qemuu7263da78045dc91cc207f350911efe4259e99b3c

Last test of basis67708  2016-09-13 23:44:23 Z1 days
Testing same since67712  2016-09-14 19:16:26 Z0 days1 attempts


People who touched revisions under test:
  Alberto Garcia 
  Artyom Tarasenko 
  Bruce Rogers 
  Cao jin 
  Changlong Xie 
  Eric Blake 
  Fam Zheng 
  Gerd Hoffmann 
  Gonglei 
  Jeff Cody 
  Laurent Vivier 
  Laurent Vivier 
  Li Zhijian 
  Michael S. Tsirkin 
  Michael Tokarev 
  Paolo Bonzini 
  Peter Maydell 
  Prasad J Pandit 
  Prasanna Kumar Kalever 
  Riku Voipio 
  Roman Pen 
  Samuel Thibault 
  Stefan Hajnoczi 
  Thomas Huth 
  Wang WeiWei 
  Wen Congyang 
  zhanghailiang 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf   

Re: [Xen-devel] [PATCH v2 1/2] xen_platform: unplug also SCSI disks

[Stefano Stabellini]

On Fri, 2 Sep 2016, Olaf Hering wrote:
> Using 'vdev=sd[a-o]' will create an emulated LSI controller, which can
> be used by the emulated BIOS to boot from disk. If the HVM domU has also
> PV driver the disk may appear twice in the guest. To avoid this an
> unplug of the emulated hardware is needed, similar to what is done for
> IDE and NIC drivers already.
> 
> Since the SCSI controller provides only disks the entire controller can
> be unplugged at once.
> 
> Impact of the change for classic and pvops based guest kernels:
> 
>  vdev=sda:disk0
> before: pvops:   disk0=pv xvda + emulated sda
> classic: disk0=pv sda  + emulated sdq
> after:  pvops:   disk0=pv xvda
> classic: disk0=pv sda
> 
>  vdev=hda:disk0, vdev=sda:disk1
> before: pvops:   disk0=pv xvda
>  disk1=emulated sda
> classic: disk0=pv hda
>  disk1=pv sda  + emulated sdq
> after:  pvops:   disk0=pv xvda
>  disk1=not accessible by blkfront, index hda==index sda
> classic: disk0=pv hda
>  disk1=pv sda
> 
>  vdev=hda:disk0, vdev=sda:disk1, vdev=sdb:disk2
> before: pvops:   disk0=pv xvda
>  disk1=emulated sda
>  disk2=pv xvdb + emulated sdb
> classic: disk0=pv hda
>  disk1=pv sda  + emulated sdq
>  disk2=pv sdb  + emulated sdr
> after:  pvops:   disk0=pv xvda
>  disk1=not accessible by blkfront, index hda==index sda
>  disk2=pv xvdb
> classic: disk0=pv hda
>  disk1=pv sda
>  disk2=pv sda
> 
> Signed-off-by: Olaf Hering 
> ---
>  hw/i386/xen/xen_platform.c | 4 
>  1 file changed, 4 insertions(+)
> 
> diff --git a/hw/i386/xen/xen_platform.c b/hw/i386/xen/xen_platform.c
> index aa78393..53be3c7 100644
> --- a/hw/i386/xen/xen_platform.c
> +++ b/hw/i386/xen/xen_platform.c
> @@ -114,6 +114,10 @@ static void unplug_disks(PCIBus *b, PCIDevice *d, void 
> *o)
>  PCI_CLASS_STORAGE_IDE
>  && strcmp(d->name, "xen-pci-passthrough") != 0) {
>  pci_piix3_xen_ide_unplug(DEVICE(d));
> +} else if (pci_get_word(d->config + PCI_CLASS_DEVICE) ==
> +PCI_CLASS_STORAGE_SCSI
> +&& strcmp(d->name, "xen-pci-passthrough") != 0) {
> +object_unparent(OBJECT(d));
>  }
>  }

Written like this, the code will unplug any Xen SCSI disks together with
Xen IDE disks when the guest writes "1" to ioport `0x10`. I am sorry to
be pedantic, but the recent changes introduced to
docs/misc/hvm-emulated-unplug.markdown do not cover any changes in
behavior to the existing ioport address (I am looking specifically at
point 6).  Sorry to only notice this now.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-4.6-testing test] 100957: tolerable FAIL - PUSHED

[osstest service owner]

flight 100957 xen-4.6-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/100957/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds 15 guest-start/debian.repeatfail  like 100895
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stopfail like 100895
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stopfail like 100895
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop fail like 100895
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop fail like 100895

Tests which did not succeed, but are not blocking:
 test-amd64-i386-rumprun-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-rumprun-amd64  1 build-check(1)   blocked  n/a
 build-amd64-rumprun   5 rumprun-buildfail   never pass
 build-i386-rumprun5 rumprun-buildfail   never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass

version targeted for testing:
 xen  3cffa34537767dfdb6a0fa02c1a54fdfc7644b6d
baseline version:
 xen  6b5bb502a93bcb7617ea1f3f5a8712f2b9f33d90

Last test of basis   100895  2016-09-12 14:15:38 Z2 days
Testing same since   100907  2016-09-13 00:53:24 Z2 days4 attempts


People who touched revisions under test:
  Jan Beulich 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64-xtf  pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-prev pass
 build-i386-prev 

[Xen-devel] [ovmf baseline-only test] 67713: all pass

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 67713 ovmf real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/67713/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf 062f9fd2cfaad09f6dd0e302094bc030827eb706
baseline version:
 ovmf 18b00c8c8773e62b1e2c4a36f490996325570b67

Last test of basis67711  2016-09-14 12:46:15 Z0 days
Testing same since67713  2016-09-14 21:50:50 Z0 days1 attempts


People who touched revisions under test:
  Ard Biesheuvel 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xs.citrite.net/~osstest/testlogs/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


commit 062f9fd2cfaad09f6dd0e302094bc030827eb706
Author: Ard Biesheuvel 
Date:   Fri Sep 9 18:58:56 2016 +0100

ArmPkg/DefaultExceptionHandlerLib: improve formatting of backtrace

Implement the backtrace formattting suggested by Andrew, i.e.,

  IRQ Exception at 0x5BE182B0
  PC 0x5BE182B0 (0x5BE14000+0x42B0) [ 0] ArmCpuDxe.dll
  PC 0x5BE15770 (0x5BE14000+0x1770) [ 0] ArmCpuDxe.dll
  PC 0x5EF08CC8 (0x5EEEB000+0x0001DCC8) [ 1] DxeCore.dll
  PC 0x5EF09008 (0x5EEEB000+0x0001E008) [ 1] DxeCore.dll
  PC 0x5EEF6ACC (0x5EEEB000+0xBACC) [ 1] DxeCore.dll
  PC 0x5EF0963C (0x5EEEB000+0x0001E63C) [ 1] DxeCore.dll
  PC 0x5EF09A38 (0x5EEEB000+0x0001EA38) [ 1] DxeCore.dll
  PC 0x5EF01560 (0x5EEEB000+0x00016560) [ 1] DxeCore.dll
  PC 0x5EF02FB4 (0x5EEEB000+0x00017FB4) [ 1] DxeCore.dll
  PC 0x5EF03098 (0x5EEEB000+0x00018098) [ 1] DxeCore.dll
  PC 0x5EF0460C (0x5EEEB000+0x0001960C) [ 1] DxeCore.dll
  PC 0x5EF005C4 (0x5EEEB000+0x000155C4) [ 1] DxeCore.dll
  PC 0x5EF00360 (0x5EEEB000+0x00015360) [ 1] DxeCore.dll
  PC 0x5EF003C8 (0x5EEEB000+0x000153C8) [ 1] DxeCore.dll
  PC 0x5EF128D0 (0x5EEEB000+0x000278D0) [ 1] DxeCore.dll
  PC 0x5EF12930 (0x5EEEB000+0x00027930) [ 1] DxeCore.dll
  PC 0x5EEEF594 (0x5EEEB000+0x4594) [ 1] DxeCore.dll
  PC 0x5EF0CF18 (0x5EEEB000+0x00021F18) [ 1] DxeCore.dll
  PC 0x5EF1A310 (0x5EEEB000+0x0002F310) [ 1] DxeCore.dll
  PC 0x5EEF1808 (0x5EEEB000+0x6808) [ 1] DxeCore.dll
  PC 0x5EEF1E48 (0x5EEEB000+0x6E48) [ 1] DxeCore.dll
  PC 0x5EF0A838 (0x5EEEB000+0x0001F838) [ 1] DxeCore.dll
  PC 0x5EEED70C (0x5EEEB000+0x270C) [ 1] DxeCore.dll
  PC 0x5EEEC93C (0x5EEEB000+0x193C) [ 1] DxeCore.dll
  PC 0x5EEEC024 (0x5EEEB000+0x1024) [ 1] DxeCore.dll

  [ 0] 
/home/ard/build/edk2/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/ArmPkg/Drivers/CpuDxe/CpuDxe/DEBUG/ArmCpuDxe.dll
  [ 1] 
/home/ard/build/edk2/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Ard Biesheuvel 
Reviewed-by: Leif Lindholm 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v7 0/2] qemu-qdisk: Implementation of grant copy operation.

[Stefano Stabellini]

Hi Wei,

I am happy to queue up this for QEMU, but I'll wait for the first patch
to be committed to Xen before sending a pull request. Is that OK?

Cheers,

Stefano

On Wed, 14 Sep 2016, Paulina Szubarczyk wrote:
> Hi,
> 
> It is a proposition for implementation of grant copy operation in qemu-qdisk 
> and interface in libxc/libs. 
> 
> Changes since v6:
> qemu-qdisk:
> -removed blank lines
> -renamed functions free_buffers -> ioreq_free_copy_buffers,
>  ioreq_copy -> ioreq_grant_copy
> -merged the if(ioreq_copy) with the conditions above
> 
> Changes since v5:
> qemu-qdisk:
> -added checking of every interface in the configure file. Based on
>  the Roger's comment that xengnttab_map_grant_ref was added prior
>  xengnttab_grant_copy, thus do not need to be check again here
>  I dropped this check.
> 
> Changes since v4:
> Interface:
> - changed the subject line
> - changed the comment in libs/gnttab/include/xengnttab.h according
>   to the David's suggestion.
> - removed unnecessary braces.
> 
> qemu-qdisk:
> - in the configure file check only if xengnttab_grant_copy is
>   implemented to verify 480 version of Xen.
> - remove r variable and initialization of count to 0 in
>   ioreq_copy.
> 
> - surround free_buffers, ioreq_init_copy_buffers and ioreq_copy
>   by "#if CONFIG_XEN_CTRL_INTERFACE_VERSION >= 480" abort in else
>   path because the function should not be called in that case.
> - replace the definition of struct xengnttab_grant_copy_segment
>   and a typedef to it with
>   'typedef void* xengnttab_grant_copy_segment_t'.
> - moved the new code in the xen_common.h to the end of the file.
> 
> Changes since v3:
> Interface:
> - revert to cast from xengnttab_grant_copy_segment_t
>   to ioctl_gntdev_grant_copy.
> - added compile-time check to compare the libs
>   xengnttab_grant_copy_segment_t with the ioctl structure.
>   The patch relies on Wei patch introducing XENGNTTAB_BUILD_BUG_ON in 
> libs/gnttab.
> 
> qemu-qdisk:
> - qemu_memalign/qemu_free is used instead function allocating
>   memory from xc.
> - removed the get_buffer function instead there is a direct call
>   to qemu_memalign.
> - moved ioreq_copy for write operation to ioreq_runio_qemu_aio.
> - added struct xengnttab_grant_copy_segment_t and stub in
>   xen_common.h for version of Xen earlier then 480.
> - added checking for version 480 to configure. The test repeats
>   all the operation that are required for version < 480 and
>   checks if xengnttab_grant_copy() is implemented.
> 
> Changes since v2:
> Interface:
> - dropped the changes in libxc/include/xenctrl_compat
> - changed the MINOR version in Makefile
> - replaced 'return -1' -> 'abort()'in libs/gnttab/gnttab_unimp.c
> - moved the struct 'xengnttab_copy_grant_segment' to 
>   libs/gnttab/include/xengnttab.h
> - added explicit assingment to ioctl_gntdev_grant_copy_segment 
>   to the linux part
> 
> qemu-qdisk:
> - to use the xengnttab_* function directly added -lxengnttab to configure
>   and include  in include/hw/xen/xen_common.h
> - in ioreq_copy removed an out path, changed a log level, made explicit 
>   assignement to 'xengnttab_copy_grant_segment'
> * I did not change the way of testing if grant_copy operation is implemented.
>   As far as I understand if the code from gnttab_unimp.c is used then the 
> gnttab 
>   device is unavailable and the handler to gntdev would be invalid. But 
>   if the handler is valid then the ioctl should return operation 
> unimplemented 
>   if the gntdev does not implement the operation.
> 
> 
> Changes since v1:
> Interface:
> - changed the interface to call grant copy operation to match ioctl
>   int xengnttab_grant_copy(xengnttab_handle *xgt,
>uint32_t count,
>xengnttab_grant_copy_segment_t* segs)
> 
> - added a struct 'xengnttab_copy_grant_segment' definition to tools/libs  
>   /gnttab/private.h, tools/libxc/include/xenctrl_compat.h
> 
> - changed the function 'osdep_gnttab_grant_copy' which right now just
>   call the ioctl
> 
> - added a new VER1.1 to tools/libs/gnttab/libxengnttab.map 
> 
> qemu-qdisk:
> - removed the 'ioreq_write','ioreq_read_init','ioreq_read' functions 
> - implemented 'ioreq_init_copy_buffers', 'ioreq_copy' 
> - reverted the removal of grant map and introduced conditional invoking
>   grant copy or grant map
> - resigned from caching the local buffers on behalf of allocating the 
>   required amount of pages at once. The cached structure would require 
>   to have an lock guard and I suppose that the performance improvement 
>   would degraded. 
>  
> 
> For the functional test I attached the device with a qdisk backend to the 
> guest, mounted, performed some reads and writes.
> 
> I run fio tests[1] with different iodepth and size of the block. The test can 
> be 
> accessed on my github[2] but mainly after the warm up I run for 60 seconds:
> fio --time_based \
>   --clocksource=clock_gettime \
>   

Re: [Xen-devel] [PATCH v2 2/3] x86 Test and expose CPUID faulting capabilities in /proc/cpuinfo

[Andy Lutomirski]

On Wed, Sep 14, 2016 at 3:03 PM, Kyle Huey  wrote:
> On Wed, Sep 14, 2016 at 2:35 PM, Dave Hansen
>  wrote:
>> On 09/14/2016 02:01 PM, Kyle Huey wrote:

>> Is any of this useful to optimize away at compile-time?  We have config
>> options for when we're running as a guest, and this seems like a feature
>> that isn't available when running on bare metal.
>
> On the contrary, this is only available when we're on bare metal.
> Neither Xen nor KVM virtualize CPUID faulting (although KVM correctly
> suppresses MSR_PLATFORM_INFO's report of support for it).

KVM could easily support this.  If rr starts using it, I think KVM
*should* add support, possibly even for older CPUs that don't support
the feature in hardware.

It's too bad that x86 doesn't give us the instruction bytes on a
fault.  Otherwise we could lazily switch this feature.

--Andy

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH V1] xen/arm: domain_build: introduce dom0_lowmem bootargs

[Peng Fan]

Hi Edgar,
On Wed, Sep 14, 2016 at 04:16:58PM +0200, Edgar E. Iglesias wrote:
>On Wed, Sep 14, 2016 at 08:40:09PM +0800, Peng Fan wrote:
>> On Wed, Sep 14, 2016 at 01:34:10PM +0100, Julien Grall wrote:
>> >
>> >
>> >On 14/09/16 13:18, Peng Fan wrote:
>> >>Hello Julien,
>> >>
>> >>On Wed, Sep 14, 2016 at 01:06:01PM +0100, Julien Grall wrote:
>> >>>
>> >>>
>> >>>On 14/09/16 13:03, Peng Fan wrote:
>> Hello Julien,
>> >>>
>> >>>Hello Peng,
>> >>>
>> On Wed, Sep 14, 2016 at 11:47:10AM +0100, Julien Grall wrote:
>> >Hello,
>> >
>> >On 14/09/16 08:41, Peng Fan wrote:
>> >>On Wed, Sep 14, 2016 at 08:23:24AM +0100, Julien Grall wrote:
>> >>diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>> >>index 35ab08d..cc71e6f 100644
>> >>--- a/xen/arch/arm/domain_build.c
>> >>+++ b/xen/arch/arm/domain_build.c
>> >>@@ -28,6 +28,8 @@
>> >>
>> >>static unsigned int __initdata opt_dom0_max_vcpus;
>> >>integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
>> >>+static bool_t __initdata opt_dom0_use_lowmem;
>> >>+boolean_param("dom0_use_lowmem", opt_dom0_use_lowmem);
>> >>
>> >>int dom0_11_mapping = 1;
>> >>
>> >>@@ -244,7 +246,7 @@ static void allocate_memory(struct domain *d, 
>> >>struct
>> >>kernel_info *kinfo)
>> >>   unsigned int order = get_11_allocation_size(kinfo->unassigned_mem);
>> >>   int i;
>> >>   
>> >>-bool_t lowmem = is_32bit_domain(d);
>> >>+bool_t lowmem = is_32bit_domain(d) || opt_dom0_use_lowmem;
>> >>   unsigned int bits;
>> >>
>> >>
>> >>Pass "dom0_use_lowmem=1" to xen to allocate lowmem as much as possible.
>> >
>> >Again, what is the benefit to have a command line option for that?
>> 
>> Then you prefer directly change "bool_t lowmem = is_32bit_domain(d);" to 
>> "bool_t lowmem = true" ?
>> I just want to give user a choice.
>> >>>
>> >>>We don't add new command line parameter just because they look cool to 
>> >>>have.
>> >>>So far, you did not explain why it would be good to let the choice to the
>> >>>user and how it could be used.
>> >>
>> >>I have not try, if there is no lowmem.
>> >>
>> >>I have not look into alloc_domheap_pages.
>> >>I am not sure whether there is such a platform or not,
>> >>just thinking if there is soc that dram memory starts from 4GB, and there 
>> >>is no dram
>> >>below 4GB. If we still can get memory when lowmem is true, I am ok to 
>> >>change directly assign
>> >>lowmem with value true. Anyway I have not look into the internals of 
>> >>domheap and
>> >>not sure whether there is such a platform that no lowmem (:-
>> >
>> >We cannot exclude this possibility. However, the only reason that Xen is
>> >requiring to allocate a bank below 4GB for 32-bit domain is to handle
>> >non-LPAE kernel.
>> 
>> Now also need to handle device that have DMA limitation -:)
>
>Hi Peng,
>
>Doesn't your platform have an IOMMU/SMMU?

We have SMMU. This is not related to SMMU. Dom0 use 1:1 mapping and no SMMU 
involved,
the physical memory assigned to Dom0 maybe higher than 4GB, but Some IPs only
supports 32bits DMA in Dom0. Then assign a 64bits dma address to a device only 
supports 32
bits device in Linux will hang the device or else.

Regards,
Peng.
>
>Cheers,
>Edgar

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-unstable test] 100954: tolerable FAIL - PUSHED

[osstest service owner]

flight 100954 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/100954/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop fail like 100933
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop fail like 100933
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stopfail like 100933
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stopfail like 100933
 test-amd64-amd64-xl-rtds  9 debian-install   fail  like 100933

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-rumprun-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-rumprun-i386  1 build-check(1)   blocked  n/a
 build-amd64-rumprun   5 rumprun-buildfail   never pass
 build-i386-rumprun5 rumprun-buildfail   never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass

version targeted for testing:
 xen  773522000cc17f6f4323a4d97423790138ea98f2
baseline version:
 xen  d45fae589b8d8b6d36c211dcc46d767dda730b61

Last test of basis   100933  2016-09-13 13:03:38 Z1 days
Testing same since   100944  2016-09-14 02:56:07 Z0 days2 attempts


People who touched revisions under test:
  Andrew Cooper 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64-xtf  pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-oldkern  pass
 build-i386-oldkern   

[Xen-devel] [DOC v6] PV Calls protocol design

[Stefano Stabellini]

Hi all,

This is the design document of the PV Calls protocol. You can find
prototypes of the Linux frontend and backend drivers here:

git://git.kernel.org/pub/scm/linux/kernel/git/sstabellini/xen.git pvcalls-6

To use them, make sure to enable CONFIG_XEN_PVCALLS in your kernel
config and add "pvcalls=1" to the command line of your DomU Linux
kernel. You also need the toolstack to create the initial xenstore nodes
for the protocol. To do that, please apply the attached patch to libxl
(the patch is based on Xen 4.7.0-rc3) and add "pvcalls=1" to your DomU
config file.


In this version I added a "reuse" field to the release command, to give
an hint to the backend that data ring pages and evtchn will be reused.
The backend is free to unmap them immediately as usual or keep them
mapped until they get reused (or until it decides it is time to free
them). Thanks to this optimization pv calls can handle more requests per
second than Docker with apachebench in my test environment.

I also added a "networking-calls" backend node on xenstore to allow the
backend to specify that is supporting networking calls. In a future when
other POSIX syscall sets are supported, a backend could decide to
implement only one set.


Please review!

Cheers,

Stefano


Changes in v6:
- add reuse field in release command
- add "networking-calls" backend node on xenstore
- fixed tab/whitespace indentation

Changes in v5:
- clarify text
- rename id to req_id
- rename sockid to id
- move id to request and response specific fields
- add version node to xenstore

Changes in v4:
- rename xensock to pvcalls

Changes in v3:
- add a dummy element to struct xen_xensock_request to make sure the
  size of the struct is the same on both x86_32 and x86_64

Changes in v2:
- add max-dataring-page-order
- add "Publish backend features and transport parameters" to backend
  xenbus workflow
- update new cmd values
- update xen_xensock_request
- add backlog parameter to listen and binary layout
- add description of new data ring format (interface+data)
- modify connect and accept to reflect new data ring format
- add link to POSIX docs
- add error numbers
- add address format section and relevant numeric definitions
- add explicit mention of unimplemented commands
- add protocol node name
- add xenbus shutdown diagram
- add socket operation

---

# PV Calls Protocol version 1

## Rationale

PV Calls is a paravirtualized protocol that allows the implementation of
a set of POSIX functions in a different domain. The PV Calls frontend
sends POSIX function calls to the backend, which implements them and
returns a value to the frontend.

This version of the document covers networking function calls, such as
connect, accept, bind, release, listen, poll, recvmsg and sendmsg; but
the protocol is meant to be easily extended to cover different sets of
calls. Unimplemented commands return ENOTSUPP.

PV Calls provide the following benefits:
* full visibility of the guest behavior on the backend domain, allowing
  for inexpensive filtering and manipulation of any guest calls
* excellent performance

Specifically, PV Calls for networking offer these advantages:
* guest networking works out of the box with VPNs, wireless networks and
  any other complex configurations on the host
* guest services listen on ports bound directly to the backend domain IP
  addresses
* localhost becomes a secure namespace for inter-VMs communications


## Design

### Xenstore

The frontend and the backend connect to each other exchanging information via
xenstore. The toolstack creates front and back nodes with state
XenbusStateInitialising. The protocol node name is **pvcalls**. There can only
be one PV Calls frontend per domain.

 Frontend XenBus Nodes

port
 Values: 

 The identifier of the Xen event channel used to signal activity
 in the ring buffer.

ring-ref
 Values: 

 The Xen grant reference granting permission for the backend to map
 the sole page in a single page sized ring buffer.

 Backend XenBus Nodes

version
 Values: 

 Protocol version supported by the backend.

max-dataring-page-order
 Values: 

 The maximum supported size of the data ring in units of lb(machine
 pages). (e.g. 0 == 1 page,  1 = 2 pages, 2 == 4 pages, etc.).

networking-calls
 Values: 

 Value "1" means that the backend support networking calls.
 Value "0" means that it does not.

 State Machine

Initialization:

*Front*   *Back*
XenbusStateInitialising   XenbusStateInitialising
- Query virtual device- Query backend device
  properties.   identification data.
- Setup OS device instance.   - Publish backend features
- Allocate and initialize the   and transport parameters
  request ring.  |
- Publish transport parameters  

Re: [Xen-devel] [PATCH v4 1/2] xen: replace complicated tlbflush check with an inline function

[Dongli Zhang]

> I don't think you should pass this into the function ...
> 
> > +{
> > +return page->u.free.need_tlbflush &&
> > +   page->tlbflush_timestamp <= tlbflush_current_time &&
> 
> ... and use tlbflush_current_time() here instead.

I rewrite the inline function in xen/include/xen/mm.h to:

+#include 
+
+static inline bool accumulate_tlbflush(bool need_tlbflush,
+   const struct page_info *page,
+   uint32_t tlbflush_timestamp)
+{
+return page->u.free.need_tlbflush &&
+   page->tlbflush_timestamp <= tlbflush_current_time() &&
+   (!need_tlbflush ||
+page->tlbflush_timestamp > tlbflush_timestamp);
+}

However, to use tlbflush_current_time and "asm/flushtlb.h" would lead
to the following compiling error:

>>>
In file included from 
/home/zhang/test/mainline-xen/xen/include/asm/flushtlb.h:14:0,
 from suspend.c:13:
/home/zhang/test/mainline-xen/xen/include/xen/mm.h: In function 
‘accumulate_tlbflush’:
/home/zhang/test/mainline-xen/xen/include/xen/mm.h:577:12: error: implicit 
declaration of function ‘tlbflush_current_time’ 
[-Werror=implicit-function-declaration]
page->tlbflush_timestamp <= tlbflush_current_time() &&
^
/home/zhang/test/mainline-xen/xen/include/xen/mm.h:577:12: error: nested extern 
declaration of ‘tlbflush_current_time’ [-Werror=nested-externs]
cc1: all warnings being treated as errors
make[4]: *** [suspend.o] Error 1
>>>

I can workaround the issue by removing "#include " from
xen/arch/x86/acpi/suspend.c and then everything works fine.


Can I just rewrite the inline function to a #define macro? This minimizes the
changes to the code.

+#define accumulate_tlbflush(need_tlbflush, page, tlbflush_timestamp) \
+(page)->u.free.need_tlbflush &&  \
+(page)->tlbflush_timestamp <= tlbflush_current_time() && \
+(!need_tlbflush ||   \
+ (page)->tlbflush_timestamp > tlbflush_timestamp)

Thank you very much!

Dongli Zhang

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 2/3] x86 Test and expose CPUID faulting capabilities in /proc/cpuinfo

[Kyle Huey]

Xen advertises the underlying support for CPUID faulting but not does pass
through writes to the relevant MSR, nor does it virtualize it, so it does
not actually work. For now mask off the relevant bit on MSR_PLATFORM_INFO.

Signed-off-by: Kyle Huey 
---
 arch/x86/include/asm/cpufeatures.h |  1 +
 arch/x86/include/asm/msr-index.h   |  1 +
 arch/x86/kernel/cpu/scattered.c| 14 ++
 arch/x86/xen/enlighten.c   |  3 +++
 4 files changed, 19 insertions(+)

diff --git a/arch/x86/include/asm/cpufeatures.h 
b/arch/x86/include/asm/cpufeatures.h
index 92a8308..78b9d06 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -190,6 +190,7 @@
 
 #define X86_FEATURE_CPB( 7*32+ 2) /* AMD Core Performance 
Boost */
 #define X86_FEATURE_EPB( 7*32+ 3) /* IA32_ENERGY_PERF_BIAS 
support */
+#define X86_FEATURE_CPUID_FAULT ( 7*32+ 4) /* Intel CPUID faulting */
 
 #define X86_FEATURE_HW_PSTATE  ( 7*32+ 8) /* AMD HW-PState */
 #define X86_FEATURE_PROC_FEEDBACK ( 7*32+ 9) /* AMD ProcFeedbackInterface */
diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index 56f4c66..83908d5 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -41,6 +41,7 @@
 #define MSR_IA32_PERFCTR1  0x00c2
 #define MSR_FSB_FREQ   0x00cd
 #define MSR_PLATFORM_INFO  0x00ce
+#define CPUID_FAULTING_SUPPORT (1UL << 31)
 
 #define MSR_NHM_SNB_PKG_CST_CFG_CTL0x00e2
 #define NHM_C3_AUTO_DEMOTE (1UL << 25)
diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattered.c
index 8cb57df..d502da1 100644
--- a/arch/x86/kernel/cpu/scattered.c
+++ b/arch/x86/kernel/cpu/scattered.c
@@ -24,6 +24,17 @@ enum cpuid_regs {
CR_EBX
 };
 
+static int supports_cpuid_faulting(void)
+{
+   unsigned int lo, hi;
+
+   if (rdmsr_safe(MSR_PLATFORM_INFO, , ) == 0 &&
+   (lo & CPUID_FAULTING_SUPPORT))
+   return 1;
+   else
+   return 0;
+}
+
 void init_scattered_cpuid_features(struct cpuinfo_x86 *c)
 {
u32 max_level;
@@ -54,4 +65,7 @@ void init_scattered_cpuid_features(struct cpuinfo_x86 *c)
if (regs[cb->reg] & (1 << cb->bit))
set_cpu_cap(c, cb->feature);
}
+
+   if (supports_cpuid_faulting())
+   set_cpu_cap(c, X86_FEATURE_CPUID_FAULT);
 }
diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index b86ebb1..2c47f0c 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -1050,6 +1050,9 @@ static u64 xen_read_msr_safe(unsigned int msr, int *err)
 #endif
val &= ~X2APIC_ENABLE;
break;
+   case MSR_PLATFORM_INFO:
+   val &= ~CPUID_FAULTING_SUPPORT;
+   break;
}
return val;
 }
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 2/3] x86 Test and expose CPUID faulting capabilities in /proc/cpuinfo

[Dave Hansen]

On 09/14/2016 02:01 PM, Kyle Huey wrote:
> Xen advertises the underlying support for CPUID faulting but not does pass
> through writes to the relevant MSR, nor does it virtualize it, so it does
> not actually work. For now mask off the relevant bit on MSR_PLATFORM_INFO.

That needs to make it into a comment, please.

That *is* a Xen bug, right?

> Signed-off-by: Kyle Huey 
> ---
>  arch/x86/include/asm/cpufeatures.h |  1 +
>  arch/x86/include/asm/msr-index.h   |  1 +
>  arch/x86/kernel/cpu/scattered.c| 14 ++
>  arch/x86/xen/enlighten.c   |  3 +++
>  4 files changed, 19 insertions(+)
> 
> diff --git a/arch/x86/include/asm/cpufeatures.h 
> b/arch/x86/include/asm/cpufeatures.h
> index 92a8308..78b9d06 100644
> --- a/arch/x86/include/asm/cpufeatures.h
> +++ b/arch/x86/include/asm/cpufeatures.h
> @@ -190,6 +190,7 @@
>  
>  #define X86_FEATURE_CPB  ( 7*32+ 2) /* AMD Core Performance 
> Boost */
>  #define X86_FEATURE_EPB  ( 7*32+ 3) /* IA32_ENERGY_PERF_BIAS 
> support */
> +#define X86_FEATURE_CPUID_FAULT ( 7*32+ 4) /* Intel CPUID faulting */
>  
>  #define X86_FEATURE_HW_PSTATE( 7*32+ 8) /* AMD HW-PState */
>  #define X86_FEATURE_PROC_FEEDBACK ( 7*32+ 9) /* AMD ProcFeedbackInterface */
> diff --git a/arch/x86/include/asm/msr-index.h 
> b/arch/x86/include/asm/msr-index.h
> index 56f4c66..83908d5 100644
> --- a/arch/x86/include/asm/msr-index.h
> +++ b/arch/x86/include/asm/msr-index.h
> @@ -41,6 +41,7 @@
>  #define MSR_IA32_PERFCTR10x00c2
>  #define MSR_FSB_FREQ 0x00cd
>  #define MSR_PLATFORM_INFO0x00ce
> +#define CPUID_FAULTING_SUPPORT   (1UL << 31)
>  
>  #define MSR_NHM_SNB_PKG_CST_CFG_CTL  0x00e2
>  #define NHM_C3_AUTO_DEMOTE   (1UL << 25)
> diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattered.c
> index 8cb57df..d502da1 100644
> --- a/arch/x86/kernel/cpu/scattered.c
> +++ b/arch/x86/kernel/cpu/scattered.c
> @@ -24,6 +24,17 @@ enum cpuid_regs {
>   CR_EBX
>  };
>  
> +static int supports_cpuid_faulting(void)
> +{
> + unsigned int lo, hi;
> +
> + if (rdmsr_safe(MSR_PLATFORM_INFO, , ) == 0 &&
> + (lo & CPUID_FAULTING_SUPPORT))
> + return 1;
> + else
> + return 0;
> +}

Is any of this useful to optimize away at compile-time?  We have config
options for when we're running as a guest, and this seems like a feature
that isn't available when running on bare metal.

> diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
> index b86ebb1..2c47f0c 100644
> --- a/arch/x86/xen/enlighten.c
> +++ b/arch/x86/xen/enlighten.c
> @@ -1050,6 +1050,9 @@ static u64 xen_read_msr_safe(unsigned int msr, int *err)
>  #endif
>   val &= ~X2APIC_ENABLE;
>   break;
> + case MSR_PLATFORM_INFO:
> + val &= ~CPUID_FAULTING_SUPPORT;
> + break;
>   }
>   return val;
>  }

Does this mean that Xen guests effectively can't take advantage of this
feature?


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [RESEND][PATCH v2 2/3] x86 Test and expose CPUID faulting capabilities in /proc/cpuinfo

[Kyle Huey]

Xen advertises the underlying support for CPUID faulting but not does pass
through writes to the relevant MSR, nor does it virtualize it, so it does
not actually work. For now mask off the relevant bit on MSR_PLATFORM_INFO.

Signed-off-by: Kyle Huey 
---
 arch/x86/include/asm/cpufeatures.h |  1 +
 arch/x86/include/asm/msr-index.h   |  1 +
 arch/x86/kernel/cpu/scattered.c| 14 ++
 arch/x86/xen/enlighten.c   |  3 +++
 4 files changed, 19 insertions(+)

diff --git a/arch/x86/include/asm/cpufeatures.h 
b/arch/x86/include/asm/cpufeatures.h
index 92a8308..78b9d06 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -190,6 +190,7 @@
 
 #define X86_FEATURE_CPB( 7*32+ 2) /* AMD Core Performance 
Boost */
 #define X86_FEATURE_EPB( 7*32+ 3) /* IA32_ENERGY_PERF_BIAS 
support */
+#define X86_FEATURE_CPUID_FAULT ( 7*32+ 4) /* Intel CPUID faulting */
 
 #define X86_FEATURE_HW_PSTATE  ( 7*32+ 8) /* AMD HW-PState */
 #define X86_FEATURE_PROC_FEEDBACK ( 7*32+ 9) /* AMD ProcFeedbackInterface */
diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index 56f4c66..83908d5 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -41,6 +41,7 @@
 #define MSR_IA32_PERFCTR1  0x00c2
 #define MSR_FSB_FREQ   0x00cd
 #define MSR_PLATFORM_INFO  0x00ce
+#define CPUID_FAULTING_SUPPORT (1UL << 31)
 
 #define MSR_NHM_SNB_PKG_CST_CFG_CTL0x00e2
 #define NHM_C3_AUTO_DEMOTE (1UL << 25)
diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattered.c
index 8cb57df..d502da1 100644
--- a/arch/x86/kernel/cpu/scattered.c
+++ b/arch/x86/kernel/cpu/scattered.c
@@ -24,6 +24,17 @@ enum cpuid_regs {
CR_EBX
 };
 
+static int supports_cpuid_faulting(void)
+{
+   unsigned int lo, hi;
+
+   if (rdmsr_safe(MSR_PLATFORM_INFO, , ) == 0 &&
+   (lo & CPUID_FAULTING_SUPPORT))
+   return 1;
+   else
+   return 0;
+}
+
 void init_scattered_cpuid_features(struct cpuinfo_x86 *c)
 {
u32 max_level;
@@ -54,4 +65,7 @@ void init_scattered_cpuid_features(struct cpuinfo_x86 *c)
if (regs[cb->reg] & (1 << cb->bit))
set_cpu_cap(c, cb->feature);
}
+
+   if (supports_cpuid_faulting())
+   set_cpu_cap(c, X86_FEATURE_CPUID_FAULT);
 }
diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index b86ebb1..2c47f0c 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -1050,6 +1050,9 @@ static u64 xen_read_msr_safe(unsigned int msr, int *err)
 #endif
val &= ~X2APIC_ENABLE;
break;
+   case MSR_PLATFORM_INFO:
+   val &= ~CPUID_FAULTING_SUPPORT;
+   break;
}
return val;
 }
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [ovmf test] 100955: all pass - PUSHED

[osstest service owner]

flight 100955 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/100955/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf 062f9fd2cfaad09f6dd0e302094bc030827eb706
baseline version:
 ovmf 18b00c8c8773e62b1e2c4a36f490996325570b67

Last test of basis   100947  2016-09-14 05:41:50 Z0 days
Testing same since   100955  2016-09-14 14:14:52 Z0 days1 attempts


People who touched revisions under test:
  Ard Biesheuvel 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=ovmf
+ revision=062f9fd2cfaad09f6dd0e302094bc030827eb706
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push ovmf 
062f9fd2cfaad09f6dd0e302094bc030827eb706
+ branch=ovmf
+ revision=062f9fd2cfaad09f6dd0e302094bc030827eb706
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=ovmf
+ xenbranch=xen-unstable
+ '[' xovmf = xlinux ']'
+ linuxbranch=
+ '[' x = x ']'
+ qemuubranch=qemu-upstream-unstable
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable
+ prevxenbranch=xen-4.7-testing
+ '[' x062f9fd2cfaad09f6dd0e302094bc030827eb706 = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/xtf.git
++ : osst...@xenbits.xen.org:/home/xen/git/xtf.git
++ : git://xenbits.xen.org/xtf.git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/rumprun.git
++ : git://git.seabios.org/seabios.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/seabios.git
++ : git://xenbits.xen.org/osstest/seabios.git
++ : https://github.com/tianocore/edk2.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/linux-firmware.git
++ : osst...@xenbits.xen.org:/home/osstest/ext/linux-firmware.git
++ : git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
++ : 

Re: [Xen-devel] [PATCH v5 00/16] Xen ARM DomU ACPI support

[Julien Grall]

Hi Stefano,

On 14/09/2016 21:48, Stefano Stabellini wrote:

On Wed, 14 Sep 2016, Julien Grall wrote:

On 14/09/2016 02:06, Stefano Stabellini wrote:

On Wed, 14 Sep 2016, Shannon Zhao wrote:

On 2016/9/13 23:17, Julien Grall wrote:



On 13/09/16 14:06, Shannon Zhao wrote:

Hi Julien,


Hello Shannon,


On 2016/9/13 19:56, Julien Grall wrote:

Hi Shannon,

On 02/09/16 03:55, Shannon Zhao wrote:

From: Shannon Zhao 

The design of this feature is described as below.
Firstly, the toolstack (libxl) generates the ACPI tables according
the
number of vcpus and gic controller.

Then, it copies these ACPI tables to DomU non-RAM memory map space
and
passes them to UEFI firmware through the "ARM multiboot" protocol.

At last, UEFI gets the ACPI tables through the "ARM multiboot"
protocol
and installs these tables like the usual way and passes both ACPI
and DT
information to the Xen DomU.

Currently libxl only generates RSDP, XSDT, GTDT, MADT, FADT, DSDT
tables
since it's enough now.

This has been tested using guest kernel with the Dom0 ACPI support
patches which could be fetched from linux master or:
https://git.kernel.org/cgit/linux/kernel/git/mfleming/efi.git/log/?h=efi/arm-xen



The UEFI binary could be fetched from or built from edk2 master
branch:
http://people.linaro.org/~shannon.zhao/DomU_ACPI/XEN_EFI.fd


On which commit this EFI binary is based? I am trying to rebuild
myself,
and go no luck to boot it so far.


I forgot the exact commit. But I just tried below commit which adds
the
support to edk2 and the guest can boot up successfully with ACPI.

402dde6 ArmVirtPkg/ArmVirtXen: Add ACPI support for Virt Xen ARM


Thanks, the commit does not build on my platform. After some help for
Ard I managed to boot UEFI with the patch [1] applied.

However Linux does not boot when passing acpi=on and abort with the
following message:

(d86) 6RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=1
(d86) 6NR_IRQS:64 nr_irqs:64 0
(d86) 3No valid GICC entries exist
(d86) 0Kernel panic - not syncing: No interrupt controller found.
(d86) dCPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.8.0-rc6+ #420
(d86) dHardware name: XENVM-4.8 (DT)
(d86) Call trace:
(d86) [] dump_backtrace+0x0/0x1a8
(d86) [] show_stack+0x14/0x20
(d86) [] dump_stack+0x94/0xb8
(d86) [] panic+0x10c/0x250
(d86) [] init_IRQ+0x24/0x2c
(d86) [] start_kernel+0x238/0x394
(d86) [] __primary_switched+0x30/0x74
(d86) 0---[ end Kernel panic - not syncing: No interrupt controller
found.

This is because the header.length for GICC is not valid for ACPI 5.1
(see BAD_MADT_GICC_ENTRY). So please check all the size of each table
against ACPI 5.1.


Oops. The reason is that acpi_madt_generic_interrupt in Xen is already
updated to ACPI 6.0 and the length is 80 not 76 of ACPI 5.1.
One solution is that we still use ACPI 5.1 and make gicc->header.length
76. Other one is that we update to ACPI 6.0 since the Xen ARM ACPI
support in Linux was introduced after ACPI 6.0.

Which one do you prefer?


Certainly the versions of all tables need to be consistent. I would
prefer to have ACPI 6.0 but 5.1 is acceptable too (especially if
upgrading to 6.0 causes a large amount of changes to your patches).


I disagree on this, we should use the first version of ACPI that is fully
supporting ARM because a guest operating system may choose to support the
first one (there is a lot hardware platform out which only provides ACPI 5.1).


And I thought that compatibility was supposed to be ACPI's strong suit.
I mistakenly had the impression that new ACPI releases weren't suppose
to break old OSes. I take back my comment, you are right that we should
stay on 5.1 (including all the erratas, many are important for ARM).



IIRC, early version of ACPI used to have some incompatibility. I will 
have to go through the ACPI spec to find the main differences between 
5.1 and 6.0 for ARM.


Assuming the newer versions are backward compatible, it might be good to 
written down somewhere (maybe a public headers) that the guest OS should 
not assume a specific version of ACPI. This would avoid to tie us on 
ACPI 5.1 and allow us to upgrade the tables on a next release of Xen.


In any case, we should be consistent accross all the ACPI tables (e.g 
version, size of the tables...) to accommodate picky OSes. For now, I 
would stay on ACPI 5.1 for safety.


Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v5 00/16] Xen ARM DomU ACPI support

[Stefano Stabellini]

On Wed, 14 Sep 2016, Julien Grall wrote:

> Hello,
> 
> On 14/09/2016 02:06, Stefano Stabellini wrote:
> > On Wed, 14 Sep 2016, Shannon Zhao wrote:
> > > On 2016/9/13 23:17, Julien Grall wrote:
> > > > 
> > > > 
> > > > On 13/09/16 14:06, Shannon Zhao wrote:
> > > > > Hi Julien,
> > > > 
> > > > Hello Shannon,
> > > > 
> > > > > On 2016/9/13 19:56, Julien Grall wrote:
> > > > > > Hi Shannon,
> > > > > > 
> > > > > > On 02/09/16 03:55, Shannon Zhao wrote:
> > > > > > > From: Shannon Zhao 
> > > > > > > 
> > > > > > > The design of this feature is described as below.
> > > > > > > Firstly, the toolstack (libxl) generates the ACPI tables according
> > > > > > > the
> > > > > > > number of vcpus and gic controller.
> > > > > > > 
> > > > > > > Then, it copies these ACPI tables to DomU non-RAM memory map space
> > > > > > > and
> > > > > > > passes them to UEFI firmware through the "ARM multiboot" protocol.
> > > > > > > 
> > > > > > > At last, UEFI gets the ACPI tables through the "ARM multiboot"
> > > > > > > protocol
> > > > > > > and installs these tables like the usual way and passes both ACPI
> > > > > > > and DT
> > > > > > > information to the Xen DomU.
> > > > > > > 
> > > > > > > Currently libxl only generates RSDP, XSDT, GTDT, MADT, FADT, DSDT
> > > > > > > tables
> > > > > > > since it's enough now.
> > > > > > > 
> > > > > > > This has been tested using guest kernel with the Dom0 ACPI support
> > > > > > > patches which could be fetched from linux master or:
> > > > > > > https://git.kernel.org/cgit/linux/kernel/git/mfleming/efi.git/log/?h=efi/arm-xen
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > The UEFI binary could be fetched from or built from edk2 master
> > > > > > > branch:
> > > > > > > http://people.linaro.org/~shannon.zhao/DomU_ACPI/XEN_EFI.fd
> > > > > > 
> > > > > > On which commit this EFI binary is based? I am trying to rebuild
> > > > > > myself,
> > > > > > and go no luck to boot it so far.
> > > > > > 
> > > > > I forgot the exact commit. But I just tried below commit which adds
> > > > > the
> > > > > support to edk2 and the guest can boot up successfully with ACPI.
> > > > > 
> > > > > 402dde6 ArmVirtPkg/ArmVirtXen: Add ACPI support for Virt Xen ARM
> > > > 
> > > > Thanks, the commit does not build on my platform. After some help for
> > > > Ard I managed to boot UEFI with the patch [1] applied.
> > > > 
> > > > However Linux does not boot when passing acpi=on and abort with the
> > > > following message:
> > > > 
> > > > (d86) 6RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=1
> > > > (d86) 6NR_IRQS:64 nr_irqs:64 0
> > > > (d86) 3No valid GICC entries exist
> > > > (d86) 0Kernel panic - not syncing: No interrupt controller found.
> > > > (d86) dCPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.8.0-rc6+ #420
> > > > (d86) dHardware name: XENVM-4.8 (DT)
> > > > (d86) Call trace:
> > > > (d86) [] dump_backtrace+0x0/0x1a8
> > > > (d86) [] show_stack+0x14/0x20
> > > > (d86) [] dump_stack+0x94/0xb8
> > > > (d86) [] panic+0x10c/0x250
> > > > (d86) [] init_IRQ+0x24/0x2c
> > > > (d86) [] start_kernel+0x238/0x394
> > > > (d86) [] __primary_switched+0x30/0x74
> > > > (d86) 0---[ end Kernel panic - not syncing: No interrupt controller
> > > > found.
> > > > 
> > > > This is because the header.length for GICC is not valid for ACPI 5.1
> > > > (see BAD_MADT_GICC_ENTRY). So please check all the size of each table
> > > > against ACPI 5.1.
> > > > 
> > > Oops. The reason is that acpi_madt_generic_interrupt in Xen is already
> > > updated to ACPI 6.0 and the length is 80 not 76 of ACPI 5.1.
> > > One solution is that we still use ACPI 5.1 and make gicc->header.length
> > > 76. Other one is that we update to ACPI 6.0 since the Xen ARM ACPI
> > > support in Linux was introduced after ACPI 6.0.
> > > 
> > > Which one do you prefer?
> > 
> > Certainly the versions of all tables need to be consistent. I would
> > prefer to have ACPI 6.0 but 5.1 is acceptable too (especially if
> > upgrading to 6.0 causes a large amount of changes to your patches).
> 
> I disagree on this, we should use the first version of ACPI that is fully
> supporting ARM because a guest operating system may choose to support the
> first one (there is a lot hardware platform out which only provides ACPI 5.1).

And I thought that compatibility was supposed to be ACPI's strong suit.
I mistakenly had the impression that new ACPI releases weren't suppose
to break old OSes. I take back my comment, you are right that we should
stay on 5.1 (including all the erratas, many are important for ARM).

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] prctl, x86 Add PR_[GET|SET]_CPUID for controlling the CPUID instruction.

[Andrew Cooper]

On 14/09/2016 20:36, Andy Lutomirski wrote:
> On Wed, Sep 14, 2016 at 12:28 PM, Andrew Cooper
>  wrote:
>> On 14/09/2016 20:23, Boris Ostrovsky wrote:
>>> On 09/14/2016 02:52 PM, Andy Lutomirski wrote:
 On Tue, Sep 13, 2016 at 11:13 PM, Kyle Huey  wrote:
> On Mon, Sep 12, 2016 at 9:56 AM, Andy Lutomirski  
> wrote:
>> You should explicitly check that, if the
>> feature is set under Xen PV, then the MSR actually works as
>> advertised.  This may require talking to the Xen folks to make sure
>> you're testing the right configuration.
> This is interesting.  When running under Xen PV the kernel is allowed
> to read the real value of MSR_PLATFORM_INFO and see that CPUID
> faulting is supported.  But as you suggested, writing to
> MSR_MISC_FEATURES_ENABLES doesn't actually enable CPUID faulting, at
> least not in any way that works.
>
> It's not obvious to me how to test this, because when this feature
> works, CPUID only faults in userspace, not in the kernel.  Is there
> existing code somewhere that runs tests like this in userspace?
>
 Andrew, Boris: should we expect Xen PV to do anything sensible when we
 write to MSR_PLATFORM_INFO to turn on CPUID faulting?  Should the Xen
 PV rdmsr hooks or perhaps the hypervisor mask out the feature if it
 isn't going to be supported?
>>> The hypervisor uses CPUID faulting so we shouldn't advertise this
>>> feature to guests.
>> In the case that the hardware has faulting, or for any HVM guest, the
>> extra cost to making the feature available to the guest is a single
>> conditional test in the cpuid path.  This is about as close to zero as a
>> feature gets.  We really should be offering the feature to guests, and
>> have it actually working.  The issue here is that it is leaking when we
>> weren't intending to offer it.
> As long as Xen can fix this one way or the other in reasonably short
> order, I think I'm okay with having Linux incorrectly think it works
> on old Xen hypervisors.

For now, unilaterally hiding CPUID faulting is easy, and simple to backport.

Making the feature available for guests to use is slightly more tricky,
as the toolstack still depends on not being faulted to construct HVM
domains properly.  This is the subject of my current CPUID project,
which will result in dom0 being no more special than any other domain
(in terms of hypervisor-side cpuid handling).

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v7 2/2] qdisk - hw/block/xen_disk: grant copy implementation

[Paulina Szubarczyk]

Copy data operated on during request from/to local buffers to/from
the grant references.

Before grant copy operation local buffers must be allocated what is
done by calling ioreq_init_copy_buffers. For the 'read' operation,
first, the qemu device invokes the read operation on local buffers
and on the completion grant copy is called and buffers are freed.
For the 'write' operation grant copy is performed before invoking
write by qemu device.

A new value 'feature_grant_copy' is added to recognize when the
grant copy operation is supported by a guest.

Signed-off-by: Paulina Szubarczyk 
Acked-by: Anthony PERARD 
Acked-by: Roger Pau Monné 
---
Changes since v6:
-removed blank lines
-renamed functions free_buffers -> ioreq_free_copy_buffers,
 ioreq_copy -> ioreq_grant_copy
-merged the if(ioreq_copy) with the conditions above
---
 configure   |  55 
 hw/block/xen_disk.c | 153 ++--
 include/hw/xen/xen_common.h |  14 
 3 files changed, 217 insertions(+), 5 deletions(-)

diff --git a/configure b/configure
index f4c589f..74b3d93 100755
--- a/configure
+++ b/configure
@@ -1972,6 +1972,61 @@ EOF
 /*
  * If we have stable libs the we don't want the libxc compat
  * layers, regardless of what CFLAGS we may have been given.
+ *
+ * Also, check if xengnttab_grant_copy_segment_t is defined and
+ * grant copy operation is implemented.
+ */
+#undef XC_WANT_COMPAT_EVTCHN_API
+#undef XC_WANT_COMPAT_GNTTAB_API
+#undef XC_WANT_COMPAT_MAP_FOREIGN_API
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#if !defined(HVM_MAX_VCPUS)
+# error HVM_MAX_VCPUS not defined
+#endif
+int main(void) {
+  xc_interface *xc = NULL;
+  xenforeignmemory_handle *xfmem;
+  xenevtchn_handle *xe;
+  xengnttab_handle *xg;
+  xen_domain_handle_t handle;
+  xengnttab_grant_copy_segment_t* seg = NULL;
+
+  xs_daemon_open();
+
+  xc = xc_interface_open(0, 0, 0);
+  xc_hvm_set_mem_type(0, 0, HVMMEM_ram_ro, 0, 0);
+  xc_domain_add_to_physmap(0, 0, XENMAPSPACE_gmfn, 0, 0);
+  xc_hvm_inject_msi(xc, 0, 0xf000, 0x);
+  xc_hvm_create_ioreq_server(xc, 0, HVM_IOREQSRV_BUFIOREQ_ATOMIC, NULL);
+  xc_domain_create(xc, 0, handle, 0, NULL, NULL);
+
+  xfmem = xenforeignmemory_open(0, 0);
+  xenforeignmemory_map(xfmem, 0, 0, 0, 0, 0);
+
+  xe = xenevtchn_open(0, 0);
+  xenevtchn_fd(xe);
+
+  xg = xengnttab_open(0, 0);
+  xengnttab_grant_copy(xg, 0, seg);
+
+  return 0;
+}
+EOF
+  compile_prog "" "$xen_libs $xen_stable_libs"
+then
+xen_ctrl_version=480
+xen=yes
+  elif
+  cat > $TMPC <= 480
+
+static void ioreq_free_copy_buffers(struct ioreq *ioreq)
+{
+int i;
+
+for (i = 0; i < ioreq->v.niov; i++) {
+ioreq->page[i] = NULL;
+}
+
+qemu_vfree(ioreq->pages);
+}
+
+static int ioreq_init_copy_buffers(struct ioreq *ioreq)
+{
+int i;
+
+if (ioreq->v.niov == 0) {
+return 0;
+}
+
+ioreq->pages = qemu_memalign(XC_PAGE_SIZE, ioreq->v.niov * XC_PAGE_SIZE);
+
+for (i = 0; i < ioreq->v.niov; i++) {
+ioreq->page[i] = ioreq->pages + i * XC_PAGE_SIZE;
+ioreq->v.iov[i].iov_base = ioreq->page[i];
+}
+
+return 0;
+}
+
+static int ioreq_grant_copy(struct ioreq *ioreq)
+{
+xengnttab_handle *gnt = ioreq->blkdev->xendev.gnttabdev;
+xengnttab_grant_copy_segment_t segs[BLKIF_MAX_SEGMENTS_PER_REQUEST];
+int i, count, rc;
+int64_t file_blk = ioreq->blkdev->file_blk;
+
+if (ioreq->v.niov == 0) {
+return 0;
+}
+
+count = ioreq->v.niov;
+
+for (i = 0; i < count; i++) {
+if (ioreq->req.operation == BLKIF_OP_READ) {
+segs[i].flags = GNTCOPY_dest_gref;
+segs[i].dest.foreign.ref = ioreq->refs[i];
+segs[i].dest.foreign.domid = ioreq->domids[i];
+segs[i].dest.foreign.offset = ioreq->req.seg[i].first_sect * 
file_blk;
+segs[i].source.virt = ioreq->v.iov[i].iov_base;
+} else {
+segs[i].flags = GNTCOPY_source_gref;
+segs[i].source.foreign.ref = ioreq->refs[i];
+segs[i].source.foreign.domid = ioreq->domids[i];
+segs[i].source.foreign.offset = ioreq->req.seg[i].first_sect * 
file_blk;
+segs[i].dest.virt = ioreq->v.iov[i].iov_base;
+}
+segs[i].len = (ioreq->req.seg[i].last_sect
+   - ioreq->req.seg[i].first_sect + 1) * file_blk;
+}
+
+rc = xengnttab_grant_copy(gnt, count, segs);
+
+if (rc) {
+xen_be_printf(>blkdev->xendev, 0,
+  "failed to copy data %d\n", rc);
+ioreq->aio_errors++;
+return -1;
+}
+
+for (i = 0; i < count; i++) {
+if (segs[i].status != GNTST_okay) {
+xen_be_printf(>blkdev->xendev, 3,
+  "failed to copy data %d for gref %d, domid %d\n",
+  segs[i].status, 

[Xen-devel] [PATCH v7 1/2] libs/gnttab: introduce grant copy interface

[Paulina Szubarczyk]

In a linux part an ioctl(gntdev, IOCTL_GNTDEV_GRANT_COPY, ..)
system call is invoked. In mini-os the operation is yet not
implemented. For the OSs that does not implement gnttab the
call of the grant copy operation causes abort.

Signed-off-by: Paulina Szubarczyk 
Reviewed-by: David Vrabel 
---
 tools/include/xen-sys/Linux/gntdev.h  | 21 ++
 tools/libs/gnttab/Makefile|  2 +-
 tools/libs/gnttab/gnttab_core.c   |  6 +++
 tools/libs/gnttab/gnttab_unimp.c  |  6 +++
 tools/libs/gnttab/include/xengnttab.h | 28 +
 tools/libs/gnttab/libxengnttab.map|  5 +++
 tools/libs/gnttab/linux.c | 77 +++
 tools/libs/gnttab/minios.c|  6 +++
 tools/libs/gnttab/private.h   |  4 ++
 9 files changed, 154 insertions(+), 1 deletion(-)

diff --git a/tools/include/xen-sys/Linux/gntdev.h 
b/tools/include/xen-sys/Linux/gntdev.h
index caf6fb4..0ca07c9 100644
--- a/tools/include/xen-sys/Linux/gntdev.h
+++ b/tools/include/xen-sys/Linux/gntdev.h
@@ -147,4 +147,25 @@ struct ioctl_gntdev_unmap_notify {
 /* Send an interrupt on the indicated event channel */
 #define UNMAP_NOTIFY_SEND_EVENT 0x2
 
+struct ioctl_gntdev_grant_copy_segment {
+union {
+void *virt;
+struct {
+uint32_t ref;
+uint16_t offset;
+uint16_t domid;
+} foreign;
+} source, dest;
+uint16_t len;
+uint16_t flags;
+int16_t status;
+};
+
+#define IOCTL_GNTDEV_GRANT_COPY \
+_IOC(_IOC_NONE, 'G', 8, sizeof(struct ioctl_gntdev_grant_copy))
+struct ioctl_gntdev_grant_copy {
+unsigned int count;
+struct ioctl_gntdev_grant_copy_segment *segments;
+};
+
 #endif /* __LINUX_PUBLIC_GNTDEV_H__ */
diff --git a/tools/libs/gnttab/Makefile b/tools/libs/gnttab/Makefile
index af64542..95c2cd8 100644
--- a/tools/libs/gnttab/Makefile
+++ b/tools/libs/gnttab/Makefile
@@ -2,7 +2,7 @@ XEN_ROOT = $(CURDIR)/../../..
 include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR= 1
-MINOR= 0
+MINOR= 1
 SHLIB_LDFLAGS += -Wl,--version-script=libxengnttab.map
 
 CFLAGS   += -Werror -Wmissing-prototypes
diff --git a/tools/libs/gnttab/gnttab_core.c b/tools/libs/gnttab/gnttab_core.c
index 5d0474d..968c833 100644
--- a/tools/libs/gnttab/gnttab_core.c
+++ b/tools/libs/gnttab/gnttab_core.c
@@ -113,6 +113,12 @@ int xengnttab_unmap(xengnttab_handle *xgt, void 
*start_address, uint32_t count)
 return osdep_gnttab_unmap(xgt, start_address, count);
 }
 
+int xengnttab_grant_copy(xengnttab_handle *xgt,
+ uint32_t count,
+ xengnttab_grant_copy_segment_t *segs)
+{
+return osdep_gnttab_grant_copy(xgt, count, segs);
+}
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/gnttab/gnttab_unimp.c b/tools/libs/gnttab/gnttab_unimp.c
index b3a4a20..829eced 100644
--- a/tools/libs/gnttab/gnttab_unimp.c
+++ b/tools/libs/gnttab/gnttab_unimp.c
@@ -78,6 +78,12 @@ int xengnttab_unmap(xengnttab_handle *xgt, void 
*start_address, uint32_t count)
 abort();
 }
 
+int xengnttab_copy_grant(xengnttab_handle *xgt,
+ uint32_t count,
+ xengnttab_copy_grant_segment_t *segs)
+{
+abort();
+}
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/gnttab/include/xengnttab.h 
b/tools/libs/gnttab/include/xengnttab.h
index 0431dcf..35be6c1 100644
--- a/tools/libs/gnttab/include/xengnttab.h
+++ b/tools/libs/gnttab/include/xengnttab.h
@@ -258,6 +258,34 @@ int xengnttab_unmap(xengnttab_handle *xgt, void 
*start_address, uint32_t count);
 int xengnttab_set_max_grants(xengnttab_handle *xgt,
  uint32_t nr_grants);
 
+struct xengnttab_grant_copy_segment {
+union xengnttab_copy_ptr {
+void *virt;
+struct {
+uint32_t ref;
+uint16_t offset;
+uint16_t domid;
+} foreign;
+} source, dest;
+uint16_t len;
+uint16_t flags;
+int16_t status;
+};
+
+typedef struct xengnttab_grant_copy_segment xengnttab_grant_copy_segment_t;
+
+/**
+ * Copy memory from or to grant references. The information of each operations
+ * are contained in 'xengnttab_grant_copy_segment_t'. The @flag value indicate
+ * the direction of an operation (GNTCOPY_source_gref\GNTCOPY_dest_gref).
+ *
+ * For each segment, @virt may cross a page boundary but @offset + @len
+ * must not exceed XEN_PAGE_SIZE.
+ */
+int xengnttab_grant_copy(xengnttab_handle *xgt,
+ uint32_t count,
+ xengnttab_grant_copy_segment_t *segs);
+
 /*
  * Grant Sharing Interface (allocating and granting pages to others)
  */
diff --git a/tools/libs/gnttab/libxengnttab.map 
b/tools/libs/gnttab/libxengnttab.map
index dc737ac..f78da22 100644
--- a/tools/libs/gnttab/libxengnttab.map
+++ b/tools/libs/gnttab/libxengnttab.map
@@ -21,3 +21,8 @@ VERS_1.0 {
xengntshr_unshare;
local: *; /* Do not expose 

[Xen-devel] [PATCH v7 0/2] qemu-qdisk: Implementation of grant copy operation.

[Paulina Szubarczyk]

Hi,

It is a proposition for implementation of grant copy operation in qemu-qdisk 
and interface in libxc/libs. 

Changes since v6:
qemu-qdisk:
-removed blank lines
-renamed functions free_buffers -> ioreq_free_copy_buffers,
 ioreq_copy -> ioreq_grant_copy
-merged the if(ioreq_copy) with the conditions above

Changes since v5:
qemu-qdisk:
-added checking of every interface in the configure file. Based on
 the Roger's comment that xengnttab_map_grant_ref was added prior
 xengnttab_grant_copy, thus do not need to be check again here
 I dropped this check.

Changes since v4:
Interface:
- changed the subject line
- changed the comment in libs/gnttab/include/xengnttab.h according
  to the David's suggestion.
- removed unnecessary braces.

qemu-qdisk:
- in the configure file check only if xengnttab_grant_copy is
  implemented to verify 480 version of Xen.
- remove r variable and initialization of count to 0 in
  ioreq_copy.

- surround free_buffers, ioreq_init_copy_buffers and ioreq_copy
  by "#if CONFIG_XEN_CTRL_INTERFACE_VERSION >= 480" abort in else
  path because the function should not be called in that case.
- replace the definition of struct xengnttab_grant_copy_segment
  and a typedef to it with
  'typedef void* xengnttab_grant_copy_segment_t'.
- moved the new code in the xen_common.h to the end of the file.

Changes since v3:
Interface:
- revert to cast from xengnttab_grant_copy_segment_t
  to ioctl_gntdev_grant_copy.
- added compile-time check to compare the libs
  xengnttab_grant_copy_segment_t with the ioctl structure.
  The patch relies on Wei patch introducing XENGNTTAB_BUILD_BUG_ON in 
libs/gnttab.

qemu-qdisk:
- qemu_memalign/qemu_free is used instead function allocating
  memory from xc.
- removed the get_buffer function instead there is a direct call
  to qemu_memalign.
- moved ioreq_copy for write operation to ioreq_runio_qemu_aio.
- added struct xengnttab_grant_copy_segment_t and stub in
  xen_common.h for version of Xen earlier then 480.
- added checking for version 480 to configure. The test repeats
  all the operation that are required for version < 480 and
  checks if xengnttab_grant_copy() is implemented.

Changes since v2:
Interface:
- dropped the changes in libxc/include/xenctrl_compat
- changed the MINOR version in Makefile
- replaced 'return -1' -> 'abort()'in libs/gnttab/gnttab_unimp.c
- moved the struct 'xengnttab_copy_grant_segment' to 
  libs/gnttab/include/xengnttab.h
- added explicit assingment to ioctl_gntdev_grant_copy_segment 
  to the linux part

qemu-qdisk:
- to use the xengnttab_* function directly added -lxengnttab to configure
  and include  in include/hw/xen/xen_common.h
- in ioreq_copy removed an out path, changed a log level, made explicit 
  assignement to 'xengnttab_copy_grant_segment'
* I did not change the way of testing if grant_copy operation is implemented.
  As far as I understand if the code from gnttab_unimp.c is used then the 
gnttab 
  device is unavailable and the handler to gntdev would be invalid. But 
  if the handler is valid then the ioctl should return operation unimplemented 
  if the gntdev does not implement the operation.


Changes since v1:
Interface:
- changed the interface to call grant copy operation to match ioctl
int xengnttab_grant_copy(xengnttab_handle *xgt,
 uint32_t count,
 xengnttab_grant_copy_segment_t* segs)

- added a struct 'xengnttab_copy_grant_segment' definition to tools/libs
  /gnttab/private.h, tools/libxc/include/xenctrl_compat.h

- changed the function 'osdep_gnttab_grant_copy' which right now just
  call the ioctl

- added a new VER1.1 to tools/libs/gnttab/libxengnttab.map 

qemu-qdisk:
- removed the 'ioreq_write','ioreq_read_init','ioreq_read' functions 
- implemented 'ioreq_init_copy_buffers', 'ioreq_copy' 
- reverted the removal of grant map and introduced conditional invoking
  grant copy or grant map
- resigned from caching the local buffers on behalf of allocating the 
  required amount of pages at once. The cached structure would require 
  to have an lock guard and I suppose that the performance improvement 
  would degraded. 
 

For the functional test I attached the device with a qdisk backend to the 
guest, mounted, performed some reads and writes.

I run fio tests[1] with different iodepth and size of the block. The test can 
be 
accessed on my github[2] but mainly after the warm up I run for 60 seconds:
fio --time_based \
--clocksource=clock_gettime \
--rw=randread \
--random_distribution=pareto:0.9 \
--size=10g \
--direct='1' \
--ioengine=libaio \
--filename=$DEV \
--iodepth=$IODEPTH \
--bs=$BS \
--name=$NAME \
--runtime=$RUNTIME >> $FILENAME
The test were repeated at least three times. 

[1] 

[Xen-devel] [qemu-mainline test] 100951: tolerable FAIL - PUSHED

[osstest service owner]

flight 100951 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/100951/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt-qcow2 9 debian-di-install fail in 100941 pass in 
100951
 test-armhf-armhf-xl-rtds 11 guest-start  fail in 100941 pass in 100951
 test-armhf-armhf-xl-vhd  9 debian-di-install fail in 100941 pass in 100951
 test-armhf-armhf-libvirt  9 debian-install fail pass in 100941
 test-armhf-armhf-libvirt-xsm 11 guest-startfail pass in 100941

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds15 guest-start/debian.repeat fail REGR. vs. 100915
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop fail like 100915
 test-amd64-amd64-xl-rtds  9 debian-install   fail  like 100915
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stopfail like 100915

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt12 migrate-support-check fail in 100941 never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail in 100941 never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-check fail in 100941 never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail in 100941 never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass

version targeted for testing:
 qemuu507e4ddc3abf67391bcbc9624fd60b969c159b78
baseline version:
 qemuu7263da78045dc91cc207f350911efe4259e99b3c

Last test of basis   100915  2016-09-13 06:17:35 Z1 days
Testing same since   100941  2016-09-13 23:49:29 Z0 days2 attempts


People who touched revisions under test:
  Alberto Garcia 
  Artyom Tarasenko 
  Bruce Rogers 
  Cao jin 
  Changlong Xie 
  Eric Blake 
  Fam Zheng 
  Gerd Hoffmann 
  Gonglei 
  Jeff Cody 
  Laurent Vivier 
  Laurent Vivier 
  Li Zhijian 
  Michael S. Tsirkin 
  Michael Tokarev 
  Paolo Bonzini 
  Peter Maydell 
  Prasad J Pandit 
  Prasanna Kumar Kalever 
  Riku Voipio 
  Roman Pen 
  

Re: [Xen-devel] [PATCH 09/17] SVM: use generic instruction decoding

[Andrew Cooper]

On 08/09/16 14:14, Jan Beulich wrote:
> @@ -89,141 +54,96 @@ static unsigned long svm_nextrip_insn_le
>  return vmcb->nextrip - vmcb->rip;
>  }
>  
> -/* First byte: Length. Following bytes: Opcode bytes. */
> -#define MAKE_INSTR(nm, ...) static const u8 OPCODE_##nm[] = { __VA_ARGS__ }
> -MAKE_INSTR(INVD,   2, 0x0f, 0x08);
> -MAKE_INSTR(WBINVD, 2, 0x0f, 0x09);
> -MAKE_INSTR(CPUID,  2, 0x0f, 0xa2);
> -MAKE_INSTR(RDMSR,  2, 0x0f, 0x32);
> -MAKE_INSTR(WRMSR,  2, 0x0f, 0x30);
> -MAKE_INSTR(VMCALL, 3, 0x0f, 0x01, 0xd9);
> -MAKE_INSTR(HLT,1, 0xf4);
> -MAKE_INSTR(INT3,   1, 0xcc);
> -MAKE_INSTR(RDTSC,  2, 0x0f, 0x31);
> -MAKE_INSTR(PAUSE,  1, 0x90);
> -MAKE_INSTR(XSETBV, 3, 0x0f, 0x01, 0xd1);
> -MAKE_INSTR(VMRUN,  3, 0x0f, 0x01, 0xd8);
> -MAKE_INSTR(VMLOAD, 3, 0x0f, 0x01, 0xda);
> -MAKE_INSTR(VMSAVE, 3, 0x0f, 0x01, 0xdb);
> -MAKE_INSTR(STGI,   3, 0x0f, 0x01, 0xdc);
> -MAKE_INSTR(CLGI,   3, 0x0f, 0x01, 0xdd);
> -MAKE_INSTR(INVLPGA,3, 0x0f, 0x01, 0xdf);
> -
> -static const u8 *const opc_bytes[INSTR_MAX_COUNT] =
> -{
> -[INSTR_INVD]   = OPCODE_INVD,
> -[INSTR_WBINVD] = OPCODE_WBINVD,
> -[INSTR_CPUID]  = OPCODE_CPUID,
> -[INSTR_RDMSR]  = OPCODE_RDMSR,
> -[INSTR_WRMSR]  = OPCODE_WRMSR,
> -[INSTR_VMCALL] = OPCODE_VMCALL,
> -[INSTR_HLT]= OPCODE_HLT,
> -[INSTR_INT3]   = OPCODE_INT3,
> -[INSTR_RDTSC]  = OPCODE_RDTSC,
> -[INSTR_PAUSE]  = OPCODE_PAUSE,
> -[INSTR_XSETBV] = OPCODE_XSETBV,
> -[INSTR_VMRUN]  = OPCODE_VMRUN,
> -[INSTR_VMLOAD] = OPCODE_VMLOAD,
> -[INSTR_VMSAVE] = OPCODE_VMSAVE,
> -[INSTR_STGI]   = OPCODE_STGI,
> -[INSTR_CLGI]   = OPCODE_CLGI,
> -[INSTR_INVLPGA] = OPCODE_INVLPGA,
> +static const struct {
> +unsigned int opcode;
> +struct {
> +unsigned int rm:3;
> +unsigned int reg:3;
> +unsigned int mod:2;
> +#define MODRM(mod, reg, rm) { rm, reg, mod }
> +} modrm;
> +} const opc_tab[INSTR_MAX_COUNT] = {
> +[INSTR_PAUSE]  = { X86EMUL_OPC_F3(0, 0x90) },
> +[INSTR_INT3]   = { X86EMUL_OPC(   0, 0xcc) },
> +[INSTR_HLT]= { X86EMUL_OPC(   0, 0xf4) },
> +[INSTR_XSETBV] = { X86EMUL_OPC(0x0f, 0x01), MODRM(3, 2, 1) },
> +[INSTR_VMRUN]  = { X86EMUL_OPC(0x0f, 0x01), MODRM(3, 3, 0) },
> +[INSTR_VMCALL] = { X86EMUL_OPC(0x0f, 0x01), MODRM(3, 3, 1) },
> +[INSTR_VMLOAD] = { X86EMUL_OPC(0x0f, 0x01), MODRM(3, 3, 2) },
> +[INSTR_VMSAVE] = { X86EMUL_OPC(0x0f, 0x01), MODRM(3, 3, 3) },
> +[INSTR_STGI]   = { X86EMUL_OPC(0x0f, 0x01), MODRM(3, 3, 4) },
> +[INSTR_CLGI]   = { X86EMUL_OPC(0x0f, 0x01), MODRM(3, 3, 5) },
> +[INSTR_INVLPGA] = { X86EMUL_OPC(0x0f, 0x01), MODRM(3, 3, 7) },
> +[INSTR_INVD]   = { X86EMUL_OPC(0x0f, 0x08) },
> +[INSTR_WBINVD] = { X86EMUL_OPC(0x0f, 0x09) },
> +[INSTR_WRMSR]  = { X86EMUL_OPC(0x0f, 0x30) },
> +[INSTR_RDTSC]  = { X86EMUL_OPC(0x0f, 0x31) },
> +[INSTR_RDMSR]  = { X86EMUL_OPC(0x0f, 0x32) },
> +[INSTR_CPUID]  = { X86EMUL_OPC(0x0f, 0xa2) },
>  };
>  
> -static bool_t fetch(const struct vmcb_struct *vmcb, u8 *buf,
> -unsigned long addr, unsigned int len)
> -{
> -uint32_t pfec = (vmcb_get_cpl(vmcb) == 3) ? PFEC_user_mode : 0;
> -
> -switch ( hvm_fetch_from_guest_virt(buf, addr, len, pfec) )
> -{
> -case HVMCOPY_okay:
> -break;
> -case HVMCOPY_bad_gva_to_gfn:
> -/* OK just to give up; we'll have injected #PF already */
> -return 0;
> -default:
> -/* Not OK: fetches from non-RAM pages are not supportable. */
> -gdprintk(XENLOG_WARNING, "Bad instruction fetch at %#lx (%#lx)\n",
> - vmcb->rip, addr);
> -hvm_inject_hw_exception(TRAP_gp_fault, 0);
> -return 0;
> -}
> -return 1;
> -}
> -
>  int __get_instruction_length_from_list(struct vcpu *v,
>  const enum instruction_index *list, unsigned int list_count)
>  {
>  struct vmcb_struct *vmcb = v->arch.hvm_svm.vmcb;
> -unsigned int i, j, inst_len = 0;
> -enum instruction_index instr = 0;
> -u8 buf[MAX_INST_LEN];
> -const u8 *opcode = NULL;
> -unsigned long fetch_addr, fetch_limit;
> -unsigned int fetch_len, max_len;
> +struct hvm_emulate_ctxt ctxt;
> +struct x86_emulate_state *state;
> +unsigned int inst_len, j, modrm_rm, modrm_reg;
> +int modrm_mod;
>  
> +#ifdef NDEBUG

Presumably this is just for your testing?

>  if ( (inst_len = svm_nextrip_insn_length(v)) != 0 )
>  return inst_len;
>  
>  if ( vmcb->exitcode == VMEXIT_IOIO )
>  return vmcb->exitinfo2 - vmcb->rip;
> +#endif
>  
> -/* Fetch up to the next page break; we'll fetch from the next page
> - * later if we have to. */
> -fetch_addr = svm_rip2pointer(v, _limit);
> -if ( vmcb->rip > fetch_limit )
> -return 0;
> -max_len = min(fetch_limit - vmcb->rip + 1, MAX_INST_LEN + 0UL);
> -fetch_len = min_t(unsigned int, max_len,
> -  PAGE_SIZE - (fetch_addr & ~PAGE_MASK));
> -if ( 

[Xen-devel] [PATCH v4 1/5] x86/time: refactor init_platform_time()

[Joao Martins]

And accomodate platform time source initialization in
try_platform_time(). This is a preparatory patch for deferring
TSC clocksource initialization to the stage where all CPUS are
up (verify_tsc_reliability init call).

Signed-off-by: Joao Martins 
Reviewed-by: Konrad Rzeszutek Wilk 
Reviewed-by: Jan Beulich 
---
 xen/arch/x86/time.c | 30 --
 1 file changed, 20 insertions(+), 10 deletions(-)

diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c
index 73e0f98..0c1ad45 100644
--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -576,6 +576,24 @@ static void resume_platform_timer(void)
 plt_stamp = plt_src.read_counter();
 }
 
+static s64 __init try_platform_timer(struct platform_timesource *pts)
+{
+s64 rc = pts->init(pts);
+
+if ( rc <= 0 )
+return rc;
+
+plt_mask = (u64)~0ull >> (64 - pts->counter_bits);
+
+set_time_scale(_scale, pts->frequency);
+
+plt_overflow_period = scale_delta(
+1ull << (pts->counter_bits - 1), _scale);
+plt_src = *pts;
+
+return rc;
+}
+
 static u64 __init init_platform_timer(void)
 {
 static struct platform_timesource * __initdata plt_timers[] = {
@@ -593,7 +611,7 @@ static u64 __init init_platform_timer(void)
 pts = plt_timers[i];
 if ( !strcmp(opt_clocksource, pts->id) )
 {
-rc = pts->init(pts);
+rc = try_platform_timer(pts);
 break;
 }
 }
@@ -609,21 +627,13 @@ static u64 __init init_platform_timer(void)
 for ( i = 0; i < ARRAY_SIZE(plt_timers); i++ )
 {
 pts = plt_timers[i];
-if ( (rc = pts->init(pts)) > 0 )
+if ( (rc = try_platform_timer(pts)) > 0 )
 break;
 }
 }
 
 BUG_ON(rc <= 0);
 
-plt_mask = (u64)~0ull >> (64 - pts->counter_bits);
-
-set_time_scale(_scale, pts->frequency);
-
-plt_overflow_period = scale_delta(
-1ull << (pts->counter_bits-1), _scale);
-plt_src = *pts;
-
 printk("Platform timer is %s %s\n",
freq_string(pts->frequency), pts->name);
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v4 5/5] x86/time: extend "tsc" param with "stable:socket"

[Joao Martins]

Extend the "tsc" boot parameter is to further relax TSC restrictions and
allow it to be used on machines that guarantee reliable TSC across
sockets. This is up to board manufacturers and there's no way for the OS
to probe this property, therefore user needs to explicitly set this option.

Also make one style adjustment that is to remove the unnecessary
parenthesis around clearing TSC_RELIABLE.

Signed-off-by: Joao Martins 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 

Alternatively to having tsc_flags, I could instead introduce a new
X86_FEATURE in the Xen defined mapping. This would mean
introducing a "set_caps" similar to "cleared_caps" in order to set the
feature bit after x86_capability get's zeroed out in common
identify_cpu(). It was unclear to me what would maintainers prefer so
I went for the simplest for starters. If you prefer the other way
I can redo it.

NB: Didn't add "stable:node" (and consequently tsc=stable as it would
refer to both) because I don't have a host with multiple nodes that
I can test with.
---
 docs/misc/xen-command-line.markdown |  6 --
 xen/arch/x86/time.c | 11 ---
 2 files changed, 12 insertions(+), 5 deletions(-)

diff --git a/docs/misc/xen-command-line.markdown 
b/docs/misc/xen-command-line.markdown
index f92fb3f..7161788 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -270,7 +270,9 @@ If set, override Xen's default choice for the platform 
timer.
 Having TSC as platform timer requires being explicitly set. This is because
 TSC can only be safely used if CPU hotplug isn't performed on the system. In
 some platforms, "maxcpus" parameter may require further adjustment to the
-number of online cpus.
+number of online cpus. When running under platforms that can guarantee a
+monotonic TSC across sockets you require adjusting "tsc" command line parameter
+parameter to "stable:sockets".
 
 ### cmci-threshold
 > `= `
@@ -1508,7 +1510,7 @@ pages) must also be specified via the tbuf\_size 
parameter.
 > `= `
 
 ### tsc
-> `= unstable | skewed`
+> `= unstable | skewed | stable:socket`
 
 ### ucode
 > `= [ | scan]`
diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c
index 0c1badc..c1255db 100644
--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -477,6 +477,10 @@ uint64_t ns_to_acpi_pm_tick(uint64_t ns)
 /
  * PLATFORM TIMER 4: TSC
  */
+static unsigned int __read_mostly tsc_flags;
+
+/* TSC is reliable across sockets */
+#define TSC_RELIABLE_SOCKET (1 << 0)
 
 /*
  * Called in verify_tsc_reliability() under reliable TSC conditions
@@ -492,7 +496,7 @@ static s64 __init init_tsc(struct platform_timesource *pts)
 ret = 0;
 }
 
-if ( nr_sockets > 1 )
+if ( nr_sockets > 1 && !(tsc_flags & TSC_RELIABLE_SOCKET) )
 {
 printk(XENLOG_INFO "TSC: Not invariant across sockets\n");
 ret = 0;
@@ -1851,6 +1855,7 @@ int hwdom_pit_access(struct ioreq *ioreq)
 /*
  * tsc=unstable: Override all tests; assume TSC is unreliable.
  * tsc=skewed: Assume TSCs are individually reliable, but skewed across CPUs.
+ * tsc=stable:socket: Assume TSCs are reliable across sockets.
  */
 static void __init tsc_parse(const char *s)
 {
@@ -1861,9 +1866,9 @@ static void __init tsc_parse(const char *s)
 setup_clear_cpu_cap(X86_FEATURE_TSC_RELIABLE);
 }
 else if ( !strcmp(s, "skewed") )
-{
 setup_clear_cpu_cap(X86_FEATURE_TSC_RELIABLE);
-}
+else if ( !strcmp(s, "stable:socket") )
+tsc_flags |= TSC_RELIABLE_SOCKET;
 }
 custom_param("tsc", tsc_parse);
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v4 3/5] x86/time: refactor read_platform_stime()

[Joao Martins]

To allow the caller to fetch the last read from the clocksource which
was used to calculate system_time. This is a prerequisite for a
subsequent patch that will use this last read.

Signed-off-by: Joao Martins 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 

Changes since v3:
 - Add mention of this being a prerequisite to a later patch.
---
 xen/arch/x86/time.c | 22 +-
 1 file changed, 13 insertions(+), 9 deletions(-)

diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c
index e5001d5..af9e31f 100644
--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -581,18 +581,22 @@ static void plt_overflow(void *unused)
 set_timer(_overflow_timer, NOW() + plt_overflow_period);
 }
 
-static s_time_t read_platform_stime(void)
+static s_time_t read_platform_stime(u64 *stamp)
 {
-u64 count;
+u64 plt_counter, count;
 s_time_t stime;
 
 ASSERT(!local_irq_is_enabled());
 
 spin_lock(_timer_lock);
-count = plt_stamp64 + ((plt_src.read_counter() - plt_stamp) & plt_mask);
+plt_counter = plt_src.read_counter();
+count = plt_stamp64 + ((plt_counter - plt_stamp) & plt_mask);
 stime = __read_platform_stime(count);
 spin_unlock(_timer_lock);
 
+if ( stamp )
+*stamp = plt_counter;
+
 return stime;
 }
 
@@ -726,7 +730,7 @@ void cstate_restore_tsc(void)
 if ( boot_cpu_has(X86_FEATURE_NONSTOP_TSC) )
 return;
 
-write_tsc(stime2tsc(read_platform_stime()));
+write_tsc(stime2tsc(read_platform_stime(NULL)));
 }
 
 /***
@@ -1045,7 +1049,7 @@ int cpu_frequency_change(u64 freq)
 
 local_irq_disable();
 /* Platform time /first/, as we may be delayed by platform_timer_lock. */
-t->stamp.master_stime = read_platform_stime();
+t->stamp.master_stime = read_platform_stime(NULL);
 curr_tsc = rdtsc_ordered();
 /* TSC-extrapolated time may be bogus after frequency change. */
 /*t->stamp.local_stime = get_s_time_fixed(curr_tsc);*/
@@ -1350,7 +1354,7 @@ static void time_calibration_tsc_rendezvous(void *_r)
 
 if ( r->master_stime == 0 )
 {
-r->master_stime = read_platform_stime();
+r->master_stime = read_platform_stime(NULL);
 r->master_tsc_stamp = rdtsc_ordered();
 }
 atomic_inc(>semaphore);
@@ -1390,7 +1394,7 @@ static void time_calibration_std_rendezvous(void *_r)
 {
 while ( atomic_read(>semaphore) != (total_cpus - 1) )
 cpu_relax();
-r->master_stime = read_platform_stime();
+r->master_stime = read_platform_stime(NULL);
 smp_wmb(); /* write r->master_stime /then/ signal */
 atomic_inc(>semaphore);
 }
@@ -1429,7 +1433,7 @@ void time_latch_stamps(void)
 unsigned long flags;
 
 local_irq_save(flags);
-ap_bringup_ref.master_stime = read_platform_stime();
+ap_bringup_ref.master_stime = read_platform_stime(NULL);
 ap_bringup_ref.local_tsc = rdtsc_ordered();
 local_irq_restore(flags);
 
@@ -1447,7 +1451,7 @@ void init_percpu_time(void)
 t->tsc_scale = per_cpu(cpu_time, 0).tsc_scale;
 
 local_irq_save(flags);
-now = read_platform_stime();
+now = read_platform_stime(NULL);
 tsc = rdtsc_ordered();
 local_irq_restore(flags);
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v4 2/5] x86/time: implement tsc as clocksource

[Joao Martins]

Recent x86/time changes improved a lot of the monotonicity in xen
timekeeping, making it much harder to observe time going backwards.
Although platform timer can't be expected to be perfectly in sync with
TSC and so get_s_time won't be guaranteed to always return
monotonically increasing values across cpus.  This is the case in some
of the boxes I am testing with, observing sometimes ~100 warps (of
very few nanoseconds each) after a few hours.

This patch introduces support for using TSC as platform time source
which is the highest resolution time and most performant to get.
Though there are also several problems associated with its usage, and
there isn't a complete (and architecturally defined) guarantee that
all machines will provide reliable and monotonic TSC in all cases (I
believe Intel to be the only that can guarantee that?) For this reason
it's set with less priority when compared to HPET unless adminstrator
changes "clocksource" boot option to "tsc". Initializing TSC
clocksource requires all CPUs up to have the tsc reliability checks
performed. init_xen_time is called before all CPUs are up, so for
example we would start with HPET (or ACPI, PIT) at boot time, and
switch later to TSC. The switch then happens on verify_tsc_reliability
initcall that is invoked when all CPUs are up. When attempting to
initialize TSC we also check for time warps and if it has invariant
TSC. Note that while we deem reliable a CONSTANT_TSC with no deep
C-states, it might not always be the case, so we're conservative and
allow TSC to be used as platform timer only with invariant TSC.
Additionally we check if CPU Hotplug isn't meant to be performed on
the host which will either be when max vcpus and num_present_cpu are
the same. This is because a newly hotplugged CPU may not satisfy the
condition of having all TSCs synchronized - so when having tsc
clocksource being used we allow offlining CPUs but not onlining any
ones back. Finally we prevent TSC from being used as clocksource on
multiple sockets because it isn't guaranteed to be invariant. Further
relaxing of this last requirement is added in a separate patch, such
that we allow vendors with such guarantee to use TSC as clocksource.
In case any of these conditions is not met, we keep the clocksource
that was previously initialized on init_xen_time.

Since b64438c7c ("x86/time: use correct (local) time stamp in
constant-TSC calibration fast path") updates to cpu time use local
stamps, which means platform timer is only used to seed the initial
cpu time.  With clocksource=tsc there is no need to be in sync with
another clocksource, so we reseed the local/master stamps to be values
of TSC and update the platform time stamps accordingly. Time
calibration is set to 1sec after we switch to TSC, thus these stamps
are reseeded to also ensure monotonic returning values right after the
point we switch to TSC. This is also to avoid the possibility of
having inconsistent readings in this short period (i.e. until
calibration fires).

Signed-off-by: Joao Martins 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 

Changes since v3:
 - Really fix "HPET switching to TSC" comment. Despite mentioned in the
 in previous version, the change wasn't there.
 - Remove parenthesis around the function call in init_platform_timer
 - Merge if on verify_tsc_reliability with opt_clocksource check
 - Removed comment above ".init = init_tsctimer"
 - Fixup docs updated into this patch.
 - Move host_tsc_is_clocksource() and CPU hotplug possibility check to this
 patch.
 - s/host_tsc_is_clocksource/clocksource_is_tsc
 - Use bool instead of bool_t
 - Add a comment above init_tsctimer() declaration mentioning the
   reliable TSC checks on verify_tsc_reliability(), under which the
   function is invoked.
 - Prevent clocksource=tsc on platforms with multiple sockets. Further
 relaxing of this requirement is added in a separate patch, as
 extension of "tsc" boot parameter.
 - Removed control group to update cpu_time and do instead with
   on_selected_cpus to avoid any potential races.
 - Accomodate common path between init_xen_time and TSC switch into
 try_platform_timer_tail, such that finishing platform timer
 initialization is done in the same place (including platform timer
 overflow which was previously was removed in previous versions).
 - Changed TSC counter_bits 63 to avoid mishandling of TSC counter
 wrap-around in platform timer overflow timer.
 - Moved paragraph CPU Hotplug from last patch and add comment on
   commit message about multiple sockets TSC sync.
 - s/init_tsctimer/init_tsc/g to be consistent with other TSC platform
 timer functions.

Changes since v2:
 - Suggest "HPET switching to TSC" only as an example as otherwise it
 would be misleading on platforms not having one.
 - Change init_tsctimer to skip all the tests and assume it's called
 only on reliable TSC conditions and no warps observed. Tidy
 initialization on 

[Xen-devel] [PATCH v4 4/5] x86/time: implement PVCLOCK_TSC_STABLE_BIT

[Joao Martins]

This patch proposes relying on host TSC synchronization and
passthrough to the guest, when running on a TSC-safe platform. On
time_calibration we retrieve the platform time in ns and the counter
read by the clocksource that was used to compute system time. We
introduce a new rendezous function which doesn't require
synchronization between master and slave CPUS and just reads
calibration_rendezvous struct and writes it down the stime and stamp
to the cpu_calibration struct to be used later on. We can guarantee
that on a platform with a constant and reliable TSC, that the time
read on vcpu B right after A is bigger independently of the VCPU
calibration error. Since pvclock time infos are monotonic as seen by
any vCPU set PVCLOCK_TSC_STABLE_BIT, which then enables usage of VDSO
on Linux.  IIUC, this is similar to how it's implemented on KVM. Add
also a comment regarding this bit changing and that guests are
expected to check this bit on every read.

Should note that I've yet to see time going backwards in a long running
test for 2 weeks (in a dual socket machine), plus few other tests I did
on older platforms, including migration.

Signed-off-by: Joao Martins 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 

Changes since v3:
 - Do not adjust time_calibration_rendezvous_tail for nop_rendezvous and
 instead set cpu_time_stamp directly on the rendezvous function.
 - Move CPU Hotplug checks into patch 2
 - Add a commit and code comment regarding guests cope with this bit
   changing on hosts.
 - s/host_tsc_is_clocksource/clocksource_is_tsc

Changes since v2:
 - Add XEN_ prefix to pvclock flags.
 - Adapter time_calibration_rendezvous_tail to have the case of setting master
 tsc/stime and use it for the nop_rendezvous.
 - Removed hotplug CPU option that was added in v1
 - Prevent online of CPUs when clocksource is tsc.
 - Remove use_tsc_stable_bit, since clocksource is only used to seed
 values. So instead we test if hotplug is possible, and prevent clocksource=tsc
 to be used.
 - Remove 1st paragrah of commit message since the behaviour described
   no longer applies since b64438c.

Changes since v1:
 - Change approach to skip std_rendezvous by introducing a
   nop_rendezvous
 - Change commit message reflecting the change above.
 - Use TSC_STABLE_BIT only if cpu hotplug isn't possible.
 - Add command line option to override it if no cpu hotplug is
 intended.
---
 xen/arch/x86/time.c | 38 ++
 1 file changed, 38 insertions(+)

diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c
index af9e31f..0c1badc 100644
--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -951,6 +951,14 @@ static void __update_vcpu_system_time(struct vcpu *v, int 
force)
 _u.tsc_timestamp = tsc_stamp;
 _u.system_time   = t->stamp.local_stime;
 
+/*
+ * It's expected that domains cope with this bit changing on every
+ * pvclock read to check whether they can resort solely on this tuple
+ * or if it further requires monotonicity checks with other vcpus.
+ */
+if ( clocksource_is_tsc() )
+_u.flags|= XEN_PVCLOCK_TSC_STABLE_BIT;
+
 if ( is_hvm_domain(d) )
 _u.tsc_timestamp += v->arch.hvm_vcpu.cache_tsc_offset;
 
@@ -1409,6 +1417,22 @@ static void time_calibration_std_rendezvous(void *_r)
 time_calibration_rendezvous_tail(r);
 }
 
+/*
+ * Rendezvous function used when clocksource is TSC and
+ * no CPU hotplug will be performed.
+ */
+static void time_calibration_nop_rendezvous(void *rv)
+{
+const struct calibration_rendezvous *r = rv;
+struct cpu_time_stamp *c = _cpu(cpu_calibration);
+
+c->local_tsc= r->master_tsc_stamp;
+c->local_stime  = r->master_stime;
+c->master_stime = r->master_stime;
+
+raise_softirq(TIME_CALIBRATE_SOFTIRQ);
+}
+
 static void (*time_calibration_rendezvous_fn)(void *) =
 time_calibration_std_rendezvous;
 
@@ -1418,6 +1442,13 @@ static void time_calibration(void *unused)
 .semaphore = ATOMIC_INIT(0)
 };
 
+if ( clocksource_is_tsc() )
+{
+local_irq_disable();
+r.master_stime = read_platform_stime(_tsc_stamp);
+local_irq_enable();
+}
+
 cpumask_copy(_calibration_map, _online_map);
 
 /* @wait=1 because we must wait for all cpus before freeing @r. */
@@ -1587,6 +1618,13 @@ static int __init verify_tsc_reliability(void)
  */
 on_selected_cpus(_online_map, reset_percpu_time, NULL, 1);
 
+/*
+ * We won't do CPU Hotplug and TSC clocksource is being used which
+ * means we have a reliable TSC, plus we don't sync with any other
+ * clocksource so no need for rendezvous.
+ */
+time_calibration_rendezvous_fn = time_calibration_nop_rendezvous;
+
 /* Finish platform timer switch. */
 try_platform_timer_tail();
 
-- 
2.1.4


___

[Xen-devel] [PATCH v4 0/5] x86/time: PVCLOCK_TSC_STABLE_BIT support

[Joao Martins]

Hey,

This is v4 on the pvclock TSC series addressing comments from previous
version, see individual patches for complete changelog.

This series is divided as follows:

 R  * Patch 1: Small refactor around init_platform_time to reuse
   initialization code when switching to TSC.
 U  * Patch 2: Adds a new clocksource based on TSC
 U  * Patch 3: Prerequisite for patch 4
 U  * Patch 4: Implements the PVCLOCK_TSC_STABLE_BIT
 N  * Patch 5: Extend "tsc" param to relax monotonicity restriction
   across sockets.

 [ R := Reviewed-by ;; U := Updated ;; N := New ]

The benefit of this series is two-fold:
 
 1. Provide the guarantee of monotonic results on xen own system time as seen
 by any cpu when using TSC as clocksource.

 2. Provide this same guarantee to guests and thus set the
 TSC_STABLE_BIT (both FreeBSD and Linux support it) which then allows guests to
 skip expensive monotonicity check between PV CPU time infos. Plus, on Linux
 specifically this also means that it could support vDSO which greatly increases
 performance (x10) for gettimeofday and clock_gettime since it would no
 longer need to do the system call to get a reliable snapshot of system time.
 For a reference on my laptop the speed of gettimeofday under xen pvclock is 
 ~2 Mops/sec (Million ops per sec) whereas with vDSO it's on the range
 of ~22 Mops/sec on <= 4.4 kernels and ~37 Mops on >= 4.5.
 
 Doing a long running time warp test for 2 weeks on a dual-socket Haswell
 machine and I haven't yet seen time going backwards (plus tests on older
 multi socket machines). Furthermore double checked migration to/from hosts
 with/without the bit while guest was running the time warp test and no issues
 occurred too.

Thanks!

Joao Martins (5):
  x86/time: refactor init_platform_time()
  x86/time: implement tsc as clocksource
  x86/time: refactor read_platform_stime()
  x86/time: implement PVCLOCK_TSC_STABLE_BIT
  x86/time: extend "tsc" param with "stable:socket"

 docs/misc/xen-command-line.markdown |  10 +-
 xen/arch/x86/platform_hypercall.c   |   3 +-
 xen/arch/x86/time.c | 226 +++-
 xen/include/asm-x86/time.h  |   1 +
 4 files changed, 206 insertions(+), 34 deletions(-)

-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 08/17] x86emul: generate and make use of canonical opcode representation

[Andrew Cooper]

On 08/09/16 14:14, Jan Beulich wrote:

"of a canonical opcode representation".

You appear to be inventing your own here, but it isn't the only
canonical form you could represent x86 opcodes with.

> --- a/xen/arch/x86/x86_emulate/x86_emulate.h
> +++ b/xen/arch/x86/x86_emulate/x86_emulate.h
> @@ -415,12 +415,15 @@ struct x86_emulate_ctxt
>  /* Stack pointer width in bits (16, 32 or 64). */
>  unsigned int sp_size;
>  
> -/* Set this if writes may have side effects. */
> -uint8_t force_writeback;
> +/* Canonical opcode (see below). */
> +unsigned int opcode;
>  
>  /* Software event injection support. */
>  enum x86_swint_emulation swint_emulate;
>  
> +/* Set this if writes may have side effects. */
> +uint8_t force_writeback;
> +
>  /* Retirement state, set by the emulator (valid only on X86EMUL_OKAY). */
>  union {
>  struct {
> @@ -435,6 +438,51 @@ struct x86_emulate_ctxt
>  void *data;
>  };
>  
> +/*
> + * This encodes the opcode extension in a "natural" way:

I am not sure what you mean by natural way here.  All you seem to mean
is that you are encoding instructions with the following method

> + *0x0f for 0f-prefixed opcodes (or their VEX/EVEX equivalents)
> + *  0x0f38 for 0f38-prefixed opcodes (or their VEX/EVEX equivalents)
> + *  0x0f3a for 0f3a-prefixed opcodes (or their VEX/EVEX equivalents)
> + *  0x8f08 for 8f/8-prefixed XOP opcodes
> + *  0x8f09 for 8f/9-prefixed XOP opcodes
> + *  0x8f0a for 8f/a-prefixed XOP opcodes
> + * Hence no separate #define-s get added.

Please also describe what the  fields mean.  Looking below, I guess
that the bottom byte is the opcode itself, and some bits of the 2nd byte
are legacy prefixes?

> + */
> +#define X86EMUL_OPC_EXT_MASK 0x
> +#define X86EMUL_OPC(ext, byte)   ((byte) | \
> +  MASK_INSR((ext), X86EMUL_OPC_EXT_MASK))

I would highly suggest using ((byte) & 0xff).  In the case that a change
is slightly out of range, this should cause a compiler error (duplicate
case statement) rather than a very subtle bug.

> +/*
> + * This includes the 0x66, 0xF3, and 0xF2 prefixes when used to alter
> + * functionality instead of just insn attributes, as well as VEX/EVEX:
> + */
> +#define X86EMUL_OPC_MASK (0x00ff | X86EMUL_OPC_PFX_MASK | \
> + X86EMUL_OPC_KIND_MASK)

The definition should presumably live after introducing the PFX_MASK and
KIND_MASK ?

> +
> +#define X86EMUL_OPC_PFX_MASK 0x0300
> +# define X86EMUL_OPC_66(ext, byte)   (X86EMUL_OPC(ext, byte) | 0x0100)
> +# define X86EMUL_OPC_F3(ext, byte)   (X86EMUL_OPC(ext, byte) | 0x0200)
> +# define X86EMUL_OPC_F2(ext, byte)   (X86EMUL_OPC(ext, byte) | 0x0300)

The PFX mask is moderately obvious from here, but a sentence describing
what is legitimate to add in the future wouldn't go amiss.

> +
> +#define X86EMUL_OPC_KIND_MASK0x3000
> +#define X86EMUL_OPC_VEX_ 0x1000

OTOH, I am rather more confused about what is eligible for inclusion
into "kind".  Also, what does a kind of 0 indicate?

> +# define X86EMUL_OPC_VEX(ext, byte) \
> +(X86EMUL_OPC(ext, byte) | X86EMUL_OPC_VEX_)
> +# define X86EMUL_OPC_VEX_66(ext, byte) \
> +(X86EMUL_OPC_66(ext, byte) | X86EMUL_OPC_VEX_)
> +# define X86EMUL_OPC_VEX_F3(ext, byte) \
> +(X86EMUL_OPC_F3(ext, byte) | X86EMUL_OPC_VEX_)
> +# define X86EMUL_OPC_VEX_F2(ext, byte) \
> +(X86EMUL_OPC_F2(ext, byte) | X86EMUL_OPC_VEX_)
> +#define X86EMUL_OPC_EVEX_0x2000
> +# define X86EMUL_OPC_EVEX(ext, byte) \
> +(X86EMUL_OPC(ext, byte) | X86EMUL_OPC_EVEX_)
> +# define X86EMUL_OPC_EVEX_66(ext, byte) \
> +(X86EMUL_OPC_66(ext, byte) | X86EMUL_OPC_EVEX_)
> +# define X86EMUL_OPC_EVEX_F3(ext, byte) \
> +(X86EMUL_OPC_F3(ext, byte) | X86EMUL_OPC_EVEX_)
> +# define X86EMUL_OPC_EVEX_F2(ext, byte) \
> +(X86EMUL_OPC_F2(ext, byte) | X86EMUL_OPC_EVEX_)

Why do we go to the effort of spelling out the individual VEX/EVEX
possibilities, but not the XOP ones?

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 07/17] x86emul: move x86_execute() common epilogue code

[Andrew Cooper]

On 08/09/16 14:13, Jan Beulich wrote:
> Only code movement, no functional change.
>
> Signed-off-by: Jan Beulich 

Reviewed-by: Andrew Cooper 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] x86: fold code in load_segments()

[Andrew Cooper]

On 14/09/16 16:24, Jan Beulich wrote:
> No need to have the same logic twice.
>
> Signed-off-by: Jan Beulich 
>
> --- a/xen/arch/x86/domain.c
> +++ b/xen/arch/x86/domain.c
> @@ -1745,22 +1745,22 @@ static void load_segments(struct vcpu *n
>  (unsigned long *)pv->kernel_sp;
>  unsigned long cs_and_mask, rflags;
>  
> +/* Fold upcall mask and architectural IOPL into RFLAGS.IF. */
> +rflags  = regs->rflags & ~(X86_EFLAGS_IF|X86_EFLAGS_IOPL);
> +rflags |= !vcpu_info(n, evtchn_upcall_mask) << 9;
> +if ( VM_ASSIST(n->domain, architectural_iopl) )
> +rflags |= n->arch.pv_vcpu.iopl;
> +
>  if ( is_pv_32bit_vcpu(n) )
>  {
>  unsigned int *esp = ring_1(regs) ?
>  (unsigned int *)regs->rsp :
>  (unsigned int *)pv->kernel_sp;
> -unsigned int cs_and_mask, eflags;

The unshadowed cs_and_mask is unsigned long, not int, which means the
put_user() below will clobber a 32bit PV guests stack frame.

Other than that, Reviewed-by: Andrew Cooper 
for the intended change.

~Andrew

>  int ret = 0;
>  
>  /* CS longword also contains full evtchn_upcall_mask. */
>  cs_and_mask = (unsigned short)regs->cs |
>  ((unsigned int)vcpu_info(n, evtchn_upcall_mask) << 16);
> -/* Fold upcall mask into RFLAGS.IF. */
> -eflags  = regs->_eflags & ~(X86_EFLAGS_IF|X86_EFLAGS_IOPL);
> -eflags |= !vcpu_info(n, evtchn_upcall_mask) << 9;
> -if ( VM_ASSIST(n->domain, architectural_iopl) )
> -eflags |= n->arch.pv_vcpu.iopl;
>  
>  if ( !ring_1(regs) )
>  {
> @@ -1770,7 +1770,7 @@ static void load_segments(struct vcpu *n
>  }
>  
>  if ( ret |
> - put_user(eflags,  esp-1) |
> + put_user(rflags,  esp-1) |
>   put_user(cs_and_mask, esp-2) |
>   put_user(regs->_eip,  esp-3) |
>   put_user(uregs->gs,   esp-4) |
> @@ -1805,12 +1805,6 @@ static void load_segments(struct vcpu *n
>  cs_and_mask = (unsigned long)regs->cs |
>  ((unsigned long)vcpu_info(n, evtchn_upcall_mask) << 32);
>  
> -/* Fold upcall mask into RFLAGS.IF. */
> -rflags  = regs->rflags & ~(X86_EFLAGS_IF|X86_EFLAGS_IOPL);
> -rflags |= !vcpu_info(n, evtchn_upcall_mask) << 9;
> -if ( VM_ASSIST(n->domain, architectural_iopl) )
> -rflags |= n->arch.pv_vcpu.iopl;
> -
>  if ( put_user(regs->ss,rsp- 1) |
>   put_user(regs->rsp,   rsp- 2) |
>   put_user(rflags,  rsp- 3) |
>
>
>
>
>
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 06/17] x86emul: add EVEX decoding

[Andrew Cooper]

On 08/09/16 14:12, Jan Beulich wrote:
> This way we can at least size (and e.g. skip) them if needed, and we
> also won't raise the wrong fault due to not having read all relevant
> bytes.
>
> Signed-off-by: Jan Beulich 
> ---
> TBD: I'm kind of undecided whether to right away propagate evex.R into
>  modrm_reg (and then also deal with the new meaning of evex.x for
>  modrm_rm). Since that doesn't affect GPRs (and the extra bits
>  would need masking off when accessing GPRs) I've left this out for
>  now.
>
> --- a/xen/arch/x86/x86_emulate/x86_emulate.c
> +++ b/xen/arch/x86/x86_emulate/x86_emulate.c
> @@ -336,6 +336,27 @@ union vex {
>  ptr[1] = rex | REX_PREFIX; \
>  } while (0)
>  
> +union evex {
> +uint8_t raw[3];
> +struct {
> +uint8_t opcx:2;
> +uint8_t :2;

Is this legal syntax?  I am guessing it compiles for you, so is it
perhaps a GCCism?

> +uint8_t R:1;
> +uint8_t b:1;
> +uint8_t x:1;
> +uint8_t r:1;
> +uint8_t pfx:2;
> +uint8_t evex:1;
> +uint8_t reg:4;
> +uint8_t w:1;
> +uint8_t opmsk:3;
> +uint8_t RX:1;
> +uint8_t bcst:1;
> +uint8_t lr:2;
> +uint8_t z:1;
> +};
> +};
> +
>  #define rep_prefix()   (vex.pfx >= vex_f3)
>  #define repe_prefix()  (vex.pfx == vex_f3)
>  #define repne_prefix() (vex.pfx == vex_f2)
> @@ -1596,6 +1617,7 @@ struct x86_emulate_state {
>  bool lock_prefix;
>  opcode_desc_t desc;
>  union vex vex;
> +union evex evex;
>  int override_seg;
>  
>  /*
> @@ -1623,6 +1645,7 @@ struct x86_emulate_state {
>  #define rex_prefix (state->rex_prefix)
>  #define lock_prefix (state->lock_prefix)
>  #define vex (state->vex)
> +#define evex (state->evex)
>  #define override_seg (state->override_seg)
>  #define ea (state->ea)
>  
> @@ -1811,7 +1834,8 @@ x86_decode(
>  modrm = insn_fetch_type(uint8_t);
>  modrm_mod = (modrm & 0xc0) >> 6;
>  
> -if ( !ext && ((b & ~1) == 0xc4 || (b == 0x8f && (modrm & 0x18))) )
> +if ( !ext && ((b & ~1) == 0xc4 || (b == 0x8f && (modrm & 0x18)) ||
> +  b == 0x62) )
>  switch ( def_ad_bytes )
>  {
>  default:
> @@ -1825,7 +1849,7 @@ x86_decode(
>  break;
>  /* fall through */
>  case 8:
> -/* VEX / XOP */
> +/* VEX / XOP / EVEX */
>  generate_exception_if(rex_prefix || vex.pfx, EXC_UD, -1);
>  
>  vex.raw[0] = modrm;
> @@ -1852,6 +1876,14 @@ x86_decode(
>  op_bytes = 8;
>  }
>  }
> +if ( b == 0x62 )
> +{
> +evex.raw[0] = vex.raw[0];
> +evex.raw[1] = vex.raw[1];
> +evex.raw[2] = insn_fetch_type(uint8_t);
> +
> +vex.opcx = evex.opcx;

What is the meaning of opcx? The manuals list these as the mm fields.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] BUG_ON() vs ASSERT()

[Mihai Donțu]

On Tue, 13 Sep 2016 19:25:54 +0100 Andrew Cooper wrote:
> On 13/09/16 14:46, Mihai Donțu wrote:
> > On Tue, 13 Sep 2016 09:10:32 -0400 Konrad Rzeszutek Wilk wrote:  
> >> On Mon, Sep 12, 2016 at 09:23:41AM -0600, Jan Beulich wrote:  
> >>> All,
> >>>
> >>> in
> >>> https://lists.xenproject.org/archives/html/xen-devel/2016-09/msg01201.html
> >>> and
> >>> https://lists.xenproject.org/archives/html/xen-devel/2016-09/msg01210.html
> >>> Andrew basically suggests that we should switch away from using
> >>> ASSERT() and over to BUG_ON() in perhaps quite broad a set of
> >>> cases. And honestly I'm not convinced of this: We've been adding
> >>> quite a few ASSERT()s over the last years with the aim of doing
> >>> sanity checking in debug builds, without adding overhead to non-
> >>> debug builds. I can certainly see possible cases where using
> >>> BUG_ON() to prevent further possible damage is appropriate, but
> >>> I don't think we should overdo here.
> >>>
> >>> Thanks for other's opinions,
> >> I am in the mindset that ASSERTS are in the cases where a check
> >> has been done earlier and the ASSERT is more of a catch if we ended up
> >> somehow in the wrong state. We can then slowly follow the breadcrumbs to
> >> see what changed the state. In other words - something that the hypervisor
> >> has checked for and that invariant should have not changed.
> >>
> >> But a BUG_ON is in the same category - it should not have happend.
> >>
> >> Perhaps the distinction is that for ASSERTS() it is to catch me messing
> >> things up. While BUG_ON() is something (or somebody) else messing things 
> >> up.
> >>
> >> It is kind of hard to describe the semantic of an ASSERT vs BUG_ON now
> >> that I think of it ..  
> > I would see ASSERT() used to check for conditions that have immediate
> > and visible consequences, like the ones that lead to lack of
> > functionality (like a hw feature misdetection) or straight crashes
> > (like NULL-dereference).  
> 
> NULL dereferences in the context of PV guests are also a security issue,
> as the PV guest controls what is mapped at 0.
> 
> >  BUG_ON(), on the other hand, would be an early
> > warning for subtle corruptions that can lead to a hypervisor crash or
> > corrupted data after many hours of use (close to impossible to track
> > down).
> >
> > For example, a while ago I posted a small patch that would BUG_ON()
> > when it detected that the heap chunks were not properly linked. That's
> > the type of bug that's a pain to detect.  
> 
> Speaking of, what happened to that patch?

I did not post and updated version after the last review because I
wanted to produce some numbers too (wrt performance impact) ... and I
stopped there as I got distracted by other issues. I have a good mind
to update it, write a quick test and publish them. I hope I can do this
all sometime next week.

-- 
Mihai Donțu

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v4 2/2] xen: move TLB-flush filtering out into populate_physmap during vm creation

[Dario Faggioli]

On Mon, 2016-09-12 at 16:16 +0800, Dongli Zhang wrote:
> This patch implemented parts of TODO left in commit id
> a902c12ee45fc9389eb8fe54eeddaf267a555c58. 
>
We usually put both the (not necessarily full) hash and the subject
line of the commit in here.

> Signed-off-by: Dongli Zhang 
> 
> diff --git a/xen/common/domain.c b/xen/common/domain.c
> index a8804e4..7be1bee 100644
> @@ -303,6 +303,8 @@ struct domain *domain_create(domid_t domid,
> unsigned int domcr_flags,
>  if ( !zalloc_cpumask_var(>domain_dirty_cpumask) )
>  goto fail;
>  
> +d->is_ever_unpaused = false;
> +
>
I'd go for something like "first_unpaused" or "creation_finished", but
if maintainers are happy with this one already, I'm fine too.

> @@ -1004,6 +1006,15 @@ int domain_unpause_by_systemcontroller(struct
> domain *d)
>  {
>  int old, new, prev = d->controller_pause_count;
>  
> +/*
> + * Set is_ever_unpaused to true when this domain gets unpaused
> for the
> + * first time. We record this information here to help
> populate_physmap
> + * verify whether the domain has ever been unpaused.
> MEMF_no_tlbflush
> + * is allowed to be set by populate_physmap only during vm
> creation.
> + */

"We record this information here for populate_physmap to figure out
 that the domain has already been unpaused, after finishing being
 created. That's because we're allowed to set MEMF_no_tlbflush only
 during VM creation."

Or, de-focusing the unpausing even more:

"We record this information here for populate_physmap to figure out
 tha
t the domain has finished being created. In fact, we're only
 allowed to
set the MEMF_no_tlbflush flag during VM creation."

I.e., the important thing is not really the unpausing (that's where we
found it handy to put the check), it's the fact that something should
only happen at creation time and why (see below).

> +if ( unlikely(!d->is_ever_unpaused) )
> +d->is_ever_unpaused = true;
> +
>  do
>  {
>  old = prev;

> diff --git a/xen/common/memory.c b/xen/common/memory.c
> index cc0f69e..f3a733b 100644
> @@ -150,6 +152,14 @@ static void populate_physmap(struct memop_args
> *a)
>  max_order(curr_d)) )
>  return;
>  
> +/*
> + * MEMF_no_tlbflush can be set only during vm creation phase
> when
> + * is_ever_unpaused is still false before this domain gets
> unpaused for
> + * the first time.
> + */
>
What about, 'citing' from the changelog:

"With MEMF_no_tlbflush set, alloc_heap_pages() will ignore TLB-
 flushes. After VM creation, this is a security issue (it can make
 pages accessible to guest B, when guest A may still have a cached
 mapping to them). So we only do this only during domain creation,
 when the domain itself has not yet been unpaused for the first
 time."

> +if ( unlikely(!d->is_ever_unpaused) )
> +a->memflags |= MEMF_no_tlbflush;
> +
>  for ( i = a->nr_done; i < a->nr_extents; i++ )
>  {
>  if ( i != a->nr_done && hypercall_preempt_check() )

> diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
> index 2f9c15f..7fe8841 100644
> @@ -474,6 +474,9 @@ struct domain
>  unsigned int guest_request_enabled   : 1;
>  unsigned int guest_request_sync  : 1;
>  } monitor;
> +
> +/* set to true the first time this domain gets unpaused. */
>
I think it's relevant to say _when_ that is. What about:

/*
 * Set to true at the very end of domain creation, when the domain is 
 * unpaused for the first time by the systemcontroller.
 */

(not 100% happy about the "by the systemcontroller" part... but that's
the idea.)

> +bool_t is_ever_unpaused;
>
As said by Jan already --here and elsewhere-- new code should use
'bool'.

Regards,
Dario
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R Ltd., Cambridge (UK)



signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Outreachy Winter Internship

[George Dunlap]

On 11/09/16 11:52, Kavya Sharma wrote:
> Hello Sir,I am Kavya Sharma, an aspiring Outreachy intern.It would be my
> privilege to be an intern with xenproject.org this winter.I have read
> about Xen Hypervisor Userspace Tools and I am interested in your project
> 'golang bindings for libxl'.
> 
> Sir,can you please guide me on this and also give some suggestions so
> that I can work on the bite-sized projects so that I can fulfil the
> requirements.

Kavya,

Thanks for your interest in the Xen Project!  We've actually found
that one of the difficult parts of getting going with our project is
making sure that you understand how to get your whole system and
environment set up.  And another thing we want to see is to what
degree you can balance figuring things out, finding the answers on the
web, and asking for help when you need it.

So with that in mind, we've started experimenting with tasks which
don't contribute very much to the project directly, but provide a
really solid base of knowledge to do further contributions.

So here's my challenge for you.

---
OUTCOME

Attached is the very beginnings of a set of golang bindings that I
wrote for a project of my own.  They contain an implementation of
Context.Open() and Context.DomainInfo().

Write a simple go program that will take as an argument a single
domain id, and will output something that looks like the output of "xl
info [domid]".

That is, if your program was called "dominfo", then the output of
"dominfo 0" should look similar to the output of "xl info 0".

Please post a copy of your .go program, along with the results of output
both for domain 0, and another running (non-domain-0) domain.

STEPS

1. Set up a system running Linux

If you don't have one, Ubuntu, Fedora, or Debian should all be fine.

2. Download, build, and install the latest development
version of Xen.  The following page should be useful:

https://wiki.xenproject.org/wiki/Compiling_Xen_From_Source

I would recommend using "make debball" or "make rpmball" over the
"make install".

3. You'll need to build an image for at least one guest VM.

There are tons of options here, but one really simple thing would be
to follow this HOWTO from a previous OPW intern:

https://umasharma17.wordpress.com/2015/02/13/creating-guests-using-xl-in-xen/

4. Write your go program

The go program will need to Open() the context, then call DomainInfo()
on the target domain ID, and output the required info based on "xl
list".

libxl.go uses cgo to compile a library againt C.  In order for the cgo
builder to get the right libraries, you'll at least need to set the
CGO_LIBS environmetn variable to include both libyajl and libxl, like
this:

  export CGO_LIBS="-lyajl -lxenlight"

--

That's it!  Remember that the goal of this is to see how well you
balance figuring things out on your own vs asking questions.  So try
to figure things out on your own, but when you run into a bit of
difficultly, don't hesitate to ask questions or clarification --
particularly at the beginning.

You can ask questions either here on xen-devel or on the #xendevel or
#xen-opw channels on freenode IRC.  My nickname is 'gwd'.

Good luck,
 -George

package main

/*
#include 
*/
import "C"

import (
	"unsafe"
	"fmt"
	"time"
)

type Context struct {
	ctx *C.libxl_ctx
}

func NewContext() (Ctx *Context, err error) {
	Ctx = {}
	
	err = Ctx.Open()

	return
}

func (Ctx *Context) IsOpen() bool {
	return Ctx.ctx != nil
}

func (Ctx *Context) Open() (err error) {
	ret := C.libxl_ctx_alloc(unsafe.Pointer(), C.LIBXL_VERSION, 0, nil)

	if ret != 0 {
		err = fmt.Errorf("Allocating libxl context: %d", ret)
	}
	return
}

func (Ctx *Context) Close() (err error) {
	ret := C.libxl_ctx_free(unsafe.Pointer(Ctx.ctx))
	Ctx.ctx = nil

	if ret != 0 {
		err = fmt.Errorf("Freeing libxl context: %d", ret)
	}
	return
}

type Domid uint32

type MemKB uint64

// FIXME: Use the idl to generate types
type Dominfo struct {
	// FIXME: uuid
	Domid Domid
	Running   bool
	Blocked   bool
	Pausedbool
	Shutdown  bool
	Dying bool
	Never_stopbool
	
	Shutdown_reason   int32 // FIXME shutdown_reason enumeration
	Outstanding_memkb MemKB
	Current_memkb MemKB
	Shared_memkb  MemKB
	Paged_memkb   MemKB
	Max_memkb MemKB
	Cpu_time  time.Duration
	Vcpu_max_id   uint32
	Vcpu_online   uint32
	Cpupool   uint32
	Domain_type   int32 //FIXME libxl_domain_type enumeration

}

func (Ctx *Context) DomainInfo(Id Domid) (di *Dominfo, err error) {
	if Ctx.ctx == nil {
		err = fmt.Errorf("Context not opened")
		return
	}

		
	var cdi C.libxl_dominfo

	ret := C.libxl_domain_info(Ctx.ctx, unsafe.Pointer(), C.uint32_t(Id))

	// FIXME: IsDomainNotPresentError
	if ret != 0 {
		err = fmt.Errorf("libxl_domain_info failed: %d", ret)
		return
	}

	// FIXME -- use introspection to make this more robust
	di = {}
	di.Domid = Domid(cdi.domid)
	di.Running = bool(cdi.running)
	di.Blocked = 

Re: [Xen-devel] [PATCH 05/17] x86emul: add XOP decoding

[Jan Beulich]

>>> On 14.09.16 at 18:11,  wrote:
> On 08/09/16 14:11, Jan Beulich wrote:
>> @@ -1580,6 +1586,9 @@ struct x86_emulate_state {
>>  ext_0f   = vex_0f,
>>  ext_0f38 = vex_0f38,
>>  ext_0f3a = vex_0f3a,
>> +ext_8f08 = 8,
>> +ext_8f09,
>> +ext_8f0a,
> 
> What is this = 8 for?  I presume you didn't slip it in accidentally, but
> I still can't figure out why.

So I can use the value directly from vex.opcx, without further
adjustment.

>> @@ -1848,18 +1857,30 @@ x86_decode(
>>  rex_prefix |= REX_R;
>>  
>>  b = insn_fetch_type(uint8_t);
>> -switch ( ext = vex.opcx )
>> +ext = vex.opcx;
>> +if ( b != 0x8f )
>> +{
>> +switch ( ext )
>> +{
>> +case vex_0f:
>> +d = twobyte_table[b];
>> +break;
>> +case vex_0f38:
>> +d = twobyte_table[0x38];
>> +break;
>> +case vex_0f3a:
>> +d = twobyte_table[0x3a];
>> +break;
>> +default:
>> +rc = X86EMUL_UNHANDLEABLE;
>> +goto done;
>> +}
>> +}
>> +else if ( ext < ext_8f08 +
>> +sizeof(xop_table) / sizeof(*xop_table) )
> 
> ARRAY_SIZE() ?

If we want to add another helper #define to the test code, yes. It
being a single instance, that addition didn't seem worth it to me.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 2/2] x86/vm_event: Allow returning i-cache for emulation

[Tamas K Lengyel]

On Wed, Sep 14, 2016 at 9:55 AM, Jan Beulich  wrote:
 On 13.09.16 at 20:12,  wrote:
>> When emulating instructions the emulator maintains a small i-cache fetched
>> from the guest memory. This patch extends the vm_event interface to allow
>> returning this i-cache via the vm_event response instead.
>
> I guess I'm sightly confused: Isn't the purpose to have the monitoring
> app _write_ to the cache maintained in Xen? Or else, who is
> "emulator" here? If that's the app, then I think subject and description
> for hypervisor patches would better be written taking the perspective
> of the hypervisor, especially when using potentially ambiguous terms.

Well, I thought it was obvious that the built-in emulator in Xen is
what this patch is referring to. Otherwise none of this makes sense.

>
>> Note: This patch only has been compile-tested.
>
> This certainly needs to change before this can be committed.

I know, it's in process. That's why I put this note here to not get
too hasty about merging it.

>> @@ -1793,7 +1793,14 @@ static int _hvm_emulate_one(struct hvm_emulate_ctxt 
>> *hvmemul_ctxt,
>>  pfec |= PFEC_user_mode;
>>
>>  hvmemul_ctxt->insn_buf_eip = regs->eip;
>> -if ( !vio->mmio_insn_bytes )
>> +
>> +if ( unlikely(hvmemul_ctxt->set_context_insn) && curr->arch.vm_event )
>> +{
>> +hvmemul_ctxt->insn_buf_bytes = 
>> sizeof(curr->arch.vm_event->emul_insn);
>> +memcpy(hvmemul_ctxt->insn_buf, >arch.vm_event->emul_insn,
>> +   hvmemul_ctxt->insn_buf_bytes);
>> +}
>> +else if ( !vio->mmio_insn_bytes )
>>  {
>>  hvmemul_ctxt->insn_buf_bytes =
>>  hvm_get_insn_bytes(curr, hvmemul_ctxt->insn_buf) ?:
>
>
>
>> @@ -1944,11 +1951,19 @@ void hvm_mem_access_emulate_one(enum emul_kind kind, 
>> unsigned int trapnr,
>>  case EMUL_KIND_NOWRITE:
>>  rc = hvm_emulate_one_no_write();
>>  break;
>> -case EMUL_KIND_SET_CONTEXT:
>> -ctx.set_context = 1;
>> -/* Intentional fall-through. */
>> -default:
>> +case EMUL_KIND_SET_CONTEXT_DATA:
>> +ctx.set_context_data = 1;
>> +rc = hvm_emulate_one();
>> +break;
>> +case EMUL_KIND_SET_CONTEXT_INSN:
>> +ctx.set_context_insn = 1;
>>  rc = hvm_emulate_one();
>> +break;
>> +case EMUL_KIND_NORMAL:
>> +rc = hvm_emulate_one();
>> +break;
>
> One of the former two surely can be made fall through into the latter?

That's what I did before by turning this into if-else's and you
requested to go back to a switch. With a switch though I'll rather
make the cases explicit as a fall-through just makes it harder to read
for no real benefit.

>
>> +default:
>> +return;
>
> Why don't you retain the previous default handling?

The default label is unused and this makes it more apparent when
reading the code.

>
>> --- a/xen/include/asm-x86/hvm/emulate.h
>> +++ b/xen/include/asm-x86/hvm/emulate.h
>> @@ -34,20 +34,22 @@ struct hvm_emulate_ctxt {
>>
>>  uint32_t intr_shadow;
>>
>> -bool_t set_context;
>> +bool_t set_context_data;
>> +bool_t set_context_insn;
>
> As you have been told by others on patch 1 already - please take
> the opportunity to switch to plain bool.

Ack.

>
>> --- a/xen/include/asm-x86/vm_event.h
>> +++ b/xen/include/asm-x86/vm_event.h
>> @@ -27,7 +27,8 @@
>>   */
>>  struct arch_vm_event {
>>  uint32_t emulate_flags;
>> -struct vm_event_emul_read_data emul_read_data;
>> +struct vm_event_emul_read_data emul_read;
>> +struct vm_event_emul_insn_data emul_insn;
>
> Why don't these get put in a union, when ...
>
>> --- a/xen/include/public/vm_event.h
>> +++ b/xen/include/public/vm_event.h
>> @@ -97,6 +97,13 @@
>>   * Requires the vCPU to be paused already (synchronous events only).
>>   */
>>  #define VM_EVENT_FLAG_SET_REGISTERS  (1 << 8)
>> +/*
>> + * Instruction cache is being sent back to the hypervisor in the event 
>> response
>> + * to be used by the emulator. This flag is only useful when combined with
>> + * VM_EVENT_FLAG_EMULATE and is incompatible with also setting
>> + * VM_EVENT_FLAG_EMULATE_NOWRITE or VM_EVENT_FLAG_SET_EMUL_READ_DATA.
>> + */
>> +#define VM_EVENT_FLAG_SET_EMUL_INSN_DATA (1 << 9)
>
> ... place these restrictions and use a union in in the public header?

True, they can be put into a union there as well.

>
>> @@ -265,6 +272,10 @@ struct vm_event_emul_read_data {
>>  uint8_t  data[sizeof(struct vm_event_regs_x86) - sizeof(uint32_t)];
>>  };
>>
>> +struct vm_event_emul_insn_data {
>> +uint8_t data[16]; /* Has to be completely filled */
>> +};
>
> Any reason for the 16 (rather than the architectural 15) here?

Yes, see the definition in include/asm-x86/hvm/emulate.h:

struct hvm_emulate_ctxt {
struct x86_emulate_ctxt ctxt;

/* Cache of 16 bytes of instruction. */
uint8_t insn_buf[16];

Tamas

___

Re: [Xen-devel] [Help] Trigger Watchdog when adding an IPI in vcpu_wake

[Dario Faggioli]

On Wed, 2016-09-14 at 18:44 +0800, Wei Yang wrote:
> On Tue, Sep 13, 2016 at 01:30:17PM +0200, Dario Faggioli wrote:
> > 
> > Do you mind sharing just a bit more, such as:
> >  - number of pcpus
> >  - number of vcpus of the various VMs
> 160 pcpus
> 16 vcpus in VM and 8 VMs
> 
So, 16x8=128, which means you're not even oversubscribing. Maybe some
of the pcpus are hyperthreads and not full cores (are they?), but
still, this is not something I'd describe "super intensive cpu load".

Oh, well, there's dom0 of course. So, if dom0 has more than 160-128=32
vcpus (is this the case?), you technically are oversubscribing. But
then, what does dom0 do? Is it very busy doing some stuff on its own,
or running the backends/etc for the VMs? Can you check that with top,
xentop, and alike?

If the system is not overbooked, it's a bit strange that the scheduler
is the bottleneck.

> Yes, the "Blue Screen" is what we want to mimic the behavior from
> client.
> 
> The "Blue Screen" will force the hypervisor to do load balance in my
> mind.
> 
I've no idea what this means (but I don't know much of Windows and of
what happens when it crashes with a blue screen).

> > I'll have a quick look at how __runq_tickle() looks like in Xen
> > 4.1,
> > but there's very few chances I'll be available to provide detailed
> > review, advice, testing, etc., on such an old codebase. :-(
> > 
> > > 
> > > By looking at the code, what I missed may be in __runq_tickle(),
> > > which in case
> > > there are idle pcpu, schedule_softirq is also raised on them. By
> > > doing so,
> > > those idle pcpus would steal other busy tasks and may in chance
> > > get
> > > d2v2?
> > > 
> > Yes, it looks like, in Xen 4.1, this is more or less what happens.
> > The
> > idea is to always tickle pcpu 6, if the priority of the waking vcpu
> > is
> > higher than what is running there. 
> > 
> Hmm... in case there are idle pcpus and the priority of the waking
> vcpu is
> higher than what is running on pcpu 6, would pcpu 6 have more chance
> to run it?
> or other idle pcup would steal it from pcpu6? or they have equal
> chance?
> 
No, it works like this:

 - pcpu 6 is eithe idle or it is running someone already (say d3v4)
   + if pcpu 6 is idle, we tickle (i.e., we raise SCHEDULE_SOFTIRQ)
     pcpu 6 itself, and we're done
   + if pcpu 6 is running d3v4 and there is no other idle pcpu:
     * if prio(d2v2) > prio(d3v4) we tickle pcpu 6 and we're done
     * if prio(d2v2) < prio(d3v4) we just don't tickle anyone
   + if pcpu 6 is running d3v4 and there are some idle pcpus:
     * if prio(d2v2) > prio(d3v4) we tickle pcpu 6 _AND_ one or  [1]
       more of the idle pcpus
     * if prio(d2v2) < prio(d3v4) we _ONLY_ tickle one or more   [2]
       of the idle pcpus

Idea behind [1] is that d2v2 should preempt d3v4 on pcpu 6, and that's
why we tickle pcpu 6. However, that would mean that d3v4 would be put
back in pcpu's 6 runqueue. So, if there are idle pcpus, we tickle them
so that one can come and steam d3v4.

Idea behind [2] is that d3v4 should continue to run on pcpu 6, while
d2v2 will be put in pcpu's 6 runqueue. However, if there are idle
pcpus, we tickle them so that one can come and steal d2v2.

What really happens is not always what is expected, because it's
possible that, even if prio(d2v2)>prio(d3v4), an idler is quicker in
waking up and stealing d2v2 from pcpu's 6 runqueue where it was stashed
while pcpu 6 itself reschedules and enacts the preemption... But that's
unlikely and, all in all, not catastrophic (although, of course, worse
in terms of overhead, locality, etc)

> > If pcpu 6 was not idle, we also tickle one or more idle pcpus so
> > that:
> >  - if the waking vcpu preempted what was running on pcpu 6, an
> > idler
> >    can pick-up ("steal") such preempted vcpu and run it;
> >  - if the waking vcpu ended up in the runqueue, an idler can steal
> > it
> > 
> Hmm... I don't get the difference between these two cases.
> 
> Looks both are an idler steal the vcpu.
> 
I hope it's more clear now. :-)

> > Yes. But both in 4.1 and in current staging, we only raise
> > SCHEDULE_SOFTIRQ on pcpu 6, if and only if the priority of the
> > waking
> > vcpu is higher and it should preempt the currently running vcpu.
> > 
> Oh, you are right. I didn't catch this either.
> 
> This means in case 
> a) the priority is lower than current one 
> b) no idler in system
> 
> The vcpu will sit quietly in the runq and waiting for the schedule
> next time.
> 
Well, yes. It will stay in pcpu's 6 runqueue until either:
 - what's running on pcpu 6 blocks, or finishes its credits, etc.,
   and pcpu 6 reschedules and picks it up
 - some other pcpu becomes idle and, when poking around the various
   pcpus for stealing work, picks it up

Of course, both depends on in what position in pcpu's 6 runqueue the
vcpu is when the specific event happens.

This is a limit of the Credit1 scheduler, especially of the version
that you find in 4.1. Newer code is a bit better wrt this (although not

Re: [Xen-devel] [PATCH v4 1/2] xen: replace complicated tlbflush check with an inline function

[Jan Beulich]

>>> On 12.09.16 at 10:16,  wrote:
> This patch cleaned up the code by replacing complicated tlbflush check with
> an inline function. We should use this inline function to avoid the long
> and complicated to read tlbflush check when implementing TODOs left in
> commit a902c12ee45fc9389eb8fe54eeddaf267a555c58.



> --- a/xen/include/xen/mm.h
> +++ b/xen/include/xen/mm.h
> @@ -567,4 +567,15 @@ int prepare_ring_for_helper(struct domain *d, unsigned 
> long gmfn,
>  struct page_info **_page, void **_va);
>  void destroy_ring_for_helper(void **_va, struct page_info *page);
>  
> +static inline int page_needs_tlbflush(struct page_info *page,

bool and const please.

> +  bool_t need_tlbflush,

bool

But really passing this into a function with this name is kind of
awkward. Perhaps a better function name would be e.g.
accumulate_tlbflush(), and then this parameter would maybe
better be the first one.

> +  uint32_t tlbflush_timestamp,
> +  uint32_t tlbflush_current_time)

I don't think you should pass this into the function ...

> +{
> +return page->u.free.need_tlbflush &&
> +   page->tlbflush_timestamp <= tlbflush_current_time &&

... and use tlbflush_current_time() here instead.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 05/17] x86emul: add XOP decoding

[Andrew Cooper]

On 08/09/16 14:11, Jan Beulich wrote:
> This way we can at least size (and e.g. skip) them if needed, and we
> also won't raise the wrong fault due to not having read all relevant
> bytes.
>
> Signed-off-by: Jan Beulich 
>
> --- a/xen/arch/x86/x86_emulate/x86_emulate.c
> +++ b/xen/arch/x86/x86_emulate/x86_emulate.c
> @@ -279,6 +279,12 @@ static const opcode_desc_t twobyte_table
>  ModRM, ModRM, ModRM, ModRM, ModRM, ModRM, ModRM, ModRM
>  };
>  
> +static const opcode_desc_t xop_table[] = {
> +DstReg|SrcImmByte|ModRM,
> +DstReg|SrcMem|ModRM,
> +DstReg|SrcImm|ModRM,
> +};
> +
>  #define REX_PREFIX 0x40
>  #define REX_B 0x01
>  #define REX_X 0x02
> @@ -1580,6 +1586,9 @@ struct x86_emulate_state {
>  ext_0f   = vex_0f,
>  ext_0f38 = vex_0f38,
>  ext_0f3a = vex_0f3a,
> +ext_8f08 = 8,
> +ext_8f09,
> +ext_8f0a,

What is this = 8 for?  I presume you didn't slip it in accidentally, but
I still can't figure out why.

>  } ext;
>  uint8_t opcode;
>  uint8_t modrm, modrm_mod, modrm_reg, modrm_rm;
> @@ -1802,7 +1811,7 @@ x86_decode(
>  modrm = insn_fetch_type(uint8_t);
>  modrm_mod = (modrm & 0xc0) >> 6;
>  
> -if ( !ext && ((b & ~1) == 0xc4) )
> +if ( !ext && ((b & ~1) == 0xc4 || (b == 0x8f && (modrm & 0x18))) )
>  switch ( def_ad_bytes )
>  {
>  default:
> @@ -1816,11 +1825,11 @@ x86_decode(
>  break;
>  /* fall through */
>  case 8:
> -/* VEX */
> +/* VEX / XOP */
>  generate_exception_if(rex_prefix || vex.pfx, EXC_UD, -1);
>  
>  vex.raw[0] = modrm;
> -if ( b & 1 )
> +if ( b == 0xc5 )
>  {
>  vex.raw[1] = modrm;
>  vex.opcx = vex_0f;
> @@ -1848,18 +1857,30 @@ x86_decode(
>  rex_prefix |= REX_R;
>  
>  b = insn_fetch_type(uint8_t);
> -switch ( ext = vex.opcx )
> +ext = vex.opcx;
> +if ( b != 0x8f )
> +{
> +switch ( ext )
> +{
> +case vex_0f:
> +d = twobyte_table[b];
> +break;
> +case vex_0f38:
> +d = twobyte_table[0x38];
> +break;
> +case vex_0f3a:
> +d = twobyte_table[0x3a];
> +break;
> +default:
> +rc = X86EMUL_UNHANDLEABLE;
> +goto done;
> +}
> +}
> +else if ( ext < ext_8f08 +
> +sizeof(xop_table) / sizeof(*xop_table) )

ARRAY_SIZE() ?

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-4.6-testing test] 100948: regressions - trouble: blocked/broken/fail/pass

[osstest service owner]

flight 100948 xen-4.6-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/100948/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf-pvops4 host-build-prep fail in 100936 REGR. vs. 100895

Tests which are failing intermittently (not blocking):
 test-amd64-i386-migrupgrade   3 host-install/src_host(3) broken pass in 100936
 test-amd64-i386-freebsd10-i386 18 guest-start/freebsd.repeat fail in 100936 
pass in 100948
 test-amd64-amd64-xl-qemut-debianhvm-amd64 9 debian-hvm-install fail in 100936 
pass in 100948
 test-amd64-amd64-amd64-pvgrub 9 debian-di-install fail in 100936 pass in 100948
 test-amd64-amd64-xl-qemuu-debianhvm-amd64 9 debian-hvm-install fail in 100936 
pass in 100948
 test-amd64-i386-xl-qemuu-debianhvm-amd64 17 guest-start/debianhvm.repeat fail 
in 100936 pass in 100948
 test-amd64-i386-xl-qemut-debianhvm-amd64 9 debian-hvm-install fail pass in 
100936

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds 15 guest-start/debian.repeatfail  like 100895
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stopfail like 100895
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stopfail like 100895
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop fail like 100895
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop fail like 100895

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt  1 build-check(1)   blocked in 100936 n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked in 100936 n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked in 100936 n/a
 test-armhf-armhf-xl-cubietruck  1 build-check(1) blocked in 100936 n/a
 test-armhf-armhf-libvirt-xsm  1 build-check(1)   blocked in 100936 n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1) blocked in 100936 n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked in 100936 n/a
 test-armhf-armhf-xl-rtds  1 build-check(1)   blocked in 100936 n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)   blocked in 100936 n/a
 test-armhf-armhf-xl-xsm   1 build-check(1)   blocked in 100936 n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)  blocked in 100936 n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked in 100936 n/a
 test-amd64-i386-rumprun-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-rumprun-amd64  1 build-check(1)   blocked  n/a
 build-amd64-rumprun   5 rumprun-buildfail   never pass
 build-i386-rumprun5 rumprun-buildfail   never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 

Re: [Xen-devel] [PATCH 2/2] x86/vm_event: Allow returning i-cache for emulation

[Jan Beulich]

>>> On 13.09.16 at 20:12,  wrote:
> When emulating instructions the emulator maintains a small i-cache fetched
> from the guest memory. This patch extends the vm_event interface to allow
> returning this i-cache via the vm_event response instead.

I guess I'm sightly confused: Isn't the purpose to have the monitoring
app _write_ to the cache maintained in Xen? Or else, who is
"emulator" here? If that's the app, then I think subject and description
for hypervisor patches would better be written taking the perspective
of the hypervisor, especially when using potentially ambiguous terms.

> Note: This patch only has been compile-tested.

This certainly needs to change before this can be committed.

> @@ -1793,7 +1793,14 @@ static int _hvm_emulate_one(struct hvm_emulate_ctxt 
> *hvmemul_ctxt,
>  pfec |= PFEC_user_mode;
>  
>  hvmemul_ctxt->insn_buf_eip = regs->eip;
> -if ( !vio->mmio_insn_bytes )
> +
> +if ( unlikely(hvmemul_ctxt->set_context_insn) && curr->arch.vm_event )
> +{
> +hvmemul_ctxt->insn_buf_bytes = 
> sizeof(curr->arch.vm_event->emul_insn);
> +memcpy(hvmemul_ctxt->insn_buf, >arch.vm_event->emul_insn,
> +   hvmemul_ctxt->insn_buf_bytes);
> +}
> +else if ( !vio->mmio_insn_bytes )
>  {
>  hvmemul_ctxt->insn_buf_bytes =
>  hvm_get_insn_bytes(curr, hvmemul_ctxt->insn_buf) ?:



> @@ -1944,11 +1951,19 @@ void hvm_mem_access_emulate_one(enum emul_kind kind, 
> unsigned int trapnr,
>  case EMUL_KIND_NOWRITE:
>  rc = hvm_emulate_one_no_write();
>  break;
> -case EMUL_KIND_SET_CONTEXT:
> -ctx.set_context = 1;
> -/* Intentional fall-through. */
> -default:
> +case EMUL_KIND_SET_CONTEXT_DATA:
> +ctx.set_context_data = 1;
> +rc = hvm_emulate_one();
> +break;
> +case EMUL_KIND_SET_CONTEXT_INSN:
> +ctx.set_context_insn = 1;
>  rc = hvm_emulate_one();
> +break;
> +case EMUL_KIND_NORMAL:
> +rc = hvm_emulate_one();
> +break;

One of the former two surely can be made fall through into the latter?

> +default:
> +return;

Why don't you retain the previous default handling?

> --- a/xen/include/asm-x86/hvm/emulate.h
> +++ b/xen/include/asm-x86/hvm/emulate.h
> @@ -34,20 +34,22 @@ struct hvm_emulate_ctxt {
>  
>  uint32_t intr_shadow;
>  
> -bool_t set_context;
> +bool_t set_context_data;
> +bool_t set_context_insn;

As you have been told by others on patch 1 already - please take
the opportunity to switch to plain bool.

> --- a/xen/include/asm-x86/vm_event.h
> +++ b/xen/include/asm-x86/vm_event.h
> @@ -27,7 +27,8 @@
>   */
>  struct arch_vm_event {
>  uint32_t emulate_flags;
> -struct vm_event_emul_read_data emul_read_data;
> +struct vm_event_emul_read_data emul_read;
> +struct vm_event_emul_insn_data emul_insn;

Why don't these get put in a union, when ...

> --- a/xen/include/public/vm_event.h
> +++ b/xen/include/public/vm_event.h
> @@ -97,6 +97,13 @@
>   * Requires the vCPU to be paused already (synchronous events only).
>   */
>  #define VM_EVENT_FLAG_SET_REGISTERS  (1 << 8)
> +/*
> + * Instruction cache is being sent back to the hypervisor in the event 
> response
> + * to be used by the emulator. This flag is only useful when combined with
> + * VM_EVENT_FLAG_EMULATE and is incompatible with also setting
> + * VM_EVENT_FLAG_EMULATE_NOWRITE or VM_EVENT_FLAG_SET_EMUL_READ_DATA.
> + */
> +#define VM_EVENT_FLAG_SET_EMUL_INSN_DATA (1 << 9)

... place these restrictions and use a union in in the public header?

> @@ -265,6 +272,10 @@ struct vm_event_emul_read_data {
>  uint8_t  data[sizeof(struct vm_event_regs_x86) - sizeof(uint32_t)];
>  };
>  
> +struct vm_event_emul_insn_data {
> +uint8_t data[16]; /* Has to be completely filled */
> +};

Any reason for the 16 (rather than the architectural 15) here?

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Xen-users] Xen 4.7.0 graphics pass through problems (Win10, AMD Radeon)

[Peter Milesson]

On 2016-09-14 16:17, George Dunlap wrote:

On Wed, Sep 14, 2016 at 3:04 PM, Peter Milesson  wrote:


On 2016-09-14 14:50, Jan Beulich wrote:

On 14.09.16 at 13:34,  wrote:

   On 2016-09-09 09:51, Peter Milesson wrote:

[snip]

I've been using Xen for 3 years, starting out with 4.2, and I've been
upgrading regularly, and installed 4.7.0 today.

I'm using Windows 10, 64-bit with PCI pass through in a VM with
PV-drivers, using a AMD Radeon HD6450 card (AMD graphics cards don't
seem to
need graphics pass through)

Previously (up till Xen 4.6.3), the graphics output has displayed some
shorter lines, a bit like thin coarse snow, when watching videos. The
distortions stayed within the movie, and was tolerable.

After upgrade to Xen 4.7.0, the video performance is seriously ugly.
Just
for example, I open cnn.com and move the mouse pointer up and down over
the
photos, which creates a bunch of flickering lines over the display. The
same
with moving content in Youtube for example. Or opening a Cygwin
terminal
window and scrolling through a file. Terrible.

Nothing else has changed, only the Xen version.

[snip]

On Fri, Sep 9, 2016 at 10:35 AM, Peter Milesson  wrote:

Hi again,

I've been playing around a bit more.

It seems one of the problems is that I passed a couple of PCI-devices
(USB
controllers) to the VM, beside the graphics card.

When I pass the USB-devices individually in the VM configuration file
(usbdevice=['tablet','host:1.4','host:1:9','host:x.y']), the display
problems seem to be more or less gone.

Jan / Andy,

Can you think of anything that's changed between 4.6 and 4.7 that
would cause the performance problems he's describing when both a video
card and a usb controller are passed through, but not when only the
video card is passed through?

Not really, no. Peter - are there any indications of problems in
one or more of the logs (Xen, xl, qemu)? Did you try running a
debug build of all Xen components?

Jan


Hi Jan,

No, I did not try a debug build. However, the system seems fairly stable
now, only the display driver crashing once, or twice a day. That did occur
with Xen 4.6.3 also, but maybe once a week.

Well crashing twice a day isn't what most people would call "stable".
:-)  In any case, I was trying to debug the issue you had with passing
through both the video card and the usb controller -- that is
obviously an important configuration, but if there's a bug in that
configuration it probably affects a large number of wider
configurations as well.


Looking at the log files, I
cannot see anything particular that stands out, except for the domU log.

The domU log file qemu-dm-.log started to contain the following lines
after upgrading to 4.7:

[00:06.0] xen_pt_bar_read: Error: Should not read BAR through QEMU.
@0x0004
[00:06.0] xen_pt_bar_write: Error: Should not write BAR through QEMU.
@0x

There are only a couple of lines with write, the rest with read. I don't
know what significance it's got, but there are around 350 lines every day.

Yeah, those definitely look important.  This is with just the video
card passed through, or with both passed through?

  -George

Hi George,

This is with both the USB controller, and the graphics card.

Best regards,

Peter


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Xen-users] Xen 4.7.0 graphics pass through problems (Win10, AMD Radeon)

[Peter Milesson]

On 2016-09-14 15:30, George Dunlap wrote:

On 14/09/16 14:23, Andrew Cooper wrote:

On 14/09/16 13:50, Jan Beulich wrote:

On 14.09.16 at 13:34,  wrote:

  On 2016-09-09 09:51, Peter Milesson wrote:

[snip]

I've been using Xen for 3 years, starting out with 4.2, and I've been
upgrading regularly, and installed 4.7.0 today.

I'm using Windows 10, 64-bit with PCI pass through in a VM with
PV-drivers, using a AMD Radeon HD6450 card (AMD graphics cards don't seem to
need graphics pass through)

Previously (up till Xen 4.6.3), the graphics output has displayed some
shorter lines, a bit like thin coarse snow, when watching videos. The
distortions stayed within the movie, and was tolerable.

After upgrade to Xen 4.7.0, the video performance is seriously ugly. Just
for example, I open cnn.com and move the mouse pointer up and down over the
photos, which creates a bunch of flickering lines over the display. The same
with moving content in Youtube for example. Or opening a Cygwin terminal
window and scrolling through a file. Terrible.

Nothing else has changed, only the Xen version.

[snip]

On Fri, Sep 9, 2016 at 10:35 AM, Peter Milesson  wrote:

Hi again,

I've been playing around a bit more.

It seems one of the problems is that I passed a couple of PCI-devices (USB
controllers) to the VM, beside the graphics card.

When I pass the USB-devices individually in the VM configuration file
(usbdevice=['tablet','host:1.4','host:1:9','host:x.y']), the display
problems seem to be more or less gone.

Jan / Andy,

Can you think of anything that's changed between 4.6 and 4.7 that
would cause the performance problems he's describing when both a video
card and a usb controller are passed through, but not when only the
video card is passed through?

Not really, no. Peter - are there any indications of problems in
one or more of the logs (Xen, xl, qemu)? Did you try running a
debug build of all Xen components?

Furthermore, does "Nothing else has changed, only the Xen version." mean
switching your distro packages between two versions of xen, or literally
only switching the hypervisor itself.

I assume he's using "Xen" to mean the whole Xen system -- hypervisor +
tools + qemu.


A substantial quantity of the complexity here is in qemu, rather than Xen.

Rather than the hypervisor, you mean.

Another thing worth asking, Peter: Have you tried running with
qemu-traditional (by adding the line below to your config file) rather
than qemu-upstream, to see if that makes a difference?

device_model_version="qemu-xen-traditional"

qemu-traditional almost ever changes, so if it worked better, then that
would point the finger at changes in qemu (at which point we would have
to bring in a different set of people to help diagnose it).

  -George

Hi George,

I have always been using the default. I never had the need to set a 
specific device model. So it's qemu-upstream in my case.


Best regards,

Peter



___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Xen-users] Xen 4.7.0 graphics pass through problems (Win10, AMD Radeon)

[Peter Milesson]

On 2016-09-14 16:49, Jan Beulich wrote:

On 14.09.16 at 16:04,  wrote:

On 2016-09-14 14:50, Jan Beulich wrote:

On 14.09.16 at 13:34,  wrote:

Can you think of anything that's changed between 4.6 and 4.7 that
would cause the performance problems he's describing when both a video
card and a usb controller are passed through, but not when only the
video card is passed through?

Not really, no. Peter - are there any indications of problems in
one or more of the logs (Xen, xl, qemu)? Did you try running a
debug build of all Xen components?

No, I did not try a debug build. However, the system seems fairly stable
now, only the display driver crashing once, or twice a day. That did
occur with Xen 4.6.3 also, but maybe once a week. Looking at the log
files, I cannot see anything particular that stands out, except for the
domU log.

The domU log file qemu-dm-.log started to contain the following
lines after upgrading to 4.7:

[00:06.0] xen_pt_bar_read: Error: Should not read BAR through QEMU.
@0x0004
[00:06.0] xen_pt_bar_write: Error: Should not write BAR through QEMU.
@0x

There are only a couple of lines with write, the rest with read. I don't
know what significance it's got, but there are around 350 lines every day.

Looks wrong in any case, but I have no clue whether it's indicative
of the problem you're having. In any event in the qemu tree in
hw/xen/xen_pt.h there's an "#ifdef XEN_PT_LOGGING_ENABLED"
conditional close to the top. Could you make this "#if 1" instead
(or simply #define XEN_PT_LOGGING_ENABLED in the line above)
and re-build at least qemu? The log of a subsequent guest run
should then be quite a bit more helpful in possibly spotting what's
going wrong to cause those messages.

And then it's of course rather relevant to know what device 00:06.0
actually is (iirc these are guest BDF values), considering that you're
passing through more than one.

Jan



Hi Jan,

I had a look at the guest devices, and Windows says "Standard Enhanced 
PCI to USB Host Controller". Having a look at the Properties, in the 
Device status field is the following message:

---
This device cannot start. (Code 10)

{Operation Failed}
The requested operation was unsuccessful.
---

This USB device wasn't there with Xen 4.6.3. Has it got something to do 
with the new USB passthrough? However, I have tried USB passthrough (xl 
usbctrl-attach), which doesn't work. But that is possibly another problem.


No problem to recompile Xen with the logging flag, if it would be required.

Best regards,

Peter

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 19/38] arm/p2m: Add HVMOP_altp2m_switch_p2m

[Sergej Proskurin]

Hi Julien,


On 09/14/2016 12:57 PM, Julien Grall wrote:
>
>
> On 13/09/16 14:00, Sergej Proskurin wrote:
>> Hi Julien,
>
> Hello Sergej,
>
>>
>> On 09/12/2016 10:47 AM, Julien Grall wrote:
>>> Hello Sergej,
>>>
>>> On 16/08/2016 23:16, Sergej Proskurin wrote:
 Signed-off-by: Sergej Proskurin 
 ---
 Cc: Stefano Stabellini 
 Cc: Julien Grall 
 ---
 v3: Extended the function "altp2m_switch_domain_altp2m_by_id" so
 that if
 the guest domain indirectly calles this function, the current
 vcpu also
 changes the altp2m view without performing an explicit context
 switch.

 Exchanged the check "altp2m_vttbr[idx] == INVALID_VTTBR" for
 "altp2m_p2m[idx] == NULL" in "altp2m_switch_domain_altp2m_by_id".
 ---
  xen/arch/arm/altp2m.c| 48
 
  xen/arch/arm/hvm.c   |  2 +-
  xen/include/asm-arm/altp2m.h |  4 
  3 files changed, 53 insertions(+), 1 deletion(-)

 diff --git a/xen/arch/arm/altp2m.c b/xen/arch/arm/altp2m.c
 index c14ab0b..ba345b9 100644
 --- a/xen/arch/arm/altp2m.c
 +++ b/xen/arch/arm/altp2m.c
 @@ -32,6 +32,54 @@ struct p2m_domain *altp2m_get_altp2m(struct vcpu
 *v)
  return v->domain->arch.altp2m_p2m[index];
  }

 +int altp2m_switch_domain_altp2m_by_id(struct domain *d, unsigned int
 idx)
 +{
 +struct vcpu *v;
 +int rc = -EINVAL;
 +
 +if ( idx >= MAX_ALTP2M )
 +return rc;
 +
 +domain_pause_except_self(d);
 +
 +altp2m_lock(d);
 +
 +if ( d->arch.altp2m_p2m[idx] != NULL )
 +{
 +for_each_vcpu( d, v )
 +if ( idx != altp2m_vcpu(v).p2midx )
>>>
>>> Could you invert the condition to avoid another layer of nested if?
>>>
>>
>> Did you mean checking for (idx == altp2m_vcpu(v).p2midx) and continue
>> the loop if the condition should be satisfied? If so, then sure thing :)
>
> Correct.
>

Ok, done.

>>
 +{
 +atomic_dec(_get_altp2m(v)->active_vcpus);
 +altp2m_vcpu(v).p2midx = idx;
 +atomic_inc(_get_altp2m(v)->active_vcpus);
 +
 +/*
 + * In case it is the guest domain, which indirectly
 called this
 + * function, the current vcpu will not switch its
 context
 + * within the function "p2m_restore_state". That is,
 changing
 + * the altp2m_vcpu(v).p2midx will not lead to the
 requested
 + * altp2m switch on that specific vcpu. To achieve
 the desired
 + * behavior, we write to VTTBR_EL2 directly.
 + */
 +if ( v->domain == d && v == current )
>>>
>>> v == current is enough.
>>>
>>
>> Ok.
>>
 +{
 +struct p2m_domain *ap2m =
 d->arch.altp2m_p2m[idx];
 +
 +WRITE_SYSREG64(ap2m->vttbr, VTTBR_EL2);
 +isb();
>>>
>>> I don't like the open-coding of VTTBR_EL2. I would much prefer a
>>> separate helper to update it.
>>>
>>
>> Sure, I can introduce an UPDATE_VTTBR macro (including the isb call) in
>> p2m.h and adjust the remaining writes to VTTBR_EL2 in p2m.c as well. I
>> hope, I understood you correctly.
>
> Actually, I was concerned about having the following sequence spread:
>VTTBR_EL2
>isb
>
> However, the isb is not necessary here because VTTBR_EL2 will not be
> used until we return to the guest.
>
> So I would be fine to keep WRITE_SYSREG64(ap2m->vttbr, VTTBR_EL2);

I will change that, thank you.

Cheers,
~Sergej


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH] x86: fold code in load_segments()

[Jan Beulich]

No need to have the same logic twice.

Signed-off-by: Jan Beulich 

--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -1745,22 +1745,22 @@ static void load_segments(struct vcpu *n
 (unsigned long *)pv->kernel_sp;
 unsigned long cs_and_mask, rflags;
 
+/* Fold upcall mask and architectural IOPL into RFLAGS.IF. */
+rflags  = regs->rflags & ~(X86_EFLAGS_IF|X86_EFLAGS_IOPL);
+rflags |= !vcpu_info(n, evtchn_upcall_mask) << 9;
+if ( VM_ASSIST(n->domain, architectural_iopl) )
+rflags |= n->arch.pv_vcpu.iopl;
+
 if ( is_pv_32bit_vcpu(n) )
 {
 unsigned int *esp = ring_1(regs) ?
 (unsigned int *)regs->rsp :
 (unsigned int *)pv->kernel_sp;
-unsigned int cs_and_mask, eflags;
 int ret = 0;
 
 /* CS longword also contains full evtchn_upcall_mask. */
 cs_and_mask = (unsigned short)regs->cs |
 ((unsigned int)vcpu_info(n, evtchn_upcall_mask) << 16);
-/* Fold upcall mask into RFLAGS.IF. */
-eflags  = regs->_eflags & ~(X86_EFLAGS_IF|X86_EFLAGS_IOPL);
-eflags |= !vcpu_info(n, evtchn_upcall_mask) << 9;
-if ( VM_ASSIST(n->domain, architectural_iopl) )
-eflags |= n->arch.pv_vcpu.iopl;
 
 if ( !ring_1(regs) )
 {
@@ -1770,7 +1770,7 @@ static void load_segments(struct vcpu *n
 }
 
 if ( ret |
- put_user(eflags,  esp-1) |
+ put_user(rflags,  esp-1) |
  put_user(cs_and_mask, esp-2) |
  put_user(regs->_eip,  esp-3) |
  put_user(uregs->gs,   esp-4) |
@@ -1805,12 +1805,6 @@ static void load_segments(struct vcpu *n
 cs_and_mask = (unsigned long)regs->cs |
 ((unsigned long)vcpu_info(n, evtchn_upcall_mask) << 32);
 
-/* Fold upcall mask into RFLAGS.IF. */
-rflags  = regs->rflags & ~(X86_EFLAGS_IF|X86_EFLAGS_IOPL);
-rflags |= !vcpu_info(n, evtchn_upcall_mask) << 9;
-if ( VM_ASSIST(n->domain, architectural_iopl) )
-rflags |= n->arch.pv_vcpu.iopl;
-
 if ( put_user(regs->ss,rsp- 1) |
  put_user(regs->rsp,   rsp- 2) |
  put_user(rflags,  rsp- 3) |



x86: fold code in load_segments()

No need to have the same logic twice.

Signed-off-by: Jan Beulich 

--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -1745,22 +1745,22 @@ static void load_segments(struct vcpu *n
 (unsigned long *)pv->kernel_sp;
 unsigned long cs_and_mask, rflags;
 
+/* Fold upcall mask and architectural IOPL into RFLAGS.IF. */
+rflags  = regs->rflags & ~(X86_EFLAGS_IF|X86_EFLAGS_IOPL);
+rflags |= !vcpu_info(n, evtchn_upcall_mask) << 9;
+if ( VM_ASSIST(n->domain, architectural_iopl) )
+rflags |= n->arch.pv_vcpu.iopl;
+
 if ( is_pv_32bit_vcpu(n) )
 {
 unsigned int *esp = ring_1(regs) ?
 (unsigned int *)regs->rsp :
 (unsigned int *)pv->kernel_sp;
-unsigned int cs_and_mask, eflags;
 int ret = 0;
 
 /* CS longword also contains full evtchn_upcall_mask. */
 cs_and_mask = (unsigned short)regs->cs |
 ((unsigned int)vcpu_info(n, evtchn_upcall_mask) << 16);
-/* Fold upcall mask into RFLAGS.IF. */
-eflags  = regs->_eflags & ~(X86_EFLAGS_IF|X86_EFLAGS_IOPL);
-eflags |= !vcpu_info(n, evtchn_upcall_mask) << 9;
-if ( VM_ASSIST(n->domain, architectural_iopl) )
-eflags |= n->arch.pv_vcpu.iopl;
 
 if ( !ring_1(regs) )
 {
@@ -1770,7 +1770,7 @@ static void load_segments(struct vcpu *n
 }
 
 if ( ret |
- put_user(eflags,  esp-1) |
+ put_user(rflags,  esp-1) |
  put_user(cs_and_mask, esp-2) |
  put_user(regs->_eip,  esp-3) |
  put_user(uregs->gs,   esp-4) |
@@ -1805,12 +1805,6 @@ static void load_segments(struct vcpu *n
 cs_and_mask = (unsigned long)regs->cs |
 ((unsigned long)vcpu_info(n, evtchn_upcall_mask) << 32);
 
-/* Fold upcall mask into RFLAGS.IF. */
-rflags  = regs->rflags & ~(X86_EFLAGS_IF|X86_EFLAGS_IOPL);
-rflags |= !vcpu_info(n, evtchn_upcall_mask) << 9;
-if ( VM_ASSIST(n->domain, architectural_iopl) )
-rflags |= n->arch.pv_vcpu.iopl;
-
 if ( put_user(regs->ss,rsp- 1) |
  put_user(regs->rsp,   rsp- 2) |
  put_user(rflags,  rsp- 3) |

Re: [Xen-devel] [PATCH v3 00/19] Make ACPI builder available to components other than hvmloader

[Boris Ostrovsky]

On 09/07/2016 02:59 PM, Boris Ostrovsky wrote:
> The goal here is to build ACPI tables for PVHv2/HVMlite guests while reusing 
> existing
> hvmloader's ACPI builder code. The builder is provided as a library in 
> tools/libacpi.
>
> This is version 3 of the series, see individual patches for changes. It can
> be fetched from git://oss.oracle.com/git/bostrovs/xen.git:acpi_v3

Wei, Ian --- are there any comments from the toolstack perspective?

>
> The series is probably still gated by lack of an ACK from Lenovo for the
> relicensing patch (included here as patch 1).
>

So I looked again at commit 801d469ad8b2b88f669326327df03d03200efbfb
(which is the patch from Lenovo) and it only does 2 things (assuming I
parsed ASL correctly):
* Adds _PIC method
* Controls and describes legacy PCI interrupt routing. This
functionality has actually been moved into mk_dsdt.c and so this ASL
code is now generated.

Neither of these two things is used by libxl so technically we may not
need the ACK from Lenovo. OTOH, we may forget about this restriction in
the future and accidentally include those two later.


-boris


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 29/38] arm/p2m: Add HVMOP_altp2m_set_mem_access

[Sergej Proskurin]

Hi Julien,


On 09/12/2016 02:08 PM, Julien Grall wrote:
> Hi Sergej,
>
> On 16/08/16 23:17, Sergej Proskurin wrote:
>> The HVMOP_altp2m_set_mem_access allows to set gfn permissions of
>> (currently one page at a time) of a specific altp2m view. In case the
>> view does not hold the requested gfn entry, it will be first copied from
>> the host's p2m table and then modified as requested.
>>
>> Signed-off-by: Sergej Proskurin 
>> ---
>> Cc: Stefano Stabellini 
>> Cc: Julien Grall 
>> ---
>> v2: Prevent the page reference count from being falsely updated on
>> altp2m modification. Therefore, we add a check determining whether
>> the target p2m is a hostp2m before p2m_put_l3_page is called.
>>
>> v3: Cosmetic fixes.
>>
>> Added the functionality to set/get the default_access also in/from
>> the requested altp2m view.
>>
>> Read-locked hp2m in "altp2m_set_mem_access".
>>
>> Moved the functions "p2m_is_(hostp2m|altp2m)" out of this commit.
>>
>> Moved the funtion "modify_altp2m_entry" out of this commit.
>>
>> Moved the function "p2m_lookup_attr" out of this commit.
>>
>> Moved guards for "p2m_put_l3_page" out of this commit.
>> ---
>>  xen/arch/arm/altp2m.c| 53 
>>  xen/arch/arm/hvm.c   |  7 +++-
>>  xen/arch/arm/p2m.c   | 82
>> 
>>  xen/include/asm-arm/altp2m.h | 12 +++
>>  4 files changed, 131 insertions(+), 23 deletions(-)
>>
>> diff --git a/xen/arch/arm/altp2m.c b/xen/arch/arm/altp2m.c
>> index ba345b9..03b8ce5 100644
>> --- a/xen/arch/arm/altp2m.c
>> +++ b/xen/arch/arm/altp2m.c
>> @@ -80,6 +80,59 @@ int altp2m_switch_domain_altp2m_by_id(struct
>> domain *d, unsigned int idx)
>>  return rc;
>>  }
>>
>> +int altp2m_set_mem_access(struct domain *d,
>> +  struct p2m_domain *hp2m,
>> +  struct p2m_domain *ap2m,
>> +  p2m_access_t a,
>> +  gfn_t gfn)
>> +{
>> +p2m_type_t p2mt;
>> +p2m_access_t old_a;
>> +mfn_t mfn;
>> +unsigned int page_order;
>> +int rc;
>> +
>> +altp2m_lock(d);
>
> Why do you have this lock? This will serialize all the mem access
> modification even if the ap2m is different.
>
> I am also surprised that you only lock the altp2m in
> modify_altp2m_entry. IHMO, the change in the same a2pm should be
> serialized.
>

You are correct, the implementation in the upcoming patch write-locks
the altp2m view and hence also removes the need for the locking the
altp2m_lock. Thank you.

>> +p2m_read_lock(hp2m);
>> +
>> +/* Check if entry is part of the altp2m view. */
>> +mfn = p2m_lookup_attr(ap2m, gfn, , NULL, _order);
>> +
>> +/* Check host p2m if no valid entry in ap2m. */
>> +if ( mfn_eq(mfn, INVALID_MFN) )
>> +{
>> +/* Check if entry is part of the host p2m view. */
>> +mfn = p2m_lookup_attr(hp2m, gfn, , _a, _order);
>
> As mentioned in patch #27, p2m_lookup_attr will take a read reference
> on the hp2m lock. However you already did it above.
>
> Anyway, I really think that p2m_lookup_attr should not exist and ever
> caller be replaced by a direct call to p2m_get_entry.
>

Sounds good. I thought you did not want have the functions
p2m_(set|get)_entry exported, which is the reason for the wrappers. I
have already removed these wrappers in the upcoming patch. Thank you.

>
>> +if ( mfn_eq(mfn, INVALID_MFN) ||
>> + ((p2mt != p2m_ram_rw) && (p2mt != p2m_ram_ro)) )
>
> Please use p2m_is_ram rather than open-coding the check on p2mt.
>
> However, I don't understand why access restriction on altp2m is
> limited to RAM whilst the hostp2m allows restriction on any type of page.
>

I agree: It makes sense to allow setting memory permissions without such
restrictions. It it seems to be a remnant of the old implementation, in
which invalid mfn's were not guaranteed to be marked as INVALID_MFN.
Anyway, I will remove this check. Thank you.

>> +{
>> +rc = -ESRCH;
>> +goto out;
>> +}
>> +
>> +/* If this is a superpage, copy that first. */
>> +if ( page_order != THIRD_ORDER )
>> +{
>> +rc = modify_altp2m_entry(ap2m, gfn, mfn, p2mt, old_a,
>> page_order);
>> +if ( rc < 0 )
>> +{
>> +rc = -ESRCH;
>> +goto out;
>> +}
>> +}
>> +}
>> +
>> +/* Set mem access attributes - currently supporting only one
>> (4K) page. */
>> +page_order = THIRD_ORDER;
>
> This looks pointless, please use THIRD_ORDER directly as argument.
>
>> +rc = modify_altp2m_entry(ap2m, gfn, mfn, p2mt, a, page_order);
>> +
>> +out:
>> +p2m_read_unlock(hp2m);
>> +altp2m_unlock(d);
>> +
>> +return rc;
>> +}
>> +
>>  static void altp2m_vcpu_reset(struct vcpu *v)
>>  {
>>  struct altp2mvcpu *av = 

Re: [Xen-devel] [PATCH 1/2] vm_event: Sanitize vm_event response handling

[Tamas K Lengyel]

On Wed, Sep 14, 2016 at 9:15 AM, Julien Grall  wrote:
>
>
> On 14/09/16 16:14, Tamas K Lengyel wrote:
>>
>> On Wed, Sep 14, 2016 at 3:33 AM, Julien Grall 
>> wrote:
>>>
>>> Hello Tamas,
>>>
>>> On 13/09/16 19:12, Tamas K Lengyel wrote:


 diff --git a/xen/include/asm-arm/p2m.h b/xen/include/asm-arm/p2m.h
 index 53c4d78..5e9bc54 100644
 --- a/xen/include/asm-arm/p2m.h
 +++ b/xen/include/asm-arm/p2m.h
 @@ -121,10 +121,10 @@ typedef enum {
   p2m_to_mask(p2m_map_foreign)))

  static inline
 -void p2m_mem_access_emulate_check(struct vcpu *v,
 +bool_t p2m_mem_access_emulate_check(struct vcpu *v,
const vm_event_response_t *rsp)
>>>
>>>
>>>
>>> s/bool_t/bool/ and please indent properly the second line.
>>
>>
>> Fine by me but bool_t is used throughout p2m.h and I see no use of
>> just bool. Is there actually any difference between the two to warrant
>> enforcing one over the other?
>
>
> bool_t has been turned into an alias to bool recently. Moving all the source
> code from one to another is a long task (very similar to mfn/gfn typesafe).
>
> However, new code should use bool and not bool_t.

Ack.

Tamas

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/2] vm_event: Sanitize vm_event response handling

[Julien Grall]

On 14/09/16 16:14, Tamas K Lengyel wrote:

On Wed, Sep 14, 2016 at 3:33 AM, Julien Grall  wrote:

Hello Tamas,

On 13/09/16 19:12, Tamas K Lengyel wrote:


diff --git a/xen/include/asm-arm/p2m.h b/xen/include/asm-arm/p2m.h
index 53c4d78..5e9bc54 100644
--- a/xen/include/asm-arm/p2m.h
+++ b/xen/include/asm-arm/p2m.h
@@ -121,10 +121,10 @@ typedef enum {
  p2m_to_mask(p2m_map_foreign)))

 static inline
-void p2m_mem_access_emulate_check(struct vcpu *v,
+bool_t p2m_mem_access_emulate_check(struct vcpu *v,
   const vm_event_response_t *rsp)



s/bool_t/bool/ and please indent properly the second line.


Fine by me but bool_t is used throughout p2m.h and I see no use of
just bool. Is there actually any difference between the two to warrant
enforcing one over the other?


bool_t has been turned into an alias to bool recently. Moving all the 
source code from one to another is a long task (very similar to mfn/gfn 
typesafe).


However, new code should use bool and not bool_t.

Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/2] vm_event: Sanitize vm_event response handling

[Tamas K Lengyel]

On Wed, Sep 14, 2016 at 3:33 AM, Julien Grall  wrote:
> Hello Tamas,
>
> On 13/09/16 19:12, Tamas K Lengyel wrote:
>>
>> diff --git a/xen/include/asm-arm/p2m.h b/xen/include/asm-arm/p2m.h
>> index 53c4d78..5e9bc54 100644
>> --- a/xen/include/asm-arm/p2m.h
>> +++ b/xen/include/asm-arm/p2m.h
>> @@ -121,10 +121,10 @@ typedef enum {
>>   p2m_to_mask(p2m_map_foreign)))
>>
>>  static inline
>> -void p2m_mem_access_emulate_check(struct vcpu *v,
>> +bool_t p2m_mem_access_emulate_check(struct vcpu *v,
>>const vm_event_response_t *rsp)
>
>
> s/bool_t/bool/ and please indent properly the second line.

Fine by me but bool_t is used throughout p2m.h and I see no use of
just bool. Is there actually any difference between the two to warrant
enforcing one over the other?

>
> Please indent properly the second line.

Ack.

>
> Regards,
>
> --
> Julien Grall

Thanks,
Tamas

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/2] vm_event: Sanitize vm_event response handling

[Tamas K Lengyel]

On Wed, Sep 14, 2016 at 7:38 AM, George Dunlap  wrote:
> On 13/09/16 19:12, Tamas K Lengyel wrote:
>> Setting response flags in vm_event are only ever safe if the vCPUs are 
>> paused.
>> To reflect this we move all checks within the if block that already checks
>> whether this is the case. Checks that are only supported on one architecture
>> we relocate the bitmask operations to the arch-specific handlers to avoid
>> the overhead on architectures that don't support it.
>>
>> Furthermore, we clean-up the emulation checks so it more clearly represents 
>> the
>> decision-logic when emulation should take place. As part of this we also
>> set the stage to allow emulation in response to other types of events, not 
>> just
>> mem_access violations.
>>
>> Signed-off-by: Tamas K Lengyel 
>
> Tamas,
>
> Would you like a detailed review from me for this?  I'm happy to ack the
> p2m bits on the basis that they're only touching mem_access code.  A
> full review may get stuck behind a pretty long review backlog. :-(
>
>  -George

Hi George,
acking just the p2m bits should suffice!

Thanks,
Tamas

>> ---
>> Cc: George Dunlap 
>> Cc: Jan Beulich 
>> Cc: Andrew Cooper 
>> Cc: Razvan Cojocaru 
>> Cc: Stefano Stabellini 
>> Cc: Julien Grall 
>> ---
>>  xen/arch/x86/mm/p2m.c  | 81 
>> +++---
>>  xen/arch/x86/vm_event.c| 35 +-
>>  xen/common/vm_event.c  | 53 ++-
>>  xen/include/asm-arm/p2m.h  |  4 +--
>>  xen/include/asm-arm/vm_event.h |  9 -
>>  xen/include/asm-x86/p2m.h  |  4 +--
>>  xen/include/asm-x86/vm_event.h |  5 ++-
>>  xen/include/xen/mem_access.h   | 12 ---
>>  8 files changed, 113 insertions(+), 90 deletions(-)
>>
>> diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
>> index 7d14c3b..6c01868 100644
>> --- a/xen/arch/x86/mm/p2m.c
>> +++ b/xen/arch/x86/mm/p2m.c
>> @@ -1588,62 +1588,55 @@ void p2m_mem_paging_resume(struct domain *d, 
>> vm_event_response_t *rsp)
>>  }
>>  }
>>
>> -void p2m_mem_access_emulate_check(struct vcpu *v,
>> -  const vm_event_response_t *rsp)
>> +bool_t p2m_mem_access_emulate_check(struct vcpu *v,
>> +const vm_event_response_t *rsp)
>>  {
>> -/* Mark vcpu for skipping one instruction upon rescheduling. */
>> -if ( rsp->flags & VM_EVENT_FLAG_EMULATE )
>> -{
>> -xenmem_access_t access;
>> -bool_t violation = 1;
>> -const struct vm_event_mem_access *data = >u.mem_access;
>> +xenmem_access_t access;
>> +bool_t violation = 1;
>> +const struct vm_event_mem_access *data = >u.mem_access;
>>
>> -if ( p2m_get_mem_access(v->domain, _gfn(data->gfn), ) == 0 )
>> +if ( p2m_get_mem_access(v->domain, _gfn(data->gfn), ) == 0 )
>> +{
>> +switch ( access )
>>  {
>> -switch ( access )
>> -{
>> -case XENMEM_access_n:
>> -case XENMEM_access_n2rwx:
>> -default:
>> -violation = data->flags & MEM_ACCESS_RWX;
>> -break;
>> +case XENMEM_access_n:
>> +case XENMEM_access_n2rwx:
>> +default:
>> +violation = data->flags & MEM_ACCESS_RWX;
>> +break;
>>
>> -case XENMEM_access_r:
>> -violation = data->flags & MEM_ACCESS_WX;
>> -break;
>> +case XENMEM_access_r:
>> +violation = data->flags & MEM_ACCESS_WX;
>> +break;
>>
>> -case XENMEM_access_w:
>> -violation = data->flags & MEM_ACCESS_RX;
>> -break;
>> +case XENMEM_access_w:
>> +violation = data->flags & MEM_ACCESS_RX;
>> +break;
>>
>> -case XENMEM_access_x:
>> -violation = data->flags & MEM_ACCESS_RW;
>> -break;
>> +case XENMEM_access_x:
>> +violation = data->flags & MEM_ACCESS_RW;
>> +break;
>>
>> -case XENMEM_access_rx:
>> -case XENMEM_access_rx2rw:
>> -violation = data->flags & MEM_ACCESS_W;
>> -break;
>> +case XENMEM_access_rx:
>> +case XENMEM_access_rx2rw:
>> +violation = data->flags & MEM_ACCESS_W;
>> +break;
>>
>> -case XENMEM_access_wx:
>> -violation = data->flags & MEM_ACCESS_R;
>> -break;
>> +case XENMEM_access_wx:
>> +violation = data->flags & MEM_ACCESS_R;
>> +break;
>>
>> -case XENMEM_access_rw:
>> -violation = data->flags & MEM_ACCESS_X;
>> -break;
>> +case XENMEM_access_rw:
>> +violation = 

Re: [Xen-devel] [PATCH 04/17] x86emul: complete decoding of two-byte instructions

[Jan Beulich]

>>> On 14.09.16 at 16:22,  wrote:
> On 08/09/16 14:10, Jan Beulich wrote:
>> This way we can at least size (and e.g. skip) them if needed, and we
>> also won't raise the wrong fault due to not having read all relevant
>> bytes.
> 
> What faults are you referring to?  #UD vs #GP from hitting the %cs limit?

Or #PF.

>> This at once adds correct raising of #UD for the three "ud" flavors
>> (Intel names only "ud2", but AMD names all three of them in their
>> opcode maps), as that may make a difference to callers compared to
>> getting back X86EMUL_UNHANDLEABLE.
> 
> Definitely a good improvement.  I have been meaning to do this for a while.
> 
> Intel does references 0FB9 in a footnote in the opcode map, but I can't
> see any mention of 0FFF at all.

Check AMD's.

>> Note on opcodes 0FA6 and 0FA7: These are VIA's PadLock instructions,
>> which have a ModRM like byte where only register forms are valid. I.e.
>> we could also use SrcImmByte there, but ModRM is more likely to be
>> correct for a hypothetical extension allowing non-register operations.
> 
> Won't the use of ModRM possibly cause us to read too much if it end up
> with SIB and displacement encoding?  OTOH, do we really care?

That's why I've added that paragraph: I'd be fine either way, but I
do think the intention is a ModRM byte. Which is then also in line with
these opcodes' uses in early 386 and 486 processors (xbts/ibts/
cmpxchg).

>> Note on opcode 0FB8: I think we're safe to ignore JMPE (which doesn't
>> take a ModRM byte, but an immediate).
> 
> It took a while to find out what this instruction is.  Mind indicating
> that it is Itanium-specific in the commit message?

Sure.

> POPCNT, the aliased instruction takes a full ModRM byte with no space to
> distinguish.

Well, distinguishing them is possible in principle, as by the time we
process bytes past the main opcode one we already know whether
an F3 prefix was present. I simply think it's not worth trying to do
so.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 4/6] Pause/Unpause the domain before/after assigning PI hooks

[Dario Faggioli]

On Wed, 2016-09-14 at 02:23 +, Wu, Feng wrote:
> Then I tried to implement the function like the following:
> 
> /* This function is called when pcidevs_lock is held */
> void vmx_pi_hooks_assign(struct domain *d)
> {
> if ( !iommu_intpost || !has_hvm_container_domain(d) )
> return;
> 
> ASSERT(!d->arch.hvm_domain.vmx.vcpu_block);
> 
> /*
>  * We carefully handle the timing here:
>  * - Install the context switch first
>  * - Then set the NDST field
>  * - Install the block and resume hooks in the end
>  *
>  * This can make sure the PI (especially the NDST feild) is
>  * in proper state when we call vmx_vcpu_block().
>  */
> d->arch.hvm_domain.vmx.pi_switch_from = vmx_pi_switch_from;
> d->arch.hvm_domain.vmx.pi_switch_to = vmx_pi_switch_to;
> 
> for_each_vcpu ( d, v )
> {
> unsigned int dest = cpu_physical_id(v->processor);
> struct pi_desc *pi_desc = >arch.hvm_vmx.pi_desc;
> 
> /* spot 1 */
> 
> write_atomic(_desc->ndst,
>  x2apic_enabled ? dest : MASK_INSR(dest,
> PI_xAPIC_NDST_MASK));
> }
> 
> d->arch.hvm_domain.vmx.vcpu_block = vmx_vcpu_block;
> d->arch.hvm_domain.vmx.pi_do_resume = vmx_pi_do_resume;
> }
> 
> However, I still think it is problematic. Consider the _spot 1_
> above, we get
> the pCPU of the vCPU and update the NDST, but the vCPU can be happily
> rescheduled to another pCPU before updating the NDST. The problem is
> that it is not safe to update NDST here, since vCPU can be scheduled
> behind us at any time. And if this is the case, it is hard to safely
> do this
> without guarantee the vCPU is in a known state. Any further advice
> on this? Thanks a lot!
> 
So, I'm sorry if this is me missing or not remembering something... but
I do remember that, at some point, in the early days of this series,
there were concerns about the use of v->processor behind the back of
the scheduler, i.e., without holding the proper scheduler related
locks.

Pausing the domain was not even being remotely considered at the time,
it (again, at least AFAICR) came up later for trying to address other
issues.

No, the whole point of why that was not a problem in the first place is
that what counts here is on the wait list of what pcpu the vcpu is put,
not really where the vcpu is being or has been scheduled last. Of
course it'd be better --and it would also be true most of the times--
if there where a match, but that was not a correctness concern.

So why this is all of the sudden becoming one? Am I completely off with
my recollection (or in general :-P)? Or what am I missing about the
issue we are trying to address with this new bits of the work?

Thanks and Regards,
Dario
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R Ltd., Cambridge (UK)



signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Xen-users] Xen 4.7.0 graphics pass through problems (Win10, AMD Radeon)

[Jan Beulich]

>>> On 14.09.16 at 16:04,  wrote:
> On 2016-09-14 14:50, Jan Beulich wrote:
> On 14.09.16 at 13:34,  wrote:
>>> Can you think of anything that's changed between 4.6 and 4.7 that
>>> would cause the performance problems he's describing when both a video
>>> card and a usb controller are passed through, but not when only the
>>> video card is passed through?
>> Not really, no. Peter - are there any indications of problems in
>> one or more of the logs (Xen, xl, qemu)? Did you try running a
>> debug build of all Xen components?
> 
> No, I did not try a debug build. However, the system seems fairly stable 
> now, only the display driver crashing once, or twice a day. That did 
> occur with Xen 4.6.3 also, but maybe once a week. Looking at the log 
> files, I cannot see anything particular that stands out, except for the 
> domU log.
> 
> The domU log file qemu-dm-.log started to contain the following 
> lines after upgrading to 4.7:
> 
> [00:06.0] xen_pt_bar_read: Error: Should not read BAR through QEMU. 
> @0x0004
> [00:06.0] xen_pt_bar_write: Error: Should not write BAR through QEMU. 
> @0x
> 
> There are only a couple of lines with write, the rest with read. I don't 
> know what significance it's got, but there are around 350 lines every day.

Looks wrong in any case, but I have no clue whether it's indicative
of the problem you're having. In any event in the qemu tree in
hw/xen/xen_pt.h there's an "#ifdef XEN_PT_LOGGING_ENABLED"
conditional close to the top. Could you make this "#if 1" instead
(or simply #define XEN_PT_LOGGING_ENABLED in the line above)
and re-build at least qemu? The log of a subsequent guest run
should then be quite a bit more helpful in possibly spotting what's
going wrong to cause those messages.

And then it's of course rather relevant to know what device 00:06.0
actually is (iirc these are guest BDF values), considering that you're
passing through more than one.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 04/17] x86emul: complete decoding of two-byte instructions

[Andrew Cooper]

On 08/09/16 14:10, Jan Beulich wrote:
> This way we can at least size (and e.g. skip) them if needed, and we
> also won't raise the wrong fault due to not having read all relevant
> bytes.

What faults are you referring to?  #UD vs #GP from hitting the %cs limit?

>
> This at once adds correct raising of #UD for the three "ud" flavors
> (Intel names only "ud2", but AMD names all three of them in their
> opcode maps), as that may make a difference to callers compared to
> getting back X86EMUL_UNHANDLEABLE.

Definitely a good improvement.  I have been meaning to do this for a while.

Intel does references 0FB9 in a footnote in the opcode map, but I can't
see any mention of 0FFF at all.

>
> Note on opcodes 0FA6 and 0FA7: These are VIA's PadLock instructions,
> which have a ModRM like byte where only register forms are valid. I.e.
> we could also use SrcImmByte there, but ModRM is more likely to be
> correct for a hypothetical extension allowing non-register operations.

Won't the use of ModRM possibly cause us to read too much if it end up
with SIB and displacement encoding?  OTOH, do we really care?

>
> Note on opcode 0FB8: I think we're safe to ignore JMPE (which doesn't
> take a ModRM byte, but an immediate).

It took a while to find out what this instruction is.  Mind indicating
that it is Itanium-specific in the commit message?

POPCNT, the aliased instruction takes a full ModRM byte with no space to
distinguish.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [ovmf baseline-only test] 67711: all pass

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 67711 ovmf real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/67711/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf 18b00c8c8773e62b1e2c4a36f490996325570b67
baseline version:
 ovmf d947fbed72226011961e5e2691f09baebf128795

Last test of basis67709  2016-09-14 05:54:11 Z0 days
Testing same since67711  2016-09-14 12:46:15 Z0 days1 attempts


People who touched revisions under test:
  Jeff Fan 
  lushifex 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xs.citrite.net/~osstest/testlogs/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


commit 18b00c8c8773e62b1e2c4a36f490996325570b67
Author: lushifex 
Date:   Thu Sep 1 13:02:55 2016 +0800

Vlv2TbltDevicePkg:Signal End of Dxe Event.

According to PI spec,DxeSmmReadyToLock protocol is published immediately 
after signaling of the End of Dxe Event.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: lushifex 

commit 8a5b8cef678711060cc9014e0fac1886839de943
Author: Jeff Fan 
Date:   Fri Sep 9 15:14:32 2016 +0800

UefiCpuPkg/SecCore: Re-install SEC platform information(2) PPI

In SecTemporaryRamDone(), we will build one privated GUIDed-HOB to save CPU 
BIST
Data and re-install SEC platform information(2) PPI. Then other PEI drivers
could get CPU BIST data from the private GUIDed-HOB by new installed PPI.

Cc: Michael Kinney 
Cc: Feng Tian 
Cc: Giri P Mudusuru 
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jeff Fan 
Reviewed-by: Feng Tian 

commit d157de8b5603fb24feed6ec7b46be8677083c25a
Author: Jeff Fan 
Date:   Fri Sep 9 15:08:56 2016 +0800

UefiCpuPkg/SecCore: Abstract worker function GetBistFromHob()

Abstract one worker function to get CPU BIST from the GUIDed-HOB. Add
SecPlatformInformationBist() and SecPlatformInformation2Bist() to invoke
GetBistFromHob(). Add in/out for parameter in function header.

Cc: Michael Kinney 
Cc: Feng Tian 
Cc: Giri P Mudusuru 
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jeff Fan 
Reviewed-by: Feng Tian 

commit 863c738c4af0d200c8789e4786e1722e07f407fa
Author: Jeff Fan 
Date:   Fri Sep 9 15:00:30 2016 +0800

UefiCpuPkg/SecCore: Add SecBist.c

Add SecBist.c and copy GetBistInfoFromPpi() and SecPlatformInformation2() 
from
UefiCpuPkg/CpuMpPei/CpuBist.c. And update SecMain.c, SecMain.inf and
UefiCpuPkg.dsc accordinlgy to pass build.

Cc: Michael Kinney 
Cc: Feng Tian 
Cc: Giri P Mudusuru 
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jeff Fan 
Reviewed-by: Feng Tian 

commit 8643299420ed5e8305eebda669c68672266096ff
Author: Jeff Fan 
Date:   Fri Sep 9 14:36:39 2016 +0800

UefiCpuPkg/CpuMpPei: Build GUIDed-HOB to store all CPU BIST Data

Build gEfiSecPlatformInformation2PpiGuid GUIDed-HOB to store all CPU BIST 
data
that could be used not only by SecPlatformInformation2(), but also by CPU 
MP Dxe
driver to get CPU BIST data.

Cc: Michael Kinney 
Cc: Feng Tian 
Cc: Giri P Mudusuru 
Contributed-under: TianoCore 

[Xen-devel] per-domain logging

[Cedric Bosdonnat]

Hi all,

I wanted to get libvirt's libxl driver have per-domain logs like all other 
drivers. After
looking at the libxl and XenToolLogger it seems I'll need to add the feature in 
either libxl
XenToolLogger. Would anyone already have an idea how best to add API to allow 
this?

Thanks for your help
--
Cedric

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Xen-users] Xen 4.7.0 graphics pass through problems (Win10, AMD Radeon)

[Peter Milesson]

On 2016-09-14 15:23, Andrew Cooper wrote:

On 14/09/16 13:50, Jan Beulich wrote:

On 14.09.16 at 13:34,  wrote:

  On 2016-09-09 09:51, Peter Milesson wrote:

[snip]

I've been using Xen for 3 years, starting out with 4.2, and I've been
upgrading regularly, and installed 4.7.0 today.

I'm using Windows 10, 64-bit with PCI pass through in a VM with
PV-drivers, using a AMD Radeon HD6450 card (AMD graphics cards don't seem to
need graphics pass through)

Previously (up till Xen 4.6.3), the graphics output has displayed some
shorter lines, a bit like thin coarse snow, when watching videos. The
distortions stayed within the movie, and was tolerable.

After upgrade to Xen 4.7.0, the video performance is seriously ugly. Just
for example, I open cnn.com and move the mouse pointer up and down over the
photos, which creates a bunch of flickering lines over the display. The same
with moving content in Youtube for example. Or opening a Cygwin terminal
window and scrolling through a file. Terrible.

Nothing else has changed, only the Xen version.

[snip]

On Fri, Sep 9, 2016 at 10:35 AM, Peter Milesson  wrote:

Hi again,

I've been playing around a bit more.

It seems one of the problems is that I passed a couple of PCI-devices (USB
controllers) to the VM, beside the graphics card.

When I pass the USB-devices individually in the VM configuration file
(usbdevice=['tablet','host:1.4','host:1:9','host:x.y']), the display
problems seem to be more or less gone.

Jan / Andy,

Can you think of anything that's changed between 4.6 and 4.7 that
would cause the performance problems he's describing when both a video
card and a usb controller are passed through, but not when only the
video card is passed through?

Not really, no. Peter - are there any indications of problems in
one or more of the logs (Xen, xl, qemu)? Did you try running a
debug build of all Xen components?

Furthermore, does "Nothing else has changed, only the Xen version." mean
switching your distro packages between two versions of xen, or literally
only switching the hypervisor itself.

A substantial quantity of the complexity here is in qemu, rather than Xen.

~Andrew

Hi Andrew,

I previously compiled and run Xen 4.6.3 in the same environment. Without 
making any changes to anything else, then downloading, compiling and 
installing Xen 4.7.0.


But read my following 2 posts. I stopped passing the original USB 
controller (2 PCI devices), and switched to another USB controller. All 
the previous problems completely gone. Logically, it doesn't make sense, 
because passing those 2 PCI devices, or these 2, should be completely 
irrelevant.


Best regards,

Peter


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Xen-users] Xen 4.7.0 graphics pass through problems (Win10, AMD Radeon)

[George Dunlap]

On Wed, Sep 14, 2016 at 3:04 PM, Peter Milesson  wrote:
>
>
> On 2016-09-14 14:50, Jan Beulich wrote:
>
> On 14.09.16 at 13:34,  wrote:

   On 2016-09-09 09:51, Peter Milesson wrote:
>>>
>>> [snip]
>
> I've been using Xen for 3 years, starting out with 4.2, and I've been
> upgrading regularly, and installed 4.7.0 today.
>
> I'm using Windows 10, 64-bit with PCI pass through in a VM with
> PV-drivers, using a AMD Radeon HD6450 card (AMD graphics cards don't
> seem to
> need graphics pass through)
>
> Previously (up till Xen 4.6.3), the graphics output has displayed some
> shorter lines, a bit like thin coarse snow, when watching videos. The
> distortions stayed within the movie, and was tolerable.
>
> After upgrade to Xen 4.7.0, the video performance is seriously ugly.
> Just
> for example, I open cnn.com and move the mouse pointer up and down over
> the
> photos, which creates a bunch of flickering lines over the display. The
> same
> with moving content in Youtube for example. Or opening a Cygwin
> terminal
> window and scrolling through a file. Terrible.
>
> Nothing else has changed, only the Xen version.
>>>
>>> [snip]
>>>
>>> On Fri, Sep 9, 2016 at 10:35 AM, Peter Milesson  wrote:

 Hi again,

 I've been playing around a bit more.

 It seems one of the problems is that I passed a couple of PCI-devices
 (USB
 controllers) to the VM, beside the graphics card.

 When I pass the USB-devices individually in the VM configuration file
 (usbdevice=['tablet','host:1.4','host:1:9','host:x.y']), the display
 problems seem to be more or less gone.
>>>
>>> Jan / Andy,
>>>
>>> Can you think of anything that's changed between 4.6 and 4.7 that
>>> would cause the performance problems he's describing when both a video
>>> card and a usb controller are passed through, but not when only the
>>> video card is passed through?
>>
>> Not really, no. Peter - are there any indications of problems in
>> one or more of the logs (Xen, xl, qemu)? Did you try running a
>> debug build of all Xen components?
>>
>> Jan
>>
> Hi Jan,
>
> No, I did not try a debug build. However, the system seems fairly stable
> now, only the display driver crashing once, or twice a day. That did occur
> with Xen 4.6.3 also, but maybe once a week.

Well crashing twice a day isn't what most people would call "stable".
:-)  In any case, I was trying to debug the issue you had with passing
through both the video card and the usb controller -- that is
obviously an important configuration, but if there's a bug in that
configuration it probably affects a large number of wider
configurations as well.

> Looking at the log files, I
> cannot see anything particular that stands out, except for the domU log.
>
> The domU log file qemu-dm-.log started to contain the following lines
> after upgrading to 4.7:
>
> [00:06.0] xen_pt_bar_read: Error: Should not read BAR through QEMU.
> @0x0004
> [00:06.0] xen_pt_bar_write: Error: Should not write BAR through QEMU.
> @0x
>
> There are only a couple of lines with write, the rest with read. I don't
> know what significance it's got, but there are around 350 lines every day.

Yeah, those definitely look important.  This is with just the video
card passed through, or with both passed through?

 -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH V1] xen/arm: domain_build: introduce dom0_lowmem bootargs

[Edgar E. Iglesias]

On Wed, Sep 14, 2016 at 08:40:09PM +0800, Peng Fan wrote:
> On Wed, Sep 14, 2016 at 01:34:10PM +0100, Julien Grall wrote:
> >
> >
> >On 14/09/16 13:18, Peng Fan wrote:
> >>Hello Julien,
> >>
> >>On Wed, Sep 14, 2016 at 01:06:01PM +0100, Julien Grall wrote:
> >>>
> >>>
> >>>On 14/09/16 13:03, Peng Fan wrote:
> Hello Julien,
> >>>
> >>>Hello Peng,
> >>>
> On Wed, Sep 14, 2016 at 11:47:10AM +0100, Julien Grall wrote:
> >Hello,
> >
> >On 14/09/16 08:41, Peng Fan wrote:
> >>On Wed, Sep 14, 2016 at 08:23:24AM +0100, Julien Grall wrote:
> >>diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> >>index 35ab08d..cc71e6f 100644
> >>--- a/xen/arch/arm/domain_build.c
> >>+++ b/xen/arch/arm/domain_build.c
> >>@@ -28,6 +28,8 @@
> >>
> >>static unsigned int __initdata opt_dom0_max_vcpus;
> >>integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
> >>+static bool_t __initdata opt_dom0_use_lowmem;
> >>+boolean_param("dom0_use_lowmem", opt_dom0_use_lowmem);
> >>
> >>int dom0_11_mapping = 1;
> >>
> >>@@ -244,7 +246,7 @@ static void allocate_memory(struct domain *d, struct
> >>kernel_info *kinfo)
> >>   unsigned int order = get_11_allocation_size(kinfo->unassigned_mem);
> >>   int i;
> >>
> >>-bool_t lowmem = is_32bit_domain(d);
> >>+bool_t lowmem = is_32bit_domain(d) || opt_dom0_use_lowmem;
> >>   unsigned int bits;
> >>
> >>
> >>Pass "dom0_use_lowmem=1" to xen to allocate lowmem as much as possible.
> >
> >Again, what is the benefit to have a command line option for that?
> 
> Then you prefer directly change "bool_t lowmem = is_32bit_domain(d);" to 
> "bool_t lowmem = true" ?
> I just want to give user a choice.
> >>>
> >>>We don't add new command line parameter just because they look cool to 
> >>>have.
> >>>So far, you did not explain why it would be good to let the choice to the
> >>>user and how it could be used.
> >>
> >>I have not try, if there is no lowmem.
> >>
> >>I have not look into alloc_domheap_pages.
> >>I am not sure whether there is such a platform or not,
> >>just thinking if there is soc that dram memory starts from 4GB, and there 
> >>is no dram
> >>below 4GB. If we still can get memory when lowmem is true, I am ok to 
> >>change directly assign
> >>lowmem with value true. Anyway I have not look into the internals of 
> >>domheap and
> >>not sure whether there is such a platform that no lowmem (:-
> >
> >We cannot exclude this possibility. However, the only reason that Xen is
> >requiring to allocate a bank below 4GB for 32-bit domain is to handle
> >non-LPAE kernel.
> 
> Now also need to handle device that have DMA limitation -:)

Hi Peng,

Doesn't your platform have an IOMMU/SMMU?

Cheers,
Edgar

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Xen-users] Xen 4.7.0 graphics pass through problems (Win10, AMD Radeon)

[Peter Milesson]

On 2016-09-14 14:50, Jan Beulich wrote:

On 14.09.16 at 13:34,  wrote:

  On 2016-09-09 09:51, Peter Milesson wrote:

[snip]

I've been using Xen for 3 years, starting out with 4.2, and I've been
upgrading regularly, and installed 4.7.0 today.

I'm using Windows 10, 64-bit with PCI pass through in a VM with
PV-drivers, using a AMD Radeon HD6450 card (AMD graphics cards don't seem to
need graphics pass through)

Previously (up till Xen 4.6.3), the graphics output has displayed some
shorter lines, a bit like thin coarse snow, when watching videos. The
distortions stayed within the movie, and was tolerable.

After upgrade to Xen 4.7.0, the video performance is seriously ugly. Just
for example, I open cnn.com and move the mouse pointer up and down over the
photos, which creates a bunch of flickering lines over the display. The same
with moving content in Youtube for example. Or opening a Cygwin terminal
window and scrolling through a file. Terrible.

Nothing else has changed, only the Xen version.

[snip]

On Fri, Sep 9, 2016 at 10:35 AM, Peter Milesson  wrote:

Hi again,

I've been playing around a bit more.

It seems one of the problems is that I passed a couple of PCI-devices (USB
controllers) to the VM, beside the graphics card.

When I pass the USB-devices individually in the VM configuration file
(usbdevice=['tablet','host:1.4','host:1:9','host:x.y']), the display
problems seem to be more or less gone.

Jan / Andy,

Can you think of anything that's changed between 4.6 and 4.7 that
would cause the performance problems he's describing when both a video
card and a usb controller are passed through, but not when only the
video card is passed through?

Not really, no. Peter - are there any indications of problems in
one or more of the logs (Xen, xl, qemu)? Did you try running a
debug build of all Xen components?

Jan


Hi Jan,

No, I did not try a debug build. However, the system seems fairly stable 
now, only the display driver crashing once, or twice a day. That did 
occur with Xen 4.6.3 also, but maybe once a week. Looking at the log 
files, I cannot see anything particular that stands out, except for the 
domU log.


The domU log file qemu-dm-.log started to contain the following 
lines after upgrading to 4.7:


[00:06.0] xen_pt_bar_read: Error: Should not read BAR through QEMU. 
@0x0004
[00:06.0] xen_pt_bar_write: Error: Should not write BAR through QEMU. 
@0x


There are only a couple of lines with write, the rest with read. I don't 
know what significance it's got, but there are around 350 lines every day.


Thanks a lot for your interest and good work.

Regards,

Peter




___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Help] Trigger Watchdog when adding an IPI in vcpu_wake

[Wei Yang]

On Tue, Sep 13, 2016 at 01:30:17PM +0200, Dario Faggioli wrote:
>[using xendevel correct address]
>
>On Tue, 2016-09-13 at 16:54 +0800, Wei Yang wrote:
>> On Fri, 2016-09-09 at 17:41 +0800, Wei Yang wrote:
>> > 
>> > I'm not surprised by that. Yet, I'd be interested in hearing more
>> > about this profiling you have done (things like, how you captured
>> > the data, what workloads you are exactly considering, how you
>> > determined what is the bottleneck, etc).
>> Let me try to explain this.
>> 
>> Workload: a. Blue Screen in Windows Guests
>>    b. some client using Windows to do some video
>> processing
>>   which need precise timestamp (we are not sure the
>> core
>>   reason but we see the system is very slow)
>>
>Do you mind sharing just a bit more, such as:
> - number of pcpus
> - number of vcpus of the various VMs

160 pcpus
16 vcpus in VM and 8 VMs

>
>I also am not sure what "a. Blue screen in Windows guests" above
>means... is there a workload called "Blue Screen"? Or is it that you
>are hitting a BSOD in some of your guests (which ones, what were they
>doing)? Or is it that you _want_ to provoke a BSOD on some of your
>guests? Or is that something else? :-P
>

Yes, the "Blue Screen" is what we want to mimic the behavior from client.

The "Blue Screen" will force the hypervisor to do load balance in my mind.

>> Capture the data: lock profile
>> Bottleneck Check: The schedule lock wait time is really high  
>> 
>Ok, cool. Interesting that lock profiling works on 4.1! :-O
>
>> > The scheduler tries to see whether the v->processor of the waking
>> > vcpu can be re-used, but that's not at all guaranteed, and again,
>> > on a very loaded system, not even that likely!
>> > 
>> Hmm... I may missed something.
>> 
>> Took your assumption below for example. 
>> In my mind, the process looks like this:
>> 
>> csched_vcpu_wake()
>> __runq_insert(),  insert the vcpu in pcpu's 6 runq
>> __runq_tickle(),  raise SCHEDULE_SOFTIRQ on pcpu 6 or other
>> (1)
>> 
>Well, yes. More precisely, at least in current staging,
>SCHEDULE_SOFTIRQ is raised for pcpu 6:
> - if pcpu 6 is idle, or
> - if pcpu 6 is not idle but there actually isn't any idle vcpu, and
>   the waking vcpu is higher in priority than what is running on pcpu 6
>
>> __do_softirq()
>> schedule()
>> csched_schedule(),  for pcpu 6, it may wake up d2v2 based
>> on it's
>> priority
>>
>Yes, but it is pcpu 6 that will run csched_schedule() only if the
>conditions mentioned above are met. If not, it will be some other pcpu
>that will do so (or none!).
>
>But I just realized that the fact that you are actually working on 4.1
>is going to be an issue. In fact, the code that it is now in Xen has
>changed **a lot** since 4.1. In fact, you're missing all the soft-
>affinity work (which you may or may not find useful) but also
>improvements and bugfixing.
>

That's true...

>I'll have a quick look at how __runq_tickle() looks like in Xen 4.1,
>but there's very few chances I'll be available to provide detailed
>review, advice, testing, etc., on such an old codebase. :-(
>
>> By looking at the code, what I missed may be in __runq_tickle(),
>> which in case
>> there are idle pcpu, schedule_softirq is also raised on them. By
>> doing so,
>> those idle pcpus would steal other busy tasks and may in chance get
>> d2v2?
>> 
>Yes, it looks like, in Xen 4.1, this is more or less what happens. The
>idea is to always tickle pcpu 6, if the priority of the waking vcpu is
>higher than what is running there. 
>

Hmm... in case there are idle pcpus and the priority of the waking vcpu is
higher than what is running on pcpu 6, would pcpu 6 have more chance to run it?
or other idle pcup would steal it from pcpu6? or they have equal chance?

>If pcpu 6 was not idle, we also tickle one or more idle pcpus so that:
> - if the waking vcpu preempted what was running on pcpu 6, an idler
>   can pick-up ("steal") such preempted vcpu and run it;
> - if the waking vcpu ended up in the runqueue, an idler can steal it
>

Hmm... I don't get the difference between these two cases.

Looks both are an idler steal the vcpu.

>> BTW, if the system is in heavy load, how much chance would we have
>> idle pcpu?
>> 
>It's not very likely that there will be idle pcpus in a very busy
>system, I agree.
>
>> > > We can divide this in three steps:
>> > > a) Send IPI to the target CPU and tell it which vcpu we want it
>> > > to 
>> > > wake up.
>> > > b) The interrupt handler on target cpu insert vcpu to a percpu
>> > > queue 
>> > > and raise
>> > >    softirq.
>> > > c) softirq will dequeue the vcpu and wake it up.
>> > > 
>> > I'm not sure I see how you think this would improve the situation.
>> > 
>> > So, quickly, right now:
>> > 
>> > - let's say vcpu 2 of domain 2 (from now d2v2) is waking up
>> > - let's assume d2v2->processor = 6
>> > - let's assume the wakeup is 

Re: [Xen-devel] [PATCH 03/24] xen: credit1: return the 'time remaining to the limit' as next timeslice.

[Dario Faggioli]

On Wed, 2016-09-14 at 10:34 +0100, George Dunlap wrote:
> On 12/09/16 18:00, Dario Faggioli wrote:
> > 
> > I also agree on the fact that most of the times ratelimit_us and
> > MIN_TIMER will be close enough (like in the example above) that it
> > won't probably matter much... but if someone set ratelimit_us to
> > something higher (say, 10ms --we accept values as big as the
> > timeslice,
> > which is 30ms b default) it may matter a bit.
> > 
> > What do you think?
> > 
> > If we decide not to care, and leave things as they are, I'd add a
> > comment saying that code is like that on purpose, so we won't trip
> > over
> > this again in 1 or 2 years. :-)
> Yeah, I think while we're thinking about it we might as well add in
> the
> MIN_TIMER thing you mention above (if you don't mind doing it).
> 
Sure I don't mind. I'll do it.

Thanks and Regards,
Dario
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R Ltd., Cambridge (UK)



signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [GIT PULL] xen: regression fix for 4.8-rc6

[David Vrabel]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Linus,

Please git pull the following tag:

 git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git
for-linus-4.8b-rc6-tag

xen: regression fix for 4.8-rc6

- - Fix SMP boot in arm guests.

Thanks.

David

 arch/arm/xen/enlighten.c | 7 +++
 1 file changed, 3 insertions(+), 4 deletions(-)

Vitaly Kuznetsov (1):
  arm/xen: fix SMP guests boot
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQEcBAEBCAAGBQJX2VWXAAoJEFxbo/MsZsTRNQkIAIxyFCgLWpNHjOc5xfqHIFXD
rQgVixD+7CzgcQ6uC4JliGykLdWwqhy+2lx/69OJtEQ/49SQqainUeMf+GVZSwjg
jBT2v7jfdCPWu2qcTTG2xWYtNDjP4Wb0A+tMpw0pDZj33eYcoQuCl0lxLjjgOUb8
dUIHo47F2ptxMeugg2HetwhkPOqP9+NfGb3DaoruQy5RfYqul1VniRlfDTurLfIY
nVi0jnhLGrHIDFgtxIZaSHDGl+g5piOlPTXT3UEaTKcsxUJNmWdgGfnFJit7T50h
GkZ8riN/pHrSnT3lEcbia4TsiTIL9MF6Bi9u4yL+yCkINatmL63qDwMxMeFHzxs=
=6N4A
-END PGP SIGNATURE-

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH Altp2m cleanup v4 2/4] altp2m cleanup work

[George Dunlap]

On 13/09/16 18:38, Lai, Paul wrote:
> On Mon, Sep 12, 2016 at 11:47:35AM +0100, George Dunlap wrote:
>> On 08/09/16 17:45, Lai, Paul C wrote:
>>> [Paul2] in-line
>>
>> If you're going to engage in discussions on xen-devel it would really be
>> worth your time to find a mail setup that allows you to actually quote
>> properly such that you can reply in-line without these manual mark-ups.
>>
>> If you can't configure your mail reader to do proper nested quoting, and
>> you can't / don't want to change your mail reader, then you could
>> consider doing what I do and signing up for a gmail account to get all
>> the xen-devel mail and replying from there.
>>
>> Thanks,
>>  -George
>>
> 
> George,
> Thanks for the suggestions.

No problem -- and I hope the tone of my message didn't come off too
wrong.  I was trying to find solutions that work for everyone, not
criticize. :-)

 -George


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/2] vm_event: Sanitize vm_event response handling

[George Dunlap]

On 13/09/16 19:12, Tamas K Lengyel wrote:
> Setting response flags in vm_event are only ever safe if the vCPUs are paused.
> To reflect this we move all checks within the if block that already checks
> whether this is the case. Checks that are only supported on one architecture
> we relocate the bitmask operations to the arch-specific handlers to avoid
> the overhead on architectures that don't support it.
> 
> Furthermore, we clean-up the emulation checks so it more clearly represents 
> the
> decision-logic when emulation should take place. As part of this we also
> set the stage to allow emulation in response to other types of events, not 
> just
> mem_access violations.
> 
> Signed-off-by: Tamas K Lengyel 

Tamas,

Would you like a detailed review from me for this?  I'm happy to ack the
p2m bits on the basis that they're only touching mem_access code.  A
full review may get stuck behind a pretty long review backlog. :-(

 -George

> ---
> Cc: George Dunlap 
> Cc: Jan Beulich 
> Cc: Andrew Cooper 
> Cc: Razvan Cojocaru 
> Cc: Stefano Stabellini 
> Cc: Julien Grall 
> ---
>  xen/arch/x86/mm/p2m.c  | 81 
> +++---
>  xen/arch/x86/vm_event.c| 35 +-
>  xen/common/vm_event.c  | 53 ++-
>  xen/include/asm-arm/p2m.h  |  4 +--
>  xen/include/asm-arm/vm_event.h |  9 -
>  xen/include/asm-x86/p2m.h  |  4 +--
>  xen/include/asm-x86/vm_event.h |  5 ++-
>  xen/include/xen/mem_access.h   | 12 ---
>  8 files changed, 113 insertions(+), 90 deletions(-)
> 
> diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
> index 7d14c3b..6c01868 100644
> --- a/xen/arch/x86/mm/p2m.c
> +++ b/xen/arch/x86/mm/p2m.c
> @@ -1588,62 +1588,55 @@ void p2m_mem_paging_resume(struct domain *d, 
> vm_event_response_t *rsp)
>  }
>  }
>  
> -void p2m_mem_access_emulate_check(struct vcpu *v,
> -  const vm_event_response_t *rsp)
> +bool_t p2m_mem_access_emulate_check(struct vcpu *v,
> +const vm_event_response_t *rsp)
>  {
> -/* Mark vcpu for skipping one instruction upon rescheduling. */
> -if ( rsp->flags & VM_EVENT_FLAG_EMULATE )
> -{
> -xenmem_access_t access;
> -bool_t violation = 1;
> -const struct vm_event_mem_access *data = >u.mem_access;
> +xenmem_access_t access;
> +bool_t violation = 1;
> +const struct vm_event_mem_access *data = >u.mem_access;
>  
> -if ( p2m_get_mem_access(v->domain, _gfn(data->gfn), ) == 0 )
> +if ( p2m_get_mem_access(v->domain, _gfn(data->gfn), ) == 0 )
> +{
> +switch ( access )
>  {
> -switch ( access )
> -{
> -case XENMEM_access_n:
> -case XENMEM_access_n2rwx:
> -default:
> -violation = data->flags & MEM_ACCESS_RWX;
> -break;
> +case XENMEM_access_n:
> +case XENMEM_access_n2rwx:
> +default:
> +violation = data->flags & MEM_ACCESS_RWX;
> +break;
>  
> -case XENMEM_access_r:
> -violation = data->flags & MEM_ACCESS_WX;
> -break;
> +case XENMEM_access_r:
> +violation = data->flags & MEM_ACCESS_WX;
> +break;
>  
> -case XENMEM_access_w:
> -violation = data->flags & MEM_ACCESS_RX;
> -break;
> +case XENMEM_access_w:
> +violation = data->flags & MEM_ACCESS_RX;
> +break;
>  
> -case XENMEM_access_x:
> -violation = data->flags & MEM_ACCESS_RW;
> -break;
> +case XENMEM_access_x:
> +violation = data->flags & MEM_ACCESS_RW;
> +break;
>  
> -case XENMEM_access_rx:
> -case XENMEM_access_rx2rw:
> -violation = data->flags & MEM_ACCESS_W;
> -break;
> +case XENMEM_access_rx:
> +case XENMEM_access_rx2rw:
> +violation = data->flags & MEM_ACCESS_W;
> +break;
>  
> -case XENMEM_access_wx:
> -violation = data->flags & MEM_ACCESS_R;
> -break;
> +case XENMEM_access_wx:
> +violation = data->flags & MEM_ACCESS_R;
> +break;
>  
> -case XENMEM_access_rw:
> -violation = data->flags & MEM_ACCESS_X;
> -break;
> +case XENMEM_access_rw:
> +violation = data->flags & MEM_ACCESS_X;
> +break;
>  
> -case XENMEM_access_rwx:
> -violation = 0;
> -break;
> -}
> +case XENMEM_access_rwx:
> +violation = 0;
> +break;
>  }
> 

Re: [Xen-devel] [Xen-users] Xen 4.7.0 graphics pass through problems (Win10, AMD Radeon)

[George Dunlap]

On 14/09/16 14:23, Andrew Cooper wrote:
> On 14/09/16 13:50, Jan Beulich wrote:
> On 14.09.16 at 13:34,  wrote:
  On 2016-09-09 09:51, Peter Milesson wrote:
>>> [snip]
> I've been using Xen for 3 years, starting out with 4.2, and I've been
> upgrading regularly, and installed 4.7.0 today.
>
> I'm using Windows 10, 64-bit with PCI pass through in a VM with
> PV-drivers, using a AMD Radeon HD6450 card (AMD graphics cards don't seem 
> to
> need graphics pass through)
>
> Previously (up till Xen 4.6.3), the graphics output has displayed some
> shorter lines, a bit like thin coarse snow, when watching videos. The
> distortions stayed within the movie, and was tolerable.
>
> After upgrade to Xen 4.7.0, the video performance is seriously ugly. Just
> for example, I open cnn.com and move the mouse pointer up and down over 
> the
> photos, which creates a bunch of flickering lines over the display. The 
> same
> with moving content in Youtube for example. Or opening a Cygwin terminal
> window and scrolling through a file. Terrible.
>
> Nothing else has changed, only the Xen version.
>>> [snip]
>>>
>>> On Fri, Sep 9, 2016 at 10:35 AM, Peter Milesson  wrote:
 Hi again,

 I've been playing around a bit more.

 It seems one of the problems is that I passed a couple of PCI-devices (USB
 controllers) to the VM, beside the graphics card.

 When I pass the USB-devices individually in the VM configuration file
 (usbdevice=['tablet','host:1.4','host:1:9','host:x.y']), the display
 problems seem to be more or less gone.
>>> Jan / Andy,
>>>
>>> Can you think of anything that's changed between 4.6 and 4.7 that
>>> would cause the performance problems he's describing when both a video
>>> card and a usb controller are passed through, but not when only the
>>> video card is passed through?
>> Not really, no. Peter - are there any indications of problems in
>> one or more of the logs (Xen, xl, qemu)? Did you try running a
>> debug build of all Xen components?
> 
> Furthermore, does "Nothing else has changed, only the Xen version." mean
> switching your distro packages between two versions of xen, or literally
> only switching the hypervisor itself.

I assume he's using "Xen" to mean the whole Xen system -- hypervisor +
tools + qemu.

> A substantial quantity of the complexity here is in qemu, rather than Xen.

Rather than the hypervisor, you mean.

Another thing worth asking, Peter: Have you tried running with
qemu-traditional (by adding the line below to your config file) rather
than qemu-upstream, to see if that makes a difference?

device_model_version="qemu-xen-traditional"

qemu-traditional almost ever changes, so if it worked better, then that
would point the finger at changes in qemu (at which point we would have
to bring in a different set of people to help diagnose it).

 -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Xen-users] Xen 4.7.0 graphics pass through problems (Win10, AMD Radeon)

[Andrew Cooper]

On 14/09/16 13:50, Jan Beulich wrote:
 On 14.09.16 at 13:34,  wrote:
>>>  On 2016-09-09 09:51, Peter Milesson wrote:
>> [snip]
 I've been using Xen for 3 years, starting out with 4.2, and I've been
 upgrading regularly, and installed 4.7.0 today.

 I'm using Windows 10, 64-bit with PCI pass through in a VM with
 PV-drivers, using a AMD Radeon HD6450 card (AMD graphics cards don't seem 
 to
 need graphics pass through)

 Previously (up till Xen 4.6.3), the graphics output has displayed some
 shorter lines, a bit like thin coarse snow, when watching videos. The
 distortions stayed within the movie, and was tolerable.

 After upgrade to Xen 4.7.0, the video performance is seriously ugly. Just
 for example, I open cnn.com and move the mouse pointer up and down over the
 photos, which creates a bunch of flickering lines over the display. The 
 same
 with moving content in Youtube for example. Or opening a Cygwin terminal
 window and scrolling through a file. Terrible.

 Nothing else has changed, only the Xen version.
>> [snip]
>>
>> On Fri, Sep 9, 2016 at 10:35 AM, Peter Milesson  wrote:
>>> Hi again,
>>>
>>> I've been playing around a bit more.
>>>
>>> It seems one of the problems is that I passed a couple of PCI-devices (USB
>>> controllers) to the VM, beside the graphics card.
>>>
>>> When I pass the USB-devices individually in the VM configuration file
>>> (usbdevice=['tablet','host:1.4','host:1:9','host:x.y']), the display
>>> problems seem to be more or less gone.
>> Jan / Andy,
>>
>> Can you think of anything that's changed between 4.6 and 4.7 that
>> would cause the performance problems he's describing when both a video
>> card and a usb controller are passed through, but not when only the
>> video card is passed through?
> Not really, no. Peter - are there any indications of problems in
> one or more of the logs (Xen, xl, qemu)? Did you try running a
> debug build of all Xen components?

Furthermore, does "Nothing else has changed, only the Xen version." mean
switching your distro packages between two versions of xen, or literally
only switching the hypervisor itself.

A substantial quantity of the complexity here is in qemu, rather than Xen.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-unstable test] 100944: regressions - FAIL

[osstest service owner]

flight 100944 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/100944/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemut-debianhvm-amd64 15 guest-localmigrate/x10 fail REGR. 
vs. 100933

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop fail like 100933
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop fail like 100933
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stopfail like 100933
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stopfail like 100933
 test-amd64-amd64-xl-rtds  9 debian-install   fail  like 100933

Tests which did not succeed, but are not blocking:
 test-amd64-i386-rumprun-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-rumprun-amd64  1 build-check(1)   blocked  n/a
 build-amd64-rumprun   5 rumprun-buildfail   never pass
 build-i386-rumprun5 rumprun-buildfail   never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass

version targeted for testing:
 xen  773522000cc17f6f4323a4d97423790138ea98f2
baseline version:
 xen  d45fae589b8d8b6d36c211dcc46d767dda730b61

Last test of basis   100933  2016-09-13 13:03:38 Z0 days
Testing same since   100944  2016-09-14 02:56:07 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64-xtf  pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt

Re: [Xen-devel] [PATCH] x86: add a user configurable Kconfig option for the VGA

[Julien Grall]

Hello Derek,

On 13/09/16 20:40, Derek Straka wrote:

diff --git a/xen/drivers/video/Kconfig b/xen/drivers/video/Kconfig
index 0ffbbd9..0f208fe 100644
--- a/xen/drivers/video/Kconfig
+++ b/xen/drivers/video/Kconfig
@@ -3,7 +3,8 @@ config VIDEO
bool

 config VGA
-   bool
+   bool "VGA"
+   default y if X86


Same remark as for the NS16550 change. This option cannot be enabled on 
ARM and will break compilation of randconfig.


Regards,


select VIDEO

 config HAS_ARM_HDLCD


Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Xen-users] Xen 4.7.0 graphics pass through problems (Win10, AMD Radeon)

[Jan Beulich]

>>> On 14.09.16 at 13:34,  wrote:
>>  On 2016-09-09 09:51, Peter Milesson wrote:
> [snip]
>>> I've been using Xen for 3 years, starting out with 4.2, and I've been
>>> upgrading regularly, and installed 4.7.0 today.
>>>
>>> I'm using Windows 10, 64-bit with PCI pass through in a VM with
>>> PV-drivers, using a AMD Radeon HD6450 card (AMD graphics cards don't seem to
>>> need graphics pass through)
>>>
>>> Previously (up till Xen 4.6.3), the graphics output has displayed some
>>> shorter lines, a bit like thin coarse snow, when watching videos. The
>>> distortions stayed within the movie, and was tolerable.
>>>
>>> After upgrade to Xen 4.7.0, the video performance is seriously ugly. Just
>>> for example, I open cnn.com and move the mouse pointer up and down over the
>>> photos, which creates a bunch of flickering lines over the display. The same
>>> with moving content in Youtube for example. Or opening a Cygwin terminal
>>> window and scrolling through a file. Terrible.
>>>
>>> Nothing else has changed, only the Xen version.
> 
> [snip]
> 
> On Fri, Sep 9, 2016 at 10:35 AM, Peter Milesson  wrote:
>> Hi again,
>>
>> I've been playing around a bit more.
>>
>> It seems one of the problems is that I passed a couple of PCI-devices (USB
>> controllers) to the VM, beside the graphics card.
>>
>> When I pass the USB-devices individually in the VM configuration file
>> (usbdevice=['tablet','host:1.4','host:1:9','host:x.y']), the display
>> problems seem to be more or less gone.
> 
> Jan / Andy,
> 
> Can you think of anything that's changed between 4.6 and 4.7 that
> would cause the performance problems he's describing when both a video
> card and a usb controller are passed through, but not when only the
> video card is passed through?

Not really, no. Peter - are there any indications of problems in
one or more of the logs (Xen, xl, qemu)? Did you try running a
debug build of all Xen components?

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 12/19] libacpi: Build DSDT for PVH guests

[Boris Ostrovsky]

On 09/14/2016 12:13 AM, Shannon Zhao wrote:
>
> On 2016/9/8 2:59, Boris Ostrovsky wrote:
>> PVH guests require DSDT with only ACPI INFO (Xen-specific) and Processor
>> objects. We separate ASL's ACPI INFO definition into dsdt_acpi_info.asl so
>> that it can be included in ASLs for both HVM and PVH2.
>>
>> Signed-off-by: Boris Ostrovsky 
>> ---
>> Changes in v3:
>> * Added comment to dsdt_acpi_info.asl indicating that the structure
>>   there must match struct acpi_info
>> * Use QEMU_NONE in mk_dsdt.c
>> * Makefile tweaks
>>
>>  tools/libacpi/Makefile   | 13 ++---
>>  tools/libacpi/dsdt.asl   | 20 
>>  tools/libacpi/dsdt_acpi_info.asl | 26 ++
>>  tools/libacpi/mk_dsdt.c  |  8 
>>  4 files changed, 44 insertions(+), 23 deletions(-)
>>  create mode 100644 tools/libacpi/dsdt_acpi_info.asl
>>
>> diff --git a/tools/libacpi/Makefile b/tools/libacpi/Makefile
>> index 6325cd0..12b081e 100644
>> --- a/tools/libacpi/Makefile
>> +++ b/tools/libacpi/Makefile
>> @@ -18,7 +18,7 @@ include $(XEN_ROOT)/tools/firmware/Rules.mk
>>  MK_DSDT = $(ACPI_BUILD_DIR)/mk_dsdt
>>  
>>  # Sources to be generated
>> -C_SRC = $(addprefix $(ACPI_BUILD_DIR)/, dsdt_anycpu.c dsdt_15cpu.c  
>> dsdt_anycpu_qemu_xen.c)
>> +C_SRC = $(addprefix $(ACPI_BUILD_DIR)/, dsdt_anycpu.c dsdt_15cpu.c  
>> dsdt_anycpu_qemu_xen.c dsdt_pvh.c)
>>  H_SRC = $(addprefix $(ACPI_BUILD_DIR)/, ssdt_s3.h ssdt_s4.h ssdt_pm.h 
>> ssdt_tpm.h)
>>  
> Hi Bros,
>
> It looks like you forgot to add 'C_SRC-$(CONFIG_X86)' here. Will you add
> it in your next version?

I know I said that I'd do it but then I realized that it would probably
make more sense to add this macro when a non-x86 target is first
introduced (which I believe is patch 3 in your series). It's a one-line
change IIRC so it shouldn't be too disruptive to your series.

I should have mentioned it when I posted v3 of my patches, I apologize
for forgetting it.

-boris



___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH V1] xen/arm: domain_build: introduce dom0_lowmem bootargs

[Peng Fan]

On Wed, Sep 14, 2016 at 01:34:10PM +0100, Julien Grall wrote:
>
>
>On 14/09/16 13:18, Peng Fan wrote:
>>Hello Julien,
>>
>>On Wed, Sep 14, 2016 at 01:06:01PM +0100, Julien Grall wrote:
>>>
>>>
>>>On 14/09/16 13:03, Peng Fan wrote:
Hello Julien,
>>>
>>>Hello Peng,
>>>
On Wed, Sep 14, 2016 at 11:47:10AM +0100, Julien Grall wrote:
>Hello,
>
>On 14/09/16 08:41, Peng Fan wrote:
>>On Wed, Sep 14, 2016 at 08:23:24AM +0100, Julien Grall wrote:
>>diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>>index 35ab08d..cc71e6f 100644
>>--- a/xen/arch/arm/domain_build.c
>>+++ b/xen/arch/arm/domain_build.c
>>@@ -28,6 +28,8 @@
>>
>>static unsigned int __initdata opt_dom0_max_vcpus;
>>integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
>>+static bool_t __initdata opt_dom0_use_lowmem;
>>+boolean_param("dom0_use_lowmem", opt_dom0_use_lowmem);
>>
>>int dom0_11_mapping = 1;
>>
>>@@ -244,7 +246,7 @@ static void allocate_memory(struct domain *d, struct
>>kernel_info *kinfo)
>>   unsigned int order = get_11_allocation_size(kinfo->unassigned_mem);
>>   int i;
>>  
>>-bool_t lowmem = is_32bit_domain(d);
>>+bool_t lowmem = is_32bit_domain(d) || opt_dom0_use_lowmem;
>>   unsigned int bits;
>>
>>
>>Pass "dom0_use_lowmem=1" to xen to allocate lowmem as much as possible.
>
>Again, what is the benefit to have a command line option for that?

Then you prefer directly change "bool_t lowmem = is_32bit_domain(d);" to 
"bool_t lowmem = true" ?
I just want to give user a choice.
>>>
>>>We don't add new command line parameter just because they look cool to have.
>>>So far, you did not explain why it would be good to let the choice to the
>>>user and how it could be used.
>>
>>I have not try, if there is no lowmem.
>>
>>I have not look into alloc_domheap_pages.
>>I am not sure whether there is such a platform or not,
>>just thinking if there is soc that dram memory starts from 4GB, and there is 
>>no dram
>>below 4GB. If we still can get memory when lowmem is true, I am ok to change 
>>directly assign
>>lowmem with value true. Anyway I have not look into the internals of domheap 
>>and
>>not sure whether there is such a platform that no lowmem (:-
>
>We cannot exclude this possibility. However, the only reason that Xen is
>requiring to allocate a bank below 4GB for 32-bit domain is to handle
>non-LPAE kernel.

Now also need to handle device that have DMA limitation -:)

>
>I personally don't think this is a hard requirement and instead of panicking
>we should print a warning to say "no bank has been allocated below 4GB" or "a
>bank of N MB has been allocated below 4GB".
>
>So my suggestion is to allocate as much as possible lowmem (i.e below 4GB)
>and if it does not work print a warning then allocate the first bank above
>4GB.
>
>I much prefer to let Xen decide itself what to do when possible rather than
>asking the user to specify himself whether lowmem is required.
>
>This is making life easier for distribution to support multiple platforms
>with Xen without having to modify the command line.

Thanks. I'll follow you suggestion and do some test. Then send out a
new patch.

Thanks,
Peng.

>
>Regards,
>
>-- 
>Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH V1] xen/arm: domain_build: introduce dom0_lowmem bootargs

[Julien Grall]

On 14/09/16 13:18, Peng Fan wrote:

Hello Julien,

On Wed, Sep 14, 2016 at 01:06:01PM +0100, Julien Grall wrote:



On 14/09/16 13:03, Peng Fan wrote:

Hello Julien,


Hello Peng,


On Wed, Sep 14, 2016 at 11:47:10AM +0100, Julien Grall wrote:

Hello,

On 14/09/16 08:41, Peng Fan wrote:

On Wed, Sep 14, 2016 at 08:23:24AM +0100, Julien Grall wrote:
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 35ab08d..cc71e6f 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -28,6 +28,8 @@

static unsigned int __initdata opt_dom0_max_vcpus;
integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
+static bool_t __initdata opt_dom0_use_lowmem;
+boolean_param("dom0_use_lowmem", opt_dom0_use_lowmem);

int dom0_11_mapping = 1;

@@ -244,7 +246,7 @@ static void allocate_memory(struct domain *d, struct
kernel_info *kinfo)
   unsigned int order = get_11_allocation_size(kinfo->unassigned_mem);
   int i;

-bool_t lowmem = is_32bit_domain(d);
+bool_t lowmem = is_32bit_domain(d) || opt_dom0_use_lowmem;
   unsigned int bits;


Pass "dom0_use_lowmem=1" to xen to allocate lowmem as much as possible.


Again, what is the benefit to have a command line option for that?


Then you prefer directly change "bool_t lowmem = is_32bit_domain(d);" to "bool_t 
lowmem = true" ?
I just want to give user a choice.


We don't add new command line parameter just because they look cool to have.
So far, you did not explain why it would be good to let the choice to the
user and how it could be used.


I have not try, if there is no lowmem.

I have not look into alloc_domheap_pages.
I am not sure whether there is such a platform or not,
just thinking if there is soc that dram memory starts from 4GB, and there is no 
dram
below 4GB. If we still can get memory when lowmem is true, I am ok to change 
directly assign
lowmem with value true. Anyway I have not look into the internals of domheap and
not sure whether there is such a platform that no lowmem (:-


We cannot exclude this possibility. However, the only reason that Xen is 
requiring to allocate a bank below 4GB for 32-bit domain is to handle 
non-LPAE kernel.


I personally don't think this is a hard requirement and instead of 
panicking we should print a warning to say "no bank has been allocated 
below 4GB" or "a bank of N MB has been allocated below 4GB".


So my suggestion is to allocate as much as possible lowmem (i.e below 
4GB) and if it does not work print a warning then allocate the first 
bank above 4GB.


I much prefer to let Xen decide itself what to do when possible rather 
than asking the user to specify himself whether lowmem is required.


This is making life easier for distribution to support multiple 
platforms with Xen without having to modify the command line.


Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [ovmf test] 100947: all pass - PUSHED

[osstest service owner]

flight 100947 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/100947/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf 18b00c8c8773e62b1e2c4a36f490996325570b67
baseline version:
 ovmf d947fbed72226011961e5e2691f09baebf128795

Last test of basis   100940  2016-09-13 23:19:04 Z0 days
Testing same since   100947  2016-09-14 05:41:50 Z0 days1 attempts


People who touched revisions under test:
  Jeff Fan 
  lushifex 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=ovmf
+ revision=18b00c8c8773e62b1e2c4a36f490996325570b67
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push ovmf 
18b00c8c8773e62b1e2c4a36f490996325570b67
+ branch=ovmf
+ revision=18b00c8c8773e62b1e2c4a36f490996325570b67
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=ovmf
+ xenbranch=xen-unstable
+ '[' xovmf = xlinux ']'
+ linuxbranch=
+ '[' x = x ']'
+ qemuubranch=qemu-upstream-unstable
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable
+ prevxenbranch=xen-4.7-testing
+ '[' x18b00c8c8773e62b1e2c4a36f490996325570b67 = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/xtf.git
++ : osst...@xenbits.xen.org:/home/xen/git/xtf.git
++ : git://xenbits.xen.org/xtf.git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/rumprun.git
++ : git://git.seabios.org/seabios.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/seabios.git
++ : git://xenbits.xen.org/osstest/seabios.git
++ : https://github.com/tianocore/edk2.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/linux-firmware.git
++ : osst...@xenbits.xen.org:/home/osstest/ext/linux-firmware.git
++ : git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git

Re: [Xen-devel] [PATCH V1] xen/arm: domain_build: introduce dom0_lowmem bootargs

[Peng Fan]

Hello Julien,

On Wed, Sep 14, 2016 at 01:06:01PM +0100, Julien Grall wrote:
>
>
>On 14/09/16 13:03, Peng Fan wrote:
>>Hello Julien,
>
>Hello Peng,
>
>>On Wed, Sep 14, 2016 at 11:47:10AM +0100, Julien Grall wrote:
>>>Hello,
>>>
>>>On 14/09/16 08:41, Peng Fan wrote:
On Wed, Sep 14, 2016 at 08:23:24AM +0100, Julien Grall wrote:
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 35ab08d..cc71e6f 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -28,6 +28,8 @@

static unsigned int __initdata opt_dom0_max_vcpus;
integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
+static bool_t __initdata opt_dom0_use_lowmem;
+boolean_param("dom0_use_lowmem", opt_dom0_use_lowmem);

int dom0_11_mapping = 1;

@@ -244,7 +246,7 @@ static void allocate_memory(struct domain *d, struct
kernel_info *kinfo)
unsigned int order = get_11_allocation_size(kinfo->unassigned_mem);
int i;

-bool_t lowmem = is_32bit_domain(d);
+bool_t lowmem = is_32bit_domain(d) || opt_dom0_use_lowmem;
unsigned int bits;


Pass "dom0_use_lowmem=1" to xen to allocate lowmem as much as possible.
>>>
>>>Again, what is the benefit to have a command line option for that?
>>
>>Then you prefer directly change "bool_t lowmem = is_32bit_domain(d);" to 
>>"bool_t lowmem = true" ?
>>I just want to give user a choice.
>
>We don't add new command line parameter just because they look cool to have.
>So far, you did not explain why it would be good to let the choice to the
>user and how it could be used.

I have not try, if there is no lowmem.

I have not look into alloc_domheap_pages.
I am not sure whether there is such a platform or not,
just thinking if there is soc that dram memory starts from 4GB, and there is no 
dram
below 4GB. If we still can get memory when lowmem is true, I am ok to change 
directly assign
lowmem with value true. Anyway I have not look into the internals of domheap and
not sure whether there is such a platform that no lowmem (:-

while ( order >= min_low_order )
{
for ( bits = order ; bits <= (lowmem ? 32 : PADDR_BITS); bits++ )
{
pg = alloc_domheap_pages(d, order, MEMF_bits(bits));
if ( pg != NULL )
goto got_bank0;
}
order--;
}

panic("Unable to allocate first memory bank");

Thanks,
Peng.
>
>Regards,
>
>-- 
>Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [libvirt test] 100945: tolerable FAIL - PUSHED

[osstest service owner]

flight 100945 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/100945/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass

version targeted for testing:
 libvirt  d53fa838e18d8c192296487ff40c3a37420b6d06
baseline version:
 libvirt  2692304c94e631720e8e9444e34a3e445e8da61a

Last test of basis   100866  2016-09-10 04:25:40 Z4 days
Failing since100912  2016-09-13 04:22:08 Z1 days2 attempts
Testing same since   100945  2016-09-14 04:22:08 Z0 days1 attempts


People who touched revisions under test:
  Daniel P. Berrange 
  Erik Skultety 
  Jiri Denemark 
  Joao Martins 
  John Ferlan 
  Laine Stump 
  Martin Kletzander 
  Michal Privoznik 
  Nikolay Shirokovskiy 
  Peter Krempa 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmpass
 test-amd64-amd64-libvirt-xsm pass
 test-armhf-armhf-libvirt-xsm fail
 test-amd64-i386-libvirt-xsm  pass
 test-amd64-amd64-libvirt pass
 test-armhf-armhf-libvirt fail
 test-amd64-i386-libvirt  pass
 test-amd64-amd64-libvirt-pairpass
 test-amd64-i386-libvirt-pair pass
 test-armhf-armhf-libvirt-qcow2   fail
 test-armhf-armhf-libvirt-raw fail
 test-amd64-amd64-libvirt-vhd pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=libvirt
+ revision=d53fa838e18d8c192296487ff40c3a37420b6d06
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local 

Re: [Xen-devel] [PATCH V1] xen/arm: domain_build: introduce dom0_lowmem bootargs

[Julien Grall]

On 14/09/16 13:03, Peng Fan wrote:

Hello Julien,


Hello Peng,


On Wed, Sep 14, 2016 at 11:47:10AM +0100, Julien Grall wrote:

Hello,

On 14/09/16 08:41, Peng Fan wrote:

On Wed, Sep 14, 2016 at 08:23:24AM +0100, Julien Grall wrote:
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 35ab08d..cc71e6f 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -28,6 +28,8 @@

static unsigned int __initdata opt_dom0_max_vcpus;
integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
+static bool_t __initdata opt_dom0_use_lowmem;
+boolean_param("dom0_use_lowmem", opt_dom0_use_lowmem);

int dom0_11_mapping = 1;

@@ -244,7 +246,7 @@ static void allocate_memory(struct domain *d, struct
kernel_info *kinfo)
unsigned int order = get_11_allocation_size(kinfo->unassigned_mem);
int i;

-bool_t lowmem = is_32bit_domain(d);
+bool_t lowmem = is_32bit_domain(d) || opt_dom0_use_lowmem;
unsigned int bits;


Pass "dom0_use_lowmem=1" to xen to allocate lowmem as much as possible.


Again, what is the benefit to have a command line option for that?


Then you prefer directly change "bool_t lowmem = is_32bit_domain(d);" to "bool_t 
lowmem = true" ?
I just want to give user a choice.


We don't add new command line parameter just because they look cool to 
have. So far, you did not explain why it would be good to let the choice 
to the user and how it could be used.


Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH V1] xen/arm: domain_build: introduce dom0_lowmem bootargs

[Peng Fan]

Hello Julien,
On Wed, Sep 14, 2016 at 11:47:10AM +0100, Julien Grall wrote:
>Hello,
>
>On 14/09/16 08:41, Peng Fan wrote:
>>On Wed, Sep 14, 2016 at 08:23:24AM +0100, Julien Grall wrote:
>>diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>>index 35ab08d..cc71e6f 100644
>>--- a/xen/arch/arm/domain_build.c
>>+++ b/xen/arch/arm/domain_build.c
>>@@ -28,6 +28,8 @@
>>
>>static unsigned int __initdata opt_dom0_max_vcpus;
>>integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
>>+static bool_t __initdata opt_dom0_use_lowmem;
>>+boolean_param("dom0_use_lowmem", opt_dom0_use_lowmem);
>>
>>int dom0_11_mapping = 1;
>>
>>@@ -244,7 +246,7 @@ static void allocate_memory(struct domain *d, struct
>>kernel_info *kinfo)
>> unsigned int order = get_11_allocation_size(kinfo->unassigned_mem);
>> int i;
>>  
>>-bool_t lowmem = is_32bit_domain(d);
>>+bool_t lowmem = is_32bit_domain(d) || opt_dom0_use_lowmem;
>> unsigned int bits;
>>
>>
>>Pass "dom0_use_lowmem=1" to xen to allocate lowmem as much as possible.
>
>Again, what is the benefit to have a command line option for that?

Then you prefer directly change "bool_t lowmem = is_32bit_domain(d);" to 
"bool_t lowmem = true" ?
I just want to give user a choice.

Thanks,
Peng.

>
>Regards,
>
>-- 
>Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Xen-users] Xen 4.7.0 graphics pass through problems (Win10, AMD Radeon)

[George Dunlap]

> On 2016-09-09 09:51, Peter Milesson wrote:
[snip]
>> I've been using Xen for 3 years, starting out with 4.2, and I've been
>> upgrading regularly, and installed 4.7.0 today.
>>
>> I'm using Windows 10, 64-bit with PCI pass through in a VM with
>> PV-drivers, using a AMD Radeon HD6450 card (AMD graphics cards don't seem to
>> need graphics pass through)
>>
>> Previously (up till Xen 4.6.3), the graphics output has displayed some
>> shorter lines, a bit like thin coarse snow, when watching videos. The
>> distortions stayed within the movie, and was tolerable.
>>
>> After upgrade to Xen 4.7.0, the video performance is seriously ugly. Just
>> for example, I open cnn.com and move the mouse pointer up and down over the
>> photos, which creates a bunch of flickering lines over the display. The same
>> with moving content in Youtube for example. Or opening a Cygwin terminal
>> window and scrolling through a file. Terrible.
>>
>> Nothing else has changed, only the Xen version.

[snip]

On Fri, Sep 9, 2016 at 10:35 AM, Peter Milesson  wrote:
> Hi again,
>
> I've been playing around a bit more.
>
> It seems one of the problems is that I passed a couple of PCI-devices (USB
> controllers) to the VM, beside the graphics card.
>
> When I pass the USB-devices individually in the VM configuration file
> (usbdevice=['tablet','host:1.4','host:1:9','host:x.y']), the display
> problems seem to be more or less gone.

Jan / Andy,

Can you think of anything that's changed between 4.6 and 4.7 that
would cause the performance problems he's describing when both a video
card and a usb controller are passed through, but not when only the
video card is passed through?

Details on hardware / config / guest OS below.

 -George

>> *Hardware*
>> CPU: AMD FX-6100 (6-core)
>> Chipset: AMD 970A (has got a working IOMMU)
>> RAM: 16GB DDR3-1600MHz
>> Video: AMD Radeon HD6450
>>
>> *OS*
>> Linux Slackware 14.2 (latest)
>> Kernel: 4.4.19 long term stable
>>
>> *Xen*
>> Version: 4.7.0 (configure same as 4.6.3 and previous versions)
>>  ./configure --prefix=/usr --localstatedir=/var --sysconfdir=/etc
>> --libdir=/usr/lib64 --with-xenstored=oxenstored --disable-qemu-traditional
>> --disable-rombios --disable-stubdom
>> Xen dom0 boot entry (grub2):
>> menuentry "Xen 4.7.0 / Slackware 14.2 (kernel 4.4.19-dom0)" {
>>set root=(hd0,1)
>>multiboot /boot/xen-4.7.0.gz iommu=1 dom0_mem=2048M
>> dom0_max_vcpus=2 dom0_vcpus_pin console=vga cpufreq=xen:performance noreboot
>>module /boot/vmlinuz-4.4.19-dom0 console=vga earlyprintk=xen
>> root=/dev/sda1 ro clocksource=xen xencons=off
>> }
>>
>> *Guest OS*
>> Windows 10 Pro 64-bit (2VCPUs)
>> Latest PV Drivers (8.2 series)
>> Graphics driver: Latest WHQL driver 15.201.1151.1008

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Xen-users] pv-grub in xen 4.7, ok with kernel 3.x, but fails with 4.x

[Wei Liu]

On Wed, Sep 14, 2016 at 12:13:00PM +0100, George Dunlap wrote:
> On Sat, Sep 10, 2016 at 9:04 PM, Sarah Newman  wrote:
> > On 09/10/2016 07:58 AM, Mark Pryor wrote:
> >> Hello,
> >>
> >> In a xen 4.7 source build, pv-grub works with domU kernel 3.x.x, but not 
> >> with 4.x.x. So, the recent fc24, xenial, and debian sid/stretch can't use
> >> pv-grub.
> >
> > While this should be fixed, in the meantime can you use an older pv-grub 
> > build? I don't know of a reason why you have to use the version of pv-grub
> > that was built along with the rest of xen.
> 
> Wei, doesn't osstest test pv-grub? Do we not have any tests of the
> in-tree pv-grub booting on new versions of Linux?
> 

First question, yes. Second question, no, we don't boot newer versions
of Linux.

Wei.

>  -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 33/38] arm/p2m: Add altp2m paging mechanism

[Julien Grall]

On 14/09/16 08:53, Sergej Proskurin wrote:

Hi Julien


[...]


 static void do_trap_instr_abort_guest(struct cpu_user_regs *regs,
   const union hsr hsr)
 {
@@ -2445,6 +2465,14 @@ static void do_trap_instr_abort_guest(struct
cpu_user_regs *regs,
 break;
 case FSC_FLT_TRANS:
 /*
+ * The guest shall retry accessing the page if the altp2m
handler
+ * succeeds. Otherwise, we continue injecting an instruction
abort
+ * exception.
+ */
+if ( try_handle_altp2m(current, gpa, npfec) )
+return;

I would have expected that altp2m is the last thing we want to check
in the abort handler. I.e after p2m_lookup.


Alright, I will reorder both calls, thank you.


Actually, reordering of these calls (try_handle_altp2m/altp2m_lazy_copy
and p2m_lookup) would make the system stall if altp2m is active. This is
because the p2m_lookup routine considers only the host's p2m, which will
most likely return a mfn != INVALID_MFN and thus entirely skip the call
to altp2m_lazy_copy. Thus, both calls should rather be conditional as
shown in the following:
if ( likely(!altp2m_active(current->domain)) )
{
/*
 * The PT walk may have failed because someone was playing
 * with the Stage-2 page table. Walk the Stage-2 PT to check
 * if the entry exists. If it's the case, return to the guest
 */
mfn = p2m_lookup(current->domain, _gfn(paddr_to_pfn(gpa)), NULL);
if ( !mfn_eq(mfn, INVALID_MFN) )
return;
}
else
/*
 * The guest shall retry accessing the page if the altp2m handler
 * succeeds. Otherwise, we continue injecting an instruction abort
 * exception.
 */
if ( altp2m_lazy_copy(current, _gfn(paddr_to_pfn(gpa))) )
return;


I think this solution is racy because altp2m_active could be changed by 
another physical CPU whilst we are in the abort handler.


So the best here is to keep the previous solution:

if ( altp2m_lazy_copy(current, _gfn(paddr_to_pfn(gpa))) )
  return;

if ( p2m_lookup(current->domain, _gfn(paddr_to_pfn(gpa)), NULL )
  return;

Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Xen-users] pv-grub in xen 4.7, ok with kernel 3.x, but fails with 4.x

[George Dunlap]

On Sat, Sep 10, 2016 at 9:04 PM, Sarah Newman  wrote:
> On 09/10/2016 07:58 AM, Mark Pryor wrote:
>> Hello,
>>
>> In a xen 4.7 source build, pv-grub works with domU kernel 3.x.x, but not 
>> with 4.x.x. So, the recent fc24, xenial, and debian sid/stretch can't use
>> pv-grub.
>
> While this should be fixed, in the meantime can you use an older pv-grub 
> build? I don't know of a reason why you have to use the version of pv-grub
> that was built along with the rest of xen.

Wei, doesn't osstest test pv-grub? Do we not have any tests of the
in-tree pv-grub booting on new versions of Linux?

 -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 16/38] arm/p2m: Add HVMOP_altp2m_set_domain_state

[Julien Grall]

Hello Sergej,

On 16/08/16 23:16, Sergej Proskurin wrote:

diff --git a/xen/arch/arm/hvm.c b/xen/arch/arm/hvm.c
index 180154e..c69da36 100644
--- a/xen/arch/arm/hvm.c
+++ b/xen/arch/arm/hvm.c
@@ -83,8 +83,40 @@ static int do_altp2m_op(XEN_GUEST_HANDLE_PARAM(void) arg)
 break;

 case HVMOP_altp2m_set_domain_state:
-rc = -EOPNOTSUPP;
+{
+struct vcpu *v;
+bool_t ostate, nstate;
+
+ostate = d->arch.altp2m_active;
+nstate = !!a.u.domain_state.state;
+
+/* If the alternate p2m state has changed, handle appropriately */
+if ( (nstate != ostate) &&
+ (ostate || !(rc = altp2m_init_by_id(d, 0))) )
+{
+for_each_vcpu( d, v )
+{
+if ( !ostate )
+{
+altp2m_vcpu_initialise(v);
+d->arch.altp2m_active = nstate;


Why do you set d->arch.altp2m_active for each vCPU?


+}
+else
+{
+d->arch.altp2m_active = nstate;
+altp2m_vcpu_destroy(v);
+}
+}
+
+/*
+ * The altp2m_active state has been deactivated. It is now safe to
+ * flush all altp2m views -- including altp2m[0].
+ */
+if ( ostate )
+altp2m_flush(d);
+}
 break;
+}

 case HVMOP_altp2m_vcpu_enable_notify:
 rc = -EOPNOTSUPP;


Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-unstable-coverity test] 100952: all pass - PUSHED

[osstest service owner]

flight 100952 xen-unstable-coverity real [real]
http://logs.test-lab.xenproject.org/osstest/logs/100952/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 xen  773522000cc17f6f4323a4d97423790138ea98f2
baseline version:
 xen  a3fe74e4345e66ddb7aa514395260a5e5f8b0cdc

Last test of basis   100886  2016-09-11 09:20:57 Z3 days
Testing same since   100952  2016-09-14 09:18:52 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Jan Beulich 
  Juergen Gross 
  Tim Deegan 
  Wei Liu 

jobs:
 coverity-amd64   pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=xen-unstable-coverity
+ revision=773522000cc17f6f4323a4d97423790138ea98f2
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push 
xen-unstable-coverity 773522000cc17f6f4323a4d97423790138ea98f2
+ branch=xen-unstable-coverity
+ revision=773522000cc17f6f4323a4d97423790138ea98f2
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=xen
+ xenbranch=xen-unstable-coverity
+ qemuubranch=qemu-upstream-unstable-coverity
+ qemuubranch=qemu-upstream-unstable
+ '[' xxen = xlinux ']'
+ linuxbranch=
+ '[' xqemu-upstream-unstable = x ']'
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable-coverity
+ prevxenbranch=xen-4.7-testing
+ '[' x773522000cc17f6f4323a4d97423790138ea98f2 = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/xtf.git
++ : osst...@xenbits.xen.org:/home/xen/git/xtf.git
++ : git://xenbits.xen.org/xtf.git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/rumprun.git
++ : git://git.seabios.org/seabios.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/seabios.git
++ : git://xenbits.xen.org/osstest/seabios.git
++ : https://github.com/tianocore/edk2.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/linux-firmware.git
++ : osst...@xenbits.xen.org:/home/osstest/ext/linux-firmware.git
++ : git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
++ : osst...@xenbits.xen.org:/home/xen/git/linux-pvops.git
++ : git://xenbits.xen.org/linux-pvops.git
++ : tested/linux-3.14
++ : tested/linux-arm-xen
++ '[' xgit://xenbits.xen.org/linux-pvops.git = x ']'
++ '[' x = x ']'
++ : git://xenbits.xen.org/linux-pvops.git
++ : tested/linux-arm-xen
++ : git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git
++ : tested/2.6.39.x
++ : 

Re: [Xen-devel] [PATCH v3 19/38] arm/p2m: Add HVMOP_altp2m_switch_p2m

[Julien Grall]

On 13/09/16 14:00, Sergej Proskurin wrote:

Hi Julien,


Hello Sergej,



On 09/12/2016 10:47 AM, Julien Grall wrote:

Hello Sergej,

On 16/08/2016 23:16, Sergej Proskurin wrote:

Signed-off-by: Sergej Proskurin 
---
Cc: Stefano Stabellini 
Cc: Julien Grall 
---
v3: Extended the function "altp2m_switch_domain_altp2m_by_id" so that if
the guest domain indirectly calles this function, the current
vcpu also
changes the altp2m view without performing an explicit context
switch.

Exchanged the check "altp2m_vttbr[idx] == INVALID_VTTBR" for
"altp2m_p2m[idx] == NULL" in "altp2m_switch_domain_altp2m_by_id".
---
 xen/arch/arm/altp2m.c| 48

 xen/arch/arm/hvm.c   |  2 +-
 xen/include/asm-arm/altp2m.h |  4 
 3 files changed, 53 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/altp2m.c b/xen/arch/arm/altp2m.c
index c14ab0b..ba345b9 100644
--- a/xen/arch/arm/altp2m.c
+++ b/xen/arch/arm/altp2m.c
@@ -32,6 +32,54 @@ struct p2m_domain *altp2m_get_altp2m(struct vcpu *v)
 return v->domain->arch.altp2m_p2m[index];
 }

+int altp2m_switch_domain_altp2m_by_id(struct domain *d, unsigned int
idx)
+{
+struct vcpu *v;
+int rc = -EINVAL;
+
+if ( idx >= MAX_ALTP2M )
+return rc;
+
+domain_pause_except_self(d);
+
+altp2m_lock(d);
+
+if ( d->arch.altp2m_p2m[idx] != NULL )
+{
+for_each_vcpu( d, v )
+if ( idx != altp2m_vcpu(v).p2midx )


Could you invert the condition to avoid another layer of nested if?



Did you mean checking for (idx == altp2m_vcpu(v).p2midx) and continue
the loop if the condition should be satisfied? If so, then sure thing :)


Correct.




+{
+atomic_dec(_get_altp2m(v)->active_vcpus);
+altp2m_vcpu(v).p2midx = idx;
+atomic_inc(_get_altp2m(v)->active_vcpus);
+
+/*
+ * In case it is the guest domain, which indirectly
called this
+ * function, the current vcpu will not switch its
context
+ * within the function "p2m_restore_state". That is,
changing
+ * the altp2m_vcpu(v).p2midx will not lead to the
requested
+ * altp2m switch on that specific vcpu. To achieve
the desired
+ * behavior, we write to VTTBR_EL2 directly.
+ */
+if ( v->domain == d && v == current )


v == current is enough.



Ok.


+{
+struct p2m_domain *ap2m = d->arch.altp2m_p2m[idx];
+
+WRITE_SYSREG64(ap2m->vttbr, VTTBR_EL2);
+isb();


I don't like the open-coding of VTTBR_EL2. I would much prefer a
separate helper to update it.



Sure, I can introduce an UPDATE_VTTBR macro (including the isb call) in
p2m.h and adjust the remaining writes to VTTBR_EL2 in p2m.c as well. I
hope, I understood you correctly.


Actually, I was concerned about having the following sequence spread:
   VTTBR_EL2
   isb

However, the isb is not necessary here because VTTBR_EL2 will not be 
used until we return to the guest.


So I would be fine to keep WRITE_SYSREG64(ap2m->vttbr, VTTBR_EL2);

Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Getting started with Xen Project

[Anthony PERARD]

On Mon, Sep 12, 2016 at 02:30:39PM +0200, Roger Pau Monné wrote:
> On Sat, Sep 10, 2016 at 03:51:33PM +0530, Aashaka Shah wrote:
> > Hello! I am Aashaka. I like computer architecture and have a good knowledge
> > of C. While browsing Outreachy projects, I came across the project about
> > adding PVH support to OVMF binaries. Since it looked interesting, I started
> > reading the mailing-list link mentioned in the project description and also
> > the Beginners section in navigation by audience column.
> > 
> > Currently, I have very little, in fact, no, experience developing with the
> > Xen Project. I would like to know how to continue with getting familiar
> > with the terms, setting up the codebase and making contributions.
> 
> Hello,
> 
> Anthony (who I've added to the Cc) already started looking into this, so I 
> think you should better coordinate with him regarding this project.

Hello, yes, I'm actually already working on it. Sorry.

There is part of this project where we would need to enable network
support in OVMF via the Xen PV interface. The challenge here would be to
work without interrupt, the only interrupt available in OVMF is a timer.
Then, one can use the implementation of netfront from FreeBSD.

> Please note that this project will involve quite a lot of low-level coding 
> in OVMF, and some knowledge of x86 assembly is required, again Anthony will 
> be able to provide more insight, and whether this is still a suitable 
> project for OPW or not.

-- 
Anthony PERARD

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 14/38] arm/p2m: Add altp2m init/teardown routines

[Julien Grall]

On 14/09/16 07:28, Sergej Proskurin wrote:

Hi Julien,



 struct arch_vcpu
diff --git a/xen/include/asm-arm/p2m.h b/xen/include/asm-arm/p2m.h
index 1a004ed..de0c90a 100644
--- a/xen/include/asm-arm/p2m.h
+++ b/xen/include/asm-arm/p2m.h
@@ -9,6 +9,8 @@
 #include 
 #include 

+#define MAX_ALTP2M 10   /* ARM might contain an arbitrary
number of
+   altp2m views. */

This should belong to altp2m.h and not p2m.h


Unfortunately, this won't work. We cannot move this define into altp2m.h
as it would result in an header conflict: The header asm/altp2m.h
includes xen/domain.h, which in turn includes asm/domain.h. So by
including asm/altp2m.h in asm/domain.h (as MAX_ALTP2M is used there), we
would create a loop of header dependencies.


Sorry, I forgot one header: altp2m.h includes xen/sched.h, which in turn
includes xen/domain.h and so on.


Fair enough, let's keep like that for now.

Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] x86: add a user configurable Kconfig option for the VGA

[Jan Beulich]

>>> On 13.09.16 at 21:40,  wrote:
> Allows for the conditional inclusion of VGA driver on the x86 platform
> rather than having it always enabled.

So I guess with all three of these patches an overview mail is missing.
What are you trying to accomplish? Solely reducing the binary size of
Xen doesn't seem like a very important goal to me, and eliminating
these drivers from the build doesn't appear to help make Xen more
stable of secure.

> @@ -672,6 +675,7 @@ void __init noreturn __start_xen(unsigned long mbi_p)
>  
>  printk("Command line: %s\n", cmdline);
>  
> +#ifdef CONFIG_VGA
>  printk("Video information:\n");

Some of the other conditionals you add may be affected too, but
here it is most prominent at the first glance - considering that we also
have CONFIG_VIDEO, wouldn't it rather be that one to be used in a
place like this one?

> --- a/xen/include/xen/console.h
> +++ b/xen/include/xen/console.h
> @@ -19,7 +19,15 @@ void console_init_postirq(void);
>  void console_endboot(void);
>  int console_has(const char *device);
>  
> +#ifdef CONFIG_VGA
>  int fill_console_start_info(struct dom0_vga_console_info *);
> +#else
> +#include 
> +static inline int fill_console_start_info(struct dom0_vga_console_info *ci) 
> {
> +(void) memset(ci, 0, sizeof(*ci));

What is this cast to void goo for?

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH V1] xen/arm: domain_build: introduce dom0_lowmem bootargs

[Julien Grall]

Hello,

On 14/09/16 08:41, Peng Fan wrote:

On Wed, Sep 14, 2016 at 08:23:24AM +0100, Julien Grall wrote:
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 35ab08d..cc71e6f 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -28,6 +28,8 @@

static unsigned int __initdata opt_dom0_max_vcpus;
integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
+static bool_t __initdata opt_dom0_use_lowmem;
+boolean_param("dom0_use_lowmem", opt_dom0_use_lowmem);

int dom0_11_mapping = 1;

@@ -244,7 +246,7 @@ static void allocate_memory(struct domain *d, struct
kernel_info *kinfo)
 unsigned int order = get_11_allocation_size(kinfo->unassigned_mem);
 int i;

-bool_t lowmem = is_32bit_domain(d);
+bool_t lowmem = is_32bit_domain(d) || opt_dom0_use_lowmem;
 unsigned int bits;


Pass "dom0_use_lowmem=1" to xen to allocate lowmem as much as possible.


Again, what is the benefit to have a command line option for that?

Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 2/2] x86: add a user configurable Kconfig option for the EHCI UART

[Jan Beulich]

>>> On 13.09.16 at 19:35,  wrote:
> --- a/xen/drivers/char/Kconfig
> +++ b/xen/drivers/char/Kconfig
> @@ -45,7 +45,8 @@ config HAS_SCIF
> say Y.
>  
>  config HAS_EHCI
> - bool
> + bool "EHCI UART" if EXPERT = "y"

Please make the prompt say e.g "EHCI debug port" - UART is not a
term commonly used with this hardware. And the variable renaming
comment from patch 1 applies here too, of course.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/2] x86: add a user configurable Kconfig option for the NS16550 UART

[Jan Beulich]

>>> On 13.09.16 at 19:35,  wrote:
> --- a/xen/drivers/char/Kconfig
> +++ b/xen/drivers/char/Kconfig
> @@ -1,6 +1,6 @@
>  config HAS_NS16550
> - bool
> - default y
> + bool "NS16550 UART" if EXPERT = "y"

This need more adjustment: HAS_* options should never have
prompts. I.e. you need to rename the config variable at once.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Unable to build with gcc 6 because of etherboot

[Wei Liu]

On Thu, Sep 08, 2016 at 10:38:01AM -0400, Daniel E. Shub wrote:
> This is a follow on to a message I sent to xen-users:
> https://lists.xen.org/archives/html/xen-devel/2015-08/msg01924.html
> 
> I am trying to compile Xen 4.7.0 with gcc 6.1.1, but I get an error
> related to etherboot. It was suggested to update the etherboot
> Makefile to the head of the etherboot repository. Another possibility
> would be to just pull in the gcc 6 patches from upstream (e.g.,
> https://git.ipxe.org/ipxe.git?a=search=refs%2Fheads%2Fmaster=commit=gcc+6).
> 
> For 4.6.0 and gcc 5 the recommendation was to just pull in the patches
> (cf. https://lists.xen.org/archives/html/xen-devel/2015-08/msg01924.html)
> but to update the etherboot version at some point. Is the recommended
> course still to just patch as needed or has there been work to update
> the etherboot version?
> 

To fix your issue immediately, pulling in a patch is ok. You are welcome
to submit a patch against xen-unstable for that if you have time.

Because ipxe doesn't do regular release so I don't really know what to
pull in. 

Long term it might be useful for user to just supply a ipxe binary when
building xen.

Wei.

> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel