[Xen-devel] [xen-unstable-smoke test] 133244: trouble: blocked/broken/pass

2019-02-13 Thread osstest service owner 
flight 133244 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133244/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken
 build-arm64-xsm   4 host-install(4)broken REGR. vs. 133005

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  f178a00c30173c0b268d99160e19ad299b1823a2
baseline version:
 xen  455301716e1ff358cb79367213003fba771dd466

Last test of basis   133005  2019-02-07 14:00:32 Z6 days
Failing since133011  2019-02-07 18:00:36 Z6 days   38 attempts
Testing same since   133200  2019-02-12 16:00:56 Z1 days   15 attempts


People who touched revisions under test:
  Andrew Cooper 
  George Dunlap 
  Ian Jackson 
  Jan Beulich 
  Juergen Gross 
  Kevin Tian 
  Michael Tautschnig 
  Norbert Manthey 
  Norbert Manthey 
  Stefano Stabellini 
  Stefano Stabellini 
  Tim Deegan 
  Wei Liu 

jobs:
 build-arm64-xsm  broken  
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64-xsm broken
broken-step build-arm64-xsm host-install(4)

Not pushing.


commit f178a00c30173c0b268d99160e19ad299b1823a2
Author: Norbert Manthey 
Date:   Tue Feb 12 15:20:15 2019 +0100

x86/hvm: block speculative out-of-bound accesses

There are multiple arrays in the HVM interface that are accessed
with indices that are provided by the guest. To avoid speculative
out-of-bound accesses, we use the array_index_nospec macro.

When blocking speculative out-of-bound accesses, we can classify arrays
into dynamic arrays and static arrays. Where the former are allocated
during run time, the size of the latter is known during compile time.
On static arrays, compiler might be able to block speculative accesses
in the future.

This is part of the speculative hardening effort.

Reported-by: Pawel Wieczorkiewicz 
Signed-off-by: Norbert Manthey 
Reviewed-by: Jan Beulich 
Release-acked-by: Juergen Gross 

commit 56d8d0119d270f846c6c4943712b8a21fbe5d4d0
Author: Jan Beulich 
Date:   Tue Feb 12 11:54:57 2019 +0100

VMX: don't ignore P2M setup error

set_mmio_p2m_entry() may fail, in particular with -ENOMEM. Don't ignore
such an error, but instead cause domain creation to fail in such a case.

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 
Acked-by: Kevin Tian 
Release-acked-by: Juergen Gross 

commit 88b92c3820cffed4b4abeb139edc2cbd8286cb12
Author: Juergen Gross 
Date:   Tue Feb 12 11:54:07 2019 +0100

iommu: fix iommu_ops initialization

Commit 32a5ea00ec75ef53e ("IOMMU/x86: remove indirection from certain
IOMMU hook accesses") introduced iommu_ops initialized at boot time
with data declared as __initconstrel.

On Intel systems there is another path where iommu_ops is initialized
and this path is relevant on resume after returning from system suspend.
As the initialization data is no longer accessible in this case that
second initialization must be dropped in case the system isn't just
booting.

Signed-off-by: Juergen Gross 
Reviewed-by: Jan Beulich 
Acked-by: Kevin Tian 
Release-acked-by: Juergen Gross 

commit 09fc4de4a8ebb389641b8b8a632efcb7ca880e08
Author: Norbert Manthey 
Date:   Wed Feb 6 15:09:33 2019 +0100

asm:

[Xen-devel] [xen-4.10-testing test] 133201: trouble: blocked/broken/fail/pass

2019-02-13 Thread osstest service owner 
flight 133201 xen-4.10-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133201/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken
 build-arm64-pvopsbroken
 build-arm64  broken

Regressions which are regarded as allowable (not blocking):
 build-arm64   2 hosts-allocate broken REGR. vs. 132966
 build-arm64-pvops 2 hosts-allocate broken REGR. vs. 132966
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 132966

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked  n/a
 build-arm64-pvops 3 capture-logs  broken blocked in 132966
 build-arm64   3 capture-logs  broken blocked in 132966
 build-arm64-xsm   3 capture-logs  broken blocked in 132966
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop fail like 132966
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail never pass
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass

version targeted for testing:
 xen  a016b8f207c7a3fe8bdd2b6f7c080020e3e1c823
baseline version:
 xen

Re: [Xen-devel] Upstream Dom0 DRM problems regarding swiotlb

2019-02-13 Thread Juergen Gross 
On 14/02/2019 01:11, Andrew Cooper wrote:
> On 13/02/2019 21:08, Michael Labriola wrote:
>> On Wed, Feb 13, 2019 at 3:21 PM Andrew Cooper  
>> wrote:
>>> On 13/02/2019 20:15, Michael Labriola wrote:
 On Wed, Feb 13, 2019 at 2:16 PM Konrad Rzeszutek Wilk
  wrote:
> On Wed, Feb 13, 2019 at 01:38:21PM -0500, Michael Labriola wrote:
>> On Wed, Feb 13, 2019 at 1:16 PM Michael Labriola
>>  wrote:
>>> On Wed, Feb 13, 2019 at 11:57 AM Konrad Rzeszutek Wilk
>>>  wrote:
 On Wed, Feb 13, 2019 at 09:09:32AM -0700, Jan Beulich wrote:
 On 13.02.19 at 17:00,  wrote:
>> On Wed, Feb 13, 2019 at 9:28 AM Jan Beulich  
>> wrote:
>> On 13.02.19 at 15:10,  wrote:
 Ah, so this isn't necessarily Xen-specific but rather any 
 paravirtual
 guest?  That hadn't crossed my mind.  Is there an easy way to find 
 out
 if we're a pv guest in the need_swiotlb conditionals?
>>> There's xen_pv_domain(), but I think xen_swiotlb would be more to
>>> the point if the check is already to be Xen-specific. There's no 
>>> generic
>>> "is PV" predicate that I'm aware of.
>> Well, that makes doing conditional code right more difficult.  I
>> assume since there isn't a generic predicate, and PV isn't new, that
>> it's absence is by design?  To reign in the temptation to sprinkle
>> conditional code all over the kernel?  ;-)
> Well, with lguest gone, Xen is the only PV environment the kernel
> can run in, afaik at least. I guess to decide between the suggested
> options or the need for some abstracting macro (or yet something
> else), you'll really need to ask the driver maintainers. Or simply
> send a patch their way implementing one of them, and see what
> their reaction is.
 Could you try this out and see if it works and I will send it out:

>> *snip*
>>> Yes, that works for me.  However, I feel like the conditional should
>>> be in drm_get_max_iomem() instead of directly after it everywhere it's
>>> used...  and is just checking xen_pv_domain() enough?  Jan made it
>>> sound like there were possibly other PV cases that would also still
>>> need swiotlb.
>> How about this?  It strcmp's pv_info to see if we're bare metal, does
>> the comparison in a single place, moves the bit shifting comparison
>> into the function (simplifying the drm driver code), and renames the
>> function to more aptly describe what's going on.
>  That looks much better.
 Great!  Now the only question left is:  KVM, VMware, Xen PVH, Xen HVM,
 and Xen PV all populate pv_info.  Do any of those other than Xen PV
 *really* need swiotlb.  That's slightly over my head.  As written, my
 patch would require swiotlb for all of them because I was attempting
 to not be Xen-specific.
>>> Its far more complicated that "Xen PV requires swiotlb".
>>>
>>> I presume the underlying problem here is DRM being special and not
>>> DMA-mapping its buffers, and trying to DMA to a buffer crossing a 4k
>>> boundary?
>> Well, I don't 100% understand how all these things work...  but here's
>> what I do know.  There are a series of commits in v4.17 that try to
>> optimize the radeon and amdgpu drivers by skipping calls to
>> ttm_dma_populate() and ttm_dma_unpopulate() unless they're "really
>> needed".  The original commit determines if swiotlb is needed by
>> checking to see if the max io mapping address of system memory is over
>> the video card's accessing range.  I can no longer log into Gnome on a
>> Xen dom0 after upgrading my kernel to v4.20 because the code that's no
>> longer happening was actually needed in a paravirtualized environment.
> 
> But from the log you provided, your bug was space exhaustion in the
> swiotlb, no?
> 
>> So, I'm trying to get all my details straight so I can submit a patch
>> to fix it w/out saying anything factually incorrect.
> 
> The thing which is different between Xen PV guests and most others (all
> others(?), now that Lguest and UML have been dropped) is that what Linux
> thinks of as PFN $N isn't necessarily adjacent to PFN $N+1 in system
> physical address space.
> 
> Therefore, code which has a buffer spanning a page boundary can't just
> convert a pointer to the buffer into a physical address, and hand that
> address to a device.  You generally end up with either memory corruption
> (DMA hitting the wrong page allocated to the guest), or an IOMMU fault
> (DMA hitting a pages which isn't allocated to the guest).
> 
> Xen PV is very good at finding DMA bugs in drivers.  The way to resolve
> this is to fix the driver to use the proper DMA APIs - not to add even
> more magic corner cases.
> 
> In general, a lot of devices can do 4k scatter/gather, or end up making
> requests to buffers which fit within a single page, but the SWIOTLB does

Re: [Xen-devel] Upstream Dom0 DRM problems regarding swiotlb

2019-02-13 Thread Juergen Gross 
On 13/02/2019 19:38, Michael Labriola wrote:
> On Wed, Feb 13, 2019 at 1:16 PM Michael Labriola
>  wrote:
>>
>> On Wed, Feb 13, 2019 at 11:57 AM Konrad Rzeszutek Wilk
>>  wrote:
>>>
>>> On Wed, Feb 13, 2019 at 09:09:32AM -0700, Jan Beulich wrote:
>>> On 13.02.19 at 17:00,  wrote:
> On Wed, Feb 13, 2019 at 9:28 AM Jan Beulich  wrote:
> On 13.02.19 at 15:10,  wrote:
>>> Ah, so this isn't necessarily Xen-specific but rather any paravirtual
>>> guest?  That hadn't crossed my mind.  Is there an easy way to find out
>>> if we're a pv guest in the need_swiotlb conditionals?
>>
>> There's xen_pv_domain(), but I think xen_swiotlb would be more to
>> the point if the check is already to be Xen-specific. There's no generic
>> "is PV" predicate that I'm aware of.
>
> Well, that makes doing conditional code right more difficult.  I
> assume since there isn't a generic predicate, and PV isn't new, that
> it's absence is by design?  To reign in the temptation to sprinkle
> conditional code all over the kernel?  ;-)

 Well, with lguest gone, Xen is the only PV environment the kernel
 can run in, afaik at least. I guess to decide between the suggested
 options or the need for some abstracting macro (or yet something
 else), you'll really need to ask the driver maintainers. Or simply
 send a patch their way implementing one of them, and see what
 their reaction is.
>>>
>>> Could you try this out and see if it works and I will send it out:
>>>
> *snip*
>>
>> Yes, that works for me.  However, I feel like the conditional should
>> be in drm_get_max_iomem() instead of directly after it everywhere it's
>> used...  and is just checking xen_pv_domain() enough?  Jan made it
>> sound like there were possibly other PV cases that would also still
>> need swiotlb.
> 
> How about this?  It strcmp's pv_info to see if we're bare metal, does
> the comparison in a single place, moves the bit shifting comparison
> into the function (simplifying the drm driver code), and renames the
> function to more aptly describe what's going on.

...

> diff --git a/drivers/gpu/drm/drm_memory.c b/drivers/gpu/drm/drm_memory.c
> index d69e4fc1ee77..f22f6a0d20b3 100644
> --- a/drivers/gpu/drm/drm_memory.c
> +++ b/drivers/gpu/drm/drm_memory.c
> @@ -35,6 +35,7 @@
> 
>  #include 
>  #include 
> +#include 
>  #include 
>  #include "drm_legacy.h"
> 
> @@ -150,15 +151,24 @@ void drm_legacy_ioremapfree(struct drm_local_map
> *map, struct drm_device *dev)
>  }
>  EXPORT_SYMBOL(drm_legacy_ioremapfree);
> 
> -u64 drm_get_max_iomem(void)
> +bool drm_need_swiotlb_for_dma(int dma_bits)
>  {
>  struct resource *tmp;
>  resource_size_t max_iomem = 0;
> 
> +#ifdef CONFIG_PARAVIRT
> +/*
> + * Paravirtual hosts require swiotlb regardless of requested dma
> + * transfer size.
> + */
> +if (strcmp(pv_info.name, "bare hardware") != 0)
> +return true;
> +#endif
> +

No, this is really not acceptable.

Apart from Andrew's comments on using the DMA API (which I really
support) relying on the pv_info.name string is a very bad interface.
You'd need to add something like a "need_swiotlb" boolean to the
pv_info struct and set it for Xen PV and maybe others.


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH V2 0/7] Add FOLL_LONGTERM to GUP fast and use it

2019-02-13 Thread ira . weiny 
From: Ira Weiny 

NOTE: This series depends on my clean up patch to remove the write parameter
from gup_fast_permitted()[1]

HFI1, qib, and mthca, use get_user_pages_fast() due to it performance
advantages.  These pages can be held for a significant time.  But
get_user_pages_fast() does not protect against mapping of FS DAX pages.

Introduce FOLL_LONGTERM and use this flag in get_user_pages_fast() which
retains the performance while also adding the FS DAX checks.  XDP has also
shown interest in using this functionality.[2]

In addition we change get_user_pages() to use the new FOLL_LONGTERM flag and
remove the specialized get_user_pages_longterm call.

[1] https://lkml.org/lkml/2019/2/11/237
[2] https://lkml.org/lkml/2019/2/11/1789

Ira Weiny (7):
  mm/gup: Replace get_user_pages_longterm() with FOLL_LONGTERM
  mm/gup: Change write parameter to flags in fast walk
  mm/gup: Change GUP fast to use flags rather than a write 'bool'
  mm/gup: Add FOLL_LONGTERM capability to GUP fast
  IB/hfi1: Use the new FOLL_LONGTERM flag to get_user_pages_fast()
  IB/qib: Use the new FOLL_LONGTERM flag to get_user_pages_fast()
  IB/mthca: Use the new FOLL_LONGTERM flag to get_user_pages_fast()

 arch/mips/mm/gup.c  |  11 +-
 arch/powerpc/kvm/book3s_64_mmu_hv.c |   4 +-
 arch/powerpc/kvm/e500_mmu.c |   2 +-
 arch/powerpc/mm/mmu_context_iommu.c |   4 +-
 arch/s390/kvm/interrupt.c   |   2 +-
 arch/s390/mm/gup.c  |  12 +-
 arch/sh/mm/gup.c|  11 +-
 arch/sparc/mm/gup.c |   9 +-
 arch/x86/kvm/paging_tmpl.h  |   2 +-
 arch/x86/kvm/svm.c  |   2 +-
 drivers/fpga/dfl-afu-dma-region.c   |   2 +-
 drivers/gpu/drm/via/via_dmablit.c   |   3 +-
 drivers/infiniband/core/umem.c  |   5 +-
 drivers/infiniband/hw/hfi1/user_pages.c |   5 +-
 drivers/infiniband/hw/mthca/mthca_memfree.c |   3 +-
 drivers/infiniband/hw/qib/qib_user_pages.c  |   8 +-
 drivers/infiniband/hw/qib/qib_user_sdma.c   |   2 +-
 drivers/infiniband/hw/usnic/usnic_uiom.c|   9 +-
 drivers/media/v4l2-core/videobuf-dma-sg.c   |   6 +-
 drivers/misc/genwqe/card_utils.c|   2 +-
 drivers/misc/vmw_vmci/vmci_host.c   |   2 +-
 drivers/misc/vmw_vmci/vmci_queue_pair.c |   6 +-
 drivers/platform/goldfish/goldfish_pipe.c   |   3 +-
 drivers/rapidio/devices/rio_mport_cdev.c|   4 +-
 drivers/sbus/char/oradax.c  |   2 +-
 drivers/scsi/st.c   |   3 +-
 drivers/staging/gasket/gasket_page_table.c  |   4 +-
 drivers/tee/tee_shm.c   |   2 +-
 drivers/vfio/vfio_iommu_spapr_tce.c |   3 +-
 drivers/vfio/vfio_iommu_type1.c |   3 +-
 drivers/vhost/vhost.c   |   2 +-
 drivers/video/fbdev/pvr2fb.c|   2 +-
 drivers/virt/fsl_hypervisor.c   |   2 +-
 drivers/xen/gntdev.c|   2 +-
 fs/orangefs/orangefs-bufmap.c   |   2 +-
 include/linux/mm.h  |  17 +-
 kernel/futex.c  |   2 +-
 lib/iov_iter.c  |   7 +-
 mm/gup.c| 220 
 mm/gup_benchmark.c  |   5 +-
 mm/util.c   |   8 +-
 net/ceph/pagevec.c  |   2 +-
 net/rds/info.c  |   2 +-
 net/rds/rdma.c  |   3 +-
 44 files changed, 232 insertions(+), 180 deletions(-)

-- 
2.20.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH V2 3/7] mm/gup: Change GUP fast to use flags rather than a write 'bool'

2019-02-13 Thread Ira Weiny 
On Wed, Feb 13, 2019 at 04:11:10PM -0700, Jason Gunthorpe wrote:
> On Wed, Feb 13, 2019 at 03:04:51PM -0800, ira.we...@intel.com wrote:
> > From: Ira Weiny 
> > 
> > To facilitate additional options to get_user_pages_fast() change the
> > singular write parameter to be gup_flags.
> 
> So now we have:
> 
> long get_user_pages_unlocked(unsigned long start, unsigned long nr_pages,
>   struct page **pages, unsigned int gup_flags);
> 
> and 
> 
> int get_user_pages_fast(unsigned long start, int nr_pages,
>   unsigned int gup_flags, struct page **pages)
> 
> Does this make any sense? At least the arguments should be in the same
> order, I think.

Yes...  and no.  see below.

> 
> Also this comment:
> /*
>  * get_user_pages_unlocked() is suitable to replace the form:
>  *
>  *  down_read(>mmap_sem);
>  *  get_user_pages(tsk, mm, ..., pages, NULL);
>  *  up_read(>mmap_sem);
>  *
>  *  with:
>  *
>  *  get_user_pages_unlocked(tsk, mm, ..., pages);
>  *
>  * It is functionally equivalent to get_user_pages_fast so
>  * get_user_pages_fast should be used instead if specific gup_flags
>  * (e.g. FOLL_FORCE) are not required.
>  */
> 
> Needs some attention as the recommendation is now nonsense.

IMO they are not functionally equivalent.

We can't remove *_unlocked() as it is used as both a helper for the arch
specific *_fast() calls, _and_ in drivers.  Again I don't know the history here
but it could be that the drivers should never have used the call in the first
place???  Or been converted at some point?

I could change the comment to be something like

/*
 * get_user_pages_unlocked() is only to be used by arch specific
 * get_user_pages_fast() calls.  Drivers should be calling
 * get_user_pages_fast()
 */

Instead of the current comment.

And change the drivers to get_user_pages_fast().

However, I'm not sure if these drivers need the FOLL_TOUCH flag which
*_unlocked() adds for them.  And adding FOLL_TOUCH to *_fast() is not going to
give the same functionality.

It _looks_ like we can add FOLL_TOUCH functionality to the fast path in the
generic code.  I'm not sure about the arch's.

If we did that then we can have those drivers use FOLL_TOUCH or not in *_fast()
if they want/need.

> 
> Honestly a proper explanation of why two functions exist would be
> great at this point :)

I've not researched it.  I do agree that there seems to be a lot of calls in
this file and the differences are subtle.

Ira

> 
> Jason

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH V2 5/7] IB/hfi1: Use the new FOLL_LONGTERM flag to get_user_pages_fast()

2019-02-13 Thread ira . weiny 
From: Ira Weiny 

Use the new FOLL_LONGTERM to get_user_pages_fast() to protect against
FS DAX pages being mapped.

Signed-off-by: Ira Weiny 
---
 drivers/infiniband/hw/hfi1/user_pages.c | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/hw/hfi1/user_pages.c 
b/drivers/infiniband/hw/hfi1/user_pages.c
index 78ccacaf97d0..6a7f9cd5a94e 100644
--- a/drivers/infiniband/hw/hfi1/user_pages.c
+++ b/drivers/infiniband/hw/hfi1/user_pages.c
@@ -104,9 +104,11 @@ int hfi1_acquire_user_pages(struct mm_struct *mm, unsigned 
long vaddr, size_t np
bool writable, struct page **pages)
 {
int ret;
+   unsigned int gup_flags = writable ? FOLL_WRITE : 0;
 
-   ret = get_user_pages_fast(vaddr, npages, writable ? FOLL_WRITE : 0,
- pages);
+   gup_flags |= FOLL_LONGTERM;
+
+   ret = get_user_pages_fast(vaddr, npages, gup_flags, pages);
if (ret < 0)
return ret;
 
-- 
2.20.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH V2 2/7] mm/gup: Change write parameter to flags in fast walk

2019-02-13 Thread ira . weiny 
From: Ira Weiny 

In order to support more options in the GUP fast walk, change
the write parameter to flags throughout the call stack.

This patch does not change functionality and passes FOLL_WRITE
where write was previously used.

Signed-off-by: Ira Weiny 
---
 mm/gup.c | 52 ++--
 1 file changed, 26 insertions(+), 26 deletions(-)

diff --git a/mm/gup.c b/mm/gup.c
index ee96eaff118c..681388236106 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -1417,7 +1417,7 @@ static void undo_dev_pagemap(int *nr, int nr_start, 
struct page **pages)
 
 #ifdef CONFIG_ARCH_HAS_PTE_SPECIAL
 static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end,
-int write, struct page **pages, int *nr)
+unsigned int flags, struct page **pages, int *nr)
 {
struct dev_pagemap *pgmap = NULL;
int nr_start = *nr, ret = 0;
@@ -1435,7 +1435,7 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, 
unsigned long end,
if (pte_protnone(pte))
goto pte_unmap;
 
-   if (!pte_access_permitted(pte, write))
+   if (!pte_access_permitted(pte, flags & FOLL_WRITE))
goto pte_unmap;
 
if (pte_devmap(pte)) {
@@ -1487,7 +1487,7 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, 
unsigned long end,
  * useful to have gup_huge_pmd even if we can't operate on ptes.
  */
 static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end,
-int write, struct page **pages, int *nr)
+unsigned int flags, struct page **pages, int *nr)
 {
return 0;
 }
@@ -1570,12 +1570,12 @@ static int __gup_device_huge_pud(pud_t pud, pud_t 
*pudp, unsigned long addr,
 #endif
 
 static int gup_huge_pmd(pmd_t orig, pmd_t *pmdp, unsigned long addr,
-   unsigned long end, int write, struct page **pages, int *nr)
+   unsigned long end, unsigned int flags, struct page **pages, int 
*nr)
 {
struct page *head, *page;
int refs;
 
-   if (!pmd_access_permitted(orig, write))
+   if (!pmd_access_permitted(orig, flags & FOLL_WRITE))
return 0;
 
if (pmd_devmap(orig))
@@ -1608,12 +1608,12 @@ static int gup_huge_pmd(pmd_t orig, pmd_t *pmdp, 
unsigned long addr,
 }
 
 static int gup_huge_pud(pud_t orig, pud_t *pudp, unsigned long addr,
-   unsigned long end, int write, struct page **pages, int *nr)
+   unsigned long end, unsigned int flags, struct page **pages, int 
*nr)
 {
struct page *head, *page;
int refs;
 
-   if (!pud_access_permitted(orig, write))
+   if (!pud_access_permitted(orig, flags & FOLL_WRITE))
return 0;
 
if (pud_devmap(orig))
@@ -1646,13 +1646,13 @@ static int gup_huge_pud(pud_t orig, pud_t *pudp, 
unsigned long addr,
 }
 
 static int gup_huge_pgd(pgd_t orig, pgd_t *pgdp, unsigned long addr,
-   unsigned long end, int write,
+   unsigned long end, unsigned int flags,
struct page **pages, int *nr)
 {
int refs;
struct page *head, *page;
 
-   if (!pgd_access_permitted(orig, write))
+   if (!pgd_access_permitted(orig, flags & FOLL_WRITE))
return 0;
 
BUILD_BUG_ON(pgd_devmap(orig));
@@ -1683,7 +1683,7 @@ static int gup_huge_pgd(pgd_t orig, pgd_t *pgdp, unsigned 
long addr,
 }
 
 static int gup_pmd_range(pud_t pud, unsigned long addr, unsigned long end,
-   int write, struct page **pages, int *nr)
+   unsigned int flags, struct page **pages, int *nr)
 {
unsigned long next;
pmd_t *pmdp;
@@ -1705,7 +1705,7 @@ static int gup_pmd_range(pud_t pud, unsigned long addr, 
unsigned long end,
if (pmd_protnone(pmd))
return 0;
 
-   if (!gup_huge_pmd(pmd, pmdp, addr, next, write,
+   if (!gup_huge_pmd(pmd, pmdp, addr, next, flags,
pages, nr))
return 0;
 
@@ -1715,9 +1715,9 @@ static int gup_pmd_range(pud_t pud, unsigned long addr, 
unsigned long end,
 * pmd format and THP pmd format
 */
if (!gup_huge_pd(__hugepd(pmd_val(pmd)), addr,
-PMD_SHIFT, next, write, pages, nr))
+PMD_SHIFT, next, flags, pages, nr))
return 0;
-   } else if (!gup_pte_range(pmd, addr, next, write, pages, nr))
+   } else if (!gup_pte_range(pmd, addr, next, flags, pages, nr))
return 0;
} while (pmdp++, addr = next, addr != end);
 
@@ -1725,7 +1725,7 @@ static int gup_pmd_range(pud_t pud, unsigned long addr, 
unsigned long end,
 }
 
 static int

[Xen-devel] [PATCH V2 7/7] IB/mthca: Use the new FOLL_LONGTERM flag to get_user_pages_fast()

2019-02-13 Thread ira . weiny 
From: Ira Weiny 

Use the new FOLL_LONGTERM to get_user_pages_fast() to protect against
FS DAX pages being mapped.

Signed-off-by: Ira Weiny 
---
 drivers/infiniband/hw/mthca/mthca_memfree.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/mthca/mthca_memfree.c 
b/drivers/infiniband/hw/mthca/mthca_memfree.c
index 112d2f38e0de..8ff0e90d7564 100644
--- a/drivers/infiniband/hw/mthca/mthca_memfree.c
+++ b/drivers/infiniband/hw/mthca/mthca_memfree.c
@@ -472,7 +472,8 @@ int mthca_map_user_db(struct mthca_dev *dev, struct 
mthca_uar *uar,
goto out;
}
 
-   ret = get_user_pages_fast(uaddr & PAGE_MASK, 1, FOLL_WRITE, pages);
+   ret = get_user_pages_fast(uaddr & PAGE_MASK, 1,
+ FOLL_WRITE | FOLL_LONGTERM, pages);
if (ret < 0)
goto out;
 
-- 
2.20.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH V2 1/7] mm/gup: Replace get_user_pages_longterm() with FOLL_LONGTERM

2019-02-13 Thread ira . weiny 
From: Ira Weiny 

Rather than have a separate get_user_pages_longterm() call,
introduce FOLL_LONGTERM and change the longterm callers to use
it.

This patch does not change any functionality.

FOLL_LONGTERM can only be supported with get_user_pages() as it
requires vmas to determine if DAX is in use.

Signed-off-by: Ira Weiny 
---
 drivers/infiniband/core/umem.c |   5 +-
 drivers/infiniband/hw/qib/qib_user_pages.c |   8 +-
 drivers/infiniband/hw/usnic/usnic_uiom.c   |   9 +-
 drivers/media/v4l2-core/videobuf-dma-sg.c  |   6 +-
 drivers/vfio/vfio_iommu_type1.c|   3 +-
 include/linux/mm.h |  13 +-
 mm/gup.c   | 138 -
 mm/gup_benchmark.c |   5 +-
 8 files changed, 101 insertions(+), 86 deletions(-)

diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c
index b69d3efa8712..120a40df91b4 100644
--- a/drivers/infiniband/core/umem.c
+++ b/drivers/infiniband/core/umem.c
@@ -185,10 +185,11 @@ struct ib_umem *ib_umem_get(struct ib_udata *udata, 
unsigned long addr,
 
while (npages) {
down_read(>mmap_sem);
-   ret = get_user_pages_longterm(cur_base,
+   ret = get_user_pages(cur_base,
 min_t(unsigned long, npages,
   PAGE_SIZE / sizeof (struct page *)),
-gup_flags, page_list, vma_list);
+gup_flags | FOLL_LONGTERM,
+page_list, vma_list);
if (ret < 0) {
up_read(>mmap_sem);
goto umem_release;
diff --git a/drivers/infiniband/hw/qib/qib_user_pages.c 
b/drivers/infiniband/hw/qib/qib_user_pages.c
index ef8bcf366ddc..1b9368261035 100644
--- a/drivers/infiniband/hw/qib/qib_user_pages.c
+++ b/drivers/infiniband/hw/qib/qib_user_pages.c
@@ -114,10 +114,10 @@ int qib_get_user_pages(unsigned long start_page, size_t 
num_pages,
 
down_read(>mm->mmap_sem);
for (got = 0; got < num_pages; got += ret) {
-   ret = get_user_pages_longterm(start_page + got * PAGE_SIZE,
- num_pages - got,
- FOLL_WRITE | FOLL_FORCE,
- p + got, NULL);
+   ret = get_user_pages(start_page + got * PAGE_SIZE,
+num_pages - got,
+FOLL_LONGTERM | FOLL_WRITE | FOLL_FORCE,
+p + got, NULL);
if (ret < 0) {
up_read(>mm->mmap_sem);
goto bail_release;
diff --git a/drivers/infiniband/hw/usnic/usnic_uiom.c 
b/drivers/infiniband/hw/usnic/usnic_uiom.c
index 06862a6af185..1d9a182ac163 100644
--- a/drivers/infiniband/hw/usnic/usnic_uiom.c
+++ b/drivers/infiniband/hw/usnic/usnic_uiom.c
@@ -143,10 +143,11 @@ static int usnic_uiom_get_pages(unsigned long addr, 
size_t size, int writable,
ret = 0;
 
while (npages) {
-   ret = get_user_pages_longterm(cur_base,
-   min_t(unsigned long, npages,
-   PAGE_SIZE / sizeof(struct page *)),
-   gup_flags, page_list, NULL);
+   ret = get_user_pages(cur_base,
+min_t(unsigned long, npages,
+PAGE_SIZE / sizeof(struct page *)),
+gup_flags | FOLL_LONGTERM,
+page_list, NULL);
 
if (ret < 0)
goto out;
diff --git a/drivers/media/v4l2-core/videobuf-dma-sg.c 
b/drivers/media/v4l2-core/videobuf-dma-sg.c
index 08929c087e27..870a2a526e0b 100644
--- a/drivers/media/v4l2-core/videobuf-dma-sg.c
+++ b/drivers/media/v4l2-core/videobuf-dma-sg.c
@@ -186,12 +186,12 @@ static int videobuf_dma_init_user_locked(struct 
videobuf_dmabuf *dma,
dprintk(1, "init user [0x%lx+0x%lx => %d pages]\n",
data, size, dma->nr_pages);
 
-   err = get_user_pages_longterm(data & PAGE_MASK, dma->nr_pages,
-flags, dma->pages, NULL);
+   err = get_user_pages(data & PAGE_MASK, dma->nr_pages,
+flags | FOLL_LONGTERM, dma->pages, NULL);
 
if (err != dma->nr_pages) {
dma->nr_pages = (err >= 0) ? err : 0;
-   dprintk(1, "get_user_pages_longterm: err=%d [%d]\n", err,
+   dprintk(1, "get_user_pages: err=%d [%d]\n", err,
dma->nr_pages);
return err < 0 ? err : -EINVAL;
}
diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
index 73652e21efec..1500bd0bb6da 100644
---

[Xen-devel] [PATCH V2 6/7] IB/qib: Use the new FOLL_LONGTERM flag to get_user_pages_fast()

2019-02-13 Thread ira . weiny 
From: Ira Weiny 

Use the new FOLL_LONGTERM to get_user_pages_fast() to protect against
FS DAX pages being mapped.

Signed-off-by: Ira Weiny 
---
 drivers/infiniband/hw/qib/qib_user_sdma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/qib/qib_user_sdma.c 
b/drivers/infiniband/hw/qib/qib_user_sdma.c
index 31c523b2a9f5..b53cc0240e02 100644
--- a/drivers/infiniband/hw/qib/qib_user_sdma.c
+++ b/drivers/infiniband/hw/qib/qib_user_sdma.c
@@ -673,7 +673,7 @@ static int qib_user_sdma_pin_pages(const struct qib_devdata 
*dd,
else
j = npages;
 
-   ret = get_user_pages_fast(addr, j, 0, pages);
+   ret = get_user_pages_fast(addr, j, FOLL_LONGTERM, pages);
if (ret != j) {
i = 0;
j = ret;
-- 
2.20.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH V2 3/7] mm/gup: Change GUP fast to use flags rather than a write 'bool'

2019-02-13 Thread ira . weiny 
From: Ira Weiny 

To facilitate additional options to get_user_pages_fast() change the
singular write parameter to be gup_flags.

This patch does not change any functionality.  New functionality will
follow in subsequent patches.

Some of the get_user_pages_fast() call sites were unchanged because they
already passed FOLL_WRITE or 0 for the write parameter.

Signed-off-by: Ira Weiny 
---
 arch/mips/mm/gup.c | 11 ++-
 arch/powerpc/kvm/book3s_64_mmu_hv.c|  4 ++--
 arch/powerpc/kvm/e500_mmu.c|  2 +-
 arch/powerpc/mm/mmu_context_iommu.c|  4 ++--
 arch/s390/kvm/interrupt.c  |  2 +-
 arch/s390/mm/gup.c | 12 ++--
 arch/sh/mm/gup.c   | 11 ++-
 arch/sparc/mm/gup.c|  9 +
 arch/x86/kvm/paging_tmpl.h |  2 +-
 arch/x86/kvm/svm.c |  2 +-
 drivers/fpga/dfl-afu-dma-region.c  |  2 +-
 drivers/gpu/drm/via/via_dmablit.c  |  3 ++-
 drivers/infiniband/hw/hfi1/user_pages.c|  3 ++-
 drivers/misc/genwqe/card_utils.c   |  2 +-
 drivers/misc/vmw_vmci/vmci_host.c  |  2 +-
 drivers/misc/vmw_vmci/vmci_queue_pair.c|  6 --
 drivers/platform/goldfish/goldfish_pipe.c  |  3 ++-
 drivers/rapidio/devices/rio_mport_cdev.c   |  4 +++-
 drivers/sbus/char/oradax.c |  2 +-
 drivers/scsi/st.c  |  3 ++-
 drivers/staging/gasket/gasket_page_table.c |  4 ++--
 drivers/tee/tee_shm.c  |  2 +-
 drivers/vfio/vfio_iommu_spapr_tce.c|  3 ++-
 drivers/vhost/vhost.c  |  2 +-
 drivers/video/fbdev/pvr2fb.c   |  2 +-
 drivers/virt/fsl_hypervisor.c  |  2 +-
 drivers/xen/gntdev.c   |  2 +-
 fs/orangefs/orangefs-bufmap.c  |  2 +-
 include/linux/mm.h |  4 ++--
 kernel/futex.c |  2 +-
 lib/iov_iter.c |  7 +--
 mm/gup.c   | 10 +-
 mm/util.c  |  8 
 net/ceph/pagevec.c |  2 +-
 net/rds/info.c |  2 +-
 net/rds/rdma.c |  3 ++-
 36 files changed, 81 insertions(+), 65 deletions(-)

diff --git a/arch/mips/mm/gup.c b/arch/mips/mm/gup.c
index 0d14e0d8eacf..4c2b4483683c 100644
--- a/arch/mips/mm/gup.c
+++ b/arch/mips/mm/gup.c
@@ -235,7 +235,7 @@ int __get_user_pages_fast(unsigned long start, int 
nr_pages, int write,
  * get_user_pages_fast() - pin user pages in memory
  * @start: starting user address
  * @nr_pages:  number of pages from start to pin
- * @write: whether pages will be written to
+ * @gup_flags: flags modifying pin behaviour
  * @pages: array that receives pointers to the pages pinned.
  * Should be at least nr_pages long.
  *
@@ -247,8 +247,8 @@ int __get_user_pages_fast(unsigned long start, int 
nr_pages, int write,
  * requested. If nr_pages is 0 or negative, returns 0. If no pages
  * were pinned, returns -errno.
  */
-int get_user_pages_fast(unsigned long start, int nr_pages, int write,
-   struct page **pages)
+int get_user_pages_fast(unsigned long start, int nr_pages,
+   unsigned int gup_flags, struct page **pages)
 {
struct mm_struct *mm = current->mm;
unsigned long addr, len, end;
@@ -273,7 +273,8 @@ int get_user_pages_fast(unsigned long start, int nr_pages, 
int write,
next = pgd_addr_end(addr, end);
if (pgd_none(pgd))
goto slow;
-   if (!gup_pud_range(pgd, addr, next, write, pages, ))
+   if (!gup_pud_range(pgd, addr, next, gup_flags & FOLL_WRITE,
+  pages, ))
goto slow;
} while (pgdp++, addr = next, addr != end);
local_irq_enable();
@@ -289,7 +290,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, 
int write,
pages += nr;
 
ret = get_user_pages_unlocked(start, (end - start) >> PAGE_SHIFT,
- pages, write ? FOLL_WRITE : 0);
+ pages, gup_flags);
 
/* Have to be a bit careful with return values */
if (nr > 0) {
diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c 
b/arch/powerpc/kvm/book3s_64_mmu_hv.c
index bd2dcfbf00cd..8fcb0a921e46 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c
@@ -582,7 +582,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct 
kvm_vcpu *vcpu,
/* If writing != 0, then the HPTE must allow writing, if we get here */
write_ok = writing;
hva = gfn_to_hva_memslot(memslot, gfn);
-   npages = get_user_pages_fast(hva, 1, writing, pages);
+   npages = get_user_pages_fast(hva, 1,

[Xen-devel] [PATCH V2 4/7] mm/gup: Add FOLL_LONGTERM capability to GUP fast

2019-02-13 Thread ira . weiny 
From: Ira Weiny 

DAX pages were previously unprotected from longterm pins when users
called get_user_pages_fast().

Use the new FOLL_LONGTERM flag to check for DEVMAP pages and fall
back to regular GUP processing if a DEVMAP page is encountered.

Signed-off-by: Ira Weiny 
---
 mm/gup.c | 24 +---
 1 file changed, 21 insertions(+), 3 deletions(-)

diff --git a/mm/gup.c b/mm/gup.c
index 6f32d36b3c5b..f7e759c523bb 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -1439,6 +1439,9 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, 
unsigned long end,
goto pte_unmap;
 
if (pte_devmap(pte)) {
+   if (unlikely(flags & FOLL_LONGTERM))
+   goto pte_unmap;
+
pgmap = get_dev_pagemap(pte_pfn(pte), pgmap);
if (unlikely(!pgmap)) {
undo_dev_pagemap(nr, nr_start, pages);
@@ -1578,8 +1581,11 @@ static int gup_huge_pmd(pmd_t orig, pmd_t *pmdp, 
unsigned long addr,
if (!pmd_access_permitted(orig, flags & FOLL_WRITE))
return 0;
 
-   if (pmd_devmap(orig))
+   if (pmd_devmap(orig)) {
+   if (unlikely(flags & FOLL_LONGTERM))
+   return 0;
return __gup_device_huge_pmd(orig, pmdp, addr, end, pages, nr);
+   }
 
refs = 0;
page = pmd_page(orig) + ((addr & ~PMD_MASK) >> PAGE_SHIFT);
@@ -1904,8 +1910,20 @@ int get_user_pages_fast(unsigned long start, int 
nr_pages,
start += nr << PAGE_SHIFT;
pages += nr;
 
-   ret = get_user_pages_unlocked(start, nr_pages - nr, pages,
- gup_flags);
+   if (gup_flags & FOLL_LONGTERM) {
+   down_read(>mm->mmap_sem);
+   ret = __gup_longterm_locked(current, current->mm,
+   start, nr_pages - nr,
+   pages, NULL, gup_flags);
+   up_read(>mm->mmap_sem);
+   } else {
+   /*
+* retain FAULT_FOLL_ALLOW_RETRY optimization if
+* possible
+*/
+   ret = get_user_pages_unlocked(start, nr_pages - nr,
+ pages, gup_flags);
+   }
 
/* Have to be a bit careful with return values */
if (nr > 0) {
-- 
2.20.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [ovmf test] 133203: all pass - PUSHED

2019-02-13 Thread osstest service owner 
flight 133203 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133203/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf 1a35dd723bbf9333a11f6397dac77f1a5dadd3c5
baseline version:
 ovmf 3103389043bd7389fd7cef3eb291a2150af8b929

Last test of basis   133148  2019-02-11 13:43:01 Z2 days
Testing same since   133203  2019-02-12 18:45:18 Z1 days1 attempts


People who touched revisions under test:
  Antoine Coeur 
  Chen A Chen 
  Coeur 
  Jiaxin Wu 
  Michael Turner 
  Wu Jiaxin 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   3103389043..1a35dd723b  1a35dd723bbf9333a11f6397dac77f1a5dadd3c5 -> 
xen-tested-master

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-unstable-smoke test] 133241: trouble: blocked/broken/pass

2019-02-13 Thread osstest service owner 
flight 133241 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133241/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken
 build-arm64-xsm   4 host-install(4)broken REGR. vs. 133005

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  f178a00c30173c0b268d99160e19ad299b1823a2
baseline version:
 xen  455301716e1ff358cb79367213003fba771dd466

Last test of basis   133005  2019-02-07 14:00:32 Z6 days
Failing since133011  2019-02-07 18:00:36 Z6 days   37 attempts
Testing same since   133200  2019-02-12 16:00:56 Z1 days   14 attempts


People who touched revisions under test:
  Andrew Cooper 
  George Dunlap 
  Ian Jackson 
  Jan Beulich 
  Juergen Gross 
  Kevin Tian 
  Michael Tautschnig 
  Norbert Manthey 
  Norbert Manthey 
  Stefano Stabellini 
  Stefano Stabellini 
  Tim Deegan 
  Wei Liu 

jobs:
 build-arm64-xsm  broken  
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64-xsm broken
broken-step build-arm64-xsm host-install(4)

Not pushing.


commit f178a00c30173c0b268d99160e19ad299b1823a2
Author: Norbert Manthey 
Date:   Tue Feb 12 15:20:15 2019 +0100

x86/hvm: block speculative out-of-bound accesses

There are multiple arrays in the HVM interface that are accessed
with indices that are provided by the guest. To avoid speculative
out-of-bound accesses, we use the array_index_nospec macro.

When blocking speculative out-of-bound accesses, we can classify arrays
into dynamic arrays and static arrays. Where the former are allocated
during run time, the size of the latter is known during compile time.
On static arrays, compiler might be able to block speculative accesses
in the future.

This is part of the speculative hardening effort.

Reported-by: Pawel Wieczorkiewicz 
Signed-off-by: Norbert Manthey 
Reviewed-by: Jan Beulich 
Release-acked-by: Juergen Gross 

commit 56d8d0119d270f846c6c4943712b8a21fbe5d4d0
Author: Jan Beulich 
Date:   Tue Feb 12 11:54:57 2019 +0100

VMX: don't ignore P2M setup error

set_mmio_p2m_entry() may fail, in particular with -ENOMEM. Don't ignore
such an error, but instead cause domain creation to fail in such a case.

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 
Acked-by: Kevin Tian 
Release-acked-by: Juergen Gross 

commit 88b92c3820cffed4b4abeb139edc2cbd8286cb12
Author: Juergen Gross 
Date:   Tue Feb 12 11:54:07 2019 +0100

iommu: fix iommu_ops initialization

Commit 32a5ea00ec75ef53e ("IOMMU/x86: remove indirection from certain
IOMMU hook accesses") introduced iommu_ops initialized at boot time
with data declared as __initconstrel.

On Intel systems there is another path where iommu_ops is initialized
and this path is relevant on resume after returning from system suspend.
As the initialization data is no longer accessible in this case that
second initialization must be dropped in case the system isn't just
booting.

Signed-off-by: Juergen Gross 
Reviewed-by: Jan Beulich 
Acked-by: Kevin Tian 
Release-acked-by: Juergen Gross 

commit 09fc4de4a8ebb389641b8b8a632efcb7ca880e08
Author: Norbert Manthey 
Date:   Wed Feb 6 15:09:33 2019 +0100

asm:

[Xen-devel] [linux-4.9 test] 133198: trouble: blocked/broken/fail/pass

2019-02-13 Thread osstest service owner 
flight 133198 linux-4.9 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133198/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64  broken
 build-arm64-xsm  broken
 build-arm64-pvopsbroken

Tests which are failing intermittently (not blocking):
 test-amd64-i386-freebsd10-amd64 17 guest-localmigrate/x10 fail in 133142 pass 
in 133198
 test-amd64-amd64-xl-qemuu-debianhvm-amd64 16 guest-localmigrate/x10 fail pass 
in 133142

Regressions which are regarded as allowable (not blocking):
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 132748
 build-arm64   2 hosts-allocate broken REGR. vs. 132748
 build-arm64-pvops 2 hosts-allocate broken REGR. vs. 132748

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-examine  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked  n/a
 build-arm64-pvops 3 capture-logs  broken blocked in 132748
 build-arm64   3 capture-logs  broken blocked in 132748
 build-arm64-xsm   3 capture-logs  broken blocked in 132748
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 132748
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 132748
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 132748
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 132748
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 132748
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-amd64-xl-pvhv2-intel 12 guest-start fail never pass
 test-amd64-amd64-xl-pvhv2-amd 12 guest-start  fail  never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass

[Xen-devel] [xen-unstable-smoke test] 133237: trouble: blocked/broken/pass

2019-02-13 Thread osstest service owner 
flight 133237 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133237/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken
 build-arm64-xsm   4 host-install(4)broken REGR. vs. 133005

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  f178a00c30173c0b268d99160e19ad299b1823a2
baseline version:
 xen  455301716e1ff358cb79367213003fba771dd466

Last test of basis   133005  2019-02-07 14:00:32 Z6 days
Failing since133011  2019-02-07 18:00:36 Z6 days   36 attempts
Testing same since   133200  2019-02-12 16:00:56 Z1 days   13 attempts


People who touched revisions under test:
  Andrew Cooper 
  George Dunlap 
  Ian Jackson 
  Jan Beulich 
  Juergen Gross 
  Kevin Tian 
  Michael Tautschnig 
  Norbert Manthey 
  Norbert Manthey 
  Stefano Stabellini 
  Stefano Stabellini 
  Tim Deegan 
  Wei Liu 

jobs:
 build-arm64-xsm  broken  
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64-xsm broken
broken-step build-arm64-xsm host-install(4)

Not pushing.


commit f178a00c30173c0b268d99160e19ad299b1823a2
Author: Norbert Manthey 
Date:   Tue Feb 12 15:20:15 2019 +0100

x86/hvm: block speculative out-of-bound accesses

There are multiple arrays in the HVM interface that are accessed
with indices that are provided by the guest. To avoid speculative
out-of-bound accesses, we use the array_index_nospec macro.

When blocking speculative out-of-bound accesses, we can classify arrays
into dynamic arrays and static arrays. Where the former are allocated
during run time, the size of the latter is known during compile time.
On static arrays, compiler might be able to block speculative accesses
in the future.

This is part of the speculative hardening effort.

Reported-by: Pawel Wieczorkiewicz 
Signed-off-by: Norbert Manthey 
Reviewed-by: Jan Beulich 
Release-acked-by: Juergen Gross 

commit 56d8d0119d270f846c6c4943712b8a21fbe5d4d0
Author: Jan Beulich 
Date:   Tue Feb 12 11:54:57 2019 +0100

VMX: don't ignore P2M setup error

set_mmio_p2m_entry() may fail, in particular with -ENOMEM. Don't ignore
such an error, but instead cause domain creation to fail in such a case.

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 
Acked-by: Kevin Tian 
Release-acked-by: Juergen Gross 

commit 88b92c3820cffed4b4abeb139edc2cbd8286cb12
Author: Juergen Gross 
Date:   Tue Feb 12 11:54:07 2019 +0100

iommu: fix iommu_ops initialization

Commit 32a5ea00ec75ef53e ("IOMMU/x86: remove indirection from certain
IOMMU hook accesses") introduced iommu_ops initialized at boot time
with data declared as __initconstrel.

On Intel systems there is another path where iommu_ops is initialized
and this path is relevant on resume after returning from system suspend.
As the initialization data is no longer accessible in this case that
second initialization must be dropped in case the system isn't just
booting.

Signed-off-by: Juergen Gross 
Reviewed-by: Jan Beulich 
Acked-by: Kevin Tian 
Release-acked-by: Juergen Gross 

commit 09fc4de4a8ebb389641b8b8a632efcb7ca880e08
Author: Norbert Manthey 
Date:   Wed Feb 6 15:09:33 2019 +0100

asm:

[Xen-devel] [linux-4.19 test] 133196: regressions - trouble: blocked/broken/fail/pass

2019-02-13 Thread osstest service owner 
flight 133196 linux-4.19 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133196/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-pvopsbroken
 build-arm64-xsm  broken
 build-arm64  broken
 test-amd64-i386-qemut-rhel6hvm-amd 12 guest-start/redhat.repeat fail REGR. vs. 
129313
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm 16 guest-localmigrate/x10 fail 
REGR. vs. 129313
 test-amd64-amd64-qemuu-nested-intel 17 debian-hvm-install/l1/l2 fail REGR. vs. 
129313

Regressions which are regarded as allowable (not blocking):
 build-arm64   2 hosts-allocate broken REGR. vs. 129313
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 129313
 build-arm64-pvops 2 hosts-allocate broken REGR. vs. 129313
 test-amd64-amd64-rumprun-amd64 17 rumprun-demo-xenstorels/xenstorels.repeat 
fail REGR. vs. 129313

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-examine  1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 build-arm64-xsm   3 capture-logs  broken blocked in 129313
 build-arm64-pvops 3 capture-logs  broken blocked in 129313
 build-arm64   3 capture-logs  broken blocked in 129313
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail

Re: [Xen-devel] Upstream Dom0 DRM problems regarding swiotlb

2019-02-13 Thread Andrew Cooper 
On 13/02/2019 21:08, Michael Labriola wrote:
> On Wed, Feb 13, 2019 at 3:21 PM Andrew Cooper  
> wrote:
>> On 13/02/2019 20:15, Michael Labriola wrote:
>>> On Wed, Feb 13, 2019 at 2:16 PM Konrad Rzeszutek Wilk
>>>  wrote:
 On Wed, Feb 13, 2019 at 01:38:21PM -0500, Michael Labriola wrote:
> On Wed, Feb 13, 2019 at 1:16 PM Michael Labriola
>  wrote:
>> On Wed, Feb 13, 2019 at 11:57 AM Konrad Rzeszutek Wilk
>>  wrote:
>>> On Wed, Feb 13, 2019 at 09:09:32AM -0700, Jan Beulich wrote:
>>> On 13.02.19 at 17:00,  wrote:
> On Wed, Feb 13, 2019 at 9:28 AM Jan Beulich  wrote:
> On 13.02.19 at 15:10,  wrote:
>>> Ah, so this isn't necessarily Xen-specific but rather any 
>>> paravirtual
>>> guest?  That hadn't crossed my mind.  Is there an easy way to find 
>>> out
>>> if we're a pv guest in the need_swiotlb conditionals?
>> There's xen_pv_domain(), but I think xen_swiotlb would be more to
>> the point if the check is already to be Xen-specific. There's no 
>> generic
>> "is PV" predicate that I'm aware of.
> Well, that makes doing conditional code right more difficult.  I
> assume since there isn't a generic predicate, and PV isn't new, that
> it's absence is by design?  To reign in the temptation to sprinkle
> conditional code all over the kernel?  ;-)
 Well, with lguest gone, Xen is the only PV environment the kernel
 can run in, afaik at least. I guess to decide between the suggested
 options or the need for some abstracting macro (or yet something
 else), you'll really need to ask the driver maintainers. Or simply
 send a patch their way implementing one of them, and see what
 their reaction is.
>>> Could you try this out and see if it works and I will send it out:
>>>
> *snip*
>> Yes, that works for me.  However, I feel like the conditional should
>> be in drm_get_max_iomem() instead of directly after it everywhere it's
>> used...  and is just checking xen_pv_domain() enough?  Jan made it
>> sound like there were possibly other PV cases that would also still
>> need swiotlb.
> How about this?  It strcmp's pv_info to see if we're bare metal, does
> the comparison in a single place, moves the bit shifting comparison
> into the function (simplifying the drm driver code), and renames the
> function to more aptly describe what's going on.
  That looks much better.
>>> Great!  Now the only question left is:  KVM, VMware, Xen PVH, Xen HVM,
>>> and Xen PV all populate pv_info.  Do any of those other than Xen PV
>>> *really* need swiotlb.  That's slightly over my head.  As written, my
>>> patch would require swiotlb for all of them because I was attempting
>>> to not be Xen-specific.
>> Its far more complicated that "Xen PV requires swiotlb".
>>
>> I presume the underlying problem here is DRM being special and not
>> DMA-mapping its buffers, and trying to DMA to a buffer crossing a 4k
>> boundary?
> Well, I don't 100% understand how all these things work...  but here's
> what I do know.  There are a series of commits in v4.17 that try to
> optimize the radeon and amdgpu drivers by skipping calls to
> ttm_dma_populate() and ttm_dma_unpopulate() unless they're "really
> needed".  The original commit determines if swiotlb is needed by
> checking to see if the max io mapping address of system memory is over
> the video card's accessing range.  I can no longer log into Gnome on a
> Xen dom0 after upgrading my kernel to v4.20 because the code that's no
> longer happening was actually needed in a paravirtualized environment.

But from the log you provided, your bug was space exhaustion in the
swiotlb, no?

> So, I'm trying to get all my details straight so I can submit a patch
> to fix it w/out saying anything factually incorrect.

The thing which is different between Xen PV guests and most others (all
others(?), now that Lguest and UML have been dropped) is that what Linux
thinks of as PFN $N isn't necessarily adjacent to PFN $N+1 in system
physical address space.

Therefore, code which has a buffer spanning a page boundary can't just
convert a pointer to the buffer into a physical address, and hand that
address to a device.  You generally end up with either memory corruption
(DMA hitting the wrong page allocated to the guest), or an IOMMU fault
(DMA hitting a pages which isn't allocated to the guest).

Xen PV is very good at finding DMA bugs in drivers.  The way to resolve
this is to fix the driver to use the proper DMA APIs - not to add even
more magic corner cases.

In general, a lot of devices can do 4k scatter/gather, or end up making
requests to buffers which fit within a single page, but the SWIOTLB does
act as a mechanism of last resort.  It has a massive performance penalty
(due to double buffering), and does have a tendency to fragment (due to
asymmetric size

[Xen-devel] [linux-3.18 test] 133191: regressions - trouble: blocked/broken/fail/pass

2019-02-13 Thread osstest service owner 
flight 133191 linux-3.18 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133191/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64  broken
 build-arm64-pvopsbroken
 build-arm64-xsm  broken
 build-arm64-pvops 3 capture-logs   broken REGR. vs. 128858
 test-amd64-i386-rumprun-i386  7 xen-boot fail REGR. vs. 128858
 test-amd64-i386-examine   8 reboot   fail REGR. vs. 128858
 test-amd64-i386-libvirt-xsm   7 xen-boot fail REGR. vs. 128858
 test-amd64-amd64-xl-multivcpu  7 xen-bootfail REGR. vs. 128858
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  7 xen-bootfail REGR. vs. 128858
 test-amd64-amd64-libvirt  7 xen-boot fail REGR. vs. 128858
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-boot fail REGR. 
vs. 128858
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-boot fail REGR. 
vs. 128858
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-boot fail REGR. vs. 
128858
 test-amd64-i386-pair 10 xen-boot/src_hostfail REGR. vs. 128858
 test-amd64-i386-pair 11 xen-boot/dst_hostfail REGR. vs. 128858
 test-amd64-i386-freebsd10-i386  7 xen-boot   fail REGR. vs. 128858
 test-amd64-i386-xl7 xen-boot fail REGR. vs. 128858
 test-amd64-i386-xl-raw7 xen-boot fail REGR. vs. 128858
 test-amd64-amd64-libvirt-pair 10 xen-boot/src_host   fail REGR. vs. 128858
 test-amd64-i386-freebsd10-amd64  7 xen-boot  fail REGR. vs. 128858
 test-amd64-amd64-xl-credit2   7 xen-boot fail REGR. vs. 128858
 test-amd64-amd64-qemuu-nested-intel  7 xen-boot  fail REGR. vs. 128858
 test-amd64-amd64-libvirt-pair 11 xen-boot/dst_host   fail REGR. vs. 128858
 test-amd64-i386-libvirt-pair 10 xen-boot/src_hostfail REGR. vs. 128858
 test-amd64-i386-libvirt-pair 11 xen-boot/dst_hostfail REGR. vs. 128858
 test-amd64-amd64-xl   7 xen-boot fail REGR. vs. 128858
 test-amd64-amd64-rumprun-amd64  7 xen-boot   fail REGR. vs. 128858
 test-amd64-amd64-xl-shadow7 xen-boot fail REGR. vs. 128858
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm 7 xen-boot fail REGR. vs. 128858
 test-amd64-amd64-i386-pvgrub  7 xen-boot fail REGR. vs. 128858
 test-amd64-amd64-amd64-pvgrub  7 xen-bootfail REGR. vs. 128858
 test-amd64-amd64-xl-pvshim7 xen-boot fail REGR. vs. 128858
 test-amd64-amd64-xl-xsm   7 xen-boot fail REGR. vs. 128858
 test-amd64-amd64-pair10 xen-boot/src_hostfail REGR. vs. 128858
 test-amd64-amd64-pair11 xen-boot/dst_hostfail REGR. vs. 128858
 test-amd64-amd64-libvirt-xsm  7 xen-boot fail REGR. vs. 128858
 test-amd64-amd64-xl-qemuu-ovmf-amd64  7 xen-boot fail REGR. vs. 128858
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm 7 xen-boot fail REGR. vs. 128858

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt  6 xen-installfail pass in 133132
 test-amd64-i386-qemut-rhel6hvm-amd 12 guest-start/redhat.repeat fail pass in 
133132

Regressions which are regarded as allowable (not blocking):
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 128858
 build-arm64   2 hosts-allocate broken REGR. vs. 128858
 build-arm64-pvops 2 hosts-allocate broken REGR. vs. 128858
 test-amd64-amd64-xl-rtds  7 xen-boot fail REGR. vs. 128858

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-examine  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64   3 capture-logs  broken blocked in 128858
 build-arm64-xsm   3 capture-logs  broken blocked in 128858
 test-armhf-armhf-xl-vhd  10 debian-di-install   fail in 133132 like 128841
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 16 guest-localmigrate/x10 
fail in 133132 like 128858
 test-armhf-armhf-libvirt 14 saverestore-support-check fail in 133132 like 
128858
 test-armhf-armhf-libvirt-raw 13 saverestore-support-check fail in 133132 like 
128858
 test-armhf-armhf-libvirt13 migrate-support-check fail in 133132 never pass
 test-armhf-armhf-libvirt-raw

[Xen-devel] [libvirt test] 133199: trouble: blocked/broken/pass

2019-02-13 Thread osstest service owner 
flight 133199 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133199/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-pvopsbroken
 build-arm64-xsm  broken
 build-arm64  broken

Regressions which are regarded as allowable (not blocking):
 build-arm64-pvops 2 hosts-allocate broken REGR. vs. 132941
 build-arm64   2 hosts-allocate broken REGR. vs. 132941
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 132941

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 build-arm64-pvops 3 capture-logs  broken blocked in 132941
 build-arm64   3 capture-logs  broken blocked in 132941
 build-arm64-xsm   3 capture-logs  broken blocked in 132941
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 132941
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 132941
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass

version targeted for testing:
 libvirt  9916f2a3c82575678565c7eb9c5ddfffd981d446
baseline version:
 libvirt  620d9dd598fde388f56ac37bcd3b31168c2f9fc6

Last test of basis   132941  2019-02-05 14:57:44 Z8 days
Failing since132978  2019-02-06 20:25:32 Z7 days6 attempts
Testing same since   133199  2019-02-12 14:32:54 Z1 days1 attempts


People who touched revisions under test:
  Andrea Bolognani 
  Cole Robinson 
  Erik Skultety 
  Jie Wang 
  Jiri Denemark 
  John Ferlan 
  Ján Tomko 
  Michal Privoznik 
  Nikolay Shirokovskiy 
  Peter Krempa 

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  broken  
 build-i386-xsm   pass
 build-amd64  pass
 build-arm64  broken  
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-arm64-libvirt  blocked 
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-arm64-pvopsbroken  
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmpass
 test-amd64-amd64-libvirt-xsm pass
 test-arm64-arm64-libvirt-xsm blocked 
 test-amd64-i386-libvirt-xsm  pass
 test-amd64-amd64-libvirt pass
 test-arm64-arm64-libvirt blocked 
 test-armhf-armhf-libvirt pass
 test-amd64-i386-libvirt  pass
 test-amd64-amd64-libvirt-pairpass
 test-amd64-i386-libvirt-pair pass
 test-arm64-arm64-libvirt-qcow2   blocked 
 test-armhf-armhf-libvirt-raw pass
 test-amd64-amd64-libvirt-vhd pass



sg-report-flight on

Re: [Xen-devel] [PATCH v9 2/5] xen: introduce SYMBOLS_SUBTRACT and SYMBOLS_COMPARE

2019-02-13 Thread Stefano Stabellini 
On Wed, 13 Feb 2019, Jan Beulich wrote:
> >>> On 13.02.19 at 02:17,  wrote:
> > On Tue, 12 Feb 2019, Jan Beulich wrote:
> >> >>> On 12.02.19 at 13:01,  wrote:
> >> > I would particularly welcome the opinion of hypervisor maintainers on
> >> > my type safety point, below.
> >> 
> >> I agree with the requirements you put forward; I think I'd
> >> prefer the inline function versions I had suggested (or
> >> something similar) over macros though, not the least because
> >> they come with "built-in" type safety, rather than grafted one
> >> (by adding "pseudo" comparisons).
> > 
> > I don't mind the type checks in principle, I didn't add them to this
> > version because, as I wrote in a previous email, with have occurrences
> > of all three this possible calls:
> > 
> >   SYMBOL_COMPARE/SUBTRACT( symbol, symbol )
> >   SYMBOL_COMPARE/SUBTRACT( symbol, non-symbol )
> >   SYMBOL_COMPARE/SUBTRACT( non-symbol, symbol )
> > 
> > If you look through the patches you should be able to spot all three.
> > As you know "non-symbol" and "symbol" are of different type:
> > 
> >   non-symbol would be like a "struct wombat*"
> >   symbol would be like a "struct wombat[]"
> 
> These are declared types. Effective type when used as rvalue
> (i.e. also when passed as arguments to functions) is struct
> wombat * in both cases.

I see..


> The important aspect (and new idea) Ian has been introducing
> really is that the "end" symbols intentionally no longer be of
> the same type as the start ones (but some type derived
> thereof).

I missed that part of his proposal. Reading back your suggested static
inline functions and WHATEVER macro I can see that it works. But I don't
understand why is it desirable to have the "end" symbols intentionally
no longer be of the same type as the start ones.

Also, I would ask to consinder the impact of WHATEVER on the code versus
the var.S approach, which at least has the benefit of being simpler.


> > However, it is not possible to have symbol as struct wombar[] and
> > non-symbol as something entirely different like char*.
> > 
> > So, my question is: do we need three different variations of these
> > macros for the types checks?
> > 
> > 
> > I don't understand from IanJ email whether the suggestion is to change
> > the type of all the linker symbols. If so, why are we doing this instead
> > of the var.S approach? If we go and change the type of all the linker
> > symbols in C-land, this series will become much bigger and at least as
> > invasive as the var.S approach, but with added weird macros. It is kind
> > of a lose - lose situation.
> > 
> > Similarly I would prefer to avoid Jan's proposed inline function approach
> > because we have a few different array types for the linker symbols
> > (vpci_register_init_t*, struct scheduler *, etc.), it looks far more
> > work, and I am already wy over-budget for this series (as in 700%
> > over budget). I would be very happy to "gift it" to somebody else
> > willing to take it over :-)
> > 
> > Seriously, now that all the calls sites are marked appropriately and we
> > all agree on the compare/subtract macro approach, it wouldn't be hard
> > for somebody else to jump in and write the macros in their favorite way.
> > Let me know if you would like to volunteer!
> 
> I've indeed been considering this already, as I expected the point
> would come up sooner or later. Thing is though that, in particular
> with Adrian not having replied at all so far, I'm still unconvinced that
> we need to make this many changes (i.e. other than to work around
> compiler deficiencies, which would boil down to using SYMBOL_HIDE()
> from v7, but only in places where it is known certain compiler versions
> might mis-optimize it, and with a clear reference to the involved
> compiler bug/versions) at all. It's just that what we're now discussing
> is the approach I have the least problem following _if_ such a global
> "marking" of linker symbol uses ends up being necessary.

Thank you for taking this into consideration, I really appreciate it!
When/If we decide that we need a global "marking", and we also want
"fancy" type-checking, I would really appreciate your help.


> >> Furthermore - do we really need both a subtract and a compare
> >> construct? The result subtract produces can be used for
> >> comparison purposes as well, after all (just like all CPUs I know
> >> details of implement [integer] compare as a subtract discarding
> >> its numeric result, instead [or only] updating certain status flags).
> > 
> > No, we don't. In my first attempt I only had one macro. I am happy to
> > follow your suggestion and keep only SUBTRACT.
> 
> Except that, as I think Ian has also suggested, DIFFERENCE() (or
> SYMBOL_DIFFERENCE()) might be better, as it (hopefully)
> reduces the connections to the - operator, and hence the risk
> of possibly getting the argument order wrong. Otoh with the
> type safety added wrong argument order would cause a
> compile time error.

I don't

[Xen-devel] [xen-unstable-smoke test] 133236: trouble: blocked/broken/pass

2019-02-13 Thread osstest service owner 
flight 133236 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133236/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken
 build-arm64-xsm   4 host-install(4)broken REGR. vs. 133005

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  f178a00c30173c0b268d99160e19ad299b1823a2
baseline version:
 xen  455301716e1ff358cb79367213003fba771dd466

Last test of basis   133005  2019-02-07 14:00:32 Z6 days
Failing since133011  2019-02-07 18:00:36 Z6 days   35 attempts
Testing same since   133200  2019-02-12 16:00:56 Z1 days   12 attempts


People who touched revisions under test:
  Andrew Cooper 
  George Dunlap 
  Ian Jackson 
  Jan Beulich 
  Juergen Gross 
  Kevin Tian 
  Michael Tautschnig 
  Norbert Manthey 
  Norbert Manthey 
  Stefano Stabellini 
  Stefano Stabellini 
  Tim Deegan 
  Wei Liu 

jobs:
 build-arm64-xsm  broken  
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64-xsm broken
broken-step build-arm64-xsm host-install(4)

Not pushing.


commit f178a00c30173c0b268d99160e19ad299b1823a2
Author: Norbert Manthey 
Date:   Tue Feb 12 15:20:15 2019 +0100

x86/hvm: block speculative out-of-bound accesses

There are multiple arrays in the HVM interface that are accessed
with indices that are provided by the guest. To avoid speculative
out-of-bound accesses, we use the array_index_nospec macro.

When blocking speculative out-of-bound accesses, we can classify arrays
into dynamic arrays and static arrays. Where the former are allocated
during run time, the size of the latter is known during compile time.
On static arrays, compiler might be able to block speculative accesses
in the future.

This is part of the speculative hardening effort.

Reported-by: Pawel Wieczorkiewicz 
Signed-off-by: Norbert Manthey 
Reviewed-by: Jan Beulich 
Release-acked-by: Juergen Gross 

commit 56d8d0119d270f846c6c4943712b8a21fbe5d4d0
Author: Jan Beulich 
Date:   Tue Feb 12 11:54:57 2019 +0100

VMX: don't ignore P2M setup error

set_mmio_p2m_entry() may fail, in particular with -ENOMEM. Don't ignore
such an error, but instead cause domain creation to fail in such a case.

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 
Acked-by: Kevin Tian 
Release-acked-by: Juergen Gross 

commit 88b92c3820cffed4b4abeb139edc2cbd8286cb12
Author: Juergen Gross 
Date:   Tue Feb 12 11:54:07 2019 +0100

iommu: fix iommu_ops initialization

Commit 32a5ea00ec75ef53e ("IOMMU/x86: remove indirection from certain
IOMMU hook accesses") introduced iommu_ops initialized at boot time
with data declared as __initconstrel.

On Intel systems there is another path where iommu_ops is initialized
and this path is relevant on resume after returning from system suspend.
As the initialization data is no longer accessible in this case that
second initialization must be dropped in case the system isn't just
booting.

Signed-off-by: Juergen Gross 
Reviewed-by: Jan Beulich 
Acked-by: Kevin Tian 
Release-acked-by: Juergen Gross 

commit 09fc4de4a8ebb389641b8b8a632efcb7ca880e08
Author: Norbert Manthey 
Date:   Wed Feb 6 15:09:33 2019 +0100

asm:

[Xen-devel] [PATCH] xen/pciback: Don't disable PCI_COMMAND on PCI device reset.

2019-02-13 Thread Prarit Bhargava 
From: Konrad Rzeszutek Wilk 

This was submitted in 2015 here

https://marc.info/?l=linux-kernel=142807132515973=2

and has been included in Fedora builds ever since.  No issues have been
reported with the patch.

P.

8<

There is no need for this at all. Worst it means that if
the guest tries to write to BARs it could lead (on certain
platforms) to PCI SERR errors.

Please note that with af6fc858a35b90e89ea7a7ee58e66628c55c776b
"xen-pciback: limit guest control of command register"
a guest is still allowed to enable those control bits (safely), but
is not allowed to disable them and that therefore a well behaved
frontend which enables things before using them will still
function correctly.

This is done via an write to the configuration register 0x4 which
triggers on the backend side:
command_write
  \- pci_enable_device
 \- pci_enable_device_flags
\- do_pci_enable_device
   \- pcibios_enable_device
  \-pci_enable_resourcess
[which enables the PCI_COMMAND_MEMORY|PCI_COMMAND_IO]

However guests (and drivers) which don't do this could cause
problems, including the security issues which XSA-120 sought
to address.

Reported-by: Jan Beulich 
Signed-off-by: Konrad Rzeszutek Wilk 
Reviewed-by: Prarit Bhargava 
Cc: Juergen Gross 
---
 drivers/xen/xen-pciback/pciback_ops.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/drivers/xen/xen-pciback/pciback_ops.c 
b/drivers/xen/xen-pciback/pciback_ops.c
index ea4a08b83fa0..787966f44589 100644
--- a/drivers/xen/xen-pciback/pciback_ops.c
+++ b/drivers/xen/xen-pciback/pciback_ops.c
@@ -127,8 +127,6 @@ void xen_pcibk_reset_device(struct pci_dev *dev)
if (pci_is_enabled(dev))
pci_disable_device(dev);
 
-   pci_write_config_word(dev, PCI_COMMAND, 0);
-
dev->is_busmaster = 0;
} else {
pci_read_config_word(dev, PCI_COMMAND, );
-- 
2.18.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH V2 3/7] mm/gup: Change GUP fast to use flags rather than a write 'bool'

2019-02-13 Thread Jason Gunthorpe 
On Wed, Feb 13, 2019 at 03:04:51PM -0800, ira.we...@intel.com wrote:
> From: Ira Weiny 
> 
> To facilitate additional options to get_user_pages_fast() change the
> singular write parameter to be gup_flags.

So now we have:

long get_user_pages_unlocked(unsigned long start, unsigned long nr_pages,
struct page **pages, unsigned int gup_flags);

and 

int get_user_pages_fast(unsigned long start, int nr_pages,
unsigned int gup_flags, struct page **pages)

Does this make any sense? At least the arguments should be in the same
order, I think.

Also this comment:
/*
 * get_user_pages_unlocked() is suitable to replace the form:
 *
 *  down_read(>mmap_sem);
 *  get_user_pages(tsk, mm, ..., pages, NULL);
 *  up_read(>mmap_sem);
 *
 *  with:
 *
 *  get_user_pages_unlocked(tsk, mm, ..., pages);
 *
 * It is functionally equivalent to get_user_pages_fast so
 * get_user_pages_fast should be used instead if specific gup_flags
 * (e.g. FOLL_FORCE) are not required.
 */

Needs some attention as the recommendation is now nonsense.

Honestly a proper explanation of why two functions exist would be
great at this point :)

Jason

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v5 00/15] Argo: hypervisor-mediated interdomain communication

2019-02-13 Thread Stefano Stabellini 
On Wed, 13 Feb 2019, Rich Persaud wrote:
> On Feb 4, 2019, at 13:22, Stefano Stabellini  wrote:
> 
>   On Mon, 4 Feb 2019, Roger Pau Monné wrote:
>   Yes, v7 was sent to address Jan and Julien's review 
> comments in parallel
> 
>   with our ongoing discussion on v5 macros. v7 also provided 
> a checkpoint
> 
>   for Argo testers to maximize test coverage as the series 
> converges into
> 
>   a Xen 4.12 merge candidate for Juergen. It addressed:
> 
> 
>   - Jan's v6 review comments
> 
>   - Julien's v1 review comment
> 
>   - most of your xen-devel and offline review comments
> 
> 
> I think it will benefit the community to give this review in 
> public,
> 
> so other reviewers know whats going on. IMO getting this private
> 
> review makes it harder for me (as a reviewer) to know the 
> motivation
> 
> of some of the changes between versions, and likely also makes it
> 
> harder for you since you have to keep track of comments from 
> multiple
> 
> sources on different channels.
> 
> 
>   There is one more reason to require public comments which I have only
>   learned recently: for safety certifications we need to keep a record of
>   all review comments and patches that address them for traceability.
> 
> 
> Do you mean:
> 
> (A) all _merged_ patches and their review comments
> 
>  or
> 
> (B) all comments and patches (merged or not) that address them
> 
> i.e. would the certification process be seeking traceability of 
> safety-impacting patches (code, scenario A) or decisions
> (including decisions to leave code unchanged, scenario B)?

I meant (A), however, I don't know specifically if anything from (B) is
also required.


> If you mean (B), would we need an update to the Xen Security Problem Response 
> Process [1]?  e.g. public archive of all comments
> from pre-disclosure discussion, along with content hashes stored immutably?  
> Rich
> 
> [1] https://www.xenproject.org/security-policy.html

I don't think the archives of the pre-disclosure security discussions
need to be public, but they probably need to be available.___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Upstream Dom0 DRM problems regarding swiotlb

2019-02-13 Thread Michael Labriola 
On Wed, Feb 13, 2019 at 3:21 PM Andrew Cooper  wrote:
>
> On 13/02/2019 20:15, Michael Labriola wrote:
> > On Wed, Feb 13, 2019 at 2:16 PM Konrad Rzeszutek Wilk
> >  wrote:
> >> On Wed, Feb 13, 2019 at 01:38:21PM -0500, Michael Labriola wrote:
> >>> On Wed, Feb 13, 2019 at 1:16 PM Michael Labriola
> >>>  wrote:
>  On Wed, Feb 13, 2019 at 11:57 AM Konrad Rzeszutek Wilk
>   wrote:
> > On Wed, Feb 13, 2019 at 09:09:32AM -0700, Jan Beulich wrote:
> > On 13.02.19 at 17:00,  wrote:
> >>> On Wed, Feb 13, 2019 at 9:28 AM Jan Beulich  wrote:
> >>> On 13.02.19 at 15:10,  wrote:
> > Ah, so this isn't necessarily Xen-specific but rather any 
> > paravirtual
> > guest?  That hadn't crossed my mind.  Is there an easy way to find 
> > out
> > if we're a pv guest in the need_swiotlb conditionals?
>  There's xen_pv_domain(), but I think xen_swiotlb would be more to
>  the point if the check is already to be Xen-specific. There's no 
>  generic
>  "is PV" predicate that I'm aware of.
> >>> Well, that makes doing conditional code right more difficult.  I
> >>> assume since there isn't a generic predicate, and PV isn't new, that
> >>> it's absence is by design?  To reign in the temptation to sprinkle
> >>> conditional code all over the kernel?  ;-)
> >> Well, with lguest gone, Xen is the only PV environment the kernel
> >> can run in, afaik at least. I guess to decide between the suggested
> >> options or the need for some abstracting macro (or yet something
> >> else), you'll really need to ask the driver maintainers. Or simply
> >> send a patch their way implementing one of them, and see what
> >> their reaction is.
> > Could you try this out and see if it works and I will send it out:
> >
> >>> *snip*
>  Yes, that works for me.  However, I feel like the conditional should
>  be in drm_get_max_iomem() instead of directly after it everywhere it's
>  used...  and is just checking xen_pv_domain() enough?  Jan made it
>  sound like there were possibly other PV cases that would also still
>  need swiotlb.
> >>> How about this?  It strcmp's pv_info to see if we're bare metal, does
> >>> the comparison in a single place, moves the bit shifting comparison
> >>> into the function (simplifying the drm driver code), and renames the
> >>> function to more aptly describe what's going on.
> >>  That looks much better.
> > Great!  Now the only question left is:  KVM, VMware, Xen PVH, Xen HVM,
> > and Xen PV all populate pv_info.  Do any of those other than Xen PV
> > *really* need swiotlb.  That's slightly over my head.  As written, my
> > patch would require swiotlb for all of them because I was attempting
> > to not be Xen-specific.
>
> Its far more complicated that "Xen PV requires swiotlb".
>
> I presume the underlying problem here is DRM being special and not
> DMA-mapping its buffers, and trying to DMA to a buffer crossing a 4k
> boundary?

Well, I don't 100% understand how all these things work...  but here's
what I do know.  There are a series of commits in v4.17 that try to
optimize the radeon and amdgpu drivers by skipping calls to
ttm_dma_populate() and ttm_dma_unpopulate() unless they're "really
needed".  The original commit determines if swiotlb is needed by
checking to see if the max io mapping address of system memory is over
the video card's accessing range.  I can no longer log into Gnome on a
Xen dom0 after upgrading my kernel to v4.20 because the code that's no
longer happening was actually needed in a paravirtualized environment.

So, I'm trying to get all my details straight so I can submit a patch
to fix it w/out saying anything factually incorrect.

> Buffers sitting entirely within one 4k frame never need the swiotlb
> unless you've only got a 32-bit capable graphics card, and there is
> separate mode dma-mapping mode in the process of being upstreamed where
> frames which most of Linux things are adjacent do appear adjacent in
> device-virtual address space.
>
> ~Andrew

-- 
Michael D Labriola
21 Rip Van Winkle Cir
Warwick, RI 02886
401-316-9844 (cell)
401-848-8871 (work)
401-234-1306 (home)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-unstable-smoke test] 133233: trouble: blocked/broken/pass

2019-02-13 Thread osstest service owner 
flight 133233 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133233/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken

Regressions which are regarded as allowable (not blocking):
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 133005

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64-xsm   3 capture-logs  broken blocked in 133005
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  f178a00c30173c0b268d99160e19ad299b1823a2
baseline version:
 xen  455301716e1ff358cb79367213003fba771dd466

Last test of basis   133005  2019-02-07 14:00:32 Z6 days
Failing since133011  2019-02-07 18:00:36 Z6 days   34 attempts
Testing same since   133200  2019-02-12 16:00:56 Z1 days   11 attempts


People who touched revisions under test:
  Andrew Cooper 
  George Dunlap 
  Ian Jackson 
  Jan Beulich 
  Juergen Gross 
  Kevin Tian 
  Michael Tautschnig 
  Norbert Manthey 
  Norbert Manthey 
  Stefano Stabellini 
  Stefano Stabellini 
  Tim Deegan 
  Wei Liu 

jobs:
 build-arm64-xsm  broken  
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64-xsm broken
broken-step build-arm64-xsm hosts-allocate
broken-step build-arm64-xsm capture-logs

Not pushing.


commit f178a00c30173c0b268d99160e19ad299b1823a2
Author: Norbert Manthey 
Date:   Tue Feb 12 15:20:15 2019 +0100

x86/hvm: block speculative out-of-bound accesses

There are multiple arrays in the HVM interface that are accessed
with indices that are provided by the guest. To avoid speculative
out-of-bound accesses, we use the array_index_nospec macro.

When blocking speculative out-of-bound accesses, we can classify arrays
into dynamic arrays and static arrays. Where the former are allocated
during run time, the size of the latter is known during compile time.
On static arrays, compiler might be able to block speculative accesses
in the future.

This is part of the speculative hardening effort.

Reported-by: Pawel Wieczorkiewicz 
Signed-off-by: Norbert Manthey 
Reviewed-by: Jan Beulich 
Release-acked-by: Juergen Gross 

commit 56d8d0119d270f846c6c4943712b8a21fbe5d4d0
Author: Jan Beulich 
Date:   Tue Feb 12 11:54:57 2019 +0100

VMX: don't ignore P2M setup error

set_mmio_p2m_entry() may fail, in particular with -ENOMEM. Don't ignore
such an error, but instead cause domain creation to fail in such a case.

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 
Acked-by: Kevin Tian 
Release-acked-by: Juergen Gross 

commit 88b92c3820cffed4b4abeb139edc2cbd8286cb12
Author: Juergen Gross 
Date:   Tue Feb 12 11:54:07 2019 +0100

iommu: fix iommu_ops initialization

Commit 32a5ea00ec75ef53e ("IOMMU/x86: remove indirection from certain
IOMMU hook accesses") introduced iommu_ops initialized at boot time
with data declared as __initconstrel.

On Intel systems there is another path where iommu_ops is initialized
and this path is relevant on resume after returning from system suspend.
As the initialization data is no longer accessible in this case that
second initialization must be dropped in case the system isn't just
booting.

Signed-off-by: Juergen Gross 
Reviewed-by: Jan Beulich

Re: [Xen-devel] Upstream Dom0 DRM problems regarding swiotlb

2019-02-13 Thread Andrew Cooper 
On 13/02/2019 20:15, Michael Labriola wrote:
> On Wed, Feb 13, 2019 at 2:16 PM Konrad Rzeszutek Wilk
>  wrote:
>> On Wed, Feb 13, 2019 at 01:38:21PM -0500, Michael Labriola wrote:
>>> On Wed, Feb 13, 2019 at 1:16 PM Michael Labriola
>>>  wrote:
 On Wed, Feb 13, 2019 at 11:57 AM Konrad Rzeszutek Wilk
  wrote:
> On Wed, Feb 13, 2019 at 09:09:32AM -0700, Jan Beulich wrote:
> On 13.02.19 at 17:00,  wrote:
>>> On Wed, Feb 13, 2019 at 9:28 AM Jan Beulich  wrote:
>>> On 13.02.19 at 15:10,  wrote:
> Ah, so this isn't necessarily Xen-specific but rather any paravirtual
> guest?  That hadn't crossed my mind.  Is there an easy way to find out
> if we're a pv guest in the need_swiotlb conditionals?
 There's xen_pv_domain(), but I think xen_swiotlb would be more to
 the point if the check is already to be Xen-specific. There's no 
 generic
 "is PV" predicate that I'm aware of.
>>> Well, that makes doing conditional code right more difficult.  I
>>> assume since there isn't a generic predicate, and PV isn't new, that
>>> it's absence is by design?  To reign in the temptation to sprinkle
>>> conditional code all over the kernel?  ;-)
>> Well, with lguest gone, Xen is the only PV environment the kernel
>> can run in, afaik at least. I guess to decide between the suggested
>> options or the need for some abstracting macro (or yet something
>> else), you'll really need to ask the driver maintainers. Or simply
>> send a patch their way implementing one of them, and see what
>> their reaction is.
> Could you try this out and see if it works and I will send it out:
>
>>> *snip*
 Yes, that works for me.  However, I feel like the conditional should
 be in drm_get_max_iomem() instead of directly after it everywhere it's
 used...  and is just checking xen_pv_domain() enough?  Jan made it
 sound like there were possibly other PV cases that would also still
 need swiotlb.
>>> How about this?  It strcmp's pv_info to see if we're bare metal, does
>>> the comparison in a single place, moves the bit shifting comparison
>>> into the function (simplifying the drm driver code), and renames the
>>> function to more aptly describe what's going on.
>>  That looks much better.
> Great!  Now the only question left is:  KVM, VMware, Xen PVH, Xen HVM,
> and Xen PV all populate pv_info.  Do any of those other than Xen PV
> *really* need swiotlb.  That's slightly over my head.  As written, my
> patch would require swiotlb for all of them because I was attempting
> to not be Xen-specific.

Its far more complicated that "Xen PV requires swiotlb".

I presume the underlying problem here is DRM being special and not
DMA-mapping its buffers, and trying to DMA to a buffer crossing a 4k
boundary?

Buffers sitting entirely within one 4k frame never need the swiotlb
unless you've only got a 32-bit capable graphics card, and there is
separate mode dma-mapping mode in the process of being upstreamed where
frames which most of Linux things are adjacent do appear adjacent in
device-virtual address space.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Upstream Dom0 DRM problems regarding swiotlb

2019-02-13 Thread Michael Labriola 
On Wed, Feb 13, 2019 at 2:16 PM Konrad Rzeszutek Wilk
 wrote:
>
> On Wed, Feb 13, 2019 at 01:38:21PM -0500, Michael Labriola wrote:
> > On Wed, Feb 13, 2019 at 1:16 PM Michael Labriola
> >  wrote:
> > >
> > > On Wed, Feb 13, 2019 at 11:57 AM Konrad Rzeszutek Wilk
> > >  wrote:
> > > >
> > > > On Wed, Feb 13, 2019 at 09:09:32AM -0700, Jan Beulich wrote:
> > > > > >>> On 13.02.19 at 17:00,  wrote:
> > > > > > On Wed, Feb 13, 2019 at 9:28 AM Jan Beulich  
> > > > > > wrote:
> > > > > >> >>> On 13.02.19 at 15:10,  wrote:
> > > > > >> > Ah, so this isn't necessarily Xen-specific but rather any 
> > > > > >> > paravirtual
> > > > > >> > guest?  That hadn't crossed my mind.  Is there an easy way to 
> > > > > >> > find out
> > > > > >> > if we're a pv guest in the need_swiotlb conditionals?
> > > > > >>
> > > > > >> There's xen_pv_domain(), but I think xen_swiotlb would be more to
> > > > > >> the point if the check is already to be Xen-specific. There's no 
> > > > > >> generic
> > > > > >> "is PV" predicate that I'm aware of.
> > > > > >
> > > > > > Well, that makes doing conditional code right more difficult.  I
> > > > > > assume since there isn't a generic predicate, and PV isn't new, that
> > > > > > it's absence is by design?  To reign in the temptation to sprinkle
> > > > > > conditional code all over the kernel?  ;-)
> > > > >
> > > > > Well, with lguest gone, Xen is the only PV environment the kernel
> > > > > can run in, afaik at least. I guess to decide between the suggested
> > > > > options or the need for some abstracting macro (or yet something
> > > > > else), you'll really need to ask the driver maintainers. Or simply
> > > > > send a patch their way implementing one of them, and see what
> > > > > their reaction is.
> > > >
> > > > Could you try this out and see if it works and I will send it out:
> > > >
> > *snip*
> > >
> > > Yes, that works for me.  However, I feel like the conditional should
> > > be in drm_get_max_iomem() instead of directly after it everywhere it's
> > > used...  and is just checking xen_pv_domain() enough?  Jan made it
> > > sound like there were possibly other PV cases that would also still
> > > need swiotlb.
> >
> > How about this?  It strcmp's pv_info to see if we're bare metal, does
> > the comparison in a single place, moves the bit shifting comparison
> > into the function (simplifying the drm driver code), and renames the
> > function to more aptly describe what's going on.
>
>  That looks much better.

Great!  Now the only question left is:  KVM, VMware, Xen PVH, Xen HVM,
and Xen PV all populate pv_info.  Do any of those other than Xen PV
*really* need swiotlb.  That's slightly over my head.  As written, my
patch would require swiotlb for all of them because I was attempting
to not be Xen-specific.

> Would love to see this posted upstream!

I'm assuming dri-devel is the appropriate place to post this?

> >
> >
> > diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
> > b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
> > index 73ad02aea2b2..328d45b8b2ec 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
> > @@ -885,7 +885,7 @@ static int gmc_v6_0_sw_init(void *handle)
> >  pci_set_consistent_dma_mask(adev->pdev, DMA_BIT_MASK(32));
> >  dev_warn(adev->dev, "amdgpu: No coherent DMA available.\n");
> >  }
> > -adev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits);
> > +adev->need_swiotlb = drm_need_swiotlb_for_dma(dma_bits);
> >
> >  r = gmc_v6_0_init_microcode(adev);
> >  if (r) {
> > diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> > b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> > index 910c4ce19cb3..3d49eff28448 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> > @@ -1029,7 +1029,7 @@ static int gmc_v7_0_sw_init(void *handle)
> >  pci_set_consistent_dma_mask(adev->pdev, DMA_BIT_MASK(32));
> >  pr_warn("amdgpu: No coherent DMA available\n");
> >  }
> > -adev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits);
> > +adev->need_swiotlb = drm_need_swiotlb_for_dma(dma_bits);
> >
> >  r = gmc_v7_0_init_microcode(adev);
> >  if (r) {
> > diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> > b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> > index 747c068379dc..9247dd6316f1 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> > @@ -1155,7 +1155,7 @@ static int gmc_v8_0_sw_init(void *handle)
> >  pci_set_consistent_dma_mask(adev->pdev, DMA_BIT_MASK(32));
> >  pr_warn("amdgpu: No coherent DMA available\n");
> >  }
> > -adev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits);
> > +adev->need_swiotlb = drm_need_swiotlb_for_dma(dma_bits);
> >
> >  r = gmc_v8_0_init_microcode(adev);
> >  if (r) {
> > diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
> > b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
> > index

Re: [Xen-devel] Upstream Dom0 DRM problems regarding swiotlb

2019-02-13 Thread Konrad Rzeszutek Wilk 
On Wed, Feb 13, 2019 at 01:38:21PM -0500, Michael Labriola wrote:
> On Wed, Feb 13, 2019 at 1:16 PM Michael Labriola
>  wrote:
> >
> > On Wed, Feb 13, 2019 at 11:57 AM Konrad Rzeszutek Wilk
> >  wrote:
> > >
> > > On Wed, Feb 13, 2019 at 09:09:32AM -0700, Jan Beulich wrote:
> > > > >>> On 13.02.19 at 17:00,  wrote:
> > > > > On Wed, Feb 13, 2019 at 9:28 AM Jan Beulich  wrote:
> > > > >> >>> On 13.02.19 at 15:10,  wrote:
> > > > >> > Ah, so this isn't necessarily Xen-specific but rather any 
> > > > >> > paravirtual
> > > > >> > guest?  That hadn't crossed my mind.  Is there an easy way to find 
> > > > >> > out
> > > > >> > if we're a pv guest in the need_swiotlb conditionals?
> > > > >>
> > > > >> There's xen_pv_domain(), but I think xen_swiotlb would be more to
> > > > >> the point if the check is already to be Xen-specific. There's no 
> > > > >> generic
> > > > >> "is PV" predicate that I'm aware of.
> > > > >
> > > > > Well, that makes doing conditional code right more difficult.  I
> > > > > assume since there isn't a generic predicate, and PV isn't new, that
> > > > > it's absence is by design?  To reign in the temptation to sprinkle
> > > > > conditional code all over the kernel?  ;-)
> > > >
> > > > Well, with lguest gone, Xen is the only PV environment the kernel
> > > > can run in, afaik at least. I guess to decide between the suggested
> > > > options or the need for some abstracting macro (or yet something
> > > > else), you'll really need to ask the driver maintainers. Or simply
> > > > send a patch their way implementing one of them, and see what
> > > > their reaction is.
> > >
> > > Could you try this out and see if it works and I will send it out:
> > >
> *snip*
> >
> > Yes, that works for me.  However, I feel like the conditional should
> > be in drm_get_max_iomem() instead of directly after it everywhere it's
> > used...  and is just checking xen_pv_domain() enough?  Jan made it
> > sound like there were possibly other PV cases that would also still
> > need swiotlb.
> 
> How about this?  It strcmp's pv_info to see if we're bare metal, does
> the comparison in a single place, moves the bit shifting comparison
> into the function (simplifying the drm driver code), and renames the
> function to more aptly describe what's going on.

 That looks much better.

Would love to see this posted upstream!
> 
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
> b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
> index 73ad02aea2b2..328d45b8b2ec 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
> @@ -885,7 +885,7 @@ static int gmc_v6_0_sw_init(void *handle)
>  pci_set_consistent_dma_mask(adev->pdev, DMA_BIT_MASK(32));
>  dev_warn(adev->dev, "amdgpu: No coherent DMA available.\n");
>  }
> -adev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits);
> +adev->need_swiotlb = drm_need_swiotlb_for_dma(dma_bits);
> 
>  r = gmc_v6_0_init_microcode(adev);
>  if (r) {
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> index 910c4ce19cb3..3d49eff28448 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> @@ -1029,7 +1029,7 @@ static int gmc_v7_0_sw_init(void *handle)
>  pci_set_consistent_dma_mask(adev->pdev, DMA_BIT_MASK(32));
>  pr_warn("amdgpu: No coherent DMA available\n");
>  }
> -adev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits);
> +adev->need_swiotlb = drm_need_swiotlb_for_dma(dma_bits);
> 
>  r = gmc_v7_0_init_microcode(adev);
>  if (r) {
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> index 747c068379dc..9247dd6316f1 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> @@ -1155,7 +1155,7 @@ static int gmc_v8_0_sw_init(void *handle)
>  pci_set_consistent_dma_mask(adev->pdev, DMA_BIT_MASK(32));
>  pr_warn("amdgpu: No coherent DMA available\n");
>  }
> -adev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits);
> +adev->need_swiotlb = drm_need_swiotlb_for_dma(dma_bits);
> 
>  r = gmc_v8_0_init_microcode(adev);
>  if (r) {
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
> b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
> index f35d7a554ad5..89f3fe981ac5 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
> @@ -989,7 +989,7 @@ static int gmc_v9_0_sw_init(void *handle)
>  pci_set_consistent_dma_mask(adev->pdev, DMA_BIT_MASK(32));
>  printk(KERN_WARNING "amdgpu: No coherent DMA available.\n");
>  }
> -adev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits);
> +adev->need_swiotlb = drm_need_swiotlb_for_dma(dma_bits);
> 
>  if (adev->asic_type == CHIP_VEGA20) {
>  r = gfxhub_v1_1_get_xgmi_info(adev);
> diff --git a/drivers/gpu/drm/drm_memory.c

Re: [Xen-devel] Enhancing Xen's Kconfig infrastructure to support tailored solutions

2019-02-13 Thread Stefano Stabellini 
On Wed, 13 Feb 2019, Wei Liu wrote:
> On Tue, Feb 12, 2019 at 09:34:25PM -0500, Daniel P. Smith wrote:
> > Greetings,
> > 
> > On the 11/14/18 Xen x86 community call a discussion was initiated about
> > using Kconfig to build minimized versions of Xen for security, safety
> > and other certification requirements. After some offline discussions
> > with Xen contributors I realized that a variety of efforts each with
> > their own respective goals are underway,
> > 
> >  - nested virtualization
> >  - mixed criticality architectures
> >  - reducing trusted componentary
> >  - combining hardware protection of virtualization with performance and
> > ease-of-use of containers
> > 
> > These efforts use hypervisors in different roles, all which Xen is
> > capable of meeting. Today Xen's utility comes at the expense of carrying
> > features necessary for one role to be present in another role where it
> > is not required, e.g. PV interfaces that may not be essential in an ARM
> > mixed criticality deployment.
> > 
> > The initial focus will be to explore and document the range of possible
> > use cases that are of interest to the Xen community. This will be the
> > input to a design document that is crafted in conjunction with the Xen
> > maintainers, to identify possible approaches to extend the existing
> > Kconfig infrastructure to produce tailored instances of Xen.
> > 
> > If you are interested in participating in this effort, please reply to
> > this thread to outline possible use cases, design constraints and other
> > considerations for improving Xen's Kconfig infrastructure to support
> > tailoring for specific use cases.
> > 
> 
> My impression from the community call is that you want to provide
> smallish configurations for different use cases.
> 
> The Kconfig infrastructure is already able to do what you want as far as
> I can tell.  You can easily feed it a base config file -- see files
> under automation/configs/x86/.  What sort of extension is needed in your
> opinion?
> 
> As use case goes, it would be a good start if you just submit something
> you care about.

I mentioned on the call that a good first start could be a kconfig that
allows to build an hypervisor binary with only support for PVH and only
support for recent Intel machines, with the goal of minimizing the code
base to less than 100K LOC.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Upstream Dom0 DRM problems regarding swiotlb

2019-02-13 Thread Michael Labriola 
On Wed, Feb 13, 2019 at 1:16 PM Michael Labriola
 wrote:
>
> On Wed, Feb 13, 2019 at 11:57 AM Konrad Rzeszutek Wilk
>  wrote:
> >
> > On Wed, Feb 13, 2019 at 09:09:32AM -0700, Jan Beulich wrote:
> > > >>> On 13.02.19 at 17:00,  wrote:
> > > > On Wed, Feb 13, 2019 at 9:28 AM Jan Beulich  wrote:
> > > >> >>> On 13.02.19 at 15:10,  wrote:
> > > >> > Ah, so this isn't necessarily Xen-specific but rather any paravirtual
> > > >> > guest?  That hadn't crossed my mind.  Is there an easy way to find 
> > > >> > out
> > > >> > if we're a pv guest in the need_swiotlb conditionals?
> > > >>
> > > >> There's xen_pv_domain(), but I think xen_swiotlb would be more to
> > > >> the point if the check is already to be Xen-specific. There's no 
> > > >> generic
> > > >> "is PV" predicate that I'm aware of.
> > > >
> > > > Well, that makes doing conditional code right more difficult.  I
> > > > assume since there isn't a generic predicate, and PV isn't new, that
> > > > it's absence is by design?  To reign in the temptation to sprinkle
> > > > conditional code all over the kernel?  ;-)
> > >
> > > Well, with lguest gone, Xen is the only PV environment the kernel
> > > can run in, afaik at least. I guess to decide between the suggested
> > > options or the need for some abstracting macro (or yet something
> > > else), you'll really need to ask the driver maintainers. Or simply
> > > send a patch their way implementing one of them, and see what
> > > their reaction is.
> >
> > Could you try this out and see if it works and I will send it out:
> >
*snip*
>
> Yes, that works for me.  However, I feel like the conditional should
> be in drm_get_max_iomem() instead of directly after it everywhere it's
> used...  and is just checking xen_pv_domain() enough?  Jan made it
> sound like there were possibly other PV cases that would also still
> need swiotlb.

How about this?  It strcmp's pv_info to see if we're bare metal, does
the comparison in a single place, moves the bit shifting comparison
into the function (simplifying the drm driver code), and renames the
function to more aptly describe what's going on.


diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
index 73ad02aea2b2..328d45b8b2ec 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
@@ -885,7 +885,7 @@ static int gmc_v6_0_sw_init(void *handle)
 pci_set_consistent_dma_mask(adev->pdev, DMA_BIT_MASK(32));
 dev_warn(adev->dev, "amdgpu: No coherent DMA available.\n");
 }
-adev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits);
+adev->need_swiotlb = drm_need_swiotlb_for_dma(dma_bits);

 r = gmc_v6_0_init_microcode(adev);
 if (r) {
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
index 910c4ce19cb3..3d49eff28448 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
@@ -1029,7 +1029,7 @@ static int gmc_v7_0_sw_init(void *handle)
 pci_set_consistent_dma_mask(adev->pdev, DMA_BIT_MASK(32));
 pr_warn("amdgpu: No coherent DMA available\n");
 }
-adev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits);
+adev->need_swiotlb = drm_need_swiotlb_for_dma(dma_bits);

 r = gmc_v7_0_init_microcode(adev);
 if (r) {
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
index 747c068379dc..9247dd6316f1 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
@@ -1155,7 +1155,7 @@ static int gmc_v8_0_sw_init(void *handle)
 pci_set_consistent_dma_mask(adev->pdev, DMA_BIT_MASK(32));
 pr_warn("amdgpu: No coherent DMA available\n");
 }
-adev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits);
+adev->need_swiotlb = drm_need_swiotlb_for_dma(dma_bits);

 r = gmc_v8_0_init_microcode(adev);
 if (r) {
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
index f35d7a554ad5..89f3fe981ac5 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
@@ -989,7 +989,7 @@ static int gmc_v9_0_sw_init(void *handle)
 pci_set_consistent_dma_mask(adev->pdev, DMA_BIT_MASK(32));
 printk(KERN_WARNING "amdgpu: No coherent DMA available.\n");
 }
-adev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits);
+adev->need_swiotlb = drm_need_swiotlb_for_dma(dma_bits);

 if (adev->asic_type == CHIP_VEGA20) {
 r = gfxhub_v1_1_get_xgmi_info(adev);
diff --git a/drivers/gpu/drm/drm_memory.c b/drivers/gpu/drm/drm_memory.c
index d69e4fc1ee77..f22f6a0d20b3 100644
--- a/drivers/gpu/drm/drm_memory.c
+++ b/drivers/gpu/drm/drm_memory.c
@@ -35,6 +35,7 @@

 #include 
 #include 
+#include 
 #include 
 #include "drm_legacy.h"

@@ -150,15 +151,24 @@ void drm_legacy_ioremapfree(struct drm_local_map
*map, struct drm_device *dev)
 }

Re: [Xen-devel] [OSSTEST PATCH 0/4] Use snapshot for jessie-backports

2019-02-13 Thread Ian Jackson 
Ian Jackson writes ("[OSSTEST PATCH 0/4] Use snapshot for jessie-backports"):
> Things are being removed from jessie-backports, which we rely on for
> arm64 tests.  Switch to using snapshot.debian.org.
> 
> In my moderately-formal test, this has got as far as booting one of
> the machines into Xen.  It's currently doing the guest install, which
> I expect to be fine.
...
> Accordingly, I have convinced myself that this code will only affect
> arm64 in the Xen Project CI lab.

I have pushed this to osstest pretest.

Juergen, I wasn't sure if this met the formal conditions for putting
your release ack on it, but I felt you would want me to go ahead.

If you disagree, I can rewind pretest and kill the self-test flight,
putting us back in the same situation (although having maybe wasted
some machine effort).

BTW, in my semiformal test this has got as far as the guest repeat
stop/start test so it is looking good.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 0/4] Use snapshot for jessie-backports

2019-02-13 Thread Ian Jackson 
Things are being removed from jessie-backports, which we rely on for
arm64 tests.  Switch to using snapshot.debian.org.

In my moderately-formal test, this has got as far as booting one of
the machines into Xen.  It's currently doing the guest install, which
I expect to be fine.


Impact on the Xen Project Massachusetts CI, and the Xen release:

This enables the new code in preseed_backports_packages when $suite is
jessie.  preseed_backports_packages has three call sites; two in
preseed_create_guest but only where $suite is wheezy or stretch, and
one in preseed_create in a call to di_special_kernel.

di_special_kernel looks for host flags like `need-kernel-deb-SUITE'
(and a compat case for wheezy).  In the Massachusetts osstest
instance, such flags are need-kernel-deb-jessie-backports (for the
laxtons) and need-kernel-deb-jessie-special (for a handful of x86
boxes).  But the call to preseed_backports_packages only triggers if
$kp eq 'backports', so only for the laxtons.

Accordingly, I have convinced myself that this code will only affect
arm64 in the Xen Project CI lab.


Ian Jackson (4):
  backports snapshot: Honour DebianSnapshotBackports_ config var
  backports snapshot: Provide for $apt_insert and $extra_rune
  backports snapshot: Disable apt timestamp checking (sometimes)
  backports snapshot: Use 20190206T211314Z for jessie-backports

 Osstest/Debian.pm | 21 -
 production-config |  2 ++
 2 files changed, 22 insertions(+), 1 deletion(-)

-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 3/4] backports snapshot: Disable apt timestamp checking (sometimes)

2019-02-13 Thread Ian Jackson 
In jessie and earlier, this has to be done with a global option.

In later releases, it can be done by putting some options in [ ]
in the relevant sources list entry.

Signed-off-by: Ian Jackson 
---
 Osstest/Debian.pm | 13 +
 1 file changed, 13 insertions(+)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 5e74e86e..c1e75020 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -951,6 +951,19 @@ sub preseed_backports_packages (@) {
 
my $apt_insert='';
my $extra_rune='';
+   if ($suite =~ m/wheezy|jessie/) {
+   # this has global effect, unfortunately
+   $extra_rune = <\$d/50osstestsnapshot

[Xen-devel] [OSSTEST PATCH 4/4] backports snapshot: Use 20190206T211314Z for jessie-backports

2019-02-13 Thread Ian Jackson 
Some time on 2019-02-07, Debian removed linux-base from
jessie-backports.  This caused everything to break: apt wasn't happy
to get linux-base from jessie-security (because of our -t
jessie-backports, probably) and that meant there was no linux-base
suitable for linux-image-4.9.x on arm64.  We ended up trying to
boot the installed system with 3.16, which does not work on our two
SoftIron arm64 test boxes.

Also, jessie-backports about to be completely removed.

Signed-off-by: Ian Jackson 
---
 production-config   | 2 ++
 production-config-cambridge | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/production-config b/production-config
index 6b743d4f..59c74cca 100644
--- a/production-config
+++ b/production-config
@@ -91,6 +91,8 @@ TftpNetbootGroup osstest
 TftpDiVersion_wheezy 2016-06-08
 TftpDiVersion_jessie 2018-06-26
 
+DebianSnapshotBackports_jessie 
http://snapshot.debian.org/archive/debian/20190206T211314Z/
+
 # For ISO installs
 DebianImageVersion_wheezy 7.2.0
 DebianImageVersion_jessie 8.2.0
diff --git a/production-config-cambridge b/production-config-cambridge
index 8e2eadd2..ce6239bd 100644
--- a/production-config-cambridge
+++ b/production-config-cambridge
@@ -78,6 +78,8 @@ TftpNetbootGroup osstest
 TftpDiVersion_wheezy 2016-06-08
 TftpDiVersion_jessie 2018-06-26
 
+DebianSnapshotBackports_jessie 
http://snapshot.debian.org/archive/debian/20190206T211314Z/
+
 # For ISO installs
 DebianImageVersion_wheezy 7.2.0
 DebianImageVersion_jessie 8.2.0
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 2/4] backports snapshot: Provide for $apt_insert and $extra_rune

2019-02-13 Thread Ian Jackson 
No functional change.

Signed-off-by: Ian Jackson 
---
 Osstest/Debian.pm | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index ee7e03cf..5e74e86e 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -949,6 +949,8 @@ sub preseed_backports_packages (@) {
my $bp_url = $c{"DebianSnapshotBackports_$suite"};
$bp_url ||= "http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath};;
 
+   my $apt_insert='';
+   my $extra_rune='';
preseed_hook_command($ho, 'late_command', $sfx, <>/target/etc/apt/sources.list
 
 # $suite backports
-deb $bp_url $suite-backports main
+deb $apt_insert $bp_url $suite-backports main
 EOF
+$extra_rune
 in-target apt-get update
 END
 }
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 1/4] backports snapshot: Honour DebianSnapshotBackports_ config var

2019-02-13 Thread Ian Jackson 
If this is set, use it instead of the usual DebianMirrorHost and
Subpath.  No functional change with configs that don't set it.

This is not sufficient to work right yet, because snapshots
repositories have out-of-date signatures...

Signed-off-by: Ian Jackson 
---
 Osstest/Debian.pm | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index c6858253..ee7e03cf 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -946,6 +946,9 @@ sub preseed_backports_packages (@) {
 my ($ho, $sfx, $xopts, $suite, @pkgs) = @_;
 
 if (! $xopts->{BackportsSourcesAlreadyAdded}++) {
+   my $bp_url = $c{"DebianSnapshotBackports_$suite"};
+   $bp_url ||= "http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath};;
+
preseed_hook_command($ho, 'late_command', $sfx, <>/target/etc/apt/sources.list
 
 # $suite backports
-deb http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath} $suite-backports main
+deb $bp_url $suite-backports main
 EOF
 in-target apt-get update
 END
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Enhancing Xen's Kconfig infrastructure to support tailored solutions

2019-02-13 Thread Wei Liu 
On Tue, Feb 12, 2019 at 09:34:25PM -0500, Daniel P. Smith wrote:
> Greetings,
> 
> On the 11/14/18 Xen x86 community call a discussion was initiated about
> using Kconfig to build minimized versions of Xen for security, safety
> and other certification requirements. After some offline discussions
> with Xen contributors I realized that a variety of efforts each with
> their own respective goals are underway,
> 
>  - nested virtualization
>  - mixed criticality architectures
>  - reducing trusted componentary
>  - combining hardware protection of virtualization with performance and
> ease-of-use of containers
> 
> These efforts use hypervisors in different roles, all which Xen is
> capable of meeting. Today Xen's utility comes at the expense of carrying
> features necessary for one role to be present in another role where it
> is not required, e.g. PV interfaces that may not be essential in an ARM
> mixed criticality deployment.
> 
> The initial focus will be to explore and document the range of possible
> use cases that are of interest to the Xen community. This will be the
> input to a design document that is crafted in conjunction with the Xen
> maintainers, to identify possible approaches to extend the existing
> Kconfig infrastructure to produce tailored instances of Xen.
> 
> If you are interested in participating in this effort, please reply to
> this thread to outline possible use cases, design constraints and other
> considerations for improving Xen's Kconfig infrastructure to support
> tailoring for specific use cases.
> 

My impression from the community call is that you want to provide
smallish configurations for different use cases.

The Kconfig infrastructure is already able to do what you want as far as
I can tell.  You can easily feed it a base config file -- see files
under automation/configs/x86/.  What sort of extension is needed in your
opinion?

As use case goes, it would be a good start if you just submit something
you care about.

Wei.

> V/r,
> Daniel P. Smith
> Apertus Solutions, LLC

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Upstream Dom0 DRM problems regarding swiotlb

2019-02-13 Thread Michael Labriola 
On Wed, Feb 13, 2019 at 11:57 AM Konrad Rzeszutek Wilk
 wrote:
>
> On Wed, Feb 13, 2019 at 09:09:32AM -0700, Jan Beulich wrote:
> > >>> On 13.02.19 at 17:00,  wrote:
> > > On Wed, Feb 13, 2019 at 9:28 AM Jan Beulich  wrote:
> > >> >>> On 13.02.19 at 15:10,  wrote:
> > >> > Ah, so this isn't necessarily Xen-specific but rather any paravirtual
> > >> > guest?  That hadn't crossed my mind.  Is there an easy way to find out
> > >> > if we're a pv guest in the need_swiotlb conditionals?
> > >>
> > >> There's xen_pv_domain(), but I think xen_swiotlb would be more to
> > >> the point if the check is already to be Xen-specific. There's no generic
> > >> "is PV" predicate that I'm aware of.
> > >
> > > Well, that makes doing conditional code right more difficult.  I
> > > assume since there isn't a generic predicate, and PV isn't new, that
> > > it's absence is by design?  To reign in the temptation to sprinkle
> > > conditional code all over the kernel?  ;-)
> >
> > Well, with lguest gone, Xen is the only PV environment the kernel
> > can run in, afaik at least. I guess to decide between the suggested
> > options or the need for some abstracting macro (or yet something
> > else), you'll really need to ask the driver maintainers. Or simply
> > send a patch their way implementing one of them, and see what
> > their reaction is.
>
> Could you try this out and see if it works and I will send it out:
>
>
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c 
> b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
> index 9fc3296592fe..96bf1df0ed28 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
> @@ -23,6 +23,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>  #include "amdgpu.h"
>  #include "gmc_v6_0.h"
>  #include "amdgpu_ucode.h"
> @@ -887,6 +888,8 @@ static int gmc_v6_0_sw_init(void *handle)
> dev_warn(adev->dev, "amdgpu: No coherent DMA available.\n");
> }
> adev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits);
> +   if (xen_pv_domain())
> +   adev->need_swiotlb = 1;
>
> r = gmc_v6_0_init_microcode(adev);
> if (r) {
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c 
> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> index 761dcfb2fec0..710ac0ece1b0 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> @@ -23,6 +23,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>  #include "amdgpu.h"
>  #include "cikd.h"
>  #include "cik.h"
> @@ -1031,6 +1032,8 @@ static int gmc_v7_0_sw_init(void *handle)
> pr_warn("amdgpu: No coherent DMA available\n");
> }
> adev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits);
> +   if (xen_pv_domain())
> +   adev->need_swiotlb = 1;
>
> r = gmc_v7_0_init_microcode(adev);
> if (r) {
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c 
> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> index 1ad7e6b8ed1d..c418a129bb32 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> @@ -23,6 +23,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>  #include "amdgpu.h"
>  #include "gmc_v8_0.h"
>  #include "amdgpu_ucode.h"
> @@ -1156,6 +1157,8 @@ static int gmc_v8_0_sw_init(void *handle)
> pr_warn("amdgpu: No coherent DMA available\n");
> }
> adev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits);
> +   if (xen_pv_domain())
> +   adev->need_swiotlb = 1;
>
> r = gmc_v8_0_init_microcode(adev);
> if (r) {
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c 
> b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
> index bacdaef77b6c..85c0762c37ae 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
> @@ -22,6 +22,7 @@
>   */
>  #include 
>  #include 
> +#include 
>  #include "amdgpu.h"
>  #include "gmc_v9_0.h"
>  #include "amdgpu_atomfirmware.h"
> @@ -1004,6 +1005,8 @@ static int gmc_v9_0_sw_init(void *handle)
> printk(KERN_WARNING "amdgpu: No coherent DMA available.\n");
> }
> adev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits);
> +   if (xen_pv_domain())
> +   adev->need_swiotlb = 1;
>
> if (adev->gmc.xgmi.supported) {
> r = gfxhub_v1_1_get_xgmi_info(adev);
> diff --git a/drivers/gpu/drm/radeon/radeon_device.c 
> b/drivers/gpu/drm/radeon/radeon_device.c
> index 59c8a6647ff2..02fba6829936 100644
> --- a/drivers/gpu/drm/radeon/radeon_device.c
> +++ b/drivers/gpu/drm/radeon/radeon_device.c
> @@ -35,6 +35,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>  #include "radeon_reg.h"
>  #include "radeon.h"
>  #include "atom.h"
> @@ -1388,6 +1389,8 @@ int radeon_device_init(struct radeon_device *rdev,
> pr_warn("radeon: No coherent DMA available\n");
> }
> rdev->need_swiotlb = drm_get_max_iomem() >

Re: [Xen-devel] [PATCH] x86/pv: Fix construction of 32bit dom0's

2019-02-13 Thread Wei Liu 
On Thu, Feb 07, 2019 at 05:58:56AM -0700, Jan Beulich wrote:
> >>> On 06.02.19 at 21:41,  wrote:
> > Slightly RFC:
> > 
> > 1) I've not worked out exactly what the
> > 
> >  v->vcpu_info = (void *)>shared_info->compat.vcpu_info[0];
> > 
> >line is supposed to be doing and whether it is needed, but it doesn't
> >appear to matter.  It is perhaps another redundant opencoding.
> 
> Afaict this is just to be independent of the fact that the vcpu_info
> array is first in struct shared_info. I'd be fine with it getting replaced
> by a respective BUILD_BUG_ON(), but I'd like to ask that it not be
> dropped without replacement.

What do you mean by "be independent of" here? Perhaps you meant "be sure
of"? But I still fail to understand how would an assignment makes sure
a member is first in a struct.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Upstream Dom0 DRM problems regarding swiotlb

2019-02-13 Thread Michael Labriola 
On Wed, Feb 13, 2019 at 11:09 AM Jan Beulich  wrote:
>
> >>> On 13.02.19 at 17:00,  wrote:
> > On Wed, Feb 13, 2019 at 9:28 AM Jan Beulich  wrote:
> >> >>> On 13.02.19 at 15:10,  wrote:
> >> > Ah, so this isn't necessarily Xen-specific but rather any paravirtual
> >> > guest?  That hadn't crossed my mind.  Is there an easy way to find out
> >> > if we're a pv guest in the need_swiotlb conditionals?
> >>
> >> There's xen_pv_domain(), but I think xen_swiotlb would be more to
> >> the point if the check is already to be Xen-specific. There's no generic
> >> "is PV" predicate that I'm aware of.
> >
> > Well, that makes doing conditional code right more difficult.  I
> > assume since there isn't a generic predicate, and PV isn't new, that
> > it's absence is by design?  To reign in the temptation to sprinkle
> > conditional code all over the kernel?  ;-)
>
> Well, with lguest gone, Xen is the only PV environment the kernel
> can run in, afaik at least. I guess to decide between the suggested
> options or the need for some abstracting macro (or yet something
> else), you'll really need to ask the driver maintainers. Or simply
> send a patch their way implementing one of them, and see what
> their reaction is.

Thanks, I'll do that.

When you said any PV guest would need swiotlb, not just Xen, does that
mean anything that's using CONFIG_PARAVIRT?  That appears to include
KVM, VMware, Xen PVH, and Xen HVM in addition to Xen PV, all of which
populate the global pv_info structure at kernel bootup.  Is Xen PV the
only one of those that requires swiotlb?

-Mike

-- 
Michael D Labriola
21 Rip Van Winkle Cir
Warwick, RI 02886
401-316-9844 (cell)
401-848-8871 (work)
401-234-1306 (home)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-unstable-smoke test] 133231: trouble: blocked/broken/pass

2019-02-13 Thread osstest service owner 
flight 133231 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133231/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken

Regressions which are regarded as allowable (not blocking):
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 133005

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64-xsm   3 capture-logs  broken blocked in 133005
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  f178a00c30173c0b268d99160e19ad299b1823a2
baseline version:
 xen  455301716e1ff358cb79367213003fba771dd466

Last test of basis   133005  2019-02-07 14:00:32 Z6 days
Failing since133011  2019-02-07 18:00:36 Z5 days   33 attempts
Testing same since   133200  2019-02-12 16:00:56 Z1 days   10 attempts


People who touched revisions under test:
  Andrew Cooper 
  George Dunlap 
  Ian Jackson 
  Jan Beulich 
  Juergen Gross 
  Kevin Tian 
  Michael Tautschnig 
  Norbert Manthey 
  Norbert Manthey 
  Stefano Stabellini 
  Stefano Stabellini 
  Tim Deegan 
  Wei Liu 

jobs:
 build-arm64-xsm  broken  
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64-xsm broken
broken-step build-arm64-xsm hosts-allocate
broken-step build-arm64-xsm capture-logs

Not pushing.


commit f178a00c30173c0b268d99160e19ad299b1823a2
Author: Norbert Manthey 
Date:   Tue Feb 12 15:20:15 2019 +0100

x86/hvm: block speculative out-of-bound accesses

There are multiple arrays in the HVM interface that are accessed
with indices that are provided by the guest. To avoid speculative
out-of-bound accesses, we use the array_index_nospec macro.

When blocking speculative out-of-bound accesses, we can classify arrays
into dynamic arrays and static arrays. Where the former are allocated
during run time, the size of the latter is known during compile time.
On static arrays, compiler might be able to block speculative accesses
in the future.

This is part of the speculative hardening effort.

Reported-by: Pawel Wieczorkiewicz 
Signed-off-by: Norbert Manthey 
Reviewed-by: Jan Beulich 
Release-acked-by: Juergen Gross 

commit 56d8d0119d270f846c6c4943712b8a21fbe5d4d0
Author: Jan Beulich 
Date:   Tue Feb 12 11:54:57 2019 +0100

VMX: don't ignore P2M setup error

set_mmio_p2m_entry() may fail, in particular with -ENOMEM. Don't ignore
such an error, but instead cause domain creation to fail in such a case.

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 
Acked-by: Kevin Tian 
Release-acked-by: Juergen Gross 

commit 88b92c3820cffed4b4abeb139edc2cbd8286cb12
Author: Juergen Gross 
Date:   Tue Feb 12 11:54:07 2019 +0100

iommu: fix iommu_ops initialization

Commit 32a5ea00ec75ef53e ("IOMMU/x86: remove indirection from certain
IOMMU hook accesses") introduced iommu_ops initialized at boot time
with data declared as __initconstrel.

On Intel systems there is another path where iommu_ops is initialized
and this path is relevant on resume after returning from system suspend.
As the initialization data is no longer accessible in this case that
second initialization must be dropped in case the system isn't just
booting.

Signed-off-by: Juergen Gross 
Reviewed-by: Jan Beulich

Re: [Xen-devel] [PATCH for-4.12 v2 4/7] pvh/dom0: warn when dom0_mem is not set

2019-02-13 Thread Roger Pau Monné 
On Wed, Feb 13, 2019 at 09:01:07AM -0700, Jan Beulich wrote:
> >>> On 11.02.19 at 18:46,  wrote:
> > There have been several reports of the dom0 builder running out of
> > memory when building a PVH dom0 without having specified a dom0_mem
> > value. Print a warning message if dom0_mem is not set when booting in
> > PVH mode.
> > 
> > This is a temporary workaround until accounting for internal memory
> > required by Xen (ie: paging structures) is improved.
> > 
> > Signed-off-by: Roger Pau Monné 
> 
> Acked-by: Jan Beulich 
> 
> I take it that ...
> 
> > --- a/xen/arch/x86/dom0_build.c
> > +++ b/xen/arch/x86/dom0_build.c
> > @@ -378,8 +378,18 @@ unsigned long __init dom0_compute_nr_pages(
> >   * maximum of 128MB.
> >   */
> >  if ( !nr_pages )
> > +{
> >  nr_pages = avail - (pv_shim ? pv_shim_mem(avail)
> >   : min(avail / 16, 128UL << (20 - 
> > PAGE_SHIFT)));
> > +if ( is_hvm_domain(d) )
> > +/*
> > + * Temporary workaround message until internal (paging) 
> > memory
> > + * accounting required to build a pvh dom0 is improved.
> > + */
> > +printk("WARNING: PVH dom0 without dom0_mem set is still 
> > unstable. "
> > +   "If you get crashes during boot, try adding a 
> > dom0_mem parameter\n");
> > +}
> 
> ... you consider it acceptable for the message to be logged twice
> in certain cases?

Right, nr_pages could be set to 0 again if there are 2 iterations of
the parent for loop, in which case using a boolean would be better. I
can fix this in the next version.

Thanks, Roger.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Upstream Dom0 DRM problems regarding swiotlb

2019-02-13 Thread Konrad Rzeszutek Wilk 
On Wed, Feb 13, 2019 at 09:09:32AM -0700, Jan Beulich wrote:
> >>> On 13.02.19 at 17:00,  wrote:
> > On Wed, Feb 13, 2019 at 9:28 AM Jan Beulich  wrote:
> >> >>> On 13.02.19 at 15:10,  wrote:
> >> > Ah, so this isn't necessarily Xen-specific but rather any paravirtual
> >> > guest?  That hadn't crossed my mind.  Is there an easy way to find out
> >> > if we're a pv guest in the need_swiotlb conditionals?
> >>
> >> There's xen_pv_domain(), but I think xen_swiotlb would be more to
> >> the point if the check is already to be Xen-specific. There's no generic
> >> "is PV" predicate that I'm aware of.
> > 
> > Well, that makes doing conditional code right more difficult.  I
> > assume since there isn't a generic predicate, and PV isn't new, that
> > it's absence is by design?  To reign in the temptation to sprinkle
> > conditional code all over the kernel?  ;-)
> 
> Well, with lguest gone, Xen is the only PV environment the kernel
> can run in, afaik at least. I guess to decide between the suggested
> options or the need for some abstracting macro (or yet something
> else), you'll really need to ask the driver maintainers. Or simply
> send a patch their way implementing one of them, and see what
> their reaction is.

Could you try this out and see if it works and I will send it out:



diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c 
b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
index 9fc3296592fe..96bf1df0ed28 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
@@ -23,6 +23,7 @@
 #include 
 #include 
 #include 
+#include 
 #include "amdgpu.h"
 #include "gmc_v6_0.h"
 #include "amdgpu_ucode.h"
@@ -887,6 +888,8 @@ static int gmc_v6_0_sw_init(void *handle)
dev_warn(adev->dev, "amdgpu: No coherent DMA available.\n");
}
adev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits);
+   if (xen_pv_domain())
+   adev->need_swiotlb = 1;
 
r = gmc_v6_0_init_microcode(adev);
if (r) {
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c 
b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
index 761dcfb2fec0..710ac0ece1b0 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
@@ -23,6 +23,7 @@
 #include 
 #include 
 #include 
+#include 
 #include "amdgpu.h"
 #include "cikd.h"
 #include "cik.h"
@@ -1031,6 +1032,8 @@ static int gmc_v7_0_sw_init(void *handle)
pr_warn("amdgpu: No coherent DMA available\n");
}
adev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits);
+   if (xen_pv_domain())
+   adev->need_swiotlb = 1;
 
r = gmc_v7_0_init_microcode(adev);
if (r) {
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c 
b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
index 1ad7e6b8ed1d..c418a129bb32 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
@@ -23,6 +23,7 @@
 #include 
 #include 
 #include 
+#include 
 #include "amdgpu.h"
 #include "gmc_v8_0.h"
 #include "amdgpu_ucode.h"
@@ -1156,6 +1157,8 @@ static int gmc_v8_0_sw_init(void *handle)
pr_warn("amdgpu: No coherent DMA available\n");
}
adev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits);
+   if (xen_pv_domain())
+   adev->need_swiotlb = 1;
 
r = gmc_v8_0_init_microcode(adev);
if (r) {
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c 
b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
index bacdaef77b6c..85c0762c37ae 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
@@ -22,6 +22,7 @@
  */
 #include 
 #include 
+#include 
 #include "amdgpu.h"
 #include "gmc_v9_0.h"
 #include "amdgpu_atomfirmware.h"
@@ -1004,6 +1005,8 @@ static int gmc_v9_0_sw_init(void *handle)
printk(KERN_WARNING "amdgpu: No coherent DMA available.\n");
}
adev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits);
+   if (xen_pv_domain())
+   adev->need_swiotlb = 1;
 
if (adev->gmc.xgmi.supported) {
r = gfxhub_v1_1_get_xgmi_info(adev);
diff --git a/drivers/gpu/drm/radeon/radeon_device.c 
b/drivers/gpu/drm/radeon/radeon_device.c
index 59c8a6647ff2..02fba6829936 100644
--- a/drivers/gpu/drm/radeon/radeon_device.c
+++ b/drivers/gpu/drm/radeon/radeon_device.c
@@ -35,6 +35,7 @@
 #include 
 #include 
 #include 
+#include 
 #include "radeon_reg.h"
 #include "radeon.h"
 #include "atom.h"
@@ -1388,6 +1389,8 @@ int radeon_device_init(struct radeon_device *rdev,
pr_warn("radeon: No coherent DMA available\n");
}
rdev->need_swiotlb = drm_get_max_iomem() > ((u64)1 << dma_bits);
+   if (xen_pv_domain())
+   rdev->need_swiotlb = 1;
 
/* Registers mapping */
/* TODO: block userspace mapping of io register */

> 
> Jan
> 
> 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org

Re: [Xen-devel] [PATCH] vm_event: Add a new opcode to get VM_EVENT_INTERFACE_VERSION

2019-02-13 Thread Petre Ovidiu PIRCALABU 
On Wed, 2019-02-13 at 08:27 -0700, Jan Beulich wrote:
> > > > On 13.02.19 at 14:25,  wrote:
> > 
> > @@ -592,6 +592,19 @@ int vm_event_domctl(struct domain *d, struct
> > xen_domctl_vm_event_op *vec,
> >  {
> >  int rc;
> >  
> > +if ( vec->op == XEN_VM_EVENT_GET_INTERFACE_VERSION )
> > +{
> > +vec->u.get_interface_version.value =
> > VM_EVENT_INTERFACE_VERSION;
> > +return 0;
> > +}
> > +
> > +if ( unlikely(d == NULL) )
> > +{
> > +gdprintk(XENLOG_INFO,
> > + "Tried to do a memory event op on an invalid
> > domain.\n");
> > +return -EINVAL;
> > +}
> 
> To be compatible with previous behavior you want to return
> -ESRCH here. I'm also unconvinced of the need to add a log
> message here - there was none before in that case. But I'm
> not a maintainer of this code.
I will update the patch and post a new version.
> 
> > --- a/xen/include/public/domctl.h
> > +++ b/xen/include/public/domctl.h
> > @@ -778,9 +778,10 @@ struct xen_domctl_gdbsx_domstatus {
> >   * to the hypervisor to pull responses (resume) from the given
> >   * ring.
> >   */
> > -#define XEN_VM_EVENT_ENABLE   0
> > -#define XEN_VM_EVENT_DISABLE  1
> > -#define XEN_VM_EVENT_RESUME   2
> > +#define XEN_VM_EVENT_ENABLE 0
> > +#define XEN_VM_EVENT_DISABLE1
> > +#define XEN_VM_EVENT_RESUME 2
> > +#define XEN_VM_EVENT_GET_INTERFACE_VERSION  3
> 
> Perhaps just XEN_VM_EVENT_GET_VERSION?
I agree, it's simpler. I have used "INTERFACE" only to match the
VM_EVENT_INTERFACE_VERSION macro.

> 
> > @@ -843,7 +844,15 @@ struct xen_domctl_vm_event_op {
> >  uint32_t   op;   /* XEN_VM_EVENT_* */
> >  uint32_t   mode; /* XEN_DOMCTL_VM_EVENT_OP_* */
> >  
> > -uint32_t port;  /* OUT: event channel for ring */
> > +union {
> > +struct {
> > +uint32_t port;   /* OUT: event channel for ring */
> > +} enable;
> > +
> > +struct {
> > +uint32_t value;
> > +} get_interface_version;
> 
> Why the wrapper structs? Having just a "port" and "version"
> field inside the union would be good enough, wouldn't it? But
> even if you want to stick to that, the new structure's name
> could be simply "version", thus also allowing re-use for a
> hypothetical "set-version" operation (in case multiple versions
> would want/need to be supported going forward).
I chose to wrap the "port" field in a structure because it's valid only
for the XEN_VM_EVENT_ENABLE operation (I have used the wrapping structs
in order to match the operation name). 
The "version" domctl should only return the vm_event version
(VM_EVENT_INTERFACE_VERSION is a macro and is defined at compile time).
At least in this case I think the backward compatibility issues should
be handled only at the client level (easier maintainance and
deployment).

Many thanks for your comments,
Petre


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH] viridian: fix the HvFlushVirtualAddress/List hypercall implementation

2019-02-13 Thread Paul Durrant 
The current code uses hvm_asid_flush_vcpu() but this is insufficient for
a guest running in shadow mode, which results in guest crashes early in
boot if the 'hcall_remote_tlb_flush' is enabled.

This patch, instead of open coding a new flush algorithm, adapts the one
already used by the HVMOP_flush_tlbs Xen hypercall. The implementation is
modified to allow TLB flushing a subset of a domain's vCPUs. A callback
function determines whether or not a vCPU requires flushing. This mechanism
was chosen because, while it is the case that the currently implemented
viridian hypercalls specify a vCPU mask, there are newer variants which
specify a sparse HV_VP_SET and thus use of a callback will avoid needing to
expose details of this outside of the viridian subsystem if and when those
newer variants are implemented.

NOTE: Use of the common flush function requires that the hypercalls are
  restartable and so, with this patch applied, viridian_hypercall()
  can now return HVM_HCALL_preempted. This is safe as no modification
  to struct cpu_user_regs is done before the return.

Signed-off-by: Paul Durrant 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 
Cc: Wei Liu 
Cc: "Roger Pau Monné" 
---
 xen/arch/x86/hvm/hvm.c   | 55 +---
 xen/arch/x86/hvm/viridian/viridian.c | 41 +
 xen/include/asm-x86/hvm/hvm.h|  2 +
 3 files changed, 53 insertions(+), 45 deletions(-)

diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 410623d437..5de2c2aec7 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -3964,45 +3964,72 @@ static void hvm_s3_resume(struct domain *d)
 }
 }
 
-static int hvmop_flush_tlb_all(void)
+static DEFINE_PER_CPU(cpumask_t, flush_cpumask);
+
+bool hvm_flush_vcpu_tlb(bool (*flush_vcpu)(void *ctxt, struct vcpu *v),
+void *ctxt)
 {
+cpumask_t *mask = _cpu(flush_cpumask);
 struct domain *d = current->domain;
 struct vcpu *v;
 
-if ( !is_hvm_domain(d) )
-return -EINVAL;
-
 /* Avoid deadlock if more than one vcpu tries this at the same time. */
 if ( !spin_trylock(>hypercall_deadlock_mutex) )
-return -ERESTART;
+return false;
 
 /* Pause all other vcpus. */
 for_each_vcpu ( d, v )
-if ( v != current )
+if ( v != current && flush_vcpu(ctxt, v) )
 vcpu_pause_nosync(v);
 
+cpumask_clear(mask);
+
 /* Now that all VCPUs are signalled to deschedule, we wait... */
 for_each_vcpu ( d, v )
-if ( v != current )
-while ( !vcpu_runnable(v) && v->is_running )
-cpu_relax();
+{
+unsigned int cpu;
+
+if ( v == current || !flush_vcpu(ctxt, v) )
+continue;
+
+while ( !vcpu_runnable(v) && v->is_running )
+cpu_relax();
+
+cpu = read_atomic(>dirty_cpu);
+if ( is_vcpu_dirty_cpu(cpu) )
+__cpumask_set_cpu(cpu, mask);
+}
 
 /* All other vcpus are paused, safe to unlock now. */
 spin_unlock(>hypercall_deadlock_mutex);
 
 /* Flush paging-mode soft state (e.g., va->gfn cache; PAE PDPE cache). */
 for_each_vcpu ( d, v )
-paging_update_cr3(v, false);
+if ( flush_vcpu(ctxt, v) )
+paging_update_cr3(v, false);
 
-/* Flush all dirty TLBs. */
-flush_tlb_mask(d->dirty_cpumask);
+/* Flush TLBs on all CPUs with dirty vcpu state. */
+flush_tlb_mask(mask);
 
 /* Done. */
 for_each_vcpu ( d, v )
-if ( v != current )
+if ( v != current && flush_vcpu(ctxt, v) )
 vcpu_unpause(v);
 
-return 0;
+return true;
+}
+
+static bool always_flush(void *ctxt, struct vcpu *v)
+{
+return true;
+}
+
+static int hvmop_flush_tlb_all(void)
+{
+if ( !is_hvm_domain(current->domain) )
+return -EINVAL;
+
+return hvm_flush_vcpu_tlb(always_flush, NULL) ? 0 : -ERESTART;
 }
 
 static int hvmop_set_evtchn_upcall_vector(
diff --git a/xen/arch/x86/hvm/viridian/viridian.c 
b/xen/arch/x86/hvm/viridian/viridian.c
index c78b2918d9..6bb702f27e 100644
--- a/xen/arch/x86/hvm/viridian/viridian.c
+++ b/xen/arch/x86/hvm/viridian/viridian.c
@@ -430,7 +430,12 @@ void viridian_domain_deinit(struct domain *d)
 viridian_vcpu_deinit(v);
 }
 
-static DEFINE_PER_CPU(cpumask_t, ipi_cpumask);
+static bool need_flush(void *ctxt, struct vcpu *v)
+{
+uint64_t vcpu_mask = *(uint64_t *)ctxt;
+
+return vcpu_mask & (1ul << v->vcpu_id);
+}
 
 int viridian_hypercall(struct cpu_user_regs *regs)
 {
@@ -494,8 +499,6 @@ int viridian_hypercall(struct cpu_user_regs *regs)
 case HvFlushVirtualAddressSpace:
 case HvFlushVirtualAddressList:
 {
-cpumask_t *pcpu_mask;
-struct vcpu *v;
 struct {
 uint64_t address_space;
 uint64_t flags;
@@ -521,36 +524,12 @@ int viridian_hypercall(struct cpu_user_regs *regs)
 if ( input_params.flags & HV_FLUSH_ALL_PROCESSORS )

[Xen-devel] [PATCH] viridian: fix the HvFlushVirtualAddress/List hypercall implementation

2019-02-13 Thread Paul Durrant 
The current code uses hvm_asid_flush_vcpu() but this is insufficient for
a guest running in shadow mode, which results in guest crashes early in
boot if the 'hcall_remote_tlb_flush' is enabled.

This patch, instead of open coding a new flush algorithm, adapts the one
already used by the HVMOP_flush_tlbs Xen hypercall. The implementation is
modified to allow TLB flushing a subset of a domain's vCPUs. A callback
function determines whether or not a vCPU requires flushing. This mechanism
was chosen because, while it is the case that the currently implemented
viridian hypercalls specify a vCPU mask, there are newer variants which
specify a sparse HV_VP_SET and thus use of a callback will avoid needing to
expose details of this outside of the viridian subsystem if and when those
newer variants are implemented.

NOTE: Use of the common flush function requires that the hypercalls are
  restartable and so, with this patch applied, viridian_hypercall()
  can now return HVM_HCALL_preempted. This is safe as no modification
  to struct cpu_user_regs is done before the return.

Signed-off-by: Paul Durrant 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 
Cc: Wei Liu 
Cc: "Roger Pau Monné" 
---
 xen/arch/x86/hvm/hvm.c   | 55 +---
 xen/arch/x86/hvm/viridian/viridian.c | 41 +
 xen/include/asm-x86/hvm/hvm.h|  2 +
 3 files changed, 53 insertions(+), 45 deletions(-)

diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 410623d437..5de2c2aec7 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -3964,45 +3964,72 @@ static void hvm_s3_resume(struct domain *d)
 }
 }
 
-static int hvmop_flush_tlb_all(void)
+static DEFINE_PER_CPU(cpumask_t, flush_cpumask);
+
+bool hvm_flush_vcpu_tlb(bool (*flush_vcpu)(void *ctxt, struct vcpu *v),
+void *ctxt)
 {
+cpumask_t *mask = _cpu(flush_cpumask);
 struct domain *d = current->domain;
 struct vcpu *v;
 
-if ( !is_hvm_domain(d) )
-return -EINVAL;
-
 /* Avoid deadlock if more than one vcpu tries this at the same time. */
 if ( !spin_trylock(>hypercall_deadlock_mutex) )
-return -ERESTART;
+return false;
 
 /* Pause all other vcpus. */
 for_each_vcpu ( d, v )
-if ( v != current )
+if ( v != current && flush_vcpu(ctxt, v) )
 vcpu_pause_nosync(v);
 
+cpumask_clear(mask);
+
 /* Now that all VCPUs are signalled to deschedule, we wait... */
 for_each_vcpu ( d, v )
-if ( v != current )
-while ( !vcpu_runnable(v) && v->is_running )
-cpu_relax();
+{
+unsigned int cpu;
+
+if ( v == current || !flush_vcpu(ctxt, v) )
+continue;
+
+while ( !vcpu_runnable(v) && v->is_running )
+cpu_relax();
+
+cpu = read_atomic(>dirty_cpu);
+if ( is_vcpu_dirty_cpu(cpu) )
+__cpumask_set_cpu(cpu, mask);
+}
 
 /* All other vcpus are paused, safe to unlock now. */
 spin_unlock(>hypercall_deadlock_mutex);
 
 /* Flush paging-mode soft state (e.g., va->gfn cache; PAE PDPE cache). */
 for_each_vcpu ( d, v )
-paging_update_cr3(v, false);
+if ( flush_vcpu(ctxt, v) )
+paging_update_cr3(v, false);
 
-/* Flush all dirty TLBs. */
-flush_tlb_mask(d->dirty_cpumask);
+/* Flush TLBs on all CPUs with dirty vcpu state. */
+flush_tlb_mask(mask);
 
 /* Done. */
 for_each_vcpu ( d, v )
-if ( v != current )
+if ( v != current && flush_vcpu(ctxt, v) )
 vcpu_unpause(v);
 
-return 0;
+return true;
+}
+
+static bool always_flush(void *ctxt, struct vcpu *v)
+{
+return true;
+}
+
+static int hvmop_flush_tlb_all(void)
+{
+if ( !is_hvm_domain(current->domain) )
+return -EINVAL;
+
+return hvm_flush_vcpu_tlb(always_flush, NULL) ? 0 : -ERESTART;
 }
 
 static int hvmop_set_evtchn_upcall_vector(
diff --git a/xen/arch/x86/hvm/viridian/viridian.c 
b/xen/arch/x86/hvm/viridian/viridian.c
index c78b2918d9..6bb702f27e 100644
--- a/xen/arch/x86/hvm/viridian/viridian.c
+++ b/xen/arch/x86/hvm/viridian/viridian.c
@@ -430,7 +430,12 @@ void viridian_domain_deinit(struct domain *d)
 viridian_vcpu_deinit(v);
 }
 
-static DEFINE_PER_CPU(cpumask_t, ipi_cpumask);
+static bool need_flush(void *ctxt, struct vcpu *v)
+{
+uint64_t vcpu_mask = *(uint64_t *)ctxt;
+
+return vcpu_mask & (1ul << v->vcpu_id);
+}
 
 int viridian_hypercall(struct cpu_user_regs *regs)
 {
@@ -494,8 +499,6 @@ int viridian_hypercall(struct cpu_user_regs *regs)
 case HvFlushVirtualAddressSpace:
 case HvFlushVirtualAddressList:
 {
-cpumask_t *pcpu_mask;
-struct vcpu *v;
 struct {
 uint64_t address_space;
 uint64_t flags;
@@ -521,36 +524,12 @@ int viridian_hypercall(struct cpu_user_regs *regs)
 if ( input_params.flags & HV_FLUSH_ALL_PROCESSORS )

Re: [Xen-devel] [PATCH] viridian: fix the HvFlushVirtualAddress/List hypercall implementation

2019-02-13 Thread Paul Durrant 
Apologies if you get multiple copies of this but I seem to be having problems 
sending to the list.

  Paul

> -Original Message-
> From: Paul Durrant [mailto:paul.durr...@citrix.com]
> Sent: 13 February 2019 16:03
> To: Paul Durrant 
> Cc: Jan Beulich ; Andrew Cooper
> ; Wei Liu ; Roger Pau
> Monne 
> Subject: [PATCH] viridian: fix the HvFlushVirtualAddress/List hypercall
> implementation
> 
> The current code uses hvm_asid_flush_vcpu() but this is insufficient for
> a guest running in shadow mode, which results in guest crashes early in
> boot if the 'hcall_remote_tlb_flush' is enabled.
> 
> This patch, instead of open coding a new flush algorithm, adapts the one
> already used by the HVMOP_flush_tlbs Xen hypercall. The implementation is
> modified to allow TLB flushing a subset of a domain's vCPUs. A callback
> function determines whether or not a vCPU requires flushing. This
> mechanism
> was chosen because, while it is the case that the currently implemented
> viridian hypercalls specify a vCPU mask, there are newer variants which
> specify a sparse HV_VP_SET and thus use of a callback will avoid needing
> to
> expose details of this outside of the viridian subsystem if and when those
> newer variants are implemented.
> 
> NOTE: Use of the common flush function requires that the hypercalls are
>   restartable and so, with this patch applied, viridian_hypercall()
>   can now return HVM_HCALL_preempted. This is safe as no modification
>   to struct cpu_user_regs is done before the return.
> 
> Signed-off-by: Paul Durrant 
> ---
> Cc: Jan Beulich 
> Cc: Andrew Cooper 
> Cc: Wei Liu 
> Cc: "Roger Pau Monné" 
> ---
>  xen/arch/x86/hvm/hvm.c   | 55 +---
>  xen/arch/x86/hvm/viridian/viridian.c | 41 +
>  xen/include/asm-x86/hvm/hvm.h|  2 +
>  3 files changed, 53 insertions(+), 45 deletions(-)
> 
> diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
> index 410623d437..5de2c2aec7 100644
> --- a/xen/arch/x86/hvm/hvm.c
> +++ b/xen/arch/x86/hvm/hvm.c
> @@ -3964,45 +3964,72 @@ static void hvm_s3_resume(struct domain *d)
>  }
>  }
> 
> -static int hvmop_flush_tlb_all(void)
> +static DEFINE_PER_CPU(cpumask_t, flush_cpumask);
> +
> +bool hvm_flush_vcpu_tlb(bool (*flush_vcpu)(void *ctxt, struct vcpu *v),
> +void *ctxt)
>  {
> +cpumask_t *mask = _cpu(flush_cpumask);
>  struct domain *d = current->domain;
>  struct vcpu *v;
> 
> -if ( !is_hvm_domain(d) )
> -return -EINVAL;
> -
>  /* Avoid deadlock if more than one vcpu tries this at the same time.
> */
>  if ( !spin_trylock(>hypercall_deadlock_mutex) )
> -return -ERESTART;
> +return false;
> 
>  /* Pause all other vcpus. */
>  for_each_vcpu ( d, v )
> -if ( v != current )
> +if ( v != current && flush_vcpu(ctxt, v) )
>  vcpu_pause_nosync(v);
> 
> +cpumask_clear(mask);
> +
>  /* Now that all VCPUs are signalled to deschedule, we wait... */
>  for_each_vcpu ( d, v )
> -if ( v != current )
> -while ( !vcpu_runnable(v) && v->is_running )
> -cpu_relax();
> +{
> +unsigned int cpu;
> +
> +if ( v == current || !flush_vcpu(ctxt, v) )
> +continue;
> +
> +while ( !vcpu_runnable(v) && v->is_running )
> +cpu_relax();
> +
> +cpu = read_atomic(>dirty_cpu);
> +if ( is_vcpu_dirty_cpu(cpu) )
> +__cpumask_set_cpu(cpu, mask);
> +}
> 
>  /* All other vcpus are paused, safe to unlock now. */
>  spin_unlock(>hypercall_deadlock_mutex);
> 
>  /* Flush paging-mode soft state (e.g., va->gfn cache; PAE PDPE
> cache). */
>  for_each_vcpu ( d, v )
> -paging_update_cr3(v, false);
> +if ( flush_vcpu(ctxt, v) )
> +paging_update_cr3(v, false);
> 
> -/* Flush all dirty TLBs. */
> -flush_tlb_mask(d->dirty_cpumask);
> +/* Flush TLBs on all CPUs with dirty vcpu state. */
> +flush_tlb_mask(mask);
> 
>  /* Done. */
>  for_each_vcpu ( d, v )
> -if ( v != current )
> +if ( v != current && flush_vcpu(ctxt, v) )
>  vcpu_unpause(v);
> 
> -return 0;
> +return true;
> +}
> +
> +static bool always_flush(void *ctxt, struct vcpu *v)
> +{
> +return true;
> +}
> +
> +static int hvmop_flush_tlb_all(void)
> +{
> +if ( !is_hvm_domain(current->domain) )
> +return -EINVAL;
> +
> +return hvm_flush_vcpu_tlb(always_flush, NULL) ? 0 : -ERESTART;
>  }
> 
>  static int hvmop_set_evtchn_upcall_vector(
> diff --git a/xen/arch/x86/hvm/viridian/viridian.c
> b/xen/arch/x86/hvm/viridian/viridian.c
> index c78b2918d9..6bb702f27e 100644
> --- a/xen/arch/x86/hvm/viridian/viridian.c
> +++ b/xen/arch/x86/hvm/viridian/viridian.c
> @@ -430,7 +430,12 @@ void viridian_domain_deinit(struct domain *d)
>  viridian_vcpu_deinit(v);
>  }
> 
> -static DEFINE_PER_CPU(cpumask_t, ipi_cpumask);
>

Re: [Xen-devel] Upstream Dom0 DRM problems regarding swiotlb

2019-02-13 Thread Jan Beulich 
>>> On 13.02.19 at 17:00,  wrote:
> On Wed, Feb 13, 2019 at 9:28 AM Jan Beulich  wrote:
>> >>> On 13.02.19 at 15:10,  wrote:
>> > Ah, so this isn't necessarily Xen-specific but rather any paravirtual
>> > guest?  That hadn't crossed my mind.  Is there an easy way to find out
>> > if we're a pv guest in the need_swiotlb conditionals?
>>
>> There's xen_pv_domain(), but I think xen_swiotlb would be more to
>> the point if the check is already to be Xen-specific. There's no generic
>> "is PV" predicate that I'm aware of.
> 
> Well, that makes doing conditional code right more difficult.  I
> assume since there isn't a generic predicate, and PV isn't new, that
> it's absence is by design?  To reign in the temptation to sprinkle
> conditional code all over the kernel?  ;-)

Well, with lguest gone, Xen is the only PV environment the kernel
can run in, afaik at least. I guess to decide between the suggested
options or the need for some abstracting macro (or yet something
else), you'll really need to ask the driver maintainers. Or simply
send a patch their way implementing one of them, and see what
their reaction is.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Upstream Dom0 DRM problems regarding swiotlb

2019-02-13 Thread Michael Labriola 
On Wed, Feb 13, 2019 at 9:28 AM Jan Beulich  wrote:
>
> >>> On 13.02.19 at 15:10,  wrote:
> > On Wed, Feb 13, 2019 at 5:34 AM Jan Beulich  wrote:
> >>
> >> >>> On 12.02.19 at 19:46,  wrote:
> >> > Konrad,
> >> >
> >> > Starting w/ v4.17, I cannot log in to GNOME w/out getting the
> >> > following mess in dmesg and ending up back at the GDM login screen.
> >> >
> >> > [   28.554259] radeon_dp_aux_transfer_native: 200 callbacks suppressed
> >> > [   31.219821] radeon :01:00.0: swiotlb buffer is full (sz: 2097152
> >> > bytes)
> >> > [   31.220030] [drm:radeon_gem_object_create [radeon]] *ERROR* Failed
> >> > to allocate GEM object (16384000, 2, 4096, -14)
> >> > [   31.226109] radeon :01:00.0: swiotlb buffer is full (sz: 2097152
> >> > bytes)
> >> > [   31.226300] [drm:radeon_gem_object_create [radeon]] *ERROR* Failed
> >> > to allocate GEM object (16384000, 2, 4096, -14)
> >> > [   31.300734] gnome-shell[1935]: segfault at 88 ip 7f39151cd904
> >> > sp 7ffc97611ad8 error 4 in libmutter-cogl.so[7f3915178000+aa000]
> >> > [   31.300745] Code: 5f c3 0f 1f 40 00 48 8b 47 78 48 8b 40 40 ff e0
> >> > 66 0f 1f 44 00 00 48 8b 47 78 48 8b 40 48 ff e0 66 0f 1f 44 00 00 48
> >> > 8b 47 78 <48> 8b 80 88 00 00 00 ff e0 0f 1f 00 48 8b 47 78 48 8b 40 68
> >> > ff e0
> >> > [   38.193302] radeon_dp_aux_transfer_native: 116 callbacks suppressed
> >> > [   40.009317] radeon :01:00.0: swiotlb buffer is full (sz: 2097152
> >> > bytes)
> >> > [   40.009488] [drm:radeon_gem_object_create [radeon]] *ERROR* Failed
> >> > to allocate GEM object (16384000, 2, 4096, -14)
> >> > [   40.015114] radeon :01:00.0: swiotlb buffer is full (sz: 2097152
> >> > bytes)
> >> > [   40.015297] [drm:radeon_gem_object_create [radeon]] *ERROR* Failed
> >> > to allocate GEM object (16384000, 2, 4096, -14)
> >> > [   40.028302] gnome-shell[2431]: segfault at 2dadf40 ip
> >> > 02dadf40 sp 7ffcd24ea5f8 error 15
> >> > [   40.028306] Code: 20 6e 31 00 00 00 00 00 00 00 00 37 e3 3d 2d 7f
> >> > 00 00 80 f4 e6 3d 2d 7f 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> >> > 00 00 00 <00> 00 00 00 00 00 00 00 c1 00 00 00 00 00 00 00 80 e1 d2 03
> >> > 00 00
> >> >
> >> >
> >> > This happens w/ both radeon and amdgpu.
> >> >
> >> > I bisected down to the following range of commits, which basically add
> >> > conditional code to radeon and amdgpu to NOT use swiotlb if dma_bits
> >> > is smaller than the system's max iomem address...  but that very much
> >> > doesn't work on a Xen dom0.
> >>
> >> Well, not so much a Xen Dom0, but a Xen PV domain.
> >>
> >> > 82626363 drm: add func to get max iomem address v2
> >> > fd5fd480 drm/amdgpu: only enable swiotlb alloc when need v2
> >> > 1bc3d3cc drm/radeon: only enable swiotlb path when need v2
> >> >
> >> > Reverting the offending commits gives me a usable v4.20 dom0 kernel w/
> >> > working 3d support.  Not sure what the appropriate upstream fix for
> >> > this would be, as I don't 100% understand this.  Could you enlighten
> >> > me?  ;-)
> >>
> >> Well, this depends on how much abstraction we want, and how
> >> much abstraction the maintainers of the DRM drivers demand.
> >> It could be as simple as adding xen_swiotlb checks into the
> >> conditionals setting ->need_swiotlb, but in an abstract sense
> >> the issue of course exists for PV guests of any hypervisor.
> >> (Altering drm_get_max_iomem() itself would seem wrong to me,
> >> unless its name was also changed.)

So, the commit message for the patch that added drm_get_max_iomem()
specifically states that the function "will be used to check if the
driver needs swiotlb"...  Sounds like a logical place to put some type
of PV conditional to me.  I get that that seems a bit sneaky and
unrelated to the function's name... but it definitely plays directly
into the function's stated purpose.

> >
> > Ah, so this isn't necessarily Xen-specific but rather any paravirtual
> > guest?  That hadn't crossed my mind.  Is there an easy way to find out
> > if we're a pv guest in the need_swiotlb conditionals?
>
> There's xen_pv_domain(), but I think xen_swiotlb would be more to
> the point if the check is already to be Xen-specific. There's no generic
> "is PV" predicate that I'm aware of.

Well, that makes doing conditional code right more difficult.  I
assume since there isn't a generic predicate, and PV isn't new, that
it's absence is by design?  To reign in the temptation to sprinkle
conditional code all over the kernel?  ;-)

>
> >  If not, we
> > should at least add a module parameter to force swiotlb usage to both
> > radeon and amdgpu.  I'd be more than happy to gin up a patch to do
> > either and submit to upstream (dri-devel, I guess).
>
> I don't think module parameters are a good way forward here.
> They may do as a temporary workaround, but not as a solution.

Agreed.  I suppose not many people have bumped into this problem in
the wild, since it's been in mainline since 4.17.  Am I really the
only person running a development

Re: [Xen-devel] [PATCH for-4.12 v2 4/7] pvh/dom0: warn when dom0_mem is not set

2019-02-13 Thread Jan Beulich 
>>> On 11.02.19 at 18:46,  wrote:
> There have been several reports of the dom0 builder running out of
> memory when building a PVH dom0 without having specified a dom0_mem
> value. Print a warning message if dom0_mem is not set when booting in
> PVH mode.
> 
> This is a temporary workaround until accounting for internal memory
> required by Xen (ie: paging structures) is improved.
> 
> Signed-off-by: Roger Pau Monné 

Acked-by: Jan Beulich 

I take it that ...

> --- a/xen/arch/x86/dom0_build.c
> +++ b/xen/arch/x86/dom0_build.c
> @@ -378,8 +378,18 @@ unsigned long __init dom0_compute_nr_pages(
>   * maximum of 128MB.
>   */
>  if ( !nr_pages )
> +{
>  nr_pages = avail - (pv_shim ? pv_shim_mem(avail)
>   : min(avail / 16, 128UL << (20 - 
> PAGE_SHIFT)));
> +if ( is_hvm_domain(d) )
> +/*
> + * Temporary workaround message until internal (paging) 
> memory
> + * accounting required to build a pvh dom0 is improved.
> + */
> +printk("WARNING: PVH dom0 without dom0_mem set is still 
> unstable. "
> +   "If you get crashes during boot, try adding a 
> dom0_mem parameter\n");
> +}

... you consider it acceptable for the message to be logged twice
in certain cases?

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-4.12 v2 3/7] x86/pvh: reorder PVH dom0 iommu initialization

2019-02-13 Thread Jan Beulich 
>>> On 11.02.19 at 18:46,  wrote:
> So that the iommu is initialized before populating the p2m, and
> entries added get the corresponding iommu page table entries if
> required. This requires splitting the current pvh_setup_p2m into two
> different functions. One that crafts dom0 physmap and sets the paging
> allocation, and another one that actually populates the p2m with RAM
> regions.
> 
> Note that this allows to remove the special casing done for the low
> 1MB in hwdom_iommu_map.
> 
> Signed-off-by: Roger Pau Monné 

Reviewed-by: Jan Beulich 


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 1/2] amd-iommu: use a bitfield for PTE/PDE

2019-02-13 Thread Paul Durrant 
> -Original Message-
> From: Xen-devel [mailto:xen-devel-boun...@lists.xenproject.org] On Behalf
> Of Woods, Brian
> Sent: 13 February 2019 15:34
> To: Paul Durrant ; xen-devel@lists.xenproject.org
> Cc: Andrew Cooper ; Wei Liu
> ; Jan Beulich ; Suthikulpanit,
> Suravee ; Roger Pau Monne
> 
> Subject: Re: [Xen-devel] [PATCH 1/2] amd-iommu: use a bitfield for PTE/PDE
> 
> On 2/13/19 3:45 AM, Paul Durrant wrote:
> >> -Original Message-
> >> From: Woods, Brian [mailto:brian.wo...@amd.com]
> >> Sent: 12 February 2019 20:14
> >> To: Paul Durrant ; xen-
> de...@lists.xenproject.org
> >> Cc: Suthikulpanit, Suravee ; Jan Beulich
> >> ; Andrew Cooper ; Wei Liu
> >> ; Roger Pau Monne 
> >> Subject: Re: [PATCH 1/2] amd-iommu: use a bitfield for PTE/PDE
> >>
> >> On 2/4/19 5:19 AM, Paul Durrant wrote:
> >>> The current use of get/set_field_from/in_reg_u32() is both inefficient
> >> and
> >>> requires some ugly casting.
> >>>
> >>> This patch defines a new bitfield structure (amd_iommu_pte) and uses
> >> this
> >>> structure in all PTE/PDE manipulation, resulting in much more readable
> >>> and compact code.
> >>>
> >>> NOTE: This commit also fixes one malformed comment in
> >>> set_iommu_pte_present().
> >>>
> >>> Signed-off-by: Paul Durrant 
> >>
> >> Sorry about the delay.
> >>
> >> Nitpick here, but I'd rather have !!IOMMUF_{writable,readable} than
> >> true.
> >
> > That's pretty ugly. How about I pass an OR of the flags through to lower
> level functions rather than a pair of bools? If you're ok with that I'll
> send a v2.
> >
> >Paul
> >
> 
> There's no need for a v2 based on that, that's just me nitpicking.
> There's no real nice way to do it without turning
> IOMMUF_{writable,readable} into bools or your suggested way which has
> more code to decode a flag.  Assuming everyone else is ok with the
> patches as are, it's fine.  If there's going to be a v2 for other
> reasons, I'll just leave it up to your discretion (other people may have
> stronger opinions about it anyway).
> 

Ok. Thanks Brian,

  Paul
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-4.12 v2 2/7] amd/npt/shadow: replace assert that prevents creating 2M/1G MMIO entries

2019-02-13 Thread Jan Beulich 
>>> On 11.02.19 at 18:46,  wrote:
> The assert was originally added to make sure that higher order
> regions (> PAGE_ORDER_4K) could not be used to bypass the
> mmio_ro_ranges check performed by p2m_type_to_flags.
> 
> This however is already checked in set_mmio_p2m_entry, which makes
> sure that higher order mappings don't overlap with mmio_ro_ranges,
> thus allowing the creation of high order MMIO mappings safely.
> 
> Replace the assert to allow 2M/1G entries to be created for MMIO
> regions and add some extra asserts as a replacement to make sure
> there's no overlapping with MMIO read-only ranges.
> 
> Note that 1G MMIO entries will not be created unless mmio_order is
> changed to allow it.
> 
> Suggested-by: George Dunlap 

Is this still the case? Iirc the original suggestion was to remove
the assertion altogether?

> --- a/xen/arch/x86/mm/p2m-pt.c
> +++ b/xen/arch/x86/mm/p2m-pt.c
> @@ -576,7 +576,15 @@ p2m_pt_set_entry(struct p2m_domain *p2m, gfn_t gfn_, 
> mfn_t mfn,
>  }
>  
>  ASSERT(p2m_flags_to_type(flags) != p2m_ioreq_server);
> -ASSERT(!mfn_valid(mfn) || p2mt != p2m_mmio_direct);
> +if ( p2mt == p2m_mmio_direct )
> +ASSERT(!mfn_eq(mfn, INVALID_MFN) &&
> +   !rangeset_overlaps_range(mmio_ro_ranges, mfn_x(mfn),
> +mfn_x(mfn) + PFN_DOWN(MB(2;
> +else if ( p2m_allows_invalid_mfn(p2mt) || p2mt == p2m_invalid ||
> +  p2mt == p2m_mmio_dm )
> +ASSERT(mfn_valid(mfn) || mfn_eq(mfn, INVALID_MFN));
> +else
> +ASSERT(mfn_valid(mfn));
>  l3e_content = mfn_valid(mfn) || p2m_allows_invalid_mfn(p2mt)
>  ? p2m_l3e_from_pfn(mfn_x(mfn),
> p2m_type_to_flags(p2m, p2mt, mfn, 2))
> @@ -668,7 +676,15 @@ p2m_pt_set_entry(struct p2m_domain *p2m, gfn_t gfn_, 
> mfn_t mfn,
>  }
>  
>  ASSERT(p2m_flags_to_type(flags) != p2m_ioreq_server);
> -ASSERT(!mfn_valid(mfn) || p2mt != p2m_mmio_direct);
> +if ( p2mt == p2m_mmio_direct )
> +ASSERT(!mfn_eq(mfn, INVALID_MFN) &&
> +   !rangeset_overlaps_range(mmio_ro_ranges, mfn_x(mfn),
> +mfn_x(mfn) + PFN_DOWN(MB(2;
> +else if ( p2m_allows_invalid_mfn(p2mt) || p2mt == p2m_invalid ||
> +  p2mt == p2m_mmio_dm )
> +ASSERT(mfn_valid(mfn) || mfn_eq(mfn, INVALID_MFN));
> +else
> +ASSERT(mfn_valid(mfn));

Seeing this supposedly almost the same (but actually entirely the same,
due to the wrong MB(2) in the first hunk) code I wonder whether this
wouldn't better be put in a helper (macro or function), together with
adjacent assertion in context, immediately ahead of the line you alter.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 1/2] amd-iommu: use a bitfield for PTE/PDE

2019-02-13 Thread Woods, Brian 
On 2/13/19 3:45 AM, Paul Durrant wrote:
>> -Original Message-
>> From: Woods, Brian [mailto:brian.wo...@amd.com]
>> Sent: 12 February 2019 20:14
>> To: Paul Durrant ; xen-devel@lists.xenproject.org
>> Cc: Suthikulpanit, Suravee ; Jan Beulich
>> ; Andrew Cooper ; Wei Liu
>> ; Roger Pau Monne 
>> Subject: Re: [PATCH 1/2] amd-iommu: use a bitfield for PTE/PDE
>>
>> On 2/4/19 5:19 AM, Paul Durrant wrote:
>>> The current use of get/set_field_from/in_reg_u32() is both inefficient
>> and
>>> requires some ugly casting.
>>>
>>> This patch defines a new bitfield structure (amd_iommu_pte) and uses
>> this
>>> structure in all PTE/PDE manipulation, resulting in much more readable
>>> and compact code.
>>>
>>> NOTE: This commit also fixes one malformed comment in
>>> set_iommu_pte_present().
>>>
>>> Signed-off-by: Paul Durrant 
>>
>> Sorry about the delay.
>>
>> Nitpick here, but I'd rather have !!IOMMUF_{writable,readable} than
>> true.
> 
> That's pretty ugly. How about I pass an OR of the flags through to lower 
> level functions rather than a pair of bools? If you're ok with that I'll send 
> a v2.
> 
>Paul
> 

There's no need for a v2 based on that, that's just me nitpicking. 
There's no real nice way to do it without turning 
IOMMUF_{writable,readable} into bools or your suggested way which has 
more code to decode a flag.  Assuming everyone else is ok with the 
patches as are, it's fine.  If there's going to be a v2 for other 
reasons, I'll just leave it up to your discretion (other people may have 
stronger opinions about it anyway).

Brian

>>   Not worth a revision if there isn't anything else though (and is
>> debatable).
>>
>> Acked-by: Brian Woods 
>>
>>> ---
>>> Cc: Suravee Suthikulpanit 
>>> Cc: Brian Woods 
>>> Cc: Jan Beulich 
>>> Cc: Andrew Cooper 
>>> Cc: Wei Liu 
>>> Cc: "Roger Pau Monné" 
>>> ---
>>>xen/drivers/passthrough/amd/iommu_map.c   | 143 --
>>>xen/drivers/passthrough/amd/pci_amd_iommu.c   |  50 +++---
>>>xen/include/asm-x86/hvm/svm/amd-iommu-defs.h  |  47 ++
>>>xen/include/asm-x86/hvm/svm/amd-iommu-proto.h |  15 --
>>>4 files changed, 64 insertions(+), 191 deletions(-)
>>>
>>> diff --git a/xen/drivers/passthrough/amd/iommu_map.c
>> b/xen/drivers/passthrough/amd/iommu_map.c
>>> index 67329b0c95..5fda6063df 100644
>>> --- a/xen/drivers/passthrough/amd/iommu_map.c
>>> +++ b/xen/drivers/passthrough/amd/iommu_map.c
>>> @@ -38,100 +38,45 @@ static unsigned int pfn_to_pde_idx(unsigned long
>> pfn, unsigned int level)
>>>static unsigned int clear_iommu_pte_present(unsigned long l1_mfn,
>>>unsigned long dfn)
>>>{
>>> -uint64_t *table, *pte;
>>> +struct amd_iommu_pte *table, *pte;
>>>unsigned int flush_flags;
>>>
>>>table = map_domain_page(_mfn(l1_mfn));
>>> +pte = [pfn_to_pde_idx(dfn, 1)];
>>>
>>> -pte = (table + pfn_to_pde_idx(dfn, 1));
>>> +flush_flags = pte->pr ? IOMMU_FLUSHF_modified : 0;
>>> +memset(pte, 0, sizeof(*pte));
>>>
>>> -flush_flags = get_field_from_reg_u32(*pte, IOMMU_PTE_PRESENT_MASK,
>>> - IOMMU_PTE_PRESENT_SHIFT) ?
>>> - IOMMU_FLUSHF_modified : 0;
>>> -
>>> -*pte = 0;
>>>unmap_domain_page(table);
>>>
>>>return flush_flags;
>>>}
>>>
>>> -static unsigned int set_iommu_pde_present(uint32_t *pde,
>>> +static unsigned int set_iommu_pde_present(struct amd_iommu_pte *pte,
>>>  unsigned long next_mfn,
>>>  unsigned int next_level,
>> bool iw,
>>>  bool ir)
>>>{
>>> -uint64_t maddr_next;
>>> -uint32_t addr_lo, addr_hi, entry;
>>> -bool old_present;
>>>unsigned int flush_flags = IOMMU_FLUSHF_added;
>>>
>>> -maddr_next = __pfn_to_paddr(next_mfn);
>>> -
>>> -old_present = get_field_from_reg_u32(pde[0],
>> IOMMU_PTE_PRESENT_MASK,
>>> - IOMMU_PTE_PRESENT_SHIFT);
>>> -if ( old_present )
>>> -{
>>> -bool old_r, old_w;
>>> -unsigned int old_level;
>>> -uint64_t maddr_old;
>>> -
>>> -addr_hi = get_field_from_reg_u32(pde[1],
>>> - IOMMU_PTE_ADDR_HIGH_MASK,
>>> - IOMMU_PTE_ADDR_HIGH_SHIFT);
>>> -addr_lo = get_field_from_reg_u32(pde[0],
>>> - IOMMU_PTE_ADDR_LOW_MASK,
>>> - IOMMU_PTE_ADDR_LOW_SHIFT);
>>> -old_level = get_field_from_reg_u32(pde[0],
>>> -   IOMMU_PDE_NEXT_LEVEL_MASK,
>>> -   IOMMU_PDE_NEXT_LEVEL_SHIFT);
>>> -old_w = get_field_from_reg_u32(pde[1],
>>> -
>> IOMMU_PTE_IO_WRITE_PERMISSION_MASK,
>>> -
>>

Re: [Xen-devel] [PATCH for-4.12 v2 1/7] dom0/pvh: align allocation and mapping order to start address

2019-02-13 Thread Jan Beulich 
>>> On 11.02.19 at 18:46,  wrote:
> The p2m and iommu mapping code always had the requirement that
> addresses and orders must be aligned when populating the p2m or the
> iommu page tables.
> 
> PVH dom0 builder didn't take this requirement into account, and can
> call into the p2m/iommu mapping helpers with addresses and orders that
> are not aligned.
> 
> Fix this by making sure the orders passed to the physmap population
> helpers are always aligned to the guest address to be populated.
> 
> Signed-off-by: Roger Pau Monné 

Reviewed-by: Jan Beulich 


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] vm_event: Add a new opcode to get VM_EVENT_INTERFACE_VERSION

2019-02-13 Thread Jan Beulich 
>>> On 13.02.19 at 14:25,  wrote:
> @@ -592,6 +592,19 @@ int vm_event_domctl(struct domain *d, struct 
> xen_domctl_vm_event_op *vec,
>  {
>  int rc;
>  
> +if ( vec->op == XEN_VM_EVENT_GET_INTERFACE_VERSION )
> +{
> +vec->u.get_interface_version.value = VM_EVENT_INTERFACE_VERSION;
> +return 0;
> +}
> +
> +if ( unlikely(d == NULL) )
> +{
> +gdprintk(XENLOG_INFO,
> + "Tried to do a memory event op on an invalid domain.\n");
> +return -EINVAL;
> +}

To be compatible with previous behavior you want to return
-ESRCH here. I'm also unconvinced of the need to add a log
message here - there was none before in that case. But I'm
not a maintainer of this code.

> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -778,9 +778,10 @@ struct xen_domctl_gdbsx_domstatus {
>   * to the hypervisor to pull responses (resume) from the given
>   * ring.
>   */
> -#define XEN_VM_EVENT_ENABLE   0
> -#define XEN_VM_EVENT_DISABLE  1
> -#define XEN_VM_EVENT_RESUME   2
> +#define XEN_VM_EVENT_ENABLE 0
> +#define XEN_VM_EVENT_DISABLE1
> +#define XEN_VM_EVENT_RESUME 2
> +#define XEN_VM_EVENT_GET_INTERFACE_VERSION  3

Perhaps just XEN_VM_EVENT_GET_VERSION?

> @@ -843,7 +844,15 @@ struct xen_domctl_vm_event_op {
>  uint32_t   op;   /* XEN_VM_EVENT_* */
>  uint32_t   mode; /* XEN_DOMCTL_VM_EVENT_OP_* */
>  
> -uint32_t port;  /* OUT: event channel for ring */
> +union {
> +struct {
> +uint32_t port;   /* OUT: event channel for ring */
> +} enable;
> +
> +struct {
> +uint32_t value;
> +} get_interface_version;

Why the wrapper structs? Having just a "port" and "version"
field inside the union would be good enough, wouldn't it? But
even if you want to stick to that, the new structure's name
could be simply "version", thus also allowing re-use for a
hypothetical "set-version" operation (in case multiple versions
would want/need to be supported going forward).

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-unstable-smoke test] 133229: trouble: blocked/broken/pass

2019-02-13 Thread osstest service owner 
flight 133229 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133229/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken

Regressions which are regarded as allowable (not blocking):
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 133005

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64-xsm   3 capture-logs  broken blocked in 133005
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  f178a00c30173c0b268d99160e19ad299b1823a2
baseline version:
 xen  455301716e1ff358cb79367213003fba771dd466

Last test of basis   133005  2019-02-07 14:00:32 Z6 days
Failing since133011  2019-02-07 18:00:36 Z5 days   32 attempts
Testing same since   133200  2019-02-12 16:00:56 Z0 days9 attempts


People who touched revisions under test:
  Andrew Cooper 
  George Dunlap 
  Ian Jackson 
  Jan Beulich 
  Juergen Gross 
  Kevin Tian 
  Michael Tautschnig 
  Norbert Manthey 
  Norbert Manthey 
  Stefano Stabellini 
  Stefano Stabellini 
  Tim Deegan 
  Wei Liu 

jobs:
 build-arm64-xsm  broken  
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64-xsm broken
broken-step build-arm64-xsm hosts-allocate
broken-step build-arm64-xsm capture-logs

Not pushing.


commit f178a00c30173c0b268d99160e19ad299b1823a2
Author: Norbert Manthey 
Date:   Tue Feb 12 15:20:15 2019 +0100

x86/hvm: block speculative out-of-bound accesses

There are multiple arrays in the HVM interface that are accessed
with indices that are provided by the guest. To avoid speculative
out-of-bound accesses, we use the array_index_nospec macro.

When blocking speculative out-of-bound accesses, we can classify arrays
into dynamic arrays and static arrays. Where the former are allocated
during run time, the size of the latter is known during compile time.
On static arrays, compiler might be able to block speculative accesses
in the future.

This is part of the speculative hardening effort.

Reported-by: Pawel Wieczorkiewicz 
Signed-off-by: Norbert Manthey 
Reviewed-by: Jan Beulich 
Release-acked-by: Juergen Gross 

commit 56d8d0119d270f846c6c4943712b8a21fbe5d4d0
Author: Jan Beulich 
Date:   Tue Feb 12 11:54:57 2019 +0100

VMX: don't ignore P2M setup error

set_mmio_p2m_entry() may fail, in particular with -ENOMEM. Don't ignore
such an error, but instead cause domain creation to fail in such a case.

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 
Acked-by: Kevin Tian 
Release-acked-by: Juergen Gross 

commit 88b92c3820cffed4b4abeb139edc2cbd8286cb12
Author: Juergen Gross 
Date:   Tue Feb 12 11:54:07 2019 +0100

iommu: fix iommu_ops initialization

Commit 32a5ea00ec75ef53e ("IOMMU/x86: remove indirection from certain
IOMMU hook accesses") introduced iommu_ops initialized at boot time
with data declared as __initconstrel.

On Intel systems there is another path where iommu_ops is initialized
and this path is relevant on resume after returning from system suspend.
As the initialization data is no longer accessible in this case that
second initialization must be dropped in case the system isn't just
booting.

Signed-off-by: Juergen Gross 
Reviewed-by: Jan Beulich

Re: [Xen-devel] Upstream Dom0 DRM problems regarding swiotlb

2019-02-13 Thread Jan Beulich 
>>> On 13.02.19 at 15:10,  wrote:
> On Wed, Feb 13, 2019 at 5:34 AM Jan Beulich  wrote:
>>
>> >>> On 12.02.19 at 19:46,  wrote:
>> > Konrad,
>> >
>> > Starting w/ v4.17, I cannot log in to GNOME w/out getting the
>> > following mess in dmesg and ending up back at the GDM login screen.
>> >
>> > [   28.554259] radeon_dp_aux_transfer_native: 200 callbacks suppressed
>> > [   31.219821] radeon :01:00.0: swiotlb buffer is full (sz: 2097152
>> > bytes)
>> > [   31.220030] [drm:radeon_gem_object_create [radeon]] *ERROR* Failed
>> > to allocate GEM object (16384000, 2, 4096, -14)
>> > [   31.226109] radeon :01:00.0: swiotlb buffer is full (sz: 2097152
>> > bytes)
>> > [   31.226300] [drm:radeon_gem_object_create [radeon]] *ERROR* Failed
>> > to allocate GEM object (16384000, 2, 4096, -14)
>> > [   31.300734] gnome-shell[1935]: segfault at 88 ip 7f39151cd904
>> > sp 7ffc97611ad8 error 4 in libmutter-cogl.so[7f3915178000+aa000]
>> > [   31.300745] Code: 5f c3 0f 1f 40 00 48 8b 47 78 48 8b 40 40 ff e0
>> > 66 0f 1f 44 00 00 48 8b 47 78 48 8b 40 48 ff e0 66 0f 1f 44 00 00 48
>> > 8b 47 78 <48> 8b 80 88 00 00 00 ff e0 0f 1f 00 48 8b 47 78 48 8b 40 68
>> > ff e0
>> > [   38.193302] radeon_dp_aux_transfer_native: 116 callbacks suppressed
>> > [   40.009317] radeon :01:00.0: swiotlb buffer is full (sz: 2097152
>> > bytes)
>> > [   40.009488] [drm:radeon_gem_object_create [radeon]] *ERROR* Failed
>> > to allocate GEM object (16384000, 2, 4096, -14)
>> > [   40.015114] radeon :01:00.0: swiotlb buffer is full (sz: 2097152
>> > bytes)
>> > [   40.015297] [drm:radeon_gem_object_create [radeon]] *ERROR* Failed
>> > to allocate GEM object (16384000, 2, 4096, -14)
>> > [   40.028302] gnome-shell[2431]: segfault at 2dadf40 ip
>> > 02dadf40 sp 7ffcd24ea5f8 error 15
>> > [   40.028306] Code: 20 6e 31 00 00 00 00 00 00 00 00 37 e3 3d 2d 7f
>> > 00 00 80 f4 e6 3d 2d 7f 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> > 00 00 00 <00> 00 00 00 00 00 00 00 c1 00 00 00 00 00 00 00 80 e1 d2 03
>> > 00 00
>> >
>> >
>> > This happens w/ both radeon and amdgpu.
>> >
>> > I bisected down to the following range of commits, which basically add
>> > conditional code to radeon and amdgpu to NOT use swiotlb if dma_bits
>> > is smaller than the system's max iomem address...  but that very much
>> > doesn't work on a Xen dom0.
>>
>> Well, not so much a Xen Dom0, but a Xen PV domain.
>>
>> > 82626363 drm: add func to get max iomem address v2
>> > fd5fd480 drm/amdgpu: only enable swiotlb alloc when need v2
>> > 1bc3d3cc drm/radeon: only enable swiotlb path when need v2
>> >
>> > Reverting the offending commits gives me a usable v4.20 dom0 kernel w/
>> > working 3d support.  Not sure what the appropriate upstream fix for
>> > this would be, as I don't 100% understand this.  Could you enlighten
>> > me?  ;-)
>>
>> Well, this depends on how much abstraction we want, and how
>> much abstraction the maintainers of the DRM drivers demand.
>> It could be as simple as adding xen_swiotlb checks into the
>> conditionals setting ->need_swiotlb, but in an abstract sense
>> the issue of course exists for PV guests of any hypervisor.
>> (Altering drm_get_max_iomem() itself would seem wrong to me,
>> unless its name was also changed.)
> 
> Ah, so this isn't necessarily Xen-specific but rather any paravirtual
> guest?  That hadn't crossed my mind.  Is there an easy way to find out
> if we're a pv guest in the need_swiotlb conditionals?

There's xen_pv_domain(), but I think xen_swiotlb would be more to
the point if the check is already to be Xen-specific. There's no generic
"is PV" predicate that I'm aware of.

>  If not, we
> should at least add a module parameter to force swiotlb usage to both
> radeon and amdgpu.  I'd be more than happy to gin up a patch to do
> either and submit to upstream (dri-devel, I guess).

I don't think module parameters are a good way forward here.
They may do as a temporary workaround, but not as a solution.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Upstream Dom0 DRM problems regarding swiotlb

2019-02-13 Thread Michael Labriola 
On Wed, Feb 13, 2019 at 5:34 AM Jan Beulich  wrote:
>
> >>> On 12.02.19 at 19:46,  wrote:
> > Konrad,
> >
> > Starting w/ v4.17, I cannot log in to GNOME w/out getting the
> > following mess in dmesg and ending up back at the GDM login screen.
> >
> > [   28.554259] radeon_dp_aux_transfer_native: 200 callbacks suppressed
> > [   31.219821] radeon :01:00.0: swiotlb buffer is full (sz: 2097152
> > bytes)
> > [   31.220030] [drm:radeon_gem_object_create [radeon]] *ERROR* Failed
> > to allocate GEM object (16384000, 2, 4096, -14)
> > [   31.226109] radeon :01:00.0: swiotlb buffer is full (sz: 2097152
> > bytes)
> > [   31.226300] [drm:radeon_gem_object_create [radeon]] *ERROR* Failed
> > to allocate GEM object (16384000, 2, 4096, -14)
> > [   31.300734] gnome-shell[1935]: segfault at 88 ip 7f39151cd904
> > sp 7ffc97611ad8 error 4 in libmutter-cogl.so[7f3915178000+aa000]
> > [   31.300745] Code: 5f c3 0f 1f 40 00 48 8b 47 78 48 8b 40 40 ff e0
> > 66 0f 1f 44 00 00 48 8b 47 78 48 8b 40 48 ff e0 66 0f 1f 44 00 00 48
> > 8b 47 78 <48> 8b 80 88 00 00 00 ff e0 0f 1f 00 48 8b 47 78 48 8b 40 68
> > ff e0
> > [   38.193302] radeon_dp_aux_transfer_native: 116 callbacks suppressed
> > [   40.009317] radeon :01:00.0: swiotlb buffer is full (sz: 2097152
> > bytes)
> > [   40.009488] [drm:radeon_gem_object_create [radeon]] *ERROR* Failed
> > to allocate GEM object (16384000, 2, 4096, -14)
> > [   40.015114] radeon :01:00.0: swiotlb buffer is full (sz: 2097152
> > bytes)
> > [   40.015297] [drm:radeon_gem_object_create [radeon]] *ERROR* Failed
> > to allocate GEM object (16384000, 2, 4096, -14)
> > [   40.028302] gnome-shell[2431]: segfault at 2dadf40 ip
> > 02dadf40 sp 7ffcd24ea5f8 error 15
> > [   40.028306] Code: 20 6e 31 00 00 00 00 00 00 00 00 37 e3 3d 2d 7f
> > 00 00 80 f4 e6 3d 2d 7f 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > 00 00 00 <00> 00 00 00 00 00 00 00 c1 00 00 00 00 00 00 00 80 e1 d2 03
> > 00 00
> >
> >
> > This happens w/ both radeon and amdgpu.
> >
> > I bisected down to the following range of commits, which basically add
> > conditional code to radeon and amdgpu to NOT use swiotlb if dma_bits
> > is smaller than the system's max iomem address...  but that very much
> > doesn't work on a Xen dom0.
>
> Well, not so much a Xen Dom0, but a Xen PV domain.
>
> > 82626363 drm: add func to get max iomem address v2
> > fd5fd480 drm/amdgpu: only enable swiotlb alloc when need v2
> > 1bc3d3cc drm/radeon: only enable swiotlb path when need v2
> >
> > Reverting the offending commits gives me a usable v4.20 dom0 kernel w/
> > working 3d support.  Not sure what the appropriate upstream fix for
> > this would be, as I don't 100% understand this.  Could you enlighten
> > me?  ;-)
>
> Well, this depends on how much abstraction we want, and how
> much abstraction the maintainers of the DRM drivers demand.
> It could be as simple as adding xen_swiotlb checks into the
> conditionals setting ->need_swiotlb, but in an abstract sense
> the issue of course exists for PV guests of any hypervisor.
> (Altering drm_get_max_iomem() itself would seem wrong to me,
> unless its name was also changed.)

Ah, so this isn't necessarily Xen-specific but rather any paravirtual
guest?  That hadn't crossed my mind.  Is there an easy way to find out
if we're a pv guest in the need_swiotlb conditionals?  If not, we
should at least add a module parameter to force swiotlb usage to both
radeon and amdgpu.  I'd be more than happy to gin up a patch to do
either and submit to upstream (dri-devel, I guess).

Thanks!

-Mike

-- 
Michael D Labriola
21 Rip Van Winkle Cir
Warwick, RI 02886
401-316-9844 (cell)
401-848-8871 (work)
401-234-1306 (home)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 8/9] xen/gntdev.c: Convert to use vm_map_pages()

2019-02-13 Thread Souptick Joarder 
Convert to use vm_map_pages() to map range of kernel
memory to user vma.

map->count is passed to vm_map_pages() and internal API
verify map->count against count ( count = vma_pages(vma))
for page array boundary overrun. With this count is not
needed inside gntdev_mmap() and it could be replaced with
vma_pages(vma).

Signed-off-by: Souptick Joarder 
Reviewed-by: Boris Ostrovsky 
---
 drivers/xen/gntdev.c | 16 ++--
 1 file changed, 6 insertions(+), 10 deletions(-)

diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c
index 5efc5ee..7f65ba3 100644
--- a/drivers/xen/gntdev.c
+++ b/drivers/xen/gntdev.c
@@ -1082,18 +1082,17 @@ static int gntdev_mmap(struct file *flip, struct 
vm_area_struct *vma)
 {
struct gntdev_priv *priv = flip->private_data;
int index = vma->vm_pgoff;
-   int count = vma_pages(vma);
struct gntdev_grant_map *map;
-   int i, err = -EINVAL;
+   int err = -EINVAL;
 
if ((vma->vm_flags & VM_WRITE) && !(vma->vm_flags & VM_SHARED))
return -EINVAL;
 
pr_debug("map %d+%d at %lx (pgoff %lx)\n",
-   index, count, vma->vm_start, vma->vm_pgoff);
+   index, vma_pages(vma), vma->vm_start, vma->vm_pgoff);
 
mutex_lock(>lock);
-   map = gntdev_find_map_index(priv, index, count);
+   map = gntdev_find_map_index(priv, index, vma_pages(vma));
if (!map)
goto unlock_out;
if (use_ptemod && map->vma)
@@ -1145,12 +1144,9 @@ static int gntdev_mmap(struct file *flip, struct 
vm_area_struct *vma)
goto out_put_map;
 
if (!use_ptemod) {
-   for (i = 0; i < count; i++) {
-   err = vm_insert_page(vma, vma->vm_start + i*PAGE_SIZE,
-   map->pages[i]);
-   if (err)
-   goto out_put_map;
-   }
+   err = vm_map_pages(vma, map->pages, map->count);
+   if (err)
+   goto out_put_map;
} else {
 #ifdef CONFIG_X86
/*
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 9/9] xen/privcmd-buf.c: Convert to use vm_map_pages_zero()

2019-02-13 Thread Souptick Joarder 
Convert to use vm_map_pages_zero() to map range of kernel
memory to user vma.

This driver has ignored vm_pgoff. We could later "fix" these drivers
to behave according to the normal vm_pgoff offsetting simply by
removing the _zero suffix on the function name and if that causes
regressions, it gives us an easy way to revert.

Signed-off-by: Souptick Joarder 
Reviewed-by: Boris Ostrovsky 
---
 drivers/xen/privcmd-buf.c | 8 ++--
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/drivers/xen/privcmd-buf.c b/drivers/xen/privcmd-buf.c
index de01a6d..d02dc43 100644
--- a/drivers/xen/privcmd-buf.c
+++ b/drivers/xen/privcmd-buf.c
@@ -166,12 +166,8 @@ static int privcmd_buf_mmap(struct file *file, struct 
vm_area_struct *vma)
if (vma_priv->n_pages != count)
ret = -ENOMEM;
else
-   for (i = 0; i < vma_priv->n_pages; i++) {
-   ret = vm_insert_page(vma, vma->vm_start + i * PAGE_SIZE,
-vma_priv->pages[i]);
-   if (ret)
-   break;
-   }
+   ret = vm_map_pages_zero(vma, vma_priv->pages,
+   vma_priv->n_pages);
 
if (ret)
privcmd_buf_vmapriv_free(vma_priv);
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 5/9] drm/xen/xen_drm_front_gem.c: Convert to use vm_map_pages()

2019-02-13 Thread Souptick Joarder 
Convert to use vm_map_pages() to map range of kernel
memory to user vma.

Signed-off-by: Souptick Joarder 
Reviewed-by: Oleksandr Andrushchenko 
---
 drivers/gpu/drm/xen/xen_drm_front_gem.c | 18 +-
 1 file changed, 5 insertions(+), 13 deletions(-)

diff --git a/drivers/gpu/drm/xen/xen_drm_front_gem.c 
b/drivers/gpu/drm/xen/xen_drm_front_gem.c
index 28bc501..dd0602d 100644
--- a/drivers/gpu/drm/xen/xen_drm_front_gem.c
+++ b/drivers/gpu/drm/xen/xen_drm_front_gem.c
@@ -224,8 +224,7 @@ struct drm_gem_object *
 static int gem_mmap_obj(struct xen_gem_object *xen_obj,
struct vm_area_struct *vma)
 {
-   unsigned long addr = vma->vm_start;
-   int i;
+   int ret;
 
/*
 * clear the VM_PFNMAP flag that was set by drm_gem_mmap(), and set the
@@ -246,18 +245,11 @@ static int gem_mmap_obj(struct xen_gem_object *xen_obj,
 * FIXME: as we insert all the pages now then no .fault handler must
 * be called, so don't provide one
 */
-   for (i = 0; i < xen_obj->num_pages; i++) {
-   int ret;
-
-   ret = vm_insert_page(vma, addr, xen_obj->pages[i]);
-   if (ret < 0) {
-   DRM_ERROR("Failed to insert pages into vma: %d\n", ret);
-   return ret;
-   }
+   ret = vm_map_pages(vma, xen_obj->pages, xen_obj->num_pages);
+   if (ret < 0)
+   DRM_ERROR("Failed to map pages into vma: %d\n", ret);
 
-   addr += PAGE_SIZE;
-   }
-   return 0;
+   return ret;
 }
 
 int xen_drm_front_gem_mmap(struct file *filp, struct vm_area_struct *vma)
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 1/9] mm: Introduce new vm_map_pages() and vm_map_pages_zero() API

2019-02-13 Thread Souptick Joarder 
Previouly drivers have their own way of mapping range of
kernel pages/memory into user vma and this was done by
invoking vm_insert_page() within a loop.

As this pattern is common across different drivers, it can
be generalized by creating new functions and use it across
the drivers.

vm_map_pages() is the API which could be used to mapped
kernel memory/pages in drivers which has considered vm_pgoff

vm_map_pages_zero() is the API which could be used to map
range of kernel memory/pages in drivers which has not considered
vm_pgoff. vm_pgoff is passed default as 0 for those drivers.

We _could_ then at a later "fix" these drivers which are using
vm_map_pages_zero() to behave according to the normal vm_pgoff
offsetting simply by removing the _zero suffix on the function
name and if that causes regressions, it gives us an easy way to revert.

Tested on Rockchip hardware and display is working, including talking
to Lima via prime.

Signed-off-by: Souptick Joarder 
Suggested-by: Russell King 
Suggested-by: Matthew Wilcox 
Reviewed-by: Mike Rapoport 
Tested-by: Heiko Stuebner 
---
 include/linux/mm.h |  4 +++
 mm/memory.c| 81 ++
 mm/nommu.c | 14 ++
 3 files changed, 99 insertions(+)

diff --git a/include/linux/mm.h b/include/linux/mm.h
index 80bb640..e0aaa73 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -2565,6 +2565,10 @@ unsigned long change_prot_numa(struct vm_area_struct 
*vma,
 int remap_pfn_range(struct vm_area_struct *, unsigned long addr,
unsigned long pfn, unsigned long size, pgprot_t);
 int vm_insert_page(struct vm_area_struct *, unsigned long addr, struct page *);
+int vm_map_pages(struct vm_area_struct *vma, struct page **pages,
+   unsigned long num);
+int vm_map_pages_zero(struct vm_area_struct *vma, struct page **pages,
+   unsigned long num);
 vm_fault_t vmf_insert_pfn(struct vm_area_struct *vma, unsigned long addr,
unsigned long pfn);
 vm_fault_t vmf_insert_pfn_prot(struct vm_area_struct *vma, unsigned long addr,
diff --git a/mm/memory.c b/mm/memory.c
index e11ca9d..cad3e27 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -1520,6 +1520,87 @@ int vm_insert_page(struct vm_area_struct *vma, unsigned 
long addr,
 }
 EXPORT_SYMBOL(vm_insert_page);
 
+/*
+ * __vm_map_pages - maps range of kernel pages into user vma
+ * @vma: user vma to map to
+ * @pages: pointer to array of source kernel pages
+ * @num: number of pages in page array
+ * @offset: user's requested vm_pgoff
+ *
+ * This allows drivers to map range of kernel pages into a user vma.
+ *
+ * Return: 0 on success and error code otherwise.
+ */
+static int __vm_map_pages(struct vm_area_struct *vma, struct page **pages,
+   unsigned long num, unsigned long offset)
+{
+   unsigned long count = vma_pages(vma);
+   unsigned long uaddr = vma->vm_start;
+   int ret, i;
+
+   /* Fail if the user requested offset is beyond the end of the object */
+   if (offset > num)
+   return -ENXIO;
+
+   /* Fail if the user requested size exceeds available object size */
+   if (count > num - offset)
+   return -ENXIO;
+
+   for (i = 0; i < count; i++) {
+   ret = vm_insert_page(vma, uaddr, pages[offset + i]);
+   if (ret < 0)
+   return ret;
+   uaddr += PAGE_SIZE;
+   }
+
+   return 0;
+}
+
+/**
+ * vm_map_pages - maps range of kernel pages starts with non zero offset
+ * @vma: user vma to map to
+ * @pages: pointer to array of source kernel pages
+ * @num: number of pages in page array
+ *
+ * Maps an object consisting of @num pages, catering for the user's
+ * requested vm_pgoff
+ *
+ * If we fail to insert any page into the vma, the function will return
+ * immediately leaving any previously inserted pages present.  Callers
+ * from the mmap handler may immediately return the error as their caller
+ * will destroy the vma, removing any successfully inserted pages. Other
+ * callers should make their own arrangements for calling unmap_region().
+ *
+ * Context: Process context. Called by mmap handlers.
+ * Return: 0 on success and error code otherwise.
+ */
+int vm_map_pages(struct vm_area_struct *vma, struct page **pages,
+   unsigned long num)
+{
+   return __vm_map_pages(vma, pages, num, vma->vm_pgoff);
+}
+EXPORT_SYMBOL(vm_map_pages);
+
+/**
+ * vm_map_pages_zero - map range of kernel pages starts with zero offset
+ * @vma: user vma to map to
+ * @pages: pointer to array of source kernel pages
+ * @num: number of pages in page array
+ *
+ * Similar to vm_map_pages(), except that it explicitly sets the offset
+ * to 0. This function is intended for the drivers that did not consider
+ * vm_pgoff.
+ *
+ * Context: Process context. Called by mmap handlers.
+ * Return: 0 on success and error

[Xen-devel] [PATCH v3 0/9] mm: Use vm_map_pages() and vm_map_pages_zero() API

2019-02-13 Thread Souptick Joarder 
Previouly drivers have their own way of mapping range of
kernel pages/memory into user vma and this was done by
invoking vm_insert_page() within a loop.

As this pattern is common across different drivers, it can
be generalized by creating new functions and use it across
the drivers.

vm_map_pages() is the API which could be used to map
kernel memory/pages in drivers which has considered vm_pgoff.

vm_map_pages_zero() is the API which could be used to map
range of kernel memory/pages in drivers which has not considered
vm_pgoff. vm_pgoff is passed default as 0 for those drivers.

We _could_ then at a later "fix" these drivers which are using
vm_map_pages_zero() to behave according to the normal vm_pgoff
offsetting simply by removing the _zero suffix on the function
name and if that causes regressions, it gives us an easy way to revert.

Tested on Rockchip hardware and display is working fine, including talking
to Lima via prime.

v1 -> v2:
Few Reviewed-by.

Updated the change log in [8/9]

In [7/9], vm_pgoff is treated in V4L2 API as a 'cookie'
to select a buffer, not as a in-buffer offset by design
and it always want to mmap a whole buffer from its beginning.
Added additional changes after discussing with Marek and
vm_map_pages() could be used instead of vm_map_pages_zero().

v2 -> v3:
Corrected the documentation as per review comment.

As suggested in v2, renaming the interfaces to -
*vm_insert_range() -> vm_map_pages()* and
*vm_insert_range_buggy() -> vm_map_pages_zero()*.
As the interface is renamed, modified the code accordingly,
updated the change logs and modified the subject lines to use the
new interfaces. There is no other change apart from renaming and
using the new interface.

Patch[1/9] & [4/9], Tested on Rockchip hardware.

Souptick Joarder (9):
  mm: Introduce new vm_map_pages() and vm_map_pages_zero() API
  arm: mm: dma-mapping: Convert to use vm_map_pages()
  drivers/firewire/core-iso.c: Convert to use vm_map_pages_zero()
  drm/rockchip/rockchip_drm_gem.c: Convert to use vm_map_pages()
  drm/xen/xen_drm_front_gem.c: Convert to use vm_map_pages()
  iommu/dma-iommu.c: Convert to use vm_map_pages()
  videobuf2/videobuf2-dma-sg.c: Convert to use vm_map_pages()
  xen/gntdev.c: Convert to use vm_map_pages()
  xen/privcmd-buf.c: Convert to use vm_map_pages_zero()

 arch/arm/mm/dma-mapping.c  | 22 ++
 drivers/firewire/core-iso.c| 15 +---
 drivers/gpu/drm/rockchip/rockchip_drm_gem.c| 17 +
 drivers/gpu/drm/xen/xen_drm_front_gem.c| 18 ++---
 drivers/iommu/dma-iommu.c  | 12 +---
 drivers/media/common/videobuf2/videobuf2-core.c|  7 ++
 .../media/common/videobuf2/videobuf2-dma-contig.c  |  6 --
 drivers/media/common/videobuf2/videobuf2-dma-sg.c  | 22 ++
 drivers/xen/gntdev.c   | 16 ++---
 drivers/xen/privcmd-buf.c  |  8 +--
 include/linux/mm.h |  4 ++
 mm/memory.c| 81 ++
 mm/nommu.c | 14 
 13 files changed, 136 insertions(+), 106 deletions(-)

-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [linux-4.4 test] 133160: trouble: blocked/broken/fail/pass

2019-02-13 Thread osstest service owner 
flight 133160 linux-4.4 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133160/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-pvopsbroken
 build-arm64  broken
 build-arm64-xsm  broken

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-examine  1 build-check(1)   blocked  n/a
 build-arm64-xsm   2 hosts-allocatebroken baseline untested
 build-arm64-pvops 2 hosts-allocatebroken baseline untested
 build-arm64   2 hosts-allocatebroken baseline untested
 build-arm64   3 capture-logs  broken baseline untested
 build-arm64-pvops 3 capture-logs  broken baseline untested
 build-arm64-xsm   3 capture-logs  broken baseline untested
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-amd64-xl-pvhv2-amd 12 guest-start  fail  never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvhv2-intel 12 guest-start fail never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop fail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-install

[Xen-devel] [PATCH] vm_event: Add a new opcode to get VM_EVENT_INTERFACE_VERSION

2019-02-13 Thread Petre Pircalabu 
Currently, the VM_EVENT_INTERFACE_VERSION is determined at runtime, by
inspecting the corresponding field in a vm_event_request. This helper
opcode will query the hypervisor supported version before the vm_event
related structures and layout are set-up.

Signed-off-by: Petre Pircalabu 
---
 tools/libxc/include/xenctrl.h |  5 +
 tools/libxc/xc_vm_event.c | 18 +-
 xen/common/domctl.c   |  1 +
 xen/common/vm_event.c | 15 ++-
 xen/include/public/domctl.h   | 17 +
 5 files changed, 50 insertions(+), 6 deletions(-)

diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 31cdda7..b801b85 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2003,6 +2003,11 @@ int xc_set_mem_access_multi(xc_interface *xch, uint32_t 
domain_id,
 int xc_get_mem_access(xc_interface *xch, uint32_t domain_id,
   uint64_t pfn, xenmem_access_t *access);
 
+/*
+ * Returns the VM_EVENT_INTERFACE version.
+ */
+int xc_vm_event_get_interface_version(xc_interface *xch);
+
 /***
  * Monitor control operations.
  *
diff --git a/tools/libxc/xc_vm_event.c b/tools/libxc/xc_vm_event.c
index 8674607..10ebc3c 100644
--- a/tools/libxc/xc_vm_event.c
+++ b/tools/libxc/xc_vm_event.c
@@ -35,7 +35,7 @@ int xc_vm_event_control(xc_interface *xch, uint32_t 
domain_id, unsigned int op,
 
 rc = do_domctl(xch, );
 if ( !rc && port )
-*port = domctl.u.vm_event_op.port;
+*port = domctl.u.vm_event_op.u.enable.port;
 return rc;
 }
 
@@ -156,6 +156,22 @@ void *xc_vm_event_enable(xc_interface *xch, uint32_t 
domain_id, int param,
 return ring_page;
 }
 
+int xc_vm_event_get_interface_version(xc_interface *xch)
+{
+DECLARE_DOMCTL;
+int rc;
+
+domctl.cmd = XEN_DOMCTL_vm_event_op;
+domctl.domain = DOMID_INVALID;
+domctl.u.vm_event_op.op = XEN_VM_EVENT_GET_INTERFACE_VERSION;
+domctl.u.vm_event_op.mode = XEN_DOMCTL_VM_EVENT_OP_MONITOR;
+
+rc = do_domctl(xch, );
+if ( !rc )
+rc = domctl.u.vm_event_op.u.get_interface_version.value;
+return rc;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/common/domctl.c b/xen/common/domctl.c
index d08b627..bade9a6 100644
--- a/xen/common/domctl.c
+++ b/xen/common/domctl.c
@@ -392,6 +392,7 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) 
u_domctl)
 switch ( op->cmd )
 {
 case XEN_DOMCTL_test_assign_device:
+case XEN_DOMCTL_vm_event_op:
 if ( op->domain == DOMID_INVALID )
 {
 case XEN_DOMCTL_createdomain:
diff --git a/xen/common/vm_event.c b/xen/common/vm_event.c
index 26cfa2c..cbdade3 100644
--- a/xen/common/vm_event.c
+++ b/xen/common/vm_event.c
@@ -88,7 +88,7 @@ static int vm_event_enable(
 if ( rc < 0 )
 goto err;
 
-(*ved)->xen_port = vec->port = rc;
+(*ved)->xen_port = vec->u.enable.port = rc;
 
 /* Prepare ring buffer */
 FRONT_RING_INIT(&(*ved)->front_ring,
@@ -592,6 +592,19 @@ int vm_event_domctl(struct domain *d, struct 
xen_domctl_vm_event_op *vec,
 {
 int rc;
 
+if ( vec->op == XEN_VM_EVENT_GET_INTERFACE_VERSION )
+{
+vec->u.get_interface_version.value = VM_EVENT_INTERFACE_VERSION;
+return 0;
+}
+
+if ( unlikely(d == NULL) )
+{
+gdprintk(XENLOG_INFO,
+ "Tried to do a memory event op on an invalid domain.\n");
+return -EINVAL;
+}
+
 rc = xsm_vm_event_control(XSM_PRIV, d, vec->mode, vec->op);
 if ( rc )
 return rc;
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 7e1cf21..f222d1e 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -778,9 +778,10 @@ struct xen_domctl_gdbsx_domstatus {
  * to the hypervisor to pull responses (resume) from the given
  * ring.
  */
-#define XEN_VM_EVENT_ENABLE   0
-#define XEN_VM_EVENT_DISABLE  1
-#define XEN_VM_EVENT_RESUME   2
+#define XEN_VM_EVENT_ENABLE 0
+#define XEN_VM_EVENT_DISABLE1
+#define XEN_VM_EVENT_RESUME 2
+#define XEN_VM_EVENT_GET_INTERFACE_VERSION  3
 
 /*
  * Domain memory paging
@@ -843,7 +844,15 @@ struct xen_domctl_vm_event_op {
 uint32_t   op;   /* XEN_VM_EVENT_* */
 uint32_t   mode; /* XEN_DOMCTL_VM_EVENT_OP_* */
 
-uint32_t port;  /* OUT: event channel for ring */
+union {
+struct {
+uint32_t port;   /* OUT: event channel for ring */
+} enable;
+
+struct {
+uint32_t value;
+} get_interface_version;
+} u;
 };
 
 /*
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-unstable-smoke test] 133227: trouble: blocked/broken/pass

2019-02-13 Thread osstest service owner 
flight 133227 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133227/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken

Regressions which are regarded as allowable (not blocking):
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 133005

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64-xsm   3 capture-logs  broken blocked in 133005
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  f178a00c30173c0b268d99160e19ad299b1823a2
baseline version:
 xen  455301716e1ff358cb79367213003fba771dd466

Last test of basis   133005  2019-02-07 14:00:32 Z5 days
Failing since133011  2019-02-07 18:00:36 Z5 days   31 attempts
Testing same since   133200  2019-02-12 16:00:56 Z0 days8 attempts


People who touched revisions under test:
  Andrew Cooper 
  George Dunlap 
  Ian Jackson 
  Jan Beulich 
  Juergen Gross 
  Kevin Tian 
  Michael Tautschnig 
  Norbert Manthey 
  Norbert Manthey 
  Stefano Stabellini 
  Stefano Stabellini 
  Tim Deegan 
  Wei Liu 

jobs:
 build-arm64-xsm  broken  
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64-xsm broken
broken-step build-arm64-xsm hosts-allocate
broken-step build-arm64-xsm capture-logs

Not pushing.


commit f178a00c30173c0b268d99160e19ad299b1823a2
Author: Norbert Manthey 
Date:   Tue Feb 12 15:20:15 2019 +0100

x86/hvm: block speculative out-of-bound accesses

There are multiple arrays in the HVM interface that are accessed
with indices that are provided by the guest. To avoid speculative
out-of-bound accesses, we use the array_index_nospec macro.

When blocking speculative out-of-bound accesses, we can classify arrays
into dynamic arrays and static arrays. Where the former are allocated
during run time, the size of the latter is known during compile time.
On static arrays, compiler might be able to block speculative accesses
in the future.

This is part of the speculative hardening effort.

Reported-by: Pawel Wieczorkiewicz 
Signed-off-by: Norbert Manthey 
Reviewed-by: Jan Beulich 
Release-acked-by: Juergen Gross 

commit 56d8d0119d270f846c6c4943712b8a21fbe5d4d0
Author: Jan Beulich 
Date:   Tue Feb 12 11:54:57 2019 +0100

VMX: don't ignore P2M setup error

set_mmio_p2m_entry() may fail, in particular with -ENOMEM. Don't ignore
such an error, but instead cause domain creation to fail in such a case.

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 
Acked-by: Kevin Tian 
Release-acked-by: Juergen Gross 

commit 88b92c3820cffed4b4abeb139edc2cbd8286cb12
Author: Juergen Gross 
Date:   Tue Feb 12 11:54:07 2019 +0100

iommu: fix iommu_ops initialization

Commit 32a5ea00ec75ef53e ("IOMMU/x86: remove indirection from certain
IOMMU hook accesses") introduced iommu_ops initialized at boot time
with data declared as __initconstrel.

On Intel systems there is another path where iommu_ops is initialized
and this path is relevant on resume after returning from system suspend.
As the initialization data is no longer accessible in this case that
second initialization must be dropped in case the system isn't just
booting.

Signed-off-by: Juergen Gross 
Reviewed-by: Jan Beulich

Re: [Xen-devel] xen: credit2: credit2 can’t reach the throughput as expected

2019-02-13 Thread Dario Faggioli 
On Tue, 2019-02-12 at 13:36 +, 郑 川 wrote:
> Hi, George,
> 
Hi (although I'm not George :-D),

> I found Credit2 can’t reach the throughput as expected under my test
> workload, compared to Credit and CFS. It is easy to reproduce, and I
> think the problem is really exist.
> It really took me a long time to find out why due to my lack of
> knowledge, and I cannot find a good way to solve it.
> Please do help to take a look at it. Thx.
> 
Ok, thanks for your testing, and for reporting this to us.

A few questions.

> ***
> [How to reproduce]
> ***
> I use openSUSE-Tumbleweed with xen-4.11 version.
> Here is the test workload like:
> I have guest_1 with 4 vCPU and guest_2 with 8 vCPU running on 4 pCPU,
> that is, the relation of pCPU:vCPU is 1:3.
> Then I add pressure with 20% CPU usage for each vCPU, which results
> in total 240% pCPU usage.
> The 20% pressure model is that, I start one process on each vCPU,
> which runs 20ms indefinitely and then goes to sleep 80ms within the
> period of 100ms.
> I use xentop to observe guest cpu usage in dom0, as I expect, the
> guest cpu usage is 80% and 160% for guest_1 and guest_2 ,
> respectively.
> 
Do you have the sources for this somewhere, so that we can try to
reproduce it ourself. I'm thinking to the source code for the periodic
apps (if you used a custom made one), or the repository (if you used
one from any) or the name of the benchmarking suite --and the parameter
used to create this scenario?

> **
> [Why it happens]
> **
> The test workload likes the polling from the long term to see.
> As showed in the figure below, the - - - - means the cputime the
> vcpus is running and the ——— means the idle.
> As we can see from Fig.1, if vcpu_1 and vcpu_2 can run staggeredly,
> the throughput looks fine, however, if vcpu_1 and vcpu_2 runs at the
> same time, they will compete for pCPU, which results in poor
> throughput.
> 
> vcpu_1- - - - - - -   - - - - - - -  - - - -
> -
>   ||   | 
>   |   |
> vcpu_2- - - - - - -   - - - - - - -
> 
>   |  vcpu1|   vcpu2   |   |  vcpu1   
>  |   vcpu2   |  |  vcpu1
> cpu usage   - - - - - - - - - - - - -  - - - - - - - - - - - - -
> -  - - - - - - - 
>Fig.1
> 
> vcpu_1   - - - - - - -   - - - - - - -
> ———
>  |
> vcpu_2   - - - - - - -   - - - - - - -
> ———
>  |  compete running |   both
> sleep |  compete running|   both sleep|
> cpu usage   - - - - - - - - - - - - - -  - - - - - - - - - -
> - - - - 
>Fig.2
> 
Ok, I'm not entirely sure I follow all this, but let's put it aside for
a second. The question I have is, is this analysis coming from looking
at actual traces? If yes, can you upload somewhere/share the trace
files?

> As we do reset_credit() when snext->credit is negative which makes
> the credit value is too close between each vcpu.
> As a result, from long term to observe, the time-slice of each vcpu
> becomes smaller, they compete for pCPU at the same time just like
> shown in Fig.2 above.
> Thus, i think the reason why it can't reach the expected throughput
> is that reset_credit() for all vcpu will make the time-slice smaller
> which is different from
> Credit and CFS.
> 
Ok, so you're saying this drop of "throughput" can be caused by
scheduling happening too frequently in Credit2.

Well, I guess that is a possibility, although, as I said above, I'd
need to think a bit more about this, as well as trying to reproduce it,
and look at actual traces.

Perhaps, one thing that can be done to try to confirm this analysis,
would be to make the scheduling less frequent in Credit2 and, on the
other hand, to make it more frequent in Credit1. In theory, if the
analysis is correct, you would observe the behavior of this specific
workload improving on Credit2 and degrading in Credit1, when doing so.

If you fancy trying that, for Credit1, you can play with the
sched_credit_tslice_ms Xen boot time parameter (e.g., try pushing it
down to 1ms).

For Credit2, it's a little trickier, as the scheduler does not have a
real timeslice. So, either you alter CSCHED2_MIN_TIMER, in the code, or
you "mimic" the timeslice increase by setting sched_ratelimit_us to a
higher value (like, e.g., 10ms).

It's not a conclusive test, but I think it is a good enough one for
gaining some more understanding of the issue.

Regards,
Dario
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Software Engineer @ SUSE https://www.suse.com/


signature.asc

Re: [Xen-devel] [PATCH v4 1/1] cameraif: add ABI for para-virtual camera

2019-02-13 Thread Oleksandr Andrushchenko 

Konrad, could you please review? So, I can send v5 with Hans'
comments addressed

Thank you,
Oleksandr

On 1/23/19 10:14 AM, Oleksandr Andrushchenko wrote:

Any comments from Xen community?
Konrad?

On 1/15/19 4:44 PM, Hans Verkuil wrote:

Hi Oleksandr,

Just two remaining comments:

On 1/15/19 10:38 AM, Oleksandr Andrushchenko wrote:

From: Oleksandr Andrushchenko 

This is the ABI for the two halves of a para-virtualized
camera driver which extends Xen's reach multimedia capabilities even
farther enabling it for video conferencing, In-Vehicle Infotainment,
high definition maps etc.

The initial goal is to support most needed functionality with the
final idea to make it possible to extend the protocol if need be:

1. Provide means for base virtual device configuration:
   - pixel formats
   - resolutions
   - frame rates
2. Support basic camera controls:
   - contrast
   - brightness
   - hue
   - saturation
3. Support streaming control

Signed-off-by: Oleksandr Andrushchenko 
---
   xen/include/public/io/cameraif.h | 1364 ++
   1 file changed, 1364 insertions(+)
   create mode 100644 xen/include/public/io/cameraif.h

diff --git a/xen/include/public/io/cameraif.h b/xen/include/public/io/cameraif.h
new file mode 100644
index ..246eb2457f40
--- /dev/null
+++ b/xen/include/public/io/cameraif.h
@@ -0,0 +1,1364 @@




+/*
+ **
+ * EVENT CODES
+ **
+ */
+#define XENCAMERA_EVT_FRAME_AVAIL  0x00
+#define XENCAMERA_EVT_CTRL_CHANGE  0x01
+
+/* Resolution has changed. */
+#define XENCAMERA_EVT_CFG_FLG_RESOL(1 << 0)

I think this flag is a left-over from v2 and should be removed.




+ * Request number of buffers to be used:
+ * 01 2   3octet
+ * +++++
+ * |   id| _OP_BUF_REQUEST|   reserved | 4
+ * +++++
+ * | reserved  | 8
+ * +++++
+ * |num_bufs| reserved | 12
+ * +++++
+ * | reserved  | 16
+ * +++++
+ * |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/|
+ * +++++
+ * | reserved  | 64
+ * +++++
+ *
+ * num_bufs - uint8_t, desired number of buffers to be used. This is
+ *   limited to the value configured in XenStore.max-buffers.
+ *   Passing zero num_bufs in this request (after streaming has stopped
+ *   and all buffers destroyed) unblocks camera configuration changes.

I think the phrase 'unblocks camera configuration changes' is confusing.

In v3 this sentence came after the third note below, and so it made sense
in that context, but now the order has been reversed and it became hard to
understand.

I'm not sure what the best approach is to fix this. One option is to remove
the third note and integrate it somehow in the sentence above. Or perhaps
do away with the 'notes' at all and just write a more extensive documentation
for this op. I leave that up to you.


+ *
+ * See response format for this request.
+ *
+ * Notes:
+ *  - frontend must check the corresponding response in order to see
+ *if the values reported back by the backend do match the desired ones
+ *and can be accepted.
+ *  - frontend may send multiple XENCAMERA_OP_BUF_REQUEST requests before
+ *sending XENCAMERA_OP_STREAM_START request to update or tune the
+ *configuration.
+ *  - after this request camera configuration cannot be changed, unless

camera configuration -> the camera configuration


+ *streaming is stopped and buffers destroyed
+ */

Regards,

Hans



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH SpectreV1+L1TF v6 8/9] common/grant_table: block speculative out-of-bound accesses

2019-02-13 Thread Jan Beulich 
>>> On 08.02.19 at 14:44,  wrote:
> Guests can issue grant table operations and provide guest controlled
> data to them. This data is also used for memory loads. To avoid
> speculative out-of-bound accesses, we use the array_index_nospec macro
> where applicable. However, there are also memory accesses that cannot
> be protected by a single array protection, or multiple accesses in a
> row. To protect these, a nospec barrier is placed between the actual
> range check and the access via the block_speculation macro.
> 
> As different versions of grant tables use structures of different size,
> and the status is encoded in an array for version 2, speculative
> execution might touch zero-initialized structures of version 2 while
> the table is actually using version 1.

Why zero-initialized? Did I still not succeed demonstrating to you
that speculation along a v2 path can actually overrun v1 arrays,
not just access parts with may still be zero-initialized?

> @@ -203,8 +204,9 @@ static inline unsigned int nr_status_frames(const struct 
> grant_table *gt)
>  }
>  
>  #define MAPTRACK_PER_PAGE (PAGE_SIZE / sizeof(struct grant_mapping))
> -#define maptrack_entry(t, e) \
> -((t)->maptrack[(e)/MAPTRACK_PER_PAGE][(e)%MAPTRACK_PER_PAGE])
> +#define maptrack_entry(t, e) 
>   \
> +((t)->maptrack[array_index_nospec(e, (t)->maptrack_limit)
>   \
> + 
> /MAPTRACK_PER_PAGE][(e)%MAPTRACK_PER_PAGE])

I would have hoped that the pointing out of similar formatting
issues elsewhere would have had an impact here as well, but
I see the / is still wrongly at the beginning of a line, and is still
not followed by a blank (would be "preceded" if it was well
placed). And while I realize it's only code movement, adding
the missing blanks around % would be appreciated too at this
occasion.

> @@ -963,9 +965,13 @@ map_grant_ref(
>  PIN_FAIL(unlock_out, GNTST_bad_gntref, "Bad ref %#x for d%d\n",
>   op->ref, rgt->domain->domain_id);
>  
> +/* Make sure the above check is not bypassed speculatively */
> +block_speculation();
> +
>  act = active_entry_acquire(rgt, op->ref);
>  shah = shared_entry_header(rgt, op->ref);
> -status = rgt->gt_version == 1 ? >flags : _entry(rgt, 
> op->ref);
> +status = evaluate_nospec(rgt->gt_version == 1) ? >flags
> + : _entry(rgt, 
> op->ref);

Did you consider folding the two pairs of fences you emit into
one? Moving up the assignment to status ought to achieve this,
as then the block_speculation() could be dropped afaict.

Then again you don't alter shared_entry_header(). If there's
a reason for you not having done so, then a second fence
here is needed in any event.

What about the version check in nr_grant_entries()? It appears
to me as if at least its use in grant_map_exists() (which simply is
the first one I've found) is problematic without an adjustment.
Even worse, ...

> @@ -1321,7 +1327,8 @@ unmap_common(
>  goto unlock_out;
>  }
>  
> -act = active_entry_acquire(rgt, op->ref);
> +act = active_entry_acquire(rgt, array_index_nospec(op->ref,
> +   
> nr_grant_entries(rgt)));

... you add a use e.g. here to _guard_ against speculation.

And what about _set_status(), unmap_common_complete(),
gnttab_grow_table(), gnttab_setup_table(),
release_grant_for_copy(), the 2nd one in acquire_grant_for_copy(),
several ones in gnttab_set_version(), gnttab_release_mappings(),
the 3rd one in mem_sharing_gref_to_gfn(), gnttab_map_frame(),
and gnttab_get_status_frame()?

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Xen (both x86 and Arm) Community Call: Feb 13 - 16:00 - 17:00 UTC - Call for Agenda Items

2019-02-13 Thread George Dunlap 
On 2/13/19 1:31 AM, Rich Persaud wrote:
>> On Feb 11, 2019, at 05:05, Lars Kurth  wrote:
>>
>> Hi all, 
>>
>> we have the community call for February coming up this Wednesday. My sincere 
>> apologies, that I have not asked for agenda items last week. A current 
>> agenda (primarily a skeleton) is available at  
>> https://docs.google.com/document/d/15ZLzQcH794jufDZW1oNYVY2D12CnVqxQ-klFAqkd2bU/edit#heading=h.mz1wjb9vekjn
>>
>> Please propose topics by either editing the running agenda document at 
>> https://docs.google.com/document/d/1Ufv9XcQO0zIAVeFbFCAHAeEIB9Ap4Y4srAm4vI8I01I/edit#
>>  or by replying to the mail. Ideally by a few hours before the meeting!
> 
> Proposed agenda items:
> 
> 1.  Tailored instances of Xen: continuing the Nov 2018 discussion of 
> KCONFIG/L0  hypervisor use cases.  More details upcoming via wiki page.
> 
> 2.  Macro supply chains:  what are best practices for maintaining Xen macros 
> which originate in other open-source communities, e.g. QEMU or Linux?  Would 
> each macro benefit from a documented status (e.g. "Ignore upstream changes", 
> "Monitor upstream changes", "Mirror upstream changes") with associated 
> tooling?
> 
> 3. Go toolchain:  is there community interest in collaborating on the 
> development of golang tools for local management of Xen?  Historically, 
> OpenXT used a combination of Haskell and Ocaml tools.  Some OpenXT community 
> members are using golang with Xen. Could these new tools find a home in 
> upstream Xen?

FWIW we did start to make golang bindings for libxl (see
xen.git/toools/golang/xenlight).  I think golang is an obvious language
to write Xen control stacks in.

The main issue I see at the moment is that while the *API* between a
toolstack and libxl is backwards-compatible, the *ABI* between the
resulting binary and Xen is not; meaning that every time you change your
Xen version, you need to recompile your golang programs.  (This of
course is not limited to golang, but it was highlighted to me when I
started to use go-xenlight.)

But I'd definitely be interested in seeing what your community members
are doing, and if we can come up with a useful solution.

 -George

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Enhancing Xen's Kconfig infrastructure to support tailored solutions

2019-02-13 Thread Jan Beulich 
>>> On 13.02.19 at 03:34,  wrote:
> The initial focus will be to explore and document the range of possible
> use cases that are of interest to the Xen community. This will be the
> input to a design document that is crafted in conjunction with the Xen
> maintainers, to identify possible approaches to extend the existing
> Kconfig infrastructure to produce tailored instances of Xen.

In EXPERT mode, tailoring is already possible to a certain degree,
and personally I don't see why this couldn't become more fine
grained. EXPERT mode, however, is being disliked in general, first
and foremost due to how one needs to enable it (and keep it
enabled).

To move away from a model where only a very limited set of
configurations is security supported, the original question would
need to be addressed: How could such a variety remain
manageable both in terms of testing and in terms of the handling
of bug reports? randconfig build testing is a small piece here, but
since it ensures successful builds only, it doesn't go quite far
enough. Plus, of course, the more variations are possible, the
longer it would take for even just a build issue to be noticed.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [qemu-mainline test] 133158: trouble: blocked/broken/fail/pass

2019-02-13 Thread osstest service owner 
flight 133158 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133158/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken
 build-arm64-pvopsbroken
 build-arm64  broken

Regressions which are regarded as allowable (not blocking):
 build-arm64-pvops 2 hosts-allocate broken REGR. vs. 132847
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 132847
 build-arm64   2 hosts-allocate broken REGR. vs. 132847

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 build-arm64   3 capture-logs  broken blocked in 132847
 build-arm64-xsm   3 capture-logs  broken blocked in 132847
 build-arm64-pvops 3 capture-logs  broken blocked in 132847
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 132847
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 132847
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 132847
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 132847
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 132847
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass

version targeted for testing:
 qemuu22c5f446514a2a4bb0dbe1fea26713da92fc85fa
baseline version:
 qemuua61faa3d02159d24d4fa984733dbc0c905508752

Last test of basis   132847  2019-02-04 13:19:31 Z8 days
Failing since132945  2019-02-05 16:32:07 Z7 days4 attempts
Testing same since   133158  2019-02-11 19:08:21 Z1 days1 attempts


People who touched revisions under test:
  Alex Bennée 
  Anthony PERARD 
  Brendan Shanks 
  Catherine Ho 
  Changpeng Liu 
  Chen Zhang 
  Cleber Rosa 
  Cornelia Huck 
  Daniel P. Berrangé 
  David Hildenbrand 
  Dima Stepanov 
  Doug Gale 
  Ed Maste 
  Emilio G. Cota 
  Eric Blake

Re: [Xen-devel] [PATCH v9 1/5] xen: introduce ptrdiff_t, fix uintptr_t

2019-02-13 Thread Jan Beulich 
>>> On 13.02.19 at 11:28,  wrote:
> On 2/12/19 4:03 PM, Jan Beulich wrote:
> On 12.02.19 at 16:32,  wrote:
>>> On 12/02/2019 15:26, Ian Jackson wrote:

> But if we really want to have ptrdiff_t, then I think we should either
> follow the uintptr_t model and use attribute((mode())), or we should
> leverage the compiler's __PTRDIFF_TYPE__ (and then also
> __UINTPTR_TYPE__ for uintptr_t, at least if available - not sure what
> its availability depends on, but it's conditional in gcc's
> c_stddef_cpp_builtins()).
 It is not unusual for porting something like Xen to a new architecture
 to involve writing a short header file with these kind of type
 definitions.  I don't know why we couldn't take that approach.
>>>
>>> stdint.h and inttypes.h are a freestanding header files, and are
>>> intended for uses just like this.
>>>
>>> Lets stop second guessing our build environment, and use the solution to
>>> the problem given to us by the C specification.
>>>
>>> And to be crystal clear.  This means including  and
>>>  in xen/types.h and deleting all of these typedefs
>> 
>> Well, this would certainly be a viable route if
>> - these headers were truly freestanding (rather than coming in their
>>   own incarnation with every gcc version, and perhaps also with
>>   any other compiler)
> 
> Why would this be a problem?  Isn't that a feature -- that your
> compiler/header file combination provides you with the proper runes to
> get a working uintptr_t?  That seems a lot more robust and reliable than
> trying to keep our own header that works on all possible compiler
> combinations.

Well, that depends on which instance of the header one ends up
including - the one in /usr/include/, or the specific one coming
with the specific compiler version. It is certainly possible that I'm
too biased due to my DOS / OS/2 / Windows / NetWare heritage,
where compilers didn't necessarily know where to take headers
from without being told, but I'm simply uncertain whether of all
compilers we care about (and also all ones we want to care
about in the future) we can expect that they'll find the right
headers without our help, and they won't fall back to e.g. the
ones in /usr/include/, or not find anything at all.

>> - were guaranteed bug free on every distro we care about building on
> 
> This argument seems kind of backwards to me.
> 
> You are implicitly asserting that there are distros we care about
> building on where the system stdint.h and inttypes.h have bugs.  It's
> not up to others to show that this is false, but up to you to show that
> it's true; by providing at least some examples (even if historical).
> 
> Or if you don't have examples ready to hand, saying something like:
> 
> "I seem to recall some distros having bugs in stdint.h in the past.  We
> should make sure to test it with all distro versions we care about
> before committing to it."  This should be somewhat easier now that we
> have the docker build-testing infrastructure.

Oh, that's pretty easy: Why would gcc care to produce fixed up
host headers (see the fixincludes/ subdirectory in its sources and
how it's used by the build process), if there wasn't the problem of
distros getting their headers wrong?

But then again I don't think I agree with you considering this
backwards: Bugs exist, there's no question. For something as
fundamental as producing correct basic types (which _can_ be
got right without relying on any headers outside of a project's
source tree), the mere risk of there possibly being bugs is bad
enough imo.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v5 00/15] Argo: hypervisor-mediated interdomain communication

2019-02-13 Thread Rich Persaud 

> On Feb 4, 2019, at 05:07, Roger Pau Monné  wrote:
> 
>> On Sun, Feb 03, 2019 at 10:04:29AM -0800, Christopher Clark wrote:
>> On Thu, Jan 31, 2019 at 5:39 AM Roger Pau Monné  
>> wrote:
>> 
>> On Wed, Jan 30, 2019 at 08:05:30PM -0800, Christopher Clark wrote:
>> On Tue, Jan 22, 2019 at 6:19 AM Roger Pau Monné  
>> wrote:
>> 
>> On Mon, Jan 21, 2019 at 01:59:40AM -0800, Christopher Clark wrote:
>> Version five of this patch series:
>> 
>> * Changes are primarily addressing feedback from the v4 series reviews.
>> Many points noted on the invididual commit posts.
>> 
>> * Critical sections have been shrunk, with allocations and frees
>> pulled outside where possible, reordering logic within hypercall ops.
>> 
>> * A new ring hash function implemented, derived from the djb2 string
>> hash function.
>> 
>> * Flags returned by the notify op have been simplified.
>> 
>> * Now uses a single argo boot parameter, taking a list:
>> - top level boolean to enable/disable Argo
>> - mac-permissive option to enable/disable wildcard rings
>> - command line doc edit: no "CONFIG_ARGO" but refers to build config
>> 
>> * Switched to use the standard list data structures used by Xen's
>> common code.
> 
> AFAIK this was not requested by any reviewer, so I wonder why you made
> such change. The more that you open coded some of the list_ macros
> instead of just doing a s/hlist_/list_/ replacement.
> I'm fine with using list instead of hlist,
 
 At your request, v7 replaces open coding with Xen's list macros. The
 hlist macros were not used by any of the common code in Xen.
 
> but I don't understand why
> you decided to open code list_for_each and list_for_each_safe instead
> of using the macros provided by Xen. Is there an issue with such
> macros?
 
 As discussed offline:
 
 - Using Xen's list macros will expedite Argo's merge for Xen 4.12
 - List macros in Xen list.h originated in Linux list.h and have diverged
 - OpenXT has use cases for measured launch and nested virtualization,
 which influence downstream performance and security requirements for
 Argo and Xen
 - OpenXT can temporarily patch Xen 4.12 for downstream use
 
> I've made a couple of minor comments, but I think the current status
> is good, and fixing those minor comments is going to be trivial.
 
 Ack, thanks. Hopefully v7 looks good.
>>> 
>>> As a note, the common flow of interactions usually involves the
>>> contributor replying to the comments made by the reviewer in order to
>>> try to reach an agreement before sending a new version.
>> 
>> Yes, v7 was sent to address Jan and Julien's review comments in parallel
>> with our ongoing discussion on v5 macros. v7 also provided a checkpoint
>> for Argo testers to maximize test coverage as the series converges into
>> a Xen 4.12 merge candidate for Juergen. It addressed:
>> 
>> - Jan's v6 review comments
>> - Julien's v1 review comment
>> - most of your xen-devel and offline review comments
> 
> I think it will benefit the community to give this review in public,
> so other reviewers know whats going on. IMO getting this private
> review makes it harder for me (as a reviewer) to know the motivation
> of some of the changes between versions, and likely also makes it
> harder for you since you have to keep track of comments from multiple
> sources on different channels.
> 
> Is there anything that prevents those people from making the review
> comments publicly on xen-devel?
> 
> We should very much try to fix that so everyone can make review
> comments on the public mailing list.

I've advocated for open-source principles in several large organizations.  At 
XenSource and Citrix, we created organizational separation between the OSS Xen 
dev team and product teams.  I don't know if that structure remains today, but 
it was once helpful in reducing conflict between public OSS and private product 
roadmaps.

The separation between server and client Xen product teams was less ideal, 
which eventually lead to OpenXT.  Six years after v4v was posted to xen-devel, 
Xen Argo is the first step to possible reunification, a small chance at 
reversal, via public open-source, of architectural and resource fragmentation 
that took place privately.

Like QubesOS, OpenXT (and predecessor Citrix XenClient) development is spread 
across many open-source projects, including Xen, enabling user workflows that 
balance hardware-assisted security with usability.  Spanning ecosystems, OpenXT 
is:

- unbundling OSS capabilities, e.g. TrenchBoot and coreboot for launch integrity
- moving code upstream (Argo, stubdom, blktap, Qemu, OpenEmbedded meta-virt)
- refactoring for peer & downstream derivatives, on client devices and beyond

To achieve this cross-community integration, we work with many stakeholders

Re: [Xen-devel] Upstream Dom0 DRM problems regarding swiotlb

2019-02-13 Thread Jan Beulich 
>>> On 12.02.19 at 19:46,  wrote:
> Konrad,
> 
> Starting w/ v4.17, I cannot log in to GNOME w/out getting the
> following mess in dmesg and ending up back at the GDM login screen.
> 
> [   28.554259] radeon_dp_aux_transfer_native: 200 callbacks suppressed
> [   31.219821] radeon :01:00.0: swiotlb buffer is full (sz: 2097152 
> bytes)
> [   31.220030] [drm:radeon_gem_object_create [radeon]] *ERROR* Failed
> to allocate GEM object (16384000, 2, 4096, -14)
> [   31.226109] radeon :01:00.0: swiotlb buffer is full (sz: 2097152 
> bytes)
> [   31.226300] [drm:radeon_gem_object_create [radeon]] *ERROR* Failed
> to allocate GEM object (16384000, 2, 4096, -14)
> [   31.300734] gnome-shell[1935]: segfault at 88 ip 7f39151cd904
> sp 7ffc97611ad8 error 4 in libmutter-cogl.so[7f3915178000+aa000]
> [   31.300745] Code: 5f c3 0f 1f 40 00 48 8b 47 78 48 8b 40 40 ff e0
> 66 0f 1f 44 00 00 48 8b 47 78 48 8b 40 48 ff e0 66 0f 1f 44 00 00 48
> 8b 47 78 <48> 8b 80 88 00 00 00 ff e0 0f 1f 00 48 8b 47 78 48 8b 40 68
> ff e0
> [   38.193302] radeon_dp_aux_transfer_native: 116 callbacks suppressed
> [   40.009317] radeon :01:00.0: swiotlb buffer is full (sz: 2097152 
> bytes)
> [   40.009488] [drm:radeon_gem_object_create [radeon]] *ERROR* Failed
> to allocate GEM object (16384000, 2, 4096, -14)
> [   40.015114] radeon :01:00.0: swiotlb buffer is full (sz: 2097152 
> bytes)
> [   40.015297] [drm:radeon_gem_object_create [radeon]] *ERROR* Failed
> to allocate GEM object (16384000, 2, 4096, -14)
> [   40.028302] gnome-shell[2431]: segfault at 2dadf40 ip
> 02dadf40 sp 7ffcd24ea5f8 error 15
> [   40.028306] Code: 20 6e 31 00 00 00 00 00 00 00 00 37 e3 3d 2d 7f
> 00 00 80 f4 e6 3d 2d 7f 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 00 00 00 <00> 00 00 00 00 00 00 00 c1 00 00 00 00 00 00 00 80 e1 d2 03
> 00 00
> 
> 
> This happens w/ both radeon and amdgpu.
> 
> I bisected down to the following range of commits, which basically add
> conditional code to radeon and amdgpu to NOT use swiotlb if dma_bits
> is smaller than the system's max iomem address...  but that very much
> doesn't work on a Xen dom0.

Well, not so much a Xen Dom0, but a Xen PV domain.

> 82626363 drm: add func to get max iomem address v2
> fd5fd480 drm/amdgpu: only enable swiotlb alloc when need v2
> 1bc3d3cc drm/radeon: only enable swiotlb path when need v2
> 
> Reverting the offending commits gives me a usable v4.20 dom0 kernel w/
> working 3d support.  Not sure what the appropriate upstream fix for
> this would be, as I don't 100% understand this.  Could you enlighten
> me?  ;-)

Well, this depends on how much abstraction we want, and how
much abstraction the maintainers of the DRM drivers demand.
It could be as simple as adding xen_swiotlb checks into the
conditionals setting ->need_swiotlb, but in an abstract sense
the issue of course exists for PV guests of any hypervisor.
(Altering drm_get_max_iomem() itself would seem wrong to me,
unless its name was also changed.)

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v5 00/15] Argo: hypervisor-mediated interdomain communication

2019-02-13 Thread Rich Persaud 
> On Feb 4, 2019, at 13:22, Stefano Stabellini  wrote:
> 
> On Mon, 4 Feb 2019, Roger Pau Monné wrote:
>>> Yes, v7 was sent to address Jan and Julien's review comments in parallel
>>> with our ongoing discussion on v5 macros. v7 also provided a checkpoint
>>> for Argo testers to maximize test coverage as the series converges into
>>> a Xen 4.12 merge candidate for Juergen. It addressed:
>>> 
>>> - Jan's v6 review comments
>>> - Julien's v1 review comment
>>> - most of your xen-devel and offline review comments
>> 
>> I think it will benefit the community to give this review in public,
>> so other reviewers know whats going on. IMO getting this private
>> review makes it harder for me (as a reviewer) to know the motivation
>> of some of the changes between versions, and likely also makes it
>> harder for you since you have to keep track of comments from multiple
>> sources on different channels.
> 
> There is one more reason to require public comments which I have only
> learned recently: for safety certifications we need to keep a record of
> all review comments and patches that address them for traceability.

Do you mean:

(A) all _merged_ patches and their review comments

 or

(B) all comments and patches (merged or not) that address them

i.e. would the certification process be seeking traceability of 
safety-impacting patches (code, scenario A) or decisions (including decisions 
to leave code unchanged, scenario B)?

If you mean (B), would we need an update to the Xen Security Problem Response 
Process [1]?  e.g. public archive of all comments from pre-disclosure 
discussion, along with content hashes stored immutably?  

Rich

[1] https://www.xenproject.org/security-policy.html


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v9 1/5] xen: introduce ptrdiff_t, fix uintptr_t

2019-02-13 Thread George Dunlap 
On 2/12/19 4:03 PM, Jan Beulich wrote:
 On 12.02.19 at 16:32,  wrote:
>> On 12/02/2019 15:26, Ian Jackson wrote:
>>>
 But if we really want to have ptrdiff_t, then I think we should either
 follow the uintptr_t model and use attribute((mode())), or we should
 leverage the compiler's __PTRDIFF_TYPE__ (and then also
 __UINTPTR_TYPE__ for uintptr_t, at least if available - not sure what
 its availability depends on, but it's conditional in gcc's
 c_stddef_cpp_builtins()).
>>> It is not unusual for porting something like Xen to a new architecture
>>> to involve writing a short header file with these kind of type
>>> definitions.  I don't know why we couldn't take that approach.
>>
>> stdint.h and inttypes.h are a freestanding header files, and are
>> intended for uses just like this.
>>
>> Lets stop second guessing our build environment, and use the solution to
>> the problem given to us by the C specification.
>>
>> And to be crystal clear.  This means including  and
>>  in xen/types.h and deleting all of these typedefs
> 
> Well, this would certainly be a viable route if
> - these headers were truly freestanding (rather than coming in their
>   own incarnation with every gcc version, and perhaps also with
>   any other compiler)

Why would this be a problem?  Isn't that a feature -- that your
compiler/header file combination provides you with the proper runes to
get a working uintptr_t?  That seems a lot more robust and reliable than
trying to keep our own header that works on all possible compiler
combinations.

> - were guaranteed bug free on every distro we care about building on

This argument seems kind of backwards to me.

You are implicitly asserting that there are distros we care about
building on where the system stdint.h and inttypes.h have bugs.  It's
not up to others to show that this is false, but up to you to show that
it's true; by providing at least some examples (even if historical).

Or if you don't have examples ready to hand, saying something like:

"I seem to recall some distros having bugs in stdint.h in the past.  We
should make sure to test it with all distro versions we care about
before committing to it."  This should be somewhat easier now that we
have the docker build-testing infrastructure.

 -George

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-unstable-smoke test] 133221: trouble: blocked/broken/pass

2019-02-13 Thread osstest service owner 
flight 133221 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133221/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken

Regressions which are regarded as allowable (not blocking):
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 133005

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64-xsm   3 capture-logs  broken blocked in 133005
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  f178a00c30173c0b268d99160e19ad299b1823a2
baseline version:
 xen  455301716e1ff358cb79367213003fba771dd466

Last test of basis   133005  2019-02-07 14:00:32 Z5 days
Failing since133011  2019-02-07 18:00:36 Z5 days   30 attempts
Testing same since   133200  2019-02-12 16:00:56 Z0 days7 attempts


People who touched revisions under test:
  Andrew Cooper 
  George Dunlap 
  Ian Jackson 
  Jan Beulich 
  Juergen Gross 
  Kevin Tian 
  Michael Tautschnig 
  Norbert Manthey 
  Norbert Manthey 
  Stefano Stabellini 
  Stefano Stabellini 
  Tim Deegan 
  Wei Liu 

jobs:
 build-arm64-xsm  broken  
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64-xsm broken
broken-step build-arm64-xsm hosts-allocate
broken-step build-arm64-xsm capture-logs

Not pushing.


commit f178a00c30173c0b268d99160e19ad299b1823a2
Author: Norbert Manthey 
Date:   Tue Feb 12 15:20:15 2019 +0100

x86/hvm: block speculative out-of-bound accesses

There are multiple arrays in the HVM interface that are accessed
with indices that are provided by the guest. To avoid speculative
out-of-bound accesses, we use the array_index_nospec macro.

When blocking speculative out-of-bound accesses, we can classify arrays
into dynamic arrays and static arrays. Where the former are allocated
during run time, the size of the latter is known during compile time.
On static arrays, compiler might be able to block speculative accesses
in the future.

This is part of the speculative hardening effort.

Reported-by: Pawel Wieczorkiewicz 
Signed-off-by: Norbert Manthey 
Reviewed-by: Jan Beulich 
Release-acked-by: Juergen Gross 

commit 56d8d0119d270f846c6c4943712b8a21fbe5d4d0
Author: Jan Beulich 
Date:   Tue Feb 12 11:54:57 2019 +0100

VMX: don't ignore P2M setup error

set_mmio_p2m_entry() may fail, in particular with -ENOMEM. Don't ignore
such an error, but instead cause domain creation to fail in such a case.

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 
Acked-by: Kevin Tian 
Release-acked-by: Juergen Gross 

commit 88b92c3820cffed4b4abeb139edc2cbd8286cb12
Author: Juergen Gross 
Date:   Tue Feb 12 11:54:07 2019 +0100

iommu: fix iommu_ops initialization

Commit 32a5ea00ec75ef53e ("IOMMU/x86: remove indirection from certain
IOMMU hook accesses") introduced iommu_ops initialized at boot time
with data declared as __initconstrel.

On Intel systems there is another path where iommu_ops is initialized
and this path is relevant on resume after returning from system suspend.
As the initialization data is no longer accessible in this case that
second initialization must be dropped in case the system isn't just
booting.

Signed-off-by: Juergen Gross 
Reviewed-by: Jan Beulich

Re: [Xen-devel] [PATCH v2 5/7] x86/mm: split p2m ioreq server pages special handling into helper

2019-02-13 Thread Paul Durrant 
> -Original Message-
> From: Roger Pau Monne [mailto:roger@citrix.com]
> Sent: 11 February 2019 17:47
> To: xen-devel@lists.xenproject.org
> Cc: Roger Pau Monne ; George Dunlap
> ; Jan Beulich ; Andrew Cooper
> ; Wei Liu ; Jun Nakajima
> ; Kevin Tian ; Paul Durrant
> 
> Subject: [PATCH v2 5/7] x86/mm: split p2m ioreq server pages special
> handling into helper
> 
> So that it can be shared by both ept, npt and shadow code, instead of
> duplicating it.
> 
> No change in functionality intended.
> 
> Signed-off-by: Roger Pau Monné 

LGTM.

Reviewed-by: Paul Durrant 

> ---
> Cc: George Dunlap 
> Cc: Jan Beulich 
> Cc: Andrew Cooper 
> Cc: Wei Liu 
> Cc: Jun Nakajima 
> Cc: Kevin Tian 
> Cc: Paul Durrant 
> ---
> Changes since v1:
>  - Remove unused p2mt_old from p2m_pt_set_entry.
> ---
>  xen/arch/x86/mm/hap/hap.c   |  3 ++
>  xen/arch/x86/mm/p2m-ept.c   | 55 +++--
>  xen/arch/x86/mm/p2m-pt.c| 24 --
>  xen/arch/x86/mm/shadow/common.c |  3 ++
>  xen/include/asm-x86/p2m.h   | 32 +++
>  5 files changed, 56 insertions(+), 61 deletions(-)
> 
> diff --git a/xen/arch/x86/mm/hap/hap.c b/xen/arch/x86/mm/hap/hap.c
> index 3d651b94c3..dc46d5e14f 100644
> --- a/xen/arch/x86/mm/hap/hap.c
> +++ b/xen/arch/x86/mm/hap/hap.c
> @@ -734,6 +734,9 @@ hap_write_p2m_entry(struct domain *d, unsigned long
> gfn, l1_pgentry_t *p,
>  && perms_strictly_increased(old_flags, l1e_get_flags(new)) );
>  }
> 
> +p2m_entry_modify(p2m_get_hostp2m(d),
> p2m_flags_to_type(l1e_get_flags(new)),
> + p2m_flags_to_type(old_flags), level);
> +
>  safe_write_pte(p, new);
>  if ( old_flags & _PAGE_PRESENT )
>  flush_tlb_mask(d->dirty_cpumask);
> diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c
> index bb562607f7..0ece6608cb 100644
> --- a/xen/arch/x86/mm/p2m-ept.c
> +++ b/xen/arch/x86/mm/p2m-ept.c
> @@ -46,7 +46,8 @@ static inline bool_t is_epte_valid(ept_entry_t *e)
>  }
> 
>  /* returns : 0 for success, -errno otherwise */
> -static int atomic_write_ept_entry(ept_entry_t *entryptr, ept_entry_t new,
> +static int atomic_write_ept_entry(struct p2m_domain *p2m,
> +  ept_entry_t *entryptr, ept_entry_t new,
>int level)
>  {
>  int rc;
> @@ -89,6 +90,8 @@ static int atomic_write_ept_entry(ept_entry_t *entryptr,
> ept_entry_t new,
>  if ( unlikely(p2m_is_foreign(entryptr->sa_p2mt)) && check_foreign )
>  oldmfn = entryptr->mfn;
> 
> +p2m_entry_modify(p2m, new.sa_p2mt, entryptr->sa_p2mt, level);
> +
>  write_atomic(>epte, new.epte);
> 
>  if ( unlikely(oldmfn != mfn_x(INVALID_MFN)) )
> @@ -390,7 +393,8 @@ static int ept_next_level(struct p2m_domain *p2m,
> bool_t read_only,
>   * present entries in the given page table, optionally marking the
> entries
>   * also for their subtrees needing P2M type re-calculation.
>   */
> -static bool_t ept_invalidate_emt(mfn_t mfn, bool_t recalc, int level)
> +static bool_t ept_invalidate_emt(struct p2m_domain *p2m, mfn_t mfn,
> + bool_t recalc, int level)
>  {
>  int rc;
>  ept_entry_t *epte = map_domain_page(mfn);
> @@ -408,7 +412,7 @@ static bool_t ept_invalidate_emt(mfn_t mfn, bool_t
> recalc, int level)
>  e.emt = MTRR_NUM_TYPES;
>  if ( recalc )
>  e.recalc = 1;
> -rc = atomic_write_ept_entry([i], e, level);
> +rc = atomic_write_ept_entry(p2m, [i], e, level);
>  ASSERT(rc == 0);
>  changed = 1;
>  }
> @@ -459,7 +463,7 @@ static int ept_invalidate_emt_range(struct p2m_domain
> *p2m,
>  rc = -ENOMEM;
>  goto out;
>  }
> -wrc = atomic_write_ept_entry([index], split_ept_entry, i);
> +wrc = atomic_write_ept_entry(p2m, [index], split_ept_entry,
> i);
>  ASSERT(wrc == 0);
> 
>  for ( ; i > target; --i )
> @@ -479,7 +483,7 @@ static int ept_invalidate_emt_range(struct p2m_domain
> *p2m,
>  {
>  e.emt = MTRR_NUM_TYPES;
>  e.recalc = 1;
> -wrc = atomic_write_ept_entry([index], e, target);
> +wrc = atomic_write_ept_entry(p2m, [index], e, target);
>  ASSERT(wrc == 0);
>  rc = 1;
>  }
> @@ -549,17 +553,11 @@ static int resolve_misconfig(struct p2m_domain *p2m,
> unsigned long gfn)
>  nt = p2m_recalc_type(e.recalc, e.sa_p2mt, p2m, gfn +
> i);
>  if ( nt != e.sa_p2mt )
>  {
> -if ( e.sa_p2mt == p2m_ioreq_server )
> -{
> -ASSERT(p2m->ioreq.entry_count > 0);
> -p2m->ioreq.entry_count--;
> -}
> -
>  e.sa_p2mt = nt;
>  ept_p2m_type_to_flags(p2m, , e.sa_p2mt,
> e.access);
>  }
>

Re: [Xen-devel] Xen (both x86 and Arm) Community Call: Feb 13 - 16:00 - 17:00 UTC - Call for Agenda Items

2019-02-13 Thread Jan Beulich 
>>> On 13.02.19 at 02:31,  wrote:
> 4. NVME passthrough performance: this is improved when VMEXITs are avoided 
> by using "posted interrupts" [1] available on Broadwell and later Xeon 
> processors or AWS nested hypervisor "metal" [2].  For commodity x86 CPUs 
> which do not have posted interrupts, Linux [3] and Hyper-V [4] have used 
> "hybrid polling" to achieve good I/O performance at the cost of CPU cycles.  
> Is this applicable to Xen?

The number of references you make alone already suggest to me that
this is not a topic usefully discussed on a phone call. May I suggest to
use the mailing list for topics requiring a lot of context?

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-unstable-coverity test] 133223: regressions - ALL FAIL

2019-02-13 Thread osstest service owner 
flight 133223 xen-unstable-coverity real [real]
http://logs.test-lab.xenproject.org/osstest/logs/133223/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 coverity-amd647 coverity-upload  fail REGR. vs. 132424

version targeted for testing:
 xen  455301716e1ff358cb79367213003fba771dd466
baseline version:
 xen  08b908ba63dee8bc313983c5e412852cbcbcda85

Last test of basis   132424  2019-01-23 09:19:14 Z   21 days
Failing since132506  2019-01-27 09:18:42 Z   17 days6 attempts
Testing same since   133101  2019-02-10 09:18:30 Z3 days2 attempts


People who touched revisions under test:
  Andrew Cooper 
  Andrii Anisov 
  Anthony PERARD 
  Brian Woods 
  Chris Patterson 
  Christopher Clark 
  Christopher Clark 
  Daniel De Graaf 
  Doug Goldstein 
  George Dunlap 
  Hans van Kranenburg 
  Ian Jackson 
  Jan Beulich 
  Julien Grall 
  Lars Kurth 
  Norbert Manthey 
  Peng Fan 
  Roger Pau Monne 
  Roger Pau Monné 
  Tamas K Lengyel 
  Wei Liu 

jobs:
 coverity-amd64   fail



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 1231 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] Macro supply chains

2019-02-13 Thread Rich Persaud 
Synopsys, which owns both Coverity and Black Duck, wrote about software 
supply-chain integrity for a library with almost two million downloads per week:

"EventStream, a highly popular JavaScript library, was compromised with the 
addition of a third-party dependency, flatmap-stream, containing encrypted 
malicious code. The attack targeted other Node.js libraries used in 
cryptocurrency wallets."
"Keep a bill of materials (BoM), a list of components and dependencies in your 
codebase. Just knowing what your code depends on will help make you aware of 
the third-party risks that you might be exposed to."

Are there existing best practices for tracking and maintaining macros which 
originated in other open-source communities, e.g. QEMU or Linux?  Some Xen 
macros have diverged [2][3][4] from the versions used by other communities.  
Would such macros benefit from a documented relationship with upstream, e.g.

- "Ignore upstream changes"
- "Mirror upstream changes"
- "Review upstream changes"

For the latter case, build/test tooling could trigger manual macro review when 
a change is detected in an upstream dependency.  Which other cases should be 
considered?

Rich

[1] Compromised npm packages lead to supply chain attack

https://www.synopsys.com/blogs/software-security/malicious-dependency-supply-chain/

https://github.com/dominictarr/event-stream/issues/116

[2] bitops.h

http://xenbits.xen.org/gitweb/?p=xen.git;a=history;f=xen/include/xen/bitops.h;h=a103e490894829a445cc743b98f8956b7d44e022;hb=HEAD

https://github.com/torvalds/linux/commits/master/arch/x86/include/asm/bitops.h

[3] list.h

http://xenbits.xen.org/gitweb/?p=xen.git;a=history;f=xen/include/xen/list.h;h=1387abb21192668f4103620520cf4256902a45aa;hb=HEAD

https://github.com/torvalds/linux/commits/master/include/linux/list.h

[4] delay.h

http://xenbits.xen.org/gitweb/?p=xen.git;a=history;f=xen/include/xen/delay.h;h=9d70ef035fc20bb6708a5ff33eb83a55c6ad460c;hb=HEAD

https://github.com/torvalds/linux/commits/master/include/linux/delay.h


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v5 8/8] microcode: update microcode on cores in parallel

2019-02-13 Thread Jan Beulich 
>>> On 13.02.19 at 09:50,  wrote:
> On Wed, Feb 13, 2019 at 12:20:51AM -0700, Jan Beulich wrote:
> On 13.02.19 at 03:30,  wrote:
>>> On Tue, Feb 12, 2019 at 06:55:20AM -0700, Jan Beulich wrote:
>>> On 12.02.19 at 14:25,  wrote:
> On Tue, Feb 12, 2019 at 05:51:41AM -0700, Jan Beulich wrote:
>> >>> On 28.01.19 at 08:06,  wrote:
>> > @@ -314,9 +310,7 @@ static int apply_microcode(unsigned int cpu)
>> >  
>> >  mc_intel = patch->data;
>> >  BUG_ON(!mc_intel);
>> > -
>> > -/* serialize access to the physical write to MSR 0x79 */
>> > -spin_lock_irqsave(_update_lock, flags);
>> > +BUG_ON(local_irq_is_enabled());
>> >  
>> >  /* write microcode via MSR 0x79 */
>> >  wrmsrl(MSR_IA32_UCODE_WRITE, (unsigned long)mc_intel->bits);
>> > @@ -329,7 +323,6 @@ static int apply_microcode(unsigned int cpu)
>> >  rdmsrl(MSR_IA32_UCODE_REV, msr_content);
>> >  val[1] = (uint32_t)(msr_content >> 32);
>> >  
>> > -spin_unlock_irqrestore(_update_lock, flags);
>> >  if ( val[1] != mc_intel->hdr.rev )
>> >  {
>> >  printk(KERN_ERR "microcode: CPU%d update from revision "
>> 
>> Am I understanding right that you now rely on upper layers in the
>> call tree to avoid calling into here in parallel for two hyperthreads
>> of the same core? I can't see how you avoid this situation during
>> AP bringup, for example. Did I overlook anything in this regard?
> 
> IIRC microcode update is done in the serialized part of AP bringup,
> before the call to smp_callin, which guarantees serialization.

Hmm, yes, right now it is. But I'd call this "happens to be that way"
rather than "is guaranteed to be that way" - prior to commit
f97838bbd9 it did happen later.
>>> 
>>> How about employing another lock, "early_ucode_update_lock", to
>>> guarantee serialization.
>>> 
>>> In particular, early_microcode_update_cpu() and microcode_resume_cpu()
>>> will acquire this lock before ucode update.
>>
>>That's a (temporary) option, but would over-serialize things again.
>>I was rather trying to hint towards a per-core lock.
> 
> To implement a per-core lock, a core should be conscious of its siblings.
> Unfortunately, during AP bringup, ucode update is done prior to the
> initialization of 'cpu_sibling_mask'.

Okay, in this case a global lock will perhaps do for now (as boot
time AP bringup is serialized in this regard already anyway). I do
think though that the sibling maps should be set up earlier, but
this doesn't look to be as simple as moving ahead the call to
set_cpu_sibling_map(). I wonder anyway why all sorts of stuff
have accumulated ahead of smp_callin(), thus delaying the boot
process in particular on large systems. Clearly something taking
as long as microcode load should, if at all possible, happen
afterwards.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 1/2] amd-iommu: use a bitfield for PTE/PDE

2019-02-13 Thread Paul Durrant 
> -Original Message-
> From: Woods, Brian [mailto:brian.wo...@amd.com]
> Sent: 12 February 2019 20:14
> To: Paul Durrant ; xen-devel@lists.xenproject.org
> Cc: Suthikulpanit, Suravee ; Jan Beulich
> ; Andrew Cooper ; Wei Liu
> ; Roger Pau Monne 
> Subject: Re: [PATCH 1/2] amd-iommu: use a bitfield for PTE/PDE
> 
> On 2/4/19 5:19 AM, Paul Durrant wrote:
> > The current use of get/set_field_from/in_reg_u32() is both inefficient
> and
> > requires some ugly casting.
> >
> > This patch defines a new bitfield structure (amd_iommu_pte) and uses
> this
> > structure in all PTE/PDE manipulation, resulting in much more readable
> > and compact code.
> >
> > NOTE: This commit also fixes one malformed comment in
> >set_iommu_pte_present().
> >
> > Signed-off-by: Paul Durrant 
> 
> Sorry about the delay.
> 
> Nitpick here, but I'd rather have !!IOMMUF_{writable,readable} than
> true.

That's pretty ugly. How about I pass an OR of the flags through to lower level 
functions rather than a pair of bools? If you're ok with that I'll send a v2.

  Paul

>  Not worth a revision if there isn't anything else though (and is
> debatable).
> 
> Acked-by: Brian Woods 
> 
> > ---
> > Cc: Suravee Suthikulpanit 
> > Cc: Brian Woods 
> > Cc: Jan Beulich 
> > Cc: Andrew Cooper 
> > Cc: Wei Liu 
> > Cc: "Roger Pau Monné" 
> > ---
> >   xen/drivers/passthrough/amd/iommu_map.c   | 143 --
> >   xen/drivers/passthrough/amd/pci_amd_iommu.c   |  50 +++---
> >   xen/include/asm-x86/hvm/svm/amd-iommu-defs.h  |  47 ++
> >   xen/include/asm-x86/hvm/svm/amd-iommu-proto.h |  15 --
> >   4 files changed, 64 insertions(+), 191 deletions(-)
> >
> > diff --git a/xen/drivers/passthrough/amd/iommu_map.c
> b/xen/drivers/passthrough/amd/iommu_map.c
> > index 67329b0c95..5fda6063df 100644
> > --- a/xen/drivers/passthrough/amd/iommu_map.c
> > +++ b/xen/drivers/passthrough/amd/iommu_map.c
> > @@ -38,100 +38,45 @@ static unsigned int pfn_to_pde_idx(unsigned long
> pfn, unsigned int level)
> >   static unsigned int clear_iommu_pte_present(unsigned long l1_mfn,
> >   unsigned long dfn)
> >   {
> > -uint64_t *table, *pte;
> > +struct amd_iommu_pte *table, *pte;
> >   unsigned int flush_flags;
> >
> >   table = map_domain_page(_mfn(l1_mfn));
> > +pte = [pfn_to_pde_idx(dfn, 1)];
> >
> > -pte = (table + pfn_to_pde_idx(dfn, 1));
> > +flush_flags = pte->pr ? IOMMU_FLUSHF_modified : 0;
> > +memset(pte, 0, sizeof(*pte));
> >
> > -flush_flags = get_field_from_reg_u32(*pte, IOMMU_PTE_PRESENT_MASK,
> > - IOMMU_PTE_PRESENT_SHIFT) ?
> > - IOMMU_FLUSHF_modified : 0;
> > -
> > -*pte = 0;
> >   unmap_domain_page(table);
> >
> >   return flush_flags;
> >   }
> >
> > -static unsigned int set_iommu_pde_present(uint32_t *pde,
> > +static unsigned int set_iommu_pde_present(struct amd_iommu_pte *pte,
> > unsigned long next_mfn,
> > unsigned int next_level,
> bool iw,
> > bool ir)
> >   {
> > -uint64_t maddr_next;
> > -uint32_t addr_lo, addr_hi, entry;
> > -bool old_present;
> >   unsigned int flush_flags = IOMMU_FLUSHF_added;
> >
> > -maddr_next = __pfn_to_paddr(next_mfn);
> > -
> > -old_present = get_field_from_reg_u32(pde[0],
> IOMMU_PTE_PRESENT_MASK,
> > - IOMMU_PTE_PRESENT_SHIFT);
> > -if ( old_present )
> > -{
> > -bool old_r, old_w;
> > -unsigned int old_level;
> > -uint64_t maddr_old;
> > -
> > -addr_hi = get_field_from_reg_u32(pde[1],
> > - IOMMU_PTE_ADDR_HIGH_MASK,
> > - IOMMU_PTE_ADDR_HIGH_SHIFT);
> > -addr_lo = get_field_from_reg_u32(pde[0],
> > - IOMMU_PTE_ADDR_LOW_MASK,
> > - IOMMU_PTE_ADDR_LOW_SHIFT);
> > -old_level = get_field_from_reg_u32(pde[0],
> > -   IOMMU_PDE_NEXT_LEVEL_MASK,
> > -   IOMMU_PDE_NEXT_LEVEL_SHIFT);
> > -old_w = get_field_from_reg_u32(pde[1],
> > -
> IOMMU_PTE_IO_WRITE_PERMISSION_MASK,
> > -
> IOMMU_PTE_IO_WRITE_PERMISSION_SHIFT);
> > -old_r = get_field_from_reg_u32(pde[1],
> > -
> IOMMU_PTE_IO_READ_PERMISSION_MASK,
> > -
> IOMMU_PTE_IO_READ_PERMISSION_SHIFT);
> > -
> > -maddr_old = ((uint64_t)addr_hi << 32) |
> > -((uint64_t)addr_lo << PAGE_SHIFT);
> > -
> > -if ( maddr_old != maddr_next || iw != old_w || ir != old_r ||
> > - old_level != next_level )
> > +if ( pte->pr &&
> > + (pte->mfn != next_mfn ||
> > +  pte->iw != iw ||
> > +  pte->ir != ir ||
> > +

Re: [Xen-devel] Xen (both x86 and Arm) Community Call: Feb 13 - 16:00 - 17:00 UTC - Call for Agenda Items

2019-02-13 Thread Lars Kurth 
Apologies: it is 16:00--17:00 as in the URL, etc. - I had forgotten to update 
the UTC time

I would like to add an agenda item about the timing of the meeting under AOB
* Originally the meeting was held 15:00 - 16:00 and Stefano requested it to be 
moved
* However, it turns out that normally key people such as Andrew Cooper can't 
attend. 
* To solve this, we may have to pick another day

Also, I came across Daniel's thread called "Enhancing Xen's Kconfig 
infrastructure to support tailored solutions", which I think would be 
worthwhile discussing at this meeting (or maybe the next). The overall proposal 
makes a lot of sense to me

Best Regards
Lars

> On 13 Feb 2019, at 06:22, Julien Grall  wrote:
> 
> Hi Lars,
> 
> The title says "16:00 - 17:00 UTC" but the text below says "15:00 - 16:00 
> UTC". Can you confirm what time is the meeting?
> 
> Cheers,
> 
> 
> On Mon, 11 Feb 2019, 11:07 Lars Kurth,  > wrote:
> Hi all, 
> 
> we have the community call for February coming up this Wednesday. My sincere 
> apologies, that I have not asked for agenda items last week. A current agenda 
> (primarily a skeleton) is available at  
> https://docs.google.com/document/d/15ZLzQcH794jufDZW1oNYVY2D12CnVqxQ-klFAqkd2bU/edit#heading=h.mz1wjb9vekjn
>  
> 
> 
> Please propose topics by either editing the running agenda document at 
> https://docs.google.com/document/d/1Ufv9XcQO0zIAVeFbFCAHAeEIB9Ap4Y4srAm4vI8I01I/edit#
>  
> 
>  or by replying to the mail. Ideally by a few hours before the meeting!
> 
> Best Regards
> Lars
>  
> 
> == Dial-in Information ==
> 
>   ## Future Community Call schedule
>   Feb 13, Mar 12
> 
>   ## Meeting time
>   16:00 - 17:00 UTC
>8:00 -  9:00 EDT (San Francisco)
>   11:00 - 12:00 EDT (New York)
>   16:00 - 17:00 BST (London)
>   17:00 - 18:00 CEST (Berlin)
>   00:00 - 01:00 CST (Beijing)
>   Further International meeting times: 
>   
> https://www.timeanddate.com/worldclock/meetingdetails.html?year=2019=2=13=16=0=0=224=24=179=136=37=33
>  
> 
>  
> 
>   ## Dial in details
>   Web: https://www.gotomeet.me/larskurth 
> 
>   You can also dial in using your phone.
>   Access Code: 906-886-965
> 
>   China (Toll Free): 4008 811084
>   Germany: +49 692 5736 7317
>   Poland (Toll Free): 00 800 1124759
>   United Kingdom: +44 330 221 0088
>   United States: +1 (571) 317-3129
> 
>   More phone numbers
>   Australia: +61 2 9087 3604
>   Austria: +43 7 2081 5427
>   Argentina (Toll Free): 0 800 444 3375
>   Bahrain (Toll Free): 800 81 111
>   Belarus (Toll Free): 8 820 0011 0400
>   Belgium: +32 28 93 7018
>   Brazil (Toll Free): 0 800 047 4906
>   Bulgaria (Toll Free): 00800 120 4417
>   Canada: +1 (647) 497-9391
>   Chile (Toll Free): 800 395 150
>   Colombia (Toll Free): 01 800 518 4483
>Czech Republic (Toll Free): 800 500448
>   Denmark: +45 32 72 03 82
>   Finland: +358 923 17 0568
>   France: +33 170 950 594
>   Greece (Toll Free): 00 800 4414 3838
>   Hong Kong (Toll Free): 30713169
>   Hungary (Toll Free): (06) 80 986 255
>   Iceland (Toll Free): 800 7204
>   India (Toll Free): 18002669272
>   Indonesia (Toll Free): 007 803 020 5375
>   Ireland: +353 15 360 728
>   Israel (Toll Free): 1 809 454 830
>   Italy: +39 0 247 92 13 01
>   Japan (Toll Free): 0 120 663 800
>   Korea, Republic of (Toll Free): 00798 14 207 4914
>   Luxembourg (Toll Free): 800 85158
>   Malaysia (Toll Free): 1 800 81 6854
>   Mexico (Toll Free): 01 800 522 1133
>   Netherlands: +31 207 941 377
>   New Zealand: +64 9 280 6302
>   Norway: +47 21 93 37 51
>   Panama (Toll Free): 00 800 226 7928
>   Peru (Toll Free): 0 800 77023
>   Philippines (Toll Free): 1 800 1110 1661
>   Portugal (Toll Free): 800 819 575
>   Romania (Toll Free): 0 800 410 029
>   Russian Federation (Toll Free): 8 800 100 6203
>   Saudi Arabia (Toll Free): 800 844 3633
>   Singapore (Toll Free): 18007231323
>   South Africa (Toll Free): 0 800 555 447
>   Spain: +34 932 75 2004
>   Sweden: +46 853 527 827
>   Switzerland: +41 225 4599 78
>   Taiwan (Toll Free): 0 800 666 854
>   Thailand (Toll Free): 001 800 011 023
>   Turkey (Toll Free): 00 800 4488 23683
>   Ukraine (Toll Free): 0 800 50 1733
>   United Arab Emirates (Toll Free): 800 044 40439
>   Uruguay (Toll Free): 0004 019 1018
>   Viet Nam (Toll Free): 122 80 481
> 
>   First GoToMeeting? Let's do a quick system check:
>   https://link.gotomeeting.com/system-check 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xenproject.org 
> https://lists.xenproject.org/mailman/listinfo/xen-devel 
>

Re: [Xen-devel] [PATCH v5 8/8] microcode: update microcode on cores in parallel

2019-02-13 Thread Chao Gao 
On Wed, Feb 13, 2019 at 12:20:51AM -0700, Jan Beulich wrote:
 On 13.02.19 at 03:30,  wrote:
>> On Tue, Feb 12, 2019 at 06:55:20AM -0700, Jan Beulich wrote:
>> On 12.02.19 at 14:25,  wrote:
 On Tue, Feb 12, 2019 at 05:51:41AM -0700, Jan Beulich wrote:
> >>> On 28.01.19 at 08:06,  wrote:
> > @@ -314,9 +310,7 @@ static int apply_microcode(unsigned int cpu)
> >  
> >  mc_intel = patch->data;
> >  BUG_ON(!mc_intel);
> > -
> > -/* serialize access to the physical write to MSR 0x79 */
> > -spin_lock_irqsave(_update_lock, flags);
> > +BUG_ON(local_irq_is_enabled());
> >  
> >  /* write microcode via MSR 0x79 */
> >  wrmsrl(MSR_IA32_UCODE_WRITE, (unsigned long)mc_intel->bits);
> > @@ -329,7 +323,6 @@ static int apply_microcode(unsigned int cpu)
> >  rdmsrl(MSR_IA32_UCODE_REV, msr_content);
> >  val[1] = (uint32_t)(msr_content >> 32);
> >  
> > -spin_unlock_irqrestore(_update_lock, flags);
> >  if ( val[1] != mc_intel->hdr.rev )
> >  {
> >  printk(KERN_ERR "microcode: CPU%d update from revision "
> 
> Am I understanding right that you now rely on upper layers in the
> call tree to avoid calling into here in parallel for two hyperthreads
> of the same core? I can't see how you avoid this situation during
> AP bringup, for example. Did I overlook anything in this regard?
 
 IIRC microcode update is done in the serialized part of AP bringup,
 before the call to smp_callin, which guarantees serialization.
>>>
>>>Hmm, yes, right now it is. But I'd call this "happens to be that way"
>>>rather than "is guaranteed to be that way" - prior to commit
>>>f97838bbd9 it did happen later.
>> 
>> How about employing another lock, "early_ucode_update_lock", to
>> guarantee serialization.
>> 
>> In particular, early_microcode_update_cpu() and microcode_resume_cpu()
>> will acquire this lock before ucode update.
>
>That's a (temporary) option, but would over-serialize things again.
>I was rather trying to hint towards a per-core lock.

To implement a per-core lock, a core should be conscious of its siblings.
Unfortunately, during AP bringup, ucode update is done prior to the
initialization of 'cpu_sibling_mask'.

Thanks
Chao

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v9 2/5] xen: introduce SYMBOLS_SUBTRACT and SYMBOLS_COMPARE

2019-02-13 Thread Jan Beulich 
>>> On 13.02.19 at 02:17,  wrote:
> On Tue, 12 Feb 2019, Jan Beulich wrote:
>> >>> On 12.02.19 at 13:01,  wrote:
>> > I would particularly welcome the opinion of hypervisor maintainers on
>> > my type safety point, below.
>> 
>> I agree with the requirements you put forward; I think I'd
>> prefer the inline function versions I had suggested (or
>> something similar) over macros though, not the least because
>> they come with "built-in" type safety, rather than grafted one
>> (by adding "pseudo" comparisons).
> 
> I don't mind the type checks in principle, I didn't add them to this
> version because, as I wrote in a previous email, with have occurrences
> of all three this possible calls:
> 
>   SYMBOL_COMPARE/SUBTRACT( symbol, symbol )
>   SYMBOL_COMPARE/SUBTRACT( symbol, non-symbol )
>   SYMBOL_COMPARE/SUBTRACT( non-symbol, symbol )
> 
> If you look through the patches you should be able to spot all three.
> As you know "non-symbol" and "symbol" are of different type:
> 
>   non-symbol would be like a "struct wombat*"
>   symbol would be like a "struct wombat[]"

These are declared types. Effective type when used as rvalue
(i.e. also when passed as arguments to functions) is struct
wombat * in both cases.

The important aspect (and new idea) Ian has been introducing
really is that the "end" symbols intentionally no longer be of
the same type as the start ones (but some type derived
thereof).

> However, it is not possible to have symbol as struct wombar[] and
> non-symbol as something entirely different like char*.
> 
> So, my question is: do we need three different variations of these
> macros for the types checks?
> 
> 
> I don't understand from IanJ email whether the suggestion is to change
> the type of all the linker symbols. If so, why are we doing this instead
> of the var.S approach? If we go and change the type of all the linker
> symbols in C-land, this series will become much bigger and at least as
> invasive as the var.S approach, but with added weird macros. It is kind
> of a lose - lose situation.
> 
> Similarly I would prefer to avoid Jan's proposed inline function approach
> because we have a few different array types for the linker symbols
> (vpci_register_init_t*, struct scheduler *, etc.), it looks far more
> work, and I am already wy over-budget for this series (as in 700%
> over budget). I would be very happy to "gift it" to somebody else
> willing to take it over :-)
> 
> Seriously, now that all the calls sites are marked appropriately and we
> all agree on the compare/subtract macro approach, it wouldn't be hard
> for somebody else to jump in and write the macros in their favorite way.
> Let me know if you would like to volunteer!

I've indeed been considering this already, as I expected the point
would come up sooner or later. Thing is though that, in particular
with Adrian not having replied at all so far, I'm still unconvinced that
we need to make this many changes (i.e. other than to work around
compiler deficiencies, which would boil down to using SYMBOL_HIDE()
from v7, but only in places where it is known certain compiler versions
might mis-optimize it, and with a clear reference to the involved
compiler bug/versions) at all. It's just that what we're now discussing
is the approach I have the least problem following _if_ such a global
"marking" of linker symbol uses ends up being necessary.

>> Furthermore - do we really need both a subtract and a compare
>> construct? The result subtract produces can be used for
>> comparison purposes as well, after all (just like all CPUs I know
>> details of implement [integer] compare as a subtract discarding
>> its numeric result, instead [or only] updating certain status flags).
> 
> No, we don't. In my first attempt I only had one macro. I am happy to
> follow your suggestion and keep only SUBTRACT.

Except that, as I think Ian has also suggested, DIFFERENCE() (or
SYMBOL_DIFFERENCE()) might be better, as it (hopefully)
reduces the connections to the - operator, and hence the risk
of possibly getting the argument order wrong. Otoh with the
type safety added wrong argument order would cause a
compile time error.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel