[Xen-devel] [xen-unstable test] 112416: regressions - trouble: blocked/broken/fail/pass

[osstest service owner]

flight 112416 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/112416/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-localmigrate/x10 fail REGR. vs. 
112286
 test-armhf-armhf-xl-credit2 16 guest-start/debian.repeat fail REGR. vs. 112286

Regressions which are regarded as allowable (not blocking):
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 112286
 build-arm64-pvops 2 hosts-allocate broken REGR. vs. 112286
 build-arm64   2 hosts-allocate broken REGR. vs. 112286

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-examine  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64-xsm   3 capture-logs  broken blocked in 112286
 build-arm64-pvops 3 capture-logs  broken blocked in 112286
 build-arm64   3 capture-logs  broken blocked in 112286
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeatfail  like 112274
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 112286
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-localmigrate/x10 fail like 112286
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 112286
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 112286
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 112286
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 112286
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 13 guest-saverestore   fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-ws16-amd64 13 guest-saverestore   fail never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 10 windows-installfail never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass

version targeted for testing:
 xen  b8029db62eb2a06a204a8e2b69437d0927bd1ac4
baseline version:
 xen  55924baf2211ddcf5ba8f702c9a4c07730e0c8e8

Last test of basis   

[Xen-devel] [linux-linus test] 112412: regressions - trouble: blocked/broken/fail/pass

[osstest service owner]

flight 112412 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/112412/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-ovmf-amd64  7 xen-boot fail REGR. vs. 110515
 test-amd64-amd64-examine  7 reboot   fail REGR. vs. 110515
 test-amd64-amd64-libvirt-vhd  7 xen-boot fail REGR. vs. 110515
 test-amd64-amd64-libvirt-pair 10 xen-boot/src_host   fail REGR. vs. 110515
 test-amd64-amd64-pair10 xen-boot/src_hostfail REGR. vs. 110515
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-boot fail REGR. vs. 
110515
 test-amd64-amd64-rumprun-amd64  7 xen-boot   fail REGR. vs. 110515
 test-amd64-amd64-pair11 xen-boot/dst_hostfail REGR. vs. 110515
 test-amd64-amd64-libvirt-pair 11 xen-boot/dst_host   fail REGR. vs. 110515
 test-amd64-amd64-xl-qemut-debianhvm-amd64  7 xen-bootfail REGR. vs. 110515
 test-amd64-amd64-qemuu-nested-intel  7 xen-boot  fail REGR. vs. 110515
 test-amd64-amd64-xl-pvh-intel  7 xen-bootfail REGR. vs. 110515
 test-amd64-amd64-pygrub   7 xen-boot fail REGR. vs. 110515
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-boot fail REGR. 
vs. 110515
 test-amd64-amd64-libvirt  7 xen-boot fail REGR. vs. 110515
 test-amd64-amd64-xl-qcow2 7 xen-boot fail REGR. vs. 110515
 test-amd64-amd64-i386-pvgrub  7 xen-boot fail REGR. vs. 110515
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-localmigrate/x10 fail REGR. vs. 
110515
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-localmigrate/x10 fail REGR. vs. 
110515
 test-amd64-amd64-amd64-pvgrub  7 xen-bootfail REGR. vs. 110515

Regressions which are regarded as allowable (not blocking):
 build-arm64   2 hosts-allocate broken REGR. vs. 110515
 build-arm64-pvops 2 hosts-allocate broken REGR. vs. 110515
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 110515

Tests which did not succeed, but are not blocking:
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-examine  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64-xsm   3 capture-logs  broken blocked in 110515
 build-arm64-pvops 3 capture-logs  broken blocked in 110515
 build-arm64   3 capture-logs  broken blocked in 110515
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 110515
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeatfail  like 110515
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 110515
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 110515
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 110515
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 110515
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 110515
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 13 guest-saverestore   fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 13 guest-saverestore   fail never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 

Re: [Xen-devel] [PATCH v2] xen: get rid of paravirt op adjust_exception_frame

[Andy Lutomirski]

On Tue, Aug 1, 2017 at 4:38 PM, Andrew Cooper  wrote:
> On 01/08/2017 20:45, Andy Lutomirski wrote:
>> Also, IMO it would be nice to fully finish the job.  Remaining steps are:
>>
>> 1. Unsuck the SYSCALL entries on Xen PV.
>> 2. Unsuck the SYENTER entry on Xen PV.
>> 3. Make a xen_nmi that's actually correct (should be trivial)
>>
>> #1 is here:
>>
>> https://git.kernel.org/pub/scm/linux/kernel/git/luto/linux.git/commit/?h=x86/entry_syscall=14fee348e3e86c994400d68085217d1232a637d6
>
> If the
>
> /* Zero-extending 32-bit regs, do not remove */
> movl%eax, %eax
>
> comments are to be believed, then the entry logic needs reordering
> slightly to:
>
> GLOBAL(entry_*_compat_after_hwframe)
> movl%eax, %eax/* Zero-extending 32-bit regs, do not remove */
> pushq%rax/* pt_regs->orig_ax */

D'oh, right.  Juergen, want to make that change?

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2] xen: get rid of paravirt op adjust_exception_frame

[Andrew Cooper]

On 01/08/2017 20:45, Andy Lutomirski wrote:
> Also, IMO it would be nice to fully finish the job.  Remaining steps are:
>
> 1. Unsuck the SYSCALL entries on Xen PV.
> 2. Unsuck the SYENTER entry on Xen PV.
> 3. Make a xen_nmi that's actually correct (should be trivial)
>
> #1 is here:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/luto/linux.git/commit/?h=x86/entry_syscall=14fee348e3e86c994400d68085217d1232a637d6

If the

/* Zero-extending 32-bit regs, do not remove */
movl%eax, %eax

comments are to be believed, then the entry logic needs reordering
slightly to:

GLOBAL(entry_*_compat_after_hwframe)
movl%eax, %eax/* Zero-extending 32-bit regs, do not remove */
pushq%rax/* pt_regs->orig_ax */

~Andrew

(It is unfortunate this can't be simplified to pushq %eax)

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/2] arm: smccc: handle SMCs/HVCs according to SMCCC

[Julien Grall]

Hi Edgar,

On 01/08/2017 22:22, Edgar E. Iglesias wrote:

On Tue, Aug 01, 2017 at 10:02:45PM +0100, Julien Grall wrote:

Hi,

On 01/08/2017 21:40, Stefano Stabellini wrote:

On Tue, 1 Aug 2017, Edgar E. Iglesias wrote:

On Tue, Aug 01, 2017 at 11:59:00AM +0100, Julien Grall wrote:

(+ Edgar, Mark, Dave)

Hi,


Hi Julien,

I'll share some thoughts based on our platforms.



On 14/06/17 15:10, Volodymyr Babchuk wrote:

SMCCC (SMC Call Convention) describes how to handle both HVCs and SMCs.
SMCCC states that both HVC and SMC are valid conduits to call to a different
firmware functions. Thus, for example PSCI calls can be made both by
SMC or HVC. Also SMCCC defines function number coding for such calls.
Besides functional calls there are query calls, which allows underling
OS determine version, UID and number of functions provided by service
provider.

This patch adds new file `smccc.c`, which handles both generic SMCs
and HVC according to SMC. At this moment it implements only one
service: Standard Hypervisor Service.

Standard Hypervisor Service only supports query calls, so caller can
ask about hypervisor UID and determine that it is XEN running.

This change allows more generic handling for SMCs and HVCs and it can
be easily extended to support new services and functions.


I have already reviewed the code and one thing I missed is how a domain will
know that Xen supports SMCCC.

Currently, Xen will:
- inject an undefined instruction for any SMC issued by a guest
- crash the guest (quite bad) for any unknown HCV #0

So a guest needs to be aware whether Xen supports SMCCC convention or not. I
am not aware of any bindings in the device-tree for doing that.


On our platforms, SW probes the DT for specific service classes and then
probes for specific versions via SMC calls using the standard Version FIDs.
If the DT does not specify the firmware node, I don't think any SMCs will be
issued but the guest may not be functional (as the firmware interface is
mandatory).

I don't know of a generic DT node/compat that tells guests if SMCC probing
is allowed or not. Perhaps there should be one, or there should be yet
another service specific one for Hypervisors. I don't know.

For example, these are the nodes we've got (PSCI and EEMI/SIP):
   psci {
   compatible = "arm,psci-0.2";
   method = "smc";
   };

   pmufw: firmware {
   compatible = "xlnx,zynqmp-pm";
   method = "smc";
   interrupt-parent = <>;
   interrupts = <0 35 4>;
   };

SW that does not have DT support, will either directly probe the SMC
interface or in some cases just assume it's there and use it.

ZynqMP-wise, Xen is in a little bit of an akward position by messing
guests up if they probe for non-existing SMC services but I don't think
it's that bad. Mainly because there's really very little ZynqMP guests
that need the firmware would be able todo without it.

For other platforms and services, I guess FW may very well be optional
and probing makes more sence. So getting support for gracefully returning
Unknown FID still important...


Indeed, but unfortunately older versions of Xen don't do that. That's
why I think we'll have to introduce a feature flag under the Xen
hypervisor node on device tree. The presence of the flag could mean "it
is safe to probe via SMC/HVC". I think it makes sense for the flag to be
Xen specific, because we are talking about Xen behavior. Applications
that know they are running on a new enough Xen can skip the check (this
is going to be the case in most embedded scenarios).


This is not Xen specific behavior. Per ARM ARM, SMC will be undefined when
EL3 is not present. Today Xen is behaving the same way as EL3 was not
existing on the platform.

So we need a generic way to say "SMC is supported and is SMCCC capable".


Would it be unthinkable to treat the lack of "Unknown FID" returns as a bug
and backport the "fix" and leave it at that?


At the moment, this is not an option for me. New Linux should work on 
any Xen we shipped in the past. We should really avoid to break that 
unless there are a strong reason too.


Plus that will not solve the problem that we may want support other 
convention in the future.


I think the way forward is to define a binding that will advertise this 
feature.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [qemu-mainline test] 112407: regressions - trouble: blocked/broken/fail/pass

[osstest service owner]

flight 112407 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/112407/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemuu-win7-amd64 10 windows-install   fail REGR. vs. 111765
 test-amd64-i386-xl-qemuu-ws16-amd64 10 windows-install   fail REGR. vs. 111765
 test-amd64-amd64-xl-qemuu-win7-amd64 10 windows-install  fail REGR. vs. 111765

Regressions which are regarded as allowable (not blocking):
 build-arm64-pvops 2 hosts-allocate broken REGR. vs. 111765
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 111765
 build-arm64   2 hosts-allocate broken REGR. vs. 111765

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64   3 capture-logs  broken blocked in 111765
 build-arm64-pvops 3 capture-logs  broken blocked in 111765
 build-arm64-xsm   3 capture-logs  broken blocked in 111765
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 111765
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 111765
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeatfail  like 111765
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 111765
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 111765
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass

version targeted for testing:
 qemuu5619c179057e24195ff19c8fe6d6a6cbcb16ed28
baseline version:
 qemuu31fe1c414501047cbb91b695bdccc0068496dcf6

Last test of basis   111765  2017-07-13 10:20:16 Z   19 days
Failing since111790  2017-07-14 04:20:46 Z   18 days   27 attempts
Testing same since   112407  2017-08-01 08:19:38 Z0 days1 attempts


People who touched revisions under test:
  Alex Bennée 
  Alex Williamson 
  Alexander Graf 
  Alexey G 
  Alexey Gerasimenko 
  Alexey Kardashevskiy 
  Alistair Francis 

[Xen-devel] [ovmf baseline-only test] 71929: all pass

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 71929 ovmf real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/71929/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf c1acb0f9b8222a97d2ad72dbebbcefc214d9ce03
baseline version:
 ovmf c65df5d9a14331d2b6d583359f1cf88c3b710d34

Last test of basis71925  2017-08-01 13:47:14 Z0 days
Testing same since71929  2017-08-01 19:47:26 Z0 days1 attempts


People who touched revisions under test:
  Star Zeng 
  Yonghong Zhu 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xs.citrite.net/~osstest/testlogs/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


commit c1acb0f9b8222a97d2ad72dbebbcefc214d9ce03
Author: Star Zeng 
Date:   Fri Jul 28 10:05:19 2017 +0800

MdeModulePkg FirmwarePerfPei: Remove SEC performance data getting code

Current SEC performance data getting code in FirmwarePerformancePei
may get wrong SEC performance data if FirmwarePerformancePei executes
after memory discovered.

And as SecCore has added SecPerformancePpiCallBack to get SEC performance
data and build HOB to convey the SEC performance data to DXE phase.

This patch is to remove the SEC performance data getting code in
FirmwarePerformancePei.

Cc: Liming Gao 
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Star Zeng 
Reviewed-by: Liming Gao 

commit f2e70629742fe1d387186d9f2bb5cdb338b46009
Author: Star Zeng 
Date:   Fri Jul 28 10:05:08 2017 +0800

UefiCpuPkg SecCore: Add SecPerformancePpiCallBack

Add SecPerformancePpiCallBack to get SEC performance data and
build HOB to convey the SEC performance data to DXE phase.

Cc: Liming Gao 
Cc: Jeff Fan 
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Star Zeng 
Reviewed-by: Jeff Fan 
Reviewed-by: Liming Gao 

commit 9e9ca2100f22be29f1a53129d741f4305ff34a71
Author: Star Zeng 
Date:   Fri Jul 28 22:13:00 2017 +0800

UefiCpuPkg SecCore: Adjust PeiTemporaryRamBase to be 8byte aligned

As HOB which has 8byte aligned requirement will be built based on them
in PEI phase.

Cc: Liming Gao 
Cc: Jeff Fan 
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Star Zeng 
Reviewed-by: Jeff Fan 
Reviewed-by: Liming Gao 

commit 884200f95f7fbf7ffad8b304fdfb331570c74677
Author: Star Zeng 
Date:   Wed Jul 26 15:55:26 2017 +0800

MdeModulePkg PeiCore: Handle notification PPI from SEC

InstallPpi() will be used for normal PPI in PPI list from SEC,
and NotifyPpi() will be used for notification PPI in PPI list from SEC.

Cc: Liming Gao 
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Star Zeng 
Reviewed-by: Liming Gao 

commit cc9488476fe7dc551305eb9aaa5aab09b1df5581
Author: Star Zeng 
Date:   Wed Jul 26 11:19:22 2017 +0800

MdePkg PiPeiCis.h: Add description for notification PPI from SEC

This patch is to follow latest (>= 1.5) PI spec to add description
for notification PPI from SEC

Cc: Liming Gao 
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Star Zeng 
Reviewed-by: Liming Gao 

commit 

Re: [Xen-devel] [PATCH 1/2] arm: smccc: handle SMCs/HVCs according to SMCCC

[Edgar E. Iglesias]

On Tue, Aug 01, 2017 at 10:02:45PM +0100, Julien Grall wrote:
> Hi,
> 
> On 01/08/2017 21:40, Stefano Stabellini wrote:
> >On Tue, 1 Aug 2017, Edgar E. Iglesias wrote:
> >>On Tue, Aug 01, 2017 at 11:59:00AM +0100, Julien Grall wrote:
> >>>(+ Edgar, Mark, Dave)
> >>>
> >>>Hi,
> >>
> >>Hi Julien,
> >>
> >>I'll share some thoughts based on our platforms.
> >>
> >>
> >>>On 14/06/17 15:10, Volodymyr Babchuk wrote:
> SMCCC (SMC Call Convention) describes how to handle both HVCs and SMCs.
> SMCCC states that both HVC and SMC are valid conduits to call to a 
> different
> firmware functions. Thus, for example PSCI calls can be made both by
> SMC or HVC. Also SMCCC defines function number coding for such calls.
> Besides functional calls there are query calls, which allows underling
> OS determine version, UID and number of functions provided by service
> provider.
> 
> This patch adds new file `smccc.c`, which handles both generic SMCs
> and HVC according to SMC. At this moment it implements only one
> service: Standard Hypervisor Service.
> 
> Standard Hypervisor Service only supports query calls, so caller can
> ask about hypervisor UID and determine that it is XEN running.
> 
> This change allows more generic handling for SMCs and HVCs and it can
> be easily extended to support new services and functions.
> >>>
> >>>I have already reviewed the code and one thing I missed is how a domain 
> >>>will
> >>>know that Xen supports SMCCC.
> >>>
> >>>Currently, Xen will:
> >>>   - inject an undefined instruction for any SMC issued by a guest
> >>>   - crash the guest (quite bad) for any unknown HCV #0
> >>>
> >>>So a guest needs to be aware whether Xen supports SMCCC convention or not. 
> >>>I
> >>>am not aware of any bindings in the device-tree for doing that.
> >>
> >>On our platforms, SW probes the DT for specific service classes and then
> >>probes for specific versions via SMC calls using the standard Version FIDs.
> >>If the DT does not specify the firmware node, I don't think any SMCs will be
> >>issued but the guest may not be functional (as the firmware interface is
> >>mandatory).
> >>
> >>I don't know of a generic DT node/compat that tells guests if SMCC probing
> >>is allowed or not. Perhaps there should be one, or there should be yet
> >>another service specific one for Hypervisors. I don't know.
> >>
> >>For example, these are the nodes we've got (PSCI and EEMI/SIP):
> >>psci {
> >>compatible = "arm,psci-0.2";
> >>method = "smc";
> >>};
> >>
> >>pmufw: firmware {
> >>compatible = "xlnx,zynqmp-pm";
> >>method = "smc";
> >>interrupt-parent = <>;
> >>interrupts = <0 35 4>;
> >>};
> >>
> >>SW that does not have DT support, will either directly probe the SMC
> >>interface or in some cases just assume it's there and use it.
> >>
> >>ZynqMP-wise, Xen is in a little bit of an akward position by messing
> >>guests up if they probe for non-existing SMC services but I don't think
> >>it's that bad. Mainly because there's really very little ZynqMP guests
> >>that need the firmware would be able todo without it.
> >>
> >>For other platforms and services, I guess FW may very well be optional
> >>and probing makes more sence. So getting support for gracefully returning
> >>Unknown FID still important...
> >
> >Indeed, but unfortunately older versions of Xen don't do that. That's
> >why I think we'll have to introduce a feature flag under the Xen
> >hypervisor node on device tree. The presence of the flag could mean "it
> >is safe to probe via SMC/HVC". I think it makes sense for the flag to be
> >Xen specific, because we are talking about Xen behavior. Applications
> >that know they are running on a new enough Xen can skip the check (this
> >is going to be the case in most embedded scenarios).
> 
> This is not Xen specific behavior. Per ARM ARM, SMC will be undefined when
> EL3 is not present. Today Xen is behaving the same way as EL3 was not
> existing on the platform.
> 
> So we need a generic way to say "SMC is supported and is SMCCC capable".

Would it be unthinkable to treat the lack of "Unknown FID" returns as a bug
and backport the "fix" and leave it at that?

Cheers,
Edgar

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/2] arm: smccc: handle SMCs/HVCs according to SMCCC

[Julien Grall]

Hi,

On 01/08/2017 21:40, Stefano Stabellini wrote:

On Tue, 1 Aug 2017, Edgar E. Iglesias wrote:

On Tue, Aug 01, 2017 at 11:59:00AM +0100, Julien Grall wrote:

(+ Edgar, Mark, Dave)

Hi,


Hi Julien,

I'll share some thoughts based on our platforms.



On 14/06/17 15:10, Volodymyr Babchuk wrote:

SMCCC (SMC Call Convention) describes how to handle both HVCs and SMCs.
SMCCC states that both HVC and SMC are valid conduits to call to a different
firmware functions. Thus, for example PSCI calls can be made both by
SMC or HVC. Also SMCCC defines function number coding for such calls.
Besides functional calls there are query calls, which allows underling
OS determine version, UID and number of functions provided by service
provider.

This patch adds new file `smccc.c`, which handles both generic SMCs
and HVC according to SMC. At this moment it implements only one
service: Standard Hypervisor Service.

Standard Hypervisor Service only supports query calls, so caller can
ask about hypervisor UID and determine that it is XEN running.

This change allows more generic handling for SMCs and HVCs and it can
be easily extended to support new services and functions.


I have already reviewed the code and one thing I missed is how a domain will
know that Xen supports SMCCC.

Currently, Xen will:
- inject an undefined instruction for any SMC issued by a guest
- crash the guest (quite bad) for any unknown HCV #0

So a guest needs to be aware whether Xen supports SMCCC convention or not. I
am not aware of any bindings in the device-tree for doing that.


On our platforms, SW probes the DT for specific service classes and then
probes for specific versions via SMC calls using the standard Version FIDs.
If the DT does not specify the firmware node, I don't think any SMCs will be
issued but the guest may not be functional (as the firmware interface is
mandatory).

I don't know of a generic DT node/compat that tells guests if SMCC probing
is allowed or not. Perhaps there should be one, or there should be yet
another service specific one for Hypervisors. I don't know.

For example, these are the nodes we've got (PSCI and EEMI/SIP):
psci {
compatible = "arm,psci-0.2";
method = "smc";
};

pmufw: firmware {
compatible = "xlnx,zynqmp-pm";
method = "smc";
interrupt-parent = <>;
interrupts = <0 35 4>;
};

SW that does not have DT support, will either directly probe the SMC
interface or in some cases just assume it's there and use it.

ZynqMP-wise, Xen is in a little bit of an akward position by messing
guests up if they probe for non-existing SMC services but I don't think
it's that bad. Mainly because there's really very little ZynqMP guests
that need the firmware would be able todo without it.

For other platforms and services, I guess FW may very well be optional
and probing makes more sence. So getting support for gracefully returning
Unknown FID still important...


Indeed, but unfortunately older versions of Xen don't do that. That's
why I think we'll have to introduce a feature flag under the Xen
hypervisor node on device tree. The presence of the flag could mean "it
is safe to probe via SMC/HVC". I think it makes sense for the flag to be
Xen specific, because we are talking about Xen behavior. Applications
that know they are running on a new enough Xen can skip the check (this
is going to be the case in most embedded scenarios).


This is not Xen specific behavior. Per ARM ARM, SMC will be undefined 
when EL3 is not present. Today Xen is behaving the same way as EL3 was 
not existing on the platform.


So we need a generic way to say "SMC is supported and is SMCCC capable".

Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/2] arm: smccc: handle SMCs/HVCs according to SMCCC

[Stefano Stabellini]

On Tue, 1 Aug 2017, Edgar E. Iglesias wrote:
> On Tue, Aug 01, 2017 at 11:59:00AM +0100, Julien Grall wrote:
> > (+ Edgar, Mark, Dave)
> > 
> > Hi,
> 
> Hi Julien,
> 
> I'll share some thoughts based on our platforms.
> 
> 
> > On 14/06/17 15:10, Volodymyr Babchuk wrote:
> > >SMCCC (SMC Call Convention) describes how to handle both HVCs and SMCs.
> > >SMCCC states that both HVC and SMC are valid conduits to call to a 
> > >different
> > >firmware functions. Thus, for example PSCI calls can be made both by
> > >SMC or HVC. Also SMCCC defines function number coding for such calls.
> > >Besides functional calls there are query calls, which allows underling
> > >OS determine version, UID and number of functions provided by service
> > >provider.
> > >
> > >This patch adds new file `smccc.c`, which handles both generic SMCs
> > >and HVC according to SMC. At this moment it implements only one
> > >service: Standard Hypervisor Service.
> > >
> > >Standard Hypervisor Service only supports query calls, so caller can
> > >ask about hypervisor UID and determine that it is XEN running.
> > >
> > >This change allows more generic handling for SMCs and HVCs and it can
> > >be easily extended to support new services and functions.
> > 
> > I have already reviewed the code and one thing I missed is how a domain will
> > know that Xen supports SMCCC.
> > 
> > Currently, Xen will:
> > - inject an undefined instruction for any SMC issued by a guest
> > - crash the guest (quite bad) for any unknown HCV #0
> > 
> > So a guest needs to be aware whether Xen supports SMCCC convention or not. I
> > am not aware of any bindings in the device-tree for doing that.
> 
> On our platforms, SW probes the DT for specific service classes and then
> probes for specific versions via SMC calls using the standard Version FIDs.
> If the DT does not specify the firmware node, I don't think any SMCs will be
> issued but the guest may not be functional (as the firmware interface is
> mandatory).
> 
> I don't know of a generic DT node/compat that tells guests if SMCC probing
> is allowed or not. Perhaps there should be one, or there should be yet
> another service specific one for Hypervisors. I don't know.
> 
> For example, these are the nodes we've got (PSCI and EEMI/SIP):
> psci {
> compatible = "arm,psci-0.2";
> method = "smc";
> };
> 
> pmufw: firmware {
> compatible = "xlnx,zynqmp-pm";
> method = "smc";
> interrupt-parent = <>;
> interrupts = <0 35 4>;
> };
> 
> SW that does not have DT support, will either directly probe the SMC
> interface or in some cases just assume it's there and use it.
> 
> ZynqMP-wise, Xen is in a little bit of an akward position by messing
> guests up if they probe for non-existing SMC services but I don't think
> it's that bad. Mainly because there's really very little ZynqMP guests
> that need the firmware would be able todo without it.
> 
> For other platforms and services, I guess FW may very well be optional
> and probing makes more sence. So getting support for gracefully returning
> Unknown FID still important...

Indeed, but unfortunately older versions of Xen don't do that. That's
why I think we'll have to introduce a feature flag under the Xen
hypervisor node on device tree. The presence of the flag could mean "it
is safe to probe via SMC/HVC". I think it makes sense for the flag to be
Xen specific, because we are talking about Xen behavior. Applications
that know they are running on a new enough Xen can skip the check (this
is going to be the case in most embedded scenarios).


> > The other issue is not all the firmware may be SMCCC capable. We may want in
> > the future to support other convention to allow baremetal OS running on Xen.
> > This means a guest should be able to detect the convention used.
> 
> Perhaps this could be done by injecting DT fragments like we do for 
> passthrough?

Ideally the firmware protocol could be detected by HVC/SMC probing (once
that is deemed to be safe via device tree). However, if it is not
possible to establish the right protocol via HVC/SMC probing, then the
supported firmware protocol could be advertised via device tree, ideally
in a non-Xen specific fashion.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v1 0/1] xen/arm: zynqmp: Disable PCIe

[Stefano Stabellini]

On Tue, 1 Aug 2017, Edgar E. Iglesias wrote:
> On Mon, Jul 31, 2017 at 11:11:39PM +0100, Julien Grall wrote:
> > 
> > 
> > On 31/07/2017 20:37, Edgar E. Iglesias wrote:
> > >From: "Edgar E. Iglesias" 
> > >
> > >Hi,
> > 
> > Hi Edgar,
> > 
> > 
> > >We're seeing panics in dom0 with PCIe enabled due to what seems
> > >to be wrongly created mappings by Xen. With older kernels we
> > >didn't see the panics but PCIe wasn't functional in dom0.
> > >
> > >This disables the PCIe nodes on the ZynqMP until Xen/ARM gets
> > >more PCIe support.
> > 
> > I feel a bit sad to ack a patch disabling PCIe in the ZynqMP.
> > 
> > Before doing that. Can you describe what is the exact problem with Xen? It
> > might be possible that we don't parse correctly the device-tree.
> 
> Yes, it was related to the DTB since we had a workaround with a modified
> DTB.
> 
> Anyway, I was to quick with sending out the patch.
> 
> We've been carrying this for a while and your questions prompted me
> to have a look again and dig out the details. Turns out I can't reproduce it
> with our latest kernel and the latest Xen. I don't know what fixed
> it yet but PCIe works ATM both on QEMU models and on real HW.
> 
> I don't think it ever has before without modified DTBs.

Fantastic!!


> So please ignore this patch for now :-)


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2] xen: get rid of paravirt op adjust_exception_frame

[Andy Lutomirski]

On Tue, Aug 1, 2017 at 3:39 AM, Juergen Gross  wrote:
> When running as Xen pv-guest the exception frame on the stack contains
> %r11 and %rcx additional to the other data pushed by the processor.
>
> Instead of having a paravirt op being called for each exception type
> prepend the Xen specific code to each exception entry. When running as
> Xen pv-guest just use the exception entry with prepended instructions,
> otherwise use the entry without the Xen specific code.
>
> Signed-off-by: Juergen Gross 
> ---
>  arch/x86/entry/entry_64.S | 22 ++
>  arch/x86/entry/entry_64_compat.S  |  1 -
>  arch/x86/include/asm/desc.h   | 16 ++
>  arch/x86/include/asm/paravirt.h   |  5 ---
>  arch/x86/include/asm/paravirt_types.h |  4 ---
>  arch/x86/include/asm/proto.h  |  3 ++
>  arch/x86/include/asm/traps.h  | 51 +--
>  arch/x86/kernel/asm-offsets_64.c  |  1 -
>  arch/x86/kernel/paravirt.c|  3 --
>  arch/x86/kernel/traps.c   | 57 
> ++-
>  arch/x86/xen/enlighten_pv.c   | 16 +-
>  arch/x86/xen/irq.c|  3 --
>  arch/x86/xen/xen-asm_64.S | 45 ---
>  arch/x86/xen/xen-ops.h|  1 -
>  14 files changed, 147 insertions(+), 81 deletions(-)
>
> diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
> index a9a8027a6c0e..602bcf68a32c 100644
> --- a/arch/x86/entry/entry_64.S
> +++ b/arch/x86/entry/entry_64.S
> @@ -745,7 +745,6 @@ ENTRY(\sym)
> .endif
>
> ASM_CLAC
> -   PARAVIRT_ADJUST_EXCEPTION_FRAME
>
> .ifeq \has_error_code
> pushq   $-1 /* ORIG_RAX: no syscall to 
> restart */
> @@ -901,7 +900,7 @@ ENTRY(do_softirq_own_stack)
>  END(do_softirq_own_stack)
>
>  #ifdef CONFIG_XEN
> -idtentry xen_hypervisor_callback xen_do_hypervisor_callback has_error_code=0
> +idtentry hypervisor_callback xen_do_hypervisor_callback has_error_code=0
>
>  /*
>   * A note on the "critical region" in our callback handler.
> @@ -967,8 +966,6 @@ ENTRY(xen_failsafe_callback)
> movq8(%rsp), %r11
> addq$0x30, %rsp
> pushq   $0  /* RIP */
> -   pushq   %r11
> -   pushq   %rcx
> jmp general_protection
>  1: /* Segment mismatch => Category 1 (Bad segment). Retry the IRET. */
> movq(%rsp), %rcx
> @@ -997,9 +994,8 @@ idtentry int3   do_int3   
>   has_error_code=0paranoid=1 shift_ist=DEBUG_STACK
>  idtentry stack_segment do_stack_segmenthas_error_code=1
>
>  #ifdef CONFIG_XEN
> -idtentry xen_debug do_debughas_error_code=0
> -idtentry xen_int3  do_int3 has_error_code=0
> -idtentry xen_stack_segment do_stack_segmenthas_error_code=1
> +idtentry xendebug  do_debughas_error_code=0
> +idtentry xenint3   do_int3 has_error_code=0
>  #endif
>
>  idtentry general_protectiondo_general_protection   has_error_code=1
> @@ -1161,18 +1157,6 @@ END(error_exit)
>  /* Runs on exception stack */
>  ENTRY(nmi)
> /*
> -* Fix up the exception frame if we're on Xen.
> -* PARAVIRT_ADJUST_EXCEPTION_FRAME is guaranteed to push at most
> -* one value to the stack on native, so it may clobber the rdx
> -* scratch slot, but it won't clobber any of the important
> -* slots past it.
> -*
> -* Xen is a different story, because the Xen frame itself overlaps
> -* the "NMI executing" variable.
> -*/

Based on Andrew Cooper's email, it sounds like this function is just
straight-up broken on Xen PV.  Maybe change the comment to "XXX:
broken on Xen PV" or similar.

> +#if defined(CONFIG_X86_64) && defined(CONFIG_XEN_PV)
> +#define pv_trap_entry(name) (void *)(xen_pv_domain() ? xen_ ## name : name)
> +#else
> +#define pv_trap_entry(name) (void *)(name)
> +#endif
> +

Seems reasonable to me.

>  #ifdef CONFIG_X86_64
>
>  static inline void pack_gate(gate_desc *gate, unsigned type, unsigned long 
> func,
> @@ -482,6 +490,14 @@ static inline void _set_gate(int gate, unsigned type, 
> void *addr,
> 0, 0, __KERNEL_CS); \
> } while (0)
>
> +#define set_intr_gate_pv(n, addr)  \
> +   do {\
> +   set_intr_gate_notrace(n, pv_trap_entry(addr));  \
> +   _trace_set_gate(n, GATE_INTERRUPT,  \
> +   pv_trap_entry(trace_##addr),\
> +   0, 0, __KERNEL_CS); \
> +   } while (0)

Any reason this can't be set_intr_gate((n), 

[Xen-devel] [PATCH v4] xen: rtds: only tickle non-already tickled CPUs

[Meng Xu]

When more than one idle VCPUs that have the same PCPU as their
previous running core invoke runq_tickle(), they will tickle the same
PCPU. The tickled PCPU will only pick at most one VCPU, i.e., the
highest-priority one, to execute. The other VCPUs will not be
scheduled for a period, even when there is an idle core, making these
VCPUs unnecessarily starve for one period.

Therefore, always make sure that we only tickle PCPUs that have not
been tickled already.

Signed-off-by: Haoran Li 
Signed-off-by: Meng Xu 

---
The initial discussion of this patch can be found at
https://lists.xenproject.org/archives/html/xen-devel/2017-02/msg02857.html

Changes in v4:
1) Take Dario's suggestions:
   Search the new->cpu first for the cpu to tickle.
   This get rid of the if statement in previous versions.
2) Reword the comments and commit messages.
3) Rebased on staging branch.

Issues in v2 and v3:
Did not rebase on the latest staging branch.
Did not solve the comments/issues in v1.
Please ignore the v2 and v3.
---
 xen/common/sched_rt.c | 29 ++---
 1 file changed, 14 insertions(+), 15 deletions(-)

diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c
index 39f6bee..5fec95f 100644
--- a/xen/common/sched_rt.c
+++ b/xen/common/sched_rt.c
@@ -1147,9 +1147,9 @@ rt_vcpu_sleep(const struct scheduler *ops, struct vcpu 
*vc)
  * Called by wake() and context_saved()
  * We have a running candidate here, the kick logic is:
  * Among all the cpus that are within the cpu affinity
- * 1) if the new->cpu is idle, kick it. This could benefit cache hit
- * 2) if there are any idle vcpu, kick it.
- * 3) now all pcpus are busy;
+ * 1) if there are any idle vcpu, kick it.
+  For cache benefit,we first search new->cpu.
+ * 2) now all pcpus are busy;
  *among all the running vcpus, pick lowest priority one
  *if snext has higher priority, kick it.
  *
@@ -1177,17 +1177,13 @@ runq_tickle(const struct scheduler *ops, struct rt_vcpu 
*new)
 cpumask_and(_tickled, online, new->vcpu->cpu_hard_affinity);
 cpumask_andnot(_tickled, _tickled, >tickled);
 
-/* 1) if new's previous cpu is idle, kick it for cache benefit */
-if ( is_idle_vcpu(curr_on_cpu(new->vcpu->processor)) )
-{
-SCHED_STAT_CRANK(tickled_idle_cpu);
-cpu_to_tickle = new->vcpu->processor;
-goto out;
-}
-
-/* 2) if there are any idle pcpu, kick it */
-/* The same loop also find the one with lowest priority */
-for_each_cpu(cpu, _tickled)
+/*
+ * 1) If there are any idle vcpu, kick it.
+ *For cache benefit,we first search new->cpu.
+ *The same loop also find the one with lowest priority.
+ */
+cpu = cpumask_test_or_cycle(new->vcpu->processor, _tickled);
+while ( cpu!= nr_cpu_ids )
 {
 iter_vc = curr_on_cpu(cpu);
 if ( is_idle_vcpu(iter_vc) )
@@ -1200,9 +1196,12 @@ runq_tickle(const struct scheduler *ops, struct rt_vcpu 
*new)
 if ( latest_deadline_vcpu == NULL ||
  iter_svc->cur_deadline > latest_deadline_vcpu->cur_deadline )
 latest_deadline_vcpu = iter_svc;
+
+cpumask_clear_cpu(cpu, _tickled);
+cpu = cpumask_cycle(cpu, _tickled);
 }
 
-/* 3) candicate has higher priority, kick out lowest priority vcpu */
+/* 2) candicate has higher priority, kick out lowest priority vcpu */
 if ( latest_deadline_vcpu != NULL &&
  new->cur_deadline < latest_deadline_vcpu->cur_deadline )
 {
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v6] x86/monitor: Notify monitor if an emulation fails.

[Razvan Cojocaru]

On 07/25/2017 08:40 PM, Razvan Cojocaru wrote:
> On 07/18/2017 01:20 PM, Razvan Cojocaru wrote:
>> On 07/18/2017 01:09 PM, Andrew Cooper wrote:
>>> On 18/07/17 10:37, Petre Pircalabu wrote:
 If case of a vm_event with the emulate_flags set, if the instruction
 cannot be emulated, the monitor should be notified instead of directly
 injecting a hw exception.
 This behavior can be used to re-execute an instruction not supported by
 the emulator using the real processor (e.g. altp2m) instead of just
 crashing.

 Signed-off-by: Petre Pircalabu 
>>>
>>> There are many situations which end up failing an emulation with
>>> UNHANDLEABLE.
>>>
>>> What exact scenario are you looking to catch?  Is it just instructions
>>> which aren't implemented in the emulator?
>>
>> Instructions that are not implemented in the emulator are our main use
>> case for this, yes. In which case, we'd like a chance to be able to
>> single-step them (using altp2m), so that the guest will continue to run
>> even with an incomplete emulator.
>>
>> We don't care about instructions that would have failed to run in both
>> scenarios (emulated or single-stepped). I'm not sure if there are other
>> cases in which an instruction, although supported by the emulator, would
>> fail emulation but pass single-stepping.
> 
> Is there further action required on our part the get this patch
> commited? FWIW, while not ideal, from our perspective trying to
> single-step an instruction that was UNHANDLEABLE when attempting to
> emulate it is acceptable (even if UNHANDLEABLE doesn't mean that the
> instruction is not supported by the emulator). At worst it won't run
> when single-stepped either, and that'll be that.
Since this seems to be blocked as-is, I propose transforming this patch
into a series, with one patch adding a new return code specifically for
unsupported instructions (X86_EMUL_UNIMPLEMENTED or
X86_EMUL_UNSUPPORTED?), and this patch sending the vm_event out only for
that. (In which case the event's name should probably change as well to
reflect the name of the new error code.)

Thoughts?


Thanks,
Razvan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [ovmf test] 112415: all pass - PUSHED

[osstest service owner]

flight 112415 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/112415/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf c1acb0f9b8222a97d2ad72dbebbcefc214d9ce03
baseline version:
 ovmf c65df5d9a14331d2b6d583359f1cf88c3b710d34

Last test of basis   112404  2017-08-01 02:49:30 Z0 days
Testing same since   112415  2017-08-01 13:50:01 Z0 days1 attempts


People who touched revisions under test:
  Star Zeng 
  Yonghong Zhu 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=ovmf
+ revision=c1acb0f9b8222a97d2ad72dbebbcefc214d9ce03
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push ovmf 
c1acb0f9b8222a97d2ad72dbebbcefc214d9ce03
+ branch=ovmf
+ revision=c1acb0f9b8222a97d2ad72dbebbcefc214d9ce03
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=ovmf
+ xenbranch=xen-unstable
+ '[' xovmf = xlinux ']'
+ linuxbranch=
+ '[' x = x ']'
+ qemuubranch=qemu-upstream-unstable
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable
+ prevxenbranch=xen-4.9-testing
+ '[' xc1acb0f9b8222a97d2ad72dbebbcefc214d9ce03 = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/xtf.git
++ : osst...@xenbits.xen.org:/home/xen/git/xtf.git
++ : git://xenbits.xen.org/xtf.git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/rumprun.git
++ : git://git.seabios.org/seabios.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/seabios.git
++ : git://xenbits.xen.org/osstest/seabios.git
++ : https://github.com/tianocore/edk2.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/linux-firmware.git
++ : osst...@xenbits.xen.org:/home/osstest/ext/linux-firmware.git
++ : 

Re: [Xen-devel] [PATCH 5/5] xen: RCU: avoid busy waiting until the end of grace period.

[Stefano Stabellini]

On Tue, 1 Aug 2017, Dario Faggioli wrote:
> On Mon, 2017-07-31 at 16:58 -0700, Stefano Stabellini wrote:
> > On Tue, 1 Aug 2017, Dario Faggioli wrote:
> > > On Mon, 2017-07-31 at 14:20 -0700, Stefano Stabellini wrote:
> > > > On Thu, 27 Jul 2017, Dario Faggioli wrote:
> > > > > 
> > > > > diff --git a/xen/common/rcupdate.c b/xen/common/rcupdate.c
> > > > > index f0fdc87..4586f2a 100644
> > > > > --- a/xen/common/rcupdate.c
> > > > > +++ b/xen/common/rcupdate.c
> > > > > @@ -84,8 +84,14 @@ struct rcu_data {
> > > > >  int cpu;
> > > > >  struct rcu_head barrier;
> > > > >  longlast_rs_qlen; /* qlen during the last
> > > > > resched */
> > > > > +
> > > > > +/* 3) idle CPUs handling */
> > > > > +struct timer idle_timer;
> > > > > +bool idle_timer_active;
> > > > >  };
> > > > >  
> > > > > +#define RCU_IDLE_TIMER_PERIOD MILLISECS(10)
> > > > 
> > > > Isn't this a bit too short? How is it chosen?
> > > > 
> > > 
> > > What makes you think it's short?
> > 
> > In terms of power saving and CPU sleep states, 10ms is not much to
> > sleep
> > for. I wonder if there are any power saving benefits in sleeping for
> > only 10ms (especially on x86 where entering and exiting CPU sleep
> > states
> > takes longer, to be confirmed).
> >
> I *think* we should be fine with, say, 100ms. But that's again,
> guess/rule of thumb, nothing more than that. And, TBH, I'm not even
> sure what a good experiment/benchmark would be, to assess whether a
> particular value is good or bad. :-/

Given the description below, it's possible that the new timer has to
fire several times before the callback can be finally invoked, right?

I would make some measurements to check how many times the timer has to
fire (how long we actually need to wait before calling the callback) in
various scenarios. Then I would choose a value to minimize the number of
unnecessary wake-ups.


> >   We might as well do the thing we need
> > to do immediately? I guess we cannot do that?
> >
> You're guessing correct, we can't. It's indeed a bit tricky, and it
> took it a little bit to me as well to figure all of it out properly.
> 
> Basically, let's say that, at time t1, on CPU1, someone calls
> call_rcu(). The situation on the other CPUs is: CPU0 busy; CPU2 idle
> (no timer pending); CPU3 busy.
> 
> So, a new grace period starts, and its exact end will be when CPU0,
> CPU1 and CPU3 have quiesced once (in Xen, quiescence means: "going
> through do_softirq()").
> 
> But RCU it's a passive mechanism, i.e., we rely on each CPU coming to
> the RCU core logic, and tell <>.
> Let's say that CPU0 quiesces at time t2 > t1. CPU1 quiesces at time
> t3 > t2, and goes fully idle (no timer pending). CPU3 quiesces at time
> t4 > t3. Now, at time t4, CPU1 can actually invoke the callbeck, queued
> at time t1, from within call_rcu().
> 
> This patch series solves two problems, of our current RCU
> implementation:
> 
> 1) right now, we don't only wait for CPU0, CPU1 and CPU3, we also wait 
>for CPU2. But since CPU2 is fully idle, it just won't bother 
>telling RCU that it has quiesced (well, on x86, that would actually 
>happen at some point, while on ARM, it is really possible that this 
>never happens!). This is solved in patch 3, by introducing the 
>cpumask;
> 
> 2) now (after patch 3) we know that we just can avoid waiting for 
>CPU2. Good. But at time t4, when CPU3 quiesces, marking the end of
>the grace period, how would CPU1 --which is fully idle-- know that
>it can now safely invoke the callback? Again, RCU is passive, so it
>relies on CPU1 to figure that out on its own, next time it wakes
>up, e.g., because of the periodic tick timer. But we don't have a
>periodic tick timer! Well, this again means that, on x86, CPU1 will
>actually figure that out at some (unpredictable) point in future.
>On ARM, not so much. The purpose of the timer in this patch is to
>make sure it always will.
>In fact, with patches 4 and 5 applied, at time t3, we let CPU1 go 
>idle, but we program the timer. It will fire at t3+T (with T=10ms, 
>right now). When this happens, if t3+T > t4, CPU1 invokes the
>callback, and we're done. If not (and CPU1 is still idle) we retry
>in another T.
> 
> So, when you say "immediately", *when* do you actually mean?
> 
> We can't invoke the callback at t3 (i.e., immediately after CPU1
> quiesces), because we need to wait for CPU3 to do the same.
> 
> We can't invoke the callback when CPU3 quiesces, at t4 (i.e.,
> immediately after the grace period ends) either, because the callback
> it's on CPU1, not on CPU3.
> 
> Linux's solution is not to stop the tick for CPU1 at time t3. It will
> be stopped only after the first firing of the tick itself at time
> t > t4 (if CPU1 is still idle, of course). This timer thing is, IMO,
> pretty similar. The only difference is that we don't have a tick to not
> stop... So I had to make up a 

Re: [Xen-devel] [PATCH RFC v1 0/3] Enable XL to set and get per-VCPU work conserving flag for RTDS scheduler

[Meng Xu]

On Tue, Aug 1, 2017 at 2:33 PM, Meng Xu  wrote:
>
> This series of patches enable the toolstack to
> set and get per-VCPU work-conserving flag.
> With the toolstack, system administrators can decide
> which VCPUs will be made work-conserving.
>
> The design of the work-conserving RTDS was discussed in
> https://www.mail-archive.com/xen-devel@lists.xen.org/msg77150.html
>
> We plan to perform two steps in making RTDS scheduler work-conserving:
> (1) First make all VCPUs work-conserving by default,
> which was sent as a separate patch. This work aims for Xen 4.10 release.
> (2) After that, we enable the XL to set and get per-VCPU work-conserving flag,
> which is this series of patches.


The series of patches that have both steps done can be found at the
following repo: https://github.com/PennPanda/RT-Xen
under the branch xenbits/rtds/work-conserving-RFCv1.

Thanks,

Meng


Meng Xu
PhD Student in Computer and Information Science
University of Pennsylvania
http://www.cis.upenn.edu/~mengxu/

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] PV drivers and zero copying

[Stefano Stabellini]

On Tue, 1 Aug 2017, Oleksandr Andrushchenko wrote:
> Hi, Stefano!
> 
> On 07/31/2017 11:28 PM, Stefano Stabellini wrote:
> > On Mon, 31 Jul 2017, Oleksandr Andrushchenko wrote:
> > > 3 Sharing with page exchange (XENMEM_exchange)
> > > ==
> > > 
> > > This API was pointed to me by Stefano Stabellini as one of the possible
> > > ways
> > > to
> > > achieve zero copying and share physically contiguous buffers. It is used
> > > by
> > > x86
> > > SWIOTLB code (xen_create_contiguous_region, [5]), but as per my
> > > understanding
> > > this API cannot be used on ARM as of now [6].  Conclusion: not an option
> > > for
> > > ARM
> > > at the moment
> > Let me elaborate on this. The purpose of XENMEM_exchange is to exchange
> > a number of memory pages with an equal number of contiguous memory
> > pages, possibly even under 4G. The original purpose of the hypercall was
> > to get DMA-able memory.
> this is good to know
> > 
> > So far, it has only been used by Dom0 on x86. Dom0 on ARM doesn't need
> > it because it is mapped 1:1 by default and device assignment is not
> > allowed without an IOMMU. However it should work on ARM too, as the
> > implementation is all common code in Xen.
> well, according to [6]:
> "Currently XENMEM_exchange is not supported on ARM because the steal_page is
> left unimplemented.
> 
> However, even if steal_page is implemented, the hypercall can't work for ARM
> because:
> - Direct mapped domain is not supported
> - ARM doesn't have a M2P and therefore usage of mfn_to_gmfn is
> invalid"
> And what I see at [7] is that it is still EOPNOTSUPP
> So, yes, common code is usable for both ARM and x86, but
> underlying support for ARM is till not there.
> Please correct me if I am wrong here

Ops, I forgot about that! Implementing steal_page on ARM is not be a
problem, and direct mapped domains are not a concern in this scenario.

The issue is mfn_to_gmfn. However, we do not actually need mfn_to_gmfn
to implement xen/common/memory.c:memory_exchange as Julien pointed out
in http://marc.info/?l=xen-devel=145037009127660.

Julien, Jan, two years have passed. Do you think we can find a way to
make that old series work for everybody?


> >   Also, looking at the
> > implementation (xen/common/memory.c:memory_exchange) it would seem that
> > it can be called from a DomU too (but I have never tried).
> good
> > Thus, if you have a platform without IOMMU and you disabled the IOMMU
> > checks in Xen to assign a device to a DomU anyway, then you could use
> > this hypercall from DomU to get memory under 4G to be used for DMA with
> > this device.
> There is no real device assigned to DomU, but a PV frontend
> > 
> > As far as I can tell XENMEM_exchange could help in the design of
> > zero-copy PV protocols only to address this specific use case:
> > 
> > - you have a frontend in DomU and a backend in Dom0
> > - pages shared by DomU get mapped in Dom0 and potentially used for DMA
> yes, this is crucial for zero copying in my case: DMA
> > - the device has under 4G DMA restrictions
> > 
> > Normally Dom0 maps a DomU page, then at the time of using the mapped
> > page for DMA it checks whether it is suitable for DMA (under 4G if the
> > device requires so). If it is not, Dom0 uses a bounce buffer borrowed
> > from the swiotlb. Obviously this introduces one or two memcpys.
> > 
> > Instead, if DomU calls XENMEM_exchange to get memory under 4G, and
> > shares one of the pages with Dom0 via PV frontends, then Dom0 wouldn't
> > have to use a bounce buffer to do DMA to this page.
> > 
> > Does it make sense?
> yes, it does, thank you, but [6], [7] :(
> 
> [7]
> https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/mm.c;h=411bab1ea9f7f14789a134056ebff9f68fd4a4c7;hb=a15516c0cf21d7ac84799f1e2e500b0bb22d2300#l1161


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] stage1-xen for Fedora

[Stefano Stabellini]

CC'ing xen-devel in case somebody else in the community has a better
answer for you

On Tue, 1 Aug 2017, Rajiv Ranganath wrote:
> Hi Stefano,
> 
> I was wondering if you had instructions on building stage1-xen on Fedora?
> 
> If so can you please share.
> 
> Thank you!

Hello Rajiv,

It's great to see interest in the project! I haven't tried building
stage1-xen on Fedora yet, but I don't think it should be very different
from building it on Debian or Ubuntu. Please refer to BUILDING.md.

The main thing that will be different is the list of dependencies you
need to install to build Xen. On Fedora it should be (I am using
Raisin[1] as a reference):

  make gcc python-devel gettext libuuid-devel ncurses-devel glib2-devel 
libaio-devel openssl-devel yajl-devel patch pixman-devel glibc-devel 
bridge-utils grub2 wget tar bzip2 glibc-devel.i686

Similarly, the list of dependencies to build QEMU will be different:

  make gcc glib2-devel pixman-devel zlib-devel

Finally the list of dependencies to build rkt will be different, but in
this case I don't have a pre-made list for Fedora, but it shouldn't be
hard to find the corresponding packets for golang automake libacl1-dev
and libsystemd-dev.

stage1-xen per se has only busybox-static and jq as dependencies that
most probably are simply called "busybox-static" and "jq" on Fedora too.

Let me know if you find any issues! I would be very happy to take a
patch (or pull request) for BUILDING.md to document how to do this on
Fedora.

Cheers,

Stefano

[1]: https://wiki.xenproject.org/wiki/Raisin

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC v1 0/3] Enable XL to set and get per-VCPU work conserving flag for RTDS scheduler

[Meng Xu]

This series of patches enable the toolstack to
set and get per-VCPU work-conserving flag.
With the toolstack, system administrators can decide
which VCPUs will be made work-conserving.

The design of the work-conserving RTDS was discussed in
https://www.mail-archive.com/xen-devel@lists.xen.org/msg77150.html

We plan to perform two steps in making RTDS scheduler work-conserving:
(1) First make all VCPUs work-conserving by default,
which was sent as a separate patch. This work aims for Xen 4.10 release.
(2) After that, we enable the XL to set and get per-VCPU work-conserving flag,
which is this series of patches.

Signed-off-by: Meng Xu 


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC v1 3/3] xl: enable per-VCPU work conserving flag for RTDS

[Meng Xu]

Change main_sched_rtds and related output functions to support
per-VCPU work conserving flag.

Signed-off-by: Meng Xu 
---
 tools/xl/xl_cmdtable.c |  3 ++-
 tools/xl/xl_sched.c| 56 ++
 2 files changed, 40 insertions(+), 19 deletions(-)

diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 30eb93c..95997e1 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -272,12 +272,13 @@ struct cmd_spec cmd_table[] = {
 { "sched-rtds",
   _sched_rtds, 0, 1,
   "Get/set rtds scheduler parameters",
-  "[-d  [-v[=VCPUID/all]] [-p[=PERIOD]] [-b[=BUDGET]]]",
+  "[-d  [-v[=VCPUID/all]] [-p[=PERIOD]] [-b[=BUDGET]]] 
[-w[=WORKCONSERVING]]",
   "-d DOMAIN, --domain=DOMAIN Domain to modify\n"
   "-v VCPUID/all, --vcpuid=VCPUID/allVCPU to modify or output;\n"
   "   Using '-v all' to modify/output all vcpus\n"
   "-p PERIOD, --period=PERIOD Period (us)\n"
   "-b BUDGET, --budget=BUDGET Budget (us)\n"
+  "-w WORKCONSERVING, --workconserving=WORKCONSERVINGWORKCONSERVING 
(1=yes,0=no)\n"
 },
 { "domid",
   _domid, 0, 0,
diff --git a/tools/xl/xl_sched.c b/tools/xl/xl_sched.c
index 85722fe..35a64e1 100644
--- a/tools/xl/xl_sched.c
+++ b/tools/xl/xl_sched.c
@@ -251,7 +251,7 @@ static int sched_rtds_domain_output(
 libxl_domain_sched_params scinfo;
 
 if (domid < 0) {
-printf("%-33s %4s %9s %9s\n", "Name", "ID", "Period", "Budget");
+printf("%-33s %4s %9s %9s %15s\n", "Name", "ID", "Period", "Budget", 
"Work conserving");
 return 0;
 }
 
@@ -262,11 +262,12 @@ static int sched_rtds_domain_output(
 }
 
 domname = libxl_domid_to_name(ctx, domid);
-printf("%-33s %4d %9d %9d\n",
+printf("%-33s %4d %9d %9d %15d\n",
 domname,
 domid,
 scinfo.period,
-scinfo.budget);
+scinfo.budget,
+scinfo.is_work_conserving);
 free(domname);
 libxl_domain_sched_params_dispose();
 return 0;
@@ -279,8 +280,8 @@ static int sched_rtds_vcpu_output(int domid, 
libxl_vcpu_sched_params *scinfo)
 int i;
 
 if (domid < 0) {
-printf("%-33s %4s %4s %9s %9s\n", "Name", "ID",
-   "VCPU", "Period", "Budget");
+printf("%-33s %4s %4s %9s %9s %15s\n", "Name", "ID",
+   "VCPU", "Period", "Budget", "Work conserving");
 return 0;
 }
 
@@ -290,12 +291,13 @@ static int sched_rtds_vcpu_output(int domid, 
libxl_vcpu_sched_params *scinfo)
 
 domname = libxl_domid_to_name(ctx, domid);
 for ( i = 0; i < scinfo->num_vcpus; i++ ) {
-printf("%-33s %4d %4d %9"PRIu32" %9"PRIu32"\n",
+printf("%-33s %4d %4d %9"PRIu32" %9"PRIu32" %15d\n",
domname,
domid,
scinfo->vcpus[i].vcpuid,
scinfo->vcpus[i].period,
-   scinfo->vcpus[i].budget);
+   scinfo->vcpus[i].budget,
+   scinfo->vcpus[i].is_work_conserving );
 }
 free(domname);
 return 0;
@@ -309,8 +311,8 @@ static int sched_rtds_vcpu_output_all(int domid,
 int i;
 
 if (domid < 0) {
-printf("%-33s %4s %4s %9s %9s\n", "Name", "ID",
-   "VCPU", "Period", "Budget");
+printf("%-33s %4s %4s %9s %9s %15s\n", "Name", "ID",
+   "VCPU", "Period", "Budget", "Work conserving");
 return 0;
 }
 
@@ -321,12 +323,13 @@ static int sched_rtds_vcpu_output_all(int domid,
 
 domname = libxl_domid_to_name(ctx, domid);
 for ( i = 0; i < scinfo->num_vcpus; i++ ) {
-printf("%-33s %4d %4d %9"PRIu32" %9"PRIu32"\n",
+printf("%-33s %4d %4d %9"PRIu32" %9"PRIu32" %15d\n",
domname,
domid,
scinfo->vcpus[i].vcpuid,
scinfo->vcpus[i].period,
-   scinfo->vcpus[i].budget);
+   scinfo->vcpus[i].budget,
+   scinfo->vcpus[i].is_work_conserving);
 }
 free(domname);
 return 0;
@@ -702,14 +705,18 @@ int main_sched_rtds(int argc, char **argv)
 int *vcpus = (int *)xmalloc(sizeof(int)); /* IDs of VCPUs that change */
 int *periods = (int *)xmalloc(sizeof(int)); /* period is in microsecond */
 int *budgets = (int *)xmalloc(sizeof(int)); /* budget is in microsecond */
+int *workconservings = (int *)xmalloc(sizeof(int)); /* budget is in 
microsecond */
 int v_size = 1; /* size of vcpus array */
 int p_size = 1; /* size of periods array */
 int b_size = 1; /* size of budgets array */
+int w_size = 1; /* size of work conserving array */
 int v_index = 0; /* index in vcpus array */
 int p_index =0; /* index in periods array */
 int b_index =0; /* index for in budgets array */
+int w_index =0; /* index for in work conserving array */
 bool opt_p = false;
 bool opt_b = false;
+bool opt_w = false;
 bool opt_v = false;
 bool opt_all = false; /* output 

[Xen-devel] [PATCH RFC v1 1/3] xen:rtds: enable XL to set and get vcpu work conserving flag

[Meng Xu]

Extend the hypercalls(XEN_DOMCTL_SCHEDOP_getvcpuinfo/putvcpuinfo) to
get/set a domain's per-VCPU work conserving parameters.

Signed-off-by: Meng Xu 
---
 xen/common/sched_rt.c   | 2 ++
 xen/include/public/domctl.h | 1 +
 2 files changed, 3 insertions(+)

diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c
index 740a712..76ed4cb 100644
--- a/xen/common/sched_rt.c
+++ b/xen/common/sched_rt.c
@@ -1442,6 +1442,7 @@ rt_dom_cntl(
 svc = rt_vcpu(d->vcpu[local_sched.vcpuid]);
 local_sched.u.rtds.budget = svc->budget / MICROSECS(1);
 local_sched.u.rtds.period = svc->period / MICROSECS(1);
+local_sched.u.rtds.is_work_conserving = 
svc->is_work_conserving;
 spin_unlock_irqrestore(>lock, flags);
 
 if ( copy_to_guest_offset(op->u.v.vcpus, index,
@@ -1466,6 +1467,7 @@ rt_dom_cntl(
 svc = rt_vcpu(d->vcpu[local_sched.vcpuid]);
 svc->period = period;
 svc->budget = budget;
+svc->is_work_conserving = 
local_sched.u.rtds.is_work_conserving;
 spin_unlock_irqrestore(>lock, flags);
 }
 /* Process a most 64 vCPUs without checking for preemptions. */
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index ff39762..e67cd9e 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -360,6 +360,7 @@ typedef struct xen_domctl_sched_credit2 {
 typedef struct xen_domctl_sched_rtds {
 uint32_t period;
 uint32_t budget;
+bool is_work_conserving;
 } xen_domctl_sched_rtds_t;
 
 typedef struct xen_domctl_schedparam_vcpu {
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] DESIGN v2: CPUID part 3

[Andrew Cooper]

On 31/07/2017 20:49, Konrad Rzeszutek Wilk wrote:
> On Wed, Jul 05, 2017 at 02:22:00PM +0100, Joao Martins wrote:
>> On 07/05/2017 12:16 PM, Andrew Cooper wrote:
>>> On 05/07/17 10:46, Joao Martins wrote:
 Hey Andrew,

 On 07/04/2017 03:55 PM, Andrew Cooper wrote:

> (RFC: Decide exactly where to fit this.  _XEN\_DOMCTL\_max\_vcpus_ 
> perhaps?)
> The toolstack shall also have a mechanism to explicitly select topology
> configuration for the guest, which primarily affects the virtual APIC ID
> layout, and has a knock on effect for the APIC ID of the virtual IO-APIC.
> Xen's auditing shall ensure that guests observe values consistent with the
> guarantees made by the vendor manuals.
>
 Why choose max_vcpus domctl?
>>> Despite its name, the max_vcpus hypercall is the one which allocates all
>>> the vcpus in the hypervisor.  I don't want there to be any opportunity
>>> for vcpus to exist but no topology information to have been provided.
>>>
>> /nods
>>
>> So then doing this at vcpus allocation we would need to pass an additional 
>> CPU
>> topology argument on the max_vcpus hypercall? Otherwise it's sort of guess 
>> work
>> wrt sockets, cores, threads ... no?
> Andrew, thoughts on this and the one below?

Urgh sorry.  I've been distracted with some high priority interrupts (of
the non-maskable variety).

So, bad news is that the CPUID and MSR policy handling has become
substantially more complicated and entwined than I had first planned.  A
change in either of the data alters the auditing of the other, so I am
leaning towards implementing everything with a single set hypercall (as
this is the only way to get a plausibly-consistent set of data).

The good news is that I don't think we actually need any changes to the
XEN_DOMCTL_max_vcpus.  I now think there is sufficient expressibility in
the static cpuid policy to work.

>> There could be other uses too on passing this info to Xen, say e.g. the
>> scheduler knowing the guest CPU topology it would allow better selection of
>> core+sibling pair such that it could match cache/cpu topology passed on the
>> guest (for unpinned SMT guests).

I remain to be convinced (i.e. with some real performance numbers) that
the added complexity in the scheduler for that logic is a benefit in the
general case.

In practice, customers are either running very specific and dedicated
workloads (at which point pinning is used and there is no
oversubscription, and exposing the actual SMT topology is a good thing),
or customers are running general workloads with no pinning (or perhaps
cpupool-numa-split) with a moderate amount of oversubscription (at which
point exposing SMT is a bad move).

Counterintuitively, exposing NUMA in general oversubscribed scenarios is
terrible for net system performance.  What happens in practice is that
VMs which see NUMA spend their idle cycles trying to balance their own
userspace processes, rather than yielding to the hypervisor so another
guest can get a go.

>>
 With multiple sockets/nodes and having supported extended topology leaf 
 the APIC
 ID layout will change considerably requiring fixup if... say we set vNUMA 
 (I
 know numa node != socket spec wise, but on the machines we have seen so 
 far,
 it's a 1:1 mapping).
>>> AMD Fam15h and later (may) have multiple NUMA nodes per socket, which
>>> will need to be accounted for in how the information is represented,
>>> especially in leaf 0x801e.
>>>
>>> Intel on the other hand (as far as I can tell), has no interaction
>>> between NUMA and topology as far as CPUID is concerned.
>>>
>> Sorry, I should probably have mentioned earlier that "machines we have seen 
>> so
>> far" were Intel - I am bit unaware of the AMD added possibilities.
>>
 Another question since we are speaking about topology is would be: how do 
 we
 make hvmloader aware of each the APIC_ID layout? Right now, it is too 
 hardcoded
 2 * APIC_ID :( Probably a xenstore entry 'hvmloader/cputopology-threads' 
 and
 'hvmloader/cputopology-sockets' (or use vnuma_topo.nr_nodes for the 
 latter)?
>>> ACPI table writing is in the toolstack now, but even if it weren't,
>>> HVMLoader would have to do what all real firmware needs to do, and look
>>> at CPUID.
> I think the real hardware when constructing interesting topologies uses
> platform specific MSRs or other hidden gems (like AMD Northbridge).

It was my understanding that APIC IDs are negotiated at power-on time,
as they are the base layer of addressing in the system.

>
>> Right, but the mp tables (and lapic ids) are still adjusted/created by 
>> hvmloader
>> unless ofc I am reading it wrong. But anyhow - if you're planning to be 
>> based on
> 
>
> I can't see how the CPUID would allow to construct the proper APIC MADT 
> entries so that
> the APIC IDs match as of right now?
>
> Unless hvmloader is changed to do full SMP bootup (it does that now at some 
> point)
> and 

[Xen-devel] [PATCH RFC v1 2/3] libxl: enable per-VCPU work conserving flag for RTDS

[Meng Xu]

Modify libxl_vcpu_sched_params_get/set and sched_rtds_vcpu_get/set
functions to support per-VCPU work conserving flag

Signed-off-by: Meng Xu 
---
 tools/libxl/libxl.h | 1 +
 tools/libxl/libxl_sched.c   | 3 +++
 tools/libxl/libxl_types.idl | 2 ++
 3 files changed, 6 insertions(+)

diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 7cf0f31..dd9c926 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -2058,6 +2058,7 @@ int libxl_sched_credit2_params_set(libxl_ctx *ctx, 
uint32_t poolid,
 #define LIBXL_DOMAIN_SCHED_PARAM_LATENCY_DEFAULT   -1
 #define LIBXL_DOMAIN_SCHED_PARAM_EXTRATIME_DEFAULT -1
 #define LIBXL_DOMAIN_SCHED_PARAM_BUDGET_DEFAULT-1
+#define LIBXL_DOMAIN_SCHED_PARAM_IS_WORK_CONSERVING_DEFAULT-1
 
 /* Per-VCPU parameters */
 #define LIBXL_SCHED_PARAM_VCPU_INDEX_DEFAULT   -1
diff --git a/tools/libxl/libxl_sched.c b/tools/libxl/libxl_sched.c
index faa604e..fe92747 100644
--- a/tools/libxl/libxl_sched.c
+++ b/tools/libxl/libxl_sched.c
@@ -558,6 +558,7 @@ static int sched_rtds_vcpu_get_all(libxl__gc *gc, uint32_t 
domid,
 for (i = 0; i < num_vcpus; i++) {
 scinfo->vcpus[i].period = vcpus[i].u.rtds.period;
 scinfo->vcpus[i].budget = vcpus[i].u.rtds.budget;
+scinfo->vcpus[i].is_work_conserving = 
vcpus[i].u.rtds.is_work_conserving;
 scinfo->vcpus[i].vcpuid = vcpus[i].vcpuid;
 }
 rc = 0;
@@ -607,6 +608,7 @@ static int sched_rtds_vcpu_set(libxl__gc *gc, uint32_t 
domid,
 vcpus[i].vcpuid = scinfo->vcpus[i].vcpuid;
 vcpus[i].u.rtds.period = scinfo->vcpus[i].period;
 vcpus[i].u.rtds.budget = scinfo->vcpus[i].budget;
+vcpus[i].u.rtds.is_work_conserving = 
scinfo->vcpus[i].is_work_conserving;
 }
 
 r = xc_sched_rtds_vcpu_set(CTX->xch, domid,
@@ -655,6 +657,7 @@ static int sched_rtds_vcpu_set_all(libxl__gc *gc, uint32_t 
domid,
 vcpus[i].vcpuid = i;
 vcpus[i].u.rtds.period = scinfo->vcpus[0].period;
 vcpus[i].u.rtds.budget = scinfo->vcpus[0].budget;
+vcpus[i].u.rtds.is_work_conserving = 
scinfo->vcpus[0].is_work_conserving;
 }
 
 r = xc_sched_rtds_vcpu_set(CTX->xch, domid,
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 8a9849c..f6c3ead 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -401,6 +401,7 @@ libxl_sched_params = Struct("sched_params",[
 ("period",   integer, {'init_val': 
'LIBXL_DOMAIN_SCHED_PARAM_PERIOD_DEFAULT'}),
 ("extratime",integer, {'init_val': 
'LIBXL_DOMAIN_SCHED_PARAM_EXTRATIME_DEFAULT'}),
 ("budget",   integer, {'init_val': 
'LIBXL_DOMAIN_SCHED_PARAM_BUDGET_DEFAULT'}),
+("is_work_conserving", integer, {'init_val': 
'LIBXL_DOMAIN_SCHED_PARAM_IS_WORK_CONSERVING_DEFAULT'}),
 ])
 
 libxl_vcpu_sched_params = Struct("vcpu_sched_params",[
@@ -414,6 +415,7 @@ libxl_domain_sched_params = Struct("domain_sched_params",[
 ("cap",  integer, {'init_val': 
'LIBXL_DOMAIN_SCHED_PARAM_CAP_DEFAULT'}),
 ("period",   integer, {'init_val': 
'LIBXL_DOMAIN_SCHED_PARAM_PERIOD_DEFAULT'}),
 ("budget",   integer, {'init_val': 
'LIBXL_DOMAIN_SCHED_PARAM_BUDGET_DEFAULT'}),
+("is_work_conserving", integer, {'init_val': 
'LIBXL_DOMAIN_SCHED_PARAM_IS_WORK_CONSERVING_DEFAULT'}),
 
 # The following three parameters ('slice', 'latency' and 'extratime') are 
deprecated,
 # and will have no effect if used, since the SEDF scheduler has been 
removed.
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 00/13] "Non-shared" IOMMU support on ARM

[Oleksandr Tyshchenko]

СС persons from all patches in current series into the cover letter.

On Tue, Aug 1, 2017 at 9:09 PM, Julien Grall  wrote:
>
>
> On 01/08/17 04:06, Tian, Kevin wrote:
>>>
>>> From: Oleksandr Tyshchenko [mailto:olekst...@gmail.com]
>>> Sent: Monday, July 31, 2017 7:58 PM
>>>
>>> Hi, Kevin
>>>
>>> On Mon, Jul 31, 2017 at 8:57 AM, Tian, Kevin 
>>> wrote:
>
> From: Oleksandr Tyshchenko
> Sent: Wednesday, July 26, 2017 1:27 AM
>
> From: Oleksandr Tyshchenko 
>
> Hi, all.
>
> The purpose of this patch series is to create a base for porting
> any "Non-shared" IOMMUs to Xen on ARM. Saying "Non-shared" IOMMU
>>>
>>> I
>
> mean
> the IOMMU that can't share the page table with the CPU.


 Is "non-shared" IOMMU a standard terminology in ARM side? I quickly
 searched to find it mostly used in this thread...
>>>
>>> I don't think that it is a standard terminology.
>>>

 On the other hand, all IOMMUs support a basic DMA remapping
 mechanism with page table not shared with CPU. Then some IOMMUs
 may optional support Shared Virtual Memory (SVM) through page
 sharing with CPU. Then I'm not sure why need to highlight the
 "non-shared" manner in this thread, instead of just saying
 IPMMU-VMSA support...
>>>
>>> I wouldn't use "IPMMU-VMSA support" in this thread since it may be any
>>> other IOMMUs which can't share page table
>>> with CPU because of format incompatibilities.
>>
>>
>> As I commented you can assume all IOMMUs cannot share page
>> table with CPU as the starting point. It's not good to name an IOMMU
>> driver based on such fact.
>
>
> As said in a previous reply. This is a wrong assumption, you can share
> page-tables with the IOMMU if the format is the same. You may still need to
> do TLB flush manually on the IOMMU, but you still save memory.
>
> x86 is already supporting that and also call "sharing" and is done by
> default (see iommu_hap_pt_share). If the naming is wrong, then feel free to
> send a patch.
>
> But I don't think you should complain about Oleksandr use this name.
>
> Cheers,
>
> --
> Julien Grall



-- 
Regards,

Oleksandr Tyshchenko

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC v1] xen:rtds: towards work conserving RTDS

[Meng Xu]

Make RTDS scheduler work conserving to utilize the idle resource,
without breaking the real-time guarantees.

VCPU model:
Each real-time VCPU is extended to have a work conserving flag
and a priority_level field.
When a VCPU's budget is depleted in the current period,
if it has work conserving flag set,
its priority_level will increase by 1 and its budget will be refilled;
othewrise, the VCPU will be moved to the depletedq.

Scheduling policy: modified global EDF:
A VCPU v1 has higher priority than another VCPU v2 if
(i) v1 has smaller priority_leve; or
(ii) v1 has the same priority_level but has a smaller deadline

Signed-off-by: Meng Xu 
---
 xen/common/sched_rt.c | 71 ++-
 1 file changed, 59 insertions(+), 12 deletions(-)

diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c
index 39f6bee..740a712 100644
--- a/xen/common/sched_rt.c
+++ b/xen/common/sched_rt.c
@@ -49,13 +49,16 @@
  * A PCPU is feasible if the VCPU can run on this PCPU and (the PCPU is idle or
  * has a lower-priority VCPU running on it.)
  *
- * Each VCPU has a dedicated period and budget.
+ * Each VCPU has a dedicated period, budget and is_work_conserving flag
  * The deadline of a VCPU is at the end of each period;
  * A VCPU has its budget replenished at the beginning of each period;
  * While scheduled, a VCPU burns its budget.
  * The VCPU needs to finish its budget before its deadline in each period;
  * The VCPU discards its unused budget at the end of each period.
- * If a VCPU runs out of budget in a period, it has to wait until next period.
+ * A work conserving VCPU has is_work_conserving flag set to true;
+ * When a VCPU runs out of budget in a period, if it is work conserving,
+ * it increases its priority_level by 1 and refill its budget; otherwise,
+ * it has to wait until next period.
  *
  * Each VCPU is implemented as a deferable server.
  * When a VCPU has a task running on it, its budget is continuously burned;
@@ -63,7 +66,8 @@
  *
  * Queue scheme:
  * A global runqueue and a global depletedqueue for each CPU pool.
- * The runqueue holds all runnable VCPUs with budget, sorted by deadline;
+ * The runqueue holds all runnable VCPUs with budget,
+ * sorted by priority_level and deadline;
  * The depletedqueue holds all VCPUs without budget, unsorted;
  *
  * Note: cpumask and cpupool is supported.
@@ -191,6 +195,7 @@ struct rt_vcpu {
 /* VCPU parameters, in nanoseconds */
 s_time_t period;
 s_time_t budget;
+bool_t is_work_conserving;   /* is vcpu work conserving */
 
 /* VCPU current infomation in nanosecond */
 s_time_t cur_budget; /* current budget */
@@ -201,6 +206,8 @@ struct rt_vcpu {
 struct rt_dom *sdom;
 struct vcpu *vcpu;
 
+unsigned priority_level;
+
 unsigned flags;  /* mark __RTDS_scheduled, etc.. */
 };
 
@@ -245,6 +252,11 @@ static inline struct list_head *rt_replq(const struct 
scheduler *ops)
 return _priv(ops)->replq;
 }
 
+static inline bool_t is_work_conserving(const struct rt_vcpu *svc)
+{
+return svc->is_work_conserving;
+}
+
 /*
  * Helper functions for manipulating the runqueue, the depleted queue,
  * and the replenishment events queue.
@@ -273,6 +285,20 @@ vcpu_on_replq(const struct rt_vcpu *svc)
 return !list_empty(>replq_elem);
 }
 
+/* If v1 priority >= v2 priority, return value > 0
+ * Otherwise, return value < 0
+ */
+static int
+compare_vcpu_priority(const struct rt_vcpu *v1, const struct rt_vcpu *v2)
+{
+if ( v1->priority_level < v2->priority_level ||
+ ( v1->priority_level == v2->priority_level && 
+ v1->cur_deadline <= v2->cur_deadline ) )
+return 1;
+else
+return -1;
+}
+
 /*
  * Debug related code, dump vcpu/cpu information
  */
@@ -303,6 +329,7 @@ rt_dump_vcpu(const struct scheduler *ops, const struct 
rt_vcpu *svc)
 cpulist_scnprintf(keyhandler_scratch, sizeof(keyhandler_scratch), mask);
 printk("[%5d.%-2u] cpu %u, (%"PRI_stime", %"PRI_stime"),"
" cur_b=%"PRI_stime" cur_d=%"PRI_stime" last_start=%"PRI_stime"\n"
+   " \t\t priority_level=%d work_conserving=%d\n"
" \t\t onQ=%d runnable=%d flags=%x effective hard_affinity=%s\n",
 svc->vcpu->domain->domain_id,
 svc->vcpu->vcpu_id,
@@ -312,6 +339,8 @@ rt_dump_vcpu(const struct scheduler *ops, const struct 
rt_vcpu *svc)
 svc->cur_budget,
 svc->cur_deadline,
 svc->last_start,
+svc->priority_level,
+is_work_conserving(svc),
 vcpu_on_q(svc),
 vcpu_runnable(svc->vcpu),
 svc->flags,
@@ -423,15 +452,18 @@ rt_update_deadline(s_time_t now, struct rt_vcpu *svc)
  */
 svc->last_start = now;
 svc->cur_budget = svc->budget;
+svc->priority_level = 0;
 
 /* TRACE */
 {
 struct __packed {
 unsigned vcpu:16, dom:16;
+unsigned priority_level;
 

Re: [Xen-devel] [PATCH v2 00/13] "Non-shared" IOMMU support on ARM

[Julien Grall]

On 01/08/17 04:06, Tian, Kevin wrote:

From: Oleksandr Tyshchenko [mailto:olekst...@gmail.com]
Sent: Monday, July 31, 2017 7:58 PM

Hi, Kevin

On Mon, Jul 31, 2017 at 8:57 AM, Tian, Kevin  wrote:

From: Oleksandr Tyshchenko
Sent: Wednesday, July 26, 2017 1:27 AM

From: Oleksandr Tyshchenko 

Hi, all.

The purpose of this patch series is to create a base for porting
any "Non-shared" IOMMUs to Xen on ARM. Saying "Non-shared" IOMMU

I

mean
the IOMMU that can't share the page table with the CPU.


Is "non-shared" IOMMU a standard terminology in ARM side? I quickly
searched to find it mostly used in this thread...

I don't think that it is a standard terminology.



On the other hand, all IOMMUs support a basic DMA remapping
mechanism with page table not shared with CPU. Then some IOMMUs
may optional support Shared Virtual Memory (SVM) through page
sharing with CPU. Then I'm not sure why need to highlight the
"non-shared" manner in this thread, instead of just saying
IPMMU-VMSA support...

I wouldn't use "IPMMU-VMSA support" in this thread since it may be any
other IOMMUs which can't share page table
with CPU because of format incompatibilities.


As I commented you can assume all IOMMUs cannot share page
table with CPU as the starting point. It's not good to name an IOMMU
driver based on such fact.


As said in a previous reply. This is a wrong assumption, you can share 
page-tables with the IOMMU if the format is the same. You may still need 
to do TLB flush manually on the IOMMU, but you still save memory.


x86 is already supporting that and also call "sharing" and is done by 
default (see iommu_hap_pt_share). If the naming is wrong, then feel free 
to send a patch.


But I don't think you should complain about Oleksandr use this name.

Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 00/13] "Non-shared" IOMMU support on ARM

[Julien Grall]

On 31/07/17 06:57, Tian, Kevin wrote:

From: Oleksandr Tyshchenko
Sent: Wednesday, July 26, 2017 1:27 AM

From: Oleksandr Tyshchenko 

Hi, all.

The purpose of this patch series is to create a base for porting
any "Non-shared" IOMMUs to Xen on ARM. Saying "Non-shared" IOMMU I
mean
the IOMMU that can't share the page table with the CPU.


Is "non-shared" IOMMU a standard terminology in ARM side? I quickly
searched to find it mostly used in this thread...

On the other hand, all IOMMUs support a basic DMA remapping
mechanism with page table not shared with CPU. Then some IOMMUs
may optional support Shared Virtual Memory (SVM) through page
sharing with CPU. Then I'm not sure why need to highlight the
"non-shared" manner in this thread, instead of just saying
IPMMU-VMSA support...


This is not entirely true. You can share the page-table with the IOMMU 
as long as the page-table are the same. This may still involve some TLB 
flush on the IOMMU side, but you don't have allocate twice the 
page-table. Therefore you save memory.


This is actually the case of SMMUv{1,2}, we share the page table but 
still have to do the TLB flush and potential clean the page entry table.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [linux-4.9 test] 112405: tolerable trouble: blocked/broken/fail/pass - PUSHED

[osstest service owner]

flight 112405 linux-4.9 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/112405/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 112193
 build-arm64-pvops 2 hosts-allocate broken REGR. vs. 112193
 build-arm64   2 hosts-allocate broken REGR. vs. 112193

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-examine  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64-pvops 3 capture-logs  broken blocked in 112193
 build-arm64   3 capture-logs  broken blocked in 112193
 build-arm64-xsm   3 capture-logs broken never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 112086
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-localmigrate/x10 fail like 112117
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-localmigrate/x10 fail like 112117
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeatfail  like 112193
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 112193
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 10 windows-installfail never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 13 guest-saverestore   fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-xl-qemut-win7-amd64 18 guest-start/win.repeat  fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 13 guest-saverestore   fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass

version targeted for testing:
 linuxefcfbfb1d8bf756d1b58fe215bf4e419d176435b
baseline version:
 linuxc03917de04aa68017a737e90ea01338d991eaff5

Last test of basis   112193  2017-07-23 01:49:45 Z9 days
Testing same since   112350  2017-07-27 

Re: [Xen-devel] [PATCH OSSTEST v2 11/11] sg-run-job: hook the memdisk test into examine

[Ian Jackson]

Roger Pau Monne writes ("[PATCH OSSTEST v2 11/11] sg-run-job: hook the memdisk 
test into examine"):
> Hook the memdisk parameter detection and the saving of the host
> properties into the examine jobs.
...
>  catching-otherwise fail {
> + run-ts .   =ts-memdisk-try-append + host

I think -. would be better here, so that if this step fails we
continue with the rest of the job.  (See the comment about IFFAIL in
sg-run-job.)

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH OSSTEST v2 09/11] ts-examine-hostprops-save: introduce a script to save properties

[Ian Jackson]

Roger Pau Monne writes ("[PATCH OSSTEST v2 09/11] ts-examine-hostprops-save: 
introduce a script to save properties"):
> The introduce script turns the properties stored in the runvars using
> the format hostprop_$hotname_$prop=$val into host properties stored in
> the database.

"This script ..."


> +our $blessing = intended_blessing();
> +if ($blessing ne "real")
> +{

{ should be on the same line as the if.  (Several times.)

I think it is fine to check the intended blessing both here and in
$mjobdb.  Here it might be useful to check it so that you can print
useful debug output in non-real flights.

We discussed the semantics of this script on irc - specifically, that
it should operate on all hosts for which a relevant runvar exists.

Thanks,
Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH OSSTEST v2 08/11] ts-memdisk-try-append: introduce a script to test memdisk options

[Ian Jackson]

Roger Pau Monne writes ("[PATCH OSSTEST v2 08/11] ts-memdisk-try-append: 
introduce a script to test memdisk options"):
> The intended usage is to run this script against every host in order
> to record the possible needed memdisk flags.

Acked-by: Ian Jackson 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH OSSTEST v2 04/11] HostDB: introduce set_property

[Ian Jackson]

Roger Pau Monne writes ("[PATCH OSSTEST v2 04/11] HostDB: introduce 
set_property"):
> And provide a helper in TestSupport to use it. This allows osstest to
> set host properties from test script themselves (instead of using
> the mg-hosts clu).
...
> +sub set_property() {
> +my ($hd, $ho, $prop, $val) = @_;
> +my $rmq = $dbh_tests->prepare(< +DELETE FROM resource_properties
> +   WHERE restype='host' and resname=? AND name=?

This is where the test for the intended blessing should occur IMO.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH OSSTEST v2 07/11] ts-freebsd-host-install: add arguments to test memdisk append options

[Ian Jackson]

Roger Pau Monne writes ("[PATCH OSSTEST v2 07/11] ts-freebsd-host-install: add 
arguments to test memdisk append options"):
> This is needed in order to figure out which memdisk options should be
> used to boot the images on each specific box.
> 
> Note that upon success the script stores the tentative host property
> in the runvars.
...
> +} elsif ($ARGV[0] eq "--recordappend") {
> +$record_append = 1;
...
> +if ($bootonly) {
> +hostprop_putative_record($ho, "MemdiskAppend", $memdisk_append)
> +if $record_append;
> +exit 0;

This is surely wrong.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Next Xen ARM community call - Wednesday 2nd August 2017

[Stefano Stabellini]

On Tue, 1 Aug 2017, Julien Grall wrote:
> On 01/08/17 17:44, Edgar E. Iglesias wrote:
> > On Wed, Jul 26, 2017 at 04:59:43PM +0100, Julien Grall wrote:
> > > Hi all,
> > > 
> > > The next Xen ARM community call will be Wednesday 2nd August 2017 5pm BST.
> > > 
> > > Do you have any specific topic you would like to discuss?
> > 
> > CC: Davorin and Mirella from Aggios
> > 
> > Hi Julien,
> 
> Hi,
> 
> > I was talking with the Aggios folks today and they were wondering if
> > it's possible to share screens to present slides during the call?
> > I'm guessing not, since the info below only has dial in info.
> 
> It sounds like it is possible to share screen but I can't find a public link
> for it :/.
> 
> I can look for an alternative (I have plenty of choice at Arm :)) if it is
> something people wants to do in the future. But for tomorrow, it might be
> difficult to get something up by tomorrow.
> 
> Can you upload the slides somewhere and send a link by e-mail?

Sending the slides by email beforehand is always a good idea. However,
it's 2017 and we *have* to be able to share slides live during a meeting
:-)

I talked with Julien: we are going to use my uberconference details for
the call, which supports dialing in by phone, from your PC and slide
sharing:


Join the call: https://www.uberconference.com/stefano-stabellini
US dial-in number: 669-999-0613
No PIN needed

For the international numbers, go to
https://www.uberconference.com/stefano-stabellini and choose "Join by
Phone", a list of international numbers and a pin will be provided.

I'll also send another reminder tomorrow.


Cheers,

Stefano

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH OSSTEST v2 01/11] netboot_memdisk: allow each host to have different append values

[Ian Jackson]

Roger Pau Monne writes ("[PATCH OSSTEST v2 01/11] netboot_memdisk: allow each 
host to have different append values"):
> Some hosts require "append raw" [0] when booting with memdisk, while
> others don't. This is based on the hardware/BIOS, and needs to be set
> on a per-host basis.

Acked-by: Ian Jackson 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [libvirt test] 112406: tolerable trouble: blocked/broken/pass - PUSHED

[osstest service owner]

flight 112406 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/112406/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 build-arm64   2 hosts-allocate broken REGR. vs. 112276
 build-arm64-pvops 2 hosts-allocate broken REGR. vs. 112276
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 112276

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt  1 build-check(1)   blocked  n/a
 build-arm64   3 capture-logs  broken blocked in 112276
 build-arm64-xsm   3 capture-logs  broken blocked in 112276
 build-arm64-pvops 3 capture-logs  broken blocked in 112276
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 112276
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 112276
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 112276
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass

version targeted for testing:
 libvirt  861dd1234f1776e149c9fe5486f7efc07294a655
baseline version:
 libvirt  f7237d63e8f02f3689f9b63b413fae7d4221faa9

Last test of basis   112276  2017-07-25 04:21:09 Z7 days
Failing since112310  2017-07-26 04:21:38 Z6 days7 attempts
Testing same since   112406  2017-08-01 04:20:17 Z0 days1 attempts


People who touched revisions under test:
  Andrea Bolognani 
  Daniel P. Berrange 
  Erik Skultety 
  John Ferlan 
  Ján Tomko 
  Martin Kletzander 
  Michal Privoznik 
  Nitesh Konkar 
  Nitesh Konkar 
  Pavel Hrdina 
  Peter Krempa 
  Scott Garfinkle 

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  broken  
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-arm64  broken  
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-arm64-libvirt  blocked 
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-arm64-pvopsbroken  
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmpass
 test-amd64-amd64-libvirt-xsm pass
 test-arm64-arm64-libvirt-xsm blocked 
 test-armhf-armhf-libvirt-xsm pass
 test-amd64-i386-libvirt-xsm  pass
 test-amd64-amd64-libvirt pass
 test-arm64-arm64-libvirt blocked 
 test-armhf-armhf-libvirt pass
 test-amd64-i386-libvirt  pass
 test-amd64-amd64-libvirt-pairpass
 

Re: [Xen-devel] [RFC PATCH v1 0/7] IPMMU-VMSA support on ARM

[Oleksandr Tyshchenko]

Hi, Julien

On Tue, Aug 1, 2017 at 3:27 PM, Julien Grall  wrote:
> On 26/07/17 16:09, Oleksandr Tyshchenko wrote:
>>
>> From: Oleksandr Tyshchenko 
>>
>> Hi, all.
>
>
> Hi,
>
> Please CC maintainers and any relevant person on the cover letter. This is
> quite useful to have in the inbox.
Yes. I CCed guys who, I think, are/were involved in IPMMU-VMSA
development from Linux side +
IOMMU maintainers (mostly ARM). Sorry, if I missed someone or mistakenly added.

>
>> The purpose of this patch series is to add IPMMU-VMSA support to Xen on
>> ARM.
>> It is VMSA-compatible IOMMU that integrated in the newest Renesas R-Car
>> Gen3 SoCs (ARM).
>> And this IOMMU can't share the page table with the CPU since it doesn't
>> use the same page-table format
>> as the CPU on ARM therefore I name it "Non-shared" IOMMU.
>> This all means that current patch series must be based on "Non-shared"
>> IOMMU support [1]
>> for the IPMMU-VMSA to be functional inside Xen.
>>
>> The IPMMU-VMSA driver as well as the ARM LPAE allocator were directly
>> ported from BSP for Linux the vendor provides:
>> git://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas-bsp.git
>> rcar-3.5.3
>
>
> I think this is probably a good starting point to discuss about IOMMU
> support in Xen. I skimmed through the patches and saw the words "rfc" and
> "ported from BSP".
Well, at the time of porting IPMMU-VMSA driver, BSP [1] had more
complete support than mainline [2]
and seems to have at the moment.
For example, mainline driver still has single IPMMU context while BSP
driver can have up to 8 contexts,
the maximum uTLBs mainline driver can handle is 32, but for BSP driver
this value was increased to 48, etc.
But, I see attempts to get all required support in [3]. So, when this
support reaches upsteam, I hope,
it won't be a big problem to rebase on mainline driver if we decide to
align with it.

>
> At the moment for IOMMU we rely on the Linux community to do the review, but
> this is not the case here as it is an RFC.  I can definitely help to check
> if it comply for Xen,
yes, please.

> but I don't have the competence to tell whether it is
> valid for the hardware.
>
> We may want to find a compromise to get it merged in Xen, but surely we
> don't want to build it by default at least until we had feedback from the
> community about the validity of the code here.
agree.

>
> As I mentioned above, we are currently borrowing drivers from Linux and
> adapting for Xen. Today we support SMMUv{1,2} (we need to resync it) and
> there are plan to add IPMMU-VMSA (this series) and SMMUv3.
It would be really nice to have IPMMU-VMSA support in Xen. Without
this support the SCF [4] we are developing right now
and even the Passthrough feature won't be fully functional on R-Car
Gen3 based boards powered by Xen hypervisor.

>
> I am aware that Linux IOMMU subsystem has growing quite a lot making more
> tricky to get support in Xen. I wanted to get feedback how complex from you
> and Sameer how complex it was and whether we should consider doing our own.

Yes, the IPMMU-VMSA Linux driver relies on some Linux functional
(IOMMU/DMA/io-pgtable frameworks) the Xen doesn't have (it is
expected). So, it took *some time*
to make Linux driver happy inside Xen). Moreover, this all resulted in
the fact that the driver looks complicated a bit).
A lot of different wrappers, #if 0, code style, etc.
On the other hand, I think, I will be able to fairly quickly align
with new BSP, etc.

But, I really don't know should we continue to follow this direction
or not, perhaps it will depend on
how complex the entity is and how much things we must pull together
with it to make it happy.

>
>> Patch series was rebased on Xen 4.9.0 release and tested on Renesas R-Car
>> Gen3 H3 ES2.0/M3 based boards
>> with devices assigned to different domains.
>>
>> You can find patch series here:
>> repo: https://github.com/otyshchenko1/xen.git branch: ipmmu_v2
>>
>> P.S. There is one more patch which needs to be brought back to life [2]
>> Any reasons why this patch hasn't been upstremed yet?
>
>
> The series didn't make it upstream. Feel free to resend it separately.
ok.

>
>>
>> Thank you.
>>
>> [1] [Xen-devel] [PATCH v2 00/13] "Non-shared" IOMMU support on ARM
>> https://www.mail-archive.com/xen-devel@lists.xen.org/msg115901.html
>>
>> [2] [Xen-devel] [PATCH v8 02/28] xen: Add log2 functionality
>> https://lists.xenproject.org/archives/html/xen-devel/2016-02/msg00031.html
>>
>> Oleksandr Tyshchenko (7):
>>   iommu/arm: ipmmu-vmsa: Add IPMMU-VMSA support
>>   iommu/arm: ipmmu-vmsa: Add Xen changes for main driver
>>   iommu/arm: ipmmu-vmsa: Add io-pgtables support
>>   iommu/arm: ipmmu-vmsa: Add Xen changes for io-pgtables
>>   iommu/arm: Build IPMMU-VMSA related stuff
>>   iommu/arm: ipmmu-vmsa: Deallocate page table asynchronously
>>   iommu/arm: ipmmu-vmsa: Enable VMSAv8-64 mode if IPMMU HW supports it
>>
>>  xen/drivers/passthrough/arm/Makefile  

Re: [Xen-devel] [XenSummit 2017] Shared coprocessor framework followup

[Edgar E. Iglesias]

On Tue, Aug 01, 2017 at 08:04:09PM +0300, Andrii Anisov wrote:
> Hello Edgar,
> 
> 
> On 01.08.17 17:56, Edgar E. Iglesias wrote:
> >On the PL, there's a chunk of programmable logic that allows you to
> >create your own custom accellerators or devices.
> >Some devices are tied to specific boards (e.g when they depend on specific 
> >IO)
> >but others are not (for example memory to memory computational accelerators).
> >To communicate with these devices, they have memory slave and master ports
> >(for register accesses and for DMA). They also have interrupts both ways.
> Are master ports behind IOMMU?

Yes, they are.

> 
> >It's possible to reprogram the configuration of the PL and swap accelerators 
> >in
> >and out on the fly. It's probably going to be too slow for trying to
> >context switch between guests
> So let us assume it is a FW-less resource we need to share between domains.
> Context switch will be stripped to mapping its mmio (or passing mmio
> accesses) next domain to serve and IOMMU configuration switching.
> Yep, IOMMU matters.

OK. I think the PL is more like a firmware enabled resource.
The PL configuration needs to be loaded much like firmware
otherwise the accelerators can't change shape and all guests
must use the same kind.

For example, one guest might want a crypto accelerator while
another might want some kind of machine-learning accelerator.

I think each guest may want to provide it's own accelerator "config".


> >  so I think primarily we would be looking at
> >a way to lets say, "allocate" and "release" the resources for batch use.
> 
> Kind of voluntary preemption?

Right. That could be a start.
In the future perhaps it makes sense to context-switch.

> 
> >If a guest cannot allocate an accelerator, it could fall back to emulation
> >or just to using SW libraries until an accelerator slot is available.
> What about the thing I called "an access emulation" [1]? From the domain's
> point of view it would be reflected in a delayed response (via IRQ or
> register polling) from an accelerator.
> 
> I guess the concept described above is feasible even with current SCF code
> and will not take too much efforts.

I'll have a look, thanks!

Cheers,
Edgar

> 
> [1]
> https://lists.xenproject.org/archives/html/xen-devel/2016-11/msg01935.html
> 
> -- 
> 
> *Andrii Anisov*
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XenSummit 2017] Shared coprocessor framework followup

[Andrii Anisov]

Hello Edgar,


On 01.08.17 17:56, Edgar E. Iglesias wrote:

On the PL, there's a chunk of programmable logic that allows you to
create your own custom accellerators or devices.
Some devices are tied to specific boards (e.g when they depend on specific IO)
but others are not (for example memory to memory computational accelerators).
To communicate with these devices, they have memory slave and master ports
(for register accesses and for DMA). They also have interrupts both ways.

Are master ports behind IOMMU?


It's possible to reprogram the configuration of the PL and swap accelerators in
and out on the fly. It's probably going to be too slow for trying to
context switch between guests

So let us assume it is a FW-less resource we need to share between domains.
Context switch will be stripped to mapping its mmio (or passing mmio 
accesses) next domain to serve and IOMMU configuration switching.

Yep, IOMMU matters.


  so I think primarily we would be looking at
a way to lets say, "allocate" and "release" the resources for batch use.


Kind of voluntary preemption?


If a guest cannot allocate an accelerator, it could fall back to emulation
or just to using SW libraries until an accelerator slot is available.
What about the thing I called "an access emulation" [1]? From the 
domain's point of view it would be reflected in a delayed response (via 
IRQ or register polling) from an accelerator.


I guess the concept described above is feasible even with current SCF 
code and will not take too much efforts.


[1] 
https://lists.xenproject.org/archives/html/xen-devel/2016-11/msg01935.html


--

*Andrii Anisov*


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Next Xen ARM community call - Wednesday 2nd August 2017

[Julien Grall]

Hi Edgar,

On 01/08/17 17:44, Edgar E. Iglesias wrote:

On Wed, Jul 26, 2017 at 04:59:43PM +0100, Julien Grall wrote:

Hi all,

The next Xen ARM community call will be Wednesday 2nd August 2017 5pm BST.

Do you have any specific topic you would like to discuss?


CC: Davorin and Mirella from Aggios

Hi Julien,


Hi,


I was talking with the Aggios folks today and they were wondering if
it's possible to share screens to present slides during the call?
I'm guessing not, since the info below only has dial in info.


It sounds like it is possible to share screen but I can't find a public 
link for it :/.


I can look for an alternative (I have plenty of choice at Arm :)) if it 
is something people wants to do in the future. But for tomorrow, it 
might be difficult to get something up by tomorrow.


Can you upload the slides somewhere and send a link by e-mail?

Cheers,



We can also email out slides before the call instead (reply to this email).

Thanks,
Edgar




Call+44 1223 406065 (Local dial in)
and enter the access code below followed by # key.
Participant code: 4915191

Mobile Auto Dial:
VoIP: voip://+441223406065;4915191#
iOS devices: +44 1223 406065,4915191 and press #
Other devices: +44 1223 406065x4915191#

Additional Calling Information:

UK +44 1142828002
US CA +1 4085761502
US TX +1 5123141073
JP +81 453455355
DE +49 8945604050
NO +47 73187518
SE +46 46313131
FR +33 497235101
TW +886 35657119
HU +36 13275600
IE +353 91337900

Toll Free

UK 0800 1412084
US +1 8668801148
CN +86 4006782367
IN 0008009868365
IN +918049282778
TW 08000 22065
HU 0680981587
IE 1800800022
KF +972732558877

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Next Xen ARM community call - Wednesday 2nd August 2017

[Edgar E. Iglesias]

On Wed, Jul 26, 2017 at 04:59:43PM +0100, Julien Grall wrote:
> Hi all,
> 
> The next Xen ARM community call will be Wednesday 2nd August 2017 5pm BST.
> 
> Do you have any specific topic you would like to discuss?

CC: Davorin and Mirella from Aggios

Hi Julien,

I was talking with the Aggios folks today and they were wondering if
it's possible to share screens to present slides during the call?
I'm guessing not, since the info below only has dial in info.

We can also email out slides before the call instead (reply to this email).

Thanks,
Edgar


> 
> Call+44 1223 406065 (Local dial in)
> and enter the access code below followed by # key.
> Participant code: 4915191
> 
> Mobile Auto Dial:
> VoIP: voip://+441223406065;4915191#
> iOS devices: +44 1223 406065,4915191 and press #
> Other devices: +44 1223 406065x4915191#
> 
> Additional Calling Information:
> 
> UK +44 1142828002
> US CA +1 4085761502
> US TX +1 5123141073
> JP +81 453455355
> DE +49 8945604050
> NO +47 73187518
> SE +46 46313131
> FR +33 497235101
> TW +886 35657119
> HU +36 13275600
> IE +353 91337900
> 
> Toll Free
> 
> UK 0800 1412084
> US +1 8668801148
> CN +86 4006782367
> IN 0008009868365
> IN +918049282778
> TW 08000 22065
> HU 0680981587
> IE 1800800022
> KF +972732558877
> 
> -- 
> Julien Grall
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [ovmf baseline-only test] 71925: all pass

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 71925 ovmf real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/71925/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf c65df5d9a14331d2b6d583359f1cf88c3b710d34
baseline version:
 ovmf fff2623cc2a5e3d85db201a4cf1ca8c595e20075

Last test of basis71920  2017-07-29 09:20:12 Z3 days
Testing same since71925  2017-08-01 13:47:14 Z0 days1 attempts


People who touched revisions under test:
  Yonghong Zhu 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xs.citrite.net/~osstest/testlogs/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


commit c65df5d9a14331d2b6d583359f1cf88c3b710d34
Author: Yonghong Zhu 
Date:   Thu Jul 27 11:20:26 2017 +0800

BaseTools: Fix the bug to correctly check Pcd type that in FDF file

We set Pcd value in FDF and used this Pcd as PatchableInModule type in
module, it cause build report generate failure. because we incorrectly
set the Pcd type during check whether the Pcd is used.

Cc: Liming Gao 
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Yonghong Zhu 
Reviewed-by: Liming Gao 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH OSSTEST 04/11] TestSupport: introduce set_host_prop

[Roger Pau Monne]

On Tue, Aug 01, 2017 at 04:07:18PM +0100, Ian Jackson wrote:
> Roger Pau Monne writes ("Re: [PATCH OSSTEST 04/11] TestSupport: introduce 
> set_host_prop"):
> > After reading README osstest doesn't seem to have any limitation on
> > the characters that can be used for host idents, would you be fine
> > with me modifying it to add that '-' cannot be used in host idents,
> > and then storing the putative host properties using the following
> > runvar format:
> > 
> > hostprops--=
> 
> Existing runvars use `_' so we have the choice of using a new
> structured name delimiter, or massaging the ident.  I think using a
> new delimiter is fine, but we should plan to have one new delimiter
> for all future uses.
> 
> If we are going to have runvars which have a different delimiter than
> `_', I think we should probabaly not choose `-'; the chance is too
> high that this will be used somewhere else.
> 
> Indeed, we currently already have runvars `diversehosts_xtf-x86'
> and `one.guest.osstest_tcpcheckport' in the history.
> 
> I did this
>select * from runvars where name ~* '[^-._0-9a-z]';
> and got hits only in play flights containing things which are
> obviously garbage generated by broken flight construction tools.
> 
> So we can choose any delimter except those.  I suggest `/' ?
> I think if we are trying to encode pathnames in runvar names,
> we are doing something wrong.

That seems fine to me.

Thanks, Roger.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2] xl: add --clear option to dmesg command

[Wei Liu]

On Tue, Aug 01, 2017 at 11:57:50PM +0800, Xiao Liang wrote:
> From: xiliang 
> 
> The manual of xl says --clear option is supported and that option worked for 
> xm. Add that to xl now.

I will wrap this long line to 72 columns while committing.

> 
> Signed-off-by: xiliang 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2] x86/hvm: Allow guest_request vm_events coming from userspace

[Tamas K Lengyel]

On Tue, Aug 1, 2017 at 4:30 AM, Andrew Cooper  wrote:
> On 01/08/17 10:46, Alexandru Isaila wrote:
>> Allow guest userspace code to request that a vm_event be sent out
>> via VMCALL. This functionality seems to be handy for a number of
>> Xen developers, as stated on the mailing list (thread "[Xen-devel]
>> HVMOP_guest_request_vm_event only works from guest in ring0").
>> This is a use case in communication between a userspace application
>> in the guest and the introspection application in dom0.
>>
>> Signed-off-by: Alexandru Isaila 
>
> This issue has been argued several times before, and while I am in
> favour of the change, there is a legitimate argument that it breaks one
> of our security boundaries.
>
> One intermediate option comes to mind however.
>
> Could we introduce a new monitor op which permits the use of
> HVMOP_guest_request_vm_event from userspace?  This way, it requires a
> positive action on behalf of the introspection agent to relax the CPL
> check, rather than having the CPL check unconditionally relaxed.

I agree, it would be required to gate this on a monitor option that is
disabled by default.

Tamas

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v4]Proposal to allow setting up shared memory areas between VMs from xl config file

[Zhongze Liu]

Hi Edgar,

Thank you for being interested in this project.

2017-08-01 5:55 GMT+08:00 Edgar E. Iglesias :
> On Mon, Jul 31, 2017 at 02:30:47PM -0700, Stefano Stabellini wrote:
>> On Mon, 31 Jul 2017, Edgar E. Iglesias wrote:
>> > On Fri, Jul 28, 2017 at 09:03:15PM +0800, Zhongze Liu wrote:
>> > > 
>> > > 1. Motivation and Description
>> >
>> > Hi,
>> >
>> > I think this looks quite useful. I have a few comments inline.
>>
>> Hi Edgar, thanks for giving it a look!
>>
>>
>> > > 
>> > > Virtual machines use grant table hypercalls to setup a share page for
>> > > inter-VMs communications. These hypercalls are used by all PV
>> > > protocols today. However, very simple guests, such as baremetal
>> > > applications, might not have the infrastructure to handle the grant 
>> > > table.
>> > > This project is about setting up several shared memory areas for 
>> > > inter-VMs
>> > > communications directly from the VM config file.
>> > > So that the guest kernel doesn't have to have grant table support (in the
>> > > embedded space, this is not unusual) to be able to communicate with
>> > > other guests.
>> > >
>> > > 
>> > > 2. Implementation Plan:
>> > > 
>> > >
>> > > ==
>> > > 2.1 Introduce a new VM config option in xl:
>> > > ==
>> > >
>> > > 2.1.1 Design Goals
>> > > ~~~
>> > >
>> > > The shared areas should be shareable among several (>=2) VMs, so every 
>> > > shared
>> > > physical memory area is assigned to a set of VMs. Therefore, a “token” or
>> > > “identifier” should be used here to uniquely identify a backing memory 
>> > > area.
>> > > A string no longer than 128 bytes is used here to serve the purpose.
>> > >
>> > > The backing area would be taken from one domain, which we will regard
>> > > as the "master domain", and this domain should be created prior to any
>> > > other "slave domain"s. Again, we have to use some kind of tag to tell who
>> > > is the "master domain".
>> > >
>> > > And the ability to specify the permissions and cacheability (and 
>> > > shareability
>> > > for ARM guest's) of the pages to be shared should also be given to the 
>> > > user.
>> > >
>> > > 2.2.2 Syntax and Behavior
>> > > ~
>> > > The following example illustrates the syntax of the proposed config entry
>> > > (suppose that we're on x86):
>> > >
>> > > In xl config file of vm1:
>> > >   static_shm = [ 'id=ID1, begin=0x10, end=0x20, role=master,
>> > >   cache_policy=x86_normal, prot=r2',
>> > >
>> > >  'id=ID2, begin=0x30, end=0x40, role=master' ]
>> > >
>> > > In xl config file of vm2:
>> > >   static_shm = [ 'id=ID1, offset = 0, begin=0x50, end=0x60,
>> > > role=slave, prot=ro' ]
>> > >
>> > > In xl config file of vm3:
>> > >   static_shm = [ 'id=ID2, offset = 1, begin=0x69,
>> > > end=0x80, role=slave, prot=ro' ]
>> > >
>> > > where:
>> > >   @id   The identifier of the backing memory area.
>> > > Can be any string that matches the regexp "[^ 
>> > > \t\n,]+"
>> > > and no longer than 128 characters
>> > >
>> > >   @offset   Can only appear when @role = slave. The sharing 
>> > > will
>> > > start from the beginning of backing memory area 
>> > > plus
>> > > this offset. If not set, it defaults to zero.
>> > > Can be decimals or hexadecimals of the form 
>> > > "0x2",
>> > > and should be the multiple of the hypervisor page
>> > > granularity (currently 4K on both ARM and x86).
>> > >
>> > >   @begin/endThe boundaries of the shared memory area. The 
>> > > format
>> > > requirements are the same with @offset.
>> >
>> > I'm assuming this is all specified in GFN and also not MFN contigous?
>> > Would it be possible to allow the specification of MFN mappings
>> > that are contigous?
>>
>> That could be done with the iomem= parameter?
>
> The missing part from the iomem paramater is the attributes, cacheability, 
> shared etc.
> But we could perhaps add that to iomem somehow.
>
>
>>
>>
>> > This would be useful to map specific kinds of memory (e.g On Chip RAMs).
>> >
>> > Other use-cases are when there are not only guests sharing
>> > the pages but also devices. In some cases these devs may be locked in
>> > with low-latency access to specific memory regions.
>> >
>> > Perhaps something like the following?
>> > addr=gfn@
>> > size=0x1000
>> >
>> > with mfn being optional?
>>
>> I can see that it might be useful, but we are trying to keep the scope
>> small to be able to complete the project within the limited 

[Xen-devel] [PATCH v2] xl: add --clear option to dmesg command

[Xiao Liang]

From: xiliang 

The manual of xl says --clear option is supported and that option worked for 
xm. Add that to xl now.

Signed-off-by: xiliang 
---
 tools/xl/xl_info.c | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/tools/xl/xl_info.c b/tools/xl/xl_info.c
index 94bd1fd9ab..e9890ca5f2 100644
--- a/tools/xl/xl_info.c
+++ b/tools/xl/xl_info.c
@@ -884,8 +884,12 @@ int main_dmesg(int argc, char **argv)
 libxl_xen_console_reader *cr;
 char *line;
 int opt, ret = 1;
+static struct option opts[] = {
+{"clear", 0, 0, 'c'},
+COMMON_LONG_OPTS
+};
 
-SWITCH_FOREACH_OPT(opt, "c", NULL, "dmesg", 0) {
+SWITCH_FOREACH_OPT(opt, "c", opts, "dmesg", 0) {
 case 'c':
 clear = 1;
 break;
-- 
2.13.3


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 11/13] xen/pvcalls: implement release command

[Boris Ostrovsky]

>> BTW, I also noticed that in rcvmsg you are calling
>> wait_event_interruptible() while holding the lock. Have you tested with
>> CONFIG_DEBUG_ATOMIC_SLEEP? (or maybe it's some other config  option that
>> would complain about those sorts of thing)
> I believe sleeping while holding a mutex is allowed. Sleeping in
> spinlocked paths is bad.

Oh, right. I was thinking about spinlocks. Sorry.

-boris

>
> BTW: You are looking for CONFIG_DEBUG_MUTEXES (see
> Documentation/locking/mutex-design.txt ).
>
>
> Juergen


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 11/13] xen/pvcalls: implement release command

[Juergen Gross]

On 01/08/17 17:23, Boris Ostrovsky wrote:
> On 07/31/2017 06:34 PM, Stefano Stabellini wrote:
>> On Thu, 27 Jul 2017, Boris Ostrovsky wrote:
 +int pvcalls_front_release(struct socket *sock)
 +{
 +  struct pvcalls_bedata *bedata;
 +  struct sock_mapping *map;
 +  int req_id, notify;
 +  struct xen_pvcalls_request *req;
 +
 +  if (!pvcalls_front_dev)
 +  return -EIO;
 +  bedata = dev_get_drvdata(_front_dev->dev);
 +  if (!bedata)
 +  return -EIO;
>>> Some (all?) other ops don't check bedata validity. Should they all do?
>> No, I don't think they should: dev_set_drvdata is called in the probe
>> function (pvcalls_front_probe). I'll remove it.
>>
>>
 +
 +  if (sock->sk == NULL)
 +  return 0;
 +
 +  map = (struct sock_mapping *) READ_ONCE(sock->sk->sk_send_head);
 +  if (map == NULL)
 +  return 0;
 +
 +  spin_lock(>pvcallss_lock);
 +  req_id = bedata->ring.req_prod_pvt & (RING_SIZE(>ring) - 1);
 +  if (RING_FULL(>ring) ||
 +  READ_ONCE(bedata->rsp[req_id].req_id) != PVCALLS_INVALID_ID) {
 +  spin_unlock(>pvcallss_lock);
 +  return -EAGAIN;
 +  }
 +  WRITE_ONCE(sock->sk->sk_send_head, NULL);
 +
 +  req = RING_GET_REQUEST(>ring, req_id);
 +  req->req_id = req_id;
 +  req->cmd = PVCALLS_RELEASE;
 +  req->u.release.id = (uint64_t)sock;
 +
 +  bedata->ring.req_prod_pvt++;
 +  RING_PUSH_REQUESTS_AND_CHECK_NOTIFY(>ring, notify);
 +  spin_unlock(>pvcallss_lock);
 +  if (notify)
 +  notify_remote_via_irq(bedata->irq);
 +
 +  wait_event(bedata->inflight_req,
 +  READ_ONCE(bedata->rsp[req_id].req_id) == req_id);
 +
 +  if (map->active_socket) {
 +  /* 
 +   * Set in_error and wake up inflight_conn_req to force
 +   * recvmsg waiters to exit.
 +   */
 +  map->active.ring->in_error = -EBADF;
 +  wake_up_interruptible(>active.inflight_conn_req);
 +
 +  mutex_lock(>active.in_mutex);
 +  mutex_lock(>active.out_mutex);
 +  pvcalls_front_free_map(bedata, map);
 +  mutex_unlock(>active.out_mutex);
 +  mutex_unlock(>active.in_mutex);
 +  kfree(map);
>>> Since you are locking here I assume you expect that someone else might
>>> also be trying to lock the map. But you are freeing it immediately after
>>> unlocking. Wouldn't that mean that whoever is trying to grab the lock
>>> might then dereference freed memory?
>> The lock is to make sure there are no recvmsg or sendmsg in progress. We
>> are sure that no newer sendmsg or recvmsg are waiting for
>> pvcalls_front_release to release the lock because before send a message
>> to the backend we set sk_send_head to NULL.
> 
> Is there a chance that whoever is potentially calling send/rcvmsg has
> checked that sk_send_head is non-NULL but hasn't grabbed the lock yet?
> 
> Freeing a structure containing a lock right after releasing the lock
> looks weird (to me). Is there any other way to synchronize with
> sender/receiver? Any other lock?

Right. This looks fishy. Either you don't need the locks or you can't
just free the area right after releasing the lock.

> BTW, I also noticed that in rcvmsg you are calling
> wait_event_interruptible() while holding the lock. Have you tested with
> CONFIG_DEBUG_ATOMIC_SLEEP? (or maybe it's some other config  option that
> would complain about those sorts of thing)

I believe sleeping while holding a mutex is allowed. Sleeping in
spinlocked paths is bad.

BTW: You are looking for CONFIG_DEBUG_MUTEXES (see
Documentation/locking/mutex-design.txt ).


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 11/13] xen/pvcalls: implement release command

[Boris Ostrovsky]

On 07/31/2017 06:34 PM, Stefano Stabellini wrote:
> On Thu, 27 Jul 2017, Boris Ostrovsky wrote:
>>> +int pvcalls_front_release(struct socket *sock)
>>> +{
>>> +   struct pvcalls_bedata *bedata;
>>> +   struct sock_mapping *map;
>>> +   int req_id, notify;
>>> +   struct xen_pvcalls_request *req;
>>> +
>>> +   if (!pvcalls_front_dev)
>>> +   return -EIO;
>>> +   bedata = dev_get_drvdata(_front_dev->dev);
>>> +   if (!bedata)
>>> +   return -EIO;
>> Some (all?) other ops don't check bedata validity. Should they all do?
> No, I don't think they should: dev_set_drvdata is called in the probe
> function (pvcalls_front_probe). I'll remove it.
>
>
>>> +
>>> +   if (sock->sk == NULL)
>>> +   return 0;
>>> +
>>> +   map = (struct sock_mapping *) READ_ONCE(sock->sk->sk_send_head);
>>> +   if (map == NULL)
>>> +   return 0;
>>> +
>>> +   spin_lock(>pvcallss_lock);
>>> +   req_id = bedata->ring.req_prod_pvt & (RING_SIZE(>ring) - 1);
>>> +   if (RING_FULL(>ring) ||
>>> +   READ_ONCE(bedata->rsp[req_id].req_id) != PVCALLS_INVALID_ID) {
>>> +   spin_unlock(>pvcallss_lock);
>>> +   return -EAGAIN;
>>> +   }
>>> +   WRITE_ONCE(sock->sk->sk_send_head, NULL);
>>> +
>>> +   req = RING_GET_REQUEST(>ring, req_id);
>>> +   req->req_id = req_id;
>>> +   req->cmd = PVCALLS_RELEASE;
>>> +   req->u.release.id = (uint64_t)sock;
>>> +
>>> +   bedata->ring.req_prod_pvt++;
>>> +   RING_PUSH_REQUESTS_AND_CHECK_NOTIFY(>ring, notify);
>>> +   spin_unlock(>pvcallss_lock);
>>> +   if (notify)
>>> +   notify_remote_via_irq(bedata->irq);
>>> +
>>> +   wait_event(bedata->inflight_req,
>>> +   READ_ONCE(bedata->rsp[req_id].req_id) == req_id);
>>> +
>>> +   if (map->active_socket) {
>>> +   /* 
>>> +* Set in_error and wake up inflight_conn_req to force
>>> +* recvmsg waiters to exit.
>>> +*/
>>> +   map->active.ring->in_error = -EBADF;
>>> +   wake_up_interruptible(>active.inflight_conn_req);
>>> +
>>> +   mutex_lock(>active.in_mutex);
>>> +   mutex_lock(>active.out_mutex);
>>> +   pvcalls_front_free_map(bedata, map);
>>> +   mutex_unlock(>active.out_mutex);
>>> +   mutex_unlock(>active.in_mutex);
>>> +   kfree(map);
>> Since you are locking here I assume you expect that someone else might
>> also be trying to lock the map. But you are freeing it immediately after
>> unlocking. Wouldn't that mean that whoever is trying to grab the lock
>> might then dereference freed memory?
> The lock is to make sure there are no recvmsg or sendmsg in progress. We
> are sure that no newer sendmsg or recvmsg are waiting for
> pvcalls_front_release to release the lock because before send a message
> to the backend we set sk_send_head to NULL.

Is there a chance that whoever is potentially calling send/rcvmsg has
checked that sk_send_head is non-NULL but hasn't grabbed the lock yet?

Freeing a structure containing a lock right after releasing the lock
looks weird (to me). Is there any other way to synchronize with
sender/receiver? Any other lock?

BTW, I also noticed that in rcvmsg you are calling
wait_event_interruptible() while holding the lock. Have you tested with
CONFIG_DEBUG_ATOMIC_SLEEP? (or maybe it's some other config  option that
would complain about those sorts of thing)

-boris




___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [linux-3.18 test] 112403: regressions - trouble: blocked/broken/fail/pass

[osstest service owner]

flight 112403 linux-3.18 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/112403/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 112102
 build-arm64-pvops 2 hosts-allocate broken REGR. vs. 112102
 build-arm64   2 hosts-allocate broken REGR. vs. 112102
 build-arm64-pvops 3 capture-logs   broken REGR. vs. 112102
 build-armhf-libvirt   5 host-build-prep  fail REGR. vs. 112102

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm 16 guest-localmigrate/x10 fail 
in 112394 pass in 112403
 test-armhf-armhf-xl-cubietruck 17 guest-start.2fail pass in 112394

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-arm64-arm64-examine  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64-xsm   3 capture-logs  broken blocked in 112102
 build-arm64   3 capture-logs  broken blocked in 112102
 test-amd64-amd64-xl-qemut-win7-amd64 18 guest-start/win.repeat fail blocked in 
112102
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop  fail blocked in 112102
 test-amd64-amd64-xl-qemuu-win7-amd64 18 guest-start/win.repeat fail in 112394 
blocked in 112102
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop  fail in 112394 like 112085
 test-armhf-armhf-libvirt 14 saverestore-support-check fail in 112394 like 
112102
 test-armhf-armhf-libvirt-raw 13 saverestore-support-check fail in 112394 like 
112102
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-check fail in 112394 like 
112102
 test-armhf-armhf-libvirt13 migrate-support-check fail in 112394 never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-check fail in 112394 never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-check fail in 112394 never pass
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 112085
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-localmigrate/x10 fail like 112102
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 112102
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 13 guest-saverestore   fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 13 guest-saverestore   fail never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 

Re: [Xen-devel] [PATCH] vtpmmgr: make inline functions static

[Olaf Hering]

Ping

On Fri, Jun 23, Olaf Hering wrote:

> gcc7 is more strict with functions marked as inline. They are not
> automatically inlined. Instead a function call is generated, but the
> actual code is not visible by the linker.
> 
> Do a mechanical change and mark every 'inline' as 'static inline'. For
> simpler review the static goes into an extra line.
> 
> Signed-off-by: Olaf Hering 
> ---
>  stubdom/vtpmmgr/marshal.h  | 76 
> ++
>  stubdom/vtpmmgr/tcg.h  | 14 
>  stubdom/vtpmmgr/tpm2_marshal.h | 58 
>  stubdom/vtpmmgr/tpmrsa.h   |  1 +
>  4 files changed, 149 insertions(+)
> 
> diff --git a/stubdom/vtpmmgr/marshal.h b/stubdom/vtpmmgr/marshal.h
> index d826f19d89..dce19c6439 100644
> --- a/stubdom/vtpmmgr/marshal.h
> +++ b/stubdom/vtpmmgr/marshal.h
> @@ -47,16 +47,19 @@ typedef enum UnpackPtr {
>   UNPACK_ALLOC
>  } UnpackPtr;
>  
> +static
>  inline BYTE* pack_BYTE(BYTE* ptr, BYTE t) {

...


signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH OSSTEST 04/11] TestSupport: introduce set_host_prop

[Ian Jackson]

Roger Pau Monne writes ("Re: [PATCH OSSTEST 04/11] TestSupport: introduce 
set_host_prop"):
> After reading README osstest doesn't seem to have any limitation on
> the characters that can be used for host idents, would you be fine
> with me modifying it to add that '-' cannot be used in host idents,
> and then storing the putative host properties using the following
> runvar format:
> 
> hostprops--=

Existing runvars use `_' so we have the choice of using a new
structured name delimiter, or massaging the ident.  I think using a
new delimiter is fine, but we should plan to have one new delimiter
for all future uses.

If we are going to have runvars which have a different delimiter than
`_', I think we should probabaly not choose `-'; the chance is too
high that this will be used somewhere else.

Indeed, we currently already have runvars `diversehosts_xtf-x86'
and `one.guest.osstest_tcpcheckport' in the history.

I did this
   select * from runvars where name ~* '[^-._0-9a-z]';
and got hits only in play flights containing things which are
obviously garbage generated by broken flight construction tools.

So we can choose any delimter except those.  I suggest `/' ?
I think if we are trying to encode pathnames in runvar names,
we are doing something wrong.

> Then I will remove the host parameter to the ts-save-props... script
> and call selecthost on every ident found in the runvars. This is going
> to generate some noise on the log I guess, but it should be fine.

Yes.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XenSummit 2017] Shared coprocessor framework followup

[Edgar E. Iglesias]

On Tue, Aug 01, 2017 at 02:52:22PM +0300, Andrii Anisov wrote:
> Dear Edgar,
> 
> 
> On 31.07.17 23:42, Edgar E. Iglesias wrote:
> >Yes I'm interested in this.
> It's good to hear at least one vote for the stuff :)
> 
> >  I'm not sure how much time I'll be able to contribute but at least I can 
> > review proposals and hopefully look at implementing a driver/backend that 
> > may be useful for our FPGA platforms.
> I really hope we can start from small things:
> - Could you please describe use-cases you have in your mind. What
> functionality do you expect? It is really important for us, we need some
> side view on the framework.
> - Also it would be good for us to have some view on the coprocessor (fpga?)
> you are going to share using SCF. How is it exposed to the system? Does it
> have mmio, ram, irq, iommu connection, whatever?

Hi,

I don't really have a defined list of requirements but I can do some initial
hand-waiving :-)

On the ZynqMP we have to classes of HW (on the chip) that I think may benefit.
1. The Cortex-R5s
2. The Programmable Logic (FPGA)

The Cortex-R5s are two real-time co-processors that can be programmed.
They have local RAMs and control registers to reset, halt etc.
I'll leave these out for now, they are probably more similar to the
use-cases you had in mind.

On the PL, there's a chunk of programmable logic that allows you to
create your own custom accellerators or devices.
Some devices are tied to specific boards (e.g when they depend on specific IO)
but others are not (for example memory to memory computational accelerators).
To communicate with these devices, they have memory slave and master ports
(for register accesses and for DMA). They also have interrupts both ways.

It's possible to reprogram the configuration of the PL and swap accelerators in
and out on the fly. It's probably going to be too slow for trying to
context switch between guests so I think primarily we would be looking at
a way to lets say, "allocate" and "release" the resources for batch use.

If a guest cannot allocate an accelerator, it could fall back to emulation
or just to using SW libraries until an accelerator slot is available.

Best regards,
Edgar


> - Please comment on SCF configuration followup letter [1]
> 
> [1]
> https://lists.xenproject.org/archives/html/xen-devel/2017-07/msg02124.html
> 
> -- 
> 
> *Andrii Anisov*
> 
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH OSSTEST 04/11] TestSupport: introduce set_host_prop

[Roger Pau Monne]

On Tue, Aug 01, 2017 at 03:13:33PM +0100, Ian Jackson wrote:
> Roger Pau Monne writes ("Re: [PATCH OSSTEST 04/11] TestSupport: introduce 
> set_host_prop"):
> > On Tue, Aug 01, 2017 at 02:01:35PM +0100, Ian Jackson wrote:
> > > TBH, since this is only being called in the one
> > > ts-set-host-properties-from-runvars script (or whatever you're calling
> > > it), I think you can use $mjobdb-> directly.  That's not too bad a
> > > layer violation.
> > 
> > In the new version that I've sent I've already added a helper to
> > TestSupport, it's just two lines of code so unless you feel really
> > annoyed by it I would probably leave it there.
> 
> No, sure, the helper is probably a bit better.
> 
> > > I think your runvars should probably be named after the ident, not the
> > > hostname.  That may involve rethinking your encoding, since idents can
> > > contain _ (hostnames can contain - but not _).
> > 
> > Does it make sense to have the hostname or the ident?
> > 
> > After all ts-set-host-properties-from-runvars is only going to save
> > properties for the host passed as 'host' ident, and I don't see much
> > reason for allowing it to support two different idents like src_host
> > or dst_host, or in general for having a test script that sets host
> > properties for multiple hosts.
> 
> I don't think this can be right.
> 
> You have a helper function that you pass a $ho, and which records the
> runvars, and which therefore pretends to arrange for the properties to
> be set (later), for that $ho.  It should do as it implicity promises.
> 
> That means the second actually-modifying-the-db ts-* script must
> process all of the specified runvars.  So it must call selecthost.  So
> really it should have the ident.  So the runvars should be named after
> the ident.

OK, I was calling selecthost using ARGS[0] ATM in the ts-save-props...
script, and I was passing 'host' to it from sg-run-job.

I can change ts-save-props... to instead call selecthost on every
ident that's found in the hostprops-- runvar.

After reading README osstest doesn't seem to have any limitation on
the characters that can be used for host idents, would you be fine
with me modifying it to add that '-' cannot be used in host idents,
and then storing the putative host properties using the following
runvar format:

hostprops--=

Then I will remove the host parameter to the ts-save-props... script
and call selecthost on every ident found in the runvars. This is going
to generate some noise on the log I guess, but it should be fine.

Roger.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH]tools: updated "xl_info.c" to accept "--clear" as parameter following "xl dmesg"

[Wei Liu]

On Tue, Aug 01, 2017 at 10:19:35PM +0800, Xiao Liang wrote:
> From 85c4bb378cb456fba96bbe6cdc8734f493daeb0c Mon Sep 17 00:00:00 2001
> From: xiliang 
> Date: Tue, 1 Aug 2017 17:33:02 +0800
> Subject: [PATCH] tools: updated "xl_info.c" to accept "--clear" as parameter
>  following "xl dmesg" In xl man page, adding "-c" or "--clear" following "xl
>  dmesg" can clear Xen's message buffer. It works in old "xm", so added support
>  to xl.

Also you might want to rework the commit message a bit.

When writing the commit message, the first line is going to be the
subject line of the email, and the rest the body of the commit message.

The subject line can be:

xl: add --clear option to dmesg command

The body can be:

The manual of xl says --clear option is supported and that option worked
for xm. Add that to xl now.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH]tools: updated "xl_info.c" to accept "--clear" as parameter following "xl dmesg"

[Wei Liu]

On Tue, Aug 01, 2017 at 10:19:35PM +0800, Xiao Liang wrote:
> Hello,
> 
> Today I found a minor problem that "xl dmesg --clear" failed to clear Xen's
> message buffer. From manual page, it should accept "-c" or "--clear".
> I proposed a fix and please help review. I also attached the path in the
> mail in case mail client format problem. Thanks
> Fail log:
> # xl dmesg --clear
> option `' not supported.

Yes we would accept a patch to add the long option.

> 
> Author: xiliang 
> Date:   Tue Aug 1 17:33:02 2017 +0800
> 
> tools: updated "xl_info.c" to accept "--clear" as parameter following
> "xl dmesg"
> In xl man page, adding "-c" or "--clear" following "xl dmesg" can clear
> Xen's message buffer. It works in old "xm", so added support to xl.
> 
> Signed-off-by: xiliang 
> 
> diff --git a/tools/xl/xl_info.c b/tools/xl/xl_info.c
> index 94bd1fd9ab..d6f723b4ad 100644
> --- a/tools/xl/xl_info.c
> +++ b/tools/xl/xl_info.c
> @@ -884,8 +884,11 @@ int main_dmesg(int argc, char **argv)
>  libxl_xen_console_reader *cr;
>  char *line;
>  int opt, ret = 1;
> +static struct option opts[] = {
> +{"clear", 0, 0, 'c'}

Missing COMMON_LONG_OPTS

And please use git send-email in the future to send your patch directly.

See https://wiki.xenproject.org/wiki/Submitting_Xen_Project_Patches

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-unstable test] 112401: trouble: blocked/broken/fail/pass

[osstest service owner]

flight 112401 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/112401/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 112286
 build-arm64-pvops 2 hosts-allocate broken REGR. vs. 112286
 build-arm64   2 hosts-allocate broken REGR. vs. 112286

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-localmigrate/x10 fail in 112391 
pass in 112401
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-localmigrate/x10 fail in 112391 
pass in 112401
 test-armhf-armhf-xl-rtds 12 guest-start  fail in 112391 pass in 112401
 test-amd64-amd64-rumprun-amd64 17 rumprun-demo-xenstorels/xenstorels.repeat 
fail in 112391 pass in 112401
 test-armhf-armhf-xl-credit2  16 guest-start/debian.repeat  fail pass in 112391

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop   fail REGR. vs. 112286

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-examine  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64-xsm   3 capture-logs  broken blocked in 112286
 build-arm64-pvops 3 capture-logs  broken blocked in 112286
 build-arm64   3 capture-logs  broken blocked in 112286
 test-amd64-amd64-xl-qemut-win7-amd64 18 guest-start/win.repeat fail blocked in 
112286
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop  fail in 112391 like 112286
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeatfail  like 112274
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 112286
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-localmigrate/x10 fail like 112286
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 112286
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 112286
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 112286
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 112286
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 13 guest-saverestore   fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-ws16-amd64 13 guest-saverestore   fail never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 10 windows-installfail never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail 

Re: [Xen-devel] Xen 4.9 + kernel 4.13rc2 -- ballooning regression? reappearance of "Over-allocation for domain 1" errors

[PGNet Dev]

On 7/28/17 9:02 AM, PGNet Dev wrote:

On 7/27/17 11:23 PM, Juergen Gross wrote:

Can you please post the domain's config file used to create the domain
and the kernel config?


Sure.

   https://pastebin.com/M6cr2pX7



Any add'l info needed?


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/2] arm: smccc: handle SMCs/HVCs according to SMCCC

[Edgar E. Iglesias]

On Tue, Aug 01, 2017 at 11:59:00AM +0100, Julien Grall wrote:
> (+ Edgar, Mark, Dave)
> 
> Hi,

Hi Julien,

I'll share some thoughts based on our platforms.


> On 14/06/17 15:10, Volodymyr Babchuk wrote:
> >SMCCC (SMC Call Convention) describes how to handle both HVCs and SMCs.
> >SMCCC states that both HVC and SMC are valid conduits to call to a different
> >firmware functions. Thus, for example PSCI calls can be made both by
> >SMC or HVC. Also SMCCC defines function number coding for such calls.
> >Besides functional calls there are query calls, which allows underling
> >OS determine version, UID and number of functions provided by service
> >provider.
> >
> >This patch adds new file `smccc.c`, which handles both generic SMCs
> >and HVC according to SMC. At this moment it implements only one
> >service: Standard Hypervisor Service.
> >
> >Standard Hypervisor Service only supports query calls, so caller can
> >ask about hypervisor UID and determine that it is XEN running.
> >
> >This change allows more generic handling for SMCs and HVCs and it can
> >be easily extended to support new services and functions.
> 
> I have already reviewed the code and one thing I missed is how a domain will
> know that Xen supports SMCCC.
> 
> Currently, Xen will:
>   - inject an undefined instruction for any SMC issued by a guest
>   - crash the guest (quite bad) for any unknown HCV #0
> 
> So a guest needs to be aware whether Xen supports SMCCC convention or not. I
> am not aware of any bindings in the device-tree for doing that.

On our platforms, SW probes the DT for specific service classes and then
probes for specific versions via SMC calls using the standard Version FIDs.
If the DT does not specify the firmware node, I don't think any SMCs will be
issued but the guest may not be functional (as the firmware interface is
mandatory).

I don't know of a generic DT node/compat that tells guests if SMCC probing
is allowed or not. Perhaps there should be one, or there should be yet
another service specific one for Hypervisors. I don't know.

For example, these are the nodes we've got (PSCI and EEMI/SIP):
psci {
compatible = "arm,psci-0.2";
method = "smc";
};

pmufw: firmware {
compatible = "xlnx,zynqmp-pm";
method = "smc";
interrupt-parent = <>;
interrupts = <0 35 4>;
};

SW that does not have DT support, will either directly probe the SMC
interface or in some cases just assume it's there and use it.

ZynqMP-wise, Xen is in a little bit of an akward position by messing
guests up if they probe for non-existing SMC services but I don't think
it's that bad. Mainly because there's really very little ZynqMP guests
that need the firmware would be able todo without it.

For other platforms and services, I guess FW may very well be optional
and probing makes more sence. So getting support for gracefully returning
Unknown FID still important...


> The other issue is not all the firmware may be SMCCC capable. We may want in
> the future to support other convention to allow baremetal OS running on Xen.
> This means a guest should be able to detect the convention used.

Perhaps this could be done by injecting DT fragments like we do for passthrough?

Cheers,
Edgar

> 
> I don't have a clear answer here yet, but thought it would be good to start
> a conversation.
> 
> Cheers,
> 
> -- 
> Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH]tools: updated "xl_info.c" to accept "--clear" as parameter following "xl dmesg"

[Xiao Liang]

Hello,

Today I found a minor problem that "xl dmesg --clear" failed to clear 
Xen's message buffer. From manual page, it should accept "-c" or "--clear".
I proposed a fix and please help review. I also attached the path in the 
mail in case mail client format problem. Thanks

Fail log:
# xl dmesg --clear
option `' not supported.

Author: xiliang 
Date:   Tue Aug 1 17:33:02 2017 +0800

tools: updated "xl_info.c" to accept "--clear" as parameter 
following "xl dmesg"
In xl man page, adding "-c" or "--clear" following "xl dmesg" can 
clear Xen's message buffer. It works in old "xm", so added support to xl.


Signed-off-by: xiliang 

diff --git a/tools/xl/xl_info.c b/tools/xl/xl_info.c
index 94bd1fd9ab..d6f723b4ad 100644
--- a/tools/xl/xl_info.c
+++ b/tools/xl/xl_info.c
@@ -884,8 +884,11 @@ int main_dmesg(int argc, char **argv)
 libxl_xen_console_reader *cr;
 char *line;
 int opt, ret = 1;
+static struct option opts[] = {
+{"clear", 0, 0, 'c'}
+};

-SWITCH_FOREACH_OPT(opt, "c", NULL, "dmesg", 0) {
+SWITCH_FOREACH_OPT(opt, "c", opts, "dmesg", 0) {
 case 'c':
 clear = 1;
 break;

Thanks,
Xiao Liang

>From 85c4bb378cb456fba96bbe6cdc8734f493daeb0c Mon Sep 17 00:00:00 2001
From: xiliang 
Date: Tue, 1 Aug 2017 17:33:02 +0800
Subject: [PATCH] tools: updated "xl_info.c" to accept "--clear" as parameter
 following "xl dmesg" In xl man page, adding "-c" or "--clear" following "xl
 dmesg" can clear Xen's message buffer. It works in old "xm", so added support
 to xl.

Signed-off-by: xiliang 
---
 tools/xl/xl_info.c | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/tools/xl/xl_info.c b/tools/xl/xl_info.c
index 94bd1fd9ab..d6f723b4ad 100644
--- a/tools/xl/xl_info.c
+++ b/tools/xl/xl_info.c
@@ -884,8 +884,11 @@ int main_dmesg(int argc, char **argv)
 libxl_xen_console_reader *cr;
 char *line;
 int opt, ret = 1;
+static struct option opts[] = {
+{"clear", 0, 0, 'c'}
+};
 
-SWITCH_FOREACH_OPT(opt, "c", NULL, "dmesg", 0) {
+SWITCH_FOREACH_OPT(opt, "c", opts, "dmesg", 0) {
 case 'c':
 clear = 1;
 break;
-- 
2.13.3


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH OSSTEST 04/11] TestSupport: introduce set_host_prop

[Ian Jackson]

Roger Pau Monne writes ("Re: [PATCH OSSTEST 04/11] TestSupport: introduce 
set_host_prop"):
> On Tue, Aug 01, 2017 at 02:01:35PM +0100, Ian Jackson wrote:
> > TBH, since this is only being called in the one
> > ts-set-host-properties-from-runvars script (or whatever you're calling
> > it), I think you can use $mjobdb-> directly.  That's not too bad a
> > layer violation.
> 
> In the new version that I've sent I've already added a helper to
> TestSupport, it's just two lines of code so unless you feel really
> annoyed by it I would probably leave it there.

No, sure, the helper is probably a bit better.

> > I think your runvars should probably be named after the ident, not the
> > hostname.  That may involve rethinking your encoding, since idents can
> > contain _ (hostnames can contain - but not _).
> 
> Does it make sense to have the hostname or the ident?
> 
> After all ts-set-host-properties-from-runvars is only going to save
> properties for the host passed as 'host' ident, and I don't see much
> reason for allowing it to support two different idents like src_host
> or dst_host, or in general for having a test script that sets host
> properties for multiple hosts.

I don't think this can be right.

You have a helper function that you pass a $ho, and which records the
runvars, and which therefore pretends to arrange for the properties to
be set (later), for that $ho.  It should do as it implicity promises.

That means the second actually-modifying-the-db ts-* script must
process all of the specified runvars.  So it must call selecthost.  So
really it should have the ident.  So the runvars should be named after
the ident.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] libxc: check pointer is not null before printing

[Ian Jackson]

Wei Liu writes ("[PATCH] libxc: check pointer is not null before printing"):
> Signed-off-by: Wei Liu 

Acked-by: Ian Jackson 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [OSSTEST PATCH] mg-repro-setup: Slightly better document the alloc: syntax

[Wei Liu]

On Tue, Aug 01, 2017 at 02:19:34PM +0100, Ian Jackson wrote:
> Provide a clearer indication that the  is passed to
> mg-allocate (and therefore, implicitly, that mg-allocate's docs should
> be consulted).
> 
> Signed-off-by: Ian Jackson 
> CC: Wei Liu 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [ovmf test] 112404: all pass - PUSHED

[osstest service owner]

flight 112404 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/112404/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf c65df5d9a14331d2b6d583359f1cf88c3b710d34
baseline version:
 ovmf fff2623cc2a5e3d85db201a4cf1ca8c595e20075

Last test of basis   112367  2017-07-28 18:48:49 Z3 days
Testing same since   112404  2017-08-01 02:49:30 Z0 days1 attempts


People who touched revisions under test:
  Yonghong Zhu 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=ovmf
+ revision=c65df5d9a14331d2b6d583359f1cf88c3b710d34
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push ovmf 
c65df5d9a14331d2b6d583359f1cf88c3b710d34
+ branch=ovmf
+ revision=c65df5d9a14331d2b6d583359f1cf88c3b710d34
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=ovmf
+ xenbranch=xen-unstable
+ '[' xovmf = xlinux ']'
+ linuxbranch=
+ '[' x = x ']'
+ qemuubranch=qemu-upstream-unstable
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable
+ prevxenbranch=xen-4.9-testing
+ '[' xc65df5d9a14331d2b6d583359f1cf88c3b710d34 = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/xtf.git
++ : osst...@xenbits.xen.org:/home/xen/git/xtf.git
++ : git://xenbits.xen.org/xtf.git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/rumprun.git
++ : git://git.seabios.org/seabios.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/seabios.git
++ : git://xenbits.xen.org/osstest/seabios.git
++ : https://github.com/tianocore/edk2.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/linux-firmware.git
++ : osst...@xenbits.xen.org:/home/osstest/ext/linux-firmware.git
++ : git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
++ : 

Re: [Xen-devel] [PATCH OSSTEST 04/11] TestSupport: introduce set_host_prop

[Roger Pau Monne]

On Tue, Aug 01, 2017 at 02:01:35PM +0100, Ian Jackson wrote:
> Roger Pau Monne writes ("Re: [PATCH OSSTEST 04/11] TestSupport: introduce 
> set_host_prop"):
> > IMHO, I think the right approach is to leave mg-hosts as it is now,
> 
> Yes.
> 
> > and implement a set_property in HostDB/{Executive/Static}.pm and
> > implement a helper in TestSupport that makes use of it
> > ($mhostdb->set_property(...)), do you agree?
> 
> TBH, since this is only being called in the one
> ts-set-host-properties-from-runvars script (or whatever you're calling
> it), I think you can use $mjobdb-> directly.  That's not too bad a
> layer violation.

In the new version that I've sent I've already added a helper to
TestSupport, it's just two lines of code so unless you feel really
annoyed by it I would probably leave it there.

> I think your runvars should probably be named after the ident, not the
> hostname.  That may involve rethinking your encoding, since idents can
> contain _ (hostnames can contain - but not _).

Does it make sense to have the hostname or the ident?

After all ts-set-host-properties-from-runvars is only going to save
properties for the host passed as 'host' ident, and I don't see much
reason for allowing it to support two different idents like src_host
or dst_host, or in general for having a test script that sets host
properties for multiple hosts.

Roger.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH] mg-repro-setup: Slightly better document the alloc: syntax

[Ian Jackson]

Provide a clearer indication that the  is passed to
mg-allocate (and therefore, implicitly, that mg-allocate's docs should
be consulted).

Signed-off-by: Ian Jackson 
CC: Wei Liu 
---
 mg-repro-setup | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mg-repro-setup b/mg-repro-setup
index 9663497..a30534a 100755
--- a/mg-repro-setup
+++ b/mg-repro-setup
@@ -28,7 +28,7 @@ usage () { cat <   host must be allocated, skip host-install
[host=]wipe:  host must be allocated, wipe it
-   [host=]alloc:mg-allocate and wipe
+   [host=]alloc:\`mg-allocate ', and wipe
none:   no hosts (should be only HOSTPSEC)
 
  OPTIONs
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 0/5] xen: RCU: x86/ARM: Add support of rcu_idle_{enter, exit}

[Dario Faggioli]

On Tue, 2017-08-01 at 13:02 +, Tomas Thoresen wrote:
> With the patch series I was able to create, destroy and re-create the
> domU.
>  
> "Tested-by: Tomas Thoresen "
>  
Thanks.

I'll have to resend the series. If I will not have to change the code
much (as it, so far, seems), I'll incorporate this tag.

Thanks again for testing. :-)

Dario
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R Ltd., Cambridge (UK)

signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [linux-linus test] 112400: regressions - trouble: blocked/broken/fail/pass

[osstest service owner]

flight 112400 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/112400/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64   2 hosts-allocate broken REGR. vs. 110515
 build-arm64-pvops 2 hosts-allocate broken REGR. vs. 110515
 build-arm64-xsm   2 hosts-allocate broken REGR. vs. 110515
 test-amd64-amd64-examine  7 reboot   fail REGR. vs. 110515
 test-amd64-amd64-libvirt-vhd  7 xen-boot fail REGR. vs. 110515
 test-amd64-amd64-libvirt-pair 10 xen-boot/src_host   fail REGR. vs. 110515
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-boot fail REGR. vs. 
110515
 test-amd64-amd64-rumprun-amd64  7 xen-boot   fail REGR. vs. 110515
 test-amd64-amd64-libvirt-pair 11 xen-boot/dst_host   fail REGR. vs. 110515
 test-amd64-amd64-xl-qemut-debianhvm-amd64  7 xen-bootfail REGR. vs. 110515
 test-amd64-amd64-qemuu-nested-intel  7 xen-boot  fail REGR. vs. 110515
 test-amd64-amd64-xl-pvh-intel  7 xen-bootfail REGR. vs. 110515
 test-amd64-amd64-pygrub   7 xen-boot fail REGR. vs. 110515
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-boot fail REGR. 
vs. 110515
 test-amd64-amd64-libvirt  7 xen-boot fail REGR. vs. 110515
 test-amd64-amd64-xl-qcow2 7 xen-boot fail REGR. vs. 110515
 test-amd64-amd64-i386-pvgrub  7 xen-boot fail REGR. vs. 110515
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-localmigrate/x10 fail REGR. vs. 
110515
 test-amd64-amd64-amd64-pvgrub  7 xen-bootfail REGR. vs. 110515
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-localmigrate/x10 fail in 112390 
REGR. vs. 110515

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-win7-amd64 15 guest-saverestore.2 fail pass in 112390
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-localmigrate/x10 fail pass in 
112390

Tests which did not succeed, but are not blocking:
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-examine  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64-xsm   3 capture-logs  broken blocked in 110515
 build-arm64-pvops 3 capture-logs  broken blocked in 110515
 build-arm64   3 capture-logs  broken blocked in 110515
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop   fail in 112390 like 110515
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 110515
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeatfail  like 110515
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 110515
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 110515
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 110515
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 110515
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 13 guest-saverestore   fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 13 guest-saverestore   fail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd 

[Xen-devel] [PATCH 0/5] xen: RCU: x86/ARM: Add support of rcu_idle_{enter, exit}

[Tomas Thoresen]

With the patch series I was able to create, destroy and re-create the domU.



"Tested-by: Tomas Thoresen >"


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH OSSTEST 04/11] TestSupport: introduce set_host_prop

[Ian Jackson]

Roger Pau Monne writes ("Re: [PATCH OSSTEST 04/11] TestSupport: introduce 
set_host_prop"):
> IMHO, I think the right approach is to leave mg-hosts as it is now,

Yes.

> and implement a set_property in HostDB/{Executive/Static}.pm and
> implement a helper in TestSupport that makes use of it
> ($mhostdb->set_property(...)), do you agree?

TBH, since this is only being called in the one
ts-set-host-properties-from-runvars script (or whatever you're calling
it), I think you can use $mjobdb-> directly.  That's not too bad a
layer violation.

I think your runvars should probably be named after the ident, not the
hostname.  That may involve rethinking your encoding, since idents can
contain _ (hostnames can contain - but not _).

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v4 06/13] libxl: change p9 to use generec add function

[Wei Liu]

On Tue, Aug 01, 2017 at 02:58:19PM +0300, Oleksandr Grytsov wrote:
> On Mon, Jul 31, 2017 at 5:36 PM, Wei Liu  wrote:
> > On Sun, Jul 30, 2017 at 09:42:09PM +0300, Oleksandr Grytsov wrote:
> >> On Fri, Jul 28, 2017 at 7:23 PM, Wei Liu  wrote:
> >> > On Fri, Jul 28, 2017 at 03:11:34PM +0100, Wei Liu wrote:
> >> >> On Tue, Jul 18, 2017 at 05:25:23PM +0300, Oleksandr Grytsov wrote:
> >> >> [...]
> >> >> >  /* Waits for the passed device to reach state XenbusStateInitWait.
> >> >> >   * This is not really useful by itself, but is important when 
> >> >> > executing
> >> >> >   * hotplug scripts, since we need to be sure the device is in the 
> >> >> > correct
> >> >> > @@ -3565,6 +3559,7 @@ extern const struct libxl_device_type 
> >> >> > libxl__usbctrl_devtype;
> >> >> >  extern const struct libxl_device_type libxl__usbdev_devtype;
> >> >> >  extern const struct libxl_device_type libxl__pcidev_devtype;
> >> >> >  extern const struct libxl_device_type libxl__vdispl_devtype;
> >> >> > +extern const struct libxl_device_type libxl__p9_devtype;
> >> >> >
> >> >> >  extern const struct libxl_device_type *device_type_tbl[];
> >> >> >
> >> >> > diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> >> >> > index 25563cf..96dbaed 100644
> >> >> > --- a/tools/libxl/libxl_types.idl
> >> >> > +++ b/tools/libxl/libxl_types.idl
> >> >> > @@ -804,7 +804,7 @@ libxl_domain_config = Struct("domain_config", [
> >> >> >  ("vfbs", Array(libxl_device_vfb, "num_vfbs")),
> >> >> >  ("vkbs", Array(libxl_device_vkb, "num_vkbs")),
> >> >> >  ("vtpms", Array(libxl_device_vtpm, "num_vtpms")),
> >> >> > -("p9", Array(libxl_device_p9, "num_p9s")),
> >> >> > +("p9s", Array(libxl_device_p9, "num_p9s")),
> >> >>
> >> >> Oh, no, please don't do this. We can't change the name of the fields.
> >> >>
> >> >> There is already on irregular device type -- the PCI device. I suppose
> >> >> you probably need another hook somewhere. And please convert PCI devices
> >> >> if you can.
> >> >
> >> > OK, going through the code I think we need to come to a conclusion if we
> >> > want an extra callback to handle the irregular device names first
> >> > because that's likely to affect the code of the framework in previous
> >> > patch.
> >>
> >> Actually creating new callback to handle irregular device name looks
> >> not so good.
> >> There is the pattern which all namings should follow. May be it has to
> >> be documented
> >
> > The normal pattern is DEVTYPEs.
> >
> >> somewhere. p9 was added recently we can ask the author to review this 
> >> rename.
> >
> > Once it is released we can't change it, of course unless we deem it
> > unstable. I'm two minded here. P9 was released in 4.9, which was only a
> > few months old.
> >
> 
> IMHO this the bug I mean wrong naming and it should be fixed.
> 
> > But we definitely can't change the PCI type. It has been around since
> > forever. And there is provision in code to deal with that.
> 
> Agree, I didn't touch PCI.
> 
> >> From other side this rename touches only internals changes: no changes
> >> in config file
> >> or CLI interface.
> >>
> >
> > As said, the framework need to be ready to deal with PCI anyway.
> >
> > What sort of issues do you foresee here?
> 
> Do you mean in case to rework PCI to use the device framework?
> 

No, I mean adding the new hook. You said "handle irregular device name
looks not so good"

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH v1 0/7] IPMMU-VMSA support on ARM

[Julien Grall]

On 26/07/17 16:09, Oleksandr Tyshchenko wrote:

From: Oleksandr Tyshchenko 

Hi, all.


Hi,

Please CC maintainers and any relevant person on the cover letter. This 
is quite useful to have in the inbox.



The purpose of this patch series is to add IPMMU-VMSA support to Xen on ARM.
It is VMSA-compatible IOMMU that integrated in the newest Renesas R-Car Gen3 
SoCs (ARM).
And this IOMMU can't share the page table with the CPU since it doesn't use the 
same page-table format
as the CPU on ARM therefore I name it "Non-shared" IOMMU.
This all means that current patch series must be based on "Non-shared" IOMMU 
support [1]
for the IPMMU-VMSA to be functional inside Xen.

The IPMMU-VMSA driver as well as the ARM LPAE allocator were directly ported 
from BSP for Linux the vendor provides:
git://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas-bsp.git rcar-3.5.3


I think this is probably a good starting point to discuss about IOMMU 
support in Xen. I skimmed through the patches and saw the words "rfc" 
and "ported from BSP".


At the moment for IOMMU we rely on the Linux community to do the review, 
but this is not the case here as it is an RFC.  I can definitely help to 
check if it comply for Xen, but I don't have the competence to tell 
whether it is valid for the hardware.


We may want to find a compromise to get it merged in Xen, but surely we 
don't want to build it by default at least until we had feedback from 
the community about the validity of the code here.


As I mentioned above, we are currently borrowing drivers from Linux and 
adapting for Xen. Today we support SMMUv{1,2} (we need to resync it) and 
there are plan to add IPMMU-VMSA (this series) and SMMUv3.


I am aware that Linux IOMMU subsystem has growing quite a lot making 
more tricky to get support in Xen. I wanted to get feedback how complex 
from you and Sameer how complex it was and whether we should consider 
doing our own.



Patch series was rebased on Xen 4.9.0 release and tested on Renesas R-Car Gen3 
H3 ES2.0/M3 based boards
with devices assigned to different domains.

You can find patch series here:
repo: https://github.com/otyshchenko1/xen.git branch: ipmmu_v2

P.S. There is one more patch which needs to be brought back to life [2]
Any reasons why this patch hasn't been upstremed yet?


The series didn't make it upstream. Feel free to resend it separately.



Thank you.

[1] [Xen-devel] [PATCH v2 00/13] "Non-shared" IOMMU support on ARM
https://www.mail-archive.com/xen-devel@lists.xen.org/msg115901.html

[2] [Xen-devel] [PATCH v8 02/28] xen: Add log2 functionality
https://lists.xenproject.org/archives/html/xen-devel/2016-02/msg00031.html

Oleksandr Tyshchenko (7):
  iommu/arm: ipmmu-vmsa: Add IPMMU-VMSA support
  iommu/arm: ipmmu-vmsa: Add Xen changes for main driver
  iommu/arm: ipmmu-vmsa: Add io-pgtables support
  iommu/arm: ipmmu-vmsa: Add Xen changes for io-pgtables
  iommu/arm: Build IPMMU-VMSA related stuff
  iommu/arm: ipmmu-vmsa: Deallocate page table asynchronously
  iommu/arm: ipmmu-vmsa: Enable VMSAv8-64 mode if IPMMU HW supports it

 xen/drivers/passthrough/arm/Makefile |3 +
 xen/drivers/passthrough/arm/io-pgtable-arm.c | 1331 +
 xen/drivers/passthrough/arm/io-pgtable.c |   91 +
 xen/drivers/passthrough/arm/io-pgtable.h |  220 +++
 xen/drivers/passthrough/arm/ipmmu-vmsa.c | 2611 ++
 5 files changed, 4256 insertions(+)
 create mode 100644 xen/drivers/passthrough/arm/io-pgtable-arm.c
 create mode 100644 xen/drivers/passthrough/arm/io-pgtable.c
 create mode 100644 xen/drivers/passthrough/arm/io-pgtable.h
 create mode 100644 xen/drivers/passthrough/arm/ipmmu-vmsa.c



Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH OSSTEST v2 07/11] ts-freebsd-host-install: add arguments to test memdisk append options

[Roger Pau Monne]

This is needed in order to figure out which memdisk options should be
used to boot the images on each specific box.

Note that upon success the script stores the tentative host property
in the runvars.

Signed-off-by: Roger Pau Monné 
---
Changes since v1:
 - Provide a --recordappend argument to force the recording the
   memdisk parameters.
 - Exit gracefully if a bootonly test is attempted against a
   non-supported architecture.
 - Use NONE instead of an empty string when calling
   setup_netboot_memdisk if nothing should be appended.
 - Do not perform any arch test in ts-freebsd-host-install.
---
 ts-freebsd-host-install | 25 -
 1 file changed, 24 insertions(+), 1 deletion(-)

diff --git a/ts-freebsd-host-install b/ts-freebsd-host-install
index 483b9aec..50af5dd3 100755
--- a/ts-freebsd-host-install
+++ b/ts-freebsd-host-install
@@ -41,6 +41,23 @@ use Osstest::TestSupport;
 
 tsreadconfig();
 
+our $bootonly;
+our $memdisk_append;
+our $record_append;
+while (@ARGV && $ARGV[0] =~ m/^-/g) {
+if ($ARGV[0] =~ m/^--memdiskappend=(.*)/) {
+$memdisk_append = $1;
+} elsif ($ARGV[0] eq "--testboot") {
+$memdisk_append //= "NONE";
+$bootonly = 1;
+} elsif ($ARGV[0] eq "--recordappend") {
+$record_append = 1;
+} else {
+die "Unknown argument $ARGV[0]";
+}
+shift @ARGV;
+}
+
 our ($whhost) = @ARGV;
 $whhost ||= 'host';
 our $ho= selecthost($whhost);
@@ -95,7 +112,7 @@ END
 
 # Setup the pxelinux config file
 logm("Booting from installer image at $pxeimg");
-setup_netboot_memdisk($ho, $pxeimg);
+setup_netboot_memdisk($ho, $pxeimg, $memdisk_append);
 }
 
 sub install () {
@@ -247,6 +264,12 @@ power_state($ho, 1);
 logm("Waiting for the installer to boot");
 await_tcp(get_timeout($ho,'reboot',$timeout), 5, $ho);
 
+if ($bootonly) {
+hostprop_putative_record($ho, "MemdiskAppend", $memdisk_append)
+if $record_append;
+exit 0;
+}
+
 # Next boot will be from local disk
 setup_netboot_local($ho);
 
-- 
2.11.0 (Apple Git-81)


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH OSSTEST v2 04/11] HostDB: introduce set_property

[Roger Pau Monne]

And provide a helper in TestSupport to use it. This allows osstest to
set host properties from test script themselves (instead of using
the mg-hosts clu).

Signed-off-by: Roger Pau Monné 
---
 Osstest/HostDB/Executive.pm | 19 +++
 Osstest/HostDB/Static.pm|  7 +++
 Osstest/TestSupport.pm  |  8 +++-
 3 files changed, 33 insertions(+), 1 deletion(-)

diff --git a/Osstest/HostDB/Executive.pm b/Osstest/HostDB/Executive.pm
index 300178bb..403c2f02 100644
--- a/Osstest/HostDB/Executive.pm
+++ b/Osstest/HostDB/Executive.pm
@@ -51,6 +51,25 @@ END
 }
 }
 
+sub set_property() {
+my ($hd, $ho, $prop, $val) = @_;
+my $rmq = $dbh_tests->prepare(<prepare(<execute($ho->{Name}, $prop);
+if (length $val) {
+$addq->execute($ho->{Name}, $prop, $val);
+   }
+});
+}
+
 sub get_flags ($$) {
 my ($hd, $ho) = @_;
 
diff --git a/Osstest/HostDB/Static.pm b/Osstest/HostDB/Static.pm
index 60f5d3c2..3191c565 100644
--- a/Osstest/HostDB/Static.pm
+++ b/Osstest/HostDB/Static.pm
@@ -40,6 +40,13 @@ sub get_properties ($$$) { #method
 my ($hd, $name, $hp) = @_;
 }
 
+sub set_property() {
+my ($hd, $ho, $prop, $val) = @_;
+
+die
+"Cannot set property in standalone mode for $ho->{Name} $prop = $val\n";
+}
+
 sub get_flags ($$) { #method
 my ($hd, $ho) = @_;
 
diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 0af5..27b2342c 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -80,7 +80,7 @@ BEGIN {
   get_target_property get_host_native_linux_console
   hostnamepath hostnamepath_list set_runtime_hostflag
   power_state power_cycle power_cycle_sleep
-  serial_fetch_logs
+  serial_fetch_logs set_host_property
   propname_massage propname_check
  
   get_stashed open_unique_stashfile compress_stashed
@@ -1183,6 +1183,12 @@ sub get_host_property ($$;$) {
 return defined($val) ? $val : $defval;
 }
 
+sub set_host_property ($$$) {
+my ($ho,$prop,$val) = @_;
+
+$mhostdb->set_property($ho, $prop, $val);
+}
+
 sub get_target_property ($$;$);
 sub get_target_property ($$;$) {
 my ($ho, $prop, $defval) = @_;
-- 
2.11.0 (Apple Git-81)


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH OSSTEST v2 02/11] ts-freebsd-host-install: fix image permissions

[Roger Pau Monne]

Make sure images copied to the tftp path have the right permissions,
so use dd instead of cp, which will obviously not preserve the
original permissions.

Signed-off-by: Roger Pau Monné 
Acked-by: Ian Jackson 
---
 ts-freebsd-host-install | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/ts-freebsd-host-install b/ts-freebsd-host-install
index 321763b0..483b9aec 100755
--- a/ts-freebsd-host-install
+++ b/ts-freebsd-host-install
@@ -76,7 +76,8 @@ targetpath=$4
 cd $basedir
 mkdir -p `dirname $sharedpath`
 if [ ! -f $sharedpath ]; then
-cp $imagepath $sharedpath.tmp
+rm $sharedpath.tmp
+dd if=$imagepath of=$sharedpath.tmp
 mv $sharedpath.tmp $sharedpath
 fi
 rm -f $targetpath
-- 
2.11.0 (Apple Git-81)


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH OSSTEST v2 00/11] Add support to examine the needed memdisk flags for each host

[Roger Pau Monne]

Hello,

This builds on top of my previous osstest FreeBSD support series, and
expands the examine flight in order to test which memdisk options
should be used for each host. Hopefully all of this will be automatic
upon running a examine flight. The required options are detected by
ts-memdisk-try-append and stored into the database by
ts-examine-hostprops-save.

This has been tested except for ts-examine-hostprops-save because I
haven't run it in a flight with "real" blessing.

The series can be found at:

git://xenbits.xen.org/people/royger/osstest.git freebsd_v10

Which as said is already rebased on top of the previous FreeBSD
series.

Note that before running the FreeBSD branch flights an examine flight
with "real" intended blessing must be run, so that the proper memdisk
append options are set for each host. After that FreeBSD flights can
be run as normal.

Thanks, Roger.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH OSSTEST v2 10/11] make-hosts-flight: set runvars for FreeBSD test

[Roger Pau Monne]

This is needed in order to run the memdisk test.

Signed-off-by: Roger Pau Monné 
Acked-by: Ian Jackson 
---
 make-hosts-flight | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/make-hosts-flight b/make-hosts-flight
index 0152dfe1..d5670857 100755
--- a/make-hosts-flight
+++ b/make-hosts-flight
@@ -69,10 +69,13 @@ hosts_iterate () {
 case $kern in
   xen|linux)
 local di_version=`getconfig_TftpDiVersion_suite $suite`
+local freebsd_runvars
+set_freebsd_runvars
 runvars+=" 
kernkind=pvops
all_host_di_version=$di_version
all_host_suite=$suite
+   $freebsd_runvars
  "
 ;;
 esac
-- 
2.11.0 (Apple Git-81)


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH OSSTEST v2 08/11] ts-memdisk-try-append: introduce a script to test memdisk options

[Roger Pau Monne]

The intended usage is to run this script against every host in order
to record the possible needed memdisk flags.

Signed-off-by: Roger Pau Monné 
---
Changes since v1:
 - Get the arch of the job and exit with 0 if it's not supported.
 - Pass the --recordappend argument to ts-memdisk-try-append.
---
 ts-memdisk-try-append | 45 +
 1 file changed, 45 insertions(+)
 create mode 100755 ts-memdisk-try-append

diff --git a/ts-memdisk-try-append b/ts-memdisk-try-append
new file mode 100755
index ..86c6ee41
--- /dev/null
+++ b/ts-memdisk-try-append
@@ -0,0 +1,45 @@
+#!/bin/bash
+
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2017 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see .
+
+set -xe -o posix
+
+arch=`perl -e '
+use Osstest;
+use Osstest::TestSupport;
+
+tsreadconfig();
+print $r{"arch"} or die $!;
+  '`
+
+case "$arch" in
+amd64)
+;;
+*)
+echo "Arch $arch not supported for memdisk tests"
+exit 0
+;;
+esac
+
+if ./ts-freebsd-host-install --testboot --recordappend $@; then
+exit 0
+elif ./ts-freebsd-host-install --testboot --recordappend \
+   --memdiskappend="raw" $@; then
+exit 0
+fi
+
+exit 1
-- 
2.11.0 (Apple Git-81)


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH OSSTEST v2 09/11] ts-examine-hostprops-save: introduce a script to save properties

[Roger Pau Monne]

The introduce script turns the properties stored in the runvars using
the format hostprop_$hotname_$prop=$val into host properties stored in
the database.

Signed-off-by: Roger Pau Monné 
---
Changes since v1:
 - Select a host for setting the properties.
 - Print a message before exiting if blessing != real.
 - Skip properties that don't contain the selected host.
---
 ts-examine-hostprops-save | 55 +++
 1 file changed, 55 insertions(+)
 create mode 100755 ts-examine-hostprops-save

diff --git a/ts-examine-hostprops-save b/ts-examine-hostprops-save
new file mode 100755
index ..7622a95c
--- /dev/null
+++ b/ts-examine-hostprops-save
@@ -0,0 +1,55 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2017 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see .
+
+use strict qw(vars);
+use DBI;
+use POSIX;
+
+unshift @INC, qw(.);
+use Osstest;
+use Osstest::TestSupport;
+
+tsreadconfig();
+
+our ($whhost) = @ARGV;
+$whhost ||= 'host';
+our $ho= selecthost($whhost);
+
+our $blessing = intended_blessing();
+if ($blessing ne "real")
+{
+logm("Not setting host propeties because blessing $blessing != real");
+exit 0;
+}
+
+logm("setting host properties");
+
+foreach my $k (sort keys %r) {
+next unless $k =~ m/^hostprop_([^_]*)_([^_]*)$/;
+my $hostname= $1;
+my $prop=$2;
+
+if ($hostname ne $ho->{Name})
+{
+logm("invalid hostname $hostname found, expecting $ho->{Name}");
+logm("skipping putative hostprop runvar $k=$r{$k}");
+next;
+}
+
+logm("recording for $hostname $prop=$r{$k}");
+set_host_property($ho, $prop, $r{$k});
+}
-- 
2.11.0 (Apple Git-81)


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH OSSTEST v2 03/11] sg-run-job: fix typo in the examine jobs

[Roger Pau Monne]

proc prep-job/host-examine-xen is declared twice, one of them should
be prep-job/host-examine-linux instead.

Signed-off-by: Roger Pau Monné 
Acked-by: Ian Jackson 
---
 sg-run-job | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sg-run-job b/sg-run-job
index b7ce963a..ed1ed3c8 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -671,7 +671,7 @@ proc prep-job/host-examine-xen {} { examine-host-prep }
 proc run-job/host-examine-xen {} { examine-host-examine xen }
 
 proc need-hosts/host-examine-linux {} { return {} }
-proc prep-job/host-examine-xen {} { examine-host-prep }
+proc prep-job/host-examine-linux {} { examine-host-prep }
 proc need-hosts/host-examine-linux {} { examine-host-examine debian }
 
 #-- builds --
-- 
2.11.0 (Apple Git-81)


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH OSSTEST v2 11/11] sg-run-job: hook the memdisk test into examine

[Roger Pau Monne]

Hook the memdisk parameter detection and the saving of the host
properties into the examine jobs.

Signed-off-by: Roger Pau Monné 
---
Changes since v1:
 - Run the memdisk test first (so that we don't leave the host in a
   weird state).
 - Pass a host to the examine-hostprops-save.
---
 sg-run-job | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/sg-run-job b/sg-run-job
index ed1ed3c8..b6a072f3 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -655,6 +655,7 @@ proc examine-host-install-xen {} {
 proc examine-host-examine {install} {
 global ok
 catching-otherwise fail {
+   run-ts .   =ts-memdisk-try-append + host
examine-host-install-$install
run-ts .   =ts-examine-serial-pre + host
run-ts .   reboot   ts-host-reboot+ host
@@ -663,6 +664,7 @@ proc examine-host-examine {install} {
 if {$ok} {
run-ts -.  =   ts-examine-serial-post + host
run-ts .   =   ts-examine-logs-save   + host
+   run-ts .   =   ts-examine-hostprops-save + host
 }
 }
 
-- 
2.11.0 (Apple Git-81)


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH OSSTEST v2 05/11] mfi-common: move set_freebsd_runvars to mfi-common

[Roger Pau Monne]

So that it can also be used by make-hosts-flight. No functional change
intended.

Signed-off-by: Roger Pau Monné 
Acked-by: Ian Jackson 
---
 make-freebsd-flight | 31 ---
 mfi-common  | 31 +++
 2 files changed, 31 insertions(+), 31 deletions(-)

diff --git a/make-freebsd-flight b/make-freebsd-flight
index 64dfe9a6..72695742 100755
--- a/make-freebsd-flight
+++ b/make-freebsd-flight
@@ -36,37 +36,6 @@ job_create_build_filter_callback () {
 :
 }
 
-set_freebsd_runvars () {
-# Caller should have done if required:
-# local freebsd_runvars
-#
-# Figure out where are the installer binaries. The order is the
-# following:
-#
-# 1. Env variable FREEBSD__BUILDJOB: use the output from a
-# previous build--freebsd.
-#
-# 2. Env variables FREEBSD_DIST, FREEBSD_VERSION: set before calling
-# into make-flight, provide the path to the installer image, the sets
-# to install and the version being installed.
-#
-# 3. Config file FreeBSDDist, FreeBSDVersion: same as 2. except that
-# they are set on the config file.
-#
-envvar="FREEBSD_${arch^^}_BUILDJOB"
-if [ -n "${!envvar}" ]; then
-freebsd_runvars="freebsdbuildjob=${!envvar}"
-elif [ -n "$FREEBSD_DIST" ] && [ -n "$FREEBSD_VERSION" ]; then
-freebsd_runvars="freebsd_distpath=$FREEBSD_DIST/$arch \
- freebsd_version=$FREEBSD_VERSION"
-else
-distpath=`getconfig "FreeBSDDist"`
-version=`getconfig "FreeBSDVersion"`
-freebsd_runvars="freebsd_distpath=$distpath/$arch \
- freebsd_version=$version"
-fi
-}
-
 for arch in "$arches"; do
 set_freebsd_runvars
 job_create_build build-$arch-freebsd build-freebsd  \
diff --git a/mfi-common b/mfi-common
index 4827c827..8a9546ab 100644
--- a/mfi-common
+++ b/mfi-common
@@ -113,6 +113,37 @@ set_hostos_runvars () {
   esac
 }
 
+set_freebsd_runvars () {
+# Caller should have done if required:
+# local freebsd_runvars
+#
+# Figure out where are the installer binaries. The order is the
+# following:
+#
+# 1. Env variable FREEBSD__BUILDJOB: use the output from a
+# previous build--freebsd.
+#
+# 2. Env variables FREEBSD_DIST, FREEBSD_VERSION: set before calling
+# into make-flight, provide the path to the installer image, the sets
+# to install and the version being installed.
+#
+# 3. Config file FreeBSDDist, FreeBSDVersion: same as 2. except that
+# they are set on the config file.
+#
+local envvar="FREEBSD_${arch^^}_BUILDJOB"
+if [ -n "${!envvar}" ]; then
+freebsd_runvars="freebsdbuildjob=${!envvar}"
+elif [ -n "$FREEBSD_DIST" ] && [ -n "$FREEBSD_VERSION" ]; then
+freebsd_runvars="freebsd_distpath=$FREEBSD_DIST/$arch \
+ freebsd_version=$FREEBSD_VERSION"
+else
+local distpath=`getconfig "FreeBSDDist"`
+local version=`getconfig "FreeBSDVersion"`
+freebsd_runvars="freebsd_distpath=$distpath/$arch \
+ freebsd_version=$version"
+fi
+}
+
 create_build_jobs () {
 
   local arch
-- 
2.11.0 (Apple Git-81)


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH OSSTEST v2 06/11] TestSupport: introduce hostprop_putative_record

[Roger Pau Monne]

This is used to store tentative host properties in the runvars of a
job, with the expectation that at some point (ie: at the end of the
job) they will be turned into real properties stored in the database.

Signed-off-by: Roger Pau Monné 
Acked-by: Ian Jackson 
---
 Osstest/TestSupport.pm | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 27b2342c..3c627e2a 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -82,6 +82,7 @@ BEGIN {
   power_state power_cycle power_cycle_sleep
   serial_fetch_logs set_host_property
   propname_massage propname_check
+  hostprop_putative_record
  
   get_stashed open_unique_stashfile compress_stashed
   dir_identify_vcs
@@ -1189,6 +1190,12 @@ sub set_host_property ($$$) {
 $mhostdb->set_property($ho, $prop, $val);
 }
 
+sub hostprop_putative_record ($$$) {
+my ($ho, $prop, $val) = @_;
+
+store_runvar("hostprop_$ho->{Name}_$prop", $val);
+}
+
 sub get_target_property ($$;$);
 sub get_target_property ($$;$) {
 my ($ho, $prop, $defval) = @_;
-- 
2.11.0 (Apple Git-81)


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH OSSTEST v2 01/11] netboot_memdisk: allow each host to have different append values

[Roger Pau Monne]

Some hosts require "append raw" [0] when booting with memdisk, while
others don't. This is based on the hardware/BIOS, and needs to be set
on a per-host basis.

In order to do this, add a new "MemdiskAppend" host property and make
use of it in the setup_netboot_memdisk helper.

[0] http://www.syslinux.org/wiki/index.php?title=MEMDISK#Memory_access_method

Signed-off-by: Roger Pau Monné 
---
Changes since v1:
 - Explicitly use NONE for no options set (instead of an empty string,
   which is the default).
 - Allow to manually pass append parameters.
---
 Osstest/TestSupport.pm | 14 ++
 1 file changed, 6 insertions(+), 8 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index b858ac82..0af5 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -2672,8 +2672,11 @@ default local
 END
 }
 
-sub setup_netboot_memdisk ($$) {
-my ($ho, $img) = @_;
+sub setup_netboot_memdisk ($$;$) {
+my ($ho, $img, $append) = @_;
+
+$append //= get_host_property($ho, "MemdiskAppend", "NONE");
+$append = $append eq "NONE" ? "" : "append $append";
 setup_netboot_bootcfg($ho, <

Re: [Xen-devel] [PATCH v2 1/2] libxl: use xen-blkback for 'vbd' disk types by default

[Marek Marczykowski-Górecki]

On Mon, Jul 31, 2017 at 05:01:08PM +0100, Wei Liu wrote:
> On Mon, Jul 31, 2017 at 04:56:04PM +0100, Wei Liu wrote:
> > On Fri, Jul 28, 2017 at 06:42:13PM +0200, Marek Marczykowski-Górecki wrote:
> > > This will allow later to make HVM domain without qemu in dom0 (in
> > > addition to the one in stubdomain).
> > > 
> > > Signed-off-by: Marek Marczykowski-Górecki 
> > > 
> > > 
> > > ---
> > > This is extracted from v1 of "libxl: do not start dom0 qemu for
> > > stubdomain when not needed".
> > > 
> > > Signed-off-by: Marek Marczykowski-Górecki 
> > > 
> > > ---
> > >  tools/libxl/libxl_disk.c | 4 +++-
> > >  1 file changed, 3 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/tools/libxl/libxl_disk.c b/tools/libxl/libxl_disk.c
> > > index 63de75c..7842d9b 100644
> > > --- a/tools/libxl/libxl_disk.c
> > > +++ b/tools/libxl/libxl_disk.c
> > > @@ -56,10 +56,12 @@ static void disk_eject_xswatch_callback(libxl__egc 
> > > *egc, libxl__ev_xswatch *w,
> > >  "/local/domain/%d/backend/%" TOSTRING(BACKEND_STRING_SIZE)
> > > "[a-z]/%*d/%*d",
> > > >backend_domid, backend_type);
> > > -if (!strcmp(backend_type, "tap") || !strcmp(backend_type, "vbd")) {
> > > +if (!strcmp(backend_type, "tap")) {
> > >  disk->backend = LIBXL_DISK_BACKEND_TAP;
> > >  } else if (!strcmp(backend_type, "qdisk")) {
> > >  disk->backend = LIBXL_DISK_BACKEND_QDISK;
> > > +} else if (!strcmp(backend_type, "vbd")) {
> > > +disk->backend = LIBXL_DISK_BACKEND_PHY;
> > 
> > Wait, it only occurred to me until now this patch is changing
> > disk_eject_xswatch_callback.
> > 
> > Is this a bug fix? How is it possible for the backend_type to be "vbd"
> > when there isn't such thing in libxl_types.idl?
> 
> Oh, I'm an idiot. That's read from xenstore path. I think this patch is
> correct. But I still tend to think this is a bug fix. How do you
> discover this problem? Has disk eject ever worked for you?

Hmm, I think I've failed with forward-porting this patch. In current Xen
it isn't possible to create cdrom with backend_type!=qdisk...

So, this patch alone isn't sufficient. Please ignore it.

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?


signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v4 06/13] libxl: change p9 to use generec add function

[Oleksandr Grytsov]

On Mon, Jul 31, 2017 at 5:36 PM, Wei Liu  wrote:
> On Sun, Jul 30, 2017 at 09:42:09PM +0300, Oleksandr Grytsov wrote:
>> On Fri, Jul 28, 2017 at 7:23 PM, Wei Liu  wrote:
>> > On Fri, Jul 28, 2017 at 03:11:34PM +0100, Wei Liu wrote:
>> >> On Tue, Jul 18, 2017 at 05:25:23PM +0300, Oleksandr Grytsov wrote:
>> >> [...]
>> >> >  /* Waits for the passed device to reach state XenbusStateInitWait.
>> >> >   * This is not really useful by itself, but is important when executing
>> >> >   * hotplug scripts, since we need to be sure the device is in the 
>> >> > correct
>> >> > @@ -3565,6 +3559,7 @@ extern const struct libxl_device_type 
>> >> > libxl__usbctrl_devtype;
>> >> >  extern const struct libxl_device_type libxl__usbdev_devtype;
>> >> >  extern const struct libxl_device_type libxl__pcidev_devtype;
>> >> >  extern const struct libxl_device_type libxl__vdispl_devtype;
>> >> > +extern const struct libxl_device_type libxl__p9_devtype;
>> >> >
>> >> >  extern const struct libxl_device_type *device_type_tbl[];
>> >> >
>> >> > diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
>> >> > index 25563cf..96dbaed 100644
>> >> > --- a/tools/libxl/libxl_types.idl
>> >> > +++ b/tools/libxl/libxl_types.idl
>> >> > @@ -804,7 +804,7 @@ libxl_domain_config = Struct("domain_config", [
>> >> >  ("vfbs", Array(libxl_device_vfb, "num_vfbs")),
>> >> >  ("vkbs", Array(libxl_device_vkb, "num_vkbs")),
>> >> >  ("vtpms", Array(libxl_device_vtpm, "num_vtpms")),
>> >> > -("p9", Array(libxl_device_p9, "num_p9s")),
>> >> > +("p9s", Array(libxl_device_p9, "num_p9s")),
>> >>
>> >> Oh, no, please don't do this. We can't change the name of the fields.
>> >>
>> >> There is already on irregular device type -- the PCI device. I suppose
>> >> you probably need another hook somewhere. And please convert PCI devices
>> >> if you can.
>> >
>> > OK, going through the code I think we need to come to a conclusion if we
>> > want an extra callback to handle the irregular device names first
>> > because that's likely to affect the code of the framework in previous
>> > patch.
>>
>> Actually creating new callback to handle irregular device name looks
>> not so good.
>> There is the pattern which all namings should follow. May be it has to
>> be documented
>
> The normal pattern is DEVTYPEs.
>
>> somewhere. p9 was added recently we can ask the author to review this rename.
>
> Once it is released we can't change it, of course unless we deem it
> unstable. I'm two minded here. P9 was released in 4.9, which was only a
> few months old.
>

IMHO this the bug I mean wrong naming and it should be fixed.

> But we definitely can't change the PCI type. It has been around since
> forever. And there is provision in code to deal with that.

Agree, I didn't touch PCI.

>> From other side this rename touches only internals changes: no changes
>> in config file
>> or CLI interface.
>>
>
> As said, the framework need to be ready to deal with PCI anyway.
>
> What sort of issues do you foresee here?

Do you mean in case to rework PCI to use the device framework?


-- 
Best Regards,
Oleksandr Grytsov.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XenSummit 2017] Shared coprocessor framework followup

[Andrii Anisov]

Dear Edgar,


On 31.07.17 23:42, Edgar E. Iglesias wrote:

Yes I'm interested in this.

It's good to hear at least one vote for the stuff :)


  I'm not sure how much time I'll be able to contribute but at least I can 
review proposals and hopefully look at implementing a driver/backend that may 
be useful for our FPGA platforms.

I really hope we can start from small things:
- Could you please describe use-cases you have in your mind. What 
functionality do you expect? It is really important for us, we need some 
side view on the framework.
- Also it would be good for us to have some view on the coprocessor 
(fpga?) you are going to share using SCF. How is it exposed to the 
system? Does it have mmio, ram, irq, iommu connection, whatever?

- Please comment on SCF configuration followup letter [1]

[1] 
https://lists.xenproject.org/archives/html/xen-devel/2017-07/msg02124.html


--

*Andrii Anisov*



___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH V2 2/2] xen:arm: earlyprintk configuration for R-Car Gen3 boards

[Julien Grall]

Hi Andrii,

On 27/07/17 15:50, Andrii Anisov wrote:

From: Andrii Anisov 

Introduce an earlyprintk configuration for R-Car Gen3 SoC based development
boards, like:
 - Salvator-X [http://elinux.org/R-Car/Boards/Salvator-X]
 - M3ULCB [http://elinux.org/R-Car/Boards/M3SK]
 - H3ULCB [http://elinux.org/R-Car/Boards/H3SK]

Signed-off-by: Iurii Konovalenko 
Signed-off-by: Iurii Mykhalskyi 
Signed-off-by: Andrii Anisov 


Acked-by: Julien Grall 

Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH V2 1/2] xen:arm64: Add SCIF UART support for earlyprintk

[Julien Grall]

Hi Andrii,

On 27/07/17 15:50, Andrii Anisov wrote:

From: Iurii Konovalenko 

Add support for a SCIF compatible UART found in Renesas R-Car Gen3 SoCs.

Signed-off-by: Iurii Konovalenko 
Signed-off-by: Iurii Mykhalskyi 
Signed-off-by: Andrii Anisov 


Acked-by: Julien Grall 

Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v2 3/6] xen/arm: Allow platform_hvc to handle guest SMC calls

[Edgar E. Iglesias]

On Tue, Aug 01, 2017 at 11:37:05AM +0100, Julien Grall wrote:
> Hi Edgar,
> 
> On 31/07/17 23:23, Edgar E. Iglesias wrote:
> >On Thu, Feb 09, 2017 at 12:32:09PM -0700, Tamas K Lengyel wrote:
> >>On Thu, Feb 9, 2017 at 11:43 AM, Stefano Stabellini
> >> wrote:
> >>>On Thu, 9 Feb 2017, Tamas K Lengyel wrote:
> On Thu, Feb 9, 2017 at 11:22 AM, Stefano Stabellini
>  wrote:
> >On Thu, 9 Feb 2017, Edgar E. Iglesias wrote:
> >>On Thu, Feb 09, 2017 at 10:12:41AM +0100, Edgar E. Iglesias wrote:
> >>>On Wed, Feb 08, 2017 at 05:20:44PM -0800, Stefano Stabellini wrote:
> On Thu, 9 Feb 2017, Julien Grall wrote:
> >On 08/02/2017 23:28, Tamas K Lengyel wrote:
> >>On Wed, Feb 8, 2017 at 3:04 PM, Julien Grall  
> >>wrote:
> >>>Hi Tamas,
> >
> >..
> >
> In principle I have nothing against a command line option, but I don't
> really follow how that would help. The monitor system is disabled by
> default for all domains, so there is no problem with dom0 booting or
> any other domain needing to access the firmware. You specifically have
> to enable the monitoring for domains. Why is it a problem to have it
> be exclusive for just those domains where it is enabled?
> >>>
> >>>I am suggesting this solution because I expect many use-cases for memory
> >>>introspection that don't actually require any platform_hvc events to be
> >>>monitored at all. On the other end, I expect that on platforms where
> >>>platform_hvc is implemented, such as the ZynqMP, those calls are
> >>>important and should be handled in Xen in most cases.
> >>>
> >>>Looking at the code, does monitor.privileged_call_enabled only cover
> >>>SMC? Is monitor.privileged_call_enabled disabled by default?
> >>>If so, monitor.privileged_call_enabled could be the tunable I was
> >>>talking about. As long as enabling memory introspection doesn't
> >>>automatically forward platform_hvc events to the monitor, I am fine with
> >>>it.
> >>
> >>Yes, monitor.privileged_call_enabled only covers SMCs right now and it
> >>is disabled by default. It has to be enabled specifically for a
> >>domain.  Memory introspection is separate from this, that is handled
> >>by the mem_access system and it can be enabled separately from SMC
> >>monitoring.
> >>
> >>As for hypercalls that get handled by Xen, I don't really need to
> >>monitor those. If Xen would on the other hand go and call some
> >>firmware as a result of the hypercall, I would need to be able to deny
> >>that. So as long as XSM can be used to control HVC calls, that works
> >>for me just fine too.
> >
> >Hi again!
> >
> >This was quite a while ago but I think we kind of ended up with
> >monitor.privileged_call_enabled being a possible flag to conditionalize
> >the forwarding of firmware calls or not.
> >
> >There are at least 3 cases to consider at the moment:
> >1. Firmware calls over SMC (PSCI or other platform calls like EEMI)
> >2. Firmware calls over HVC Handled by Xen (PSCI and XEN Hypercalls)
> >3. Firmware calls over HVC Handled by platform specific code (e.g EEMI)
> >
> >
> >For #1 Firmware calls over SMC:
> >I've conditionalized all of it on monitor.privileged_call_enabled.
> >It's either the monitor or the firmware call handling, they
> >are mutually exclusive. Guests can still do PSCI over HVC.
> >
> >For #2, things work like today. This is PSCI and the Xen Hypercallsi over 
> >HVC.
> >
> >For #3, only platform code knows if the specific call will be handled
> >in Xen completely or if it will result in some kind of SMC to lower layers.
> >If monitor.privileged_call_enabled is on, I've made the ZynqMP
> >implementation gracefully NACK any call that would result in an SMC
> >issued by Xen.
> >
> >Are there any concerns around this?
> 
> You may want to have a look at the discussion about SMC and Xen:
> 
> https://lists.xenproject.org/archives/html/xen-devel/2017-02/msg01226.html
> 
> IIRC, the consensus is to exclusively forward SMC to the monitor if enabled:
> 
> "10. Domains on which the monitor privileged call feature is enabled
> (which is by default disabled for all domains) should not be able to
> issue firmware calls via SMCs/HVCs so that such calls reach the
> firmware directly. Xen should not bounce such calls to the firmware on
> behalf of the domain. Xen should not alter the state of the domain
> automatically (ie. incrementing PC). These calls should be exclusively
> transfered to the monitor subscriber for further processing.
> Hypercalls, virtual PSCI calls, virtual CPU services calls and virtual
> ARM architecture service calls remain unaffected.".
>

Thanks Julien.

I think it's better for EEMI to wait for the SMCC patches to go in and
base the EEMI mediator on top of them. I'll hold back a little more with this.

Cheers,
Edgar 

___
Xen-devel mailing list
Xen-devel@lists.xen.org

[Xen-devel] [PATCH] libxc: check pointer is not null before printing

[Wei Liu]

Signed-off-by: Wei Liu 
---
 tools/libxc/xc_dom_core.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tools/libxc/xc_dom_core.c b/tools/libxc/xc_dom_core.c
index cf403432d2..b5f316a1dc 100644
--- a/tools/libxc/xc_dom_core.c
+++ b/tools/libxc/xc_dom_core.c
@@ -754,7 +754,8 @@ struct xc_dom_image *xc_dom_allocate(xc_interface *xch,
 struct xc_dom_image *dom;
 
 xc_dom_printf(xch, "%s: cmdline=\"%s\", features=\"%s\"",
-  __FUNCTION__, cmdline, features);
+  __FUNCTION__, cmdline ? cmdline : "",
+  features ? features : "");
 dom = malloc(sizeof(*dom));
 if ( !dom )
 goto err;
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 00/13] "Non-shared" IOMMU support on ARM

[Oleksandr Tyshchenko]

Hi, Kevin

On Tue, Aug 1, 2017 at 6:06 AM, Tian, Kevin  wrote:
>> From: Oleksandr Tyshchenko [mailto:olekst...@gmail.com]
>> Sent: Monday, July 31, 2017 7:58 PM
>>
>> Hi, Kevin
>>
>> On Mon, Jul 31, 2017 at 8:57 AM, Tian, Kevin  wrote:
>> >> From: Oleksandr Tyshchenko
>> >> Sent: Wednesday, July 26, 2017 1:27 AM
>> >>
>> >> From: Oleksandr Tyshchenko 
>> >>
>> >> Hi, all.
>> >>
>> >> The purpose of this patch series is to create a base for porting
>> >> any "Non-shared" IOMMUs to Xen on ARM. Saying "Non-shared" IOMMU
>> I
>> >> mean
>> >> the IOMMU that can't share the page table with the CPU.
>> >
>> > Is "non-shared" IOMMU a standard terminology in ARM side? I quickly
>> > searched to find it mostly used in this thread...
>> I don't think that it is a standard terminology.
>>
>> >
>> > On the other hand, all IOMMUs support a basic DMA remapping
>> > mechanism with page table not shared with CPU. Then some IOMMUs
>> > may optional support Shared Virtual Memory (SVM) through page
>> > sharing with CPU. Then I'm not sure why need to highlight the
>> > "non-shared" manner in this thread, instead of just saying
>> > IPMMU-VMSA support...
>> I wouldn't use "IPMMU-VMSA support" in this thread since it may be any
>> other IOMMUs which can't share page table
>> with CPU because of format incompatibilities.
>
> As I commented you can assume all IOMMUs cannot share page
> table with CPU as the starting point. It's not good to name an IOMMU
> driver based on such fact.
>
>> I needed something short to describe such IOMMUs, but, If title
>> "non-shared" IOMMU sounds confusing
>> I won't use it anymore. Do you have something in mind?
>
> IOMMU driver needs to be vendor specific. Is your driver working
> for all IPMMU-VMSA compatible IOMMUs or only for Renesas?
> If the latter, you may make the name explicit for such purpose.
>
> btw since you're porting Linux driver to Xen. What 's the name
> used in Linux side? that should be a good reference to you.

I am afraid a misunderstanding took place. Let me elaborate a bit more
about this.

The IOMMU driver I am porting to Xen is IPMMU-VMSA [1]. This name is
used in Linux
and this name was retained during porting to Xen. This driver is
intended to work with Renesas VMSA-compatible IPMMUs.
But, IPMMU-VMSA support is not a target for the current thread, there
is another thread for adding it [2].

The purpose of the current thread is to create ground for IPMMU-VMSA IOMMUs
(as well as other IOMMUs which can't share page table with CPU because
of format incompatibilities) to be functional inside Xen on ARM.
The only IOMMU supported today in Xen on ARM is the ARM SMMU (which
uses the same page table format as the CPU and can share page table
with it).
And ARM specific code assumes that P2M table is *always* shared and
acts accordingly. So, this patch series is trying
to, let's say, brake this assumption and create environment to handle
such IOMMUs as well.
So, I may use the whole sentence as a patch series title in order not
to confuse people:
"Support IOMMUs which don't share page table with the CPU on ARM"
No objections?

P.S. This patch series also touches x86 specific code, it's because I
had to modify common code.

[1] 
http://elixir.free-electrons.com/linux/latest/source/drivers/iommu/ipmmu-vmsa.c
[2] https://lists.xen.org/archives/html/xen-devel/2017-07/msg02679.html

>
>>
>> >
>> >> Primarily, we are interested in IPMMU-VMSA and I hope that it will be the
>> >> first candidate.
>> >> It is VMSA-compatible IOMMU that integrated in the newest Renesas R-
>> Car
>> >> Gen3 SoCs (ARM).
>> >> I am about to push IPMMU-VMSA support in a while.
>> >>
>> >> With regard to the patch series, it was rebased on Xen 4.9.0 release and
>> >> tested on Renesas R-Car Gen3
>> >> H3/M3 based boards with applied IPMMU-VMSA support:
>> >> - Patches 1 and 3 have Julien's Rb.
>> >> - Patch 2 has Jan's Rb but only for x86 and generic parts.
>> >> - Patch 4 has Julien's Ab.
>> >> - Patches 5,6,9,10 were slightly reworked.
>> >> - Patch 7 was significantly reworked. The previous patch -> iommu: Split
>> >> iommu_hwdom_init() into arch specific parts
>> >> - Patches 8,11,12,13 are new.
>> >>
>> >> Not really sure about x86-related changes since I had no possibility to
>> check.
>> >> So, compile-tested on x86.
>> >>
>> >> You can find current patch series here:
>> >> repo: https://github.com/otyshchenko1/xen.git branch:
>> >> non_shared_iommu_v2
>> >>
>> >> Previous patch series here:
>> >> [PATCH v1 00/10] "Non-shared" IOMMU support on ARM
>> >> https://www.mail-archive.com/xen-devel@lists.xen.org/msg107532.html
>> >>
>> >> [RFC PATCH 0/9] "Non-shared" IOMMU support on ARM
>> >> https://www.mail-archive.com/xen-devel@lists.xen.org/msg100468.html
>> >>
>> >> Thank you.
>> >>
>> >> Oleksandr Tyshchenko (13):
>> >>   xen/device-tree: Add dt_count_phandle_with_args helper
>> >>   iommu: Add extra order argument to the IOMMU APIs 

Re: [Xen-devel] RT-Xen on ARM

[Andrii Anisov]

Hello Meng Xu,

I've get back to this stuff.


On 03.07.17 17:58, Andrii Anisov wrote:
That's why we are going to keep configuration (of guests and 
workloads) close to [1] for evaluation, but on our target SoC.

I'm wondering if there are known issues or specifics for ARM.

[1] https://www.cis.upenn.edu/~linhphan/papers/emsoft14-rt-xen.pdf
Currently I have a setup with dom0 and domU's with Litmus-RT. Following 
the document I need workload tasks.
Maybe you have mentioned workload tasks sources you can share, so that 
would shorten my steps.


--

*Andrii Anisov*



___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2] x86/hvm: Allow guest_request vm_events coming from userspace

[Alexandru Stefan ISAILA]

I'm sure we can to this and use a monitor op together with the 
HVMOP_guest_request_vm_event event. We have discussed this
and have a good idea on how to do it. 

~Alex

From: Andrew Cooper 
Sent: Tuesday, August 1, 2017 1:30 PM
To: Alexandru Stefan ISAILA; xen-devel@lists.xen.org
Cc: jbeul...@suse.com; Razvan Cojocaru; Tamas K Lengyel
Subject: Re: [PATCH v2] x86/hvm: Allow guest_request vm_events coming from 
userspace

On 01/08/17 10:46, Alexandru Isaila wrote:
> Allow guest userspace code to request that a vm_event be sent out
> via VMCALL. This functionality seems to be handy for a number of
> Xen developers, as stated on the mailing list (thread "[Xen-devel]
> HVMOP_guest_request_vm_event only works from guest in ring0").
> This is a use case in communication between a userspace application
> in the guest and the introspection application in dom0.
>
> Signed-off-by: Alexandru Isaila 

This issue has been argued several times before, and while I am in
favour of the change, there is a legitimate argument that it breaks one
of our security boundaries.

One intermediate option comes to mind however.

Could we introduce a new monitor op which permits the use of
HVMOP_guest_request_vm_event from userspace?  This way, it requires a
positive action on behalf of the introspection agent to relax the CPL
check, rather than having the CPL check unconditionally relaxed.

I think this would be sufficient to alleviate the previous objections.

~Andrew

>
> ---
> Changes since V1:
>   - Added Fallthrough check on mode == 2
> ---
>  xen/arch/x86/hvm/hypercall.c | 6 ++
>  1 file changed, 6 insertions(+)
>
> diff --git a/xen/arch/x86/hvm/hypercall.c b/xen/arch/x86/hvm/hypercall.c
> index e7238ce..1c067c3 100644
> --- a/xen/arch/x86/hvm/hypercall.c
> +++ b/xen/arch/x86/hvm/hypercall.c
> @@ -152,9 +152,15 @@ int hvm_hypercall(struct cpu_user_regs *regs)
>  {
>  case 8:
>  eax = regs->rax;
> +if ( eax == __HYPERVISOR_hvm_op &&
> + regs->rdi == HVMOP_guest_request_vm_event )
> +break;
>  /* Fallthrough to permission check. */
>  case 4:
>  case 2:
> +if ( mode != 8 && eax == __HYPERVISOR_hvm_op &&
> + regs->ebx == HVMOP_guest_request_vm_event )
> +break;
>  if ( unlikely(hvm_get_cpl(curr)) )
>  {
>  default:



This email was scanned by Bitdefender

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/2] arm: smccc: handle SMCs/HVCs according to SMCCC

[Julien Grall]

(+ Edgar, Mark, Dave)

Hi,

On 14/06/17 15:10, Volodymyr Babchuk wrote:

SMCCC (SMC Call Convention) describes how to handle both HVCs and SMCs.
SMCCC states that both HVC and SMC are valid conduits to call to a different
firmware functions. Thus, for example PSCI calls can be made both by
SMC or HVC. Also SMCCC defines function number coding for such calls.
Besides functional calls there are query calls, which allows underling
OS determine version, UID and number of functions provided by service
provider.

This patch adds new file `smccc.c`, which handles both generic SMCs
and HVC according to SMC. At this moment it implements only one
service: Standard Hypervisor Service.

Standard Hypervisor Service only supports query calls, so caller can
ask about hypervisor UID and determine that it is XEN running.

This change allows more generic handling for SMCs and HVCs and it can
be easily extended to support new services and functions.


I have already reviewed the code and one thing I missed is how a domain 
will know that Xen supports SMCCC.


Currently, Xen will:
- inject an undefined instruction for any SMC issued by a guest
- crash the guest (quite bad) for any unknown HCV #0

So a guest needs to be aware whether Xen supports SMCCC convention or 
not. I am not aware of any bindings in the device-tree for doing that.


The other issue is not all the firmware may be SMCCC capable. We may 
want in the future to support other convention to allow baremetal OS 
running on Xen. This means a guest should be able to detect the 
convention used.


I don't have a clear answer here yet, but thought it would be good to 
start a conversation.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2] xen: get rid of paravirt op adjust_exception_frame

[Juergen Gross]

When running as Xen pv-guest the exception frame on the stack contains
%r11 and %rcx additional to the other data pushed by the processor.

Instead of having a paravirt op being called for each exception type
prepend the Xen specific code to each exception entry. When running as
Xen pv-guest just use the exception entry with prepended instructions,
otherwise use the entry without the Xen specific code.

Signed-off-by: Juergen Gross 
---
 arch/x86/entry/entry_64.S | 22 ++
 arch/x86/entry/entry_64_compat.S  |  1 -
 arch/x86/include/asm/desc.h   | 16 ++
 arch/x86/include/asm/paravirt.h   |  5 ---
 arch/x86/include/asm/paravirt_types.h |  4 ---
 arch/x86/include/asm/proto.h  |  3 ++
 arch/x86/include/asm/traps.h  | 51 +--
 arch/x86/kernel/asm-offsets_64.c  |  1 -
 arch/x86/kernel/paravirt.c|  3 --
 arch/x86/kernel/traps.c   | 57 ++-
 arch/x86/xen/enlighten_pv.c   | 16 +-
 arch/x86/xen/irq.c|  3 --
 arch/x86/xen/xen-asm_64.S | 45 ---
 arch/x86/xen/xen-ops.h|  1 -
 14 files changed, 147 insertions(+), 81 deletions(-)

diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
index a9a8027a6c0e..602bcf68a32c 100644
--- a/arch/x86/entry/entry_64.S
+++ b/arch/x86/entry/entry_64.S
@@ -745,7 +745,6 @@ ENTRY(\sym)
.endif
 
ASM_CLAC
-   PARAVIRT_ADJUST_EXCEPTION_FRAME
 
.ifeq \has_error_code
pushq   $-1 /* ORIG_RAX: no syscall to 
restart */
@@ -901,7 +900,7 @@ ENTRY(do_softirq_own_stack)
 END(do_softirq_own_stack)
 
 #ifdef CONFIG_XEN
-idtentry xen_hypervisor_callback xen_do_hypervisor_callback has_error_code=0
+idtentry hypervisor_callback xen_do_hypervisor_callback has_error_code=0
 
 /*
  * A note on the "critical region" in our callback handler.
@@ -967,8 +966,6 @@ ENTRY(xen_failsafe_callback)
movq8(%rsp), %r11
addq$0x30, %rsp
pushq   $0  /* RIP */
-   pushq   %r11
-   pushq   %rcx
jmp general_protection
 1: /* Segment mismatch => Category 1 (Bad segment). Retry the IRET. */
movq(%rsp), %rcx
@@ -997,9 +994,8 @@ idtentry int3   do_int3 
has_error_code=0paranoid=1 shift_ist=DEBUG_STACK
 idtentry stack_segment do_stack_segmenthas_error_code=1
 
 #ifdef CONFIG_XEN
-idtentry xen_debug do_debughas_error_code=0
-idtentry xen_int3  do_int3 has_error_code=0
-idtentry xen_stack_segment do_stack_segmenthas_error_code=1
+idtentry xendebug  do_debughas_error_code=0
+idtentry xenint3   do_int3 has_error_code=0
 #endif
 
 idtentry general_protectiondo_general_protection   has_error_code=1
@@ -1161,18 +1157,6 @@ END(error_exit)
 /* Runs on exception stack */
 ENTRY(nmi)
/*
-* Fix up the exception frame if we're on Xen.
-* PARAVIRT_ADJUST_EXCEPTION_FRAME is guaranteed to push at most
-* one value to the stack on native, so it may clobber the rdx
-* scratch slot, but it won't clobber any of the important
-* slots past it.
-*
-* Xen is a different story, because the Xen frame itself overlaps
-* the "NMI executing" variable.
-*/
-   PARAVIRT_ADJUST_EXCEPTION_FRAME
-
-   /*
 * We allow breakpoints in NMIs. If a breakpoint occurs, then
 * the iretq it performs will take us out of NMI context.
 * This means that we can have nested NMIs where the next
diff --git a/arch/x86/entry/entry_64_compat.S b/arch/x86/entry/entry_64_compat.S
index e1721dafbcb1..843d2f08397b 100644
--- a/arch/x86/entry/entry_64_compat.S
+++ b/arch/x86/entry/entry_64_compat.S
@@ -294,7 +294,6 @@ ENTRY(entry_INT80_compat)
/*
 * Interrupts are off on entry.
 */
-   PARAVIRT_ADJUST_EXCEPTION_FRAME
ASM_CLAC/* Do this early to minimize exposure */
SWAPGS
 
diff --git a/arch/x86/include/asm/desc.h b/arch/x86/include/asm/desc.h
index d0a21b12dd58..f581e61ffcaa 100644
--- a/arch/x86/include/asm/desc.h
+++ b/arch/x86/include/asm/desc.h
@@ -9,6 +9,8 @@
 #include 
 #include 
 
+#include 
+
 static inline void fill_ldt(struct desc_struct *desc, const struct user_desc 
*info)
 {
desc->limit0= info->limit & 0x0;
@@ -83,6 +85,12 @@ static inline phys_addr_t get_cpu_gdt_paddr(unsigned int cpu)
return per_cpu_ptr_to_phys(get_cpu_gdt_rw(cpu));
 }
 
+#if defined(CONFIG_X86_64) && defined(CONFIG_XEN_PV)
+#define pv_trap_entry(name) (void *)(xen_pv_domain() ? xen_ ## name : name)
+#else
+#define pv_trap_entry(name) (void *)(name)
+#endif
+
 #ifdef CONFIG_X86_64
 
 static 

Re: [Xen-devel] [RFC v2 3/6] xen/arm: Allow platform_hvc to handle guest SMC calls

[Julien Grall]

Hi Edgar,

On 31/07/17 23:23, Edgar E. Iglesias wrote:

On Thu, Feb 09, 2017 at 12:32:09PM -0700, Tamas K Lengyel wrote:

On Thu, Feb 9, 2017 at 11:43 AM, Stefano Stabellini
 wrote:

On Thu, 9 Feb 2017, Tamas K Lengyel wrote:

On Thu, Feb 9, 2017 at 11:22 AM, Stefano Stabellini
 wrote:

On Thu, 9 Feb 2017, Edgar E. Iglesias wrote:

On Thu, Feb 09, 2017 at 10:12:41AM +0100, Edgar E. Iglesias wrote:

On Wed, Feb 08, 2017 at 05:20:44PM -0800, Stefano Stabellini wrote:

On Thu, 9 Feb 2017, Julien Grall wrote:

On 08/02/2017 23:28, Tamas K Lengyel wrote:

On Wed, Feb 8, 2017 at 3:04 PM, Julien Grall  wrote:

Hi Tamas,


..


In principle I have nothing against a command line option, but I don't
really follow how that would help. The monitor system is disabled by
default for all domains, so there is no problem with dom0 booting or
any other domain needing to access the firmware. You specifically have
to enable the monitoring for domains. Why is it a problem to have it
be exclusive for just those domains where it is enabled?


I am suggesting this solution because I expect many use-cases for memory
introspection that don't actually require any platform_hvc events to be
monitored at all. On the other end, I expect that on platforms where
platform_hvc is implemented, such as the ZynqMP, those calls are
important and should be handled in Xen in most cases.

Looking at the code, does monitor.privileged_call_enabled only cover
SMC? Is monitor.privileged_call_enabled disabled by default?
If so, monitor.privileged_call_enabled could be the tunable I was
talking about. As long as enabling memory introspection doesn't
automatically forward platform_hvc events to the monitor, I am fine with
it.


Yes, monitor.privileged_call_enabled only covers SMCs right now and it
is disabled by default. It has to be enabled specifically for a
domain.  Memory introspection is separate from this, that is handled
by the mem_access system and it can be enabled separately from SMC
monitoring.

As for hypercalls that get handled by Xen, I don't really need to
monitor those. If Xen would on the other hand go and call some
firmware as a result of the hypercall, I would need to be able to deny
that. So as long as XSM can be used to control HVC calls, that works
for me just fine too.


Hi again!

This was quite a while ago but I think we kind of ended up with
monitor.privileged_call_enabled being a possible flag to conditionalize
the forwarding of firmware calls or not.

There are at least 3 cases to consider at the moment:
1. Firmware calls over SMC (PSCI or other platform calls like EEMI)
2. Firmware calls over HVC Handled by Xen (PSCI and XEN Hypercalls)
3. Firmware calls over HVC Handled by platform specific code (e.g EEMI)


For #1 Firmware calls over SMC:
I've conditionalized all of it on monitor.privileged_call_enabled.
It's either the monitor or the firmware call handling, they
are mutually exclusive. Guests can still do PSCI over HVC.

For #2, things work like today. This is PSCI and the Xen Hypercallsi over HVC.

For #3, only platform code knows if the specific call will be handled
in Xen completely or if it will result in some kind of SMC to lower layers.
If monitor.privileged_call_enabled is on, I've made the ZynqMP
implementation gracefully NACK any call that would result in an SMC
issued by Xen.

Are there any concerns around this?


You may want to have a look at the discussion about SMC and Xen:

https://lists.xenproject.org/archives/html/xen-devel/2017-02/msg01226.html

IIRC, the consensus is to exclusively forward SMC to the monitor if enabled:

"10. Domains on which the monitor privileged call feature is enabled
(which is by default disabled for all domains) should not be able to
issue firmware calls via SMCs/HVCs so that such calls reach the
firmware directly. Xen should not bounce such calls to the firmware on
behalf of the domain. Xen should not alter the state of the domain
automatically (ie. incrementing PC). These calls should be exclusively
transfered to the monitor subscriber for further processing.
Hypercalls, virtual PSCI calls, virtual CPU services calls and virtual
ARM architecture service calls remain unaffected.".

Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 5/5] xen: RCU: avoid busy waiting until the end of grace period.

[Dario Faggioli]

On Tue, 2017-08-01 at 11:22 +0100, Julien Grall wrote:
> On 01/08/17 10:17, Dario Faggioli wrote:
> > As soon as this (callbacks being invoked) will have happened, we
> > won't
> > interrupt it any longer.
> > 
> > And idle CPUs _without_ queued RCU callbacks, won't be interrupted
> > at
> > all.
> 
> Oh, the commit message is not clear about it. The wording gives the 
> impression the timer will always be here on every idle CPU(s). In
> the 
> case on active CPU(s) you specific mention the end of the grace
> period.
> 
Ok, I'll try to improve the changelog and resend.

Thanks and Regards,
Dario
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R Ltd., Cambridge (UK)

signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel