[Xen-devel] [qemu-mainline test] 146060: regressions - FAIL

2020-01-13 Thread osstest service owner
flight 146060 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146060/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm   6 xen-buildfail REGR. vs. 144861
 build-arm64   6 xen-buildfail REGR. vs. 144861
 build-amd64   6 xen-buildfail REGR. vs. 144861
 build-i386-xsm6 xen-buildfail REGR. vs. 144861
 build-armhf   6 xen-buildfail REGR. vs. 144861
 build-amd64-xsm   6 xen-buildfail REGR. vs. 144861
 build-i3866 xen-buildfail REGR. vs. 144861

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-xl-pvshim 1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked  n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-rtds  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1) blocked n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-seattle   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvshim1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)   blocked  n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit1   1 

[Xen-devel] [qemu-mainline test] 146057: regressions - FAIL

2020-01-13 Thread osstest service owner
flight 146057 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146057/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm   6 xen-buildfail REGR. vs. 144861
 build-arm64   6 xen-buildfail REGR. vs. 144861
 build-amd64   6 xen-buildfail REGR. vs. 144861
 build-i386-xsm6 xen-buildfail REGR. vs. 144861
 build-armhf   6 xen-buildfail REGR. vs. 144861
 build-amd64-xsm   6 xen-buildfail REGR. vs. 144861
 build-i3866 xen-buildfail REGR. vs. 144861

Tests which did not succeed, but are not blocking:
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-armhf-armhf-xl-cubietruck  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-shadow 1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-rtds  1 build-check(1)   blocked  n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-seattle   1 build-check(1)   blocked  n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-shadow1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-pvshim 1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-rtds  1 build-check(1)   blocked  n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 

[Xen-devel] [ovmf test] 146056: regressions - trouble: broken/fail/pass

2020-01-13 Thread osstest service owner
flight 146056 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146056/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-ovmf-amd64 broken
 test-amd64-amd64-xl-qemuu-ovmf-amd64 4 host-install(4) broken REGR. vs. 145767
 test-amd64-i386-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail REGR. vs. 145767

version targeted for testing:
 ovmf b112ec225f1c5f85b7b7db4110b4d4f1a0626d6b
baseline version:
 ovmf 70911f1f4aee0366b6122f2b90d367ec0f066beb

Last test of basis   145767  2020-01-08 00:39:09 Z6 days
Failing since145774  2020-01-08 02:50:20 Z5 days   33 attempts
Testing same since   146056  2020-01-13 23:39:24 Z0 days1 attempts


People who touched revisions under test:
  Albecki, Mateusz 
  Ard Biesheuvel 
  Ashish Singhal 
  Eric Dong 
  Fan, ZhijuX 
  Jason Voelz 
  Laszlo Ersek 
  Mateusz Albecki 
  Michael D Kinney 
  Pavana.K 
  Philippe Mathieu-Daud? 
  Philippe Mathieu-Daude 
  Siyuan Fu 
  Siyuan, Fu 
  Vitaly Cheptsov 
  Vitaly Cheptsov via Groups.Io 
  Zhiju.Fan 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 broken  
 test-amd64-i386-xl-qemuu-ovmf-amd64  fail



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job test-amd64-amd64-xl-qemuu-ovmf-amd64 broken
broken-step test-amd64-amd64-xl-qemuu-ovmf-amd64 host-install(4)

Not pushing.

(No revision log; it would be 445 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] docs/misc: pvcalls: Verbatim block should be indented with 4 spaces

2020-01-13 Thread Stefano Stabellini
On Mon, 13 Jan 2020, Julien Grall wrote:
> At the moment, the diagram is only indented with 2 spaces. So pandoc
> will try to badly interpret it and not display it correctly.
> 
> Fix it by indenting all the block by 4 spaces (i.e an extra 2 spaces).
> 
> Fixes: d661611d08 ("docs/markdown: Switch to using pandoc, and fix underscore 
> escaping")
> Signed-off-by: Julien Grall 

Acked-by: Stefano Stabellini 

Thanks!

> ---
>  docs/misc/pvcalls.pandoc | 36 ++--
>  1 file changed, 18 insertions(+), 18 deletions(-)
> 
> diff --git a/docs/misc/pvcalls.pandoc b/docs/misc/pvcalls.pandoc
> index 0c48b29842..729cf97bdf 100644
> --- a/docs/misc/pvcalls.pandoc
> +++ b/docs/misc/pvcalls.pandoc
> @@ -867,24 +867,24 @@ and the second half to the **out** array. They are used 
> as circular
>  buffers for transferring data, and, together, they are the data ring.
>  
>  
> -  +---+ Indexes page
> -  | Command ring: | +--+
> -  | @0: xen_pvcalls_connect:  | |@0 pvcalls_data_intf: |
> -  | @44: ref  +>+@76: ring_order = 1   |
> -  |   | |@80: ref[0]+  |
> -  +---+ |@84: ref[1]+  |
> -|   |  |
> -|   |  |
> -+--+
> -|
> -v (data ring)
> -+---+---+
> -|  @0->4098: in |
> -|  ref[0]   |
> -|---|
> -|  @4099->8196: out |
> -|  ref[1]   |
> -+---+
> ++---+ Indexes page
> +| Command ring: | 
> +--+
> +| @0: xen_pvcalls_connect:  | |@0 pvcalls_data_intf: 
> |
> +| @44: ref  +>+@76: ring_order = 1   
> |
> +|   | |@80: ref[0]+  
> |
> ++---+ |@84: ref[1]+  
> |
> +  |   |  
> |
> +  |   |  
> |
> +  
> +--+
> +  |
> +  v (data 
> ring)
> +  
> +---+---+
> +  |  @0->4098: in
>  |
> +  |  ref[0]  
>  |
> +  
> |---|
> +  |  @4099->8196: 
> out |
> +  |  ref[1]  
>  |
> +  
> +---+
>  
>  
>   Indexes Page Structure
> -- 
> 2.17.1
> 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-unstable test] 146050: tolerable FAIL - PUSHED

2020-01-13 Thread osstest service owner
flight 146050 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146050/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeatfail  like 146006
 test-amd64-amd64-xl-rtds 16 guest-localmigrate   fail  like 146018
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 146039
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 146039
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 146039
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 146039
 test-amd64-amd64-qemuu-nested-intel 17 debian-hvm-install/l1/l2 fail like 
146039
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 146039
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 146039
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 146039
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 146039
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop fail like 146039
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-arm64-arm64-xl-seattle  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass

version targeted for testing:
 xen  03bfe526ecadc86f31eda433b91dc90be0563919
baseline version:
 xen  8842d01b300919e20bca2e1138c458a8483600f8

Last test of basis   146039  2020-01-13 03:13:30 Z0 days
Testing same since   146050  2020-01-13 16:36:37 Z0 days1 attempts


People who touched revisions under test:
  George Dunlap 
  Jan Beulich 
  Julien Grall 


[Xen-devel] [qemu-mainline test] 146055: regressions - FAIL

2020-01-13 Thread osstest service owner
flight 146055 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146055/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm   6 xen-buildfail REGR. vs. 144861
 build-arm64   6 xen-buildfail REGR. vs. 144861
 build-amd64   6 xen-buildfail REGR. vs. 144861
 build-i386-xsm6 xen-buildfail REGR. vs. 144861
 build-armhf   6 xen-buildfail REGR. vs. 144861
 build-amd64-xsm   6 xen-buildfail REGR. vs. 144861
 build-i3866 xen-buildfail REGR. vs. 144861

Tests which did not succeed, but are not blocking:
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-arm64-arm64-xl-seattle   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-rtds  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-armhf-armhf-xl-cubietruck  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 test-amd64-i386-xl-pvshim 1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvshim1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-rtds  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-pygrub   

Re: [Xen-devel] [RFC PATCH V2 11/11] x86: tsc: avoid system instability in hibernation

2020-01-13 Thread Rafael J. Wysocki
On Mon, Jan 13, 2020 at 10:50 PM Rafael J. Wysocki  wrote:
>
> On Mon, Jan 13, 2020 at 1:43 PM Peter Zijlstra  wrote:
> >
> > On Mon, Jan 13, 2020 at 11:43:18AM +, Singh, Balbir wrote:
> > > For your original comment, just wanted to clarify the following:
> > >
> > > 1. After hibernation, the machine can be resumed on a different but 
> > > compatible
> > > host (these are VM images hibernated)
> > > 2. This means the clock between host1 and host2 can/will be different
> > >
> > > In your comments are you making the assumption that the host(s) is/are the
> > > same? Just checking the assumptions being made and being on the same page 
> > > with
> > > them.
> >
> > I would expect this to be the same problem we have as regular suspend,
> > after power off the TSC will have been reset, so resume will have to
> > somehow bridge that gap. I've no idea if/how it does that.
>
> In general, this is done by timekeeping_resume() and the only special
> thing done for the TSC appears to be the tsc_verify_tsc_adjust(true)
> call in tsc_resume().

And I forgot about tsc_restore_sched_clock_state() that gets called
via restore_processor_state() on x86, before calling
timekeeping_resume().

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [ovmf test] 146051: regressions - trouble: broken/fail/pass

2020-01-13 Thread osstest service owner
flight 146051 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146051/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-ovmf-amd64 broken
 test-amd64-i386-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail REGR. vs. 145767
 test-amd64-amd64-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail in 146047 
REGR. vs. 145767

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-ovmf-amd64  4 host-install(4)  broken pass in 146047

version targeted for testing:
 ovmf 4465cd124fbcf5490faad6a1a834299b30b5d009
baseline version:
 ovmf 70911f1f4aee0366b6122f2b90d367ec0f066beb

Last test of basis   145767  2020-01-08 00:39:09 Z5 days
Failing since145774  2020-01-08 02:50:20 Z5 days   32 attempts
Testing same since   146041  2020-01-13 04:39:23 Z0 days4 attempts


People who touched revisions under test:
  Albecki, Mateusz 
  Ard Biesheuvel 
  Ashish Singhal 
  Eric Dong 
  Fan, ZhijuX 
  Jason Voelz 
  Laszlo Ersek 
  Mateusz Albecki 
  Pavana.K 
  Philippe Mathieu-Daud? 
  Philippe Mathieu-Daude 
  Siyuan Fu 
  Siyuan, Fu 
  Vitaly Cheptsov 
  Vitaly Cheptsov via Groups.Io 
  Zhiju.Fan 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 broken  
 test-amd64-i386-xl-qemuu-ovmf-amd64  fail



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job test-amd64-amd64-xl-qemuu-ovmf-amd64 broken
broken-step test-amd64-amd64-xl-qemuu-ovmf-amd64 host-install(4)

Not pushing.

(No revision log; it would be 422 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [qemu-mainline test] 146054: regressions - FAIL

2020-01-13 Thread osstest service owner
flight 146054 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146054/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm   6 xen-buildfail REGR. vs. 144861
 build-arm64   6 xen-buildfail REGR. vs. 144861
 build-amd64   6 xen-buildfail REGR. vs. 144861
 build-i386-xsm6 xen-buildfail REGR. vs. 144861
 build-armhf   6 xen-buildfail REGR. vs. 144861
 build-amd64-xsm   6 xen-buildfail REGR. vs. 144861
 build-i3866 xen-buildfail REGR. vs. 144861

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-armhf-armhf-xl-cubietruck  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-shadow1 build-check(1)   blocked  n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-shadow 1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-rtds  1 build-check(1)   blocked  n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-pvshim 1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-rtds  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvshim1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow  1 

[Xen-devel] [PATCH] docs/misc: livepatch: Espace backslash

2020-01-13 Thread Julien Grall
pandoc is currently failing to generate the pdf with the following
error:
! Undefined control sequence.
l.1048   metadata string format is: key=value\0

In this case, we want to print \0 so we need to backslash-escape the
first character.

Interestingly pandoc will not complain when creating html and will just
ignore \0 completely.

Fixes: 5083e0ff93 ("livepatch: Add metadata runtime retrieval mechanism")
Signed-off-by: Julien Grall 
Cc: Pawel Wieczorkiewicz 
---
 docs/misc/livepatch.pandoc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/misc/livepatch.pandoc b/docs/misc/livepatch.pandoc
index 2f3f95ed37..9473ad5991 100644
--- a/docs/misc/livepatch.pandoc
+++ b/docs/misc/livepatch.pandoc
@@ -739,7 +739,7 @@ The caller provides:
Caller *MUST* allocate enough space to be able to store all received data
(i.e. total allocated space *MUST* match the `metadata_total_size` value
provided by the hypervisor). Individual payload metadata string can be of
-   arbitrary length. The metadata string format is: key=value\0...key=value\0.
+   arbitrary length. The metadata string format is: 
key=value\\0...key=value\\0.
  * `metadata_len` - Virtual address of where to write the length of each 
metadata
string of the payload. Caller *MUST* allocate up to `nr` of them. Each 
*MUST*
be of sizeof(uint32_t) (4 bytes).
-- 
2.17.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] docs/misc: pvcalls: Verbatim block should be indented with 4 spaces

2020-01-13 Thread Andrew Cooper
On 13/01/2020 21:57, Julien Grall wrote:
> At the moment, the diagram is only indented with 2 spaces. So pandoc
> will try to badly interpret it and not display it correctly.
>
> Fix it by indenting all the block by 4 spaces (i.e an extra 2 spaces).
>
> Fixes: d661611d08 ("docs/markdown: Switch to using pandoc, and fix underscore 
> escaping")
> Signed-off-by: Julien Grall 

Acked-by: Andrew Cooper 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH] docs/misc: pvcalls: Verbatim block should be indented with 4 spaces

2020-01-13 Thread Julien Grall
At the moment, the diagram is only indented with 2 spaces. So pandoc
will try to badly interpret it and not display it correctly.

Fix it by indenting all the block by 4 spaces (i.e an extra 2 spaces).

Fixes: d661611d08 ("docs/markdown: Switch to using pandoc, and fix underscore 
escaping")
Signed-off-by: Julien Grall 
---
 docs/misc/pvcalls.pandoc | 36 ++--
 1 file changed, 18 insertions(+), 18 deletions(-)

diff --git a/docs/misc/pvcalls.pandoc b/docs/misc/pvcalls.pandoc
index 0c48b29842..729cf97bdf 100644
--- a/docs/misc/pvcalls.pandoc
+++ b/docs/misc/pvcalls.pandoc
@@ -867,24 +867,24 @@ and the second half to the **out** array. They are used 
as circular
 buffers for transferring data, and, together, they are the data ring.
 
 
-  +---+ Indexes page
-  | Command ring: | +--+
-  | @0: xen_pvcalls_connect:  | |@0 pvcalls_data_intf: |
-  | @44: ref  +>+@76: ring_order = 1   |
-  |   | |@80: ref[0]+  |
-  +---+ |@84: ref[1]+  |
-|   |  |
-|   |  |
-+--+
-|
-v (data ring)
-+---+---+
-|  @0->4098: in |
-|  ref[0]   |
-|---|
-|  @4099->8196: out |
-|  ref[1]   |
-+---+
++---+ Indexes page
+| Command ring: | +--+
+| @0: xen_pvcalls_connect:  | |@0 pvcalls_data_intf: |
+| @44: ref  +>+@76: ring_order = 1   |
+|   | |@80: ref[0]+  |
++---+ |@84: ref[1]+  |
+  |   |  |
+  |   |  |
+  +--+
+  |
+  v (data ring)
+  +---+---+
+  |  @0->4098: in |
+  |  ref[0]   |
+  |---|
+  |  @4099->8196: out |
+  |  ref[1]   |
+  +---+
 
 
  Indexes Page Structure
-- 
2.17.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC PATCH V2 11/11] x86: tsc: avoid system instability in hibernation

2020-01-13 Thread Rafael J. Wysocki
On Mon, Jan 13, 2020 at 1:43 PM Peter Zijlstra  wrote:
>
> On Mon, Jan 13, 2020 at 11:43:18AM +, Singh, Balbir wrote:
> > For your original comment, just wanted to clarify the following:
> >
> > 1. After hibernation, the machine can be resumed on a different but 
> > compatible
> > host (these are VM images hibernated)
> > 2. This means the clock between host1 and host2 can/will be different
> >
> > In your comments are you making the assumption that the host(s) is/are the
> > same? Just checking the assumptions being made and being on the same page 
> > with
> > them.
>
> I would expect this to be the same problem we have as regular suspend,
> after power off the TSC will have been reset, so resume will have to
> somehow bridge that gap. I've no idea if/how it does that.

In general, this is done by timekeeping_resume() and the only special
thing done for the TSC appears to be the tsc_verify_tsc_adjust(true)
call in tsc_resume().

> I remember some BIOSes had crazy TSC ideas for suspend2ram, and we grew
> tsc_restore_sched_clock_state() for it.
>
> Playing crazy games like what you're doing just isn't it though.

Right.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3] xen-pciback: optionally allow interrupt enable flag writes

2020-01-13 Thread Marek Marczykowski-Górecki
On Mon, Jan 13, 2020 at 04:25:02PM -0500, Boris Ostrovsky wrote:
> 
> 
> On 1/10/20 10:43 PM, Marek Marczykowski-Górecki wrote:
> > @@ -117,6 +117,24 @@ static int command_write(struct pci_dev *dev, int 
> > offset, u16 value, void *data)
> > pci_clear_mwi(dev);
> > }
> > +   if (dev_data && dev_data->allow_interrupt_control) {
> > +   if ((cmd->val ^ val) & PCI_COMMAND_INTX_DISABLE) {
> > +   if (value & PCI_COMMAND_INTX_DISABLE) {
> > +   pci_intx(dev, 0);
> > +   } else {
> > +   /* Do not allow enabling INTx together with MSI 
> > or MSI-X. */
> > +   switch (xen_pcibk_get_interrupt_type(dev)) {
> > +   case INTERRUPT_TYPE_NONE:
> > +   case INTERRUPT_TYPE_INTX:
> > +   pci_intx(dev, 1);
> 
> If INTERRUPT_TYPE_INTX , why call pci_intx(1)?

Not needed indeed.

> 
> (I think I asked this last time as well).
> 
> 
> -boris
> 
> 

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?


signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 0/4] xen/x86: Rework inclusion between struct pirq and

2020-01-13 Thread Julien Grall
Hi all,

The main goal of this series is to make easier to understand and use
struct pirq. Patch #1 and #3 are cleanups.

Cheers,

Julien Grall (4):
  xen/x86: Remove unused forward declaration in asm-x86/irq.h
  xen/char: ehci: Directly include xen/timer.h rather rely on dependency
  xen/domain: Remove #ifndef surrounding alloc_pirq_struct()
  xen/x86: Rework inclusion between struct pirq and struct hvm_pirq_dpci

 xen/arch/arm/irq.c|  5 +
 xen/arch/x86/hvm/irq.c|  7 ---
 xen/arch/x86/irq.c| 39 ---
 xen/common/domain.c   |  7 +--
 xen/drivers/char/ehci-dbgp.c  |  1 +
 xen/drivers/passthrough/io.c  |  1 +
 xen/include/asm-x86/hvm/irq.h | 19 +
 xen/include/asm-x86/irq.h | 20 +++---
 xen/include/xen/domain.h  |  5 +++--
 9 files changed, 64 insertions(+), 40 deletions(-)

-- 
2.24.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 2/4] xen/char: ehci: Directly include xen/timer.h rather rely on dependency

2020-01-13 Thread Julien Grall
From: Julien Grall 

The ehci char driver is using timers but relying on the header
xen/timer.h to be included via asm-x86/hvm/irq.h which is not even
directly included!

Future rework will reduce the number of places where asm-x86/hvm/irq.h
will be included. Include xen/timer.h directly to avoid any breakage.

Signed-off-by: Julien Grall 
---
 xen/drivers/char/ehci-dbgp.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/xen/drivers/char/ehci-dbgp.c b/xen/drivers/char/ehci-dbgp.c
index b6e155d17b..8124e0aad8 100644
--- a/xen/drivers/char/ehci-dbgp.c
+++ b/xen/drivers/char/ehci-dbgp.c
@@ -10,6 +10,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
-- 
2.24.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 4/4] xen/x86: Rework inclusion between struct pirq and struct hvm_pirq_dpci

2020-01-13 Thread Julien Grall
From: Julien Grall 

At the moment, alloc_pirq_struct() relies on the field 'arch' to be the
last member of the structure.

As this is used for computing the size of the structure, the value will
be miscomputed if a new field is added afterwards.

Such quirkiness makes quite difficult to understand how struct pirq
works. Given that struct hvm_pirq_dpci is only used in combination of a
struct pirq, we can inverse the inclusion. i.e pirq will now be
contained in struct hvm_pirq_dpci.

As the field pirq.arch.hvm.emuirq is as well HVM specific, this is now
moved in struct hvm_pirq_dpci.

There is a few side effects with this changes:
- We now need to distinguish between PIRQ allocated for HVM and PV
  guests. This is to allow us to know what we are freeing.
- container_of is not able to cater with const and non-const at the
  same time. So we need to introduce two macros (const and
  non-const).

Lastly all the HVM specific pirq code can now be moved in hvm/irq.h
allowing use to drop the include from irq.h. This is one less header
included treewide.

Signed-off-by: Julien Grall 
---
 xen/arch/arm/irq.c|  5 +
 xen/arch/x86/hvm/irq.c|  7 ---
 xen/arch/x86/irq.c| 39 ---
 xen/common/domain.c   |  7 +--
 xen/drivers/passthrough/io.c  |  1 +
 xen/include/asm-x86/hvm/irq.h | 19 +
 xen/include/asm-x86/irq.h | 19 +++--
 xen/include/xen/domain.h  |  3 +++
 8 files changed, 63 insertions(+), 37 deletions(-)

diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 3877657a52..fd108ea3a5 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -582,6 +582,11 @@ struct pirq *alloc_pirq_struct(struct domain *d)
 return NULL;
 }
 
+void arch_free_pirq_struct(struct rcu_head *head)
+{
+ASSERT_UNREACHABLE();
+}
+
 /*
  * These are all unreachable given an alloc_pirq_struct
  * which returns NULL, all callers try to lookup struct pirq first
diff --git a/xen/arch/x86/hvm/irq.c b/xen/arch/x86/hvm/irq.c
index c684422b24..e0bb0a8b90 100644
--- a/xen/arch/x86/hvm/irq.c
+++ b/xen/arch/x86/hvm/irq.c
@@ -29,7 +29,8 @@
 
 bool hvm_domain_use_pirq(const struct domain *d, const struct pirq *pirq)
 {
-return is_hvm_domain(d) && pirq && pirq->arch.hvm.emuirq != IRQ_UNBOUND;
+return is_hvm_domain(d) && pirq &&
+const_pirq_dpci(pirq)->emuirq != IRQ_UNBOUND;
 }
 
 /* Must be called with hvm_domain->irq_lock hold */
@@ -396,7 +397,7 @@ int hvm_inject_msi(struct domain *d, uint64_t addr, 
uint32_t data)
 struct pirq *info = pirq_info(d, pirq);
 
 /* if it is the first time, allocate the pirq */
-if ( !info || info->arch.hvm.emuirq == IRQ_UNBOUND )
+if ( !info || pirq_dpci(info)->emuirq == IRQ_UNBOUND )
 {
 int rc;
 
@@ -409,7 +410,7 @@ int hvm_inject_msi(struct domain *d, uint64_t addr, 
uint32_t data)
 if ( !info )
 return -EBUSY;
 }
-else if ( info->arch.hvm.emuirq != IRQ_MSI_EMU )
+else if ( pirq_dpci(info)->emuirq != IRQ_MSI_EMU )
 return -EINVAL;
 send_guest_pirq(d, info);
 return 0;
diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index 310ac00a60..3e01101f88 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -1286,22 +1286,37 @@ void cleanup_domain_irq_mapping(struct domain *d)
 
 struct pirq *alloc_pirq_struct(struct domain *d)
 {
-size_t sz = is_hvm_domain(d) ? sizeof(struct pirq) :
-   offsetof(struct pirq, arch.hvm);
-struct pirq *pirq = xzalloc_bytes(sz);
+struct pirq *pirq;
 
-if ( pirq )
+if ( is_hvm_domain(d) )
 {
-if ( is_hvm_domain(d) )
+struct hvm_pirq_dpci *dpci = xzalloc(struct hvm_pirq_dpci);
+
+if ( dpci )
 {
-pirq->arch.hvm.emuirq = IRQ_UNBOUND;
-pt_pirq_init(d, >arch.hvm.dpci);
+pt_pirq_init(d, dpci);
+pirq = dpci_pirq(dpci);
+pirq->arch.hvm = true;
 }
+else
+pirq = NULL;
 }
+else
+pirq = xzalloc(struct pirq);
 
 return pirq;
 }
 
+void arch_free_pirq_struct(struct rcu_head *head)
+{
+struct pirq *pirq = container_of(head, struct pirq, rcu_head);
+
+if ( pirq->arch.hvm )
+xfree(pirq_dpci(pirq));
+else
+xfree(pirq);
+}
+
 void (pirq_cleanup_check)(struct pirq *pirq, struct domain *d)
 {
 /*
@@ -1315,9 +1330,9 @@ void (pirq_cleanup_check)(struct pirq *pirq, struct 
domain *d)
 
 if ( is_hvm_domain(d) )
 {
-if ( pirq->arch.hvm.emuirq != IRQ_UNBOUND )
+if ( pirq_dpci(pirq)->emuirq != IRQ_UNBOUND )
 return;
-if ( !pt_pirq_cleanup_check(>arch.hvm.dpci) )
+if ( !pt_pirq_cleanup_check(pirq_dpci(pirq)) )
 return;
 }
 
@@ -2029,7 +2044,7 @@ static inline bool is_free_pirq(const 

[Xen-devel] [PATCH 1/4] xen/x86: Remove unused forward declaration in asm-x86/irq.h

2020-01-13 Thread Julien Grall
From: Julien Grall 

None of the prototypes within the header asm-x86/irq.h actually requires
the forward declaration of "struct pirq". So remove it.

No functional changes intended.

Signed-off-by: Julien Grall 
---
 xen/include/asm-x86/irq.h | 1 -
 1 file changed, 1 deletion(-)

diff --git a/xen/include/asm-x86/irq.h b/xen/include/asm-x86/irq.h
index 7c825e9d9c..44aefc8f03 100644
--- a/xen/include/asm-x86/irq.h
+++ b/xen/include/asm-x86/irq.h
@@ -131,7 +131,6 @@ extern unsigned int io_apic_irqs;
 
 DECLARE_PER_CPU(unsigned int, irq_count);
 
-struct pirq;
 struct arch_pirq {
 int irq;
 union {
-- 
2.24.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 3/4] xen/domain: Remove #ifndef surrounding alloc_pirq_struct()

2020-01-13 Thread Julien Grall
From: Julien Grall 

None of the supported architecture override alloc_pirq_struct() with
a macro. So remove the #ifdef surrounding the prototype.

Signed-off-by: Julien Grall 
---
 xen/include/xen/domain.h | 2 --
 1 file changed, 2 deletions(-)

diff --git a/xen/include/xen/domain.h b/xen/include/xen/domain.h
index 1cb205d977..89bf0a1721 100644
--- a/xen/include/xen/domain.h
+++ b/xen/include/xen/domain.h
@@ -41,9 +41,7 @@ struct vcpu *alloc_vcpu_struct(const struct domain *d);
 void free_vcpu_struct(struct vcpu *v);
 
 /* Allocate/free a PIRQ structure. */
-#ifndef alloc_pirq_struct
 struct pirq *alloc_pirq_struct(struct domain *);
-#endif
 void free_pirq_struct(void *);
 
 /*
-- 
2.24.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3] xen-pciback: optionally allow interrupt enable flag writes

2020-01-13 Thread Boris Ostrovsky



On 1/10/20 10:43 PM, Marek Marczykowski-Górecki wrote:

@@ -117,6 +117,24 @@ static int command_write(struct pci_dev *dev, int offset, 
u16 value, void *data)
pci_clear_mwi(dev);
}
  
+	if (dev_data && dev_data->allow_interrupt_control) {

+   if ((cmd->val ^ val) & PCI_COMMAND_INTX_DISABLE) {
+   if (value & PCI_COMMAND_INTX_DISABLE) {
+   pci_intx(dev, 0);
+   } else {
+   /* Do not allow enabling INTx together with MSI 
or MSI-X. */
+   switch (xen_pcibk_get_interrupt_type(dev)) {
+   case INTERRUPT_TYPE_NONE:
+   case INTERRUPT_TYPE_INTX:
+   pci_intx(dev, 1);


If INTERRUPT_TYPE_INTX , why call pci_intx(1)?

(I think I asked this last time as well).


-boris



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 4/6] libxl: allow creation of domains with a specified or random domid

2020-01-13 Thread Julien Grall

Hi Paul,

On 13/01/2020 16:54, Durrant, Paul wrote:

-Original Message-
From: jandr...@gmail.com 
Sent: 13 January 2020 16:16
To: Durrant, Paul 
Cc: xen-devel ; Anthony PERARD
; Ian Jackson ; Wei
Liu 
Subject: Re: [Xen-devel] [PATCH v2 4/6] libxl: allow creation of domains
with a specified or random domid

On Thu, Jan 9, 2020 at 6:50 AM Paul Durrant  wrote:


This patch adds a 'domid' field to libxl_domain_create_info and then
modifies do_domain_create() to use that value if it is valid. Any valid
domid will be checked against the retired domid list before being passed
to libxl__domain_make().
If the domid value is invalid then Xen will choose the domid, as before,
unless the value is the new special RANDOM_DOMID value added to the API.
This value instructs libxl__domain_make() to select a random domid

value,

check it for validity, verify it does not match a retired domain, and

then

pass it to Xen's XEN_DOMCTL_createdomain operation. If Xen determines

that

it co-incides with an existing domain, a new random value will be
selected and the operation will be re-tried.

NOTE: libxl__logv() is also modified to only log valid domid values in
   messages rather than any domid, valid or otherwise, that is not
   INVALID_DOMID.

Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 
Cc: Anthony PERARD 

v2:
  - Re-worked to use a value from libxl_domain_create_info
---
  tools/libxl/libxl.h  |  9 +
  tools/libxl/libxl_create.c   | 32 +++-
  tools/libxl/libxl_internal.c |  2 +-
  tools/libxl/libxl_types.idl  |  1 +
  4 files changed, 42 insertions(+), 2 deletions(-)






diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index 1835a5502c..ee76dee364 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -600,9 +600,39 @@ int libxl__domain_make(libxl__gc *gc,

libxl_domain_config *d_config,

  goto out;
  }

-ret = xc_domain_create(ctx->xch, domid, );
+if (libxl_domid_valid_guest(info->domid)) {
+*domid = info->domid;
+
+if (libxl__is_retired_domid(gc, *domid)) {
+LOGED(ERROR, *domid, "domain id is retired");
+rc = ERROR_FAIL;
+goto out;
+}
+} else if (info->domid == RANDOM_DOMID) {
+*domid = 0; /* Zero-out initial value */
+}
+
+for (;;) {
+if (info->domid == RANDOM_DOMID) {
+/* Randomize lower order bytes */
+ret = libxl__random_bytes(gc, (void *)domid,
+  sizeof(uint16_t));


Casting to void * assumes little endian.


I think that's a fairly safe assumption as far as Xen goes...


Not really, there are technically nothing (other than bug fixes) 
preventing us to use a big endian guest on Xen on Arm.


I actually did play with big endian on Xen in the past and managed to 
get a guest running. The main annoying part is Linux as it is assuming 
to use the same endian as the hypervisor. But other OS may not have this 
issues...


The hypervisor itself is likely going to stay little endian, so does the 
interface. For the tools, we should aim to not introduce more assumption 
that the software will be little endian.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v4 6/7] Add guide on Communication Best Practice

2020-01-13 Thread Lars Kurth


On 13/01/2020, 19:54, "George Dunlap"  wrote:


> On Dec 30, 2019, at 7:32 PM, Lars Kurth  wrote:
> 
> From: Lars Kurth 
> 
> This guide covers the bulk on Best Practice related to code review
> It primarily focusses on code review interactions
> It also covers how to deal with Misunderstandings and Cultural
> Differences
> 
> +### Avoid opinion: stick to the facts

In my talk on this subject I said “Avoid *inflammatory language*”.  At some 
level it’s good to have strong opinions on what code should look like.  It’s 
not opinions that are a problem, or even expressing opinions, but expressing 
them in a provocative or inflammatory way.

Let me look at this again: I don't feel strongly about it

I changed the title because I felt that the bulk of the 
example is actually about sticking to the facts an opinion 
and the inflammatory element was secondary. So it felt more
natural to me to change the title.

But then looking at the definition of inflammatory language,
aka  "an inflammatory question or an inflammatory statement
would be one which would somehow predispose the listeners
towards a subject in an unreasonable, prejudiced way."
It is clearly also true that the example is inflammatory.

I think I may have tripped over an area where there is no good
language match: the German translations of inflammatory
aufrührerisch & aufwieglerisch have an element of rebellion
and mischief to them (at least when I grew up).

I am wondering though, whether it is necessary to include 
a definition of an inflammatory question or an inflammatory
statement if we stick with it in the title

> 
> +> Foot binding was the custom of applying tight binding to the feet of 
young
> +> girls to modify the shape and size of their feet. ... foot binding was 
a
> +> painful practice and significantly limited the mobility of women, 
resulting
> +> in lifelong disabilities for most of its subjects. ... Binding usually
> +> started during the winter months since the feet were more likely to be 
numb,
> +> and therefore the pain would not be as extreme. …The toes on each foot
> +> were curled under, then pressed with great force downwards and squeezed
> +> into the sole of the foot until the toes broke…

In my talk I covered the last three words behind a blue square, since this 
image is pretty violent — and is gendered violence at that.  Some people joke 
about “triggering”, but there are certainly people who  have experienced 
violence, who when they come across descriptions of it unexpectedly suddenly 
have loads of unwelcome emotions to deal with; and I venture to guess that most 
people skimming through such a guide wouldn’t be expecting to come across 
something like this.

Personally I would replace the last three words with [redacted].  The point 
can be made without being so explicit.  Anyone who wants to know what happens 
can go look up the entry themselves.

OK. I can do that. 
I copied the text from the content outline on slide share and wasn't even 
looking at the slides themselves

Lars



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [qemu-mainline test] 146053: regressions - FAIL

2020-01-13 Thread osstest service owner
flight 146053 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146053/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm   6 xen-buildfail REGR. vs. 144861
 build-arm64   6 xen-buildfail REGR. vs. 144861
 build-amd64   6 xen-buildfail REGR. vs. 144861
 build-i386-xsm6 xen-buildfail REGR. vs. 144861
 build-armhf   6 xen-buildfail REGR. vs. 144861
 build-amd64-xsm   6 xen-buildfail REGR. vs. 144861
 build-i3866 xen-buildfail REGR. vs. 144861

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-seattle   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-shadow 1 build-check(1)   blocked  n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-pvshim 1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-shadow1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)   blocked  n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvshim1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)  blocked n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-rtds  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-armhf-armhf-xl-cubietruck  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 

Re: [Xen-devel] [PATCH v4 6/7] Add guide on Communication Best Practice

2020-01-13 Thread George Dunlap

> On Dec 30, 2019, at 7:32 PM, Lars Kurth  wrote:
> 
> From: Lars Kurth 
> 
> This guide covers the bulk on Best Practice related to code review
> It primarily focusses on code review interactions
> It also covers how to deal with Misunderstandings and Cultural
> Differences
> 
> +### Avoid opinion: stick to the facts

In my talk on this subject I said “Avoid *inflammatory language*”.  At some 
level it’s good to have strong opinions on what code should look like.  It’s 
not opinions that are a problem, or even expressing opinions, but expressing 
them in a provocative or inflammatory way.

> 
> +> Foot binding was the custom of applying tight binding to the feet of young
> +> girls to modify the shape and size of their feet. ... foot binding was a
> +> painful practice and significantly limited the mobility of women, resulting
> +> in lifelong disabilities for most of its subjects. ... Binding usually
> +> started during the winter months since the feet were more likely to be 
> numb,
> +> and therefore the pain would not be as extreme. …The toes on each foot
> +> were curled under, then pressed with great force downwards and squeezed
> +> into the sole of the foot until the toes broke…

In my talk I covered the last three words behind a blue square, since this 
image is pretty violent — and is gendered violence at that.  Some people joke 
about “triggering”, but there are certainly people who  have experienced 
violence, who when they come across descriptions of it unexpectedly suddenly 
have loads of unwelcome emotions to deal with; and I venture to guess that most 
people skimming through such a guide wouldn’t be expecting to come across 
something like this.

Personally I would replace the last three words with [redacted].  The point can 
be made without being so explicit.  Anyone who wants to know what happens can 
go look up the entry themselves.

Everything else looks good!

 -George

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Ping: [PATCH] x86/HVM: use single (atomic) MOV for aligned emulated writes

2020-01-13 Thread Jason Andryuk
On Fri, Dec 27, 2019 at 11:09 AM Andrew Cooper
 wrote:
>
> On 20/12/2019 16:23, Jan Beulich wrote:
> > On 16.09.2019 11:40, Jan Beulich wrote:
> >> Using memcpy() may result in multiple individual byte accesses
> >> (dependening how memcpy() is implemented and how the resulting insns,
> >> e.g. REP MOVSB, get carried out in hardware), which isn't what we
> >> want/need for carrying out guest insns as correctly as possible. Fall
> >> back to memcpy() only for accesses not 2, 4, or 8 bytes in size.
> >>
> >> Suggested-by: Andrew Cooper 
> >> Signed-off-by: Jan Beulich 
>
> Acked-by: Andrew Cooper 

Should xen/arch/x86/mm/shadow/hvm.c:hvm_emulate_write() be similarly changed?

Thanks,
Jason

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 6/6] xen-pt: Round pci regions sizes to XEN_PAGE_SIZE

2020-01-13 Thread Jason Andryuk
On Fri, Mar 22, 2019 at 3:43 PM Jason Andryuk  wrote:
>
> On Thu, Mar 21, 2019 at 11:09 PM Roger Pau Monné  wrote:
> >
> > On Wed, Mar 20, 2019 at 01:28:47PM -0400, Jason Andryuk wrote:
> > > On Fri, Mar 15, 2019 at 12:28 PM Andrew Cooper
> > >  wrote:
> > > >
> > > > On 15/03/2019 09:17, Paul Durrant wrote:
> > > > >> -Original Message-
> > > > >> From: Jason Andryuk [mailto:jandr...@gmail.com]
> > > > >> Sent: 14 March 2019 18:16
> > > > >> To: Paul Durrant 
> > > > >> Cc: qemu-de...@nongnu.org; xen-devel@lists.xenproject.org; 
> > > > >> marma...@invisiblethingslab.com; Simon
> > > > >> Gaiser ; Stefano Stabellini 
> > > > >> ; Anthony Perard
> > > > >> 
> > > > >> Subject: Re: [PATCH 6/6] xen-pt: Round pci regions sizes to 
> > > > >> XEN_PAGE_SIZE
> > > > >>
> > > > >> On Wed, Mar 13, 2019 at 11:09 AM Paul Durrant 
> > > > >>  wrote:
> > > >  -Original Message-
> > > >  From: Jason Andryuk [mailto:jandr...@gmail.com]
> > > >  Sent: 11 March 2019 18:02
> > > >  To: qemu-de...@nongnu.org
> > > >  Cc: xen-devel@lists.xenproject.org; 
> > > >  marma...@invisiblethingslab.com; Simon Gaiser
> > > >  ; Jason Andryuk 
> > > >  ; Stefano Stabellini
> > > >  ; Anthony Perard 
> > > >  ; Paul Durrant
> > > >  
> > > >  Subject: [PATCH 6/6] xen-pt: Round pci regions sizes to 
> > > >  XEN_PAGE_SIZE
> > > > 
> > > >  From: Simon Gaiser 
> > > > 
> > > >  If a pci memory region has a size < XEN_PAGE_SIZE it can get 
> > > >  located at
> > > >  an address which is not page aligned.
> > > > >>> IIRC the PCI spec says that the minimum memory region size should 
> > > > >>> be at least 4k. Should we even be
> > > > >> tolerating BARs smaller than that?
> > > > >>>   Paul
> > > > >>>
> > > > >> Hi, Paul.
> > > > >>
> > > > >> Simon found this, so it affects a real device.  Simon, do you recall
> > > > >> which device was affected?
> > > > >>
> > > > >> I think BARs only need to be power-of-two size and aligned, and 4k is
> > > > >> not a minimum.  16bytes may be a minimum, but I don't know what the
> > > > >> spec says.
> > > > >>
> > > > >> On an Ivy Bridge system, here are some of the devices with BARs 
> > > > >> smaller than 4K:
> > > > >> 00:16.0 Communication controller: Intel Corporation 7 Series/C210
> > > > >> Series Chipset Family MEI Controller #1 (rev 04)
> > > > >>Memory at d0735000 (64-bit, non-prefetchable) [disabled] [size=16]
> > > > >> 00:1d.0 USB controller: Intel Corporation 7 Series/C210 Series 
> > > > >> Chipset
> > > > >> Family USB Enhanced Host Controller #1 (rev 04) (prog-if 20 [EHCI])
> > > > >>Memory at d0739000 (32-bit, non-prefetchable) [disabled] [size=1K]
> > > > >> 00:1f.3 SMBus: Intel Corporation 7 Series/C210 Series Chipset Family
> > > > >> SMBus Controller (rev 04)
> > > > >>Memory at d0734000 (64-bit, non-prefetchable) [disabled] 
> > > > >> [size=256]
> > > > >> 02:00.0 System peripheral: JMicron Technology Corp. SD/MMC Host
> > > > >> Controller (rev 30)
> > > > >>Memory at d0503000 (32-bit, non-prefetchable) [disabled] 
> > > > >> [size=256]
> > > > >>
> > > > >> These examples are all 4K aligned, so this is not an issue on this 
> > > > >> machine.
> > > > >>
> > > > >> Reviewing the code, I'm now wondering if the following in
> > > > >> hw/xen/xen_pt.c:xen_pt_region_update is wrong:rc =
> > > > >> xc_domain_memory_mapping(xen_xc, xen_domid,
> > > > >>  XEN_PFN(guest_addr + 
> > > > >> XC_PAGE_SIZE - 1),
> > > > >>  XEN_PFN(machine_addr + 
> > > > >> XC_PAGE_SIZE - 1),
> > > > >>  XEN_PFN(size + XC_PAGE_SIZE - 
> > > > >> 1),
> > > > >>  op);
> > > > >>
> > > > >> If a bar of size 0x100 is at 0xd0500800, then the machine_addr passed
> > > > >> in would be 0xd0501000 which is past the actual location.  Should the
> > > > >> call arguments just be XEN_PFN(guest_addr) & XEN_PFN(machine_addr)?
> > > > >>
> > > > >> BARs smaller than a page would also be a problem if BARs for 
> > > > >> different
> > > > >> devices shared the same page.
> > > > > Exactly. We cannot pass them through with any degree of safety (not 
> > > > > that passthrough of an arbitrary device is a particularly safe thing 
> > > > > to do anyway). The xen-pt code would instead need to trap those BARs 
> > > > > and perform the accesses to the real BAR itself. Ultimately though I 
> > > > > think we should be retiring the xen-pt code in favour of a standalone 
> > > > > emulator.
> > > >
> > > > It doesn't matter if the BAR is smaller than 4k, if there are holes next
> > > > to it.
> > > >
> > > > Do we know what the case is in practice for these USB controllers?
> > > >
> > > > If the worst comes to the worst, we can re-enumerate the PCI bus to
> > > > ensure that all bars smaller than 4k still have 4k alignment between
> > > > them.  That way we can 

[Xen-devel] [qemu-mainline test] 146052: regressions - FAIL

2020-01-13 Thread osstest service owner
flight 146052 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146052/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm   6 xen-buildfail REGR. vs. 144861
 build-arm64   6 xen-buildfail REGR. vs. 144861
 build-amd64   6 xen-buildfail REGR. vs. 144861
 build-i386-xsm6 xen-buildfail REGR. vs. 144861
 build-armhf   6 xen-buildfail REGR. vs. 144861
 build-amd64-xsm   6 xen-buildfail REGR. vs. 144861
 build-i3866 xen-buildfail REGR. vs. 144861

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1)  blocked n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-arm64-arm64-xl-seattle   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)  blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-pvshim 1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-shadow1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-rtds  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-rtds  1 build-check(1)   blocked  n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-cubietruck  1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 

Re: [Xen-devel] [PATCH v2 4/6] libxl: allow creation of domains with a specified or random domid

2020-01-13 Thread Jason Andryuk
On Mon, Jan 13, 2020 at 11:55 AM Durrant, Paul  wrote:
>
> > -Original Message-
> > From: jandr...@gmail.com 
> > Sent: 13 January 2020 16:16
> > To: Durrant, Paul 
> > Cc: xen-devel ; Anthony PERARD
> > ; Ian Jackson ; Wei
> > Liu 
> > Subject: Re: [Xen-devel] [PATCH v2 4/6] libxl: allow creation of domains
> > with a specified or random domid
> >
> > On Thu, Jan 9, 2020 at 6:50 AM Paul Durrant  wrote:
> > >
> > > This patch adds a 'domid' field to libxl_domain_create_info and then
> > > modifies do_domain_create() to use that value if it is valid. Any valid
> > > domid will be checked against the retired domid list before being passed
> > > to libxl__domain_make().
> > > If the domid value is invalid then Xen will choose the domid, as before,
> > > unless the value is the new special RANDOM_DOMID value added to the API.
> > > This value instructs libxl__domain_make() to select a random domid
> > value,
> > > check it for validity, verify it does not match a retired domain, and
> > then
> > > pass it to Xen's XEN_DOMCTL_createdomain operation. If Xen determines
> > that
> > > it co-incides with an existing domain, a new random value will be
> > > selected and the operation will be re-tried.
> > >
> > > NOTE: libxl__logv() is also modified to only log valid domid values in
> > >   messages rather than any domid, valid or otherwise, that is not
> > >   INVALID_DOMID.
> > >
> > > Signed-off-by: Paul Durrant 
> > > ---
> > > Cc: Ian Jackson 
> > > Cc: Wei Liu 
> > > Cc: Anthony PERARD 
> > >
> > > v2:
> > >  - Re-worked to use a value from libxl_domain_create_info
> > > ---
> > >  tools/libxl/libxl.h  |  9 +
> > >  tools/libxl/libxl_create.c   | 32 +++-
> > >  tools/libxl/libxl_internal.c |  2 +-
> > >  tools/libxl/libxl_types.idl  |  1 +
> > >  4 files changed, 42 insertions(+), 2 deletions(-)
> > >
> >
> > 
> >
> > > diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
> > > index 1835a5502c..ee76dee364 100644
> > > --- a/tools/libxl/libxl_create.c
> > > +++ b/tools/libxl/libxl_create.c
> > > @@ -600,9 +600,39 @@ int libxl__domain_make(libxl__gc *gc,
> > libxl_domain_config *d_config,
> > >  goto out;
> > >  }
> > >
> > > -ret = xc_domain_create(ctx->xch, domid, );
> > > +if (libxl_domid_valid_guest(info->domid)) {
> > > +*domid = info->domid;
> > > +
> > > +if (libxl__is_retired_domid(gc, *domid)) {
> > > +LOGED(ERROR, *domid, "domain id is retired");
> > > +rc = ERROR_FAIL;
> > > +goto out;
> > > +}
> > > +} else if (info->domid == RANDOM_DOMID) {
> > > +*domid = 0; /* Zero-out initial value */
> > > +}
> > > +
> > > +for (;;) {
> > > +if (info->domid == RANDOM_DOMID) {
> > > +/* Randomize lower order bytes */
> > > +ret = libxl__random_bytes(gc, (void *)domid,
> > > +  sizeof(uint16_t));
> >
> > Casting to void * assumes little endian.
>
> I think that's a fairly safe assumption as far as Xen goes...
>
> > Using a temporary uint16_t
>
> ...but, yes, that might be neater.
>
> > would avoid that assumption.  Also, masking down to 0x7fff would clear
> > the top bit which is never valid.
>
> That seems like a bit of a layering violation and the check in 
> libxl_domid_valid_guest() is going to cause a pretty fast turn round the loop 
> if the top bit is set so masking is not going to gain that much.

Yeah, there isn't a define or constant exposed for 0x7fff, so masking
is a little dirty.  Since about ~half of random 16bit numbers will
have the high bit set, we'll have to read a second one.  My natural
instinct is to avoid those extra reads :)

Regards,
Jason

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 4/4] x86/boot: Size the boot/directmap mappings dynamically

2020-01-13 Thread Andrew Cooper
... rather than presuming that 16M will do.  With this fixed, Xen is now
capable of booting even when its compiled size is larger than 16M.

On the EFI side, use l2e_add_flags() to reduce the code-generation overhead of
using l2e_from_paddr() twice.

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
CC: Wei Liu 
CC: Roger Pau Monné 

Can be tested by trying to boot with:

  diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
  index 759827a19a..fa83a9a28f 100644
  --- a/xen/arch/x86/setup.c
  +++ b/xen/arch/x86/setup.c
  @@ -52,6 +52,8 @@
   #include 
   #include 

  +static uint8_t __used big_data[MB(16)] = { 42, };
  +
   /* opt_nosmp: If true, secondary processors are ignored. */
   static bool __initdata opt_nosmp;
   boolean_param("nosmp", opt_nosmp);

Before this series, Xen will triple fault in one of two places (both on the
transition to Xen's high mappings), both ultimately because of cpu0_stack[]
getting shifted off the top of the mappings.
---
 xen/arch/x86/boot/head.S| 21 +
 xen/arch/x86/efi/efi-boot.h | 23 ++-
 xen/arch/x86/xen.lds.S  |  3 ---
 3 files changed, 31 insertions(+), 16 deletions(-)

diff --git a/xen/arch/x86/boot/head.S b/xen/arch/x86/boot/head.S
index 94bed4a2d3..eda3161fb1 100644
--- a/xen/arch/x86/boot/head.S
+++ b/xen/arch/x86/boot/head.S
@@ -687,14 +687,19 @@ trampoline_setup:
  * handling/walking), and identity map Xen into bootmap (needed for
  * the transition into long mode), using 2M superpages.
  */
-lea sym_esi(start),%ebx
-lea 
(1<> L2_PAGETABLE_SHIFT) & (4 * L2_PAGETABLE_ENTRIES - 1))
+
+for ( i  = l2_4G_offset(_start);
+  i <= l2_4G_offset(_end - 1); ++i )
 {
-unsigned int slot = (xen_phys_start >> L2_PAGETABLE_SHIFT) + i;
-paddr_t addr = slot << L2_PAGETABLE_SHIFT;
+l2_pgentry_t pte = l2e_from_paddr(i << L2_PAGETABLE_SHIFT,
+  __PAGE_HYPERVISOR | _PAGE_PSE);
+
+l2_bootmap[i] = pte;
+
+/* Bootmap RWX/Non-global.  Directmap RW/Global. */
+l2e_add_flags(pte, PAGE_HYPERVISOR);
 
-l2_directmap[slot] = l2e_from_paddr(addr, PAGE_HYPERVISOR|_PAGE_PSE);
-l2_bootmap[slot] = l2e_from_paddr(addr, __PAGE_HYPERVISOR|_PAGE_PSE);
+l2_directmap[i] = pte;
 }
+#undef l2_4G_offset
 }
 
 static void __init efi_arch_handle_module(struct file *file, const CHAR16 
*name,
diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S
index 7c351b9df3..a71853a856 100644
--- a/xen/arch/x86/xen.lds.S
+++ b/xen/arch/x86/xen.lds.S
@@ -384,6 +384,3 @@ ASSERT((trampoline_end - trampoline_start) < 
TRAMPOLINE_SPACE - MBI_SPACE_MIN,
 "not enough room for trampoline and mbi data")
 ASSERT((wakeup_stack - wakeup_stack_start) >= WAKEUP_STACK_MIN,
 "wakeup stack too small")
-
-/* Plenty of boot code assumes that Xen isn't larger than 16M. */
-ASSERT(_end - _start <= MB(16), "Xen too large for early-boot assumptions")
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 2/4] x86/page: Remove bifrucated PAGE_HYPERVISOR constant

2020-01-13 Thread Andrew Cooper
Despite being vaguely aware, the difference between PAGE_HYPERVISOR in ASM and
C code has nevertheless caused several bugs I should have known better about,
and contributed to review confusion.

There are exactly 4 uses of these constants in asm code (and one is shortly
going to disappear).

Instead of creating the constants which behave differently between ASM and C
code, expose all the constants and use non-ambuguous non-NX ones in ASM.

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
CC: Wei Liu 
CC: Roger Pau Monné 
---
 xen/arch/x86/boot/head.S  | 2 +-
 xen/arch/x86/boot/x86_64.S| 6 +++---
 xen/include/asm-x86/x86_64/page.h | 7 ---
 3 files changed, 4 insertions(+), 11 deletions(-)

diff --git a/xen/arch/x86/boot/head.S b/xen/arch/x86/boot/head.S
index aaf0e119db..c5acbf56ae 100644
--- a/xen/arch/x86/boot/head.S
+++ b/xen/arch/x86/boot/head.S
@@ -674,7 +674,7 @@ trampoline_setup:
  * the transition into long mode), using 2M superpages.
  */
 lea sym_esi(start),%ebx
-lea 
(1<= 0xa0 && pfn < 0xc0
-.quad (pfn << PAGE_SHIFT) | PAGE_HYPERVISOR_UCMINUS | MAP_SMALL_PAGES
+.quad (pfn << PAGE_SHIFT) | __PAGE_HYPERVISOR_UCMINUS | _PAGE_GLOBAL | 
MAP_SMALL_PAGES
 .else
-.quad (pfn << PAGE_SHIFT) | PAGE_HYPERVISOR | MAP_SMALL_PAGES
+.quad (pfn << PAGE_SHIFT) | PAGE_HYPERVISOR_RWX | MAP_SMALL_PAGES
 .endif
 pfn = pfn + 1
 .endr
@@ -89,7 +89,7 @@ GLOBAL(l2_xenmap)
 .quad 0
 idx = 1
 .rept 7
-.quad sym_offs(__image_base__) + (idx << L2_PAGETABLE_SHIFT) + 
(PAGE_HYPERVISOR | _PAGE_PSE)
+.quad sym_offs(__image_base__) + (idx << L2_PAGETABLE_SHIFT) + 
(PAGE_HYPERVISOR_RWX | _PAGE_PSE)
 idx = idx + 1
 .endr
 .fill L2_PAGETABLE_ENTRIES - 8, 8, 0
diff --git a/xen/include/asm-x86/x86_64/page.h 
b/xen/include/asm-x86/x86_64/page.h
index 4fe0205553..1a4af85469 100644
--- a/xen/include/asm-x86/x86_64/page.h
+++ b/xen/include/asm-x86/x86_64/page.h
@@ -172,18 +172,11 @@ static inline intpte_t put_pte_flags(unsigned int x)
 #define PAGE_HYPERVISOR_RX  (__PAGE_HYPERVISOR_RX  | _PAGE_GLOBAL)
 #define PAGE_HYPERVISOR_RWX (__PAGE_HYPERVISOR | _PAGE_GLOBAL)
 
-#ifdef __ASSEMBLY__
-/* Dependency on NX being available can't be expressed. */
-# define PAGE_HYPERVISOR PAGE_HYPERVISOR_RWX
-# define PAGE_HYPERVISOR_UCMINUS (__PAGE_HYPERVISOR_UCMINUS | _PAGE_GLOBAL)
-# define PAGE_HYPERVISOR_UC  (__PAGE_HYPERVISOR_UC  | _PAGE_GLOBAL)
-#else
 # define PAGE_HYPERVISOR PAGE_HYPERVISOR_RW
 # define PAGE_HYPERVISOR_UCMINUS (__PAGE_HYPERVISOR_UCMINUS | \
   _PAGE_GLOBAL | _PAGE_NX)
 # define PAGE_HYPERVISOR_UC  (__PAGE_HYPERVISOR_UC | \
   _PAGE_GLOBAL | _PAGE_NX)
-#endif
 
 #endif /* __X86_64_PAGE_H__ */
 
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 3/4] x86/boot: Create the l2_xenmap[] mappings dynamically

2020-01-13 Thread Andrew Cooper
The build-time construction of l2_xenmap[] imposes an arbitrary limit of 16M
total, which is a limit looking to be lifted.

Move l2_xenmap[] into the bss, and adjust both the BIOS and EFI paths to fill
it in dynamically, based on the final linked size of Xen.  For current builds,
this reduces the number of .text/etc mappings from 7 to 4.

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
CC: Wei Liu 
CC: Roger Pau Monné 

In principle, the non-EFI case could be made to work by having a post-link
script fill in a suitable number of _PAGE_PRESENT entries in l2_xenmap[].
This doesn't work for the EFI case, because pagetable relocation is instead
triggered on the ad-hoc relocation table, which would require the
_PAGE_PRESENT references to be in place before the link takes place.
---
 xen/arch/x86/boot/head.S| 14 ++
 xen/arch/x86/boot/x86_64.S  | 23 ---
 xen/arch/x86/efi/efi-boot.h | 14 ++
 xen/arch/x86/xen.lds.S  |  3 +++
 4 files changed, 39 insertions(+), 15 deletions(-)

diff --git a/xen/arch/x86/boot/head.S b/xen/arch/x86/boot/head.S
index c5acbf56ae..94bed4a2d3 100644
--- a/xen/arch/x86/boot/head.S
+++ b/xen/arch/x86/boot/head.S
@@ -668,6 +668,20 @@ trampoline_setup:
 add %esi,sym_fs(__page_tables_start)-8(,%ecx,8)
 2:  loop1b
 
+/* Map Xen into the higher mappings using 2M superpages. */
+lea _PAGE_PSE + PAGE_HYPERVISOR_RWX + sym_esi(_start), %eax
+mov $sym_offs(_start),   %ecx   /* %eax = PTE to write*/
+mov $sym_offs(_end - 1), %edx
+shr $L2_PAGETABLE_SHIFT, %ecx   /* %ecx = First slot to write */
+shr $L2_PAGETABLE_SHIFT, %edx   /* %edx = Final slot to write */
+
+1:  mov %eax, sym_offs(l2_xenmap)(%esi, %ecx, 8)
+add $1, %ecx
+add $1 << L2_PAGETABLE_SHIFT, %eax
+
+cmp %edx, %ecx
+jbe 1b
+
 /*
  * Map Xen into the directmap (needed for early-boot pagetable
  * handling/walking), and identity map Xen into bootmap (needed for
diff --git a/xen/arch/x86/boot/x86_64.S b/xen/arch/x86/boot/x86_64.S
index aabf561b23..e63bece460 100644
--- a/xen/arch/x86/boot/x86_64.S
+++ b/xen/arch/x86/boot/x86_64.S
@@ -43,6 +43,14 @@ multiboot_ptr:
 GLOBAL(stack_start)
 .quad   cpu0_stack + STACK_SIZE - CPUINFO_sizeof
 
+.section .bss.page_aligned, "aw", @nobits
+.align PAGE_SIZE, 0
+
+/* L2 mapping the Xen text/data/bss region.  Uses 1x 4k page. */
+GLOBAL(l2_xenmap)
+.fill L2_PAGETABLE_ENTRIES, 8, 0
+.size l2_xenmap, . - l2_xenmap
+
 .section .data.page_aligned, "aw", @progbits
 .align PAGE_SIZE, 0
 /*
@@ -80,21 +88,6 @@ GLOBAL(l2_directmap)
 .fill 4 * L2_PAGETABLE_ENTRIES - 1, 8, 0
 .size l2_directmap, . - l2_directmap
 
-/*
- * L2 mapping the 1GB Xen text/data/bss region.  At boot it maps 16MB from
- * __image_base__, and is modified when Xen relocates itself.  Uses 1x 4k
- * page.
- */
-GLOBAL(l2_xenmap)
-.quad 0
-idx = 1
-.rept 7
-.quad sym_offs(__image_base__) + (idx << L2_PAGETABLE_SHIFT) + 
(PAGE_HYPERVISOR_RWX | _PAGE_PSE)
-idx = idx + 1
-.endr
-.fill L2_PAGETABLE_ENTRIES - 8, 8, 0
-.size l2_xenmap, . - l2_xenmap
-
 /* L2 mapping the fixmap.  Uses 1x 4k page. */
 l2_fixmap:
 idx = 0
diff --git a/xen/arch/x86/efi/efi-boot.h b/xen/arch/x86/efi/efi-boot.h
index 50d1499867..e750db6f5c 100644
--- a/xen/arch/x86/efi/efi-boot.h
+++ b/xen/arch/x86/efi/efi-boot.h
@@ -585,6 +585,20 @@ static void __init efi_arch_memory_setup(void)
 if ( !efi_enabled(EFI_LOADER) )
 return;
 
+/*
+ * Map Xen into the higher mappings, using 2M superpages.
+ *
+ * NB: We are currently in physical mode, so a RIP-relative relocation
+ * against _start/_end gets their position as placed by the bootloader,
+ * not as expected in the final build.  This has arbitrary 2M alignment,
+ * so subtract xen_phys_start to get the appropriate slots in l2_xenmap[].
+ */
+for ( i =  l2_table_offset((UINTN)_start   - xen_phys_start);
+  i <= l2_table_offset((UINTN)_end - 1 - xen_phys_start); ++i )
+l2_xenmap[i] =
+l2e_from_paddr(xen_phys_start + (i << L2_PAGETABLE_SHIFT),
+   PAGE_HYPERVISOR_RWX | _PAGE_PSE);
+
 /* Check that there is at least 4G of mapping space in l2_*map[] */
 BUILD_BUG_ON((sizeof(l2_bootmap)   / L2_PAGETABLE_ENTRIES) < 4);
 BUILD_BUG_ON((sizeof(l2_directmap) / L2_PAGETABLE_ENTRIES) < 4);
diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S
index 7f82f64078..7c351b9df3 100644
--- a/xen/arch/x86/xen.lds.S
+++ b/xen/arch/x86/xen.lds.S
@@ -359,6 +359,9 @@ ASSERT(__image_base__ > XEN_VIRT_START |
 ASSERT(kexec_reloc_size - kexec_reloc <= PAGE_SIZE, "kexec_reloc is too large")
 #endif
 
+/* The Multiboot setup paths depend on this to simplify superpage PTE 

[Xen-devel] [PATCH 2/4] x86/page: Remove bifurcated PAGE_HYPERVISOR constant

2020-01-13 Thread Andrew Cooper
Despite being vaguely aware, the difference between PAGE_HYPERVISOR in ASM and
C code has nevertheless caused several bugs I should have known better about,
and contributed to review confusion.

There are exactly 4 uses of these constants in asm code (and one is shortly
going to disappear).

Instead of creating the constants which behave differently between ASM and C
code, expose all the constants and use non-ambiguous non-NX ones in ASM.

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
CC: Wei Liu 
CC: Roger Pau Monné 
---
 xen/arch/x86/boot/head.S  | 2 +-
 xen/arch/x86/boot/x86_64.S| 6 +++---
 xen/include/asm-x86/x86_64/page.h | 7 ---
 3 files changed, 4 insertions(+), 11 deletions(-)

diff --git a/xen/arch/x86/boot/head.S b/xen/arch/x86/boot/head.S
index aaf0e119db..c5acbf56ae 100644
--- a/xen/arch/x86/boot/head.S
+++ b/xen/arch/x86/boot/head.S
@@ -674,7 +674,7 @@ trampoline_setup:
  * the transition into long mode), using 2M superpages.
  */
 lea sym_esi(start),%ebx
-lea 
(1<= 0xa0 && pfn < 0xc0
-.quad (pfn << PAGE_SHIFT) | PAGE_HYPERVISOR_UCMINUS | MAP_SMALL_PAGES
+.quad (pfn << PAGE_SHIFT) | __PAGE_HYPERVISOR_UCMINUS | _PAGE_GLOBAL | 
MAP_SMALL_PAGES
 .else
-.quad (pfn << PAGE_SHIFT) | PAGE_HYPERVISOR | MAP_SMALL_PAGES
+.quad (pfn << PAGE_SHIFT) | PAGE_HYPERVISOR_RWX | MAP_SMALL_PAGES
 .endif
 pfn = pfn + 1
 .endr
@@ -89,7 +89,7 @@ GLOBAL(l2_xenmap)
 .quad 0
 idx = 1
 .rept 7
-.quad sym_offs(__image_base__) + (idx << L2_PAGETABLE_SHIFT) + 
(PAGE_HYPERVISOR | _PAGE_PSE)
+.quad sym_offs(__image_base__) + (idx << L2_PAGETABLE_SHIFT) + 
(PAGE_HYPERVISOR_RWX | _PAGE_PSE)
 idx = idx + 1
 .endr
 .fill L2_PAGETABLE_ENTRIES - 8, 8, 0
diff --git a/xen/include/asm-x86/x86_64/page.h 
b/xen/include/asm-x86/x86_64/page.h
index 4fe0205553..1a4af85469 100644
--- a/xen/include/asm-x86/x86_64/page.h
+++ b/xen/include/asm-x86/x86_64/page.h
@@ -172,18 +172,11 @@ static inline intpte_t put_pte_flags(unsigned int x)
 #define PAGE_HYPERVISOR_RX  (__PAGE_HYPERVISOR_RX  | _PAGE_GLOBAL)
 #define PAGE_HYPERVISOR_RWX (__PAGE_HYPERVISOR | _PAGE_GLOBAL)
 
-#ifdef __ASSEMBLY__
-/* Dependency on NX being available can't be expressed. */
-# define PAGE_HYPERVISOR PAGE_HYPERVISOR_RWX
-# define PAGE_HYPERVISOR_UCMINUS (__PAGE_HYPERVISOR_UCMINUS | _PAGE_GLOBAL)
-# define PAGE_HYPERVISOR_UC  (__PAGE_HYPERVISOR_UC  | _PAGE_GLOBAL)
-#else
 # define PAGE_HYPERVISOR PAGE_HYPERVISOR_RW
 # define PAGE_HYPERVISOR_UCMINUS (__PAGE_HYPERVISOR_UCMINUS | \
   _PAGE_GLOBAL | _PAGE_NX)
 # define PAGE_HYPERVISOR_UC  (__PAGE_HYPERVISOR_UC | \
   _PAGE_GLOBAL | _PAGE_NX)
-#endif
 
 #endif /* __X86_64_PAGE_H__ */
 
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 1/4] x86/boot: Rename l?_identmap to l?_directmap

2020-01-13 Thread Andrew Cooper
Since c/s faa85d4fb3 "x86/boot: Don't map 0 during boot", l1_identmap no
longer has an alias mapped at 0, meaning that none of the l?_identmap[]
pagetables are actually an identity map.

Rename them to l?_directmap, which avoids any kind of implication that they
might be mapped at 0.

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
CC: Wei Liu 
CC: Roger Pau Monné 
---
 xen/arch/x86/boot/head.S|  2 +-
 xen/arch/x86/boot/x86_64.S  | 22 +++---
 xen/arch/x86/efi/efi-boot.h |  8 
 xen/arch/x86/setup.c|  6 +++---
 xen/include/asm-x86/page.h  |  2 +-
 5 files changed, 20 insertions(+), 20 deletions(-)

diff --git a/xen/arch/x86/boot/head.S b/xen/arch/x86/boot/head.S
index d246e374f1..aaf0e119db 100644
--- a/xen/arch/x86/boot/head.S
+++ b/xen/arch/x86/boot/head.S
@@ -678,7 +678,7 @@ trampoline_setup:
 shr $(L2_PAGETABLE_SHIFT-3),%ebx
 mov $8,%ecx
 1:  mov %eax,sym_fs(l2_bootmap)-8(%ebx,%ecx,8)
-mov %eax,sym_fs(l2_identmap)-8(%ebx,%ecx,8)
+mov %eax,sym_fs(l2_directmap)-8(%ebx,%ecx,8)
 sub $(1<> L2_PAGETABLE_SHIFT) + i;
 paddr_t addr = slot << L2_PAGETABLE_SHIFT;
 
-l2_identmap[slot] = l2e_from_paddr(addr, PAGE_HYPERVISOR|_PAGE_PSE);
+l2_directmap[slot] = l2e_from_paddr(addr, PAGE_HYPERVISOR|_PAGE_PSE);
 l2_bootmap[slot] = l2e_from_paddr(addr, __PAGE_HYPERVISOR|_PAGE_PSE);
 }
 }
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 1b6ca4a47d..5bdc229bd6 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1031,7 +1031,7 @@ void __init noreturn __start_xen(unsigned long mbi_p)
 for ( i = boot_e820.nr_map-1; i >= 0; i-- )
 {
 uint64_t s, e, mask = (1UL << L2_PAGETABLE_SHIFT) - 1;
-uint64_t end, limit = ARRAY_SIZE(l2_identmap) << L2_PAGETABLE_SHIFT;
+uint64_t end, limit = ARRAY_SIZE(l2_directmap) << L2_PAGETABLE_SHIFT;
 
 if ( boot_e820.map[i].type != E820_RAM )
 continue;
@@ -1136,7 +1136,7 @@ void __init noreturn __start_xen(unsigned long mbi_p)
 /* The only data mappings to be relocated are in the Xen area. */
 pl2e = __va(__pa(l2_xenmap));
 /*
- * Undo the temporary-hooking of the l1_identmap.  __2M_text_start
+ * Undo the temporary-hooking of the l1_directmap.  __2M_text_start
  * is contained in this PTE.
  */
 BUG_ON(using_2M_mapping() &&
@@ -1349,7 +1349,7 @@ void __init noreturn __start_xen(unsigned long mbi_p)
 /* Need to create mappings above PREBUILT_MAP_LIMIT. */
 map_s = max_t(uint64_t, s, PREBUILT_MAP_LIMIT);
 map_e = min_t(uint64_t, e,
-  ARRAY_SIZE(l2_identmap) << L2_PAGETABLE_SHIFT);
+  ARRAY_SIZE(l2_directmap) << L2_PAGETABLE_SHIFT);
 
 /* Pass mapped memory to allocator /before/ creating new mappings. */
 init_boot_pages(s, min(map_s, e));
diff --git a/xen/include/asm-x86/page.h b/xen/include/asm-x86/page.h
index 05a8b1efa6..4b9a4fa33f 100644
--- a/xen/include/asm-x86/page.h
+++ b/xen/include/asm-x86/page.h
@@ -293,7 +293,7 @@ extern unsigned int   m2p_compat_vstart;
 extern l2_pgentry_t l2_xenmap[L2_PAGETABLE_ENTRIES],
 l2_bootmap[4*L2_PAGETABLE_ENTRIES];
 extern l3_pgentry_t l3_bootmap[L3_PAGETABLE_ENTRIES];
-extern l2_pgentry_t l2_identmap[4*L2_PAGETABLE_ENTRIES];
+extern l2_pgentry_t l2_directmap[4*L2_PAGETABLE_ENTRIES];
 extern l1_pgentry_t l1_fixmap[L1_PAGETABLE_ENTRIES];
 void paging_init(void);
 void efi_update_l4_pgtable(unsigned int l4idx, l4_pgentry_t);
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 0/4] x86: Remove 16M total-size restriction

2020-01-13 Thread Andrew Cooper
It turns out that the note in c/s a8d27a54cc9cc

  Finally, leave a linker assert covering the fact that plenty of code blindly
  assumes that Xen is less that 16M.  This wants fixing in due course.

was easier to address than I had originally anticipated.  This series does so.

The end result can be tested by trying to boot with:

  diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
  index 759827a19a..fa83a9a28f 100644
  --- a/xen/arch/x86/setup.c
  +++ b/xen/arch/x86/setup.c
  @@ -52,6 +52,8 @@
   #include 
   #include 

  +static uint8_t __used big_data[MB(16)] = { 42, };
  +
   /* opt_nosmp: If true, secondary processors are ignored. */
   static bool __initdata opt_nosmp;
   boolean_param("nosmp", opt_nosmp);

Before this series, Xen will triple fault in one of several places (first and
most obviously, __high_start on the first stack access, as cpu0_stack[] is
very near the end of Xen's linked image).

Andrew Cooper (4):
  x86/boot: Rename l?_identmap to l?_directmap
  x86/page: Remove bifurcated PAGE_HYPERVISOR constant
  x86/boot: Create the l2_xenmap[] mappings dynamically
  x86/boot: Size the boot/directmap mappings dynamically

 xen/arch/x86/boot/head.S  | 35 +---
 xen/arch/x86/boot/x86_64.S| 49 +--
 xen/arch/x86/efi/efi-boot.h   | 43 +++---
 xen/arch/x86/setup.c  |  6 ++---
 xen/arch/x86/xen.lds.S|  6 ++---
 xen/include/asm-x86/page.h|  2 +-
 xen/include/asm-x86/x86_64/page.h |  7 --
 7 files changed, 90 insertions(+), 58 deletions(-)

-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] live migration from 4.12 to 4.13 fails due to qemu-xen bug

2020-01-13 Thread Olaf Hering
Am Mon, 13 Jan 2020 11:36:27 +0100
schrieb Olaf Hering :

> qemu-system-i386: Unknown savevm section type 111

Looks like hw/i386/pc_piix.c:xenfv_machine_options must set 
m->smbus_no_migration_support to true.
Not sure why this remained unnoticed for so long.

Olaf


pgpffQqWBqgtH.pgp
Description: Digitale Signatur von OpenPGP
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3] xen-pciback: optionally allow interrupt enable flag writes

2020-01-13 Thread Nick Desaulniers
Hi Marek,
Below is a report from 0day bot build w/ Clang. The warning looks
legit, can you please take a look? Apologies if this has already been
reported.

On Sat, Jan 11, 2020 at 7:48 AM kbuild test robot  wrote:
>
> CC: kbuild-...@lists.01.org
> In-Reply-To: <20200111034347.5270-1-marma...@invisiblethingslab.com>
> References: <20200111034347.5270-1-marma...@invisiblethingslab.com>
> TO: "Marek Marczykowski-Górecki" 
> CC: xen-devel@lists.xenproject.org, "Marek Marczykowski-Górecki" 
> , Jan Beulich , Simon 
> Gaiser , Boris Ostrovsky 
> , Juergen Gross , Stefano 
> Stabellini , YueHaibing , open 
> list , "Marek Marczykowski-Górecki" 
> , Jan Beulich , Simon 
> Gaiser , Boris Ostrovsky 
> , Juergen Gross , Stefano 
> Stabellini , YueHaibing , open 
> list 
> CC: "Marek Marczykowski-Górecki" , Jan 
> Beulich , Simon Gaiser , 
> Boris Ostrovsky , Juergen Gross 
> , Stefano Stabellini , YueHaibing 
> , open list 
>
> Hi "Marek,
>
> Thank you for the patch! Perhaps something to improve:
>
> [auto build test WARNING on xen-tip/linux-next]
> [also build test WARNING on linux/master linus/master v5.5-rc5 next-20200110]
> [if your patch is applied to the wrong git tree, please drop us a note to help
> improve the system. BTW, we also suggest to use '--base' option to specify the
> base tree in git format-patch, please see 
> https://stackoverflow.com/a/37406982]
>
> url:
> https://github.com/0day-ci/linux/commits/Marek-Marczykowski-G-recki/xen-pciback-optionally-allow-interrupt-enable-flag-writes/20200111-162243
> base:   https://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git linux-next
> config: x86_64-allyesconfig (attached as .config)
> compiler: clang version 10.0.0 (git://gitmirror/llvm_project 
> 016bf03ef6fcd9dce43b0c17971f76323f07a684)
> reproduce:
> # save the attached .config to linux build tree
> make ARCH=x86_64
>
> If you fix the issue, kindly add following tag
> Reported-by: kbuild test robot 
>
> All warnings (new ones prefixed by >>):
>
> >> drivers/xen/xen-pciback/conf_space_header.c:121:19: warning: variable 
> >> 'val' is uninitialized when used here [-Wuninitialized]
>if ((cmd->val ^ val) & PCI_COMMAND_INTX_DISABLE) {
>^~~
>drivers/xen/xen-pciback/conf_space_header.c:65:9: note: initialize the 
> variable 'val' to silence this warning
>u16 val;
>   ^
>= 0
>1 warning generated.
>
> vim +/val +121 drivers/xen/xen-pciback/conf_space_header.c
>
> 60
> 61  static int command_write(struct pci_dev *dev, int offset, u16 value, 
> void *data)
> 62  {
> 63  struct xen_pcibk_dev_data *dev_data;
> 64  int err;
> 65  u16 val;
> 66  struct pci_cmd_info *cmd = data;
> 67
> 68  dev_data = pci_get_drvdata(dev);
> 69  if (!pci_is_enabled(dev) && is_enable_cmd(value)) {
> 70  if (unlikely(verbose_request))
> 71  printk(KERN_DEBUG DRV_NAME ": %s: enable\n",
> 72 pci_name(dev));
> 73  err = pci_enable_device(dev);
> 74  if (err)
> 75  return err;
> 76  if (dev_data)
> 77  dev_data->enable_intx = 1;
> 78  } else if (pci_is_enabled(dev) && !is_enable_cmd(value)) {
> 79  if (unlikely(verbose_request))
> 80  printk(KERN_DEBUG DRV_NAME ": %s: disable\n",
> 81 pci_name(dev));
> 82  pci_disable_device(dev);
> 83  if (dev_data)
> 84  dev_data->enable_intx = 0;
> 85  }
> 86
> 87  if (!dev->is_busmaster && is_master_cmd(value)) {
> 88  if (unlikely(verbose_request))
> 89  printk(KERN_DEBUG DRV_NAME ": %s: set bus 
> master\n",
> 90 pci_name(dev));
> 91  pci_set_master(dev);
> 92  } else if (dev->is_busmaster && !is_master_cmd(value)) {
> 93  if (unlikely(verbose_request))
> 94  printk(KERN_DEBUG DRV_NAME ": %s: clear bus 
> master\n",
> 95 pci_name(dev));
> 96  pci_clear_master(dev);
> 97  }
> 98
> 99  if (!(cmd->val & PCI_COMMAND_INVALIDATE) &&
>100  (value & PCI_COMMAND_INVALIDATE)) {
>101  if (unlikely(verbose_request))
>102  printk(KERN_DEBUG
>103 DRV_NAME ": %s: enable 
> memory-write-invalidate\n",
>104 pci_name(dev));
>105  err = pci_set_mwi(dev);
>106  if (err) {
>107 

[Xen-devel] [PATCH v2 09/10] libxl: event: Fix possible hang with libxl_osevent_beforepoll

2020-01-13 Thread Ian Jackson
If the application uses libxl_osevent_beforepoll, a similar hang is
possible to the one described and fixed in
   libxl: event: Fix hang when mixing blocking and eventy calls
Application behaviour would have to be fairly unusual, but it
doesn't seem sensible to just leave this latent bug.

We fix the latent bug by waking up the "poller_app" pipe every time we
add osevents.  If the application does not ever call beforepoll, we
write one byte to the pipe and set pipe_nonempty and then we ignore
it.  We only write another byte if beforepoll is called again.

Normally in an eventy program there would only be one thread calling
libxl_osevent_beforepoll.  The effect in such a program is to
sometimes needlessly go round the poll loop again if a timeout
callback becomes interested in a new osevent.  We'll fix that in a
moment.

Signed-off-by: Ian Jackson 
---
v2: New addition to correctness arguments in libxl_event.c comment.
---
 tools/libxl/libxl_event.c | 54 +--
 1 file changed, 43 insertions(+), 11 deletions(-)

diff --git a/tools/libxl/libxl_event.c b/tools/libxl/libxl_event.c
index 45cc67942d..5f6a607d80 100644
--- a/tools/libxl/libxl_event.c
+++ b/tools/libxl/libxl_event.c
@@ -41,18 +41,25 @@ static void ao__check_destroy(libxl_ctx *ctx, libxl__ao 
*ao);
  *
  * We need the following property (the "unstale liveness property"):
  *
- * Whenever any thread is blocking in the libxl event loop[1], at
- * least one thread must be using an up to date osevent set.  It is OK
- * for all but one threads to have stale event sets, because so long
- * as one waiting thread has the right event set, any actually
- * interesting event will, if nothing else, wake that "right" thread
- * up.  It will then make some progress and/or, if it exits, ensure
- * that some other thread becomes the "right" thread.
+ * Whenever any thread is blocking as a result of being given an fd
+ * set or timeout by libxl, at least one thread must be using an up to
+ * date osevent set.  It is OK for all but one threads to have stale
+ * event sets, because so long as one waiting thread has the right
+ * event set, any actually interesting event will, if nothing else,
+ * wake that "right" thread up.  It will then make some progress
+ * and/or, if it exits, ensure that some other thread becomes the
+ * "right" thread.
  *
- * [1] TODO: Right now we are considering only the libxl event loop.
- * We need to consider application event loop outside libxl too.
+ * For threads blocking outside libxl and which are receiving libxl's
+ * fd and timeout information via the libxl_osevent_hooks callbacks,
+ * libxl calls this function as soon as it becomes interested.  It is
+ * the responsiblity of a provider of these functions in a
+ * multithreaded environment to make arrangements to wake up event
+ * waiting thread(s) with stale event sets.
  *
- * Argument that our approach is sound:
+ * Waiters outside libxl using _beforepoll are dealt with below.
+ *
+ * For the libxl event loop, the argument is as follows:
  *
  * The issue we are concerned about is libxl sleeping on an out of
  * date fd set, or too long a timeout, so that it doesn't make
@@ -132,7 +139,29 @@ static void ao__check_destroy(libxl_ctx *ctx, libxl__ao 
*ao);
  * will reenter libxl when it gains the lock and necessarily then
  * becomes a baton holder in category (a).
  *
- * So the "baton invariant" is maintained.  QED.
+ * So the "baton invariant" is maintained.
+ * QED (for waiters in libxl).
+ *
+ *
+ * For waiters outside libxl which used libxl_osevent_beforepoll
+ * to get the fd set:
+ *
+ * As above, adding an osevent involves having an egc or an ao.
+ * It sets poller->osevents_added on all active pollers.  Notably
+ * it sets it on poller_app, which is always active.
+ *
+ * The thread which does this will dispose of its egc or ao before
+ * exiting libxl so it will always wake up the poller_app if the last
+ * call to _beforepoll was before the osevents were added.  So the
+ * application's fd set contains at least a wakeup in the form of the
+ * poller_app fd.  The application cannot sleep on the libxl fd set
+ * until it has called _afterpoll which empties the pipe, and it
+ * is expected to then call _beforepoll again before sleeping.
+ *
+ * So all the application's event waiting thread(s) will always have
+ * an up to date osevent set, and will be woken up if necessary to
+ * achieve this.  (This is in contrast libxl's own event loop where
+ * only one thread need be up to date, as discussed above.)
  */
 static void pollers_note_osevent_added(libxl_ctx *ctx) {
 libxl__poller *poller;
@@ -157,6 +186,9 @@ void libxl__egc_ao_cleanup_1_baton(libxl__gc *gc)
 {
 libxl__poller *search, *wake=0;
 
+if (CTX->poller_app->osevents_added)
+baton_wake(gc, CTX->poller_app);
+
 LIBXL_LIST_FOREACH(search, >pollers_active, active_entry) {
 if (search == CTX->poller_app)
 /* This one is special.  We 

[Xen-devel] [PATCH v2 03/10] libxl: event: Introduce CTX_UNLOCK_EGC_FREE

2020-01-13 Thread Ian Jackson
This is a very common exit pattern.  We are going to want to change
this pattern.  So we should make it into a macro of its own.

No functional change.

Signed-off-by: Ian Jackson 
---
 tools/libxl/libxl_event.c| 18 ++
 tools/libxl/libxl_fork.c |  6 ++
 tools/libxl/libxl_internal.h |  2 ++
 3 files changed, 10 insertions(+), 16 deletions(-)

diff --git a/tools/libxl/libxl_event.c b/tools/libxl/libxl_event.c
index 5b12a45e70..be37e12bb0 100644
--- a/tools/libxl/libxl_event.c
+++ b/tools/libxl/libxl_event.c
@@ -1152,8 +1152,7 @@ int libxl_osevent_beforepoll(libxl_ctx *ctx, int *nfds_io,
 CTX_LOCK;
 int rc = beforepoll_internal(gc, ctx->poller_app,
  nfds_io, fds, timeout_upd, now);
-CTX_UNLOCK;
-EGC_FREE;
+CTX_UNLOCK_EGC_FREE;
 return rc;
 }
 
@@ -1305,8 +1304,7 @@ void libxl_osevent_afterpoll(libxl_ctx *ctx, int nfds, 
const struct pollfd *fds,
 EGC_INIT(ctx);
 CTX_LOCK;
 afterpoll_internal(egc, ctx->poller_app, nfds, fds, now);
-CTX_UNLOCK;
-EGC_FREE;
+CTX_UNLOCK_EGC_FREE;
 }
 
 /*
@@ -1342,8 +1340,7 @@ void libxl_osevent_occurred_fd(libxl_ctx *ctx, void 
*for_libxl,
 fd_occurs(egc, ev, revents_ign);
 
  out:
-CTX_UNLOCK;
-EGC_FREE;
+CTX_UNLOCK_EGC_FREE;
 }
 
 void libxl_osevent_occurred_timeout(libxl_ctx *ctx, void *for_libxl)
@@ -1365,8 +1362,7 @@ void libxl_osevent_occurred_timeout(libxl_ctx *ctx, void 
*for_libxl)
 time_occurs(egc, ev, ERROR_TIMEDOUT);
 
  out:
-CTX_UNLOCK;
-EGC_FREE;
+CTX_UNLOCK_EGC_FREE;
 }
 
 void libxl__event_disaster(libxl__egc *egc, const char *msg, int errnoval,
@@ -1546,8 +1542,7 @@ int libxl_event_check(libxl_ctx *ctx, libxl_event 
**event_r,
 EGC_INIT(ctx);
 CTX_LOCK;
 int rc = event_check_internal(egc, event_r, typemask, pred, pred_user);
-CTX_UNLOCK;
-EGC_FREE;
+CTX_UNLOCK_EGC_FREE;
 return rc;
 }
 
@@ -1772,8 +1767,7 @@ int libxl_event_wait(libxl_ctx *ctx, libxl_event 
**event_r,
  out:
 libxl__poller_put(ctx, poller);
 
-CTX_UNLOCK;
-EGC_FREE;
+CTX_UNLOCK_EGC_FREE;
 return rc;
 }
 
diff --git a/tools/libxl/libxl_fork.c b/tools/libxl/libxl_fork.c
index 0f1b6b518c..cf170b9085 100644
--- a/tools/libxl/libxl_fork.c
+++ b/tools/libxl/libxl_fork.c
@@ -483,8 +483,7 @@ int libxl_childproc_reaped(libxl_ctx *ctx, pid_t pid, int 
status)
 assert(CTX->childproc_hooks->chldowner
== libxl_sigchld_owner_mainloop);
 int rc = childproc_reaped(egc, pid, status);
-CTX_UNLOCK;
-EGC_FREE;
+CTX_UNLOCK_EGC_FREE;
 return rc;
 }
 
@@ -529,8 +528,7 @@ void libxl_childproc_sigchld_occurred(libxl_ctx *ctx)
 assert(CTX->childproc_hooks->chldowner
== libxl_sigchld_owner_mainloop);
 childproc_checkall(egc);
-CTX_UNLOCK;
-EGC_FREE;
+CTX_UNLOCK_EGC_FREE;
 }
 
 static void sigchld_selfpipe_handler(libxl__egc *egc, libxl__ev_fd *ev,
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 581d64b99c..983fffac7a 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -2363,6 +2363,8 @@ _hidden void libxl__egc_cleanup(libxl__egc *egc);
 
 #define EGC_FREE   libxl__egc_cleanup(egc)
 
+#define CTX_UNLOCK_EGC_FREE  do{ CTX_UNLOCK; EGC_FREE; }while(0)
+
 
 /*
  * Machinery for asynchronous operations ("ao")
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 08/10] libxl: event: Break out baton_wake

2020-01-13 Thread Ian Jackson
No functional change.

Signed-off-by: Ian Jackson 
---
v2: Now it takes a gc, not an egc.
---
 tools/libxl/libxl_event.c | 21 +
 1 file changed, 13 insertions(+), 8 deletions(-)

diff --git a/tools/libxl/libxl_event.c b/tools/libxl/libxl_event.c
index 3e76fa5af5..45cc67942d 100644
--- a/tools/libxl/libxl_event.c
+++ b/tools/libxl/libxl_event.c
@@ -140,6 +140,18 @@ static void pollers_note_osevent_added(libxl_ctx *ctx) {
 poller->osevents_added = 1;
 }
 
+static void baton_wake(libxl__gc *gc, libxl__poller *wake)
+{
+libxl__poller_wakeup(gc, wake);
+
+wake->osevents_added = 0;
+/* This serves to make _1_baton idempotent.  It is OK even though
+ * that poller may currently be sleeping on only old osevents,
+ * because it is going to wake up because we've just prodded it,
+ * and it pick up new osevents on its next iteration (or pass
+ * on the baton). */
+}
+
 void libxl__egc_ao_cleanup_1_baton(libxl__gc *gc)
 /* Any poller we had must have been `put' already. */
 {
@@ -160,14 +172,7 @@ void libxl__egc_ao_cleanup_1_baton(libxl__gc *gc)
 /* no-one in libxl waiting for any events */
 return;
 
-libxl__poller_wakeup(gc, wake);
-
-wake->osevents_added = 0;
-/* This serves to make _1_baton idempotent.  It is OK even though
- * that poller may currently be sleeping on only old osevents,
- * because it is going to wake up because we've just prodded it,
- * and it pick up new osevents on its next iteration (or pass
- * on the baton). */
+baton_wake(gc, wake);
 }
 
 /*
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 02/10] libxl: event: Rename ctx.pollers_fd_changed to .pollers_active

2020-01-13 Thread Ian Jackson
We are going to use this a bit more widely.  Make the name more
general.

Signed-off-by: Ian Jackson 
---
 tools/libxl/libxl.c  | 4 ++--
 tools/libxl/libxl_event.c| 8 
 tools/libxl/libxl_internal.h | 6 +++---
 3 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index a0d84281d0..f60fd3e4fd 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -48,7 +48,7 @@ int libxl_ctx_alloc(libxl_ctx **pctx, int version,
 ctx->poller_app = 0;
 LIBXL_LIST_INIT(>pollers_event);
 LIBXL_LIST_INIT(>pollers_idle);
-LIBXL_LIST_INIT(>pollers_fds_changed);
+LIBXL_LIST_INIT(>pollers_active);
 
 LIBXL_LIST_INIT(>efds);
 LIBXL_TAILQ_INIT(>etimes);
@@ -177,7 +177,7 @@ int libxl_ctx_free(libxl_ctx *ctx)
 libxl__poller_put(ctx, ctx->poller_app);
 ctx->poller_app = NULL;
 assert(LIBXL_LIST_EMPTY(>pollers_event));
-assert(LIBXL_LIST_EMPTY(>pollers_fds_changed));
+assert(LIBXL_LIST_EMPTY(>pollers_active));
 libxl__poller *poller, *poller_tmp;
 LIBXL_LIST_FOREACH_SAFE(poller, >pollers_idle, entry, poller_tmp) {
 libxl__poller_dispose(poller);
diff --git a/tools/libxl/libxl_event.c b/tools/libxl/libxl_event.c
index 1210c1bfb3..5b12a45e70 100644
--- a/tools/libxl/libxl_event.c
+++ b/tools/libxl/libxl_event.c
@@ -238,7 +238,7 @@ void libxl__ev_fd_deregister(libxl__gc *gc, libxl__ev_fd 
*ev)
 LIBXL_LIST_REMOVE(ev, entry);
 ev->fd = -1;
 
-LIBXL_LIST_FOREACH(poller, >pollers_fds_changed, fds_changed_entry)
+LIBXL_LIST_FOREACH(poller, >pollers_active, active_entry)
 poller->fds_deregistered = 1;
 
  out:
@@ -1663,15 +1663,15 @@ libxl__poller *libxl__poller_get(libxl__gc *gc)
 }
 }
 
-LIBXL_LIST_INSERT_HEAD(>pollers_fds_changed, p,
-   fds_changed_entry);
+LIBXL_LIST_INSERT_HEAD(>pollers_active, p,
+   active_entry);
 return p;
 }
 
 void libxl__poller_put(libxl_ctx *ctx, libxl__poller *p)
 {
 if (!p) return;
-LIBXL_LIST_REMOVE(p, fds_changed_entry);
+LIBXL_LIST_REMOVE(p, active_entry);
 LIBXL_LIST_INSERT_HEAD(>pollers_idle, p, entry);
 }
 
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index c5b71d15f0..581d64b99c 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -629,13 +629,13 @@ struct libxl__poller {
 /*
  * We also use the poller to record whether any fds have been
  * deregistered since we entered poll.  Each poller which is not
- * idle is on the list pollers_fds_changed.  fds_deregistered is
+ * idle is on the list pollers_active.  fds_deregistered is
  * cleared by beforepoll, and tested by afterpoll.  Whenever an fd
  * event is deregistered, we set the fds_deregistered of all non-idle
  * pollers.  So afterpoll can tell whether any POLLNVAL is
  * plausibly due to an fd being closed and reopened.
  */
-LIBXL_LIST_ENTRY(libxl__poller) fds_changed_entry;
+LIBXL_LIST_ENTRY(libxl__poller) active_entry;
 bool fds_deregistered;
 };
 
@@ -678,7 +678,7 @@ struct libxl__ctx {
 
 libxl__poller *poller_app; /* libxl_osevent_beforepoll and _afterpoll */
 LIBXL_LIST_HEAD(, libxl__poller) pollers_event, pollers_idle;
-LIBXL_LIST_HEAD(, libxl__poller) pollers_fds_changed;
+LIBXL_LIST_HEAD(, libxl__poller) pollers_active;
 
 LIBXL_SLIST_HEAD(libxl__osevent_hook_nexi, libxl__osevent_hook_nexus)
 hook_fd_nexi_idle, hook_timeout_nexi_idle;
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 04/10] libxl: event: Make LIBXL__EVENT_DISASTER take a gc, not an egc

2020-01-13 Thread Ian Jackson
We are going to want to change libxl__poller_wakeup to take a gc.

In theory there is a risk here that it would be called inappropriately
in a future patch but this seems unlikely.

Signed-off-by: Ian Jackson 
---
v2: New patch
---
 tools/libxl/libxl_aoutils.c  |  2 +-
 tools/libxl/libxl_disk.c |  4 ++--
 tools/libxl/libxl_domain.c   |  2 +-
 tools/libxl/libxl_event.c| 21 ++---
 tools/libxl/libxl_fork.c | 11 ++-
 tools/libxl/libxl_internal.h | 10 +-
 6 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/tools/libxl/libxl_aoutils.c b/tools/libxl/libxl_aoutils.c
index e24e4eed53..1be858c93c 100644
--- a/tools/libxl/libxl_aoutils.c
+++ b/tools/libxl/libxl_aoutils.c
@@ -282,7 +282,7 @@ static void datacopier_readable(libxl__egc *egc, 
libxl__ev_fd *ev,
 hupchk.revents = 0;
 r = poll(, 1, 0);
 if (r < 0)
-LIBXL__EVENT_DISASTER(egc,
+LIBXL__EVENT_DISASTER(gc,
  "unexpected failure polling fd for datacopier eof hup check",
   errno, 0);
 if (datacopier_pollhup_handled(egc, dc, fd, hupchk.revents, 0))
diff --git a/tools/libxl/libxl_disk.c b/tools/libxl/libxl_disk.c
index 64a6691424..a463334130 100644
--- a/tools/libxl/libxl_disk.c
+++ b/tools/libxl/libxl_disk.c
@@ -33,7 +33,7 @@ static void disk_eject_xswatch_callback(libxl__egc *egc, 
libxl__ev_xswatch *w,
 return;
 
 if (libxl__xs_printf(gc, XBT_NULL, wpath, "")) {
-LIBXL__EVENT_DISASTER(egc, "xs_write failed acknowledging eject",
+LIBXL__EVENT_DISASTER(gc, "xs_write failed acknowledging eject",
   errno, LIBXL_EVENT_TYPE_DISK_EJECT);
 return;
 }
@@ -43,7 +43,7 @@ static void disk_eject_xswatch_callback(libxl__egc *egc, 
libxl__ev_xswatch *w,
 
 rc = libxl__xs_read_checked(gc, XBT_NULL, evg->be_ptr_path, );
 if (rc) {
-LIBXL__EVENT_DISASTER(egc, "xs_read failed reading be_ptr_path",
+LIBXL__EVENT_DISASTER(gc, "xs_read failed reading be_ptr_path",
   errno, LIBXL_EVENT_TYPE_DISK_EJECT);
 return;
 }
diff --git a/tools/libxl/libxl_domain.c b/tools/libxl/libxl_domain.c
index 5714501778..b59cc65750 100644
--- a/tools/libxl/libxl_domain.c
+++ b/tools/libxl/libxl_domain.c
@@ -892,7 +892,7 @@ static void domain_death_xswatch_callback(libxl__egc *egc, 
libxl__ev_xswatch *w,
 
 rc = xc_domain_getinfolist(CTX->xch, evg->domid, nentries, 
domaininfos);
 if (rc == -1) {
-LIBXL__EVENT_DISASTER(egc, "xc_domain_getinfolist failed while"
+LIBXL__EVENT_DISASTER(gc, "xc_domain_getinfolist failed while"
   " processing @releaseDomain watch event",
   errno, 0);
 goto out;
diff --git a/tools/libxl/libxl_event.c b/tools/libxl/libxl_event.c
index be37e12bb0..16e6786889 100644
--- a/tools/libxl/libxl_event.c
+++ b/tools/libxl/libxl_event.c
@@ -261,7 +261,7 @@ short libxl__fd_poll_recheck(libxl__egc *egc, int fd, short 
events) {
 break;
 assert(r<0);
 if (errno != EINTR) {
-LIBXL__EVENT_DISASTER(egc, "failed poll to check for fd", errno, 
0);
+LIBXL__EVENT_DISASTER(gc, "failed poll to check for fd", errno, 0);
 return 0;
 }
 }
@@ -509,14 +509,14 @@ static void watchfd_callback(libxl__egc *egc, 
libxl__ev_fd *ev,
 EGC_GC;
 
 if (revents & (POLLERR|POLLHUP))
-LIBXL__EVENT_DISASTER(egc, "unexpected poll event on watch fd", 0, 0);
+LIBXL__EVENT_DISASTER(gc, "unexpected poll event on watch fd", 0, 0);
 
 for (;;) {
 char **event = xs_check_watch(CTX->xsh);
 if (!event) {
 if (errno == EAGAIN) break;
 if (errno == EINTR) continue;
-LIBXL__EVENT_DISASTER(egc, "cannot check/read watches", errno, 0);
+LIBXL__EVENT_DISASTER(gc, "cannot check/read watches", errno, 0);
 return;
 }
 
@@ -705,7 +705,7 @@ static int evtchn_revents_check(libxl__egc *egc, int 
revents)
 
 if (revents & ~POLLIN) {
 LOG(ERROR, "unexpected poll event on event channel fd: %x", revents);
-LIBXL__EVENT_DISASTER(egc,
+LIBXL__EVENT_DISASTER(gc,
"unexpected poll event on event channel fd", 0, 0);
 libxl__ev_fd_deregister(gc, >evtchn_efd);
 return ERROR_FAIL;
@@ -746,7 +746,7 @@ static void evtchn_fd_callback(libxl__egc *egc, 
libxl__ev_fd *ev,
 if (port < 0) {
 if (errno == EAGAIN)
 break;
-LIBXL__EVENT_DISASTER(egc,
+LIBXL__EVENT_DISASTER(gc,
  "unexpected failure fetching occurring event port number from evtchn",
   errno, 0);
 return;
@@ -966,7 +966,7 @@ static void domaindeathcheck_callback(libxl__egc *egc, 
libxl__ev_xswatch *w,
 

[Xen-devel] [PATCH v2 07/10] libxl: event: poller pipe optimisation

2020-01-13 Thread Ian Jackson
Track in userland whether the poller pipe is nonempty.  This saves us
writing many many bytes to the pipe if nothing ever reads them.

This is going to be relevant in a moment, where we are going to create
a situation where this will happen quite a lot.

Signed-off-by: Ian Jackson 

squash! libxl: event: poller pipe optimisation
---
 tools/libxl/libxl_event.c| 3 +++
 tools/libxl/libxl_internal.h | 1 +
 2 files changed, 4 insertions(+)

diff --git a/tools/libxl/libxl_event.c b/tools/libxl/libxl_event.c
index b50d4e5074..3e76fa5af5 100644
--- a/tools/libxl/libxl_event.c
+++ b/tools/libxl/libxl_event.c
@@ -1417,6 +1417,7 @@ static void afterpoll_internal(libxl__egc *egc, 
libxl__poller *poller,
 }
 
 if (afterpoll_check_fd(poller,fds,nfds, poller->wakeup_pipe[0],POLLIN)) {
+poller->pipe_nonempty = 0;
 int e = libxl__self_pipe_eatall(poller->wakeup_pipe[0]);
 if (e) LIBXL__EVENT_DISASTER(gc, "read wakeup", e, 0);
 }
@@ -1809,6 +1810,8 @@ void libxl__poller_put(libxl_ctx *ctx, libxl__poller *p)
 
 void libxl__poller_wakeup(libxl__gc *gc, libxl__poller *p)
 {
+if (p->pipe_nonempty) return;
+p->pipe_nonempty = 1;
 int e = libxl__self_pipe_wakeup(p->wakeup_pipe[1]);
 if (e) LIBXL__EVENT_DISASTER(gc, "cannot poke watch pipe", e, 0);
 }
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index eec4bf767d..0ab324102b 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -625,6 +625,7 @@ struct libxl__poller {
 int (*fd_rindices)[3]; /* see libxl_event.c:beforepoll_internal */
 
 int wakeup_pipe[2]; /* 0 means no fd allocated */
+bool pipe_nonempty;
 
 /*
  * We also use the poller to record whether any fds have been
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 05/10] libxl: event: Make libxl__poller_wakeup take a gc, not an egc

2020-01-13 Thread Ian Jackson
We are going to want to call this in the following situation:

 * We have just set up an ao, which is to call back - so a
   non-synchronous one.  It ought not to call the application
   back right away, so no egc.

 * There is a libxl thread blocking somewhere but it is using
   using an out of date fd or timeout set, which does not take into
   account the ao we have just started.

 * We try to wake that thread up, but libxl__poller_wakeup fails.

Signed-off-by: Ian Jackson 
---
v2: New patch
---
 tools/libxl/libxl_event.c| 7 +++
 tools/libxl/libxl_internal.h | 2 +-
 2 files changed, 4 insertions(+), 5 deletions(-)

diff --git a/tools/libxl/libxl_event.c b/tools/libxl/libxl_event.c
index 16e6786889..268a5da120 100644
--- a/tools/libxl/libxl_event.c
+++ b/tools/libxl/libxl_event.c
@@ -1477,7 +1477,7 @@ void libxl__event_occurred(libxl__egc *egc, libxl_event 
*event)
 libxl__poller *poller;
 LIBXL_TAILQ_INSERT_TAIL(>occurred, event, link);
 LIBXL_LIST_FOREACH(poller, >pollers_event, entry)
-libxl__poller_wakeup(egc, poller);
+libxl__poller_wakeup(gc, poller);
 }
 }
 
@@ -1668,9 +1668,8 @@ void libxl__poller_put(libxl_ctx *ctx, libxl__poller *p)
 LIBXL_LIST_INSERT_HEAD(>pollers_idle, p, entry);
 }
 
-void libxl__poller_wakeup(libxl__egc *egc, libxl__poller *p)
+void libxl__poller_wakeup(libxl__gc *gc, libxl__poller *p)
 {
-EGC_GC;
 int e = libxl__self_pipe_wakeup(p->wakeup_pipe[1]);
 if (e) LIBXL__EVENT_DISASTER(gc, "cannot poke watch pipe", e, 0);
 }
@@ -1924,7 +1923,7 @@ void libxl__ao_complete_check_progress_reports(libxl__egc 
*egc, libxl__ao *ao)
 assert(ao->in_initiator);
 if (!ao->constructing)
 /* don't bother with this if we're not in the event loop */
-libxl__poller_wakeup(egc, ao->poller);
+libxl__poller_wakeup(gc, ao->poller);
 } else if (ao->how.callback) {
 LOG(DEBUG, "ao %p: complete for callback", ao);
 LIBXL_TAILQ_INSERT_TAIL(>aos_for_callback, ao, 
entry_for_callback);
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 328ecf3e1e..b68ab218b6 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -1311,7 +1311,7 @@ _hidden void libxl__poller_put(libxl_ctx*, libxl__poller 
*p /* may be NULL */);
 
 /* Notifies whoever is polling using p that they should wake up.
  * ctx must be locked. */
-_hidden void libxl__poller_wakeup(libxl__egc *egc, libxl__poller *p);
+_hidden void libxl__poller_wakeup(libxl__gc *egc, libxl__poller *p);
 
 /* Internal to fork and child reaping machinery */
 extern const libxl_childproc_hooks libxl__childproc_default_hooks;
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 06/10] libxl: event: Fix hang when mixing blocking and eventy calls

2020-01-13 Thread Ian Jackson
If the application calls libxl with ao_how==0 and also makes calls
like _occurred, libxl will sometimes get stuck.

The bug happens as follows (for example):

  Thread A
   libxl_do_thing(,ao_how==0)
   libxl_do_thing starts, sets up some callbacks
   libxl_do_thing exit path calls AO_INPROGRESS
   libxl__ao_inprogress goes into event loop
   eventloop_iteration sleeps on:
  - do_thing's current fd set
  - sigchld pipe if applicable
  - its poller

  Thread B
   libxl_something_occurred
   the something is to do with do_thing, above
   do_thing_next_callback does some more work
   do_thing_next_callback becomes interested in fd N
   thread B returns to application

Note that nothing wakes up thread A.  A is not listening on fd N.  So
do_thing_* will not spot when fd N signals.  do_thing will not make
further timely progress.  If there is no timeout thread A will never
wake up.

The problem here occurs because thread A is waiting on an out of date
osevent set.

There is also the possibility that a thread might block waiting for
libxl osevents but outside libxl, eg if the application used
libxl_osevent_beforepoll.  We will deal with that in a moment.

See the big comment in libxl_event.c for a fairly formal correctness
argument.

This depends on libxl__egc_ao_cleanup_1_baton being called everywhere
an egc or ao is disposed of.  Firstly egcs: in this patch we rename
libxl__egc_cleanup, which means we catch all the disposal sites.
Secondly aos: these are disposed of by (i) AO_CREATE_FAIL
(ii) ao__inprogress and (iii) an event which completes the ao later.
(i) and (ii) we handle by adding the call to _baton.  In the case of
(iii) any such function must be an event-generating function so it has
an egc too, so it will pass on the baton when the egc is disposed.

Reported-by: George Dunlap 
Signed-off-by: Ian Jackson 
---
v2: Call libxl__egc_ao_cleanup_1_baton (renamed from __egc_cleanup) on
all exits from ao_inprogress, even requests for async processing.
Fixes a remaining instance of this bug (!)
This involves disposing of ao->poller somewhat earlier.

v2: New correctness arguments in libxl_event.c comment and
in commit message.
---
 tools/libxl/libxl_event.c| 178 ---
 tools/libxl/libxl_internal.h |  33 ++--
 2 files changed, 194 insertions(+), 17 deletions(-)

diff --git a/tools/libxl/libxl_event.c b/tools/libxl/libxl_event.c
index 268a5da120..b50d4e5074 100644
--- a/tools/libxl/libxl_event.c
+++ b/tools/libxl/libxl_event.c
@@ -37,6 +37,140 @@ static void ao__check_destroy(libxl_ctx *ctx, libxl__ao 
*ao);
 
 
 /*
+ * osevent update baton handling
+ *
+ * We need the following property (the "unstale liveness property"):
+ *
+ * Whenever any thread is blocking in the libxl event loop[1], at
+ * least one thread must be using an up to date osevent set.  It is OK
+ * for all but one threads to have stale event sets, because so long
+ * as one waiting thread has the right event set, any actually
+ * interesting event will, if nothing else, wake that "right" thread
+ * up.  It will then make some progress and/or, if it exits, ensure
+ * that some other thread becomes the "right" thread.
+ *
+ * [1] TODO: Right now we are considering only the libxl event loop.
+ * We need to consider application event loop outside libxl too.
+ *
+ * Argument that our approach is sound:
+ *
+ * The issue we are concerned about is libxl sleeping on an out of
+ * date fd set, or too long a timeout, so that it doesn't make
+ * progress.  If the property above is satisfied, then if any thread
+ * is waiting in libxl at least one such thread will be waiting on a
+ * sufficient osevent set, so any relevant osevent will wake up a
+ * libxl thread which will either handle the event, or arrange that at
+ * least one other libxl thread has the right set.
+ *
+ * There are two calls to poll in libxl: one is the fd recheck, which
+ * is not blocking.  There is only the one blocking call, in
+ * eventloop_iteration.  poll runs with the ctx unlocked, so osevents
+ * might be added after it unlocks the ctx - that is what we are
+ * worried about.
+ *
+ * To demonstrate that the unstale liveness property is satisfied:
+ *
+ * We define a baton holder as follows: a libxl thread is a baton
+ * holder if
+ *   (a) it has an egc or an ao and holds the ctx lock, or
+ *   (b) it has an active non-app poller and no osevents have been
+ *   added since it released the lock, or
+ *   (c) it has an active non-app poller which has been woken
+ *   (by writing to its pipe), so it will not sleep
+ * We will maintain the invariant (the "baton invariant") that
+ * whenever there is any active poller, there is at least
+ * one baton holder.  ("non-app" means simply "not poller_app".)
+ *
+ * No thread outside libxl can have an active non-app poller: pollers
+ * are put on the active list by poller_get which is called in three
+ * 

[Xen-devel] [PATCH v2 00/10] libxl: event: Fix hang for some applications

2020-01-13 Thread Ian Jackson
The meat here, including a description of the bug, is in:
  libxl: event: Fix hang when mixing blocking and eventy calls

Re v1 I wrote:
  I suggest we try to convince ourselves of its correctness
  via a second round of code review.

I put this into practice by writing an informal proof of correctness.
This found a bug, the fixing of which was not entirely trivial.

George tells me he tested v1 of this series.  As with v1, I have
compiled this v2 but not executed it.

Ian Jackson (10):
  libxl: event: Rename poller.fds_changed to .fds_deregistered
  libxl: event: Rename ctx.pollers_fd_changed to .pollers_active
  libxl: event: Introduce CTX_UNLOCK_EGC_FREE
  libxl: event: Make LIBXL__EVENT_DISASTER take a gc, not an egc
  libxl: event: Make libxl__poller_wakeup take a gc, not an egc
  libxl: event: Fix hang when mixing blocking and eventy calls
  libxl: event: poller pipe optimisation
  libxl: event: Break out baton_wake
  libxl: event: Fix possible hang with libxl_osevent_beforepoll
  libxl: event: Move poller pipe emptying to the end of afterpoll

 tools/libxl/libxl.c  |   4 +-
 tools/libxl/libxl_aoutils.c  |   2 +-
 tools/libxl/libxl_disk.c |   4 +-
 tools/libxl/libxl_domain.c   |   2 +-
 tools/libxl/libxl_event.c| 286 +++
 tools/libxl/libxl_fork.c |  17 ++-
 tools/libxl/libxl_internal.h |  54 +---
 7 files changed, 290 insertions(+), 79 deletions(-)

-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 01/10] libxl: event: Rename poller.fds_changed to .fds_deregistered

2020-01-13 Thread Ian Jackson
This is only for deregistration.  We are going to add another variable
for new events, with different semantics, and this overly-general name
will become confusing.

Signed-off-by: Ian Jackson 
---
 tools/libxl/libxl_event.c| 8 
 tools/libxl/libxl_internal.h | 6 +++---
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/tools/libxl/libxl_event.c b/tools/libxl/libxl_event.c
index aa8b7d1945..1210c1bfb3 100644
--- a/tools/libxl/libxl_event.c
+++ b/tools/libxl/libxl_event.c
@@ -239,7 +239,7 @@ void libxl__ev_fd_deregister(libxl__gc *gc, libxl__ev_fd 
*ev)
 ev->fd = -1;
 
 LIBXL_LIST_FOREACH(poller, >pollers_fds_changed, fds_changed_entry)
-poller->fds_changed = 1;
+poller->fds_deregistered = 1;
 
  out:
 CTX_UNLOCK;
@@ -1120,7 +1120,7 @@ static int beforepoll_internal(libxl__gc *gc, 
libxl__poller *poller,
 
 *nfds_io = used;
 
-poller->fds_changed = 0;
+poller->fds_deregistered = 0;
 
 libxl__ev_time *etime = LIBXL_TAILQ_FIRST(>etimes);
 if (etime) {
@@ -1186,7 +1186,7 @@ static int afterpoll_check_fd(libxl__poller *poller,
 /* again, stale slot entry */
 continue;
 
-assert(poller->fds_changed || !(fds[slot].revents & POLLNVAL));
+assert(poller->fds_deregistered || !(fds[slot].revents & POLLNVAL));
 
 /* we mask in case requested events have changed */
 int slot_revents = fds[slot].revents & events;
@@ -1626,7 +1626,7 @@ int libxl__poller_init(libxl__gc *gc, libxl__poller *p)
 int rc;
 p->fd_polls = 0;
 p->fd_rindices = 0;
-p->fds_changed = 0;
+p->fds_deregistered = 0;
 
 rc = libxl__pipe_nonblock(CTX, p->wakeup_pipe);
 if (rc) goto out;
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index ba8c9b41ab..c5b71d15f0 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -629,14 +629,14 @@ struct libxl__poller {
 /*
  * We also use the poller to record whether any fds have been
  * deregistered since we entered poll.  Each poller which is not
- * idle is on the list pollers_fds_changed.  fds_changed is
+ * idle is on the list pollers_fds_changed.  fds_deregistered is
  * cleared by beforepoll, and tested by afterpoll.  Whenever an fd
- * event is deregistered, we set the fds_changed of all non-idle
+ * event is deregistered, we set the fds_deregistered of all non-idle
  * pollers.  So afterpoll can tell whether any POLLNVAL is
  * plausibly due to an fd being closed and reopened.
  */
 LIBXL_LIST_ENTRY(libxl__poller) fds_changed_entry;
-bool fds_changed;
+bool fds_deregistered;
 };
 
 struct libxl__gc {
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 10/10] libxl: event: Move poller pipe emptying to the end of afterpoll

2020-01-13 Thread Ian Jackson
If a timer event callback causes this poller to be woken (not very
unlikely) we would go round the poll loop twice rather than once.

Do the poller pipe emptying at the end; this is slightly more
efficient because it can't cause any callbacks, so it happens after
all the callbacks have been run.

(This pipe-emptying has to happen in afterpoll rather than the
apparently more logical beforepoll, because the application calling
beforepoll doesn't constitute a promise to actually do anything.)

Signed-off-by: Ian Jackson 
---
 tools/libxl/libxl_event.c | 12 ++--
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/tools/libxl/libxl_event.c b/tools/libxl/libxl_event.c
index 5f6a607d80..7c5387e94f 100644
--- a/tools/libxl/libxl_event.c
+++ b/tools/libxl/libxl_event.c
@@ -1453,12 +1453,6 @@ static void afterpoll_internal(libxl__egc *egc, 
libxl__poller *poller,
 fd_occurs(egc, efd, revents);
 }
 
-if (afterpoll_check_fd(poller,fds,nfds, poller->wakeup_pipe[0],POLLIN)) {
-poller->pipe_nonempty = 0;
-int e = libxl__self_pipe_eatall(poller->wakeup_pipe[0]);
-if (e) LIBXL__EVENT_DISASTER(gc, "read wakeup", e, 0);
-}
-
 for (;;) {
 libxl__ev_time *etime = LIBXL_TAILQ_FIRST(>etimes);
 if (!etime)
@@ -1473,6 +1467,12 @@ static void afterpoll_internal(libxl__egc *egc, 
libxl__poller *poller,
 
 time_occurs(egc, etime, ERROR_TIMEDOUT);
 }
+
+if (afterpoll_check_fd(poller,fds,nfds, poller->wakeup_pipe[0],POLLIN)) {
+poller->pipe_nonempty = 0;
+int e = libxl__self_pipe_eatall(poller->wakeup_pipe[0]);
+if (e) LIBXL__EVENT_DISASTER(gc, "read wakeup", e, 0);
+}
 }
 
 void libxl_osevent_afterpoll(libxl_ctx *ctx, int nfds, const struct pollfd 
*fds,
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 4/6] libxl: allow creation of domains with a specified or random domid

2020-01-13 Thread Durrant, Paul
> -Original Message-
> From: jandr...@gmail.com 
> Sent: 13 January 2020 16:16
> To: Durrant, Paul 
> Cc: xen-devel ; Anthony PERARD
> ; Ian Jackson ; Wei
> Liu 
> Subject: Re: [Xen-devel] [PATCH v2 4/6] libxl: allow creation of domains
> with a specified or random domid
> 
> On Thu, Jan 9, 2020 at 6:50 AM Paul Durrant  wrote:
> >
> > This patch adds a 'domid' field to libxl_domain_create_info and then
> > modifies do_domain_create() to use that value if it is valid. Any valid
> > domid will be checked against the retired domid list before being passed
> > to libxl__domain_make().
> > If the domid value is invalid then Xen will choose the domid, as before,
> > unless the value is the new special RANDOM_DOMID value added to the API.
> > This value instructs libxl__domain_make() to select a random domid
> value,
> > check it for validity, verify it does not match a retired domain, and
> then
> > pass it to Xen's XEN_DOMCTL_createdomain operation. If Xen determines
> that
> > it co-incides with an existing domain, a new random value will be
> > selected and the operation will be re-tried.
> >
> > NOTE: libxl__logv() is also modified to only log valid domid values in
> >   messages rather than any domid, valid or otherwise, that is not
> >   INVALID_DOMID.
> >
> > Signed-off-by: Paul Durrant 
> > ---
> > Cc: Ian Jackson 
> > Cc: Wei Liu 
> > Cc: Anthony PERARD 
> >
> > v2:
> >  - Re-worked to use a value from libxl_domain_create_info
> > ---
> >  tools/libxl/libxl.h  |  9 +
> >  tools/libxl/libxl_create.c   | 32 +++-
> >  tools/libxl/libxl_internal.c |  2 +-
> >  tools/libxl/libxl_types.idl  |  1 +
> >  4 files changed, 42 insertions(+), 2 deletions(-)
> >
> 
> 
> 
> > diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
> > index 1835a5502c..ee76dee364 100644
> > --- a/tools/libxl/libxl_create.c
> > +++ b/tools/libxl/libxl_create.c
> > @@ -600,9 +600,39 @@ int libxl__domain_make(libxl__gc *gc,
> libxl_domain_config *d_config,
> >  goto out;
> >  }
> >
> > -ret = xc_domain_create(ctx->xch, domid, );
> > +if (libxl_domid_valid_guest(info->domid)) {
> > +*domid = info->domid;
> > +
> > +if (libxl__is_retired_domid(gc, *domid)) {
> > +LOGED(ERROR, *domid, "domain id is retired");
> > +rc = ERROR_FAIL;
> > +goto out;
> > +}
> > +} else if (info->domid == RANDOM_DOMID) {
> > +*domid = 0; /* Zero-out initial value */
> > +}
> > +
> > +for (;;) {
> > +if (info->domid == RANDOM_DOMID) {
> > +/* Randomize lower order bytes */
> > +ret = libxl__random_bytes(gc, (void *)domid,
> > +  sizeof(uint16_t));
> 
> Casting to void * assumes little endian.

I think that's a fairly safe assumption as far as Xen goes...

> Using a temporary uint16_t

...but, yes, that might be neater.

> would avoid that assumption.  Also, masking down to 0x7fff would clear
> the top bit which is never valid.

That seems like a bit of a layering violation and the check in 
libxl_domid_valid_guest() is going to cause a pretty fast turn round the loop 
if the top bit is set so masking is not going to gain that much.

  Paul

> 
> Regards,
> Jason
> 
> > +if (ret < 0)
> > +break;
> > +
> > +if (!libxl_domid_valid_guest(*domid) ||
> > +libxl__is_retired_domid(gc, *domid))
> > +continue;
> > +}
> > +
> > +ret = xc_domain_create(ctx->xch, domid, );
> > +if (ret == 0 || errno != EEXIST || info->domid !=
> RANDOM_DOMID)
> > +break;
> > +}
> > +
> >  if (ret < 0) {
> >  LOGED(ERROR, *domid, "domain creation fail");
> > +*domid = INVALID_DOMID;
> >  rc = ERROR_FAIL;
> >  goto out;
> >  }
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] EFI development issues

2020-01-13 Thread Jan Beulich
On 13.01.2020 17:02, Andrew Cooper wrote:
> My recent boot pagetable changes have caused me to work with the EFI
> build of Xen rather more than previously.
> 
> First, there is a dependency tracking bug in the build system.  Edits to
> xen/arch/x86/efi/efi-boot.h don't cause xen.efi to be regenerated.  From
> what I can tell, the file doesn't even get recompiled, because syntax
> errors even go unnoticed.

Not an issue here, I've just now tried it out. .boot.o.d also
correctly names the file here.

> Second, and the main point of the email.
> 
> The EFI code has some logic which does:
> 
> if ( !base_video )
> {
>     ...
> 
>     if ( best != StdOut->Mode->Mode )
>         StdOut->SetMode(StdOut, best);
> }
> 
> just before printing out the Xen banner.  This has a side effect of
> causing all further use of StdOut/StdErr to cease working, and
> interfering completely with debugging activities.

Interesting, and certainly unintended. Obviously the "normal" output
works (for me at least, with or without serial console, albeit I
don't think I've ever tried headless, i.e. _just_ a serial console),
so it's not exactly clear to me what other "debugging activities" it
may interfere with. A broken StdOut / StdErr protocol implementation
in the firmware?

>  (Waiting for a
> keypress on StdIn however does work, which is how I eventually diagnosed
> that it was an output problem.)  Skipping this logic allows debugging to
> work.

As should then do -basevideo.

> The code appeared in bf6501a62 "x86-64: EFI boot code" and has no
> specific description of what it is doing and/or trying to achieve.

efi_console_set_mode() is simple enough I think: It tries to maximize
screen dimensions. (There's some historical context here, as the
code wasn't written from scratch: Serial consoles often weren't
available when colleagues of mine and I did some of the original EFI
enabling work for a long canceled project. Plus there we had a rather
better (tm) kernel debugger, wanting to utilize as high resolution a
screen as possible to show as much useful information as possible at
any point in time.)

> It is also not entirely clear why it is gated on having a cfg file in
> the first place (c/s ,c38cf865ec8, not that there is adequate context
> for why)

The description of the cited commit is clear enough, isn't it?
This is just the same distinction (just placed differently) for
Arm as that between efi_start() (doing most of this stuff) and
efi_multiboot2() (not doing so, in particular the command line
parsing, and e.g. not even providing a means to suppress the
call to efi_console_set_mode()).

For anything beyond this I have to defer to the Arm folks, who
wanted it this way.

> or why there is a Xen command line argument "-basevideo"
> introduced in the beginning to skip this behaviour.

Traditionally video mode setting had its problems, hence we
anticipated there may be problems also with EFI doing so. As a
result we wanted, from the very beginning, a simply means to
get past any such.

> As a point of reference, I don't see Linux making any SetMode calls.

Well, if I'm not mistaken Xen's support for booting as an EFI
application predates Linux'es by quite a bit, so there was nothing
to reference there. As said, the origin of this code is elsewhere.

> What is the purpose of changing to a different mode?  Certainly as far
> as serial consoles go, sticking with the mode the loader uses certainly
> feels like a safer option.

Does a serial console report a "resolution" in the first place? And
if we were able to (sufficiently easily) tell video from serial
console, how would we deal with the case of StdOut / StdErr being
multiplexed to both?

Jan

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [qemu-mainline test] 146049: regressions - FAIL

2020-01-13 Thread osstest service owner
flight 146049 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146049/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm   6 xen-buildfail REGR. vs. 144861
 build-arm64   6 xen-buildfail REGR. vs. 144861
 build-amd64   6 xen-buildfail REGR. vs. 144861
 build-i386-xsm6 xen-buildfail REGR. vs. 144861
 build-armhf   6 xen-buildfail REGR. vs. 144861
 build-amd64-xsm   6 xen-buildfail REGR. vs. 144861
 build-i3866 xen-buildfail REGR. vs. 144861

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)  blocked n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvshim1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-rtds  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1) blocked n/a
 test-arm64-arm64-xl-seattle   1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-i386-xl-pvshim 1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-shadow1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)   blocked  n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 

[Xen-devel] [ovmf test] 146047: regressions - FAIL

2020-01-13 Thread osstest service owner
flight 146047 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146047/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail REGR. vs. 145767
 test-amd64-amd64-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail REGR. vs. 
145767

version targeted for testing:
 ovmf 4465cd124fbcf5490faad6a1a834299b30b5d009
baseline version:
 ovmf 70911f1f4aee0366b6122f2b90d367ec0f066beb

Last test of basis   145767  2020-01-08 00:39:09 Z5 days
Failing since145774  2020-01-08 02:50:20 Z5 days   31 attempts
Testing same since   146041  2020-01-13 04:39:23 Z0 days3 attempts


People who touched revisions under test:
  Albecki, Mateusz 
  Ard Biesheuvel 
  Ashish Singhal 
  Eric Dong 
  Fan, ZhijuX 
  Jason Voelz 
  Laszlo Ersek 
  Mateusz Albecki 
  Pavana.K 
  Philippe Mathieu-Daud? 
  Philippe Mathieu-Daude 
  Siyuan Fu 
  Siyuan, Fu 
  Vitaly Cheptsov 
  Vitaly Cheptsov via Groups.Io 
  Zhiju.Fan 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 fail
 test-amd64-i386-xl-qemuu-ovmf-amd64  fail



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 422 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 4/6] libxl: allow creation of domains with a specified or random domid

2020-01-13 Thread Jason Andryuk
On Thu, Jan 9, 2020 at 6:50 AM Paul Durrant  wrote:
>
> This patch adds a 'domid' field to libxl_domain_create_info and then
> modifies do_domain_create() to use that value if it is valid. Any valid
> domid will be checked against the retired domid list before being passed
> to libxl__domain_make().
> If the domid value is invalid then Xen will choose the domid, as before,
> unless the value is the new special RANDOM_DOMID value added to the API.
> This value instructs libxl__domain_make() to select a random domid value,
> check it for validity, verify it does not match a retired domain, and then
> pass it to Xen's XEN_DOMCTL_createdomain operation. If Xen determines that
> it co-incides with an existing domain, a new random value will be
> selected and the operation will be re-tried.
>
> NOTE: libxl__logv() is also modified to only log valid domid values in
>   messages rather than any domid, valid or otherwise, that is not
>   INVALID_DOMID.
>
> Signed-off-by: Paul Durrant 
> ---
> Cc: Ian Jackson 
> Cc: Wei Liu 
> Cc: Anthony PERARD 
>
> v2:
>  - Re-worked to use a value from libxl_domain_create_info
> ---
>  tools/libxl/libxl.h  |  9 +
>  tools/libxl/libxl_create.c   | 32 +++-
>  tools/libxl/libxl_internal.c |  2 +-
>  tools/libxl/libxl_types.idl  |  1 +
>  4 files changed, 42 insertions(+), 2 deletions(-)
>



> diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
> index 1835a5502c..ee76dee364 100644
> --- a/tools/libxl/libxl_create.c
> +++ b/tools/libxl/libxl_create.c
> @@ -600,9 +600,39 @@ int libxl__domain_make(libxl__gc *gc, 
> libxl_domain_config *d_config,
>  goto out;
>  }
>
> -ret = xc_domain_create(ctx->xch, domid, );
> +if (libxl_domid_valid_guest(info->domid)) {
> +*domid = info->domid;
> +
> +if (libxl__is_retired_domid(gc, *domid)) {
> +LOGED(ERROR, *domid, "domain id is retired");
> +rc = ERROR_FAIL;
> +goto out;
> +}
> +} else if (info->domid == RANDOM_DOMID) {
> +*domid = 0; /* Zero-out initial value */
> +}
> +
> +for (;;) {
> +if (info->domid == RANDOM_DOMID) {
> +/* Randomize lower order bytes */
> +ret = libxl__random_bytes(gc, (void *)domid,
> +  sizeof(uint16_t));

Casting to void * assumes little endian.  Using a temporary uint16_t
would avoid that assumption.  Also, masking down to 0x7fff would clear
the top bit which is never valid.

Regards,
Jason

> +if (ret < 0)
> +break;
> +
> +if (!libxl_domid_valid_guest(*domid) ||
> +libxl__is_retired_domid(gc, *domid))
> +continue;
> +}
> +
> +ret = xc_domain_create(ctx->xch, domid, );
> +if (ret == 0 || errno != EEXIST || info->domid != RANDOM_DOMID)
> +break;
> +}
> +
>  if (ret < 0) {
>  LOGED(ERROR, *domid, "domain creation fail");
> +*domid = INVALID_DOMID;
>  rc = ERROR_FAIL;
>  goto out;
>  }

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-unstable-smoke test] 146048: tolerable all pass - PUSHED

2020-01-13 Thread osstest service owner
flight 146048 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146048/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  03bfe526ecadc86f31eda433b91dc90be0563919
baseline version:
 xen  8842d01b300919e20bca2e1138c458a8483600f8

Last test of basis   145992  2020-01-11 13:01:43 Z2 days
Testing same since   146048  2020-01-13 14:00:30 Z0 days1 attempts


People who touched revisions under test:
  George Dunlap 
  Jan Beulich 
  Julien Grall 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   8842d01b30..03bfe526ec  03bfe526ecadc86f31eda433b91dc90be0563919 -> smoke

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] EFI development issues

2020-01-13 Thread Andrew Cooper
Hello,

My recent boot pagetable changes have caused me to work with the EFI
build of Xen rather more than previously.

First, there is a dependency tracking bug in the build system.  Edits to
xen/arch/x86/efi/efi-boot.h don't cause xen.efi to be regenerated.  From
what I can tell, the file doesn't even get recompiled, because syntax
errors even go unnoticed.

Second, and the main point of the email.

The EFI code has some logic which does:

if ( !base_video )
{
    ...

    if ( best != StdOut->Mode->Mode )
        StdOut->SetMode(StdOut, best);
}

just before printing out the Xen banner.  This has a side effect of
causing all further use of StdOut/StdErr to cease working, and
interfering completely with debugging activities.  (Waiting for a
keypress on StdIn however does work, which is how I eventually diagnosed
that it was an output problem.)  Skipping this logic allows debugging to
work.

The code appeared in bf6501a62 "x86-64: EFI boot code" and has no
specific description of what it is doing and/or trying to achieve.

It is also not entirely clear why it is gated on having a cfg file in
the first place (c/s ,c38cf865ec8, not that there is adequate context
for why) or why there is a Xen command line argument "-basevideo"
introduced in the beginning to skip this behaviour.

As a point of reference, I don't see Linux making any SetMode calls.

What is the purpose of changing to a different mode?  Certainly as far
as serial consoles go, sticking with the mode the loader uses certainly
feels like a safer option.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2] Introduce CHANGELOG.md

2020-01-13 Thread Paul Durrant
As agreed during the 2020-01 community call [1] this patch introduces a
changelog, based on the principles explained at keepachangelog.com [2].
A new MAINTAINERS entry is also added, with myself as (currently sole)
maintainer.

[1] See C.2 at https://cryptpad.fr/pad/#/2/pad/edit/ERZtMYD5j6k0sv-NG6Htl-AJ/
[2] https://keepachangelog.com/en/1.0.0/

Signed-off-by: Paul Durrant 
Acked-by: Lars Kurth 
---
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Jan Beulich 
Cc: Julien Grall 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Wei Liu 

v2:
 - Dropped 'All' from 'All notable changes'
 - Added Lars as a designated reviewer
---
 CHANGELOG.md | 14 ++
 MAINTAINERS  |  6 ++
 2 files changed, 20 insertions(+)
 create mode 100644 CHANGELOG.md

diff --git a/CHANGELOG.md b/CHANGELOG.md
new file mode 100644
index 00..b11e9bc4e3
--- /dev/null
+++ b/CHANGELOG.md
@@ -0,0 +1,14 @@
+# Changelog
+
+Notable changes to Xen will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
+
+## [Unreleased](https://xenbits.xen.org/gitweb/?p=xen.git;a=shortlog)
+
+### Added
+ - This file and MAINTAINERS entry.
+
+## 
[4.13.0](https://xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=RELEASE-4.13.0) 
- 2019-12-17
+
+> Pointer to release from which CHANGELOG tracking starts
diff --git a/MAINTAINERS b/MAINTAINERS
index d5bd83073c..1ffc3dc600 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -198,6 +198,12 @@ F: xen/include/asm-arm/
 F: xen/include/public/arch-arm/
 F: xen/include/public/arch-arm.h
 
+Change Log
+M: Paul Durrant 
+R: Lars Kurth 
+S: Maintained
+F: CHANGELOG.md
+
 Continuous Integration (CI)
 M: Doug Goldstein 
 W: https://gitlab.com/xen-project/xen
-- 
2.17.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v4] MAINTAINERS: Add explicit check-in policy section

2020-01-13 Thread Jan Beulich
On 13.01.2020 16:04, George Dunlap wrote:
> The "nesting" section in the MAINTAINERS file was not initially
> intended to describe the check-in policy for patches, but only how
> nesting worked; but since there was no check-in policy, it has been
> acting as a de-facto policy.
> 
> One problem with this is that the policy is not complete: It doesn't
> cover open objections, time to check-in, or so on.  The other problem
> with the policy is that, as written, it doesn't account for
> maintainers submitting patches to files which they themselves
> maintain.  This is fine for situations where there are are multiple
> maintainers, but not for situations where there is only one
> maintainer.
> 
> Add an explicit "Check-in policy" section to the MAINTAINERS document
> to serve as the canonical reference for the check-in policy.  Move
> paragraphs not explicitly related to nesting into it.
> 
> While here, "promote" the "The meaning of nesting" section title.
> 
> DISCUSSION
> 
> This seems to be a change from people's understanding of the current
> policy.  Most people's understanding of the current policy seems to be:
> 
> 1.  In order to get a change to a given file committed, it must have
> an Ack or Review from at least one *maintainer* of that file other
> than the submitter.
> 
> 2. In the case where a file has only one maintainer, it must have an
> Ack or Review from a "nested" maintainer.
> 
> I.e., if I submitted something to x86/mm, it would require an Ack from
> Jan or Andy, or (in exceptional circumstances) The Rest; but an Ack from
> (say) Roger or Juergen wouldn't suffice.
> 
> Let's call this the "maintainer-ack" approach (because it must have an
> ack or r-b from a maintainer to be checked in), and the proposal in
> this patch the "maintainer-approval" (since SoB from a maintainer
> indicates approval).
> 
> The core issue I have with "maintainer-ack" is that it makes the
> maintainer less privileged with regard to writing code than
> non-maintainers.  If component X has maintainers A and B, then a
> non-maintainer can have code checked in if reviewed either by A or B.
> If A or B wants code checked in, they have to wait for exactly one
> person to review it.
> 
> In fact, if B is quite busy, the easiest way for A really to get their
> code checked in might be to hand it to a non-maintainer N, and ask N
> to submit it as their own.  Then A can Ack the patches and check them
> in.
> 
> The current system, therefore, either sets up a perverse incentive (if
> you think the behavior described above is unacceptable) or unnecessary
> bureaucracy (if you think it's acceptable).  Either way I think we
> should set up our system to avoid it.
> 
> Other variations on "maintainer-ack" have been proposed:
> 
> - Allow maintainer's patches to go in with an R-b from "designated
>   reviewers"
> 
> - Allow maintainer's patches to go in with an Ack from more general
>   maintainer
> 
> Both fundamentally make it harder for maintainers to get their code in
> and/or reviewed effectively than non-maintainers, setting up the
> perverse incentive / unnecessary bureaucracy.
> 
> Signed-off-by: George Dunlap 

Acked-by: Jan Beulich 

Albeit I guess this is rather something which needs voting on.

Jan

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] MAINTAINERS: Add explicit check-in policy section

2020-01-13 Thread George Dunlap
On 1/7/20 4:44 PM, Jan Beulich wrote:
> On 07.01.2020 17:17, George Dunlap wrote:
>> On 1/7/20 1:05 PM, Jan Beulich wrote:
>> 2. It must have either a an Acked-by from a maintainer, or a
>>Reviewed-by.  This must come from someone other than the submitter.
> 
> Better, but leaving ambiguous whether "maintainer" means "any one"
> or "of the code being touched". I think you mean the former, in
> which case I'd prefer to see it amended along the lines of "...
> from a maintainer (of any component), or ...". Or possibly you
> mean any maintainer up the "nesting" chain, in which case the
> wording would need to be yet different?

I've tried to reword this to make it more clear (see v4).  Just in
general, though, it would be helpful if when you found some wording
insufficient, if you tried to craft something you thought was better.
Even if I don't use it, it gives me a clearer idea the direction you'd
like to go in.

Thanks,
 -George

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v4] MAINTAINERS: Add explicit check-in policy section

2020-01-13 Thread George Dunlap
The "nesting" section in the MAINTAINERS file was not initially
intended to describe the check-in policy for patches, but only how
nesting worked; but since there was no check-in policy, it has been
acting as a de-facto policy.

One problem with this is that the policy is not complete: It doesn't
cover open objections, time to check-in, or so on.  The other problem
with the policy is that, as written, it doesn't account for
maintainers submitting patches to files which they themselves
maintain.  This is fine for situations where there are are multiple
maintainers, but not for situations where there is only one
maintainer.

Add an explicit "Check-in policy" section to the MAINTAINERS document
to serve as the canonical reference for the check-in policy.  Move
paragraphs not explicitly related to nesting into it.

While here, "promote" the "The meaning of nesting" section title.

DISCUSSION

This seems to be a change from people's understanding of the current
policy.  Most people's understanding of the current policy seems to be:

1.  In order to get a change to a given file committed, it must have
an Ack or Review from at least one *maintainer* of that file other
than the submitter.

2. In the case where a file has only one maintainer, it must have an
Ack or Review from a "nested" maintainer.

I.e., if I submitted something to x86/mm, it would require an Ack from
Jan or Andy, or (in exceptional circumstances) The Rest; but an Ack from
(say) Roger or Juergen wouldn't suffice.

Let's call this the "maintainer-ack" approach (because it must have an
ack or r-b from a maintainer to be checked in), and the proposal in
this patch the "maintainer-approval" (since SoB from a maintainer
indicates approval).

The core issue I have with "maintainer-ack" is that it makes the
maintainer less privileged with regard to writing code than
non-maintainers.  If component X has maintainers A and B, then a
non-maintainer can have code checked in if reviewed either by A or B.
If A or B wants code checked in, they have to wait for exactly one
person to review it.

In fact, if B is quite busy, the easiest way for A really to get their
code checked in might be to hand it to a non-maintainer N, and ask N
to submit it as their own.  Then A can Ack the patches and check them
in.

The current system, therefore, either sets up a perverse incentive (if
you think the behavior described above is unacceptable) or unnecessary
bureaucracy (if you think it's acceptable).  Either way I think we
should set up our system to avoid it.

Other variations on "maintainer-ack" have been proposed:

- Allow maintainer's patches to go in with an R-b from "designated
  reviewers"

- Allow maintainer's patches to go in with an Ack from more general
  maintainer

Both fundamentally make it harder for maintainers to get their code in
and/or reviewed effectively than non-maintainers, setting up the
perverse incentive / unnecessary bureaucracy.

Signed-off-by: George Dunlap 
---
v4:
- Try to reword the "someone other than the submitter" section to be
  more clear.
- Make a distinction between "waiting sufficient time for anyone to
  respond" and "waiting a sufficient time for a co-maintainer to
  respond"
v3:
- Be more specific about kind of tag (R-b or Ack) is required from
  what kind of person ('normal' person or maintainer)
- Separate "reasonable amount of time for anyone to object in general"
  from "reasonable amount of time / notification for checking in
  without a maintainer ack".
v2:
- Modify "sufficient time" to "sufficient time and/or warning".
- Add a comment explicitly stating that there are exceptions.
- Move some of the alternate proposals into the changelog itself

CC: Ian Jackson 
CC: Wei Liu 
CC: Andrew Cooper 
CC: Jan Beulich 
CC: Tim Deegan 
CC: Konrad Wilk 
CC: Stefano Stabellini 
CC: Julien Grall 
CC: Lars Kurth 

This is a follow-up to the discussion in `[PATCH for-4.12]
passthrough/vtd: Drop the "workaround_bios_bug" logic entirely`, specifically
Message-ID: <5c9cf25a027800222...@prv1-mh.provo.novell.com>

Another approach would be to say that in the case of multiple
maintainers, the maintainers themselves can decide to mandate each
other's Ack.  For instance, Dario and I could agree that we don't need
each others' ack for changes to the scheduler, but Andy and Jan could
agree that they do need each other's Ack for changes to the x86 code.
Checks that maintainers themselves have agreed on will produce neither
perverse incentives, nor be considered "unnecessary".
---
 MAINTAINERS | 63 -
 1 file changed, 57 insertions(+), 6 deletions(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index d5bd83073c..79f665b5b2 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -104,7 +104,63 @@ Descriptions of section entries:
   xen-maintainers-
 
 
-The meaning of nesting:
+   Check-in policy
+   ===
+
+In order for a patch to be checked in, in general, several conditions
+must be 

Re: [Xen-devel] [RFC PATCH V2 11/11] x86: tsc: avoid system instability in hibernation

2020-01-13 Thread Singh, Balbir
On Mon, 2020-01-13 at 13:01 +, Andrew Cooper wrote:
> On 13/01/2020 11:43, Singh, Balbir wrote:
> > On Mon, 2020-01-13 at 11:16 +0100, Peter Zijlstra wrote:
> > > On Fri, Jan 10, 2020 at 07:35:20AM -0800, Eduardo Valentin wrote:
> > > > Hey Peter,
> > > > 
> > > > On Wed, Jan 08, 2020 at 11:50:11AM +0100, Peter Zijlstra wrote:
> > > > > On Tue, Jan 07, 2020 at 11:45:26PM +, Anchal Agarwal wrote:
> > > > > > From: Eduardo Valentin 
> > > > > > 
> > > > > > System instability are seen during resume from hibernation when
> > > > > > system
> > > > > > is under heavy CPU load. This is due to the lack of update of
> > > > > > sched
> > > > > > clock data, and the scheduler would then think that heavy CPU hog
> > > > > > tasks need more time in CPU, causing the system to freeze
> > > > > > during the unfreezing of tasks. For example, threaded irqs,
> > > > > > and kernel processes servicing network interface may be delayed
> > > > > > for several tens of seconds, causing the system to be unreachable.
> > > > > > The fix for this situation is to mark the sched clock as unstable
> > > > > > as early as possible in the resume path, leaving it unstable
> > > > > > for the duration of the resume process. This will force the
> > > > > > scheduler to attempt to align the sched clock across CPUs using
> > > > > > the delta with time of day, updating sched clock data. In a post
> > > > > > hibernation event, we can then mark the sched clock as stable
> > > > > > again, avoiding unnecessary syncs with time of day on systems
> > > > > > in which TSC is reliable.
> > > > > 
> > > > > This makes no frigging sense what so bloody ever. If the clock is
> > > > > stable, we don't care about sched_clock_data. When it is stable you
> > > > > get
> > > > > a linear function of the TSC without complicated bits on.
> > > > > 
> > > > > When it is unstable, only then do we care about the
> > > > > sched_clock_data.
> > > > > 
> > > > 
> > > > Yeah, maybe what is not clear here is that we covering for situation
> > > > where clock stability changes over time, e.g. at regular boot clock is
> > > > stable, hibernation happens, then restore happens in a non-stable
> > > > clock.
> > > 
> > > Still confused, who marks the thing unstable? The patch seems to suggest
> > > you do yourself, but it is not at all clear why.
> > > 
> > > If TSC really is unstable, then it needs to remain unstable. If the TSC
> > > really is stable then there is no point in marking is unstable.
> > > 
> > > Either way something is off, and you're not telling me what.
> > > 
> > 
> > Hi, Peter
> > 
> > For your original comment, just wanted to clarify the following:
> > 
> > 1. After hibernation, the machine can be resumed on a different but
> > compatible
> > host (these are VM images hibernated)
> > 2. This means the clock between host1 and host2 can/will be different
> 
> The guests TSC value is part of all save/migrate/resume state.  Given
> this bug, I presume you've actually discarded all register state on
> hibernate, and the TSC is starting again from 0?
> 
> The frequency of the new TSC might very likely be different, but the
> scale/offset in the paravirtual clock information should let Linux's
> view of time stay consistent.
> 

I am looking at my old dmesg logs, which I seem to have lost to revalidate,
but I think Eduardo had a different point. I should point out that I was
adding to the list of potentially missed assumptions


> > In your comments are you making the assumption that the host(s) is/are the
> > same? Just checking the assumptions being made and being on the same page
> > with
> > them.
> 
> TSCs are a massive source of "fun".  I'm not surprised that there are
> yet more bugs around.
> 
> Does anyone actually know what does/should happen to the real TSC on
> native S4?  The default course of action should be for virtualisation to
> follow suit.
> 
> ~Andrew

Balbir
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2] xsm: hide detailed Xen version from unprivileged guests

2020-01-13 Thread Julien Grall

Hi,

On 13/01/2020 12:51, George Dunlap wrote:

On 1/12/20 6:26 PM, Doug Goldstein wrote:

On 1/11/20 3:02 AM, George Dunlap wrote:

1. Block XENVER_extraversion at the hypervisor level.  Change the
xen_deny() string to "".  (This is v1 of sergey's patch.)

2. Block XENVER_extraversion at the hypervisor level.  Leave xen_deny()
as returning "", but replace "" with "" in hvmloader so
it doesn't show up in the System Info and scare users.

3. Block XENVER_extraversion at the hypervisor level.  Change xen_deny()
to return a more benign string like "".  (Perhaps also filter it
in hvmloader, just for good measure.)

4. Block XENVER_extraversion at the hypervisor level.  Make the
xen_deny() string configurable in KConfig.


A Kconfig option is indeed ideal as some of the stakeholder may want to 
keep control of the string exposed.


But if we go the Kconfig route, then maybe we want to allow each bits 
(extraversion, compiler...) to be separatly configurable.


I would be more than happy to help writing such a patch if there is an 
interest for it.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2] xsm: hide detailed Xen version from unprivileged guests

2020-01-13 Thread Andrew Cooper
On 13/01/2020 12:51, George Dunlap wrote:
> On 1/12/20 6:26 PM, Doug Goldstein wrote:
>> On 1/11/20 3:02 AM, George Dunlap wrote:
>>>
 On Jan 11, 2020, at 4:02 AM, Doug Goldstein  wrote:



 On 1/10/20 4:37 AM, Sergey Dyasli wrote:
> Hide the following information that can help identify the running Xen
> binary version: XENVER_extraversion, XENVER_compile_info,
> XENVER_changeset.
> Add explicit cases for XENVER_commandline and XENVER_build_id as well.
> Introduce xsm_filter_denied() to hvmloader to remove "" string
> from guest's DMI tables that otherwise would be shown in tools like
> dmidecode.
> Signed-off-by: Sergey Dyasli 
> ---
> v1 --> v2:
> - Added xsm_filter_denied() to hvmloader instead of modifying
> xen_deny()
 So 100% this version of the patch won't fly with the various
 downstreams that run the v1 of this patch. Those various consumers
 will stick with v1.

 If the goal of this is to reduce the burden of the downstreams and
 their customers to carry a patch against Xen then I wouldn't even
 bother with this version.
>>> If the goal is to come up with a solution that works for everyone, it
>>> would be helpful if you said *why* “various consumers” would find this
>>> patch unacceptable; and also what they might think about the alternate
>>> solutions proposed (and why).
>>>
>>>   -George
>>>
> [snip]
>
>> Now I know someone is going to read this and say "Look at Doug and him
>> advocating for security through obscurity".
> FWIW I'd be the first person to contradict them, and say you were
> practicing "defense in depth". :-)
>
>> Ultimately my point is if the goal of this patch is to upstream a patch
>> that's carried by various downstreams, why not actually listen to what
>> caused them to write the patch?
> Right, that's what I'm trying to do; but I don't seem to be making much
> progress.
>
> Here's my summary of the situation and arguments so far:
>
> 1. The xen_version hypercall can return strings for a number of
> different values, including XENVER_extraversion, which gives the point
> release and build id.
>
> 2. The XSM dummy module has code to filter which of these are allowed
> for unprivileged guests.  When access to a given value is filtered, no
> error is returned; rather, the string "" is returned.

Given an ABI which is capable of expressing -EPERM, it is perhaps worth
noting that this behaviour is because some drivers will BSOD if the call
fails...

> 3. Knowledge about the specific instantiation of Xen on which they are
> running makes it easier for attackers to know how to attack t he system;
> the XENVER_extraversion provides little value to legitimate users, but a
> lot of value to attackers.   As a defense-in-depth measure, it's
> important to be able to hide this information.
>
> 4. There's currently a patch carried by many downstreams, which changes
> the XSM dummy module to deny XENVER_extraversion to unprivileged guests.
>
> 5. However, this caused "" to show up in various user-visible
> places, which caused customer support headaches.  So this out-of-tree
> patch also replaced the string returned when denying access to ""
> instead.  Note that this is not *only* for XENVER_extraversion; with
> that patch, *any* time the value requested in xen_version is denied by
> policy, "" will be returned.
>
> 6. Silently returning an empty string is considered bad interface design
> by several developers.

Several others disagree with this claim.  Not least because it is very
common signal for "no information".

This is why several downstreams really have gotten away with using the
empty string, for products which have been in the field for years.

>   So Sergey's second patch:
>  - Still denies XENVER_extraversion at the hypervisor level
>  - Leaves the value returned by the hypervisor as ""
>  - Filters the "" string at the hvmloader level, to prevent it
> leaking into a GUI and scaring customers.

The SMBios table isn't the only way XENVER_extraversion leaks up into
the UI.

XENVER_extraversion isn't the only source of redacted information
leaking up into the UI.

Linux for example exports it all via sysfs.  The windows drivers put
XENVER_extraversion into several other logs.

>
> Now we get to Andy's objection on the 10th:
>
> ---
> The reason for this (which ought to be obvious, but I guess only to
> those who actually do customer support) is basic human physiology.
> "denied" means something has gone wrong.  It scares people, and causes
> them to seek help to change fix whatever is broken.
>
> It is not appropriate for it to find its way into the guest in the first
> place, and that includes turning up in `dmesg` and other logs, and
> expecting guest runtime to filter for it is complete nonsense.
> ---
>
> Basically, Andy says that *anywhere* it might show up is way too scary,
> even a guest dmesg log.
>
> Well, I disagree; I look in "dmesg" and I see loads of "scary" things.


[Xen-devel] [qemu-mainline test] 146046: regressions - FAIL

2020-01-13 Thread osstest service owner
flight 146046 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146046/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm   6 xen-buildfail REGR. vs. 144861
 build-arm64   6 xen-buildfail REGR. vs. 144861
 build-amd64   6 xen-buildfail REGR. vs. 144861
 build-i386-xsm6 xen-buildfail REGR. vs. 144861
 build-armhf   6 xen-buildfail REGR. vs. 144861
 build-amd64-xsm   6 xen-buildfail REGR. vs. 144861
 build-i3866 xen-buildfail REGR. vs. 144861

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-rtds  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-shadow 1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-cubietruck  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1) blocked n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)  blocked n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-seattle   1 build-check(1)   blocked  n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)  blocked n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvshim1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 

Re: [Xen-devel] [PATCH v2] xsm: hide detailed Xen version from unprivileged guests

2020-01-13 Thread Julien Grall



On 13/01/2020 14:07, George Dunlap wrote:

On 1/13/20 2:01 PM, Andrew Cooper wrote:

On 13/01/2020 13:39, Julien Grall wrote:

Hi George,

Thank you for summarising the possibility. One question below.

On 13/01/2020 12:51, George Dunlap wrote:

2. Block XENVER_extraversion at the hypervisor level.  Leave xen_deny()
as returning "", but replace "" with "" in hvmloader so
it doesn't show up in the System Info and scare users.

3. Block XENVER_extraversion at the hypervisor level.  Change xen_deny()
to return a more benign string like "".  (Perhaps also filter it
in hvmloader, just for good measure.)


My knowledge of live migration on x86 is a bit limited, but if I
understand correctly those two options would require a guest to reboot
in order to pick up the changes. Am I correct?


Not in the slightest.  The content returned changes whenever the
hypervisor changes.


I guess Julien is talking about the filtering done in hvmloader.  That
filtering is about what's in the guest's ACPI tables; and *that* happens
only once at guest boot; so whatever the scary message is in the Windows
System Information page (or wherever it is) would stay there until the
guest reboots, regardless of which option we go with.


Yes, I was speaking about the filtering done in hvmloader. Thank you 
both for the explanation.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 2/2] libxl: Add new "notify-only" childproc mode

2020-01-13 Thread Ian Jackson
George Dunlap writes ("Re: [PATCH v2 2/2] libxl: Add new "notify-only" 
childproc mode"):
> FAOD, with the fixes in your other series, I consider this patch to now
> be moot.

Right.  FTOAD, I don't think there was a problem with this patch
principle; it's just not needed now.  If someone comes up with a use
for it in the future then it can be resurrected and we should do a
detailed code review of it then.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2] xsm: hide detailed Xen version from unprivileged guests

2020-01-13 Thread George Dunlap
On 1/13/20 2:01 PM, Andrew Cooper wrote:
> On 13/01/2020 13:39, Julien Grall wrote:
>> Hi George,
>>
>> Thank you for summarising the possibility. One question below.
>>
>> On 13/01/2020 12:51, George Dunlap wrote:
>>> 2. Block XENVER_extraversion at the hypervisor level.  Leave xen_deny()
>>> as returning "", but replace "" with "" in hvmloader so
>>> it doesn't show up in the System Info and scare users.
>>>
>>> 3. Block XENVER_extraversion at the hypervisor level.  Change xen_deny()
>>> to return a more benign string like "".  (Perhaps also filter it
>>> in hvmloader, just for good measure.)
>>
>> My knowledge of live migration on x86 is a bit limited, but if I
>> understand correctly those two options would require a guest to reboot
>> in order to pick up the changes. Am I correct?
> 
> Not in the slightest.  The content returned changes whenever the
> hypervisor changes.

I guess Julien is talking about the filtering done in hvmloader.  That
filtering is about what's in the guest's ACPI tables; and *that* happens
only once at guest boot; so whatever the scary message is in the Windows
System Information page (or wherever it is) would stay there until the
guest reboots, regardless of which option we go with.

 -George

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2] xsm: hide detailed Xen version from unprivileged guests

2020-01-13 Thread Andrew Cooper
On 13/01/2020 13:39, Julien Grall wrote:
> Hi George,
>
> Thank you for summarising the possibility. One question below.
>
> On 13/01/2020 12:51, George Dunlap wrote:
>> 2. Block XENVER_extraversion at the hypervisor level.  Leave xen_deny()
>> as returning "", but replace "" with "" in hvmloader so
>> it doesn't show up in the System Info and scare users.
>>
>> 3. Block XENVER_extraversion at the hypervisor level.  Change xen_deny()
>> to return a more benign string like "".  (Perhaps also filter it
>> in hvmloader, just for good measure.)
>
> My knowledge of live migration on x86 is a bit limited, but if I
> understand correctly those two options would require a guest to reboot
> in order to pick up the changes. Am I correct?

Not in the slightest.  The content returned changes whenever the
hypervisor changes.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2] xsm: hide detailed Xen version from unprivileged guests

2020-01-13 Thread Ian Jackson
Doug Goldstein writes ("Re: [Xen-devel] [PATCH v2] xsm: hide detailed Xen 
version from unprivileged guests"):
> I'd be happy if we had a Kconfig option behind what the string is. Give 
> me a blank as an option but default it to whatever string like 
> "" that you'd like. Every shipping Xen distro I've worked on has 
> had its own v1 variant of the patch and none of them authored by me.

Firstly: I generally agree with George's comments about not liking
silent failure, and I disagree with Andrew.  My inclination would be
to have this string be "" (or similar) - and to not filter it
in the DMI tables, so it would be "" everywhere.

Doug, I can't figure out from your messages whether that would meet
your needs ?  Is George right that the reason for filtering what used
to be "" is simply to reduce support burden due to the
negative connotations of "denied" ?  Would "hidden" fix that ?

If I am wrong, what is the reason for the filtering ?

If we have to have this string be configurable then so be it but I
would rather see if we can find one thing that meets downstreams'
needs.

FTR I agree with many of the points you make and I understand why you
think this is frustrating.  I hope we can clear this up.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC PATCH V2 11/11] x86: tsc: avoid system instability in hibernation

2020-01-13 Thread David Woodhouse
On Mon, 2020-01-13 at 13:01 +, Andrew Cooper wrote:
> On 13/01/2020 11:43, Singh, Balbir wrote:
> > On Mon, 2020-01-13 at 11:16 +0100, Peter Zijlstra wrote:
> > > On Fri, Jan 10, 2020 at 07:35:20AM -0800, Eduardo Valentin wrote:
> > > > Hey Peter,
> > > > 
> > > > On Wed, Jan 08, 2020 at 11:50:11AM +0100, Peter Zijlstra wrote:
> > > > > On Tue, Jan 07, 2020 at 11:45:26PM +, Anchal Agarwal wrote:
> > > > > > From: Eduardo Valentin 
> > > > > > 
> > > > > > System instability are seen during resume from hibernation when 
> > > > > > system
> > > > > > is under heavy CPU load. This is due to the lack of update of sched
> > > > > > clock data, and the scheduler would then think that heavy CPU hog
> > > > > > tasks need more time in CPU, causing the system to freeze
> > > > > > during the unfreezing of tasks. For example, threaded irqs,
> > > > > > and kernel processes servicing network interface may be delayed
> > > > > > for several tens of seconds, causing the system to be unreachable.
> > > > > > The fix for this situation is to mark the sched clock as unstable
> > > > > > as early as possible in the resume path, leaving it unstable
> > > > > > for the duration of the resume process. This will force the
> > > > > > scheduler to attempt to align the sched clock across CPUs using
> > > > > > the delta with time of day, updating sched clock data. In a post
> > > > > > hibernation event, we can then mark the sched clock as stable
> > > > > > again, avoiding unnecessary syncs with time of day on systems
> > > > > > in which TSC is reliable.
> > > > > 
> > > > > This makes no frigging sense what so bloody ever. If the clock is
> > > > > stable, we don't care about sched_clock_data. When it is stable you 
> > > > > get
> > > > > a linear function of the TSC without complicated bits on.
> > > > > 
> > > > > When it is unstable, only then do we care about the sched_clock_data.
> > > > > 
> > > > 
> > > > Yeah, maybe what is not clear here is that we covering for situation
> > > > where clock stability changes over time, e.g. at regular boot clock is
> > > > stable, hibernation happens, then restore happens in a non-stable clock.
> > > 
> > > Still confused, who marks the thing unstable? The patch seems to suggest
> > > you do yourself, but it is not at all clear why.
> > > 
> > > If TSC really is unstable, then it needs to remain unstable. If the TSC
> > > really is stable then there is no point in marking is unstable.
> > > 
> > > Either way something is off, and you're not telling me what.
> > > 
> > 
> > Hi, Peter
> > 
> > For your original comment, just wanted to clarify the following:
> > 
> > 1. After hibernation, the machine can be resumed on a different but 
> > compatible
> > host (these are VM images hibernated)
> > 2. This means the clock between host1 and host2 can/will be different
> 
> The guests TSC value is part of all save/migrate/resume state.  Given
> this bug, I presume you've actually discarded all register state on
> hibernate, and the TSC is starting again from 0?

Right. This is a guest-driven suspend to disk, followed by starting up
later on a different — but identical — host. There is no guest state
being saved as part of a Xen save/restore.

> The frequency of the new TSC might very likely be different, but the
> scale/offset in the paravirtual clock information should let Linux's
> view of time stay consistent.

The frequency as seen by the guest really needs to be the same. That
hibernated instance may only be booted again on a host which would have
been suitable for live migration. That's either because the TSC
frequency *is* the same, or with TSC scaling to make it appear that
way.

If the environment doesn't provide that then all bets are off and we
shouldn't be trying to hack around it in the guest kernel.

Across the hibernation we do expect a single step change in the TSC
value, just as on real hardware. Like Peter, I assume that the resume
code does cope with that but haven't checked precisely how/where it
does so.



smime.p7s
Description: S/MIME cryptographic signature
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH V7 2/4] x86/altp2m: Add hypercall to set a range of sve bits

2020-01-13 Thread Alexandru Stefan ISAILA


On 13.01.2020 14:53, Jan Beulich wrote:
> On 13.01.2020 11:32, Alexandru Stefan ISAILA wrote:
>> On 10.01.2020 18:20, Jan Beulich wrote:
>>> On 08.01.2020 15:08, Alexandru Stefan ISAILA wrote:
 +if ( !(rc = p2m_set_suppress_ve_multi(d, )) && sve.first_error )
 +rc = sve.first_error;
>>>
>>> Why the right side of the && ?
>>
>> This is intended to have p2m_set_suppress_ve() call
>> p2m_set_suppress_ve_multi(). So here first I call the _multi version and
>> the check if there was any error from the set/get (that is what
>> p2m_set_suppress_ve did before).
> 
> To put my original question differently: from a functionality pov,
> how would
> 
>  if ( !(rc = p2m_set_suppress_ve_multi(d, )) )
>  rc = sve.first_error; >
> be different from your variant (as long as the field indeed starts
> out as zero)?

It will be the same in this case and it can be dropped.

> 
>> I don't know why git made the patch so ugly.
> 
> I have no idea what ugliness you refer to here.
> 

I was talking about the fact that the changes in p2m_set_suppress_ve() 
got mixed with the ones in p2m_set_suppress_ve_multi().
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2] xsm: hide detailed Xen version from unprivileged guests

2020-01-13 Thread Julien Grall

Hi George,

Thank you for summarising the possibility. One question below.

On 13/01/2020 12:51, George Dunlap wrote:

2. Block XENVER_extraversion at the hypervisor level.  Leave xen_deny()
as returning "", but replace "" with "" in hvmloader so
it doesn't show up in the System Info and scare users.

3. Block XENVER_extraversion at the hypervisor level.  Change xen_deny()
to return a more benign string like "".  (Perhaps also filter it
in hvmloader, just for good measure.)


My knowledge of live migration on x86 is a bit limited, but if I 
understand correctly those two options would require a guest to reboot 
in order to pick up the changes. Am I correct?


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [ovmf test] 146044: regressions - FAIL

2020-01-13 Thread osstest service owner
flight 146044 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146044/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail REGR. vs. 145767
 test-amd64-amd64-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail REGR. vs. 
145767

version targeted for testing:
 ovmf 4465cd124fbcf5490faad6a1a834299b30b5d009
baseline version:
 ovmf 70911f1f4aee0366b6122f2b90d367ec0f066beb

Last test of basis   145767  2020-01-08 00:39:09 Z5 days
Failing since145774  2020-01-08 02:50:20 Z5 days   30 attempts
Testing same since   146041  2020-01-13 04:39:23 Z0 days2 attempts


People who touched revisions under test:
  Albecki, Mateusz 
  Ard Biesheuvel 
  Ashish Singhal 
  Eric Dong 
  Fan, ZhijuX 
  Jason Voelz 
  Laszlo Ersek 
  Mateusz Albecki 
  Pavana.K 
  Philippe Mathieu-Daud? 
  Philippe Mathieu-Daude 
  Siyuan Fu 
  Siyuan, Fu 
  Vitaly Cheptsov 
  Vitaly Cheptsov via Groups.Io 
  Zhiju.Fan 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 fail
 test-amd64-i386-xl-qemuu-ovmf-amd64  fail



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 422 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC PATCH V2 11/11] x86: tsc: avoid system instability in hibernation

2020-01-13 Thread Andrew Cooper
On 13/01/2020 11:43, Singh, Balbir wrote:
> On Mon, 2020-01-13 at 11:16 +0100, Peter Zijlstra wrote:
>> On Fri, Jan 10, 2020 at 07:35:20AM -0800, Eduardo Valentin wrote:
>>> Hey Peter,
>>>
>>> On Wed, Jan 08, 2020 at 11:50:11AM +0100, Peter Zijlstra wrote:
 On Tue, Jan 07, 2020 at 11:45:26PM +, Anchal Agarwal wrote:
> From: Eduardo Valentin 
>
> System instability are seen during resume from hibernation when system
> is under heavy CPU load. This is due to the lack of update of sched
> clock data, and the scheduler would then think that heavy CPU hog
> tasks need more time in CPU, causing the system to freeze
> during the unfreezing of tasks. For example, threaded irqs,
> and kernel processes servicing network interface may be delayed
> for several tens of seconds, causing the system to be unreachable.
> The fix for this situation is to mark the sched clock as unstable
> as early as possible in the resume path, leaving it unstable
> for the duration of the resume process. This will force the
> scheduler to attempt to align the sched clock across CPUs using
> the delta with time of day, updating sched clock data. In a post
> hibernation event, we can then mark the sched clock as stable
> again, avoiding unnecessary syncs with time of day on systems
> in which TSC is reliable.
 This makes no frigging sense what so bloody ever. If the clock is
 stable, we don't care about sched_clock_data. When it is stable you get
 a linear function of the TSC without complicated bits on.

 When it is unstable, only then do we care about the sched_clock_data.

>>> Yeah, maybe what is not clear here is that we covering for situation
>>> where clock stability changes over time, e.g. at regular boot clock is
>>> stable, hibernation happens, then restore happens in a non-stable clock.
>> Still confused, who marks the thing unstable? The patch seems to suggest
>> you do yourself, but it is not at all clear why.
>>
>> If TSC really is unstable, then it needs to remain unstable. If the TSC
>> really is stable then there is no point in marking is unstable.
>>
>> Either way something is off, and you're not telling me what.
>>
> Hi, Peter
>
> For your original comment, just wanted to clarify the following:
>
> 1. After hibernation, the machine can be resumed on a different but compatible
> host (these are VM images hibernated)
> 2. This means the clock between host1 and host2 can/will be different

The guests TSC value is part of all save/migrate/resume state.  Given
this bug, I presume you've actually discarded all register state on
hibernate, and the TSC is starting again from 0?

The frequency of the new TSC might very likely be different, but the
scale/offset in the paravirtual clock information should let Linux's
view of time stay consistent.

> In your comments are you making the assumption that the host(s) is/are the
> same? Just checking the assumptions being made and being on the same page with
> them.

TSCs are a massive source of "fun".  I'm not surprised that there are
yet more bugs around.

Does anyone actually know what does/should happen to the real TSC on
native S4?  The default course of action should be for virtualisation to
follow suit.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH V7 2/4] x86/altp2m: Add hypercall to set a range of sve bits

2020-01-13 Thread Jan Beulich
On 13.01.2020 11:32, Alexandru Stefan ISAILA wrote:
> On 10.01.2020 18:20, Jan Beulich wrote:
>> On 08.01.2020 15:08, Alexandru Stefan ISAILA wrote:
>>> +if ( !(rc = p2m_set_suppress_ve_multi(d, )) && sve.first_error )
>>> +rc = sve.first_error;
>>
>> Why the right side of the && ?
> 
> This is intended to have p2m_set_suppress_ve() call 
> p2m_set_suppress_ve_multi(). So here first I call the _multi version and 
> the check if there was any error from the set/get (that is what 
> p2m_set_suppress_ve did before).

To put my original question differently: from a functionality pov,
how would

if ( !(rc = p2m_set_suppress_ve_multi(d, )) )
rc = sve.first_error;

be different from your variant (as long as the field indeed starts
out as zero)?

> I don't know why git made the patch so ugly.

I have no idea what ugliness you refer to here.

Jan

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2] xsm: hide detailed Xen version from unprivileged guests

2020-01-13 Thread George Dunlap
On 1/12/20 6:26 PM, Doug Goldstein wrote:
> On 1/11/20 3:02 AM, George Dunlap wrote:
>>
>>
>>> On Jan 11, 2020, at 4:02 AM, Doug Goldstein  wrote:
>>>
>>>
>>>
>>> On 1/10/20 4:37 AM, Sergey Dyasli wrote:
 Hide the following information that can help identify the running Xen
 binary version: XENVER_extraversion, XENVER_compile_info,
 XENVER_changeset.
 Add explicit cases for XENVER_commandline and XENVER_build_id as well.
 Introduce xsm_filter_denied() to hvmloader to remove "" string
 from guest's DMI tables that otherwise would be shown in tools like
 dmidecode.
 Signed-off-by: Sergey Dyasli 
 ---
 v1 --> v2:
 - Added xsm_filter_denied() to hvmloader instead of modifying
 xen_deny()
>>>
>>> So 100% this version of the patch won't fly with the various
>>> downstreams that run the v1 of this patch. Those various consumers
>>> will stick with v1.
>>>
>>> If the goal of this is to reduce the burden of the downstreams and
>>> their customers to carry a patch against Xen then I wouldn't even
>>> bother with this version.
>>
>> If the goal is to come up with a solution that works for everyone, it
>> would be helpful if you said *why* “various consumers” would find this
>> patch unacceptable; and also what they might think about the alternate
>> solutions proposed (and why).
>>
>>   -George
>>
> 

[snip]

> Now I know someone is going to read this and say "Look at Doug and him
> advocating for security through obscurity".

FWIW I'd be the first person to contradict them, and say you were
practicing "defense in depth". :-)

> Ultimately my point is if the goal of this patch is to upstream a patch
> that's carried by various downstreams, why not actually listen to what
> caused them to write the patch?

Right, that's what I'm trying to do; but I don't seem to be making much
progress.

Here's my summary of the situation and arguments so far:

1. The xen_version hypercall can return strings for a number of
different values, including XENVER_extraversion, which gives the point
release and build id.

2. The XSM dummy module has code to filter which of these are allowed
for unprivileged guests.  When access to a given value is filtered, no
error is returned; rather, the string "" is returned.

3. Knowledge about the specific instantiation of Xen on which they are
running makes it easier for attackers to know how to attack t he system;
the XENVER_extraversion provides little value to legitimate users, but a
lot of value to attackers.   As a defense-in-depth measure, it's
important to be able to hide this information.

4. There's currently a patch carried by many downstreams, which changes
the XSM dummy module to deny XENVER_extraversion to unprivileged guests.

5. However, this caused "" to show up in various user-visible
places, which caused customer support headaches.  So this out-of-tree
patch also replaced the string returned when denying access to ""
instead.  Note that this is not *only* for XENVER_extraversion; with
that patch, *any* time the value requested in xen_version is denied by
policy, "" will be returned.

6. Silently returning an empty string is considered bad interface design
by several developers.  So Sergey's second patch:
 - Still denies XENVER_extraversion at the hypervisor level
 - Leaves the value returned by the hypervisor as ""
 - Filters the "" string at the hvmloader level, to prevent it
leaking into a GUI and scaring customers.

Now we get to Andy's objection on the 10th:

---
The reason for this (which ought to be obvious, but I guess only to
those who actually do customer support) is basic human physiology.
"denied" means something has gone wrong.  It scares people, and causes
them to seek help to change fix whatever is broken.

It is not appropriate for it to find its way into the guest in the first
place, and that includes turning up in `dmesg` and other logs, and
expecting guest runtime to filter for it is complete nonsense.
---

Basically, Andy says that *anywhere* it might show up is way too scary,
even a guest dmesg log.

Well, I disagree; I look in "dmesg" and I see loads of "scary" things.
But if "" is too scary, then we can try "".

Then we come to your mail.

You spend two paragraphs justifying why we need to do #4 (hide the value
from unprivileged guests), basically reiterating point #3 and dealing
with potential objections.  But nobody objects to #4, or disagrees with #3.

You then have a paragraph arguing why it's important that information be
stripped at the hypervisor rather than in the toolstack.

But Sergey's v2 patch *does* strip the information at the hypervisor.
His patch makes it so that XENVER_extraversion returns "".  The
code which converts "" to "" in hvmloader is purely a UI thing,
so that people looking in their Windows System Info don't get scary
messages.

> I'd be happy if we had a Kconfig option behind what the string is. Give
> me a blank as an option but default it to whatever string like
> "" that you'd like. 

[Xen-devel] [qemu-mainline test] 146045: regressions - FAIL

2020-01-13 Thread osstest service owner
flight 146045 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146045/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm   6 xen-buildfail REGR. vs. 144861
 build-arm64   6 xen-buildfail REGR. vs. 144861
 build-amd64   6 xen-buildfail REGR. vs. 144861
 build-i386-xsm6 xen-buildfail REGR. vs. 144861
 build-armhf   6 xen-buildfail REGR. vs. 144861
 build-amd64-xsm   6 xen-buildfail REGR. vs. 144861
 build-i3866 xen-buildfail REGR. vs. 144861

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-rtds  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvshim1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)  blocked n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1) blocked n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-cubietruck  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-pvshim 1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-seattle   1 build-check(1)   blocked  n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1)  blocked n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-shadow 1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-rtds  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-multivcpu  1 

Re: [Xen-devel] [RFC PATCH V2 11/11] x86: tsc: avoid system instability in hibernation

2020-01-13 Thread Peter Zijlstra
On Mon, Jan 13, 2020 at 11:43:18AM +, Singh, Balbir wrote:
> For your original comment, just wanted to clarify the following:
> 
> 1. After hibernation, the machine can be resumed on a different but compatible
> host (these are VM images hibernated)
> 2. This means the clock between host1 and host2 can/will be different
> 
> In your comments are you making the assumption that the host(s) is/are the
> same? Just checking the assumptions being made and being on the same page with
> them.

I would expect this to be the same problem we have as regular suspend,
after power off the TSC will have been reset, so resume will have to
somehow bridge that gap. I've no idea if/how it does that.

I remember some BIOSes had crazy TSC ideas for suspend2ram, and we grew
tsc_restore_sched_clock_state() for it.

Playing crazy games like what you're doing just isn't it though.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 4/6] x86/boot: Clean up l?_bootmap[] construction

2020-01-13 Thread Andrew Cooper
On 07/01/2020 16:30, Jan Beulich wrote:
> On 07.01.2020 17:16, Jan Beulich wrote:
>> On 06.01.2020 16:54, Andrew Cooper wrote:
>>> --- a/xen/arch/x86/efi/efi-boot.h
>>> +++ b/xen/arch/x86/efi/efi-boot.h
>>> @@ -584,21 +584,24 @@ static void __init efi_arch_memory_setup(void)
>>>  if ( !efi_enabled(EFI_LOADER) )
>>>  return;
>>>  
>>> -/* Initialise L2 identity-map and boot-map page table entries (16MB). 
>>> */
>>> +/*
>>> + * Map Xen into the directmap (NX, needed for early-boot pagetable
>>> + * handling/walking), and identity map Xen into bootmap (X, needed for 
>>> the
>>> + * transition from the EFI pagetables to Xen), using 2M superpages.
>>> + */
>> How does NX vs X matter for the code below here? PAGE_HYPERVISOR and
>> __PAGE_HYPERVISOR, as used below, differ by just _PAGE_GLOBAL. Did
>> you mean to make further changes?

Nope.  The comments were actually correct (and the code, remained correct).

PAGE_HYPERVISOR and __PAGE_HYPERVISOR really do differ by NX as well,
outside of asm code.  I'm going to fix this because its too complicated
to reason about.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-unstable test] 146039: tolerable FAIL

2020-01-13 Thread osstest service owner
flight 146039 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146039/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-rtds 18 guest-localmigrate/x10   fail  like 146030
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 146030
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 146030
 test-armhf-armhf-xl-rtds 12 guest-start  fail  like 146030
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 146030
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 146030
 test-amd64-amd64-qemuu-nested-intel 17 debian-hvm-install/l1/l2 fail like 
146030
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 146030
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 146030
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 146030
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 146030
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop fail like 146030
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-arm64-arm64-xl-seattle  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass

version targeted for testing:
 xen  8842d01b300919e20bca2e1138c458a8483600f8
baseline version:
 xen  8842d01b300919e20bca2e1138c458a8483600f8

Last test of basis   146039  2020-01-13 03:13:30 Z0 days
Testing same since  (not found) 0 attempts

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  pass
 build-i386-xsm   pass
 build-amd64-xtf  pass
 build-amd64

Re: [Xen-devel] [RFC PATCH 0/3] Live update boot memory management

2020-01-13 Thread David Woodhouse
On Wed, 2020-01-08 at 17:24 +, David Woodhouse wrote:
> When doing a live update, Xen needs to be very careful not to scribble
> on pages which contain guest memory or state information for the
> domains which are being preserved.
> 
> The information about which pages are in use is contained in the live
> update state passed from the previous Xen — which is mostly just a
> guest-transparent live migration data stream, except that it points to
> the page tables in place in memory while traditional live migration
> obviously copies the pages separately.
> 
> Our initial implementation actually prepended a list of 'in-use' ranges
> to the live update state, and made the boot allocator treat them the
> same as 'bad pages'. That worked well enough for initial development
> but wouldn't scale to a live production system, mainly because the boot
> allocator has a limit of 512 memory ranges that it can keep track of,
> and a real system would end up more fragmented than that.
> 
> My other concern with that approach is that it required two passes over
> the domain-owned pages. We have to do a later pass *anyway*, as we set
> up ownership in the frametable for each page — and that has to happen
> after we've managed to allocate a 'struct domain' for each page_info to
> point to. If we want to keep the pause time due to a live update down
> to a bare minimum, doing two passes over the full set of domain pages
> isn't my favourite strategy.
> 
> So we've settled on a simpler approach — reserve a contiguous region
> of physical memory which *won't* be used for domain pages. Let the boot
> allocator see *only* that region of memory, and plug the rest of the
> memory in later only after doing a full pass of the live update state.
> 
> This means that we have to ensure the reserved region is large enough,
> but ultimately we had that problem either way — even if we were
> processing the actual free ranges, if the page_info grew and we didn't
> have enough contiguous space for the new frametable we were hosed
> anyway.
> 
> So the straw man patch ends up being really simple, as a seed for
> bikeshedding. Just take a 'liveupdate=' region on the command line,
> which kexec(8) can find from the running Xen. The initial Xen needs to
> ensure that it *won't* allocate any pages from that range which will
> subsequently need to be preserved across live update, which isn't done
> yet. We just need to make sure that any page which might be given to
> share_xen_page_with_guest() is allocated appropriately.
> 
> The part which actually hands over the live update state isn't included
> yet, so this really does just *defer* the addition of the memory until
> a little bit later in __start_xen(). Actually taking ranges out of it
> will come later.

What isn't addressed in this series is actually *honouring* the promise
not to put pages into the reserved LU bootmem region that need to be
preserved over live update. As things stand, we just add them to the
heap anyway in end_boot_allocator().

It isn't even sufficient to use these pages for xenheap allocations and
not domheap, since there are cases where we allocate from the xenheap
and then share pages to a domain.

Hongyan's patches to kill the directmap have already started addressing
a bunch of the places that do that, so what I'm inclined to do in the
short term is just *not* use the remaining space in the reserved LU
bootmem region. Use it for boot time allocations (including the
frametable) only, and *not* insert the rest of those pages into the
heap allocator in end_boot_allocator() for now. If sized appropriately,
there shouldn't be much wastage anyway. We can refine it and ensure
that we can use those pages but *not* for domain allocations, once the
dust has settled on the directmap removal.




smime.p7s
Description: S/MIME cryptographic signature
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC PATCH V2 11/11] x86: tsc: avoid system instability in hibernation

2020-01-13 Thread Rafael J. Wysocki
On Mon, Jan 13, 2020 at 12:43 PM Singh, Balbir  wrote:
>
> On Mon, 2020-01-13 at 11:16 +0100, Peter Zijlstra wrote:
> > On Fri, Jan 10, 2020 at 07:35:20AM -0800, Eduardo Valentin wrote:
> > > Hey Peter,
> > >
> > > On Wed, Jan 08, 2020 at 11:50:11AM +0100, Peter Zijlstra wrote:
> > > > On Tue, Jan 07, 2020 at 11:45:26PM +, Anchal Agarwal wrote:
> > > > > From: Eduardo Valentin 
> > > > >
> > > > > System instability are seen during resume from hibernation when system
> > > > > is under heavy CPU load. This is due to the lack of update of sched
> > > > > clock data, and the scheduler would then think that heavy CPU hog
> > > > > tasks need more time in CPU, causing the system to freeze
> > > > > during the unfreezing of tasks. For example, threaded irqs,
> > > > > and kernel processes servicing network interface may be delayed
> > > > > for several tens of seconds, causing the system to be unreachable.
> > > > > The fix for this situation is to mark the sched clock as unstable
> > > > > as early as possible in the resume path, leaving it unstable
> > > > > for the duration of the resume process. This will force the
> > > > > scheduler to attempt to align the sched clock across CPUs using
> > > > > the delta with time of day, updating sched clock data. In a post
> > > > > hibernation event, we can then mark the sched clock as stable
> > > > > again, avoiding unnecessary syncs with time of day on systems
> > > > > in which TSC is reliable.
> > > >
> > > > This makes no frigging sense what so bloody ever. If the clock is
> > > > stable, we don't care about sched_clock_data. When it is stable you get
> > > > a linear function of the TSC without complicated bits on.
> > > >
> > > > When it is unstable, only then do we care about the sched_clock_data.
> > > >
> > >
> > > Yeah, maybe what is not clear here is that we covering for situation
> > > where clock stability changes over time, e.g. at regular boot clock is
> > > stable, hibernation happens, then restore happens in a non-stable clock.
> >
> > Still confused, who marks the thing unstable? The patch seems to suggest
> > you do yourself, but it is not at all clear why.
> >
> > If TSC really is unstable, then it needs to remain unstable. If the TSC
> > really is stable then there is no point in marking is unstable.
> >
> > Either way something is off, and you're not telling me what.
> >
>
> Hi, Peter
>
> For your original comment, just wanted to clarify the following:
>
> 1. After hibernation, the machine can be resumed on a different but compatible
> host (these are VM images hibernated)
> 2. This means the clock between host1 and host2 can/will be different

So the problem is specific to this particular use case.

I'm not sure why to impose this hack on hibernation in all cases.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC PATCH V2 11/11] x86: tsc: avoid system instability in hibernation

2020-01-13 Thread Singh, Balbir
On Mon, 2020-01-13 at 11:16 +0100, Peter Zijlstra wrote:
> On Fri, Jan 10, 2020 at 07:35:20AM -0800, Eduardo Valentin wrote:
> > Hey Peter,
> > 
> > On Wed, Jan 08, 2020 at 11:50:11AM +0100, Peter Zijlstra wrote:
> > > On Tue, Jan 07, 2020 at 11:45:26PM +, Anchal Agarwal wrote:
> > > > From: Eduardo Valentin 
> > > > 
> > > > System instability are seen during resume from hibernation when system
> > > > is under heavy CPU load. This is due to the lack of update of sched
> > > > clock data, and the scheduler would then think that heavy CPU hog
> > > > tasks need more time in CPU, causing the system to freeze
> > > > during the unfreezing of tasks. For example, threaded irqs,
> > > > and kernel processes servicing network interface may be delayed
> > > > for several tens of seconds, causing the system to be unreachable.
> > > > The fix for this situation is to mark the sched clock as unstable
> > > > as early as possible in the resume path, leaving it unstable
> > > > for the duration of the resume process. This will force the
> > > > scheduler to attempt to align the sched clock across CPUs using
> > > > the delta with time of day, updating sched clock data. In a post
> > > > hibernation event, we can then mark the sched clock as stable
> > > > again, avoiding unnecessary syncs with time of day on systems
> > > > in which TSC is reliable.
> > > 
> > > This makes no frigging sense what so bloody ever. If the clock is
> > > stable, we don't care about sched_clock_data. When it is stable you get
> > > a linear function of the TSC without complicated bits on.
> > > 
> > > When it is unstable, only then do we care about the sched_clock_data.
> > > 
> > 
> > Yeah, maybe what is not clear here is that we covering for situation
> > where clock stability changes over time, e.g. at regular boot clock is
> > stable, hibernation happens, then restore happens in a non-stable clock.
> 
> Still confused, who marks the thing unstable? The patch seems to suggest
> you do yourself, but it is not at all clear why.
> 
> If TSC really is unstable, then it needs to remain unstable. If the TSC
> really is stable then there is no point in marking is unstable.
> 
> Either way something is off, and you're not telling me what.
> 

Hi, Peter

For your original comment, just wanted to clarify the following:

1. After hibernation, the machine can be resumed on a different but compatible
host (these are VM images hibernated)
2. This means the clock between host1 and host2 can/will be different

In your comments are you making the assumption that the host(s) is/are the
same? Just checking the assumptions being made and being on the same page with
them.

Balbir Singh.

> > > > Reviewed-by: Erik Quanstrom 
> > > > Reviewed-by: Frank van der Linden 
> > > > Reviewed-by: Balbir Singh 
> > > > Reviewed-by: Munehisa Kamata 
> > > > Tested-by: Anchal Agarwal 
> > > > Signed-off-by: Eduardo Valentin 
> > > > ---
> > > 
> > > NAK, the code very much relies on never getting marked stable again
> > > after it gets set to unstable.
> > > 
> > 
> > Well actually, at the PM_POST_HIBERNATION, we do the check and set stable
> > if
> > known to be stable.
> > 
> > The issue only really happens during the restoration path under scheduling
> > pressure,
> > which takes forever to finish, as described in the commit.
> > 
> > Do you see a better solution for this issue?
> 
> I still have no clue what your actual problem is. You say scheduling
> goes wobbly because sched_clock_data is stale, but when stable that
> doesn't matter.
> 
> So what is the actual problem?
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [qemu-mainline test] 146043: regressions - FAIL

2020-01-13 Thread osstest service owner
flight 146043 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146043/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm   6 xen-buildfail REGR. vs. 144861
 build-arm64   6 xen-buildfail REGR. vs. 144861
 build-amd64   6 xen-buildfail REGR. vs. 144861
 build-i386-xsm6 xen-buildfail REGR. vs. 144861
 build-armhf   6 xen-buildfail REGR. vs. 144861
 build-amd64-xsm   6 xen-buildfail REGR. vs. 144861
 build-i3866 xen-buildfail REGR. vs. 144861

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvshim1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)  blocked n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)   blocked  n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-seattle   1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)  blocked n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-armhf-armhf-xl-cubietruck  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-shadow 1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-rtds  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-multivcpu  1 

Re: [Xen-devel] [PATCH v2] xsm: hide detailed Xen version from unprivileged guests

2020-01-13 Thread Sergey Dyasli
On 10/01/2020 11:02, Andrew Cooper wrote:
> On 10/01/2020 10:37, Sergey Dyasli wrote:
>> Hide the following information that can help identify the running Xen
>> binary version: XENVER_extraversion, XENVER_compile_info, XENVER_changeset.
>> Add explicit cases for XENVER_commandline and XENVER_build_id as well.
>>
>> Introduce xsm_filter_denied() to hvmloader to remove "" string
>> from guest's DMI tables that otherwise would be shown in tools like
>> dmidecode.
>>
>> Signed-off-by: Sergey Dyasli 
>> ---
>> v1 --> v2:
>> - Added xsm_filter_denied() to hvmloader instead of modifying xen_deny()
>> - Made behaviour the same for both Release and Debug builds
>> - XENVER_capabilities is no longer hided
>>
>> CC: Andrew Cooper 
>> CC: George Dunlap 
>> CC: Ian Jackson 
>> CC: Jan Beulich 
>> CC: Julien Grall 
>> CC: Konrad Rzeszutek Wilk 
>> CC: Stefano Stabellini 
>> CC: Wei Liu 
>> CC: Daniel De Graaf 
>
> I realise there are arguments over how to fix this, but we (the Xen
> community) have already f*cked up once here, and this is doing so a
> second time.
>
> Nack.
>
> Fixing it anywhere other than Xen is simply not appropriate.
>
> The reason for this (which ought to be obvious, but I guess only to
> those who actually do customer support) is basic human physiology.
> "denied" means something has gone wrong.  It scares people, and causes
> them to seek help to change fix whatever is broken.

But the patch takes care of that by removing "denied" from DMI tables.
Functionally it should have the same effect as v1 to ordinary guests.

> It is not appropriate for it to find its way into the guest in the first
> place, and that includes turning up in `dmesg` and other logs, and
> expecting guest runtime to filter for it is complete nonsense.

`dmesg` will have only Xen major version (e.g. Xen 4.13) with this patch
applied. Even if there exists a tool which uses xen_version hypercall
for information gathering, it would show you "" for fields like
commandline and build_id already (without the patch). So extending this
behaviour for other sensitive fields is not a regression IMHO.

> As said several times before, the empty string is completely fine ABI
> wise, doesn't confuse customers, and really really does work in practice.

I agree with the other opinion that returning an empty string is too
ambiguous. I'd prefer to retain the current behaviour with (whatever)
non-empty descriptive string.

--
Thanks,
Sergey

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 3/4] drm/cirrus: Let DRM core send VBLANK events

2020-01-13 Thread Thomas Zimmermann
In drm_atomic_helper_fake_vblank(), the DRM core sends out VBLANK
events if struct drm_crtc_state.no_vblank is enabled. Replace cirrus'
VBLANK events with the DRM core's functionality.

v2:
* set struct_drm_crtc_state.no_vblank in cirrus_pipe_check()

Signed-off-by: Thomas Zimmermann 
---
 drivers/gpu/drm/cirrus/cirrus.c | 10 ++
 1 file changed, 2 insertions(+), 8 deletions(-)

diff --git a/drivers/gpu/drm/cirrus/cirrus.c b/drivers/gpu/drm/cirrus/cirrus.c
index 248c9f765c45..5ff15e8a2a0a 100644
--- a/drivers/gpu/drm/cirrus/cirrus.c
+++ b/drivers/gpu/drm/cirrus/cirrus.c
@@ -38,7 +38,6 @@
 #include 
 #include 
 #include 
-#include 
 
 #define DRIVER_NAME "cirrus"
 #define DRIVER_DESC "qemu cirrus vga"
@@ -404,6 +403,8 @@ static int cirrus_pipe_check(struct drm_simple_display_pipe 
*pipe,
 {
struct drm_framebuffer *fb = plane_state->fb;
 
+   crtc_state->no_vblank = true;
+
if (!fb)
return 0;
return cirrus_check_size(fb->width, fb->height, fb);
@@ -434,13 +435,6 @@ static void cirrus_pipe_update(struct 
drm_simple_display_pipe *pipe,
 
if (drm_atomic_helper_damage_merged(old_state, state, ))
cirrus_fb_blit_rect(pipe->plane.state->fb, );
-
-   if (crtc->state->event) {
-   spin_lock_irq(>dev->event_lock);
-   drm_crtc_send_vblank_event(crtc, crtc->state->event);
-   crtc->state->event = NULL;
-   spin_unlock_irq(>dev->event_lock);
-   }
 }
 
 static const struct drm_simple_display_pipe_funcs cirrus_pipe_funcs = {
-- 
2.24.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 4/4] drm/simple-kms: Let DRM core send VBLANK events by default

2020-01-13 Thread Thomas Zimmermann
In drm_atomic_helper_fake_vblank() the DRM core sends out VBLANK events
if struct drm_crtc_state.no_vblank is enabled in the check() callbacks.

For drivers that have neither an enable_vblank() callback nor a check()
callback, the simple-KMS helpers enable VBLANK generation by default. This
simplifies bochs, udl, several tiny drivers, and drivers based upon MIPI
DPI helpers. The driver for Xen explicitly disables no_vblank, as it has
its own logic for sending these events.

Signed-off-by: Thomas Zimmermann 
---
 drivers/gpu/drm/bochs/bochs_kms.c   |  9 -
 drivers/gpu/drm/drm_mipi_dbi.c  |  9 -
 drivers/gpu/drm/drm_simple_kms_helper.c | 19 +++
 drivers/gpu/drm/tiny/gm12u320.c |  9 -
 drivers/gpu/drm/tiny/ili9225.c  |  9 -
 drivers/gpu/drm/tiny/repaper.c  |  9 -
 drivers/gpu/drm/tiny/st7586.c   |  9 -
 drivers/gpu/drm/udl/udl_modeset.c   | 11 ---
 drivers/gpu/drm/xen/xen_drm_front_kms.c | 13 +
 9 files changed, 28 insertions(+), 69 deletions(-)

diff --git a/drivers/gpu/drm/bochs/bochs_kms.c 
b/drivers/gpu/drm/bochs/bochs_kms.c
index 3f0006c2470d..ff275faee88d 100644
--- a/drivers/gpu/drm/bochs/bochs_kms.c
+++ b/drivers/gpu/drm/bochs/bochs_kms.c
@@ -7,7 +7,6 @@
 #include 
 #include 
 #include 
-#include 
 
 #include "bochs.h"
 
@@ -57,16 +56,8 @@ static void bochs_pipe_update(struct drm_simple_display_pipe 
*pipe,
  struct drm_plane_state *old_state)
 {
struct bochs_device *bochs = pipe->crtc.dev->dev_private;
-   struct drm_crtc *crtc = >crtc;
 
bochs_plane_update(bochs, pipe->plane.state);
-
-   if (crtc->state->event) {
-   spin_lock_irq(>dev->event_lock);
-   drm_crtc_send_vblank_event(crtc, crtc->state->event);
-   crtc->state->event = NULL;
-   spin_unlock_irq(>dev->event_lock);
-   }
 }
 
 static const struct drm_simple_display_pipe_funcs bochs_pipe_funcs = {
diff --git a/drivers/gpu/drm/drm_mipi_dbi.c b/drivers/gpu/drm/drm_mipi_dbi.c
index 16bff1be4b8a..13b753cb3f67 100644
--- a/drivers/gpu/drm/drm_mipi_dbi.c
+++ b/drivers/gpu/drm/drm_mipi_dbi.c
@@ -24,7 +24,6 @@
 #include 
 #include 
 #include 
-#include 
 #include 
 
 #define MIPI_DBI_MAX_SPI_READ_SPEED 200 /* 2MHz */
@@ -299,18 +298,10 @@ void mipi_dbi_pipe_update(struct drm_simple_display_pipe 
*pipe,
  struct drm_plane_state *old_state)
 {
struct drm_plane_state *state = pipe->plane.state;
-   struct drm_crtc *crtc = >crtc;
struct drm_rect rect;
 
if (drm_atomic_helper_damage_merged(old_state, state, ))
mipi_dbi_fb_dirty(state->fb, );
-
-   if (crtc->state->event) {
-   spin_lock_irq(>dev->event_lock);
-   drm_crtc_send_vblank_event(crtc, crtc->state->event);
-   spin_unlock_irq(>dev->event_lock);
-   crtc->state->event = NULL;
-   }
 }
 EXPORT_SYMBOL(mipi_dbi_pipe_update);
 
diff --git a/drivers/gpu/drm/drm_simple_kms_helper.c 
b/drivers/gpu/drm/drm_simple_kms_helper.c
index 15fb516ae2d8..4414c7a5b2ce 100644
--- a/drivers/gpu/drm/drm_simple_kms_helper.c
+++ b/drivers/gpu/drm/drm_simple_kms_helper.c
@@ -146,10 +146,21 @@ static int drm_simple_kms_plane_atomic_check(struct 
drm_plane *plane,
if (!plane_state->visible)
return 0;
 
-   if (!pipe->funcs || !pipe->funcs->check)
-   return 0;
-
-   return pipe->funcs->check(pipe, plane_state, crtc_state);
+   if (pipe->funcs) {
+   if (pipe->funcs->check)
+   return pipe->funcs->check(pipe, plane_state,
+ crtc_state);
+   if (pipe->funcs->enable_vblank)
+   return 0;
+   }
+
+   /* Drivers without VBLANK support have to fake VBLANK events. As
+* there's no check() callback to enable this, set the no_vblank
+* field by default.
+*/
+   crtc_state->no_vblank = true;
+
+   return 0;
 }
 
 static void drm_simple_kms_plane_atomic_update(struct drm_plane *plane,
diff --git a/drivers/gpu/drm/tiny/gm12u320.c b/drivers/gpu/drm/tiny/gm12u320.c
index 94fb1f593564..a48173441ae0 100644
--- a/drivers/gpu/drm/tiny/gm12u320.c
+++ b/drivers/gpu/drm/tiny/gm12u320.c
@@ -22,7 +22,6 @@
 #include 
 #include 
 #include 
-#include 
 
 static bool eco_mode;
 module_param(eco_mode, bool, 0644);
@@ -610,18 +609,10 @@ static void gm12u320_pipe_update(struct 
drm_simple_display_pipe *pipe,
 struct drm_plane_state *old_state)
 {
struct drm_plane_state *state = pipe->plane.state;
-   struct drm_crtc *crtc = >crtc;
struct drm_rect rect;
 
if (drm_atomic_helper_damage_merged(old_state, state, ))
gm12u320_fb_mark_dirty(pipe->plane.state->fb, );
-
-   if (crtc->state->event) {
-   

[Xen-devel] [PATCH v2 0/4] Use no_vblank property for drivers without VBLANK

2020-01-13 Thread Thomas Zimmermann
Instead of faking VBLANK events by themselves, drivers without VBLANK
support can enable drm_crtc_vblank.no_vblank and let DRM do the rest.
The patchset makes this official and converts over several drivers.

Ast already uses the functionality and just needs a cleanup. Cirrus can
be converted easily by setting the field in the check() callback and
removing the existing VBLANK code. For most other simple-KMS drivers
without enable_vblank() and check(), simple-KMS helpers can enable the
faked VBLANK by default. The only exception is Xen, which comes with
its own VBLANK logic and should rather to disable no_vblank.

v2:
* document functionality (Daniel)
* cleanup ast (Daniel)
* let simple-kms handle no_vblank where possible

Thomas Zimmermann (4):
  drm: Document struct drm_crtc_state.no_vblank for faking VBLANK events
  drm/ast: Set struct drm_crtc_state.no_vblank in atomic_check()
  drm/cirrus: Let DRM core send VBLANK events
  drm/simple-kms: Let DRM core send VBLANK events by default

 drivers/gpu/drm/ast/ast_mode.c  |  4 ++--
 drivers/gpu/drm/bochs/bochs_kms.c   |  9 -
 drivers/gpu/drm/cirrus/cirrus.c | 10 ++
 drivers/gpu/drm/drm_atomic_helper.c |  4 +++-
 drivers/gpu/drm/drm_mipi_dbi.c  |  9 -
 drivers/gpu/drm/drm_simple_kms_helper.c | 19 +++
 drivers/gpu/drm/tiny/gm12u320.c |  9 -
 drivers/gpu/drm/tiny/ili9225.c  |  9 -
 drivers/gpu/drm/tiny/repaper.c  |  9 -
 drivers/gpu/drm/tiny/st7586.c   |  9 -
 drivers/gpu/drm/udl/udl_modeset.c   | 11 ---
 drivers/gpu/drm/xen/xen_drm_front_kms.c | 13 +
 include/drm/drm_crtc.h  |  9 +++--
 include/drm/drm_simple_kms_helper.h |  7 +--
 14 files changed, 47 insertions(+), 84 deletions(-)

--
2.24.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 2/4] drm/ast: Set struct drm_crtc_state.no_vblank in atomic_check()

2020-01-13 Thread Thomas Zimmermann
CRTC state properties should be computed in atomic_check(). Do so for
the no_vblank field.

Signed-off-by: Thomas Zimmermann 
---
 drivers/gpu/drm/ast/ast_mode.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/ast/ast_mode.c b/drivers/gpu/drm/ast/ast_mode.c
index 34608f0499eb..ef7a0b08cc05 100644
--- a/drivers/gpu/drm/ast/ast_mode.c
+++ b/drivers/gpu/drm/ast/ast_mode.c
@@ -800,6 +800,8 @@ static int ast_crtc_helper_atomic_check(struct drm_crtc 
*crtc,
return -EINVAL;
}
 
+   state->no_vblank = true;
+
ast_state = to_ast_crtc_state(state);
 
format = ast_state->format;
@@ -833,8 +835,6 @@ static void ast_crtc_helper_atomic_flush(struct drm_crtc 
*crtc,
struct ast_vbios_mode_info *vbios_mode_info;
struct drm_display_mode *adjusted_mode;
 
-   crtc->state->no_vblank = true;
-
ast_state = to_ast_crtc_state(crtc->state);
 
format = ast_state->format;
-- 
2.24.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 1/4] drm: Document struct drm_crtc_state.no_vblank for faking VBLANK events

2020-01-13 Thread Thomas Zimmermann
Drivers for CRTC hardware without support for VBLANK interrupts can set
struct drm_crtc_state.no_vblank and let DRM's atomic commit helpers
generate the VBLANK events automatically. Document this in order to make
it official.

Signed-off-by: Thomas Zimmermann 
---
 drivers/gpu/drm/drm_atomic_helper.c | 4 +++-
 include/drm/drm_crtc.h  | 9 +++--
 include/drm/drm_simple_kms_helper.h | 7 +--
 3 files changed, 15 insertions(+), 5 deletions(-)

diff --git a/drivers/gpu/drm/drm_atomic_helper.c 
b/drivers/gpu/drm/drm_atomic_helper.c
index 4511c2e07bb9..ce30a37971e4 100644
--- a/drivers/gpu/drm/drm_atomic_helper.c
+++ b/drivers/gpu/drm/drm_atomic_helper.c
@@ -2215,7 +2215,9 @@ EXPORT_SYMBOL(drm_atomic_helper_wait_for_dependencies);
  * when a job is queued, and any change to the pipeline that does not touch the
  * connector is leading to timeouts when calling
  * drm_atomic_helper_wait_for_vblanks() or
- * drm_atomic_helper_wait_for_flip_done().
+ * drm_atomic_helper_wait_for_flip_done(). In addition to writeback
+ * connectors, this function can also fake VBLANK events for CRTCs without
+ * VBLANK interrupt.
  *
  * This is part of the atomic helper support for nonblocking commits, see
  * drm_atomic_helper_setup_commit() for an overview.
diff --git a/include/drm/drm_crtc.h b/include/drm/drm_crtc.h
index 5e9b15a0e8c5..01caf5160596 100644
--- a/include/drm/drm_crtc.h
+++ b/include/drm/drm_crtc.h
@@ -179,7 +179,9 @@ struct drm_crtc_state {
 * In this case the VBLANK event is only generated when a job is queued
 * to the writeback connector, and we want the core to fake VBLANK
 * events when this part of the pipeline hasn't changed but others had
-* or when the CRTC and connectors are being disabled.
+* or when the CRTC and connectors are being disabled. In addition to
+* writeback connectors, this function can also fake VBLANK events for
+* CRTCs without VBLANK interrupt.
 *
 * __drm_atomic_helper_crtc_duplicate_state() will not reset the value
 * from the current state, the CRTC driver is then responsible for
@@ -335,7 +337,10 @@ struct drm_crtc_state {
 *  - Events for disabled CRTCs are not allowed, and drivers can ignore
 *that case.
 *
-* This can be handled by the drm_crtc_send_vblank_event() function,
+* For very simple hardware without VBLANK interrupt, enabling
+*  drm_crtc_state.no_vblank makes DRM's atomic commit helpers
+* send the event at an appropriate time. For more complex hardware this
+* can be handled by the drm_crtc_send_vblank_event() function,
 * which the driver should call on the provided event upon completion of
 * the atomic commit. Note that if the driver supports vblank signalling
 * and timestamping the vblank counters and timestamps must agree with
diff --git a/include/drm/drm_simple_kms_helper.h 
b/include/drm/drm_simple_kms_helper.h
index 15afee9cf049..e253ba7bea9d 100644
--- a/include/drm/drm_simple_kms_helper.h
+++ b/include/drm/drm_simple_kms_helper.h
@@ -100,8 +100,11 @@ struct drm_simple_display_pipe_funcs {
 * This is the function drivers should submit the
 * _pending_vblank_event from. Using either
 * drm_crtc_arm_vblank_event(), when the driver supports vblank
-* interrupt handling, or drm_crtc_send_vblank_event() directly in case
-* the hardware lacks vblank support entirely.
+* interrupt handling, or drm_crtc_send_vblank_event() for more
+* complex case. In case the hardware lacks vblank support entirely,
+* drivers can set  drm_crtc_state.no_vblank in
+*  drm_simple_display_pipe_funcs.check and let DRM's
+* atomic helper fake a vblank event.
 */
void (*update)(struct drm_simple_display_pipe *pipe,
   struct drm_plane_state *old_plane_state);
-- 
2.24.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] live migration from 4.12 to 4.13 fails due to qemu-xen bug

2020-01-13 Thread Olaf Hering
I did not find anything in the Xen 4.13 release notes, so I'm asking here:

This HVM domU fails to live migrate from staging-4.12 to staging-4.13:

name='hvm'
serial='pty'
vcpus='4'
memory='888'
disk=[ 'vdev=xvda, format=raw, access=rw, target=/netshare/disk0.raw' ]
vif=[ 'bridge=br0,mac=3c:27:63:58:ca:35' ]
builder="hvm"
device_model_version="qemu-xen"

The receiving qemu fails like that:

char device redirected to /dev/pts/3 (label serial0)
xen_ram_alloc: do not alloc 3700 bytes of ram at 0 when runstate is 
INMIGRATE
xen_ram_alloc: do not alloc 80 bytes of ram at 3700 when runstate is 
INMIGRATE
xen_ram_alloc: do not alloc 1 bytes of ram at 3780 when runstate is 
INMIGRATE
xen_ram_alloc: do not alloc 4 bytes of ram at 3784 when runstate is 
INMIGRATE
VNC server running on 127.0.0.1:5900
qemu-system-i386: Unknown savevm section type 111
qemu-system-i386: load of migration failed: Invalid argument

Google does not seem to know this specific error.
In fact, every upstream qemu-3.x fails to migrate to qemu-4.x.
Does anyone know what incompatible change was done to qemu.git?

Olaf


pgp2Hrba6E2Fw.pgp
Description: Digitale Signatur von OpenPGP
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH V7 2/4] x86/altp2m: Add hypercall to set a range of sve bits

2020-01-13 Thread Alexandru Stefan ISAILA


On 10.01.2020 18:20, Jan Beulich wrote:
> On 08.01.2020 15:08, Alexandru Stefan ISAILA wrote:
>> By default the sve bits are not set.
>> This patch adds a new hypercall, xc_altp2m_set_supress_ve_multi(),
>> to set a range of sve bits.
>> The core function, p2m_set_suppress_ve_multi(), does not break in case
>> of a error and it is doing a best effort for setting the bits in the
>> given range. A check for continuation is made in order to have
>> preemption on large ranges.
>> The gfn of the first error is stored in
>> xen_hvm_altp2m_suppress_ve_multi.first_error and the error code is
>> stored in xen_hvm_altp2m_suppress_ve_multi.first_error_code.
>> If no error occurred the values will be 0.
> 
> These last two sentences must have been stale for a while. IOW ...

I just saw that now, yes you are right, from when I changed the field names.

> 
>> Changes since V6:
>>  - Fix commit message
> 
> ... has this really happened?

This was done for the "braek" typo and for large/big ranges but I will 
fix the field names as well, thanks for pointing that out.

> 
>> --- a/xen/arch/x86/mm/p2m.c
>> +++ b/xen/arch/x86/mm/p2m.c
>> @@ -3030,45 +3030,87 @@ out:
>>*/
>>   int p2m_set_suppress_ve(struct domain *d, gfn_t gfn, bool suppress_ve,
>>   unsigned int altp2m_idx)
>> +{
>> +int rc;
>> +struct xen_hvm_altp2m_suppress_ve_multi sve = {0};
>> +
>> +sve.view = altp2m_idx;
>> +sve.suppress_ve = suppress_ve;
>> +sve.first_gfn = gfn_x(gfn);
>> +sve.last_gfn = gfn_x(gfn);
> 
> Can't all of these move into the initializer, instead of the
> somewhat odd 0?

Sure, it's no trouble.

> 
>> +if ( !(rc = p2m_set_suppress_ve_multi(d, )) && sve.first_error )
>> +rc = sve.first_error;
> 
> Why the right side of the && ?

This is intended to have p2m_set_suppress_ve() call 
p2m_set_suppress_ve_multi(). So here first I call the _multi version and 
the check if there was any error from the set/get (that is what 
p2m_set_suppress_ve did before). I don't know why git made the patch so 
ugly.

> 
>> +/*
>> + * Set/clear the #VE suppress bit for multiple pages.  Only available on 
>> VMX.
>> + */
>> +int p2m_set_suppress_ve_multi(struct domain *d,
>> +  struct xen_hvm_altp2m_suppress_ve_multi *sve)
>>   {
>>   struct p2m_domain *host_p2m = p2m_get_hostp2m(d);
>>   struct p2m_domain *ap2m = NULL;
>> -struct p2m_domain *p2m;
>> -mfn_t mfn;
>> -p2m_access_t a;
>> -p2m_type_t t;
>> -int rc;
>> +struct p2m_domain *p2m = host_p2m;
>> +uint64_t start = sve->first_gfn;
>> +int rc = 0;
>>   
>> -if ( altp2m_idx > 0 )
>> +if ( sve->view > 0 )
>>   {
>> -if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
>> - d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] 
>> ==
>> +if ( sve->view >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
>> + d->arch.altp2m_eptp[array_index_nospec(sve->view, MAX_EPTP)] ==
>>mfn_x(INVALID_MFN) )
>>   return -EINVAL;
>>   
>> -p2m = ap2m = d->arch.altp2m_p2m[array_index_nospec(altp2m_idx,
>> +p2m = ap2m = d->arch.altp2m_p2m[array_index_nospec(sve->view,
>>   ARRAY_SIZE(d->arch.altp2m_p2m))];
>>   }
>> -else
>> -p2m = host_p2m;
>>   
>> -gfn_lock(host_p2m, gfn, 0);
>> +p2m_lock(host_p2m);
>>   
>>   if ( ap2m )
>>   p2m_lock(ap2m);
>>   
>> -rc = altp2m_get_effective_entry(p2m, gfn, , , , AP2MGET_query);
>> +while ( sve->last_gfn >= start )
>> +{
>> +p2m_access_t a;
>> +p2m_type_t t;
>> +mfn_t mfn;
>> +int err = 0;
>>   
>> -if ( rc )
>> -goto out;
>> +if ( (err = altp2m_get_effective_entry(p2m, _gfn(start), , , 
>> ,
>> +   AP2MGET_query)) &&
>> + !sve->first_error )
>> +{
>> +sve->first_error_gfn = start; /* Save the gfn of the first 
>> error */
>> +sve->first_error = err; /* Save the first error code */
>> +}
>>   
>> -rc = p2m->set_entry(p2m, gfn, mfn, PAGE_ORDER_4K, t, a, suppress_ve);
>> +if ( !err && (err = p2m->set_entry(p2m, _gfn(start), mfn,
>> +   PAGE_ORDER_4K, t, a,
>> +   sve->suppress_ve)) &&
>> + !sve->first_error )
>> +{
>> +sve->first_error_gfn = start; /* Save the gfn of the first 
>> error */
>> +sve->first_error = err; /* Save the first error code */
>> +}
> 
> I think it would help readability if you didn't do this error
> saving twice.
> 


George requested this as well as having the single version call the 
_multi version.

Alex
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org

Re: [Xen-devel] [RFC PATCH V2 11/11] x86: tsc: avoid system instability in hibernation

2020-01-13 Thread Peter Zijlstra
On Fri, Jan 10, 2020 at 07:35:20AM -0800, Eduardo Valentin wrote:
> Hey Peter,
> 
> On Wed, Jan 08, 2020 at 11:50:11AM +0100, Peter Zijlstra wrote:
> > On Tue, Jan 07, 2020 at 11:45:26PM +, Anchal Agarwal wrote:
> > > From: Eduardo Valentin 
> > > 
> > > System instability are seen during resume from hibernation when system
> > > is under heavy CPU load. This is due to the lack of update of sched
> > > clock data, and the scheduler would then think that heavy CPU hog
> > > tasks need more time in CPU, causing the system to freeze
> > > during the unfreezing of tasks. For example, threaded irqs,
> > > and kernel processes servicing network interface may be delayed
> > > for several tens of seconds, causing the system to be unreachable.
> > 
> > > The fix for this situation is to mark the sched clock as unstable
> > > as early as possible in the resume path, leaving it unstable
> > > for the duration of the resume process. This will force the
> > > scheduler to attempt to align the sched clock across CPUs using
> > > the delta with time of day, updating sched clock data. In a post
> > > hibernation event, we can then mark the sched clock as stable
> > > again, avoiding unnecessary syncs with time of day on systems
> > > in which TSC is reliable.
> > 
> > This makes no frigging sense what so bloody ever. If the clock is
> > stable, we don't care about sched_clock_data. When it is stable you get
> > a linear function of the TSC without complicated bits on.
> > 
> > When it is unstable, only then do we care about the sched_clock_data.
> > 
> 
> Yeah, maybe what is not clear here is that we covering for situation
> where clock stability changes over time, e.g. at regular boot clock is
> stable, hibernation happens, then restore happens in a non-stable clock.

Still confused, who marks the thing unstable? The patch seems to suggest
you do yourself, but it is not at all clear why.

If TSC really is unstable, then it needs to remain unstable. If the TSC
really is stable then there is no point in marking is unstable.

Either way something is off, and you're not telling me what.

> > > Reviewed-by: Erik Quanstrom 
> > > Reviewed-by: Frank van der Linden 
> > > Reviewed-by: Balbir Singh 
> > > Reviewed-by: Munehisa Kamata 
> > > Tested-by: Anchal Agarwal 
> > > Signed-off-by: Eduardo Valentin 
> > > ---
> > 
> > NAK, the code very much relies on never getting marked stable again
> > after it gets set to unstable.
> > 
> 
> Well actually, at the PM_POST_HIBERNATION, we do the check and set stable if
> known to be stable.
> 
> The issue only really happens during the restoration path under scheduling 
> pressure,
> which takes forever to finish, as described in the commit.
> 
> Do you see a better solution for this issue?

I still have no clue what your actual problem is. You say scheduling
goes wobbly because sched_clock_data is stale, but when stable that
doesn't matter.

So what is the actual problem?

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [ovmf test] 146041: regressions - FAIL

2020-01-13 Thread osstest service owner
flight 146041 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146041/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail REGR. vs. 145767
 test-amd64-amd64-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail REGR. vs. 
145767

version targeted for testing:
 ovmf 4465cd124fbcf5490faad6a1a834299b30b5d009
baseline version:
 ovmf 70911f1f4aee0366b6122f2b90d367ec0f066beb

Last test of basis   145767  2020-01-08 00:39:09 Z5 days
Failing since145774  2020-01-08 02:50:20 Z5 days   29 attempts
Testing same since   146041  2020-01-13 04:39:23 Z0 days1 attempts


People who touched revisions under test:
  Albecki, Mateusz 
  Ard Biesheuvel 
  Ashish Singhal 
  Eric Dong 
  Fan, ZhijuX 
  Jason Voelz 
  Laszlo Ersek 
  Mateusz Albecki 
  Pavana.K 
  Philippe Mathieu-Daud? 
  Philippe Mathieu-Daude 
  Siyuan Fu 
  Siyuan, Fu 
  Vitaly Cheptsov 
  Vitaly Cheptsov via Groups.Io 
  Zhiju.Fan 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 fail
 test-amd64-i386-xl-qemuu-ovmf-amd64  fail



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 422 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v13 0/5] xenbus/backend: Add memory pressure handler callback

2020-01-13 Thread SeongJae Park
On Mon, 13 Jan 2020 10:55:07 +0100 "Roger Pau Monné"  
wrote:

> On Mon, Jan 13, 2020 at 10:49:52AM +0100, SeongJae Park wrote:
> > Every patch of this patchset got at least one 'Reviewed-by' or 'Acked-by' 
> > from
> > appropriate maintainers by last Wednesday, and after that, got no comment 
> > yet.
> > May I ask some more comments?
> 
> I'm not sure why more comments are needed, patches have all the
> relevant Acks and will be pushed in due time unless someone has
> objections.
> 
> Please be patient and wait at least until the next merge window, this
> patches are not bug fixes so pushing them now would be wrong.

Ok, I will.  Thank you for your quick and nice reply.


Thanks,
SeongJae Park

> 
> Roger.
> 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v13 0/5] xenbus/backend: Add memory pressure handler callback

2020-01-13 Thread Roger Pau Monné
On Mon, Jan 13, 2020 at 10:49:52AM +0100, SeongJae Park wrote:
> Every patch of this patchset got at least one 'Reviewed-by' or 'Acked-by' from
> appropriate maintainers by last Wednesday, and after that, got no comment yet.
> May I ask some more comments?

I'm not sure why more comments are needed, patches have all the
relevant Acks and will be pushed in due time unless someone has
objections.

Please be patient and wait at least until the next merge window, this
patches are not bug fixes so pushing them now would be wrong.

Roger.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v13 0/5] xenbus/backend: Add memory pressure handler callback

2020-01-13 Thread SeongJae Park
Every patch of this patchset got at least one 'Reviewed-by' or 'Acked-by' from
appropriate maintainers by last Wednesday, and after that, got no comment yet.
May I ask some more comments?


Thanks,
SeongJae Park

On Wed, 18 Dec 2019 19:37:13 +0100 SeongJae Park  wrote:

> Granting pages consumes backend system memory.  In systems configured
> with insufficient spare memory for those pages, it can cause a memory
> pressure situation.  However, finding the optimal amount of the spare
> memory is challenging for large systems having dynamic resource
> utilization patterns.  Also, such a static configuration might lack
> flexibility.
> 
> To mitigate such problems, this patchset adds a memory reclaim callback
> to 'xenbus_driver' (patch 1) and then introduce a lock for race
> condition avoidance (patch 2).  After that, patch 3 applies the callback
> mechanism to mitigate the problem in 'xen-blkback'.  The fourth and
> fifth patches are trivial cleanups; those fix nits we found during the
> development of this patchset.
> 
> Note that patches 1, 4, and 5 are not changed since v9.
> 
> 
> Base Version
> 
> 
> This patch is based on v5.4.  A complete tree is also available at my
> public git repo:
> https://github.com/sjp38/linux/tree/patches/blkback/buffer_squeeze/v13
> 
> 
> Patch History
> -
> 
> Changes from v12
> (https://lore.kernel.org/xen-devel/20191218104232.9606-1-sjp...@amazon.com/)
>  - Do not unnecessarily disable interrupts (suggested by Juergen)
>  - Hold lock from xenbus side (suggested by Juergen)
> 
> Changes from v11
> (https://lore.kernel.org/xen-devel/20191217160748.693-2-sjp...@amazon.com/)
>  - Fix wrong trylock use (reported by Juergen)
>  - Merge patch 3 and 4 (suggested by Juergen)
>  - Update test result
> 
> Changes from v10
> (https://lore.kernel.org/xen-devel/20191216124527.30306-1-sjp...@amazon.com/)
>  - Fix race condition (reported by SeongJae, suggested by Juergen)
> 
> Changes from v9
> (https://lore.kernel.org/xen-devel/20191213153546.17425-1-sjp...@amazon.de/)
>  - Add 'Reviewed-by' and 'Acked-by' from Roger Pau Monné
>  - Update the commit message for overhead test of the 2nd path
> 
> Changes from v8
> (https://lore.kernel.org/xen-devel/20191213130211.24011-1-sjp...@amazon.de/)
>  - Drop 'Reviewed-by: Juergen' from the second patch
>(suggested by Roger Pau Monné)
>  - Update contact of the new module param to SeongJae Park
>
>(suggested by Roger Pau Monné)
>  - Wordsmith the description of the parameter
>(suggested by Roger Pau Monné)
>  - Fix dumb bugs
>(suggested by Roger Pau Monné)
>  - Move module param definition to xenbus.c and reduce the number of
>lines for this change
>(suggested by Roger Pau Monné)
>  - Add a comment for the new callback, reclaim_memory, as other
>callbacks also have
>  - Add another trivial cleanup of xenbus.c file (4th patch)
> 
> Changes from v7
> (https://lore.kernel.org/xen-devel/20191211181016.14366-1-sjp...@amazon.de/)
>  - Update sysfs-driver-xen-blkback for new parameter
>(suggested by Roger Pau Monné)
>  - Use per-xen_blkif buffer_squeeze_end instead of global variable
>(suggested by Roger Pau Monné)
> 
> Changes from v6
> (https://lore.kernel.org/linux-block/20191211042428.5961-1-sjp...@amazon.de/)
>  - Remove more unnecessary prefixes (suggested by Roger Pau Monné)
>  - Constify a variable (suggested by Roger Pau Monné)
>  - Rename 'reclaim' into 'reclaim_memory' (suggested by Roger Pau Monné)
>  - More wordsmith of the commit message (suggested by Roger Pau Monné)
> 
> Changes from v5
> (https://lore.kernel.org/linux-block/20191210080628.5264-1-sjp...@amazon.de/)
>  - Wordsmith the commit messages (suggested by Roger Pau Monné)
>  - Change the reclaim callback return type (suggested by Roger Pau
>Monné)
>  - Change the type of the blkback squeeze duration variable
>(suggested by Roger Pau Monné)
>  - Add a patch for removal of unnecessary static variable name prefixes
>(suggested by Roger Pau Monné)
>  - Fix checkpatch.pl warnings
> 
> Changes from v4
> (https://lore.kernel.org/xen-devel/20191209194305.20828-1-sjp...@amazon.com/)
>  - Remove domain id parameter from the callback (suggested by Juergen
>Gross)
>  - Rename xen-blkback module parameter (suggested by Stefan Nuernburger)
> 
> Changes from v3
> (https://lore.kernel.org/xen-devel/20191209085839.21215-1-sjp...@amazon.com/)
>  - Add general callback in xen_driver and use it (suggested by Juergen
>Gross)
> 
> Changes from v2
> (https://lore.kernel.org/linux-block/af195033-23d5-38ed-b73b-f6e2e3b34...@amazon.com)
>  - Rename the module parameter and variables for brevity
>(aggressive shrinking -> squeezing)
> 
> Changes from v1
> (https://lore.kernel.org/xen-devel/20191204113419.2298-1-sjp...@amazon.com/)
>  - Adjust the description to not use the term, `arbitrarily`
>(suggested by Paul Durrant)
>  - Specify time unit of the duration in the parameter description,
>(suggested by 

[Xen-devel] [qemu-mainline test] 146042: regressions - FAIL

2020-01-13 Thread osstest service owner
flight 146042 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146042/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm   6 xen-buildfail REGR. vs. 144861
 build-arm64   6 xen-buildfail REGR. vs. 144861
 build-amd64   6 xen-buildfail REGR. vs. 144861
 build-i386-xsm6 xen-buildfail REGR. vs. 144861
 build-armhf   6 xen-buildfail REGR. vs. 144861
 build-amd64-xsm   6 xen-buildfail REGR. vs. 144861
 build-i3866 xen-buildfail REGR. vs. 144861

Tests which did not succeed, but are not blocking:
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-i386-xl-shadow 1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvshim1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-cubietruck  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-rtds  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)  blocked n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-shadow1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1) blocked n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-pvshim 1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 

Re: [Xen-devel] [PATCH v6 11/11] xen: introduce ERRP_AUTO_PROPAGATE

2020-01-13 Thread Vladimir Sementsov-Ogievskiy
13.01.2020 11:57, Paul Durrant wrote:
> On Fri, 10 Jan 2020 at 19:42, Vladimir Sementsov-Ogievskiy
>  wrote:
>>
>> If we want to add some info to errp (by error_prepend() or
>> error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
>> Otherwise, this info will not be added when errp == _fatal
>> (the program will exit prior to the error_append_hint() or
>> error_prepend() call).  Fix such cases.
>>
>> If we want to check error after errp-function call, we need to
>> introduce local_err and then propagate it to errp. Instead, use
>> ERRP_AUTO_PROPAGATE macro, benefits are:
>> 1. No need of explicit error_propagate call
>> 2. No need of explicit local_err variable: use errp directly
>> 3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
>> _fatal, this means that we don't break error_abort
>> (we'll abort on error_set, not on error_propagate)
>>
>> This commit is generated by command
>>
>>  sed -n '/^X86 Xen CPUs$/,/^$/{s/^F: //p}' MAINTAINERS | \
>>  xargs git ls-files | grep '\.[hc]$' | \
>>  xargs spatch \
>>  --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>>  --macro-file scripts/cocci-macro-file.h \
>>  --in-place --no-show-diff --max-width 80
>>
>> Reported-by: Kevin Wolf 
>> Reported-by: Greg Kurz 
>> Signed-off-by: Vladimir Sementsov-Ogievskiy 
> 
> Acked-by: Paul Durrant 
> 

Thanks!

-- 
Best regards,
Vladimir
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v6 02/11] error: auto propagated local_err

2020-01-13 Thread Vladimir Sementsov-Ogievskiy
13.01.2020 11:50, Paul Durrant wrote:
> On Fri, 10 Jan 2020 at 19:42, Vladimir Sementsov-Ogievskiy
>  wrote:
> [snip]
>> +/*
>> + * ERRP_AUTO_PROPAGATE
>> + *
>> + * This macro is created to be the first line of a function which use
>> + * Error **errp parameter to report error. It's needed only in cases where 
>> we
>> + * want to use error_prepend, error_append_hint or dereference *errp. It's
>> + * still safe (but useless) in other cases.
>> + *
>> + * If errp is NULL or points to error_fatal, it is rewritten to point to a
>> + * local Error object, which will be automatically propagated to the 
>> original
>> + * errp on function exit (see error_propagator_cleanup).
>> + *
>> + * After invocation of this macro it is always safe to dereference errp
>> + * (as it's not NULL anymore) and to add information (by error_prepend or
>> + * error_append_hint)
>> + * (as, if it was error_fatal, we swapped it with a local_error to be
>> + * propagated on cleanup).
>> + *
>> + * Note: we don't wrap the error_abort case, as we want resulting coredump
>> + * to point to the place where the error happened, not to error_propagate.
>> + */
>> +#define ERRP_AUTO_PROPAGATE()  \
>> +g_auto(ErrorPropagator) _auto_errp_prop = {.errp = errp};  \
>> +errp = ((errp == NULL || *errp == error_fatal) \
> 
> Perhaps !errp rather than errp == NULL, for brevity.
> 

I mostly prefer !ptr notation.. But may be here, I'd keep it as is,
to stress special-casing NULL in this non-trivial place.. And it is in good
relation with phrasing "If errp is NULL or points to error_fatal".
But !errp is OK for me to. Let it be as Markus prefer, he is maintainer.

> 
>> +? &_auto_errp_prop.local_err : errp)
>> +
>>   /*
>>* Special error destination to abort on error.
>>* See error_setg() and error_propagate() for details.
>> --
>> 2.21.0
>>


-- 
Best regards,
Vladimir
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v6 11/11] xen: introduce ERRP_AUTO_PROPAGATE

2020-01-13 Thread Paul Durrant
On Fri, 10 Jan 2020 at 19:42, Vladimir Sementsov-Ogievskiy
 wrote:
>
> If we want to add some info to errp (by error_prepend() or
> error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
> Otherwise, this info will not be added when errp == _fatal
> (the program will exit prior to the error_append_hint() or
> error_prepend() call).  Fix such cases.
>
> If we want to check error after errp-function call, we need to
> introduce local_err and then propagate it to errp. Instead, use
> ERRP_AUTO_PROPAGATE macro, benefits are:
> 1. No need of explicit error_propagate call
> 2. No need of explicit local_err variable: use errp directly
> 3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
>_fatal, this means that we don't break error_abort
>(we'll abort on error_set, not on error_propagate)
>
> This commit is generated by command
>
> sed -n '/^X86 Xen CPUs$/,/^$/{s/^F: //p}' MAINTAINERS | \
> xargs git ls-files | grep '\.[hc]$' | \
> xargs spatch \
> --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
> --macro-file scripts/cocci-macro-file.h \
> --in-place --no-show-diff --max-width 80
>
> Reported-by: Kevin Wolf 
> Reported-by: Greg Kurz 
> Signed-off-by: Vladimir Sementsov-Ogievskiy 

Acked-by: Paul Durrant 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v6 02/11] error: auto propagated local_err

2020-01-13 Thread Paul Durrant
On Fri, 10 Jan 2020 at 19:42, Vladimir Sementsov-Ogievskiy
 wrote:
[snip]
> +/*
> + * ERRP_AUTO_PROPAGATE
> + *
> + * This macro is created to be the first line of a function which use
> + * Error **errp parameter to report error. It's needed only in cases where we
> + * want to use error_prepend, error_append_hint or dereference *errp. It's
> + * still safe (but useless) in other cases.
> + *
> + * If errp is NULL or points to error_fatal, it is rewritten to point to a
> + * local Error object, which will be automatically propagated to the original
> + * errp on function exit (see error_propagator_cleanup).
> + *
> + * After invocation of this macro it is always safe to dereference errp
> + * (as it's not NULL anymore) and to add information (by error_prepend or
> + * error_append_hint)
> + * (as, if it was error_fatal, we swapped it with a local_error to be
> + * propagated on cleanup).
> + *
> + * Note: we don't wrap the error_abort case, as we want resulting coredump
> + * to point to the place where the error happened, not to error_propagate.
> + */
> +#define ERRP_AUTO_PROPAGATE()  \
> +g_auto(ErrorPropagator) _auto_errp_prop = {.errp = errp};  \
> +errp = ((errp == NULL || *errp == error_fatal) \

Perhaps !errp rather than errp == NULL, for brevity.

  Paul

> +? &_auto_errp_prop.local_err : errp)
> +
>  /*
>   * Special error destination to abort on error.
>   * See error_setg() and error_propagate() for details.
> --
> 2.21.0
>

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

  1   2   >