[Xen-devel] [ovmf test] 99761: regressions - FAIL

[osstest service owner]

flight 99761 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/99761/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm   5 xen-build fail REGR. vs. 99721

version targeted for testing:
 ovmf 58a4bff071832e587d18f97597b5d571dcebc9d7
baseline version:
 ovmf 39dbc4d5534790b5efcd67ce6b0f82ac23c6db6d

Last test of basis99721  2016-07-27 18:00:18 Z2 days
Testing same since99761  2016-07-28 18:15:40 Z1 days1 attempts


People who touched revisions under test:
  Jeremy Linton 

jobs:
 build-amd64-xsm  fail
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.


commit 58a4bff071832e587d18f97597b5d571dcebc9d7
Author: Jeremy Linton 
Date:   Wed Jul 27 14:24:36 2016 -0500

ArmPlatformPkg: Convert ArmJunoDxe to use common juno revision code

Now that the code to detect the Juno revision is in
the header go ahead and covert the ArmJunoDxe to use it.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jeremy Linton 
Reviewed-by: Leif Lindholm 

commit 7ac29b544f4fd2f9d604e1fb39ea83c6f538a6d2
Author: Jeremy Linton 
Date:   Wed Jul 27 14:24:35 2016 -0500

ArmPlatformPkg: break out juno revision detection

The code to detect what juno revision we are running on
is fairly small put it in a common header where it may be
used in a couple places.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jeremy Linton 
Reviewed-by: Leif Lindholm 

commit 25654e2469d5387740addd4a2d24f61dd665a3b0
Author: Jeremy Linton 
Date:   Wed Jul 27 14:24:34 2016 -0500

ArmPkg: Add Cortex-A72 CPU type

Add the Cortex-A72 CPU type which is used in JunoR2.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jeremy Linton 
Reviewed-by: Leif Lindholm 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-4.4-testing test] 99757: tolerable FAIL - PUSHED

[osstest service owner]

flight 99757 xen-4.4-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/99757/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl   3 host-install(3)  broken in 99711 pass in 99757
 test-amd64-i386-pv3 host-install(3)  broken in 99711 pass in 99757
 test-amd64-i386-xl-raw3 host-install(3)  broken in 99711 pass in 99757
 test-amd64-amd64-xl-qemuu-win7-amd64 3 host-install(3) broken in 99711 pass in 
99757
 test-amd64-amd64-xl-qemut-debianhvm-amd64 15 guest-localmigrate/x10 fail pass 
in 99711

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail like 95550
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop  fail like 95615
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop  fail like 95615
 test-armhf-armhf-xl-multivcpu 15 guest-start/debian.repeatfail  like 95615
 test-amd64-i386-xend-qemut-winxpsp3  9 windows-install fail like 95615

Tests which did not succeed, but are not blocking:
 test-amd64-i386-rumpuserxen-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-rumpuserxen-amd64  1 build-check(1)   blocked n/a
 build-amd64-rumpuserxen   6 xen-buildfail   never pass
 build-i386-rumpuserxen6 xen-buildfail   never pass
 test-armhf-armhf-libvirt-qcow2  9 debian-di-installfail never pass
 test-armhf-armhf-xl-vhd   9 debian-di-installfail   never pass
 test-armhf-armhf-libvirt-raw  9 debian-di-installfail   never pass
 test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stop fail never pass
 test-armhf-armhf-libvirt 11 guest-start  fail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass

version targeted for testing:
 xen  0fe7d6961755812503694e9a4741b5f35a09d1f7
baseline version:
 xen  36a5a8785065ad4e3110a4bd30967b1410f99138

Last test of basis95615  2016-06-12 17:47:03 Z   47 days
Testing same since99711  2016-07-27 17:59:34 Z2 days2 attempts


People who touched revisions under test:
  Andrew Cooper 

jobs:
 build-amd64-xend pass
 build-i386-xend  pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 build-amd64-rumpuserxen  fail
 build-i386-rumpuserxen   fail
 test-amd64-amd64-xl  pass
 test-armhf-armhf-xl  pass
 test-amd64-i386-xl   pass
 test-amd64-amd64-qemuu-nested-amdfail
 test-amd64-i386-qemut-rhel6hvm-amd   pass
 test-amd64-i386-qemuu-rhel6hvm-amd   pass
 test-amd64-amd64-xl-qemut-debianhvm-amd64fail
 test-amd64-i386-xl-qemut-debianhvm-amd64 pass
 test-amd64-amd64-xl-qemuu-debian

[Xen-devel] [xen-4.5-testing baseline-only test] 66863: trouble: blocked/broken/fail/pass

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 66863 xen-4.5-testing real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/66863/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemut-debianhvm-amd64 3 host-install(3) broken REGR. vs. 
66514
 test-amd64-amd64-i386-pvgrub  3 host-install(3) broken REGR. vs. 66514
 test-amd64-amd64-libvirt  3 host-install(3) broken REGR. vs. 66514
 build-i3863 host-install(3) broken REGR. vs. 66514
 test-amd64-amd64-xl-qemut-win7-amd64  3 host-install(3) broken REGR. vs. 66514
 test-amd64-amd64-libvirt-vhd  3 host-install(3) broken REGR. vs. 66514
 test-amd64-amd64-xl-qemut-winxpsp3  3 host-install(3)   broken REGR. vs. 66514
 test-amd64-amd64-migrupgrade 4 host-install/dst_host(4) broken REGR. vs. 66514

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds  3 host-install(3) broken REGR. vs. 66514
 test-amd64-amd64-pair 4 host-install/dst_host(4) broken like 66514
 test-amd64-amd64-libvirt-pair  4 host-install/dst_host(4)broken like 66514
 test-amd64-amd64-rumpuserxen-amd64 15 
rumpuserxen-demo-xenstorels/xenstorels.repeat fail REGR. vs. 66514
 test-amd64-amd64-qemuu-nested-intel 13 xen-boot/l1 fail like 66514
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail like 66514
 test-amd64-amd64-xl-qemuu-winxpsp3  9 windows-install  fail like 66514

Tests which did not succeed, but are not blocking:
 build-i386-rumpuserxen1 build-check(1)   blocked  n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1  1 build-check(1) blocked n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-winxpsp3  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemut-winxpsp3  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemut-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-qemut-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemut-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-qemut-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-i386-rumpuserxen-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 test-amd64-i386-migrupgrade   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-armhf-armhf-libvirt-qcow2 10 guest-start  fail never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-midway   13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-midway   12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 10 guest-start  fail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  10 guest-start  fail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass

version targeted for testing:
 xen  c4c0312efaf8

[Xen-devel] [xen-4.7-testing test] 99754: tolerable FAIL - PUSHED

[osstest service owner]

flight 99754 xen-4.7-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/99754/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop  fail like 96660
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop  fail like 96660

Tests which did not succeed, but are not blocking:
 test-amd64-i386-rumpuserxen-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-rumpuserxen-amd64  1 build-check(1)   blocked n/a
 build-i386-rumpuserxen6 xen-buildfail   never pass
 build-amd64-rumpuserxen   6 xen-buildfail   never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stop fail never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  899495b60a6e55fc2afa69d4616cb08af212de12
baseline version:
 xen  a492556c40aaf5a84fe0ecb1d4d865bb20cec4d3

Last test of basis96660  2016-07-05 01:11:46 Z   25 days
Testing same since99713  2016-07-27 17:59:11 Z2 days2 attempts


People who touched revisions under test:
  Andrew Cooper 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-prev pass
 build-i386-prev  pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 build-amd64-rum

[Xen-devel] [PATCH] x86/vMsi-x: check whether msixtbl_list in msixtbl_pt_register()

[Chao Gao]

MSI-x tables' initializtion had been deferred in the commit
74c6dc2d0ac4dcab0c6243cdf6ed550c1532b798. If an assigned device does not support
MSI-x, the msixtbl_list won't be initialized. However, the following paths
XEN_DOMCTL_bind_pt_irq
pt_irq_create_bind
msixtbl_pt_register
do not check this case. Some errors(malwares, etc.) may lead to calling
XEN_DOMCTL_bind_pt_irq without a clear gtable and will cause Xen panic.

Signed-off-by: Chao Gao 
---
 xen/arch/x86/hvm/vmsi.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
index ef1dfff..d81c5d4 100644
--- a/xen/arch/x86/hvm/vmsi.c
+++ b/xen/arch/x86/hvm/vmsi.c
@@ -459,7 +459,7 @@ int msixtbl_pt_register(struct domain *d, struct pirq 
*pirq, uint64_t gtable)
 ASSERT(pcidevs_locked());
 ASSERT(spin_is_locked(&d->event_lock));
 
-if ( !has_vlapic(d) )
+if ( !msixtbl_initialised(d) )
 return -ENODEV;
 
 /*
-- 
1.8.3.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [linux-3.14 baseline-only test] 66862: trouble: blocked/broken/fail/pass

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 66862 linux-3.14 real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/66862/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-libvirt   3 host-install(3) broken REGR. vs. 66493
 test-amd64-i386-qemuu-rhel6hvm-intel  3 host-install(3) broken REGR. vs. 66493
 test-amd64-i386-xl-qemut-winxpsp3  3 host-install(3)broken REGR. vs. 66493
 test-amd64-i386-xl-qemuu-debianhvm-amd64 3 host-install(3) broken REGR. vs. 
66493
 test-amd64-i386-libvirt-xsm   3 host-install(3) broken REGR. vs. 66493
 test-amd64-amd64-xl-qemuu-debianhvm-amd64 3 host-install(3) broken REGR. vs. 
66493
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 3 host-install(3) broken 
REGR. vs. 66493
 test-amd64-amd64-xl-qcow2 3 host-install(3) broken REGR. vs. 66493
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm 3 host-install(3) broken REGR. 
vs. 66493
 test-amd64-i386-xl-qemuu-win7-amd64  3 host-install(3)  broken REGR. vs. 66493
 test-amd64-i386-xl-qemut-win7-amd64  3 host-install(3)  broken REGR. vs. 66493
 test-amd64-i386-qemut-rhel6hvm-amd  3 host-install(3)   broken REGR. vs. 66493
 test-amd64-i386-xl3 host-install(3) broken REGR. vs. 66493
 test-amd64-amd64-xl-qemut-winxpsp3  3 host-install(3)   broken REGR. vs. 66493
 test-amd64-amd64-xl-credit2   3 host-install(3) broken REGR. vs. 66493
 test-amd64-amd64-libvirt-vhd  3 host-install(3) broken REGR. vs. 66493
 test-amd64-amd64-xl-qemuu-winxpsp3  3 host-install(3)   broken REGR. vs. 66493
 test-amd64-amd64-amd64-pvgrub  3 host-install(3)broken REGR. vs. 66493
 test-amd64-i386-pair 4 host-install/dst_host(4) broken REGR. vs. 66493
 test-amd64-amd64-libvirt-pair 3 host-install/src_host(3) broken REGR. vs. 66493
 test-amd64-amd64-pair3 host-install/src_host(3) broken REGR. vs. 66493

Regressions which are regarded as allowable (not blocking):
 build-amd64-rumpuserxen   6 xen-buildfail   like 66493
 build-i386-rumpuserxen6 xen-buildfail   like 66493
 test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail like 66493
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stop fail like 66493
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail like 66493

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-rumpuserxen-amd64  1 build-check(1)   blocked n/a
 test-amd64-i386-rumpuserxen-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass

version targeted for testing:
 linuxda99423b3cd3e48c42c0d64b79aba58d828f9648
baseline version:
 linux44dd5e6b1cf505485d839bd62d47e29a36232d67

Last test of basis66493  2016-07-01 10:54:57 Z   28 days
Testing same since66862  2016-07-29 20:17:38 Z0 days1 attempts


People who touched revisions under test:
  Al Viro 
  Alan Stern 
  Alex Deucher 
  Andreas Gruenbacher 
  Andrew Goodbody 
  Andrew Morton 
  Andrey Ryabinin 
  Anna Schumaker 
  Anthony Romano 
  Ben Hutchings 
  Bin Liu 
  Bjørn Mork 
  Bob Copeland 
  Borislav Petkov 
  Catalin Marinas 
  Christoph Hellwig 
  Crestez Dan Leonard 
  Cyril Bur 
  Dan Carpenter 
  Daniel Vetter 
  David Howells 
  David S. Miller 
  David Vrabel 
  Dmitry Torokhov 
  Dmitry Vyukov 
  Doug Ledford 
  Feng Tang 
  Fred Veldini 
  Gavin Shan 
  Greg Kroah-Hartman 
  Guilherme G. Piccoli 
  H. Peter Anvin 
  Hans de Goede 
  Herbert Xu 
  Hugh Dickins 
  Imre Palik 
  Ingo Molnar 
  J. Bruce Fields 
  James Hogan 
  Jan Beulich 
  Jan Willeke 
  Jason Gunthorpe 
  Jiri Kosina 
  Jiri Slaby 
  Johannes Berg 
  Jonathan Cameron 
  Kirill A. Shutemov 
  Linus Torvalds 
  Linus Walleij 
  Luis de Bethencourt 
  Lyude 
  Mark Brown 
  Martin K. Petersen 
  Martin Schwidefsky 
  Martin Willi 
  Masami Hiramatsu 
  Michael Ellerman 
  Namhyung Kim 
  Ole Lukoie 
  Oleg Drokin 
  Oleg Nesterov 
  Oliver Neukum 
  Palik, Imre 
  Paolo Bonzini 
  Pavel Shilovsky 
  Peter Zijlstra (Intel) 
  Richard Weinberger 
  Russell King 
  Scott Bauer 
  Simon Horman 
  Steve French 
  Steve French 
  Steven Rostedt (Red Hat) 
  Steven Rostedt 
  Takashi Iwai 
  Tom Goff 
  Trond Myklebust 
  Vlad

Re: [Xen-devel] [PATCH] x86/vMsi-x: check whether the msixtbl_list has been initialized or not when accessing it

[gao, chao]

On Fri, Jul 29, 2016 at 10:30:07AM +0100, Andrew Cooper wrote:
>On 29/07/16 02:35, Chao Gao wrote:
>> MSI-x tables' initialization had been detered in the commit
>> 74c6dc2d0ac4dcab0c6243cdf6ed550c1532b798. If an assigned device does not 
>> support
>> MSI-x, the msixtbl_list won't be initialized. Howerver, both of following 
>> paths
>> XEN_DOMCTL_bind_pt_irq
>> pt_irq_create_bind
>> msixtbl_pt_register
>> and
>> XEN_DOMCTL_unbind_pt_irq
>> pt_irq_destroy_bind
>> msixtbl_pt_unregister
>> do not check this case and will cause Xen panic consequently.
>>
>> Signed-off-by: Chao Gao 
>
>This issue was already reported and I provided a fix in
>
>https://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=db0eee0a071e2e3e18e79d21a9b1d6724edeeeb3

I'm sorry for the mistake.

>However, looking at your patch, I forgot to fix the
>msixtbl_pt_register() path, so your patch is still necessary.

Actually, the msixtbl_pt_register() path never causes a panic unless wrong 
hypercall
paramters are given. Specially, we assign a msi capable but not msi-x capable 
device
to guest, but some errors(malwares, etc.) lead to calling 
XEN_DOMCTL_bind_pt_irq 
without a clear gtable.
>Please rebase this patch onto the staging branch which has the
>aformentioned fix in, at which point it can be accepted.  Just one note.

Thanks for your advice.
>> ---
>>  xen/arch/x86/hvm/vmsi.c | 4 ++--
>>  1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
>> index e418b98..e0d710b 100644
>> --- a/xen/arch/x86/hvm/vmsi.c
>> +++ b/xen/arch/x86/hvm/vmsi.c
>> @@ -449,7 +449,7 @@ int msixtbl_pt_register(struct domain *d, struct pirq 
>> *pirq, uint64_t gtable)
>>  ASSERT(pcidevs_locked());
>>  ASSERT(spin_is_locked(&d->event_lock));
>>  
>> -if ( !has_vlapic(d) )
>> +if ( !has_vlapic(d) || !d->arch.hvm_domain.msixtbl_list.next )
>
>You can drop the vlapic() check, as it is redundant with whether msixtbl
>is enabled or not.
>
>~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-4.5-testing test] 99752: tolerable FAIL - PUSHED

[osstest service owner]

flight 99752 xen-4.5-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/99752/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-xl-qemuu-win7-amd64 13 guest-localmigrate fail REGR. vs. 96516
 test-amd64-amd64-xl-rtds  6 xen-boot fail   like 96516
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail like 96516
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop  fail like 96516
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stop fail like 96516
 test-armhf-armhf-xl-rtds 11 guest-start  fail   like 96516

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  10 guest-start  fail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 10 guest-start  fail never pass
 test-armhf-armhf-libvirt-raw 10 guest-start  fail   never pass

version targeted for testing:
 xen  c4c0312efaf8bd252ff06d55d6bf5b542a0a9421
baseline version:
 xen  eadd6636fae2abe1608207569e32c8457e37c653

Last test of basis96516  2016-07-02 03:48:30 Z   27 days
Testing same since99712  2016-07-27 17:58:29 Z2 days2 attempts


People who touched revisions under test:
  Andrew Cooper 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-prev pass
 build-i386-prev  pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 build-amd64-rumpuserxen  pass
 build-i386-rumpuserxen   pass
 test-amd64-amd64-xl  pass
 test-armhf-armhf-xl  pass
 test-amd64-i386-xl   pass
 test-amd64-amd64-qemuu-nested-amdfail
 test-amd64-amd64-xl-pvh-amd  fail
 test-amd64-i386-qemut-rhel6hvm-amd   pass
 test-amd64-i386-qemuu-rhel6hvm-amd   pass
 test-amd64-amd64-xl-qemut-debianhvm-amd64pass
 test-amd64-i386-xl-qemut-debianhvm-amd64 pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-i386-xl-qemuu-debianhvm-amd64 pass
 test-amd64-i386-freebsd10-amd64  pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass
 test-amd64-amd64-rumpuserxen-amd64   pass
 test-amd64-amd64-xl-qemut-win7-amd64 fail
 test-amd64-i386-xl-qemut-win7-amd64  fail
 test-amd64-amd64-xl-qemuu

Re: [Xen-devel] [PATCH] arm/vm_event: get/set registers

[Tamas K Lengyel]

On Thu, Jul 28, 2016 at 10:16 PM, Razvan Cojocaru
 wrote:
> On 07/29/16 00:25, Julien Grall wrote:
>>
>>
>> On 28/07/2016 22:05, Tamas K Lengyel wrote:
>>> On Thu, Jul 28, 2016 at 3:01 PM, Julien Grall 
>>> wrote:
>>> That's not how we do it with vm_event. Even on x86 we only selectively
>>> set registers using the VM_EVENT_FLAG_SET_REGISTERS flag (albeit it
>>> not being documented in the header). As for "not exposing them" it's a
>>> waste to declare separate structures for getting and setting. I'll
>>> change my mind about that if Razvan is on the side that we should
>>> start doing that, but I don't think that's the case at the moment.
>>
>> Is there any rationale to only set a subset of the information you
>> retrieved?
>
> The perennial speed optimization, but mainly that setting everything can
> have side-effects (on x86 I remember that at the time I wrote the
> initial patch this had something to do with the control registers - if
> you'd like I can try to follow the code again and try to remember what
> the exact issue was).
>
> My main use-case at the time was to simply set EIP (I needed to be able
> to skip the current instruction if it happened to be deemed to be
> malicious by the introspection engine). I believe it has been assumed at
> the time that setting the GPRS is enough, and that can be extended in
> the future by interested parties.
>

Yeap, it's pretty much the same case here, we just need to be able to
set the address of the next instruction without needing to get and set
all registers. Changing the translation table underneath the OS or
other control registers is just not necessary (even if technically it
would be possible).

Thanks,
Tamas

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] xen/arm: p2m: Don't use default access permission when shattering a superpage

[Tamas K Lengyel]

On Fri, Jul 29, 2016 at 12:53 PM, Julien Grall  wrote:
> The following message flood the console when memaccess is enabled on
> various platforms:
>
> traps.c:2510:d1v0 HSR=0x9383004f pc=0x08b7d4c4 gva=0x08eeb8e0 
> gpa=0x004903f8e0
>
> This is because a data abort from a guest was received due to a
> permission fault but memaccess thought there are no permission fault.
>
> On ARM, memaccess permissions are stored in a radix tree because there
> are not enough available bits in the p2m entry to store the access
> restriction. When memaccess is restricting the access (i.e any other
> access than p2m_access_rwx), the access will be added in the radix tree
> using the GFN as a key. This will be done for all 4KB pages.
>
> This means that memaccess has to shatter all the superpages in a given
> region to set the permission on a 4KB granularity. Currently, when a
> superpage is shattered, the new entries are using the value
> p2m->default_access which will restrict permission (because memaccess
> has been enabled). However the radix tree does not yet contain
> an entry for this GFN.
>
> If a guest VCPU is running at the same time and trying to access the
> modified region, it will result to a stage-2 permission fault. As
> the radix tree does not yet contain an entry for the GFN, memaccess will
> deduce that the fault was not valid and a data abort will be injecting
> to the guest (and crash it).
>
> Furthermore, the permission may be restricted outside of the requested
> region if it is only a subset of a 1GB/2MB superpage.
>
> The two issues can be fixed by re-using the permission of the superpage
> entry and override the necessary fields. This is not a problem because
> memaccess cannot work on superpage.
>
> Lastly, document the code which call mfn_to_p2m_entry when creating a
> the p2m entry for a table to explain that create the p2m entry to page table
> to explain that permission are ignored by the hardware (See D4.3.1 in ARM DDI
> 0487A.j). so the value of the parameter 'access' of mfn_to_p2m_entry does
> not matter.
>
> Signed-off-by: Julien Grall 

Thanks for looking into this, it is very strange that this issue has
not surfaced before as mem_access was extensively tested on the
Arndale and the XU during the 4.6 merge. Maybe it just happened that
the superpage shattering path was never hit during the tests. It still
does work fine on my Cubietruck as of the latest staging..

Tamas

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH] docs: define semantics of vncpasswd in xl.cfg

[Jim Fehlig]

A recent discussion around LSN-2016-0001 [1] included defining
the sematics of an empty string for a VNC password. It was stated
that "libxl interprets an empty password in the caller's
configuration to mean that passwordless access should be permitted".

The same applies for vncpasswd setting in xl.cfg. This patch
extends to xl.cfg documentation to define the semantics of setting
vncpasswd to an empty string.

Signed-off-by: Jim Fehlig 
---
 docs/man/xl.cfg.pod.5.in | 8 ++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/docs/man/xl.cfg.pod.5.in b/docs/man/xl.cfg.pod.5.in
index 3bb27d0..48c9c0d 100644
--- a/docs/man/xl.cfg.pod.5.in
+++ b/docs/man/xl.cfg.pod.5.in
@@ -561,7 +561,9 @@ The actual display used can be accessed with C.
 
 =item C
 
-Specifies the password for the VNC server.
+Specifies the password for the VNC server. If password is set to an
+empty string, authentication on the VNC server will be disabled
+allowing any user to connect.
 
 =item C
 
@@ -1689,7 +1691,9 @@ The actual display used can be accessed with C.
 
 =item B
 
-Specifies the password for the VNC server.
+Specifies the password for the VNC server. If password is set to an
+empty string, authentication on the VNC server will be disabled
+allowing any user to connect.
 
 =item B
 
-- 
1.8.0.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [qemu-mainline test] 99748: regressions - FAIL

[osstest service owner]

flight 99748 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/99748/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-libvirt-raw  9 debian-di-install fail REGR. vs. 99672
 test-armhf-armhf-libvirt-qcow2  9 debian-di-install   fail REGR. vs. 99672
 test-amd64-i386-libvirt-pair 22 guest-migrate/dst_host/src_host fail REGR. vs. 
99672
 test-amd64-i386-xl-qemuu-ovmf-amd64  9 debian-hvm-install fail REGR. vs. 99672

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail like 99672
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop  fail like 99672
 test-amd64-amd64-xl-rtds  9 debian-install   fail   like 99672

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass

version targeted for testing:
 qemuu21a21b853a1bb606358af61e738abfb9aecbd720
baseline version:
 qemuu2d2e632ad00d11867c6c5625605b1fbc022dd62f

Last test of basis99672  2016-07-25 21:49:19 Z4 days
Failing since 99715  2016-07-27 17:57:14 Z2 days2 attempts
Testing same since99748  2016-07-28 09:31:07 Z1 days1 attempts


People who touched revisions under test:
  Cao jin 
  Daniel P. Berrange 
  David Gibson 
  Denis V. Lunev 
  Eduardo Habkost 
  Greg Kurz 
  Igor Mammedov 
  Jeff Cody 
  Laurent Vivier 
  lviv...@redhat.com 
  Marc-André Lureau 
  Max Reitz 
  Michael Roth 
  Michael Walle 
  Peter Maydell 
  Prasanna Kumar Kalever 
  Stefan Hajnoczi 
  Thomas Huth 
  Vladimir Sementsov-Ogievskiy 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass   

Re: [Xen-devel] [RFC 13/22] xen/arm: p2m: Replace all usage of __p2m_lookup with p2m_get_entry

[Tamas K Lengyel]

On Fri, Jul 29, 2016 at 9:06 AM, Julien Grall  wrote:
>
>
> On 28/07/16 18:36, Tamas K Lengyel wrote:
>>
>> On Thu, Jul 28, 2016 at 11:29 AM, Tamas K Lengyel 
>> wrote:
>>>
>>> On Thu, Jul 28, 2016 at 8:51 AM, Julien Grall 
>>> wrote:

 __p2m_lookup is just a wrapper to p2m_get_entry.

 Signed-off-by: Julien Grall 
 Cc: Razvan Cojocaru 
 Cc: Tamas K Lengyel 

 ---
 It might be possible to rework the memaccess code to take advantage
 of all the parameters. I will defer this to the memaccess folks.
>>>
>>>
>>> Could you elaborate on what you mean?
>>>
>>
>> Never mind, I see it. Yes, doing __p2m_get_mem_access and then
>> p2m_get_entry later duplicates work. I would suggest just replacing
>> __p2m_get_mem_access with a single call to p2m_get_entry to get both
>> the type and the mem_access setting on the page in a single run.
>
>
> I am not planning to clean-up the memaccess code. Feel free to send a
> follow-up patch for that.
>

Since you are already touching this code you might as well but
whatever, up to you.

Tamas

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] mem_access: Use monitor_traps instead of mem_access_send_req

[Tamas K Lengyel]

On Fri, Jul 29, 2016 at 3:38 PM, Julien Grall  wrote:
>
>
> On 29/07/2016 22:02, Tamas K Lengyel wrote:
>>
>> On Fri, Jul 29, 2016 at 11:38 AM, Stefano Stabellini
>>  wrote:
>>>
>>> On Fri, 29 Jul 2016, Tamas K Lengyel wrote:

 On Jul 29, 2016 02:50, "Julien Grall"  wrote:
>
>
>
>
> On 28/07/16 23:54, Tamas K Lengyel wrote:
>>
>>
>> On Thu, Jul 28, 2016 at 2:38 PM, Julien Grall 
>> wrote:
>>>
>>>
>>> On 28/07/2016 20:35, Tamas K Lengyel wrote:
>>> This patch is doing more than it is claimed in the commit message.
>>>
>>> In general, moving the code and introducing changes within the same
>>> patch
>>> should really be avoided. So please split it in 2 patches.
>>
>>
>>
>> Well, the changes are largely cosmetic so doing a whole separate patch
>> IMHO is an overkill. How about adjusting the commit message to
>> something like "sanitize code surrounding sending mem_access
>> vm_events" to better describe the adjustments made in this patch?
>
>
>
> I think the wiki page "Submitting Xen Project patches" [1] should
> answer to your question.
>
> If not, trivial patches are easy to review, merging multiple trivial
> patches in a single patch is not. Moving

 code and at the same time as changing the behavior is fairly difficult
 to review because it hides the real
 modifications.
>
>
> Regards,
>
> [1]
> http://wiki.xenproject.org/wiki/Submitting_Xen_Project_Patches#Break_down_your_patches
>

 The behavior didn't change at all, this whole patch is code
 sanitization. It's not worth doing a separate patch
 for each minor change. The few change on the arm side is the vm_event
 request allocation going from xzalloc to
 stack based and using monitor_traps now in a split-out function. It
 really should be no problem reviewing it. Even
 Andrew requested minor adjustments to be included in this patch. Anyway,
 I'm not looking to change this into a
 series. If it's a no go from your side I'm just going to cut down the
 ARM side sanitization to the bare minimum of
 using monitor_traps as the rest just does not worth the effort.
>>>
>>>
>>> Hi Tamas,
>>> let me premise that, like you wrote, the patch is just code
>>> sanitization, certainly not worth turning it into an argument.
>>>
>>> I think different maintainers have different styles. I for one always
>>> dislike to have code movement, renaming or code style fixes together
>>> with any other more meaningful changes. In fact when people suggest
>>> "could you please also rename this variable" or "could you please also
>>> move the function to common code", I usually reply: "I can, but it will
>>> be in another patch".
>>>
>>> So I agree with Julien that I would prefer to see two patches. In fact
>>> if I were you, I would prefer to write two separate patches because it
>>> would be less troubles for me as a developer. But clearly not everybody
>>> agrees with this style as I get requests for cosmetic changes together
>>> with other changes by many other seasoned maintainers. And that's OK, it
>>> just means that different people think differently, which is a good
>>> thing for the project as a whole.
>>>
>>> You are a valued contributor and maintainer of this project -- if you
>>> strongly feel this way, I am sure we can find a way to make this work
>>> anyway.
>>
>>
>> I would highly appreciate that as I said it's just not worth the time
>> it takes to break this down into smaller patches. It really doubles
>> the effort it takes to test it.
>
>
> I find this paragraph really offensive for reviewers. This requires more
> efforts for reviewers to review a such patch. My time is as valuable as
> yours. I hope you will reconsider what you've written.
>

I don't know which part you find offensive and I certainly didn't mean
to offend anyone. I do value your time. I understand this may require
a bit more to review but honestly, I don't think the difference from
your side should be significant. From my side it is and it's not worth
the extra effort to go rounds about this and do a whole series so I'll
just drop the portions you are not happy about. It is your right as
maintainer to reject it as this code right now lives in a generic part
of the ARM code-base. However, as this patch really only touches
things that belong to mem_access/monitor components maybe we should
split these from the generic ARM code altogether to avoid such
conflicts in the future.

Tamas

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] mem_access: Use monitor_traps instead of mem_access_send_req

[Julien Grall]

On 29/07/2016 22:02, Tamas K Lengyel wrote:

On Fri, Jul 29, 2016 at 11:38 AM, Stefano Stabellini
 wrote:

On Fri, 29 Jul 2016, Tamas K Lengyel wrote:

On Jul 29, 2016 02:50, "Julien Grall"  wrote:




On 28/07/16 23:54, Tamas K Lengyel wrote:


On Thu, Jul 28, 2016 at 2:38 PM, Julien Grall  wrote:


On 28/07/2016 20:35, Tamas K Lengyel wrote:
This patch is doing more than it is claimed in the commit message.

In general, moving the code and introducing changes within the same patch
should really be avoided. So please split it in 2 patches.



Well, the changes are largely cosmetic so doing a whole separate patch
IMHO is an overkill. How about adjusting the commit message to
something like "sanitize code surrounding sending mem_access
vm_events" to better describe the adjustments made in this patch?



I think the wiki page "Submitting Xen Project patches" [1] should answer to 
your question.

If not, trivial patches are easy to review, merging multiple trivial patches in 
a single patch is not. Moving

code and at the same time as changing the behavior is fairly difficult to 
review because it hides the real
modifications.


Regards,

[1] 
http://wiki.xenproject.org/wiki/Submitting_Xen_Project_Patches#Break_down_your_patches



The behavior didn't change at all, this whole patch is code sanitization. It's 
not worth doing a separate patch
for each minor change. The few change on the arm side is the vm_event request 
allocation going from xzalloc to
stack based and using monitor_traps now in a split-out function. It really 
should be no problem reviewing it. Even
Andrew requested minor adjustments to be included in this patch. Anyway, I'm 
not looking to change this into a
series. If it's a no go from your side I'm just going to cut down the ARM side 
sanitization to the bare minimum of
using monitor_traps as the rest just does not worth the effort.


Hi Tamas,
let me premise that, like you wrote, the patch is just code
sanitization, certainly not worth turning it into an argument.

I think different maintainers have different styles. I for one always
dislike to have code movement, renaming or code style fixes together
with any other more meaningful changes. In fact when people suggest
"could you please also rename this variable" or "could you please also
move the function to common code", I usually reply: "I can, but it will
be in another patch".

So I agree with Julien that I would prefer to see two patches. In fact
if I were you, I would prefer to write two separate patches because it
would be less troubles for me as a developer. But clearly not everybody
agrees with this style as I get requests for cosmetic changes together
with other changes by many other seasoned maintainers. And that's OK, it
just means that different people think differently, which is a good
thing for the project as a whole.

You are a valued contributor and maintainer of this project -- if you
strongly feel this way, I am sure we can find a way to make this work
anyway.


I would highly appreciate that as I said it's just not worth the time
it takes to break this down into smaller patches. It really doubles
the effort it takes to test it.


I find this paragraph really offensive for reviewers. This requires more 
efforts for reviewers to review a such patch. My time is as valuable as 
yours. I hope you will reconsider what you've written.


Regards,



Thanks,
Tamas



--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] mem_access: Use monitor_traps instead of mem_access_send_req

[Tamas K Lengyel]

On Fri, Jul 29, 2016 at 11:38 AM, Stefano Stabellini
 wrote:
> On Fri, 29 Jul 2016, Tamas K Lengyel wrote:
>> On Jul 29, 2016 02:50, "Julien Grall"  wrote:
>> >
>> >
>> >
>> > On 28/07/16 23:54, Tamas K Lengyel wrote:
>> >>
>> >> On Thu, Jul 28, 2016 at 2:38 PM, Julien Grall  
>> >> wrote:
>> >>>
>> >>> On 28/07/2016 20:35, Tamas K Lengyel wrote:
>> >>> This patch is doing more than it is claimed in the commit message.
>> >>>
>> >>> In general, moving the code and introducing changes within the same patch
>> >>> should really be avoided. So please split it in 2 patches.
>> >>
>> >>
>> >> Well, the changes are largely cosmetic so doing a whole separate patch
>> >> IMHO is an overkill. How about adjusting the commit message to
>> >> something like "sanitize code surrounding sending mem_access
>> >> vm_events" to better describe the adjustments made in this patch?
>> >
>> >
>> > I think the wiki page "Submitting Xen Project patches" [1] should answer 
>> > to your question.
>> >
>> > If not, trivial patches are easy to review, merging multiple trivial 
>> > patches in a single patch is not. Moving
>> code and at the same time as changing the behavior is fairly difficult to 
>> review because it hides the real
>> modifications.
>> >
>> > Regards,
>> >
>> > [1] 
>> > http://wiki.xenproject.org/wiki/Submitting_Xen_Project_Patches#Break_down_your_patches
>> >
>>
>> The behavior didn't change at all, this whole patch is code sanitization. 
>> It's not worth doing a separate patch
>> for each minor change. The few change on the arm side is the vm_event 
>> request allocation going from xzalloc to
>> stack based and using monitor_traps now in a split-out function. It really 
>> should be no problem reviewing it. Even
>> Andrew requested minor adjustments to be included in this patch. Anyway, I'm 
>> not looking to change this into a
>> series. If it's a no go from your side I'm just going to cut down the ARM 
>> side sanitization to the bare minimum of
>> using monitor_traps as the rest just does not worth the effort.
>
> Hi Tamas,
> let me premise that, like you wrote, the patch is just code
> sanitization, certainly not worth turning it into an argument.
>
> I think different maintainers have different styles. I for one always
> dislike to have code movement, renaming or code style fixes together
> with any other more meaningful changes. In fact when people suggest
> "could you please also rename this variable" or "could you please also
> move the function to common code", I usually reply: "I can, but it will
> be in another patch".
>
> So I agree with Julien that I would prefer to see two patches. In fact
> if I were you, I would prefer to write two separate patches because it
> would be less troubles for me as a developer. But clearly not everybody
> agrees with this style as I get requests for cosmetic changes together
> with other changes by many other seasoned maintainers. And that's OK, it
> just means that different people think differently, which is a good
> thing for the project as a whole.
>
> You are a valued contributor and maintainer of this project -- if you
> strongly feel this way, I am sure we can find a way to make this work
> anyway.

I would highly appreciate that as I said it's just not worth the time
it takes to break this down into smaller patches. It really doubles
the effort it takes to test it. It's within your and Julien's right to
not accept such patches touching your code-base as Maintainer and
that's fine. If that's the case though we might want to look into
adjusting the layout of the code so that mem_access/monitor/vm_event
components are more isolated into separate files so that we can move
at a different phase and different style then the rest of the code
here without getting into arguments about that stuff.

Thanks,
Tamas

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] mem_access: Use monitor_traps instead of mem_access_send_req

[Tamas K Lengyel]

On Fri, Jul 29, 2016 at 10:27 AM, Andrew Cooper
 wrote:
> On 29/07/16 15:21, Tamas K Lengyel wrote:
>
> On Jul 29, 2016 02:50, "Julien Grall"  wrote:
>>
>>
>>
>> On 28/07/16 23:54, Tamas K Lengyel wrote:
>>>
>>> On Thu, Jul 28, 2016 at 2:38 PM, Julien Grall 
>>> wrote:

 On 28/07/2016 20:35, Tamas K Lengyel wrote:
 This patch is doing more than it is claimed in the commit message.

 In general, moving the code and introducing changes within the same
 patch
 should really be avoided. So please split it in 2 patches.
>>>
>>>
>>> Well, the changes are largely cosmetic so doing a whole separate patch
>>> IMHO is an overkill. How about adjusting the commit message to
>>> something like "sanitize code surrounding sending mem_access
>>> vm_events" to better describe the adjustments made in this patch?
>>
>>
>> I think the wiki page "Submitting Xen Project patches" [1] should answer
>> to your question.
>>
>> If not, trivial patches are easy to review, merging multiple trivial
>> patches in a single patch is not. Moving code and at the same time as
>> changing the behavior is fairly difficult to review because it hides the
>> real modifications.
>>
>> Regards,
>>
>> [1]
>> http://wiki.xenproject.org/wiki/Submitting_Xen_Project_Patches#Break_down_your_patches
>>
>
> The behavior didn't change at all, this whole patch is code sanitization.
> It's not worth doing a separate patch for each minor change. The few change
> on the arm side is the vm_event request allocation going from xzalloc to
> stack based and using monitor_traps now in a split-out function. It really
> should be no problem reviewing it. Even Andrew requested minor adjustments
> to be included in this patch. Anyway, I'm not looking to change this into a
> series. If it's a no go from your side I'm just going to cut down the ARM
> side sanitization to the bare minimum of using monitor_traps as the rest
> just does not worth the effort.
>
>
> To offer an alternative opinion.
>
> Looking at this patch, I personally would have a hard time justifying
> breaking it down any further.  Each of the changes are closely related.
>
> However, the commit message could be a lot clearer about which steps are
> taken.  If I were writing the commit message, I would go with something a
> bit more like this:
>
> 8<
> The two functions monitor_traps and mem_access_send_req duplicate some of
> the same functionality. The mem_access_send_req however leaves a lot of the
> standard vm_event fields to be filled by other functions.
>
> Remove mem_access_send_req() completely, making use of monitor_traps() to
> put requests into the monitor ring.  This in turn causes some cleanup around
> the old callsites of mem_access_send_req(), and on ARM, the introduction of
> the __p2m_mem_access_send_req() helper to fill in common mem_access
> information.
>
> Finally, this change identifies that errors from mem_access_send_req() were
> never checked.  As errors constitute a problem with the monitor ring,
> crashing the domain is the most appropriate action to take.
> 8<
>
> If in doubt, always spell out each of the changes, and their logical
> relationships.  If nothing else, it helps people trying to review the patch.
>

Thanks and I agree, adjusting the commit message is what I would think
would make sense as well here (and what I offered as an alternative to
breaking down the series into tiny patches).

Tamas

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [linux-3.14 test] 99747: tolerable FAIL - PUSHED

[osstest service owner]

flight 99747 linux-3.14 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/99747/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 build-amd64-rumpuserxen   6 xen-buildfail   like 96226
 build-i386-rumpuserxen6 xen-buildfail   like 96226
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop  fail like 96226
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stop fail like 96226
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail like 96226
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop  fail like 96226

Tests which did not succeed, but are not blocking:
 test-amd64-i386-rumpuserxen-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-rumpuserxen-amd64  1 build-check(1)   blocked n/a
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass

version targeted for testing:
 linuxda99423b3cd3e48c42c0d64b79aba58d828f9648
baseline version:
 linux44dd5e6b1cf505485d839bd62d47e29a36232d67

Last test of basis96226  2016-06-24 17:53:30 Z   35 days
Testing same since99716  2016-07-27 17:57:45 Z2 days2 attempts


People who touched revisions under test:
  Al Viro 
  Alan Stern 
  Alex Deucher 
  Andreas Gruenbacher 
  Andrew Goodbody 
  Andrew Morton 
  Andrey Ryabinin 
  Anna Schumaker 
  Anthony Romano 
  Ben Hutchings 
  Bin Liu 
  Bjørn Mork 
  Bob Copeland 
  Borislav Petkov 
  Catalin Marinas 
  Christoph Hellwig 
  Crestez Dan Leonard 
  Cyril Bur 
  Dan Carpenter 
  Daniel Vetter 
  David Howells 
  David S. Miller 
  David Vrabel 
  Dmitry Torokhov 
  Dmitry Vyukov 
  Doug Ledford 
  Feng Tang 
  Fred Veldini 
  Gavin Shan 
  Greg Kroah-Hartman 
  Guilherme G. Piccoli 
  H. Peter Anvin 
  Hans de Goede 
  Herbert Xu 
  Hugh Dickins 
  Imre Palik 
  Ingo Molnar 
  J. Bruce Fields 
  James Hogan 
  Jan Beulich 
  Jan Willeke 
  Jason Gunthorpe 
  Jiri Kosina 
  Jiri Slaby 
  Johannes Berg 
  Jonathan Cameron 
  Kirill A. Shutemov 
  Linus Torvalds 
  Linus Walleij 
  Luis de Bethencourt 
  Lyude 
  Mark Brown 
  Martin K. Petersen 
  Martin Schwidefsky 
  Martin Willi 
  Masami Hiramatsu 
  Michael Ellerman 
  Namhyung Kim 
  Ole Lukoie 
  Oleg Drokin 
  Oleg Nesterov 
  Oliver Neukum 
  Palik, Imre 
  Paolo Bonzini 
  Pavel Shilovsky 
  Peter Zijlstra (Intel) 
  Richard Weinberger 
  Russell King 
  Scott Bauer 
  Simon Horman 
  Steve French 
  Steve French 
  Steven Rostedt (Red Hat) 
  Steven Rostedt 
  Takashi Iwai 
  Tom Goff 
  Trond Myklebust 
  Vladimir Davydov 
  Wei Fang 
  Wei Tang 
  Wilfried Klaebe 
  Will Deacon 
  Xiubo Li 
  YOSHIFUJI Hideaki 
  Zhang Zhuoyu 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 build-amd64-rumpuserxen  fail
 build-i386-rumpuserxen   fail
 test-amd64-amd64-xl  pass
 test-amd64-i386-xl   pass
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsmpass
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmpass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsmpass
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm pass
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsmpass
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm pa

Re: [Xen-devel] [RFC 00/22] xen/arm: Rework the P2M code to follow break-before-make sequence

[Julien Grall]

On 29/07/16 17:23, Julien Grall wrote:



On 28/07/16 18:46, Tamas K Lengyel wrote:

Hi Julien,


Hello,


I sent this patch series as an RFC because there are still some TODOs
in the code (mostly sanity check and possible optimization) and I have
done limited testing. However, I think it is a good shape to start reviewing,
get more feedback and have wider testing on different board.


I've tested this series on my Cubietruck but when I try to enable
xen-access on a domain I get the following errors:

~/xen/tools/tests/xen-access# ./xen-access 1 write
xenaccess init
max_gpfn = 48000
starting write 1
(XEN) traps.c:2569:d1v0 HSR=0x904f pc=0xc029eb10 gva=0xc0e013c0
gpa=0x0040e013c0
Error -1 setting all memory to access type 5

The same thing works fine on the latest staging build, so this series
introduces some regression along the way.


Well, memaccess is not working on staging. As soon as it is enabled, the console
is flooded with:

traps.c:2503:d1v0 HSR=0x924f pc=0xff8008579b90 gva=0xffc0011bf000 
gpa=0x00411bf000

And the guest is crashing soon after because it received a data abort.


I found the problem and sent a patch for unstable (see [1]). I can hit 
easily on all the platform I tested. So I am not sure why you don't see 
the issue on the cubietruck.


I will update patch #18 to apply a similar fix in p2m_split_superpage.

Regards,

[1] 
https://lists.xenproject.org/archives/html/xen-devel/2016-07/msg03133.html




Regards,



--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH RFC 06/12] xen/x86: populate PVHv2 Dom0 physical memory map

[Andrew Cooper]

On 29/07/16 17:29, Roger Pau Monne wrote:
> Craft the Dom0 e820 memory map and populate it.
>
> Signed-off-by: Roger Pau Monné 
> ---
> Cc: Jan Beulich 
> Cc: Andrew Cooper 
> ---
>  xen/arch/x86/domain_build.c | 199 
> ++--
>  1 file changed, 193 insertions(+), 6 deletions(-)
>
> diff --git a/xen/arch/x86/domain_build.c b/xen/arch/x86/domain_build.c
> index c0ef40f..cb8ecbd 100644
> --- a/xen/arch/x86/domain_build.c
> +++ b/xen/arch/x86/domain_build.c
> @@ -43,6 +43,11 @@ static long __initdata dom0_nrpages;
>  static long __initdata dom0_min_nrpages;
>  static long __initdata dom0_max_nrpages = LONG_MAX;
>  
> +/* GFN of the identity map for EPT. */
> +#define HVM_IDENT_PT_GFN  0xfeffeu

The IDENT_PT is only needed on Gen1 VT-x hardware which doesn't support
unrestricted-guest mode.  OTOH, it needs to be matched with VM86_TSS,
which is also needed if hardware doesn't support unrestricted-guest. 
Also, the IDENT_PT can live anywhere in GFN space.  0xfeffe is just
convention for HVM guests as nothing else interesting lives there, but a
PVH dom0 will want to ensure it doesn't alias anything interesting it
might wish to map.

Now I look at it, there is substantial WTF.  The domain builder makes
IDENT_PT, but HVMLoader makes VM86_TSS.  I presume this is a historical
side effect of IDENT_PT being a prerequisite to even executing
hvmloader, while VM86_TSS was only a prerequisite to executing the
rombios payload.  Either way, eww :(

I think the VM86_TSS setup needs to move to unconditionally being beside
IDENT_PT setup in the domain builder, and mirrored here in dom0_hvm()
creation.  I don't think it is reasonable to expect an HVMLite payload
to set up its own VM86_TSS if it didn't set up IDENT_PT.  (OTOH, the
chances of an HVMLite guest ever needing to return to real mode is slim,
but in the name of flexibility, it would be nice not to rule it out).

Longterm, it would be nice to disentangle the unrestricted-guest support
from the main vmx code, and make it able to be compiled out.  There is
lots of it, and it all-but-dead on modern hardware.
> @@ -591,6 +610,8 @@ static __init void pvh_setup_e820(struct domain *d, 
> unsigned long nr_pages)
>  {
>  cur_pages += pages;
>  }
> +ASSERT((entry_guest->addr & ~PAGE_MASK) == 0 &&
> +   (entry_guest->size & ~PAGE_MASK) == 0);

This would be clearer as ASSERT(IS_ALIGNED(entry_guest->addr, PAGE_SIZE));

(although I suspect the IS_ALIGNED() predicate is newer than your first
iteration of this code.)

>   next:
>  d->arch.nr_e820++;
>  entry_guest++;
> @@ -1631,7 +1652,7 @@ static int __init construct_dom0_pv(
>  dom0_update_physmap(d, pfn, mfn, 0);
>  
>  pvh_map_all_iomem(d, nr_pages);
> -pvh_setup_e820(d, nr_pages);
> +hvm_setup_e820(d, nr_pages);
>  }
>  
>  if ( d->domain_id == hardware_domid )
> @@ -1647,15 +1668,181 @@ out:
>  return rc;
>  }
>  
> +/* Helper to convert from bytes into human-readable form. */
> +static void __init pretty_print_bytes(uint64_t size)
> +{
> +const char* units[] = {"B", "KB", "MB", "GB", "TB"};

static const char *units[] __initconst

However, this particular array would be far smaller as

static const char units[][3] __initconst = { "B", ... };

as it avoids embedding 5x8byte pointers to get at 14 useful bytes of
information.

> +int i = 0;
> +
> +while ( ++i < sizeof(units) && size >= 1024 )
> +size >>= 10; /* size /= 1024 */
> +
> +printk("%4" PRIu64 "%2s", size, units[i-1]);

Wouldn't it be better to introduce a new %p format identifier to do
this?  (Linux doesn't appear to have an existing format identifier which
we can 'borrow')

It looks potentially useful elsewhere, and avoids the awkward

printk("Order %2u: ", MAX_ORDER - i);
pretty_print_bytes(((uint64_t)1 << (MAX_ORDER - i + PAGE_SHIFT)) *
   hvm_mem_stats[MAX_ORDER - i]);
printk("\n");

constructs you have below.


> +}
> +
> +/* Calculate the biggest usable order given a size in bytes. */
> +static inline unsigned int get_order(uint64_t size)
> +{
> +unsigned int order;
> +uint64_t pg;
> +
> +ASSERT((size & ~PAGE_MASK) == 0);
> +pg = PFN_DOWN(size);
> +for ( order = 0; pg >= (1 << (order + 1)); order++ );
> +
> +return order;
> +}

We already have get_order_from_bytes() and get_order_from_pages(), the
latter of which looks like it will suit your usecase.

As a TODO item, I really would like to borrow some of the Linux
infrastructure to calculate orders of constants at compile time, because
a large number of callers of the existing get_order_*() functions are
performing unnecessary runtime calculation.  For those which need
runtime calculation, some careful use of ffs() would be preferable to a
loop.

> +
> +/* Populate an HVM memory range using the biggest possible order. */
> +static void __init hvm_populate_memory_range(struct domain *d, uint64_t 
> start,
> +  

[Xen-devel] [linux-4.1 test] 99741: regressions - FAIL

[osstest service owner]

flight 99741 linux-4.1 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/99741/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemut-debianhvm-amd64 9 debian-hvm-install fail REGR. vs. 
96211
 test-amd64-i386-xl-raw9 debian-di-install fail REGR. vs. 96211
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm 9 debian-hvm-install fail REGR. 
vs. 96211
 test-amd64-i386-libvirt   9 debian-installfail REGR. vs. 96211
 test-amd64-amd64-xl-qcow2 6 xen-boot  fail REGR. vs. 96211
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 9 debian-hvm-install fail 
REGR. vs. 96211
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 9 windows-install fail REGR. vs. 96211
 test-amd64-i386-qemut-rhel6hvm-amd  9 redhat-install  fail REGR. vs. 96211
 test-armhf-armhf-xl-multivcpu  9 debian-install   fail REGR. vs. 96211
 test-amd64-amd64-qemuu-nested-intel  6 xen-boot   fail REGR. vs. 96211
 test-amd64-i386-qemuu-rhel6hvm-amd  9 redhat-install  fail REGR. vs. 96211
 test-armhf-armhf-libvirt-xsm  9 debian-installfail REGR. vs. 96211
 test-amd64-amd64-xl-qemut-debianhvm-amd64  6 xen-boot fail REGR. vs. 96211
 test-amd64-amd64-i386-pvgrub  6 xen-boot  fail REGR. vs. 96211
 test-armhf-armhf-xl-cubietruck  9 debian-install  fail REGR. vs. 96211
 test-amd64-amd64-xl-qemut-winxpsp3  6 xen-bootfail REGR. vs. 96211
 test-armhf-armhf-xl-xsm   9 debian-installfail REGR. vs. 96211
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  6 xen-boot fail REGR. vs. 96211
 test-amd64-amd64-xl-credit2   6 xen-boot  fail REGR. vs. 96211
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1 9 windows-install fail REGR. vs. 96211
 test-amd64-i386-freebsd10-amd64  9 freebsd-installfail REGR. vs. 96211
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm 9 debian-hvm-install fail REGR. 
vs. 96211
 test-armhf-armhf-xl   9 debian-installfail REGR. vs. 96211
 test-amd64-i386-xl-qemut-winxpsp3  9 windows-install  fail REGR. vs. 96211
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 6 xen-boot fail REGR. 
vs. 96211
 test-amd64-i386-xl-qemuu-debianhvm-amd64 9 debian-hvm-install fail REGR. vs. 
96211
 test-amd64-amd64-xl-xsm   6 xen-boot  fail REGR. vs. 96211
 test-amd64-amd64-xl-qemuu-ovmf-amd64  6 xen-boot  fail REGR. vs. 96211
 test-amd64-amd64-xl-qemuu-winxpsp3  6 xen-bootfail REGR. vs. 96211
 test-armhf-armhf-xl-credit2   9 debian-installfail REGR. vs. 96211
 test-amd64-amd64-xl-multivcpu  6 xen-boot fail REGR. vs. 96211
 test-amd64-i386-xl-xsm9 debian-installfail REGR. vs. 96211
 test-amd64-amd64-xl-qemuu-win7-amd64  6 xen-boot  fail REGR. vs. 96211
 test-amd64-amd64-libvirt-xsm  6 xen-boot  fail REGR. vs. 96211
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 6 xen-boot fail REGR. vs. 
96211
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm  6 xen-boot fail REGR. vs. 96211
 test-amd64-amd64-xl-pvh-amd   6 xen-boot  fail REGR. vs. 96211
 test-amd64-amd64-libvirt  6 xen-boot  fail REGR. vs. 96211
 test-amd64-amd64-amd64-pvgrub  6 xen-boot fail REGR. vs. 96211
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm  6 xen-boot fail REGR. vs. 96211
 test-amd64-amd64-libvirt-vhd  6 xen-boot  fail REGR. vs. 96211
 test-amd64-amd64-qemuu-nested-amd  6 xen-boot fail REGR. vs. 96211
 test-amd64-i386-qemuu-rhel6hvm-intel  9 redhat-installfail REGR. vs. 96211
 test-amd64-amd64-xl   6 xen-boot  fail REGR. vs. 96211
 test-amd64-i386-xl-qemuu-winxpsp3  9 windows-install  fail REGR. vs. 96211
 test-armhf-armhf-xl-arndale   9 debian-installfail REGR. vs. 96211
 test-amd64-amd64-xl-qemut-win7-amd64  6 xen-boot  fail REGR. vs. 96211
 test-amd64-amd64-xl-pvh-intel  6 xen-boot fail REGR. vs. 96211
 test-amd64-i386-qemut-rhel6hvm-intel  9 redhat-installfail REGR. vs. 96211
 test-amd64-amd64-pygrub   6 xen-boot  fail REGR. vs. 96211
 test-amd64-i386-xl-qemuu-ovmf-amd64  9 debian-hvm-install fail REGR. vs. 96211
 test-amd64-i386-freebsd10-i386  9 freebsd-install fail REGR. vs. 96211
 test-armhf-armhf-libvirt-qcow2  9 debian-di-install   fail REGR. vs. 96211
 test-armhf-armhf-libvirt  9 debian-installfail REGR. vs. 96211
 test-amd64-i386-libvirt-pair 15 debian-install/dst_host   fail REGR. vs. 96211
 test-amd64-i386-libvirt-xsm   9 debian-installfail REGR. vs. 96211
 test-amd64-i386-xl9 debian-installfail REGR. vs. 96211
 test-amd64-i386-pair 15 debian-install/dst_host   fail REGR. vs. 96211
 test-amd64-i386-xl-qemut-win7-amd64  9 windows-installfail REGR. vs. 96211
 test-amd64-amd64-pair  

[Xen-devel] [PATCH] xen/arm: p2m: Don't use default access permission when shattering a superpage

[Julien Grall]

The following message flood the console when memaccess is enabled on
various platforms:

traps.c:2510:d1v0 HSR=0x9383004f pc=0x08b7d4c4 gva=0x08eeb8e0 
gpa=0x004903f8e0

This is because a data abort from a guest was received due to a
permission fault but memaccess thought there are no permission fault.

On ARM, memaccess permissions are stored in a radix tree because there
are not enough available bits in the p2m entry to store the access
restriction. When memaccess is restricting the access (i.e any other
access than p2m_access_rwx), the access will be added in the radix tree
using the GFN as a key. This will be done for all 4KB pages.

This means that memaccess has to shatter all the superpages in a given
region to set the permission on a 4KB granularity. Currently, when a
superpage is shattered, the new entries are using the value
p2m->default_access which will restrict permission (because memaccess
has been enabled). However the radix tree does not yet contain
an entry for this GFN.

If a guest VCPU is running at the same time and trying to access the
modified region, it will result to a stage-2 permission fault. As
the radix tree does not yet contain an entry for the GFN, memaccess will
deduce that the fault was not valid and a data abort will be injecting
to the guest (and crash it).

Furthermore, the permission may be restricted outside of the requested
region if it is only a subset of a 1GB/2MB superpage.

The two issues can be fixed by re-using the permission of the superpage
entry and override the necessary fields. This is not a problem because
memaccess cannot work on superpage.

Lastly, document the code which call mfn_to_p2m_entry when creating a
the p2m entry for a table to explain that create the p2m entry to page table
to explain that permission are ignored by the hardware (See D4.3.1 in ARM DDI
0487A.j). so the value of the parameter 'access' of mfn_to_p2m_entry does
not matter.

Signed-off-by: Julien Grall 

---
This patch needs to be backported up to Xen 4.6 (first release
which introduced memaccess on ARM). This may require few adjustement
because the code has changed a bit.

Without it, the guest will randomly crash because the permission has
been overriden whilst shattering a superpage and before adding the access
permission in the radix tree.

Also I am wondering if it would be better to pass p2m_access_rwx
to mfn_to_p2m_entry when creating a table entry. This would save
a couple of instructions.
---
 xen/arch/arm/p2m.c | 18 +-
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index 40a0b80..d60fbbf 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -434,7 +434,6 @@ static int p2m_create_table(struct p2m_domain *p2m, lpae_t 
*entry,
 p = __map_domain_page(page);
 if ( splitting )
 {
-p2m_type_t t = entry->p2m.type;
 mfn_t mfn = _mfn(entry->p2m.base);
 int i;
 
@@ -444,15 +443,20 @@ static int p2m_create_table(struct p2m_domain *p2m, 
lpae_t *entry,
  */
  for ( i=0 ; i < LPAE_ENTRIES; i++ )
  {
- pte = mfn_to_p2m_entry(mfn_add(mfn, i << (level_shift - 
LPAE_SHIFT)),
-t, p2m->default_access);
+ /*
+  * Use the content of the superpage entry and override
+  * the necessary fields. So the correct permissions are
+  * kept.
+  */
+ pte = *entry;
+ pte.p2m.base = mfn_x(mfn_add(mfn,
+  i << (level_shift - LPAE_SHIFT)));
 
  /*
   * First and second level super pages set p2m.table = 0, but
   * third level entries set table = 1.
   */
- if ( level_shift - LPAE_SHIFT )
- pte.p2m.table = 0;
+ pte.p2m.table = !(level_shift - LPAE_SHIFT);
 
  write_pte(&p[i], pte);
  }
@@ -467,6 +471,10 @@ static int p2m_create_table(struct p2m_domain *p2m, lpae_t 
*entry,
 
 unmap_domain_page(p);
 
+/*
+ * The access value does not matter because the hardware will ignore
+ * the permission fields for table entry.
+ */
 pte = mfn_to_p2m_entry(_mfn(page_to_mfn(page)), p2m_invalid,
p2m->default_access);
 
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH RFC 05/12] xen/x86: make print_e820_memory_map global

[Andrew Cooper]

On 29/07/16 17:29, Roger Pau Monne wrote:
> So that it can be called from the Dom0 builder.
>
> Signed-off-by: Roger Pau Monné 

Reviewed-by: Andrew Cooper 
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH RFC 04/12] xen/x86: split Dom0 build into PV and PVHv2

[Andrew Cooper]

On 29/07/16 17:28, Roger Pau Monne wrote:
> Split the Dom0 builder into two different functions, one for PV (and classic
> PVH), and another one for PVHv2. Introduce a new command line parameter,
> dom0hvm in order to request the creation of a PVHv2 Dom0.
>
> Signed-off-by: Roger Pau Monné 
> ---
> Cc: Jan Beulich 
> Cc: Andrew Cooper 
> ---
>  xen/arch/x86/domain_build.c | 27 ++-
>  xen/arch/x86/setup.c|  9 +

A patch to docs/misc/xen-command-line.markdown please.

>  2 files changed, 35 insertions(+), 1 deletion(-)
>
> diff --git a/xen/arch/x86/domain_build.c b/xen/arch/x86/domain_build.c
> index 09d79be..c0ef40f 100644
> --- a/xen/arch/x86/domain_build.c
> +++ b/xen/arch/x86/domain_build.c
> @@ -952,7 +952,7 @@ static int __init setup_permissions(struct domain *d)
>  return rc;
>  }
>  
> -int __init construct_dom0(
> +static int __init construct_dom0_pv(
>  struct domain *d,
>  const module_t *image, unsigned long image_headroom,
>  module_t *initrd,
> @@ -1647,6 +1647,31 @@ out:
>  return rc;
>  }
>  
> +static int __init construct_dom0_hvm(struct domain *d, const module_t *image,
> + unsigned long image_headroom,
> + module_t *initrd,
> + void *(*bootstrap_map)(const module_t 
> *),
> + char *cmdline)
> +{
> +
> +printk("** Building a PVH Dom0 **\n");

Some naming curiosities here, especially given the parameter name.

> +
> +return 0;
> +}
> +
> +int __init construct_dom0(struct domain *d, const module_t *image,
> +  unsigned long image_headroom, module_t *initrd,
> +  void *(*bootstrap_map)(const module_t *),
> +  char *cmdline)
> +{
> +
> +return is_hvm_domain(d) ?
> +construct_dom0_hvm(d, image, image_headroom, initrd,
> +   bootstrap_map, cmdline) :
> +construct_dom0_pv(d, image, image_headroom, initrd, 
> bootstrap_map,
> +  cmdline);

This could be slightly less awkwardly split as:

(is_hvm_domain(d) ? construct_dom0_hvm : construct_dom0_pv)
(d, image, image_headroom, initrd, bootstrap_map, cmdline);

with some appropriate indentation/alignment.

~Andrew


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH RFC 03/12] xen/x86: allow the emulated APICs to be enbled for the hardware domain

[Andrew Cooper]

On 29/07/16 17:28, Roger Pau Monne wrote:
>  if ( is_hardware_domain(d) )
> -config->emulation_flags |= XEN_X86_EMU_PIT;
> -if ( config->emulation_flags != 0 &&
> - (config->emulation_flags !=
> -  (is_hvm_domain(d) ? XEN_X86_EMU_ALL : XEN_X86_EMU_PIT)) )
> +emflags |= XEN_X86_EMU_PIT;
> +
> +if ( (is_hardware_domain(d) ?
> +  (is_hvm_domain(d) && emflags !=
> +  (XEN_X86_EMU_PIT|XEN_X86_EMU_LAPIC|XEN_X86_EMU_IOAPIC)) :
> +  (emflags && (is_hvm_domain(d) ? (emflags != XEN_X86_EMU_ALL) :
> +  (emflags != 
> XEN_X86_EMU_PIT )

This is getting very complicated.

It is rather important (security wise) and not a hotpath.  Could I
please request that you move this logic to a helper such as:

static bool emulation_flags_ok(const struct domain *d, uint32_t emflags)

and make this more readable.  Any decent compiler will inline and
simplify it appropriately.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] mem_access: Use monitor_traps instead of mem_access_send_req

[Stefano Stabellini]

On Fri, 29 Jul 2016, Tamas K Lengyel wrote:
> On Jul 29, 2016 02:50, "Julien Grall"  wrote:
> >
> >
> >
> > On 28/07/16 23:54, Tamas K Lengyel wrote:
> >>
> >> On Thu, Jul 28, 2016 at 2:38 PM, Julien Grall  wrote:
> >>>
> >>> On 28/07/2016 20:35, Tamas K Lengyel wrote:
> >>> This patch is doing more than it is claimed in the commit message.
> >>>
> >>> In general, moving the code and introducing changes within the same patch
> >>> should really be avoided. So please split it in 2 patches.
> >>
> >>
> >> Well, the changes are largely cosmetic so doing a whole separate patch
> >> IMHO is an overkill. How about adjusting the commit message to
> >> something like "sanitize code surrounding sending mem_access
> >> vm_events" to better describe the adjustments made in this patch?
> >
> >
> > I think the wiki page "Submitting Xen Project patches" [1] should answer to 
> > your question.
> >
> > If not, trivial patches are easy to review, merging multiple trivial 
> > patches in a single patch is not. Moving
> code and at the same time as changing the behavior is fairly difficult to 
> review because it hides the real
> modifications.
> >
> > Regards,
> >
> > [1] 
> > http://wiki.xenproject.org/wiki/Submitting_Xen_Project_Patches#Break_down_your_patches
> >
> 
> The behavior didn't change at all, this whole patch is code sanitization. 
> It's not worth doing a separate patch
> for each minor change. The few change on the arm side is the vm_event request 
> allocation going from xzalloc to
> stack based and using monitor_traps now in a split-out function. It really 
> should be no problem reviewing it. Even
> Andrew requested minor adjustments to be included in this patch. Anyway, I'm 
> not looking to change this into a
> series. If it's a no go from your side I'm just going to cut down the ARM 
> side sanitization to the bare minimum of
> using monitor_traps as the rest just does not worth the effort.

Hi Tamas,
let me premise that, like you wrote, the patch is just code
sanitization, certainly not worth turning it into an argument.

I think different maintainers have different styles. I for one always
dislike to have code movement, renaming or code style fixes together
with any other more meaningful changes. In fact when people suggest
"could you please also rename this variable" or "could you please also
move the function to common code", I usually reply: "I can, but it will
be in another patch".

So I agree with Julien that I would prefer to see two patches. In fact
if I were you, I would prefer to write two separate patches because it
would be less troubles for me as a developer. But clearly not everybody
agrees with this style as I get requests for cosmetic changes together
with other changes by many other seasoned maintainers. And that's OK, it
just means that different people think differently, which is a good
thing for the project as a whole.

You are a valued contributor and maintainer of this project -- if you
strongly feel this way, I am sure we can find a way to make this work
anyway.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v3 05/13] sections.h: add sections header to collect all section info

[Steven Rostedt]

On Fri, 22 Jul 2016 22:37:16 +0100
James Hogan  wrote:


> > --- /dev/null
> > +++ b/arch/alpha/include/asm/sections.h
> > @@ -0,0 +1,6 @@
> > +#ifndef _ASM_ALPHA_SECTIONS_H
> > +#define _ASM_ALPHA_SECTIONS_H
> > +
> > +#include 
> > +
> > +#endif /* _ASM_ALPHA_SECTIONS_H */  
> 
> Any particular reason not to use generic-y in the Kbuild files for
> sections.h, ranges.h, and tables.h?

One of my TODOs is to simplify that process. That is, to get rid of the
having to add it to all archs and get rid of the Kbuild file altogether.

-- Steve

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH linux v2] xen: change the type of xen_vcpu_id to uint32_t

[Stefano Stabellini]

On Fri, 29 Jul 2016, Vitaly Kuznetsov wrote:
> We pass xen_vcpu_id mapping information to hypercalls which require
> uint32_t type so it would be cleaner to have it as uint32_t. The
> initializer to -1 can be dropped as we always do the mapping before using
> it and we never check the 'not set' value anyway.
> 
> Signed-off-by: Vitaly Kuznetsov 

It would probably be better to keep the initializer and add an assert to
xen_vcpu_nr. But I don't want to be too picky, so:

Reviewed-by: Stefano Stabellini 


> Changes since v2:
> - Change the return value type of xen_vcpu_nr to uint32_t [David Vrabel]
> ---
>  arch/arm/xen/enlighten.c | 2 +-
>  arch/x86/xen/enlighten.c | 2 +-
>  include/xen/xen-ops.h| 4 ++--
>  3 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/arch/arm/xen/enlighten.c b/arch/arm/xen/enlighten.c
> index 6d3a171..1752116 100644
> --- a/arch/arm/xen/enlighten.c
> +++ b/arch/arm/xen/enlighten.c
> @@ -47,7 +47,7 @@ DEFINE_PER_CPU(struct vcpu_info *, xen_vcpu);
>  static struct vcpu_info __percpu *xen_vcpu_info;
>  
>  /* Linux <-> Xen vCPU id mapping */
> -DEFINE_PER_CPU(int, xen_vcpu_id) = -1;
> +DEFINE_PER_CPU(uint32_t, xen_vcpu_id);
>  EXPORT_PER_CPU_SYMBOL(xen_vcpu_id);
>  
>  /* These are unused until we support booting "pre-ballooned" */
> diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
> index 54eef1a..78a14a0 100644
> --- a/arch/x86/xen/enlighten.c
> +++ b/arch/x86/xen/enlighten.c
> @@ -120,7 +120,7 @@ DEFINE_PER_CPU(struct vcpu_info *, xen_vcpu);
>  DEFINE_PER_CPU(struct vcpu_info, xen_vcpu_info);
>  
>  /* Linux <-> Xen vCPU id mapping */
> -DEFINE_PER_CPU(int, xen_vcpu_id) = -1;
> +DEFINE_PER_CPU(uint32_t, xen_vcpu_id);
>  EXPORT_PER_CPU_SYMBOL(xen_vcpu_id);
>  
>  enum xen_domain_type xen_domain_type = XEN_NATIVE;
> diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h
> index a4926f1..aac1be8 100644
> --- a/include/xen/xen-ops.h
> +++ b/include/xen/xen-ops.h
> @@ -9,8 +9,8 @@
>  
>  DECLARE_PER_CPU(struct vcpu_info *, xen_vcpu);
>  
> -DECLARE_PER_CPU(int, xen_vcpu_id);
> -static inline int xen_vcpu_nr(int cpu)
> +DECLARE_PER_CPU(uint32_t, xen_vcpu_id);
> +static inline uint32_t xen_vcpu_nr(int cpu)
>  {
>   return per_cpu(xen_vcpu_id, cpu);
>  }
> -- 
> 2.7.4
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH RFC 01/12] x86/paging: introduce paging_set_allocation

[Andrew Cooper]

On 29/07/16 17:28, Roger Pau Monne wrote:
> diff --git a/xen/arch/x86/mm/paging.c b/xen/arch/x86/mm/paging.c
> index 107fc8c..1b270df 100644
> --- a/xen/arch/x86/mm/paging.c
> +++ b/xen/arch/x86/mm/paging.c
> @@ -953,6 +953,22 @@ void paging_write_p2m_entry(struct p2m_domain *p2m, 
> unsigned long gfn,
>  safe_write_pte(p, new);
>  }
>  
> +int paging_set_allocation(struct domain *d, unsigned long pages)
> +{
> +int rc;
> +
> +ASSERT(paging_mode_enabled(d));
> +
> +paging_lock(d);
> +if ( hap_enabled(d) )
> +rc = hap_set_allocation(d, pages, NULL);
> +else
> +rc = sh_set_allocation(d, pages, NULL);

(without looking at the rest of the series) Your NMI is probably a
watchdog timeout from this call, as passing NULL means "non-preemptible".

As this is for the construction of dom0, it would be better to take a
preemptible pointer, loop in construct_dom0(), with a
process_pending_softirqs() call in between.

> +paging_unlock(d);
> +
> +return rc;
> +}
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
> index c22362f..452e22e 100644
> --- a/xen/arch/x86/mm/shadow/common.c
> +++ b/xen/arch/x86/mm/shadow/common.c
> @@ -1604,9 +1604,8 @@ shadow_free_p2m_page(struct domain *d, struct page_info 
> *pg)
>   * Input will be rounded up to at least shadow_min_acceptable_pages(),
>   * plus space for the p2m table.
>   * Returns 0 for success, non-zero for failure. */
> -static unsigned int sh_set_allocation(struct domain *d,
> -  unsigned int pages,
> -  int *preempted)
> +unsigned int sh_set_allocation(struct domain *d, unsigned int pages,
> +   int *preempted)
>  {
>  struct page_info *sp;
>  unsigned int lower_bound;
> diff --git a/xen/include/asm-x86/hap.h b/xen/include/asm-x86/hap.h
> index c613836..e3c9c98 100644
> --- a/xen/include/asm-x86/hap.h
> +++ b/xen/include/asm-x86/hap.h
> @@ -46,7 +46,8 @@ int   hap_track_dirty_vram(struct domain *d,
> XEN_GUEST_HANDLE_64(uint8) dirty_bitmap);
>  
>  extern const struct paging_mode *hap_paging_get_mode(struct vcpu *);
> -void hap_set_alloc_for_pvh_dom0(struct domain *d, unsigned long num_pages);
> +unsigned int hap_set_allocation(struct domain *d, unsigned int pages,
> + int *preempted);

I also note from this change that there is an unsigned long => unsigned
int truncation in the internals of *_set_allocation().  This should
definitely be fixed, although possibly wants to be a separate change.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XTF PATCH] xtf-runner: fix two synchronisation issues

[Wei Liu]

Thanks for writing this up.

The whole email described accurately what we discussed.

On Fri, Jul 29, 2016 at 05:18:10PM +0100, Ian Jackson wrote:
> The three of us had an IRL conversation.  Here is what I think we
> agreed:
> 
> * We intend to make the XTF runner capable of reading
>   xenconsoled-created logfiles.  Both XenRT and osstest configure
>   xenconsoled appropriately.
> 
>   The xtf runner will need to
> 
>  - on each test, wait for the test domain to shutdown, and then
> 
>  - look backwards through the xenconsoled logfile for the
>indication that the domain started (eg, a banner printed by the
>domain, or message from xenconsoled), and parse the relevant
>output there.  (This is needed because starting the same-named
>domain multiple times results in concatenations of the console
>output in the xenconsoled guest output logfile.)
> 
>  - arrange for the guest to be preserved on crash (at least)
>so that if the domain crashed, we don't risk parsing the
>output from a previous run.  Instead we can see it having
>crashed and report that as a failure.
> 
> * We intend to make `xl create -c' work to find all of the output,
>   by (i) starting the domain paused (ii) spawning xenconsoled
>   (iii) awaiting a new startup indication from xenconsoled
>   (iv) unpausing the domain.  This will be done in xen-unstable.
> 
>   I propose the following startup protocol: xl runs
>  xenconsole --startup-notify-fd=FD
>   where FD is the writing end of a pipe.  xl waits for xenconsole to
>   write the byte 0x00 to the FD.  If xenconsoled crashes, xl can tell
>   by the EOF on the pipe.  (This approach saves xl from having to try
>   to wait for either SIGCHLD or fd readability.)
> 
>   (The systemd startup notification protocol is too complex to
>   reimplement and would therefore introduce a dependency on
>   libsystemd's sd_notify, which would be awkward.  There is also the
>   upstart SIGSTOP protocol but it could interact badly with an
>   interactive user who uses ^Z.)
> 
>   The xtf runner would also be able to use `xl create -c' and simply
>   expect to see all the console output.
> 
>   (We also discussed making xenconsole print something to its stdout
>   or stderr when it has successfully connected, and when it
>   disconnects.  While we're editing xenconsole it would probably be
>   nice to do this, but with our plans it's not needed for XTF.)
> 

FTR, s/xenconsoled/xenconsole/ in this part.

I'm going to ditch this patch and look into implementing this. With this
I can ditch all the crazy gross hack in this patch.

> As a result, the xtf runner can be used with a default install of
> xen-unstable.  For older versions of Xen it will be necessary to
> reconfigure xenconsoled, if the user wants to get reliable pass/fail
> reports from the xtf runner.
> 
> * We discussed changing xenconsoled to not tear down the guest console
>   until the guest is destroyed, rather than already tearing it down
>   when the guest is shut down.  This is not now needed for the above,
>   but I still think it would be nice.  However it is done, it should
>   arrange that `xl console' doesn't hang waiting for further output
>   from a crashed or shutdown domain (but perhaps should wait for
>   output from a rebooting domain?).  It would probably be a good idea
>   to put this work item in a bucket with `overhaul the console stuff'.
> 
> Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH RFC 01/12] PVHv2 Dom0

[Roger Pau Monne]

Hello,

Sorry, I've failed to Cc both Konrad and Boris on this...

On Fri, Jul 29, 2016 at 06:28:55PM +0200, Roger Pau Monne wrote:
> Hello,
> 
> This is a very rough implementation of a PVHv2 Dom0. There are still a bunch 
> of things that will not work properly (like SR-IOV, MSI, MSI-X...), but it 
> *should* be able to boot a Dom0 kernel and it doesn't require using PIRQs.
> 
> There are some changes compared to a traditional PV or PVH Dom0:
> 
>  - An emulated local APIC (for each vCPU) and IO APIC is provided to Dom0.
>  - At the start of day only the low 1MB and the ACPI e820 regions are mapped 
>into the guest using 1:1 mappings (for UEFI systems mapping the low 1MB 
>probably doesn't make any sense, but alas).
>  - The MADT has been replaced in order to reflect the real topology seen by 
>Dom0.
>  - In order to have the BARs of PCI devices mapped Dom0 must call 
>PHYSDEVOP_pci_device_add. See the notes on patch 11 for more information 
>(and what's missing).
>  - ATM only legacy PCI interrupts are supported. This is implemented by 
>looking at the writes Dom0 makes to the emulated IO APIC and translating 
>them into the real hardware. Note that MSI or MSI-X capabilities are 
>_not_ masked from the PCI capabilities list, so the user must avoid using
>them.
>  - PCIe Enhanced Configuration Mechanism regions are not yet mapped into 
>Dom0 physmap.
>  - Some ACPI tables are zapped (it's signature is inverted) to prevent Dom0 
>from poking at them, those are: HPET, SLIT, SRAT, MPST and PMTT.
> 
> This is still very experimental, I've been able to boot a FreeBSD Dom0 using 
> 2GB and 4GB of memory assigned to it, but if I try to use 6GB Xen gets a NMI 
> (I've got no idea why yet). I don't think it's worth delaying this more, 
> because it's going to take me some time to finish all this work (MSI, 
> MSI-X, bug hunting...), and in the meantime people can already take a look 
> at what's done (or half-done).
> 
> I've pushed the whole series to my git repository:
> 
> git://xenbits.xen.org/people/royger/xen.git pvhv2_dom0_rfc
> 
> It contains two patches from Boris and Anthony, that are a pre-requisite to 
> this series.
> 
> As usual, thanks for taking the time to look into it, hope it doesn't make 
> your eyes bleed much (slight bleeding is expected).
> 
> Roger.
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XTF PATCH] xtf-runner: fix two synchronisation issues

[Andrew Cooper]

On 29/07/16 17:18, Ian Jackson wrote:
> The three of us had an IRL conversation.  Here is what I think we
> agreed:
>
> * We intend to make the XTF runner capable of reading
>   xenconsoled-created logfiles.  Both XenRT and osstest configure
>   xenconsoled appropriately.
>
>   The xtf runner will need to
>
>  - on each test, wait for the test domain to shutdown, and then
>
>  - look backwards through the xenconsoled logfile for the
>indication that the domain started (eg, a banner printed by the
>domain, or message from xenconsoled), and parse the relevant
>output there.  (This is needed because starting the same-named
>domain multiple times results in concatenations of the console
>output in the xenconsoled guest output logfile.)
>
>  - arrange for the guest to be preserved on crash (at least)
>so that if the domain crashed, we don't risk parsing the
>output from a previous run.  Instead we can see it having
>crashed and report that as a failure.

I suppose it is worth noting that this is by far and away the easiest
solution (we can think of) which also works with older versions of Xen.

>
> * We intend to make `xl create -c' work to find all of the output,
>   by (i) starting the domain paused (ii) spawning xenconsoled
>   (iii) awaiting a new startup indication from xenconsoled
>   (iv) unpausing the domain.  This will be done in xen-unstable.
>
>   I propose the following startup protocol: xl runs
>  xenconsole --startup-notify-fd=FD
>   where FD is the writing end of a pipe.  xl waits for xenconsole to
>   write the byte 0x00 to the FD.  If xenconsoled crashes, xl can tell
>   by the EOF on the pipe.  (This approach saves xl from having to try
>   to wait for either SIGCHLD or fd readability.)
>
>   (The systemd startup notification protocol is too complex to
>   reimplement and would therefore introduce a dependency on
>   libsystemd's sd_notify, which would be awkward.  There is also the
>   upstart SIGSTOP protocol but it could interact badly with an
>   interactive user who uses ^Z.)
>
>   The xtf runner would also be able to use `xl create -c' and simply
>   expect to see all the console output.
>
>   (We also discussed making xenconsole print something to its stdout
>   or stderr when it has successfully connected, and when it
>   disconnects.  While we're editing xenconsole it would probably be
>   nice to do this, but with our plans it's not needed for XTF.)
>
> As a result, the xtf runner can be used with a default install of
> xen-unstable.  For older versions of Xen it will be necessary to
> reconfigure xenconsoled, if the user wants to get reliable pass/fail
> reports from the xtf runner.

I expect that the common usecase will be that people will develop tests
against unstable, and only automated test systems will be running tests
against older versions.

I don't expect it to be common for a human to need to develop tests
against older versions, but if someone does need to, there is at least a
way of doing so.

>
> * We discussed changing xenconsoled to not tear down the guest console
>   until the guest is destroyed, rather than already tearing it down
>   when the guest is shut down.  This is not now needed for the above,
>   but I still think it would be nice.  However it is done, it should
>   arrange that `xl console' doesn't hang waiting for further output
>   from a crashed or shutdown domain (but perhaps should wait for
>   output from a rebooting domain?).  It would probably be a good idea
>   to put this work item in a bucket with `overhaul the console stuff'.

+1.

As for the rest of the email, this matches my understanding from the
conversation.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 03/12] xen/x86: allow the emulated APICs to be enbled for the hardware domain

[Roger Pau Monne]

Allow the used of both the emulated local APIC and IO APIC for the hardware
domain.

Signed-off-by: Roger Pau Monné 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/x86/domain.c | 22 ++
 1 file changed, 14 insertions(+), 8 deletions(-)

diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 1133ea2..4d47228 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -545,25 +545,31 @@ int arch_domain_create(struct domain *d, unsigned int 
domcr_flags,
 }
 else
 {
-if ( (config->emulation_flags & ~XEN_X86_EMU_ALL) != 0 )
+uint32_t emflags = config->emulation_flags;
+
+if ( (emflags & ~XEN_X86_EMU_ALL) != 0 )
 {
 printk(XENLOG_G_ERR "d%d: Invalid emulation bitmap: %#x\n",
-   d->domain_id, config->emulation_flags);
+   d->domain_id, emflags);
 return -EINVAL;
 }
+
 if ( is_hardware_domain(d) )
-config->emulation_flags |= XEN_X86_EMU_PIT;
-if ( config->emulation_flags != 0 &&
- (config->emulation_flags !=
-  (is_hvm_domain(d) ? XEN_X86_EMU_ALL : XEN_X86_EMU_PIT)) )
+emflags |= XEN_X86_EMU_PIT;
+
+if ( (is_hardware_domain(d) ?
+  (is_hvm_domain(d) && emflags !=
+  (XEN_X86_EMU_PIT|XEN_X86_EMU_LAPIC|XEN_X86_EMU_IOAPIC)) :
+  (emflags && (is_hvm_domain(d) ? (emflags != XEN_X86_EMU_ALL) :
+  (emflags != XEN_X86_EMU_PIT )
 {
 printk(XENLOG_G_ERR "d%d: Xen does not allow %s domain creation "
"with the current selection of emulators: %#x\n",
d->domain_id, is_hvm_domain(d) ? "HVM" : "PV",
-   config->emulation_flags);
+   emflags);
 return -EOPNOTSUPP;
 }
-d->arch.emulation_flags = config->emulation_flags;
+d->arch.emulation_flags = emflags;
 }
 
 if ( has_hvm_container_domain(d) )
-- 
2.7.4 (Apple Git-66)


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 12/12] xen/x86: route legacy PCI interrupts to Dom0

[Roger Pau Monne]

This is done adding some Dom0 specific logic to the IO APIC emulation inside
of Xen, so that writes to the IO APIC registers that should unmask an
interrupt will take care of setting up this interrupt with Xen. A Dom0
specific EIO handler also has to be used, since Xen doesn't know the
topology of the PCI devices and it just has to passthrough what Dom0 does.

Signed-off-by: Roger Pau Monné 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 
Cc: Paul Durrant 
---
 xen/arch/x86/hvm/irq.c   |   9 +++
 xen/arch/x86/hvm/vioapic.c   |  28 -
 xen/arch/x86/physdev.c   |   6 +-
 xen/drivers/passthrough/io.c | 144 ++-
 xen/include/asm-x86/hvm/io.h |   2 +
 xen/include/asm-x86/irq.h|   5 ++
 xen/include/xen/hvm/irq.h|   3 +
 xen/include/xen/iommu.h  |   1 +
 8 files changed, 178 insertions(+), 20 deletions(-)

diff --git a/xen/arch/x86/hvm/irq.c b/xen/arch/x86/hvm/irq.c
index 5323d7c..be9b648 100644
--- a/xen/arch/x86/hvm/irq.c
+++ b/xen/arch/x86/hvm/irq.c
@@ -88,6 +88,15 @@ void hvm_pci_intx_assert(
 spin_unlock(&d->arch.hvm_domain.irq_lock);
 }
 
+void hvm_hw_gsi_assert(struct domain *d, unsigned int gsi)
+{
+
+ASSERT(is_hardware_domain(d));
+spin_lock(&d->arch.hvm_domain.irq_lock);
+assert_gsi(d, gsi);
+spin_unlock(&d->arch.hvm_domain.irq_lock);
+}
+
 static void __hvm_pci_intx_deassert(
 struct domain *d, unsigned int device, unsigned int intx)
 {
diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c
index 611be87..18305be 100644
--- a/xen/arch/x86/hvm/vioapic.c
+++ b/xen/arch/x86/hvm/vioapic.c
@@ -148,6 +148,29 @@ static void vioapic_write_redirent(
 unmasked = unmasked && !ent.fields.mask;
 }
 
+if ( is_hardware_domain(d) && unmasked )
+{
+int ret, gsi;
+
+/* Interrupt has been unmasked */
+gsi = idx;
+ret = mp_register_gsi(gsi, ent.fields.trig_mode, ent.fields.polarity);
+if ( ret && ret != -EEXIST )
+{
+gdprintk(XENLOG_WARNING,
+ "%s: error registering GSI %d\n", __func__, ret);
+}
+if ( !ret )
+{
+ret = physdev_map_pirq(DOMID_SELF, MAP_PIRQ_TYPE_GSI, &gsi, &gsi,
+   NULL);
+BUG_ON(ret);
+
+ret = pt_irq_bind_hw_domain(gsi);
+BUG_ON(ret);
+}
+}
+
 *pent = ent;
 
 if ( idx == 0 )
@@ -409,7 +432,10 @@ void vioapic_update_EOI(struct domain *d, u8 vector)
 if ( iommu_enabled )
 {
 spin_unlock(&d->arch.hvm_domain.irq_lock);
-hvm_dpci_eoi(d, gsi, ent);
+if ( is_hardware_domain(d) )
+hvm_hw_dpci_eoi(d, gsi, ent);
+else
+hvm_dpci_eoi(d, gsi, ent);
 spin_lock(&d->arch.hvm_domain.irq_lock);
 }
 
diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c
index 5a49796..8b13a36 100644
--- a/xen/arch/x86/physdev.c
+++ b/xen/arch/x86/physdev.c
@@ -19,10 +19,6 @@
 #include 
 #include 
 
-int physdev_map_pirq(domid_t, int type, int *index, int *pirq_p,
- struct msi_info *);
-int physdev_unmap_pirq(domid_t, int pirq);
-
 #include "x86_64/mmconfig.h"
 
 #ifndef COMPAT
@@ -94,7 +90,7 @@ int physdev_map_pirq(domid_t domid, int type, int *index, int 
*pirq_p,
 int pirq, irq, ret = 0;
 void *map_data = NULL;
 
-if ( domid == DOMID_SELF && is_hvm_domain(d) )
+if ( domid == DOMID_SELF && !is_hardware_domain(d) && is_hvm_domain(d) )
 {
 /*
  * Only makes sense for vector-based callback, else HVM-IRQ logic
diff --git a/xen/drivers/passthrough/io.c b/xen/drivers/passthrough/io.c
index 9e6b46c..56af98c 100644
--- a/xen/drivers/passthrough/io.c
+++ b/xen/drivers/passthrough/io.c
@@ -159,26 +159,29 @@ static int pt_irq_guest_eoi(struct domain *d, struct 
hvm_pirq_dpci *pirq_dpci,
 static void pt_irq_time_out(void *data)
 {
 struct hvm_pirq_dpci *irq_map = data;
-const struct hvm_irq_dpci *dpci;
 const struct dev_intx_gsi_link *digl;
 
 spin_lock(&irq_map->dom->event_lock);
 
-dpci = domain_get_irq_dpci(irq_map->dom);
-ASSERT(dpci);
-list_for_each_entry ( digl, &irq_map->digl_list, list )
+if ( !is_hardware_domain(irq_map->dom) )
 {
-unsigned int guest_gsi = hvm_pci_intx_gsi(digl->device, digl->intx);
-const struct hvm_girq_dpci_mapping *girq;
-
-list_for_each_entry ( girq, &dpci->girq[guest_gsi], list )
+const struct hvm_irq_dpci *dpci = domain_get_irq_dpci(irq_map->dom);
+ASSERT(dpci);
+list_for_each_entry ( digl, &irq_map->digl_list, list )
 {
-struct pirq *pirq = pirq_info(irq_map->dom, girq->machine_gsi);
+unsigned int guest_gsi = hvm_pci_intx_gsi(digl->device, 
digl->intx);
+const struct hvm_girq_dpci_mapping *girq;
+
+list_for_each_entry ( girq, &dpci->girq[guest_gsi], list )
+{
+str

[Xen-devel] [PATCH RFC 11/12] xen/x86: allow a PVHv2 Dom0 to register PCI devices with Xen

[Roger Pau Monne]

This patch allows a PVHv2 Dom0 to register PCI devices with Xen using
PHYSDEVOP_pci_mmcfg_reserved, PHYSDEVOP_pci_device_add and
PHYSDEVOP_pci_device_remove. The functionality of the pci_device_add
function has been expanded so that for PVHv2 Dom0 it also sizes the PCI
device BARs and adds them to the Dom0 memory map using a 1:1 mapping.

Signed-off-by: Roger Pau Monne 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
This is incomplete, devices with SR-IOV BARs are not going to be properly
mapped, pci_remove_device will not unmap the BARs, and if the Dom0 OS
changes the position of the BARs the 1:1 mappings are not going to be
updated, so Dom0 is going to lose access to them...
---
 xen/arch/x86/hvm/hvm.c|   6 ++
 xen/drivers/passthrough/pci.c | 153 +-
 2 files changed, 127 insertions(+), 32 deletions(-)

diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index db4b2d6..9434540 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -4029,6 +4029,12 @@ static long hvm_physdev_op(int cmd, 
XEN_GUEST_HANDLE_PARAM(void) arg)
 if ( !is_pvh_vcpu(current) || !is_hardware_domain(current->domain) )
 return -ENOSYS;
 /* fall through */
+case PHYSDEVOP_pci_mmcfg_reserved:
+case PHYSDEVOP_pci_device_add:
+case PHYSDEVOP_pci_device_remove:
+if ( !is_hardware_domain(current->domain) )
+return -ENOSYS;
+/* fall through */
 case PHYSDEVOP_map_pirq:
 case PHYSDEVOP_unmap_pirq:
 case PHYSDEVOP_eoi:
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index e3595a9..18687e0 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -587,6 +587,52 @@ static void pci_enable_acs(struct pci_dev *pdev)
 pci_conf_write16(seg, bus, dev, func, pos + PCI_ACS_CTRL, ctrl);
 }
 
+static int pci_size_bar(unsigned int seg, unsigned int bus, unsigned int slot,
+unsigned int func, unsigned int base,
+unsigned int max_bars, unsigned int *index,
+uint64_t *addr, uint64_t *size)
+{
+unsigned int idx = base + *index * 4;
+u32 bar = pci_conf_read32(seg, bus, slot, func, idx);
+u32 hi = 0;
+
+*addr = *size = 0;
+
+ASSERT((bar & PCI_BASE_ADDRESS_SPACE) == PCI_BASE_ADDRESS_SPACE_MEMORY);
+pci_conf_write32(seg, bus, slot, func, idx, ~0);
+if ( (bar & PCI_BASE_ADDRESS_MEM_TYPE_MASK) ==
+ PCI_BASE_ADDRESS_MEM_TYPE_64 )
+{
+if ( *index >= max_bars )
+{
+printk(XENLOG_WARNING
+   "device %04x:%02x:%02x.%u with 64-bit BAR in last slot\n",
+   seg, bus, slot, func);
+return -EINVAL;
+}
+hi = pci_conf_read32(seg, bus, slot, func, idx + 4);
+pci_conf_write32(seg, bus, slot, func, idx + 4, ~0);
+}
+*size = pci_conf_read32(seg, bus, slot, func, idx) &
+PCI_BASE_ADDRESS_MEM_MASK;
+if ( (bar & PCI_BASE_ADDRESS_MEM_TYPE_MASK) ==
+ PCI_BASE_ADDRESS_MEM_TYPE_64 )
+{
+*size |= (u64)pci_conf_read32(seg, bus, slot, func, idx + 4) << 32;
+pci_conf_write32(seg, bus, slot, func, idx + 4, hi);
+}
+else if ( *size )
+*size |= (u64)~0 << 32;
+pci_conf_write32(seg, bus, slot, func, idx, bar);
+*size = - *size;
+*addr = (bar & PCI_BASE_ADDRESS_MEM_MASK) | ((u64)hi << 32);
+if ( (bar & PCI_BASE_ADDRESS_MEM_TYPE_MASK) ==
+ PCI_BASE_ADDRESS_MEM_TYPE_64 )
+++*index;
+
+return 0;
+}
+
 int pci_add_device(u16 seg, u8 bus, u8 devfn,
const struct pci_dev_info *info, nodeid_t node)
 {
@@ -651,7 +697,7 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn,
 {
 unsigned int idx = pos + PCI_SRIOV_BAR + i * 4;
 u32 bar = pci_conf_read32(seg, bus, slot, func, idx);
-u32 hi = 0;
+uint64_t addr;
 
 if ( (bar & PCI_BASE_ADDRESS_SPACE) ==
  PCI_BASE_ADDRESS_SPACE_IO )
@@ -662,38 +708,13 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn,
seg, bus, slot, func, i);
 continue;
 }
-pci_conf_write32(seg, bus, slot, func, idx, ~0);
-if ( (bar & PCI_BASE_ADDRESS_MEM_TYPE_MASK) ==
- PCI_BASE_ADDRESS_MEM_TYPE_64 )
-{
-if ( i >= PCI_SRIOV_NUM_BARS )
-{
+ret = pci_size_bar(seg, bus, slot, func, pos + PCI_SRIOV_BAR,
+   PCI_SRIOV_NUM_BARS, &i, &addr,
+   &pdev->vf_rlen[i]);
+if ( ret )
 printk(XENLOG_WARNING
-   "SR-IOV device %04x:%02x:%02x.%u with 64-bit"
-   " vf BAR in last slot\n",
-   seg, bus, slot, func);
-  

[Xen-devel] [PATCH RFC 08/12] xen/x86: setup PVHv2 Dom0 CPUs

[Roger Pau Monne]

Initialize Dom0 BSP/APs and setup the memory and IO permissions.

Signed-off-by: Roger Pau Monné 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
The logic used to setup the CPUID leaves is extremely simplistic (and
probably wrong for hardware different than mine). I'm not sure what's the
best way to deal with this, the code that currently sets the CPUID leaves
for HVM guests lives in libxc, maybe moving it xen/common would be better?
---
 xen/arch/x86/domain_build.c | 103 
 1 file changed, 103 insertions(+)

diff --git a/xen/arch/x86/domain_build.c b/xen/arch/x86/domain_build.c
index df6354a..89ef59e 100644
--- a/xen/arch/x86/domain_build.c
+++ b/xen/arch/x86/domain_build.c
@@ -40,6 +40,7 @@
 
 #include 
 #include 
+#include 
 
 static long __initdata dom0_nrpages;
 static long __initdata dom0_min_nrpages;
@@ -1948,6 +1949,101 @@ out:
 return rc;
 }
 
+static int __init hvm_setup_cpus(struct domain *d, paddr_t entry,
+ paddr_t start_info)
+{
+vcpu_hvm_context_t cpu_ctx;
+struct vcpu *v = d->vcpu[0];
+int cpu, i, rc;
+struct {
+uint32_t index;
+uint32_t count;
+} cpuid_leaves[] = {
+{0, XEN_CPUID_INPUT_UNUSED},
+{1, XEN_CPUID_INPUT_UNUSED},
+{2, XEN_CPUID_INPUT_UNUSED},
+{4, 0},
+{4, 1},
+{4, 2},
+{4, 3},
+{4, 4},
+{7, 0},
+{0xa, XEN_CPUID_INPUT_UNUSED},
+{0xd, 0},
+{0x8000, XEN_CPUID_INPUT_UNUSED},
+{0x8001, XEN_CPUID_INPUT_UNUSED},
+{0x8002, XEN_CPUID_INPUT_UNUSED},
+{0x8003, XEN_CPUID_INPUT_UNUSED},
+{0x8004, XEN_CPUID_INPUT_UNUSED},
+{0x8005, XEN_CPUID_INPUT_UNUSED},
+{0x8006, XEN_CPUID_INPUT_UNUSED},
+{0x8007, XEN_CPUID_INPUT_UNUSED},
+{0x8008, XEN_CPUID_INPUT_UNUSED},
+};
+
+printk("** Setting up BSP/APs **\n");
+
+cpu = v->processor;
+for ( i = 1; i < d->max_vcpus; i++ )
+{
+cpu = cpumask_cycle(cpu, &dom0_cpus);
+setup_dom0_vcpu(d, i, cpu);
+}
+
+memset(&cpu_ctx, 0, sizeof(cpu_ctx));
+
+cpu_ctx.mode = VCPU_HVM_MODE_32B;
+
+cpu_ctx.cpu_regs.x86_32.ebx = start_info;
+cpu_ctx.cpu_regs.x86_32.eip = entry;
+cpu_ctx.cpu_regs.x86_32.cr0 = X86_CR0_PE | X86_CR0_ET;
+
+cpu_ctx.cpu_regs.x86_32.cs_limit = ~0u;
+cpu_ctx.cpu_regs.x86_32.ds_limit = ~0u;
+cpu_ctx.cpu_regs.x86_32.ss_limit = ~0u;
+cpu_ctx.cpu_regs.x86_32.tr_limit = 0x67;
+cpu_ctx.cpu_regs.x86_32.cs_ar = 0xc9b;
+cpu_ctx.cpu_regs.x86_32.ds_ar = 0xc93;
+cpu_ctx.cpu_regs.x86_32.ss_ar = 0xc93;
+cpu_ctx.cpu_regs.x86_32.tr_ar = 0x8b;
+
+rc = arch_set_info_hvm_guest(v, &cpu_ctx);
+if ( rc )
+{
+printk("Unable to setup Dom0 BSP context: %d\n", rc);
+return rc;
+}
+clear_bit(_VPF_down, &v->pause_flags);
+
+for ( i = 0; i < ARRAY_SIZE(cpuid_leaves); i++ )
+{
+d->arch.cpuids[i].input[0] = cpuid_leaves[i].index;
+d->arch.cpuids[i].input[1] = cpuid_leaves[i].count;
+if ( d->arch.cpuids[i].input[1] == XEN_CPUID_INPUT_UNUSED )
+cpuid(d->arch.cpuids[i].input[0], &d->arch.cpuids[i].eax,
+  &d->arch.cpuids[i].ebx, &d->arch.cpuids[i].ecx,
+  &d->arch.cpuids[i].edx);
+else
+cpuid_count(d->arch.cpuids[i].input[0], d->arch.cpuids[i].input[1],
+&d->arch.cpuids[i].eax, &d->arch.cpuids[i].ebx,
+&d->arch.cpuids[i].ecx, &d->arch.cpuids[i].edx);
+/* XXX: we need to do much more filtering here. */
+if ( d->arch.cpuids[i].input[0] == 1 )
+d->arch.cpuids[i].ecx &= ~X86_FEATURE_VMX;
+}
+
+rc = setup_permissions(d);
+if ( rc )
+{
+panic("Unable to setup Dom0 permissions: %d\n", rc);
+return rc;
+}
+
+update_domain_wallclock_time(d);
+
+return 0;
+}
+
 static int __init construct_dom0_hvm(struct domain *d, const module_t *image,
  unsigned long image_headroom,
  module_t *initrd,
@@ -1982,6 +2078,13 @@ static int __init construct_dom0_hvm(struct domain *d, 
const module_t *image,
 return rc;
 }
 
+rc = hvm_setup_cpus(d, entry, start_info);
+if ( rc )
+{
+printk("Failed to setup Dom0 CPUs: %d\n", rc);
+return rc;
+}
+
 return 0;
 }
 
-- 
2.7.4 (Apple Git-66)


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 02/12] xen/x86: split the setup of Dom0 permissions to a function

[Roger Pau Monne]

So that it can also be used by the PVH-specific domain builder. This is just
code motion, it should not introduce any functional change.

Signed-off-by: Roger Pau Monné 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/x86/domain_build.c | 164 +++-
 1 file changed, 86 insertions(+), 78 deletions(-)

diff --git a/xen/arch/x86/domain_build.c b/xen/arch/x86/domain_build.c
index d7d4afc..09d79be 100644
--- a/xen/arch/x86/domain_build.c
+++ b/xen/arch/x86/domain_build.c
@@ -869,6 +869,89 @@ static __init void setup_pv_physmap(struct domain *d, 
unsigned long pgtbl_pfn,
 unmap_domain_page(l4start);
 }
 
+static int __init setup_permissions(struct domain *d)
+{
+unsigned long mfn;
+int i, rc = 0;
+
+/* The hardware domain is initially permitted full I/O capabilities. */
+rc |= ioports_permit_access(d, 0, 0x);
+rc |= iomem_permit_access(d, 0UL, (1UL << (paddr_bits - PAGE_SHIFT)) - 1);
+rc |= irqs_permit_access(d, 1, nr_irqs_gsi - 1);
+
+/*
+ * Modify I/O port access permissions.
+ */
+/* Master Interrupt Controller (PIC). */
+rc |= ioports_deny_access(d, 0x20, 0x21);
+/* Slave Interrupt Controller (PIC). */
+rc |= ioports_deny_access(d, 0xA0, 0xA1);
+/* Interval Timer (PIT). */
+rc |= ioports_deny_access(d, 0x40, 0x43);
+/* PIT Channel 2 / PC Speaker Control. */
+rc |= ioports_deny_access(d, 0x61, 0x61);
+/* ACPI PM Timer. */
+if ( pmtmr_ioport )
+rc |= ioports_deny_access(d, pmtmr_ioport, pmtmr_ioport + 3);
+/* PCI configuration space (NB. 0xcf8 has special treatment). */
+rc |= ioports_deny_access(d, 0xcfc, 0xcff);
+/* Command-line I/O ranges. */
+process_dom0_ioports_disable(d);
+
+/*
+ * Modify I/O memory access permissions.
+ */
+/* Local APIC. */
+if ( mp_lapic_addr != 0 )
+{
+mfn = paddr_to_pfn(mp_lapic_addr);
+rc |= iomem_deny_access(d, mfn, mfn);
+}
+/* I/O APICs. */
+for ( i = 0; i < nr_ioapics; i++ )
+{
+mfn = paddr_to_pfn(mp_ioapics[i].mpc_apicaddr);
+if ( !rangeset_contains_singleton(mmio_ro_ranges, mfn) )
+rc |= iomem_deny_access(d, mfn, mfn);
+}
+/* MSI range. */
+rc |= iomem_deny_access(d, paddr_to_pfn(MSI_ADDR_BASE_LO),
+paddr_to_pfn(MSI_ADDR_BASE_LO +
+ MSI_ADDR_DEST_ID_MASK));
+/* HyperTransport range. */
+if ( boot_cpu_data.x86_vendor == X86_VENDOR_AMD )
+rc |= iomem_deny_access(d, paddr_to_pfn(0xfdULL << 32),
+paddr_to_pfn((1ULL << 40) - 1));
+
+/* Remove access to E820_UNUSABLE I/O regions above 1MB. */
+for ( i = 0; i < e820.nr_map; i++ )
+{
+unsigned long sfn, efn;
+sfn = max_t(unsigned long, paddr_to_pfn(e820.map[i].addr), 0x100ul);
+efn = paddr_to_pfn(e820.map[i].addr + e820.map[i].size - 1);
+if ( (e820.map[i].type == E820_UNUSABLE) &&
+ (e820.map[i].size != 0) &&
+ (sfn <= efn) )
+rc |= iomem_deny_access(d, sfn, efn);
+}
+
+/* Prevent access to HPET */
+if ( hpet_address )
+{
+u8 prot_flags = hpet_flags & ACPI_HPET_PAGE_PROTECT_MASK;
+
+mfn = paddr_to_pfn(hpet_address);
+if ( prot_flags == ACPI_HPET_PAGE_PROTECT4 )
+rc |= iomem_deny_access(d, mfn, mfn);
+else if ( prot_flags == ACPI_HPET_PAGE_PROTECT64 )
+rc |= iomem_deny_access(d, mfn, mfn + 15);
+else if ( ro_hpet )
+rc |= rangeset_add_singleton(mmio_ro_ranges, mfn);
+}
+
+return rc;
+}
+
 int __init construct_dom0(
 struct domain *d,
 const module_t *image, unsigned long image_headroom,
@@ -1529,84 +1612,9 @@ int __init construct_dom0(
 if ( test_bit(XENFEAT_supervisor_mode_kernel, parms.f_required) )
 panic("Dom0 requires supervisor-mode execution");
 
-rc = 0;
-
-/* The hardware domain is initially permitted full I/O capabilities. */
-rc |= ioports_permit_access(d, 0, 0x);
-rc |= iomem_permit_access(d, 0UL, (1UL << (paddr_bits - PAGE_SHIFT)) - 1);
-rc |= irqs_permit_access(d, 1, nr_irqs_gsi - 1);
-
-/*
- * Modify I/O port access permissions.
- */
-/* Master Interrupt Controller (PIC). */
-rc |= ioports_deny_access(d, 0x20, 0x21);
-/* Slave Interrupt Controller (PIC). */
-rc |= ioports_deny_access(d, 0xA0, 0xA1);
-/* Interval Timer (PIT). */
-rc |= ioports_deny_access(d, 0x40, 0x43);
-/* PIT Channel 2 / PC Speaker Control. */
-rc |= ioports_deny_access(d, 0x61, 0x61);
-/* ACPI PM Timer. */
-if ( pmtmr_ioport )
-rc |= ioports_deny_access(d, pmtmr_ioport, pmtmr_ioport + 3);
-/* PCI configuration space (NB. 0xcf8 has special treatment). */
-rc |= ioports_deny_access(d, 0xcfc, 0xcff);
-/* Command-line I/O ranges. */
-process_dom0_ioports_disable(d);
-
-/*
- * Modify I/O memory

[Xen-devel] [PATCH RFC 10/12] xen/dcpi: add a dpci passthrough handler for hardware domain

[Roger Pau Monne]

This is very similar to the PCI trap used for the traditional PV(H) Dom0.

Signed-off-by: Roger Pau Monné 
---
Cc: Paul Durrant 
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/x86/hvm/io.c | 72 ++-
 xen/arch/x86/traps.c  | 39 ---
 xen/drivers/passthrough/pci.c | 39 +++
 xen/include/xen/pci.h |  2 ++
 4 files changed, 112 insertions(+), 40 deletions(-)

diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c
index 1e7a5f9..31d54dc 100644
--- a/xen/arch/x86/hvm/io.c
+++ b/xen/arch/x86/hvm/io.c
@@ -247,12 +247,79 @@ static int dpci_portio_write(const struct hvm_io_handler 
*handler,
 return X86EMUL_OKAY;
 }
 
+static bool_t hw_dpci_portio_accept(const struct hvm_io_handler *handler,
+const ioreq_t *p)
+{
+if ( (p->addr == 0xcf8 && p->size == 4) || (p->addr & 0xfffc) == 0xcfc)
+{
+return 1;
+}
+
+return 0;
+}
+
+static int hw_dpci_portio_read(const struct hvm_io_handler *handler,
+uint64_t addr,
+uint32_t size,
+uint64_t *data)
+{
+struct domain *currd = current->domain;
+
+if ( addr == 0xcf8 )
+{
+ASSERT(size == 4);
+*data = currd->arch.pci_cf8;
+return X86EMUL_OKAY;
+}
+
+ASSERT((addr & 0xfffc) == 0xcfc);
+size = min(size, 4 - ((uint32_t)addr & 3));
+if ( size == 3 )
+size = 2;
+if ( pci_cfg_ok(currd, addr & 3, size, NULL) )
+*data = pci_conf_read(currd->arch.pci_cf8, addr & 3, size);
+
+return X86EMUL_OKAY;
+}
+
+static int hw_dpci_portio_write(const struct hvm_io_handler *handler,
+uint64_t addr,
+uint32_t size,
+uint64_t data)
+{
+struct domain *currd = current->domain;
+uint32_t data32;
+
+if ( addr == 0xcf8 )
+{
+ASSERT(size == 4);
+currd->arch.pci_cf8 = data;
+return X86EMUL_OKAY;
+}
+
+ASSERT((addr & 0xfffc) == 0xcfc);
+size = min(size, 4 - ((uint32_t)addr & 3));
+if ( size == 3 )
+size = 2;
+data32 = data;
+if ( pci_cfg_ok(currd, addr & 3, size, &data32) )
+pci_conf_write(currd->arch.pci_cf8, addr & 3, size, data);
+
+return X86EMUL_OKAY;
+}
+
 static const struct hvm_io_ops dpci_portio_ops = {
 .accept = dpci_portio_accept,
 .read = dpci_portio_read,
 .write = dpci_portio_write
 };
 
+static const struct hvm_io_ops hw_dpci_portio_ops = {
+.accept = hw_dpci_portio_accept,
+.read = hw_dpci_portio_read,
+.write = hw_dpci_portio_write
+};
+
 void register_dpci_portio_handler(struct domain *d)
 {
 struct hvm_io_handler *handler = hvm_next_io_handler(d);
@@ -261,7 +328,10 @@ void register_dpci_portio_handler(struct domain *d)
 return;
 
 handler->type = IOREQ_TYPE_PIO;
-handler->ops = &dpci_portio_ops;
+if ( is_hardware_domain(d) )
+handler->ops = &hw_dpci_portio_ops;
+else
+handler->ops = &dpci_portio_ops;
 }
 
 /*
diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index 767d0b0..4333bc1 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -2031,45 +2031,6 @@ static bool_t admin_io_okay(unsigned int port, unsigned 
int bytes,
 return ioports_access_permitted(d, port, port + bytes - 1);
 }
 
-static bool_t pci_cfg_ok(struct domain *currd, unsigned int start,
- unsigned int size, uint32_t *write)
-{
-uint32_t machine_bdf;
-
-if ( !is_hardware_domain(currd) )
-return 0;
-
-if ( !CF8_ENABLED(currd->arch.pci_cf8) )
-return 1;
-
-machine_bdf = CF8_BDF(currd->arch.pci_cf8);
-if ( write )
-{
-const unsigned long *ro_map = pci_get_ro_map(0);
-
-if ( ro_map && test_bit(machine_bdf, ro_map) )
-return 0;
-}
-start |= CF8_ADDR_LO(currd->arch.pci_cf8);
-/* AMD extended configuration space access? */
-if ( CF8_ADDR_HI(currd->arch.pci_cf8) &&
- boot_cpu_data.x86_vendor == X86_VENDOR_AMD &&
- boot_cpu_data.x86 >= 0x10 && boot_cpu_data.x86 <= 0x17 )
-{
-uint64_t msr_val;
-
-if ( rdmsr_safe(MSR_AMD64_NB_CFG, msr_val) )
-return 0;
-if ( msr_val & (1ULL << AMD64_NB_CFG_CF8_EXT_ENABLE_BIT) )
-start |= CF8_ADDR_HI(currd->arch.pci_cf8);
-}
-
-return !write ?
-   xsm_pci_config_permission(XSM_HOOK, currd, machine_bdf,
- start, start + size - 1, 0) == 0 :
-   pci_conf_write_intercept(0, machine_bdf, start, size, write) >= 0;
-}
-
 uint32_t guest_io_read(unsigned int port, unsigned int bytes,
struct domain *currd)
 {
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 8bce213..e3595a9 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers

[Xen-devel] [PATCH RFC 07/12] xen/x86: parse Dom0 kernel for PVHv2

[Roger Pau Monne]

Introduce a helper to parse the Dom0 kernel.

Signed-off-by: Roger Pau Monné 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/x86/domain_build.c | 139 
 1 file changed, 139 insertions(+)

diff --git a/xen/arch/x86/domain_build.c b/xen/arch/x86/domain_build.c
index cb8ecbd..df6354a 100644
--- a/xen/arch/x86/domain_build.c
+++ b/xen/arch/x86/domain_build.c
@@ -22,6 +22,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -38,6 +39,7 @@
 #include 
 
 #include 
+#include 
 
 static long __initdata dom0_nrpages;
 static long __initdata dom0_min_nrpages;
@@ -1818,12 +1820,141 @@ static int __init hvm_setup_p2m(struct domain *d)
 return 0;
 }
 
+static int __init hvm_load_kernel(struct domain *d, const module_t *image,
+  unsigned long image_headroom,
+  module_t *initrd, char *image_base,
+  char *cmdline, paddr_t *entry,
+  paddr_t *start_info_addr)
+{
+char *image_start = image_base + image_headroom;
+unsigned long image_len = image->mod_end;
+struct elf_binary elf;
+struct elf_dom_parms parms;
+paddr_t last_addr;
+struct hvm_start_info start_info;
+struct hvm_modlist_entry mod;
+struct vcpu *saved_current, *v = d->vcpu[0];
+int rc;
+
+printk("** Parsing Dom0 kernel **\n");
+
+if ( (rc = bzimage_parse(image_base, &image_start, &image_len)) != 0 )
+{
+printk("Error trying to detect bz compressed kernel\n");
+return rc;
+}
+
+if ( (rc = elf_init(&elf, image_start, image_len)) != 0 )
+{
+printk("Unable to init ELF\n");
+return rc;
+}
+#ifdef VERBOSE
+elf_set_verbose(&elf);
+#endif
+elf_parse_binary(&elf);
+if ( (rc = elf_xen_parse(&elf, &parms)) != 0 )
+{
+printk("Unable to parse kernel for ELFNOTES\n");
+return rc;
+}
+
+if ( parms.phys_entry == UNSET_ADDR32 ) {
+printk("Unable to find kernel entry point, aborting\n");
+return -EINVAL;
+}
+
+printk("OS: %s version: %s loader: %s bitness: %s\n", parms.guest_os,
+   parms.guest_ver, parms.loader,
+   elf_64bit(&elf) ? "64-bit" : "32-bit");
+
+printk("** Loading Dom0 kernel **\n");
+/* Copy the OS image and free temporary buffer. */
+elf.dest_base = (void *)(parms.virt_kstart - parms.virt_base);
+elf.dest_size = parms.virt_kend - parms.virt_kstart;
+
+saved_current = current;
+set_current(v);
+
+rc = elf_load_binary(&elf);
+if ( rc < 0 )
+{
+printk("Failed to load kernel: %d\n", rc);
+printk("Xen dom0 kernel broken ELF: %s\n", elf_check_broken(&elf));
+goto out;
+}
+
+last_addr = ROUNDUP(parms.virt_kend - parms.virt_base, PAGE_SIZE);
+printk("** Copying Dom0 modules **\n");
+
+rc = hvm_copy_to_guest_phys(last_addr, mfn_to_virt(initrd->mod_start),
+initrd->mod_end);
+if ( rc != HVMCOPY_okay )
+{
+printk("Unable to copy initrd to guest\n");
+rc = -EFAULT;
+goto out;
+}
+
+mod.paddr = last_addr;
+mod.size = initrd->mod_end;
+last_addr += ROUNDUP(initrd->mod_end, PAGE_SIZE);
+
+/* Free temporary buffers. */
+discard_initial_images();
+
+printk("** Setting up start-of-day info **\n");
+
+memset(&start_info, 0, sizeof(start_info));
+if ( cmdline != NULL )
+{
+rc = hvm_copy_to_guest_phys(last_addr, cmdline, strlen(cmdline) + 1);
+if ( rc != HVMCOPY_okay )
+{
+printk("Unable to copy guest command line\n");
+rc = -EFAULT;
+goto out;
+}
+start_info.cmdline_paddr = last_addr;
+last_addr += ROUNDUP(strlen(cmdline) + 1, 8);
+}
+rc = hvm_copy_to_guest_phys(last_addr, &mod, sizeof(mod));
+if ( rc != HVMCOPY_okay )
+{
+printk("Unable to copy guest modules\n");
+rc = -EFAULT;
+goto out;
+}
+
+start_info.modlist_paddr = last_addr;
+start_info.nr_modules = 1;
+last_addr += sizeof(mod);
+start_info.magic = XEN_HVM_START_MAGIC_VALUE;
+start_info.flags = SIF_PRIVILEGED | SIF_INITDOMAIN;
+rc = hvm_copy_to_guest_phys(last_addr, &start_info, sizeof(start_info));
+if ( rc != HVMCOPY_okay )
+{
+printk("Unable to copy start info to guest\n");
+rc = -EFAULT;
+goto out;
+}
+
+*entry = parms.phys_entry;
+*start_info_addr = last_addr;
+rc = 0;
+
+out:
+set_current(saved_current);
+return rc;
+}
+
 static int __init construct_dom0_hvm(struct domain *d, const module_t *image,
  unsigned long image_headroom,
  module_t *initrd,
  void *(*bootstrap_map)(const module_t *),
  char *cmdline)
 {
+   

[Xen-devel] [PATCH RFC 09/12] xen/x86: setup PVHv2 Dom0 ACPI tables

[Roger Pau Monne]

This maps all the regions in the e820 marked as E820_ACPI or E820_NVS to
Dom0 1:1. It also shadows the page(s) where the native MADT is placed by
mapping a RAM page over it, copying the original data and modifying it
afterwards in order to represent the real CPU topology exposed to Dom0.

Signed-off-by: Roger Pau Monné 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
FWIW, I think that the current approach that I've used in order to craft the
MADT is not the best one, IMHO it would be better to place the MADT at the
end of the E820_ACPI region (expanding it's size one page), and modify the
XSDT/RSDT in order to point to it, that way we avoid shadowing any other
ACPI data that might be at the same page as the native MADT (and that needs
to be modified by Dom0) .
---
 xen/arch/x86/domain_build.c | 250 
 1 file changed, 250 insertions(+)

diff --git a/xen/arch/x86/domain_build.c b/xen/arch/x86/domain_build.c
index 89ef59e..fad4f5c 100644
--- a/xen/arch/x86/domain_build.c
+++ b/xen/arch/x86/domain_build.c
@@ -23,6 +23,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -38,6 +39,8 @@
 #include 
 #include 
 
+#include 
+
 #include 
 #include 
 #include 
@@ -50,6 +53,8 @@ static long __initdata dom0_max_nrpages = LONG_MAX;
 #define HVM_IDENT_PT_GFN  0xfeffeu
 
 static unsigned int __initdata hvm_mem_stats[MAX_ORDER + 1];
+static unsigned int __initdata acpi_intr_overrrides = 0;
+static struct acpi_madt_interrupt_override __initdata *intsrcovr = NULL;
 
 /*
  * dom0_mem=[min:,][max:,][]
@@ -1932,6 +1937,7 @@ static int __init hvm_load_kernel(struct domain *d, const 
module_t *image,
 last_addr += sizeof(mod);
 start_info.magic = XEN_HVM_START_MAGIC_VALUE;
 start_info.flags = SIF_PRIVILEGED | SIF_INITDOMAIN;
+start_info.rsdp_paddr = acpi_os_get_root_pointer();
 rc = hvm_copy_to_guest_phys(last_addr, &start_info, sizeof(start_info));
 if ( rc != HVMCOPY_okay )
 {
@@ -2044,6 +2050,243 @@ static int __init hvm_setup_cpus(struct domain *d, 
paddr_t entry,
 return 0;
 }
 
+static int __init acpi_count_intr_ov(struct acpi_subtable_header *header,
+ const unsigned long end)
+{
+
+acpi_intr_overrrides++;
+return 0;
+}
+
+static int __init acpi_set_intr_ov(struct acpi_subtable_header *header,
+   const unsigned long end)
+{
+struct acpi_madt_interrupt_override *intr =
+container_of(header, struct acpi_madt_interrupt_override, header);
+
+ACPI_MEMCPY(intsrcovr, intr, sizeof(*intr));
+intsrcovr++;
+
+return 0;
+}
+
+static void __init acpi_zap_table_signature(char *name)
+{
+struct acpi_table_header *table;
+acpi_status status;
+union {
+char str[ACPI_NAME_SIZE];
+uint32_t bits;
+} signature;
+char tmp;
+int i;
+
+status = acpi_get_table(name, 0, &table);
+if ( ACPI_SUCCESS(status) )
+{
+memcpy(&signature.str[0], &table->signature[0], ACPI_NAME_SIZE);
+for ( i = 0; i < ACPI_NAME_SIZE / 2; i++ )
+{
+tmp = signature.str[ACPI_NAME_SIZE - i - 1];
+signature.str[ACPI_NAME_SIZE - i - 1] = signature.str[i];
+signature.str[i] = tmp;
+}
+write_atomic((uint32_t*)&table->signature[0], signature.bits);
+}
+}
+
+static int __init acpi_map(struct domain *d, unsigned long pfn,
+   unsigned long nr_pages)
+{
+int rc;
+
+while ( nr_pages > 0 )
+{
+rc = map_mmio_regions(d, _gfn(pfn), nr_pages, _mfn(pfn));
+if ( rc == 0 )
+break;
+if ( rc < 0 )
+{
+printk("Failed to map %#lx - %#lx into Dom0 memory map: %d\n",
+   pfn, pfn + nr_pages, rc);
+return rc;
+}
+nr_pages -= rc;
+pfn += rc;
+process_pending_softirqs();
+}
+
+return rc;
+}
+
+static int __init hvm_setup_acpi(struct domain *d)
+{
+struct vcpu *saved_current, *v = d->vcpu[0];
+unsigned long pfn, nr_pages;
+uint64_t size, start_addr, end_addr;
+uint64_t madt_addr[2] = { 0, 0 };
+struct acpi_table_header *table;
+struct acpi_table_madt *madt;
+struct acpi_madt_io_apic *io_apic;
+struct acpi_madt_local_apic *local_apic;
+acpi_status status;
+int rc, i;
+
+printk("** Setup ACPI tables **\n");
+
+/* ZAP the HPET, SLIT, SRAT, MPST and PMTT tables. */
+acpi_zap_table_signature(ACPI_SIG_HPET);
+acpi_zap_table_signature(ACPI_SIG_SLIT);
+acpi_zap_table_signature(ACPI_SIG_SRAT);
+acpi_zap_table_signature(ACPI_SIG_MPST);
+acpi_zap_table_signature(ACPI_SIG_PMTT);
+
+/* Map ACPI tables 1:1 */
+for ( i = 0; i < d->arch.nr_e820; i++ )
+{
+if ( d->arch.e820[i].type != E820_ACPI &&
+ d->arch.e820[i].type != E820_NVS )
+continue;
+
+pfn = PFN_DOWN(d->arch.e820[i].addr);
+nr_pages = DIV_ROUND_U

[Xen-devel] [PATCH RFC 01/12] x86/paging: introduce paging_set_allocation

[Roger Pau Monne]

... and remove hap_set_alloc_for_pvh_dom0.

Signed-off-by: Roger Pau Monné 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Tim Deegan 
---
 xen/arch/x86/domain_build.c |  7 +++
 xen/arch/x86/mm/hap/hap.c   | 14 +-
 xen/arch/x86/mm/paging.c| 16 
 xen/arch/x86/mm/shadow/common.c |  5 ++---
 xen/include/asm-x86/hap.h   |  3 ++-
 xen/include/asm-x86/paging.h|  3 +++
 xen/include/asm-x86/shadow.h|  6 ++
 7 files changed, 33 insertions(+), 21 deletions(-)

diff --git a/xen/arch/x86/domain_build.c b/xen/arch/x86/domain_build.c
index 0a02d65..d7d4afc 100644
--- a/xen/arch/x86/domain_build.c
+++ b/xen/arch/x86/domain_build.c
@@ -35,7 +35,6 @@
 #include 
 #include  /* for bzimage_parse */
 #include 
-#include 
 #include 
 
 #include 
@@ -1383,15 +1382,15 @@ int __init construct_dom0(
  nr_pages);
 }
 
-if ( is_pvh_domain(d) )
-hap_set_alloc_for_pvh_dom0(d, dom0_paging_pages(d, nr_pages));
-
 /*
  * We enable paging mode again so guest_physmap_add_page will do the
  * right thing for us.
  */
 d->arch.paging.mode = save_pvh_pg_mode;
 
+if ( is_pvh_domain(d) )
+paging_set_allocation(d, dom0_paging_pages(d, nr_pages));
+
 /* Write the phys->machine and machine->phys table entries. */
 for ( pfn = 0; pfn < count; pfn++ )
 {
diff --git a/xen/arch/x86/mm/hap/hap.c b/xen/arch/x86/mm/hap/hap.c
index 3218fa2..b49b38f 100644
--- a/xen/arch/x86/mm/hap/hap.c
+++ b/xen/arch/x86/mm/hap/hap.c
@@ -334,7 +334,7 @@ hap_get_allocation(struct domain *d)
 
 /* Set the pool of pages to the required number of pages.
  * Returns 0 for success, non-zero for failure. */
-static unsigned int
+unsigned int
 hap_set_allocation(struct domain *d, unsigned int pages, int *preempted)
 {
 struct page_info *pg;
@@ -638,18 +638,6 @@ int hap_domctl(struct domain *d, xen_domctl_shadow_op_t 
*sc,
 }
 }
 
-void __init hap_set_alloc_for_pvh_dom0(struct domain *d,
-   unsigned long hap_pages)
-{
-int rc;
-
-paging_lock(d);
-rc = hap_set_allocation(d, hap_pages, NULL);
-paging_unlock(d);
-
-BUG_ON(rc);
-}
-
 static const struct paging_mode hap_paging_real_mode;
 static const struct paging_mode hap_paging_protected_mode;
 static const struct paging_mode hap_paging_pae_mode;
diff --git a/xen/arch/x86/mm/paging.c b/xen/arch/x86/mm/paging.c
index 107fc8c..1b270df 100644
--- a/xen/arch/x86/mm/paging.c
+++ b/xen/arch/x86/mm/paging.c
@@ -953,6 +953,22 @@ void paging_write_p2m_entry(struct p2m_domain *p2m, 
unsigned long gfn,
 safe_write_pte(p, new);
 }
 
+int paging_set_allocation(struct domain *d, unsigned long pages)
+{
+int rc;
+
+ASSERT(paging_mode_enabled(d));
+
+paging_lock(d);
+if ( hap_enabled(d) )
+rc = hap_set_allocation(d, pages, NULL);
+else
+rc = sh_set_allocation(d, pages, NULL);
+paging_unlock(d);
+
+return rc;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index c22362f..452e22e 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -1604,9 +1604,8 @@ shadow_free_p2m_page(struct domain *d, struct page_info 
*pg)
  * Input will be rounded up to at least shadow_min_acceptable_pages(),
  * plus space for the p2m table.
  * Returns 0 for success, non-zero for failure. */
-static unsigned int sh_set_allocation(struct domain *d,
-  unsigned int pages,
-  int *preempted)
+unsigned int sh_set_allocation(struct domain *d, unsigned int pages,
+   int *preempted)
 {
 struct page_info *sp;
 unsigned int lower_bound;
diff --git a/xen/include/asm-x86/hap.h b/xen/include/asm-x86/hap.h
index c613836..e3c9c98 100644
--- a/xen/include/asm-x86/hap.h
+++ b/xen/include/asm-x86/hap.h
@@ -46,7 +46,8 @@ int   hap_track_dirty_vram(struct domain *d,
XEN_GUEST_HANDLE_64(uint8) dirty_bitmap);
 
 extern const struct paging_mode *hap_paging_get_mode(struct vcpu *);
-void hap_set_alloc_for_pvh_dom0(struct domain *d, unsigned long num_pages);
+unsigned int hap_set_allocation(struct domain *d, unsigned int pages,
+   int *preempted);
 
 #endif /* XEN_HAP_H */
 
diff --git a/xen/include/asm-x86/paging.h b/xen/include/asm-x86/paging.h
index a1401ab..6598007 100644
--- a/xen/include/asm-x86/paging.h
+++ b/xen/include/asm-x86/paging.h
@@ -347,6 +347,9 @@ void pagetable_dying(struct domain *d, paddr_t gpa);
 void paging_dump_domain_info(struct domain *d);
 void paging_dump_vcpu_info(struct vcpu *v);
 
+/* Set pool of pages for paging. */
+int paging_set_allocation(struct domain *d, unsigned long pages);
+
 #endif /* XEN_PAGING_H */
 
 /*
diff --git a/xen/include/asm-x86/shadow.h b/xen/include/asm-x86/shadow.h
index 6d0aefb..bc068ec 100644
--- a/xen

[Xen-devel] [PATCH RFC 04/12] xen/x86: split Dom0 build into PV and PVHv2

[Roger Pau Monne]

Split the Dom0 builder into two different functions, one for PV (and classic
PVH), and another one for PVHv2. Introduce a new command line parameter,
dom0hvm in order to request the creation of a PVHv2 Dom0.

Signed-off-by: Roger Pau Monné 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/x86/domain_build.c | 27 ++-
 xen/arch/x86/setup.c|  9 +
 2 files changed, 35 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/domain_build.c b/xen/arch/x86/domain_build.c
index 09d79be..c0ef40f 100644
--- a/xen/arch/x86/domain_build.c
+++ b/xen/arch/x86/domain_build.c
@@ -952,7 +952,7 @@ static int __init setup_permissions(struct domain *d)
 return rc;
 }
 
-int __init construct_dom0(
+static int __init construct_dom0_pv(
 struct domain *d,
 const module_t *image, unsigned long image_headroom,
 module_t *initrd,
@@ -1647,6 +1647,31 @@ out:
 return rc;
 }
 
+static int __init construct_dom0_hvm(struct domain *d, const module_t *image,
+ unsigned long image_headroom,
+ module_t *initrd,
+ void *(*bootstrap_map)(const module_t *),
+ char *cmdline)
+{
+
+printk("** Building a PVH Dom0 **\n");
+
+return 0;
+}
+
+int __init construct_dom0(struct domain *d, const module_t *image,
+  unsigned long image_headroom, module_t *initrd,
+  void *(*bootstrap_map)(const module_t *),
+  char *cmdline)
+{
+
+return is_hvm_domain(d) ?
+construct_dom0_hvm(d, image, image_headroom, initrd,
+   bootstrap_map, cmdline) :
+construct_dom0_pv(d, image, image_headroom, initrd, bootstrap_map,
+  cmdline);
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 217c775..8d9c3a0 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -75,6 +75,10 @@ unsigned long __read_mostly cr4_pv32_mask;
 static bool_t __initdata opt_dom0pvh;
 boolean_param("dom0pvh", opt_dom0pvh);
 
+/* Boot dom0 in HVM mode */
+static bool_t __initdata opt_dom0hvm;
+boolean_param("dom0hvm", opt_dom0hvm);
+
 /*  Linux config option: propagated to domain0. */
 /* "acpi=off":Sisables both ACPI table parsing and interpreter. */
 /* "acpi=force":  Override the disable blacklist.   */
@@ -1492,6 +1496,11 @@ void __init noreturn __start_xen(unsigned long mbi_p)
 if ( opt_dom0pvh )
 domcr_flags |= DOMCRF_pvh | DOMCRF_hap;
 
+if ( opt_dom0hvm ) {
+domcr_flags |= DOMCRF_hvm | (hvm_funcs.hap_supported ? DOMCRF_hap : 0);
+config.emulation_flags = XEN_X86_EMU_LAPIC|XEN_X86_EMU_IOAPIC;
+}
+
 /*
  * Create initial domain 0.
  * x86 doesn't support arch-configuration. So it's fine to pass
-- 
2.7.4 (Apple Git-66)


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 05/12] xen/x86: make print_e820_memory_map global

[Roger Pau Monne]

So that it can be called from the Dom0 builder.

Signed-off-by: Roger Pau Monné 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/x86/e820.c| 2 +-
 xen/include/asm-x86/e820.h | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/e820.c b/xen/arch/x86/e820.c
index ef077a5..48e35f9 100644
--- a/xen/arch/x86/e820.c
+++ b/xen/arch/x86/e820.c
@@ -87,7 +87,7 @@ static void __init add_memory_region(unsigned long long start,
 e820.nr_map++;
 }
 
-static void __init print_e820_memory_map(struct e820entry *map, unsigned int 
entries)
+void __init print_e820_memory_map(struct e820entry *map, unsigned int entries)
 {
 unsigned int i;
 
diff --git a/xen/include/asm-x86/e820.h b/xen/include/asm-x86/e820.h
index d9ff4eb..9dad76a 100644
--- a/xen/include/asm-x86/e820.h
+++ b/xen/include/asm-x86/e820.h
@@ -31,6 +31,7 @@ extern int e820_change_range_type(
 extern int e820_add_range(
 struct e820map *, uint64_t s, uint64_t e, uint32_t type);
 extern unsigned long init_e820(const char *, struct e820entry *, unsigned int 
*);
+extern void print_e820_memory_map(struct e820entry *map, unsigned int entries);
 extern struct e820map e820;
 
 /* These symbols live in the boot trampoline. */
-- 
2.7.4 (Apple Git-66)


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 01/12] PVHv2 Dom0

[Roger Pau Monne]

Hello,

This is a very rough implementation of a PVHv2 Dom0. There are still a bunch 
of things that will not work properly (like SR-IOV, MSI, MSI-X...), but it 
*should* be able to boot a Dom0 kernel and it doesn't require using PIRQs.

There are some changes compared to a traditional PV or PVH Dom0:

 - An emulated local APIC (for each vCPU) and IO APIC is provided to Dom0.
 - At the start of day only the low 1MB and the ACPI e820 regions are mapped 
   into the guest using 1:1 mappings (for UEFI systems mapping the low 1MB 
   probably doesn't make any sense, but alas).
 - The MADT has been replaced in order to reflect the real topology seen by 
   Dom0.
 - In order to have the BARs of PCI devices mapped Dom0 must call 
   PHYSDEVOP_pci_device_add. See the notes on patch 11 for more information 
   (and what's missing).
 - ATM only legacy PCI interrupts are supported. This is implemented by 
   looking at the writes Dom0 makes to the emulated IO APIC and translating 
   them into the real hardware. Note that MSI or MSI-X capabilities are 
   _not_ masked from the PCI capabilities list, so the user must avoid using
   them.
 - PCIe Enhanced Configuration Mechanism regions are not yet mapped into 
   Dom0 physmap.
 - Some ACPI tables are zapped (it's signature is inverted) to prevent Dom0 
   from poking at them, those are: HPET, SLIT, SRAT, MPST and PMTT.

This is still very experimental, I've been able to boot a FreeBSD Dom0 using 
2GB and 4GB of memory assigned to it, but if I try to use 6GB Xen gets a NMI 
(I've got no idea why yet). I don't think it's worth delaying this more, 
because it's going to take me some time to finish all this work (MSI, 
MSI-X, bug hunting...), and in the meantime people can already take a look 
at what's done (or half-done).

I've pushed the whole series to my git repository:

git://xenbits.xen.org/people/royger/xen.git pvhv2_dom0_rfc

It contains two patches from Boris and Anthony, that are a pre-requisite to 
this series.

As usual, thanks for taking the time to look into it, hope it doesn't make 
your eyes bleed much (slight bleeding is expected).

Roger.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH RFC 06/12] xen/x86: populate PVHv2 Dom0 physical memory map

[Roger Pau Monne]

Craft the Dom0 e820 memory map and populate it.

Signed-off-by: Roger Pau Monné 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/x86/domain_build.c | 199 ++--
 1 file changed, 193 insertions(+), 6 deletions(-)

diff --git a/xen/arch/x86/domain_build.c b/xen/arch/x86/domain_build.c
index c0ef40f..cb8ecbd 100644
--- a/xen/arch/x86/domain_build.c
+++ b/xen/arch/x86/domain_build.c
@@ -43,6 +43,11 @@ static long __initdata dom0_nrpages;
 static long __initdata dom0_min_nrpages;
 static long __initdata dom0_max_nrpages = LONG_MAX;
 
+/* GFN of the identity map for EPT. */
+#define HVM_IDENT_PT_GFN  0xfeffeu
+
+static unsigned int __initdata hvm_mem_stats[MAX_ORDER + 1];
+
 /*
  * dom0_mem=[min:,][max:,][]
  * 
@@ -304,7 +309,8 @@ static unsigned long __init compute_dom0_nr_pages(
 avail -= max_pdx >> s;
 }
 
-need_paging = opt_dom0_shadow || (is_pvh_domain(d) && !iommu_hap_pt_share);
+need_paging = opt_dom0_shadow || (has_hvm_container_domain(d) &&
+  (!iommu_hap_pt_share || !paging_mode_hap(d)));
 for ( ; ; need_paging = 0 )
 {
 nr_pages = dom0_nrpages;
@@ -336,7 +342,8 @@ static unsigned long __init compute_dom0_nr_pages(
 avail -= dom0_paging_pages(d, nr_pages);
 }
 
-if ( (parms->p2m_base == UNSET_ADDR) && (dom0_nrpages <= 0) &&
+if ( is_pv_domain(d) &&
+ (parms->p2m_base == UNSET_ADDR) && (dom0_nrpages <= 0) &&
  ((dom0_min_nrpages <= 0) || (nr_pages > min_pages)) )
 {
 /*
@@ -547,11 +554,12 @@ static __init void pvh_map_all_iomem(struct domain *d, 
unsigned long nr_pages)
 ASSERT(nr_holes == 0);
 }
 
-static __init void pvh_setup_e820(struct domain *d, unsigned long nr_pages)
+static __init void hvm_setup_e820(struct domain *d, unsigned long nr_pages)
 {
 struct e820entry *entry, *entry_guest;
 unsigned int i;
 unsigned long pages, cur_pages = 0;
+uint64_t start, end;
 
 /*
  * Craft the e820 memory map for Dom0 based on the hardware e820 map.
@@ -579,8 +587,19 @@ static __init void pvh_setup_e820(struct domain *d, 
unsigned long nr_pages)
 continue;
 }
 
-*entry_guest = *entry;
-pages = PFN_UP(entry_guest->size);
+/*
+ * Make sure the start and length are aligned to PAGE_SIZE, because
+ * that's the minimum granularity of the 2nd stage translation.
+ */
+start = ROUNDUP(entry->addr, PAGE_SIZE);
+end = (entry->addr + entry->size) & PAGE_MASK;
+if ( start >= end )
+continue;
+
+entry_guest->type = E820_RAM;
+entry_guest->addr = start;
+entry_guest->size = end - start;
+pages = PFN_DOWN(entry_guest->size);
 if ( (cur_pages + pages) > nr_pages )
 {
 /* Truncate region */
@@ -591,6 +610,8 @@ static __init void pvh_setup_e820(struct domain *d, 
unsigned long nr_pages)
 {
 cur_pages += pages;
 }
+ASSERT((entry_guest->addr & ~PAGE_MASK) == 0 &&
+   (entry_guest->size & ~PAGE_MASK) == 0);
  next:
 d->arch.nr_e820++;
 entry_guest++;
@@ -1631,7 +1652,7 @@ static int __init construct_dom0_pv(
 dom0_update_physmap(d, pfn, mfn, 0);
 
 pvh_map_all_iomem(d, nr_pages);
-pvh_setup_e820(d, nr_pages);
+hvm_setup_e820(d, nr_pages);
 }
 
 if ( d->domain_id == hardware_domid )
@@ -1647,15 +1668,181 @@ out:
 return rc;
 }
 
+/* Helper to convert from bytes into human-readable form. */
+static void __init pretty_print_bytes(uint64_t size)
+{
+const char* units[] = {"B", "KB", "MB", "GB", "TB"};
+int i = 0;
+
+while ( ++i < sizeof(units) && size >= 1024 )
+size >>= 10; /* size /= 1024 */
+
+printk("%4" PRIu64 "%2s", size, units[i-1]);
+}
+
+/* Calculate the biggest usable order given a size in bytes. */
+static inline unsigned int get_order(uint64_t size)
+{
+unsigned int order;
+uint64_t pg;
+
+ASSERT((size & ~PAGE_MASK) == 0);
+pg = PFN_DOWN(size);
+for ( order = 0; pg >= (1 << (order + 1)); order++ );
+
+return order;
+}
+
+/* Populate an HVM memory range using the biggest possible order. */
+static void __init hvm_populate_memory_range(struct domain *d, uint64_t start,
+ uint64_t size)
+{
+static unsigned int __initdata memflags = MEMF_no_dma|MEMF_exact_node;
+unsigned int order;
+struct page_info *page;
+int rc;
+
+ASSERT((size & ~PAGE_MASK) == 0 && (start & ~PAGE_MASK) == 0);
+
+order = MAX_ORDER;
+while ( size != 0 )
+{
+order = min(get_order(size), order);
+page = alloc_domheap_pages(d, order, memflags);
+if ( page == NULL )
+{
+if ( order == 0 && memflags )
+{
+/* Try again without any memflags. */
+memflags = 0;
+order = MAX_ORDER;
+continu

Re: [Xen-devel] [PATCH] mem_access: Use monitor_traps instead of mem_access_send_req

[Andrew Cooper]

On 29/07/16 15:21, Tamas K Lengyel wrote:
>
> On Jul 29, 2016 02:50, "Julien Grall"  > wrote:
> >
> >
> >
> > On 28/07/16 23:54, Tamas K Lengyel wrote:
> >>
> >> On Thu, Jul 28, 2016 at 2:38 PM, Julien Grall  > wrote:
> >>>
> >>> On 28/07/2016 20:35, Tamas K Lengyel wrote:
> >>> This patch is doing more than it is claimed in the commit message.
> >>>
> >>> In general, moving the code and introducing changes within the
> same patch
> >>> should really be avoided. So please split it in 2 patches.
> >>
> >>
> >> Well, the changes are largely cosmetic so doing a whole separate patch
> >> IMHO is an overkill. How about adjusting the commit message to
> >> something like "sanitize code surrounding sending mem_access
> >> vm_events" to better describe the adjustments made in this patch?
> >
> >
> > I think the wiki page "Submitting Xen Project patches" [1] should
> answer to your question.
> >
> > If not, trivial patches are easy to review, merging multiple trivial
> patches in a single patch is not. Moving code and at the same time as
> changing the behavior is fairly difficult to review because it hides
> the real modifications.
> >
> > Regards,
> >
> > [1]
> http://wiki.xenproject.org/wiki/Submitting_Xen_Project_Patches#Break_down_your_patches
> >
>
> The behavior didn't change at all, this whole patch is code
> sanitization. It's not worth doing a separate patch for each minor
> change. The few change on the arm side is the vm_event request
> allocation going from xzalloc to stack based and using monitor_traps
> now in a split-out function. It really should be no problem reviewing
> it. Even Andrew requested minor adjustments to be included in this
> patch. Anyway, I'm not looking to change this into a series. If it's a
> no go from your side I'm just going to cut down the ARM side
> sanitization to the bare minimum of using monitor_traps as the rest
> just does not worth the effort.
>

To offer an alternative opinion.

Looking at this patch, I personally would have a hard time justifying
breaking it down any further.  Each of the changes are closely related.

However, the commit message could be a lot clearer about which steps are
taken.  If I were writing the commit message, I would go with something
a bit more like this:

8<
The two functions monitor_traps and mem_access_send_req duplicate some
of the same functionality. The mem_access_send_req however leaves a lot
of the standard vm_event fields to be filled by other functions.

Remove mem_access_send_req() completely, making use of monitor_traps()
to put requests into the monitor ring.  This in turn causes some cleanup
around the old callsites of mem_access_send_req(), and on ARM, the
introduction of the __p2m_mem_access_send_req() helper to fill in common
mem_access information.

Finally, this change identifies that errors from mem_access_send_req()
were never checked.  As errors constitute a problem with the monitor
ring, crashing the domain is the most appropriate action to take.
8<

If in doubt, always spell out each of the changes, and their logical
relationships.  If nothing else, it helps people trying to review the patch.

~Andrew
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XTF PATCH] xtf-runner: fix two synchronisation issues

[Ian Jackson]

The three of us had an IRL conversation.  Here is what I think we
agreed:

* We intend to make the XTF runner capable of reading
  xenconsoled-created logfiles.  Both XenRT and osstest configure
  xenconsoled appropriately.

  The xtf runner will need to

 - on each test, wait for the test domain to shutdown, and then

 - look backwards through the xenconsoled logfile for the
   indication that the domain started (eg, a banner printed by the
   domain, or message from xenconsoled), and parse the relevant
   output there.  (This is needed because starting the same-named
   domain multiple times results in concatenations of the console
   output in the xenconsoled guest output logfile.)

 - arrange for the guest to be preserved on crash (at least)
   so that if the domain crashed, we don't risk parsing the
   output from a previous run.  Instead we can see it having
   crashed and report that as a failure.

* We intend to make `xl create -c' work to find all of the output,
  by (i) starting the domain paused (ii) spawning xenconsoled
  (iii) awaiting a new startup indication from xenconsoled
  (iv) unpausing the domain.  This will be done in xen-unstable.

  I propose the following startup protocol: xl runs
 xenconsole --startup-notify-fd=FD
  where FD is the writing end of a pipe.  xl waits for xenconsole to
  write the byte 0x00 to the FD.  If xenconsoled crashes, xl can tell
  by the EOF on the pipe.  (This approach saves xl from having to try
  to wait for either SIGCHLD or fd readability.)

  (The systemd startup notification protocol is too complex to
  reimplement and would therefore introduce a dependency on
  libsystemd's sd_notify, which would be awkward.  There is also the
  upstart SIGSTOP protocol but it could interact badly with an
  interactive user who uses ^Z.)

  The xtf runner would also be able to use `xl create -c' and simply
  expect to see all the console output.

  (We also discussed making xenconsole print something to its stdout
  or stderr when it has successfully connected, and when it
  disconnects.  While we're editing xenconsole it would probably be
  nice to do this, but with our plans it's not needed for XTF.)

As a result, the xtf runner can be used with a default install of
xen-unstable.  For older versions of Xen it will be necessary to
reconfigure xenconsoled, if the user wants to get reliable pass/fail
reports from the xtf runner.

* We discussed changing xenconsoled to not tear down the guest console
  until the guest is destroyed, rather than already tearing it down
  when the guest is shut down.  This is not now needed for the above,
  but I still think it would be nice.  However it is done, it should
  arrange that `xl console' doesn't hang waiting for further output
  from a crashed or shutdown domain (but perhaps should wait for
  output from a rebooting domain?).  It would probably be a good idea
  to put this work item in a bucket with `overhaul the console stuff'.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC 00/22] xen/arm: Rework the P2M code to follow break-before-make sequence

[Julien Grall]

On 28/07/16 18:46, Tamas K Lengyel wrote:
> Hi Julien,

Hello,

>> I sent this patch series as an RFC because there are still some TODOs
>> in the code (mostly sanity check and possible optimization) and I have
>> done limited testing. However, I think it is a good shape to start reviewing,
>> get more feedback and have wider testing on different board.
> 
> I've tested this series on my Cubietruck but when I try to enable
> xen-access on a domain I get the following errors:
> 
> ~/xen/tools/tests/xen-access# ./xen-access 1 write
> xenaccess init
> max_gpfn = 48000
> starting write 1
> (XEN) traps.c:2569:d1v0 HSR=0x904f pc=0xc029eb10 gva=0xc0e013c0
> gpa=0x0040e013c0
> Error -1 setting all memory to access type 5
> 
> The same thing works fine on the latest staging build, so this series
> introduces some regression along the way.

Well, memaccess is not working on staging. As soon as it is enabled, the console
is flooded with:

traps.c:2503:d1v0 HSR=0x924f pc=0xff8008579b90 gva=0xffc0011bf000 
gpa=0x00411bf000

And the guest is crashing soon after because it received a data abort.

Regards,

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On Fri, Jul 29, 2016, at 09:03 AM, Konrad Rzeszutek Wilk wrote:
> It may very well be added.

Just fyi, it's not in here.  Yet.

> But having extra test-confirmation is always good.

Right, and I'm glad to do that.  I'd like to.  Goal is to keep moving the ball 
forward.

And I've been testing what I've been asked to when I can do it with packages 
that I have and that work.

I just haven't figured out yet how to use their 'build system' to build a 
kernel and get it working to boot.

I built a kernel the 'upstream' way but that won't boot Xen at all.  Sure it's 
something I'm not doing right, I know.

So

(1) I'll keep trying to DIY and get something running
(2) As soon as the distro patches their packages with the patches you all say 
it should have, then I'll test them and report back

IF there's a different combination of different OS + packages for a Kernel 
version + Xen version that is known to work with UEFI Dom0 & UEFI Guests then 
I'm all ears.

A friend's starting to test KVM with QEMU + OVMF/UEFI on another box, and so 
far so good I guess.  I don't want to use it for a bunch of reasons, so I'm 
sticking it out in here.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[Konrad Rzeszutek Wilk]

On Fri, Jul 29, 2016 at 08:57:59AM -0700, li...@ssl-mail.com wrote:
> 
> 
> On Fri, Jul 29, 2016, at 08:42 AM, Konrad Rzeszutek Wilk wrote:
> > did you apply the patch that Vitaly pointed out?
> 
> No.  It wasn't clear that it was anything more than a question to 
> "double-check".   There wasn't any further comment on my reply.
> 
> I'm depending on working packages for now.  Like I said earlier building my 
> own packages is something I haven't gotten to work to the point that I can 
> boot up without causing other problems yet.
> 
> I'm trying to get to that point, but not there today.
> 
> If that's a 'must fix this' patch, then I don't quite get why it doesn't just 
> get added to the distro packages.  The devs from the distro are obviously 
> here.

It may very well be added.

But having extra test-confirmation is always good.




___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On Fri, Jul 29, 2016, at 08:42 AM, Konrad Rzeszutek Wilk wrote:
> did you apply the patch that Vitaly pointed out?

No.  It wasn't clear that it was anything more than a question to 
"double-check".   There wasn't any further comment on my reply.

I'm depending on working packages for now.  Like I said earlier building my own 
packages is something I haven't gotten to work to the point that I can boot up 
without causing other problems yet.

I'm trying to get to that point, but not there today.

If that's a 'must fix this' patch, then I don't quite get why it doesn't just 
get added to the distro packages.  The devs from the distro are obviously here.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] OVMF very slow on AMD

[Anthony PERARD]

On Thu, Jul 28, 2016 at 03:54:34PM -0400, Boris Ostrovsky wrote:
> On 07/28/2016 03:44 PM, Andrew Cooper wrote:
>  As far as Intel vs AMD implementation in Xen, we have vmx_handle_cd()
>  but no corresponding SVM code. Could it be that we need to set gPAT, for
>  example?
> >>> A better approach would be to find out why ovmf insists on disabling
> >>> caches at all.  Even if we optimise the non-PCI-device case in the
> >>> hypervisor, a passthrough case will still run like treacle if caches are
> >>> disabled.
> >> True, we should understand why OVMF does this. But I think we also need
> >> to understand what makes Intel run faster. Or is it already clear from
> >> vmx_handle_cd()?
> > Wow this code is hard to follow :(
> >
> > handle_cd() is only called when an IOMMU is enabled and the domain in
> > question has access to real ioports or PCI devices.
> >
> > However, I really can't spot anything that ends up eliding the
> > cache-disable setting even for Intel.  This clearly needs further
> > investigation.
> 
> So as an easy start perhaps Anthony could check whether this call is
> made with his guest running on Intel.

No, handle_cd is never called on my guest.

-- 
Anthony PERARD

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[Konrad Rzeszutek Wilk]

On Fri, Jul 29, 2016 at 07:36:57AM -0700, li...@ssl-mail.com wrote:
> On Wed, Jul 27, 2016, at 09:34 AM, Andrew Cooper wrote:
> > This looks suspiciously like the issue which was fixed by c/s
> > d6b186c1e2d852a92c43f090d0d8fad4704d51ef "x86/xen: avoid m2p lookup when
> > setting early page table entries", but that fix is present in Linux 4.7.0
> > 
> > Can you check to see whether the commit is included in your kernel?
> > 
> > Failing that, can you find out exactly where the kernel crashed?  You
> > need to manually decode 81f6374c with the debug symbols.
> > 
> > ~Andrew
> 
> Is an eventual fix to whatever is causing this crash likely in the kernel or 
> Xen code?
> 
> I've managed to try a couple of different kernels, older version, with no 
> luck.  Crash still happens.
> 
> Getting my hands on an older Xen package is turning out to be a bit tougher.  
> Still working on it.
> 
> Not sure what specifically changed to break this , since this server clearly 
> was running not that long ago.

did you apply the patch that Vitaly pointed out?
> 
> I'd like to at least figure out what I can drop back to so I can get my 
> server and guests back up and running.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XTF PATCH] xtf-runner: fix two synchronisation issues

[Andrew Cooper]

On 29/07/16 15:31, Ian Jackson wrote:
> Wei Liu writes ("Re: [XTF PATCH] xtf-runner: fix two synchronisation issues"):
>> On Fri, Jul 29, 2016 at 01:43:42PM +0100, Andrew Cooper wrote:
>>> The runner existing before xl has torn down the guest is very
>>> deliberate, because some part of hvm guests is terribly slow to tear
>>> down; waiting synchronously for teardown tripled the wallclock time to
>>> run a load of tests back-to-back.
>> Then you won't know if a guest is leaked or it is being slowly destroyed
>> when a dead guest shows up in the snapshot of 'xl list'.
>>
>> Also consider that would make back-to-back tests that happen to have a
>> guest that has the same name as the one in previous test fail.
>>
>> I don't think getting blocked for a few more seconds is a big issue.
>> It's is important to eliminate such race conditions so that osstest can
>> work properly.
> IMO the biggest reason for waiting for teardown is that that will make
> it possible to accurately identify the xtf test which was responsible
> for the failure if a test reveals a bug which causes problems for the
> whole host.

That is perfectly reasonable.

>
> Suppose there is a test T1 which, in buggy hypervisors, creates an
> anomalous data structure, such that the hypervisor crashes when T1's
> guest is finally torn down.
>
> If we start to run the next test T2 immediately we see success output
> from T1, we will observe the host crashing "due to T2", and T1 would
> be regarded as having succeeded.
>
> This is why in an in-person conversation with Wei yesterday I
> recommended that osstest should after each xtf test (i) wait for
> everything to be torn down and (ii) then check that the dom0 is still
> up.  (And these two activities are regarded as part of the preceding
> test step.)

That is also my understanding of how the intended OSSTest integration is
going to work.

OSSTest asks `./xtf-runner --list` for all tests, then iterates over all
tests, running them one at a time, with suitable liveness checks
inbetween.  This is not using xtf-runner's ability to run multiple tests
back to back.


The dev usecase on the other hand is for something like, for checking a
test case refactoring or new bit of functionality.

$ ./xtf-runner sefltest

Combined test results:
test-pv64-selftest   SUCCESS
test-pv32pae-selftestSUCCESS
test-hvm64-selftest  SUCCESS
test-hvm32pae-selftest   SUCCESS
test-hvm32pse-selftest   SUCCESS
test-hvm32-selftest  SUCCESS


FWIW, I have just put a synchronous wait in to demonstrate.

Without wait:

$ time ./xtf-runner sefltest


real0m0.571s
user0m0.060s
sys0m0.228s

With wait:
$ time ./xtf-runner sefltest


real0m8.870s
user0m0.048s
sys0m0.280s


That is more than 8 wallclock seconds elapsed where nothing useful is
happening from the point of view of a human using ./xtf-runner.  All of
this time is spent between @releaseDomain and `xl create -F` finally
exiting.

>
> If this leads to over-consumption of machine resources because this
> serialisation is too slow then the right approach would be explicit
> parallelisation in osstest.  That would still mean that in the
> scenario above, T1 would be regarded as having failed, because T1
> wouldn't be regarded as having passed until osstest had seen that all
> of T1's cleanup had been done and the host was still up.  (T2 would
> _also_ be regarded as failed, and that might look like a heisenbug,
> but that would be tolerable.)

OSSTest shouldn't run multiple tests at once, and I have taken exactly
the same decision for XenRT.  Easy identification of what went bang is
the most important properly in these cases.

We are going to have to get to a vast test library before the wallclock
time of XTF tests approaches anything similar to installing a VM from
scratch.  I am not worried at the moment.

>
> Wei: I need to check what happens with multiple failing test steps in
> the same job.  Specifically, I need to check which one the bisector
> is likely to try to attack.

For individual XTF tests, it is entirely possible that every failure is
from a different change, so should be treated individually.

Having said that, it is also quite likely that, given a lot of similar
microckernels, one hypervisor bug would take a large number out at once,
and we really don't want to bisect each individual XTF test.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC 13/22] xen/arm: p2m: Replace all usage of __p2m_lookup with p2m_get_entry

[Julien Grall]

On 28/07/16 18:36, Tamas K Lengyel wrote:

On Thu, Jul 28, 2016 at 11:29 AM, Tamas K Lengyel  wrote:

On Thu, Jul 28, 2016 at 8:51 AM, Julien Grall  wrote:

__p2m_lookup is just a wrapper to p2m_get_entry.

Signed-off-by: Julien Grall 
Cc: Razvan Cojocaru 
Cc: Tamas K Lengyel 

---
It might be possible to rework the memaccess code to take advantage
of all the parameters. I will defer this to the memaccess folks.


Could you elaborate on what you mean?



Never mind, I see it. Yes, doing __p2m_get_mem_access and then
p2m_get_entry later duplicates work. I would suggest just replacing
__p2m_get_mem_access with a single call to p2m_get_entry to get both
the type and the mem_access setting on the page in a single run.


I am not planning to clean-up the memaccess code. Feel free to send a 
follow-up patch for that.


Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v7 00/15] Load BIOS via toolstack instead of been embedded in hvmloader.

[Boris Ostrovsky]

On 07/29/2016 10:50 AM, Wei Liu wrote:
>
>>> You need to run ./autogen.sh. Anthony didn't commit the changes to
>>> ./configure.
>>
>> Yes, that did the trick. Except for the machine on which I actually
>> wanted it to run:
>>
>> root@ovs104> ./autogen.sh
>> configure.ac:4: error: Autoconf version 2.67 or higher is required
>> configure.ac:4: the top level
>> autom4te: /usr/bin/m4 failed with exit status: 63
>> root@ovs104> autoconf --version
>> autoconf (GNU Autoconf) 2.63
>> Copyright (C) 2008 Free Software Foundation, Inc.
>> License GPLv2+: GNU GPL version 2 or later
>> 
>> This is free software: you are free to change and redistribute it.
>> There is NO WARRANTY, to the extent permitted by law.
>>
>> Written by David J. MacKenzie and Akim Demaille.
>> root@ovs104>
>>
>> Is 2.67 really a pre-req?
>>
> It is not written down. But I guess there is a reason for it.
>
> Do you need me or Anthony to generate that for you?

No, I am good --- I can build on a system with software from this decade.

-boris


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v7 00/15] Load BIOS via toolstack instead of been embedded in hvmloader.

[Wei Liu]

On Fri, Jul 29, 2016 at 10:36:02AM -0400, Boris Ostrovsky wrote:
> On 07/29/2016 04:29 AM, Wei Liu wrote:
> > On Fri, Jul 29, 2016 at 01:28:02AM -0400, Boris Ostrovsky wrote:
> >>
> >> On 07/28/2016 06:49 AM, Anthony PERARD wrote:
> >>> Hi all,
> >>>
> >>> Changes in V7:
> >>>   - There is one new patch at the end to fix the doc.
> >>>   - Patch 6 as been change.
> >>>   that's it.
> >>>
> >>>   There is just a few missing ackes:
> >>> 6 xen: Move the hvm_start_info C representation from libxc to 
> >>> public/xen.h
> >>> 8 hvmloader: Locate the BIOS blob
> >>> 9 hvmloader: Check modules whereabouts in perform_tests
> >>>15 docs/misc/hvmlite: Point to the canonical definition of 
> >>> hvm_start_info
> >>>
> >>> Thanks.
> >>>
> >>> A git tree can be found here:
> >>> git://xenbits.xen.org/people/aperard/xen-unstable.git
> >>> tag: hvmloader-with-separated-bios-v7
> >>
> >> I am unable to build this:
> >>
> >> libxl_paths.c: In function ‘libxl__seabios_path’:
> >> libxl_paths.c:40: error: ‘SEABIOS_PATH’ undeclared (first use in this
> >> function)
> >> libxl_paths.c:40: error: (Each undeclared identifier is reported only once
> >> libxl_paths.c:40: error: for each function it appears in.)
> >> libxl_paths.c: In function ‘libxl__ovmf_path’:
> >> libxl_paths.c:45: error: ‘OVMF_PATH’ undeclared (first use in this 
> >> function)
> >>
> >> IIUIC these two are supposed to be generated into tools/config.h but they
> >> were not for me. I haven't looked any further yet.
> >>
> >> -boris
> >>
> > You need to run ./autogen.sh. Anthony didn't commit the changes to
> > ./configure.
> 
> 
> Yes, that did the trick. Except for the machine on which I actually
> wanted it to run:
> 
> root@ovs104> ./autogen.sh
> configure.ac:4: error: Autoconf version 2.67 or higher is required
> configure.ac:4: the top level
> autom4te: /usr/bin/m4 failed with exit status: 63
> root@ovs104> autoconf --version
> autoconf (GNU Autoconf) 2.63
> Copyright (C) 2008 Free Software Foundation, Inc.
> License GPLv2+: GNU GPL version 2 or later
> 
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
> 
> Written by David J. MacKenzie and Akim Demaille.
> root@ovs104>
> 
> Is 2.67 really a pre-req?
> 

It is not written down. But I guess there is a reason for it.

Do you need me or Anthony to generate that for you?

Wei.

> -boris
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XTF PATCH] xtf-runner: fix two synchronisation issues

[Wei Liu]

On Fri, Jul 29, 2016 at 03:31:30PM +0100, Ian Jackson wrote:
> Wei Liu writes ("Re: [XTF PATCH] xtf-runner: fix two synchronisation issues"):
> > On Fri, Jul 29, 2016 at 01:43:42PM +0100, Andrew Cooper wrote:
> > > The runner existing before xl has torn down the guest is very
> > > deliberate, because some part of hvm guests is terribly slow to tear
> > > down; waiting synchronously for teardown tripled the wallclock time to
> > > run a load of tests back-to-back.
> > 
> > Then you won't know if a guest is leaked or it is being slowly destroyed
> > when a dead guest shows up in the snapshot of 'xl list'.
> > 
> > Also consider that would make back-to-back tests that happen to have a
> > guest that has the same name as the one in previous test fail.
> > 
> > I don't think getting blocked for a few more seconds is a big issue.
> > It's is important to eliminate such race conditions so that osstest can
> > work properly.
> 
> IMO the biggest reason for waiting for teardown is that that will make
> it possible to accurately identify the xtf test which was responsible
> for the failure if a test reveals a bug which causes problems for the
> whole host.
> 
> Suppose there is a test T1 which, in buggy hypervisors, creates an
> anomalous data structure, such that the hypervisor crashes when T1's
> guest is finally torn down.
> 
> If we start to run the next test T2 immediately we see success output
> from T1, we will observe the host crashing "due to T2", and T1 would
> be regarded as having succeeded.
> 
> This is why in an in-person conversation with Wei yesterday I
> recommended that osstest should after each xtf test (i) wait for
> everything to be torn down and (ii) then check that the dom0 is still
> up.  (And these two activities are regarded as part of the preceding
> test step.)
> 
> If this leads to over-consumption of machine resources because this
> serialisation is too slow then the right approach would be explicit
> parallelisation in osstest.  That would still mean that in the
> scenario above, T1 would be regarded as having failed, because T1
> wouldn't be regarded as having passed until osstest had seen that all
> of T1's cleanup had been done and the host was still up.  (T2 would
> _also_ be regarded as failed, and that might look like a heisenbug,
> but that would be tolerable.)
> 
> Wei: I need to check what happens with multiple failing test steps in
> the same job.  Specifically, I need to check which one the bisector
> is likely to try to attack.
> 

Yes. I think my current code can meet both you and Andrew's
requirement.

1. The runner waits for all tests to finish, which amortise the clean up
   time. This is what Andrew needs.
2. In osstest, we run one test case at a time. So "all tests" is only
   one test. This is what you need.

Wei.

> Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XTF PATCH] xtf-runner: fix two synchronisation issues

[Wei Liu]

On Fri, Jul 29, 2016 at 03:35:06PM +0100, Ian Jackson wrote:
> Wei Liu writes ("Re: [XTF PATCH] xtf-runner: fix two synchronisation issues"):
> > On Fri, Jul 29, 2016 at 03:21:52PM +0100, Ian Jackson wrote:
> > > Wei Liu writes ("[XTF PATCH] xtf-runner: fix two synchronisation issues"):
> > > > There were two synchronisation issues for the old code:
> > > > 
> > > > 1. There was no guarantee that guest console was ready before "xl
> > > >console" invocation.
> > > 
> > > Is this not a bug in xl console ?
> > 
> > I don't think so. It gives up when it can't get tty from xenstore. I
> > think that's reasonable.
> 
> If you say
>   xl create /etc/xen/foo.cfg
>   xl console foo
> then surely you shouldn't find that xl console fails because of some
> race.
> 

What if the guest just doesn't have / want a console? In that case there
will never be a tty node in xenstore, so xl console should wait forever
for one? What is the expectation for this case?

> > > > 2. Poll xenstore guest console node to make sure console is available
> > > >before "xl console" invocation.
> > > 
> > > Users of things like xl shouldn't need to prat about in xenstore too.
> > 
> > What does this mean?
> 
> I mean that xenstore is part of the implementation of libxl, not part
> of its public interface.  In this sense it is a bit like libxc.
> libxl callers should (in general) not look at xenstore - and of
> course they should therefore;2~ not need to do so.
> 

In this particular case, all nodes that xtf-runner code reles on are
documented, so I already consider them public interfaces.

> If the xtf runner needs to look in xenstore (other than perhaps if
> it's doing strange things there as part of its test cases) then that
> means there is some interface or capability missing in libxl.
> 

I don't disagree.

But the missing capability won't be backported to older version of Xen.
I don't want to eliminate the possibility for running xtf on older
versions of Xen.

> OTOH I'm slightly confused because I was under the impression that
> there was a polling loop in xl console (xenconsole) already.
> 
> Maybe I don't understand your problem.
> 

The issue is that when it tries to get tty from xenstore and there isn't
one, it gives up. Your polling loop is something else.

http://osstest.xs.citrite.net/~osstest/testlogs/logs/66855/test-xtf-amd64-amd64-1/10.ts-xtf-fep.log

Wei.

> Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On Wed, Jul 27, 2016, at 09:34 AM, Andrew Cooper wrote:
> This looks suspiciously like the issue which was fixed by c/s
> d6b186c1e2d852a92c43f090d0d8fad4704d51ef "x86/xen: avoid m2p lookup when
> setting early page table entries", but that fix is present in Linux 4.7.0
> 
> Can you check to see whether the commit is included in your kernel?
> 
> Failing that, can you find out exactly where the kernel crashed?  You
> need to manually decode 81f6374c with the debug symbols.
> 
> ~Andrew

Is an eventual fix to whatever is causing this crash likely in the kernel or 
Xen code?

I've managed to try a couple of different kernels, older version, with no luck. 
 Crash still happens.

Getting my hands on an older Xen package is turning out to be a bit tougher.  
Still working on it.

Not sure what specifically changed to break this , since this server clearly 
was running not that long ago.

I'd like to at least figure out what I can drop back to so I can get my server 
and guests back up and running.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v7 00/15] Load BIOS via toolstack instead of been embedded in hvmloader.

[Boris Ostrovsky]

On 07/29/2016 04:29 AM, Wei Liu wrote:
> On Fri, Jul 29, 2016 at 01:28:02AM -0400, Boris Ostrovsky wrote:
>>
>> On 07/28/2016 06:49 AM, Anthony PERARD wrote:
>>> Hi all,
>>>
>>> Changes in V7:
>>>   - There is one new patch at the end to fix the doc.
>>>   - Patch 6 as been change.
>>>   that's it.
>>>
>>>   There is just a few missing ackes:
>>> 6 xen: Move the hvm_start_info C representation from libxc to 
>>> public/xen.h
>>> 8 hvmloader: Locate the BIOS blob
>>> 9 hvmloader: Check modules whereabouts in perform_tests
>>>15 docs/misc/hvmlite: Point to the canonical definition of hvm_start_info
>>>
>>> Thanks.
>>>
>>> A git tree can be found here:
>>> git://xenbits.xen.org/people/aperard/xen-unstable.git
>>> tag: hvmloader-with-separated-bios-v7
>>
>> I am unable to build this:
>>
>> libxl_paths.c: In function ‘libxl__seabios_path’:
>> libxl_paths.c:40: error: ‘SEABIOS_PATH’ undeclared (first use in this
>> function)
>> libxl_paths.c:40: error: (Each undeclared identifier is reported only once
>> libxl_paths.c:40: error: for each function it appears in.)
>> libxl_paths.c: In function ‘libxl__ovmf_path’:
>> libxl_paths.c:45: error: ‘OVMF_PATH’ undeclared (first use in this function)
>>
>> IIUIC these two are supposed to be generated into tools/config.h but they
>> were not for me. I haven't looked any further yet.
>>
>> -boris
>>
> You need to run ./autogen.sh. Anthony didn't commit the changes to
> ./configure.


Yes, that did the trick. Except for the machine on which I actually
wanted it to run:

root@ovs104> ./autogen.sh
configure.ac:4: error: Autoconf version 2.67 or higher is required
configure.ac:4: the top level
autom4te: /usr/bin/m4 failed with exit status: 63
root@ovs104> autoconf --version
autoconf (GNU Autoconf) 2.63
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv2+: GNU GPL version 2 or later

This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by David J. MacKenzie and Akim Demaille.
root@ovs104>

Is 2.67 really a pre-req?

-boris


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XTF PATCH] xtf-runner: fix two synchronisation issues

[Ian Jackson]

Wei Liu writes ("Re: [XTF PATCH] xtf-runner: fix two synchronisation issues"):
> On Fri, Jul 29, 2016 at 03:21:52PM +0100, Ian Jackson wrote:
> > Wei Liu writes ("[XTF PATCH] xtf-runner: fix two synchronisation issues"):
> > > There were two synchronisation issues for the old code:
> > > 
> > > 1. There was no guarantee that guest console was ready before "xl
> > >console" invocation.
> > 
> > Is this not a bug in xl console ?
> 
> I don't think so. It gives up when it can't get tty from xenstore. I
> think that's reasonable.

If you say
  xl create /etc/xen/foo.cfg
  xl console foo
then surely you shouldn't find that xl console fails because of some
race.

> > > 2. Poll xenstore guest console node to make sure console is available
> > >before "xl console" invocation.
> > 
> > Users of things like xl shouldn't need to prat about in xenstore too.
> 
> What does this mean?

I mean that xenstore is part of the implementation of libxl, not part
of its public interface.  In this sense it is a bit like libxc.
libxl callers should (in general) not look at xenstore - and of
course they should therefore;2~ not need to do so.

If the xtf runner needs to look in xenstore (other than perhaps if
it's doing strange things there as part of its test cases) then that
means there is some interface or capability missing in libxl.

OTOH I'm slightly confused because I was under the impression that
there was a polling loop in xl console (xenconsole) already.

Maybe I don't understand your problem.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [qemu-upstream-4.5-testing baseline-only test] 66860: regressions - trouble: broken/fail/pass

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 66860 qemu-upstream-4.5-testing real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/66860/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl   3 host-install(3) broken REGR. vs. 66489
 test-amd64-amd64-xl-pvh-intel  3 host-install(3)broken REGR. vs. 66489
 test-amd64-i386-qemuu-rhel6hvm-amd  3 host-install(3)   broken REGR. vs. 66489
 test-amd64-amd64-xl-credit2   3 host-install(3) broken REGR. vs. 66489
 test-amd64-amd64-xl-pvh-amd   3 host-install(3) broken REGR. vs. 66489
 test-amd64-amd64-xl-qcow2 3 host-install(3) broken REGR. vs. 66489
 test-amd64-amd64-qemuu-nested-amd  3 host-install(3)broken REGR. vs. 66489
 test-amd64-amd64-xl-qemuu-winxpsp3  3 host-install(3)   broken REGR. vs. 66489
 test-amd64-amd64-libvirt-pair 4 host-install/dst_host(4) broken REGR. vs. 66489
 test-amd64-i386-libvirt-pair 4 host-install/dst_host(4) broken REGR. vs. 66489
 test-amd64-i386-xl-qemuu-winxpsp3 15 guest-localmigrate/x10 fail REGR. vs. 
66489

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds  3 host-install(3) broken REGR. vs. 66489
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop  fail like 66489
 test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail like 66489

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 10 guest-start  fail never pass
 test-armhf-armhf-libvirt-raw 10 guest-start  fail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-midway   13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-midway   12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  10 guest-start  fail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass

version targeted for testing:
 qemuu835c204f1196ab8f5213a9dc5299ed76e748cdca
baseline version:
 qemuu5e40cec825a2582d8a91119c485f5130cc2648e9

Last test of basis66489  2016-07-01 10:51:42 Z   28 days
Testing same since66860  2016-07-29 09:48:09 Z0 days1 attempts


People who touched revisions under test:
  P J P 
  Stefan Hajnoczi 
  Stefano Stabellini 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl  broken  
 test-armhf-armhf-xl  pass
 test-amd64-i386-xl   pass
 test-amd64-amd64-qemuu-nested-amdbroken  
 test-amd64-amd64-xl-pvh-amd  broken  
 test-amd64-i386-qemuu-rhel6hvm-amd   broken  
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-i386-xl-qemuu-debianhvm-amd64 pass
 test-amd64-i386-freebsd10-amd64  pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass
 test-amd64-amd64-xl-qemuu-win7-amd64 pass
 test-amd64-i386-xl-qemuu-win7-amd64  fail
 test-amd64-amd64-xl-cr

Re: [Xen-devel] [PATCH] mem_access: Use monitor_traps instead of mem_access_send_req

[Tamas K Lengyel]

On Jul 29, 2016 01:27, "Razvan Cojocaru"  wrote:
>
> On 07/28/2016 10:35 PM, Tamas K Lengyel wrote:
> > The two functions monitor_traps and mem_access_send_req duplicate
> > some of the same functionality. The mem_access_send_req however leaves a
> > lot of the standard vm_event fields to be filled by other functions.
> >
> > Since mem_access events go on the monitor ring in this patch we
consolidate
> > all paths to use monitor_traps to place events on the ring and to fill
in
> > the common parts of the requests.
> >
> > Signed-off-by: Tamas K Lengyel 
> > ---
> > Cc: Stefano Stabellini 
> > Cc: Julien Grall 
> > Cc: Jan Beulich 
> > Cc: Andrew Cooper 
> > Cc: Razvan Cojocaru 
> > Cc: George Dunlap 
> > ---
> >  xen/arch/arm/p2m.c| 69
+++
> >  xen/arch/x86/hvm/hvm.c| 16 ++---
> >  xen/arch/x86/hvm/monitor.c|  6 
> >  xen/arch/x86/mm/p2m.c | 24 ++
> >  xen/common/mem_access.c   | 11 ---
> >  xen/include/asm-x86/hvm/monitor.h |  2 ++
> >  xen/include/asm-x86/p2m.h | 13 +---
> >  xen/include/xen/mem_access.h  |  7 
> >  8 files changed, 63 insertions(+), 85 deletions(-)
> >
> > diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
> > index d82349c..df898a3 100644
> > --- a/xen/arch/arm/p2m.c
> > +++ b/xen/arch/arm/p2m.c
> > @@ -5,7 +5,7 @@
> >  #include 
> >  #include 
> >  #include 
> > -#include 
> > +#include 
> >  #include 
> >  #include 
> >  #include 
> > @@ -1642,12 +1642,41 @@ void __init setup_virt_paging(void)
> >  smp_call_function(setup_virt_paging_one, (void *)val, 1);
> >  }
> >
> > +static int
> > +__p2m_mem_access_send_req(paddr_t gpa, vaddr_t gla, const struct npfec
npfec,
> > +  xenmem_access_t xma)
> > +{
> > +struct vcpu *v = current;
> > +vm_event_request_t req = {};
> > +bool_t sync = (xma == XENMEM_access_n2rwx) ? 0 : 1;
> > +
> > +req.reason = VM_EVENT_REASON_MEM_ACCESS;
> > +
> > +/* Send request to mem access subscriber */
> > +req.u.mem_access.gfn = gpa >> PAGE_SHIFT;
> > +req.u.mem_access.offset = gpa & ((1 << PAGE_SHIFT) - 1);
> > +if ( npfec.gla_valid )
> > +{
> > +req.u.mem_access.flags |= MEM_ACCESS_GLA_VALID;
> > +req.u.mem_access.gla = gla;
> > +
> > +if ( npfec.kind == npfec_kind_with_gla )
> > +req.u.mem_access.flags |= MEM_ACCESS_FAULT_WITH_GLA;
> > +else if ( npfec.kind == npfec_kind_in_gpt )
> > +req.u.mem_access.flags |= MEM_ACCESS_FAULT_IN_GPT;
> > +}
> > +req.u.mem_access.flags |= npfec.read_access? MEM_ACCESS_R : 0;
> > +req.u.mem_access.flags |= npfec.write_access   ? MEM_ACCESS_W : 0;
> > +req.u.mem_access.flags |= npfec.insn_fetch ? MEM_ACCESS_X : 0;
> > +
> > +return monitor_traps(v, sync, &req);
> > +}
> > +
> >  bool_t p2m_mem_access_check(paddr_t gpa, vaddr_t gla, const struct
npfec npfec)
> >  {
> >  int rc;
> >  bool_t violation;
> >  xenmem_access_t xma;
> > -vm_event_request_t *req;
> >  struct vcpu *v = current;
> >  struct p2m_domain *p2m = p2m_get_hostp2m(v->domain);
> >
> > @@ -1734,40 +1763,8 @@ bool_t p2m_mem_access_check(paddr_t gpa, vaddr_t
gla, const struct npfec npfec)
> >  return false;
> >  }
> >
> > -req = xzalloc(vm_event_request_t);
> > -if ( req )
> > -{
> > -req->reason = VM_EVENT_REASON_MEM_ACCESS;
> > -
> > -/* Pause the current VCPU */
> > -if ( xma != XENMEM_access_n2rwx )
> > -req->flags |= VM_EVENT_FLAG_VCPU_PAUSED;
> > -
> > -/* Send request to mem access subscriber */
> > -req->u.mem_access.gfn = gpa >> PAGE_SHIFT;
> > -req->u.mem_access.offset =  gpa & ((1 << PAGE_SHIFT) - 1);
> > -if ( npfec.gla_valid )
> > -{
> > -req->u.mem_access.flags |= MEM_ACCESS_GLA_VALID;
> > -req->u.mem_access.gla = gla;
> > -
> > -if ( npfec.kind == npfec_kind_with_gla )
> > -req->u.mem_access.flags |= MEM_ACCESS_FAULT_WITH_GLA;
> > -else if ( npfec.kind == npfec_kind_in_gpt )
> > -req->u.mem_access.flags |= MEM_ACCESS_FAULT_IN_GPT;
> > -}
> > -req->u.mem_access.flags |= npfec.read_access? MEM_ACCESS_R
: 0;
> > -req->u.mem_access.flags |= npfec.write_access   ? MEM_ACCESS_W
: 0;
> > -req->u.mem_access.flags |= npfec.insn_fetch ? MEM_ACCESS_X
: 0;
> > -req->vcpu_id = v->vcpu_id;
>
> The line setting req->vcpu_id has been removed here ...
>
> > -
> > -mem_access_send_req(v->domain, req);
> > -xfree(req);
> > -}
> > -
> > -/* Pause the current VCPU */
> > -if ( xma != XENMEM_access_n2rwx )
> > -vm_event_vcpu_pause(v);
> > +if ( __p2m_mem_access_send_req(gpa, gla, npfec, xma) < 0 )
> > +domain_crash(v->domain);
> >
> >  return false;
> >  }
> > diff --git a/xen/arch/x86/hvm/hvm.c

Re: [Xen-devel] [PATCH] arm/vm_event: get/set registers

[Tamas K Lengyel]

On Jul 29, 2016 08:21, "Razvan Cojocaru"  wrote:
>
> On 07/29/2016 05:11 PM, Julien Grall wrote:
> > Which lead to my next question. For instance, we have an app A which is
> > built for Xen 4.N and they want to also support Xen 4.(N + 1) which will
> > set more registers and take advantage of it. How the app will
> > differentiate the 2 versions?
>
> We currenly have macros like VM_EVENT_INTERFACE_VERSION that can be
> incremented so that applications can know at compile-time which
> interface they get.
>
> At runtime, I suppose we could use code similar to what runs when "xl
> info" is called (which outputs xen_major, xen_minor and xen_extra, or
> the complete xen_version).
>
>

The vm_event interface version is included in all requests and responses so
the user knows at runtime what it is running on. We enforce on the Xen side
that the response matches the interface expected.

Tamas
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XTF PATCH] xtf-runner: fix two synchronisation issues

[Ian Jackson]

Wei Liu writes ("Re: [XTF PATCH] xtf-runner: fix two synchronisation issues"):
> On Fri, Jul 29, 2016 at 01:43:42PM +0100, Andrew Cooper wrote:
> > The runner existing before xl has torn down the guest is very
> > deliberate, because some part of hvm guests is terribly slow to tear
> > down; waiting synchronously for teardown tripled the wallclock time to
> > run a load of tests back-to-back.
> 
> Then you won't know if a guest is leaked or it is being slowly destroyed
> when a dead guest shows up in the snapshot of 'xl list'.
> 
> Also consider that would make back-to-back tests that happen to have a
> guest that has the same name as the one in previous test fail.
> 
> I don't think getting blocked for a few more seconds is a big issue.
> It's is important to eliminate such race conditions so that osstest can
> work properly.

IMO the biggest reason for waiting for teardown is that that will make
it possible to accurately identify the xtf test which was responsible
for the failure if a test reveals a bug which causes problems for the
whole host.

Suppose there is a test T1 which, in buggy hypervisors, creates an
anomalous data structure, such that the hypervisor crashes when T1's
guest is finally torn down.

If we start to run the next test T2 immediately we see success output
from T1, we will observe the host crashing "due to T2", and T1 would
be regarded as having succeeded.

This is why in an in-person conversation with Wei yesterday I
recommended that osstest should after each xtf test (i) wait for
everything to be torn down and (ii) then check that the dom0 is still
up.  (And these two activities are regarded as part of the preceding
test step.)

If this leads to over-consumption of machine resources because this
serialisation is too slow then the right approach would be explicit
parallelisation in osstest.  That would still mean that in the
scenario above, T1 would be regarded as having failed, because T1
wouldn't be regarded as having passed until osstest had seen that all
of T1's cleanup had been done and the host was still up.  (T2 would
_also_ be regarded as failed, and that might look like a heisenbug,
but that would be tolerable.)

Wei: I need to check what happens with multiple failing test steps in
the same job.  Specifically, I need to check which one the bisector
is likely to try to attack.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XTF PATCH] xtf-runner: fix two synchronisation issues

[Andrew Cooper]

On 29/07/16 15:21, Ian Jackson wrote:
> Wei Liu writes ("[XTF PATCH] xtf-runner: fix two synchronisation issues"):
>> There were two synchronisation issues for the old code:
>>
>> 1. There was no guarantee that guest console was ready before "xl
>>console" invocation.
> Is this not a bug in xl console ?
>
>> 2. Poll xenstore guest console node to make sure console is available
>>before "xl console" invocation.
> Users of things like xl shouldn't need to prat about in xenstore too.

Ideally, what xtf-runner wants is for `xl create -c $FOO.cfg` to work as
one would logically intend, but it doesn't.  It does a create, then
falls into the weeds trying to locate the console for a domain which has
already exited.

As a workaround, xtf-runner currently does

`xl create -p $foo.cfg`
`xl console $test &`
`xl unpause $text`
wait on xl console to complete.

but it turns out that this even isn't good enough.

xl should definitely be fixed, but xtf-runner needs to be able to cope
with older broken xl as well.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XTF PATCH] xtf-runner: fix two synchronisation issues

[Wei Liu]

On Fri, Jul 29, 2016 at 03:21:52PM +0100, Ian Jackson wrote:
> Wei Liu writes ("[XTF PATCH] xtf-runner: fix two synchronisation issues"):
> > There were two synchronisation issues for the old code:
> > 
> > 1. There was no guarantee that guest console was ready before "xl
> >console" invocation.
> 
> Is this not a bug in xl console ?
> 

I don't think so. It gives up when it can't get tty from xenstore. I
think that's reasonable.

> > 2. Poll xenstore guest console node to make sure console is available
> >before "xl console" invocation.
> 
> Users of things like xl shouldn't need to prat about in xenstore too.
> 

What does this mean?

Wei.

> Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XTF PATCH] xtf-runner: fix two synchronisation issues

[Ian Jackson]

Wei Liu writes ("[XTF PATCH] xtf-runner: fix two synchronisation issues"):
> There were two synchronisation issues for the old code:
> 
> 1. There was no guarantee that guest console was ready before "xl
>console" invocation.

Is this not a bug in xl console ?

> 2. Poll xenstore guest console node to make sure console is available
>before "xl console" invocation.

Users of things like xl shouldn't need to prat about in xenstore too.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] arm/vm_event: get/set registers

[Razvan Cojocaru]

On 07/29/2016 05:11 PM, Julien Grall wrote:
> Which lead to my next question. For instance, we have an app A which is
> built for Xen 4.N and they want to also support Xen 4.(N + 1) which will
> set more registers and take advantage of it. How the app will
> differentiate the 2 versions?

We currenly have macros like VM_EVENT_INTERFACE_VERSION that can be
incremented so that applications can know at compile-time which
interface they get.

At runtime, I suppose we could use code similar to what runs when "xl
info" is called (which outputs xen_major, xen_minor and xen_extra, or
the complete xen_version).


Thanks,
Razvan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] mem_access: Use monitor_traps instead of mem_access_send_req

[Tamas K Lengyel]

On Jul 29, 2016 02:50, "Julien Grall"  wrote:
>
>
>
> On 28/07/16 23:54, Tamas K Lengyel wrote:
>>
>> On Thu, Jul 28, 2016 at 2:38 PM, Julien Grall 
wrote:
>>>
>>> On 28/07/2016 20:35, Tamas K Lengyel wrote:
>>> This patch is doing more than it is claimed in the commit message.
>>>
>>> In general, moving the code and introducing changes within the same
patch
>>> should really be avoided. So please split it in 2 patches.
>>
>>
>> Well, the changes are largely cosmetic so doing a whole separate patch
>> IMHO is an overkill. How about adjusting the commit message to
>> something like "sanitize code surrounding sending mem_access
>> vm_events" to better describe the adjustments made in this patch?
>
>
> I think the wiki page "Submitting Xen Project patches" [1] should answer
to your question.
>
> If not, trivial patches are easy to review, merging multiple trivial
patches in a single patch is not. Moving code and at the same time as
changing the behavior is fairly difficult to review because it hides the
real modifications.
>
> Regards,
>
> [1]
http://wiki.xenproject.org/wiki/Submitting_Xen_Project_Patches#Break_down_your_patches
>

The behavior didn't change at all, this whole patch is code sanitization.
It's not worth doing a separate patch for each minor change. The few change
on the arm side is the vm_event request allocation going from xzalloc to
stack based and using monitor_traps now in a split-out function. It really
should be no problem reviewing it. Even Andrew requested minor adjustments
to be included in this patch. Anyway, I'm not looking to change this into a
series. If it's a no go from your side I'm just going to cut down the ARM
side sanitization to the bare minimum of using monitor_traps as the rest
just does not worth the effort.

Tamas
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] arm/vm_event: get/set registers

[Julien Grall]

Hi Tamas,

On 28/07/16 23:45, Tamas K Lengyel wrote:

On Thu, Jul 28, 2016 at 4:03 PM, Julien Grall  wrote:



On 28/07/2016 22:33, Tamas K Lengyel wrote:


On Jul 28, 2016 15:25, "Julien Grall" mailto:julien.gr...@arm.com>> wrote:





On 28/07/2016 22:05, Tamas K Lengyel wrote:



On Thu, Jul 28, 2016 at 3:01 PM, Julien Grall 

> wrote:


That's not how we do it with vm_event. Even on x86 we only selectively
set registers using the VM_EVENT_FLAG_SET_REGISTERS flag (albeit it
not being documented in the header). As for "not exposing them" it's a
waste to declare separate structures for getting and setting. I'll
change my mind about that if Razvan is on the side that we should
start doing that, but I don't think that's the case at the moment.




Is there any rationale to only set a subset of the information you


retrieved?





I just did a testrun with setting every register through this method to
0 other then pc and it resulted in hypervisor crash. Not sure if it's
just my setup or not though so I'm still poking at it. However, I don't
really see a usecase where setting ttbr regs to be required either via
the fast method so it simply may not worth digging into it more at this
time.



To confirm, do you mean setting CPSR, TTBR0_EL1, TTBR1_EL1 to 0?

TTBR*_EL1 are safe to set to any values (they are directly accessible by the
guest anyway). However this is not the case of CPSR. From my understanding
of the ARM ARM (B1-1150 in ARM DDI 0406C.b) is writing 0 to M[4:0] will lead
to an unpredictable behavior (that could cause an hypervisor trap).

Can you copy/paste the hypervisor crash log?



OK, the issue I had was that right now mem_access on ARM doesn't send
the registers (yet) as it needs my monitor_traps work from the other
patch so I kept setting all registers to 0. Rebasing on top of my
other patch now I was able to verify that indeed only setting cpsr to
arbitrary values (like 0) can cause hypervisor crash as follows:


Thank you for the stack trace. I managed to reproduce it and did further 
testing.


We need to sanitize the CPSR mode in Xen before writing it. So far, this 
can be only set by XEN_DOMCTL_setvcpucontext.


[...]


Setting ttb/cr/r0/r1 is fine however as it only causes an in-guest
kernel crash. So we still need to selectively set registers and as at
this point only PC makes sense we will start with just that.


Which lead to my next question. For instance, we have an app A which is 
built for Xen 4.N and they want to also support Xen 4.(N + 1) which will 
set more registers and take advantage of it. How the app will 
differentiate the 2 versions?


Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XTF PATCH] xtf-runner: fix two synchronisation issues

[Andrew Cooper]

On 29/07/16 13:07, Wei Liu wrote:
>  
>  if __name__ == "__main__":
> +signal.signal(signal.SIGALRM, sigalrm_handler)
>  try:
>  sys.exit(main())
>  except RunnerError, e:

One final thing before I forget, this addition should be in main(),
rather than here.

This bit of code is just to allow main() to work per its normal
expectations.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XTF PATCH] xtf-runner: fix two synchronisation issues

[Wei Liu]

On Fri, Jul 29, 2016 at 02:23:20PM +0100, Andrew Cooper wrote:
> On 29/07/16 14:12, Wei Liu wrote:
> > On Fri, Jul 29, 2016 at 02:06:56PM +0100, Andrew Cooper wrote:
> >> On 29/07/16 13:58, Wei Liu wrote:
> >>> On Fri, Jul 29, 2016 at 01:43:42PM +0100, Andrew Cooper wrote:
>  On 29/07/16 13:07, Wei Liu wrote:
> > There were two synchronisation issues for the old code:
> >
> > 1. There was no guarantee that guest console was ready before "xl
> >console" invocation.
> > 2. There was no guarantee that runner wouldn't not exit before all test
> >>> s/not//
> >>>
> >guests were gone.
>  Sorry, but I can't parse this.
> 
>  The runner existing before xl has torn down the guest is very
>  deliberate, because some part of hvm guests is terribly slow to tear
>  down; waiting synchronously for teardown tripled the wallclock time to
>  run a load of tests back-to-back.
> 
> >>> Then you won't know if a guest is leaked or it is being slowly destroyed
> >>> when a dead guest shows up in the snapshot of 'xl list'.
> >>>
> >>> Also consider that would make back-to-back tests that happen to have a
> >>> guest that has the same name as the one in previous test fail.
> >> test names are globally unique, so this isn't an issue.
> >>
> >> Also, the wait for `xl console` to complete shows that @releasedomain
> >> has been fired for the domain.
> >>
> > Are you suggesting waiting for "xl console" only is good enough?
> 
> The "stdout, _ = console.communicate()" line waits for `xl console` to
> exit.  (In fact, `xl` exec()'s `xenconosle`)
> 
> As we never put a CTRL-] into stdin, `xl console` only exits when the
> PTY shuts, which is when xenconsoled decided the domain has terminated.
> 
> So, yes - I think this is safe.
> 

TBH I would rather to be explicit in this regard. I would rather stick
with checking "xl create".

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XTF PATCH] xtf-runner: fix two synchronisation issues

[Andrew Cooper]

On 29/07/16 14:12, Wei Liu wrote:
> On Fri, Jul 29, 2016 at 02:06:56PM +0100, Andrew Cooper wrote:
>> On 29/07/16 13:58, Wei Liu wrote:
>>> On Fri, Jul 29, 2016 at 01:43:42PM +0100, Andrew Cooper wrote:
 On 29/07/16 13:07, Wei Liu wrote:
> There were two synchronisation issues for the old code:
>
> 1. There was no guarantee that guest console was ready before "xl
>console" invocation.
> 2. There was no guarantee that runner wouldn't not exit before all test
>>> s/not//
>>>
>guests were gone.
 Sorry, but I can't parse this.

 The runner existing before xl has torn down the guest is very
 deliberate, because some part of hvm guests is terribly slow to tear
 down; waiting synchronously for teardown tripled the wallclock time to
 run a load of tests back-to-back.

>>> Then you won't know if a guest is leaked or it is being slowly destroyed
>>> when a dead guest shows up in the snapshot of 'xl list'.
>>>
>>> Also consider that would make back-to-back tests that happen to have a
>>> guest that has the same name as the one in previous test fail.
>> test names are globally unique, so this isn't an issue.
>>
>> Also, the wait for `xl console` to complete shows that @releasedomain
>> has been fired for the domain.
>>
> Are you suggesting waiting for "xl console" only is good enough?

The "stdout, _ = console.communicate()" line waits for `xl console` to
exit.  (In fact, `xl` exec()'s `xenconosle`)

As we never put a CTRL-] into stdin, `xl console` only exits when the
PTY shuts, which is when xenconsoled decided the domain has terminated.

So, yes - I think this is safe.

>
>>> I don't think getting blocked for a few more seconds is a big issue.
>>> It's is important to eliminate such race conditions so that osstest can
>>> work properly.
>> Amortising the teardown cost in the background is fine (which is what
>> your "for child in wait_list" ends up doing), but an extra few seconds
>> per test is very important to be avoided for manual use of XTF.
>>
> So let's default to not wait and add an option to wait. Osstest will use
> that option.

The code you currently have should be fine (in this respect).  This way,
you are tearing down a dying domain at the same time as starting the
next one up, which is fine.

What I want to avoid is waiting for complete teardown before starting
the next domain up.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] live migrating hvm from 4.4 to 4.7 fails in ioreq server

[Paul Durrant]

> -Original Message-
> From: Olaf Hering [mailto:o...@aepfle.de]
> Sent: 29 July 2016 14:11
> To: Paul Durrant
> Cc: Stefano Stabellini; Wei Liu; Andrew Cooper; xen-devel@lists.xen.org;
> Anthony Perard; zhigang.x.w...@oracle.com
> Subject: Re: [Xen-devel] live migrating hvm from 4.4 to 4.7 fails in ioreq
> server
> 
> On Fri, Jul 29, Paul Durrant wrote:
> 
> > >   Could you give the attached patch a try? I believe it should solve the
> > > problem.
> 
> Thanks Paul. I tested it with 4.4.20160610T12110 dom0 as sender and
> 4.6.20160620T12082 as receiver.
> 
> 
> Initially the result was this:
> 
> # cat /var/log/xen/qemu-dm-fv-x64-sles12sp1-clean--incoming.log
> char device redirected to /dev/pts/3 (label serial0)
> xen: ioreq server create: Cannot allocate memory
> qemu-system-x86_64: xen hardware virtual machine initialisation failed
> 
> 
> Now the result it this:
> root@anonymi:~ # cat /var/log/xen/qemu-dm-fv-x64-sles12sp1-clean--
> incoming.log
> char device redirected to /dev/pts/3 (label serial0)
> xen_ram_alloc: do not alloc f80 bytes of ram at 0 when runstate is
> INMIGRATE
> xen_ram_alloc: do not alloc 80 bytes of ram at f80 when runstate is
> INMIGRATE
> xen_ram_alloc: do not alloc 1 bytes of ram at 1000 when runstate is
> INMIGRATE
> xen_ram_alloc: do not alloc 4 bytes of ram at 1001 when runstate is
> INMIGRATE
> Unknown savevm section or instance 'kvm-tpr-opt' 0
> qemu-system-i386: load of migration failed: Invalid argument
> 
> So appearently it got past the point of the ioreq server issue.
> 

Yes, it certainly looks like it.

> 
> The "kvm-tpr-opt" thing was briefly discussed on 12 May 2016.
> 

I'll have a look for that.

Thanks,

  Paul

> Olaf
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Default controller type for USB devices

[George Dunlap]

On 29/07/16 14:00, Juergen Gross wrote:
> When specifying no USB controller type for a usb device the default is
> chosen in libxl__device_usbctrl_setdefault(). For a HVM guest this is
> currently the not yet supported "LIBXL_USBCTRL_TYPE_DEVICEMODEL".
> 
> Wouldn't it make sense to handle HVM guests in the same way as PV guests
> as long as emulated USB devices are not implemented in libxl? Or would
> this create future incompatibilities which we don't want to run into?
> 
> I'd be happy to send a patch if this is the way to go.

I think the big thing is that we can pretty much expect a typical HVM
guest OS to have drivers for the emulated hardware; we *cannot* expect
an average HVM guest OS to have pvfront drivers.  This will probably
always apply to Windows, but at the moment it even applies to Linux.

So what happens right now if a user simply asks for a usbctrl for an HVM
guest?  They get an error on domain creation.  This will hopefully
prompt them to look into either switching back to the old format, or
finding out about pvusb, at which point they can maybe find a pvusbfront
for their OS.

If we make the switch you propose, then the user will create a usb
controller, which will succeed; but if the guest OS doesn't have
pvusbfront drivers (which is likely), then they will mysteriously just
not see the controller and devices they've plugged in.

As a user, I would personally rather have an error message up front
which gives me a clue as to what's wrong, rather than have things
mysteriously not work and spend a lot of time going around trying to
figure out what's wrong.

So I think the current default is the best.  Wei / Ian, any thoughts?

 -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XTF PATCH] xtf-runner: fix two synchronisation issues

[Wei Liu]

On Fri, Jul 29, 2016 at 02:06:56PM +0100, Andrew Cooper wrote:
> On 29/07/16 13:58, Wei Liu wrote:
> > On Fri, Jul 29, 2016 at 01:43:42PM +0100, Andrew Cooper wrote:
> >> On 29/07/16 13:07, Wei Liu wrote:
> >>> There were two synchronisation issues for the old code:
> >>>
> >>> 1. There was no guarantee that guest console was ready before "xl
> >>>console" invocation.
> >>> 2. There was no guarantee that runner wouldn't not exit before all test
> > s/not//
> >
> >>>guests were gone.
> >> Sorry, but I can't parse this.
> >>
> >> The runner existing before xl has torn down the guest is very
> >> deliberate, because some part of hvm guests is terribly slow to tear
> >> down; waiting synchronously for teardown tripled the wallclock time to
> >> run a load of tests back-to-back.
> >>
> > Then you won't know if a guest is leaked or it is being slowly destroyed
> > when a dead guest shows up in the snapshot of 'xl list'.
> >
> > Also consider that would make back-to-back tests that happen to have a
> > guest that has the same name as the one in previous test fail.
> 
> test names are globally unique, so this isn't an issue.
> 
> Also, the wait for `xl console` to complete shows that @releasedomain
> has been fired for the domain.
> 

Are you suggesting waiting for "xl console" only is good enough?

> >
> > I don't think getting blocked for a few more seconds is a big issue.
> > It's is important to eliminate such race conditions so that osstest can
> > work properly.
> 
> Amortising the teardown cost in the background is fine (which is what
> your "for child in wait_list" ends up doing), but an extra few seconds
> per test is very important to be avoided for manual use of XTF.
> 

So let's default to not wait and add an option to wait. Osstest will use
that option.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] live migrating hvm from 4.4 to 4.7 fails in ioreq server

[Olaf Hering]

On Fri, Jul 29, Paul Durrant wrote:

> >   Could you give the attached patch a try? I believe it should solve the
> > problem.

Thanks Paul. I tested it with 4.4.20160610T12110 dom0 as sender and
4.6.20160620T12082 as receiver.


Initially the result was this:

# cat /var/log/xen/qemu-dm-fv-x64-sles12sp1-clean--incoming.log
char device redirected to /dev/pts/3 (label serial0)
xen: ioreq server create: Cannot allocate memory
qemu-system-x86_64: xen hardware virtual machine initialisation failed


Now the result it this:
root@anonymi:~ # cat /var/log/xen/qemu-dm-fv-x64-sles12sp1-clean--incoming.log
char device redirected to /dev/pts/3 (label serial0)
xen_ram_alloc: do not alloc f80 bytes of ram at 0 when runstate is INMIGRATE
xen_ram_alloc: do not alloc 80 bytes of ram at f80 when runstate is 
INMIGRATE
xen_ram_alloc: do not alloc 1 bytes of ram at 1000 when runstate is 
INMIGRATE
xen_ram_alloc: do not alloc 4 bytes of ram at 1001 when runstate is 
INMIGRATE
Unknown savevm section or instance 'kvm-tpr-opt' 0
qemu-system-i386: load of migration failed: Invalid argument

So appearently it got past the point of the ioreq server issue.


The "kvm-tpr-opt" thing was briefly discussed on 12 May 2016.

Olaf


signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XTF PATCH] xtf-runner: fix two synchronisation issues

[Andrew Cooper]

On 29/07/16 13:58, Wei Liu wrote:
> On Fri, Jul 29, 2016 at 01:43:42PM +0100, Andrew Cooper wrote:
>> On 29/07/16 13:07, Wei Liu wrote:
>>> There were two synchronisation issues for the old code:
>>>
>>> 1. There was no guarantee that guest console was ready before "xl
>>>console" invocation.
>>> 2. There was no guarantee that runner wouldn't not exit before all test
> s/not//
>
>>>guests were gone.
>> Sorry, but I can't parse this.
>>
>> The runner existing before xl has torn down the guest is very
>> deliberate, because some part of hvm guests is terribly slow to tear
>> down; waiting synchronously for teardown tripled the wallclock time to
>> run a load of tests back-to-back.
>>
> Then you won't know if a guest is leaked or it is being slowly destroyed
> when a dead guest shows up in the snapshot of 'xl list'.
>
> Also consider that would make back-to-back tests that happen to have a
> guest that has the same name as the one in previous test fail.

test names are globally unique, so this isn't an issue.

Also, the wait for `xl console` to complete shows that @releasedomain
has been fired for the domain.

>
> I don't think getting blocked for a few more seconds is a big issue.
> It's is important to eliminate such race conditions so that osstest can
> work properly.

Amortising the teardown cost in the background is fine (which is what
your "for child in wait_list" ends up doing), but an extra few seconds
per test is very important to be avoided for manual use of XTF.

>>> +
>>> +cmd = ['xenstore-read', '/local/domain/'+domid+'/name']
>>> +print "Executing '%s'" % (" ".join(cmd), )
>>> +gname = check_output(cmd).splitlines()[0]
>>> +if gname != test: continue
>>> +
>>> +cmd = ['xenstore-read', '/local/domain/'+domid+'/console/tty']
>>> +print "Executing '%s'" % (" ".join(cmd), )
>>> +con = check_output(cmd).splitlines()[0]
>>> +if con != '': break
>> Somewhere in this loop, you should poll the call to xl create.  In the
>> case that there is an xl configuration error, this setup will wait for 5
>> seconds, then discard all trace of the error.
>>
> Right. I will see what I can do here.

Popen().poll() seems to do what you want.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Kernel panic on Xen virtualisation in Debian

[Wei Liu]

On Fri, Jul 29, 2016 at 01:45:55PM +0200, Andreas Ziegler wrote:
> Hello Wei,
> 
> we tried with kernel 4.6 now, the crashed happened again, though.
> 
> next we want to try the Xen debug build, but we couldn't find any
> information on how to enable debug for the build, perhaps you could give
> us a hint.
> 

First please read:

http://wiki.xenproject.org/wiki/Compiling_Xen_From_Source

for general information.

Then in the cloned xen.git tree -- assuming you use Xen 4.7 (maybe 4.6 as
well)

  make -C xen menuconfig

to get a Linux style menuconfig experience

If you're using earlier version of Xen, you need to modify
xen.git/Config.mk to set the debug variable.

Feel free to ask if you have more questions.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] Default controller type for USB devices

[Juergen Gross]

When specifying no USB controller type for a usb device the default is
chosen in libxl__device_usbctrl_setdefault(). For a HVM guest this is
currently the not yet supported "LIBXL_USBCTRL_TYPE_DEVICEMODEL".

Wouldn't it make sense to handle HVM guests in the same way as PV guests
as long as emulated USB devices are not implemented in libxl? Or would
this create future incompatibilities which we don't want to run into?

I'd be happy to send a patch if this is the way to go.


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XTF PATCH] xtf-runner: fix two synchronisation issues

[Wei Liu]

On Fri, Jul 29, 2016 at 01:43:42PM +0100, Andrew Cooper wrote:
> On 29/07/16 13:07, Wei Liu wrote:
> > There were two synchronisation issues for the old code:
> >
> > 1. There was no guarantee that guest console was ready before "xl
> >console" invocation.
> > 2. There was no guarantee that runner wouldn't not exit before all test

s/not//

> >guests were gone.
> 
> Sorry, but I can't parse this.
> 
> The runner existing before xl has torn down the guest is very
> deliberate, because some part of hvm guests is terribly slow to tear
> down; waiting synchronously for teardown tripled the wallclock time to
> run a load of tests back-to-back.
> 

Then you won't know if a guest is leaked or it is being slowly destroyed
when a dead guest shows up in the snapshot of 'xl list'.

Also consider that would make back-to-back tests that happen to have a
guest that has the same name as the one in previous test fail.

I don't think getting blocked for a few more seconds is a big issue.
It's is important to eliminate such race conditions so that osstest can
work properly.

> > @@ -132,24 +141,53 @@ def list_tests(args):
> >  
> >  print name
> >  
> > +# A list of processes that we need to wait until they exits.
> > +wait_list = []
> >  
> >  def run_test(test):
> >  """ Run a specific test """
> >  
> > +console_path = '/local/domain/0/backend/console'
> > +# Setup watch for console
> > +cmd = ['xenstore-watch', console_path]
> > +print "Executing '%s'" % (" ".join(cmd), )
> > +xs_watch = Popen(cmd, stdout = PIPE, stderr = PIPE)
> > +
> >  _, _, name = test.split('-', 2)
> >  
> >  cfg = path.join("tests", name, test + ".cfg")
> >  
> > -cmd = ['xl', 'create', '-p', cfg]
> > +cmd = ['xl', 'create', '-Fp', cfg]
> >  
> >  print "Executing '%s'" % (" ".join(cmd), )
> > -rc = subproc_call(cmd)
> > -if rc:
> > -raise RunnerError("Failed to create VM")
> > +wait = Popen(cmd, stdout = DEVNULL, stderr = DEVNULL)
> 
> I would name this "create" rather than "wait"
> 

NP.

> > +wait_list.append(wait)
> > +
> > +# Wait up to 5 seconds for console to show up
> > +signal.alarm(5)
> > +while True:
> > +l = xs_watch.stdout.readline()
> > +domid = l[len(console_path)+1:].split('/')[0]
> > +if domid == '': continue
> 
> Please put continues and breaks on newlines.
> 
> if not domid:
> continue

Fixed.

> 
> > +
> > +cmd = ['xenstore-read', '/local/domain/'+domid+'/name']
> > +print "Executing '%s'" % (" ".join(cmd), )
> > +gname = check_output(cmd).splitlines()[0]
> > +if gname != test: continue
> > +
> > +cmd = ['xenstore-read', '/local/domain/'+domid+'/console/tty']
> > +print "Executing '%s'" % (" ".join(cmd), )
> > +con = check_output(cmd).splitlines()[0]
> > +if con != '': break
> 
> Somewhere in this loop, you should poll the call to xl create.  In the
> case that there is an xl configuration error, this setup will wait for 5
> seconds, then discard all trace of the error.
> 

Right. I will see what I can do here.

> > +
> > +# Tear down watch and timer
> > +signal.alarm(0)
> > +xs_watch.kill()
> >  
> >  cmd = ['xl', 'console', test]
> >  print "Executing '%s'" % (" ".join(cmd), )
> >  console = Popen(cmd, stdout = PIPE)
> > +wait_list.append(console)
> >  
> >  cmd = ['xl', 'unpause', test]
> >  print "Executing '%s'" % (" ".join(cmd), )
> > @@ -327,12 +365,17 @@ def main():
> >  opts, args = parser.parse_args()
> >  
> >  if opts.list_tests:
> > -return list_tests(args)
> > +ret = list_tests(args)
> >  else:
> > -return run_tests(args)
> > +ret = run_tests(args)
> > +
> > +for child in wait_list: child.wait()
> 
> On a newline please.
> 
> You should use stdout=PIPE, stderr=STDOUT (to link stdout and stderr to
> the same fd) and .communicate() to get the results.  In any case that a
> child exists non-zero, you mustn't discard the error.  As a result, I
> don't think you need DEVNULL any more.
> 

OK, this makes sense.

Wei.

> ~Andrew
> 
> > +
> > +return ret
> >  
> >  
> >  if __name__ == "__main__":
> > +signal.signal(signal.SIGALRM, sigalrm_handler)
> >  try:
> >  sys.exit(main())
> >  except RunnerError, e:
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [libvirt test] 99738: tolerable FAIL - PUSHED

[osstest service owner]

flight 99738 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/99738/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass

version targeted for testing:
 libvirt  a48c714115c3cfde1aac90b75152204f9a628209
baseline version:
 libvirt  cabae6319452a504cad981566235a4b297d2ac4f

Last test of basis99696  2016-07-26 04:20:34 Z3 days
Testing same since99738  2016-07-28 04:30:53 Z1 days1 attempts


People who touched revisions under test:
  Anton Khramov 
  Daniel P. Berrange 
  Derbyshev Dmitry 
  Erik Skultety 
  Henning Schild 
  Joao Martins 
  John Ferlan 
  Jovanka Gulicoska 
  Ján Tomko 
  Michal Privoznik 
  Pavel Hrdina 
  Peter Krempa 
  Prasanna Kumar Kalever 
  Ramon Medeiros 
  Shivaprasad G Bhat 
  Tomasz Flendrich 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmpass
 test-amd64-amd64-libvirt-xsm pass
 test-armhf-armhf-libvirt-xsm fail
 test-amd64-i386-libvirt-xsm  pass
 test-amd64-amd64-libvirt pass
 test-armhf-armhf-libvirt fail
 test-amd64-i386-libvirt  pass
 test-amd64-amd64-libvirt-pairpass
 test-amd64-i386-libvirt-pair pass
 test-armhf-armhf-libvirt-qcow2   fail
 test-armhf-armhf-libvirt-raw fail
 test-amd64-amd64-libvirt-vhd pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=libvirt
+ revision=a48c714115c3cfde1aac90b75152204f9a628209
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest

Re: [Xen-devel] [XTF PATCH] xtf-runner: fix two synchronisation issues

[Andrew Cooper]

On 29/07/16 13:07, Wei Liu wrote:
> There were two synchronisation issues for the old code:
>
> 1. There was no guarantee that guest console was ready before "xl
>console" invocation.
> 2. There was no guarantee that runner wouldn't not exit before all test
>guests were gone.

Sorry, but I can't parse this.

The runner existing before xl has torn down the guest is very
deliberate, because some part of hvm guests is terribly slow to tear
down; waiting synchronously for teardown tripled the wallclock time to
run a load of tests back-to-back.

> @@ -132,24 +141,53 @@ def list_tests(args):
>  
>  print name
>  
> +# A list of processes that we need to wait until they exits.
> +wait_list = []
>  
>  def run_test(test):
>  """ Run a specific test """
>  
> +console_path = '/local/domain/0/backend/console'
> +# Setup watch for console
> +cmd = ['xenstore-watch', console_path]
> +print "Executing '%s'" % (" ".join(cmd), )
> +xs_watch = Popen(cmd, stdout = PIPE, stderr = PIPE)
> +
>  _, _, name = test.split('-', 2)
>  
>  cfg = path.join("tests", name, test + ".cfg")
>  
> -cmd = ['xl', 'create', '-p', cfg]
> +cmd = ['xl', 'create', '-Fp', cfg]
>  
>  print "Executing '%s'" % (" ".join(cmd), )
> -rc = subproc_call(cmd)
> -if rc:
> -raise RunnerError("Failed to create VM")
> +wait = Popen(cmd, stdout = DEVNULL, stderr = DEVNULL)

I would name this "create" rather than "wait"

> +wait_list.append(wait)
> +
> +# Wait up to 5 seconds for console to show up
> +signal.alarm(5)
> +while True:
> +l = xs_watch.stdout.readline()
> +domid = l[len(console_path)+1:].split('/')[0]
> +if domid == '': continue

Please put continues and breaks on newlines.

if not domid:
continue

> +
> +cmd = ['xenstore-read', '/local/domain/'+domid+'/name']
> +print "Executing '%s'" % (" ".join(cmd), )
> +gname = check_output(cmd).splitlines()[0]
> +if gname != test: continue
> +
> +cmd = ['xenstore-read', '/local/domain/'+domid+'/console/tty']
> +print "Executing '%s'" % (" ".join(cmd), )
> +con = check_output(cmd).splitlines()[0]
> +if con != '': break

Somewhere in this loop, you should poll the call to xl create.  In the
case that there is an xl configuration error, this setup will wait for 5
seconds, then discard all trace of the error.

> +
> +# Tear down watch and timer
> +signal.alarm(0)
> +xs_watch.kill()
>  
>  cmd = ['xl', 'console', test]
>  print "Executing '%s'" % (" ".join(cmd), )
>  console = Popen(cmd, stdout = PIPE)
> +wait_list.append(console)
>  
>  cmd = ['xl', 'unpause', test]
>  print "Executing '%s'" % (" ".join(cmd), )
> @@ -327,12 +365,17 @@ def main():
>  opts, args = parser.parse_args()
>  
>  if opts.list_tests:
> -return list_tests(args)
> +ret = list_tests(args)
>  else:
> -return run_tests(args)
> +ret = run_tests(args)
> +
> +for child in wait_list: child.wait()

On a newline please.

You should use stdout=PIPE, stderr=STDOUT (to link stdout and stderr to
the same fd) and .communicate() to get the results.  In any case that a
child exists non-zero, you mustn't discard the error.  As a result, I
don't think you need DEVNULL any more.

~Andrew

> +
> +return ret
>  
>  
>  if __name__ == "__main__":
> +signal.signal(signal.SIGALRM, sigalrm_handler)
>  try:
>  sys.exit(main())
>  except RunnerError, e:


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [qemu-upstream-unstable test] 99728: regressions - trouble: broken/fail/pass

[osstest service owner]

flight 99728 qemu-upstream-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/99728/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-credit2   3 host-install(3) broken REGR. vs. 94800
 test-amd64-amd64-libvirt-xsm  5 xen-install   fail REGR. vs. 94800
 test-amd64-i386-qemuu-rhel6hvm-intel 10 guest-stopfail REGR. vs. 94800
 test-amd64-i386-xl-qemuu-debianhvm-amd64 17 guest-start/debianhvm.repeat fail 
REGR. vs. 94800
 test-armhf-armhf-xl-xsm  15 guest-start/debian.repeat fail REGR. vs. 94800

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop fail REGR. vs. 94800
 test-armhf-armhf-xl-rtds 16 guest-start.2fail   like 94800

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass

version targeted for testing:
 qemuud145386f52950c0c5d4587dbb6c3b9cdf3a58309
baseline version:
 qemuu44a072f0de0d57c95c2212bbce0232b7b74f

Last test of basis94800  2016-05-26 17:40:42 Z   63 days
Testing same since99728  2016-07-27 18:41:26 Z1 days1 attempts


People who touched revisions under test:
  P J P 
  Stefan Hajnoczi 
  Stefano Stabellini 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl  pass
 test-armhf-armhf-xl  pass
 test-amd64-i386-xl   pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   

[Xen-devel] [qemu-upstream-4.6-testing baseline-only test] 66859: regressions - trouble: blocked/broken/fail/pass

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 66859 qemu-upstream-4.6-testing real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/66859/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-i386-xsm3 host-install(3) broken REGR. vs. 66491
 test-amd64-amd64-xl-qemuu-winxpsp3  3 host-install(3)   broken REGR. vs. 66491
 test-amd64-amd64-xl-qcow2 3 host-install(3) broken REGR. vs. 66491
 test-amd64-amd64-qemuu-nested-amd  3 host-install(3)broken REGR. vs. 66491
 build-i3863 host-install(3) broken REGR. vs. 66491
 build-i386-pvops  3 host-install(3) broken REGR. vs. 66491
 test-amd64-amd64-xl-multivcpu  3 host-install(3)broken REGR. vs. 66491
 test-amd64-amd64-libvirt-xsm  3 host-install(3) broken REGR. vs. 66491
 test-amd64-amd64-pair4 host-install/dst_host(4) broken REGR. vs. 66491
 test-amd64-amd64-libvirt-pair 4 host-install/dst_host(4) broken REGR. vs. 66491
 test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail REGR. vs. 
66491

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-xsm  11 guest-start  fail blocked in 66491

Tests which did not succeed, but are not blocking:
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1  1 build-check(1) blocked n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-winxpsp3  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-midway   13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-midway   12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail never pass

version targeted for testing:
 qemuuebfc90b51d09e0a3330a4702bb23223c

[Xen-devel] [XTF PATCH] xtf-runner: fix two synchronisation issues

[Wei Liu]

There were two synchronisation issues for the old code:

1. There was no guarantee that guest console was ready before "xl
   console" invocation.
2. There was no guarantee that runner wouldn't not exit before all test
   guests were gone.

For xtf-runner to be suitable to run in an automatic testing system like
osstest, we need to eliminate those two issues, otherwise we risk seeing
failures that are caused by races.

To fix these two issues:

1. Use "xl create -F" to force the process that creates a guest to
   stay until the guest is dead. With this we can wait for that
   particular subprocess in runner to make sure the guest is gone.
2. Poll xenstore guest console node to make sure console is available
   before "xl console" invocation.

To avoid the runner loops forever in that console checking loop, use
SIGALRM to make it time out after 5 seconds.

Redirect output that we're not interested in to DEVNULL.

Signed-off-by: Wei Liu 
---
 xtf-runner | 57 ++---
 1 file changed, 50 insertions(+), 7 deletions(-)

diff --git a/xtf-runner b/xtf-runner
index 66b2fb7..3296cdf 100755
--- a/xtf-runner
+++ b/xtf-runner
@@ -7,7 +7,7 @@ xtf-runner - A utility for enumerating and running XTF tests.
 Currently assumes the presence and availability of the `xl` toolstack.
 """
 
-import sys, os, os.path as path
+import sys, os, os.path as path, signal
 
 from optparse import OptionParser
 from subprocess import Popen, PIPE, call as subproc_call, check_output
@@ -17,6 +17,11 @@ try:
 except ImportError:
 import simplejson as json
 
+try:
+from subprocess import DEVNULL
+except ImportError:
+DEVNULL = open(os.devnull, 'wb')
+
 # All results of a test, keep in sync with C code report.h.
 # Note that warning is not a result on its own.
 all_results = ('SUCCESS', 'SKIP', 'ERROR', 'FAILURE')
@@ -41,6 +46,10 @@ all_environments = pv_environments + hvm_environments
 class RunnerError(Exception):
 """ Errors relating to xtf-runner itself """
 
+def sigalrm_handler(signum, frame):
+""" Signal handler for SIGALRM """
+raise RunnerError("Operation timed out")
+
 def open_test_info():
 """ Open and collate each test-info.json """
 
@@ -132,24 +141,53 @@ def list_tests(args):
 
 print name
 
+# A list of processes that we need to wait until they exits.
+wait_list = []
 
 def run_test(test):
 """ Run a specific test """
 
+console_path = '/local/domain/0/backend/console'
+# Setup watch for console
+cmd = ['xenstore-watch', console_path]
+print "Executing '%s'" % (" ".join(cmd), )
+xs_watch = Popen(cmd, stdout = PIPE, stderr = PIPE)
+
 _, _, name = test.split('-', 2)
 
 cfg = path.join("tests", name, test + ".cfg")
 
-cmd = ['xl', 'create', '-p', cfg]
+cmd = ['xl', 'create', '-Fp', cfg]
 
 print "Executing '%s'" % (" ".join(cmd), )
-rc = subproc_call(cmd)
-if rc:
-raise RunnerError("Failed to create VM")
+wait = Popen(cmd, stdout = DEVNULL, stderr = DEVNULL)
+wait_list.append(wait)
+
+# Wait up to 5 seconds for console to show up
+signal.alarm(5)
+while True:
+l = xs_watch.stdout.readline()
+domid = l[len(console_path)+1:].split('/')[0]
+if domid == '': continue
+
+cmd = ['xenstore-read', '/local/domain/'+domid+'/name']
+print "Executing '%s'" % (" ".join(cmd), )
+gname = check_output(cmd).splitlines()[0]
+if gname != test: continue
+
+cmd = ['xenstore-read', '/local/domain/'+domid+'/console/tty']
+print "Executing '%s'" % (" ".join(cmd), )
+con = check_output(cmd).splitlines()[0]
+if con != '': break
+
+# Tear down watch and timer
+signal.alarm(0)
+xs_watch.kill()
 
 cmd = ['xl', 'console', test]
 print "Executing '%s'" % (" ".join(cmd), )
 console = Popen(cmd, stdout = PIPE)
+wait_list.append(console)
 
 cmd = ['xl', 'unpause', test]
 print "Executing '%s'" % (" ".join(cmd), )
@@ -327,12 +365,17 @@ def main():
 opts, args = parser.parse_args()
 
 if opts.list_tests:
-return list_tests(args)
+ret = list_tests(args)
 else:
-return run_tests(args)
+ret = run_tests(args)
+
+for child in wait_list: child.wait()
+
+return ret
 
 
 if __name__ == "__main__":
+signal.signal(signal.SIGALRM, sigalrm_handler)
 try:
 sys.exit(main())
 except RunnerError, e:
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Kernel panic on Xen virtualisation in Debian

[Andreas Ziegler]

Hello Wei,

we tried with kernel 4.6 now, the crashed happened again, though.

next we want to try the Xen debug build, but we couldn't find any
information on how to enable debug for the build, perhaps you could give
us a hint.

- Andreas


 Original-Nachricht 
Betreff: Re: [Xen-devel] Kernel panic on Xen virtualisation in Debian
Von: Wei Liu 
An: Ingo Jürgensmann 
Datum: 25.7.2016, 15:13:06

> On Mon, Jul 25, 2016 at 01:41:41PM +0200, Ingo Jürgensmann wrote:
>> On 25.07.2016 12:23, Wei Liu wrote:
>>
>> First, thank you for replying! Very much appreciated! :)
>>
>>> I did skim your emails. But the oops was happening in memcpy+0x6 which
>>> indicated it came back to the origin question why would it got an
>>> exception there.
>>>
>>> Just by staring at the code doesn't get me anywhere. Without a concrete
>>> reproduction of the issue, I'm afraid I can't provide more input here.
>>
>> Well, from my point of view, it happens quite often when accessing the
>> server via SSH. For example today it crashed when I wanted to add something
>> and after I clicked into putty and typed the first char. In another putty,
>> where I have my netconsole log open, I instantly saw the oops.
>>
>> But what exactly causing these kinds of reboots, I'm clueless as you too.
>> Only that I do experience far more frequent crashes when accessing the
>> server from workplace via putty on Windows than via SSH on OSX or Debian
>> Linux.
>>
>>> There are several moving parts:
>>> 0. Hardware
>>> 1. Xen hypervisor
>>> 2. Dom0 kernel
>>> Your report and the debian report suggested that Dom0 kernel is less
>>> likely to be the culprit because you've tried different Dom0 kernels.
>>
>> As just written in the other mail, I already tried kernel 4.5 from
>> backports. Still crashing.
>>
>>> As for Xen, not sure if you would be up for trying a debug build from
>>> source tree. That would help provide information on whether this is a
>>> bug in Xen or not.
>>
>> Will try to build from Debian source, but how to enable debug build?
>>
> 
> I was thinking about building directly from xen.git.
> 
> http://wiki.xenproject.org/wiki/Compiling_Xen_From_Source
> 
> Probably try the Xen 4.7 release.
> 
> Wei.
> 
>> -- 
>> Ciao...  //http://blog.windfluechter.net
>>   Ingo \X/ XMPP: i...@jabber.windfluechter.net
>>
>>
>> gpg pubkey:  http://www.juergensmann.de/ij_public_key.asc

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [qemu-upstream-4.3-testing baseline-only test] 66861: tolerable trouble: blocked/broken/fail

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 66861 qemu-upstream-4.3-testing real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/66861/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 build-amd64-pvops 3 host-install(3)   broken baseline untested
 build-amd64   3 host-install(3)   broken baseline untested
 build-i386-pvops  5 kernel-buildfail baseline untested
 build-i3865 xen-build   fail baseline untested

Tests which did not succeed, but are not blocking:
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-winxpsp3  1 build-check(1)   blocked n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked  n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-pv   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1  1 build-check(1) blocked n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 test-amd64-i386-pv1 build-check(1)   blocked  n/a

version targeted for testing:
 qemuu22f4cb04e3660b6be75ee9b6a4a8f8cfe8f8070f
baseline version:
 qemuu12e8fccf5b5460be7aecddc71d27eceaba6e1f15

Last test of basis66487  2016-07-01 10:50:18 Z   28 days
Testing same since66861  2016-07-29 10:14:36 Z0 days1 attempts


People who touched revisions under test:
  P J P 
  Stefan Hajnoczi 
  Stefano Stabellini 

jobs:
 build-amd64  broken  
 build-i386   fail
 build-amd64-libvirt  blocked 
 build-i386-libvirt   blocked 
 build-amd64-pvopsbroken  
 build-i386-pvops fail
 test-amd64-amd64-xl  blocked 
 test-amd64-i386-xl   blocked 
 test-amd64-i386-qemuu-rhel6hvm-amd   blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64blocked 
 test-amd64-i386-xl-qemuu-debianhvm-amd64 blocked 
 test-amd64-i386-freebsd10-amd64  blocked 
 test-amd64-amd64-xl-qemuu-ovmf-amd64 blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64  blocked 
 test-amd64-amd64-xl-qemuu-win7-amd64 blocked 
 test-amd64-i386-xl-qemuu-win7-amd64  blocked 
 test-amd64-amd64-xl-credit2  blocked 
 test-amd64-i386-freebsd10-i386   blocked 
 test-amd64-i386-qemuu-rhel6hvm-intel blocked 
 test-amd64-amd64-libvirt blocked 
 test-amd64-i386-libvirt  blocked 
 test-amd64-amd64-xl-multivcpublocked 
 test-amd64-amd64-pair 

[Xen-devel] [PATCH 2/2] xen: drain submit queue in xen-usb before removing device

[Juergen Gross]

When unplugging a device in the Xen pvusb backend drain the submit
queue before deallocation of the control structures. Otherwise there
will be bogus memory accesses when I/O contracts are finished.

Correlated to this issue is the handling of cancel requests: a packet
cancelled will still lead to the call of complete, so add a flag
to the request indicating it should be just dropped on complete.

Signed-off-by: Juergen Gross 
---
 hw/usb/xen-usb.c | 93 
 1 file changed, 60 insertions(+), 33 deletions(-)

diff --git a/hw/usb/xen-usb.c b/hw/usb/xen-usb.c
index 7992456..6ad666e 100644
--- a/hw/usb/xen-usb.c
+++ b/hw/usb/xen-usb.c
@@ -90,6 +90,8 @@ struct usbback_req {
 void *buffer;
 void *isoc_buffer;
 struct libusb_transfer   *xfer;
+
+bool cancelled;
 };
 
 struct usbback_hotplug {
@@ -301,20 +303,23 @@ static void usbback_do_response(struct usbback_req 
*usbback_req, int32_t status,
 usbback_req->isoc_buffer = NULL;
 }
 
-res = RING_GET_RESPONSE(&usbif->urb_ring, usbif->urb_ring.rsp_prod_pvt);
-res->id = usbback_req->req.id;
-res->status = status;
-res->actual_length = actual_length;
-res->error_count = error_count;
-res->start_frame = 0;
-usbif->urb_ring.rsp_prod_pvt++;
-RING_PUSH_RESPONSES_AND_CHECK_NOTIFY(&usbif->urb_ring, notify);
+if (usbif->urb_sring) {
+res = RING_GET_RESPONSE(&usbif->urb_ring, 
usbif->urb_ring.rsp_prod_pvt);
+res->id = usbback_req->req.id;
+res->status = status;
+res->actual_length = actual_length;
+res->error_count = error_count;
+res->start_frame = 0;
+usbif->urb_ring.rsp_prod_pvt++;
+RING_PUSH_RESPONSES_AND_CHECK_NOTIFY(&usbif->urb_ring, notify);
 
-if (notify) {
-xen_be_send_notify(xendev);
+if (notify) {
+xen_be_send_notify(xendev);
+}
 }
 
-usbback_put_req(usbback_req);
+if (!usbback_req->cancelled)
+usbback_put_req(usbback_req);
 }
 
 static void usbback_do_response_ret(struct usbback_req *usbback_req,
@@ -366,15 +371,14 @@ static void usbback_set_address(struct usbback_info 
*usbif,
 }
 }
 
-static bool usbback_cancel_req(struct usbback_req *usbback_req)
+static void usbback_cancel_req(struct usbback_req *usbback_req)
 {
-bool ret = false;
-
 if (usb_packet_is_inflight(&usbback_req->packet)) {
 usb_cancel_packet(&usbback_req->packet);
-ret = true;
+QTAILQ_REMOVE(&usbback_req->stub->submit_q, usbback_req, q);
+usbback_req->cancelled = true;
+usbback_do_response_ret(usbback_req, -EPROTO);
 }
-return ret;
 }
 
 static void usbback_process_unlink_req(struct usbback_req *usbback_req)
@@ -391,7 +395,7 @@ static void usbback_process_unlink_req(struct usbback_req 
*usbback_req)
 devnum = usbif_pipedevice(usbback_req->req.pipe);
 if (unlikely(devnum == 0)) {
 usbback_req->stub = usbif->ports +
-usbif_pipeportnum(usbback_req->req.pipe);
+usbif_pipeportnum(usbback_req->req.pipe) - 1;
 if (unlikely(!usbback_req->stub)) {
 ret = -ENODEV;
 goto fail_response;
@@ -406,9 +410,7 @@ static void usbback_process_unlink_req(struct usbback_req 
*usbback_req)
 
 QTAILQ_FOREACH(unlink_req, &usbback_req->stub->submit_q, q) {
 if (unlink_req->req.id == id) {
-if (usbback_cancel_req(unlink_req)) {
-usbback_do_response_ret(unlink_req, -EPROTO);
-}
+usbback_cancel_req(unlink_req);
 break;
 }
 }
@@ -681,6 +683,31 @@ static void usbback_hotplug_enq(struct usbback_info 
*usbif, unsigned port)
 usbback_hotplug_notify(usbif);
 }
 
+static void usbback_portid_drain(struct usbback_info *usbif, unsigned port)
+{
+struct usbback_req *req, *tmp;
+bool sched = false;
+
+QTAILQ_FOREACH_SAFE(req, &usbif->ports[port - 1].submit_q, q, tmp) {
+usbback_cancel_req(req);
+sched = true;
+}
+
+if (sched)
+qemu_bh_schedule(usbif->bh);
+}
+
+static void usbback_portid_detach(struct usbback_info *usbif, unsigned port)
+{
+if (!usbif->ports[port - 1].attached)
+return;
+
+usbif->ports[port - 1].speed = USBIF_SPEED_NONE;
+usbif->ports[port - 1].attached = false;
+usbback_portid_drain(usbif, port);
+usbback_hotplug_enq(usbif, port);
+}
+
 static void usbback_portid_remove(struct usbback_info *usbif, unsigned port)
 {
 USBPort *p;
@@ -694,9 +721,7 @@ static void usbback_portid_remove(struct usbback_info 
*usbif, unsigned port)
 
 object_unparent(OBJECT(usbif->ports[port - 1].dev));
 usbif->ports[port - 1].dev = NULL;
-usbif->ports[port - 1].speed = USBIF_SPEED_NONE;
-usbif->ports[port - 1].attached = false;
-usbback_hotplug_enq(usbif, port);
+usbback_portid_detach(usbif, port);
 

[Xen-devel] [PATCH 0/2] xen: bug fixes in Xen backend handling

[Juergen Gross]

When testing qemu based pvusb backend two bugs have been discovered:
- detaching of a usb controller leads to memory clobbering in qemu
- detaching of a usb device with active I/O requests could result in
  crash of qemu

Juergen Gross (2):
  xen: when removing a backend don't remove many of them
  xen: drain submit queue in xen-usb before removing device

 hw/usb/xen-usb.c | 93 +---
 hw/xen/xen_backend.c | 58 +++-
 2 files changed, 79 insertions(+), 72 deletions(-)

-- 
2.6.6


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 1/2] xen: when removing a backend don't remove many of them

[Juergen Gross]

When a Xenstore watch fires indicating a backend has to be removed
don't remove all backends for that domain with the specified device
index, but just the one which has the correct type.

The easiest way to achieve this is to use the already determined
xendev as parameter for xen_be_del_xendev() instead of only the domid
and device index.

This at once removes the open coded QTAILQ_FOREACH_SAVE() in
xen_be_del_xendev() as there is no need to search for the correct
xendev any longer.

Signed-off-by: Juergen Gross 
---
 hw/xen/xen_backend.c | 58 +---
 1 file changed, 19 insertions(+), 39 deletions(-)

diff --git a/hw/xen/xen_backend.c b/hw/xen/xen_backend.c
index bab79b1..3ceb778 100644
--- a/hw/xen/xen_backend.c
+++ b/hw/xen/xen_backend.c
@@ -321,48 +321,28 @@ static struct XenDevice *xen_be_get_xendev(const char 
*type, int dom, int dev,
 /*
  * release xen backend device.
  */
-static struct XenDevice *xen_be_del_xendev(int dom, int dev)
+static void xen_be_del_xendev(struct XenDevice *xendev)
 {
-struct XenDevice *xendev, *xnext;
-
-/*
- * This is pretty much like QTAILQ_FOREACH(xendev, &xendevs, next) but
- * we save the next pointer in xnext because we might free xendev.
- */
-xnext = xendevs.tqh_first;
-while (xnext) {
-xendev = xnext;
-xnext = xendev->next.tqe_next;
-
-if (xendev->dom != dom) {
-continue;
-}
-if (xendev->dev != dev && dev != -1) {
-continue;
-}
-
-if (xendev->ops->free) {
-xendev->ops->free(xendev);
-}
-
-if (xendev->fe) {
-char token[XEN_BUFSIZE];
-snprintf(token, sizeof(token), "fe:%p", xendev);
-xs_unwatch(xenstore, xendev->fe, token);
-g_free(xendev->fe);
-}
+if (xendev->ops->free) {
+xendev->ops->free(xendev);
+}
 
-if (xendev->evtchndev != NULL) {
-xenevtchn_close(xendev->evtchndev);
-}
-if (xendev->gnttabdev != NULL) {
-xengnttab_close(xendev->gnttabdev);
-}
+if (xendev->fe) {
+char token[XEN_BUFSIZE];
+snprintf(token, sizeof(token), "fe:%p", xendev);
+xs_unwatch(xenstore, xendev->fe, token);
+g_free(xendev->fe);
+}
 
-QTAILQ_REMOVE(&xendevs, xendev, next);
-g_free(xendev);
+if (xendev->evtchndev != NULL) {
+xenevtchn_close(xendev->evtchndev);
 }
-return NULL;
+if (xendev->gnttabdev != NULL) {
+xengnttab_close(xendev->gnttabdev);
+}
+
+QTAILQ_REMOVE(&xendevs, xendev, next);
+g_free(xendev);
 }
 
 /*
@@ -682,7 +662,7 @@ static void xenstore_update_be(char *watch, char *type, int 
dom,
 if (xendev != NULL) {
 bepath = xs_read(xenstore, 0, xendev->be, &len);
 if (bepath == NULL) {
-xen_be_del_xendev(dom, dev);
+xen_be_del_xendev(xendev);
 } else {
 free(bepath);
 xen_be_backend_changed(xendev, path);
-- 
2.6.6


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] live migrating hvm from 4.4 to 4.7 fails in ioreq server

[Paul Durrant]

> -Original Message-
> From: Paul Durrant
> Sent: 29 July 2016 11:12
> To: Paul Durrant; Olaf Hering
> Cc: Stefano Stabellini; Wei Liu; Andrew Cooper; xen-devel@lists.xen.org;
> Anthony Perard; zhigang.x.w...@oracle.com
> Subject: RE: [Xen-devel] live migrating hvm from 4.4 to 4.7 fails in ioreq
> server
> 
> > -Original Message-
> > From: Xen-devel [mailto:xen-devel-boun...@lists.xen.org] On Behalf Of
> > Paul Durrant
> > Sent: 26 July 2016 16:49
> > To: Olaf Hering
> > Cc: Stefano Stabellini; Wei Liu; Andrew Cooper; xen-devel@lists.xen.org;
> > Anthony Perard; zhigang.x.w...@oracle.com
> > Subject: Re: [Xen-devel] live migrating hvm from 4.4 to 4.7 fails in ioreq
> > server
> >
> > > -Original Message-
> > > From: Olaf Hering [mailto:o...@aepfle.de]
> > > Sent: 26 July 2016 16:45
> > > To: Paul Durrant
> > > Cc: Konrad Rzeszutek Wilk; zhigang.x.w...@oracle.com; Wei Liu; Stefano
> > > Stabellini; Andrew Cooper; xen-devel@lists.xen.org; Anthony Perard
> > > Subject: Re: [Xen-devel] live migrating hvm from 4.4 to 4.7 fails in ioreq
> > > server
> > >
> > > On Thu, May 26, Paul Durrant wrote:
> > >
> > > > It's likely to be a while before I could find some time for this;
> > > > rough guess would be a month... It depends how other stuff pans out.
> > >
> > > Any news, Paul? Did you have a chance to compose a fix?
> > >
> >
> > Nope, not yet. I may get some time this week now that other stuff has died
> > down.
> 
> Olaf,
> 
>   Could you give the attached patch a try? I believe it should solve the
> problem.
>

For some reason the attached patch was a previous version that did not compile. 
Here's the one I actually tested...

  Paul

 
>   Paul
> 
> >
> >   Paul
> >
> > > Olaf
> > ___
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > https://lists.xen.org/xen-devel


0001-xen-handle-inbound-migration-of-VMs-without-ioreq-se.patch
Description: 0001-xen-handle-inbound-migration-of-VMs-without-ioreq-se.patch
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [qemu-upstream-4.3-testing test] 99727: tolerable FAIL - PUSHED

[osstest service owner]

flight 99727 qemu-upstream-4.3-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/99727/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail like 96374
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop  fail like 96374

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-ovmf-amd64  9 debian-hvm-install fail never pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  9 debian-hvm-install  fail never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass

version targeted for testing:
 qemuu22f4cb04e3660b6be75ee9b6a4a8f8cfe8f8070f
baseline version:
 qemuu12e8fccf5b5460be7aecddc71d27eceaba6e1f15

Last test of basis96374  2016-06-29 11:49:22 Z   29 days
Testing same since99727  2016-07-27 18:40:24 Z1 days1 attempts


People who touched revisions under test:
  P J P 
  Stefan Hajnoczi 
  Stefano Stabellini 

jobs:
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl  pass
 test-amd64-i386-xl   pass
 test-amd64-i386-qemuu-rhel6hvm-amd   pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-i386-xl-qemuu-debianhvm-amd64 pass
 test-amd64-i386-freebsd10-amd64  pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 fail
 test-amd64-i386-xl-qemuu-ovmf-amd64  fail
 test-amd64-amd64-xl-qemuu-win7-amd64 fail
 test-amd64-i386-xl-qemuu-win7-amd64  fail
 test-amd64-amd64-xl-credit2  pass
 test-amd64-i386-freebsd10-i386   pass
 test-amd64-i386-qemuu-rhel6hvm-intel pass
 test-amd64-amd64-libvirt pass
 test-amd64-i386-libvirt  pass
 test-amd64-amd64-xl-multivcpupass
 test-amd64-amd64-pairpass
 test-amd64-i386-pair pass
 test-amd64-amd64-pv  pass
 test-amd64-i386-pv   pass
 test-amd64-amd64-amd64-pvgrubpass
 test-amd64-amd64-i386-pvgrub pass
 test-amd64-amd64-pygrub  pass
 test-amd64-amd64-xl-qcow2pass
 test-amd64-i386-xl-raw   pass
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 pass
 test-amd64-amd64-libvirt-vhd pass
 test-amd64-amd64-xl-qemuu-winxpsp3   pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=qemu-upstream-4.3-testing
+ revision=22f4cb04e3660b6be75ee9b6a4a8f8cfe8f8070f
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push 
qemu-upst

Re: [Xen-devel] live migrating hvm from 4.4 to 4.7 fails in ioreq server

[Paul Durrant]

> -Original Message-
> From: Xen-devel [mailto:xen-devel-boun...@lists.xen.org] On Behalf Of
> Paul Durrant
> Sent: 26 July 2016 16:49
> To: Olaf Hering
> Cc: Stefano Stabellini; Wei Liu; Andrew Cooper; xen-devel@lists.xen.org;
> Anthony Perard; zhigang.x.w...@oracle.com
> Subject: Re: [Xen-devel] live migrating hvm from 4.4 to 4.7 fails in ioreq
> server
> 
> > -Original Message-
> > From: Olaf Hering [mailto:o...@aepfle.de]
> > Sent: 26 July 2016 16:45
> > To: Paul Durrant
> > Cc: Konrad Rzeszutek Wilk; zhigang.x.w...@oracle.com; Wei Liu; Stefano
> > Stabellini; Andrew Cooper; xen-devel@lists.xen.org; Anthony Perard
> > Subject: Re: [Xen-devel] live migrating hvm from 4.4 to 4.7 fails in ioreq
> > server
> >
> > On Thu, May 26, Paul Durrant wrote:
> >
> > > It's likely to be a while before I could find some time for this;
> > > rough guess would be a month... It depends how other stuff pans out.
> >
> > Any news, Paul? Did you have a chance to compose a fix?
> >
> 
> Nope, not yet. I may get some time this week now that other stuff has died
> down.

Olaf,

  Could you give the attached patch a try? I believe it should solve the 
problem.

  Paul

> 
>   Paul
> 
> > Olaf
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel


0001-xen-handle-inbound-migration-of-VMs-without-ioreq-se.patch
Description: 0001-xen-handle-inbound-migration-of-VMs-without-ioreq-se.patch
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v3 07/13] tables.h: add linker table support

[Borislav Petkov]

On Fri, Jul 22, 2016 at 02:24:41PM -0700, Luis R. Rodriguez wrote:
> A linker table is a data structure that is stitched together from items
> in multiple object files. Linux has historically implicitly used linker
> tables for ages, however they were all built in an adhoc manner which
> requires linker script modifications, per architecture. This adds a
> general linker table solution so that a new linker table can be
> implemented by changing C code only. The Linux linker table was
> originally based on Michael Brown's iPXE's linker table solution but
> has been significantly modified to fit Linux's use in its integration.

...

> diff --git a/include/asm-generic/tables.h b/include/asm-generic/tables.h
> new file mode 100644
> index ..5cf655590a19
> --- /dev/null
> +++ b/include/asm-generic/tables.h
> @@ -0,0 +1,70 @@
> +#ifndef _ASM_GENERIC_TABLES_H_
> +#define _ASM_GENERIC_TABLES_H_
> +/*
> + * Linux linker tables
> + *
> + * Copyright (C) 2016 Luis R. Rodriguez 
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of copyleft-next (version 0.3.1 or later) as published
> + * at http://copyleft-next.org/.
> + */
> +
> +#ifdef __KERNEL__
> +# include 
> +#endif /* __KERNEL__ */
> +
> +#define SECTION_TYPE_TABLES  tbl

What is that for? Section type? Do we have more types or is it useless
and can go?

> +
> +#define SECTION_TBL(section, name, level)\
> + SECTION_TYPE(section, SECTION_TYPE_TABLES, name, level)
> +
> +#define SECTION_TBL_ALL(section) \
> + SECTION_TYPE_ALL(section,SECTION_TYPE_TABLES)
> +
> +#ifndef section_tbl
> +# define section_tbl(section, name, level, flags)\
> +  section_type(section, SECTION_TYPE_TABLES, name,   \
> +  level, flags)

That section_type macro is actually saying the following code should
be in it. Can we make its name have a verb, i.e., something like
"set_section" to make it more clear?

Also, that type SECTION_TYPE_TABLES looks redundant to me but it
might start making more sense after I've gone through the rest of the
series...

> +#endif
> +
> +#ifndef section_tbl_any
> +# define section_tbl_any(section, name, flags)   
> \
> +  section_type(section, SECTION_TYPE_TABLES, name,   \
> +  SECTION_ORDER_ANY, flags)
> +#endif
> +
> +#ifndef section_tbl_asmtype
> +# define section_tbl_asmtype(section, name, level, flags, asmtype)   \

And here's the confusion: there's "asmtype" but there's also
SECTION_TYPE_TABLES.

That asmtype is the *actual* section type in gas speak, i.e., @progbits,
@note, etc.

Now *that* should be your type. The SECTION_TYPE_TABLES should be called
something else or completely gone.

> +  section_type_asmtype(section, SECTION_TYPE_TABLES, name,   \
> +  level, flags, asmtype)
> +#endif
> +
> +#ifndef push_section_tbl
> +# define push_section_tbl(section, name, level, flags)   
> \
> +  push_section_type(section, SECTION_TYPE_TABLES, name,  \
> +   level, flags)
> +#endif
> +
> +#ifndef push_section_tbl_any
> +# define push_section_tbl_any(section, name, flags)  \
> +  push_section_type(section, SECTION_TYPE_TABLES, name,  \
> +   SECTION_ORDER_ANY, flags)
> +#endif
> +
> +#if defined(__ASSEMBLER__) || defined(__ASSEMBLY__)
> +
> +# ifndef DECLARE_SECTION_TBL
> +#  define DECLARE_SECTION_TBL(section, name) \
> +  push_section_tbl(section, name,,) ;
> \
> +  .globl name ;  
> \
> +name: ;  
> \
> +  .popsection
> \
> + \
> +  push_section_tbl(section, name, ~,) ;  
> \
> +  .popsection
> +# endif

#endif /* DECLARE_SECTION_TBL */

Btw, what does that macro do? I don't see it used anywhere.

> +
> +#endif /* defined(__ASSEMBLER__) || defined(__ASSEMBLY__) */
> +
> +#endif /* _ASM_GENERIC_TABLES_H_ */


...


> diff --git a/include/linux/tables.h b/include/linux/tables.h
> new file mode 100644
> index ..89d46f4c5739
> --- /dev/null
> +++ b/include/linux/tables.h
> @@ -0,0 +1,597 @@
> +#ifndef _LINUX_LINKER_TABLES_H
> +#define _LINUX_LINKER_TABLES_H
> +/*
> + * Linux linker tables
> + *
> + * Copyright (C) 2016 Luis R. Rodriguez 
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of copyleft-next (version 0.3.1 or later) as published
> + * at http://copyleft-next.org/.
> + */
> +#include 
> +#ifdef __KERNEL__
> +# include 
> +#endif /* __KERNEL__ */
> +
> +#ifn

Re: [Xen-devel] Xen 4.6.1 crash with altp2m enabled by default

[Andrew Cooper]

On 29/07/16 08:33, kevin.ma...@gdata.de wrote:
>
> Hi guys
>
>  
>
> We are using Xen 4.6.1 to manage our virtual machines on x86-64-servers.
>
> We start dozens of VMs and destroy them again after 60 seconds, which
> works fine as it is, but the next step in our approach requires the
> use of the altp2m functionality.
>
> Since libvirt does not pass the altp2m-enable flag to the hypervisor
> we enabled altp2m unconditionally by patching the hvm.c . Since all of
> our machines support the altp2m this seemed to be ok.
>

altp2m is emulated in software when hardware support isn't available, so
it should work on all hardware (albeit with rather higher overhead).

>  
>
>  d->arch.hvm_domain.params[HVM_PARAM_HPET_ENABLED] = 1;
>
>  d->arch.hvm_domain.params[HVM_PARAM_TRIPLE_FAULT_REASON] =
> SHUTDOWN_reboot;
>
> +d->arch.hvm_domain.params[HVM_PARAM_ALTP2M] = 1;
>
> +
>

This looks to be ok, given your situation.

>  vpic_init(d);
>
>  rc = vioapic_init(d);
>
>  
>
> Since applying this patch the hypervisor crashes after several hundred
> restarted VMs (without any altp2m-functionality used by us) with the
> following dmesg:
>
>  
>
> (XEN) [ Xen-4.6.1  x86_64  debug=n  Not tainted ]
>

As a start, please always use a debug hypervisor for investigating
issues like this.

> (XEN) CPU:7
>
> (XEN) RIP:e008:[] vmx_vmenter_helper+0x2b5/0x340
>
> (XEN) RFLAGS: 00010003   CONTEXT: hypervisor (d0v3)
>
> (XEN) rax: 8005003b   rbx: 8300e7038000   rcx:
> 0008
>
> (XEN) rdx: 6c00   rsi: 83062eb5e000   rdi:
> 8300e7038000
>
> (XEN) rbp: 830c17e3f000   rsp: 830617fc7d70   r8: 
> 
>
> (XEN) r9:  83014f8d7028   r10: 02700f858000   r11:
> 2201be6861f0
>
> (XEN) r12: 83062eb5e000   r13: 8300e752f000   r14:
> 82d08030ea40
>
> (XEN) r15: 0007   cr0: 8005003b   cr4:
> 26e0
>
> (XEN) cr3: 0001bf4da000   cr2: dd840c00
>
> (XEN) ds:    es:    fs:    gs:    ss:    cs: e008
>
> (XEN) Xen stack trace from rsp=830617fc7d70:
>
> (XEN)8300e7038000 82d080170c04 
> 000780109f6a
>
> (XEN)830617fc7f18 831e 
> 8300e752f19c
>
> (XEN)0286 8300e752f000 8300e72fc000
> 0007
>
> (XEN)830c17e3f000 830c14ee1000 82d08030ea40
> 82d080173d6a
>
> (XEN)  
> 
>
> (XEN)82d08030ea40 8300e72fc000 02700f481091
> 0001
>
> (XEN)82d080324560 82d08030ea40 8300e752f000
> 82d080128004
>
> (XEN)0001 01c9c380 830c14ef60e8
> 17fce600
>
> (XEN)0001 82d0801bd18b 82d0801d9e88
> 8300e752f000
>
> (XEN)01c9c380 82d08012e700 006e0171
> 
>
> (XEN)830617fc 82d0802f8f80 
> 83062eb5e000
>
> (XEN)82d08030ea40 82d08012b040 8300e7038000
> 830617fc
>
> (XEN)8300e7038000  830c14ee1000
> 82d080170970
>
> (XEN)8300e72fc000  
> 
>
> (XEN) 80550f50 ffdffc70
> 
>
> (XEN)  
> 2fcffe19
>
> (XEN)ffdffc70  ffdffc50
> 853b0918
>
> (XEN)00fa f0e48162 
> 0246
>
> (XEN)80550f34  
> 
>
> (XEN)  0007
> 8300e752f000
>
> (XEN) Xen call trace:
>
> (XEN)[] vmx_vmenter_helper+0x2b5/0x340
>
> (XEN)[] __context_switch+0xb4/0x350
>
> (XEN)[] context_switch+0xca/0xef0
>
> (XEN)[] schedule+0x264/0x5f0
>
> (XEN)[] mwait_idle+0x25b/0x3a0
>
> (XEN)[] hvm_vcpu_has_pending_irq+0x58/0xc0
>
> (XEN)[] timer_softirq_action+0x80/0x250
>
> (XEN)[] __do_softirq+0x60/0x90
>
> (XEN)[] idle_loop+0x20/0x50
>
> (XEN)
>
> (XEN)
>
> (XEN) 
>
> (XEN) Panic on CPU 7:
>
> (XEN) FATAL TRAP: vector = 6 (invalid opcode)
>
> (XEN) 
>
> (XEN)
>
> (XEN) Reboot in five seconds...
>
> (XEN) Executing kexec image on cpu7
>
> (XEN) Shot down all CPUs
>
>  
>
> The RIP points to ud2
>
> 0x82d0801f5a55:  ud2
>
> From the RFLAGS we concluded that the vmwrite failed due to an invalid
> vmcs-pointer (CF = 1), but this is where we are stuck since we have no
> idea how the pointer could have gotten corrupted.
>
> crash> vcpu
>
> gives vmcs = 0x817cbc20 for vcpu_id = 7,
>
>  
>
> and vcpus gives
>
>  
>
>VCID  PCID   VCPU   ST T DOMID  DOMAIN
>
>   0 0 8300e75f2000 RU I 32767 830c14ee1000
>
>  

Re: [Xen-devel] Xen 4.6.1 crash with altp2m enabled by default

[Dario Faggioli]

On Fri, 2016-07-29 at 07:33 +, kevin.ma...@gdata.de wrote:
> Hi guys
>  
Hi,

I'm pretty much just Cc-ing maintainers/key people, to see if they have
ideas.

Only one thing. Since you are rebuilding Xen anyway, I think it could
be helpful to try a debug build, and post the dump it will produce.

> (XEN) [ Xen-4.6.1  x86_64  debug=n  Not tainted ]
>
I.e., this would need to become debug=y.

Since you said you're using 4.6.x, I think putting "debug=y" in a
.config file (and then rebuilding and reinstalling, of course) should
be enough.

Regards,
Dario
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)



signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [qemu-upstream-4.4-testing baseline-only test] 66857: regressions - trouble: broken/fail/pass

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 66857 qemu-upstream-4.4-testing real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/66857/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-multivcpu  3 host-install(3)broken REGR. vs. 66488
 test-amd64-i386-pv3 host-install(3) broken REGR. vs. 66488
 test-amd64-amd64-xl-qemuu-debianhvm-amd64 3 host-install(3) broken REGR. vs. 
66488
 test-amd64-amd64-xl-qemuu-winxpsp3  3 host-install(3)   broken REGR. vs. 66488
 test-amd64-amd64-libvirt  3 host-install(3) broken REGR. vs. 66488
 test-amd64-amd64-xl-qemuu-win7-amd64  3 host-install(3) broken REGR. vs. 66488
 test-amd64-i386-libvirt-pair 4 host-install/dst_host(4) broken REGR. vs. 66488
 test-amd64-i386-pair 4 host-install/dst_host(4) broken REGR. vs. 66488
 test-amd64-amd64-pair3 host-install/src_host(3) broken REGR. vs. 66488
 test-amd64-amd64-libvirt-pair 3 host-install/src_host(3) broken REGR. vs. 66488
 test-amd64-i386-xl-qemuu-win7-amd64 15 guest-localmigrate/x10 fail REGR. vs. 
66488

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-pv  17 guest-localmigrate/x10   fail   like 66488
 test-amd64-amd64-xl-qcow221 leak-check/check fail   like 66488

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass

version targeted for testing:
 qemuue72488cdcf2208f2df334fa88c35b33e695fa93b
baseline version:
 qemuuca3d95113903ccffe52af3922bbf7bd14a9acc78

Last test of basis66488  2016-07-01 10:51:34 Z   27 days
Testing same since66857  2016-07-29 04:46:53 Z0 days1 attempts


People who touched revisions under test:
  P J P 
  Stefan Hajnoczi 
  Stefano Stabellini 

jobs:
 build-amd64-xend pass
 build-i386-xend  pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl  pass
 test-amd64-i386-xl   pass
 test-amd64-amd64-qemuu-nested-amdfail
 test-amd64-i386-qemuu-rhel6hvm-amd   pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64broken  
 test-amd64-i386-xl-qemuu-debianhvm-amd64 pass
 test-amd64-i386-freebsd10-amd64  pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass
 test-amd64-amd64-xl-qemuu-win7-amd64 broken  
 test-amd64-i386-xl-qemuu-win7-amd64  fail
 test-amd64-amd64-xl-credit2  pass
 test-amd64-i386-freebsd10-i386   pass
 test-amd64-amd64-qemuu-nested-intel  fail
 test-amd64-i386-qemuu-rhel6hvm-intel pass
 test-amd64-amd64-libvirt broken  
 test-amd64-i386-libvirt  pass
 test-amd64-amd64-xl-multivcpubroken  
 test-amd64-amd64-pairbroken  
 test-amd64-i386-pair broken  
 test-amd64-amd64-libvirt-pairbroken  
 test-amd64-i386-libvirt-pair broken  
 test-amd64-amd64-pv  fail
 test-amd64-i386-pv   broken  
 test-amd64-amd64-amd64-pvgrubpass
 test-amd64-amd64-i386-pvgrub pass
 test-amd64-amd64-pygrub  pass
 test-amd64-amd64-xl-qcow2fail
 test-amd64-i386-xl-raw   pass
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 pass
 test-amd64-amd64-libvirt-vhd pass
 test-amd