[Xen-devel] [xen-unstable-smoke test] 114318: regressions - FAIL

[osstest service owner]

flight 114318 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/114318/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl   7 xen-boot fail REGR. vs. 114299

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass

version targeted for testing:
 xen  f17d2cd2ffeda70aba8788910e9d088415562c8b
baseline version:
 xen  3b40cfcd1a1912c2e4c4eb353dc77cbf35c63c3a

Last test of basis   114299  2017-10-10 21:02:54 Z0 days
Failing since114308  2017-10-10 23:01:10 Z0 days2 attempts
Testing same since   114318  2017-10-11 02:19:34 Z0 days1 attempts


People who touched revisions under test:
  Andre Przywara 
  Julien Grall 
  Stefano Stabellini 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  fail
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.


commit f17d2cd2ffeda70aba8788910e9d088415562c8b
Author: Andre Przywara 
Date:   Sat Oct 7 01:06:40 2017 +0100

ARM: sunxi: support more Allwinner SoCs

So far we only supported the Allwinner A20 SoC. Add support for most
of the other virtualization capable Allwinner SoCs by:
- supporting the watchdog in newer (sun8i) SoCs
- getting the watchdog address from DT
- adding compatible strings for other 32-bit SoCs
- adding compatible strings for 64-bit SoCs

As all 64-bit SoCs support system reset via PSCI, we don't use the
platform specific reset routine there. Should the 32-bit SoCs start to
properly support the PSCI 0.2 SYSTEM_RESET call, we will use it for them
automatically, as we try PSCI first, then fall back to platform reset.

Signed-off-by: Andre Przywara 
Signed-off-by: Stefano Stabellini 
Reviewed-by: Stefano Stabellini 

commit e2dfe4a037b0c6ccfd2375e4b60668109a0118e5
Author: Julien Grall 
Date:   Mon Oct 9 14:23:41 2017 +0100

xen/arm: mm: Use memory flags for modify_xen_mappings rather than custom one

This will help to consolidate the page-table code and avoid different
path depending on the action to perform.

Signed-off-by: Julien Grall 
Reviewed-by: Andre Przywara 
Reviewed-by: Stefano Stabellini 
Reviewed-by: Konrad Rzeszutek Wilk 

commit 6b88beed40c756aaff018d286f4de31351240a93
Author: Julien Grall 
Date:   Mon Oct 9 14:23:40 2017 +0100

xen/arm: mm: Handle permission flags when adding a new mapping

Currently, all the new mappings will be read-write non-executable. Allow the
caller to use other permissions.

Signed-off-by: Julien Grall 
Reviewed-by: Stefano Stabellini 

commit 28f2ad440a08908010abec43b7ccc3283051e943
Author: Julien Grall 
Date:   Mon Oct 9 14:23:39 2017 +0100

xen/arm: mm: Embed permission in the flags

Currently, it is not possible to specify the permission of a new
mapping. It would be necessary to use the function modify_xen_mappings
with a different set of flags.

Introduce a couple of new flags for the permissions (Non-eXecutable,
Read-Only) and also provides definition that combine the memory attribute
and permission for common combinations.

PAGE_HYPERVISOR is now an alias to PAGE_HYPERVISOR_RW (read-write,
non-executable mappings). This does not affect the current mapping using
PAGE_HYPERVISOR because Xen is currently forcing all the mapping to be
non-executable by default (see mfn_to_xen_entry).

A 

[Xen-devel] [linux-3.18 bisection] complete test-amd64-amd64-xl-pvh-intel

[osstest service owner]

branch xen-unstable
xenbranch xen-unstable
job test-amd64-amd64-xl-pvh-intel
testid guest-start

Tree: linux 
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  xen git://xenbits.xen.org/xen.git
  Bug introduced:  c7dfe4ac58dd9c8678126b78da961b233a49f3f9
  Bug not present: 3c44f8ed44ab559c7e5b58316751bea53adfd83b
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/114323/


  commit c7dfe4ac58dd9c8678126b78da961b233a49f3f9
  Author: Roger Pau Monne 
  Date:   Fri Sep 22 16:25:06 2017 +0100
  
  xl: introduce a domain type option
  
  Introduce a new type option to xl configuration files in order to
  specify the domain type. This supersedes the current builder option.
  
  The new option is documented in the xl.cfg man page, and the previous
  builder option is marked as deprecated.
  
  Signed-off-by: Roger Pau Monné 
  Acked-by: Ian Jackson 


For bisection revision-tuple graph see:
   
http://logs.test-lab.xenproject.org/osstest/results/bisect/linux-3.18/test-amd64-amd64-xl-pvh-intel.guest-start.html
Revision IDs in each graph node refer, respectively, to the Trees above.


Running cs-bisection-step 
--graph-out=/home/logs/results/bisect/linux-3.18/test-amd64-amd64-xl-pvh-intel.guest-start
 --summary-out=tmp/114323.bisection-summary --basis-template=114034 
--blessings=real,real-bisect linux-3.18 test-amd64-amd64-xl-pvh-intel 
guest-start
Searching for failure / basis pass:
 114225 fail [host=chardonnay0] / 114034 [host=fiano1] 113869 [host=nobling1] 
113856 [host=huxelrebe1] 113503 [host=huxelrebe0] 113476 [host=fiano1] 113455 
[host=nobling0] 113424 [host=godello1] 113158 [host=baroque0] 113144 
[host=italia1] 112102 [host=huxelrebe0] 111920 [host=chardonnay1] 111893 
[host=godello1] 111867 [host=nobling1] 111839 [host=elbling0] 111812 
[host=nobling0] 111793 [host=godello0] 111770 [host=elbling1] 111745 
[host=italia1] 111724 [host=baroque0] 111706 ok.
Failure / basis pass flights: 114225 / 111706
(tree with no url: minios)
(tree with no url: ovmf)
(tree with no url: seabios)
Tree: linux 
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git
Latest ac0058305d83e8e50a9652a003bc2ec468df9f87 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
8051789e982499050680a26febeada7467e18a8d 
7434775abf8fb2ca3b9e805d30656f4da8c08816 
dbc4b6e13a5d0dd8967cde7ff7000ab1ed88625e
Basis pass 8686d7d2e93b977f4cc3cb28cd0c64dcfcc8a865 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
8051789e982499050680a26febeada7467e18a8d 
414d069b38ab114b89085e44989bf57604ea86d7 
d23afa6399a78ca7d0ed3294119632535828c9d8
Generating revisions with ./adhoc-revtuple-generator  
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git#8686d7d2e93b977f4cc3cb28cd0c64dcfcc8a865-ac0058305d83e8e50a9652a003bc2ec468df9f87
 
git://xenbits.xen.org/osstest/linux-firmware.git#c530a75c1e6a472b0eb9558310b518f0dfcd8860-c530a75c1e6a472b0eb9558310b518f0dfcd8860
 
git://xenbits.xen.org/qemu-xen-traditional.git#8051789e982499050680a26febeada7467e18a8d-8051789e982499050680a26febeada7467e18a8d
 
git://xenbits.xen.org/qemu-xen.git#414d069b38ab114b89085e44989bf57604ea86d7-7434775abf8fb2ca3b9e805d30656f4da8c08816
 
git://xenbits.xen.org/xen.git#d23afa6399a78ca7d0ed3294119632535828c9d8-dbc4b6e13a5d0dd8967cde7ff7000ab1ed88625e
adhoc-revtuple-generator: tree discontiguous: qemu-xen
Loaded 2002 nodes in revision graph
Searching for test results:
 111425 [host=baroque0]
 111491 [host=fiano1]
 111539 [host=huxelrebe0]
 111523 pass irrelevant
 111614 [host=godello1]
 111591 [host=godello0]
 111644 [host=baroque1]
 111673 [host=huxelrebe1]
 111655 [host=italia0]
 111706 pass 8686d7d2e93b977f4cc3cb28cd0c64dcfcc8a865 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
8051789e982499050680a26febeada7467e18a8d 
414d069b38ab114b89085e44989bf57604ea86d7 
d23afa6399a78ca7d0ed3294119632535828c9d8
 111724 [host=baroque0]
 111745 [host=italia1]
 111770 [host=elbling1]
 111793 [host=godello0]
 111812 [host=nobling0]
 111867 [host=nobling1]
 111839 [host=elbling0]
 111920 [host=chardonnay1]
 111893 [host=godello1]
 112102 [host=huxelrebe0]
 113144 [host=italia1]
 113158 [host=baroque0]
 113424 [host=godello1]
 113455 [host=nobling0]
 113476 [host=fiano1]
 113503 [host=huxelrebe0]
 113856 [host=huxelrebe1]
 113869 [host=nobling1]
 114034 [host=fiano1]
 114179 pass 8686d7d2e93b977f4cc3cb28cd0c64dcfcc8a865 

[Xen-devel] [qemu-upstream-unstable test] 114273: regressions - FAIL

[osstest service owner]

flight 114273 qemu-upstream-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/114273/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-pvh-intel 12 guest-start fail REGR. vs. 114029
 test-amd64-amd64-xl-pvh-amd  12 guest-start  fail REGR. vs. 114029

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds 10 debian-install   fail REGR. vs. 114029

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeatfail  like 114014
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 114029
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 114029
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 114029
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qcow2 12 migrate-support-checkfail  never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 13 guest-saverestore   fail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass

version targeted for testing:
 qemuu5cd7ce5dde3f228b3b669ed9ca432f588947bd40
baseline version:
 qemuu7434775abf8fb2ca3b9e805d30656f4da8c08816

Last test of basis   114029  2017-10-05 04:17:58 Z6 days
Testing same since   114273  2017-10-10 11:03:21 Z0 days1 attempts


People who touched revisions under test:
  Alex Bennée 
  Alex Williamson 
  Anthony PERARD 
  Christian Borntraeger 
  Cornelia Huck 
  Dr. David Alan Gilbert 
  Eric Blake 
  Farhan Ali 
  Gerd Hoffmann 
  Greg Kurz 
  Hannes Reinecke 
  Hannes Reinecke 
  Igor Mammedov 
  Jan Dakinevich 
  John Snow 
  Kevin Wolf 
  Manos Pitsidianakis 
  Marc-André Lureau 
  Max Reitz 
  Michael Roth 
  Michael S. Tsirkin 
  Michael Tokarev 
  Paolo Bonzini 
  Pavel Butsykin 
  Peter Lieven 
  Peter Maydell 
  Pranith Kumar 
  Prasad J Pandit 
  Richard 

[Xen-devel] [xen-4.5-testing test] 114263: tolerable FAIL - PUSHED

[osstest service owner]

flight 114263 xen-4.5-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/114263/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-winxpsp3 16 guest-localmigrate/x10 fail in 114101 
pass in 114263
 test-amd64-amd64-libvirt-vhd 14 guest-saverestore fail in 114190 pass in 114263
 test-armhf-armhf-xl-rtds 12 guest-start  fail in 114190 pass in 114263
 test-armhf-armhf-xl-vhd 10 debian-di-install fail in 114190 pass in 114263
 test-amd64-amd64-libvirt-vhd 15 guest-saverestore.2fail pass in 114101
 test-armhf-armhf-xl-credit2  12 guest-startfail pass in 114190

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 16 guest-saverestore.2 fail in 114101 like 113369
 test-amd64-i386-xl-qemuu-winxpsp3 16 guest-localmigrate/x10 fail in 114101 
like 113408
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop   fail in 114101 like 113448
 test-armhf-armhf-xl-credit2 13 migrate-support-check fail in 114101 never pass
 test-armhf-armhf-xl-credit2 14 saverestore-support-check fail in 114101 never 
pass
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 16 guest-localmigrate/x10 fail like 
113369
 test-xtf-amd64-amd64-2   59 leak-check/check fail  like 113448
 test-xtf-amd64-amd64-1   59 leak-check/check fail  like 113448
 test-xtf-amd64-amd64-5   59 leak-check/check fail  like 113448
 test-xtf-amd64-amd64-4   59 leak-check/check fail  like 113448
 test-amd64-amd64-xl-rtds  7 xen-boot fail  like 113448
 test-xtf-amd64-amd64-3   59 leak-check/check fail  like 113448
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 113448
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeatfail  like 113448
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 113448
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 113448
 test-xtf-amd64-amd64-2   19 xtf/test-hvm32-cpuid-faulting fail  never pass
 test-xtf-amd64-amd64-2 33 xtf/test-hvm32pae-cpuid-faulting fail never pass
 test-xtf-amd64-amd64-2 40 xtf/test-hvm32pse-cpuid-faulting fail never pass
 test-xtf-amd64-amd64-2   44 xtf/test-hvm64-cpuid-faulting fail  never pass
 test-xtf-amd64-amd64-4   19 xtf/test-hvm32-cpuid-faulting fail  never pass
 test-xtf-amd64-amd64-5   19 xtf/test-hvm32-cpuid-faulting fail  never pass
 test-xtf-amd64-amd64-1   19 xtf/test-hvm32-cpuid-faulting fail  never pass
 test-xtf-amd64-amd64-4 33 xtf/test-hvm32pae-cpuid-faulting fail never pass
 test-xtf-amd64-amd64-1 33 xtf/test-hvm32pae-cpuid-faulting fail never pass
 test-xtf-amd64-amd64-4 40 xtf/test-hvm32pse-cpuid-faulting fail never pass
 test-xtf-amd64-amd64-1 40 xtf/test-hvm32pse-cpuid-faulting fail never pass
 test-xtf-amd64-amd64-4   44 xtf/test-hvm64-cpuid-faulting fail  never pass
 test-xtf-amd64-amd64-1   44 xtf/test-hvm64-cpuid-faulting fail  never pass
 test-xtf-amd64-amd64-5 33 xtf/test-hvm32pae-cpuid-faulting fail never pass
 test-xtf-amd64-amd64-5 40 xtf/test-hvm32pse-cpuid-faulting fail never pass
 test-xtf-amd64-amd64-5   44 xtf/test-hvm64-cpuid-faulting fail  never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-xtf-amd64-amd64-2   58 xtf/test-hvm64-xsa-195   fail   never pass
 test-xtf-amd64-amd64-1   58 xtf/test-hvm64-xsa-195   fail   never pass
 test-xtf-amd64-amd64-5   58 xtf/test-hvm64-xsa-195   fail   never pass
 test-xtf-amd64-amd64-4   58 xtf/test-hvm64-xsa-195   fail   never pass
 test-amd64-i386-libvirt-qcow2 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-xtf-amd64-amd64-3   58 xtf/test-hvm64-xsa-195   fail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 13 guest-saverestore  fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 

Re: [Xen-devel] [PATCH v3 09/12] fuzz/x86_emulate: Make input more compact

[George Dunlap]

> On Oct 10, 2017, at 6:26 PM, Ian Jackson  wrote:
> 
> George Dunlap writes ("[PATCH v3 09/12] fuzz/x86_emulate: Make input more 
> compact"):
>> At the moment, AFL reckons that for any given input, 87% of it is
>> completely irrelevant: that is, it can change it as much as it wants
>> but have no impact on the result of the test; and yet it can't remove
>> it.
>> 
>> This is largely because we interpret the blob handed to us as a large
>> struct, including CR values, MSR values, segment registers, and a full
>> cpu_user_regs.
>> 
>> Instead, modify our interpretation to have a "set state" stanza at the
>> front.  Begin by reading a 16-bit value; if it is lower than a certain
>> threshold, set some state according to what byte it is, and repeat.
>> Continue until the byte is above a certain threshold.
>> 
>> This allows AFL to compact any given test case much smaller; to the
>> point where now it reckons there is not a single byte of the test file
>> which becomes irrelevant.  Testing have shown that this option both
>> allows AFL to reach coverage much faster, and to have a total coverage
>> higher than with the old format.
> 
> This is basically a compression scheme.  How odd that it should help.

Well I’m pretty sure the size of the input file is more or less the precise 
cause for the difference in speed: Fuzzing a 32-byte file is just a lot faster 
than fuzzing a 1-k file.  Running them side by side makes the effect more 
obvious — I’ll show you tomorrow if you’re interested.

Since the file size is the direct cause of the speed difference, having a 
“compressed” file will naturally make things faster.


> 
> Acked-by: Ian Jackson 

Thanks.

 -George
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [ovmf baseline-only test] 72226: all pass

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 72226 ovmf real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/72226/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf 3673214c6e0eb94de9e52221cca454a3ba5976ab
baseline version:
 ovmf 728d74973c9262b6c7b7ef4be213223d55affec3

Last test of basis72221  2017-10-09 14:17:32 Z1 days
Testing same since72226  2017-10-11 02:23:02 Z0 days1 attempts


People who touched revisions under test:
  Eric Dong 
  Hao Wu 
  Ladi Prosek 
  Laszlo Ersek 
  Liming Gao 
  Ruiyu Ni 
  Yonghong Zhu 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xs.citrite.net/~osstest/testlogs/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.

(No revision log; it would be 498 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 07/12] fuzz/x86_emulate: Move all state into fuzz_state

[Andrew Cooper]

On 10/10/17 17:20, George Dunlap wrote:
> This is in preparation for adding the option for a more "compact"
> interpretation of the fuzzing data, in which we only change select
> bits of the state.
>
> Signed-off-by: George Dunlap 
> Acked-by: Jan Beulich 
> ---
> v3:
>  - Move DATA_OFFSET inside the structure
>  - Remove a stray blank line
> v2: Port over previous changes
>
> CC: Ian Jackson 
> CC: Wei Liu 
> CC: Andrew Cooper 
> CC: Jan Beulich 
> ---
>  tools/fuzz/x86_instruction_emulator/fuzz-emul.c | 89 
> +
>  1 file changed, 45 insertions(+), 44 deletions(-)
>
> diff --git a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c 
> b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
> index 8998f21fe1..20d52b33f8 100644
> --- a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
> +++ b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
> @@ -24,14 +24,8 @@
>  /* Layout of data expected as fuzzing input. */
>  struct fuzz_corpus
>  {
> -unsigned long cr[5];
> -uint64_t msr[MSR_INDEX_MAX];
> -struct cpu_user_regs regs;
> -struct segment_register segments[SEG_NUM];
> -unsigned long options;
>  unsigned char data[4096];
>  } input;
> -#define DATA_OFFSET offsetof(struct fuzz_corpus, data)
>  
>  /*
>   * Internal state of the fuzzing harness.  Calculated initially from the 
> input
> @@ -39,7 +33,14 @@ struct fuzz_corpus
>   */

You've invalidated a number of the comments describing behaviours,
including the description of the difference between fuzz_state and
fuzz_corpus.

Why do you need to move any of this in the first place?  If you insist
on keeping the compact mode, what's wrong with conditionally reading the
AFL input into either fuzz_copus entirely, or into fuzz_corpus.data[]
and then conditionally deriving this state?

That way, you don't block work to fix the root cause, which needs to end
up with architectural values in fuzz_state, derived from a bitfields in
fuzz_corpus.

~Andrew

>  struct fuzz_state
>  {
> +unsigned long options;
> +unsigned long cr[5];
> +uint64_t msr[MSR_INDEX_MAX];
> +struct segment_register segments[SEG_NUM];
> +struct cpu_user_regs regs;
> +
>  /* Fuzzer's input data. */
> +#define DATA_OFFSET offsetof(struct fuzz_state, corpus)
>  struct fuzz_corpus *corpus;
>  
>  /* Real amount of data backing corpus->data[]. */
>


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 10/12] fuzz/x86_emulate: Add --rerun option to try to track down instability

[Andrew Cooper]

On 10/10/17 17:20, George Dunlap wrote:
> @@ -659,7 +667,10 @@ static void setup_state(struct x86_emulate_ctxt *ctxt)
>  {
>  /* Fuzz all of the state in one go */
>  if ( !input_read(s, s, DATA_SIZE_FULL) )
> +{
> +printf("Input size too small\n");
>  exit(-1);
> +}

Is this hunk intended to be in the previous patch?

> @@ -886,21 +896,144 @@ int LLVMFuzzerInitialize(int *argc, char ***argv)
>  return 0;
>  }
>  
> -int LLVMFuzzerTestOneInput(const uint8_t *data_p, size_t size)
> +static void setup_fuzz_state(struct fuzz_state *state, const void *data_p, 
> size_t size)
>  {
> -struct fuzz_state state = {
> -.ops = all_fuzzer_ops,
> -};
> -struct x86_emulate_ctxt ctxt = {
> -.data = ,
> -.regs = ,
> -.addr_size = 8 * sizeof(void *),
> -.sp_size = 8 * sizeof(void *),
> -};
> +memset(state, 0, sizeof(*state));
> +state->corpus = data_p;
> +state->data_num = size;
> +}
> +
> +static int runtest(struct fuzz_state *state) {
>  int rc;
>  
> -/* Reset all global state variables */
> -memset(, 0, sizeof(input));
> +struct x86_emulate_ctxt *ctxt = >ctxt;
> +
> +state->ops = all_fuzzer_ops;
> +
> +ctxt->data = state;
> +ctxt->regs = >regs;
> +
> +setup_state(ctxt);
> +
> +sanitize_state(ctxt);
> +
> +disable_hooks(state);
> +
> +do {
> +/* FIXME: Until we actually implement SIGFPE handling properly */
> +setup_fpu_exception_handler();
> +
> +set_sizes(ctxt);
> +dump_state(ctxt);
> +
> +rc = x86_emulate(ctxt, >ops);
> +printf("Emulation result: %d\n", rc);
> +} while ( rc == X86EMUL_OKAY );
> +
> +return 0;
> +}
> +
> +static void compare_states(struct fuzz_state state[2])
> +{
> +// First zero any "internal" pointers

/* */

> +state[0].corpus = state[1].corpus = NULL;
> +state[0].ctxt.data = state[1].ctxt.data = NULL;
> +state[0].ctxt.regs = state[1].ctxt.regs = NULL;
> +
> +if ( memcmp([0], [1], sizeof(struct fuzz_state)) )
> +{
> +unsigned int i;
> +
> +printf("State mismatch\n");
> +
> +for ( i=0; i +if ( state[0].cr[i] != state[1].cr[i] )
> +printf("cr[%u]: %lx != %lx\n",
> +   i, state[0].cr[i], state[1].cr[i]);
> +
> +for ( i=0; i +if ( state[0].msr[i] != state[1].msr[i] )
> +printf("msr[%u]: %lx != %lx\n",
> +   i, state[0].msr[i], state[1].msr[i]);
> +
> +for ( i=0; i +if ( memcmp([0].segments[i], [1].segments[i],
> +sizeof(state[0].segments[0])) )
> +printf("segments[%u]: [%x:%x:%x:%lx] != [%x:%x:%x:%lx]!\n", 
> i,
> +   (unsigned)state[0].segments[i].sel,
> +   (unsigned)state[0].segments[i].attr,
> +   state[0].segments[i].limit,
> +   state[0].segments[i].base,
> +   (unsigned)state[1].segments[i].sel,
> +   (unsigned)state[1].segments[i].attr,
> +   state[1].segments[i].limit,
> +   state[1].segments[i].base);
> +
> +if ( state[0].data_num != state[1].data_num )
> +printf("data_num: %lx !=  %lx\n", state[0].data_num,
> +   state[1].data_num);
> +if ( state[0].data_index != state[1].data_index )
> +printf("data_index: %lx !=  %lx\n", state[0].data_index,
> +   state[1].data_index);
> +
> +if ( memcmp([0].regs, [1].regs, sizeof(state[0].regs)) )
> +{
> +printf("registers differ!\n");
> +/* Print If Not Equal */
> +#define PINE(elem)\

PRINT_NE() ?

> +if ( state[0].elem != state[1].elem ) \
> +printf(#elem " differ: %lx != %lx\n", \
> +   (unsigned long)state[0].elem, \
> +   (unsigned long)state[1].elem)
> +PINE(regs.r15);

Hmm - this is going to cause problems for the 32bit build.  I can't
think of an easy suggestion to fix it.

> +PINE(regs.r14);
> +PINE(regs.r13);
> +PINE(regs.r12);
> +PINE(regs.rbp);
> +PINE(regs.rbx);
> +PINE(regs.r10);
> +PINE(regs.r11);
> +PINE(regs.r9);
> +PINE(regs.r8);
> +PINE(regs.rax);
> +PINE(regs.rcx);
> +PINE(regs.rdx);
> +PINE(regs.rsi);
> +PINE(regs.rdi);
> +
> +for ( i = offsetof(struct cpu_user_regs, error_code) / 
> sizeof(unsigned);
> +  i < sizeof(state[1].regs)/sizeof(unsigned); i++ )
> +{
> +printf("[%04lu] %08x 

[Xen-devel] [seabios baseline-only test] 72225: tolerable FAIL

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 72225 seabios real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/72225/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-qemuu-nested-intel 17 debian-hvm-install/l1/l2 fail blocked 
in 72216
 test-amd64-amd64-xl-qemuu-win10-i386 16 guest-localmigrate/x10 fail blocked in 
72216
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail like 72216
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail like 72216
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail like 72216
 test-amd64-i386-xl-qemuu-ws16-amd64 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass

version targeted for testing:
 seabios  5c1a2c75951c4a59f1bf2d3c82ca7447244513ad
baseline version:
 seabios  f703604b30958312e64a5b7fc74c606a2ececc15

Last test of basis72216  2017-10-08 13:20:20 Z2 days
Testing same since72225  2017-10-10 18:52:22 Z0 days1 attempts


People who touched revisions under test:
  Kevin O'Connor 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmpass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsmpass
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm pass
 test-amd64-amd64-qemuu-nested-amdfail
 test-amd64-i386-qemuu-rhel6hvm-amd   pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-i386-xl-qemuu-debianhvm-amd64 pass
 test-amd64-amd64-xl-qemuu-win7-amd64 fail
 test-amd64-i386-xl-qemuu-win7-amd64  fail
 test-amd64-amd64-xl-qemuu-ws16-amd64 fail
 test-amd64-i386-xl-qemuu-ws16-amd64  fail
 test-amd64-amd64-xl-qemuu-win10-i386 fail
 test-amd64-i386-xl-qemuu-win10-i386  fail
 test-amd64-amd64-qemuu-nested-intel  fail
 test-amd64-i386-qemuu-rhel6hvm-intel pass



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xs.citrite.net/~osstest/testlogs/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


commit 5c1a2c75951c4a59f1bf2d3c82ca7447244513ad
Author: Kevin O'Connor 
Date:   Tue Oct 3 11:29:12 2017 -0400

xhci: Verify the device is still present in xhci_cmd_submit()

Make sure the USB device is still present before altering the xhci
"slot" for it.  It appears some controllers will hang if a request is
sent to a port no longer connected.

Signed-off-by: Kevin O'Connor 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2] VT-d: use two 32-bit writes to update DMAR fault address registers

[Haozhong Zhang]

The 64-bit DMAR fault address is composed of two 32 bits registers
DMAR_FEADDR_REG and DMAR_FEUADDR_REG. According to VT-d spec:
"Software is expected to access 32-bit registers as aligned doublewords",
a hypervisor should use two 32-bit writes to DMAR_FEADDR_REG and
DMAR_FEUADDR_REG separately in order to update a 64-bit fault address,
rather than a 64-bit write to DMAR_FEADDR_REG. Note that when x2APIC
is not enabled DMAR_FEUADDR_REG is reserved and it's not necessary to
update it.

Though I haven't seen any errors caused by such one 64-bit write on
real machines, it's still better to follow the specification.

Fixes: ae05fd3912b ("VT-d: use qword MMIO access for MSI address writes")
Reviewed-by: Roger Pau Monné 
Signed-off-by: Haozhong Zhang 
---
Changes in v2:
 * Explain in commit message and code comment why not updating DMAR_FEUADDR_REG
   when x2APIC is not enabled

This patch actually reverts part of commit ae05fd3912b
("VT-d: use qword MMIO access for MSI address writes"). The latter
was included in XSA-120, 128..131 follow-up patch series [1]. I
don't know whether my patch breaks those XSA fixes. If it does,
please drop my patch.

[1] https://lists.xenproject.org/archives/html/xen-devel/2015-06/msg00638.html
---
 xen/drivers/passthrough/vtd/iommu.c | 8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/xen/drivers/passthrough/vtd/iommu.c 
b/xen/drivers/passthrough/vtd/iommu.c
index daaed0abbd..81dd2085c7 100644
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -1105,7 +1105,13 @@ static void dma_msi_set_affinity(struct irq_desc *desc, 
const cpumask_t *mask)
 
 spin_lock_irqsave(>register_lock, flags);
 dmar_writel(iommu->reg, DMAR_FEDATA_REG, msg.data);
-dmar_writeq(iommu->reg, DMAR_FEADDR_REG, msg.address);
+dmar_writel(iommu->reg, DMAR_FEADDR_REG, msg.address_lo);
+/*
+ * When x2APIC is not enabled, DMAR_FEUADDR_REG is reserved and
+ * it's not necessary to update it.
+ */
+if (x2apic_enabled)
+dmar_writel(iommu->reg, DMAR_FEUADDR_REG, msg.address_hi);
 spin_unlock_irqrestore(>register_lock, flags);
 }
 
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v9] x86/hvm: Implement hvmemul_write() using real mappings

[Paul Durrant]

> -Original Message-
> From: Alexandru Isaila [mailto:aisa...@bitdefender.com]
> Sent: 09 October 2017 11:56
> To: xen-devel@lists.xen.org
> Cc: jbeul...@suse.com; Andrew Cooper ;
> Paul Durrant ; Alexandru Isaila
> 
> Subject: [PATCH v9] x86/hvm: Implement hvmemul_write() using real
> mappings
> 
> From: Andrew Cooper 
> 
> An access which crosses a page boundary is performed atomically by x86
> hardware, albeit with a severe performance penalty.  An important corner
> case
> is when a straddled access hits two pages which differ in whether a
> translation exists, or in net access rights.
> 
> The use of hvm_copy*() in hvmemul_write() is problematic, because it
> performs
> a translation then completes the partial write, before moving onto the next
> translation.
> 
> If an individual emulated write straddles two pages, the first of which is
> writable, and the second of which is not, the first half of the write will
> complete before #PF is raised from the second half.
> 
> This results in guest state corruption as a side effect of emulation, which
> has been observed to cause windows to crash while under introspection.
> 
> Introduce the hvmemul_{,un}map_linear_addr() helpers, which translate an
> entire contents of a linear access, and vmap() the underlying frames to
> provide a contiguous virtual mapping for the emulator to use.  This is the
> same mechanism as used by the shadow emulation code.
> 
> This will catch any translation issues and abort the emulation before any
> modifications occur.
> 
> Signed-off-by: Andrew Cooper 
> Signed-off-by: Alexandru Isaila 

Reviewed-by: Paul Durrant 

> 
> ---
> Changes since V8:
>   - Removed comment
>   - Changed the addr formula
>   - Added blank space in the for statement.
> 
> Note: Tested with win32/64 and ubuntu64 guests.
> ---
>  xen/arch/x86/hvm/emulate.c| 179
> ++
>  xen/include/asm-x86/hvm/emulate.h |   7 ++
>  2 files changed, 170 insertions(+), 16 deletions(-)
> 
> diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c
> index cc874ce..957fc46 100644
> --- a/xen/arch/x86/hvm/emulate.c
> +++ b/xen/arch/x86/hvm/emulate.c
> @@ -498,6 +498,160 @@ static int hvmemul_do_mmio_addr(paddr_t
> mmio_gpa,
>  }
> 
>  /*
> + * Map the frame(s) covering an individual linear access, for writeable
> + * access.  May return NULL for MMIO, or ERR_PTR(~X86EMUL_*) for other
> errors
> + * including ERR_PTR(~X86EMUL_OKAY) for write-discard mappings.
> + *
> + * In debug builds, map() checks that each slot in hvmemul_ctxt->mfn[] is
> + * clean before use, and poisions unused slots with INVALID_MFN.
> + */
> +static void *hvmemul_map_linear_addr(
> +unsigned long linear, unsigned int bytes, uint32_t pfec,
> +struct hvm_emulate_ctxt *hvmemul_ctxt)
> +{
> +struct vcpu *curr = current;
> +void *err, *mapping;
> +unsigned int nr_frames = ((linear + bytes - !!bytes) >> PAGE_SHIFT) -
> +(linear >> PAGE_SHIFT) + 1;
> +unsigned int i;
> +
> +/*
> + * mfn points to the next free slot.  All used slots have a page 
> reference
> + * held on them.
> + */
> +mfn_t *mfn = _ctxt->mfn[0];
> +
> +/*
> + * The caller has no legitimate reason for trying a zero-byte write, but
> + * final is calculate to fail safe in release builds.
> + *
> + * The maximum write size depends on the number of adjacent mfns[]
> which
> + * can be vmap()'d, accouting for possible misalignment within the 
> region.
> + * The higher level emulation callers are responsible for ensuring that
> + * mfns[] is large enough for the requested write size.
> + */
> +if ( bytes == 0 ||
> + nr_frames > ARRAY_SIZE(hvmemul_ctxt->mfn) )
> +{
> +ASSERT_UNREACHABLE();
> +printk("goto unhandle ERROR~!~~\n");
> +goto unhandleable;
> +}
> +
> +for ( i = 0; i < nr_frames; i++ )
> +{
> +enum hvm_translation_result res;
> +struct page_info *page;
> +pagefault_info_t pfinfo;
> +p2m_type_t p2mt;
> +unsigned long addr = i ? (linear + (i << PAGE_SHIFT)) & PAGE_MASK :
> linear;
> +
> +if ( hvmemul_ctxt->ctxt.addr_size < 64 )
> +addr = (uint32_t)addr;
> +
> +/* Error checking.  Confirm that the current slot is clean. */
> +ASSERT(mfn_x(*mfn) == 0);
> +
> +res = hvm_translate_get_page(curr, addr, true, pfec,
> + , , NULL, );
> +
> +switch ( res )
> +{
> +case HVMTRANS_okay:
> +break;
> +
> +case HVMTRANS_bad_linear_to_gfn:
> +x86_emul_pagefault(pfinfo.ec, pfinfo.linear, 
> _ctxt->ctxt);
> +err = ERR_PTR(~X86EMUL_EXCEPTION);
> +goto out;
> +
> +

Re: [Xen-devel] [PATCH v9 10/11] common: add a new mappable resource type: XENMEM_resource_grant_table

[Paul Durrant]

> -Original Message-
> From: Xen-devel [mailto:xen-devel-boun...@lists.xen.org] On Behalf Of Jan
> Beulich
> Sent: 10 October 2017 11:26
> To: Paul Durrant 
> Cc: Stefano Stabellini ; Wei Liu
> ; KonradRzeszutek Wilk ;
> George Dunlap ; Andrew Cooper
> ; Ian Jackson ; Tim
> (Xen.org) ; xen-de...@lists.xenproject.org
> Subject: Re: [Xen-devel] [PATCH v9 10/11] common: add a new mappable
> resource type: XENMEM_resource_grant_table
> 
> >>> On 06.10.17 at 14:25,  wrote:
> > --- a/xen/common/grant_table.c
> > +++ b/xen/common/grant_table.c
> > @@ -3756,14 +3756,13 @@ int mem_sharing_gref_to_gfn(struct
> grant_table *gt, grant_ref_t ref,
> >  }
> >  #endif
> >
> > -int gnttab_map_frame(struct domain *d, unsigned long idx, gfn_t gfn,
> > - mfn_t *mfn)
> > +/* Caller must hold write lock as version may change and table may grow
> */
> > +static int gnttab_get_frame_locked(struct domain *d, unsigned long idx,
> > +   mfn_t *mfn)
> 
> I don't think this helper function needs to be passed d; gt appears
> to suffice.

Alas it does not. It calls through to gnttab_grow_table() which requires the 
domain.

> 
> > @@ -3787,6 +3786,19 @@ int gnttab_map_frame(struct domain *d,
> unsigned long idx, gfn_t gfn,
> >  rc = -EINVAL;
> >  }
> >
> > +return rc;
> > +}
> > +
> > +int gnttab_map_frame(struct domain *d, unsigned long idx, gfn_t gfn,
> > + mfn_t *mfn)
> > +{
> > +struct grant_table *gt = d->grant_table;
> > +int rc;
> > +
> > +grant_write_lock(gt);
> > +
> > +rc = gnttab_get_frame_locked(d, idx, mfn);
> > +
> >  if ( !rc )
> >  gnttab_set_frame_gfn(gt, idx, gfn);
> >
> > @@ -3795,6 +3807,18 @@ int gnttab_map_frame(struct domain *d,
> unsigned long idx, gfn_t gfn,
> >  return rc;
> >  }
> >
> > +int gnttab_get_frame(struct domain *d, unsigned long idx, mfn_t *mfn)
> 
> const struct domain * (I realize now that the same should have
> been done for gnttab_map_frame() when it was introduced;
> perhaps you could change that at the same time).

Again, the problem is that grow_table and functions it calls don't take a const 
pointer. I tried cascading the const through to the underlying function but the 
patch started to balloon so I think such work should be deferred.

> 
> > @@ -966,6 +967,30 @@ static long xatp_permission_check(struct domain
> *d,
> > unsigned int space)
> >  }
> >
> >  #ifdef CONFIG_X86
> > +static int acquire_grant_table(struct domain *d, unsigned int id,
> 
> This clearly isn't x86-specific code.

Ok.

> 
> > +   unsigned long frame,
> > +   unsigned long nr_frames,
> > +   unsigned long mfn_list[])
> > +{
> > +unsigned int i = nr_frames;
> 
> Silent truncation just like in the earlier patch.

Yes, nr_frames should be an unsigned int.

> 
> > +if ( id != 0 )
> 
> Do we want a constant here again? Also acquiring the status page
> MFNs via separate ID would seem better than re-using
> XENMAPIDX_grant_table_status here, the more that this uses bit
> 31 while right now your interface structure field is still 64 bits wide.
> 

Ok, that's a better way to do it.

> > @@ -993,6 +1018,11 @@ static int acquire_resource(const
> xen_mem_acquire_resource_t *xmar)
> >   xmar->nr_frames, mfn_list);
> >  break;
> >
> > +case XENMEM_resource_grant_table:
> > +rc = acquire_grant_table(d, xmar->id, xmar->frame,
> > + xmar->nr_frames, mfn_list);
> > +break;
> 
> Is this really generally useful with mfn_list[] having just two entries?
> 

Good point. I'll increase the size of the array in this patch (to the default 
table size of 32... I think that's a reasonable value to choose).

> Jan
> 
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [linux-4.9 test] 114230: regressions - FAIL

[osstest service owner]

flight 114230 linux-4.9 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/114230/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-pvh-intel 12 guest-start fail REGR. vs. 114036
 test-amd64-amd64-xl-pvh-amd  12 guest-start  fail REGR. vs. 114036

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 114036
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 114036
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 114036
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 10 windows-installfail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qcow2 12 migrate-support-checkfail  never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 13 guest-saverestore   fail never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-ws16-amd64 13 guest-saverestore   fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 linuxf37eb7b586f1dd24a86c50278c65322fc6787722
baseline version:
 linux1852eae92c460813692808234da35d142a405ab7

Last test of basis   114036  2017-10-05 08:21:13 Z5 days
Testing same since   114134  2017-10-08 09:26:45 Z2 days3 attempts


People who touched revisions under test:
  Aaron Brown 
  Afzal Mohammed 
  Alden Tondettar 
  Alexander Potapenko 
  Alexandre Belloni 
  Alexey Brodkin 
  Alexey Brodkin 
  Andreas Klinger 
  Andrew Morton 
  Anna Schumaker 
  Ansis Atteka 
  Archit Taneja 
  Ard Biesheuvel 
  Arnd Bergmann 
  Arvind Yadav 
  Axel Köllhofer 
  Balbir Singh 
  Bart Van Assche 
  Bartlomiej 

Re: [Xen-devel] [PATCH v6 11/11] vpci/msix: add MSI-X handlers

[Roger Pau Monné]

On Wed, Oct 04, 2017 at 08:34:43AM +, Jan Beulich wrote:
> >>> On 19.09.17 at 17:29,  wrote:
> > +const struct vpci_bar *bars;
> > +struct vpci_msix *msix;
> > +const struct vpci_msix_entry *entry;
> > +unsigned int offset;
> > +
> > +*data = ~0ul;
> > +
> > +msix = vpci_msix_find(d, addr);
> > +if ( !msix || !vpci_msix_access_allowed(msix->pdev, addr, len) )
> > +return X86EMUL_OKAY;
> 
> In the !msix case I'm once again not convinced returning OKAY is correct
> here.

From what we have spoken in the mmcfg case, for the !msix case Xen
should return _RETRY. This error can only happen when the msix table
is unmapped in between a accept and read/write call, so calling the
accept handler again will return the correct value.

> > --- a/xen/include/xen/vpci.h
> > +++ b/xen/include/xen/vpci.h
> > @@ -100,6 +100,40 @@ struct vpci {
> >  /* 64-bit address capable? */
> >  bool address64;
> >  } *msi;
> > +
> > +/* MSI-X data. */
> > +struct vpci_msix {
> > +struct pci_dev *pdev;
> > +/* List link. */
> > +struct list_head next;
> > +/* Table information. */
> > +struct vpci_msix_mem {
> > +/* MSI-X table offset. */
> > +unsigned int offset;
> > +/* MSI-X table BIR. */
> > +unsigned int bir;
> > +/* Table size. */
> > +unsigned int size;
> > +#define VPCI_MSIX_TABLE 0
> > +#define VPCI_MSIX_PBA   1
> > +#define VPCI_MSIX_MEM_NUM   2
> > +} mem[VPCI_MSIX_MEM_NUM];
> > +/* Maximum number of vectors supported by the device. */
> > +unsigned int max_entries;
> > +/* MSI-X enabled? */
> > +bool enabled;
> > +/* Masked? */
> > +bool masked;
> > +/* Entries. */
> > +struct vpci_msix_entry {
> > +uint64_t addr;
> > +uint32_t data;
> > +unsigned int nr;
> > +struct vpci_arch_msix_entry arch;
> > +bool masked;
> > +bool updated;
> > +} entries[];
> > +} *msix;
> 
> Same remark as for MSI regarding optimizing structure size.

Going over the fields, bir can be turned into a uint8_t, and size into
a uint16_t. max_entries can also be converted to a uint16_t together
with nr.

Apart from that I don't see much more optimization, unless we start
packaging fields (ie: offset and bir could reside in a uint32_t), but
IMHO that's going to make the code harder to parse for little gain,
and will involve more calculations in the handlers.

Thanks, Roger.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 06/12] fuzz/x86_emulate: Take multiple test files for inputs

[Andrew Cooper]

On 10/10/17 17:58, George Dunlap wrote:
> On 10/10/2017 05:56 PM, Andrew Cooper wrote:
>> On 10/10/17 17:20, George Dunlap wrote:
>>> @@ -65,12 +68,15 @@ int main(int argc, char **argv)
>>>  #ifdef __AFL_HAVE_MANUAL_CONTROL
>>>  __AFL_INIT();
>>>  
>>> -while ( __AFL_LOOP(1000) )
>>> +for( count = 0; __AFL_LOOP(1000); )
>>> +#else
>>> +for( count = 0; count < max; count++ )
>>>  #endif
>>>  {
>>>  if ( fp != stdin ) /* If not using stdin, open the provided file. 
>>> */
>>>  {
>>> -fp = fopen(argv[optind], "rb");
>>> +printf("Opening file %s\n", argv[optind]);
>>> +fp = fopen(argv[optind + count], "rb");
>> I presume the printf() wants adjusting to match the fopen() ?
> Oh!  I thought I'd fixed that.  Indeed it does.
>
> I can fix that on check-in, if we don't find anything bigger worth
> re-sending for.

I can't see anything else needing fixing.  Acked-by: Andrew Cooper


~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 09/12] fuzz/x86_emulate: Make input more compact

[Andrew Cooper]

On 10/10/17 18:13, George Dunlap wrote:
> On 10/10/2017 06:11 PM, Andrew Cooper wrote:
>> On 10/10/17 18:01, George Dunlap wrote:
>>> On 10/10/2017 05:59 PM, Andrew Cooper wrote:
 On 10/10/17 17:20, George Dunlap wrote:
> At the moment, AFL reckons that for any given input, 87% of it is
> completely irrelevant: that is, it can change it as much as it wants
> but have no impact on the result of the test; and yet it can't remove
> it.
>
> This is largely because we interpret the blob handed to us as a large
> struct, including CR values, MSR values, segment registers, and a full
> cpu_user_regs.
>
> Instead, modify our interpretation to have a "set state" stanza at the
> front.  Begin by reading a 16-bit value; if it is lower than a certain
> threshold, set some state according to what byte it is, and repeat.
> Continue until the byte is above a certain threshold.
>
> This allows AFL to compact any given test case much smaller; to the
> point where now it reckons there is not a single byte of the test file
> which becomes irrelevant.  Testing have shown that this option both
> allows AFL to reach coverage much faster, and to have a total coverage
> higher than with the old format.
>
> Make this an option (rather than a unilateral change) to enable
> side-by-side performance comparison of the old and new formats.
>
> Signed-off-by: George Dunlap 
 I am still of the opinion that this is a waste of effort, which would be
 better spent actually removing the irrelevant state in the first place;
 not building an obfuscation algorithm.

 I'm not going to nack the patch because that is probably over the top,
 but I'm not in favour if this change going in.
>>> Did you look at the evidence I presented, demonstrating that this
>>> significantly increases the effectiveness of AFL?
>> I can easily believe that you've found an obfucation algorithm which
>> does better than the current state layout.
>>
>> I do not believe that any amount of obfuscation will be better than
>> actually fixing the root cause of the problem; that the current state
>> really is mostly irrelevant, and can easily be shrunk.
> Right; well I've already explained why I don't think "obfuscation" is
> the right term.

How else would describe it?  You are purposefully hiding the structure
of the data by doing conditional initialisation based on earlier values.

> For the time being, we have something which improves
> efficiency;

How much of this measured efficiently is actually ALF finding paths
around setup_state() rather than finding new paths in the emulator
itself?  I can't think of a test which would isolate this properly.

> let's check it in now, and in the future if you or someone
> else finds a way to fix it "properly" we can do that.

I wouldn't really mind so much if this change didn't make it harder to
fix the root cause.  As it is, your prerequisite patch prohibits any
ability to overlay a minimal structure over the fuzzing corpus.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 05/12] fuzz/x86_emulate: Add 'afl-cov' target

[Andrew Cooper]

On 10/10/17 17:20, George Dunlap wrote:
> ...to generate a "normal" coverage-instrumented binary, suitable for
> use with gcov or afl-cov.
>
> This is slightly annoying because:
>
>  - Every object file needs to have been instrumented to work
>effectively
>
>  - You generally want to have both an afl-instrumented binary and a
>gcov-instrumented binary at the same time, but
>
>  - gcov instrumentation and afl instrumentation are mutually exclusive
>
> So when making the `afl-cov` target, generate a second set of object
> files and a second binary with the `-cov` suffix.
>
> While we're here, remove the redundant x86-emulate.c dependency for
> x86-emulate.o.
>
> Signed-off-by: George Dunlap 

Acked-by: Andrew Cooper 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 09/12] fuzz/x86_emulate: Make input more compact

[Andrew Cooper]

On 10/10/17 18:01, George Dunlap wrote:
> On 10/10/2017 05:59 PM, Andrew Cooper wrote:
>> On 10/10/17 17:20, George Dunlap wrote:
>>> At the moment, AFL reckons that for any given input, 87% of it is
>>> completely irrelevant: that is, it can change it as much as it wants
>>> but have no impact on the result of the test; and yet it can't remove
>>> it.
>>>
>>> This is largely because we interpret the blob handed to us as a large
>>> struct, including CR values, MSR values, segment registers, and a full
>>> cpu_user_regs.
>>>
>>> Instead, modify our interpretation to have a "set state" stanza at the
>>> front.  Begin by reading a 16-bit value; if it is lower than a certain
>>> threshold, set some state according to what byte it is, and repeat.
>>> Continue until the byte is above a certain threshold.
>>>
>>> This allows AFL to compact any given test case much smaller; to the
>>> point where now it reckons there is not a single byte of the test file
>>> which becomes irrelevant.  Testing have shown that this option both
>>> allows AFL to reach coverage much faster, and to have a total coverage
>>> higher than with the old format.
>>>
>>> Make this an option (rather than a unilateral change) to enable
>>> side-by-side performance comparison of the old and new formats.
>>>
>>> Signed-off-by: George Dunlap 
>> I am still of the opinion that this is a waste of effort, which would be
>> better spent actually removing the irrelevant state in the first place;
>> not building an obfuscation algorithm.
>>
>> I'm not going to nack the patch because that is probably over the top,
>> but I'm not in favour if this change going in.
> Did you look at the evidence I presented, demonstrating that this
> significantly increases the effectiveness of AFL?

I can easily believe that you've found an obfucation algorithm which
does better than the current state layout.

I do not believe that any amount of obfuscation will be better than
actually fixing the root cause of the problem; that the current state
really is mostly irrelevant, and can easily be shrunk.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 09/12] fuzz/x86_emulate: Make input more compact

[Andrew Cooper]

On 10/10/17 17:20, George Dunlap wrote:
> At the moment, AFL reckons that for any given input, 87% of it is
> completely irrelevant: that is, it can change it as much as it wants
> but have no impact on the result of the test; and yet it can't remove
> it.
>
> This is largely because we interpret the blob handed to us as a large
> struct, including CR values, MSR values, segment registers, and a full
> cpu_user_regs.
>
> Instead, modify our interpretation to have a "set state" stanza at the
> front.  Begin by reading a 16-bit value; if it is lower than a certain
> threshold, set some state according to what byte it is, and repeat.
> Continue until the byte is above a certain threshold.
>
> This allows AFL to compact any given test case much smaller; to the
> point where now it reckons there is not a single byte of the test file
> which becomes irrelevant.  Testing have shown that this option both
> allows AFL to reach coverage much faster, and to have a total coverage
> higher than with the old format.
>
> Make this an option (rather than a unilateral change) to enable
> side-by-side performance comparison of the old and new formats.
>
> Signed-off-by: George Dunlap 

I am still of the opinion that this is a waste of effort, which would be
better spent actually removing the irrelevant state in the first place;
not building an obfuscation algorithm.

I'm not going to nack the patch because that is probably over the top,
but I'm not in favour if this change going in.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 03/12] fuzz/x86_emulate: Implement input_read() and input_avail()

[Andrew Cooper]

On 10/10/17 17:20, George Dunlap wrote:
> Rather than open-coding the "read" from the input file.
>
> Signed-off-by: George Dunlap 

Acked-by: Andrew Cooper 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 06/12] fuzz/x86_emulate: Take multiple test files for inputs

[Andrew Cooper]

On 10/10/17 17:20, George Dunlap wrote:
> @@ -65,12 +68,15 @@ int main(int argc, char **argv)
>  #ifdef __AFL_HAVE_MANUAL_CONTROL
>  __AFL_INIT();
>  
> -while ( __AFL_LOOP(1000) )
> +for( count = 0; __AFL_LOOP(1000); )
> +#else
> +for( count = 0; count < max; count++ )
>  #endif
>  {
>  if ( fp != stdin ) /* If not using stdin, open the provided file. */
>  {
> -fp = fopen(argv[optind], "rb");
> +printf("Opening file %s\n", argv[optind]);
> +fp = fopen(argv[optind + count], "rb");

I presume the printf() wants adjusting to match the fopen() ?

~Andrew

>  if ( fp == NULL )
>  {
>  perror("fopen");
> @@ -100,7 +106,10 @@ int main(int argc, char **argv)
>  if ( !feof(fp) )
>  {
>  printf("Input too large\n");
> -exit(-1);
> +/* Don't exit if we're doing batch processing */
> +if ( max == 1 )
> +exit(-1);
> +continue;
>  }
>  
>  if ( fp != stdin )


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 01/12] fuzz/x86_emulate: Clear errors after each iteration

[Andrew Cooper]

On 10/10/17 17:47, George Dunlap wrote:
> On 10/10/2017 05:20 PM, George Dunlap wrote:
>> Once feof() returns true for a stream, it will continue to return true
>> for that stream until clearerr() is called (or the stream is closed
>> and re-opened).
>>
>> In llvm-clang-fast-mode, the same file descriptor is used for each
>> iteration of the loop, meaning that the "Input too large" check was
>> broken -- feof() would return true even if the fread() hadn't hit the
>> end of the file.  The result is that AFL generates testcases of
>> arbitrary size.
>>
>> Fix this by fseek'ing to the beginning of the file on every iteration;
>> this resets the EOF marker and other state.
>>
>> Signed-off-by: George Dunlap 
>> ---
>> Changes in v3:
>> - Fix the issue in the official sanctioned way
> Hmm, seems v2 of this patch was checked in; review had flagged up that
> "clearerr()" was too big of a hammer.
>
> Attached is a revised v1/12 patch that fixes this.
>
>  -George

Acked-by: Andrew Cooper 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v4] x86: psr: support co-exist features' values setting

[Yi Sun]

The whole value array is transferred into 'do_write_psr_msrs'. Then, we can
write all features values on the cos id into MSRs.

Because multiple features may co-exist, we need handle all features to write
values of them into a COS register with new COS ID. E.g:
1. L3 CAT and L2 CAT co-exist.
2. Dom1 and Dom2 share the same COS ID (2). The L3 CAT CBM of Dom1 is 0x1ff,
   the L2 CAT CBM of Dom1 is 0x1f.
3. User wants to change L2 CBM of Dom1 to be 0xf. Because COS ID 2 is
   used by Dom2 too, we have to pick a new COS ID 3. The values of Dom1 on
   COS ID 3 are all default values as below:
   -
   | COS 3 |
   -
   L3 CAT  | 0x7ff |
   -
   L2 CAT  | 0xff  |
   -
4. After setting, the L3 CAT CBM value of Dom1 should be kept and the new L2
   CAT CBM is set. So, the values on COS ID 3 should be below.
   -
   | COS 3 |
   -
   L3 CAT  | 0x1ff |
   -
   L2 CAT  | 0xf   |
   -

Signed-off-by: Yi Sun 
---
CC: Jan Beulich 
CC: Andrew Cooper 
CC: Wei Liu 
CC: Roger Pau Monné 
CC: Julien Grall 

v4:
- remove init of 'result'.
  (suggested by Roger Pau Monné)
- remove 'features' in 'cos_write_info' and get socket info in
  'do_write_psr_msrs' to get features array.
  (suggested by Jan Beulich)
- fix a typo in commit message.
  (suggested by Kent R. Spillner)
v3:
- add 'result' in 'cos_write_info' to return error code.
  (suggested by Roger Pau Monné)
v2:
- fix issues in commit message.
  (suggested by Roger Pau Monné)
- remove unnecessary local variable 'val_array'.
  (suggested by Roger Pau Monné)
---
 xen/arch/x86/psr.c | 62 +++---
 1 file changed, 36 insertions(+), 26 deletions(-)

diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index daa2aeb..a812124 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -,25 +,48 @@ static unsigned int get_socket_cpu(unsigned int socket)
 struct cos_write_info
 {
 unsigned int cos;
-struct feat_node *feature;
 const uint32_t *val;
-const struct feat_props *props;
+unsigned int array_len;
+int result;
 };
 
 static void do_write_psr_msrs(void *data)
 {
-const struct cos_write_info *info = data;
-struct feat_node *feat = info->feature;
-const struct feat_props *props = info->props;
-unsigned int i, cos = info->cos, cos_num = props->cos_num;
+struct cos_write_info *info = data;
+unsigned int i, index = 0, cos = info->cos;
+struct psr_socket_info *socket_info =
+get_socket_info(cpu_to_socket(smp_processor_id()));
 
-for ( i = 0; i < cos_num; i++ )
+/*
+ * Iterate all featuers to write different value (not same as MSR) for
+ * each feature.
+ */
+for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
 {
-if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
+struct feat_node *feat = socket_info->features[i];
+const struct feat_props *props = feat_props[i];
+unsigned int cos_num, j;
+
+if ( !feat || !props )
+continue;
+
+cos_num = props->cos_num;
+if ( info->array_len < index + cos_num )
+{
+info->result = -ENOSPC;
+return;
+}
+
+for ( j = 0; j < cos_num; j++ )
 {
-feat->cos_reg_val[cos * cos_num + i] = info->val[i];
-props->write_msr(cos, info->val[i], props->type[i]);
+if ( feat->cos_reg_val[cos * cos_num + j] != info->val[index + j] )
+{
+feat->cos_reg_val[cos * cos_num + j] = info->val[index + j];
+props->write_msr(cos, info->val[index + j], props->type[j]);
+}
 }
+
+index += cos_num;
 }
 }
 
@@ -1137,30 +1160,17 @@ static int write_psr_msrs(unsigned int socket, unsigned 
int cos,
   const uint32_t val[], unsigned int array_len,
   enum psr_feat_type feat_type)
 {
-int ret;
 struct psr_socket_info *info = get_socket_info(socket);
 struct cos_write_info data =
 {
 .cos = cos,
-.feature = info->features[feat_type],
-.props = feat_props[feat_type],
+.val = val,
+.array_len = array_len,
 };
 
 if ( cos > info->features[feat_type]->cos_max )
 return -EINVAL;
 
-/* Skip to the feature's value head. */
-ret = skip_prior_features(_len, feat_type);
-if ( ret < 0 )
-return ret;
-
-val += ret;
-
-if ( array_len < feat_props[feat_type]->cos_num )
-return -ENOSPC;
-
-data.val = val;
-
 if ( socket == cpu_to_socket(smp_processor_id()) )
 do_write_psr_msrs();
 else
@@ -1172,7 +1182,7 @@ static 

[Xen-devel] [ovmf test] 114270: all pass - PUSHED

[osstest service owner]

flight 114270 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/114270/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf 3673214c6e0eb94de9e52221cca454a3ba5976ab
baseline version:
 ovmf 728d74973c9262b6c7b7ef4be213223d55affec3

Last test of basis   114172  2017-10-09 02:47:37 Z1 days
Failing since114197  2017-10-09 14:48:23 Z1 days2 attempts
Testing same since   114270  2017-10-10 11:02:07 Z0 days1 attempts


People who touched revisions under test:
  Eric Dong 
  Hao Wu 
  Ladi Prosek 
  Laszlo Ersek 
  Liming Gao 
  Ruiyu Ni 
  Yonghong Zhu 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=ovmf
+ revision=3673214c6e0eb94de9e52221cca454a3ba5976ab
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
 export PERLLIB=.:.
 PERLLIB=.:.
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push ovmf 
3673214c6e0eb94de9e52221cca454a3ba5976ab
+ branch=ovmf
+ revision=3673214c6e0eb94de9e52221cca454a3ba5976ab
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
 export PERLLIB=.:.:.
 PERLLIB=.:.:.
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
+++ export PERLLIB=.:.:.:.
+++ PERLLIB=.:.:.:.
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=ovmf
+ xenbranch=xen-unstable
+ '[' xovmf = xlinux ']'
+ linuxbranch=
+ '[' x = x ']'
+ qemuubranch=qemu-upstream-unstable
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable
+ prevxenbranch=xen-4.9-testing
+ '[' x3673214c6e0eb94de9e52221cca454a3ba5976ab = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/xtf.git
++ : osst...@xenbits.xen.org:/home/xen/git/xtf.git
++ : git://xenbits.xen.org/xtf.git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/rumprun.git
++ : git://git.seabios.org/seabios.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/seabios.git
++ : 

[Xen-devel] [xen-unstable-smoke test] 114308: regressions - FAIL

[osstest service owner]

flight 114308 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/114308/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl   7 xen-boot fail REGR. vs. 114299

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass

version targeted for testing:
 xen  e2dfe4a037b0c6ccfd2375e4b60668109a0118e5
baseline version:
 xen  3b40cfcd1a1912c2e4c4eb353dc77cbf35c63c3a

Last test of basis   114299  2017-10-10 21:02:54 Z0 days
Testing same since   114308  2017-10-10 23:01:10 Z0 days1 attempts


People who touched revisions under test:
  Julien Grall 
  Stefano Stabellini 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  fail
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.


commit e2dfe4a037b0c6ccfd2375e4b60668109a0118e5
Author: Julien Grall 
Date:   Mon Oct 9 14:23:41 2017 +0100

xen/arm: mm: Use memory flags for modify_xen_mappings rather than custom one

This will help to consolidate the page-table code and avoid different
path depending on the action to perform.

Signed-off-by: Julien Grall 
Reviewed-by: Andre Przywara 
Reviewed-by: Stefano Stabellini 
Reviewed-by: Konrad Rzeszutek Wilk 

commit 6b88beed40c756aaff018d286f4de31351240a93
Author: Julien Grall 
Date:   Mon Oct 9 14:23:40 2017 +0100

xen/arm: mm: Handle permission flags when adding a new mapping

Currently, all the new mappings will be read-write non-executable. Allow the
caller to use other permissions.

Signed-off-by: Julien Grall 
Reviewed-by: Stefano Stabellini 

commit 28f2ad440a08908010abec43b7ccc3283051e943
Author: Julien Grall 
Date:   Mon Oct 9 14:23:39 2017 +0100

xen/arm: mm: Embed permission in the flags

Currently, it is not possible to specify the permission of a new
mapping. It would be necessary to use the function modify_xen_mappings
with a different set of flags.

Introduce a couple of new flags for the permissions (Non-eXecutable,
Read-Only) and also provides definition that combine the memory attribute
and permission for common combinations.

PAGE_HYPERVISOR is now an alias to PAGE_HYPERVISOR_RW (read-write,
non-executable mappings). This does not affect the current mapping using
PAGE_HYPERVISOR because Xen is currently forcing all the mapping to be
non-executable by default (see mfn_to_xen_entry).

A follow-up patch will change modify_xen_mappings to use the new flags.

Signed-off-by: Julien Grall 
Reviewed-by: Stefano Stabellini 
Signed-off-by: Stefano Stabellini 

commit 5f3edb5f32e511b915d173403d0b7b5ea38e00ad
Author: Julien Grall 
Date:   Mon Oct 9 14:23:38 2017 +0100

xen/arm: page: Describe the layout of flags used to update page tables

Currently, the flags used to update page tables (i.e PAGE_HYPERVISOR_*)
only contains the memory attribute index. Follow-up patches will add
more information in it. So document the current layout.

At the same time introduce PAGE_AI_MASK to get the memory attribute
index easily.

Signed-off-by: Julien Grall 
Reviewed-by: Andre Przywara 
Reviewed-by: Stefano Stabellini 

commit 7d68c8db25f0ffc7f39af2fc929f1c77c1affa01
Author: Julien Grall 
Date:   Mon Oct 9 14:23:37 2017 +0100

xen/arm: mm: Use PAGE_HYPERVISOR_* 

Re: [Xen-devel] [PATCH v9 05/11] x86/mm: add HYPERVISOR_memory_op to acquire guest resources

[Paul Durrant]

> -Original Message-
> From: Paul Durrant
> Sent: 10 October 2017 15:10
> To: 'Jan Beulich' 
> Cc: Andrew Cooper ; Wei Liu
> ; George Dunlap ; Ian
> Jackson ; Stefano Stabellini
> ; xen-de...@lists.xenproject.org; Konrad Rzeszutek
> Wilk ; Tim (Xen.org) 
> Subject: RE: [PATCH v9 05/11] x86/mm: add HYPERVISOR_memory_op to
> acquire guest resources
> 
> > -Original Message-
> > From: Jan Beulich [mailto:jbeul...@suse.com]
> > Sent: 09 October 2017 15:23
> > To: Paul Durrant 
> > Cc: Andrew Cooper ; Wei Liu
> > ; George Dunlap ; Ian
> > Jackson ; Stefano Stabellini
> > ; xen-de...@lists.xenproject.org; Konrad
> Rzeszutek
> > Wilk ; Tim (Xen.org) 
> > Subject: Re: [PATCH v9 05/11] x86/mm: add HYPERVISOR_memory_op to
> > acquire guest resources
> >
> > >>> On 06.10.17 at 14:25,  wrote:
> > > --- a/xen/common/memory.c
> > > +++ b/xen/common/memory.c
> > > @@ -965,6 +965,67 @@ static long xatp_permission_check(struct domain
> > *d, unsigned int space)
> > >  return xsm_add_to_physmap(XSM_TARGET, current->domain, d);
> > >  }
> > >
> > > +#ifdef CONFIG_X86
> > > +static int acquire_resource(const xen_mem_acquire_resource_t *xmar)
> > > +{
> > > +struct domain *d, *currd = current->domain;
> > > +unsigned long mfn_list[2];
> > > +int rc;
> > > +
> > > +if ( xmar->nr_frames == 0 || xmar->pad != 0 )
> > > +return -EINVAL;
> > > +
> > > +if ( xmar->nr_frames > ARRAY_SIZE(mfn_list) )
> > > +return -E2BIG;
> > > +
> > > +d = rcu_lock_domain_by_any_id(xmar->domid);
> > > +if ( d == NULL )
> > > +return -ESRCH;
> > > +
> > > +rc = xsm_domain_memory_map(XSM_TARGET, d);
> >
> > Looking at the description of patch 6 - why is this XSM_TARGET
> > rather than XSM_DM_PRIV?
> 
> Good point. I was using the priv mapping code as a guide, but XSM_DM_PRIV
> is probably the right thing to use in this case.
> 

Actually that's not possible. There is an assertion in xsm_domain_memory_map() 
that the action is XSM_TARGET.

  Paul

>   Paul
> 
> >
> > Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-unstable baseline-only test] 72224: regressions - trouble: blocked/broken/fail/pass

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 72224 xen-unstable real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/72224/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-libvirt-raw 15 guest-start/debian.repeat fail REGR. vs. 72198
 test-amd64-amd64-xl-qemut-win10-i386 16 guest-localmigrate/x10 fail REGR. vs. 
72198

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-rumprun-i386 17 rumprun-demo-xenstorels/xenstorels.repeat fail 
REGR. vs. 72198

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-examine  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64   2 hosts-allocate   broken never pass
 build-arm64-pvops 2 hosts-allocate   broken never pass
 build-arm64-xsm   2 hosts-allocate   broken never pass
 build-arm64-pvops 3 capture-logs broken never pass
 build-arm64-xsm   3 capture-logs broken never pass
 build-arm64   3 capture-logs broken never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail like 72198
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail like 72198
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   like 72198
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail   like 72198
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   like 72198
 test-amd64-amd64-qemuu-nested-intel 17 debian-hvm-install/l1/l2 fail like 72198
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail like 72198
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop  fail like 72198
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail like 72198
 test-armhf-armhf-xl-midway   13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-midway   14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 10 windows-install fail never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 10 windows-installfail never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 13 guest-saverestore   fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemut-win10-i386 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail never pass

version targeted for testing:
 xen  572a78190403e5f2acbd01fa72c35fafe9700169
baseline version:
 xen  dbc4b6e13a5d0dd8967cde7ff7000ab1ed88625e

Last test of basis72198  

Re: [Xen-devel] [PATCH v9 06/11] x86/hvm/ioreq: add a new mappable resource type...

[Paul Durrant]

> -Original Message-
> From: Jan Beulich [mailto:jbeul...@suse.com]
> Sent: 09 October 2017 16:21
> To: Paul Durrant 
> Cc: Andrew Cooper ; Ian Jackson
> ; Stefano Stabellini ; xen-
> de...@lists.xenproject.org; Konrad Rzeszutek Wilk
> ; Tim (Xen.org) 
> Subject: Re: [PATCH v9 06/11] x86/hvm/ioreq: add a new mappable resource
> type...
> 
> >>> On 06.10.17 at 14:25,  wrote:
> > @@ -288,6 +301,61 @@ static int hvm_map_ioreq_gfn(struct
> hvm_ioreq_server *s, bool buf)
> >  return rc;
> >  }
> >
> > +static int hvm_alloc_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
> > +{
> > +struct domain *currd = current->domain;
> > +struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
> > +
> > +if ( iorp->page )
> > +{
> > +/*
> > + * If a guest frame has already been mapped (which may happen
> > + * on demand if hvm_get_ioreq_server_info() is called), then
> > + * allocating a page is not permitted.
> > + */
> > +if ( !gfn_eq(iorp->gfn, INVALID_GFN) )
> > +return -EPERM;
> > +
> > +return 0;
> > +}
> > +
> > +/*
> > + * Allocated IOREQ server pages are assigned to the emulating
> > + * domain, not the target domain. This is because the emulator is
> > + * likely to be destroyed after the target domain has been torn
> > + * down, and we must use MEMF_no_refcount otherwise page
> allocation
> > + * could fail if the emulating domain has already reached its
> > + * maximum allocation.
> > + */
> > +iorp->page = alloc_domheap_page(currd, MEMF_no_refcount);
> 
> Whichever domain you assign the page to, you need to prevent
> it becoming usable as e.g. a page table or descriptor table page.
> IOW I think your missing a get_page_type(..., PGT_writable)
> here, with the put_page() on the free path below then needing
> to become put_page_and_type().

Ok.

> 
> > @@ -784,6 +885,45 @@ int hvm_get_ioreq_server_info(struct domain *d,
> ioservid_t id,
> >  return rc;
> >  }
> >
> > +int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
> > +   unsigned int idx, mfn_t *mfn)
> > +{
> > +struct hvm_ioreq_server *s;
> > +int rc;
> > +
> > +spin_lock_recursive(>arch.hvm_domain.ioreq_server.lock);
> > +
> > +if ( id == DEFAULT_IOSERVID )
> > +return -EOPNOTSUPP;
> > +
> > +s = get_ioreq_server(d, id);
> > +
> > +ASSERT(!IS_DEFAULT(s));
> > +
> > +rc = hvm_ioreq_server_alloc_pages(s);
> > +if ( rc )
> > +goto out;
> > +
> > +if ( idx == 0 )
> 
> switch() ?

Yes, but if idx can exceed 1 in future (which would need to be the case to 
support greater numbers of vcpus) then I guess it may change back.

> 
> > @@ -3866,6 +3867,27 @@ int xenmem_add_to_physmap_one(
> >  return rc;
> >  }
> >
> > +int xenmem_acquire_ioreq_server(struct domain *d, unsigned int id,
> > +unsigned long frame,
> > +unsigned long nr_frames,
> > +unsigned long mfn_list[])
> > +{
> > +unsigned int i;
> > +
> > +for ( i = 0; i < nr_frames; i++ )
> > +{
> > +mfn_t mfn;
> > +int rc = hvm_get_ioreq_server_frame(d, id, frame + i, );
> 
> Coming back to the question of the size of the "frame" interface
> structure field, note how you silently truncate the upper 32 bits
> here.

OK. For this resource type I can't see 64-bits being needed, but I'll carry 
them through.

> 
> > --- a/xen/include/public/memory.h
> > +++ b/xen/include/public/memory.h
> > @@ -609,15 +609,26 @@ struct xen_mem_acquire_resource {
> >  domid_t domid;
> >  /* IN - the type of resource */
> >  uint16_t type;
> > +
> > +#define XENMEM_resource_ioreq_server 0
> > +
> >  /*
> >   * IN - a type-specific resource identifier, which must be zero
> >   *  unless stated otherwise.
> > + *
> > + * type == XENMEM_resource_ioreq_server -> id == ioreq server id
> >   */
> >  uint32_t id;
> >  /* IN - number of (4K) frames of the resource to be mapped */
> >  uint32_t nr_frames;
> >  uint32_t pad;
> > -/* IN - the index of the initial frame to be mapped */
> > +/* IN - the index of the initial frame to be mapped
> > + *
> > + * type == XENMEM_resource_ioreq_server -> frame == 0 -> bufioreq
> > + *   page
> > + * frame == 1 -> ioreq
> > + *   page
> > + */
> 
> Long comment or not I think you want to introduce constants
> for these two numbers.
> 

Yes, that would probably be better although increasing the number of supported 
vcpus may mean that >1 becomes valid in future.

  

Re: [Xen-devel] [PATCH] ARM: sunxi: support more Allwinner SoCs

[Stefano Stabellini]

On Sat, 7 Oct 2017, Andre Przywara wrote:
> So far we only supported the Allwinner A20 SoC. Add support for most
> of the other virtualization capable Allwinner SoCs by:
> - supporting the watchdog in newer (sun8i) SoCs
> - getting the watchdog address from DT
> - adding compatible strings for other 32-bit SoCs
> - adding compatible strings for 64-bit SoCs
> 
> As all 64-bit SoCs support system reset via PSCI, we don't use the
> platform specific reset routine there. Should the 32-bit SoCs start to
> properly support the PSCI 0.2 SYSTEM_RESET call, we will use it for them
> automatically, as we try PSCI first, then fall back to platform reset.
> 
> Signed-off-by: Andre Przywara 

Reviewed-by: Stefano Stabellini 

There were a couple of tabs, which I fixed on commit.


> ---
> Hi,
> 
> this is based on staging, which has the required UART fix.
> Tested on:
> - BananaPi M1 (A20)
> - OrangePi Zero (H2+, which is almost the same as H3)
> - OrangePi PC 2 (H5, arm64)
> - Pine64+ (A64, arm64)
> 
> On the 64-bit boards I could boot into Dom0 prompt.
> I had issues with U-Boot's fdt command on the two 32-bit boards, so couldn't
> inject the Dom0 magic into the DT. But at least Xen booted and reset
> worked with both the "old" and "new" watchdog.
> The newer boards require "clk_ignore_unused" on the Linux command line at
> the moment, I will try to find a more sustainable solution next week.
> Will try to update the Wiki later on.
> 
> Please let me know if this is worth splitting up into multiple patches
> (watchdog address from DT, new watchdog support, arm64 support).
> 
> Many thanks to Awais for the idea and his original patch, and for testing
> this one!
> 
> Cheers,
> Andre.
> 
>  xen/arch/arm/platforms/Makefile |  2 +-
>  xen/arch/arm/platforms/sunxi.c  | 96 
> +++--
>  2 files changed, 85 insertions(+), 13 deletions(-)
> 
> diff --git a/xen/arch/arm/platforms/Makefile b/xen/arch/arm/platforms/Makefile
> index 49fa683780..53a47e48d2 100644
> --- a/xen/arch/arm/platforms/Makefile
> +++ b/xen/arch/arm/platforms/Makefile
> @@ -5,6 +5,6 @@ obj-$(CONFIG_ARM_32) += midway.o
>  obj-$(CONFIG_ARM_32) += omap5.o
>  obj-$(CONFIG_ARM_32) += rcar2.o
>  obj-$(CONFIG_ARM_64) += seattle.o
> -obj-$(CONFIG_ARM_32) += sunxi.o
> +obj-y += sunxi.o
>  obj-$(CONFIG_ARM_64) += xgene-storm.o
>  obj-$(CONFIG_ARM_64) += xilinx-zynqmp.o
> diff --git a/xen/arch/arm/platforms/sunxi.c b/xen/arch/arm/platforms/sunxi.c
> index 0ba7b3d9b4..c8a3e8eec8 100644
> --- a/xen/arch/arm/platforms/sunxi.c
> +++ b/xen/arch/arm/platforms/sunxi.c
> @@ -1,7 +1,7 @@
>  /*
>   * xen/arch/arm/platforms/sunxi.c
>   *
> - * SUNXI (AllWinner A20/A31) specific settings
> + * SUNXI (Allwinner ARM SoCs) specific settings
>   *
>   * Copyright (c) 2013 Citrix Systems.
>   *
> @@ -22,36 +22,103 @@
>  #include 
>  
>  /* Watchdog constants: */
> -#define SUNXI_WDT_BASE0x01c20c90
> -#define SUNXI_WDT_MODE0x04
> -#define SUNXI_WDT_MODEADDR(SUNXI_WDT_BASE + SUNXI_WDT_MODE)
> +#define SUNXI_WDT_MODE_REG0x04
>  #define SUNXI_WDT_MODE_EN (1 << 0)
>  #define SUNXI_WDT_MODE_RST_EN (1 << 1)
>  
> +#define SUNXI_WDT_CONFIG_SYSTEM_RESET   (1 << 0)
> +#define SUNXI_WDOG0_CFG_REG 0x14
> +#define SUNXI_WDOG0_MODE_REG0x18
>  
> -static void sunxi_reset(void)
> +static void __iomem *sunxi_map_watchdog(bool *new_wdt)
>  {
>  void __iomem *wdt;
> +struct dt_device_node *node;
> +paddr_t wdt_start, wdt_len;
> +bool _new_wdt = false;
> +int ret;
> +
> +node = dt_find_compatible_node(NULL, NULL, "allwinner,sun6i-a31-wdt");
> +if ( node )
> + _new_wdt = true;
> +else
> +node = dt_find_compatible_node(NULL, NULL, 
> "allwinner,sun4i-a10-wdt");
> +
> +if ( !node )
> +{
> +dprintk(XENLOG_ERR, "Cannot find matching watchdog node in DT\n");
> +return NULL;
> +}
>  
> -wdt = ioremap_nocache(SUNXI_WDT_MODEADDR & PAGE_MASK, PAGE_SIZE);
> +ret = dt_device_get_address(node, 0, _start, _len);
> +if ( ret )
> +{
> +dprintk(XENLOG_ERR, "Cannot read watchdog register address\n");
> +return NULL;
> +}
> +
> +wdt = ioremap_nocache(wdt_start & PAGE_MASK, PAGE_SIZE);
>  if ( !wdt )
>  {
>  dprintk(XENLOG_ERR, "Unable to map watchdog register!\n");
> -return;
> +return NULL;
>  }
>  
> -/* Enable watchdog to trigger a reset after 500 ms: */
> +if ( new_wdt )
> + *new_wdt = _new_wdt;
> +
> +return wdt + (wdt_start & ~PAGE_MASK);
> +}
> +
> +/* Enable watchdog to trigger a reset after 500 ms */
> +static void sunxi_old_wdt_reset(void __iomem *wdt)
> +{
>  writel(SUNXI_WDT_MODE_EN | SUNXI_WDT_MODE_RST_EN,
> -  wdt + (SUNXI_WDT_MODEADDR & ~PAGE_MASK));
> +   wdt + SUNXI_WDT_MODE_REG);
> +}
> +
> +static void sunxi_new_wdt_reset(void __iomem *wdt)
> +{
> +

Re: [Xen-devel] [PATCH v3 09/12] fuzz/x86_emulate: Make input more compact

[George Dunlap]

> On Oct 10, 2017, at 6:31 PM, Andrew Cooper  wrote:
> 
> On 10/10/17 18:13, George Dunlap wrote:
>> On 10/10/2017 06:11 PM, Andrew Cooper wrote:
>>> On 10/10/17 18:01, George Dunlap wrote:
 On 10/10/2017 05:59 PM, Andrew Cooper wrote:
> On 10/10/17 17:20, George Dunlap wrote:
>> At the moment, AFL reckons that for any given input, 87% of it is
>> completely irrelevant: that is, it can change it as much as it wants
>> but have no impact on the result of the test; and yet it can't remove
>> it.
>> 
>> This is largely because we interpret the blob handed to us as a large
>> struct, including CR values, MSR values, segment registers, and a full
>> cpu_user_regs.
>> 
>> Instead, modify our interpretation to have a "set state" stanza at the
>> front.  Begin by reading a 16-bit value; if it is lower than a certain
>> threshold, set some state according to what byte it is, and repeat.
>> Continue until the byte is above a certain threshold.
>> 
>> This allows AFL to compact any given test case much smaller; to the
>> point where now it reckons there is not a single byte of the test file
>> which becomes irrelevant.  Testing have shown that this option both
>> allows AFL to reach coverage much faster, and to have a total coverage
>> higher than with the old format.
>> 
>> Make this an option (rather than a unilateral change) to enable
>> side-by-side performance comparison of the old and new formats.
>> 
>> Signed-off-by: George Dunlap 
> I am still of the opinion that this is a waste of effort, which would be
> better spent actually removing the irrelevant state in the first place;
> not building an obfuscation algorithm.
> 
> I'm not going to nack the patch because that is probably over the top,
> but I'm not in favour if this change going in.
 Did you look at the evidence I presented, demonstrating that this
 significantly increases the effectiveness of AFL?
>>> I can easily believe that you've found an obfucation algorithm which
>>> does better than the current state layout.
>>> 
>>> I do not believe that any amount of obfuscation will be better than
>>> actually fixing the root cause of the problem; that the current state
>>> really is mostly irrelevant, and can easily be shrunk.
>> Right; well I've already explained why I don't think "obfuscation" is
>> the right term.
> 
> 

> How else would describe it?  You are purposefully hiding the structure
> of the data by doing conditional initialisation based on earlier values.

Consider the following progression:

1. Have a big structure, which includes RAX, and have the corpus be loaded into 
the whole thing (meaning we always fuzz the whole thing).

2. Have  tuples which will set specific state to specific values; 
e.g., .  

3. Have  tuples which write $value into a specific offset of the 
processor state (including RAX).

I would consider #3 a generalized form of #2.


>> For the time being, we have something which improves
>> efficiency;
> 
> How much of this measured efficiently is actually ALF finding paths
> around setup_state() rather than finding new paths in the emulator
> itself?  I can't think of a test which would isolate this properly.

Come now; we’re talking about thousands of unique “tuples” difference between 
“all paths found after 24 hours” for the compact and non-compact versions.  
(Like, 12k vs 8k.)  Do you really think there are 4000 unique tuples inside 
that one function?

We now have a way to generate gcov data from AFL test cases, so that’s a theory 
that can be tested now, if you care to do so.

> 
>> let's check it in now, and in the future if you or someone
>> else finds a way to fix it "properly" we can do that.
> 
> I wouldn't really mind so much if this change didn't make it harder to
> fix the root cause.  As it is, your prerequisite patch prohibits any
> ability to overlay a minimal structure over the fuzzing corpus.

I’d be interested to know what you mean by “a minimal structure” (and 
“architectural values in fuzz_state derived from bitfields in fuzz_corpus” in 
another email).

Consider the following three input formats.

1. We bits to fuzz_state->options for whether we should set the GPRs to fuzzed 
data or not.  If, for instance, the FUZZ_rax bit is set, then we read 8 bytes 
out of the fuzz state into RAX as part of the setup.

2. A format similar to what I have, but we have an explicit enumeration.  Read 
“fuzz_target” byte.  If fuzz_target == FUZZ_rax, then you read 64 bytes and 
write it into regs->rax.

3. The format I have: read an offset, write bytes into that offset.

Clearly there will be similarly ‘compact’ corpus files in all three that 
specify: “Set RAX to 0xfefefefefefefefe, then execute mov RAX, RBX”.  In all 
three formats AFL can efficiently mutate the corpus 

Re: [Xen-devel] [PATCH v9 05/11] x86/mm: add HYPERVISOR_memory_op to acquire guest resources

[Paul Durrant]

> -Original Message-
> From: Jan Beulich [mailto:jbeul...@suse.com]
> Sent: 09 October 2017 15:23
> To: Paul Durrant 
> Cc: Andrew Cooper ; Wei Liu
> ; George Dunlap ; Ian
> Jackson ; Stefano Stabellini
> ; xen-de...@lists.xenproject.org; Konrad Rzeszutek
> Wilk ; Tim (Xen.org) 
> Subject: Re: [PATCH v9 05/11] x86/mm: add HYPERVISOR_memory_op to
> acquire guest resources
> 
> >>> On 06.10.17 at 14:25,  wrote:
> > --- a/xen/common/memory.c
> > +++ b/xen/common/memory.c
> > @@ -965,6 +965,67 @@ static long xatp_permission_check(struct domain
> *d, unsigned int space)
> >  return xsm_add_to_physmap(XSM_TARGET, current->domain, d);
> >  }
> >
> > +#ifdef CONFIG_X86
> > +static int acquire_resource(const xen_mem_acquire_resource_t *xmar)
> > +{
> > +struct domain *d, *currd = current->domain;
> > +unsigned long mfn_list[2];
> > +int rc;
> > +
> > +if ( xmar->nr_frames == 0 || xmar->pad != 0 )
> > +return -EINVAL;
> > +
> > +if ( xmar->nr_frames > ARRAY_SIZE(mfn_list) )
> > +return -E2BIG;
> > +
> > +d = rcu_lock_domain_by_any_id(xmar->domid);
> > +if ( d == NULL )
> > +return -ESRCH;
> > +
> > +rc = xsm_domain_memory_map(XSM_TARGET, d);
> 
> Looking at the description of patch 6 - why is this XSM_TARGET
> rather than XSM_DM_PRIV?

Good point. I was using the priv mapping code as a guide, but XSM_DM_PRIV is 
probably the right thing to use in this case.

  Paul

> 
> Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3] xen/ns16550: Fix ISR lockup on Allwinner uart

[Stefano Stabellini]

On Wed, 4 Oct 2017, Awais Masood wrote:
> This patch fixes an ISR lockup seen on Allwinner uart
> 
> On Allwinner H5, serial driver goes into an infinite loop
> when interrupts are enabled. The reason is a residual
> "busy detect" interrupt. Since the condition UART_IIR_NOINT
> will not be true unless this interrupt is cleared, the
> interrupt handler will remain locked up in this while loop.
> 
> A HW quirk fix was previously added for designware uart under
> commit:
> 50417cd978aa54930d065ac1f139f935d14af76d
> 
> It checks for a busy condition during setup and clears the
> condition by reading UART_USR register.
> 
> On Allwinner hardware, the "busy detect" condition occurs
> later because an LCR write is performed during setup 'after'
> this clear and if uart is busy, the "busy detect" condition
> will trigger again and cause the ISR lockup.
> 
> To solve this problem, the same UART_USR read operation needs
> to be performed within the interrupt handler to clear this
> condition.
> 
> Linux dw 8250 driver also handles this condition within
> interrupt handler
> http://elixir.free-electrons.com/linux/latest/source/drivers/tty/serial/8250/8250_dw.c#L233
> 
> Tested on Orange Pi PC2 (H5). This issue is seen on H3
> as well and the same fix works.
> 
> Signed-off-by: Awais Masood 

Acked-by: Stefano Stabellini 


> ---
> 
> CC: Andrew Cooper 
> CC: George Dunlap 
> CC: Ian Jackson 
> CC: Jan Beulich 
> CC: Konrad Rzeszutek Wilk 
> CC: Stefano Stabellini 
> CC: Tim Deegan 
> CC: Wei Liu 
> 
> Changes since v2
>  - Updated comments to clarify that fix is for Allwinner hardware
>  - Removed ns16550 prefix from local function
> 
> Changes since v1
>  - Common quirk fix code moved to a helper function
>  - Patch description improved with earlier commit link
> ---
>  xen/drivers/char/ns16550.c | 38 +-
>  1 file changed, 29 insertions(+), 9 deletions(-)
> 
> diff --git a/xen/drivers/char/ns16550.c b/xen/drivers/char/ns16550.c
> index 6ab5ec3..e0f8199 100644
> --- a/xen/drivers/char/ns16550.c
> +++ b/xen/drivers/char/ns16550.c
> @@ -505,6 +505,23 @@ static int ns16550_ioport_invalid(struct ns16550 *uart)
>  return ns_read_reg(uart, UART_IER) == 0xff;
>  }
>  
> +static void handle_dw_usr_busy_quirk(struct ns16550 *uart)
> +{
> +if ( uart->dw_usr_bsy &&
> + (ns_read_reg(uart, UART_IIR) & UART_IIR_BSY) == UART_IIR_BSY )
> +{
> +/* DesignWare 8250 detects if LCR is written while the UART is
> + * busy and raises a "busy detect" interrupt. Read the UART
> + * Status Register to clear this state.
> + *
> + * Allwinner/sunxi UART hardware is similar to DesignWare 8250
> + * and also contains a "busy detect" interrupt. So this quirk
> + * fix will also be used for Allwinner UART.
> + */
> +ns_read_reg(uart, UART_USR);
> +}
> +}
> +
>  static void ns16550_interrupt(
>  int irq, void *dev_id, struct cpu_user_regs *regs)
>  {
> @@ -521,6 +538,16 @@ static void ns16550_interrupt(
>  serial_tx_interrupt(port, regs);
>  if ( lsr & UART_LSR_DR )
>  serial_rx_interrupt(port, regs);
> +
> +/* A "busy-detect" condition is observed on Allwinner/sunxi UART
> + * after LCR is written during setup. It needs to be cleared at
> + * this point or UART_IIR_NOINT will never be set and this loop
> + * will continue forever.
> + *
> + * This state can be cleared by calling the dw_usr_busy quirk
> + * handler that resolves "busy-detect" for  DesignWare uart.
> + */
> +handle_dw_usr_busy_quirk(uart);
>  }
>  }
>  
> @@ -623,15 +650,8 @@ static void ns16550_setup_preirq(struct ns16550 *uart)
>  /* No interrupts. */
>  ns_write_reg(uart, UART_IER, 0);
>  
> -if ( uart->dw_usr_bsy &&
> - (ns_read_reg(uart, UART_IIR) & UART_IIR_BSY) == UART_IIR_BSY )
> -{
> -/* DesignWare 8250 detects if LCR is written while the UART is
> - * busy and raises a "busy detect" interrupt. Read the UART
> - * Status Register to clear this state.
> - */
> -ns_read_reg(uart, UART_USR);
> -}
> +/* Handle the DesignWare 8250 'busy-detect' quirk. */
> +handle_dw_usr_busy_quirk(uart);
>  
>  /* Line control and baud-rate generator. */
>  ns_write_reg(uart, UART_LCR, lcr | UART_LCR_DLAB);
> -- 
> 2.7.4
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [linux-4.9 bisection] complete test-amd64-amd64-xl-pvh-intel

[osstest service owner]

branch xen-unstable
xenbranch xen-unstable
job test-amd64-amd64-xl-pvh-intel
testid guest-start

Tree: linux 
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  xen git://xenbits.xen.org/xen.git
  Bug introduced:  c7dfe4ac58dd9c8678126b78da961b233a49f3f9
  Bug not present: 3c44f8ed44ab559c7e5b58316751bea53adfd83b
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/114307/


  commit c7dfe4ac58dd9c8678126b78da961b233a49f3f9
  Author: Roger Pau Monne 
  Date:   Fri Sep 22 16:25:06 2017 +0100
  
  xl: introduce a domain type option
  
  Introduce a new type option to xl configuration files in order to
  specify the domain type. This supersedes the current builder option.
  
  The new option is documented in the xl.cfg man page, and the previous
  builder option is marked as deprecated.
  
  Signed-off-by: Roger Pau Monné 
  Acked-by: Ian Jackson 


For bisection revision-tuple graph see:
   
http://logs.test-lab.xenproject.org/osstest/results/bisect/linux-4.9/test-amd64-amd64-xl-pvh-intel.guest-start.html
Revision IDs in each graph node refer, respectively, to the Trees above.


Running cs-bisection-step 
--graph-out=/home/logs/results/bisect/linux-4.9/test-amd64-amd64-xl-pvh-intel.guest-start
 --summary-out=tmp/114307.bisection-summary --basis-template=114036 
--blessings=real,real-bisect linux-4.9 test-amd64-amd64-xl-pvh-intel guest-start
Searching for failure / basis pass:
 114185 fail [host=chardonnay1] / 114036 [host=elbling1] 113872 [host=italia1] 
113860 [host=chardonnay0] 113736 [host=fiano1] 113680 [host=nobling1] 113655 
[host=godello1] 113640 [host=baroque1] 113620 [host=nobling0] 113479 
[host=huxelrebe0] 113458 [host=godello0] 113425 [host=elbling0] 113202 
[host=italia0] 113168 [host=chardonnay0] 113154 [host=huxelrebe1] 112193 
[host=godello0] 112117 [host=godello1] 112086 [host=nobling0] 111883 
[host=elbling0] 111843 [host=baroque0] 111809 [host=huxelrebe0] 111786 
[host=fiano1] 111763 [host=italia1] 111737 [host=chardonnay0] 111411 
[host=baroque1] 111336 [host=nobling0] 111294 [host=elbling1] 111228 
[host=baroque0] 84 [host=fiano0] 110557 [host=italia1] 110545 
[host=italia0] 110535 [host=nobling0] 110513 [host=huxelrebe1] 110456 
[host=fiano0] 110423 [host=elbling0] 110396 [host=baroque0] 110371 
[host=godello0] 110336 [host=elbling1] 110266 [host=nobling0] 110200 
[host=chardonnay0] 110151 [host=godello1] 110112 ok.
Failure / basis pass flights: 114185 / 110112
(tree with no url: minios)
(tree with no url: ovmf)
(tree with no url: seabios)
Tree: linux 
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git
Latest f37eb7b586f1dd24a86c50278c65322fc6787722 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
8051789e982499050680a26febeada7467e18a8d 
7434775abf8fb2ca3b9e805d30656f4da8c08816 
dbc4b6e13a5d0dd8967cde7ff7000ab1ed88625e
Basis pass f1aa865ae5d4608cbfbb02f42baa1ef5ed95fce2 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
8051789e982499050680a26febeada7467e18a8d 
e97832ec6b2a7ddd48b8e6d1d848ffdfee6a31c7 
3d2010f9ffeacc8836811420460e15f2c1233695
Generating revisions with ./adhoc-revtuple-generator  
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git#f1aa865ae5d4608cbfbb02f42baa1ef5ed95fce2-f37eb7b586f1dd24a86c50278c65322fc6787722
 
git://xenbits.xen.org/osstest/linux-firmware.git#c530a75c1e6a472b0eb9558310b518f0dfcd8860-c530a75c1e6a472b0eb9558310b518f0dfcd8860
 
git://xenbits.xen.org/qemu-xen-traditional.git#8051789e982499050680a26febeada7467e18a8d-8051789e982499050680a26febeada7467e18a8d
 
git://xenbits.xen.org/qemu-xen.git#e97832ec6b2a7ddd48b8e6d1d848ffdfee6a31c7-7434775abf8fb2ca3b9e805d30656f4da8c08816
 
git://xenbits.xen.org/xen.git#3d2010f9ffeacc8836811420460e15f2c1233695-dbc4b6e13a5d0dd8967cde7ff7000ab1ed88625e
adhoc-revtuple-generator: tree discontiguous: linux-stable
adhoc-revtuple-generator: tree discontiguous: qemu-xen
Loaded 1003 nodes in revision graph
Searching for test results:
 110112 pass f1aa865ae5d4608cbfbb02f42baa1ef5ed95fce2 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
8051789e982499050680a26febeada7467e18a8d 
e97832ec6b2a7ddd48b8e6d1d848ffdfee6a31c7 
3d2010f9ffeacc8836811420460e15f2c1233695
 110050 [host=nobling1]
 110151 [host=godello1]
 110200 [host=chardonnay0]
 110336 [host=elbling1]
 110266 [host=nobling0]
 110371 [host=godello0]
 110396 [host=baroque0]
 110456 

Re: [Xen-devel] [PATCH v2 0/6] Allow setting up shared memory areas between VMs from xl config files

[Stefano Stabellini]

On Sun, 27 Aug 2017, Zhongze Liu wrote:
> This series implements the new xl config entry proposed in [1]. Users can use
> the new config entry to statically setup shared memory areas among VMs that
> don't have grant table support so that they could communicate with each other
> through the static shared memory areas.
> 
> [1] Proposla to allow setting up shared memory areas between VMs from xl 
> config file:
>   https://lists.xen.org/archives/html/xen-devel/2017-08/msg03242.html
> 
> v2:
>   * fixed several code style issues.
>   * introduce MMU__SHARE_MEM in flask av permissions, and add a check to this.
> permission in the flask hook for xsm_map_gmfn_foreign.
>   * support rolling back during creation on partial failure.
>   * refcounting the sshm path instead of using "alive" and "zombie" to label 
> the
> master and counting the slaves.

Hey Zhongze,

any plans on sending an update of this series?


> Cheers,
> 
> Zhongze Liu (6):
>   libxc: add xc_domain_remove_from_physmap to wrap
> XENMEM_remove_from_physmap
>   libxl: introduce a new structure to represent static shared memory
> regions
>   libxl:xl: add parsing code to parse "libxl_static_sshm" from xl config
> files
>   xsm: flask: change the dummy xsm policy and flask hook for
> map_gmfn_foregin
>   libxl: support mapping static shared memory areas during domain
> creation
>   libxl: support unmapping static shared memory areas during domain
> destruction
> 
>  tools/flask/policy/modules/xen.if   |   2 +
>  tools/libxc/include/xenctrl.h   |   4 +
>  tools/libxc/xc_domain.c |  11 +
>  tools/libxl/Makefile|   4 +-
>  tools/libxl/libxl.h |   4 +
>  tools/libxl/libxl_arch.h|   6 +
>  tools/libxl/libxl_arm.c |  15 ++
>  tools/libxl/libxl_create.c  |  27 ++
>  tools/libxl/libxl_domain.c  |   5 +
>  tools/libxl/libxl_internal.h|  15 ++
>  tools/libxl/libxl_sshm.c| 480 
> 
>  tools/libxl/libxl_types.idl |  34 ++-
>  tools/libxl/libxl_x86.c |  18 ++
>  tools/libxl/libxlu_sshm.c   | 210 
>  tools/libxl/libxlutil.h |   6 +
>  tools/xl/xl_parse.c |  24 +-
>  xen/arch/arm/mm.c   |   2 +-
>  xen/arch/x86/mm/p2m.c   |   2 +-
>  xen/include/xsm/dummy.h |   8 +-
>  xen/include/xsm/xsm.h   |   7 +-
>  xen/xsm/flask/hooks.c   |  10 +-
>  xen/xsm/flask/policy/access_vectors |   4 +
>  22 files changed, 883 insertions(+), 15 deletions(-)
>  create mode 100644 tools/libxl/libxl_sshm.c
>  create mode 100644 tools/libxl/libxlu_sshm.c
> 
> -- 
> 2.14.0
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 3/3 v4] xenfb: Add [feature|request]-raw-pointer

[Stefano Stabellini]

On Mon, 2 Oct 2017, Anthony PERARD wrote:
> On Tue, Sep 26, 2017 at 02:43:39PM +, Owen Smith wrote:
> > Writes "feature-raw-pointer" during init to indicate the backend
> > can pass raw unscaled values for absolute axes to the frontend.
> > Frontends set "request-raw-pointer" to indicate the backend should
> > not attempt to scale absolute values to console size.
> > "request-raw-pointer" is only valid if "request-abs-pointer" is
> > also set. Raw unscaled pointer values are in the range [0, 0x7fff]
> > 
> > Signed-off-by: Owen Smith 
> 
> Hi Owen,
> 
> Why did you remove the following from the commit description?
> > "feature-raw-pointer" and "request-raw-pointer" added to Xen
> > header in commit 7868654ff7fe5e4a2eeae2b277644fa884a5031e
> 
> I think that with it, you could have kept stefano's reviewed-by tag.

Hi Anthony,

Have you tested this series with a few of different guests? Do you
consider it safe to merge? If so, we can send it upstream (either via
xen or via ui as Gerd kindly offered). 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH FOR-4.10] xen/arm: guest_walk: Fix check again the IPS

[Stefano Stabellini]

On Tue, 10 Oct 2017, Julien Grall wrote:
> The function get_ipa_output_size is check whether the input size
> configured by the guest is valid and will return it.
> 
> The check is done with the IPS already shifted against
> TCR_EL1_IPS_48_BIT. However the constant has been defined with the
> shift included, resulting the check always been false.
> 
> Fix it by doing the check on the non-shifted value.
> 
> This was introduced by commit 7d623b358a "arm/mem_access: Add long-descriptor
> based gpt" introduced software page-table walk for stage-1.
> 
> Coverity-ID: 1457707
> Signed-off-by: Julien Grall 
> 
> ---
> 
> Cc: Sergej Proskurin 
> ---
>  xen/arch/arm/guest_walk.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/xen/arch/arm/guest_walk.c b/xen/arch/arm/guest_walk.c
> index c38bedcf65..a6de325572 100644
> --- a/xen/arch/arm/guest_walk.c
> +++ b/xen/arch/arm/guest_walk.c
> @@ -185,7 +185,7 @@ static int guest_walk_sd(const struct vcpu *v,
>  static int get_ipa_output_size(struct domain *d, register_t tcr,
> unsigned int *output_size)
>  {
> -unsigned int ips;
> +register_t ips;
>  
>  static const unsigned int ipa_sizes[7] = {
>  TCR_EL1_IPS_32_BIT_VAL,
> @@ -200,7 +200,7 @@ static int get_ipa_output_size(struct domain *d, 
> register_t tcr,
>  if ( is_64bit_domain(d) )
>  {
>  /* Get the intermediate physical address size. */
> -ips = (tcr & TCR_EL1_IPS_MASK) >> TCR_EL1_IPS_SHIFT;
> +ips = tcr & TCR_EL1_IPS_MASK;
>  
>  /*
>   * Return an error on reserved IPA output-sizes and if the IPA
> @@ -211,7 +211,7 @@ static int get_ipa_output_size(struct domain *d, 
> register_t tcr,
>  if ( ips > TCR_EL1_IPS_48_BIT )
>  return -EFAULT;
>  
> -*output_size = ipa_sizes[ips];
> +*output_size = ipa_sizes[ips >> TCR_EL1_IPS_SHIFT];
>  }
>  else
>  *output_size = TCR_EL1_IPS_40_BIT_VAL;

on arm32, I get:

guest_walk.c: In function 'get_ipa_output_size':
guest_walk.c:214:9: error: right shift count >= width of type [-Werror]
 *output_size = ipa_sizes[ips >> TCR_EL1_IPS_SHIFT];
 ^
cc1: all warnings being treated as errors

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v9 05/11] x86/mm: add HYPERVISOR_memory_op to acquire guest resources

[Paul Durrant]

> -Original Message-
> From: Jan Beulich [mailto:jbeul...@suse.com]
> Sent: 09 October 2017 14:06
> To: Paul Durrant 
> Cc: Andrew Cooper ; Wei Liu
> ; George Dunlap ; Ian
> Jackson ; Stefano Stabellini
> ; xen-de...@lists.xenproject.org; Konrad Rzeszutek
> Wilk ; Tim (Xen.org) 
> Subject: Re: [PATCH v9 05/11] x86/mm: add HYPERVISOR_memory_op to
> acquire guest resources
> 
> >>> On 06.10.17 at 14:25,  wrote:
> > @@ -395,6 +397,39 @@ int compat_memory_op(unsigned int cmd,
> XEN_GUEST_HANDLE_PARAM(void) compat)
> >  }
> >  #endif
> >
> > +case XENMEM_acquire_resource:
> > +{
> > +xen_ulong_t *gmfn_list = (xen_ulong_t *)(nat.mar + 1);
> > +
> > +if ( copy_from_guest(, compat, 1) ||
> > + !compat_handle_okay(cmp.mar.gmfn_list,
> > + cmp.mar.nr_frames) )
> > +return -EFAULT;
> > +
> > +if ( sizeof(*gmfn_list) * cmp.mar.nr_frames >
> > + COMPAT_ARG_XLAT_SIZE - sizeof(*nat.mar) )
> > +return -E2BIG;
> 
> With the actual handler accepting no more than 2 entries this is
> certainly good enough for now, but since larger arrays could be
> handled here perhaps a comment would be helpful to clarify
> why this is sufficient atm.

Ok.

> 
> > @@ -535,6 +570,23 @@ int compat_memory_op(unsigned int cmd,
> XEN_GUEST_HANDLE_PARAM(void) compat)
> >  rc = -EFAULT;
> >  break;
> >
> > +case XENMEM_acquire_resource:
> > +{
> > +xen_ulong_t *gmfn_list = (xen_ulong_t *)(nat.mar + 1);
> > +
> > +for ( i = 0; i < cmp.mar.nr_frames; i++ )
> > +{
> > +compat_ulong_t gmfn = gmfn_list[i];
> > +
> > +if ( gmfn != gmfn_list[i] )
> > +return -ERANGE;
> > +
> > +if ( __copy_to_compat_offset(cmp.mar.gmfn_list, i,
> > + , 1) )
> > +return -EFAULT;
> 
> While I consider it acceptable to leave kind of inconsistent state in
> this latter case (as it's under guest control to avoid the situation),
> I'm not sure the -ERANGE return above wouldn't better be
> accompanied by undoing the operation. Undoing in both cases
> would become imperative once set_foreign_p2m_entry() acquires
> proper page references.

Ok. I'll try to make sure things get left in a consistent state.

> 
> > --- a/xen/common/memory.c
> > +++ b/xen/common/memory.c
> > @@ -965,6 +965,67 @@ static long xatp_permission_check(struct domain
> *d, unsigned int space)
> >  return xsm_add_to_physmap(XSM_TARGET, current->domain, d);
> >  }
> >
> > +#ifdef CONFIG_X86
> 
> Could you try to (a) have only a single such #ifdef and (b) reduce
> its scope as much as possible? Even if for now the code is dead on
> ARM, making sure it continues to compile there would be helpful.
> 

Ok. I don't have an arm machine to build on so I can't tell if anything is 
broken though.

> > +static int acquire_resource(const xen_mem_acquire_resource_t *xmar)
> > +{
> > +struct domain *d, *currd = current->domain;
> > +unsigned long mfn_list[2];
> > +int rc;
> > +
> > +if ( xmar->nr_frames == 0 || xmar->pad != 0 )
> > +return -EINVAL;
> > +
> > +if ( xmar->nr_frames > ARRAY_SIZE(mfn_list) )
> > +return -E2BIG;
> > +
> > +d = rcu_lock_domain_by_any_id(xmar->domid);
> > +if ( d == NULL )
> > +return -ESRCH;
> > +
> > +rc = xsm_domain_memory_map(XSM_TARGET, d);
> > +if ( rc )
> > +goto out;
> > +
> > +switch ( xmar->type )
> > +{
> > +default:
> > +rc = -EOPNOTSUPP;
> > +break;
> > +}
> > +
> > +if ( rc )
> > +goto out;
> > +
> > +if ( !paging_mode_translate(currd) )
> > +{
> > +if ( copy_to_guest_offset(xmar->gmfn_list, 0, mfn_list,
> > +  xmar->nr_frames) )
> 
> Just copy_to_guest()?
> 

Ok, if that is preferable.

> > +rc = -EFAULT;
> > +}
> > +else
> > +{
> > +unsigned int i;
> > +
> > +for ( i = 0; i < xmar->nr_frames; i++ )
> > +{
> > +xen_pfn_t gfn;
> > +
> > +rc = -EFAULT;
> > +if ( copy_from_guest_offset(, xmar->gmfn_list, i, 1) )
> > +goto out;
> > +
> > +rc = set_foreign_p2m_entry(currd, gfn, _mfn(mfn_list[i]));
> > +if ( rc )
> > +goto out;
> 
> Perhaps partial success indication would be necessary here, so
> the caller knows what to undo later. Or alternatively (just like
> said for the compat wrapper) you may want/need to clean up
> yourself.

OK.

> 
> > --- a/xen/include/public/memory.h
> > +++ 

Re: [Xen-devel] [PATCH v9 04/11] public: xen.h: add definitions for UUID handling

[Stefano Stabellini]

On Tue, 10 Oct 2017, Volodymyr Babchuk wrote:
> Added type xen_uuid_t. This type represents UUID as an array of 16
> bytes in big endian format.
> 
> Added macro XEN_DEFINE_UUID that constructs UUID in the usual way:
> 
>  XEN_DEFINE_UUID(0x00112233, 0x4455, 0x6677, 0x8899,
>   0xaa, 0xbb, 0xcc, 0xdd, 0xee, 0xff)
> 
> will construct UUID 00112233-4455-6677-8899-aabbccddeeff presented as
>  {0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88,
>   0x99, 0xaa, 0xbb, 0xcc, 0xdd, 0xee, 0xff}
> 
> NB: We define a new structure here rather than re-using EFI_GUID.
> EFI_GUID uses a Microsoft-style encoding which, among other things,
> mixes little-endian and big-endian. The structure defined in this
> patch, unlike EFI_GUID, is compatible with the Linux kernel and libuuid.
> 
> Signed-off-by: Volodymyr Babchuk 
> ---
> 
>  * Fixed code formatting
>  * Added parenthess around (xen_uuid_t)XEN_DEFINE_UUID_(..)
> 
> ---
>  xen/include/public/xen.h | 33 +
>  1 file changed, 33 insertions(+)
> 
> diff --git a/xen/include/public/xen.h b/xen/include/public/xen.h
> index 2ac6b1e..3d5edc6 100644
> --- a/xen/include/public/xen.h
> +++ b/xen/include/public/xen.h
> @@ -930,6 +930,39 @@ __DEFINE_XEN_GUEST_HANDLE(uint16, uint16_t);
>  __DEFINE_XEN_GUEST_HANDLE(uint32, uint32_t);
>  __DEFINE_XEN_GUEST_HANDLE(uint64, uint64_t);
>  
> +typedef struct {
> +uint8_t a[16];
> +} xen_uuid_t;
> +
> +/*
> + * XEN_DEFINE_UUID(0x00112233, 0x4455, 0x6677, 0x8899,
> + * 0xaa, 0xbb, 0xcc, 0xdd, 0xee, 0xff)
> + * will construct UUID 00112233-4455-6677-8899-aabbccddeeff presented as
> + * {0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88,
> + * 0x99, 0xaa, 0xbb, 0xcc, 0xdd, 0xee, 0xff};
> + *
> + * NB: This is compatible with Linux kernel and with libuuid, but it is not
> + * compatible with Microsoft, as they use mixed-endian encoding (some
> + * components are little-endian, some are big-endian).
> + */
> +#define XEN_DEFINE_UUID_(a, b, c, d, e1, e2, e3, e4, e5, e6)\
> +{{((a) >> 24) & 0xFF, ((a) >> 16) & 0xFF,   \
> +  ((a) >>  8) & 0xFF, ((a) >>  0) & 0xFF,   \
> +  ((b) >>  8) & 0xFF, ((b) >>  0) & 0xFF,   \
> +  ((c) >>  8) & 0xFF, ((c) >>  0) & 0xFF,   \
> +  ((d) >>  8) & 0xFF, ((d) >>  0) & 0xFF,   \
> +e1, e2, e3, e4, e5, e6}}
> +
> +/* Compound literals are supported in C99 and later. */
> +#if defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L
> +#define XEN_DEFINE_UUID(a, b, c, d, e1, e2, e3, e4, e5, e6) \
> +((xen_uuid_t)XEN_DEFINE_UUID_(a, b, c, d, e1, e2, e3, e4, e5, e6))
> +#else
> +#define XEN_DEFINE_UUID(a, b, c, d, e1, e2, e3, e4, e5, e6) \
> +XEN_DEFINE_UUID_(a, b, c, d, e1, e2, e3, e4, e5, e6)
> +
> +#endif /* defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L */
> +
>  #endif /* !__ASSEMBLY__ */
>  
>  /* Default definitions for macros used by domctl/sysctl. */

This looks good to me, but I would like to get Jan's opinion on this.
Ideally we would commit the series tomorrow before the code freeze.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v3 1/5] xen:rtds: towards work conserving RTDS

[Meng Xu]

Make RTDS scheduler work conserving without breaking the real-time guarantees.

VCPU model:
Each real-time VCPU is extended to have an extratime flag
and a priority_level field.
When a VCPU's budget is depleted in the current period,
if it has extratime flag set,
its priority_level will increase by 1 and its budget will be refilled;
othewrise, the VCPU will be moved to the depletedq.

Scheduling policy is modified global EDF:
A VCPU v1 has higher priority than another VCPU v2 if
(i) v1 has smaller priority_leve; or
(ii) v1 has the same priority_level but has a smaller deadline

Queue management:
Run queue holds VCPUs with extratime flag set and VCPUs with
remaining budget. Run queue is sorted in increasing order of VCPUs priorities.
Depleted queue holds VCPUs which have extratime flag cleared and depleted 
budget.
Replenished queue is not modified.

Distribution of spare bandwidth
Spare bandwidth is distributed among all VCPUs with extratime flag set,
proportional to these VCPUs utilizations

Signed-off-by: Meng Xu 

---
Changes from v2
Explain how to distribute spare bandwidth in commit log
Minor change in has_extratime function without functionality change.

Changes from v1
Change XEN_DOMCTL_SCHED_RTDS_extratime to XEN_DOMCTL_SCHEDRT_extra as
suggested by Dario

Changes from RFC v1
Rewording comments and commit message
Remove is_work_conserving field from rt_vcpu structure
Use one bit in VCPU's flag to indicate if a VCPU will have extra time
Correct comments style
---
 xen/common/sched_rt.c   | 90 ++---
 xen/include/public/domctl.h |  4 ++
 2 files changed, 80 insertions(+), 14 deletions(-)

diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c
index 5c51cd9..b770287 100644
--- a/xen/common/sched_rt.c
+++ b/xen/common/sched_rt.c
@@ -49,13 +49,15 @@
  * A PCPU is feasible if the VCPU can run on this PCPU and (the PCPU is idle or
  * has a lower-priority VCPU running on it.)
  *
- * Each VCPU has a dedicated period and budget.
+ * Each VCPU has a dedicated period, budget and a extratime flag
  * The deadline of a VCPU is at the end of each period;
  * A VCPU has its budget replenished at the beginning of each period;
  * While scheduled, a VCPU burns its budget.
  * The VCPU needs to finish its budget before its deadline in each period;
  * The VCPU discards its unused budget at the end of each period.
- * If a VCPU runs out of budget in a period, it has to wait until next period.
+ * When a VCPU runs out of budget in a period, if its extratime flag is set,
+ * the VCPU increases its priority_level by 1 and refills its budget; 
otherwise,
+ * it has to wait until next period.
  *
  * Each VCPU is implemented as a deferable server.
  * When a VCPU has a task running on it, its budget is continuously burned;
@@ -63,7 +65,8 @@
  *
  * Queue scheme:
  * A global runqueue and a global depletedqueue for each CPU pool.
- * The runqueue holds all runnable VCPUs with budget, sorted by deadline;
+ * The runqueue holds all runnable VCPUs with budget,
+ * sorted by priority_level and deadline;
  * The depletedqueue holds all VCPUs without budget, unsorted;
  *
  * Note: cpumask and cpupool is supported.
@@ -151,6 +154,14 @@
 #define RTDS_depleted (1<<__RTDS_depleted)
 
 /*
+ * RTDS_extratime: Can the vcpu run in the time that is
+ * not part of any real-time reservation, and would therefore
+ * be otherwise left idle?
+ */
+#define __RTDS_extratime4
+#define RTDS_extratime (1<<__RTDS_extratime)
+
+/*
  * rt tracing events ("only" 512 available!). Check
  * include/public/trace.h for more details.
  */
@@ -201,6 +212,8 @@ struct rt_vcpu {
 struct rt_dom *sdom;
 struct vcpu *vcpu;
 
+unsigned priority_level;
+
 unsigned flags;  /* mark __RTDS_scheduled, etc.. */
 };
 
@@ -245,6 +258,11 @@ static inline struct list_head *rt_replq(const struct 
scheduler *ops)
 return _priv(ops)->replq;
 }
 
+static inline bool has_extratime(const struct rt_vcpu *svc)
+{
+return svc->flags & RTDS_extratime;
+}
+
 /*
  * Helper functions for manipulating the runqueue, the depleted queue,
  * and the replenishment events queue.
@@ -274,6 +292,21 @@ vcpu_on_replq(const struct rt_vcpu *svc)
 }
 
 /*
+ * If v1 priority >= v2 priority, return value > 0
+ * Otherwise, return value < 0
+ */
+static s_time_t
+compare_vcpu_priority(const struct rt_vcpu *v1, const struct rt_vcpu *v2)
+{
+int prio = v2->priority_level - v1->priority_level;
+
+if ( prio == 0 )
+return v2->cur_deadline - v1->cur_deadline;
+
+return prio;
+}
+
+/*
  * Debug related code, dump vcpu/cpu information
  */
 static void
@@ -303,6 +336,7 @@ rt_dump_vcpu(const struct scheduler *ops, const struct 
rt_vcpu *svc)
 cpulist_scnprintf(keyhandler_scratch, sizeof(keyhandler_scratch), mask);
 printk("[%5d.%-2u] cpu %u, (%"PRI_stime", %"PRI_stime"),"
" cur_b=%"PRI_stime" cur_d=%"PRI_stime" last_start=%"PRI_stime"\n"
+   " \t\t 

[Xen-devel] [PATCH v3 0/5] Towards work-conserving RTDS

[Meng Xu]

This series of patches make RTDS scheduler work-conserving
without breaking real-time guarantees.
VCPUs with extratime flag set can get extra time
from the unreserved system resource.
System administrators can decide which VCPUs have extratime flag set.

Example:
Set the extratime bit of all VCPUs of domain 1:
# xl sched-rtds -d 1 -v all -p 1 -b 2000 -e 1
Each VCPU of domain 1 will be guaranteed to have 2000ms every 1ms
(if the system is schedulable).
If there is a CPU having no work to do,
domain 1's VCPUs will be scheduled onto the CPU,
even though the VCPUs have got 2000ms in 1ms.

Clear the extra bit of all VCPUs of domain 1:
# xl sched-rtds -d 1 -v all -p 1 -b 2000 -e 0

Set/Clear the extratime bit of one specific VCPU of domain 1:
# xl sched-rtds -d 1 -v 1 -p 1 -b 2000 -e 1
# xl sched-rtds -d 1 -v 1 -p 1 -b 2000 -e 0


The original design of the work-conserving RTDS was discussed at
https://www.mail-archive.com/xen-devel@lists.xen.org/msg77150.html

The first version was discussed at
https://www.mail-archive.com/xen-devel@lists.xen.org/msg117361.html

The second version was discussed at
https://www.mail-archive.com/xen-devel@lists.xen.org/msg120618.html

The series of patch can be found at github:
https://github.com/PennPanda/RT-Xen
under the branch:
xenbits/rtds/work-conserving-v3.1

Changes from v2
Sanity check the input of -e option which can only be 0 or 1
Set -e to 1 by default if 3rd party library does not set -e option
Set vcpu extratime in sched_rtds_vcpu_get function function, which
fixes a bug in previous version.
Change EXTRATIME to Extratime in the xl output

Changes from v1
Change XEN_DOMCTL_SCHED_RTDS_extratime to XEN_DOMCTL_SCHEDRT_extra
Revise xentrace, xenalyze, and docs
Add LIBXL_HAVE_SCHED_RTDS_VCPU_EXTRA symbol in libxl.h

Changes from RFC v1
Merge changes in sched_rt.c into one patch;
Minor change in variable name and comments.

Signed-off-by: Meng Xu 

[PATCH v3 1/5] xen:rtds: towards work conserving RTDS
[PATCH v3 2/5] libxl: enable per-VCPU extratime flag for RTDS
[PATCH v3 3/5] xl: enable per-VCPU extratime flag for RTDS
[PATCH v3 4/5] xentrace: enable per-VCPU extratime flag for RTDS
[PATCH v3 5/5] docs: enable per-VCPU extratime flag for RTDS

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v3 3/5] xl: enable per-VCPU extratime flag for RTDS

[Meng Xu]

Change main_sched_rtds and related output functions to support
per-VCPU extratime flag.

Signed-off-by: Meng Xu 

---
Changes from v2
Validate the -e option input that can only be 0 or 1
Update docs/man/xl.pod.1.in
Change EXTRATIME to Extratime

Changes from v1
No change because we agree on using -e 0/1 option to
set if a vcpu will get extra time or not

Changes from RFC v1
Changes work_conserving flag to extratime flag
---
 docs/man/xl.pod.1.in   | 59 +--
 tools/xl/xl_cmdtable.c |  3 ++-
 tools/xl/xl_sched.c| 62 +++---
 3 files changed, 78 insertions(+), 46 deletions(-)

diff --git a/docs/man/xl.pod.1.in b/docs/man/xl.pod.1.in
index cd8bb1c..486a24f 100644
--- a/docs/man/xl.pod.1.in
+++ b/docs/man/xl.pod.1.in
@@ -1117,11 +1117,11 @@ as B<--ratelimit_us> in B
 Set or get rtds (Real Time Deferrable Server) scheduler parameters.
 This rt scheduler applies Preemptive Global Earliest Deadline First
 real-time scheduling algorithm to schedule VCPUs in the system.
-Each VCPU has a dedicated period and budget.
-VCPUs in the same domain have the same period and budget.
+Each VCPU has a dedicated period, budget and extratime.
 While scheduled, a VCPU burns its budget.
 A VCPU has its budget replenished at the beginning of each period;
 Unused budget is discarded at the end of each period.
+A VCPU with extratime set gets extra time from the unreserved system resource.
 
 B
 
@@ -1145,6 +1145,11 @@ Period of time, in microseconds, over which to replenish 
the budget.
 Amount of time, in microseconds, that the VCPU will be allowed
 to run every period.
 
+=item B<-e Extratime>, B<--extratime=Extratime>
+
+Binary flag to decide if the VCPU will be allowed to get extra time from
+the unreserved system resource.
+
 =item B<-c CPUPOOL>, B<--cpupool=CPUPOOL>
 
 Restrict output to domains in the specified cpupool.
@@ -1160,57 +1165,57 @@ all the domains:
 
 xl sched-rtds -v all
 Cpupool Pool-0: sched=RTDS
-NameID VCPUPeriodBudget
-Domain-0 00 1  4000
-vm1  10   300   150
-vm1  11   400   200
-vm1  12 1  4000
-vm1  13  1000   500
-vm2  20 1  4000
-vm2  21 1  4000
+NameID VCPUPeriodBudget  Extratime
+Domain-0 00 1  4000yes
+vm1  20   300   150yes
+vm1  21   400   200yes
+vm1  22 1  4000yes
+vm1  23  1000   500yes
+vm2  40 1  4000yes
+vm2  41 1  4000yes
 
 Without any arguments, it will output the default scheduling
 parameters for each domain:
 
 xl sched-rtds
 Cpupool Pool-0: sched=RTDS
-NameIDPeriodBudget
-Domain-0 0 1  4000
-vm1  1 1  4000
-vm2  2 1  4000
+NameIDPeriodBudget  Extratime
+Domain-0 0 1  4000yes
+vm1  2 1  4000yes
+vm2  4 1  4000yes
 
 
-2) Use, for instancei, B<-d vm1, -v all> to see the budget and
+2) Use, for instance, B<-d vm1, -v all> to see the budget and
 period of all VCPUs of a specific domain (B):
 
 xl sched-rtds -d vm1 -v all
-NameID VCPUPeriodBudget
-vm1  10   300   150
-vm1  11   400   200
-vm1  12 1  4000
-vm1  13  1000   500
+NameID VCPUPeriodBudget  Extratime
+vm1  20   300   150yes
+vm1  21   400   200yes
+vm1  22 1  4000yes
+vm1  23  1000   500yes
 
 To see the parameters of a subset of the VCPUs of a domain, use:
 
 xl sched-rtds -d vm1 -v 0 -v 3
-NameID VCPUPeriodBudget
-vm1  10   300   150
-vm1  13  1000   500
+NameID VCPUPeriodBudget  Extratime
+vm1   

[Xen-devel] [PATCH v3 5/5] docs: enable per-VCPU extratime flag for RTDS

[Meng Xu]

Revise xl tool use case by adding -e option
Remove work-conserving from TODO list

Signed-off-by: Meng Xu 

---
No change from v2

Changes from v1
Revise rtds docs
---
 docs/features/sched_rtds.pandoc | 7 ---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/docs/features/sched_rtds.pandoc b/docs/features/sched_rtds.pandoc
index 354097b..d51b499 100644
--- a/docs/features/sched_rtds.pandoc
+++ b/docs/features/sched_rtds.pandoc
@@ -40,7 +40,7 @@ as follows:
 
 It is possible, for a multiple vCPUs VM, to change the parameters of
 each vCPU individually:
-* `xl sched-rtds -d vm-rt -v 0 -p 2 -b 1 -v 1 -p 45000 -b 12000`
+* `xl sched-rtds -d vm-rt -v 0 -p 2 -b 1 -e 1 -v 1 -p 45000 -b 
12000 -e 0`
 
 # Technical details
 
@@ -53,7 +53,8 @@ the presence of the LIBXL\_HAVE\_SCHED\_RTDS symbol. The 
ability of
 specifying different scheduling parameters for each vcpu has been
 introduced later, and is available if the following symbols are defined:
 * `LIBXL\_HAVE\_VCPU\_SCHED\_PARAMS`,
-* `LIBXL\_HAVE\_SCHED\_RTDS\_VCPU\_PARAMS`.
+* `LIBXL\_HAVE\_SCHED\_RTDS\_VCPU\_PARAMS`,
+* `LIBXL\_HAVE\_SCHED\_RTDS\_VCPU\_EXTRA`.
 
 # Limitations
 
@@ -95,7 +96,6 @@ at a macroscopic level), the following should be done:
 
 # Areas for improvement
 
-* Work-conserving mode to be added;
 * performance assessment, especially focusing on what level of real-time
   behavior the scheduler enables.
 
@@ -118,4 +118,5 @@ at a macroscopic level), the following should be done:
 Date   Revision Version  Notes
 --   ---
 2016-10-14 1Xen 4.8  Document written
+2017-08-31 2Xen 4.10 Revise for work conserving feature
 --   ---
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v3 2/5] libxl: enable per-VCPU extratime flag for RTDS

[Meng Xu]

Modify libxl_vcpu_sched_params_get/set and sched_rtds_vcpu_get/set
functions to support per-VCPU extratime flag

Signed-off-by: Meng Xu 

---
Changes from v2
1) Move extratime out of the section
   that is marked as depreciated in libxl_domain_sched_params.
2) Set vcpu extratime in sched_rtds_vcpu_get function function;
   This fix a bug in previous version
   when run command "xl sched-rtds -d 0 -v 1" which
   outputs vcpu extratime value incorrectly.

Changes from v1
1) Add LIBXL_HAVE_SCHED_RTDS_VCPU_EXTRA to indicate if extratime flag is
supported
2) Change flag name in domctl.h from XEN_DOMCTL_SCHED_RTDS_extratime to
XEN_DOMCTL_SCHEDRT_extra

Changes from RFC v1
Change work_conserving flag to extratime flag
---
 tools/libxl/libxl.h |  6 ++
 tools/libxl/libxl_sched.c   | 17 +
 tools/libxl/libxl_types.idl |  8 
 3 files changed, 27 insertions(+), 4 deletions(-)

diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index f82b91e..5e9aed7 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -257,6 +257,12 @@
 #define LIBXL_HAVE_SCHED_RTDS_VCPU_PARAMS 1
 
 /*
+ * LIBXL_HAVE_SCHED_RTDS_VCPU_EXTRA indicates RTDS scheduler
+ * now supports per-vcpu extratime settings.
+ */
+#define LIBXL_HAVE_SCHED_RTDS_VCPU_EXTRA 1
+
+/*
  * libxl_domain_build_info has the arm.gic_version field.
  */
 #define LIBXL_HAVE_BUILDINFO_ARM_GIC_VERSION 1
diff --git a/tools/libxl/libxl_sched.c b/tools/libxl/libxl_sched.c
index 7d144d0..512788f 100644
--- a/tools/libxl/libxl_sched.c
+++ b/tools/libxl/libxl_sched.c
@@ -532,6 +532,8 @@ static int sched_rtds_vcpu_get(libxl__gc *gc, uint32_t 
domid,
 for (i = 0; i < num_vcpus; i++) {
 scinfo->vcpus[i].period = vcpus[i].u.rtds.period;
 scinfo->vcpus[i].budget = vcpus[i].u.rtds.budget;
+scinfo->vcpus[i].extratime =
+!!(vcpus[i].u.rtds.flags & XEN_DOMCTL_SCHEDRT_extra);
 scinfo->vcpus[i].vcpuid = vcpus[i].vcpuid;
 }
 rc = 0;
@@ -579,6 +581,8 @@ static int sched_rtds_vcpu_get_all(libxl__gc *gc, uint32_t 
domid,
 for (i = 0; i < num_vcpus; i++) {
 scinfo->vcpus[i].period = vcpus[i].u.rtds.period;
 scinfo->vcpus[i].budget = vcpus[i].u.rtds.budget;
+scinfo->vcpus[i].extratime =
+!!(vcpus[i].u.rtds.flags & XEN_DOMCTL_SCHEDRT_extra);
 scinfo->vcpus[i].vcpuid = vcpus[i].vcpuid;
 }
 rc = 0;
@@ -628,6 +632,10 @@ static int sched_rtds_vcpu_set(libxl__gc *gc, uint32_t 
domid,
 vcpus[i].vcpuid = scinfo->vcpus[i].vcpuid;
 vcpus[i].u.rtds.period = scinfo->vcpus[i].period;
 vcpus[i].u.rtds.budget = scinfo->vcpus[i].budget;
+if (scinfo->vcpus[i].extratime)
+vcpus[i].u.rtds.flags |= XEN_DOMCTL_SCHEDRT_extra;
+else
+vcpus[i].u.rtds.flags &= ~XEN_DOMCTL_SCHEDRT_extra;
 }
 
 r = xc_sched_rtds_vcpu_set(CTX->xch, domid,
@@ -676,6 +684,10 @@ static int sched_rtds_vcpu_set_all(libxl__gc *gc, uint32_t 
domid,
 vcpus[i].vcpuid = i;
 vcpus[i].u.rtds.period = scinfo->vcpus[0].period;
 vcpus[i].u.rtds.budget = scinfo->vcpus[0].budget;
+if (scinfo->vcpus[0].extratime)
+vcpus[i].u.rtds.flags |= XEN_DOMCTL_SCHEDRT_extra;
+else
+vcpus[i].u.rtds.flags &= ~XEN_DOMCTL_SCHEDRT_extra;
 }
 
 r = xc_sched_rtds_vcpu_set(CTX->xch, domid,
@@ -726,6 +738,11 @@ static int sched_rtds_domain_set(libxl__gc *gc, uint32_t 
domid,
 sdom.period = scinfo->period;
 if (scinfo->budget != LIBXL_DOMAIN_SCHED_PARAM_BUDGET_DEFAULT)
 sdom.budget = scinfo->budget;
+/* Set extratime by default */
+if (scinfo->extratime)
+sdom.flags |= XEN_DOMCTL_SCHEDRT_extra;
+else
+sdom.flags &= ~XEN_DOMCTL_SCHEDRT_extra;
 if (sched_rtds_validate_params(gc, sdom.period, sdom.budget))
 return ERROR_INVAL;
 
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 2d0bb8a..dd7d364 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -421,14 +421,14 @@ libxl_domain_sched_params = Struct("domain_sched_params",[
 ("cap",  integer, {'init_val': 
'LIBXL_DOMAIN_SCHED_PARAM_CAP_DEFAULT'}),
 ("period",   integer, {'init_val': 
'LIBXL_DOMAIN_SCHED_PARAM_PERIOD_DEFAULT'}),
 ("budget",   integer, {'init_val': 
'LIBXL_DOMAIN_SCHED_PARAM_BUDGET_DEFAULT'}),
+("extratime",integer, {'init_val': 
'LIBXL_DOMAIN_SCHED_PARAM_EXTRATIME_DEFAULT'}),
 
-# The following three parameters ('slice', 'latency' and 'extratime') are 
deprecated,
+# The following three parameters ('slice' and 'latency') are deprecated,
 # and will have no effect if used, since the SEDF scheduler has been 
removed.
-# Note that 'period' was an SDF parameter too, but it is still effective 
as it is
-# now used (together with 'budget') by the RTDS scheduler.
+# Note that 'period' and 'extratime' was an SDF parameter too, but it is 

[Xen-devel] [PATCH v3 4/5] xentrace: enable per-VCPU extratime flag for RTDS

[Meng Xu]

Change repl_budget event output for xentrace formats and xenalyze

Signed-off-by: Meng Xu 

---
No changes from v2

Changes from v1
Add this changes from v1
---
 tools/xentrace/formats| 2 +-
 tools/xentrace/xenalyze.c | 8 +---
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/tools/xentrace/formats b/tools/xentrace/formats
index d6e7e3f..7d3a209 100644
--- a/tools/xentrace/formats
+++ b/tools/xentrace/formats
@@ -75,7 +75,7 @@
 0x00022801  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:tickle[ cpu = 
%(1)d ]
 0x00022802  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:runq_pick [ dom:vcpu 
= 0x%(1)08x, cur_deadline = 0x%(3)08x%(2)08x, cur_budget = 0x%(5)08x%(4)08x ]
 0x00022803  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:burn_budget   [ dom:vcpu 
= 0x%(1)08x, cur_budget = 0x%(3)08x%(2)08x, delta = %(4)d ]
-0x00022804  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:repl_budget   [ dom:vcpu 
= 0x%(1)08x, cur_deadline = 0x%(3)08x%(2)08x, cur_budget = 0x%(5)08x%(4)08x ]
+0x00022804  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:repl_budget   [ dom:vcpu 
= 0x%(1)08x, priority_level = 0x%(2)08d cur_deadline = 0x%(4)08x%(3)08x, 
cur_budget = 0x%(6)08x%(5)08x ]
 0x00022805  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:sched_tasklet
 0x00022806  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  rtds:schedule  [ 
cpu[16]:tasklet[8]:idle[4]:tickled[4] = %(1)08x ]
 
diff --git a/tools/xentrace/xenalyze.c b/tools/xentrace/xenalyze.c
index 79bdba7..2783204 100644
--- a/tools/xentrace/xenalyze.c
+++ b/tools/xentrace/xenalyze.c
@@ -7946,12 +7946,14 @@ void sched_process(struct pcpu_info *p)
 if(opt.dump_all) {
 struct {
 unsigned int vcpuid:16, domid:16;
+unsigned int priority_level;
 uint64_t cur_dl, cur_bg;
 } __attribute__((packed)) *r = (typeof(r))ri->d;
 
-printf(" %s rtds:repl_budget d%uv%u, deadline = %"PRIu64", "
-   "budget = %"PRIu64"\n", ri->dump_header,
-   r->domid, r->vcpuid, r->cur_dl, r->cur_bg);
+printf(" %s rtds:repl_budget d%uv%u, priority_level = %u,"
+   "deadline = %"PRIu64", budget = %"PRIu64"\n",
+   ri->dump_header, r->domid, r->vcpuid,
+   r->priority_level, r->cur_dl, r->cur_bg);
 }
 break;
 case TRC_SCHED_CLASS_EVT(RTDS, 5): /* SCHED_TASKLET*/
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH for-4.10] common/gnttab: Improve logging message by including relevent domid

[Andrew Cooper]

Several logging messages cite "bad ref %#x", without identifying which domain
the ref belongs to.  Add a domain back-pointer to struct grant_table to
improve the debugability.

While editing the messages, clean up some others:

 * Remove extranious punctuation
 * Use d%d rather than Dom%d
 * Remove "gnttab_transfer:" prefixes, as it is included by the gdprintk()
 * Reflow several messages to not be split across multiple lines

Signed-off-by: Andrew Cooper 
---
CC: George Dunlap 
CC: Jan Beulich 
CC: Konrad Rzeszutek Wilk 
CC: Stefano Stabellini 
CC: Tim Deegan 
CC: Wei Liu 
CC: Julien Grall 
---
 xen/common/grant_table.c | 91 +---
 1 file changed, 48 insertions(+), 43 deletions(-)

diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index 6d20b17..3a7a7e4 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -76,6 +76,9 @@ struct grant_table {
 /* Mapping tracking table per vcpu. */
 struct grant_mapping **maptrack;
 
+/* Domain to which this struct grant_table belongs. */
+struct domain *domain;
+
 struct grant_table_arch arch;
 };
 
@@ -651,7 +654,7 @@ static int _set_status_v1(domid_t  domid,
GTF_permit_access) ||
   (scombo.shorts.domid != domid)) )
 PIN_FAIL(done, GNTST_general_error,
- "Bad flags (%x) or dom (%d). (expected dom %d)\n",
+ "Bad flags (%x) or dom (%d); expected d%d\n",
  scombo.shorts.flags, scombo.shorts.domid,
  domid);
 
@@ -663,7 +666,7 @@ static int _set_status_v1(domid_t  domid,
 new_scombo.shorts.flags |= GTF_writing;
 if ( unlikely(scombo.shorts.flags & GTF_readonly) )
 PIN_FAIL(done, GNTST_general_error,
- "Attempt to write-pin a r/o grant entry.\n");
+ "Attempt to write-pin a r/o grant entry\n");
 }
 
 prev_scombo.word = cmpxchg((u32 *)shah,
@@ -673,7 +676,7 @@ static int _set_status_v1(domid_t  domid,
 
 if ( retries++ == 4 )
 PIN_FAIL(done, GNTST_general_error,
- "Shared grant entry is unstable.\n");
+ "Shared grant entry is unstable\n");
 
 scombo = prev_scombo;
 }
@@ -715,7 +718,7 @@ static int _set_status_v2(domid_t  domid,
 ((flags & mask) != GTF_transitive)) ||
   (id != domid)) )
 PIN_FAIL(done, GNTST_general_error,
- "Bad flags (%x) or dom (%d). (expected dom %d, flags %x)\n",
+ "Bad flags (%x) or dom (%d); expected d%d, flags %x\n",
  flags, id, domid, mask);
 
 if ( readonly )
@@ -726,7 +729,7 @@ static int _set_status_v2(domid_t  domid,
 {
 if ( unlikely(flags & GTF_readonly) )
 PIN_FAIL(done, GNTST_general_error,
- "Attempt to write-pin a r/o grant entry.\n");
+ "Attempt to write-pin a r/o grant entry\n");
 *status |= GTF_reading | GTF_writing;
 }
 
@@ -749,8 +752,7 @@ static int _set_status_v2(domid_t  domid,
 gnttab_clear_flag(_GTF_writing, status);
 gnttab_clear_flag(_GTF_reading, status);
 PIN_FAIL(done, GNTST_general_error,
- "Unstable flags (%x) or dom (%d). (expected dom %d) "
- "(r/w: %d)\n",
+ "Unstable flags (%x) or dom (%d); expected d%d (r/w: 
%d)\n",
  flags, id, domid, !readonly);
 }
 }
@@ -896,7 +898,7 @@ map_grant_ref(
 
 if ( unlikely((op->flags & (GNTMAP_device_map|GNTMAP_host_map)) == 0) )
 {
-gdprintk(XENLOG_INFO, "Bad flags in grant map op (%x).\n", op->flags);
+gdprintk(XENLOG_INFO, "Bad flags in grant map op: %x\n", op->flags);
 op->status = GNTST_bad_gntref;
 return;
 }
@@ -905,7 +907,7 @@ map_grant_ref(
   (op->flags & (GNTMAP_device_map|GNTMAP_application_map|
 GNTMAP_contains_pte))) )
 {
-gdprintk(XENLOG_INFO, "No device mapping in HVM domain.\n");
+gdprintk(XENLOG_INFO, "No device mapping in HVM domain\n");
 op->status = GNTST_general_error;
 return;
 }
@@ -930,7 +932,7 @@ map_grant_ref(
 if ( unlikely(handle == INVALID_MAPTRACK_HANDLE) )
 {
 rcu_unlock_domain(rd);
-gdprintk(XENLOG_INFO, "Failed to obtain maptrack handle.\n");
+gdprintk(XENLOG_INFO, "Failed to obtain maptrack handle\n");
 op->status = GNTST_no_device_space;
 return;
 }
@@ -940,7 +942,8 @@ map_grant_ref(
 
 /* Bounds check on the grant ref */
 if ( unlikely(op->ref >= nr_grant_entries(rgt)))
-PIN_FAIL(unlock_out, GNTST_bad_gntref, "Bad ref 

Re: [Xen-devel] [dpdk-dev] Can xenvirt pmd work in xen guest (aka DomU) without xen-vhost in Dom0 ?

[Stephen Hemminger]

On Mon, 9 Oct 2017 00:13:47 +0800
"Tan, Jianfeng"  wrote:

> Hi,
> 
> 
> On 10/8/2017 12:54 PM, Bill Bonaparte wrote:
> > Thanks Jianfeng for taking time to reply.
> >
> > please allow me to briefly explain why I want to run dpdk on xen.
> > our system is based on dpdk, which means we use dpdk as packet 
> > receive/transmit engine,
> > and with integrated dpdk virtio/vmxnet3 driver, our system can run on 
> > KVM/VMware platform .
> > this year, we have plan to run our system on AWS cloud, but I found 
> > that AWS
> > uses xen as its virtualization platform, and the bus-info of nic is 
> > vif-x (x could be 0,1,2...),
> > the driver used in kernel is vif. this should be para-virtualized nic 
> > used on xen.  
> 
> My guess is exactly as you describe. In AWS, we lack of a PMD for xen 
> netfront (vif) nic. And even we got such a PMD, we still need a PMD for 
> xen netback. Both are missing.
> 
> >
> > I don't know which dpdk drvier can manage this pv nic. then I see 
> > xenvirt, I think this driver can
> > did this job, like virtio can manage virtio nic which is used on kvm.
> > unfortunately, after some study work, I run testpmd successfully on 
> > xen, but no packets received.
> >
> > with the informain got from you, I know It's need to run vhost_xen at 
> > dom0 so that xenvirt at domU can work.
> > but for my case, I have no change to run vhost_xen at dom0, because I 
> > only can operate my own domU.
> >
> > for this case, If I want to run system which is based on dpdk at domU, 
> > what should I do?
> > appreciate any idea or suggestion from you.  
> 
> What kind of performance are you seeking? Only accelerating the frontend 
> by a new PMD, i.e. netfront, we can bypass the VM kernel (). But without 
> accelerating the backend, it only brings limited improvement.

Brocade wrote a xen net-front driver. It was never integrated upstream
and had many issues (such as pretending to be PCI and it wasn't).



___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v10 05/11] x86/mm: add HYPERVISOR_memory_op to acquire guest resources

[Paul Durrant]

Certain memory resources associated with a guest are not necessarily
present in the guest P2M.

This patch adds the boilerplate for new memory op to allow such a resource
to be priv-mapped directly, by either a PV or HVM tools domain.

NOTE: Whilst the new op is not intrinsicly specific to the x86 architecture,
  I have no means to test it on an ARM platform and so cannot verify
  that it functions correctly. Hence it is currently only implemented
  for x86.

Signed-off-by: Paul Durrant 
---
Cc: George Dunlap 
Cc: Jan Beulich 
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Wei Liu 

v9:
 - Addressed more comments from Jan.

v8:
 - Move the code into common as requested by Jan.
 - Make the gmfn_list handle a 64-bit type to avoid limiting the MFN
   range for a 32-bit tools domain.
 - Add missing pad.
 - Add compat code.
 - Make this patch deal with purely boilerplate.
 - Drop George's A-b and Wei's R-b because the changes are non-trivial,
   and update Cc list now the boilerplate is common.

v5:
 - Switched __copy_to/from_guest_offset() to copy_to/from_guest_offset().
---
 xen/arch/x86/mm/p2m.c   |  3 +-
 xen/common/compat/memory.c  | 65 ++
 xen/common/memory.c | 85 +
 xen/include/asm-x86/p2m.h   |  3 ++
 xen/include/public/memory.h | 32 -
 xen/include/xlat.lst|  1 +
 6 files changed, 186 insertions(+), 3 deletions(-)

diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index 3fbc537da6..ecc69d0093 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -1131,8 +1131,7 @@ static int set_typed_p2m_entry(struct domain *d, unsigned 
long gfn_l,
 }
 
 /* Set foreign mfn in the given guest's p2m table. */
-static int set_foreign_p2m_entry(struct domain *d, unsigned long gfn,
- mfn_t mfn)
+int set_foreign_p2m_entry(struct domain *d, unsigned long gfn, mfn_t mfn)
 {
 return set_typed_p2m_entry(d, gfn, mfn, PAGE_ORDER_4K, p2m_map_foreign,
p2m_get_hostp2m(d)->default_access);
diff --git a/xen/common/compat/memory.c b/xen/common/compat/memory.c
index 35bb259808..b59bd3f44b 100644
--- a/xen/common/compat/memory.c
+++ b/xen/common/compat/memory.c
@@ -71,6 +71,7 @@ int compat_memory_op(unsigned int cmd, 
XEN_GUEST_HANDLE_PARAM(void) compat)
 struct xen_remove_from_physmap *xrfp;
 struct xen_vnuma_topology_info *vnuma;
 struct xen_mem_access_op *mao;
+struct xen_mem_acquire_resource *mar;
 } nat;
 union {
 struct compat_memory_reservation rsrv;
@@ -79,6 +80,7 @@ int compat_memory_op(unsigned int cmd, 
XEN_GUEST_HANDLE_PARAM(void) compat)
 struct compat_add_to_physmap_batch atpb;
 struct compat_vnuma_topology_info vnuma;
 struct compat_mem_access_op mao;
+struct compat_mem_acquire_resource mar;
 } cmp;
 
 set_xen_guest_handle(nat.hnd, COMPAT_ARG_XLAT_VIRT_BASE);
@@ -395,6 +397,45 @@ int compat_memory_op(unsigned int cmd, 
XEN_GUEST_HANDLE_PARAM(void) compat)
 }
 #endif
 
+case XENMEM_acquire_resource:
+{
+xen_ulong_t *gmfn_list = (xen_ulong_t *)(nat.mar + 1);
+
+if ( copy_from_guest(, compat, 1) ||
+ !compat_handle_okay(cmp.mar.gmfn_list,
+ cmp.mar.nr_frames) )
+return -EFAULT;
+
+/*
+ * The number of frames handled is currently limited to a
+ * small number by the underlying implementation, so the
+ * scratch space should be sufficient for bouncing the
+ * frame addresses.
+ */
+if ( sizeof(*gmfn_list) * cmp.mar.nr_frames >
+ COMPAT_ARG_XLAT_SIZE - sizeof(*nat.mar) )
+return -E2BIG;
+
+for ( i = 0; i < cmp.mar.nr_frames; i++ )
+{
+compat_ulong_t gmfn;
+
+if ( __copy_from_compat_offset(, cmp.mar.gmfn_list,
+   i, 1) )
+return -EFAULT;
+
+gmfn_list[i] = gmfn;
+}
+
+#define XLAT_mem_acquire_resource_HNDL_gmfn_list(_d_, _s_) \
+set_xen_guest_handle((_d_)->gmfn_list, gmfn_list)
+
+XLAT_mem_acquire_resource(nat.mar, );
+
+#undef XLAT_mem_acquire_resource_HNDL_gmfn_list
+
+break;
+}
 default:
 return compat_arch_memory_op(cmd, compat);
 }
@@ -535,6 +576,30 @@ int compat_memory_op(unsigned int cmd, 
XEN_GUEST_HANDLE_PARAM(void) 

Re: [Xen-devel] [Qemu-devel] [PATCH 7/8] os-posix: Provide new -runasid option

[Ian Jackson]

Markus Armbruster writes ("Re: [Qemu-devel] [PATCH 7/8] os-posix: Provide new 
-runasid option"):
> Actually, a numeric UID without group name or ID could be made to work
> just fine as long as it maps to a user name.  The use case may not be
> worth the bother, though.

In libxl's use case, it wouldn't map to a name.

> Using '.' to separate user and group is suboptimal, because POSIX
> portable user and group names may contain it:
...
> Coreutils uses ':'.  Let's follow its lead.

I'm happy to change it to use `:'.

Can you confirm that this command line interface is then OK ?
We would like to stablise the corresponding code in libxl...

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v5 00/26 (PARTIAL POSTING)] qemu restrict final fixes

[Ian Jackson]

These two patches
 [PATCH 04/26] xentoolcore, _restrict_all: Introduce new library and
 [PATCH 24/26] libxl: dm_restrict: Support uid range user
need fixes.  See the commit messages.

I am not resending the unchanged patches.
I intend to push the whole series tomorrow.

Thanks,
Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/8] xen: link against xentoolcore

[Ian Jackson]

Anthony PERARD writes ("Re: [PATCH 1/8] xen: link against xentoolcore"):
> On Mon, Oct 09, 2017 at 05:28:08PM +0100, Ian Jackson wrote:
> > The xentoolcore library was just committed to xen.git#staging, so at
> > least this patch (or something like it) should go into qemu RSN.
> 
> I don't think it is necessary to do anything in qemu. The linker should
> find on its own the new libxentoolcore as long as an option
> -Wl,-rpath-link= provide the right path to xentoolcore when building
> qemu from xen.git.

But, the -rpath-link= specification in libxendevicemodel refers not to
the xen.git build tree, but to the eventual installed copy.

In my tests, this does in fact break the build.

>  In other cases, the pkg-config files should be
> enough (configure doesn't need to known about a new xentoolcore.pc
> file).

In that case, yes, the .pc works.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 04/26] xentoolcore, _restrict_all: Introduce new library and implementation

[Ian Jackson]

In practice, qemu opens a great many fds.  Tracking them all down and
playing whack-a-mole is unattractive.  It is also potentially fragile
in that future changes might accidentally undo our efforts.

Instead, we are going to teach all the Xen libraries how to register
their fds so that they can be neutered with one qemu call.

Right now, nothing will go wrong if some tries to link without
-ltoolcore, but that will stop working as soon as the first other Xen
library starts to register.  So this patch will be followed by the
stubdom build update, and should be followed by a
MINIOS_UPSTREAM_REVISION updated.

Sadly qemu upstream's configuration arrangements are too crude, being
keyed solely off the Xen version number.  So they cannot provide
forward/backward build compatibility across changes in xen-unstable,
like this one.  qemu patches to link against xentoolcore should be
applied in qemu upstream so avoid the qemu build breaking against the
released version of Xen 4.10.

Signed-off-by: Ian Jackson 
Acked-by: Wei Liu 
---
v5: Fix lock() call to actually call pthread_mutex_lock!
Spotted by Anthony Perard.

v3: Change %.o %.opic rules for extra dependency to $(LIB_OBJS) and
$(PIC_OBJS) instead.  (Report from Ross Lagerwall.)

v2: Remove obsolete "xxx" comment.
No longer claim to provide idempotency.
Add paragraphs to commit message about compatibility.

Signed-off-by: Ian Jackson 
---
 .gitignore |   4 +
 tools/Rules.mk |   6 ++
 tools/libs/Makefile|   1 +
 tools/libs/toolcore/Makefile   | 101 
 tools/libs/toolcore/handlereg.c|  77 
 tools/libs/toolcore/include/xentoolcore.h  |  73 +++
 tools/libs/toolcore/include/xentoolcore_internal.h | 102 +
 tools/libs/toolcore/libxentoolcore.map |   7 ++
 tools/libs/toolcore/xentoolcore.pc.in  |   9 ++
 9 files changed, 380 insertions(+)
 create mode 100644 tools/libs/toolcore/Makefile
 create mode 100644 tools/libs/toolcore/handlereg.c
 create mode 100644 tools/libs/toolcore/include/xentoolcore.h
 create mode 100644 tools/libs/toolcore/include/xentoolcore_internal.h
 create mode 100644 tools/libs/toolcore/libxentoolcore.map
 create mode 100644 tools/libs/toolcore/xentoolcore.pc.in

diff --git a/.gitignore b/.gitignore
index f36ddd2..95f40f1 100644
--- a/.gitignore
+++ b/.gitignore
@@ -73,6 +73,7 @@ stubdom/libxencall-*
 stubdom/libxenevtchn-*
 stubdom/libxenforeignmemory-*
 stubdom/libxengnttab-*
+stubdom/libxentoolcore-*
 stubdom/libxentoollog-*
 stubdom/lwip-*
 stubdom/lwip/
@@ -98,6 +99,8 @@ tools/config.cache
 config/Tools.mk
 config/Stubdom.mk
 config/Docs.mk
+tools/libs/toolcore/headers.chk
+tools/libs/toolcore/xentoolcore.pc
 tools/libs/toollog/headers.chk
 tools/libs/toollog/xentoollog.pc
 tools/libs/evtchn/headers.chk
@@ -352,6 +355,7 @@ tools/include/xen-foreign/arm64.h
 .git
 tools/misc/xen-hptool
 tools/misc/xen-mfndump
+tools/libs/toolcore/include/_*.h
 tools/libxc/_*.[ch]
 tools/libxl/_*.[ch]
 tools/libxl/testidl
diff --git a/tools/Rules.mk b/tools/Rules.mk
index dbc7635..5e1c7cb 100644
--- a/tools/Rules.mk
+++ b/tools/Rules.mk
@@ -10,6 +10,7 @@ export _INSTALL := $(INSTALL)
 INSTALL = $(XEN_ROOT)/tools/cross-install
 
 XEN_INCLUDE= $(XEN_ROOT)/tools/include
+XEN_LIBXENTOOLCORE  = $(XEN_ROOT)/tools/libs/toolcore
 XEN_LIBXENTOOLLOG  = $(XEN_ROOT)/tools/libs/toollog
 XEN_LIBXENEVTCHN   = $(XEN_ROOT)/tools/libs/evtchn
 XEN_LIBXENGNTTAB   = $(XEN_ROOT)/tools/libs/gnttab
@@ -102,6 +103,11 @@ SHDEPS_libxentoollog =
 LDLIBS_libxentoollog = $(SHDEPS_libxentoollog) 
$(XEN_LIBXENTOOLLOG)/libxentoollog$(libextension)
 SHLIB_libxentoollog  = $(SHDEPS_libxentoollog) 
-Wl,-rpath-link=$(XEN_LIBXENTOOLLOG)
 
+CFLAGS_libxentoolcore = -I$(XEN_LIBXENTOOLCORE)/include $(CFLAGS_xeninclude)
+SHDEPS_libxentoolcore =
+LDLIBS_libxentoolcore = $(SHDEPS_libxentoolcore) 
$(XEN_LIBXENTOOLCORE)/libxentoolcore$(libextension)
+SHLIB_libxentoolcore  = $(SHDEPS_libxentoolcore) 
-Wl,-rpath-link=$(XEN_LIBXENTOOLCORE)
+
 CFLAGS_libxenevtchn = -I$(XEN_LIBXENEVTCHN)/include $(CFLAGS_xeninclude)
 SHDEPS_libxenevtchn =
 LDLIBS_libxenevtchn = $(SHDEPS_libxenevtchn) 
$(XEN_LIBXENEVTCHN)/libxenevtchn$(libextension)
diff --git a/tools/libs/Makefile b/tools/libs/Makefile
index 2035873..ea9a64d 100644
--- a/tools/libs/Makefile
+++ b/tools/libs/Makefile
@@ -2,6 +2,7 @@ XEN_ROOT = $(CURDIR)/../..
 include $(XEN_ROOT)/tools/Rules.mk
 
 SUBDIRS-y :=
+SUBDIRS-y += toolcore
 SUBDIRS-y += toollog
 SUBDIRS-y += evtchn
 SUBDIRS-y += gnttab
diff --git a/tools/libs/toolcore/Makefile b/tools/libs/toolcore/Makefile
new file mode 100644
index 000..73db0bd
--- /dev/null
+++ b/tools/libs/toolcore/Makefile
@@ -0,0 +1,101 @@
+XEN_ROOT = $(CURDIR)/../../..
+include 

[Xen-devel] [xen-unstable-smoke test] 114299: tolerable all pass - PUSHED

[osstest service owner]

flight 114299 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/114299/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  3b40cfcd1a1912c2e4c4eb353dc77cbf35c63c3a
baseline version:
 xen  a164e14c6e140d792aee644990f8fea0fa8f8da2

Last test of basis   114289  2017-10-10 18:02:42 Z0 days
Testing same since   114299  2017-10-10 21:02:54 Z0 days1 attempts


People who touched revisions under test:
  Julien Grall 
  Julien Grall 
  Manish Jaggi 
  Stefano Stabellini 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=xen-unstable-smoke
+ revision=3b40cfcd1a1912c2e4c4eb353dc77cbf35c63c3a
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
 export PERLLIB=.:.
 PERLLIB=.:.
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push xen-unstable-smoke 
3b40cfcd1a1912c2e4c4eb353dc77cbf35c63c3a
+ branch=xen-unstable-smoke
+ revision=3b40cfcd1a1912c2e4c4eb353dc77cbf35c63c3a
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
 export PERLLIB=.:.:.
 PERLLIB=.:.:.
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
+++ export PERLLIB=.:.:.:.
+++ PERLLIB=.:.:.:.
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=xen
+ xenbranch=xen-unstable-smoke
+ qemuubranch=qemu-upstream-unstable
+ '[' xxen = xlinux ']'
+ linuxbranch=
+ '[' xqemu-upstream-unstable = x ']'
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable-smoke
+ prevxenbranch=xen-4.9-testing
+ '[' x3b40cfcd1a1912c2e4c4eb353dc77cbf35c63c3a = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/xtf.git
++ : osst...@xenbits.xen.org:/home/xen/git/xtf.git
++ : git://xenbits.xen.org/xtf.git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/rumprun.git
++ : git://git.seabios.org/seabios.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/seabios.git
++ : git://xenbits.xen.org/osstest/seabios.git
++ : https://github.com/tianocore/edk2.git
++ : 

[Xen-devel] [PATCH v10 07/11] x86/mm: add an extra command to HYPERVISOR_mmu_update...

[Paul Durrant]

...to allow the calling domain to prevent translation of specified l1e
value.

Despite what the comment in public/xen.h might imply, specifying a
command value of MMU_NORMAL_PT_UPDATE will not simply update an l1e with
the specified value. Instead, mod_l1_entry() tests whether foreign_dom
has PG_translate set in its paging mode and, if it does, assumes that the
the pfn value in the l1e is a gfn rather than an mfn.

To allow PV tools domain to map mfn values from a previously issued
HYPERVISOR_memory_op:XENMEM_acquire_resource, there needs to be a way
to tell HYPERVISOR_mmu_update that the specific l1e value does not
require translation regardless of the paging mode of foreign_dom. This
patch therefore defines a new command value, MMU_PT_UPDATE_NO_TRANSLATE,
which has the same semantics as MMU_NORMAL_PT_UPDATE except that the
paging mode of foreign_dom is ignored and the l1e value is used verbatim.

Signed-off-by: Paul Durrant 
Reviewed-by: Jan Beulich 
---
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Wei Liu 

v8:
 - New in this version, replacing "allow a privileged PV domain to map
   guest mfns".
---
 xen/arch/x86/mm.c| 17 ++---
 xen/include/public/xen.h | 12 +---
 2 files changed, 19 insertions(+), 10 deletions(-)

diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index c9bc4a4e92..3dd5b2c00f 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -1619,9 +1619,10 @@ void page_unlock(struct page_info *page)
 
 /* Update the L1 entry at pl1e to new value nl1e. */
 static int mod_l1_entry(l1_pgentry_t *pl1e, l1_pgentry_t nl1e,
-unsigned long gl1mfn, int preserve_ad,
+unsigned long gl1mfn, unsigned int cmd,
 struct vcpu *pt_vcpu, struct domain *pg_dom)
 {
+bool preserve_ad = (cmd == MMU_PT_UPDATE_PRESERVE_AD);
 l1_pgentry_t ol1e;
 struct domain *pt_dom = pt_vcpu->domain;
 int rc = 0;
@@ -1643,7 +1644,8 @@ static int mod_l1_entry(l1_pgentry_t *pl1e, l1_pgentry_t 
nl1e,
 return -EINVAL;
 }
 
-if ( paging_mode_translate(pg_dom) )
+if ( cmd != MMU_PT_UPDATE_NO_TRANSLATE &&
+ paging_mode_translate(pg_dom) )
 {
 page = get_page_from_gfn(pg_dom, l1e_get_pfn(nl1e), NULL, 
P2M_ALLOC);
 if ( !page )
@@ -3258,6 +3260,7 @@ long do_mmu_update(
  */
 case MMU_NORMAL_PT_UPDATE:
 case MMU_PT_UPDATE_PRESERVE_AD:
+case MMU_PT_UPDATE_NO_TRANSLATE:
 {
 p2m_type_t p2mt;
 
@@ -3323,7 +3326,8 @@ long do_mmu_update(
 p2m_query_t q = (l1e_get_flags(l1e) & _PAGE_RW) ?
 P2M_UNSHARE : P2M_ALLOC;
 
-if ( paging_mode_translate(pg_owner) )
+if ( cmd != MMU_PT_UPDATE_NO_TRANSLATE &&
+ paging_mode_translate(pg_owner) )
 target = get_page_from_gfn(pg_owner, l1e_get_pfn(l1e),
_p2mt, q);
 
@@ -3350,9 +3354,7 @@ long do_mmu_update(
 break;
 }
 
-rc = mod_l1_entry(va, l1e, mfn,
-  cmd == MMU_PT_UPDATE_PRESERVE_AD, v,
-  pg_owner);
+rc = mod_l1_entry(va, l1e, mfn, cmd, v, pg_owner);
 if ( target )
 put_page(target);
 }
@@ -3630,7 +3632,8 @@ static int __do_update_va_mapping(
 goto out;
 }
 
-rc = mod_l1_entry(pl1e, val, mfn_x(gl1mfn), 0, v, pg_owner);
+rc = mod_l1_entry(pl1e, val, mfn_x(gl1mfn), MMU_NORMAL_PT_UPDATE, v,
+  pg_owner);
 
 page_unlock(gl1pg);
 put_page(gl1pg);
diff --git a/xen/include/public/xen.h b/xen/include/public/xen.h
index 2ac6b1e24d..d2014a39eb 100644
--- a/xen/include/public/xen.h
+++ b/xen/include/public/xen.h
@@ -268,6 +268,10 @@ DEFINE_XEN_GUEST_HANDLE(xen_ulong_t);
  * As MMU_NORMAL_PT_UPDATE above, but A/D bits currently in the PTE are ORed
  * with those in @val.
  *
+ * ptr[1:0] == MMU_PT_UPDATE_NO_TRANSLATE:
+ * As MMU_NORMAL_PT_UPDATE above, but @val is not translated though FD
+ * page tables.
+ *
  * @val is usually the machine frame number along with some attributes.
  * The attributes by default follow the architecture defined bits. Meaning that
  * if this is a X86_64 machine and four page table layout is used, the layout
@@ -334,9 +338,11 @@ DEFINE_XEN_GUEST_HANDLE(xen_ulong_t);
  *
  * PAT (bit 7 on) --> PWT (bit 3 on) and clear bit 7.
  */
-#define MMU_NORMAL_PT_UPDATE  0 /* checked '*ptr = val'. ptr is MA.   

[Xen-devel] [PATCH v10 03/11] x86/hvm/ioreq: use gfn_t in struct hvm_ioreq_page

[Paul Durrant]

This patch adjusts the ioreq server code to use type-safe gfn_t values
where possible. No functional change.

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
Reviewed-by: Wei Liu 
Acked-by: Jan Beulich 
---
Cc: Andrew Cooper 
---
 xen/arch/x86/hvm/ioreq.c | 44 
 xen/include/asm-x86/hvm/domain.h |  2 +-
 2 files changed, 23 insertions(+), 23 deletions(-)

diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 6e750538a3..4f20ef7108 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -210,7 +210,7 @@ bool handle_hvm_io_completion(struct vcpu *v)
 return true;
 }
 
-static unsigned long hvm_alloc_ioreq_gfn(struct hvm_ioreq_server *s)
+static gfn_t hvm_alloc_ioreq_gfn(struct hvm_ioreq_server *s)
 {
 struct domain *d = s->domain;
 unsigned int i;
@@ -220,20 +220,19 @@ static unsigned long hvm_alloc_ioreq_gfn(struct 
hvm_ioreq_server *s)
 for ( i = 0; i < sizeof(d->arch.hvm_domain.ioreq_gfn.mask) * 8; i++ )
 {
 if ( test_and_clear_bit(i, >arch.hvm_domain.ioreq_gfn.mask) )
-return d->arch.hvm_domain.ioreq_gfn.base + i;
+return _gfn(d->arch.hvm_domain.ioreq_gfn.base + i);
 }
 
-return gfn_x(INVALID_GFN);
+return INVALID_GFN;
 }
 
-static void hvm_free_ioreq_gfn(struct hvm_ioreq_server *s,
-   unsigned long gfn)
+static void hvm_free_ioreq_gfn(struct hvm_ioreq_server *s, gfn_t gfn)
 {
 struct domain *d = s->domain;
-unsigned int i = gfn - d->arch.hvm_domain.ioreq_gfn.base;
+unsigned int i = gfn_x(gfn) - d->arch.hvm_domain.ioreq_gfn.base;
 
 ASSERT(!IS_DEFAULT(s));
-ASSERT(gfn != gfn_x(INVALID_GFN));
+ASSERT(!gfn_eq(gfn, INVALID_GFN));
 
 set_bit(i, >arch.hvm_domain.ioreq_gfn.mask);
 }
@@ -242,7 +241,7 @@ static void hvm_unmap_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 {
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
 
-if ( iorp->gfn == gfn_x(INVALID_GFN) )
+if ( gfn_eq(iorp->gfn, INVALID_GFN) )
 return;
 
 destroy_ring_for_helper(>va, iorp->page);
@@ -251,7 +250,7 @@ static void hvm_unmap_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 if ( !IS_DEFAULT(s) )
 hvm_free_ioreq_gfn(s, iorp->gfn);
 
-iorp->gfn = gfn_x(INVALID_GFN);
+iorp->gfn = INVALID_GFN;
 }
 
 static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
@@ -264,16 +263,17 @@ static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 return -EINVAL;
 
 if ( IS_DEFAULT(s) )
-iorp->gfn = buf ?
-d->arch.hvm_domain.params[HVM_PARAM_BUFIOREQ_PFN] :
-d->arch.hvm_domain.params[HVM_PARAM_IOREQ_PFN];
+iorp->gfn = _gfn(buf ?
+ d->arch.hvm_domain.params[HVM_PARAM_BUFIOREQ_PFN] :
+ d->arch.hvm_domain.params[HVM_PARAM_IOREQ_PFN]);
 else
 iorp->gfn = hvm_alloc_ioreq_gfn(s);
 
-if ( iorp->gfn == gfn_x(INVALID_GFN) )
+if ( gfn_eq(iorp->gfn, INVALID_GFN) )
 return -ENOMEM;
 
-rc = prepare_ring_for_helper(d, iorp->gfn, >page, >va);
+rc = prepare_ring_for_helper(d, gfn_x(iorp->gfn), >page,
+ >va);
 
 if ( rc )
 hvm_unmap_ioreq_gfn(s, buf);
@@ -309,10 +309,10 @@ static void hvm_remove_ioreq_gfn(struct hvm_ioreq_server 
*s, bool buf)
 struct domain *d = s->domain;
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
 
-if ( IS_DEFAULT(s) || iorp->gfn == gfn_x(INVALID_GFN) )
+if ( IS_DEFAULT(s) || gfn_eq(iorp->gfn, INVALID_GFN) )
 return;
 
-if ( guest_physmap_remove_page(d, _gfn(iorp->gfn),
+if ( guest_physmap_remove_page(d, iorp->gfn,
_mfn(page_to_mfn(iorp->page)), 0) )
 domain_crash(d);
 clear_page(iorp->va);
@@ -324,12 +324,12 @@ static int hvm_add_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
 int rc;
 
-if ( IS_DEFAULT(s) || iorp->gfn == gfn_x(INVALID_GFN) )
+if ( IS_DEFAULT(s) || gfn_eq(iorp->gfn, INVALID_GFN) )
 return 0;
 
 clear_page(iorp->va);
 
-rc = guest_physmap_add_page(d, _gfn(iorp->gfn),
+rc = guest_physmap_add_page(d, iorp->gfn,
 _mfn(page_to_mfn(iorp->page)), 0);
 if ( rc == 0 )
 paging_mark_dirty(d, _mfn(page_to_mfn(iorp->page)));
@@ -590,8 +590,8 @@ static int hvm_ioreq_server_init(struct hvm_ioreq_server *s,
 INIT_LIST_HEAD(>ioreq_vcpu_list);
 spin_lock_init(>bufioreq_lock);
 
-s->ioreq.gfn = gfn_x(INVALID_GFN);
-s->bufioreq.gfn = gfn_x(INVALID_GFN);
+s->ioreq.gfn = INVALID_GFN;
+s->bufioreq.gfn = INVALID_GFN;
 
 rc = hvm_ioreq_server_alloc_rangesets(s, id);
 if ( rc )
@@ -757,11 +757,11 @@ int 

Re: [Xen-devel] [PATCH v3 09/12] fuzz/x86_emulate: Make input more compact

[George Dunlap]

On 10/10/2017 06:11 PM, Andrew Cooper wrote:
> On 10/10/17 18:01, George Dunlap wrote:
>> On 10/10/2017 05:59 PM, Andrew Cooper wrote:
>>> On 10/10/17 17:20, George Dunlap wrote:
 At the moment, AFL reckons that for any given input, 87% of it is
 completely irrelevant: that is, it can change it as much as it wants
 but have no impact on the result of the test; and yet it can't remove
 it.

 This is largely because we interpret the blob handed to us as a large
 struct, including CR values, MSR values, segment registers, and a full
 cpu_user_regs.

 Instead, modify our interpretation to have a "set state" stanza at the
 front.  Begin by reading a 16-bit value; if it is lower than a certain
 threshold, set some state according to what byte it is, and repeat.
 Continue until the byte is above a certain threshold.

 This allows AFL to compact any given test case much smaller; to the
 point where now it reckons there is not a single byte of the test file
 which becomes irrelevant.  Testing have shown that this option both
 allows AFL to reach coverage much faster, and to have a total coverage
 higher than with the old format.

 Make this an option (rather than a unilateral change) to enable
 side-by-side performance comparison of the old and new formats.

 Signed-off-by: George Dunlap 
>>> I am still of the opinion that this is a waste of effort, which would be
>>> better spent actually removing the irrelevant state in the first place;
>>> not building an obfuscation algorithm.
>>>
>>> I'm not going to nack the patch because that is probably over the top,
>>> but I'm not in favour if this change going in.
>> Did you look at the evidence I presented, demonstrating that this
>> significantly increases the effectiveness of AFL?
> 
> I can easily believe that you've found an obfucation algorithm which
> does better than the current state layout.
> 
> I do not believe that any amount of obfuscation will be better than
> actually fixing the root cause of the problem; that the current state
> really is mostly irrelevant, and can easily be shrunk.

Right; well I've already explained why I don't think "obfuscation" is
the right term.  For the time being, we have something which improves
efficiency; let's check it in now, and in the future if you or someone
else finds a way to fix it "properly" we can do that.

 -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 3/8] xen: defer call to xen_restrict until after os_setup_post

[Ian Jackson]

Anthony PERARD writes ("Re: [PATCH 3/8] xen: defer call to xen_restrict until 
after os_setup_post"):
> I'm tring to find out what does calling xen_restrict_all(0), when
> running an non-Xen guest. I think it would just lock(), then unlock()
> then there should not be any handle to restrict, and return 0; is that
> right?

Yes.  If the process has not opened any Xen control handles,
xentoolcore_restrict_all is a no-op.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 24/26] libxl: dm_restrict: Support uid range user

[Ian Jackson]

Signed-off-by: Ian Jackson 
Acked-by: Wei Liu 
---
v5: Use -runas :, as suggested on qemu-devel
by Markus Armbruster

v3: Use -runas ., as suggested on qemu-devel
by Markus Armbruster

squash! libxl: dm_restrict: Support uid range user
---
 docs/man/xl.cfg.pod.5.in | 11 ++-
 tools/libxl/libxl_dm.c   | 32 
 tools/libxl/libxl_internal.h |  1 +
 3 files changed, 43 insertions(+), 1 deletion(-)

diff --git a/docs/man/xl.cfg.pod.5.in b/docs/man/xl.cfg.pod.5.in
index ee84511..cb32d04 100644
--- a/docs/man/xl.cfg.pod.5.in
+++ b/docs/man/xl.cfg.pod.5.in
@@ -2243,7 +2243,16 @@ For example, cdrom insert will fail.
 =item
 
 You must create user(s) for qemu to run as.
-Currently, you should either create
+
+Ideally, set aside a range of 32752 uids
+(from N to N+32751)
+and create a user
+whose name is B
+and whose uid is N
+and whose gid is a plain unprivileged gid.
+libxl will use one such user for each domid.
+
+Alternatively, either create
 B
 for every $domid from 1 to 32751 inclusive,
 or
diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
index b1e6796..0a5b0f8 100644
--- a/tools/libxl/libxl_dm.c
+++ b/tools/libxl/libxl_dm.c
@@ -23,6 +23,7 @@
 #include 
 #include 
 #include 
+#include 
 
 static const char *libxl_tapif_script(libxl__gc *gc)
 {
@@ -753,6 +754,9 @@ libxl__detect_gfx_passthru_kind(libxl__gc *gc,
  *  userlookup_helper_getpwnam(libxl__gc*, const char *user,
  * struct passwd **pwd_r);
  *
+ *  userlookup_helper_getpwuid(libxl__gc*, uid_t uid,
+ * struct passwd **pwd_r);
+ *
  *  returns 1 if the user was found, 0 if it was not, -1 on error
  */
 #define DEFINE_USERLOOKUP_HELPER(NAME,SPEC_TYPE,STRUCTNAME,SYSCONF) \
@@ -791,6 +795,7 @@ libxl__detect_gfx_passthru_kind(libxl__gc *gc,
 }
 
 DEFINE_USERLOOKUP_HELPER(getpwnam, const char*, passwd, _SC_GETPW_R_SIZE_MAX);
+DEFINE_USERLOOKUP_HELPER(getpwuid, uid_t,   passwd, _SC_GETPW_R_SIZE_MAX);
 
 /* colo mode */
 enum {
@@ -951,6 +956,7 @@ static int libxl__build_device_model_args_new(libxl__gc *gc,
 uint64_t ram_size;
 const char *path, *chardev;
 char *user = NULL;
+struct passwd *user_base;
 
 dm_args = flexarray_make(gc, 16, 1);
 dm_envs = flexarray_make(gc, 16, 1);
@@ -1660,6 +1666,32 @@ static int libxl__build_device_model_args_new(libxl__gc 
*gc,
 if (ret > 0)
 goto end_search;
 
+ret = userlookup_helper_getpwnam(gc, LIBXL_QEMU_USER_RANGE_BASE,
+ _base);
+if (ret < 0)
+return ret;
+if (ret > 0) {
+struct passwd *user_clash;
+uid_t intended_uid = user_base->pw_uid + guest_domid;
+ret = userlookup_helper_getpwuid(gc, intended_uid, _clash);
+if (ret < 0)
+return ret;
+if (ret > 0) {
+LOGD(ERROR, guest_domid,
+ "wanted to use uid %ld (%s + %d) but that is user %s !",
+ (long)intended_uid, LIBXL_QEMU_USER_RANGE_BASE,
+ guest_domid, user_clash->pw_name);
+return ERROR_FAIL;
+}
+LOGD(DEBUG, guest_domid, "using uid %ld", (long)intended_uid);
+flexarray_append(dm_args, "-runas");
+flexarray_append(dm_args,
+ GCSPRINTF("%ld:%ld", (long)intended_uid,
+   (long)user_base->pw_gid));
+user = NULL; /* we have taken care of it */
+goto end_search;
+}
+
 user = LIBXL_QEMU_USER_SHARED;
 ret = userlookup_helper_getpwnam(gc, user, 0);
 if (ret < 0)
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 530183f..6d51d47 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -4314,6 +4314,7 @@ _hidden int libxl__read_sysfs_file_contents(libxl__gc *gc,
 #define LIBXL_QEMU_USER_PREFIX "xen-qemuuser"
 #define LIBXL_QEMU_USER_BASE   LIBXL_QEMU_USER_PREFIX"-domid"
 #define LIBXL_QEMU_USER_SHARED LIBXL_QEMU_USER_PREFIX"-shared"
+#define LIBXL_QEMU_USER_RANGE_BASE LIBXL_QEMU_USER_PREFIX"-range-base"
 
 static inline bool libxl__acpi_defbool_val(const libxl_domain_build_info 
*b_info)
 {
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [linux-3.18 test] 114225: regressions - FAIL

[osstest service owner]

flight 114225 linux-3.18 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/114225/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-pvh-intel 12 guest-start fail REGR. vs. 114034
 test-amd64-amd64-xl-pvh-amd  12 guest-start  fail REGR. vs. 114034

Tests which are failing intermittently (not blocking):
 test-amd64-i386-xl-raw 19 guest-start/debian.repeat fail in 114133 pass in 
114225
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail pass in 114133

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 114034
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 114034
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 114034
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 114034
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 114034
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 114034
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 114034
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qcow2 12 migrate-support-checkfail  never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 13 guest-saverestore   fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 13 guest-saverestore   fail never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass

version targeted for testing:
 linuxac0058305d83e8e50a9652a003bc2ec468df9f87
baseline version:
 linuxffc97d4dde1d3c77beebddbbd2a0be5f8f18236a

Last test of basis   114034  2017-10-05 07:56:53 Z5 days
Testing same since   114133  2017-10-08 09:26:23 Z2 days3 attempts


People who touched revisions under test:
  Afzal Mohammed 
  Alden Tondettar 
  Alexander Potapenko 
  Andrey Ryabinin 
  Archit Taneja 
  Ard Biesheuvel 
  Arnd Bergmann 
  Arvind Yadav 
  Bartosz Golaszewski 
  Christophe JAILLET 
  Connor O'Brien 
  Darrick 

Re: [Xen-devel] [PATCH v3 09/12] fuzz/x86_emulate: Make input more compact

[Ian Jackson]

George Dunlap writes ("[PATCH v3 09/12] fuzz/x86_emulate: Make input more 
compact"):
> At the moment, AFL reckons that for any given input, 87% of it is
> completely irrelevant: that is, it can change it as much as it wants
> but have no impact on the result of the test; and yet it can't remove
> it.
> 
> This is largely because we interpret the blob handed to us as a large
> struct, including CR values, MSR values, segment registers, and a full
> cpu_user_regs.
> 
> Instead, modify our interpretation to have a "set state" stanza at the
> front.  Begin by reading a 16-bit value; if it is lower than a certain
> threshold, set some state according to what byte it is, and repeat.
> Continue until the byte is above a certain threshold.
> 
> This allows AFL to compact any given test case much smaller; to the
> point where now it reckons there is not a single byte of the test file
> which becomes irrelevant.  Testing have shown that this option both
> allows AFL to reach coverage much faster, and to have a total coverage
> higher than with the old format.

This is basically a compression scheme.  How odd that it should help.

Acked-by: Ian Jackson 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 03/12] fuzz/x86_emulate: Implement input_read() and input_avail()

[Ian Jackson]

George Dunlap writes ("[PATCH v3 03/12] fuzz/x86_emulate: Implement 
input_read() and input_avail()"):
> Rather than open-coding the "read" from the input file.
> 
> Signed-off-by: George Dunlap 

Reviewed-by: Ian Jackson 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 01/12] fuzz/x86_emulate: Clear errors after each iteration

[Ian Jackson]

George Dunlap writes ("[PATCH v3 01/12] fuzz/x86_emulate: Clear errors after 
each iteration"):
> Once feof() returns true for a stream, it will continue to return true
> for that stream until clearerr() is called (or the stream is closed
> and re-opened).
> 
> In llvm-clang-fast-mode, the same file descriptor is used for each
> iteration of the loop, meaning that the "Input too large" check was
> broken -- feof() would return true even if the fread() hadn't hit the
> end of the file.  The result is that AFL generates testcases of
> arbitrary size.
> 
> Fix this by fseek'ing to the beginning of the file on every iteration;
> this resets the EOF marker and other state.

Acked-by: Ian Jackson 

> This is a candidate for backport to 4.9.

Please let me know when it is committed and I will add it to my
backport list.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 04/26] xentoolcore, _restrict_all: Introduce new library and implementation

[Ian Jackson]

Anthony PERARD writes ("Re: [Xen-devel] [PATCH 04/26] xentoolcore, 
_restrict_all: Introduce new library and implementation"):
> On Mon, Oct 09, 2017 at 04:57:06PM +0100, Ian Jackson wrote:
> > +static pthread_mutex_t handles_lock = PTHREAD_MUTEX_INITIALIZER;
> > +static XENTOOLCORE_LIST_HEAD(, Xentoolcore__Active_Handle) handles;
> > +
> > +static void lock(void) {
> > +int e = pthread_mutex_unlock(_lock);
> 
> Shouldn't this call pthread_mutex_lock? Right now lock and unlock do
> the same thing.

Wow.  Sorry about that.  We should definitely fix that.

It's committed already but I will send a followup patch.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 0/*] xen: xen-domid-restrict improvements

[Ian Jackson]

Ross Lagerwall writes ("Re: [Xen-devel] [PATCH v2 0/*] xen: xen-domid-restrict 
improvements"):
> If no one objects, I propose adding the following calls to 
> libxendevicemodel (with underlying Xen implementations, of course) that 
> would be usable after the xendevicemodel handle has been restricted.
> 
> xendevicemodel_add_to_physmap(xendevicemodel_handle *dmod,
...
> xendevicemodel_pin_memory_cacheattr(xendevicemodel_handle *dmod,
...
> This is equivalent to xc_domain_pin_memory_cacheattr().

These seem fine to me.  If you want this in Xen 4.10 I think you will
want to make a case to Julien (not CC'd) for a release ack.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 09/12] fuzz/x86_emulate: Make input more compact

[George Dunlap]

On 10/10/2017 05:59 PM, Andrew Cooper wrote:
> On 10/10/17 17:20, George Dunlap wrote:
>> At the moment, AFL reckons that for any given input, 87% of it is
>> completely irrelevant: that is, it can change it as much as it wants
>> but have no impact on the result of the test; and yet it can't remove
>> it.
>>
>> This is largely because we interpret the blob handed to us as a large
>> struct, including CR values, MSR values, segment registers, and a full
>> cpu_user_regs.
>>
>> Instead, modify our interpretation to have a "set state" stanza at the
>> front.  Begin by reading a 16-bit value; if it is lower than a certain
>> threshold, set some state according to what byte it is, and repeat.
>> Continue until the byte is above a certain threshold.
>>
>> This allows AFL to compact any given test case much smaller; to the
>> point where now it reckons there is not a single byte of the test file
>> which becomes irrelevant.  Testing have shown that this option both
>> allows AFL to reach coverage much faster, and to have a total coverage
>> higher than with the old format.
>>
>> Make this an option (rather than a unilateral change) to enable
>> side-by-side performance comparison of the old and new formats.
>>
>> Signed-off-by: George Dunlap 
> 
> I am still of the opinion that this is a waste of effort, which would be
> better spent actually removing the irrelevant state in the first place;
> not building an obfuscation algorithm.
> 
> I'm not going to nack the patch because that is probably over the top,
> but I'm not in favour if this change going in.

Did you look at the evidence I presented, demonstrating that this
significantly increases the effectiveness of AFL?

 -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 08/12] fuzz/x86_emulate: Move definitions into a header

[Ian Jackson]

George Dunlap writes ("[PATCH v3 08/12] fuzz/x86_emulate: Move definitions into 
a header"):
> Move fuzz-emul.c function prototypes into a header.  Also share the
> definition of the input size (rather than hard-coding it in
> fuzz-emul.c).
> 
> Signed-off-by: George Dunlap 

Reviewed-by: Ian Jackson 

> RFC: Worth trying to BUILD_BUG_ON(INPUT_SIZE < DATA_SIZE_FULL)?

I don't mind.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v10 08/11] tools/libxenforeignmemory: add support for resource mapping

[Paul Durrant]

A previous patch introduced a new HYPERVISOR_memory_op to acquire guest
resources for direct priv-mapping.

This patch adds new functionality into libxenforeignmemory to make use
of a new privcmd ioctl [1] that uses the new memory op to make such
resources available via mmap(2).

[1] 
http://xenbits.xen.org/gitweb/?p=people/pauldu/linux.git;a=commit;h=ce59a05e6712

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
Reviewed-by: Wei Liu 
---
Cc: Ian Jackson 

v4:
 - Fixed errno and removed single-use label
 - The unmap call now returns a status
 - Use C99 initialization for ioctl struct

v2:
 - Bump minor version up to 3.
---
 tools/include/xen-sys/Linux/privcmd.h  | 11 +
 tools/libs/foreignmemory/Makefile  |  2 +-
 tools/libs/foreignmemory/core.c| 53 ++
 .../libs/foreignmemory/include/xenforeignmemory.h  | 41 +
 tools/libs/foreignmemory/libxenforeignmemory.map   |  5 ++
 tools/libs/foreignmemory/linux.c   | 45 ++
 tools/libs/foreignmemory/private.h | 31 +
 7 files changed, 187 insertions(+), 1 deletion(-)

diff --git a/tools/include/xen-sys/Linux/privcmd.h 
b/tools/include/xen-sys/Linux/privcmd.h
index 732ff7c15a..9531b728f9 100644
--- a/tools/include/xen-sys/Linux/privcmd.h
+++ b/tools/include/xen-sys/Linux/privcmd.h
@@ -86,6 +86,15 @@ typedef struct privcmd_dm_op {
const privcmd_dm_op_buf_t __user *ubufs;
 } privcmd_dm_op_t;
 
+typedef struct privcmd_mmap_resource {
+   domid_t dom;
+   __u32 type;
+   __u32 id;
+   __u32 idx;
+   __u64 num;
+   __u64 addr;
+} privcmd_mmap_resource_t;
+
 /*
  * @cmd: IOCTL_PRIVCMD_HYPERCALL
  * @arg: _hypercall_t
@@ -103,5 +112,7 @@ typedef struct privcmd_dm_op {
_IOC(_IOC_NONE, 'P', 5, sizeof(privcmd_dm_op_t))
 #define IOCTL_PRIVCMD_RESTRICT \
_IOC(_IOC_NONE, 'P', 6, sizeof(domid_t))
+#define IOCTL_PRIVCMD_MMAP_RESOURCE\
+   _IOC(_IOC_NONE, 'P', 7, sizeof(privcmd_mmap_resource_t))
 
 #endif /* __LINUX_PUBLIC_PRIVCMD_H__ */
diff --git a/tools/libs/foreignmemory/Makefile 
b/tools/libs/foreignmemory/Makefile
index ab7f873f26..5c7f78f61d 100644
--- a/tools/libs/foreignmemory/Makefile
+++ b/tools/libs/foreignmemory/Makefile
@@ -2,7 +2,7 @@ XEN_ROOT = $(CURDIR)/../../..
 include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR= 1
-MINOR= 2
+MINOR= 3
 SHLIB_LDFLAGS += -Wl,--version-script=libxenforeignmemory.map
 
 CFLAGS   += -Werror -Wmissing-prototypes
diff --git a/tools/libs/foreignmemory/core.c b/tools/libs/foreignmemory/core.c
index a6897dc561..8d3f9f178f 100644
--- a/tools/libs/foreignmemory/core.c
+++ b/tools/libs/foreignmemory/core.c
@@ -17,6 +17,8 @@
 #include 
 #include 
 
+#include 
+
 #include "private.h"
 
 xenforeignmemory_handle *xenforeignmemory_open(xentoollog_logger *logger,
@@ -120,6 +122,57 @@ int xenforeignmemory_restrict(xenforeignmemory_handle 
*fmem,
 return osdep_xenforeignmemory_restrict(fmem, domid);
 }
 
+xenforeignmemory_resource_handle *xenforeignmemory_map_resource(
+xenforeignmemory_handle *fmem, domid_t domid, unsigned int type,
+unsigned int id, unsigned long frame, unsigned long nr_frames,
+void **paddr, int prot, int flags)
+{
+xenforeignmemory_resource_handle *fres;
+int rc;
+
+/* Check flags only contains POSIX defined values */
+if ( flags & ~(MAP_SHARED | MAP_PRIVATE) )
+{
+errno = EINVAL;
+return NULL;
+}
+
+fres = calloc(1, sizeof(*fres));
+if ( !fres )
+{
+errno = ENOMEM;
+return NULL;
+}
+
+fres->domid = domid;
+fres->type = type;
+fres->id = id;
+fres->frame = frame;
+fres->nr_frames = nr_frames;
+fres->addr = *paddr;
+fres->prot = prot;
+fres->flags = flags;
+
+rc = osdep_xenforeignmemory_map_resource(fmem, fres);
+if ( rc )
+{
+free(fres);
+fres = NULL;
+} else
+*paddr = fres->addr;
+
+return fres;
+}
+
+int xenforeignmemory_unmap_resource(
+xenforeignmemory_handle *fmem, xenforeignmemory_resource_handle *fres)
+{
+int rc = osdep_xenforeignmemory_unmap_resource(fmem, fres);
+
+free(fres);
+return rc;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/foreignmemory/include/xenforeignmemory.h 
b/tools/libs/foreignmemory/include/xenforeignmemory.h
index f4814c390f..d594be8df0 100644
--- a/tools/libs/foreignmemory/include/xenforeignmemory.h
+++ b/tools/libs/foreignmemory/include/xenforeignmemory.h
@@ -138,6 +138,47 @@ int xenforeignmemory_unmap(xenforeignmemory_handle *fmem,
 int xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
   domid_t domid);
 
+typedef struct xenforeignmemory_resource_handle 
xenforeignmemory_resource_handle;
+
+/**
+ * This 

[Xen-devel] [PATCH v10 02/11] x86/hvm/ioreq: simplify code and use consistent naming

[Paul Durrant]

This patch re-works much of the ioreq server initialization and teardown
code:

- The hvm_map/unmap_ioreq_gfn() functions are expanded to call through
  to hvm_alloc/free_ioreq_gfn() rather than expecting them to be called
  separately by outer functions.
- Several functions now test the validity of the hvm_ioreq_page gfn value
  to determine whether they need to act. This means can be safely called
  for the bufioreq page even when it is not used.
- hvm_add/remove_ioreq_gfn() simply return in the case of the default
  IOREQ server so callers no longer need to test before calling.
- hvm_ioreq_server_setup_pages() is renamed to hvm_ioreq_server_map_pages()
  to mirror the existing hvm_ioreq_server_unmap_pages().

All of this significantly shortens the code.

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
Reviewed-by: Wei Liu 
Acked-by: Jan Beulich 
---
Cc: Andrew Cooper 

v3:
 - Rebased on top of 's->is_default' to 'IS_DEFAULT(s)' changes.
 - Minor updates in response to review comments from Roger.
---
 xen/arch/x86/hvm/ioreq.c | 182 ++-
 1 file changed, 69 insertions(+), 113 deletions(-)

diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 3403440762..6e750538a3 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -210,63 +210,75 @@ bool handle_hvm_io_completion(struct vcpu *v)
 return true;
 }
 
-static int hvm_alloc_ioreq_gfn(struct domain *d, unsigned long *gfn)
+static unsigned long hvm_alloc_ioreq_gfn(struct hvm_ioreq_server *s)
 {
+struct domain *d = s->domain;
 unsigned int i;
-int rc;
 
-rc = -ENOMEM;
+ASSERT(!IS_DEFAULT(s));
+
 for ( i = 0; i < sizeof(d->arch.hvm_domain.ioreq_gfn.mask) * 8; i++ )
 {
 if ( test_and_clear_bit(i, >arch.hvm_domain.ioreq_gfn.mask) )
-{
-*gfn = d->arch.hvm_domain.ioreq_gfn.base + i;
-rc = 0;
-break;
-}
+return d->arch.hvm_domain.ioreq_gfn.base + i;
 }
 
-return rc;
+return gfn_x(INVALID_GFN);
 }
 
-static void hvm_free_ioreq_gfn(struct domain *d, unsigned long gfn)
+static void hvm_free_ioreq_gfn(struct hvm_ioreq_server *s,
+   unsigned long gfn)
 {
+struct domain *d = s->domain;
 unsigned int i = gfn - d->arch.hvm_domain.ioreq_gfn.base;
 
-if ( gfn != gfn_x(INVALID_GFN) )
-set_bit(i, >arch.hvm_domain.ioreq_gfn.mask);
+ASSERT(!IS_DEFAULT(s));
+ASSERT(gfn != gfn_x(INVALID_GFN));
+
+set_bit(i, >arch.hvm_domain.ioreq_gfn.mask);
 }
 
-static void hvm_unmap_ioreq_page(struct hvm_ioreq_server *s, bool buf)
+static void hvm_unmap_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
 {
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
 
+if ( iorp->gfn == gfn_x(INVALID_GFN) )
+return;
+
 destroy_ring_for_helper(>va, iorp->page);
+iorp->page = NULL;
+
+if ( !IS_DEFAULT(s) )
+hvm_free_ioreq_gfn(s, iorp->gfn);
+
+iorp->gfn = gfn_x(INVALID_GFN);
 }
 
-static int hvm_map_ioreq_page(
-struct hvm_ioreq_server *s, bool buf, unsigned long gfn)
+static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
 {
 struct domain *d = s->domain;
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
-struct page_info *page;
-void *va;
 int rc;
 
-if ( (rc = prepare_ring_for_helper(d, gfn, , )) )
-return rc;
-
-if ( (iorp->va != NULL) || d->is_dying )
-{
-destroy_ring_for_helper(, page);
+if ( d->is_dying )
 return -EINVAL;
-}
 
-iorp->va = va;
-iorp->page = page;
-iorp->gfn = gfn;
+if ( IS_DEFAULT(s) )
+iorp->gfn = buf ?
+d->arch.hvm_domain.params[HVM_PARAM_BUFIOREQ_PFN] :
+d->arch.hvm_domain.params[HVM_PARAM_IOREQ_PFN];
+else
+iorp->gfn = hvm_alloc_ioreq_gfn(s);
 
-return 0;
+if ( iorp->gfn == gfn_x(INVALID_GFN) )
+return -ENOMEM;
+
+rc = prepare_ring_for_helper(d, iorp->gfn, >page, >va);
+
+if ( rc )
+hvm_unmap_ioreq_gfn(s, buf);
+
+return rc;
 }
 
 bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
@@ -279,8 +291,7 @@ bool is_ioreq_server_page(struct domain *d, const struct 
page_info *page)
 
 FOR_EACH_IOREQ_SERVER(d, id, s)
 {
-if ( (s->ioreq.va && s->ioreq.page == page) ||
- (s->bufioreq.va && s->bufioreq.page == page) )
+if ( (s->ioreq.page == page) || (s->bufioreq.page == page) )
 {
 found = true;
 break;
@@ -292,20 +303,30 @@ bool is_ioreq_server_page(struct domain *d, const struct 
page_info *page)
 return found;
 }
 
-static void hvm_remove_ioreq_gfn(
-struct domain *d, struct hvm_ioreq_page *iorp)
+static void hvm_remove_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
+
 {
+

[Xen-devel] [PATCH v10 09/11] tools/libxenforeignmemory: reduce xenforeignmemory_restrict code footprint

[Paul Durrant]

By using a static inline stub in private.h for OS where this functionality
is not implemented, the various duplicate stubs in the OS-specific source
modules can be avoided.

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
Acked-by: Wei Liu 
---
Cc: Ian Jackson 

v4:
 - Removed extraneous freebsd code.

v3:
 - Patch added in response to review comments.
---
 tools/libs/foreignmemory/freebsd.c |  7 ---
 tools/libs/foreignmemory/minios.c  |  7 ---
 tools/libs/foreignmemory/netbsd.c  |  7 ---
 tools/libs/foreignmemory/private.h | 12 +---
 tools/libs/foreignmemory/solaris.c |  7 ---
 5 files changed, 9 insertions(+), 31 deletions(-)

diff --git a/tools/libs/foreignmemory/freebsd.c 
b/tools/libs/foreignmemory/freebsd.c
index dec447485a..6e6bc4b11f 100644
--- a/tools/libs/foreignmemory/freebsd.c
+++ b/tools/libs/foreignmemory/freebsd.c
@@ -95,13 +95,6 @@ int osdep_xenforeignmemory_unmap(xenforeignmemory_handle 
*fmem,
 return munmap(addr, num << PAGE_SHIFT);
 }
 
-int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
-domid_t domid)
-{
-errno = -EOPNOTSUPP;
-return -1;
-}
-
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/foreignmemory/minios.c 
b/tools/libs/foreignmemory/minios.c
index 75f340122e..43341ca301 100644
--- a/tools/libs/foreignmemory/minios.c
+++ b/tools/libs/foreignmemory/minios.c
@@ -58,13 +58,6 @@ int osdep_xenforeignmemory_unmap(xenforeignmemory_handle 
*fmem,
 return munmap(addr, num << PAGE_SHIFT);
 }
 
-int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
-domid_t domid)
-{
-errno = -EOPNOTSUPP;
-return -1;
-}
-
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/foreignmemory/netbsd.c 
b/tools/libs/foreignmemory/netbsd.c
index 9bf95ef4f0..54a418ebd6 100644
--- a/tools/libs/foreignmemory/netbsd.c
+++ b/tools/libs/foreignmemory/netbsd.c
@@ -100,13 +100,6 @@ int osdep_xenforeignmemory_unmap(xenforeignmemory_handle 
*fmem,
 return munmap(addr, num*XC_PAGE_SIZE);
 }
 
-int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
-domid_t domid)
-{
-errno = -EOPNOTSUPP;
-return -1;
-}
-
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/foreignmemory/private.h 
b/tools/libs/foreignmemory/private.h
index 80b22bdbfc..b5d5f0a354 100644
--- a/tools/libs/foreignmemory/private.h
+++ b/tools/libs/foreignmemory/private.h
@@ -32,9 +32,6 @@ void *osdep_xenforeignmemory_map(xenforeignmemory_handle 
*fmem,
 int osdep_xenforeignmemory_unmap(xenforeignmemory_handle *fmem,
  void *addr, size_t num);
 
-int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
-domid_t domid);
-
 #if defined(__NetBSD__) || defined(__sun__)
 /* Strictly compat for those two only only */
 void *compat_mapforeign_batch(xenforeignmem_handle *fmem, uint32_t dom,
@@ -54,6 +51,13 @@ struct xenforeignmemory_resource_handle {
 };
 
 #ifndef __linux__
+static inline int osdep_xenforeignmemory_restrict(xenforeignmemory_handle 
*fmem,
+  domid_t domid)
+{
+errno = EOPNOTSUPP;
+return -1;
+}
+
 static inline int osdep_xenforeignmemory_map_resource(
 xenforeignmemory_handle *fmem, xenforeignmemory_resource_handle *fres)
 {
@@ -67,6 +71,8 @@ static inline int osdep_xenforeignmemory_unmap_resource(
 return 0;
 }
 #else
+int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
+domid_t domid);
 int osdep_xenforeignmemory_map_resource(
 xenforeignmemory_handle *fmem, xenforeignmemory_resource_handle *fres);
 int osdep_xenforeignmemory_unmap_resource(
diff --git a/tools/libs/foreignmemory/solaris.c 
b/tools/libs/foreignmemory/solaris.c
index a33decb4ae..ee8aae4fbd 100644
--- a/tools/libs/foreignmemory/solaris.c
+++ b/tools/libs/foreignmemory/solaris.c
@@ -97,13 +97,6 @@ int osdep_xenforeignmemory_unmap(xenforeignmemory_handle 
*fmem,
 return munmap(addr, num*XC_PAGE_SIZE);
 }
 
-int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
-domid_t domid)
-{
-errno = -EOPNOTSUPP;
-return -1;
-}
-
 /*
  * Local variables:
  * mode: C
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v10 11/11] tools/libxenctrl: use new xenforeignmemory API to seed grant table

[Paul Durrant]

A previous patch added support for priv-mapping guest resources directly
(rather than having to foreign-map, which requires P2M modification for
HVM guests).

This patch makes use of the new API to seed the guest grant table unless
the underlying infrastructure (i.e. privcmd) doesn't support it, in which
case the old scheme is used.

NOTE: The call to xc_dom_gnttab_hvm_seed() in hvm_build_set_params() was
  actually unnecessary, as the grant table has already been seeded
  by a prior call to xc_dom_gnttab_init() made by libxl__build_dom().

Signed-off-by: Paul Durrant 
Acked-by: Marek Marczykowski-Górecki 
Reviewed-by: Roger Pau Monné 
Acked-by: Wei Liu 
---
Cc: Ian Jackson 

v10:
 - Use new id constant for grant table.

v4:
 - Minor cosmetic fix suggested by Roger.

v3:
 - Introduced xc_dom_set_gnttab_entry() to avoid duplicated code.
---
 tools/libxc/include/xc_dom.h|   8 +--
 tools/libxc/xc_dom_boot.c   | 114 +---
 tools/libxc/xc_sr_restore_x86_hvm.c |  10 ++--
 tools/libxc/xc_sr_restore_x86_pv.c  |   2 +-
 tools/libxl/libxl_dom.c |   1 -
 tools/python/xen/lowlevel/xc/xc.c   |   6 +-
 6 files changed, 92 insertions(+), 49 deletions(-)

diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
index 6e06ef1dec..4216d63462 100644
--- a/tools/libxc/include/xc_dom.h
+++ b/tools/libxc/include/xc_dom.h
@@ -325,12 +325,8 @@ void *xc_dom_boot_domU_map(struct xc_dom_image *dom, 
xen_pfn_t pfn,
 int xc_dom_boot_image(struct xc_dom_image *dom);
 int xc_dom_compat_check(struct xc_dom_image *dom);
 int xc_dom_gnttab_init(struct xc_dom_image *dom);
-int xc_dom_gnttab_hvm_seed(xc_interface *xch, domid_t domid,
-   xen_pfn_t console_gmfn,
-   xen_pfn_t xenstore_gmfn,
-   domid_t console_domid,
-   domid_t xenstore_domid);
-int xc_dom_gnttab_seed(xc_interface *xch, domid_t domid,
+int xc_dom_gnttab_seed(xc_interface *xch, domid_t guest_domid,
+   bool is_hvm,
xen_pfn_t console_gmfn,
xen_pfn_t xenstore_gmfn,
domid_t console_domid,
diff --git a/tools/libxc/xc_dom_boot.c b/tools/libxc/xc_dom_boot.c
index 8a376d097c..0fe94aa255 100644
--- a/tools/libxc/xc_dom_boot.c
+++ b/tools/libxc/xc_dom_boot.c
@@ -282,11 +282,29 @@ static xen_pfn_t xc_dom_gnttab_setup(xc_interface *xch, 
domid_t domid)
 return gmfn;
 }
 
-int xc_dom_gnttab_seed(xc_interface *xch, domid_t domid,
-   xen_pfn_t console_gmfn,
-   xen_pfn_t xenstore_gmfn,
-   domid_t console_domid,
-   domid_t xenstore_domid)
+static void xc_dom_set_gnttab_entry(xc_interface *xch,
+grant_entry_v1_t *gnttab,
+unsigned int idx,
+domid_t guest_domid,
+domid_t backend_domid,
+xen_pfn_t backend_gmfn)
+{
+if ( guest_domid == backend_domid || backend_gmfn == -1)
+return;
+
+xc_dom_printf(xch, "%s: [%u] -> 0x%"PRI_xen_pfn,
+  __FUNCTION__, idx, backend_gmfn);
+
+gnttab[idx].flags = GTF_permit_access;
+gnttab[idx].domid = backend_domid;
+gnttab[idx].frame = backend_gmfn;
+}
+
+static int compat_gnttab_seed(xc_interface *xch, domid_t domid,
+  xen_pfn_t console_gmfn,
+  xen_pfn_t xenstore_gmfn,
+  domid_t console_domid,
+  domid_t xenstore_domid)
 {
 
 xen_pfn_t gnttab_gmfn;
@@ -310,18 +328,10 @@ int xc_dom_gnttab_seed(xc_interface *xch, domid_t domid,
 return -1;
 }
 
-if ( domid != console_domid  && console_gmfn != -1)
-{
-gnttab[GNTTAB_RESERVED_CONSOLE].flags = GTF_permit_access;
-gnttab[GNTTAB_RESERVED_CONSOLE].domid = console_domid;
-gnttab[GNTTAB_RESERVED_CONSOLE].frame = console_gmfn;
-}
-if ( domid != xenstore_domid && xenstore_gmfn != -1)
-{
-gnttab[GNTTAB_RESERVED_XENSTORE].flags = GTF_permit_access;
-gnttab[GNTTAB_RESERVED_XENSTORE].domid = xenstore_domid;
-gnttab[GNTTAB_RESERVED_XENSTORE].frame = xenstore_gmfn;
-}
+xc_dom_set_gnttab_entry(xch, gnttab, GNTTAB_RESERVED_CONSOLE,
+domid, console_domid, console_gmfn);
+xc_dom_set_gnttab_entry(xch, gnttab, GNTTAB_RESERVED_XENSTORE,
+domid, xenstore_domid, xenstore_gmfn);
 
 if ( munmap(gnttab, PAGE_SIZE) == -1 )
 {
@@ -339,11 +349,11 @@ int xc_dom_gnttab_seed(xc_interface *xch, domid_t domid,
 return 0;
 }
 
-int xc_dom_gnttab_hvm_seed(xc_interface *xch, domid_t 

Re: [Xen-devel] [PATCH v3 06/12] fuzz/x86_emulate: Take multiple test files for inputs

[George Dunlap]

On 10/10/2017 05:56 PM, Andrew Cooper wrote:
> On 10/10/17 17:20, George Dunlap wrote:
>> @@ -65,12 +68,15 @@ int main(int argc, char **argv)
>>  #ifdef __AFL_HAVE_MANUAL_CONTROL
>>  __AFL_INIT();
>>  
>> -while ( __AFL_LOOP(1000) )
>> +for( count = 0; __AFL_LOOP(1000); )
>> +#else
>> +for( count = 0; count < max; count++ )
>>  #endif
>>  {
>>  if ( fp != stdin ) /* If not using stdin, open the provided file. */
>>  {
>> -fp = fopen(argv[optind], "rb");
>> +printf("Opening file %s\n", argv[optind]);
>> +fp = fopen(argv[optind + count], "rb");
> 
> I presume the printf() wants adjusting to match the fopen() ?

Oh!  I thought I'd fixed that.  Indeed it does.

I can fix that on check-in, if we don't find anything bigger worth
re-sending for.

 -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v10 10/11] common: add a new mappable resource type: XENMEM_resource_grant_table

[Paul Durrant]

This patch allows grant table frames to be mapped using the
XENMEM_acquire_resource memory op.

Signed-off-by: Paul Durrant 
---
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Jan Beulich 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Wei Liu 

v10:
 - Addressed comments from Jan.

v8:
 - The functionality was originally incorporated into the earlier patch
   "x86/mm: add HYPERVISOR_memory_op to acquire guest resources".
---
 xen/common/grant_table.c  | 63 ++-
 xen/common/memory.c   | 44 +-
 xen/include/public/memory.h   |  6 +
 xen/include/xen/grant_table.h |  4 +++
 4 files changed, 110 insertions(+), 7 deletions(-)

diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index 6d20b17739..e42c1b6bf3 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -1608,7 +1608,8 @@ fault:
 }
 
 static int
-gnttab_populate_status_frames(struct domain *d, struct grant_table *gt,
+gnttab_populate_status_frames(struct domain *d,
+  struct grant_table *gt,
   unsigned int req_nr_frames)
 {
 unsigned i;
@@ -3756,13 +3757,12 @@ int mem_sharing_gref_to_gfn(struct grant_table *gt, 
grant_ref_t ref,
 }
 #endif
 
-int gnttab_map_frame(struct domain *d, unsigned long idx, gfn_t gfn,
- mfn_t *mfn)
+/* Caller must hold write lock as version may change and table may grow */
+static int gnttab_get_frame(struct domain *d, unsigned long idx,
+mfn_t *mfn)
 {
-int rc = 0;
 struct grant_table *gt = d->grant_table;
-
-grant_write_lock(gt);
+int rc = 0;
 
 if ( gt->gt_version == 0 )
 gt->gt_version = 1;
@@ -3787,6 +3787,19 @@ int gnttab_map_frame(struct domain *d, unsigned long 
idx, gfn_t gfn,
 rc = -EINVAL;
 }
 
+return rc;
+}
+
+int gnttab_map_frame(struct domain *d, unsigned long idx, gfn_t gfn,
+ mfn_t *mfn)
+{
+struct grant_table *gt = d->grant_table;
+int rc;
+
+grant_write_lock(gt);
+
+rc = gnttab_get_frame(d, idx, mfn);
+
 if ( !rc )
 gnttab_set_frame_gfn(gt, idx, gfn);
 
@@ -3795,6 +3808,44 @@ int gnttab_map_frame(struct domain *d, unsigned long 
idx, gfn_t gfn,
 return rc;
 }
 
+int gnttab_get_grant_frame(struct domain *d, unsigned long idx,
+   mfn_t *mfn)
+{
+struct grant_table *gt = d->grant_table;
+int rc;
+
+/* write lock required as version may change and/or table may grow */
+grant_write_lock(gt);
+
+rc = (gt->gt_version == 2 &&
+  idx > XENMAPIDX_grant_table_status) ?
+-EINVAL :
+gnttab_get_frame(d, idx, mfn);
+
+grant_write_unlock(gt);
+
+return rc;
+}
+
+int gnttab_get_status_frame(struct domain *d, unsigned long idx,
+mfn_t *mfn)
+{
+struct grant_table *gt = d->grant_table;
+int rc;
+
+/* write lock required as version may change and/or table may grow */
+grant_write_lock(gt);
+
+rc = (gt->gt_version != 2 ||
+  idx > XENMAPIDX_grant_table_status) ?
+-EINVAL :
+gnttab_get_frame(d, idx & XENMAPIDX_grant_table_status, mfn);
+
+grant_write_unlock(gt);
+
+return rc;
+}
+
 static void gnttab_usage_print(struct domain *rd)
 {
 int first = 1;
diff --git a/xen/common/memory.c b/xen/common/memory.c
index 6f01cc9663..42d5440979 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -23,6 +23,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -965,10 +966,46 @@ static long xatp_permission_check(struct domain *d, 
unsigned int space)
 return xsm_add_to_physmap(XSM_TARGET, current->domain, d);
 }
 
+static int acquire_grant_table(struct domain *d, unsigned int id,
+   unsigned long frame,
+   unsigned int nr_frames,
+   unsigned long mfn_list[])
+{
+unsigned int i = nr_frames;
+
+while ( i-- != 0 )
+{
+mfn_t mfn = INVALID_MFN;
+int rc;
+
+switch ( id )
+{
+case XENMEM_resource_grant_table_id_grant:
+rc = gnttab_get_grant_frame(d, frame + i, );
+break;
+
+case XENMEM_resource_grant_table_id_status:
+rc = gnttab_get_status_frame(d, frame + i, );
+break;
+
+default:
+rc = -EINVAL;
+break;
+}
+
+if ( rc )
+return rc;
+
+mfn_list[i] = mfn_x(mfn);
+}
+
+return 0;
+}
+
 static int acquire_resource(const xen_mem_acquire_resource_t *xmar)
 {
 struct domain *d, *currd = current->domain;
-unsigned long 

[Xen-devel] [PATCH v10 00/11] x86: guest resource mapping

[Paul Durrant]

This series introduces support for direct mapping of guest resources.
The resources are:
 - IOREQ server pages
 - Grant tables

v10:
 - Responded to comments from Jan.

v9:
 - Change to patch #1 only.

v8:
 - Re-ordered series and dropped two patches that have already been
   committed.

v7:
 - Fixed assertion failure hit during domain destroy.

v6:
 - Responded to missed comments from Roger.

v5:
 - Responded to review comments from Wei.

v4:
 - Responded to further review comments from Roger.

v3:
 - Dropped original patch #1 since it is covered by Juergen's patch.
 - Added new xenforeignmemorycleanup patch (#4).
 - Replaced the patch introducing the ioreq server 'is_default' flag with one
   that changes the ioreq server list into an array (#8).

Paul Durrant (11):
  x86/hvm/ioreq: maintain an array of ioreq servers rather than a list
  x86/hvm/ioreq: simplify code and use consistent naming
  x86/hvm/ioreq: use gfn_t in struct hvm_ioreq_page
  x86/hvm/ioreq: defer mapping gfns until they are actually requsted
  x86/mm: add HYPERVISOR_memory_op to acquire guest resources
  x86/hvm/ioreq: add a new mappable resource type...
  x86/mm: add an extra command to HYPERVISOR_mmu_update...
  tools/libxenforeignmemory: add support for resource mapping
  tools/libxenforeignmemory: reduce xenforeignmemory_restrict code
footprint
  common: add a new mappable resource type: XENMEM_resource_grant_table
  tools/libxenctrl: use new xenforeignmemory API to seed grant table

 tools/include/xen-sys/Linux/privcmd.h  |  11 +
 tools/libs/devicemodel/core.c  |   8 +
 tools/libs/devicemodel/include/xendevicemodel.h|   6 +-
 tools/libs/foreignmemory/Makefile  |   2 +-
 tools/libs/foreignmemory/core.c|  53 ++
 tools/libs/foreignmemory/freebsd.c |   7 -
 .../libs/foreignmemory/include/xenforeignmemory.h  |  41 +
 tools/libs/foreignmemory/libxenforeignmemory.map   |   5 +
 tools/libs/foreignmemory/linux.c   |  45 ++
 tools/libs/foreignmemory/minios.c  |   7 -
 tools/libs/foreignmemory/netbsd.c  |   7 -
 tools/libs/foreignmemory/private.h |  43 +-
 tools/libs/foreignmemory/solaris.c |   7 -
 tools/libxc/include/xc_dom.h   |   8 +-
 tools/libxc/xc_dom_boot.c  | 114 ++-
 tools/libxc/xc_sr_restore_x86_hvm.c|  10 +-
 tools/libxc/xc_sr_restore_x86_pv.c |   2 +-
 tools/libxl/libxl_dom.c|   1 -
 tools/python/xen/lowlevel/xc/xc.c  |   6 +-
 xen/arch/x86/hvm/dm.c  |   9 +-
 xen/arch/x86/hvm/ioreq.c   | 829 -
 xen/arch/x86/mm.c  |  39 +-
 xen/arch/x86/mm/p2m.c  |   3 +-
 xen/common/compat/memory.c |  65 ++
 xen/common/grant_table.c   |  63 +-
 xen/common/memory.c| 132 
 xen/include/asm-x86/hvm/domain.h   |  14 +-
 xen/include/asm-x86/hvm/ioreq.h|   2 +
 xen/include/asm-x86/mm.h   |   5 +
 xen/include/asm-x86/p2m.h  |   3 +
 xen/include/public/hvm/dm_op.h |  36 +-
 xen/include/public/memory.h|  47 +-
 xen/include/public/xen.h   |  12 +-
 xen/include/xen/grant_table.h  |   4 +
 xen/include/xlat.lst   |   1 +
 35 files changed, 1153 insertions(+), 494 deletions(-)

---
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Jan Beulich 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Wei Liu 

-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v10 01/11] x86/hvm/ioreq: maintain an array of ioreq servers rather than a list

[Paul Durrant]

A subsequent patch will remove the current implicit limitation on creation
of ioreq servers which is due to the allocation of gfns for the ioreq
structures and buffered ioreq ring.

It will therefore be necessary to introduce an explicit limit and, since
this limit should be small, it simplifies the code to maintain an array of
that size rather than using a list.

Also, by reserving an array slot for the default server and populating
array slots early in create, the need to pass an 'is_default' boolean
to sub-functions can be avoided.

Some function return values are changed by this patch: Specifically, in
the case where the id of the default ioreq server is passed in, -EOPNOTSUPP
is now returned rather than -ENOENT.

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 

v10:
 - modified FOR_EACH... macro as suggested by Jan.
 - check for NULL in IS_DEFAULT macro as suggested by Jan.

v9:
 - modified FOR_EACH... macro as requested by Andrew.

v8:
 - Addressed various comments from Jan.

v7:
 - Fixed assertion failure found in testing.

v6:
 - Updated according to comments made by Roger on v4 that I'd missed.

v5:
 - Switched GET/SET_IOREQ_SERVER() macros to get/set_ioreq_server()
   functions to avoid possible double-evaluation issues.

v4:
 - Introduced more helper macros and relocated them to the top of the
   code.

v3:
 - New patch (replacing "move is_default into struct hvm_ioreq_server") in
   response to review comments.
---
 xen/arch/x86/hvm/ioreq.c | 502 +++
 xen/include/asm-x86/hvm/domain.h |  10 +-
 2 files changed, 245 insertions(+), 267 deletions(-)

diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index f2e0b3f74a..3403440762 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -33,6 +33,37 @@
 
 #include 
 
+static void set_ioreq_server(struct domain *d, unsigned int id,
+ struct hvm_ioreq_server *s)
+{
+ASSERT(id < MAX_NR_IOREQ_SERVERS);
+ASSERT(!s || !d->arch.hvm_domain.ioreq_server.server[id]);
+
+d->arch.hvm_domain.ioreq_server.server[id] = s;
+}
+
+#define GET_IOREQ_SERVER(d, id) \
+(d)->arch.hvm_domain.ioreq_server.server[id]
+
+static struct hvm_ioreq_server *get_ioreq_server(const struct domain *d,
+ unsigned int id)
+{
+if ( id >= MAX_NR_IOREQ_SERVERS )
+return NULL;
+
+return GET_IOREQ_SERVER(d, id);
+}
+
+#define IS_DEFAULT(s) \
+((s) && (s) == get_ioreq_server((s)->domain, DEFAULT_IOSERVID))
+
+/* Iterate over all possible ioreq servers */
+#define FOR_EACH_IOREQ_SERVER(d, id, s) \
+for ( (id) = 0; (id) < MAX_NR_IOREQ_SERVERS; (id)++ ) \
+if ( !(s = GET_IOREQ_SERVER((d), (id))) ) \
+continue; \
+else
+
 static ioreq_t *get_ioreq(struct hvm_ioreq_server *s, struct vcpu *v)
 {
 shared_iopage_t *p = s->ioreq.va;
@@ -47,10 +78,9 @@ bool hvm_io_pending(struct vcpu *v)
 {
 struct domain *d = v->domain;
 struct hvm_ioreq_server *s;
+unsigned int id;
 
-list_for_each_entry ( s,
-  >arch.hvm_domain.ioreq_server.list,
-  list_entry )
+FOR_EACH_IOREQ_SERVER(d, id, s)
 {
 struct hvm_ioreq_vcpu *sv;
 
@@ -127,10 +157,9 @@ bool handle_hvm_io_completion(struct vcpu *v)
 struct hvm_vcpu_io *vio = >arch.hvm_vcpu.hvm_io;
 struct hvm_ioreq_server *s;
 enum hvm_io_completion io_completion;
+unsigned int id;
 
-  list_for_each_entry ( s,
-  >arch.hvm_domain.ioreq_server.list,
-  list_entry )
+FOR_EACH_IOREQ_SERVER(d, id, s)
 {
 struct hvm_ioreq_vcpu *sv;
 
@@ -243,13 +272,12 @@ static int hvm_map_ioreq_page(
 bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
 {
 const struct hvm_ioreq_server *s;
+unsigned int id;
 bool found = false;
 
 spin_lock_recursive(>arch.hvm_domain.ioreq_server.lock);
 
-list_for_each_entry ( s,
-  >arch.hvm_domain.ioreq_server.list,
-  list_entry )
+FOR_EACH_IOREQ_SERVER(d, id, s)
 {
 if ( (s->ioreq.va && s->ioreq.page == page) ||
  (s->bufioreq.va && s->bufioreq.page == page) )
@@ -302,7 +330,7 @@ static void hvm_update_ioreq_evtchn(struct hvm_ioreq_server 
*s,
 }
 
 static int hvm_ioreq_server_add_vcpu(struct hvm_ioreq_server *s,
- bool is_default, struct vcpu *v)
+ struct vcpu *v)
 {
 struct hvm_ioreq_vcpu *sv;
 int rc;
@@ -331,7 +359,7 @@ static int hvm_ioreq_server_add_vcpu(struct 
hvm_ioreq_server *s,
 goto fail3;
 
 s->bufioreq_evtchn = rc;
-if ( is_default )
+if ( IS_DEFAULT(s) )
 

Re: [Xen-devel] [PATCH v3 01/12] fuzz/x86_emulate: Clear errors after each iteration

[George Dunlap]

On 10/10/2017 05:20 PM, George Dunlap wrote:
> Once feof() returns true for a stream, it will continue to return true
> for that stream until clearerr() is called (or the stream is closed
> and re-opened).
> 
> In llvm-clang-fast-mode, the same file descriptor is used for each
> iteration of the loop, meaning that the "Input too large" check was
> broken -- feof() would return true even if the fread() hadn't hit the
> end of the file.  The result is that AFL generates testcases of
> arbitrary size.
> 
> Fix this by fseek'ing to the beginning of the file on every iteration;
> this resets the EOF marker and other state.
> 
> Signed-off-by: George Dunlap 
> ---
> Changes in v3:
> - Fix the issue in the official sanctioned way

Hmm, seems v2 of this patch was checked in; review had flagged up that
"clearerr()" was too big of a hammer.

Attached is a revised v1/12 patch that fixes this.

 -George
From d07b2d68085957bf3d7a2567dce9c4f031fb5966 Mon Sep 17 00:00:00 2001
From: George Dunlap 
Date: Wed, 4 Oct 2017 17:09:10 +0100
Subject: [PATCH] fuzz/x86_emulate: Clear errors in the officially sanctioned
 way

Commit 849a1f10c9 was checked in in appropriately; review flagged up
that clearerr() was too big a hammer, as it would clear both the EOF
flag and stream errors.

Stream errors shouldn't be cleared; we only want the EOF and other
stream-related state cleared.  To do this, it is sufficient to fseek()
to zero.

Signed-off-by: George Dunlap 
---
This is a candidate for backport to 4.9.

CC: Ian Jackson 
CC: Wei Liu 
CC: Andrew Cooper 
CC: Jan Beulich 
---
 tools/fuzz/x86_instruction_emulator/afl-harness.c | 13 +++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/tools/fuzz/x86_instruction_emulator/afl-harness.c b/tools/fuzz/x86_instruction_emulator/afl-harness.c
index b4d15451b5..31ae1daef1 100644
--- a/tools/fuzz/x86_instruction_emulator/afl-harness.c
+++ b/tools/fuzz/x86_instruction_emulator/afl-harness.c
@@ -77,6 +77,17 @@ int main(int argc, char **argv)
 exit(-1);
 }
 }
+#ifdef __AFL_HAVE_MANUAL_CONTROL
+else
+{
+/* 
+ * This will ensure we're dealing with a clean stream
+ * state after the afl-fuzz process messes with the open
+ * file handle.
+ */
+fseek(fp, 0, SEEK_SET);
+}
+#endif
 
 size = fread(input, 1, INPUT_SIZE, fp);
 
@@ -97,8 +108,6 @@ int main(int argc, char **argv)
 fclose(fp);
 fp = NULL;
 }
-else
-clearerr(fp);
 
 LLVMFuzzerTestOneInput(input, size);
 }
-- 
2.14.2

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v10 04/11] x86/hvm/ioreq: defer mapping gfns until they are actually requsted

[Paul Durrant]

A subsequent patch will introduce a new scheme to allow an emulator to
map ioreq server pages directly from Xen rather than the guest P2M.

This patch lays the groundwork for that change by deferring mapping of
gfns until their values are requested by an emulator. To that end, the
pad field of the xen_dm_op_get_ioreq_server_info structure is re-purposed
to a flags field and new flag, XEN_DMOP_no_gfns, defined which modifies the
behaviour of XEN_DMOP_get_ioreq_server_info to allow the caller to avoid
requesting the gfn values.

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
Acked-by: Wei Liu 
Reviewed-by: Jan Beulich 
---
Cc: Ian Jackson 
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 

v8:
 - For safety make all of the pointers passed to
   hvm_get_ioreq_server_info() optional.
 - Shrink bufioreq_handling down to a uint8_t.

v3:
 - Updated in response to review comments from Wei and Roger.
 - Added a HANDLE_BUFIOREQ macro to make the code neater.
 - This patch no longer introduces a security vulnerability since there
   is now an explicit limit on the number of ioreq servers that may be
   created for any one domain.
---
 tools/libs/devicemodel/core.c   |  8 +
 tools/libs/devicemodel/include/xendevicemodel.h |  6 ++--
 xen/arch/x86/hvm/dm.c   |  9 +++--
 xen/arch/x86/hvm/ioreq.c| 47 ++---
 xen/include/asm-x86/hvm/domain.h|  2 +-
 xen/include/public/hvm/dm_op.h  | 32 ++---
 6 files changed, 63 insertions(+), 41 deletions(-)

diff --git a/tools/libs/devicemodel/core.c b/tools/libs/devicemodel/core.c
index 0f2c1a791f..91c69d103b 100644
--- a/tools/libs/devicemodel/core.c
+++ b/tools/libs/devicemodel/core.c
@@ -188,6 +188,14 @@ int xendevicemodel_get_ioreq_server_info(
 
 data->id = id;
 
+/*
+ * If the caller is not requesting gfn values then instruct the
+ * hypercall not to retrieve them as this may cause them to be
+ * mapped.
+ */
+if (!ioreq_gfn && !bufioreq_gfn)
+data->flags |= XEN_DMOP_no_gfns;
+
 rc = xendevicemodel_op(dmod, domid, 1, , sizeof(op));
 if (rc)
 return rc;
diff --git a/tools/libs/devicemodel/include/xendevicemodel.h 
b/tools/libs/devicemodel/include/xendevicemodel.h
index 13216db04a..d73a76da35 100644
--- a/tools/libs/devicemodel/include/xendevicemodel.h
+++ b/tools/libs/devicemodel/include/xendevicemodel.h
@@ -61,11 +61,11 @@ int xendevicemodel_create_ioreq_server(
  * @parm domid the domain id to be serviced
  * @parm id the IOREQ Server id.
  * @parm ioreq_gfn pointer to a xen_pfn_t to receive the synchronous ioreq
- *  gfn
+ *  gfn. (May be NULL if not required)
  * @parm bufioreq_gfn pointer to a xen_pfn_t to receive the buffered ioreq
- *gfn
+ *gfn. (May be NULL if not required)
  * @parm bufioreq_port pointer to a evtchn_port_t to receive the buffered
- * ioreq event channel
+ * ioreq event channel. (May be NULL if not required)
  * @return 0 on success, -1 on failure.
  */
 int xendevicemodel_get_ioreq_server_info(
diff --git a/xen/arch/x86/hvm/dm.c b/xen/arch/x86/hvm/dm.c
index 9cf53b551c..22fa5b51e3 100644
--- a/xen/arch/x86/hvm/dm.c
+++ b/xen/arch/x86/hvm/dm.c
@@ -416,16 +416,19 @@ static int dm_op(const struct dmop_args *op_args)
 {
 struct xen_dm_op_get_ioreq_server_info *data =
 _ioreq_server_info;
+const uint16_t valid_flags = XEN_DMOP_no_gfns;
 
 const_op = false;
 
 rc = -EINVAL;
-if ( data->pad )
+if ( data->flags & ~valid_flags )
 break;
 
 rc = hvm_get_ioreq_server_info(d, data->id,
-   >ioreq_gfn,
-   >bufioreq_gfn,
+   (data->flags & XEN_DMOP_no_gfns) ?
+   NULL : >ioreq_gfn,
+   (data->flags & XEN_DMOP_no_gfns) ?
+   NULL : >bufioreq_gfn,
>bufioreq_port);
 break;
 }
diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 4f20ef7108..52b1381ee3 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -350,6 +350,9 @@ static void hvm_update_ioreq_evtchn(struct hvm_ioreq_server 
*s,
 }
 }
 
+#define HANDLE_BUFIOREQ(s) \
+((s)->bufioreq_handling != HVM_IOREQSRV_BUFIOREQ_OFF)
+
 static int hvm_ioreq_server_add_vcpu(struct hvm_ioreq_server *s,
  struct vcpu *v)
 {
@@ 

[Xen-devel] [PATCH v10 06/11] x86/hvm/ioreq: add a new mappable resource type...

[Paul Durrant]

... XENMEM_resource_ioreq_server

This patch adds support for a new resource type that can be mapped using
the XENMEM_acquire_resource memory op.

If an emulator makes use of this resource type then, instead of mapping
gfns, the IOREQ server will allocate pages from the heap. These pages
will never be present in the P2M of the guest at any point and so are
not vulnerable to any direct attack by the guest. They are only ever
accessible by Xen and any domain that has mapping privilege over the
guest (which may or may not be limited to the domain running the emulator).

NOTE: Use of the new resource type is not compatible with use of
  XEN_DMOP_get_ioreq_server_info unless the XEN_DMOP_no_gfns flag is
  set.

Signed-off-by: Paul Durrant 
Acked-by: George Dunlap 
Reviewed-by: Wei Liu 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 
Cc: Ian Jackson 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 

v10:
 - Addressed comments from Jan.

v8:
 - Re-base on new boilerplate.
 - Adjust function signature of hvm_get_ioreq_server_frame(), and test
   whether the bufioreq page is present.

v5:
 - Use get_ioreq_server() function rather than indexing array directly.
 - Add more explanation into comments to state than mapping guest frames
   and allocation of pages for ioreq servers are not simultaneously
   permitted.
 - Add a comment into asm/ioreq.h stating the meaning of the index
   value passed to hvm_get_ioreq_server_frame().
---
 xen/arch/x86/hvm/ioreq.c| 154 
 xen/arch/x86/mm.c   |  22 ++
 xen/common/memory.c |   5 ++
 xen/include/asm-x86/hvm/ioreq.h |   2 +
 xen/include/asm-x86/mm.h|   5 ++
 xen/include/public/hvm/dm_op.h  |   4 ++
 xen/include/public/memory.h |   9 +++
 7 files changed, 201 insertions(+)

diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 52b1381ee3..b7e6617f44 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -259,6 +259,19 @@ static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
 int rc;
 
+if ( iorp->page )
+{
+/*
+ * If a page has already been allocated (which will happen on
+ * demand if hvm_get_ioreq_server_frame() is called), then
+ * mapping a guest frame is not permitted.
+ */
+if ( gfn_eq(iorp->gfn, INVALID_GFN) )
+return -EPERM;
+
+return 0;
+}
+
 if ( d->is_dying )
 return -EINVAL;
 
@@ -281,6 +294,69 @@ static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 return rc;
 }
 
+static int hvm_alloc_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
+{
+struct domain *currd = current->domain;
+struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
+
+if ( iorp->page )
+{
+/*
+ * If a guest frame has already been mapped (which may happen
+ * on demand if hvm_get_ioreq_server_info() is called), then
+ * allocating a page is not permitted.
+ */
+if ( !gfn_eq(iorp->gfn, INVALID_GFN) )
+return -EPERM;
+
+return 0;
+}
+
+/*
+ * Allocated IOREQ server pages are assigned to the emulating
+ * domain, not the target domain. This is because the emulator is
+ * likely to be destroyed after the target domain has been torn
+ * down, and we must use MEMF_no_refcount otherwise page allocation
+ * could fail if the emulating domain has already reached its
+ * maximum allocation.
+ */
+iorp->page = alloc_domheap_page(currd, MEMF_no_refcount);
+if ( !iorp->page )
+return -ENOMEM;
+
+if ( !get_page_type(iorp->page, PGT_writable_page) )
+{
+put_page(iorp->page);
+iorp->page = NULL;
+return -ENOMEM;
+}
+
+iorp->va = __map_domain_page_global(iorp->page);
+if ( !iorp->va )
+{
+put_page_and_type(iorp->page);
+iorp->page = NULL;
+return -ENOMEM;
+}
+
+clear_page(iorp->va);
+return 0;
+}
+
+static void hvm_free_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
+{
+struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
+
+if ( !iorp->page )
+return;
+
+unmap_domain_page_global(iorp->va);
+iorp->va = NULL;
+
+put_page_and_type(iorp->page);
+iorp->page = NULL;
+}
+
 bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
 {
 const struct hvm_ioreq_server *s;
@@ -484,6 +560,27 @@ static void hvm_ioreq_server_unmap_pages(struct 
hvm_ioreq_server *s)
 hvm_unmap_ioreq_gfn(s, false);
 }
 
+static int hvm_ioreq_server_alloc_pages(struct hvm_ioreq_server *s)
+{
+int rc;
+
+rc = hvm_alloc_ioreq_mfn(s, false);

Re: [Xen-devel] [PATCH] x86emul: handle address wrapping for VMASKMOVP{S, D}

[Andrew Cooper]

On 10/10/17 11:43, Jan Beulich wrote:
> I failed to recognize the need to mirror the changes done by 7869e2bafe
> ("x86emul/fuzz: add rudimentary limit checking") into the earlier
> written but later committed 2fe43d333f ("x86emul: support remaining AVX
> insns"): Behavior here is the same as for multi-part reads or writes.
>
> Reported-by: Andrew Cooper 
> Signed-off-by: Jan Beulich 

Acked-by: Andrew Cooper 

> ---
> There's another issue here, but I'll first have to think about possible
> (preferably non-intrusive) solutions: An access crossing a page
> boundary and having
> - a set mask bit corresponding to an element fully living in the first
>   page,
> - one or more clear mask bits corresponding to the initial elements on
>   the second page,
> - another higher mask bit being set
> would result in a wrong CR2 value to be reported in case the access to
> the second page would cause a fault (it would point to the start of the
> page instead of the element being accessed). Neither splitting the
> access here into multiple ones nor uniformly passing a byte or word
> enable mask into ->write() seem very desirable.

Is this just supposition, or have you confirmed what really happens on
hardware?

I expect that the mask operations turn into multi-part operations, which
means their behaviour on a straddled fault is implementation defined
(and behaves differently between Atoms and Xeons).

One option we could do is to have a variation of the "Implement
hvmemul_write() using real mappings" logic where we pull mappings into
the vmap individually, but that would require some part of the code to
convert ea + mask => linear address of each unit, so the eventual
mapping can be constructed piece-wise.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v3 07/12] fuzz/x86_emulate: Move all state into fuzz_state

[George Dunlap]

This is in preparation for adding the option for a more "compact"
interpretation of the fuzzing data, in which we only change select
bits of the state.

Signed-off-by: George Dunlap 
Acked-by: Jan Beulich 
---
v3:
 - Move DATA_OFFSET inside the structure
 - Remove a stray blank line
v2: Port over previous changes

CC: Ian Jackson 
CC: Wei Liu 
CC: Andrew Cooper 
CC: Jan Beulich 
---
 tools/fuzz/x86_instruction_emulator/fuzz-emul.c | 89 +
 1 file changed, 45 insertions(+), 44 deletions(-)

diff --git a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c 
b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
index 8998f21fe1..20d52b33f8 100644
--- a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
+++ b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
@@ -24,14 +24,8 @@
 /* Layout of data expected as fuzzing input. */
 struct fuzz_corpus
 {
-unsigned long cr[5];
-uint64_t msr[MSR_INDEX_MAX];
-struct cpu_user_regs regs;
-struct segment_register segments[SEG_NUM];
-unsigned long options;
 unsigned char data[4096];
 } input;
-#define DATA_OFFSET offsetof(struct fuzz_corpus, data)
 
 /*
  * Internal state of the fuzzing harness.  Calculated initially from the input
@@ -39,7 +33,14 @@ struct fuzz_corpus
  */
 struct fuzz_state
 {
+unsigned long options;
+unsigned long cr[5];
+uint64_t msr[MSR_INDEX_MAX];
+struct segment_register segments[SEG_NUM];
+struct cpu_user_regs regs;
+
 /* Fuzzer's input data. */
+#define DATA_OFFSET offsetof(struct fuzz_state, corpus)
 struct fuzz_corpus *corpus;
 
 /* Real amount of data backing corpus->data[]. */
@@ -392,11 +393,10 @@ static int fuzz_read_segment(
 struct x86_emulate_ctxt *ctxt)
 {
 const struct fuzz_state *s = ctxt->data;
-const struct fuzz_corpus *c = s->corpus;
 
 assert(is_x86_user_segment(seg) || is_x86_system_segment(seg));
 
-*reg = c->segments[seg];
+*reg = s->segments[seg];
 
 return X86EMUL_OKAY;
 }
@@ -407,7 +407,6 @@ static int fuzz_write_segment(
 struct x86_emulate_ctxt *ctxt)
 {
 struct fuzz_state *s = ctxt->data;
-struct fuzz_corpus *c = s->corpus;
 int rc;
 
 assert(is_x86_user_segment(seg) || is_x86_system_segment(seg));
@@ -415,7 +414,7 @@ static int fuzz_write_segment(
 rc = maybe_fail(ctxt, "write_segment", true);
 
 if ( rc == X86EMUL_OKAY )
-c->segments[seg] = *reg;
+s->segments[seg] = *reg;
 
 return rc;
 }
@@ -426,12 +425,11 @@ static int fuzz_read_cr(
 struct x86_emulate_ctxt *ctxt)
 {
 const struct fuzz_state *s = ctxt->data;
-const struct fuzz_corpus *c = s->corpus;
 
-if ( reg >= ARRAY_SIZE(c->cr) )
+if ( reg >= ARRAY_SIZE(s->cr) )
 return X86EMUL_UNHANDLEABLE;
 
-*val = c->cr[reg];
+*val = s->cr[reg];
 
 return X86EMUL_OKAY;
 }
@@ -442,17 +440,16 @@ static int fuzz_write_cr(
 struct x86_emulate_ctxt *ctxt)
 {
 struct fuzz_state *s = ctxt->data;
-struct fuzz_corpus *c = s->corpus;
 int rc;
 
-if ( reg >= ARRAY_SIZE(c->cr) )
+if ( reg >= ARRAY_SIZE(s->cr) )
 return X86EMUL_UNHANDLEABLE;
 
 rc = maybe_fail(ctxt, "write_cr", true);
 if ( rc != X86EMUL_OKAY )
 return rc;
 
-c->cr[reg] = val;
+s->cr[reg] = val;
 
 return X86EMUL_OKAY;
 }
@@ -487,7 +484,6 @@ static int fuzz_read_msr(
 struct x86_emulate_ctxt *ctxt)
 {
 const struct fuzz_state *s = ctxt->data;
-const struct fuzz_corpus *c = s->corpus;
 unsigned int idx;
 
 switch ( reg )
@@ -501,10 +497,10 @@ static int fuzz_read_msr(
  */
 return data_read(ctxt, x86_seg_none, "read_msr", val, sizeof(*val));
 case MSR_EFER:
-*val = c->msr[MSRI_EFER];
+*val = s->msr[MSRI_EFER];
 *val &= ~EFER_LMA;
-if ( (*val & EFER_LME) && (c->cr[4] & X86_CR4_PAE) &&
- (c->cr[0] & X86_CR0_PG) )
+if ( (*val & EFER_LME) && (s->cr[4] & X86_CR4_PAE) &&
+ (s->cr[0] & X86_CR0_PG) )
 {
 printf("Setting EFER_LMA\n");
 *val |= EFER_LMA;
@@ -516,7 +512,7 @@ static int fuzz_read_msr(
 {
 if ( msr_index[idx] == reg )
 {
-*val = c->msr[idx];
+*val = s->msr[idx];
 return X86EMUL_OKAY;
 }
 }
@@ -531,7 +527,6 @@ static int fuzz_write_msr(
 struct x86_emulate_ctxt *ctxt)
 {
 struct fuzz_state *s = ctxt->data;
-struct fuzz_corpus *c = s->corpus;
 unsigned int idx;
 int rc;
 
@@ -550,7 +545,7 @@ static int fuzz_write_msr(
 {
 if ( msr_index[idx] == reg )
 {
-c->msr[idx] = val;
+s->msr[idx] = val;
 return X86EMUL_OKAY;
 }
 }
@@ -600,15 +595,14 @@ static void setup_fpu_exception_handler(void)
 static void dump_state(struct x86_emulate_ctxt *ctxt)
 {
 struct fuzz_state 

[Xen-devel] [PATCH v3 10/12] fuzz/x86_emulate: Add --rerun option to try to track down instability

[George Dunlap]

Current stability numbers are not 100%.  In order to help track this
down, add a --rerun option which will run the same input twice,
resetting the state between each run, and comparing the state
afterwards.  If the state differs, call abort().

This allows AFL to help the process of tracking down what state is not
being reset properly between runs by proving testcases that
demonstrate the behavior.

To do this:

- Move ctxt into struct fuzz-state to simplify handling

- Rather than copying the data into input, treat the data handed as
  immutable and point each "copy" to it

- Factor out various steps (setting up fuzz state, running an
  individual test) so that they can be efficiently run either once or
  twice (as necessary)

- Compare the states afterwards, printing what's different and calling
  abort() if anything is found.

Signed-off-by: George Dunlap 
---
v3:
- Make new local functions static
- Avoid losing const-ness of pointer in setup_fuzz_state()
- Remove useless *_size initialization
- Remove extra blank line
- Use ARRAY_SIZE() when appropriate
- Move opt_rerun declaration into fuzz-emul.h
- Change loop variable to unsigned int
- Print segment contents when segments differ
v2:
- Fix some coding style issues
- Port over previous changes

CC: Ian Jackson 
CC: Wei Liu 
CC: Andrew Cooper 
CC: Jan Beulich 
---
 tools/fuzz/x86_instruction_emulator/afl-harness.c |   8 +-
 tools/fuzz/x86_instruction_emulator/fuzz-emul.c   | 194 ++
 tools/fuzz/x86_instruction_emulator/fuzz-emul.h   |   1 +
 3 files changed, 171 insertions(+), 32 deletions(-)

diff --git a/tools/fuzz/x86_instruction_emulator/afl-harness.c 
b/tools/fuzz/x86_instruction_emulator/afl-harness.c
index 052239cea4..4a55ac3c3f 100644
--- a/tools/fuzz/x86_instruction_emulator/afl-harness.c
+++ b/tools/fuzz/x86_instruction_emulator/afl-harness.c
@@ -23,10 +23,12 @@ int main(int argc, char **argv)
 enum {
 OPT_MIN_SIZE,
 OPT_COMPACT,
+OPT_RERUN,
 };
 static const struct option lopts[] = {
 { "min-input-size", no_argument, NULL, OPT_MIN_SIZE },
 { "compact", required_argument, NULL, OPT_COMPACT },
+{ "rerun", no_argument, NULL, OPT_RERUN },
 { 0, 0, 0, 0 }
 };
 int c = getopt_long_only(argc, argv, "", lopts, NULL);
@@ -45,8 +47,12 @@ int main(int argc, char **argv)
 opt_compact = atoi(optarg);
 break;
 
+case OPT_RERUN:
+opt_rerun = true;
+break;
+
 case '?':
-printf("Usage: %s [--compact=0|1] $FILE [$FILE...] | 
[--min-input-size]\n", argv[0]);
+printf("Usage: %s [--compact=0|1] [--rerun] $FILE [$FILE...] | 
[--min-input-size]\n", argv[0]);
 exit(-1);
 break;
 
diff --git a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c 
b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
index b6ebcebc19..7685e976b8 100644
--- a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
+++ b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
@@ -43,7 +43,7 @@ struct fuzz_state
 
 /* Fuzzer's input data. */
 #define DATA_SIZE_FULL offsetof(struct fuzz_state, corpus)
-struct fuzz_corpus *corpus;
+const struct fuzz_corpus *corpus;
 
 /* Real amount of data backing corpus->data[]. */
 size_t data_num;
@@ -53,6 +53,7 @@ struct fuzz_state
 
 /* Emulation ops, some of which are disabled based on corpus->options. */
 struct x86_emulate_ops ops;
+struct x86_emulate_ctxt ctxt;
 };
 
 bool opt_compact = true;
@@ -495,6 +496,12 @@ static int fuzz_read_msr(
 const struct fuzz_state *s = ctxt->data;
 unsigned int idx;
 
+/* 
+ * NB at the moment dump_state() relies on the fact that this
+ * cannot fail.  If we add in fuzzed failures we'll have to handle
+ * that differently.
+ */
+
 switch ( reg )
 {
 case MSR_TSC_AUX:
@@ -615,6 +622,7 @@ static void dump_state(struct x86_emulate_ctxt *ctxt)
 
 printf(" rip: %"PRIx64"\n", regs->rip);
 
+/* read_msr() never fails at the moment */
 fuzz_read_msr(MSR_EFER, , ctxt);
 printf("EFER: %"PRIx64"\n", val);
 }
@@ -659,7 +667,10 @@ static void setup_state(struct x86_emulate_ctxt *ctxt)
 {
 /* Fuzz all of the state in one go */
 if ( !input_read(s, s, DATA_SIZE_FULL) )
+{
+printf("Input size too small\n");
 exit(-1);
+}
 return;
 }
 
@@ -789,9 +800,8 @@ enum {
 printf("Disabling hook "#h"\n");   \
 }
 
-static void disable_hooks(struct x86_emulate_ctxt *ctxt)
+static void disable_hooks(struct fuzz_state *s)
 {
-struct fuzz_state *s = ctxt->data;
 unsigned long bitmap = s->options;
 
 /* See also sanitize_input, some hooks can't be disabled. */
@@ -839,7 +849,7 @@ static void 

[Xen-devel] [PATCH v3 08/12] fuzz/x86_emulate: Move definitions into a header

[George Dunlap]

Move fuzz-emul.c function prototypes into a header.  Also share the
definition of the input size (rather than hard-coding it in
fuzz-emul.c).

Signed-off-by: George Dunlap 
---
RFC: Worth trying to BUILD_BUG_ON(INPUT_SIZE < DATA_SIZE_FULL)?

v3:
- New in this version

CC: Ian Jackson 
CC: Wei Liu 
CC: Andrew Cooper 
CC: Jan Beulich 
---
 tools/fuzz/x86_instruction_emulator/afl-harness.c |  6 +-
 tools/fuzz/x86_instruction_emulator/fuzz-emul.c   |  3 ++-
 tools/fuzz/x86_instruction_emulator/fuzz-emul.h   | 10 ++
 3 files changed, 13 insertions(+), 6 deletions(-)
 create mode 100644 tools/fuzz/x86_instruction_emulator/fuzz-emul.h

diff --git a/tools/fuzz/x86_instruction_emulator/afl-harness.c 
b/tools/fuzz/x86_instruction_emulator/afl-harness.c
index 26b710cb3f..891e56f448 100644
--- a/tools/fuzz/x86_instruction_emulator/afl-harness.c
+++ b/tools/fuzz/x86_instruction_emulator/afl-harness.c
@@ -4,12 +4,8 @@
 #include 
 #include 
 #include 
+#include "fuzz-emul.h"
 
-extern int LLVMFuzzerInitialize(int *argc, char ***argv);
-extern int LLVMFuzzerTestOneInput(const uint8_t *data_p, size_t size);
-extern unsigned int fuzz_minimal_input_size(void);
-
-#define INPUT_SIZE  4096
 static uint8_t input[INPUT_SIZE];
 
 int main(int argc, char **argv)
diff --git a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c 
b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
index 20d52b33f8..9bbe973fd0 100644
--- a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
+++ b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
@@ -16,6 +16,7 @@
 #include 
 
 #include "x86-emulate.h"
+#include "fuzz-emul.h"
 
 #define MSR_INDEX_MAX 16
 
@@ -24,7 +25,7 @@
 /* Layout of data expected as fuzzing input. */
 struct fuzz_corpus
 {
-unsigned char data[4096];
+unsigned char data[INPUT_SIZE];
 } input;
 
 /*
diff --git a/tools/fuzz/x86_instruction_emulator/fuzz-emul.h 
b/tools/fuzz/x86_instruction_emulator/fuzz-emul.h
new file mode 100644
index 00..30dd8de21e
--- /dev/null
+++ b/tools/fuzz/x86_instruction_emulator/fuzz-emul.h
@@ -0,0 +1,10 @@
+#ifndef FUZZ_EMUL_H
+# define FUZZ_EMUL_H
+
+extern int LLVMFuzzerInitialize(int *argc, char ***argv);
+extern int LLVMFuzzerTestOneInput(const uint8_t *data_p, size_t size);
+extern unsigned int fuzz_minimal_input_size(void);
+
+#define INPUT_SIZE  4096
+
+#endif /* ifdef FUZZ_EMUL_H */
-- 
2.14.2


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v3 02/12] fuzz/x86_emulate: Improve failure descriptions in x86_emulate harness

[George Dunlap]

- Print the symbolic name rather than the number
- Explicitly state when data_read() fails due to EOI

Signed-off-by: George Dunlap 
Reviewed-by: Wei Liu 
Reviewed-by: Jan Beulich 
---
Changes in v4:
- Make array 'static const char* const'
Changes in v2:
- Add spaces around '='

CC: Ian Jackson 
CC: Wei Liu 
CC: Andrew Cooper 
CC: Jan Beulich 
---
 tools/fuzz/x86_instruction_emulator/fuzz-emul.c | 11 ++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c 
b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
index 48a879cc88..999f417716 100644
--- a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
+++ b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
@@ -52,6 +52,14 @@ struct fuzz_state
 struct x86_emulate_ops ops;
 };
 
+static const char* const x86emul_return_string[] = {
+[X86EMUL_OKAY] = "X86EMUL_OKAY",
+[X86EMUL_UNHANDLEABLE] = "X86EMUL_UNHANDLEABLE",
+[X86EMUL_EXCEPTION] = "X86EMUL_EXCEPTION",
+[X86EMUL_RETRY] = "X86EMUL_RETRY",
+[X86EMUL_DONE] = "X86EMUL_DONE",
+};
+
 /*
  * Randomly return success or failure when processing data.  If
  * `exception` is false, this function turns _EXCEPTION to _OKAY.
@@ -84,7 +92,7 @@ static int maybe_fail(struct x86_emulate_ctxt *ctxt,
 if ( rc == X86EMUL_EXCEPTION && !exception )
 rc = X86EMUL_OKAY;
 
-printf("maybe_fail %s: %d\n", why, rc);
+printf("maybe_fail %s: %s\n", why, x86emul_return_string[rc]);
 
 if ( rc == X86EMUL_EXCEPTION )
 /* Fake up a pagefault. */
@@ -113,6 +121,7 @@ static int data_read(struct x86_emulate_ctxt *ctxt,
 x86_emul_hw_exception(13, 0, ctxt);
 
 rc = X86EMUL_EXCEPTION;
+printf("data_read %s: X86EMUL_EXCEPTION (end of input)\n", why);
 }
 else
 rc = maybe_fail(ctxt, why, true);
-- 
2.14.2


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v3 04/12] fuzz/x86_emulate: Rename the file containing the wrapper code

[George Dunlap]

When generating coverage output, by default gcov generates output
filenames based only on the coverage file and the "leaf" source file,
not the full path.  As a result, it uses the same name for
x86_emulate.c and x86_emulate/x86_emulate.c, generally overwriting the
second (which we actually are about) with the first (which is just a
wrapper).

Rename the user-space wrapper helpers to x86-emulate.[ch], so
that it generates separate files.

There is actually an option to gcov, `--preserve-paths`, which will
cause the full path name to be included in the filename, properly
distinguishing between the two.  However, given that the user-space
wrapper doesn't actually do any emulation (and the poor state of gcov
documentation making it difficult to find the option in the first
place), it seems to make more sense to rename the file anyway.

Signed-off-by: George Dunlap 
Acked-by: Wei Liu 
---
v3:
- x86-emulate.* rather than x86_emulate_user.*
- Also update .gitignore to ignore the new files

NB: I discovered the `-p` option to gcov after writing this patch.
But I think the patch itself still makes sense.

CC: Ian Jackson 
CC: Wei Liu 
CC: Andrew Cooper 
CC: Jan Beulich 
---
 .gitignore|  3 ++-
 tools/fuzz/x86_instruction_emulator/Makefile  | 12 ++--
 tools/fuzz/x86_instruction_emulator/fuzz-emul.c   |  2 +-
 tools/tests/x86_emulator/Makefile |  6 +++---
 tools/tests/x86_emulator/test_x86_emulator.c  |  2 +-
 tools/tests/x86_emulator/{x86_emulate.c => x86-emulate.c} |  2 +-
 tools/tests/x86_emulator/{x86_emulate.h => x86-emulate.h} |  0
 7 files changed, 14 insertions(+), 13 deletions(-)
 rename tools/tests/x86_emulator/{x86_emulate.c => x86-emulate.c} (99%)
 rename tools/tests/x86_emulator/{x86_emulate.h => x86-emulate.h} (100%)

diff --git a/.gitignore b/.gitignore
index f36ddd284d..ef27553a2d 100644
--- a/.gitignore
+++ b/.gitignore
@@ -160,7 +160,8 @@ tools/flask/utils/flask-set-bool
 tools/flask/utils/flask-label-pci
 tools/fuzz/libelf/afl-libelf-fuzzer
 tools/fuzz/x86_instruction_emulator/asm
-tools/fuzz/x86_instruction_emulator/x86_emulate*
+tools/fuzz/x86_instruction_emulator/x86_emulate
+tools/fuzz/x86_instruction_emulator/x86-emulate.[ch]
 tools/fuzz/x86_instruction_emulator/afl-harness
 tools/helpers/_paths.h
 tools/helpers/init-xenstore-domain
diff --git a/tools/fuzz/x86_instruction_emulator/Makefile 
b/tools/fuzz/x86_instruction_emulator/Makefile
index a3f6b2c754..107bf62a21 100644
--- a/tools/fuzz/x86_instruction_emulator/Makefile
+++ b/tools/fuzz/x86_instruction_emulator/Makefile
@@ -18,22 +18,22 @@ asm:
 
 asm/%: asm ;
 
-x86_emulate.c x86_emulate.h: %:
+x86-emulate.c x86-emulate.h: %:
[ -L $* ] || ln -sf $(XEN_ROOT)/tools/tests/x86_emulator/$*
 
 CFLAGS += $(CFLAGS_xeninclude) -D__XEN_TOOLS__ -I.
 
 x86.h := asm/x86-vendors.h asm/x86-defns.h asm/msr-index.h
-x86_emulate.h := x86_emulate.h x86_emulate/x86_emulate.h $(x86.h)
+x86_emulate.h := x86-emulate.h x86_emulate/x86_emulate.h $(x86.h)
 
-x86_emulate.o: x86_emulate.c x86_emulate/x86_emulate.c $(x86_emulate.h)
+x86-emulate.o: x86-emulate.c x86_emulate/x86_emulate.c $(x86_emulate.h)
 
 fuzz-emul.o: $(x86_emulate.h)
 
-x86-insn-fuzzer.a: fuzz-emul.o x86_emulate.o
+x86-insn-fuzzer.a: fuzz-emul.o x86-emulate.o
$(AR) rc $@ $^
 
-afl-harness: afl-harness.o fuzz-emul.o x86_emulate.o
+afl-harness: afl-harness.o fuzz-emul.o x86-emulate.o
$(CC) $(CFLAGS) $^ -o $@
 
 # Common targets
@@ -42,7 +42,7 @@ all: x86-insn-fuzz-all
 
 .PHONY: distclean
 distclean: clean
-   rm -f x86_emulate x86_emulate.c x86_emulate.h asm
+   rm -f x86_emulate x86-emulate.c x86-emulate.h asm
 
 .PHONY: clean
 clean:
diff --git a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c 
b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
index 5fb8586955..8998f21fe1 100644
--- a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
+++ b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
@@ -15,7 +15,7 @@
 #include 
 #include 
 
-#include "x86_emulate.h"
+#include "x86-emulate.h"
 
 #define MSR_INDEX_MAX 16
 
diff --git a/tools/tests/x86_emulator/Makefile 
b/tools/tests/x86_emulator/Makefile
index d7be77d98a..ed0fd9710e 100644
--- a/tools/tests/x86_emulator/Makefile
+++ b/tools/tests/x86_emulator/Makefile
@@ -77,7 +77,7 @@ $(addsuffix .h,$(TESTCASES)): %.h: %.c testcase.mk Makefile
 $(addsuffix .c,$(SIMD)) $(addsuffix -avx.c,$(filter sse%,$(SIMD))):
ln -sf simd.c $@
 
-$(TARGET): x86_emulate.o test_x86_emulator.o
+$(TARGET): x86-emulate.o test_x86_emulator.o
$(HOSTCC) $(HOSTCFLAGS) -o $@ $^
 
 .PHONY: clean
@@ -105,9 +105,9 @@ $(call cc-option-add,HOSTCFLAGS-x86_64,HOSTCC,-no-pie)
 HOSTCFLAGS += $(CFLAGS_xeninclude) -I. $(HOSTCFLAGS-$(XEN_COMPILE_ARCH))
 
 x86.h := asm/x86-vendors.h asm/x86-defns.h 

[Xen-devel] [PATCH v3 05/12] fuzz/x86_emulate: Add 'afl-cov' target

[George Dunlap]

...to generate a "normal" coverage-instrumented binary, suitable for
use with gcov or afl-cov.

This is slightly annoying because:

 - Every object file needs to have been instrumented to work
   effectively

 - You generally want to have both an afl-instrumented binary and a
   gcov-instrumented binary at the same time, but

 - gcov instrumentation and afl instrumentation are mutually exclusive

So when making the `afl-cov` target, generate a second set of object
files and a second binary with the `-cov` suffix.

While we're here, remove the redundant x86-emulate.c dependency for
x86-emulate.o.

Signed-off-by: George Dunlap 
---
v3:
- Rebase on new versions of previous patch (mainly x86-emulate.* rename)
- Tighten up build rules
- Add newline at the end of README.afl
- Use := for GCOV_FLAGS in Makefile
Changes in v2:
- Pull 'inputs' to x86_emulate_user* into a make variable to avoid duplication

CC: Ian Jackson 
CC: Wei Liu 
CC: Andrew Cooper 
CC: Jan Beulich 
---
 .gitignore   |  1 +
 tools/fuzz/README.afl| 14 ++
 tools/fuzz/x86_instruction_emulator/Makefile | 17 ++---
 3 files changed, 29 insertions(+), 3 deletions(-)

diff --git a/.gitignore b/.gitignore
index ef27553a2d..0514842dae 100644
--- a/.gitignore
+++ b/.gitignore
@@ -163,6 +163,7 @@ tools/fuzz/x86_instruction_emulator/asm
 tools/fuzz/x86_instruction_emulator/x86_emulate
 tools/fuzz/x86_instruction_emulator/x86-emulate.[ch]
 tools/fuzz/x86_instruction_emulator/afl-harness
+tools/fuzz/x86_instruction_emulator/afl-harness-cov
 tools/helpers/_paths.h
 tools/helpers/init-xenstore-domain
 tools/helpers/xen-init-dom0
diff --git a/tools/fuzz/README.afl b/tools/fuzz/README.afl
index 4758de2490..8b58b8cdea 100644
--- a/tools/fuzz/README.afl
+++ b/tools/fuzz/README.afl
@@ -41,3 +41,17 @@ Use the x86 instruction emulator fuzzer as an example.
$ $AFLPATH/afl-fuzz -t 1000 -i testcase_dir -o findings_dir -- ./afl-harness
 
 Please see AFL documentation for more information.
+
+# GENERATING COVERAGE INFORMATION
+
+To use afl-cov or gcov, you need a separate binary instrumented to
+generate coverage data.  To do this, use the target `afl-cov`:
+
+$ make afl-cov #produces afl-harness-cov
+
+NOTE: Please also note that the coverage instrumentation hard-codes
+the absolute path for the instrumentation read and write files in the
+binary; so coverage data will always show up in the build directory no
+matter where you run the binary from.
+
+Please see afl-cov and/or gcov documentation for more information.
diff --git a/tools/fuzz/x86_instruction_emulator/Makefile 
b/tools/fuzz/x86_instruction_emulator/Makefile
index 107bf62a21..cb561aec3f 100644
--- a/tools/fuzz/x86_instruction_emulator/Makefile
+++ b/tools/fuzz/x86_instruction_emulator/Makefile
@@ -23,12 +23,17 @@ x86-emulate.c x86-emulate.h: %:
 
 CFLAGS += $(CFLAGS_xeninclude) -D__XEN_TOOLS__ -I.
 
+GCOV_FLAGS := --coverage
+%-cov.o: %.c
+   $(CC) -c $(CFLAGS) $(GCOV_FLAGS) $< -o $@
+
 x86.h := asm/x86-vendors.h asm/x86-defns.h asm/msr-index.h
 x86_emulate.h := x86-emulate.h x86_emulate/x86_emulate.h $(x86.h)
 
-x86-emulate.o: x86-emulate.c x86_emulate/x86_emulate.c $(x86_emulate.h)
+# x86-emulate.c will be implicit for both
+x86-emulate.o x86-emulate-cov.o: x86_emulate/x86_emulate.c $(x86_emulate.h)
 
-fuzz-emul.o: $(x86_emulate.h)
+fuzz-emul.o fuzz-emulate-cov.o: $(x86_emulate.h)
 
 x86-insn-fuzzer.a: fuzz-emul.o x86-emulate.o
$(AR) rc $@ $^
@@ -36,6 +41,9 @@ x86-insn-fuzzer.a: fuzz-emul.o x86-emulate.o
 afl-harness: afl-harness.o fuzz-emul.o x86-emulate.o
$(CC) $(CFLAGS) $^ -o $@
 
+afl-harness-cov: afl-harness-cov.o fuzz-emul-cov.o x86-emulate-cov.o
+   $(CC) $(CFLAGS) $(GCOV_FLAGS) $^ -o $@
+
 # Common targets
 .PHONY: all
 all: x86-insn-fuzz-all
@@ -46,7 +54,7 @@ distclean: clean
 
 .PHONY: clean
 clean:
-   rm -f *.a *.o .*.d afl-harness
+   rm -f *.a *.o .*.d afl-harness afl-harness-cov *.gcda *.gcno *.gcov
 
 .PHONY: install
 install: all
@@ -55,3 +63,6 @@ install: all
 
 .PHONY: afl
 afl: afl-harness
+
+.PHONY: afl-cov
+afl-cov: afl-harness-cov
-- 
2.14.2


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v3 03/12] fuzz/x86_emulate: Implement input_read() and input_avail()

[George Dunlap]

Rather than open-coding the "read" from the input file.

Signed-off-by: George Dunlap 
---
v3:
 - s/input_available/input_avail/;
 - Constify argument to input_avail
 - Fix off-by-one error in input_avail
 - Return false / true rather than 0 / 1 in input_read
v2:
- Use less dread-ful names
- Return bool rather than int

CC: Ian Jackson 
CC: Wei Liu 
CC: Andrew Cooper 
CC: Jan Beulich 
---
 tools/fuzz/x86_instruction_emulator/fuzz-emul.c | 31 ++---
 1 file changed, 22 insertions(+), 9 deletions(-)

diff --git a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c 
b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
index 999f417716..5fb8586955 100644
--- a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
+++ b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
@@ -52,6 +52,22 @@ struct fuzz_state
 struct x86_emulate_ops ops;
 };
 
+static inline bool input_avail(const struct fuzz_state *s, size_t size)
+{
+return s->data_index + size <= s->data_num;
+}
+
+static inline bool input_read(struct fuzz_state *s, void *dst, size_t size)
+{
+if ( !input_avail(s, size) )
+return false;
+
+memcpy(dst, >corpus->data[s->data_index], size);
+s->data_index += size;
+
+return true;
+}
+
 static const char* const x86emul_return_string[] = {
 [X86EMUL_OKAY] = "X86EMUL_OKAY",
 [X86EMUL_UNHANDLEABLE] = "X86EMUL_UNHANDLEABLE",
@@ -68,10 +84,10 @@ static int maybe_fail(struct x86_emulate_ctxt *ctxt,
   const char *why, bool exception)
 {
 struct fuzz_state *s = ctxt->data;
-const struct fuzz_corpus *c = s->corpus;
+unsigned char c;
 int rc;
 
-if ( s->data_index >= s->data_num )
+if ( !input_read(s, , sizeof(c)) )
 rc = X86EMUL_EXCEPTION;
 else
 {
@@ -80,13 +96,12 @@ static int maybe_fail(struct x86_emulate_ctxt *ctxt,
  * 25% unhandlable
  * 25% exception
  */
-if ( c->data[s->data_index] > 0xc0 )
+if ( c > 0xc0 )
 rc = X86EMUL_EXCEPTION;
-else if ( c->data[s->data_index] > 0x80 )
+else if ( c > 0x80 )
 rc = X86EMUL_UNHANDLEABLE;
 else
 rc = X86EMUL_OKAY;
-s->data_index++;
 }
 
 if ( rc == X86EMUL_EXCEPTION && !exception )
@@ -106,11 +121,10 @@ static int data_read(struct x86_emulate_ctxt *ctxt,
  const char *why, void *dst, unsigned int bytes)
 {
 struct fuzz_state *s = ctxt->data;
-const struct fuzz_corpus *c = s->corpus;
 unsigned int i;
 int rc;
 
-if ( s->data_index + bytes > s->data_num )
+if ( !input_avail(s, bytes) )
 {
 /*
  * Fake up a segment limit violation.  System segment limit volations
@@ -128,8 +142,7 @@ static int data_read(struct x86_emulate_ctxt *ctxt,
 
 if ( rc == X86EMUL_OKAY )
 {
-memcpy(dst, >data[s->data_index], bytes);
-s->data_index += bytes;
+input_read(s, dst, bytes);
 
 printf("%s: ", why);
 for ( i = 0; i < bytes; i++ )
-- 
2.14.2


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v3 09/12] fuzz/x86_emulate: Make input more compact

[George Dunlap]

At the moment, AFL reckons that for any given input, 87% of it is
completely irrelevant: that is, it can change it as much as it wants
but have no impact on the result of the test; and yet it can't remove
it.

This is largely because we interpret the blob handed to us as a large
struct, including CR values, MSR values, segment registers, and a full
cpu_user_regs.

Instead, modify our interpretation to have a "set state" stanza at the
front.  Begin by reading a 16-bit value; if it is lower than a certain
threshold, set some state according to what byte it is, and repeat.
Continue until the byte is above a certain threshold.

This allows AFL to compact any given test case much smaller; to the
point where now it reckons there is not a single byte of the test file
which becomes irrelevant.  Testing have shown that this option both
allows AFL to reach coverage much faster, and to have a total coverage
higher than with the old format.

Make this an option (rather than a unilateral change) to enable
side-by-side performance comparison of the old and new formats.

Signed-off-by: George Dunlap 
---
v3:
 - Set default for opt_compact statically in fuzz-emul.c rather than afl-harness
 - Make input size / unconditional input reading more consistent
 - Require only minimum input read, not first instruction byte
 - Use ARRAY_SIZE() for hardcoded values
 - Use `for ( ; ; )` rather than `while(1)`
 - Some style issues
 - Move opt_compact declaration into fuzz-emul.h
v2: Port over previous changes

CC: Ian Jackson 
CC: Wei Liu 
CC: Andrew Cooper 
CC: Jan Beulich 
---
 tools/fuzz/x86_instruction_emulator/afl-harness.c |  9 ++-
 tools/fuzz/x86_instruction_emulator/fuzz-emul.c   | 96 ---
 tools/fuzz/x86_instruction_emulator/fuzz-emul.h   |  2 +
 3 files changed, 96 insertions(+), 11 deletions(-)

diff --git a/tools/fuzz/x86_instruction_emulator/afl-harness.c 
b/tools/fuzz/x86_instruction_emulator/afl-harness.c
index 891e56f448..052239cea4 100644
--- a/tools/fuzz/x86_instruction_emulator/afl-harness.c
+++ b/tools/fuzz/x86_instruction_emulator/afl-harness.c
@@ -4,6 +4,7 @@
 #include 
 #include 
 #include 
+#include 
 #include "fuzz-emul.h"
 
 static uint8_t input[INPUT_SIZE];
@@ -21,9 +22,11 @@ int main(int argc, char **argv)
 {
 enum {
 OPT_MIN_SIZE,
+OPT_COMPACT,
 };
 static const struct option lopts[] = {
 { "min-input-size", no_argument, NULL, OPT_MIN_SIZE },
+{ "compact", required_argument, NULL, OPT_COMPACT },
 { 0, 0, 0, 0 }
 };
 int c = getopt_long_only(argc, argv, "", lopts, NULL);
@@ -38,8 +41,12 @@ int main(int argc, char **argv)
 exit(0);
 break;
 
+case OPT_COMPACT:
+opt_compact = atoi(optarg);
+break;
+
 case '?':
-printf("Usage: %s $FILE [$FILE...] | [--min-input-size]\n", 
argv[0]);
+printf("Usage: %s [--compact=0|1] $FILE [$FILE...] | 
[--min-input-size]\n", argv[0]);
 exit(-1);
 break;
 
diff --git a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c 
b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
index 9bbe973fd0..b6ebcebc19 100644
--- a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
+++ b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
@@ -35,13 +35,14 @@ struct fuzz_corpus
 struct fuzz_state
 {
 unsigned long options;
+#define DATA_SIZE_COMPACT offsetof(struct fuzz_state, cr)
 unsigned long cr[5];
 uint64_t msr[MSR_INDEX_MAX];
 struct segment_register segments[SEG_NUM];
 struct cpu_user_regs regs;
 
 /* Fuzzer's input data. */
-#define DATA_OFFSET offsetof(struct fuzz_state, corpus)
+#define DATA_SIZE_FULL offsetof(struct fuzz_state, corpus)
 struct fuzz_corpus *corpus;
 
 /* Real amount of data backing corpus->data[]. */
@@ -54,6 +55,13 @@ struct fuzz_state
 struct x86_emulate_ops ops;
 };
 
+bool opt_compact = true;
+
+unsigned int fuzz_minimal_input_size(void)
+{
+return opt_compact ? DATA_SIZE_COMPACT : DATA_SIZE_FULL;
+}
+
 static inline bool input_avail(const struct fuzz_state *s, size_t size)
 {
 return s->data_index + size <= s->data_num;
@@ -647,9 +655,82 @@ static void setup_state(struct x86_emulate_ctxt *ctxt)
 {
 struct fuzz_state *s = ctxt->data;
 
-/* Fuzz all of the state in one go */
-if (!input_read(s, s, DATA_OFFSET))
-exit(-1);
+if ( !opt_compact )
+{
+/* Fuzz all of the state in one go */
+if ( !input_read(s, s, DATA_SIZE_FULL) )
+exit(-1);
+return;
+}
+
+/* Modify only select bits of state */
+
+/* Always read 'options' */
+if ( !input_read(s, s, DATA_SIZE_COMPACT) )
+return;
+
+for ( ; ; )
+{
+uint16_t offset;
+
+/* Read 16 bits to decide what bit of state to modify */

[Xen-devel] [PATCH v3 06/12] fuzz/x86_emulate: Take multiple test files for inputs

[George Dunlap]

Finding aggregate coverage for a set of test files means running each
afl-generated test case through the harness.  At the moment, this is
done by re-executing afl-harness-cov with each input file.  When a
large number of test cases have been generated, this can take a
significant amonut of time; a recent test with 30k total files
generated by 4 parallel fuzzers took over 7 minutes.

The vast majority of this time is taken up with 'exec', however.
Since the harness is already designed to loop over multiple inputs for
llvm "persistent mode", just allow it to take a large number of inputs
on the same when *not* running in llvm "persistent mode"..  Then the
command can be efficiently executed like this:

  ls */queue/id* | xargs $path/afl-harness-cov

For the above-mentioned test on 30k files, the time to generate
coverage data was reduced from 7 minutes to under 30 seconds.

Signed-off-by: George Dunlap 
Acked-by: Jan Beulich 
---
v3:
- Combine some variable declarations
- Make sure that count is set only once no matter how it's compiled
v2:
- Make check for batch processing more clear

CC: Ian Jackson 
CC: Wei Liu 
CC: Andrew Cooper 
CC: Jan Beulich 
---
 tools/fuzz/README.afl |  7 +++
 tools/fuzz/x86_instruction_emulator/afl-harness.c | 25 +++
 2 files changed, 24 insertions(+), 8 deletions(-)

diff --git a/tools/fuzz/README.afl b/tools/fuzz/README.afl
index 8b58b8cdea..a59564985a 100644
--- a/tools/fuzz/README.afl
+++ b/tools/fuzz/README.afl
@@ -49,6 +49,13 @@ generate coverage data.  To do this, use the target 
`afl-cov`:
 
 $ make afl-cov #produces afl-harness-cov
 
+In order to speed up the process of checking total coverage,
+`afl-harness-cov` can take several test inputs on its command-line;
+the speed-up effect should be similar to that of using afl-clang-fast.
+You can use xargs to do this most efficiently, like so:
+
+$ ls queue/id* | xargs $path/afl-harness-cov
+
 NOTE: Please also note that the coverage instrumentation hard-codes
 the absolute path for the instrumentation read and write files in the
 binary; so coverage data will always show up in the build directory no
diff --git a/tools/fuzz/x86_instruction_emulator/afl-harness.c 
b/tools/fuzz/x86_instruction_emulator/afl-harness.c
index 57b4542556..26b710cb3f 100644
--- a/tools/fuzz/x86_instruction_emulator/afl-harness.c
+++ b/tools/fuzz/x86_instruction_emulator/afl-harness.c
@@ -16,6 +16,7 @@ int main(int argc, char **argv)
 {
 size_t size;
 FILE *fp = NULL;
+int max, count;
 
 setbuf(stdin, NULL);
 setbuf(stdout, NULL);
@@ -42,8 +43,7 @@ int main(int argc, char **argv)
 break;
 
 case '?':
-usage:
-printf("Usage: %s $FILE | [--min-input-size]\n", argv[0]);
+printf("Usage: %s $FILE [$FILE...] | [--min-input-size]\n", 
argv[0]);
 exit(-1);
 break;
 
@@ -54,10 +54,13 @@ int main(int argc, char **argv)
 }
 }
 
-if ( optind == argc ) /* No positional parameters.  Use stdin. */
+max = argc - optind;
+
+if ( !max ) /* No positional parameters.  Use stdin. */
+{
+max = 1;
 fp = stdin;
-else if ( optind != (argc - 1) )
-goto usage;
+}
 
 if ( LLVMFuzzerInitialize(, ) )
 exit(-1);
@@ -65,12 +68,15 @@ int main(int argc, char **argv)
 #ifdef __AFL_HAVE_MANUAL_CONTROL
 __AFL_INIT();
 
-while ( __AFL_LOOP(1000) )
+for( count = 0; __AFL_LOOP(1000); )
+#else
+for( count = 0; count < max; count++ )
 #endif
 {
 if ( fp != stdin ) /* If not using stdin, open the provided file. */
 {
-fp = fopen(argv[optind], "rb");
+printf("Opening file %s\n", argv[optind]);
+fp = fopen(argv[optind + count], "rb");
 if ( fp == NULL )
 {
 perror("fopen");
@@ -100,7 +106,10 @@ int main(int argc, char **argv)
 if ( !feof(fp) )
 {
 printf("Input too large\n");
-exit(-1);
+/* Don't exit if we're doing batch processing */
+if ( max == 1 )
+exit(-1);
+continue;
 }
 
 if ( fp != stdin )
-- 
2.14.2


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v3 01/12] fuzz/x86_emulate: Clear errors after each iteration

[George Dunlap]

Once feof() returns true for a stream, it will continue to return true
for that stream until clearerr() is called (or the stream is closed
and re-opened).

In llvm-clang-fast-mode, the same file descriptor is used for each
iteration of the loop, meaning that the "Input too large" check was
broken -- feof() would return true even if the fread() hadn't hit the
end of the file.  The result is that AFL generates testcases of
arbitrary size.

Fix this by fseek'ing to the beginning of the file on every iteration;
this resets the EOF marker and other state.

Signed-off-by: George Dunlap 
---
Changes in v3:
- Fix the issue in the official sanctioned way

This is a candidate for backport to 4.9.

CC: Ian Jackson 
CC: Wei Liu 
CC: Andrew Cooper 
CC: Jan Beulich 
---
 tools/fuzz/x86_instruction_emulator/afl-harness.c | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/tools/fuzz/x86_instruction_emulator/afl-harness.c 
b/tools/fuzz/x86_instruction_emulator/afl-harness.c
index b4d15451b5..57b4542556 100644
--- a/tools/fuzz/x86_instruction_emulator/afl-harness.c
+++ b/tools/fuzz/x86_instruction_emulator/afl-harness.c
@@ -77,6 +77,17 @@ int main(int argc, char **argv)
 exit(-1);
 }
 }
+#ifdef __AFL_HAVE_MANUAL_CONTROL
+else
+{
+/* 
+ * This will ensure we're dealing with a clean stream
+ * state after the afl-fuzz process messes with the open
+ * file handle.
+ */
+fseek(fp, 0, SEEK_SET);
+}
+#endif
 
 size = fread(input, 1, INPUT_SIZE, fp);
 
-- 
2.14.2


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v3 11/12] fuzz/x86_emulate: Set and fuzz more CPU state

[George Dunlap]

x86_emulate() operates not only on state passed to it in
cpu_user_regs, but also on state currently found on the cpu: namely,
the FPU and XMM registers.  At the moment, we re-zero (and/or
re-initialize) cpu_user_regs on every invocation, but leave the
cpu-stored state alone.  In "persistent mode", this causes test cases
to behave differently -- sometimes significantly so -- depending on
which test cases have been run beforehand.

Zero out the state before each test run, and then fuzz it based on the
corpus input.

Signed-off-by: George Dunlap 
---
v3:
- Make type 512 bytes rather than 464
- Style changes
- Change argument from 'store' to 'write'
- Add a comment explaining why we always 'save' even for a write
- Sanitize mxcsr with mxcrs_mask when writing instead of zeroing it in 
sanitize_state
- Get rid of redundant mxcsr_mask setting
- Add comments explaining why we're arbitrarily writing 32 bits
v2: Rebase on top of previous changes

CC: Ian Jackson 
CC: Wei Liu 
CC: Andrew Cooper 
CC: Jan Beulich 
---
 tools/fuzz/x86_instruction_emulator/fuzz-emul.c | 82 -
 1 file changed, 81 insertions(+), 1 deletion(-)

diff --git a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c 
b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
index 7685e976b8..79dd36ec30 100644
--- a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
+++ b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
@@ -40,6 +40,8 @@ struct fuzz_state
 uint64_t msr[MSR_INDEX_MAX];
 struct segment_register segments[SEG_NUM];
 struct cpu_user_regs regs;
+char fxsave[512] __attribute__((aligned(16)));
+
 
 /* Fuzzer's input data. */
 #define DATA_SIZE_FULL offsetof(struct fuzz_state, corpus)
@@ -596,6 +598,54 @@ static const struct x86_emulate_ops all_fuzzer_ops = {
 };
 #undef SET
 
+/*
+ * This funciton will read or write fxsave to the fpu.  When writing,
+ * it 'sanitizes' the state: It will mask off the appropriate bits in
+ * the mxcsr, 'restore' the state to the fpu, then 'save' it again so
+ * that the data in fxsave reflects what's actually in the FPU.
+ *
+ * TODO: Extend state beyond just FPU (ymm registers, )
+ */
+static void _set_fpu_state(char *fxsave, bool write)
+{
+if ( cpu_has_fxsr )
+{
+static union __attribute__((__aligned__(16))) {
+char x[512];
+struct {
+uint32_t other[6];
+uint32_t mxcsr;
+uint32_t mxcsr_mask;
+/* ... */
+};
+} *fxs;
+
+fxs = (typeof(fxs)) fxsave;
+
+if ( write )
+{
+char null[512] __attribute__((aligned(16))) = { };
+
+fxs->mxcsr &= mxcsr_mask;
+
+asm volatile( "fxrstor %0" :: "m" (*null) );
+asm volatile( "fxrstor %0" :: "m" (*fxs) );
+}
+
+asm volatile( "fxsave %0" : "=m" (*fxs) );
+}
+}
+
+static void set_fpu_state(char *fxsave)
+{
+_set_fpu_state(fxsave, true);
+}
+
+static void save_fpu_state(char *fxsave)
+{
+_set_fpu_state(fxsave, false);
+}
+
 static void setup_fpu_exception_handler(void)
 {
 /* FIXME - just disable exceptions for now */
@@ -674,7 +724,11 @@ static void setup_state(struct x86_emulate_ctxt *ctxt)
 return;
 }
 
-/* Modify only select bits of state */
+/*
+ * Modify only select bits of state.  In general, try not to fuzz less
+ * than 32 bits at a time; otherwise we're reading 2 bytes in order to 
fuzz only
+ * one byte. 
+ */
 
 /* Always read 'options' */
 if ( !input_read(s, s, DATA_SIZE_COMPACT) )
@@ -737,6 +791,18 @@ static void setup_state(struct x86_emulate_ctxt *ctxt)
 printf("Setting cpu_user_regs offset %x\n", offset);
 continue;
 }
+offset -= sizeof(struct cpu_user_regs);
+
+/* Fuzz fxsave state */
+if ( offset < 128 )
+{
+/* 32-bit size is arbitrary; see comment above */
+if ( !input_read(s, s->fxsave + (offset * 4), 4) )
+return;
+printf("Setting fxsave offset %x\n", offset * 4);
+continue;
+}
+offset -= 128;
 
 /* None of the above -- take that as "start emulating" */
 
@@ -919,6 +985,8 @@ static int runtest(struct fuzz_state *state) {
 
 disable_hooks(state);
 
+set_fpu_state(state->fxsave);
+
 do {
 /* FIXME: Until we actually implement SIGFPE handling properly */
 setup_fpu_exception_handler();
@@ -930,6 +998,8 @@ static int runtest(struct fuzz_state *state) {
 printf("Emulation result: %d\n", rc);
 } while ( rc == X86EMUL_OKAY );
 
+save_fpu_state(state->fxsave);
+
 return 0;
 }
 
@@ -1013,6 +1083,16 @@ static void compare_states(struct fuzz_state state[2])
 if ( memcmp([0].ops, [1].ops, sizeof(state[0].ops)) )
 

[Xen-devel] [PATCH v3 12/12] fuzz/x86_emulate: Add an option to limit the number of instructions executed

[George Dunlap]

AFL considers a testcase to be a useful addition not only if there are
tuples exercised by that testcase which were not exercised otherwise,
but also if the *number* of times an individual tuple is exercised
changes significantly; in particular, if the number of the highest
non-zero bit changes (i.e., if it is run 1, 2-3, 4-7, 8-15, ).

One simple way to increase these stats it to execute the same (or
similar) instructions multiple times: If executing a given instruction
once hits a particular tuple 2 times, executing it twice will hit the
tuple 4 times, four times will hit the tuple 8 times, and so on.  All
of these will look different to AFL, and so it is likely that many of
the "unique test cases" will simply be the same instruction repeated
powers of 2 times until the tuple counts max out (at 128).

It is unlikely that executing a single instruction more than a handful
of times will generate any state we actually care about; but such long
testcases take exponentially longer to fuzz: the fuzzer spends more
time flipping bits looking for meaningful changes, and each execution
takes longer because it is doing more things.  So long paths which add
nothing to the actual code coverage but effectively "distract" the
fuzzer, making it less effective.

Experiments have shown that not allowing infinite number of
instruction retries for the old (non-compact) format does indeed speed
up and increase code coverage.  However, it has also shown that on the
new, more compact format, having no instruction limit causes the highest
throughput in code coverage.

So leave the option in, but have it default to 0 (no limit).

Signed-off-by: George Dunlap 
---
v3:
- Change opt_instruction_limit to unsigned, default to UINT_MAX
- Simplify limit checking (now that the actual variable itself will never be 0)
- Change counter to unsigned
- Update changelog to try to be a bit more clear


CC: Ian Jackson 
CC: Wei Liu 
CC: Andrew Cooper 
CC: Jan Beulich 
---
 tools/fuzz/x86_instruction_emulator/afl-harness.c | 11 ++-
 tools/fuzz/x86_instruction_emulator/fuzz-emul.c   |  5 -
 tools/fuzz/x86_instruction_emulator/fuzz-emul.h   |  1 +
 3 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/tools/fuzz/x86_instruction_emulator/afl-harness.c 
b/tools/fuzz/x86_instruction_emulator/afl-harness.c
index 4a55ac3c3f..7d09cc29c6 100644
--- a/tools/fuzz/x86_instruction_emulator/afl-harness.c
+++ b/tools/fuzz/x86_instruction_emulator/afl-harness.c
@@ -1,4 +1,5 @@
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -24,11 +25,13 @@ int main(int argc, char **argv)
 OPT_MIN_SIZE,
 OPT_COMPACT,
 OPT_RERUN,
+OPT_INSTRUCTION_LIMIT,
 };
 static const struct option lopts[] = {
 { "min-input-size", no_argument, NULL, OPT_MIN_SIZE },
 { "compact", required_argument, NULL, OPT_COMPACT },
 { "rerun", no_argument, NULL, OPT_RERUN },
+{ "instruction-limit", required_argument, NULL, 
OPT_INSTRUCTION_LIMIT },
 { 0, 0, 0, 0 }
 };
 int c = getopt_long_only(argc, argv, "", lopts, NULL);
@@ -51,8 +54,14 @@ int main(int argc, char **argv)
 opt_rerun = true;
 break;
 
+case OPT_INSTRUCTION_LIMIT:
+opt_instruction_limit = atoi(optarg);
+if ( !opt_instruction_limit )
+opt_instruction_limit = UINT_MAX;
+break;
+
 case '?':
-printf("Usage: %s [--compact=0|1] [--rerun] $FILE [$FILE...] | 
[--min-input-size]\n", argv[0]);
+printf("Usage: %s [--compact=0|1] [--rerun] 
[--instruction-limit=N] $FILE [$FILE...] | [--min-input-size]\n", argv[0]);
 exit(-1);
 break;
 
diff --git a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c 
b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
index 79dd36ec30..8aaec93973 100644
--- a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
+++ b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c
@@ -969,10 +969,13 @@ static void setup_fuzz_state(struct fuzz_state *state, 
const void *data_p, size_
 state->data_num = size;
 }
 
+unsigned int opt_instruction_limit = UINT_MAX;
+
 static int runtest(struct fuzz_state *state) {
 int rc;
 
 struct x86_emulate_ctxt *ctxt = >ctxt;
+unsigned int icount = 0;
 
 state->ops = all_fuzzer_ops;
 
@@ -996,7 +999,7 @@ static int runtest(struct fuzz_state *state) {
 
 rc = x86_emulate(ctxt, >ops);
 printf("Emulation result: %d\n", rc);
-} while ( rc == X86EMUL_OKAY );
+} while ( rc == X86EMUL_OKAY && ++icount < opt_instruction_limit );
 
 save_fpu_state(state->fxsave);
 
diff --git a/tools/fuzz/x86_instruction_emulator/fuzz-emul.h 
b/tools/fuzz/x86_instruction_emulator/fuzz-emul.h
index 4863bf2166..746e1b542d 100644
--- 

Re: [Xen-devel] [PATCH v2 12/13] fuzz/x86_emulate: Set and fuzz more CPU state

[George Dunlap]

On 10/06/2017 12:56 PM, Jan Beulich wrote:
 On 25.09.17 at 16:26,  wrote:
>> @@ -597,6 +599,47 @@ static const struct x86_emulate_ops all_fuzzer_ops = {
>>  };
>>  #undef SET
>>  
>> +static void _set_fpu_state(char *fxsave, bool store)
>> +{
>> +if ( cpu_has_fxsr )
>> +{
>> +static union __attribute__((__aligned__(16))) {
>> +char x[464];
>> +struct {
>> +uint32_t other[6];
>> +uint32_t mxcsr;
>> +uint32_t mxcsr_mask;
>> +/* ... */
>> +};
>> +} *fxs;
>> +
>> +fxs = (typeof(fxs)) fxsave;
>> +
>> +if ( store ) {
>> +char null[512] __attribute__((aligned(16))) = { 0 };
>> +asm volatile(" fxrstor %0; "::"m"(*null));
>> +asm volatile(" fxrstor %0; "::"m"(*fxsave));
>> +}
>> +
>> +asm volatile( "fxsave %0" : "=m" (*fxs) );
>> +
>> +if ( fxs->mxcsr_mask )
>> +mxcsr_mask = fxs->mxcsr_mask;
>> +else
>> +mxcsr_mask = 0x000ffbf;
> 
> Actually - why is this necessary? I.e. why isn't emul_test_init()
> setting mxcsr_mask sufficient?

This is me not understanding what's going on.  I've removed this bit,
and modified this function to do the 'sanitation' -- to mask off mxcsr
before doing the fxrstor (and removed the change from "sanitize_input").

 -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v4 00/10] xen/arm: Memory subsystem clean-up

[Stefano Stabellini]

On Mon, 9 Oct 2017, Julien Grall wrote:
> Hi all,
> 
> This patch series contains clean-up for the ARM memory subsystem in 
> preparation
> of reworking the page tables handling.
> 
> For all changes, see in each patch.

The series is committed, thanks!


> Cheers,
> 
> Julien Grall (10):
>   xen/arm: page: Use ARMv8 naming to improve readability
>   xen/arm: page: Clean-up the definition of MAIRVAL
>   xen/arm: mm: Rename and clarify AP[1] in the stage-1 page table
>   xen/arm: Switch to SYS_STATE_boot just after end_boot_allocator()
>   xen/arm: mm: Rename 'ai' into 'flags' in create_xen_entries
>   xen/arm: mm: Use PAGE_HYPERVISOR_* instead of MT_* when calling
> set_fixmap
>   xen/arm: page: Describe the layout of flags used to update page tables
>   xen/arm: mm: Embed permission in the flags
>   xen/arm: mm: Handle permission flags when adding a new mapping
>   xen/arm: mm: Use memory flags for modify_xen_mappings rather than
> custom one
> 
>  xen/arch/arm/kernel.c |  2 +-
>  xen/arch/arm/livepatch.c  |  6 +--
>  xen/arch/arm/mm.c | 54 
>  xen/arch/arm/platforms/vexpress.c |  3 +-
>  xen/arch/arm/setup.c  |  8 +++-
>  xen/drivers/video/arm_hdlcd.c |  2 +-
>  xen/include/asm-arm/lpae.h|  2 +-
>  xen/include/asm-arm/page.h| 88 
> +--
>  8 files changed, 99 insertions(+), 66 deletions(-)
> 
> -- 
> 2.11.0
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 0/*] xen: xen-domid-restrict improvements

[Ross Lagerwall]

On 10/06/2017 02:19 PM, Paul Durrant wrote:

-Original Message-
From: Xen-devel [mailto:xen-devel-boun...@lists.xen.org] On Behalf Of
Ross Lagerwall
Sent: 06 October 2017 13:58
To: Ian Jackson ; qemu-de...@nongnu.org
Cc: Anthony Perard ; xen-
de...@lists.xenproject.org; Stefano Stabellini ;
Juergen Gross 
Subject: Re: [Xen-devel] [PATCH v2 0/*] xen: xen-domid-restrict
improvements

On 10/04/2017 05:18 PM, Ian Jackson wrote:

(Resending this because 1. I got the CC for xen-devel wrong; 2. I got
the subject wrong: there are actually 8 patches; 3. I mangled
Anthony's name in theheaders.  Sorry for the noise.)

I have been working on trying to get qemu, when running as a Xen
device model, to _actually_ not have power equivalent to root.

I think I have achieved this, with some limitations (which will be
discussed in my series against xen.git.

However, there are changes to qemu needed.  In particular

   * The -xen-domid-restrict option does not work properly right now.
 It only restricts a small subset of the descriptors qemu has open.
 I am introducing a new library call in the Xen libraries for this,
 xentoolcore_restrict_all.



Hi Ian,

I'm testing your QEMU and Xen patch series and found that after being
restricted, QEMU fails to setup up the VGA memory properly which causes
a complete stall with stdvga. With cirrus it mostly works although it
seems to have reduced performance.

I think it happens when the VM sets up the BAR some time after
xen_restrict() has been called. The failure comes from QEMU calling
xc_domain_add_to_physmap() which calls do_memory_op() and finally
xencall2(). But the underlying xencall fd has been replaced with /dev/null.


Oh, that's a PITA. This is because of the slightly hacky way that the VRAM is 
dealt with (it needing to be moved into the BAR of the PCI VGA device at the 
point where guest firmware programs it). Maybe we need a new dm_op for this?

If no one objects, I propose adding the following calls to 
libxendevicemodel (with underlying Xen implementations, of course) that 
would be usable after the xendevicemodel handle has been restricted.


xendevicemodel_add_to_physmap(xendevicemodel_handle *dmod,
  domid_t domid,
  unsigned long idx,
  xen_pfn_t gpfn);

This is equivalent to xc_domain_add_to_physmap() with
space==XENMAPSPACE_gmfn.

xendevicemodel_pin_memory_cacheattr(xendevicemodel_handle *dmod,
domid_t domid,
uint64_t start,
uint64_t end,
uint32_t type);

This is equivalent to xc_domain_pin_memory_cacheattr().

--
Ross Lagerwall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v4 06/10] xen/arm: mm: Use PAGE_HYPERVISOR_* instead of MT_* when calling set_fixmap

[Stefano Stabellini]

On Mon, 9 Oct 2017, Julien Grall wrote:
> At the moment, PAGE_HYPERVISOR_* and MT_* have exactly the same value.
> In a follow-up patch the former will be extended to carry more
> information.
> 
> It looks like the caller of set_fixmap are mixing the both. Stay
> consistent and only use PAGE_HYPERVISOR_*. This is also match the
> behavior of create_xen_entries and would potentially allow to share some
> part in the future.
> 
> Also rename the parameter 'attributes' to 'flags' so it is clearer what
> is the interface.
> 
> Signed-off-by: Julien Grall 

Acked-by: Stefano Stabellini 


> ---
> Changes in v4:
> - Patch added.
> ---
>  xen/arch/arm/kernel.c | 2 +-
>  xen/arch/arm/mm.c | 4 ++--
>  xen/arch/arm/platforms/vexpress.c | 3 ++-
>  xen/drivers/video/arm_hdlcd.c | 2 +-
>  4 files changed, 6 insertions(+), 5 deletions(-)
> 
> diff --git a/xen/arch/arm/kernel.c b/xen/arch/arm/kernel.c
> index a12baa86e7..c2755a9ab9 100644
> --- a/xen/arch/arm/kernel.c
> +++ b/xen/arch/arm/kernel.c
> @@ -54,7 +54,7 @@ void copy_from_paddr(void *dst, paddr_t paddr, unsigned 
> long len)
>  s = paddr & (PAGE_SIZE-1);
>  l = min(PAGE_SIZE - s, len);
>  
> -set_fixmap(FIXMAP_MISC, maddr_to_mfn(paddr), MT_NORMAL_NC);
> +set_fixmap(FIXMAP_MISC, maddr_to_mfn(paddr), PAGE_HYPERVISOR_WC);
>  memcpy(dst, src + s, l);
>  clean_dcache_va_range(dst, l);
>  
> diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c
> index 39bade63f5..70a03015ec 100644
> --- a/xen/arch/arm/mm.c
> +++ b/xen/arch/arm/mm.c
> @@ -329,9 +329,9 @@ static inline lpae_t mfn_to_xen_entry(mfn_t mfn, unsigned 
> attr)
>  }
>  
>  /* Map a 4k page in a fixmap entry */
> -void set_fixmap(unsigned map, mfn_t mfn, unsigned attributes)
> +void set_fixmap(unsigned map, mfn_t mfn, unsigned int flags)
>  {
> -lpae_t pte = mfn_to_xen_entry(mfn, attributes);
> +lpae_t pte = mfn_to_xen_entry(mfn, flags);
>  pte.pt.table = 1; /* 4k mappings always have this bit set */
>  pte.pt.xn = 1;
>  write_pte(xen_fixmap + third_table_offset(FIXMAP_ADDR(map)), pte);
> diff --git a/xen/arch/arm/platforms/vexpress.c 
> b/xen/arch/arm/platforms/vexpress.c
> index df2c4b5bec..39b6bcc70e 100644
> --- a/xen/arch/arm/platforms/vexpress.c
> +++ b/xen/arch/arm/platforms/vexpress.c
> @@ -65,7 +65,8 @@ int vexpress_syscfg(int write, int function, int device, 
> uint32_t *data)
>  uint32_t *syscfg = (uint32_t *) FIXMAP_ADDR(FIXMAP_MISC);
>  int ret = -1;
>  
> -set_fixmap(FIXMAP_MISC, maddr_to_mfn(V2M_SYS_MMIO_BASE), 
> MT_DEVICE_nGnRE);
> +set_fixmap(FIXMAP_MISC, maddr_to_mfn(V2M_SYS_MMIO_BASE),
> +   PAGE_HYPERVISOR_NOCACHE);
>  
>  if ( syscfg[V2M_SYS_CFGCTRL/4] & V2M_SYS_CFG_START )
>  goto out;
> diff --git a/xen/drivers/video/arm_hdlcd.c b/xen/drivers/video/arm_hdlcd.c
> index 1175399dbc..e1174b223f 100644
> --- a/xen/drivers/video/arm_hdlcd.c
> +++ b/xen/drivers/video/arm_hdlcd.c
> @@ -227,7 +227,7 @@ void __init video_init(void)
>  /* uses FIXMAP_MISC */
>  set_pixclock(videomode->pixclock);
>  
> -set_fixmap(FIXMAP_MISC, maddr_to_mfn(hdlcd_start), MT_DEVICE_nGnRE);
> +set_fixmap(FIXMAP_MISC, maddr_to_mfn(hdlcd_start), 
> PAGE_HYPERVISOR_NOCACHE);
>  HDLCD[HDLCD_COMMAND] = 0;
>  
>  HDLCD[HDLCD_LINELENGTH] = videomode->xres * bytes_per_pixel;
> -- 
> 2.11.0
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v4 09/10] xen/arm: mm: Handle permission flags when adding a new mapping

[Stefano Stabellini]

On Mon, 9 Oct 2017, Julien Grall wrote:
> Currently, all the new mappings will be read-write non-executable. Allow the
> caller to use other permissions.
> 
> Signed-off-by: Julien Grall 

Reviewed-by: Stefano Stabellini 

> ---
> Changes in v2:
> - Switch the runtime check to a BUG_ON()
> ---
>  xen/arch/arm/mm.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c
> index c1dad61a20..2329ccee83 100644
> --- a/xen/arch/arm/mm.c
> +++ b/xen/arch/arm/mm.c
> @@ -1022,6 +1022,9 @@ static int create_xen_entries(enum xenmap_operation op,
>  if ( op == RESERVE )
>  break;
>  pte = mfn_to_xen_entry(mfn, PAGE_AI_MASK(flags));
> +pte.pt.ro = PAGE_RO_MASK(flags);
> +pte.pt.xn = PAGE_XN_MASK(flags);
> +BUG_ON(!pte.pt.ro && !pte.pt.xn);
>  pte.pt.table = 1;
>  write_pte(entry, pte);
>  break;
> -- 
> 2.11.0
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v4 08/10] xen/arm: mm: Embed permission in the flags

[Stefano Stabellini]

On Mon, 9 Oct 2017, Julien Grall wrote:
> Currently, it is not possible to specify the permission of a new
> mapping. It would be necessary to use the function modify_xen_mappings
> with a different set of flags.
> 
> Introduce a couple of new flags for the permissions (Non-eXecutable,
> Read-Only) and also provides definition that combine the memory attribute
> and permission for common combinations.
> 
> PAGE_HYPERVISOR is now an alias to PAGE_HYPERVISOR_RW (read-write,
> non-executable mappings). This does not affect the current mapping using
> PAGE_HYPERVISOR because Xen is currently forcing all the mapping to be
> non-executable by default (see mfn_to_xen_entry).
> 
> A follow-up patch will change modify_xen_mappings to use the new flags.
> 
> Signed-off-by: Julien Grall 
> 
> ---
> 
> Changes in v3:
> - Add a comment about _PAGE_DEVICE and _PAGE_NORMAL
> 
> Changes in v2:
> - Update the commit message
> ---
>  xen/include/asm-arm/page.h | 25 ++---
>  1 file changed, 22 insertions(+), 3 deletions(-)
> 
> diff --git a/xen/include/asm-arm/page.h b/xen/include/asm-arm/page.h
> index aa3e83f5b4..e2b3e402d0 100644
> --- a/xen/include/asm-arm/page.h
> +++ b/xen/include/asm-arm/page.h
> @@ -69,12 +69,31 @@
>   * Layout of the flags used for updating the hypervisor page tables
>   *
>   * [0:2] Memory Attribute Index
> + * [3:4] Permission flags
>   */
>  #define PAGE_AI_MASK(x) ((x) & 0x7U)
>  
> -#define PAGE_HYPERVISOR (MT_NORMAL)
> -#define PAGE_HYPERVISOR_NOCACHE (MT_DEVICE_nGnRE)
> -#define PAGE_HYPERVISOR_WC  (MT_NORMAL_NC)
> +#define _PAGE_XN_BIT3
> +#define _PAGE_RO_BIT4
> +#define _PAGE_XN(1U << _PAGE_XN_BIT)
> +#define _PAGE_RO(1U << _PAGE_RO_BIT)
> +#define PAGE_XN_MASK(x) (((x) >> _PAGE_XN_BIT) & 0x1U)
> +#define PAGE_RO_MASK(x) (((x) >> _PAGE_RO_BIT) & 0x1U)
> +
> +/*
> + * _PAGE_DEVICE and _PAGE_NORMAL are conveniences defines. They are not
> + * meant to be used outside of the headers.

just grammar NITs:
  _PAGE_DEVICE and _PAGE_NORMAL are convenience defines. They are not
  meant to be used outside of this header.
I'll fix on commit

Reviewed-by: Stefano Stabellini 

> + */
> +#define _PAGE_DEVICE_PAGE_XN
> +#define _PAGE_NORMALMT_NORMAL
> +
> +#define PAGE_HYPERVISOR_RO  (_PAGE_NORMAL|_PAGE_RO|_PAGE_XN)
> +#define PAGE_HYPERVISOR_RX  (_PAGE_NORMAL|_PAGE_RO)
> +#define PAGE_HYPERVISOR_RW  (_PAGE_NORMAL|_PAGE_XN)
> +
> +#define PAGE_HYPERVISOR PAGE_HYPERVISOR_RW
> +#define PAGE_HYPERVISOR_NOCACHE (_PAGE_DEVICE|MT_DEVICE_nGnRE)
> +#define PAGE_HYPERVISOR_WC  (_PAGE_DEVICE|MT_NORMAL_NC)
>  
>  /*
>   * Defines for changing the hypervisor PTE .ro and .nx bits. This is only to 
> be
> -- 
> 2.11.0
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v4 01/10] xen/arm: page: Use ARMv8 naming to improve readability

[Stefano Stabellini]

On Mon, 9 Oct 2017, Julien Grall wrote:
> This is based on the Linux ARMv8 naming scheme (see arch/arm64/mm/proc.S). 
> Each
> type will contain "NORMAL" or "DEVICE" to make clear whether each attribute
> targets device or normal memory.
> 
> Signed-off-by: Julien Grall 

Reviewed-by: Stefano Stabellini 


> ---
> 
> Changes in v3:
> - The ai '000' is named MT_DEVICE_nGnRnE and not
> MT_DEVICE_nGnRE. The definition is still valid.
> - Expand the comment to point to "Device Memory" section in the
> ARM ARM.
> 
> Changes in v2:
> * Move the patch before "xen/arm: page: Clean-up the definition
> of MAIRVAL"
> ---
>  xen/arch/arm/kernel.c |  2 +-
>  xen/arch/arm/mm.c | 28 ++--
>  xen/arch/arm/platforms/vexpress.c |  2 +-
>  xen/drivers/video/arm_hdlcd.c |  2 +-
>  xen/include/asm-arm/page.h| 35 +++
>  5 files changed, 36 insertions(+), 33 deletions(-)
> 
> diff --git a/xen/arch/arm/kernel.c b/xen/arch/arm/kernel.c
> index 9c183f96da..a12baa86e7 100644
> --- a/xen/arch/arm/kernel.c
> +++ b/xen/arch/arm/kernel.c
> @@ -54,7 +54,7 @@ void copy_from_paddr(void *dst, paddr_t paddr, unsigned 
> long len)
>  s = paddr & (PAGE_SIZE-1);
>  l = min(PAGE_SIZE - s, len);
>  
> -set_fixmap(FIXMAP_MISC, maddr_to_mfn(paddr), MT_BUFFERABLE);
> +set_fixmap(FIXMAP_MISC, maddr_to_mfn(paddr), MT_NORMAL_NC);
>  memcpy(dst, src + s, l);
>  clean_dcache_va_range(dst, l);
>  
> diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c
> index 9a37f29ce6..f41c6ce6f1 100644
> --- a/xen/arch/arm/mm.c
> +++ b/xen/arch/arm/mm.c
> @@ -290,7 +290,7 @@ static inline lpae_t mfn_to_xen_entry(mfn_t mfn, unsigned 
> attr)
>  
>  switch ( attr )
>  {
> -case MT_BUFFERABLE:
> +case MT_NORMAL_NC:
>  /*
>   * ARM ARM: Overlaying the shareability attribute (DDI
>   * 0406C.b B3-1376 to 1377)
> @@ -305,8 +305,8 @@ static inline lpae_t mfn_to_xen_entry(mfn_t mfn, unsigned 
> attr)
>   */
>  e.pt.sh = LPAE_SH_OUTER;
>  break;
> -case MT_UNCACHED:
> -case MT_DEV_SHARED:
> +case MT_DEVICE_nGnRnE:
> +case MT_DEVICE_nGnRE:
>  /*
>   * Shareability is ignored for non-Normal memory, Outer is as
>   * good as anything.
> @@ -369,7 +369,7 @@ static void __init create_mappings(lpae_t *second,
>  
>  count = nr_mfns / LPAE_ENTRIES;
>  p = second + second_linear_offset(virt_offset);
> -pte = mfn_to_xen_entry(_mfn(base_mfn), MT_WRITEALLOC);
> +pte = mfn_to_xen_entry(_mfn(base_mfn), MT_NORMAL);
>  if ( granularity == 16 * LPAE_ENTRIES )
>  pte.pt.contig = 1;  /* These maps are in 16-entry contiguous chunks. 
> */
>  for ( i = 0; i < count; i++ )
> @@ -422,7 +422,7 @@ void *map_domain_page(mfn_t mfn)
>  else if ( map[slot].pt.avail == 0 )
>  {
>  /* Commandeer this 2MB slot */
> -pte = mfn_to_xen_entry(_mfn(slot_mfn), MT_WRITEALLOC);
> +pte = mfn_to_xen_entry(_mfn(slot_mfn), MT_NORMAL);
>  pte.pt.avail = 1;
>  write_pte(map + slot, pte);
>  break;
> @@ -543,7 +543,7 @@ static inline lpae_t pte_of_xenaddr(vaddr_t va)
>  {
>  paddr_t ma = va + phys_offset;
>  
> -return mfn_to_xen_entry(maddr_to_mfn(ma), MT_WRITEALLOC);
> +return mfn_to_xen_entry(maddr_to_mfn(ma), MT_NORMAL);
>  }
>  
>  /* Map the FDT in the early boot page table */
> @@ -652,7 +652,7 @@ void __init setup_pagetables(unsigned long 
> boot_phys_offset, paddr_t xen_paddr)
>  /* Initialise xen second level entries ... */
>  /* ... Xen's text etc */
>  
> -pte = mfn_to_xen_entry(maddr_to_mfn(xen_paddr), MT_WRITEALLOC);
> +pte = mfn_to_xen_entry(maddr_to_mfn(xen_paddr), MT_NORMAL);
>  pte.pt.xn = 0;/* Contains our text mapping! */
>  xen_second[second_table_offset(XEN_VIRT_START)] = pte;
>  
> @@ -669,7 +669,7 @@ void __init setup_pagetables(unsigned long 
> boot_phys_offset, paddr_t xen_paddr)
>  
>  /* ... Boot Misc area for xen relocation */
>  dest_va = BOOT_RELOC_VIRT_START;
> -pte = mfn_to_xen_entry(maddr_to_mfn(xen_paddr), MT_WRITEALLOC);
> +pte = mfn_to_xen_entry(maddr_to_mfn(xen_paddr), MT_NORMAL);
>  /* Map the destination in xen_second. */
>  xen_second[second_table_offset(dest_va)] = pte;
>  /* Map the destination in boot_second. */
> @@ -700,7 +700,7 @@ void __init setup_pagetables(unsigned long 
> boot_phys_offset, paddr_t xen_paddr)
>  unsigned long va = XEN_VIRT_START + (i << PAGE_SHIFT);
>  if ( !is_kernel(va) )
>  break;
> -pte = mfn_to_xen_entry(mfn, MT_WRITEALLOC);
> +pte = mfn_to_xen_entry(mfn, MT_NORMAL);
>  pte.pt.table = 1; /* 4k mappings always have this bit set */
>  if ( is_kernel_text(va) || 

Re: [Xen-devel] [PATCH v2 01/17] x86emul: support remaining AVX insns

[George Dunlap]

On 10/09/2017 05:12 PM, Jan Beulich wrote:
 On 09.10.17 at 17:36,  wrote:
>> On 09/14/2017 04:12 PM, Jan Beulich wrote:
>>> @@ -7119,6 +7142,18 @@ x86_emulate(
>>>  fic.insn_bytes = PFX_BYTES + 3;
>>>  break;
>>>  
>>> +case X86EMUL_OPC_VEX_66(0x0f38, 0x19): /* vbroadcastsd m64,ymm */
>>> +case X86EMUL_OPC_VEX_66(0x0f38, 0x1a): /* vbroadcastf128 m128,ymm */
>>> +generate_exception_if(!vex.l, EXC_UD);
>>> +/* fall through */
>>> +case X86EMUL_OPC_VEX_66(0x0f38, 0x18): /* vbroadcastss m32,{x,y}mm */
>>> +generate_exception_if(ea.type != OP_MEM, EXC_UD);
>>
>> Just checking my understanding here: The reason for this check is that
>> as of this patch, we still only support AVX instructions, not AVX2 (and
>> later) variants, which have non-memory-source variants?
> 
> That's a good point actually. Yes, for this patch alone it's fine to
> handle just memory operands. But the later AVX2 patch doesn't
> generalize this - I've managed to overlook this aspect. Based on
> how other support has been added, it could have been done the
> other way as well (using suitable conditionals), but now that this
> patch has gone in, I'll have to do it in the AVX2 patch.
> 
>> I'm not trying to complain, but I want to reflect back some of what I'm
>> experiencing trying to review this series.  After having gotten somewhat
>> up to speed on the instructions and terminology, and the general layout
>> of the existing code, I understand that the basic method for adding a
>> new instruction of this type is:
>>
>> 1. Add instruction re-execution support
>>   a. Add decode information into the appropriate tables (in this case
>> ext0f38_table[] and ext0f3a_table[]
>>   b. Add case statements which
>>- Check for the appropriate conditions
>>- Set up anything that needs setting up for the instruction
>> re-execution (in the "if (state->simd_size)" clause).
>>
>> 2. Add testing
> 
> Whatever that means. Generally I'm trying to add tests for new
> code paths, or for insns using unusual operand sizes. But I'm not
> going to claim the testing being added is always completely
> covering all new insns.
> 
>> And of course 1b may involve extending functionality, such as adding
>> simd_128 or handling new source / destination modes or combinations.
>>
>> So a proper review would involve making sure that all the above had been
>> done properly: That the right instruction was decoded, the right tables
>> set up, the right conditions checked, the right inputs made to the
>> re-execution code further down; and then of course making sure that
>> these instructions were properly added to the testing matrix.
>>
>> So I look up the instructions named above (vbroadcast*) and verified
>> that they matched the codes listed.  The manual says first two generate
>> an exception if vex.l == 0; so far so good.
>>
>> But what's the ea.type != MEM thing?  The manual certainly says there
>> are instructions that don't reference memory.  A bit of looking and
>> poking around, and I formulated the question above.  The test for AVX
>> support is made in a completely different part of the file, and no
>> mention of AVX2 / whatever is done here.
>>
>> OK, go up and check the tables -- additions to ext0f38 at 0x18, 0x19,
>> and 0x1a seem to match the description.
>>
>> Now what?  How do I verify that everything else on the path will work as
>> it should?
> 
> Well, if we assume that the code paths outside the big switch
> statements work, then it is really only the table addition and the
> customization in the new case label which needs verifying. If otoh
> you suspect that the glue between existing (common) and new
> (per opcode) code isn't right, then going through the common
> parts with the specifics of the new instruction in mind won't be
> avoidable. But isn't that how you'd review other code as well?

Well the question isn't whether I suspect there's something wrong; it's
checking to see if the author haven't missed something.  In order to
know that a particular instruction will DTRT when going through the
common code, I need to have an idea not only what the instruction needs,
but what the common code is doing.

And no, this process isn't different than what a reviewer has to do for
any patch; it's just particularly difficult for this patch.

>> And I still don't have any idea how to start on verifying that these
>> instructions have been added to the test framework properly.
> 
> Is seeing the respective compiler intrinsics being used (or, where
> those aren't suitable, inline asm()-s with the very instructions) not
> enough? Plus, as said above, don't expect every single insn to be
> tested, or even every single flavor of one insn.
> 
>> I'm only 3 instructions in, with at least 20 more to go.
>>
>> There is certainly something to be said for saying that if you're going
>> to review a change to code you have to understand the underlying code
>> itself; and with a piece of 

[Xen-devel] [xen-unstable-smoke test] 114289: tolerable all pass - PUSHED

[osstest service owner]

flight 114289 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/114289/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  a164e14c6e140d792aee644990f8fea0fa8f8da2
baseline version:
 xen  3c1ca29bd53570ffce049a297d18956f5d93ec8a

Last test of basis   114267  2017-10-10 10:38:15 Z0 days
Testing same since   114289  2017-10-10 18:02:42 Z0 days1 attempts


People who touched revisions under test:
  Wei Liu 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=xen-unstable-smoke
+ revision=a164e14c6e140d792aee644990f8fea0fa8f8da2
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
 export PERLLIB=.:.
 PERLLIB=.:.
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push xen-unstable-smoke 
a164e14c6e140d792aee644990f8fea0fa8f8da2
+ branch=xen-unstable-smoke
+ revision=a164e14c6e140d792aee644990f8fea0fa8f8da2
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
 export PERLLIB=.:.:.
 PERLLIB=.:.:.
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
+++ export PERLLIB=.:.:.:.
+++ PERLLIB=.:.:.:.
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=xen
+ xenbranch=xen-unstable-smoke
+ qemuubranch=qemu-upstream-unstable
+ '[' xxen = xlinux ']'
+ linuxbranch=
+ '[' xqemu-upstream-unstable = x ']'
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable-smoke
+ prevxenbranch=xen-4.9-testing
+ '[' xa164e14c6e140d792aee644990f8fea0fa8f8da2 = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/xtf.git
++ : osst...@xenbits.xen.org:/home/xen/git/xtf.git
++ : git://xenbits.xen.org/xtf.git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/rumprun.git
++ : git://git.seabios.org/seabios.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/seabios.git
++ : git://xenbits.xen.org/osstest/seabios.git
++ : https://github.com/tianocore/edk2.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/ovmf.git
++ : 

Re: [Xen-devel] [PATCH v9] x86/hvm: Implement hvmemul_write() using real mappings

[Andrew Cooper]

On 10/10/17 13:01, Alexandru Stefan ISAILA wrote:
>> I'd be fine taking care of all the comments while committing (and
>> then adding my R-b), provided you (and ideally also Andrew)
>> agree, and of course assuming Paul would ack the patch, plus
>> no-one else finds yet another problem which once again I may
>> have overlooked.
>>
> Hi Jan,
>
> Thank you for your help and I agree with the suggested changes.

Ok from me too.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v6 5/5] ARM: ITS: Expose ITS in the MADT table

[Julien Grall]

Hi,

On 10/10/2017 21:07, Stefano Stabellini wrote:

CC'ing Jan and Andrew, just in case they disagree on this.

On Tue, 10 Oct 2017, Julien Grall wrote:

+unsigned long gicv3_its_make_hwdom_madt(const struct domain *d, void
*base_ptr)
+{
+unsigned int i;
+void *fw_its;
+struct acpi_madt_generic_translator *hwdom_its;
+
+hwdom_its = base_ptr;
+
+for ( i = 0; i < vgic_v3_its_count(d); i++ )
+{
+fw_its =
acpi_table_get_entry_madt(ACPI_MADT_TYPE_GENERIC_TRANSLATOR,
+   i);
+memcpy(hwdom_its, fw_its, sizeof(struct
acpi_madt_generic_translator));


I think we are supposed to use ACPI_MEMCPY for this kind of operations.
If that's OK for you, I'll fix on commit.


I don't think we should use ACPI_MEMCPY. The macro is here because acpica (our
drivers/acpi) is meant to be OS-agnostic. So you need a way to tell how your
OS copies memory.

I had a look on the usage of ACPI_MEMCPY, it seems that only the arch/arm and
drivers/acpi is using it. This seem to confirm that probably we used it by
mistake in the Arm code.


It looks like you are right, ACPI_MEMCPY does not make sense in Xen
code outside of drivers/acpi.

Consistency is important, so if we are not going to use ACPI_MEMCPY, then
I'll write a patch (or I'll take a patch if you volunteer in writing it)
to s/ACPI_MEMCPY/memcpy/g everywhere under arch/arm. The worst we could
end up with is a mixed environment.


Feel free to write a patch, but I don't think you should block this 
series for that.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel