[Xen-devel] [xen-unstable-smoke test] 134620: regressions - trouble: blocked/broken/fail

2019-04-10 Thread osstest service owner
flight 134620 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/134620/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken
 build-arm64-xsm   4 host-install(4)broken REGR. vs. 133991
 build-amd64   6 xen-buildfail REGR. vs. 133991
 build-armhf   6 xen-buildfail REGR. vs. 133991

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-debianhvm-i386  1 build-check(1) blocked n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a

version targeted for testing:
 xen  0c07431cf712d990b3c60341b0b435c903bbf4f4
baseline version:
 xen  cb70a26f78848fe45f593f7ebc9cfaac760a791b

Last test of basis   133991  2019-03-22 15:00:46 Z   19 days
Failing since134068  2019-03-25 12:00:51 Z   16 days   74 attempts
Testing same since   134610  2019-04-10 16:00:43 Z0 days3 attempts


People who touched revisions under test:
  Alexandru Isaila 
  Alexandru Stefan ISAILA 
  Amit Singh Tomar 
  Andrew Cooper 
  Anthony PERARD 
  Brian Woods 
  Christian Lindig 
  Doug Goldstein 
  George Dunlap 
  Igor Druzhinin 
  Jan Beulich 
  Juergen Gross 
  Julien Grall 
  Kevin Tian 
  Lukas Juenger 
  M A Young 
  Norbert Manthey 
  Oleksandr Andrushchenko 
  Paul Durrant 
  Pawel Wieczorkiewicz 
  Petre Pircalabu 
  Razvan Cojocaru 
  Tamas K Lengyel 
  Wei Liu 
  Xiaochen Wang 
  Xin Li 

jobs:
 build-arm64-xsm  broken  
 build-amd64  fail
 build-armhf  fail
 build-amd64-libvirt  blocked 
 test-armhf-armhf-xl  blocked 
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386 blocked 
 test-amd64-amd64-libvirt blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64-xsm broken
broken-step build-arm64-xsm host-install(4)

Not pushing.

(No revision log; it would be 2016 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [linux-linus test] 134517: regressions - trouble: blocked/broken/fail/pass

2019-04-10 Thread osstest service owner
flight 134517 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/134517/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-pvopsbroken
 build-arm64  broken
 build-arm64-xsm  broken
 build-arm64   4 host-install(4)broken REGR. vs. 133580
 build-arm64-pvops 4 host-install(4)broken REGR. vs. 133580
 build-arm64-xsm   4 host-install(4)broken REGR. vs. 133580
 test-amd64-amd64-xl-credit2  12 guest-start  fail REGR. vs. 133580
 test-amd64-i386-xl   12 guest-start  fail REGR. vs. 133580
 test-amd64-amd64-libvirt 12 guest-start  fail REGR. vs. 133580
 test-amd64-i386-xl-shadow12 guest-start  fail REGR. vs. 133580
 test-amd64-i386-qemut-rhel6hvm-amd 12 guest-start/redhat.repeat fail REGR. vs. 
133580
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 10 debian-hvm-install 
fail REGR. vs. 133580
 test-amd64-i386-libvirt-xsm 18 guest-start/debian.repeat fail REGR. vs. 133580
 test-amd64-amd64-xl-pvhv2-amd 20 guest-start/debian.repeat fail REGR. vs. 
133580
 test-armhf-armhf-xl  12 guest-start  fail REGR. vs. 133580
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 18 
guest-start/debianhvm.repeat fail REGR. vs. 133580
 test-amd64-amd64-xl-qemuu-ovmf-amd64 18 guest-start/debianhvm.repeat fail 
REGR. vs. 133580

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds 12 guest-start  fail REGR. vs. 133580
 test-amd64-i386-rumprun-i386 17 rumprun-demo-xenstorels/xenstorels.repeat fail 
REGR. vs. 133580

Tests which did not succeed, but are not blocking:
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-arm64-arm64-examine  1 build-check(1)   blocked  n/a
 test-amd64-amd64-rumprun-amd64 17 rumprun-demo-xenstorels/xenstorels.repeat 
fail like 133580
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 133580
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 133580
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 133580
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 133580
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 133580
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 133580
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 133580
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 133580
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 

[Xen-devel] [xen-unstable-smoke test] 134612: regressions - trouble: blocked/broken/fail/pass

2019-04-10 Thread osstest service owner
flight 134612 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/134612/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken
 build-arm64-xsm   4 host-install(4)broken REGR. vs. 133991
 build-amd64   6 xen-buildfail REGR. vs. 133991
 test-armhf-armhf-xl  12 guest-start  fail REGR. vs. 133991

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386  1 build-check(1) blocked n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a

version targeted for testing:
 xen  0c07431cf712d990b3c60341b0b435c903bbf4f4
baseline version:
 xen  cb70a26f78848fe45f593f7ebc9cfaac760a791b

Last test of basis   133991  2019-03-22 15:00:46 Z   19 days
Failing since134068  2019-03-25 12:00:51 Z   16 days   73 attempts
Testing same since   134610  2019-04-10 16:00:43 Z0 days2 attempts


People who touched revisions under test:
  Alexandru Isaila 
  Alexandru Stefan ISAILA 
  Amit Singh Tomar 
  Andrew Cooper 
  Anthony PERARD 
  Brian Woods 
  Christian Lindig 
  Doug Goldstein 
  George Dunlap 
  Igor Druzhinin 
  Jan Beulich 
  Juergen Gross 
  Julien Grall 
  Kevin Tian 
  Lukas Juenger 
  M A Young 
  Norbert Manthey 
  Oleksandr Andrushchenko 
  Paul Durrant 
  Pawel Wieczorkiewicz 
  Petre Pircalabu 
  Razvan Cojocaru 
  Tamas K Lengyel 
  Wei Liu 
  Xiaochen Wang 
  Xin Li 

jobs:
 build-arm64-xsm  broken  
 build-amd64  fail
 build-armhf  pass
 build-amd64-libvirt  blocked 
 test-armhf-armhf-xl  fail
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386 blocked 
 test-amd64-amd64-libvirt blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64-xsm broken
broken-step build-arm64-xsm host-install(4)

Not pushing.

(No revision log; it would be 2016 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH RFC 00/49] xen: add core scheduling support

2019-04-10 Thread Dario Faggioli
On Fri, 2019-03-29 at 19:16 +0100, Dario Faggioli wrote:
> On Fri, 2019-03-29 at 16:08 +0100, Juergen Gross wrote:
> > I have done some very basic performance testing: on a 4 cpu system
> > (2 cores with 2 threads each) I did a "make -j 4" for building the
> > Xen
> > hypervisor. With This test has been run on dom0, once with no other
> > guest active and once with another guest with 4 vcpus running the
> > same
> > test.
> Just as an heads up for people (as Juergen knows this already :-D),
> I'm
> planning to run some performance evaluation of this patches.
> 
> I've got an 8 CPUs system (4 cores, 2 threads each, no-NUMA) and an
> 16
> CPUs system (2 sockets/NUMA nodes, 4 cores each, 2 threads each) on
> which I should be able to get some bench suite running relatively
> easy
> and (hopefully) quick.
> 
> I'm planning to evaluate:
> - vanilla (i.e., without this series), SMT enabled in BIOS
> - vanilla (i.e., without this series), SMT disabled in BIOS
> - patched (i.e., with this series), granularity=thread
> - patched (i.e., with this series), granularity=core
> 
> I'll do start with no overcommitment, and then move to 2x
> overcommitment (as you did above).
> 
I've got the first set of results. It's fewer than I wanted/expected to
have at this point in time, but still...

Also, it's Phoronix again. I don't especially love it, but I'm still
working on convincing our own internal automated benchmarking tool
(which I like a lot more :-) ) to be a good friend of Xen. :-P

It's a not too big set of tests, done in the following conditions:
- hardware: Intel Xeon E5620; 2 NUMA nodes, 4 cores and 2 threads each
- slow disk (old rotational HDD)
- benchmarks run in dom0
- CPU, memory and some disk IO benchmarks
- all Spec mitigations disabled both at Xen and dom0 kernel level
- cpufreq governor = performance, max_cstate = C1
- *non* debug hypervisor

In just one sentence, what I'd say is "So far so god" :-D

https://openbenchmarking.org/result/1904105-SP-1904100DA38

1) 'Xen dom0, SMT On, vanilla' is staging *without* this series even 
applied
2) 'Xen dom0, SMT on, patched, sched_granularity=thread' is with this 
series applied, but scheduler behavior as right now
3) 'Xen dom0, SMT on, patched, sched_granularity=core' is with this 
series applied, and core-scheduling enabled
4) 'Xen dom0, SMT Off, vanilla' is staging *without* this series 
applied, and SMT turned off in BIOS (i.e., we only have 8 CPUs)

So, comparing 1 and 4, we see, for each specific benchmark, what is the
cost of disabling SMT (or vice-versa, the gain of using SMT).

Comparing 1 and 2, we see the overhead introduced by this series, when
it is not used to achieve core-scheduling.

Compating 1 and 3, we see the differences with what we have right now,
and what we'll have with core-scheduling enabled, as it is implemented
in this series.

Some of the things we can see from the results:
- disabling SMT (i.e., 1 vs 4) is not always bad, but it is bad 
  overall, i.e., if you look at how many tests are better and at how 
  many are slower, with SMT off (and also, by how much). Of course, 
  this can be considered true for these specific benchmarks, on this 
  specific hardware and with this configuration
- the overhead introduced by this series is, overall, pretty small, 
  apart from not more than a couple of exceptions (e.g., Stream Triad 
  or zstd compression). OTOH, there seem to be cases where this series 
  improves performance (e.g., Stress-NG Socket Activity)
- the performance we achieve with core-scheduling are more than 
  acceptable
- between core-scheduling and disabling SMT, core-scheduling wins and
  I wouldn't even call it a match :-P

Of course, other thoughts, comments, alternative analysis are welcome.

As said above, this is less that what I wanted to have, and in fact I'm
running more stuff.

I have a much more comprehensive set of benchmarks running in these
days. It being "much more comprehensive", however, also means it takes
more time.

I have a newer and faster (both CPU and disk) machine, but I need to
re-purpose it for benchmarking purposes.

At least now that the old Xeon NUMA box is done with this first round,
I can use it for:
- running the tests inside a "regular" PV domain
- running the tests inside more than one PV domain, i.e. with some 
  degree of overcommitment

I'll push out results as soon as I have them.

Regards
-- 
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
---
<> (Raistlin Majere)



signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 1/2] libxl: Add virtio vga interface support for qemu

2019-04-10 Thread Chris Patterson
For anyone looking at this... while I have tested and verified that
both virtio-gpu and VirGL work, it's not without some hiccup.

I have been running Ubuntu 19.04 with this config for a few days and I
have had a couple VM freezes. 'xl dmesg' is spamming the following
repeatedly:
(XEN) irq.c:2479: dom3: pirq 143 or emuirq -3 already mapped

Seems to be related to MSI-X for the device, but I haven't yet had a
chance to dig into it.  Maybe someone knows if this is to be expected
given the state of things? :)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [qemu-mainline test] 134560: trouble: blocked/broken/fail/pass

2019-04-10 Thread osstest service owner
flight 134560 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/134560/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64  broken
 build-arm64-xsm  broken
 build-arm64-pvopsbroken
 build-arm64-xsm   4 host-install(4)broken REGR. vs. 133909
 build-arm64   4 host-install(4)broken REGR. vs. 133909
 build-arm64-pvops 4 host-install(4)broken REGR. vs. 133909

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 133909
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 133909
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 133909
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 133909
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 133909
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass

version targeted for testing:
 qemuu5263724b78f89cdea2354c8e92c53bac1b4641a3
baseline version:
 qemuu082c0543baa6f237704c83a51658bd7f6ae316d5

Last test of basis   133909  2019-03-18 17:20:53 Z   23 days
Failing since133939  2019-03-20 04:22:12 Z   21 days   16 attempts
Testing same since   134560  2019-04-09 04:26:43 Z1 days1 attempts


People who touched revisions under test:
  "Cédric Le Goater" 
  Alberto Garcia 
  Alex Bennée 
  Alex Williamson mailto:alex.william...@redhat.com; 
target="_blank" rel="noreferrer">alex.william...@redhat.com
  Alex Williamson 
  Alistair Francis 
  Andrew Jones 
  Anthony PERARD 
  BALATON Zoltan 
  Bandan Das 
  Benjamin Herrenschmidt 
  Bin Meng 
  Bishara AbuHattoum 
  Brijesh Singh 
  Chih-Min Chao 
  Cleber Rosa 
  Cornelia Huck 
  Cédric Le Goater 
  Daniel Henrique Barboza 
  Daniel P. Berrange 
  Daniel P. Berrangé 
  David Gibson 
  David Hildenbrand 
  Dr. David Alan 

[Xen-devel] [linux-4.19 test] 134556: regressions - trouble: blocked/broken/fail/pass

2019-04-10 Thread osstest service owner
flight 134556 linux-4.19 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/134556/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-pvopsbroken
 build-arm64  broken
 build-arm64-xsm  broken
 build-arm64-pvops 4 host-install(4)broken REGR. vs. 129313
 build-arm64-xsm   4 host-install(4)broken REGR. vs. 129313
 build-arm64   4 host-install(4)broken REGR. vs. 129313
 test-amd64-amd64-examine  4 memdisk-try-append   fail REGR. vs. 129313

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-arm64-arm64-examine  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass

version targeted for testing:
 linux4d552acf337038028f7e2f63a927afb7adf65fc1
baseline version:
 linux84df9525b0c27f3ebc2ebb1864fa62a97fdedb7d

Last test of basis   129313  2018-11-02 05:39:08 Z  159 days
Failing since129412  2018-11-04 14:10:15 Z  157 days  104 attempts
Testing same since   134461  2019-04-05 21:43:26 Z4 days2 attempts


Re: [Xen-devel] Xen ARM dom0less use PV drivers

2019-04-10 Thread Stefano Stabellini
On Wed, 10 Apr 2019, Julien Grall wrote:
> Hi,
> 
> CCing Stefano who is looking after dom0less.
>
> On 08/04/2019 03:17, jinchen wrote:
> > Hello experts:
> >  ?0?2 ?0?2 The xen 4.12 support the Dom0less VM that start domu from xen not
> > xl, but the PV drivers hasn't been supported.

That's right. Please refer to docs/misc/arm/device-tree/booting.txt for
examples on how to boot other VMs from Xen. Also, I wrote a more
detailed tutorial on how to use dom0less that I have yet to publish.
Stay tuned.


> > Do you have some guidance for how to enable PV drivers when using dom0less?

The first question would be, do you actually need PV drivers between the
dom0less VMs? Because PV drivers are still available for all VMs started
later from dom0. Also, if you just need a simple communication mechanism
between the dom0less VMs, it would be pretty simple to setup a shared
memory area between them. Otherwise, see below.


> >  ?0?2 ?0?2 Due to kernel use xenbus and xenstore to probe PV drivers, these
> > mechanism should?0?2remain for compatibility?
> >  ?0?2 ?0?2 Removal the xenstore mechanism to xen internal or some other way?
> >  ?0?2 ?0?2 Thank you very much??

If you really need PV drivers in dom0less VMs, then the first thing to
do would be to get xenstore up and running in a dom0less domU. It is a
matter of allocating the proper resources, such as the evtchn page. It
is also a matter of getting xenstore up and running even if it might have
to wait for xenstored in dom0 to be initialized before being able to
make a connection. I expect this not to be trivial. 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-4.8-testing test] 134552: regressions - trouble: blocked/broken/fail/pass

2019-04-10 Thread osstest service owner
flight 134552 xen-4.8-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/134552/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-pvopsbroken
 build-arm64  broken
 build-arm64-xsm  broken
 test-armhf-armhf-xl-vhd  broken
 test-amd64-amd64-libvirt-pair 22 guest-migrate/src_host/dst_host fail REGR. 
vs. 130965

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-vhd   4 host-install(4)  broken pass in 134442
 test-xtf-amd64-amd64-2 50 xtf/test-hvm64-lbr-tsx-vmentry fail in 134442 pass 
in 134552
 test-xtf-amd64-amd64-5 69 xtf/test-hvm64-xsa-278 fail in 134442 pass in 134552
 test-xtf-amd64-amd64-1 69 xtf/test-hvm64-xsa-278 fail in 134442 pass in 134552
 test-amd64-i386-libvirt-pair 22 guest-migrate/src_host/dst_host fail pass in 
133662
 test-xtf-amd64-amd64-4   50 xtf/test-hvm64-lbr-tsx-vmentry fail pass in 134442

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 build-arm64-xsm   4 host-install(4)   broken baseline untested
 build-arm64-pvops 4 host-install(4)   broken baseline untested
 build-arm64   4 host-install(4)   broken baseline untested
 test-arm64-arm64-xl-xsm 13 migrate-support-check fail in 133662 never pass
 test-arm64-arm64-xl-credit1 13 migrate-support-check fail in 133662 never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-check fail in 133662 never pass
 test-arm64-arm64-xl 13 migrate-support-check fail in 133662 never pass
 test-arm64-arm64-xl-credit2 13 migrate-support-check fail in 133662 never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-check fail in 133662 never 
pass
 test-arm64-arm64-xl-credit2 14 saverestore-support-check fail in 133662 never 
pass
 test-arm64-arm64-xl 14 saverestore-support-check fail in 133662 never pass
 test-arm64-arm64-xl-xsm 14 saverestore-support-check fail in 133662 never pass
 test-arm64-arm64-xl-credit1 14 saverestore-support-check fail in 133662 never 
pass
 test-xtf-amd64-amd64-5 50 xtf/test-hvm64-lbr-tsx-vmentry fail in 134442 like 
130965
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeat fail in 134442 like 
130965
 test-armhf-armhf-xl-vhd 12 migrate-support-check fail in 134442 never pass
 test-armhf-armhf-xl-vhd 13 saverestore-support-check fail in 134442 never pass
 test-xtf-amd64-amd64-1  50 xtf/test-hvm64-lbr-tsx-vmentry fail like 130965
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 130965
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 130965
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 130965
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 130965
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop fail like 130965
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 130965
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 130965
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 130965
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop fail like 130965
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 

Re: [Xen-devel] [PATCH RFC 00/39] x86/KVM: Xen HVM guest support

2019-04-10 Thread Stefano Stabellini
On Tue, 9 Apr 2019, Ankur Arora wrote:
> On 2019-04-08 5:35 p.m., Stefano Stabellini wrote:
> > On Mon, 8 Apr 2019, Joao Martins wrote:
> > > On 4/8/19 11:42 AM, Juergen Gross wrote:
> > > > On 08/04/2019 12:36, Joao Martins wrote:
> > > > > On 4/8/19 7:44 AM, Juergen Gross wrote:
> > > > > > On 12/03/2019 18:14, Joao Martins wrote:
> > > > > > > On 2/22/19 4:59 PM, Paolo Bonzini wrote:
> > > > > > > > On 21/02/19 12:45, Joao Martins wrote:
> > > > > > > > > On 2/20/19 9:09 PM, Paolo Bonzini wrote:
> > > > > > > > > > On 20/02/19 21:15, Joao Martins wrote:
> > > > > > > > > > >   2. PV Driver support (patches 17 - 39)
> > > > > > > > > > > 
> > > > > > > > > > >   We start by redirecting hypercalls from the backend to
> > > > > > > > > > > routines
> > > > > > > > > > >   which emulate the behaviour that PV backends expect i.e.
> > > > > > > > > > > grant
> > > > > > > > > > >   table and interdomain events. Next, we add support for
> > > > > > > > > > > late
> > > > > > > > > > >   initialization of xenbus, followed by implementing
> > > > > > > > > > >   frontend/backend communication mechanisms (i.e. grant
> > > > > > > > > > > tables and
> > > > > > > > > > >   interdomain event channels). Finally, introduce
> > > > > > > > > > > xen-shim.ko,
> > > > > > > > > > >   which will setup a limited Xen environment. This uses
> > > > > > > > > > > the added
> > > > > > > > > > >   functionality of Xen specific shared memory (grant
> > > > > > > > > > > tables) and
> > > > > > > > > > >   notifications (event channels).
> > > > > > > > > > 
> > > > > > > > > > I am a bit worried by the last patches, they seem really
> > > > > > > > > > brittle and
> > > > > > > > > > prone to breakage.  I don't know Xen well enough to
> > > > > > > > > > understand if the
> > > > > > > > > > lack of support for GNTMAP_host_map is fixable, but if not,
> > > > > > > > > > you have to
> > > > > > > > > > define a completely different hypercall.
> > > > > > > > > > 
> > > > > > > > > I guess Ankur already answered this; so just to stack this on
> > > > > > > > > top of his comment.
> > > > > > > > > 
> > > > > > > > > The xen_shim_domain() is only meant to handle the case where
> > > > > > > > > the backend
> > > > > > > > > has/can-have full access to guest memory [i.e. netback and
> > > > > > > > > blkback would work
> > > > > > > > > with similar assumptions as vhost?]. For the normal case,
> > > > > > > > > where a backend *in a
> > > > > > > > > guest* maps and unmaps other guest memory, this is not
> > > > > > > > > applicable and these
> > > > > > > > > changes don't affect that case.
> > > > > > > > > 
> > > > > > > > > IOW, the PV backend here sits on the hypervisor, and the
> > > > > > > > > hypercalls aren't
> > > > > > > > > actual hypercalls but rather invoking shim_hypercall(). The
> > > > > > > > > call chain would go
> > > > > > > > > more or less like:
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > >   gnttab_map_refs(map_ops, pages)
> > > > > > > > > HYPERVISOR_grant_table_op(GNTTABOP_map_grant_ref,...)
> > > > > > > > >   shim_hypercall()
> > > > > > > > > shim_hcall_gntmap()
> > > > > > > > > 
> > > > > > > > > Our reasoning was that given we are already in KVM, why
> > > > > > > > > mapping a page if the
> > > > > > > > > user (i.e. the kernel PV backend) is himself? The lack of
> > > > > > > > > GNTMAP_host_map is how
> > > > > > > > > the shim determines its user doesn't want to map the page.
> > > > > > > > > Also, there's another
> > > > > > > > > issue where PV backends always need a struct page to reference
> > > > > > > > > the device
> > > > > > > > > inflight data as Ankur pointed out.
> > > > > > > > 
> > > > > > > > Ultimately it's up to the Xen people.  It does make their API
> > > > > > > > uglier,
> > > > > > > > especially the in/out change for the parameter.  If you can at
> > > > > > > > least
> > > > > > > > avoid that, it would alleviate my concerns quite a bit.
> > > > > > > 
> > > > > > > In my view, we have two options overall:
> > > > > > > 
> > > > > > > 1) Make it explicit, the changes the PV drivers we have to make in
> > > > > > > order to support xen_shim_domain(). This could mean e.g. a) add a
> > > > > > > callback
> > > > > > > argument to gnttab_map_refs() that is invoked for every page that
> > > > > > > gets looked up
> > > > > > > successfully, and inside this callback the PV driver may update
> > > > > > > it's tracking
> > > > > > > page. Here we no longer have this in/out parameter in
> > > > > > > gnttab_map_refs, and all
> > > > > > > shim_domain specific bits would be a little more abstracted from
> > > > > > > Xen PV
> > > > > > > backends. See netback example below the scissors mark. Or b) have
> > > > > > > sort of a
> > > > > > > translate_gref() and put_gref() API that Xen PV drivers use which
> > > > > > > make it even
> > > > > > > more explicit that there's no grant ops involved. The latter is
> > > > > > > more invasive.
> > > > > > > 
> 

Re: [Xen-devel] status of non-live migration of HVM with libvirt

2019-04-10 Thread Olaf Hering
Am Wed, 10 Apr 2019 17:25:04 +0100
schrieb Anthony PERARD :

> Because "Snapshots don't need to inactivate images", see:
> https://lists.xenproject.org/archives/html/xen-devel/2017-10/msg02782.html
> 
> I had to add the `live' parameter to fix the lock issue.

Thanks, I will double check this part.

Olaf


pgpfAaJ8wMhgo.pgp
Description: Digitale Signatur von OpenPGP
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [edk2-devel] [PATCH v2 02/31] OvmfPkg: Create platform XenOvmf

2019-04-10 Thread Jordan Justen
On 2019-04-10 07:27:15, Laszlo Ersek wrote:
> On 04/10/19 11:48, Jordan Justen wrote:
> > On 2019-04-09 04:08:15, Anthony PERARD wrote:
> >> This is a copy of OvmfX64, removing VirtIO and some SMM.
> >>
> >> This new platform will be changed to make it works on two types of Xen
> >> guest: HVM and PVH.
> >>
> >> Compare to OvmfX64, this patch:
> >>
> >> - changed: PLATFORM_GUID, OUTPUT_DIRECTORY, FLASH_DEFINITION
> >> - removed: VirtioLib class resolution
> >> - removed: all UEFI_DRIVER modules for virtio devices
> >> - removed: DXE_SMM_DRIVER and SMM_CORE lib class resolutions
> >> - removed: DXE_SMM_DRIVER and SMM_CORE FDF rules
> >> - removed: Everything related to SMM_REQUIRE==true
> >> - removed: Everything related to SECURE_BOOT_ENABLE==true
> >> - removed: Everything related to TPM2_ENABLE==true
> >> - changed: PcdPciDisableBusEnumeration dynamic default flipped to TRUE
> >> - changed: default FD_SIZE_IN_KB to 2M.
> >>
> >> Contributed-under: TianoCore Contribution Agreement 1.1
> >> Signed-off-by: Anthony PERARD 
> >> ---
> >>  OvmfPkg/{OvmfPkgX64.dsc => XenOvmf.dsc} | 202 +---
> > 
> > I guess we all want our name to be first :), but OvmfXen seems more
> > like the pattern that edk2 uses when making sub-platforms.

For example: ArmVirtPkg/ArmVirtXen.dsc

> > 
> > Also, in some cases we've used !ifdef type things to keep dsc and fdf
> > files common. Would that not be workable here? Maybe it would get too
> > ugly. :\
> 
> I've been happy with a similar Xen<->QEMU split under ArmVirtPkg.
> Duplications in updates are usually not a huge burden, and keeping the
> files separate has proved very helpful for determining
> maintainer/reviewer/tester responsibility.
> 
> > It could help to prevent having to sync dsc changes across, and
> > prevent changes from being omitted for Xen on accident.
> 
> True, but in my experience that's been the smaller problem. The larger
> problem has been modifying Xen stuff in unintended ways (= regressing
> Xen), because we can't test (or don't even notice) the Xen-side
> implications of changes made to common source.

Does that mean you avoid changing ArmVirtXen.dsc entirely, or you try
to update it when it seems likely to not cause trouble? I could see
unintended breakage either way.

Anyway, it sounds like it's generally working out okay with
ArmVirtXen, so it seems okay to follow that under OvmfPkg.

-Jordan

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#38833): https://edk2.groups.io/g/devel/message/38833
Mute This Topic: https://groups.io/mt/30997887/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[Xen-devel] [xen-unstable-smoke test] 134610: regressions - trouble: blocked/broken/fail

2019-04-10 Thread osstest service owner
flight 134610 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/134610/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken
 build-arm64-xsm   4 host-install(4)broken REGR. vs. 133991
 build-amd64   6 xen-buildfail REGR. vs. 133991
 build-armhf   6 xen-buildfail REGR. vs. 133991

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386  1 build-check(1) blocked n/a

version targeted for testing:
 xen  0c07431cf712d990b3c60341b0b435c903bbf4f4
baseline version:
 xen  cb70a26f78848fe45f593f7ebc9cfaac760a791b

Last test of basis   133991  2019-03-22 15:00:46 Z   19 days
Failing since134068  2019-03-25 12:00:51 Z   16 days   72 attempts
Testing same since   134610  2019-04-10 16:00:43 Z0 days1 attempts


People who touched revisions under test:
  Alexandru Isaila 
  Alexandru Stefan ISAILA 
  Amit Singh Tomar 
  Andrew Cooper 
  Anthony PERARD 
  Brian Woods 
  Christian Lindig 
  Doug Goldstein 
  George Dunlap 
  Igor Druzhinin 
  Jan Beulich 
  Juergen Gross 
  Julien Grall 
  Kevin Tian 
  Lukas Juenger 
  M A Young 
  Norbert Manthey 
  Oleksandr Andrushchenko 
  Paul Durrant 
  Pawel Wieczorkiewicz 
  Petre Pircalabu 
  Razvan Cojocaru 
  Tamas K Lengyel 
  Wei Liu 
  Xiaochen Wang 
  Xin Li 

jobs:
 build-arm64-xsm  broken  
 build-amd64  fail
 build-armhf  fail
 build-amd64-libvirt  blocked 
 test-armhf-armhf-xl  blocked 
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386 blocked 
 test-amd64-amd64-libvirt blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64-xsm broken
broken-step build-arm64-xsm host-install(4)

Not pushing.

(No revision log; it would be 2016 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-4.9-testing test] 134514: regressions - trouble: blocked/broken/fail/pass

2019-04-10 Thread osstest service owner
flight 134514 xen-4.9-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/134514/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64  broken
 build-arm64-xsm  broken
 build-arm64-pvopsbroken
 build-arm64   4 host-install(4)broken REGR. vs. 132889
 build-arm64-xsm   4 host-install(4)broken REGR. vs. 132889
 build-arm64-pvops 4 host-install(4)broken REGR. vs. 132889
 test-amd64-i386-libvirt-pair 22 guest-migrate/src_host/dst_host fail REGR. vs. 
132889
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail in 134283 REGR. vs. 
132889
 test-amd64-amd64-libvirt-pair 22 guest-migrate/src_host/dst_host fail in 
134473 REGR. vs. 132889

Tests which are failing intermittently (not blocking):
 test-amd64-i386-pair   21 guest-start/debian fail in 134374 pass in 134514
 test-amd64-i386-xl  20 guest-start/debian.repeat fail in 134374 pass in 134514
 test-amd64-amd64-xl-xsm 20 guest-start/debian.repeat fail in 134374 pass in 
134514
 test-armhf-armhf-xl-arndale  12 guest-start  fail in 134374 pass in 134514
 test-amd64-i386-libvirt-xsm 18 guest-start/debian.repeat fail in 134374 pass 
in 134514
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 16 
guest-start/debianhvm.repeat fail in 134374 pass in 134514
 test-amd64-amd64-pygrub 19 guest-start/debian.repeat fail in 134374 pass in 
134514
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 18 
guest-start/debianhvm.repeat fail in 134374 pass in 134514
 test-amd64-amd64-libvirt-vhd 17 guest-start/debian.repeat fail in 134374 pass 
in 134514
 test-armhf-armhf-xl-credit2 16 guest-start/debian.repeat fail in 134374 pass 
in 134514
 test-amd64-amd64-xl-qemuu-debianhvm-amd64 18 guest-start/debianhvm.repeat fail 
in 134374 pass in 134514
 test-amd64-i386-xl-qemuu-ovmf-amd64 18 guest-start/debianhvm.repeat fail in 
134374 pass in 134514
 test-amd64-amd64-xl-qemut-ws16-amd64 16 guest-localmigrate/x10 fail in 134473 
pass in 134283
 test-amd64-i386-xl-qemut-ws16-amd64 16 guest-localmigrate/x10 fail in 134473 
pass in 134514
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-localmigrate/x10 fail pass in 
134374
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-localmigrate/x10 fail pass in 
134374
 test-amd64-i386-xl-shadow12 guest-startfail pass in 134473
 test-amd64-i386-xl-xsm   12 guest-startfail pass in 134473
 test-amd64-amd64-xl-credit1  12 guest-startfail pass in 134473
 test-amd64-amd64-xl-credit2  12 guest-startfail pass in 134473
 test-amd64-amd64-xl-shadow   12 guest-startfail pass in 134473
 test-amd64-amd64-libvirt-pair 21 guest-start/debianfail pass in 134473
 test-amd64-amd64-libvirt-xsm 18 guest-start/debian.repeat  fail pass in 134473
 test-amd64-amd64-xl  20 guest-start/debian.repeat  fail pass in 134473
 test-amd64-amd64-xl-multivcpu 20 guest-start/debian.repeat fail pass in 134473
 test-amd64-amd64-xl-qemut-ws16-amd64 14 guest-localmigrate fail pass in 134473
 test-armhf-armhf-xl-multivcpu 16 guest-start/debian.repeat fail pass in 134473
 test-armhf-armhf-libvirt 12 guest-startfail pass in 134473
 test-amd64-amd64-xl-qemuu-ws16-amd64 16 guest-localmigrate/x10 fail pass in 
134473

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop  fail blocked in 132889
 test-armhf-armhf-xl-rtds   16 guest-start/debian.repeat fail blocked in 132889
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop   fail blocked in 132889
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail in 134374 blocked in 
132889
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop  fail in 134374 like 132889
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop   fail in 134374 like 132889
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop  fail in 134473 like 132889
 test-armhf-armhf-libvirt13 migrate-support-check fail in 134473 never pass
 test-armhf-armhf-libvirt 14 saverestore-support-check fail in 134473 never pass
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-localmigrate/x10 fail like 132889
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 132889
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop fail like 132889
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   

Re: [edk2-devel] [PATCH v2 02/31] OvmfPkg: Create platform XenOvmf

2019-04-10 Thread Ard Biesheuvel
On Wed, 10 Apr 2019 at 07:27, Laszlo Ersek  wrote:
>
> On 04/10/19 11:48, Jordan Justen wrote:
> > On 2019-04-09 04:08:15, Anthony PERARD wrote:
> >> This is a copy of OvmfX64, removing VirtIO and some SMM.
> >>
> >> This new platform will be changed to make it works on two types of Xen
> >> guest: HVM and PVH.
> >>
> >> Compare to OvmfX64, this patch:
> >>
> >> - changed: PLATFORM_GUID, OUTPUT_DIRECTORY, FLASH_DEFINITION
> >> - removed: VirtioLib class resolution
> >> - removed: all UEFI_DRIVER modules for virtio devices
> >> - removed: DXE_SMM_DRIVER and SMM_CORE lib class resolutions
> >> - removed: DXE_SMM_DRIVER and SMM_CORE FDF rules
> >> - removed: Everything related to SMM_REQUIRE==true
> >> - removed: Everything related to SECURE_BOOT_ENABLE==true
> >> - removed: Everything related to TPM2_ENABLE==true
> >> - changed: PcdPciDisableBusEnumeration dynamic default flipped to TRUE
> >> - changed: default FD_SIZE_IN_KB to 2M.
> >>
> >> Contributed-under: TianoCore Contribution Agreement 1.1
> >> Signed-off-by: Anthony PERARD 
> >> ---
> >>  OvmfPkg/{OvmfPkgX64.dsc => XenOvmf.dsc} | 202 +---
> >
> > I guess we all want our name to be first :), but OvmfXen seems more
> > like the pattern that edk2 uses when making sub-platforms.
> >
> > Also, in some cases we've used !ifdef type things to keep dsc and fdf
> > files common. Would that not be workable here? Maybe it would get too
> > ugly. :\
>
> I've been happy with a similar Xen<->QEMU split under ArmVirtPkg.
> Duplications in updates are usually not a huge burden, and keeping the
> files separate has proved very helpful for determining
> maintainer/reviewer/tester responsibility.
>
> > It could help to prevent having to sync dsc changes across, and
> > prevent changes from being omitted for Xen on accident.
>
> True, but in my experience that's been the smaller problem. The larger
> problem has been modifying Xen stuff in unintended ways (= regressing
> Xen), because we can't test (or don't even notice) the Xen-side
> implications of changes made to common source.
>
> Personally I prefer another (DSC + FDF) pair under OvmfPkg (beyond the
> three we already have) to another layer of (conditional?) includes. The
> !if's that are being eliminated in this Xen-customized copy (i.e., in
> this patch) are complex enough already :)
>
> ... It's not that I *generally* prefer duplication; as you say, we do
> have a number of !includes already. I do prefer duplication specifically
> for Xen vs. QEMU however.
>

+1

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#38825): https://edk2.groups.io/g/devel/message/38825
Mute This Topic: https://groups.io/mt/30997887/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [Xen-devel] status of non-live migration of HVM with libvirt

2019-04-10 Thread Anthony PERARD
On Mon, Jan 07, 2019 at 11:07:43AM +0100, Olaf Hering wrote:
> Am Fri, 4 Jan 2019 17:48:31 +0100
> schrieb Olaf Hering :
> 
> > Am Fri, 4 Jan 2019 16:57:55 +0100
> > schrieb Olaf Hering :
> > 
> > > worth keeping (and fixing) the concept of an "offline migration"  
> > 
> > And regarding the fix, it looks like qmp_xen_save_devices_state does not 
> > need the concept of "live". Just shutdown the blockdevices and be done with 
> > it?
> 
> Anthony, it looks like 5d6c599fe1 ("migration, xen: Fix block image
> lock issue on live migration") went too far. Why does stopping the
> disks at the very end of the save/migrate process depend on "live"?

Because "Snapshots don't need to inactivate images", see:
https://lists.xenproject.org/archives/html/xen-devel/2017-10/msg02782.html

I had to add the `live' parameter to fix the lock issue.

> During a migration the disks have to be released either way. During "xl save" 
> the domU may continue to run, if '-c' was specified. 

xl save -c is a snapshot, right? There isn't a second QEMU process that
is going to read/write the same disk.

> It seems the 'live' parameter for xen-save-devices-state is not needed.
> 
> Olaf



-- 
Anthony PERARD

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [edk2-devel] [PATCH 0/4] OvmfPkg: replace MIT license blocks with SPDX IDs

2019-04-10 Thread Ard Biesheuvel
On Wed, 10 Apr 2019 at 05:58, Laszlo Ersek  wrote:
>
> Repo:   https://github.com/lersek/edk2.git
> Branch: ovmf_spdx_mit
>
> For , we replaced
> open-coded license text blocks with "SPDX-License-Identifier"s, almost
> all over the edk2 tree.
>
> That change however was tied to a license update / CLA update too
> (please see details in TianoCore#1373, referenced above), and therefore
> the MIT-covered files under OvmfPkg were out of scope. We filed a
> reminder at , and
> this series intends to address that -- i.e., to replace the MIT license
> blocks with the corresponding SPDX License IDs.
>
> Cc: Anthony Perard 
> Cc: Ard Biesheuvel 
> Cc: Jordan Justen 
> Cc: Julien Grall 
> Cc: Lars Kurth 
> Cc: xen-devel@lists.xenproject.org
>
> Thanks,
> Laszlo
>
> Laszlo Ersek (4):
>   OvmfPkg/License.txt: remove XenPvBlkDxe from the MIT licensed dir list
>   OvmfPkg/License.txt: refresh the MIT license text and include the SPDX
> ID
>   OvmfPkg/IndustryStandard/Xen: replace MIT license text with SPDX ID
>   OvmfPkg/XenBusDxe: replace MIT license text with SPDX ID
>

For the series

Reviewed-by: Ard Biesheuvel 

>  OvmfPkg/Include/IndustryStandard/Xen/arch-arm/xen.h| 18 
> +-
>  OvmfPkg/Include/IndustryStandard/Xen/arch-x86/xen-x86_32.h | 18 
> +-
>  OvmfPkg/Include/IndustryStandard/Xen/arch-x86/xen-x86_64.h | 18 
> +-
>  OvmfPkg/Include/IndustryStandard/Xen/arch-x86/xen.h| 18 
> +-
>  OvmfPkg/Include/IndustryStandard/Xen/event_channel.h   | 18 
> +-
>  OvmfPkg/Include/IndustryStandard/Xen/grant_table.h | 18 
> +-
>  OvmfPkg/Include/IndustryStandard/Xen/hvm/hvm_op.h  | 18 
> +-
>  OvmfPkg/Include/IndustryStandard/Xen/hvm/params.h  | 18 
> +-
>  OvmfPkg/Include/IndustryStandard/Xen/io/blkif.h| 18 
> +-
>  OvmfPkg/Include/IndustryStandard/Xen/io/console.h  | 18 
> +-
>  OvmfPkg/Include/IndustryStandard/Xen/io/protocols.h| 18 
> +-
>  OvmfPkg/Include/IndustryStandard/Xen/io/ring.h | 18 
> +-
>  OvmfPkg/Include/IndustryStandard/Xen/io/xenbus.h   | 18 
> +-
>  OvmfPkg/Include/IndustryStandard/Xen/io/xs_wire.h  | 18 
> +-
>  OvmfPkg/Include/IndustryStandard/Xen/memory.h  | 18 
> +-
>  OvmfPkg/Include/IndustryStandard/Xen/xen-compat.h  | 18 
> +-
>  OvmfPkg/Include/IndustryStandard/Xen/xen.h | 18 
> +-
>  OvmfPkg/License.txt|  8 +---
>  OvmfPkg/XenBusDxe/XenBus.c | 18 
> +-
>  OvmfPkg/XenBusDxe/XenStore.c   | 18 
> +-
>  OvmfPkg/XenBusDxe/XenStore.h   | 18 
> +-
>  21 files changed, 25 insertions(+), 343 deletions(-)
>
> --
> 2.19.1.3.g30247aa5d201
>

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#38823): https://edk2.groups.io/g/devel/message/38823
Mute This Topic: https://groups.io/mt/31018509/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [Xen-devel] [PATCH v3 1/3] x86/mm: Introduce altp2m_get_gfn_type_access

2019-04-10 Thread George Dunlap
On 4/9/19 1:03 PM, Alexandru Stefan ISAILA wrote:
> This patch moves common code from p2m_set_altp2m_mem_access() and
> p2m_change_altp2m_gfn() into one function
> 
> Signed-off-by: Alexandru Isaila 

This patch contains a lot of behavioral changes which aren't mentioned
or explained.  For instance...

> ---
> Changes since V2:
>   - Change var name from found_in_hostp2m to copied_from_hostp2m
>   - Move the type check from altp2m_get_gfn_type_access() to the
>   callers.
> ---
>  xen/arch/x86/mm/mem_access.c | 32 
>  xen/arch/x86/mm/p2m.c| 41 ++--
>  xen/include/asm-x86/p2m.h| 19 +
>  3 files changed, 49 insertions(+), 43 deletions(-)
> 
> diff --git a/xen/arch/x86/mm/mem_access.c b/xen/arch/x86/mm/mem_access.c
> index 56c06a4fc6..bf67ddb15a 100644
> --- a/xen/arch/x86/mm/mem_access.c
> +++ b/xen/arch/x86/mm/mem_access.c
> @@ -265,31 +265,27 @@ int p2m_set_altp2m_mem_access(struct domain *d, struct 
> p2m_domain *hp2m,
>  unsigned int page_order;
>  unsigned long gfn_l = gfn_x(gfn);
>  int rc;
> +bool copied_from_hostp2m;
>  
> -mfn = ap2m->get_entry(ap2m, gfn, , _a, 0, NULL, NULL);
> +mfn = altp2m_get_gfn_type_access(ap2m, gfn, , _a, _order, 
> _from_hostp2m);
>  
> -/* Check host p2m if no valid entry in alternate */
>  if ( !mfn_valid(mfn) )
> +return -ESRCH;
> +
> +/* If this is a superpage, copy that first */
> +if ( page_order != PAGE_ORDER_4K && copied_from_hostp2m )
>  {
> +unsigned long mask = ~((1UL << page_order) - 1);
> +gfn_t gfn2 = _gfn(gfn_l & mask);
> +mfn_t mfn2 = _mfn(mfn_x(mfn) & mask);
>  
> -mfn = __get_gfn_type_access(hp2m, gfn_l, , _a,
> -P2M_ALLOC | P2M_UNSHARE, _order, 0);
> +/* Note: currently it is not safe to remap to a shared entry */
> +if ( t != p2m_ram_rw )
> +return -ESRCH;
>  
> -rc = -ESRCH;
> -if ( !mfn_valid(mfn) || t != p2m_ram_rw )
> +rc = ap2m->set_entry(ap2m, gfn2, mfn2, page_order, t, old_a, 1);
> +if ( rc )
>  return rc;
> -
> -/* If this is a superpage, copy that first */
> -if ( page_order != PAGE_ORDER_4K )
> -{
> -unsigned long mask = ~((1UL << page_order) - 1);
> -gfn_t gfn2 = _gfn(gfn_l & mask);
> -mfn_t mfn2 = _mfn(mfn_x(mfn) & mask);
> -
> -rc = ap2m->set_entry(ap2m, gfn2, mfn2, page_order, t, old_a, 1);
> -if ( rc )
> -return rc;
> -}
>  }
>  
>  /*
> diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
> index b9bbb8f485..d38d7c29ca 100644
> --- a/xen/arch/x86/mm/p2m.c
> +++ b/xen/arch/x86/mm/p2m.c
> @@ -2626,6 +2626,7 @@ int p2m_change_altp2m_gfn(struct domain *d, unsigned 
> int idx,
>  mfn_t mfn;
>  unsigned int page_order;
>  int rc = -EINVAL;
> +bool copied_from_hostp2m;
>  
>  if ( idx >= MAX_ALTP2M || d->arch.altp2m_eptp[idx] == mfn_x(INVALID_MFN) 
> )
>  return rc;
> @@ -2636,7 +2637,7 @@ int p2m_change_altp2m_gfn(struct domain *d, unsigned 
> int idx,
>  p2m_lock(hp2m);
>  p2m_lock(ap2m);
>  
> -mfn = ap2m->get_entry(ap2m, old_gfn, , , 0, NULL, NULL);
> +mfn = altp2m_get_gfn_type_access(ap2m, old_gfn, , , _order, 
> _from_hostp2m);

Before, if new_gfn was INVALID_GFN, then the host p2m wasn't read at
all.  Now, the hostp2m will have __get_gfn_type_access() called with
P2M_ALLOC | P2M_UNSHARE.  Is that change intentional, and if so, why?

>  
>  if ( gfn_eq(new_gfn, INVALID_GFN) )
>  {
> @@ -2646,37 +2647,27 @@ int p2m_change_altp2m_gfn(struct domain *d, unsigned 
> int idx,
>  goto out;
>  }
>  
> -/* Check host p2m if no valid entry in alternate */
> -if ( !mfn_valid(mfn) )
> -{
> -mfn = __get_gfn_type_access(hp2m, gfn_x(old_gfn), , ,
> -P2M_ALLOC, _order, 0);
> +if ( !mfn_valid(mfn) || (t != p2m_ram_rw && copied_from_hostp2m) )
> + goto out;
>  
> -if ( !mfn_valid(mfn) || t != p2m_ram_rw )
> -goto out;
> -
> -/* If this is a superpage, copy that first */
> -if ( page_order != PAGE_ORDER_4K )
> -{
> -gfn_t gfn;
> -unsigned long mask;
> +/* If this is a superpage, copy that first */
> +if ( page_order != PAGE_ORDER_4K && copied_from_hostp2m )
> +{
> +gfn_t gfn;
> +unsigned long mask;
>  
> -mask = ~((1UL << page_order) - 1);
> -gfn = _gfn(gfn_x(old_gfn) & mask);
> -mfn = _mfn(mfn_x(mfn) & mask);
> +mask = ~((1UL << page_order) - 1);
> +gfn = _gfn(gfn_x(old_gfn) & mask);
> +mfn = _mfn(mfn_x(mfn) & mask);
>  
> -if ( ap2m->set_entry(ap2m, gfn, mfn, page_order, t, a, 1) )
> -goto out;
> -}
> +if ( 

Re: [Xen-devel] [OSSTEST PATCH 55/62] make-flight: shadow test: Disable kpti in guests

2019-04-10 Thread Ian Jackson
Andrew Cooper writes ("Re: [OSSTEST PATCH 55/62] make-flight: shadow test: 
Disable kpti in guests"):
> On 10/04/2019 15:24, Ian Jackson wrote:
> > Since Spectre/Meltdown, shadow has been a lot slower, especially with
> > KPTI in the guest.  Empirically, too slow (with the kernel from Debian
> > stretch).
> 
> The speed of shadow pagetables hasn't changed - I don't think we even
> touched the shadow code at all for XSA-254.
> 
> The problem is the change in guest behaviour as a consequence of needing
> KPTI for a Meltdown mitigation.
> 
> The guest now flushes its pagetables on every
> syscall/interrupt/exception rather than once on a process=>process
> context switch, which is why running a guest using KPTI in shadow mode
> is boarderline unusable.
> 
> The actual change to use nopti looks fine.

Thanks for the explanation.  Unfortunately for the reasons I've
explained I won't change the commit message because that would involve
rewriting the commit, but it is useful to have this correction in the
archives.

Regards,
Ian.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] xen-block: support feature-large-sector-size

2019-04-10 Thread Paul Durrant
> -Original Message-
> From: Anthony PERARD [mailto:anthony.per...@citrix.com]
> Sent: 10 April 2019 16:52
> To: Paul Durrant 
> Cc: qemu-de...@nongnu.org; xen-devel@lists.xenproject.org; 
> qemu-bl...@nongnu.org; Stefano Stabellini
> ; Stefan Hajnoczi ; Kevin Wolf 
> ; Max
> Reitz 
> Subject: Re: [PATCH] xen-block: support feature-large-sector-size
> 
> On Tue, Apr 09, 2019 at 05:40:38PM +0100, Paul Durrant wrote:
> > A recent Xen commit [1] clarified the semantics of sector based quantities
> > used in the blkif protocol such that it is now safe to create a xen-block
> > device with a logical_block_size != 512, as long as the device only
> > connects to a frontend advertizing 'feature-large-block-size'.
> >
> > This patch modifies xen-block accordingly. It also uses a stack variable
> > for the BlockBackend in xen_block_realize() to avoid repeated dereferencing
> > of the BlockConf pointer, and changes the parameters of
> > xen_block_dataplane_create() so that the BlockBackend pointer and sector
> > size are passed expicitly rather than implicitly via the BlockConf.
> >
> > These modifications have been tested against a recent Windows PV XENVBD
> > driver [2] using a xen-disk device with a 4kB logical block size.
> >
> > [1] 
> > http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=67e1c050e36b2c9900cca83618e56189effbad98
> > [2] https://winpvdrvbuild.xenproject.org:8080/job/XENVBD-master/126
> >
> > Signed-off-by: Paul Durrant 
> > ---
> > diff --git a/hw/block/xen-block.c b/hw/block/xen-block.c
> > index ef635be4c2..05e890ad78 100644
> > --- a/hw/block/xen-block.c
> > +++ b/hw/block/xen-block.c
> > @@ -51,11 +51,25 @@ static void xen_block_connect(XenDevice *xendev, Error 
> > **errp)
> [...]
> > +if (xen_device_frontend_scanf(xendev, "feature-large-sector-size", 
> > "%u",
> > +  _large_sector_size) != 1) {
> > +feature_large_sector_size = 0;
> > +}
> > +
> > +if (feature_large_sector_size != 1 &&
> > +conf->logical_block_size != XEN_BLKIF_SECTOR_SIZE) {
> > +error_setg(errp, "logical_block_size != %u not supported",
> 
> Maybe add "by frontend" to the error message?

Yes, I'm fine with that addition.

> 
> > +   XEN_BLKIF_SECTOR_SIZE);
> > +return;
> > +}
> > +
> 
> With the question answered:
> Reviewed-by: Anthony PERARD 
> 

Thanks,

  Paul

> Thanks,
> 
> --
> Anthony PERARD

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-4.12] x86/svm: Fix handling of ICEBP intercepts

2019-04-10 Thread Woods, Brian
On 2/1/19 8:49 AM, Andrew Cooper wrote:
> c/s 9338a37d "x86/svm: implement debug events" added support for introspecting
> ICEBP debug exceptions, but didn't account for the fact that
> svm_get_insn_len() (previously __get_instruction_length) can fail and may
> already raise #GP for the guest.
> 
> If svm_get_insn_len() fails, return back to guest context rather than
> continuing and mistaking a trap-style VMExit for a fault-style one.
> 
> Spotted by Coverity.
> 
> Signed-off-by: Andrew Cooper 
> ---
> CC: Jan Beulich 
> CC: Wei Liu 
> CC: Roger Pau Monné 
> CC: Boris Ostrovsky 
> CC: Suravee Suthikulpanit 
> CC: Brian Woods 
> CC: Juergen Gross 
> CC: Razvan Cojocaru 
> CC: Tamas K Lengyel 
> 
> This wants backporting to Xen 4.11
> ---
>   xen/arch/x86/hvm/svm/svm.c | 3 +++
>   1 file changed, 3 insertions(+)
> 
> diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
> index 2584b90..e21091c 100644
> --- a/xen/arch/x86/hvm/svm/svm.c
> +++ b/xen/arch/x86/hvm/svm/svm.c
> @@ -2758,6 +2758,9 @@ void svm_vmexit_handler(struct cpu_user_regs *regs)
>   {
>   trap_type = X86_EVENTTYPE_PRI_SW_EXCEPTION;
>   inst_len = svm_get_insn_len(v, INSTR_ICEBP);
> +
> +if ( !instr_len )
> +break;
>   }
>   
>   rc = hvm_monitor_debug(regs->rip,
> 

Acked-by: Brian Woods 
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 2/3] xen-bus: allow AioContext to be specified for each event channel

2019-04-10 Thread Paul Durrant
> -Original Message-
> From: Anthony PERARD [mailto:anthony.per...@citrix.com]
> Sent: 10 April 2019 16:23
> To: Paul Durrant 
> Cc: qemu-de...@nongnu.org; qemu-bl...@nongnu.org; 
> xen-devel@lists.xenproject.org; Stefano Stabellini
> ; Stefan Hajnoczi ; Kevin Wolf 
> ; Max
> Reitz 
> Subject: Re: [PATCH 2/3] xen-bus: allow AioContext to be specified for each 
> event channel
> 
> On Wed, Apr 10, 2019 at 04:20:05PM +0100, Paul Durrant wrote:
> > > -Original Message-
> > > From: Anthony PERARD [mailto:anthony.per...@citrix.com]
> > > Sent: 10 April 2019 13:57
> > > To: Paul Durrant 
> > > Cc: qemu-de...@nongnu.org; qemu-bl...@nongnu.org; 
> > > xen-devel@lists.xenproject.org; Stefano
> Stabellini
> > > ; Stefan Hajnoczi ; Kevin 
> > > Wolf ;
> Max
> > > Reitz 
> > > Subject: Re: [PATCH 2/3] xen-bus: allow AioContext to be specified for 
> > > each event channel
> > >
> > > On Mon, Apr 08, 2019 at 04:16:16PM +0100, Paul Durrant wrote:
> > > > This patch adds an AioContext parameter to 
> > > > xen_device_bind_event_channel()
> > > > and then uses aio_set_fd_handler() to set the callback rather than
> > > > qemu_set_fd_handler().
> > > >
> > > > Signed-off-by: Paul Durrant 
> > > > ---
> > > > @@ -943,6 +944,7 @@ static void xen_device_event(void *opaque)
> > > >  }
> > > >
> > > >  XenEventChannel *xen_device_bind_event_channel(XenDevice *xendev,
> > > > +   AioContext *ctx,
> > > > unsigned int port,
> > > > XenEventHandler handler,
> > > > void *opaque, Error 
> > > > **errp)
> > > > @@ -968,8 +970,9 @@ XenEventChannel 
> > > > *xen_device_bind_event_channel(XenDevice *xendev,
> > > >  channel->handler = handler;
> > > >  channel->opaque = opaque;
> > > >
> > > > -qemu_set_fd_handler(xenevtchn_fd(channel->xeh), xen_device_event, 
> > > > NULL,
> > > > -channel);
> > > > +channel->ctx = ctx;
> > > > +aio_set_fd_handler(channel->ctx, xenevtchn_fd(channel->xeh), false,
> > > > +   xen_device_event, NULL, NULL, channel);
> > >
> > > I wonder if the `'is_external' parameter of aio_set_fd_handler shoud be
> > > `true' here, instead. That flag seems to be used when making a snapshot
> > > of a blockdev, for example.
> > >
> > > That was introduced by:
> > > dca21ef23ba48f6f1428c59f295a857e5dc203c8^..c07bc2c1658fffeee08eb46402b2f66d55b07586
> > >
> > > What do you think?
> >
> > Interesting. I admit I was merely transcribing what qemu_set_fd_handler() 
> > passes without really
> looking into the values. Looking at the arguments that virtio-blk passes to 
> aio_set_event_notifier()
> though, and what 'is_external' means, it would appear that setting it to true 
> is probably the right
> thing to do. Do you want me to send a v2 of the series or can you fix it up?
> 
> I'll fix that up.

Thanks,

  Paul

> 
> Reviewed-by: Anthony PERARD 
> 
> Thanks,
> 
> --
> Anthony PERARD

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] xen-block: support feature-large-sector-size

2019-04-10 Thread Anthony PERARD
On Tue, Apr 09, 2019 at 05:40:38PM +0100, Paul Durrant wrote:
> A recent Xen commit [1] clarified the semantics of sector based quantities
> used in the blkif protocol such that it is now safe to create a xen-block
> device with a logical_block_size != 512, as long as the device only
> connects to a frontend advertizing 'feature-large-block-size'.
> 
> This patch modifies xen-block accordingly. It also uses a stack variable
> for the BlockBackend in xen_block_realize() to avoid repeated dereferencing
> of the BlockConf pointer, and changes the parameters of
> xen_block_dataplane_create() so that the BlockBackend pointer and sector
> size are passed expicitly rather than implicitly via the BlockConf.
> 
> These modifications have been tested against a recent Windows PV XENVBD
> driver [2] using a xen-disk device with a 4kB logical block size.
> 
> [1] 
> http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=67e1c050e36b2c9900cca83618e56189effbad98
> [2] https://winpvdrvbuild.xenproject.org:8080/job/XENVBD-master/126
> 
> Signed-off-by: Paul Durrant 
> ---
> diff --git a/hw/block/xen-block.c b/hw/block/xen-block.c
> index ef635be4c2..05e890ad78 100644
> --- a/hw/block/xen-block.c
> +++ b/hw/block/xen-block.c
> @@ -51,11 +51,25 @@ static void xen_block_connect(XenDevice *xendev, Error 
> **errp)
[...]
> +if (xen_device_frontend_scanf(xendev, "feature-large-sector-size", "%u",
> +  _large_sector_size) != 1) {
> +feature_large_sector_size = 0;
> +}
> +
> +if (feature_large_sector_size != 1 &&
> +conf->logical_block_size != XEN_BLKIF_SECTOR_SIZE) {
> +error_setg(errp, "logical_block_size != %u not supported",

Maybe add "by frontend" to the error message?

> +   XEN_BLKIF_SECTOR_SIZE);
> +return;
> +}
> +

With the question answered:
Reviewed-by: Anthony PERARD 

Thanks,

-- 
Anthony PERARD

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] Ping SVM: [PATCH for-4.12] x86/svm: Fix handling of ICEBP intercepts

2019-04-10 Thread Andrew Cooper
The discovery of further cascade errors doesn't change the need for this
patch.

Its obviously missed the boat for 4.12, but still needed.

I haven't reposed with the typo which Tamas fixed, but will fold that in
on commit.  (I didn't think it worthy of sending out a v2, given the
obviousness of the fix.)

~Andrew

On 01/02/2019 14:49, Andrew Cooper wrote:
> c/s 9338a37d "x86/svm: implement debug events" added support for introspecting
> ICEBP debug exceptions, but didn't account for the fact that
> svm_get_insn_len() (previously __get_instruction_length) can fail and may
> already raise #GP for the guest.
>
> If svm_get_insn_len() fails, return back to guest context rather than
> continuing and mistaking a trap-style VMExit for a fault-style one.
>
> Spotted by Coverity.
>
> Signed-off-by: Andrew Cooper 
> ---
> CC: Jan Beulich 
> CC: Wei Liu 
> CC: Roger Pau Monné 
> CC: Boris Ostrovsky 
> CC: Suravee Suthikulpanit 
> CC: Brian Woods 
> CC: Juergen Gross 
> CC: Razvan Cojocaru 
> CC: Tamas K Lengyel 
>
> This wants backporting to Xen 4.11
> ---
>  xen/arch/x86/hvm/svm/svm.c | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
> index 2584b90..e21091c 100644
> --- a/xen/arch/x86/hvm/svm/svm.c
> +++ b/xen/arch/x86/hvm/svm/svm.c
> @@ -2758,6 +2758,9 @@ void svm_vmexit_handler(struct cpu_user_regs *regs)
>  {
>  trap_type = X86_EVENTTYPE_PRI_SW_EXCEPTION;
>  inst_len = svm_get_insn_len(v, INSTR_ICEBP);
> +
> +if ( !instr_len )
> +break;
>  }
>  
>  rc = hvm_monitor_debug(regs->rip,


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [OSSTEST PATCH 13/62] Extend workaround `clk_ignore_unused' to stretch

2019-04-10 Thread Julien Grall

Hi,

On 10/04/2019 15:23, Ian Jackson wrote:

From: Wei Liu 

This is https://bugs.xenproject.org/xen/bug/45

Still no resolution for this one :/.



Without that parameter we lose uart output.

Signed-off-by: Wei Liu 
CC: Stefano Stabellini 
CC: Julien Grall 


Acked-by: Julien Grall 

Cheers,


---
  Osstest/Debian.pm | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 82b5fb40..91bffdff 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -241,7 +241,7 @@ END
push @xenkopt, $xenkopt;
# http://bugs.xenproject.org/xen/bug/45
push @xenkopt, "clk_ignore_unused"
-   if $ho->{Suite} =~ m/wheezy|jessie/;
+   if $ho->{Suite} =~ m/wheezy|jessie|stretch/;
  
  	$xenkopt = join ' ', @xenkopt;

logm("Dom0 Linux options: $xenkopt");



--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [OSSTEST PATCH 21/62] ts-kernel-build: disable host1x, which doesn't build

2019-04-10 Thread Julien Grall

Hi,

On 10/04/2019 16:27, Thierry Reding wrote:

On Wed, Apr 10, 2019 at 03:23:27PM +0100, Ian Jackson wrote:

From: Wei Liu 

Empirically, on stretch armhf:

   drivers/gpu/host1x/cdma.c: In function `host1x_pushbuffer_init':
   drivers/gpu/host1x/cdma.c:94:48: error: passing argument 3 of `dma_alloc_wc' 
from incompatible pointer type [-Werror=incompatible-pointer-types]
  pb->mapped = dma_alloc_wc(host1x->dev, size, >phys,
  ^
etc.


This was fixed in v4.18 by this commit:

commit 2f8a6da866eff746a9f8c7745790f3765baeb589
Author: Emil Goode 
Date:   Wed May 16 12:22:04 2018 +0200

gpu: host1x: Fix compiler errors by converting to dma_addr_t

The compiler is complaining with the following errors:

drivers/gpu/host1x/cdma.c:94:48: error:
passing argument 3 of ‘dma_alloc_wc’ from incompatible 
pointer type
[-Werror=incompatible-pointer-types]

drivers/gpu/host1x/cdma.c:113:48: error:
passing argument 3 of ‘dma_alloc_wc’ from incompatible 
pointer type
[-Werror=incompatible-pointer-types]

The expected pointer type of the third argument to dma_alloc_wc() is
dma_addr_t but phys_addr_t is passed.

Change the phys member of struct push_buffer to be dma_addr_t so 
that we
pass the correct type to dma_alloc_wc().
Also check pb->mapped for non-NULL in the destroy function as that 
is the
right way of checking if dma_alloc_wc() was successful.

Signed-off-by: Emil Goode 
Signed-off-by: Thierry Reding 

It should be fairly easy to backport this to older releases, though I'm
not sure exactly what made this trigger. This wasn't causing any build
errors for a very long time, since this type mismatch has existed ever
since the driver was merged all the way back in v3.10.


Likely no-one ever tried to build this module with CONFIG_XEN=y. Xen requires 
dma_addr_t to be 64-bit regardless the page-tables format selected (e.g short vs 
LPAE).


AFAIK, this is the only case on Arm32 where phys_addr_t and dma_addr_t would 
mismatched.


Ian are we using a different config between Jessie and Stretch?

Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 2/3] xen-bus: allow AioContext to be specified for each event channel

2019-04-10 Thread Paul Durrant
> -Original Message-
> From: Anthony PERARD [mailto:anthony.per...@citrix.com]
> Sent: 10 April 2019 13:57
> To: Paul Durrant 
> Cc: qemu-de...@nongnu.org; qemu-bl...@nongnu.org; 
> xen-devel@lists.xenproject.org; Stefano Stabellini
> ; Stefan Hajnoczi ; Kevin Wolf 
> ; Max
> Reitz 
> Subject: Re: [PATCH 2/3] xen-bus: allow AioContext to be specified for each 
> event channel
> 
> On Mon, Apr 08, 2019 at 04:16:16PM +0100, Paul Durrant wrote:
> > This patch adds an AioContext parameter to xen_device_bind_event_channel()
> > and then uses aio_set_fd_handler() to set the callback rather than
> > qemu_set_fd_handler().
> >
> > Signed-off-by: Paul Durrant 
> > ---
> > @@ -943,6 +944,7 @@ static void xen_device_event(void *opaque)
> >  }
> >
> >  XenEventChannel *xen_device_bind_event_channel(XenDevice *xendev,
> > +   AioContext *ctx,
> > unsigned int port,
> > XenEventHandler handler,
> > void *opaque, Error **errp)
> > @@ -968,8 +970,9 @@ XenEventChannel 
> > *xen_device_bind_event_channel(XenDevice *xendev,
> >  channel->handler = handler;
> >  channel->opaque = opaque;
> >
> > -qemu_set_fd_handler(xenevtchn_fd(channel->xeh), xen_device_event, NULL,
> > -channel);
> > +channel->ctx = ctx;
> > +aio_set_fd_handler(channel->ctx, xenevtchn_fd(channel->xeh), false,
> > +   xen_device_event, NULL, NULL, channel);
> 
> I wonder if the `'is_external' parameter of aio_set_fd_handler shoud be
> `true' here, instead. That flag seems to be used when making a snapshot
> of a blockdev, for example.
> 
> That was introduced by:
> dca21ef23ba48f6f1428c59f295a857e5dc203c8^..c07bc2c1658fffeee08eb46402b2f66d55b07586
> 
> What do you think?

Interesting. I admit I was merely transcribing what qemu_set_fd_handler() 
passes without really looking into the values. Looking at the arguments that 
virtio-blk passes to aio_set_event_notifier() though, and what 'is_external' 
means, it would appear that setting it to true is probably the right thing to 
do. Do you want me to send a v2 of the series or can you fix it up?

  Cheers,

Paul

> 
> 
> --
> Anthony PERARD

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [OSSTEST PATCH 21/62] ts-kernel-build: disable host1x, which doesn't build

2019-04-10 Thread Thierry Reding
On Wed, Apr 10, 2019 at 03:23:27PM +0100, Ian Jackson wrote:
> From: Wei Liu 
> 
> Empirically, on stretch armhf:
> 
>   drivers/gpu/host1x/cdma.c: In function `host1x_pushbuffer_init':
>   drivers/gpu/host1x/cdma.c:94:48: error: passing argument 3 of 
> `dma_alloc_wc' from incompatible pointer type 
> [-Werror=incompatible-pointer-types]
>  pb->mapped = dma_alloc_wc(host1x->dev, size, >phys,
> ^
> etc.

This was fixed in v4.18 by this commit:

commit 2f8a6da866eff746a9f8c7745790f3765baeb589
Author: Emil Goode 
Date:   Wed May 16 12:22:04 2018 +0200

gpu: host1x: Fix compiler errors by converting to dma_addr_t

The compiler is complaining with the following errors:

drivers/gpu/host1x/cdma.c:94:48: error:
passing argument 3 of ‘dma_alloc_wc’ from incompatible 
pointer type
[-Werror=incompatible-pointer-types]

drivers/gpu/host1x/cdma.c:113:48: error:
passing argument 3 of ‘dma_alloc_wc’ from incompatible 
pointer type
[-Werror=incompatible-pointer-types]

The expected pointer type of the third argument to dma_alloc_wc() is
dma_addr_t but phys_addr_t is passed.

Change the phys member of struct push_buffer to be dma_addr_t so 
that we
pass the correct type to dma_alloc_wc().
Also check pb->mapped for non-NULL in the destroy function as that 
is the
right way of checking if dma_alloc_wc() was successful.

Signed-off-by: Emil Goode 
Signed-off-by: Thierry Reding 

It should be fairly easy to backport this to older releases, though I'm
not sure exactly what made this trigger. This wasn't causing any build
errors for a very long time, since this type mismatch has existed ever
since the driver was merged all the way back in v3.10.

Thierry

> This is blocking the upgrade of the Xen Project CI to Debian stretch
> so disable it for now.
> 
> Signed-off-by: Wei Liu 
> CC: Julien Grall 
> CC: Stefano Stabellini 
> CC: Thierry Reding 
> CC: dri-de...@lists.freedesktop.org
> ---
>  ts-kernel-build | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/ts-kernel-build b/ts-kernel-build
> index 21b8f78a..0bc443de 100755
> --- a/ts-kernel-build
> +++ b/ts-kernel-build
> @@ -594,6 +594,9 @@ case ${XEN_TARGET_ARCH} in
>  *) ;;
>  esac
>  
> +# Disable components that don't build
> +setopt CONFIG_TEGRA_HOST1X n
> +
>  exit 0
>  END
>  }
> -- 
> 2.11.0
> 


signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 2/3] xen-bus: allow AioContext to be specified for each event channel

2019-04-10 Thread Anthony PERARD
On Wed, Apr 10, 2019 at 04:20:05PM +0100, Paul Durrant wrote:
> > -Original Message-
> > From: Anthony PERARD [mailto:anthony.per...@citrix.com]
> > Sent: 10 April 2019 13:57
> > To: Paul Durrant 
> > Cc: qemu-de...@nongnu.org; qemu-bl...@nongnu.org; 
> > xen-devel@lists.xenproject.org; Stefano Stabellini
> > ; Stefan Hajnoczi ; Kevin Wolf 
> > ; Max
> > Reitz 
> > Subject: Re: [PATCH 2/3] xen-bus: allow AioContext to be specified for each 
> > event channel
> > 
> > On Mon, Apr 08, 2019 at 04:16:16PM +0100, Paul Durrant wrote:
> > > This patch adds an AioContext parameter to xen_device_bind_event_channel()
> > > and then uses aio_set_fd_handler() to set the callback rather than
> > > qemu_set_fd_handler().
> > >
> > > Signed-off-by: Paul Durrant 
> > > ---
> > > @@ -943,6 +944,7 @@ static void xen_device_event(void *opaque)
> > >  }
> > >
> > >  XenEventChannel *xen_device_bind_event_channel(XenDevice *xendev,
> > > +   AioContext *ctx,
> > > unsigned int port,
> > > XenEventHandler handler,
> > > void *opaque, Error 
> > > **errp)
> > > @@ -968,8 +970,9 @@ XenEventChannel 
> > > *xen_device_bind_event_channel(XenDevice *xendev,
> > >  channel->handler = handler;
> > >  channel->opaque = opaque;
> > >
> > > -qemu_set_fd_handler(xenevtchn_fd(channel->xeh), xen_device_event, 
> > > NULL,
> > > -channel);
> > > +channel->ctx = ctx;
> > > +aio_set_fd_handler(channel->ctx, xenevtchn_fd(channel->xeh), false,
> > > +   xen_device_event, NULL, NULL, channel);
> > 
> > I wonder if the `'is_external' parameter of aio_set_fd_handler shoud be
> > `true' here, instead. That flag seems to be used when making a snapshot
> > of a blockdev, for example.
> > 
> > That was introduced by:
> > dca21ef23ba48f6f1428c59f295a857e5dc203c8^..c07bc2c1658fffeee08eb46402b2f66d55b07586
> > 
> > What do you think?
> 
> Interesting. I admit I was merely transcribing what qemu_set_fd_handler() 
> passes without really looking into the values. Looking at the arguments that 
> virtio-blk passes to aio_set_event_notifier() though, and what 'is_external' 
> means, it would appear that setting it to true is probably the right thing to 
> do. Do you want me to send a v2 of the series or can you fix it up?

I'll fix that up.

Reviewed-by: Anthony PERARD 

Thanks,

-- 
Anthony PERARD

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 3/3] xen-bus / xen-block: add support for event channel polling

2019-04-10 Thread Anthony PERARD
On Mon, Apr 08, 2019 at 04:16:17PM +0100, Paul Durrant wrote:
> This patch introduces a poll callback for event channel fd-s and uses
> this to invoke the channel callback function.
> 
> To properly support polling, it is necessary for the event channel callback
> function to return a boolean saying whether it has done any useful work or
> not. Thus xen_block_dataplane_event() is modified to directly invoke
> xen_block_handle_requests() and the latter only returns true if it actually
> processes any requests. This also means that the call to qemu_bh_schedule()
> is moved into xen_block_complete_aio(), which is more intuitive since the
> only reason for doing a deferred poll of the shared ring should be because
> there were previously insufficient resources to fully complete a previous
> poll.
> 
> Signed-off-by: Paul Durrant 
> ---

Reviewed-by: Anthony PERARD 

Thanks,

-- 
Anthony PERARD

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-unstable-smoke test] 134597: regressions - trouble: blocked/broken/fail/pass

2019-04-10 Thread osstest service owner
flight 134597 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/134597/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm  broken
 build-arm64-xsm   4 host-install(4)broken REGR. vs. 133991
 build-amd64   6 xen-buildfail REGR. vs. 133991
 test-armhf-armhf-xl  12 guest-start  fail REGR. vs. 133991

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-debianhvm-i386  1 build-check(1) blocked n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a

version targeted for testing:
 xen  8228577ad1ba9f4b49370b76c90b75fb9243ee2f
baseline version:
 xen  cb70a26f78848fe45f593f7ebc9cfaac760a791b

Last test of basis   133991  2019-03-22 15:00:46 Z   19 days
Failing since134068  2019-03-25 12:00:51 Z   16 days   71 attempts
Testing same since   134590  2019-04-09 22:00:37 Z0 days3 attempts


People who touched revisions under test:
  Amit Singh Tomar 
  Andrew Cooper 
  Anthony PERARD 
  Brian Woods 
  Christian Lindig 
  Doug Goldstein 
  George Dunlap 
  Igor Druzhinin 
  Jan Beulich 
  Juergen Gross 
  Julien Grall 
  Kevin Tian 
  Lukas Juenger 
  M A Young 
  Norbert Manthey 
  Oleksandr Andrushchenko 
  Paul Durrant 
  Pawel Wieczorkiewicz 
  Petre Pircalabu 
  Razvan Cojocaru 
  Tamas K Lengyel 
  Wei Liu 
  Xiaochen Wang 
  Xin Li 

jobs:
 build-arm64-xsm  broken  
 build-amd64  fail
 build-armhf  pass
 build-amd64-libvirt  blocked 
 test-armhf-armhf-xl  fail
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-i386 blocked 
 test-amd64-amd64-libvirt blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-arm64-xsm broken
broken-step build-arm64-xsm host-install(4)

Not pushing.

(No revision log; it would be 1980 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v1] libxl: fix migration of PV and PVH domUs with and without qemu

2019-04-10 Thread Olaf Hering
Am Wed, 10 Apr 2019 12:33:38 +0200
schrieb Olaf Hering :

>  goto error_out;

This one should have been removed, now libxl fails right away.
Will do more runtime testing.

Waiting for feedback before sending v2.

Olaf


pgpVcn1cnjcYW.pgp
Description: Digitale Signatur von OpenPGP
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [OSSTEST PATCH 55/62] make-flight: shadow test: Disable kpti in guests

2019-04-10 Thread Andrew Cooper
On 10/04/2019 15:24, Ian Jackson wrote:
> Since Spectre/Meltdown, shadow has been a lot slower, especially with
> KPTI in the guest.  Empirically, too slow (with the kernel from Debian
> stretch).

The speed of shadow pagetables hasn't changed - I don't think we even
touched the shadow code at all for XSA-254.

The problem is the change in guest behaviour as a consequence of needing
KPTI for a Meltdown mitigation.

The guest now flushes its pagetables on every
syscall/interrupt/exception rather than once on a process=>process
context switch, which is why running a guest using KPTI in shadow mode
is boarderline unusable.

The actual change to use nopti looks fine.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 04/62] ts-xen-build-prep: install packages for suites >jessie

2019-04-10 Thread Ian Jackson
From: Wei Liu 

The stubdom build needs texinfo.
The libvirt build needs autopoint.
The QEMU build needs libpciaccess-dev.

Signed-off-by: Wei Liu 
Acked-off-by: Ian Jackson 
---
 ts-xen-build-prep | 4 
 1 file changed, 4 insertions(+)

diff --git a/ts-xen-build-prep b/ts-xen-build-prep
index a15ab3df..ca5735a1 100755
--- a/ts-xen-build-prep
+++ b/ts-xen-build-prep
@@ -222,6 +222,10 @@ sub prep () {
# jessie (>jessie?)
push(@packages, "libnl-route-3-dev");
 }
+if ($ho->{Suite} !~ m/squeeze|wheezy|jessie/) {
+push(@packages, qw(texinfo autopoint libpciaccess-dev));
+}
+
 target_install_packages($ho, @packages);
 target_cmd_root($ho, "chmod -R a+r /usr/share/git-core/templates");
 # workaround for Debian #595728
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 31/62] persistent-net: Set net.ifnames=0 in di_installcmdline_core

2019-04-10 Thread Ian Jackson
Signed-off-by: Wei Liu 
---
 Osstest/Debian.pm | 5 +
 1 file changed, 5 insertions(+)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 6309b246..aff5acd5 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -81,6 +81,11 @@ sub debian_boot_setup ($;$) {
 $kopt .= ' '.$targkopt;
 }
 
+# 
https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/
+# In fact these are anything but predictable.  We use the scheme
+# from Debian jessie and earlier, persistent-net-generator etc.
+$kopt .= ' net.ifnames=0';
+
 foreach my $hook ($hooks ? @$hooks : ()) {
 my $bo_hook= $hook->{EditBootOptions};
 $bo_hook->($ho, \$xenhopt, \$kopt) if $bo_hook;
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 28/62] ts-host-install: Unconditionally mkdir -p /etc/udev/rules.d

2019-04-10 Thread Ian Jackson
We are going to want this directory to exist so that we can put a
canary in 70-persistent-net.rules.

In the cases where the behaviour of osstest changes, the empty
directory does not result in any overall change.

Signed-off-by: Ian Jackson 
---
 ts-host-install | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/ts-host-install b/ts-host-install
index f80a151c..7423eb9b 100755
--- a/ts-host-install
+++ b/ts-host-install
@@ -198,6 +198,8 @@ sub setup_netboot_firstboot($) {
 my $persistent_net_rules =
"$initrd_overlay.d/etc/udev/rules.d/70-persistent-net.rules";
 
+system_checked(qw(mkdir -p --), "$initrd_overlay.d/etc/udev/rules.d");
+
 my $ipappend = 2;
 my $wantphysif= get_host_property($ho,'interface force','auto');
 logm("Forcing interface $wantphysif");
@@ -205,7 +207,6 @@ sub setup_netboot_firstboot($) {
$ipappend = 0;
die "need Ether for $ho->{Name} ($wantphysif)"
unless defined $ho->{Ether};
-system_checked(qw(mkdir -p --), "$initrd_overlay.d/etc/udev/rules.d");
# ip(8) moved to /sbin in Jessie
my $ipcmd = $ho->{Suite} =~ m/wheezy/ ? "/bin/ip" : "/sbin/ip";
 file_simple_write_contents
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 14/62] adjust how to skip bootloader installation for arm32, in Stretch

2019-04-10 Thread Ian Jackson
From: Wei Liu 

Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 
---
v4: Change case of suite names in comment.
---
 Osstest/Debian.pm | 4 
 1 file changed, 4 insertions(+)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 91bffdff..85b1890d 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1041,7 +1041,11 @@ END
# actually does what we need, but the installer doesn't treat
# that as a "bootloader".
$preseed_file.= (

[Xen-devel] [OSSTEST PATCH 62/62] Switch to Debian Stretch

2019-04-10 Thread Ian Jackson
From: Wei Liu 

Signed-off-by: Wei Liu 
Signed-off-by: Ian Jackson 
---
 Osstest.pm| 2 +-
 production-config | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/Osstest.pm b/Osstest.pm
index 92b1a0ea..7ce53fcb 100644
--- a/Osstest.pm
+++ b/Osstest.pm
@@ -87,7 +87,7 @@ our %c = qw(
 
 Images images
 
-DebianSuite jessie
+DebianSuite stretch
 DebianMirrorSubpath debian
 
 TestHostKeypairPath id_rsa_osstest
diff --git a/production-config b/production-config
index fadfe8b9..5b9f5c82 100644
--- a/production-config
+++ b/production-config
@@ -90,12 +90,14 @@ TftpNetbootGroup osstest
 # Update with ./mg-debian-installer-update(-all)
 TftpDiVersion_wheezy 2016-06-08
 TftpDiVersion_jessie 2018-06-26
+TftpDiVersion_stretch 2018-11-27
 
 DebianSnapshotBackports_jessie 
http://snapshot.debian.org/archive/debian/20190206T211314Z/
 
 # For ISO installs
 DebianImageVersion_wheezy 7.2.0
 DebianImageVersion_jessie 8.2.0
+DebianImageVersion_stretch 9.4.0
 
 # These should normally be the same.
 # Update with ./mg-cpu-microcode-update
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 47/62] Debian: Add reference to bug numbers for erase-other-disks

2019-04-10 Thread Ian Jackson
Signed-off-by: Ian Jackson 
---
 Osstest/Debian.pm | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 073b776c..79b7960d 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1171,6 +1171,7 @@ END2
 chmod +x parted_devices
 END
 
+# Work around Debian bugs #601363 #687160
 preseed_hook_installscript($ho, $sfx,
   '/lib/partman/init.d', '25erase-other-disks', 

[Xen-devel] [OSSTEST PATCH 39/62] preseed_base: chmod ssh host private keys to placate sshd

2019-04-10 Thread Ian Jackson
Otherwise:
  Could not load host key: /etc/ssh/ssh_host_ecdsa_key
  @@@
  @ WARNING: UNPROTECTED PRIVATE KEY FILE!  @
  @@@
  Permissions 0640 for '/etc/ssh/ssh_host_ed25519_key' are too open.
  It is recommended that your private key files are NOT accessible by others.
  This private key will be ignored.
  key_load_private: bad permissions
  Could not load host key: /etc/ssh/ssh_host_ed25519_key

This seems to start happening with stretch.  Presumably stretch is
more annoyingly picky than jessie.

Signed-off-by: Ian Jackson 
---
 Osstest/Debian.pm | 8 
 1 file changed, 8 insertions(+)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index aff5acd5..d76dd03d 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -871,6 +871,14 @@ END
preseed_hook_overlay($ho, $sfx, $srcdir, $tfilename);
 });
 
+# Host private keys in the overlays have to be group-readable
+# at least, or no-one can use them.  But ssh is very fussy.
+preseed_hook_command($ho, 'late_command', $sfx, 

[Xen-devel] [OSSTEST PATCH 21/62] ts-kernel-build: disable host1x, which doesn't build

2019-04-10 Thread Ian Jackson
From: Wei Liu 

Empirically, on stretch armhf:

  drivers/gpu/host1x/cdma.c: In function `host1x_pushbuffer_init':
  drivers/gpu/host1x/cdma.c:94:48: error: passing argument 3 of `dma_alloc_wc' 
from incompatible pointer type [-Werror=incompatible-pointer-types]
 pb->mapped = dma_alloc_wc(host1x->dev, size, >phys,
  ^
etc.

This is blocking the upgrade of the Xen Project CI to Debian stretch
so disable it for now.

Signed-off-by: Wei Liu 
CC: Julien Grall 
CC: Stefano Stabellini 
CC: Thierry Reding 
CC: dri-de...@lists.freedesktop.org
---
 ts-kernel-build | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/ts-kernel-build b/ts-kernel-build
index 21b8f78a..0bc443de 100755
--- a/ts-kernel-build
+++ b/ts-kernel-build
@@ -594,6 +594,9 @@ case ${XEN_TARGET_ARCH} in
 *) ;;
 esac
 
+# Disable components that don't build
+setopt CONFIG_TEGRA_HOST1X n
+
 exit 0
 END
 }
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 36/62] TestSupport: target_somefile_leaf rename and change a variable

2019-04-10 Thread Ian Jackson
Rename this function.  `getleaf' contains `get' which makes it sound
like the function copies something, or returns answers suitable for
getting, or something.

Also rename `$rdest' to `$rfile' since it might be a source too.
(Although we are not about to make it a source...)

Signed-off-by: Ian Jackson 
---
 Osstest/TestSupport.pm | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 4e2f120a..a5870e4d 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -504,10 +504,10 @@ sub remote_perl_script_done ($) {
 !$? or die "$thing->{What} $?";
 }
 
-sub target_somefile_getleaf ($$$) {
-my ($lleaf_ref, $rdest, $ho) = @_;
+sub target_somefile_leaf ($$$) {
+my ($lleaf_ref, $rfile, $ho) = @_;
 if (!defined $$lleaf_ref) {
-$$lleaf_ref= $rdest;
+$$lleaf_ref= $rfile;
 $$lleaf_ref =~ s,.*/,,;
 }
 $$lleaf_ref= hostnamepath($ho)."--$$lleaf_ref";
@@ -602,7 +602,7 @@ sub target_install_packages_norec ($@) {
 
 sub tpfcs_core {
 my ($tputfilef,$ho,$timeout,$filedata, $rdest,$lleaf) = @_;
-target_somefile_getleaf(\$lleaf,$rdest,$ho);
+target_somefile_leaf(\$lleaf,$rdest,$ho);
 
 my $h= new IO::File "$stash/$lleaf", 'w' or die "$lleaf $!";
 print $h $filedata or die $!;
@@ -654,7 +654,7 @@ sub teditfileex {
 if (!defined $rdest) {
 $rdest= $rfile;
 }
-target_somefile_getleaf(\$lleaf,$rdest,$ho);
+target_somefile_leaf(\$lleaf,$rdest,$ho);
 my $lfile;
 
 for (;;) {
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [edk2-devel] [PATCH v2 02/31] OvmfPkg: Create platform XenOvmf

2019-04-10 Thread Laszlo Ersek
On 04/10/19 11:48, Jordan Justen wrote:
> On 2019-04-09 04:08:15, Anthony PERARD wrote:
>> This is a copy of OvmfX64, removing VirtIO and some SMM.
>>
>> This new platform will be changed to make it works on two types of Xen
>> guest: HVM and PVH.
>>
>> Compare to OvmfX64, this patch:
>>
>> - changed: PLATFORM_GUID, OUTPUT_DIRECTORY, FLASH_DEFINITION
>> - removed: VirtioLib class resolution
>> - removed: all UEFI_DRIVER modules for virtio devices
>> - removed: DXE_SMM_DRIVER and SMM_CORE lib class resolutions
>> - removed: DXE_SMM_DRIVER and SMM_CORE FDF rules
>> - removed: Everything related to SMM_REQUIRE==true
>> - removed: Everything related to SECURE_BOOT_ENABLE==true
>> - removed: Everything related to TPM2_ENABLE==true
>> - changed: PcdPciDisableBusEnumeration dynamic default flipped to TRUE
>> - changed: default FD_SIZE_IN_KB to 2M.
>>
>> Contributed-under: TianoCore Contribution Agreement 1.1
>> Signed-off-by: Anthony PERARD 
>> ---
>>  OvmfPkg/{OvmfPkgX64.dsc => XenOvmf.dsc} | 202 +---
> 
> I guess we all want our name to be first :), but OvmfXen seems more
> like the pattern that edk2 uses when making sub-platforms.
> 
> Also, in some cases we've used !ifdef type things to keep dsc and fdf
> files common. Would that not be workable here? Maybe it would get too
> ugly. :\

I've been happy with a similar Xen<->QEMU split under ArmVirtPkg.
Duplications in updates are usually not a huge burden, and keeping the
files separate has proved very helpful for determining
maintainer/reviewer/tester responsibility.

> It could help to prevent having to sync dsc changes across, and
> prevent changes from being omitted for Xen on accident.

True, but in my experience that's been the smaller problem. The larger
problem has been modifying Xen stuff in unintended ways (= regressing
Xen), because we can't test (or don't even notice) the Xen-side
implications of changes made to common source.

Personally I prefer another (DSC + FDF) pair under OvmfPkg (beyond the
three we already have) to another layer of (conditional?) includes. The
!if's that are being eliminated in this Xen-customized copy (i.e., in
this patch) are complex enough already :)

... It's not that I *generally* prefer duplication; as you say, we do
have a number of !includes already. I do prefer duplication specifically
for Xen vs. QEMU however.

Thanks
Laszlo

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#38814): https://edk2.groups.io/g/devel/message/38814
Mute This Topic: https://groups.io/mt/30997887/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[Xen-devel] [OSSTEST PATCH 53/62] ts-xen-install: Install libpciaccess0

2019-04-10 Thread Ian Jackson
In "ts-xen-build-prep: install packages for suites >jessie",
libpciaccess-dev was added for the benefit of qemu.  libvirt needs it
too.

We also need the runtime library.  Without it, libvirt does not start:

  2019-04-04 22:35:36.760+: 3623: error : virModuleLoadFile:53 : internal 
error: Failed to load module 
'/usr/local/lib/libvirt/connection-driver/libvirt_driver_nodedev.so': 
libpciaccess.so.0: cannot open shared object file: No such file or directory

Signed-off-by: Ian Jackson 
---
 ts-xen-install | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/ts-xen-install b/ts-xen-install
index 03f8c03e..9f78a75f 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -53,7 +53,8 @@ sub packages () {
   qemu-utils
netcat-openbsd));
 if ($ho->{Suite} !~ m/squeeze|wheezy|jessie/) {
-target_install_packages($ho, qw(net-tools libnl-route-3-200));
+target_install_packages($ho, qw(net-tools libnl-route-3-200 
+   libpciaccess0));
 }
 if ($ho->{Suite} =~ m/jessie/) {
 target_install_packages($ho, qw(libnl-route-3-200));
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 59/62] Debian: Fix /lib/udev/ifupdown-hotplug to not run if / is ro

2019-04-10 Thread Ian Jackson
Empirically, without this, on Debian stretch:

 1. udev starts before / is remounted rw
 2. udev spots eth0 and runs /lib/udev/ifupdown-hotplug
 3. ifupdown-hotplug runs ifup which runs isc-dhcp-client
 4. isc-dhcp-client talks to the dhcp server to get an address
 5. isc-dhcp-client tries to write the lease and run the hook
scripts but something here fails with EROFS
 6. isc-dhcp-client sends DHCPDECLINE to the server
 7. GOTO 4

This loop continues for several seconds, until / is remounted rw.

None of this is appears in any of the guest's logs because syslog is
not running yet, and none of this stuff goes to the console either.
But it does appear in the dhcp *server* logs so that a conscientious
administrator will suffer consternation and concern.

It is not ever sensible to run /lib/udev/ifupdown-hotplug with /
mounted ro.  (Maybe it is not sensible to run udev so early.)
Skipping it is fine because the boot sequence contains an explicit
call to ifup which occurs *after* / is remounted, and that will
collect any interfaces which were skipped earlier.

In my osstest stretch series development tests I don't think I saw any
actual host install failures due to this situation.  The timeouts are
generous enough not to matter, and of course when we install Xen we
reconfigure the host networking to have a static IP address so then
the problem goes away.

In this patch we do this for the host.  We provide a function to
return the appropriate rune which we will use in a moment.

I have not yet reported this situation to the appropriate Debian
channels.  That's on my backlog.  But in any case I have limited the
workaround to stretch so we will revisit this for buster.

The approach to fixing this is somewhat awkward.

Firstly, since the bug is in /lib/udev/ifupdown-hotplug we want to
edit that script.  But we need to do it in the installer environment
as a late_command, because after first boot, via ssh, is too late.
The installer environment has no `patch'.  I didn't want to just
supply a whole new script.  So instead we use sed and mv.

Secondly, as for the contents of /lib/udev/ifupdown-hotplug: I wasn't
able to think of a convenient shell command which will tell us the
errno value from trying to write a file.  Plenty will print the
strerror but I balked at LC_MESSAGES=C and grep.  Perl, however, can
do this, and is always available on Debian.  So perl it is.

Thirdly, the code has a bad case of toothpicks (lots of \), because it
needs to pass through (i) perl (ii) the shell (iii) sed's regexp
syntax and/or i command.

Signed-off-by: Ian Jackson 
---
 Osstest/Debian.pm | 22 ++
 1 file changed, 22 insertions(+)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 9d7d1518..41aa28b0 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -48,6 +48,7 @@ BEGIN {
   preseed_hook_cmds
   di_installcmdline_core
   di_vg_name
+  debian_dhcp_rofs_fix
   );
 %EXPORT_TAGS = ( );
 
@@ -931,6 +932,9 @@ END
 set -ex
 END
 
+preseed_hook_command($ho, 'late_command', $sfx,
+debian_dhcp_rofs_fix($ho, '/target'));
+
 my $preseed = <<"END";
 d-i debian-installer/locale string en_GB
 d-i console-keymaps-at/keymap select gb
@@ -1563,4 +1567,22 @@ sub debian_guest_di_version ($) {
 return $gho->{DiVersion};
 }
 
+sub debian_dhcp_rofs_fix ($$) {
+my ($ho, $rootdir) = @_;
+# Works around bug where /lib/udev/ifupdown-hotplug runs while
+# / is still ro.  In stretch, the isc dhcp client spins requesting
+# an address and then sending a DHCPDECLINE (and then, usually,
+# eventually works).
+return '' unless $ho->{Suite} =~ m/stretch/;
+my $script = "$rootdir/lib/udev/ifupdown-hotplug";
+<'$script.new' \\
+END
+'/^if \[ -d \/run\/systemd\/system ]; then/ i perl -e '\''use POSIX; my 
$f="/var/lib/dhcp/rw-fs-check"; exit 0 if open T, ">", $f; die "quitting: $f: 
$!\\n" if $!==EROFS; warn "warning: $f: $!\\n"'\'' || exit 0'
+ENDQ
+mv '$script.new' '$script'
+END
+}
+
 1;
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 55/62] make-flight: shadow test: Disable kpti in guests

2019-04-10 Thread Ian Jackson
Since Spectre/Meltdown, shadow has been a lot slower, especially with
KPTI in the guest.  Empirically, too slow (with the kernel from Debian
stretch).

CC: Andrew Cooper 
Signed-off-by: Ian Jackson 
---
 make-flight | 1 +
 1 file changed, 1 insertion(+)

diff --git a/make-flight b/make-flight
index 151b1435..1e3ebd5c 100755
--- a/make-flight
+++ b/make-flight
@@ -493,6 +493,7 @@ do_shadow_tests () {
 $debian_runvars all_hostflags=$most_hostflags
 
   do_hvm_debian_test_one debianhvm xl seabios false '' -shadow \
+ all_guest_linux_boot_append=nopti \
  "xen_boot_append=hap=false"
 }
 
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 58/62] dm restrict, fishdescriptor: Update to a fixed chiark-scripts

2019-04-10 Thread Ian Jackson
I have only just fixed a bug which stops our test from working
and the fix is not upstream yet.

Signed-off-by: Ian Jackson 
---
 production-config | 1 +
 1 file changed, 1 insertion(+)

diff --git a/production-config b/production-config
index 59c74cca..fadfe8b9 100644
--- a/production-config
+++ b/production-config
@@ -106,6 +106,7 @@ MicrocodeUpdateI386 microcode.x86.2015-06-12.cpio
 TftpGrubVersion -XX-XX
 
 DebianExtraPackages_jessie chiark-scripts_6.0.3~citrix1_all.deb
+DebianExtraPackages_stretch chiark-scripts_6.0.4~citrix1_all.deb
 
 DebianExtraPackages_uefi_i386_jessie   extradebs-uefi-i386-2018-04-01/
 DebianExtraPackages_uefi_amd64_jessie  extradebs-uefi-amd64-2018-04-01/
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 57/62] platforms: Honour suite in get_arch_platforms

2019-04-10 Thread Ian Jackson
The available platforms may depend on the suite to be used.

Actually implement that for HostDB::Executive.
For Static, we leave it to the user.

Signed-off-by: Ian Jackson 
---
 Osstest/HostDB/Executive.pm | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/Osstest/HostDB/Executive.pm b/Osstest/HostDB/Executive.pm
index bf6ec57d..7ffca6c4 100644
--- a/Osstest/HostDB/Executive.pm
+++ b/Osstest/HostDB/Executive.pm
@@ -99,18 +99,20 @@ SELECT DISTINCT hostflag
FROM hostflags h0
WHERE EXISTS (
SELECT *
- FROM hostflags h1, hostflags h2, hostflags h3
+ FROM hostflags h1, hostflags h2, hostflags h3, hostflags h4
 WHERE h0.hostname = h1.hostname
   AND h1.hostname = h2.hostname
   AND h2.hostname = h3.hostname
+  AND h3.hostname = h4.hostname
   AND h1.hostflag = ?
   AND h2.hostflag = ?
   AND h3.hostflag = 'purpose-test'
+  AND h4.hostflag = ?
)
AND hostflag like 'platform-%';
 END
 
-$platsq->execute("blessed-$blessing", "arch-$arch");
+$platsq->execute("blessed-$blessing", "arch-$arch", "suite-$suite");
 
 while (my ($plat) = $platsq->fetchrow_array()) {
$plat =~ s/^platform-//g or die;
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 45/62] Debian: partman scripts: Run right away too

2019-04-10 Thread Ian Jackson
We are switching the installation of these to partman/early_command
which runs as a result of a /lib/partman/init.d hook.  That means that
things we install don't get picked up, so run them right away (too).

Signed-off-by: Ian Jackson 
---
 Osstest/Debian.pm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index d1b52b5b..9dec321f 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1157,7 +1157,7 @@ sub preseed_create ($$;@) {
 my $d_i = $ho->{Tftp}{Path}.'/'.di_installer_path($ho);
 
 preseed_hook_installscript($ho, $sfx,
-  '/lib/partman/init.d', '000override-parted-devices', 

[Xen-devel] [OSSTEST PATCH 49/62] Debian: Move preseed_backports_packages earlier

2019-04-10 Thread Ian Jackson
No functional change.

Signed-off-by: Ian Jackson 
---
 Osstest/Debian.pm | 88 +++
 1 file changed, 44 insertions(+), 44 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 9aa88822..25bf8e88 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -823,6 +823,50 @@ chmod 600 $subdir/etc/ssh/ssh_host_*_key ||:
 END
 }
 
+sub preseed_backports_packages (@) {
+my ($ho, $sfx, $xopts, $suite, @pkgs) = @_;
+
+if (! $xopts->{BackportsSourcesAlreadyAdded}++) {
+   my $bp_url = $c{"DebianSnapshotBackports_$suite"};
+   $bp_url ||= "http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath};;
+
+   my $apt_insert='';
+   my $extra_rune='';
+   if ($suite =~ m/wheezy|jessie/) {
+   # this has global effect, unfortunately
+   $extra_rune = <\$d/50osstestsnapshot <>/target/etc/apt/sources.list
+
+# $suite backports
+deb $apt_insert $bp_url $suite-backports main
+EOF
+$extra_rune
+in-target apt-get update
+END
+}
+
+preseed_hook_command($ho, 'late_command', $sfx, <{BackportsSourcesAlreadyAdded}++) {
-   my $bp_url = $c{"DebianSnapshotBackports_$suite"};
-   $bp_url ||= "http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath};;
-
-   my $apt_insert='';
-   my $extra_rune='';
-   if ($suite =~ m/wheezy|jessie/) {
-   # this has global effect, unfortunately
-   $extra_rune = <\$d/50osstestsnapshot <>/target/etc/apt/sources.list
-
-# $suite backports
-deb $apt_insert $bp_url $suite-backports main
-EOF
-$extra_rune
-in-target apt-get update
-END
-}
-
-preseed_hook_command($ho, 'late_command', $sfx, 

[Xen-devel] [OSSTEST PATCH 51/62] dm restrict audit: always install (some) chiark-scripts

2019-04-10 Thread Ian Jackson
In
  dm restrict audit: install newer chiark-scripts for fishdescriptor
arrangements were made to install suitable chiark-scripts for
for jessie and stretch.

For buster and later, the mainline Debian version of chiark-scripts is
indeed sufficient, but nothing installed it.  Do that.

Signed-off-by: Ian Jackson 
---
 Osstest/Debian.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 8d53bbe1..9d7d1518 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -997,7 +997,7 @@ d-i apt-setup/another boolean false
 d-i apt-setup/non-free boolean false
 d-i apt-setup/contrib boolean false
 
-d-i pkgsel/include string openssh-server, ntp, ntpdate, ethtool, 
chiark-utils-bin, wget, $extra_packages
+d-i pkgsel/include string openssh-server, ntp, ntpdate, ethtool, 
chiark-utils-bin, chiark-scripts, wget, $extra_packages
 
 d-i grub-installer/force-efi-extra-removable boolean true
 
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 46/62] Debian: set partman-lvm/device_remove_lvm_span

2019-04-10 Thread Ian Jackson
Web searching[1] suggests that this suppresses this error:

  !! ERROR: Unable to automatically remove LVM data
  Because the volume group(s) on the selected device also consist of
  physical volumes on other devices, it is not considered safe to
  remove its LVM data automatically. If you wish to use this device
  for partitioning, please remove its LVM data first.

[1] eg 
https://serverfault.com/questions/571363/unable-to-automatically-remove-lvm-data

It doesn't, though.  I am only experiencing it now because the ad-hoc
disk-erasing (25erase-other-disks) is broken for other reasons.  But
let's have it anyway as it sounds like a thing we might want.

Signed-off-by: Ian Jackson 
---
 Osstest/Debian.pm | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 9dec321f..073b776c 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -907,6 +907,7 @@ d-i partman-partitioning/confirm_write_new_label boolean 
true
 d-i partman/choose_partition select finish
 d-i partman/confirm boolean true
 d-i partman-lvm/confirm boolean true
+d-i partman-lvm/device_remove_lvm_span boolean true
 
 d-i partman/confirm_nooverwrite true
 d-i partman-lvm/confirm_nooverwrite true
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 07/62] ts-leak-check: suppress systemd-shim, which leaks in stretch

2019-04-10 Thread Ian Jackson
From: Wei Liu 

Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 
---
 ts-leak-check | 1 +
 1 file changed, 1 insertion(+)

diff --git a/ts-leak-check b/ts-leak-check
index 678d0696..41e6245d 100755
--- a/ts-leak-check
+++ b/ts-leak-check
@@ -202,6 +202,7 @@ xenstore /vm
 xenstore /libxl
 
 process .* udevd
+process .* /.+/systemd-shim
 
 file /var/run/xenstored/db
 file /var/run/xenstored/db.debug
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 54/62] ts-debian-hvm-install: Honour linux_boot_append target var

2019-04-10 Thread Ian Jackson
This looks for:
  _linux_boot_append
  all_guest_linux_boot_append

Nothing sets these yet.

Signed-off-by: Ian Jackson 
---
 ts-debian-hvm-install | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/ts-debian-hvm-install b/ts-debian-hvm-install
index 54d5d1c2..ed8803ed 100755
--- a/ts-debian-hvm-install
+++ b/ts-debian-hvm-install
@@ -129,6 +129,9 @@ sub gcmdline (;$) {
PreseedScheme => 'file');
 push @dicmdline, $extra if $extra;
 
+my $append = target_var($gho,'linux_boot_append');
+push @dicmdline, $append if $append;
+
 push @dicmdline, "--";
 # See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=762007 for
 # why console= is repeated.
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 48/62] dm restrict audit: actually install right package for fishdescriptor

2019-04-10 Thread Ian Jackson
In
  dm restrict audit: install newer chiark-scripts for fishdescriptor
a locally-provided chiark-scripts_6.0.2_all.deb was installed for
jessie.  For stretch a backport was installed, but mistakenly
of chiark-utils-bin rather than chiark-scripts.

Signed-off-by: Ian Jackson 
---
 Osstest/Debian.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 79b7960d..9aa88822 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1060,7 +1060,7 @@ sub preseed_create_guest ($$$;@) {
 if (grep { m/_dmrestrict$/ && $r{$_} } keys %r and
$suite =~ m/stretch/) {
preseed_backports_packages($ho, $sfx, \%xopts, $suite,
-  qw(chiark-utils-bin));
+  qw(chiark-scripts));
 }
 
 my $preseed_file= preseed_base($ho, $sfx, $extra_packages, %xopts);
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 52/62] target_install_packages: Consistently use qw(...) rather than '...'

2019-04-10 Thread Ian Jackson
qw(...) splits its argument into words.

There is one semantic change, where two package names were passed in a
single argument.  That worked by accident.

Signed-off-by: Ian Jackson 
---
 ts-xen-install | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/ts-xen-install b/ts-xen-install
index 80952857..03f8c03e 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -53,18 +53,18 @@ sub packages () {
   qemu-utils
netcat-openbsd));
 if ($ho->{Suite} !~ m/squeeze|wheezy|jessie/) {
-target_install_packages($ho, 'net-tools libnl-route-3-200');
+target_install_packages($ho, qw(net-tools libnl-route-3-200));
 }
 if ($ho->{Suite} =~ m/jessie/) {
-target_install_packages($ho, 'libnl-route-3-200');
+target_install_packages($ho, qw(libnl-route-3-200));
 }
 target_install_packages($ho,
$ho->{Suite} =~ /squeeze/ ? "libyajl1" : 
"libyajl2");
 if ($ho->{Suite} !~ m/lenny|squeeze/) {
-target_install_packages($ho, 'libfdt1');
+target_install_packages($ho, qw(libfdt1));
 }
 if ($r{arch} eq 'i386') {
-   target_install_packages($ho, 'libc6-xen');
+   target_install_packages($ho, qw(libc6-xen));
 }
 target_install_packages($ho, @{toolstack($ho)->{ExtraPackages}})
 if toolstack($ho)->{ExtraPackages};
@@ -88,7 +88,7 @@ sub some_extradebs ($) {
$ontarget = "extrapackages-$cfgvar-$counter"; $counter++;
$dpkgopts = '-iGROEB';
logm("$cfgvar: updating packages from directory $path");
-   target_install_packages($ho, 'rsync') unless $rsync_installed++;
+   target_install_packages($ho, qw(rsync)) unless $rsync_installed++;
target_putfile_root($ho,300, "$path/.", $ontarget, '-r');
} elsif ($path =~ m{\.deb$}) {
$path =~ s{_\.deb}{ "_$r{arch}.deb" }e;
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 61/62] production-config-cambridge: Provide TftpDiVersion_stretch

2019-04-10 Thread Ian Jackson
Signed-off-by: Ian Jackson 
---
 production-config-cambridge | 1 +
 1 file changed, 1 insertion(+)

diff --git a/production-config-cambridge b/production-config-cambridge
index ce6239bd..a50b19b3 100644
--- a/production-config-cambridge
+++ b/production-config-cambridge
@@ -77,6 +77,7 @@ TftpNetGrubTemplatesReal Netgrub.cfg/%ether%
 TftpNetbootGroup osstest
 TftpDiVersion_wheezy 2016-06-08
 TftpDiVersion_jessie 2018-06-26
+TftpDiVersion_stretch 2019-04-10
 
 DebianSnapshotBackports_jessie 
http://snapshot.debian.org/archive/debian/20190206T211314Z/
 
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 50/62] dm restrict audit: actually install fishdescriptor in host

2019-04-10 Thread Ian Jackson
In
  dm restrict audit: install newer chiark-scripts for fishdescriptor
arrangements were made to install a backport of chiark-scripts
but the code was mistakenly placed in preseed_create_guest but
of course it's needed in the host.

Signed-off-by: Ian Jackson 
---
 Osstest/Debian.pm | 11 ++-
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 25bf8e88..8d53bbe1 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1022,6 +1022,12 @@ END
 # security.d.o CDN seems unreliable right now
 # and jessie-updates is no more
 
+if (grep { m/_dmrestrict$/ && $r{$_} } keys %r and
+   $suite =~ m/stretch/) {
+   preseed_backports_packages($ho, $sfx, \%xopts, $suite,
+  qw(chiark-scripts));
+}
+
 $preseed .= <<"END";
 
 ### END OF DEBIAN PRESEED BASE
@@ -1057,11 +1063,6 @@ sub preseed_create_guest ($$$;@) {
 $extra_packages = "pv-grub-menu";
 }
 }
-if (grep { m/_dmrestrict$/ && $r{$_} } keys %r and
-   $suite =~ m/stretch/) {
-   preseed_backports_packages($ho, $sfx, \%xopts, $suite,
-  qw(chiark-scripts));
-}
 
 my $preseed_file= preseed_base($ho, $sfx, $extra_packages, %xopts);
 $preseed_file.= (

[Xen-devel] [OSSTEST PATCH 43/62] preseed_hook_installscript: Use partman/early_command, not preseed/

2019-04-10 Thread Ian Jackson
On iso-based installs, with stretch, preseed/early_command runs before
the network is up.  This causes the install to fail.

Our existing call sites add things to
   /usr/lib/base-installer.d/
   /lib/partman/init.d/
for which this is still early enough.

Signed-off-by: Ian Jackson 
---
 Osstest/Debian.pm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index f8ff4f46..3afea62b 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1484,11 +1484,12 @@ sub preseed_hook_command () {
 
 sub preseed_hook_installscript ($) {
 my ($ho, $sfx, $installer_dir, $installer_leaf, $data) = @_;
+# the specified script is installed via partman/early_command
 my $installer_pathname= "$installer_dir/$installer_leaf";
 my $urlfile= $installer_pathname;
 $urlfile =~ s/[^-_0-9a-z]/ sprintf "X%02x", ord($&) /ge;
 my $url= create_webfile($ho, $urlfile, $data);
-preseed_hook_command($ho, 'early_command', $sfx, 

[Xen-devel] [OSSTEST PATCH 19/62] Debian: Fix http:// url for bugs.xenproject.org

2019-04-10 Thread Ian Jackson
Signed-off-by: Ian Jackson 
---
 Osstest/Debian.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 80b4cf37..414cd897 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -239,7 +239,7 @@ END
# Dom0 specific kernel options
my @xenkopt = @kopt;
push @xenkopt, $xenkopt;
-   # http://bugs.xenproject.org/xen/bug/45
+   # https://bugs.xenproject.org/xen/bug/45
push @xenkopt, "clk_ignore_unused"
if $ho->{Suite} =~ m/wheezy|jessie|stretch/;
 
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 35/62] TestSupport: Move `target_somefile_getleaf' earlier

2019-04-10 Thread Ian Jackson
We are going to make more use of this in intervening code.

Pure code motion.

Signed-off-by: Ian Jackson 
---
 Osstest/TestSupport.pm | 18 +-
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index ce346097..4e2f120a 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -504,6 +504,15 @@ sub remote_perl_script_done ($) {
 !$? or die "$thing->{What} $?";
 }
 
+sub target_somefile_getleaf ($$$) {
+my ($lleaf_ref, $rdest, $ho) = @_;
+if (!defined $$lleaf_ref) {
+$$lleaf_ref= $rdest;
+$$lleaf_ref =~ s,.*/,,;
+}
+$$lleaf_ref= hostnamepath($ho)."--$$lleaf_ref";
+}
+
 sub sshuho ($$) { my ($user,$ho)= @_; return "$user\@$ho->{Ip}"; }
 
 sub sshopts () {
@@ -591,15 +600,6 @@ sub target_install_packages_norec ($@) {
 target_run_pkgmanager_install($ho,\@packages,1);
 }
 
-sub target_somefile_getleaf ($$$) {
-my ($lleaf_ref, $rdest, $ho) = @_;
-if (!defined $$lleaf_ref) {
-$$lleaf_ref= $rdest;
-$$lleaf_ref =~ s,.*/,,;
-}
-$$lleaf_ref= hostnamepath($ho)."--$$lleaf_ref";
-}
-
 sub tpfcs_core {
 my ($tputfilef,$ho,$timeout,$filedata, $rdest,$lleaf) = @_;
 target_somefile_getleaf(\$lleaf,$rdest,$ho);
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 08/62] ts-host-install: don't use the new nic naming scheme

2019-04-10 Thread Ian Jackson
From: Wei Liu 

Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 
---
 ts-host-install | 4 
 1 file changed, 4 insertions(+)

diff --git a/ts-host-install b/ts-host-install
index 292733ba..9ab3de62 100755
--- a/ts-host-install
+++ b/ts-host-install
@@ -244,6 +244,10 @@ END
 # why this is repeated.
 push @hocmdline, "console=$console" unless $console eq "NONE";
 
+# Don't use "Predictable Network Interface Names"
+# 
https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/
+push @hocmdline, "net.ifnames=0" if $ho->{Suite} =~ m/stretch/;
+
 push @hocmdline,
 get_host_property($ho, "linux-boot-append $ho->{Suite}", ''),
 get_host_property($ho, "linux-boot-append $ho->{Suite} $r{arch}", '');
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 38/62] ts-guests-nbd-mirror: Use target_getfile_root_stash

2019-04-10 Thread Ian Jackson
That removes the rather ad-hoc open-coding.

Signed-off-by: Ian Jackson 
---
 ts-guests-nbd-mirror | 5 +
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/ts-guests-nbd-mirror b/ts-guests-nbd-mirror
index 06903aaa..0365c47b 100755
--- a/ts-guests-nbd-mirror
+++ b/ts-guests-nbd-mirror
@@ -154,10 +154,7 @@ sub shuffleconfigs () {
my $gn= $gns[$i];
my $gho= $ghos[$i];
my $cfgpath= $r{ "$gho->{Guest}_cfgpath" };
-   my $file= $cfgpath;
-   $file=~ s,/,-,g;
-   $file= "$stash/".hostnamepath($cho)."--$file";
-   target_getfile_root($sho, 60, $cfgpath, $file);
+   my $file = target_getfile_root_stash($sho, 60, $cfgpath);
target_putfile_root($cho, 60, $file, $cfgpath);
 }
 }
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 29/62] ts-host-install: Put canary in 70-persistent-net.rules

2019-04-10 Thread Ian Jackson
This will allow us to see if the initramfs's network names are being
properly copied to the installed system.  Ie, this is just a debugging
aid.

Signed-off-by: Ian Jackson 
---
 ts-host-install | 4 
 1 file changed, 4 insertions(+)

diff --git a/ts-host-install b/ts-host-install
index 7423eb9b..ea087a25 100755
--- a/ts-host-install
+++ b/ts-host-install
@@ -218,6 +218,10 @@ SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", 
ATTR{address}=="$ho->{Ether}", A
 END
 }
 
+open CANARY, '>>', "$persistent_net_rules" or die $!;
+print CANARY "\n# - canary - came via initramfs\n" or die $!;
+close CANARY or die $!;
+
 my %xopts;
 
 di_special_kernel($ho, sub {
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 41/62] ts-debian-fixup: Use debian_overlays_fixup_cmd

2019-04-10 Thread Ian Jackson
Otherwise we get the same error for guests as was prevented for hosts
by "preseed_base: chmod ssh host private keys to placate sshd".

Signed-off-by: Ian Jackson 
---
 ts-debian-fixup | 5 +
 1 file changed, 5 insertions(+)

diff --git a/ts-debian-fixup b/ts-debian-fixup
index 0e553d47..7d9d0398 100755
--- a/ts-debian-fixup
+++ b/ts-debian-fixup
@@ -206,7 +206,12 @@ savecfg();
 ether();
 target_kernkind_check($gho);
 access();
+
 debian_overlays($ho, \);
+target_cmd_root($ho, 

[Xen-devel] [OSSTEST PATCH 22/62] contents_make_cpio: Include symlinks

2019-04-10 Thread Ian Jackson
We are going to introduce some symlinks into one of our preprepared
overlays.  We must therefore arrange to copy them as appropriate.

The syntax `-type f,l' is an extension in GNU find.  If this causes
trouble in the future we will then have to introduce the obvious
circumlocution involving ( ).

Signed-off-by: Ian Jackson 
---
 Osstest/TestSupport.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index d35a784b..1f01ac6a 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -1518,7 +1518,7 @@ sub contents_make_cpio ($$$) {
 if (!$child) {
 postfork();
 chdir($srcdir) or die $!;
-open STDIN, 'find ! -name "*~" ! -name "#*" -type f -print0 |'
+open STDIN, 'find ! -name "*~" ! -name "#*" -type f,l -print0 |'
 or die $!;
 open STDOUT, '>&', $fh or die $!;
 system "cpio -H$format -o --quiet -0 -R 1000:1000";
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 05/62] ts-xen-install: install some packages on stretch

2019-04-10 Thread Ian Jackson
From: Wei Liu 

The "route" command is now in that package.

libnl is needed when running xl.

Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 
---
v3: invert condition.
---
 ts-xen-install | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/ts-xen-install b/ts-xen-install
index 8de94ac2..80952857 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -52,6 +52,9 @@ sub packages () {
libsdl1.2debian libglib2.0-0 liblzma5
   qemu-utils
netcat-openbsd));
+if ($ho->{Suite} !~ m/squeeze|wheezy|jessie/) {
+target_install_packages($ho, 'net-tools libnl-route-3-200');
+}
 if ($ho->{Suite} =~ m/jessie/) {
 target_install_packages($ho, 'libnl-route-3-200');
 }
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 42/62] preseed_hook_command: allow specifying di keys other than preseed/*

2019-04-10 Thread Ian Jackson
Ie, only add preseed/ if there is not already a slash.

No functional change with existing call sites other than urls and
temporary filenames.

Signed-off-by: Ian Jackson 
---
 Osstest/Debian.pm | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 78d242e4..f8ff4f46 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1472,9 +1472,12 @@ END
 
 sub preseed_hook_command () {
 my ($ho, $di_key, $sfx, $text) = @_;
+$di_key = "preseed/$di_key" unless $di_key =~ m{/};
+my $basename = $di_key;
+$basename =~ s{/}{--};
 my $ix= $#{ $preseed_cmds{$di_key} } + 1;
-my $url= create_webfile($ho, "$di_key-$ix$sfx", $text);
-my $file= "/tmp/$di_key-$ix";
+my $url= create_webfile($ho, "$basename-$ix$sfx", $text);
+my $file= "/tmp/$basename-$ix";
 my $cmd_cmd= "$preseed_wget -O $file '$url' && chmod +x $file && $file";
 push @{ $preseed_cmds{$di_key} }, $cmd_cmd;
 }
@@ -1521,7 +1524,7 @@ END
 sub preseed_hook_cmds () {
 my $preseed;
 foreach my $di_key (keys %preseed_cmds) {
-$preseed .= "d-i preseed/$di_key string ".
+$preseed .= "d-i $di_key string ".
 (join ' && ', @{ $preseed_cmds{$di_key} }). "\n";
 }
 return $preseed;
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 18/62] Drop rumprun tests

2019-04-10 Thread Ian Jackson
From: Wei Liu 

These have been failing for some time and it doesn't any more look
like this will be an attractive route to stub device models.  (At
least two Xen downstream projects are using Linux-based stub device
models.)

Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 
---
v4: Expand commit message.
---
 Osstest/RumpRun.pm|  68 -
 Osstest/Toolstack/rumprun.pm  |  33 ---
 allow.all |   1 -
 allow.rumprun |   1 -
 ap-common |   7 --
 ap-fetch-version  |   4 -
 ap-fetch-version-old  |   5 -
 ap-print-url  |   3 -
 ap-push   |   5 -
 cr-daily-branch   |   9 --
 cri-common|   1 -
 crontab   |   4 +-
 daily-cron-email-real--rumpuserxen|   4 -
 daily-cron-email-real-bisectcomplete--rumpuserxen |   4 -
 make-flight   |  35 ---
 mfi-common|  37 ---
 sg-run-job|  26 -
 ts-rumprun-bake   |  88 -
 ts-rumprun-build  | 103 
 ts-rumprun-demo-build |  67 -
 ts-rumprun-demo-setup |  54 ---
 ts-rumprun-demo-xenstorels| 113 --
 ts-rumprun-test-prep  |  37 ---
 23 files changed, 2 insertions(+), 707 deletions(-)
 delete mode 100644 Osstest/RumpRun.pm
 delete mode 100644 Osstest/Toolstack/rumprun.pm
 delete mode 100644 allow.rumprun
 delete mode 100644 daily-cron-email-real--rumpuserxen
 delete mode 100644 daily-cron-email-real-bisectcomplete--rumpuserxen
 delete mode 100755 ts-rumprun-bake
 delete mode 100755 ts-rumprun-build
 delete mode 100755 ts-rumprun-demo-build
 delete mode 100755 ts-rumprun-demo-setup
 delete mode 100755 ts-rumprun-demo-xenstorels
 delete mode 100755 ts-rumprun-test-prep

diff --git a/Osstest/RumpRun.pm b/Osstest/RumpRun.pm
deleted file mode 100644
index f46d520b..
--- a/Osstest/RumpRun.pm
+++ /dev/null
@@ -1,68 +0,0 @@
-# This is part of "osstest", an automated testing framework for Xen.
-# Copyright (C) 2009-2013 Citrix Inc.
-# 
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-# 
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Affero General Public License for more details.
-# 
-# You should have received a copy of the GNU Affero General Public License
-# along with this program.  If not, see .
-
-
-package Osstest::RumpRun;
-
-use strict;
-use warnings;
-
-use Osstest::TestSupport;
-
-BEGIN {
-use Exporter ();
-our ($VERSION, @ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS);
-$VERSION = 1.00;
-@ISA = qw(Exporter);
-@EXPORT  = qw(
- rumprun_guest_create
-   );
-%EXPORT_TAGS = ( );
-
-@EXPORT_OK   = qw();
-}
-
-sub rumprun_guest_create ($) {
-my ($gho) = @_;
-my $ho = $gho->{Host};
-my $gn = $gho->{Guest};
-guest_prepare_disk($gho);
-
-my $rumprun = target_var($gho, 'rumprun_path');
-if (!$rumprun) {
-   logm("finding rumprun to use for $gho->{Name} on $ho->{Name}");
-   my $buildjob = $r{guests_rumprunbuildjob} // # todo: eliminate this
-   target_var($gho, 'rumprunbuildjob');
-   my $rumprundist = target_extract_jobdistpath_subdir
-   ($ho, "rumprun-rumprun-g-$gho->{Name}", "rumprun", $buildjob);
-   $rumprun = "$rumprundist/rumprun";
-   store_runvar("${gn}_rumprun_path", $rumprun);
-}
-
-my $imagepath = $r{"${gn}_imagepath"};
-my $cmdline = guest_var($gho, 'cmdline', undef);
-
-my $cmd = "$rumprun xen";
-$cmd .= " -N $gn";
-$cmd .= " -I xenif0,xenif,mac=$gho->{Ether}";
-$cmd .= " -W xenif0,inet,dhcp";
-$cmd .= " $imagepath";
-$cmd .= " $cmdline";
-
-target_cmd_root($ho, $cmd, 100);
-}
-
-1;
diff --git a/Osstest/Toolstack/rumprun.pm b/Osstest/Toolstack/rumprun.pm
deleted file mode 100644
index 74742c45..
--- a/Osstest/Toolstack/rumprun.pm
+++ /dev/null
@@ -1,33 +0,0 @@
-# This is part of "osstest", an automated testing framework for Xen.
-# Copyright (C) 2014 Citrix Inc.
-#
-# This program is free software: 

[Xen-devel] [OSSTEST PATCH 56/62] platforms: Pass suite to get_arch_platforms

2019-04-10 Thread Ian Jackson
The available platforms may depend on the suite to be used.

For now we use $defsuite from make-flight, which is not entirely right
but it will do for now because we don't use other suites much.

No functional change yet since neither implementation uses it.

Signed-off-by: Ian Jackson 
---
 Osstest/HostDB/Executive.pm | 4 ++--
 Osstest/HostDB/Static.pm| 4 ++--
 cri-getplatforms| 2 +-
 make-flight | 4 ++--
 4 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/Osstest/HostDB/Executive.pm b/Osstest/HostDB/Executive.pm
index d42250f6..bf6ec57d 100644
--- a/Osstest/HostDB/Executive.pm
+++ b/Osstest/HostDB/Executive.pm
@@ -90,8 +90,8 @@ END
 return $flags;
 }
 
-sub get_arch_platforms ($$) {
-my ($hd, $blessing, $arch) = @_;
+sub get_arch_platforms ($$$) {
+my ($hd, $blessing, $arch, $suite) = @_;
 
 my @plats = ( );
 my $platsq = $dbh_tests->prepare(get_arch_platforms("'$blessing'", 
"'$1'", "'$2'") or die $!;
 '
 }
diff --git a/make-flight b/make-flight
index 1e3ebd5c..92dacb35 100755
--- a/make-flight
+++ b/make-flight
@@ -27,9 +27,9 @@ buildflight=$4
 flight=`./cs-flight-create $blessing $branch`
 
 . ./cri-common
-. ./cri-getplatforms
 . ./ap-common
 . ./mfi-common
+. ./cri-getplatforms
 
 # Older versions of Xen may not build with the current default.  Note
 # that branches older than 4.3 might need something even older than
@@ -612,7 +612,7 @@ do_pv_debian_tests () {
 
   for xsm in $xsms ; do
 # Basic PV Linux test with xl
-for platform in '' `getplatforms $xenarch` ; do
+for platform in '' `getplatforms $xenarch $defsuite` ; do
 
   # xsm test is not platform specific
   if [ x$xsm = xtrue -a x$platform != x ]; then
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 23/62] contents_make_cpio: Make files owned by root

2019-04-10 Thread Ian Jackson
This function is called to generate overlays for use, mainly, by the
initramfs.

We are going to use it to ship udev rules.  Annoyingly, udev hates
files which aren't owned by root - it simply ignores them.

Signed-off-by: Ian Jackson 
---
 Osstest/TestSupport.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 1f01ac6a..41f6f5f8 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -1521,7 +1521,7 @@ sub contents_make_cpio ($$$) {
 open STDIN, 'find ! -name "*~" ! -name "#*" -type f,l -print0 |'
 or die $!;
 open STDOUT, '>&', $fh or die $!;
-system "cpio -H$format -o --quiet -0 -R 1000:1000";
+system "cpio -H$format -o --quiet -0 -R 0:0";
 $? and die $?;
 $!=0; close STDIN; die "$! $?" if $! or $?;
 exit 0;
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 60/62] Debian: Fix /lib/udev/ifupdown-hotplug in guests from debootstrap

2019-04-10 Thread Ian Jackson
The ifupdown-hotplug problem just described affects guests too.

Empirically, this occurs only with the guests from xen-tools.  In my
osstest stretch series development tests this resulted in occasional
failures of ts-guest-start.  The problem is not deterministic; and it
may be that it is a race between the guest's random dhcp retry and
osstest's timeout, or something.  The race affected both x86 and ARM.

I conjecture that it only affects xen-tools created guests because
those guests use sysvinit.  Presumably the other guests have systemd
which has some different ordering.  I conjecture that the sysvinit
boot arrangements were damaged by some changes made in stretch to
shared components (udev, probably) to support systemd.

An alternative explanation for seeing the failure only with xen-tools
created guests might be that all guests are affected, but only
xen-tools created guests are booted with a short timeout; whereas the
d-i created ones have a long timeout for firstboot at least.  I don't
believe this theory because all guests are restarted with
ts-guest-start and a short timeout.

So: in ts-debian-fixup, execute the sed rune to fix
/lib/udev/ifupdown-hotplug.  This then happens before first boot.

Debian guests installed via d-i are not affected by this patch.

Signed-off-by: Ian Jackson 
---
 ts-debian-fixup | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/ts-debian-fixup b/ts-debian-fixup
index 7d9d0398..fef9836e 100755
--- a/ts-debian-fixup
+++ b/ts-debian-fixup
@@ -212,6 +212,8 @@ target_cmd_root($ho, 

[Xen-devel] [OSSTEST PATCH 24/62] overlay-persistent-net: Copy from jessie

2019-04-10 Thread Ian Jackson
These were copied from a system running Debian jessie.

The nontrivial files are:
 # Copyright (C) 2006 Marco d'Itri 
 # Copyright (C) 2007 Kay Sievers 
and licenced GPLv2+.  That is compatible with osstest's AGPLv3+.

Right now we do nothing with these.

Signed-off-by: Ian Jackson 
---
 overlay-persistent-net/etc/init.d/udev-finish  |  22 +++
 .../etc/udev/rule_generator.functions  | 113 +++
 .../etc/udev/rules.d/73-usb-net-by-mac.rules   |   0
 .../etc/udev/rules.d/75-net-description.rules  |   0
 .../udev/rules.d/75-persistent-net-generator.rules | 143 +++
 overlay-persistent-net/etc/udev/udev-finish|   9 ++
 overlay-persistent-net/etc/udev/write_net_rules| 152 +
 .../lib/udev/rule_generator.functions  |   1 +
 overlay-persistent-net/lib/udev/udev-finish|   1 +
 overlay-persistent-net/lib/udev/write_net_rules|   1 +
 10 files changed, 442 insertions(+)
 create mode 100755 overlay-persistent-net/etc/init.d/udev-finish
 create mode 100644 overlay-persistent-net/etc/udev/rule_generator.functions
 create mode 100644 
overlay-persistent-net/etc/udev/rules.d/73-usb-net-by-mac.rules
 create mode 100644 
overlay-persistent-net/etc/udev/rules.d/75-net-description.rules
 create mode 100644 
overlay-persistent-net/etc/udev/rules.d/75-persistent-net-generator.rules
 create mode 100755 overlay-persistent-net/etc/udev/udev-finish
 create mode 100755 overlay-persistent-net/etc/udev/write_net_rules
 create mode 12 overlay-persistent-net/lib/udev/rule_generator.functions
 create mode 12 overlay-persistent-net/lib/udev/udev-finish
 create mode 12 overlay-persistent-net/lib/udev/write_net_rules

diff --git a/overlay-persistent-net/etc/init.d/udev-finish 
b/overlay-persistent-net/etc/init.d/udev-finish
new file mode 100755
index ..10602017
--- /dev/null
+++ b/overlay-persistent-net/etc/init.d/udev-finish
@@ -0,0 +1,22 @@
+#!/bin/sh -e
+### BEGIN INIT INFO
+# Provides:  udev-finish
+# Required-Start:udev $local_fs
+# Required-Stop: 
+# Default-Start: S
+# Default-Stop:
+# Short-Description: Copy rules generated while the root was ro
+### END INIT INFO
+
+PATH="/sbin:/bin"
+
+. /lib/lsb/init-functions
+
+case "$1" in
+  start) ;;
+  stop|restart|force-reload) exit 0 ;;
+  *) echo "Usage: $0 {start|stop|restart|force-reload}" >&2; exit 1 ;;
+esac
+
+exec /lib/udev/udev-finish
+
diff --git a/overlay-persistent-net/etc/udev/rule_generator.functions 
b/overlay-persistent-net/etc/udev/rule_generator.functions
new file mode 100644
index ..925193e4
--- /dev/null
+++ b/overlay-persistent-net/etc/udev/rule_generator.functions
@@ -0,0 +1,113 @@
+# functions used by the udev rule generator
+
+# Copyright (C) 2006 Marco d'Itri 
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+
+PATH='/sbin:/bin'
+
+# Read a single line from file $1 in the $DEVPATH directory.
+# The function must not return an error even if the file does not exist.
+sysread() {
+   local file="$1"
+   [ -e "/sys$DEVPATH/$file" ] || return 0
+   local value
+   read value < "/sys$DEVPATH/$file" || return 0
+   echo "$value"
+}
+
+sysreadlink() {
+   local file="$1"
+   [ -e "/sys$DEVPATH/$file" ] || return 0
+   readlink -f /sys$DEVPATH/$file 2> /dev/null || true
+}
+
+# Return true if a directory is writeable.
+writeable() {
+   if ln -s test-link $1/.is-writeable 2> /dev/null; then
+   rm -f $1/.is-writeable
+   return 0
+   else
+   return 1
+   fi
+}
+
+# Create a lock file for the current rules file.
+lock_rules_file() {
+   RUNDIR=/run/udev
+   [ -e "$RUNDIR" ] || return 0
+
+   RULES_LOCK="$RUNDIR/.lock-${RULES_FILE##*/}"
+
+   retry=30
+   while ! mkdir $RULES_LOCK 2> /dev/null; do
+   if [ $retry -eq 0 ]; then
+echo "Cannot lock $RULES_FILE!" >&2
+exit 2
+   fi
+   sleep 1
+   retry=$(($retry - 1))
+   done
+}
+
+unlock_rules_file() {
+   [ "$RULES_LOCK" ] || return 0
+   rmdir $RULES_LOCK || true
+}
+
+# Choose the real rules file if it is writeable or a temporary file if not.
+# Both files should be checked later when looking for existing rules.
+choose_rules_file() {
+   RUNDIR=/run/udev
+   local 

[Xen-devel] [OSSTEST PATCH 16/62] make-flight: guest should use jessie to test pvgrub

2019-04-10 Thread Ian Jackson
From: Wei Liu 

stretch has 64bit feature enabled for ext4, which pvgrub can't cope.
We want to continue to test pvgrub, so specify jessie in the guest
suite field.

A consequence is that this test will test jessie forever.  Eventually
jessie will rot so badly that this test fails and then we will no
longer be testing pvgrub1.  Hopefully by then no-one will be using it.

CC: Juergen Gross 
Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 
---
v4: Adjust commit message slightly.
---
 make-flight | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/make-flight b/make-flight
index 93c82a21..2f875125 100755
--- a/make-flight
+++ b/make-flight
@@ -608,8 +608,8 @@ do_pvgrub_tests () {
   job_create_test test-$xenarch$kern-$dom0arch-amd64-pvgrub \
 test-debian-di xl $xenarch $dom0arch\
   debian_arch=amd64 \
-  debian_suite=$guestsuite  \
-  debian_di_version=$guest_di_version   \
+  debian_suite=jessie   \
+  debian_di_version=`getconfig_TftpDiVersion_suite jessie`  \
   debian_method=netboot \
   debian_bootloader=pvgrub  \
   all_hostflags=$most_hostflags \
@@ -617,8 +617,8 @@ do_pvgrub_tests () {
   job_create_test test-$xenarch$kern-$dom0arch-i386-pvgrub  \
 test-debian-di xl $xenarch $dom0arch\
   debian_arch=i386  \
-  debian_suite=$guestsuite  \
-  debian_di_version=$guest_di_version   \
+  debian_suite=jessie   \
+  debian_di_version=`getconfig_TftpDiVersion_suite jessie`  \
   debian_method=netboot \
   debian_bootloader=pvgrub  \
   all_hostflags=$most_hostflags
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 32/62] ts-kernel-build: Enable some additional drivers for Thunder-X

2019-04-10 Thread Ian Jackson
Without this, our kernels do not find the storage.

Suggested-by: Julien Grall 
Signed-off-by: Ian Jackson 
---
 ts-kernel-build | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/ts-kernel-build b/ts-kernel-build
index 0bc443de..3dad7d36 100755
--- a/ts-kernel-build
+++ b/ts-kernel-build
@@ -242,6 +242,17 @@ setopt CONFIG_MMC_DW m
 setopt CONFIG_MMC_DW_EXYNOS m
 setopt CONFIG_REGULATOR_S5M8767 m
 
+# Enable some additonal drivers for Thunder-X
+setopt CONFIG_PCI_HOST_THUNDER_PEM=y
+setopt CONFIG_PCI_HOST_THUNDER_ECAM=y
+setopt CONFIG_THUNDER_NIC_PF=m
+setopt CONFIG_THUNDER_NIC_VF=m
+setopt CONFIG_THUNDER_NIC_BGX=m
+setopt CONFIG_THUNDER_NIC_RGX=m
+setopt CONFIG_MDIO_THUNDER=m
+setopt CONFIG_I2C_THUNDERX=m
+setopt CONFIG_SPI_THUNDERX=m
+
 
 
 END
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 15/62] ts-debian-di-install: use gho to pick d-i

2019-04-10 Thread Ian Jackson
From: Wei Liu 

The original code used ho which gave us the host suite, but we wanted
the guest suite.

Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 
---
 ts-debian-di-install | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/ts-debian-di-install b/ts-debian-di-install
index 60079712..361a1710 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -152,8 +152,8 @@ sub setup_netboot($$$)
die if $r{ "$gho->{Guest}_netboot_kernel" }
|| $r{ "$gho->{Guest}_netboot_ramdisk" };
 
-   my $di_path = $c{TftpPath}.'/'.$ho->{Tftp}{DiBase}.'/'.${arch}.'/'.
-   debian_guest_di_version($ho).'-'.$ho->{Suite};
+   my $di_path = $c{TftpPath}.'/'.$gho->{Tftp}{DiBase}.'/'.${arch}.'/'.
+   debian_guest_di_version($gho).'-'.$gho->{Suite};
 
 if (${arch} =~ m/amd64|i386/) {
$kernel = "$di_path/vmlinuz-xen";
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 30/62] persistent-net: Include initramfs script to copy to target

2019-04-10 Thread Ian Jackson
This is the piece which actually copies the installer's network names
to the target.  It should not appear on the installed system, so it's
not in overlay-persistent-net.

Technically this is only useful when the installer has the
overlay-persistent-net in it, which is done only in ts-host-install
and not in all the places where setup_netboot_firstboot is used.
But without overlay-persistent-net it is harmless, and it is most
convenient to put it here.

The little script fragment was copied out of a jessie debian-installer
initramfs environment.

Signed-off-by: Ian Jackson 
---
 Osstest/Debian.pm | 13 +
 1 file changed, 13 insertions(+)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index e246c012..6309b246 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -848,6 +848,19 @@ if [ -f /target$grub ] ; then
 fi
 END
 
+# See comment in debian_overlays re net.ifnames=1
+$ho->{Suite} =~ m/jessie|wheezy/ or
+   preseed_hook_installscript($ho, $sfx,
+'/usr/lib/base-installer.d/', '05udev', <<'END');
+#!/bin/sh -e
+
+RULESDIR=etc/udev/rules.d
+
+mkdir -p /target/$RULESDIR
+cp /$RULESDIR/70-persistent-*.rules /target/$RULESDIR 2>/dev/null || true
+
+END
+
 debian_overlays($ho, sub {
my ($srcdir, $tfilename) = @_;
preseed_hook_overlay($ho, $sfx, $srcdir, $tfilename);
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 11/62] ts-xen-build-prep: install e2fslibs-dev

2019-04-10 Thread Ian Jackson
From: Wei Liu 

The in-tree libfsimage ext2fs implementation can't handle 64bit
enabled ext4, which is the default in stretch.

Installing e2fslibs-dev causes libfsimage to pick up the packaged
ext2fs implementation.

Signed-off-by: Wei Liu 
---
 ts-xen-build-prep | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/ts-xen-build-prep b/ts-xen-build-prep
index ca5735a1..c38ab36d 100755
--- a/ts-xen-build-prep
+++ b/ts-xen-build-prep
@@ -226,6 +226,12 @@ sub prep () {
 push(@packages, qw(texinfo autopoint libpciaccess-dev));
 }
 
+# The in-tree ext4 support in libfsimage can't cope with 64bit ext4 on
+# 32bit build. Use the packaged library.
+if ($ho->{Suite} !~ m/squeeze|wheezy|jessie/) {
+push(@packages, qw(e2fslibs-dev));
+}
+
 target_install_packages($ho, @packages);
 target_cmd_root($ho, "chmod -R a+r /usr/share/git-core/templates");
 # workaround for Debian #595728
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 34/62] TestSupport: Move `stashed files' and `next_unique_name' earlier

2019-04-10 Thread Ian Jackson
We are going to make more use of this in intervening code.

Pure code motion.

Signed-off-by: Ian Jackson 
---
 Osstest/TestSupport.pm | 78 +-
 1 file changed, 39 insertions(+), 39 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 41f6f5f8..ce346097 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -300,6 +300,45 @@ sub get_filecontents ($;$) {
 return $data;
 }
 
+#-- stashed files --
+
+sub next_unique_name ($) {
+my ($fnref) = @_;
+my $num = $$fnref =~ s/\+([1-9]\d*)$// ? $1 : 0;
+$$fnref .= '+'.($num+1);
+}
+
+sub open_unique_stashfile ($) {
+my ($leafref) = @_;
+my $dh;
+for (;;) {
+my $df= $$leafref;
+$dh= new IO::File "$stash/$df", O_RDWR|O_EXCL|O_CREAT;
+last if $dh;
+die "$df $!" unless $!==
+next_unique_name $leafref;
+}
+return $dh;
+}
+
+sub get_stashed ($$) {
+my ($param, $otherflightjob) = @_; 
+# may be run outside transaction, or with flights locked
+my ($oflight, $ojob) = otherflightjob($otherflightjob);
+my $path= get_runvar($param, $otherflightjob);
+die "$path $& " if
+$path =~ m,[^-+._0-9a-zA-Z/], or
+$path =~ m/\.\./;
+return "$c{Stash}/$oflight/$ojob/$path";
+}
+
+sub compress_stashed($) {
+my ($path) = @_;
+return unless -e "$stash/$path";
+my $r= system 'gzip','-9vf','--',"$stash/$path";
+die "$r $!" if $r;
+}
+
 #-- runvars --
 
 sub store_runvar ($$) {
@@ -597,12 +636,6 @@ sub target_file_exists ($$) {
 die "$rfile $out ?";
 }
 
-sub next_unique_name ($) {
-my ($fnref) = @_;
-my $num = $$fnref =~ s/\+([1-9]\d*)$// ? $1 : 0;
-$$fnref .= '+'.($num+1);
-}
-
 our $target_editfile_cancel_exception =
 bless { }, 'Osstest::TestSupport::TargetEditfileCancelException';
 
@@ -1369,39 +1402,6 @@ sub hostnamepath ($) {
 join '_', reverse @l;
 }
 
-#-- stashed files --
-
-sub open_unique_stashfile ($) {
-my ($leafref) = @_;
-my $dh;
-for (;;) {
-my $df= $$leafref;
-$dh= new IO::File "$stash/$df", O_RDWR|O_EXCL|O_CREAT;
-last if $dh;
-die "$df $!" unless $!==
-next_unique_name $leafref;
-}
-return $dh;
-}
-
-sub get_stashed ($$) {
-my ($param, $otherflightjob) = @_; 
-# may be run outside transaction, or with flights locked
-my ($oflight, $ojob) = otherflightjob($otherflightjob);
-my $path= get_runvar($param, $otherflightjob);
-die "$path $& " if
-$path =~ m,[^-+._0-9a-zA-Z/], or
-$path =~ m/\.\./;
-return "$c{Stash}/$oflight/$ojob/$path";
-}
-
-sub compress_stashed($) {
-my ($path) = @_;
-return unless -e "$stash/$path";
-my $r= system 'gzip','-9vf','--',"$stash/$path";
-die "$r $!" if $r;
-}
-
 #-- other stuff --
 
 sub common_toolstack ($) {
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 17/62] stretch: Use chainloading when booting using GRUB on Arm64

2019-04-10 Thread Ian Jackson
From: Julien Grall 

The GRUB package in stretch is not able to boot Xen on Arm64.
Use chainloading as we did for jessie for the time being.

Note that a bug has been filled on Debian to integrate Xen
pactches for the next release (see [1]).

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=884770

Signed-off-by: Julien Grall 
Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 
---
v4: Correct suite name capitalisation in commit message and comment.
---
 Osstest/Debian.pm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 85b1890d..80b4cf37 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -435,10 +435,10 @@ sub setupboot_grub2 () {
 my $rmenu= '/boot/grub/grub.cfg';
 my $kernkey= (defined $xenhopt ? 'KernDom0' : 'KernOnly');
 
-# Grub2 on Jessie/arm* doesn't do multiboot, so we must chainload.
+# Grub2 on jessie/stretch ARM* doesn't do multiboot, so we must chainload.
 my $need_uefi_chainload =
 get_host_property($ho, "firmware", "") eq "uefi" &&
-$ho->{Suite} =~ m/jessie/ && $r{arch} =~ m/^arm/;
+$ho->{Suite} =~ m/jessie|stretch/ && $r{arch} =~ m/^arm/;
 
 my $parsemenu= sub {
 my $f= bl_getmenu_open($ho, $rmenu, "$stash/$ho->{Name}--grub.cfg.1");
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 40/62] preseed_base: break out debian_overlays_fixup_cmd

2019-04-10 Thread Ian Jackson
We are going to want this for guests too.

Signed-off-by: Ian Jackson 
---
 Osstest/Debian.pm | 14 +++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index d76dd03d..78d242e4 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -36,7 +36,7 @@ BEGIN {
 @EXPORT  = qw(debian_boot_setup
   di_installer_path di_special_kernel
   setupboot_bootloader_edited_rune
-  debian_overlays
+  debian_overlays debian_overlays_fixup_cmd
   debian_guest_suite debian_guest_di_version
   %preseed_cmds
   preseed_base
@@ -815,6 +815,14 @@ sub debian_overlays ($$) {
 $maybe->("$c{OverlayLocal}-$suite", 'overlay-local-$suite.tar');
 }
 
+sub debian_overlays_fixup_cmd ($;$) {
+my ($ho, $subdir) = @_;
+$subdir //= '';
+return 

[Xen-devel] [OSSTEST PATCH 10/62] ts-debian-fixup: append noresume

2019-04-10 Thread Ian Jackson
From: Wei Liu 

See code comment for explanation.

Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 
---
v4: Fix grammar error in comment.
---
 ts-debian-fixup | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/ts-debian-fixup b/ts-debian-fixup
index 478dc2af..0e553d47 100755
--- a/ts-debian-fixup
+++ b/ts-debian-fixup
@@ -178,6 +178,16 @@ sub otherfixupcfg () {
 $extra .= " iommu=soft";
 }
 
+# There might be stale entries in /etc/initramfs-tools/conf.d/resume which
+# get stored in the initramfs. That introduces delay in guest booting which
+# might cause tests to fail.
+#
+# This is particularly prominent in stretch when it tries to scan for the
+# nonexistent device(s) for a long time. See also Debian bug #784810.
+#
+# Append noresume to fix the issue.
+$extra .= " noresume";
+
 if ($cfg =~ m/extra\s*=\s*['"](.*)['"]/) {
$cfg =~ s/extra\s*=\s*['"](.*)['"]/extra = '$1 $extra'/;
 } else {
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 06/62] Debian.pm: use sysvinit-core instead of systemd

2019-04-10 Thread Ian Jackson
From: Wei Liu 

Install that packages for suites >wheezy, because they use systemd as
the default init.

Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 
---
 Osstest/Debian.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 59c60d40..82b5fb40 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -812,7 +812,7 @@ sub preseed_base ($$$;@) {
 
 # Systemd doesn't honor osstest-confirm-booted service, which
 # breaks ts-leak-check.  Fall back to SysV init for now.
-if ( $suite =~ /jessie/ ) {
+if ( $suite !~ /squeeze|wheezy/ ) {
preseed_hook_command($ho, 'late_command', $sfx, 

[Xen-devel] [OSSTEST PATCH 27/62] ts-host-install: Break out $persistent_net_rules

2019-04-10 Thread Ian Jackson
We're going to want to reuse this value.

Signed-off-by: Ian Jackson 
---
 ts-host-install | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/ts-host-install b/ts-host-install
index 8ea81e24..f80a151c 100755
--- a/ts-host-install
+++ b/ts-host-install
@@ -195,6 +195,9 @@ sub setup_netboot_firstboot($) {
system_checked @cmd;
 }
 
+my $persistent_net_rules =
+   "$initrd_overlay.d/etc/udev/rules.d/70-persistent-net.rules";
+
 my $ipappend = 2;
 my $wantphysif= get_host_property($ho,'interface force','auto');
 logm("Forcing interface $wantphysif");
@@ -206,7 +209,7 @@ sub setup_netboot_firstboot($) {
# ip(8) moved to /sbin in Jessie
my $ipcmd = $ho->{Suite} =~ m/wheezy/ ? "/bin/ip" : "/sbin/ip";
 file_simple_write_contents
-("$initrd_overlay.d/etc/udev/rules.d/70-persistent-net.rules",
+($persistent_net_rules,
  $ho->{Flags}{'force-mac-address'} ? 

[Xen-devel] [OSSTEST PATCH 26/62] persistent-net: Add overlay in installer >= stretch

2019-04-10 Thread Ian Jackson
We are going to need this in the installer so that the interface names
from the installer environment are captured so that they can be the
same on the host.

This prepares the ground for turning off net.ifnames.  The actual
rules are gated on net.ifnames so right now there is no change.

Signed-off-by: Ian Jackson 
---
 ts-host-install | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/ts-host-install b/ts-host-install
index 9ab3de62..8ea81e24 100755
--- a/ts-host-install
+++ b/ts-host-install
@@ -188,6 +188,13 @@ sub setup_netboot_firstboot($) {
 system qw(rm -rf --),"$initrd_overlay.d";
 mkdir "$initrd_overlay.d" or die "$initrd_overlay.d: $!";
 
+if ($ho->{Suite} !~ m/wheezy|jessie/) {
+   my @cmd = (qw(cp -dR overlay-persistent-net/.),
+  "$initrd_overlay.d/.");
+   logm("using persistent-net-generator: @cmd");
+   system_checked @cmd;
+}
+
 my $ipappend = 2;
 my $wantphysif= get_host_property($ho,'interface force','auto');
 logm("Forcing interface $wantphysif");
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 09/62] ts-debian-fixup: merge origin extra= to our own if necessary

2019-04-10 Thread Ian Jackson
From: Wei Liu 

The original extra= was not removed, so there were two extra= in the
resulting config file.

It wasn't a problem for xl because the second extra= took precedence.
However libvirt tests would only pick up the first extra= so they
worked by chance.

Fix this issue by merging the original. If there isn't already extra=
in $cfg, use our own.

Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 
---
v3: handle situation when no extra= is in $cfg
---
 ts-debian-fixup | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/ts-debian-fixup b/ts-debian-fixup
index 3e5cdb97..478dc2af 100755
--- a/ts-debian-fixup
+++ b/ts-debian-fixup
@@ -178,7 +178,11 @@ sub otherfixupcfg () {
 $extra .= " iommu=soft";
 }
 
-$cfg .= "\nextra='$extra'\n";
+if ($cfg =~ m/extra\s*=\s*['"](.*)['"]/) {
+   $cfg =~ s/extra\s*=\s*['"](.*)['"]/extra = '$1 $extra'/;
+} else {
+   $cfg .= "extra = '$extra'\n";
+}
 };
 
 sub writecfg () {
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 37/62] TestSupport: Provide target_getfile_root_stash

2019-04-10 Thread Ian Jackson
This convenient function selects a local filename based on a target
filename, and copies the target file to the selected stash file.

Signed-off-by: Ian Jackson 
---
 Osstest/TestSupport.pm | 9 +
 1 file changed, 9 insertions(+)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index a5870e4d..ec867e4f 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -61,6 +61,7 @@ BEGIN {
   target_cmd_inputfh_root sshuho
   target_getfile target_getfile_root
   target_putfile target_putfile_root
+ target_getfile_root_stash
   target_putfilecontents_stash
  target_putfilecontents_root_stash
   target_put_guest_image target_editfile
@@ -553,6 +554,14 @@ sub target_getfile_root () {
 my ($ho,$timeout, $rsrc,$ldst) = @_;
 tgetfileex('root', @_);
 }
+sub target_getfile_root_stash ($$$;$) {
+my ($ho,$timeout,$rsrc, $lleaf) = @_; # => full path of local file
+target_somefile_leaf(\$lleaf, $rsrc, $ho);
+open_unique_stashfile(\$lleaf); # discard filehandle, leave file
+my $lfile = "$stash/$lleaf";
+target_getfile_root($ho,$timeout,$rsrc,$lfile);
+return $lfile;
+}
 
 sub tputfileex {
 my ($ruser, $ho,$timeout, $lsrc,$rdst, $rsync) = @_;
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 12/62] ts-guests-nbd-mirror: make it work with stretch

2019-04-10 Thread Ian Jackson
From: Wei Liu 

On the server side, only add oldstyle= and port= on wheezy and jessie.
stretch doesn't support or need those anymore.

On the client side, generate new style configuration file.

Reorder nbd-client setup a bit. Install it first, then write our own
configuration file, then start it.  This stops dpkg asking what to
do regarding configuration files.

Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 
---
v3: invert some tests, rearrange client setup code.
v4: Fix commit message grammar.
---
 ts-guests-nbd-mirror | 47 +++
 1 file changed, 43 insertions(+), 4 deletions(-)

diff --git a/ts-guests-nbd-mirror b/ts-guests-nbd-mirror
index ca8300db..7ea31f7c 100755
--- a/ts-guests-nbd-mirror
+++ b/ts-guests-nbd-mirror
@@ -60,15 +60,19 @@ sub configserver () {
 [generic]
 user = root
 END
-$scfg .= <{Suite} =~ m/sarge|lenny|squeeze/;
+
+$scfg .= <{Suite} =~ m/wheezy|jessie/;
 oldstyle = true
 END
+
 foreach my $v (@vols) {
$v->{Port}= unique_incrementing_runvar("${srvhost}_nextport",4000);
$v->{Path}= "/dev/$v->{Gho}{Vg}/$v->{Lv}";
$scfg.=<{Ix}]
 exportname = $v->{Path}
+END
+   $scfg.=<{Suite} =~ m/wheezy|jessie/;
 port = $v->{Port}
 END
 }
@@ -79,9 +83,7 @@ END
 target_install_packages($sho, qw(nbd-server));
 }
 
-sub configclient () {
-target_cmd_root($cho, "dpkg --purge nbd-client ||:");
-
+sub configclient_pre_stretch () {
 my $mydaemon= '/root/nbd-client-async';
 target_putfilecontents_root_stash($cho,10,<<'END',$mydaemon);
 #!/bin/sh
@@ -107,7 +109,44 @@ NBD_PORT[$v->{Ix}]=$v->{Port}
 END
 }
 target_putfilecontents_root_stash($cho,10,$ccfg,"/etc/nbd-client");
+}
+
+sub configclient_stretch_and_later () {
+my $ccfg = <{Name} export$v->{Ix}
+END
+}
+
+target_putfilecontents_root_stash($cho,10,$ccfg,"/etc/nbdtab");
+}
+
+sub configclient () {
+target_cmd_root($cho, "dpkg --purge nbd-client ||:");
+
 target_install_packages($cho, qw(nbd-client));
+
+target_cmd_root($cho, "/etc/init.d/nbd-client stop ||:");
+
+if ($cho->{Suite} =~ m/wheezy|jessie/) {
+configclient_pre_stretch();
+} else {
+configclient_stretch_and_later();
+   foreach my $v (@vols) {
+   my $nbddev = "nbd$v->{Ix}";
+   target_cmd_root($cho, <{Gho}{Vg}
+if ! test -L $v->{Path}; then ln -s /dev/$nbddev $v->{Path}; fi
+END
+   }
+}
+
+target_cmd_root($cho, "/etc/init.d/nbd-client start");
 }
 
 sub shuffleconfigs () {
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 20/62] ts-guests-nbd-mirror: use target_{get, put}file_root to transfter cfg

2019-04-10 Thread Ian Jackson
From: Wei Liu 

The original code used target_cmd_output_root which caused a trailing
new line to be deleted, which caused libvirt converter to fail.

It wasn't discovered until now because we appended too many "\n".

Use target_{get,put}file_root to do the job.

Signed-off-by: Wei Liu 
---
 ts-guests-nbd-mirror | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/ts-guests-nbd-mirror b/ts-guests-nbd-mirror
index 7ea31f7c..06903aaa 100755
--- a/ts-guests-nbd-mirror
+++ b/ts-guests-nbd-mirror
@@ -154,8 +154,11 @@ sub shuffleconfigs () {
my $gn= $gns[$i];
my $gho= $ghos[$i];
my $cfgpath= $r{ "$gho->{Guest}_cfgpath" };
-   my $cfgdata= target_cmd_output_root($sho,"cat $cfgpath");
-   target_putfilecontents_root_stash($cho,10,$cfgdata,$cfgpath);
+   my $file= $cfgpath;
+   $file=~ s,/,-,g;
+   $file= "$stash/".hostnamepath($cho)."--$file";
+   target_getfile_root($sho, 60, $cfgpath, $file);
+   target_putfile_root($cho, 60, $file, $cfgpath);
 }
 }
 
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 00/62] Update to Debian stable (stretch)

2019-04-10 Thread Ian Jackson
This, finally, is the series to update osstest to Debian stretch.
This is very urgent because Debian have been removing parts of jessie
from their servers, in a rather unhelpful and not entirely controlled
manner.  Consequently, currently osstest is not reliable with jessie.

I am going to push this series to osstest pretest immediately without
waiting for acks/reviews/etc.  I expect it to trip two regressions in
the osstest push gate, which I will force push.

One effect of this series is to put into service the two ThunderX
arm64 machines, rochester[01], which are supportable in stretch but
not jessie.

There seem to be a number of outstanding problems which I decided were
not blockers:

 * test-amd64-amd64-xl-qcow2 fails because the xvd driver fails to
   freeze during the Linux guest suspend.  This seems to be a real bug
   of some kind and needs investigation.

 * I experienced difficulties with the 4 Arndale devboards: high
   probability guest start failures.  For now I have marked those
   nodes as unsuitable for use with stretch, which will, effectively,
   take them out of service - and leave us with a lack of armhf
   capacity.  It is possible that this problem is due to the
   ifupdown-hotplug issue, now addressed, so I plan to retest.

 * The serial output from the new rochester arm64 machines is missing
   some of the grub bootloader messages (and this is detected by one
   of the examine jobs).  I may need help from someone familiar with
   these machines' hardware/firmware.

 * stretch did not appear to work properly on joubertin[01] and
   merlot[01] (x86 boxes).  I have marked them to be not used with
   stretch.  They will need to be retested and the problems
   investigated (if indeed the problems still remain after other fixes
   which are now in this series).

 * I have two bug reports / patches to go upstream to Debian: a boot
   ordering bug with ifupdown and udev, and a bug in fishdescriptor.

There are a couple of occurrences in this series of changes which
break things followed by patches to fix them up.  I have not bothered
squashing/reorganising those.

Ian Jackson (41):
  power: Fix uninitialised variable warning
  Debian: Fix http:// url for bugs.xenproject.org
  contents_make_cpio: Include symlinks
  contents_make_cpio: Make files owned by root
  overlay-persistent-net: Copy from jessie
  persistent-net: Add overlay on installed systems >= stretch
  persistent-net: Add overlay in installer >= stretch
  ts-host-install: Break out $persistent_net_rules
  ts-host-install: Unconditionally mkdir -p /etc/udev/rules.d
  ts-host-install: Put canary in 70-persistent-net.rules
  persistent-net: Include initramfs script to copy to target
  persistent-net: Set net.ifnames=0 in di_installcmdline_core
  ts-kernel-build: Enable some additional drivers for Thunder-X
  TestSupport: Move `stashed files' and `next_unique_name' earlier
  TestSupport: Move `target_somefile_getleaf' earlier
  TestSupport: target_somefile_leaf rename and change a variable
  TestSupport: Provide target_getfile_root_stash
  ts-guests-nbd-mirror: Use target_getfile_root_stash
  preseed_base: chmod ssh host private keys to placate sshd
  preseed_base: break out debian_overlays_fixup_cmd
  ts-debian-fixup: Use debian_overlays_fixup_cmd
  preseed_hook_command: allow specifying di keys other than preseed/*
  preseed_hook_installscript: Use partman/early_command, not preseed/
  Debian: preseed_hook_installscript: New $atonce option
  Debian: partman scripts: Run right away too
  Debian: set partman-lvm/device_remove_lvm_span
  Debian: Add reference to bug numbers for erase-other-disks
  dm restrict audit: actually install right package for fishdescriptor
  Debian: Move preseed_backports_packages earlier
  dm restrict audit: actually install fishdescriptor in host
  dm restrict audit: always install (some) chiark-scripts
  target_install_packages: Consistently use qw(...) rather than '...'
  ts-xen-install: Install libpciaccess0
  ts-debian-hvm-install: Honour linux_boot_append target var
  make-flight: shadow test: Disable kpti in guests
  platforms: Pass suite to get_arch_platforms
  platforms: Honour suite in get_arch_platforms
  dm restrict, fishdescriptor: Update to a fixed chiark-scripts
  Debian: Fix /lib/udev/ifupdown-hotplug to not run if / is ro
  Debian: Fix /lib/udev/ifupdown-hotplug in guests from debootstrap
  production-config-cambridge: Provide TftpDiVersion_stretch

Julien Grall (2):
  stretch: Use chainloading when booting using GRUB on Arm64
  ts-xen-build: Enable ITS driver in Xen

Wei Liu (19):
  gitignore: ignore vim swap file
  ts-xen-build-prep: only install w3c-dtd-xhtml for suites jessie
  ts-xen-install: install some packages on stretch
  Debian.pm: use sysvinit-core instead of systemd
  ts-leak-check: suppress systemd-shim, which leaks in stretch
  ts-host-install: don't use the new nic naming scheme
  ts-debian-fixup: merge origin extra= to our own if necessary
  ts-debian-fixup: append 

[Xen-devel] [OSSTEST PATCH 13/62] Extend workaround `clk_ignore_unused' to stretch

2019-04-10 Thread Ian Jackson
From: Wei Liu 

This is https://bugs.xenproject.org/xen/bug/45

Without that parameter we lose uart output.

Signed-off-by: Wei Liu 
CC: Stefano Stabellini 
CC: Julien Grall 
---
 Osstest/Debian.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 82b5fb40..91bffdff 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -241,7 +241,7 @@ END
push @xenkopt, $xenkopt;
# http://bugs.xenproject.org/xen/bug/45
push @xenkopt, "clk_ignore_unused"
-   if $ho->{Suite} =~ m/wheezy|jessie/;
+   if $ho->{Suite} =~ m/wheezy|jessie|stretch/;
 
$xenkopt = join ' ', @xenkopt;
logm("Dom0 Linux options: $xenkopt");
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 03/62] ts-xen-build-prep: only install w3c-dtd-xhtml for suites

2019-04-10 Thread Ian Jackson
From: Wei Liu 

That package is not included in stretch.

That package was installed because the libvirt build needed it.
However libvirt builds fine without it in stretch.

Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 
---
 ts-xen-build-prep | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/ts-xen-build-prep b/ts-xen-build-prep
index 23bbbeb9..a15ab3df 100755
--- a/ts-xen-build-prep
+++ b/ts-xen-build-prep
@@ -206,11 +206,13 @@ sub prep () {
   libglib2.0-dev liblzma-dev pkg-config
   autoconf automake libtool xsltproc
   libxml2-utils libxml2-dev
-  libdevmapper-dev w3c-dtd-xhtml libxml-xpath-perl
-  libelf-dev
+  libdevmapper-dev libxml-xpath-perl libelf-dev
   ccache nasm checkpolicy ebtables
   libgnutls28-dev);
 
+if ($ho->{Suite} =~ m/squeeze|wheezy|jessie/) {
+   push(@packages, "w3c-dtd-xhtml");
+}
 if ($ho->{Suite} !~ m/squeeze|wheezy/) {
push(@packages, qw(ocaml-nox ocaml-findlib));
 }
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 33/62] ts-xen-build: Enable ITS driver in Xen

2019-04-10 Thread Ian Jackson
From: Julien Grall 

The ITS driver was added in Xen 4.10 as a technical preview feature.
However, it is required in order to boot Xen as Thunder-X because
PCI devices don't support legacy interrupt.

So enable CONFIG_ITS in our Xen build.

Signed-off-by: Julien Grall 
---
 ts-xen-build | 4 
 1 file changed, 4 insertions(+)

diff --git a/ts-xen-build b/ts-xen-build
index 6ddfc533..1762cd61 100755
--- a/ts-xen-build
+++ b/ts-xen-build
@@ -128,6 +128,10 @@ END
echo >>xen/.config CONFIG_EXPERT=y
echo >>xen/.config CONFIG_HVM_FEP=y
echo >>xen/.config CONFIG_VERBOSE_DEBUG=y
+   # ITS driver is required to boot the Hardware Domain
+   # on Xen. For now (Xen 4.10/4.11 at at least),
+   # will be not built by default and gated by expert mode
+   echo >>xen/.config CONFIG_HAS_ITS=y
fi
 END
);
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 01/62] gitignore: ignore vim swap file

2019-04-10 Thread Ian Jackson
From: Wei Liu 

Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 
---
 .gitignore | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.gitignore b/.gitignore
index 425506b6..f7e5b77f 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,5 +1,6 @@
 *~
 *.bak
+*.swp
 tmp
 *.tmp
 bisection.ps
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 02/62] power: Fix uninitialised variable warning

2019-04-10 Thread Ian Jackson
In
  power: Record approach used for power cycles in runvars
we introduced a reference to $r{$rv} which might be undef,
resulting in this:
  Use of uninitialized value in concatenation (.) or string at 
Osstest/TestSupport.pm line 1069.

Signed-off-by: Ian Jackson 
---
 Osstest/TestSupport.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 334cc2cb..d35a784b 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -1023,7 +1023,7 @@ sub power_reboot_attempts ($$$;$$) {
logm("power: rebooted $ho->{Name} (using $approach->{Name})");
if (defined $record_runvar_tail) {
my $rv = "$ho->{Ident}_power_${record_runvar_tail}";
-   my $newval = $r{$rv}.(!!length($r{$rv}) and ',')
+   my $newval = ($r{$rv} // '').(!!length($r{$rv}) and ',')
.$approach->{Name};
store_runvar($rv, $newval);
}
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 25/62] persistent-net: Add overlay on installed systems >= stretch

2019-04-10 Thread Ian Jackson
This prepares the ground for turning off net.ifnames.  The actual
rules are gated on net.ifnames so right now there is no change.

Signed-off-by: Ian Jackson 
---
 Osstest/Debian.pm | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 414cd897..e246c012 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -798,6 +798,13 @@ sub debian_overlays ($$) {
 };
 my $suite = $ho->{Suite};
 $maybe->('overlay');
+
+# In stretch and later, net.ifnames=1, the fdo-style `predictable
+# network device names', is the default; but it is anything but
+# predictable, so we disable it.  Instead, we restore the
+# 75-persistent-net-generator mechanism from jessie and earlier.
+$maybe->("overlay-persistent-net") if $ho->{Suite} !~ m/wheezy|jessie/;
+
 $maybe->("overlay-$suite");
 $maybe->($c{OverlayLocal}, 'overlay-local.tar');
 $maybe->("$c{OverlayLocal}-$suite", 'overlay-local-$suite.tar');
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 2/3] x86/mm: Introduce altp2m_set_entry_by_page_order

2019-04-10 Thread Alexandru Stefan ISAILA


On 10.04.2019 17:18, George Dunlap wrote:
> On 4/9/19 1:03 PM, Alexandru Stefan ISAILA wrote:
>> This patch moves common code from p2m_set_altp2m_mem_access() and
>> p2m_change_altp2m_gfn() into one function
>>
>> Signed-off-by: Alexandru Isaila 
>> ---
>>   xen/arch/x86/mm/mem_access.c |  2 +-
>>   xen/include/asm-x86/p2m.h| 11 +++
>>   2 files changed, 12 insertions(+), 1 deletion(-)
>>
>> diff --git a/xen/arch/x86/mm/mem_access.c b/xen/arch/x86/mm/mem_access.c
>> index bf67ddb15a..6a22512555 100644
>> --- a/xen/arch/x86/mm/mem_access.c
>> +++ b/xen/arch/x86/mm/mem_access.c
>> @@ -279,7 +279,7 @@ int p2m_set_altp2m_mem_access(struct domain *d, struct 
>> p2m_domain *hp2m,
>>   gfn_t gfn2 = _gfn(gfn_l & mask);
>>   mfn_t mfn2 = _mfn(mfn_x(mfn) & mask);
>>   
>> -/* Note: currently it is not safe to remap to a shared entry */
>> +/* Note: currently it is not safe to remap to a shared entry */
>>   if ( t != p2m_ram_rw )
>>   return -ESRCH;
>>   
>> diff --git a/xen/include/asm-x86/p2m.h b/xen/include/asm-x86/p2m.h
>> index 6de1546d76..90a6c135a7 100644
>> --- a/xen/include/asm-x86/p2m.h
>> +++ b/xen/include/asm-x86/p2m.h
>> @@ -467,6 +467,17 @@ static inline mfn_t altp2m_get_gfn_type_access(
>>   return mfn;
>>   }
>>   
>> +static inline int altp2m_set_entry_by_page_order(
>> +struct p2m_domain *ap2m, unsigned long gfn,  mfn_t mfn,
>> +unsigned int page_order, p2m_type_t t, p2m_access_t a)
> 
> This function doesn't seem to be called anywhere in this series.

Yes I saw that yesterday after sending the patch. The call got lost in 
the  re-base/merge process. Sorry about that, it will go on again in v4

Alex
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 2/3] x86/mm: Introduce altp2m_set_entry_by_page_order

2019-04-10 Thread George Dunlap
On 4/9/19 1:03 PM, Alexandru Stefan ISAILA wrote:
> This patch moves common code from p2m_set_altp2m_mem_access() and
> p2m_change_altp2m_gfn() into one function
> 
> Signed-off-by: Alexandru Isaila 
> ---
>  xen/arch/x86/mm/mem_access.c |  2 +-
>  xen/include/asm-x86/p2m.h| 11 +++
>  2 files changed, 12 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/x86/mm/mem_access.c b/xen/arch/x86/mm/mem_access.c
> index bf67ddb15a..6a22512555 100644
> --- a/xen/arch/x86/mm/mem_access.c
> +++ b/xen/arch/x86/mm/mem_access.c
> @@ -279,7 +279,7 @@ int p2m_set_altp2m_mem_access(struct domain *d, struct 
> p2m_domain *hp2m,
>  gfn_t gfn2 = _gfn(gfn_l & mask);
>  mfn_t mfn2 = _mfn(mfn_x(mfn) & mask);
>  
> -/* Note: currently it is not safe to remap to a shared entry */
> + /* Note: currently it is not safe to remap to a shared entry */
>  if ( t != p2m_ram_rw )
>  return -ESRCH;
>  
> diff --git a/xen/include/asm-x86/p2m.h b/xen/include/asm-x86/p2m.h
> index 6de1546d76..90a6c135a7 100644
> --- a/xen/include/asm-x86/p2m.h
> +++ b/xen/include/asm-x86/p2m.h
> @@ -467,6 +467,17 @@ static inline mfn_t altp2m_get_gfn_type_access(
>  return mfn;
>  }
>  
> +static inline int altp2m_set_entry_by_page_order(
> +struct p2m_domain *ap2m, unsigned long gfn,  mfn_t mfn,
> +unsigned int page_order, p2m_type_t t, p2m_access_t a)

This function doesn't seem to be called anywhere in this series.

 -George

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] x86 guest EOI timer issues / questions

2019-04-10 Thread Jan Beulich
All,

Andrew's observation mentioned in "[PATCH v2] x86/smt: Support for
enabling/disabling SMT at runtime" and some discussions we've already
had with him made me look into how exactly the EOI timer works. I
think there are a number of quirks. I'll enumerate all questions I've run
into below. At the end of the mail is a patch carrying out some of the
adjustments implied by a subset of these questions, plus the addition
of some debugging code. It is perhaps worthwhile to note that the
printk()-s irq_guest_eoi_timer_fn() both were observed to trigger
prior to making the non-debugging code changes. With the patch in
its current shape, I've seen trigger (every once in a while) only the
printk() added to __do_IRQ_guest().

- Why does the timer not get stopped in desc_guest_eoi() when
  ->in_flight is zero?

- Why does irq_guest_eoi_timer_fn() not re-arm the timer when
  ->in_flight is still non-zero after all the decrements? Can this case
  happen at all, considering that the only increment in
  __do_IRQ_guest() happens under the same desc lock? (If it can
  happen, I think it would be against the purpose of 37eb6d05fe,
  as this would mean the IRQ can then remain un-acked for an
  indefinite period of time, unless something else re-armed the
  timer.)

- Why does irq_guest_eoi_timer_fn() issue ->end() / set_eoi_ready()
  even when no decrement of any ->in_flight was done at all? Neither
  for ACKTYPE_UNMASK nor for ACKTYPE_EOI this looks to be fully
  race free.

- Wouldn't __do_IRQ_guest() better stop the timer early on?

- Wouldn't __do_IRQ_guest() better avoid re-programming the timer
  if no ->in_flight increment was done (which I would have thought
  shouldn't happen anyway, but which I've observed in practice)? Of
  course this would mean the timer may only be stopped when the
 first increment occurs.

- What about the timer triggering while __do_IRQ_guest() is active
  (holding the desc lock)? This looks to result in immediate expiry of
  the EOI deferral for the current interrupt instance (and I've
  observed this in practice, luckily with no other visible bad effects).

- Is it okay for fixup_eoi() to fiddle with action->cpu_eoi_map
  without holding the desc lock? (I guess being in stop-machine /
  SMP-boot context makes this acceptable.)

Commits of interest:

f3922f4084 x86: Support CPU hotplug offline
37eb6d05fe x86: Automatically EOI guest-bound interrupts if guest takes too long

Other notes / observations:
- irq_guest_eoi_timer_fn() should not need to re-acquire the lock
  after on_selected_cpus()
- irq_guest_eoi_timer_fn() should not need to use irqsave/irqrestore
  spin lock/unlock (plain irq ones should suffice in a timer handler)
- irq_guest_eoi_timer_fn() could ASSERT(action->ack_type !=
  ACKTYPE_NONE) instead of having such a conditional
- fixup_eoi() may better avoid setting all peoi[].ready and instead
  pass a "force" boolean to flush_ready_eoi()

Thoughts / opinions appreciated,
Jan

--- unstable.orig/xen/arch/x86/irq.c
+++ unstable/xen/arch/x86/irq.c
@@ -1103,7 +1103,7 @@ static void set_eoi_ready(void *data);
 static void irq_guest_eoi_timer_fn(void *data)
 {
 struct irq_desc *desc = data;
-unsigned int irq = desc - irq_desc;
+unsigned int irq = desc - irq_desc, done = 0;
 irq_guest_action_t *action;
 cpumask_t cpu_eoi_map;
 unsigned long flags;
@@ -1115,6 +1115,16 @@ static void irq_guest_eoi_timer_fn(void
 
 action = (irq_guest_action_t *)desc->action;
 
+if ( action->eoi_timer.status >= TIMER_STATUS_in_heap )
+{//temp
+ static unsigned long cnt, thr;
+ if(++cnt > thr) {
+  thr |= cnt;
+  printk("IRQ%u: i=%u a=%d n=%u\n", irq, action->in_flight, action->ack_type, 
action->nr_guests);
+ }
+goto out;
+}
+
 if ( action->ack_type != ACKTYPE_NONE )
 {
 unsigned int i;
@@ -1122,11 +1132,29 @@ static void irq_guest_eoi_timer_fn(void
 {
 struct domain *d = action->guest[i];
 unsigned int pirq = domain_irq_to_pirq(d, irq);
+
 if ( test_and_clear_bool(pirq_info(d, pirq)->masked) )
-action->in_flight--;
+++done;
 }
+action->in_flight -= done;
 }
 
+{//temp
+ static unsigned long done_cnt, done_thr;
+ static unsigned long infl_cnt, infl_thr;
+ bool log = false;
+ if(action->in_flight && ++infl_cnt > infl_thr) {
+  infl_thr |= infl_cnt;
+  log = true;
+ } else if(!done && ++done_cnt > done_thr) {
+  done_thr |= done_cnt;
+  log = true;
+ }
+ if(log)
+  printk("IRQ%u: d=%u i=%u a=%d n=%u (%06lx,%06lx)\n",  irq, done, 
action->in_flight,
+ action->ack_type, action->nr_guests, done_cnt, infl_cnt);
+}
+//todo Instead of the below, assert that ->in_flight is zero and bail if done 
is zero?
 if ( action->in_flight != 0 )
 goto out;
 
@@ -1156,6 +1184,7 @@ static void __do_IRQ_guest(int irq)
 int i, sp;
 struct pending_eoi *peoi = this_cpu(pending_eoi);
 unsigned intvector = 

Re: [Xen-devel] [PATCH] x86/mem-sharing: statically initialize audit list head and lock

2019-04-10 Thread Tamas K Lengyel
On Wed, Apr 10, 2019 at 6:17 AM Jan Beulich  wrote:
>
> >>> On 10.04.19 at 13:20,  wrote:
> > On 4/10/19 12:13 PM, Andrew Cooper wrote:
> >> On 10/04/2019 11:58, Jan Beulich wrote:
> >>> There's no need to execute any instructions for doing so.
> >>>
> >>> Signed-off-by: Jan Beulich 
> >>> ---
> >>> I wonder whether mem_sharing_init() shouldn't go away altogether then.
> >>
> >> I vote for removing it completely.  The printk is a out-of-character
> >> compared to other subsystems in Xen.
> >
> > +1
>
> In any event I'll wait for Tamas'es opinion. There might be plans to put
> further meat into the function, after all.

+1 from me too, no plans on adding stuff here.

Thanks,
Tamas

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

  1   2   >