[Xen-devel] [ovmf test] 146186: regressions - FAIL

[osstest service owner]

flight 146186 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146186/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail REGR. vs. 145767
 test-amd64-amd64-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail REGR. vs. 
145767

version targeted for testing:
 ovmf 302eb57b18d9ac70270c18a9a2a2630cba98f8ab
baseline version:
 ovmf 70911f1f4aee0366b6122f2b90d367ec0f066beb

Last test of basis   145767  2020-01-08 00:39:09 Z   10 days
Failing since145774  2020-01-08 02:50:20 Z   10 days   40 attempts
Testing same since   146186  2020-01-17 07:50:59 Z0 days1 attempts


People who touched revisions under test:
  Aaron Li 
  Albecki, Mateusz 
  Ard Biesheuvel 
  Ashish Singhal 
  Brian R Haug 
  Eric Dong 
  Fan, ZhijuX 
  Hao A Wu 
  Jason Voelz 
  Krzysztof Koch 
  Laszlo Ersek 
  Li, Aaron 
  Liming Gao 
  Mateusz Albecki 
  Michael D Kinney 
  Michael Kubacki 
  Pavana.K 
  Philippe Mathieu-Daud? 
  Philippe Mathieu-Daude 
  Siyuan Fu 
  Siyuan, Fu 
  Sudipto Paul 
  Vitaly Cheptsov 
  Vitaly Cheptsov via Groups.Io 
  Zhiguang Liu 
  Zhiju.Fan 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 fail
 test-amd64-i386-xl-qemuu-ovmf-amd64  fail



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 737 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-4.9-testing test] 146180: regressions - trouble: fail/pass/starved

[osstest service owner]

flight 146180 xen-4.9-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146180/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-ws16-amd64 16 guest-localmigrate/x10 fail REGR. vs. 
144758

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemuu-ovmf-amd64 15 guest-saverestore.2 fail in 146097 
pass in 146180
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-localmigrate/x10 fail in 146097 
pass in 146180
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-localmigrate/x10 fail in 146097 
pass in 146180
 test-amd64-i386-xl-qemuu-ovmf-amd64 10 debian-hvm-install  fail pass in 146097

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemut-ws16-amd64 18 guest-start/win.repeat fail blocked in 
144758
 test-amd64-i386-xl-qemut-ws16-amd64 16 guest-localmigrate/x10 fail in 146097 
like 144758
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 144723
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 144758
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 144758
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 144758
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop fail like 144758
 test-amd64-amd64-xl-qemut-ws16-amd64 16 guest-localmigrate/x10 fail like 144758
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx  2 hosts-allocate   starved  n/a

version targeted for testing:
 xen  cf2e9cc0ba0432f05cdca36dcd46be5fdfd7ca0c
baseline version:
 xen  43ab30b13fe8b1d5f92a9ad2ca7d61f4c77b6cac

Last test of basis   144758  2019-12-12 10:24:41 Z   36 days
Testing same since   146075  2020-01-14 

[Xen-devel] [libvirt test] 146182: tolerable all pass - PUSHED

[osstest service owner]

flight 146182 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146182/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 145969
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 145969
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-arm64-arm64-libvirt 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-qcow2 12 migrate-support-checkfail never pass
 test-arm64-arm64-libvirt-qcow2 13 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass

version targeted for testing:
 libvirt  a1cd25b919509be2645dbe6f952d5263e0d4e4e5
baseline version:
 libvirt  4a09c143f6c467230ab60c20fea560e710ddeee0

Last test of basis   145969  2020-01-11 04:18:42 Z6 days
Failing since146061  2020-01-14 04:19:22 Z3 days4 attempts
Testing same since   146182  2020-01-17 06:00:23 Z0 days1 attempts


People who touched revisions under test:
  Andrea Bolognani 
  Daniel P. Berrangé 
  Daniel Veillard 
  Han Han 
  Jiri Denemark 
  Jonathon Jongsma 
  Julio Faracco 
  Ján Tomko 
  Michal Privoznik 
  Pavel Hrdina 
  Peter Krempa 
  Thomas Huth 

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-arm64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-arm64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-arm64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmpass
 test-amd64-amd64-libvirt-xsm pass
 test-arm64-arm64-libvirt-xsm pass
 test-amd64-i386-libvirt-xsm  pass
 test-amd64-amd64-libvirt pass
 test-arm64-arm64-libvirt pass
 test-armhf-armhf-libvirt pass
 test-amd64-i386-libvirt  pass
 test-amd64-amd64-libvirt-pairpass
 test-amd64-i386-libvirt-pair pass
 test-arm64-arm64-libvirt-qcow2   pass
 test-armhf-armhf-libvirt-raw pass
 test-amd64-amd64-libvirt-vhd pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   4a09c143f6..a1cd25b919  

Re: [Xen-devel] [PATCH] xen: xen-pciback: Reset MSI-X state when exposing a device

[Chao Gao]

On Fri, Jan 17, 2020 at 01:57:43PM -0500, Rich Persaud wrote:
>On Sep 26, 2019, at 06:17, Pasi Kärkkäinen  wrote:
>> 
>> Hello Stanislav,
>> 
>>> On Fri, Sep 13, 2019 at 11:28:20PM +0800, Chao Gao wrote:
 On Fri, Sep 13, 2019 at 10:02:24AM +, Spassov, Stanislav wrote:
 On Thu, Dec 13, 2018 at 07:54, Chao Gao wrote:
> On Thu, Dec 13, 2018 at 12:54:52AM -0700, Jan Beulich wrote:
> On 13.12.18 at 04:46,  wrote:
>>> On Wed, Dec 12, 2018 at 08:21:39AM -0700, Jan Beulich wrote:
>>> On 12.12.18 at 16:18,  wrote:
> On Wed, Dec 12, 2018 at 01:51:01AM -0700, Jan Beulich wrote:
> On 12.12.18 at 08:06,  wrote:
>>> On Wed, Dec 05, 2018 at 09:01:33AM -0500, Boris Ostrovsky wrote:
 On 12/5/18 4:32 AM, Roger Pau Monné wrote:
> On Wed, Dec 05, 2018 at 10:19:17AM +0800, Chao Gao wrote:
>> I find some pass-thru devices don't work any more across guest 
>> reboot.
>> Assigning it to another guest also meets the same issue. And the 
>> only
>> way to make it work again is un-binding and binding it to 
>> pciback.
>> Someone reported this issue one year ago [1]. More detail also 
>> can be
>> found in [2].
>> 
>> The root-cause is Xen's internal MSI-X state isn't reset properly
>> during reboot or re-assignment. In the above case, Xen set 
>> maskall bit
>> to mask all MSI interrupts after it detected a potential security
>> issue. Even after device reset, Xen didn't reset its internal 
>> maskall
>> bit. As a result, maskall bit would be set again in next write to
>> MSI-X message control register.
>> 
>> Given that PHYSDEVOPS_prepare_msix() also triggers Xen resetting 
>> MSI-X
>> internal state of a device, we employ it to fix this issue 
>> rather than
>> introducing another dedicated sub-hypercall.
>> 
>> Note that PHYSDEVOPS_release_msix() will fail if the mapping 
>> between
>> the device's msix and pirq has been created. This limitation 
>> prevents
>> us calling this function when detaching a device from a guest 
>> during
>> guest shutdown. Thus it is called right before calling
>> PHYSDEVOPS_prepare_msix().
> s/PHYSDEVOPS/PHYSDEVOP/ (no final S). And then I would also drop 
> the
> () at the end of the hypercall name since it's not a function.
> 
> I'm also wondering why the release can't be done when the device 
> is
> detached from the guest (or the guest has been shut down). This 
> makes
> me worry about the raciness of the attach/detach procedure: if 
> there's
> a state where pciback assumes the device has been detached from 
> the
> guest, but there are still pirqs bound, an attempt to attach to
> another guest in such state will fail.
 
 I wonder whether this additional reset functionality could be done 
 out
 of xen_pcibk_xenbus_remove(). We first do a (best effort) device 
 reset
 and then do the extra things that are not properly done there.
>>> 
>>> No. It cannot be done in xen_pcibk_xenbus_remove() without modifying
>>> the handler of PHYSDEVOP_release_msix. To do a successful Xen 
>>> internal
>>> MSI-X state reset, PHYSDEVOP_{release, prepare}_msix should be 
>>> finished
>>> without error. But ATM, xen expects that no msi is bound to pirq 
>>> when
>>> doing PHYSDEVOP_release_msix. Otherwise it fails with error code 
>>> -EBUSY.
>>> However, the expectation isn't guaranteed in 
>>> xen_pcibk_xenbus_remove().
>>> In some cases, if qemu fails to unmap MSIs, MSIs are unmapped by Xen
>>> at last minute, which happens after device reset in 
>>> xen_pcibk_xenbus_remove().
>> 
>> But that may need taking care of: I don't think it is a good idea to 
>> have
>> anything left from the prior owning domain when the device gets 
>> reset.
>> I.e. left over IRQ bindings should perhaps be forcibly cleared before
>> invoking the reset;
> 
> Agree. How about pciback to track the established IRQ bindings? Then
> pciback can clear irq binding before invoking the reset.
 
 How would pciback even know of those mappings, when it's qemu
 who establishes (and manages) them?
>>> 
>>> I meant to expose some interfaces from pciback. And pciback serves
>>> as the proxy of IRQ 

[Xen-devel] [xen-unstable-smoke test] 146209: tolerable all pass - PUSHED

[osstest service owner]

flight 146209 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146209/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  17a6c03701bf65c0b4e8b5ed5a3970cd0248c47f
baseline version:
 xen  32772fbb3cf7498817304b53b087e325c6991716

Last test of basis   146202  2020-01-17 16:00:42 Z0 days
Testing same since   146209  2020-01-17 20:01:06 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Jan Beulich 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   32772fbb3c..17a6c03701  17a6c03701bf65c0b4e8b5ed5a3970cd0248c47f -> smoke

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 2/2] xen/arm: sign extend writes to TimerValue

[Jeff Kubascik]

On 12/18/2019 9:24 AM, Julien Grall wrote:
> Hi Jeff,
> 
> On 11/12/2019 21:13, Jeff Kubascik wrote:
>> Per the ARMv8 Reference Manual (ARM DDI 0487E.a), section D11.2.4
>> specifies that the values in the TimerValue view of the timers are
>> signed in standard two's complement form. When writing to the TimerValue
> 
> Do you mean CompareValue register instead of TimerValue register?

I'm fairly certain TimerValue register is correct. When the guest writes to the
TimerValue register, the equation below is used to convert it to a CompareValue
equivalent.

>> register, it should be signed extended as described by the equation
>>
>> CompareValue = (Counter[63:0] + SignExtend(TimerValue))[63:0]
> This explains the signed part, but it does not explain why the 32-bit
> case. So I would mention that TimerValue is a 32-bit signed integer.
> 
> Maybe saying "are 32-bit signed in standard ..."

I pulled this equation directly from the ARMv8 Reference Manual - the manual
goes into detail about the sign extension. This is referenced earlier in the
commit message.

>>
>> Signed-off-by: Jeff Kubascik 
>> ---
>>   xen/arch/arm/vtimer.c | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/xen/arch/arm/vtimer.c b/xen/arch/arm/vtimer.c
>> index 21b98ec20a..872181d9b6 100644
>> --- a/xen/arch/arm/vtimer.c
>> +++ b/xen/arch/arm/vtimer.c
>> @@ -211,7 +211,7 @@ static bool vtimer_cntp_tval(struct cpu_user_regs *regs, 
>> uint32_t *r,
>>   }
>>   else
>>   {
>> -v->arch.phys_timer.cval = cntpct + *r;
>> +v->arch.phys_timer.cval = cntpct + (uint64_t)(int32_t)*r;
>>   if ( v->arch.phys_timer.ctl & CNTx_CTL_ENABLE )
>>   {
>>   v->arch.phys_timer.ctl &= ~CNTx_CTL_PENDING;
>>
> 
> Cheers,
> 
> --
> Julien Grall
> 

Sincerely,
Jeff Kubascik

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 1/2] xen/arm: remove physical timer offset

[Jeff Kubascik]

On 12/18/2019 9:20 AM, Julien Grall wrote:
> Hi Jeff,
> 
> On 11/12/2019 21:13, Jeff Kubascik wrote:
>> The physical timer traps apply an offset so that time starts at 0 for
>> the guest. However, this offset is not currently applied to the physical
>> counter. Per the ARMv8 Reference Manual (ARM DDI 0487E.a), section
>> D11.2.4 Timers, the "Offset" between the counter and timer should be
>> zero for a physical timer. This removes the offset to make the timer and
>> counter consistent.
>>
>> This also cleans up the physical timer implementation to better match
>> the virtual timer - both cval's now hold the hardware value.
>>
>> Signed-off-by: Jeff Kubascik 
>> ---
>>   xen/arch/arm/vtimer.c| 34 ++
>>   xen/include/asm-arm/domain.h |  3 ---
>>   2 files changed, 18 insertions(+), 19 deletions(-)
>>
>> diff --git a/xen/arch/arm/vtimer.c b/xen/arch/arm/vtimer.c
>> index e6aebdac9e..21b98ec20a 100644
>> --- a/xen/arch/arm/vtimer.c
>> +++ b/xen/arch/arm/vtimer.c
>> @@ -62,7 +62,6 @@ static void virt_timer_expired(void *data)
>>
>>   int domain_vtimer_init(struct domain *d, struct xen_arch_domainconfig 
>> *config)
>>   {
>> -d->arch.phys_timer_base.offset = NOW();
>>   d->arch.virt_timer_base.offset = READ_SYSREG64(CNTPCT_EL0);
>>   d->time_offset_seconds = ticks_to_ns(d->arch.virt_timer_base.offset - 
>> boot_count);
>>   do_div(d->time_offset_seconds, 10);
>> @@ -108,7 +107,6 @@ int vcpu_vtimer_init(struct vcpu *v)
>>
>>   init_timer(>timer, phys_timer_expired, t, v->processor);
>>   t->ctl = 0;
>> -t->cval = NOW();
>>   t->irq = d0
>>   ? timer_get_irq(TIMER_PHYS_NONSECURE_PPI)
>>   : GUEST_TIMER_PHYS_NS_PPI;
>> @@ -167,6 +165,7 @@ void virt_timer_restore(struct vcpu *v)
>>   static bool vtimer_cntp_ctl(struct cpu_user_regs *regs, uint32_t *r, bool 
>> read)
>>   {
>>   struct vcpu *v = current;
>> +s_time_t expires;
>>
>>   if ( !ACCESS_ALLOWED(regs, EL0PTEN) )
>>   return false;
>> @@ -184,8 +183,9 @@ static bool vtimer_cntp_ctl(struct cpu_user_regs *regs, 
>> uint32_t *r, bool read)
>>
>>   if ( v->arch.phys_timer.ctl & CNTx_CTL_ENABLE )
>>   {
>> -set_timer(>arch.phys_timer.timer,
>> -  v->arch.phys_timer.cval + 
>> v->domain->arch.phys_timer_base.offset);
>> +expires = v->arch.phys_timer.cval > boot_count
>> +  ? ticks_to_ns(v->arch.phys_timer.cval - boot_count) : 
>> 0;
>> +set_timer(>arch.phys_timer.timer, expires);
>>   }
>>   else
>>   stop_timer(>arch.phys_timer.timer);
>> @@ -197,26 +197,27 @@ static bool vtimer_cntp_tval(struct cpu_user_regs 
>> *regs, uint32_t *r,
>>bool read)
>>   {
>>   struct vcpu *v = current;
>> -s_time_t now;
>> +uint64_t cntpct;
>> +s_time_t expires;
>>
>>   if ( !ACCESS_ALLOWED(regs, EL0PTEN) )
>>   return false;
>>
>> -now = NOW() - v->domain->arch.phys_timer_base.offset;
>> +cntpct = get_cycles();
>>
>>   if ( read )
>>   {
>> -*r = (uint32_t)(ns_to_ticks(v->arch.phys_timer.cval - now) & 
>> 0xull);
>> +*r = (uint32_t)((v->arch.phys_timer.cval - cntpct) & 0xull);
>>   }
>>   else
>>   {
>> -v->arch.phys_timer.cval = now + ticks_to_ns(*r);
>> +v->arch.phys_timer.cval = cntpct + *r;
>>   if ( v->arch.phys_timer.ctl & CNTx_CTL_ENABLE )
>>   {
>>   v->arch.phys_timer.ctl &= ~CNTx_CTL_PENDING;
>> -set_timer(>arch.phys_timer.timer,
>> -  v->arch.phys_timer.cval +
>> -  v->domain->arch.phys_timer_base.offset);
>> +expires = v->arch.phys_timer.cval > boot_count
>> +  ? ticks_to_ns(v->arch.phys_timer.cval - boot_count) : 
>> 0;
> 
> You probably want a comment to explain why you set to 0 here.

This code is repeated in 3 places - it probably doesn't make sense to have 3
duplicate comments as well. I think it would fit well with your function
suggestion below.

>> +set_timer(>arch.phys_timer.timer, expires);
>>   }
>>   }
>>   return true;
>> @@ -226,23 +227,24 @@ static bool vtimer_cntp_cval(struct cpu_user_regs 
>> *regs, uint64_t *r,
>>bool read)
>>   {
>>   struct vcpu *v = current;
>> +s_time_t expires;
>>
>>   if ( !ACCESS_ALLOWED(regs, EL0PTEN) )
>>   return false;
>>
>>   if ( read )
>>   {
>> -*r = ns_to_ticks(v->arch.phys_timer.cval);
>> +*r = v->arch.phys_timer.cval;
>>   }
>>   else
>>   {
>> -v->arch.phys_timer.cval = ticks_to_ns(*r);
>> +v->arch.phys_timer.cval = *r;
>>   if ( v->arch.phys_timer.ctl & CNTx_CTL_ENABLE )
>>   {
>>   v->arch.phys_timer.ctl &= ~CNTx_CTL_PENDING;
>> -set_timer(>arch.phys_timer.timer,
>> -

Re: [Xen-devel] [PATCH v2 2/4] x86/microcode: avoid unnecessary xmalloc/memcpy of ucode data

[Andrew Cooper]

On 20/12/2019 09:57, Jan Beulich wrote:
> On 19.12.2019 22:25, Eslam Elnikety wrote:
>> On 18.12.19 13:05, Jan Beulich wrote:
>>> On 18.12.2019 02:32, Eslam Elnikety wrote:
 @@ -725,7 +701,7 @@ static int __init microcode_init(void)
*/
   if ( ucode_blob.size )
   {
 -xfree(ucode_blob.data);
 +bootstrap_map(NULL);
>>> As much as I like the change, I wholeheartedly disagree with this
>>> aspect of it: You make it largely unpredictable when the boot
>>> mappings will go away - it becomes entirely dependent upon link
>>> order. And of course we really want these mappings to be gone,
>>> the very latest (I think), by the time we start bringing up APs
>>> (but generally the sooner the better). This is (one of?) the main
>>> reason(s) why it hadn't been done this way to begin with. The
>>> alternative is more complicated (set up a proper, long term
>>> mapping), but it's going to be more clean (including the mapping
>>> then also being suitable to post-boot CPU onlining).
>>>
>> This change is an improvement on the current status. We get to avoid 
>> xmalloc/memcpy in the case of a successful ucode=scan. The problematic 
>> aspect you highlight is anyway there regardless of this patch (ref. to 
>> the "else if ( ucode_mod.mod_end )" in microcode_init).
> Hmm, fair point. I'm not a fan of making a bad situation worse,
> but I think it's acceptable here:
> Acked-by: Jan Beulich 

Specifically relevant to this conversation is point 2 of
https://lore.kernel.org/xen-devel/20200109193241.14542-1-andrew.coop...@citrix.com/
where having dynamic bootmap mappings seems pointless when all we're
doing is mapping RAM below 4G.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 4/5] x86/boot: Simplify pagetable manipulation loops

[Andrew Cooper]

For __page_tables_{start,end} and L3 bootmap initialisation, the logic is
unnecesserily complicated owing to its attempt to use the LOOP instruction,
which results in an off-by-8 memory address owing to LOOP's termination
condition.

Rewrite both loops for improved clarity and speed.

Misc notes:
 * TEST $IMM, MEM can't macrofuse.  The loop has 0x1200 iterations, so pull
   the $_PAGE_PRESENT constant out into a spare register to turn the TEST into
   its %REG, MEM form, which can macrofuse.
 * Avoid the use of %fs-relative references.  %esi-relative is the more common
   form in the code, and doesn't suffer an address generation overhead.
 * Avoid LOOP.  CMP/JB isn't microcoded and faster to execute in all cases.
 * For a 4 interation trivial loop, even compilers unroll these.  The
   generated code size is a fraction larger, but this is init and the asm is
   far easier to follow.
 * Reposition the l2=>l1 bootmap construction so the asm reads in pagetable
   level order.

No functional change.

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
CC: Wei Liu 
CC: Roger Pau Monné 
---
 xen/arch/x86/boot/head.S | 41 ++---
 1 file changed, 26 insertions(+), 15 deletions(-)

diff --git a/xen/arch/x86/boot/head.S b/xen/arch/x86/boot/head.S
index 1deeae2f2a..1acaf817ba 100644
--- a/xen/arch/x86/boot/head.S
+++ b/xen/arch/x86/boot/head.S
@@ -662,11 +662,17 @@ trampoline_setup:
 mov %edx,sym_fs(boot_tsc_stamp)+4
 
 /* Relocate pagetables to point at Xen's current location in memory. */
-mov $((__page_tables_end-__page_tables_start)/8),%ecx
-1:  testl   $_PAGE_PRESENT,sym_fs(__page_tables_start)-8(,%ecx,8)
+mov $_PAGE_PRESENT, %edx
+lea sym_esi(__page_tables_start), %eax
+lea sym_esi(__page_tables_end), %edi
+
+1:  testb   %dl, (%eax)  /* if page present */
 jz  2f
-add %esi,sym_fs(__page_tables_start)-8(,%ecx,8)
-2:  loop1b
+add %esi, (%eax) /* pte += base */
+2:  add $8, %eax
+
+cmp %edi, %eax
+jb  1b
 
 /* Map Xen into the higher mappings using 2M superpages. */
 lea _PAGE_PSE + PAGE_HYPERVISOR_RWX + sym_esi(_start), %eax
@@ -701,22 +707,27 @@ trampoline_setup:
 cmp %edx, %ecx
 jbe 1b
 
-/* Initialize L3 boot-map page directory entries. */
-lea 
__PAGE_HYPERVISOR+(L2_PAGETABLE_ENTRIES*8)*3+sym_esi(l2_bootmap),%eax
-mov $4,%ecx
-1:  mov %eax,sym_fs(l3_bootmap)-8(,%ecx,8)
-sub $(L2_PAGETABLE_ENTRIES*8),%eax
-loop1b
-
-/* Map the permanent trampoline page into l{1,2}_bootmap[]. */
+/* Map 4x l2_bootmap[] into l3_bootmap[0...3] */
+lea __PAGE_HYPERVISOR + sym_esi(l2_bootmap), %eax
+mov $PAGE_SIZE, %edx
+mov %eax, 0  + sym_esi(l3_bootmap)
+add %edx, %eax
+mov %eax, 8  + sym_esi(l3_bootmap)
+add %edx, %eax
+mov %eax, 16 + sym_esi(l3_bootmap)
+add %edx, %eax
+mov %eax, 24 + sym_esi(l3_bootmap)
+
+/* Map l1_bootmap[] into l2_bootmap[0]. */
+lea __PAGE_HYPERVISOR + sym_esi(l1_bootmap), %eax
+mov %eax, sym_esi(l2_bootmap)
+
+/* Map the permanent trampoline page into l1_bootmap[]. */
 mov sym_esi(trampoline_phys), %ecx
 lea __PAGE_HYPERVISOR_RX(%ecx), %edx /* %edx = PTE to write  */
 shr $PAGE_SHIFT, %ecx/* %ecx = Slot to write */
 mov %edx, sym_offs(l1_bootmap)(%esi, %ecx, 8)
 
-lea __PAGE_HYPERVISOR + sym_esi(l1_bootmap), %edx
-mov %edx, sym_esi(l2_bootmap)
-
 /* Apply relocations to bootstrap trampoline. */
 mov sym_esi(trampoline_phys), %edx
 lea sym_esi(__trampoline_rel_start), %edi
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 2/5] x86/boot: Size the boot/directmap mappings dynamically

[Andrew Cooper]

... rather than presuming that 16M will do.  On the EFI side, use
l2e_add_flags() to reduce the code-generation overhead of using
l2e_from_paddr() twice.

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
CC: Wei Liu 
CC: Roger Pau Monné 

v2:
 * Drop adjustment to the linker script.  There are more 16M issues to find.
---
 xen/arch/x86/boot/head.S| 21 +
 xen/arch/x86/efi/efi-boot.h | 23 ++-
 2 files changed, 31 insertions(+), 13 deletions(-)

diff --git a/xen/arch/x86/boot/head.S b/xen/arch/x86/boot/head.S
index ef9f562505..0137ee99a4 100644
--- a/xen/arch/x86/boot/head.S
+++ b/xen/arch/x86/boot/head.S
@@ -687,14 +687,19 @@ trampoline_setup:
  * handling/walking), and identity map Xen into bootmap (needed for
  * the transition into long mode), using 2M superpages.
  */
-lea sym_esi(start),%ebx
-lea 
(1<> L2_PAGETABLE_SHIFT) & (4 * L2_PAGETABLE_ENTRIES - 1))
+
+for ( i  = l2_4G_offset(_start);
+  i <= l2_4G_offset(_end - 1); ++i )
 {
-unsigned int slot = (xen_phys_start >> L2_PAGETABLE_SHIFT) + i;
-paddr_t addr = slot << L2_PAGETABLE_SHIFT;
+l2_pgentry_t pte = l2e_from_paddr(i << L2_PAGETABLE_SHIFT,
+  __PAGE_HYPERVISOR | _PAGE_PSE);
+
+l2_bootmap[i] = pte;
+
+/* Bootmap RWX/Non-global.  Directmap RW/Global. */
+l2e_add_flags(pte, PAGE_HYPERVISOR);
 
-l2_directmap[slot] = l2e_from_paddr(addr, PAGE_HYPERVISOR|_PAGE_PSE);
-l2_bootmap[slot] = l2e_from_paddr(addr, __PAGE_HYPERVISOR|_PAGE_PSE);
+l2_directmap[i] = pte;
 }
+#undef l2_4G_offset
 }
 
 static void __init efi_arch_handle_module(struct file *file, const CHAR16 
*name,
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 1/5] x86/boot: Create the l2_xenmap[] mappings dynamically

[Andrew Cooper]

The build-time construction of l2_xenmap[] imposes an arbitrary limit of 16M
total, which is a limit looking to be lifted.

Move l2_xenmap[] into the BSS, and adjust both the BIOS and EFI paths to fill
it in dynamically, based on the final linked size of Xen.

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
CC: Wei Liu 
CC: Roger Pau Monné 

v2:
 * Rewrite several comments
---
 xen/arch/x86/boot/head.S| 14 ++
 xen/arch/x86/boot/x86_64.S  | 23 ---
 xen/arch/x86/efi/efi-boot.h | 14 ++
 xen/arch/x86/xen.lds.S  |  3 +++
 4 files changed, 39 insertions(+), 15 deletions(-)

diff --git a/xen/arch/x86/boot/head.S b/xen/arch/x86/boot/head.S
index c5acbf56ae..ef9f562505 100644
--- a/xen/arch/x86/boot/head.S
+++ b/xen/arch/x86/boot/head.S
@@ -668,6 +668,20 @@ trampoline_setup:
 add %esi,sym_fs(__page_tables_start)-8(,%ecx,8)
 2:  loop1b
 
+/* Map Xen into the higher mappings using 2M superpages. */
+lea _PAGE_PSE + PAGE_HYPERVISOR_RWX + sym_esi(_start), %eax
+mov $sym_offs(_start),   %ecx   /* %eax = PTE to write ^  */
+mov $sym_offs(_end - 1), %edx
+shr $L2_PAGETABLE_SHIFT, %ecx   /* %ecx = First slot to write */
+shr $L2_PAGETABLE_SHIFT, %edx   /* %edx = Final slot to write */
+
+1:  mov %eax, sym_offs(l2_xenmap)(%esi, %ecx, 8)
+add $1, %ecx
+add $1 << L2_PAGETABLE_SHIFT, %eax
+
+cmp %edx, %ecx
+jbe 1b
+
 /*
  * Map Xen into the directmap (needed for early-boot pagetable
  * handling/walking), and identity map Xen into bootmap (needed for
diff --git a/xen/arch/x86/boot/x86_64.S b/xen/arch/x86/boot/x86_64.S
index aabf561b23..e63bece460 100644
--- a/xen/arch/x86/boot/x86_64.S
+++ b/xen/arch/x86/boot/x86_64.S
@@ -43,6 +43,14 @@ multiboot_ptr:
 GLOBAL(stack_start)
 .quad   cpu0_stack + STACK_SIZE - CPUINFO_sizeof
 
+.section .bss.page_aligned, "aw", @nobits
+.align PAGE_SIZE, 0
+
+/* L2 mapping the Xen text/data/bss region.  Uses 1x 4k page. */
+GLOBAL(l2_xenmap)
+.fill L2_PAGETABLE_ENTRIES, 8, 0
+.size l2_xenmap, . - l2_xenmap
+
 .section .data.page_aligned, "aw", @progbits
 .align PAGE_SIZE, 0
 /*
@@ -80,21 +88,6 @@ GLOBAL(l2_directmap)
 .fill 4 * L2_PAGETABLE_ENTRIES - 1, 8, 0
 .size l2_directmap, . - l2_directmap
 
-/*
- * L2 mapping the 1GB Xen text/data/bss region.  At boot it maps 16MB from
- * __image_base__, and is modified when Xen relocates itself.  Uses 1x 4k
- * page.
- */
-GLOBAL(l2_xenmap)
-.quad 0
-idx = 1
-.rept 7
-.quad sym_offs(__image_base__) + (idx << L2_PAGETABLE_SHIFT) + 
(PAGE_HYPERVISOR_RWX | _PAGE_PSE)
-idx = idx + 1
-.endr
-.fill L2_PAGETABLE_ENTRIES - 8, 8, 0
-.size l2_xenmap, . - l2_xenmap
-
 /* L2 mapping the fixmap.  Uses 1x 4k page. */
 l2_fixmap:
 idx = 0
diff --git a/xen/arch/x86/efi/efi-boot.h b/xen/arch/x86/efi/efi-boot.h
index 50d1499867..ce07aedf45 100644
--- a/xen/arch/x86/efi/efi-boot.h
+++ b/xen/arch/x86/efi/efi-boot.h
@@ -585,6 +585,20 @@ static void __init efi_arch_memory_setup(void)
 if ( !efi_enabled(EFI_LOADER) )
 return;
 
+/*
+ * Map Xen into the higher mappings, using 2M superpages.
+ *
+ * NB: We are currently in physical mode, so a RIP-relative relocation
+ * against _start/_end result in our arbitrary placement by the bootloader
+ * in memory, rather than the intended high mappings position.  Subtract
+ * xen_phys_start to get the appropriate slots in l2_xenmap[].
+ */
+for ( i =  l2_table_offset((UINTN)_start   - xen_phys_start);
+  i <= l2_table_offset((UINTN)_end - 1 - xen_phys_start); ++i )
+l2_xenmap[i] =
+l2e_from_paddr(xen_phys_start + (i << L2_PAGETABLE_SHIFT),
+   PAGE_HYPERVISOR_RWX | _PAGE_PSE);
+
 /* Check that there is at least 4G of mapping space in l2_*map[] */
 BUILD_BUG_ON((sizeof(l2_bootmap)   / L2_PAGETABLE_ENTRIES) < 4);
 BUILD_BUG_ON((sizeof(l2_directmap) / L2_PAGETABLE_ENTRIES) < 4);
diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S
index 29ef507432..07c6448dbb 100644
--- a/xen/arch/x86/xen.lds.S
+++ b/xen/arch/x86/xen.lds.S
@@ -360,6 +360,9 @@ ASSERT(__2M_rwdata_end <= XEN_VIRT_END - XEN_VIRT_START + 
__XEN_VIRT_START -
 ASSERT(kexec_reloc_size - kexec_reloc <= PAGE_SIZE, "kexec_reloc is too large")
 #endif
 
+/* The Multiboot setup paths relies on this to simplify superpage PTE 
creation. */
+ASSERT(IS_ALIGNED(_start,MB(2)), "_start misaligned")
+
 ASSERT(IS_ALIGNED(__2M_text_end, SECTION_ALIGN), "__2M_text_end 
misaligned")
 ASSERT(IS_ALIGNED(__2M_rodata_start, SECTION_ALIGN), "__2M_rodata_start 
misaligned")
 ASSERT(IS_ALIGNED(__2M_rodata_end,   SECTION_ALIGN), "__2M_rodata_end 
misaligned")
-- 
2.11.0

[Xen-devel] [PATCH v2 3/5] x86/boot: Drop explicit %fs uses

[Andrew Cooper]

The trampoline relocation code uses %fs for accessing Xen, and this comes with
an arbitrary 16M limitation.  We could adjust the limit, but the boot code is
a confusing mix of %ds/%esi-based and %fs-based accesses, and the use of %fs
is longer to encode, and incurs an address generation overhead.

Rewrite the logic to use %ds, for better consistency with the surrounding
code, and a marginal performance improvement.

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
CC: Wei Liu 
CC: Roger Pau Monné 
---
 xen/arch/x86/boot/head.S | 26 +++---
 1 file changed, 15 insertions(+), 11 deletions(-)

diff --git a/xen/arch/x86/boot/head.S b/xen/arch/x86/boot/head.S
index 0137ee99a4..1deeae2f2a 100644
--- a/xen/arch/x86/boot/head.S
+++ b/xen/arch/x86/boot/head.S
@@ -718,23 +718,27 @@ trampoline_setup:
 mov %edx, sym_esi(l2_bootmap)
 
 /* Apply relocations to bootstrap trampoline. */
-mov sym_fs(trampoline_phys),%edx
-mov $sym_offs(__trampoline_rel_start),%edi
+mov sym_esi(trampoline_phys), %edx
+lea sym_esi(__trampoline_rel_start), %edi
+lea sym_esi(__trampoline_rel_stop), %ecx
 1:
-mov %fs:(%edi),%eax
-add %edx,%fs:(%edi,%eax)
+mov (%edi), %eax
+add %edx, (%edi, %eax)
 add $4,%edi
-cmp $sym_offs(__trampoline_rel_stop),%edi
+
+cmp %ecx, %edi
 jb  1b
 
 /* Patch in the trampoline segment. */
 shr $4,%edx
-mov $sym_offs(__trampoline_seg_start),%edi
+lea sym_esi(__trampoline_seg_start), %edi
+lea sym_esi(__trampoline_seg_stop), %ecx
 1:
-mov %fs:(%edi),%eax
-mov %dx,%fs:(%edi,%eax)
+mov (%edi), %eax
+mov %dx, (%edi, %eax)
 add $4,%edi
-cmp $sym_offs(__trampoline_seg_stop),%edi
+
+cmp %ecx, %edi
 jb  1b
 
 /* Do not parse command line on EFI platform here. */
@@ -760,9 +764,9 @@ trampoline_setup:
 push%eax
 
 /* Copy bootstrap trampoline to low memory, below 1MB. */
-mov $sym_offs(trampoline_start),%esi
+lea sym_esi(trampoline_start), %esi
 mov $((trampoline_end - trampoline_start) / 4),%ecx
-rep movsl %fs:(%esi),%es:(%edi)
+rep movsl
 
 /* Jump into the relocated trampoline. */
 lret
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 0/5] x86: Remove more 16M total-size restrictions

[Andrew Cooper]

The boot/directmap pagetables, high Xen pagetables, and use of BOOT_FS all
come with arbitrary limitations to Xen's total size.  Remove these limits.

Testing of the EFI build indicates that there is still an issue lurking
somewhere:

  (XEN) [ Xen-4.14-unstable  x86_64  debug=y   Not tainted ]
  (XEN) CPU:0
  (XEN) RIP:e008:[] 
drivers/char/ns16550.c#ns16550_poll+0x1d/0x21
  ...
  (XEN) Xen call trace:
  (XEN)[] R drivers/char/ns16550.c#ns16550_poll+0x1d/0x21
  (XEN)[] F common/timer.c#execute_timer+0x49/0x64
  (XEN)[] F common/timer.c#timer_softirq_action+0xa2/0x276
  (XEN)[] F common/softirq.c#__do_softirq+0x71/0x9a
  (XEN)[] F process_pending_softirqs+0x33/0x37
  (XEN)[] F __cpu_up+0x652/0x719
  (XEN)[] F cpu_up+0x75/0xe3
  (XEN)[] F __start_xen+0x251a/0x29b1
  (XEN)
  (XEN)
  (XEN) 
  (XEN) Panic on CPU 0:
  (XEN) FATAL TRAP: vector = 6 (invalid opcode)
  (XEN) 

which is run_in_exception_handler() not being recognised as BUGFRAME_run_fn.
Therefore, I've left the linker assert in place for now.

Andrew Cooper (5):
  x86/boot: Create the l2_xenmap[] mappings dynamically
  x86/boot: Size the boot/directmap mappings dynamically
  x86/boot: Drop explicit %fs uses
  x86/boot: Simplify pagetable manipulation loops
  x86/boot: Drop sym_fs()

 xen/arch/x86/boot/head.S   | 145 +++--
 xen/arch/x86/boot/trampoline.S |   1 -
 xen/arch/x86/boot/x86_64.S |  23 +++
 xen/arch/x86/efi/efi-boot.h|  37 +--
 xen/arch/x86/xen.lds.S |   3 +
 5 files changed, 126 insertions(+), 83 deletions(-)

-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 5/5] x86/boot: Drop sym_fs()

[Andrew Cooper]

All remaining users of sym_fs() can trivially be switched to using sym_esi()
instead.  This is shorter to encode and faster to execute.

This removes the final uses of %fs during boot, which allows us to drop
BOOT_FS from the trampoline GDT, which drops an 16M arbitrary limit on Xen's
compiled size.

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
CC: Wei Liu 
CC: Roger Pau Monné 
---
 xen/arch/x86/boot/head.S   | 41 ++---
 xen/arch/x86/boot/trampoline.S |  1 -
 2 files changed, 14 insertions(+), 28 deletions(-)

diff --git a/xen/arch/x86/boot/head.S b/xen/arch/x86/boot/head.S
index 1acaf817ba..aea6744c80 100644
--- a/xen/arch/x86/boot/head.S
+++ b/xen/arch/x86/boot/head.S
@@ -14,14 +14,12 @@
 
 #define sym_offs(sym) ((sym) - __XEN_VIRT_START)
 #define sym_esi(sym)  sym_offs(sym)(%esi)
-#define sym_fs(sym)   %fs:sym_offs(sym)
 
 #define BOOT_CS320x0008
 #define BOOT_CS640x0010
 #define BOOT_DS  0x0018
 #define BOOT_PSEUDORM_CS 0x0020
 #define BOOT_PSEUDORM_DS 0x0028
-#define BOOT_FS  0x0030
 
 #define MB2_HT(name)  (MULTIBOOT2_HEADER_TAG_##name)
 #define MB2_TT(name)  (MULTIBOOT2_TAG_TYPE_##name)
@@ -555,24 +553,13 @@ trampoline_bios_setup:
 trampoline_setup:
 /*
  * Called on legacy BIOS and EFI platforms.
- *
- * Set the BOOT_FS descriptor base address to %esi.
  */
-mov %esi, %edx
-shr $16, %edx
-mov %si, BOOT_FS + 2 + sym_esi(trampoline_gdt) /* Bits  0-15 */
-mov %dl, BOOT_FS + 4 + sym_esi(trampoline_gdt) /* Bits 16-23 */
-mov %dh, BOOT_FS + 7 + sym_esi(trampoline_gdt) /* Bits 24-31 */
-
-/* Load %fs to allow for access to Xen data. */
-mov $BOOT_FS, %edx
-mov %edx, %fs
 
 /* Save Xen image load base address for later use. */
-mov %esi,sym_fs(xen_phys_start)
-mov %esi,sym_fs(trampoline_xen_phys_start)
+mov %esi, sym_esi(xen_phys_start)
+mov %esi, sym_esi(trampoline_xen_phys_start)
 
-mov sym_fs(trampoline_phys),%ecx
+mov sym_esi(trampoline_phys), %ecx
 
 /* Get bottom-most low-memory stack address. */
 add $TRAMPOLINE_SPACE,%ecx
@@ -583,13 +570,13 @@ trampoline_setup:
 push%eax/* Magic number. */
 callreloc
 #ifdef CONFIG_PVH_GUEST
-cmpb$0, sym_fs(pvh_boot)
+cmpb$0, sym_esi(pvh_boot)
 je  1f
-mov %eax, sym_fs(pvh_start_info_pa)
+mov %eax, sym_esi(pvh_start_info_pa)
 jmp 2f
 #endif
 1:
-mov %eax, sym_fs(multiboot_ptr)
+mov %eax, sym_esi(multiboot_ptr)
 2:
 
 /*
@@ -613,7 +600,7 @@ trampoline_setup:
  * Do not zero BSS on EFI platform here.
  * It was initialized earlier.
  */
-cmpb$0,sym_fs(efi_platform)
+cmpb$0, sym_esi(efi_platform)
 jnz 1f
 
 /*
@@ -632,7 +619,7 @@ trampoline_setup:
 /* Interrogate CPU extended features via CPUID. */
 mov $1, %eax
 cpuid
-mov %ecx, sym_fs(boot_cpu_data) + 
CPUINFO_FEATURE_OFFSET(X86_FEATURE_HYPERVISOR)
+mov %ecx, CPUINFO_FEATURE_OFFSET(X86_FEATURE_HYPERVISOR) + 
sym_esi(boot_cpu_data)
 
 mov $0x8000,%eax
 cpuid
@@ -644,7 +631,7 @@ trampoline_setup:
 jbe 1f
 mov $0x8001,%eax
 cpuid
-1:  mov %edx, sym_fs(boot_cpu_data) + 
CPUINFO_FEATURE_OFFSET(X86_FEATURE_LM)
+1:  mov %edx, CPUINFO_FEATURE_OFFSET(X86_FEATURE_LM) + 
sym_esi(boot_cpu_data)
 
 /* Check for NX. Adjust EFER setting if available. */
 bt  $cpufeat_bit(X86_FEATURE_NX), %edx
@@ -658,8 +645,8 @@ trampoline_setup:
 
 /* Stash TSC to calculate a good approximation of time-since-boot */
 rdtsc
-mov %eax,sym_fs(boot_tsc_stamp)
-mov %edx,sym_fs(boot_tsc_stamp)+4
+mov %eax, sym_esi(boot_tsc_stamp)
+mov %edx, 4 + sym_esi(boot_tsc_stamp)
 
 /* Relocate pagetables to point at Xen's current location in memory. */
 mov $_PAGE_PRESENT, %edx
@@ -753,11 +740,11 @@ trampoline_setup:
 jb  1b
 
 /* Do not parse command line on EFI platform here. */
-cmpb$0,sym_fs(efi_platform)
+cmpb$0, sym_esi(efi_platform)
 jnz 1f
 
 /* Bail if there is no command line to parse. */
-mov sym_fs(multiboot_ptr),%ebx
+mov sym_esi(multiboot_ptr), %ebx
 testl   $MBI_CMDLINE,MB_flags(%ebx)
 jz  1f
 
@@ -768,7 +755,7 @@ trampoline_setup:
 
 1:
 /* Switch to low-memory stack which lives at the end of trampoline 
region. */
-mov sym_fs(trampoline_phys),%edi
+mov sym_esi(trampoline_phys), %edi
 lea 

[Xen-devel] Interaction between ACPI and dt_unreserved_regions() (WAS: Re: [PATCH] xen/arm: vgic-v3: Fix the typo of GICD IRQ active status range)

[Julien Grall]

(Renaming the title to avoid confusion)

On 17/01/2020 09:06, Wei Xu wrote:

Hi Julien,


Hi Wei,


On 2020/1/7 23:12, Julien Grall wrote:
Sorry for the late reply!


Don't worry, thank you for looking into the bug!


The PC refers to fdt_num_mem_rsv during init_done.
But the device_tree_flattened is NULL that the data abort happened.


Ah, I didn't realize that device_tree_flattened where still used 
afterwards. Sorry for the breakage. I really need to setup a devbox with 
ACPI so I can test changes properly.



So I added below changes and the XEN dom0 can be booted.

     diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
     index 1e83351..1ab80a1 100644
     --- a/xen/arch/arm/setup.c
     +++ b/xen/arch/arm/setup.c
     @@ -392,7 +392,8 @@ void __init discard_initial_modules(void)
   !mfn_valid(maddr_to_mfn(e)) )
  continue;

     -    dt_unreserved_regions(s, e, init_domheap_pages, 0);
     +   if( acpi_disabled )
     +   dt_unreserved_regions(s, e, init_domheap_pages, 0);


While I understand how this is fixing your problem, this unfortunately 
means the memory ranges used by the inital modules (e.g Kernel, Initrd) 
will not be re-used by Xen. So this is a "slight" waste of memory.


There are a few other places where dt_unreserved_regions() is called 
(see setup_mm()). However, in the case of setup_mm() we have a pointer 
to DT as we don't know yet we are running on ACPI systems.


But it feels wrong to try to find out the reserved memory range through 
the DT when ACPI will be used. The DT is either going to be nearly 
empty, or contain the full description of the platform. I don't know 
enough to be able to say if something is going to go wrong.


I am thinking to suggest to create an helper that will do the job for 
you. Something like:


void fwtable_unreserved_regions(paddr_t s, paddr_t e,
void (*cb)(paddr_t, paddr_t), int first)
{
   if ( acpi_disabled )
 dt_unreserved_regions(s, e, cb, first);
   else
 cb(s, e);
}

Regarding the else part, this may need some adjustment if we need to 
skip some reserved region for ACPI. On Xen 4.13, we should only have 
usuable RAM in hand (the EFI stub is doing to sorting for us). Do you 
know whether ACPI describes something similar to reserved-memory in DT 
(i.e RAM regions reserved for cma...)?


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 4/4] x86/microcode: Support builtin CPU microcode

[Eslam Elnikety]

On 20.12.19 11:34, Jürgen Groß wrote:

On 20.12.19 11:12, Jan Beulich wrote:

On 19.12.2019 23:11, Eslam Elnikety wrote:

On 18.12.19 13:42, Jan Beulich wrote:

On 18.12.2019 02:32, Eslam Elnikety wrote:

--- /dev/null
+++ b/xen/arch/x86/microcode/Makefile
@@ -0,0 +1,46 @@
+# Copyright (C) 2019 Amazon.com, Inc. or its affiliates.
+# Author: Eslam Elnikety 
+#
+# This program is free software; you can redistribute it and/or 
modify
+# it under the terms of the GNU General Public License as 
published by

+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# Remove quotes and excess spaces from configuration strings
+UCODE_DIR=$(strip $(subst $\",,$(CONFIG_BUILTIN_UCODE_DIR)))
+UCODE_AMD=$(strip $(subst $\",,$(CONFIG_BUILTIN_UCODE_AMD)))
+UCODE_INTEL=$(strip $(subst $\",,$(CONFIG_BUILTIN_UCODE_INTEL)))
+
+# AMD and INTEL microcode blobs. Use 'wildcard' to filter for 
existing blobs.

+amd-blobs := $(wildcard $(addprefix $(UCODE_DIR),$(UCODE_AMD)))
+intel-blobs := $(wildcard $(addprefix $(UCODE_DIR),$(UCODE_INTEL)))
+
+ifneq ($(amd-blobs),)
+obj-y += ucode_amd.o
+endif
+
+ifneq ($(intel-blobs),)
+obj-y += ucode_intel.o
+endif
+
+ifeq ($(amd-blobs)$(intel-blobs),)
+obj-y += ucode_dummy.o
+endif
+
+ucode_amd.o: Makefile $(amd-blobs)
+    cat $(amd-blobs) > $@.bin
+    $(OBJCOPY) -I binary -O elf64-x86-64 -B i386:x86-64 
--rename-section 
.data=.builtin_amd_ucode,alloc,load,readonly,data,contents $@.bin $@

+    rm -f $@.bin
+
+ucode_intel.o: Makefile $(intel-blobs)
+    cat $(intel-blobs) > $@.bin
+    $(OBJCOPY) -I binary -O elf64-x86-64 -B i386:x86-64 
--rename-section 
.data=.builtin_intel_ucode,alloc,load,readonly,data,contents $@.bin $@

+    rm -f $@.bin


This can be had with a pattern rule (with the vendor being the stem)
and hence without duplication, I think.

Also - is simply concatenating the blobs reliable enough? There's no
build time diagnostic that the result would actually be understood
at runtime.



Concatenation is reliable (as long as the individual microcode blobs are
not malformed, and in that case the builtin is not making matters worse
compared to presenting the malformed update via  | scan).


A malformed update found the other way is a bug in the tools
constructing the respective images. A malformed built-in
update is a bug in the Xen build system. The put the question
differently: Is it specified somewhere that the blobs all have
to have certain properties, which the straight concatenation
relies upon?


+ucode_dummy.o: Makefile
+    $(CC) $(CFLAGS) -c -x c /dev/null -o $@;


Since the commit message doesn't explain why this is needed, I
have to ask (I guess we somewhere have a dependency on $(obj-y)
not being empty).


Your guess is correct. All sub-directories of xen/arch/x86 are expected
to produce built_in.o. If there are not amd nor intel microcode blobs,
there will be no build dependencies and the build fails preparing the
built_in.o


That's rather poor, but it's of course not your task to get this
fixed (it shouldn't be very difficult to create an empty
built_in.o for an empty $(obj-y)).


_If_ it is needed, I don't see why you need
ifeq() around its use. In fact you could have

obj-y := ucode-dummy.o

right at the top of the file.

Furthermore I don't really understand why you need this in the
first place. While cat won't do what you want with an empty
argument list, can't you simply prepend / append /dev/null?



To make sure we are on the same page. You are suggesting using
"/dev/null" in case there are no amd/intel ucode to generate the
ucode_amd/intel.o? If so, objcopy does not allow using /dev/null as
input (complains about empty binary).


That's again rather poor, this time of the utility - it should be
easy enough to produce an object with an empty .data (or whatever
it is) section. As above - I'm fine with you keeping the logic
then as is, provided you say in the description why it can't be
simplified.


What about using the attached patch for including the binary files?

I wanted to post that for my hypervisor-fs series, but I think it would
fit here quite nice.


Thanks, Jürgen. That tool is indeed useful on its own right for 
flask/policies. However, it seems to me that creating a built_in.o right 
out of the microcode blobs is simpler and keeps the whole business 
contained within xen/arch/x86/microcode/.


-- Eslam




Juergen



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 4/4] xen/x86: Rework inclusion between struct pirq and struct hvm_pirq_dpci

[Julien Grall]

Hi Jan,

On 15/01/2020 10:44, Jan Beulich wrote:

On 14.01.2020 18:03, Julien Grall wrote:

On 14/01/2020 16:50, Jan Beulich wrote:

On 14.01.2020 17:26, Julien Grall wrote:

On 14/01/2020 16:08, Jan Beulich wrote:

On 13.01.2020 22:33, Julien Grall wrote:

--- a/xen/arch/x86/hvm/irq.c
+++ b/xen/arch/x86/hvm/irq.c
@@ -29,7 +29,8 @@

bool hvm_domain_use_pirq(const struct domain *d, const struct pirq *pirq)

{
-return is_hvm_domain(d) && pirq && pirq->arch.hvm.emuirq != IRQ_UNBOUND;
+return is_hvm_domain(d) && pirq &&
+const_pirq_dpci(pirq)->emuirq != IRQ_UNBOUND;
}

/* Must be called with hvm_domain->irq_lock hold */

@@ -396,7 +397,7 @@ int hvm_inject_msi(struct domain *d, uint64_t addr, 
uint32_t data)
struct pirq *info = pirq_info(d, pirq);

/* if it is the first time, allocate the pirq */

-if ( !info || info->arch.hvm.emuirq == IRQ_UNBOUND )
+if ( !info || pirq_dpci(info)->emuirq == IRQ_UNBOUND )
{
int rc;

@@ -409,7 +410,7 @@ int hvm_inject_msi(struct domain *d, uint64_t addr, uint32_t data)

if ( !info )
return -EBUSY;
}
-else if ( info->arch.hvm.emuirq != IRQ_MSI_EMU )
+else if ( pirq_dpci(info)->emuirq != IRQ_MSI_EMU )
return -EINVAL;
send_guest_pirq(d, info);
return 0;


All of these uses (and others further down) make pretty clear
that the emuirq field doesn't belong in the structure you put it
in - the 'd' in dpci stands for "direct" afaik, and the field is
for a certain variant of emulation of interrupt delivery into
guests, i.e. not really pass-through focused at all.


I am happy to keep emuirq in struct pirq if you are happy with slightly
increasing the size allocated on PV.

The main thing I want to get rid of is the weird allocation size we do
today.


While I understand this, to be honest I'd rather not see the size
grow for no good (to PV) reason. I don't think the current model is
_this_ bad.


Well, I did lost two days debugging a problem because of the allocation
(the memory were getting corrupted randomly). The comment you added may
help to avoid this problem but I still think that trying to allocate
half a pirq is a pretty bad idea.


To me, not significantly different from your container_of() approach.


I guess it is a matter of perspective. The implementation of alloc/free 
is not much better, but a user trying to add a new field will not fall 
into the trap again (comments can often be overlooked).





But if you really want to push for it, why can't the
two parts continue to live in a wrapper HVM structure, just like
they do today?


I am not sure what you are suggesting here. Could you extend your thought?


Right now we have

struct arch_pirq {
 int irq;
 union {
 struct hvm_pirq {
 int emuirq;
 struct hvm_pirq_dpci dpci;
 } hvm;
 };
};

What I'm suggesting is to keep

struct hvm_pirq {
  int emuirq;
  struct hvm_pirq_dpci dpci;
};

and add struct arch_pirq into there. Arguably it could even
be first in there, thus allowing xfree() to free the whole
thing no matter whether passed a struct hvm_pirq * or a
struct arch_pirq * (and eliminating the need for a per-
arch abstraction of the freeing).


I guess you mean struct pirq instead of struct arch_pirq. If so, I will 
have a look. The code should be much cleaner than what I have submitted.





@@ -171,8 +172,26 @@ struct hvm_pirq_dpci {
struct hvm_gmsi_info gmsi;
struct timer timer;
struct list_head softirq_list;
+int emuirq;
+struct pirq pirq;
};

+#define pirq_dpci(p)\

+((p) ? container_of(p, struct hvm_pirq_dpci, pirq) : NULL)
+#define const_pirq_dpci(p)  \
+((p) ? container_of(p, const struct hvm_pirq_dpci, pirq) : NULL)
+
+#define dpci_pirq(pd) (&(pd)->pirq)
+
+#define domain_pirq_to_emuirq(d, p) ({  \
+struct pirq *__pi = pirq_info(d, p);\
+__pi ? pirq_dpci(__pi)->emuirq : IRQ_UNBOUND;   \
+})
+#define domain_emuirq_to_pirq(d, emuirq) ({ \
+void *__ret = radix_tree_lookup(&(d)->arch.hvm.emuirq_pirq, emuirq);\
+__ret ? radix_tree_ptr_to_int(__ret) : IRQ_UNBOUND; \
+})


While for the latter you merely move the bogus double-leading-
underscore macro local variable (which on this occasion I'd
like to ask anyway to be changed), you actively introduce a
new similar name space violation in the domain_pirq_to_emuirq().


AFAIK, there is nothing in the coding style forbidding your "bogus"
naming. So I just followed the rest of the code.


Our coding style document is not to re-iterate C standard rules,
I think, 

Re: [Xen-devel] [PATCH v2 4/4] x86/microcode: Support builtin CPU microcode

[Eslam Elnikety]

On 20.12.19 11:12, Jan Beulich wrote:

On 19.12.2019 23:11, Eslam Elnikety wrote:

On 18.12.19 13:42, Jan Beulich wrote:

On 18.12.2019 02:32, Eslam Elnikety wrote:

--- /dev/null
+++ b/xen/arch/x86/microcode/Makefile
@@ -0,0 +1,46 @@
+# Copyright (C) 2019 Amazon.com, Inc. or its affiliates.
+# Author: Eslam Elnikety 
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# Remove quotes and excess spaces from configuration strings
+UCODE_DIR=$(strip $(subst $\",,$(CONFIG_BUILTIN_UCODE_DIR)))
+UCODE_AMD=$(strip $(subst $\",,$(CONFIG_BUILTIN_UCODE_AMD)))
+UCODE_INTEL=$(strip $(subst $\",,$(CONFIG_BUILTIN_UCODE_INTEL)))
+
+# AMD and INTEL microcode blobs. Use 'wildcard' to filter for existing blobs.
+amd-blobs := $(wildcard $(addprefix $(UCODE_DIR),$(UCODE_AMD)))
+intel-blobs := $(wildcard $(addprefix $(UCODE_DIR),$(UCODE_INTEL)))
+
+ifneq ($(amd-blobs),)
+obj-y += ucode_amd.o
+endif
+
+ifneq ($(intel-blobs),)
+obj-y += ucode_intel.o
+endif
+
+ifeq ($(amd-blobs)$(intel-blobs),)
+obj-y += ucode_dummy.o
+endif
+
+ucode_amd.o: Makefile $(amd-blobs)
+   cat $(amd-blobs) > $@.bin
+   $(OBJCOPY) -I binary -O elf64-x86-64 -B i386:x86-64 --rename-section 
.data=.builtin_amd_ucode,alloc,load,readonly,data,contents $@.bin $@
+   rm -f $@.bin
+
+ucode_intel.o: Makefile $(intel-blobs)
+   cat $(intel-blobs) > $@.bin
+   $(OBJCOPY) -I binary -O elf64-x86-64 -B i386:x86-64 --rename-section 
.data=.builtin_intel_ucode,alloc,load,readonly,data,contents $@.bin $@
+   rm -f $@.bin


This can be had with a pattern rule (with the vendor being the stem)
and hence without duplication, I think.

Also - is simply concatenating the blobs reliable enough? There's no
build time diagnostic that the result would actually be understood
at runtime.



Concatenation is reliable (as long as the individual microcode blobs are
not malformed, and in that case the builtin is not making matters worse
compared to presenting the malformed update via  | scan).


A malformed update found the other way is a bug in the tools
constructing the respective images. A malformed built-in
update is a bug in the Xen build system. The put the question
differently: Is it specified somewhere that the blobs all have
to have certain properties, which the straight concatenation
relies upon?



Refer to ( https://www.kernel.org/doc/Documentation/x86/microcode.txt ). 
AuthenticAMD.bin and GenuineIntel.bin are both concatenations of 
individual microcode blobs.



+ucode_dummy.o: Makefile
+   $(CC) $(CFLAGS) -c -x c /dev/null -o $@;


Since the commit message doesn't explain why this is needed, I
have to ask (I guess we somewhere have a dependency on $(obj-y)
not being empty).


Your guess is correct. All sub-directories of xen/arch/x86 are expected
to produce built_in.o. If there are not amd nor intel microcode blobs,
there will be no build dependencies and the build fails preparing the
built_in.o


That's rather poor, but it's of course not your task to get this
fixed (it shouldn't be very difficult to create an empty
built_in.o for an empty $(obj-y)).


_If_ it is needed, I don't see why you need
ifeq() around its use. In fact you could have

obj-y := ucode-dummy.o

right at the top of the file.

Furthermore I don't really understand why you need this in the
first place. While cat won't do what you want with an empty
argument list, can't you simply prepend / append /dev/null?



To make sure we are on the same page. You are suggesting using
"/dev/null" in case there are no amd/intel ucode to generate the
ucode_amd/intel.o? If so, objcopy does not allow using /dev/null as
input (complains about empty binary).


That's again rather poor, this time of the utility - it should be
easy enough to produce an object with an empty .data (or whatever
it is) section. As above - I'm fine with you keeping the logic
then as is, provided you say in the description why it can't be
simplified.


Ack. Will justify the logic in comments.

-- Eslam



Jan




___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [Vote] For Xen Project Code of Conduct (deadline March 31st)

[Lars Kurth]

Apologies,

some of the links I added for convenience do not work

> It should be read in the following order
The correct links are

* 
http://xenbits.xenproject.org/gitweb/?p=people/larsk/code-of-conduct.git;a=blob;f=code-of-conduct.md;hb=refs/heads/CoC-v5
 
* 
http://xenbits.xenproject.org/gitweb/?p=people/larsk/code-of-conduct.git;a=blob;f=communication-guide.md;hb=refs/heads/CoC-v5
* 
http://xenbits.xenproject.org/gitweb/?p=people/larsk/code-of-conduct.git;a=blob;f=code-review-guide.md;hb=refs/heads/CoC-v5
* 
http://xenbits.xenproject.org/gitweb/?p=people/larsk/code-of-conduct.git;a=blob;f=communication-practice.md;hb=refs/heads/CoC-v5
* 
http://xenbits.xenproject.org/gitweb/?p=people/larsk/code-of-conduct.git;a=blob;f=resolving-disagreement.md;hb=refs/heads/CoC-v5

Lars


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH V3 2/2] Xen/PCIback: Implement PCI flr/slot/bus reset with 'reset' SysFS attribute

[Rich Persaud]

On Aug 26, 2019, at 17:08, Pasi Kärkkäinen  wrote:
> Hi,
> 
>> On Mon, Oct 08, 2018 at 10:32:45AM -0400, Boris Ostrovsky wrote:
>>> On 10/3/18 11:51 AM, Pasi Kärkkäinen wrote:
>>> On Wed, Sep 19, 2018 at 11:05:26AM +0200, Roger Pau Monné wrote:
 On Tue, Sep 18, 2018 at 02:09:53PM -0400, Boris Ostrovsky wrote:
> On 9/18/18 5:32 AM, George Dunlap wrote:
>>> On Sep 18, 2018, at 8:15 AM, Pasi Kärkkäinen  wrote:
>>> Hi,
>>> On Mon, Sep 17, 2018 at 02:06:02PM -0400, Boris Ostrovsky wrote:
 What about the toolstack changes? Have they been accepted? I vaguely
 recall there was a discussion about those changes but don't remember 
 how
 it ended.
>>> I don't think toolstack/libxl patch has been applied yet either.
>>> "[PATCH V1 0/1] Xen/Tools: PCI reset using 'reset' SysFS attribute":
>>> https://lists.xen.org/archives/html/xen-devel/2017-12/msg00664.html
>>> "[PATCH V1 1/1] Xen/libxl: Perform PCI reset using 'reset' SysFS 
>>> attribute":
>>> https://lists.xen.org/archives/html/xen-devel/2017-12/msg00663.html
> Will this patch work for *BSD? Roger?
 At least FreeBSD don't support pci-passthrough, so none of this works
 ATM. There's no sysfs on BSD, so much of what's in libxl_pci.c will
 have to be moved to libxl_linux.c when BSD support is added.
>>> Ok. That sounds like it's OK for the initial pci 'reset' implementation in 
>>> xl/libxl to be linux-only..
>> 
>> Are these two patches still needed? ISTR they were  written originally
>> to deal with guest trying to use device that was previously assigned to
>> another guest. But pcistub_put_pci_dev() calls
>> __pci_reset_function_locked() which first tries FLR, and it looks like
>> it was added relatively recently.
> 
> Replying to an old thread.. I only now realized I forgot to reply to this 
> message earlier.
> 
> afaik these patches are still needed. Håkon (CC'd) wrote to me in private that
> he gets a (dom0) Linux kernel crash if he doesn't have these patches applied.
> 
> 
> Here are the links to both the linux kernel and libxl patches:
> 
> 
> "[Xen-devel] [PATCH V3 0/2] Xen/PCIback: PCI reset using 'reset' SysFS 
> attribute":
> https://lists.xen.org/archives/html/xen-devel/2017-12/msg00659.html
> 
> [Note that PATCH V3 1/2 "Drivers/PCI: Export pcie_has_flr() interface" is 
> already applied in upstream linux kernel, so it's not needed anymore]
> 
> "[Xen-devel] [PATCH V3 2/2] Xen/PCIback: Implement PCI flr/slot/bus reset 
> with 'reset' SysFS attribute":
> https://lists.xen.org/archives/html/xen-devel/2017-12/msg00661.html
> 
> 
> "[Xen-devel] [PATCH V1 0/1] Xen/Tools: PCI reset using 'reset' SysFS 
> attribute":
> https://lists.xen.org/archives/html/xen-devel/2017-12/msg00664.html
> 
> "[Xen-devel] [PATCH V1 1/1] Xen/libxl: Perform PCI reset using 'reset' SysFS 
> attribute":
> https://lists.xen.org/archives/html/xen-devel/2017-12/msg00663.html

[dropping Linux mailing lists]

What is required to get the Xen patches merged?  Rebasing against Xen master?  
OpenXT has been carrying a similar patch for many years and we would like to 
move to an upstream implementation.  Xen users of PCI passthrough would benefit 
from more reliable device reset.

  2017 thread, including OpenXT patch: https://lists.gt.net/xen/devel/492945
  2017-2019 thread: https://lists.gt.net/xen/devel/532648

Rich___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [Vote] For Xen Project Code of Conduct (deadline March 31st)

[Lars Kurth]

Hi all,

for some time now we have been discussing the Xen Project Code of
Conduct. The most recent set of feedback has been primarily around
minor language issues (US vs UL English, etc.), which indicates to me 
that the proposal is ready to be voted on

The final version which addresses all the latest minor feedback can be
found at 
http://xenbits.xenproject.org/gitweb/?p=people/larsk/code-of-conduct.git;a=tree;h=refs/heads/CoC-v5
 

It should be read in the following order
* 
http://xenbits.xenproject.org/gitweb/?p=people/larsk/code-of-conduct.git;a=blob;f=code-of-conduct.md
 
* 
http://xenbits.xenproject.org/gitweb/?p=people/larsk/code-of-conduct.git;a=blob;f=communication-guide.md
* 
http://xenbits.xenproject.org/gitweb/?p=people/larsk/code-of-conduct.git;a=blob;f=code-review-guide.md
* 
http://xenbits.xenproject.org/gitweb/?p=people/larsk/code-of-conduct.git;a=blob;f=communication-practice.md
 
* 
http://xenbits.xenproject.org/gitweb/?p=people/larsk/code-of-conduct.git;a=blob;f=resolving-disagreement.md
 

In accordance with https://xenproject.org/developers/governance/, I need the
leadership teams of the three mature projects: the Hypervisor, the XAPI
project and the Windows PV Driver project to vote on this proposal.

The specific voting rules in this case are outlined in section
https://www.xenproject.org/governance.html#project-decisions 

People allowed to vote on behalf of the Hypervisor project are:
Julien Grall, Andy Cooper, George Dunlap, Ian Jackson, Jan Beulich, Konrad R
Wilk, Stefano Stabellini, Wei Liu and Paul Durrant (as Release Manager).

People allowed to vote on behalf of the XAPI project are:
Chandrika Srinivasan, Christian Lindig, Konstantina Chremmou,
Rob Hoes, Zhang Li

People allowed to vote on behalf of the Windows PV Driver Project are:
Paul Durrant, Ben Chalmers, Owen Smith

I propose to tally the votes after March 31st. You can reply via
+1: for proposal
-1: against proposal
in public or private.

Votes will be tallied by subproject – aka the Hypervisor and XAPI project by %
for the proposal - and then averaged across sub-projects that achieved the
quorum. The vote needs to achieve a 2/3 majority to pass.

Sub-project needs to achieve the following quorum of votes in favour for the
sub-project’s vote to count
Hypervisor: 3 + votes
XAPI: 2 + votes
Windows PV Drivers: 1 + votes

Regards
Lars


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-unstable-smoke test] 146202: tolerable all pass - PUSHED

[osstest service owner]

flight 146202 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146202/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  32772fbb3cf7498817304b53b087e325c6991716
baseline version:
 xen  97f10daf5f4bac91db732ef45c562839686f2c04

Last test of basis   146174  2020-01-16 21:00:38 Z0 days
Testing same since   146202  2020-01-17 16:00:42 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Igor Druzhinin 
  Jan Beulich 
  Jason Andryuk 
  Lars Kurth 
  Tim Deegan 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   97f10daf5f..32772fbb3c  32772fbb3cf7498817304b53b087e325c6991716 -> smoke

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 1/4] x86/microcode: Improve documentation and parsing for ucode=

[Eslam Elnikety]

Picking this up again after the break. Apologies for the delay.

On 20.12.19 10:53, Jan Beulich wrote:

On 19.12.2019 22:08, Eslam Elnikety wrote:

On 18.12.19 12:49, Jan Beulich wrote:

On 18.12.2019 02:32, Eslam Elnikety wrote:

Decouple the microcode referencing mechanism when using GRUB to that
when using EFI. This allows us to avoid the "unspecified effect" of
using ` | scan` along xen.efi.


I guess "unspecified effect" in the doc was pretty pointless - such
options have been ignored before; in fact ...


With that, Xen can explicitly
ignore those named options when using EFI.


... I don't see things becoming any more explicit (not even parsing
the options was quite explicit to me).



I agree that those options have been ignored so far in the case of EFI.
The documentation contradicts that however. The documentation:
* says  has unspecified effect.
* does not mention anything about scan being ignored.

With this patch, it is explicit in code and in documentation that both
options are ignored in case of EFI.


But isn't it rather that ucode=scan could (and hence perhaps should)
also have its value on EFI?



I do not see "ucode=scan" applicable in anyway in the case of EFI. In 
EFI, there are not "modules" to scan through, but rather the efi config 
points exactly to the microcode blob.



--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -2128,7 +2128,13 @@ logic applies:
   ### ucode (x86)
   > `= List of [  | scan=, nmi= ]`
   
-Specify how and where to find CPU microcode update blob.

+Applicability: x86
+Default: `nmi`
+
+Controls for CPU microcode loading. For early loading, this parameter can
+specify how and where to find the microcode update blob. For late loading,
+this parameter specifies if the update happens within a NMI handler or in
+a stop_machine context.


It's always stop_machine context, isn't it? I also don't think this
implementation detail belongs here.



Needs a better wording indeed. Let me know if you have particular
suggestions, and I will incorporate in v3.


Just drop everything from "or" onwards?



Ack


@@ -105,16 +105,10 @@ static struct microcode_patch *microcode_cache;
   
   void __init microcode_set_module(unsigned int idx)

   {
-ucode_mod_idx = idx;
-ucode_mod_forced = 1;
+ucode_mod_efi_idx = idx;


Is it guaranteed (now and forever) that the index passed in is
non-zero? You changes to microcode_grab_module() imply so, but
just looking at the call site of the function I can't convince
myself this is the case. _If_ it is (thought to be) guaranteed,
then I think you at least want to ASSERT() here, perhaps with
a comment.



For x86, it seems we have that guarantee (given that we must have a
dom0). Right?


For fully bringing up the system - yes. But there's no reason to
have a Dom0 if all you're after is testing early Xen boot. There'll
be a panic() in the case, but there shouldn't be anything breaking
proper behavior prior to this point.



That's a valid point indeed. v3 will handle index being zero.


   }
   
-/*

- * The format is '[|scan=, nmi=]'. Both options are
- * optional. If the EFI has forced which of the multiboot payloads is to be
- * used, only nmi= is parsed.
- */
-static int __init parse_ucode(const char *s)
+static int __init parse_ucode_param(const char *s)


Any particular reason for the renaming? The function name was quite
fine imo.



To me, "parse_ucode" is a misnomer.


Well, parse_"ucode= isn't a valid identifier. parse_ucode is what
results when stripping everything that makes it invalid. I can
see the ambiguity with parsing actual ucode, but the context here
makes it pretty clear what the function is about. Personally I'd
prefer such unnecessary renames to be avoided.


Ack

-- Eslam

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [linux-5.4 test] 146178: regressions - FAIL

[osstest service owner]

flight 146178 linux-5.4 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146178/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail REGR. vs. 
146121

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-arm64-arm64-xl-seattle  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop fail never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail never pass

version targeted for testing:
 linuxadc0acf587768b7db6ca1d7c395a9116865c9e07
baseline version:
 linux122179cb7d648a6f36b20dd6bf34f953cb384c30

Last test of basis   146121  2020-01-15 17:42:04 Z2 days
Testing same since   146178  2020-01-17 02:59:07 Z0 days1 attempts


527 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  

Re: [Xen-devel] [PATCH] xen: xen-pciback: Reset MSI-X state when exposing a device

[Rich Persaud]

On Sep 26, 2019, at 06:17, Pasi Kärkkäinen  wrote:
> 
> Hello Stanislav,
> 
>> On Fri, Sep 13, 2019 at 11:28:20PM +0800, Chao Gao wrote:
>>> On Fri, Sep 13, 2019 at 10:02:24AM +, Spassov, Stanislav wrote:
>>> On Thu, Dec 13, 2018 at 07:54, Chao Gao wrote:
 On Thu, Dec 13, 2018 at 12:54:52AM -0700, Jan Beulich wrote:
 On 13.12.18 at 04:46,  wrote:
>> On Wed, Dec 12, 2018 at 08:21:39AM -0700, Jan Beulich wrote:
>> On 12.12.18 at 16:18,  wrote:
 On Wed, Dec 12, 2018 at 01:51:01AM -0700, Jan Beulich wrote:
 On 12.12.18 at 08:06,  wrote:
>> On Wed, Dec 05, 2018 at 09:01:33AM -0500, Boris Ostrovsky wrote:
>>> On 12/5/18 4:32 AM, Roger Pau Monné wrote:
 On Wed, Dec 05, 2018 at 10:19:17AM +0800, Chao Gao wrote:
> I find some pass-thru devices don't work any more across guest 
> reboot.
> Assigning it to another guest also meets the same issue. And the 
> only
> way to make it work again is un-binding and binding it to pciback.
> Someone reported this issue one year ago [1]. More detail also 
> can be
> found in [2].
> 
> The root-cause is Xen's internal MSI-X state isn't reset properly
> during reboot or re-assignment. In the above case, Xen set 
> maskall bit
> to mask all MSI interrupts after it detected a potential security
> issue. Even after device reset, Xen didn't reset its internal 
> maskall
> bit. As a result, maskall bit would be set again in next write to
> MSI-X message control register.
> 
> Given that PHYSDEVOPS_prepare_msix() also triggers Xen resetting 
> MSI-X
> internal state of a device, we employ it to fix this issue rather 
> than
> introducing another dedicated sub-hypercall.
> 
> Note that PHYSDEVOPS_release_msix() will fail if the mapping 
> between
> the device's msix and pirq has been created. This limitation 
> prevents
> us calling this function when detaching a device from a guest 
> during
> guest shutdown. Thus it is called right before calling
> PHYSDEVOPS_prepare_msix().
 s/PHYSDEVOPS/PHYSDEVOP/ (no final S). And then I would also drop 
 the
 () at the end of the hypercall name since it's not a function.
 
 I'm also wondering why the release can't be done when the device is
 detached from the guest (or the guest has been shut down). This 
 makes
 me worry about the raciness of the attach/detach procedure: if 
 there's
 a state where pciback assumes the device has been detached from the
 guest, but there are still pirqs bound, an attempt to attach to
 another guest in such state will fail.
>>> 
>>> I wonder whether this additional reset functionality could be done 
>>> out
>>> of xen_pcibk_xenbus_remove(). We first do a (best effort) device 
>>> reset
>>> and then do the extra things that are not properly done there.
>> 
>> No. It cannot be done in xen_pcibk_xenbus_remove() without modifying
>> the handler of PHYSDEVOP_release_msix. To do a successful Xen 
>> internal
>> MSI-X state reset, PHYSDEVOP_{release, prepare}_msix should be 
>> finished
>> without error. But ATM, xen expects that no msi is bound to pirq when
>> doing PHYSDEVOP_release_msix. Otherwise it fails with error code 
>> -EBUSY.
>> However, the expectation isn't guaranteed in 
>> xen_pcibk_xenbus_remove().
>> In some cases, if qemu fails to unmap MSIs, MSIs are unmapped by Xen
>> at last minute, which happens after device reset in 
>> xen_pcibk_xenbus_remove().
> 
> But that may need taking care of: I don't think it is a good idea to 
> have
> anything left from the prior owning domain when the device gets reset.
> I.e. left over IRQ bindings should perhaps be forcibly cleared before
> invoking the reset;
 
 Agree. How about pciback to track the established IRQ bindings? Then
 pciback can clear irq binding before invoking the reset.
>>> 
>>> How would pciback even know of those mappings, when it's qemu
>>> who establishes (and manages) them?
>> 
>> I meant to expose some interfaces from pciback. And pciback serves
>> as the proxy of IRQ (un)binding APIs.
> 
> If at all possible we should avoid having to change more parties (qemu,
> libxc, kernel, hypervisor) than really necessary. Remember that such
> a bug fix may want backporting, and making sure 

Re: [Xen-devel] [PATCH v4 11/16] tools: add simple vchan-socket-proxy

[Marek Marczykowski-Górecki]

On Fri, Jan 17, 2020 at 01:44:20PM -0500, Rich Persaud wrote:
> On Jan 14, 2020, at 21:42, Marek Marczykowski-Górecki 
>  wrote:
> > @@ -66,6 +70,7 @@ install: all
> >$(INSTALL_PROG) libxenvchan.so.$(MAJOR).$(MINOR) $(DESTDIR)$(libdir)
> >ln -sf libxenvchan.so.$(MAJOR).$(MINOR) 
> > $(DESTDIR)$(libdir)/libxenvchan.so.$(MAJOR)
> >ln -sf libxenvchan.so.$(MAJOR) $(DESTDIR)$(libdir)/libxenvchan.so
> > +$(INSTALL_PROG) vchan-socket-proxy $(DESTDIR)$(bindir)
> 
> Does this need directory creation, to avoid vchan binary being named "bin"?
> +   $(INSTALL_DIR) $(DESTDIR)$(bindir)

I guess it depends on makefile execution order. I'll add it to be on the
safe side.

> > +int main(int argc, char **argv)
> > +{
> > +int is_server = 0;
> > +int socket_fd;
> 
> When compiled for OpenEmbedded / OpenXT, gcc complained about socket_fd being 
> uninitialized before possible use.

I think gcc is wrong here - in all the paths socket_fd is used, it is
initialized under the same conditions (socket_path != "-" and
!is_server). But I'll add initializer to mute this warning.

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?


signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-unstable test] 146176: regressions - FAIL

[osstest service owner]

flight 146176 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146176/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 10 debian-hvm-install 
fail REGR. vs. 146058
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 10 debian-hvm-install 
fail REGR. vs. 146058

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-rtds 18 guest-localmigrate/x10  fail blocked in 146058
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 146058
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 146058
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 146058
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 146058
 test-amd64-amd64-qemuu-nested-intel 17 debian-hvm-install/l1/l2 fail like 
146058
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 146058
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 146058
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 146058
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 146058
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop fail like 146058
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-arm64-arm64-xl-seattle  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass

version targeted for testing:
 xen  97f10daf5f4bac91db732ef45c562839686f2c04
baseline version:
 xen  03bfe526ecadc86f31eda433b91dc90be0563919

Last test of basis   146058  2020-01-14 01:51:38 Z3 days
Failing since146094  2020-01-14 

Re: [Xen-devel] [PATCH v4 11/16] tools: add simple vchan-socket-proxy

[Rich Persaud]

On Jan 14, 2020, at 21:42, Marek Marczykowski-Górecki 
 wrote:
> 
> diff --git a/tools/libvchan/Makefile b/tools/libvchan/Makefile
> index 7892750..1c845ca 100644
> --- a/tools/libvchan/Makefile
> +++ b/tools/libvchan/Makefile
> @@ -13,6 +13,7 @@ LIBVCHAN_PIC_OBJS = $(patsubst %.o,%.opic,$(LIBVCHAN_OBJS))
> LIBVCHAN_LIBS = $(LDLIBS_libxenstore) $(LDLIBS_libxengnttab) 
> $(LDLIBS_libxenevtchn)
> $(LIBVCHAN_OBJS) $(LIBVCHAN_PIC_OBJS): CFLAGS += $(CFLAGS_libxenstore) 
> $(CFLAGS_libxengnttab) $(CFLAGS_libxenevtchn)
> $(NODE_OBJS) $(NODE2_OBJS): CFLAGS += $(CFLAGS_libxengnttab) 
> $(CFLAGS_libxenevtchn)
> +vchan-socket-proxy.o: CFLAGS += $(CFLAGS_libxenstore) $(CFLAGS_libxenctrl) 
> $(CFLAGS_libxengnttab) $(CFLAGS_libxenevtchn)
> 
> MAJOR = 4.14
> MINOR = 0
> @@ -39,7 +40,7 @@ $(PKG_CONFIG_LOCAL): PKG_CONFIG_LIBDIR = $(CURDIR)
> $(PKG_CONFIG_LOCAL): PKG_CONFIG_CFLAGS_LOCAL = $(CFLAGS_xeninclude)
> 
> .PHONY: all
> -all: libxenvchan.so vchan-node1 vchan-node2 libxenvchan.a $(PKG_CONFIG_INST) 
> $(PKG_CONFIG_LOCAL)
> +all: libxenvchan.so vchan-node1 vchan-node2 vchan-socket-proxy libxenvchan.a 
> $(PKG_CONFIG_INST) $(PKG_CONFIG_LOCAL)
> 
> libxenvchan.so: libxenvchan.so.$(MAJOR)
>ln -sf $< $@
> @@ -59,6 +60,9 @@ vchan-node1: $(NODE_OBJS) libxenvchan.so
> vchan-node2: $(NODE2_OBJS) libxenvchan.so
>$(CC) $(LDFLAGS) -o $@ $(NODE2_OBJS) $(LDLIBS_libxenvchan) 
> $(APPEND_LDFLAGS)
> 
> +vchan-socket-proxy: vchan-socket-proxy.o libxenvchan.so
> +$(CC) $(LDFLAGS) -o $@ $< $(LDLIBS_libxenvchan) $(LDLIBS_libxenstore) 
> $(LDLIBS_libxenctrl) $(APPEND_LDFLAGS)
> +
> .PHONY: install
> install: all
>$(INSTALL_DIR) $(DESTDIR)$(libdir)
> @@ -66,6 +70,7 @@ install: all
>$(INSTALL_PROG) libxenvchan.so.$(MAJOR).$(MINOR) $(DESTDIR)$(libdir)
>ln -sf libxenvchan.so.$(MAJOR).$(MINOR) 
> $(DESTDIR)$(libdir)/libxenvchan.so.$(MAJOR)
>ln -sf libxenvchan.so.$(MAJOR) $(DESTDIR)$(libdir)/libxenvchan.so
> +$(INSTALL_PROG) vchan-socket-proxy $(DESTDIR)$(bindir)

Does this need directory creation, to avoid vchan binary being named "bin"?
+   $(INSTALL_DIR) $(DESTDIR)$(bindir)


> diff --git a/tools/libvchan/vchan-socket-proxy.c 
> b/tools/libvchan/vchan-socket-proxy.c
> new file mode 100644
> index 000..6b4ae09
> --- /dev/null
> +++ b/tools/libvchan/vchan-socket-proxy.c
> @@ -0,0 +1,469 @@
> +/**
> + * @file
> + * @section AUTHORS
> + *
> + * Copyright (C) 2010  Rafal Wojtczuk  
> + *
> + *  Authors:
> + *   Rafal Wojtczuk  
> + *   Daniel De Graaf 
> + *   Marek Marczykowski-Górecki  
> + *
> + * @section LICENSE
> + *
> + *  This program is free software; you can redistribute it and/or
> + *  modify it under the terms of the GNU Lesser General Public
> + *  License as published by the Free Software Foundation; either
> + *  version 2.1 of the License, or (at your option) any later version.
> + *
> + *  This program is distributed in the hope that it will be useful,
> + *  but WITHOUT ANY WARRANTY; without even the implied warranty of
> + *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + *  Lesser General Public License for more details.
> + *
> + *  You should have received a copy of the GNU Lesser General Public
> + *  License along with this program; If not, see 
> .
> + *
> + * @section DESCRIPTION
> + *
> + * This is a vchan to unix socket proxy. Vchan server is set, and on client
> + * connection, local socket connection is established. Communication is 
> bidirectional.
> + * One client is served at a time, clients needs to coordinate this 
> themselves.
> + */
> +
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +
> +#include 
> +#include 
> +#include 
> +
> +static void usage(char** argv)
> +{
> +fprintf(stderr, "usage:\n"
> +"\t%s [options] domainid nodepath [socket-path|file-no|-]\n"
> +"\n"
> +"options:\n"
> +"\t-m, --mode=client|server - vchan connection mode\n"
> +"\t-m, --state-path=path - xenstore path where write \"running\" to 
> at startup\n"
> +"\t-v, --verbose - verbose logging\n"
> +"\n"
> +"client: client of a vchan connection, fourth parameter can be:\n"
> +"\tsocket-path: listen on a UNIX socket at this path and connect to 
> vchan\n"
> +"\t  whenever new connection is accepted;\n"
> +"\t  handle multiple _subsequent_ connections, until terminated\n"
> +"\tfile-no: except open FD of a socket in listen mode; otherwise 
> similar to socket-path\n"
> +"\t-: open vchan connection immediately and pass the data from 
> stdin/stdout;\n"
> +"\t  terminate when vchan connection is closed\n"
> +"server: server of a vchan connection, fourth parameter can be:\n"
> +"\tsocket-path: connect to this UNIX socket when new vchan 
> connection is accepted\n"
> +"\t  handle multiple _subsequent_ connections, until terminated\n"

Re: [Xen-devel] [PATCH v3 8/8] RFC: Sketch constructors, DomainCreateNew

[George Dunlap]

On 1/17/20 3:57 PM, George Dunlap wrote:
> This is a sketch of functionality suitable for creating a basic
> domain, with a disk and a vif.  DomainConfig, DeviceDisk, and
> DeviceNic types are all created using constructor functions, which
> initialize them with libxl's defaults.
> 
> DomainCreateNew takes the config and calls without any updates.
> 
> Obviously some of these will need to be changed it we switch to
> passing references to .toC() rather than passing back by value.
> 
> The main purpose of this is to allow testing of creating a hard-coded
> domain.
> 
> Creating a domain would look like this:
> 
>   // type = "pv"
>   dconf, err := xl.NewDomainConfig(xl.DomainTypePv)
>   if err != nil {
>   fmt.Printf("NewDomainConfig: %v\n", err)
>   return
>   }
>   dconf.CInfo.Type = xl.DomainTypePv
>   // name = "c6-01"
>   dconf.CInfo.Name = "c6-01"
>   // vcpus=4
>   dconf.BInfo.MaxVcpus = 4
>   // memory = "2048"
>   dconf.BInfo.MaxMemkb = 2048 * 1024
>   dconf.BInfo.TargetMemkb = 2048 * 1024
>   // on_crash = 'destroy'
>   dconf.OnCrash = xl.ActionOnShutdownDestroy
>   // bootloader = "pygrub"
>   dconf.BInfo.Bootloader = "pygrub"
>   // disk = [ 'vdev=hda,format=raw,target=/images/c6-01.raw']
>   {
>   disk, err := xl.NewDeviceDisk()
>   if err != nil {
>   fmt.Printf("NewDeviceDisk: %v\n", err)
>   return
>   }
>   disk.Vdev = "hda"
>   //disk.DiskBackend = xl.DiskBackendPhy
>   disk.Format = xl.DiskFormatRaw
>   disk.Readwrite = 1
>   disk.PdevPath = "/images/c6-01.raw"
>   dconf.Disks = append(dconf.Disks, *disk)
>   }
>   // vif = [ 'mac=5a:5b:d6:f1:d6:b4' ]
>   {
>   vif, err := xl.NewDeviceNic()
>   if err != nil {
>   fmt.Printf("NewDeviceNic: %v\n", err)
>   return
>   }
>   vif.Mac = xl.Mac{ 0x5a, 0x5b, 0xd6, 0x31, 0xd6, 0xb4 }
>   dconf.Nics = append(dconf.Nics, *vif)
>   }
>   // serial='pty' # HVM only
> 
>   did, err := ctx.DomainCreateNew(dconf)
> 
>   if err != nil {
>   fmt.Printf("Creating domain: %v\n", err)
>   return
>   }
> 
>   fmt.Printf("Domain %s(%d) created successfully", dconf.CInfo.Name, did)
> 
> 
> Signed-off-by: George Dunlap 
> ---
> CC: Nick Rosbrook 
> ---
>  tools/golang/xenlight/xenlight.go | 66 +++
>  1 file changed, 66 insertions(+)
> 
> diff --git a/tools/golang/xenlight/xenlight.go 
> b/tools/golang/xenlight/xenlight.go
> index c462e4bb42..5a21a2b9b8 100644
> --- a/tools/golang/xenlight/xenlight.go
> +++ b/tools/golang/xenlight/xenlight.go
> @@ -1113,3 +1113,69 @@ func (Ctx *Context) PrimaryConsoleGetTty(domid uint32) 
> (path string, err error)
>   path = C.GoString(cpath)
>   return
>  }
> +
> +func NewDomainConfig(t DomainType) (*DomainConfig, error) {
> + var cconfig C.libxl_domain_config
> +
> + C.libxl_domain_config_init()
> + C.libxl_domain_build_info_init_type(_info, 
> C.libxl_domain_type(t))
> +
> + gconfig := {}
> + err := gconfig.fromC()
> + if err != nil {
> + return nil, err
> + }
> +
> + return gconfig, nil
> +}
> +
> +func NewDeviceDisk() (*DeviceDisk, error) {
> + var ctype C.libxl_device_disk
> +
> + C.libxl_device_disk_init()
> +
> + gtype := {}
> + err := gtype.fromC()
> +
> + if err != nil {
> + return nil, err
> + }
> +
> + return gtype, nil
> +}
> +
> +func NewDeviceNic() (*DeviceNic, error) {
> + var ctype C.libxl_device_nic
> +
> + C.libxl_device_nic_init()
> +
> + gtype := {}
> + err := gtype.fromC()
> +
> + if err != nil {
> + return nil, err
> + }
> +
> + return gtype, nil
> +}
> +
> +// int libxl_domain_create_new(libxl_ctx *ctx, libxl_domain_config *d_config,
> +// uint32_t *domid,
> +// const libxl_asyncop_how *ao_how,
> +// const libxl_asyncprogress_how 
> *aop_console_how)
> +func (Ctx *Context) DomainCreateNew(config *DomainConfig) (Domid, error) {
> + var cdomid C.uint32_t
> + var cconfig C.libxl_domain_config
> + err := config.toC()
> + if err != nil {
> + return Domid(0), fmt.Errorf("converting domain config to C: 
> %v", err)
> + }
> + defer C.libxl_domain_config_dispose()
> +
> + ret := C.libxl_domain_create_new(Ctx.ctx, , , nil, nil)
> + if ret != 0 {
> + return Domid(0), Error(ret)
> + }
> +
> + return Domid(cdomid), nil
> +}

An alternate way to do this would be something like this:

---
func NewDomainConfig(t DomainType, populate func(*DomainConfig))
*DomainConfig {
var ctype C.libxl_domain_config

Re: [Xen-devel] [PATCH v3 7/8] golang/xenlight: Notify xenlight of SIGCHLD

[George Dunlap]

On 1/17/20 6:13 PM, Nick Rosbrook wrote:
>>  // Context represents a libxl_ctx.
>>  type Context struct {
>> -   ctx*C.libxl_ctx
>> -   logger *C.xentoollog_logger_stdiostream
>> +   ctx *C.libxl_ctx
>> +   logger  *C.xentoollog_logger_stdiostream
>> +   sigchld chan os.Signal
>> +   sigchldDone chan bool
> 
> It's preferred to use `chan struct{}` for this pattern; it makes it
> clear that the data sent over the channel has no meaning, and is only
> intended to be used for synchronization.

OK.  I think it looks ugly, but there's certainly a signalling value to
having it really be empty.

> 
>> +   // ...and arrange to keep that promise.
>> +   ctx.sigchld = make(chan os.Signal, 2)
>> +   ctx.sigchldDone = make(chan bool, 1)
>> +   signal.Notify(ctx.sigchld, syscall.SIGCHLD)
>> +
>> +   go sigchldHandler(ctx)
> 
> It could be useful to add a comment here that explains the lifetime of
> this goroutine, i.e. it returns when the context is Close()'d.

Ack.

>>  // Close closes the Context.
>>  func (ctx *Context) Close() error {
>> +   // Tell our SIGCHLD notifier to shut down, and wait for it to exit
>> +   // before we free the context.
>> +   if ctx.sigchld == nil {
> 
> Shouldn't this be `if ctx.sigchld != nil`?

Er, yes, indeed it should.  This has gone through too many iterations...

 -George

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [qemu-mainline test] 146185: regressions - FAIL

[osstest service owner]

flight 146185 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/146185/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-arm64-xsm   6 xen-buildfail REGR. vs. 144861
 build-arm64   6 xen-buildfail REGR. vs. 144861
 build-i386-xsm6 xen-buildfail REGR. vs. 144861
 build-armhf   6 xen-buildfail REGR. vs. 144861
 build-amd64-xsm   6 xen-buildfail REGR. vs. 144861
 build-i3866 xen-buildfail REGR. vs. 144861
 build-amd64   6 xen-buildfail REGR. vs. 144861

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)  blocked n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-rtds  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-pvshim 1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-cubietruck  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-shadow 1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)  blocked n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1) blocked n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-seattle   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-xsm   1 

Re: [Xen-devel] [PATCH v3 7/8] golang/xenlight: Notify xenlight of SIGCHLD

[Nick Rosbrook]

>  // Context represents a libxl_ctx.
>  type Context struct {
> -   ctx*C.libxl_ctx
> -   logger *C.xentoollog_logger_stdiostream
> +   ctx *C.libxl_ctx
> +   logger  *C.xentoollog_logger_stdiostream
> +   sigchld chan os.Signal
> +   sigchldDone chan bool

It's preferred to use `chan struct{}` for this pattern; it makes it
clear that the data sent over the channel has no meaning, and is only
intended to be used for synchronization.

> +   // ...and arrange to keep that promise.
> +   ctx.sigchld = make(chan os.Signal, 2)
> +   ctx.sigchldDone = make(chan bool, 1)
> +   signal.Notify(ctx.sigchld, syscall.SIGCHLD)
> +
> +   go sigchldHandler(ctx)

It could be useful to add a comment here that explains the lifetime of
this goroutine, i.e. it returns when the context is Close()'d.

>  // Close closes the Context.
>  func (ctx *Context) Close() error {
> +   // Tell our SIGCHLD notifier to shut down, and wait for it to exit
> +   // before we free the context.
> +   if ctx.sigchld == nil {

Shouldn't this be `if ctx.sigchld != nil`?

-NR

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH] libxl: event: Document lifetime API for libxl_childproc_setmode

[Ian Jackson]

There is already an identical comment for
libxl_osevent_register_hooks.

libxl_childproc_setmode's hooks parameter has the same property and
this should be documented.

Reported-by; George Dunlap 
Signed-off-by: Ian Jackson 
---
 tools/libxl/libxl_event.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/libxl/libxl_event.h b/tools/libxl/libxl_event.h
index d1517f7456..8d0aa6417e 100644
--- a/tools/libxl/libxl_event.h
+++ b/tools/libxl/libxl_event.h
@@ -548,6 +548,8 @@ typedef struct {
  * May not be called when libxl might have any child processes, or the
  * behaviour is undefined.  So it is best to call this at
  * initialisation.
+ *
+ * The value *hooks is not copied and must outlast the libxl_ctx.
  */
 void libxl_childproc_setmode(libxl_ctx *ctx, const libxl_childproc_hooks 
*hooks,
  void *user);
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 7/8] golang/xenlight: Notify xenlight of SIGCHLD

[George Dunlap]

On 1/17/20 4:52 PM, Ian Jackson wrote:
> George Dunlap writes ("[PATCH v3 7/8] golang/xenlight: Notify xenlight of 
> SIGCHLD"):
>> libxl forks external processes and waits for them to complete; it
>> therefore needs to be notified when children exit.
>>
>> In absence of instructions to the contrary, libxl sets up its own
>> SIGCHLD handlers.
>>
>> Golang always unmasks and handles SIGCHLD itself.  libxl thankfully
>> notices this and throws an assert() rather than clobbering SIGCHLD
>> handlers.
>>
>> Tell libxl that we'll be responsible for getting SIGCHLD notifications
>> to it.  Arrange for a channel in the context to receive notifications
>> on SIGCHLD, and set up a goroutine that will pass these on to libxl.
>>
>> NB that every libxl context needs a notification; so multiple contexts
>> will each spin up their own goroutine when opening a context, and shut
>> it down on close.
>>
>> libxl also wants to hold on to a const pointer to
>> xenlight_childproc_hooks rather than do a copy; so make a global
>> structure in C space and initialize it once on library creation.
>>
>> While here, add a few comments to make the context set-up a bit easier
>> to follow.
> 
> For what it's worth,
> 
> Reviewed-by: Ian Jackson 
> 
> However, I don't think I understand golang (and particularly the
> threading model etc.) well enough for that to mean I'm confident that
> this correct.

Thanks -- I mainly just wanted to give you the opportunity to spot any
obvious things I was doing wrong wrt libxl.  (For instance, an earlier
version of this patch had me destroying the libxl context before
shutting down the sigchld helper, which is obviously wrong.)

>> +func init() {
>> +// libxl for some reason wants to:
>> +// 1. Retain a copy to this pointer as long as the context is open, and
>> +// 2. Not free it when it's done
> 
> I found this comment a bit rude.  This is not an unusual approach for
> a pointer in a C API.>
> In Rust this would be called an `immutable borrow': the ctx borrows
> the contents of the pointer, promises not to modify it (and the caller
> ought to promise not to modify it either); but the caller retains
> ownership so when the ctx is done the borrow ends.

I'm sorry to be rude; I've deleted the comment.  But none of what you
said is obvious from the documentation; on the contrary:

---
...is equivalent to &{ libxl_sigchld_owner_libxl, 0, 0 }
---

...seems to imply that you can pass it a pointer to the stack.  And,
from an interface perspective, that seems obviously better to me --
rather than make the caller promise not to change the contents (to
who-knows-what result if they forget), it's much easier to just take a
local copy and update it with locks next time the function is called.

I was slightly more annoyed because Go's rule about C functions not
retaining pointers to Go memory meant I had to do some un-Golike things
to make this work; but that's nothing to do with libxl.

> Normally in C the struct would be `static const', so there is no need
> to allocate it or free it.
> 
> I think that ...
> 
>> +// Rather than alloc and free multiple copies, just keep a single
>> +// static copy in the C space (since C code isn't allowed to retain 
>> pointers
>> +// to go code), and initialize it once.
>> +C.xenlight_childproc_hooks.chldowner = C.libxl_sigchld_owner_mainloop
> 
> ... this is what this is doing ?

In fact, there's a global C variable declared here:

---
 #include 
+
+libxl_childproc_hooks xenlight_childproc_hooks;
 */
 import "C"
---

...and the line above just initialized it.  But on reflection I've
decided to do this:

---
/*

#cgo LDFLAGS: -lxenlight -lyajl -lxentoollog
#include 
#include 

static const libxl_childproc_hooks childproc_hooks = { .chldowner =
libxl_sigchld_owner_mainloop };

void xenlight_set_chldproc(libxl_ctx *ctx) {
libxl_childproc_setmode(ctx, _hooks, NULL);
}

*/
import "C"
---

That declares childproc_hooks as static const in the C space; and then
defines a C function which takes a libxl_ctx* and calls
libxl_childproc_setmode appropriately.  That way childproc_hooks can
enjoy the safety of static const.

>> +// The alternate would be to register a fork callback, such that the
>> +// xenlight package can make a single list of all children, and only
>> +// notify the specific libxl context(s) that have children woken.  But
>> +// it's not clear to me this will be much more work than having the
>> +// xenlight go library do the same thing; doing it in separate go
>> +// threads has the potential to do it in parallel.  Leave that as an
>> +// optimization for later if it turns out to be a bottleneck.
> 
> I think this is fine.

Thanks.

 -George

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [XEN PATCH v2 06/12] xen/test/livepatch: remove include of Config.mk

[Ross Lagerwall]

On 1/17/20 10:53 AM, Anthony PERARD wrote:
> livepatch/Makefile seems to only be used via Rules.mk, which already
> includes Config.mk, avoid the second include.
> 
> Signed-off-by: Anthony PERARD 

Reviewed-by: Ross Lagerwall 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH V8 4/4] x86/mm: Make use of the default access param from xc_altp2m_create_view

[Tamas K Lengyel]

On Fri, Jan 17, 2020 at 6:31 AM Alexandru Stefan ISAILA
 wrote:
>
> At this moment the default_access param from xc_altp2m_create_view is
> not used.
>
> This patch assigns default_access to p2m->default_access at the time of
> initializing a new altp2m view.
>
> Signed-off-by: Alexandru Isaila 
> Acked-by: Jan Beulich 

For the mem_access bits:
Acked-by: Tamas K Lengyel 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v4 11/18] x86/mem_sharing: ASSERT that p2m_set_entry succeeds

[Tamas K Lengyel]

On Fri, Jan 17, 2020 at 2:23 AM Jan Beulich  wrote:
>
> On 16.01.2020 17:12, Tamas K Lengyel wrote:
> > On Thu, Jan 16, 2020 at 9:07 AM Jan Beulich  wrote:
> >>
> >> On 08.01.2020 18:14, Tamas K Lengyel wrote:
> >>> Signed-off-by: Tamas K Lengyel 
> >>> ---
> >>>  xen/arch/x86/mm/mem_sharing.c | 42 +--
> >>>  1 file changed, 21 insertions(+), 21 deletions(-)
> >>>
> >>> diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c
> >>> index 93e7605900..3f36cd6bbc 100644
> >>> --- a/xen/arch/x86/mm/mem_sharing.c
> >>> +++ b/xen/arch/x86/mm/mem_sharing.c
> >>> @@ -1117,11 +1117,19 @@ int add_to_physmap(struct domain *sd, unsigned 
> >>> long sgfn, shr_handle_t sh,
> >>>  goto err_unlock;
> >>>  }
> >>>
> >>> +/*
> >>> + * Must succeed, we just read the entry and hold the p2m lock
> >>> + * via get_two_gfns.
> >>> + */
> >>>  ret = p2m_set_entry(p2m, _gfn(cgfn), smfn, PAGE_ORDER_4K,
> >>>  p2m_ram_shared, a);
> >>> +ASSERT(!ret);
> >>
> >> And there's no risk of -ENOMEM because of needing to split a
> >> larger order page? At the very least the reasoning in the
> >> comment would need further extending.
> >
> > No because we are plugging a hole in the domain. There is no larger
> > page mapped in here that would need to be broken up.
>
> p2m_is_hole() also covers p2m_mmio_dm and p2m_invalid. The former
> (should really be the latter) is what you'll get back for e.g. a
> GFN beyond max_mapped_pfn. Aiui such a "set" may then require
> table population, which may still yield -ENOMEM (at least EPT
> looks to return -ENOENT in this case instead; I guess I'll make
> a patch).

Yes, actually that is what's expected in the fork case to happen since
the fork has no entries in its EPT when it starts at all. So there
will be allocations happening there for the pagetable entries. But for
forks that's not of concern since we'll setup the same HAP allocation
the parent VM has during the fork hypercall. So it is guaranteed that
the fork will have the same amount of memory for its pagetables its
parent has.

Now as for using add_to_physmap on a non-forked VM when plugging a
hole like that, yes, I guess there is the possibility that the VM is
going to run out of space for its pagetable. So I guess we should skip
this patch.

Thanks,
Tamas

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 7/8] golang/xenlight: Notify xenlight of SIGCHLD

[Ian Jackson]

George Dunlap writes ("[PATCH v3 7/8] golang/xenlight: Notify xenlight of 
SIGCHLD"):
> libxl forks external processes and waits for them to complete; it
> therefore needs to be notified when children exit.
> 
> In absence of instructions to the contrary, libxl sets up its own
> SIGCHLD handlers.
> 
> Golang always unmasks and handles SIGCHLD itself.  libxl thankfully
> notices this and throws an assert() rather than clobbering SIGCHLD
> handlers.
> 
> Tell libxl that we'll be responsible for getting SIGCHLD notifications
> to it.  Arrange for a channel in the context to receive notifications
> on SIGCHLD, and set up a goroutine that will pass these on to libxl.
> 
> NB that every libxl context needs a notification; so multiple contexts
> will each spin up their own goroutine when opening a context, and shut
> it down on close.
> 
> libxl also wants to hold on to a const pointer to
> xenlight_childproc_hooks rather than do a copy; so make a global
> structure in C space and initialize it once on library creation.
> 
> While here, add a few comments to make the context set-up a bit easier
> to follow.

For what it's worth,

Reviewed-by: Ian Jackson 

However, I don't think I understand golang (and particularly the
threading model etc.) well enough for that to mean I'm confident that
this correct.

> +func init() {
> + // libxl for some reason wants to:
> + // 1. Retain a copy to this pointer as long as the context is open, and
> + // 2. Not free it when it's done

I found this comment a bit rude.  This is not an unusual approach for
a pointer in a C API.

In Rust this would be called an `immutable borrow': the ctx borrows
the contents of the pointer, promises not to modify it (and the caller
ought to promise not to modify it either); but the caller retains
ownership so when the ctx is done the borrow ends.

Normally in C the struct would be `static const', so there is no need
to allocate it or free it.

I think that ...

> + // Rather than alloc and free multiple copies, just keep a single
> + // static copy in the C space (since C code isn't allowed to retain 
> pointers
> + // to go code), and initialize it once.
> + C.xenlight_childproc_hooks.chldowner = C.libxl_sigchld_owner_mainloop

... this is what this is doing ?

> +// This should "play nicely" with other users of SIGCHLD as long as
> +// they don't reap libxl's processes.

I assume that nothing in golang will do this.  If something calls a
non-process-specific variant of wait* then you would need to somehow
capture the results and feed them to libxl_childproc_exited.

> +// The alternate would be to register a fork callback, such that the
> +// xenlight package can make a single list of all children, and only
> +// notify the specific libxl context(s) that have children woken.  But
> +// it's not clear to me this will be much more work than having the
> +// xenlight go library do the same thing; doing it in separate go
> +// threads has the potential to do it in parallel.  Leave that as an
> +// optimization for later if it turns out to be a bottleneck.

I think this is fine.

Thanks,
Ian.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 2/2] xsm: hide detailed Xen version from unprivileged guests

[Sergey Dyasli]

Hide the following information that can help identify the running Xen
binary version: XENVER_extraversion, XENVER_compile_info, XENVER_changeset.
This makes harder for malicious guests to fingerprint Xen to identify
exploitable systems.

Add explicit cases for XENVER_commandline and XENVER_build_id as well
for better code readability.

Signed-off-by: Sergey Dyasli 
---
v2 --> v3:
- Remove hvmloader filtering
- Add ASSERT_UNREACHABLE

v1 --> v2:
- Added xsm_filter_denied() to hvmloader instead of modifying xen_deny()
- Made behaviour the same for both Release and Debug builds
- XENVER_capabilities is no longer hided

CC: Andrew Cooper 
CC: George Dunlap 
CC: Ian Jackson 
CC: Jan Beulich 
CC: Julien Grall 
CC: Konrad Rzeszutek Wilk 
CC: Stefano Stabellini 
CC: Wei Liu 
CC: Daniel De Graaf 
CC: Doug Goldstein 
---
 xen/include/xsm/dummy.h | 15 +++
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
index b8e185e6fa..c00186d7b6 100644
--- a/xen/include/xsm/dummy.h
+++ b/xen/include/xsm/dummy.h
@@ -750,16 +750,23 @@ static XSM_INLINE int xsm_xen_version (XSM_DEFAULT_ARG 
uint32_t op)
 case XENVER_get_features:
 /* These sub-ops ignore the permission checks and return data. */
 return 0;
-case XENVER_extraversion:
-case XENVER_compile_info:
+
 case XENVER_capabilities:
-case XENVER_changeset:
 case XENVER_pagesize:
 case XENVER_guest_handle:
 /* These MUST always be accessible to any guest by default. */
 return xsm_default_action(XSM_HOOK, current->domain, NULL);
-default:
+
+case XENVER_extraversion:
+case XENVER_compile_info:
+case XENVER_changeset:
+case XENVER_commandline:
+case XENVER_build_id:
 return xsm_default_action(XSM_PRIV, current->domain, NULL);
+
+default:
+ASSERT_UNREACHABLE();
+return -EPERM;
 }
 }
 
-- 
2.17.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 1/2] xsm: add config option for denied string

[Sergey Dyasli]

Signed-off-by: Sergey Dyasli 
---
v2 --> v3:
- new patch

CC: Andrew Cooper 
CC: George Dunlap 
CC: Ian Jackson 
CC: Jan Beulich 
CC: Julien Grall 
CC: Konrad Rzeszutek Wilk 
CC: Stefano Stabellini 
CC: Wei Liu 
CC: Daniel De Graaf 
CC: Doug Goldstein 
---
 xen/common/Kconfig   | 8 
 xen/common/version.c | 2 +-
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index b3d161d057..f0a3f0da0f 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -236,6 +236,14 @@ choice
bool "SILO" if XSM_SILO
 endchoice
 
+config XSM_DENIED_STRING
+   string "xen_version denied string"
+   default ""
+   depends on XSM
+   ---help---
+ A string which substitutes sensitive information returned via
+ xen_version hypercall to non-privileged guests
+
 config LATE_HWDOM
bool "Dedicated hardware domain"
default n
diff --git a/xen/common/version.c b/xen/common/version.c
index 937eb1281c..14b205af48 100644
--- a/xen/common/version.c
+++ b/xen/common/version.c
@@ -67,7 +67,7 @@ const char *xen_banner(void)
 
 const char *xen_deny(void)
 {
-return "";
+return CONFIG_XSM_DENIED_STRING;
 }
 
 static const void *build_id_p __read_mostly;
-- 
2.17.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH] EPT: do away with hidden GUEST_TABLE_MAP_FAILED == 0 assumptions

[Jan Beulich]

The code is quite a bit easier to read and to reason about this way,
I think.

In ept_set_entry() additionally change the function's return value in
the MAP_FAILED case to -ENOMEM; -ENOENT would be applicable only when
ept_next_entry() was invoked with "read_only" set to true.

In two cases, where ept_next_level() follows an ept_split_superpage()
invocation, actually tighten the loop exit condition from
"== MAP_FAILED" to "!= NORMAL_PAGE". Continuing these loops for other
than NORMAL_PAGE is invalid, and there are ASSERT()s in place after
these loops.

Also reduce the scope of "ret" variables where possible, in particular
to better distinguish them from "rc" often used in the same function.

Finally drop pointless "else" in a few areas touched anyway.

Signed-off-by: Jan Beulich 

--- a/xen/arch/x86/mm/p2m-ept.c
+++ b/xen/arch/x86/mm/p2m-ept.c
@@ -292,8 +292,8 @@ static bool_t ept_split_super_page(struc
  * and map the next table, if available.  If the entry is empty
  * and read_only is set, 
  * Return values:
- *  0: Failed to map.  Either read_only was set and the entry was
- *   empty, or allocating a new page failed.
+ *  GUEST_TABLE_MAP_FAILED: Failed to map.  Either read_only was set and the
+ *   entry was empty, or allocating a new page failed.
  *  GUEST_TABLE_NORMAL_PAGE: next level mapped normally
  *  GUEST_TABLE_SUPER_PAGE:
  *   The next entry points to a superpage, and caller indicates
@@ -404,12 +404,13 @@ static int ept_invalidate_emt_range(stru
 ept_entry_t *table;
 unsigned long gfn_remainder = first_gfn;
 unsigned int i, index;
-int wrc, rc = 0, ret = GUEST_TABLE_MAP_FAILED;
+int wrc, rc = 0;
 
 table = map_domain_page(pagetable_get_mfn(p2m_get_pagetable(p2m)));
 for ( i = p2m->ept.wl; i > target; --i )
 {
-ret = ept_next_level(p2m, 1, , _remainder, i);
+int ret = ept_next_level(p2m, 1, , _remainder, i);
+
 if ( ret == GUEST_TABLE_MAP_FAILED )
 goto out;
 if ( ret != GUEST_TABLE_NORMAL_PAGE )
@@ -434,8 +435,10 @@ static int ept_invalidate_emt_range(stru
 ASSERT(wrc == 0);
 
 for ( ; i > target; --i )
-if ( !ept_next_level(p2m, 1, , _remainder, i) )
+if ( ept_next_level(p2m, 1, , _remainder, i) !=
+ GUEST_TABLE_NORMAL_PAGE )
 break;
+/* We just installed the pages we need. */
 ASSERT(i == target);
 }
 
@@ -694,12 +697,12 @@ ept_set_entry(struct p2m_domain *p2m, gf
 for ( i = ept->wl; i > target; i-- )
 {
 ret = ept_next_level(p2m, 0, , _remainder, i);
-if ( !ret )
+if ( ret == GUEST_TABLE_MAP_FAILED )
 {
-rc = -ENOENT;
+rc = -ENOMEM;
 goto out;
 }
-else if ( ret != GUEST_TABLE_NORMAL_PAGE )
+if ( ret != GUEST_TABLE_NORMAL_PAGE )
 break;
 }
 
@@ -756,7 +759,8 @@ ept_set_entry(struct p2m_domain *p2m, gf
 
 /* then move to the level we want to make real changes */
 for ( ; i > target; i-- )
-if ( !ept_next_level(p2m, 0, , _remainder, i) )
+if ( ept_next_level(p2m, 0, , _remainder, i) !=
+ GUEST_TABLE_NORMAL_PAGE )
 break;
 /* We just installed the pages we need. */
 ASSERT(i == target);
@@ -859,7 +863,6 @@ static mfn_t ept_get_entry(struct p2m_do
 ept_entry_t *ept_entry;
 u32 index;
 int i;
-int ret = 0;
 bool_t recalc = 0;
 mfn_t mfn = INVALID_MFN;
 struct ept_data *ept = >ept;
@@ -883,13 +886,15 @@ static mfn_t ept_get_entry(struct p2m_do
 
 for ( i = ept->wl; i > 0; i-- )
 {
+int ret;
+
 retry:
 if ( table[gfn_remainder >> (i * EPT_TABLE_ORDER)].recalc )
 recalc = 1;
 ret = ept_next_level(p2m, 1, , _remainder, i);
-if ( !ret )
+if ( ret == GUEST_TABLE_MAP_FAILED )
 goto out;
-else if ( ret == GUEST_TABLE_POD_PAGE )
+if ( ret == GUEST_TABLE_POD_PAGE )
 {
 if ( !(q & P2M_ALLOC) )
 {
@@ -905,10 +910,9 @@ static mfn_t ept_get_entry(struct p2m_do
 
 if ( p2m_pod_demand_populate(p2m, gfn_, i * EPT_TABLE_ORDER) )
 goto retry;
-else
-goto out;
+goto out;
 }
-else if ( ret == GUEST_TABLE_SUPER_PAGE )
+if ( ret == GUEST_TABLE_SUPER_PAGE )
 break;
 }
 
@@ -1289,7 +1293,6 @@ static void ept_dump_p2m_table(unsigned
 ept_entry_t *table, *ept_entry;
 int order;
 int i;
-int ret = 0;
 unsigned long gfn, gfn_remainder;
 unsigned long record_counter = 0;
 struct p2m_domain *p2m;
@@ -1307,6 +1310,7 @@ static void ept_dump_p2m_table(unsigned
 for ( gfn = 0; gfn <= p2m->max_mapped_pfn; gfn += 1UL << order )
 {
 char c = 0;
+int ret = GUEST_TABLE_MAP_FAILED;
 
 gfn_remainder = gfn;
 table = 

Re: [Xen-devel] [PATCH] build: fix dependency file generation with ENFORCE_UNIQUE_SYMBOLS=y

[Andrew Cooper]

On 17/01/2020 15:44, Jan Beulich wrote:
> The recorded file, unless overridden by -MQ (or -MT) is that specified
> by -o, which doesn't produce correct dependencies and hence will cause
> failure to re-build when included files change.
>
> Fixes: 81ecb38b83b0 ("build: provide option to disambiguate symbol names")
> Reported-by: Andrew Cooper 
> Signed-off-by: Jan Beulich 

Thanks - this fixes my issue.

Tested-by: Andrew Cooper 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] x86/apic: fix disabling LVT0 in disconnect_bsp_APIC

[Jan Beulich]

On 17.01.2020 17:08, Roger Pau Monné wrote:
> On Fri, Jan 17, 2020 at 04:56:00PM +0100, Jan Beulich wrote:
>> On 17.01.2020 16:09, Roger Pau Monne wrote:
>>> The Intel SDM states:
>>>
>>> "When an illegal vector value (0 to 15) is written to a LVT entry and
>>> the delivery mode is Fixed (bits 8-11 equal 0), the APIC may signal an
>>> illegal vector error, without regard to whether the mask bit is set or
>>> whether an interrupt is actually seen on the input."
>>>
>>> And that's exactly what's currently done in disconnect_bsp_APIC when
>>> virt_wire_setup is true and LVT LINT0 is being masked. By writing only
>>> APIC_LVT_MASKED Xen is actually setting the vector to 0 and the
>>> delivery mode to Fixed (0), and hence it triggers an APIC error even
>>> when the LVT entry is masked.
>>
>> But there are many more instances where we (have a risk to) do so,
>> most notably in clear_local_APIC(). The two step logic there is
>> anyway somewhat in conflict with the citation above.
> 
> clear_local_APIC masks the error vector before doing any write, and
> clears ESR afterwards, there's a comment at the top:
> 
> "Masking an LVT entry on a P6 can trigger a local APIC error
> if the vector is zero. Mask LVTERR first to prevent this."
> 
> We could do the same (ie: mask LVTERR first and clear ESR afterwards)
> if that seems preferable. There's a maxlvt check in clear_local_APIC,
> but the sdm doesn't specify anyway to check if the lapic will accept a
> masked vector 0 write or not, so not sure whether we should replicate
> that check or just do it unconditionally on both disconnect_bsp_APIC
> and clear_local_APIC.

I think doing it the most careful way is going to be best. I find it
surprising anyway that disconnect_bsp_APIC() doesn't write LVTERR
(or other LVTs except for LVT1) at all. The function looks to have a
goal of putting the APIC back into the state that we found it when
booting.

Jan

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [XEN PATCH v3 2/6] xen: Have Kconfig check $(CC)'s version

[Anthony PERARD]

On Thu, Jan 16, 2020 at 01:40:39PM +0100, Jan Beulich wrote:
> On 16.01.2020 13:29, Anthony PERARD wrote:
> Indeed, hence also my "as suggested before". I remain unconvinced
> that is it useful to have e.g.
> 
> CONFIG_GCC_VERSION=80300
> CONFIG_CLANG_VERSION=0
> 
> in xen/.config. This is at best confusing, because it may not
> represent what the system actually has installed (which I realize
> is also not the intention, but the variable naming suggests that
> this is what was found on the system; I have no better naming
> suggestion, to preempt a possible question to this effect).

After a talk on #xendevel yesterday, I have George agreeing that we
should keep the same behavior as the one Linux have. And Ian saying that
we should copy entire files where we can. If we modify the behavior of
%_VERSION, it would make it more difficult to copy entire files from
%Linux later.

So, now, can we finally commit the patch series, with both %_VERSION
set, and let this bikeshedding rest, and move on?

Thank you,

-- 
Anthony PERARD

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH V8 1/4] x86/mm: Add array_index_nospec to guest provided index values

[Jan Beulich]

On 17.01.2020 14:31, Alexandru Stefan ISAILA wrote:
> This patch aims to sanitize indexes, potentially guest provided
> values, for altp2m_eptp[] and altp2m_p2m[] arrays.
> 
> Requested-by: Jan Beulich 
> Signed-off-by: Alexandru Isaila 
> Acked-by: Tamas K Lengyel 

Reviewed-by: Jan Beulich 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] x86/apic: fix disabling LVT0 in disconnect_bsp_APIC

[Roger Pau Monné]

On Fri, Jan 17, 2020 at 03:30:44PM +, Andrew Cooper wrote:
> On 17/01/2020 15:09, Roger Pau Monne wrote:
> > The Intel SDM states:
> >
> > "When an illegal vector value (0 to 15) is written to a LVT entry and
> > the delivery mode is Fixed (bits 8-11 equal 0), the APIC may signal an
> > illegal vector error, without regard to whether the mask bit is set or
> > whether an interrupt is actually seen on the input."
> >
> > And that's exactly what's currently done in disconnect_bsp_APIC when
> > virt_wire_setup is true and LVT LINT0 is being masked. By writing only
> > APIC_LVT_MASKED Xen is actually setting the vector to 0 and the
> > delivery mode to Fixed (0), and hence it triggers an APIC error even
> > when the LVT entry is masked.
> >
> > This would usually manifest when Xen is being shut down, as that's
> > where disconnect_bsp_APIC is called:
> >
> > (XEN) APIC error on CPU0: 40(00)
> >
> > Fix this by reusing the current LVT LINT0 value and just adding the
> > mask bit to it.
> >
> > Reported-by: Andrew Cooper 
> > Signed-off-by: Roger Pau Monné 
> > ---
> >  xen/arch/x86/apic.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/xen/arch/x86/apic.c b/xen/arch/x86/apic.c
> > index a6a7754d77..e4363639bd 100644
> > --- a/xen/arch/x86/apic.c
> > +++ b/xen/arch/x86/apic.c
> > @@ -281,7 +281,8 @@ void disconnect_bsp_APIC(int virt_wire_setup)
> >  }
> >  else {
> >  /* Disable LVT0 */
> > -apic_write(APIC_LVT0, APIC_LVT_MASKED);
> > +value = apic_read(APIC_LVT0);
> > +apic_write(APIC_LVT0, value | APIC_LVT_MASKED);
> >  }
> 
> This really is ugly.  It seems that we can't write LVT0 to the same
> state that it has after reset/INIT.
> 
> For the code however, both halves of the if() condition do a
> read/modify/write.  It would be nicer to have the read and write common,
> with modify alone having the if().

As said on my reply to Jan, we could do the same as clear_local_APIC
if that seems preferable.

Thanks, Roger.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] x86/apic: fix disabling LVT0 in disconnect_bsp_APIC

[Roger Pau Monné]

On Fri, Jan 17, 2020 at 04:56:00PM +0100, Jan Beulich wrote:
> On 17.01.2020 16:09, Roger Pau Monne wrote:
> > The Intel SDM states:
> > 
> > "When an illegal vector value (0 to 15) is written to a LVT entry and
> > the delivery mode is Fixed (bits 8-11 equal 0), the APIC may signal an
> > illegal vector error, without regard to whether the mask bit is set or
> > whether an interrupt is actually seen on the input."
> > 
> > And that's exactly what's currently done in disconnect_bsp_APIC when
> > virt_wire_setup is true and LVT LINT0 is being masked. By writing only
> > APIC_LVT_MASKED Xen is actually setting the vector to 0 and the
> > delivery mode to Fixed (0), and hence it triggers an APIC error even
> > when the LVT entry is masked.
> 
> But there are many more instances where we (have a risk to) do so,
> most notably in clear_local_APIC(). The two step logic there is
> anyway somewhat in conflict with the citation above.

clear_local_APIC masks the error vector before doing any write, and
clears ESR afterwards, there's a comment at the top:

"Masking an LVT entry on a P6 can trigger a local APIC error
if the vector is zero. Mask LVTERR first to prevent this."

We could do the same (ie: mask LVTERR first and clear ESR afterwards)
if that seems preferable. There's a maxlvt check in clear_local_APIC,
but the sdm doesn't specify anyway to check if the lapic will accept a
masked vector 0 write or not, so not sure whether we should replicate
that check or just do it unconditionally on both disconnect_bsp_APIC
and clear_local_APIC.

Thanks, Roger.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2] x86/hvmloader: round up memory BAR size to 4K

[Jason Andryuk]

On Fri, Jan 17, 2020 at 6:08 AM Roger Pau Monne  wrote:
>
> When placing memory BARs with sizes smaller than 4K multiple memory
> BARs can end up mapped to the same guest physical address, and thus
> won't work correctly.
>
> Round up all memory BAR sizes to be at least 4K, so that they are
> naturally aligned to a page size and thus don't end up sharing a page.
> Also add a couple of asserts to the current code to make sure the MMIO
> hole is properly sized and aligned.
>
> Note that the guest can still move the BARs around and create this
> collisions, and that BARs not filling up a physical page might leak
> access to other MMIO regions placed in the same host physical page.
>
> This is however no worse than what's currently done, and hence should
> be considered an improvement over the current state.
>
> Reported-by: Jason Andryuk 
> Signed-off-by: Roger Pau Monné 
> ---
> Cc: Jason Andryuk 
> ---
> Changes since v1:
>  - Do the round up when sizing the BARs, so that the MMIO hole is
>correctly sized.
>  - Add some asserts that the hole is properly sized and size-aligned.
>  - Dropped Jason Tested-by since the code has changed.
> ---
> Jason, can you give this a spin? Thanks.

Tested-by: Jason Andryuk 

Thanks!

-Jason

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 1/8] golang/xenlight: Don't try to marshall zero-length arrays in fromC

[George Dunlap]

On 1/17/20 3:57 PM, George Dunlap wrote:
> The current fromC array code will do the "magic" casting and
> martialling even when num_foo variable is 0.  Go crashes when doing
> the cast.
> 
> Only do array marshalling if the number of elements is non-zero;
> otherwise, leave the target pointer empty (nil for Go slices, NULL for
> C arrays).
> 
> Signed-off-by: George Dunlap 
> ---
> v2:
> - Remove toC part of this, which has been folded into Nick's patch
>   series.

Er, obviously the subject line should say "v2", for the whole series. :-/

 -George

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 8/8] RFC: Sketch constructors, DomainCreateNew

[George Dunlap]

This is a sketch of functionality suitable for creating a basic
domain, with a disk and a vif.  DomainConfig, DeviceDisk, and
DeviceNic types are all created using constructor functions, which
initialize them with libxl's defaults.

DomainCreateNew takes the config and calls without any updates.

Obviously some of these will need to be changed it we switch to
passing references to .toC() rather than passing back by value.

The main purpose of this is to allow testing of creating a hard-coded
domain.

Creating a domain would look like this:

// type = "pv"
dconf, err := xl.NewDomainConfig(xl.DomainTypePv)
if err != nil {
fmt.Printf("NewDomainConfig: %v\n", err)
return
}
dconf.CInfo.Type = xl.DomainTypePv
// name = "c6-01"
dconf.CInfo.Name = "c6-01"
// vcpus=4
dconf.BInfo.MaxVcpus = 4
// memory = "2048"
dconf.BInfo.MaxMemkb = 2048 * 1024
dconf.BInfo.TargetMemkb = 2048 * 1024
// on_crash = 'destroy'
dconf.OnCrash = xl.ActionOnShutdownDestroy
// bootloader = "pygrub"
dconf.BInfo.Bootloader = "pygrub"
// disk = [ 'vdev=hda,format=raw,target=/images/c6-01.raw']
{
disk, err := xl.NewDeviceDisk()
if err != nil {
fmt.Printf("NewDeviceDisk: %v\n", err)
return
}
disk.Vdev = "hda"
//disk.DiskBackend = xl.DiskBackendPhy
disk.Format = xl.DiskFormatRaw
disk.Readwrite = 1
disk.PdevPath = "/images/c6-01.raw"
dconf.Disks = append(dconf.Disks, *disk)
}
// vif = [ 'mac=5a:5b:d6:f1:d6:b4' ]
{
vif, err := xl.NewDeviceNic()
if err != nil {
fmt.Printf("NewDeviceNic: %v\n", err)
return
}
vif.Mac = xl.Mac{ 0x5a, 0x5b, 0xd6, 0x31, 0xd6, 0xb4 }
dconf.Nics = append(dconf.Nics, *vif)
}
// serial='pty' # HVM only

did, err := ctx.DomainCreateNew(dconf)

if err != nil {
fmt.Printf("Creating domain: %v\n", err)
return
}

fmt.Printf("Domain %s(%d) created successfully", dconf.CInfo.Name, did)


Signed-off-by: George Dunlap 
---
CC: Nick Rosbrook 
---
 tools/golang/xenlight/xenlight.go | 66 +++
 1 file changed, 66 insertions(+)

diff --git a/tools/golang/xenlight/xenlight.go 
b/tools/golang/xenlight/xenlight.go
index c462e4bb42..5a21a2b9b8 100644
--- a/tools/golang/xenlight/xenlight.go
+++ b/tools/golang/xenlight/xenlight.go
@@ -1113,3 +1113,69 @@ func (Ctx *Context) PrimaryConsoleGetTty(domid uint32) 
(path string, err error)
path = C.GoString(cpath)
return
 }
+
+func NewDomainConfig(t DomainType) (*DomainConfig, error) {
+   var cconfig C.libxl_domain_config
+
+   C.libxl_domain_config_init()
+   C.libxl_domain_build_info_init_type(_info, 
C.libxl_domain_type(t))
+
+   gconfig := {}
+   err := gconfig.fromC()
+   if err != nil {
+   return nil, err
+   }
+
+   return gconfig, nil
+}
+
+func NewDeviceDisk() (*DeviceDisk, error) {
+   var ctype C.libxl_device_disk
+
+   C.libxl_device_disk_init()
+
+   gtype := {}
+   err := gtype.fromC()
+
+   if err != nil {
+   return nil, err
+   }
+
+   return gtype, nil
+}
+
+func NewDeviceNic() (*DeviceNic, error) {
+   var ctype C.libxl_device_nic
+
+   C.libxl_device_nic_init()
+
+   gtype := {}
+   err := gtype.fromC()
+
+   if err != nil {
+   return nil, err
+   }
+
+   return gtype, nil
+}
+
+// int libxl_domain_create_new(libxl_ctx *ctx, libxl_domain_config *d_config,
+// uint32_t *domid,
+// const libxl_asyncop_how *ao_how,
+// const libxl_asyncprogress_how *aop_console_how)
+func (Ctx *Context) DomainCreateNew(config *DomainConfig) (Domid, error) {
+   var cdomid C.uint32_t
+   var cconfig C.libxl_domain_config
+   err := config.toC()
+   if err != nil {
+   return Domid(0), fmt.Errorf("converting domain config to C: 
%v", err)
+   }
+   defer C.libxl_domain_config_dispose()
+
+   ret := C.libxl_domain_create_new(Ctx.ctx, , , nil, nil)
+   if ret != 0 {
+   return Domid(0), Error(ret)
+   }
+
+   return Domid(cdomid), nil
+}
-- 
2.24.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 3/8] go/xenlight: More informative error messages

[George Dunlap]

If an error is encountered deep in a complicated data structure, it's
often difficult to tell where the error actually is.  Make the error
message from the generated toC() and fromC() structures more
informative by tagging which field being converted encountered the
error.  This will have the effect of giving a "stack trace" of the
failure inside a nested data structure.

NB that my version of python insists on reordering a couple of switch
statements for some reason; In other patches I've reverted those
changes, but in this case it's more difficult because they interact
with actual code changes.  I'll leave this here for now, as we're
going to remove helpers.gen.go from being tracked by git at some point
in the near future anyway.

Signed-off-by: George Dunlap 
---
v2:
- Keep error messages lower case
- Actually implement .toC changes

CC: Nick Rosbrook 
---
 tools/golang/xenlight/gengotypes.py  |  10 +-
 tools/golang/xenlight/helpers.gen.go | 530 +--
 2 files changed, 270 insertions(+), 270 deletions(-)

diff --git a/tools/golang/xenlight/gengotypes.py 
b/tools/golang/xenlight/gengotypes.py
index ad2c573da9..17b0ca00bd 100644
--- a/tools/golang/xenlight/gengotypes.py
+++ b/tools/golang/xenlight/gengotypes.py
@@ -314,7 +314,7 @@ def xenlight_golang_convert_from_C(ty = None, outer_name = 
None, cvarname = None
 # If the type is not castable, we need to call its fromC
 # function.
 s += 'if err := x.{}.fromC(&{}.{});'.format(goname,cvarname,cname)
-s += 'err != nil {\n return err \n}\n'
+s += 'err != nil {{\nreturn fmt.Errorf("converting field {}: %v", err) 
\n}}\n'.format(goname)
 
 elif gotypename == 'string':
 # Use the cgo helper for converting C strings.
@@ -389,7 +389,7 @@ def xenlight_golang_union_from_C(ty = None, union_name = 
'', struct_name = ''):
 
 s += 'var {} {}\n'.format(goname, gotype)
 s += 'if err := {}.fromC(xc);'.format(goname)
-s += 'err != nil {\n return err \n}\n'
+s += 'err != nil {{\n return fmt.Errorf("converting field {}: %v", 
err) \n}}\n'.format(goname)
 
 field_name = xenlight_golang_fmt_name('{}_union'.format(keyname))
 s += 'x.{} = {}\n'.format(field_name, goname)
@@ -432,7 +432,7 @@ def xenlight_golang_array_from_C(ty = None):
 s += 'x.{}[i] = {}(v)\n'.format(goname, gotypename)
 else:
 s += 'if err := x.{}[i].fromC(); err != nil {{\n'.format(goname)
-s += 'return err }\n'
+s += 'return fmt.Errorf("converting field {}: %v", err) 
}}\n'.format(goname)
 
 s += '}\n}\n'
 
@@ -513,7 +513,7 @@ def xenlight_golang_convert_to_C(ty = None, outer_name = 
None,
 if not is_castable:
 s += 'if err := {}.{}.toC(&{}.{}); err != nil 
{{\n'.format(govarname,goname,

cvarname,cname)
-s += 'return err\n}\n'
+s += 'return fmt.Errorf("converting field {}: %v", err) 
\n}}\n'.format(goname)
 
 elif gotypename == 'string':
 # Use the cgo helper for converting C strings.
@@ -615,7 +615,7 @@ def xenlight_golang_array_to_C(ty = None):
  
golenvar,golenvar)
 s += 'for i,v := range x.{} {{\n'.format(goname)
 s += 'if err := v.toC({}[i]); err != nil {{\n'.format(goname)
-s += 'return err\n'
+s += 'return fmt.Errorf("converting field {}: %v", err) \n'.format(goname)
 s += '}\n}\n}\n'
 
 return s
diff --git a/tools/golang/xenlight/helpers.gen.go 
b/tools/golang/xenlight/helpers.gen.go
index 889807d928..078c37f1c8 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -92,13 +92,13 @@ func (x *VgaInterfaceInfo) toC(xc 
*C.libxl_vga_interface_info) (err error) {
 
 func (x *VncInfo) fromC(xc *C.libxl_vnc_info) error {
if err := x.Enable.fromC(); err != nil {
-   return err
+   return fmt.Errorf("converting field Enable: %v", err)
}
x.Listen = C.GoString(xc.listen)
x.Passwd = C.GoString(xc.passwd)
x.Display = int(xc.display)
if err := x.Findunused.fromC(); err != nil {
-   return err
+   return fmt.Errorf("converting field Findunused: %v", err)
}
 
return nil
@@ -112,7 +112,7 @@ func (x *VncInfo) toC(xc *C.libxl_vnc_info) (err error) {
}()
 
if err := x.Enable.toC(); err != nil {
-   return err
+   return fmt.Errorf("converting field Enable: %v", err)
}
if x.Listen != "" {
xc.listen = C.CString(x.Listen)
@@ -122,7 +122,7 @@ func (x *VncInfo) toC(xc *C.libxl_vnc_info) (err error) {
}
xc.display = C.int(x.Display)
if err := x.Findunused.toC(); err != nil {
-   return err
+   return fmt.Errorf("converting field Findunused: %v", err)
}
 
return nil
@@ -130,23 

[Xen-devel] [PATCH v3 7/8] golang/xenlight: Notify xenlight of SIGCHLD

[George Dunlap]

libxl forks external processes and waits for them to complete; it
therefore needs to be notified when children exit.

In absence of instructions to the contrary, libxl sets up its own
SIGCHLD handlers.

Golang always unmasks and handles SIGCHLD itself.  libxl thankfully
notices this and throws an assert() rather than clobbering SIGCHLD
handlers.

Tell libxl that we'll be responsible for getting SIGCHLD notifications
to it.  Arrange for a channel in the context to receive notifications
on SIGCHLD, and set up a goroutine that will pass these on to libxl.

NB that every libxl context needs a notification; so multiple contexts
will each spin up their own goroutine when opening a context, and shut
it down on close.

libxl also wants to hold on to a const pointer to
xenlight_childproc_hooks rather than do a copy; so make a global
structure in C space and initialize it once on library creation.

While here, add a few comments to make the context set-up a bit easier
to follow.

Signed-off-by: George Dunlap 
---
v2:
- Fix unsafe libxl_childproc_hooks pointer behavior
- Close down the SIGCHLD handler first, and make sure it's exited
  before closing the context
- Explicitly decide to have a separate goroutine per ctx

NB that due to a bug in libxl, this will hang without Ian's "[PATCH v2
00/10] libxl: event: Fix hang for some applications" series.

CC: Nick Rosbrook 
CC: Ian Jackson 
---
 tools/golang/xenlight/xenlight.go | 72 ++-
 1 file changed, 70 insertions(+), 2 deletions(-)

diff --git a/tools/golang/xenlight/xenlight.go 
b/tools/golang/xenlight/xenlight.go
index 662b266250..c462e4bb42 100644
--- a/tools/golang/xenlight/xenlight.go
+++ b/tools/golang/xenlight/xenlight.go
@@ -20,6 +20,8 @@ package xenlight
 #cgo LDFLAGS: -lxenlight -lyajl -lxentoollog
 #include 
 #include 
+
+libxl_childproc_hooks xenlight_childproc_hooks;
 */
 import "C"
 
@@ -33,6 +35,9 @@ import "C"
 
 import (
"fmt"
+   "os"
+   "os/signal"
+   "syscall"
"unsafe"
 )
 
@@ -72,10 +77,49 @@ func (e Error) Error() string {
return fmt.Sprintf("libxl error: %d", e)
 }
 
+func init() {
+   // libxl for some reason wants to:
+   // 1. Retain a copy to this pointer as long as the context is open, and
+   // 2. Not free it when it's done
+   //
+   // Rather than alloc and free multiple copies, just keep a single
+   // static copy in the C space (since C code isn't allowed to retain 
pointers
+   // to go code), and initialize it once.
+   C.xenlight_childproc_hooks.chldowner = C.libxl_sigchld_owner_mainloop
+}
+
 // Context represents a libxl_ctx.
 type Context struct {
-   ctx*C.libxl_ctx
-   logger *C.xentoollog_logger_stdiostream
+   ctx *C.libxl_ctx
+   logger  *C.xentoollog_logger_stdiostream
+   sigchld chan os.Signal
+   sigchldDone chan bool
+}
+
+// Golang always unmasks SIGCHLD, and internally has ways of
+// distributing SIGCHLD to multiple recipients.  libxl has provision
+// for this model: just tell it when a SIGCHLD happened, and it will
+// look after its own processes.
+//
+// This should "play nicely" with other users of SIGCHLD as long as
+// they don't reap libxl's processes.
+//
+// Every context needs to be notified on each SIGCHLD; so spin up a
+// new goroutine for each context.  If there are a large number of contexts,
+// this means each context will be woken up looking through its own list of 
children.
+//
+// The alternate would be to register a fork callback, such that the
+// xenlight package can make a single list of all children, and only
+// notify the specific libxl context(s) that have children woken.  But
+// it's not clear to me this will be much more work than having the
+// xenlight go library do the same thing; doing it in separate go
+// threads has the potential to do it in parallel.  Leave that as an
+// optimization for later if it turns out to be a bottleneck.
+func sigchldHandler(ctx *Context) {
+   for _ = range ctx.sigchld {
+   go C.libxl_childproc_sigchld_occurred(ctx.ctx)
+   }
+   close(ctx.sigchldDone)
 }
 
 // NewContext returns a new Context.
@@ -89,19 +133,43 @@ func NewContext() (ctx *Context, err error) {
}
}()
 
+   // Create a logger
ctx.logger = C.xtl_createlogger_stdiostream(C.stderr, C.XTL_DEBUG, 0)
 
+   // Allocate a context
ret := C.libxl_ctx_alloc(, C.LIBXL_VERSION, 0,
(*C.xentoollog_logger)(unsafe.Pointer(ctx.logger)))
if ret != 0 {
return ctx, Error(ret)
}
 
+   // Tell libxl that we'll be dealing with SIGCHLD...
+   C.libxl_childproc_setmode(ctx.ctx, _childproc_hooks, nil)
+
+   // ...and arrange to keep that promise.
+   ctx.sigchld = make(chan os.Signal, 2)
+   ctx.sigchldDone = make(chan bool, 1)
+   signal.Notify(ctx.sigchld, syscall.SIGCHLD)
+
+   go sigchldHandler(ctx)
+
return 

[Xen-devel] [PATCH v3 4/8] golang/xenlight: Errors are negative

[George Dunlap]

Commit 871e51d2d4 changed the sign on the xenlight error types (making
the values negative, same as the C-generated constants), but failed to
flip the sign in the Error() string function.  The result is that
ErrorNonspecific.String() prints "libxl error: 1" rather than the
human-readable error message.

Get rid of the whole issue by making libxlErrors a map, and mapping
actual error values to string, falling back to printing the actual
value of the Error type if it's not present.

Signed-off-by: George Dunlap 
---
v2:
- Convert libxlErrors to a map.

CC: Nick Rosbrook 
---
 tools/golang/xenlight/xenlight.go | 62 +++
 1 file changed, 30 insertions(+), 32 deletions(-)

diff --git a/tools/golang/xenlight/xenlight.go 
b/tools/golang/xenlight/xenlight.go
index 1299981713..aa1e63a61a 100644
--- a/tools/golang/xenlight/xenlight.go
+++ b/tools/golang/xenlight/xenlight.go
@@ -36,42 +36,40 @@ import (
"unsafe"
 )
 
-var libxlErrors = [...]string{
-   -ErrorNonspecific:  "Non-specific error",
-   -ErrorVersion:  "Wrong version",
-   -ErrorFail: "Failed",
-   -ErrorNi:   "Not Implemented",
-   -ErrorNomem:"No memory",
-   -ErrorInval:"Invalid argument",
-   -ErrorBadfail:  "Bad Fail",
-   -ErrorGuestTimedout:"Guest timed out",
-   -ErrorTimedout: "Timed out",
-   -ErrorNoparavirt:   "No Paravirtualization",
-   -ErrorNotReady: "Not ready",
-   -ErrorOseventRegFail:   "OS event registration failed",
-   -ErrorBufferfull:   "Buffer full",
-   -ErrorUnknownChild: "Unknown child",
-   -ErrorLockFail: "Lock failed",
-   -ErrorJsonConfigEmpty:  "JSON config empty",
-   -ErrorDeviceExists: "Device exists",
-   -ErrorCheckpointDevopsDoesNotMatch: "Checkpoint devops does not match",
-   -ErrorCheckpointDeviceNotSupported: "Checkpoint device not supported",
-   -ErrorVnumaConfigInvalid:   "VNUMA config invalid",
-   -ErrorDomainNotfound:   "Domain not found",
-   -ErrorAborted:  "Aborted",
-   -ErrorNotfound: "Not found",
-   -ErrorDomainDestroyed:  "Domain destroyed",
-   -ErrorFeatureRemoved:   "Feature removed",
+var libxlErrors = map[Error]string{
+   ErrorNonspecific:  "Non-specific error",
+   ErrorVersion:  "Wrong version",
+   ErrorFail: "Failed",
+   ErrorNi:   "Not Implemented",
+   ErrorNomem:"No memory",
+   ErrorInval:"Invalid argument",
+   ErrorBadfail:  "Bad Fail",
+   ErrorGuestTimedout:"Guest timed out",
+   ErrorTimedout: "Timed out",
+   ErrorNoparavirt:   "No Paravirtualization",
+   ErrorNotReady: "Not ready",
+   ErrorOseventRegFail:   "OS event registration failed",
+   ErrorBufferfull:   "Buffer full",
+   ErrorUnknownChild: "Unknown child",
+   ErrorLockFail: "Lock failed",
+   ErrorJsonConfigEmpty:  "JSON config empty",
+   ErrorDeviceExists: "Device exists",
+   ErrorCheckpointDevopsDoesNotMatch: "Checkpoint devops does not match",
+   ErrorCheckpointDeviceNotSupported: "Checkpoint device not supported",
+   ErrorVnumaConfigInvalid:   "VNUMA config invalid",
+   ErrorDomainNotfound:   "Domain not found",
+   ErrorAborted:  "Aborted",
+   ErrorNotfound: "Not found",
+   ErrorDomainDestroyed:  "Domain destroyed",
+   ErrorFeatureRemoved:   "Feature removed",
 }
 
 func (e Error) Error() string {
-   if 0 < int(e) && int(e) < len(libxlErrors) {
-   s := libxlErrors[e]
-   if s != "" {
-   return s
-   }
+   if s, ok := libxlErrors[e]; ok {
+   return s
}
-   return fmt.Sprintf("libxl error: %d", -e)
+
+   return fmt.Sprintf("libxl error: %d", e)
 }
 
 // Context represents a libxl_ctx.
-- 
2.24.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 2/8] go/xenlight: Fix CpuidPoliclyList conversion

[George Dunlap]

Empty Go strings should be converted to `nil` libxl_cpuid_policy_list;
otherwise libxl_cpuid_parse_config gets confused.

Also, libxl_cpuid_policy_list returns a weird error, not a "normal"
libxl error; if it returns one of these non-standard errors, convert
it to ErrorInval.

Finally, make the fromC() method take a pointer, and set the value of
CpuidPolicyList such that it will generate a valid CpuidPolicyList in
response.

Signed-off-by: George Dunlap 
---
v2:
- Port over toC() function signature change
- Have fromC set the string to ""

CC: Nick Rosbrook 
---
 tools/golang/xenlight/xenlight.go | 10 --
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/tools/golang/xenlight/xenlight.go 
b/tools/golang/xenlight/xenlight.go
index b1587b964f..1299981713 100644
--- a/tools/golang/xenlight/xenlight.go
+++ b/tools/golang/xenlight/xenlight.go
@@ -306,9 +306,14 @@ func (el *EvLink) toC(cel *C.libxl_ev_link) (err error) { 
return }
 // empty when it is returned from libxl.
 type CpuidPolicyList string
 
-func (cpl CpuidPolicyList) fromC(ccpl *C.libxl_cpuid_policy_list) error { 
return nil }
+func (cpl *CpuidPolicyList) fromC(ccpl *C.libxl_cpuid_policy_list) error { 
*cpl = ""; return nil }
 
 func (cpl CpuidPolicyList) toC(ccpl *C.libxl_cpuid_policy_list) error {
+   if cpl == "" {
+   *ccpl = nil
+   return nil
+   }
+
s := C.CString(string(cpl))
defer C.free(unsafe.Pointer(s))
 
@@ -316,7 +321,8 @@ func (cpl CpuidPolicyList) toC(ccpl 
*C.libxl_cpuid_policy_list) error {
if ret != 0 {
C.libxl_cpuid_dispose(ccpl)
 
-   return Error(-ret)
+   // libxl_cpuid_parse_config doesn't return a normal libxl error.
+   return ErrorInval
}
 
return nil
-- 
2.24.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 1/8] golang/xenlight: Don't try to marshall zero-length arrays in fromC

[George Dunlap]

The current fromC array code will do the "magic" casting and
martialling even when num_foo variable is 0.  Go crashes when doing
the cast.

Only do array marshalling if the number of elements is non-zero;
otherwise, leave the target pointer empty (nil for Go slices, NULL for
C arrays).

Signed-off-by: George Dunlap 
---
v2:
- Remove toC part of this, which has been folded into Nick's patch
  series.

CC: Nick Rosbrook 
---
 tools/golang/xenlight/gengotypes.py  |   5 +-
 tools/golang/xenlight/helpers.gen.go | 452 +++
 2 files changed, 261 insertions(+), 196 deletions(-)

diff --git a/tools/golang/xenlight/gengotypes.py 
b/tools/golang/xenlight/gengotypes.py
index 27edf66241..ad2c573da9 100644
--- a/tools/golang/xenlight/gengotypes.py
+++ b/tools/golang/xenlight/gengotypes.py
@@ -419,7 +419,8 @@ def xenlight_golang_array_from_C(ty = None):
 clenvar= ty.type.lenvar.name
 golenvar   = xenlight_golang_fmt_name(clenvar,exported=False)
 
-s += '{} := int(xc.{})\n'.format(golenvar, clenvar)
+s += 'x.{} = nil\n'.format(goname)
+s += 'if {} := int(xc.{}); {} > 0 {{\n'.format(golenvar, clenvar, golenvar)
 s += '{} := '.format(cslice)
 s +='(*[1<<28]C.{})(unsafe.Pointer(xc.{}))[:{}:{}]\n'.format(ctypename, 
cname,
 golenvar, 
golenvar)
@@ -433,7 +434,7 @@ def xenlight_golang_array_from_C(ty = None):
 s += 'if err := x.{}[i].fromC(); err != nil {{\n'.format(goname)
 s += 'return err }\n'
 
-s += '}\n'
+s += '}\n}\n'
 
 return s
 
diff --git a/tools/golang/xenlight/helpers.gen.go 
b/tools/golang/xenlight/helpers.gen.go
index b9a7e828a0..889807d928 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -623,12 +623,14 @@ func (x *SchedParams) toC(xc *C.libxl_sched_params) (err 
error) {
 
 func (x *VcpuSchedParams) fromC(xc *C.libxl_vcpu_sched_params) error {
x.Sched = Scheduler(xc.sched)
-   numVcpus := int(xc.num_vcpus)
-   cVcpus := (*[1 << 
28]C.libxl_sched_params)(unsafe.Pointer(xc.vcpus))[:numVcpus:numVcpus]
-   x.Vcpus = make([]SchedParams, numVcpus)
-   for i, v := range cVcpus {
-   if err := x.Vcpus[i].fromC(); err != nil {
-   return err
+   x.Vcpus = nil
+   if numVcpus := int(xc.num_vcpus); numVcpus > 0 {
+   cVcpus := (*[1 << 
28]C.libxl_sched_params)(unsafe.Pointer(xc.vcpus))[:numVcpus:numVcpus]
+   x.Vcpus = make([]SchedParams, numVcpus)
+   for i, v := range cVcpus {
+   if err := x.Vcpus[i].fromC(); err != nil {
+   return err
+   }
}
}
 
@@ -691,11 +693,13 @@ func (x *DomainSchedParams) toC(xc 
*C.libxl_domain_sched_params) (err error) {
 
 func (x *VnodeInfo) fromC(xc *C.libxl_vnode_info) error {
x.Memkb = uint64(xc.memkb)
-   numDistances := int(xc.num_distances)
-   cDistances := (*[1 << 
28]C.uint32_t)(unsafe.Pointer(xc.distances))[:numDistances:numDistances]
-   x.Distances = make([]uint32, numDistances)
-   for i, v := range cDistances {
-   x.Distances[i] = uint32(v)
+   x.Distances = nil
+   if numDistances := int(xc.num_distances); numDistances > 0 {
+   cDistances := (*[1 << 
28]C.uint32_t)(unsafe.Pointer(xc.distances))[:numDistances:numDistances]
+   x.Distances = make([]uint32, numDistances)
+   for i, v := range cDistances {
+   x.Distances[i] = uint32(v)
+   }
}
x.Pnode = uint32(xc.pnode)
if err := x.Vcpus.fromC(); err != nil {
@@ -760,20 +764,24 @@ func (x *DomainBuildInfo) fromC(xc 
*C.libxl_domain_build_info) error {
if err := x.Nodemap.fromC(); err != nil {
return err
}
-   numVcpuHardAffinity := int(xc.num_vcpu_hard_affinity)
-   cVcpuHardAffinity := (*[1 << 
28]C.libxl_bitmap)(unsafe.Pointer(xc.vcpu_hard_affinity))[:numVcpuHardAffinity:numVcpuHardAffinity]
-   x.VcpuHardAffinity = make([]Bitmap, numVcpuHardAffinity)
-   for i, v := range cVcpuHardAffinity {
-   if err := x.VcpuHardAffinity[i].fromC(); err != nil {
-   return err
+   x.VcpuHardAffinity = nil
+   if numVcpuHardAffinity := int(xc.num_vcpu_hard_affinity); 
numVcpuHardAffinity > 0 {
+   cVcpuHardAffinity := (*[1 << 
28]C.libxl_bitmap)(unsafe.Pointer(xc.vcpu_hard_affinity))[:numVcpuHardAffinity:numVcpuHardAffinity]
+   x.VcpuHardAffinity = make([]Bitmap, numVcpuHardAffinity)
+   for i, v := range cVcpuHardAffinity {
+   if err := x.VcpuHardAffinity[i].fromC(); err != nil {
+   return err
+   }
}
}
-   numVcpuSoftAffinity := int(xc.num_vcpu_soft_affinity)
-   cVcpuSoftAffinity := (*[1 << 

[Xen-devel] [PATCH v3 5/8] golang/xenlight: Default loglevel to DEBUG until we get everything working

[George Dunlap]

Signed-off-by: George Dunlap 
---

The other option would be to expose the XTL logging levels and let the
caller set them somehow.

CC: Nick Rosbrook 
---
 tools/golang/xenlight/xenlight.go | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/golang/xenlight/xenlight.go 
b/tools/golang/xenlight/xenlight.go
index aa1e63a61a..0e71f6ca7d 100644
--- a/tools/golang/xenlight/xenlight.go
+++ b/tools/golang/xenlight/xenlight.go
@@ -82,7 +82,7 @@ type Context struct {
 func NewContext() (*Context, error) {
var ctx Context
 
-   ctx.logger = C.xtl_createlogger_stdiostream(C.stderr, C.XTL_ERROR, 0)
+   ctx.logger = C.xtl_createlogger_stdiostream(C.stderr, C.XTL_DEBUG, 0)
 
ret := C.libxl_ctx_alloc(, C.LIBXL_VERSION, 0,
(*C.xentoollog_logger)(unsafe.Pointer(ctx.logger)))
-- 
2.24.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 6/8] golang/xenlight: Don't leak memory on context open failure

[George Dunlap]

If libxl_ctx_alloc() returns an error, we need to destroy the logger
that we made.

Restructure the Close() method such that it checks for each resource
to be freed and then frees it.  This allows Close() to be come
idempotent, as well as to be a useful clean-up to a partially-created
context.

Signed-off-by: George Dunlap 
---
CC: Nick Rosbrook 
---
 tools/golang/xenlight/xenlight.go | 30 +-
 1 file changed, 21 insertions(+), 9 deletions(-)

diff --git a/tools/golang/xenlight/xenlight.go 
b/tools/golang/xenlight/xenlight.go
index 0e71f6ca7d..662b266250 100644
--- a/tools/golang/xenlight/xenlight.go
+++ b/tools/golang/xenlight/xenlight.go
@@ -79,28 +79,40 @@ type Context struct {
 }
 
 // NewContext returns a new Context.
-func NewContext() (*Context, error) {
-   var ctx Context
+func NewContext() (ctx *Context, err error) {
+   ctx = {}
+
+   defer func() {
+   if err != nil {
+   ctx.Close()
+   ctx = nil
+   }
+   }()
 
ctx.logger = C.xtl_createlogger_stdiostream(C.stderr, C.XTL_DEBUG, 0)
 
ret := C.libxl_ctx_alloc(, C.LIBXL_VERSION, 0,
(*C.xentoollog_logger)(unsafe.Pointer(ctx.logger)))
if ret != 0 {
-   return nil, Error(ret)
+   return ctx, Error(ret)
}
 
-   return , nil
+   return ctx, nil
 }
 
 // Close closes the Context.
 func (ctx *Context) Close() error {
-   ret := C.libxl_ctx_free(ctx.ctx)
-   ctx.ctx = nil
-   C.xtl_logger_destroy((*C.xentoollog_logger)(unsafe.Pointer(ctx.logger)))
+   if ctx.ctx != nil {
+   ret := C.libxl_ctx_free(ctx.ctx)
+   if ret != 0 {
+   return Error(ret)
+   }
+   ctx.ctx = nil
+   }
 
-   if ret != 0 {
-   return Error(ret)
+   if ctx.logger != nil {
+   
C.xtl_logger_destroy((*C.xentoollog_logger)(unsafe.Pointer(ctx.logger)))
+   ctx.logger = nil
}
 
return nil
-- 
2.24.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] x86/apic: fix disabling LVT0 in disconnect_bsp_APIC

[Jan Beulich]

On 17.01.2020 16:09, Roger Pau Monne wrote:
> The Intel SDM states:
> 
> "When an illegal vector value (0 to 15) is written to a LVT entry and
> the delivery mode is Fixed (bits 8-11 equal 0), the APIC may signal an
> illegal vector error, without regard to whether the mask bit is set or
> whether an interrupt is actually seen on the input."
> 
> And that's exactly what's currently done in disconnect_bsp_APIC when
> virt_wire_setup is true and LVT LINT0 is being masked. By writing only
> APIC_LVT_MASKED Xen is actually setting the vector to 0 and the
> delivery mode to Fixed (0), and hence it triggers an APIC error even
> when the LVT entry is masked.

But there are many more instances where we (have a risk to) do so,
most notably in clear_local_APIC(). The two step logic there is
anyway somewhat in conflict with the citation above.

Jan

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH] build: fix dependency file generation with ENFORCE_UNIQUE_SYMBOLS=y

[Jan Beulich]

The recorded file, unless overridden by -MQ (or -MT) is that specified
by -o, which doesn't produce correct dependencies and hence will cause
failure to re-build when included files change.

Fixes: 81ecb38b83b0 ("build: provide option to disambiguate symbol names")
Reported-by: Andrew Cooper 
Signed-off-by: Jan Beulich 

--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -195,7 +195,7 @@ SRCPATH := $(patsubst $(BASEDIR)/%,%,$(C
 
 %.o: %.c Makefile
 ifeq ($(CONFIG_ENFORCE_UNIQUE_SYMBOLS),y)
-   $(CC) $(CFLAGS) -c $< -o $(@D)/.$(@F).tmp
+   $(CC) $(CFLAGS) -c $< -o $(@D)/.$(@F).tmp -MQ $@
 ifeq ($(clang),y)
$(OBJCOPY) --redefine-sym $<=$(SRCPATH)/$< $(@D)/.$(@F).tmp $@
 else

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 4/6] libxl: allow creation of domains with a specified or random domid

[Ian Jackson]

Durrant, Paul writes ("RE: [PATCH v3 4/6] libxl: allow creation of domains with 
a specified or random domid"):
> Ok, to cover all bases then it seems like checking the domid after creation 
> and then destroying if it is too recent is the better option.

I think so, yes.  I think the recent timestamp should be updated in
this case.  (Faff!)

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] x86/apic: fix disabling LVT0 in disconnect_bsp_APIC

[Andrew Cooper]

On 17/01/2020 15:09, Roger Pau Monne wrote:
> The Intel SDM states:
>
> "When an illegal vector value (0 to 15) is written to a LVT entry and
> the delivery mode is Fixed (bits 8-11 equal 0), the APIC may signal an
> illegal vector error, without regard to whether the mask bit is set or
> whether an interrupt is actually seen on the input."
>
> And that's exactly what's currently done in disconnect_bsp_APIC when
> virt_wire_setup is true and LVT LINT0 is being masked. By writing only
> APIC_LVT_MASKED Xen is actually setting the vector to 0 and the
> delivery mode to Fixed (0), and hence it triggers an APIC error even
> when the LVT entry is masked.
>
> This would usually manifest when Xen is being shut down, as that's
> where disconnect_bsp_APIC is called:
>
> (XEN) APIC error on CPU0: 40(00)
>
> Fix this by reusing the current LVT LINT0 value and just adding the
> mask bit to it.
>
> Reported-by: Andrew Cooper 
> Signed-off-by: Roger Pau Monné 
> ---
>  xen/arch/x86/apic.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/xen/arch/x86/apic.c b/xen/arch/x86/apic.c
> index a6a7754d77..e4363639bd 100644
> --- a/xen/arch/x86/apic.c
> +++ b/xen/arch/x86/apic.c
> @@ -281,7 +281,8 @@ void disconnect_bsp_APIC(int virt_wire_setup)
>  }
>  else {
>  /* Disable LVT0 */
> -apic_write(APIC_LVT0, APIC_LVT_MASKED);
> +value = apic_read(APIC_LVT0);
> +apic_write(APIC_LVT0, value | APIC_LVT_MASKED);
>  }

This really is ugly.  It seems that we can't write LVT0 to the same
state that it has after reset/INIT.

For the code however, both halves of the if() condition do a
read/modify/write.  It would be nicer to have the read and write common,
with modify alone having the if().

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] EFI development issues

[Jan Beulich]

On 13.01.2020 17:02, Andrew Cooper wrote:
> First, there is a dependency tracking bug in the build system.  Edits to
> xen/arch/x86/efi/efi-boot.h don't cause xen.efi to be regenerated.  From
> what I can tell, the file doesn't even get recompiled, because syntax
> errors even go unnoticed.

I've just now seen this too, and also see why it is: My enforce-unique-
symbols change causes .*.o.d files to start like this

.apic.o.tmp: apic.c \
 /build/xen/unstable-git/2019-12-20-livepatch/xen/include/generated/config.h \
 ...

when the option is enabled. I'll need to figure how to tell the compiler
to emit the proper apic.o instead of the intermediate file. Iirc there's
a command line option ...

Jan

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] xen/blkfront: limit allocated memory size to actual use case

[Roger Pau Monné]

On Fri, Jan 17, 2020 at 03:39:55PM +0100, Juergen Gross wrote:
> Today the Xen blkfront driver allocates memory for one struct
> blkfront_ring_info for each communication ring. This structure is
> statically sized for the maximum supported configuration resulting
> in a size of more than 90 kB.
> 
> As the main size contributor is one array inside the struct, the
> memory allocation can easily be limited by moving this array to be
> the last structure element and to allocate only the memory for the
> actually needed array size.
> 
> Signed-off-by: Juergen Gross 

Acked-by: Roger Pau Monné 

Thanks! It would be nice to backport this, but I'm not sure it would
qualify as a bug fix.

Roger.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH] x86/apic: fix disabling LVT0 in disconnect_bsp_APIC

[Roger Pau Monne]

The Intel SDM states:

"When an illegal vector value (0 to 15) is written to a LVT entry and
the delivery mode is Fixed (bits 8-11 equal 0), the APIC may signal an
illegal vector error, without regard to whether the mask bit is set or
whether an interrupt is actually seen on the input."

And that's exactly what's currently done in disconnect_bsp_APIC when
virt_wire_setup is true and LVT LINT0 is being masked. By writing only
APIC_LVT_MASKED Xen is actually setting the vector to 0 and the
delivery mode to Fixed (0), and hence it triggers an APIC error even
when the LVT entry is masked.

This would usually manifest when Xen is being shut down, as that's
where disconnect_bsp_APIC is called:

(XEN) APIC error on CPU0: 40(00)

Fix this by reusing the current LVT LINT0 value and just adding the
mask bit to it.

Reported-by: Andrew Cooper 
Signed-off-by: Roger Pau Monné 
---
 xen/arch/x86/apic.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/apic.c b/xen/arch/x86/apic.c
index a6a7754d77..e4363639bd 100644
--- a/xen/arch/x86/apic.c
+++ b/xen/arch/x86/apic.c
@@ -281,7 +281,8 @@ void disconnect_bsp_APIC(int virt_wire_setup)
 }
 else {
 /* Disable LVT0 */
-apic_write(APIC_LVT0, APIC_LVT_MASKED);
+value = apic_read(APIC_LVT0);
+apic_write(APIC_LVT0, value | APIC_LVT_MASKED);
 }
 
 /* For LVT1 make it edge triggered, active high, nmi and enabled */
-- 
2.25.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 2/4] x86/xen: add basic KASAN support for PV kernel

[Boris Ostrovsky]

On 1/17/20 7:58 AM, Sergey Dyasli wrote:

--- a/arch/x86/mm/kasan_init_64.c
+++ b/arch/x86/mm/kasan_init_64.c
@@ -13,6 +13,9 @@
  #include 
  #include 
  
+#include 

+#include 
+
  #include 
  #include 
  #include 
@@ -332,6 +335,11 @@ void __init kasan_early_init(void)
for (i = 0; pgtable_l5_enabled() && i < PTRS_PER_P4D; i++)
kasan_early_shadow_p4d[i] = __p4d(p4d_val);
  
+	if (xen_pv_domain()) {

+   pgd_t *pv_top_pgt = xen_pv_kasan_early_init();
+   kasan_map_early_shadow(pv_top_pgt);
+   }
+



I'd suggest replacing this with xen_kasan_early_init() and doing 
everything, including PV check, there. This way non-Xen code won't need 
to be aware of Xen-specific details such as guest types.




kasan_map_early_shadow(early_top_pgt);
kasan_map_early_shadow(init_top_pgt);
  }
@@ -369,6 +377,8 @@ void __init kasan_init(void)
__pgd(__pa(tmp_p4d_table) | _KERNPG_TABLE));
}
  
+	xen_pv_kasan_pin_pgd(early_top_pgt);

+


And drop "_pv" here (and below) for the same reason.

-boris


load_cr3(early_top_pgt);
__flush_tlb_all();
  
@@ -433,6 +443,8 @@ void __init kasan_init(void)

load_cr3(init_top_pgt);
__flush_tlb_all();
  
+	xen_pv_kasan_unpin_pgd(early_top_pgt);

+



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v6 03/11] scripts: add coccinelle script to use auto propagated errp

[Vladimir Sementsov-Ogievskiy]

10.01.2020 22:41, Vladimir Sementsov-Ogievskiy wrote:
> Signed-off-by: Vladimir Sementsov-Ogievskiy 
> ---
> 
> CC: Cornelia Huck 
> CC: Eric Blake 
> CC: Kevin Wolf 
> CC: Max Reitz 
> CC: Greg Kurz 
> CC: Stefan Hajnoczi 
> CC: Stefano Stabellini 
> CC: Anthony Perard 
> CC: Paul Durrant 
> CC: "Philippe Mathieu-Daudé" 
> CC: Laszlo Ersek 
> CC: Gerd Hoffmann 
> CC: Stefan Berger 
> CC: Markus Armbruster 
> CC: Michael Roth 
> CC: qemu-bl...@nongnu.org
> CC: xen-devel@lists.xenproject.org
> 
>   include/qapi/error.h  |   3 +
>   scripts/coccinelle/auto-propagated-errp.cocci | 139 ++
>   2 files changed, 142 insertions(+)
>   create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
> 
> diff --git a/include/qapi/error.h b/include/qapi/error.h
> index 532b9afb9e..dcfb77e107 100644
> --- a/include/qapi/error.h
> +++ b/include/qapi/error.h
> @@ -141,6 +141,9 @@
>* ...
>* }
>*
> + * For mass conversion use script
> + *   scripts/coccinelle/auto-propagated-errp.cocci
> + *
>*
>* Receive and accumulate multiple errors (first one wins):
>* Error *err = NULL, *local_err = NULL;
> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci 
> b/scripts/coccinelle/auto-propagated-errp.cocci
> new file mode 100644
> index 00..6c72a5049f
> --- /dev/null
> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
> @@ -0,0 +1,139 @@
> +// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
> +//
> +// Copyright (c) 2020 Virtuozzo International GmbH.
> +//
> +// This program is free software; you can redistribute it and/or modify
> +// it under the terms of the GNU General Public License as published by
> +// the Free Software Foundation; either version 2 of the License, or
> +// (at your option) any later version.
> +//
> +// This program is distributed in the hope that it will be useful,
> +// but WITHOUT ANY WARRANTY; without even the implied warranty of
> +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +// GNU General Public License for more details.
> +//
> +// You should have received a copy of the GNU General Public License
> +// along with this program.  If not, see .
> +//
> +// Usage example:
> +// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
> +//  --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
> +//  blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
> +
> +@@
> +// Add invocation to errp-functions where necessary
> +// We should skip functions with "Error *const *errp"
> +// parameter, but how to do it with coccinelle?
> +// I don't know, so, I skip them by function name regex.
> +// It's safe: if we not skip some functions with
> +// "Error *const *errp", ERRP_AUTO_PROPAGATE invocation
> +// will fail to compile, because of const violation.
> +identifier fn !~ "error_append_.*_hint";
> +identifier local_err, errp;

Hmm.

Note, that in new version I define errp as "identifier", which means,
that we'll match Error ** parameters with other names..

Still, our ERRP_AUTO_PROPAGATE assumes that parameter called errp, and
I'd prefere not to change it.

We can ignore this fact for now: inappropriately named errp parameter will
break compilation in ERRP_AUTO_PROPAGATE() invocation, so it's safe enough.
(Hope, there are no functions with two Error** parameters)

Or we can revert errp to be symbol again.

> +@@
> +
> + fn(..., Error **errp, ...)
> + {
> ++   ERRP_AUTO_PROPAGATE();
> +<+...
> +when != ERRP_AUTO_PROPAGATE();
> +(
> +error_append_hint(errp, ...);
> +|
> +error_prepend(errp, ...);
> +|
> +Error *local_err = NULL;
> +)
> +...+>
> + }
> +
> +@rule1@
> +// We do not inherit from previous rule, as we want to match
> +// also functions, which already had ERRP_AUTO_PROPAGATE
> +// invocation.
> +identifier fn !~ "error_append_.*_hint";
> +identifier local_err, errp;
> +@@
> +
> + fn(..., Error **errp, ...)
> + {
> + <...
> +-Error *local_err = NULL;
> + ...>
> + }
> +
> +@@
> +// Handle pattern with goto, otherwise we'll finish up
> +// with labels at function end which will not compile.
> +identifier rule1.fn, rule1.local_err, rule1.errp;
> +identifier OUT;
> +@@
> +
> + fn(...)
> + {
> + <...
> +-goto OUT;
> ++return;
> + ...>
> +- OUT:
> +-error_propagate(errp, local_err);
> + }
> +
> +@@
> +identifier rule1.fn, rule1.local_err, rule1.errp;
> +expression list args; // to reindent error_propagate_prepend
> +@@
> +
> + fn(...)
> + {
> + <...
> +(
> +-error_free(local_err);
> +-local_err = NULL;
> ++error_free_errp(errp);
> +|
> +-error_free(local_err);
> ++error_free_errp(errp);
> +|
> +-error_report_err(local_err);
> ++error_report_errp(errp);
> +|
> +-warn_report_err(local_err);
> ++warn_report_errp(errp);
> +|
> +-error_propagate_prepend(errp, local_err, args);
> ++error_prepend(errp, args);
> +|
> +-

[Xen-devel] [PATCH v3 04/10] libxl: event: Make LIBXL__EVENT_DISASTER take a gc, not an egc

[Ian Jackson]

We are going to want to change libxl__poller_wakeup to take a gc.

In theory there is a risk here that it would be called inappropriately
in a future patch but this seems unlikely.

Signed-off-by: Ian Jackson 
Tested-by: George Dunlap 
Reviewed-by: George Dunlap 
---
v2: New patch
---
 tools/libxl/libxl_aoutils.c  |  2 +-
 tools/libxl/libxl_disk.c |  4 ++--
 tools/libxl/libxl_domain.c   |  2 +-
 tools/libxl/libxl_event.c| 21 ++---
 tools/libxl/libxl_fork.c | 11 ++-
 tools/libxl/libxl_internal.h | 10 +-
 6 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/tools/libxl/libxl_aoutils.c b/tools/libxl/libxl_aoutils.c
index e24e4eed53..1be858c93c 100644
--- a/tools/libxl/libxl_aoutils.c
+++ b/tools/libxl/libxl_aoutils.c
@@ -282,7 +282,7 @@ static void datacopier_readable(libxl__egc *egc, 
libxl__ev_fd *ev,
 hupchk.revents = 0;
 r = poll(, 1, 0);
 if (r < 0)
-LIBXL__EVENT_DISASTER(egc,
+LIBXL__EVENT_DISASTER(gc,
  "unexpected failure polling fd for datacopier eof hup check",
   errno, 0);
 if (datacopier_pollhup_handled(egc, dc, fd, hupchk.revents, 0))
diff --git a/tools/libxl/libxl_disk.c b/tools/libxl/libxl_disk.c
index 64a6691424..a463334130 100644
--- a/tools/libxl/libxl_disk.c
+++ b/tools/libxl/libxl_disk.c
@@ -33,7 +33,7 @@ static void disk_eject_xswatch_callback(libxl__egc *egc, 
libxl__ev_xswatch *w,
 return;
 
 if (libxl__xs_printf(gc, XBT_NULL, wpath, "")) {
-LIBXL__EVENT_DISASTER(egc, "xs_write failed acknowledging eject",
+LIBXL__EVENT_DISASTER(gc, "xs_write failed acknowledging eject",
   errno, LIBXL_EVENT_TYPE_DISK_EJECT);
 return;
 }
@@ -43,7 +43,7 @@ static void disk_eject_xswatch_callback(libxl__egc *egc, 
libxl__ev_xswatch *w,
 
 rc = libxl__xs_read_checked(gc, XBT_NULL, evg->be_ptr_path, );
 if (rc) {
-LIBXL__EVENT_DISASTER(egc, "xs_read failed reading be_ptr_path",
+LIBXL__EVENT_DISASTER(gc, "xs_read failed reading be_ptr_path",
   errno, LIBXL_EVENT_TYPE_DISK_EJECT);
 return;
 }
diff --git a/tools/libxl/libxl_domain.c b/tools/libxl/libxl_domain.c
index 5714501778..b59cc65750 100644
--- a/tools/libxl/libxl_domain.c
+++ b/tools/libxl/libxl_domain.c
@@ -892,7 +892,7 @@ static void domain_death_xswatch_callback(libxl__egc *egc, 
libxl__ev_xswatch *w,
 
 rc = xc_domain_getinfolist(CTX->xch, evg->domid, nentries, 
domaininfos);
 if (rc == -1) {
-LIBXL__EVENT_DISASTER(egc, "xc_domain_getinfolist failed while"
+LIBXL__EVENT_DISASTER(gc, "xc_domain_getinfolist failed while"
   " processing @releaseDomain watch event",
   errno, 0);
 goto out;
diff --git a/tools/libxl/libxl_event.c b/tools/libxl/libxl_event.c
index be37e12bb0..16e6786889 100644
--- a/tools/libxl/libxl_event.c
+++ b/tools/libxl/libxl_event.c
@@ -261,7 +261,7 @@ short libxl__fd_poll_recheck(libxl__egc *egc, int fd, short 
events) {
 break;
 assert(r<0);
 if (errno != EINTR) {
-LIBXL__EVENT_DISASTER(egc, "failed poll to check for fd", errno, 
0);
+LIBXL__EVENT_DISASTER(gc, "failed poll to check for fd", errno, 0);
 return 0;
 }
 }
@@ -509,14 +509,14 @@ static void watchfd_callback(libxl__egc *egc, 
libxl__ev_fd *ev,
 EGC_GC;
 
 if (revents & (POLLERR|POLLHUP))
-LIBXL__EVENT_DISASTER(egc, "unexpected poll event on watch fd", 0, 0);
+LIBXL__EVENT_DISASTER(gc, "unexpected poll event on watch fd", 0, 0);
 
 for (;;) {
 char **event = xs_check_watch(CTX->xsh);
 if (!event) {
 if (errno == EAGAIN) break;
 if (errno == EINTR) continue;
-LIBXL__EVENT_DISASTER(egc, "cannot check/read watches", errno, 0);
+LIBXL__EVENT_DISASTER(gc, "cannot check/read watches", errno, 0);
 return;
 }
 
@@ -705,7 +705,7 @@ static int evtchn_revents_check(libxl__egc *egc, int 
revents)
 
 if (revents & ~POLLIN) {
 LOG(ERROR, "unexpected poll event on event channel fd: %x", revents);
-LIBXL__EVENT_DISASTER(egc,
+LIBXL__EVENT_DISASTER(gc,
"unexpected poll event on event channel fd", 0, 0);
 libxl__ev_fd_deregister(gc, >evtchn_efd);
 return ERROR_FAIL;
@@ -746,7 +746,7 @@ static void evtchn_fd_callback(libxl__egc *egc, 
libxl__ev_fd *ev,
 if (port < 0) {
 if (errno == EAGAIN)
 break;
-LIBXL__EVENT_DISASTER(egc,
+LIBXL__EVENT_DISASTER(gc,
  "unexpected failure fetching occurring event port number from evtchn",
   errno, 0);
 return;
@@ -966,7 +966,7 @@ static void 

[Xen-devel] [PATCH v3 08/10] libxl: event: Break out baton_wake

[Ian Jackson]

No functional change.

Signed-off-by: Ian Jackson 
Reviewed-by: George Dunlap 
Tested-by: George Dunlap 
---
v2: Now it takes a gc, not an egc.
---
 tools/libxl/libxl_event.c | 21 +
 1 file changed, 13 insertions(+), 8 deletions(-)

diff --git a/tools/libxl/libxl_event.c b/tools/libxl/libxl_event.c
index 3e76fa5af5..45cc67942d 100644
--- a/tools/libxl/libxl_event.c
+++ b/tools/libxl/libxl_event.c
@@ -140,6 +140,18 @@ static void pollers_note_osevent_added(libxl_ctx *ctx) {
 poller->osevents_added = 1;
 }
 
+static void baton_wake(libxl__gc *gc, libxl__poller *wake)
+{
+libxl__poller_wakeup(gc, wake);
+
+wake->osevents_added = 0;
+/* This serves to make _1_baton idempotent.  It is OK even though
+ * that poller may currently be sleeping on only old osevents,
+ * because it is going to wake up because we've just prodded it,
+ * and it pick up new osevents on its next iteration (or pass
+ * on the baton). */
+}
+
 void libxl__egc_ao_cleanup_1_baton(libxl__gc *gc)
 /* Any poller we had must have been `put' already. */
 {
@@ -160,14 +172,7 @@ void libxl__egc_ao_cleanup_1_baton(libxl__gc *gc)
 /* no-one in libxl waiting for any events */
 return;
 
-libxl__poller_wakeup(gc, wake);
-
-wake->osevents_added = 0;
-/* This serves to make _1_baton idempotent.  It is OK even though
- * that poller may currently be sleeping on only old osevents,
- * because it is going to wake up because we've just prodded it,
- * and it pick up new osevents on its next iteration (or pass
- * on the baton). */
+baton_wake(gc, wake);
 }
 
 /*
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 06/10] libxl: event: Fix hang when mixing blocking and eventy calls

[Ian Jackson]

If the application calls libxl with ao_how==0 and also makes calls
like _occurred, libxl will sometimes get stuck.

The bug happens as follows (for example):

  Thread A
   libxl_do_thing(,ao_how==0)
   libxl_do_thing starts, sets up some callbacks
   libxl_do_thing exit path calls AO_INPROGRESS
   libxl__ao_inprogress goes into event loop
   eventloop_iteration sleeps on:
  - do_thing's current fd set
  - sigchld pipe if applicable
  - its poller

  Thread B
   libxl_something_occurred
   the something is to do with do_thing, above
   do_thing_next_callback does some more work
   do_thing_next_callback becomes interested in fd N
   thread B returns to application

Note that nothing wakes up thread A.  A is not listening on fd N.  So
do_thing_* will not spot when fd N signals.  do_thing will not make
further timely progress.  If there is no timeout thread A will never
wake up.

The problem here occurs because thread A is waiting on an out of date
osevent set.

There is also the possibility that a thread might block waiting for
libxl osevents but outside libxl, eg if the application used
libxl_osevent_beforepoll.  We will deal with that in a moment.

See the big comment in libxl_event.c for a fairly formal correctness
argument.

This depends on libxl__egc_ao_cleanup_1_baton being called everywhere
an egc or ao is disposed of.  Firstly egcs: in this patch we rename
libxl__egc_cleanup, which means we catch all the disposal sites.
Secondly aos: these are disposed of by (i) AO_CREATE_FAIL
(ii) ao__inprogress and (iii) an event which completes the ao later.
(i) and (ii) we handle by adding the call to _baton.  In the case of
(iii) any such function must be an event-generating function so it has
an egc too, so it will pass on the baton when the egc is disposed.

Reported-by: George Dunlap 
Signed-off-by: Ian Jackson 
Reviewed-by: George Dunlap 
Tested-by: George Dunlap 
---
v2: Call libxl__egc_ao_cleanup_1_baton (renamed from __egc_cleanup) on
all exits from ao_inprogress, even requests for async processing.
Fixes a remaining instance of this bug (!)
This involves disposing of ao->poller somewhat earlier.

v2: New correctness arguments in libxl_event.c comment and
in commit message.
---
 tools/libxl/libxl_event.c| 178 ---
 tools/libxl/libxl_internal.h |  33 ++--
 2 files changed, 194 insertions(+), 17 deletions(-)

diff --git a/tools/libxl/libxl_event.c b/tools/libxl/libxl_event.c
index 268a5da120..b50d4e5074 100644
--- a/tools/libxl/libxl_event.c
+++ b/tools/libxl/libxl_event.c
@@ -37,6 +37,140 @@ static void ao__check_destroy(libxl_ctx *ctx, libxl__ao 
*ao);
 
 
 /*
+ * osevent update baton handling
+ *
+ * We need the following property (the "unstale liveness property"):
+ *
+ * Whenever any thread is blocking in the libxl event loop[1], at
+ * least one thread must be using an up to date osevent set.  It is OK
+ * for all but one threads to have stale event sets, because so long
+ * as one waiting thread has the right event set, any actually
+ * interesting event will, if nothing else, wake that "right" thread
+ * up.  It will then make some progress and/or, if it exits, ensure
+ * that some other thread becomes the "right" thread.
+ *
+ * [1] TODO: Right now we are considering only the libxl event loop.
+ * We need to consider application event loop outside libxl too.
+ *
+ * Argument that our approach is sound:
+ *
+ * The issue we are concerned about is libxl sleeping on an out of
+ * date fd set, or too long a timeout, so that it doesn't make
+ * progress.  If the property above is satisfied, then if any thread
+ * is waiting in libxl at least one such thread will be waiting on a
+ * sufficient osevent set, so any relevant osevent will wake up a
+ * libxl thread which will either handle the event, or arrange that at
+ * least one other libxl thread has the right set.
+ *
+ * There are two calls to poll in libxl: one is the fd recheck, which
+ * is not blocking.  There is only the one blocking call, in
+ * eventloop_iteration.  poll runs with the ctx unlocked, so osevents
+ * might be added after it unlocks the ctx - that is what we are
+ * worried about.
+ *
+ * To demonstrate that the unstale liveness property is satisfied:
+ *
+ * We define a baton holder as follows: a libxl thread is a baton
+ * holder if
+ *   (a) it has an egc or an ao and holds the ctx lock, or
+ *   (b) it has an active non-app poller and no osevents have been
+ *   added since it released the lock, or
+ *   (c) it has an active non-app poller which has been woken
+ *   (by writing to its pipe), so it will not sleep
+ * We will maintain the invariant (the "baton invariant") that
+ * whenever there is any active poller, there is at least
+ * one baton holder.  ("non-app" means simply "not poller_app".)
+ *
+ * No thread outside libxl can have an active non-app poller: pollers
+ * are put on the 

[Xen-devel] [PATCH v3 03/10] libxl: event: Introduce CTX_UNLOCK_EGC_FREE

[Ian Jackson]

This is a very common exit pattern.  We are going to want to change
this pattern.  So we should make it into a macro of its own.

No functional change.

Signed-off-by: Ian Jackson 
Reviewed-by: George Dunlap 
Tested-by: George Dunlap 
---
 tools/libxl/libxl_event.c| 18 ++
 tools/libxl/libxl_fork.c |  6 ++
 tools/libxl/libxl_internal.h |  2 ++
 3 files changed, 10 insertions(+), 16 deletions(-)

diff --git a/tools/libxl/libxl_event.c b/tools/libxl/libxl_event.c
index 5b12a45e70..be37e12bb0 100644
--- a/tools/libxl/libxl_event.c
+++ b/tools/libxl/libxl_event.c
@@ -1152,8 +1152,7 @@ int libxl_osevent_beforepoll(libxl_ctx *ctx, int *nfds_io,
 CTX_LOCK;
 int rc = beforepoll_internal(gc, ctx->poller_app,
  nfds_io, fds, timeout_upd, now);
-CTX_UNLOCK;
-EGC_FREE;
+CTX_UNLOCK_EGC_FREE;
 return rc;
 }
 
@@ -1305,8 +1304,7 @@ void libxl_osevent_afterpoll(libxl_ctx *ctx, int nfds, 
const struct pollfd *fds,
 EGC_INIT(ctx);
 CTX_LOCK;
 afterpoll_internal(egc, ctx->poller_app, nfds, fds, now);
-CTX_UNLOCK;
-EGC_FREE;
+CTX_UNLOCK_EGC_FREE;
 }
 
 /*
@@ -1342,8 +1340,7 @@ void libxl_osevent_occurred_fd(libxl_ctx *ctx, void 
*for_libxl,
 fd_occurs(egc, ev, revents_ign);
 
  out:
-CTX_UNLOCK;
-EGC_FREE;
+CTX_UNLOCK_EGC_FREE;
 }
 
 void libxl_osevent_occurred_timeout(libxl_ctx *ctx, void *for_libxl)
@@ -1365,8 +1362,7 @@ void libxl_osevent_occurred_timeout(libxl_ctx *ctx, void 
*for_libxl)
 time_occurs(egc, ev, ERROR_TIMEDOUT);
 
  out:
-CTX_UNLOCK;
-EGC_FREE;
+CTX_UNLOCK_EGC_FREE;
 }
 
 void libxl__event_disaster(libxl__egc *egc, const char *msg, int errnoval,
@@ -1546,8 +1542,7 @@ int libxl_event_check(libxl_ctx *ctx, libxl_event 
**event_r,
 EGC_INIT(ctx);
 CTX_LOCK;
 int rc = event_check_internal(egc, event_r, typemask, pred, pred_user);
-CTX_UNLOCK;
-EGC_FREE;
+CTX_UNLOCK_EGC_FREE;
 return rc;
 }
 
@@ -1772,8 +1767,7 @@ int libxl_event_wait(libxl_ctx *ctx, libxl_event 
**event_r,
  out:
 libxl__poller_put(ctx, poller);
 
-CTX_UNLOCK;
-EGC_FREE;
+CTX_UNLOCK_EGC_FREE;
 return rc;
 }
 
diff --git a/tools/libxl/libxl_fork.c b/tools/libxl/libxl_fork.c
index 0f1b6b518c..cf170b9085 100644
--- a/tools/libxl/libxl_fork.c
+++ b/tools/libxl/libxl_fork.c
@@ -483,8 +483,7 @@ int libxl_childproc_reaped(libxl_ctx *ctx, pid_t pid, int 
status)
 assert(CTX->childproc_hooks->chldowner
== libxl_sigchld_owner_mainloop);
 int rc = childproc_reaped(egc, pid, status);
-CTX_UNLOCK;
-EGC_FREE;
+CTX_UNLOCK_EGC_FREE;
 return rc;
 }
 
@@ -529,8 +528,7 @@ void libxl_childproc_sigchld_occurred(libxl_ctx *ctx)
 assert(CTX->childproc_hooks->chldowner
== libxl_sigchld_owner_mainloop);
 childproc_checkall(egc);
-CTX_UNLOCK;
-EGC_FREE;
+CTX_UNLOCK_EGC_FREE;
 }
 
 static void sigchld_selfpipe_handler(libxl__egc *egc, libxl__ev_fd *ev,
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 581d64b99c..983fffac7a 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -2363,6 +2363,8 @@ _hidden void libxl__egc_cleanup(libxl__egc *egc);
 
 #define EGC_FREE   libxl__egc_cleanup(egc)
 
+#define CTX_UNLOCK_EGC_FREE  do{ CTX_UNLOCK; EGC_FREE; }while(0)
+
 
 /*
  * Machinery for asynchronous operations ("ao")
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 09/10] libxl: event: Fix possible hang with libxl_osevent_beforepoll

[Ian Jackson]

If the application uses libxl_osevent_beforepoll, a similar hang is
possible to the one described and fixed in
   libxl: event: Fix hang when mixing blocking and eventy calls
Application behaviour would have to be fairly unusual, but it
doesn't seem sensible to just leave this latent bug.

We fix the latent bug by waking up the "poller_app" pipe every time we
add osevents.  If the application does not ever call beforepoll, we
write one byte to the pipe and set pipe_nonempty and then we ignore
it.  We only write another byte if beforepoll is called again.

Normally in an eventy program there would only be one thread calling
libxl_osevent_beforepoll.  The effect in such a program is to
sometimes needlessly go round the poll loop again if a timeout
callback becomes interested in a new osevent.  We'll fix that in a
moment.

Signed-off-by: Ian Jackson 
Reviewed-by: George Dunlap 
Tested-by: George Dunlap 
---
v2: New addition to correctness arguments in libxl_event.c comment.
---
 tools/libxl/libxl_event.c | 54 +--
 1 file changed, 43 insertions(+), 11 deletions(-)

diff --git a/tools/libxl/libxl_event.c b/tools/libxl/libxl_event.c
index 45cc67942d..5f6a607d80 100644
--- a/tools/libxl/libxl_event.c
+++ b/tools/libxl/libxl_event.c
@@ -41,18 +41,25 @@ static void ao__check_destroy(libxl_ctx *ctx, libxl__ao 
*ao);
  *
  * We need the following property (the "unstale liveness property"):
  *
- * Whenever any thread is blocking in the libxl event loop[1], at
- * least one thread must be using an up to date osevent set.  It is OK
- * for all but one threads to have stale event sets, because so long
- * as one waiting thread has the right event set, any actually
- * interesting event will, if nothing else, wake that "right" thread
- * up.  It will then make some progress and/or, if it exits, ensure
- * that some other thread becomes the "right" thread.
+ * Whenever any thread is blocking as a result of being given an fd
+ * set or timeout by libxl, at least one thread must be using an up to
+ * date osevent set.  It is OK for all but one threads to have stale
+ * event sets, because so long as one waiting thread has the right
+ * event set, any actually interesting event will, if nothing else,
+ * wake that "right" thread up.  It will then make some progress
+ * and/or, if it exits, ensure that some other thread becomes the
+ * "right" thread.
  *
- * [1] TODO: Right now we are considering only the libxl event loop.
- * We need to consider application event loop outside libxl too.
+ * For threads blocking outside libxl and which are receiving libxl's
+ * fd and timeout information via the libxl_osevent_hooks callbacks,
+ * libxl calls this function as soon as it becomes interested.  It is
+ * the responsiblity of a provider of these functions in a
+ * multithreaded environment to make arrangements to wake up event
+ * waiting thread(s) with stale event sets.
  *
- * Argument that our approach is sound:
+ * Waiters outside libxl using _beforepoll are dealt with below.
+ *
+ * For the libxl event loop, the argument is as follows:
  *
  * The issue we are concerned about is libxl sleeping on an out of
  * date fd set, or too long a timeout, so that it doesn't make
@@ -132,7 +139,29 @@ static void ao__check_destroy(libxl_ctx *ctx, libxl__ao 
*ao);
  * will reenter libxl when it gains the lock and necessarily then
  * becomes a baton holder in category (a).
  *
- * So the "baton invariant" is maintained.  QED.
+ * So the "baton invariant" is maintained.
+ * QED (for waiters in libxl).
+ *
+ *
+ * For waiters outside libxl which used libxl_osevent_beforepoll
+ * to get the fd set:
+ *
+ * As above, adding an osevent involves having an egc or an ao.
+ * It sets poller->osevents_added on all active pollers.  Notably
+ * it sets it on poller_app, which is always active.
+ *
+ * The thread which does this will dispose of its egc or ao before
+ * exiting libxl so it will always wake up the poller_app if the last
+ * call to _beforepoll was before the osevents were added.  So the
+ * application's fd set contains at least a wakeup in the form of the
+ * poller_app fd.  The application cannot sleep on the libxl fd set
+ * until it has called _afterpoll which empties the pipe, and it
+ * is expected to then call _beforepoll again before sleeping.
+ *
+ * So all the application's event waiting thread(s) will always have
+ * an up to date osevent set, and will be woken up if necessary to
+ * achieve this.  (This is in contrast libxl's own event loop where
+ * only one thread need be up to date, as discussed above.)
  */
 static void pollers_note_osevent_added(libxl_ctx *ctx) {
 libxl__poller *poller;
@@ -157,6 +186,9 @@ void libxl__egc_ao_cleanup_1_baton(libxl__gc *gc)
 {
 libxl__poller *search, *wake=0;
 
+if (CTX->poller_app->osevents_added)
+baton_wake(gc, CTX->poller_app);
+
 LIBXL_LIST_FOREACH(search, >pollers_active, active_entry) {
 if (search == 

[Xen-devel] [PATCH v3 01/10] libxl: event: Rename poller.fds_changed to .fds_deregistered

[Ian Jackson]

This is only for deregistration.  We are going to add another variable
for new events, with different semantics, and this overly-general name
will become confusing.

Signed-off-by: Ian Jackson 
Reviewed-by: George Dunlap 
Tested-by: George Dunlap 
---
 tools/libxl/libxl_event.c| 8 
 tools/libxl/libxl_internal.h | 6 +++---
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/tools/libxl/libxl_event.c b/tools/libxl/libxl_event.c
index aa8b7d1945..1210c1bfb3 100644
--- a/tools/libxl/libxl_event.c
+++ b/tools/libxl/libxl_event.c
@@ -239,7 +239,7 @@ void libxl__ev_fd_deregister(libxl__gc *gc, libxl__ev_fd 
*ev)
 ev->fd = -1;
 
 LIBXL_LIST_FOREACH(poller, >pollers_fds_changed, fds_changed_entry)
-poller->fds_changed = 1;
+poller->fds_deregistered = 1;
 
  out:
 CTX_UNLOCK;
@@ -1120,7 +1120,7 @@ static int beforepoll_internal(libxl__gc *gc, 
libxl__poller *poller,
 
 *nfds_io = used;
 
-poller->fds_changed = 0;
+poller->fds_deregistered = 0;
 
 libxl__ev_time *etime = LIBXL_TAILQ_FIRST(>etimes);
 if (etime) {
@@ -1186,7 +1186,7 @@ static int afterpoll_check_fd(libxl__poller *poller,
 /* again, stale slot entry */
 continue;
 
-assert(poller->fds_changed || !(fds[slot].revents & POLLNVAL));
+assert(poller->fds_deregistered || !(fds[slot].revents & POLLNVAL));
 
 /* we mask in case requested events have changed */
 int slot_revents = fds[slot].revents & events;
@@ -1626,7 +1626,7 @@ int libxl__poller_init(libxl__gc *gc, libxl__poller *p)
 int rc;
 p->fd_polls = 0;
 p->fd_rindices = 0;
-p->fds_changed = 0;
+p->fds_deregistered = 0;
 
 rc = libxl__pipe_nonblock(CTX, p->wakeup_pipe);
 if (rc) goto out;
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index ba8c9b41ab..c5b71d15f0 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -629,14 +629,14 @@ struct libxl__poller {
 /*
  * We also use the poller to record whether any fds have been
  * deregistered since we entered poll.  Each poller which is not
- * idle is on the list pollers_fds_changed.  fds_changed is
+ * idle is on the list pollers_fds_changed.  fds_deregistered is
  * cleared by beforepoll, and tested by afterpoll.  Whenever an fd
- * event is deregistered, we set the fds_changed of all non-idle
+ * event is deregistered, we set the fds_deregistered of all non-idle
  * pollers.  So afterpoll can tell whether any POLLNVAL is
  * plausibly due to an fd being closed and reopened.
  */
 LIBXL_LIST_ENTRY(libxl__poller) fds_changed_entry;
-bool fds_changed;
+bool fds_deregistered;
 };
 
 struct libxl__gc {
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 05/10] libxl: event: Make libxl__poller_wakeup take a gc, not an egc

[Ian Jackson]

We are going to want to call this in the following situation:

 * We have just set up an ao, which is to call back - so a
   non-synchronous one.  It ought not to call the application
   back right away, so no egc.

 * There is a libxl thread blocking somewhere but it is using
   using an out of date fd or timeout set, which does not take into
   account the ao we have just started.

 * We try to wake that thread up, but libxl__poller_wakeup fails.

In more detail:

The idea before was that these two functions take an egc, not so much
because it actually uses the egc, but to make sure it's only called in a
restricted set of conditions; and now we're relaxing those conditions.

Specifically, we need to make one exception, relating to ao's.

In the situation described above, there is no egc, but we need to call
libxl__poller_wakeup.  Introducing an egc is wrong because that would
imply that this situation might result in application callbacks, but
it shouldn't (and not having an egc prevents that).

libxl__poller_wakeup and LIBXL__EVENT_DISASTER only take an egc for
form's sake; they don't use any part of it other than the gc.  The
"form's sake" is to stop them being called from libxl entrypoints that
are not involved in event generation.

Before this patch this is enforced by the types: you can't call it in
the wrong place because it wants an egc which you don't have.

After this patch this is no longer enforced.  But the mistake
(principally, calling _DISASTER) seems unlikely.  The type enforcement
I mention above was done because it was possible and easy, not because
it was important.

Signed-off-by: Ian Jackson 
Reviewed-by: George Dunlap 
Tested-by: George Dunlap 
---
v3: Significantly expanded commit message based on irc comments
v2: New patch
---
 tools/libxl/libxl_event.c| 7 +++
 tools/libxl/libxl_internal.h | 2 +-
 2 files changed, 4 insertions(+), 5 deletions(-)

diff --git a/tools/libxl/libxl_event.c b/tools/libxl/libxl_event.c
index 16e6786889..268a5da120 100644
--- a/tools/libxl/libxl_event.c
+++ b/tools/libxl/libxl_event.c
@@ -1477,7 +1477,7 @@ void libxl__event_occurred(libxl__egc *egc, libxl_event 
*event)
 libxl__poller *poller;
 LIBXL_TAILQ_INSERT_TAIL(>occurred, event, link);
 LIBXL_LIST_FOREACH(poller, >pollers_event, entry)
-libxl__poller_wakeup(egc, poller);
+libxl__poller_wakeup(gc, poller);
 }
 }
 
@@ -1668,9 +1668,8 @@ void libxl__poller_put(libxl_ctx *ctx, libxl__poller *p)
 LIBXL_LIST_INSERT_HEAD(>pollers_idle, p, entry);
 }
 
-void libxl__poller_wakeup(libxl__egc *egc, libxl__poller *p)
+void libxl__poller_wakeup(libxl__gc *gc, libxl__poller *p)
 {
-EGC_GC;
 int e = libxl__self_pipe_wakeup(p->wakeup_pipe[1]);
 if (e) LIBXL__EVENT_DISASTER(gc, "cannot poke watch pipe", e, 0);
 }
@@ -1924,7 +1923,7 @@ void libxl__ao_complete_check_progress_reports(libxl__egc 
*egc, libxl__ao *ao)
 assert(ao->in_initiator);
 if (!ao->constructing)
 /* don't bother with this if we're not in the event loop */
-libxl__poller_wakeup(egc, ao->poller);
+libxl__poller_wakeup(gc, ao->poller);
 } else if (ao->how.callback) {
 LOG(DEBUG, "ao %p: complete for callback", ao);
 LIBXL_TAILQ_INSERT_TAIL(>aos_for_callback, ao, 
entry_for_callback);
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 328ecf3e1e..b68ab218b6 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -1311,7 +1311,7 @@ _hidden void libxl__poller_put(libxl_ctx*, libxl__poller 
*p /* may be NULL */);
 
 /* Notifies whoever is polling using p that they should wake up.
  * ctx must be locked. */
-_hidden void libxl__poller_wakeup(libxl__egc *egc, libxl__poller *p);
+_hidden void libxl__poller_wakeup(libxl__gc *egc, libxl__poller *p);
 
 /* Internal to fork and child reaping machinery */
 extern const libxl_childproc_hooks libxl__childproc_default_hooks;
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 02/10] libxl: event: Rename ctx.pollers_fd_changed to .pollers_active

[Ian Jackson]

We are going to use this a bit more widely.  Make the name more
general.

Signed-off-by: Ian Jackson 
Reviewed-by: George Dunlap 
Tested-by: George Dunlap 
---
 tools/libxl/libxl.c  | 4 ++--
 tools/libxl/libxl_event.c| 8 
 tools/libxl/libxl_internal.h | 6 +++---
 3 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index a0d84281d0..f60fd3e4fd 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -48,7 +48,7 @@ int libxl_ctx_alloc(libxl_ctx **pctx, int version,
 ctx->poller_app = 0;
 LIBXL_LIST_INIT(>pollers_event);
 LIBXL_LIST_INIT(>pollers_idle);
-LIBXL_LIST_INIT(>pollers_fds_changed);
+LIBXL_LIST_INIT(>pollers_active);
 
 LIBXL_LIST_INIT(>efds);
 LIBXL_TAILQ_INIT(>etimes);
@@ -177,7 +177,7 @@ int libxl_ctx_free(libxl_ctx *ctx)
 libxl__poller_put(ctx, ctx->poller_app);
 ctx->poller_app = NULL;
 assert(LIBXL_LIST_EMPTY(>pollers_event));
-assert(LIBXL_LIST_EMPTY(>pollers_fds_changed));
+assert(LIBXL_LIST_EMPTY(>pollers_active));
 libxl__poller *poller, *poller_tmp;
 LIBXL_LIST_FOREACH_SAFE(poller, >pollers_idle, entry, poller_tmp) {
 libxl__poller_dispose(poller);
diff --git a/tools/libxl/libxl_event.c b/tools/libxl/libxl_event.c
index 1210c1bfb3..5b12a45e70 100644
--- a/tools/libxl/libxl_event.c
+++ b/tools/libxl/libxl_event.c
@@ -238,7 +238,7 @@ void libxl__ev_fd_deregister(libxl__gc *gc, libxl__ev_fd 
*ev)
 LIBXL_LIST_REMOVE(ev, entry);
 ev->fd = -1;
 
-LIBXL_LIST_FOREACH(poller, >pollers_fds_changed, fds_changed_entry)
+LIBXL_LIST_FOREACH(poller, >pollers_active, active_entry)
 poller->fds_deregistered = 1;
 
  out:
@@ -1663,15 +1663,15 @@ libxl__poller *libxl__poller_get(libxl__gc *gc)
 }
 }
 
-LIBXL_LIST_INSERT_HEAD(>pollers_fds_changed, p,
-   fds_changed_entry);
+LIBXL_LIST_INSERT_HEAD(>pollers_active, p,
+   active_entry);
 return p;
 }
 
 void libxl__poller_put(libxl_ctx *ctx, libxl__poller *p)
 {
 if (!p) return;
-LIBXL_LIST_REMOVE(p, fds_changed_entry);
+LIBXL_LIST_REMOVE(p, active_entry);
 LIBXL_LIST_INSERT_HEAD(>pollers_idle, p, entry);
 }
 
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index c5b71d15f0..581d64b99c 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -629,13 +629,13 @@ struct libxl__poller {
 /*
  * We also use the poller to record whether any fds have been
  * deregistered since we entered poll.  Each poller which is not
- * idle is on the list pollers_fds_changed.  fds_deregistered is
+ * idle is on the list pollers_active.  fds_deregistered is
  * cleared by beforepoll, and tested by afterpoll.  Whenever an fd
  * event is deregistered, we set the fds_deregistered of all non-idle
  * pollers.  So afterpoll can tell whether any POLLNVAL is
  * plausibly due to an fd being closed and reopened.
  */
-LIBXL_LIST_ENTRY(libxl__poller) fds_changed_entry;
+LIBXL_LIST_ENTRY(libxl__poller) active_entry;
 bool fds_deregistered;
 };
 
@@ -678,7 +678,7 @@ struct libxl__ctx {
 
 libxl__poller *poller_app; /* libxl_osevent_beforepoll and _afterpoll */
 LIBXL_LIST_HEAD(, libxl__poller) pollers_event, pollers_idle;
-LIBXL_LIST_HEAD(, libxl__poller) pollers_fds_changed;
+LIBXL_LIST_HEAD(, libxl__poller) pollers_active;
 
 LIBXL_SLIST_HEAD(libxl__osevent_hook_nexi, libxl__osevent_hook_nexus)
 hook_fd_nexi_idle, hook_timeout_nexi_idle;
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 10/10] libxl: event: Move poller pipe emptying to the end of afterpoll

[Ian Jackson]

This seems neater.  It doesn't have any significant effect because:

The poller fd wouldn't be emptied by time_occurs.  It would only be
woken by time_occurs as a result of an ao completing, or by
libxl__egc_ao_cleanup_1_baton.  But ...1_baton won't be called in
between (for one thing, this would violate the rule of not still
having the active caller when ...1_baton is called).

While discussing this patch, I noticed that there is a possibility (in
libxl in general) that poller_put might be called on a woken poller.
It would probably be sensible at some point to make poller_get empty
the pipe, at least if the pipe_nonempty flag is set.

Signed-off-by: Ian Jackson 
Tested-by: George Dunlap 
Reviewed-by: George Dunlap 
---
v3: Completely revised commit message; now we think this is just
cleanup.
---
 tools/libxl/libxl_event.c | 12 ++--
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/tools/libxl/libxl_event.c b/tools/libxl/libxl_event.c
index 5f6a607d80..7c5387e94f 100644
--- a/tools/libxl/libxl_event.c
+++ b/tools/libxl/libxl_event.c
@@ -1453,12 +1453,6 @@ static void afterpoll_internal(libxl__egc *egc, 
libxl__poller *poller,
 fd_occurs(egc, efd, revents);
 }
 
-if (afterpoll_check_fd(poller,fds,nfds, poller->wakeup_pipe[0],POLLIN)) {
-poller->pipe_nonempty = 0;
-int e = libxl__self_pipe_eatall(poller->wakeup_pipe[0]);
-if (e) LIBXL__EVENT_DISASTER(gc, "read wakeup", e, 0);
-}
-
 for (;;) {
 libxl__ev_time *etime = LIBXL_TAILQ_FIRST(>etimes);
 if (!etime)
@@ -1473,6 +1467,12 @@ static void afterpoll_internal(libxl__egc *egc, 
libxl__poller *poller,
 
 time_occurs(egc, etime, ERROR_TIMEDOUT);
 }
+
+if (afterpoll_check_fd(poller,fds,nfds, poller->wakeup_pipe[0],POLLIN)) {
+poller->pipe_nonempty = 0;
+int e = libxl__self_pipe_eatall(poller->wakeup_pipe[0]);
+if (e) LIBXL__EVENT_DISASTER(gc, "read wakeup", e, 0);
+}
 }
 
 void libxl_osevent_afterpoll(libxl_ctx *ctx, int nfds, const struct pollfd 
*fds,
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 07/10] libxl: event: poller pipe optimisation

[Ian Jackson]

Track in userland whether the poller pipe is nonempty.  This saves us
writing many many bytes to the pipe if nothing ever reads them.

This is going to be relevant in a moment, where we are going to create
a situation where this will happen quite a lot.

Signed-off-by: Ian Jackson 
Reviewed-by: George Dunlap 
Tested-by: George Dunlap 
---
 tools/libxl/libxl_event.c| 3 +++
 tools/libxl/libxl_internal.h | 1 +
 2 files changed, 4 insertions(+)

diff --git a/tools/libxl/libxl_event.c b/tools/libxl/libxl_event.c
index b50d4e5074..3e76fa5af5 100644
--- a/tools/libxl/libxl_event.c
+++ b/tools/libxl/libxl_event.c
@@ -1417,6 +1417,7 @@ static void afterpoll_internal(libxl__egc *egc, 
libxl__poller *poller,
 }
 
 if (afterpoll_check_fd(poller,fds,nfds, poller->wakeup_pipe[0],POLLIN)) {
+poller->pipe_nonempty = 0;
 int e = libxl__self_pipe_eatall(poller->wakeup_pipe[0]);
 if (e) LIBXL__EVENT_DISASTER(gc, "read wakeup", e, 0);
 }
@@ -1809,6 +1810,8 @@ void libxl__poller_put(libxl_ctx *ctx, libxl__poller *p)
 
 void libxl__poller_wakeup(libxl__gc *gc, libxl__poller *p)
 {
+if (p->pipe_nonempty) return;
+p->pipe_nonempty = 1;
 int e = libxl__self_pipe_wakeup(p->wakeup_pipe[1]);
 if (e) LIBXL__EVENT_DISASTER(gc, "cannot poke watch pipe", e, 0);
 }
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index eec4bf767d..0ab324102b 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -625,6 +625,7 @@ struct libxl__poller {
 int (*fd_rindices)[3]; /* see libxl_event.c:beforepoll_internal */
 
 int wakeup_pipe[2]; /* 0 means no fd allocated */
+bool pipe_nonempty;
 
 /*
  * We also use the poller to record whether any fds have been
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 00/10] libxl: event: Fix hang for some applications

[Ian Jackson]

The meat here, including a description of the bug, is in:
  libxl: event: Fix hang when mixing blocking and eventy calls

This is all now Reviewed-by and Tested-by George, so it is ready to be
committed.  But I will be away for a bit soon and reverting something
of this form is probably undesirable.  So I will commit this in
something over a week (assuming no further comments arise).

The changes here from v2 are only to two of the commit messages
(marked m in the list below).

I am not sure whether this series is a backport candidate.  It is not
impossible that the bug we are fixing here is affecting (say) libvirt.
But if so presumably not in a significant way as we haven't seen
reports.  So even though this is a bugfix, I'm sceptical.

Ian Jackson (10):
   libxl: event: Rename poller.fds_changed to .fds_deregistered
   libxl: event: Rename ctx.pollers_fd_changed to .pollers_active
   libxl: event: Introduce CTX_UNLOCK_EGC_FREE
   libxl: event: Make LIBXL__EVENT_DISASTER take a gc, not an egc
 m libxl: event: Make libxl__poller_wakeup take a gc, not an egc
   libxl: event: Fix hang when mixing blocking and eventy calls
   libxl: event: poller pipe optimisation
   libxl: event: Break out baton_wake
   libxl: event: Fix possible hang with libxl_osevent_beforepoll
 m libxl: event: Move poller pipe emptying to the end of afterpoll

 tools/libxl/libxl.c  |   4 +-
 tools/libxl/libxl_aoutils.c  |   2 +-
 tools/libxl/libxl_disk.c |   4 +-
 tools/libxl/libxl_domain.c   |   2 +-
 tools/libxl/libxl_event.c| 286 +++
 tools/libxl/libxl_fork.c |  17 ++-
 tools/libxl/libxl_internal.h |  54 +---
 7 files changed, 290 insertions(+), 79 deletions(-)

-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH] xen/blkfront: limit allocated memory size to actual use case

[Juergen Gross]

Today the Xen blkfront driver allocates memory for one struct
blkfront_ring_info for each communication ring. This structure is
statically sized for the maximum supported configuration resulting
in a size of more than 90 kB.

As the main size contributor is one array inside the struct, the
memory allocation can easily be limited by moving this array to be
the last structure element and to allocate only the memory for the
actually needed array size.

Signed-off-by: Juergen Gross 
---
 drivers/block/xen-blkfront.c | 8 +++-
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
index c02be06c5299..61491167da19 100644
--- a/drivers/block/xen-blkfront.c
+++ b/drivers/block/xen-blkfront.c
@@ -151,9 +151,6 @@ MODULE_PARM_DESC(max_ring_page_order, "Maximum order of 
pages to be used for the
 #define BLK_RING_SIZE(info)\
__CONST_RING_SIZE(blkif, XEN_PAGE_SIZE * (info)->nr_ring_pages)
 
-#define BLK_MAX_RING_SIZE  \
-   __CONST_RING_SIZE(blkif, XEN_PAGE_SIZE * XENBUS_MAX_RING_GRANTS)
-
 /*
  * ring-ref%u i=(-1UL) would take 11 characters + 'ring-ref' is 8, so 19
  * characters are enough. Define to 20 to keep consistent with backend.
@@ -177,12 +174,12 @@ struct blkfront_ring_info {
unsigned int evtchn, irq;
struct work_struct work;
struct gnttab_free_callback callback;
-   struct blk_shadow shadow[BLK_MAX_RING_SIZE];
struct list_head indirect_pages;
struct list_head grants;
unsigned int persistent_gnts_c;
unsigned long shadow_free;
struct blkfront_info *dev_info;
+   struct blk_shadow shadow[];
 };
 
 /*
@@ -1915,7 +1912,8 @@ static int negotiate_mq(struct blkfront_info *info)
info->nr_rings = 1;
 
info->rinfo = kvcalloc(info->nr_rings,
-  sizeof(struct blkfront_ring_info),
+  struct_size(info->rinfo, shadow,
+  BLK_RING_SIZE(info)),
   GFP_KERNEL);
if (!info->rinfo) {
xenbus_dev_fatal(info->xbdev, -ENOMEM, "allocating ring_info 
structure");
-- 
2.16.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 10/10] libxl: event: Move poller pipe emptying to the end of afterpoll

[George Dunlap]

On 1/17/20 2:33 PM, Ian Jackson wrote:
> Ian Jackson writes ("Re: [PATCH v2 10/10] libxl: event: Move poller pipe 
> emptying to the end of afterpoll"):
>> TBH I still think this patch tidies the code up a bit.
> 
> Given you tested it with this change, and I think it makes it a bit
> tidier and no less correct, I would like to keep it.
> 
> I rewrote the commit message - see below.
> 
> Ian.
> 
> libxl: event: Move poller pipe emptying to the end of afterpoll
> 
> This seems neater.  It doesn't have any significant effect because:
> 
> The poller fd wouldn't be emptied by time_occurs.  It would only be
> woken by time_occurs as a result of an ao completing, or by
> libxl__egc_ao_cleanup_1_baton.  But ...1_baton won't be called in
> between (for one thing, this would violate the rule of not still
> having the active caller when ...1_baton is called).
> 
> While discussing this patch, I noticed that there is a possibility (in
> libxl in general) that poller_put might be called on a woken poller.
> It would probably be sensible at some point to make poller_get empty
> the pipe, at least if the pipe_nonempty flag is set.
> 
> Signed-off-by: Ian Jackson 
> Tested-by: George Dunlap 

With the new commit message:

Reviewed-by: George Dunlap 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 10/10] libxl: event: Move poller pipe emptying to the end of afterpoll

[George Dunlap]

On 1/17/20 2:24 PM, Ian Jackson wrote:
> George Dunlap writes ("Re: [PATCH v2 10/10] libxl: event: Move poller pipe 
> emptying to the end of afterpoll"):
>> On 1/13/20 5:08 PM, Ian Jackson wrote:
>>> If a timer event callback causes this poller to be woken (not very
>>> unlikely) we would go round the poll loop twice rather than once.
>>>
>>> Do the poller pipe emptying at the end; this is slightly more
>>> efficient because it can't cause any callbacks, so it happens after
>>> all the callbacks have been run.
>>>
>>> (This pipe-emptying has to happen in afterpoll rather than the
>>> apparently more logical beforepoll, because the application calling
>>> beforepoll doesn't constitute a promise to actually do anything.)
>>>
>>> Signed-off-by: Ian Jackson 
>>
>> I can't quite figure out: why would you end up going around the loop
>> twice, and how does this fix it?
> 
> I now think this is not true and the situation I describe cannot
> happen.
> 
> What I was thinking was that pollers_note_osevent_added might be
> called by something from time_occurs, and that would write a byte into
> the poller pipe.  But pollers_note_osevent_added doesn't wake up
> pollers; it just tags them osevents_added.
> 
> I now think the spurious wakeup cannot happen because:
> 
> For this patch to make any difference, the poller pipe would have to
> be woken up by something in the time scan loop in afterpoll_internal.
> 
> But poller pipes are only woken up by ao completion or by
> cleanup_1_baton.
> 
> cleanup_1_baton is not called anywhere there (as an argument against:
> any such call would violate the rule that cleanup_1_baton may not be
> called with a poller in hand).
> 
> And as for ao completion, we would indeed wake up the poller.  But we
> also mark the ao as complete, so ao_inprogress would spot
> !ao_work_outstanding, and not reenter eventloop_iteration at all.
> The woken-up poller would be put by ao_inprogress.
> 
> This leads me to this observation: poller_get might give you a
> woken-up poller.  This is not incorrect, but it is pointless.  So
> maybe I should write a patch that puts a call to
> libxl__self_pipe_eatall in libxl__poller_get.
> 
> TBH I still think this patch tidies the code up a bit.

No objection to it on those grounds. :-)

Thanks for the explanation,
 -George

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 10/10] libxl: event: Move poller pipe emptying to the end of afterpoll

[Ian Jackson]

Ian Jackson writes ("Re: [PATCH v2 10/10] libxl: event: Move poller pipe 
emptying to the end of afterpoll"):
> TBH I still think this patch tidies the code up a bit.

Given you tested it with this change, and I think it makes it a bit
tidier and no less correct, I would like to keep it.

I rewrote the commit message - see below.

Ian.

libxl: event: Move poller pipe emptying to the end of afterpoll

This seems neater.  It doesn't have any significant effect because:

The poller fd wouldn't be emptied by time_occurs.  It would only be
woken by time_occurs as a result of an ao completing, or by
libxl__egc_ao_cleanup_1_baton.  But ...1_baton won't be called in
between (for one thing, this would violate the rule of not still
having the active caller when ...1_baton is called).

While discussing this patch, I noticed that there is a possibility (in
libxl in general) that poller_put might be called on a woken poller.
It would probably be sensible at some point to make poller_get empty
the pipe, at least if the pipe_nonempty flag is set.

Signed-off-by: Ian Jackson 
Tested-by: George Dunlap 
---
v2: Completely revised commit message; now we think this is just
cleanup.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH V8 2/4] x86/altp2m: Add hypercall to set a range of sve bits

[Jan Beulich]

On 17.01.2020 14:31, Alexandru Stefan ISAILA wrote:
> By default the sve bits are not set.
> This patch adds a new hypercall, xc_altp2m_set_supress_ve_multi(),
> to set a range of sve bits.
> The core function, p2m_set_suppress_ve_multi(), does not break in case
> of a error and it is doing a best effort for setting the bits in the
> given range. A check for continuation is made in order to have
> preemption on large ranges.
> The gfn of the first error is stored in
> xen_hvm_altp2m_suppress_ve_multi.first_error_gfn and the error code is
> stored in xen_hvm_altp2m_suppress_ve_multi.first_error.
> If no error occurred the values will be 0.

I'm sorry for being nitpicky here, but this still isn't fully in
line with ...

> --- a/xen/include/public/hvm/hvm_op.h
> +++ b/xen/include/public/hvm/hvm_op.h
> @@ -46,6 +46,16 @@ struct xen_hvm_altp2m_suppress_ve {
>  uint64_t gfn;
>  };
>  
> +struct xen_hvm_altp2m_suppress_ve_multi {
> +uint16_t view;
> +uint8_t suppress_ve; /* Boolean type. */
> +uint8_t pad1;
> +int32_t first_error; /* Should be set to 0. */
> +uint64_t first_gfn; /* Value may be updated. */
> +uint64_t last_gfn;
> +uint64_t first_error_gfn; /* Gfn of the first error. */
> +};

... this: There's nothing said here about zeroing first_error_gfn
(and FAOD there doesn't need to be), and even first_error correctly
says only "should". Hence the values will be non-zero when there
was no error only if the caller had set them to zero. Anyway, this
alone surely is no reason for a v9, so take it just as a benign
(for the moment) remark.

Jan

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v4 00/18] VM forking

[Tamas K Lengyel]

> > Provided this is v4 now of the series and no issues
> > were raised so far for these particular patches they can be merged
> > with your Reviewed-by.
>
> I don't think so, under the current (sufficiently) common
> understanding of the rules. See George's proposal to change to a
> model like what you imply:
> https://lists.xenproject.org/archives/html/xen-devel/2020-01/msg00885.html
>

Ah OK, I though that was already agreed upon. I would certainly prefer
that model to speed things up and reduce the hassle to work with code
noone else maintains then me.

Tamas

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 10/10] libxl: event: Move poller pipe emptying to the end of afterpoll

[Ian Jackson]

George Dunlap writes ("Re: [PATCH v2 10/10] libxl: event: Move poller pipe 
emptying to the end of afterpoll"):
> On 1/13/20 5:08 PM, Ian Jackson wrote:
> > If a timer event callback causes this poller to be woken (not very
> > unlikely) we would go round the poll loop twice rather than once.
> > 
> > Do the poller pipe emptying at the end; this is slightly more
> > efficient because it can't cause any callbacks, so it happens after
> > all the callbacks have been run.
> > 
> > (This pipe-emptying has to happen in afterpoll rather than the
> > apparently more logical beforepoll, because the application calling
> > beforepoll doesn't constitute a promise to actually do anything.)
> > 
> > Signed-off-by: Ian Jackson 
> 
> I can't quite figure out: why would you end up going around the loop
> twice, and how does this fix it?

I now think this is not true and the situation I describe cannot
happen.

What I was thinking was that pollers_note_osevent_added might be
called by something from time_occurs, and that would write a byte into
the poller pipe.  But pollers_note_osevent_added doesn't wake up
pollers; it just tags them osevents_added.

I now think the spurious wakeup cannot happen because:

For this patch to make any difference, the poller pipe would have to
be woken up by something in the time scan loop in afterpoll_internal.

But poller pipes are only woken up by ao completion or by
cleanup_1_baton.

cleanup_1_baton is not called anywhere there (as an argument against:
any such call would violate the rule that cleanup_1_baton may not be
called with a poller in hand).

And as for ao completion, we would indeed wake up the poller.  But we
also mark the ao as complete, so ao_inprogress would spot
!ao_work_outstanding, and not reenter eventloop_iteration at all.
The woken-up poller would be put by ao_inprogress.

This leads me to this observation: poller_get might give you a
woken-up poller.  This is not incorrect, but it is pointless.  So
maybe I should write a patch that puts a call to
libxl__self_pipe_eatall in libxl__poller_get.

TBH I still think this patch tidies the code up a bit.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v6 03/11] scripts: add coccinelle script to use auto propagated errp

[Eric Blake]

On 1/10/20 1:41 PM, Vladimir Sementsov-Ogievskiy wrote:

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---


Rather light on the commit message. If nothing else, a comment about 
typical command-line usage would be helpful (yes, it's in the patch 
body, but sometimes I just refer to git log).



diff --git a/include/qapi/error.h b/include/qapi/error.h
index 532b9afb9e..dcfb77e107 100644
--- a/include/qapi/error.h
+++ b/include/qapi/error.h
@@ -141,6 +141,9 @@
   * ...
   * }
   *
+ * For mass conversion use script
+ *   scripts/coccinelle/auto-propagated-errp.cocci
+ *
   *
   * Receive and accumulate multiple errors (first one wins):
   * Error *err = NULL, *local_err = NULL;
diff --git a/scripts/coccinelle/auto-propagated-errp.cocci 
b/scripts/coccinelle/auto-propagated-errp.cocci
new file mode 100644
index 00..6c72a5049f
--- /dev/null
+++ b/scripts/coccinelle/auto-propagated-errp.cocci
@@ -0,0 +1,139 @@
+// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
+//



+// Usage example:
+// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
+//  --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
+//  blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
+
+@@
+// Add invocation to errp-functions where necessary
+// We should skip functions with "Error *const *errp"
+// parameter, but how to do it with coccinelle?
+// I don't know, so, I skip them by function name regex.
+// It's safe: if we not skip some functions with


s/not/did not/


+// "Error *const *errp", ERRP_AUTO_PROPAGATE invocation
+// will fail to compile, because of const violation.
+identifier fn !~ "error_append_.*_hint";
+identifier local_err, errp;
+@@
+
+ fn(..., Error **errp, ...)
+ {
++   ERRP_AUTO_PROPAGATE();
+<+...
+when != ERRP_AUTO_PROPAGATE();
+(
+error_append_hint(errp, ...);
+|
+error_prepend(errp, ...);
+|
+Error *local_err = NULL;
+)
+...+>
+ }
+


Looks like it should catch all functions that require adding the macro.


+@rule1@
+// We do not inherit from previous rule, as we want to match
+// also functions, which already had ERRP_AUTO_PROPAGATE
+// invocation.


Grammar suggestion:

// We want to patch error propagation in functions regardless of
// whether the function already uses ERRP_AUTO_PROPAGATE, hence
// this one does not inherit from the first rule.

Reviewed-by: Eric Blake 

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v4 00/18] VM forking

[Tamas K Lengyel]

On Fri, Jan 17, 2020 at 4:15 AM Anthony PERARD
 wrote:
>
> On Fri, Jan 17, 2020 at 10:12:14AM +0100, Jan Beulich wrote:
> > Please note that my previous mail was _to_ George, with you only
> > _cc_-ed. Hence the question was to George, not you. (It is a
> > common issue which I keep mentioning on meetings that the
> > distinction of To and Cc is often not being honored, albeit
> > typically more by senders than recipients.)
>
> Tip: Jan, you could also have started the sentence by "George, " in
> addition to properly setting the "To:", it would help a lot I think.
>
> Teaching people about setting properly "To:", and reading it before
> reading the email is a lost fight I think. Even so it can be useful to
> filter email which needs a response.

Yea, +1 for that, it would make addressed questions more apparent.
Gmail (which is what I use) doesn't break out the email header by
default with separate lines for to: and cc:, all recipients are in a
single line with no distinction between them.

Tamas

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v6 02/11] error: auto propagated local_err

[Eric Blake]

On 1/10/20 1:41 PM, Vladimir Sementsov-Ogievskiy wrote:

Here is introduced ERRP_AUTO_PROPAGATE macro, to be used at start of
functions with errp OUT parameter.


s/with/with an/



It has three goals:

1. Fix issue with error_fatal & error_prepend/error_append_hint: user


maybe s/&/and/ so it doesn't look like the C & operator.


can't see this additional information, because exit() happens in
error_setg earlier than information is added. [Reported by Greg Kurz]

2. Fix issue with error_abort & error_propagate: when we wrap


and again


error_abort by local_err+error_propagate, resulting coredump will


s/,/, the/


refer to error_propagate and not to the place where error happened.
(the macro itself doesn't fix the issue, but it allows to [3.] drop all


s/allows/allows us/
s/all/the/


local_err+error_propagate pattern, which will definitely fix the issue)
[Reported by Kevin Wolf]

3. Drop local_err+error_propagate pattern, which is used to workaround
void functions with errp parameter, when caller wants to know resulting
status. (Note: actually these functions could be merely updated to
return int error code).

To achieve these goals, we need to add invocation of the macro at start
of functions, which needs error_prepend/error_append_hint (1.); add
invocation of the macro at start of functions which do
local_err+error_propagate scenario the check errors, drop local errors
from them and just use *errp instead (2., 3.).


To achieve these goals, later patches will add invocations of this macro 
at the start of functions with either use 
error_prepend/error_append_hint (solving 1) or which use 
local_err+error_propagate to check errors, switching those functions to 
use *errp instead (solving 2 and 3).




Signed-off-by: Vladimir Sementsov-Ogievskiy 
---




- * Receive an error and pass it on to the caller:
+ * Receive an error and pass it on to the caller (DEPRECATED*):
   * Error *err = NULL;
   * foo(arg, );
   * if (err) {
@@ -98,6 +98,50 @@
   * foo(arg, errp);
   * for readability.
   *
+ * DEPRECATED* This pattern is deprecated now, use ERRP_AUTO_PROPAGATE macro


s/use/use the/


+ * instead (defined below).
+ * It's deprecated because of two things:
+ *
+ * 1. Issue with error_abort & error_propagate: when we wrap error_abort by


s/&/and/


+ * local_err+error_propagate, resulting coredump will refer to error_propagate


s/,/, the/


+ * and not to the place where error happened.
+ *
+ * 2. A lot of extra code of the same pattern
+ *



+/*
+ * ERRP_AUTO_PROPAGATE
+ *
+ * This macro is created to be the first line of a function which use
+ * Error **errp parameter to report error. It's needed only in cases where we
+ * want to use error_prepend, error_append_hint or dereference *errp. It's
+ * still safe (but useless) in other cases.


It doesn't _have_ to be the first line to compile (we require C99+ 
compilers, which allow declarations after statements); but rather 
because it makes it easier for our Coccinelle conversion script to catch 
outliers.  But I think this text is okay, without calling out that extra 
information (maybe the commit message should mention it, though).



+ *
+ * If errp is NULL or points to error_fatal, it is rewritten to point to a
+ * local Error object, which will be automatically propagated to the original
+ * errp on function exit (see error_propagator_cleanup).
+ *
+ * After invocation of this macro it is always safe to dereference errp
+ * (as it's not NULL anymore) and to add information (by error_prepend or
+ * error_append_hint)
+ * (as, if it was error_fatal, we swapped it with a local_error to be
+ * propagated on cleanup).


double () () looks odd, as does the mid-sentence newline.


+ *
+ * Note: we don't wrap the error_abort case, as we want resulting coredump
+ * to point to the place where the error happened, not to error_propagate.
+ */
+#define ERRP_AUTO_PROPAGATE()  \
+g_auto(ErrorPropagator) _auto_errp_prop = {.errp = errp};  \
+errp = ((errp == NULL || *errp == error_fatal) \
+? &_auto_errp_prop.local_err : errp)
+
  /*
   * Special error destination to abort on error.
   * See error_setg() and error_propagate() for details.



The macro itself is fine, my comments are solely on the commit message 
and comments.  Depending on how much cleanup Markus is willing to do 
rather than require a respin, you can add:


Reviewed-by: Eric Blake 

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2] xen/balloon: Support xend-based toolstack take two

[Juergen Gross]

Commit 3aa6c19d2f38be ("xen/balloon: Support xend-based toolstack")
tried to fix a regression with running on rather ancient Xen versions.
Unfortunately the fix was based on the assumption that xend would
just use another Xenstore node, but in reality only some downstream
versions of xend are doing that. The upstream xend does not write
that Xenstore node at all, so the problem must be fixed in another
way.

The easiest way to achieve that is to fall back to the behavior
before commit 96edd61dcf4436 ("xen/balloon: don't online new memory
initially") in case the static memory maximum can't be read.

This is achieved by setting static_max to the current number of
memory pages known by the system resulting in target_diff becoming
zero.

Fixes: 3aa6c19d2f38be ("xen/balloon: Support xend-based toolstack")
Signed-off-by: Juergen Gross 
Reviewed-by: Boris Ostrovsky 
Cc:  # 4.13
---
V2: better commit message
---
 drivers/xen/xen-balloon.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/xen/xen-balloon.c b/drivers/xen/xen-balloon.c
index 6d12fc368210..a8d24433c8e9 100644
--- a/drivers/xen/xen-balloon.c
+++ b/drivers/xen/xen-balloon.c
@@ -94,7 +94,7 @@ static void watch_target(struct xenbus_watch *watch,
  "%llu", _max) == 1))
static_max >>= PAGE_SHIFT - 10;
else
-   static_max = new_target;
+   static_max = balloon_stats.current_pages;
 
target_diff = (xen_pv_domain() || xen_initial_domain()) ? 0
: static_max - balloon_stats.target_pages;
-- 
2.16.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 05/10] libxl: event: Make libxl__poller_wakeup take a gc, not an egc

[George Dunlap]

On 1/17/20 1:46 PM, Ian Jackson wrote:
> George Dunlap writes ("Re: [PATCH v2 05/10] libxl: event: Make 
> libxl__poller_wakeup take a gc, not an egc"):
>> On 1/13/20 5:08 PM, Ian Jackson wrote:
>>> We are going to want to call this in the following situation:
>>>
>>>  * We have just set up an ao, which is to call back - so a
>>>non-synchronous one.  It ought not to call the application
>>>back right away, so no egc.
>>>
>>>  * There is a libxl thread blocking somewhere but it is using
>>>using an out of date fd or timeout set, which does not take into
>>>account the ao we have just started.
>>>
>>>  * We try to wake that thread up, but libxl__poller_wakeup fails.
>>
>> So the idea before was that these two functions take an egc, not so much
>> because it actually uses the egc, but to make sure it's only called in a
>> restricted set of conditions; and now we're relaxing those conditions?
> 
> Yes.  Specifically, we need to make one exception, relating to ao's.
> 
> In the situation described above, there is no egc, but we need to call
> libxl__poller_wakeup.  Introducing an egc is wrong because that would
> imply that this situation might result in application callbacks, but
> it shouldn't (and not having an egc prevents that).
> 
> libxl__poller_wakeup and LIBXL__EVENT_DISASTER only take an egc for
> form's sake; they don't use any part of it other than the gc.  The
> "form's sake" is to stop them being called from libxl entrypoints that
> are not involved in event generation.
> 
> Before this patch this is enforced by the types: you can't call it in
> the wrong place because it wants an egc which you don't have.
> 
> After this patch this is no longer enforced.  But the mistake
> (principally, calling _DISASTER) seems unlikely.  The type enforcement
> I mention above was done because it was possible and easy, not because
> it was important.

That makes sense; just trying partly to make sure I have it right,
partly to have things in the public record.  In which case, re the code:

Reviewed-by: George Dunlap 


> Does more of this want to be in the commit message ?

I was going to say I'm not sure we need another round-trip.  I'd be OK
with checking it in as-is; or you could edit the commit message on
check-in if you wanted.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 05/10] libxl: event: Make libxl__poller_wakeup take a gc, not an egc

[Ian Jackson]

George Dunlap writes ("Re: [PATCH v2 05/10] libxl: event: Make 
libxl__poller_wakeup take a gc, not an egc"):
> On 1/13/20 5:08 PM, Ian Jackson wrote:
> > We are going to want to call this in the following situation:
> > 
> >  * We have just set up an ao, which is to call back - so a
> >non-synchronous one.  It ought not to call the application
> >back right away, so no egc.
> > 
> >  * There is a libxl thread blocking somewhere but it is using
> >using an out of date fd or timeout set, which does not take into
> >account the ao we have just started.
> > 
> >  * We try to wake that thread up, but libxl__poller_wakeup fails.
> 
> So the idea before was that these two functions take an egc, not so much
> because it actually uses the egc, but to make sure it's only called in a
> restricted set of conditions; and now we're relaxing those conditions?

Yes.  Specifically, we need to make one exception, relating to ao's.

In the situation described above, there is no egc, but we need to call
libxl__poller_wakeup.  Introducing an egc is wrong because that would
imply that this situation might result in application callbacks, but
it shouldn't (and not having an egc prevents that).

libxl__poller_wakeup and LIBXL__EVENT_DISASTER only take an egc for
form's sake; they don't use any part of it other than the gc.  The
"form's sake" is to stop them being called from libxl entrypoints that
are not involved in event generation.

Before this patch this is enforced by the types: you can't call it in
the wrong place because it wants an egc which you don't have.

After this patch this is no longer enforced.  But the mistake
(principally, calling _DISASTER) seems unlikely.  The type enforcement
I mention above was done because it was possible and easy, not because
it was important.

Does more of this want to be in the commit message ?

Thanks,
Ian.
(much text of this mail first written on irc)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 07/10] libxl: event: poller pipe optimisation

[Ian Jackson]

George Dunlap writes ("Re: [PATCH v2 07/10] libxl: event: poller pipe 
optimisation"):
> On 1/13/20 5:08 PM, Ian Jackson wrote:
> > squash! libxl: event: poller pipe optimisation
> 
> Stray "squash" detrius.

Oops.

> Other than that:
> 
> Reviewed-by: George Dunlap 

Thanks, fixed.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 00/10] libxl: event: Fix hang for some applications

[George Dunlap]

On 1/13/20 5:08 PM, Ian Jackson wrote:
> The meat here, including a description of the bug, is in:
>   libxl: event: Fix hang when mixing blocking and eventy calls
> 
> Re v1 I wrote:
>   I suggest we try to convince ourselves of its correctness
>   via a second round of code review.
> 
> I put this into practice by writing an informal proof of correctness.
> This found a bug, the fixing of which was not entirely trivial.
> 
> George tells me he tested v1 of this series.  As with v1, I have
> compiled this v2 but not executed it.

I have tested this series both with my C-based proof of concept, and now
with the golang bindings, and it solves my problem and seems to work as
advertised.

Tested-by: George Dunlap 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 09/10] libxl: event: Fix possible hang with libxl_osevent_beforepoll

[George Dunlap]

On 1/13/20 5:08 PM, Ian Jackson wrote:
> If the application uses libxl_osevent_beforepoll, a similar hang is
> possible to the one described and fixed in
>libxl: event: Fix hang when mixing blocking and eventy calls
> Application behaviour would have to be fairly unusual, but it
> doesn't seem sensible to just leave this latent bug.
> 
> We fix the latent bug by waking up the "poller_app" pipe every time we
> add osevents.  If the application does not ever call beforepoll, we
> write one byte to the pipe and set pipe_nonempty and then we ignore
> it.  We only write another byte if beforepoll is called again.
> 
> Normally in an eventy program there would only be one thread calling
> libxl_osevent_beforepoll.  The effect in such a program is to
> sometimes needlessly go round the poll loop again if a timeout
> callback becomes interested in a new osevent.  We'll fix that in a
> moment.
> 
> Signed-off-by: Ian Jackson 

Same as the comment on patch 5:

Reviewed-by: George Dunlap 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [Xen-users] 4.13RC3 and PVHVM makes drive drops just after boot

[Anthony PERARD]

On Sat, Jan 04, 2020 at 10:24:37PM +1000, Andrew wrote:
> Hi Anthony,
> 
> 
> I have been trying to keep an eye on the mailing list, but I might have
> missed it. Do you mind if I ask if you had any luck with the below (and/or
> if there is a subject line or content I should be keeping an eye on to see
> if a patch has been released so we can re-test?)

CCing xen-devel

Hi Andrew,

Sorry, I haven't sent anything yet.

I've managed to workaround one part of the problem, but when I test it
with an nbd backend, my patch isn't enought. But it might work with
Ceph/rbd., I've attatch a patch that you could try.

The issue it that now QEMU wants to connect twice at the same time to
the backend (rbd, nbd) for the same disk. There were only one connection
at a time before, most of the time.

Cheers,

-- 
Anthony PERARD
>From 1b8d77f1f8709a6ef1960111ea022cfb6d74 Mon Sep 17 00:00:00 2001
From: Anthony PERARD 
Date: Fri, 17 Jan 2020 12:05:09 +
Subject: [PATCH] xen-block: Fix parsing of legacy options

Even though the xen-disk PV backend can be instantiated via QMP, we
still need to handle the case where the backend is created via
xenstore. This means that we need to be able to parse legacy disk
options such as "aio:nbd://host:1234/disk".

Signed-off-by: Anthony PERARD 
---
 block.c|  6 ++
 hw/block/xen-block.c   | 25 +
 include/sysemu/block-backend.h |  3 +++
 3 files changed, 30 insertions(+), 4 deletions(-)

diff --git a/block.c b/block.c
index ecd09dbbfd89..13b8690e5006 100644
--- a/block.c
+++ b/block.c
@@ -1705,6 +1705,12 @@ static int bdrv_fill_options(QDict **options, const char 
*filename,
 
 return 0;
 }
+int bdrv_fill_options_legacy(QDict **options, const char *filename,
+ int *flags, Error **errp)
+{
+return bdrv_fill_options(options, filename, flags, errp);
+}
+
 
 static int bdrv_child_check_perm(BdrvChild *c, BlockReopenQueue *q,
  uint64_t perm, uint64_t shared,
diff --git a/hw/block/xen-block.c b/hw/block/xen-block.c
index 879fc310a4c5..1cc97a001e1f 100644
--- a/hw/block/xen-block.c
+++ b/hw/block/xen-block.c
@@ -28,6 +28,7 @@
 #include "sysemu/iothread.h"
 #include "dataplane/xen-block.h"
 #include "trace.h"
+#include "include/block/qdict.h"
 
 static char *xen_block_get_name(XenDevice *xendev, Error **errp)
 {
@@ -687,7 +688,12 @@ static char *xen_block_blockdev_add(const char *id, QDict 
*qdict,
 
 trace_xen_block_blockdev_add(node_name);
 
-v = qobject_input_visitor_new(QOBJECT(qdict));
+qdict_flatten(qdict);
+v = qobject_input_visitor_new_flat_confused(qdict, _err);
+if (local_err) {
+error_propagate(errp, local_err);
+goto fail;
+}
 visit_type_BlockdevOptions(v, NULL, , _err);
 visit_free(v);
 
@@ -782,8 +788,14 @@ static XenBlockDrive *xen_block_drive_create(const char 
*id,
 file_layer = qdict_new();
 driver_layer = qdict_new();
 
-qdict_put_str(file_layer, "driver", "file");
-qdict_put_str(file_layer, "filename", filename);
+int flags = BDRV_O_PROTOCOL | BDRV_O_RDWR;
+if (mode && *mode != 'w') {
+flags &= ~BDRV_O_RDWR;
+}
+bdrv_fill_options_legacy(_layer, filename, , _err);
+if (local_err)
+goto done;
+
 g_free(filename);
 
 if (mode && *mode != 'w') {
@@ -816,7 +828,12 @@ static XenBlockDrive *xen_block_drive_create(const char 
*id,
  * It is necessary to turn file locking off as an emulated device
  * may have already opened the same image file.
  */
-qdict_put_str(file_layer, "locking", "off");
+const char *file_driver = qdict_get_str(file_layer, "driver");
+if (!strcmp("file", file_driver) ||
+!strcmp("host_device", file_driver) ||
+!strcmp("host_cdrom", file_driver)
+)
+qdict_put_str(file_layer, "locking", "off");
 
 qdict_put_str(driver_layer, "driver", driver);
 g_free(driver);
diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h
index b198deca0b24..93efded0ab61 100644
--- a/include/sysemu/block-backend.h
+++ b/include/sysemu/block-backend.h
@@ -98,6 +98,9 @@ void blk_remove_bs(BlockBackend *blk);
 int blk_insert_bs(BlockBackend *blk, BlockDriverState *bs, Error **errp);
 bool bdrv_has_blk(BlockDriverState *bs);
 bool bdrv_is_root_node(BlockDriverState *bs);
+/* deprecated, not to be used for new backends */
+int bdrv_fill_options_legacy(QDict **options, const char *filename,
+ int *flags, Error **errp);
 int blk_set_perm(BlockBackend *blk, uint64_t perm, uint64_t shared_perm,
  Error **errp);
 void blk_get_perm(BlockBackend *blk, uint64_t *perm, uint64_t *shared_perm);
-- 
Anthony PERARD

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 06/10] libxl: event: Fix hang when mixing blocking and eventy calls

[George Dunlap]

On 1/13/20 5:08 PM, Ian Jackson wrote:
> If the application calls libxl with ao_how==0 and also makes calls
> like _occurred, libxl will sometimes get stuck.
> 
> The bug happens as follows (for example):
> 
>   Thread A
>libxl_do_thing(,ao_how==0)
>libxl_do_thing starts, sets up some callbacks
>libxl_do_thing exit path calls AO_INPROGRESS
>libxl__ao_inprogress goes into event loop
>eventloop_iteration sleeps on:
>   - do_thing's current fd set
>   - sigchld pipe if applicable
>   - its poller
> 
>   Thread B
>libxl_something_occurred
>the something is to do with do_thing, above
>do_thing_next_callback does some more work
>do_thing_next_callback becomes interested in fd N
>thread B returns to application
> 
> Note that nothing wakes up thread A.  A is not listening on fd N.  So
> do_thing_* will not spot when fd N signals.  do_thing will not make
> further timely progress.  If there is no timeout thread A will never
> wake up.
> 
> The problem here occurs because thread A is waiting on an out of date
> osevent set.
> 
> There is also the possibility that a thread might block waiting for
> libxl osevents but outside libxl, eg if the application used
> libxl_osevent_beforepoll.  We will deal with that in a moment.
> 
> See the big comment in libxl_event.c for a fairly formal correctness
> argument.
> 
> This depends on libxl__egc_ao_cleanup_1_baton being called everywhere
> an egc or ao is disposed of.  Firstly egcs: in this patch we rename
> libxl__egc_cleanup, which means we catch all the disposal sites.
> Secondly aos: these are disposed of by (i) AO_CREATE_FAIL
> (ii) ao__inprogress and (iii) an event which completes the ao later.
> (i) and (ii) we handle by adding the call to _baton.  In the case of
> (iii) any such function must be an event-generating function so it has
> an egc too, so it will pass on the baton when the egc is disposed.
> 
> Reported-by: George Dunlap 
> Signed-off-by: Ian Jackson 

This all looks very plausible.  I don't feel confident I have enough of
a grasp of the situation to say that I would notice anything missing;
but I think it's worth putting in and letting osstest give it some
exercise (via libvirt).

Reviewed-by: George Dunlap 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH V8 4/4] x86/mm: Make use of the default access param from xc_altp2m_create_view

[Alexandru Stefan ISAILA]

At this moment the default_access param from xc_altp2m_create_view is
not used.

This patch assigns default_access to p2m->default_access at the time of
initializing a new altp2m view.

Signed-off-by: Alexandru Isaila 
Acked-by: Jan Beulich 
---
CC: Jan Beulich 
CC: Andrew Cooper 
CC: Wei Liu 
CC: "Roger Pau Monné" 
CC: George Dunlap 
CC: Ian Jackson 
CC: Julien Grall 
CC: Konrad Rzeszutek Wilk 
CC: Stefano Stabellini 
CC: Razvan Cojocaru 
CC: Tamas K Lengyel 
CC: Petre Pircalabu 
CC: George Dunlap 
---
Changes since V6:
- Remove the NULL check for p2m in xenmem_access_to_p2m_access()
- Use hostp2m for default access in p2m_init_next_altp2m()
- Remove the artifact line from p2m_init_next_altp2m().
---
 xen/arch/x86/hvm/hvm.c  |  3 ++-
 xen/arch/x86/mm/mem_access.c|  6 +++---
 xen/arch/x86/mm/p2m.c   | 20 +++-
 xen/include/asm-x86/p2m.h   |  3 ++-
 xen/include/public/hvm/hvm_op.h |  2 --
 xen/include/xen/mem_access.h|  4 
 6 files changed, 26 insertions(+), 12 deletions(-)

diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 4d79b4934e..b96fafed65 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -4654,7 +4654,8 @@ static int do_altp2m_op(
 }
 
 case HVMOP_altp2m_create_p2m:
-if ( !(rc = p2m_init_next_altp2m(d, )) )
+if ( !(rc = p2m_init_next_altp2m(d, ,
+ a.u.view.hvmmem_default_access)) )
 rc = __copy_to_guest(arg, , 1) ? -EFAULT : 0;
 break;
 
diff --git a/xen/arch/x86/mm/mem_access.c b/xen/arch/x86/mm/mem_access.c
index 31ff826393..d16540a9aa 100644
--- a/xen/arch/x86/mm/mem_access.c
+++ b/xen/arch/x86/mm/mem_access.c
@@ -314,9 +314,9 @@ static int set_mem_access(struct domain *d, struct 
p2m_domain *p2m,
 return rc;
 }
 
-static bool xenmem_access_to_p2m_access(struct p2m_domain *p2m,
-xenmem_access_t xaccess,
-p2m_access_t *paccess)
+bool xenmem_access_to_p2m_access(const struct p2m_domain *p2m,
+ xenmem_access_t xaccess,
+ p2m_access_t *paccess)
 {
 static const p2m_access_t memaccess[] = {
 #define ACCESS(ac) [XENMEM_access_##ac] = p2m_access_##ac
diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index 696946697a..4599a0bc24 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -25,6 +25,7 @@
 
 #include  /* copy_from_guest() */
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -2536,7 +2537,8 @@ void p2m_flush_altp2m(struct domain *d)
 altp2m_list_unlock(d);
 }
 
-static int p2m_activate_altp2m(struct domain *d, unsigned int idx)
+static int p2m_activate_altp2m(struct domain *d, unsigned int idx,
+   p2m_access_t hvmmem_default_access)
 {
 struct p2m_domain *hostp2m, *p2m;
 int rc;
@@ -2562,7 +2564,7 @@ static int p2m_activate_altp2m(struct domain *d, unsigned 
int idx)
 goto out;
 }
 
-p2m->default_access = hostp2m->default_access;
+p2m->default_access = hvmmem_default_access;
 p2m->domain = hostp2m->domain;
 p2m->global_logdirty = hostp2m->global_logdirty;
 p2m->min_remapped_gfn = gfn_x(INVALID_GFN);
@@ -2579,6 +2581,7 @@ static int p2m_activate_altp2m(struct domain *d, unsigned 
int idx)
 int p2m_init_altp2m_by_id(struct domain *d, unsigned int idx)
 {
 int rc = -EINVAL;
+struct p2m_domain *hostp2m = p2m_get_hostp2m(d);
 
 if ( idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) )
 return rc;
@@ -2587,16 +2590,23 @@ int p2m_init_altp2m_by_id(struct domain *d, unsigned 
int idx)
 
 if ( d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] ==
  mfn_x(INVALID_MFN) )
-rc = p2m_activate_altp2m(d, idx);
+rc = p2m_activate_altp2m(d, idx, hostp2m->default_access);
 
 altp2m_list_unlock(d);
 return rc;
 }
 
-int p2m_init_next_altp2m(struct domain *d, uint16_t *idx)
+int p2m_init_next_altp2m(struct domain *d, uint16_t *idx,
+ xenmem_access_t hvmmem_default_access)
 {
 int rc = -EINVAL;
 unsigned int i;
+p2m_access_t a;
+struct p2m_domain *hostp2m = p2m_get_hostp2m(d);
+
+if ( hvmmem_default_access > XENMEM_access_default ||
+ !xenmem_access_to_p2m_access(hostp2m, hvmmem_default_access, ) )
+return rc;
 
 altp2m_list_lock(d);
 
@@ -2605,7 +2615,7 @@ int p2m_init_next_altp2m(struct domain *d, uint16_t *idx)
 if ( d->arch.altp2m_eptp[i] != mfn_x(INVALID_MFN) )
 continue;
 
-rc = p2m_activate_altp2m(d, i);
+rc = p2m_activate_altp2m(d, i, a);
 
 if ( !rc )
 *idx = i;
diff --git a/xen/include/asm-x86/p2m.h b/xen/include/asm-x86/p2m.h
index 94285db1b4..ac2d2787f4 100644
--- a/xen/include/asm-x86/p2m.h
+++ b/xen/include/asm-x86/p2m.h
@@ -884,7 +884,8 @@ bool p2m_altp2m_get_or_propagate(struct p2m_domain 

[Xen-devel] [PATCH V8 2/4] x86/altp2m: Add hypercall to set a range of sve bits

[Alexandru Stefan ISAILA]

By default the sve bits are not set.
This patch adds a new hypercall, xc_altp2m_set_supress_ve_multi(),
to set a range of sve bits.
The core function, p2m_set_suppress_ve_multi(), does not break in case
of a error and it is doing a best effort for setting the bits in the
given range. A check for continuation is made in order to have
preemption on large ranges.
The gfn of the first error is stored in
xen_hvm_altp2m_suppress_ve_multi.first_error_gfn and the error code is
stored in xen_hvm_altp2m_suppress_ve_multi.first_error.
If no error occurred the values will be 0.

Signed-off-by: Alexandru Isaila 

---
CC: Ian Jackson 
CC: Wei Liu 
CC: Andrew Cooper 
CC: George Dunlap 
CC: Jan Beulich 
CC: Julien Grall 
CC: Konrad Rzeszutek Wilk 
CC: Stefano Stabellini 
CC: "Roger Pau Monné" 
CC: George Dunlap 
CC: Razvan Cojocaru 
CC: Tamas K Lengyel 
CC: Petre Pircalabu 
---
Changes since V7:
- Fix commit message
- Move all in values in the sve initializer
- Drop sve.first_error check.
---
 tools/libxc/include/xenctrl.h   |  4 ++
 tools/libxc/xc_altp2m.c | 33 +++
 xen/arch/x86/hvm/hvm.c  | 20 +
 xen/arch/x86/mm/p2m.c   | 75 +
 xen/include/public/hvm/hvm_op.h | 13 ++
 xen/include/xen/mem_access.h|  3 ++
 6 files changed, 130 insertions(+), 18 deletions(-)

diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 75f191ae3a..cc4eb1e3d3 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -1923,6 +1923,10 @@ int xc_altp2m_switch_to_view(xc_interface *handle, 
uint32_t domid,
  uint16_t view_id);
 int xc_altp2m_set_suppress_ve(xc_interface *handle, uint32_t domid,
   uint16_t view_id, xen_pfn_t gfn, bool sve);
+int xc_altp2m_set_supress_ve_multi(xc_interface *handle, uint32_t domid,
+   uint16_t view_id, xen_pfn_t first_gfn,
+   xen_pfn_t last_gfn, bool sve,
+   xen_pfn_t *error_gfn, int32_t *error_code);
 int xc_altp2m_get_suppress_ve(xc_interface *handle, uint32_t domid,
   uint16_t view_id, xen_pfn_t gfn, bool *sve);
 int xc_altp2m_set_mem_access(xc_interface *handle, uint32_t domid,
diff --git a/tools/libxc/xc_altp2m.c b/tools/libxc/xc_altp2m.c
index 09dad0355e..46fb725806 100644
--- a/tools/libxc/xc_altp2m.c
+++ b/tools/libxc/xc_altp2m.c
@@ -234,6 +234,39 @@ int xc_altp2m_set_suppress_ve(xc_interface *handle, 
uint32_t domid,
 return rc;
 }
 
+int xc_altp2m_set_supress_ve_multi(xc_interface *handle, uint32_t domid,
+   uint16_t view_id, xen_pfn_t first_gfn,
+   xen_pfn_t last_gfn, bool sve,
+   xen_pfn_t *error_gfn, int32_t *error_code)
+{
+int rc;
+DECLARE_HYPERCALL_BUFFER(xen_hvm_altp2m_op_t, arg);
+
+arg = xc_hypercall_buffer_alloc(handle, arg, sizeof(*arg));
+if ( arg == NULL )
+return -1;
+
+arg->version = HVMOP_ALTP2M_INTERFACE_VERSION;
+arg->cmd = HVMOP_altp2m_set_suppress_ve_multi;
+arg->domain = domid;
+arg->u.suppress_ve_multi.view = view_id;
+arg->u.suppress_ve_multi.first_gfn = first_gfn;
+arg->u.suppress_ve_multi.last_gfn = last_gfn;
+arg->u.suppress_ve_multi.suppress_ve = sve;
+
+rc = xencall2(handle->xcall, __HYPERVISOR_hvm_op, HVMOP_altp2m,
+  HYPERCALL_BUFFER_AS_ARG(arg));
+
+if ( arg->u.suppress_ve_multi.first_error )
+{
+*error_gfn = arg->u.suppress_ve_multi.first_error_gfn;
+*error_code = arg->u.suppress_ve_multi.first_error;
+}
+
+xc_hypercall_buffer_free(handle, arg);
+return rc;
+}
+
 int xc_altp2m_set_mem_access(xc_interface *handle, uint32_t domid,
  uint16_t view_id, xen_pfn_t gfn,
  xenmem_access_t access)
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 4723f5d09c..4d79b4934e 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -4520,6 +4520,7 @@ static int do_altp2m_op(
 case HVMOP_altp2m_destroy_p2m:
 case HVMOP_altp2m_switch_p2m:
 case HVMOP_altp2m_set_suppress_ve:
+case HVMOP_altp2m_set_suppress_ve_multi:
 case HVMOP_altp2m_get_suppress_ve:
 case HVMOP_altp2m_set_mem_access:
 case HVMOP_altp2m_set_mem_access_multi:
@@ -4678,6 +4679,25 @@ static int do_altp2m_op(
 }
 break;
 
+case HVMOP_altp2m_set_suppress_ve_multi:
+{
+uint64_t max_phys_addr = (1UL << d->arch.cpuid->extd.maxphysaddr) - 1;
+
+a.u.suppress_ve_multi.last_gfn = min(a.u.suppress_ve_multi.last_gfn,
+ max_phys_addr);
+
+if ( a.u.suppress_ve_multi.pad1 ||
+ a.u.suppress_ve_multi.first_gfn > a.u.suppress_ve_multi.last_gfn )
+rc = -EINVAL;
+else
+   

[Xen-devel] [PATCH V8 1/4] x86/mm: Add array_index_nospec to guest provided index values

[Alexandru Stefan ISAILA]

This patch aims to sanitize indexes, potentially guest provided
values, for altp2m_eptp[] and altp2m_p2m[] arrays.

Requested-by: Jan Beulich 
Signed-off-by: Alexandru Isaila 
Acked-by: Tamas K Lengyel 
---
CC: Razvan Cojocaru 
CC: Tamas K Lengyel 
CC: Petre Pircalabu 
CC: George Dunlap 
CC: Jan Beulich 
CC: Andrew Cooper 
CC: Wei Liu 
CC: "Roger Pau Monné" 
CC: Jun Nakajima 
CC: Kevin Tian 
---
Changes since V7:
- Make use of array_access_nospec() over
array_index_nospec(altp2m_idx, ARRAY_SIZE(d->arch.altp2m_p2m).
---
 xen/arch/x86/mm/mem_access.c | 21 ++-
 xen/arch/x86/mm/p2m-ept.c|  4 ++--
 xen/arch/x86/mm/p2m.c| 39 +---
 3 files changed, 37 insertions(+), 27 deletions(-)

diff --git a/xen/arch/x86/mm/mem_access.c b/xen/arch/x86/mm/mem_access.c
index 320b9fe621..31ff826393 100644
--- a/xen/arch/x86/mm/mem_access.c
+++ b/xen/arch/x86/mm/mem_access.c
@@ -366,11 +366,12 @@ long p2m_set_mem_access(struct domain *d, gfn_t gfn, 
uint32_t nr,
 #ifdef CONFIG_HVM
 if ( altp2m_idx )
 {
-if ( altp2m_idx >= MAX_ALTP2M ||
- d->arch.altp2m_eptp[altp2m_idx] == mfn_x(INVALID_MFN) )
+if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
+ d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
+ mfn_x(INVALID_MFN) )
 return -EINVAL;
 
-ap2m = d->arch.altp2m_p2m[altp2m_idx];
+ap2m = array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
 }
 #else
 ASSERT(!altp2m_idx);
@@ -425,11 +426,12 @@ long p2m_set_mem_access_multi(struct domain *d,
 #ifdef CONFIG_HVM
 if ( altp2m_idx )
 {
-if ( altp2m_idx >= MAX_ALTP2M ||
- d->arch.altp2m_eptp[altp2m_idx] == mfn_x(INVALID_MFN) )
+if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
+ d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
+ mfn_x(INVALID_MFN) )
 return -EINVAL;
 
-ap2m = d->arch.altp2m_p2m[altp2m_idx];
+ap2m = array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
 }
 #else
 ASSERT(!altp2m_idx);
@@ -491,11 +493,12 @@ int p2m_get_mem_access(struct domain *d, gfn_t gfn, 
xenmem_access_t *access,
 }
 else if ( altp2m_idx ) /* altp2m view 0 is treated as the hostp2m */
 {
-if ( altp2m_idx >= MAX_ALTP2M ||
- d->arch.altp2m_eptp[altp2m_idx] == mfn_x(INVALID_MFN) )
+if ( altp2m_idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) ||
+ d->arch.altp2m_eptp[array_index_nospec(altp2m_idx, MAX_EPTP)] ==
+ mfn_x(INVALID_MFN) )
 return -EINVAL;
 
-p2m = d->arch.altp2m_p2m[altp2m_idx];
+p2m = array_access_nospec(d->arch.altp2m_p2m, altp2m_idx);
 }
 #else
 ASSERT(!altp2m_idx);
diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c
index b5517769c9..b078a9a59e 100644
--- a/xen/arch/x86/mm/p2m-ept.c
+++ b/xen/arch/x86/mm/p2m-ept.c
@@ -1353,7 +1353,7 @@ void setup_ept_dump(void)
 
 void p2m_init_altp2m_ept(struct domain *d, unsigned int i)
 {
-struct p2m_domain *p2m = d->arch.altp2m_p2m[i];
+struct p2m_domain *p2m = array_access_nospec(d->arch.altp2m_p2m, i);
 struct p2m_domain *hostp2m = p2m_get_hostp2m(d);
 struct ept_data *ept;
 
@@ -1366,7 +1366,7 @@ void p2m_init_altp2m_ept(struct domain *d, unsigned int i)
 p2m->max_mapped_pfn = p2m->max_remapped_gfn = 0;
 ept = >ept;
 ept->mfn = pagetable_get_pfn(p2m_get_pagetable(p2m));
-d->arch.altp2m_eptp[i] = ept->eptp;
+d->arch.altp2m_eptp[array_index_nospec(i, MAX_EPTP)] = ept->eptp;
 }
 
 unsigned int p2m_find_altp2m_by_eptp(struct domain *d, uint64_t eptp)
diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index 3119269073..00b24342fc 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -2502,7 +2502,7 @@ static void p2m_reset_altp2m(struct domain *d, unsigned 
int idx,
 struct p2m_domain *p2m;
 
 ASSERT(idx < MAX_ALTP2M);
-p2m = d->arch.altp2m_p2m[idx];
+p2m = array_access_nospec(d->arch.altp2m_p2m, idx);
 
 p2m_lock(p2m);
 
@@ -2543,7 +2543,7 @@ static int p2m_activate_altp2m(struct domain *d, unsigned 
int idx)
 
 ASSERT(idx < MAX_ALTP2M);
 
-p2m = d->arch.altp2m_p2m[idx];
+p2m = array_access_nospec(d->arch.altp2m_p2m, idx);
 hostp2m = p2m_get_hostp2m(d);
 
 p2m_lock(p2m);
@@ -2574,12 +2574,13 @@ int p2m_init_altp2m_by_id(struct domain *d, unsigned 
int idx)
 {
 int rc = -EINVAL;
 
-if ( idx >= MAX_ALTP2M )
+if ( idx >= min(ARRAY_SIZE(d->arch.altp2m_p2m), MAX_EPTP) )
 return rc;
 
 altp2m_list_lock(d);
 
-if ( d->arch.altp2m_eptp[idx] == mfn_x(INVALID_MFN) )
+if ( d->arch.altp2m_eptp[array_index_nospec(idx, MAX_EPTP)] ==
+ mfn_x(INVALID_MFN) )
 rc = p2m_activate_altp2m(d, idx);
 
 altp2m_list_unlock(d);
@@ -2615,7 +2616,7 @@ int p2m_destroy_altp2m_by_id(struct domain *d, unsigned 

[Xen-devel] [PATCH V8 3/4] x86/mm: Pull vendor-independent altp2m code out of p2m-ept.c and into p2m.c

[Alexandru Stefan ISAILA]

No functional changes.

Requested-by: Jan Beulich 
Signed-off-by: Alexandru Isaila 
Reviewed-by: Jan Beulich 

---
CC: Jun Nakajima 
CC: Kevin Tian 
CC: George Dunlap 
CC: Jan Beulich 
CC: Andrew Cooper 
CC: Wei Liu 
CC: "Roger Pau Monné" 
---
 xen/arch/x86/mm/p2m-ept.c | 6 --
 xen/arch/x86/mm/p2m.c | 6 ++
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c
index b078a9a59e..05a5526e08 100644
--- a/xen/arch/x86/mm/p2m-ept.c
+++ b/xen/arch/x86/mm/p2m-ept.c
@@ -1357,13 +1357,7 @@ void p2m_init_altp2m_ept(struct domain *d, unsigned int 
i)
 struct p2m_domain *hostp2m = p2m_get_hostp2m(d);
 struct ept_data *ept;
 
-p2m->default_access = hostp2m->default_access;
-p2m->domain = hostp2m->domain;
-
-p2m->global_logdirty = hostp2m->global_logdirty;
 p2m->ept.ad = hostp2m->ept.ad;
-p2m->min_remapped_gfn = gfn_x(INVALID_GFN);
-p2m->max_mapped_pfn = p2m->max_remapped_gfn = 0;
 ept = >ept;
 ept->mfn = pagetable_get_pfn(p2m_get_pagetable(p2m));
 d->arch.altp2m_eptp[array_index_nospec(i, MAX_EPTP)] = ept->eptp;
diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index 3a2929c365..696946697a 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -2562,6 +2562,12 @@ static int p2m_activate_altp2m(struct domain *d, 
unsigned int idx)
 goto out;
 }
 
+p2m->default_access = hostp2m->default_access;
+p2m->domain = hostp2m->domain;
+p2m->global_logdirty = hostp2m->global_logdirty;
+p2m->min_remapped_gfn = gfn_x(INVALID_GFN);
+p2m->max_mapped_pfn = p2m->max_remapped_gfn = 0;
+
 p2m_init_altp2m_ept(d, idx);
 
  out:
-- 
2.17.1

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v6 01/11] qapi/error: add (Error **errp) cleaning APIs

[Eric Blake]

On 1/10/20 1:41 PM, Vladimir Sementsov-Ogievskiy wrote:

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---


Sparse commit message; it might be nice (for future 'git log' 
greppability) to at least mention the names of the functions being added.


  
+/*

+ * Functions to clean Error **errp: call corresponding Error *err cleaning
+ * function an set pointer to NULL


s/ an/, then/

Missing a '.' at the end of the sentence.

Otherwise,
Reviewed-by: Eric Blake 

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel