Re: Porting Xen to Jetson Nano

2020-07-22 Thread Christopher Clark
On Wed, Jul 22, 2020 at 10:59 AM Srinivas Bangalore  wrote:
> Dear Xen experts,
>
> Would greatly appreciate some hints on how to move forward with this one…

Hi Srini,

I don't have any strong recommendations for you, but I do want to say
that I'm very happy to see you taking this project on and I am hoping
for your success. I have a newly-arrived Jetson Nano sitting on my
desk here, purchased with the intention of getting Xen up and running
on it, that I just haven't got to work on yet. I'm also familiar with
Chris Patterson, Kyle Temkin and Ian Campbell's previous Tegra Jetson
patches and it would be great to see some further progress made from
those.

In my recent experience with the Raspberry Pi 4, one basic observation
with ARM kernel bringup is that if your device tree isn't good, your
dom0 kernel can be missing the configuration it needs to use the
serial port correctly and you don't get any diagnostics from it after
Xen attempts to launch it, so I would just patch the right serial port
config directly into your Linux kernel (eg. hardcode specific things
onto the kernel command line) so you're not messing about with that
any more.

The other thing I would recommend is patching in some printks into the
earliest part of the Xen parts of the Dom0 Linux kernel start code.
Others who are more familar with Xen on ARM may have some better
recommendations, but linux/arch/arm/xen/enlighten.c has a function
xen_guest_init that looks like a good place to stuff some extra
printks for some early proof-of-entry from your kernel, and that way
you'll have some indication whether execution has actually commenced
in there.

I don't think you're going to get a great deal of enthusiasm on this
list for Xen 4.8.5, unfortunately; most people around here work off
Xen's staging branch, and I'd be surprised to hear of anyone having
tried a 5.7 Linux kernel with Xen 4.8.5. I can understand why you
might start there from the existing patch series though.

Best of luck,

Christopher



[linux-linus test] 152097: regressions - trouble: fail/pass/starved

2020-07-22 Thread osstest service owner
flight 152097 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/152097/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-xsm 16 guest-start/debian.repeat fail REGR. vs. 151214

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-vhd  11 guest-start  fail in 152070 pass in 152097
 test-amd64-amd64-xl-rtds 18 guest-localmigrate/x10 fail pass in 152070
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeat  fail pass in 152070

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 151214
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 151214
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 151214
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 151214
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 151214
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 151214
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 151214
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 151214
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-qemuu-freebsd12-amd64  2 hosts-allocate   starved n/a
 test-amd64-amd64-qemuu-freebsd11-amd64  2 hosts-allocate   starved n/a

version targeted for testing:
 linux4fa640dc52302b5e62b01b05c755b055549633ae
baseline 

Re: [PATCH v2 01/11] xen/manage: keep track of the on-going suspend mode

2020-07-22 Thread Stefano Stabellini
On Wed, 22 Jul 2020, Anchal Agarwal wrote:
> On Tue, Jul 21, 2020 at 05:18:34PM -0700, Stefano Stabellini wrote:
> > On Tue, 21 Jul 2020, Boris Ostrovsky wrote:
> > > >> +static int xen_setup_pm_notifier(void)
> > > >> +{
> > > >> + if (!xen_hvm_domain())
> > > >> + return -ENODEV;
> > > >>
> > > >> I forgot --- what did we decide about non-x86 (i.e. ARM)?
> > > > It would be great to support that however, its  out of
> > > > scope for this patch set.
> > > > I’ll be happy to discuss it separately.
> > > 
> > >  I wasn't implying that this *should* work on ARM but rather whether 
> > >  this
> > >  will break ARM somehow (because xen_hvm_domain() is true there).
> > > 
> > > 
> > > >>> Ok makes sense. TBH, I haven't tested this part of code on ARM and 
> > > >>> the series
> > > >>> was only support x86 guests hibernation.
> > > >>> Moreover, this notifier is there to distinguish between 2 PM
> > > >>> events PM SUSPEND and PM hibernation. Now since we only care about PM
> > > >>> HIBERNATION I may just remove this code and rely on 
> > > >>> "SHUTDOWN_SUSPEND" state.
> > > >>> However, I may have to fix other patches in the series where this 
> > > >>> check may
> > > >>> appear and cater it only for x86 right?
> > > >>
> > > >>
> > > >> I don't know what would happen if ARM guest tries to handle hibernation
> > > >> callbacks. The only ones that you are introducing are in block and net
> > > >> fronts and that's arch-independent.
> > > >>
> > > >>
> > > >> You do add a bunch of x86-specific code though (syscore ops), would
> > > >> something similar be needed for ARM?
> > > >>
> > > >>
> > > > I don't expect this to work out of the box on ARM. To start with 
> > > > something
> > > > similar will be needed for ARM too.
> > > > We may still want to keep the driver code as-is.
> > > >
> > > > I understand the concern here wrt ARM, however, currently the support 
> > > > is only
> > > > proposed for x86 guests here and similar work could be carried out for 
> > > > ARM.
> > > > Also, if regular hibernation works correctly on arm, then all is needed 
> > > > is to
> > > > fix Xen side of things.
> > > >
> > > > I am not sure what could be done to achieve any assurances on arm side 
> > > > as far as
> > > > this series is concerned.
> > 
> > Just to clarify: new features don't need to work on ARM or cause any
> > addition efforts to you to make them work on ARM. The patch series only
> > needs not to break existing code paths (on ARM and any other platforms).
> > It should also not make it overly difficult to implement the ARM side of
> > things (if there is one) at some point in the future.
> > 
> > FYI drivers/xen/manage.c is compiled and working on ARM today, however
> > Xen suspend/resume is not supported. I don't know for sure if
> > guest-initiated hibernation works because I have not tested it.
> > 
> > 
> > 
> > > If you are not sure what the effects are (or sure that it won't work) on
> > > ARM then I'd add IS_ENABLED(CONFIG_X86) check, i.e.
> > >
> > >
> > > if (!IS_ENABLED(CONFIG_X86) || !xen_hvm_domain())
> > >   return -ENODEV;
> > 
> > That is a good principle to have and thanks for suggesting it. However,
> > in this specific case there is nothing in this patch that doesn't work
> > on ARM. From an ARM perspective I think we should enable it and
> > _pm_notifier_block should be registered.
> > 
> This question is for Boris, I think you we decided to get rid of the notifier
> in V3 as all we need  to check is SHUTDOWN_SUSPEND state which sounds 
> plausible
> to me. So this check may go away. It may still be needed for sycore_ops
> callbacks registration.
> > Given that all guests are HVM guests on ARM, it should work fine as is.
> > 
> > 
> > I gave a quick look at the rest of the series and everything looks fine
> > to me from an ARM perspective. I cannot imaging that the new freeze,
> > thaw, and restore callbacks for net and block are going to cause any
> > trouble on ARM. The two main x86-specific functions are
> > xen_syscore_suspend/resume and they look trivial to implement on ARM (in
> > the sense that they are likely going to look exactly the same.)
> > 
> Yes but for now since things are not tested I will put this
> !IS_ENABLED(CONFIG_X86) on syscore_ops calls registration part just to be safe
> and not break anything.
> > 
> > One question for Anchal: what's going to happen if you trigger a
> > hibernation, you have the new callbacks, but you are missing
> > xen_syscore_suspend/resume?
> > 
> > Is it any worse than not having the new freeze, thaw and restore
> > callbacks at all and try to do a hibernation?
> If callbacks are not there, I don't expect hibernation to work correctly.
> These callbacks takes care of xen primitives like shared_info_page,
> grant table, sched clock, runstate time which are important to save the 
> correct
> state of the guest and bring it back up. Other patches in the series, 

Re: [PATCH v2 01/11] xen/manage: keep track of the on-going suspend mode

2020-07-22 Thread Boris Ostrovsky
On 7/22/20 2:02 PM, Anchal Agarwal wrote:
> On Tue, Jul 21, 2020 at 05:18:34PM -0700, Stefano Stabellini wrote:
>>
>>
>>> If you are not sure what the effects are (or sure that it won't work) on
>>> ARM then I'd add IS_ENABLED(CONFIG_X86) check, i.e.
>>>
>>>
>>> if (!IS_ENABLED(CONFIG_X86) || !xen_hvm_domain())
>>>   return -ENODEV;
>> That is a good principle to have and thanks for suggesting it. However,
>> in this specific case there is nothing in this patch that doesn't work
>> on ARM. From an ARM perspective I think we should enable it and
>> _pm_notifier_block should be registered.
>>
> This question is for Boris, I think you we decided to get rid of the notifier
> in V3 as all we need  to check is SHUTDOWN_SUSPEND state which sounds 
> plausible
> to me. So this check may go away. It may still be needed for sycore_ops
> callbacks registration.


If this check is going away then I guess there is nothing to do here.


My concern isn't about this particular notifier but rather whether this
feature may affect existing functionality (ARM and PVH dom0). If Stefano
feels this should be fine for ARM then so be it.


-boris


>> Given that all guests are HVM guests on ARM, it should work fine as is.
>>
>>
>> I gave a quick look at the rest of the series and everything looks fine
>> to me from an ARM perspective. I cannot imaging that the new freeze,
>> thaw, and restore callbacks for net and block are going to cause any
>> trouble on ARM. The two main x86-specific functions are
>> xen_syscore_suspend/resume and they look trivial to implement on ARM (in
>> the sense that they are likely going to look exactly the same.)
>>
> Yes but for now since things are not tested I will put this
> !IS_ENABLED(CONFIG_X86) on syscore_ops calls registration part just to be safe
> and not break anything.
>> One question for Anchal: what's going to happen if you trigger a
>> hibernation, you have the new callbacks, but you are missing
>> xen_syscore_suspend/resume?
>>
>> Is it any worse than not having the new freeze, thaw and restore
>> callbacks at all and try to do a hibernation?
> If callbacks are not there, I don't expect hibernation to work correctly.
> These callbacks takes care of xen primitives like shared_info_page,
> grant table, sched clock, runstate time which are important to save the 
> correct
> state of the guest and bring it back up. Other patches in the series, adds all
> the logic to these syscore callbacks. Freeze/thaw/restore are just there for 
> at driver
> level.
>
> Thanks,
> Anchal






Re: [PATCH v2 01/11] xen/manage: keep track of the on-going suspend mode

2020-07-22 Thread Anchal Agarwal
On Tue, Jul 21, 2020 at 05:18:34PM -0700, Stefano Stabellini wrote:
> CAUTION: This email originated from outside of the organization. Do not click 
> links or open attachments unless you can confirm the sender and know the 
> content is safe.
> 
> 
> 
> On Tue, 21 Jul 2020, Boris Ostrovsky wrote:
> > >> +static int xen_setup_pm_notifier(void)
> > >> +{
> > >> + if (!xen_hvm_domain())
> > >> + return -ENODEV;
> > >>
> > >> I forgot --- what did we decide about non-x86 (i.e. ARM)?
> > > It would be great to support that however, its  out of
> > > scope for this patch set.
> > > I’ll be happy to discuss it separately.
> > 
> >  I wasn't implying that this *should* work on ARM but rather whether 
> >  this
> >  will break ARM somehow (because xen_hvm_domain() is true there).
> > 
> > 
> > >>> Ok makes sense. TBH, I haven't tested this part of code on ARM and the 
> > >>> series
> > >>> was only support x86 guests hibernation.
> > >>> Moreover, this notifier is there to distinguish between 2 PM
> > >>> events PM SUSPEND and PM hibernation. Now since we only care about PM
> > >>> HIBERNATION I may just remove this code and rely on "SHUTDOWN_SUSPEND" 
> > >>> state.
> > >>> However, I may have to fix other patches in the series where this check 
> > >>> may
> > >>> appear and cater it only for x86 right?
> > >>
> > >>
> > >> I don't know what would happen if ARM guest tries to handle hibernation
> > >> callbacks. The only ones that you are introducing are in block and net
> > >> fronts and that's arch-independent.
> > >>
> > >>
> > >> You do add a bunch of x86-specific code though (syscore ops), would
> > >> something similar be needed for ARM?
> > >>
> > >>
> > > I don't expect this to work out of the box on ARM. To start with something
> > > similar will be needed for ARM too.
> > > We may still want to keep the driver code as-is.
> > >
> > > I understand the concern here wrt ARM, however, currently the support is 
> > > only
> > > proposed for x86 guests here and similar work could be carried out for 
> > > ARM.
> > > Also, if regular hibernation works correctly on arm, then all is needed 
> > > is to
> > > fix Xen side of things.
> > >
> > > I am not sure what could be done to achieve any assurances on arm side as 
> > > far as
> > > this series is concerned.
> 
> Just to clarify: new features don't need to work on ARM or cause any
> addition efforts to you to make them work on ARM. The patch series only
> needs not to break existing code paths (on ARM and any other platforms).
> It should also not make it overly difficult to implement the ARM side of
> things (if there is one) at some point in the future.
> 
> FYI drivers/xen/manage.c is compiled and working on ARM today, however
> Xen suspend/resume is not supported. I don't know for sure if
> guest-initiated hibernation works because I have not tested it.
> 
> 
> 
> > If you are not sure what the effects are (or sure that it won't work) on
> > ARM then I'd add IS_ENABLED(CONFIG_X86) check, i.e.
> >
> >
> > if (!IS_ENABLED(CONFIG_X86) || !xen_hvm_domain())
> >   return -ENODEV;
> 
> That is a good principle to have and thanks for suggesting it. However,
> in this specific case there is nothing in this patch that doesn't work
> on ARM. From an ARM perspective I think we should enable it and
> _pm_notifier_block should be registered.
> 
This question is for Boris, I think you we decided to get rid of the notifier
in V3 as all we need  to check is SHUTDOWN_SUSPEND state which sounds plausible
to me. So this check may go away. It may still be needed for sycore_ops
callbacks registration.
> Given that all guests are HVM guests on ARM, it should work fine as is.
> 
> 
> I gave a quick look at the rest of the series and everything looks fine
> to me from an ARM perspective. I cannot imaging that the new freeze,
> thaw, and restore callbacks for net and block are going to cause any
> trouble on ARM. The two main x86-specific functions are
> xen_syscore_suspend/resume and they look trivial to implement on ARM (in
> the sense that they are likely going to look exactly the same.)
> 
Yes but for now since things are not tested I will put this
!IS_ENABLED(CONFIG_X86) on syscore_ops calls registration part just to be safe
and not break anything.
> 
> One question for Anchal: what's going to happen if you trigger a
> hibernation, you have the new callbacks, but you are missing
> xen_syscore_suspend/resume?
> 
> Is it any worse than not having the new freeze, thaw and restore
> callbacks at all and try to do a hibernation?
If callbacks are not there, I don't expect hibernation to work correctly.
These callbacks takes care of xen primitives like shared_info_page,
grant table, sched clock, runstate time which are important to save the correct
state of the guest and bring it back up. Other patches in the series, adds all
the logic to these syscore callbacks. Freeze/thaw/restore are just there 

[xen-unstable test] 152091: regressions - trouble: fail/pass/preparing/running/starved

2020-07-22 Thread osstest service owner
flight 152091 xen-unstable running [real]
http://logs.test-lab.xenproject.org/osstest/logs/152091/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-multivcpu 18 guest-localmigrate/x10  fail REGR. vs. 152045
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 10 debian-hvm-install fail REGR. 
vs. 152045
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 19 guest-start.2 fail REGR. 
vs. 152045
 test-amd64-amd64-xl-qemut-win7-amd64  2 hosts-allocate   running
 test-amd64-amd64-xl-qemut-ws16-amd64 10 windows-install  running
 test-amd64-amd64-xl-qemut-ws16-amd64  3 syslog-serverrunning
 test-amd64-amd64-xl-qemuu-debianhvm-amd64 18 guest-start/debianhvm.repeat 
running
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  3 syslog-server running

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 152045
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 152045
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 152045
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 152045
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 152045
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 152045
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop fail like 152045
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-qemuu-freebsd11-amd64  2 hosts-allocate   starved n/a
 test-amd64-amd64-qemuu-freebsd12-amd64  2 hosts-allocate 

[xen-unstable-smoke test] 152121: tolerable all pass - PUSHED

2020-07-22 Thread osstest service owner
flight 152121 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/152121/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  26707b747feb5d707f659989c0f8f2e847e8020a
baseline version:
 xen  f3885e8c3ceaef101e466466e879e97103ecce18

Last test of basis   152077  2020-07-21 16:02:01 Z1 days
Testing same since   152121  2020-07-22 15:07:52 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   f3885e8c3c..26707b747f  26707b747feb5d707f659989c0f8f2e847e8020a -> smoke



[xen-4.14-testing test] 152123: trouble: preparing/queued

2020-07-22 Thread osstest service owner
flight 152123 xen-4.14-testing running [real]
http://logs.test-lab.xenproject.org/osstest/logs/152123/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-i386-libvirtqueued
 test-armhf-armhf-xl-vhd   queued
 test-amd64-i386-libvirt-xsm   queued
 test-amd64-i386-xl-qemuu-ws16-amd64  queued
 test-amd64-amd64-xl   queued
 test-amd64-amd64-xl-qemuu-win7-amd64  queued
 test-amd64-i386-xl-qemut-debianhvm-amd64 queued
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   queued
 test-armhf-armhf-libvirt-raw  queued
 test-arm64-arm64-xl-credit2   queued
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm  queued
 test-xtf-amd64-amd64-2queued
 test-amd64-amd64-xl-qemut-debianhvm-amd64queued
 test-amd64-i386-xl-shadow queued
 test-armhf-armhf-xl-rtds  queued
 test-amd64-amd64-xl-rtds  queued
 test-amd64-amd64-dom0pvh-xl-amd  queued
 test-amd64-i386-qemuu-rhel6hvm-intel  queued
 test-arm64-arm64-xl-xsm   queued
 test-amd64-amd64-dom0pvh-xl-intel  queued
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm queued
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow  queued
 test-amd64-coresched-i386-xl  queued
 test-amd64-amd64-qemuu-nested-amd  queued
 test-amd64-i386-xl-xsmqueued
 test-amd64-amd64-xl-qemuu-ovmf-amd64  queued
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmqueued
 test-amd64-i386-freebsd10-amd64  queued
 test-amd64-i386-qemut-rhel6hvm-amd  queued
 build-arm64-libvirt   queued
 test-amd64-i386-xlqueued
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm  queued
 test-amd64-i386-freebsd10-i386  queued
 test-arm64-arm64-xl-thunderx  queued
 test-amd64-amd64-amd64-pvgrub  queued
 test-amd64-amd64-pair queued
 test-amd64-amd64-xl-credit2   queued
 test-amd64-amd64-livepatchqueued
 test-amd64-i386-migrupgrade   queued
 test-armhf-armhf-xl-arndale   queued
 test-arm64-arm64-xl-credit1   queued
 test-amd64-amd64-xl-multivcpu  queued
 test-amd64-i386-xl-rawqueued
 test-amd64-i386-pair  queued
 build-armhf-libvirt   queued
 test-amd64-coresched-amd64-xl  queued
 test-armhf-armhf-libvirt  queued
 test-amd64-amd64-xl-qcow2 queued
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow queued
 test-arm64-arm64-libvirt-xsm  queued
 test-amd64-amd64-libvirt-pair  queued
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrictqueued
 test-amd64-amd64-xl-pvshimqueued
 test-amd64-i386-libvirt-pair  queued
 test-amd64-i386-xl-qemuu-debianhvm-amd64 queued
 test-amd64-amd64-libvirt-vhd  queued
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsmqueued
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm queued
 build-amd64-libvirt   queued
 test-amd64-i386-xl-qemut-win7-amd64  queued
 test-amd64-i386-xl-pvshim queued
 test-amd64-i386-qemut-rhel6hvm-intel  queued
 test-xtf-amd64-amd64-5queued
 test-armhf-armhf-xl-multivcpu  queued
 test-amd64-amd64-i386-pvgrub  queued
 test-amd64-amd64-xl-qemuu-ws16-amd64  queued
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict queued
 test-armhf-armhf-xl   queued
 test-amd64-amd64-qemuu-nested-intel  queued
 test-amd64-i386-xl-qemut-ws16-amd64  queued
 test-arm64-arm64-xl-seattle   queued
 test-amd64-amd64-xl-credit1   queued
 test-xtf-amd64-amd64-3queued
 test-amd64-amd64-xl-xsm   queued
 test-amd64-amd64-xl-qemut-ws16-amd64  queued
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm queued
 test-amd64-amd64-libvirt  queued
 test-amd64-amd64-libvirt-xsm  queued
 test-amd64-amd64-qemuu-freebsd11-amd64  queued
 test-xtf-amd64-amd64-4queued
 test-amd64-amd64-pygrub   queued
 test-amd64-amd64-xl-qemuu-debianhvm-amd64queued
 test-arm64-arm64-xl   

RE: Porting Xen to Jetson Nano

2020-07-22 Thread Srinivas Bangalore
Dear Xen experts,

 

Would greatly appreciate some hints on how to move forward with this one.

I have included further details on Xen diagnostics below:

 

(XEN) *** LOADING DOMAIN 0 ***

(XEN) Loading kernel from boot module @ e100

(XEN) Allocating 1:1 mappings totalling 512MB for dom0:

(XEN) BANK[0] 0x00a000-0x00c000 (512MB)

(XEN) Grant table range: 0x00fec0-0x00fec6

(XEN) Loading zImage from e100 to
a008-a223c808

(XEN) Allocating PPI 16 for event channel interrupt

(XEN) Loading dom0 DTB to 0xa800-0xa803435c

(XEN) Scrubbing Free RAM on 1 nodes using 4 CPUs

(XEN) done.

(XEN) Initial low memory virq threshold set at 0x4000 pages.

(XEN) Std. Loglevel: Errors and warnings

(XEN) Guest Loglevel: All

(XEN) ***

(XEN) WARNING: CONSOLE OUTPUT IS SYNCHRONOUS

(XEN) This option is intended to aid debugging of Xen by ensuring

(XEN) that all output is synchronously delivered on the serial line.

(XEN) However it can introduce SIGNIFICANT latencies and affect

(XEN) timekeeping. It is NOT recommended for production use!

(XEN) ***

(XEN) 3... 2... 1...

(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to
Xen)

(XEN) Freed 300kB init memory.

(XEN) *** Serial input -> Xen (type 'CTRL-a' three times to switch input to
DOM0)

(XEN) 'h' pressed -> showing installed handlers

(XEN)  key '%' (ascii '25') => trap to xendbg

(XEN)  key '*' (ascii '2a') => print all diagnostics

(XEN)  key '0' (ascii '30') => dump Dom0 registers

(XEN)  key 'A' (ascii '41') => toggle alternative key handling

(XEN)  key 'H' (ascii '48') => dump heap info

(XEN)  key 'R' (ascii '52') => reboot machine

(XEN)  key 'a' (ascii '61') => dump timer queues

(XEN)  key 'd' (ascii '64') => dump registers

(XEN)  key 'e' (ascii '65') => dump evtchn info

(XEN)  key 'g' (ascii '67') => print grant table usage

(XEN)  key 'h' (ascii '68') => show this message

(XEN)  key 'm' (ascii '6d') => memory info

(XEN)  key 'q' (ascii '71') => dump domain (and guest debug) info

(XEN)  key 'r' (ascii '72') => dump run queues

(XEN)  key 't' (ascii '74') => display multi-cpu clock info

(XEN)  key 'w' (ascii '77') => synchronously dump console ring buffer
(dmesg)

(XEN) '*' pressed -> firing all diagnostic keyhandlers

(XEN) [d: dump registers]

(XEN) 'd' pressed -> dumping registers

(XEN)

(XEN) *** Dumping CPU0 guest state (d0v0): ***

(XEN) [ Xen-4.8.5  arm64  debug=n   Tainted:  C   ]

(XEN) CPU:0

(XEN) PC: a008

(XEN) LR: 

(XEN) SP_EL0: 

(XEN) SP_EL1: 

(XEN) CPSR:   01c5 MODE:64-bit EL1h (Guest Kernel, handler)

(XEN)  X0: a800  X1:   X2: 

(XEN)  X3:   X4:   X5: 

(XEN)  X6:   X7:   X8: 

(XEN)  X9:  X10:  X11: 

(XEN) X12:  X13:  X14: 

(XEN) X15:  X16:  X17: 

(XEN) X18:  X19:  X20: 

(XEN) X21:  X22:  X23: 

(XEN) X24:  X25:  X26: 

(XEN) X27:  X28:   FP: 

(XEN)

(XEN)ELR_EL1: 

(XEN)ESR_EL1: 

(XEN)FAR_EL1: 

(XEN)

(XEN)  SCTLR_EL1: 00c50838

(XEN)TCR_EL1: 

(XEN)  TTBR0_EL1: 

(XEN)  TTBR1_EL1: 

(XEN)

(XEN)   VTCR_EL2: 80043594

(XEN)  VTTBR_EL2: 000100017f0f9000

(XEN)

(XEN)  SCTLR_EL2: 30cd183d

(XEN)HCR_EL2: 8038663f

(XEN)  TTBR0_EL2: fecfc000

(XEN)

(XEN)ESR_EL2: 820d

(XEN)  HPFAR_EL2: 

(XEN)FAR_EL2: a008

(XEN)

(XEN) Guest stack trace from sp=0:

(XEN)   Failed to convert stack to physical address

(XEN)

(XEN) *** Dumping CPU1 host state: ***

(XEN) [ Xen-4.8.5  arm64  debug=n   Tainted:  C   ]

(XEN) CPU:1

(XEN) PC: 00243ad8 idle_loop+0x74/0x11c

(XEN) LR: 00243ae0

(XEN) SP: 8000ff1bfe70

(XEN) CPSR:   2249 MODE:64-bit EL2h (Hypervisor, handler)

(XEN)  X0:   X1: 8000feeb8680  X2: 0001

(XEN)  X3: fed4  X4: 8000feeb8680  X5: 

(XEN)  X6: 8000ff16dc40  X7: 8000ff16dc58  X8: 8000ff1bfe08

(XEN)  X9: 00262458 X10: 000a X11: 8000ff1bfbe9

(XEN) X12: 0031 X13: 0001 X14: 8000ff1bfbe8

(XEN) X15: 0020 X16:  X17: 

(XEN)  

Re: [PATCH-for-4.14] SUPPORT.md: Set version and release/support dates

2020-07-22 Thread Julien Grall




On 22/07/2020 18:13, Paul Durrant wrote:

-Original Message-
From: Julien Grall 
Sent: 22 July 2020 17:59
To: Paul Durrant ; xen-devel@lists.xenproject.org
Cc: Paul Durrant ; Andrew Cooper 
; George Dunlap
; Ian Jackson ; Jan Beulich 
;
Stefano Stabellini ; Wei Liu 
Subject: Re: [PATCH-for-4.14] SUPPORT.md: Set version and release/support dates



On 22/07/2020 17:55, Paul Durrant wrote:

From: Paul Durrant 

Signed-off-by: Paul Durrant 


Acked-by: Julien Grall 


---
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Jan Beulich 
Cc: Julien Grall 
Cc: Stefano Stabellini 
Cc: Wei Liu 

Obviously this has my implicit Release-acked-by and is to be committed to
the staging-4.14 branch only.


I will commit it.


Thanks,


I ended up to revert the patch as there was some unhappiness on 
#xendevel about committing it.


I will let Ian doing it as part of release deliverables.

Cheers,

--
Julien Grall



Re: [PATCH-for-4.14] SUPPORT.md: Set version and release/support dates

2020-07-22 Thread Ian Jackson
Paul Durrant writes ("[PATCH-for-4.14] SUPPORT.md: Set version and 
release/support dates"):
> From: Paul Durrant 
> 
> Signed-off-by: Paul Durrant 
> ---
> Cc: Andrew Cooper 
> Cc: George Dunlap 
> Cc: Ian Jackson 
> Cc: Jan Beulich 
> Cc: Julien Grall 
> Cc: Stefano Stabellini 
> Cc: Wei Liu 
> 
> Obviously this has my implicit Release-acked-by and is to be committed to
> the staging-4.14 branch only.

Acked-by: Ian Jackson 

The plan is to commit this as part of the 4.14 release deliverables
prep.

Ian.



RE: [PATCH-for-4.14] SUPPORT.md: Set version and release/support dates

2020-07-22 Thread Paul Durrant
> -Original Message-
> From: Julien Grall 
> Sent: 22 July 2020 17:59
> To: Paul Durrant ; xen-devel@lists.xenproject.org
> Cc: Paul Durrant ; Andrew Cooper 
> ; George Dunlap
> ; Ian Jackson ; Jan 
> Beulich ;
> Stefano Stabellini ; Wei Liu 
> Subject: Re: [PATCH-for-4.14] SUPPORT.md: Set version and release/support 
> dates
> 
> 
> 
> On 22/07/2020 17:55, Paul Durrant wrote:
> > From: Paul Durrant 
> >
> > Signed-off-by: Paul Durrant 
> 
> Acked-by: Julien Grall 
> 
> > ---
> > Cc: Andrew Cooper 
> > Cc: George Dunlap 
> > Cc: Ian Jackson 
> > Cc: Jan Beulich 
> > Cc: Julien Grall 
> > Cc: Stefano Stabellini 
> > Cc: Wei Liu 
> >
> > Obviously this has my implicit Release-acked-by and is to be committed to
> > the staging-4.14 branch only.
> 
> I will commit it.

Thanks,

  Paul

> 
> > ---
> >   SUPPORT.md | 8 
> >   1 file changed, 4 insertions(+), 4 deletions(-)
> >
> > diff --git a/SUPPORT.md b/SUPPORT.md
> > index efbcb26ddf..88a182ac31 100644
> > --- a/SUPPORT.md
> > +++ b/SUPPORT.md
> > @@ -9,10 +9,10 @@ for the definitions of the support status levels etc.
> >
> >   # Release Support
> >
> > -Xen-Version: 4.14-rc
> > -Initial-Release: n/a
> > -Supported-Until: TBD
> > -Security-Support-Until: Unreleased - not yet security-supported
> > +Xen-Version: 4.14
> > +Initial-Release: 2020-07-24
> > +Supported-Until: 2022-01-24
> > +Security-Support-Until: 2023-07-24
> >
> >   Release Notes
> >   :  > href="https://wiki.xenproject.org/wiki/Xen_Project_4.14_Release_Notes;>RN
> >
> 
> --
> Julien Grall




Re: [PATCH-for-4.14] SUPPORT.md: Set version and release/support dates

2020-07-22 Thread Julien Grall




On 22/07/2020 17:55, Paul Durrant wrote:

From: Paul Durrant 

Signed-off-by: Paul Durrant 


Acked-by: Julien Grall 


---
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Jan Beulich 
Cc: Julien Grall 
Cc: Stefano Stabellini 
Cc: Wei Liu 

Obviously this has my implicit Release-acked-by and is to be committed to
the staging-4.14 branch only.


I will commit it.


---
  SUPPORT.md | 8 
  1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/SUPPORT.md b/SUPPORT.md
index efbcb26ddf..88a182ac31 100644
--- a/SUPPORT.md
+++ b/SUPPORT.md
@@ -9,10 +9,10 @@ for the definitions of the support status levels etc.
  
  # Release Support
  
-Xen-Version: 4.14-rc

-Initial-Release: n/a
-Supported-Until: TBD
-Security-Support-Until: Unreleased - not yet security-supported
+Xen-Version: 4.14
+Initial-Release: 2020-07-24
+Supported-Until: 2022-01-24
+Security-Support-Until: 2023-07-24
  
  Release Notes

  : https://wiki.xenproject.org/wiki/Xen_Project_4.14_Release_Notes;>RN



--
Julien Grall



[PATCH-for-4.14] SUPPORT.md: Set version and release/support dates

2020-07-22 Thread Paul Durrant
From: Paul Durrant 

Signed-off-by: Paul Durrant 
---
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Jan Beulich 
Cc: Julien Grall 
Cc: Stefano Stabellini 
Cc: Wei Liu 

Obviously this has my implicit Release-acked-by and is to be committed to
the staging-4.14 branch only.
---
 SUPPORT.md | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/SUPPORT.md b/SUPPORT.md
index efbcb26ddf..88a182ac31 100644
--- a/SUPPORT.md
+++ b/SUPPORT.md
@@ -9,10 +9,10 @@ for the definitions of the support status levels etc.
 
 # Release Support
 
-Xen-Version: 4.14-rc
-Initial-Release: n/a
-Supported-Until: TBD
-Security-Support-Until: Unreleased - not yet security-supported
+Xen-Version: 4.14
+Initial-Release: 2020-07-24
+Supported-Until: 2022-01-24
+Security-Support-Until: 2023-07-24
 
 Release Notes
 : https://wiki.xenproject.org/wiki/Xen_Project_4.14_Release_Notes;>RN
-- 
2.17.1




[PATCH] xen/x86: irq: Avoid a TOCTOU race in pirq_spin_lock_irq_desc()

2020-07-22 Thread Julien Grall
From: Julien Grall 

Even if we assigned pirq->arch.irq to a variable, a compile is still
allowed to read pirq->arch.irq multiple time. This means that the value
checked may be different from the value used to get the desc.

Force the compiler to only do one read access by using read_atomic().

Signed-off-by: Julien Grall 
---
 xen/arch/x86/irq.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index a69937c840b9..25f2eb611692 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -1187,7 +1187,7 @@ struct irq_desc *pirq_spin_lock_irq_desc(
 
 for ( ; ; )
 {
-int irq = pirq->arch.irq;
+int irq = read_atomic(>arch.irq);
 
 if ( irq <= 0 )
 return NULL;
-- 
2.17.1




Re: [RFC v2 1/2] arm,smmu: switch to using iommu_fwspec functions

2020-07-22 Thread Stefano Stabellini
On Tue, 21 Jul 2020, Brian Woods wrote:
> Modify the smmu driver so that it uses the iommu_fwspec helper
> functions.  This means both ARM IOMMU drivers will both use the
> iommu_fwspec helper functions.
> 
> Signed-off-by: Brian Woods 

[...]

> @@ -1924,14 +1924,21 @@ static int arm_smmu_add_device(struct device *dev)
>   ret = -ENOMEM;
>   goto out_put_group;
>   }
> + cfg->fwspec = kzalloc(sizeof(struct iommu_fwspec), GFP_KERNEL);
> + if (!cfg->fwspec) {
> + kfree(cfg);
> + ret = -ENOMEM;
> + goto out_put_group;
> + }
> + iommu_fwspec_init(dev, smmu->dev);

Normally the fwspec structure is initialized in
xen/drivers/passthrough/device_tree.c:iommu_add_dt_device. However here
we are trying to use it with the legacy bindings, that of course don't
initialize in iommu_add_dt_device because they are different.

So I imagine this is the reason why we have to initialize iommu_fwspec here
indepdendently from iommu_add_dt_device.

However, why are we allocating the struct iommu_fwspec twice?

We are calling kzalloc, then iommu_fwspec_init is calling xzalloc.

Similarly, we are storing the pointer to struct iommu_fwspec in
cfg->fwspec but actually there is already a pointer to struct
iommu_fwspec in struct device (the one set by dev_iommu_fwspec_set.)

Do we actually need both?



Re: [PATCH] x86/hvm: Clean up track_dirty_vram() calltree

2020-07-22 Thread Jan Beulich
On 22.07.2020 17:15, Andrew Cooper wrote:
>  * Rename nr to nr_frames.  A plain 'nr' is confusing to follow in the the
>lower levels.
>  * Use DIV_ROUND_UP() rather than opencoding it in several different ways
>  * The hypercall input is capped at uint32_t, so there is no need for
>nr_frames to be unsigned long in the lower levels.
> 
> No functional change.
> 
> Signed-off-by: Andrew Cooper 

Reviewed-by: Jan Beulich 

I'd like to note though that ...

> --- a/xen/arch/x86/mm/hap/hap.c
> +++ b/xen/arch/x86/mm/hap/hap.c
> @@ -58,16 +58,16 @@
>  
>  int hap_track_dirty_vram(struct domain *d,
>   unsigned long begin_pfn,
> - unsigned long nr,
> + unsigned int nr_frames,
>   XEN_GUEST_HANDLE(void) guest_dirty_bitmap)
>  {
>  long rc = 0;
>  struct sh_dirty_vram *dirty_vram;
>  uint8_t *dirty_bitmap = NULL;
>  
> -if ( nr )
> +if ( nr_frames )
>  {
> -int size = (nr + BITS_PER_BYTE - 1) / BITS_PER_BYTE;
> +unsigned int size = DIV_ROUND_UP(nr_frames, BITS_PER_BYTE);

... with the change from long to int this construct will now no
longer be correct for the (admittedly absurd) case of a hypercall
input in the range of [0xfff9,0x]. We now fully
depend on this getting properly rejected at the top level hypercall
handler (which limits to 1Gb worth of tracked space).

Jan



Re: [PATCH] x86/svm: Misc coding style corrections

2020-07-22 Thread Jan Beulich
On 22.07.2020 16:39, Andrew Cooper wrote:
> No functional change.
> 
> Signed-off-by: Andrew Cooper 

Acked-by: Jan Beulich 




Re: [PATCH] xen: credit2: document that min_rqd is valid and ok to use

2020-07-22 Thread Dario Faggioli
On Thu, 2020-03-26 at 18:08 +0100, Dario Faggioli wrote:
> Code is a bit involved, and it is not easy to tell that min_rqd,
> inside
> csched2_res_pick() is actually pointing to a runqueue, when it is
> dereferenced.
> 
> Add a comment and an ASSERT() for that.
> 
> Suggested-by: Jan Beulich 
> Signed-off-by: Dario Faggioli 
>
Ping.

Actually, for George, this is more a:

<> :-)

Regards
-- 
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
---
<> (Raistlin Majere)


signature.asc
Description: This is a digitally signed message part


Re: [PATCH v2 0/7] xen: credit2: limit the number of CPUs per runqueue

2020-07-22 Thread Dario Faggioli
On Tue, 2020-07-21 at 14:08 +0200, Jan Beulich wrote:
> On 28.05.2020 23:29, Dario Faggioli wrote:
> > Dario Faggioli (7):
> >   xen: credit2: factor cpu to runqueue matching in a function
> >   xen: credit2: factor runqueue initialization in its own
> > function.
> >   xen: cpupool: add a back-pointer from a scheduler to its pool
> >   xen: credit2: limit the max number of CPUs in a runqueue
> >   xen: credit2: compute cpus per-runqueue more dynamically.
> >   cpupool: create an the 'cpupool sync' infrastructure
> >   xen: credit2: rebalance the number of CPUs in the scheduler
> > runqueues
> 
> I still have the last three patches here as well as "xen: credit2:
> document that min_rqd is valid and ok to use" in my "waiting for
> sufficient acks" folder. 
>
Ok.

> Would you mind indicating if they should
> stay there (and you will take care of pinging or whatever is
> needed), or whether I may drop them (and you'll eventually re-
> submit)?
> 
So, the last 3 patches of this series, despite their status is indeed
"waiting for sufficient acks", in the sense that they've not been
looked at due to code freeze being imminent back then, but it still
would be ok for people to look at them, I'm happy for you to drop this
from your queue.

I will take care of resending just them in a new series and everything.
And thanks.

For the min_rqd one... That should be quite easy, in theory, so let me
ping the thread right now. Especially considering that I just looked
back at it, and noticed that I forgot to Cc George in the first place!
:-O

Regards
-- 
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
---
<> (Raistlin Majere)


signature.asc
Description: This is a digitally signed message part


Re: [osstest PATCH] dom0pvh: assign 1GB of memory to PVH dom0

2020-07-22 Thread Roger Pau Monné
On Wed, Jul 22, 2020 at 04:11:21PM +0100, Ian Jackson wrote:
> Roger Pau Monne writes ("[osstest PATCH] dom0pvh: assign 1GB of memory to PVH 
> dom0"):
> > Current tests use 512MB of memory for dom0, but that's too low for a
> > PVH dom0 on some hosts and will cause errors because memory is
> > ballooned out in order to obtain physical memory ranges to map foreign
> > pages.
> > 
> > Using ballooned out pages for foreign mappings also doesn't seem to
> > work properly with the current Linux kernel version, so increase the
> > memory assigned to dom0 to 1GB for PVH dom0 tests. We should see about
> > reverting this when using ballooned pages is fixed.
> > 
> > The runvar diff is:
> > 
> > +test-amd64-amd64-dom0pvh-xl-amd   dom0_mem 1024
> > +test-amd64-amd64-dom0pvh-xl-intel dom0_mem 1024
> > 
> > I've done a repro of the failed test on elbling0 with dom0_mem set to
> > 1GB and it seems to prevent the issue, the flight is 152111.
> > 
> > Signed-off-by: Roger Pau Monné 
> 
> Acked-by: Ian Jackson 
> 
> And queued.

Thanks! Forgot to add that I've checked x86 hosts and they all have at
least 8GB of RAM, so using 1GB for dom0 should be fine, as I don't
think osstest runs guests close to 7GB of RAM.

Roger.



[libvirt test] 152094: regressions - FAIL

2020-07-22 Thread osstest service owner
flight 152094 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/152094/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-i386-libvirt6 libvirt-buildfail REGR. vs. 151777
 build-arm64-libvirt   6 libvirt-buildfail REGR. vs. 151777
 build-amd64-libvirt   6 libvirt-buildfail REGR. vs. 151777
 build-armhf-libvirt   6 libvirt-buildfail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a

version targeted for testing:
 libvirt  6f59749e4e1ecaef0f7c252e5a57f5cd569f7ea3
baseline version:
 libvirt  2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z   12 days
Failing since151818  2020-07-11 04:18:52 Z   11 days   12 attempts
Testing same since   152094  2020-07-22 04:18:47 Z0 days1 attempts


People who touched revisions under test:
  Andrea Bolognani 
  Bastien Orivel 
  Boris Fiuczynski 
  Côme Borsoi 
  Daniel Henrique Barboza 
  Daniel P. Berrangé 
  Jianan Gao 
  Jin Yan 
  Jiri Denemark 
  Ján Tomko 
  Laine Stump 
  Liao Pingfang 
  Michal Privoznik 
  Nikolay Shirokovskiy 
  Pavel Hrdina 
  Peter Krempa 
  Prathamesh Chavan 
  Roman Bogorodskiy 
  Ryan Schmidt 
  Stefan Berger 
  Yi Wang 

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-arm64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  fail
 build-arm64-libvirt  fail
 build-armhf-libvirt  fail
 build-i386-libvirt   fail
 build-amd64-pvopspass
 build-arm64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmblocked 
 test-amd64-amd64-libvirt-xsm blocked 
 test-arm64-arm64-libvirt-xsm blocked 
 test-amd64-i386-libvirt-xsm  blocked 
 test-amd64-amd64-libvirt blocked 
 test-arm64-arm64-libvirt blocked 
 test-armhf-armhf-libvirt blocked 
 test-amd64-i386-libvirt  blocked 
 test-amd64-amd64-libvirt-pairblocked 
 test-amd64-i386-libvirt-pair blocked 
 test-arm64-arm64-libvirt-qcow2   blocked 
 test-armhf-armhf-libvirt-raw blocked 
 test-amd64-amd64-libvirt-vhd blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master

[PATCH] x86/hvm: Clean up track_dirty_vram() calltree

2020-07-22 Thread Andrew Cooper
 * Rename nr to nr_frames.  A plain 'nr' is confusing to follow in the the
   lower levels.
 * Use DIV_ROUND_UP() rather than opencoding it in several different ways
 * The hypercall input is capped at uint32_t, so there is no need for
   nr_frames to be unsigned long in the lower levels.

No functional change.

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
CC: Wei Liu 
CC: Roger Pau Monné 
CC: Tim Deegan 
---
 xen/arch/x86/hvm/dm.c| 13 +++--
 xen/arch/x86/mm/hap/hap.c| 21 +++--
 xen/arch/x86/mm/shadow/hvm.c | 16 
 xen/include/asm-x86/hap.h|  2 +-
 xen/include/asm-x86/shadow.h |  2 +-
 5 files changed, 28 insertions(+), 26 deletions(-)

diff --git a/xen/arch/x86/hvm/dm.c b/xen/arch/x86/hvm/dm.c
index e3f845165d..9930d68860 100644
--- a/xen/arch/x86/hvm/dm.c
+++ b/xen/arch/x86/hvm/dm.c
@@ -62,9 +62,10 @@ static bool _raw_copy_from_guest_buf_offset(void *dst,
 sizeof(dst))
 
 static int track_dirty_vram(struct domain *d, xen_pfn_t first_pfn,
-unsigned int nr, const struct xen_dm_op_buf *buf)
+unsigned int nr_frames,
+const struct xen_dm_op_buf *buf)
 {
-if ( nr > (GB(1) >> PAGE_SHIFT) )
+if ( nr_frames > (GB(1) >> PAGE_SHIFT) )
 return -EINVAL;
 
 if ( d->is_dying )
@@ -73,12 +74,12 @@ static int track_dirty_vram(struct domain *d, xen_pfn_t 
first_pfn,
 if ( !d->max_vcpus || !d->vcpu[0] )
 return -EINVAL;
 
-if ( ((nr + 7) / 8) > buf->size )
+if ( DIV_ROUND_UP(nr_frames, BITS_PER_BYTE) > buf->size )
 return -EINVAL;
 
-return shadow_mode_enabled(d) ?
-shadow_track_dirty_vram(d, first_pfn, nr, buf->h) :
-hap_track_dirty_vram(d, first_pfn, nr, buf->h);
+return shadow_mode_enabled(d)
+? shadow_track_dirty_vram(d, first_pfn, nr_frames, buf->h)
+:hap_track_dirty_vram(d, first_pfn, nr_frames, buf->h);
 }
 
 static int set_pci_intx_level(struct domain *d, uint16_t domain,
diff --git a/xen/arch/x86/mm/hap/hap.c b/xen/arch/x86/mm/hap/hap.c
index 7f84d0c6ea..4eedd1a995 100644
--- a/xen/arch/x86/mm/hap/hap.c
+++ b/xen/arch/x86/mm/hap/hap.c
@@ -58,16 +58,16 @@
 
 int hap_track_dirty_vram(struct domain *d,
  unsigned long begin_pfn,
- unsigned long nr,
+ unsigned int nr_frames,
  XEN_GUEST_HANDLE(void) guest_dirty_bitmap)
 {
 long rc = 0;
 struct sh_dirty_vram *dirty_vram;
 uint8_t *dirty_bitmap = NULL;
 
-if ( nr )
+if ( nr_frames )
 {
-int size = (nr + BITS_PER_BYTE - 1) / BITS_PER_BYTE;
+unsigned int size = DIV_ROUND_UP(nr_frames, BITS_PER_BYTE);
 
 if ( !paging_mode_log_dirty(d) )
 {
@@ -97,13 +97,13 @@ int hap_track_dirty_vram(struct domain *d,
 }
 
 if ( begin_pfn != dirty_vram->begin_pfn ||
- begin_pfn + nr != dirty_vram->end_pfn )
+ begin_pfn + nr_frames != dirty_vram->end_pfn )
 {
 unsigned long ostart = dirty_vram->begin_pfn;
 unsigned long oend = dirty_vram->end_pfn;
 
 dirty_vram->begin_pfn = begin_pfn;
-dirty_vram->end_pfn = begin_pfn + nr;
+dirty_vram->end_pfn = begin_pfn + nr_frames;
 
 paging_unlock(d);
 
@@ -115,7 +115,7 @@ int hap_track_dirty_vram(struct domain *d,
  * Switch vram to log dirty mode, either by setting l1e entries of
  * P2M table to be read-only, or via hardware-assisted log-dirty.
  */
-p2m_change_type_range(d, begin_pfn, begin_pfn + nr,
+p2m_change_type_range(d, begin_pfn, begin_pfn + nr_frames,
   p2m_ram_rw, p2m_ram_logdirty);
 
 guest_flush_tlb_mask(d, d->dirty_cpumask);
@@ -132,7 +132,7 @@ int hap_track_dirty_vram(struct domain *d,
 p2m_flush_hardware_cached_dirty(d);
 
 /* get the bitmap */
-paging_log_dirty_range(d, begin_pfn, nr, dirty_bitmap);
+paging_log_dirty_range(d, begin_pfn, nr_frames, dirty_bitmap);
 
 domain_unpause(d);
 }
@@ -153,14 +153,15 @@ int hap_track_dirty_vram(struct domain *d,
  * then stop tracking
  */
 begin_pfn = dirty_vram->begin_pfn;
-nr = dirty_vram->end_pfn - dirty_vram->begin_pfn;
+nr_frames = dirty_vram->end_pfn - dirty_vram->begin_pfn;
 xfree(dirty_vram);
 d->arch.hvm.dirty_vram = NULL;
 }
 
 paging_unlock(d);
-if ( nr )
-p2m_change_type_range(d, begin_pfn, begin_pfn + nr,
+
+if ( nr_frames )
+p2m_change_type_range(d, begin_pfn, begin_pfn + nr_frames,
   p2m_ram_logdirty, p2m_ram_rw);
 }
 out:
diff --git a/xen/arch/x86/mm/shadow/hvm.c b/xen/arch/x86/mm/shadow/hvm.c
index 

Re: [osstest PATCH] dom0pvh: assign 1GB of memory to PVH dom0

2020-07-22 Thread Ian Jackson
Roger Pau Monne writes ("[osstest PATCH] dom0pvh: assign 1GB of memory to PVH 
dom0"):
> Current tests use 512MB of memory for dom0, but that's too low for a
> PVH dom0 on some hosts and will cause errors because memory is
> ballooned out in order to obtain physical memory ranges to map foreign
> pages.
> 
> Using ballooned out pages for foreign mappings also doesn't seem to
> work properly with the current Linux kernel version, so increase the
> memory assigned to dom0 to 1GB for PVH dom0 tests. We should see about
> reverting this when using ballooned pages is fixed.
> 
> The runvar diff is:
> 
> +test-amd64-amd64-dom0pvh-xl-amd   dom0_mem 1024
> +test-amd64-amd64-dom0pvh-xl-intel dom0_mem 1024
> 
> I've done a repro of the failed test on elbling0 with dom0_mem set to
> 1GB and it seems to prevent the issue, the flight is 152111.
> 
> Signed-off-by: Roger Pau Monné 

Acked-by: Ian Jackson 

And queued.

Ian.



[osstest PATCH] dom0pvh: assign 1GB of memory to PVH dom0

2020-07-22 Thread Roger Pau Monne
Current tests use 512MB of memory for dom0, but that's too low for a
PVH dom0 on some hosts and will cause errors because memory is
ballooned out in order to obtain physical memory ranges to map foreign
pages.

Using ballooned out pages for foreign mappings also doesn't seem to
work properly with the current Linux kernel version, so increase the
memory assigned to dom0 to 1GB for PVH dom0 tests. We should see about
reverting this when using ballooned pages is fixed.

The runvar diff is:

+test-amd64-amd64-dom0pvh-xl-amd   dom0_mem 1024
+test-amd64-amd64-dom0pvh-xl-intel dom0_mem 1024

I've done a repro of the failed test on elbling0 with dom0_mem set to
1GB and it seems to prevent the issue, the flight is 152111.

Signed-off-by: Roger Pau Monné 
---
 make-flight | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/make-flight b/make-flight
index b8942c1c..85559c68 100755
--- a/make-flight
+++ b/make-flight
@@ -903,7 +903,7 @@ test_matrix_do_one () {
   job_create_test test-$xenarch$kern-$dom0arch-dom0pvh-xl-$cpuvendor \
 test-debian xl $xenarch $dom0arch $debian_runvars \
 all_hostflags=$most_hostflags,hvm-$cpuvendor,iommu \
-xen_boot_append='dom0=pvh,verbose'
+xen_boot_append='dom0=pvh,verbose' dom0_mem=1024
 
 done
 
-- 
2.27.0




[PATCH] x86/svm: Misc coding style corrections

2020-07-22 Thread Andrew Cooper
No functional change.

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
CC: Wei Liu 
CC: Roger Pau Monné 

These almost certainly aren't all the style issues, but the end result is
certainly far more consistent.
---
 xen/arch/x86/hvm/svm/intr.c  |  19 ++-
 xen/arch/x86/hvm/svm/nestedsvm.c | 291 +++
 xen/arch/x86/hvm/svm/svm.c   |  76 +-
 3 files changed, 225 insertions(+), 161 deletions(-)

diff --git a/xen/arch/x86/hvm/svm/intr.c b/xen/arch/x86/hvm/svm/intr.c
index 38011bd4e2..7f815d2307 100644
--- a/xen/arch/x86/hvm/svm/intr.c
+++ b/xen/arch/x86/hvm/svm/intr.c
@@ -1,6 +1,6 @@
 /*
  * intr.c: Interrupt handling for SVM.
- * Copyright (c) 2005, AMD Inc. 
+ * Copyright (c) 2005, AMD Inc.
  * Copyright (c) 2004, Intel Corporation.
  *
  * This program is free software; you can redistribute it and/or modify it
@@ -83,9 +83,12 @@ static void svm_enable_intr_window(struct vcpu *v, struct 
hvm_intack intack)
 
 ASSERT(intack.source != hvm_intsrc_none);
 
-if ( nestedhvm_enabled(v->domain) ) {
+if ( nestedhvm_enabled(v->domain) )
+{
 struct nestedvcpu *nv = _nestedhvm(v);
-if ( nv->nv_vmentry_pending ) {
+
+if ( nv->nv_vmentry_pending )
+{
 struct vmcb_struct *gvmcb = nv->nv_vvmcx;
 
 /* check if l1 guest injects interrupt into l2 guest via vintr.
@@ -131,7 +134,7 @@ static void svm_enable_intr_window(struct vcpu *v, struct 
hvm_intack intack)
 vmcb, general1_intercepts | GENERAL1_INTERCEPT_VINTR);
 }
 
-void svm_intr_assist(void) 
+void svm_intr_assist(void)
 {
 struct vcpu *v = current;
 struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
@@ -151,7 +154,8 @@ void svm_intr_assist(void)
 return;
 
 intblk = hvm_interrupt_blocked(v, intack);
-if ( intblk == hvm_intblk_svm_gif ) {
+if ( intblk == hvm_intblk_svm_gif )
+{
 ASSERT(nestedhvm_enabled(v->domain));
 return;
 }
@@ -167,10 +171,11 @@ void svm_intr_assist(void)
  * the l1 guest occurred.
  */
 rc = nestedsvm_vcpu_interrupt(v, intack);
-switch (rc) {
+switch ( rc )
+{
 case NSVM_INTR_NOTINTERCEPTED:
 /* Inject interrupt into 2nd level guest directly. */
-break; 
+break;
 case NSVM_INTR_NOTHANDLED:
 case NSVM_INTR_FORCEVMEXIT:
 return;
diff --git a/xen/arch/x86/hvm/svm/nestedsvm.c b/xen/arch/x86/hvm/svm/nestedsvm.c
index a193d9de45..fcfccf75df 100644
--- a/xen/arch/x86/hvm/svm/nestedsvm.c
+++ b/xen/arch/x86/hvm/svm/nestedsvm.c
@@ -30,7 +30,7 @@
 
 #define NSVM_ERROR_VVMCB1
 #define NSVM_ERROR_VMENTRY  2
- 
+
 static void
 nestedsvm_vcpu_clgi(struct vcpu *v)
 {
@@ -51,7 +51,8 @@ int nestedsvm_vmcb_map(struct vcpu *v, uint64_t vmcbaddr)
 {
 struct nestedvcpu *nv = _nestedhvm(v);
 
-if (nv->nv_vvmcx != NULL && nv->nv_vvmcxaddr != vmcbaddr) {
+if ( nv->nv_vvmcx != NULL && nv->nv_vvmcxaddr != vmcbaddr )
+{
 ASSERT(vvmcx_valid(v));
 hvm_unmap_guest_frame(nv->nv_vvmcx, 1);
 nv->nv_vvmcx = NULL;
@@ -87,24 +88,24 @@ int nsvm_vcpu_initialise(struct vcpu *v)
 
 msrpm = alloc_xenheap_pages(get_order_from_bytes(MSRPM_SIZE), 0);
 svm->ns_cached_msrpm = msrpm;
-if (msrpm == NULL)
+if ( msrpm == NULL )
 goto err;
 memset(msrpm, 0x0, MSRPM_SIZE);
 
 msrpm = alloc_xenheap_pages(get_order_from_bytes(MSRPM_SIZE), 0);
 svm->ns_merged_msrpm = msrpm;
-if (msrpm == NULL)
+if ( msrpm == NULL )
 goto err;
 memset(msrpm, 0x0, MSRPM_SIZE);
 
 nv->nv_n2vmcx = alloc_vmcb();
-if (nv->nv_n2vmcx == NULL)
+if ( nv->nv_n2vmcx == NULL )
 goto err;
 nv->nv_n2vmcx_pa = virt_to_maddr(nv->nv_n2vmcx);
 
 return 0;
 
-err:
+ err:
 nsvm_vcpu_destroy(v);
 return -ENOMEM;
 }
@@ -120,28 +121,33 @@ void nsvm_vcpu_destroy(struct vcpu *v)
  * in order to avoid double free of l2 vmcb and the possible memory leak
  * of l1 vmcb page.
  */
-if (nv->nv_n1vmcx)
+if ( nv->nv_n1vmcx )
 v->arch.hvm.svm.vmcb = nv->nv_n1vmcx;
 
-if (svm->ns_cached_msrpm) {
+if ( svm->ns_cached_msrpm )
+{
 free_xenheap_pages(svm->ns_cached_msrpm,
get_order_from_bytes(MSRPM_SIZE));
 svm->ns_cached_msrpm = NULL;
 }
-if (svm->ns_merged_msrpm) {
+
+if ( svm->ns_merged_msrpm )
+{
 free_xenheap_pages(svm->ns_merged_msrpm,
get_order_from_bytes(MSRPM_SIZE));
 svm->ns_merged_msrpm = NULL;
 }
+
 hvm_unmap_guest_frame(nv->nv_vvmcx, 1);
 nv->nv_vvmcx = NULL;
-if (nv->nv_n2vmcx) {
+if ( nv->nv_n2vmcx )
+{
 free_vmcb(nv->nv_n2vmcx);
 nv->nv_n2vmcx = NULL;
 nv->nv_n2vmcx_pa = INVALID_PADDR;
 }
-if (svm->ns_iomap)
-

[OSSTEST PATCH 15/14] Executive: Drop redundant AND clause

2020-07-22 Thread Ian Jackson
In "Executive: Use index for report__find_test" we changed an EXISTS
subquery into a JOIN.

Now, the condition r.flight=f.flight is redundant because this is the
join column (from USING).

No functional change.

CC: George Dunlap 
Signed-off-by: Ian Jackson 
---
 Osstest/Executive.pm | 1 -
 1 file changed, 1 deletion(-)

diff --git a/Osstest/Executive.pm b/Osstest/Executive.pm
index 66c93ab9..33de3708 100644
--- a/Osstest/Executive.pm
+++ b/Osstest/Executive.pm
@@ -433,7 +433,6 @@ END
   WHERE name=?
  AND name LIKE 'revision_%'
 AND val=?
-AND r.flight=f.flight
  AND ${\ main_revision_job_cond('r.job') }
 END
 push @params, "revision_$tree", $revision;
-- 
2.20.1




Re: [OSSTEST PATCH 05/14] sg-report-flight: Use WITH to use best index use for $flightsq

2020-07-22 Thread Ian Jackson
George Dunlap writes ("Re: [OSSTEST PATCH 05/14] sg-report-flight: Use WITH to 
use best index use for $flightsq"):
> On Jul 21, 2020, at 7:41 PM, Ian Jackson  wrote:
> > After:
> >  WITH sub AS (
> >SELECT DISTINCT flight, blessing
> > FROM flights
> > JOIN runvars r1 USING (flight)
> > 
> >WHERE (branch='xen-unstable')
> >  AND ( (TRUE AND flight <= 151903) AND (blessing='real') )
> >  AND r1.name LIKE 'built_revision_%'
> >  AND r1.name = ?
> >  AND r1.val= ?
> > 
> >ORDER BY flight DESC
> >LIMIT 1000
> >  )
> >  SELECT *
> >FROM sub
> >JOIN jobs USING (flight)
> > 
> >   WHERE (1=1)
> >  AND jobs.job = ?
> > 
> >  ORDER BY blessing ASC, flight DESC
> 
> I was wondering if it would be useful converting this to a join would be 
> useful. :-)
...
> The main thing I see here is that there’s nothing *in the query*
> that guarantees you won’t get multiple flights if there are multiple
> jobs for that flight whose ‘job’ value; but given the naming scheme
> so far, I’m guessing job is unique…?  As long as there’s something
> else preventing duplication I think it’s fine.

(flight,job) is the primary key for the jobs table.

I can probably produce a schema dump if that would make reading this
stuff easier.

Ian.



Re: [OSSTEST PATCH 04/14] sg-report-flight: Ask the db for flights of interest

2020-07-22 Thread Ian Jackson
George Dunlap writes ("Re: [OSSTEST PATCH 04/14] sg-report-flight: Ask the db 
for flights of interest"):
> > On Jul 21, 2020, at 7:41 PM, Ian Jackson  wrote:
> > Example query before (from the Perl DBI trace):
> > 
> >  SELECT * FROM (
> >SELECT flight, blessing FROM flights
...
> >  AND ( (TRUE AND flight <= 151903) AND (blessing='real') )
...
> But why are we selecting ‘blessing’ from these, if we’ve specified that 
> blessing = “real”? Isn’t that redundant?

That condition is programmatically constructed.  Sometimes it will ask
for multiple different blessings and then it wants to know which.

> > After:
...
> So this says:
> 
> Find me the most 1000 recent flights
> Where:
>   branch is “xen-unstable”
>   flight <= 15903
>   blessing is “real”
>   One of its jobs is $job
>   It has a runvar matching given $name and $val
> 
> And of course it uses the ’name LIKE ‘built_revision_%’ index.

Yes.

> Still don’t understand the ’TRUE AND’ and ‘AS sub’ bits, but it
> looks to me like it’s substantially the same query, with additional
> $name = $val runvar restriction.

That's my intent, ytes.

> And given that you say, "This condition is strictly broader than
> that implemented inside the flight search loop”, I take it that it’s
> again mainly to take advantage of the new index?

Right.  The previous approach was "iterate over recent flights,
figure out precisely what they built, and decide if they meet the
(complex) requirements".

Now we only iterate over a subset of recent flights: those which have
at least one such runvar.  The big commennt is meant to be a
demonstration that the "(complex) requirements" are a narrower
condition than the new condition on the initial flights query.

So I think the result is that it will look deeper into history, and be
faster, but not otherwise change its beaviour.

Ian.



Re: [OSSTEST PATCH 08/14] Executive: Use index for report__find_test

2020-07-22 Thread Ian Jackson
George Dunlap writes ("Re: [OSSTEST PATCH 08/14] Executive: Use index for 
report__find_test"):
> > On Jul 21, 2020, at 7:41 PM, Ian Jackson  wrote:
> > Example query before (from the Perl DBI trace):
...
> So this says:
> 
> Get me all the columns
> for the highest-numbered flight
> Where:
>   There is at least one runvar for that flight has the specified $name and 
> $value
>   And the job is *not* like build-%-prev or build-%-freebsd
>   The flight number (?) is <= 151903, and blessing = real
>   For the specified $branch

Yes.

> What’s the “TRUE and flight <= 151903” for?

These queries are programmetically constructed.  In this case, the
flight condition is not always there.  My test case had a
--max-flight=151903 on the command line: this is a debugging option.
It avoids newly added stuff in the db confusing me and generally
disturbing things.  This is implemented with a condition variable
which contains either "" or "and flight <= 151903".  Doing it this way
simplifies the generation code.

> And this says (effectively)
> 
> Get me 
> From the highest-numbered flight
> Where
>   That flight has a runvar with specified name and value
>   The job *doesn’t* look like “build-%-prev” or “build-%-freebsd”
>   flight & blessing as appropriate
>   branch as specified.

I think so, yes.

> Isn’t the r.flight = f.flight redundant if we’re joining on flight?

Indeed it is.  I guess I can add a patch at theend to delete that.

> Also, in spite of the paragraph attempting to explain it, I’m afraid
> I don’t understand what the “AS sub WHERE TRUE” is for.

The reason for the subquery is not evident in the SQL.  It's because
of the Perl code which generates this query.  The same code is used to
generate queries that start with things like
   SELECT * ...
   SELECT COUNT(*) AS count ...
The perl code gets told "*" or "COUNT(*) AS count".  The call sites
that pass "*" expect to see fields from flights.  It would be
possible to change "*" to the explicit field list everywhere, but
it was much easier to do it this way.

(The WHERE TRUE is another one of these stubs where a condition might
appear.)

> But it looks like the new query should do the same thing as the old
> query, assuming that the columns from the subquery are all the
> columns that you need in the correct order.

The subquery columns are precisely the columns currently existing in
he flights table.

Thanks,
Ian.



Re: [OSSTEST PATCH 05/14] sg-report-flight: Use WITH to use best index use for $flightsq

2020-07-22 Thread George Dunlap


> On Jul 21, 2020, at 7:41 PM, Ian Jackson  wrote:
> 
> While we're here, convert this EXISTS subquery to a JOIN.
> 
> Perf: runtime of my test case now ~200-300s.
> 
> Example query before (from the Perl DBI trace):
> 
>  SELECT * FROM (
>SELECT DISTINCT flight, blessing
> FROM flights
> JOIN runvars r1 USING (flight)
> 
>WHERE (branch='xen-unstable')
>  AND ( (TRUE AND flight <= 151903) AND (blessing='real') )
>  AND EXISTS (SELECT 1
>FROM jobs
>   WHERE jobs.flight = flights.flight
> AND jobs.job = ?)
> 
>  AND r1.name LIKE 'built_revision_%'
>  AND r1.name = ?
>  AND r1.val= ?
> 
>ORDER BY flight DESC
>LIMIT 1000
>  ) AS sub
>  ORDER BY blessing ASC, flight DESC
> 
> With bind variables:
> 
> "test-armhf-armhf-libvirt"
> 'built_revision_xen'
> '165f3afbfc3db70fcfdccad07085cde0a03c858b'
> 
> After:
> 
>  WITH sub AS (
>SELECT DISTINCT flight, blessing
> FROM flights
> JOIN runvars r1 USING (flight)
> 
>WHERE (branch='xen-unstable')
>  AND ( (TRUE AND flight <= 151903) AND (blessing='real') )
>  AND r1.name LIKE 'built_revision_%'
>  AND r1.name = ?
>  AND r1.val= ?
> 
>ORDER BY flight DESC
>LIMIT 1000
>  )
>  SELECT *
>FROM sub
>JOIN jobs USING (flight)
> 
>   WHERE (1=1)
>  AND jobs.job = ?
> 
>  ORDER BY blessing ASC, flight DESC

I was wondering if it would be useful converting this to a join would be 
useful. :-)

Again, not sure what the “(1=1) AND” bit is for; something to poke the query 
planner somehow?

The main thing I see here is that there’s nothing *in the query* that 
guarantees you won’t get multiple flights if there are multiple jobs for that 
flight whose ‘job’ value; but given the naming scheme so far, I’m guessing job 
is unique…?  As long as there’s something else preventing duplication I think 
it’s fine.

 -George



Re: [OSSTEST PATCH 04/14] sg-report-flight: Ask the db for flights of interest

2020-07-22 Thread George Dunlap


> On Jul 21, 2020, at 7:41 PM, Ian Jackson  wrote:
> 
> Specifically, we narrow the initial query to flights which have at
> least some job with the built_revision_foo we are looking for.
> 
> This condition is strictly broader than that implemented inside the
> flight search loop, so there is no functional change.
> 
> Perf: runtime of my test case now ~300s-500s.
> 
> Example query before (from the Perl DBI trace):
> 
>  SELECT * FROM (
>SELECT flight, blessing FROM flights
>WHERE (branch='xen-unstable')
>  AND   EXISTS (SELECT 1
>FROM jobs
>   WHERE jobs.flight = flights.flight
> AND jobs.job = ?)
> 
>  AND ( (TRUE AND flight <= 151903) AND (blessing='real') )
>ORDER BY flight DESC
>LIMIT 1000
>  ) AS sub
>  ORDER BY blessing ASC, flight DESC

This one says:

Find the 1000 most recent flights
Where 
  branch is "xen-unstable”
  one of its jobs is $job
  And blessing is “real”

But why are we selecting ‘blessing’ from these, if we’ve specified that 
blessing = “real”? Isn’t that redundant?

> 
> With these bind variables:
> 
>"test-armhf-armhf-libvirt"
> 
> After:
> 
>  SELECT * FROM (
>SELECT DISTINCT flight, blessing
> FROM flights
> JOIN runvars r1 USING (flight)
> 
>WHERE (branch='xen-unstable')
>  AND ( (TRUE AND flight <= 151903) AND (blessing='real') )
>  AND EXISTS (SELECT 1
>FROM jobs
>   WHERE jobs.flight = flights.flight
> AND jobs.job = ?)
> 
>  AND r1.name LIKE 'built_revision_%'
>  AND r1.name = ?
>  AND r1.val= ?
> 
>ORDER BY flight DESC
>LIMIT 1000
>  ) AS sub
>  ORDER BY blessing ASC, flight DESC

So this says:

Find me the most 1000 recent flights
Where:
  branch is “xen-unstable”
  flight <= 15903
  blessing is “real”
  One of its jobs is $job
  It has a runvar matching given $name and $val

And of course it uses the ’name LIKE ‘built_revision_%’ index.

Still don’t understand the ’TRUE AND’ and ‘AS sub’ bits, but it looks to me 
like it’s substantially the same query, with additional $name = $val runvar 
restriction.

And given that you say, "This condition is strictly broader than that 
implemented inside the flight search loop”, I take it that it’s again mainly to 
take advantage of the new index?

 -George

[xen-4.14-testing test] 152081: tolerable trouble: fail/pass/starved - PUSHED

2020-07-22 Thread osstest service owner
flight 152081 xen-4.14-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/152081/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop fail never pass
 test-amd64-amd64-qemuu-freebsd12-amd64  2 hosts-allocate   starved n/a
 test-amd64-amd64-qemuu-freebsd11-amd64  2 hosts-allocate   starved n/a

version targeted for testing:
 xen  827031adfeb3c2656baa2156d3e7caaea8aec739
baseline version:
 xen  23fe1b8d5170dfd5039c39181e82bfd5e20f3c18

Last test of basis   152043  2020-07-20 12:10:42 Z1 days
Failing since152061  2020-07-21 01:41:43 Z1 days2 attempts
Testing same since   152081  2020-07-21 16:52:47 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Julien Grall 

jobs:
 

Re: [PATCH] x86/svm: Fold nsvm_{wr,rd}msr() into svm_msr_{read,write}_intercept()

2020-07-22 Thread Jan Beulich
On 22.07.2020 12:10, Andrew Cooper wrote:
> On 22/07/2020 10:16, Jan Beulich wrote:
>> On 21.07.2020 19:22, Andrew Cooper wrote:
>>> ... to simplify the default cases.
>>>
>>> There are multiple errors with the handling of these three MSRs, but they 
>>> are
>>> deliberately not addressed this point.
>>>
>>> This removes the dance converting -1/0/1 into X86EMUL_*, allowing for the
>>> removal of the 'ret' variable.
>>>
>>> While cleaning this up, drop the gdprintk()'s for #GP conditions, and the
>>> 'result' variable from svm_msr_write_intercept() is it is never modified.
>>>
>>> No functional change.
>>>
>>> Signed-off-by: Andrew Cooper 
>> Reviewed-by: Jan Beulich 
>>
>> However, ...
>>
>>> @@ -2085,6 +2091,22 @@ static int svm_msr_write_intercept(unsigned int msr, 
>>> uint64_t msr_content)
>>>  goto gpf;
>>>  break;
>>>  
>>> +case MSR_K8_VM_CR:
>>> +/* ignore write. handle all bits as read-only. */
>>> +break;
>>> +
>>> +case MSR_K8_VM_HSAVE_PA:
>>> +if ( (msr_content & ~PAGE_MASK) || msr_content > 0xfdULL )
>> ... while you're moving this code here, wouldn't it be worthwhile
>> to at least fix the > to be >= ?
> 
> I'd prefer not to, to avoid breaking the "No Functional Change" aspect.

Well, so be it then.

> In reality, this needs to be a path which takes an extra ref on the
> nominated frame and globally maps it, seeing as we memcpy to/from it on
> every virtual vmentry/exit.  The check against the HT range is quite bogus.

I agree; I merely found the > so very obviously off-by-one that I
thought I'd at least inquire.

Jan



[qemu-mainline test] 152076: regressions - trouble: fail/pass/starved

2020-07-22 Thread osstest service owner
flight 152076 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/152076/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail REGR. vs. 
151065
 test-amd64-amd64-xl-qemuu-debianhvm-amd64 10 debian-hvm-install fail REGR. vs. 
151065
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail REGR. vs. 151065
 test-amd64-amd64-qemuu-nested-intel 10 debian-hvm-install fail REGR. vs. 151065
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 10 debian-hvm-install fail REGR. 
vs. 151065
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail REGR. vs. 151065
 test-amd64-i386-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail REGR. vs. 151065
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 10 debian-hvm-install fail REGR. 
vs. 151065
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 10 debian-hvm-install fail 
REGR. vs. 151065
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 debian-hvm-install fail 
REGR. vs. 151065
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 debian-hvm-install fail 
REGR. vs. 151065
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 10 debian-hvm-install fail 
REGR. vs. 151065
 test-amd64-i386-xl-qemuu-debianhvm-amd64 10 debian-hvm-install fail REGR. vs. 
151065
 test-amd64-i386-qemuu-rhel6hvm-intel 10 redhat-install   fail REGR. vs. 151065
 test-amd64-i386-freebsd10-i386 11 guest-startfail REGR. vs. 151065
 test-amd64-i386-freebsd10-amd64 11 guest-start   fail REGR. vs. 151065
 test-amd64-amd64-qemuu-nested-amd 10 debian-hvm-install  fail REGR. vs. 151065
 test-amd64-i386-qemuu-rhel6hvm-amd 10 redhat-install fail REGR. vs. 151065
 test-amd64-amd64-xl-qemuu-win7-amd64 10 windows-install  fail REGR. vs. 151065
 test-amd64-i386-xl-qemuu-win7-amd64 10 windows-install   fail REGR. vs. 151065
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-install  fail REGR. vs. 151065
 test-amd64-i386-xl-qemuu-ws16-amd64 10 windows-install   fail REGR. vs. 151065

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds 12 guest-start  fail REGR. vs. 151065

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 151065
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 151065
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-arm64-arm64-xl-seattle  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-check

Re: [osstest PATCH] freebsd: remove freebsd- hostflags request from guest tests

2020-07-22 Thread Ian Jackson
Roger Pau Monne writes ("[osstest PATCH] freebsd: remove freebsd- hostflags 
request from guest tests"):
> Guest tests shouldn't care about the capabilities or firmware of the
> underlying hosts, so drop the request of specific freebsd-
> hostflags for FreeBSD guest tests.
> 
> While there request the presence of the hvm hostflag since the FreeBSD
> guest tests are run in HVM mode.
> 
> Signed-off-by: Roger Pau Monné 

Acked-by: Ian Jackson 

I have queued this for the next push to pretest which I hope to do
some time today.

Thanks,
Ian.



Re: Virtio in Xen on Arm (based on IOREQ concept)

2020-07-22 Thread Roger Pau Monné
On Wed, Jul 22, 2020 at 12:17:26PM +0100, Julien Grall wrote:
> 
> 
> On 22/07/2020 12:10, Roger Pau Monné wrote:
> > On Wed, Jul 22, 2020 at 11:47:18AM +0100, Julien Grall wrote:
> > > > 
> > > > You can still use the map-on-fault behaviour as above, but I would
> > > > recommend that you try to limit the number of hypercalls issued.
> > > > Having to issue a single hypercall for each page fault it's going to
> > > > be slow, so I would instead use mmap batch to map the hole range in
> > > > unpopulated physical memory and then the OS fault handler just needs to
> > > > fill the page tables with the corresponding address.
> > > IIUC your proposal, you are assuming that you will have enough free space 
> > > in
> > > the physical address space to map the foreign mapping.
> > > 
> > > However that amount of free space is not unlimited and may be quite small
> > > (see above). It would be fairly easy to exhaust it given that a userspace
> > > application can map many times the same guest physical address.
> > > 
> > > So I still think we need to be able to allow Linux to swap a foreign page
> > > with another page.
> > 
> > Right, but you will have to be careful to make sure physical addresses
> > are not swapped while being used for IO with devices, as in that case
> > you won't get a recoverable fault. This is safe now because physical
> > mappings created by privcmd are never swapped out, but if you go the
> > route you propose you will have to figure a way to correctly populate
> > physical ranges used for IO with devices, even when the CPU hasn't
> > accessed them.
> > 
> > Relying solely on CPU page faults to populate them will not be enough,
> > as the CPU won't necessarily access all the pages that would be send
> > to devices for IO.
> 
> The problem you described here doesn't seem to be specific to foreign
> mapping. So I would really be surprised if Linux doesn't already have
> generic mechanism to deal with this.

Right, Linux will pre-fault and lock the pages before using them for
IO, and unlock them afterwards, in which case it should be safe.

> Hence why I suggested before to deal with foreign mapping the same way as
> Linux would do with user memory.

Should work, on FreeBSD privcmd I also populate the pages in the page
fault handler, but the hypercall to create the foreign mappings is
executed only once when the ioctl is issued.

Roger.



Re: [OSSTEST PATCH 08/14] Executive: Use index for report__find_test

2020-07-22 Thread George Dunlap


> On Jul 21, 2020, at 7:41 PM, Ian Jackson  wrote:
> 
> After we refactor this query then we can enable the index use.
> (Both of these things together in this commit because I haven't perf
> tested the version with just the refactoring.)
> 
> (We have provided an index that can answer this question really
> quickly if a version is specified.  But the query planner couldn't see
> that because it works without seeing the bind variables, so doesn't
> know that the value of name is going to be suitable for this index.)
> 
> * Convert the two EXISTS subqueries into JOIN/AND with a DISTINCT
>  clause naming the fields on flights, so as to replicate the previous
>  result rows.  Then do $selection field last.  The subquery is a
>  convenient way to let this do the previous thing for all the values
>  of $selection (including, notably, *).
> 
> * Add the additional AND clause for r.name, which has no logical
>  effect given the actual values of name, enabling the query planner
>  to use this index.
> 
> Perf: In my test case the sg-report-flight runtime is now ~8s.  I am
> reasonably confident that this will not make other use cases of this
> code worse.
> 
> Perf: runtime of my test case now ~11s
> 
> Example query before (from the Perl DBI trace):
> 
>SELECT *
> FROM flights f
>WHERE
>EXISTS (
>   SELECT 1
>FROM runvars r
>   WHERE name=?
> AND val=?
> AND r.flight=f.flight
> AND (  (CASE
>   WHEN (r.job) LIKE 'build-%-prev' THEN 'xprev'
>   WHEN ((r.job) LIKE 'build-%-freebsd'
> AND 'x' = 'freebsdbuildjob') THEN 'DISCARD'
>   ELSE  ''
>   END)
> = '')
> )
>  AND ( (TRUE AND flight <= 151903) AND (blessing='real') )
>  AND (branch=?)
>ORDER BY flight DESC
>LIMIT 1

So this says:

Get me all the columns
for the highest-numbered flight
Where:
  There is at least one runvar for that flight has the specified $name and 
$value
  And the job is *not* like build-%-prev or build-%-freebsd
  The flight number (?) is <= 151903, and blessing = real
  For the specified $branch

What’s the “TRUE and flight <= 151903” for?

> 
> After:
> 
>SELECT *
>  FROM ( SELECT DISTINCT
>  flight, started, blessing, branch, intended
> FROM flights f
>JOIN runvars r USING (flight)
>   WHERE name=?
> AND name LIKE 'revision_%'
> AND val=?
> AND r.flight=f.flight
> AND (  (CASE
>   WHEN (r.job) LIKE 'build-%-prev' THEN 'xprev'
>   WHEN ((r.job) LIKE 'build-%-freebsd'
> AND 'x' = 'freebsdbuildjob') THEN 'DISCARD'
>   ELSE  ''
>   END)
> = '')
>  AND ( (TRUE AND flight <= 151903) AND (blessing='real') )
>  AND (branch=?)
> ) AS sub WHERE TRUE
>ORDER BY flight DESC
>LIMIT 1

And this says (effectively)

Get me 
From the highest-numbered flight
Where
  That flight has a runvar with specified name and value
  The job *doesn’t* look like “build-%-prev” or “build-%-freebsd”
  flight & blessing as appropriate
  branch as specified.
  
Isn’t the r.flight = f.flight redundant if we’re joining on flight?

Also, in spite of the paragraph attempting to explain it, I’m afraid I don’t 
understand what the “AS sub WHERE TRUE” is for.

But it looks like the new query should do the same thing as the old query, 
assuming that the columns from the subquery are all the columns that you need 
in the correct order.

 -George



[PATCH v2] tools/configure: drop BASH configure variable

2020-07-22 Thread Andrew Cooper
This is a weird variable to have in the first place.  The only user of it is
XSM's CONFIG_SHELL, which opencodes a fallback to sh, and the only two scripts
run with this are shebang sh anyway, so don't need bash in the first place.

Make the mkflask.sh and mkaccess_vector.sh scripts executable, drop the
CONFIG_SHELL, and drop the $BASH variable to prevent further use.

Signed-off-by: Andrew Cooper 
---
CC: Ian Jackson 
CC: Wei Liu 
CC: Daniel De Graaf 

./autogen.sh needs rerunning on commit.

v2:
 * Use $(SHELL)

There is a separate check for [BASH] in the configure script, which is
checking the requirement for the Linux hotplug scripts.  Really, that is a
runtime requirement not a build time requirement, and it is rude to require
bash in a build environment on this basis.  IMO, that check wants dropping as
well.
---
 config/Tools.mk.in  | 1 -
 tools/configure.ac  | 1 -
 xen/xsm/flask/Makefile  | 8 ++--
 xen/xsm/flask/policy/mkaccess_vector.sh | 0
 xen/xsm/flask/policy/mkflask.sh | 0
 5 files changed, 2 insertions(+), 8 deletions(-)
 mode change 100644 => 100755 xen/xsm/flask/policy/mkaccess_vector.sh
 mode change 100644 => 100755 xen/xsm/flask/policy/mkflask.sh

diff --git a/config/Tools.mk.in b/config/Tools.mk.in
index 23df47af8d..48bd9ab731 100644
--- a/config/Tools.mk.in
+++ b/config/Tools.mk.in
@@ -12,7 +12,6 @@ PYTHON  := @PYTHON@
 PYTHON_PATH := @PYTHONPATH@
 PY_NOOPT_CFLAGS := @PY_NOOPT_CFLAGS@
 PERL:= @PERL@
-BASH:= @BASH@
 XGETTTEXT   := @XGETTEXT@
 AS86:= @AS86@
 LD86:= @LD86@
diff --git a/tools/configure.ac b/tools/configure.ac
index 9d126b7a14..6614a4f130 100644
--- a/tools/configure.ac
+++ b/tools/configure.ac
@@ -297,7 +297,6 @@ AC_ARG_VAR([PYTHON], [Path to the Python parser])
 AC_ARG_VAR([PERL], [Path to Perl parser])
 AC_ARG_VAR([BISON], [Path to Bison parser generator])
 AC_ARG_VAR([FLEX], [Path to Flex lexical analyser generator])
-AC_ARG_VAR([BASH], [Path to bash shell])
 AC_ARG_VAR([XGETTEXT], [Path to xgetttext tool])
 AC_ARG_VAR([AS86], [Path to as86 tool])
 AC_ARG_VAR([LD86], [Path to ld86 tool])
diff --git a/xen/xsm/flask/Makefile b/xen/xsm/flask/Makefile
index 07f36d075d..50bec20a1e 100644
--- a/xen/xsm/flask/Makefile
+++ b/xen/xsm/flask/Makefile
@@ -8,10 +8,6 @@ CFLAGS-y += -I./include
 
 AWK = awk
 
-CONFIG_SHELL := $(shell if [ -x "$$BASH" ]; then echo $$BASH; \
-  else if [ -x /bin/bash ]; then echo /bin/bash; \
-  else echo sh; fi ; fi)
-
 FLASK_H_DEPEND = policy/security_classes policy/initial_sids
 AV_H_DEPEND = policy/access_vectors
 
@@ -24,14 +20,14 @@ extra-y += $(ALL_H_FILES)
 
 mkflask := policy/mkflask.sh
 quiet_cmd_mkflask = MKFLASK $@
-cmd_mkflask = $(CONFIG_SHELL) $(mkflask) $(AWK) include $(FLASK_H_DEPEND)
+cmd_mkflask = $(SHELL) $(mkflask) $(AWK) include $(FLASK_H_DEPEND)
 
 $(subst include/,%/,$(FLASK_H_FILES)): $(FLASK_H_DEPEND) $(mkflask) FORCE
$(call if_changed,mkflask)
 
 mkaccess := policy/mkaccess_vector.sh
 quiet_cmd_mkaccess = MKACCESS VECTOR $@
-cmd_mkaccess = $(CONFIG_SHELL) $(mkaccess) $(AWK) $(AV_H_DEPEND)
+cmd_mkaccess = $(SHELL) $(mkaccess) $(AWK) $(AV_H_DEPEND)
 
 $(subst include/,%/,$(AV_H_FILES)): $(AV_H_DEPEND) $(mkaccess) FORCE
$(call if_changed,mkaccess)
diff --git a/xen/xsm/flask/policy/mkaccess_vector.sh 
b/xen/xsm/flask/policy/mkaccess_vector.sh
old mode 100644
new mode 100755
diff --git a/xen/xsm/flask/policy/mkflask.sh b/xen/xsm/flask/policy/mkflask.sh
old mode 100644
new mode 100755
-- 
2.11.0




Re: Virtio in Xen on Arm (based on IOREQ concept)

2020-07-22 Thread Julien Grall




On 22/07/2020 12:10, Roger Pau Monné wrote:

On Wed, Jul 22, 2020 at 11:47:18AM +0100, Julien Grall wrote:


You can still use the map-on-fault behaviour as above, but I would
recommend that you try to limit the number of hypercalls issued.
Having to issue a single hypercall for each page fault it's going to
be slow, so I would instead use mmap batch to map the hole range in
unpopulated physical memory and then the OS fault handler just needs to
fill the page tables with the corresponding address.

IIUC your proposal, you are assuming that you will have enough free space in
the physical address space to map the foreign mapping.

However that amount of free space is not unlimited and may be quite small
(see above). It would be fairly easy to exhaust it given that a userspace
application can map many times the same guest physical address.

So I still think we need to be able to allow Linux to swap a foreign page
with another page.


Right, but you will have to be careful to make sure physical addresses
are not swapped while being used for IO with devices, as in that case
you won't get a recoverable fault. This is safe now because physical
mappings created by privcmd are never swapped out, but if you go the
route you propose you will have to figure a way to correctly populate
physical ranges used for IO with devices, even when the CPU hasn't
accessed them.

Relying solely on CPU page faults to populate them will not be enough,
as the CPU won't necessarily access all the pages that would be send
to devices for IO.


The problem you described here doesn't seem to be specific to foreign 
mapping. So I would really be surprised if Linux doesn't already have 
generic mechanism to deal with this.


Hence why I suggested before to deal with foreign mapping the same way 
as Linux would do with user memory.


Cheers,

--
Julien Grall



Re: Virtio in Xen on Arm (based on IOREQ concept)

2020-07-22 Thread Roger Pau Monné
On Wed, Jul 22, 2020 at 11:47:18AM +0100, Julien Grall wrote:
> Hi Roger,
> 
> On 22/07/2020 09:21, Roger Pau Monné wrote:
> > On Tue, Jul 21, 2020 at 10:12:40PM +0100, Julien Grall wrote:
> > > Hi Oleksandr,
> > > 
> > > On 21/07/2020 19:16, Oleksandr wrote:
> > > > 
> > > > On 21.07.20 17:27, Julien Grall wrote:
> > > > > On a similar topic, I am a bit surprised you didn't encounter memory
> > > > > exhaustion when trying to use virtio. Because on how Linux currently
> > > > > works (see XSA-300), the backend domain as to have a least as much
> > > > > RAM as the domain it serves. For instance, you have serve two
> > > > > domains with 1GB of RAM each, then your backend would need at least
> > > > > 2GB + some for its own purpose.
> > > > > 
> > > > > This probably wants to be resolved by allowing foreign mapping to be
> > > > > "paging" out as you would for memory assigned to a userspace.
> > > > 
> > > > Didn't notice the last sentence initially. Could you please explain your
> > > > idea in detail if possible. Does it mean if implemented it would be
> > > > feasible to map all guest memory regardless of how much memory the guest
> > > > has?
> > > > 
> > > > Avoiding map/unmap memory each guest request would allow us to have
> > > > better performance (of course with taking care of the fact that guest
> > > > memory layout could be changed)...
> > > 
> > > I will explain that below. Before let me comment on KVM first.
> > > 
> > > > Actually what I understand looking at kvmtool is the fact it does not
> > > > map/unmap memory dynamically, just calculate virt addresses according to
> > > > the gfn provided.
> > > 
> > > The memory management between KVM and Xen is quite different. In the case 
> > > of
> > > KVM, the guest RAM is effectively memory from the userspace (allocated via
> > > mmap) and then shared with the guest.
> > > 
> > >  From the userspace PoV, the guest memory will always be accessible from 
> > > the
> > > same virtual region. However, behind the scene, the pages may not always
> > > reside in memory. They are basically managed the same way as "normal"
> > > userspace memory.
> > > 
> > > In the case of Xen, we are basically stealing a guest physical page
> > > allocated via kmalloc() and provide no facilities for Linux to reclaim the
> > > page if it needs to do it before the userspace decide to unmap the foreign
> > > mapping.
> > > 
> > > I think it would be good to handle the foreing mapping the same way as
> > > userspace memory. By that I mean, that Linux could reclaim the physical 
> > > page
> > > used by the foreing mapping if it needs to.
> > > 
> > > The process for reclaiming the page would look like:
> > >  1) Unmap the foreign page
> > >  2) Ballon in the backend domain physical address used by the foreing
> > > mapping (allocate the page in the physmap)
> > > 
> > > The next time the userspace is trying to access the foreign page, Linux 
> > > will
> > > receive a data abort that would result to:
> > >  1) Allocate a backend domain physical page
> > >  2) Balloon out the physical address (remove the page from the 
> > > physmap)
> > >  3) Map the foreing mapping at the new guest physical address
> > >  4) Map the guest physical page in the userspace address space
> > 
> > This is going to shatter all the super pages in the stage-2
> > translation.
> 
> Yes, but this is nothing really new as ballooning would result to (AFAICT)
> the same behavior on Linux.
> 
> > 
> > > With this approach, we should be able to have backend domain that can 
> > > handle
> > > frontend domain without require a lot of memory.
> > 
> > Linux on x86 has the option to use empty hotplug memory ranges to map
> > foreign memory: the balloon driver hotplugs an unpopulated physical
> > memory range that's not made available to the OS free memory allocator
> > and it's just used as scratch space to map foreign memory. Not sure
> > whether Arm has something similar, or if it could be implemented.
> 
> We already discussed that last year :). This was attempted in the past (I
> was still at Citrix) and indefinitely paused for Arm.
> 
> /proc/iomem can be incomplete on Linux if we didn't load a driver for all
> the devices. This means that Linux doesn't have the full view of what is
> physical range is freed.
> 
> Additionally, in the case of Dom0, all the regions corresponding to the host
> RAM are unusable when using the SMMU. This is because we would do 1:1
> mapping for the foreign mapping as well.

Right, that's a PITA because on x86 PVH dom0 I was planning to use
those RAM regions as scratch space for foreign mapping lacking a
better alternative ATM.

> It might be possible to take advantage of the direct mapping property if
> Linux do some bookeeping. Although, this wouldn't work for 32-bit Dom0 using
> short page tables (e.g some version of Debian does) as it may not be able to
> access all the host RAM. Whether we still care about is a different
> situation :).
> 
> 

[ovmf test] 152088: all pass - PUSHED

2020-07-22 Thread osstest service owner
flight 152088 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/152088/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf 9132a31b9c8381197eee75eb66c809182b264110
baseline version:
 ovmf 02539e900854488343a1efa435d4dded1ddd66a2

Last test of basis   152068  2020-07-21 07:11:01 Z1 days
Testing same since   152088  2020-07-21 23:41:49 Z0 days1 attempts


People who touched revisions under test:
  Jeff Brasen 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   02539e9008..9132a31b9c  9132a31b9c8381197eee75eb66c809182b264110 -> 
xen-tested-master



[PATCH] x86/msr: Unify the real {rd, wr}msr() paths in guest_{rd, wr}msr()

2020-07-22 Thread Andrew Cooper
Both the read and write side have commonalities which can be abstracted away.
This also allows for additional safety in release builds, and slightly more
helpful diagnostics in debug builds.

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
CC: Wei Liu 
CC: Roger Pau Monné 

I'm not a massive fan of the global scope want_rdmsr_safe boolean, but I can't
think of a reasonable way to fix it without starting to use other
flexibiltiies offered to us by C99.  (And to preempt the other question, an
extra set of braces makes extremely confusing to read logic.)
---
 xen/arch/x86/msr.c | 54 ++
 1 file changed, 42 insertions(+), 12 deletions(-)

diff --git a/xen/arch/x86/msr.c b/xen/arch/x86/msr.c
index 22f921cc71..68f3aadeab 100644
--- a/xen/arch/x86/msr.c
+++ b/xen/arch/x86/msr.c
@@ -154,6 +154,7 @@ int guest_rdmsr(struct vcpu *v, uint32_t msr, uint64_t *val)
 const struct cpuid_policy *cp = d->arch.cpuid;
 const struct msr_policy *mp = d->arch.msr;
 const struct vcpu_msrs *msrs = v->arch.msrs;
+bool want_rdmsr_safe = false;
 int ret = X86EMUL_OKAY;
 
 switch ( msr )
@@ -204,10 +205,9 @@ int guest_rdmsr(struct vcpu *v, uint32_t msr, uint64_t 
*val)
  */
 if ( !(cp->x86_vendor & (X86_VENDOR_INTEL | X86_VENDOR_AMD)) ||
  !(boot_cpu_data.x86_vendor &
-   (X86_VENDOR_INTEL | X86_VENDOR_AMD)) ||
- rdmsr_safe(MSR_AMD_PATCHLEVEL, *val) )
+   (X86_VENDOR_INTEL | X86_VENDOR_AMD)) )
 goto gp_fault;
-break;
+goto read_from_hw_safe;
 
 case MSR_SPEC_CTRL:
 if ( !cp->feat.ibrsb )
@@ -278,7 +278,7 @@ int guest_rdmsr(struct vcpu *v, uint32_t msr, uint64_t *val)
  */
 #ifdef CONFIG_HVM
 if ( v == current && is_hvm_domain(d) && v->arch.hvm.flag_dr_dirty )
-rdmsrl(msr, *val);
+goto read_from_hw;
 else
 #endif
 *val = msrs->dr_mask[
@@ -303,6 +303,23 @@ int guest_rdmsr(struct vcpu *v, uint32_t msr, uint64_t 
*val)
 
 return ret;
 
+ read_from_hw_safe:
+want_rdmsr_safe = true;
+ read_from_hw:
+if ( !rdmsr_safe(msr, *val) )
+return X86EMUL_OKAY;
+
+/*
+ * Paths which didn't want rdmsr_safe() and get here took a #GP fault.
+ * Something is broken with the above logic, so make it obvious in debug
+ * builds, and fail safe by handing #GP back to guests in release builds.
+ */
+if ( !want_rdmsr_safe )
+{
+gprintk(XENLOG_ERR, "Bad rdmsr %#x\n", msr);
+ASSERT_UNREACHABLE();
+}
+
  gp_fault:
 return X86EMUL_EXCEPTION;
 }
@@ -402,9 +419,7 @@ int guest_wrmsr(struct vcpu *v, uint32_t msr, uint64_t val)
 if ( val & ~PRED_CMD_IBPB )
 goto gp_fault; /* Rsvd bit set? */
 
-if ( v == curr )
-wrmsrl(MSR_PRED_CMD, val);
-break;
+goto maybe_write_to_hw;
 
 case MSR_FLUSH_CMD:
 if ( !cp->feat.l1d_flush )
@@ -413,9 +428,7 @@ int guest_wrmsr(struct vcpu *v, uint32_t msr, uint64_t val)
 if ( val & ~FLUSH_CMD_L1D )
 goto gp_fault; /* Rsvd bit set? */
 
-if ( v == curr )
-wrmsrl(MSR_FLUSH_CMD, val);
-break;
+goto maybe_write_to_hw;
 
 case MSR_INTEL_MISC_FEATURES_ENABLES:
 {
@@ -493,8 +506,8 @@ int guest_wrmsr(struct vcpu *v, uint32_t msr, uint64_t val)
? 0 : (msr - MSR_AMD64_DR1_ADDRESS_MASK + 1),
ARRAY_SIZE(msrs->dr_mask))] = val;
 
-if ( v == curr && (curr->arch.dr7 & DR7_ACTIVE_MASK) )
-wrmsrl(msr, val);
+if ( curr->arch.dr7 & DR7_ACTIVE_MASK )
+goto maybe_write_to_hw;
 break;
 
 default:
@@ -509,6 +522,23 @@ int guest_wrmsr(struct vcpu *v, uint32_t msr, uint64_t val)
 
 return ret;
 
+ maybe_write_to_hw:
+/*
+ * All paths potentially updating a value in hardware need to check
+ * whether the call is in current context or not, so the logic is
+ * implemented here.  Remote context need do nothing more.
+ */
+if ( v != curr || !wrmsr_safe(msr, val) )
+return X86EMUL_OKAY;
+
+/*
+ * Paths which end up here took a #GP fault in wrmsr_safe().  Something is
+ * broken with the logic above, so make it obvious in debug builds, and
+ * fail safe by handing #GP back to the guests in release builds.
+ */
+gprintk(XENLOG_ERR, "Bad wrmsr %#x val %016"PRIx64"\n", msr, val);
+ASSERT_UNREACHABLE();
+
  gp_fault:
 return X86EMUL_EXCEPTION;
 }
-- 
2.11.0




Re: Virtio in Xen on Arm (based on IOREQ concept)

2020-07-22 Thread Julien Grall

Hi Roger,

On 22/07/2020 09:21, Roger Pau Monné wrote:

On Tue, Jul 21, 2020 at 10:12:40PM +0100, Julien Grall wrote:

Hi Oleksandr,

On 21/07/2020 19:16, Oleksandr wrote:


On 21.07.20 17:27, Julien Grall wrote:

On a similar topic, I am a bit surprised you didn't encounter memory
exhaustion when trying to use virtio. Because on how Linux currently
works (see XSA-300), the backend domain as to have a least as much
RAM as the domain it serves. For instance, you have serve two
domains with 1GB of RAM each, then your backend would need at least
2GB + some for its own purpose.

This probably wants to be resolved by allowing foreign mapping to be
"paging" out as you would for memory assigned to a userspace.


Didn't notice the last sentence initially. Could you please explain your
idea in detail if possible. Does it mean if implemented it would be
feasible to map all guest memory regardless of how much memory the guest
has?

Avoiding map/unmap memory each guest request would allow us to have
better performance (of course with taking care of the fact that guest
memory layout could be changed)...


I will explain that below. Before let me comment on KVM first.


Actually what I understand looking at kvmtool is the fact it does not
map/unmap memory dynamically, just calculate virt addresses according to
the gfn provided.


The memory management between KVM and Xen is quite different. In the case of
KVM, the guest RAM is effectively memory from the userspace (allocated via
mmap) and then shared with the guest.

 From the userspace PoV, the guest memory will always be accessible from the
same virtual region. However, behind the scene, the pages may not always
reside in memory. They are basically managed the same way as "normal"
userspace memory.

In the case of Xen, we are basically stealing a guest physical page
allocated via kmalloc() and provide no facilities for Linux to reclaim the
page if it needs to do it before the userspace decide to unmap the foreign
mapping.

I think it would be good to handle the foreing mapping the same way as
userspace memory. By that I mean, that Linux could reclaim the physical page
used by the foreing mapping if it needs to.

The process for reclaiming the page would look like:
 1) Unmap the foreign page
 2) Ballon in the backend domain physical address used by the foreing
mapping (allocate the page in the physmap)

The next time the userspace is trying to access the foreign page, Linux will
receive a data abort that would result to:
 1) Allocate a backend domain physical page
 2) Balloon out the physical address (remove the page from the physmap)
 3) Map the foreing mapping at the new guest physical address
 4) Map the guest physical page in the userspace address space


This is going to shatter all the super pages in the stage-2
translation.


Yes, but this is nothing really new as ballooning would result to 
(AFAICT) the same behavior on Linux.





With this approach, we should be able to have backend domain that can handle
frontend domain without require a lot of memory.


Linux on x86 has the option to use empty hotplug memory ranges to map
foreign memory: the balloon driver hotplugs an unpopulated physical
memory range that's not made available to the OS free memory allocator
and it's just used as scratch space to map foreign memory. Not sure
whether Arm has something similar, or if it could be implemented.


We already discussed that last year :). This was attempted in the past 
(I was still at Citrix) and indefinitely paused for Arm.


/proc/iomem can be incomplete on Linux if we didn't load a driver for 
all the devices. This means that Linux doesn't have the full view of 
what is physical range is freed.


Additionally, in the case of Dom0, all the regions corresponding to the 
host RAM are unusable when using the SMMU. This is because we would do 
1:1 mapping for the foreign mapping as well.


It might be possible to take advantage of the direct mapping property if 
Linux do some bookeeping. Although, this wouldn't work for 32-bit Dom0 
using short page tables (e.g some version of Debian does) as it may not 
be able to access all the host RAM. Whether we still care about is a 
different situation :).


For all the other domains, I think we would want the toolstack to 
provide a region that can be safely used for foreign mapping (similar to 
what we already do for the grant-table).




You can still use the map-on-fault behaviour as above, but I would
recommend that you try to limit the number of hypercalls issued.
Having to issue a single hypercall for each page fault it's going to
be slow, so I would instead use mmap batch to map the hole range in
unpopulated physical memory and then the OS fault handler just needs to
fill the page tables with the corresponding address.
IIUC your proposal, you are assuming that you will have enough free 
space in the physical address space to map the foreign mapping.


However that amount of free space is not 

[xen-unstable-coverity test] 152103: all pass - PUSHED

2020-07-22 Thread osstest service owner
flight 152103 xen-unstable-coverity real [real]
http://logs.test-lab.xenproject.org/osstest/logs/152103/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 xen  f3885e8c3ceaef101e466466e879e97103ecce18
baseline version:
 xen  fb024b779336a0f73b3aee885b2ce082e812881f

Last test of basis   152012  2020-07-19 09:18:28 Z3 days
Testing same since   152103  2020-07-22 09:24:23 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Christian Lindig 
  Edwin Török 
  Elliott Mitchell 
  George Dunlap 
  Igor Druzhinin 
  Jan Beulich 
  Juergen Gross 
  Julien Grall 
  Michal Leszczynski 
  Nick Rosbrook 
  Nick Rosbrook 
  Stefano Stabellini 
  Tim Deegan 
  Wei Liu 

jobs:
 coverity-amd64   pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   fb024b7793..f3885e8c3c  f3885e8c3ceaef101e466466e879e97103ecce18 -> 
coverity-tested/smoke



[PATCH] x86/vmce: Dispatch vmce_{rd,wr}msr() from guest_{rd,wr}msr()

2020-07-22 Thread Andrew Cooper
... rather than from the default clauses of the PV and HVM MSR handlers.

This means that we no longer take the vmce lock for any unknown MSR, and
accesses to architectural MCE banks outside of the subset implemented for the
guest no longer fall further through the unknown MSR path.

With the vmce calls removed, the hvm alternative_call()'s expression can be
simplified substantially.

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
CC: Wei Liu 
CC: Roger Pau Monné 
---
 xen/arch/x86/hvm/hvm.c | 16 ++--
 xen/arch/x86/msr.c | 16 
 xen/arch/x86/pv/emul-priv-op.c | 15 ---
 3 files changed, 18 insertions(+), 29 deletions(-)

diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 5bb47583b3..a9d1685549 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -3560,13 +3560,7 @@ int hvm_msr_read_intercept(unsigned int msr, uint64_t 
*msr_content)
  break;
 
 default:
-if ( (ret = vmce_rdmsr(msr, msr_content)) < 0 )
-goto gp_fault;
-/* If ret == 0 then this is not an MCE MSR, see other MSRs. */
-ret = ((ret == 0)
-   ? alternative_call(hvm_funcs.msr_read_intercept,
-  msr, msr_content)
-   : X86EMUL_OKAY);
+ret = alternative_call(hvm_funcs.msr_read_intercept, msr, msr_content);
 break;
 }
 
@@ -3696,13 +3690,7 @@ int hvm_msr_write_intercept(unsigned int msr, uint64_t 
msr_content,
 break;
 
 default:
-if ( (ret = vmce_wrmsr(msr, msr_content)) < 0 )
-goto gp_fault;
-/* If ret == 0 then this is not an MCE MSR, see other MSRs. */
-ret = ((ret == 0)
-   ? alternative_call(hvm_funcs.msr_write_intercept,
-  msr, msr_content)
-   : X86EMUL_OKAY);
+ret = alternative_call(hvm_funcs.msr_write_intercept, msr, 
msr_content);
 break;
 }
 
diff --git a/xen/arch/x86/msr.c b/xen/arch/x86/msr.c
index 22f921cc71..ca4307e19f 100644
--- a/xen/arch/x86/msr.c
+++ b/xen/arch/x86/msr.c
@@ -227,6 +227,14 @@ int guest_rdmsr(struct vcpu *v, uint32_t msr, uint64_t 
*val)
 *val = msrs->misc_features_enables.raw;
 break;
 
+case MSR_IA32_MCG_CAP ... MSR_IA32_MCG_CTL:  /* 0x179 -> 0x17b */
+case MSR_IA32_MCx_CTL2(0) ... MSR_IA32_MCx_CTL2(31): /* 0x280 -> 0x29f */
+case MSR_IA32_MCx_CTL(0)  ... MSR_IA32_MCx_MISC(31): /* 0x400 -> 0x47f */
+case MSR_IA32_MCG_EXT_CTL:   /* 0x4d0 */
+if ( vmce_rdmsr(msr, val) < 0 )
+goto gp_fault;
+break;
+
 case MSR_X2APIC_FIRST ... MSR_X2APIC_LAST:
 if ( !is_hvm_domain(d) || v != curr )
 goto gp_fault;
@@ -436,6 +444,14 @@ int guest_wrmsr(struct vcpu *v, uint32_t msr, uint64_t val)
 break;
 }
 
+case MSR_IA32_MCG_CAP ... MSR_IA32_MCG_CTL:  /* 0x179 -> 0x17b */
+case MSR_IA32_MCx_CTL2(0) ... MSR_IA32_MCx_CTL2(31): /* 0x280 -> 0x29f */
+case MSR_IA32_MCx_CTL(0)  ... MSR_IA32_MCx_MISC(31): /* 0x400 -> 0x47f */
+case MSR_IA32_MCG_EXT_CTL:   /* 0x4d0 */
+if ( vmce_wrmsr(msr, val) < 0 )
+goto gp_fault;
+break;
+
 case MSR_X2APIC_FIRST ... MSR_X2APIC_LAST:
 if ( !is_hvm_domain(d) || v != curr )
 goto gp_fault;
diff --git a/xen/arch/x86/pv/emul-priv-op.c b/xen/arch/x86/pv/emul-priv-op.c
index 254da2b849..f14552cb4b 100644
--- a/xen/arch/x86/pv/emul-priv-op.c
+++ b/xen/arch/x86/pv/emul-priv-op.c
@@ -855,8 +855,6 @@ static int read_msr(unsigned int reg, uint64_t *val,
 
 switch ( reg )
 {
-int rc;
-
 case MSR_FS_BASE:
 if ( is_pv_32bit_domain(currd) )
 break;
@@ -955,12 +953,6 @@ static int read_msr(unsigned int reg, uint64_t *val,
 }
 /* fall through */
 default:
-rc = vmce_rdmsr(reg, val);
-if ( rc < 0 )
-break;
-if ( rc )
-return X86EMUL_OKAY;
-/* fall through */
 normal:
 /* Everyone can read the MSR space. */
 /* gdprintk(XENLOG_WARNING, "Domain attempted RDMSR %08x\n", reg); */
@@ -991,7 +983,6 @@ static int write_msr(unsigned int reg, uint64_t val,
 switch ( reg )
 {
 uint64_t temp;
-int rc;
 
 case MSR_FS_BASE:
 if ( is_pv_32bit_domain(currd) || !is_canonical_address(val) )
@@ -1122,12 +1113,6 @@ static int write_msr(unsigned int reg, uint64_t val,
 }
 /* fall through */
 default:
-rc = vmce_wrmsr(reg, val);
-if ( rc < 0 )
-break;
-if ( rc )
-return X86EMUL_OKAY;
-
 if ( (rdmsr_safe(reg, temp) != 0) || (val != temp) )
 invalid:
 gdprintk(XENLOG_WARNING,
-- 
2.11.0




Re: [PATCH] x86/svm: Fold nsvm_{wr,rd}msr() into svm_msr_{read,write}_intercept()

2020-07-22 Thread Andrew Cooper
On 22/07/2020 10:16, Jan Beulich wrote:
> On 21.07.2020 19:22, Andrew Cooper wrote:
>> ... to simplify the default cases.
>>
>> There are multiple errors with the handling of these three MSRs, but they are
>> deliberately not addressed this point.
>>
>> This removes the dance converting -1/0/1 into X86EMUL_*, allowing for the
>> removal of the 'ret' variable.
>>
>> While cleaning this up, drop the gdprintk()'s for #GP conditions, and the
>> 'result' variable from svm_msr_write_intercept() is it is never modified.
>>
>> No functional change.
>>
>> Signed-off-by: Andrew Cooper 
> Reviewed-by: Jan Beulich 
>
> However, ...
>
>> @@ -2085,6 +2091,22 @@ static int svm_msr_write_intercept(unsigned int msr, 
>> uint64_t msr_content)
>>  goto gpf;
>>  break;
>>  
>> +case MSR_K8_VM_CR:
>> +/* ignore write. handle all bits as read-only. */
>> +break;
>> +
>> +case MSR_K8_VM_HSAVE_PA:
>> +if ( (msr_content & ~PAGE_MASK) || msr_content > 0xfdULL )
> ... while you're moving this code here, wouldn't it be worthwhile
> to at least fix the > to be >= ?

I'd prefer not to, to avoid breaking the "No Functional Change" aspect.

In reality, this needs to be a path which takes an extra ref on the
nominated frame and globally maps it, seeing as we memcpy to/from it on
every virtual vmentry/exit.  The check against the HT range is quite bogus.

~Andrew



Re: [PATCH] x86/svm: Fold nsvm_{wr,rd}msr() into svm_msr_{read,write}_intercept()

2020-07-22 Thread Andrew Cooper
On 22/07/2020 10:34, Jan Beulich wrote:
> On 22.07.2020 11:26, Roger Pau Monné wrote:
>> On Tue, Jul 21, 2020 at 06:22:08PM +0100, Andrew Cooper wrote:
>>> @@ -2085,6 +2091,22 @@ static int svm_msr_write_intercept(unsigned int msr, 
>>> uint64_t msr_content)
>>>  goto gpf;
>>>  break;
>>>  
>>> +case MSR_K8_VM_CR:
>>> +/* ignore write. handle all bits as read-only. */
>>> +break;
>>> +
>>> +case MSR_K8_VM_HSAVE_PA:
>>> +if ( (msr_content & ~PAGE_MASK) || msr_content > 0xfdULL )
>> Regarding the address check, the PM states "the maximum supported
>> physical address for this implementation", but I don't seem to be able
>> to find where is this actually announced.
> I think you'd typically find this information in the BKDG or PPR only.
> The PM is generic, while the named two are specific to particular
> families or even just models.

Furthermore, the BKDG/PPR's are misleading/wrong.

For pre Fam17h, it is MAXPHYSADDR - 12G, which gives a limit lower than
0xfd on various SoC and embedded platforms.

On Fam17h, it is also lowered dynamically by how much memory encryption
is turned on (and therefore steals bits from the upper end of MAXPHYSADDR).


However, neither of these points are relevant in the slightest to
nested-svm because we don't ever map the HyperTransport range into
guests to start with - we'd get #PF[Rsvd] if we ever tried to use these
mappings.

Last time I presented this patch (nearly 2 years ago, and in the middle
of a series), it got very bogged down in a swamp of nested virt work,
which is why this time I've gone for no functional change, and punting
all the nested virt work to some future point where I've got time to
deal with it, and its not blocking the improvement wanted here.

~Andrew



Re: [xen-unstable test] 152067: regressions - trouble: fail/pass/starved

2020-07-22 Thread Jürgen Groß

On 22.07.20 11:30, Roger Pau Monné wrote:

On Wed, Jul 22, 2020 at 11:23:20AM +0200, Jürgen Groß wrote:

On 22.07.20 11:02, Roger Pau Monné wrote:

On Wed, Jul 22, 2020 at 10:59:48AM +0200, Jürgen Groß wrote:

On 22.07.20 10:38, Roger Pau Monné wrote:

On Wed, Jul 22, 2020 at 12:37:46AM +, osstest service owner wrote:

flight 152067 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/152067/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
test-amd64-amd64-dom0pvh-xl-intel 18 guest-localmigrate/x10 fail REGR. vs. 
152045


Failure was caused by:

Jul 21 16:20:58.985209 [  530.412043] libxl-save-help: page allocation failure: 
order:4, mode:0x60c0c0(GFP_KERNEL|__GFP_COMP|__GFP_ZERO), nodemask=(null)
Jul 21 16:21:00.378548 [  530.412261] libxl-save-help cpuset=/ mems_allowed=0
Jul 21 16:21:00.378622 [  530.412318] CPU: 1 PID: 15485 Comm: libxl-save-help 
Not tainted 4.19.80+ #1
Jul 21 16:21:00.390740 [  530.412377] Hardware name: Dell Inc. PowerEdge 
R420/0K29HN, BIOS 2.4.2 01/29/2015
Jul 21 16:21:00.390810 [  530.412448] Call Trace:
Jul 21 16:21:00.402721 [  530.412499]  dump_stack+0x72/0x8c
Jul 21 16:21:00.402801 [  530.412541]  warn_alloc.cold.140+0x68/0xe8
Jul 21 16:21:00.402841 [  530.412585]  __alloc_pages_slowpath+0xc73/0xcb0
Jul 21 16:21:00.414737 [  530.412640]  ? __do_page_fault+0x249/0x4d0
Jul 21 16:21:00.414786 [  530.412681]  __alloc_pages_nodemask+0x235/0x250
Jul 21 16:21:00.426555 [  530.412734]  kmalloc_order+0x13/0x60
Jul 21 16:21:00.426619 [  530.412774]  kmalloc_order_trace+0x18/0xa0
Jul 21 16:21:00.426671 [  530.412816]  alloc_empty_pages.isra.15+0x24/0x60
Jul 21 16:21:00.438447 [  530.412867]  
privcmd_ioctl_mmap_batch.isra.18+0x303/0x320
Jul 21 16:21:00.438507 [  530.412918]  ? vmacache_find+0xb0/0xb0
Jul 21 16:21:00.450475 [  530.412957]  privcmd_ioctl+0x253/0xa9b
Jul 21 16:21:00.450540 [  530.412996]  ? mmap_region+0x226/0x630
Jul 21 16:21:00.450592 [  530.413043]  ? selinux_mmap_file+0xb0/0xb0
Jul 21 16:21:00.462757 [  530.413084]  ? selinux_file_ioctl+0x15c/0x200
Jul 21 16:21:00.462823 [  530.413136]  do_vfs_ioctl+0x9f/0x630
Jul 21 16:21:00.474698 [  530.413177]  ksys_ioctl+0x5b/0x90
Jul 21 16:21:00.474762 [  530.413224]  __x64_sys_ioctl+0x11/0x20
Jul 21 16:21:00.474813 [  530.413264]  do_syscall_64+0x57/0x130
Jul 21 16:21:00.486480 [  530.413305]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
Jul 21 16:21:00.486548 [  530.413357] RIP: 0033:0x7f4f7ecde427
Jul 21 16:21:00.486600 [  530.413395] Code: 00 00 90 48 8b 05 69 aa 0c 00 64 c7 00 26 
00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 
<48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 39 aa 0c 00 f7 d8 64 89 01 48
Jul 21 16:21:00.510766 [  530.413556] RSP: 002b:7ffc1ef6eb38 EFLAGS: 
0213 ORIG_RAX: 0010
Jul 21 16:21:00.522758 [  530.413629] RAX: ffda RBX: 
 RCX: 7f4f7ecde427
Jul 21 16:21:00.534632 [  530.413699] RDX: 7ffc1ef6eb90 RSI: 
00205004 RDI: 0007
Jul 21 16:21:00.534702 [  530.413810] RBP: 7ffc1ef6ebe0 R08: 
0007 R09: 
Jul 21 16:21:00.547013 [  530.413881] R10: 0001 R11: 
0213 R12: 55d754136200
Jul 21 16:21:00.558751 [  530.413951] R13: 7ffc1ef6f340 R14: 
 R15: 
Jul 21 16:21:00.558846 [  530.414079] Mem-Info:
Jul 21 16:21:00.558928 [  530.414123] active_anon:1724 inactive_anon:3931 
isolated_anon:0
Jul 21 16:21:00.570481 [  530.414123]  active_file:7862 inactive_file:86530 
isolated_file:0
Jul 21 16:21:00.582599 [  530.414123]  unevictable:0 dirty:18 writeback:0 
unstable:0
Jul 21 16:21:00.582668 [  530.414123]  slab_reclaimable:4704 
slab_unreclaimable:4036
Jul 21 16:21:00.594782 [  530.414123]  mapped:3461 shmem:124 pagetables:372 
bounce:0
Jul 21 16:21:00.594849 [  530.414123]  free:1863 free_pcp:16 free_cma:0
Jul 21 16:21:00.606733 [  530.414579] Node 0 active_anon:6896kB 
inactive_anon:15724kB active_file:31448kB inactive_file:346120kB 
unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:13844kB dirty:72kB 
writeback:0kB shmem:496kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
Jul 21 16:21:00.630626 [  530.414870] DMA free:1816kB min:92kB low:112kB 
high:132kB active_anon:0kB inactive_anon:0kB active_file:76kB 
inactive_file:9988kB unevictable:0kB writepending:0kB present:15980kB 
managed:14328kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB 
free_pcp:0kB local_pcp:0kB free_cma:0kB
Jul 21 16:21:00.658448 [  530.415329] lowmem_reserve[]: 0 431 431 431
Jul 21 16:21:00.658513 [  530.415404] DMA32 free:5512kB min:2608kB low:3260kB 
high:3912kB active_anon:6896kB inactive_anon:15724kB active_file:31372kB 
inactive_file:336132kB unevictable:0kB writepending:72kB present:508300kB 
managed:451760kB mlocked:0kB kernel_stack:2848kB pagetables:1488kB bounce:0kB 
free_pcp:184kB local_pcp:0kB free_cma:0kB
Jul 21 16:21:00.694702 [  

Re: [PATCH] x86/svm: Fold nsvm_{wr,rd}msr() into svm_msr_{read,write}_intercept()

2020-07-22 Thread Jan Beulich
On 22.07.2020 11:26, Roger Pau Monné wrote:
> On Tue, Jul 21, 2020 at 06:22:08PM +0100, Andrew Cooper wrote:
>> @@ -2085,6 +2091,22 @@ static int svm_msr_write_intercept(unsigned int msr, 
>> uint64_t msr_content)
>>  goto gpf;
>>  break;
>>  
>> +case MSR_K8_VM_CR:
>> +/* ignore write. handle all bits as read-only. */
>> +break;
>> +
>> +case MSR_K8_VM_HSAVE_PA:
>> +if ( (msr_content & ~PAGE_MASK) || msr_content > 0xfdULL )
> 
> Regarding the address check, the PM states "the maximum supported
> physical address for this implementation", but I don't seem to be able
> to find where is this actually announced.

I think you'd typically find this information in the BKDG or PPR only.
The PM is generic, while the named two are specific to particular
families or even just models.

Jan



Re: [xen-unstable test] 152067: regressions - trouble: fail/pass/starved

2020-07-22 Thread Roger Pau Monné
On Wed, Jul 22, 2020 at 11:23:20AM +0200, Jürgen Groß wrote:
> On 22.07.20 11:02, Roger Pau Monné wrote:
> > On Wed, Jul 22, 2020 at 10:59:48AM +0200, Jürgen Groß wrote:
> > > On 22.07.20 10:38, Roger Pau Monné wrote:
> > > > On Wed, Jul 22, 2020 at 12:37:46AM +, osstest service owner wrote:
> > > > > flight 152067 xen-unstable real [real]
> > > > > http://logs.test-lab.xenproject.org/osstest/logs/152067/
> > > > > 
> > > > > Regressions :-(
> > > > > 
> > > > > Tests which did not succeed and are blocking,
> > > > > including tests which could not be run:
> > > > >test-amd64-amd64-dom0pvh-xl-intel 18 guest-localmigrate/x10 fail 
> > > > > REGR. vs. 152045
> > > > 
> > > > Failure was caused by:
> > > > 
> > > > Jul 21 16:20:58.985209 [  530.412043] libxl-save-help: page allocation 
> > > > failure: order:4, mode:0x60c0c0(GFP_KERNEL|__GFP_COMP|__GFP_ZERO), 
> > > > nodemask=(null)
> > > > Jul 21 16:21:00.378548 [  530.412261] libxl-save-help cpuset=/ 
> > > > mems_allowed=0
> > > > Jul 21 16:21:00.378622 [  530.412318] CPU: 1 PID: 15485 Comm: 
> > > > libxl-save-help Not tainted 4.19.80+ #1
> > > > Jul 21 16:21:00.390740 [  530.412377] Hardware name: Dell Inc. 
> > > > PowerEdge R420/0K29HN, BIOS 2.4.2 01/29/2015
> > > > Jul 21 16:21:00.390810 [  530.412448] Call Trace:
> > > > Jul 21 16:21:00.402721 [  530.412499]  dump_stack+0x72/0x8c
> > > > Jul 21 16:21:00.402801 [  530.412541]  warn_alloc.cold.140+0x68/0xe8
> > > > Jul 21 16:21:00.402841 [  530.412585]  
> > > > __alloc_pages_slowpath+0xc73/0xcb0
> > > > Jul 21 16:21:00.414737 [  530.412640]  ? __do_page_fault+0x249/0x4d0
> > > > Jul 21 16:21:00.414786 [  530.412681]  
> > > > __alloc_pages_nodemask+0x235/0x250
> > > > Jul 21 16:21:00.426555 [  530.412734]  kmalloc_order+0x13/0x60
> > > > Jul 21 16:21:00.426619 [  530.412774]  kmalloc_order_trace+0x18/0xa0
> > > > Jul 21 16:21:00.426671 [  530.412816]  
> > > > alloc_empty_pages.isra.15+0x24/0x60
> > > > Jul 21 16:21:00.438447 [  530.412867]  
> > > > privcmd_ioctl_mmap_batch.isra.18+0x303/0x320
> > > > Jul 21 16:21:00.438507 [  530.412918]  ? vmacache_find+0xb0/0xb0
> > > > Jul 21 16:21:00.450475 [  530.412957]  privcmd_ioctl+0x253/0xa9b
> > > > Jul 21 16:21:00.450540 [  530.412996]  ? mmap_region+0x226/0x630
> > > > Jul 21 16:21:00.450592 [  530.413043]  ? selinux_mmap_file+0xb0/0xb0
> > > > Jul 21 16:21:00.462757 [  530.413084]  ? selinux_file_ioctl+0x15c/0x200
> > > > Jul 21 16:21:00.462823 [  530.413136]  do_vfs_ioctl+0x9f/0x630
> > > > Jul 21 16:21:00.474698 [  530.413177]  ksys_ioctl+0x5b/0x90
> > > > Jul 21 16:21:00.474762 [  530.413224]  __x64_sys_ioctl+0x11/0x20
> > > > Jul 21 16:21:00.474813 [  530.413264]  do_syscall_64+0x57/0x130
> > > > Jul 21 16:21:00.486480 [  530.413305]  
> > > > entry_SYSCALL_64_after_hwframe+0x44/0xa9
> > > > Jul 21 16:21:00.486548 [  530.413357] RIP: 0033:0x7f4f7ecde427
> > > > Jul 21 16:21:00.486600 [  530.413395] Code: 00 00 90 48 8b 05 69 aa 0c 
> > > > 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 
> > > > 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 39 aa 
> > > > 0c 00 f7 d8 64 89 01 48
> > > > Jul 21 16:21:00.510766 [  530.413556] RSP: 002b:7ffc1ef6eb38 
> > > > EFLAGS: 0213 ORIG_RAX: 0010
> > > > Jul 21 16:21:00.522758 [  530.413629] RAX: ffda RBX: 
> > > >  RCX: 7f4f7ecde427
> > > > Jul 21 16:21:00.534632 [  530.413699] RDX: 7ffc1ef6eb90 RSI: 
> > > > 00205004 RDI: 0007
> > > > Jul 21 16:21:00.534702 [  530.413810] RBP: 7ffc1ef6ebe0 R08: 
> > > > 0007 R09: 
> > > > Jul 21 16:21:00.547013 [  530.413881] R10: 0001 R11: 
> > > > 0213 R12: 55d754136200
> > > > Jul 21 16:21:00.558751 [  530.413951] R13: 7ffc1ef6f340 R14: 
> > > >  R15: 
> > > > Jul 21 16:21:00.558846 [  530.414079] Mem-Info:
> > > > Jul 21 16:21:00.558928 [  530.414123] active_anon:1724 
> > > > inactive_anon:3931 isolated_anon:0
> > > > Jul 21 16:21:00.570481 [  530.414123]  active_file:7862 
> > > > inactive_file:86530 isolated_file:0
> > > > Jul 21 16:21:00.582599 [  530.414123]  unevictable:0 dirty:18 
> > > > writeback:0 unstable:0
> > > > Jul 21 16:21:00.582668 [  530.414123]  slab_reclaimable:4704 
> > > > slab_unreclaimable:4036
> > > > Jul 21 16:21:00.594782 [  530.414123]  mapped:3461 shmem:124 
> > > > pagetables:372 bounce:0
> > > > Jul 21 16:21:00.594849 [  530.414123]  free:1863 free_pcp:16 free_cma:0
> > > > Jul 21 16:21:00.606733 [  530.414579] Node 0 active_anon:6896kB 
> > > > inactive_anon:15724kB active_file:31448kB inactive_file:346120kB 
> > > > unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:13844kB 
> > > > dirty:72kB writeback:0kB shmem:496kB writeback_tmp:0kB unstable:0kB 
> > > > all_unreclaimable? no
> > > > Jul 21 16:21:00.630626 [  530.414870] DMA free:1816kB min:92kB 
> > > > low:112kB high:132kB active_anon:0kB 

Re: [PATCH] x86/svm: Fold nsvm_{wr,rd}msr() into svm_msr_{read,write}_intercept()

2020-07-22 Thread Roger Pau Monné
On Tue, Jul 21, 2020 at 06:22:08PM +0100, Andrew Cooper wrote:
> ... to simplify the default cases.
> 
> There are multiple errors with the handling of these three MSRs, but they are
> deliberately not addressed this point.
^ at
> 
> This removes the dance converting -1/0/1 into X86EMUL_*, allowing for the
> removal of the 'ret' variable.
> 
> While cleaning this up, drop the gdprintk()'s for #GP conditions, and the
> 'result' variable from svm_msr_write_intercept() is it is never modified.
   ^ extra is
> 
> No functional change.
> 
> Signed-off-by: Andrew Cooper 

Reviewed-by: Roger Pau Monné 

I've got one question (not really patch related).

> @@ -1956,10 +1962,10 @@ static int svm_msr_read_intercept(unsigned int msr, 
> uint64_t *msr_content)
>  
>  static int svm_msr_write_intercept(unsigned int msr, uint64_t msr_content)
>  {
> -int ret, result = X86EMUL_OKAY;
>  struct vcpu *v = current;
>  struct domain *d = v->domain;
>  struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
> +struct nestedsvm *nsvm = _nestedsvm(v);
>  
>  switch ( msr )
>  {
> @@ -2085,6 +2091,22 @@ static int svm_msr_write_intercept(unsigned int msr, 
> uint64_t msr_content)
>  goto gpf;
>  break;
>  
> +case MSR_K8_VM_CR:
> +/* ignore write. handle all bits as read-only. */
> +break;
> +
> +case MSR_K8_VM_HSAVE_PA:
> +if ( (msr_content & ~PAGE_MASK) || msr_content > 0xfdULL )

Regarding the address check, the PM states "the maximum supported
physical address for this implementation", but I don't seem to be able
to find where is this actually announced.

Roger.



Re: [xen-unstable test] 152067: regressions - trouble: fail/pass/starved

2020-07-22 Thread Jürgen Groß

On 22.07.20 11:02, Roger Pau Monné wrote:

On Wed, Jul 22, 2020 at 10:59:48AM +0200, Jürgen Groß wrote:

On 22.07.20 10:38, Roger Pau Monné wrote:

On Wed, Jul 22, 2020 at 12:37:46AM +, osstest service owner wrote:

flight 152067 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/152067/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
   test-amd64-amd64-dom0pvh-xl-intel 18 guest-localmigrate/x10 fail REGR. vs. 
152045


Failure was caused by:

Jul 21 16:20:58.985209 [  530.412043] libxl-save-help: page allocation failure: 
order:4, mode:0x60c0c0(GFP_KERNEL|__GFP_COMP|__GFP_ZERO), nodemask=(null)
Jul 21 16:21:00.378548 [  530.412261] libxl-save-help cpuset=/ mems_allowed=0
Jul 21 16:21:00.378622 [  530.412318] CPU: 1 PID: 15485 Comm: libxl-save-help 
Not tainted 4.19.80+ #1
Jul 21 16:21:00.390740 [  530.412377] Hardware name: Dell Inc. PowerEdge 
R420/0K29HN, BIOS 2.4.2 01/29/2015
Jul 21 16:21:00.390810 [  530.412448] Call Trace:
Jul 21 16:21:00.402721 [  530.412499]  dump_stack+0x72/0x8c
Jul 21 16:21:00.402801 [  530.412541]  warn_alloc.cold.140+0x68/0xe8
Jul 21 16:21:00.402841 [  530.412585]  __alloc_pages_slowpath+0xc73/0xcb0
Jul 21 16:21:00.414737 [  530.412640]  ? __do_page_fault+0x249/0x4d0
Jul 21 16:21:00.414786 [  530.412681]  __alloc_pages_nodemask+0x235/0x250
Jul 21 16:21:00.426555 [  530.412734]  kmalloc_order+0x13/0x60
Jul 21 16:21:00.426619 [  530.412774]  kmalloc_order_trace+0x18/0xa0
Jul 21 16:21:00.426671 [  530.412816]  alloc_empty_pages.isra.15+0x24/0x60
Jul 21 16:21:00.438447 [  530.412867]  
privcmd_ioctl_mmap_batch.isra.18+0x303/0x320
Jul 21 16:21:00.438507 [  530.412918]  ? vmacache_find+0xb0/0xb0
Jul 21 16:21:00.450475 [  530.412957]  privcmd_ioctl+0x253/0xa9b
Jul 21 16:21:00.450540 [  530.412996]  ? mmap_region+0x226/0x630
Jul 21 16:21:00.450592 [  530.413043]  ? selinux_mmap_file+0xb0/0xb0
Jul 21 16:21:00.462757 [  530.413084]  ? selinux_file_ioctl+0x15c/0x200
Jul 21 16:21:00.462823 [  530.413136]  do_vfs_ioctl+0x9f/0x630
Jul 21 16:21:00.474698 [  530.413177]  ksys_ioctl+0x5b/0x90
Jul 21 16:21:00.474762 [  530.413224]  __x64_sys_ioctl+0x11/0x20
Jul 21 16:21:00.474813 [  530.413264]  do_syscall_64+0x57/0x130
Jul 21 16:21:00.486480 [  530.413305]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
Jul 21 16:21:00.486548 [  530.413357] RIP: 0033:0x7f4f7ecde427
Jul 21 16:21:00.486600 [  530.413395] Code: 00 00 90 48 8b 05 69 aa 0c 00 64 c7 00 26 
00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 
<48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 39 aa 0c 00 f7 d8 64 89 01 48
Jul 21 16:21:00.510766 [  530.413556] RSP: 002b:7ffc1ef6eb38 EFLAGS: 
0213 ORIG_RAX: 0010
Jul 21 16:21:00.522758 [  530.413629] RAX: ffda RBX: 
 RCX: 7f4f7ecde427
Jul 21 16:21:00.534632 [  530.413699] RDX: 7ffc1ef6eb90 RSI: 
00205004 RDI: 0007
Jul 21 16:21:00.534702 [  530.413810] RBP: 7ffc1ef6ebe0 R08: 
0007 R09: 
Jul 21 16:21:00.547013 [  530.413881] R10: 0001 R11: 
0213 R12: 55d754136200
Jul 21 16:21:00.558751 [  530.413951] R13: 7ffc1ef6f340 R14: 
 R15: 
Jul 21 16:21:00.558846 [  530.414079] Mem-Info:
Jul 21 16:21:00.558928 [  530.414123] active_anon:1724 inactive_anon:3931 
isolated_anon:0
Jul 21 16:21:00.570481 [  530.414123]  active_file:7862 inactive_file:86530 
isolated_file:0
Jul 21 16:21:00.582599 [  530.414123]  unevictable:0 dirty:18 writeback:0 
unstable:0
Jul 21 16:21:00.582668 [  530.414123]  slab_reclaimable:4704 
slab_unreclaimable:4036
Jul 21 16:21:00.594782 [  530.414123]  mapped:3461 shmem:124 pagetables:372 
bounce:0
Jul 21 16:21:00.594849 [  530.414123]  free:1863 free_pcp:16 free_cma:0
Jul 21 16:21:00.606733 [  530.414579] Node 0 active_anon:6896kB 
inactive_anon:15724kB active_file:31448kB inactive_file:346120kB 
unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:13844kB dirty:72kB 
writeback:0kB shmem:496kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
Jul 21 16:21:00.630626 [  530.414870] DMA free:1816kB min:92kB low:112kB 
high:132kB active_anon:0kB inactive_anon:0kB active_file:76kB 
inactive_file:9988kB unevictable:0kB writepending:0kB present:15980kB 
managed:14328kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB 
free_pcp:0kB local_pcp:0kB free_cma:0kB
Jul 21 16:21:00.658448 [  530.415329] lowmem_reserve[]: 0 431 431 431
Jul 21 16:21:00.658513 [  530.415404] DMA32 free:5512kB min:2608kB low:3260kB 
high:3912kB active_anon:6896kB inactive_anon:15724kB active_file:31372kB 
inactive_file:336132kB unevictable:0kB writepending:72kB present:508300kB 
managed:451760kB mlocked:0kB kernel_stack:2848kB pagetables:1488kB bounce:0kB 
free_pcp:184kB local_pcp:0kB free_cma:0kB
Jul 21 16:21:00.694702 [  530.415742] lowmem_reserve[]: 0 0 0 0
Jul 21 16:21:00.694778 [  530.415806] DMA: 8*4kB (UM) 3*8kB (UM) 

Re: [PATCH] x86/svm: Fold nsvm_{wr,rd}msr() into svm_msr_{read,write}_intercept()

2020-07-22 Thread Jan Beulich
On 21.07.2020 19:22, Andrew Cooper wrote:
> ... to simplify the default cases.
> 
> There are multiple errors with the handling of these three MSRs, but they are
> deliberately not addressed this point.
> 
> This removes the dance converting -1/0/1 into X86EMUL_*, allowing for the
> removal of the 'ret' variable.
> 
> While cleaning this up, drop the gdprintk()'s for #GP conditions, and the
> 'result' variable from svm_msr_write_intercept() is it is never modified.
> 
> No functional change.
> 
> Signed-off-by: Andrew Cooper 

Reviewed-by: Jan Beulich 

However, ...

> @@ -2085,6 +2091,22 @@ static int svm_msr_write_intercept(unsigned int msr, 
> uint64_t msr_content)
>  goto gpf;
>  break;
>  
> +case MSR_K8_VM_CR:
> +/* ignore write. handle all bits as read-only. */
> +break;
> +
> +case MSR_K8_VM_HSAVE_PA:
> +if ( (msr_content & ~PAGE_MASK) || msr_content > 0xfdULL )

... while you're moving this code here, wouldn't it be worthwhile
to at least fix the > to be >= ?

Jan



Re: [PATCH v2 04/11] x86/xen: add system core suspend and resume callbacks

2020-07-22 Thread Julien Grall

Hi,

On 02/07/2020 19:22, Anchal Agarwal wrote:

diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h
index 2521d6a306cd..9fa8a4082d68 100644
--- a/include/xen/xen-ops.h
+++ b/include/xen/xen-ops.h
@@ -41,6 +41,8 @@ u64 xen_steal_clock(int cpu);
  int xen_setup_shutdown_event(void);
  
  bool xen_is_xen_suspend(void);

+void xen_setup_syscore_ops(void);


The function is only implemented and used by x86. So shouldn't this be 
declared in an x86 header?


Cheers,

--
Julien Grall



Re: [xen-unstable test] 152067: regressions - trouble: fail/pass/starved

2020-07-22 Thread Roger Pau Monné
On Wed, Jul 22, 2020 at 10:59:48AM +0200, Jürgen Groß wrote:
> On 22.07.20 10:38, Roger Pau Monné wrote:
> > On Wed, Jul 22, 2020 at 12:37:46AM +, osstest service owner wrote:
> > > flight 152067 xen-unstable real [real]
> > > http://logs.test-lab.xenproject.org/osstest/logs/152067/
> > > 
> > > Regressions :-(
> > > 
> > > Tests which did not succeed and are blocking,
> > > including tests which could not be run:
> > >   test-amd64-amd64-dom0pvh-xl-intel 18 guest-localmigrate/x10 fail REGR. 
> > > vs. 152045
> > 
> > Failure was caused by:
> > 
> > Jul 21 16:20:58.985209 [  530.412043] libxl-save-help: page allocation 
> > failure: order:4, mode:0x60c0c0(GFP_KERNEL|__GFP_COMP|__GFP_ZERO), 
> > nodemask=(null)
> > Jul 21 16:21:00.378548 [  530.412261] libxl-save-help cpuset=/ 
> > mems_allowed=0
> > Jul 21 16:21:00.378622 [  530.412318] CPU: 1 PID: 15485 Comm: 
> > libxl-save-help Not tainted 4.19.80+ #1
> > Jul 21 16:21:00.390740 [  530.412377] Hardware name: Dell Inc. PowerEdge 
> > R420/0K29HN, BIOS 2.4.2 01/29/2015
> > Jul 21 16:21:00.390810 [  530.412448] Call Trace:
> > Jul 21 16:21:00.402721 [  530.412499]  dump_stack+0x72/0x8c
> > Jul 21 16:21:00.402801 [  530.412541]  warn_alloc.cold.140+0x68/0xe8
> > Jul 21 16:21:00.402841 [  530.412585]  __alloc_pages_slowpath+0xc73/0xcb0
> > Jul 21 16:21:00.414737 [  530.412640]  ? __do_page_fault+0x249/0x4d0
> > Jul 21 16:21:00.414786 [  530.412681]  __alloc_pages_nodemask+0x235/0x250
> > Jul 21 16:21:00.426555 [  530.412734]  kmalloc_order+0x13/0x60
> > Jul 21 16:21:00.426619 [  530.412774]  kmalloc_order_trace+0x18/0xa0
> > Jul 21 16:21:00.426671 [  530.412816]  alloc_empty_pages.isra.15+0x24/0x60
> > Jul 21 16:21:00.438447 [  530.412867]  
> > privcmd_ioctl_mmap_batch.isra.18+0x303/0x320
> > Jul 21 16:21:00.438507 [  530.412918]  ? vmacache_find+0xb0/0xb0
> > Jul 21 16:21:00.450475 [  530.412957]  privcmd_ioctl+0x253/0xa9b
> > Jul 21 16:21:00.450540 [  530.412996]  ? mmap_region+0x226/0x630
> > Jul 21 16:21:00.450592 [  530.413043]  ? selinux_mmap_file+0xb0/0xb0
> > Jul 21 16:21:00.462757 [  530.413084]  ? selinux_file_ioctl+0x15c/0x200
> > Jul 21 16:21:00.462823 [  530.413136]  do_vfs_ioctl+0x9f/0x630
> > Jul 21 16:21:00.474698 [  530.413177]  ksys_ioctl+0x5b/0x90
> > Jul 21 16:21:00.474762 [  530.413224]  __x64_sys_ioctl+0x11/0x20
> > Jul 21 16:21:00.474813 [  530.413264]  do_syscall_64+0x57/0x130
> > Jul 21 16:21:00.486480 [  530.413305]  
> > entry_SYSCALL_64_after_hwframe+0x44/0xa9
> > Jul 21 16:21:00.486548 [  530.413357] RIP: 0033:0x7f4f7ecde427
> > Jul 21 16:21:00.486600 [  530.413395] Code: 00 00 90 48 8b 05 69 aa 0c 00 
> > 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 
> > b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 39 aa 0c 00 f7 
> > d8 64 89 01 48
> > Jul 21 16:21:00.510766 [  530.413556] RSP: 002b:7ffc1ef6eb38 EFLAGS: 
> > 0213 ORIG_RAX: 0010
> > Jul 21 16:21:00.522758 [  530.413629] RAX: ffda RBX: 
> >  RCX: 7f4f7ecde427
> > Jul 21 16:21:00.534632 [  530.413699] RDX: 7ffc1ef6eb90 RSI: 
> > 00205004 RDI: 0007
> > Jul 21 16:21:00.534702 [  530.413810] RBP: 7ffc1ef6ebe0 R08: 
> > 0007 R09: 
> > Jul 21 16:21:00.547013 [  530.413881] R10: 0001 R11: 
> > 0213 R12: 55d754136200
> > Jul 21 16:21:00.558751 [  530.413951] R13: 7ffc1ef6f340 R14: 
> >  R15: 
> > Jul 21 16:21:00.558846 [  530.414079] Mem-Info:
> > Jul 21 16:21:00.558928 [  530.414123] active_anon:1724 inactive_anon:3931 
> > isolated_anon:0
> > Jul 21 16:21:00.570481 [  530.414123]  active_file:7862 inactive_file:86530 
> > isolated_file:0
> > Jul 21 16:21:00.582599 [  530.414123]  unevictable:0 dirty:18 writeback:0 
> > unstable:0
> > Jul 21 16:21:00.582668 [  530.414123]  slab_reclaimable:4704 
> > slab_unreclaimable:4036
> > Jul 21 16:21:00.594782 [  530.414123]  mapped:3461 shmem:124 pagetables:372 
> > bounce:0
> > Jul 21 16:21:00.594849 [  530.414123]  free:1863 free_pcp:16 free_cma:0
> > Jul 21 16:21:00.606733 [  530.414579] Node 0 active_anon:6896kB 
> > inactive_anon:15724kB active_file:31448kB inactive_file:346120kB 
> > unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:13844kB 
> > dirty:72kB writeback:0kB shmem:496kB writeback_tmp:0kB unstable:0kB 
> > all_unreclaimable? no
> > Jul 21 16:21:00.630626 [  530.414870] DMA free:1816kB min:92kB low:112kB 
> > high:132kB active_anon:0kB inactive_anon:0kB active_file:76kB 
> > inactive_file:9988kB unevictable:0kB writepending:0kB present:15980kB 
> > managed:14328kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB 
> > free_pcp:0kB local_pcp:0kB free_cma:0kB
> > Jul 21 16:21:00.658448 [  530.415329] lowmem_reserve[]: 0 431 431 431
> > Jul 21 16:21:00.658513 [  530.415404] DMA32 free:5512kB min:2608kB 
> > low:3260kB high:3912kB active_anon:6896kB inactive_anon:15724kB 
> > active_file:31372kB 

Re: [xen-unstable test] 152067: regressions - trouble: fail/pass/starved

2020-07-22 Thread Jürgen Groß

On 22.07.20 10:38, Roger Pau Monné wrote:

On Wed, Jul 22, 2020 at 12:37:46AM +, osstest service owner wrote:

flight 152067 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/152067/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
  test-amd64-amd64-dom0pvh-xl-intel 18 guest-localmigrate/x10 fail REGR. vs. 
152045


Failure was caused by:

Jul 21 16:20:58.985209 [  530.412043] libxl-save-help: page allocation failure: 
order:4, mode:0x60c0c0(GFP_KERNEL|__GFP_COMP|__GFP_ZERO), nodemask=(null)
Jul 21 16:21:00.378548 [  530.412261] libxl-save-help cpuset=/ mems_allowed=0
Jul 21 16:21:00.378622 [  530.412318] CPU: 1 PID: 15485 Comm: libxl-save-help 
Not tainted 4.19.80+ #1
Jul 21 16:21:00.390740 [  530.412377] Hardware name: Dell Inc. PowerEdge 
R420/0K29HN, BIOS 2.4.2 01/29/2015
Jul 21 16:21:00.390810 [  530.412448] Call Trace:
Jul 21 16:21:00.402721 [  530.412499]  dump_stack+0x72/0x8c
Jul 21 16:21:00.402801 [  530.412541]  warn_alloc.cold.140+0x68/0xe8
Jul 21 16:21:00.402841 [  530.412585]  __alloc_pages_slowpath+0xc73/0xcb0
Jul 21 16:21:00.414737 [  530.412640]  ? __do_page_fault+0x249/0x4d0
Jul 21 16:21:00.414786 [  530.412681]  __alloc_pages_nodemask+0x235/0x250
Jul 21 16:21:00.426555 [  530.412734]  kmalloc_order+0x13/0x60
Jul 21 16:21:00.426619 [  530.412774]  kmalloc_order_trace+0x18/0xa0
Jul 21 16:21:00.426671 [  530.412816]  alloc_empty_pages.isra.15+0x24/0x60
Jul 21 16:21:00.438447 [  530.412867]  
privcmd_ioctl_mmap_batch.isra.18+0x303/0x320
Jul 21 16:21:00.438507 [  530.412918]  ? vmacache_find+0xb0/0xb0
Jul 21 16:21:00.450475 [  530.412957]  privcmd_ioctl+0x253/0xa9b
Jul 21 16:21:00.450540 [  530.412996]  ? mmap_region+0x226/0x630
Jul 21 16:21:00.450592 [  530.413043]  ? selinux_mmap_file+0xb0/0xb0
Jul 21 16:21:00.462757 [  530.413084]  ? selinux_file_ioctl+0x15c/0x200
Jul 21 16:21:00.462823 [  530.413136]  do_vfs_ioctl+0x9f/0x630
Jul 21 16:21:00.474698 [  530.413177]  ksys_ioctl+0x5b/0x90
Jul 21 16:21:00.474762 [  530.413224]  __x64_sys_ioctl+0x11/0x20
Jul 21 16:21:00.474813 [  530.413264]  do_syscall_64+0x57/0x130
Jul 21 16:21:00.486480 [  530.413305]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
Jul 21 16:21:00.486548 [  530.413357] RIP: 0033:0x7f4f7ecde427
Jul 21 16:21:00.486600 [  530.413395] Code: 00 00 90 48 8b 05 69 aa 0c 00 64 c7 00 26 
00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 
<48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 39 aa 0c 00 f7 d8 64 89 01 48
Jul 21 16:21:00.510766 [  530.413556] RSP: 002b:7ffc1ef6eb38 EFLAGS: 
0213 ORIG_RAX: 0010
Jul 21 16:21:00.522758 [  530.413629] RAX: ffda RBX: 
 RCX: 7f4f7ecde427
Jul 21 16:21:00.534632 [  530.413699] RDX: 7ffc1ef6eb90 RSI: 
00205004 RDI: 0007
Jul 21 16:21:00.534702 [  530.413810] RBP: 7ffc1ef6ebe0 R08: 
0007 R09: 
Jul 21 16:21:00.547013 [  530.413881] R10: 0001 R11: 
0213 R12: 55d754136200
Jul 21 16:21:00.558751 [  530.413951] R13: 7ffc1ef6f340 R14: 
 R15: 
Jul 21 16:21:00.558846 [  530.414079] Mem-Info:
Jul 21 16:21:00.558928 [  530.414123] active_anon:1724 inactive_anon:3931 
isolated_anon:0
Jul 21 16:21:00.570481 [  530.414123]  active_file:7862 inactive_file:86530 
isolated_file:0
Jul 21 16:21:00.582599 [  530.414123]  unevictable:0 dirty:18 writeback:0 
unstable:0
Jul 21 16:21:00.582668 [  530.414123]  slab_reclaimable:4704 
slab_unreclaimable:4036
Jul 21 16:21:00.594782 [  530.414123]  mapped:3461 shmem:124 pagetables:372 
bounce:0
Jul 21 16:21:00.594849 [  530.414123]  free:1863 free_pcp:16 free_cma:0
Jul 21 16:21:00.606733 [  530.414579] Node 0 active_anon:6896kB 
inactive_anon:15724kB active_file:31448kB inactive_file:346120kB 
unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:13844kB dirty:72kB 
writeback:0kB shmem:496kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
Jul 21 16:21:00.630626 [  530.414870] DMA free:1816kB min:92kB low:112kB 
high:132kB active_anon:0kB inactive_anon:0kB active_file:76kB 
inactive_file:9988kB unevictable:0kB writepending:0kB present:15980kB 
managed:14328kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB 
free_pcp:0kB local_pcp:0kB free_cma:0kB
Jul 21 16:21:00.658448 [  530.415329] lowmem_reserve[]: 0 431 431 431
Jul 21 16:21:00.658513 [  530.415404] DMA32 free:5512kB min:2608kB low:3260kB 
high:3912kB active_anon:6896kB inactive_anon:15724kB active_file:31372kB 
inactive_file:336132kB unevictable:0kB writepending:72kB present:508300kB 
managed:451760kB mlocked:0kB kernel_stack:2848kB pagetables:1488kB bounce:0kB 
free_pcp:184kB local_pcp:0kB free_cma:0kB
Jul 21 16:21:00.694702 [  530.415742] lowmem_reserve[]: 0 0 0 0
Jul 21 16:21:00.694778 [  530.415806] DMA: 8*4kB (UM) 3*8kB (UM) 4*16kB (UM) 
3*32kB (M) 5*64kB (UM) 2*128kB (UM) 4*256kB (UM) 0*512kB 0*1024kB 0*2048kB 
0*4096kB = 1816kB
Jul 

Xen 4.11.4 released

2020-07-22 Thread Jan Beulich
All,

better late than never, I am pleased to announce the release of Xen 4.11.4.
This has been available immediately from its git repository
http://xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=refs/heads/stable-4.11
(tag RELEASE-4.11.4) or from the XenProject download page
https://xenproject.org/downloads/xen-project-archives/xen-project-4-11-series/xen-project-4-11-4/
(where a list of changes can also be found).

We recommend all users of the 4.11 stable series to update to this last
point release to be made by the XenProject team from this stable branch.

Apologies for the much delayed announcement.

Regards, Jan



Re: [xen-unstable test] 152067: regressions - trouble: fail/pass/starved

2020-07-22 Thread Roger Pau Monné
On Wed, Jul 22, 2020 at 12:37:46AM +, osstest service owner wrote:
> flight 152067 xen-unstable real [real]
> http://logs.test-lab.xenproject.org/osstest/logs/152067/
> 
> Regressions :-(
> 
> Tests which did not succeed and are blocking,
> including tests which could not be run:
>  test-amd64-amd64-dom0pvh-xl-intel 18 guest-localmigrate/x10 fail REGR. vs. 
> 152045

Failure was caused by:

Jul 21 16:20:58.985209 [  530.412043] libxl-save-help: page allocation failure: 
order:4, mode:0x60c0c0(GFP_KERNEL|__GFP_COMP|__GFP_ZERO), nodemask=(null)
Jul 21 16:21:00.378548 [  530.412261] libxl-save-help cpuset=/ mems_allowed=0
Jul 21 16:21:00.378622 [  530.412318] CPU: 1 PID: 15485 Comm: libxl-save-help 
Not tainted 4.19.80+ #1
Jul 21 16:21:00.390740 [  530.412377] Hardware name: Dell Inc. PowerEdge 
R420/0K29HN, BIOS 2.4.2 01/29/2015
Jul 21 16:21:00.390810 [  530.412448] Call Trace:
Jul 21 16:21:00.402721 [  530.412499]  dump_stack+0x72/0x8c
Jul 21 16:21:00.402801 [  530.412541]  warn_alloc.cold.140+0x68/0xe8
Jul 21 16:21:00.402841 [  530.412585]  __alloc_pages_slowpath+0xc73/0xcb0
Jul 21 16:21:00.414737 [  530.412640]  ? __do_page_fault+0x249/0x4d0
Jul 21 16:21:00.414786 [  530.412681]  __alloc_pages_nodemask+0x235/0x250
Jul 21 16:21:00.426555 [  530.412734]  kmalloc_order+0x13/0x60
Jul 21 16:21:00.426619 [  530.412774]  kmalloc_order_trace+0x18/0xa0
Jul 21 16:21:00.426671 [  530.412816]  alloc_empty_pages.isra.15+0x24/0x60
Jul 21 16:21:00.438447 [  530.412867]  
privcmd_ioctl_mmap_batch.isra.18+0x303/0x320
Jul 21 16:21:00.438507 [  530.412918]  ? vmacache_find+0xb0/0xb0
Jul 21 16:21:00.450475 [  530.412957]  privcmd_ioctl+0x253/0xa9b
Jul 21 16:21:00.450540 [  530.412996]  ? mmap_region+0x226/0x630
Jul 21 16:21:00.450592 [  530.413043]  ? selinux_mmap_file+0xb0/0xb0
Jul 21 16:21:00.462757 [  530.413084]  ? selinux_file_ioctl+0x15c/0x200
Jul 21 16:21:00.462823 [  530.413136]  do_vfs_ioctl+0x9f/0x630
Jul 21 16:21:00.474698 [  530.413177]  ksys_ioctl+0x5b/0x90
Jul 21 16:21:00.474762 [  530.413224]  __x64_sys_ioctl+0x11/0x20
Jul 21 16:21:00.474813 [  530.413264]  do_syscall_64+0x57/0x130
Jul 21 16:21:00.486480 [  530.413305]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
Jul 21 16:21:00.486548 [  530.413357] RIP: 0033:0x7f4f7ecde427
Jul 21 16:21:00.486600 [  530.413395] Code: 00 00 90 48 8b 05 69 aa 0c 00 64 c7 
00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 
00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 39 aa 0c 00 f7 d8 64 89 01 48
Jul 21 16:21:00.510766 [  530.413556] RSP: 002b:7ffc1ef6eb38 EFLAGS: 
0213 ORIG_RAX: 0010
Jul 21 16:21:00.522758 [  530.413629] RAX: ffda RBX: 
 RCX: 7f4f7ecde427
Jul 21 16:21:00.534632 [  530.413699] RDX: 7ffc1ef6eb90 RSI: 
00205004 RDI: 0007
Jul 21 16:21:00.534702 [  530.413810] RBP: 7ffc1ef6ebe0 R08: 
0007 R09: 
Jul 21 16:21:00.547013 [  530.413881] R10: 0001 R11: 
0213 R12: 55d754136200
Jul 21 16:21:00.558751 [  530.413951] R13: 7ffc1ef6f340 R14: 
 R15: 
Jul 21 16:21:00.558846 [  530.414079] Mem-Info:
Jul 21 16:21:00.558928 [  530.414123] active_anon:1724 inactive_anon:3931 
isolated_anon:0
Jul 21 16:21:00.570481 [  530.414123]  active_file:7862 inactive_file:86530 
isolated_file:0
Jul 21 16:21:00.582599 [  530.414123]  unevictable:0 dirty:18 writeback:0 
unstable:0
Jul 21 16:21:00.582668 [  530.414123]  slab_reclaimable:4704 
slab_unreclaimable:4036
Jul 21 16:21:00.594782 [  530.414123]  mapped:3461 shmem:124 pagetables:372 
bounce:0
Jul 21 16:21:00.594849 [  530.414123]  free:1863 free_pcp:16 free_cma:0
Jul 21 16:21:00.606733 [  530.414579] Node 0 active_anon:6896kB 
inactive_anon:15724kB active_file:31448kB inactive_file:346120kB 
unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:13844kB dirty:72kB 
writeback:0kB shmem:496kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
Jul 21 16:21:00.630626 [  530.414870] DMA free:1816kB min:92kB low:112kB 
high:132kB active_anon:0kB inactive_anon:0kB active_file:76kB 
inactive_file:9988kB unevictable:0kB writepending:0kB present:15980kB 
managed:14328kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB 
free_pcp:0kB local_pcp:0kB free_cma:0kB
Jul 21 16:21:00.658448 [  530.415329] lowmem_reserve[]: 0 431 431 431
Jul 21 16:21:00.658513 [  530.415404] DMA32 free:5512kB min:2608kB low:3260kB 
high:3912kB active_anon:6896kB inactive_anon:15724kB active_file:31372kB 
inactive_file:336132kB unevictable:0kB writepending:72kB present:508300kB 
managed:451760kB mlocked:0kB kernel_stack:2848kB pagetables:1488kB bounce:0kB 
free_pcp:184kB local_pcp:0kB free_cma:0kB
Jul 21 16:21:00.694702 [  530.415742] lowmem_reserve[]: 0 0 0 0
Jul 21 16:21:00.694778 [  530.415806] DMA: 8*4kB (UM) 3*8kB (UM) 4*16kB (UM) 
3*32kB (M) 5*64kB (UM) 2*128kB (UM) 4*256kB (UM) 0*512kB 0*1024kB 0*2048kB 
0*4096kB = 1816kB
Jul 21 16:21:00.706798 [  

Re: [xen-unstable test] 152067: regressions - trouble: fail/pass/starved

2020-07-22 Thread Jan Beulich
On 22.07.2020 02:37, osstest service owner wrote:
> flight 152067 xen-unstable real [real]
> http://logs.test-lab.xenproject.org/osstest/logs/152067/
> 
> Regressions :-(
> 
> Tests which did not succeed and are blocking,
> including tests which could not be run:
>  test-amd64-amd64-dom0pvh-xl-intel 18 guest-localmigrate/x10 fail REGR. vs. 
> 152045

Jul 21 16:20:58.985209 [  530.412043] libxl-save-help: page allocation failure: 
order:4, mode:0x60c0c0(GFP_KERNEL|__GFP_COMP|__GFP_ZERO), nodemask=(null)

My first reaction to this would be to ask if Dom0 was given too little
memory here. Or of course there could be a memory leak somewhere. But
the system isn't entirely out of memory (about 7Mb left), so perhaps
the "order:4" aspect here also plays a meaningful role. Hence ...

Jul 21 16:21:00.390810 [  530.412448] Call Trace:
Jul 21 16:21:00.402721 [  530.412499]  dump_stack+0x72/0x8c
Jul 21 16:21:00.402801 [  530.412541]  warn_alloc.cold.140+0x68/0xe8
Jul 21 16:21:00.402841 [  530.412585]  __alloc_pages_slowpath+0xc73/0xcb0
Jul 21 16:21:00.414737 [  530.412640]  ? __do_page_fault+0x249/0x4d0
Jul 21 16:21:00.414786 [  530.412681]  __alloc_pages_nodemask+0x235/0x250
Jul 21 16:21:00.426555 [  530.412734]  kmalloc_order+0x13/0x60
Jul 21 16:21:00.426619 [  530.412774]  kmalloc_order_trace+0x18/0xa0
Jul 21 16:21:00.426671 [  530.412816]  alloc_empty_pages.isra.15+0x24/0x60
Jul 21 16:21:00.438447 [  530.412867]  
privcmd_ioctl_mmap_batch.isra.18+0x303/0x320
Jul 21 16:21:00.438507 [  530.412918]  ? vmacache_find+0xb0/0xb0
Jul 21 16:21:00.450475 [  530.412957]  privcmd_ioctl+0x253/0xa9b

... perhaps we ought to consider re-working this code path to avoid
order > 0 allocations (may be as simple as switching to vmalloc(),
but I say this without having looked at the code).

Jan



Re: [PATCH v2 01/11] xen/manage: keep track of the on-going suspend mode

2020-07-22 Thread Roger Pau Monné
On Tue, Jul 21, 2020 at 07:55:09PM +, Anchal Agarwal wrote:
> On Tue, Jul 21, 2020 at 10:30:18AM +0200, Roger Pau Monné wrote:
> > CAUTION: This email originated from outside of the organization. Do not 
> > click links or open attachments unless you can confirm the sender and know 
> > the content is safe.
> > 
> > 
> > 
> > Marek: I'm adding you in case you could be able to give this a try and
> > make sure it doesn't break suspend for dom0.
> > 
> > On Tue, Jul 21, 2020 at 12:17:36AM +, Anchal Agarwal wrote:
> > > On Mon, Jul 20, 2020 at 11:37:05AM +0200, Roger Pau Monné wrote:
> > > > CAUTION: This email originated from outside of the organization. Do not 
> > > > click links or open attachments unless you can confirm the sender and 
> > > > know the content is safe.
> > > >
> > > >
> > > >
> > > > On Sat, Jul 18, 2020 at 09:47:04PM -0400, Boris Ostrovsky wrote:
> > > > > (Roger, question for you at the very end)
> > > > >
> > > > > On 7/17/20 3:10 PM, Anchal Agarwal wrote:
> > > > > > On Wed, Jul 15, 2020 at 05:18:08PM -0400, Boris Ostrovsky wrote:
> > > > > >> CAUTION: This email originated from outside of the organization. 
> > > > > >> Do not click links or open attachments unless you can confirm the 
> > > > > >> sender and know the content is safe.
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> On 7/15/20 4:49 PM, Anchal Agarwal wrote:
> > > > > >>> On Mon, Jul 13, 2020 at 11:52:01AM -0400, Boris Ostrovsky wrote:
> > > > >  CAUTION: This email originated from outside of the organization. 
> > > > >  Do not click links or open attachments unless you can confirm 
> > > > >  the sender and know the content is safe.
> > > > > 
> > > > > 
> > > > > 
> > > > >  On 7/2/20 2:21 PM, Anchal Agarwal wrote:
> > > > >  And PVH dom0.
> > > > > >>> That's another good use case to make it work with however, I still
> > > > > >>> think that should be tested/worked upon separately as the feature 
> > > > > >>> itself
> > > > > >>> (PVH Dom0) is very new.
> > > > > >>
> > > > > >> Same question here --- will this break PVH dom0?
> > > > > >>
> > > > > > I haven't tested it as a part of this series. Is that a blocker 
> > > > > > here?
> > > > >
> > > > >
> > > > > I suspect dom0 will not do well now as far as hibernation goes, in 
> > > > > which
> > > > > case you are not breaking anything.
> > > > >
> > > > >
> > > > > Roger?
> > > >
> > > > I sadly don't have any box ATM that supports hibernation where I
> > > > could test it. We have hibernation support for PV dom0, so while I
> > > > haven't done anything specific to support or test hibernation on PVH
> > > > dom0 I would at least aim to not make this any worse, and hence the
> > > > check should at least also fail for a PVH dom0?
> > > >
> > > > if (!xen_hvm_domain() || xen_initial_domain())
> > > > return -ENODEV;
> > > >
> > > > Ie: none of this should be applied to a PVH dom0, as it doesn't have
> > > > PV devices and hence should follow the bare metal device suspend.
> > > >
> > > So from what I understand you meant for any guest running on pvh dom0 
> > > should not
> > > hibernate if hibernation is triggered from within the guest or should 
> > > they?
> > 
> > Er no to both I think. What I meant is that a PVH dom0 should be able
> > to properly suspend, and we should make sure this work doesn't make
> > this any harder (or breaks it if it's currently working).
> > 
> > Or at least that's how I understood the question raised by Boris.
> > 
> > You are adding code to the generic suspend path that's also used by dom0
> > in order to perform bare metal suspension. This is fine now for a PV
> > dom0 because the code is gated on xen_hvm_domain, but you should also
> > take into account that a PVH dom0 is considered a HVM domain, and
> > hence will get the notifier registered.
> >
> Ok that makes sense now. This is good to be safe, but my patch series is only 
> to
> support domU hibernation, so I am not sure if this will affect pvh dom0.
> However, since I do not have a good way of testing sure I will add the check.
> 
> Moreover, in Patch-0004, I do register suspend/resume syscore_ops 
> specifically for domU
> hibernation only if its xen_hvm_domain.

So if the hooks are only registered for domU, do you still need this
xen_hvm_domain check here?

I have to admit I'm not familiar with Linux PM suspend.

> I don't see any reason that should not
> be registered for domU's running on pvh dom0.

To be clear: it should be registered for all HVM domUs, regardless of
whether they are running on a PV or a PVH dom0. My intention was never
to suggest otherwise. It should be enabled for all HVM domUs, but
shouldn't be enabled for HVM dom0.

> Those suspend/resume callbacks will
> only be invoked in case hibernation and will be skipped if generic suspend 
> path
> is in progress. Do you see any issue with that?

No, I think it's fine.

Roger.



[PATCH-for-5.2] hw/i386/q35: Remove unreachable Xen code on Q35 machine

2020-07-22 Thread Philippe Mathieu-Daudé
Xen accelerator requires specific changes to a machine to be able
to use it. See for example the 'Xen PC' machine configure its PCI
bus calling pc_xen_hvm_init_pci(). There is no 'Xen Q35' machine
declared. This code was probably added while introducing the Q35
machine, based on the existing PC machine (see commit df2d8b3ed4
"Introduce q35 pc based chipset emulator"). Remove the unreachable
code to simplify this file.

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/i386/pc_q35.c | 13 ++---
 1 file changed, 2 insertions(+), 11 deletions(-)

diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
index a3e607a544..12f5934241 100644
--- a/hw/i386/pc_q35.c
+++ b/hw/i386/pc_q35.c
@@ -34,9 +34,7 @@
 #include "sysemu/arch_init.h"
 #include "hw/i2c/smbus_eeprom.h"
 #include "hw/rtc/mc146818rtc.h"
-#include "hw/xen/xen.h"
 #include "sysemu/kvm.h"
-#include "sysemu/xen.h"
 #include "hw/kvm/clock.h"
 #include "hw/pci-host/q35.h"
 #include "hw/qdev-properties.h"
@@ -179,10 +177,6 @@ static void pc_q35_init(MachineState *machine)
 x86ms->below_4g_mem_size = machine->ram_size;
 }
 
-if (xen_enabled()) {
-xen_hvm_init(pcms, _memory);
-}
-
 x86_cpus_init(x86ms, pcmc->default_cpu_version);
 
 kvmclock_create();
@@ -208,10 +202,7 @@ static void pc_q35_init(MachineState *machine)
 }
 
 /* allocate ram and load rom/bios */
-if (!xen_enabled()) {
-pc_memory_init(pcms, get_system_memory(),
-   rom_memory, _memory);
-}
+pc_memory_init(pcms, get_system_memory(), rom_memory, _memory);
 
 /* create pci host bus */
 q35_host = Q35_HOST_DEVICE(qdev_new(TYPE_Q35_HOST_DEVICE));
@@ -271,7 +262,7 @@ static void pc_q35_init(MachineState *machine)
 
 assert(pcms->vmport != ON_OFF_AUTO__MAX);
 if (pcms->vmport == ON_OFF_AUTO_AUTO) {
-pcms->vmport = xen_enabled() ? ON_OFF_AUTO_OFF : ON_OFF_AUTO_ON;
+pcms->vmport = ON_OFF_AUTO_ON;
 }
 
 /* init basic PC hardware */
-- 
2.21.3




Re: Virtio in Xen on Arm (based on IOREQ concept)

2020-07-22 Thread Roger Pau Monné
On Tue, Jul 21, 2020 at 10:12:40PM +0100, Julien Grall wrote:
> Hi Oleksandr,
> 
> On 21/07/2020 19:16, Oleksandr wrote:
> > 
> > On 21.07.20 17:27, Julien Grall wrote:
> > > On a similar topic, I am a bit surprised you didn't encounter memory
> > > exhaustion when trying to use virtio. Because on how Linux currently
> > > works (see XSA-300), the backend domain as to have a least as much
> > > RAM as the domain it serves. For instance, you have serve two
> > > domains with 1GB of RAM each, then your backend would need at least
> > > 2GB + some for its own purpose.
> > > 
> > > This probably wants to be resolved by allowing foreign mapping to be
> > > "paging" out as you would for memory assigned to a userspace.
> > 
> > Didn't notice the last sentence initially. Could you please explain your
> > idea in detail if possible. Does it mean if implemented it would be
> > feasible to map all guest memory regardless of how much memory the guest
> > has?
> >
> > Avoiding map/unmap memory each guest request would allow us to have
> > better performance (of course with taking care of the fact that guest
> > memory layout could be changed)...
> 
> I will explain that below. Before let me comment on KVM first.
> 
> > Actually what I understand looking at kvmtool is the fact it does not
> > map/unmap memory dynamically, just calculate virt addresses according to
> > the gfn provided.
> 
> The memory management between KVM and Xen is quite different. In the case of
> KVM, the guest RAM is effectively memory from the userspace (allocated via
> mmap) and then shared with the guest.
> 
> From the userspace PoV, the guest memory will always be accessible from the
> same virtual region. However, behind the scene, the pages may not always
> reside in memory. They are basically managed the same way as "normal"
> userspace memory.
> 
> In the case of Xen, we are basically stealing a guest physical page
> allocated via kmalloc() and provide no facilities for Linux to reclaim the
> page if it needs to do it before the userspace decide to unmap the foreign
> mapping.
> 
> I think it would be good to handle the foreing mapping the same way as
> userspace memory. By that I mean, that Linux could reclaim the physical page
> used by the foreing mapping if it needs to.
> 
> The process for reclaiming the page would look like:
> 1) Unmap the foreign page
> 2) Ballon in the backend domain physical address used by the foreing
> mapping (allocate the page in the physmap)
> 
> The next time the userspace is trying to access the foreign page, Linux will
> receive a data abort that would result to:
> 1) Allocate a backend domain physical page
> 2) Balloon out the physical address (remove the page from the physmap)
> 3) Map the foreing mapping at the new guest physical address
> 4) Map the guest physical page in the userspace address space

This is going to shatter all the super pages in the stage-2
translation.

> With this approach, we should be able to have backend domain that can handle
> frontend domain without require a lot of memory.

Linux on x86 has the option to use empty hotplug memory ranges to map
foreign memory: the balloon driver hotplugs an unpopulated physical
memory range that's not made available to the OS free memory allocator
and it's just used as scratch space to map foreign memory. Not sure
whether Arm has something similar, or if it could be implemented.

You can still use the map-on-fault behaviour as above, but I would
recommend that you try to limit the number of hypercalls issued.
Having to issue a single hypercall for each page fault it's going to
be slow, so I would instead use mmap batch to map the hole range in
unpopulated physical memory and then the OS fault handler just needs to
fill the page tables with the corresponding address.

Roger.



[PATCH] x86/xen/time: set the X86_FEATURE_TSC_KNOWN_FREQ flag in xen_tsc_khz()

2020-07-22 Thread Hayato Ohhashi
If the TSC frequency is known from the pvclock page,
the TSC frequency does not need to be recalibrated.
We can avoid recalibration by setting X86_FEATURE_TSC_KNOWN_FREQ.

Signed-off-by: Hayato Ohhashi 
---
 arch/x86/xen/time.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c
index c8897aad13cd..91f5b330dcc6 100644
--- a/arch/x86/xen/time.c
+++ b/arch/x86/xen/time.c
@@ -39,6 +39,7 @@ static unsigned long xen_tsc_khz(void)
struct pvclock_vcpu_time_info *info =
_shared_info->vcpu_info[0].time;
 
+   setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
return pvclock_tsc_khz(info);
 }
 
-- 
2.23.3




[PATCH] xen-netfront: fix potential deadlock in xennet_remove()

2020-07-22 Thread Andrea Righi
There's a potential race in xennet_remove(); this is what the driver is
doing upon unregistering a network device:

  1. state = read bus state
  2. if state is not "Closed":
  3.request to set state to "Closing"
  4.wait for state to be set to "Closing"
  5.request to set state to "Closed"
  6.wait for state to be set to "Closed"

If the state changes to "Closed" immediately after step 1 we are stuck
forever in step 4, because the state will never go back from "Closed" to
"Closing".

Make sure to check also for state == "Closed" in step 4 to prevent the
deadlock.

Also add a 5 sec timeout any time we wait for the bus state to change,
to avoid getting stuck forever in wait_event() and add a debug message
to help tracking down potential similar issues.

Signed-off-by: Andrea Righi 
---
 drivers/net/xen-netfront.c | 79 +++---
 1 file changed, 57 insertions(+), 22 deletions(-)

diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c
index 482c6c8b0fb7..e09caba93dd9 100644
--- a/drivers/net/xen-netfront.c
+++ b/drivers/net/xen-netfront.c
@@ -63,6 +63,8 @@ module_param_named(max_queues, xennet_max_queues, uint, 0644);
 MODULE_PARM_DESC(max_queues,
 "Maximum number of queues per virtual interface");
 
+#define XENNET_TIMEOUT  (5 * HZ)
+
 static const struct ethtool_ops xennet_ethtool_ops;
 
 struct netfront_cb {
@@ -1334,12 +1336,20 @@ static struct net_device *xennet_create_dev(struct 
xenbus_device *dev)
 
netif_carrier_off(netdev);
 
-   xenbus_switch_state(dev, XenbusStateInitialising);
-   wait_event(module_wq,
-  xenbus_read_driver_state(dev->otherend) !=
-  XenbusStateClosed &&
-  xenbus_read_driver_state(dev->otherend) !=
-  XenbusStateUnknown);
+   do {
+   dev_dbg(>dev,
+   "%s: switching to XenbusStateInitialising\n",
+   dev->nodename);
+   xenbus_switch_state(dev, XenbusStateInitialising);
+   err = wait_event_timeout(module_wq,
+xenbus_read_driver_state(dev->otherend) !=
+XenbusStateClosed &&
+xenbus_read_driver_state(dev->otherend) !=
+XenbusStateUnknown, XENNET_TIMEOUT);
+   dev_dbg(>dev, "%s: state = %d\n", dev->nodename,
+   xenbus_read_driver_state(dev->otherend));
+   } while (!err);
+
return netdev;
 
  exit:
@@ -2139,28 +2149,53 @@ static const struct attribute_group xennet_dev_group = {
 };
 #endif /* CONFIG_SYSFS */
 
-static int xennet_remove(struct xenbus_device *dev)
+static void xennet_bus_close(struct xenbus_device *dev)
 {
-   struct netfront_info *info = dev_get_drvdata(>dev);
-
-   dev_dbg(>dev, "%s\n", dev->nodename);
+   int ret;
 
-   if (xenbus_read_driver_state(dev->otherend) != XenbusStateClosed) {
+   if (xenbus_read_driver_state(dev->otherend) == XenbusStateClosed)
+   return;
+   do {
+   dev_dbg(>dev, "%s: switching to XenbusStateClosing\n",
+   dev->nodename);
xenbus_switch_state(dev, XenbusStateClosing);
-   wait_event(module_wq,
-  xenbus_read_driver_state(dev->otherend) ==
-  XenbusStateClosing ||
-  xenbus_read_driver_state(dev->otherend) ==
-  XenbusStateUnknown);
+   ret = wait_event_timeout(module_wq,
+  xenbus_read_driver_state(dev->otherend) ==
+  XenbusStateClosing ||
+  xenbus_read_driver_state(dev->otherend) ==
+  XenbusStateClosed ||
+  xenbus_read_driver_state(dev->otherend) ==
+  XenbusStateUnknown,
+  XENNET_TIMEOUT);
+   dev_dbg(>dev, "%s: state = %d\n", dev->nodename,
+   xenbus_read_driver_state(dev->otherend));
+   } while (!ret);
+
+   if (xenbus_read_driver_state(dev->otherend) == XenbusStateClosed)
+   return;
 
+   do {
+   dev_dbg(>dev, "%s: switching to XenbusStateClosed\n",
+   dev->nodename);
xenbus_switch_state(dev, XenbusStateClosed);
-   wait_event(module_wq,
-  xenbus_read_driver_state(dev->otherend) ==
-  XenbusStateClosed ||
-  xenbus_read_driver_state(dev->otherend) ==
-  XenbusStateUnknown);
-   }
+   ret = wait_event_timeout(module_wq,
+  xenbus_read_driver_state(dev->otherend) ==
+  XenbusStateClosed ||
+ 

[linux-linus test] 152070: regressions - trouble: fail/pass/starved

2020-07-22 Thread osstest service owner
flight 152070 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/152070/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-xsm 16 guest-start/debian.repeat fail REGR. vs. 151214
 test-armhf-armhf-xl-vhd  11 guest-start  fail REGR. vs. 151214

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 151214
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 151214
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 151214
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 151214
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 151214
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 151214
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 151214
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 151214
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-qemuu-freebsd12-amd64  2 hosts-allocate   starved n/a
 test-amd64-amd64-qemuu-freebsd11-amd64  2 hosts-allocate   starved n/a

version targeted for testing:
 linux4fa640dc52302b5e62b01b05c755b055549633ae
baseline version:
 linux1b5044021070efa3259f3e9548dc35d1eb6aa844

Last test of basis   151214  2020-06-18 02:27:46 Z   34 days
Failing since151236  2020-06-19 19:10:35 Z   32 days   51 attempts
Testing same since   152070  2020-07-21 09:35:59 Z0 days1 attempts


830 people touched revisions