Re: [Xen-devel] q35 support in Xen

[Yi Sun]

On 17-09-19 13:35:19, Alexey G wrote:
> On Sun, 17 Sep 2017 10:49:12 +0800
> Yi Sun  wrote:
> 
> > On 17-09-15 23:12:58, Alexey G wrote:
> > > On Thu, 14 Sep 2017 16:39:32 +0800
> > > Yi Sun  wrote:
> > >   
> > > > Hi, Alexey,
> > > > 
> > > > Have you submitted the patches? If yes, could you please share the
> > > > link?  
> > > 
> > > Sorry, had a lot of work recently -- so far I've managed to submit only
> > > the bugfix for mentioned xen-mapcache issue with emulated AHCI DMA I/O,
> > > which was a main prerequisite for Q35 on Xen.
> > > 
> > > Right now I need to rebase Q35 support patches on recent changes --
> > > there were multiple commits to related parts of code in both Xen and
> > > QEMU while Q35 patches were initially made for Xen 4.8.0 (which I'm
> > > still using at the moment). I'll try rebasing Q35 patches on this
> > > weekend, hopefully there will be no big differences with 4.8.
> > >   
> > Thanks a lot for the update! Do you have a plan for the whole feature?
> > When do you expect all changes can be submitted or get merged?
>  
> Will send the whole set this week, once I refactor patches and write

That is great. Thank you!

> descriptions for them -- for at least one patch it will be a lenghty one,
> I'm afraid. Most Q35 support patches are depend on each other, so it's
> easier to send them in a single batch (one for QEMU and one for Xen in
> fact).
> 
Per my experiences, it would be better to split big patch into several small
pieces so that reviewers are easier to understand. Maintainer Wei, Liu provided
a good method before: you can write what the patch does into commit message,
then you can find if it is possible to split it into some small patches.
Hope it can help.

> Rebasing was done only for stable-4.9.0 currently. As patches will be sent
> as RFC first, I've decided to delay rebasing to latest staging/master a bit
> -- it will require a bit of effort to check the impact of some recent
> commits, so it may be a better idea to collect some feedback on RFC patches
> targeting stable-4.9.0 and rebase a more or less updated version to staging
> then.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [linux-linus test] 113583: regressions - FAIL

[osstest service owner]

flight 113583 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/113583/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-localmigrate/x10 fail REGR. vs. 
113497
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-localmigrate/x10 fail REGR. vs. 
113497
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-localmigrate/x10 fail REGR. vs. 
113497

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 18 guest-start/win.repeat fail blocked in 
113497
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 113497
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 113497
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 113497
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeatfail  like 113497
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 113497
 test-amd64-amd64-xl-qemut-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qcow2 12 migrate-support-checkfail  never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 13 guest-saverestore   fail never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 13 guest-saverestore   fail never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass

version targeted for testing:
 linuxebb2c2437d8008d46796902ff390653822af6cc4
baseline version:
 linux7318413077a5141a50a753b1fab687b7907eef16

Last test of basis   113497  2017-09-16 05:31:48 Z2 days
Failing since113516  2017-09-16 19:00:20 Z2 days6 attempts
Testing same since   113583  2017-09-18 19:54:12 Z0 days1 attempts


People who touched revisions under test:
  Adrian Hunter 
  Alexei Starovoitov 
  Andrey Konovalov 
  Arkadi Sharshevsky 
  Arnd Bergmann 
  Bastien Nocera 
  Ben Dooks 
  Borislav Petkov 
  Cameron Gutman 
  Christophe JAILLET 
  Colin Ian King 
  Cong Wang 
  Dan Carpenter 
  Daniel Borkmann 
  David Ahern 

Re: [Xen-devel] q35 support in Xen

[Alexey G]

On Sun, 17 Sep 2017 10:49:12 +0800
Yi Sun  wrote:

> On 17-09-15 23:12:58, Alexey G wrote:
> > On Thu, 14 Sep 2017 16:39:32 +0800
> > Yi Sun  wrote:
> >   
> > > Hi, Alexey,
> > > 
> > > Have you submitted the patches? If yes, could you please share the
> > > link?  
> > 
> > Sorry, had a lot of work recently -- so far I've managed to submit only
> > the bugfix for mentioned xen-mapcache issue with emulated AHCI DMA I/O,
> > which was a main prerequisite for Q35 on Xen.
> > 
> > Right now I need to rebase Q35 support patches on recent changes --
> > there were multiple commits to related parts of code in both Xen and
> > QEMU while Q35 patches were initially made for Xen 4.8.0 (which I'm
> > still using at the moment). I'll try rebasing Q35 patches on this
> > weekend, hopefully there will be no big differences with 4.8.
> >   
> Thanks a lot for the update! Do you have a plan for the whole feature?
> When do you expect all changes can be submitted or get merged?
 
Will send the whole set this week, once I refactor patches and write
descriptions for them -- for at least one patch it will be a lenghty one,
I'm afraid. Most Q35 support patches are depend on each other, so it's
easier to send them in a single batch (one for QEMU and one for Xen in
fact).

Rebasing was done only for stable-4.9.0 currently. As patches will be sent
as RFC first, I've decided to delay rebasing to latest staging/master a bit
-- it will require a bit of effort to check the impact of some recent
commits, so it may be a better idea to collect some feedback on RFC patches
targeting stable-4.9.0 and rebase a more or less updated version to staging
then.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-unstable test] 113582: regressions - FAIL

[osstest service owner]

flight 113582 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/113582/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-i386-xsm6 xen-buildfail REGR. vs. 113387
 test-armhf-armhf-xl-cubietruck  6 xen-installfail REGR. vs. 113387
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-localmigrate/x10 fail REGR. vs. 
113387
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-localmigrate/x10 fail REGR. vs. 
113387
 test-amd64-amd64-xl-credit2  15 guest-saverestorefail REGR. vs. 113387

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm  1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked 
n/a
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 113387
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 113387
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-localmigrate/x10 fail like 113387
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 113387
 test-armhf-armhf-xl-rtds 12 guest-start  fail  like 113387
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 113387
 test-amd64-amd64-xl-qemut-ws16-amd64 10 windows-installfail never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 13 guest-saverestore   fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-amd64-i386-xl-qemut-ws16-amd64 13 guest-saverestore   fail never pass
 test-amd64-i386-libvirt-qcow2 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass

version targeted for testing:
 xen  e25fc4cba8439c068a05f29531811cba71069837
baseline version:
 xen  16b1414de91b5a82a0996c67f6db3af7d7e32873

Last test of basis   113387  2017-09-12 23:20:09 Z6 days
Failing since113430  2017-09-14 01:24:48 Z5 days9 attempts
Testing same since   113582  2017-09-18 16:51:18 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Bhupinder Thakur 
  Boris Ostrovsky 
  Dario Faggioli 
  George Dunlap 
  Haozhong Zhang 
  Ian Jackson 
  Jan Beulich 
  Julien Grall 
  Kevin Tian 
  Oleksandr Grytsov 
  

Re: [Xen-devel] [RFC PATCH V3 1/3] Xen: Increase hap/shadow page pool size to support more vcpus support

[Lan Tianyu]

Hi Wei:

On 2017年09月18日 21:06, Wei Liu wrote:
> On Wed, Sep 13, 2017 at 12:52:47AM -0400, Lan Tianyu wrote:
>> This patch is to increase page pool size when max vcpu number is larger
>> than 128.
>>
>> Signed-off-by: Lan Tianyu 
>> ---
>>  xen/arch/arm/domain.c|  5 +
>>  xen/arch/x86/domain.c| 25 +
>>  xen/common/domctl.c  |  3 +++
>>  xen/include/xen/domain.h |  2 ++
>>  4 files changed, 35 insertions(+)
>>
>> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
>> index 6512f01..94cf70b 100644
>> --- a/xen/arch/arm/domain.c
>> +++ b/xen/arch/arm/domain.c
>> @@ -824,6 +824,11 @@ int arch_vcpu_reset(struct vcpu *v)
>>  return 0;
>>  }
>>  
>> +int arch_domain_set_max_vcpus(struct domain *d)
>> +{
>> +return 0;
>> +}
>> +
>>  static int relinquish_memory(struct domain *d, struct page_list_head *list)
>>  {
>>  struct page_info *page, *tmp;
>> diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
>> index dbddc53..0e230f9 100644
>> --- a/xen/arch/x86/domain.c
>> +++ b/xen/arch/x86/domain.c
>> @@ -1161,6 +1161,31 @@ int arch_vcpu_reset(struct vcpu *v)
>>  return 0;
>>  }
>>  
>> +int arch_domain_set_max_vcpus(struct domain *d)
> 
> The name doesn't match what the function does.
> 

I originally hoped to introduce a hook for each arch when set max vcpus.
Each arch function can do customized thing and so named
"arch_domain_set_max_vcpus".

How about "arch_domain_setup_vcpus_resource"?


>> +{
>> +int ret;
>> +
>> +/* Increase page pool in order to support more vcpus. */
>> +if ( d->max_vcpus > 128 )
>> +{
>> +unsigned long nr_pages;
>> +
>> +if (hap_enabled(d))
> 
> Coding style.

Will update. Thanks.

> 
>> +nr_pages = 1024;
>> +else
>> +nr_pages = 4096;
>> +
>> +ret = paging_set_allocation(d, nr_pages, NULL);
> 
> Does this work on PV guests?


Sorry. This code should not run for PV guest. Will add a domain type
check here.

> 
>> +if ( ret != 0 )
>> +{
>> +paging_set_allocation(d, 0, NULL);
>> +return ret;
>> +}
>> +}
>> +
>> +return 0;
>> +}
>> +
>>  long
>>  arch_do_vcpu_op(
>>  int cmd, struct vcpu *v, XEN_GUEST_HANDLE_PARAM(void) arg)
>> diff --git a/xen/common/domctl.c b/xen/common/domctl.c
>> index 42658e5..64357a3 100644
>> --- a/xen/common/domctl.c
>> +++ b/xen/common/domctl.c
>> @@ -631,6 +631,9 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) 
>> u_domctl)
>>  d->max_vcpus = max;
>>  }
>>  
>> +if ( arch_domain_set_max_vcpus(d) < 0)
> 
> != 0 please.
> 

Sure.

>> +goto maxvcpu_out;
>> +
>>  for ( i = 0; i < max; i++ )
>>  {
>>  if ( d->vcpu[i] != NULL )
>> diff --git a/xen/include/xen/domain.h b/xen/include/xen/domain.h
>> index 347f264..e1ece3a 100644
>> --- a/xen/include/xen/domain.h
>> +++ b/xen/include/xen/domain.h
>> @@ -81,6 +81,8 @@ void arch_dump_domain_info(struct domain *d);
>>  
>>  int arch_vcpu_reset(struct vcpu *);
>>  
>> +int arch_domain_set_max_vcpus(struct domain *d);
>> +
>>  extern spinlock_t vcpu_alloc_lock;
>>  bool_t domctl_lock_acquire(void);
>>  void domctl_lock_release(void);
>> -- 
>> 1.8.3.1
>>


-- 
Best regards
Tianyu Lan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH] xen: credit2: fix spinlock irq-safety violation

[Dario Faggioli]

In commit ad4b3e1e9df34 ("xen: credit2: implement
utilization cap") xfree() was being called (for
deallocating the budget replenishment timer, during
domain destruction) inside an IRQ disabled critical
section.

That must not happen, as it uses the mem-pool's lock,
which needs to be taken with IRQ enabled. And, in fact,
we crash (in debug builds):

(XEN) 
(XEN) Panic on CPU 0:
(XEN) Xen BUG at spinlock.c:47
(XEN) 

Let's, therefore, kill and deallocate the timer outside of
the critical sections, when IRQs are enabled.

Signed-off-by: Dario Faggioli 
---
Cc: osstest service owner 
Cc: George Dunlap 
Cc: Wei Liu 
---
This was spotted by OSSTest's flight 113562:

 http://logs.test-lab.xenproject.org/osstest/logs/113562/
 
http://logs.test-lab.xenproject.org/osstest/logs/113562/test-amd64-amd64-xl-credit2/serial-godello0.log
---
 xen/common/sched_credit2.c |6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index 32234ac..7a550db 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -2923,13 +2923,13 @@ csched2_free_domdata(const struct scheduler *ops, void 
*data)
 
 write_lock_irqsave(>lock, flags);
 
-kill_timer(sdom->repl_timer);
-xfree(sdom->repl_timer);
-
 list_del_init(>sdom_elem);
 
 write_unlock_irqrestore(>lock, flags);
 
+kill_timer(sdom->repl_timer);
+xfree(sdom->repl_timer);
+
 xfree(data);
 }
 


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 09/17] livepatch/arm[32, 64]: Modify livepatch_funcs

[Konrad Rzeszutek Wilk]

On Thu, Sep 14, 2017 at 02:20:42PM +0100, Julien Grall wrote:
> Hi Konrad,
> 
> On 12/09/17 01:37, Konrad Rzeszutek Wilk wrote:
> > This was found when porting livepatch-build-tools to ARM64/32.
> > 
> > When livepatch-build-tools are built (and test-case thanks to:
> > livepatch/tests: Make sure all .livepatch.funcs sections are read-only)
> > the .livepatch.funcs are in read-only section.
> > 
> > However the hypervisor uses the 'opaque' for its own purpose, that
> > is stashing the original code. But the .livepatch_funcs section is
> > in the RO vmap area so on ARM[32,64] we get a fault.
> 
> This is because the payload is secure at loading and therefore before it get
> applied, right?

Yes.
> 
> I was wondering if we could either defer the call to secure_payload or make
> the region temporarily writeable?

This patch creates a temporary writeable virtual address space.

But the idea of making the region temporarily writeable is also possible.
Is there a specific register I can use for this?

> 
> > 
> > On x86 the same protection is in place. In 'arch_livepatch_quiesce'
> > we disable WP to allow changes to read-only pages (and in arch_live_resume
> 
> I can't find any function call arch_live_resume in Xen code. Do you mean
> arch_livepatch_revive?

Yes, let me update that.
> 
> > we enable the WP protection).
> > 
> > On ARM[32,64] we do not have the luxury of a global register that can
> > be changed after boot. In lieu of that we use the vmap to create
> > a temporary virtual address in which we can use instead.
> > 
> > To do this we need to stash during livepatch: vmap of the hypervisor
> > code, vmap of the .livepatch_funcs (vmap comes in page aligned virtual
> > addresses), offset in the vmap (in case it is not nicely aligned), and
> > the original first livepatch_funcs to figure out the index.
> > 
> > Equipped with that we can patch livepatch functions which have
> >   .livepatch_funcs in rodata section.
> > 
> > An alternative is to add the 'W' flag during loading of the
> > .livepatch_funcs which would result the section being in writeable
> > region from the gecko. >
> > Note that this vmap solution could be extended to x86 as well.

And also this, as there is more to it (As Andrew pointed out).
> > 
> > Signed-off-by: Konrad Rzeszutek Wilk 
> > ---
> >   xen/arch/arm/arm32/livepatch.c  | 11 ++---
> >   xen/arch/arm/arm64/livepatch.c  | 11 ++---
> >   xen/arch/arm/livepatch.c| 52 
> > -
> >   xen/arch/x86/livepatch.c|  2 +-
> >   xen/common/livepatch.c  |  5 ++--
> >   xen/include/asm-arm/livepatch.h | 13 ---
> >   xen/include/xen/livepatch.h |  2 +-
> >   7 files changed, 71 insertions(+), 25 deletions(-)
> > 
> > diff --git a/xen/arch/arm/arm32/livepatch.c b/xen/arch/arm/arm32/livepatch.c
> > index 10887ace81..d793ebcaad 100644
> > --- a/xen/arch/arm/arm32/livepatch.c
> > +++ b/xen/arch/arm/arm32/livepatch.c
> > @@ -16,18 +16,23 @@ void arch_livepatch_apply(struct livepatch_func *func)
> >   uint32_t insn;
> >   uint32_t *new_ptr;
> >   unsigned int i, len;
> > +struct livepatch_func *f;
> >   BUILD_BUG_ON(ARCH_PATCH_INSN_SIZE > sizeof(func->opaque));
> >   BUILD_BUG_ON(ARCH_PATCH_INSN_SIZE != sizeof(insn));
> > -ASSERT(vmap_of_xen_text);
> > +ASSERT(livepatch_vmap.text);
> >   len = livepatch_insn_len(func);
> >   if ( !len )
> >   return;
> > +/* Index in the vmap region. */
> > +i = livepatch_vmap.va - func;
> > +f = (struct livepatch_func *)(livepatch_vmap.funcs + 
> > livepatch_vmap.offset) + i;
> > +
> >   /* Save old ones. */
> > -memcpy(func->opaque, func->old_addr, len);
> > +memcpy(f->opaque, func->old_addr, len);
> >   if ( func->new_addr )
> >   {
> > @@ -56,7 +61,7 @@ void arch_livepatch_apply(struct livepatch_func *func)
> >   else
> >   insn = 0xe1a0; /* mov r0, r0 */
> > -new_ptr = func->old_addr - (void *)_start + vmap_of_xen_text;
> > +new_ptr = func->old_addr - (void *)_start + livepatch_vmap.text;
> >   len = len / sizeof(uint32_t);
> >   /* PATCH! */
> > diff --git a/xen/arch/arm/arm64/livepatch.c b/xen/arch/arm/arm64/livepatch.c
> > index 2728e2a125..662bedabc3 100644
> > --- a/xen/arch/arm/arm64/livepatch.c
> > +++ b/xen/arch/arm/arm64/livepatch.c
> > @@ -20,18 +20,23 @@ void arch_livepatch_apply(struct livepatch_func *func)
> >   uint32_t insn;
> >   uint32_t *new_ptr;
> >   unsigned int i, len;
> > +struct livepatch_func *f;
> >   BUILD_BUG_ON(ARCH_PATCH_INSN_SIZE > sizeof(func->opaque));
> >   BUILD_BUG_ON(ARCH_PATCH_INSN_SIZE != sizeof(insn));
> > -ASSERT(vmap_of_xen_text);
> > +ASSERT(livepatch_vmap.text);
> >   len = livepatch_insn_len(func);
> >   if ( !len )
> >   return;
> > +/* Index in the vmap region. */
> > +i = livepatch_vmap.va - func;
> > +f = (struct livepatch_func 

Re: [Xen-devel] [PATCH v3 08/17] livepatch/tests: Make sure all .livepatch.funcs sections are read-only

[Konrad Rzeszutek Wilk]

On Tue, Sep 12, 2017 at 08:49:55AM -0600, Jan Beulich wrote:
> >>> On 12.09.17 at 02:37,  wrote:
> > --- a/xen/test/livepatch/Makefile
> > +++ b/xen/test/livepatch/Makefile
> > @@ -54,6 +54,7 @@ xen_hello_world.o: config.h livepatch_depends.h
> >  $(LIVEPATCH): xen_hello_world_func.o xen_hello_world.o
> > $(LD) $(LDFLAGS) $(build_id_linker) -r -o $(LIVEPATCH) $^
> > $(OBJCOPY) --strip-debug --strip-symbol=$(NOTE_SYMBOL) $@
> > +   $(OBJCOPY) --set-section-flags .livepatch.funcs=alloc,readonly $@
> 
> Why multiple objcopy invocations?

 I honestly have no idea. I converted it over to be  \
and then have --set-section-flags on the same column as --strip-debug.
> 
> Jan
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 06/17] xen/livepatch/x86/arm32: Force .livepatch.depends section to be uint32_t aligned.

[Konrad Rzeszutek Wilk]

> > +.PHONY: livepatch_depends.h
> > +livepatch_depends.h: note.bin
> > +   $(shell (../../../tools/firmware/hvmloader/mkhex $(NOTE_DEPENDS) $^ > 
> > $@))
> 
> It looks quite odd to use a file in firmware/hvmloader for livepatch. Would
> it be possible to move mkhex to a generic place?

Like so?

diff --git a/tools/firmware/hvmloader/Makefile 
b/tools/firmware/hvmloader/Makefile
index 7c4c0ce535..a5b4c32c1a 100644
--- a/tools/firmware/hvmloader/Makefile
+++ b/tools/firmware/hvmloader/Makefile
@@ -91,23 +91,23 @@ roms.inc: $(ROMS)
 
 ifneq ($(ROMBIOS_ROM),)
echo "#ifdef ROM_INCLUDE_ROMBIOS" >> $@.new
-   sh ./mkhex rombios $(ROMBIOS_ROM) >> $@.new
+   sh ../../misc/mkhex rombios $(ROMBIOS_ROM) >> $@.new
echo "#endif" >> $@.new
 endif
 
 ifneq ($(STDVGA_ROM),)
echo "#ifdef ROM_INCLUDE_VGABIOS" >> $@.new
-   sh ./mkhex vgabios_stdvga $(STDVGA_ROM) >> $@.new
+   sh ../../misc/mkhex vgabios_stdvga $(STDVGA_ROM) >> $@.new
echo "#endif" >> $@.new
 endif
 ifneq ($(CIRRUSVGA_ROM),)
echo "#ifdef ROM_INCLUDE_VGABIOS" >> $@.new
-   sh ./mkhex vgabios_cirrusvga $(CIRRUSVGA_ROM) >> $@.new
+   sh ../../misc/mkhex vgabios_cirrusvga $(CIRRUSVGA_ROM) >> $@.new
echo "#endif" >> $@.new
 endif
 ifneq ($(ETHERBOOT_ROMS),)
echo "#ifdef ROM_INCLUDE_ETHERBOOT" >> $@.new
-   sh ./mkhex etherboot $(ETHERBOOT_ROMS) >> $@.new
+   sh ../../misc/mkhex etherboot $(ETHERBOOT_ROMS) >> $@.new
echo "#endif" >> $@.new
 endif
 
diff --git a/tools/firmware/hvmloader/mkhex b/tools/misc/mkhex
similarity index 100%
rename from tools/firmware/hvmloader/mkhex
rename to tools/misc/mkhex
diff --git a/xen/test/livepatch/Makefile b/xen/test/livepatch/Makefile
index 8ac9f5e426..f0365305ba 100644
--- a/xen/test/livepatch/Makefile
+++ b/xen/test/livepatch/Makefile
@@ -73,7 +73,7 @@ note.bin:
 
 .PHONY: livepatch_depends.h
 livepatch_depends.h: note.bin
-   $(shell (../../../tools/firmware/hvmloader/mkhex $(NOTE_DEPENDS) $^ > 
$@))
+   $(shell (../../../tools/misc/mkhex $(NOTE_DEPENDS) $^ > $@))
 
 #
 # Extract the build-id of the xen_hello_world.livepatch
@@ -85,7 +85,7 @@ hello_world_note.bin: $(LIVEPATCH)
 
 .PHONY: hello_world_livepatch_depends.h
 hello_world_livepatch_depends.h: hello_world_note.bin
-   $(shell (../../../tools/firmware/hvmloader/mkhex $(NOTE_DEPENDS) $^ > 
$@))
+   $(shell (../../../tools/misc/mkhex $(NOTE_DEPENDS) $^ > $@))
 
 xen_bye_world.o: config.h hello_world_livepatch_depends.h
 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-unstable-smoke test] 113584: tolerable all pass - PUSHED

[osstest service owner]

flight 113584 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/113584/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  cd02f96d54813139e14e2847566d744358b55c1c
baseline version:
 xen  9fbd31071538a1c696f2b75d1d91a83e72113d32

Last test of basis   113581  2017-09-18 16:01:21 Z0 days
Testing same since   113584  2017-09-18 22:01:16 Z0 days1 attempts


People who touched revisions under test:
  Jan Beulich 
  Konrad Rzeszutek Wilk 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=xen-unstable-smoke
+ revision=cd02f96d54813139e14e2847566d744358b55c1c
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
 export PERLLIB=.:.
 PERLLIB=.:.
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push xen-unstable-smoke 
cd02f96d54813139e14e2847566d744358b55c1c
+ branch=xen-unstable-smoke
+ revision=cd02f96d54813139e14e2847566d744358b55c1c
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
 export PERLLIB=.:.:.
 PERLLIB=.:.:.
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
+++ export PERLLIB=.:.:.:.
+++ PERLLIB=.:.:.:.
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=xen
+ xenbranch=xen-unstable-smoke
+ qemuubranch=qemu-upstream-unstable
+ '[' xxen = xlinux ']'
+ linuxbranch=
+ '[' xqemu-upstream-unstable = x ']'
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable-smoke
+ prevxenbranch=xen-4.9-testing
+ '[' xcd02f96d54813139e14e2847566d744358b55c1c = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/xtf.git
++ : osst...@xenbits.xen.org:/home/xen/git/xtf.git
++ : git://xenbits.xen.org/xtf.git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/rumprun.git
++ : git://git.seabios.org/seabios.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/seabios.git
++ : git://xenbits.xen.org/osstest/seabios.git
++ : https://github.com/tianocore/edk2.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/ovmf.git
++ : 

[Xen-devel] [qemu-mainline test] 113580: regressions - FAIL

[osstest service owner]

flight 113580 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/113580/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-localmigrate/x10 fail REGR. vs. 
113302
 test-amd64-amd64-xl-multivcpu 20 guest-start/debian.repeat fail REGR. vs. 
113302

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop   fail REGR. vs. 113302

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 113302
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 113302
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 113302
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 113302
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qcow2 12 migrate-support-checkfail  never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 13 guest-saverestore   fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass

version targeted for testing:
 qemuua9158a5cba955b79d580a252cc58ff44d154e370
baseline version:
 qemuua6e8c1dacfd37d34542e33600dcc50b7683b735a

Last test of basis   113302  2017-09-11 10:18:16 Z7 days
Failing since113345  2017-09-12 00:21:07 Z6 days   12 attempts
Testing same since   113580  2017-09-18 13:19:38 Z0 days1 attempts


People who touched revisions under test:
  Alex Bennée 
  Alexander Graf 
  Alexey Kardashevskiy 
  Alistair Francis 
  Amador Pahim 
  Benjamin Herrenschmidt 
  Cornelia Huck 
  Cédric Le Goater 
  Daniel Henrique Barboza 
  David Gibson 
  Dr. David Alan Gilbert 
  Eduardo Habkost 
  Eduardo Otubo 
  Eduardo Otubo 
  Eric Auger 
  Eric Blake 
  Fam Zheng 
  Feng Kan 
  Gerd Hoffmann 
  Greg Kurz 
  Jaroslaw Pelczar 
  Kevin Wolf 
  Laurent Vivier 
  Lluís Vilanova 
  Lukáš Doktor 
  Mark Cave-Ayland 
  Matt 

[Xen-devel] [PATCH] xen, arm64: drop dummy lookup_address()

[Tycho Andersen]

This is unused, and conflicts with the definition that we'll add for XPFO.

Signed-off-by: Tycho Andersen 
CC: Boris Ostrovsky 
CC: Juergen Gross 
CC: Stefano Stabellini 
---
The patch this depends on is in for-linus-4.14b, so it would be easiest to
carry this one too; Stefano can you ack it and Boris can you carry it?

Thanks!
---
 include/xen/arm/page.h | 10 --
 1 file changed, 10 deletions(-)

diff --git a/include/xen/arm/page.h b/include/xen/arm/page.h
index 415dbc6e43fd..6adc2a955340 100644
--- a/include/xen/arm/page.h
+++ b/include/xen/arm/page.h
@@ -84,16 +84,6 @@ static inline xmaddr_t arbitrary_virt_to_machine(void *vaddr)
BUG();
 }
 
-/* TODO: this shouldn't be here but it is because the frontend drivers
- * are using it (its rolled in headers) even though we won't hit the code path.
- * So for right now just punt with this.
- */
-static inline pte_t *lookup_address(unsigned long address, unsigned int *level)
-{
-   BUG();
-   return NULL;
-}
-
 extern int set_foreign_p2m_mapping(struct gnttab_map_grant_ref *map_ops,
   struct gnttab_map_grant_ref *kmap_ops,
   struct page **pages, unsigned int count);
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [linux-next test] 113577: regressions - FAIL

[osstest service owner]

flight 113577 linux-next real [real]
http://logs.test-lab.xenproject.org/osstest/logs/113577/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-libvirt-qcow2 17 guest-start/debian.repeat fail REGR. vs. 
113576
 test-armhf-armhf-xl-arndale  10 debian-install   fail REGR. vs. 113576
 test-armhf-armhf-xl-credit2  10 debian-install   fail REGR. vs. 113576
 test-armhf-armhf-libvirt-xsm 10 debian-install   fail REGR. vs. 113576
 test-armhf-armhf-libvirt-raw 10 debian-di-installfail REGR. vs. 113576
 test-armhf-armhf-xl  10 debian-install   fail REGR. vs. 113576

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop   fail REGR. vs. 113565
 test-armhf-armhf-xl-rtds 10 debian-install   fail REGR. vs. 113576

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop   fail blocked in 113576
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 113565
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 113576
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 10 windows-installfail never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qcow2 12 migrate-support-checkfail  never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 13 guest-saverestore   fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 13 guest-saverestore   fail never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass

version targeted for testing:
 linuxfc2e8b1a47c14b22c33eb087fca0db58e1f4ed0e
baseline version:
 linux0666f560b71b899cd11a7caf39fd45129e9030fd

Last test of basis  (not found) 
Failing since   (not found) 
Testing same since   113577  2017-09-18 09:25:39 Z0 days1 attempts

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 build-amd64-rumprun  pass
 build-i386-rumprun   pass
 test-amd64-amd64-xl  pass
 test-armhf-armhf-xl  fail
 test-amd64-i386-xl   pass
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsmpass
 

Re: [Xen-devel] [PATCH] xen: don't compile pv-specific parts if XEN_PV isn't configured

[Boris Ostrovsky]

On 09/14/2017 08:38 AM, Juergen Gross wrote:
> xenbus_client.c contains some functions specific for pv guests.
> Enclose them with #ifdef CONFIG_XEN_PV to avoid compiling them when
> they are not needed (e.g. on ARM).
>
> Signed-off-by: Juergen Gross 

Applied to for-linus-14b.

-boris


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 0/4] xen: sched: optimize exclusive pinning and soft-affinity checking

[Anshul Makkar]

On 9/15/17 6:35 PM, Dario Faggioli wrote:

Hello,

Dario Faggioli (4):
   xen: sched: introduce 'adjust_affinity' hook.
   xen: sched: optimize exclusive pinning case (Credit1 & 2)
   xen: sched: improve checking soft-affinity
   xen: sched: simplify (and speedup) checking soft-affinity

  xen/arch/x86/dom0_build.c|4 +
  xen/common/sched_credit.c|  114 +++---
  xen/common/sched_credit2.c   |   50 --
  xen/common/sched_null.c  |8 +--
  xen/common/schedule.c|   67 +++--
  xen/include/xen/perfc_defn.h |1
  xen/include/xen/sched-if.h   |   16 +++---
  xen/include/xen/sched.h  |5 ++
  8 files changed, 188 insertions(+), 77 deletions(-)
--
Please can you share the link  for the branch on which these changes 
have been done .


Anshul

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [linux-linus test] 113576: regressions - FAIL

[osstest service owner]

flight 113576 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/113576/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-localmigrate/x10 fail REGR. vs. 
113497
 build-armhf-xsm   6 xen-build  fail in 113565 REGR. vs. 113497

Tests which are failing intermittently (not blocking):
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-localmigrate/x10 fail in 113565 
pass in 113576
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-localmigrate/x10 fail pass in 
113565
 test-amd64-amd64-xl-qemuu-win7-amd64 10 windows-installfail pass in 113565

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-xsm  1 build-check(1)   blocked in 113565 n/a
 test-armhf-armhf-xl-xsm   1 build-check(1)   blocked in 113565 n/a
 test-amd64-amd64-xl-qemuu-win7-amd64 18 guest-start/win.repeat fail in 113565 
like 113497
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop  fail in 113565 like 113497
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 113497
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 113497
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 113497
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 113497
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeatfail  like 113497
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 113497
 test-amd64-amd64-xl-qemut-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qcow2 12 migrate-support-checkfail  never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 13 guest-saverestore   fail never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 13 guest-saverestore   fail never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass

version targeted for testing:
 linux0666f560b71b899cd11a7caf39fd45129e9030fd
baseline version:
 linux7318413077a5141a50a753b1fab687b7907eef16

Last test of basis   113497  2017-09-16 05:31:48 Z2 days
Failing since113516  2017-09-16 19:00:20 Z2 days5 attempts
Testing same since   113565  2017-09-18 00:49:23 Z0 days2 attempts


People who touched revisions under test:
  Alexei Starovoitov 
  Andrey Konovalov 

Re: [Xen-devel] How to prepare the COLO test environment

[Zhang Chen]

山本真吾 于2017年9月18日周一 下午6:27写道：

> Hello,
>
> I am looking for ways to try out COLO.
> I tried a wiki and mailing list, but I failed.
> I do not know whether the wiki and the mailing list are correct.
> Could you tell me the latest way to try it?
>
> I have read the following documents:
>
> COLO - Coarse Grain Lock Stepping
> https://wiki.xen.org/wiki/COLO_-_Coarse_Grain_Lock_Stepping
>
> Mailing list
> https://lists.xen.org/archives/html/xen-devel/2016-03/msg00376.html
>
> If there are some mistakes in English, I'd like apologize.
> Thank you.
>

Unfortunately, wiki and the mail can't be updated. Latest COLO use user
space COLO-Proxy in qemu,
Wiki just guide you to setup kernel space COLO-Proxy(this version
COLO-proxy not being maintained).
For some reason, Xen COLO need use some KVM COLO codes, but we haven't push
all qemu code
to upstream, just in progress.
So, you can try our internal latest version:
Xen:
https://github.com/zhangckid/xen/tree/xen-colo-v2
Qemu:
https://github.com/zhangckid/qemu/tree/qemu-colo-for-xen

The start scripts is same with wiki (you just need remove the modprobe
commands).
Thank you for the question, I will update Xen wiki this week.

Thanks
Zhang Chen


>
> Yamamoto
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel
>
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-unstable-smoke test] 113581: tolerable all pass - PUSHED

[osstest service owner]

flight 113581 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/113581/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  9fbd31071538a1c696f2b75d1d91a83e72113d32
baseline version:
 xen  e25fc4cba8439c068a05f29531811cba71069837

Last test of basis   113579  2017-09-18 11:02:17 Z0 days
Testing same since   113581  2017-09-18 16:01:21 Z0 days1 attempts


People who touched revisions under test:
  Ian Jackson 
  Wei Liu 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=xen-unstable-smoke
+ revision=9fbd31071538a1c696f2b75d1d91a83e72113d32
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
 export PERLLIB=.:.
 PERLLIB=.:.
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push xen-unstable-smoke 
9fbd31071538a1c696f2b75d1d91a83e72113d32
+ branch=xen-unstable-smoke
+ revision=9fbd31071538a1c696f2b75d1d91a83e72113d32
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
 export PERLLIB=.:.:.
 PERLLIB=.:.:.
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
+++ export PERLLIB=.:.:.:.
+++ PERLLIB=.:.:.:.
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=xen
+ xenbranch=xen-unstable-smoke
+ qemuubranch=qemu-upstream-unstable
+ '[' xxen = xlinux ']'
+ linuxbranch=
+ '[' xqemu-upstream-unstable = x ']'
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable-smoke
+ prevxenbranch=xen-4.9-testing
+ '[' x9fbd31071538a1c696f2b75d1d91a83e72113d32 = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/xtf.git
++ : osst...@xenbits.xen.org:/home/xen/git/xtf.git
++ : git://xenbits.xen.org/xtf.git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/rumprun.git
++ : git://git.seabios.org/seabios.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/seabios.git
++ : git://xenbits.xen.org/osstest/seabios.git
++ : https://github.com/tianocore/edk2.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/ovmf.git
++ : 

Re: [Xen-devel] [xen-unstable test] 113562: regressions - FAIL

[Dario Faggioli]

On Mon, 2017-09-18 at 11:29 +0100, George Dunlap wrote:
> > > On Mon, Sep 18, 2017 at 10:37:58AM +0100, Wei Liu wrote:
> > > > On Mon, Sep 18, 2017 at 08:36:03AM +, osstest service owner
> > > > wrote:
> > > > > flight 113562 xen-unstable real [real]
> > > > > http://logs.test-lab.xenproject.org/osstest/logs/113562/
> > > > > 
> > > > > Regressions :-(
> > > > > 
> > > > > Tests which did not succeed and are blocking,
> > > > > including tests which could not be run:
> > > > >  test-amd64-amd64-xl-credit2  15 guest-
> > > > > saverestorefail REGR. vs. 113387
> > > > 
> > > > There appears to be a bug:
> > > > 
> > > > http://logs.test-lab.xenproject.org/osstest/logs/113562/test-am
> > > > d64-amd64-xl-credit2/serial-godello0.log
> > > > 
> > > > Sep 18 01:14:28.803062 (XEN) Xen BUG at spinlock.c:47
> > > 
> In fact in this case it appears to be the xfree(sdom->repl_timer) in
> csched2_free_domdata() being inside the critical section (which
> disables
> irqs); there's actually an xfree() right in that function outside the
> critical section.
> 
So, during the afternoon, there was an glitch here, in the local
network/NAS of my home office.

It took me a bit to fix it, and that delayed the work on the (trivial)
patch to fix this problem.

It's solved now, and I will work on and send the patch later (after
dinner).

Sorry again,
Dario
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R Ltd., Cambridge (UK)

signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2] xen: grant-table: Simplify get_paged_frame

[Julien Grall]

Hi Roger,

On 18/09/17 17:58, Roger Pau Monné wrote:

On Mon, Sep 18, 2017 at 05:27:52PM +0100, Julien Grall wrote:

The implementation of get_paged_frame is currently different whether the
architecture support sharing memory or paging memory. Both
version are extremely similar so it is possible to consolidate in a
single implementation.

The main difference is the x86 version will allow grant on foreign page
when using HVM/PVH whilst Arm does not. At the moment, on x86 foreign pages
are only allowed for PVH Dom0. It seems that foreign pages should never
be granted so deny them

The check for shared/paged memory are now gated with the respective ifdef.
Potentially, dummy p2m_is_shared/p2m_is_paging could be implemented for
Arm.

Signed-off-by: Julien Grall 

---

Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Jan Beulich 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Wei Liu 

 Changes in v2:
 - Deny grant on foreign page (aligned with the ARM code)
 - Use #ifdef rather than #if defined
 - Update commit message
 - Fix typo in the title

get_page_from_gfn will be able to get reference on foreign page and as
per my understanding will allow to grant page on foreign memory.

This was not allowed with a simple get_page(...) on the ARM
implementation (no sharing nor paging supprot) but is allowed on the x86
implementation due to get_page_from_gfn.

On x86, foreign pages are currently only allowed for PVH dom0, so I
think it is not a big deal for now.

On Arm, foreign pages can be present on any domain. So this patch would
permit grant on foreing pages.

This patch will deny granting foreign pages. Jan Beulich is happy with
it. Any other opinions?


Won't this break QEMU running in stub domains?

I haven't tested it, but I'm afraid QEMU running in a stub domain
might try to grant a foreign frame. Ie: the emulated network code in
QEMU might try to grant a foreign frame in order to forward operations
from emulated devices to PV frontends.


I don't think it will break any existing setup because foreign mapping 
are only allowed for the hardware domain (see p2m_add_foreign).


Now, what should be expected in the future? I am not sure why we would 
allow grant on foreign mapping. It is very similar to QEMU granting a 
page that was granted by another domain, this is not allowed today.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 01/15] docs: create Memory Bandwidth Allocation (MBA) feature document

[Roger Pau Monné]

On Tue, Sep 05, 2017 at 05:32:23PM +0800, Yi Sun wrote:
> +* xl interfaces:
> +
> +  1. `psr-mba-show [domain-id]`:

Is this limited to domain-id, or one can also use the domain name?
Most of the xl commands accept either a domain-id or a domain-name.

> +
> + Show memory bandwidth throttling for domain. Under different modes, it
> + shows different type of data.
> +
> + There are two modes:
> + Linear mode: the response of throttling value is linear.
> + Non-linear mode: the response of throttling value is non-linear.
> +
> + For linear mode, it shows the decimal value. For non-linear mode, it 
> shows
> + hexadecimal value.
> +
> +  2. `psr-mba-set [OPTIONS]  `:
> +
> + Set memory bandwidth throttling for domain.
> +
> + Options:
> + '-s': Specify the socket to process, otherwise all sockets are 
> processed.
> +
> + Throttling value set in register implies the approximate amount of 
> delaying
> + the traffic between core and memory. The higher throttling value 
> results in
> + lower bandwidth. The max throttling value (MBA_MAX) supported can be got

s/got/obtained/

> + through CPUID.

How can one get this value empirically? Do I need to use a external
tool?

> +
> + Linear mode: the input precision is defined as 100-(MBA_MAX). For 
> instance,
> + if the MBA_MAX value is 90, the input precision is 10%. Values not an 
> even
> + multiple of the precision (e.g., 12%) will be rounded down (e.g., to 10%
> + delay applied) by HW automatically.
> +
> + Non-linear mode: input delay values are powers-of-two from zero to the
> + MBA_MAX value from CPUID. In this case any values not a power of two 
> will
> + be rounded down the next nearest power of two by HW automatically.

Both of the above descriptions should be moved to mba-show IMHO, the
description there is incomplete and not helpful.

> +
> +# Technical details
> +
> +MBA is a member of Intel PSR features, it shares the base PSR infrastructure
> +in Xen.
> +
> +## Hardware perspective
> +
> +  MBA defines a range of MSRs to support specifying a delay value (Thrtl) per
> +  COS, with details below.
> +
> +  ```
> +   +++
> +   | MSR (per socket)   |Address |
> +   +++
> +   | IA32_L2_QOS_Ext_BW_Thrtl_0 | 0xD50  |
> +   +++
> +   | ...|  ...   |
> +   +++
> +   | IA32_L2_QOS_Ext_BW_Thrtl_n | 0xD50+n|
> +   +++
> +  ```
> +
> +  When context switch happens, the COS ID of domain is written to per-thread 
> MSR
> +  `IA32_PQR_ASSOC`, and then hardware enforces bandwidth allocation according

I think this is missing some context of the relation between a thread
and the MSR. I assume it's related to IA32_PQR_ASSOC, but I have no
idea what that constant means.

What's more, Xen doesn't have threads, so you should maybe speak about
vCPUs instead?

> +  to the throttling value stored in the Thrtl MSR register.
> +
> +## The relationship between MBA and CAT/CDP
> +
> +  Generally speaking, MBA is completely independent of CAT/CDP, and any
> +  combination may be applied at any time, e.g. enabling MBA with CAT
> +  disabled.
> +
> +  But it needs to be noticed that MBA shares COS infrastructure with CAT,
> +  although MBA is enumerated by different CPUID leaf from CAT (which
> +  indicates that the max COS of MBA may be different from CAT). In some
> +  cases, a domain is permitted to have a COS that is beyond one (or more)
> +  of PSR features but within the others. For instance, let's assume the max
> +  COS of MBA is 8 but the max COS of L3 CAT is 16, when a domain is assigned
> +  9 as COS, the L3 CAT CBM associated to COS 9 would be enforced, but for 
> MBA,
> +  the HW works as default value is set since COS 9 is beyond the max COS (8)
> +  of MBA.
> +
> +## Design Overview
> +
> +* Core COS/Thrtl association
> +
> +  When enforcing Memory Bandwidth Allocation, all cores of domains have
> +  the same default Thrtl MSR (COS0) which stores the same Thrtl (0). The
> +  default Thrtl MSR is used only in hypervisor and is transparent to tool 
> stack
> +  and user.
> +
> +  System administrators can change PSR allocation policy at runtime by
> +  using the tool stack. Since MBA shares COS ID with CAT/CDP, a COS ID
> +  corresponds to a 2-tuple, like [CBM, Thrtl] with only-CAT enabled, when CDP
> +  is enabled, the COS ID corresponds to a 3-tuple, like [Code_CBM, Data_CBM,
> +  Thrtl]. If neither CAT nor CDP is enabled, things are easier, since one COS
> +  ID corresponds to one Thrtl.
> +
> +* VCPU schedule
> +
> +  This part reuses CAT COS infrastructure.
> +
> +* Multi-sockets
> +
> +  Different sockets may have different MBA ability (like max COS)
> +  although it is consistent on the same socket. 

Re: [Xen-devel] [PATCH 02/22] tools: libxendevicemodel: Provide xendevicemodel_shutdown

[Ian Jackson]

Wei Liu writes ("Re: [PATCH 02/22] tools: libxendevicemodel: Provide 
xendevicemodel_shutdown"):
> We need to have:
> 
> VERS_1.1 {
...
> And also bump the minor number in Makefile.

Done, thanks.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] x86/domctl: Don't pause the whole domain if only getting vcpu state

[Razvan Cojocaru]

On 09/18/2017 06:35 PM, Jan Beulich wrote:
 On 12.09.17 at 15:53,  wrote:
>> --- a/xen/arch/x86/domctl.c
>> +++ b/xen/arch/x86/domctl.c
>> @@ -625,6 +625,26 @@ long arch_do_domctl(
>>   !is_hvm_domain(d) )
>>  break;
>>  
>> +if ( domctl->u.hvmcontext_partial.type == HVM_SAVE_CODE(CPU) &&
>> + domctl->u.hvmcontext_partial.instance < d->max_vcpus )
> 
> I have to admit that I'm not in favor of such special casing, even
> less so without any code comment saying why this is so special.
> What if someone else wanted some other piece of vCPU state
> without pausing the entire domain? Wouldn't it be possible to
> generalize this to cover all such state elements?

There's no reason why all the other cases where this would the possible
shouldn't be optimized. What has made this one stand out for us is that
we're using it a lot with introspection, and the optimization counts.

But judging by the code reorganization (the addition of
hvm_save_one_cpu_ctxt()), the changes would need to be done on a
one-by-one case anyway (different queries may require different ways of
chaging the code).



Thanks,
Razvan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2] xen: grant-table: Simplify get_paged_frame

[Roger Pau Monné]

On Mon, Sep 18, 2017 at 05:27:52PM +0100, Julien Grall wrote:
> The implementation of get_paged_frame is currently different whether the
> architecture support sharing memory or paging memory. Both
> version are extremely similar so it is possible to consolidate in a
> single implementation.
> 
> The main difference is the x86 version will allow grant on foreign page
> when using HVM/PVH whilst Arm does not. At the moment, on x86 foreign pages
> are only allowed for PVH Dom0. It seems that foreign pages should never
> be granted so deny them
> 
> The check for shared/paged memory are now gated with the respective ifdef.
> Potentially, dummy p2m_is_shared/p2m_is_paging could be implemented for
> Arm.
> 
> Signed-off-by: Julien Grall 
> 
> ---
> 
> Cc: Andrew Cooper 
> Cc: George Dunlap 
> Cc: Ian Jackson 
> Cc: Jan Beulich 
> Cc: Konrad Rzeszutek Wilk 
> Cc: Stefano Stabellini 
> Cc: Tim Deegan 
> Cc: Wei Liu 
> 
> Changes in v2:
> - Deny grant on foreign page (aligned with the ARM code)
> - Use #ifdef rather than #if defined
> - Update commit message
> - Fix typo in the title
> 
> get_page_from_gfn will be able to get reference on foreign page and as
> per my understanding will allow to grant page on foreign memory.
> 
> This was not allowed with a simple get_page(...) on the ARM
> implementation (no sharing nor paging supprot) but is allowed on the x86
> implementation due to get_page_from_gfn.
> 
> On x86, foreign pages are currently only allowed for PVH dom0, so I
> think it is not a big deal for now.
> 
> On Arm, foreign pages can be present on any domain. So this patch would
> permit grant on foreing pages.
> 
> This patch will deny granting foreign pages. Jan Beulich is happy with
> it. Any other opinions?

Won't this break QEMU running in stub domains?

I haven't tested it, but I'm afraid QEMU running in a stub domain
might try to grant a foreign frame. Ie: the emulated network code in
QEMU might try to grant a foreign frame in order to forward operations
from emulated devices to PV frontends.

Roger.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2] xen: grant-table: Simplify get_paged_frame

[Julien Grall]

The implementation of get_paged_frame is currently different whether the
architecture support sharing memory or paging memory. Both
version are extremely similar so it is possible to consolidate in a
single implementation.

The main difference is the x86 version will allow grant on foreign page
when using HVM/PVH whilst Arm does not. At the moment, on x86 foreign pages
are only allowed for PVH Dom0. It seems that foreign pages should never
be granted so deny them

The check for shared/paged memory are now gated with the respective ifdef.
Potentially, dummy p2m_is_shared/p2m_is_paging could be implemented for
Arm.

Signed-off-by: Julien Grall 

---

Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Jan Beulich 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Wei Liu 

Changes in v2:
- Deny grant on foreign page (aligned with the ARM code)
- Use #ifdef rather than #if defined
- Update commit message
- Fix typo in the title

get_page_from_gfn will be able to get reference on foreign page and as
per my understanding will allow to grant page on foreign memory.

This was not allowed with a simple get_page(...) on the ARM
implementation (no sharing nor paging supprot) but is allowed on the x86
implementation due to get_page_from_gfn.

On x86, foreign pages are currently only allowed for PVH dom0, so I
think it is not a big deal for now.

On Arm, foreign pages can be present on any domain. So this patch would
permit grant on foreing pages.

This patch will deny granting foreign pages. Jan Beulich is happy with
it. Any other opinions?
---
 xen/common/grant_table.c | 19 ---
 1 file changed, 8 insertions(+), 11 deletions(-)

diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index c3895e6201..a6a168df6e 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -259,7 +259,6 @@ static int get_paged_frame(unsigned long gfn, unsigned long 
*frame,
struct domain *rd)
 {
 int rc = GNTST_okay;
-#if defined(P2M_PAGED_TYPES) || defined(P2M_SHARED_TYPES)
 p2m_type_t p2mt;
 
 *page = get_page_from_gfn(rd, gfn, ,
@@ -267,26 +266,24 @@ static int get_paged_frame(unsigned long gfn, unsigned 
long *frame,
 if ( !(*page) )
 {
 *frame = mfn_x(INVALID_MFN);
+#ifdef P2M_SHARED_TYPES
 if ( p2m_is_shared(p2mt) )
 return GNTST_eagain;
+#endif
+#ifdef P2M_PAGES_TYPES
 if ( p2m_is_paging(p2mt) )
 {
 p2m_mem_paging_populate(rd, gfn);
 return GNTST_eagain;
 }
+#endif
 return GNTST_bad_page;
 }
+
+if ( p2m_is_foreign(p2mt) )
+return GNTST_bad_page;
+
 *frame = page_to_mfn(*page);
-#else
-*frame = mfn_x(gfn_to_mfn(rd, _gfn(gfn)));
-*page = mfn_valid(_mfn(*frame)) ? mfn_to_page(*frame) : NULL;
-if ( (!(*page)) || (!get_page(*page, rd)) )
-{
-*frame = mfn_x(INVALID_MFN);
-*page = NULL;
-rc = GNTST_bad_page;
-}
-#endif
 
 return rc;
 }
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-unstable test] 113575: regressions - FAIL

[osstest service owner]

flight 113575 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/113575/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-credit2  15 guest-saverestorefail REGR. vs. 113387

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-rtds   16 guest-start/debian.repeat fail blocked in 113387
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 113387
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 113387
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-localmigrate/x10 fail like 113387
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 113387
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 113387
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 113387
 test-amd64-amd64-xl-qemut-ws16-amd64 10 windows-installfail never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 13 guest-saverestore   fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-amd64-i386-xl-qemut-ws16-amd64 13 guest-saverestore   fail never pass
 test-amd64-i386-libvirt-qcow2 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass

version targeted for testing:
 xen  abd91b2a2bcd05618a71f7e5fe571dd10a5727bc
baseline version:
 xen  16b1414de91b5a82a0996c67f6db3af7d7e32873

Last test of basis   113387  2017-09-12 23:20:09 Z5 days
Failing since113430  2017-09-14 01:24:48 Z4 days8 attempts
Testing same since   113511  2017-09-16 13:59:07 Z2 days5 attempts


People who touched revisions under test:
  Andrew Cooper 
  Bhupinder Thakur 
  Boris Ostrovsky 
  Dario Faggioli 
  Haozhong Zhang 
  Ian Jackson 
  Jan Beulich 
  Julien Grall 
  Oleksandr Grytsov 
  Oleksandr Tyshchenko 
  Petre Pircalabu 
  Stefano Stabellini 
  Wei Liu 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm   

Re: [Xen-devel] [PATCH v7 12/12] x86/hvm/ioreq: add a new mappable resource type...

[Roger Pau Monné]

On Mon, Sep 18, 2017 at 04:31:26PM +0100, Paul Durrant wrote:
> ... XENMEM_resource_ioreq_server
> 
> This patch adds support for a new resource type that can be mapped using
> the XENMEM_acquire_resource memory op.
> 
> If an emulator makes use of this resource type then, instead of mapping
> gfns, the IOREQ server will allocate pages from the heap. These pages
> will never be present in the P2M of the guest at any point and so are
> not vulnerable to any direct attack by the guest. They are only ever
> accessible by Xen and any domain that has mapping privilege over the
> guest (which may or may not be limited to the domain running the emulator).
> 
> NOTE: Use of the new resource type is not compatible with use of
>   XEN_DMOP_get_ioreq_server_info unless the XEN_DMOP_no_gfns flag is
>   set.
> 
> Signed-off-by: Paul Durrant 
> Acked-by: George Dunlap 
> Reviewed-by: Wei Liu 

Reviewed-by: Roger Pau Monné 

Just one nit below.

> --- a/xen/arch/x86/hvm/ioreq.c
> +++ b/xen/arch/x86/hvm/ioreq.c
> +mfn_t hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
> + unsigned int idx)
> +{
> +struct hvm_ioreq_server *s;
> +mfn_t mfn = INVALID_MFN;
> +
> +spin_lock_recursive(>arch.hvm_domain.ioreq_server.lock);
> +
> +s = get_ioreq_server(d, id);
> +
> +if ( id >= MAX_NR_IOREQ_SERVERS || !s || IS_DEFAULT(s) )

If you use get_ioreq_server the id >= MAX_NR_IOREQ_SERVERS check is
pointless, get_ioreq_server will already return NULL in that case.

Thanks, Roger.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v7 03/12] tools/libxenforeignmemory: add support for resource mapping

[Ian Jackson]

Paul Durrant writes ("[PATCH v7 03/12] tools/libxenforeignmemory: add support 
for resource mapping"):
> A previous patch introduced a new HYPERVISOR_memory_op to acquire guest
> resources for direct priv-mapping.
> 
> This patch adds new functionality into libxenforeignmemory to make use
> of a new privcmd ioctl [1] that uses the new memory op to make such
> resources available via mmap(2).
> 
> [1] 
> http://xenbits.xen.org/gitweb/?p=people/pauldu/linux.git;a=commit;h=ce59a05e6712

This looks plausible to me.

I wonder whether this, particularly for the ioreq server page, will
make it possible to deprivilege earlier than I did in my own series on
Friday.

(With my series, I do the depriv on entering the `running' state,
which is quite late.  It's after reading the migration stream, which
is not ideal.  But it did mean that qemu had already aquired the ioreq
page by then so it worked.  Unless that's just because my Xen was a
bit old?)

Anyway,

Acked-by: Ian Jackson 

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 03/22] xentoolcore, _restrict_all: Introduce new library and implementation

[Ian Jackson]

Wei Liu writes ("Re: [PATCH 03/22] xentoolcore, _restrict_all: Introduce new 
library and implementation"):
> On Fri, Sep 15, 2017 at 07:48:40PM +0100, Ian Jackson wrote:
...
> > +void xentoolcore__register_active_handle(Xentoolcore__Active_Handle*);
> > +void xentoolcore__deregister_active_handle(Xentoolcore__Active_Handle*);
> 
> Why use two underscores in those function names?

These functions are for use by other Xen libraries only, not by
external callers.

> Is this library supposed to be stable? We only expect this to be tech
> review, right?  I think it is worth explicitly stating that if that's
> the case.

The API/ABI to xentoolcore_restrict_all is intended to be stable.
The internal API may vary.

The head comment of xentoolcore_internal.h says
   * Interfaces of xentoolcore directed internally at other Xen libraries
but I can make this more explicit if you want.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 16/22] xentoolcore, _restrict_all: Document implementation "complete"

[Ian Jackson]

Wei Liu writes ("Re: [PATCH 16/22] xentoolcore, _restrict_all: Document 
implementation "complete""):
> On Fri, Sep 15, 2017 at 07:48:53PM +0100, Ian Jackson wrote:
> > - *  This function will be implemented insofar as it appears necessary
> > - *  for the purposes of running a deprivileged qemu.
> > + *  This function has been implemented insofar as it appears necessary
> > + *  for the purposes of running a deprivileged qemu, and is believed to
> > + *  be sufficient (subject to the caveats discussed in the appropriate
> > + *  libxl documatation for this feature).
> 
> documentation

Fixed, thanks.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v7 12/12] x86/hvm/ioreq: add a new mappable resource type...

[Paul Durrant]

... XENMEM_resource_ioreq_server

This patch adds support for a new resource type that can be mapped using
the XENMEM_acquire_resource memory op.

If an emulator makes use of this resource type then, instead of mapping
gfns, the IOREQ server will allocate pages from the heap. These pages
will never be present in the P2M of the guest at any point and so are
not vulnerable to any direct attack by the guest. They are only ever
accessible by Xen and any domain that has mapping privilege over the
guest (which may or may not be limited to the domain running the emulator).

NOTE: Use of the new resource type is not compatible with use of
  XEN_DMOP_get_ioreq_server_info unless the XEN_DMOP_no_gfns flag is
  set.

Signed-off-by: Paul Durrant 
Acked-by: George Dunlap 
Reviewed-by: Wei Liu 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 
Cc: Ian Jackson 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 

v5:
 - Use get_ioreq_server() function rather than indexing array directly.
 - Add more explanation into comments to state than mapping guest frames
   and allocation of pages for ioreq servers are not simultaneously
   permitted.
 - Add a comment into asm/ioreq.h stating the meaning of the index
   value passed to hvm_get_ioreq_server_frame().
---
 xen/arch/x86/hvm/ioreq.c| 131 +++-
 xen/arch/x86/mm.c   |  27 +
 xen/include/asm-x86/hvm/ioreq.h |   6 ++
 xen/include/public/hvm/dm_op.h  |   4 ++
 xen/include/public/memory.h |   3 +
 5 files changed, 170 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 1fbc81fb15..0aacd7d2c2 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -260,6 +260,19 @@ static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
 int rc;
 
+if ( iorp->page )
+{
+/*
+ * If a page has already been allocated (which will happen on
+ * demand if hvm_get_ioreq_server_frame() is called), then
+ * mapping a guest frame is not permitted.
+ */
+if ( gfn_eq(iorp->gfn, INVALID_GFN) )
+return -EPERM;
+
+return 0;
+}
+
 if ( d->is_dying )
 return -EINVAL;
 
@@ -282,6 +295,61 @@ static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 return rc;
 }
 
+static int hvm_alloc_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
+{
+struct domain *currd = current->domain;
+struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
+
+if ( iorp->page )
+{
+/*
+ * If a guest frame has already been mapped (which may happen
+ * on demand if hvm_get_ioreq_server_info() is called), then
+ * allocating a page is not permitted.
+ */
+if ( !gfn_eq(iorp->gfn, INVALID_GFN) )
+return -EPERM;
+
+return 0;
+}
+
+/*
+ * Allocated IOREQ server pages are assigned to the emulating
+ * domain, not the target domain. This is because the emulator is
+ * likely to be destroyed after the target domain has been torn
+ * down, and we must use MEMF_no_refcount otherwise page allocation
+ * could fail if the emulating domain has already reached its
+ * maximum allocation.
+ */
+iorp->page = alloc_domheap_page(currd, MEMF_no_refcount);
+if ( !iorp->page )
+return -ENOMEM;
+
+iorp->va = __map_domain_page_global(iorp->page);
+if ( !iorp->va )
+{
+iorp->page = NULL;
+return -ENOMEM;
+}
+
+clear_page(iorp->va);
+return 0;
+}
+
+static void hvm_free_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
+{
+struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
+
+if ( !iorp->page )
+return;
+
+unmap_domain_page_global(iorp->va);
+iorp->va = NULL;
+
+put_page(iorp->page);
+iorp->page = NULL;
+}
+
 bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
 {
 const struct hvm_ioreq_server *s;
@@ -488,6 +556,27 @@ static void hvm_ioreq_server_unmap_pages(struct 
hvm_ioreq_server *s)
 hvm_unmap_ioreq_gfn(s, false);
 }
 
+static int hvm_ioreq_server_alloc_pages(struct hvm_ioreq_server *s)
+{
+int rc;
+
+rc = hvm_alloc_ioreq_mfn(s, false);
+
+if ( !rc && (s->bufioreq_handling != HVM_IOREQSRV_BUFIOREQ_OFF) )
+rc = hvm_alloc_ioreq_mfn(s, true);
+
+if ( rc )
+hvm_free_ioreq_mfn(s, false);
+
+return rc;
+}
+
+static void hvm_ioreq_server_free_pages(struct hvm_ioreq_server *s)
+{
+hvm_free_ioreq_mfn(s, true);
+hvm_free_ioreq_mfn(s, false);
+}
+
 static void hvm_ioreq_server_free_rangesets(struct hvm_ioreq_server *s)
 {
 unsigned int i;
@@ -614,7 +703,18 @@ 

[Xen-devel] [PATCH v7 11/12] x86/hvm/ioreq: defer mapping gfns until they are actually requsted

[Paul Durrant]

A subsequent patch will introduce a new scheme to allow an emulator to
map ioreq server pages directly from Xen rather than the guest P2M.

This patch lays the groundwork for that change by deferring mapping of
gfns until their values are requested by an emulator. To that end, the
pad field of the xen_dm_op_get_ioreq_server_info structure is re-purposed
to a flags field and new flag, XEN_DMOP_no_gfns, defined which modifies the
behaviour of XEN_DMOP_get_ioreq_server_info to allow the caller to avoid
requesting the gfn values.

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
Acked-by: Wei Liu 
---
Cc: Ian Jackson 
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Jan Beulich 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 

v3:
 - Updated in response to review comments from Wei and Roger.
 - Added a HANDLE_BUFIOREQ macro to make the code neater.
 - This patch no longer introduces a security vulnerability since there
   is now an explicit limit on the number of ioreq servers that may be
   created for any one domain.
---
 tools/libs/devicemodel/core.c   |  8 +
 tools/libs/devicemodel/include/xendevicemodel.h |  6 ++--
 xen/arch/x86/hvm/dm.c   |  9 --
 xen/arch/x86/hvm/ioreq.c| 41 +
 xen/include/asm-x86/hvm/domain.h|  2 +-
 xen/include/public/hvm/dm_op.h  | 32 +++
 6 files changed, 59 insertions(+), 39 deletions(-)

diff --git a/tools/libs/devicemodel/core.c b/tools/libs/devicemodel/core.c
index fcb260d29b..28958934bf 100644
--- a/tools/libs/devicemodel/core.c
+++ b/tools/libs/devicemodel/core.c
@@ -188,6 +188,14 @@ int xendevicemodel_get_ioreq_server_info(
 
 data->id = id;
 
+/*
+ * If the caller is not requesting gfn values then instruct the
+ * hypercall not to retrieve them as this may cause them to be
+ * mapped.
+ */
+if (!ioreq_gfn && !bufioreq_gfn)
+data->flags |= XEN_DMOP_no_gfns;
+
 rc = xendevicemodel_op(dmod, domid, 1, , sizeof(op));
 if (rc)
 return rc;
diff --git a/tools/libs/devicemodel/include/xendevicemodel.h 
b/tools/libs/devicemodel/include/xendevicemodel.h
index 13216db04a..d73a76da35 100644
--- a/tools/libs/devicemodel/include/xendevicemodel.h
+++ b/tools/libs/devicemodel/include/xendevicemodel.h
@@ -61,11 +61,11 @@ int xendevicemodel_create_ioreq_server(
  * @parm domid the domain id to be serviced
  * @parm id the IOREQ Server id.
  * @parm ioreq_gfn pointer to a xen_pfn_t to receive the synchronous ioreq
- *  gfn
+ *  gfn. (May be NULL if not required)
  * @parm bufioreq_gfn pointer to a xen_pfn_t to receive the buffered ioreq
- *gfn
+ *gfn. (May be NULL if not required)
  * @parm bufioreq_port pointer to a evtchn_port_t to receive the buffered
- * ioreq event channel
+ * ioreq event channel. (May be NULL if not required)
  * @return 0 on success, -1 on failure.
  */
 int xendevicemodel_get_ioreq_server_info(
diff --git a/xen/arch/x86/hvm/dm.c b/xen/arch/x86/hvm/dm.c
index 87ef4b6ca9..c020f0c99f 100644
--- a/xen/arch/x86/hvm/dm.c
+++ b/xen/arch/x86/hvm/dm.c
@@ -418,16 +418,19 @@ static int dm_op(const struct dmop_args *op_args)
 {
 struct xen_dm_op_get_ioreq_server_info *data =
 _ioreq_server_info;
+const uint16_t valid_flags = XEN_DMOP_no_gfns;
 
 const_op = false;
 
 rc = -EINVAL;
-if ( data->pad )
+if ( data->flags & ~valid_flags )
 break;
 
 rc = hvm_get_ioreq_server_info(d, data->id,
-   >ioreq_gfn,
-   >bufioreq_gfn,
+   (data->flags & XEN_DMOP_no_gfns) ?
+   NULL : >ioreq_gfn,
+   (data->flags & XEN_DMOP_no_gfns) ?
+   NULL : >bufioreq_gfn,
>bufioreq_port);
 break;
 }
diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 3e2a3f62ba..1fbc81fb15 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -354,6 +354,9 @@ static void hvm_update_ioreq_evtchn(struct hvm_ioreq_server 
*s,
 }
 }
 
+#define HANDLE_BUFIOREQ(s) \
+(s->bufioreq_handling != HVM_IOREQSRV_BUFIOREQ_OFF)
+
 static int hvm_ioreq_server_add_vcpu(struct hvm_ioreq_server *s,
  struct vcpu *v)
 {
@@ -375,7 +378,7 @@ static int hvm_ioreq_server_add_vcpu(struct 
hvm_ioreq_server *s,
 
 sv->ioreq_evtchn = rc;
 
-if ( v->vcpu_id == 0 && 

[Xen-devel] [PATCH v7 10/12] x86/hvm/ioreq: use gfn_t in struct hvm_ioreq_page

[Paul Durrant]

This patch adjusts the ioreq server code to use type-safe gfn_t values
where possible. No functional change.

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
Reviewed-by: Wei Liu 
---
Cc: Andrew Cooper 
Cc: Jan Beulich 
---
 xen/arch/x86/hvm/ioreq.c | 44 
 xen/include/asm-x86/hvm/domain.h |  2 +-
 2 files changed, 23 insertions(+), 23 deletions(-)

diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 5f38d39ce2..3e2a3f62ba 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -211,7 +211,7 @@ bool handle_hvm_io_completion(struct vcpu *v)
 return true;
 }
 
-static unsigned long hvm_alloc_ioreq_gfn(struct hvm_ioreq_server *s)
+static gfn_t hvm_alloc_ioreq_gfn(struct hvm_ioreq_server *s)
 {
 struct domain *d = s->domain;
 unsigned int i;
@@ -221,20 +221,19 @@ static unsigned long hvm_alloc_ioreq_gfn(struct 
hvm_ioreq_server *s)
 for ( i = 0; i < sizeof(d->arch.hvm_domain.ioreq_gfn.mask) * 8; i++ )
 {
 if ( test_and_clear_bit(i, >arch.hvm_domain.ioreq_gfn.mask) )
-return d->arch.hvm_domain.ioreq_gfn.base + i;
+return _gfn(d->arch.hvm_domain.ioreq_gfn.base + i);
 }
 
-return gfn_x(INVALID_GFN);
+return INVALID_GFN;
 }
 
-static void hvm_free_ioreq_gfn(struct hvm_ioreq_server *s,
-   unsigned long gfn)
+static void hvm_free_ioreq_gfn(struct hvm_ioreq_server *s, gfn_t gfn)
 {
 struct domain *d = s->domain;
-unsigned int i = gfn - d->arch.hvm_domain.ioreq_gfn.base;
+unsigned int i = gfn_x(gfn) - d->arch.hvm_domain.ioreq_gfn.base;
 
 ASSERT(!IS_DEFAULT(s));
-ASSERT(gfn != gfn_x(INVALID_GFN));
+ASSERT(!gfn_eq(gfn, INVALID_GFN));
 
 set_bit(i, >arch.hvm_domain.ioreq_gfn.mask);
 }
@@ -243,7 +242,7 @@ static void hvm_unmap_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 {
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
 
-if ( iorp->gfn == gfn_x(INVALID_GFN) )
+if ( gfn_eq(iorp->gfn, INVALID_GFN) )
 return;
 
 destroy_ring_for_helper(>va, iorp->page);
@@ -252,7 +251,7 @@ static void hvm_unmap_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 if ( !IS_DEFAULT(s) )
 hvm_free_ioreq_gfn(s, iorp->gfn);
 
-iorp->gfn = gfn_x(INVALID_GFN);
+iorp->gfn = INVALID_GFN;
 }
 
 static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
@@ -265,16 +264,17 @@ static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 return -EINVAL;
 
 if ( IS_DEFAULT(s) )
-iorp->gfn = buf ?
-d->arch.hvm_domain.params[HVM_PARAM_BUFIOREQ_PFN] :
-d->arch.hvm_domain.params[HVM_PARAM_IOREQ_PFN];
+iorp->gfn = _gfn(buf ?
+ d->arch.hvm_domain.params[HVM_PARAM_BUFIOREQ_PFN] :
+ d->arch.hvm_domain.params[HVM_PARAM_IOREQ_PFN]);
 else
 iorp->gfn = hvm_alloc_ioreq_gfn(s);
 
-if ( iorp->gfn == gfn_x(INVALID_GFN) )
+if ( gfn_eq(iorp->gfn, INVALID_GFN) )
 return -ENOMEM;
 
-rc = prepare_ring_for_helper(d, iorp->gfn, >page, >va);
+rc = prepare_ring_for_helper(d, gfn_x(iorp->gfn), >page,
+ >va);
 
 if ( rc )
 hvm_unmap_ioreq_gfn(s, buf);
@@ -313,10 +313,10 @@ static void hvm_remove_ioreq_gfn(struct hvm_ioreq_server 
*s, bool buf)
 struct domain *d = s->domain;
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
 
-if ( IS_DEFAULT(s) || iorp->gfn == gfn_x(INVALID_GFN) )
+if ( IS_DEFAULT(s) || gfn_eq(iorp->gfn, INVALID_GFN) )
 return;
 
-if ( guest_physmap_remove_page(d, _gfn(iorp->gfn),
+if ( guest_physmap_remove_page(d, iorp->gfn,
_mfn(page_to_mfn(iorp->page)), 0) )
 domain_crash(d);
 clear_page(iorp->va);
@@ -328,12 +328,12 @@ static int hvm_add_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
 int rc;
 
-if ( IS_DEFAULT(s) || iorp->gfn == gfn_x(INVALID_GFN) )
+if ( IS_DEFAULT(s) || gfn_eq(iorp->gfn, INVALID_GFN) )
 return 0;
 
 clear_page(iorp->va);
 
-rc = guest_physmap_add_page(d, _gfn(iorp->gfn),
+rc = guest_physmap_add_page(d, iorp->gfn,
 _mfn(page_to_mfn(iorp->page)), 0);
 if ( rc == 0 )
 paging_mark_dirty(d, _mfn(page_to_mfn(iorp->page)));
@@ -592,8 +592,8 @@ static int hvm_ioreq_server_init(struct hvm_ioreq_server *s,
 INIT_LIST_HEAD(>ioreq_vcpu_list);
 spin_lock_init(>bufioreq_lock);
 
-s->ioreq.gfn = gfn_x(INVALID_GFN);
-s->bufioreq.gfn = gfn_x(INVALID_GFN);
+s->ioreq.gfn = INVALID_GFN;
+s->bufioreq.gfn = INVALID_GFN;
 
 rc = hvm_ioreq_server_alloc_rangesets(s, id);
 if ( rc )
@@ -762,11 +762,11 @@ int 

Re: [Xen-devel] [PATCH 2/2] Introduce migration precopy policy

[Ian Jackson]

Jennifer Herbert writes ("Re: [PATCH 2/2] Introduce migration precopy policy"):
> The original patch had a small amount of code to pass this though such 
> that libxl could provide one.  I'm not entity sure how the perl code 
> worked, but I've not made any changes that wouldn't stop this working.  
> That section would form a follow on patch.  Since I don't use or have 
> much understanding of libxl, I didn't really feel I was the best person 
> to do this.

Can you post that extra bit of code as an RFC 3/2 ?  I think it would
help my understanding of the future path, and maybe I'd like to fix it
up for you :-).

Thanks,
Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 0/3] Some coverity fixes for xl/libxl

[Ian Jackson]

Wei Liu writes ("[PATCH v2 0/3] Some coverity fixes for xl/libxl"):
> v2: use _mandatory variant

All three

Acked-by: Ian Jackson 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] x86/domctl: Don't pause the whole domain if only getting vcpu state

[Jan Beulich]

>>> On 12.09.17 at 15:53,  wrote:
> --- a/xen/arch/x86/domctl.c
> +++ b/xen/arch/x86/domctl.c
> @@ -625,6 +625,26 @@ long arch_do_domctl(
>   !is_hvm_domain(d) )
>  break;
>  
> +if ( domctl->u.hvmcontext_partial.type == HVM_SAVE_CODE(CPU) &&
> + domctl->u.hvmcontext_partial.instance < d->max_vcpus )

I have to admit that I'm not in favor of such special casing, even
less so without any code comment saying why this is so special.
What if someone else wanted some other piece of vCPU state
without pausing the entire domain? Wouldn't it be possible to
generalize this to cover all such state elements?

> +{
> + struct vcpu *v = d->vcpu[domctl->u.hvmcontext_partial.instance];
> + struct hvm_hw_cpu ctx;
> +
> + vcpu_pause(v);
> +
> + hvm_save_one_cpu_ctxt(v, );
> +
> + vcpu_unpause(v);
> +
> + if ( copy_to_guest(domctl->u.hvmcontext_partial.buffer,
> +(void *), sizeof(ctx)) )

Indentation.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v7 03/12] tools/libxenforeignmemory: add support for resource mapping

[Paul Durrant]

A previous patch introduced a new HYPERVISOR_memory_op to acquire guest
resources for direct priv-mapping.

This patch adds new functionality into libxenforeignmemory to make use
of a new privcmd ioctl [1] that uses the new memory op to make such
resources available via mmap(2).

[1] 
http://xenbits.xen.org/gitweb/?p=people/pauldu/linux.git;a=commit;h=ce59a05e6712

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
Reviewed-by: Wei Liu 
---
Cc: Ian Jackson 

v4:
 - Fixed errno and removed single-use label
 - The unmap call now returns a status
 - Use C99 initialization for ioctl struct

v2:
 - Bump minor version up to 3.
---
 tools/include/xen-sys/Linux/privcmd.h  | 11 +
 tools/libs/foreignmemory/Makefile  |  2 +-
 tools/libs/foreignmemory/core.c| 53 ++
 .../libs/foreignmemory/include/xenforeignmemory.h  | 41 +
 tools/libs/foreignmemory/libxenforeignmemory.map   |  5 ++
 tools/libs/foreignmemory/linux.c   | 45 ++
 tools/libs/foreignmemory/private.h | 31 +
 7 files changed, 187 insertions(+), 1 deletion(-)

diff --git a/tools/include/xen-sys/Linux/privcmd.h 
b/tools/include/xen-sys/Linux/privcmd.h
index 732ff7c15a..9531b728f9 100644
--- a/tools/include/xen-sys/Linux/privcmd.h
+++ b/tools/include/xen-sys/Linux/privcmd.h
@@ -86,6 +86,15 @@ typedef struct privcmd_dm_op {
const privcmd_dm_op_buf_t __user *ubufs;
 } privcmd_dm_op_t;
 
+typedef struct privcmd_mmap_resource {
+   domid_t dom;
+   __u32 type;
+   __u32 id;
+   __u32 idx;
+   __u64 num;
+   __u64 addr;
+} privcmd_mmap_resource_t;
+
 /*
  * @cmd: IOCTL_PRIVCMD_HYPERCALL
  * @arg: _hypercall_t
@@ -103,5 +112,7 @@ typedef struct privcmd_dm_op {
_IOC(_IOC_NONE, 'P', 5, sizeof(privcmd_dm_op_t))
 #define IOCTL_PRIVCMD_RESTRICT \
_IOC(_IOC_NONE, 'P', 6, sizeof(domid_t))
+#define IOCTL_PRIVCMD_MMAP_RESOURCE\
+   _IOC(_IOC_NONE, 'P', 7, sizeof(privcmd_mmap_resource_t))
 
 #endif /* __LINUX_PUBLIC_PRIVCMD_H__ */
diff --git a/tools/libs/foreignmemory/Makefile 
b/tools/libs/foreignmemory/Makefile
index ab7f873f26..5c7f78f61d 100644
--- a/tools/libs/foreignmemory/Makefile
+++ b/tools/libs/foreignmemory/Makefile
@@ -2,7 +2,7 @@ XEN_ROOT = $(CURDIR)/../../..
 include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR= 1
-MINOR= 2
+MINOR= 3
 SHLIB_LDFLAGS += -Wl,--version-script=libxenforeignmemory.map
 
 CFLAGS   += -Werror -Wmissing-prototypes
diff --git a/tools/libs/foreignmemory/core.c b/tools/libs/foreignmemory/core.c
index a6897dc561..8d3f9f178f 100644
--- a/tools/libs/foreignmemory/core.c
+++ b/tools/libs/foreignmemory/core.c
@@ -17,6 +17,8 @@
 #include 
 #include 
 
+#include 
+
 #include "private.h"
 
 xenforeignmemory_handle *xenforeignmemory_open(xentoollog_logger *logger,
@@ -120,6 +122,57 @@ int xenforeignmemory_restrict(xenforeignmemory_handle 
*fmem,
 return osdep_xenforeignmemory_restrict(fmem, domid);
 }
 
+xenforeignmemory_resource_handle *xenforeignmemory_map_resource(
+xenforeignmemory_handle *fmem, domid_t domid, unsigned int type,
+unsigned int id, unsigned long frame, unsigned long nr_frames,
+void **paddr, int prot, int flags)
+{
+xenforeignmemory_resource_handle *fres;
+int rc;
+
+/* Check flags only contains POSIX defined values */
+if ( flags & ~(MAP_SHARED | MAP_PRIVATE) )
+{
+errno = EINVAL;
+return NULL;
+}
+
+fres = calloc(1, sizeof(*fres));
+if ( !fres )
+{
+errno = ENOMEM;
+return NULL;
+}
+
+fres->domid = domid;
+fres->type = type;
+fres->id = id;
+fres->frame = frame;
+fres->nr_frames = nr_frames;
+fres->addr = *paddr;
+fres->prot = prot;
+fres->flags = flags;
+
+rc = osdep_xenforeignmemory_map_resource(fmem, fres);
+if ( rc )
+{
+free(fres);
+fres = NULL;
+} else
+*paddr = fres->addr;
+
+return fres;
+}
+
+int xenforeignmemory_unmap_resource(
+xenforeignmemory_handle *fmem, xenforeignmemory_resource_handle *fres)
+{
+int rc = osdep_xenforeignmemory_unmap_resource(fmem, fres);
+
+free(fres);
+return rc;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/foreignmemory/include/xenforeignmemory.h 
b/tools/libs/foreignmemory/include/xenforeignmemory.h
index f4814c390f..d594be8df0 100644
--- a/tools/libs/foreignmemory/include/xenforeignmemory.h
+++ b/tools/libs/foreignmemory/include/xenforeignmemory.h
@@ -138,6 +138,47 @@ int xenforeignmemory_unmap(xenforeignmemory_handle *fmem,
 int xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
   domid_t domid);
 
+typedef struct xenforeignmemory_resource_handle 
xenforeignmemory_resource_handle;
+
+/**
+ * This 

[Xen-devel] [PATCH v7 04/12] tools/libxenforeignmemory: reduce xenforeignmemory_restrict code footprint

[Paul Durrant]

By using a static inline stub in private.h for OS where this functionality
is not implemented, the various duplicate stubs in the OS-specific source
modules can be avoided.

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
Acked-by: Wei Liu 
---
Cc: Ian Jackson 

v4:
 - Removed extraneous freebsd code.

v3:
 - Patch added in response to review comments.
---
 tools/libs/foreignmemory/freebsd.c |  7 ---
 tools/libs/foreignmemory/minios.c  |  7 ---
 tools/libs/foreignmemory/netbsd.c  |  7 ---
 tools/libs/foreignmemory/private.h | 12 +---
 tools/libs/foreignmemory/solaris.c |  7 ---
 5 files changed, 9 insertions(+), 31 deletions(-)

diff --git a/tools/libs/foreignmemory/freebsd.c 
b/tools/libs/foreignmemory/freebsd.c
index dec447485a..6e6bc4b11f 100644
--- a/tools/libs/foreignmemory/freebsd.c
+++ b/tools/libs/foreignmemory/freebsd.c
@@ -95,13 +95,6 @@ int osdep_xenforeignmemory_unmap(xenforeignmemory_handle 
*fmem,
 return munmap(addr, num << PAGE_SHIFT);
 }
 
-int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
-domid_t domid)
-{
-errno = -EOPNOTSUPP;
-return -1;
-}
-
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/foreignmemory/minios.c 
b/tools/libs/foreignmemory/minios.c
index 75f340122e..43341ca301 100644
--- a/tools/libs/foreignmemory/minios.c
+++ b/tools/libs/foreignmemory/minios.c
@@ -58,13 +58,6 @@ int osdep_xenforeignmemory_unmap(xenforeignmemory_handle 
*fmem,
 return munmap(addr, num << PAGE_SHIFT);
 }
 
-int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
-domid_t domid)
-{
-errno = -EOPNOTSUPP;
-return -1;
-}
-
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/foreignmemory/netbsd.c 
b/tools/libs/foreignmemory/netbsd.c
index 9bf95ef4f0..54a418ebd6 100644
--- a/tools/libs/foreignmemory/netbsd.c
+++ b/tools/libs/foreignmemory/netbsd.c
@@ -100,13 +100,6 @@ int osdep_xenforeignmemory_unmap(xenforeignmemory_handle 
*fmem,
 return munmap(addr, num*XC_PAGE_SIZE);
 }
 
-int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
-domid_t domid)
-{
-errno = -EOPNOTSUPP;
-return -1;
-}
-
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/foreignmemory/private.h 
b/tools/libs/foreignmemory/private.h
index 80b22bdbfc..b5d5f0a354 100644
--- a/tools/libs/foreignmemory/private.h
+++ b/tools/libs/foreignmemory/private.h
@@ -32,9 +32,6 @@ void *osdep_xenforeignmemory_map(xenforeignmemory_handle 
*fmem,
 int osdep_xenforeignmemory_unmap(xenforeignmemory_handle *fmem,
  void *addr, size_t num);
 
-int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
-domid_t domid);
-
 #if defined(__NetBSD__) || defined(__sun__)
 /* Strictly compat for those two only only */
 void *compat_mapforeign_batch(xenforeignmem_handle *fmem, uint32_t dom,
@@ -54,6 +51,13 @@ struct xenforeignmemory_resource_handle {
 };
 
 #ifndef __linux__
+static inline int osdep_xenforeignmemory_restrict(xenforeignmemory_handle 
*fmem,
+  domid_t domid)
+{
+errno = EOPNOTSUPP;
+return -1;
+}
+
 static inline int osdep_xenforeignmemory_map_resource(
 xenforeignmemory_handle *fmem, xenforeignmemory_resource_handle *fres)
 {
@@ -67,6 +71,8 @@ static inline int osdep_xenforeignmemory_unmap_resource(
 return 0;
 }
 #else
+int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
+domid_t domid);
 int osdep_xenforeignmemory_map_resource(
 xenforeignmemory_handle *fmem, xenforeignmemory_resource_handle *fres);
 int osdep_xenforeignmemory_unmap_resource(
diff --git a/tools/libs/foreignmemory/solaris.c 
b/tools/libs/foreignmemory/solaris.c
index a33decb4ae..ee8aae4fbd 100644
--- a/tools/libs/foreignmemory/solaris.c
+++ b/tools/libs/foreignmemory/solaris.c
@@ -97,13 +97,6 @@ int osdep_xenforeignmemory_unmap(xenforeignmemory_handle 
*fmem,
 return munmap(addr, num*XC_PAGE_SIZE);
 }
 
-int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
-domid_t domid)
-{
-errno = -EOPNOTSUPP;
-return -1;
-}
-
 /*
  * Local variables:
  * mode: C
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v7 07/12] x86/hvm/ioreq: use bool rather than bool_t

[Paul Durrant]

This patch changes use of bool_t to bool in the ioreq server code. It also
fixes an incorrect indentation in a continuation line.

This patch is purely cosmetic. No semantic or functional change.

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
Reviewed-by: Wei Liu 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/x86/hvm/dm.c|   2 +-
 xen/arch/x86/hvm/hvm.c   |   2 +-
 xen/arch/x86/hvm/io.c|   4 +-
 xen/arch/x86/hvm/ioreq.c | 100 +++
 xen/include/asm-x86/hvm/domain.h |   6 +--
 xen/include/asm-x86/hvm/ioreq.h  |  14 +++---
 6 files changed, 64 insertions(+), 64 deletions(-)

diff --git a/xen/arch/x86/hvm/dm.c b/xen/arch/x86/hvm/dm.c
index f7cb883fec..87ef4b6ca9 100644
--- a/xen/arch/x86/hvm/dm.c
+++ b/xen/arch/x86/hvm/dm.c
@@ -409,7 +409,7 @@ static int dm_op(const struct dmop_args *op_args)
 if ( data->pad[0] || data->pad[1] || data->pad[2] )
 break;
 
-rc = hvm_create_ioreq_server(d, curr_d->domain_id, 0,
+rc = hvm_create_ioreq_server(d, curr_d->domain_id, false,
  data->handle_bufioreq, >id);
 break;
 }
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 58b4afa1d1..031d07baf0 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -4361,7 +4361,7 @@ static int hvmop_get_param(
 {
 domid_t domid = d->arch.hvm_domain.params[HVM_PARAM_DM_DOMAIN];
 
-rc = hvm_create_ioreq_server(d, domid, 1,
+rc = hvm_create_ioreq_server(d, domid, true,
  HVM_IOREQSRV_BUFIOREQ_LEGACY, NULL);
 if ( rc != 0 && rc != -EEXIST )
 goto out;
diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c
index bf41954f59..1ddcaba52e 100644
--- a/xen/arch/x86/hvm/io.c
+++ b/xen/arch/x86/hvm/io.c
@@ -59,7 +59,7 @@ void send_timeoffset_req(unsigned long timeoff)
 if ( timeoff == 0 )
 return;
 
-if ( hvm_broadcast_ioreq(, 1) != 0 )
+if ( hvm_broadcast_ioreq(, true) != 0 )
 gprintk(XENLOG_ERR, "Unsuccessful timeoffset update\n");
 }
 
@@ -73,7 +73,7 @@ void send_invalidate_req(void)
 .data = ~0UL, /* flush all */
 };
 
-if ( hvm_broadcast_ioreq(, 0) != 0 )
+if ( hvm_broadcast_ioreq(, false) != 0 )
 gprintk(XENLOG_ERR, "Unsuccessful map-cache invalidate\n");
 }
 
diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 69913cf3cd..f2e0b3f74a 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -43,7 +43,7 @@ static ioreq_t *get_ioreq(struct hvm_ioreq_server *s, struct 
vcpu *v)
 return >vcpu_ioreq[v->vcpu_id];
 }
 
-bool_t hvm_io_pending(struct vcpu *v)
+bool hvm_io_pending(struct vcpu *v)
 {
 struct domain *d = v->domain;
 struct hvm_ioreq_server *s;
@@ -59,11 +59,11 @@ bool_t hvm_io_pending(struct vcpu *v)
   list_entry )
 {
 if ( sv->vcpu == v && sv->pending )
-return 1;
+return true;
 }
 }
 
-return 0;
+return false;
 }
 
 static void hvm_io_assist(struct hvm_ioreq_vcpu *sv, uint64_t data)
@@ -82,10 +82,10 @@ static void hvm_io_assist(struct hvm_ioreq_vcpu *sv, 
uint64_t data)
 msix_write_completion(v);
 vcpu_end_shutdown_deferral(v);
 
-sv->pending = 0;
+sv->pending = false;
 }
 
-static bool_t hvm_wait_for_io(struct hvm_ioreq_vcpu *sv, ioreq_t *p)
+static bool hvm_wait_for_io(struct hvm_ioreq_vcpu *sv, ioreq_t *p)
 {
 while ( sv->pending )
 {
@@ -112,16 +112,16 @@ static bool_t hvm_wait_for_io(struct hvm_ioreq_vcpu *sv, 
ioreq_t *p)
 break;
 default:
 gdprintk(XENLOG_ERR, "Weird HVM iorequest state %u\n", state);
-sv->pending = 0;
+sv->pending = false;
 domain_crash(sv->vcpu->domain);
-return 0; /* bail */
+return false; /* bail */
 }
 }
 
-return 1;
+return true;
 }
 
-bool_t handle_hvm_io_completion(struct vcpu *v)
+bool handle_hvm_io_completion(struct vcpu *v)
 {
 struct domain *d = v->domain;
 struct hvm_vcpu_io *vio = >arch.hvm_vcpu.hvm_io;
@@ -141,7 +141,7 @@ bool_t handle_hvm_io_completion(struct vcpu *v)
 if ( sv->vcpu == v && sv->pending )
 {
 if ( !hvm_wait_for_io(sv, get_ioreq(s, v)) )
-return 0;
+return false;
 
 break;
 }
@@ -178,7 +178,7 @@ bool_t handle_hvm_io_completion(struct vcpu *v)
 break;
 }
 
-return 1;
+return true;
 }
 
 static int hvm_alloc_ioreq_gfn(struct domain *d, unsigned long *gfn)
@@ -208,7 +208,7 @@ static void hvm_free_ioreq_gfn(struct domain *d, unsigned 
long gfn)
 set_bit(i, 

[Xen-devel] [PATCH v7 08/12] x86/hvm/ioreq: maintain an array of ioreq servers rather than a list

[Paul Durrant]

A subsequent patch will remove the current implicit limitation on creation
of ioreq servers which is due to the allocation of gfns for the ioreq
structures and buffered ioreq ring.

It will therefore be necessary to introduce an explicit limit and, since
this limit should be small, it simplifies the code to maintain an array of
that size rather than using a list.

Also, by reserving an array slot for the default server and populating
array slots early in create, the need to pass an 'is_default' boolean
to sub-functions can be avoided.

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 

v7:
 - Fixed assertion failure found in testing.

v6:
 - Updated according to comments made by Roger on v4 that I'd missed.

v5:
 - Switched GET/SET_IOREQ_SERVER() macros to get/set_ioreq_server()
   functions to avoid possible double-evaluation issues.

v4:
 - Introduced more helper macros and relocated them to the top of the
   code.

v3:
 - New patch (replacing "move is_default into struct hvm_ioreq_server") in
   response to review comments.
---
 xen/arch/x86/hvm/ioreq.c | 512 +++
 xen/include/asm-x86/hvm/domain.h |  11 +-
 2 files changed, 261 insertions(+), 262 deletions(-)

diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index f2e0b3f74a..fe29004e87 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -33,6 +33,32 @@
 
 #include 
 
+static void set_ioreq_server(struct domain *d, unsigned int id,
+ struct hvm_ioreq_server *s)
+{
+ASSERT(id < MAX_NR_IOREQ_SERVERS);
+ASSERT(!s || !d->arch.hvm_domain.ioreq_server.server[id]);
+
+d->arch.hvm_domain.ioreq_server.server[id] = s;
+}
+
+static struct hvm_ioreq_server *get_ioreq_server(struct domain *d,
+ unsigned int id)
+{
+if ( id >= MAX_NR_IOREQ_SERVERS )
+return NULL;
+
+return d->arch.hvm_domain.ioreq_server.server[id];
+}
+
+#define IS_DEFAULT(s) \
+((s) == get_ioreq_server((s)->domain, DEFAULT_IOSERVID))
+
+#define FOR_EACH_IOREQ_SERVER(d, id, s) \
+for ( (id) = 0, (s) = get_ioreq_server((d), (id)); \
+  (id) < MAX_NR_IOREQ_SERVERS; \
+  (s) = get_ioreq_server((d), ++(id)) )
+
 static ioreq_t *get_ioreq(struct hvm_ioreq_server *s, struct vcpu *v)
 {
 shared_iopage_t *p = s->ioreq.va;
@@ -47,13 +73,15 @@ bool hvm_io_pending(struct vcpu *v)
 {
 struct domain *d = v->domain;
 struct hvm_ioreq_server *s;
+unsigned int id;
 
-list_for_each_entry ( s,
-  >arch.hvm_domain.ioreq_server.list,
-  list_entry )
+FOR_EACH_IOREQ_SERVER(d, id, s)
 {
 struct hvm_ioreq_vcpu *sv;
 
+if ( !s )
+continue;
+
 list_for_each_entry ( sv,
   >ioreq_vcpu_list,
   list_entry )
@@ -127,13 +155,15 @@ bool handle_hvm_io_completion(struct vcpu *v)
 struct hvm_vcpu_io *vio = >arch.hvm_vcpu.hvm_io;
 struct hvm_ioreq_server *s;
 enum hvm_io_completion io_completion;
+unsigned int id;
 
-  list_for_each_entry ( s,
-  >arch.hvm_domain.ioreq_server.list,
-  list_entry )
+FOR_EACH_IOREQ_SERVER(d, id, s)
 {
 struct hvm_ioreq_vcpu *sv;
 
+if ( !s )
+continue;
+
 list_for_each_entry ( sv,
   >ioreq_vcpu_list,
   list_entry )
@@ -243,14 +273,16 @@ static int hvm_map_ioreq_page(
 bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
 {
 const struct hvm_ioreq_server *s;
+unsigned int id;
 bool found = false;
 
 spin_lock_recursive(>arch.hvm_domain.ioreq_server.lock);
 
-list_for_each_entry ( s,
-  >arch.hvm_domain.ioreq_server.list,
-  list_entry )
+FOR_EACH_IOREQ_SERVER(d, id, s)
 {
+if ( !s )
+continue;
+
 if ( (s->ioreq.va && s->ioreq.page == page) ||
  (s->bufioreq.va && s->bufioreq.page == page) )
 {
@@ -301,8 +333,9 @@ static void hvm_update_ioreq_evtchn(struct hvm_ioreq_server 
*s,
 }
 }
 
+
 static int hvm_ioreq_server_add_vcpu(struct hvm_ioreq_server *s,
- bool is_default, struct vcpu *v)
+ struct vcpu *v)
 {
 struct hvm_ioreq_vcpu *sv;
 int rc;
@@ -331,7 +364,7 @@ static int hvm_ioreq_server_add_vcpu(struct 
hvm_ioreq_server *s,
 goto fail3;
 
 s->bufioreq_evtchn = rc;
-if ( is_default )
+if ( IS_DEFAULT(s) )
 d->arch.hvm_domain.params[HVM_PARAM_BUFIOREQ_EVTCHN] =
 s->bufioreq_evtchn;
 }
@@ -431,7 +464,6 @@ static int 

[Xen-devel] [PATCH v7 02/12] x86/mm: add HYPERVISOR_memory_op to acquire guest resources

[Paul Durrant]

Certain memory resources associated with a guest are not necessarily
present in the guest P2M and so are not necessarily available to be
foreign-mapped by a tools domain unless they are inserted, which risks
shattering a super-page mapping.

This patch adds a new memory op to allow such a resource to be priv-mapped
directly, by either a PV or HVM tools domain: grant table frames.

NOTE: Whilst the new op is not intrinsicly specific to the x86 architecture,
  I have no means to test it on an ARM platform and so cannot verify
  that it functions correctly. Hence it is currently only implemented
  for x86.

Signed-off-by: Paul Durrant 
Acked-by: George Dunlap 
Reviewed-by: Wei Liu 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 

v5:
 - Switched __copy_to/from_guest_offset() to copy_to/from_guest_offset().
---
 xen/arch/x86/mm.c | 111 ++
 xen/arch/x86/mm/p2m.c |   3 +-
 xen/common/grant_table.c  |  56 ++---
 xen/include/asm-x86/p2m.h |   3 ++
 xen/include/public/memory.h   |  38 ++-
 xen/include/xen/grant_table.h |   1 +
 6 files changed, 191 insertions(+), 21 deletions(-)

diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index cb0189efae..c8f50f3bb0 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -4768,6 +4768,107 @@ int xenmem_add_to_physmap_one(
 return rc;
 }
 
+static int xenmem_acquire_grant_table(struct domain *d,
+  unsigned long frame,
+  unsigned long nr_frames,
+  unsigned long mfn_list[])
+{
+unsigned int i;
+
+/*
+ * Iterate through the list backwards so that gnttab_get_frame() is
+ * first called for the highest numbered frame. This means that the
+ * out-of-bounds check will be done on the first iteration and, if
+ * the table needs to grow, it will only grow once.
+ */
+i = nr_frames;
+while ( i-- != 0 )
+{
+mfn_t mfn = gnttab_get_frame(d, frame + i);
+
+if ( mfn_eq(mfn, INVALID_MFN) )
+return -EINVAL;
+
+mfn_list[i] = mfn_x(mfn);
+}
+
+return 0;
+}
+
+static int xenmem_acquire_resource(xen_mem_acquire_resource_t *xmar)
+{
+struct domain *d, *currd = current->domain;
+unsigned long *mfn_list;
+int rc;
+
+if ( xmar->nr_frames == 0 )
+return -EINVAL;
+
+d = rcu_lock_domain_by_any_id(xmar->domid);
+if ( d == NULL )
+return -ESRCH;
+
+rc = xsm_domain_memory_map(XSM_TARGET, d);
+if ( rc )
+goto out;
+
+mfn_list = xmalloc_array(unsigned long, xmar->nr_frames);
+
+rc = -ENOMEM;
+if ( !mfn_list )
+goto out;
+
+switch ( xmar->type )
+{
+case XENMEM_resource_grant_table:
+rc = -EINVAL;
+if ( xmar->id ) /* must be zero for grant_table */
+break;
+
+rc = xenmem_acquire_grant_table(d, xmar->frame, xmar->nr_frames,
+mfn_list);
+break;
+
+default:
+rc = -EOPNOTSUPP;
+break;
+}
+
+if ( rc )
+goto free_and_out;
+
+if ( !paging_mode_translate(currd) )
+{
+if ( copy_to_guest_offset(xmar->gmfn_list, 0, mfn_list,
+  xmar->nr_frames) )
+rc = -EFAULT;
+}
+else
+{
+unsigned int i;
+
+for ( i = 0; i < xmar->nr_frames; i++ )
+{
+xen_pfn_t gfn;
+
+rc = -EFAULT;
+if ( copy_from_guest_offset(, xmar->gmfn_list, i, 1) )
+goto free_and_out;
+
+rc = set_foreign_p2m_entry(currd, gfn, _mfn(mfn_list[i]));
+if ( rc )
+goto free_and_out;
+}
+}
+
+ free_and_out:
+xfree(mfn_list);
+
+ out:
+rcu_unlock_domain(d);
+return rc;
+}
+
 long arch_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
 {
 int rc;
@@ -4990,6 +5091,16 @@ long arch_memory_op(unsigned long cmd, 
XEN_GUEST_HANDLE_PARAM(void) arg)
 return rc;
 }
 
+case XENMEM_acquire_resource:
+{
+xen_mem_acquire_resource_t xmar;
+
+if ( copy_from_guest(, arg, 1) )
+return -EFAULT;
+
+return xenmem_acquire_resource();
+}
+
 default:
 return subarch_memory_op(cmd, arg);
 }
diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index 0b479105b9..d0f8fc249b 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -1121,8 +1121,7 @@ static int set_typed_p2m_entry(struct domain *d, unsigned 
long gfn, mfn_t mfn,
 }
 
 /* Set foreign mfn in the given guest's p2m table. */
-static int set_foreign_p2m_entry(struct domain *d, unsigned long gfn,
- mfn_t mfn)
+int set_foreign_p2m_entry(struct domain *d, unsigned long gfn, 

[Xen-devel] [PATCH v7 09/12] x86/hvm/ioreq: simplify code and use consistent naming

[Paul Durrant]

This patch re-works much of the ioreq server initialization and teardown
code:

- The hvm_map/unmap_ioreq_gfn() functions are expanded to call through
  to hvm_alloc/free_ioreq_gfn() rather than expecting them to be called
  separately by outer functions.
- Several functions now test the validity of the hvm_ioreq_page gfn value
  to determine whether they need to act. This means can be safely called
  for the bufioreq page even when it is not used.
- hvm_add/remove_ioreq_gfn() simply return in the case of the default
  IOREQ server so callers no longer need to test before calling.
- hvm_ioreq_server_setup_pages() is renamed to hvm_ioreq_server_map_pages()
  to mirror the existing hvm_ioreq_server_unmap_pages().

All of this significantly shortens the code.

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
Reviewed-by: Wei Liu 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 

v3:
 - Rebased on top of 's->is_default' to 'IS_DEFAULT(s)' changes.
 - Minor updates in response to review comments from Roger.
---
 xen/arch/x86/hvm/ioreq.c | 183 ++-
 1 file changed, 69 insertions(+), 114 deletions(-)

diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index fe29004e87..5f38d39ce2 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -211,63 +211,75 @@ bool handle_hvm_io_completion(struct vcpu *v)
 return true;
 }
 
-static int hvm_alloc_ioreq_gfn(struct domain *d, unsigned long *gfn)
+static unsigned long hvm_alloc_ioreq_gfn(struct hvm_ioreq_server *s)
 {
+struct domain *d = s->domain;
 unsigned int i;
-int rc;
 
-rc = -ENOMEM;
+ASSERT(!IS_DEFAULT(s));
+
 for ( i = 0; i < sizeof(d->arch.hvm_domain.ioreq_gfn.mask) * 8; i++ )
 {
 if ( test_and_clear_bit(i, >arch.hvm_domain.ioreq_gfn.mask) )
-{
-*gfn = d->arch.hvm_domain.ioreq_gfn.base + i;
-rc = 0;
-break;
-}
+return d->arch.hvm_domain.ioreq_gfn.base + i;
 }
 
-return rc;
+return gfn_x(INVALID_GFN);
 }
 
-static void hvm_free_ioreq_gfn(struct domain *d, unsigned long gfn)
+static void hvm_free_ioreq_gfn(struct hvm_ioreq_server *s,
+   unsigned long gfn)
 {
+struct domain *d = s->domain;
 unsigned int i = gfn - d->arch.hvm_domain.ioreq_gfn.base;
 
-if ( gfn != gfn_x(INVALID_GFN) )
-set_bit(i, >arch.hvm_domain.ioreq_gfn.mask);
+ASSERT(!IS_DEFAULT(s));
+ASSERT(gfn != gfn_x(INVALID_GFN));
+
+set_bit(i, >arch.hvm_domain.ioreq_gfn.mask);
 }
 
-static void hvm_unmap_ioreq_page(struct hvm_ioreq_server *s, bool buf)
+static void hvm_unmap_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
 {
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
 
+if ( iorp->gfn == gfn_x(INVALID_GFN) )
+return;
+
 destroy_ring_for_helper(>va, iorp->page);
+iorp->page = NULL;
+
+if ( !IS_DEFAULT(s) )
+hvm_free_ioreq_gfn(s, iorp->gfn);
+
+iorp->gfn = gfn_x(INVALID_GFN);
 }
 
-static int hvm_map_ioreq_page(
-struct hvm_ioreq_server *s, bool buf, unsigned long gfn)
+static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
 {
 struct domain *d = s->domain;
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
-struct page_info *page;
-void *va;
 int rc;
 
-if ( (rc = prepare_ring_for_helper(d, gfn, , )) )
-return rc;
-
-if ( (iorp->va != NULL) || d->is_dying )
-{
-destroy_ring_for_helper(, page);
+if ( d->is_dying )
 return -EINVAL;
-}
 
-iorp->va = va;
-iorp->page = page;
-iorp->gfn = gfn;
+if ( IS_DEFAULT(s) )
+iorp->gfn = buf ?
+d->arch.hvm_domain.params[HVM_PARAM_BUFIOREQ_PFN] :
+d->arch.hvm_domain.params[HVM_PARAM_IOREQ_PFN];
+else
+iorp->gfn = hvm_alloc_ioreq_gfn(s);
+
+if ( iorp->gfn == gfn_x(INVALID_GFN) )
+return -ENOMEM;
 
-return 0;
+rc = prepare_ring_for_helper(d, iorp->gfn, >page, >va);
+
+if ( rc )
+hvm_unmap_ioreq_gfn(s, buf);
+
+return rc;
 }
 
 bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
@@ -283,8 +295,7 @@ bool is_ioreq_server_page(struct domain *d, const struct 
page_info *page)
 if ( !s )
 continue;
 
-if ( (s->ioreq.va && s->ioreq.page == page) ||
- (s->bufioreq.va && s->bufioreq.page == page) )
+if ( (s->ioreq.page == page) || (s->bufioreq.page == page) )
 {
 found = true;
 break;
@@ -296,20 +307,30 @@ bool is_ioreq_server_page(struct domain *d, const struct 
page_info *page)
 return found;
 }
 
-static void hvm_remove_ioreq_gfn(
-struct domain *d, struct hvm_ioreq_page *iorp)
+static void hvm_remove_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
+
 {
+struct 

[Xen-devel] [PATCH v7 06/12] x86/hvm/ioreq: rename .*pfn and .*gmfn to .*gfn

[Paul Durrant]

Since ioreq servers are only relevant to HVM guests and all the names in
question unequivocally refer to guest frame numbers, name them all .*gfn
to avoid any confusion.

This patch is purely cosmetic. No semantic or functional change.

Signed-off-by: Paul Durrant 
Reviewed-by: Wei Liu 
Reviewed-by: Roger Pau Monné 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 tools/libs/devicemodel/core.c   | 10 ++--
 tools/libs/devicemodel/include/xendevicemodel.h | 12 ++--
 xen/arch/x86/hvm/dm.c   |  4 +-
 xen/arch/x86/hvm/hvm.c  |  6 +-
 xen/arch/x86/hvm/ioreq.c| 74 -
 xen/include/asm-x86/hvm/domain.h|  4 +-
 xen/include/asm-x86/hvm/ioreq.h |  4 +-
 xen/include/public/hvm/dm_op.h  | 20 +++
 8 files changed, 67 insertions(+), 67 deletions(-)

diff --git a/tools/libs/devicemodel/core.c b/tools/libs/devicemodel/core.c
index d7c6476006..fcb260d29b 100644
--- a/tools/libs/devicemodel/core.c
+++ b/tools/libs/devicemodel/core.c
@@ -174,7 +174,7 @@ int xendevicemodel_create_ioreq_server(
 
 int xendevicemodel_get_ioreq_server_info(
 xendevicemodel_handle *dmod, domid_t domid, ioservid_t id,
-xen_pfn_t *ioreq_pfn, xen_pfn_t *bufioreq_pfn,
+xen_pfn_t *ioreq_gfn, xen_pfn_t *bufioreq_gfn,
 evtchn_port_t *bufioreq_port)
 {
 struct xen_dm_op op;
@@ -192,11 +192,11 @@ int xendevicemodel_get_ioreq_server_info(
 if (rc)
 return rc;
 
-if (ioreq_pfn)
-*ioreq_pfn = data->ioreq_pfn;
+if (ioreq_gfn)
+*ioreq_gfn = data->ioreq_gfn;
 
-if (bufioreq_pfn)
-*bufioreq_pfn = data->bufioreq_pfn;
+if (bufioreq_gfn)
+*bufioreq_gfn = data->bufioreq_gfn;
 
 if (bufioreq_port)
 *bufioreq_port = data->bufioreq_port;
diff --git a/tools/libs/devicemodel/include/xendevicemodel.h 
b/tools/libs/devicemodel/include/xendevicemodel.h
index 580fad2f49..13216db04a 100644
--- a/tools/libs/devicemodel/include/xendevicemodel.h
+++ b/tools/libs/devicemodel/include/xendevicemodel.h
@@ -60,17 +60,17 @@ int xendevicemodel_create_ioreq_server(
  * @parm dmod a handle to an open devicemodel interface.
  * @parm domid the domain id to be serviced
  * @parm id the IOREQ Server id.
- * @parm ioreq_pfn pointer to a xen_pfn_t to receive the synchronous ioreq
- *  gmfn
- * @parm bufioreq_pfn pointer to a xen_pfn_t to receive the buffered ioreq
- *gmfn
+ * @parm ioreq_gfn pointer to a xen_pfn_t to receive the synchronous ioreq
+ *  gfn
+ * @parm bufioreq_gfn pointer to a xen_pfn_t to receive the buffered ioreq
+ *gfn
  * @parm bufioreq_port pointer to a evtchn_port_t to receive the buffered
  * ioreq event channel
  * @return 0 on success, -1 on failure.
  */
 int xendevicemodel_get_ioreq_server_info(
 xendevicemodel_handle *dmod, domid_t domid, ioservid_t id,
-xen_pfn_t *ioreq_pfn, xen_pfn_t *bufioreq_pfn,
+xen_pfn_t *ioreq_gfn, xen_pfn_t *bufioreq_gfn,
 evtchn_port_t *bufioreq_port);
 
 /**
@@ -168,7 +168,7 @@ int xendevicemodel_destroy_ioreq_server(
  * This function sets IOREQ Server state. An IOREQ Server
  * will not be passed emulation requests until it is in
  * the enabled state.
- * Note that the contents of the ioreq_pfn and bufioreq_pfn are
+ * Note that the contents of the ioreq_gfn and bufioreq_gfn are
  * not meaningful until the IOREQ Server is in the enabled state.
  *
  * @parm dmod a handle to an open devicemodel interface.
diff --git a/xen/arch/x86/hvm/dm.c b/xen/arch/x86/hvm/dm.c
index 4cf6deedc7..f7cb883fec 100644
--- a/xen/arch/x86/hvm/dm.c
+++ b/xen/arch/x86/hvm/dm.c
@@ -426,8 +426,8 @@ static int dm_op(const struct dmop_args *op_args)
 break;
 
 rc = hvm_get_ioreq_server_info(d, data->id,
-   >ioreq_pfn,
-   >bufioreq_pfn,
+   >ioreq_gfn,
+   >bufioreq_gfn,
>bufioreq_port);
 break;
 }
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 6cb903def5..58b4afa1d1 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -4185,20 +4185,20 @@ static int hvmop_set_param(
 rc = -EINVAL;
 break;
 case HVM_PARAM_IOREQ_SERVER_PFN:
-d->arch.hvm_domain.ioreq_gmfn.base = a.value;
+d->arch.hvm_domain.ioreq_gfn.base = a.value;
 break;
 case HVM_PARAM_NR_IOREQ_SERVER_PAGES:
 {
 unsigned int i;
 
 if ( a.value == 0 ||
- a.value > sizeof(d->arch.hvm_domain.ioreq_gmfn.mask) * 8 )
+ a.value > sizeof(d->arch.hvm_domain.ioreq_gfn.mask) * 8 )
 {
 rc = -EINVAL;

[Xen-devel] [PATCH v7 00/12] x86: guest resource mapping

[Paul Durrant]

This series introduces support for direct mapping of guest resources.
The resources are:
 - Grant tables
 - IOREQ server pages

NOTE: This series is based on a master re-base of Juergen Gross's patch "xen: 
move
XENMAPSPACE_grant_table code into grant_table.c". For convenience the code is 
also available
on a branch at:

http://xenbits.xen.org/gitweb/?p=people/pauldu/xen.git;a=shortlog;h=refs/heads/ioreq11

v7:
 - Fixed assertion failure hit during domain destroy.

v6:
 - Responded to missed comments from Roger.

v5:
 - Responded to review comments from Wei.

v4:
 - Responded to further review comments from Roger.

v3:
 - Dropped original patch #1 since it is covered by Juergen's patch.
 - Added new xenforeignmemorycleanup patch (#4).
 - Replaced the patch introducing the ioreq server 'is_default' flag with one
   that changes the ioreq server list into an array (#8).

Paul Durrant (12):
  x86/mm: allow a privileged PV domain to map guest mfns
  x86/mm: add HYPERVISOR_memory_op to acquire guest resources
  tools/libxenforeignmemory: add support for resource mapping
  tools/libxenforeignmemory: reduce xenforeignmemory_restrict code
footprint
  tools/libxenctrl: use new xenforeignmemory API to seed grant table
  x86/hvm/ioreq: rename .*pfn and .*gmfn to .*gfn
  x86/hvm/ioreq: use bool rather than bool_t
  x86/hvm/ioreq: maintain an array of ioreq servers rather than a list
  x86/hvm/ioreq: simplify code and use consistent naming
  x86/hvm/ioreq: use gfn_t in struct hvm_ioreq_page
  x86/hvm/ioreq: defer mapping gfns until they are actually requsted
  x86/hvm/ioreq: add a new mappable resource type...

 tools/include/xen-sys/Linux/privcmd.h  |  11 +
 tools/libs/devicemodel/core.c  |  18 +-
 tools/libs/devicemodel/include/xendevicemodel.h|  14 +-
 tools/libs/foreignmemory/Makefile  |   2 +-
 tools/libs/foreignmemory/core.c|  53 ++
 tools/libs/foreignmemory/freebsd.c |   7 -
 .../libs/foreignmemory/include/xenforeignmemory.h  |  41 +
 tools/libs/foreignmemory/libxenforeignmemory.map   |   5 +
 tools/libs/foreignmemory/linux.c   |  45 ++
 tools/libs/foreignmemory/minios.c  |   7 -
 tools/libs/foreignmemory/netbsd.c  |   7 -
 tools/libs/foreignmemory/private.h |  43 +-
 tools/libs/foreignmemory/solaris.c |   7 -
 tools/libxc/include/xc_dom.h   |   8 +-
 tools/libxc/xc_dom_boot.c  | 114 ++-
 tools/libxc/xc_sr_restore_x86_hvm.c|  10 +-
 tools/libxc/xc_sr_restore_x86_pv.c |   2 +-
 tools/libxl/libxl_dom.c|   1 -
 tools/python/xen/lowlevel/xc/xc.c  |   6 +-
 xen/arch/x86/hvm/dm.c  |  11 +-
 xen/arch/x86/hvm/hvm.c |   8 +-
 xen/arch/x86/hvm/io.c  |   4 +-
 xen/arch/x86/hvm/ioreq.c   | 869 +++--
 xen/arch/x86/mm.c  | 151 +++-
 xen/arch/x86/mm/p2m.c  |   3 +-
 xen/common/grant_table.c   |  56 +-
 xen/include/asm-x86/hvm/domain.h   |  21 +-
 xen/include/asm-x86/hvm/ioreq.h|  24 +-
 xen/include/asm-x86/p2m.h  |   3 +
 xen/include/public/hvm/dm_op.h |  46 +-
 xen/include/public/memory.h|  41 +-
 xen/include/xen/grant_table.h  |   1 +
 32 files changed, 1081 insertions(+), 558 deletions(-)

---
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Jan Beulich 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Wei Liu 

-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v7 05/12] tools/libxenctrl: use new xenforeignmemory API to seed grant table

[Paul Durrant]

A previous patch added support for priv-mapping guest resources directly
(rather than having to foreign-map, which requires P2M modification for
HVM guests).

This patch makes use of the new API to seed the guest grant table unless
the underlying infrastructure (i.e. privcmd) doesn't support it, in which
case the old scheme is used.

NOTE: The call to xc_dom_gnttab_hvm_seed() in hvm_build_set_params() was
  actually unnecessary, as the grant table has already been seeded
  by a prior call to xc_dom_gnttab_init() made by libxl__build_dom().

Signed-off-by: Paul Durrant 
Acked-by: Marek Marczykowski-Górecki 
Reviewed-by: Roger Pau Monné 
Acked-by: Wei Liu 
---
Cc: Ian Jackson 

v4:
 - Minor cosmetic fix suggested by Roger.

v3:
 - Introduced xc_dom_set_gnttab_entry() to avoid duplicated code.
---
 tools/libxc/include/xc_dom.h|   8 +--
 tools/libxc/xc_dom_boot.c   | 114 +---
 tools/libxc/xc_sr_restore_x86_hvm.c |  10 ++--
 tools/libxc/xc_sr_restore_x86_pv.c  |   2 +-
 tools/libxl/libxl_dom.c |   1 -
 tools/python/xen/lowlevel/xc/xc.c   |   6 +-
 6 files changed, 92 insertions(+), 49 deletions(-)

diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
index ce47058c41..d6ca0a8680 100644
--- a/tools/libxc/include/xc_dom.h
+++ b/tools/libxc/include/xc_dom.h
@@ -323,12 +323,8 @@ void *xc_dom_boot_domU_map(struct xc_dom_image *dom, 
xen_pfn_t pfn,
 int xc_dom_boot_image(struct xc_dom_image *dom);
 int xc_dom_compat_check(struct xc_dom_image *dom);
 int xc_dom_gnttab_init(struct xc_dom_image *dom);
-int xc_dom_gnttab_hvm_seed(xc_interface *xch, domid_t domid,
-   xen_pfn_t console_gmfn,
-   xen_pfn_t xenstore_gmfn,
-   domid_t console_domid,
-   domid_t xenstore_domid);
-int xc_dom_gnttab_seed(xc_interface *xch, domid_t domid,
+int xc_dom_gnttab_seed(xc_interface *xch, domid_t guest_domid,
+   bool is_hvm,
xen_pfn_t console_gmfn,
xen_pfn_t xenstore_gmfn,
domid_t console_domid,
diff --git a/tools/libxc/xc_dom_boot.c b/tools/libxc/xc_dom_boot.c
index c3b44dd399..dc0a1fdee8 100644
--- a/tools/libxc/xc_dom_boot.c
+++ b/tools/libxc/xc_dom_boot.c
@@ -280,11 +280,29 @@ static xen_pfn_t xc_dom_gnttab_setup(xc_interface *xch, 
domid_t domid)
 return gmfn;
 }
 
-int xc_dom_gnttab_seed(xc_interface *xch, domid_t domid,
-   xen_pfn_t console_gmfn,
-   xen_pfn_t xenstore_gmfn,
-   domid_t console_domid,
-   domid_t xenstore_domid)
+static void xc_dom_set_gnttab_entry(xc_interface *xch,
+grant_entry_v1_t *gnttab,
+unsigned int idx,
+domid_t guest_domid,
+domid_t backend_domid,
+xen_pfn_t backend_gmfn)
+{
+if ( guest_domid == backend_domid || backend_gmfn == -1)
+return;
+
+xc_dom_printf(xch, "%s: [%u] -> 0x%"PRI_xen_pfn,
+  __FUNCTION__, idx, backend_gmfn);
+
+gnttab[idx].flags = GTF_permit_access;
+gnttab[idx].domid = backend_domid;
+gnttab[idx].frame = backend_gmfn;
+}
+
+static int compat_gnttab_seed(xc_interface *xch, domid_t domid,
+  xen_pfn_t console_gmfn,
+  xen_pfn_t xenstore_gmfn,
+  domid_t console_domid,
+  domid_t xenstore_domid)
 {
 
 xen_pfn_t gnttab_gmfn;
@@ -308,18 +326,10 @@ int xc_dom_gnttab_seed(xc_interface *xch, domid_t domid,
 return -1;
 }
 
-if ( domid != console_domid  && console_gmfn != -1)
-{
-gnttab[GNTTAB_RESERVED_CONSOLE].flags = GTF_permit_access;
-gnttab[GNTTAB_RESERVED_CONSOLE].domid = console_domid;
-gnttab[GNTTAB_RESERVED_CONSOLE].frame = console_gmfn;
-}
-if ( domid != xenstore_domid && xenstore_gmfn != -1)
-{
-gnttab[GNTTAB_RESERVED_XENSTORE].flags = GTF_permit_access;
-gnttab[GNTTAB_RESERVED_XENSTORE].domid = xenstore_domid;
-gnttab[GNTTAB_RESERVED_XENSTORE].frame = xenstore_gmfn;
-}
+xc_dom_set_gnttab_entry(xch, gnttab, GNTTAB_RESERVED_CONSOLE,
+domid, console_domid, console_gmfn);
+xc_dom_set_gnttab_entry(xch, gnttab, GNTTAB_RESERVED_XENSTORE,
+domid, xenstore_domid, xenstore_gmfn);
 
 if ( munmap(gnttab, PAGE_SIZE) == -1 )
 {
@@ -337,11 +347,11 @@ int xc_dom_gnttab_seed(xc_interface *xch, domid_t domid,
 return 0;
 }
 
-int xc_dom_gnttab_hvm_seed(xc_interface *xch, domid_t domid,
-   xen_pfn_t 

[Xen-devel] [PATCH v7 01/12] x86/mm: allow a privileged PV domain to map guest mfns

[Paul Durrant]

In the case where a PV domain is mapping guest resources then it needs make
the HYPERVISOR_mmu_update call using DOMID_SELF, rather than the guest
domid, so that the passed in gmfn values are correctly treated as mfns
rather than gfns present in the guest p2m.

This patch removes a check which currently disallows mapping of a page when
the owner of the page tables matches the domain passed to
HYPERVISOR_mmu_update, but that domain is not the real owner of the page.
The check was introduced by patch d3c6a215ca9 ("x86: Clean up
get_page_from_l1e() to correctly distinguish between owner-of-pte and
owner-of-data-page in all cases") but it's not clear why it was needed.

Signed-off-by: Paul Durrant 
---
Cc: Jan Beulich 
Cc: Andrew Cooper 
---
 xen/arch/x86/mm.c | 13 -
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 2e5b15e7a2..cb0189efae 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -1024,12 +1024,15 @@ get_page_from_l1e(
(real_pg_owner != dom_cow) ) )
 {
 /*
- * Let privileged domains transfer the right to map their target
- * domain's pages. This is used to allow stub-domain pvfb export to
- * dom0, until pvfb supports granted mappings. At that time this
- * minor hack can go away.
+ * If the real page owner is not the domain specified in the
+ * hypercall then establish that the specified domain has
+ * mapping privilege over the page owner.
+ * This is used to allow stub-domain pvfb export to dom0. It is
+ * also used to allow a privileged PV domain to map mfns using
+ * DOMID_SELF, which is needed for mapping guest resources such
+ * grant table frames.
  */
-if ( (real_pg_owner == NULL) || (pg_owner == l1e_owner) ||
+if ( (real_pg_owner == NULL) ||
  xsm_priv_mapping(XSM_TARGET, pg_owner, real_pg_owner) )
 {
 gdprintk(XENLOG_WARNING,
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Xen-users] UEFI Secure Boot Xen 4.9

[Tamas K Lengyel]

On Tue, Sep 5, 2017 at 12:26 PM, Tamas K Lengyel
 wrote:
> On Mon, Sep 4, 2017 at 6:40 AM, Daniel Kiper  wrote:
>> On Wed, Aug 30, 2017 at 10:16:23AM -0600, Tamas K Lengyel wrote:
>>> On Tue, Aug 29, 2017 at 2:01 PM, Daniel Kiper  
>>> wrote:
>>> > Hey Tamas,
>>> >
>>> > Sorry for late reply. I was on vacation.
>>> >
>>> > On Tue, Aug 22, 2017 at 09:01:06PM -0600, Tamas K Lengyel wrote:
>>> >> On Tue, May 16, 2017 at 5:04 AM, Daniel Kiper  
>>> >> wrote:
>>> >
>>> > [...]
>>> >
>>> >> > UEFI will verify shim secure boot signature then shim will verify GRUB2
>>> >> > signature then GRUB2 will verify (with shim protocol) Xen signature and
>>> >> > finally Xen will verify (with shim protocol) Linux kernel signature. 
>>> >> > Then
>>> >> > your kernel can verify modules using whatever you want.
>>> >> >
>>> >> >> I would be happy to work to help achieve this.
>>> >> >
>>> >> > There is a chance that I will have something very raw at the beginning
>>> >> > of June. If you wish to do tests drop me a line.
>>> >>
>>> >> Hi Daniel,
>>> >> is there any news on this? I would be interested in giving this a shot 
>>> >> too.
>>> >
>>> > Please look at
>>> >
>>> >   https://lists.xen.org/archives/html/xen-devel/2017-07/msg00982.html
>>> >
>>> > and at
>>> >
>>> >   https://lists.xen.org/archives/html/xen-devel/2017-07/msg00985.html
>>> >
>>> > Attachments contain the same patches as above but rebased on latest
>>> > GRUB2 and Xen git repositories.
>>> >
>>> > Due to some travel I am going to restart work on this in the second
>>> > half of September.
>>> >
>>> > If you have any questions please drop me a line.
>>> >
>>>
>>> Hi Daniel,
>>> thanks for the update, I'll give it a shot today to set it up. In a
>>> somewhat related note, are you aware of any work on getting secure
>>> boot + UEFI working in a guest? There is a PoC patch on OpenXT
>>> (https://github.com/OpenXT/xenclient-oe/pull/729) but was wondering if
>>> there are any parallel efforts ongoing.
>>
>> I do not follow this issue in detail. However, I suppose that if OVMF
>> supports UEFI secure boot (well, QEMU has to enable SMM support too;
>> I do not know does it work with Xen or not) then guest should work
>> without any issue. Just guessing...
>>
>
> Sure, was just wondering if you are aware of anyone looking at that.
>
> In other news I was able to get your patches working and have been
> able to boot with Secure boot enabled as far as shim -> signed grub ->
> signed linux without initrd. If I boot a signed version of Xen from
> grub it goes as far as setup_efi_pci but then the system reboots
> without anything else being printed on the screen. I haven't been able
> to debug it any further yet.
>

Daniel,
just FYI the xen.mb.efi generated with your patches causes pesign to segfault:

cms_pe_common.c:generate_digest:198 PE section ".text" has invalid address
Segmentation fault

Tamas

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v6 06/12] tools: set grant limits for xenstore stubdom

[Wei Liu]

On Wed, Sep 13, 2017 at 05:46:45PM +0200, Juergen Gross wrote:
> When creating a Xenstore stubdom set the grant limits: the stubdom
> will need to setup a very limited amount of grants only, so 1 grant
> frame is enough. For being able to support up to 32768 domains it
> will need 128 maptrack frames (1 mapping per domain, 256 maptrack
> entries per maptrack frame).
> 
> Signed-off-by: Juergen Gross 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v6 01/12] xen: correct gnttab_get_status_frames()

[Wei Liu]

On Wed, Sep 13, 2017 at 05:46:40PM +0200, Juergen Gross wrote:
> In gnttab_get_status_frames() all accesses to nr_status_frames should
> be done with the grant table lock held.
> 
> While at it correct coding style: labels should be indented by one
> space.
> 
> Signed-off-by: Juergen Gross 

Reviewed-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 03/22] xentoolcore, _restrict_all: Introduce new library and implementation

[Wei Liu]

On Fri, Sep 15, 2017 at 07:48:40PM +0100, Ian Jackson wrote:
> diff --git a/tools/libs/toolcore/include/xentoolcore.h 
> b/tools/libs/toolcore/include/xentoolcore.h
> new file mode 100644
> index 000..1ab646e
> --- /dev/null
> +++ b/tools/libs/toolcore/include/xentoolcore.h
> @@ -0,0 +1,71 @@
> +/*
> + * xentoolcore.h
> + *
> + * Copyright (c) 2017 Citrix
> + * 
> + * Common features used/provided by all Xen tools libraries
> + *
> + * This library is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation;
> + * version 2.1 of the License.
> + *
> + * This library is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with this library; If not, see 
> .
> + */
> +
> +#ifndef XENTOOLCORE_H
> +#define XENTOOLCORE_H
> +
> +#include 
> +
> +/*
> + * int xentoolcore_restrict_all(uint32_t domid);
> + *
> + * Arranges that Xen library handles (fds etc.) which are currently held
> + * by Xen libraries, can no longer be used other than to affect domid.
> + *
> + * If this cannot be achieved, returns -1 and sets errno.
> + * Idempotent if domid is always the same.
> + *
> + *  
> + *  IMPORTANT - IMPLEMENTATION STATUS
> + *
> + *  This function will be implemented insofar as it appears necessary
> + *  for the purposes of running a deprivileged qemu.
> + *
> + *  However, this function is NOT implemented for all Xen libraries.
> + *  For each use case of this function, the designer must evaluate and
> + *  audit whether the implementation is sufficient in their specific
> + *  context.
> + *
> + *  Of course, patches to extend the implementation are very welcome.
> + *  
> + *
> + * Thread safe.
> + *
> + * We expect that no callers do the following:
> + *   - in one thread call xen_somelibrary_open|close
> + *   - in another thread call fork
> + *   - in the child of the fork, before exec, call
> + * xen_some[other]library_open|close or xentoolcore_restrict_all
> + *
> + */
> +int xentoolcore_restrict_all(uint32_t domid);
> +
> +#endif /* XENTOOLCORE_H */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/tools/libs/toolcore/include/xentoolcore_internal.h 
> b/tools/libs/toolcore/include/xentoolcore_internal.h
> new file mode 100644
> index 000..670e29d
> --- /dev/null
> +++ b/tools/libs/toolcore/include/xentoolcore_internal.h
> @@ -0,0 +1,102 @@
> +/*
> + * xentoolcore_internal.h
> + *
> + * Interfaces of xentoolcore directed internally at other Xen libraries
> + *
> + * Copyright (c) 2017 Citrix
> + * 
> + * Common code used by all Xen tools libraries
> + *
> + * This library is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation;
> + * version 2.1 of the License.
> + *
> + * This library is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with this library; If not, see 
> .
> + */
> +
> +#ifndef XENTOOLCORE_INTERNAL_H
> +#define XENTOOLCORE_INTERNAL_H
> +
> +#include "xentoolcore.h"
> +#include "_xentoolcore_list.h"
> +
> +/*-- active handle registration --*/
> +
> +/*
> + * This is all to support xentoolcore_restrict_all
> + *
> + * Any libxl library that opens a Xen control handle of any kind which
> + * might allow manipulation of dom0, of other domains, or of the whole
> + * machine, must:
> + *   I. arrange that their own datastructure contains a
> + *  Xentoolcore__Active_Handle
> + * 
> + *   II. during the "open handle" function
> + * 1. allocate the memory for the own datastructure and initialise it
> + * 2. set Xentoolcore__Active_Handle.restrict_callback
> + * 3. call xentoolcore__register_active_handle
> + *   3a. if the open fails, call xentoolcore__deregister_active_handle
> + * 4. ONLY THEN actually open the relevant fd or whatever
> + *
> + *   III. during the "close handle" function
> + * 1. FIRST close the relevant fd or whatever
> + * 2. call xentoolcore__deregister_active_handle
> + *
> + *   IV. in the 

Re: [Xen-devel] [PATCH v3 1/1] public/io/netif.h: add gref mapping control messages

[Joao Martins]

On Mon, Sep 18, 2017 at 12:11:04PM +, Paul Durrant wrote:
> > -Original Message-
> > From: Joao Martins [mailto:joao.m.mart...@oracle.com]
> > Sent: 18 September 2017 12:56
> > To: Paul Durrant 
> > Cc: Xen-devel ; Wei Liu ;
> > Konrad Rzeszutek Wilk 
> > Subject: Re: [PATCH v3 1/1] public/io/netif.h: add gref mapping control
> > messages
> > 
> > On Mon, Sep 18, 2017 at 09:53:18AM +, Paul Durrant wrote:
> > > > -Original Message-
> > > > From: Joao Martins [mailto:joao.m.mart...@oracle.com]
> > > > Sent: 13 September 2017 19:11
> > > > To: Xen-devel 
> > > > Cc: Wei Liu ; Paul Durrant
> > ;
> > > > Konrad Rzeszutek Wilk ; Joao Martins
> > > > 
> > > > Subject: [PATCH v3 1/1] public/io/netif.h: add gref mapping control
> > messages

[snip]

> > > > + * XEN_NETIF_CTRL_TYPE_ADD_GREF_MAPPING
> > > > + * 
> > > > + *
> > > > + * This is sent by the frontend for backend to map a list of grant
> > > > + * references.
> > > > + *
> > > > + * Request:
> > > > + *
> > > > + *  type= XEN_NETIF_CTRL_TYPE_ADD_GREF_MAPPING
> > > > + *  data[0] = queue index
> > > > + *  data[1] = grant reference of page containing the mapping list
> > > > + *(r/w and assumed to start at beginning of page)
> > > > + *  data[2] = size of list in entries
> > > > + *
> > > > + * Response:
> > > > + *
> > > > + *  status = XEN_NETIF_CTRL_STATUS_NOT_SUPPORTED - Operation
> > not
> > > > + * supported
> > > > + *   XEN_NETIF_CTRL_STATUS_INVALID_PARAMETER - Operation
> > failed
> > > > + *   XEN_NETIF_CTRL_STATUS_SUCCESS   - Operation 
> > > > successful
> > > > + *  data   = number of entries that were mapped
> > > > + *
> > > > + * NOTE: Each entry in the input table has the format outlined
> > > > + *   in struct xen_netif_gref.
> > >
> > > You may want to put words here about the 'all or nothing' semantics of
> > > this operation vs. the semantics of the 'del' operation below.
> > >
> > Good point I'll add a paragraph about that.
> > 
> > For the unmap it is clear that status should be per-entry for reasons
> > discussed on v2. Do you think ADD 'all or nothing' like I had on v2 ?
> > If so I should remove the 'data' return part since it is not really
> > useful here.
> 
> The 'all or nothing' semantic is easier for the frontend to deal with,
> so I think that's the way to go. Otherwise you'd need the per-entry
> status, as you say. Either way, I don't think the data return is
> particularly useful.
> 
Yeap.

The 'data' return was to allow both cases but leaving the decision to
implementors meaning if number of mapped entries was the same as the
input size (data[2]) then frontend wouldn't need to check all entries.
But it would still need to unmap on partial success, as that
is not guaranteed by design. On a 'all or nothing', 'data' doesn't really has
any meaning and definitely makes life easier for frontend.

> > 
> > > > + *
> > > > + * XEN_NETIF_CTRL_TYPE_DEL_GREF_MAPPING
> > > > + * 
> > > > + *
> > > > + * This is sent by the frontend for backend to unmap a list of grant
> > > > + * references.
> > > > + *
> > > > + * Request:
> > > > + *
> > > > + *  type= XEN_NETIF_CTRL_TYPE_DEL_GREF_MAPPING
> > > > + *  data[0] = queue index
> > > > + *  data[1] = grant reference of page containing the mapping list
> > > > + *(r/w and assumed to start at beginning of page)
> > > > + *  data[2] = size of list in entries
> > > > + *
> > > > + * Response:
> > > > + *
> > > > + *  status = XEN_NETIF_CTRL_STATUS_NOT_SUPPORTED - Operation
> > not
> > > > + * supported
> > > > + *   XEN_NETIF_CTRL_STATUS_INVALID_PARAMETER - Operation
> > failed
> > > > + *   XEN_NETIF_CTRL_STATUS_SUCCESS   - Operation 
> > > > successful
> > > > + *  data   = number of entries that were unmapped
> > > > + *
> > > > + * NOTE: Each entry in the input table has the format outlined in 
> > > > struct
> > > > + *   xen_netif_gref. The only valid entries are those previously 
> > > > added
> > with
> > > > + *   message XEN_NETIF_CTRL_TYPE_ADD_GREF_MAPPING are valid,
> > > > otherwise
> > > > + *   entries status are set to
> > > > XEN_NETIF_CTRL_STATUS_INVALID_PARAMETER.
> > > > + *   It will also have the same error for entries inflight i.e. 
> > > > used in
> > > > + *   requests for which haven't been sent responses to the the
> > frontend.
> > >
> > > This last paragraph is a bit garbled.
> > 
> > Hmm, sorry for the twisted english. I removed the inflight part which
> > was making it more confusing and made it clear in the paragraph that the
> > only 

Re: [Xen-devel] [PATCH 18/22] libxl: Rationalise calculation of user to run qemu as

[Wei Liu]

On Fri, Sep 15, 2017 at 07:48:55PM +0100, Ian Jackson wrote:
> If the config specifies a user we use that.  Otherwise:
> 
> When we are not restricting qemu, there is very little point running
> it as a different user than root.  Indeed, previously, creating the
> "magic" users would cause qemu to become slightly dysfunctional (for
> example, you can't insert a cd that the qemu user can't read).
> So, in that case, default to running it as root.
> 
> Conversely, if restriction is requested, we must insist on running
> qemu as a non-root user.
> 
> Sadly the admin is still required to create 2^16-epsilon users!
> 

I will let Stefano comment on this.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 16/22] xentoolcore, _restrict_all: Document implementation "complete"

[Wei Liu]

On Fri, Sep 15, 2017 at 07:48:53PM +0100, Ian Jackson wrote:
> Signed-off-by: Ian Jackson 
> ---
>  tools/libs/toolcore/include/xentoolcore.h | 6 --
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/libs/toolcore/include/xentoolcore.h 
> b/tools/libs/toolcore/include/xentoolcore.h
> index 1210d7d..52f7aa3 100644
> --- a/tools/libs/toolcore/include/xentoolcore.h
> +++ b/tools/libs/toolcore/include/xentoolcore.h
> @@ -41,8 +41,10 @@
>   *  
>   *  IMPORTANT - IMPLEMENTATION STATUS
>   *
> - *  This function will be implemented insofar as it appears necessary
> - *  for the purposes of running a deprivileged qemu.
> + *  This function has been implemented insofar as it appears necessary
> + *  for the purposes of running a deprivileged qemu, and is believed to
> + *  be sufficient (subject to the caveats discussed in the appropriate
> + *  libxl documatation for this feature).

documentation


>   *
>   *  However, this function is NOT implemented for all Xen libraries.
>   *  For each use case of this function, the designer must evaluate and
> -- 
> 2.1.4
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 2/2] Introduce migration precopy policy

[Jennifer Herbert]

On 18/09/17 14:30, Ian Jackson wrote:

Jennifer Herbert writes ("[PATCH 2/2] Introduce migration precopy policy"):

This Patch allows a migration precopy policy to be specified.

But only for direct libxc users.  How do you think this should be
exposed via libxl ?


The original patch had a small amount of code to pass this though such 
that libxl could provide one.  I'm not entity sure how the perl code 
worked, but I've not made any changes that wouldn't stop this working.  
That section would form a follow on patch.  Since I don't use or have 
much understanding of libxl, I didn't really feel I was the best person 
to do this.


-jenny


The general approach, so far, seems sound.

Thanks,
Ian.



___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] ARM64:Porting xen to new hardware

[Konrad Rzeszutek Wilk]

On Fri, Sep 08, 2017 at 10:19:55PM +0300, Oleksandr Tyshchenko wrote:
> Hi Bharat
> 
> On Thu, Sep 7, 2017 at 4:30 PM, bharat gohil  wrote:
> > Hello Olensandr,
> >
> > I able to boot xen and trying to boot dom0 but there are no console log for
> > dom0.
> >
> > following log for xen and it stuck booting dom0.
> >
> > (XEN) I/O virtualisation disabled
> > (XEN) build-id: 7c2a3c70fb94754801d18c4cb9e3db3ffa01d8c4
> > (XEN) alternatives: Patching with alt table 400d2e08 ->
> > 400d32dc
> > (XEN) *** LOADING DOMAIN 0 ***
> > (XEN) Loading kernel from boot module @ 40148158
> > (XEN) Allocating 1:1 mappings totalling 128MB for dom0:
> > (XEN) BANK[0] 0x004800-0x005000 (128MB)
> > (XEN) Grant table range: 0x00bfe0-0x00bfe65000
> > (XEN) Loading zImage from 40148158 to
> > 4808-4948
> > (XEN) Allocating PPI 16 for event channel interrupt
> > (XEN) Loading dom0 DTB to 0x4fe0-0x4fe0f31e
> > (XEN) Scrubbing Free RAM on 1 nodes using 3 CPUs
> > (XEN) ..done.
> > (XEN) Initial low memory virq threshold set at 0x4000 pages.
> > (XEN) Std. Loglevel: All
> > (XEN) Guest Loglevel: All
> > (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to
> > Xen)
> > (XEN) Freed 272kB init memory.
> >
> > I have done all the xen configuration in linux kernel 4.9. This kernel
> > booting fine without xen.
> >
> > following are the DTB changes,
> >
> > chosen {
> > #address-cells = <1>;
> > #size-cells = <1>;
> > bootargs = "console=dtuart dtuart=serial0 dom0_mem=128M";
> > stdout-path = "serial0";
> > module: module@0 {
> > compatible = "xen,linux-zimage", "xen,multiboot-module";
> > reg = <0x40148158 0x140>;
> > bootargs = "console=hvc0,921600n8 earlyprintk=xen debug

It should be just 'console=hvc0', not 'console=hvc0,921600n8'

> > ignore_loglevel rw root=/dev/mmcblk0p7";
> > };
> >
> > };
> >
> > Can you tell me how to debug dom0 booting or anything which i can check?
> 
> Don't now much about "debug dom0 booting", I leave it for competent people.
> 
> Looks weird, even with earlyprintk no logs.
> Do you have DEBUG_LL and all related options enabled in your dom0 kernel 
> config?
> 
> 1. Check that following options are enabled in your kernel config file:
> 
> CONFIG_HVC_XEN=y
> CONFIG_HVC_XEN_FRONTEND=y
> 
> 2. Check that dom0 kernel doesn't disable clock for console.
> 
> BTW, could you post full Xen log, kernel config and device-tree you are using?
> If you have some changes on top of Xen, post them too.
> These may help people to identify what is wrong.
> 
> >
> >
> > Thanks,
> > Bharat
> >
> > On Wed, Sep 6, 2017 at 3:49 PM, Oleksandr Tyshchenko 
> > wrote:
> >>
> >> Hi Bharat
> >>
> >> On Wed, Sep 6, 2017 at 10:01 AM, bharat gohil  wrote:
> >> > Hello Oleksandr,
> >> >
> >> > Thank you very much.It resolved my issue.
> >> Sounds great!
> >>
> >> >
> >> > Thanks,
> >> > Bharat
> >> >
> >> > On Mon, Sep 4, 2017 at 6:24 PM, Oleksandr Tyshchenko
> >> > 
> >> > wrote:
> >> >>
> >> >> Hi Bharat
> >> >>
> >> >> On Mon, Sep 4, 2017 at 7:13 AM, bharat gohil 
> >> >> wrote:
> >> >> > Hello Oleksandr,
> >> >> >
> >> >> > I have corrected  GIC settings but no success.Following line
> >> >> > disappear
> >> >> > from
> >> >> > log.
> >> >> >>>XEN) GICv2: WARNING: The GICC size is too small: 0x1000 expected
> >> >> >>> 0x2000
> >> >> >
> >> >> > Is anything else which can I try.
> >> >> >
> >> >> > I don’t know much about xen internal for ARM architecture. As you
> >> >> > mentioned,
> >> >> >>>Wrong GIC settings might lead to that IPIs won't work as expected.
> >> >> >>> And
> >> >> >>>boot CPU will get stuck waiting for another CPU.
> >> >> >
> >> >> > Can you explain it with some boot sequence and relation with IPI?
> >> >>
> >> >> Well, we faced similar issue with R-Car Gen3 H3 SoC. Xen hung at
> >> >> smp_call_function (one CPU didn't receive interrupt from another one).
> >> >> Next patch helped us to fix this issue:
> >> >> https://patchwork.kernel.org/patch/9163065/
> >> >>
> >> >> I assume the SoC you are working with has "arm,gic-400" compatible GIC.
> >> >> Can you take a look at the patch, maybe it is your case too.
> >> >>
> >> >> >
> >> >> > Thanks,
> >> >> > Bharat
> >> >> >
> >> >> >
> >> >> > On Thu, Aug 31, 2017 at 5:28 PM, Oleksandr Tyshchenko
> >> >> > 
> >> >> > wrote:
> >> >> >>
> >> >> >> On Thu, Aug 31, 2017 at 2:13 PM, bharat gohil 
> >> >> >> wrote:
> >> >> >> > Hello Oleksandr,
> >> >> >> Hi Bharat
> >> >> >>
> >> >> >> >
> >> >> >> > I had removed A72 cluster and tried to boot only two A35 but I got
> >> >> >> > same
> >> >> >> > error.
> >> >> >> >
> >> >> >> > Is anything added or missing in A35 compare to A53?
> >> >> >> Unfortunately, I don't 

Re: [Xen-devel] [xen-unstable test] 113562: regressions - FAIL

[Roger Pau Monné]

On Mon, Sep 18, 2017 at 01:11:07PM +0200, Juergen Gross wrote:
> On 18/09/17 13:05, George Dunlap wrote:
> > On 09/18/2017 11:46 AM, Roger Pau Monné wrote:
> >> On Mon, Sep 18, 2017 at 11:15:03AM +0100, George Dunlap wrote:
> >>> On 09/18/2017 10:45 AM, Roger Pau Monné wrote:
>  On Mon, Sep 18, 2017 at 10:37:58AM +0100, Wei Liu wrote:
> > On Mon, Sep 18, 2017 at 08:36:03AM +, osstest service owner wrote:
> >> flight 113562 xen-unstable real [real]
> >> http://logs.test-lab.xenproject.org/osstest/logs/113562/
> >>
> >> Regressions :-(
> >>
> >> Tests which did not succeed and are blocking,
> >> including tests which could not be run:
> >>  test-amd64-amd64-xl-credit2  15 guest-saverestorefail REGR. 
> >> vs. 113387
> >
> > There appears to be a bug:
> >
> > http://logs.test-lab.xenproject.org/osstest/logs/113562/test-amd64-amd64-xl-credit2/serial-godello0.log
> >
> > Sep 18 01:14:28.803062 (XEN) Xen BUG at spinlock.c:47
> 
>  Seem to be caused because budget_lock is sometimes locked with irqsave
>  while others not.
> >>>
> >>> Just wondering where you're getting the budget lock from?  The call
> >>> stack in that link makes it look like it's the RCU clean-up triggering a
> >>> domain destroy.  (Haven't looked deeper into the specific line numbers.)
> >>
> >> Just skimmed over the commit and jumped into conclusions too fast. As
> >> you mention later the issue is calling xfree with interrupts disabled
> >> in csched2_free_domdata.
> >>
> >> I would rather prefer budget_lock to be always locked with the
> >> irqsave/restore variant to make what you mention above more obvious,
> >> but that's just a question of taste.
> > 
> > I *think* at some point in the past we had a discussion about this and
> > someone (perhaps Jan?) said if we always know the irqs are disabled we
> > shouldn't call the _irqsave() version, to save cpu cycles.
> > 
> > Personally I think the ASSERT()s are clear enough to people familiar
> > with the scheduling code.
> 
> Why don't we add _irqoff variants of the locks containing the ASSERTion
> that interrupts are really off? This would save the additional
> instructions of the irqsave/restore variants and make it very clear that
> no violation of the lock interface is happening.

+1

Roger.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 02/22] tools: libxendevicemodel: Provide xendevicemodel_shutdown

[Wei Liu]

On Fri, Sep 15, 2017 at 07:48:39PM +0100, Ian Jackson wrote:
> diff --git a/tools/libs/devicemodel/libxendevicemodel.map 
> b/tools/libs/devicemodel/libxendevicemodel.map
> index 130222c..fd57e79 100644
> --- a/tools/libs/devicemodel/libxendevicemodel.map
> +++ b/tools/libs/devicemodel/libxendevicemodel.map
> @@ -18,6 +18,7 @@ VERS_1.0 {
>   xendevicemodel_modified_memory;
>   xendevicemodel_set_mem_type;
>   xendevicemodel_inject_event;
> + xendevicemodel_shutdown;
>   xendevicemodel_restrict;
>   xendevicemodel_close;
>   local: *; /* Do not expose anything by default */

We need to have:

VERS_1.1 {
   global:
xendevicemodel_shutdown;
} VERS_1.0;

And also bump the minor number in Makefile.

The code looks fine to me.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 01/22] xen: Provide XEN_DMOP_remote_shutdown

[Wei Liu]

On Fri, Sep 15, 2017 at 07:48:38PM +0100, Ian Jackson wrote:
> diff --git a/xen/include/xlat.lst b/xen/include/xlat.lst
> index 0f17000..20055b4 100644
> --- a/xen/include/xlat.lst
> +++ b/xen/include/xlat.lst
> @@ -70,6 +70,7 @@
>  ?dm_op_set_pci_intx_levelhvm/dm_op.h
>  ?dm_op_set_pci_link_routehvm/dm_op.h
>  ?dm_op_track_dirty_vram  hvm/dm_op.h
> +?dm_op_remote_shutdown   hvm/dm_op.h

Can you please move this a few lines above to make the dm_op section
sorted alphabetically?

With that and Jan's comments addressed:

Reviewed-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 4/4] x86/hvm: Implement hvmemul_write() using real mappings

[Alexandru Stefan ISAILA]

On Lu, 2017-09-18 at 07:43 -0600, Jan Beulich wrote:
> >
> > >
> > > >
> > > > On 08.09.17 at 18:05,  wrote:
> > Changes since V1:
> > - Moved ASSERT to the begining of the loop
> > - Corrected the decrement on mfn int the while statement
> > - Modified the comment to PAGE_SIZE+1
> While several of my v1 comments were taken care of verbally, some
> haven't been addressed here or during the discussion.
Sorry about that, I must have lost some or some emails have not been
indexed. I'll address all from now on.
>
> >
> > While the maximum size of linear mapping is capped at 1 page, the
> > logic
> > in the helpers is written to work properly as hvmemul_ctxt->mfn[]
> > gets
> > longer,
> > specifically with XSAVE instruction emulation in mind.
> >
> > This has only had light testing so far.
> Has this changed in the meantime?
This has not changed so far.
>
> >
> > +static void *hvmemul_map_linear_addr(
> > +unsigned long linear, unsigned int bytes, uint32_t pfec,
> > +struct hvm_emulate_ctxt *hvmemul_ctxt)
> > +{
> > +struct vcpu *curr = current;
> > +void *err, *mapping;
> > +
> > +/* First and final gfns which need mapping. */
> > +unsigned long frame = linear >> PAGE_SHIFT, first = frame;
> > +unsigned long final = (linear + bytes - !!bytes) >>
> > PAGE_SHIFT;
> > +
> > +/*
> > + * mfn points to the next free slot.  All used slots have a
> > page reference
> > + * held on them.
> > + */
> > +mfn_t *mfn = _ctxt->mfn[0];
> > +
> > +/*
> > + * The caller has no legitimate reason for trying a zero-byte
> > write, but
> > + * final is calculate to fail safe in release builds.
> > + *
> > + * The maximum write size depends on the number of adjacent
> > mfns[] which
> > + * can be vmap()'d, accouting for possible misalignment within
> > the region.
> > + * The higher level emulation callers are responsible for
> > ensuring that
> > + * mfns[] is large enough for the requested write size.
> > + */
> > +if ( bytes == 0 ||
> > + final - first > ARRAY_SIZE(hvmemul_ctxt->mfn) - 1 )
> > +{
> > +ASSERT_UNREACHABLE();
> > +goto unhandleable;
> > +}
> > +
> > +do {
> > +enum hvm_translation_result res;
> > +struct page_info *page;
> > +pagefault_info_t pfinfo;
> > +p2m_type_t p2mt;
> > +
> > +/* Error checking.  Confirm that the current slot is
> > clean. */
> > +ASSERT(mfn_x(*mfn) == 0);
> > +
> > +res = hvm_translate_get_page(curr, frame << PAGE_SHIFT,
> > true, pfec,
> > + , , NULL, );
> > +
> > +switch ( res )
> > +{
> > +case HVMTRANS_okay:
> > +break;
> > +
> > +case HVMTRANS_bad_linear_to_gfn:
> > +x86_emul_pagefault(pfinfo.ec, pfinfo.linear,
> > _ctxt->ctxt);
> > +err = ERR_PTR(~(long)X86EMUL_EXCEPTION);
> Why the casts to long here and further down?
>
> >
> > +goto out;
> > +
> > +case HVMTRANS_bad_gfn_to_mfn:
> > +err = NULL;
> > +goto out;
> > +
> > +case HVMTRANS_gfn_paged_out:
> > +case HVMTRANS_gfn_shared:
> > +err = ERR_PTR(~(long)X86EMUL_RETRY);
> > +goto out;
> > +
> > +default:
> > +goto unhandleable;
> > +}
> > +
> > +*mfn++ = _mfn(page_to_mfn(page));
> > +frame++;
> > +
> > +if ( p2m_is_discard_write(p2mt) )
> > +{
> > +err = ERR_PTR(~(long)X86EMUL_OKAY);
> > +goto out;
> > +}
> > +
> > +} while ( frame < final );
> > +
> > +/* Entire access within a single frame? */
> > +if ( first == final )
> > +mapping = map_domain_page(hvmemul_ctxt->mfn[0]) + (linear
> > & ~PAGE_MASK);
> > +/* Multiple frames? Need to vmap(). */
> > +else if ( (mapping = vmap(hvmemul_ctxt->mfn,
> > +  mfn - hvmemul_ctxt->mfn)) == NULL )
> v1 comment was "final - first + 1 would likely yield better code."
will do.
>
> >
> > +goto unhandleable;
> > +
> > +#ifndef NDEBUG /* Poision unused mfn[]s with INVALID_MFN. */
> > +while ( mfn < hvmemul_ctxt->mfn + ARRAY_SIZE(hvmemul_ctxt-
> > >mfn) )
> > +{
> > +ASSERT(mfn_x(*mfn) == 0);
> > +*mfn++ = INVALID_MFN;
> > +}
> > +#endif
> > +
> > +return mapping;
> > +
> > + unhandleable:
> > +err = ERR_PTR(~(long)X86EMUL_UNHANDLEABLE);
> > +
> > + out:
> > +/* Drop all held references. */
> > +while ( mfn-- > hvmemul_ctxt->mfn )
> > +put_page(mfn_to_page(mfn_x(*mfn)));
> > +
> > +return err;
> > +}
> > +
> > +static void hvmemul_unmap_linear_addr(
> > +void *mapping, unsigned long linear, unsigned int bytes,
> While this was discussed in response to v1, I still think "mapping"
> should be const void *, or a prereq patch (which I would object
> to) should be submitted to drop the const from vunmap() 

Re: [Xen-devel] [PATCH 01/22] xen: Provide XEN_DMOP_remote_shutdown

[Jan Beulich]

>>> On 18.09.17 at 15:57,  wrote:
> Jan Beulich writes ("Re: [PATCH 01/22] xen: Provide 
> XEN_DMOP_remote_shutdown"):
>> >>> On 15.09.17 at 20:48,  wrote:
>> > SCHEDOP_remote_shutdown should be a DMOP so that a deprivileged qemu
>> > can do the propery tidying up.
>> > 
>> > We should remove SCHEDOP_remote_shutdown at some point.
>> 
>> Except we can't for ABI stability reasons, plus how would you
>> remote-shutdown a PV guest then?
> 
> Thanks for the review.  I have replaced that sentence in the commit
> message with this:
> 
>  We need to keep SCHEDOP_remote_shutdown for ABI stability reasons and
>  because it is needed for PV guests.

Sounds good.

>> > --- a/xen/arch/x86/hvm/dm.c
>> > +++ b/xen/arch/x86/hvm/dm.c
>> > @@ -630,6 +630,14 @@ static int dm_op(const struct dmop_args *op_args)
>> >  rc = hvm_inject_msi(d, data->addr, data->data);
>> >  break;
>> >  }
>> > +case XEN_DMOP_remote_shutdown:
>> 
>> With a blank line added above here,
> 
> Thanks.  I copied the lack of newline from between
> XEN_DMOP_inject_event and XEN_DMOP_inject_msi.

Oh, I see - the only bad example in this function.

> I will add a trivial extra patch to add that missing newline (unless
> you object).

Feel free to put my ack on it, or even merge it into the patch here.

>> Reviewed-by: Jan Beulich 
> 
> Thanks.  In the expectation that what I say above in the commit
> message meets with your approval, I will include that R-B in my next
> posting of the series.

Yes please.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] mm: Scrub pages returned back to heap if MEMF_no_scrub is set

[Wei Liu]

On Fri, Sep 15, 2017 at 10:05:22AM -0600, Jan Beulich wrote:
> >>> On 15.09.17 at 16:04,  wrote:
> > Set free_heap_pages()'s need_scrub to true if alloc_domheap_pages()
> > returns pages back to heap as result of assign_pages() error when those
> > pages were requested with MEMF_no_scrub flag.
> > 
> > We need to do this because there is a possibility that
> > alloc_heap_pages() might clear buddy's PGC_need_scrubs flag without
> > actually clearing the page.
> > 
> > Signed-off-by: Boris Ostrovsky 
> > ---
> > We are declaring a likely clean (or almost clean) chunk to be dirty. Since
> > this only happend on assign_pages() error I figured it would be acceptable.
> 
> I think that's fine, but let's wait a little to see whether others
> have differing opinions.
> 

I think that's fine, too.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 01/22] xen: Provide XEN_DMOP_remote_shutdown

[Ian Jackson]

Jan Beulich writes ("Re: [PATCH 01/22] xen: Provide XEN_DMOP_remote_shutdown"):
> >>> On 15.09.17 at 20:48,  wrote:
> > SCHEDOP_remote_shutdown should be a DMOP so that a deprivileged qemu
> > can do the propery tidying up.
> > 
> > We should remove SCHEDOP_remote_shutdown at some point.
> 
> Except we can't for ABI stability reasons, plus how would you
> remote-shutdown a PV guest then?

Thanks for the review.  I have replaced that sentence in the commit
message with this:

 We need to keep SCHEDOP_remote_shutdown for ABI stability reasons and
 because it is needed for PV guests.

> > --- a/xen/arch/x86/hvm/dm.c
> > +++ b/xen/arch/x86/hvm/dm.c
> > @@ -630,6 +630,14 @@ static int dm_op(const struct dmop_args *op_args)
> >  rc = hvm_inject_msi(d, data->addr, data->data);
> >  break;
> >  }
> > +case XEN_DMOP_remote_shutdown:
> 
> With a blank line added above here,

Thanks.  I copied the lack of newline from between
XEN_DMOP_inject_event and XEN_DMOP_inject_msi.

I will add a trivial extra patch to add that missing newline (unless
you object).

> Reviewed-by: Jan Beulich 

Thanks.  In the expectation that what I say above in the commit
message meets with your approval, I will include that R-B in my next
posting of the series.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 0/3] Some coverity fixes for xl/libxl

[Wei Liu]

v2: use _mandatory variant

Wei Liu (3):
  libxl: use libxl__read_xenstore_mandatory in vtpm function
  libxl: use libxl__read_xenstore_mandatory in vdispl function
  xl: avoid leaking memory in vdispl parser

 tools/libxl/libxl_vdispl.c | 8 ++--
 tools/libxl/libxl_vtpm.c   | 7 +--
 tools/xl/xl_parse.c| 2 ++
 3 files changed, 13 insertions(+), 4 deletions(-)

-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 3/3] xl: avoid leaking memory in vdispl parser

[Wei Liu]

Coverity-ID: 1418095

Signed-off-by: Wei Liu 
---
 tools/xl/xl_parse.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 9965b83c44..0678fbc1b0 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -832,6 +832,8 @@ int parse_vdispl_config(libxl_device_vdispl *vdispl, char 
*token)
 
 rc= sscanf(resolution, "%ux%u", >connectors[i].width,
>connectors[i].height);
+free(resolution);
+
 if (rc != 2) {
 fprintf(stderr, "Can't parse connector resolution\n");
 goto out;
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 1/3] libxl: use libxl__read_xenstore_mandatory in vtpm function

[Wei Liu]

libxl__read_xenstore can return NULL. Use the _mandatory variant to
return early when the read fails.

Coverity-ID: 1418098

Signed-off-by: Wei Liu 
---
 tools/libxl/libxl_vtpm.c | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/tools/libxl/libxl_vtpm.c b/tools/libxl/libxl_vtpm.c
index 21320870d4..6182cfc49c 100644
--- a/tools/libxl/libxl_vtpm.c
+++ b/tools/libxl/libxl_vtpm.c
@@ -79,12 +79,15 @@ static int libxl__vtpm_from_xenstore(libxl__gc *gc, const 
char *libxl_path,
  libxl_device_vtpm *vtpm)
 {
 int rc;
-char *be_path;
+const char *be_path;
 char *uuid;
 
 vtpm->devid = devid;
 
-be_path = libxl__xs_read(gc, XBT_NULL, GCSPRINTF("%s/backend", 
libxl_path));
+rc = libxl__xs_read_mandatory(gc, XBT_NULL,
+  GCSPRINTF("%s/backend", libxl_path),
+  _path);
+if (rc) return rc;
 
 rc = libxl__backendpath_parse_domid(gc, be_path, >backend_domid);
 if (rc) return rc;
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 2/3] libxl: use libxl__read_xenstore_mandatory in vdispl function

[Wei Liu]

Coverity-ID: 1418097

Signed-off-by: Wei Liu 
---
 tools/libxl/libxl_vdispl.c | 8 ++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/tools/libxl/libxl_vdispl.c b/tools/libxl/libxl_vdispl.c
index 5740c89fad..befc56bdeb 100644
--- a/tools/libxl/libxl_vdispl.c
+++ b/tools/libxl/libxl_vdispl.c
@@ -40,10 +40,14 @@ static int libxl__vdispl_from_xenstore(libxl__gc *gc, const 
char *libxl_path,
libxl_devid devid,
libxl_device_vdispl *vdispl)
 {
-char *be_path;
+const char *be_path;
+int rc;
 
 vdispl->devid = devid;
-be_path = libxl__xs_read(gc, XBT_NULL, GCSPRINTF("%s/backend", 
libxl_path));
+rc = libxl__xs_read_mandatory(gc, XBT_NULL,
+  GCSPRINTF("%s/backend", libxl_path),
+  _path);
+if (rc) return rc;
 
 return libxl__backendpath_parse_domid(gc, be_path, >backend_domid);
 }
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 2/2] Introduce migration precopy policy

[Wei Liu]

On Thu, Sep 14, 2017 at 04:33:58PM +0100, Jennifer Herbert wrote:
> This Patch allows a migration precopy policy to be specified.
> 
> The precopy phase of the xc_domain_save() live migration algorithm has
> historically been implemented to run until either a) (almost) no pages
> are dirty or b) some fixed, hard-coded maximum number of precopy
> iterations has been exceeded.  This policy and its implementation are
> less than ideal for a few reasons:
> - the logic of the policy is intertwined with the control flow of the
>   mechanism of the precopy stage
> - it can't take into account facts external to the immediate
>   migration context, such external state transfer state, interactive
>   user input, or the passage of wall-clock time.
> - it does not permit the user to change their mind, over time, about
>   what to do at the end of the precopy (they get an unconditional
>   transition into the stop-and-copy phase of the migration)
> 
> To permit callers to implement arbitrary higher-level policies governing
> when the live migration precopy phase should end, and what should be
> done next:
> - add a precopy_policy() callback to the xc_domain_save() user-supplied
>   callbacks
> - during the precopy phase of live migrations, consult this policy after
>   each batch of pages transmitted and take the dictated action, which
>   may be to a) abort the migration entirely, b) continue with the
>   precopy, or c) proceed to the stop-and-copy phase.
> - provide an implementation of the old policy, used when
>   precopy_policy callback  is not provided.
> 
> Signed-off-by: Jennifer Herbert 
> 
> ---
> 
> This is updated/modified subset of patch 7/20, part of
> Joshua Otto's "Add postcopy live migration support." patch,
> dated 27th March 2017.  As indicated on the original thread,
> I wish to make use of this this within the XenServer product.
> I hope this will aid Josh in pushing the remainder of his series.
> ---
>  tools/libxc/include/xenguest.h |  19 
>  tools/libxc/xc_sr_common.h |   7 ++-
>  tools/libxc/xc_sr_save.c   | 102 
> +
>  3 files changed, 94 insertions(+), 34 deletions(-)
> 
> diff --git a/tools/libxc/include/xenguest.h b/tools/libxc/include/xenguest.h
> index 6626f0c..d5908dc 100644
> --- a/tools/libxc/include/xenguest.h
> +++ b/tools/libxc/include/xenguest.h
> @@ -39,6 +39,14 @@
>   */
>  struct xenevtchn_handle;
>  
> +/* For save's precopy_policy(). */
> +struct precopy_stats
> +{
> +unsigned iteration;
> +unsigned total_written;
> +long dirty_count; /* -1 if unknown */
> +};
> +
>  /* callbacks provided by xc_domain_save */
>  struct save_callbacks {
>  /* Called after expiration of checkpoint interval,
> @@ -46,6 +54,17 @@ struct save_callbacks {
>   */
>  int (*suspend)(void* data);
>  
> +/* Called after every batch of page data sent during the precopy phase 
> of a
> + * live migration to ask the caller what to do next based on the current
> + * state of the precopy migration.
> + */
> +#define XGS_POLICY_ABORT  (-1) /* Abandon the migration entirely and
> +* tidy up. */
> +#define XGS_POLICY_CONTINUE_PRECOPY 0  /* Remain in the precopy phase. */
> +#define XGS_POLICY_STOP_AND_COPY1  /* Immediately suspend and transmit 
> the
> +* remaining dirty pages. */
> +int (*precopy_policy)(struct precopy_stats stats, void *data);
> +
>  /* Called after the guest's dirty pages have been
>   *  copied into an output buffer.
>   * Callback function resumes the guest & the device model,
> diff --git a/tools/libxc/xc_sr_common.h b/tools/libxc/xc_sr_common.h
> index a83f22a..2bc261b 100644
> --- a/tools/libxc/xc_sr_common.h
> +++ b/tools/libxc/xc_sr_common.h
> @@ -198,12 +198,11 @@ struct xc_sr_context
>  /* Further debugging information in the stream. */
>  bool debug;
>  
> -/* Parameters for tweaking live migration. */
> -unsigned max_iterations;
> -unsigned dirty_threshold;
> -
>  unsigned long p2m_size;
>  
> +struct precopy_stats stats;
> +int policy_decision;
> +
>  xen_pfn_t *batch_pfns;
>  unsigned nr_batch_pfns;
>  unsigned long *deferred_pages;
> diff --git a/tools/libxc/xc_sr_save.c b/tools/libxc/xc_sr_save.c
> index 1e7502d..03dfa61 100644
> --- a/tools/libxc/xc_sr_save.c
> +++ b/tools/libxc/xc_sr_save.c
> @@ -452,8 +452,7 @@ static int update_progress_string(struct xc_sr_context 
> *ctx,
>  xc_interface *xch = ctx->xch;
>  char *new_str = NULL;
>  
> -if ( asprintf(_str, "Frames iteration %u of %u",
> -  iter, ctx->save.max_iterations) == -1 )
> +if ( asprintf(_str, "Frames iteration %u", iter) == -1 )
>  {
>  PERROR("Unable to allocate new progress string");
>  return -1;
> @@ 

Re: [Xen-devel] [xen-unstable test] 113562: regressions - FAIL

[George Dunlap]

On 09/18/2017 12:11 PM, Juergen Gross wrote:
> On 18/09/17 13:05, George Dunlap wrote:
>> On 09/18/2017 11:46 AM, Roger Pau Monné wrote:
>>> On Mon, Sep 18, 2017 at 11:15:03AM +0100, George Dunlap wrote:
 On 09/18/2017 10:45 AM, Roger Pau Monné wrote:
> On Mon, Sep 18, 2017 at 10:37:58AM +0100, Wei Liu wrote:
>> On Mon, Sep 18, 2017 at 08:36:03AM +, osstest service owner wrote:
>>> flight 113562 xen-unstable real [real]
>>> http://logs.test-lab.xenproject.org/osstest/logs/113562/
>>>
>>> Regressions :-(
>>>
>>> Tests which did not succeed and are blocking,
>>> including tests which could not be run:
>>>  test-amd64-amd64-xl-credit2  15 guest-saverestorefail REGR. 
>>> vs. 113387
>>
>> There appears to be a bug:
>>
>> http://logs.test-lab.xenproject.org/osstest/logs/113562/test-amd64-amd64-xl-credit2/serial-godello0.log
>>
>> Sep 18 01:14:28.803062 (XEN) Xen BUG at spinlock.c:47
>
> Seem to be caused because budget_lock is sometimes locked with irqsave
> while others not.

 Just wondering where you're getting the budget lock from?  The call
 stack in that link makes it look like it's the RCU clean-up triggering a
 domain destroy.  (Haven't looked deeper into the specific line numbers.)
>>>
>>> Just skimmed over the commit and jumped into conclusions too fast. As
>>> you mention later the issue is calling xfree with interrupts disabled
>>> in csched2_free_domdata.
>>>
>>> I would rather prefer budget_lock to be always locked with the
>>> irqsave/restore variant to make what you mention above more obvious,
>>> but that's just a question of taste.
>>
>> I *think* at some point in the past we had a discussion about this and
>> someone (perhaps Jan?) said if we always know the irqs are disabled we
>> shouldn't call the _irqsave() version, to save cpu cycles.
>>
>> Personally I think the ASSERT()s are clear enough to people familiar
>> with the scheduling code.
> 
> Why don't we add _irqoff variants of the locks containing the ASSERTion
> that interrupts are really off? This would save the additional
> instructions of the irqsave/restore variants and make it very clear that
> no violation of the lock interface is happening.

I'd be OK with such a patch -- but obviously at this point it would have
to wait for 4.11. :-)

 -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 4/4] x86/hvm: Implement hvmemul_write() using real mappings

[Jan Beulich]

>>> On 08.09.17 at 18:05,  wrote:
> Changes since V1:
>   - Moved ASSERT to the begining of the loop
>   - Corrected the decrement on mfn int the while statement
>   - Modified the comment to PAGE_SIZE+1

While several of my v1 comments were taken care of verbally, some
haven't been addressed here or during the discussion.

> While the maximum size of linear mapping is capped at 1 page, the logic
> in the helpers is written to work properly as hvmemul_ctxt->mfn[] gets 
> longer,
> specifically with XSAVE instruction emulation in mind.
> 
> This has only had light testing so far.

Has this changed in the meantime?

> +static void *hvmemul_map_linear_addr(
> +unsigned long linear, unsigned int bytes, uint32_t pfec,
> +struct hvm_emulate_ctxt *hvmemul_ctxt)
> +{
> +struct vcpu *curr = current;
> +void *err, *mapping;
> +
> +/* First and final gfns which need mapping. */
> +unsigned long frame = linear >> PAGE_SHIFT, first = frame;
> +unsigned long final = (linear + bytes - !!bytes) >> PAGE_SHIFT;
> +
> +/*
> + * mfn points to the next free slot.  All used slots have a page 
> reference
> + * held on them.
> + */
> +mfn_t *mfn = _ctxt->mfn[0];
> +
> +/*
> + * The caller has no legitimate reason for trying a zero-byte write, but
> + * final is calculate to fail safe in release builds.
> + *
> + * The maximum write size depends on the number of adjacent mfns[] which
> + * can be vmap()'d, accouting for possible misalignment within the 
> region.
> + * The higher level emulation callers are responsible for ensuring that
> + * mfns[] is large enough for the requested write size.
> + */
> +if ( bytes == 0 ||
> + final - first > ARRAY_SIZE(hvmemul_ctxt->mfn) - 1 )
> +{
> +ASSERT_UNREACHABLE();
> +goto unhandleable;
> +}
> +
> +do {
> +enum hvm_translation_result res;
> +struct page_info *page;
> +pagefault_info_t pfinfo;
> +p2m_type_t p2mt;
> +
> +/* Error checking.  Confirm that the current slot is clean. */
> +ASSERT(mfn_x(*mfn) == 0);
> +
> +res = hvm_translate_get_page(curr, frame << PAGE_SHIFT, true, pfec,
> + , , NULL, );
> +
> +switch ( res )
> +{
> +case HVMTRANS_okay:
> +break;
> +
> +case HVMTRANS_bad_linear_to_gfn:
> +x86_emul_pagefault(pfinfo.ec, pfinfo.linear, 
> _ctxt->ctxt);
> +err = ERR_PTR(~(long)X86EMUL_EXCEPTION);

Why the casts to long here and further down?

> +goto out;
> +
> +case HVMTRANS_bad_gfn_to_mfn:
> +err = NULL;
> +goto out;
> +
> +case HVMTRANS_gfn_paged_out:
> +case HVMTRANS_gfn_shared:
> +err = ERR_PTR(~(long)X86EMUL_RETRY);
> +goto out;
> +
> +default:
> +goto unhandleable;
> +}
> +
> +*mfn++ = _mfn(page_to_mfn(page));
> +frame++;
> +
> +if ( p2m_is_discard_write(p2mt) )
> +{
> +err = ERR_PTR(~(long)X86EMUL_OKAY);
> +goto out;
> +}
> +
> +} while ( frame < final );
> +
> +/* Entire access within a single frame? */
> +if ( first == final )
> +mapping = map_domain_page(hvmemul_ctxt->mfn[0]) + (linear & 
> ~PAGE_MASK);
> +/* Multiple frames? Need to vmap(). */
> +else if ( (mapping = vmap(hvmemul_ctxt->mfn,
> +  mfn - hvmemul_ctxt->mfn)) == NULL )

v1 comment was "final - first + 1 would likely yield better code."

> +goto unhandleable;
> +
> +#ifndef NDEBUG /* Poision unused mfn[]s with INVALID_MFN. */
> +while ( mfn < hvmemul_ctxt->mfn + ARRAY_SIZE(hvmemul_ctxt->mfn) )
> +{
> +ASSERT(mfn_x(*mfn) == 0);
> +*mfn++ = INVALID_MFN;
> +}
> +#endif
> +
> +return mapping;
> +
> + unhandleable:
> +err = ERR_PTR(~(long)X86EMUL_UNHANDLEABLE);
> +
> + out:
> +/* Drop all held references. */
> +while ( mfn-- > hvmemul_ctxt->mfn )
> +put_page(mfn_to_page(mfn_x(*mfn)));
> +
> +return err;
> +}
> +
> +static void hvmemul_unmap_linear_addr(
> +void *mapping, unsigned long linear, unsigned int bytes,

While this was discussed in response to v1, I still think "mapping"
should be const void *, or a prereq patch (which I would object
to) should be submitted to drop the const from vunmap() and
unmap_domain_page().

> @@ -1007,23 +1160,15 @@ static int hvmemul_write(
>   (vio->mmio_gla == (addr & PAGE_MASK)) )
>  return hvmemul_linear_mmio_write(addr, bytes, p_data, pfec, 
> hvmemul_ctxt, 1);
>  
> -rc = hvm_copy_to_guest_linear(addr, p_data, bytes, pfec, );
> -
> -switch ( rc )
> -{
> -case HVMTRANS_okay:
> -break;
> -case HVMTRANS_bad_linear_to_gfn:
> -x86_emul_pagefault(pfinfo.ec, pfinfo.linear, _ctxt->ctxt);
> -return 

Re: [Xen-devel] [PATCH 2/2] Introduce migration precopy policy

[Ian Jackson]

Jennifer Herbert writes ("[PATCH 2/2] Introduce migration precopy policy"):
> This Patch allows a migration precopy policy to be specified.

But only for direct libxc users.  How do you think this should be
exposed via libxl ?

The general approach, so far, seems sound.

Thanks,
Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 2/2] Introduce migration precopy policy

[Paul Durrant]

> -Original Message-
> From: Wei Liu [mailto:wei.l...@citrix.com]
> Sent: 18 September 2017 14:20
> To: Paul Durrant 
> Cc: Jennifer Herbert ; Ian Jackson
> ; Wei Liu ; xen-
> de...@lists.xenproject.org; jto...@uwaterloo.ca
> Subject: Re: [Xen-devel] [PATCH 2/2] Introduce migration precopy policy
> 
> On Mon, Sep 18, 2017 at 10:05:07AM +0100, Paul Durrant wrote:
> > > -Original Message-
> > > From: Xen-devel [mailto:xen-devel-boun...@lists.xen.org] On Behalf Of
> > > Jennifer Herbert
> > > Sent: 14 September 2017 16:34
> > > To: Ian Jackson ; Wei Liu ;
> > > xen-de...@lists.xenproject.org; jto...@uwaterloo.ca
> > > Cc: Jennifer Herbert 
> > > Subject: [Xen-devel] [PATCH 2/2] Introduce migration precopy policy
> > >
> > > This Patch allows a migration precopy policy to be specified.
> > >
> > > The precopy phase of the xc_domain_save() live migration algorithm has
> > > historically been implemented to run until either a) (almost) no pages
> > > are dirty or b) some fixed, hard-coded maximum number of precopy
> > > iterations has been exceeded.  This policy and its implementation are
> > > less than ideal for a few reasons:
> > > - the logic of the policy is intertwined with the control flow of the
> > >   mechanism of the precopy stage
> > > - it can't take into account facts external to the immediate
> > >   migration context, such external state transfer state, interactive
> > >   user input, or the passage of wall-clock time.
> > > - it does not permit the user to change their mind, over time, about
> > >   what to do at the end of the precopy (they get an unconditional
> > >   transition into the stop-and-copy phase of the migration)
> > >
> > > To permit callers to implement arbitrary higher-level policies governing
> > > when the live migration precopy phase should end, and what should be
> > > done next:
> > > - add a precopy_policy() callback to the xc_domain_save() user-supplied
> > >   callbacks
> > > - during the precopy phase of live migrations, consult this policy after
> > >   each batch of pages transmitted and take the dictated action, which
> > >   may be to a) abort the migration entirely, b) continue with the
> > >   precopy, or c) proceed to the stop-and-copy phase.
> > > - provide an implementation of the old policy, used when
> > >   precopy_policy callback  is not provided.
> > >
> > > Signed-off-by: Jennifer Herbert 
> > >
> > > ---
> > >
> > > This is updated/modified subset of patch 7/20, part of
> > > Joshua Otto's "Add postcopy live migration support." patch,
> > > dated 27th March 2017.  As indicated on the original thread,
> > > I wish to make use of this this within the XenServer product.
> > > I hope this will aid Josh in pushing the remainder of his series.
> >
> > Does this patch need to carry Joshua's s-o-b, or at least 'suggested-by'?
> 
> I agree. We need to retain Joshua's s-o-b because this patch is based on
> his.
> 
> >
> > > ---
> > >  tools/libxc/include/xenguest.h |  19 
> > >  tools/libxc/xc_sr_common.h |   7 ++-
> > >  tools/libxc/xc_sr_save.c   | 102 +--
> 
> > > --
> > >  3 files changed, 94 insertions(+), 34 deletions(-)
> > >
> > > diff --git a/tools/libxc/include/xenguest.h
> b/tools/libxc/include/xenguest.h
> > > index 6626f0c..d5908dc 100644
> > > --- a/tools/libxc/include/xenguest.h
> > > +++ b/tools/libxc/include/xenguest.h
> > > @@ -39,6 +39,14 @@
> > >   */
> > >  struct xenevtchn_handle;
> > >
> > > +/* For save's precopy_policy(). */
> > > +struct precopy_stats
> > > +{
> > > +unsigned iteration;
> > > +unsigned total_written;
> > > +long dirty_count; /* -1 if unknown */
> > > +};
> > > +
> > >  /* callbacks provided by xc_domain_save */
> > >  struct save_callbacks {
> > >  /* Called after expiration of checkpoint interval,
> > > @@ -46,6 +54,17 @@ struct save_callbacks {
> > >   */
> > >  int (*suspend)(void* data);
> > >
> > > +/* Called after every batch of page data sent during the precopy
> phase of
> > > a
> > > + * live migration to ask the caller what to do next based on the 
> > > current
> > > + * state of the precopy migration.
> > > + */
> > > +#define XGS_POLICY_ABORT  (-1) /* Abandon the migration entirely
> > > and
> > > +* tidy up. */
> > > +#define XGS_POLICY_CONTINUE_PRECOPY 0  /* Remain in the precopy
> > > phase. */
> > > +#define XGS_POLICY_STOP_AND_COPY1  /* Immediately suspend
> and
> > > transmit the
> > > +* remaining dirty pages. */
> > > +int (*precopy_policy)(struct precopy_stats stats, void *data);
> >
> > Do we really want to be passing the struct, rather than a pointer to it?
> >
> 
> IIRC that was discussed in the 

Re: [Xen-devel] [PATCH 2/2] Introduce migration precopy policy

[Wei Liu]

On Mon, Sep 18, 2017 at 10:05:07AM +0100, Paul Durrant wrote:
> > -Original Message-
> > From: Xen-devel [mailto:xen-devel-boun...@lists.xen.org] On Behalf Of
> > Jennifer Herbert
> > Sent: 14 September 2017 16:34
> > To: Ian Jackson ; Wei Liu ;
> > xen-de...@lists.xenproject.org; jto...@uwaterloo.ca
> > Cc: Jennifer Herbert 
> > Subject: [Xen-devel] [PATCH 2/2] Introduce migration precopy policy
> > 
> > This Patch allows a migration precopy policy to be specified.
> > 
> > The precopy phase of the xc_domain_save() live migration algorithm has
> > historically been implemented to run until either a) (almost) no pages
> > are dirty or b) some fixed, hard-coded maximum number of precopy
> > iterations has been exceeded.  This policy and its implementation are
> > less than ideal for a few reasons:
> > - the logic of the policy is intertwined with the control flow of the
> >   mechanism of the precopy stage
> > - it can't take into account facts external to the immediate
> >   migration context, such external state transfer state, interactive
> >   user input, or the passage of wall-clock time.
> > - it does not permit the user to change their mind, over time, about
> >   what to do at the end of the precopy (they get an unconditional
> >   transition into the stop-and-copy phase of the migration)
> > 
> > To permit callers to implement arbitrary higher-level policies governing
> > when the live migration precopy phase should end, and what should be
> > done next:
> > - add a precopy_policy() callback to the xc_domain_save() user-supplied
> >   callbacks
> > - during the precopy phase of live migrations, consult this policy after
> >   each batch of pages transmitted and take the dictated action, which
> >   may be to a) abort the migration entirely, b) continue with the
> >   precopy, or c) proceed to the stop-and-copy phase.
> > - provide an implementation of the old policy, used when
> >   precopy_policy callback  is not provided.
> > 
> > Signed-off-by: Jennifer Herbert 
> > 
> > ---
> > 
> > This is updated/modified subset of patch 7/20, part of
> > Joshua Otto's "Add postcopy live migration support." patch,
> > dated 27th March 2017.  As indicated on the original thread,
> > I wish to make use of this this within the XenServer product.
> > I hope this will aid Josh in pushing the remainder of his series.
> 
> Does this patch need to carry Joshua's s-o-b, or at least 'suggested-by'?

I agree. We need to retain Joshua's s-o-b because this patch is based on
his.

> 
> > ---
> >  tools/libxc/include/xenguest.h |  19 
> >  tools/libxc/xc_sr_common.h |   7 ++-
> >  tools/libxc/xc_sr_save.c   | 102 +--
> > --
> >  3 files changed, 94 insertions(+), 34 deletions(-)
> > 
> > diff --git a/tools/libxc/include/xenguest.h b/tools/libxc/include/xenguest.h
> > index 6626f0c..d5908dc 100644
> > --- a/tools/libxc/include/xenguest.h
> > +++ b/tools/libxc/include/xenguest.h
> > @@ -39,6 +39,14 @@
> >   */
> >  struct xenevtchn_handle;
> > 
> > +/* For save's precopy_policy(). */
> > +struct precopy_stats
> > +{
> > +unsigned iteration;
> > +unsigned total_written;
> > +long dirty_count; /* -1 if unknown */
> > +};
> > +
> >  /* callbacks provided by xc_domain_save */
> >  struct save_callbacks {
> >  /* Called after expiration of checkpoint interval,
> > @@ -46,6 +54,17 @@ struct save_callbacks {
> >   */
> >  int (*suspend)(void* data);
> > 
> > +/* Called after every batch of page data sent during the precopy phase 
> > of
> > a
> > + * live migration to ask the caller what to do next based on the 
> > current
> > + * state of the precopy migration.
> > + */
> > +#define XGS_POLICY_ABORT  (-1) /* Abandon the migration entirely
> > and
> > +* tidy up. */
> > +#define XGS_POLICY_CONTINUE_PRECOPY 0  /* Remain in the precopy
> > phase. */
> > +#define XGS_POLICY_STOP_AND_COPY1  /* Immediately suspend and
> > transmit the
> > +* remaining dirty pages. */
> > +int (*precopy_policy)(struct precopy_stats stats, void *data);
> 
> Do we really want to be passing the struct, rather than a pointer to it?
> 

IIRC that was discussed in the past. We passed the struct because we
couldn't pass pointers across process boundary.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 3/4] x86/hvm: Break out __hvm_copy()'s translation logic

[Jan Beulich]

>>> On 08.09.17 at 18:05,  wrote:
> --- a/xen/arch/x86/hvm/hvm.c
> +++ b/xen/arch/x86/hvm/hvm.c
> @@ -3069,6 +3069,83 @@ void hvm_task_switch(
>  hvm_unmap_entry(nptss_desc);
>  }
>  
> +enum hvm_translation_result hvm_translate_get_page(
> +struct vcpu *v, unsigned long addr, bool linear, uint32_t pfec,
> +pagefault_info_t *pfinfo, struct page_info **page_p,
> +gfn_t *gfn_p, p2m_type_t *p2mt_p)
> +{
> +struct page_info *page;
> +p2m_type_t p2mt;
> +gfn_t gfn;
> +
> +if ( linear )
> +{
> +gfn = _gfn(paging_gva_to_gfn(v, addr, ));
> +
> +if ( gfn_eq(gfn, INVALID_GFN) )
> +{
> +if ( pfec & PFEC_page_paged )
> +return HVMTRANS_gfn_paged_out;
> +
> +if ( pfec & PFEC_page_shared )
> +return HVMTRANS_gfn_shared;
> +
> +if ( pfinfo )
> +{
> +pfinfo->linear = addr;
> +pfinfo->ec = pfec & ~PFEC_implicit;
> +}
> +
> +return HVMTRANS_bad_linear_to_gfn;
> +}
> +}
> +else
> +{
> +gfn = _gfn(addr >> PAGE_SHIFT);

This wants to become gaddr_to_gfn(), now that we have it, and ...

> +ASSERT(!pfinfo);
> +}
> +
> +/*
> + * No need to do the P2M lookup for internally handled MMIO, benefiting
> + * - 32-bit WinXP (& older Windows) on AMD CPUs for LAPIC accesses,
> + * - newer Windows (like Server 2012) for HPET accesses.
> + */
> +if ( v == current
> + && !nestedhvm_vcpu_in_guestmode(v)
> + && hvm_mmio_internal(gfn_x(gfn) << PAGE_SHIFT) )

... this one gfn_to_gaddr().

> +return HVMTRANS_bad_gfn_to_mfn;
> +
> +page = get_page_from_gfn(v->domain, gfn_x(gfn), , P2M_UNSHARE);
> +
> +if ( !page )
> +return HVMTRANS_bad_gfn_to_mfn;
> +
> +if ( p2m_is_paging(p2mt) )
> +{
> +put_page(page);
> +p2m_mem_paging_populate(v->domain, gfn_x(gfn));
> +return HVMTRANS_gfn_paged_out;
> +}
> +if ( p2m_is_shared(p2mt) )
> +{
> +put_page(page);
> +return HVMTRANS_gfn_shared;
> +}
> +if ( p2m_is_grant(p2mt) )
> +{
> +put_page(page);
> +return HVMTRANS_unhandleable;
> +}
> +
> +*page_p = page;
> +if ( gfn_p )
> +*gfn_p = gfn;
> +if ( p2mt_p )
> +*p2mt_p = p2mt;

Considering the use in patch 4, is it really useful for p2mt_p to be an
optional output? I.e. do you foresee (in the near future) a case where
p2m_is_discard_write() doesn't need handling by the caller?

> @@ -103,6 +104,17 @@ enum hvm_translation_result hvm_fetch_from_guest_linear(
>  void *buf, unsigned long addr, int size, uint32_t pfec,
>  pagefault_info_t *pfinfo);
>  
> +/*
> + * Get a reference on the page under an HVM physical or linear address.  If
> + * linear, a pagewalk is performed using pfec (fault details optionally in
> + * pfinfo).
> + * On success, returns HVMTRANS_okay with a reference taken on **_page.
> + */
> +enum hvm_translation_result hvm_translate_get_page(
> +struct vcpu *v, unsigned long addr, bool linear, uint32_t pfec,
> +pagefault_info_t *pfinfo, struct page_info **page_p,
> +gfn_t *gfn_p, p2m_type_t *p2mt_p);

The optional nature of whichever outputs we decide to be optional
should probably be called out in the comment, just like it is for pfinfo.

None of the points brought up would really require another version,
they could all easily be dealt with while committing. But of course we
first need to agree on at least this last one.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH V3 1/3] Xen: Increase hap/shadow page pool size to support more vcpus support

[Wei Liu]

On Wed, Sep 13, 2017 at 12:52:47AM -0400, Lan Tianyu wrote:
> This patch is to increase page pool size when max vcpu number is larger
> than 128.
> 
> Signed-off-by: Lan Tianyu 
> ---
>  xen/arch/arm/domain.c|  5 +
>  xen/arch/x86/domain.c| 25 +
>  xen/common/domctl.c  |  3 +++
>  xen/include/xen/domain.h |  2 ++
>  4 files changed, 35 insertions(+)
> 
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 6512f01..94cf70b 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -824,6 +824,11 @@ int arch_vcpu_reset(struct vcpu *v)
>  return 0;
>  }
>  
> +int arch_domain_set_max_vcpus(struct domain *d)
> +{
> +return 0;
> +}
> +
>  static int relinquish_memory(struct domain *d, struct page_list_head *list)
>  {
>  struct page_info *page, *tmp;
> diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
> index dbddc53..0e230f9 100644
> --- a/xen/arch/x86/domain.c
> +++ b/xen/arch/x86/domain.c
> @@ -1161,6 +1161,31 @@ int arch_vcpu_reset(struct vcpu *v)
>  return 0;
>  }
>  
> +int arch_domain_set_max_vcpus(struct domain *d)

The name doesn't match what the function does.

> +{
> +int ret;
> +
> +/* Increase page pool in order to support more vcpus. */
> +if ( d->max_vcpus > 128 )
> +{
> +unsigned long nr_pages;
> +
> +if (hap_enabled(d))

Coding style.

> +nr_pages = 1024;
> +else
> +nr_pages = 4096;
> +
> +ret = paging_set_allocation(d, nr_pages, NULL);

Does this work on PV guests?

> +if ( ret != 0 )
> +{
> +paging_set_allocation(d, 0, NULL);
> +return ret;
> +}
> +}
> +
> +return 0;
> +}
> +
>  long
>  arch_do_vcpu_op(
>  int cmd, struct vcpu *v, XEN_GUEST_HANDLE_PARAM(void) arg)
> diff --git a/xen/common/domctl.c b/xen/common/domctl.c
> index 42658e5..64357a3 100644
> --- a/xen/common/domctl.c
> +++ b/xen/common/domctl.c
> @@ -631,6 +631,9 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) 
> u_domctl)
>  d->max_vcpus = max;
>  }
>  
> +if ( arch_domain_set_max_vcpus(d) < 0)

!= 0 please.

> +goto maxvcpu_out;
> +
>  for ( i = 0; i < max; i++ )
>  {
>  if ( d->vcpu[i] != NULL )
> diff --git a/xen/include/xen/domain.h b/xen/include/xen/domain.h
> index 347f264..e1ece3a 100644
> --- a/xen/include/xen/domain.h
> +++ b/xen/include/xen/domain.h
> @@ -81,6 +81,8 @@ void arch_dump_domain_info(struct domain *d);
>  
>  int arch_vcpu_reset(struct vcpu *);
>  
> +int arch_domain_set_max_vcpus(struct domain *d);
> +
>  extern spinlock_t vcpu_alloc_lock;
>  bool_t domctl_lock_acquire(void);
>  void domctl_lock_release(void);
> -- 
> 1.8.3.1
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 2/4] x86/hvm: Rename enum hvm_copy_result to hvm_translation_result

[Jan Beulich]

>>> On 08.09.17 at 18:05,  wrote:
> From: Andrew Cooper 
> 
> Signed-off-by: Andrew Cooper 

Acked-by: Jan Beulich 



___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-unstable-smoke test] 113579: tolerable all pass - PUSHED

[osstest service owner]

flight 113579 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/113579/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  e25fc4cba8439c068a05f29531811cba71069837
baseline version:
 xen  abd91b2a2bcd05618a71f7e5fe571dd10a5727bc

Last test of basis   113493  2017-09-15 22:01:37 Z2 days
Testing same since   113579  2017-09-18 11:02:17 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  George Dunlap 
  Jan Beulich 
  Julien Grall 
  Kevin Tian 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=xen-unstable-smoke
+ revision=e25fc4cba8439c068a05f29531811cba71069837
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
 export PERLLIB=.:.
 PERLLIB=.:.
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push xen-unstable-smoke 
e25fc4cba8439c068a05f29531811cba71069837
+ branch=xen-unstable-smoke
+ revision=e25fc4cba8439c068a05f29531811cba71069837
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
 export PERLLIB=.:.:.
 PERLLIB=.:.:.
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
+++ export PERLLIB=.:.:.:.
+++ PERLLIB=.:.:.:.
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=xen
+ xenbranch=xen-unstable-smoke
+ qemuubranch=qemu-upstream-unstable
+ '[' xxen = xlinux ']'
+ linuxbranch=
+ '[' xqemu-upstream-unstable = x ']'
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable-smoke
+ prevxenbranch=xen-4.9-testing
+ '[' xe25fc4cba8439c068a05f29531811cba71069837 = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/xtf.git
++ : osst...@xenbits.xen.org:/home/xen/git/xtf.git
++ : git://xenbits.xen.org/xtf.git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : git
++ : git://xenbits.xen.org/osstest/rumprun.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/rumprun.git
++ : git://git.seabios.org/seabios.git
++ : osst...@xenbits.xen.org:/home/xen/git/osstest/seabios.git
++ : git://xenbits.xen.org/osstest/seabios.git
++ : 

[Xen-devel] [qemu-mainline test] 113569: regressions - trouble: broken/fail/pass

[osstest service owner]

flight 113569 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/113569/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-libvirt-xsm broken
 test-armhf-armhf-libvirt-raw broken
 test-amd64-amd64-xl-qemuu-win7-amd64 18 guest-start/win.repeat fail in 113560 
REGR. vs. 113302

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt-raw  4 host-install(4)  broken pass in 113560
 test-armhf-armhf-libvirt-xsm  4 host-install(4)  broken pass in 113560
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail in 113560 pass in 113569
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-localmigrate/x10 fail pass in 
113560
 test-armhf-armhf-xl-credit2  16 guest-start/debian.repeat  fail pass in 113560

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-check fail in 113560 like 
113302
 test-armhf-armhf-libvirt-raw 13 saverestore-support-check fail in 113560 like 
113302
 test-armhf-armhf-libvirt-xsm 13 migrate-support-check fail in 113560 never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-check fail in 113560 never pass
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeatfail  like 113302
 test-amd64-amd64-xl-rtds 10 debian-install   fail  like 113302
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 113302
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-installfail never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qcow2 12 migrate-support-checkfail  never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 13 guest-saverestore   fail never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass

version targeted for testing:
 qemuu4f2058ded4feb2fa815b33b57b305c81d5016307
baseline version:
 qemuua6e8c1dacfd37d34542e33600dcc50b7683b735a

Last test of basis   113302  2017-09-11 10:18:16 Z7 days
Failing since113345  2017-09-12 00:21:07 Z6 days   11 attempts
Testing same since   113560  2017-09-17 21:24:33 Z0 days2 attempts


People who touched revisions under test:
  Alexander Graf 
  Alexey Kardashevskiy 
  Alistair Francis 
  Amador Pahim 
  Benjamin Herrenschmidt 
  Cornelia Huck 
  Cédric Le Goater 
  Daniel Henrique Barboza 
  David Gibson 
  Dr. David Alan Gilbert 
  Eduardo Habkost 
  Eduardo Otubo 
  Eduardo Otubo 
  Eric Auger 

[Xen-devel] [ovmf baseline-only test] 72121: all pass

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 72121 ovmf real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/72121/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf 7f2f96f1a8af3c22bdf5d4dccb020846799f7be0
baseline version:
 ovmf 4084ccfa22dab15e2b9c3f531ba9ec18a6e08a8d

Last test of basis72120  2017-09-18 07:49:44 Z0 days
Testing same since72121  2017-09-18 09:49:09 Z0 days1 attempts


People who touched revisions under test:
  Fu Siyuan 
  Star Zeng 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xs.citrite.net/~osstest/testlogs/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


commit 7f2f96f1a8af3c22bdf5d4dccb020846799f7be0
Author: Star Zeng 
Date:   Thu Sep 14 19:24:35 2017 +0800

MdeModulePkg SmbiosMeasurementDxe: Skip measurement for OEM type

The generic driver has no way to know whether an OEM type should
be filtered or not.
This patch is to update the code to skip measurement for OEM type
and platform code can measure it by self if required.

Cc: Jiewen Yao 
Cc: Chasel Chiu 
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Star Zeng 
Reviewed-by: Jiewen Yao 
Reviewed-by: Chasel Chiu 

commit fbfe64203be8af6929c4e8c88500ea07689ea39e
Author: Fu Siyuan 
Date:   Thu Sep 14 11:13:05 2017 +0800

NetworkPkg: Remove the redundant '/' in the end of returned ISCSIMacAddr 
keyword.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Fu Siyuan 
Reviewed-by: Wu Jiaxin 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 1/1] public/io/netif.h: add gref mapping control messages

[Paul Durrant]

> -Original Message-
> From: Joao Martins [mailto:joao.m.mart...@oracle.com]
> Sent: 18 September 2017 12:56
> To: Paul Durrant 
> Cc: Xen-devel ; Wei Liu ;
> Konrad Rzeszutek Wilk 
> Subject: Re: [PATCH v3 1/1] public/io/netif.h: add gref mapping control
> messages
> 
> On Mon, Sep 18, 2017 at 09:53:18AM +, Paul Durrant wrote:
> > > -Original Message-
> > > From: Joao Martins [mailto:joao.m.mart...@oracle.com]
> > > Sent: 13 September 2017 19:11
> > > To: Xen-devel 
> > > Cc: Wei Liu ; Paul Durrant
> ;
> > > Konrad Rzeszutek Wilk ; Joao Martins
> > > 
> > > Subject: [PATCH v3 1/1] public/io/netif.h: add gref mapping control
> messages
> > >
> > > Adds 3 messages to allow guest to let backend keep grants mapped,
> > > such that 1) guests allowing fast recycling of pages can avoid doing
> > > grant ops for those cases, or otherwise 2) preferring copies over
> > > grants and 3) always using a fixed set of pages for network I/O.
> > >
> > > The three control ring messages added are:
> > >  - Add grefs to be mapped by backend
> > >  - Remove grefs mappings (If they are not in use)
> > >  - Get maximum amount of grefs kept mapped.
> > >
> > > Signed-off-by: Joao Martins 
> > > ---
> > > v3:
> > > * Use DEL for unmapping grefs instead of PUT
> > > * Rname from xen_netif_gref_alloc to xen_netif_gref
> > > * Add 'status' field on xen_netif_gref
> > > * Clarify what 'inflight' means
> > > * Use "beginning of the page" instead of "beginning of the grant"
> > > * Mention that page needs to be r/w (as it will have to modify \.status)
> > > * `data` on ADD|PUT returns number of entries mapped/unmapped.
> > > ---
> > >  xen/include/public/io/netif.h | 115
> > > ++
> > >  1 file changed, 115 insertions(+)
> > >
> > > diff --git a/xen/include/public/io/netif.h b/xen/include/public/io/netif.h
> > > index ca0061410d..0080a260fd 100644
> > > --- a/xen/include/public/io/netif.h
> > > +++ b/xen/include/public/io/netif.h
> > > @@ -353,6 +353,9 @@ struct xen_netif_ctrl_request {
> > >  #define XEN_NETIF_CTRL_TYPE_SET_HASH_MAPPING_SIZE 5
> > >  #define XEN_NETIF_CTRL_TYPE_SET_HASH_MAPPING  6
> > >  #define XEN_NETIF_CTRL_TYPE_SET_HASH_ALGORITHM7
> > > +#define XEN_NETIF_CTRL_TYPE_GET_GREF_MAPPING_SIZE 8
> > > +#define XEN_NETIF_CTRL_TYPE_ADD_GREF_MAPPING  9
> > > +#define XEN_NETIF_CTRL_TYPE_DEL_GREF_MAPPING 10
> > >
> > >  uint32_t data[3];
> > >  };
> > > @@ -391,6 +394,41 @@ struct xen_netif_ctrl_response {
> > >  };
> > >
> > >  /*
> > > + * Static Grants (struct xen_netif_gref)
> > > + * =
> > > + *
> > > + * A frontend may provide a fixed set of grant references to be mapped
> on
> > > + * the backend. The message of type
> > > XEN_NETIF_CTRL_TYPE_ADD_GREF_MAPPING
> > > + * prior its usage in the command ring allows for creation of these
> mappings.
> > > + * The backend will maintain a fixed amount of these mappings.
> > > + *
> > > + * XEN_NETIF_CTRL_TYPE_GET_GREF_MAPPING_SIZE lets a frontend
> > > query how many
> > > + * of these mappings can be kept.
> > > + *
> > > + * Each entry in the
> XEN_NETIF_CTRL_TYPE_{ADD,DEL}_GREF_MAPPING
> > > input table has
> > > + * the following format:
> > > + *
> > > + *0 1 2 3 4 5 6 7  octet
> > > + * +-+-+-+-+-+-+-+-+
> > > + * | grant ref |  flags|  status   |
> > > + * +-+-+-+-+-+-+-+-+
> > > + *
> > > + * grant ref: grant reference
> > > + * flags: flags describing the control operation
> > > + * status: XEN_NETIF_CTRL_STATUS_*
> > > + */
> >
> > You may want to add some words here pointing out that the status is an
> > 'out' field, and also whether it should be initialized to zero or not.
> >
> OK.
> 
> > > +
> > > +struct xen_netif_gref {
> > > +   grant_ref_t ref;
> > > +   uint16_t flags;
> > > +
> > > +#define _XEN_NETIF_CTRLF_GREF_readonly0
> > > +#define XEN_NETIF_CTRLF_GREF_readonly
> > > (1U<<_XEN_NETIF_CTRLF_GREF_readonly)
> > > +
> > > +   uint16_t status;
> > > +};
> > > +
> > > +/*
> > >   * Control messages
> > >   * 
> > >   *
> > > @@ -609,6 +647,83 @@ struct xen_netif_ctrl_response {
> > >   *   invalidate any table data outside that range.
> > >   *   The grant reference may be read-only and must remain valid until
> > >   *   the response has been processed.
> > > + *
> > > + * XEN_NETIF_CTRL_TYPE_GET_GREF_MAPPING_SIZE
> > > + * -
> > > + *
> > > + * This is sent by the frontend to fetch the number of grefs that can be
> kept
> > > + * mapped in the backend.
> > > + *
> > > + * Request:
> > > + *
> > > + *  type

Re: [Xen-devel] [PATCH v3 1/1] public/io/netif.h: add gref mapping control messages

[Joao Martins]

On Mon, Sep 18, 2017 at 09:53:18AM +, Paul Durrant wrote:
> > -Original Message-
> > From: Joao Martins [mailto:joao.m.mart...@oracle.com]
> > Sent: 13 September 2017 19:11
> > To: Xen-devel 
> > Cc: Wei Liu ; Paul Durrant ;
> > Konrad Rzeszutek Wilk ; Joao Martins
> > 
> > Subject: [PATCH v3 1/1] public/io/netif.h: add gref mapping control messages
> > 
> > Adds 3 messages to allow guest to let backend keep grants mapped,
> > such that 1) guests allowing fast recycling of pages can avoid doing
> > grant ops for those cases, or otherwise 2) preferring copies over
> > grants and 3) always using a fixed set of pages for network I/O.
> > 
> > The three control ring messages added are:
> >  - Add grefs to be mapped by backend
> >  - Remove grefs mappings (If they are not in use)
> >  - Get maximum amount of grefs kept mapped.
> > 
> > Signed-off-by: Joao Martins 
> > ---
> > v3:
> > * Use DEL for unmapping grefs instead of PUT
> > * Rname from xen_netif_gref_alloc to xen_netif_gref
> > * Add 'status' field on xen_netif_gref
> > * Clarify what 'inflight' means
> > * Use "beginning of the page" instead of "beginning of the grant"
> > * Mention that page needs to be r/w (as it will have to modify \.status)
> > * `data` on ADD|PUT returns number of entries mapped/unmapped.
> > ---
> >  xen/include/public/io/netif.h | 115
> > ++
> >  1 file changed, 115 insertions(+)
> > 
> > diff --git a/xen/include/public/io/netif.h b/xen/include/public/io/netif.h
> > index ca0061410d..0080a260fd 100644
> > --- a/xen/include/public/io/netif.h
> > +++ b/xen/include/public/io/netif.h
> > @@ -353,6 +353,9 @@ struct xen_netif_ctrl_request {
> >  #define XEN_NETIF_CTRL_TYPE_SET_HASH_MAPPING_SIZE 5
> >  #define XEN_NETIF_CTRL_TYPE_SET_HASH_MAPPING  6
> >  #define XEN_NETIF_CTRL_TYPE_SET_HASH_ALGORITHM7
> > +#define XEN_NETIF_CTRL_TYPE_GET_GREF_MAPPING_SIZE 8
> > +#define XEN_NETIF_CTRL_TYPE_ADD_GREF_MAPPING  9
> > +#define XEN_NETIF_CTRL_TYPE_DEL_GREF_MAPPING 10
> > 
> >  uint32_t data[3];
> >  };
> > @@ -391,6 +394,41 @@ struct xen_netif_ctrl_response {
> >  };
> > 
> >  /*
> > + * Static Grants (struct xen_netif_gref)
> > + * =
> > + *
> > + * A frontend may provide a fixed set of grant references to be mapped on
> > + * the backend. The message of type
> > XEN_NETIF_CTRL_TYPE_ADD_GREF_MAPPING
> > + * prior its usage in the command ring allows for creation of these 
> > mappings.
> > + * The backend will maintain a fixed amount of these mappings.
> > + *
> > + * XEN_NETIF_CTRL_TYPE_GET_GREF_MAPPING_SIZE lets a frontend
> > query how many
> > + * of these mappings can be kept.
> > + *
> > + * Each entry in the XEN_NETIF_CTRL_TYPE_{ADD,DEL}_GREF_MAPPING
> > input table has
> > + * the following format:
> > + *
> > + *0 1 2 3 4 5 6 7  octet
> > + * +-+-+-+-+-+-+-+-+
> > + * | grant ref |  flags|  status   |
> > + * +-+-+-+-+-+-+-+-+
> > + *
> > + * grant ref: grant reference
> > + * flags: flags describing the control operation
> > + * status: XEN_NETIF_CTRL_STATUS_*
> > + */
> 
> You may want to add some words here pointing out that the status is an
> 'out' field, and also whether it should be initialized to zero or not.
> 
OK.

> > +
> > +struct xen_netif_gref {
> > +   grant_ref_t ref;
> > +   uint16_t flags;
> > +
> > +#define _XEN_NETIF_CTRLF_GREF_readonly0
> > +#define XEN_NETIF_CTRLF_GREF_readonly
> > (1U<<_XEN_NETIF_CTRLF_GREF_readonly)
> > +
> > +   uint16_t status;
> > +};
> > +
> > +/*
> >   * Control messages
> >   * 
> >   *
> > @@ -609,6 +647,83 @@ struct xen_netif_ctrl_response {
> >   *   invalidate any table data outside that range.
> >   *   The grant reference may be read-only and must remain valid until
> >   *   the response has been processed.
> > + *
> > + * XEN_NETIF_CTRL_TYPE_GET_GREF_MAPPING_SIZE
> > + * -
> > + *
> > + * This is sent by the frontend to fetch the number of grefs that can be 
> > kept
> > + * mapped in the backend.
> > + *
> > + * Request:
> > + *
> > + *  type= XEN_NETIF_CTRL_TYPE_GET_GREF_MAPPING_SIZE
> > + *  data[0] = queue index (assumed 0 for single queue)
> > + *  data[1] = 0
> > + *  data[2] = 0
> > + *
> > + * Response:
> > + *
> > + *  status = XEN_NETIF_CTRL_STATUS_NOT_SUPPORTED - Operation not
> > + * supported
> > + *   XEN_NETIF_CTRL_STATUS_INVALID_PARAMETER - The queue index
> > is
> > + * out of range
> > + *   XEN_NETIF_CTRL_STATUS_SUCCESS   - Operation successful
> > + *  data   = maximum number of 

Re: [Xen-devel] [PATCH v3 0/1] netif: staging grants for I/O requests

[Paul Durrant]

> -Original Message-
> From: Joao Martins [mailto:joao.m.mart...@oracle.com]
> Sent: 18 September 2017 12:36
> To: Paul Durrant 
> Cc: Xen-devel ; Wei Liu ;
> Konrad Rzeszutek Wilk 
> Subject: Re: [PATCH v3 0/1] netif: staging grants for I/O requests
> 
> On Mon, Sep 18, 2017 at 09:45:06AM +, Paul Durrant wrote:
> > > -Original Message-
> > > From: Joao Martins [mailto:joao.m.mart...@oracle.com]
> > > Sent: 13 September 2017 19:11
> > > To: Xen-devel 
> > > Cc: Wei Liu ; Paul Durrant
> ;
> > > Konrad Rzeszutek Wilk ; Joao Martins
> > > 
> > > Subject: [PATCH v3 0/1] netif: staging grants for I/O requests
> > >
> > > Hey,
> > >
> > > This is v3 taking into consideration all comments received from v2
> (changelog
> > > in the first patch). The specification is right after the diffstat.
> > >
> > > Reference implementation also here (on top of net-next):
> > >
> > > https://github.com/jpemartins/linux.git xen-net-stg-gnts-v3
> > >
> > > Although I am satisfied with how things are being done above, I wanted
> > > to request some advise/input on whether there could be a simpler way
> of
> > > achieving the same. Specifically because these control messages
> > > adds up significant code on the frontend to pregrant, and in other cases
> the
> > > control message might be limitative if frontend tries to keep a 
> > > dinamically
> > > changed buffer pool in different queues. *Maybe* it could be simpler to
> > > adjust
> > > the TX/RX ring ABI in a compatible matter (Disclaimer: I haven't
> implemented
> > > this just yet):
> >
> > But the whole point of pre-granting is to separate the grant/ungrant
> > operations from the rx/tx operations, right?
> 
> /nods
> 
> > So, why would the extra
> > control messages really be an overhead?
> 
> It's not that it's an overhead, but more like the bigger amount of code
> to pregrant once ... and so I was trying to figure out if there was some
> simplification/flexibility that could be made; in the meantime I was
> experimenting a bit and it looks that won't probably make too much
> difference implementation-wise while implying higher complexity on the
> datapath and also weaker semantics.
> 
> With things like AF_PACKET v4 (pre mapping buffers) appearing in linux
> mid term, it will require stronger semantics like those provided by the
> control ring ops rather than these flags I was suggesting below.
> 
> The advantage with the flags though is that add/del mappings would be
> (by design) on the context of the queue rather than in the control
> ring thread handling it. But maybe this can be considered implementation
> specific behaviour too and we could find ways to handle that better if it
> ever becomes a problem e.g. doing the pre{un,}maps on dealloc thread
> context.
> 

Yes, let's not introduce unnecessary complexity at this stage. If optimizations 
are needed then they can be dealt with later.

Cheers,

  Paul

> Joao
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 0/1] netif: staging grants for I/O requests

[Joao Martins]

On Mon, Sep 18, 2017 at 09:45:06AM +, Paul Durrant wrote:
> > -Original Message-
> > From: Joao Martins [mailto:joao.m.mart...@oracle.com]
> > Sent: 13 September 2017 19:11
> > To: Xen-devel 
> > Cc: Wei Liu ; Paul Durrant ;
> > Konrad Rzeszutek Wilk ; Joao Martins
> > 
> > Subject: [PATCH v3 0/1] netif: staging grants for I/O requests
> > 
> > Hey,
> > 
> > This is v3 taking into consideration all comments received from v2 
> > (changelog
> > in the first patch). The specification is right after the diffstat.
> > 
> > Reference implementation also here (on top of net-next):
> > 
> > https://github.com/jpemartins/linux.git xen-net-stg-gnts-v3
> > 
> > Although I am satisfied with how things are being done above, I wanted
> > to request some advise/input on whether there could be a simpler way of
> > achieving the same. Specifically because these control messages
> > adds up significant code on the frontend to pregrant, and in other cases the
> > control message might be limitative if frontend tries to keep a dinamically
> > changed buffer pool in different queues. *Maybe* it could be simpler to
> > adjust
> > the TX/RX ring ABI in a compatible matter (Disclaimer: I haven't implemented
> > this just yet):
> 
> But the whole point of pre-granting is to separate the grant/ungrant
> operations from the rx/tx operations, right?

/nods

> So, why would the extra
> control messages really be an overhead?

It's not that it's an overhead, but more like the bigger amount of code
to pregrant once ... and so I was trying to figure out if there was some
simplification/flexibility that could be made; in the meantime I was
experimenting a bit and it looks that won't probably make too much
difference implementation-wise while implying higher complexity on the
datapath and also weaker semantics.

With things like AF_PACKET v4 (pre mapping buffers) appearing in linux
mid term, it will require stronger semantics like those provided by the
control ring ops rather than these flags I was suggesting below.

The advantage with the flags though is that add/del mappings would be
(by design) on the context of the queue rather than in the control
ring thread handling it. But maybe this can be considered implementation
specific behaviour too and we could find ways to handle that better if it
ever becomes a problem e.g. doing the pre{un,}maps on dealloc thread context.

Joao

> > 
> >  1) Add a flag `NETTXF_persist` to `netif_tx_request`
> > 
> >  2) Replace RX `netif_rx_request` padding with `flags` and adda
> >  `NETRXF_persist` with the same purpose as 1).
> > 
> >  3) This remains backwards compatible as backends not supporting this
> > wouldn't
> >  act on this new flag, and given we replace padding with flags means
> > unsupported
> >  backends won't simply be aware of RX *request* `flags`. This is under the
> >  assumption that there's no requirement that padding must be zero
> > throughout
> >  the netif.h specification.
> > 
> >  4) Keeping `GET_GREF_MAPPING_SIZE` ctrl msg for frontend to do better
> >  decisions?
> > 
> >  5) Semantics are simple: slots with flags marked as NET{RX,TX}F_persist
> >  represent a permanent mapped ref and therefore mapped if non-existent.
> >  *future* omissions of the flag signals the mapping should be removed.
> > 
> > This would allow guests which reuse buffers (apparently Windows :)) to scale
> > better as unmaps would be done on the individual queue context  plus
> > allowing
> > frontend to remain a more simple in the management of "permanent"
> > buffers. The
> > drawback seems to be the added complexity (and somewhat racy behaviour)
> > on the
> > datapath, to map or unmap accordingly. Because now we would have to
> > differentiate between long vs short lived map/unmap ops in addition to
> > looking
> > up on our mappings table. Thoughts, or perhaps people may prefer the one
> > already described in the series?
> > 
> > Cheers,
> > 
> > Joao Martins (1):
> >   public/io/netif.h: add gref mapping control messages
> > 
> >  xen/include/public/io/netif.h | 115
> > ++
> >  1 file changed, 115 insertions(+)
> > ---
> > % Staging grants for network I/O requests
> > % Joao Martins <>
> > % Revision 3
> > 
> > \clearpage
> > 
> > 
> > Architecture(s): Any
> > 
> > 
> > # Background and Motivation
> > 
> > At the Xen hackaton '16 networking session, we spoke about having a
> > permanently
> > mapped region to describe header/linear region of packet buffers. This
> > document
> > outlines the proposal covering motivation of this and applicability for 
> > other
> > use-cases alongside the necessary changes. This proposal is an RFC and also
> > includes alternative 

Re: [Xen-devel] [PATCH 06/27 v9] xen/arm: vpl011: Add a new domctl API to initialize vpl011

[Wei Liu]

On Mon, Sep 18, 2017 at 04:01:50PM +0530, Bhupinder Thakur wrote:
>  
> +int libxl__arch_build_dom_finish(libxl__gc *gc,
> + libxl_domain_build_info *info,
> + struct xc_dom_image *dom,
> + libxl__domain_build_state *state)
> +{
> +int rc, ret;
> +
> +if (info->arch_arm.vuart != LIBXL_VUART_TYPE_SBSA_UART) {
> +rc = 0;
> +goto out;
> +}
> +
> +ret = xc_dom_vuart_init(CTX->xch,
> +XEN_DOMCTL_VUART_TYPE_VPL011,
> +dom->guest_domid,
> +dom->console_domid,
> +dom->vuart_gfn,
> +>vuart_port);
> +if (ret < 0) {
> +rc = ERROR_FAIL;
> +LOG(ERROR, "xc_dom_vuart_init failed\n");
> +}
> +
> +out:
> +return rc;

rc is uninitialised in this function, which you already noticed.

>  {
> @@ -119,6 +144,33 @@ long arch_do_domctl(struct xen_domctl *domctl, struct 
> domain *d,
>  d->disable_migrate = domctl->u.disable_migrate.disable;
>  return 0;
>  
> +case XEN_DOMCTL_vuart_op:
> +{
> +int rc;
> +unsigned int i;
> +struct xen_domctl_vuart_op *vuart_op = >u.vuart_op;
> +
> +/* check that structure padding must be 0. */
> +for ( i = 0; i < sizeof(vuart_op->pad); i++ )
> +if ( vuart_op->pad[i] )
> +return -EINVAL;
> +
> +switch( vuart_op->cmd )
> +{
> +case XEN_DOMCTL_VUART_OP_INIT:
> +rc = handle_vuart_init(d, vuart_op);
> +break;
> +
> +default:
> +rc = -EINVAL;
> +break;
> +}
> +
> +if ( !rc )
> +rc = __copy_to_guest(u_domctl, domctl, 1);

Please use the non-underscored variant which has more checks.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [xen-unstable test] 113562: regressions - FAIL

[Jan Beulich]

>>> On 18.09.17 at 13:05,  wrote:
> On 09/18/2017 11:46 AM, Roger Pau Monné wrote:
>> On Mon, Sep 18, 2017 at 11:15:03AM +0100, George Dunlap wrote:
>>> On 09/18/2017 10:45 AM, Roger Pau Monné wrote:
 On Mon, Sep 18, 2017 at 10:37:58AM +0100, Wei Liu wrote:
> On Mon, Sep 18, 2017 at 08:36:03AM +, osstest service owner wrote:
>> flight 113562 xen-unstable real [real]
>> http://logs.test-lab.xenproject.org/osstest/logs/113562/ 
>>
>> Regressions :-(
>>
>> Tests which did not succeed and are blocking,
>> including tests which could not be run:
>>  test-amd64-amd64-xl-credit2  15 guest-saverestorefail REGR. vs. 
> 113387
>
> There appears to be a bug:
>
> 
> http://logs.test-lab.xenproject.org/osstest/logs/113562/test-amd64-amd64-xl-c 
> redit2/serial-godello0.log
>
> Sep 18 01:14:28.803062 (XEN) Xen BUG at spinlock.c:47

 Seem to be caused because budget_lock is sometimes locked with irqsave
 while others not.
>>>
>>> Just wondering where you're getting the budget lock from?  The call
>>> stack in that link makes it look like it's the RCU clean-up triggering a
>>> domain destroy.  (Haven't looked deeper into the specific line numbers.)
>> 
>> Just skimmed over the commit and jumped into conclusions too fast. As
>> you mention later the issue is calling xfree with interrupts disabled
>> in csched2_free_domdata.
>> 
>> I would rather prefer budget_lock to be always locked with the
>> irqsave/restore variant to make what you mention above more obvious,
>> but that's just a question of taste.
> 
> I *think* at some point in the past we had a discussion about this and
> someone (perhaps Jan?) said if we always know the irqs are disabled we
> shouldn't call the _irqsave() version, to save cpu cycles.

Regardless if it was me back then, I certainly share that position.

> Personally I think the ASSERT()s are clear enough to people familiar
> with the scheduling code.

I agree.

Jan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 06/27 v9] xen/arm: vpl011: Add a new domctl API to initialize vpl011

[Bhupinder Thakur]

Found one issue in this patch:

In libxl__arch_build_dom_finish, rc is not getting initialized.
Surprisingly, it was working fine in my testing.

Regards,
Bhupinder

On 18 September 2017 at 16:01, Bhupinder Thakur
 wrote:
> Add a new domctl API to initialize vpl011. It takes the GFN and console
> backend domid as input and returns an event channel to be used for
> sending and receiving events from Xen.
>
> Xen will communicate with xenconsole using GFN as the ring buffer and
> the event channel to transmit and receive pl011 data on the guest domain's
> behalf.
>
> Signed-off-by: Bhupinder Thakur 
> ---
> CC: Ian Jackson 
> CC: Wei Liu 
> CC: Andrew Cooper 
> CC: George Dunlap 
> CC: Jan Beulich 
> CC: Konrad Rzeszutek Wilk 
> CC: Stefano Stabellini 
> CC: Tim Deegan 
> CC: Julien Grall 
>
> Changes since v8:
> - Added explicit padding in the vuart_op structure
> - Moved vuart_op structure after the PSR structure definition
> - The input fields moved before the output fields in vuart_op structure
> - Checking explicitly that padding fields are initialized to 0
>
> Changes since v6:
> - Renamed the vuart initialization function to a generic name 
> xc_dom_vuart_init
> - Used domid_t as a type instead of uint32_t for domid
> - Checking the vuart type explicitly against vpl011 enum value
>
> Changes since v5:
> - xc_dom_vpl011_init() will be compiled for both x86/arm architectures as 
> there
>   is nothing architecture specific in this function. This function will return
>   error when called for x86.
> - Fixed coding style issues in libxl.
>
> Changes since v4:
> - Removed libxl__arch_domain_create_finish().
> - Added a new function libxl__arch_build_dom_finish(), which is called at the 
> last
>   in libxl__build_dom(). This function calls the vpl011 initialization 
> function now.
>
> Changes since v3:
> - Added a new arch specific function libxl__arch_domain_create_finish(), which
>   calls the vpl011 initialization function. For x86 this function does not do
>   anything.
> - domain_vpl011_init() takes a pointer to a structure which contains all the
>   required information such as console_domid, gfn instead of passing 
> parameters
>   separately.
> - Dropped a DOMCTL API defined for de-initializing vpl011 as that should be
>   taken care when the domain is destroyed (and not dependent on userspace
>   libraries/applications).
>
> Changes since v2:
> - Replaced the DOMCTL APIs defined for get/set of event channel and GFN with
>   a set of DOMCTL APIs for initializing and de-initializing vpl011 emulation.
>
>  tools/libxc/include/xenctrl.h | 20 +
>  tools/libxc/xc_domain.c   | 27 ++
>  tools/libxl/libxl_arch.h  |  7 ++
>  tools/libxl/libxl_arm.c   | 27 ++
>  tools/libxl/libxl_dom.c   |  4 
>  tools/libxl/libxl_x86.c   |  8 +++
>  xen/arch/arm/domain.c |  6 +
>  xen/arch/arm/domctl.c | 52 
> +++
>  xen/include/public/domctl.h   | 22 ++
>  9 files changed, 173 insertions(+)
>
> diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
> index 43151cb..37beb14 100644
> --- a/tools/libxc/include/xenctrl.h
> +++ b/tools/libxc/include/xenctrl.h
> @@ -886,6 +886,26 @@ int xc_vcpu_getcontext(xc_interface *xch,
> vcpu_guest_context_any_t *ctxt);
>
>  /**
> + * This function initializes the vuart emulation and returns
> + * the event to be used by the backend for communicating with
> + * the emulation code.
> + *
> + * @parm xch a handle to an open hypervisor interface
> + * #parm type type of vuart
> + * @parm domid the domain to get information from
> + * @parm console_domid the domid of the backend console
> + * @parm gfn the guest pfn to be used as the ring buffer
> + * @parm evtchn the event channel to be used for events
> + * @return 0 on success, negative error on failure
> + */
> +int xc_dom_vuart_init(xc_interface *xch,
> +  uint32_t type,
> +  domid_t domid,
> +  domid_t console_domid,
> +  xen_pfn_t gfn,
> +  evtchn_port_t *evtchn);
> +
> +/**
>   * This function returns information about the XSAVE state of a particular
>   * vcpu of a domain. If extstate->size and extstate->xfeature_mask are 0,
>   * the call is considered a query to retrieve them and the buffer is not
> diff --git a/tools/libxc/xc_domain.c b/tools/libxc/xc_domain.c
> index 3bab4e8..43720a2 100644
> --- a/tools/libxc/xc_domain.c
> +++ b/tools/libxc/xc_domain.c
> @@ -343,6 +343,33 @@ int xc_domain_get_guest_width(xc_interface *xch, 
> uint32_t domid,
>  return 0;
>  }
>
> 

Re: [Xen-devel] [xen-unstable test] 113562: regressions - FAIL

[Juergen Gross]

On 18/09/17 13:05, George Dunlap wrote:
> On 09/18/2017 11:46 AM, Roger Pau Monné wrote:
>> On Mon, Sep 18, 2017 at 11:15:03AM +0100, George Dunlap wrote:
>>> On 09/18/2017 10:45 AM, Roger Pau Monné wrote:
 On Mon, Sep 18, 2017 at 10:37:58AM +0100, Wei Liu wrote:
> On Mon, Sep 18, 2017 at 08:36:03AM +, osstest service owner wrote:
>> flight 113562 xen-unstable real [real]
>> http://logs.test-lab.xenproject.org/osstest/logs/113562/
>>
>> Regressions :-(
>>
>> Tests which did not succeed and are blocking,
>> including tests which could not be run:
>>  test-amd64-amd64-xl-credit2  15 guest-saverestorefail REGR. vs. 
>> 113387
>
> There appears to be a bug:
>
> http://logs.test-lab.xenproject.org/osstest/logs/113562/test-amd64-amd64-xl-credit2/serial-godello0.log
>
> Sep 18 01:14:28.803062 (XEN) Xen BUG at spinlock.c:47

 Seem to be caused because budget_lock is sometimes locked with irqsave
 while others not.
>>>
>>> Just wondering where you're getting the budget lock from?  The call
>>> stack in that link makes it look like it's the RCU clean-up triggering a
>>> domain destroy.  (Haven't looked deeper into the specific line numbers.)
>>
>> Just skimmed over the commit and jumped into conclusions too fast. As
>> you mention later the issue is calling xfree with interrupts disabled
>> in csched2_free_domdata.
>>
>> I would rather prefer budget_lock to be always locked with the
>> irqsave/restore variant to make what you mention above more obvious,
>> but that's just a question of taste.
> 
> I *think* at some point in the past we had a discussion about this and
> someone (perhaps Jan?) said if we always know the irqs are disabled we
> shouldn't call the _irqsave() version, to save cpu cycles.
> 
> Personally I think the ASSERT()s are clear enough to people familiar
> with the scheduling code.

Why don't we add _irqoff variants of the locks containing the ASSERTion
that interrupts are really off? This would save the additional
instructions of the irqsave/restore variants and make it very clear that
no violation of the lock interface is happening.


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [xen-unstable test] 113562: regressions - FAIL

[George Dunlap]

On 09/18/2017 11:46 AM, Roger Pau Monné wrote:
> On Mon, Sep 18, 2017 at 11:15:03AM +0100, George Dunlap wrote:
>> On 09/18/2017 10:45 AM, Roger Pau Monné wrote:
>>> On Mon, Sep 18, 2017 at 10:37:58AM +0100, Wei Liu wrote:
 On Mon, Sep 18, 2017 at 08:36:03AM +, osstest service owner wrote:
> flight 113562 xen-unstable real [real]
> http://logs.test-lab.xenproject.org/osstest/logs/113562/
>
> Regressions :-(
>
> Tests which did not succeed and are blocking,
> including tests which could not be run:
>  test-amd64-amd64-xl-credit2  15 guest-saverestorefail REGR. vs. 
> 113387

 There appears to be a bug:

 http://logs.test-lab.xenproject.org/osstest/logs/113562/test-amd64-amd64-xl-credit2/serial-godello0.log

 Sep 18 01:14:28.803062 (XEN) Xen BUG at spinlock.c:47
>>>
>>> Seem to be caused because budget_lock is sometimes locked with irqsave
>>> while others not.
>>
>> Just wondering where you're getting the budget lock from?  The call
>> stack in that link makes it look like it's the RCU clean-up triggering a
>> domain destroy.  (Haven't looked deeper into the specific line numbers.)
> 
> Just skimmed over the commit and jumped into conclusions too fast. As
> you mention later the issue is calling xfree with interrupts disabled
> in csched2_free_domdata.
> 
> I would rather prefer budget_lock to be always locked with the
> irqsave/restore variant to make what you mention above more obvious,
> but that's just a question of taste.

I *think* at some point in the past we had a discussion about this and
someone (perhaps Jan?) said if we always know the irqs are disabled we
shouldn't call the _irqsave() version, to save cpu cycles.

Personally I think the ASSERT()s are clear enough to people familiar
with the scheduling code.

 -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 1/3] libxl: use libxl__read_xenstore_check in vtpm function

[Wei Liu]

libxl__read_xenstore can return NULL. Use the _checked variant to
return early when the read fails.

Coverity-ID: 1418098

Signed-off-by: Wei Liu 
---
 tools/libxl/libxl_vtpm.c | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/tools/libxl/libxl_vtpm.c b/tools/libxl/libxl_vtpm.c
index 21320870d4..bdbeb1b53b 100644
--- a/tools/libxl/libxl_vtpm.c
+++ b/tools/libxl/libxl_vtpm.c
@@ -79,12 +79,14 @@ static int libxl__vtpm_from_xenstore(libxl__gc *gc, const 
char *libxl_path,
  libxl_device_vtpm *vtpm)
 {
 int rc;
-char *be_path;
+const char *be_path;
 char *uuid;
 
 vtpm->devid = devid;
 
-be_path = libxl__xs_read(gc, XBT_NULL, GCSPRINTF("%s/backend", 
libxl_path));
+rc = libxl__xs_read_checked(gc, XBT_NULL,
+GCSPRINTF("%s/backend", libxl_path), _path);
+if (rc) return rc;
 
 rc = libxl__backendpath_parse_domid(gc, be_path, >backend_domid);
 if (rc) return rc;
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 2/3] libxl: use libxl__read_xenstore_check in vdispl function

[Wei Liu]

Coverity-ID: 1418097

Signed-off-by: Wei Liu 
---
 tools/libxl/libxl_vdispl.c | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/tools/libxl/libxl_vdispl.c b/tools/libxl/libxl_vdispl.c
index 5740c89fad..cfee0f5cdf 100644
--- a/tools/libxl/libxl_vdispl.c
+++ b/tools/libxl/libxl_vdispl.c
@@ -40,10 +40,13 @@ static int libxl__vdispl_from_xenstore(libxl__gc *gc, const 
char *libxl_path,
libxl_devid devid,
libxl_device_vdispl *vdispl)
 {
-char *be_path;
+const char *be_path;
+int rc;
 
 vdispl->devid = devid;
-be_path = libxl__xs_read(gc, XBT_NULL, GCSPRINTF("%s/backend", 
libxl_path));
+rc = libxl__xs_read_checked(gc, XBT_NULL,
+GCSPRINTF("%s/backend", libxl_path), _path);
+if (rc) return rc;
 
 return libxl__backendpath_parse_domid(gc, be_path, >backend_domid);
 }
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 3/3] xl: avoid leaking memory in vdispl parser

[Wei Liu]

Coverity-ID: 1418095

Signed-off-by: Wei Liu 
---
 tools/xl/xl_parse.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 9965b83c44..0678fbc1b0 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -832,6 +832,8 @@ int parse_vdispl_config(libxl_device_vdispl *vdispl, char 
*token)
 
 rc= sscanf(resolution, "%ux%u", >connectors[i].width,
>connectors[i].height);
+free(resolution);
+
 if (rc != 2) {
 fprintf(stderr, "Can't parse connector resolution\n");
 goto out;
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 0/3] Some coverity fixes for xl/libxl

[Wei Liu]

Wei Liu (3):
  libxl: use libxl__read_xenstore_check in vtpm function
  libxl: use libxl__read_xenstore_check in vdispl function
  xl: avoid leaking memory in vdispl parser

 tools/libxl/libxl_vdispl.c | 7 +--
 tools/libxl/libxl_vtpm.c   | 6 --
 tools/xl/xl_parse.c| 2 ++
 3 files changed, 11 insertions(+), 4 deletions(-)

-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [xen-unstable test] 113562: regressions - FAIL

[Roger Pau Monné]

On Mon, Sep 18, 2017 at 11:15:03AM +0100, George Dunlap wrote:
> On 09/18/2017 10:45 AM, Roger Pau Monné wrote:
> > On Mon, Sep 18, 2017 at 10:37:58AM +0100, Wei Liu wrote:
> >> On Mon, Sep 18, 2017 at 08:36:03AM +, osstest service owner wrote:
> >>> flight 113562 xen-unstable real [real]
> >>> http://logs.test-lab.xenproject.org/osstest/logs/113562/
> >>>
> >>> Regressions :-(
> >>>
> >>> Tests which did not succeed and are blocking,
> >>> including tests which could not be run:
> >>>  test-amd64-amd64-xl-credit2  15 guest-saverestorefail REGR. vs. 
> >>> 113387
> >>
> >> There appears to be a bug:
> >>
> >> http://logs.test-lab.xenproject.org/osstest/logs/113562/test-amd64-amd64-xl-credit2/serial-godello0.log
> >>
> >> Sep 18 01:14:28.803062 (XEN) Xen BUG at spinlock.c:47
> > 
> > Seem to be caused because budget_lock is sometimes locked with irqsave
> > while others not.
> 
> Just wondering where you're getting the budget lock from?  The call
> stack in that link makes it look like it's the RCU clean-up triggering a
> domain destroy.  (Haven't looked deeper into the specific line numbers.)

Just skimmed over the commit and jumped into conclusions too fast. As
you mention later the issue is calling xfree with interrupts disabled
in csched2_free_domdata.

I would rather prefer budget_lock to be always locked with the
irqsave/restore variant to make what you mention above more obvious,
but that's just a question of taste.

Roger.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 06/27 v9] xen/arm: vpl011: Add a new domctl API to initialize vpl011

[Jan Beulich]

>>> On 18.09.17 at 12:31,  wrote:
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -36,6 +36,7 @@
>  #include "grant_table.h"
>  #include "hvm/save.h"
>  #include "memory.h"
> +#include "event_channel.h"

Please play by the alphabetic sorting here (xen.h going first being
the acceptable exception, albeit it doesn't really need explicitly
including here).

> @@ -1160,9 +1161,28 @@ struct xen_domctl_psr_cat_op {
>  uint32_t target;/* IN */
>  uint64_t data;  /* IN/OUT */
>  };
> +
>  typedef struct xen_domctl_psr_cat_op xen_domctl_psr_cat_op_t;

Stray addition of a blank line.

>  DEFINE_XEN_GUEST_HANDLE(xen_domctl_psr_cat_op_t);
>  
> +struct xen_domctl_vuart_op {

Please have a comment ahead of this mentioning the XEN_DOMCTL_*
to use with this structure, like many (but sadly not all) structures
have.

> +#define XEN_DOMCTL_VUART_OP_INIT  0
> +uint32_t cmd;   /* XEN_DOMCTL_VUART_OP_* */
> +#define XEN_DOMCTL_VUART_TYPE_VPL011 0
> +uint32_t type;  /* IN - type of vuart.
> + *  Currently only vpl011 supported.
> + */
> +uint64_aligned_t  gfn;  /* IN - guest gfn to be used as a
> + *  ring buffer.
> + */
> +domid_t console_domid;  /* IN */

And this one's different from the domid in the domctl header? It's
certainly odd that this is the only field without any meaningful
comment...

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] MAINTAINERS: add arch specific public headers to arch file groups

[Jan Beulich]

>>> On 18.09.17 at 12:10,  wrote:
> Hi Jan,
> 
> On 01/09/17 11:45, Jan Beulich wrote:
>> I've recently got sufficiently annoyed by people not applying enough
>> common sense to get_maintainer.pl output, Cc-ing all REST maintainers
>> on ARM-only public interface changes.
> 
> It seems that you didn't move arch-arm.h, arch-x86_{32,64}.h under resp. 
> ARM and x86 maintainership. Was it intentional?

No, but the two x86 ones you mention are merely compatibility
stubs, which aren't really expected to ever change, so who is
their formal maintainer doesn't matter. For arch-arm.h I agree
it should be put under ARM maintainership.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 23/27 v9] xen/arm: vpl011: Add a new vuart console type to xenconsole client

[Bhupinder Thakur]

Add a new console type VUART to connect to guest's emualated vuart
console.

Signed-off-by: Bhupinder Thakur 
Reviewed-by: Stefano Stabellini 
Acked-by: Wei Liu 
---
CC: Ian Jackson 
CC: Wei Liu 
CC: Stefano Stabellini 
CC: Julien Grall 

Changes since v4:
- Removed the vuart compile time flag so that vuart code is compiled always.

Changes since v3:
- The vuart console support is under CONFIG_VUART_CONSOLE option.
- Since there is a change from last review, I have not included
  reviewed-by tag from Stefano and acked-by tag from Wei.

 tools/console/client/main.c | 13 +++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/tools/console/client/main.c b/tools/console/client/main.c
index c340cb7..f92ad3d 100644
--- a/tools/console/client/main.c
+++ b/tools/console/client/main.c
@@ -76,7 +76,7 @@ static void usage(const char *program) {
   "\n"
   "  -h, --help   display this help and exit\n"
   "  -n, --num N  use console number N\n"
-  "  --type TYPE  console type. must be 'pv' or 'serial'\n"
+  "  --type TYPE  console type. must be 'pv', 'serial' or 
'vuart'\n"
   "  --start-notify-fd N file descriptor used to notify parent\n"
   , program);
 }
@@ -264,6 +264,7 @@ typedef enum {
CONSOLE_INVAL,
CONSOLE_PV,
CONSOLE_SERIAL,
+   CONSOLE_VUART,
 } console_type;
 
 static struct termios stdin_old_attr;
@@ -344,6 +345,7 @@ int main(int argc, char **argv)
char *end;
console_type type = CONSOLE_INVAL;
bool interactive = 0;
+   char *console_names = "serial, pv, vuart";
 
while((ch = getopt_long(argc, argv, sopt, lopt, _ind)) != -1) {
switch(ch) {
@@ -359,9 +361,12 @@ int main(int argc, char **argv)
type = CONSOLE_SERIAL;
else if (!strcmp(optarg, "pv"))
type = CONSOLE_PV;
+   else if (!strcmp(optarg, "vuart"))
+   type = CONSOLE_VUART;
else {
fprintf(stderr, "Invalid type argument\n");
-   fprintf(stderr, "Console types supported are: 
serial, pv\n");
+   fprintf(stderr, "Console types supported are: 
%s\n",
+   console_names);
exit(EINVAL);
}
break;
@@ -437,6 +442,10 @@ int main(int argc, char **argv)
else
snprintf(path, strlen(dom_path) + 
strlen("/device/console/%d/tty") + 5, "%s/device/console/%d/tty", dom_path, 
num);
}
+   if (type == CONSOLE_VUART) {
+   snprintf(path, strlen(dom_path) + strlen("/vuart/0/tty") + 1,
+"%s/vuart/0/tty", dom_path);
+   }
 
/* FIXME consoled currently does not assume domain-0 doesn't have a
   console which is good when we break domain-0 up.  To keep us
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 18/27 v9] xen/arm: vpl011: Add a new console_cleanup function in xenconsole

[Bhupinder Thakur]

This patch introduces a new console_cleanup function. This function
frees up the console resources.

Signed-off-by: Bhupinder Thakur 
Reviewed-by: Stefano Stabellini 
Acked-by: Wei Liu 
---
CC: Ian Jackson 
CC: Wei Liu 
CC: Stefano Stabellini 
CC: Julien Grall 

Changes since v6:
- Removed a null pointer check before calling free() as free() already checks 
that.

Changes since v5:
- Split this change in a separate patch.

 tools/console/daemon/io.c | 15 ++-
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/tools/console/daemon/io.c b/tools/console/daemon/io.c
index 5c6da31..ff69e52 100644
--- a/tools/console/daemon/io.c
+++ b/tools/console/daemon/io.c
@@ -768,12 +768,8 @@ static void remove_domain(struct domain *dom)
}
 }
 
-static void cleanup_domain(struct domain *d)
+static void console_cleanup(struct console *con)
 {
-   struct console *con = >console;
-
-   console_close_tty(con);
-
if (con->log_fd != -1) {
close(con->log_fd);
con->log_fd = -1;
@@ -784,6 +780,15 @@ static void cleanup_domain(struct domain *d)
 
free(con->xspath);
con->xspath = NULL;
+}
+
+static void cleanup_domain(struct domain *d)
+{
+   struct console *con = >console;
+
+   console_close_tty(con);
+
+   console_cleanup(con);
 
remove_domain(d);
 }
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 17/27 v9] xen/arm: vpl011: Add a new handle_console_tty function in xenconsole

[Bhupinder Thakur]

This patch introduces a new handle_console_tty function. This function
performs read/write from/to console tty.

Signed-off-by: Bhupinder Thakur 
Reviewed-by: Stefano Stabellini 
Acked-by: Wei Liu 
---
CC: Ian Jackson 
CC: Wei Liu 
CC: Stefano Stabellini 
CC: Julien Grall 

Changes since v5:
- Split this change in a separate patch.

 tools/console/daemon/io.c | 35 +++
 1 file changed, 19 insertions(+), 16 deletions(-)

diff --git a/tools/console/daemon/io.c b/tools/console/daemon/io.c
index c361b42..5c6da31 100644
--- a/tools/console/daemon/io.c
+++ b/tools/console/daemon/io.c
@@ -1130,6 +1130,24 @@ static void maybe_add_console_tty_fd(struct console *con)
}
 }
 
+static void handle_console_tty(struct console *con)
+{
+   if (con->master_fd != -1 && con->master_pollfd_idx != -1) {
+   if (fds[con->master_pollfd_idx].revents &
+   ~(POLLIN|POLLOUT|POLLPRI))
+   console_handle_broken_tty(con, 
domain_is_valid(con->d->domid));
+   else {
+   if (fds[con->master_pollfd_idx].revents &
+   POLLIN)
+   handle_tty_read(con);
+   if (fds[con->master_pollfd_idx].revents &
+   POLLOUT)
+   handle_tty_write(con);
+   }
+   }
+   con->master_pollfd_idx = -1;
+}
+
 void handle_io(void)
 {
int ret;
@@ -1260,22 +1278,7 @@ void handle_io(void)
 
handle_console_ring(con);
 
-   if (con->master_fd != -1 && con->master_pollfd_idx != 
-1) {
-   if (fds[con->master_pollfd_idx].revents &
-   ~(POLLIN|POLLOUT|POLLPRI))
-   console_handle_broken_tty(con,
-  domain_is_valid(d->domid));
-   else {
-   if (fds[con->master_pollfd_idx].revents 
&
-   POLLIN)
-   handle_tty_read(con);
-   if (fds[con->master_pollfd_idx].revents 
&
-   POLLOUT)
-   handle_tty_write(con);
-   }
-   }
-
-   con->master_pollfd_idx = -1;
+   handle_console_tty(con);
 
if (d->last_seen != enum_pass)
shutdown_domain(d);
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 27/27 v9] xen/arm: vpl011: Correct the logic for asserting/de-asserting SBSA UART TX interrupt

[Bhupinder Thakur]

This patch fixes the issue observed when pl011 patches were tested on
the junos hardware by Andre/Julien. It was observed that when large output is
generated such as on executing 'find /', output was getting truncated 
intermittently
due to OUT ring buffer getting full.

This issue was due to the fact that the SBSA UART driver expects that when
a TX interrupt is asserted then the FIFO queue should be atleast half empty and
that it can write N bytes in the FIFO, where N is half the FIFO queue size, 
without
the bytes getting dropped due to FIFO getting full.

The SBSA UART emulation logic was asserting the TX interrupt as soon as
any space became available in the FIFO and the SBSA UART driver tried to write
more data (upto 16 bytes) in the FIFO expecting that there is enough space
available leading to dropped bytes.

The SBSA spec [1] does not specify any register where the TX interrupt thresold 
can
be set. Due to lack of clarity on the expected behavior, it is
assumed for now that TX interrupt should be asserted only when the FIFO goes
half empty (as expected by the SBSA UART driver).

TBD: Once the SBSA spec is updated with the expected behavior, the 
implementation
will be modified to align with the spec requirement.

[1] http://infocenter.arm.com/help/topic/com.arm.doc.ddi0183f/DDI0183.pdf

Signed-off-by: Bhupinder Thakur 
---
CC: Julien Grall 
CC: Andre Przywara 
CC: Stefano Stabellini 

Changes since v8:
- Used variables fifo_level/fifo_threshold for more clarity
- Added a new macro SBSA_UART_FIFO_SIZE instead of using a magic number
- Renamed ring_qsize variables to fifo_level for consistency 

 xen/arch/arm/vpl011.c| 87 ++--
 xen/include/asm-arm/vpl011.h |  2 +
 2 files changed, 61 insertions(+), 28 deletions(-)

diff --git a/xen/arch/arm/vpl011.c b/xen/arch/arm/vpl011.c
index 36794d8..1f97261 100644
--- a/xen/arch/arm/vpl011.c
+++ b/xen/arch/arm/vpl011.c
@@ -91,20 +91,24 @@ static uint8_t vpl011_read_data(struct domain *d)
  */
 if ( xencons_queued(in_prod, in_cons, sizeof(intf->in)) > 0 )
 {
+unsigned int fifo_level;
+
 data = intf->in[xencons_mask(in_cons, sizeof(intf->in))];
 in_cons += 1;
 smp_mb();
 intf->in_cons = in_cons;
+
+fifo_level = xencons_queued(in_prod, in_cons, sizeof(intf->in));
+
+if ( fifo_level == 0 )
+{
+vpl011->uartfr |= RXFE;
+vpl011->uartris &= ~RXI;
+}
 }
 else
 gprintk(XENLOG_ERR, "vpl011: Unexpected IN ring buffer empty\n");
 
-if ( xencons_queued(in_prod, in_cons, sizeof(intf->in)) == 0 )
-{
-vpl011->uartfr |= RXFE;
-vpl011->uartris &= ~RXI;
-}
-
 vpl011->uartfr &= ~RXFF;
 
 vpl011_update_interrupt_status(d);
@@ -144,28 +148,41 @@ static void vpl011_write_data(struct domain *d, uint8_t 
data)
 if ( xencons_queued(out_prod, out_cons, sizeof(intf->out)) !=
  sizeof (intf->out) )
 {
+unsigned int fifo_level, fifo_threshold;
+
 intf->out[xencons_mask(out_prod, sizeof(intf->out))] = data;
 out_prod += 1;
 smp_wmb();
 intf->out_prod = out_prod;
-}
-else
-gprintk(XENLOG_ERR, "vpl011: Unexpected OUT ring buffer full\n");
 
-if ( xencons_queued(out_prod, out_cons, sizeof(intf->out)) ==
- sizeof (intf->out) )
-{
-vpl011->uartfr |= TXFF;
-vpl011->uartris &= ~TXI;
+fifo_level = xencons_queued(out_prod, out_cons, sizeof(intf->out));
+
+if ( fifo_level == sizeof (intf->out) )
+{
+vpl011->uartfr |= TXFF;
+
+/*
+ * This bit is set only when FIFO becomes full. This ensures that
+ * the SBSA UART driver can write the early console data as fast as
+ * possible, without waiting for the BUSY bit to get cleared before
+ * writing each byte.
+ */
+vpl011->uartfr |= BUSY;
+}
 
 /*
- * This bit is set only when FIFO becomes full. This ensures that
- * the SBSA UART driver can write the early console data as fast as
- * possible, without waiting for the BUSY bit to get cleared before
- * writing each byte.
+ * Clear the TXI bit if the fifo level exceeds fifo_size/2 mark which
+ * is the trigger level for asserting/de-assterting the TX interrupt.
  */
-vpl011->uartfr |= BUSY;
+fifo_threshold = sizeof (intf->out) - SBSA_UART_FIFO_SIZE/2;
+
+if ( fifo_level <= fifo_threshold )
+vpl011->uartris |= TXI;
+else
+vpl011->uartris &= ~TXI;
 }
+else
+gprintk(XENLOG_ERR, "vpl011: Unexpected OUT ring buffer full\n");
 
 vpl011->uartfr &= ~TXFE;
 
@@ -342,7 +359,7 @@ static void vpl011_data_avail(struct domain *d)
 struct vpl011 *vpl011 = 

[Xen-devel] [PATCH 19/27 v9] xen/arm: vpl011: Add a new console_open_log function in xenconsole

[Bhupinder Thakur]

This patch introduces a console_open_log console_cleanup function. This function
opens the console log file.

Signed-off-by: Bhupinder Thakur 
Reviewed-by: Stefano Stabellini 
Acked-by: Wei Liu 
---
CC: Ian Jackson 
CC: Wei Liu 
CC: Stefano Stabellini 
CC: Julien Grall 

Changes since v5:
- Split this change in a separate patch.

 tools/console/daemon/io.c | 13 ++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/tools/console/daemon/io.c b/tools/console/daemon/io.c
index ff69e52..cfd7273 100644
--- a/tools/console/daemon/io.c
+++ b/tools/console/daemon/io.c
@@ -1038,6 +1038,15 @@ static void handle_hv_logs(xenevtchn_handle *xce_handle, 
bool force)
(void)xenevtchn_unmask(xce_handle, port);
 }
 
+static void console_open_log(struct console *con)
+{
+   if (console_enabled(con)) {
+   if (con->log_fd != -1)
+   close(con->log_fd);
+   con->log_fd = create_console_log(con);
+   }
+}
+
 static void handle_log_reload(void)
 {
if (log_guest) {
@@ -1045,9 +1054,7 @@ static void handle_log_reload(void)
for (d = dom_head; d; d = d->next) {
struct console *con = >console;
 
-   if (con->log_fd != -1)
-   close(con->log_fd);
-   con->log_fd = create_console_log(con);
+   console_open_log(con);
}
}
 
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 25/27 v9] xen/arm: vpl011: Update documentation for vuart console support

[Bhupinder Thakur]

1. Update documentation for a new vuart option added.
2. Update documentation about SPI irq reserved for vuart.

Signed-off-by: Bhupinder Thakur 
Reviewed-by: Stefano Stabellini 
Acked-by: Wei Liu 
---
CC: Ian Jackson 
CC: Wei Liu 
CC: Andrew Cooper 
CC: George Dunlap 
CC: Jan Beulich 
CC: Konrad Rzeszutek Wilk 
CC: Stefano Stabellini 
CC: Tim Deegan 
CC: Julien Grall 

Changes since v6:
- Added a new section for vuart usage.
- I have retained the reviewed-by/acked-by tags as this is a limited change. 
Kindly
  review.

Changes since v4:
- Minor change to rename "pl011" to "sbsa_uart". Since it is a minor change I 
have
  retained the reviewed-by and acked-by tags.

 docs/man/xl.cfg.pod.5.in | 12 
 docs/misc/console.txt| 44 +---
 2 files changed, 45 insertions(+), 11 deletions(-)

diff --git a/docs/man/xl.cfg.pod.5.in b/docs/man/xl.cfg.pod.5.in
index 79cb2ea..21cf43f 100644
--- a/docs/man/xl.cfg.pod.5.in
+++ b/docs/man/xl.cfg.pod.5.in
@@ -1105,6 +1105,9 @@ Allow a guest to access specific physical IRQs.
 It is recommended to only use this option for trusted VMs under
 administrator's control.
 
+If vuart console is enabled then irq 32 is reserved for it. See
+L to know how to enable vuart console.
+
 =item