[Xen-devel] [xen-unstable test] 78129: regressions - FAIL

2016-01-15 Thread osstest service owner
flight 78129 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/78129/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 5 xen-install fail REGR. 
vs. 77892
 test-amd64-amd64-xl-xsm   5 xen-install   fail REGR. vs. 77892
 test-amd64-i386-xl-xsm5 xen-install   fail REGR. vs. 77892
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 5 xen-install fail REGR. 
vs. 77892
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm 5 xen-install fail REGR. vs. 77892
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm 5 xen-install fail REGR. vs. 
77892
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm 5 xen-install fail REGR. vs. 
77892
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm 5 xen-install fail REGR. vs. 77892
 test-armhf-armhf-xl-xsm   6 xen-boot  fail REGR. vs. 77892
 test-amd64-i386-xl-qemut-win7-amd64  9 windows-installfail REGR. vs. 77892
 test-amd64-i386-xl-qemuu-win7-amd64  9 windows-installfail REGR. vs. 77892

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-libvirt-xsm   5 xen-install   fail REGR. vs. 77892
 test-amd64-amd64-libvirt-xsm  5 xen-install   fail REGR. vs. 77892
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 5 xen-install fail REGR. vs. 
77892
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 5 xen-install fail REGR. 
vs. 77892
 test-armhf-armhf-libvirt-xsm  6 xen-boot  fail REGR. vs. 77892
 test-amd64-i386-rumpuserxen-i386 10 guest-startfail like 77892
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stop fail like 77892
 test-armhf-armhf-xl-rtds  9 debian-install   fail   like 77892
 test-amd64-amd64-libvirt-vhd  9 debian-di-installfail   like 77892

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-armhf-armhf-xl-vhd   9 debian-di-installfail   never pass

version targeted for testing:
 xen  7167222b15dc661ff0b44cc93d558f6bb4ff6492
baseline version:
 xen  f7347a282420a5edc74afb31e7c42c2765f24de5

Last test of basis77892  2016-01-12 11:30:40 Z2 days
Failing since 77945  2016-01-13 04:01:14 Z2 days4 attempts
Testing same since78129  2016-01-15 00:12:12 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Boris Ostrovsky 
  Brendan Gregg 
  Daniel De Graaf 
  Doug Goldstein 
  Haozhong Zhang 
  Jan Beulich 
  Juergen Gross 
  Kevin Tian 
  Razvan Cojocaru 
  Roger Pau Monné 
  Tamas K Lengyel 
  Wei Liu 
  Wei Liu  for tools bits

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt

Re: [Xen-devel] [v3,11/41] mips: reuse asm-generic/barrier.h

2016-01-15 Thread Peter Zijlstra
On Thu, Jan 14, 2016 at 01:29:13PM -0800, Paul E. McKenney wrote:
> So smp_mb() provides transitivity, as do pairs of smp_store_release()
> and smp_read_acquire(), 

But they provide different grades of transitivity, which is where all
the confusion lays.

smp_mb() is strongly/globally transitive, all CPUs will agree on the order.

Whereas the RCpc release+acquire is weakly so, only the two cpus
involved in the handover will agree on the order.



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [v3,11/41] mips: reuse asm-generic/barrier.h

2016-01-15 Thread Peter Zijlstra
On Fri, Jan 15, 2016 at 09:55:54AM +0100, Peter Zijlstra wrote:
> On Thu, Jan 14, 2016 at 01:29:13PM -0800, Paul E. McKenney wrote:
> > So smp_mb() provides transitivity, as do pairs of smp_store_release()
> > and smp_read_acquire(), 
> 
> But they provide different grades of transitivity, which is where all
> the confusion lays.
> 
> smp_mb() is strongly/globally transitive, all CPUs will agree on the order.
> 
> Whereas the RCpc release+acquire is weakly so, only the two cpus
> involved in the handover will agree on the order.

And the stuff we're confused about is how best to express the difference
and guarantees of these two forms of transitivity and how exactly they
interact.

And smp_load_acquire()/smp_store_release() are RCpc because TSO archs
and PPC. the atomic*_{acquire,release}() are RCpc because PPC and
LOCK,UNLOCK are similarly RCpc because of PPC.

Now we'd like PPC to stick a SYNC in either LOCK or UNLOCK so at least
the locks are RCsc again, but they resist for performance reasons but
waver because they don't want to be the ones finding all the nasty bugs
because they're the only one.

Now the thing I worry about, and still have not had an answer to is if
weakly ordered MIPS will end up being RCsc or RCpc for their locks if
they get implemented with SYNC_ACQUIRE and SYNC_RELEASE instead of the
current SYNC.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] Bug 1591 - xentop truncated VM names

2016-01-15 Thread Michael Großer
Hi!
 
Some days ago, I discovered the 'xentop' tool (somebody mentioned it here:
http://discussions.citrix.com/topic/374218-what-can-we-do-to-decrease-the-time-needed-for-snapshot-reverts/
)
 
We work with XenServer 6.5 and we use long names for our virtual
machines to distinguish between them.
 
A machine name can have about 40 characters, but xentop displays only the first
10 characters in the NAME column. The result is, that I cannot identify a
particular VM using xentop.

I read the man page and googled around to find a solution. I found
the page

http://www.bl-nk.net/2014/07/xentop-and-long-server-names/

and created my own quirky solution as follows:

# cd /usr/sbin
# cp xentop xentop_wide
# sed -i 's/10\.10/80\.80/g' xentop_wide

This works for me today, but my common sense tells me that this solution could
cease working after we update to a newer version or that I could demage
some other instructions that accidentally have similar machine codes.

I discovered this 6 years old bug:

http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1591

The composer of this bug makes recommendations that I wouldn't recommend, so
that seams to be the reason why the bug is nearly 6 years old without being
solved.

I recommend a very simple and reconcilable problem-solving approach:

- The man page should contain / describe another option, let me call it
  "-w / --column-name-width", which requires a number.

- When starting xentop with "-w 80" or "--column-name-width=80", then
  it should display (in every imaginable work mode) a name column with
  a width of 80 characters.

- The default width should stay of cause at 10 characters, but with such an
  option, I as a user could have a choice.

Please tell me, how is the chance that such an bug fix enters a next possible
release of XenServer, which release number could this be and when will it
roughly appear?

Thanks in advance,
Michael Großer

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [Qemu-devel] [PATCH 2/2] xen-hvm: Clean up xen_ram_alloc() error handling

2016-01-15 Thread Markus Armbruster
Stefano Stabellini  writes:

> On Thu, 14 Jan 2016, Markus Armbruster wrote:
>> xen_ram_alloc() dies with hw_error() on error, even though its caller
>> ram_block_add() handles errors just fine.  Add an Error **errp
>> parameter and use it.
>> 
>> Leave case RUN_STATE_INMIGRATE alone, because that looks like some
>> kind of warning.

Did you double-check this is okay?

>> Signed-off-by: Markus Armbruster 
>
> Reviewed-by: Stefano Stabellini 
>
> I am happy to queue both patches up in my next branch.

Please do, thanks!

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] libxl: Use libxl_strdup instead of strdup on libxl_version_info

2016-01-15 Thread Ian Campbell
On Thu, 2016-01-14 at 22:03 -0500, Konrad Rzeszutek Wilk wrote:
> On January 14, 2016 9:33:49 PM EST, Konrad Rzeszutek Wilk  acle.com> wrote:
> > As the libxl_strdup allows us to unwind and free all
> > of the allocations, while strdup requires the callers
> > to remember to free (which they didn't seem too).
> > 
> 
> 
> Gr.. Ignore it pls. I cherry picked it and had a preceding patch that
> added the GC_INIT which this patch missed.
> 
> (And also an GC_FREE).

Allocating these returned values via a gc would be wrong per the comment
about memory management at the top of libxl.h, where these fall into
category (b) and require the caller to free. They should normally do so by
calling libxl_version_info_dispose(), not manually (which would be quite
tedious and error prone).

Returning GC'd values to the application would be wrong since they would be
freed by the GC_FREE before return upon exit from libxl.

However you've used NOGC, which compared to raw strdup etc only has the
extra behaviour of abort-on-alloc-fail and not any "unwind and free all"
behaviour which the commit message mentions. So this would be a good
change, in that it improves error handling, rather than for any of the
reasons mentioned in the commit message.

WRT freeing the results, normally the caller would need to do so by calling
the appropriate _dispose() function. However libxl_get_version_info() is
special and returns a cached result from the ctx which cannot and should
not be freed (as evidenced by it returning a const struct). This data is
freed in libxl_ctx_free() by calling libxl_version_info_dispose(). This is
why none of the callers remember to free -- they shouldn't be doing so.

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v3 2/5] remus: resume immediately if libxl__xc_domain_save_done() completes

2016-01-15 Thread Ian Campbell
On Fri, 2016-01-15 at 13:44 +0800, Wen Congyang wrote:
> On 01/14/2016 06:21 PM, Ian Campbell wrote:
> > On Tue, 2016-01-12 at 09:40 +0800, Wen Congyang wrote:
> > > On 01/09/2016 12:27 AM, Ian Campbell wrote:
> > > > On Fri, 2016-01-08 at 14:38 +0800, Wen Congyang wrote:
> > > > > For example: if the secondary host is down, and we fail to send
> > > > > the
> > > > > data
> > > > > to
> > > > > the secondary host. xc_domain_save() returns 0.
> > > > 
> > > > Just to be check: On failure in this way xc_domain_save() returns 0
> > > > (i.e.
> > > > success)?
> > > 
> > > Yes, it returns 0. I am not sure the return value is right.
> > > 
> > > > 
> > > > >  So in the function
> > > > > libxl__xc_domain_save_done(), rc is 0(the helper program exits
> > > > > normally),
> > > > > and retval is 0(it is xc_domain_save()'s return value). In such
> > > > > case,
> > > > > we
> > > > > just need to complete the stream.
> > > > > 
> > > > > Signed-off-by: Wen Congyang 
> > > > > ---
> > > > >  tools/libxl/libxl_stream_write.c | 13 +++--
> > > > >  1 file changed, 11 insertions(+), 2 deletions(-)
> > > > > 
> > > > > diff --git a/tools/libxl/libxl_stream_write.c
> > > > > b/tools/libxl/libxl_stream_write.c
> > > > > index 80d9208..82e7719 100644
> > > > > --- a/tools/libxl/libxl_stream_write.c
> > > > > +++ b/tools/libxl/libxl_stream_write.c
> > > > > @@ -354,8 +354,17 @@ void libxl__xc_domain_save_done(libxl__egc
> > > > > *egc,
> > > > > void *dss_void,
> > > > >   * alive, and check_all_finished() may have torn it down
> > > > > around
> > > > > us.
> > > > >   * If the stream is not still alive, we must not continue
> > > > > any
> > > > > work.
> > > > >   */
> > > > > -if (libxl__stream_write_inuse(stream))
> > > > > -write_emulator_xenstore_record(egc, stream);
> > > > > +if (libxl__stream_write_inuse(stream)) {
> > > > > +if (dss->remus)
> > > > > +/*
> > > > > + * For remus, if libxl__xc_domain_save_done()
> > > > > completes,
> > > > > + * there was an error sending data to the secondary.
> > > > > + * Resume the primary ASAP.
> > > > > + */
> > > > > +stream_complete(egc, stream, 0);
> > > > 
> > > > Is there an indication to the caller that things have failed in
> > > > this
> > > > way?
> > > > Would that information be of use to the caller?
> > > 
> > > For remus, when we come here, something is wrong regardless of the
> > > return
> > > value.
> > 
> > But does the caller know this? Can it tell.
> > 
> > > 
> > > > 
> > > > Or does the called infer this has happened because
> > > > otherwise libxl_domain_remus_start is not supposed to return?
> > > 
> > > Yes, libxl_domain_remus_start() should not return unless somethins is
> > > wrong.
> > 
> > This really ought to be documented somewhere.
> 
> libxl_domain_remus_start():
> /* Point of no return */
> libxl__remus_setup(egc, dss);
> return AO_INPROGRESS;

This is (obviously) not documentation.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] QEMU as non-root and PCI passthrough do not mix

2016-01-15 Thread Ian Campbell
On Thu, 2016-01-14 at 17:42 +, Ian Jackson wrote:
> Ian Campbell writes ("Re: [PATCH] QEMU as non-root and PCI passthrough do
> not mix"):
> > That reminds me -- how does the qemu user selection work over migrate?
> > Does
> > it remember the specific user or does it try and pick it again on the
> > other
> > end? Do we require all hosts in a pool to have been setup in the same
> > way
> > i.e. either xen-qemuuser-domid$i or xen-qemuuser-shared but not a
> > mixture?
> 
> I forget how it was before my RFC privsep series, but afterwards the
> user is defaulted into the configuration during domain creation and I
> would expect it to be transferred with the config to the destination.
> 
> So the destination needs to have whatever user the source chose to
> use.  I think that's a reasonable restriction.

So do I, but I think it should be written in docs/misc/qemu-deprivilege.txt 
at least.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v3 2/5] remus: resume immediately if libxl__xc_domain_save_done() completes

2016-01-15 Thread Wen Congyang
On 01/15/2016 05:48 PM, Ian Campbell wrote:
> On Fri, 2016-01-15 at 13:44 +0800, Wen Congyang wrote:
>> On 01/14/2016 06:21 PM, Ian Campbell wrote:
>>> On Tue, 2016-01-12 at 09:40 +0800, Wen Congyang wrote:
 On 01/09/2016 12:27 AM, Ian Campbell wrote:
> On Fri, 2016-01-08 at 14:38 +0800, Wen Congyang wrote:
>> For example: if the secondary host is down, and we fail to send
>> the
>> data
>> to
>> the secondary host. xc_domain_save() returns 0.
>
> Just to be check: On failure in this way xc_domain_save() returns 0
> (i.e.
> success)?

 Yes, it returns 0. I am not sure the return value is right.

>
>>  So in the function
>> libxl__xc_domain_save_done(), rc is 0(the helper program exits
>> normally),
>> and retval is 0(it is xc_domain_save()'s return value). In such
>> case,
>> we
>> just need to complete the stream.
>>
>> Signed-off-by: Wen Congyang 
>> ---
>>  tools/libxl/libxl_stream_write.c | 13 +++--
>>  1 file changed, 11 insertions(+), 2 deletions(-)
>>
>> diff --git a/tools/libxl/libxl_stream_write.c
>> b/tools/libxl/libxl_stream_write.c
>> index 80d9208..82e7719 100644
>> --- a/tools/libxl/libxl_stream_write.c
>> +++ b/tools/libxl/libxl_stream_write.c
>> @@ -354,8 +354,17 @@ void libxl__xc_domain_save_done(libxl__egc
>> *egc,
>> void *dss_void,
>>   * alive, and check_all_finished() may have torn it down
>> around
>> us.
>>   * If the stream is not still alive, we must not continue
>> any
>> work.
>>   */
>> -if (libxl__stream_write_inuse(stream))
>> -write_emulator_xenstore_record(egc, stream);
>> +if (libxl__stream_write_inuse(stream)) {
>> +if (dss->remus)
>> +/*
>> + * For remus, if libxl__xc_domain_save_done()
>> completes,
>> + * there was an error sending data to the secondary.
>> + * Resume the primary ASAP.
>> + */
>> +stream_complete(egc, stream, 0);
>
> Is there an indication to the caller that things have failed in
> this
> way?
> Would that information be of use to the caller?

 For remus, when we come here, something is wrong regardless of the
 return
 value.
>>>
>>> But does the caller know this? Can it tell.
>>>

>
> Or does the called infer this has happened because
> otherwise libxl_domain_remus_start is not supposed to return?

 Yes, libxl_domain_remus_start() should not return unless somethins is
 wrong.
>>>
>>> This really ought to be documented somewhere.
>>
>> libxl_domain_remus_start():
>> /* Point of no return */
>> libxl__remus_setup(egc, dss);
>> return AO_INPROGRESS;
> 
> This is (obviously) not documentation.

OK, I will update the comment.

Thanks
Wen Congyang

> 
> 
> 
> .
> 




___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 03/28] libxl: Provide libxl__dm_support_*

2016-01-15 Thread Ian Campbell
On Thu, 2016-01-14 at 11:31 -0700, Jim Fehlig wrote:
> Ian Campbell wrote:
> > On Mon, 2016-01-11 at 10:00 -0700, Jim Fehlig wrote:
> > > On 01/07/2016 10:13 AM, Ian Campbell wrote:
> > > > On Tue, 2015-12-22 at 18:44 +, Ian Jackson wrote:
> > > > > This allows code elsewhere in libxl to find out what options a
> > > > > device
> > > > > model executable supports.  This is done by searching the usage
> > > > > message for fixed strings.
> > > > Has anyone (ever, not necessarily a Xen person nor in this context)
> > > > approached upstream QEMU about a machine readable output of some
> > > > sort?
> > > > 
> > > > I know libvirt does something similar to this, but they want to
> > > > support
> > > > older versions, whereas we at least have the luxury of not caring
> > > > about
> > > > versions before the point this code lands.
> > > Since qemu 1.2.0, libvirt has been using the various QMP commands to
> > > probe for
> > > qemu capabilities, instead of parsing help output.
> > 
> > As in it spawns a qemu specifically to ask the questions and then kills
> > it
> > and starts what it needs _or_ it starts the qemu with minimal command
> > line
> > cfg and then dynamically pokes in the full config via qmp?
> 
> The latter.

From the description I think you meant former?

>  When the qemu driver loads, it starts qemu, pokes it for
> capabilities via qmp, caches what it finds, then terminates it. The cached
> capabilities are used until the associated qemu binary is changed/updated.
> 
> If interested in peeking at the code, see virQEMUCapsInitQMP() and the 
> functions
> it calls in $libvirt_src/src/qemu/qemu_capabilities.c. E.g.
> 
> virQEMUCapsInitQMP()
>   -> virQEMUCapsInitQMPMonitor()
> -> virQEMUCapsInitQMPBasic()

Thanks.

When I spoke to Ian J yesterday we decided doing this properly (via QMP as
above) would be nice it was out of scope for this series for now. It would
make a nice future bit of cleanup though for sure.

Ian.

> 
> Regards,
> Jim
> 
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [v3,11/41] mips: reuse asm-generic/barrier.h

2016-01-15 Thread Will Deacon
Paul,

On Thu, Jan 14, 2016 at 02:20:46PM -0800, Paul E. McKenney wrote:
> On Thu, Jan 14, 2016 at 01:24:34PM -0800, Leonid Yegoshin wrote:
> > It is not so simple, I mean "local ordering for address and data
> > dependencies". Local ordering is NOT enough. It happens that current
> > MIPS R6 doesn't require in your example smp_read_barrier_depends()
> > but in discussion it comes out that it may not. Because without
> > smp_read_barrier_depends() your example can be a part of Will's
> > WRC+addr+addr and we found some design which easily can bump into
> > this test. And that design actually performs "local ordering for
> > address and data dependencies" too.
> 
> As noted in another email in this thread, I do not believe that
> WRC+addr+addr needs to be prohibited.  Sounds like Will and I need to
> get our story straight, though.

I think you figured this out while I was sleeping, but just to confirm:

 1. The MIPS64 ISA doc [1] talks about SYNC in a way that applies only
to memory accesses appearing in *program-order* before the SYNC

 2. We need WRC+sync+addr to work, which means that the SYNC in P1 must
also capture the store in P0 as being "before" the barrier. Leonid
reckons it works, but his explanation [2] focussed on the address
dependency in P2 as to why this works. If that is the case (i.e.
address dependency provides global transitivity), then WRC+addr+addr
should also work (even though its not required).

 3. It seems that WRC+addr+addr doesn't work, so I'm still suspicious
about WRC+sync+addr, because neither the architecture document or
Leonid's explanation tell me that it should be forbidden.

Will

[1] https://imgtec.com/?do-download=4302
[2] http://lkml.kernel.org/r/569565da.2010...@imgtec.com (scroll to the end)

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [SeaBIOS] [SEABIOS] Plans for either 1.9.1 or 1.10.0?

2016-01-15 Thread Gerd Hoffmann
  Hi,

> It's been suggested (by you :)) that
> 76327b9f32a009245c215f4a3c5d58a01b5310ae be cherry-picked into 1.9.1 as
> well, perhaps.

Yes, right.  Thanks for the reminder.  Picked up.

cheers,
  Gerd


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH] libxl: create 'drivers', 'feature' and 'attr' xenstore paths

2016-01-15 Thread Paul Durrant
My recent patch series 'docs: Document xenstore paths' included 3
patches documenting new xenstore paths to allow PV drivers/agents in
guests to advertise version information, significant features and
attributes (such as assigned IP addresses).

This patch adds the necessary code to libxl to create these paths in
xenstore when a domain is created.

Signed-off-by: Paul Durrant 
Cc: Ian Jackson 
Cc: Stefano Stabellini 
Cc: Ian Campbell 
Cc: Wei Liu 
---
 tools/libxl/libxl_create.c | 9 +
 1 file changed, 9 insertions(+)

diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index 261816a..e491d83 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -628,6 +628,15 @@ retry_transaction:
 libxl__xs_mknod(gc, t,
 GCSPRINTF("%s/data", dom_path),
 rwperm, ARRAY_SIZE(rwperm));
+libxl__xs_mknod(gc, t,
+GCSPRINTF("%s/drivers", dom_path),
+rwperm, ARRAY_SIZE(rwperm));
+libxl__xs_mknod(gc, t,
+GCSPRINTF("%s/feature", dom_path),
+rwperm, ARRAY_SIZE(rwperm));
+libxl__xs_mknod(gc, t,
+GCSPRINTF("%s/attr", dom_path),
+rwperm, ARRAY_SIZE(rwperm));
 
 if (libxl_defbool_val(info->driver_domain)) {
 /*
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v3] x86/p2m: use large pages for MMIO mappings

2016-01-15 Thread Ian Campbell
On Thu, 2016-01-14 at 03:04 -0700, Jan Beulich wrote:
> - ARM side unimplemented (and hence libxc for now made cope with both
>   models),

So, one model is the one described in the commit message:

> - zero (success, everything done)
> - positive (success, this many done, more to do: re-invoke)
> - negative (error)

What is the other one? I'd expect ARM to already implement a subset of this
(i.e. 0 or negative, perhaps with a subset of the possible errno values), which 
I'd then expect libxc to just cope with without it constituting a second model.

IOW I don't think there should be (or indeed is) any special casing of ARM
vs x86 here or one model vs another, just a case of one arch only using a
subset of the expressibility of the interface.

What have I missed?


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 1/2] libxl: fix _SC_GETPW_R_SIZE_MAX usage

2016-01-15 Thread Ian Campbell
On Wed, 2016-01-13 at 10:08 +0100, Roger Pau Monné wrote:
> El 13/01/16 a les 4.19, Doug Goldstein ha escrit:
> > On 1/12/16 7:14 AM, Roger Pau Monne wrote:
> > > According to the FreeBSD sysconf man page [0] if the variable is
> > > associated
> > > with functionality that is not supported, -1 is returned and errno is
> > > not
> > > modified. Modify libxl__dm_runas_helper so it's able to correctly
> > > deal with this situation by setting the initial buffer value to 2048.
> > > 
> > > [0] https://www.freebsd.org/cgi/man.cgi?query=sysconf

http://pubs.opengroup.org/onlinepubs/9699919799/functions/sysconf.html (whi
ch IMHO is the more canonical documentation) describes a similar error
behaviour, although for a subtly different case (no limit for the given
variable).

> > > 
> > > Signed-off-by: Roger Pau Monné 
> > > ---
> > >  tools/libxl/libxl_dm.c | 7 +++
> > >  1 file changed, 7 insertions(+)
> > > 
> > > diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
> > > index 0aaefd9..ec8fb51 100644
> > > --- a/tools/libxl/libxl_dm.c
> > > +++ b/tools/libxl/libxl_dm.c
> > > @@ -728,7 +728,14 @@ static int libxl__dm_runas_helper(libxl__gc *gc,
> > > const char *username)
> > >  long buf_size;
> > >  int ret;
> > >  
> > > +errno = 0;
> > >  buf_size = sysconf(_SC_GETPW_R_SIZE_MAX);
> > > +if (buf_size < 0 && errno == 0) {
> > > +buf_size = 2048;
> > > +LOG(DEBUG,
> > > +"sysconf(_SC_GETPW_R_SIZE_MAX) is not supported, using a buffer size of 
> > > %ld",

"... an initial buffer size of ..."

> > > +buf_size);
> > > +}
> > >  if (buf_size < 0) {
> > >  LOGE(ERROR, "sysconf(_SC_GETPW_R_SIZE_MAX) returned error
> > > %ld",
> > >  buf_size);
> > > 
> > 
> > So on Linux the behavior is somewhat similar [1]. But I took a peek at
> > the libvirt code for doing the similar thing and I notice that they
> > just
> > default if the value is returned as less than 0 [2]. Reading both man
> > pages it seems like that would be the better bet instead of failing how
> > the current code is. Your fix probably makes it fail less but it could
> > still error out senselessly. Just a suggestion for an improvement
> > overall.
> 
> Ack, it doesn't make much sense to error out if we cannot find an
> initial value for the buffer, the code below is able to adapt and expand
> the buffer as needed anyway. I'm going to resend with that fixed.

Sounds good.

Don't forget to CC all the tools maintainers next time.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [seabios baseline-only test] 38639: tolerable FAIL

2016-01-15 Thread Platform Team regression test user
This run is configured for baseline tests only.

flight 38639 seabios real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/38639/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-qemuu-nested-intel 13 xen-boot/l1 fail like 38631
 test-amd64-amd64-xl-qemuu-winxpsp3  9 windows-install  fail like 38631

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop  fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail never pass

version targeted for testing:
 seabios  3e8d75f3bef0f36a807303d58523ef5eba4a386f
baseline version:
 seabios  44250252eeaefd5e81bae2f73639bd323682217b

Last test of basis38631  2016-01-13 07:57:35 Z2 days
Testing same since38639  2016-01-15 01:55:58 Z0 days1 attempts


People who touched revisions under test:
  Kevin O'Connor 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmpass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsmpass
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm pass
 test-amd64-amd64-qemuu-nested-amdfail
 test-amd64-i386-qemuu-rhel6hvm-amd   pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-i386-xl-qemuu-debianhvm-amd64 pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass
 test-amd64-amd64-xl-qemuu-win7-amd64 fail
 test-amd64-i386-xl-qemuu-win7-amd64  fail
 test-amd64-amd64-qemuu-nested-intel  fail
 test-amd64-i386-qemuu-rhel6hvm-intel pass
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 pass
 test-amd64-amd64-xl-qemuu-winxpsp3   fail
 test-amd64-i386-xl-qemuu-winxpsp3pass



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xs.citrite.net/~osstest/testlogs/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


commit 3e8d75f3bef0f36a807303d58523ef5eba4a386f
Author: Kevin O'Connor 
Date:   Tue Jan 12 13:36:50 2016 -0500

coreboot: Check for unaligned cbfs header

If the CBFS header is invalid and points to 0x it could cause
SeaBIOS to read past the 4GB boundary and cause an exception.  Check
the alignment of the header pointer before attempting to access fields
within the header.

Reported-by: "Alex G." 
Signed-off-by: Kevin O'Connor 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [v3,11/41] mips: reuse asm-generic/barrier.h

2016-01-15 Thread Will Deacon
On Thu, Jan 14, 2016 at 02:55:10PM -0800, Paul E. McKenney wrote:
> On Thu, Jan 14, 2016 at 01:36:50PM -0800, Leonid Yegoshin wrote:
> > On 01/14/2016 01:29 PM, Paul E. McKenney wrote:
> > >
> > >>On 01/14/2016 12:34 PM, Paul E. McKenney wrote:
> > >>>
> > >>>The WRC+addr+addr is OK because data dependencies are not required to be
> > >>>transitive, in other words, they are not required to flow from one CPU to
> > >>>another without the help of an explicit memory barrier.
> > >>I don't see any reliable way to fit WRC+addr+addr into "DATA
> > >>DEPENDENCY BARRIERS" section recommendation to have data dependency
> > >>barrier between read of a shared pointer/index and read the shared
> > >>data based on that pointer. If you have this two reads, it doesn't
> > >>matter the rest of scenario, you should put the dependency barrier
> > >>in code anyway. If you don't do it in WRC+addr+addr scenario then
> > >>after years it can be easily changed to different scenario which
> > >>fits some of scenario in "DATA DEPENDENCY BARRIERS" section and
> > >>fails.
> > >The trick is that lockless_dereference() contains an
> > >smp_read_barrier_depends():
> > >
> > >#define lockless_dereference(p) \
> > >({ \
> > >   typeof(p) _p1 = READ_ONCE(p); \
> > >   smp_read_barrier_depends(); /* Dependency order vs. p above. */ \
> > >   (_p1); \
> > >})
> > >
> > >Or am I missing your point?
> > 
> > WRC+addr+addr has no any barrier. lockless_dereference() has a
> > barrier. I don't see a common points between this and that in your
> > answer, sorry.
> 
> Me, I am wondering what WRC+addr+addr has to do with anything at all.

See my earlier reply [1] (but also, your WRC Linux example looks more
like a variant on WWC and I couldn't really follow it).

> 
> 
> OK, so it looks like Will was asking not about WRC+addr+addr, but instead
> about WRC+sync+addr.  This would drop an smp_mb() into cpu2() in my
> earlier example, which needs to provide ordering.
> 
> I am guessing that the manual's "Older instructions which must be globally
> performed when the SYNC instruction completes" provides the equivalent
> of ARM/Power A-cumulativity, which can be thought of as transitivity
> backwards in time. 

I couldn't make that leap. In particular, the manual's "Detailed
Description" sections explicitly refer to program-order:

  Every synchronizable specified memory instruction (loads or stores or
  both) that occurs in the instruction stream before the SYNC
  instruction must reach a stage in the load/store datapath after which
  no instruction re-ordering is possible before any synchronizable
  specified memory instruction which occurs after the SYNC instruction
  in the instruction stream reaches the same stage in the load/store
  datapath.

Will

[1] 
http://lists.infradead.org/pipermail/linux-arm-kernel/2016-January/399765.html

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 2/2] libxl: fix UUID usage on FreeBSD

2016-01-15 Thread Ian Campbell
On Tue, 2016-01-12 at 14:14 +0100, Roger Pau Monne wrote:
> libxl makes the assumtion that libxl_uuid == uuid_t,

("assumption")

>  and that uuid_t can be
> freely used as a byte array. This is not true on FreeBSD (and NetBSD
> too, not sure about other BSD UUID implementations), where the internals
> of
> uuid don't match what libxl expects as a byte array because of endianness
> issues.
> 
> Fix this by converting the libxl_uuid type to a struct with an internal
> uuid_t field and a byte-array. Also introduce a new function that should
> be
> used in order to load a byte array into a uuid_t struct.

Do we really need to keep both the uuid_t and the byte-array representation
around? It looks to me as if we only really need the byte-array form, which
might then involve changing various uses of uuid_* internally to just be
mem*.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCHv1 net] xen-netback: use skb to determine number of required guest Rx requests

2016-01-15 Thread David Vrabel
On 14/01/16 21:54, David Miller wrote:
> From: David Vrabel 
> Date: Thu, 14 Jan 2016 15:18:30 +
> 
>> -needed = xenvif_rx_ring_slots_needed(queue->vif);
>> +skb = skb_peek(&queue->rx_queue);
>> +if (!skb)
>> +return false;
>> +
>> +needed = DIV_ROUND_UP(skb->len, XEN_PAGE_SIZE);
>> +if (skb_is_gso(skb))
>> +needed++;
> 
> If I am not mistaken, we moved away from this kind of test exactly because
> it is inaccurate and may under-estimate the needs.
> 
> It is possible for an N byte SKB to require N segments.  Therefore, the:
> 
>   DIV_ROUND_UP(skb->len, XEN_PAGE_SIZE);
> 
> calculation doesn't cut it.

After 1650d5455bd2dc6b5ee134bd6fc1a3236c266b5b (xen-netback: always
fully coalesce guest Rx packets) we always fully pack a packet into its
guest Rx slots.  Each slot has space for XEN_PAGE_SIZE bytes so this
calculation for the number of slots is correct.

Shall I resend with a more description changelog?

David

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v3] x86/p2m: use large pages for MMIO mappings

2016-01-15 Thread Jan Beulich
>>> On 15.01.16 at 11:09,  wrote:
> On Thu, 2016-01-14 at 03:04 -0700, Jan Beulich wrote:
>> - ARM side unimplemented (and hence libxc for now made cope with both
>>   models),
> 
> So, one model is the one described in the commit message:
> 
>> - zero (success, everything done)
>> - positive (success, this many done, more to do: re-invoke)
>> - negative (error)
> 
> What is the other one? I'd expect ARM to already implement a subset of this
> (i.e. 0 or negative, perhaps with a subset of the possible errno values), 
> which I'd then expect libxc to just cope with without it constituting a 
> second model.

Well, first of all ARM doesn't get switched away from the current
model (yet), i.e. returning -E2BIG out of do_domctl(). And then
the difference between what the patch implements and what the
non-commit message comment describes is how errors get handled:
The patch makes a negative error value returned upon error, with
the caller having no way to tell at what point the error occurred
(and with a best effort undo in the case of "map"). The described
alternative would return the number of succeeded entries unless
an error occurred on the very first MFN, without any attempt to
undo the part that was done successfully. I.e. it would leave it
to the caller to decide what to do, and whether/when to roll back.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2 1/2] libxl: fix _SC_GETPW_R_SIZE_MAX usage

2016-01-15 Thread Ian Campbell
On Thu, 2016-01-14 at 16:06 +0100, Roger Pau Monne wrote:
> If sysconf(_SC_GETPW_R_SIZE_MAX) fails for any reason just use an initial
> buffer size of 2048. This is not a critical failure, and the code that
> makes use of this buffer is able to expand it later if required.
> 
> Signed-off-by: Roger Pau Monné 
> ---
> Cc: Ian Jackson 
> Cc: Stefano Stabellini 
> Cc: Ian Campbell 
> Cc: Wei Liu 

Acked-by: Ian Campbell 

Well done on your prescience on fixing all my comments on v1 before you'd
even seen them!

Applied.

> ---
> Changes since v1:
>  - Use a default buffer size if sysconf(_SC_GETPW_R_SIZE_MAX) fails for
> any
>    reason, this should not be a critical failure.
> ---
>  tools/libxl/libxl_dm.c | 7 ---
>  1 file changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
> index 0aaefd9..a088d71 100644
> --- a/tools/libxl/libxl_dm.c
> +++ b/tools/libxl/libxl_dm.c
> @@ -730,9 +730,10 @@ static int libxl__dm_runas_helper(libxl__gc *gc,
> const char *username)
>  
>  buf_size = sysconf(_SC_GETPW_R_SIZE_MAX);
>  if (buf_size < 0) {
> -LOGE(ERROR, "sysconf(_SC_GETPW_R_SIZE_MAX) returned error %ld",
> -buf_size);
> -return ERROR_FAIL;
> +buf_size = 2048;
> +LOG(DEBUG,
> +"sysconf(_SC_GETPW_R_SIZE_MAX) failed, setting the initial buffer size
> to %ld",
> +buf_size);
>  }
>  
>  while (1) {

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] tools: fix python install with "xentoollog"

2016-01-15 Thread Ian Campbell
On Tue, 2016-01-12 at 22:16 -0500, Konrad Rzeszutek Wilk wrote:
> commit 5d3dc8671521ea4a4f753e77d3e7fb3a3a6f5f80
> "tools: Refactor "xentoollog" into its own library"
> with older python versions (2.6.4) will fail to
> the build if attempted to be done twice (which
> happens due to pygrub dependencies).
> 
> make -C python DESTDIR=/tmp
> make -C python DESTDIR=/tmp
> 
> The second one will fail with:
> error: -Wl, -rpath-link=../../tools/libs/toollog: No such file or
> directory
> 
> even thought the directory is there (with the libs).

"though"

> 
> Andrew pointed out that the linker additions should be in
> the "extra_link_args" rather than "depends".
> 
> And true enough - with that modification it builds.
> 
> CC: Ian Campbell 
> CC: Ian Jackson 
> CC: Wei Liu 
> CC: Boris Ostrovsky 
> Suggested-by: Andrew Cooper 
> Signed-off-by: Konrad Rzeszutek Wilk 

Acked + applied, with typo fixed.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [BUG] Assertion '(sp == 0) || (peoi[sp-1].vector < vector)' failed at irq.c:1163

2016-01-15 Thread Håkon Alstadheim

This is just a preliminary report, mostly just for the record.

I will report again if this keeps happening after 4.7 is out, or upon 
request. Anyone working on this, please mail me and request more 
information. I have available logs from dom0 boot (I dump dmesg and xl 
dmesg to disk after every boot, and log dom0 serial console to disk). I 
will send boot logs if requested. I will turn on maximum verbosity and 
provide all output. My serial console is very slow, so I can not keep 
running at max verbosity all the time.


At the end of this mail there is "xl info" and output from dom0 serial 
console.


CPUINFO:
vendor_id: GenuineIntel
cpu family: 6
model: 63
model name: Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz

# smbios-sys-info
Libsmbios version:  2.2.28
Product Name:   Z10PE-D8 WS
Vendor: ASUSTeK COMPUTER INC.
BIOS Version:   3101

Dom0 OS:
Linux gentoo 4.1.12-gentoo #1 SMP Sat Jan 2 09:36:31 CET 2016 x86_64 
Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz GenuineIntel GNU/Linux.
Kernel is gentoo-sources, with experimental use-flag. Cpu type set to 
Haswell. Issue also happened without experimental.

# cat /proc/cmdline
placeholder root=LABEL=ssdroot ro 
xen-pciback.hide=(02:00.*)(08:00.*)(00:1b.*)(81:00.*)(82:00.*)(83:00.*) 
console=hvc0 console=vga domodules domdadm dolvm intel_iommu=on 
earlyprintk=xen usbcore.autosuspend=-1


The system is mostly built with stable packages, xen and xen-tools 
keyworded to ~amd64.


I have been experiencing issues with domains with passed through PCIe 
devices since I first installed xen. Then at version 4.5.x , I'm now at 
4.6.0 with gentoo patches. Crashes SEEM mostly related to this pci pass 
through and interrupts (usb-cards, sound cards).


Recently the system has been more stable, whether it is because I pass 
through as few things as possible, or because of improvements in Xen I 
do not know. I have also taken to building with debug, which leads to 
more abrupt but less mysterious failures. Earlier (w/o debug and under 
xen 4.5 ) stuff would just gradually stop working and end up in total 
hang of everything. So, hey, things are improving :-b


---xl info: 
host   : gentoo
release: 4.1.12-gentoo
version: #1 SMP Sat Jan 2 09:36:31 CET 2016
machine: x86_64
nr_cpus: 24
max_cpu_id : 23
nr_nodes   : 2
cores_per_socket   : 6
threads_per_core   : 2
cpu_mhz: 2394
hw_caps: 
bfebfbff:2c100800::7f00:77fefbff::0021:37ab

virt_caps  : hvm hvm_directio
total_memory   : 65379
free_memory: 20123
sharing_freed_memory   : 0
sharing_used_memory: 0
outstanding_claims : 0
free_cpus  : 0
xen_major  : 4
xen_minor  : 6
xen_extra  : .0
xen_version: 4.6.0
xen_caps   : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 
hvm-3.0-x86_32p hvm-3.0-x86_64

xen_scheduler  : credit
xen_pagesize   : 4096
platform_params: virt_start=0x8000
xen_changeset  :
xen_commandline: ssd-xen-marker console_timestamps=date 
loglvl=warn/warn guest_loglvl=warn/warn iommu=1 
iommu_inclusive_mapping=1 com1=115200,8n1 console=com1 dom0_max_vcpus=4 
dom0_vcpus_pin=1 dom0_mem=8G,max:8G cpufreq=xen:performance,verbose 
tmem=1 sched_smt_power_savings=1 e820-verbose=1 core_parking=power
cc_compiler: x86_64-pc-linux-gnu-gcc (Gentoo 4.9.3 p1.4, 
pie-0.6.4) 4.9.3

cc_compile_by  :
cc_compile_domain  : alstadheim.priv.no
cc_compile_date: Tue Jan 12 22:19:05 CET 2016
xend_config_format : 4
-
 serial console output from time dom0 finished booting until crash: 
(I have more of these) 

(XEN) [2016-01-13 08:46:25] tmem: flushing tmem pools for domid=4
(XEN) [2016-01-13 08:54:27] tmem: flushing tmem pools for domid=5
(XEN) [2016-01-13 09:01:53] tmem: flushing tmem pools for domid=6
(XEN) [2016-01-13 09:04:41] tmem: flushing tmem pools for domid=7
(XEN) [2016-01-13 09:19:46] tmem: flushing tmem pools for domid=8
(XEN) [2016-01-13 09:22:42] tmem: flushing tmem pools for domid=9
(XEN) [2016-01-13 09:40:37] tmem: flushing tmem pools for domid=10
(XEN) [2016-01-13 20:59:46] [VT-D] It's risky to assign :81:00.0 
with shared RMRR at 7db92000 for Dom12.

(XEN) [2016-01-13 22:21:06] tmem: flushing tmem pools for domid=3
(XEN) [2016-01-14 19:31:12] tmem: flushing tmem pools for domid=11
(XEN) [2016-01-15 07:02:56] Assertion '(sp == 0) || (peoi[sp-1].vector < 
vector)' failed at irq.c:1163
(XEN) [2016-01-15 07:02:56] [ Xen-4.6.0  x86_64  debug=y  Not 
tainted ]

(XEN) [2016-01-15 07:02:56] CPU:19
(XEN) [2016-01-15 07:02:56] RIP:e008:[] 
do_IRQ+0x3ca/0x63b

(XEN) [2016-01-15 07:02:56] RFLAGS: 00010046   CONTEXT: hypervisor
(XEN) [2016-01-15 07:02:56] rax

Re: [Xen-devel] [PATCH] Config.mk: Update SEABIOS_UPSTREAM_TAG to 442502

2016-01-15 Thread Ian Campbell
On Thu, 2016-01-14 at 17:56 +0100, Roger Pau Monné wrote:
> 
> > commit a661f70a8aad3fe7b2c828632714785a04b1d96b
> > Author: Ian Campbell 
> > Date:   Thu Jan 14 16:43:42 2016 +
> > 
> > SEABIOS_UPSTREAM_REVISION Update
> > 
> > Signed-off-by: Ian Campbell 
> 
> Acked-by: Roger Pau Monné 

Applied.

> 
> > diff --git a/Config.mk b/Config.mk
> > index 62f8209..d654af8 100644
> > --- a/Config.mk
> > +++ b/Config.mk
> > @@ -255,9 +255,9 @@ MINIOS_UPSTREAM_REVISION ?=
> > d25773c8afa2f4dbbb466116daeb60159ddd22bd
> >  # Thu Dec 3 11:23:25 2015 +
> >  # mini-os: Include libxentoollog with libxc
> >  
> > -SEABIOS_UPSTREAM_REVISION ?= rel-1.9.0
> > -# Tue Nov 17 09:18:44 2015 -0500
> > -# docs: Note v1.9.0 release
> > +SEABIOS_UPSTREAM_REVISION ?= 3403ac4313812752be6e6aac35239ca6888a8cab
> > +# Mon Dec 28 13:50:41 2015 +0100
> > +# build: fix typo in buildversion.py
> >  
> >  ETHERBOOT_NICS ?= rtl8139 8086100e
> >  
> > 
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] gitignore: update stubdom entries

2016-01-15 Thread Ian Campbell
On Thu, 2016-01-14 at 16:17 +, Wei Liu wrote:
> Add stubdom/libs-* to gitignore, then sort stubdom/* entries
> alphabetically.
> 
> Signed-off-by: Wei Liu 

Acked + applied.

I have discarded the previous "gitignore: ignore stubdom/libs-*".


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [BUG] Assertion '(sp == 0) || (peoi[sp-1].vector < vector)' failed at irq.c:1163

2016-01-15 Thread Andrew Cooper
On 15/01/16 10:58, Håkon Alstadheim wrote:
> This is just a preliminary report, mostly just for the record.
>
> I will report again if this keeps happening after 4.7 is out, or upon
> request. Anyone working on this, please mail me and request more
> information. I have available logs from dom0 boot (I dump dmesg and xl
> dmesg to disk after every boot, and log dom0 serial console to disk).
> I will send boot logs if requested. I will turn on maximum verbosity
> and provide all output. My serial console is very slow, so I can not
> keep running at max verbosity all the time.
>
> At the end of this mail there is "xl info" and output from dom0 serial
> console.
>
> CPUINFO:
> vendor_id: GenuineIntel
> cpu family: 6
> model: 63
> model name: Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
>
> # smbios-sys-info
> Libsmbios version:  2.2.28
> Product Name:   Z10PE-D8 WS
> Vendor: ASUSTeK COMPUTER INC.
> BIOS Version:   3101
>
> Dom0 OS:
> Linux gentoo 4.1.12-gentoo #1 SMP Sat Jan 2 09:36:31 CET 2016 x86_64
> Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz GenuineIntel GNU/Linux.
> Kernel is gentoo-sources, with experimental use-flag. Cpu type set to
> Haswell. Issue also happened without experimental.
> # cat /proc/cmdline
> placeholder root=LABEL=ssdroot ro
> xen-pciback.hide=(02:00.*)(08:00.*)(00:1b.*)(81:00.*)(82:00.*)(83:00.*) 
> console=hvc0
> console=vga domodules domdadm dolvm intel_iommu=on earlyprintk=xen
> usbcore.autosuspend=-1
>
> The system is mostly built with stable packages, xen and xen-tools
> keyworded to ~amd64.
>
> I have been experiencing issues with domains with passed through PCIe
> devices since I first installed xen. Then at version 4.5.x , I'm now
> at 4.6.0 with gentoo patches. Crashes SEEM mostly related to this pci
> pass through and interrupts (usb-cards, sound cards).
>
> Recently the system has been more stable, whether it is because I pass
> through as few things as possible, or because of improvements in Xen I
> do not know. I have also taken to building with debug, which leads to
> more abrupt but less mysterious failures. Earlier (w/o debug and under
> xen 4.5 ) stuff would just gradually stop working and end up in total
> hang of everything. So, hey, things are improving :-b

This isn't the first time we have seen this on Haswell processors. Do
you have microcode loading set up?

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [qemu-mainline test] 78136: tolerable FAIL - PUSHED

2016-01-15 Thread osstest service owner
flight 78136 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/78136/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds  9 debian-install   fail   like 78082

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop  fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass

version targeted for testing:
 qemuuf02ccf53693758b65843264e077f90cf295e7d98
baseline version:
 qemuu91728bda76c1bfe49ac680af763154ec51988732

Last test of basis78082  2016-01-14 11:55:24 Z0 days
Testing same since78136  2016-01-15 01:47:34 Z0 days1 attempts


People who touched revisions under test:
  Alex Bennée 
  Amit Shah 
  Bharata B Rao 
  Cao jin 
  Christian Borntraeger 
  Cornelia Huck 
  David Hildenbrand 
  Fam Zheng 
  Halil Pasic 
  Jason J. Herne 
  Markus Armbruster 
  Markus Armbruster 
  Peter Maydell 
  Pierre Morel 
  Shmulik Ladkani 
  Yi Min Zhao 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl  pass
 test-armhf-armhf-xl  pass
 test-amd64-i386-xl   pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmpass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsmpass
 test-amd64-i386-xl-qemuu-d

Re: [Xen-devel] [PATCH v1] xen/arm: Add r1p12 to the list of supported Cadence UARTs

2016-01-15 Thread Jan Beulich
>>> On 14.01.16 at 19:44,  wrote:
> --- a/xen/drivers/char/cadence-uart.c
> +++ b/xen/drivers/char/cadence-uart.c

Considering the wider than expected Cc list - is there an entry missing
from the ARM section in ./MAINTAINERS here?

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCHv1 net] xen-netfront: request Tx response events more often

2016-01-15 Thread David Vrabel
From: Malcolm Crossley 

Trying to batch Tx response events results in poor performance because
this delays freeing the transmitted skbs.

Instead use the standard RING_FINAL_CHECK_FOR_RESPONSES() macro to be
notified once the next Tx response is placed on the ring.

Signed-off-by: Malcolm Crossley 
Signed-off-by: David Vrabel 
---
 drivers/net/xen-netfront.c | 15 +++
 1 file changed, 3 insertions(+), 12 deletions(-)

diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c
index d6abf19..9cb45be 100644
--- a/drivers/net/xen-netfront.c
+++ b/drivers/net/xen-netfront.c
@@ -364,6 +364,7 @@ static void xennet_tx_buf_gc(struct netfront_queue *queue)
RING_IDX cons, prod;
unsigned short id;
struct sk_buff *skb;
+   int more_to_do;
 
BUG_ON(!netif_carrier_ok(queue->info->netdev));
 
@@ -398,18 +399,8 @@ static void xennet_tx_buf_gc(struct netfront_queue *queue)
 
queue->tx.rsp_cons = prod;
 
-   /*
-* Set a new event, then check for race with update of tx_cons.
-* Note that it is essential to schedule a callback, no matter
-* how few buffers are pending. Even if there is space in the
-* transmit ring, higher layers may be blocked because too much
-* data is outstanding: in such cases notification from Xen is
-* likely to be the only kick that we'll get.
-*/
-   queue->tx.sring->rsp_event =
-   prod + ((queue->tx.sring->req_prod - prod) >> 1) + 1;
-   mb();   /* update shared area */
-   } while ((cons == prod) && (prod != queue->tx.sring->rsp_prod));
+   RING_FINAL_CHECK_FOR_RESPONSES(&queue->tx, more_to_do);
+   } while (more_to_do);
 
xennet_maybe_wake_tx(queue);
 }
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v1 1/1] MAINTAINERS: Add xen/drivers/char/cadence-uart.c

2016-01-15 Thread Edgar E. Iglesias
From: "Edgar E. Iglesias" 

Add xen/drivers/char/cadence-uart.c to the ARM section.

Signed-off-by: Edgar E. Iglesias 
---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 09fe823..0e8af01 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -133,6 +133,7 @@ S:  Supported
 L: xen-devel@lists.xen.org
 F: xen/arch/arm/
 F: xen/include/asm-arm/
+F: xen/drivers/char/cadence-uart.c
 F: xen/drivers/char/dt-uart.c
 F: xen/drivers/char/exynos4210-uart.c
 F: xen/drivers/char/omap-uart.c
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v1] xen/arm: Add r1p12 to the list of supported Cadence UARTs

2016-01-15 Thread Edgar E. Iglesias
On Fri, Jan 15, 2016 at 04:20:18AM -0700, Jan Beulich wrote:
> >>> On 14.01.16 at 19:44,  wrote:
> > --- a/xen/drivers/char/cadence-uart.c
> > +++ b/xen/drivers/char/cadence-uart.c
> 
> Considering the wider than expected Cc list - is there an entry missing
> from the ARM section in ./MAINTAINERS here?
>

Yes, that seems to be the case. I've just sent a patch to MAINTAINERS to fix 
that.

Cheers,
Edgar

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [linux-mingo-tip-master test] 78146: regressions - FAIL

2016-01-15 Thread osstest service owner
flight 78146 linux-mingo-tip-master real [real]
http://logs.test-lab.xenproject.org/osstest/logs/78146/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-rumpuserxen-i386 10 guest-start   fail REGR. vs. 60684
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 9 debian-hvm-install fail 
REGR. vs. 60684

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-libvirt-vhd  9 debian-di-install fail REGR. vs. 60684
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop   fail blocked in 60684

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-pvh-intel 14 guest-saverestorefail  never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-intel 13 xen-boot/l1 fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stop fail never pass
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop  fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 13 xen-boot/l1   fail never pass

version targeted for testing:
 linuxcee4676c1b4018a4c2f8ce5e0039d401c64ea3d9
baseline version:
 linux69f75ebe3b1d1e636c4ce0a0ee248edacc69cbe0

Last test of basis60684  2015-08-13 04:21:46 Z  155 days
Failing since 60712  2015-08-15 18:33:48 Z  152 days  104 attempts
Testing same since78146  2016-01-15 04:21:35 Z0 days1 attempts

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 build-amd64-rumpuserxen  pass
 build-i386-rumpuserxen   pass
 test-amd64-amd64-xl  pass
 test-amd64-i386-xl   pass
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsmpass
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmpass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsmpass
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm pass
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsmpass
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm fail
 test-amd64-amd64-libvirt-xsm pass
 test-amd64-i386-libvirt-xsm  pass
 test-amd64-amd64-xl-xsm  pass
 test-amd64-i386-xl-xsm   pass
 test-amd64-amd64-qemuu-nested-amdfail
 test-amd64-amd64-xl-pvh-amd  fail
 test-amd64-i386-qemut-rhel6hvm-amd   pass
 test-amd64-i386-qemuu-rhel6hvm-amd   pass
 test-amd64-amd64-xl-qemut-debianhvm-amd64pass
 test-amd64-i386-xl-qemut-debianhvm-amd64 pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-i386-xl-qemuu-debianhvm-amd64 pass
 test-amd64-i386-freebsd10-amd64  pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass
 test-amd64-amd64-rumpuserxen-amd64   pass
 test-amd64-amd64-xl-qemut-win7-amd64 fail
 test-amd64-i386-xl-qemut-win7-amd64  fail
 test-amd64-amd64-xl-qemuu-win7-amd64 fail
 test-amd64-i386-xl-qemuu-win7-amd64  fail
 test-amd64-amd64-xl-credit2

[Xen-devel] [PATCH OSSTEST] Allow longer timeout when creating backing file for a raw disk.

2016-01-15 Thread Ian Campbell
I noticed this dd timiung out when recommissioning the 3 cubietrucks
(picasso, metzinger, gleizes) but looking at the log shows this has
been happening on braque too.

The current code assumes 65MB/s arriving at a timeout of 153s for the
10G file. On arndale-* the logs indicate that it is achieving 95MB/s
and taking 105-107s which results in a warning but not a failure:

   execution took 105 seconds [**>153.846153846154/2**]

In experiments on a local cubietruck I observed it achieving a much
lower throughput of 40MB/s, which seems to be consistent with what
others are seeing:
https://groups.google.com/forum/#!category-topic/cubieboard/troubleshooting/7R4HlCDNCTU

Therefore calculate the timeout assuming a throughput of 20MB/s, in
practice for a 10GB file this will result in a 500s timeout.

Signed-off-by: Ian Campbell 
---
 Osstest/TestSupport.pm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 28ac572..962d773 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -1747,9 +1747,9 @@ sub make_qcow2 ($$$) {
 }
 sub make_raw ($$$) {
 my ($ho, $gho, $disk_mb) = @_;
-# In local tests this reported 130MB/s, so calculate a timeout assuming 
65MB/s.
+# In local tests this reported 40MB/s, so calculate a timeout assuming 
20MB/s.
 target_cmd_root($ho, "dd if=/dev/zero of=$gho->{Rootimg} bs=1MB 
count=${disk_mb}",
-   ${disk_mb} / 65);
+   ${disk_mb} / 20);
 }
 
 sub prepareguest_part_diskimg ($$$) {
-- 
2.6.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v1 1/1] MAINTAINERS: Add xen/drivers/char/cadence-uart.c

2016-01-15 Thread Ian Campbell
On Fri, 2016-01-15 at 13:14 +0100, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" 
> 
> Add xen/drivers/char/cadence-uart.c to the ARM section.
> 
> Signed-off-by: Edgar E. Iglesias 

Acked-by: Ian Campbell 

> ---
>  MAINTAINERS | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 09fe823..0e8af01 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -133,6 +133,7 @@ S:Supported
>  L:   xen-devel@lists.xen.org
>  F:   xen/arch/arm/
>  F:   xen/include/asm-arm/
> +F:   xen/drivers/char/cadence-uart.c
>  F:   xen/drivers/char/dt-uart.c
>  F:   xen/drivers/char/exynos4210-uart.c
>  F:   xen/drivers/char/omap-uart.c

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v1] xen/arm: Add r1p12 to the list of supported Cadence UARTs

2016-01-15 Thread Ian Campbell
On Thu, 2016-01-14 at 19:44 +0100, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" 

Where are the bindings for cdns,uart-r1p12 defined? I don't see it
mentioned in the Documentation/devicetree/bindings/serial/cdns,uart.txt in
mainline Linux.

Does the driver need to differ at all compared with what is needed for
r1p8? Normally if there is no (practical/visible) difference then in DT-
world there is no need for a new compatible string, since the device is
still compatible.

If there are differences in the h/w but they are not relevant to our
particular driven then please say so in the commit log.

Ian.

> 
> Signed-off-by: Edgar E. Iglesias 
> ---
>  xen/drivers/char/cadence-uart.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/xen/drivers/char/cadence-uart.c b/xen/drivers/char/cadence-
> uart.c
> index 933672f..7f90f8d 100644
> --- a/xen/drivers/char/cadence-uart.c
> +++ b/xen/drivers/char/cadence-uart.c
> @@ -206,6 +206,7 @@ static int __init cuart_init(struct dt_device_node
> *dev, const void *data)
>  static const struct dt_device_match cuart_dt_match[] __initconst =
>  {
>  DT_MATCH_COMPATIBLE("cdns,uart-r1p8"),
> +DT_MATCH_COMPATIBLE("cdns,uart-r1p12"),
>  { /* sentinel */ },
>  };
>  

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [BUG] Assertion '(sp == 0) || (peoi[sp-1].vector < vector)' failed at irq.c:1163

2016-01-15 Thread Håkon Alstadheim



On 01/15/2016 12:05 PM, Andrew Cooper wrote:

On 15/01/16 10:58, Håkon Alstadheim wrote:

This is just a preliminary report, mostly just for the record.

I will report again if this keeps happening after 4.7 is out, or upon
request. Anyone working on this, please mail me and request more
information. I have available logs from dom0 boot (I dump dmesg and xl
dmesg to disk after every boot, and log dom0 serial console to disk).
I will send boot logs if requested. I will turn on maximum verbosity
and provide all output. My serial console is very slow, so I can not
keep running at max verbosity all the time.

At the end of this mail there is "xl info" and output from dom0 serial
console.

CPUINFO:
vendor_id: GenuineIntel
cpu family: 6
model: 63
model name: Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz

# smbios-sys-info
Libsmbios version:  2.2.28
Product Name:   Z10PE-D8 WS
Vendor: ASUSTeK COMPUTER INC.
BIOS Version:   3101

Dom0 OS:
Linux gentoo 4.1.12-gentoo #1 SMP Sat Jan 2 09:36:31 CET 2016 x86_64
Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz GenuineIntel GNU/Linux.
Kernel is gentoo-sources, with experimental use-flag. Cpu type set to
Haswell. Issue also happened without experimental.
# cat /proc/cmdline
placeholder root=LABEL=ssdroot ro
xen-pciback.hide=(02:00.*)(08:00.*)(00:1b.*)(81:00.*)(82:00.*)(83:00.*) 
console=hvc0
console=vga domodules domdadm dolvm intel_iommu=on earlyprintk=xen
usbcore.autosuspend=-1

The system is mostly built with stable packages, xen and xen-tools
keyworded to ~amd64.

I have been experiencing issues with domains with passed through PCIe
devices since I first installed xen. Then at version 4.5.x , I'm now
at 4.6.0 with gentoo patches. Crashes SEEM mostly related to this pci
pass through and interrupts (usb-cards, sound cards).

Recently the system has been more stable, whether it is because I pass
through as few things as possible, or because of improvements in Xen I
do not know. I have also taken to building with debug, which leads to
more abrupt but less mysterious failures. Earlier (w/o debug and under
xen 4.5 ) stuff would just gradually stop working and end up in total
hang of everything. So, hey, things are improving :-b

This isn't the first time we have seen this on Haswell processors. Do
you have microcode loading set up?

Not entirely sure to be honest. Is microcode: 0x31 the newest?

I AM running the very latest bios from Asus, but I do not have 
confidence in my microcode loading setup, so I have not had one in place.


Trying now.

Downloading microcode.dat from Intel
Installing iucode_tool, which in its --help states:
  -w, --write-to=fileWrite selected microcodes to a file in binary
 format.  The binary format is suitable to be
 uploaded to the kernel

Ran "iucode_tool microcode.dat -w microcode.bin"

# ls -l micro*
-rwxr-xr-x 1 root root  693248 Jan 15 12:40 microcode.bin
-rwxr-xr-x 1 root root 2081807 Nov  6 04:04 microcode.dat

placed microcode.bin in /boot/microcode.bin

 booted with :
---
xen_commandline: ssd-xen-debug-marker console_timestamps=date 
loglvl=all guest_loglvl=all sync_console iommu=1,verbose,debug 
iommu_inclusive_mapping=1 com1=115200,8n1 console=com1 dom0_max_vcpus=4 
dom0_vcpus_pin=1 dom0_mem=8G,max:8G cpufreq=xen:performance,verbose 
tmem=1 sched_smt_power_savings=1 apic_verbosity=debug e820-verbose=1 
core_parking=power ucode=microcode.bin

---

#cat /proc/cpuinfo | grep micro
says: microcode: 0x31

This is no change from previous boot.
Now: How do I know wheter 0x31 is the newest?

Grepping the console output reveals no reference to ucode or microcode 
other than the Xen command-line.

---
Håkon


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [BUG] Assertion '(sp == 0) || (peoi[sp-1].vector < vector)' failed at irq.c:1163

2016-01-15 Thread Jan Beulich
>>> On 15.01.16 at 13:32,  wrote:
> placed microcode.bin in /boot/microcode.bin
> 
>   booted with :
> ---
> xen_commandline: ssd-xen-debug-marker console_timestamps=date 
> loglvl=all guest_loglvl=all sync_console iommu=1,verbose,debug 
> iommu_inclusive_mapping=1 com1=115200,8n1 console=com1 dom0_max_vcpus=4 
> dom0_vcpus_pin=1 dom0_mem=8G,max:8G cpufreq=xen:performance,verbose 
> tmem=1 sched_smt_power_savings=1 apic_verbosity=debug e820-verbose=1 
> core_parking=power ucode=microcode.bin
> ---

This can't work - did you look at the command line documentation?
You can't specify a file name here - there's no file system driver
inside the hypervisor, and hence it can't read files (it instead has
to rely on the boot loader bringing those into memory for it).

> #cat /proc/cpuinfo | grep micro
> says: microcode: 0x31
> 
> This is no change from previous boot.
> Now: How do I know wheter 0x31 is the newest?

By checking - for the precise model and stepping of your CPU(s) -
the information in the blob (which admittedly is a little cumbersome,
but without knowing model and stepping I also can't try to help).

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [BUG] Assertion '(sp == 0) || (peoi[sp-1].vector < vector)' failed at irq.c:1163

2016-01-15 Thread Håkon Alstadheim



On 01/15/2016 01:42 PM, Jan Beulich wrote:

On 15.01.16 at 13:32,  wrote:

placed microcode.bin in /boot/microcode.bin

   booted with :
---
xen_commandline: ssd-xen-debug-marker console_timestamps=date
loglvl=all guest_loglvl=all sync_console iommu=1,verbose,debug
iommu_inclusive_mapping=1 com1=115200,8n1 console=com1 dom0_max_vcpus=4
dom0_vcpus_pin=1 dom0_mem=8G,max:8G cpufreq=xen:performance,verbose
tmem=1 sched_smt_power_savings=1 apic_verbosity=debug e820-verbose=1
core_parking=power ucode=microcode.bin
---

This can't work - did you look at the command line documentation?
You can't specify a file name here - there's no file system driver
inside the hypervisor, and hence it can't read files (it instead has
to rely on the boot loader bringing those into memory for it).
Get with the times :-) . Under EFI it most definitely wants a file-name. 
Not entirely sure about the file FORMAT though.


From xen-command-line.html
 "Note further that use of this option has an unspecified effect when 
used with xen.efi (there the concept of modules doesn't exist, and the 
blob gets specified via the ucode= config file/section entry; 
see EFI configuration file description).


From efi.html

 "ucode=

Specifies a CPU microcode blob to load. (x86 only)



#cat /proc/cpuinfo | grep micro
says: microcode: 0x31

This is no change from previous boot.
Now: How do I know wheter 0x31 is the newest?

By checking - for the precise model and stepping of your CPU(s) -
the information in the blob (which admittedly is a little cumbersome,
but without knowing model and stepping I also can't try to help).

Jan





___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [xen-unstable-smoke test] 78168: tolerable all pass - PUSHED

2016-01-15 Thread osstest service owner
flight 78168 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/78168/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  2f3b2f4aa242759826bbc61513ea06b2d124c9a7
baseline version:
 xen  7167222b15dc661ff0b44cc93d558f6bb4ff6492

Last test of basis78074  2016-01-14 10:01:37 Z1 days
Testing same since78168  2016-01-15 11:02:27 Z0 days1 attempts


People who touched revisions under test:
  Ian Campbell 
  Konrad Rzeszutek Wilk 
  Roger Pau Monne 
  Roger Pau Monné 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=xen-unstable-smoke
+ revision=2f3b2f4aa242759826bbc61513ea06b2d124c9a7
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push xen-unstable-smoke 
2f3b2f4aa242759826bbc61513ea06b2d124c9a7
+ branch=xen-unstable-smoke
+ revision=2f3b2f4aa242759826bbc61513ea06b2d124c9a7
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=xen
+ xenbranch=xen-unstable-smoke
+ qemuubranch=qemu-upstream-unstable
+ '[' xxen = xlinux ']'
+ linuxbranch=
+ '[' xqemu-upstream-unstable = x ']'
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable-smoke
+ prevxenbranch=xen-unstable
+ '[' x2f3b2f4aa242759826bbc61513ea06b2d124c9a7 = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/rumpuser-xen.git
++ : git
++ : git://xenbits.xen.org/rumpuser-xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/rumpuser-xen.git
+++ besteffort_repo https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ cached_repo https://github.com/rumpkernel/rumpkernel-netbsd-src 
'[fetch=try]'
+++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ local 'options=[fetch=try]'
 getconfig GitCacheProxy
 perl -e '
use Osstest;
readglobalconfig();
print $c{"GitCacheProxy"} or die $!;
'
+++ local cache=git://cache:9419/
+++ '[' xgit://cache:9419/ '!=' x ']'
+++ e

Re: [Xen-devel] [PATCH v4 3/3] VT-d: Fix vt-d Device-TLB flush timeout issue.

2016-01-15 Thread Jan Beulich
>>> On 23.12.15 at 09:25,  wrote:
> --- a/xen/drivers/passthrough/vtd/qinval.c
> +++ b/xen/drivers/passthrough/vtd/qinval.c
> @@ -190,9 +190,19 @@ static int queue_invalidate_wait(struct iommu *iommu,
>  static int invalidate_sync(struct iommu *iommu)
>  {
>  struct qi_ctrl *qi_ctrl = iommu_qi_ctrl(iommu);
> +int rc;
>  
>  if ( qi_ctrl->qinval_maddr )
> -return queue_invalidate_wait(iommu, 0, 1, 1);
> +{
> +rc = queue_invalidate_wait(iommu, 0, 1, 1);
> +if ( rc )
> +{
> +if ( rc == -ETIMEDOUT )
> +panic("Queue invalidate wait descriptor was timeout.\n");

Unless I'm overlooking something this doesn't replace and existing
panic(), and I think I had indicated quite clearly that this series
shouldn't add any new ones, unless the alternative of crashing
the owning domain is completely unfeasible.

> +panic("Queue invalidate wait descriptor was timeout.\n");
> +return rc;
> +}
> +}
> +
>  return 0;
>  }

Please avoid introducing multiple return points when this can be
trivially avoided.

> @@ -229,6 +239,63 @@ int qinval_device_iotlb(struct iommu *iommu,
>  return 0;
>  }
>  
> +static void dev_invalidate_iotlb_timeout(struct iommu *iommu, u16 did,
> + u16 seg, u8 bus, u8 devfn)
> +{
> +struct domain *d;
> +struct pci_dev *pdev;
> +
> +d = rcu_lock_domain_by_id(iommu->domid_map[did]);
> +ASSERT(d);

Don't you need to acquire some lock in order to safely assert this?
Also note that unused slots hold zero, i.e. there's at least a
theoretical risk of problems here when you don't also look at
iommu->domid_bitmap.

> +for_each_pdev(d, pdev)
> +{
> +if ( (pdev->seg == seg) &&
> + (pdev->bus == bus) &&
> + (pdev->devfn == devfn) )
> +{
> +if ( pdev->domain )

If the device is on the domain's list, can this reasonably be false?
I.e. don't you rather mean ASSERT() here?

> +{
> +list_del(&pdev->domain_list);

This should happen under pcidevs_lock - you need to either acquire
it or ASSERT() it being held.

> +pdev->domain = NULL;
> +}
> +
> +if ( pci_hide_device(bus, devfn) )
> +{
> +printk(XENLOG_ERR
> +   "IOMMU hide device %04x:%02x:%02x error.",
> +   seg, bus, devfn);
> +break;
> +}
> +
> +break;
> +}

Redundant breaks.

> +}
> +
> +if ( !is_hardware_domain(d) )
> +domain_crash(d);

I wonder whether the device hiding shouldn't also be avoided if
the device is owned by hwdom.

> @@ -349,9 +416,18 @@ static int flush_iotlb_qi(
>  queue_invalidate_iotlb(iommu,
> type >> DMA_TLB_FLUSH_GRANU_OFFSET, dr,
> dw, did, size_order, 0, addr);
> +
> +/*
> + * Synchronize with hardware for invalidation request descriptors
> + * submitted before Device-TLB invalidate descriptor.
> + */
> +rc = invalidate_sync(iommu);
> +if ( rc )
> + return rc;
> +
>  if ( flush_dev_iotlb )
>  ret = dev_invalidate_iotlb(iommu, did, addr, size_order, type);
> -rc = invalidate_sync(iommu);
> +
>  if ( !ret )
>  ret = rc;
>  }

This change is because of ...?

> --- a/xen/drivers/passthrough/vtd/x86/ats.c
> +++ b/xen/drivers/passthrough/vtd/x86/ats.c
> @@ -162,6 +162,19 @@ int dev_invalidate_iotlb(struct iommu *iommu, u16 did,
>  return -EOPNOTSUPP;
>  }
>  
> +/*
> + * Synchronize with hardware for Device-TLB invalidate
> + * descriptor.
> + */
> +ret = dev_invalidate_iotlb_sync(iommu, did, pdev->seg,
> +pdev->bus, pdev->devfn);
> +if ( ret )
> +{
> +dprintk(XENLOG_WARNING VTDPREFIX,
> +"Device-TLB flush timeout.\n");

Are you aware that dprintk() compiles to nothing in non-debug builds?
Plus logging the error code and device coordinates might turn out
helpful in such cases. But first of all - if there was a timeout, we'd
make it here only for Dom0. Perhaps the printing should move into
an else to the domain_crash()? And if there was another error, the
message would be outright wrong.

Jan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [BUG] Assertion '(sp == 0) || (peoi[sp-1].vector < vector)' failed at irq.c:1163

2016-01-15 Thread Ian Campbell
On Fri, 2016-01-15 at 13:49 +0100, Håkon Alstadheim wrote:
> 
> On 01/15/2016 01:42 PM, Jan Beulich wrote:
> > > > > On 15.01.16 at 13:32,  wrote:
> > > placed microcode.bin in /boot/microcode.bin
> > > 
> > >    booted with :
> > > ---
> > > xen_commandline: ssd-xen-debug-marker console_timestamps=date
> > > loglvl=all guest_loglvl=all sync_console iommu=1,verbose,debug
> > > iommu_inclusive_mapping=1 com1=115200,8n1 console=com1
> > > dom0_max_vcpus=4
> > > dom0_vcpus_pin=1 dom0_mem=8G,max:8G cpufreq=xen:performance,verbose
> > > tmem=1 sched_smt_power_savings=1 apic_verbosity=debug e820-verbose=1
> > > core_parking=power ucode=microcode.bin
> > > ---
> > This can't work - did you look at the command line documentation?
> > You can't specify a file name here - there's no file system driver
> > inside the hypervisor, and hence it can't read files (it instead has
> > to rely on the boot loader bringing those into memory for it).
> Get with the times :-) . Under EFI it most definitely wants a file-name. 
> Not entirely sure about the file FORMAT though.
> 
>  From xen-command-line.html
>   "Note further that use of this option has an unspecified effect when 
> used with xen.efi (there the concept of modules doesn't exist, and the 
> blob gets specified via the ucode= config file/section entry; 
> see EFI configuration file description).
> 
>  From efi.html
> 
>   "ucode=
> 
>  Specifies a CPU microcode blob to load. (x86 only)

This needs to go in your xen.cfg file (alongside kernel= ramdisk= etc), not
on the xen command line.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v4 1/2] memory-hotplug: add automatic onlining policy for the newly added memory

2016-01-15 Thread Vitaly Kuznetsov
David Rientjes  writes:

> On Thu, 14 Jan 2016, Vitaly Kuznetsov wrote:
>
>> > My suggestion is to just simply document that auto-onlining can add the 
>> > memory but fail to online it and the failure is silent to userspace.  If 
>> > userspace cares, it can check the online status of the added memory blocks 
>> > itself.
>> 
>> The problem is not only that it's silent, but also that
>> /sys/devices/system/memory/*/state will lie as we create all memory
>> blocks in MEM_ONLINE state and from online_pages() error we can't figure
>> out which particular block failed. 'v5' which I sent yesterday is
>> supposed to fix the issue (blocks are onlined with
>> memory_block_change_state() which handles failures.
>> 
>
> Would you mind documenting that in the memory-hotplug.txt as an add-on 
> patch to your v5, which appears ready to go?

Sure,

I'll mention possible failures diring automatic onlining. It seems v5
wasn't picked by Andrew and I also have one nitpick in PATCH 2 to
address so I'll send v6.

Thanks,

-- 
  Vitaly

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [BUG] Assertion '(sp == 0) || (peoi[sp-1].vector < vector)' failed at irq.c:1163

2016-01-15 Thread Håkon Alstadheim



On 01/15/2016 02:09 PM, Ian Campbell wrote:

On Fri, 2016-01-15 at 13:49 +0100, Håkon Alstadheim wrote:

On 01/15/2016 01:42 PM, Jan Beulich wrote:

On 15.01.16 at 13:32,  wrote:

placed microcode.bin in /boot/microcode.bin

booted with :
---
xen_commandline: ssd-xen-debug-marker console_timestamps=date
loglvl=all guest_loglvl=all sync_console iommu=1,verbose,debug
iommu_inclusive_mapping=1 com1=115200,8n1 console=com1
dom0_max_vcpus=4
dom0_vcpus_pin=1 dom0_mem=8G,max:8G cpufreq=xen:performance,verbose
tmem=1 sched_smt_power_savings=1 apic_verbosity=debug e820-verbose=1
core_parking=power ucode=microcode.bin
---

This can't work - did you look at the command line documentation?
You can't specify a file name here - there's no file system driver
inside the hypervisor, and hence it can't read files (it instead has
to rely on the boot loader bringing those into memory for it).

Get with the times :-) . Under EFI it most definitely wants a file-name.
Not entirely sure about the file FORMAT though.

  From xen-command-line.html
   "Note further that use of this option has an unspecified effect when
used with xen.efi (there the concept of modules doesn't exist, and the
blob gets specified via the ucode= config file/section entry;
see EFI configuration file description).

  From efi.html

   "ucode=

  Specifies a CPU microcode blob to load. (x86 only)

This needs to go in your xen.cfg file (alongside kernel= ramdisk= etc), not
on the xen command line.

Ian.



Ahh (face + palm) .  It dawned on me right after I sent my previous. Now 
I DO get some acknowledgement of microcode.bin in the console-log, but 
/proc/cpuinfo still reports microcode: 0x31, so it seems stale 
microcode is not the issue :-/




___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH XEN v8 12/29] tools/libs/foreignmemory: provide xenforeignmemory_unmap.

2016-01-15 Thread Ian Campbell
And require it be used instead of direct munmap.

This will allow e.g. Valgrind hooks to help track incorrect use of
foreign mappings.

Switch all uses of xenforeignmemory_map to use
xenforeignmemory_unmap, not that foreign mappings via the libxc compat
xc_map_foreign_* interface will not take advantage of this and will
need converting.

Signed-off-by: Ian Campbell 
Acked-by: Wei Liu 
Acked-by: Ian Jackson 
---
v4: xenforeignmemory_unmap takes pages not bytes, adjust callers.
v6: Document error behaviour
Use an osdep layer, although there is no current need for common
code on _unmap there is for _map so use this indirection for
consistency.
---
 tools/libs/foreignmemory/freebsd.c  |  6 ++
 tools/libs/foreignmemory/include/xenforeignmemory.h | 11 +--
 tools/libs/foreignmemory/libxenforeignmemory.map|  1 +
 tools/libs/foreignmemory/linux.c|  6 ++
 tools/libs/foreignmemory/minios.c   |  6 ++
 tools/libs/foreignmemory/netbsd.c   |  6 ++
 tools/libs/foreignmemory/solaris.c  |  6 ++
 tools/libxc/xc_sr_restore.c |  2 +-
 tools/libxc/xc_sr_save.c|  2 +-
 tools/libxc/xc_vm_event.c   |  2 +-
 10 files changed, 43 insertions(+), 5 deletions(-)

diff --git a/tools/libs/foreignmemory/freebsd.c 
b/tools/libs/foreignmemory/freebsd.c
index 4b2815d..3c3ad09 100644
--- a/tools/libs/foreignmemory/freebsd.c
+++ b/tools/libs/foreignmemory/freebsd.c
@@ -118,6 +118,12 @@ void *osdep_xenforeignmemory_map(xenforeignmemory_handle 
*fmem,
 return addr;
 }
 
+int osdep_xenforeignmemory_unmap(xenforeignmemory_handle *fmem,
+ void *addr, unsigned int num)
+{
+   return munmap(addr, num << PAGE_SHIFT);
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/foreignmemory/include/xenforeignmemory.h 
b/tools/libs/foreignmemory/include/xenforeignmemory.h
index 5b3fa7a..b783af3 100644
--- a/tools/libs/foreignmemory/include/xenforeignmemory.h
+++ b/tools/libs/foreignmemory/include/xenforeignmemory.h
@@ -44,8 +44,8 @@ int xenforeignmemory_close(xenforeignmemory_handle *fmem);
 
 /*
  * Maps a range within one domain to a local address range.  Mappings
- * should be unmapped with munmap and should follow the same rules as mmap
- * regarding page alignment.
+ * must be unmapped with xenforeignmemory_unmap and should follow the
+ * same rules as mmap regarding page alignment.
  *
  * prot is as for mmap(2).
  *
@@ -57,6 +57,13 @@ int xenforeignmemory_close(xenforeignmemory_handle *fmem);
 void *xenforeignmemory_map(xenforeignmemory_handle *fmem, uint32_t dom,
int prot, const xen_pfn_t *arr, int *err,
unsigned int num);
+/*
+ * Unmap a mapping previous created with xenforeignmemory_map().
+ *
+ * Returns 0 on success on failure sets errno and returns -1.
+ */
+int xenforeignmemory_unmap(xenforeignmemory_handle *fmem,
+   void *addr, unsigned int num);
 
 #endif
 
diff --git a/tools/libs/foreignmemory/libxenforeignmemory.map 
b/tools/libs/foreignmemory/libxenforeignmemory.map
index 11f0d2b..df206b3 100644
--- a/tools/libs/foreignmemory/libxenforeignmemory.map
+++ b/tools/libs/foreignmemory/libxenforeignmemory.map
@@ -3,5 +3,6 @@ VERS_1.0 {
xenforeignmemory_open;
xenforeignmemory_close;
xenforeignmemory_map;
+   xenforeignmemory_unmap;
local: *; /* Do not expose anything by default */
 };
diff --git a/tools/libs/foreignmemory/linux.c b/tools/libs/foreignmemory/linux.c
index b403ca7..40ecd41 100644
--- a/tools/libs/foreignmemory/linux.c
+++ b/tools/libs/foreignmemory/linux.c
@@ -282,6 +282,12 @@ void *osdep_xenforeignmemory_map(xenforeignmemory_handle 
*fmem,
 return addr;
 }
 
+int osdep_xenforeignmemory_unmap(xenforeignmemory_handle *fmem,
+ void *addr, unsigned int num)
+{
+return munmap(addr, (unsigned long)num << PAGE_SHIFT);
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/foreignmemory/minios.c 
b/tools/libs/foreignmemory/minios.c
index a542e3d..5fcb861 100644
--- a/tools/libs/foreignmemory/minios.c
+++ b/tools/libs/foreignmemory/minios.c
@@ -51,6 +51,12 @@ void *osdep_xenforeignmemory_map(xenforeignmemory_handle 
*fmem,
 return map_frames_ex(arr, num, 1, 0, 1, dom, err, pt_prot);
 }
 
+int osdep_xenforeignmemory_unmap(xenforeignmemory_handle *fmem,
+ void *addr, unsigned int num)
+{
+   return munmap(addr, num << PAGE_SHIFT);
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/foreignmemory/netbsd.c 
b/tools/libs/foreignmemory/netbsd.c
index 704a096..740c26f 100644
--- a/tools/libs/foreignmemory/netbsd.c
+++ b/tools/libs/foreignmemory/netbsd.c
@@ -94,6 +94,12 @@ void *osdep_map_foreign_batch(xenforeignmem_handle *fmem, 
uint32_t dom,
 
 }

[Xen-devel] [PATCH XEN v8 01/29] tools/libxc: Remove osdep indirection for xc_evtchn

2016-01-15 Thread Ian Campbell
The alternative backend (a xen-api/xapi shim) is no longer around and
so this stuff is now just baggage which is getting in the way of
refactoring libxenctrl.

Note that the intention is to move this into a separate library
shortly.

Nested virt probably suffices for this use case now.

One incorrect instance of using xc_interface where xc_evtchn (in ocaml
stubs) is removed, this used to work because they were typedefs to the
same struct, but is no longer permitted.

Signed-off-by: Ian Campbell 
Acked-by: Wei Liu 
---
 tools/libxc/include/xenctrl.h |  2 +-
 tools/libxc/include/xenctrlosdep.h| 16 -
 tools/libxc/xc_evtchn.c   | 45 --
 tools/libxc/xc_freebsd_osdep.c| 79 
 tools/libxc/xc_linux_osdep.c  | 76 +--
 tools/libxc/xc_minios.c   | 63 ---
 tools/libxc/xc_netbsd.c   | 72 --
 tools/libxc/xc_private.c  | 37 +---
 tools/libxc/xc_private.h  | 17 ++
 tools/libxc/xc_solaris.c  | 71 --
 tools/libxc/xenctrl_osdep_ENOSYS.c| 87 ---
 tools/ocaml/libs/eventchn/xeneventchn_stubs.c |  4 +-
 12 files changed, 180 insertions(+), 389 deletions(-)

diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 01a6dda..881dcd5 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -117,7 +117,7 @@
  */
 
 typedef struct xc_interface_core xc_interface;
-typedef struct xc_interface_core xc_evtchn;
+typedef struct xenevtchn_handle xc_evtchn;
 typedef struct xc_interface_core xc_gnttab;
 typedef struct xc_interface_core xc_gntshr;
 
diff --git a/tools/libxc/include/xenctrlosdep.h 
b/tools/libxc/include/xenctrlosdep.h
index 5121d9b..89564e1 100644
--- a/tools/libxc/include/xenctrlosdep.h
+++ b/tools/libxc/include/xenctrlosdep.h
@@ -51,7 +51,6 @@
 
 enum xc_osdep_type {
 XC_OSDEP_PRIVCMD,
-XC_OSDEP_EVTCHN,
 XC_OSDEP_GNTTAB,
 XC_OSDEP_GNTSHR,
 };
@@ -90,21 +89,6 @@ struct xc_osdep_ops
 int nentries);
 } privcmd;
 struct {
-int (*fd)(xc_evtchn *xce, xc_osdep_handle h);
-
-int (*notify)(xc_evtchn *xce, xc_osdep_handle h, evtchn_port_t 
port);
-
-evtchn_port_or_error_t (*bind_unbound_port)(xc_evtchn *xce, 
xc_osdep_handle h, int domid);
-evtchn_port_or_error_t (*bind_interdomain)(xc_evtchn *xce, 
xc_osdep_handle h, int domid,
-   evtchn_port_t 
remote_port);
-evtchn_port_or_error_t (*bind_virq)(xc_evtchn *xce, 
xc_osdep_handle h, unsigned int virq);
-
-int (*unbind)(xc_evtchn *xce, xc_osdep_handle h, evtchn_port_t 
port);
-
-evtchn_port_or_error_t (*pending)(xc_evtchn *xce, xc_osdep_handle 
h);
-int (*unmask)(xc_evtchn *xce, xc_osdep_handle h, evtchn_port_t 
port);
-} evtchn;
-struct {
 #define XC_GRANT_MAP_SINGLE_DOMAIN 0x1
 void *(*grant_map)(xc_gnttab *xcg, xc_osdep_handle h,
uint32_t count, int flags, int prot,
diff --git a/tools/libxc/xc_evtchn.c b/tools/libxc/xc_evtchn.c
index 15f0580..ae2fe1a 100644
--- a/tools/libxc/xc_evtchn.c
+++ b/tools/libxc/xc_evtchn.c
@@ -77,51 +77,6 @@ int xc_evtchn_status(xc_interface *xch, xc_evtchn_status_t 
*status)
 sizeof(*status), 1);
 }
 
-int xc_evtchn_fd(xc_evtchn *xce)
-{
-return xce->ops->u.evtchn.fd(xce, xce->ops_handle);
-}
-
-int xc_evtchn_notify(xc_evtchn *xce, evtchn_port_t port)
-{
-return xce->ops->u.evtchn.notify(xce, xce->ops_handle, port);
-}
-
-evtchn_port_or_error_t
-xc_evtchn_bind_unbound_port(xc_evtchn *xce, int domid)
-{
-return xce->ops->u.evtchn.bind_unbound_port(xce, xce->ops_handle, domid);
-}
-
-evtchn_port_or_error_t
-xc_evtchn_bind_interdomain(xc_evtchn *xce, int domid,
-   evtchn_port_t remote_port)
-{
-return xce->ops->u.evtchn.bind_interdomain(xce, xce->ops_handle, domid, 
remote_port);
-}
-
-evtchn_port_or_error_t
-xc_evtchn_bind_virq(xc_evtchn *xce, unsigned int virq)
-{
-return xce->ops->u.evtchn.bind_virq(xce, xce->ops_handle, virq);
-}
-
-int xc_evtchn_unbind(xc_evtchn *xce, evtchn_port_t port)
-{
-return xce->ops->u.evtchn.unbind(xce, xce->ops_handle, port);
-}
-
-evtchn_port_or_error_t
-xc_evtchn_pending(xc_evtchn *xce)
-{
-return xce->ops->u.evtchn.pending(xce, xce->ops_handle);
-}
-
-int xc_evtchn_unmask(xc_evtchn *xce, evtchn_port_t port)
-{
-return xce->ops->u.evtchn.unmask(xce, xce->ops_handle, port);
-}
-
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libxc/xc_freebsd_osdep.c b/tools/libxc/xc_freebsd_osdep.c
index 4d31a1e..4323e16 100644
--- a/tools/libxc/xc_freebsd_osdep.c
+++ b/tools/libxc/xc_freebsd_osde

[Xen-devel] [PATCH XEN v8 03/29] tools: Arrange to check public headers for ANSI compatiblity

2016-01-15 Thread Ian Campbell
Using the same rune as we use for the Xen public headers, except we do
not need stdint.h here.

Signed-off-by: Ian Campbell 
Acked-by: Wei Liu 
---
v7: Drop -pendantic: Some library headers include parts of
xen/include/public, which in turn are not -pedantic clean
Retained ack, hope that's ok.
---
 .gitignore  | 2 ++
 tools/Rules.mk  | 8 
 tools/libs/evtchn/Makefile  | 4 +++-
 tools/libs/toollog/Makefile | 5 -
 4 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/.gitignore b/.gitignore
index f79a7b3..aa92d78 100644
--- a/.gitignore
+++ b/.gitignore
@@ -87,6 +87,8 @@ tools/config.cache
 config/Tools.mk
 config/Stubdom.mk
 config/Docs.mk
+tools/libs/toollog/headers.chk
+tools/libs/evtchn/headers.chk
 tools/blktap2/daemon/blktapctrl
 tools/blktap2/drivers/img2qcow
 tools/blktap2/drivers/lock-util
diff --git a/tools/Rules.mk b/tools/Rules.mk
index 75d02c4..0c83e22 100644
--- a/tools/Rules.mk
+++ b/tools/Rules.mk
@@ -176,6 +176,14 @@ INSTALL_PYTHON_PROG = \
 %.opic: %.S
$(CC) $(CPPFLAGS) -DPIC $(CFLAGS) $(CFLAGS.opic) -fPIC -c -o $@ $< 
$(APPEND_CFLAGS)
 
+headers.chk:
+   for i in $(filter %.h,$^); do \
+   $(CC) -x c -ansi -Wall -Werror $(CFLAGS_xeninclude) \
+ -S -o /dev/null $$i || exit 1; \
+   echo $$i; \
+   done >$@.new
+   mv $@.new $@
+
 subdirs-all subdirs-clean subdirs-install subdirs-distclean: .phony
@set -e; for subdir in $(SUBDIRS) $(SUBDIRS-y); do \
$(MAKE) subdir-$(patsubst subdirs-%,%,$@)-$$subdir; \
diff --git a/tools/libs/evtchn/Makefile b/tools/libs/evtchn/Makefile
index 46a807f..9917864 100644
--- a/tools/libs/evtchn/Makefile
+++ b/tools/libs/evtchn/Makefile
@@ -32,8 +32,9 @@ build:
$(MAKE) libs
 
 .PHONY: libs
-libs: $(LIB)
+libs: headers.chk $(LIB)
 
+headers.chk: $(wildcard include/*.h)
 
 libxenevtchn.a: $(LIB_OBJS)
$(AR) rc $@ $^
@@ -64,6 +65,7 @@ TAGS:
 clean:
rm -rf *.rpm $(LIB) *~ $(DEPS) $(LIB_OBJS) $(PIC_OBJS)
rm -f libxenevtchn.so.$(MAJOR).$(MINOR) libxenevtchn.so.$(MAJOR)
+   rm -f headers.chk
 
 .PHONY: distclean
 distclean: clean
diff --git a/tools/libs/toollog/Makefile b/tools/libs/toollog/Makefile
index 9bfd179..fb701be 100644
--- a/tools/libs/toollog/Makefile
+++ b/tools/libs/toollog/Makefile
@@ -27,7 +27,9 @@ build:
$(MAKE) libs
 
 .PHONY: libs
-libs: $(LIB)
+libs: headers.chk $(LIB)
+
+headers.chk: $(wildcard include/*.h)
 
 libxentoollog.a: $(LIB_OBJS)
$(AR) rc $@ $^
@@ -58,6 +60,7 @@ TAGS:
 clean:
rm -rf *.rpm $(LIB) *~ $(DEPS) $(LIB_OBJS) $(PIC_OBJS)
rm -f libxentoollog.so.$(MAJOR).$(MINOR) libxentoollog.so.$(MAJOR)
+   rm -f headers.chk
 
 .PHONY: distclean
 distclean: clean
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH XEN v8 10/29] tools: Implement xc_map_foreign_range(s) in terms of common helper

2016-01-15 Thread Ian Campbell
Both Linux and FreeBSD already implemented these functions using
identical helpers based on xc_map_foreign_pages. Make one copy of
these common helpers and switch all OSes to use them, even those which
previously had a specific lower level implementation of this
functionality.

This is makes two fewer low level interfaces to think about.

Signed-off-by: Ian Campbell 
Acked-by: Wei Liu 
---
 tools/libxc/xc_foreign_memory.c | 50 +
 tools/libxc/xc_freebsd_osdep.c  | 50 -
 tools/libxc/xc_linux_osdep.c| 49 -
 tools/libxc/xc_minios.c | 43 -
 tools/libxc/xc_netbsd.c | 70 -
 tools/libxc/xc_solaris.c| 67 ---
 6 files changed, 50 insertions(+), 279 deletions(-)

diff --git a/tools/libxc/xc_foreign_memory.c b/tools/libxc/xc_foreign_memory.c
index 2413e75..d1130e6 100644
--- a/tools/libxc/xc_foreign_memory.c
+++ b/tools/libxc/xc_foreign_memory.c
@@ -50,6 +50,56 @@ void *xc_map_foreign_pages(xc_interface *xch, uint32_t dom, 
int prot,
 return res;
 }
 
+void *xc_map_foreign_range(xc_interface *xch,
+   uint32_t dom, int size, int prot,
+   unsigned long mfn)
+{
+xen_pfn_t *arr;
+int num;
+int i;
+void *ret;
+
+num = (size + XC_PAGE_SIZE - 1) >> XC_PAGE_SHIFT;
+arr = calloc(num, sizeof(xen_pfn_t));
+if ( arr == NULL )
+return NULL;
+
+for ( i = 0; i < num; i++ )
+arr[i] = mfn + i;
+
+ret = xc_map_foreign_pages(xch, dom, prot, arr, num);
+free(arr);
+return ret;
+}
+
+void *xc_map_foreign_ranges(xc_interface *xch,
+uint32_t dom, size_t size,
+int prot, size_t chunksize,
+privcmd_mmap_entry_t entries[],
+int nentries)
+{
+xen_pfn_t *arr;
+int num_per_entry;
+int num;
+int i;
+int j;
+void *ret;
+
+num_per_entry = chunksize >> XC_PAGE_SHIFT;
+num = num_per_entry * nentries;
+arr = calloc(num, sizeof(xen_pfn_t));
+if ( arr == NULL )
+return NULL;
+
+for ( i = 0; i < nentries; i++ )
+for ( j = 0; j < num_per_entry; j++ )
+arr[i * num_per_entry + j] = entries[i].mfn + j;
+
+ret = xc_map_foreign_pages(xch, dom, prot, arr, num);
+free(arr);
+return ret;
+}
+
 /*
  * stub for all not yet converted OSes (NetBSD and Solaris). New OSes should
  * just implement xc_map_foreign_bulk.
diff --git a/tools/libxc/xc_freebsd_osdep.c b/tools/libxc/xc_freebsd_osdep.c
index 6b440ee..7745d28 100644
--- a/tools/libxc/xc_freebsd_osdep.c
+++ b/tools/libxc/xc_freebsd_osdep.c
@@ -125,56 +125,6 @@ void *xc_map_foreign_bulk(xc_interface *xch,
 return addr;
 }
 
-void *xc_map_foreign_range(xc_interface *xch,
-   uint32_t dom, int size, int prot,
-   unsigned long mfn)
-{
-xen_pfn_t *arr;
-int num;
-int i;
-void *ret;
-
-num = (size + XC_PAGE_SIZE - 1) >> XC_PAGE_SHIFT;
-arr = calloc(num, sizeof(xen_pfn_t));
-if ( arr == NULL )
-return NULL;
-
-for ( i = 0; i < num; i++ )
-arr[i] = mfn + i;
-
-ret = xc_map_foreign_pages(xch, dom, prot, arr, num);
-free(arr);
-return ret;
-}
-
-void *xc_map_foreign_ranges(xc_interface *xch,
-uint32_t dom, size_t size,
-int prot, size_t chunksize,
-privcmd_mmap_entry_t entries[],
-int nentries)
-{
-xen_pfn_t *arr;
-int num_per_entry;
-int num;
-int i;
-int j;
-void *ret;
-
-num_per_entry = chunksize >> XC_PAGE_SHIFT;
-num = num_per_entry * nentries;
-arr = calloc(num, sizeof(xen_pfn_t));
-if ( arr == NULL )
-return NULL;
-
-for ( i = 0; i < nentries; i++ )
-for ( j = 0; j < num_per_entry; j++ )
-arr[i * num_per_entry + j] = entries[i].mfn + j;
-
-ret = xc_map_foreign_pages(xch, dom, prot, arr, num);
-free(arr);
-return ret;
-}
-
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libxc/xc_linux_osdep.c b/tools/libxc/xc_linux_osdep.c
index 39c88ce..acf44ce 100644
--- a/tools/libxc/xc_linux_osdep.c
+++ b/tools/libxc/xc_linux_osdep.c
@@ -290,55 +290,6 @@ void *xc_map_foreign_bulk(xc_interface *xch,
 return addr;
 }
 
-void *xc_map_foreign_range(xc_interface *xch,
-   uint32_t dom, int size, int prot,
-   unsigned long mfn)
-{
-xen_pfn_t *arr;
-int num;
-int i;
-void *ret;
-
-num = (size + XC_PAGE_SIZE - 1) >> XC_PAGE_SHIFT;
-arr = calloc(num, sizeof(xen_pfn_t));
-if ( arr == NULL )
-return NULL;
-
-for ( i = 0; i < num; i++ )
-arr[i] = mfn + i;
-
-ret = xc_map_foreign_pages(xch, dom, prot, arr, num);
-

[Xen-devel] [PATCH XEN v8 04/29] tools/libxc: Remove osdep indirection for xc_gnt{shr, tab}

2016-01-15 Thread Ian Campbell
The alternative backend (a xen-api/xapi shim) is no longer around and
so this stuff is now just baggage which is getting in the way of
refactoring libxenctrl.

Nested virt probably suffices for this use case now.

It is now necessary to provide explicit versions of things for
platforms which do not implement this functionality, since the osdep
dispatcher cannot fulfil this need any more. These are provided by
appropriate xc_nognt???.c files which are compiled and linked on the
appropriate platforms. In them open and close return failure and
everything else aborts, since if open fails they should never be
called.

Signed-off-by: Ian Campbell 
Acked-by: Wei Liu 
---
 tools/libxc/Makefile   |  10 ++--
 tools/libxc/include/xenctrl.h  |   4 +-
 tools/libxc/include/xenctrlosdep.h |  23 ---
 tools/libxc/xc_gnttab.c|  57 --
 tools/libxc/xc_linux_osdep.c   | 119 +++--
 tools/libxc/xc_minios.c|  51 ++--
 tools/libxc/xc_nogntshr.c  |  46 ++
 tools/libxc/xc_nognttab.c  |  50 
 tools/libxc/xc_private.c   |  83 +-
 tools/libxc/xc_private.h   |  24 
 10 files changed, 274 insertions(+), 193 deletions(-)
 create mode 100644 tools/libxc/xc_nogntshr.c
 create mode 100644 tools/libxc/xc_nognttab.c

diff --git a/tools/libxc/Makefile b/tools/libxc/Makefile
index b8fc6a5..184cbb7 100644
--- a/tools/libxc/Makefile
+++ b/tools/libxc/Makefile
@@ -43,11 +43,11 @@ CTRL_SRCS-y   += xc_resource.c
 CTRL_SRCS-$(CONFIG_X86) += xc_psr.c
 CTRL_SRCS-$(CONFIG_X86) += xc_pagetab.c
 CTRL_SRCS-$(CONFIG_Linux) += xc_linux.c xc_linux_osdep.c
-CTRL_SRCS-$(CONFIG_FreeBSD) += xc_freebsd.c xc_freebsd_osdep.c
-CTRL_SRCS-$(CONFIG_SunOS) += xc_solaris.c
-CTRL_SRCS-$(CONFIG_NetBSD) += xc_netbsd.c
-CTRL_SRCS-$(CONFIG_NetBSDRump) += xc_netbsd.c
-CTRL_SRCS-$(CONFIG_MiniOS) += xc_minios.c
+CTRL_SRCS-$(CONFIG_FreeBSD) += xc_freebsd.c xc_freebsd_osdep.c xc_nognttab.c 
xc_nogntshr.c
+CTRL_SRCS-$(CONFIG_SunOS) += xc_solaris.c xc_nognttab.c xc_nogntshr.c
+CTRL_SRCS-$(CONFIG_NetBSD) += xc_netbsd.c xc_nognttab.c xc_nogntshr.c
+CTRL_SRCS-$(CONFIG_NetBSDRump) += xc_netbsd.c xc_nognttab.c xc_nogntshr.c
+CTRL_SRCS-$(CONFIG_MiniOS) += xc_minios.c xc_nogntshr.c
 CTRL_SRCS-y   += xc_evtchn_compat.c
 
 GUEST_SRCS-y :=
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 8f31c4d..0fc2a11 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -117,8 +117,8 @@
  */
 
 typedef struct xc_interface_core xc_interface;
-typedef struct xc_interface_core xc_gnttab;
-typedef struct xc_interface_core xc_gntshr;
+typedef struct xengntdev_handle xc_gnttab;
+typedef struct xengntdev_handle xc_gntshr;
 
 enum xc_error_code {
   XC_ERROR_NONE = 0,
diff --git a/tools/libxc/include/xenctrlosdep.h 
b/tools/libxc/include/xenctrlosdep.h
index 89564e1..423660d 100644
--- a/tools/libxc/include/xenctrlosdep.h
+++ b/tools/libxc/include/xenctrlosdep.h
@@ -51,8 +51,6 @@
 
 enum xc_osdep_type {
 XC_OSDEP_PRIVCMD,
-XC_OSDEP_GNTTAB,
-XC_OSDEP_GNTSHR,
 };
 
 /* Opaque handle internal to the backend */
@@ -88,27 +86,6 @@ struct xc_osdep_ops
 size_t chunksize, privcmd_mmap_entry_t 
entries[],
 int nentries);
 } privcmd;
-struct {
-#define XC_GRANT_MAP_SINGLE_DOMAIN 0x1
-void *(*grant_map)(xc_gnttab *xcg, xc_osdep_handle h,
-   uint32_t count, int flags, int prot,
-   uint32_t *domids, uint32_t *refs,
-   uint32_t notify_offset,
-   evtchn_port_t notify_port);
-int (*munmap)(xc_gnttab *xcg, xc_osdep_handle h,
-  void *start_address,
-  uint32_t count);
-int (*set_max_grants)(xc_gnttab *xcg, xc_osdep_handle h, uint32_t 
count);
-} gnttab;
-struct {
-void *(*share_pages)(xc_gntshr *xcg, xc_osdep_handle h,
- uint32_t domid, int count,
- uint32_t *refs, int writable,
- uint32_t notify_offset,
- evtchn_port_t notify_port);
-int (*munmap)(xc_gntshr *xcg, xc_osdep_handle h,
-  void *start_address, uint32_t count);
-} gntshr;
 } u;
 };
 typedef struct xc_osdep_ops xc_osdep_ops;
diff --git a/tools/libxc/xc_gnttab.c b/tools/libxc/xc_gnttab.c
index 60335d8..a51f405 100644
--- a/tools/libxc/xc_gnttab.c
+++ b/tools/libxc/xc_gnttab.c
@@ -143,68 +143,48 @@ grant_entry_v2_t *xc_gnttab_map_table_v2(xc_interface 
*xch, int domid,
 return _gnttab_map_table(xch, domid, gnt_num);
 }
 
-void *xc_gnttab_map_grant_ref(xc_gnttab *xcg,
+void *xc_gnttab_map_grant_ref(xc_gnt

[Xen-devel] [PATCH XEN v8 00/29] Begin to disentangle libxenctrl and provide some stable libraries

2016-01-15 Thread Ian Campbell
We intend to stabilise some parts of the libxenctrl interface by
splitting out some functionality into separate stable libraries.

This is the xen part of the first phase of that change.

This mail is (or is intended to be) a reply to a "0/"
super-intro mail covering all of the related patch series and which
contains more details.

Ian Campbell (29):
  tools/libxc: Remove osdep indirection for xc_evtchn
  tools: Refactor /dev/xen/evtchn wrappers into libxenevtchn.
  tools: Arrange to check public headers for ANSI compatiblity
  tools/libxc: Remove osdep indirection for xc_gnt{shr,tab}
  tools: Refactor /dev/xen/gnt{dev,shr} wrappers into libxengnttab.
  tools/libxc: Remove osdep indirection for privcmd
  tools: Refactor hypercall calling wrappers into libxencall.
  tools/libxc: drop xc_map_foreign_bulk_compat wrappers
  tools: Remove xc_map_foreign_batch
  tools: Implement xc_map_foreign_range(s) in terms of common helper
  tools: Refactor foreign memory mapping into libxenforeignmemory
  tools/libs/foreignmemory: provide xenforeignmemory_unmap.
  tools/libs/foreignmemory: use size_t for size arguments.
  tools/libs/foreignmemory: Mention restrictions on fork in docs.
  tools/libs/foreignmemory: Support err == NULL to map.
  tools/libs/foreignmemory: pull array length argument to map forward
  tools/libs/evtchn: Review and update doc comments.
  tools/libs/evtchn: Use uint32_t for domid arguments
  tools/libs: Clean up hard tabs.
  tools/libs/gnttab: Extensive updates to API documentation.
  tools/libs/call: Update some log messages to not refer to xc.
  tools/libs/call: Describe return values and error semantics for
xencall*
  tools/libs/call: Avoid xc_memalign in netbsd and solaris backends
  tools/libs/call: linux: touch newly allocated pages after madvise
lockdown
  tools/libs/{call,evtchn}: Document requirements around forking.
  tools/libs/*: Use O_CLOEXEC on Linux and FreeBSD
  tools: Update CFLAGS for qemu-xen to allow it to use new libraries
  tools/libs/*: Introduce APIs to restrict handles to a specific domain.
  HACK: Update Config.mk to pull all the right bits from my xenbits
trees

 .gitignore |   9 +
 Config.mk  |  22 +-
 config/FreeBSD.mk  |   2 -
 config/NetBSD.mk   |   3 -
 config/NetBSDRump.mk   |   1 -
 config/StdGNU.mk   |   1 -
 config/SunOS.mk|   1 -
 stubdom/Makefile   |  62 +-
 tools/Makefile |  15 +-
 tools/Rules.mk |  44 +-
 tools/console/Makefile |   3 +-
 tools/console/daemon/io.c  |  64 +-
 tools/console/daemon/utils.c   |   1 -
 tools/console/daemon/utils.h   |   1 +
 tools/libs/Makefile|   4 +
 tools/libs/call/Makefile   |  71 ++
 tools/libs/call/buffer.c   | 192 +
 tools/libs/call/core.c | 154 
 tools/libs/call/freebsd.c  | 126 +++
 tools/libs/call/include/xencall.h  | 161 
 tools/libs/call/libxencall.map |  21 +
 tools/libs/call/linux.c| 134 +++
 tools/libs/call/minios.c   |  81 ++
 tools/libs/call/netbsd.c   | 121 +++
 tools/libs/call/private.h  |  68 ++
 tools/libs/call/solaris.c  |  97 +++
 tools/libs/evtchn/Makefile |  71 ++
 tools/libs/evtchn/core.c   |  79 ++
 tools/libs/evtchn/freebsd.c| 138 
 tools/libs/evtchn/include/xenevtchn.h  | 199 +
 tools/libs/evtchn/libxenevtchn.map |  21 +
 tools/libs/evtchn/linux.c  | 140 
 tools/libs/evtchn/minios.c | 266 ++
 tools/libs/evtchn/netbsd.c | 147 
 tools/libs/evtchn/private.h|  25 +
 tools/libs/evtchn/solaris.c| 135 +++
 tools/libs/foreignmemory/Makefile  |  71 ++
 tools/libs/foreignmemory/compat.c  |  72 ++
 tools/libs/foreignmemory/core.c| 123 +++
 tools/libs/foreignmemory/freebsd.c | 115 +++
 .../libs/foreignmemory/include/xenforeignmemory.h  | 150 
 tools/libs/foreignmemory/libxenforeignmemory.map   |  11 +
 tools/libs/foreignmemory/linux.c   | 283 +++
 tools/libs/foreignmemory/minios.c  |  69 ++
 tools/libs/foreignmemory/netbsd.c  | 111 +++
 tools/libs/foreignmemory/private.h |  54 ++
 tools/libs/foreignmemory/

[Xen-devel] [Minios-devel] [PATCH v8 0/] Begin to disentangle libxenctrl and provide some stable libraries

2016-01-15 Thread Ian Campbell
In <1431963008.4944.80.ca...@citrix.com> I proposed stabilising some
parts of the libxenctrl API/ABI by disaggregating into separate
libraries.

This is v8 of that set of series against:
xen
qemu-xen
qemu-xen-traditional
mini-os

NB: Samuel+minios-devel will only get the mini-os side and Stefano+qemu
-devel the qemu-xen side.

The code for all repos can be found in:

git://xenbits.xen.org/people/ianc/libxenctrl-split/xen.git  v8
git://xenbits.xen.org/people/ianc/libxenctrl-split/qemu-xen.git v8
git://xenbits.xen.org/people/ianc/libxenctrl-split/qemu-xen-traditional.git v8
git://xenbits.xen.org/people/ianc/libxenctrl-split/mini-os.git  v8

The tip of the xen.git branch contains an extra patch hacking Config.mk
to point to all the others above, which should get the correct things for
the HEAD of the branch, but not further back in time.

The new libraries here are:

 * libxentoollog: Common logging infrastructure (already in tree)
 * libxenevtchn: Userspace access to evtchns (via /dev/xen/evtchn etc)
 * libxengnttab: Userspace access to grant tables (via /dev/xen/gnt??? etc)
 * libxencall: Making hypercalls (i.e. the IOCTL_PRIVCMD_HYPERCALL type
   functionality)
 * libxenforeignmemory: Privileged mappings of foreign memory
   (IOCTL_PRIVCMD_MMAP et al)

The first three were actually pretty distinct within libxenctrl already and
have not changed in quite some time.

Although the other two are somewhat new they are based on top of long
standing stable ioctls, which gives me some confidence.

Nonetheless I would appreciate extra review of at least the interface
headers of all of these with a particular eye to the suitability of these
interfaces being maintained in an ABI (_B_, not _P_) stable way going
forward.

Still to come would be libraries for specific out of tree purposes
(device model, kexec), which would be adding new library at the same
level as libxc I think, rather than underneath, i.e. also using the
libraries split out here, but hopefully not libxenctrl itself.

The new libraries use linker version-scripts to hopefully make future
ABI changes be possible in a compatible way.

Since last time:

 * Some early bits went in.
 * Rebased
 * Clean up the *.so in clean, added distclean targets to each lib
 * On the QEMU side use CONFIG_XEN_CTRL_INTERFACE_VERSION == 471 as the
   gate for this new setup (dropped a Reviewed-by).

Even with the dropped acks mini-os and qemu-xen-trad are fully acked (by
Samuel+Wei and Ian J respectively), while qemu-xen and xen are mostly acked
(but had a few dropped acks since last time).

Summary for qemu-xen.git:
 R  xen_console: correctly cleanup primary console on teardown.
 R  xen: Switch to libxenevtchn interface for compat shims.
 R  xen: Switch to libxengnttab interface for compat shims.
 R  xen: Switch uses of xc_map_foreign_range into xc_map_foreign_pages
  M xen: Switch uses of xc_map_foreign_{pages,bulk} to use 
libxenforeignmemory API.
  M xen: Use stable library interfaces when they are available.
A   xen: domainbuild: reopen libxenctrl interface after forking for domain 
watcher.
 R  xen: make it possible to build without the Xen PV domain builder

(A == Acked by Stefano, R == Reviewed by Stefano, M == Modified in v8)

NB: qemu-xen-traditional.git, mini-os.git and xen.git are intertwined, but
the qemu-xen.git part is independent and should be applied after all the
rest of these series.

Summary for xen.git:

 W  tools/libxc: Remove osdep indirection for xc_evtchn
MWI tools: Refactor /dev/xen/evtchn wrappers into libxenevtchn.
 W  tools: Arrange to check public headers for ANSI compatiblity
 W  tools/libxc: Remove osdep indirection for xc_gnt{shr,tab}
MW  tools: Refactor /dev/xen/gnt{dev,shr} wrappers into libxengnttab.
 WS tools/libxc: Remove osdep indirection for privcmd
MW  tools: Refactor hypercall calling wrappers into libxencall.
 W  tools/libxc: drop xc_map_foreign_bulk_compat wrappers
 W   G  tools: Remove xc_map_foreign_batch
 W  tools: Implement xc_map_foreign_range(s) in terms of common helper
MWI tools: Refactor foreign memory mapping into libxenforeignmemory
 WI tools/libs/foreignmemory: provide xenforeignmemory_unmap.
 WI tools/libs/foreignmemory: use size_t for size arguments.
tools/libs/foreignmemory: Mention restrictions on fork in docs.
 WI tools/libs/foreignmemory: Support err == NULL to map.
 WI tools/libs/foreignmemory: pull array length argument to map forward
 WI tools/libs/evtchn: Review and update doc comments.
N   tools/libs/evtchn: Use uint32_t for domid arguments
 W  tools/libs: Clean up hard tabs.
D   tools/libs/gnttab: Extensive updates to API documentation.
 W  tools/libs/call: Update some log messages to not refer to xc.
 WIRtools/libs/call: Describe return values and error semantics for xencall*
 W  tools/libs/call: Avoid xc_memalign in netbsd and sola

[Xen-devel] [PATCH XEN v8 08/29] tools/libxc: drop xc_map_foreign_bulk_compat wrappers

2016-01-15 Thread Ian Campbell
On Solaris and NetBSD xc_map_foreign_bulk is implemented by calling
xc_map_foreign_bulk_compat and xc_map_foreign_bulk_compat is exposed
as a symbol by libxenctrl.so.

Remove these wrappers and turn the compat function into the real thing
surrounded by the appropriate ifdef.

As this is a compat function all new ports should instead implement
xc_map_foreign_bulk properly, hence the ifdef should never be
expanded.

Signed-off-by: Ian Campbell 
Acked-by: Wei Liu 
---
 tools/libxc/xc_foreign_memory.c | 13 +
 tools/libxc/xc_netbsd.c |  7 ---
 tools/libxc/xc_private.h|  5 -
 tools/libxc/xc_solaris.c|  7 ---
 4 files changed, 9 insertions(+), 23 deletions(-)

diff --git a/tools/libxc/xc_foreign_memory.c b/tools/libxc/xc_foreign_memory.c
index 9c705b6..b205bca 100644
--- a/tools/libxc/xc_foreign_memory.c
+++ b/tools/libxc/xc_foreign_memory.c
@@ -50,10 +50,14 @@ void *xc_map_foreign_pages(xc_interface *xch, uint32_t dom, 
int prot,
 return res;
 }
 
-/* stub for all not yet converted OSes */
-void *xc_map_foreign_bulk_compat(xc_interface *xch,
- uint32_t dom, int prot,
- const xen_pfn_t *arr, int *err, unsigned int 
num)
+/*
+ * stub for all not yet converted OSes (NetBSD and Solaris). New OSes should
+ * just implement xc_map_foreign_bulk.
+ */
+#if defined(__NetBSD__) || defined(__sun__)
+void *xc_map_foreign_bulk(xc_interface *xch,
+  uint32_t dom, int prot,
+  const xen_pfn_t *arr, int *err, unsigned int num)
 {
 xen_pfn_t *pfn;
 unsigned int i;
@@ -90,6 +94,7 @@ void *xc_map_foreign_bulk_compat(xc_interface *xch,
 
 return ret;
 }
+#endif
 
 /*
  * Local variables:
diff --git a/tools/libxc/xc_netbsd.c b/tools/libxc/xc_netbsd.c
index 5e3b343..d7f7f31 100644
--- a/tools/libxc/xc_netbsd.c
+++ b/tools/libxc/xc_netbsd.c
@@ -67,13 +67,6 @@ int osdep_privcmd_close(xc_interface *xch)
 return close(fd);
 }
 
-void *xc_map_foreign_bulk(xc_interface *xch,
-  uint32_t dom, int prot,
-  const xen_pfn_t *arr, int *err, unsigned int num)
-{
-return xc_map_foreign_bulk_compat(xch, dom, prot, arr, err, num);
-}
-
 void *xc_map_foreign_batch(xc_interface *xch,
uint32_t dom, int prot,
xen_pfn_t *arr, int num)
diff --git a/tools/libxc/xc_private.h b/tools/libxc/xc_private.h
index c93df7f..ecf2451 100644
--- a/tools/libxc/xc_private.h
+++ b/tools/libxc/xc_private.h
@@ -107,11 +107,6 @@ int osdep_privcmd_close(xc_interface *xch);
 void *osdep_alloc_hypercall_buffer(xc_interface *xch, int npages);
 void osdep_free_hypercall_buffer(xc_interface *xch, void *ptr, int npages);
 
-/* Stub for not yet converted OSes */
-void *xc_map_foreign_bulk_compat(xc_interface *xch,
- uint32_t dom, int prot,
- const xen_pfn_t *arr, int *err, unsigned int 
num);
-
 void xc_report_error(xc_interface *xch, int code, const char *fmt, ...)
 __attribute__((format(printf,3,4)));
 void xc_reportv(xc_interface *xch, xentoollog_logger *lg, xentoollog_level,
diff --git a/tools/libxc/xc_solaris.c b/tools/libxc/xc_solaris.c
index 18622fa..5e72dee 100644
--- a/tools/libxc/xc_solaris.c
+++ b/tools/libxc/xc_solaris.c
@@ -94,13 +94,6 @@ void *xc_map_foreign_batch(xc_interface *xch,
 
 }
 
-void *xc_map_foreign_bulk(xc_interface *xch,
-  uint32_t dom, int prot,
-  const xen_pfn_t *arr, int *err, unsigned int num)
-{
-return xc_map_foreign_bulk_compat(xch, dom, prot, arr, err, num);
-}
-
 void *xc_map_foreign_range(xc_interface *xch,
uint32_t dom,
int size, int prot,
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH XEN v8 09/29] tools: Remove xc_map_foreign_batch

2016-01-15 Thread Ian Campbell
It can trivially be replaced by xc_map_foreign_pages which is the
interface I want to move to going forward (by standardising on _bulk
but handling err=NULL as _pages does).

The callers of _batch are checking a mixture of a NULL return or
looking to see if the top nibble of the (usually sole) mfn they pass
has been modified to be non-zero to detect errors. _pages never
modifies the mfn it was given (it's const) and returns NULL on
failure, so adjust the error handling where necessary. Some callers
use a copy of the mfn array, for reuse on failure with _batch, which
is no longer necessary as _pages doesn't modify the array, however I
haven't cleaned that up here.

This reduces the twist maze of xc_map_foreign_* by one, which will be
useful when trying to come up with an underlying stable interface.

NetBSD and Solaris implemented xc_map_foreign_bulk in terms of
xc_map_foreign_batch via a compat layer, so xc_map_foreign_batch
becomes an internal osdep for them.

Signed-off-by: Ian Campbell 
Acked-by: Wei Liu 
Acked-by: George Dunlap 
Cc: George Dunlap 
---
v7: Don't mention new ports implementing _bulk, which is confusing and
will be obsolete a couple of patches later.
v6: Switch to xc_map_foreign_pages not xc_map_foreign_bulk, since the
former has more similar error handling semantics to the current
usage.

Dropped acks.
---
 tools/libxc/include/xenctrl.h   | 10 ---
 tools/libxc/xc_foreign_memory.c |  4 ++-
 tools/libxc/xc_linux_osdep.c| 59 +++--
 tools/libxc/xc_minios.c | 22 ---
 tools/libxc/xc_netbsd.c | 10 +++
 tools/libxc/xc_solaris.c|  6 ++---
 tools/libxc/xc_vm_event.c   | 18 ++---
 tools/xenmon/xenbaked.c | 12 -
 tools/xenpaging/xenpaging.c | 14 +-
 tools/xentrace/xentrace.c   |  3 ++-
 10 files changed, 48 insertions(+), 110 deletions(-)

diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 78400d3..cb41c07 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -1387,16 +1387,6 @@ void *xc_map_foreign_pages(xc_interface *xch, uint32_t 
dom, int prot,
const xen_pfn_t *arr, int num );
 
 /**
- * DEPRECATED - use xc_map_foreign_bulk() instead.
- *
- * Like xc_map_foreign_pages(), except it can succeeed partially.
- * When a page cannot be mapped, its PFN in @arr is or'ed with
- * 0xF000 to indicate the error.
- */
-void *xc_map_foreign_batch(xc_interface *xch, uint32_t dom, int prot,
-   xen_pfn_t *arr, int num );
-
-/**
  * Like xc_map_foreign_pages(), except it can succeed partially.
  * When a page cannot be mapped, its respective field in @err is
  * set to the corresponding errno value.
diff --git a/tools/libxc/xc_foreign_memory.c b/tools/libxc/xc_foreign_memory.c
index b205bca..2413e75 100644
--- a/tools/libxc/xc_foreign_memory.c
+++ b/tools/libxc/xc_foreign_memory.c
@@ -55,6 +55,8 @@ void *xc_map_foreign_pages(xc_interface *xch, uint32_t dom, 
int prot,
  * just implement xc_map_foreign_bulk.
  */
 #if defined(__NetBSD__) || defined(__sun__)
+void *osdep_map_foreign_batch(xc_interface *xch, uint32_t dom, int prot,
+  xen_pfn_t *arr, int num );
 void *xc_map_foreign_bulk(xc_interface *xch,
   uint32_t dom, int prot,
   const xen_pfn_t *arr, int *err, unsigned int num)
@@ -75,7 +77,7 @@ void *xc_map_foreign_bulk(xc_interface *xch,
 }
 
 memcpy(pfn, arr, num * sizeof(*arr));
-ret = xc_map_foreign_batch(xch, dom, prot, pfn, num);
+ret = osdep_map_foreign_batch(xch, dom, prot, pfn, num);
 
 if (ret) {
 for (i = 0; i < num; ++i)
diff --git a/tools/libxc/xc_linux_osdep.c b/tools/libxc/xc_linux_osdep.c
index e68c495..39c88ce 100644
--- a/tools/libxc/xc_linux_osdep.c
+++ b/tools/libxc/xc_linux_osdep.c
@@ -91,8 +91,8 @@ int osdep_privcmd_close(xc_interface *xch)
 return close(fd);
 }
 
-static int xc_map_foreign_batch_single(int fd, uint32_t dom,
-   xen_pfn_t *mfn, unsigned long addr)
+static int map_foreign_batch_single(int fd, uint32_t dom,
+xen_pfn_t *mfn, unsigned long addr)
 {
 privcmd_mmapbatch_t ioctlx;
 int rc;
@@ -113,59 +113,6 @@ static int xc_map_foreign_batch_single(int fd, uint32_t 
dom,
 return rc;
 }
 
-void *xc_map_foreign_batch(xc_interface *xch,
-   uint32_t dom, int prot,
-   xen_pfn_t *arr, int num)
-{
-int fd = xch->privcmdfd;
-privcmd_mmapbatch_t ioctlx;
-void *addr;
-int rc;
-
-addr = mmap(NULL, num << XC_PAGE_SHIFT, prot, MAP_SHARED, fd, 0);
-if ( addr == MAP_FAILED )
-{
-PERROR("xc_map_foreign_batch: mmap failed");
-return NULL;
-}
-
-ioctlx.num = num;
-ioctlx.dom = dom;
-ioctlx.addr = (unsigned long)addr;
-ioctlx.arr = arr;
-
-  

[Xen-devel] [PATCH XEN v8 16/29] tools/libs/foreignmemory: pull array length argument to map forward

2016-01-15 Thread Ian Campbell
By having the "num" argument before the page and error arrays we can
potentially use a variable-length-array argument ("int pages[num]") in
the function prototype.

However VLAs are a C99 feature and we are currently targetting C89 and
later, so we don't actually make use of this here, merely arrange that
we can switch to VLAs in the future without changing the function ABI.

Signed-off-by: Ian Campbell 
Acked-by: Wei Liu 
Acked-by: Ian Jackson 
---
v6: New
---
 tools/libs/foreignmemory/compat.c   | 4 ++--
 tools/libs/foreignmemory/core.c | 5 +++--
 tools/libs/foreignmemory/freebsd.c  | 4 ++--
 tools/libs/foreignmemory/include/xenforeignmemory.h | 4 ++--
 tools/libs/foreignmemory/linux.c| 3 ++-
 tools/libs/foreignmemory/minios.c   | 3 ++-
 tools/libs/foreignmemory/private.h  | 3 ++-
 tools/libxc/xc_foreign_memory.c | 4 ++--
 tools/libxc/xc_sr_restore.c | 2 +-
 tools/libxc/xc_sr_save.c| 2 +-
 10 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/tools/libs/foreignmemory/compat.c 
b/tools/libs/foreignmemory/compat.c
index 039297c..b79ec1a 100644
--- a/tools/libs/foreignmemory/compat.c
+++ b/tools/libs/foreignmemory/compat.c
@@ -22,8 +22,8 @@
 #include "private.h"
 
 void *osdep_xenforeignmemory_map(xenforeignmemory_handle *fmem,
- uint32_t dom, int prot,
- const xen_pfn_t *arr, int *err, size_t num)
+ uint32_t dom, int prot, size_t num,
+ const xen_pfn_t arr[/*num*/], int 
err[/*num*/])
 {
 xen_pfn_t *pfn;
 unsigned int i;
diff --git a/tools/libs/foreignmemory/core.c b/tools/libs/foreignmemory/core.c
index 4e0541f..cfb0a73 100644
--- a/tools/libs/foreignmemory/core.c
+++ b/tools/libs/foreignmemory/core.c
@@ -64,7 +64,8 @@ int xenforeignmemory_close(xenforeignmemory_handle *fmem)
 
 void *xenforeignmemory_map(xenforeignmemory_handle *fmem,
uint32_t dom, int prot,
-   const xen_pfn_t *arr, int *err, size_t num)
+   size_t num,
+   const xen_pfn_t arr[/*num*/], int err[/*num*/])
 {
 void *ret;
 int *err_to_free = NULL;
@@ -75,7 +76,7 @@ void *xenforeignmemory_map(xenforeignmemory_handle *fmem,
 if ( err == NULL )
 return NULL;
 
-ret = osdep_xenforeignmemory_map(fmem, dom, prot, arr, err, num);
+ret = osdep_xenforeignmemory_map(fmem, dom, prot, num, arr, err);
 
 if ( ret == 0 && err_to_free )
 {
diff --git a/tools/libs/foreignmemory/freebsd.c 
b/tools/libs/foreignmemory/freebsd.c
index ed26ebb..703754f 100644
--- a/tools/libs/foreignmemory/freebsd.c
+++ b/tools/libs/foreignmemory/freebsd.c
@@ -84,8 +84,8 @@ int osdep_xenforeignmemory_close(xenforeignmemory_handle 
*fmem)
 
 void *osdep_xenforeignmemory_map(xenforeignmemory_handle *fmem,
  uint32_t dom, int prot,
- const xen_pfn_t *arr, int *err,
- size_t num)
+ size_t num,
+ const xen_pfn_t arr[/*num*/], int 
err[/*num*/])
 {
 int fd = fmem->fd;
 privcmd_mmapbatch_t ioctlx;
diff --git a/tools/libs/foreignmemory/include/xenforeignmemory.h 
b/tools/libs/foreignmemory/include/xenforeignmemory.h
index 0ec6325..3724c63 100644
--- a/tools/libs/foreignmemory/include/xenforeignmemory.h
+++ b/tools/libs/foreignmemory/include/xenforeignmemory.h
@@ -104,8 +104,8 @@ int xenforeignmemory_close(xenforeignmemory_handle *fmem);
  * of @err indicating failure to map every page.
  */
 void *xenforeignmemory_map(xenforeignmemory_handle *fmem, uint32_t dom,
-   int prot, const xen_pfn_t *arr, int *err,
-   size_t pages);
+   int prot, size_t pages,
+   const xen_pfn_t arr[/*pages*/], int err[/*pages*/]);
 
 /*
  * Unmap a mapping previous created with xenforeignmemory_map().
diff --git a/tools/libs/foreignmemory/linux.c b/tools/libs/foreignmemory/linux.c
index 6620391..32b6def 100644
--- a/tools/libs/foreignmemory/linux.c
+++ b/tools/libs/foreignmemory/linux.c
@@ -161,7 +161,8 @@ out:
 
 void *osdep_xenforeignmemory_map(xenforeignmemory_handle *fmem,
  uint32_t dom, int prot,
- const xen_pfn_t *arr, int *err, size_t num)
+ size_t num,
+ const xen_pfn_t arr[/*num*/], int 
err[/*num*/])
 {
 int fd = fmem->fd;
 privcmd_mmapbatch_v2_t ioctlx;
diff --git a/tools/libs/foreignmemory/minios.c 
b/tools/libs/foreignmemory/minios.c
index bdc1239..ca5ba71 100644
--- a/tools/libs/foreignmemory/minios.c
+++ b/tools/libs/foreignmemory/minios.c
@@ -41,

[Xen-devel] [PATCH XEN v8 17/29] tools/libs/evtchn: Review and update doc comments.

2016-01-15 Thread Ian Campbell
Remove the reference to pre-4.1, since this is now a new library.

Fixup references to xc.

Signed-off-by: Ian Campbell 
Acked-by: Wei Liu 
Acked-by: Ian Jackson 
---
 tools/libs/evtchn/include/xenevtchn.h | 6 ++
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/tools/libs/evtchn/include/xenevtchn.h 
b/tools/libs/evtchn/include/xenevtchn.h
index 3380fa3..60da2a3 100644
--- a/tools/libs/evtchn/include/xenevtchn.h
+++ b/tools/libs/evtchn/include/xenevtchn.h
@@ -46,11 +46,9 @@ typedef struct xentoollog_logger xentoollog_logger;
  * which case errno will be set appropriately.
  *
  * Note:
- * After fork a child process must not use any opened xc evtchn
+ * After fork a child process must not use any opened evtchn
  * handle inherited from their parent. They must open a new handle if
- * they want to interact with xc.
- *
- * Before Xen pre-4.1 this function would sometimes report errors with perror.
+ * they want to interact with evtchn.
  */
 /* Currently no flags are defined */
 xenevtchn_handle *xenevtchn_open(xentoollog_logger *logger, unsigned 
open_flags);
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH XEN v8 13/29] tools/libs/foreignmemory: use size_t for size arguments.

2016-01-15 Thread Ian Campbell
Surprisingly it appears no callers need updating.

Signed-off-by: Ian Campbell 
Acked-by: Wei Liu 
Acked-by: Ian Jackson 
---
v4: New patch
---
 tools/libs/foreignmemory/compat.c  |  2 +-
 tools/libs/foreignmemory/freebsd.c |  4 ++--
 .../libs/foreignmemory/include/xenforeignmemory.h  |  5 +++--
 tools/libs/foreignmemory/linux.c   | 23 +++---
 tools/libs/foreignmemory/minios.c  |  4 ++--
 tools/libs/foreignmemory/netbsd.c  |  2 +-
 tools/libs/foreignmemory/solaris.c |  2 +-
 7 files changed, 22 insertions(+), 20 deletions(-)

diff --git a/tools/libs/foreignmemory/compat.c 
b/tools/libs/foreignmemory/compat.c
index b8c6fc6..039297c 100644
--- a/tools/libs/foreignmemory/compat.c
+++ b/tools/libs/foreignmemory/compat.c
@@ -23,7 +23,7 @@
 
 void *osdep_xenforeignmemory_map(xenforeignmemory_handle *fmem,
  uint32_t dom, int prot,
- const xen_pfn_t *arr, int *err, unsigned int 
num)
+ const xen_pfn_t *arr, int *err, size_t num)
 {
 xen_pfn_t *pfn;
 unsigned int i;
diff --git a/tools/libs/foreignmemory/freebsd.c 
b/tools/libs/foreignmemory/freebsd.c
index 3c3ad09..ed26ebb 100644
--- a/tools/libs/foreignmemory/freebsd.c
+++ b/tools/libs/foreignmemory/freebsd.c
@@ -85,7 +85,7 @@ int osdep_xenforeignmemory_close(xenforeignmemory_handle 
*fmem)
 void *osdep_xenforeignmemory_map(xenforeignmemory_handle *fmem,
  uint32_t dom, int prot,
  const xen_pfn_t *arr, int *err,
- unsigned int num)
+ size_t num)
 {
 int fd = fmem->fd;
 privcmd_mmapbatch_t ioctlx;
@@ -119,7 +119,7 @@ void *osdep_xenforeignmemory_map(xenforeignmemory_handle 
*fmem,
 }
 
 int osdep_xenforeignmemory_unmap(xenforeignmemory_handle *fmem,
- void *addr, unsigned int num)
+ void *addr, size_t num)
 {
return munmap(addr, num << PAGE_SHIFT);
 }
diff --git a/tools/libs/foreignmemory/include/xenforeignmemory.h 
b/tools/libs/foreignmemory/include/xenforeignmemory.h
index b783af3..04ff548 100644
--- a/tools/libs/foreignmemory/include/xenforeignmemory.h
+++ b/tools/libs/foreignmemory/include/xenforeignmemory.h
@@ -56,14 +56,15 @@ int xenforeignmemory_close(xenforeignmemory_handle *fmem);
  */
 void *xenforeignmemory_map(xenforeignmemory_handle *fmem, uint32_t dom,
int prot, const xen_pfn_t *arr, int *err,
-   unsigned int num);
+   size_t pages);
+
 /*
  * Unmap a mapping previous created with xenforeignmemory_map().
  *
  * Returns 0 on success on failure sets errno and returns -1.
  */
 int xenforeignmemory_unmap(xenforeignmemory_handle *fmem,
-   void *addr, unsigned int num);
+   void *addr, size_t pages);
 
 #endif
 
diff --git a/tools/libs/foreignmemory/linux.c b/tools/libs/foreignmemory/linux.c
index 40ecd41..6620391 100644
--- a/tools/libs/foreignmemory/linux.c
+++ b/tools/libs/foreignmemory/linux.c
@@ -115,10 +115,11 @@ static int map_foreign_batch_single(int fd, uint32_t dom,
  * This will keep the request ring full and avoids delays.
  */
 static int retry_paged(int fd, uint32_t dom, void *addr,
-   const xen_pfn_t *arr, int *err, unsigned int num)
+   const xen_pfn_t *arr, int *err, size_t num)
 {
 privcmd_mmapbatch_v2_t ioctlx;
-int rc, paged = 0, i = 0;
+int rc, paged = 0;
+size_t i = 0;
 
 do
 {
@@ -134,7 +135,7 @@ static int retry_paged(int fd, uint32_t dom, void *addr,
 /* At least one gfn is still in paging state */
 ioctlx.num = 1;
 ioctlx.dom = dom;
-ioctlx.addr = (unsigned long)addr + ((unsigned long)i

[Xen-devel] [PATCH XEN v8 29/29] HACK: Update Config.mk to pull all the right bits from my xenbits trees

2016-01-15 Thread Ian Campbell
v4: Config.mk instead of .config
---
 Config.mk | 22 +++---
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/Config.mk b/Config.mk
index d654af8..364e790 100644
--- a/Config.mk
+++ b/Config.mk
@@ -238,22 +238,22 @@ endif
 
 ifeq ($(GIT_HTTP),y)
 OVMF_UPSTREAM_URL ?= http://xenbits.xen.org/git-http/ovmf.git
-QEMU_UPSTREAM_URL ?= http://xenbits.xen.org/git-http/qemu-xen.git
-QEMU_TRADITIONAL_URL ?= 
http://xenbits.xen.org/git-http/qemu-xen-traditional.git
+QEMU_UPSTREAM_URL ?= 
http://xenbits.xen.org/git-http/people/ianc/libxenctrl-split/qemu-xen.git
+QEMU_TRADITIONAL_URL ?= 
http://xenbits.xen.org/git-http/people/ianc/libxenctrl-split/qemu-xen-traditional.git
 SEABIOS_UPSTREAM_URL ?= http://xenbits.xen.org/git-http/seabios.git
-MINIOS_UPSTREAM_URL ?= http://xenbits.xen.org/git-http/mini-os.git
+MINIOS_UPSTREAM_URL ?= 
http://xenbits.xen.org/git-http/people/ianc/libxenctrl-split/mini-os.git
 else
 OVMF_UPSTREAM_URL ?= git://xenbits.xen.org/ovmf.git
-QEMU_UPSTREAM_URL ?= git://xenbits.xen.org/qemu-xen.git
-QEMU_TRADITIONAL_URL ?= git://xenbits.xen.org/qemu-xen-traditional.git
+QEMU_UPSTREAM_URL ?= 
git://xenbits.xen.org/people/ianc/libxenctrl-split/qemu-xen.git
+QEMU_TRADITIONAL_URL ?= 
git://xenbits.xen.org/people/ianc/libxenctrl-split/qemu-xen-traditional.git
 SEABIOS_UPSTREAM_URL ?= git://xenbits.xen.org/seabios.git
-MINIOS_UPSTREAM_URL ?= git://xenbits.xen.org/mini-os.git
+MINIOS_UPSTREAM_URL ?= 
git://xenbits.xen.org/people/ianc/libxenctrl-split/mini-os.git
 endif
 OVMF_UPSTREAM_REVISION ?= 52a99493cce88a9d4ec8a02d7f1bd1a1001ce60d
-QEMU_UPSTREAM_REVISION ?= master
-MINIOS_UPSTREAM_REVISION ?= d25773c8afa2f4dbbb466116daeb60159ddd22bd
-# Thu Dec 3 11:23:25 2015 +
-# mini-os: Include libxentoollog with libxc
+QEMU_UPSTREAM_REVISION ?= origin/v8
+MINIOS_UPSTREAM_REVISION ?= origin/v8
+# Mon Nov 23 16:34:31 2015 +
+# Add a .gitignore
 
 SEABIOS_UPSTREAM_REVISION ?= 3403ac4313812752be6e6aac35239ca6888a8cab
 # Mon Dec 28 13:50:41 2015 +0100
@@ -262,7 +262,7 @@ SEABIOS_UPSTREAM_REVISION ?= 
3403ac4313812752be6e6aac35239ca6888a8cab
 ETHERBOOT_NICS ?= rtl8139 8086100e
 
 
-QEMU_TRADITIONAL_REVISION ?= 569eac99e8ddccd15fe78e8a3af5622afe780e3b
+QEMU_TRADITIONAL_REVISION ?= origin/v8
 # Mon Jan 4 15:34:29 2016 +
 # MSI-X: avoid array overrun upon MSI-X table writes
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH QEMU-XEN v8 0/8] Begin to disentangle libxenctrl and provide some stable libraries

2016-01-15 Thread Ian Campbell
We intend to stabilise some parts of the libxenctrl interface by
splitting out some functionality into separate stable libraries.

This is the qemu-xen part of the first phase of that change.

This mail is (or is intended to be) a reply to a "0/"
super-intro mail covering all of the related patch series and which
contains more details.

Ian Campbell (8):
  xen_console: correctly cleanup primary console on teardown.
  xen: Switch to libxenevtchn interface for compat shims.
  xen: Switch to libxengnttab interface for compat shims.
  xen: Switch uses of xc_map_foreign_range into xc_map_foreign_pages
  xen: Switch uses of xc_map_foreign_{pages,bulk} to use
libxenforeignmemory API.
  xen: Use stable library interfaces when they are available.
  xen: domainbuild: reopen libxenctrl interface after forking for domain
watcher.
  xen: make it possible to build without the Xen PV domain builder

 configure|  70 
 hw/block/xen_disk.c  |  38 +--
 hw/char/xen_console.c|  19 +++---
 hw/display/xenfb.c   |  28 
 hw/net/xen_nic.c |  18 +++---
 hw/xen/xen_backend.c |  44 +++--
 hw/xenpv/Makefile.objs   |   4 +-
 hw/xenpv/xen_domainbuild.c   |   9 ++-
 hw/xenpv/xen_machine_pv.c|  15 +++--
 include/hw/xen/xen_backend.h |   5 +-
 include/hw/xen/xen_common.h  | 149 ++-
 xen-common.c |   6 ++
 xen-hvm.c|  39 +--
 xen-mapcache.c   |   6 +-
 14 files changed, 315 insertions(+), 135 deletions(-)

-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH XEN v8 19/29] tools/libs: Clean up hard tabs.

2016-01-15 Thread Ian Campbell
These were wrong in the context of libxc before this code was
extracted, clean them up.

Also add some emacs magic blocks

Signed-off-by: Ian Campbell 
Acked-by: Wei Liu 
---
 tools/libs/call/buffer.c  |  4 ++--
 tools/libs/call/core.c|  2 +-
 tools/libs/call/minios.c  |  4 ++--
 tools/libs/evtchn/minios.c| 28 ++--
 tools/libs/evtchn/netbsd.c| 12 ++--
 tools/libs/foreignmemory/freebsd.c|  2 +-
 tools/libs/foreignmemory/minios.c |  6 +++---
 tools/libs/foreignmemory/netbsd.c |  2 +-
 tools/libs/foreignmemory/private.h|  2 +-
 tools/libs/foreignmemory/solaris.c| 12 +++-
 tools/libs/gnttab/include/xengnttab.h |  4 ++--
 tools/libs/gnttab/linux.c |  6 +++---
 tools/libs/toollog/xtl_logger_stdio.c |  2 +-
 13 files changed, 48 insertions(+), 38 deletions(-)

diff --git a/tools/libs/call/buffer.c b/tools/libs/call/buffer.c
index 1a1b27a..2d8fc29 100644
--- a/tools/libs/call/buffer.c
+++ b/tools/libs/call/buffer.c
@@ -20,7 +20,7 @@
 #include "private.h"
 
 #define DBGPRINTF(_m...) \
-   xtl_log(xcall->logger, XTL_DEBUG, -1, "xencall:buffer", _m)
+xtl_log(xcall->logger, XTL_DEBUG, -1, "xencall:buffer", _m)
 
 #define ROUNDUP(_x,_w) (((unsigned long)(_x)+(1UL<<(_w))-1) & ~((1UL<<(_w))-1))
 
@@ -86,7 +86,7 @@ static int cache_free(xencall_handle *xcall, void *p, size_t 
nr_pages)
 xcall->buffer_current_allocations--;
 
 if ( nr_pages == 1 &&
-xcall->buffer_cache_nr < BUFFER_CACHE_SIZE )
+ xcall->buffer_cache_nr < BUFFER_CACHE_SIZE )
 {
 xcall->buffer_cache[xcall->buffer_cache_nr++] = p;
 rc = 1;
diff --git a/tools/libs/call/core.c b/tools/libs/call/core.c
index a342871..bbf88de 100644
--- a/tools/libs/call/core.c
+++ b/tools/libs/call/core.c
@@ -19,7 +19,7 @@
 
 xencall_handle *xencall_open(xentoollog_logger *logger, unsigned open_flags)
 {
-   xencall_handle *xcall = malloc(sizeof(*xcall));
+xencall_handle *xcall = malloc(sizeof(*xcall));
 int rc;
 
 if (!xcall) return NULL;
diff --git a/tools/libs/call/minios.c b/tools/libs/call/minios.c
index 3bee7be..f04688f 100644
--- a/tools/libs/call/minios.c
+++ b/tools/libs/call/minios.c
@@ -50,8 +50,8 @@ int osdep_hypercall(xencall_handle *xcall, 
privcmd_hypercall_t *hypercall)
 ret = HYPERVISOR_multicall(&call, 1);
 
 if (ret < 0) {
-   errno = -ret;
-   return -1;
+errno = -ret;
+return -1;
 }
 if ((long) call.result < 0) {
 errno = - (long) call.result;
diff --git a/tools/libs/evtchn/minios.c b/tools/libs/evtchn/minios.c
index 773942d..2c89952 100644
--- a/tools/libs/evtchn/minios.c
+++ b/tools/libs/evtchn/minios.c
@@ -103,8 +103,8 @@ int xenevtchn_notify(xenevtchn_handle *xce, evtchn_port_t 
port)
 ret = notify_remote_via_evtchn(port);
 
 if (ret < 0) {
-   errno = -ret;
-   ret = -1;
+errno = -ret;
+ret = -1;
 }
 return ret;
 }
@@ -138,16 +138,16 @@ evtchn_port_or_error_t 
xenevtchn_bind_unbound_port(xenevtchn_handle *xce, uint32
 assert(get_current() == main_thread);
 port_info = port_alloc(fd);
 if (port_info == NULL)
-   return -1;
+return -1;
 
 printf("xenevtchn_bind_unbound_port(%d)", domid);
 ret = evtchn_alloc_unbound(domid, evtchn_handler, (void*)(intptr_t)fd, 
&port);
 printf(" = %d\n", ret);
 
 if (ret < 0) {
-   port_dealloc(port_info);
-   errno = -ret;
-   return -1;
+port_dealloc(port_info);
+errno = -ret;
+return -1;
 }
 port_info->bound = 1;
 port_info->port = port;
@@ -166,16 +166,16 @@ evtchn_port_or_error_t 
xenevtchn_bind_interdomain(xenevtchn_handle *xce, uint32_
 assert(get_current() == main_thread);
 port_info = port_alloc(fd);
 if (port_info == NULL)
-   return -1;
+return -1;
 
 printf("xenevtchn_bind_interdomain(%d, %"PRId32")", domid, remote_port);
 ret = evtchn_bind_interdomain(domid, remote_port, evtchn_handler, 
(void*)(intptr_t)fd, &local_port);
 printf(" = %d\n", ret);
 
 if (ret < 0) {
-   port_dealloc(port_info);
-   errno = -ret;
-   return -1;
+port_dealloc(port_info);
+errno = -ret;
+return -1;
 }
 port_info->bound = 1;
 port_info->port = local_port;
@@ -208,15 +208,15 @@ evtchn_port_or_error_t 
xenevtchn_bind_virq(xenevtchn_handle *xce, unsigned int v
 assert(get_current() == main_thread);
 port_info = port_alloc(fd);
 if (port_info == NULL)
-   return -1;
+return -1;
 
 printf("xenevtchn_bind_virq(%d)", virq);
 port = bind_virq(virq, evtchn_handler, (void*)(intptr_t)fd);
 
 if (port < 0) {
-   port_dealloc(port_info);
-   errno = -port;
-   return -1;
+port_dealloc(port_info);
+errno = -port;
+return -1;
 }
 port_info->bound = 1;
 port_info->port = port;
diff --git a/tools/libs/evtchn/n

[Xen-devel] [PATCH XEN v8 26/29] tools/libs/*: Use O_CLOEXEC on Linux and FreeBSD

2016-01-15 Thread Ian Campbell
In some cases this replaces an FD_CLOEXEC dance, in others it is new.

Linux has had O_CLOEXEC since 2.6.23 (October 2007), so we can rely on
it from Xen 4.7 I think. Some libc headers may still lack the
definition, so we take care of that if need be by defining to 0 (on
the premise that such an old glibc might barf on O_CLOEXEC even if the
kernel may or may not be so old).

All stable versions of FreeBSD support O_CLOEXEC (10.2, 9.3 and 8.4),
and we assume the libc there does too.

Remove various comments about having to take responsibility for this
(since really it is just hygiene, politeness, not a requirement) and
the reasons for using O_CLOEXEC seem pretty straightforward.

Backends for other OSes are untouched.

Signed-off-by: Ian Campbell 
Acked-by: Roger Pau Monné 
Cc: roger@citrix.com
Cc: jbeul...@suse.com
---
v6: New

v7: New, replaces "tools/libs/call: Use O_CLOEXEC when opening
/dev/xen/privcmd on Linux"

v8: Define compat O_CLOEXEC of 0.
---
 tools/libs/call/freebsd.c  | 30 --
 tools/libs/call/linux.c| 38 +++---
 tools/libs/evtchn/freebsd.c|  2 +-
 tools/libs/evtchn/linux.c  |  6 +-
 tools/libs/foreignmemory/freebsd.c | 24 ++--
 tools/libs/foreignmemory/linux.c   | 36 +---
 tools/libs/gnttab/linux.c  |  6 +-
 7 files changed, 41 insertions(+), 101 deletions(-)

diff --git a/tools/libs/call/freebsd.c b/tools/libs/call/freebsd.c
index 2413966..b3cbccd 100644
--- a/tools/libs/call/freebsd.c
+++ b/tools/libs/call/freebsd.c
@@ -35,8 +35,14 @@
 
 int osdep_xencall_open(xencall_handle *xcall)
 {
-int flags, saved_errno;
-int fd = open(PRIVCMD_DEV, O_RDWR);
+int saved_errno;
+int fd = open(PRIVCMD_DEV, O_RDWR|O_CLOEXEC);
+
+/*
+ * This file descriptor is opaque to the caller, thus we are
+ * polite and try and ensure it doesn't propagate (ie leak)
+ * outside the process, by using O_CLOEXEC.
+ */
 
 if ( fd == -1 )
 {
@@ -45,26 +51,6 @@ int osdep_xencall_open(xencall_handle *xcall)
 return -1;
 }
 
-/*
- * Although we return the file handle as the 'xc handle' the API
- * does not specify / guarentee that this integer is in fact
- * a file handle. Thus we must take responsiblity to ensure
- * it doesn't propagate (ie leak) outside the process.
- */
-if ( (flags = fcntl(fd, F_GETFD)) < 0 )
-{
-PERROR("Could not get file handle flags");
-goto error;
-}
-
-flags |= FD_CLOEXEC;
-
-if ( fcntl(fd, F_SETFD, flags) < 0 )
-{
-PERROR("Could not set file handle flags");
-goto error;
-}
-
 xcall->fd = fd;
 return 0;
 
diff --git a/tools/libs/call/linux.c b/tools/libs/call/linux.c
index 651f380..e8e0311 100644
--- a/tools/libs/call/linux.c
+++ b/tools/libs/call/linux.c
@@ -26,15 +26,23 @@
 
 #include "private.h"
 
+#ifndef O_CLOEXEC
+#define O_CLOEXEC 0
+#endif
+
 int osdep_xencall_open(xencall_handle *xcall)
 {
-int flags, saved_errno;
-int fd = open("/dev/xen/privcmd", O_RDWR); /* prefer this newer interface 
*/
+int fd;
+
+/*
+ * Prefer the newer interface.
+ */
+fd = open("/dev/xen/privcmd", O_RDWR|O_CLOEXEC);
 
 if ( fd == -1 && ( errno == ENOENT || errno == ENXIO || errno == ENODEV ))
 {
 /* Fallback to /proc/xen/privcmd */
-fd = open("/proc/xen/privcmd", O_RDWR);
+fd = open("/proc/xen/privcmd", O_RDWR|O_CLOEXEC);
 }
 
 if ( fd == -1 )
@@ -43,32 +51,8 @@ int osdep_xencall_open(xencall_handle *xcall)
 return -1;
 }
 
-/* Although we return the file handle as the 'xc handle' the API
-   does not specify / guarentee that this integer is in fact
-   a file handle. Thus we must take responsiblity to ensure
-   it doesn't propagate (ie leak) outside the process */
-if ( (flags = fcntl(fd, F_GETFD)) < 0 )
-{
-PERROR("Could not get file handle flags");
-goto error;
-}
-
-flags |= FD_CLOEXEC;
-
-if ( fcntl(fd, F_SETFD, flags) < 0 )
-{
-PERROR("Could not set file handle flags");
-goto error;
-}
-
 xcall->fd = fd;
 return 0;
-
- error:
-saved_errno = errno;
-close(fd);
-errno = saved_errno;
-return -1;
 }
 
 int osdep_xencall_close(xencall_handle *xcall)
diff --git a/tools/libs/evtchn/freebsd.c b/tools/libs/evtchn/freebsd.c
index 6479f7c..ddf221d 100644
--- a/tools/libs/evtchn/freebsd.c
+++ b/tools/libs/evtchn/freebsd.c
@@ -32,7 +32,7 @@
 
 int osdep_evtchn_open(xenevtchn_handle *xce)
 {
-int fd = open(EVTCHN_DEV, O_RDWR);
+int fd = open(EVTCHN_DEV, O_RDWR|O_CLOEXEC);
 if ( fd == -1 )
 return -1;
 xce->fd = fd;
diff --git a/tools/libs/evtchn/linux.c b/tools/libs/evtchn/linux.c
index 76cf0ac..0a3c6e1 100644
--- a/tools/libs/evtchn/linux.c
+++ b/tools/libs/evtchn/linux.c
@@ -28,9 +28,13 @@
 
 #include "priva

[Xen-devel] [PATCH XEN v8 23/29] tools/libs/call: Avoid xc_memalign in netbsd and solaris backends

2016-01-15 Thread Ian Campbell
These are already arch specific, so just use the appropriate
interfaces (as determined by looking at the xc_memalign backend).

Signed-off-by: Ian Campbell 
Acked-by: Wei Liu 
---
 tools/libs/call/netbsd.c  | 4 ++--
 tools/libs/call/solaris.c | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/tools/libs/call/netbsd.c b/tools/libs/call/netbsd.c
index 2aa02f1..e96fbf1 100644
--- a/tools/libs/call/netbsd.c
+++ b/tools/libs/call/netbsd.c
@@ -74,8 +74,8 @@ void *osdep_alloc_hypercall_buffer(xencall_handle *xcall, 
size_t npages)
 size_t size = npages * XC_PAGE_SIZE;
 void *p;
 
-p = xc_memalign(xcall, XC_PAGE_SIZE, size);
-if (!p)
+ret = posix_memalign(&p, XC_PAGE_SIZE, size);
+if ( ret != 0 || !p )
 return NULL;
 
 if ( mlock(p, size) < 0 )
diff --git a/tools/libs/call/solaris.c b/tools/libs/call/solaris.c
index 945d867..5aa330e 100644
--- a/tools/libs/call/solaris.c
+++ b/tools/libs/call/solaris.c
@@ -71,7 +71,7 @@ int osdep_xencall_close(xencall_handle *xcall)
 
 void *osdep_alloc_hypercall_buffer(xencall_handle *xcall, size_t npages)
 {
-return xc_memalign(xcall, XC_PAGE_SIZE, npages * XC_PAGE_SIZE);
+return memalign(XC_PAGE_SIZE, npages * XC_PAGE_SIZE);
 }
 
 void osdep_free_hypercall_buffer(xencall_handle *xcall, void *ptr,
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH XEN v8 28/29] tools/libs/*: Introduce APIs to restrict handles to a specific domain.

2016-01-15 Thread Ian Campbell
These are intended to allow user space processes (in particular QEMU)
to lock down all the handles at start of day and then drop the
privileges which would allow them to open any new unrestricted handles
(e.g. setuid or similar). This will reduce the privileges which taking
over such a process would gain an attacker wrt other domains in the
system.

These are currently unimplemented on all platforms, however the API
semantics are defined as the basis for discussion, and so that
consumers can rely on this interface always having been present rather
than requiring compile time API checks.

It is expected that these will be implemented by adding new ioctl
calls on the underlying driver and that the restrictions will be
enforced at the kernel interface layer (most likely by the kernel
itself).

For evtchn, foreignmemory, gnttab and gntshr this is hopefully
reasonably straightforward.

For call it is not so clear cut. Clearly the kernel cannot enforce
these restrictions for hypercalls which are not stable (domctl et al)
so they can never be on the whitelist. It may also be that potential
users would like to restrict the handle further than just a given
target domain, i.e. to a specific set of functionality (e.g. "things a
device model might reasonably do"). I think we will also need some way
to discover whether a given set of interfaces is available to a
restricted handle, in order to support the addition of new
functionality.

Notes:

- On many (all?) platforms libxencall and libxenforeignmemory are
  implemented by the same underlying privcmd driver. The platform
  level ioctl interface should support restricting the handle to only
  one or the other.
- On platforms with multiple privilege mapping ioctl variants should
  consider only allowing the newest/currently preferred one on a
  restricted handle. e.g. on Linux this would allow
  IOCTL_PRIVCMD_MMAPBATCH_V2 but not IOCTL_PRIVCMD_MMAPBATCH. (Of
  course any subsequently introduced _V3 would be subject to
  compatibility concerns)

Signed-off-by: Ian Campbell 
---
v8: New

This applies on top of the Xen portion of "Begin to disentangle
libxenctrl and provide some stable libraries", v7, plus a couple of
minor fixes which will be in v8. All of this can be found in the
"vwip" branch of the tree referenced by that series at
git://xenbits.xen.org/people/ianc/libxenctrl-split/xen.git.
---
 tools/libs/call/core.c |  7 +++
 tools/libs/call/include/xencall.h  | 34 ++
 tools/libs/call/libxencall.map |  2 +
 tools/libs/evtchn/core.c   |  7 +++
 tools/libs/evtchn/include/xenevtchn.h  | 36 +++
 tools/libs/evtchn/libxenevtchn.map |  2 +
 tools/libs/foreignmemory/core.c|  7 +++
 .../libs/foreignmemory/include/xenforeignmemory.h  | 22 +
 tools/libs/foreignmemory/libxenforeignmemory.map   |  3 ++
 tools/libs/gnttab/gntshr_core.c|  8 
 tools/libs/gnttab/gnttab_core.c|  7 +++
 tools/libs/gnttab/include/xengnttab.h  | 52 ++
 tools/libs/gnttab/libxengnttab.map |  8 +++-
 13 files changed, 193 insertions(+), 2 deletions(-)

diff --git a/tools/libs/call/core.c b/tools/libs/call/core.c
index bbf88de..07283da 100644
--- a/tools/libs/call/core.c
+++ b/tools/libs/call/core.c
@@ -14,6 +14,7 @@
  */
 
 #include 
+#include 
 
 #include "private.h"
 
@@ -70,6 +71,12 @@ int xencall_close(xencall_handle *xcall)
 return rc;
 }
 
+int xencall_restrict_target(xencall_handle *xcall, uint32_t domid)
+{
+errno = ENOSYS;
+return -1;
+}
+
 int xencall0(xencall_handle *xcall, unsigned int op)
 {
 privcmd_hypercall_t call = {
diff --git a/tools/libs/call/include/xencall.h 
b/tools/libs/call/include/xencall.h
index 559624a..47c394d 100644
--- a/tools/libs/call/include/xencall.h
+++ b/tools/libs/call/include/xencall.h
@@ -73,6 +73,40 @@ xencall_handle *xencall_open(xentoollog_logger *logger, 
unsigned open_flags);
 int xencall_close(xencall_handle *xcall);
 
 /*
+ * Attempt to restrict the given xcall handle to only be able to
+ * target the given domain.
+ *
+ * On success returns 0, after which only hypercalls which are on a
+ * platform specific whitelist can be called and the arguments will be
+ * audited by the platform to ensure that the target domain is
+ * domid.
+ *
+ * Subsequent attempts to call any hypercall not on the platform
+ * specific whitelist will return -1 setting errno to ENOSYS.
+ *
+ * Subsequent attempts to call any hypercall on the platform specific
+ * whitelist with any other target domain return -1 setting errno to
+ * EPERM.
+ *
+ * These restrictions will be implemented by the platform in a way
+ * which cannot be circumvented by a userspace process. Further
+ * privilege drops (such as using setuid(2) etc) may also be required
+ * to prevent a compromised process from simply opening a second
+ * ha

[Xen-devel] [PATCH XEN v8 21/29] tools/libs/call: Update some log messages to not refer to xc.

2016-01-15 Thread Ian Campbell
Signed-off-by: Ian Campbell 
Acked-by: Wei Liu 
---
 tools/libs/call/linux.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/libs/call/linux.c b/tools/libs/call/linux.c
index 55e1e83..3641e41 100644
--- a/tools/libs/call/linux.c
+++ b/tools/libs/call/linux.c
@@ -94,7 +94,7 @@ void *osdep_alloc_pages(xencall_handle *xcall, size_t npages)
 p = mmap(NULL, size, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_ANONYMOUS|MAP_LOCKED, -1, 0);
 if ( p == MAP_FAILED )
 {
-PERROR("xc_alloc_hypercall_buffer: mmap failed");
+PERROR("alloc_pages: mmap failed");
 return NULL;
 }
 
@@ -103,7 +103,7 @@ void *osdep_alloc_pages(xencall_handle *xcall, size_t 
npages)
 rc = madvise(p, npages * PAGE_SIZE, MADV_DONTFORK);
 if ( rc < 0 )
 {
-PERROR("xc_alloc_hypercall_buffer: madvise failed");
+PERROR("alloc_pages: madvise failed");
 goto out;
 }
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH XEN v8 05/29] tools: Refactor /dev/xen/gnt{dev, shr} wrappers into libxengnttab.

2016-01-15 Thread Ian Campbell
libxengnttab will provide a stable API and ABI for accessing the
grant table devices.

The functions are moved into the xengnt{tab,shr} namespace to make a
clean break from libxc and avoid ambiguity regarding which interfaces
are stable.

All in-tree users are updated to use the new names.

Upon request (via #define XC_WANT_COMPAT_GNTTAB_API) libxenctrl will
provide a compat API for the old names. This is used by qemu-xen for
the time being. qemu-xen-traditional is updated in lockstep.

This leaves a few grant table related functions which go via privcmd
(GNTTABOP) rather than ioctls on the /dev/xen/gnt* devices in
libxenctrl. Specifically:

  - xc_gnttab_get_version
  - xc_gnttab_map_table_v1
  - xc_gnttab_map_table_v2
  - xc_gnttab_op

These functions do not appear to be needed by qemu-dm, qemu-pv
(provision of device model to HVM guests and PV backends respectively)
or by libvchan suggesting they are not needed by non-toolstack uses of
event channels.

The new library uses a version script to ensure that only expected
symbols are exported and to version them such that ABI guarantees can
be kept in the future.

After this change libxenvchan no longer needs to link against
libxenctrl. It still needs xenctrl.h in one file for xen_mb and
friends.

Signed-off-by: Ian Campbell 
Acked-by: Wei Liu 
---

Must be applied with:

 - "qemu-xen-traditional: Use libxengnttab" and a corresponding
   QEMU_TAG update folded here.
 - "mini-os: Include libxengnttab with libxc"" and a corresponding
   bump to MINIOS_UPSTREAM_REVISION folded in here.

v3:
 - Remove SHLIB_libxenctrl from SHDEPS_libxenvchan, and replace with
   SHLIB_libxentoollog.
 - Move to tools/libs/gnttab
 - Adjust for rebase over 31cf2ca75181 "tools/libxc: linux: Don't use
   getpagesize() when unmapping the grants"

v5: Allow _close(NULL).

v6: The extensive API document updates from the previous round of
review have been implemented in "tools/libs/gnttab: Extensive
updates to API documentation." later in the series, so as not to
add further functional changes than already required to this
refactoring patch.

v7: Added mk-headers-$(XEN_TARGET_ARCH) build dependency instead of
open coding the recursion.
s/gnttab_munmap/gnttab_unmap/ and s/gntshr_munmap/gntshr_unshare/
in the API and equivalents in the internals/osdeps etc.

v8: s/EVTCHNOP/GNTTABOP/ cut-&-paste error in commit log.
Remove *.so on clean, add distclean target.
---
 .gitignore |   2 +
 stubdom/Makefile   |  17 +-
 tools/Makefile |   3 +
 tools/Rules.mk |  14 +-
 tools/console/Makefile |   5 +-
 tools/console/daemon/io.c  |  21 +-
 tools/libs/Makefile|   1 +
 tools/libs/evtchn/minios.c |   5 +-
 tools/libs/gnttab/Makefile |  73 +
 tools/libs/gnttab/gntshr_core.c|  95 ++
 .../xc_nognttab.c => libs/gnttab/gntshr_unimp.c}   |  34 ++-
 tools/libs/gnttab/gnttab_core.c| 124 
 tools/libs/gnttab/gnttab_unimp.c   |  89 ++
 tools/libs/gnttab/include/xengnttab.h  | 216 ++
 tools/libs/gnttab/libxengnttab.map |  23 ++
 tools/libs/gnttab/linux.c  | 329 +
 tools/libs/gnttab/minios.c | 117 
 tools/libs/gnttab/private.h|  47 +++
 tools/libvchan/Makefile|   8 +-
 tools/libvchan/init.c  |  26 +-
 tools/libvchan/io.c|   8 +-
 tools/libvchan/libxenvchan.h   |   6 +-
 tools/libxc/Makefile   |  15 +-
 tools/libxc/include/xenctrl.h  | 168 ---
 tools/libxc/include/xenctrl_compat.h   |  48 +++
 tools/libxc/xc_gnttab.c|  53 
 tools/libxc/xc_gnttab_compat.c | 111 +++
 tools/libxc/xc_linux_osdep.c   | 280 --
 tools/libxc/xc_minios.c|  73 -
 tools/libxc/xc_nogntshr.c  |  46 ---
 tools/libxc/xc_private.c   |  80 -
 tools/libxc/xc_private.h   |  24 --
 tools/xenstore/Makefile|   4 +-
 tools/xenstore/xenstored_core.h|   4 +-
 tools/xenstore/xenstored_domain.c  |  24 +-
 tools/xenstore/xenstored_minios.c  |   5 +-
 36 files changed, 1397 insertions(+), 801 deletions(-)
 create mode 100644 tools/libs/gnttab/Makefile
 create mode 100644 tools/libs/gnttab/gntshr_core.c
 rename tools/{libxc/xc_nognttab.c => libs/gnttab/gntshr_unimp.c} (52%)
 create mode 100644 tools/li

[Xen-devel] [PATCH XEN v8 22/29] tools/libs/call: Describe return values and error semantics for xencall*

2016-01-15 Thread Ian Campbell
This behaviour has been confirmed by inspection on:

 - Linux
 - NetBSD & FreeBSD (NB: hcall->retval is the hypercall return value
   only for values >= 0. For negative values the underlying privcmd
   driver translates the value from Xen to {Net,Free}BSD errno space
   and returns it as the result of the ioctl, which becomes
   ret=-1/errno=EFOO in userspace)
 - MiniOS (which takes care of errno in this library)

Signed-off-by: Ian Campbell 
Acked-by: Wei Liu 
Acked-by: Ian Jackson 
Acked-by: Roger Pau Monné 
Cc: Roger Pau Monné 
---
v7: Noted NetBSD behaviour
v6: New patch
---
 tools/libs/call/include/xencall.h | 15 +++
 1 file changed, 15 insertions(+)

diff --git a/tools/libs/call/include/xencall.h 
b/tools/libs/call/include/xencall.h
index 0d91aa8..3f325f0 100644
--- a/tools/libs/call/include/xencall.h
+++ b/tools/libs/call/include/xencall.h
@@ -46,6 +46,21 @@ int xencall_close(xencall_handle *xcall);
 
 /*
  * Call hypercalls with varying numbers of arguments.
+ *
+ * On success the return value of the hypercall is the return value of
+ * the xencall function.  On error these functions set errno and
+ * return -1.
+ *
+ * The errno values will be either:
+ * - The Xen hypercall error return (from xen/include/public/errno.h)
+ *   translated into the corresponding local value for that POSIX error.
+ * - An errno value produced by the OS driver or the library
+ *   implementation. Such values may be defined by POSIX or by the OS.
+ *
+ * Note that under some circumstances it will not be possible to tell
+ * whether an error came from Xen or from the OS/library.
+ *
+ * These functions never log.
  */
 int xencall0(xencall_handle *xcall, unsigned int op);
 int xencall1(xencall_handle *xcall, unsigned int op,
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH XEN v8 07/29] tools: Refactor hypercall calling wrappers into libxencall.

2016-01-15 Thread Ian Campbell
libxencall will provide a stable API and ABI for calling hypercalls
(although those hypercalls themselves may not have a stable API). As
well as the hypercall buffer infrastructure needed in order to safely
provide pointer arguments to hypercalls.

libxenctrl encapsulates a instance of this interface, so users of that
library are not currently subjected to any actual changes. However all
hypercalls made internally by libxc now use the correct interface. It
is expected that most users of this library will be other libraries
providing a higher level interface, rather than applications directly.

Only the basic functionality to allocate hypercall safe memory is
moved, the type safe stuff and bounce buffers remain in libxc.

Note that the functionality to map foreign pages using privcmd is not
yet moved, meaning that an xc_interface will now contain two open
privcmd file descriptors. Foreign memory mapping is logically separate
functionality and will be moved into its own library.

The new library uses a version script to ensure that only expected
symbols are exported and to version them such that ABI guarantees can
be kept in the future.

Signed-off-by: Ian Campbell 
Acked-by: Wei Liu 
---

Must be applied with:
  - "qemu-xen-traditional: Add libxencall to rpath-link" and a
corresponding QEMU_TAG update folded here.
  - "mini-os: Include libxencall with libxc" and a corresponding bump
to MINIOS_UPSTREAM_REVISION folded in here.

v3: Moved to tools/libs/call
Ported new wrappers (altp2m)

v5: Allow _close(NULL).

v6: Use size_t for nr_pages throughout.
Rebase over "libxc: prefer using privcmd character device"

v7: Added mk-headers-$(XEN_TARGET_ARCH) build dependency instead of
open coding the recursion.
Remove incorrect/spurious APP_LDLIBS (which anyway was removed by
a later patch)

v8: Remove *.so on clean, add distclean target.
---
 .gitignore|   2 +
 stubdom/Makefile  |  17 +++-
 tools/Makefile|   1 +
 tools/Rules.mk|   7 +-
 tools/libs/Makefile   |   1 +
 tools/libs/call/Makefile  |  71 ++
 tools/libs/call/buffer.c  | 192 ++
 tools/libs/call/core.c| 147 +
 tools/libs/call/freebsd.c | 140 +++
 tools/libs/call/include/xencall.h |  84 +
 tools/libs/call/libxencall.map|  19 
 tools/libs/call/linux.c   | 138 +++
 tools/libs/call/minios.c  |  81 
 tools/libs/call/netbsd.c  | 121 
 tools/libs/call/private.h |  68 ++
 tools/libs/call/solaris.c |  97 +++
 tools/libxc/Makefile  |   7 +-
 tools/libxc/xc_altp2m.c   |  64 -
 tools/libxc/xc_domain.c   | 105 +++--
 tools/libxc/xc_evtchn.c   |   9 +-
 tools/libxc/xc_flask.c|   8 +-
 tools/libxc/xc_freebsd_osdep.c|  47 --
 tools/libxc/xc_gnttab.c   |   9 +-
 tools/libxc/xc_hcall_buf.c| 138 ++-
 tools/libxc/xc_kexec.c|  36 +++
 tools/libxc/xc_linux_osdep.c  |  49 --
 tools/libxc/xc_minios.c   |  32 ---
 tools/libxc/xc_misc.c |  79 ++--
 tools/libxc/xc_netbsd.c   |  40 
 tools/libxc/xc_private.c  |  64 +
 tools/libxc/xc_private.h  |  76 +--
 tools/libxc/xc_solaris.c  |  16 
 tools/libxc/xc_tmem.c |   7 +-
 tools/misc/Makefile   |   4 +-
 tools/xcutils/Makefile|   2 +-
 tools/xenpaging/Makefile  |   2 +-
 36 files changed, 1348 insertions(+), 632 deletions(-)
 create mode 100644 tools/libs/call/Makefile
 create mode 100644 tools/libs/call/buffer.c
 create mode 100644 tools/libs/call/core.c
 create mode 100644 tools/libs/call/freebsd.c
 create mode 100644 tools/libs/call/include/xencall.h
 create mode 100644 tools/libs/call/libxencall.map
 create mode 100644 tools/libs/call/linux.c
 create mode 100644 tools/libs/call/minios.c
 create mode 100644 tools/libs/call/netbsd.c
 create mode 100644 tools/libs/call/private.h
 create mode 100644 tools/libs/call/solaris.c

diff --git a/.gitignore b/.gitignore
index a117161..55ecfb7 100644
--- a/.gitignore
+++ b/.gitignore
@@ -62,6 +62,7 @@ stubdom/ioemu
 stubdom/ioemu/
 stubdom/libs-*
 stubdom/libxc-*
+stubdom/libxencall-*
 stubdom/libxenevtchn-*
 stubdom/libxengnttab-*
 stubdom/libxentoollog-*
@@ -91,6 +92,7 @@ config/Docs.mk
 tools/libs/toollog/headers.chk
 tools/libs/evtchn/headers.chk
 tools/libs/gnttab/headers.chk
+tools/libs/call/headers.chk
 tools/blktap2/daemon/blktapctrl
 tools/blktap2/drivers/img2qcow
 tools/blktap2/drivers/lock-util
diff --git a/stubdom/Makefile b/stubdom/Makefile
index 2dbf4a8..5bf4ed6 100644
--- a/stubdom/

[Xen-devel] [PATCH XEN v8 06/29] tools/libxc: Remove osdep indirection for privcmd

2016-01-15 Thread Ian Campbell
The alternative backend (a xen-api/xapi shim) is no longer around and
so this stuff is now just baggage which is getting in the way of
refactoring libxenctrl.

Nested virt probably suffices for this use case now.

This was the last component of the osdep infrastructure, so all the
dynamic loading etc stuff all falls away too.

As part of this I was forced to investigate the twisty
xc_map_foreign_* maze, which I have added to the
toolstack-library-apis doc in the hopes of doing something sensible.

NetBSD and Solaris now call xc_map_foreign_bulk_compat directly from
their xc_map_foreign_bulk, which could have been achieved by using
some ifdefs around a renamed function. This will fall out in the wash
when these functions move to their own library.

Signed-off-by: Ian Campbell 
Acked-by: David Scott 
Acked-by: Wei Liu 
---
v7: Stop CCing Dave's dead citrix address.
---
 config/FreeBSD.mk   |   2 -
 config/NetBSD.mk|   3 -
 config/NetBSDRump.mk|   1 -
 config/StdGNU.mk|   1 -
 config/SunOS.mk |   1 -
 tools/libxc/Makefile|  27 ++
 tools/libxc/include/xenctrl.h   |   9 --
 tools/libxc/include/xenctrlosdep.h  | 133 ---
 tools/libxc/xc_foreign_memory.c |  31 +--
 tools/libxc/xc_freebsd_osdep.c  | 100 ++---
 tools/libxc/xc_hcall_buf.c  |   6 +-
 tools/libxc/xc_linux_osdep.c|  88 ++
 tools/libxc/xc_minios.c |  82 +
 tools/libxc/xc_netbsd.c |  90 +++
 tools/libxc/xc_private.c| 174 ++--
 tools/libxc/xc_private.h|  19 ++--
 tools/libxc/xc_solaris.c|  90 +++
 tools/libxc/xenctrl_osdep_ENOSYS.c  | 123 -
 tools/ocaml/libs/xc/xenctrl.ml  |   2 -
 tools/ocaml/libs/xc/xenctrl.mli |   1 -
 tools/ocaml/libs/xc/xenctrl_stubs.c |   7 --
 tools/ocaml/xenstored/domains.ml|   6 +-
 tools/ocaml/xenstored/xenstored.ml  |   5 +-
 23 files changed, 178 insertions(+), 823 deletions(-)
 delete mode 100644 tools/libxc/include/xenctrlosdep.h
 delete mode 100644 tools/libxc/xenctrl_osdep_ENOSYS.c

diff --git a/config/FreeBSD.mk b/config/FreeBSD.mk
index 5a13d607..bb3a5d0 100644
--- a/config/FreeBSD.mk
+++ b/config/FreeBSD.mk
@@ -1,6 +1,4 @@
 include $(XEN_ROOT)/config/StdGNU.mk
 
-DLOPEN_LIBS =
-
 # No wget on FreeBSD base system
 WGET = ftp
diff --git a/config/NetBSD.mk b/config/NetBSD.mk
index 21318d6..cf766e5 100644
--- a/config/NetBSD.mk
+++ b/config/NetBSD.mk
@@ -1,6 +1,3 @@
 include $(XEN_ROOT)/config/StdGNU.mk
 
-# Override settings for this OS
-DLOPEN_LIBS =
-
 WGET = ftp
diff --git a/config/NetBSDRump.mk b/config/NetBSDRump.mk
index 2a87218..74755a1 100644
--- a/config/NetBSDRump.mk
+++ b/config/NetBSDRump.mk
@@ -1,6 +1,5 @@
 include $(XEN_ROOT)/config/StdGNU.mk
 
-DLOPEN_LIBS =
 PTHREAD_LIBS =
 
 WGET = ftp
diff --git a/config/StdGNU.mk b/config/StdGNU.mk
index 129d5c8..39d36b2 100644
--- a/config/StdGNU.mk
+++ b/config/StdGNU.mk
@@ -31,7 +31,6 @@ DEBUG_DIR ?= /usr/lib/debug
 
 SOCKET_LIBS =
 UTIL_LIBS = -lutil
-DLOPEN_LIBS = -ldl
 
 SONAME_LDFLAG = -soname
 SHLIB_LDFLAGS = -shared
diff --git a/config/SunOS.mk b/config/SunOS.mk
index db5e898..86a384d 100644
--- a/config/SunOS.mk
+++ b/config/SunOS.mk
@@ -27,7 +27,6 @@ SunOS_LIBDIR_x86_64 = /usr/sfw/lib/amd64
 SOCKET_LIBS = -lsocket
 PTHREAD_LIBS = -lpthread
 UTIL_LIBS =
-DLOPEN_LIBS = -ldl
 
 SONAME_LDFLAG = -h
 SHLIB_LDFLAGS = -R $(SunOS_LIBDIR) -shared
diff --git a/tools/libxc/Makefile b/tools/libxc/Makefile
index 33d18db..3305fdd 100644
--- a/tools/libxc/Makefile
+++ b/tools/libxc/Makefile
@@ -101,8 +101,6 @@ GUEST_SRCS-y += 
xc_dom_decompress_unsafe_lzo1x.c
 GUEST_SRCS-y += xc_dom_decompress_unsafe_xz.c
 endif
 
-OSDEP_SRCS-y += xenctrl_osdep_ENOSYS.c
-
 -include $(XEN_TARGET_ARCH)/Makefile
 
 CFLAGS   += -Werror -Wmissing-prototypes
@@ -121,11 +119,8 @@ CTRL_PIC_OBJS := $(patsubst %.c,%.opic,$(CTRL_SRCS-y))
 GUEST_LIB_OBJS := $(patsubst %.c,%.o,$(GUEST_SRCS-y))
 GUEST_PIC_OBJS := $(patsubst %.c,%.opic,$(GUEST_SRCS-y))
 
-OSDEP_LIB_OBJS := $(patsubst %.c,%.o,$(OSDEP_SRCS-y))
-OSDEP_PIC_OBJS := $(patsubst %.c,%.opic,$(OSDEP_SRCS-y))
-
-$(CTRL_LIB_OBJS) $(GUEST_LIB_OBJS) $(OSDEP_LIB_OBJS) \
-$(CTRL_PIC_OBJS) $(GUEST_PIC_OBJS) $(OSDEP_PIC_OBJS) : CFLAGS += -include 
$(XEN_ROOT)/tools/config.h
+$(CTRL_LIB_OBJS) $(GUEST_LIB_OBJS) \
+$(CTRL_PIC_OBJS) $(GUEST_PIC_OBJS): CFLAGS += -include 
$(XEN_ROOT)/tools/config.h
 
 $(CTRL_LIB_OBJS) $(CTRL_PIC_OBJS): CFLAGS += $(CFLAGS_libxengnttab) 
$(CFLAGS_libxengntshr)
 
@@ -139,17 +134,13 @@ ifneq ($(nosharedlibs),y)
 LIB += libxenguest.so libxenguest.so.$(MAJOR) libxenguest.so.$(MAJOR).$(MINOR)
 endif
 
-ifneq ($(nosharedlibs),y)
-LIB += xenctrl_osdep_ENOSYS.so
-endif
-
 genpath-target = $(call buildmakevars2header,_paths.h

[Xen-devel] [PATCH XEN v8 15/29] tools/libs/foreignmemory: Support err == NULL to map.

2016-01-15 Thread Ian Campbell
The existing xc_map_foreign_bulk-like interface encourages callers to
miss error checking for partial failure (by forgetting to scan the err
array).

Add support for passing err==NULL which behaves in a
xc_map_foreign_pages-like manner and returns a global error for any
failure.

While documenting this also clarify the overall behaviour and the
behaviour with err!=NULL.

With this the compat wrapper of xc_map_foreign_pages() can be
simplified.

Signed-off-by: Ian Campbell 
Acked-by: Wei Liu 
Acked-by: Ian Jackson 
---
v7: Check for NULL when allocating err.

v6: New
---
 tools/libs/foreignmemory/core.c| 33 +-
 .../libs/foreignmemory/include/xenforeignmemory.h  | 24 ++--
 tools/libxc/xc_foreign_memory.c| 22 +--
 3 files changed, 54 insertions(+), 25 deletions(-)

diff --git a/tools/libs/foreignmemory/core.c b/tools/libs/foreignmemory/core.c
index 21dc7ee..4e0541f 100644
--- a/tools/libs/foreignmemory/core.c
+++ b/tools/libs/foreignmemory/core.c
@@ -14,6 +14,8 @@
  */
 
 #include 
+#include 
+#include 
 
 #include "private.h"
 
@@ -64,7 +66,36 @@ void *xenforeignmemory_map(xenforeignmemory_handle *fmem,
uint32_t dom, int prot,
const xen_pfn_t *arr, int *err, size_t num)
 {
-return osdep_xenforeignmemory_map(fmem, dom, prot, arr, err, num);
+void *ret;
+int *err_to_free = NULL;
+
+if ( err == NULL )
+err = err_to_free = malloc(num * sizeof(int));
+
+if ( err == NULL )
+return NULL;
+
+ret = osdep_xenforeignmemory_map(fmem, dom, prot, arr, err, num);
+
+if ( ret == 0 && err_to_free )
+{
+int i;
+
+for ( i = 0 ; i < num ; i++ )
+{
+if ( err[i] )
+{
+errno = -err[i];
+(void)osdep_xenforeignmemory_unmap(fmem, ret, num);
+ret = NULL;
+break;
+}
+}
+}
+
+free(err_to_free);
+
+return ret;
 }
 
 int xenforeignmemory_unmap(xenforeignmemory_handle *fmem,
diff --git a/tools/libs/foreignmemory/include/xenforeignmemory.h 
b/tools/libs/foreignmemory/include/xenforeignmemory.h
index a6d1bdb..0ec6325 100644
--- a/tools/libs/foreignmemory/include/xenforeignmemory.h
+++ b/tools/libs/foreignmemory/include/xenforeignmemory.h
@@ -80,10 +80,28 @@ int xenforeignmemory_close(xenforeignmemory_handle *fmem);
  *
  * prot is as for mmap(2).
  *
- * Can partially succeed. When a page cannot be mapped, its respective
- * field in @err is set to the corresponding errno value.
+ * @arr is an array of @pages gfns to be mapped linearly in the local
+ * address range. @err is an (optional) output array used to report
+ * per-page errors, as errno values.
  *
- * Returns NULL if no pages can be mapped.
+ * If @err is given (is non-NULL) then the mapping may partially
+ * succeed and return a valid pointer while also using @err to
+ * indicate the success (0) or failure (errno value) of the individual
+ * pages. The global errno thread local variable is not valid in this
+ * case.
+ *
+ * If @err is not given (is NULL) then on failure to map any page any
+ * successful mappings will be undone and NULL will be returned. errno
+ * will be set to correspond to the first failure (which may not be
+ * the most critical).
+ *
+ * It is also possible to return NULL due to a complete failure,
+ * i.e. failure to even attempt the mapping, in this case the global
+ * errno will have been set and the contents of @err (if given) is
+ * invalid.
+ *
+ * Note that it is also possible to return non-NULL with the contents
+ * of @err indicating failure to map every page.
  */
 void *xenforeignmemory_map(xenforeignmemory_handle *fmem, uint32_t dom,
int prot, const xen_pfn_t *arr, int *err,
diff --git a/tools/libxc/xc_foreign_memory.c b/tools/libxc/xc_foreign_memory.c
index 1737c10..4b24388 100644
--- a/tools/libxc/xc_foreign_memory.c
+++ b/tools/libxc/xc_foreign_memory.c
@@ -23,32 +23,12 @@
 void *xc_map_foreign_pages(xc_interface *xch, uint32_t dom, int prot,
const xen_pfn_t *arr, int num)
 {
-void *res;
-int i, *err;
-
 if (num < 0) {
 errno = EINVAL;
 return NULL;
 }
 
-err = malloc(num * sizeof(*err));
-if (!err)
-return NULL;
-
-res = xenforeignmemory_map(xch->fmem, dom, prot, arr, err, num);
-if (res) {
-for (i = 0; i < num; i++) {
-if (err[i]) {
-errno = -err[i];
-munmap(res, num * PAGE_SIZE);
-res = NULL;
-break;
-}
-}
-}
-
-free(err);
-return res;
+return xenforeignmemory_map(xch->fmem, dom, prot, arr, NULL, num);
 }
 
 void *xc_map_foreign_range(xc_interface *xch,
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-deve

[Xen-devel] [PATCH XEN v8 27/29] tools: Update CFLAGS for qemu-xen to allow it to use new libraries

2016-01-15 Thread Ian Campbell
This means adding -L for libxen{evtchn,gnttab,foreignmemory} so that
it can link them directly (rather than using the libxenctrl compat
layer exposed via -rpath-link). Also add -I for libxenforeignmemory.

Signed-off-by: Ian Campbell 
Acked-by: Wei Liu 
---
 tools/Makefile | 4 
 1 file changed, 4 insertions(+)

diff --git a/tools/Makefile b/tools/Makefile
index c03f6db..3575f16 100644
--- a/tools/Makefile
+++ b/tools/Makefile
@@ -255,12 +255,16 @@ subdir-all-qemu-xen-dir: qemu-xen-dir-find
-I$(XEN_ROOT)/tools/libs/toollog/include \
-I$(XEN_ROOT)/tools/libs/evtchn/include \
-I$(XEN_ROOT)/tools/libs/gnttab/include \
+   -I$(XEN_ROOT)/tools/libs/foreignmemory/include \
-I$(XEN_ROOT)/tools/libxc/include \
-I$(XEN_ROOT)/tools/xenstore/include \
-I$(XEN_ROOT)/tools/xenstore/compat/include \
$(EXTRA_CFLAGS_QEMU_XEN)" \
--extra-ldflags="-L$(XEN_ROOT)/tools/libxc \
-L$(XEN_ROOT)/tools/xenstore \
+   -L$(XEN_ROOT)/tools/libs/evtchn \
+   -L$(XEN_ROOT)/tools/libs/gnttab \
+   -L$(XEN_ROOT)/tools/libs/foreignmemory \
-Wl,-rpath-link=$(XEN_ROOT)/tools/libs/toollog \
-Wl,-rpath-link=$(XEN_ROOT)/tools/libs/evtchn \
-Wl,-rpath-link=$(XEN_ROOT)/tools/libs/gnttab \
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH QEMU-XEN v8 6/8] xen: Use stable library interfaces when they are available.

2016-01-15 Thread Ian Campbell
In Xen 4.7 we are refactoring parts libxenctrl into a number of
separate libraries which will provide backward and forward API and ABI
compatiblity.

Specifically libxenevtchn, libxengnttab and libxenforeignmemory.

Previous patches have already laid the groundwork for using these by
switching the existing compatibility shims to reflect the intefaces to
these libraries.

So all which remains is to update configure to detect the libraries
and enable their use. Although they are notionally independent we take
an all or nothing approach to the three libraries since they were
added at the same time.

The only non-obvious bit is that we now open a proper xenforeignmemory
handle for xen_fmem instead of reusing the xen_xc handle.

Build tested with 4.0 .. 4.6 (inclusive) and the patches targetting
4.7 which adds these libraries.

This uses CONFIG_XEN_CTRL_INTERFACE_VERSION == 471 to cover the
introduction of these new interfaces.

Signed-off-by: Ian Campbell 
---

v8: Use CONFIG_XEN_CTRL_INTERFACE_VERSION == 471 for new interfaces.
Dropped Reviewed-by.

v6: Two minor formatting things.
Rebase onto "xen: fix usage of xc_domain_create in domain
builder", required adjusting the configure script changes.

The rebase wasn't entirely trivial (semantically), so dropped
Stefano's reviwed by.

v5: Remove ifdef check when undeffing the compat stuff.
s/now way/no way/

v4: xenforeignmemory_open is now a compat wrapper, so no ifdef.

Simplify configury by asserting that interface version 470 will
always have the libraries (lack of them makes it 460).

Ran checkpatch and fixed everything except:

ERROR: need consistent spacing around '*' (ctx:WxV)
+typedef xc_interface *XenXC;

Which I think is a false +ve.

Simpler configure stuff
---
 configure   | 55 +
 include/hw/xen/xen_common.h | 35 +++--
 2 files changed, 88 insertions(+), 2 deletions(-)

diff --git a/configure b/configure
index 44ac9ab..9ead31d 100755
--- a/configure
+++ b/configure
@@ -1938,6 +1938,7 @@ fi
 
 if test "$xen" != "no" ; then
   xen_libs="-lxenstore -lxenctrl -lxenguest"
+  xen_stable_libs="-lxenforeignmemory -lxengnttab -lxenevtchn"
 
   # First we test whether Xen headers and libraries are available.
   # If no, we are done and there is no Xen support.
@@ -1960,6 +1961,57 @@ EOF
   # Xen unstable
   elif
   cat > $TMPC <
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#if !defined(HVM_MAX_VCPUS)
+# error HVM_MAX_VCPUS not defined
+#endif
+int main(void) {
+  xc_interface *xc = NULL;
+  xenforeignmemory_handle *xfmem;
+  xenevtchn_handle *xe;
+  xengnttab_handle *xg;
+  xen_domain_handle_t handle;
+
+  xs_daemon_open();
+
+  xc = xc_interface_open(0, 0, 0);
+  xc_hvm_set_mem_type(0, 0, HVMMEM_ram_ro, 0, 0);
+  xc_domain_add_to_physmap(0, 0, XENMAPSPACE_gmfn, 0, 0);
+  xc_hvm_inject_msi(xc, 0, 0xf000, 0x);
+  xc_hvm_create_ioreq_server(xc, 0, HVM_IOREQSRV_BUFIOREQ_ATOMIC, NULL);
+  xc_domain_create(xc, 0, handle, 0, NULL, NULL);
+
+  xfmem = xenforeignmemory_open(0, 0);
+  xenforeignmemory_map(xfmem, 0, 0, 0, 0, 0);
+
+  xe = xenevtchn_open(0, 0);
+  xenevtchn_fd(xe);
+
+  xg = xengnttab_open(0, 0);
+  xengnttab_map_grant_ref(xg, 0, 0, 0);
+
+  return 0;
+}
+EOF
+  compile_prog "" "$xen_libs $xen_stable_libs"
+then
+xen_ctrl_version=471
+xen=yes
+  elif
+  cat > $TMPC <
 #include 
 int main(void) {
@@ -2153,6 +2205,9 @@ EOF
   fi
 
   if test "$xen" = yes; then
+if test $xen_ctrl_version -ge 471  ; then
+  libs_softmmu="$xen_stable_libs $libs_softmmu"
+fi
 libs_softmmu="$xen_libs $libs_softmmu"
   fi
 fi
diff --git a/include/hw/xen/xen_common.h b/include/hw/xen/xen_common.h
index 95275b3..19f1577 100644
--- a/include/hw/xen/xen_common.h
+++ b/include/hw/xen/xen_common.h
@@ -6,6 +6,15 @@
 #include 
 #include 
 
+/*
+ * If we have new enough libxenctrl then we do not want/need these compat
+ * interfaces, despite what the user supplied cflags might say. They
+ * must be undefined before including xenctrl.h
+ */
+#undef XC_WANT_COMPAT_EVTCHN_API
+#undef XC_WANT_COMPAT_GNTTAB_API
+#undef XC_WANT_COMPAT_MAP_FOREIGN_API
+
 #include 
 #if CONFIG_XEN_CTRL_INTERFACE_VERSION < 420
 #  include 
@@ -148,8 +157,8 @@ static inline void xs_close(struct xs_handle *xsh)
 }
 
 
-/* Xen 4.1 */
-#else
+/* Xen 4.1 thru 4.6 */
+#elif CONFIG_XEN_CTRL_INTERFACE_VERSION < 471
 
 typedef xc_interface *XenXC;
 typedef xc_interface *xenforeignmemory_handle;
@@ -184,6 +193,28 @@ static inline XenXC xen_xc_interface_open(void *logger, 
void *dombuild_logger,
 
 /* See below for xenforeignmemory_* APIs */
 
+/* FIXME There is no way to have the xen fd */
+static inline int xc_fd(xc_interface *xen_xc)
+{
+return -1;
+}
+#else /* CONFIG_XEN_CTRL_INTERFACE_VERSION >= 471 */
+
+typedef xc_interface *XenXC;
+
+#  define XC_INTERFACE_FMT "%p"
+#  define XC_HANDLER_INITIAL_VALUENULL
+
+#include 

[Xen-devel] [PATCH XEN v8 14/29] tools/libs/foreignmemory: Mention restrictions on fork in docs.

2016-01-15 Thread Ian Campbell
Signed-off-by: Ian Campbell 
---
v6: Also discuss recovering the memory.

v7: Further clarifications regarding forking based on ML discussions.
(Dropped Wei's ack)
---
 .../libs/foreignmemory/include/xenforeignmemory.h  | 33 +-
 1 file changed, 32 insertions(+), 1 deletion(-)

diff --git a/tools/libs/foreignmemory/include/xenforeignmemory.h 
b/tools/libs/foreignmemory/include/xenforeignmemory.h
index 04ff548..a6d1bdb 100644
--- a/tools/libs/foreignmemory/include/xenforeignmemory.h
+++ b/tools/libs/foreignmemory/include/xenforeignmemory.h
@@ -32,13 +32,44 @@ typedef struct xentoollog_logger xentoollog_logger;
 typedef struct xenforeignmemory_handle xenforeignmemory_handle;
 
 /*
- * Return a handle onto the hypercall driver.  Logs errors.
+ * Return a handle onto the foreign memory mapping driver.  Logs errors.
+ *
+ * Note: After fork(2) a child process must not use any opened
+ * foreignmemory handle inherited from their parent, nor access any
+ * grant mapped areas associated with that handle.
+ *
+ * The child must open a new handle if they want to interact with
+ * foreignmemory.
+ *
+ * Calling exec(2) in a child will safely (and reliably) reclaim any
+ * resources which were allocated via a xenforeignmemory_handle in the
+ * parent.
+ *
+ * A child which does not call exec(2) may safely call
+ * xenforeignmemory_close() on a xenforeignmemory_handle inherited
+ * from their parent. This will attempt to reclaim any resources
+ * associated with that handle. Note that in some implementations this
+ * reclamation may not be completely effective, in this case any
+ * affected resources remain allocated.
+ *
+ * Calling xenforeignmemory_close() is the only safe operation on a
+ * xenforeignmemory_handle which has been inherited.
  */
 xenforeignmemory_handle *xenforeignmemory_open(xentoollog_logger *logger,
unsigned open_flags);
 
 /*
  * Close a handle previously allocated with xenforeignmemory_open().
+ *
+ * Under normal circumstances (i.e. not in the child after a fork)
+ * xenforeignmemory_unmap() should be used on all mappings allocated
+ * by xenforeignmemory_map() prior to closing the handle in order to
+ * free up resources associated with those mappings.
+ *
+ * This is the only function which may be safely called on a
+ * xenforeignmemory_handle in a child after a
+ * fork. xenforeignmemory_unmap() must not be called under such
+ * circumstances.
  */
 int xenforeignmemory_close(xenforeignmemory_handle *fmem);
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH XEN v8 18/29] tools/libs/evtchn: Use uint32_t for domid arguments

2016-01-15 Thread Ian Campbell
Signed-off-by: Ian Campbell 
---
v8: New
---
 tools/libs/evtchn/freebsd.c   | 4 ++--
 tools/libs/evtchn/include/xenevtchn.h | 4 ++--
 tools/libs/evtchn/linux.c | 4 ++--
 tools/libs/evtchn/minios.c| 4 ++--
 tools/libs/evtchn/netbsd.c| 4 ++--
 tools/libs/evtchn/solaris.c   | 4 ++--
 6 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/tools/libs/evtchn/freebsd.c b/tools/libs/evtchn/freebsd.c
index 636f052..6479f7c 100644
--- a/tools/libs/evtchn/freebsd.c
+++ b/tools/libs/evtchn/freebsd.c
@@ -62,7 +62,7 @@ int xenevtchn_notify(xenevtchn_handle *xce, evtchn_port_t 
port)
 return ioctl(fd, IOCTL_EVTCHN_NOTIFY, ¬ify);
 }
 
-evtchn_port_or_error_t xenevtchn_bind_unbound_port(xenevtchn_handle *xce, int 
domid)
+evtchn_port_or_error_t xenevtchn_bind_unbound_port(xenevtchn_handle *xce, 
uint32_t domid)
 {
 int ret, fd = xce->fd;
 struct ioctl_evtchn_bind_unbound_port bind;
@@ -74,7 +74,7 @@ evtchn_port_or_error_t 
xenevtchn_bind_unbound_port(xenevtchn_handle *xce, int do
 }
 
 evtchn_port_or_error_t
-xenevtchn_bind_interdomain(xenevtchn_handle *xce, int domid, evtchn_port_t 
remote_port)
+xenevtchn_bind_interdomain(xenevtchn_handle *xce, uint_32 domid, evtchn_port_t 
remote_port)
 {
 int ret, fd = xce->fd;
 struct ioctl_evtchn_bind_interdomain bind;
diff --git a/tools/libs/evtchn/include/xenevtchn.h 
b/tools/libs/evtchn/include/xenevtchn.h
index 60da2a3..428d54c 100644
--- a/tools/libs/evtchn/include/xenevtchn.h
+++ b/tools/libs/evtchn/include/xenevtchn.h
@@ -86,14 +86,14 @@ int xenevtchn_notify(xenevtchn_handle *xce, evtchn_port_t 
port);
  * domain ID, or -1 on failure, in which case errno will be set appropriately.
  */
 evtchn_port_or_error_t
-xenevtchn_bind_unbound_port(xenevtchn_handle *xce, int domid);
+xenevtchn_bind_unbound_port(xenevtchn_handle *xce, uint32_t domid);
 
 /*
  * Returns a new event port bound to the remote port for the given domain ID,
  * or -1 on failure, in which case errno will be set appropriately.
  */
 evtchn_port_or_error_t
-xenevtchn_bind_interdomain(xenevtchn_handle *xce, int domid,
+xenevtchn_bind_interdomain(xenevtchn_handle *xce, uint32_t domid,
evtchn_port_t remote_port);
 
 /*
diff --git a/tools/libs/evtchn/linux.c b/tools/libs/evtchn/linux.c
index 27fd6e9..76cf0ac 100644
--- a/tools/libs/evtchn/linux.c
+++ b/tools/libs/evtchn/linux.c
@@ -61,7 +61,7 @@ int xenevtchn_notify(xenevtchn_handle *xce, evtchn_port_t 
port)
 }
 
 evtchn_port_or_error_t xenevtchn_bind_unbound_port(xenevtchn_handle *xce,
-   int domid)
+   uint32_t domid)
 {
 int fd = xce->fd;
 struct ioctl_evtchn_bind_unbound_port bind;
@@ -72,7 +72,7 @@ evtchn_port_or_error_t 
xenevtchn_bind_unbound_port(xenevtchn_handle *xce,
 }
 
 evtchn_port_or_error_t xenevtchn_bind_interdomain(xenevtchn_handle *xce,
-  int domid,
+  uint32_t domid,
   evtchn_port_t remote_port)
 {
 int fd = xce->fd;
diff --git a/tools/libs/evtchn/minios.c b/tools/libs/evtchn/minios.c
index b839cd0..773942d 100644
--- a/tools/libs/evtchn/minios.c
+++ b/tools/libs/evtchn/minios.c
@@ -128,7 +128,7 @@ static void evtchn_handler(evtchn_port_t port, struct 
pt_regs *regs, void *data)
 wake_up(&event_queue);
 }
 
-evtchn_port_or_error_t xenevtchn_bind_unbound_port(xenevtchn_handle *xce, int 
domid)
+evtchn_port_or_error_t xenevtchn_bind_unbound_port(xenevtchn_handle *xce, 
uint32_t domid)
 {
 int fd = xce->fd;
 struct evtchn_port_info *port_info;
@@ -155,7 +155,7 @@ evtchn_port_or_error_t 
xenevtchn_bind_unbound_port(xenevtchn_handle *xce, int do
 return port;
 }
 
-evtchn_port_or_error_t xenevtchn_bind_interdomain(xenevtchn_handle *xce, int 
domid,
+evtchn_port_or_error_t xenevtchn_bind_interdomain(xenevtchn_handle *xce, 
uint32_t domid,
   evtchn_port_t remote_port)
 {
 int fd = xce->fd;
diff --git a/tools/libs/evtchn/netbsd.c b/tools/libs/evtchn/netbsd.c
index c4123fe..1472ca6 100644
--- a/tools/libs/evtchn/netbsd.c
+++ b/tools/libs/evtchn/netbsd.c
@@ -62,7 +62,7 @@ int xenevtchn_notify(xenevtchn_handle *xce, evtchn_port_t 
port)
 return ioctl(fd, IOCTL_EVTCHN_NOTIFY, ¬ify);
 }
 
-evtchn_port_or_error_t xenevtchn_bind_unbound_port(xenevtchn_handle * xce, int 
domid)
+evtchn_port_or_error_t xenevtchn_bind_unbound_port(xenevtchn_handle * xce, 
uint32_t domid)
 {
 int fd = xce->fd;
 struct ioctl_evtchn_bind_unbound_port bind;
@@ -77,7 +77,7 @@ evtchn_port_or_error_t 
xenevtchn_bind_unbound_port(xenevtchn_handle * xce, int d
return -1;
 }
 
-evtchn_port_or_error_t xenevtchn_bind_interdomain(xenevtchn_handle *xce, int 
domid,
+evtchn_port_or_error_t xenevtchn_bind_interdomain(xenevtchn_handle *xce, 
uint32_t dom

[Xen-devel] [PATCH XEN v8 24/29] tools/libs/call: linux: touch newly allocated pages after madvise lockdown

2016-01-15 Thread Ian Campbell
This avoids a potential issue with a fork after allocation but before
madvise.

Signed-off-by: Ian Campbell 
---
v7: New, replacing "tools/libs/call: linux: avoid forking between mmap
and madvise".
---
 tools/libs/call/linux.c | 14 +-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/tools/libs/call/linux.c b/tools/libs/call/linux.c
index 3641e41..651f380 100644
--- a/tools/libs/call/linux.c
+++ b/tools/libs/call/linux.c
@@ -88,7 +88,7 @@ void *osdep_alloc_pages(xencall_handle *xcall, size_t npages)
 {
 size_t size = npages * PAGE_SIZE;
 void *p;
-int rc, saved_errno;
+int rc, i, saved_errno;
 
 /* Address returned by mmap is page aligned. */
 p = mmap(NULL, size, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_ANONYMOUS|MAP_LOCKED, -1, 0);
@@ -107,6 +107,18 @@ void *osdep_alloc_pages(xencall_handle *xcall, size_t 
npages)
 goto out;
 }
 
+/*
+ * Touch each page in turn to force them to be un-CoWed, in case a
+ * fork happened in another thread at an inopportune moment
+ * above. The madvise() will prevent any subsequent fork calls from
+ * causing the same problem.
+ */
+for ( i = 0; i < npages ; i++ )
+{
+char *c = (char *)p + (i*PAGE_SIZE);
+*c = 0;
+}
+
 return p;
 
 out:
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH QEMU-XEN v8 5/8] xen: Switch uses of xc_map_foreign_{pages, bulk} to use libxenforeignmemory API.

2016-01-15 Thread Ian Campbell
In Xen 4.7 we are refactoring parts libxenctrl into a number of
separate libraries which will provide backward and forward API and ABI
compatiblity.

One such library will be libxenforeignmemory which provides access to
privileged foreign mappings and which will provide an interface
equivalent to xc_map_foreign_{pages,bulk}.

The new xenforeignmemory_map() function behaves like
xc_map_foreign_pages() when the err argument is NULL and like
xc_map_foreign_bulk() when err is non-NULL, which maps into the shim
here onto checking err == NULL and calling the appropriate old
function.

Note that xenforeignmemory_map() takes the number of pages before the
arrays themselves, in order to support potentially future use of
variable-length-arrays in the prototype (in the future, when Xen's
baseline toolchain requirements are new enough to ensure VLAs are
supported).

In preparation for adding support for libxenforeignmemory add support
to the <=4.0 and <=4.6 compat code in xen_common.h to allow us to
switch to using the new API. These shims will disappear for versions
of Xen which include libxenforeignmemory.

Since libxenforeignmemory will have its own handle type but for <= 4.6
the functionality is provided by using a libxenctrl handle we
introduce a new global xen_fmem alongside the existing xen_xc. In fact
we make xen_fmem a pointer to the existing xen_xc, which then works
correctly with both <=4.0 (xc handle is an int) and <=4.6 (xc handle
is a pointer). In the latter case xen_fmem is actually a double
indirect pointer, but it all falls out in the wash.

Unlike libxenctrl libxenforeignmemory has an explicit unmap function,
rather than just specifying that munmap should be used, so the unmap
paths are updated to use xenforeignmemory_unmap, which is a shim for
munmap on these versions of xen. The mappings in xen-hvm.c do not
appear to be unmapped (which makes sense for a qemu-dm process)

In fb_disconnect this results in a change from simply mmap over the
existing mapping (with an implicit munmap) to expliclty unmapping with
xenforeignmemory_unmap and then mapping the required anonymous memory
in the same hole. I don't think this is a problem since any other
thread which was racily touching this region would already be running
the risk of hitting the mapping halfway through the call. If this is
thought to be a problem then we could consider adding an extra API to
the libxenforeignmemory interface to replace a foreign mapping with
anonymous shared memory, but I'd prefer not to.

Signed-off-by: Ian Campbell 
---
v8: Move two copies of compat xenforeignmemory_{open,unmap} to the
common location. Guard with version 471 (which will be added in
the next patch)
v7: Move two copies of compat xenforeignmemory_map to a common location.
Note that the existing xen_domain_create #ifdef will be covered by
a CONFIG_XEN_PV_DOMAIN_BUILD in a later patch, and so is not a
suitable place to put this function.
v6: Handle _pages as well as _bulk, due to changes in the previous
patches, including the dropping of "xen: Switch uses of
xc_map_foreign_pages into xc_map_foreign_bulk" which previous preceded
this patch and the change of "xen: Switch uses of
xc_map_foreign_range into xc_map_foreign_bulk" into "xen: Switch
uses of xc_map_foreign_range into xc_map_foreign_pages".

Handle reordering of arguments to xenforeignmemory_map().

Dropped Stefano's Reviewed-by due to these changes.

v4: Rebase onto "xen_console: correctly cleanup primary console on
teardown."

xenforeignmemory_unmap takes pages not bytes

Compat wrapper for xenforeignmemory_open instead of ifdef in code.

Run check patch and fix most issues. I did not fix:

ERROR: do not initialise globals to 0 or NULL
+xenforeignmemory_handle *xen_fmem = NULL;

=> This is consistent with all of the existing declarations.

ERROR: need consistent spacing around '*' (ctx:WxV)
+typedef xc_interface *xenforeignmemory_handle;

=> I think this is a false +ve since this is a pointer "*" not a multiple "*".

reorder args to xenforeignmemory_map
---
 hw/char/xen_console.c|  8 
 hw/display/xenfb.c   | 17 +
 hw/xen/xen_backend.c |  3 ++-
 include/hw/xen/xen_backend.h |  1 +
 include/hw/xen/xen_common.h  | 25 +
 xen-common.c |  6 ++
 xen-hvm.c| 12 ++--
 xen-mapcache.c   |  6 +++---
 8 files changed, 56 insertions(+), 22 deletions(-)

diff --git a/hw/char/xen_console.c b/hw/char/xen_console.c
index 3e8a57b..b92d0c6 100644
--- a/hw/char/xen_console.c
+++ b/hw/char/xen_console.c
@@ -229,9 +229,9 @@ static int con_initialise(struct XenDevice *xendev)
 
 if (!xendev->dev) {
 xen_pfn_t mfn = con->ring_ref;
-con->sring = xc_map_foreign_pages(xen_xc, con->xendev.dom,
- PROT_READ|PROT_WRITE,
- &mfn, 1);
+con->sr

[Xen-devel] [PATCH QEMU-XEN-TRADITIONAL v8 3/4] qemu-xen-traditional: Add libxencall to rpath-link

2016-01-15 Thread Ian Campbell
libxenctrl links against this library

Signed-off-by: Ian Campbell 
Acked-by: Ian Jackson 
---
v3: Library moved to tools/libs
---
 xen-hooks.mak | 1 +
 1 file changed, 1 insertion(+)

diff --git a/xen-hooks.mak b/xen-hooks.mak
index 179a6b7..229d642 100644
--- a/xen-hooks.mak
+++ b/xen-hooks.mak
@@ -25,6 +25,7 @@ LIBS += -L$(XEN_ROOT)/tools/libs/gnttab -lxengnttab
 LIBS += -L$(XEN_ROOT)/tools/libxc -lxenctrl -lxenguest
 LIBS += -L$(XEN_ROOT)/tools/xenstore -lxenstore
 LIBS += -Wl,-rpath-link=$(XEN_ROOT)/tools/libs/toollog
+LIBS += -Wl,-rpath-link=$(XEN_ROOT)/tools/libs/call
 
 LDFLAGS := $(CFLAGS) $(LDFLAGS)
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH XEN v8 11/29] tools: Refactor foreign memory mapping into libxenforeignmemory

2016-01-15 Thread Ian Campbell
libxenforeignmemory will provide a stable API and ABI for mapping
foreign domain memory (subject to appropriate privileges).

The new library exposes an interface equivalent to
xc_map_foreign_memory_bulk, which all the other
xc_map_foreign_memory_* functions (which remain in libxc) are
implemented in terms of.

Upon request (via #define XC_WANT_COMPAT_MAP_FOREIGN_API) libxenctrl
will provide a compat API for the old names. This is used by qemu-xen
and qemu-trad as well as various in tree things (which required
de-dupping various #includes in some too to get the #define before the
first).

Signed-off-by: Ian Campbell 
Acked-by: Wei Liu 
Acked-by: Ian Jackson 
---
Must be applied with:
  - "qemu-xen-traditional: qemu-xen-traditional: Add
libxenforeignmemory to rpath-link" and a corresponding QEMU_TAG
update folded here.
  - "mini-os: mini-os: Include libxenforeignmemory with libxc" and a
corresponding bump to MINIOS_UPSTREAM_REVISION folded in here.

v5: Allow close(NULL)

v6: Define XC_WANT_COMPAT_MAP_FOREIGN_API for xen-mceinj.c and memshrtool.c
Fix typo in type name in compat.c, and get the functions names to
match.
Use an osdep layer for _map, to allow for future common code.

v7: Correct comment in compat.h #error due to switch to osdep layer
for map in v6
Added mk-headers-$(XEN_TARGET_ARCH) build dependency instead of
open coding the recursion.

v8: Remove *.so on clean, add distclean target.
---
 .gitignore |   2 +
 stubdom/Makefile   |  17 +++-
 tools/Makefile |   2 +
 tools/Rules.mk |  11 ++-
 tools/console/daemon/utils.c   |   1 -
 tools/console/daemon/utils.h   |   1 +
 tools/libs/Makefile|   1 +
 tools/libs/foreignmemory/Makefile  |  71 ++
 tools/libs/foreignmemory/compat.c  |  72 ++
 tools/libs/foreignmemory/core.c|  84 +
 .../foreignmemory/freebsd.c}   |  33 +++
 .../libs/foreignmemory/include/xenforeignmemory.h  |  71 ++
 tools/libs/foreignmemory/libxenforeignmemory.map   |   7 ++
 .../foreignmemory/linux.c} |  70 ++
 tools/libs/foreignmemory/minios.c  |  62 
 tools/libs/foreignmemory/netbsd.c  | 105 +
 tools/libs/foreignmemory/private.h |  53 +++
 tools/libs/foreignmemory/solaris.c |  93 ++
 tools/libxc/Makefile   |   8 +-
 tools/libxc/include/xenctrl.h  |  26 -
 tools/libxc/include/xenctrl_compat.h   |  36 +++
 tools/libxc/xc_foreign_memory.c|  49 +-
 tools/libxc/xc_minios.c|  29 --
 tools/libxc/xc_netbsd.c|  73 --
 tools/libxc/xc_private.c   |  13 +--
 tools/libxc/xc_private.h   |  11 ++-
 tools/libxc/xc_solaris.c   |  73 --
 tools/libxc/xc_sr_restore.c|   4 +-
 tools/libxc/xc_sr_save.c   |   4 +-
 tools/libxc/xg_private.h   |   3 +-
 tools/libxl/libxl_internal.h   |   1 +
 tools/misc/xen-mfndump.c   |   1 +
 tools/ocaml/libs/xc/xenctrl_stubs.c|   1 +
 tools/python/xen/lowlevel/xc/xc.c  |   2 +
 tools/tests/mce-test/tools/xen-mceinj.c|   1 +
 tools/tests/mem-sharing/memshrtool.c   |   1 +
 tools/xenmon/xenbaked.c|   1 +
 tools/xenpaging/pagein.c   |   1 -
 tools/xenpaging/xenpaging.c|   1 -
 tools/xenpaging/xenpaging.h|   2 +
 tools/xenstore/xenstored_core.c|   1 -
 tools/xenstore/xenstored_core.h|   1 +
 tools/xentrace/xenctx.c|   3 +-
 tools/xentrace/xentrace.c  |   1 +
 44 files changed, 770 insertions(+), 333 deletions(-)
 create mode 100644 tools/libs/foreignmemory/Makefile
 create mode 100644 tools/libs/foreignmemory/compat.c
 create mode 100644 tools/libs/foreignmemory/core.c
 rename tools/{libxc/xc_freebsd_osdep.c => libs/foreignmemory/freebsd.c} (76%)
 create mode 100644 tools/libs/foreignmemory/include/xenforeignmemory.h
 create mode 100644 tools/libs/foreignmemory/libxenforeignmemory.map
 rename tools/{libxc/xc_linux_osdep.c => libs/foreignmemory/linux.c} (83%)
 create mode 100644 tools/libs/foreignmemory/minios.c
 create mode 100644 tools/libs/foreignmemory/netbsd.c
 create mode 100644 tools/libs/foreignmemory/private.h
 create mode 100644 tools/lib

[Xen-devel] [PATCH QEMU-XEN v8 2/8] xen: Switch to libxenevtchn interface for compat shims.

2016-01-15 Thread Ian Campbell
In Xen 4.7 we are refactoring parts libxenctrl into a number of
separate libraries which will provide backward and forward API and ABI
compatiblity.

One such library will be libxenevtchn which provides access to event
channels.

In preparation for this switch the compatibility layer in xen_common.h
(which support building with older versions of Xen) to use what will
be the new library API. This means that the evtchn shim will disappear
for versions of Xen which include libxenevtchn.

To simplify things for the <= 4.0.0 support we wrap the int fd in a
malloc(sizeof int) such that the handle is always a pointer. This
leads to less typedef headaches and the need for
XC_HANDLER_INITIAL_VALUE etc for these interfaces.

Note that this patch does not add any support for actually using
libxenevtchn, it just adjusts the existing shims.

Note that xc_evtchn_alloc_unbound functionality remains in libxenctrl,
since that functionality is not exposed by /dev/xen/evtchn.

Signed-off-by: Ian Campbell 
Reviewed-by: Stefano Stabellini 
---
v4: Ran checkpatch, fixed all errors
Allocate correct size for handle (i.e. not size of the ptr)
---
 hw/xen/xen_backend.c | 31 ---
 include/hw/xen/xen_backend.h |  2 +-
 include/hw/xen/xen_common.h  | 44 ++--
 xen-hvm.c| 25 +
 4 files changed, 64 insertions(+), 38 deletions(-)

diff --git a/hw/xen/xen_backend.c b/hw/xen/xen_backend.c
index 2510e2e..ae2a1f0 100644
--- a/hw/xen/xen_backend.c
+++ b/hw/xen/xen_backend.c
@@ -243,19 +243,19 @@ static struct XenDevice *xen_be_get_xendev(const char 
*type, int dom, int dev,
 xendev->debug  = debug;
 xendev->local_port = -1;
 
-xendev->evtchndev = xen_xc_evtchn_open(NULL, 0);
-if (xendev->evtchndev == XC_HANDLER_INITIAL_VALUE) {
+xendev->evtchndev = xenevtchn_open(NULL, 0);
+if (xendev->evtchndev == NULL) {
 xen_be_printf(NULL, 0, "can't open evtchn device\n");
 g_free(xendev);
 return NULL;
 }
-fcntl(xc_evtchn_fd(xendev->evtchndev), F_SETFD, FD_CLOEXEC);
+fcntl(xenevtchn_fd(xendev->evtchndev), F_SETFD, FD_CLOEXEC);
 
 if (ops->flags & DEVOPS_FLAG_NEED_GNTDEV) {
 xendev->gnttabdev = xen_xc_gnttab_open(NULL, 0);
 if (xendev->gnttabdev == XC_HANDLER_INITIAL_VALUE) {
 xen_be_printf(NULL, 0, "can't open gnttab device\n");
-xc_evtchn_close(xendev->evtchndev);
+xenevtchn_close(xendev->evtchndev);
 g_free(xendev);
 return NULL;
 }
@@ -306,8 +306,8 @@ static struct XenDevice *xen_be_del_xendev(int dom, int dev)
 g_free(xendev->fe);
 }
 
-if (xendev->evtchndev != XC_HANDLER_INITIAL_VALUE) {
-xc_evtchn_close(xendev->evtchndev);
+if (xendev->evtchndev != NULL) {
+xenevtchn_close(xendev->evtchndev);
 }
 if (xendev->gnttabdev != XC_HANDLER_INITIAL_VALUE) {
 xc_gnttab_close(xendev->gnttabdev);
@@ -691,13 +691,14 @@ static void xen_be_evtchn_event(void *opaque)
 struct XenDevice *xendev = opaque;
 evtchn_port_t port;
 
-port = xc_evtchn_pending(xendev->evtchndev);
+port = xenevtchn_pending(xendev->evtchndev);
 if (port != xendev->local_port) {
-xen_be_printf(xendev, 0, "xc_evtchn_pending returned %d (expected 
%d)\n",
+xen_be_printf(xendev, 0,
+  "xenevtchn_pending returned %d (expected %d)\n",
   port, xendev->local_port);
 return;
 }
-xc_evtchn_unmask(xendev->evtchndev, port);
+xenevtchn_unmask(xendev->evtchndev, port);
 
 if (xendev->ops->event) {
 xendev->ops->event(xendev);
@@ -740,14 +741,14 @@ int xen_be_bind_evtchn(struct XenDevice *xendev)
 if (xendev->local_port != -1) {
 return 0;
 }
-xendev->local_port = xc_evtchn_bind_interdomain
+xendev->local_port = xenevtchn_bind_interdomain
 (xendev->evtchndev, xendev->dom, xendev->remote_port);
 if (xendev->local_port == -1) {
-xen_be_printf(xendev, 0, "xc_evtchn_bind_interdomain failed\n");
+xen_be_printf(xendev, 0, "xenevtchn_bind_interdomain failed\n");
 return -1;
 }
 xen_be_printf(xendev, 2, "bind evtchn port %d\n", xendev->local_port);
-qemu_set_fd_handler(xc_evtchn_fd(xendev->evtchndev),
+qemu_set_fd_handler(xenevtchn_fd(xendev->evtchndev),
 xen_be_evtchn_event, NULL, xendev);
 return 0;
 }
@@ -757,15 +758,15 @@ void xen_be_unbind_evtchn(struct XenDevice *xendev)
 if (xendev->local_port == -1) {
 return;
 }
-qemu_set_fd_handler(xc_evtchn_fd(xendev->evtchndev), NULL, NULL, NULL);
-xc_evtchn_unbind(xendev->evtchndev, xendev->local_port);
+qemu_set_fd_handler(xenevtchn_fd(xendev->evtchndev), NULL, NULL, NULL);
+xenevtchn_unbind(xendev->evtchndev, xendev->local_port);
 xen_be_printf(xendev, 2, "unbind evtc

[Xen-devel] [PATCH QEMU-XEN v8 8/8] xen: make it possible to build without the Xen PV domain builder

2016-01-15 Thread Ian Campbell
Until the previous patch this relied on xc_fd(), which was only
implemented for Xen 4.0 and earlier.

Given this wasn't working since Xen 4.0 I have marked this as disabled
by default.

Removing this support drops the use of a bunch of symbols from
libxenctrl, specifically:

  - xc_domain_create
  - xc_domain_destroy
  - xc_domain_getinfo
  - xc_domain_max_vcpus
  - xc_domain_setmaxmem
  - xc_domain_unpause
  - xc_evtchn_alloc_unbound
  - xc_linux_build

This is another step towards only using Xen libraries which provide a
stable inteface.

Signed-off-by: Ian Campbell 
Reviewed-by: Stefano Stabellini 
---
v6: Rebase onto "xen: fix usage of xc_domain_create in domain
builder" (trivial, Reviewed-by retained)

v5: XEN_CREATE entirely wihtin CONFIG_XEN_PV_DOMAIN_BUILD ifdef.
Simplify configure'ry.

v4: Fixed all checkpatch errors.
Disabled by default.
---
 configure   | 15 +++
 hw/xenpv/Makefile.objs  |  4 +++-
 hw/xenpv/xen_machine_pv.c   | 15 +++
 include/hw/xen/xen_common.h |  2 ++
 4 files changed, 31 insertions(+), 5 deletions(-)

diff --git a/configure b/configure
index 9ead31d..3506e44 100755
--- a/configure
+++ b/configure
@@ -250,6 +250,7 @@ vnc_jpeg=""
 vnc_png=""
 xen=""
 xen_ctrl_version=""
+xen_pv_domain_build="no"
 xen_pci_passthrough=""
 linux_aio=""
 cap_ng=""
@@ -927,6 +928,10 @@ for opt do
   ;;
   --enable-xen-pci-passthrough) xen_pci_passthrough="yes"
   ;;
+  --disable-xen-pv-domain-build) xen_pv_domain_build="no"
+  ;;
+  --enable-xen-pv-domain-build) xen_pv_domain_build="yes"
+  ;;
   --disable-brlapi) brlapi="no"
   ;;
   --enable-brlapi) brlapi="yes"
@@ -2229,6 +2234,12 @@ if test "$xen_pci_passthrough" != "no"; then
   fi
 fi
 
+if test "$xen_pv_domain_build" = "yes" &&
+   test "$xen" != "yes"; then
+error_exit "User requested Xen PV domain builder support" \
+  "which requires Xen support."
+fi
+
 ##
 # libtool probe
 
@@ -4848,6 +4859,7 @@ fi
 echo "xen support   $xen"
 if test "$xen" = "yes" ; then
   echo "xen ctrl version  $xen_ctrl_version"
+  echo "pv dom build  $xen_pv_domain_build"
 fi
 echo "brlapi support$brlapi"
 echo "bluez  support$bluez"
@@ -5219,6 +5231,9 @@ fi
 if test "$xen" = "yes" ; then
   echo "CONFIG_XEN_BACKEND=y" >> $config_host_mak
   echo "CONFIG_XEN_CTRL_INTERFACE_VERSION=$xen_ctrl_version" >> 
$config_host_mak
+  if test "$xen_pv_domain_build" = "yes" ; then
+echo "CONFIG_XEN_PV_DOMAIN_BUILD=y" >> $config_host_mak
+  fi
 fi
 if test "$linux_aio" = "yes" ; then
   echo "CONFIG_LINUX_AIO=y" >> $config_host_mak
diff --git a/hw/xenpv/Makefile.objs b/hw/xenpv/Makefile.objs
index 49f6e9e..bbf5873 100644
--- a/hw/xenpv/Makefile.objs
+++ b/hw/xenpv/Makefile.objs
@@ -1,2 +1,4 @@
 # Xen PV machine support
-obj-$(CONFIG_XEN) += xen_domainbuild.o xen_machine_pv.o
+obj-$(CONFIG_XEN) += xen_machine_pv.o
+# Xen PV machine builder support
+obj-$(CONFIG_XEN_PV_DOMAIN_BUILD) += xen_domainbuild.o
diff --git a/hw/xenpv/xen_machine_pv.c b/hw/xenpv/xen_machine_pv.c
index 23d6ef0..3250b94 100644
--- a/hw/xenpv/xen_machine_pv.c
+++ b/hw/xenpv/xen_machine_pv.c
@@ -30,9 +30,6 @@
 
 static void xen_init_pv(MachineState *machine)
 {
-const char *kernel_filename = machine->kernel_filename;
-const char *kernel_cmdline = machine->kernel_cmdline;
-const char *initrd_filename = machine->initrd_filename;
 DriveInfo *dinfo;
 int i;
 
@@ -46,17 +43,27 @@ static void xen_init_pv(MachineState *machine)
 case XEN_ATTACH:
 /* nothing to do, xend handles everything */
 break;
-case XEN_CREATE:
+#ifdef CONFIG_XEN_PV_DOMAIN_BUILD
+case XEN_CREATE: {
+const char *kernel_filename = machine->kernel_filename;
+const char *kernel_cmdline = machine->kernel_cmdline;
+const char *initrd_filename = machine->initrd_filename;
 if (xen_domain_build_pv(kernel_filename, initrd_filename,
 kernel_cmdline) < 0) {
 fprintf(stderr, "xen pv domain creation failed\n");
 exit(1);
 }
 break;
+}
+#endif
 case XEN_EMULATE:
 fprintf(stderr, "xen emulation not implemented (yet)\n");
 exit(1);
 break;
+default:
+fprintf(stderr, "unhandled xen_mode %d\n", xen_mode);
+exit(1);
+break;
 }
 
 xen_be_register("console", &xen_console_ops);
diff --git a/include/hw/xen/xen_common.h b/include/hw/xen/xen_common.h
index be7a915..0d83891 100644
--- a/include/hw/xen/xen_common.h
+++ b/include/hw/xen/xen_common.h
@@ -505,6 +505,7 @@ static inline int xen_xc_domain_add_to_physmap(XenXC xch, 
uint32_t domid,
 }
 #endif
 
+#ifdef CONFIG_XEN_PV_DOMAIN_BUILD
 #if CONFIG_XEN_CTRL_INTERFACE_VERSION < 470
 static inline int xen_domain_create(XenXC xc, uint32_t ssidref,
 xen_domain_handle_t handle, uint32_t flags,
@@ -520,6 +521,7 @@ static inline int xen_domain_crea

[Xen-devel] [PATCH QEMU-XEN-TRADITIONAL v8 4/4] qemu-xen-traditional: Add libxenforeignmemory to rpath-link

2016-01-15 Thread Ian Campbell
libxenctrl links against this library.

Also, request the compat xc_map_foreign API from libxc.

Signed-off-by: Ian Campbell 
Acked-by: Ian Jackson 
---
v3: Library moved to tools/libs/
---
 xen-hooks.mak | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/xen-hooks.mak b/xen-hooks.mak
index 229d642..c1ea4be 100644
--- a/xen-hooks.mak
+++ b/xen-hooks.mak
@@ -1,6 +1,7 @@
 CPPFLAGS+= -I$(XEN_ROOT)/tools/libs/toollog/include
 CPPFLAGS+= -I$(XEN_ROOT)/tools/libs/evtchn/include
 CPPFLAGS+= -I$(XEN_ROOT)/tools/libs/gnttab/include
+CPPFLAGS+= -DXC_WANT_COMPAT_MAP_FOREIGN_API
 CPPFLAGS+= -I$(XEN_ROOT)/tools/libxc/include
 CPPFLAGS+= -I$(XEN_ROOT)/tools/xenstore/include
 CPPFLAGS+= -I$(XEN_ROOT)/tools/include
@@ -26,6 +27,7 @@ LIBS += -L$(XEN_ROOT)/tools/libxc -lxenctrl -lxenguest
 LIBS += -L$(XEN_ROOT)/tools/xenstore -lxenstore
 LIBS += -Wl,-rpath-link=$(XEN_ROOT)/tools/libs/toollog
 LIBS += -Wl,-rpath-link=$(XEN_ROOT)/tools/libs/call
+LIBS += -Wl,-rpath-link=$(XEN_ROOT)/tools/libs/foreignmemory
 
 LDFLAGS := $(CFLAGS) $(LDFLAGS)
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH XEN v8 25/29] tools/libs/{call, evtchn}: Document requirements around forking.

2016-01-15 Thread Ian Campbell
Much like for gnttab and foreignmemory xencall hypercall buffers need
care.

Evtchn is a bit simpler (no magic mappings) but may not work from
parent + child simultaneously, document "parent only" since it is
consistent with the others.

Signed-off-by: Ian Campbell 
---
v7: New
---
 tools/libs/call/include/xencall.h | 28 
 tools/libs/evtchn/include/xenevtchn.h | 23 +++
 2 files changed, 47 insertions(+), 4 deletions(-)

diff --git a/tools/libs/call/include/xencall.h 
b/tools/libs/call/include/xencall.h
index 3f325f0..559624a 100644
--- a/tools/libs/call/include/xencall.h
+++ b/tools/libs/call/include/xencall.h
@@ -36,11 +36,39 @@ typedef struct xencall_handle xencall_handle;
 
 /*
  * Return a handle onto the hypercall driver.  Logs errors.
+ * *
+ * Note: After fork(2) a child process must not use any opened
+ * xencall handle inherited from their parent, nor access any
+ * hypercall argument buffers associated with that handle.
+ *
+ * The child must open a new handle if they want to interact with
+ * xencall.
+ *
+ * Calling exec(2) in a child will safely (and reliably) reclaim any
+ * resources which were allocated via a xencall_handle in the parent.
+ *
+ * A child which does not call exec(2) may safely call xencall_close()
+ * on a xencall_handle inherited from their parent. This will attempt
+ * to reclaim any resources associated with that handle. Note that in
+ * some implementations this reclamation may not be completely
+ * effective, in this case any affected resources remain allocated.
+ *
+ * Calling xencall_close() is the only safe operation on a
+ * xencall_handle which has been inherited.
  */
 xencall_handle *xencall_open(xentoollog_logger *logger, unsigned open_flags);
 
 /*
  * Close a handle previously allocated with xencall_open().
+ *
+ * Under normal circumstances (i.e. not in the child after a fork) any
+ * allocated hypercall argument buffers should be freed using the
+ * appropriate xencall_free_*() prior to closing the handle in order
+ * to free up resources associated with those mappings.
+ *
+ * This is the only function which may be safely called on a
+ * xencall_handle in a child after a fork. xencall_free_*() must not
+ * be called under such circumstances.
  */
 int xencall_close(xencall_handle *xcall);
 
diff --git a/tools/libs/evtchn/include/xenevtchn.h 
b/tools/libs/evtchn/include/xenevtchn.h
index 428d54c..4d26161 100644
--- a/tools/libs/evtchn/include/xenevtchn.h
+++ b/tools/libs/evtchn/include/xenevtchn.h
@@ -45,10 +45,25 @@ typedef struct xentoollog_logger xentoollog_logger;
  * Return a handle to the event channel driver, or NULL on failure, in
  * which case errno will be set appropriately.
  *
- * Note:
- * After fork a child process must not use any opened evtchn
- * handle inherited from their parent. They must open a new handle if
- * they want to interact with evtchn.
+ * Note: After fork(2) a child process must not use any opened evtchn
+ * handle inherited from their parent, nor access any grant mapped
+ * areas associated with that handle.
+ *
+ * The child must open a new handle if they want to interact with
+ * evtchn.
+ *
+ * Calling exec(2) in a child will safely (and reliably) reclaim any
+ * allocated resources via a xenevtchn_handle in the parent.
+ *
+ * A child which does not call exec(2) may safely call
+ * xenevtchn_close() on a xenevtchn_handle inherited from their
+ * parent. This will attempt to reclaim any resources associated with
+ * that handle. Note that in some implementations this reclamation may
+ * not be completely effective, in this case any affected resources
+ * remain allocated.
+ *
+ * Calling xenevtchn_close() is the only safe operation on a
+ * xenevtchn_handle which has been inherited.
  */
 /* Currently no flags are defined */
 xenevtchn_handle *xenevtchn_open(xentoollog_logger *logger, unsigned 
open_flags);
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH MINI-OS v8 4/4] mini-os: Include libxenforeignmemory with libxc

2016-01-15 Thread Ian Campbell
libxenforeignmemory has just been split out from libxc. From mini-os's
point of view we don't care about the distinction, so keep things
simple by just including libxenforeignmemory if libxc is enabled.

Signed-off-by: Ian Campbell 
Acked-by: Samuel Thibault 
Acked-by: Wei Liu 
---
v2: Adjust for libs/$lib layout.
---
 Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Makefile b/Makefile
index c900540..cfe015a 100644
--- a/Makefile
+++ b/Makefile
@@ -169,6 +169,7 @@ APP_LDLIBS += 
-L$(XEN_ROOT)/stubdom/libs-$(MINIOS_TARGET_ARCH)/toollog -whole-ar
 APP_LDLIBS += -L$(XEN_ROOT)/stubdom/libs-$(MINIOS_TARGET_ARCH)/evtchn 
-whole-archive -lxenevtchn -no-whole-archive
 APP_LDLIBS += -L$(XEN_ROOT)/stubdom/libs-$(MINIOS_TARGET_ARCH)/gnttab 
-whole-archive -lxengnttab -no-whole-archive
 APP_LDLIBS += -L$(XEN_ROOT)/stubdom/libs-$(MINIOS_TARGET_ARCH)/call 
-whole-archive -lxencall -no-whole-archive
+APP_LDLIBS += -L$(XEN_ROOT)/stubdom/libs-$(MINIOS_TARGET_ARCH)/foreignmemory 
-whole-archive -lxenforeignmemory -no-whole-archive
 APP_LDLIBS += -L$(XEN_ROOT)/stubdom/libxc-$(MINIOS_TARGET_ARCH) -whole-archive 
-lxenguest -lxenctrl -no-whole-archive
 endif
 APP_LDLIBS += -lpci
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH XEN v8 20/29] tools/libs/gnttab: Extensive updates to API documentation.

2016-01-15 Thread Ian Campbell
In particular around error handling, behaviour on fork and the unmap
notification mechanism.

Behaviour of xengnttab_map_*grant_refs and xengntshr_share_pages on
partial failure has been confirmed/inferred (by inspection) on Linux
and Mini-os (the only two known implementations. Likewise the
behaviour of the notification mechanism has been confirmed/inferred
(by inspection) of the Linux implementation (currently the only
implementation) and libvchan (primary known user).

These updates are not folded into "tools: Refactor
/dev/xen/gnt{dev,shr} wrappers into libxengnttab." to try and reduce
the amount of non-movement changes in that patch.

While I'm not convinced by javadoc/doxygen cause the existing comments
which appear to use that syntax to have the appropriate /** marker.

Also fix a typo in a code comment.

Signed-off-by: Ian Campbell 
Reviewed-by: Daniel De Graaf 
Cc: Daniel De Graaf 
---

v7: Typo.
Updates based on Ian Jackson's review.
Even more updates to the discussion of forking based on ML
discussion.
v6: Rewrapped a comment.
Incorported much review from Ian on the API, retitled (was:
"tools/libs/gnttab: Review and update doc comments.") and dropped
acks
---
 tools/libs/gnttab/include/xengnttab.h | 201 +-
 tools/libs/gnttab/linux.c |  12 +-
 2 files changed, 181 insertions(+), 32 deletions(-)

diff --git a/tools/libs/gnttab/include/xengnttab.h 
b/tools/libs/gnttab/include/xengnttab.h
index 1e07672..7bf8462 100644
--- a/tools/libs/gnttab/include/xengnttab.h
+++ b/tools/libs/gnttab/include/xengnttab.h
@@ -31,28 +31,124 @@
 typedef struct xentoollog_logger xentoollog_logger;
 
 /*
+ * PRODUCING AND CONSUMING GRANT REFERENCES
+ * 
+ *
+ * The xengnttab library contains two distinct interfaces, each with
+ * their own distinct handle type and entry points. The represent the
+ * two sides of the grant table interface, producer (gntshr) and
+ * consumer (gnttab).
+ *
+ * The xengnttab_* interfaces take a xengnttab_handle and provide
+ * mechanisms for consuming (i.e. mapping or copying to/from) grant
+ * references provided by a peer.
+ *
+ * The xengntshr_* interfaces take a xengntshr_handle and provide a
+ * mechanism to produce grantable memory and grant references to that
+ * memory, which can be handed to some peer.
+ *
+ * UNMAP NOTIFICATION
+ * ==
+ *
+ * The xengnt{tab,shr}_*_notify interfaces implement a cooperative
+ * interface which is intended to allow the underlying kernel
+ * interfaces to attempt to notify the peer to perform graceful
+ * teardown upon failure (i.e. crash or exit) of the process on their
+ * end.
+ *
+ * These interfaces operate on a single page only and are intended for
+ * use on the main shared-ring page of a protocol. It is assumed that
+ * on teardown both ends would automatically teardown all grants
+ * associated with the protocol in addition to the shared ring itself.
+ *
+ * Each end is able to optionally nominate a byte offset within the
+ * shared page or an event channel or both. On exit of the process the
+ * underlying kernel driver will zero the byte at the given offset and
+ * signal the event channel.
+ *
+ * The event channel can be the same event channel used for regular
+ * ring progress notifications, or may be a dedicated event channel.
+ *
+ * Both ends may share the same notification byte offset within the
+ * shared page, or may have dedicated "client" and "server" status
+ * bytes.
+ *
+ * Since the byte is cleared on shutdown the protocol must use 0 as
+ * the "closed/dead" status, but is permitted to use any other non-0
+ * values to indicate various other "live" states (waiting for
+ * connection, connected, etc).
+ *
+ * Both ends are permitted to modify (including clear) their
+ * respective status bytes and to signal the event channel themselves
+ * from userspace.
+ *
+ * Depending on the mechanisms which have been registered an
+ * the peer may receive a shutdown notification as:
+ *
+ *   - An event channel notification on a dedicated event channel
+ *   - Observation of the other ends's status byte being cleared
+ * (whether in response to an explicit notification or in the
+ * course of normal operation).
+ *
+ * The mechanism should be defined as part of the specific ring
+ * protocol.
+ *
+ * Upon receiving notification of the peer is expected to teardown any
+ * resources (and in particular any grant mappings) in a timely
+ * manner.
+ *
+ * NOTE: this protocol is intended to allow for better error behaviour
+ * and recovery between two cooperating peers. It does not cover the
+ * case of a malivious peer who may continue to hold resources open.
+ */
+
+/*
  * Grant Table Interface (making use of grants from other domains)
  */
 
 typedef struct xengntdev_handle xengnttab_handle;
 
 /*
- * Note:
- * After fork a child process must not use any opened xc gnttab
- * handle inherited from their parent. T

[Xen-devel] [PATCH MINI-OS v8 0/4] Begin to disentangle libxenctrl and provide some stable libraries

2016-01-15 Thread Ian Campbell
We intend to stabilise some parts of the libxenctrl interface by
splitting out some functionality into separate stable libraries.

This is the mini-os part of the first phase of that change.

This mail is (or is intended to be) a reply to a "0/"
super-intro mail covering all of the related patch series and which
contains more details.

Ian Campbell (4):
  mini-os: Include libxenevtchn with libxc
  mini-os: Include libxengnttab with libxc
  mini-os: Include libxencall with libxc
  mini-os: Include libxenforeignmemory with libxc

 Makefile | 4 
 1 file changed, 4 insertions(+)

-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH QEMU-XEN v8 4/8] xen: Switch uses of xc_map_foreign_range into xc_map_foreign_pages

2016-01-15 Thread Ian Campbell
In Xen 4.7 we are refactoring parts libxenctrl into a number of
separate libraries which will provide backward and forward API and ABI
compatiblity.

One such library will be libxenforeignmemory which provides access to
privileged foreign mappings and which will provide an interface
equivalent to xc_map_foreign_{pages,bulk}.

In preparation for this switch all uses of xc_map_foreign_range to
xc_map_foreign_pages. This is trivial because size was always
XC_PAGE_SIZE so the necessary adjustments are trivial:

  * Pass &mfn (an array of length 1) instead of mfn. The function
takes a pointer to const, so there is no possibily of mfn changing
due to this change.
  * Pass nr_pages=1 instead of size=XC_PAGE_SIZE

There is one wrinkle in xen_console.c:con_initialise() where
con->ring_ref is an int but can in some code paths (when !xendev->dev)
be treated as an mfn. I think this is an existing latent truncation
hazard on platforms where xen_pfn_t is 64-bit and int is 32-bit (e.g.
amd64, both arm* variants). I'm unsure under what circumstances
xendev->dev can be NULL or if anything elsewhere ensures the value
fits into an int. For now I just use a temporary xen_pfn_t to in
effect upcast the pointer from int* to xen_pfn_t*.

In xenfb.c:common_bind we now explicitly launder the mfn into a
xen_pfn_t, so it has the correct type to be passed to
xc_map_foreign_pages and doesn't provoke warnings on 32-bit x86.

Signed-off-by: Ian Campbell 
Reviewed-by: Stefano Stabellini 
---
v7: Correct type mismatch (int64 vs xen_pfn_t) on 32 bit x86 in
xenfb.c, by explicitly laundering via a xen_pfn_t variable.

v6: Switch to xc_map_foreign_pages rather than _bulk. Switching to
_bulk without checking the value of err[0] risked missing errors.
The new xenforeignmemory_map coming later in this series will
DTRT with err==NULL.

Dropped Stefano's Reviewed-by due to this change.

v4: Ran checkpatch, fixed all errors
Mention the const-ness of the mfn array in the commit log

fixup!xen: Switch uses of xc_map_foreign_range into xc_map_foreign_pages
---
 hw/char/xen_console.c |  8 
 hw/display/xenfb.c| 15 ---
 xen-hvm.c | 14 +++---
 3 files changed, 19 insertions(+), 18 deletions(-)

diff --git a/hw/char/xen_console.c b/hw/char/xen_console.c
index ac1b324..3e8a57b 100644
--- a/hw/char/xen_console.c
+++ b/hw/char/xen_console.c
@@ -228,10 +228,10 @@ static int con_initialise(struct XenDevice *xendev)
con->buffer.max_capacity = limit;
 
 if (!xendev->dev) {
-con->sring = xc_map_foreign_range(xen_xc, con->xendev.dom,
-  XC_PAGE_SIZE,
-  PROT_READ|PROT_WRITE,
-  con->ring_ref);
+xen_pfn_t mfn = con->ring_ref;
+con->sring = xc_map_foreign_pages(xen_xc, con->xendev.dom,
+ PROT_READ|PROT_WRITE,
+ &mfn, 1);
 } else {
 con->sring = xengnttab_map_grant_ref(xendev->gnttabdev, 
con->xendev.dom,
  con->ring_ref,
diff --git a/hw/display/xenfb.c b/hw/display/xenfb.c
index 4e2a27a..bb8f6b3 100644
--- a/hw/display/xenfb.c
+++ b/hw/display/xenfb.c
@@ -95,23 +95,24 @@ struct XenFB {
 
 static int common_bind(struct common *c)
 {
-uint64_t mfn;
+uint64_t val;
+xen_pfn_t mfn;
 
-if (xenstore_read_fe_uint64(&c->xendev, "page-ref", &mfn) == -1)
+if (xenstore_read_fe_uint64(&c->xendev, "page-ref", &val) == -1)
return -1;
-assert(mfn == (xen_pfn_t)mfn);
+mfn = (xen_pfn_t)val;
+assert(val == mfn);
 
 if (xenstore_read_fe_int(&c->xendev, "event-channel", 
&c->xendev.remote_port) == -1)
return -1;
 
-c->page = xc_map_foreign_range(xen_xc, c->xendev.dom,
-  XC_PAGE_SIZE,
-  PROT_READ | PROT_WRITE, mfn);
+c->page = xc_map_foreign_pages(xen_xc, c->xendev.dom,
+   PROT_READ | PROT_WRITE, &mfn, 1);
 if (c->page == NULL)
return -1;
 
 xen_be_bind_evtchn(&c->xendev);
-xen_be_printf(&c->xendev, 1, "ring mfn %"PRIx64", remote-port %d, 
local-port %d\n",
+xen_be_printf(&c->xendev, 1, "ring mfn %"PRI_xen_pfn", remote-port %d, 
local-port %d\n",
  mfn, c->xendev.remote_port, c->xendev.local_port);
 
 return 0;
diff --git a/xen-hvm.c b/xen-hvm.c
index 6227e17..1f729f6 100644
--- a/xen-hvm.c
+++ b/xen-hvm.c
@@ -1247,8 +1247,9 @@ int xen_hvm_init(PCMachineState *pcms,
 DPRINTF("buffered io page at pfn %lx\n", bufioreq_pfn);
 DPRINTF("buffered io evtchn is %x\n", bufioreq_evtchn);
 
-state->shared_page = xc_map_foreign_range(xen_xc, xen_domid, XC_PAGE_SIZE,
-  PROT_READ|PROT_WRITE, ioreq_pfn);
+state->shared_page = xc_map_foreign_pages(xen_xc, xen_domid,
+  

[Xen-devel] [PATCH MINI-OS v8 1/4] mini-os: Include libxenevtchn with libxc

2016-01-15 Thread Ian Campbell
libxenevtchn has just been split out from libxc. From mini-os's point
of view we don't care about the distinction, so keep things simple by
just including libxenevtchn if libxc is enabled.

Signed-off-by: Ian Campbell 
Acked-by: Samuel Thibault 
Acked-by: Wei Liu 
---
v2: Adjust for libs/$lib layout.
---
 Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Makefile b/Makefile
index daee46c..d1d8dc4 100644
--- a/Makefile
+++ b/Makefile
@@ -166,6 +166,7 @@ OBJS := $(filter-out $(OBJ_DIR)/lwip%.o $(LWO), $(OBJS))
 ifeq ($(libc),y)
 ifeq ($(CONFIG_XC),y)
 APP_LDLIBS += -L$(XEN_ROOT)/stubdom/libs-$(MINIOS_TARGET_ARCH)/toollog 
-whole-archive -lxentoollog -no-whole-archive
+APP_LDLIBS += -L$(XEN_ROOT)/stubdom/libs-$(MINIOS_TARGET_ARCH)/evtchn 
-whole-archive -lxenevtchn -no-whole-archive
 APP_LDLIBS += -L$(XEN_ROOT)/stubdom/libxc-$(MINIOS_TARGET_ARCH) -whole-archive 
-lxenguest -lxenctrl -no-whole-archive
 endif
 APP_LDLIBS += -lpci
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH QEMU-XEN v8 7/8] xen: domainbuild: reopen libxenctrl interface after forking for domain watcher.

2016-01-15 Thread Ian Campbell
Using an existing libxenctrl handle after a fork was never
particularly safe (especially if foreign mappings existed at the time
of the fork) and the xc fd has been unavailable for many releases.

Reopen the handle after fork and therefore do away with xc_fd().

Signed-off-by: Ian Campbell 
Acked-by: Stefano Stabellini 
---
The fact that xc_fd hasn't been useful since at least Xen 4.1 makes me
question the utility of this domainbuild in QEMU. Perhaps we should
just nuke it?
---
 hw/xenpv/xen_domainbuild.c  |  9 ++---
 include/hw/xen/xen_common.h | 17 -
 2 files changed, 6 insertions(+), 20 deletions(-)

diff --git a/hw/xenpv/xen_domainbuild.c b/hw/xenpv/xen_domainbuild.c
index ac0e5ac..f9be029 100644
--- a/hw/xenpv/xen_domainbuild.c
+++ b/hw/xenpv/xen_domainbuild.c
@@ -174,12 +174,15 @@ static int xen_domain_watcher(void)
 for (i = 3; i < n; i++) {
 if (i == fd[0])
 continue;
-if (i == xc_fd(xen_xc)) {
-continue;
-}
 close(i);
 }
 
+/*
+ * Reopen xc interface, since the original is unsafe after fork
+ * and was closed above.
+ */
+xen_xc = xc_interface_open(0, 0, 0);
+
 /* ignore term signals */
 signal(SIGINT,  SIG_IGN);
 signal(SIGTERM, SIG_IGN);
diff --git a/include/hw/xen/xen_common.h b/include/hw/xen/xen_common.h
index 19f1577..be7a915 100644
--- a/include/hw/xen/xen_common.h
+++ b/include/hw/xen/xen_common.h
@@ -116,12 +116,6 @@ static inline XenXC xen_xc_interface_open(void *logger, 
void *dombuild_logger,
 
 /* See below for xenforeignmemory_* APIs */
 
-static inline int xc_fd(int xen_xc)
-{
-return xen_xc;
-}
-
-
 static inline int xc_domain_populate_physmap_exact
 (XenXC xc_handle, uint32_t domid, unsigned long nr_extents,
  unsigned int extent_order, unsigned int mem_flags, xen_pfn_t 
*extent_start)
@@ -193,11 +187,6 @@ static inline XenXC xen_xc_interface_open(void *logger, 
void *dombuild_logger,
 
 /* See below for xenforeignmemory_* APIs */
 
-/* FIXME There is no way to have the xen fd */
-static inline int xc_fd(xc_interface *xen_xc)
-{
-return -1;
-}
 #else /* CONFIG_XEN_CTRL_INTERFACE_VERSION >= 471 */
 
 typedef xc_interface *XenXC;
@@ -214,12 +203,6 @@ static inline XenXC xen_xc_interface_open(void *logger, 
void *dombuild_logger,
 {
 return xc_interface_open(logger, dombuild_logger, open_flags);
 }
-
-/* FIXME There is now way to have the xen fd */
-static inline int xc_fd(xc_interface *xen_xc)
-{
-return -1;
-}
 #endif
 
 /* Xen before 4.2 */
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH QEMU-XEN v8 3/8] xen: Switch to libxengnttab interface for compat shims.

2016-01-15 Thread Ian Campbell
In Xen 4.7 we are refactoring parts libxenctrl into a number of
separate libraries which will provide backward and forward API and ABI
compatiblity.

One such library will be libxengnttab which provides access to grant
tables.

In preparation for this switch the compatibility layer in xen_common.h
(which support building with older versions of Xen) to use what will
be the new library API. This means that the gnttab shim will disappear
for versions of Xen which include libxengnttab.

To simplify things for the <= 4.0.0 support we wrap the int fd in a
malloc(sizeof int) such that the handle is always a pointer. This
leads to less typedef headaches and the need for
XC_HANDLER_INITIAL_VALUE etc for these interfaces.

Note that this patch does not add any support for actually using
libxengnttab, it just adjusts the existing shims.

Signed-off-by: Ian Campbell 
Reviewed-by: Stefano Stabellini 
---
v4: Ran checkpatch, fixed all errors
Allocate correct size for handle (i.e. not size of the ptr)
Rebase onto "xen_console: correctly cleanup primary console on
teardown."

v5: Rebase over b4f72e31b924 "hw/net/xen_nic.c: Free 'netdev->txs'
when map 'netdev->rxs' fails" which added a new gnttab_munmap.

v7: s/gnttab_munmap/gnttab_unmap/ to reflect change in the library.
---
 hw/block/xen_disk.c  | 38 --
 hw/char/xen_console.c|  4 ++--
 hw/net/xen_nic.c | 18 +-
 hw/xen/xen_backend.c | 10 +-
 include/hw/xen/xen_backend.h |  2 +-
 include/hw/xen/xen_common.h  | 42 --
 6 files changed, 69 insertions(+), 45 deletions(-)

diff --git a/hw/block/xen_disk.c b/hw/block/xen_disk.c
index a48e726..326c948 100644
--- a/hw/block/xen_disk.c
+++ b/hw/block/xen_disk.c
@@ -171,11 +171,11 @@ static gint int_cmp(gconstpointer a, gconstpointer b, 
gpointer user_data)
 static void destroy_grant(gpointer pgnt)
 {
 PersistentGrant *grant = pgnt;
-XenGnttab gnt = grant->blkdev->xendev.gnttabdev;
+xengnttab_handle *gnt = grant->blkdev->xendev.gnttabdev;
 
-if (xc_gnttab_munmap(gnt, grant->page, 1) != 0) {
+if (xengnttab_unmap(gnt, grant->page, 1) != 0) {
 xen_be_printf(&grant->blkdev->xendev, 0,
-  "xc_gnttab_munmap failed: %s\n",
+  "xengnttab_unmap failed: %s\n",
   strerror(errno));
 }
 grant->blkdev->persistent_gnt_count--;
@@ -188,11 +188,11 @@ static void remove_persistent_region(gpointer data, 
gpointer dev)
 {
 PersistentRegion *region = data;
 struct XenBlkDev *blkdev = dev;
-XenGnttab gnt = blkdev->xendev.gnttabdev;
+xengnttab_handle *gnt = blkdev->xendev.gnttabdev;
 
-if (xc_gnttab_munmap(gnt, region->addr, region->num) != 0) {
+if (xengnttab_unmap(gnt, region->addr, region->num) != 0) {
 xen_be_printf(&blkdev->xendev, 0,
-  "xc_gnttab_munmap region %p failed: %s\n",
+  "xengnttab_unmap region %p failed: %s\n",
   region->addr, strerror(errno));
 }
 xen_be_printf(&blkdev->xendev, 3,
@@ -327,7 +327,7 @@ err:
 
 static void ioreq_unmap(struct ioreq *ioreq)
 {
-XenGnttab gnt = ioreq->blkdev->xendev.gnttabdev;
+xengnttab_handle *gnt = ioreq->blkdev->xendev.gnttabdev;
 int i;
 
 if (ioreq->num_unmap == 0 || ioreq->mapped == 0) {
@@ -337,8 +337,9 @@ static void ioreq_unmap(struct ioreq *ioreq)
 if (!ioreq->pages) {
 return;
 }
-if (xc_gnttab_munmap(gnt, ioreq->pages, ioreq->num_unmap) != 0) {
-xen_be_printf(&ioreq->blkdev->xendev, 0, "xc_gnttab_munmap failed: 
%s\n",
+if (xengnttab_unmap(gnt, ioreq->pages, ioreq->num_unmap) != 0) {
+xen_be_printf(&ioreq->blkdev->xendev, 0,
+  "xengnttab_unmap failed: %s\n",
   strerror(errno));
 }
 ioreq->blkdev->cnt_map -= ioreq->num_unmap;
@@ -348,8 +349,9 @@ static void ioreq_unmap(struct ioreq *ioreq)
 if (!ioreq->page[i]) {
 continue;
 }
-if (xc_gnttab_munmap(gnt, ioreq->page[i], 1) != 0) {
-xen_be_printf(&ioreq->blkdev->xendev, 0, "xc_gnttab_munmap 
failed: %s\n",
+if (xengnttab_unmap(gnt, ioreq->page[i], 1) != 0) {
+xen_be_printf(&ioreq->blkdev->xendev, 0,
+  "xengnttab_unmap failed: %s\n",
   strerror(errno));
 }
 ioreq->blkdev->cnt_map--;
@@ -361,7 +363,7 @@ static void ioreq_unmap(struct ioreq *ioreq)
 
 static int ioreq_map(struct ioreq *ioreq)
 {
-XenGnttab gnt = ioreq->blkdev->xendev.gnttabdev;
+xengnttab_handle *gnt = ioreq->blkdev->xendev.gnttabdev;
 uint32_t domids[BLKIF_MAX_SEGMENTS_PER_REQUEST];
 uint32_t refs[BLKIF_MAX_SEGMENTS_PER_REQUEST];
 void *page[BLKIF_MAX_SEGMENTS_PER_REQUEST];
@@ -412,7 +414,7 @@ static int

[Xen-devel] [PATCH QEMU-XEN v8 1/8] xen_console: correctly cleanup primary console on teardown.

2016-01-15 Thread Ian Campbell
All of the work in con_disconnect applies to the primary console case
(when xendev->dev is NULL). Therefore remove the early check and bail
and allow it to fall through. All of the existing code is correctly
conditional already.

The ->dev and ->gnttabdev handles are either both set or neither. For
consistency with con_initialise() with to the former here too.

With this con_initialise and con_disconnect now mirror each other.

Fix up a hard tab in the function while editing.

Signed-off-by: Ian Campbell 
Reviewed-by: Stefano Stabellini 

---
v4: New patch based on feedback to "xen: Switch uses of
xc_map_foreign_bulk to use libxenforeignmemory API."
---
 hw/char/xen_console.c | 7 ++-
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/hw/char/xen_console.c b/hw/char/xen_console.c
index eb7f450..63ade33 100644
--- a/hw/char/xen_console.c
+++ b/hw/char/xen_console.c
@@ -265,9 +265,6 @@ static void con_disconnect(struct XenDevice *xendev)
 {
 struct XenConsole *con = container_of(xendev, struct XenConsole, xendev);
 
-if (!xendev->dev) {
-return;
-}
 if (con->chr) {
 qemu_chr_add_handlers(con->chr, NULL, NULL, NULL, NULL);
 qemu_chr_fe_release(con->chr);
@@ -275,12 +272,12 @@ static void con_disconnect(struct XenDevice *xendev)
 xen_be_unbind_evtchn(&con->xendev);
 
 if (con->sring) {
-if (!xendev->gnttabdev) {
+if (!xendev->dev) {
 munmap(con->sring, XC_PAGE_SIZE);
 } else {
 xc_gnttab_munmap(xendev->gnttabdev, con->sring, 1);
 }
-   con->sring = NULL;
+con->sring = NULL;
 }
 }
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH QEMU-XEN-TRADITIONAL v8 0/4] Begin to disentangle libxenctrl and provide some stable libraries

2016-01-15 Thread Ian Campbell
We intend to stabilise some parts of the libxenctrl interface by
splitting out some functionality into separate stable libraries.

This is the qemu-xen-traditional part of the first phase of that change.

This mail is (or is intended to be) a reply to a "0/"
super-intro mail covering all of the related patch series and which
contains more details.

Ian Campbell (4):
  qemu-xen-traditional: Use libxenevtchn
  qemu-xen-traditional: Use libxengnttab
  qemu-xen-traditional: Add libxencall to rpath-link
  qemu-xen-traditional: Add libxenforeignmemory to rpath-link

 hw/xen_backend.c  | 30 +++---
 hw/xen_backend.h  |  4 ++--
 hw/xen_common.h   |  2 ++
 hw/xen_console.c  |  4 ++--
 hw/xen_disk.c | 24 
 i386-dm/helper2.c | 19 ++-
 xen-hooks.mak |  7 +++
 7 files changed, 50 insertions(+), 40 deletions(-)

-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH MINI-OS v8 2/4] mini-os: Include libxengnttab with libxc

2016-01-15 Thread Ian Campbell
libxengnttab has just been split out from libxc. From mini-os's point
of view we don't care about the distinction, so keep things simple by
just including libxengnttab if libxc is enabled.

Signed-off-by: Ian Campbell 
Acked-by: Samuel Thibault 
Acked-by: Wei Liu 
---
v2: Adjust for libs/$lib layout.
---
 Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Makefile b/Makefile
index d1d8dc4..521f647 100644
--- a/Makefile
+++ b/Makefile
@@ -167,6 +167,7 @@ ifeq ($(libc),y)
 ifeq ($(CONFIG_XC),y)
 APP_LDLIBS += -L$(XEN_ROOT)/stubdom/libs-$(MINIOS_TARGET_ARCH)/toollog 
-whole-archive -lxentoollog -no-whole-archive
 APP_LDLIBS += -L$(XEN_ROOT)/stubdom/libs-$(MINIOS_TARGET_ARCH)/evtchn 
-whole-archive -lxenevtchn -no-whole-archive
+APP_LDLIBS += -L$(XEN_ROOT)/stubdom/libs-$(MINIOS_TARGET_ARCH)/gnttab 
-whole-archive -lxengnttab -no-whole-archive
 APP_LDLIBS += -L$(XEN_ROOT)/stubdom/libxc-$(MINIOS_TARGET_ARCH) -whole-archive 
-lxenguest -lxenctrl -no-whole-archive
 endif
 APP_LDLIBS += -lpci
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH QEMU-XEN-TRADITIONAL v8 1/4] qemu-xen-traditional: Use libxenevtchn

2016-01-15 Thread Ian Campbell
/dev/xen/evtchn related wrappers have been moved out of libxenctrl
into their own library.

Note that i386-dm/helper2.c's xc_interface * was always really an
xc_evtchn *, it's just they used to be typedefs to the same thing...

Signed-off-by: Ian Campbell 
Acked-by: Ian Jackson 
---
v3: Library moved to tools/libs/
---
 hw/xen_backend.c  | 26 +-
 hw/xen_backend.h  |  2 +-
 hw/xen_common.h   |  1 +
 i386-dm/helper2.c | 19 ++-
 xen-hooks.mak |  2 ++
 5 files changed, 27 insertions(+), 23 deletions(-)

diff --git a/hw/xen_backend.c b/hw/xen_backend.c
index 92b3506..40f6887 100644
--- a/hw/xen_backend.c
+++ b/hw/xen_backend.c
@@ -208,19 +208,19 @@ static struct XenDevice *xen_be_get_xendev(const char 
*type, int dom, int dev,
 xendev->debug  = debug;
 xendev->local_port = -1;
 
-xendev->evtchndev = xc_evtchn_open(NULL, 0);
+xendev->evtchndev = xenevtchn_open(NULL, 0);
 if (xendev->evtchndev == NULL) {
xen_be_printf(NULL, 0, "can't open evtchn device\n");
qemu_free(xendev);
return NULL;
 }
-fcntl(xc_evtchn_fd(xendev->evtchndev), F_SETFD, FD_CLOEXEC);
+fcntl(xenevtchn_fd(xendev->evtchndev), F_SETFD, FD_CLOEXEC);
 
 if (ops->flags & DEVOPS_FLAG_NEED_GNTDEV) {
xendev->gnttabdev = xc_gnttab_open(NULL, 0);
if (xendev->gnttabdev == NULL) {
xen_be_printf(NULL, 0, "can't open gnttab device\n");
-   xc_evtchn_close(xendev->evtchndev);
+   xenevtchn_close(xendev->evtchndev);
qemu_free(xendev);
return NULL;
}
@@ -268,7 +268,7 @@ static struct XenDevice *xen_be_del_xendev(int dom, int dev)
}
 
if (xendev->evtchndev != NULL)
-   xc_evtchn_close(xendev->evtchndev);
+   xenevtchn_close(xendev->evtchndev);
if (xendev->gnttabdev != NULL)
xc_gnttab_close(xendev->gnttabdev);
 
@@ -630,13 +630,13 @@ static void xen_be_evtchn_event(void *opaque)
 struct XenDevice *xendev = opaque;
 evtchn_port_t port;
 
-port = xc_evtchn_pending(xendev->evtchndev);
+port = xenevtchn_pending(xendev->evtchndev);
 if (port != xendev->local_port) {
-   xen_be_printf(xendev, 0, "xc_evtchn_pending returned %d (expected 
%d)\n",
+   xen_be_printf(xendev, 0, "xenevtchn_pending returned %d (expected 
%d)\n",
  port, xendev->local_port);
return;
 }
-xc_evtchn_unmask(xendev->evtchndev, port);
+xenevtchn_unmask(xendev->evtchndev, port);
 
 if (xendev->ops->event)
xendev->ops->event(xendev);
@@ -683,14 +683,14 @@ int xen_be_bind_evtchn(struct XenDevice *xendev)
 {
 if (xendev->local_port != -1)
return 0;
-xendev->local_port = xc_evtchn_bind_interdomain
+xendev->local_port = xenevtchn_bind_interdomain
(xendev->evtchndev, xendev->dom, xendev->remote_port);
 if (xendev->local_port == -1) {
-   xen_be_printf(xendev, 0, "xc_evtchn_bind_interdomain failed\n");
+   xen_be_printf(xendev, 0, "xenevtchn_bind_interdomain failed\n");
return -1;
 }
 xen_be_printf(xendev, 2, "bind evtchn port %d\n", xendev->local_port);
-qemu_set_fd_handler(xc_evtchn_fd(xendev->evtchndev),
+qemu_set_fd_handler(xenevtchn_fd(xendev->evtchndev),
xen_be_evtchn_event, NULL, xendev);
 return 0;
 }
@@ -699,15 +699,15 @@ void xen_be_unbind_evtchn(struct XenDevice *xendev)
 {
 if (xendev->local_port == -1)
return;
-qemu_set_fd_handler(xc_evtchn_fd(xendev->evtchndev), NULL, NULL, NULL);
-xc_evtchn_unbind(xendev->evtchndev, xendev->local_port);
+qemu_set_fd_handler(xenevtchn_fd(xendev->evtchndev), NULL, NULL, NULL);
+xenevtchn_unbind(xendev->evtchndev, xendev->local_port);
 xen_be_printf(xendev, 2, "unbind evtchn port %d\n", xendev->local_port);
 xendev->local_port = -1;
 }
 
 int xen_be_send_notify(struct XenDevice *xendev)
 {
-return xc_evtchn_notify(xendev->evtchndev, xendev->local_port);
+return xenevtchn_notify(xendev->evtchndev, xendev->local_port);
 }
 
 /*
diff --git a/hw/xen_backend.h b/hw/xen_backend.h
index e421391..60f18f8 100644
--- a/hw/xen_backend.h
+++ b/hw/xen_backend.h
@@ -44,7 +44,7 @@ struct XenDevice {
 intremote_port;
 intlocal_port;
 
-xc_evtchn  *evtchndev;
+xenevtchn_handle   *evtchndev;
 xc_gnttab  *gnttabdev;
 
 struct XenDevOps   *ops;
diff --git a/hw/xen_common.h b/hw/xen_common.h
index a615052..cee908f 100644
--- a/hw/xen_common.h
+++ b/hw/xen_common.h
@@ -4,6 +4,7 @@
 #include 
 #include 
 
+#include 
 #include 
 #include 
 #include 
diff --git a/i386-dm/helper2.c b/i386-dm/helper2.c
index ede3105..2706f2e 100644
--- a/i386-dm/helper2.c
+++ b/i386-dm/helper2.c
@@ -48,6 +48,7 @@
 #include 
 #include 
 
+#include 
 #include 
 #include 
 #include 
@@ -104,7 +105,7 @@ buffered_iopage_t *buffered_io_page = NULL;
 QEMUTimer *buffered_io_timer;
 
 /* the evtchn fd 

[Xen-devel] [PATCH MINI-OS v8 3/4] mini-os: Include libxencall with libxc

2016-01-15 Thread Ian Campbell
libxencall has just been split out from libxc. From mini-os's point
of view we don't care about the distinction, so keep things simple by
just including libxencall if libxc is enabled.

Signed-off-by: Ian Campbell 
Acked-by: Samuel Thibault 
Acked-by: Wei Liu 
---
v2: Adjust for libs/$lib layout.
---
 Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Makefile b/Makefile
index 521f647..c900540 100644
--- a/Makefile
+++ b/Makefile
@@ -168,6 +168,7 @@ ifeq ($(CONFIG_XC),y)
 APP_LDLIBS += -L$(XEN_ROOT)/stubdom/libs-$(MINIOS_TARGET_ARCH)/toollog 
-whole-archive -lxentoollog -no-whole-archive
 APP_LDLIBS += -L$(XEN_ROOT)/stubdom/libs-$(MINIOS_TARGET_ARCH)/evtchn 
-whole-archive -lxenevtchn -no-whole-archive
 APP_LDLIBS += -L$(XEN_ROOT)/stubdom/libs-$(MINIOS_TARGET_ARCH)/gnttab 
-whole-archive -lxengnttab -no-whole-archive
+APP_LDLIBS += -L$(XEN_ROOT)/stubdom/libs-$(MINIOS_TARGET_ARCH)/call 
-whole-archive -lxencall -no-whole-archive
 APP_LDLIBS += -L$(XEN_ROOT)/stubdom/libxc-$(MINIOS_TARGET_ARCH) -whole-archive 
-lxenguest -lxenctrl -no-whole-archive
 endif
 APP_LDLIBS += -lpci
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v6 0/2] memory-hotplug: add automatic onlining policy for the newly added memory

2016-01-15 Thread Vitaly Kuznetsov
Changes since v5:
Patch 1:
- Mention possible failures during automatic onlining in memory-hotplug.txt
  [David Rientjes]
- Add Daniel's Reviewed-by: (hope it stands)

Patch2:
- Change the last 'domU' -> 'target domain' in Kconfig [Daniel Kiper]
- Add Daniel's Reviewed-by:
- Add David's Acked-by:

Original description:

Currently, all newly added memory blocks remain in 'offline' state unless
someone onlines them, some linux distributions carry special udev rules
like:

SUBSYSTEM=="memory", ACTION=="add", ATTR{state}=="offline", ATTR{state}="online"

to make this happen automatically. This is not a great solution for virtual
machines where memory hotplug is being used to address high memory pressure
situations as such onlining is slow and a userspace process doing this
(udev) has a chance of being killed by the OOM killer as it will probably
require to allocate some memory.

Introduce default policy for the newly added memory blocks in
/sys/devices/system/memory/auto_online_blocks file with two possible
values: "offline" which preserves the current behavior and "online" which
causes all newly added memory blocks to go online as soon as they're added.
The default is "offline".

Vitaly Kuznetsov (2):
  memory-hotplug: add automatic onlining policy for the newly added
memory
  xen_balloon: support memory auto onlining policy

 Documentation/memory-hotplug.txt | 23 ---
 drivers/base/memory.c| 34 +-
 drivers/xen/Kconfig  | 23 +++
 drivers/xen/balloon.c| 11 ++-
 include/linux/memory.h   |  3 +++
 include/linux/memory_hotplug.h   |  4 +++-
 mm/memory_hotplug.c  | 17 +++--
 7 files changed, 99 insertions(+), 16 deletions(-)

-- 
2.5.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v1] xen/arm: Add r1p12 to the list of supported Cadence UARTs

2016-01-15 Thread Edgar E. Iglesias
On Fri, Jan 15, 2016 at 12:30:27PM +, Ian Campbell wrote:
> On Thu, 2016-01-14 at 19:44 +0100, Edgar E. Iglesias wrote:
> > From: "Edgar E. Iglesias" 
> 
> Where are the bindings for cdns,uart-r1p12 defined? I don't see it
> mentioned in the Documentation/devicetree/bindings/serial/cdns,uart.txt in
> mainline Linux.

Hi Ian,

The bindings are currently in our queue:
https://github.com/Xilinx/linux-xlnx/blob/master/Documentation/devicetree/bindings/serial/cdns%2Cuart.txt


> Does the driver need to differ at all compared with what is needed for
> r1p8? Normally if there is no (practical/visible) difference then in DT-
> world there is no need for a new compatible string, since the device is
> still compatible.
> 
> If there are differences in the h/w but they are not relevant to our
> particular driven then please say so in the commit log.

Right, there are SW visible differences (the Linux driver uses them) but
none affecting XEN. I'll note that in the commit message.

Thanks,
Edgar


> 
> Ian.
> 
> > 
> > Signed-off-by: Edgar E. Iglesias 
> > ---
> >  xen/drivers/char/cadence-uart.c | 1 +
> >  1 file changed, 1 insertion(+)
> > 
> > diff --git a/xen/drivers/char/cadence-uart.c b/xen/drivers/char/cadence-
> > uart.c
> > index 933672f..7f90f8d 100644
> > --- a/xen/drivers/char/cadence-uart.c
> > +++ b/xen/drivers/char/cadence-uart.c
> > @@ -206,6 +206,7 @@ static int __init cuart_init(struct dt_device_node
> > *dev, const void *data)
> >  static const struct dt_device_match cuart_dt_match[] __initconst =
> >  {
> >  DT_MATCH_COMPATIBLE("cdns,uart-r1p8"),
> > +DT_MATCH_COMPATIBLE("cdns,uart-r1p12"),
> >  { /* sentinel */ },
> >  };
> >  

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v6 2/2] xen_balloon: support memory auto onlining policy

2016-01-15 Thread Vitaly Kuznetsov
Add support for the newly added kernel memory auto onlining policy to Xen
ballon driver.

Suggested-by: Daniel Kiper 
Reviewed-by: Daniel Kiper 
Acked-by: David Vrabel 
Signed-off-by: Vitaly Kuznetsov 
---
Changes since v5:
- Change the last 'domU' -> 'target domain' in Kconfig [Daniel Kiper]
---
 drivers/xen/Kconfig   | 23 +++
 drivers/xen/balloon.c | 11 ++-
 2 files changed, 25 insertions(+), 9 deletions(-)

diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
index 73708ac..979a8317 100644
--- a/drivers/xen/Kconfig
+++ b/drivers/xen/Kconfig
@@ -37,23 +37,30 @@ config XEN_BALLOON_MEMORY_HOTPLUG
 
  Memory could be hotplugged in following steps:
 
-   1) dom0: xl mem-max  
+   1) target domain: ensure that memory auto online policy is in
+  effect by checking /sys/devices/system/memory/auto_online_blocks
+  file (should be 'online').
+
+   2) control domain: xl mem-max  
   where  is >= requested memory size,
 
-   2) dom0: xl mem-set  
+   3) control domain: xl mem-set  
   where  is requested memory size; alternatively memory
   could be added by writing proper value to
   /sys/devices/system/xen_memory/xen_memory0/target or
-  /sys/devices/system/xen_memory/xen_memory0/target_kb on dumU,
+  /sys/devices/system/xen_memory/xen_memory0/target_kb on the
+  target domain.
 
-   3) domU: for i in /sys/devices/system/memory/memory*/state; do \
-  [ "`cat "$i"`" = offline ] && echo online > "$i"; done
+ Alternatively, if memory auto onlining was not requested at step 1
+ the newly added memory can be manually onlined in the target domain
+ by doing the following:
 
- Memory could be onlined automatically on domU by adding following 
line to udev rules:
+   for i in /sys/devices/system/memory/memory*/state; do \
+ [ "`cat "$i"`" = offline ] && echo online > "$i"; done
 
- SUBSYSTEM=="memory", ACTION=="add", RUN+="/bin/sh -c '[ -f 
/sys$devpath/state ] && echo online > /sys$devpath/state'"
+ or by adding the following line to udev rules:
 
- In that case step 3 should be omitted.
+ SUBSYSTEM=="memory", ACTION=="add", RUN+="/bin/sh -c '[ -f 
/sys$devpath/state ] && echo online > /sys$devpath/state'"
 
 config XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
int "Hotplugged memory limit (in GiB) for a PV guest"
diff --git a/drivers/xen/balloon.c b/drivers/xen/balloon.c
index 890c3b5..f8cca0c 100644
--- a/drivers/xen/balloon.c
+++ b/drivers/xen/balloon.c
@@ -338,7 +338,16 @@ static enum bp_state reserve_additional_memory(void)
}
 #endif
 
-   rc = add_memory_resource(nid, resource, false);
+   /*
+* add_memory_resource() will call online_pages() which in its turn
+* will call xen_online_page() callback causing deadlock if we don't
+* release balloon_mutex here. Unlocking here is safe because the
+* callers drop the mutex before trying again.
+*/
+   mutex_unlock(&balloon_mutex);
+   rc = add_memory_resource(nid, resource, memhp_auto_online);
+   mutex_lock(&balloon_mutex);
+
if (rc) {
pr_warn("Cannot add additional memory (%i)\n", rc);
goto err;
-- 
2.5.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v6 1/2] memory-hotplug: add automatic onlining policy for the newly added memory

2016-01-15 Thread Vitaly Kuznetsov
Currently, all newly added memory blocks remain in 'offline' state unless
someone onlines them, some linux distributions carry special udev rules
like:

SUBSYSTEM=="memory", ACTION=="add", ATTR{state}=="offline", ATTR{state}="online"

to make this happen automatically. This is not a great solution for virtual
machines where memory hotplug is being used to address high memory pressure
situations as such onlining is slow and a userspace process doing this
(udev) has a chance of being killed by the OOM killer as it will probably
require to allocate some memory.

Introduce default policy for the newly added memory blocks in
/sys/devices/system/memory/auto_online_blocks file with two possible
values: "offline" which preserves the current behavior and "online" which
causes all newly added memory blocks to go online as soon as they're added.
The default is "offline".

Reviewed-by: Daniel Kiper 
Signed-off-by: Vitaly Kuznetsov 
---
Changes since v5:
- Mention possible failures during automatic onlining in memory-hotplug.txt
  [David Rientjes]
---
 Documentation/memory-hotplug.txt | 23 ---
 drivers/base/memory.c| 34 +-
 drivers/xen/balloon.c|  2 +-
 include/linux/memory.h   |  3 +++
 include/linux/memory_hotplug.h   |  4 +++-
 mm/memory_hotplug.c  | 17 +++--
 6 files changed, 75 insertions(+), 8 deletions(-)

diff --git a/Documentation/memory-hotplug.txt b/Documentation/memory-hotplug.txt
index ce2cfcf..443f4b4 100644
--- a/Documentation/memory-hotplug.txt
+++ b/Documentation/memory-hotplug.txt
@@ -256,10 +256,27 @@ If the memory block is offline, you'll read "offline".
 
 5.2. How to online memory
 
-Even if the memory is hot-added, it is not at ready-to-use state.
-For using newly added memory, you have to "online" the memory block.
+When the memory is hot-added, the kernel decides whether or not to "online"
+it according to the policy which can be read from "auto_online_blocks" file:
 
-For onlining, you have to write "online" to the memory block's state file as:
+% cat /sys/devices/system/memory/auto_online_blocks
+
+The default is "offline" which means the newly added memory is not in a
+ready-to-use state and you have to "online" the newly added memory blocks
+manually. Automatic onlining can be requested by writing "online" to
+"auto_online_blocks" file:
+
+% echo online > /sys/devices/system/memory/auto_online_blocks
+
+This sets a global policy and impacts all memory blocks that will subsequently
+be hotplugged. Currently offline blocks keep their state. It is possible, under
+certain circumstances, that some memory blocks will be added but will fail to
+online. User space tools can check their "state" files
+(/sys/devices/system/memory/memoryXXX/state) and try to online them manually.
+
+If the automatic onlining wasn't requested, failed, or some memory block was
+offlined it is possible to change the individual block's state by writing to 
the
+"state" file:
 
 % echo online > /sys/devices/system/memory/memoryXXX/state
 
diff --git a/drivers/base/memory.c b/drivers/base/memory.c
index 25425d3..4fc240e 100644
--- a/drivers/base/memory.c
+++ b/drivers/base/memory.c
@@ -251,7 +251,7 @@ memory_block_action(unsigned long phys_index, unsigned long 
action, int online_t
return ret;
 }
 
-static int memory_block_change_state(struct memory_block *mem,
+int memory_block_change_state(struct memory_block *mem,
unsigned long to_state, unsigned long from_state_req)
 {
int ret = 0;
@@ -439,6 +439,37 @@ print_block_size(struct device *dev, struct 
device_attribute *attr,
 static DEVICE_ATTR(block_size_bytes, 0444, print_block_size, NULL);
 
 /*
+ * Memory auto online policy.
+ */
+
+static ssize_t
+show_auto_online_blocks(struct device *dev, struct device_attribute *attr,
+   char *buf)
+{
+   if (memhp_auto_online)
+   return sprintf(buf, "online\n");
+   else
+   return sprintf(buf, "offline\n");
+}
+
+static ssize_t
+store_auto_online_blocks(struct device *dev, struct device_attribute *attr,
+const char *buf, size_t count)
+{
+   if (sysfs_streq(buf, "online"))
+   memhp_auto_online = true;
+   else if (sysfs_streq(buf, "offline"))
+   memhp_auto_online = false;
+   else
+   return -EINVAL;
+
+   return count;
+}
+
+static DEVICE_ATTR(auto_online_blocks, 0644, show_auto_online_blocks,
+  store_auto_online_blocks);
+
+/*
  * Some architectures will have custom drivers to do this, and
  * will not need to do it from userspace.  The fake hot-add code
  * as well as ppc64 will do all of their discovery in userspace
@@ -737,6 +768,7 @@ static struct attribute *memory_root_attrs[] = {
 #endif
 
&dev_attr_block_size_bytes.attr,
+   &dev_attr_auto_online_blocks.attr,
NULL
 };
 
diff --git a/drivers/xen/balloon.c b/

[Xen-devel] [PATCH OSSTEST] ts-debian-install: increase time allowed for xen-create-image

2016-01-15 Thread Ian Campbell
This step is consistently timing out when run on cubietruck-*. Judging
from the logs it appears to be completing during the 30s slack added
by tcmdex (i.e. after the timeout message the rest of the output
appears in the test step log).

Looking at the results on arndale-* (which looks to pass reasonably
reliably) I see that the regular test-armhf-armhf-xl job takes around
550s to do the xen-create-image while test-armhf-armhf-xl-rtds
typically takes around 1100s (twice as long).

On cubietruck-braque test-armhf-armhf-xl uses 900s. One could
therefore extrapolate that test-armhf-armhf-xl-rtds might need more
than 1800s and not be too surprised that it appears to need something
a bit more than 2000s in practice. 2500s seems like sufficient
headroom.

For comparisson with arm on x86 godello takes around 210s in the
normal case and 680s with RTDS (>3x slower) while nocera takes 265s
and 640s (2.4x). (Those are from nearby but not identical flights in
order to match up the host).

Signed-off-by: Ian Campbell 
---
Cc: Dario Faggioli 
Cc: Meng Xu 

Dario, Meng, I suppose a 2-3x slow down with RTDS on a dom0 operation
(with no domains running) such as xen-create-image is not unexpected?
---
 ts-debian-install | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ts-debian-install b/ts-debian-install
index f42edbf..2a7331d 100755
--- a/ts-debian-install
+++ b/ts-debian-install
@@ -95,7 +95,7 @@ END
 $initrd_opt \\
 $archarg
 END
-target_cmd_root($ho, $cmd, 2000);
+target_cmd_root($ho, $cmd, 2500);
 
 my $cfg= "/etc/xen/$gho->{Name}.cfg";
 store_runvar("$gho->{Guest}_cfgpath", $cfg);
-- 
2.6.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [libvirt test] 78148: tolerable FAIL - PUSHED

2016-01-15 Thread osstest service owner
flight 78148 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/78148/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-libvirt-vhd  9 debian-di-installfail   like 77871

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass

version targeted for testing:
 libvirt  d6d7e2885badc24e224265283fd59f4c1c492ff9
baseline version:
 libvirt  545e5571f9bafc61c65b563ca69907f8cc935f72

Last test of basis77871  2016-01-12 01:15:10 Z3 days
Failing since 77934  2016-01-12 23:13:18 Z2 days4 attempts
Testing same since78148  2016-01-15 04:22:21 Z0 days1 attempts


People who touched revisions under test:
  Andrea Bolognani 
  Ben Gray 
  Cole Robinson 
  Cédric Bosdonnat 
  Dmitry Andreev 
  Guido Günther 
  Jim Fehlig 
  John Ferlan 
  Martin Kletzander 
  Michal Privoznik 
  Roman Bogorodskiy 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmpass
 test-amd64-amd64-libvirt-xsm pass
 test-armhf-armhf-libvirt-xsm fail
 test-amd64-i386-libvirt-xsm  pass
 test-amd64-amd64-libvirt pass
 test-armhf-armhf-libvirt fail
 test-amd64-i386-libvirt  pass
 test-amd64-amd64-libvirt-pairpass
 test-amd64-i386-libvirt-pair pass
 test-armhf-armhf-libvirt-qcow2   fail
 test-armhf-armhf-libvirt-raw fail
 test-amd64-amd64-libvirt-vhd fail



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=libvirt
+ revision=d6d7e2885badc24e224265283fd59f4c1c492ff9
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTES

Re: [Xen-devel] [PATCH v1] xen/arm: Add r1p12 to the list of supported Cadence UARTs

2016-01-15 Thread Ian Campbell
On Fri, 2016-01-15 at 14:30 +0100, Edgar E. Iglesias wrote:
> On Fri, Jan 15, 2016 at 12:30:27PM +, Ian Campbell wrote:
> > On Thu, 2016-01-14 at 19:44 +0100, Edgar E. Iglesias wrote:
> > > From: "Edgar E. Iglesias" 
> > 
> > Where are the bindings for cdns,uart-r1p12 defined? I don't see it
> > mentioned in the Documentation/devicetree/bindings/serial/cdns,uart.txt
> > in
> > mainline Linux.
> 
> Hi Ian,
> 
> The bindings are currently in our queue:
> https://github.com/Xilinx/linux-
> xlnx/blob/master/Documentation/devicetree/bindings/serial/cdns%2Cuart.txt

Great, thanks.
> 
> > Does the driver need to differ at all compared with what is needed for
> > r1p8? Normally if there is no (practical/visible) difference then in
> > DT-
> > world there is no need for a new compatible string, since the device is
> > still compatible.
> > 
> > If there are differences in the h/w but they are not relevant to our
> > particular driven then please say so in the commit log.
> 
> Right, there are SW visible differences (the Linux driver uses them) but
> none affecting XEN. I'll note that in the commit message.

If you want to just supply some words here I'll fold them in upon commit.
Or feel free to send a v2 if you prefer.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH QEMU-XEN-TRADITIONAL v8 2/4] qemu-xen-traditional: Use libxengnttab

2016-01-15 Thread Ian Campbell
/dev/xen/gntdev related wrappers have been moved out of libxenctrl
into their own library.

Signed-off-by: Ian Campbell 
Acked-by: Ian Jackson 
---
v3: Library moved to tools/libs/

v7: s/gnttab_munmap/gnttab_unmap/ to reflect change to the library
API.
---
 hw/xen_backend.c |  4 ++--
 hw/xen_backend.h |  2 +-
 hw/xen_common.h  |  1 +
 hw/xen_console.c |  4 ++--
 hw/xen_disk.c| 24 
 xen-hooks.mak|  2 ++
 6 files changed, 20 insertions(+), 17 deletions(-)

diff --git a/hw/xen_backend.c b/hw/xen_backend.c
index 40f6887..97d25da 100644
--- a/hw/xen_backend.c
+++ b/hw/xen_backend.c
@@ -217,7 +217,7 @@ static struct XenDevice *xen_be_get_xendev(const char 
*type, int dom, int dev,
 fcntl(xenevtchn_fd(xendev->evtchndev), F_SETFD, FD_CLOEXEC);
 
 if (ops->flags & DEVOPS_FLAG_NEED_GNTDEV) {
-   xendev->gnttabdev = xc_gnttab_open(NULL, 0);
+   xendev->gnttabdev = xengnttab_open(NULL, 0);
if (xendev->gnttabdev == NULL) {
xen_be_printf(NULL, 0, "can't open gnttab device\n");
xenevtchn_close(xendev->evtchndev);
@@ -270,7 +270,7 @@ static struct XenDevice *xen_be_del_xendev(int dom, int dev)
if (xendev->evtchndev != NULL)
xenevtchn_close(xendev->evtchndev);
if (xendev->gnttabdev != NULL)
-   xc_gnttab_close(xendev->gnttabdev);
+   xengnttab_close(xendev->gnttabdev);
 
TAILQ_REMOVE(&xendevs, xendev, next);
qemu_free(xendev);
diff --git a/hw/xen_backend.h b/hw/xen_backend.h
index 60f18f8..ba1e12f 100644
--- a/hw/xen_backend.h
+++ b/hw/xen_backend.h
@@ -45,7 +45,7 @@ struct XenDevice {
 intlocal_port;
 
 xenevtchn_handle   *evtchndev;
-xc_gnttab  *gnttabdev;
+xengnttab_handle  *gnttabdev;
 
 struct XenDevOps   *ops;
 TAILQ_ENTRY(XenDevice) next;
diff --git a/hw/xen_common.h b/hw/xen_common.h
index cee908f..cc48892 100644
--- a/hw/xen_common.h
+++ b/hw/xen_common.h
@@ -5,6 +5,7 @@
 #include 
 
 #include 
+#include 
 #include 
 #include 
 #include 
diff --git a/hw/xen_console.c b/hw/xen_console.c
index 80beb31..0bb4469 100644
--- a/hw/xen_console.c
+++ b/hw/xen_console.c
@@ -230,7 +230,7 @@ static int con_initialise(struct XenDevice *xendev)
   PROT_READ|PROT_WRITE,
   con->ring_ref);
 else
-con->sring = xc_gnttab_map_grant_ref(xendev->gnttabdev, 
con->xendev.dom,
+con->sring = xengnttab_map_grant_ref(xendev->gnttabdev, 
con->xendev.dom,
  con->ring_ref,
  PROT_READ|PROT_WRITE);
 if (!con->sring)
@@ -263,7 +263,7 @@ static void con_disconnect(struct XenDevice *xendev)
 if (!xendev->gnttabdev)
munmap(con->sring, XC_PAGE_SIZE);
 else
-xc_gnttab_munmap(xendev->gnttabdev, con->sring, 1);
+xengnttab_unmap(xendev->gnttabdev, con->sring, 1);
con->sring = NULL;
 }
 }
diff --git a/hw/xen_disk.c b/hw/xen_disk.c
index 250d806..bc78748 100644
--- a/hw/xen_disk.c
+++ b/hw/xen_disk.c
@@ -266,7 +266,7 @@ err:
 
 static void ioreq_unmap(struct ioreq *ioreq)
 {
-xc_gnttab *gnt = ioreq->blkdev->xendev.gnttabdev;
+xengnttab_handle *gnt = ioreq->blkdev->xendev.gnttabdev;
 int i;
 
 if (ioreq->v.niov == 0)
@@ -274,8 +274,8 @@ static void ioreq_unmap(struct ioreq *ioreq)
 if (batch_maps) {
if (!ioreq->pages)
return;
-   if (xc_gnttab_munmap(gnt, ioreq->pages, ioreq->v.niov) != 0)
-   xen_be_printf(&ioreq->blkdev->xendev, 0, "xc_gnttab_munmap failed: 
%s\n",
+   if (xengnttab_unmap(gnt, ioreq->pages, ioreq->v.niov) != 0)
+   xen_be_printf(&ioreq->blkdev->xendev, 0, "xengnttab_unmap failed: 
%s\n",
  strerror(errno));
ioreq->blkdev->cnt_map -= ioreq->v.niov;
ioreq->pages = NULL;
@@ -283,8 +283,8 @@ static void ioreq_unmap(struct ioreq *ioreq)
for (i = 0; i < ioreq->v.niov; i++) {
if (!ioreq->page[i])
continue;
-   if (xc_gnttab_munmap(gnt, ioreq->page[i], 1) != 0)
-   xen_be_printf(&ioreq->blkdev->xendev, 0, "xc_gnttab_munmap 
failed: %s\n",
+   if (xengnttab_unmap(gnt, ioreq->page[i], 1) != 0)
+   xen_be_printf(&ioreq->blkdev->xendev, 0, "xengnttab_unmap 
failed: %s\n",
  strerror(errno));
ioreq->blkdev->cnt_map--;
ioreq->page[i] = NULL;
@@ -294,13 +294,13 @@ static void ioreq_unmap(struct ioreq *ioreq)
 
 static int ioreq_map(struct ioreq *ioreq)
 {
-xc_gnttab *gnt = ioreq->blkdev->xendev.gnttabdev;
+xengnttab_handle *gnt = ioreq->blkdev->xendev.gnttabdev;
 int i;
 
 if (ioreq->v.niov == 0)
 return 0;
 if (batch_maps) {
-   ioreq->pages = xc_gnttab_map_grant_refs
+   ioreq->pages = xengnttab_map_grant_refs
(gnt, ioreq->v.niov, 

Re: [Xen-devel] [PATCH v3] x86/p2m: use large pages for MMIO mappings

2016-01-15 Thread Ian Campbell
On Fri, 2016-01-15 at 03:47 -0700, Jan Beulich wrote:
> > > > On 15.01.16 at 11:09,  wrote:
> > On Thu, 2016-01-14 at 03:04 -0700, Jan Beulich wrote:
> > > - ARM side unimplemented (and hence libxc for now made cope with both
> > >   models),
> > 
> > So, one model is the one described in the commit message:
> > 
> > > - zero (success, everything done)
> > > - positive (success, this many done, more to do: re-invoke)
> > > - negative (error)
> > 
> > What is the other one? I'd expect ARM to already implement a subset of
> > this
> > (i.e. 0 or negative, perhaps with a subset of the possible errno
> > values), 
> > which I'd then expect libxc to just cope with without it constituting a
> > second model.
> 
> Well, first of all ARM doesn't get switched away from the current
> model (yet), i.e. returning -E2BIG out of do_domctl().

Which AFAICT is a valid behaviour under the new model described in the
commit message specifically the "negative (error)" case.

I think the core of my objection/confusion here is describing this as two
different models when what is being introduced is a single API which can
fail either partially or entirely, with that being at the discretion of the
internals. In any case libxc needs to cope with the complete gamut of
behaviours of the interface.

IOW rather than describing a new API and referring obliquely to ARM only
supporting an old model I think this needs a complete description of the
interface covering the full possibilities of the API.

>  And then
> the difference between what the patch implements and what the
> non-commit message comment describes is how errors get handled:
> The patch makes a negative error value returned upon error, with
> the caller having no way to tell at what point the error occurred
> (and with a best effort undo in the case of "map"). The described
> alternative would return the number of succeeded entries unless
> an error occurred on the very first MFN, without any attempt to
> undo the part that was done successfully. I.e. it would leave it
> to the caller to decide what to do, and whether/when to roll back.

That's (probably, I don't quite follow all the details as written) fine,
but the interface should be described as a single API with the possible
failure cases each spelled out, i.e. not described as a split/contrast
between old vs. new or x86 vs. arm behaviour. The fact that x86 and arm
might currently exhibit different subsets of the possibilities offered by
the API is of at best secondary interest.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


  1   2   3   >