Re: [Xen-devel] [PATCH RFC 08/20] libxl/migration: add precopy tuning parameters

2017-04-27 Thread Joshua Otto
On Wed, Apr 12, 2017 at 04:37:16PM +0100, Wei Liu wrote: > On Thu, Mar 30, 2017 at 02:03:29AM -0400, Joshua Otto wrote: > > I guess the way I had imagined an administrator using them would be in a > > non-production/test environment - if they could run workloads > > re

Re: [Xen-devel] [PATCH RFC 00/20] Add postcopy live migration support

2017-03-30 Thread Joshua Otto
On Wed, Mar 29, 2017 at 11:50:52PM +0100, Andrew Cooper wrote: > On 27/03/2017 10:06, Joshua Otto wrote: > > Hi, > > > > We're a team of three fourth-year undergraduate software engineering > > students at > > the University of Waterloo in Canada. In late

Re: [Xen-devel] [PATCH RFC 08/20] libxl/migration: add precopy tuning parameters

2017-03-29 Thread Joshua Otto
On Wed, Mar 29, 2017 at 10:08:02PM +0100, Andrew Cooper wrote: > On 27/03/17 10:06, Joshua Otto wrote: > > In the context of the live migration algorithm, the precopy iteration > > count refers to the number of page-copying iterations performed prior to > > the suspen

Re: [Xen-devel] [PATCH RFC 07/20] migration: defer precopy policy to libxl

2017-03-29 Thread Joshua Otto
function to > replace this. Yes, you're right, will fix. Thank you for the review! Josh > > Cheers, > > -jenny > > On 27/03/17 10:06, Joshua Otto wrote: > >The precopy phase of the xc_domain_save() live migration algorithm has > >historically been i

Re: [Xen-devel] [PATCH RFC 07/20] migration: defer precopy policy to libxl

2017-03-29 Thread Joshua Otto
On Wed, Mar 29, 2017 at 09:18:10PM +0100, Andrew Cooper wrote: > On 27/03/17 10:06, Joshua Otto wrote: > > The precopy phase of the xc_domain_save() live migration algorithm has > > historically been implemented to run until either a) (almost) no pages > > are dirty or b) s

Re: [Xen-devel] [PATCH RFC 06/20] libxc/xc_sr: factor helpers out of handle_page_data()

2017-03-29 Thread Joshua Otto
On Tue, Mar 28, 2017 at 08:52:26PM +0100, Andrew Cooper wrote: > On 27/03/17 10:06, Joshua Otto wrote: > > diff --git a/tools/libxc/xc_sr_stream_format.h > > b/tools/libxc/xc_sr_stream_format.h > > index 3291b25..32400b2 100644 > > --- a/tools/libxc/xc_sr_stream_for

Re: [Xen-devel] [PATCH RFC 05/20] libxc/xc_sr: factor out filter_pages()

2017-03-29 Thread Joshua Otto
On Tue, Mar 28, 2017 at 08:27:48PM +0100, Andrew Cooper wrote: > On 27/03/17 10:06, Joshua Otto wrote: > > diff --git a/tools/libxc/xc_sr_restore.c b/tools/libxc/xc_sr_restore.c > > index 481a904..8574ee8 100644 > > --- a/tools/libxc/xc_sr_restore.c > > +++ b/

Re: [Xen-devel] [PATCH RFC 04/20] libxc/xc_sr_save.c: add WRITE_TRIVIAL_RECORD_FN()

2017-03-29 Thread Joshua Otto
On Tue, Mar 28, 2017 at 08:03:26PM +0100, Andrew Cooper wrote: > On 27/03/17 10:06, Joshua Otto wrote: > > Writing the libxc save stream requires writing a few 'trivial' records, > > consisting only of a header with a particular type. As a readability > > aid, it

Re: [Xen-devel] [PATCH RFC 00/20] Add postcopy live migration support

2017-03-29 Thread Joshua Otto
got a lot on > our plate. I think maintainers will get to this series at some point. Understood. We're currently approaching our final exams so that's probably for the best :) > > From the look of things some patches can go in because they're general > useful. >

[Xen-devel] [PATCH RFC 20/20] tools: expose postcopy live migration support in libxl and xl

2017-03-27 Thread Joshua Otto
ostcopy to xl migrate, and skip the xl-level handshaking at both sides when postcopy migration occurs. Signed-off-by: Joshua Otto --- tools/libxl/libxl.h | 6 - tools/libxl/libxl_dom_save.c | 19 ++--- tools/libxl/libxl_domain.c | 9 --- tools/libxl/libxl_inter

[Xen-devel] [PATCH RFC 19/20] libxl/migration: implement the receiver side of postcopy live migration

2017-03-27 Thread Joshua Otto
sumed', that callers can test to determine whether or not further action is required on their-part post-migration to get the guest running. A subsequent patch will introduce a mechanism by which library clients can _induce_ a postcopy live migration. Signed-off-by: Joshua Otto --- tools/lib

[Xen-devel] [PATCH RFC 18/20] libxl/migration: implement the sender side of postcopy live migration

2017-03-27 Thread Joshua Otto
mechanism is introduced yet to enable library clients to induce a postcopy live migration - this will follow after the libxl postcopy receiver logic. Signed-off-by: Joshua Otto --- docs/specs/libxl-migration-stream.pandoc | 19 - tools/libxl/libxl.h | 4 +- tools/libxl

[Xen-devel] [PATCH RFC 17/20] libxl/libxl_stream_read.c: track callback chains with an explicit phase

2017-03-27 Thread Joshua Otto
migration. No functional change. Signed-off-by: Joshua Otto --- tools/libxl/libxl_internal.h| 7 ++-- tools/libxl/libxl_stream_read.c | 83 + 2 files changed, 45 insertions(+), 45 deletions(-) diff --git a/tools/libxl/libxl_internal.h b/tools/libxl

[Xen-devel] [PATCH RFC 16/20] libxl/libxl_stream_write.c: track callback chains with an explicit phase

2017-03-27 Thread Joshua Otto
factor the existing logic to use an enum rather than booleans for callback chain tracking. No functional change. Signed-off-by: Joshua Otto --- tools/libxl/libxl_internal.h | 7 ++- tools/libxl/libxl_stream_write.c | 96 ++-- 2 files changed, 48 inserti

[Xen-devel] [PATCH RFC 15/20] libxc/migration: implement the receiver side of postcopy live migration

2017-03-27 Thread Joshua Otto
ew restore callbacks required for this migration phase are stubbed in libxl for now, to be replaced in a subsequent patch that adds libxl support for this migration phase. Signed-off-by: Joshua Otto --- tools/libxc/include/xenguest.h | 63 ++- tools/libxc/xc_sr_common.h | 82 +++- t

[Xen-devel] [PATCH RFC 14/20] libxc/migration: implement the sender side of postcopy live migration

2017-03-27 Thread Joshua Otto
tion phase are stubbed in libxl for now, to be replaced in a subsequent patch that adds libxl support for this migration phase. Support for this phase on the migration receiver side follows immediately in the next patch. Signed-off-by: Joshua Otto --- tools/libxc/include/xenguest.h | 82 +

[Xen-devel] [PATCH RFC 13/20] libxc/migration: add try_read_record()

2017-03-27 Thread Joshua Otto
Enable non-blocking migration record reads by adding a helper routine that manages the context of a record read across multiple invocations as the record's data becomes available over time. Signed-off-by: Joshua Otto --- tools/libxc/xc_private.c | 21 +++ tools/libxc/xc_priv

[Xen-devel] [PATCH RFC 12/20] libxc/migration: specify postcopy live migration

2017-03-27 Thread Joshua Otto
- allocate the new postcopy record type numbers - augment the stream format specification to include these new types and their role in the protocol Signed-off-by: Joshua Otto --- docs/specs/libxc-migration-stream.pandoc | 177 ++- tools/libxc/xc_sr_common.c

[Xen-devel] [PATCH RFC 11/20] libxc/migration: correct hvm record ordering specification

2017-03-27 Thread Joshua Otto
omain save actually writes these records in the _reverse_ order, with HVM_CONTEXT first and HVM_PARAMS next. This has been the case for the entire history of that implementation, seemingly to no ill effect, so update the spec to reflect this. Signed-off-by: Joshua Otto --- docs/specs/libxc

[Xen-devel] [PATCH RFC 10/20] libxc/xc_sr_save.c: initialise rec.data before free()

2017-03-27 Thread Joshua Otto
colo_merge_secondary_dirty_bitmap() unconditionally free()s the .data member of its local xc_sr_record structure rec on its exit path. However, if the initial call to read_record() fails then this member is uninitialised. Initialise it. Signed-off-by: Joshua Otto --- tools/libxc/xc_sr_save.c

[Xen-devel] [PATCH RFC 09/20] libxc/xc_sr_save: introduce save batch types

2017-03-27 Thread Joshua Otto
subroutine that is useable independently of write_batch(). No functional change. Signed-off-by: Joshua Otto --- tools/libxc/xc_sr_common.h| 3 + tools/libxc/xc_sr_save.c | 217 -- tools/libxc/xg_save_restore.h | 2 +- 3 files changed, 151

[Xen-devel] [PATCH RFC 06/20] libxc/xc_sr: factor helpers out of handle_page_data()

2017-03-27 Thread Joshua Otto
_data() Steps 1) and 2) are also useful at various other stages of postcopy live migrations, so factor them into reusable helper routines. No functional change. Signed-off-by: Joshua Otto --- tools/libxc/xc_sr_common.c| 38 +++- tools/libxc/xc_sr_common.h| 10 +

[Xen-devel] [PATCH RFC 07/20] migration: defer precopy policy to libxl

2017-03-27 Thread Joshua Otto
tinue with the precopy, or c) proceed to the stop-and-copy phase. - provide an implementation of the old policy as such a callback in libxl and plumb it through the IPC machinery to libxc, effectively maintaing the old policy for now Signed-off-by: Joshua Otto --- tools/libxc/include/xengu

[Xen-devel] [PATCH RFC 08/20] libxl/migration: add precopy tuning parameters

2017-03-27 Thread Joshua Otto
--postcopy-iterations and --postcopy-threshold parameters to pass through Signed-off-by: Joshua Otto --- tools/libxl/libxl.h | 10 ++ tools/libxl/libxl_dom_save.c | 20 +++- tools/libxl/libxl_domain.c | 27 +-- tools/libxl

[Xen-devel] [PATCH RFC 02/20] libxc/xc_sr: parameterise write_record() on fd

2017-03-27 Thread Joshua Otto
mechanically update all existing callsites to pass ctx->fd for it. No functional change. Signed-off-by: Joshua Otto --- tools/libxc/xc_sr_common.c | 6 +++--- tools/libxc/xc_sr_common.h | 8 tools/libxc/xc_sr_common_x86.c | 2 +- tools/libxc/xc_sr_save.c

[Xen-devel] [PATCH RFC 03/20] libxc/xc_sr_restore.c: use write_record() in send_checkpoint_dirty_pfn_list()

2017-03-27 Thread Joshua Otto
Teach send_checkpoint_dirty_pfn_list() to use write_record()'s new fd parameter, avoiding the need for a manual writev(). No functional change. Signed-off-by: Joshua Otto --- tools/libxc/xc_sr_restore.c | 27 --- 1 file changed, 4 insertions(+), 23 deletions(-)

[Xen-devel] [PATCH RFC 04/20] libxc/xc_sr_save.c: add WRITE_TRIVIAL_RECORD_FN()

2017-03-27 Thread Joshua Otto
rd type and declares the corresponding trivial record write function. Use this to re-define the two existing trivial record functions, write_end_record() and write_checkpoint_record(). No functional change. Signed-off-by: Joshua Otto --- tools/libxc/xc_sr_save.c | 26 ++

[Xen-devel] [PATCH RFC 00/20] Add postcopy live migration support

2017-03-27 Thread Joshua Otto
the time required at the destination to 'evict' all of the outstanding pages is substantial - possibly because there is no batching mechanism by which pages can be evicted - so this area in particular might require further attention. We're really interested in any feedback you might

[Xen-devel] [PATCH RFC 05/20] libxc/xc_sr: factor out filter_pages()

2017-03-27 Thread Joshua Otto
h of these steps are also required when processing postcopy records, so factor it out into a common helper. No functional change. Signed-off-by: Joshua Otto --- tools/libxc/xc_sr_restore.c | 100 +--- 1 file changed, 75 insertions(+), 25 deletions(-) di

[Xen-devel] [PATCH RFC 01/20] tools: rename COLO 'postcopy' to 'aftercopy'

2017-03-27 Thread Joshua Otto
ed memory via a network page-fault." Mechanically rename 'postcopy' -> 'aftercopy' to free up the postcopy namespace while preserving the original intent of the name in the COLO context. No functional change. Signed-off-by: Joshua Otto --- tools/libxc/include/xen

[Xen-devel] [PATCH] svm: rephrase local variable use for Coverity.

2015-12-31 Thread Joshua Otto
Coverity CID 1343310 No functional changes. Signed-off-by: Joshua Otto --- On Mon, Dec 28, 2015 at 09:34:28AM +, Andrew Cooper wrote: > The error message isn't fantastic, but the complaint that Coverity > has is that we store intr here, then unilaterally store it again > sli

[Xen-devel] [PATCH 5/5] libxl: Add explicit cast to libxl_psr_cat_set_cbm

2015-12-27 Thread Joshua Otto
From: Chester Lin Fixes Coverity CID 1343299. The call to xc_psr_cat_set_domain_data() expects type xc_psr_cat_type but is provided libxl_psr_cbm_type which is defined in IDL. This change adds an explicit cast to fix the Coverity warning, and tweaks the surrounding code to more closely conform to

[Xen-devel] Coverity tidying

2015-12-27 Thread Joshua Otto
On Mon, Dec 14, 2015 at 11:08:43AM +, Ian Campbell wrote: > On Sat, 2015-12-12 at 17:07 -0500, Joshua Otto wrote: > > On Fri, Dec 11, 2015 at 01:52:41PM +, Ian Campbell wrote: > > > Cool! Just to be clear, you are looking for one project for the 3 of > > > you

[Xen-devel] [PATCH 3/5] ns16550: widen an integer constant for Coverity.

2015-12-27 Thread Joshua Otto
From: Harley Armstrong Fix CID 1343302 by widening a constant used with uart_param.reg_shift to avoid overflow for large values of reg_shift. Signed-off-by: Harley Armstrong --- xen/drivers/char/ns16550.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/xen/drivers/char/ns16

[Xen-devel] [PATCH 2/5] libxl: make GC_FREE reachable in libxl_get_scheduler()

2015-12-27 Thread Joshua Otto
Coverity CID 1343309 This patch preserves the multiple error paths in order to avoid meaninglessly assigning the ERROR_FAIL libxl error code to the return variable, which is of type libxl_scheduler. Signed-off-by: Joshua Otto --- tools/libxl/libxl.c | 2 +- 1 file changed, 1 insertion(+), 1

[Xen-devel] [PATCH 1/5] libxl: tidy libxl_get_scheduler() according to CODING_STYLE

2015-12-27 Thread Joshua Otto
introduced by this patch. Signed-off-by: Joshua Otto --- tools/libxl/libxl.c | 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c index 9207621..ca4679b 100644 --- a/tools/libxl/libxl.c +++ b/tools/libxl/libxl.c @@ -5585,10 +5585,11

[Xen-devel] [PATCH 4/5] credit: remove pointless local variable initialization.

2015-12-27 Thread Joshua Otto
Coverity CID 1343301 No functional changes. Signed-off-by: Joshua Otto --- xen/common/sched_credit.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/xen/common/sched_credit.c b/xen/common/sched_credit.c index 0dce790..02afddf 100644 --- a/xen/common/sched_credit.c +++ b/xen

Re: [Xen-devel] Taking on a Xen development project

2015-12-14 Thread Joshua Otto
On Mon, Dec 14, 2015 at 11:08:43AM +, Ian Campbell wrote: > On Sat, 2015-12-12 at 17:07 -0500, Joshua Otto wrote: > > On Fri, Dec 11, 2015 at 01:52:41PM +, Ian Campbell wrote: > > > Cool! Just to be clear, you are looking for one project for the 3 of > > > you

Re: [Xen-devel] Taking on a Xen development project

2015-12-14 Thread Joshua Otto
On Sat, Dec 12, 2015 at 11:02:50PM +, Andrew Cooper wrote: > On 12/12/2015 22:30, Joshua Otto wrote: > > On Sat, Dec 12, 2015 at 10:19:31AM +0800, Yang Hongyang wrote: > >> On 2015年12月11日 01:23, Andrew Cooper wrote: > >>> Hello - thankyou for your interest. &

Re: [Xen-devel] Taking on a Xen development project

2015-12-12 Thread Joshua Otto
On Sat, Dec 12, 2015 at 10:19:31AM +0800, Yang Hongyang wrote: > On 2015年12月11日 01:23, Andrew Cooper wrote: > > > >Hello - thankyou for your interest. > > > >One area to look at might be the parameters to the live migration > >looping. As part of the migration v2 rework I did in the 4.6 dev > >per

Re: [Xen-devel] Taking on a Xen development project

2015-12-12 Thread Joshua Otto
On Fri, Dec 11, 2015 at 01:52:41PM +, Ian Campbell wrote: > Cool! Just to be clear, you are looking for one project for the 3 of you to > work on as a group (vs 3 individual projects), is that right? Yes, that's right. > It's been a while since there has been a scan run, I did one yesterday b