Re: [Qemu-block] [PATCH 3/4] savevm: fix savevm after migration

2017-03-29 Thread Dr. David Alan Gilbert
* Paolo Bonzini (pbonz...@redhat.com) wrote: > > > On 28/03/2017 15:16, Vladimir Sementsov-Ogievskiy wrote: > > 28.03.2017 15:09, Kevin Wolf wrote: > >> Am 28.03.2017 um 13:13 hat Dr. David Alan Gilbert geschrieben: > >>> * Kevin Wolf (kw...@redhat.com) wrot

Re: [Qemu-block] [PATCH 3/4] savevm: fix savevm after migration

2017-03-28 Thread Dr. David Alan Gilbert
* Kevin Wolf (kw...@redhat.com) wrote: > Am 28.03.2017 um 12:55 hat Dr. David Alan Gilbert geschrieben: > > * Kevin Wolf (kw...@redhat.com) wrote: > > > Am 25.02.2017 um 20:31 hat Vladimir Sementsov-Ogievskiy geschrieben: > > > > After migration all drives are inac

Re: [Qemu-block] [PATCH 3/4] savevm: fix savevm after migration

2017-03-28 Thread Dr. David Alan Gilbert
) > > Error *local_err = NULL; > > AioContext *aio_context; > > > > +if (runstate_check(RUN_STATE_FINISH_MIGRATE) || > > +runstate_check(RUN_STATE_POSTMIGRATE) || > > +runstate_check(RUN_STATE_PRELAUNCH)) > > + { > > +bdrv_inval

Re: [Qemu-block] [PATCH v2] migration/block: Avoid invoking blk_drain too frequently

2017-03-15 Thread Dr. David Alan Gilbert
; > + > > break; > > } > > sector += BDRV_SECTORS_PER_DIRTY_CHUNK; > > -- > > 1.8.3.1 > > > > Nice catch above all, thank you! > > Reviewed-by: Fam Zheng Are you taking that via a block pull? Dave -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [Qemu-devel] [PATCH v1 1/1] vmstate: fix failed iotests case 68 and 91

2017-03-14 Thread Dr. David Alan Gilbert
_load_state and vmstate_save_state. And the assert fails. > With this fix we can go back to the old behavior and support > VMS_VBUFFER with size 0 and nullptr. > > Signed-off-by: QingFeng Hao > Signed-off-by: Halil Pasic Thanks, and fixes problem with vmxnet3 migration. Reviewed-by:

Re: [Qemu-block] [PATCH 1/4] iotests: add migration corner cases test

2017-03-07 Thread Dr. David Alan Gilbert
ssert_qmp(result, 'return', {}) > + > +if __name__ == '__main__': > + iotests.main() > diff --git a/tests/qemu-iotests/175.out b/tests/qemu-iotests/175.out > new file mode 100644 > index 00..8d7e996700 > --- /dev/null > +++ b/tests/qemu-iotests/175.out > @@ -0,0 +1,5 @@ > +... > +-- > +Ran 3 tests > + > +OK > diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group > index 985b9a6a36..1f4bf03185 100644 > --- a/tests/qemu-iotests/group > +++ b/tests/qemu-iotests/group > @@ -167,3 +167,4 @@ > 172 auto > 173 rw auto > 174 auto > +175 auto quick > -- > 2.11.1 > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [PATCH 3/4] savevm: fix savevm after migration

2017-03-07 Thread Dr. David Alan Gilbert
efinitely valid. I'm only questioning if > 'savevm' (and by extension, any other command that modifies the VM or > its images) should automagically regain control of the VM, or whether > that should be more explicit. How many things other than 'cont' and 'savevm' are there? We should definitely fix it so a savevm there doesn't assert, but I'm also unsure if it's worth a separate explicit command. Dave > Kevin -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [PATCH RFC 1/1] vmstate: draft fix for failed iotests case 68 and 91

2017-03-07 Thread Dr. David Alan Gilbert
uan and Dave to CC. > > You are right, this is migration stuff and has very little to do with > qemu-block. > > > > I suspect the real question is why a field with size 0 was even stored > > in the vmstate to begin with. > > > > I have looked onto the issue. It

Re: [Qemu-block] [PATCH RFC 1/1] vmstate: draft fix for failed iotests case 68 and 91

2017-03-07 Thread Dr. David Alan Gilbert
t; > @@ -322,6 +326,10 @@ void vmstate_save_state(QEMUFile *f, const > > VMStateDescription *vmsd, > > int64_t old_offset, written_bytes; > > QJSON *vmdesc_loop = vmdesc; > > > > +if (size == 0) { > > +field++; > > +continue; > > +} > > trace_vmstate_save_state_loop(vmsd->name, field->name, > > n_elems); > > if (field->flags & VMS_POINTER) { > > first_elem = *(void **)first_elem; > > This is really a live migration fix, so I'm adding Juan and Dave to CC. > > I suspect the real question is why a field with size 0 was even stored > in the vmstate to begin with. Yes; which field is it that's failing? Dave > Kevin -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [Qemu-devel] [PATCH 4/4] migration: fix use-after-free of to_dst_file

2017-02-28 Thread Dr. David Alan Gilbert
ld not stay in global migration state > after > + * this function finished */ > +ms->to_dst_file = NULL; > + > return ret; > } > > -- > 2.11.1 > > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [PATCH 12/17] migration: add postcopy migration of dirty bitmaps

2017-02-27 Thread Dr. David Alan Gilbert
* Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > 24.02.2017 16:26, Dr. David Alan Gilbert wrote: > > * Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > > > Postcopy migration of dirty bitmaps. Only named dirty bitmaps, > > > associated w

Re: [Qemu-block] [PATCH 4/4] migration: fix use-after-free of to_dst_file

2017-02-27 Thread Dr. David Alan Gilbert
set_speed) will use it after it was freed. > > Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Dr. David Alan Gilbert > --- > migration/savevm.c | 5 + > 1 file changed, 5 insertions(+) > > diff --git a/migration/savevm.c b/migration/savevm.c > inde

Re: [Qemu-block] [PATCH 12/17] migration: add postcopy migration of dirty bitmaps

2017-02-24 Thread Dr. David Alan Gilbert
active_iterate = dirty_bitmap_is_active_iterate, > +.load_state = dirty_bitmap_load, > + .cleanup = dirty_bitmap_migration_cleanup, > +.is_active = dirty_bitmap_is_active, > +}; > + > +void dirty_bitmap_mig_init(void) > +{ > +QSIMPLEQ_INIT(&dirty_bitmap_mig_state.dbms_list); > + > +register_savevm_live(NULL, "dirty-bitmap", 0, 1, > + &savevm_dirty_bitmap_handlers, > + &dirty_bitmap_mig_state); > +} > diff --git a/migration/migration.c b/migration/migration.c > index 179bd04..edf5623 100644 > --- a/migration/migration.c > +++ b/migration/migration.c > @@ -122,6 +122,9 @@ MigrationIncomingState > *migration_incoming_get_current(void) > QLIST_INIT(&mis_current.loadvm_handlers); > qemu_mutex_init(&mis_current.rp_mutex); > qemu_event_init(&mis_current.main_thread_load_event, false); > + > +init_dirty_bitmap_incoming_migration(); > + > once = true; > } > return &mis_current; > diff --git a/migration/savevm.c b/migration/savevm.c > index 54572ef..927a680 100644 > --- a/migration/savevm.c > +++ b/migration/savevm.c > @@ -1651,6 +1651,8 @@ static void loadvm_postcopy_handle_run_bh(void *opaque) > > trace_loadvm_postcopy_handle_run_vmstart(); > > +dirty_bitmap_mig_before_vm_start(); > + > if (autostart) { > /* Hold onto your hats, starting the CPU */ > vm_start(); > diff --git a/migration/trace-events b/migration/trace-events > index 0212929..38fca41 100644 > --- a/migration/trace-events > +++ b/migration/trace-events > @@ -222,3 +222,17 @@ colo_vm_state_change(const char *old, const char *new) > "Change '%s' => '%s'" > colo_send_message(const char *msg) "Send '%s' message" > colo_receive_message(const char *msg) "Receive '%s' message" > colo_failover_set_state(const char *new_state) "new state %s" > + > +# migration/block-dirty-bitmap.c > +send_bitmap_header_enter(void) "" > +send_bitmap_bits(uint32_t flags, uint64_t start_sector, uint32_t nr_sectors, > uint64_t data_size) "\n flags:%x\n start_sector: %" PRIu64 "\n > nr_sectors: %" PRIu32 "\n data_size:%" PRIu64 "\n" > +dirty_bitmap_save_iterate(int in_postcopy) "in postcopy: %d" > +dirty_bitmap_save_complete_enter(void) "" > +dirty_bitmap_save_complete_finish(void) "" > +dirty_bitmap_save_pending(uint64_t pending, uint64_t max_size) "pending %" > PRIu64 " max: %" PRIu64 > +dirty_bitmap_load_complete(void) "" > +dirty_bitmap_load_bits_enter(uint64_t first_sector, uint32_t nr_sectors) > "chunk: %" PRIu64 " %" PRIu32 > +dirty_bitmap_load_bits_zeroes(void) "" > +dirty_bitmap_load_header(uint32_t flags) "flags %x" > +dirty_bitmap_load_enter(void) "" > +dirty_bitmap_load_success(void) "" > diff --git a/vl.c b/vl.c > index b4eaf03..f1ee9ff 100644 > --- a/vl.c > +++ b/vl.c > @@ -4405,6 +4405,7 @@ int main(int argc, char **argv, char **envp) > > blk_mig_init(); > ram_mig_init(); > +dirty_bitmap_mig_init(); > > /* If the currently selected machine wishes to override the units-per-bus > * property of its default HBA interface type, do so now. */ > -- > 1.8.3.1 > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [Qemu-devel] [PATCH 21/24] util/cutils: Let qemu_strtosz*() optionally reject trailing crap

2017-02-21 Thread Dr. David Alan Gilbert
* Markus Armbruster (arm...@redhat.com) wrote: > "Dr. David Alan Gilbert" writes: > > > * Markus Armbruster (arm...@redhat.com) wrote: > >> Change the qemu_strtosz() & friends to return -EINVAL when @endptr is > >> null and the conversion doesn'

Re: [Qemu-block] [PATCH 23/24] util/cutils: Change qemu_strtosz*() from int64_t to uint64_t

2017-02-20 Thread Dr. David Alan Gilbert
* Markus Armbruster (arm...@redhat.com) wrote: > This will permit its use in parse_option_size(). > > Cc: Dr. David Alan Gilbert > Cc: Eduardo Habkost (maintainer:X86) > Cc: Kevin Wolf (supporter:Block layer core) > Cc: Max Reitz (supporter:Block layer core) > Cc: qemu-b

Re: [Qemu-block] [PATCH 22/24] util/cutils: Return qemu_strtosz*() error and value separately

2017-02-20 Thread Dr. David Alan Gilbert
* Markus Armbruster (arm...@redhat.com) wrote: > This makes qemu_strtosz(), qemu_strtosz_mebi() and > qemu_strtosz_metric() similar to qemu_strtoi64(), except negative > values are rejected. > > Cc: Dr. David Alan Gilbert > Cc: Eduardo Habkost (maintainer:X86) > Cc: Kevin W

Re: [Qemu-block] [PATCH 21/24] util/cutils: Let qemu_strtosz*() optionally reject trailing crap

2017-02-20 Thread Dr. David Alan Gilbert
) passes a null @endptr. No functional > change there, because its conversion consumes the string. > > Simplify callers that use @endptr only to fail when it doesn't point > to '\0' to pass a null @endptr instead. > > Cc: Dr. David Alan Gilbert > Cc: Eduardo Habkost

Re: [Qemu-block] [Qemu-devel] [PATCH 15/17] iotests: add default node-name

2017-02-17 Thread Dr. David Alan Gilbert
his patch.. > > If I'm not mistaken, libvirt can be patched to explicitly set the same node > names in the QEMU command line, but that is some extra work to do there. My > point is if device names are used during migration, when available, this patch > and the libvirt change is not necessary. Always best to check with libvirt guys to see what makes sense for them; ccing in jdenemar. Dave > Fam -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [PATCH] block, migration: Use qemu_madvise inplace of madvise

2017-02-17 Thread Dr. David Alan Gilbert
e_t len, int advice) > { > if (advice == QEMU_MADV_INVALID) { > errno = EINVAL; > return -1; > } > #if defined(CONFIG_MADVISE) > return madvise(addr, len, advice); > #elif defined(CONFIG_POSIX_MADVISE) > return posix_madvise(addr, len, advice); > #else > errno = EINVAL; > return -1; > #endif > } > > > > > And this is the same case. > > > > Berto > > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [PATCH] block, migration: Use qemu_madvise inplace of madvise

2017-02-17 Thread Dr. David Alan Gilbert
- i.e. it's guaranteed to throw away the pages, where as posix_madvise *may* throw away the pages if the kernel feels like it. Dave > Kevin -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [PATCH 03/17] migration: split common postcopy out of ram postcopy

2017-02-06 Thread Dr. David Alan Gilbert
* Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > 01.02.2017 14:06, Vladimir Sementsov-Ogievskiy wrote: > > 24.01.2017 22:53, Dr. David Alan Gilbert wrote: > > > * Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > > > > 24.01.2

Re: [Qemu-block] [Qemu-devel] Non-flat command line option argument syntax

2017-02-06 Thread Dr. David Alan Gilbert
* Markus Armbruster (arm...@redhat.com) wrote: > "Dr. David Alan Gilbert" writes: > > > * Markus Armbruster (arm...@redhat.com) wrote: > >> "Dr. David Alan Gilbert" writes: > >> > >> > * Markus Armbruster (arm...@redhat.com) wrote:

Re: [Qemu-block] [Qemu-devel] Non-flat command line option argument syntax

2017-02-03 Thread Dr. David Alan Gilbert
* Markus Armbruster (arm...@redhat.com) wrote: > "Dr. David Alan Gilbert" writes: > > > * Markus Armbruster (arm...@redhat.com) wrote: > >> = Introduction = > >> > > > > > > > >> = Structured option argument synta

Re: [Qemu-block] [Qemu-devel] Non-flat command line option argument syntax

2017-02-02 Thread Dr. David Alan Gilbert
that we keep clean). > So, if escaped ',' wasn't ugly and confusing enough for you... > > === Comparison === > > In my opinion, dotted keys are weird and ugly, but at least they don't > add to the quoting mess. Structured values look better, except when > they do add to the quoting mess. > > I'm having a hard time deciding which one I like less :) > > Opinions? Other ideas? Dave > > > > > [1] [PATCH v14 00/21] QAPI/QOM work for non-scalar object properties > (actually v15) > Message-Id: <1475246744-29302-1-git-send-email-berra...@redhat.com> > http://lists.gnu.org/archive/html/qemu-devel/2016-09/msg08238.html > > [2] [RFC PATCH] block: Crude initial implementation of -blockdev > Message-Id: <1485968933-9162-1-git-send-email-arm...@redhat.com> > http://lists.gnu.org/archive/html/qemu-devel/2017-02/msg00182.html > > [3] Message-ID: <87h989ncse@dusky.pond.sub.org> > http://lists.gnu.org/archive/html/qemu-devel/2016-10/msg04046.html > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [PATCH 03/17] migration: split common postcopy out of ram postcopy

2017-01-31 Thread Dr. David Alan Gilbert
* Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > 24.01.2017 22:53, Dr. David Alan Gilbert wrote: > > * Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > > > 24.01.2017 12:24, Juan Quintela wrote: > > > > Vladimir Sementsov-Ogievskiy

Re: [Qemu-block] [PATCH 03/17] migration: split common postcopy out of ram postcopy

2017-01-24 Thread Dr. David Alan Gilbert
t; These parameters are unrelated if ram postcopy is disabled. So, I should be > better to have a possibility of skipping them, then to send unneeded numbers > and write separate code to read them from the stream (rewriting > loadvm_postcopy_handle_advise to just read these two numbers and do nothing > about ram postcopy for this case). I think I'd prefer either a new command or keeping these fields (probably all 0 ?) my worry is what happens in the case if we add a 3rd postcopy subfeature; In your case we have three possibilities: a) Postcopy RAM only - 16 bytes b) Postcopy persistent-dirty-bitmap only - 0 bytes c) Both - 16 bytes Lets say we added postcopy-foo in the future and it wanted to add another 16 bytes, what would it send if it was foo+persistent-dirty-bitmap and no RAM? We'd end up with 16 bytes sent but you'd have to be very careful never to get that confused with case (a) above. (I don't feel too strongly about it though) Dave > -- > Best regards, > Vladimir > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [Qemu-devel] [PATCH v2] migration: re-active images while migration been canceled after inactive them

2017-01-24 Thread Dr. David Alan Gilbert
qemu_savevm_state_complete_precopy(s->to_dst_file, false); > +s->block_inactive = true; > } > } > qemu_mutex_unlock_iothread(); > @@ -1755,10 +1766,14 @@ fail_invalidate: > if (s->state == MIGRATION_STATUS_A

Re: [Qemu-block] [PATCH v2] migration: re-active images while migration been canceled after inactive them

2017-01-24 Thread Dr. David Alan Gilbert
idate_cache_all() in > qmp_migrate_cancel() > directly if we find images become inactive. > > Besides, bdrv_invalidate_cache_all() in migration_completion() doesn't have > the > protection of big lock, fix it by add the missing qemu_mutex_lock_iothread(); >

Re: [Qemu-block] [PATCH] migration: re-active images while migration been canceled after inactive them

2017-01-23 Thread Dr. David Alan Gilbert
* Hailiang Zhang (zhang.zhanghaili...@huawei.com) wrote: > On 2017/1/23 21:40, Dr. David Alan Gilbert wrote: > > * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: > > > commit fe904ea8242cbae2d7e69c052c754b8f5f1ba1d6 fixed a case > > > which migration aborted QE

Re: [Qemu-block] [PATCH] migration: re-active images while migration been canceled after inactive them

2017-01-23 Thread Dr. David Alan Gilbert
;to_dst_file, false); > +s->block_inactive = true; > } > } > qemu_mutex_unlock_iothread(); > @@ -1758,6 +1769,8 @@ fail_invalidate: > bdrv_invalidate_cache_all(&local_err); > if (local_err) { > error_report_err(local_err); > +} else { > +s->block_inactive = false; > } I think the fe904 commit also add the problem that this bdrv_invalidate_cache_all is done outside the big lock (Stefan and Kevin tell me bdrv_* calls generally need the lock). Dave > } > > -- > 1.8.3.1 > > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [Qemu-devel] [PATCH] pflash_cfi01: fix per device sector length in CFI table

2017-01-16 Thread Dr. David Alan Gilbert
ure we need the nasty hack, but I'm also Ccing David for > his opinion. Hmm I don't understand enough about pflash to understand the change here; but generally if you need to keep something unchanged for older machine types, add a property and then set that property in the compa

Re: [Qemu-block] [PATCH v3 1/1] migration: disallow migrate_add_blocker during migration

2016-12-12 Thread Dr. David Alan Gilbert
.713e012 100644 > > --- a/migration/migration.c > > +++ b/migration/migration.c > > @@ -1044,6 +1044,31 @@ bool > > migration_in_postcopy_after_devices(MigrationState *s) > > return migration_in_postcopy(s) && s->postcopy_after_devices; > > } &

Re: [Qemu-block] [PATCH v3 1/1] migration: disallow migrate_add_blocker during migration

2016-12-09 Thread Dr. David Alan Gilbert
migration_blockers = g_slist_prepend(migration_blockers, reason); > +return 0; > +} > + > + error_setg(errp, "Cannot block a migration already in-progress"); I wonder if you need to add the 'reason' here as well; otherwise you get this message about blocking a migration but don't know what wanted to block it. So perhaps set errp from reason and then use error_prepend to add that extra message? Dave > +return -EBUSY; > } > > void migrate_del_blocker(Error *reason) > diff --git a/stubs/migr-blocker.c b/stubs/migr-blocker.c > index 8ab3604..a5ba18f 100644 > --- a/stubs/migr-blocker.c > +++ b/stubs/migr-blocker.c > @@ -2,8 +2,9 @@ > #include "qemu-common.h" > #include "migration/migration.h" > > -void migrate_add_blocker(Error *reason) > +int migrate_add_blocker(Error *reason, Error **errp) > { > +return 0; > } > > void migrate_del_blocker(Error *reason) > diff --git a/target-i386/kvm.c b/target-i386/kvm.c > index f62264a..8abb54d 100644 > --- a/target-i386/kvm.c > +++ b/target-i386/kvm.c > @@ -961,7 +961,11 @@ int kvm_arch_init_vcpu(CPUState *cs) > error_setg(&invtsc_mig_blocker, > "State blocked by non-migratable CPU device" > " (invtsc flag)"); > -migrate_add_blocker(invtsc_mig_blocker); > +r = migrate_add_blocker(invtsc_mig_blocker, NULL); > +if (r < 0) { > +fprintf(stderr, "migrate_add_blocker failed\n"); > +goto efail; > +} > /* for savevm */ > vmstate_x86_cpu.unmigratable = 1; > } > @@ -969,12 +973,12 @@ int kvm_arch_init_vcpu(CPUState *cs) > cpuid_data.cpuid.padding = 0; > r = kvm_vcpu_ioctl(cs, KVM_SET_CPUID2, &cpuid_data); > if (r) { > -return r; > +goto fail; > } > > r = kvm_arch_set_tsc_khz(cs); > if (r < 0) { > -return r; > +goto fail; > } > > /* vcpu's TSC frequency is either specified by user, or following > @@ -1001,6 +1005,12 @@ int kvm_arch_init_vcpu(CPUState *cs) > } > > return 0; > + > + fail: > +migrate_del_blocker(invtsc_mig_blocker); > + efail: > +error_free(invtsc_mig_blocker); > +return r; > } > > void kvm_arch_reset_vcpu(X86CPU *cpu) > -- > 2.9.3 > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [Qemu-devel] [PATCH] migration: re-active images when migration fails to complete

2016-12-08 Thread Dr. David Alan Gilbert
* Hailiang Zhang (zhang.zhanghaili...@huawei.com) wrote: > Hi, > > On 2016/12/6 23:24, Dr. David Alan Gilbert wrote: > > * Kevin Wolf (kw...@redhat.com) wrote: > > > Am 19.11.2016 um 12:43 hat zhanghailiang geschrieben: > > > > commit fe904ea8242cbae2d

Re: [Qemu-block] [Qemu-devel] [PATCH] migration: re-active images when migration fails to complete

2016-12-06 Thread Dr. David Alan Gilbert
op, which would stop the cancel sneaking in early. In the case where postcopy was never enabled (!migrate_postcopy_ram()) we can move the COMPLETED transition into the lock as well; so I think then we kind of become safe. In the case where postcopy was enabled I think we can do the COMPLETED transition before waiting for the return path to close - I think but I need to think more about that. And there seem to be some dodgy cases where we call the invalidate there after a late postcopy failure; that's bad, we shouldn't reactivate the source disks after going into postcopy. So, in summary; this function is a mess - it needs a much bigger fix than this patch. Dave > Kevin > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [PATCH 00/18] Dirty bitmaps postcopy migration

2016-08-17 Thread Dr. David Alan Gilbert
| 4 +- > qapi/block-core.json | 5 +- > tests/qemu-iotests/169 | 134 > tests/qemu-iotests/169.out | 5 + > tests/qemu-iotests/group | 2 + > tests/qemu-iotests/iotests.py | 21 +- > util/hbitmap.c | 8 + > vl.c | 1 + > 25 files changed, 1055 insertions(+), 54 deletions(-) > create mode 100644 migration/block-dirty-bitmap.c > create mode 100755 tests/qemu-iotests/169 > create mode 100644 tests/qemu-iotests/169.out > > -- > 1.8.3.1 > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [PATCH 02/18] migration: fix ram_save_pending

2016-08-17 Thread Dr. David Alan Gilbert
ated pending. > > Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Dr. David Alan Gilbert > --- > migration/ram.c | 8 ++-- > 1 file changed, 6 insertions(+), 2 deletions(-) > > diff --git a/migration/ram.c b/migration/ram.c > index fa5c72b..424ce0b 100644 > --- a/

Re: [Qemu-block] [Qemu-devel] [PATCH 0/9] major rework of drive-mirror

2016-06-15 Thread Dr. David Alan Gilbert
Vector in MirrorOp > mirror: use synch scheme for drive mirror > mirror: replace bdrv_dirty_bitmap with plain hbitmap > > block/io.c| 12 +- > block/mirror.c| 290 > +++--- > include/block/block_int.h | 1 + > 3 files changed, 229 insertions(+), 74 deletions(-) > > -- > 2.5.0 > > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [PATCH v12 2/3] quorum: implement bdrv_add_child() and bdrv_del_child()

2016-03-31 Thread Dr. David Alan Gilbert
local block device using the normal command line, and use drive_add even at startup. It solves a lot of the ordering problems with getting COLO booted. Dave > > > And the "filling up quorum's children" topic then makes me notice that > > (x-)blockdev-change shoul

Re: [Qemu-block] [PATCH v12 2/3] quorum: implement bdrv_add_child() and bdrv_del_child()

2016-03-29 Thread Dr. David Alan Gilbert
* Max Reitz (mre...@redhat.com) wrote: > On 29.03.2016 18:03, Dr. David Alan Gilbert wrote: > > * Max Reitz (mre...@redhat.com) wrote: > >> On 29.03.2016 17:54, Dr. David Alan Gilbert wrote: > >>> * Max Reitz (mre...@redhat.com) wrote: > >>>> On 2

Re: [Qemu-block] [PATCH v12 2/3] quorum: implement bdrv_add_child() and bdrv_del_child()

2016-03-29 Thread Dr. David Alan Gilbert
* Max Reitz (mre...@redhat.com) wrote: > On 29.03.2016 17:54, Dr. David Alan Gilbert wrote: > > * Max Reitz (mre...@redhat.com) wrote: > >> On 29.03.2016 17:50, Dr. David Alan Gilbert wrote: > >>> * Eric Blake (ebl...@redhat.com) wrote: > >>>> On 03/

Re: [Qemu-block] [PATCH v12 2/3] quorum: implement bdrv_add_child() and bdrv_del_child()

2016-03-29 Thread Dr. David Alan Gilbert
* Max Reitz (mre...@redhat.com) wrote: > On 29.03.2016 17:50, Dr. David Alan Gilbert wrote: > > * Eric Blake (ebl...@redhat.com) wrote: > >> On 03/29/2016 09:38 AM, Max Reitz wrote: > >>> On 17.03.2016 10:56, Wen Congyang wrote: > >>>> On 03/1

Re: [Qemu-block] [PATCH v12 2/3] quorum: implement bdrv_add_child() and bdrv_del_child()

2016-03-29 Thread Dr. David Alan Gilbert
* Eric Blake (ebl...@redhat.com) wrote: > On 03/29/2016 09:38 AM, Max Reitz wrote: > > On 17.03.2016 10:56, Wen Congyang wrote: > >> On 03/17/2016 05:48 PM, Dr. David Alan Gilbert wrote: > > > > [...] > > > >>> The children.0 notation is really co

Re: [Qemu-block] [PATCH v12 2/3] quorum: implement bdrv_add_child() and bdrv_del_child()

2016-03-19 Thread Dr. David Alan Gilbert
--> "children.1" (hd1.qcow2) > >>> s->children[1] <--> "children.0" (hd0.qcow2) > >> > >> Yes, it is correct. > >> > >>> > >>> Is this correct? Is this the indented behavior? Since you are reading > >>> in FIFO mode, now hd1.qcow2 will always be read first, so if > >>> children.1 was the secondary disk, it has just become the primary. > >> > >> Yes. > > > > And don't you need a way to control the order in which the disks must be > > read for COLO? > > I think in fifo mode, we should read the disk first that is added earlier. > > We don't need a way to control the order now. Can you document fully how it's used in COLO then? We should have the failure modes documented, and how you'll use it after failover etc Without that it's really difficult to tell if this naming is right. The children.0 notation is really confusing in the way that Berto describes; I hit this a couple of months ago and it really doesn't make sense. Dave > > Thanks > Wen Congyang > > > > > Berto > > > > > > . > > > > > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [PATCH v12 2/3] quorum: implement bdrv_add_child() and bdrv_del_child()

2016-03-19 Thread Dr. David Alan Gilbert
guest detects it? If the primary's local disk (children.0) fails then if we can failover at that point then the guest carries running on the secondary without ever knowing about the failure. Dave > > Thanks > Wen Congyang > > > > > Berto > > > > > > . > > > > > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [PATCH v12 2/3] quorum: implement bdrv_add_child() and bdrv_del_child()

2016-03-19 Thread Dr. David Alan Gilbert
* Wen Congyang (we...@cn.fujitsu.com) wrote: > On 03/17/2016 05:48 PM, Dr. David Alan Gilbert wrote: > > * Wen Congyang (we...@cn.fujitsu.com) wrote: > >> On 03/17/2016 05:10 PM, Alberto Garcia wrote: > >>> On Thu 17 Mar 2016 02:22:40 AM CET, Wen Congyang > >

Re: [Qemu-block] [PATCH v12 2/3] quorum: implement bdrv_add_child() and bdrv_del_child()

2016-03-19 Thread Dr. David Alan Gilbert
* Wen Congyang (we...@cn.fujitsu.com) wrote: > On 03/17/2016 07:25 PM, Dr. David Alan Gilbert wrote: > > * Wen Congyang (we...@cn.fujitsu.com) wrote: > >> On 03/17/2016 06:07 PM, Alberto Garcia wrote: > >>> On Thu 17 Mar 2016 10:56:09 AM CET, Wen Congyang wrote: >

Re: [Qemu-block] [PATCH v13 00/10] Block replication for continuous checkpoints

2016-02-04 Thread Dr. David Alan Gilbert
* Changlong Xie (xiecl.f...@cn.fujitsu.com) wrote: > On 02/01/2016 09:18 AM, Wen Congyang wrote: > >On 01/29/2016 06:47 PM, Dr. David Alan Gilbert wrote: > >>* Wen Congyang (we...@cn.fujitsu.com) wrote: > >>>On 01/29/2016 06:07 PM, Dr. David Alan Gilbert w

Re: [Qemu-block] [PATCH v13 00/10] Block replication for continuous checkpoints

2016-02-01 Thread Dr. David Alan Gilbert
* Wen Congyang (we...@cn.fujitsu.com) wrote: > On 01/29/2016 06:47 PM, Dr. David Alan Gilbert wrote: > > * Wen Congyang (we...@cn.fujitsu.com) wrote: > >> On 01/29/2016 06:07 PM, Dr. David Alan Gilbert wrote: > >>> * Wen Congyang (we...@cn.fujitsu.com) wrote: >

Re: [Qemu-block] [PATCH v13 00/10] Block replication for continuous checkpoints

2016-01-29 Thread Dr. David Alan Gilbert
* Wen Congyang (we...@cn.fujitsu.com) wrote: > On 01/29/2016 06:07 PM, Dr. David Alan Gilbert wrote: > > * Wen Congyang (we...@cn.fujitsu.com) wrote: > >> On 01/27/2016 07:03 PM, Dr. David Alan Gilbert wrote: > >>> Hi, > >>> I've got a block error

Re: [Qemu-block] [PATCH v13 00/10] Block replication for continuous checkpoints

2016-01-29 Thread Dr. David Alan Gilbert
* Wen Congyang (we...@cn.fujitsu.com) wrote: > On 01/27/2016 07:03 PM, Dr. David Alan Gilbert wrote: > > Hi, > > I've got a block error if I kill the secondary. > > > > Start both primary & secondary > > kill -9 secondary qemu > > x_colo_lost_hear

Re: [Qemu-block] [PATCH v13 00/10] Block replication for continuous checkpoints

2016-01-27 Thread Dr. David Alan Gilbert
ock/backup.c | 14 ++ > block/quorum.c | 78 +++ > block/replication.c| 545 > + > blockjob.c | 11 + > docs/block-replication.txt | 227 +++ > include/block/block.h | 9 + > include/block/block_int.h | 15 ++ > include/block/blockjob.h | 12 + > qapi/block-core.json | 33 ++- > 11 files changed, 1089 insertions(+), 3 deletions(-) > create mode 100644 block/replication.c > create mode 100644 docs/block-replication.txt > > -- > 1.9.3 > > > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] COLO: how to flip a secondary to a primary?

2016-01-25 Thread Dr. David Alan Gilbert
the existing connection? Dave > > > >Hailiang or Zhijian can answer this question. > > > Thanks > Li Zhijian > > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] COLO: how to flip a secondary to a primary?

2016-01-25 Thread Dr. David Alan Gilbert
* Wen Congyang (we...@cn.fujitsu.com) wrote: > On 01/23/2016 03:35 AM, Dr. David Alan Gilbert wrote: > > Hi, > > I've been looking at what's needed to add a new secondary after > > a primary failed; from the block side it doesn't look as hard > > as I&

Re: [Qemu-block] [PATCH v13 00/10] Block replication for continuous checkpoints

2016-01-25 Thread Dr. David Alan Gilbert
* Wen Congyang (we...@cn.fujitsu.com) wrote: > On 01/22/2016 11:14 PM, Dr. David Alan Gilbert wrote: > > Hi, > > I can trigger a segfault if I wire in the block replication together with > > a quorum instance; it only triggers with both of them present but, > > it

Re: [Qemu-block] [PATCH v13 00/10] Block replication for continuous checkpoints

2016-01-25 Thread Dr. David Alan Gilbert
* Wen Congyang (we...@cn.fujitsu.com) wrote: > On 01/22/2016 11:14 PM, Dr. David Alan Gilbert wrote: > > Hi, > > I can trigger a segfault if I wire in the block replication together with > > a quorum instance; it only triggers with both of them present but, > > it

Re: [Qemu-block] [PATCH v9 2/3] quorum: implement bdrv_add_child() and bdrv_del_child()

2016-01-22 Thread Dr. David Alan Gilbert
gt; > contents as the backing file. I don't think qemu has to protect us > > from user error in this case. > > But the backing file is read-only so the user can guarantee that the > destination has the same data before the shallow mirror. How do you do > that in this case

[Qemu-block] COLO: how to flip a secondary to a primary?

2016-01-22 Thread Dr. David Alan Gilbert
existing connections need to keep their sequence number offset but new connections made by what is now the primary dont need to do anything special. Dave -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [PATCH v13 00/10] Block replication for continuous checkpoints

2016-01-22 Thread Dr. David Alan Gilbert
cb=0x57815410) at /root/colo/jan-2016/qemu/block/io.c:2122 . So I guess acb->child_iter is wrong, since we only have one child on that quorum? and we're trying to do a destroy on the second child. Dave -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [Qemu-devel] [PATCH 1/1] HMP: Add equivalent to x-blockdev-change

2016-01-18 Thread Dr. David Alan Gilbert
* Markus Armbruster (arm...@redhat.com) wrote: > "Dr. David Alan Gilbert (git)" writes: > > > From: "Dr. David Alan Gilbert" > > > > x-blockdev-change has no HMP equivalent, so add x_block_change. > > Uh, I can find neither QMP command x-block

Re: [Qemu-block] [Patch v12 resend 00/10] Block replication for continuous checkpoints

2016-01-04 Thread Dr. David Alan Gilbert
onState *brs); > > void block_replication_remove(BlockReplicationState *brs); > > > > The replication block driver would add/remove itself. The quorum block > > driver probably doesn't need to be modified (I think in your current > > patches you modify it just to forward the start/stop/checkpoint calls to > > a particular quorum child). > > Yes, it is the major purpose. We also do some check in the quorum driver: > we don't allow more than one child support block replication. > > Thanks > Wen Congyang > > > > > Stefan > > > > > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [Patch v12 resend 05/10] docs: block replication's description

2016-01-04 Thread Dr. David Alan Gilbert
to take some time (seconds?) to sync Host 3 back in when you add it after a failover and the aim would be not to have distrubed the application for that long, so it should already be running on Host 2 during that resync. Dave -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [Qemu-devel] [PATCH 1/1] HMP: Add equivalent to x-blockdev-change

2015-12-17 Thread Dr. David Alan Gilbert
* Eric Blake (ebl...@redhat.com) wrote: > On 12/17/2015 03:47 AM, Dr. David Alan Gilbert (git) wrote: > > From: "Dr. David Alan Gilbert" > > > > x-blockdev-change has no HMP equivalent, so add x_block_change. > > > > Example useages are: > > s/

[Qemu-block] [PATCH 1/1] HMP: Add equivalent to x-blockdev-change

2015-12-17 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" x-blockdev-change has no HMP equivalent, so add x_block_change. Example useages are: x_block_change foo -a bah to add the node bah to the parent foo x_block_change foo -d bah to delete the node bah from the parent foo Signed-off-by: Dr.

Re: [Qemu-block] [Patch v12 00/10] Block replication for continuous checkpoints

2015-12-01 Thread Dr. David Alan Gilbert
> blockjob.c | 11 + > docs/block-replication.txt | 227 +++ > include/block/block.h | 9 + > include/block/block_int.h | 15 ++ > include/block/blockjob.h | 12 + > qapi/block-core.json | 34 ++- > 11 files changed, 1093 insertions(+), 4 deletions(-) > create mode 100644 block/replication.c > create mode 100644 docs/block-replication.txt > > -- > 2.5.0 > > > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [Qemu-devel] [PATCH 00/17] Framework for securely passing secrets to QEMU

2015-10-19 Thread Dr. David Alan Gilbert
> tests/.gitignore| 1 + > tests/Makefile | 2 + > tests/qemu-iotests/087 | 20 + > tests/qemu-iotests/087.out | 26 +- > tests/qemu-iotests/134 | 17 +- > tests/qemu-iotests/134.out | 44 +-- > tests/qemu-iotests/common.rc| 4 +- > tests/test-crypto-secret.c | 440 ++ > util/oslib-posix.c | 66 > util/oslib-win32.c | 24 -- > util/qemu-option.c | 6 + > vl.c| 8 +- > 45 files changed, 2740 insertions(+), 751 deletions(-) > create mode 100644 crypto/secret.c > create mode 100644 include/crypto/secret.h > create mode 100644 tests/test-crypto-secret.c > > -- > 2.4.3 > > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [Qemu-devel] [PATCH v5 3/4] qmp: add monitor command to add/remove a child

2015-10-12 Thread Dr. David Alan Gilbert
* Max Reitz (mre...@redhat.com) wrote: > On 09.10.2015 18:42, Dr. David Alan Gilbert wrote: > > * Max Reitz (mre...@redhat.com) wrote: > >> On 08.10.2015 08:15, Markus Armbruster wrote: > >>> Max Reitz writes: > >>> > >>>> On 22.09.2015 09

Re: [Qemu-block] [Qemu-devel] [PATCH v5 3/4] qmp: add monitor command to add/remove a child

2015-10-09 Thread Dr. David Alan Gilbert
t; so much. As I argued above, I'm not opposed to adding interface that are > actually not what we want, but that are what users want, and that are > easy to implement with the interface that we want. It's what I call a > “macro function”. > > > A secondary question is whether the incompleteness of the implementation > > demands an x- to warn users. > > We don't want to shoehorn generality into these two functions, but add a > new one anyway. We might want to add optional parameters later on (e.g. > changing the threshold), but that would be a compatible change. > > >> (My point is that with such an experimental interface, management tools > >> cannot use it, even though it'd be a very nice functionality to have) > > > > How much work is it to finish the job? Unless it's a substantial chunk, > > debating x- is much ado about nothing: just finish the job already :) > > We have proven in the past to need a significant amount of time for even > for non-substantial chunks. Often, non-substantial chunks turn out to be > substantial when actually tackling them. > > It basically comes down to whether the COLO authors are willing to wait > for us to do the job. And frankly, if I were them, I wouldn't be. > > Max > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [PATCH v2] migration: disallow migrate_add_blocker during migration

2015-10-09 Thread Dr. David Alan Gilbert
on to prohibit e.g. block jobs > from running concurrently with migration. > > Signed-off-by: John Snow For the migration code only: Reviewed-by: Dr. David Alan Gilbert Dave > --- > block/qcow.c | 6 +- > block/vdi.c | 6 +- &

Re: [Qemu-block] [PATCH v5 1/4] Add new block driver interface to add/delete a BDS's child

2015-10-08 Thread Dr. David Alan Gilbert
* Wen Congyang (we...@cn.fujitsu.com) wrote: > On 10/08/2015 03:00 AM, Dr. David Alan Gilbert wrote: > > * Wen Congyang (we...@cn.fujitsu.com) wrote: > >> In some cases, we want to take a quorum child offline, and take > >> another child online. > > > > H

Re: [Qemu-block] [PATCH v5 1/4] Add new block driver interface to add/delete a BDS's child

2015-10-07 Thread Dr. David Alan Gilbert
2f2c47b..64cbc55 100644 > --- a/include/block/block_int.h > +++ b/include/block/block_int.h > @@ -288,6 +288,11 @@ struct BlockDriver { > */ > int (*bdrv_probe_geometry)(BlockDriverState *bs, HDGeometry *geo); > > +void (*bdrv_add_child)(BlockDriverState *parent, BlockDriverState *child, > + Error **errp); > +void (*bdrv_del_child)(BlockDriverState *parent, BlockDriverState *child, > + Error **errp); > + > QLIST_ENTRY(BlockDriver) list; > }; > > -- > 2.4.3 > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [PATCH v5 0/4] qapi: child add/delete support

2015-09-23 Thread Dr. David Alan Gilbert
* Wen Congyang (we...@cn.fujitsu.com) wrote: > On 09/22/2015 07:15 PM, Dr. David Alan Gilbert wrote: > > * Wen Congyang (we...@cn.fujitsu.com) wrote: > >> If quorum's child is broken, we can use mirror job to replace it. > >> But sometimes, the user only need

Re: [Qemu-block] [PATCH v5 0/4] qapi: child add/delete support

2015-09-22 Thread Dr. David Alan Gilbert
4 ++ > qmp-commands.hx | 61 +++ > 10 files changed, 329 insertions(+), 5 deletions(-) > > -- > 2.4.3 > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [Qemu-devel] [PATCH 06/16] quorum: allow ignoring child errors

2015-09-07 Thread Dr. David Alan Gilbert
;C", "D", "E" ], 'vote-threshold':3, > 'child-errors-okay': [ "E" ] } > > The code to ignore the errors is then done in the quorum itself (the BDS > for E does not have to care about a special ignore-errors property, but > just alwa

[Qemu-block] qemu-img seg, test 082 not showing the error

2015-08-14 Thread Dr. David Alan Gilbert
more important problem. Dave -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

2015-07-09 Thread Dr. David Alan Gilbert
* Wen Congyang (we...@cn.fujitsu.com) wrote: > On 07/09/2015 06:37 PM, Dr. David Alan Gilbert wrote: > > * Wen Congyang (we...@cn.fujitsu.com) wrote: > >> On 07/09/2015 05:16 PM, Dr. David Alan Gilbert wrote: > >>> * Wen Congyang (we...@cn.fujitsu.com) wrote: > >

Re: [Qemu-block] [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

2015-07-09 Thread Dr. David Alan Gilbert
* Wen Congyang (we...@cn.fujitsu.com) wrote: > On 07/09/2015 05:16 PM, Dr. David Alan Gilbert wrote: > > * Wen Congyang (we...@cn.fujitsu.com) wrote: > > > >>>> I have sent the v8. But the usage is not changed. You can setup the > >>>> environment a

Re: [Qemu-block] [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

2015-07-09 Thread Dr. David Alan Gilbert
hat isn't working in this setup is migrate -b, I get: (qemu) Receiving block device images Error unknown block device disk1 qemu-system-x86_64: error while loading state section id 1(block) qemu-system-x86_64: load of migration failed: Invalid argument Can you explain the id=disk1 on the master s

Re: [Qemu-block] [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

2015-07-08 Thread Dr. David Alan Gilbert
* Wen Congyang (we...@cn.fujitsu.com) wrote: > On 07/08/2015 11:49 PM, Michael R. Hines wrote: > > On 07/07/2015 08:38 PM, Wen Congyang wrote: > >> On 07/08/2015 12:56 AM, Michael R. Hines wrote: > >>> On 07/07/2015 04:23 AM, Paolo Bonzini wrote: > >>>>

Re: [Qemu-block] [PATCH COLO-BLOCK v8 09/18] Backup: clear all bitmap when doing block checkpoint

2015-07-08 Thread Dr. David Alan Gilbert
t; /** > @@ -348,4 +351,13 @@ void block_job_defer_to_main_loop(BlockJob *job, >BlockJobDeferToMainLoopFn *fn, > void *opaque); > > +/** > + * block_job_do_checkpoint: > + * @job: The job. > + * @errp: Error object. > + * > + * Do block checkpoint on the specified job. > + */ > +void block_job_do_checkpoint(BlockJob *job, Error **errp); > + > #endif > -- > 2.4.3 > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

2015-07-07 Thread Dr. David Alan Gilbert
* Wen Congyang (we...@cn.fujitsu.com) wrote: > On 07/07/2015 08:25 AM, Michael R. Hines wrote: > > On 07/04/2015 07:46 AM, Wen Congyang wrote: > >> At 2015/7/3 23:30, Dr. David Alan Gilbert Wrote: > >>> * Wen Congyang (we...@cn.fujitsu.com) wrote: > >>&

Re: [Qemu-block] [Qemu-devel] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

2015-07-06 Thread Dr. David Alan Gilbert
* Wen Congyang (ghost...@gmail.com) wrote: > At 2015/7/3 23:30, Dr. David Alan Gilbert Wrote: > >* Wen Congyang (we...@cn.fujitsu.com) wrote: > >>Block replication is a very important feature which is used for > >>continuous checkpoints(for example: COLO). > >>

Re: [Qemu-block] [PATCH COLO-BLOCK v7 00/17] Block replication for continuous checkpoints

2015-07-03 Thread Dr. David Alan Gilbert
lude/qemu/option.h | 2 + > include/sysemu/block-backend.h | 3 + > include/sysemu/blockdev.h | 2 + > qapi/block.json| 16 ++ > util/qemu-option.c | 44 > 18 files changed, 1303 insertions(+), 47 deletions(-) > create mode 100644 block/replication.c > create mode 100644 docs/block-replication.txt > > -- > 2.4.3 > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [Qemu-devel] [PATCH COLO-Block v6 07/16] Add new block driver interface to connect/disconnect the remote target

2015-07-02 Thread Dr. David Alan Gilbert
* Wen Congyang (we...@cn.fujitsu.com) wrote: > On 07/02/2015 02:42 AM, Dr. David Alan Gilbert wrote: > > * Wen Congyang (we...@cn.fujitsu.com) wrote: > >> On 07/01/2015 04:11 PM, Dr. David Alan Gilbert wrote: > >>> * Wen Congyang (we...@cn.fujitsu.com) wrote: >

Re: [Qemu-block] [Qemu-devel] [PATCH COLO-Block v6 07/16] Add new block driver interface to connect/disconnect the remote target

2015-07-01 Thread Dr. David Alan Gilbert
* Wen Congyang (we...@cn.fujitsu.com) wrote: > On 07/01/2015 04:11 PM, Dr. David Alan Gilbert wrote: > > * Wen Congyang (we...@cn.fujitsu.com) wrote: > >> On 07/01/2015 03:01 AM, Dr. David Alan Gilbert wrote: > >>> * Wen Congyang (we...@cn.fujitsu.com) wrote: >

Re: [Qemu-block] [Qemu-devel] [PATCH COLO-Block v6 07/16] Add new block driver interface to connect/disconnect the remote target

2015-07-01 Thread Dr. David Alan Gilbert
* Wen Congyang (we...@cn.fujitsu.com) wrote: > On 07/01/2015 03:01 AM, Dr. David Alan Gilbert wrote: > > * Wen Congyang (we...@cn.fujitsu.com) wrote: > >> On 06/27/2015 03:03 AM, Dr. David Alan Gilbert wrote: > > > > > > > >>> Ah, I hadn

Re: [Qemu-block] [Qemu-devel] [PATCH COLO-Block v6 07/16] Add new block driver interface to connect/disconnect the remote target

2015-06-30 Thread Dr. David Alan Gilbert
* Wen Congyang (we...@cn.fujitsu.com) wrote: > On 06/27/2015 03:03 AM, Dr. David Alan Gilbert wrote: > > Ah, I hadn't realised you could do that; so do you just do: > > > > migrate_set_parameter colo on > > migrate -d -b tcp:otherhhost:port > > > &

Re: [Qemu-block] [Qemu-devel] [PATCH COLO-Block v6 07/16] Add new block driver interface to connect/disconnect the remote target

2015-06-26 Thread Dr. David Alan Gilbert
* Wen Congyang (we...@cn.fujitsu.com) wrote: > On 06/24/2015 10:07 PM, Dr. David Alan Gilbert wrote: > > * Wen Congyang (ghost...@gmail.com) wrote: > >> At 2015/6/19 18:49, Stefan Hajnoczi Wrote: > >>> On Fri, Jun 19, 2015 at 08:54:56AM +0800, Wen Congyang wrote:

Re: [Qemu-block] [Qemu-devel] [PATCH COLO-Block v6 07/16] Add new block driver interface to connect/disconnect the remote target

2015-06-24 Thread Dr. David Alan Gilbert
nd again after > migration. If the disk is not synced before migration, we will use disk > mirgation or mirror > job to sync it. Can you document the way that you use disk migration or mirror, together with your COLO setup? I think it would make it easier to understand this restriction. At the moment I don't understand how you can switch from doing a disk migration into COLO mode - there seems to be a gap between the end of disk migration and the start of COLO. > So we cannot start block replication when migration is running. We need > that nbd > client is not ready when migration is running, and it is ready between > migration ends > and checkpoint begins. Using a monotir command add the nbd client will cause > larger > downtime. So if the nbd client has been added(only not connect to the nbd > server), > we can connect to nbd server automatically. Without the disk migration or mirror, I can't see the need for the delayed bdrv_connect. I can see that you want to do a disconnect at failover, however you need to take care because if the network is broken at the point of failover you need to make sure nothing blocks. Dave > > Thanks > Wen Congyang -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [Qemu-devel] [PATCH v2 3/8] fdc: Introduce fdctrl->phase

2015-05-29 Thread Dr. David Alan Gilbert
* Peter Maydell (peter.mayd...@linaro.org) wrote: > On 29 May 2015 at 11:34, Dr. David Alan Gilbert wrote: > > It's the destination I'm worried about here, not the source; lets say > > you have two devices, a & b. 'a' gets serialised, but then '

Re: [Qemu-block] [Qemu-devel] [PATCH v2 3/8] fdc: Introduce fdctrl->phase

2015-05-29 Thread Dr. David Alan Gilbert
* Kevin Wolf (kw...@redhat.com) wrote: > Am 29.05.2015 um 11:38 hat Dr. David Alan Gilbert geschrieben: > > * Kevin Wolf (kw...@redhat.com) wrote: > > > Am 29.05.2015 um 10:33 hat Dr. David Alan Gilbert geschrieben: > > > > * Markus Armbruster (arm...@redhat.com) wr

Re: [Qemu-block] [Qemu-devel] [PATCH v2 3/8] fdc: Introduce fdctrl->phase

2015-05-29 Thread Dr. David Alan Gilbert
* Kevin Wolf (kw...@redhat.com) wrote: > Am 29.05.2015 um 10:33 hat Dr. David Alan Gilbert geschrieben: > > * Markus Armbruster (arm...@redhat.com) wrote: > > > "Dr. David Alan Gilbert" writes: > > > > > > > * John Snow (js...@redhat.com) wrote:

Re: [Qemu-block] [Qemu-devel] [PATCH v2 3/8] fdc: Introduce fdctrl->phase

2015-05-29 Thread Dr. David Alan Gilbert
* Markus Armbruster (arm...@redhat.com) wrote: > "Dr. David Alan Gilbert" writes: > > > * John Snow (js...@redhat.com) wrote: > >> > >> > >> On 05/21/2015 09:19 AM, Kevin Wolf wrote: > >> > The floppy controller spec describes thre

Re: [Qemu-block] [Qemu-devel] [PATCH v2 3/8] fdc: Introduce fdctrl->phase

2015-05-28 Thread Dr. David Alan Gilbert
WRITE; > > fdctrl->data_pos = 0; > > fdctrl->msr &= ~(FD_MSR_CMDBUSY | FD_MSR_DIO); > > @@ -1146,6 +1230,7 @@ static void fdctrl_to_command_phase(FDCtrl *fdctrl) > > * @fifo_len is the number of result bytes to be read out. */ > > static void fdctrl_to_result_phase(FDCtrl *fdctrl, int fifo_len) > > { > > +fdctrl->phase = FD_PHASE_RESULT; > > fdctrl->data_dir = FD_DIR_READ; > > fdctrl->data_len = fifo_len; > > fdctrl->data_pos = 0; > > @@ -1912,6 +1997,9 @@ static void fdctrl_handle_relative_seek_out(FDCtrl > > *fdctrl, int direction) > > fdctrl_raise_irq(fdctrl); > > } > > > > +/* > > + * Handlers for the execution phase of each command > > + */ > > static const struct { > > uint8_t value; > > uint8_t mask; > > @@ -2015,6 +2103,7 @@ static void fdctrl_write_data(FDCtrl *fdctrl, > > uint32_t value) > > /* We now have all parameters > > * and will be able to treat the command > > */ > > +fdctrl->phase = FD_PHASE_EXECUTION; > > if (fdctrl->data_state & FD_STATE_FORMAT) { > > fdctrl_format_sector(fdctrl); > > return; > > > > Acked-by: John Snow > > Looks ok to me, CC David Gilbert for a migration look-see. > > --js -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [Qemu-devel] [PATCH 3/8] fdc: Introduce fdctrl->phase

2015-05-21 Thread Dr. David Alan Gilbert
y narrow use case compared to old->new. I have to care about it downstream so I'd prefer if it wasn't broken upstream except where really hard. Dave > -- PMM > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [Qemu-devel] [PATCH 3/8] fdc: Introduce fdctrl->phase

2015-05-21 Thread Dr. David Alan Gilbert
oesn't address John's (valid) concerns - would be the most reasonable: > > 3. Don't add any VMState fields now and just do the post_load handler. >If we ever extend fdc in a way that makes it impossible to >reconstruct the phase from other migrated state, we add a subsection >that is only sent in cases where it differs from the reconstructed >value. Dave > > Kevin > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [PATCH COLO v4 00/15] Block replication for continuous checkpoints

2015-05-20 Thread Dr. David Alan Gilbert
| 16 ++ > qemu-options.hx| 4 + > tests/qemu-iotests/051 | 13 ++ > tests/qemu-iotests/051.out | 13 ++ > 16 files changed, 1260 insertions(+), 30 deletions(-) > create mode 100644 block/replication.c > create mode 100644 docs/block-replication.txt > > -- > 2.1.0 > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-block] [Qemu-devel] [PATCH COLO v4 01/15] docs: block replication's description

2015-05-14 Thread Dr. David Alan Gilbert
* Wen Congyang (ghost...@gmail.com) wrote: > At 2015/5/14 19:19, Dr. David Alan Gilbert Wrote: > >One thing I wanted to check I understand; how much RAM do the active and > >hidden > >disks use; lets say during the 1st checkpoint 10MB of disk is written, > >and

Re: [Qemu-block] [PATCH COLO v4 01/15] docs: block replication's description

2015-05-14 Thread Dr. David Alan Gilbert
e.driver=qcow2,\ > + file.backing_reference.drive_id=nbd_target1,\ > + file.backing_reference.hidden-disk.file.filename=hidden_disk.qcow2,\ > + file.backing_reference.hidden-disk.driver=qcow2,\ > + file.backing_reference.hidden-disk.allow-write-backing-file=o

Re: [Qemu-block] [PATCH COLO v3 01/14] docs: block replication's description

2015-05-08 Thread Dr. David Alan Gilbert
* Kevin Wolf (kw...@redhat.com) wrote: > Am 08.05.2015 um 10:42 hat Stefan Hajnoczi geschrieben: > > On Tue, May 05, 2015 at 04:23:56PM +0100, Dr. David Alan Gilbert wrote: > > > * Stefan Hajnoczi (stefa...@redhat.com) wrote: > > > > On Fri, Apr 24, 2015 at 11:36:3

<    1   2   3   4   5   6   >