Re: [Qemu-block] [PATCH v3 03/15] block/stream: add block job creation flags

2018-08-31 Thread Jeff Cody
On Fri, Aug 31, 2018 at 06:28:55PM -0400, John Snow wrote: > Add support for taking and passing forward job creaton flags. > > Signed-off-by: John Snow > Reviewed-by: Max Reitz (with the misspelling that Eric pointed out fixed): Reviewed-by: Jeff Cody > --- > block/stream.c| 5

Re: [Qemu-block] [PATCH v3 01/15] block/commit: add block job creation flags

2018-08-31 Thread Jeff Cody
On Fri, Aug 31, 2018 at 06:28:53PM -0400, John Snow wrote: > Add support for taking and passing forward job creation flags. > > Signed-off-by: John Snow > Reviewed-by: Max Reitz Reviewed-by: Jeff Cody > --- > block/commit.c| 5 +++-- > blockdev.c| 7 --- >

Re: [Qemu-block] [PATCH v3 02/15] block/mirror: add block job creation flags

2018-08-31 Thread Jeff Cody
On Fri, Aug 31, 2018 at 06:28:54PM -0400, John Snow wrote: > Add support for taking and passing forward job creaton flags. > > Signed-off-by: John Snow > Reviewed-by: Max Reitz > --- > block/mirror.c| 5 +++-- > blockdev.c| 3 ++- > include/block/block_int.h | 5

[Qemu-block] [PATCH v3 00/15] jobs: Job Exit Refactoring Pt 2

2018-08-31 Thread John Snow
This is part two of a two part series that refactors the exit logic of jobs. This series forces all jobs to use the "finalize" semantics that were introduced previously, but only exposed via the backup jobs. Patches 1-3 add plumbing for the auto-dismiss and auto-finalize flags but do not expose

[Qemu-block] [PATCH v3 02/15] block/mirror: add block job creation flags

2018-08-31 Thread John Snow
Add support for taking and passing forward job creaton flags. Signed-off-by: John Snow Reviewed-by: Max Reitz --- block/mirror.c| 5 +++-- blockdev.c| 3 ++- include/block/block_int.h | 5 - 3 files changed, 9 insertions(+), 4 deletions(-) diff --git

Re: [Qemu-block] [Qemu-devel] [PATCH v2 05/13] block/mirror: conservative mirror_exit refactor

2018-08-31 Thread John Snow
On 08/27/2018 08:47 AM, Max Reitz wrote: > On 2018-08-24 00:22, John Snow wrote: >> For purposes of minimum code movement, refactor the mirror_exit >> callback to use the post-finalization callbacks in a trivial way. >> >> Signed-off-by: John Snow >> --- >> block/mirror.c | 26

[Qemu-block] [PATCH v3 01/15] block/commit: add block job creation flags

2018-08-31 Thread John Snow
Add support for taking and passing forward job creation flags. Signed-off-by: John Snow Reviewed-by: Max Reitz --- block/commit.c| 5 +++-- blockdev.c| 7 --- include/block/block_int.h | 5 - 3 files changed, 11 insertions(+), 6 deletions(-) diff --git

[Qemu-block] [PATCH v3 07/15] block/commit: refactor stream to use job callbacks

2018-08-31 Thread John Snow
Signed-off-by: John Snow Reviewed-by: Max Reitz --- block/stream.c | 23 +++ 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/block/stream.c b/block/stream.c index 700eb239e4..81a7ec8ece 100644 --- a/block/stream.c +++ b/block/stream.c @@ -54,16 +54,16 @@

[Qemu-block] [PATCH v3 06/15] block/mirror: conservative mirror_exit refactor

2018-08-31 Thread John Snow
For purposes of minimum code movement, refactor the mirror_exit callback to use the post-finalization callbacks in a trivial way. Signed-off-by: John Snow --- block/mirror.c | 31 +-- 1 file changed, 25 insertions(+), 6 deletions(-) diff --git a/block/mirror.c

[Qemu-block] [PATCH v3 11/15] jobs: remove .exit callback

2018-08-31 Thread John Snow
Now that all of the jobs use the component finalization callbacks, there's no use for the heavy-hammer .exit callback anymore. job_exit becomes a glorified type shim so that we can call job_completed from aio_bh_schedule_oneshot. Move these three functions down into job.c to eliminate a forward

[Qemu-block] [PATCH v3 10/15] tests/test-blockjob-txn: move .exit to .clean

2018-08-31 Thread John Snow
The exit callback in this test actually only performs cleanup. Signed-off-by: John Snow --- tests/test-blockjob-txn.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c index ef29f35e44..86606f92b3 100644 ---

[Qemu-block] [PATCH v3 03/15] block/stream: add block job creation flags

2018-08-31 Thread John Snow
Add support for taking and passing forward job creaton flags. Signed-off-by: John Snow Reviewed-by: Max Reitz --- block/stream.c| 5 +++-- blockdev.c| 3 ++- include/block/block_int.h | 5 - 3 files changed, 9 insertions(+), 4 deletions(-) diff --git

[Qemu-block] [PATCH v3 13/15] qapi/block-mirror: expose new job properties

2018-08-31 Thread John Snow
Signed-off-by: John Snow Reviewed-by: Max Reitz --- blockdev.c | 14 ++ qapi/block-core.json | 30 -- 2 files changed, 42 insertions(+), 2 deletions(-) diff --git a/blockdev.c b/blockdev.c index 98b91e75a7..429cdf9901 100644 --- a/blockdev.c

[Qemu-block] [PATCH v3 04/15] block/commit: refactor commit to use job callbacks

2018-08-31 Thread John Snow
Use the component callbacks; prepare, abort, and clean. NB: prepare is only called when the job has not yet failed; and abort can be called after prepare. complete -> prepare -> abort -> clean complete -> abort -> clean Signed-off-by: John Snow Reviewed-by: Max Reitz --- block/commit.c | 90

[Qemu-block] [PATCH v3 08/15] tests/blockjob: replace Blockjob with Job

2018-08-31 Thread John Snow
These tests don't actually test blockjobs anymore, they test generic Job lifetimes. Change the types accordingly. Signed-off-by: John Snow Reviewed-by: Max Reitz --- tests/test-blockjob.c | 98 ++- 1 file changed, 50 insertions(+), 48

[Qemu-block] [PATCH v3 09/15] tests/test-blockjob: remove exit callback

2018-08-31 Thread John Snow
We remove the exit callback and the completed boolean along with it. We can simulate it just fine by waiting for the job to defer to the main loop, and then giving it one final kick to get the main loop portion to run. Signed-off-by: John Snow Reviewed-by: Max Reitz --- tests/test-blockjob.c |

[Qemu-block] [PATCH v3 12/15] qapi/block-commit: expose new job properties

2018-08-31 Thread John Snow
Signed-off-by: John Snow Reviewed-by: Max Reitz --- blockdev.c | 8 qapi/block-core.json | 16 +++- 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/blockdev.c b/blockdev.c index ec90eb1cf9..98b91e75a7 100644 --- a/blockdev.c +++ b/blockdev.c @@

[Qemu-block] [PATCH v3 05/15] block/mirror: don't install backing chain on abort

2018-08-31 Thread John Snow
In cases where we abort the block/mirror job, there's no point in installing the new backing chain before we finish aborting. Move this to the "success" portion of mirror_exit. Signed-off-by: John Snow --- block/mirror.c | 27 ++- 1 file changed, 14 insertions(+), 13

[Qemu-block] [PATCH v3 15/15] block/backup: qapi documentation fixup

2018-08-31 Thread John Snow
Fix documentation to match the other jobs amended for 3.1. Signed-off-by: John Snow Reviewed-by: Max Reitz --- qapi/block-core.json | 18 ++ 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index f877e9e414..c0b3d33dbb

[Qemu-block] [PATCH v3 14/15] qapi/block-stream: expose new job properties

2018-08-31 Thread John Snow
Signed-off-by: John Snow Reviewed-by: Max Reitz --- blockdev.c | 9 + hmp.c| 5 +++-- qapi/block-core.json | 16 +++- 3 files changed, 27 insertions(+), 3 deletions(-) diff --git a/blockdev.c b/blockdev.c index 429cdf9901..0cf8febe6c 100644 ---

Re: [Qemu-block] [PATCH v3 03/15] block/stream: add block job creation flags

2018-08-31 Thread Eric Blake
On 08/31/2018 05:28 PM, John Snow wrote: Add support for taking and passing forward job creaton flags. s/creaton/creation/ (here and in 2/15 as well) Signed-off-by: John Snow Reviewed-by: Max Reitz --- block/stream.c| 5 +++-- blockdev.c| 3 ++-

Re: [Qemu-block] [PATCH 3/3] RFC: delete PID file on exit

2018-08-31 Thread Marc-André Lureau
Hi On Fri, Aug 31, 2018 at 6:29 PM, Stefan Weil wrote: > Am 31.08.2018 um 16:53 schrieb Marc-André Lureau: > [...] >> +static const char *pid_file; >> + >> +static void qemu_unlink_pidfile(void) >> +{ >> +if (pid_file) { >> +unlink(pid_file); >> +} >> +} >> + >> bool

Re: [Qemu-block] [PATCH 3/3] RFC: delete PID file on exit

2018-08-31 Thread Stefan Weil
Am 31.08.2018 um 16:53 schrieb Marc-André Lureau: [...] > +static const char *pid_file; > + > +static void qemu_unlink_pidfile(void) > +{ > +if (pid_file) { > +unlink(pid_file); > +} > +} > + > bool machine_init_done; > > void qemu_add_machine_init_done_notifier(Notifier

Re: [Qemu-block] [Qemu-devel] [PATCH v3 2/9] jobs: canonize Error object

2018-08-31 Thread John Snow
On 08/31/2018 02:08 AM, Markus Armbruster wrote: > Eric Blake writes: > >> On 08/29/2018 08:57 PM, John Snow wrote: >>> Jobs presently use both an Error object in the case of the create job, >>> and char strings in the case of generic errors elsewhere. >>> >>> Unify the two paths as just

[Qemu-block] [PATCH 0/3] util: add qemu_write_pidfile()

2018-08-31 Thread Marc-André Lureau
Hi, Here are a few PID file related patches extracted from "[PATCH v4 00/29] vhost-user for input & GPU" series, with suggestions from Daniel Berrangé. thanks Marc-André Lureau (3): util: add qemu_write_pidfile() util: use fcntl() for qemu_write_pidfile() locking RFC: delete PID file on

[Qemu-block] [PATCH 1/3] util: add qemu_write_pidfile()

2018-08-31 Thread Marc-André Lureau
There are variants of qemu_create_pidfile() in qemu-pr-helper and qemu-ga. Let's have a common implementation in libqemuutil. The code is initially based from pr-helper write_pidfile(), with various improvements and suggestions from Daniel Berrangé: QEMU will leave the pidfile existing on disk

[Qemu-block] [PATCH 3/3] RFC: delete PID file on exit

2018-08-31 Thread Marc-André Lureau
Register an exit handler to remove the PID file. By the time atexit() is called, qemu_write_pidfile() guarantees QEMU owns the PID file, thus we could safely remove it when exiting. Signed-off-by: Marc-André Lureau --- vl.c | 11 ++- 1 file changed, 10 insertions(+), 1 deletion(-) diff

[Qemu-block] [PATCH 2/3] util: use fcntl() for qemu_write_pidfile() locking

2018-08-31 Thread Marc-André Lureau
Daniel Berrangé suggested to use fcntl() locks rather than lockf(). 'man lockf': On Linux, lockf() is just an interface on top of fcntl(2) locking. Many other systems implement lockf() in this way, but note that POSIX.1 leaves the relationship between lockf() and fcntl(2) locks

[Qemu-block] [PULL v2 10/10] jobs: remove job_defer_to_main_loop

2018-08-31 Thread Max Reitz
From: John Snow Now that the job infrastructure is handling the job_completed call for all implemented jobs, we can remove the interface that allowed jobs to schedule their own completion. Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20180830015734.19765-10-js...@redhat.com

[Qemu-block] [PULL v2 03/10] jobs: canonize Error object

2018-08-31 Thread Max Reitz
From: John Snow Jobs presently use both an Error object in the case of the create job, and char strings in the case of generic errors elsewhere. Unify the two paths as just j->err, and remove the extra argument from job_completed. The integer error code for job_completed is kept for now, to be

[Qemu-block] [PULL v2 09/10] jobs: remove ret argument to job_completed; privatize it

2018-08-31 Thread Max Reitz
From: John Snow Jobs are now expected to return their retcode on the stack, from the .run callback, so we can remove that argument. job_cancel does not need to set -ECANCELED because job_completed will update the return code itself if the job was canceled. While we're here, make job_completed

[Qemu-block] [PULL v2 08/10] block/backup: make function variables consistently named

2018-08-31 Thread Max Reitz
From: John Snow Rename opaque_job to job to be consistent with other job implementations. Rename 'job', the BackupBlockJob object, to 's' to also be consistent. Suggested-by: Eric Blake Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20180830015734.19765-8-js...@redhat.com

[Qemu-block] [PULL v2 00/10] Block patches

2018-08-31 Thread Max Reitz
The following changes since commit 19b599f7664b2ebfd0f405fb79c14dd241557452: Merge remote-tracking branch 'remotes/armbru/tags/pull-error-2018-08-27-v2' into staging (2018-08-27 16:44:20 +0100) are available in the Git repository at: https://git.xanclic.moe/XanClic/qemu.git

[Qemu-block] [PULL 04/11] jobs: canonize Error object

2018-08-31 Thread Max Reitz
From: John Snow Jobs presently use both an Error object in the case of the create job, and char strings in the case of generic errors elsewhere. Unify the two paths as just j->err, and remove the extra argument from job_completed. The integer error code for job_completed is kept for now, to be

[Qemu-block] [PULL 02/11] tests: fix bdrv-drain leak

2018-08-31 Thread Max Reitz
From: Marc-André Lureau Spotted by ASAN: = ==5378==ERROR: LeakSanitizer: detected memory leaks Direct leak of 65536 byte(s) in 1 object(s) allocated from: #0 0x7f788f83bc48 in malloc (/lib64/libasan.so.5+0xeec48) #1

[Qemu-block] [PULL v2 07/10] jobs: utilize job_exit shim

2018-08-31 Thread Max Reitz
From: John Snow Utilize the job_exit shim by not calling job_defer_to_main_loop, and where applicable, converting the deferred callback into the job_exit callback. This converts backup, stream, create, and the unit tests all at once. Most of these jobs do not see any changes to the order in

[Qemu-block] [PULL v2 05/10] block/commit: utilize job_exit shim

2018-08-31 Thread Max Reitz
From: John Snow Change the manual deferment to commit_complete into the implicit callback to job_exit, renaming commit_complete to commit_exit. This conversion does change the timing of when job_completed is called to after the bdrv_replace_node and bdrv_unref calls, which could have

[Qemu-block] [PULL 08/11] jobs: utilize job_exit shim

2018-08-31 Thread Max Reitz
From: John Snow Utilize the job_exit shim by not calling job_defer_to_main_loop, and where applicable, converting the deferred callback into the job_exit callback. This converts backup, stream, create, and the unit tests all at once. Most of these jobs do not see any changes to the order in

[Qemu-block] [PULL v2 06/10] block/mirror: utilize job_exit shim

2018-08-31 Thread Max Reitz
From: John Snow Change the manual deferment to mirror_exit into the implicit callback to job_exit and the mirror_exit callback. This does change the order of some bdrv_unref calls and job_completed, but thanks to the new context in which we call .exit, this is safe to defer the possible

[Qemu-block] [PULL 05/11] jobs: add exit shim

2018-08-31 Thread Max Reitz
From: John Snow All jobs do the same thing when they leave their running loop: - Store the return code in a structure - wait to receive this structure in the main thread - signal job completion via job_completed Few jobs do anything beyond exactly this. Consolidate this exit logic for a net

[Qemu-block] [PULL v2 04/10] jobs: add exit shim

2018-08-31 Thread Max Reitz
From: John Snow All jobs do the same thing when they leave their running loop: - Store the return code in a structure - wait to receive this structure in the main thread - signal job completion via job_completed Few jobs do anything beyond exactly this. Consolidate this exit logic for a net

[Qemu-block] [PULL 06/11] block/commit: utilize job_exit shim

2018-08-31 Thread Max Reitz
From: John Snow Change the manual deferment to commit_complete into the implicit callback to job_exit, renaming commit_complete to commit_exit. This conversion does change the timing of when job_completed is called to after the bdrv_replace_node and bdrv_unref calls, which could have

[Qemu-block] [PULL 09/11] block/backup: make function variables consistently named

2018-08-31 Thread Max Reitz
From: John Snow Rename opaque_job to job to be consistent with other job implementations. Rename 'job', the BackupBlockJob object, to 's' to also be consistent. Suggested-by: Eric Blake Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20180830015734.19765-8-js...@redhat.com

[Qemu-block] [PULL 10/11] jobs: remove ret argument to job_completed; privatize it

2018-08-31 Thread Max Reitz
From: John Snow Jobs are now expected to return their retcode on the stack, from the .run callback, so we can remove that argument. job_cancel does not need to set -ECANCELED because job_completed will update the return code itself if the job was canceled. While we're here, make job_completed

[Qemu-block] [PULL v2 02/10] jobs: change start callback to run callback

2018-08-31 Thread Max Reitz
From: John Snow Presently we codify the entry point for a job as the "start" callback, but a more apt name would be "run" to clarify the idea that when this function returns we consider the job to have "finished," except for any cleanup which occurs in separate callbacks later. As part of this

[Qemu-block] [PULL v2 01/10] tests: fix bdrv-drain leak

2018-08-31 Thread Max Reitz
From: Marc-André Lureau Spotted by ASAN: = ==5378==ERROR: LeakSanitizer: detected memory leaks Direct leak of 65536 byte(s) in 1 object(s) allocated from: #0 0x7f788f83bc48 in malloc (/lib64/libasan.so.5+0xeec48) #1

[Qemu-block] [PULL 01/11] file-posix: Skip effectiveless OFD lock operations

2018-08-31 Thread Max Reitz
From: Fam Zheng If we know we've already locked the bytes, don't do it again; similarly don't unlock a byte if we haven't locked it. This doesn't change the behavior, but fixes a corner case explained below. Libvirt had an error handling bug that an image can get its (ownership, file mode,

[Qemu-block] [PULL 00/11] Block patches

2018-08-31 Thread Max Reitz
The following changes since commit 19b599f7664b2ebfd0f405fb79c14dd241557452: Merge remote-tracking branch 'remotes/armbru/tags/pull-error-2018-08-27-v2' into staging (2018-08-27 16:44:20 +0100) are available in the Git repository at: https://git.xanclic.moe/XanClic/qemu.git

[Qemu-block] [PULL 03/11] jobs: change start callback to run callback

2018-08-31 Thread Max Reitz
From: John Snow Presently we codify the entry point for a job as the "start" callback, but a more apt name would be "run" to clarify the idea that when this function returns we consider the job to have "finished," except for any cleanup which occurs in separate callbacks later. As part of this

[Qemu-block] [PULL 07/11] block/mirror: utilize job_exit shim

2018-08-31 Thread Max Reitz
From: John Snow Change the manual deferment to mirror_exit into the implicit callback to job_exit and the mirror_exit callback. This does change the order of some bdrv_unref calls and job_completed, but thanks to the new context in which we call .exit, this is safe to defer the possible

[Qemu-block] [PULL 11/11] jobs: remove job_defer_to_main_loop

2018-08-31 Thread Max Reitz
From: John Snow Now that the job infrastructure is handling the job_completed call for all implemented jobs, we can remove the interface that allowed jobs to schedule their own completion. Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20180830015734.19765-10-js...@redhat.com

Re: [Qemu-block] [PULL 00/11] Block patches

2018-08-31 Thread Max Reitz
On 2018-08-31 16:24, Max Reitz wrote: > The following changes since commit 19b599f7664b2ebfd0f405fb79c14dd241557452: > > Merge remote-tracking branch 'remotes/armbru/tags/pull-error-2018-08-27-v2' > into staging (2018-08-27 16:44:20 +0100) > > are available in the Git repository at: > >

Re: [Qemu-block] [PATCH v3 0/9] jobs: Job Exit Refactoring Pt 1

2018-08-31 Thread Max Reitz
On 2018-08-30 03:57, John Snow wrote: > This is part one of a two part series that refactors the exit logic > of jobs. > > Part one removes job_defer_to_main_loop. > Part two removes the job->exit() callback introduced in part one. > > It's redundant to have each job manage deferring to the main

Re: [Qemu-block] [PATCH v3 5/9] block/mirror: utilize job_exit shim

2018-08-31 Thread Jeff Cody
On Wed, Aug 29, 2018 at 09:57:30PM -0400, John Snow wrote: > Change the manual deferment to mirror_exit into the implicit > callback to job_exit and the mirror_exit callback. > > This does change the order of some bdrv_unref calls and job_completed, > but thanks to the new context in which we

Re: [Qemu-block] [PATCH v3 4/9] block/commit: utilize job_exit shim

2018-08-31 Thread Jeff Cody
On Wed, Aug 29, 2018 at 09:57:29PM -0400, John Snow wrote: > Change the manual deferment to commit_complete into the implicit > callback to job_exit, renaming commit_complete to commit_exit. > > This conversion does change the timing of when job_completed is > called to after the

Re: [Qemu-block] [PATCH v3 3/9] jobs: add exit shim

2018-08-31 Thread Jeff Cody
On Wed, Aug 29, 2018 at 09:57:28PM -0400, John Snow wrote: > All jobs do the same thing when they leave their running loop: > - Store the return code in a structure > - wait to receive this structure in the main thread > - signal job completion via job_completed > > Few jobs do anything beyond

Re: [Qemu-block] [PATCH v3 5/9] block/mirror: utilize job_exit shim

2018-08-31 Thread Max Reitz
On 2018-08-30 03:57, John Snow wrote: > Change the manual deferment to mirror_exit into the implicit > callback to job_exit and the mirror_exit callback. > > This does change the order of some bdrv_unref calls and job_completed, > but thanks to the new context in which we call .exit, this is safe

Re: [Qemu-block] [PATCH v3 8/9] jobs: remove ret argument to job_completed; privatize it

2018-08-31 Thread Max Reitz
On 2018-08-30 03:57, John Snow wrote: > Jobs are now expected to return their retcode on the stack, from the > .run callback, so we can remove that argument. > > job_cancel does not need to set -ECANCELED because job_completed will > update the return code itself if the job was canceled. > >

Re: [Qemu-block] [PATCH v3 1/9] jobs: change start callback to run callback

2018-08-31 Thread Jeff Cody
On Wed, Aug 29, 2018 at 09:57:26PM -0400, John Snow wrote: > Presently we codify the entry point for a job as the "start" callback, > but a more apt name would be "run" to clarify the idea that when this > function returns we consider the job to have "finished," except for > any cleanup which

Re: [Qemu-block] [PATCH v2 1/9] jobs: change start callback to run callback

2018-08-31 Thread Max Reitz
On 2018-08-30 02:06, John Snow wrote: > > > On 08/27/2018 05:30 AM, Max Reitz wrote: >> On 2018-08-24 00:08, John Snow wrote: >>> Presently we codify the entry point for a job as the "start" callback, >>> but a more apt name would be "run" to clarify the idea that when this >>> function returns

Re: [Qemu-block] [Qemu-devel] [PATCH v3 2/9] jobs: canonize Error object

2018-08-31 Thread Markus Armbruster
Eric Blake writes: > On 08/29/2018 08:57 PM, John Snow wrote: >> Jobs presently use both an Error object in the case of the create job, >> and char strings in the case of generic errors elsewhere. >> >> Unify the two paths as just j->err, and remove the extra argument from >> job_completed. The