Re: [Qemu-block] [PATCH v5 11/11] block/backup: use backup-top instead of write notifiers

2019-01-18 Thread Max Reitz
On 29.12.18 13:20, Vladimir Sementsov-Ogievskiy wrote: > Drop write notifiers and use filter node instead. Changes: > > 1. copy-before-writes now handled by filter node, so, drop all >is_write_notifier arguments. > > 2. we don't have intersecting requests, so their handling is dropped. >

Re: [Qemu-block] [PATCH v4 01/21] iotests: Make 233 output more reliable

2019-01-18 Thread Eric Blake
On 1/18/19 4:02 AM, Daniel P. Berrangé wrote: > On Thu, Jan 17, 2019 at 01:36:38PM -0600, Eric Blake wrote: >> We have a race between the nbd server and the client both trying >> to report errors at once which can make the test sometimes fail >> if the output lines swap order under load. Break

Re: [Qemu-block] [PATCH v4 00/21] nbd: add qemu-nbd --list

2019-01-18 Thread Eric Blake
On 1/18/19 7:47 AM, Vladimir Sementsov-Ogievskiy wrote: >> >> Interesting, I don't get it again. Searched in outlook online, I found only >> v2 of it, >> checked spam folder and filters. Magic. Weird indeed. Maybe there's enough @ signs due to .texi content in there that a spam filter

Re: [Qemu-block] [PATCH 2/3] file: Expose some protocol-specific information

2019-01-18 Thread Eric Blake
On 1/18/19 8:08 AM, Vladimir Sementsov-Ogievskiy wrote: >> >> +## >> +# @ImageInfoSpecificFile: >> +# >> +# Details about a file protocol BDS. >> +# >> +# @align: Alignment in use >> +# >> +# @discard: True if discard operations can be attempted >> +# >> +# @write-zero: True if efficient write

Re: [Qemu-block] [PATCH v4 11/21] nbd/client: Split out nbd_send_meta_query()

2019-01-18 Thread Vladimir Sementsov-Ogievskiy
17.01.2019 22:36, Eric Blake wrote: > Refactor nbd_negotiate_simple_meta_context() to pull out the > code that can be reused to send a LIST request for 0 or 1 query. > No semantic change. The old comment about 'sizeof(uint32_t)' > being equivalent to '/* number of queries */' is no longer >

Re: [Qemu-block] [PATCH v4] qemu-io: Add generic function for reinitializing optind.

2019-01-18 Thread Daniel P . Berrangé
On Fri, Jan 18, 2019 at 10:11:14AM +, Richard W.M. Jones wrote: > On FreeBSD 11.2: > > $ nbdkit memory size=1M --run './qemu-io -f raw -c "aio_write 0 512" $nbd' > Parsing error: non-numeric argument, or extraneous/unrecognized suffix -- > aio_write > > After main option parsing, we

Re: [Qemu-block] [PATCH v3 8/9] qcow2: bdrv_co_pwritev: move encryption code out of the lock

2019-01-18 Thread Vladimir Sementsov-Ogievskiy
18.01.2019 12:51, Alberto Garcia wrote: > On Tue 08 Jan 2019 06:06:54 PM CET, Vladimir Sementsov-Ogievskiy wrote: >> Encryption will be done in threads, to take benefit of it, we should >> move it out of the lock first. >> >> Signed-off-by: Vladimir Sementsov-Ogievskiy >> --- >> block/qcow2.c |

Re: [Qemu-block] [PATCH v4] qemu-io: Add generic function for reinitializing optind.

2019-01-18 Thread Eric Blake
On 1/18/19 4:11 AM, Richard W.M. Jones wrote: > On FreeBSD 11.2: > > $ nbdkit memory size=1M --run './qemu-io -f raw -c "aio_write 0 512" $nbd' > Parsing error: non-numeric argument, or extraneous/unrecognized suffix -- > aio_write > > After main option parsing, we reinitialize optind so we

Re: [Qemu-block] [PATCH v3 8/9] qcow2: bdrv_co_pwritev: move encryption code out of the lock

2019-01-18 Thread Vladimir Sementsov-Ogievskiy
18.01.2019 17:39, Alberto Garcia wrote: > On Fri 18 Jan 2019 12:37:42 PM CET, Vladimir Sementsov-Ogievskiy wrote: >> 18.01.2019 12:51, Alberto Garcia wrote: >>> On Tue 08 Jan 2019 06:06:54 PM CET, Vladimir Sementsov-Ogievskiy wrote: Encryption will be done in threads, to take benefit of it,

Re: [Qemu-block] qemu-img info ran at 100% CPU for 45 minutes without writing a byte (stopped it)

2019-01-18 Thread Alberto Garcia
On Wed 16 Jan 2019 02:15:24 AM CET, james harvey wrote: > I ran: > > # qemu-img convert /var/lib/libvirt/images/win7.qcow2 -O raw > /mnt/tmpqcow/win7.raw > > 45 minutes later, qemu-img had been running with 100% CPU every time I > checked, and it had allocated the raw file, but still hadn't

Re: [Qemu-block] [Qemu-devel] [PATCH v4 00/21] nbd: add qemu-nbd --list

2019-01-18 Thread Eric Blake
On 1/17/19 1:36 PM, Eric Blake wrote: > I got tired of debugging whether a server was advertising the > correct things during negotiation by inspecting the trace > logs of qemu-io as client - not to mention that without SOME > sort of client tracing particular commands, we can't easily >

Re: [Qemu-block] [PATCH v3 8/9] qcow2: bdrv_co_pwritev: move encryption code out of the lock

2019-01-18 Thread Alberto Garcia
On Fri 18 Jan 2019 12:37:42 PM CET, Vladimir Sementsov-Ogievskiy wrote: > 18.01.2019 12:51, Alberto Garcia wrote: >> On Tue 08 Jan 2019 06:06:54 PM CET, Vladimir Sementsov-Ogievskiy wrote: >>> Encryption will be done in threads, to take benefit of it, we should >>> move it out of the lock first.

Re: [Qemu-block] [PATCH 0/3] block: fix last address-of-packed-member warnings

2019-01-18 Thread Kevin Wolf
Am 10.12.2018 um 12:26 hat Peter Maydell geschrieben: > This patchset fixes the remaining clang warnings in the block/ code > about taking the address of a packed struct member, which are all > in block/vpc and block/vdi code handling UUIDs. Mostly I fix > these by copying the unaligned field

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/6] Acquire the AioContext during _realize()

2019-01-18 Thread Stefan Hajnoczi
On Thu, Jan 17, 2019 at 02:24:10PM +0100, Alberto Garcia wrote: > On Wed 16 Jan 2019 02:54:44 PM CET, Stefan Hajnoczi wrote: > > This patch series makes virtio-blk and virtio-scsi more robust, > > although I haven't checked what happens if the drive is attached to a > > different IOThread than the

[Qemu-block] [PATCH v4] qemu-io: Add generic function for reinitializing optind.

2019-01-18 Thread Richard W.M. Jones
On FreeBSD 11.2: $ nbdkit memory size=1M --run './qemu-io -f raw -c "aio_write 0 512" $nbd' Parsing error: non-numeric argument, or extraneous/unrecognized suffix -- aio_write After main option parsing, we reinitialize optind so we can parse each command. However reinitializing optind to 0

[Qemu-block] [PATCH v4] qemu-io: Add generic function for reinitializing optind.

2019-01-18 Thread Richard W.M. Jones
v3 was posted here: https://lists.gnu.org/archive/html/qemu-devel/2019-01/msg01536.html In v4: - Detect optreset at ./configure time and use ifdef HAVE_OPTRESET. - Adjusted comment style. - Checkpatch is now clean. - Retested on Linux, FreeBSD 11.2 and OpenBSD 6.4. Rich.

Re: [Qemu-block] [PATCH v4 16/21] nbd/client: Refactor nbd_opt_go() to support NBD_OPT_INFO

2019-01-18 Thread Vladimir Sementsov-Ogievskiy
17.01.2019 22:36, Eric Blake wrote: > Rename the function to nbd_opt_info_or_go() with an added parameter > and slight changes to comments and trace messages, in order to > reuse the function for NBD_OPT_INFO. > > Signed-off-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy -- Best

Re: [Qemu-block] [PATCH v5 10/11] block/backup: tiny refactor backup_job_create

2019-01-18 Thread Max Reitz
On 29.12.18 13:20, Vladimir Sementsov-Ogievskiy wrote: > Move copy-bitmap find/create code. It's needed for the following > commit, as we'll need copy_bitmap before actual block job creation. Do > it in a separate commit to simplify review. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- >

Re: [Qemu-block] [PATCH v3 8/9] qcow2: bdrv_co_pwritev: move encryption code out of the lock

2019-01-18 Thread Alberto Garcia
On Tue 08 Jan 2019 06:06:54 PM CET, Vladimir Sementsov-Ogievskiy wrote: > Encryption will be done in threads, to take benefit of it, we should > move it out of the lock first. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/qcow2.c | 35 +-- > 1 file

Re: [Qemu-block] [PATCH v4 01/21] iotests: Make 233 output more reliable

2019-01-18 Thread Daniel P . Berrangé
On Fri, Jan 18, 2019 at 08:28:21AM +, Vladimir Sementsov-Ogievskiy wrote: > 17.01.2019 22:36, Eric Blake wrote: > > We have a race between the nbd server and the client both trying > > to report errors at once which can make the test sometimes fail > > if the output lines swap order under

Re: [Qemu-block] [PATCH v4 01/21] iotests: Make 233 output more reliable

2019-01-18 Thread Daniel P . Berrangé
On Thu, Jan 17, 2019 at 01:36:38PM -0600, Eric Blake wrote: > We have a race between the nbd server and the client both trying > to report errors at once which can make the test sometimes fail > if the output lines swap order under load. Break the race by > collecting server messages into a file

Re: [Qemu-block] [PATCH v4 00/21] nbd: add qemu-nbd --list

2019-01-18 Thread Vladimir Sementsov-Ogievskiy
18.01.2019 11:15, Vladimir Sementsov-Ogievskiy wrote: > 17.01.2019 22:36, Eric Blake wrote: >> I got tired of debugging whether a server was advertising the >> correct things during negotiation by inspecting the trace >> logs of qemu-io as client - not to mention that without SOME >> sort of

Re: [Qemu-block] [PATCH 2/3] file: Expose some protocol-specific information

2019-01-18 Thread Vladimir Sementsov-Ogievskiy
17.01.2019 18:33, Eric Blake wrote: > When analyzing performance, it might be nice to let 'qemu-img info' > expose details that it learned from the underlying file or block > device. Start the process by picking a few useful items determined > during raw_open_common(). > > Signed-off-by: Eric

Re: [Qemu-block] [PATCH v5 07/11] block: introduce backup-top filter driver

2019-01-18 Thread Max Reitz
On 17.01.19 13:13, Vladimir Sementsov-Ogievskiy wrote: > 16.01.2019 19:02, Max Reitz wrote: >> On 29.12.18 13:20, Vladimir Sementsov-Ogievskiy wrote: >>> Backup-top filter does copy-before-write operation. It should be >>> inserted above active disk and has a target node for CBW, like the >>>

Re: [Qemu-block] [PATCH v4 15/21] nbd/client: Pull out oldstyle size determination

2019-01-18 Thread Vladimir Sementsov-Ogievskiy
17.01.2019 22:36, Eric Blake wrote: > Another refactoring creating nbd_negotiate_finish_oldstyle() > for further reuse during 'qemu-nbd --list'. > > Signed-off-by: Eric Blake > Reviewed-by: Richard W.M. Jones you forget my r-b: Reviewed-by: Vladimir Sementsov-Ogievskiy -- Best regards,

Re: [Qemu-block] [PATCH v2 0/6] Acquire the AioContext during _realize()

2019-01-18 Thread Kevin Wolf
Am 14.01.2019 um 15:23 hat Alberto Garcia geschrieben: > This series acquires the AioContext in the _realize() functions of > several devices before making use of their block backends. This fixes > at least a couple of crashes (in virtio-blk and scsi). The other > devices don't currently support

Re: [Qemu-block] [PATCH v4 05/21] nbd/server: Hoist length check to qmp_nbd_server_add

2019-01-18 Thread Vladimir Sementsov-Ogievskiy
17.01.2019 22:36, Eric Blake wrote: > We only had two callers to nbd_export_new; qemu-nbd.c always > passed a valid offset/length pair (because it already checked > the file length, to ensure that offset was in bounds), while > blockdev-nbd.c always passed 0/-1. Then nbd_export_new reduces > the

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/6] Acquire the AioContext during _realize()

2019-01-18 Thread Kevin Wolf
Am 18.01.2019 um 10:56 hat Stefan Hajnoczi geschrieben: > On Thu, Jan 17, 2019 at 02:24:10PM +0100, Alberto Garcia wrote: > > On Wed 16 Jan 2019 02:54:44 PM CET, Stefan Hajnoczi wrote: > > > This patch series makes virtio-blk and virtio-scsi more robust, > > > although I haven't checked what

Re: [Qemu-block] [PATCH v4 19/21] qemu-nbd: Add --list option

2019-01-18 Thread Vladimir Sementsov-Ogievskiy
17.01.2019 22:36, Eric Blake wrote: > We want to be able to detect whether a given qemu NBD server is > exposing the right export(s) and dirty bitmaps, at least for > regression testing. We could use 'nbd-client -l' from the upstream > NBD project to list exports, but it's annoying to rely on >

Re: [Qemu-block] [PATCH v4 02/21] maint: Allow for EXAMPLES in texi2pod

2019-01-18 Thread Vladimir Sementsov-Ogievskiy
17.01.2019 22:36, Eric Blake wrote: > The next commit will add an EXAMPLES section to qemu-nbd.8; > for that to work, we need to recognize EXAMPLES in texi2pod. > We also need to add a dependency from all man pages against > the generator script, since a change to the generator may > cause the

Re: [Qemu-block] [Qemu-devel] block: Update flags in bdrv_set_read_only()

2019-01-18 Thread Kevin Wolf
[ Cc: qemu-block - noticed only now that it was missing ] Am 14.01.2019 um 12:01 hat Kevin Wolf geschrieben: > Am 12.01.2019 um 18:08 hat Michael Tokarev geschrieben: > > commit eeae6a596b0efc092f5101c67683053e245e6250 > > Author: Kevin Wolf > > Date: Tue Oct 9 16:57:12 2018 +0200

Re: [Qemu-block] [PATCH v4 17/21] nbd/client: Add nbd_receive_export_list()

2019-01-18 Thread Vladimir Sementsov-Ogievskiy
17.01.2019 22:36, Eric Blake wrote: > We want to be able to detect whether a given qemu NBD server is > exposing the right export(s) and dirty bitmaps, at least for > regression testing. We could use 'nbd-client -l' from the upstream > NBD project to list exports, but it's annoying to rely on >

Re: [Qemu-block] [PATCH v3 9/9] qcow2: do encryption in threads

2019-01-18 Thread Alberto Garcia
On Tue 08 Jan 2019 06:06:55 PM CET, Vladimir Sementsov-Ogievskiy wrote: > Do encryption/decryption in threads, like it is already done for > compression. This improves asynchronous encrypted io. > > Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Berto

Re: [Qemu-block] [Qemu-devel] [PATCH 0/3] block: fix last address-of-packed-member warnings

2019-01-18 Thread Peter Maydell
Ping? thanks -- PMM On Mon, 10 Dec 2018 at 11:28, Peter Maydell wrote: > > This patchset fixes the remaining clang warnings in the block/ code > about taking the address of a packed struct member, which are all > in block/vpc and block/vdi code handling UUIDs. Mostly I fix > these by copying

Re: [Qemu-block] [PATCH 1/3] block: Expose protocol-specific data to 'qemu-img info'

2019-01-18 Thread Vladimir Sementsov-Ogievskiy
17.01.2019 18:33, Eric Blake wrote: > 'qemu-img info' is useful for showing additional information > about an image - but sometimes the interesting information is > specific to the protocol rather than to the format layer. Set > the groundwork for showing this information; further patches > will

Re: [Qemu-block] [PATCH v4 00/21] nbd: add qemu-nbd --list

2019-01-18 Thread Eric Blake
On 1/18/19 7:47 AM, Vladimir Sementsov-Ogievskiy wrote: > > > Only expose MBR partition @var{num}. Understands physical partitions > > 1-4 and logical partitions 5-8. > > I'm afraid, I'm too lazy to sort out these things I don't know, so just > believe. It at least > corresponds to limits

Re: [Qemu-block] [PATCH v4 00/21] nbd: add qemu-nbd --list

2019-01-18 Thread Vladimir Sementsov-Ogievskiy
17.01.2019 22:36, Eric Blake wrote: > I got tired of debugging whether a server was advertising the > correct things during negotiation by inspecting the trace > logs of qemu-io as client - not to mention that without SOME > sort of client tracing particular commands, we can't easily > regression

Re: [Qemu-block] [PATCH v4 01/21] iotests: Make 233 output more reliable

2019-01-18 Thread Vladimir Sementsov-Ogievskiy
17.01.2019 22:36, Eric Blake wrote: > We have a race between the nbd server and the client both trying > to report errors at once which can make the test sometimes fail > if the output lines swap order under load. Break the race by > collecting server messages into a file and then replaying that