Re: [Qemu-block] [PATCH v2 6/6] iotest 201: new test for qmp nbd-server-remove

2018-01-15 Thread Eric Blake
On 01/15/2018 08:40 AM, Vladimir Sementsov-Ogievskiy wrote: >> And the list archives show several threads of people complaining that >> ./check failing with a diff that merely shows: >> >> -. >> +..E.. > > didn't see that. usually, for failed iotests I see > > -. > +..E.. > > + some

Re: [Qemu-block] [Qemu-devel] [RFC PATCH 02/10] block/qapi: Add qcow2 create options to schema

2018-01-15 Thread Daniel P. Berrange
On Mon, Jan 15, 2018 at 03:07:15PM +0100, Kevin Wolf wrote: > Am 15.01.2018 um 14:51 hat Daniel P. Berrange geschrieben: > > On Mon, Jan 15, 2018 at 02:38:48PM +0100, Kevin Wolf wrote: > > > Am 12.01.2018 um 11:53 hat Daniel P. Berrange geschrieben: > > > > On Thu, Jan 11, 2018 at 08:52:17PM

Re: [Qemu-block] [PATCH v5 08/15] qcow2: skip writing zero buffers to empty COW areas

2018-01-15 Thread Alberto Garcia
On Wed 01 Nov 2017 04:44:01 PM CET, Anton Nefedov wrote: > If COW areas of the newly allocated clusters are zeroes on the backing image, > efficient bdrv_write_zeroes(flags=BDRV_REQ_ALLOCATE) can be used on the whole > cluster instead of writing explicit zero buffers later in perform_cow(). > >

Re: [Qemu-block] [PATCH v2 3/6] qapi: add nbd-server-remove

2018-01-15 Thread Eric Blake
On 01/12/2018 03:47 AM, Vladimir Sementsov-Ogievskiy wrote: > 09.01.2018 22:52, Eric Blake wrote: >> On 12/07/2017 09:50 AM, Vladimir Sementsov-Ogievskiy wrote: >>> Add command for export removing. It is needed for cases when we >>> don't want to keep export after the operation on it was

Re: [Qemu-block] [PATCH v2 6/6] iotest 201: new test for qmp nbd-server-remove

2018-01-15 Thread Vladimir Sementsov-Ogievskiy
12.01.2018 19:54, Eric Blake wrote: On 01/12/2018 05:43 AM, Vladimir Sementsov-Ogievskiy wrote: +++ b/tests/qemu-iotests/201.out @@ -0,0 +1,5 @@ +... +-- +Ran 7 tests I'm not a fan of python tests that are difficult to

Re: [Qemu-block] [PATCH v5 07/15] qcow2: move is_zero() up

2018-01-15 Thread Alberto Garcia
On Wed 01 Nov 2017 04:44:00 PM CET, Anton Nefedov wrote: > To be used in the following commit without a forward declaration. > > Signed-off-by: Anton Nefedov Reviewed-by: Alberto Garcia Berto

Re: [Qemu-block] [PATCH v2 3/6] qapi: add nbd-server-remove

2018-01-15 Thread Vladimir Sementsov-Ogievskiy
15.01.2018 18:09, Eric Blake wrote: On 01/12/2018 03:47 AM, Vladimir Sementsov-Ogievskiy wrote: 09.01.2018 22:52, Eric Blake wrote: On 12/07/2017 09:50 AM, Vladimir Sementsov-Ogievskiy wrote: Add command for export removing. It is needed for cases when we don't want to keep export after the

Re: [Qemu-block] [Qemu-devel] qcow2 autoloading bitmaps

2018-01-15 Thread John Snow
On 01/11/2018 10:15 AM, Vladimir Sementsov-Ogievskiy wrote: > 11.01.2018 17:43, Eric Blake wrote: >> On 01/11/2018 08:26 AM, Vladimir Sementsov-Ogievskiy wrote: >> >>> # @autoload: the bitmap will be automatically loaded when the image it >>> is stored >>> #    in is opened. This flag

Re: [Qemu-block] [PATCH v5 08/15] qcow2: skip writing zero buffers to empty COW areas

2018-01-15 Thread Anton Nefedov
On 15/1/2018 6:31 PM, Alberto Garcia wrote: On Wed 01 Nov 2017 04:44:01 PM CET, Anton Nefedov wrote: If COW areas of the newly allocated clusters are zeroes on the backing image, efficient bdrv_write_zeroes(flags=BDRV_REQ_ALLOCATE) can be used on the whole cluster instead of writing explicit

Re: [Qemu-block] [PATCH v5 08/15] qcow2: skip writing zero buffers to empty COW areas

2018-01-15 Thread Alberto Garcia
On Mon 15 Jan 2018 07:21:01 PM CET, Anton Nefedov wrote: >>> /** >>> + * Indicates that both COW areas are empty (nb_bytes == 0) >>> + * or filled with zeroes and do not require any more copying >>> + */ >>> +bool zero_cow; [...] >>> -if (start->nb_bytes == 0 &&

Re: [Qemu-block] [PATCH v2 6/6] iotest 201: new test for qmp nbd-server-remove

2018-01-15 Thread Vladimir Sementsov-Ogievskiy
15.01.2018 18:05, Eric Blake wrote: On 01/15/2018 08:40 AM, Vladimir Sementsov-Ogievskiy wrote: And the list archives show several threads of people complaining that ./check failing with a diff that merely shows: -. +..E.. didn't see that. usually, for failed iotests I see -. +..E..

[Qemu-block] [PATCH v6 4/9] block: Introduce buf register API

2018-01-15 Thread Fam Zheng
Allow block driver to map and unmap a buffer for later I/O, as a performance hint. Signed-off-by: Fam Zheng Message-Id: <20180110091846.10699-5-f...@redhat.com> Reviewed-by: Stefan Hajnoczi --- block/block-backend.c | 10 ++ block/io.c

[Qemu-block] [PATCH v6 9/9] qapi: Add NVMe driver options to the schema

2018-01-15 Thread Fam Zheng
Signed-off-by: Fam Zheng Message-Id: <20180110091846.10699-10-f...@redhat.com> Reviewed-by: Stefan Hajnoczi --- qapi/block-core.json | 17 - 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/qapi/block-core.json

Re: [Qemu-block] [PATCH V5] pci: removed the is_express field since a uniform interface was inserted

2018-01-15 Thread Michael S. Tsirkin
On Mon, Dec 18, 2017 at 05:21:40PM +0200, Yoni Bettan wrote: > according to Eduardo Habkost's commit fd3b02c889 all PCIEs now implement > INTERFACE_PCIE_DEVICE so we don't need is_express field anymore. > > Devices that implements only INTERFACE_PCIE_DEVICE (is_express == 1) > or > devices that

[Qemu-block] [PATCH v6 5/9] block/nvme: Implement .bdrv_(un)register_buf

2018-01-15 Thread Fam Zheng
Forward these two calls to the IOVA manager. Signed-off-by: Fam Zheng Message-Id: <20180110091846.10699-6-f...@redhat.com> Reviewed-by: Stefan Hajnoczi --- block/nvme.c | 24 1 file changed, 24 insertions(+) diff --git

[Qemu-block] [PATCH v6 2/9] util: Introduce vfio helpers

2018-01-15 Thread Fam Zheng
This is a library to manage the host vfio interface, which could be used to implement userspace device driver code in QEMU such as NVMe or net controllers. Signed-off-by: Fam Zheng Message-Id: <20180110091846.10699-3-f...@redhat.com> Reviewed-by: Stefan Hajnoczi

[Qemu-block] [PATCH v6 1/9] stubs: Add stubs for ram block API

2018-01-15 Thread Fam Zheng
These functions will be wanted by block-obj-y but the actual definition is in obj-y, so stub them to keep the linker happy. Signed-off-by: Fam Zheng Acked-by: Paolo Bonzini Message-Id: <20180110091846.10699-2-f...@redhat.com> Reviewed-by: Stefan Hajnoczi

[Qemu-block] [PATCH v6 0/9] block: Add VFIO based driver for NVMe device

2018-01-15 Thread Fam Zheng
v6: Fix the cleaning up around nvme_init() and write cache setup in nvme_file_open(). [Stefan] Add Stefan's rev-by to the doc patch. Rebase on to Paolo's coroutine series for q->free_req_queue: Based-on: 20180115220822.10156-1-pbonz...@redhat.com ([PATCH 0/4] coroutine-lock:

[Qemu-block] [PATCH v6 6/9] qemu-img: Map bench buffer

2018-01-15 Thread Fam Zheng
Signed-off-by: Fam Zheng Message-Id: <20180110091846.10699-7-f...@redhat.com> Reviewed-by: Stefan Hajnoczi --- qemu-img.c | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/qemu-img.c b/qemu-img.c index 68b375f998..28d0e4e9f8 100644

[Qemu-block] [PULL 03/33] vhost-user-blk: introduce a new vhost-user-blk host device

2018-01-15 Thread Michael S. Tsirkin
From: Changpeng Liu This commit introduces a new vhost-user device for block, it uses a chardev to connect with the backend, same with Qemu virito-blk device, Guest OS still uses the virtio-blk frontend driver. To use it, start QEMU with command line like this:

[Qemu-block] [PATCH v6 7/9] block: Move NVMe constants to a separate header

2018-01-15 Thread Fam Zheng
Signed-off-by: Fam Zheng Message-Id: <20180110091846.10699-8-f...@redhat.com> Reviewed-by: Stefan Hajnoczi --- block/nvme.c | 7 +- hw/block/nvme.h | 698 +- include/block/nvme.h | 700

[Qemu-block] [PATCH v6 3/9] block: Add VFIO based NVMe driver

2018-01-15 Thread Fam Zheng
This is a new protocol driver that exclusively opens a host NVMe controller through VFIO. It achieves better latency than linux-aio by completely bypassing host kernel vfs/block layer. $rw-$bs-$iodepth linux-aio nvme:// randread-4k-1

[Qemu-block] [PATCH v6 8/9] docs: Add section for NVMe VFIO driver

2018-01-15 Thread Fam Zheng
Signed-off-by: Fam Zheng Message-Id: <20180110091846.10699-9-f...@redhat.com> Reviewed-by: Stefan Hajnoczi --- docs/qemu-block-drivers.texi | 37 + 1 file changed, 37 insertions(+) diff --git

[Qemu-block] [PATCH 4/4] curl: convert to CoQueue

2018-01-15 Thread Paolo Bonzini
Now that CoQueues can use a QemuMutex for thread-safety, there is no need for curl to roll its own coroutine queue. Coroutines can be placed directly on the queue instead of using a list of CURLAIOCBs. Signed-off-by: Paolo Bonzini --- block/curl.c | 20

[Qemu-block] [PATCH 1/4] lockable: add QemuLockable

2018-01-15 Thread Paolo Bonzini
QemuLockable is a polymorphic lock type that takes an object and knows which function to use for locking and unlocking. The implementation could use C11 _Generic, but since the support is not very widespread I am instead using __builtin_choose_expr and __builtin_types_compatible_p, which are

[Qemu-block] [PATCH 0/4] coroutine-lock: polymorphic CoQueue

2018-01-15 Thread Paolo Bonzini
There are cases in which a queued coroutine must be restarted from non-coroutine context (with qemu_co_enter_next). In this cases, qemu_co_enter_next also needs to be thread-safe, but it cannot use a CoMutex and so cannot qemu_co_queue_wait. This happens in curl (which right now is rolling its

Re: [Qemu-block] [PATCH v4 01/16] fdc: move object structures to header file

2018-01-15 Thread John Snow
On 01/06/2018 10:37 AM, Hervé Poussineau wrote: > We are now able to embed floppy controllers in another object. > > Acked-by: Michael S. Tsirkin > Acked-by: Paolo Bonzini > Signed-off-by: Hervé Poussineau Hmm, if it's necessary to

[Qemu-block] [PATCH 2/4] coroutine-lock: convert CoQueue to use QemuLockable

2018-01-15 Thread Paolo Bonzini
There are cases in which a queued coroutine must be restarted from non-coroutine context (with qemu_co_enter_next). In this cases, qemu_co_enter_next also needs to be thread-safe, but it cannot use a CoMutex and so cannot qemu_co_queue_wait. Use QemuLockable so that the CoQueue can

[Qemu-block] [PATCH 3/4] coroutine-lock: make qemu_co_enter_next thread-safe

2018-01-15 Thread Paolo Bonzini
qemu_co_queue_next does not need to release and re-acquire the mutex, because the queued coroutine does not run immediately. However, this does not hold for qemu_co_enter_next. Now that qemu_co_queue_wait can synchronize (via QemuLockable) with code that is not running in coroutine context, it's

[Qemu-block] [PATCH v2] file-posix: specify expected filetypes

2018-01-15 Thread John Snow
I don't think there's a legitimate reason to open directories as if they were files. This prevents QEMU from opening and attempting to probe a directory inode, which can break in exciting ways. One of those ways is lseek on ext4/xfs, which will return 0x7fff as the file size instead of

Re: [Qemu-block] [Qemu-devel] [PATCH] file-posix: refuse to open directories

2018-01-15 Thread Kevin Wolf
Am 13.01.2018 um 00:30 hat John Snow geschrieben: > On 12/22/2017 08:00 AM, Kevin Wolf wrote: > > Am 21.12.2017 um 23:44 hat John Snow geschrieben: > >> I don't think there's a legitimate reason to open directories as if > >> they were files. This prevents QEMU from opening and attempting to probe

Re: [Qemu-block] [Qemu-devel] [RFC PATCH 02/10] block/qapi: Add qcow2 create options to schema

2018-01-15 Thread Kevin Wolf
Am 12.01.2018 um 11:53 hat Daniel P. Berrange geschrieben: > On Thu, Jan 11, 2018 at 08:52:17PM +0100, Kevin Wolf wrote: > > Signed-off-by: Kevin Wolf > > --- > > qapi/block-core.json | 33 - > > 1 file changed, 32 insertions(+), 1 deletion(-) >

Re: [Qemu-block] [Qemu-devel] [RFC PATCH 02/10] block/qapi: Add qcow2 create options to schema

2018-01-15 Thread Kevin Wolf
Am 15.01.2018 um 14:51 hat Daniel P. Berrange geschrieben: > On Mon, Jan 15, 2018 at 02:38:48PM +0100, Kevin Wolf wrote: > > Am 12.01.2018 um 11:53 hat Daniel P. Berrange geschrieben: > > > On Thu, Jan 11, 2018 at 08:52:17PM +0100, Kevin Wolf wrote: > > > > Signed-off-by: Kevin Wolf

Re: [Qemu-block] [Qemu-devel] [RFC PATCH 02/10] block/qapi: Add qcow2 create options to schema

2018-01-15 Thread Daniel P. Berrange
On Mon, Jan 15, 2018 at 02:38:48PM +0100, Kevin Wolf wrote: > Am 12.01.2018 um 11:53 hat Daniel P. Berrange geschrieben: > > On Thu, Jan 11, 2018 at 08:52:17PM +0100, Kevin Wolf wrote: > > > Signed-off-by: Kevin Wolf > > > --- > > > qapi/block-core.json | 33