Re: [PATCH v1 1/4] virtio: protect non-modern devices from too big virtqueue size setting

2019-11-05 Thread Denis Plotnikov
On 05.11.2019 23:56, Michael S. Tsirkin wrote: > On Tue, Nov 05, 2019 at 07:11:02PM +0300, Denis Plotnikov wrote: >> The patch protects from creating illegal virtio device configuration >> via direct virtqueue size property setting. >> >> Signed-off-by: Denis Plotnikov >> --- >>

Re: [PATCH v1 1/4] virtio: protect non-modern devices from too big virtqueue size setting

2019-11-05 Thread Michael S. Tsirkin
On Tue, Nov 05, 2019 at 07:11:02PM +0300, Denis Plotnikov wrote: > The patch protects from creating illegal virtio device configuration > via direct virtqueue size property setting. > > Signed-off-by: Denis Plotnikov > --- > hw/virtio/virtio-blk-pci.c | 9 + >

Re: [PATCH v1 3/4] virtio: increase virtuqueue sizes in new machine types

2019-11-05 Thread Michael S. Tsirkin
On Tue, Nov 05, 2019 at 07:11:04PM +0300, Denis Plotnikov wrote: > Linux guests submit IO requests no longer than PAGE_SIZE * max_seg > field reported by SCSI controler. Thus typical sequential read with > 1 MB size results in the following pattern of the IO from the guest: > 8,16 115754

Re: [PATCH v1 2/4] virtio: make seg_max virtqueue size dependent

2019-11-05 Thread Michael S. Tsirkin
On Tue, Nov 05, 2019 at 07:11:03PM +0300, Denis Plotnikov wrote: > seg_max has a restriction to be less or equal to virtqueue size > according to Virtio 1.0 specification > > Although seg_max can't be set directly, it's worth to express this > dependancy directly in the code for sanity purpose. >

[PATCH] virtio-blk: advertise F_WCE (F_FLUSH) if F_CONFIG_WCE is advertised

2019-11-05 Thread Evgeny Yakovlev
Virtio spec 1.1 (and earlier), 5.2.5.2 Driver Requirements: Device Initialization: "Devices SHOULD always offer VIRTIO_BLK_F_FLUSH, and MUST offer it if they offer VIRTIO_BLK_F_CONFIG_WCE" Currently F_CONFIG_WCE and F_WCE are not connected to each other. Qemu will advertise F_CONFIG_WCE if

Re: [PATCH v4] qcow2-bitmap: Fix uint64_t left-shift overflow

2019-11-05 Thread Max Reitz
On 01.11.19 08:37, Tuguoyi wrote: > There are two issues in In check_constraints_on_bitmap(), > 1) The sanity check on the granularity will cause uint64_t > integer left-shift overflow when cluster_size is 2M and the > granularity is BIGGER than 32K. > 2) The way to calculate image size that the

[PATCH v1 2/4] virtio: make seg_max virtqueue size dependent

2019-11-05 Thread Denis Plotnikov
seg_max has a restriction to be less or equal to virtqueue size according to Virtio 1.0 specification Although seg_max can't be set directly, it's worth to express this dependancy directly in the code for sanity purpose. Signed-off-by: Denis Plotnikov --- hw/block/virtio-blk.c | 2 +-

[PATCH v1 3/4] virtio: increase virtuqueue sizes in new machine types

2019-11-05 Thread Denis Plotnikov
Linux guests submit IO requests no longer than PAGE_SIZE * max_seg field reported by SCSI controler. Thus typical sequential read with 1 MB size results in the following pattern of the IO from the guest: 8,16 115754 2.766095122 2071 D R 2095104 + 1008 [dd] 8,16 115755

[PATCH v1 0/4] virtio: fix IO request length in virtio SCSI/block

2019-11-05 Thread Denis Plotnikov
v1: * make seg_max size dependent on virtuqueue size * don't expose seg_max as property * add new machine types with increased queue size * add test to check the new machine types * check queue size for non-modern virtio devices --- From: "Denis V. Lunev" Linux guests submit IO

[PATCH v1 4/4] iotests: add test for virtio-scsi and virtio-blk machine type settings

2019-11-05 Thread Denis Plotnikov
It tests proper queue size settings for all available machine types. Signed-off-by: Denis Plotnikov --- tests/qemu-iotests/267 | 154 + tests/qemu-iotests/267.out | 1 + tests/qemu-iotests/group | 1 + 3 files changed, 156 insertions(+) create mode

[PATCH v1 1/4] virtio: protect non-modern devices from too big virtqueue size setting

2019-11-05 Thread Denis Plotnikov
The patch protects from creating illegal virtio device configuration via direct virtqueue size property setting. Signed-off-by: Denis Plotnikov --- hw/virtio/virtio-blk-pci.c | 9 + hw/virtio/virtio-scsi-pci.c | 10 ++ 2 files changed, 19 insertions(+) diff --git

Re: [RFC PATCH 00/18] Add qemu-storage-daemon

2019-11-05 Thread Stefan Hajnoczi
On Thu, Oct 17, 2019 at 03:01:46PM +0200, Kevin Wolf wrote: > This series adds a new tool 'qemu-storage-daemon', which can be used to > export and perform operations on block devices. There is some overlap > between qemu-img/qemu-nbd and the new qemu-storage-daemon, but there are > a few important

[PULL 10/11] image-fuzzer: Use errors parameter of subprocess.Popen()

2019-11-05 Thread Stefan Hajnoczi
From: Eduardo Habkost Instead of manually encoding stderr and stdout output, use `errors` parameter of subprocess.Popen(). This will make process.communicate() return unicode strings instead of bytes objects. Signed-off-by: Eduardo Habkost Reviewed-by: John Snow Reviewed-by: Philippe

[PULL 07/11] image-fuzzer: Use bytes constant for field values

2019-11-05 Thread Stefan Hajnoczi
From: Eduardo Habkost Field values are supposed to be bytes objects, not unicode strings. Change two constants that were declared as strings. Signed-off-by: Eduardo Habkost Reviewed-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Message-id: 20191016192430.25098-8-ehabk...@redhat.com

[PULL 08/11] image-fuzzer: Encode file name and file format to bytes

2019-11-05 Thread Stefan Hajnoczi
From: Eduardo Habkost Callers of create_image() will pass strings as arguments, but the Image class will expect bytes objects to be provided. Encode them inside create_image(). Signed-off-by: Eduardo Habkost Reviewed-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Message-id:

[PULL 03/11] image-fuzzer: Explicitly use integer division operator

2019-11-05 Thread Stefan Hajnoczi
From: Eduardo Habkost Most of the division expressions in image-fuzzer assume integer division. Use the // operator to keep the same behavior when we move to Python 3. Signed-off-by: Eduardo Habkost Reviewed-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Message-id:

[PULL 09/11] image-fuzzer: Run using python3

2019-11-05 Thread Stefan Hajnoczi
From: Eduardo Habkost image-fuzzer is now supposed to be ready to run using Python 3. Remove the __future__ imports and change the interpreter line to "#!/usr/bin/env python3". Signed-off-by: Eduardo Habkost Reviewed-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Message-id:

[PULL 06/11] image-fuzzer: Return bytes objects on string fuzzing functions

2019-11-05 Thread Stefan Hajnoczi
From: Eduardo Habkost No caller of fuzzer functions is interested in unicode string values, so replace them with bytes sequences. Signed-off-by: Eduardo Habkost Reviewed-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Message-id: 20191016192430.25098-7-ehabk...@redhat.com Message-Id:

[PULL 11/11] image-fuzzer: Use OSerror.strerror instead of tuple subscript

2019-11-05 Thread Stefan Hajnoczi
From: Eduardo Habkost OSError can't be used like a tuple on Python 3, so change the code to use `e.sterror` instead of `e[1]`. Reported-by: John Snow Signed-off-by: Eduardo Habkost Reviewed-by: John Snow Message-id: 20191021214117.18091-1-ehabk...@redhat.com Message-Id:

[PULL 04/11] image-fuzzer: Use io.StringIO

2019-11-05 Thread Stefan Hajnoczi
From: Eduardo Habkost StringIO.StringIO is not available on Python 3, but io.StringIO is available on both Python 2 and 3. io.StringIO is slightly different from the Python 2 StringIO module, though, so we need bytes coming from subprocess.Popen() to be explicitly decoded. Signed-off-by:

[PULL 02/11] image-fuzzer: Write bytes instead of string to image file

2019-11-05 Thread Stefan Hajnoczi
From: Eduardo Habkost This is necessary for Python 3 compatibility. Signed-off-by: Eduardo Habkost Reviewed-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Message-id: 20191016192430.25098-3-ehabk...@redhat.com Message-Id: <20191016192430.25098-3-ehabk...@redhat.com> Signed-off-by: Stefan

[PULL 00/11] Block patches

2019-11-05 Thread Stefan Hajnoczi
The following changes since commit 36609b4fa36f0ac934874371874416f7533a5408: Merge remote-tracking branch 'remotes/palmer/tags/palmer-for-master-4.2-sf1' into staging (2019-11-02 17:59:03 +) are available in the Git repository at: https://github.com/stefanha/qemu.git

[PULL 01/11] image-fuzzer: Open image files in binary mode

2019-11-05 Thread Stefan Hajnoczi
From: Eduardo Habkost This probably never caused problems because on Linux there's no actual newline conversion happening, but on Python 3 the binary/text distinction is stronger and we must explicitly open the image file in binary mode. Signed-off-by: Eduardo Habkost Reviewed-by: John Snow

[PULL 05/11] image-fuzzer: Use %r for all fiels at Field.__repr__()

2019-11-05 Thread Stefan Hajnoczi
From: Eduardo Habkost This makes the formatting code simpler, and safer if we change the type of self.value from str to bytes. Signed-off-by: Eduardo Habkost Reviewed-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Message-id: 20191016192430.25098-6-ehabk...@redhat.com Message-Id:

Re: [PATCH] image-fuzzer: Use OSerror.strerror instead of tuple subscript

2019-11-05 Thread Stefan Hajnoczi
On Mon, Oct 21, 2019 at 06:41:17PM -0300, Eduardo Habkost wrote: > OSError can't be used like a tuple on Python 3, so change the > code to use `e.sterror` instead of `e[1]`. > > Reported-by: John Snow > Signed-off-by: Eduardo Habkost > --- > tests/image-fuzzer/runner.py | 4 ++-- > 1 file

Re: [PATCH 00/10] image-fuzzer: Port to Python 3

2019-11-05 Thread Stefan Hajnoczi
On Wed, Oct 16, 2019 at 04:24:20PM -0300, Eduardo Habkost wrote: > This series ports image-fuzzer to Python 3. > > Eduardo Habkost (10): > image-fuzzer: Open image files in binary mode > image-fuzzer: Write bytes instead of string to image file > image-fuzzer: Explicitly use integer

Re: [RFC PATCH v2 00/26] Add subcluster allocation to qcow2

2019-11-05 Thread Max Reitz
On 26.10.19 23:25, Alberto Garcia wrote: > Hi, > > here's the new version of the patches to add subcluster allocation > support to qcow2. > > Please refer to the cover letter of the first version for a full > description of the patches: > >

Re: [RFC PATCH v2 25/26] qcow2: Allow preallocation and backing files if extended_l2 is set

2019-11-05 Thread Max Reitz
On 26.10.19 23:25, Alberto Garcia wrote: > Traditional qcow2 images don't allow preallocation if a backing file > is set. This is because once a cluster is allocated there is no way to > tell that its data should be read from the backing file. > > Extended L2 entries have individual allocation

Re: [RFC PATCH v2 24/26] qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit

2019-11-05 Thread Max Reitz
On 26.10.19 23:25, Alberto Garcia wrote: > Now that the implementation of subclusters is complete we can finally > add the necessary options to create and read images with this feature, > which we call "extended L2 entries". > > Signed-off-by: Alberto Garcia > --- > block/qcow2.c

Re: [RFC PATCH v2 22/26] qcow2: Add subcluster support to handle_alloc_space()

2019-11-05 Thread Max Reitz
On 26.10.19 23:25, Alberto Garcia wrote: > The bdrv_co_pwrite_zeroes() call here fills complete clusters with > zeroes, but it can happen that some subclusters are not part of the > write request or the copy-on-write. This patch makes sure that only > the affected subclusters are overwritten. > >

Re: [RFC PATCH v2 20/26] qcow2: Update L2 bitmap in qcow2_alloc_cluster_link_l2()

2019-11-05 Thread Max Reitz
On 26.10.19 23:25, Alberto Garcia wrote: > The L2 bitmap needs to be updated after each write to indicate what > new subclusters are now allocated. > > This needs to happen even if the cluster was already allocated and the > L2 entry was otherwise valid. > > Signed-off-by: Alberto Garcia > ---

Re: [RFC PATCH v2 18/26] qcow2: Add subcluster support to expand_zero_clusters_in_l1()

2019-11-05 Thread Max Reitz
On 26.10.19 23:25, Alberto Garcia wrote: > Two changes are needed in order to add subcluster support to this > function: deallocated clusters must have their bitmaps cleared, and > expanded clusters must have all the "subcluster allocated" bits set. > > Signed-off-by: Alberto Garcia > --- >