Re: [PATCH v3 01/33] block/nbd: fix channel object leak

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
25.05.2021 00:31, Eric Blake wrote: On Fri, Apr 16, 2021 at 11:08:39AM +0300, Vladimir Sementsov-Ogievskiy wrote: From: Roman Kagan nbd_free_connect_thread leaks the channel object if it hasn't been stolen. Unref it and fix the leak. Signed-off-by: Roman Kagan --- block/nbd.c | 1 + 1

Re: [PATCH v6 11/25] python: add pylint to pipenv

2021-05-24 Thread Cleber Rosa
On Wed, May 12, 2021 at 07:12:27PM -0400, John Snow wrote: > We are specifying >= pylint 2.7.x for several reasons: > > 1. For setup.cfg support, added in pylint 2.5.x > 2. To specify a version that has incompatibly dropped >bad-whitespace checks (2.6.x) > 3. 2.7.x fixes "unsubscriptable"

Re: [PATCH] hw/nvme/ctrl: fix functions style

2021-05-24 Thread Gollu Appalanaidu
On Fri, May 21, 2021 at 09:13:51AM +0200, Klaus Jensen wrote: On May 21 11:38, Gollu Appalanaidu wrote: Identify command related functions style fix. Signed-off-by: Gollu Appalanaidu --- hw/nvme/ctrl.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/hw/nvme/ctrl.c

[PATCH 2/2] tests/qtest/nvme-test: add boot partition read test

2021-05-24 Thread Gollu Appalanaidu
Add a test case for reading an NVMe Boot Partition without enabling the controller. Signed-off-by: Gollu Appalanaidu --- tests/qtest/nvme-test.c | 118 +++- 1 file changed, 117 insertions(+), 1 deletion(-) diff --git a/tests/qtest/nvme-test.c

[PATCH 1/2] hw/nvme: add support for boot partiotions

2021-05-24 Thread Gollu Appalanaidu
NVMe Boot Partitions provides an area that may be read by the host without initializing queues or even enabling the controller. This allows various platform initialization code to be stored on the NVMe device instead of some separete medium. This patch adds the read support for such an area, as

[PATCH 0/2] add boot partitions support and read test case

2021-05-24 Thread Gollu Appalanaidu
This series adds the boot partition feature as well test case for reading boot partition area. Gollu Appalanaidu (2): hw/nvme: add support for boot partiotions tests/qtest/nvme-test: add boot partition read test hw/nvme/ctrl.c | 200

Re: [PATCH v6 07/25] python: add MANIFEST.in

2021-05-24 Thread Cleber Rosa
On Wed, May 12, 2021 at 07:12:23PM -0400, John Snow wrote: > When creating a source distribution via 'python3 setup.py sdist', the > VERSION and PACKAGE.rst files aren't bundled by default. Create a > MANIFEST.in file that instructs the build tools to include these so that > installation from

Re: [PATCH v6 06/25] python: add directory structure README.rst files

2021-05-24 Thread Cleber Rosa
On Wed, May 12, 2021 at 07:12:22PM -0400, John Snow wrote: > Add short readmes to python/, python/qemu/, python/qemu/machine, > python/qemu/qmp, and python/qemu/utils that explain the directory > hierarchy. These readmes are visible when browsing the source on > e.g. gitlab/github and are designed

Re: [PATCH v3 01/33] block/nbd: fix channel object leak

2021-05-24 Thread Eric Blake
On Fri, Apr 16, 2021 at 11:08:39AM +0300, Vladimir Sementsov-Ogievskiy wrote: > From: Roman Kagan > > nbd_free_connect_thread leaks the channel object if it hasn't been > stolen. > > Unref it and fix the leak. > > Signed-off-by: Roman Kagan > --- > block/nbd.c | 1 + > 1 file changed, 1

Re: [PATCH v2 3/3] qapi: deprecate drive-backup

2021-05-24 Thread John Snow
On 5/24/21 10:06 AM, Vladimir Sementsov-Ogievskiy wrote: 15.05.2021 01:38, John Snow wrote: On 5/6/21 5:57 AM, Kashyap Chamarthy wrote: TODO: We also need to deprecate drive-backup transaction action.. But union members in QAPI doesn't support 'deprecated' feature. I tried to dig a bit, but

Re: [PULL 0/8] Block patches

2021-05-24 Thread Peter Maydell
On Mon, 24 May 2021 at 14:02, Stefan Hajnoczi wrote: > > The following changes since commit 6c769690ac845fa62642a5f93b4e4bd906adab95: > > Merge remote-tracking branch > 'remotes/vsementsov/tags/pull-simplebench-2021-05-04' into staging > (2021-05-21 12:02:34 +0100) > > are available in the

Re: [PATCH 0/2] vvfat: fix two crashes.

2021-05-24 Thread Programmingkid
> On May 24, 2021, at 12:56 PM, Vladimir Sementsov-Ogievskiy > wrote: > > 24.05.2021 19:06, Programmingkid wrote: >>> On May 24, 2021, at 11:55 AM, Vladimir Sementsov-Ogievskiy >>> wrote: >>> >>> 24.05.2021 18:41, Programmingkid wrote: > On May 24, 2021, at 6:12 AM, Vladimir

Re: [PATCH 0/2] vvfat: fix two crashes.

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
24.05.2021 19:06, Programmingkid wrote: On May 24, 2021, at 11:55 AM, Vladimir Sementsov-Ogievskiy wrote: 24.05.2021 18:41, Programmingkid wrote: On May 24, 2021, at 6:12 AM, Vladimir Sementsov-Ogievskiy wrote: Hi! As reported by Programmingkid, command qemu-system-ppc -usb -device

[PATCH v2 4/6] block: feature detection for host block support

2021-05-24 Thread Paolo Bonzini
From: Joelle van Dyne On Darwin (iOS), there are no system level APIs for directly accessing host block devices. We detect this at configure time. Signed-off-by: Joelle van Dyne Message-Id: <20210315180341.31638-...@getutm.app> Signed-off-by: Paolo Bonzini --- block/file-posix.c | 33

[PATCH v2 6/6] block: detect DKIOCGETBLOCKCOUNT/SIZE before use

2021-05-24 Thread Paolo Bonzini
From: Joelle van Dyne iOS hosts do not have these defined so we fallback to the default behaviour. Co-authored-by: Warner Losh Reviewed-by: Peter Maydell Signed-off-by: Joelle van Dyne Message-Id: <20210315180341.31638-...@getutm.app> Signed-off-by: Paolo Bonzini --- block/file-posix.c |

[PATCH v2 0/6] block: file-posix queue

2021-05-24 Thread Paolo Bonzini
Hi Max/Kevin, this is a combination of two series that both affect host block device support in block/file-posix.c. I suspect both of them evaded your radar, so I'm grouping them together and sending them out again. v1->v2: add missing patch Joelle van Dyne (3): block: feature detection for

[PATCH v2 5/6] block: check for sys/disk.h

2021-05-24 Thread Paolo Bonzini
From: Joelle van Dyne Some BSD platforms do not have this header. Reviewed-by: Peter Maydell Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Joelle van Dyne Message-Id: <20210315180341.31638-...@getutm.app> Signed-off-by: Paolo Bonzini --- block.c | 2 +- meson.build | 1 + 2 files

[PATCH v2 3/6] file-posix: fix max_iov for /dev/sg devices

2021-05-24 Thread Paolo Bonzini
Even though it was only called for devices that have bs->sg set (which must be character devices), sg_get_max_segments looked at /sys/dev/block which only works for block devices. On Linux the sg driver has its own way to provide the maximum number of iovecs in a scatter/gather list.

Re: [PATCH 0/5] block: file-posix queue

2021-05-24 Thread Paolo Bonzini
On 18/05/21 15:23, Kevin Wolf wrote: Am 18.05.2021 um 10:38 hat Paolo Bonzini geschrieben: Hi Max/Kevin, this is a combination of two series that both affect host block device support in block/file-posix.c. I suspect both of them evaded your radar, so I'm grouping them together and sending

[PATCH v2 2/6] file-posix: try BLKSECTGET on block devices too, do not round to power of 2

2021-05-24 Thread Paolo Bonzini
bs->sg is only true for character devices, but block devices can also be used with scsi-block and scsi-generic. Unfortunately BLKSECTGET returns bytes in an int for /dev/sgN devices, and sectors in a short for block devices, so account for that in the code. The maximum transfer also need not be

[PATCH v2 1/6] scsi-generic: pass max_segments via max_iov field in BlockLimits

2021-05-24 Thread Paolo Bonzini
I/O to a disk via read/write is not limited by the number of segments allowed by the host adapter; the kernel can split requests if needed, and the limit imposed by the host adapter can be very low (256k or so) to avoid that SG_IO returns EINVAL if memory is heavily fragmented. Since this value

Re: [PATCH 0/2] vvfat: fix two crashes.

2021-05-24 Thread Programmingkid
> On May 24, 2021, at 11:55 AM, Vladimir Sementsov-Ogievskiy > wrote: > > 24.05.2021 18:41, Programmingkid wrote: >>> On May 24, 2021, at 6:12 AM, Vladimir Sementsov-Ogievskiy >>> wrote: >>> >>> Hi! >>> >>> As reported by Programmingkid, command >>> >>> qemu-system-ppc -usb -device

Re: [PATCH 0/2] vvfat: fix two crashes.

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
24.05.2021 18:41, Programmingkid wrote: On May 24, 2021, at 6:12 AM, Vladimir Sementsov-Ogievskiy wrote: Hi! As reported by Programmingkid, command qemu-system-ppc -usb -device usb-storage,drive=fat16 -drive file=fat:rw:fat-type=16:"",id=fat16,format=raw,if=none crashes. I tested it

Re: [PATCH 0/2] vvfat: fix two crashes.

2021-05-24 Thread Programmingkid
> On May 24, 2021, at 6:12 AM, Vladimir Sementsov-Ogievskiy > wrote: > > Hi! > > As reported by Programmingkid, command > > qemu-system-ppc -usb -device usb-storage,drive=fat16 -drive > file=fat:rw:fat-type=16:"",id=fat16,format=raw,if=none > > crashes. > > I tested it with

Re: [PATCH RFC] docs: document file-posix locking protocol

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
Ping. We need to synchronize access to qcow2 images between Qemu and our internal program. For this solution to be reliable, the locking protocol should be documented at least.. 22.03.2021 21:27, Vladimir Sementsov-Ogievskiy wrote: Let's document how we use file locks in file-posix driver,

Re: [PATCH v2 0/5] block permission updated follow-up

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
Ping. Now based on master 04.05.2021 12:45, Vladimir Sementsov-Ogievskiy wrote: v2: rebased on Kevin's "[PATCH 0/2] block: Fix Transaction leaks" 1: add assertions and drop extra declaration 2: add Alberto's r-b 3: improve commit message Based-on: <20210503110555.24001-1-kw...@redhat.com>

[PATCH v3 08/10] qcow2-refcount: improve style of check_refcounts_l1()

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
- use g_autofree for l1_table - better name for size in bytes variable - reduce code blocks nesting - whitespaces, braces, newlines Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-refcount.c | 98 +- 1 file changed, 50 insertions(+), 48

[PATCH v3 06/10] qcow2-refcount: check_refcounts_l2(): check l2_bitmap

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
Check subcluster bitmap of the l2 entry for different types of clusters: - for compressed it must be zero - for allocated check consistency of two parts of the bitmap - for unallocated all subclusters should be unallocated (or zero-plain) For unallocated clusters we can safely fix the

[PATCH v3 09/10] qcow2-refcount: check_refcounts_l1(): check reserved bits

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Tested-by: Kirill Tkhai --- block/qcow2.h | 1 + block/qcow2-refcount.c | 6 ++ 2 files changed, 7 insertions(+) diff --git a/block/qcow2.h b/block/qcow2.h index b8b1093b61..58fd7f1678 100644 --- a/block/qcow2.h

[PATCH v3 05/10] qcow2-refcount: fix_l2_entry_by_zero(): also zero L2 entry bitmap

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
We'll reuse the function to fix wrong L2 entry bitmap. Support it now. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- block/qcow2-refcount.c | 18 +++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/block/qcow2-refcount.c

[PATCH v3 04/10] qcow2-refcount: introduce fix_l2_entry_by_zero()

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
Split fix_l2_entry_by_zero() out of check_refcounts_l2() to be reused in further patch. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- block/qcow2-refcount.c | 87 +- 1 file changed, 60 insertions(+), 27 deletions(-) diff --git

[PATCH v3 02/10] qcow2: compressed read: simplify cluster descriptor passing

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
Let's pass the whole L2 entry and not bother with L2E_COMPRESSED_OFFSET_SIZE_MASK. It also helps further refactoring that adds generic qcow2_parse_compressed_l2_entry() helper. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: Alberto Garcia --- block/qcow2.h

[PATCH v3 10/10] qcow2-refcount: check_refblocks(): add separate message for reserved

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
Split checking for reserved bits out of aligned offset check. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Tested-by: Kirill Tkhai --- block/qcow2.h | 1 + block/qcow2-refcount.c | 10 +- 2 files changed, 10 insertions(+), 1 deletion(-) diff --git

[PATCH v3 01/10] qcow2-refcount: improve style of check_refcounts_l2()

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
- don't use same name for size in bytes and in entries - use g_autofree for l2_table - add whitespace - fix block comment style Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- block/qcow2-refcount.c | 47 +- 1 file changed, 24

[PATCH v3 07/10] qcow2-refcount: check_refcounts_l2(): check reserved bits

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Tested-by: Kirill Tkhai --- block/qcow2.h | 1 + block/qcow2-refcount.c | 12 +++- 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/block/qcow2.h b/block/qcow2.h index c0e1e83796..b8b1093b61

[PATCH v3 03/10] qcow2: introduce qcow2_parse_compressed_l2_entry() helper

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
Add helper to parse compressed l2_entry and use it everywhere instead of open-coding. Note, that in most places we move to precise coffset/csize instead of sector-aligned. Still it should work good enough for updating refcounts. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric

[PATCH v3 00/10] qcow2 check: check some reserved bits and subcluster bitmaps

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
Hi all! Here are some good refactorings and new (qemu-img check) checks for qcow2. v3: add r-b mark by Alberto and t-b marks by Kirill 07, 09: add missed "\n" Vladimir Sementsov-Ogievskiy (10): qcow2-refcount: improve style of check_refcounts_l2() qcow2: compressed read: simplify cluster

Re: [PATCH v2 3/3] qapi: deprecate drive-backup

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
15.05.2021 01:38, John Snow wrote: On 5/6/21 5:57 AM, Kashyap Chamarthy wrote: TODO: We also need to deprecate drive-backup transaction action.. But union members in QAPI doesn't support 'deprecated' feature. I tried to dig a bit, but failed :/ Markus, could you please help with it? At least by

Re: [PULL for-6.0 0/8] Block patches

2021-05-24 Thread Stefan Hajnoczi
Please ignore. I resent the pull request with the proper subject line and CC list. Stefan signature.asc Description: PGP signature

[PULL 8/8] coroutine-sleep: introduce qemu_co_sleep

2021-05-24 Thread Stefan Hajnoczi
From: Paolo Bonzini Allow using QemuCoSleep to sleep forever until woken by qemu_co_sleep_wake. This makes the logic of qemu_co_sleep_ns_wakeable easy to understand. In the future we will introduce an API that can work even if the sleep and wake happen from different threads. For now,

[PULL 7/8] coroutine-sleep: replace QemuCoSleepState pointer with struct in the API

2021-05-24 Thread Stefan Hajnoczi
From: Paolo Bonzini Right now, users of qemu_co_sleep_ns_wakeable are simply passing a pointer to QemuCoSleepState by reference to the function. But QemuCoSleepState really is just a Coroutine*; making the content of the struct public is just as efficient and lets us skip the user_state_pointer

[PULL 4/8] coroutine-sleep: disallow NULL QemuCoSleepState** argument

2021-05-24 Thread Stefan Hajnoczi
From: Paolo Bonzini Simplify the code by removing conditionals. qemu_co_sleep_ns can simply point the argument to an on-stack temporary. Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Paolo Bonzini Message-id: 20210517100548.28806-3-pbonz...@redhat.com Signed-off-by: Stefan

[PULL 5/8] coroutine-sleep: allow qemu_co_sleep_wake that wakes nothing

2021-05-24 Thread Stefan Hajnoczi
From: Paolo Bonzini All callers of qemu_co_sleep_wake are checking whether they are passing a NULL argument inside the pointer-to-pointer: do the check in qemu_co_sleep_wake itself. As a side effect, qemu_co_sleep_wake can be called more than once and it will only wake the coroutine once; after

[PULL 3/8] coroutine-sleep: use a stack-allocated timer

2021-05-24 Thread Stefan Hajnoczi
From: Paolo Bonzini The lifetime of the timer is well-known (it cannot outlive qemu_co_sleep_ns_wakeable, because it's deleted by the time the coroutine resumes), so it is not necessary to place it on the heap. Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Paolo Bonzini Message-id:

[PULL 6/8] coroutine-sleep: move timer out of QemuCoSleepState

2021-05-24 Thread Stefan Hajnoczi
From: Paolo Bonzini This simplification is enabled by the previous patch. Now aio_co_wake will only be called once, therefore we do not care about a spurious firing of the timer after a qemu_co_sleep_wake. Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Paolo Bonzini Message-id:

[PULL 2/8] bitops.h: Improve find_xxx_bit() documentation

2021-05-24 Thread Stefan Hajnoczi
From: Philippe Mathieu-Daudé Document the following functions return the bitmap size if no matching bit is found: - find_first_bit - find_next_bit - find_last_bit - find_first_zero_bit - find_next_zero_bit Reviewed-by: Richard Henderson Signed-off-by: Philippe Mathieu-Daudé Reviewed-by:

[PULL 1/8] multi-process: Initialize variables declared with g_auto*

2021-05-24 Thread Stefan Hajnoczi
From: Zenghui Yu Quote docs/devel/style.rst (section "Automatic memory deallocation"): * Variables declared with g_auto* MUST always be initialized, otherwise the cleanup function will use uninitialized stack memory Initialize @name properly to get rid of the compilation error (using

[PULL 0/8] Block patches

2021-05-24 Thread Stefan Hajnoczi
The following changes since commit 6c769690ac845fa62642a5f93b4e4bd906adab95: Merge remote-tracking branch 'remotes/vsementsov/tags/pull-simplebench-2021-05-04' into staging (2021-05-21 12:02:34 +0100) are available in the Git repository at: https://gitlab.com/stefanha/qemu.git

[PULL for-6.0 1/8] multi-process: Initialize variables declared with g_auto*

2021-05-24 Thread Stefan Hajnoczi
From: Zenghui Yu Quote docs/devel/style.rst (section "Automatic memory deallocation"): * Variables declared with g_auto* MUST always be initialized, otherwise the cleanup function will use uninitialized stack memory Initialize @name properly to get rid of the compilation error (using

[PULL for-6.0 0/8] Block patches

2021-05-24 Thread Stefan Hajnoczi
The following changes since commit 6c769690ac845fa62642a5f93b4e4bd906adab95: Merge remote-tracking branch 'remotes/vsementsov/tags/pull-simplebench-2021-05-04' into staging (2021-05-21 12:02:34 +0100) are available in the Git repository at: https://gitlab.com/stefanha/qemu.git

Re: [PATCH v4 03/11] block: use int64_t instead of uint64_t in driver read handlers

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
11.05.2021 22:22, Eric Blake wrote: On 3/24/21 3:51 PM, Vladimir Sementsov-Ogievskiy wrote: We are going to convert .bdrv_co_preadv_part and .bdrv_co_pwritev_part to int64_t type for offset and bytes parameters (as it's already done for generic block/io.c layer). In qcow2 .bdrv_co_preadv_part

[PATCH 2/2] block/vvfat: fix vvfat_child_perm crash

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
It's wrong to rely on s->qcow in vvfat_child_perm, as on permission update during bdrv_open_child() call this field is not set yet. Still prior to aa5a04c7db27eea6b36de32f241b155f0d9ce34d, it didn't crash, as bdrv_open_child passed NULL as child to bdrv_child_perm(), and NULL was equal to NULL in

[PATCH 1/2] block/vvfat: child_vvfat_qcow: add .get_parent_aio_context, fix crash

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
Commit 3ca1f3225727419ba573673b744edac10904276f "block: BdrvChildClass: add .get_parent_aio_context handler" introduced new handler and commit 228ca37e12f97788e05bd0c92f89b3e5e4019607 "block: drop ctx argument from bdrv_root_attach_child" made a generic use of it. But

[PATCH 0/2] vvfat: fix two crashes.

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
Hi! As reported by Programmingkid, command qemu-system-ppc -usb -device usb-storage,drive=fat16 -drive file=fat:rw:fat-type=16:"",id=fat16,format=raw,if=none crashes. I tested it with qemu-system-x86_64 and it reproduces for me. I even kept "" as is :). So, here are two fixes. Vladimir

Re: Patch "block: drop ctx argument from bdrv_root_attach_child" causing crash

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
24.05.2021 11:25, Vladimir Sementsov-Ogievskiy wrote: 24.05.2021 02:36, Programmingkid wrote: After doing some git bisecting I found out a problem I was having with sharing a host folder as an USB flash drive on the guest was caused by this patch: 228ca37e12f97788e05bd0c92f89b3e5e4019607 is

Re: Patch "block: drop ctx argument from bdrv_root_attach_child" causing crash

2021-05-24 Thread Vladimir Sementsov-Ogievskiy
24.05.2021 02:36, Programmingkid wrote: After doing some git bisecting I found out a problem I was having with sharing a host folder as an USB flash drive on the guest was caused by this patch: 228ca37e12f97788e05bd0c92f89b3e5e4019607 is the first bad commit commit