Re: [PATCH 1/1] block: improve alignment detection and fix 271 test

2023-10-14 Thread Nir Soffer
the test to reflect real world usage? _reset_img 2083k I guess it works with: _reset_img 2084k Commits > commit a6b257a08e3d72219f03e461a52152672fec0612 > Author: Nir Soffer > Date: Tue Aug 13 21:21:03 2019 +0300 > file-posix: Handle undetectable alignme

[PATCH] libvhost-user: Fix update of signalled_used

2023-05-09 Thread Nir Soffer
off-by: Nir Soffer --- subprojects/libvhost-user/libvhost-user.c | 23 +-- 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/subprojects/libvhost-user/libvhost-user.c b/subprojects/libvhost-user/libvhost-user.c index 8fb61e2df2..5f26d2d378 100644 --- a/subproje

Re: [Libguestfs] [PATCH v2 1/6] spec: Recommend cap on NBD_REPLY_TYPE_BLOCK_STATUS length

2023-03-06 Thread Nir Soffer
On Sun, Mar 5, 2023 at 10:42 AM Wouter Verhelst wrote: > > On Fri, Mar 03, 2023 at 04:17:40PM -0600, Eric Blake wrote: > > On Fri, Dec 16, 2022 at 10:32:01PM +0300, Vladimir Sementsov-Ogievskiy > > wrote: > > > s-o-b line missed. > > > > I'm not sure if the NBD project has a strict policy on

Re: [PATCH v2 2/5] Support format or cache specific out file

2022-12-13 Thread Nir Soffer
On Tue, Dec 13, 2022 at 8:09 PM Hanna Reitz wrote: > > On 13.12.22 16:56, Nir Soffer wrote: > > On Mon, Dec 12, 2022 at 12:38 PM Hanna Reitz wrote: > >> On 28.11.22 15:15, Nir Soffer wrote: > >>> Extend the test finder to find tests with format (*.o

Re: [PATCH v2 2/5] Support format or cache specific out file

2022-12-13 Thread Nir Soffer
On Mon, Dec 12, 2022 at 12:38 PM Hanna Reitz wrote: > > On 28.11.22 15:15, Nir Soffer wrote: > > Extend the test finder to find tests with format (*.out.qcow2) or cache > > specific (*.out.nocache) out file. This worked before only for the > > numbered tests. > >

[PATCH v2 3/5] qemu-img: Add checksum command

2022-11-28 Thread Nir Soffer
ng checksum for 8T empty image: qemu-img checksum: 3.7s, sha256sum (estimate): 17,749s Signed-off-by: Nir Soffer --- docs/tools/qemu-img.rst | 24 ++ meson.build | 10 ++- meson_options.txt | 2 + qemu-img-cmds.hx| 8 ++ qemu-img.c | 183 +

[PATCH v2 0/5] Add qemu-img checksum command using blkhash

2022-11-28 Thread Nir Soffer
tml - https://lists.nongnu.org/archive/html/qemu-block/2022-11/msg00171.html - https://lists.nongnu.org/archive/html/qemu-block/2022-11/msg00173.html Nir Soffer (5): qemu-img.c: Move IO_BUF_SIZE to the top of the file Support format or cache specific out file qemu-img: Add checksum command iote

[PATCH v2 5/5] qemu-img: Speed up checksum

2022-11-28 Thread Nir Soffer
5s ±0.183s | 1.808s ±0.016s | x1.36 | | nbd | 6g | direct| 3.540s ±0.020s | 1.749s ±0.018s | x2.02 | [1] raw image full of zeroes [2] raw fedora 35 image with additional random data, 50% full [3] image [2] exported by qemu-nbd via unix socket Signed-off-by: Nir Soffer --- qemu-i

[PATCH v2 4/5] iotests: Test qemu-img checksum

2022-11-28 Thread Nir Soffer
debugging in case of failures, the output includes a json map of the test image. Signed-off-by: Nir Soffer --- tests/qemu-iotests/tests/qemu-img-checksum| 63 +++ .../tests/qemu-img-checksum.out.qcow2 | 11 .../tests/qemu-img-checksum.out.raw | 10 +++ 3

[PATCH v2 2/5] Support format or cache specific out file

2022-11-28 Thread Nir Soffer
Extend the test finder to find tests with format (*.out.qcow2) or cache specific (*.out.nocache) out file. This worked before only for the numbered tests. --- tests/qemu-iotests/findtests.py | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git

[PATCH v2 1/5] qemu-img.c: Move IO_BUF_SIZE to the top of the file

2022-11-28 Thread Nir Soffer
This macro is used by various commands (compare, convert, rebase) but it is defined somewhere in the middle of the file. I'm going to use it in the new checksum command so lets clean up a bit before that. --- qemu-img.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git

Re: [PATCH 2/3] iotests: Test qemu-img checksum

2022-11-28 Thread Nir Soffer
On Mon, Nov 7, 2022 at 1:41 PM Hanna Reitz wrote: > On 30.10.22 18:38, Nir Soffer wrote: > > On Wed, Oct 26, 2022 at 4:31 PM Hanna Reitz wrote: > > > > On 01.09.22 16:32, Nir Soffer wrote: > > > Add simple tests creating an image with all kinds

Re: [PATCH 1/3] qemu-img: Add checksum command

2022-11-28 Thread Nir Soffer
On Mon, Nov 7, 2022 at 12:20 PM Hanna Reitz wrote: > On 30.10.22 18:37, Nir Soffer wrote: > > On Wed, Oct 26, 2022 at 4:00 PM Hanna Reitz wrote: > > > > On 01.09.22 16:32, Nir Soffer wrote: > [...] > > > --- > > > docs/tools/qemu-i

Re: [PATCH 3/3] qemu-img: Speed up checksum

2022-10-30 Thread Nir Soffer
On Sun, Oct 30, 2022 at 7:38 PM Nir Soffer wrote: > On Wed, Oct 26, 2022 at 4:54 PM Hanna Reitz wrote: > >> On 01.09.22 16:32, Nir Soffer wrote: >> > [...] > > +/* The current chunk. */ >> > +int64_t offset; >> > +int64_t length; >>

Re: [PATCH 1/3] qemu-img: Add checksum command

2022-10-30 Thread Nir Soffer
On Wed, Oct 26, 2022 at 4:00 PM Hanna Reitz wrote: > On 01.09.22 16:32, Nir Soffer wrote: > > The checksum command compute a checksum for disk image content using the > > blkhash library[1]. The blkhash library is not packaged yet, but it is > > available via copr[2]

Re: [PATCH 3/3] qemu-img: Speed up checksum

2022-10-30 Thread Nir Soffer
On Wed, Oct 26, 2022 at 4:54 PM Hanna Reitz wrote: > On 01.09.22 16:32, Nir Soffer wrote: > > Add coroutine based loop inspired by `qemu-img convert` design. > > > > Changes compared to `qemu-img convert`: > > > > - State for the entire image is kept in

Re: [PATCH 2/3] iotests: Test qemu-img checksum

2022-10-30 Thread Nir Soffer
On Wed, Oct 26, 2022 at 4:31 PM Hanna Reitz wrote: > On 01.09.22 16:32, Nir Soffer wrote: > > Add simple tests creating an image with all kinds of extents, different > > formats, different backing chain, different protocol, and different > > image options. Since all image

Re: [PATCH 0/3] Add qemu-img checksum command using blkhash

2022-10-18 Thread Nir Soffer
On Sun, Sep 18, 2022 at 12:35 PM Nir Soffer wrote: > ping > > Kevin, Hanna, I hope you have time to take a look. > > https://lists.nongnu.org/archive/html/qemu-block/2022-09/msg00021.html Ping again, hopefully someone has time to look at this :-) > > > > On Thu,

Re: [PATCH 0/3] Add qemu-img checksum command using blkhash

2022-09-18 Thread Nir Soffer
ping Kevin, Hanna, I hope you have time to take a look. https://lists.nongnu.org/archive/html/qemu-block/2022-09/msg00021.html On Thu, Sep 1, 2022 at 5:32 PM Nir Soffer wrote: > > Since blkhash is available only via copr now, the new command is added as > optional feature, b

[PATCH 1/3] qemu-img: Add checksum command

2022-09-01 Thread Nir Soffer
ng checksum for 8T empty image: qemu-img checksum: 3.7s, sha256sum (estimate): 17,749s Signed-off-by: Nir Soffer --- docs/tools/qemu-img.rst | 22 + meson.build | 10 ++- meson_options.txt | 2 + qemu-img-cmds.hx| 8 ++ qemu-img.c | 191 +

[PATCH 0/3] Add qemu-img checksum command using blkhash

2022-09-01 Thread Nir Soffer
Since blkhash is available only via copr now, the new command is added as optional feature, built only if blkhash-devel package is installed. Nir Soffer (3): qemu-img: Add checksum command iotests: Test qemu-img checksum qemu-img: Speed up checksum docs/tools/qemu-img.rst

[PATCH 3/3] qemu-img: Speed up checksum

2022-09-01 Thread Nir Soffer
5s ±0.183s | 1.808s ±0.016s | x1.36 | | nbd | 6g | direct| 3.540s ±0.020s | 1.749s ±0.018s | x2.02 | [1] raw image full of zeroes [2] raw fedora 35 image with additional random data, 50% full [3] image [2] exported by qemu-nbd via unix socket Signed-off-by: Nir Soffer --- qemu-i

[PATCH 2/3] iotests: Test qemu-img checksum

2022-09-01 Thread Nir Soffer
includes a json map of every test image. Signed-off-by: Nir Soffer --- tests/qemu-iotests/tests/qemu-img-checksum| 149 ++ .../qemu-iotests/tests/qemu-img-checksum.out | 74 + 2 files changed, 223 insertions(+) create mode 100755 tests/qemu-iotests/tests/qemu-img

Re: [PATCH 2/6] virtio-scsi: don't waste CPU polling the event virtqueue

2022-04-27 Thread Nir Soffer
cel=kvm -m 1G -cpu host \ > --object iothread,id=iothread0 \ > --device virtio-scsi-pci,iothread=iothread0 \ > --blockdev > file,filename=test.img,aio=native,cache.direct=on,node-name=drive0 \ > --device scsi-hd,drive=drive0 > > After this patch CP

Re: [PATCH v2] nbd/server: Allow MULTI_CONN for shared writable exports

2022-02-16 Thread Nir Soffer
On Wed, Feb 16, 2022 at 10:08 AM Vladimir Sementsov-Ogievskiy wrote: > > 16.02.2022 02:24, Eric Blake wrote: > > On Tue, Feb 15, 2022 at 09:23:36PM +0200, Nir Soffer wrote: > >> On Tue, Feb 15, 2022 at 7:22 PM Eric Blake wrote: > >> > >>> Accor

Re: [PATCH v2] nbd/server: Allow MULTI_CONN for shared writable exports

2022-02-16 Thread Nir Soffer
On Wed, Feb 16, 2022 at 12:13 PM Richard W.M. Jones wrote: > On Tue, Feb 15, 2022 at 05:24:14PM -0600, Eric Blake wrote: > > Oh. The QMP command (which is immediately visible through > > nbd-server-add/block-storage-add to qemu and qemu-storage-daemon) > > gains "multi-conn":"on", but you may be

Re: [PATCH v2] nbd/server: Allow MULTI_CONN for shared writable exports

2022-02-15 Thread Nir Soffer
On Tue, Feb 15, 2022 at 7:22 PM Eric Blake wrote: > According to the NBD spec, a server advertising > NBD_FLAG_CAN_MULTI_CONN promises that multiple client connections will > not see any cache inconsistencies: when properly separated by a single > flush, actions performed by one client will be

Re: [PATCH v2 2/2] iotests/block-status-cache: New test

2022-01-18 Thread Nir Soffer
644 > index 00..ae1213e6f8 > --- /dev/null > +++ b/tests/qemu-iotests/tests/block-status-cache.out > @@ -0,0 +1,5 @@ > +. > +-- > +Ran 1 tests > + > +OK > -- > 2.33.1 > The out file is not very useful, and even fragile - if the test framework will change the output format, the test will fail. Ideally we depend only on the relevant output of our tools, and using a different version of the test framework on replacing it (e.g pytest) will not require modifying the out files. Regardless I would like to see this fix merged and this issue already exists in other tests. Some tests in tests/ do have useful output that can make debugging failures easier. Reviewed-by: Nir Soffer

Re: [PATCH 1/2] block/io: Update BSC only if want_zero is true

2022-01-17 Thread Nir Soffer
data, and so we would update the block-status cache with wrong > information. > > Therefore, we should not update the cache with want_zero=false. > > Reported-by: Nir Soffer > Fixes: 0bc329fbb009f8601cec23bf2bc48ead0c5a5fa2 >("block: block-status cache for da

Re: [PATCH 2/2] iotests/block-status-cache: New test

2022-01-17 Thread Nir Soffer
self.fail("Map information differs") > + > + > +if __name__ == '__main__': > +# The block-status cache only works on the protocol layer, so to test it, > +# we can only use the raw format > +iotests.main(supported_fmts=['raw'], > + supported_protocols=['file']) > diff --git a/tests/qemu-iotests/tests/block-status-cache.out > b/tests/qemu-iotests/tests/block-status-cache.out > new file mode 100644 > index 00..ae1213e6f8 > --- /dev/null > +++ b/tests/qemu-iotests/tests/block-status-cache.out > @@ -0,0 +1,5 @@ > +. > +-- > +Ran 1 tests > + > +OK > -- > 2.33.1 > Reviewed-by: Nir Soffer

[PATCH] nbd/server.c: Remove unused field

2022-01-11 Thread Nir Soffer
NBDRequestData struct has unused QSIMPLEQ_ENTRY filed. It seems that this field exists since the first git commit and was never used. Signed-off-by: Nir Soffer --- nbd/server.c | 1 - 1 file changed, 1 deletion(-) diff --git a/nbd/server.c b/nbd/server.c index 3927f7789d..ce5b2a1d02 100644

Re: [PATCH 3/4] qemu-img: add --shallow option for qemu-img compare --stat

2021-09-29 Thread Nir Soffer
On Wed, Sep 29, 2021 at 7:28 PM Vladimir Sementsov-Ogievskiy wrote: > > 29.09.2021 19:00, Nir Soffer wrote: > > On Wed, Sep 29, 2021 at 4:37 PM Vladimir Sementsov-Ogievskiy > > wrote: > >> > >> Allow compare only top images of backing chains. That's useful fo

Re: [PATCH 3/4] qemu-img: add --shallow option for qemu-img compare --stat

2021-09-29 Thread Nir Soffer
On Wed, Sep 29, 2021 at 4:37 PM Vladimir Sementsov-Ogievskiy wrote: > > Allow compare only top images of backing chains. That's useful for > comparing two increments from the same chain of incremental backups. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > docs/tools/qemu-img.rst | 8

Re: [PATCH] qemu-nbd: Change default cache mode to writeback

2021-09-19 Thread Nir Soffer
On Mon, Aug 16, 2021 at 6:50 PM Eric Blake wrote: > > On Fri, Aug 13, 2021 at 11:55:19PM +0300, Nir Soffer wrote: > > Both qemu and qemu-img use writeback cache mode by default, which is > > already documented in qemu(1). qemu-nbd uses writethrough cache mode by > > defaul

[PATCH] qemu-nbd: Change default cache mode to writeback

2021-08-13 Thread Nir Soffer
common use case. [1] https://bugzilla.redhat.com/1990656 Signed-off-by: Nir Soffer --- docs/tools/qemu-nbd.rst | 6 -- qemu-nbd.c | 6 -- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/docs/tools/qemu-nbd.rst b/docs/tools/qemu-nbd.rst index ee862fa0bc..5643da26

Re: [PATCH for-6.1?] iotest: Further enhance qemu-img-bitmaps

2021-07-21 Thread Nir Soffer
On 7/21/21 11:46 PM, Eric Blake wrote: Add a regression test to make sure we detect attempts to use 'qemu-img bitmap' to modify an in-use local file. Suggested-by: Nir Soffer Signed-off-by: Eric Blake --- Sadly, this missed my bitmaps pull request today. If there's any reason to respin

Re: [PATCH v2 3/3] qemu-img: Add --skip-broken-bitmaps for 'convert --bitmaps'

2021-07-13 Thread Nir Soffer
On Tue, Jul 13, 2021 at 8:53 PM Eric Blake wrote: > > On Sat, Jul 10, 2021 at 09:37:35PM +0300, Nir Soffer wrote: > > > We don't want to delete inconsistent bitmaps by default: although a > > > corrupt bitmap is only a loss of optimization rather than a corruption &g

Re: [PATCH v2 2/3] qemu-img: Fail fast on convert --bitmaps with inconsistent bitmap

2021-07-13 Thread Nir Soffer
On Tue, Jul 13, 2021 at 8:48 PM Eric Blake wrote: > > On Sat, Jul 10, 2021 at 09:06:24PM +0300, Nir Soffer wrote: > > On 7/9/21 6:39 PM, Eric Blake wrote: > > > Waiting until the end of the convert operation (a potentially > > > time-consuming task) to finally dete

Re: [PATCH v2 3/3] qemu-img: Add --skip-broken-bitmaps for 'convert --bitmaps'

2021-07-10 Thread Nir Soffer
On 7/9/21 6:39 PM, Eric Blake wrote: The point of 'qemu-img convert --bitmaps' is to be a convenience for actions that are already possible through a string of smaller 'qemu-img bitmap' sub-commands. One situation not accounted for already is that if a source image contains an inconsistent

Re: [PATCH v2 2/3] qemu-img: Fail fast on convert --bitmaps with inconsistent bitmap

2021-07-10 Thread Nir Soffer
On 7/9/21 6:39 PM, Eric Blake wrote: Waiting until the end of the convert operation (a potentially time-consuming task) to finally detect that we can't copy a bitmap is bad, comparing to failing fast up front. Furthermore, this prevents us from leaving a file behind with a bitmap that is not

Re: [PATCH v2 1/3] iotests: Improve and rename test 291 to qemu-img-bitmap

2021-07-10 Thread Nir Soffer
from disk In this context a more useful error message would be: Try "qemu-img bitmap --remove" ... but this is not a new issue. +image: TEST_DIR/t.IMGFMT.copy +file format: IMGFMT +virtual size: 10 MiB (10485760 bytes) +cluster_size: 65536 +Format specific information: +bitmaps: +[0]: +flags: +name: b0 +granularity: 65536 +[1]: +flags: +[0]: auto +name: b4 +granularity: 65536 +corrupt: false *** done Reviewed-by: Nir Soffer

Re: [PATCH v3 3/2] qemu-img: Reword 'qemu-img map --output=json' docs

2021-07-07 Thread Nir Soffer
is optional; it is absent in >cases where ``human`` format would omit the entry or exit with an error. > -- > 2.31.1 Would be nice if this could be generated from the json schema instead of repeating the type and description of the fields, but this is a nice improvement. Reviewed-by: Nir Soffer

Re: [PATCH v2] docs: document file-posix locking protocol

2021-07-03 Thread Nir Soffer
On Sat, Jul 3, 2021 at 4:51 PM Vladimir Sementsov-Ogievskiy wrote: > > Let's document how we use file locks in file-posix driver, to allow > external programs to "communicate" in this way with Qemu. This makes the locking implementation public, so qemu can never change it without breaking

Re: [PATCH v2 2/1] qemu-img: Add "backing":true to unallocated map segments

2021-06-29 Thread Nir Soffer
On Tue, Jun 29, 2021 at 5:40 PM Kevin Wolf wrote: > > Am 29.06.2021 um 09:23 hat Vladimir Sementsov-Ogievskiy geschrieben: > > 28.06.2021 20:42, Eric Blake wrote: > > > On Wed, Jun 23, 2021 at 06:04:19PM +0200, Kevin Wolf wrote: > > > > > This is fine, but it means that this flag will present in

Re: [PATCH v2 2/1] qemu-img: Add "backing":true to unallocated map segments

2021-06-23 Thread Nir Soffer
On Wed, Jun 23, 2021 at 7:04 PM Kevin Wolf wrote: > > Am 23.06.2021 um 15:58 hat Nir Soffer geschrieben: > > On Wed, Jun 23, 2021 at 11:58 AM Kevin Wolf wrote: > > > > > > Am 22.06.2021 um 18:56 hat Nir Soffer geschrieben: > > > > On Tue,

Re: [PATCH v2 2/1] qemu-img: Add "backing":true to unallocated map segments

2021-06-23 Thread Nir Soffer
On Wed, Jun 23, 2021 at 11:58 AM Kevin Wolf wrote: > > Am 22.06.2021 um 18:56 hat Nir Soffer geschrieben: > > On Tue, Jun 22, 2021 at 6:38 PM Kevin Wolf wrote: > > > > > > Am 11.06.2021 um 21:03 hat Eric Blake geschrieben: > > > > To save the user fro

Re: [PATCH v2 2/1] qemu-img: Add "backing":true to unallocated map segments

2021-06-22 Thread Nir Soffer
On Fri, Jun 11, 2021 at 10:03 PM Eric Blake wrote: > > To save the user from having to check 'qemu-img info --backing-chain' > or other followup command to determine which "depth":n goes beyond the > chain, add a boolean field "backing" that is set only for unallocated > portions of the disk. > >

Re: [PATCH v2 2/1] qemu-img: Add "backing":true to unallocated map segments

2021-06-22 Thread Nir Soffer
On Tue, Jun 22, 2021 at 6:38 PM Kevin Wolf wrote: > > Am 11.06.2021 um 21:03 hat Eric Blake geschrieben: > > To save the user from having to check 'qemu-img info --backing-chain' > > or other followup command to determine which "depth":n goes beyond the > > chain, add a boolean field "backing"

Re: [PATCH v2 2/1] qemu-img: Add "backing":true to unallocated map segments

2021-06-15 Thread Nir Soffer
On Tue, Jun 15, 2021 at 11:54 AM Vladimir Sementsov-Ogievskiy wrote: > > 11.06.2021 22:03, Eric Blake wrote: > > To save the user from having to check 'qemu-img info --backing-chain' > > or other followup command to determine which "depth":n goes beyond the > > chain, add a boolean field

Re: [PATCH 2/2] nbd: Add new qemu:joint-allocation metadata context

2021-06-14 Thread Nir Soffer
On Mon, Jun 14, 2021 at 4:56 PM Eric Blake wrote: > > On Sat, Jun 12, 2021 at 02:39:44AM +0300, Nir Soffer wrote: > > Since this change is not simple, and the chance that we also get the dirty > > bitmap included in the result seems to be very low, I decided to check the > &

Re: [PATCH 2/2] nbd: Add new qemu:joint-allocation metadata context

2021-06-11 Thread Nir Soffer
On Wed, Jun 9, 2021 at 9:01 PM Eric Blake wrote: > > When trying to reconstruct a qcow2 chain using information provided > over NBD, ovirt had been relying on an unsafe assumption that any > portion of the qcow2 file advertised as sparse would defer to the > backing image; this worked with what

Re: [PATCH] qemu-{img,nbd}: Don't report zeroed cluster as a hole

2021-06-11 Thread Nir Soffer
On Fri, Jun 11, 2021 at 9:34 PM Eric Blake wrote: > > On Fri, Jun 11, 2021 at 08:35:01PM +0300, Nir Soffer wrote: > > On Fri, Jun 11, 2021 at 4:28 PM Eric Blake wrote: > > > > > > On Fri, Jun 11, 2021 at 10:09:09AM +0200, Kevin Wolf wrote: > > > > >

Re: [PATCH v2] qemu-img: Make unallocated part of backing chain obvious in map

2021-06-11 Thread Nir Soffer
On Fri, Jun 11, 2021 at 5:59 PM Eric Blake wrote: > > On Fri, Jun 11, 2021 at 05:35:12PM +0300, Vladimir Sementsov-Ogievskiy wrote: > > > An obvious solution is to make 'qemu-img map --output=json' > > > distinguish between clusters that have a local allocation from those > > > that are found

Re: [PATCH] qemu-{img,nbd}: Don't report zeroed cluster as a hole

2021-06-11 Thread Nir Soffer
On Fri, Jun 11, 2021 at 4:28 PM Eric Blake wrote: > > On Fri, Jun 11, 2021 at 10:09:09AM +0200, Kevin Wolf wrote: > > > Yes, that might work as well. But we didn't previously document > > > depth to be optional. Removing something from output risks breaking > > > more downstream tools that

Re: [PATCH] qemu-{img,nbd}: Don't report zeroed cluster as a hole

2021-06-11 Thread Nir Soffer
> ‫ב-11 ביוני 2021, בשעה 11:14, ‏‏Vladimir Sementsov-Ogievskiy > ‏ כתב/ה:‬ > > 11.06.2021 11:09, Kevin Wolf wrote: >> Am 10.06.2021 um 22:46 hat Eric Blake geschrieben: >>>> On Thu, Jun 10, 2021 at 11:09:05PM +0300, Nir Soffer wrote: >>>>>> But

Re: [PATCH] qemu-{img,nbd}: Don't report zeroed cluster as a hole

2021-06-10 Thread Nir Soffer
On Thu, Jun 10, 2021 at 9:35 PM Eric Blake wrote: > > On Tue, Jun 08, 2021 at 07:38:10PM +0300, Nir Soffer wrote: > > The example I provided was not detailed enough, what we actually do is: > > > > qemu-nbd .. 'json:{"driver": "qcow2", "backing

Re: [RFC libnbd PATCH] info: Add support for new qemu:joint-allocation

2021-06-10 Thread Nir Soffer
On Thu, Jun 10, 2021 at 4:06 PM Eric Blake wrote: > > On Thu, Jun 10, 2021 at 01:20:13AM +0300, Nir Soffer wrote: > > > + else if (strcmp (metacontext, "qemu:joint-allocation") == 0) { > > > +/* Combo of base:allocation and stripped-down qemu:allocation-dep

Re: [PATCH 2/2] nbd: Add new qemu:joint-allocation metadata context

2021-06-10 Thread Nir Soffer
On Thu, Jun 10, 2021 at 2:52 AM Nir Soffer wrote: > > On Wed, Jun 9, 2021 at 9:01 PM Eric Blake wrote: I posted a work in progress patch implementing support for qemu:joint-allocaition in oVirt: https://gerrit.ovirt.org/c/ovirt-imageio/+/115197 The most important part is the nbd client:

Re: [PATCH 2/2] nbd: Add new qemu:joint-allocation metadata context

2021-06-09 Thread Nir Soffer
be better to report: $ ./nbdinfo --map="qemu:joint-allocation" nbd://localhost 0 10737418243 hole,zero,unallocated This is the output for the empty qcow2 image. And this also affects --allocation-depth, for raw empty image we get: $ ./nbdinfo --map="qemu:all

Re: [RFC libnbd PATCH] info: Add support for new qemu:joint-allocation

2021-06-09 Thread Nir Soffer
On Thu, Jun 10, 2021 at 12:32 AM Eric Blake wrote: > > Qemu is adding qemu:joint-allocation as a single context combining the > two bits of base:allocation and a compression of qemu:allocation-depth > into two bits [1]. Decoding the bits makes it easier for humans to > see the result of that

Re: [PATCH] qemu-{img,nbd}: Don't report zeroed cluster as a hole

2021-06-08 Thread Nir Soffer
On Tue, Jun 8, 2021 at 9:46 PM Eric Blake wrote: > > On Tue, Jun 08, 2021 at 07:38:10PM +0300, Nir Soffer wrote: > > On Tue, Jun 8, 2021 at 12:22 AM Eric Blake wrote: > > > > > > On Mon, Jun 07, 2021 at 11:22:04PM +0300, Nir Soffer wrote: > > > > When

Re: [PATCH] qemu-{img,nbd}: Don't report zeroed cluster as a hole

2021-06-08 Thread Nir Soffer
On Tue, Jun 8, 2021 at 12:22 AM Eric Blake wrote: > > On Mon, Jun 07, 2021 at 11:22:04PM +0300, Nir Soffer wrote: > > When zeroing a cluster in an image with backing file, qemu-img and > > qemu-nbd reported the area as a hole. This does not affect the guest > > since

[PATCH] qemu-{img,nbd}: Don't report zeroed cluster as a hole

2021-06-07 Thread Nir Soffer
quot;: false, "offset": 196608}] $ build/qemu-nbd -r -t -f qcow2 top.qcow2 & $ qemu-img map --output json nbd://localhost [{ "start": 0, "length": 131072, "depth": 0, "zero": false, "data": true, "offset": 0},

Re: [PATCH] block/file-posix: Fix problem with fallocate(PUNCH_HOLE) on GPFS

2021-04-16 Thread Nir Soffer
On Fri, Apr 16, 2021 at 8:23 AM Thomas Huth wrote: > > A customer reported that running > > qemu-img convert -t none -O qcow2 -f qcow2 input.qcow2 output.qcow2 > > fails for them with the following error message when the images are > stored on a GPFS file system: > > qemu-img: error while

[PATCH] qemu-iotest: Test NBD hole reporting for qcow2

2021-04-13 Thread Nir Soffer
]. Add the missing tests for single qcow2 image and qcow2 image with a backing file. [1] https://listman.redhat.com/archives/libguestfs/2021-April/msg00050.html Signed-off-by: Nir Soffer --- tests/qemu-iotests/314 | 96 tests/qemu-iotests/314.out | 34

Re: [PATCH V4] file-posix: allow -EBUSY error during ioctl(fd, BLKZEROOUT, range) on block

2021-03-26 Thread Nir Soffer
On Fri, Mar 26, 2021 at 3:21 AM ChangLimin wrote: > >On Thu, Mar 25, 2021 at 8:07 AM ChangLimin > wrote: > >>On Wed, Mar 24, 2021 at 4:52 PM Max Reitz wrote: > >>On 22.03.21 10:25, ChangLimin wrote: > >>> For Linux 5.10/5.11, qemu write zeros to a multipath device using > >>> ioctl(fd,

Re: [PATCH V4] file-posix: allow -EBUSY error during ioctl(fd, BLKZEROOUT, range) on block

2021-03-25 Thread Nir Soffer
On Thu, Mar 25, 2021 at 8:07 AM ChangLimin wrote: > >On Wed, Mar 24, 2021 at 4:52 PM Max Reitz wrote: > >On 22.03.21 10:25, ChangLimin wrote: > >> For Linux 5.10/5.11, qemu write zeros to a multipath device using > >> ioctl(fd, BLKZEROOUT, range) with cache none or directsync return -EBUSY > >>

Re: [PATCH V4] file-posix: allow -EBUSY error during ioctl(fd, BLKZEROOUT, range) on block

2021-03-24 Thread Nir Soffer
On Wed, Mar 24, 2021 at 4:52 PM Max Reitz wrote: > On 22.03.21 10:25, ChangLimin wrote: > > For Linux 5.10/5.11, qemu write zeros to a multipath device using > > ioctl(fd, BLKZEROOUT, range) with cache none or directsync return -EBUSY > > permanently. > > So as far as I can track back the

Re: [PATCH] nbd: server: Report holes for raw images

2021-03-03 Thread Nir Soffer
On Thu, Feb 25, 2021 at 8:51 PM Vladimir Sementsov-Ogievskiy < vsement...@virtuozzo.com> wrote: > 19.02.2021 19:58, Eric Blake wrote: > > On 2/19/21 10:42 AM, Eric Blake wrote: > > > >>> To me, data=false looks compatible with NBD_STATE_HOLE. From user point > >>> of view, getting same results

Re: [PATCH] file-posix: allow -EBUSY errors during write zeros on block

2021-03-02 Thread Nir Soffer
On Tue, Mar 2, 2021 at 4:08 AM ChangLimin wrote: > > After Linux 5.10, write zeros to a multipath device using > ioctl(fd, BLKZEROOUT, range) with cache none or directsync will return EBUSY. > > Similar to handle_aiocb_write_zeroes_unmap, handle_aiocb_write_zeroes_block > allow -EBUSY errors

[PATCH] nbd: server: Report holes for raw images

2021-02-19 Thread Nir Soffer
ing same results from qemu-nbd and qemu-img is more important than being more correct about allocation status. Changing nbd server to report holes using BDRV_BLOCK_DATA makes qemu-nbd results compatible with qemu-img map: $ qemu-img map --output json nbd://localhost [{ "start&quo

Re: [PATCH v3 2/2] qemu-nbd: Permit --shared=0 for unlimited clients

2021-02-10 Thread Nir Soffer
rror_report("Invalid shared device number '%s'", optarg); > exit(EXIT_FAILURE); > } > @@ -966,7 +965,7 @@ int main(int argc, char **argv) > if (socket_activation == 0) { > int backlog; > > -if (persistent) { > +if (persistent || shared == 0) { > backlog = SOMAXCONN; > } else { > backlog = MIN(shared, SOMAXCONN); > -- > 2.30.0 > Reviewed-by: Nir Soffer

Re: [PATCH v3 1/2] qemu-nbd: Use SOMAXCONN for socket listen() backlog

2021-02-10 Thread Nir Soffer
On Tue, Feb 9, 2021 at 5:28 PM Eric Blake wrote: > > Our default of a backlog of 1 connection is rather puny; it gets in > the way when we are explicitly allowing multiple clients (such as > qemu-nbd -e N [--shared], or nbd-server-start with its default > "max-connections":0 for unlimited), but

Re: [PATCH v2] qemu-nbd: Use SOMAXCONN for socket listen() backlog

2021-02-05 Thread Nir Soffer
On Fri, Feb 5, 2021 at 8:57 PM Eric Blake wrote: > > Our default of a backlog of 1 connection is rather puny, particularly > for scenarios where we expect multiple listeners to connect (such as > qemu-nbd -e X). This is especially important for Unix sockets, as a > definite benefit to clients:

Re: Potential regression in 'qemu-img convert' to LVM

2021-01-07 Thread Nir Soffer
On Tue, Sep 15, 2020 at 2:51 PM Stefan Reiter wrote: > > On 9/15/20 11:08 AM, Nir Soffer wrote: > > On Mon, Sep 14, 2020 at 3:25 PM Stefan Reiter wrote: > >> > >> Hi list, > >> > >> following command fails since 5.1 (tested on kernel 5.4.60): &g

Re: [PULL 20/21] python/qemu/qmp.py: re-raise OSError when encountered

2020-10-20 Thread Nir Soffer
On Tue, Oct 20, 2020 at 8:52 PM John Snow wrote: > > Nested if conditions don't change when the exception block fires; we > need to explicitly re-raise the error if we didn't intend to capture and > suppress it. > > Signed-off-by: John Snow > Reviewed-by: Philippe Mathieu-Daudé > Message-id:

Re: [RFC PATCH 13/21] contrib/gitdm: Add more entries to the Red Hat domain

2020-10-04 Thread Nir Soffer
On Sun, Oct 4, 2020 at 9:05 PM Philippe Mathieu-Daudé wrote: > > Cc: Frediano Ziglio > Cc: Frediano Ziglio > Cc: Nir Soffer > Cc: Nir Soffer > Signed-off-by: Philippe Mathieu-Daudé > --- > To the developers Cc'ed: If you agree with your entry, please > reply with

Re: [PATCH] docs: Better mention of qemu-img amend limitations

2020-09-23 Thread Nir Soffer
On Wed, Sep 23, 2020 at 11:38 PM Eric Blake wrote: > > Missed during merge resolution of commit bc5ee6da71. > > Signed-off-by: Eric Blake > --- > docs/tools/qemu-img.rst | 4 > 1 file changed, 4 insertions(+) > > diff --git a/docs/tools/qemu-img.rst b/docs/tools/qemu-img.rst > index

Re: Potential regression in 'qemu-img convert' to LVM

2020-09-15 Thread Nir Soffer
On Mon, Sep 14, 2020 at 3:25 PM Stefan Reiter wrote: > > Hi list, > > following command fails since 5.1 (tested on kernel 5.4.60): > > # qemu-img convert -p -f raw -O raw /dev/zvol/pool/disk-1 /dev/vg/disk-1 > qemu-img: error while writing at byte 2157968896: Device or resource busy > > (source

Re: [PATCH 2/2] block: file-posix: Replace posix_fallocate with fallocate

2020-09-15 Thread Nir Soffer
On Mon, Sep 14, 2020 at 8:32 PM Daniel P. Berrangé wrote: > > On Mon, Aug 31, 2020 at 05:01:27PM +0300, Nir Soffer wrote: > > If fallocate() is not supported, posix_fallocate() falls back to > > inefficient allocation, writing one byte for every 4k bytes[1]. This is >

Re: [PATCH 0/2] Replace posix_fallocate() with falloate()

2020-09-14 Thread Nir Soffer
On Mon, Aug 31, 2020 at 5:01 PM Nir Soffer wrote: > > Change preallocation=falloc to use fallocate() instead of > posix_fallocte(), improving performance when legacy filesystem that do > not support fallocate, and avoiding issues seen with OFD locks. > > More work is needed to

Re: [PATCH 1/2] block: file-posix: Extract preallocate helpers

2020-09-01 Thread Nir Soffer
On Tue, Sep 1, 2020 at 1:27 PM Alberto Garcia wrote: > > On Mon 31 Aug 2020 04:01:26 PM CEST, Nir Soffer wrote: > > +static int preallocate_falloc(int fd, int64_t current_length, int64_t > > offset, > > + Error **errp) > > +{ >

[PATCH 2/2] block: file-posix: Replace posix_fallocate with fallocate

2020-08-31 Thread Nir Soffer
write 24620      0          24624 24567   [1] https://code.woboq.org/userspace/glibc/sysdeps/posix/posix_fallocate.c.html#96 [2] https://bugzilla.redhat.com/1850267#c25 [3] https://bugzilla.redhat.com/1851097 Signed-off-by: Nir Soffer --- block/file-posix.c | 32

[PATCH 1/2] block: file-posix: Extract preallocate helpers

2020-08-31 Thread Nir Soffer
modes (falloc, full, off) and leave only the common preparation and cleanup code in handle_aiocb_truncate(). Signed-off-by: Nir Soffer --- block/file-posix.c | 206 ++--- 1 file changed, 120 insertions(+), 86 deletions(-) diff --git a/block/file-posix.c

[PATCH 0/2] Replace posix_fallocate() with falloate()

2020-08-31 Thread Nir Soffer
. Continuing the discussion at: https://lists.nongnu.org/archive/html/qemu-block/2020-08/msg00947.html Nir Soffer (2): block: file-posix: Extract preallocate helpers block: file-posix: Replace posix_fallocate with fallocate block/file-posix.c | 202

[PATCH v3 5/5] qemu-iotests: Simplify FilePath __init__

2020-08-28 Thread Nir Soffer
Use list comprehension instead of append loop. Signed-off-by: Nir Soffer Reviewed-by: Max Reitz --- tests/qemu-iotests/iotests.py | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 7a20c9..cd0abf37e5

[PATCH v3 4/5] qemu-iotests: Merge FilePaths and FilePath

2020-08-28 Thread Nir Soffer
emporary files instead of 3. Signed-off-by: Nir Soffer Reviewed-by: Max Reitz --- tests/qemu-iotests/194| 2 +- tests/qemu-iotests/208| 2 +- tests/qemu-iotests/222| 2 +- tests/qemu-iotests/257| 4 ++-- tests/qemu-iotests/iotests.py | 23 ++

[PATCH v3 3/5] qemu-iotests: Support varargs syntax in FilePaths

2020-08-28 Thread Nir Soffer
clear and calling optional argument as positional arguments is bad idea anyway. Signed-off-by: Nir Soffer Reviewed-by: Max Reitz --- tests/qemu-iotests/194| 4 ++-- tests/qemu-iotests/257| 10 -- tests/qemu-iotests/iotests.py | 8 3 files changed, 10 inser

[PATCH v3 2/5] qemu-iotests: Fix FilePaths docstring

2020-08-28 Thread Nir Soffer
: de263986b5dc Signed-off-by: Nir Soffer --- tests/qemu-iotests/iotests.py | 19 +-- 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 2fd5c916fa..4abfe63662 100644 --- a/tests/qemu-iotests/iotests.py

[PATCH v3 1/5] qemu-iotests: Fix FilePaths cleanup

2020-08-28 Thread Nir Soffer
If os.remove() fails to remove one of the paths, for example if the file was removed by the test, the cleanup loop would exit silently, without removing the rest of the files. Fixes: de263986b5dc Signed-off-by: Nir Soffer Reviewed-by: Max Reitz --- tests/qemu-iotests/iotests.py | 8 1

[PATCH v3 0/5] iotest.FilePath fixes and cleanups

2020-08-28 Thread Nir Soffer
/msg00780.html Changes since v1: - Remove unwanted submodule change [Eric] - Add Fixes: tag v1 was here: https://lists.nongnu.org/archive/html/qemu-block/2020-08/msg00773.html Nir Soffer (5): qemu-iotests: Fix FilePaths cleanup qemu-iotests: Fix FilePaths docstring qemu-iotests: Support varargs

Re: [PATCH v2 2/5] qemu-iotests: Fix FilePaths docstring

2020-08-25 Thread Nir Soffer
On Tue, Aug 25, 2020 at 1:48 PM Max Reitz wrote: > > On 21.08.20 01:54, Nir Soffer wrote: > > When this class was extracted from FilePath, the docstring was not > > updated for generating multiple files, and the example usage was > > referencing unrelated file. >

Re: [PATCH v5 08/10] iotests.py: add verify_o_direct helper

2020-08-21 Thread Nir Soffer
On Fri, Aug 21, 2020 at 5:12 PM Vladimir Sementsov-Ogievskiy wrote: > > Add python notrun-helper similar to _check_o_direct for bash tests. > To be used in the following commit. > > Suggested-by: Nir Soffer > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > tests

[PATCH v2 3/5] qemu-iotests: Support varargs syntax in FilePaths

2020-08-20 Thread Nir Soffer
clear and calling optional argument as positional arguments is bad idea anyway. Signed-off-by: Nir Soffer --- tests/qemu-iotests/194| 4 ++-- tests/qemu-iotests/257| 10 -- tests/qemu-iotests/iotests.py | 6 +++--- 3 files changed, 9 insertions(+), 11 deletions(-) diff -

[PATCH v2 2/5] qemu-iotests: Fix FilePaths docstring

2020-08-20 Thread Nir Soffer
When this class was extracted from FilePath, the docstring was not updated for generating multiple files, and the example usage was referencing unrelated file. Fixes: de263986b5dc Signed-off-by: Nir Soffer --- tests/qemu-iotests/iotests.py | 14 -- 1 file changed, 8 insertions(+), 6

[PATCH v2 0/5] iotest.FilePath fixes and cleanups

2020-08-20 Thread Nir Soffer
/msg00773.html Nir Soffer (5): qemu-iotests: Fix FilePaths cleanup qemu-iotests: Fix FilePaths docstring qemu-iotests: Support varargs syntax in FilePaths qemu-iotests: Merge FilePaths and FilePath qemu-iotests: Simplify FilePath __init__ tests/qemu-iotests/194| 4 +-- tests/qemu

[PATCH v2 4/5] qemu-iotests: Merge FilePaths and FilePath

2020-08-20 Thread Nir Soffer
emporary files instead of 3. Signed-off-by: Nir Soffer --- tests/qemu-iotests/194| 2 +- tests/qemu-iotests/208| 2 +- tests/qemu-iotests/222| 2 +- tests/qemu-iotests/257| 4 ++-- tests/qemu-iotests/iotests.py | 23 +++ 5 files changed, 16 in

[PATCH v2 1/5] qemu-iotests: Fix FilePaths cleanup

2020-08-20 Thread Nir Soffer
If os.remove() fails to remove one of the paths, for example if the file was removed by the test, the cleanup loop would exit silently, without removing the rest of the files. Fixes: de263986b5dc Signed-off-by: Nir Soffer --- tests/qemu-iotests/iotests.py | 8 1 file changed, 4

[PATCH v2 5/5] qemu-iotests: Simplify FilePath __init__

2020-08-20 Thread Nir Soffer
Use list comprehension instead of append loop. Signed-off-by: Nir Soffer --- tests/qemu-iotests/iotests.py | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 1b5cdd493e..7ebd0bcc92 100644 --- a/tests/qemu

Re: [PATCH] qemu-iotests: Fix FilePaths cleanup

2020-08-20 Thread Nir Soffer
On Fri, Aug 21, 2020 at 12:40 AM Nir Soffer wrote: > > On Fri, Aug 21, 2020 at 12:33 AM Eric Blake wrote: > > > > On 8/20/20 4:29 PM, Eric Blake wrote: > > > On 8/20/20 4:19 PM, Nir Soffer wrote: > > >> If os.remove() fails to remove one of the paths, for

  1   2   3   4   >