Re: [PATCH] qemu-iotests: Fix FilePaths cleanup

2020-08-20 Thread Nir Soffer
On Fri, Aug 21, 2020 at 12:33 AM Eric Blake wrote: > > On 8/20/20 4:29 PM, Eric Blake wrote: > > On 8/20/20 4:19 PM, Nir Soffer wrote: > >> If os.remove() fails to remove one of the paths, for example if the file > >> was removed by the test, the cleanup loo

Re: [PATCH] qemu-iotests: Fix FilePaths cleanup

2020-08-20 Thread Nir Soffer
On Fri, Aug 21, 2020 at 12:19 AM Nir Soffer wrote: > > If os.remove() fails to remove one of the paths, for example if the file > was removed by the test, the cleanup loop would exit silently, without > removing the rest of the files. > > Signed-off-by: Nir So

[PATCH] qemu-iotests: Fix FilePaths cleanup

2020-08-20 Thread Nir Soffer
If os.remove() fails to remove one of the paths, for example if the file was removed by the test, the cleanup loop would exit silently, without removing the rest of the files. Signed-off-by: Nir Soffer --- dtc | 2 +- tests/qemu-iotests/iotests.py | 8 2 files

Re: [PATCH v4 08/10] iotests.py: add verify_o_direct helper

2020-08-20 Thread Nir Soffer
On Thu, Aug 20, 2020 at 9:49 PM Vladimir Sementsov-Ogievskiy wrote: > > Add python notrun-helper similar to _check_o_direct for bash tests. > To be used in the following commit. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > Reviewed-by: Stefan Hajnoczi > --- > tests/qemu-iotests/iotests.py

Re: [RFC PATCH 06/22] qemu-nbd: Use raw block driver for --offset

2020-08-18 Thread Nir Soffer
On Tue, Aug 18, 2020 at 11:47 AM Kevin Wolf wrote: > Am 17.08.2020 um 19:19 hat Nir Soffer geschrieben: > > On Thu, Aug 13, 2020 at 7:36 PM Kevin Wolf wrote: > > > > > Instead of implementing qemu-nbd --offset in the NBD code, just put a > > > raw block node

Re: [RFC PATCH 06/22] qemu-nbd: Use raw block driver for --offset

2020-08-17 Thread Nir Soffer
On Thu, Aug 13, 2020 at 7:36 PM Kevin Wolf wrote: > Instead of implementing qemu-nbd --offset in the NBD code, just put a > raw block node with the requested offset on top of the user image and > rely on that doing the job. > > This does not only simplify the nbd_export_new() interface and bring

Re: [PATCH] iotests/259: Fix reference output

2020-08-11 Thread Nir Soffer
On Tue, Aug 11, 2020 at 11:08 AM Max Reitz wrote: > The error message has changed recently, breaking the test. Fix it. > > Fixes: a2b333c01880f56056d50c238834d62e32001e54 >("block: nbd: Fix convert qcow2 compressed to nbd") > Thanks for fixing this. I thought that I ran all the tests

Re: [PATCH v2 2/4] iotests: Make qemu_nbd_popen() a contextmanager

2020-08-05 Thread Nir Soffer
On Wed, Aug 5, 2020 at 10:38 AM Vladimir Sementsov-Ogievskiy wrote: > > 28.07.2020 19:05, Nir Soffer wrote: > > On Tue, Jul 28, 2020 at 4:43 PM Vladimir Sementsov-Ogievskiy > > wrote: > >> > >> 28.07.2020 00:58, Nir Soffer wrote: > >>> Instead

Re: [PATCH] schemas: Add vim modeline

2020-08-03 Thread Nir Soffer
On Mon, Aug 3, 2020 at 3:23 PM Daniel P. Berrangé wrote: > > On Mon, Aug 03, 2020 at 02:16:19PM +0200, Paolo Bonzini wrote: > > On 03/08/20 13:36, Daniel P. Berrangé wrote: > > >>> Given that QEMU needs to pass > > >>> uint64 values, JSON was simply the wrong choice of format for QMP. > > > > > >

Re: cleanups with long-term benefits (was Re: [PATCH] schemas: Add vim modeline)

2020-08-03 Thread Nir Soffer
On Mon, Aug 3, 2020 at 9:19 PM John Snow wrote: > > On 8/3/20 2:16 PM, Paolo Bonzini wrote: > > On 03/08/20 20:10, John Snow wrote: > >> Heresy: > >> > >> Docstrings could become part of the data format so they can be parsed, > >> analyzed and validated. Parsers largely treat comments like

Re: [PATCH] schemas: Add vim modeline

2020-07-31 Thread Nir Soffer
On Thu, Jul 30, 2020 at 12:38 PM Daniel P. Berrangé wrote: > > On Thu, Jul 30, 2020 at 11:07:26AM +0200, Markus Armbruster wrote: > > Andrea Bolognani writes: > > > > > The various schemas included in QEMU use a JSON-based format which > > > is, however, strictly speaking not valid JSON. > > > >

Re: [PATCH v2 3/4] iotests: Add more qemu_img helpers

2020-07-28 Thread Nir Soffer
On Tue, Jul 28, 2020 at 4:50 PM Vladimir Sementsov-Ogievskiy wrote: > > 28.07.2020 00:58, Nir Soffer wrote: > > Add 2 helpers for measuring and checking images: > > - qemu_img_measure() > > - qemu_img_check() > > > > Both use --output-json and parse t

Re: [PATCH v2 2/4] iotests: Make qemu_nbd_popen() a contextmanager

2020-07-28 Thread Nir Soffer
On Tue, Jul 28, 2020 at 4:43 PM Vladimir Sementsov-Ogievskiy wrote: > > 28.07.2020 00:58, Nir Soffer wrote: > > Instead of duplicating the code to wait until the server is ready and > > remember to terminate the server and wait for it, make it possib

[PATCH v2 2/4] iotests: Make qemu_nbd_popen() a contextmanager

2020-07-27 Thread Nir Soffer
the output since it did not consistently when starting and stopping qemu-nbd. Signed-off-by: Nir Soffer --- tests/qemu-iotests/264| 76 +-- tests/qemu-iotests/264.out| 2 + tests/qemu-iotests/iotests.py | 28 - 3 files changed, 56

[PATCH v2 3/4] iotests: Add more qemu_img helpers

2020-07-27 Thread Nir Soffer
Add 2 helpers for measuring and checking images: - qemu_img_measure() - qemu_img_check() Both use --output-json and parse the returned json to make easy to use in other tests. I'm going to use them in a new test, and I hope they will be useful in may other tests. Signed-off-by: Nir Soffer

[PATCH v2 0/4] Fix convert to qcow2 compressed to NBD

2020-07-27 Thread Nir Soffer
] - Use qcow2 for source file to avoid issues with random CI filesystem [Max] - Fix many typos [Eric, Max] - Make qemu_nbd_popen a context manager - Add more qemu_img_* helpers - Verify OVA file contents v1 was here: https://lists.nongnu.org/archive/html/qemu-block/2020-07/msg01543.html Nir Soffer

[PATCH v2 4/4] iotests: Test convert to qcow2 compressed to NBD

2020-07-27 Thread Nir Soffer
Add test for "qemu-img convert -O qcow2 -c" to NBD target. The tests     create a OVA file and write compressed qcow2 disk content directly into the OVA file via qemu-nbd. Signed-off-by: Nir Soffer --- tests/qemu-iotests/302 | 127 + tests/qe

[PATCH v2 1/4] block: nbd: Fix convert qcow2 compressed to nbd

2020-07-27 Thread Nir Soffer
01% allocated, 100.00% fragmented, 100.00% compressed clusters Image end offset: 393216 $ qemu-img compare disk.raw nbd+unix:///?socket=/tmp/nbd.sock Images are identical. Fixes: https://bugzilla.redhat.com/1860627 Signed-off-by: Nir Soffer --- block/nbd.c | 30 ++ 1 fil

Re: [PATCH 1/2] block: nbd: Fix convert qcow2 compressed to nbd

2020-07-27 Thread Nir Soffer
On Mon, Jul 27, 2020 at 5:04 PM Eric Blake wrote: > > On 7/26/20 10:25 AM, Nir Soffer wrote: > > When converting to qcow2 compressed format, the last step is a special > > zero length compressed write, ending in call to bdrv_co_truncate(). This > > call always fail

Re: [PATCH 1/2] block: nbd: Fix convert qcow2 compressed to nbd

2020-07-27 Thread Nir Soffer
On Mon, Jul 27, 2020 at 5:04 PM Eric Blake wrote: > > On 7/26/20 10:25 AM, Nir Soffer wrote: > > When converting to qcow2 compressed format, the last step is a special > > zero length compressed write, ending in call to bdrv_co_truncate(). This > > call always fail

Re: [PATCH 2/2] qemu-iotests: Test convert to qcow2 compressed to NBD

2020-07-27 Thread Nir Soffer
On Mon, Jul 27, 2020 at 1:05 PM Max Reitz wrote: > > On 26.07.20 17:25, Nir Soffer wrote: > > Add test for "qemu-img convert -O qcow2 -c" to NBD target. The use case > > is writing compressed disk content to OVA archive. > > > > Signed-off-by: Nir Sof

Re: [PATCH 2/2] qemu-iotests: Test convert to qcow2 compressed to NBD

2020-07-27 Thread Nir Soffer
On Mon, Jul 27, 2020 at 5:41 PM Eric Blake wrote: > > On 7/27/20 9:35 AM, Nir Soffer wrote: > > >> I guess it's okay that you don't create a real tar file here, but > >> listing the commands to create it (even as a comment) is better than > >> just saying &

Re: [PATCH 2/2] qemu-iotests: Test convert to qcow2 compressed to NBD

2020-07-27 Thread Nir Soffer
On Mon, Jul 27, 2020 at 5:14 PM Eric Blake wrote: > > On 7/27/20 5:04 AM, Max Reitz wrote: > > On 26.07.20 17:25, Nir Soffer wrote: > >> Add test for "qemu-img convert -O qcow2 -c" to NBD target. The use case > >> is writing compressed disk content to O

[PATCH 1/2] block: nbd: Fix convert qcow2 compressed to nbd

2020-07-26 Thread Nir Soffer
01% allocated, 100.00% fragmented, 100.00% compressed clusters Image end offset: 393216 $ qemu-img compare disk.raw nbd+unix:///?socket=/tmp/nbd.sock Images are identical. Fixes: https://bugzilla.redhat.com/1860627 Signed-off-by: Nir Soffer --- block/nbd.c | 27 +++ 1 fil

[PATCH 2/2] qemu-iotests: Test convert to qcow2 compressed to NBD

2020-07-26 Thread Nir Soffer
Add test for "qemu-img convert -O qcow2 -c" to NBD target. The use case is writing compressed disk content to OVA archive. Signed-off-by: Nir Soffer --- tests/qemu-iotests/302 | 83 ++ tests/qemu-iotests/302.out | 27 + tests/qe

[PATCH 0/2] Fix convert to qcow2 compressed to NBD

2020-07-26 Thread Nir Soffer
Fix qemu-img convert -O qcow2 -c to NBD URL and add missing test for this usage. This already works now, but unfortunately qemu-img fails when trying to truncate the target image to the same size at the end of the operation. Nir Soffer (2): block: nbd: Fix convert qcow2 compressed to nbd

Re: [PATCH for-5.1 1/2] qcow2: Implement v2 zero writes with discard if possible

2020-07-20 Thread Nir Soffer
image was empty. > > Reported-by: Nir Soffer > Signed-off-by: Kevin Wolf > --- > block/qcow2-cluster.c | 9 - > 1 file changed, 8 insertions(+), 1 deletion(-) > > diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c > index 4b5fc8c4a7..a677ba9f5c

Re: [PATCH for-5.1 2/2] iotests: Test sparseness for qemu-img convert -n

2020-07-20 Thread Nir Soffer
On Mon, Jul 20, 2020 at 4:18 PM Kevin Wolf wrote: > > Signed-off-by: Kevin Wolf > --- > tests/qemu-iotests/122 | 34 ++ > tests/qemu-iotests/122.out | 17 + > 2 files changed, 51 insertions(+) > > diff --git a/tests/qemu-iotests/122

Re: [PATCH for-5.1 1/2] block: Require aligned image size to avoid assertion failure

2020-07-15 Thread Nir Soffer
On Wed, Jul 15, 2020 at 4:42 PM Kevin Wolf wrote: > > Am 15.07.2020 um 15:22 hat Nir Soffer geschrieben: > > On Mon, Jul 13, 2020 at 7:56 PM Kevin Wolf wrote: > > > > > > Am 13.07.2020 um 18:33 hat Nir Soffer geschrieben: > > > > On Fri,

Re: [PATCH for-5.1 1/2] block: Require aligned image size to avoid assertion failure

2020-07-15 Thread Nir Soffer
On Mon, Jul 13, 2020 at 7:56 PM Kevin Wolf wrote: > > Am 13.07.2020 um 18:33 hat Nir Soffer geschrieben: > > On Fri, Jul 10, 2020 at 5:22 PM Kevin Wolf wrote: > > > > > > Unaligned requests will automatically be aligned to bl.request_alignment > > > and we

Re: [PATCH for-5.1 1/2] block: Require aligned image size to avoid assertion failure

2020-07-13 Thread Nir Soffer
On Fri, Jul 10, 2020 at 5:22 PM Kevin Wolf wrote: > > Unaligned requests will automatically be aligned to bl.request_alignment > and we don't want to extend requests to access space beyond the end of > the image, so it's required that the image size is aligned. > > With write requests, this could

Re: [PATCH for-5.1 2/2] file-posix: Allow byte-aligned O_DIRECT with NFS

2020-07-13 Thread Nir Soffer
On Fri, Jul 10, 2020 at 5:22 PM Kevin Wolf wrote: > > Since commit a6b257a08e3 ('file-posix: Handle undetectable alignment'), > we assume that if we open a file with O_DIRECT and alignment probing > returns 1, we just couldn't find out the real alignment requirement > because some filesystems

Re: [PATCH 0/2] qcow2: Force preallocation with data-file-raw

2020-06-22 Thread Nir Soffer
On Mon, Jun 22, 2020 at 12:47 PM Max Reitz wrote: > > On 22.06.20 00:25, Nir Soffer wrote: > > On Fri, Jun 19, 2020 at 1:40 PM Max Reitz wrote: > >> > >> Hi, > >> > >> As discussed here: > >> > >> https://lists.n

Re: [PATCH] qemu-img convert: Don't pre-zero images

2020-06-22 Thread Nir Soffer
e. > > Remove the pre-zeroing because these days this former optimisation has > actually turned into a pessimisation in the common case. > > Reported-by: Nir Soffer > Signed-off-by: Kevin Wolf Thanks, you can also add: Tested-by: Nir Soffer > --- > qemu-img.c | 9

Re: [PATCH 1/2] qcow2: Force preallocation with data-file-raw

2020-06-22 Thread Nir Soffer
On Mon, Jun 22, 2020 at 6:07 PM Max Reitz wrote: > > On 22.06.20 16:46, Alberto Garcia wrote: > > On Mon 22 Jun 2020 11:35:59 AM CEST, Max Reitz wrote: > +if (qcow2_opts->data_file_raw && > +qcow2_opts->preallocation == PREALLOC_MODE_OFF) > +{ > +/* >

Re: [PATCH 0/2] qcow2: Force preallocation with data-file-raw

2020-06-21 Thread Nir Soffer
On Fri, Jun 19, 2020 at 1:40 PM Max Reitz wrote: > > Hi, > > As discussed here: > > https://lists.nongnu.org/archive/html/qemu-block/2020-02/msg00644.html > https://lists.nongnu.org/archive/html/qemu-block/2020-04/msg00329.html >

Re: [PATCH] block: file-posix: Fail unmap with NO_FALLBACK on block device

2020-06-16 Thread Nir Soffer
On Tue, Jun 16, 2020 at 8:39 PM Nir Soffer wrote: > > On Tue, Jun 16, 2020 at 6:32 PM Kevin Wolf wrote: > > > > Am 15.06.2020 um 21:32 hat Nir Soffer geschrieben: > > > We can zero 2.3 g/s: > > > > > > # time blkdiscard -z test-lv > > > >

Re: [PATCH] block: file-posix: Fail unmap with NO_FALLBACK on block device

2020-06-16 Thread Nir Soffer
On Tue, Jun 16, 2020 at 6:32 PM Kevin Wolf wrote: > > Am 15.06.2020 um 21:32 hat Nir Soffer geschrieben: > > We can zero 2.3 g/s: > > > > # time blkdiscard -z test-lv > > > > real 0m43.902s > > user 0m0.002s > > sys 0m0.130s > > > We can

Re: [PATCH] block: file-posix: Fail unmap with NO_FALLBACK on block device

2020-06-15 Thread Nir Soffer
On Sat, Jun 13, 2020 at 8:08 PM Nir Soffer wrote: > > Punching holes on block device uses blkdev_issue_zeroout() with > BLKDEV_ZERO_NOFALLBACK but there is no guarantee that this is fast > enough for pre-zeroing an entire device. > > Zeroing block device can be slow as writing z

[PATCH] block: file-posix: Fail unmap with NO_FALLBACK on block device

2020-06-13 Thread Nir Soffer
format: raw virtual size: 10 GiB (10737418240 bytes) disk size: 8 GiB $ time ./qemu-img convert -f raw -O raw -t none -T none -W test.img /dev/test/lv1 Before: real1m20.483s user0m0.490s sys 0m0.739s After: real0m55.831s user0m0.610s sys 0m0.956s Signed-off-by: Nir Soffer

Re: [PATCH v5 2/5] qcow2: Expose bitmaps' size during measure

2020-05-21 Thread Nir Soffer
On Thu, May 21, 2020 at 6:09 PM Vladimir Sementsov-Ogievskiy wrote: > > 21.05.2020 16:29, Nir Soffer wrote: > > On Thu, May 21, 2020 at 2:40 PM Vladimir Sementsov-Ogievskiy > > wrote: > >> > >> 21.05.2020 02:00, Nir Soffer wrote: > >>> O

Re: [PATCH v5 4/5] qemu-img: Add convert --bitmaps option

2020-05-21 Thread Nir Soffer
On Thu, May 21, 2020 at 1:01 AM Eric Blake wrote: > > Make it easier to copy all the persistent bitmaps of (the top layer > of) a source image along with its guest-visible contents, by adding a > boolean flag for use with qemu-img convert. This is basically > shorthand, as the same effect could

Re: [PATCH v5 2/5] qcow2: Expose bitmaps' size during measure

2020-05-21 Thread Nir Soffer
On Thu, May 21, 2020 at 2:40 PM Vladimir Sementsov-Ogievskiy wrote: > > 21.05.2020 02:00, Nir Soffer wrote: > > On Thu, May 21, 2020 at 1:01 AM Eric Blake wrote: > >> > >> It's useful to know how much space can be occupied by qcow2 persistent > >> bitmaps,

Re: [PATCH v5 2/5] qcow2: Expose bitmaps' size during measure

2020-05-21 Thread Nir Soffer
On Thu, May 21, 2020 at 4:08 PM Eric Blake wrote: > > On 5/20/20 6:00 PM, Nir Soffer wrote: > > >> > >> On the command-line side, 'qemu-img measure' gains a new --bitmaps > >> flag. When present, the bitmap size is rolled into the two existing > >> m

Re: [PATCH v5 2/5] qcow2: Expose bitmaps' size during measure

2020-05-20 Thread Nir Soffer
field demonstrates why we should always > zero-initialize qapi C structs; while the qcow2 driver still fully > populates all fields, the raw and crypto drivers had to be tweaked to > avoid uninitialized data. > > See also: https://bugzilla.redhat.com/1779904 > > Reporte

Re: [PATCH v3 7/9] qcow2: Expose bitmaps' size during measure

2020-05-12 Thread Nir Soffer
On Tue, May 12, 2020 at 10:39 PM Eric Blake wrote: > > On 5/12/20 6:10 AM, Max Reitz wrote: > > > >> This does not break old code since previously we always reported only > >> guest visible content > >> here, but it changes the semantics, and now you cannot allocate > >> "required" size, you need

Re: [PATCH v3 7/9] qcow2: Expose bitmaps' size during measure

2020-05-12 Thread Nir Soffer
lways > zero-initialize qapi C structs; while the qcow2 driver still fully > populates all fields, the raw and crypto drivers had to be tweaked to > avoid uninitialized data. > > See also: https://bugzilla.redhat.com/1779904 > > Reported-by: Nir Soffer > Signed-off-by: Eric

Re: [PATCH 0/3] qemu-img: Add convert --bitmaps

2020-04-16 Thread Nir Soffer
On Thu, Apr 16, 2020 at 5:51 PM Eric Blake wrote: > > Without this series, the process for copying one qcow2 image to > another including all of its bitmaps involves running qemu and doing > the copying by hand with a series of QMP commands. This makes the > process a bit more convenient. This

Re: [PATCH 1/3] qcow2: Require that the virtual size is a multiple of the sector size

2020-01-08 Thread Nir Soffer
On Wed, Jan 8, 2020 at 7:52 PM Alberto Garcia wrote: > > The qcow2 header specifies the virtual size of the image in bytes, but > BlockDriverState stores it as a number of 512-byte sectors. > > If the user tries to create an image with a size that is not a > multiple of the sector size then this

Re: [PATCH 3/3] qcow2: Use BDRV_SECTOR_SIZE instead of the hardcoded value

2020-01-08 Thread Nir Soffer
On Wed, Jan 8, 2020 at 7:50 PM Alberto Garcia wrote: > > This replaces all remaining instances in the qcow2 code. > > Signed-off-by: Alberto Garcia > --- > block/qcow2-cluster.c | 2 +- > block/qcow2.c | 16 +--- > 2 files changed, 10 insertions(+), 8 deletions(-) > > diff

Re: [PATCH] block: nbd: Fix dirty bitmap context name

2019-12-25 Thread Nir Soffer
On Thu, Dec 19, 2019 at 5:55 PM Nir Soffer wrote: > > On Thu, Dec 19, 2019 at 5:17 PM Vladimir Sementsov-Ogievskiy > wrote: ... > > Note also, that client may use NBD_OPT_LIST_META_CONTEXT with query > > "qemu:dirty-bitmap:", to get list of all exported bitmaps. &g

Re: [PATCH] qapi/block: fix nbd-server-add spec

2019-12-19 Thread Nir Soffer
On Thu, Dec 19, 2019 at 5:25 PM Vladimir Sementsov-Ogievskiy wrote: > > 19.12.2019 18:08, Nir Soffer wrote: > > On Thu, Dec 19, 2019 at 5:00 PM Vladimir Sementsov-Ogievskiy > > wrote: > >> > >> 19.12.2019 17:42, Nir Soffer wrote: > >>> On Thu, Dec

Re: [PATCH] block: nbd: Fix dirty bitmap context name

2019-12-19 Thread Nir Soffer
On Thu, Dec 19, 2019 at 5:17 PM Vladimir Sementsov-Ogievskiy wrote: > > 19.12.2019 17:59, Nir Soffer wrote: > > On Thu, Dec 19, 2019 at 4:04 PM Kevin Wolf wrote: > >> > >> Am 19.12.2019 um 14:41 hat Vladimir Sementsov-Ogievskiy geschrieben: >

Re: [PATCH] qapi/block: fix nbd-server-add spec

2019-12-19 Thread Nir Soffer
On Thu, Dec 19, 2019 at 5:00 PM Vladimir Sementsov-Ogievskiy wrote: > > 19.12.2019 17:42, Nir Soffer wrote: > > On Thu, Dec 19, 2019 at 4:34 PM Vladimir Sementsov-Ogievskiy > > wrote: > >> > >> "NAME" here may be interpreted like it should match @nam

Re: [PATCH] block: nbd: Fix dirty bitmap context name

2019-12-19 Thread Nir Soffer
On Thu, Dec 19, 2019 at 4:04 PM Kevin Wolf wrote: > > Am 19.12.2019 um 14:41 hat Vladimir Sementsov-Ogievskiy geschrieben: > > Ahh, I see, it's documented as > > > > +# @bitmap: Also export the dirty bitmap reachable from @device, so the > > +# NBD client can use NBD_OPT_SET_META_CONTEXT

Re: [PATCH] block: nbd: Fix dirty bitmap context name

2019-12-19 Thread Nir Soffer
On Thu, Dec 19, 2019 at 3:42 PM Vladimir Sementsov-Ogievskiy wrote: > > I'd not call it a "fix".. As it implies something broken. > > [edit: OK, now I see that something is broken, and why you called it "fix", > see below] > > 19.12.2019 15:51, Ni

Re: [PATCH] qapi/block: fix nbd-server-add spec

2019-12-19 Thread Nir Soffer
On Thu, Dec 19, 2019 at 4:34 PM Vladimir Sementsov-Ogievskiy wrote: > > "NAME" here may be interpreted like it should match @name, which is > export name. But it was never mentioned in such way. Make it obvious, > that actual "" (see docs/interop/nbd.txt) > will match @bitmap parameter. But this

[PATCH] block: nbd: Fix dirty bitmap context name

2019-12-19 Thread Nir Soffer
When adding an export with a dirty bitmap, expose the bitmap at: qemu:dirty-bitmap:export-name This matches qapi documentation, and user expectations. Without this, qemu leaks libvirt implementations details to clients by exposing the bitmap using the actual bitmap name:

Re: [RFC 0/3] block/file-posix: Work around XFS bug

2019-10-26 Thread Nir Soffer
On Fri, Oct 25, 2019 at 1:11 PM Max Reitz wrote: > > Hi, > > It seems to me that there is a bug in Linux’s XFS kernel driver, as > I’ve explained here: > > https://lists.nongnu.org/archive/html/qemu-block/2019-10/msg01429.html > > In combination with our commit c8bb23cbdbe32f, this may lead to

Re: [RFC 3/3] block/file-posix: Let post-EOF fallocate serialize

2019-10-26 Thread Nir Soffer
On Fri, Oct 25, 2019 at 1:24 PM Max Reitz wrote: > > The XFS kernel driver has a bug that may cause data corruption for qcow2 > images as of qemu commit c8bb23cbdbe32f. We can work around it by > treating post-EOF fallocates as serializing up until infinity (INT64_MAX > in practice). > >

Re: [RFC 2/3] block/file-posix: Detect XFS with CONFIG_FALLOCATE

2019-10-26 Thread Nir Soffer
On Fri, Oct 25, 2019 at 1:22 PM Max Reitz wrote: > > We will need this for the next patch. > > Signed-off-by: Max Reitz > --- > block/file-posix.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/block/file-posix.c b/block/file-posix.c > index 695fcf740d..5cd54c8bff

Re: [PATCH v2 1/4] qemu-iotests: remove bash shebang from library files

2019-10-11 Thread Nir Soffer
On Fri, Oct 11, 2019, 12:36 Kevin Wolf wrote: > Am 09.10.2019 um 21:47 hat Cleber Rosa geschrieben: > > Due to not being able to find a reason to have shebangs on files that > > are not executable. > > > > While at it, add a mode hint to emacs, which would be clueless or > > plain wrong about

Re: [PULL 4/4] tests: Use iothreads during iotest 223

2019-09-24 Thread Nir Soffer
On Tue, Sep 24, 2019 at 4:18 PM Eric Blake wrote: > Doing so catches the bugs we just fixed with NBD not properly using > correct contexts. > > Signed-off-by: Eric Blake > Message-Id: <20190920220729.31801-1-ebl...@redhat.com> > --- > tests/qemu-iotests/223 | 6 -- >

Re: [Qemu-devel] [Qemu-block] [PATCH v2 0/2] Alignment checks cleanup

2019-09-05 Thread Nir Soffer
Max, can you review again? On Fri, Aug 30, 2019 at 11:25 PM Nir Soffer wrote: > On Wed, Aug 28, 2019 at 11:14 PM John Snow wrote: > >> >> >> On 8/27/19 2:59 PM, Nir Soffer wrote: >> > While working on 4k support, I noticed that there is lot of code

Re: [Qemu-devel] [Qemu-block] [PATCH v2 0/2] Alignment checks cleanup

2019-08-30 Thread Nir Soffer
On Wed, Aug 28, 2019 at 11:14 PM John Snow wrote: > > > On 8/27/19 2:59 PM, Nir Soffer wrote: > > While working on 4k support, I noticed that there is lot of code using > > BDRV_SECTOR_SIZE (512) for checking alignment. I wonder how this can > work with > >

[Qemu-devel] [PATCH v2 0/2] Alignment checks cleanup

2019-08-27 Thread Nir Soffer
based on BDRV_SECTOR_SIZE Nir Soffer (2): block: Use QEMU_IS_ALIGNED block: Remove unused masks block/bochs.c | 4 ++-- block/cloop.c | 4 ++-- block/dmg.c | 4 ++-- block/io.c| 8 block/qcow2-cluster.c | 4 ++-- block/qcow2.c | 4

[Qemu-devel] [PATCH v2 1/2] block: Use QEMU_IS_ALIGNED

2019-08-27 Thread Nir Soffer
Replace instances of: (n & (BDRV_SECTOR_SIZE - 1)) == 0 And: (n & ~BDRV_SECTOR_MASK) == 0 With: QEMU_IS_ALIGNED(n, BDRV_SECTOR_SIZE) Which reveals the intent of the code better, and makes it easier to locate the code checking alignment. Signed-off-by: Nir Soffer --

[Qemu-devel] [PATCH v2 2/2] block: Remove unused masks

2019-08-27 Thread Nir Soffer
Replace confusing usage: ~BDRV_SECTOR_MASK With more clear: (BDRV_SECTOR_SIZE - 1) Remove BDRV_SECTOR_MASK and the unused BDRV_BLOCK_OFFSET_MASK which was it's last user. Signed-off-by: Nir Soffer --- include/block/block.h | 2 -- migration/block.c | 2 +- 2 files changed, 1

Re: [Qemu-devel] [PATCH] iotests: Unify cache mode quoting

2019-08-27 Thread Nir Soffer
On Tue, Aug 27, 2019 at 8:43 PM Max Reitz wrote: > > On 27.08.19 19:38, Nir Soffer wrote: > > On Tue, Aug 27, 2019 at 8:36 PM Max Reitz > <mailto:mre...@redhat.com>> wrote: > > > > On 27.08.19 19:34, Nir Soffer wrote: > > > Quoting cache mo

Re: [Qemu-devel] [PATCH] iotests: Unify cache mode quoting

2019-08-27 Thread Nir Soffer
On Tue, Aug 27, 2019 at 8:36 PM Max Reitz wrote: > On 27.08.19 19:34, Nir Soffer wrote: > > Quoting cache mode is not needed, and most tests use unquoted values. > > Unify all test to use the same style. > > S-o-b is missing, shall I add it? > Thanks! Signed-off-b

[Qemu-devel] [PATCH] iotests: Unify cache mode quoting

2019-08-27 Thread Nir Soffer
Quoting cache mode is not needed, and most tests use unquoted values. Unify all test to use the same style. --- tests/qemu-iotests/026 | 4 ++-- tests/qemu-iotests/039 | 4 ++-- tests/qemu-iotests/052 | 2 +- tests/qemu-iotests/091 | 4 ++-- 4 files changed, 7 insertions(+), 7 deletions(-) diff

Re: [Qemu-devel] [PATCH v3 0/2] Optimize alignment probing

2019-08-27 Thread Nir Soffer
Adding Eric On Tue, Aug 27, 2019 at 4:05 AM Nir Soffer wrote: > When probing unallocated area on remote XFS filesystem we cannot detect > request > alignment and we fallback to safe value which may not be optimal. Avoid > this > fallback by always allocating the first block when

[Qemu-devel] [PATCH v3 1/2] block: posix: Always allocate the first block

2019-08-26 Thread Nir Soffer
comparing the total time spent: TypeBefore(s) After(s)Diff(%) --- real 530.028469.123 -11.4 user 17.204 10.768 -37.4 sys17.881 7.011 -60.7 We can see very clear improvement in CPU usage. Signed-off-by: Ni

[Qemu-devel] [PATCH v3 2/2] iotests: Test allocate_first_block() with O_DIRECT

2019-08-26 Thread Nir Soffer
O_DIRECT does not require any alignment. Signed-off-by: Nir Soffer Reviewed-by: Max Reitz --- tests/qemu-iotests/175 | 28 tests/qemu-iotests/175.out | 8 2 files changed, 36 insertions(+) diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175 index

[Qemu-devel] [PATCH v3 0/2] Optimize alignment probing

2019-08-26 Thread Nir Soffer
esults since I ran them only once v1 was here: https://lists.nongnu.org/archive/html/qemu-block/2019-08/msg00821.html Nir Soffer (2): block: posix: Always allocate the first block iotests: Test allocate_first_block() with O_DIRECT block/file-posix.c

Re: [Qemu-devel] [PATCH v2 1/2] block: posix: Always allocate the first block

2019-08-26 Thread Nir Soffer
On Mon, Aug 26, 2019 at 3:31 PM Max Reitz wrote: > > On 26.08.19 00:03, Nir Soffer wrote: ... > > +/* > > + * Help alignment probing by allocating the first block. > > + * > > + * When reading with direct I/O from unallocated area on Gluster backed by > > XFS,

Re: [Qemu-devel] [PATCH v2 1/2] block: posix: Always allocate the first block

2019-08-26 Thread Nir Soffer
On Mon, Aug 26, 2019 at 4:49 PM Eric Blake wrote: > > On 8/26/19 7:31 AM, Max Reitz wrote: > > >> # the file size. This function hides the resulting difference in the > >> # stat -c '%b' output. > >> # Parameter 1: Number of blocks an empty file occupies > >> -# Parameter 2: Image size in

Re: [Qemu-devel] [PATCH v2 1/2] block: posix: Always allocate the first block

2019-08-26 Thread Nir Soffer
On Mon, Aug 26, 2019 at 4:46 PM Eric Blake wrote: > > On 8/25/19 5:03 PM, Nir Soffer wrote: > > When creating an image with preallocation "off" or "falloc", the first > > block of the image is typically not allocated. When using Gluster > > storage

Re: [Qemu-devel] [PATCH v2 2/2] iotests: Test allocate_first_block() with O_DIRECT

2019-08-25 Thread Nir Soffer
On Mon, Aug 26, 2019 at 1:03 AM Nir Soffer wrote: > Using block_resize we can test allocate_first_block() with file > descriptor opened with O_DIRECT, ensuring that it works for any size > larger than 4096 bytes. > > Testing smaller sizes is tricky as the result depends on the fi

[Qemu-devel] [PATCH v2 2/2] iotests: Test allocate_first_block() with O_DIRECT

2019-08-25 Thread Nir Soffer
Using block_resize we can test allocate_first_block() with file descriptor opened with O_DIRECT, ensuring that it works for any size larger than 4096 bytes. Testing smaller sizes is tricky as the result depends on the filesystem used for testing. For example on NFS any size will work since

[Qemu-devel] [PATCH v2 0/2] Optimize alignment probing

2019-08-25 Thread Nir Soffer
for allocate_first_block() with block_resize (Max) - Drop provisioing tests results since I ran them only once v1 was here: https://lists.nongnu.org/archive/html/qemu-block/2019-08/msg00821.html Nir Soffer (2): block: posix: Always allocate the first block iotests: Test allocate_first_block

[Qemu-devel] [PATCH v2 1/2] block: posix: Always allocate the first block

2019-08-25 Thread Nir Soffer
-60.7 We can see very clear improvement in CPU usage. Signed-off-by: Nir Soffer --- block/file-posix.c| 43 +++ tests/qemu-iotests/{150.out => 150.out.qcow2} | 0 tests/qemu-iotests/150.out.raw| 12 ++

Re: [Qemu-devel] [Qemu-block] [PATCH] block: posix: Always allocate the first block

2019-08-25 Thread Nir Soffer
On Sun, Aug 25, 2019 at 10:44 AM Maxim Levitsky wrote: > On Sat, 2019-08-17 at 00:21 +0300, Nir Soffer wrote: > > When creating an image with preallocation "off" or "falloc", the first > > block of the image is typically not allocated. When using Gluster &

Re: [Qemu-devel] [PATCH] block: posix: Always allocate the first block

2019-08-24 Thread Nir Soffer
On Fri, Aug 23, 2019 at 8:53 PM Max Reitz wrote: > On 23.08.19 18:48, Nir Soffer wrote: > > On Fri, Aug 23, 2019 at 4:58 PM Max Reitz > <mailto:mre...@redhat.com>> wrote: > > [...] > > > If you have a format layer that truncates the image to a fix

Re: [Qemu-devel] [PATCH] block: posix: Always allocate the first block

2019-08-23 Thread Nir Soffer
On Fri, Aug 23, 2019 at 4:58 PM Max Reitz wrote: > On 22.08.19 21:01, Nir Soffer wrote: > > On Thu, Aug 22, 2019 at 9:11 PM Max Reitz > <mailto:mre...@redhat.com>> wrote: > > > > On 22.08.19 18:39, Nir Soffer wrote: > > > On Thu, Aug 22, 20

Re: [Qemu-devel] [PATCH] block: posix: Always allocate the first block

2019-08-23 Thread Nir Soffer
On Fri, Aug 23, 2019 at 4:58 PM Max Reitz wrote: > On 22.08.19 21:01, Nir Soffer wrote: > ... > > > > @@ -1794,6 +1815,8 @@ static int handle_aiocb_truncate(void > > *opaque) > > > > /*

Re: [Qemu-devel] [PATCH] block: gluster: Probe alignment limits

2019-08-22 Thread Nir Soffer
On Thu, Aug 22, 2019 at 10:03 AM Niels de Vos wrote: > On Wed, Aug 21, 2019 at 07:04:17PM +0200, Max Reitz wrote: > > On 17.08.19 23:21, Nir Soffer wrote: > > > Implement alignment probing similar to file-posix, by reading from the > > > first 4k of the image. >

Re: [Qemu-devel] [PATCH] block: posix: Always allocate the first block

2019-08-22 Thread Nir Soffer
On Thu, Aug 22, 2019 at 9:11 PM Max Reitz wrote: > On 22.08.19 18:39, Nir Soffer wrote: > > On Thu, Aug 22, 2019 at 5:28 PM Max Reitz > <mailto:mre...@redhat.com>> wrote: > > > > On 16.08.19 23:21, Nir Soffer wrote: > > > When creating an

Re: [Qemu-devel] [PATCH] block: posix: Always allocate the first block

2019-08-22 Thread Nir Soffer
On Thu, Aug 22, 2019 at 5:28 PM Max Reitz wrote: > On 16.08.19 23:21, Nir Soffer wrote: > > When creating an image with preallocation "off" or "falloc", the first > > block of the image is typically not allocated. When using Gluster > > storage backed by

Re: [Qemu-devel] [PATCH] block: posix: Always allocate the first block

2019-08-22 Thread Nir Soffer
Max, did you have time to look at this? On Sat, Aug 17, 2019 at 12:21 AM Nir Soffer wrote: > When creating an image with preallocation "off" or "falloc", the first > block of the image is typically not allocated. When using Gluster > storage backed by XFS filesys

Re: [Qemu-devel] [Qemu-block] [PATCH 05/13] qcrypto-luks: clear the masterkey and password before freeing them always

2019-08-21 Thread Nir Soffer
On Wed, Aug 14, 2019, 23:23 Maxim Levitsky wrote: > While there are other places where these are still stored in memory, > this is still one less key material area that can be sniffed with > various side channel attacks > > > > Signed-off-by: Maxim Levitsky > --- > crypto/block-luks.c | 52

Re: [Qemu-devel] [Qemu-block] [PATCH] nbd: Advertise multi-conn for shared read-only connections

2019-08-20 Thread Nir Soffer
On Mon, Aug 19, 2019 at 9:04 PM Eric Blake wrote: > On 8/17/19 8:31 PM, Nir Soffer wrote: > >>> Also, for qemu-nbd, shouldn't we allow -e only together with -r ? > >> > >> I'm reluctant to; it might break whatever existing user is okay exposing > >> it

Re: [Qemu-devel] [PATCH] block: Use QEMU_IS_ALIGNED instead of reinventing it

2019-08-20 Thread Nir Soffer
On Tue, Aug 20, 2019 at 10:51 PM Max Reitz wrote: > On 17.08.19 19:53, Nir Soffer wrote: > > Replace instances of: > > > > (n & (BDRV_SECTOR_SIZE - 1)) == 0) > > > > With: > > > > QEMU_IS_ALIGNED(n, BDRV_SECTOR_SIZE) > > >

Re: [Qemu-devel] [Qemu-block] [PATCH] block: Use QEMU_IS_ALIGNED instead of reinventing it

2019-08-20 Thread Nir Soffer
On Tue, Aug 20, 2019 at 10:30 PM John Snow wrote: > > > On 8/17/19 1:53 PM, Nir Soffer wrote: > > Replace instances of: > > > > (n & (BDRV_SECTOR_SIZE - 1)) == 0) > > > > With: > > > > QEMU_IS_ALIGNED(n, BDRV_SECTOR_SIZE) >

Re: [Qemu-devel] [Qemu-block] [PATCH] nbd: Advertise multi-conn for shared read-only connections

2019-08-17 Thread Nir Soffer
On Sat, Aug 17, 2019 at 5:30 PM Eric Blake wrote: > On 8/16/19 5:47 AM, Vladimir Sementsov-Ogievskiy wrote: > > >>> +++ b/blockdev-nbd.c > >>> @@ -189,7 +189,7 @@ void qmp_nbd_server_add(const char *device, bool > has_name, const char *name, > >>> } > >>> > >>> exp =

Re: [Qemu-devel] [PATCH] block: gluster: Probe alignment limits

2019-08-17 Thread Nir Soffer
On Sun, Aug 18, 2019 at 12:21 AM Nir Soffer wrote: > Implement alignment probing similar to file-posix, by reading from the > first 4k of the image. > > Before this change, provisioning a VM on storage with sector size of > 4096 bytes would fail when the installer try to create fi

[Qemu-devel] [PATCH] block: gluster: Probe alignment limits

2019-08-17 Thread Nir Soffer
value (4096) even when using storage with sector size of 512 bytes. Signed-off-by: Nir Soffer --- block/gluster.c | 47 +++ 1 file changed, 47 insertions(+) diff --git a/block/gluster.c b/block/gluster.c index f64dc5b01e..d936240b72 100644 --- a/block

[Qemu-devel] [PATCH] block: Use QEMU_IS_ALIGNED instead of reinventing it

2019-08-17 Thread Nir Soffer
ient but it is used only in assert() except one instance, so it should not matter. Signed-off-by: Nir Soffer --- block/bochs.c | 4 ++-- block/cloop.c | 4 ++-- block/dmg.c | 4 ++-- block/io.c| 8 block/qcow2.c | 4 ++-- block/vvfat.c | 8 qemu-img.c| 2 +- 7 files changed

Re: [Qemu-devel] [PULL 5/7] file-posix: Support BDRV_REQ_NO_FALLBACK for zero writes

2019-08-17 Thread Nir Soffer
On Thu, Aug 15, 2019 at 1:29 PM Kevin Wolf wrote: > Am 15.08.2019 um 04:44 hat Eric Blake geschrieben: > > On 3/26/19 10:51 AM, Kevin Wolf wrote: > > > We know that the kernel implements a slow fallback code path for > > > BLKZEROOUT, so if BDRV_REQ_NO_FALLBACK is given, we shouldn't call it. >

Re: [Qemu-devel] [Qemu-block] [PATCH] block: posix: Always allocate the first block

2019-08-16 Thread Nir Soffer
On Sat, Aug 17, 2019 at 12:57 AM John Snow wrote: > On 8/16/19 5:21 PM, Nir Soffer wrote: > > When creating an image with preallocation "off" or "falloc", the first > > block of the image is typically not allocated. When using Gluster > > storage bac

<    1   2   3   4   >