Re: [Qemu-block] [PATCH v2 2/3] block/mirror: Fix target backing BDS

2016-06-08 Thread Nir Soffer
On Wed, Jun 8, 2016 at 12:32 PM, Kevin Wolf wrote: > Am 06.06.2016 um 16:42 hat Max Reitz geschrieben: >> Currently, we are trying to move the backing BDS from the source to the >> target in bdrv_replace_in_backing_chain() which is called from >> mirror_exit(). However,

Re: [Qemu-block] [PATCH v2 2/3] block/mirror: Fix target backing BDS

2016-06-09 Thread Nir Soffer
On Thu, Jun 9, 2016 at 11:58 AM, Kevin Wolf <kw...@redhat.com> wrote: > Am 08.06.2016 um 17:39 hat Nir Soffer geschrieben: >> On Wed, Jun 8, 2016 at 12:32 PM, Kevin Wolf <kw...@redhat.com> wrote: >> > Am 06.06.2016 um 16:42 hat Max Reitz geschrieben: >>

[Qemu-block] [PATCH] qemu-img: Do not truncate before preallocation

2017-02-03 Thread Nir Soffer
= 1 This happens because posix_fallocate is checking if each block is allocated before writing a byte to the block, and when truncating the file before preallocation, all blocks are unallocated. Signed-off-by: Nir Soffer <nir...@gmail.com> --- I sent this a week ago: http://lists.no

Re: [Qemu-block] [Qemu-devel] [PATCH v3 1/3] qemu-io: Return non-zero exit code on failure

2017-01-31 Thread Nir Soffer
On Mon, Jan 30, 2017 at 6:44 PM, Eric Blake <ebl...@redhat.com> wrote: > On 01/27/2017 09:59 PM, Nir Soffer wrote: >> From: Nir Soffer <nsof...@redhat.com> >> >> The result of openfile was not checked, leading to failure deep in the >> actual command wi

[Qemu-block] [PATCH v5 1/3] qemu-io: Return non-zero exit code on failure

2017-01-31 Thread Nir Soffer
this error now: no file open, try 'help open' But some tests expected it; the line was removed from the test output. Signed-off-by: Nir Soffer <nir...@gmail.com> Reviewed-by: Eric Blake <ebl...@redhat.com> --- qemu-io.c | 8 ++-- tests/qemu-iotests/059.out |

[Qemu-block] [PATCH v5 3/3] qemu-io: Add failure regression tests

2017-01-31 Thread Nir Soffer
Add regression tests checking that qemu-io fails with non-zero exit code when reading non-existing file or using the wrong image format. Signed-off-by: Nir Soffer <nir...@gmail.com> --- tests/qemu-iotests/174 | 59 ++ tests/qemu-iotests/1

[Qemu-block] [PATCH v5 2/3] qemu-iotests: Add _unsupported_fmt helper

2017-01-31 Thread Nir Soffer
This helper allows adding tests supporting any format expect the specified formats. This may be useful to test that many formats behave in a common way. Signed-off-by: Nir Soffer <nir...@gmail.com> --- tests/qemu-iotests/common.rc | 11 +++ 1 file changed, 11 insertions(+) diff

[Qemu-block] [PATCH v5 0/3] Fix qemu-io return value on failure

2017-01-31 Thread Nir Soffer
. Note that test 059 has one unrelated test failure. Nir Soffer (3): qemu-io: Return non-zero exit code on failure qemu-iotests: Add _unsupported_fmt helper qemu-io: Add failure regression tests qemu-io.c| 8 -- tests/qemu-iotests/059.out | 3 --- tests/qemu-iotests

[Qemu-block] [PATCH v4 2/2] qemu-io: Add regression tests

2017-01-31 Thread Nir Soffer
From: Nir Soffer <nsof...@redhat.com> Add regression tests checking that qemu-io fail with non-zero exit code when reading non-existing file or using the wrong format. Signed-off-by: Nir Soffer <nir...@gmail.com> --- tests/qemu-iotests/

[Qemu-block] [PATCH v4 1/2] qemu-io: Return non-zero exit code on failure

2017-01-31 Thread Nir Soffer
From: Nir Soffer <nsof...@redhat.com> The result of openfile was not checked, leading to failure deep in the actual command with confusing error message, and exiting with exit code 0. Here is a simple example - trying to read with the wrong format: $ touch file $ qemu-io -f qc

[Qemu-block] [PATCH v4 0/2] Fix qemu-io return value on failure

2017-01-31 Thread Nir Soffer
manually and by running tests/check-block.sh. Note that test 059 has one unrelated test failure. Nir Soffer (2): qemu-io: Return non-zero exit code on failure qemu-io: Add regression tests qemu-io.c | 8 +-- tests/qemu-iotests/059.out | 3 --- tests/qemu-iotests/070.out | 1

[Qemu-block] [PATCH v3 3/3] qemu-io: Fix tests expecting the wrong output

2017-01-30 Thread Nir Soffer
From: Nir Soffer <nsof...@redhat.com> Many tests expected the wrong behavior when qemu-io call into the command with after failing to open the file, writing this error: no file open, try 'help open' Now that we fail immediately when opening a file fails, this error does not

[Qemu-block] [PATCH v3 2/3] qemu-io: Add regression tests

2017-01-30 Thread Nir Soffer
From: Nir Soffer <nsof...@redhat.com> Add regression tests checking that qemu-io fail with non-zero exit code when reading non-existing file or using the wrong format. Signed-off-by: Nir Soffer <nir...@gmail.com> --- tests/qemu-iotests/

[Qemu-block] [PATCH v3 1/3] qemu-io: Return non-zero exit code on failure

2017-01-30 Thread Nir Soffer
From: Nir Soffer <nsof...@redhat.com> The result of openfile was not checked, leading to failure deep in the actual command with confusing error message, and exiting with exit code 0. Here is a simple example - trying to read with the wrong format: $ touch file $ qemu-io -f qc

Re: [Qemu-block] [PATCH 0/3] qemu-img raw preallocation

2017-02-22 Thread Nir Soffer
On Wed, Feb 22, 2017 at 2:31 PM, Kevin Wolf <kw...@redhat.com> wrote: > Am 17.02.2017 um 01:51 hat Nir Soffer geschrieben: >> This series add missing tests for raw image preallocation, refine >> preallocation=full and improve documentation. >> >> Create on top o

Re: [Qemu-block] [PATCH 1/3] qemu-img: Add tests for raw image preallocation

2017-02-17 Thread Nir Soffer
On Fri, Feb 17, 2017 at 11:14 AM, Kevin Wolf <kw...@redhat.com> wrote: > Am 17.02.2017 um 01:51 hat Nir Soffer geschrieben: >> Add tests for creating raw image with and without the preallocation >> option. >> >> Signed-off-by: Nir Soffer <nir...@gmail.com

[Qemu-block] [PATCH 0/3] qemu-img raw preallocation

2017-02-16 Thread Nir Soffer
This series add missing tests for raw image preallocation, refine preallocation=full and improve documentation. Create on top of the commit 10ddfe7b6044 (qemu-img: Do not truncate before preallocation) from Kevin block branch. Nir Soffer (3): qemu-img: Add tests for raw image preallocation

[Qemu-block] [PATCH 1/3] qemu-img: Add tests for raw image preallocation

2017-02-16 Thread Nir Soffer
Add tests for creating raw image with and without the preallocation option. Signed-off-by: Nir Soffer <nir...@gmail.com> --- tests/qemu-iotests/175 | 61 ++ tests/qemu-iotests/175.out | 18 ++ tests/qemu-iotests/group | 1 + 3

[Qemu-block] [PATCH 3/3] qemu-img: Improve documentation for PREALLOC_MODE_FALLOC

2017-02-16 Thread Nir Soffer
Now that we are truncating the file in both PREALLOC_MODE_FULL and PREALLOC_MODE_OFF, not truncating in PREALLOC_MODE_FALLOC looks odd. Add a comment explaining why we do not truncate in this case. Signed-off-by: Nir Soffer <nir...@gmail.com> --- block/file-posix.c | 7 ++- 1 file chan

Re: [Qemu-block] [PATCH] qemu-img: Do not truncate before preallocation

2017-02-16 Thread Nir Soffer
Ping On Fri, Feb 3, 2017 at 9:50 PM, Nir Soffer <nir...@gmail.com> wrote: > When using file system that does not support fallocate() (e.g. NFS < > 4.2), truncating the file only when preallocation=OFF speeds up creating > raw file. > > Here is example run, tested on Fed

Re: [Qemu-block] [PATCH] qemu-img: Do not truncate before preallocation

2017-02-16 Thread Nir Soffer
On Thu, Feb 16, 2017 at 7:52 PM, Kevin Wolf <kw...@redhat.com> wrote: > Am 03.02.2017 um 20:50 hat Nir Soffer geschrieben: >> When using file system that does not support fallocate() (e.g. NFS < >> 4.2), truncating the file only when preallocation=OFF speeds up creating

Re: [Qemu-block] [PATCH v4] qemu-img: Check for backing image if specified during create

2017-07-12 Thread Nir Soffer
On Wed, Jul 12, 2017 at 8:00 PM Max Reitz wrote: > On 2017-07-12 16:56, Kevin Wolf wrote: > > Am 12.07.2017 um 16:42 hat Eric Blake geschrieben: > >> On 05/17/2017 07:38 AM, Max Reitz wrote: > >> > >> Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1213786 >

Re: [Qemu-block] [Qemu-devel] Supporting unsafe create when backing file is not accessible

2017-07-14 Thread Nir Soffer
On Wed, Jul 12, 2017 at 5:40 PM Eric Blake wrote: > On 07/12/2017 03:56 AM, Ala Hino wrote: > > Hi, > > > > We encountered a performance issue when creating a volume for a running > VM > > and we'd like to share the info with you. The root cause of the issue is > in > > our

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.10] block: Skip implicit nodes in query-block/blockstats

2017-07-19 Thread Nir Soffer
On Wed, Jul 19, 2017 at 4:05 PM Markus Armbruster wrote: > Kevin Wolf writes: > > > Commits 0db832f and 6cdbceb introduced the automatic insertion of filter > > nodes above the top layer of mirror and commit block jobs. The > > assumption made there was that

Re: [Qemu-block] [PATCH 0/6] QTests: use Python to run complex tests

2017-12-14 Thread Nir Soffer
On Thu, Dec 14, 2017 at 11:39 AM Stefan Hajnoczi wrote: > On Wed, Dec 13, 2017 at 06:35:51PM -0300, Philippe Mathieu-Daudé wrote: > > Hi, > > > > With this series we can now write tests using Python rather than C. > > For complex tests this can reduce the test development

Re: [Qemu-block] [ovirt-users] slow performance with export storage on glusterfs

2017-12-07 Thread Nir Soffer
t domain still using safelease), and qemu or kernel may have better options to avoid trashing the host cache, or guarantee reliable access to storage. Daivd, do you know if sanlock is effected by trashing the host cache? Adding also qemu-block mailing list. Nir > > >> >> Chee

Re: [Qemu-block] [ovirt-users] Scheduling daily Snapshot

2017-12-07 Thread Nir Soffer
On Wed, Dec 6, 2017 at 6:02 PM Jason Lelievre wrote: > Hello, > > What is the best way to set up a daily live snapshot for all VM, and have > the possibility to recover, for example, a specific VM to a specific day? > Each snapshot you create make reads and writes

Re: [Qemu-block] import thin provisioned disks with image upload

2017-12-07 Thread Nir Soffer
On Fri, Dec 8, 2017 at 1:35 AM Gianluca Cecchi <gianluca.cec...@gmail.com> wrote: > On Thu, Dec 7, 2017 at 9:33 PM, Nir Soffer <nsof...@redhat.com> wrote: > >> >> For transferring images over http, there is no way to avoid sending >> unused blocks, excep

Re: [Qemu-block] [ovirt-users] slow performance with export storage on glusterfs

2017-12-08 Thread Nir Soffer
On Fri, Dec 8, 2017 at 4:18 PM Kevin Wolf <kw...@redhat.com> wrote: > Am 07.12.2017 um 23:45 hat Nir Soffer geschrieben: > > The qemu bug https://bugzilla.redhat.com/713743 explains the issue: > > qemu-img was writing disk images using writeback and fillingup the &g

Re: [Qemu-block] [ovirt-users] Scheduling daily Snapshot

2017-12-08 Thread Nir Soffer
On Fri, Dec 8, 2017 at 4:08 PM Kevin Wolf <kw...@redhat.com> wrote: > Am 07.12.2017 um 23:19 hat Nir Soffer geschrieben: > > On Wed, Dec 6, 2017 at 6:02 PM Jason Lelievre <jlelie...@folksvfx.com> > > wrote: > > > > > Hello, > > > > > >

Re: [Qemu-block] import thin provisioned disks with image upload

2017-12-07 Thread Nir Soffer
On Thu, Dec 7, 2017 at 10:15 AM Gianluca Cecchi <gianluca.cec...@gmail.com> wrote: > On Thu, Dec 7, 2017 at 1:28 AM, Nir Soffer <nsof...@redhat.com> wrote: > >> >> >> On Thu, Dec 7, 2017 at 1:22 AM Gianluca Cecchi <gianluca.cec...@gmail.com> >> wr

Re: [Qemu-block] import thin provisioned disks with image upload

2017-12-08 Thread Nir Soffer
On Fri, Dec 8, 2017 at 5:23 PM Max Reitz <mre...@redhat.com> wrote: > On 2017-12-07 21:33, Nir Soffer wrote: > > [...] > > > > > Trying harder... > > > > > > The term "file size" is ambiguous in this context... > > > > >

Re: [Qemu-block] [ovirt-users] Scheduling daily Snapshot

2017-12-08 Thread Nir Soffer
On Fri, Dec 8, 2017 at 5:25 PM Max Reitz <mre...@redhat.com> wrote: > On 2017-12-07 23:19, Nir Soffer wrote: > > On Wed, Dec 6, 2017 at 6:02 PM Jason Lelievre <jlelie...@folksvfx.com > > <mailto:jlelie...@folksvfx.com>> wrote: > > > > Hello, >

Re: [Qemu-block] [VDSM] travis tests fail consistently since Apr 14

2018-06-11 Thread Nir Soffer
On Mon, Jun 11, 2018 at 1:03 PM Maor Lipchuk wrote: > On Tue, Jun 5, 2018 at 9:40 AM, Dan Kenigsberg wrote: > >> On Mon, Jun 4, 2018 at 7:14 PM, Nir Soffer wrote: >> > On Mon, Jun 4, 2018 at 6:56 PM Dan Kenigsberg >> wrote: >> >> >> >> On

Re: [Qemu-block] [VDSM] travis tests fail consistently since Apr 14

2018-06-11 Thread Nir Soffer
On Mon, Jun 11, 2018 at 7:20 PM Kevin Wolf wrote: > Am 11.06.2018 um 12:43 hat Nir Soffer geschrieben: > > On Mon, Jun 11, 2018 at 1:03 PM Maor Lipchuk > wrote: > > > > > On Tue, Jun 5, 2018 at 9:40 AM, Dan Kenigsberg > wrote: > > > > > >>

Re: [Qemu-block] [ovirt-users] Libvirt ERROR cannot access backing file after importing VM from OpenStack

2018-05-29 Thread Nir Soffer
n: > > > > On Thu, May 24, 2018 at 6:13 PM, Nir Soffer > wrote: > > > > > > > > > On Thu, May 24, 2018 at 6:06 PM Vrgotic, Marko < > > > m.vrgo...@activevideo.com> > > > > > wrote: > > &g

Re: [Qemu-block] [ovirt-users] Libvirt ERROR cannot access backing file after importing VM from OpenStack

2018-05-29 Thread Nir Soffer
On Tue, May 29, 2018 at 10:43 PM Eric Blake wrote: > On 05/28/2018 05:27 AM, Arik Hadas wrote: > ... > > Few months ago people from the oVirt-storage team checked the qemu > toolset > > and replied that this capability is not yet provided, therefore we > > implemented the workaround described

Re: [Qemu-block] [ovirt-users] Libvirt ERROR cannot access backing file after importing VM from OpenStack

2018-05-30 Thread Nir Soffer
On Wed, May 30, 2018 at 11:58 AM Richard W.M. Jones wrote: > On Wed, May 30, 2018 at 12:11:21AM +0300, Nir Soffer wrote: > > Exporting images or ova files: > > > > image in any format -> qemu-img -> [qcow2 byte stream] -> imageio http > > server -> http

Re: [Qemu-block] [ovirt-users] Libvirt ERROR cannot access backing file after importing VM from OpenStack

2018-05-30 Thread Nir Soffer
On Wed, May 30, 2018 at 12:35 AM Eric Blake wrote: > On 05/29/2018 04:11 PM, Nir Soffer wrote: > > >> I think real streaming is unlikely to happen because most image formats > >> that QEMU supports aren't made that way. If there is a compelling > >> reason, we c

Re: [Qemu-block] [ovirt-users] Re: Debugging ceph access

2018-05-31 Thread Nir Soffer
On Thu, May 31, 2018 at 1:55 AM Bernhard Dick wrote: > Hi, > > I found the reason for my timeout problems: It is the version of librbd1 > (which is 0.94.5) in conjunction with my CEPH test-environment which is > running the luminous release. > When I install the librbd1 (and librados2) packages

Re: [Qemu-block] [ovirt-users] Re: Debugging ceph access

2018-06-05 Thread Nir Soffer
On Fri, Jun 1, 2018 at 3:54 PM Stefan Hajnoczi wrote: > On Thu, May 31, 2018 at 11:02:01PM +0300, Nir Soffer wrote: > > On Thu, May 31, 2018 at 1:55 AM Bernhard Dick wrote: > > > > > Hi, > > > > > > I found the reason for my timeout problems: I

Re: [Qemu-block] [PATCH 0/2] qemu-img: Let "info" warn and go ahead without -U

2018-01-08 Thread Nir Soffer
On Mon, Jan 8, 2018 at 4:48 PM Kevin Wolf wrote: > Am 05.01.2018 um 07:55 hat Fam Zheng geschrieben: > > Management and users are accustomed to "qemu-img info" to query status of > > images even when they are used by guests. Since image locking was added, > the -U > >

Re: [Qemu-block] [Qemu-devel] [PATCH 0/2] qemu-img: Let "info" warn and go ahead without -U

2018-01-10 Thread Nir Soffer
On Wed, Jan 10, 2018 at 4:04 PM Kashyap Chamarthy wrote: > On Mon, Jan 08, 2018 at 03:41:36PM +0100, Kevin Wolf wrote: > > Am 05.01.2018 um 07:55 hat Fam Zheng geschrieben: > > > Management and users are accustomed to "qemu-img info" to query status > of > > > images even

Re: [Qemu-block] [ovirt-users] qcow2 images corruption

2018-02-18 Thread Nir Soffer
On Wed, Feb 7, 2018 at 7:09 PM Nicolas Ecarnot wrote: > Hello, > > TL; DR : qcow2 images keep getting corrupted. Any workaround? > > Long version: > This discussion has already been launched by me on the oVirt and on > qemu-block mailing list, under similar circumstances but

Re: [Qemu-block] NBD structured reads vs. block size

2018-08-01 Thread Nir Soffer
On Wed, Aug 1, 2018 at 4:40 PM Eric Blake wrote: > Rich Jones pointed me to questionable behavior in qemu's NBD server > implementation today: qemu advertises a minimum block size of 512 to any > client that promises to honor block sizes, but when serving up a raw > file that is not aligned to a

Re: [Qemu-block] Disconecting /dev/nbdX leaves stale partitions and device

2018-07-28 Thread Nir Soffer
On Fri, Jul 27, 2018 at 9:53 PM Nir Soffer wrote: > I also tested uploading to raw disk using qemu-nbd on my dev setup with > LIO storage over 1G nic. > For reference, blkdiscard --zeroout takes 38.5 seconds on this setup. > > imageio example upload to /dev/nbd0[1]: 40.3 sec

Re: [Qemu-block] interactive qemu-img

2018-07-30 Thread Nir Soffer
On Mon, Jul 30, 2018 at 8:00 PM Programmingkid wrote: > > > On Jul 30, 2018, at 11:09 AM, Eric Blake wrote: > > > > On 07/28/2018 08:22 PM, Programmingkid wrote: > >> I thought of a way to make qemu-img much more user-friendly. When the > user opens qemu-img without any arguments, we could

Re: [Qemu-block] Disconecting /dev/nbdX leaves stale partitions and device

2018-07-29 Thread Nir Soffer
On Sat, Jul 28, 2018 at 12:13 PM Richard W.M. Jones wrote: > On Sat, Jul 28, 2018 at 12:34:29AM +0300, Nir Soffer wrote: > > > $ time bash -c '(qemu-img convert -p -n fedora-27.img > > > nbd:unix:/var/tmp/sock; sync)' > > > (100.00/100%) > > > > &

Re: [Qemu-block] interactive qemu-img

2018-07-30 Thread Nir Soffer
On Mon, Jul 30, 2018 at 11:09 PM Eric Blake wrote: > On 07/30/2018 02:54 PM, Nir Soffer wrote: > > >>> $ qemu-img > >>> qemu-img: Not enough arguments > >>> Try 'qemu-img --help' for more information > > > > > > This is not user fri

Re: [Qemu-block] [Qemu-devel] [PATCH for-3.0] file-posix: Fix write_zeroes with unmap on block devices

2018-07-26 Thread Nir Soffer
On Thu, Jul 26, 2018 at 7:10 PM Eric Blake wrote: > On 07/26/2018 10:33 AM, Kevin Wolf wrote: > > > > > As far as I know, the comment you quoted is accurate for BLKDISCARD and > > BLKZEROOUT, but not for the fallocate() flags. > > > > I sure wish the man pages were more explicit on what

Re: [Qemu-block] [PATCH for-3.0] file-posix: Fix write_zeroes with unmap on block devices

2018-07-26 Thread Nir Soffer
On Thu, Jul 26, 2018 at 2:33 PM Kevin Wolf wrote: > The BLKDISCARD ioctl doesn't guarantee that the discarded blocks read as > all-zero afterwards, so don't try to abuse it for zero writing. We try > to only use this if BLKDISCARDZEROES tells us that it is safe, but this > is unreliable on older

Re: [Qemu-block] [Qemu-devel] [PATCH for-3.0] file-posix: Fix write_zeroes with unmap on block devices

2018-07-26 Thread Nir Soffer
On Thu, Jul 26, 2018 at 7:58 PM Kevin Wolf wrote: > Am 26.07.2018 um 18:46 hat Nir Soffer geschrieben: > > On Thu, Jul 26, 2018 at 7:10 PM Eric Blake wrote: > > > > > On 07/26/2018 10:33 AM, Kevin Wolf wrote: > > > > > > > > > >

Re: [Qemu-block] [PATCH RFC for-3.0-rc3 0/3] qemu-img: Disable copy offloading by default

2018-07-27 Thread Nir Soffer
On Fri, Jul 27, 2018 at 3:15 PM Fam Zheng wrote: > On Fri, Jul 27, 2018 at 6:29 PM Kevin Wolf wrote: > > > > Am 27.07.2018 um 05:33 hat Fam Zheng geschrieben: > > > Kevin pointed out that both glibc and kernel provides a slow fallback > of > > > copy_file_range which hurts thin provisioning.

Re: [Qemu-block] Disconecting /dev/nbdX leaves stale partitions and device

2018-07-27 Thread Nir Soffer
On Fri, Jul 27, 2018 at 4:20 PM Stefan Hajnoczi wrote: > On Thu, Jul 19, 2018 at 01:56:34PM +0300, Nir Soffer wrote: > > On Wed, Jul 18, 2018 at 6:35 PM Stefan Hajnoczi > wrote: > > > > > On Wed, Jul 11, 2018 at 08:28:38PM +0300, Nir Soffer wrote: > > > >

Re: [Qemu-block] Disconecting /dev/nbdX leaves stale partitions and device

2018-07-27 Thread Nir Soffer
On Fri, Jul 27, 2018 at 5:41 PM Richard W.M. Jones wrote: > On Fri, Jul 27, 2018 at 05:29:56PM +0300, Nir Soffer wrote: > > Do we support specifying an offset in qemu? > > Yes. virt-v2v uses this functionality to access files inside > tarballs (written by Tomas): > &

[Qemu-block] Python NBD client library

2018-07-31 Thread Nir Soffer
Thanks everyone for the good input on http://lists.nongnu.org/archive/html/qemu-block/2018-07/msg00168.html I'm convinced that the best direction for oVirt is having NBD client instead of using the kernel nbd client. I think a C library would be best, but a python version is faster to implement

Re: [Qemu-block] Python NBD client library

2018-08-03 Thread Nir Soffer
On Tue, Jul 31, 2018 at 6:17 PM Richard W.M. Jones wrote: ... > > If you like to check the prototype, see: > > https://gerrit.ovirt.org/c/93384/ > > > > qemu's nbd/client.c was very helpful so far, since NBD doc > > https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md > > is not

Re: [Qemu-block] Disconecting /dev/nbdX leaves stale partitions and device

2018-07-27 Thread Nir Soffer
On Fri, Jul 27, 2018 at 5:07 PM Richard W.M. Jones wrote: > On Fri, Jul 27, 2018 at 02:20:29PM +0100, Stefan Hajnoczi wrote: > > On Thu, Jul 19, 2018 at 01:56:34PM +0300, Nir Soffer wrote: > > > Having HTTP API makes it easy to integrate with. This is also the > protocol

Re: [Qemu-block] Disconecting /dev/nbdX leaves stale partitions and device

2018-07-27 Thread Nir Soffer
On Fri, Jul 27, 2018 at 10:36 PM Richard W.M. Jones wrote: > On Fri, Jul 27, 2018 at 09:53:32PM +0300, Nir Soffer wrote: > > I think performance is pretty good now, after I added efficient zeroing: > > https://gerrit.ovirt.org/c/92901/ > > This should be available in 4.2

Re: [Qemu-block] [Qemu-devel] [PATCH for-3.0] file-posix: Fix write_zeroes with unmap on block devices

2018-07-26 Thread Nir Soffer
On Thu, Jul 26, 2018 at 5:28 PM Eric Blake wrote: > On 07/26/2018 06:33 AM, Kevin Wolf wrote: > > The BLKDISCARD ioctl doesn't guarantee that the discarded blocks read as > > all-zero afterwards, so don't try to abuse it for zero writing. We try > > to only use this if BLKDISCARDZEROES tells us

Re: [Qemu-block] [Libguestfs] [PATCH] v2v: Add --print-estimate option to print source size estimate.

2018-08-15 Thread Nir Soffer
On Wed, Aug 15, 2018 at 1:55 PM Richard W.M. Jones wrote: > (Adding Stefan who implemented the subcommand) > > On Wed, Aug 15, 2018 at 12:44:44PM +0200, Kevin Wolf wrote: > > Am 15.08.2018 um 12:26 hat Richard W.M. Jones geschrieben: > > > On Tue, Aug 14, 2018 at 09:3

Re: [Qemu-block] Disconecting /dev/nbdX leaves stale partitions and device

2018-07-19 Thread Nir Soffer
On Wed, Jul 18, 2018 at 6:35 PM Stefan Hajnoczi wrote: > On Wed, Jul 11, 2018 at 08:28:38PM +0300, Nir Soffer wrote: > > On Wed, Jul 11, 2018 at 3:40 PM Stefan Hajnoczi > wrote: > > > > > On Thu, Jul 05, 2018 at 02:10:58AM +0300, Nir Soffer wrote: > > > >

Re: [Qemu-block] [kubevirt-dev] Re: Converting qcow2 image on the fly to raw format

2018-07-19 Thread Nir Soffer
On Mon, Jul 16, 2018 at 11:56 AM Daniel P. Berrangé wrote: ... > Recommendation is to run 'qemu-img info' to extract the metadata and sanity > check results eg no backing file list, not unreasonable size, etc. When > running 'qemu-img info' apply process limits of 30 secs CPU time, and 1 GB >

Re: [Qemu-block] [kubevirt-dev] Re: Converting qcow2 image on the fly to raw format

2018-07-19 Thread Nir Soffer
On Thu, Jul 19, 2018 at 10:24 PM Richard W.M. Jones wrote: > On Thu, Jul 19, 2018 at 09:50:00PM +0300, Nir Soffer wrote: > > On Mon, Jul 16, 2018 at 11:56 AM Daniel P. Berrangé > > > wrote: > > ... > > > > > Recommendation is to run 'qemu-img inf

Re: [Qemu-block] [kubevirt-dev] Re: Converting qcow2 image on the fly to raw format

2018-07-19 Thread Nir Soffer
On Thu, Jul 19, 2018 at 10:27 PM Richard W.M. Jones wrote: > > This is the code in libguestfs if you prefer something in C: > > > https://github.com/libguestfs/libguestfs/blob/3a87c4bb441430c9cef9223e67d10bf51a4e865c/lib/info.c#L150-L160 I see that libguestfs is using 10 seconds limit:

Re: [Qemu-block] Disconecting /dev/nbdX leaves stale partitions and device

2018-07-04 Thread Nir Soffer
On Thu, Jul 5, 2018 at 2:10 AM Nir Soffer wrote: > I'm trying to access a qcow2 image via /dev/nbdX device. > > I'm connecting a fedora 27 image (created using virt-builder) on a block > device: > > qemu-nbd -c /dev/nbd0 -f qcow2 -n --detect-zeroes=on --aio=native > /dev/27

[Qemu-block] Disconecting /dev/nbdX leaves stale partitions and device

2018-07-04 Thread Nir Soffer
I'm trying to access a qcow2 image via /dev/nbdX device. I'm connecting a fedora 27 image (created using virt-builder) on a block device: qemu-nbd -c /dev/nbd0 -f qcow2 -n --detect-zeroes=on --aio=native /dev/27837a03-64f9-4f2b-abb0-daa2195b01ae/acb196da-31fc-454d-856b-c31ab24715b3 It works,

Re: [Qemu-block] Converting qcow2 image on the fly to raw format

2018-07-09 Thread Nir Soffer
On Mon, Jul 9, 2018 at 8:06 PM Richard W.M. Jones wrote: > On Mon, Jul 09, 2018 at 07:02:50PM +0200, Kevin Wolf wrote: > > Am 09.07.2018 um 18:52 hat Richard W.M. Jones geschrieben: > > > On Mon, Jul 09, 2018 at 07:38:05PM +0300, Nir Soffer wrote: > > > > We are

Re: [Qemu-block] Disconecting /dev/nbdX leaves stale partitions and device

2018-07-11 Thread Nir Soffer
On Wed, Jul 11, 2018 at 3:40 PM Stefan Hajnoczi wrote: > On Thu, Jul 05, 2018 at 02:10:58AM +0300, Nir Soffer wrote: > > CCing Eric and Paolo, who maintain QEMU's NBD code. > > > I'm trying to access a qcow2 image via /dev/nbdX device. > > Untrusted disk images should not

[Qemu-block] Converting qcow2 image on the fly to raw format

2018-07-09 Thread Nir Soffer
We are discussing importing VM images to KubVirt. The goal is to be able to import an existing qcow2 disk, probably some appliance stored on http server, and and convert it to raw format for writing to storage. This can be also useful for for oVirt for importing OVA, since we like to pack disks

Re: [Qemu-block] [Qemu-devel] [PATCH 1/2] block: curl: Allow arbitrary HTTP request headers to be set.

2018-03-01 Thread Nir Soffer
On Thu, Mar 1, 2018 at 4:00 PM Richard W.M. Jones wrote: > Allow arbitrary HTTP request headers to be set, like this: > > qemu-img create -f qcow2 \ > -b 'json:{ "file.driver":"http", > "file.url":"http://192.168.0.249/scratch/test.img;, >

Re: [Qemu-block] [Qemu-devel] [PATCH 2/2] block: curl: Allow Certificate Authority bundle to be passed in.

2018-03-01 Thread Nir Soffer
On Thu, Mar 1, 2018 at 4:02 PM Richard W.M. Jones wrote: > This allows a Certificate Authority bundle to be passed to the curl > driver, allowing authentication against servers that check > certificates. For example this allows you to access a disk on an > oVirt node: > >

Re: [Qemu-block] [Qemu-discuss] qemu-img convert stuck

2018-04-08 Thread Nir Soffer
On Sun, Apr 8, 2018 at 9:27 PM Benny Zlotnik wrote: > Hi, > > As part of copy operation initiated by rhev got stuck for more than a day > and consumes plenty of CPU > vdsm 13024 3117 99 Apr07 ?1-06:58:43 /usr/bin/qemu-img convert > -p -t none -T none -f qcow2 >

Re: [Qemu-block] v2v: -o rhv-upload: Long time spent zeroing the disk

2018-04-10 Thread Nir Soffer
On Tue, Apr 10, 2018 at 1:44 PM Richard W.M. Jones wrote: > We now have true zeroing support in oVirt imageio, thanks for that. > > However a problem is that ‘qemu-img convert’ issues zero requests for > the whole disk before starting the transfer. It does this using 32 MB >

Re: [Qemu-block] v2v: -o rhv-upload: Long time spent zeroing the disk

2018-04-10 Thread Nir Soffer
On Tue, Apr 10, 2018 at 4:48 PM Kevin Wolf <kw...@redhat.com> wrote: > Am 10.04.2018 um 15:03 hat Nir Soffer geschrieben: > > On Tue, Apr 10, 2018 at 1:44 PM Richard W.M. Jones <rjo...@redhat.com> > > wrote: > > > > > We now have tru

Re: [Qemu-block] [Libguestfs] v2v: -o rhv-upload: Long time spent zeroing the disk

2018-04-10 Thread Nir Soffer
On Tue, Apr 10, 2018 at 6:00 PM Eric Blake wrote: > On 04/10/2018 09:40 AM, Richard W.M. Jones wrote: > >> When the destination is a block device we cannot avoid zeroing since a > block > >> device may contain junk data (we usually get dirty empty images from our > >> local >

Re: [Qemu-block] v2v: -o rhv-upload: Long time spent zeroing the disk

2018-04-10 Thread Nir Soffer
On Tue, Apr 10, 2018 at 5:50 PM Richard W.M. Jones <rjo...@redhat.com> wrote: > On Tue, Apr 10, 2018 at 02:07:33PM +0000, Nir Soffer wrote: > > This makes sense if the device is backed by a block device on oVirt side, > > and the NBD support efficient zeroing. But in

Re: [Qemu-block] v2v: -o rhv-upload: Long time spent zeroing the disk

2018-04-12 Thread Nir Soffer
On Thu, Apr 12, 2018 at 2:07 AM Nir Soffer <nir...@gmail.com> wrote: > On Tue, Apr 10, 2018 at 6:53 PM Richard W.M. Jones <rjo...@redhat.com> > wrote: > ... > Dan Berrange pointed out earlier on that it might be easier if imageio >> > just exposed NBD, or if we f

Re: [Qemu-block] v2v: -o rhv-upload: Long time spent zeroing the disk

2018-04-11 Thread Nir Soffer
On Tue, Apr 10, 2018 at 6:53 PM Richard W.M. Jones <rjo...@redhat.com> wrote: > On Tue, Apr 10, 2018 at 03:25:47PM +0000, Nir Soffer wrote: > > On Tue, Apr 10, 2018 at 5:50 PM Richard W.M. Jones <rjo...@redhat.com> > > wrote: > > > > > On Tue, Apr 10,

Re: [Qemu-block] v2v: -o rhv-upload: Long time spent zeroing the disk

2018-04-12 Thread Nir Soffer
On Thu, Apr 12, 2018 at 5:42 PM Eric Blake wrote: > On 04/12/2018 05:24 AM, Richard W.M. Jones wrote: > > > I don't think we have nbd-server in RHEL, and in any case wouldn't it > > be better to use qemu-nbd? > > > > You just start a new qemu-nbd process instead of faffing

[Qemu-block] [PATCH 3/3] qemu-iotests: Test new qemu-nbd --nolist option

2018-04-13 Thread Nir Soffer
Add new test module for tesing the --nolist option. Signed-off-by: Nir Soffer <nir...@gmail.com> --- tests/qemu-iotests/214 | 46 ++ tests/qemu-iotests/214.out | 2 ++ tests/qemu-iotests/group | 1 + 3 files changed, 49 insertions(+)

[Qemu-block] [PATCH 0/3] qemu-nbd: Disallow listing exports

2018-04-13 Thread Nir Soffer
ot iotests.py - and use the new infrastructure to add test the new option using nbd-client. Adding dependency on nbd-client may be probelematic, but I think qemu-nbd should have tests ensuring compatibility with other tools. Nir Soffer (3): nbd: Add option to disallow listing exports

[Qemu-block] [PATCH 2/3] iotests.py: Add helper for running commands

2018-04-13 Thread Nir Soffer
stdout and stderr, returning the command output or raising CommandFailed. These helpers will be used by new qemu-nbd tests. And later can be used to cleanup helpers for running qemu-* tools in iotests.py. Signed-off-by: Nir Soffer <nir...@gmail.com> --- tests/qemu-iotests/iotests.p

Re: [Qemu-block] Change in qemu 2.12 causes qemu-img convert to NBD to write more data

2018-11-07 Thread Nir Soffer
On Wed, Nov 7, 2018 at 7:27 PM Kevin Wolf wrote: > Am 07.11.2018 um 15:56 hat Nir Soffer geschrieben: > > Wed, Nov 7, 2018 at 4:36 PM Richard W.M. Jones > wrote: > > > > > Another thing I tried was to change the NBD server (nbdkit) so that it > > > doesn't

Re: [Qemu-block] Change in qemu 2.12 causes qemu-img convert to NBD to write more data

2018-11-11 Thread Nir Soffer
On Wed, Nov 7, 2018 at 6:42 PM Eric Blake wrote: > On 11/7/18 6:13 AM, Richard W.M. Jones wrote: > > (I'm not going to claim this is a bug, but it causes a large, easily > > measurable performance regression in virt-v2v). > > I haven't closely looked at at this email thread yet, but a quick

Re: [Qemu-block] Change in qemu 2.12 causes qemu-img convert to NBD to write more data

2018-11-11 Thread Nir Soffer
On Wed, Nov 7, 2018 at 7:55 PM Nir Soffer wrote: > On Wed, Nov 7, 2018 at 7:27 PM Kevin Wolf wrote: > >> Am 07.11.2018 um 15:56 hat Nir Soffer geschrieben: >> > Wed, Nov 7, 2018 at 4:36 PM Richard W.M. Jones >> wrote: >> > >> > > Another thing

Re: [Qemu-block] KVM Forum block no[td]es

2018-11-11 Thread Nir Soffer
On Mon, Nov 12, 2018 at 12:25 AM Max Reitz wrote: > This is what I’ve taken from two or three BoF-like get-togethers on > blocky things. Amendments are more than welcome, of course. ... > Bitmaps > === > > (Got this section from sneaking into a BoF I wasn’t invited to. Oh > well. Won’t

Re: [Qemu-block] KVM Forum block no[td]es

2018-11-12 Thread Nir Soffer
On Mon, Nov 12, 2018 at 5:26 PM Max Reitz wrote: > On 12.11.18 00:36, Nir Soffer wrote: > > On Mon, Nov 12, 2018 at 12:25 AM Max Reitz > <mailto:mre...@redhat.com>> wrote: > > > > This is what I’ve taken from two or three BoF-like get-togethers on > >

Re: [Qemu-block] Change in qemu 2.12 causes qemu-img convert to NBD to write more data

2018-11-15 Thread Nir Soffer
On Sun, Nov 11, 2018 at 6:11 PM Nir Soffer wrote: > On Wed, Nov 7, 2018 at 7:55 PM Nir Soffer wrote: > >> On Wed, Nov 7, 2018 at 7:27 PM Kevin Wolf wrote: >> >>> Am 07.11.2018 um 15:56 hat Nir Soffer geschrieben: >>> > Wed, Nov 7, 2018 a

Re: [Qemu-block] Change in qemu 2.12 causes qemu-img convert to NBD to write more data

2018-11-07 Thread Nir Soffer
Wed, Nov 7, 2018 at 4:36 PM Richard W.M. Jones wrote: > Another thing I tried was to change the NBD server (nbdkit) so that it > doesn't advertise zero support to the client: > > $ nbdkit --filter=log --filter=nozero memory size=6G logfile=/tmp/log \ > --run './qemu-img convert

Re: [Qemu-block] Change in qemu 2.12 causes qemu-img convert to NBD to write more data

2018-11-17 Thread Nir Soffer
On Sat, Nov 17, 2018 at 11:13 PM Richard W.M. Jones wrote: > On Sat, Nov 17, 2018 at 10:59:26PM +0200, Nir Soffer wrote: > > On Fri, Nov 16, 2018 at 5:26 PM Kevin Wolf wrote: > > > > > Am 15.11.2018 um 23:27 hat Nir Soffer geschrieben: > > > > On Sun, Nov 11,

Re: [Qemu-block] [PATCH RFC] qemu-io: Prefer stderr for error messages

2018-12-12 Thread Nir Soffer
On Thu, Dec 13, 2018 at 12:13 AM Eric Blake wrote: > > When a qemu-io command fails, it's best if the failure message > goes to stderr rather than stdout. This makes sense, but it will break users like this:

Re: [Qemu-block] [Qemu-devel] [PATCH RFC] qemu-io: Prefer stderr for error messages

2018-12-13 Thread Nir Soffer
On Thu, Dec 13, 2018 at 11:27 PM Eric Blake wrote: > On 12/13/18 11:44 AM, Nir Soffer wrote: > > >> The things is, qemu-io was never meant to be used by other > >> applications that need to process the results, it's a tool for testing > >> and debugging. If we

Re: [Qemu-block] [Qemu-devel] Request for clarification on qemu-img convert behavior zeroing target host_device

2018-12-13 Thread Nir Soffer
On Thu, Dec 13, 2018 at 11:14 PM De Backer, Fred (Nokia - BE/Antwerp) < fred.de_bac...@nokia.com> wrote: > > Indeed, performance traces are important for issues like this. > See strace of both FC27 and FC29 attached > Looks like you traced only the main thread. All the I/O is done in other

Re: [Qemu-block] do not lseek in qcow2 block_status

2018-12-24 Thread Nir Soffer
On Mon, Dec 24, 2018 at 5:50 PM Vladimir Sementsov-Ogievskiy < vsement...@virtuozzo.com> wrote: > Hi all! > > bdrv_co_block_status digs bs->file for additional, more accurate search > for hole > inside region, reported as DATA by bs since long-ago > >commit

Re: [Qemu-block] [Qemu-devel] [PATCH RFC] qemu-io: Prefer stderr for error messages

2018-12-13 Thread Nir Soffer
On Thu, Dec 13, 2018 at 4:05 PM Kevin Wolf wrote: > Am 13.12.2018 um 11:47 hat Daniel P. Berrangé geschrieben: > > On Thu, Dec 13, 2018 at 01:52:29AM +0200, Nir Soffer wrote: > > > On Thu, Dec 13, 2018 at 12:13 AM Eric Blake wrote: > > > > > > > &g

Re: [Qemu-block] [Qemu-devel] [PATCH RFC] qemu-io: Prefer stderr for error messages

2018-12-13 Thread Nir Soffer
On Thu, Dec 13, 2018 at 12:47 PM Daniel P. Berrangé wrote: > On Thu, Dec 13, 2018 at 01:52:29AM +0200, Nir Soffer wrote: > > On Thu, Dec 13, 2018 at 12:13 AM Eric Blake wrote: > > > > > > When a qemu-io command fails, it's best if the failure message > > &g

Re: [Qemu-block] 4.2 CI failure on qemu-io tests

2018-12-07 Thread Nir Soffer
f-a8cb-48353cf898e7'] failed with rc=1 out='Pattern verification failed at offset 0, 1024 bytes\nread 1024/1024 bytes at offset 0\n1 KiB, 1 ops; 0. sec (17.756 MiB/sec and 18181.8182 ops/sec)\n' err='' This is the same issue fixed in master by: commit a341dcc3ab5c36afc7a536d3d4794c37fedcda6d Autho

Re: [Qemu-block] Change in qemu 2.12 causes qemu-img convert to NBD to write more data

2018-11-17 Thread Nir Soffer
On Fri, Nov 16, 2018 at 5:26 PM Kevin Wolf wrote: > Am 15.11.2018 um 23:27 hat Nir Soffer geschrieben: > > On Sun, Nov 11, 2018 at 6:11 PM Nir Soffer wrote: > > > > > On Wed, Nov 7, 2018 at 7:55 PM Nir Soffer wrote: > > > > > >> On

Re: [Qemu-block] [libvirt] [PATCH] qmp: Deprecate query-nodes option of query-blockstats

2019-01-28 Thread Nir Soffer
On Mon, Jan 28, 2019 at 6:13 PM Daniel P. Berrangé wrote: > On Mon, Jan 28, 2019 at 04:37:50PM +0100, Kevin Wolf wrote: > > Am 28.01.2019 um 16:15 hat Anton Kuchin geschrieben: > > > This option is broken since a6baa60807 in v2.9 and returns mostly > > > zeroes instead of real stats because

  1   2   3   4   >