Re: iotest failure -- test possibly not using sufficiently unique temp filename?

2019-10-18 Thread Thomas Huth
On 18/10/2019 10.42, Max Reitz wrote: > On 18.10.19 08:20, Thomas Huth wrote: >> On 17/10/2019 18.41, Peter Maydell wrote: >>> On Fri, 27 Sep 2019 at 17:44, Max Reitz wrote: On 27.09.19 18:39, Peter Maydell wrote: > Hi; I just saw this iotest failure (on an s390x box, as it

Re: [PATCH v8 3/3] docs: qcow2: introduce compression type feature

2019-10-18 Thread Eric Blake
On 10/18/19 4:47 AM, Vladimir Sementsov-Ogievskiy wrote: The patch add new additional field to qcow2 header: compression_type, which specifies compression type. If field is absent or zero, default compression type is set: ZLIB, which corresponds to current behavior. New compression type (ZSTD)

Re: iotest failure -- test possibly not using sufficiently unique temp filename?

2019-10-18 Thread Max Reitz
On 18.10.19 08:20, Thomas Huth wrote: > On 17/10/2019 18.41, Peter Maydell wrote: >> On Fri, 27 Sep 2019 at 17:44, Max Reitz wrote: >>> >>> On 27.09.19 18:39, Peter Maydell wrote: Hi; I just saw this iotest failure (on an s390x box, as it happens): TESTiotest-qcow2: 130

Re: iotest failure -- test possibly not using sufficiently unique temp filename?

2019-10-18 Thread Thomas Huth
On 17/10/2019 18.41, Peter Maydell wrote: > On Fri, 27 Sep 2019 at 17:44, Max Reitz wrote: >> >> On 27.09.19 18:39, Peter Maydell wrote: >>> Hi; I just saw this iotest failure (on an s390x box, as it happens): >>> >>> TESTiotest-qcow2: 130 [fail] >>> QEMU -- >>>

Re: [PATCH v2 01/23] iotests: Introduce $SOCK_DIR

2019-10-18 Thread Max Reitz
On 17.10.19 16:52, Eric Blake wrote: > On 10/17/19 8:31 AM, Max Reitz wrote: >> Unix sockets generally have a maximum path length.  Depending on your >> $TEST_DIR, it may be exceeded and then all tests that create and use >> Unix sockets there may fail. >> >> Circumvent this by adding a new

Re: [PATCH v7 1/2] docs: improve qcow2 spec about extending image header

2019-10-18 Thread Vladimir Sementsov-Ogievskiy
08.10.2019 12:05, Vladimir Sementsov-Ogievskiy wrote: > 07.10.2019 23:21, Eric Blake wrote: >> On 10/7/19 11:04 AM, Vladimir Sementsov-Ogievskiy wrote: >>> Make it more obvious how to add new fields to the version 3 header and >>> how to interpret them. >>> >>> Signed-off-by: Vladimir

Re: [PATCH v7 1/2] docs: improve qcow2 spec about extending image header

2019-10-18 Thread Vladimir Sementsov-Ogievskiy
18.10.2019 11:29, Vladimir Sementsov-Ogievskiy wrote: > 08.10.2019 12:05, Vladimir Sementsov-Ogievskiy wrote: >> 07.10.2019 23:21, Eric Blake wrote: >>> On 10/7/19 11:04 AM, Vladimir Sementsov-Ogievskiy wrote: Make it more obvious how to add new fields to the version 3 header and how to

Re: [PATCH] block/backup: drop dead code from backup_job_create

2019-10-18 Thread Stefano Garzarella
Hi Vladimir, On Thu, Oct 17, 2019 at 05:21:22PM +0300, Vladimir Sementsov-Ogievskiy wrote: > After commit 00e30f05de1d195, there is no more "goto error" points > after job creation, so after "error:" @job is always NULL and we don't > need roll-back job creation. I don't know this code very

Re: [PATCH] iotests: Skip read-only cases in 118 when run as root

2019-10-18 Thread Philippe Mathieu-Daudé
Hi Kevin, On 10/18/19 1:51 PM, Kevin Wolf wrote: Some tests in 118 use chmod to remove write permissions from the file and assume that the image can indeed not be opened read-write afterwards. This doesn't work when the test is run as root, because root can still open the file as writable even

[PATCH v8 1/3] docs: improve qcow2 spec about extending image header

2019-10-18 Thread Vladimir Sementsov-Ogievskiy
Make it more obvious how to add new fields to the version 3 header and how to interpret them. The specification is adjusted so for new defined optional fields: 1. Software may support some of these optional fields and ignore the others, which means that features may be backported to

Re: [PATCH v2 01/23] iotests: Introduce $SOCK_DIR

2019-10-18 Thread Eric Blake
On 10/18/19 4:03 AM, Max Reitz wrote:   -if [ ! -e "$TEST_DIR" ]; then -    mkdir "$TEST_DIR" +tmp_sock_dir=false +if [ -z "$SOCK_DIR" ]; then +    SOCK_DIR=$(mktemp -d) +    tmp_sock_dir=true   fi +mkdir -p "$SOCK_DIR" || _init_error 'Failed to create SOCK_DIR' Thinking about this

Re: [PATCH v8 2/3] docs: define padding for qcow2 header

2019-10-18 Thread Eric Blake
On 10/18/19 4:47 AM, Vladimir Sementsov-Ogievskiy wrote: Header extensions ends are already defined to be multiply of 8. Let's gently ask for header length to be a multiply of 8 too, when we have some additional fields. Requiring this may be considered as an incompatible change, so the padding

[PATCH] iotests: Skip read-only cases in 118 when run as root

2019-10-18 Thread Kevin Wolf
Some tests in 118 use chmod to remove write permissions from the file and assume that the image can indeed not be opened read-write afterwards. This doesn't work when the test is run as root, because root can still open the file as writable even when the permission bit isn't set. Introduce a

[PATCH] blockdev: modify blockdev-change-medium to change non-removable device

2019-10-18 Thread Denis Plotnikov
The modification is useful to workaround exclusive file access restrictions, e.g. to implement VM migration with shared disk stored on a storage with the exclusive file opening model: a destination VM is started waiting for incomming migration with a fake image drive, and later, on the last

[PATCH v8 2/3] docs: define padding for qcow2 header

2019-10-18 Thread Vladimir Sementsov-Ogievskiy
Header extensions ends are already defined to be multiply of 8. Let's gently ask for header length to be a multiply of 8 too, when we have some additional fields. Requiring this may be considered as an incompatible change, so the padding is optional. Actually, padding is allowed before this patch

Re: [PATCH v7 1/2] docs: improve qcow2 spec about extending image header

2019-10-18 Thread Eric Blake
On 10/18/19 4:27 AM, Vladimir Sementsov-Ogievskiy wrote: I would suggest a stronger requirement: header_length must be a multiple of 4, and must not land in the middle of any optional 8-byte field. Or maybe even add our compression type extension with 4 bytes of padding, so that we could go

Re: [PATCH] block/backup: drop dead code from backup_job_create

2019-10-18 Thread Kevin Wolf
Am 17.10.2019 um 16:21 hat Vladimir Sementsov-Ogievskiy geschrieben: > After commit 00e30f05de1d195, there is no more "goto error" points > after job creation, so after "error:" @job is always NULL and we don't > need roll-back job creation. > > Reported-by: Coverity (CID 1406402) >

Re: [PATCH v2] doc: Describe missing generic -blockdev options

2019-10-18 Thread Peter Maydell
On Fri, 18 Oct 2019 at 13:02, Kevin Wolf wrote: > > We added more generic options after introducing -blockdev and forgot to > update the documentation (man page and --help output) accordingly. Do > that now. > > Signed-off-by: Kevin Wolf > --- > qemu-options.hx | 22 +- > 1

Re: [PULL v3 00/19] Bitmaps patches

2019-10-18 Thread Peter Maydell
On Thu, 17 Oct 2019 at 22:54, John Snow wrote: > > The following changes since commit f22f553efffd083ff624be116726f843a39f1148: > > Merge remote-tracking branch 'remotes/rth/tags/pull-tcg-20191013' into > staging (2019-10-17 16:48:56 +0100) > > are available in the Git repository at: > >

Re: [PATCH] qemu-img.texi: Describe data_file and data_file_raw

2019-10-18 Thread Eric Blake
[cc qemu-block] On 10/18/19 4:59 AM, Han Han wrote: https://bugzilla.redhat.com/show_bug.cgi?id=1763105 Signed-off-by: Han Han --- qemu-img.texi | 10 ++ 1 file changed, 10 insertions(+) Reviewed-by: Eric Blake diff --git a/qemu-img.texi b/qemu-img.texi index

Re: [PATCH] iotests: Skip read-only cases in 118 when run as root

2019-10-18 Thread Kevin Wolf
Am 18.10.2019 um 14:59 hat Philippe Mathieu-Daudé geschrieben: > Hi Kevin, > > On 10/18/19 1:51 PM, Kevin Wolf wrote: > > Some tests in 118 use chmod to remove write permissions from the file > > and assume that the image can indeed not be opened read-write > > afterwards. This doesn't work when

Re: [PATCH v8 1/3] docs: improve qcow2 spec about extending image header

2019-10-18 Thread Eric Blake
On 10/18/19 4:47 AM, Vladimir Sementsov-Ogievskiy wrote: Make it more obvious how to add new fields to the version 3 header and how to interpret them. The specification is adjusted so for new defined optional fields: The specification is adjusted to make it clear that future fields are

[PATCH v8 3/3] docs: qcow2: introduce compression type feature

2019-10-18 Thread Vladimir Sementsov-Ogievskiy
The patch add new additional field to qcow2 header: compression_type, which specifies compression type. If field is absent or zero, default compression type is set: ZLIB, which corresponds to current behavior. New compression type (ZSTD) is to be added in further commit. Suggested-by: Denis

[PATCH v8 0/3] qcow2: add zstd cluster compression

2019-10-18 Thread Vladimir Sementsov-Ogievskiy
Hi all! Here is my proposal, about how to correctly update qcow2 specification to introduce new field, keeping in mind currently existing images and downstream Qemu instances. v8: Add padding, and clarify "zero equals absence" concept. Move some points to commit message from the spec itself.

[PATCH v2] doc: Describe missing generic -blockdev options

2019-10-18 Thread Kevin Wolf
We added more generic options after introducing -blockdev and forgot to update the documentation (man page and --help output) accordingly. Do that now. Signed-off-by: Kevin Wolf --- qemu-options.hx | 22 +- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git

Re: [PATCH v7 1/2] docs: improve qcow2 spec about extending image header

2019-10-18 Thread Vladimir Sementsov-Ogievskiy
18.10.2019 16:39, Eric Blake wrote: > On 10/18/19 4:27 AM, Vladimir Sementsov-Ogievskiy wrote: > > I would suggest a stronger requirement: > > header_length must be a multiple of 4, and must not land in the middle of > any optional 8-byte field. > > Or maybe even add our

Re: [PATCH] iotests: Skip read-only cases in 118 when run as root

2019-10-18 Thread Max Reitz
On 18.10.19 16:27, Kevin Wolf wrote: > Am 18.10.2019 um 14:59 hat Philippe Mathieu-Daudé geschrieben: >> Hi Kevin, >> >> On 10/18/19 1:51 PM, Kevin Wolf wrote: >>> Some tests in 118 use chmod to remove write permissions from the file >>> and assume that the image can indeed not be opened

Re: [PATCH 08/10] nbd/server: introduce NBDExtentArray

2019-10-18 Thread Vladimir Sementsov-Ogievskiy
09.10.2019 20:02, Eric Blake wrote: > On 9/30/19 10:15 AM, Vladimir Sementsov-Ogievskiy wrote: >> Introduce NBDExtentArray class, to handle extents list creation in more >> controlled way and with less OUT parameters in functions. >> >> Signed-off-by: Vladimir Sementsov-Ogievskiy >> --- >>  

Re: [PATCH 08/10] nbd/server: introduce NBDExtentArray

2019-10-18 Thread Eric Blake
On 10/18/19 11:07 AM, Vladimir Sementsov-Ogievskiy wrote:   static int nbd_co_send_extents(NBDClient *client, uint64_t handle, -   NBDExtent *extents, unsigned int nb_extents, -   uint64_t length, bool last, -  

Re: [PATCH] iotests: Skip read-only cases in 118 when run as root

2019-10-18 Thread Kevin Wolf
Am 18.10.2019 um 17:00 hat Max Reitz geschrieben: > On 18.10.19 16:27, Kevin Wolf wrote: > > Am 18.10.2019 um 14:59 hat Philippe Mathieu-Daudé geschrieben: > >> Hi Kevin, > >> > >> On 10/18/19 1:51 PM, Kevin Wolf wrote: > >>> Some tests in 118 use chmod to remove write permissions from the file >

Re: [PATCH] iotests: Remove 130 from the "auto" group

2019-10-18 Thread Bruce Rogers
On Fri, 2019-10-18 at 18:10 +0200, Thomas Huth wrote: > Peter hit a "Could not open 'TEST_DIR/t.IMGFMT': Failed to get shared > 'write' lock - Is another process using the image > [TEST_DIR/t.IMGFMT]?" > error with 130 already twice. Looks like this test is a little bit > shaky, and currently

Re: [PATCH v8 1/3] docs: improve qcow2 spec about extending image header

2019-10-18 Thread Vladimir Sementsov-Ogievskiy
18.10.2019 17:00, Eric Blake wrote: > On 10/18/19 4:47 AM, Vladimir Sementsov-Ogievskiy wrote: >> Make it more obvious how to add new fields to the version 3 header and >> how to interpret them. >> >> The specification is adjusted so for new defined optional fields: > > The specification is

Re: [PATCH] iotests: Skip read-only cases in 118 when run as root

2019-10-18 Thread Philippe Mathieu-Daudé
On 10/18/19 4:27 PM, Kevin Wolf wrote: Am 18.10.2019 um 14:59 hat Philippe Mathieu-Daudé geschrieben: Hi Kevin, On 10/18/19 1:51 PM, Kevin Wolf wrote: Some tests in 118 use chmod to remove write permissions from the file and assume that the image can indeed not be opened read-write

Re: [PATCH v2 1/5] hbitmap: handle set/reset with zero length

2019-10-18 Thread Max Reitz
On 11.10.19 11:07, Vladimir Sementsov-Ogievskiy wrote: > Passing zero length to these functions leads to unpredicted results. > Zero-length set/reset may occur in active-mirror, on zero-length write > (which is unlikely, but not guaranteed to never happen). > > Let's just do nothing on

Re: [PATCH v2 4/5] block/mirror: support unaligned write in active mirror

2019-10-18 Thread Max Reitz
On 11.10.19 11:07, Vladimir Sementsov-Ogievskiy wrote: > Prior 9adc1cb49af8d do_sync_target_write had a bug: it reset aligned-up > region in the dirty bitmap, which means that we may not copy some bytes > and assume them copied, which actually leads to producing corrupted > target. > > So

[PATCH] iotests: Remove 130 from the "auto" group

2019-10-18 Thread Thomas Huth
Peter hit a "Could not open 'TEST_DIR/t.IMGFMT': Failed to get shared 'write' lock - Is another process using the image [TEST_DIR/t.IMGFMT]?" error with 130 already twice. Looks like this test is a little bit shaky, and currently nobody has a real clue what could be causing this issue, so for the

Re: [PATCH 4/5] iotests: Skip "make check-block" if QEMU does not support virtio-blk

2019-10-18 Thread Max Reitz
On 11.10.19 16:50, Thomas Huth wrote: > The next patch is going to add some python-based tests to the "auto" > group, and these tests require virtio-blk to work properly. Running > iotests without virtio-blk likely does not make too much sense anyway, > so instead of adding a check for the

Re: [PATCH] blockdev: modify blockdev-change-medium to change non-removable device

2019-10-18 Thread Eric Blake
On 10/18/19 7:09 AM, Denis Plotnikov wrote: The modification is useful to workaround exclusive file access restrictions, e.g. to implement VM migration with shared disk stored on a storage with the exclusive file opening model: a destination VM is started waiting for incomming migration with a

Re: [PATCH v2 00/23] iotests: Add and use $SOCK_DIR

2019-10-18 Thread Max Reitz
On 17.10.19 15:31, Max Reitz wrote: > Hi, > > Perhaps the main reason we cannot run important tests such as 041 in CI > is that when they care Unix sockets in $TEST_DIR, the path may become > too long to connect to them. > > To get by this problem, this series lets the check script create a new

Re: [PATCH 5/5] iotests: Enable more tests in the 'auto' group to improve test coverage

2019-10-18 Thread Max Reitz
On 11.10.19 16:50, Thomas Huth wrote: > According to Kevin, tests 030, 040 and 041 are among the most valuable > tests that we have, so we should always run them if possible, even if > they take a little bit longer. > > According to Max, it would be good to have a test for iothreads and >

Re: [PATCH v8 2/3] docs: define padding for qcow2 header

2019-10-18 Thread Vladimir Sementsov-Ogievskiy
18.10.2019 17:02, Eric Blake wrote: > On 10/18/19 4:47 AM, Vladimir Sementsov-Ogievskiy wrote: >> Header extensions ends are already defined to be multiply of 8. Let's >> gently ask for header length to be a multiply of 8 too, when we have >> some additional fields. Requiring this may be

Re: [PATCH v8 3/3] docs: qcow2: introduce compression type feature

2019-10-18 Thread Vladimir Sementsov-Ogievskiy
18.10.2019 17:04, Eric Blake wrote: > On 10/18/19 4:47 AM, Vladimir Sementsov-Ogievskiy wrote: >> The patch add new additional field to qcow2 header: compression_type, >> which specifies compression type. If field is absent or zero, default >> compression type is set: ZLIB, which corresponds to

Re: [PATCH] blockdev: modify blockdev-change-medium to change non-removable device

2019-10-18 Thread Max Reitz
On 18.10.19 14:09, Denis Plotnikov wrote: > The modification is useful to workaround exclusive file access restrictions, > e.g. to implement VM migration with shared disk stored on a storage with > the exclusive file opening model: a destination VM is started waiting for > incomming migration with

Re: [PATCH 2/5] iotests: Test 041 does not work on macOS

2019-10-18 Thread Max Reitz
On 11.10.19 16:50, Thomas Huth wrote: > 041 works fine on Linux, FreeBSD and OpenBSD, so let's mark it as > only supported on these systems. > > Signed-off-by: Thomas Huth > --- > tests/qemu-iotests/041 | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git

Re: [PATCH v2 0/5] active-mirror: support unaligned guest operations

2019-10-18 Thread Max Reitz
On 11.10.19 11:07, Vladimir Sementsov-Ogievskiy wrote: > Commit 9adc1cb49af8d fixed a bug about unaligned (to dirty bitmap > granularity) guest writes (and discards) by simply requesting > corresponding alignment on mirror-top filter. However forcing large > alignment obviously decreases

Re: [PATCH v2 0/5] active-mirror: support unaligned guest operations

2019-10-18 Thread Vladimir Sementsov-Ogievskiy
18.10.2019 18:54, Max Reitz wrote: > On 11.10.19 11:07, Vladimir Sementsov-Ogievskiy wrote: >> Commit 9adc1cb49af8d fixed a bug about unaligned (to dirty bitmap >> granularity) guest writes (and discards) by simply requesting >> corresponding alignment on mirror-top filter. However forcing large

Re: [PATCH 1/5] iotests: remove 'linux' from default supported platforms

2019-10-18 Thread Max Reitz
On 11.10.19 16:50, Thomas Huth wrote: > From: John Snow > > verify_platform will check an explicit whitelist and blacklist instead. > The default will now be assumed to be allowed to run anywhere. > > For tests that do not specify their platforms explicitly, this has the effect > of > enabling

Re: [PATCH v2 1/3] iotests: Fix 173

2019-10-18 Thread Eric Blake
On 10/17/19 7:17 AM, Max Reitz wrote: Why haven't we noticed the failure? Because the test rarely gets run: './check -qcow2 173' is insufficient (that defaults to using file protocol) './check -nfs 173' is insufficient (that defaults to using raw format) so the test is only run with: ./check

Re: [PATCH] iotests: Remove 130 from the "auto" group

2019-10-18 Thread John Snow
On 10/18/19 12:10 PM, Thomas Huth wrote: > Peter hit a "Could not open 'TEST_DIR/t.IMGFMT': Failed to get shared > 'write' lock - Is another process using the image [TEST_DIR/t.IMGFMT]?" > error with 130 already twice. Looks like this test is a little bit > shaky, and currently nobody has a

Re: [PATCH v6 1/4] block/replication.c: Ignore requests after failover

2019-10-18 Thread Lukas Straub
On Sat, 5 Oct 2019 15:05:23 +0200 Lukas Straub wrote: > After failover the Secondary side of replication shouldn't change state, > because > it now functions as our primary disk. > > In replication_start, replication_do_checkpoint, replication_stop, ignore > the request if current state is

Re: [PATCH] qemu-img.texi: Describe data_file and data_file_raw

2019-10-18 Thread John Snow
CC qemu-block On 10/18/19 5:59 AM, Han Han wrote: > https://bugzilla.redhat.com/show_bug.cgi?id=1763105 > > Signed-off-by: Han Han > --- > qemu-img.texi | 10 ++ > 1 file changed, 10 insertions(+) > > diff --git a/qemu-img.texi b/qemu-img.texi > index b5156d6316..44596c2d93 100644 >

qemu crashing when attaching an ISO file to a virtio-scsi CD-ROM device through libvirt

2019-10-18 Thread Fernando Casas Schössow
Hi, Today while working with two different Windows Server 2012 R2 guests I found that when I try to attach an ISO file to a SCSI CD-ROM device through libvirt (virsh or virt-manager) while the guest is running, qemu crashes and the following message is logged: Assertion failed:

qemu crashing when attaching an ISO file to a virtio-scsi CD-ROM device through libvirt

2019-10-18 Thread Fernando Casas Schössow
Hi, Today while working with two different Windows Server 2012 R2 guests I found that when I try to attach an ISO file to a SCSI CD-ROM device through libvirt (virsh or virt-manager) while the guest is running, qemu crashes and the following message is logged: Assertion failed: