Re: [Qemu-block] [PATCH RFC 0/3] block: Backdoor to skip image locking in qemu-io/qemu-img

2017-04-13 Thread Fam Zheng
On Thu, 04/13 21:14, John Snow wrote: > > > On 03/13/2017 10:39 PM, Fam Zheng wrote: > > Hi Kevin, > > > > This tries to capture your idea to add a explicit command option in qemu-io > > and > > qemu-img for user to specify "expect inconsistent data when reading image". > > It's stored as a

Re: [Qemu-block] [PATCH RFC 0/3] block: Backdoor to skip image locking in qemu-io/qemu-img

2017-04-13 Thread John Snow
On 03/13/2017 10:39 PM, Fam Zheng wrote: > Hi Kevin, > > This tries to capture your idea to add a explicit command option in qemu-io > and > qemu-img for user to specify "expect inconsistent data when reading image". > It's stored as a BDRV_O_ flag internally, I don't know if there is a

Re: [Qemu-block] [PATCH for-2.10] Revert "block/io: Comment out permission assertions"

2017-04-13 Thread John Snow
On 04/13/2017 10:34 AM, Max Reitz wrote: > On 13.04.2017 15:28, Stefan Hajnoczi wrote: >> On Tue, Apr 11, 2017 at 05:52:26PM +0200, Max Reitz wrote: >>> This reverts commit e3e0003a8f6570aba1421ef99a0b383a43371a74. >>> >>> This commit was necessary for the 2.9 release because we were unable to

Re: [Qemu-block] [Qemu-devel] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-13 Thread John Snow
On 04/13/2017 06:19 AM, Denis V. Lunev wrote: > On 04/13/2017 12:44 PM, Kevin Wolf wrote: >> Am 12.04.2017 um 21:02 hat Denis V. Lunev geschrieben: >>> On 04/12/2017 09:20 PM, Eric Blake wrote: On 04/12/2017 12:55 PM, Denis V. Lunev wrote: > Let me rephrase a bit. > > The

Re: [Qemu-block] [PATCH v6] migration/block: use blk_pwrite_zeroes for each zero cluster

2017-04-13 Thread 858585 jemmy
On Thu, Apr 13, 2017 at 10:16 PM, Stefan Hajnoczi wrote: > On Thu, Apr 13, 2017 at 10:34:28AM +0800, jemmy858...@gmail.com wrote: >> From: Lidong Chen >> >> BLOCK_SIZE is (1 << 20), qcow2 cluster size is 65536 by default, >> this may cause the qcow2

Re: [Qemu-block] [PATCH for-2.10 3/3] qemu-img/convert: Move bs_n > 1 && -B check down

2017-04-13 Thread Eric Blake
On 04/13/2017 03:34 PM, Max Reitz wrote: > It does not make much sense to use a backing image for the target when > you concatenate multiple images (because then there is no correspondence > between the source images' backing files and the target's); but it was > still possible to give one by

Re: [Qemu-block] [PATCH for-2.10 2/3] qemu-img/convert: Use @opts for one thing only

2017-04-13 Thread Eric Blake
On 04/13/2017 03:34 PM, Max Reitz wrote: > After storing the creation options for the new image into @opts, we > fetch some things for our own information, like the backing file name, > or whether to use encryption or preallocation. > > With the -n parameter, there will not be any creation

Re: [Qemu-block] [PATCH] progress: Show current progress on SIGINFO

2017-04-13 Thread John Snow
On 04/13/2017 04:38 PM, Max Reitz wrote: > On 08.02.2017 00:57, Max Reitz wrote: >> Currently we only print progress information on retrieval of SIGUSR1. >> Some systems have a dedicated SIGINFO for this, however, so it should be >> handled appropriately if it is available. >> >> Buglink:

Re: [Qemu-block] [PATCH for-2.10 1/3] qemu-img/convert: Always set ret < 0 on error

2017-04-13 Thread Eric Blake
On 04/13/2017 03:33 PM, Max Reitz wrote: > Otherwise the qemu-img process will exit with EXIT_SUCCESS instead of > EXIT_FAILURE. > > Cc: qemu-stable > Signed-off-by: Max Reitz > --- > qemu-img.c | 2 ++ > 1 file changed, 2 insertions(+) Reviewed-by:

Re: [Qemu-block] [PATCH] progress: Show current progress on SIGINFO

2017-04-13 Thread Max Reitz
On 08.02.2017 00:57, Max Reitz wrote: > Currently we only print progress information on retrieval of SIGUSR1. > Some systems have a dedicated SIGINFO for this, however, so it should be > handled appropriately if it is available. > > Buglink: https://bugs.launchpad.net/qemu/+bug/1662468 >

[Qemu-block] [PATCH for-2.10 2/3] qemu-img/convert: Use @opts for one thing only

2017-04-13 Thread Max Reitz
After storing the creation options for the new image into @opts, we fetch some things for our own information, like the backing file name, or whether to use encryption or preallocation. With the -n parameter, there will not be any creation options; this is not too bad because this just means that

[Qemu-block] [PATCH for-2.10 3/3] qemu-img/convert: Move bs_n > 1 && -B check down

2017-04-13 Thread Max Reitz
It does not make much sense to use a backing image for the target when you concatenate multiple images (because then there is no correspondence between the source images' backing files and the target's); but it was still possible to give one by using -o backing_file=X instead of -B X. Fix this by

[Qemu-block] [PATCH for-2.10 1/3] qemu-img/convert: Always set ret < 0 on error

2017-04-13 Thread Max Reitz
Otherwise the qemu-img process will exit with EXIT_SUCCESS instead of EXIT_FAILURE. Cc: qemu-stable Signed-off-by: Max Reitz --- qemu-img.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/qemu-img.c b/qemu-img.c index 37c2894678..f2809e1ab4

[Qemu-block] [PATCH for-2.10 0/3] qemu-img/convert: Some small fixes

2017-04-13 Thread Max Reitz
This series fixes some small issues I found in qemu-img's convert subcommand while reviewing Dan's convert/dd series. Nothing serious at all, so all for 2.10. Max Reitz (3): qemu-img/convert: Always set ret < 0 on error qemu-img/convert: Use @opts for one thing only qemu-img/convert: Move

Re: [Qemu-block] [PATCH v4 4/4] qemu-img: copy *key-secret opts when opening newly created files

2017-04-13 Thread Max Reitz
On 12.04.2017 18:44, Daniel P. Berrange wrote: > The qemu-img dd/convert commands will create a image file and > then try to open it. Historically it has been possible to open > new files without passing any options. With encrypted files > though, the *key-secret options are mandatory, so we need

Re: [Qemu-block] [PATCH v4 3/4] qemu-img: introduce --target-image-opts for 'convert' command

2017-04-13 Thread Max Reitz
On 12.04.2017 18:44, Daniel P. Berrange wrote: > The '--image-opts' flags indicates whether the source filename > includes options. The target filename has to remain in the > plain filename format though, since it needs to be passed to > bdrv_create(). When using --skip-create though, it would be

Re: [Qemu-block] [Qemu-devel] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-13 Thread Denis V. Lunev
On 04/13/2017 04:05 PM, Kevin Wolf wrote: > Am 13.04.2017 um 14:44 hat Denis V. Lunev geschrieben: >> On 04/13/2017 02:58 PM, Alberto Garcia wrote: >>> On Wed 12 Apr 2017 06:54:50 PM CEST, Denis V. Lunev wrote: My opinion about this approach is very negative as the problem could be

Re: [Qemu-block] [PATCH v4 1/4] qemu-img: add support for --object with 'dd' command

2017-04-13 Thread Max Reitz
On 12.04.2017 18:44, Daniel P. Berrange wrote: > The qemu-img dd command added --image-opts support, but missed > the corresponding --object support. This prevented passing > secrets (eg auth passwords) needed by certain disk images. > > Reviewed-by: Eric Blake >

Re: [Qemu-block] [Qemu-devel] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-13 Thread Denis V. Lunev
On 04/13/2017 12:44 PM, Kevin Wolf wrote: > Am 12.04.2017 um 21:02 hat Denis V. Lunev geschrieben: >> On 04/12/2017 09:20 PM, Eric Blake wrote: >>> On 04/12/2017 12:55 PM, Denis V. Lunev wrote: Let me rephrase a bit. The proposal is looking very close to the following case: -

Re: [Qemu-block] [PATCH 3/3] qemu-iotests: Test postcopy migration

2017-04-13 Thread Eric Blake
On 04/13/2017 12:23 PM, Kevin Wolf wrote: > Signed-off-by: Kevin Wolf > --- > tests/qemu-iotests/181 | 117 > + > tests/qemu-iotests/181.out | 38 +++ > tests/qemu-iotests/group | 1 + > 3 files changed, 156

Re: [Qemu-block] [PATCH 1/3] migration: Call blk_resume_after_migration() for postcopy

2017-04-13 Thread Kevin Wolf
Am 13.04.2017 um 20:03 hat Eric Blake geschrieben: > On 04/13/2017 12:54 PM, Kevin Wolf wrote: > > Am 13.04.2017 um 19:39 hat Eric Blake geschrieben: > >> On 04/13/2017 12:23 PM, Kevin Wolf wrote: > >>> Commit d35ff5e6 ('block: Ignore guest dev permissions during incoming > >>> migration') added

Re: [Qemu-block] [PATCH 1/3] migration: Call blk_resume_after_migration() for postcopy

2017-04-13 Thread Eric Blake
On 04/13/2017 12:54 PM, Kevin Wolf wrote: > Am 13.04.2017 um 19:39 hat Eric Blake geschrieben: >> On 04/13/2017 12:23 PM, Kevin Wolf wrote: >>> Commit d35ff5e6 ('block: Ignore guest dev permissions during incoming >>> migration') added blk_resume_after_migration() to the precopy migration >>>

Re: [Qemu-block] [Qemu-devel] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-13 Thread Denis V. Lunev
On 04/13/2017 04:21 PM, Alberto Garcia wrote: > On Thu 13 Apr 2017 02:44:51 PM CEST, Denis V. Lunev wrote: 1) current L2 cache management seems very wrong to me. Each cache miss means that we have to read entire L2 cache block. This means that in the worst case (when dataset

Re: [Qemu-block] [PATCH 1/3] migration: Call blk_resume_after_migration() for postcopy

2017-04-13 Thread Kevin Wolf
Am 13.04.2017 um 19:39 hat Eric Blake geschrieben: > On 04/13/2017 12:23 PM, Kevin Wolf wrote: > > Commit d35ff5e6 ('block: Ignore guest dev permissions during incoming > > migration') added blk_resume_after_migration() to the precopy migration > > path, but neglected to add it to the duplicated

Re: [Qemu-block] [Qemu-devel] [PATCH 2/3] qemu-iotests: Filter HMP readline escape characters

2017-04-13 Thread Eric Blake
On 04/13/2017 12:38 PM, Eric Blake wrote: > On 04/13/2017 12:23 PM, Kevin Wolf wrote: >> The only thing the escape characters achieve is making the reference >> output unreadable and lines that are potentially so long that git >> doesn't want to put them into an email any more. Let's filter them

Re: [Qemu-block] [PATCH 2/3] qemu-iotests: Filter HMP readline escape characters

2017-04-13 Thread Eric Blake
On 04/13/2017 12:23 PM, Kevin Wolf wrote: > The only thing the escape characters achieve is making the reference > output unreadable and lines that are potentially so long that git > doesn't want to put them into an email any more. Let's filter them out. Hear! Hear! > > Signed-off-by: Kevin

Re: [Qemu-block] [PATCH 1/3] migration: Call blk_resume_after_migration() for postcopy

2017-04-13 Thread Eric Blake
On 04/13/2017 12:23 PM, Kevin Wolf wrote: > Commit d35ff5e6 ('block: Ignore guest dev permissions during incoming > migration') added blk_resume_after_migration() to the precopy migration > path, but neglected to add it to the duplicated code that is used for > postcopy migration. This means that

[Qemu-block] [PATCH 2/3] qemu-iotests: Filter HMP readline escape characters

2017-04-13 Thread Kevin Wolf
The only thing the escape characters achieve is making the reference output unreadable and lines that are potentially so long that git doesn't want to put them into an email any more. Let's filter them out. Signed-off-by: Kevin Wolf --- tests/qemu-iotests/028.out | 2 +-

[Qemu-block] [PATCH 0/3] migration: Call blk_resume_after_migration() for postcopy

2017-04-13 Thread Kevin Wolf
This fixes one of the bugs that made us disable the op blocker assertions for the 2.9 release. Kevin Wolf (3): migration: Call blk_resume_after_migration() for postcopy qemu-iotests: Filter HMP readline escape characters qemu-iotests: Test postcopy migration migration/savevm.c

[Qemu-block] [PATCH 3/3] qemu-iotests: Test postcopy migration

2017-04-13 Thread Kevin Wolf
Signed-off-by: Kevin Wolf --- tests/qemu-iotests/181 | 117 + tests/qemu-iotests/181.out | 38 +++ tests/qemu-iotests/group | 1 + 3 files changed, 156 insertions(+) create mode 100755 tests/qemu-iotests/181

[Qemu-block] [PATCH 1/3] migration: Call blk_resume_after_migration() for postcopy

2017-04-13 Thread Kevin Wolf
Commit d35ff5e6 ('block: Ignore guest dev permissions during incoming migration') added blk_resume_after_migration() to the precopy migration path, but neglected to add it to the duplicated code that is used for postcopy migration. This means that the guest device doesn't request the necessary

Re: [Qemu-block] [Qemu-devel] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-13 Thread Roman Kagan
On Thu, Apr 13, 2017 at 04:27:35PM +0200, Kevin Wolf wrote: > Am 13.04.2017 um 16:15 hat Alberto Garcia geschrieben: > > On Thu 13 Apr 2017 03:51:55 PM CEST, Kevin Wolf wrote: > > >> This invariant is already broken by the very design of the qcow2 > > >> format, subclusters don't really add

Re: [Qemu-block] [Qemu-devel] Regression from 2.8: stuck in bdrv_drain()

2017-04-13 Thread John Snow
On 04/13/2017 11:02 AM, Jeff Cody wrote: > On Thu, Apr 13, 2017 at 03:39:59PM +0100, Stefan Hajnoczi wrote: >> On Thu, Apr 13, 2017 at 01:45:55PM +0800, Paolo Bonzini wrote: >>> >>> >>> On 13/04/2017 09:11, Jeff Cody wrote: > It didn't make it into 2.9-rc4 because of limited time. :( >

Re: [Qemu-block] [Qemu-devel] [PATCH 03/12] dirty-bitmap: Drop unused functions

2017-04-13 Thread John Snow
On 04/13/2017 05:19 AM, Vladimir Sementsov-Ogievskiy wrote: > 13.04.2017 02:40, John Snow wrote: >> >> On 04/12/2017 07:36 PM, Eric Blake wrote: >>> On 04/12/2017 05:47 PM, John Snow wrote: On 04/12/2017 01:49 PM, Eric Blake wrote: > We had several functions that no one was using,

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.9?] block: Do not unref bs->file on error in BD's open

2017-04-13 Thread Eric Blake
On 04/13/2017 10:43 AM, Max Reitz wrote: > The block layer takes care of removing the bs->file child if the block > driver's bdrv_open()/bdrv_file_open() implementation fails. The block > driver therefore does not need to do so, and indeed should not unless it > sets bs->file to NULL afterwards --

Re: [Qemu-block] [PATCH for-2.10 0/2] block: An empty filename counts as no filename

2017-04-13 Thread Eric Blake
On 04/13/2017 11:06 AM, Max Reitz wrote: > See patch 1 for what is happening when you try "qemu-img info ''" and > for the fix. > > (Patch 2 just adds a simple test.) > > > Max Reitz (2): > block: An empty filename counts as no filename > iotests/051: Add test for empty filename

[Qemu-block] [PATCH for-2.10 1/2] block: An empty filename counts as no filename

2017-04-13 Thread Max Reitz
Reproducer: $ ./qemu-img info '' qemu-img: ./block.c:1008: bdrv_open_driver: Assertion `!drv->bdrv_needs_filename || bs->filename[0]' failed. [1]26105 abort (core dumped) ./qemu-img info '' This patch fixes this to be: $ ./qemu-img info '' qemu-img: Could not open

[Qemu-block] [PATCH for-2.10 2/2] iotests/051: Add test for empty filename

2017-04-13 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/051| 1 + tests/qemu-iotests/051.out| 3 +++ tests/qemu-iotests/051.pc.out | 3 +++ 3 files changed, 7 insertions(+) diff --git a/tests/qemu-iotests/051 b/tests/qemu-iotests/051 index 630cb7a114..4fe676019b 100755

[Qemu-block] [PATCH for-2.10 0/2] block: An empty filename counts as no filename

2017-04-13 Thread Max Reitz
See patch 1 for what is happening when you try "qemu-img info ''" and for the fix. (Patch 2 just adds a simple test.) Max Reitz (2): block: An empty filename counts as no filename iotests/051: Add test for empty filename block.c | 2 +- tests/qemu-iotests/051

[Qemu-block] [PATCH for-2.10?] block: Do not unref bs->file on error in BD's open

2017-04-13 Thread Max Reitz
The block layer takes care of removing the bs->file child if the block driver's bdrv_open()/bdrv_file_open() implementation fails. The block driver therefore does not need to do so, and indeed should not unless it sets bs->file to NULL afterwards -- because if this is not done, the

Re: [Qemu-block] [Qemu-devel] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-13 Thread Denis V. Lunev
On 04/13/2017 06:04 PM, Alberto Garcia wrote: > On Thu 13 Apr 2017 03:30:43 PM CEST, Denis V. Lunev wrote: >> Yes, block size should be increased. I perfectly in agreement with >> your. But I think that we could do that by plain increase of the >> cluster size without any further dances.

Re: [Qemu-block] [Qemu-devel] Regression from 2.8: stuck in bdrv_drain()

2017-04-13 Thread Stefan Hajnoczi
On Thu, Apr 13, 2017 at 6:45 AM, Paolo Bonzini wrote: > On 13/04/2017 09:11, Jeff Cody wrote: >>> It didn't make it into 2.9-rc4 because of limited time. :( >>> >>> Looks like there is no -rc5, we'll have to document this as a known issue. >>> Users should

Re: [Qemu-block] [Qemu-devel] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-13 Thread Alberto Garcia
On Thu 13 Apr 2017 03:30:43 PM CEST, Denis V. Lunev wrote: > Yes, block size should be increased. I perfectly in agreement with > your. But I think that we could do that by plain increase of the > cluster size without any further dances. Sub-clusters as sub-clusters > will help if we are able to

Re: [Qemu-block] [Qemu-devel] Regression from 2.8: stuck in bdrv_drain()

2017-04-13 Thread Jeff Cody
On Thu, Apr 13, 2017 at 03:39:59PM +0100, Stefan Hajnoczi wrote: > On Thu, Apr 13, 2017 at 01:45:55PM +0800, Paolo Bonzini wrote: > > > > > > On 13/04/2017 09:11, Jeff Cody wrote: > > >> It didn't make it into 2.9-rc4 because of limited time. :( > > >> > > >> Looks like there is no -rc5, we'll

Re: [Qemu-block] [Qemu-devel] Regression from 2.8: stuck in bdrv_drain()

2017-04-13 Thread Peter Maydell
On 13 April 2017 at 15:33, Eric Blake wrote: > On 04/13/2017 04:48 AM, Peter Maydell wrote: >> Well, we *hope* there is no -rc5, but if the bug is genuinely >> a "we can't release like this" bug we will obviously have to >> do another rc. Basically you all as the block

Re: [Qemu-block] [Qemu-devel] Regression from 2.8: stuck in bdrv_drain()

2017-04-13 Thread Jeff Cody
On Thu, Apr 13, 2017 at 09:45:49AM -0500, Eric Blake wrote: > On 04/13/2017 09:39 AM, Stefan Hajnoczi wrote: > > On Thu, Apr 13, 2017 at 01:45:55PM +0800, Paolo Bonzini wrote: > >> > >> > >> On 13/04/2017 09:11, Jeff Cody wrote: > It didn't make it into 2.9-rc4 because of limited time. :( >

Re: [Qemu-block] [Qemu-devel] Regression from 2.8: stuck in bdrv_drain()

2017-04-13 Thread Eric Blake
On 04/13/2017 09:39 AM, Stefan Hajnoczi wrote: > On Thu, Apr 13, 2017 at 01:45:55PM +0800, Paolo Bonzini wrote: >> >> >> On 13/04/2017 09:11, Jeff Cody wrote: It didn't make it into 2.9-rc4 because of limited time. :( Looks like there is no -rc5, we'll have to document this as a

Re: [Qemu-block] [Qemu-devel] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-13 Thread Denis V. Lunev
On 04/13/2017 04:51 PM, Kevin Wolf wrote: > Am 13.04.2017 um 15:21 hat Alberto Garcia geschrieben: >> This invariant is already broken by the very design of the qcow2 format, >> subclusters don't really add anything new there. For any given cluster >> size you can write 4k in every odd cluster,

Re: [Qemu-block] [Qemu-devel] Regression from 2.8: stuck in bdrv_drain()

2017-04-13 Thread Stefan Hajnoczi
On Thu, Apr 13, 2017 at 01:45:55PM +0800, Paolo Bonzini wrote: > > > On 13/04/2017 09:11, Jeff Cody wrote: > >> It didn't make it into 2.9-rc4 because of limited time. :( > >> > >> Looks like there is no -rc5, we'll have to document this as a known issue. > >> Users should

Re: [Qemu-block] [Qemu-devel] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-13 Thread Denis V. Lunev
On 04/13/2017 04:36 PM, Alberto Garcia wrote: > On Thu 13 Apr 2017 03:09:53 PM CEST, Denis V. Lunev wrote: For nowadays SSD we are facing problems somewhere else. Right now I can achieve only 100k IOPSes on SSD capable of 350-550k. 1 Mb block with preallocation and fragmented L2

Re: [Qemu-block] [PATCH for-2.10] Revert "block/io: Comment out permission assertions"

2017-04-13 Thread Max Reitz
On 13.04.2017 15:28, Stefan Hajnoczi wrote: > On Tue, Apr 11, 2017 at 05:52:26PM +0200, Max Reitz wrote: >> This reverts commit e3e0003a8f6570aba1421ef99a0b383a43371a74. >> >> This commit was necessary for the 2.9 release because we were unable to >> fix the underlying issue(s) in time. However,

Re: [Qemu-block] [Qemu-devel] Regression from 2.8: stuck in bdrv_drain()

2017-04-13 Thread Eric Blake
On 04/13/2017 04:48 AM, Peter Maydell wrote: > On 13 April 2017 at 00:54, Fam Zheng wrote: >> John and I confirmed that this can be fixed by this pending patch: >> >> [PATCH for-2.9 4/5] block: Drain BH in bdrv_drained_begin >> >>

Re: [Qemu-block] [Qemu-devel] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-13 Thread Kevin Wolf
Am 13.04.2017 um 16:15 hat Alberto Garcia geschrieben: > On Thu 13 Apr 2017 03:51:55 PM CEST, Kevin Wolf wrote: > >> This invariant is already broken by the very design of the qcow2 > >> format, subclusters don't really add anything new there. For any > >> given cluster size you can write 4k in

Re: [Qemu-block] [Qemu-devel] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-13 Thread Denis V. Lunev
On 04/13/2017 02:58 PM, Alberto Garcia wrote: > On Wed 12 Apr 2017 06:54:50 PM CEST, Denis V. Lunev wrote: >> My opinion about this approach is very negative as the problem could >> be (partially) solved in a much better way. > Hmm... it seems to me that (some of) the problems you are describing

Re: [Qemu-block] [PATCH v6] migration/block: use blk_pwrite_zeroes for each zero cluster

2017-04-13 Thread Stefan Hajnoczi
On Thu, Apr 13, 2017 at 10:34:28AM +0800, jemmy858...@gmail.com wrote: > From: Lidong Chen > > BLOCK_SIZE is (1 << 20), qcow2 cluster size is 65536 by default, > this may cause the qcow2 file size to be bigger after migration. > This patch checks each cluster, using

Re: [Qemu-block] [Qemu-devel] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-13 Thread Alberto Garcia
On Thu 13 Apr 2017 03:51:55 PM CEST, Kevin Wolf wrote: >> This invariant is already broken by the very design of the qcow2 >> format, subclusters don't really add anything new there. For any >> given cluster size you can write 4k in every odd cluster, then do the >> same in every even cluster, and

Re: [Qemu-block] [Qemu-devel] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-13 Thread Kevin Wolf
Am 13.04.2017 um 15:30 hat Denis V. Lunev geschrieben: > On 04/13/2017 04:21 PM, Alberto Garcia wrote: > > On Thu 13 Apr 2017 02:44:51 PM CEST, Denis V. Lunev wrote: > 1) current L2 cache management seems very wrong to me. Each cache > miss means that we have to read entire L2 cache

Re: [Qemu-block] [Qemu-devel] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-13 Thread Kevin Wolf
Am 13.04.2017 um 15:21 hat Alberto Garcia geschrieben: > This invariant is already broken by the very design of the qcow2 format, > subclusters don't really add anything new there. For any given cluster > size you can write 4k in every odd cluster, then do the same in every > even cluster, and

Re: [Qemu-block] [Qemu-devel] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-13 Thread Alberto Garcia
On Thu 13 Apr 2017 03:09:53 PM CEST, Denis V. Lunev wrote: >>> For nowadays SSD we are facing problems somewhere else. Right now I >>> can achieve only 100k IOPSes on SSD capable of 350-550k. 1 Mb block >>> with preallocation and fragmented L2 cache gives same 100k. Tests >>> for initially empty

Re: [Qemu-block] [PATCH for-2.10] Revert "block/io: Comment out permission assertions"

2017-04-13 Thread Stefan Hajnoczi
On Tue, Apr 11, 2017 at 05:52:26PM +0200, Max Reitz wrote: > This reverts commit e3e0003a8f6570aba1421ef99a0b383a43371a74. > > This commit was necessary for the 2.9 release because we were unable to > fix the underlying issue(s) in time. However, we will be for 2.10. > > Signed-off-by: Max Reitz

Re: [Qemu-block] [Qemu-devel] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-13 Thread Alberto Garcia
On Thu 13 Apr 2017 02:44:51 PM CEST, Denis V. Lunev wrote: >>> 1) current L2 cache management seems very wrong to me. Each cache >>> miss means that we have to read entire L2 cache block. This means >>> that in the worst case (when dataset of the test does not fit L2 >>> cache size we

Re: [Qemu-block] [Qemu-devel] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-13 Thread Kevin Wolf
Am 13.04.2017 um 14:44 hat Denis V. Lunev geschrieben: > On 04/13/2017 02:58 PM, Alberto Garcia wrote: > > On Wed 12 Apr 2017 06:54:50 PM CEST, Denis V. Lunev wrote: > >> My opinion about this approach is very negative as the problem could > >> be (partially) solved in a much better way. > >

Re: [Qemu-block] [Qemu-devel] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-13 Thread Alberto Garcia
On Wed 12 Apr 2017 06:54:50 PM CEST, Denis V. Lunev wrote: > My opinion about this approach is very negative as the problem could > be (partially) solved in a much better way. Hmm... it seems to me that (some of) the problems you are describing are different from the ones this proposal tries to

Re: [Qemu-block] [Qemu-devel] Regression from 2.8: stuck in bdrv_drain()

2017-04-13 Thread Peter Maydell
On 13 April 2017 at 00:54, Fam Zheng wrote: > John and I confirmed that this can be fixed by this pending patch: > > [PATCH for-2.9 4/5] block: Drain BH in bdrv_drained_begin > > https://lists.gnu.org/archive/html/qemu-devel/2017-04/msg01018.html > > It didn't make it into

Re: [Qemu-block] [Qemu-devel] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-13 Thread Kevin Wolf
Am 12.04.2017 um 21:02 hat Denis V. Lunev geschrieben: > On 04/12/2017 09:20 PM, Eric Blake wrote: > > On 04/12/2017 12:55 PM, Denis V. Lunev wrote: > >> Let me rephrase a bit. > >> > >> The proposal is looking very close to the following case: > >> - raw sparse file > >> > >> In this case all

Re: [Qemu-block] [Qemu-devel] [PATCH 03/12] dirty-bitmap: Drop unused functions

2017-04-13 Thread Vladimir Sementsov-Ogievskiy
13.04.2017 02:40, John Snow wrote: On 04/12/2017 07:36 PM, Eric Blake wrote: On 04/12/2017 05:47 PM, John Snow wrote: On 04/12/2017 01:49 PM, Eric Blake wrote: We had several functions that no one was using, and which used sector-based interfaces. I'm trying to convert towards byte-based

Re: [Qemu-block] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-13 Thread Alberto Garcia
On Thu 13 Apr 2017 11:02:10 AM CEST, Kevin Wolf wrote: > I think you still need it if you don't want to look at the refcount > blocks for every write. When you take an internal snapshot, you just > increase the refcount of the L2 tables at first and keep the contents > the same,

Re: [Qemu-block] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-13 Thread Kevin Wolf
Am 13.04.2017 um 10:05 hat Alberto Garcia geschrieben: > On Wed 12 Apr 2017 04:10:46 PM CEST, Max Reitz wrote: > >> I still don't see why we can always assume OFLAG_COPIED. Before doing > >> the COW one cluster can have references from multiple snapshots, > > > > Yes... > > >

Re: [Qemu-block] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-13 Thread Alberto Garcia
On Wed 12 Apr 2017 04:10:46 PM CEST, Max Reitz wrote: >> I still don't see why we can always assume OFLAG_COPIED. Before doing >> the COW one cluster can have references from multiple snapshots, > > Yes... > >> and OFLAG_COPIED is equally valid in that context. > > In what

Re: [Qemu-block] [PATCH v3 0/2] block: Quiesce old aio context during bdrv_set_aio_context

2017-04-13 Thread Fam Zheng
On Thu, 04/13 14:51, Paolo Bonzini wrote: > > > On 11/04/2017 00:04, Stefan Hajnoczi wrote: > > On Mon, Apr 10, 2017 at 10:26:34AM +0800, Fam Zheng wrote: > >> v3: Use bdrv_parent_drained_begin/end. [Kevin] > >> Do it before releasing new_context. [Stefan] > >> > >> Fam Zheng (2): > >>

Re: [Qemu-block] [PATCH v3 0/2] block: Quiesce old aio context during bdrv_set_aio_context

2017-04-13 Thread Paolo Bonzini
On 11/04/2017 00:04, Stefan Hajnoczi wrote: > On Mon, Apr 10, 2017 at 10:26:34AM +0800, Fam Zheng wrote: >> v3: Use bdrv_parent_drained_begin/end. [Kevin] >> Do it before releasing new_context. [Stefan] >> >> Fam Zheng (2): >> block: Make bdrv_parent_drained_begin/end public >> block: