Re: [Qemu-block] [Qemu-devel] [PATCH 07/12] block/backup: add 'always' bitmap sync policy

2019-06-21 Thread John Snow
On 6/21/19 5:48 PM, Max Reitz wrote: > On 21.06.19 22:58, John Snow wrote: >> >> >> On 6/21/19 9:44 AM, Vladimir Sementsov-Ogievskiy wrote: > > [...] > > Just chiming in on this: > >>> "So on cancel and abort you synchronize bitmap too?" >> >> I will concede that this means that if you ask

Re: [Qemu-block] [Qemu-devel] [PATCH 07/12] block/backup: add 'always' bitmap sync policy

2019-06-21 Thread Max Reitz
On 21.06.19 22:58, John Snow wrote: > > > On 6/21/19 9:44 AM, Vladimir Sementsov-Ogievskiy wrote: [...] Just chiming in on this: >> "So on cancel and abort you synchronize bitmap too?" > > I will concede that this means that if you ask for a bitmap backup with > the 'always' policy and, for

Re: [Qemu-block] [Qemu-devel] [PATCH 06/12] block/dirty-bitmap: add bdrv_dirty_bitmap_claim

2019-06-21 Thread John Snow
On 6/21/19 7:58 AM, Vladimir Sementsov-Ogievskiy wrote: > 20.06.2019 19:36, John Snow wrote: >> >> >> On 6/20/19 12:02 PM, Max Reitz wrote: >>> On 20.06.19 03:03, John Snow wrote: This function can claim an hbitmap to replace its own current hbitmap. In the case that the granularities

Re: [Qemu-block] [Qemu-devel] [PATCH 07/12] block/backup: add 'always' bitmap sync policy

2019-06-21 Thread John Snow
On 6/21/19 9:44 AM, Vladimir Sementsov-Ogievskiy wrote: > 21.06.2019 16:08, Vladimir Sementsov-Ogievskiy wrote: >> 21.06.2019 15:59, Vladimir Sementsov-Ogievskiy wrote: >>> 21.06.2019 15:57, Vladimir Sementsov-Ogievskiy wrote: ^ Home Run! I'm going to reply to all four of these mails at once

Re: [Qemu-block] [Qemu-devel] [PATCH 02/12] block/backup: Add mirror sync mode 'bitmap'

2019-06-21 Thread John Snow
On 6/21/19 7:29 AM, Vladimir Sementsov-Ogievskiy wrote: > 20.06.2019 4:03, John Snow wrote: >> We don't need or want a new sync mode for simple differences in >> semantics. Create a new mode simply named "BITMAP" that is designed to >> make use of the new Bitmap Sync Mode field. >> >> Because

Re: [Qemu-block] [SeaBIOS] [PATCH v3 3/4] geometry: Add boot_lchs_find_*() utility functions

2019-06-21 Thread Kevin O'Connor
On Fri, Jun 21, 2019 at 08:42:28PM +0300, Sam Eiderman wrote: > Sounds reasonable, how do purpose to deal with: > > config BIOS_GEOMETRY > config BOOTORDER > > precompiler optouts? I think you can stick them both under BOOTORDER. That option is only there in case someone wants to reduce the

Re: [Qemu-block] [SeaBIOS] [PATCH v3 3/4] geometry: Add boot_lchs_find_*() utility functions

2019-06-21 Thread Sam Eiderman
Sounds reasonable, how do purpose to deal with: config BIOS_GEOMETRY config BOOTORDER precompiler optouts? If we don’t need any of them we also don’t need to call “get_scsi_devpath", “get_ata_devpath”, “get_pci_dev_path”. I’ll see what can be done. > On 20 Jun 2019, at 17:37, Kevin O'Connor

[Qemu-block] [PATCH] blockjob: drain all job nodes in block_job_drain

2019-06-21 Thread Vladimir Sementsov-Ogievskiy
Instead of draining additional nodes in each job code, let's do it in common block_job_drain, draining just all job's children. It's also a first step to finally get rid of blockjob->blk. Signed-off-by: Vladimir Sementsov-Ogievskiy --- Hi all! As a follow-up for "block: drop bs->job" recently

Re: [Qemu-block] [PATCH v2] blockjob: drain all job nodes in block_job_drain

2019-06-21 Thread Vladimir Sementsov-Ogievskiy
21.06.2019 18:15, Vladimir Sementsov-Ogievskiy wrote: > Instead of draining additional nodes in each job code, let's do it in > common block_job_drain, draining just all job's children. > BlockJobDriver.drain becomes unused, so, drop it at all. > > It's also a first step to finally get rid of

[Qemu-block] [PATCH v2] blockjob: drain all job nodes in block_job_drain

2019-06-21 Thread Vladimir Sementsov-Ogievskiy
Instead of draining additional nodes in each job code, let's do it in common block_job_drain, draining just all job's children. BlockJobDriver.drain becomes unused, so, drop it at all. It's also a first step to finally get rid of blockjob->blk. Signed-off-by: Vladimir Sementsov-Ogievskiy ---

Re: [Qemu-block] [Qemu-devel] [PATCH v8 01/10] escc: introduce a selector for the register bit

2019-06-21 Thread Philippe Mathieu-Daudé
On 6/20/19 12:19 AM, Laurent Vivier wrote: > On Sparc and PowerMac, the bit 0 of the address > selects the register type (control or data) and > bit 1 selects the channel (B or A). > > On m68k Macintosh, the bit 0 selects the channel and > bit 1 the register type. > > This patch introduces a new

Re: [Qemu-block] [PATCH v5 34/42] block: Inline bdrv_co_block_status_from_*()

2019-06-21 Thread Vladimir Sementsov-Ogievskiy
13.06.2019 1:09, Max Reitz wrote: > With bdrv_filtered_rw_bs(), we can easily handle this default filter > behavior in bdrv_co_block_status(). > > blkdebug wants to have an additional assertion, so it keeps its own > implementation, except bdrv_co_block_status_from_file() needs to be > inlined

Re: [Qemu-block] [PATCH 07/12] block/backup: add 'always' bitmap sync policy

2019-06-21 Thread Vladimir Sementsov-Ogievskiy
21.06.2019 16:08, Vladimir Sementsov-Ogievskiy wrote: > 21.06.2019 15:59, Vladimir Sementsov-Ogievskiy wrote: >> 21.06.2019 15:57, Vladimir Sementsov-Ogievskiy wrote: >>> 20.06.2019 4:03, John Snow wrote: This adds an "always" policy for bitmap synchronization. Regardless of if the job

[Qemu-block] [PULL 7/8] ssh: switch from libssh2 to libssh

2019-06-21 Thread Max Reitz
From: Pino Toscano Rewrite the implementation of the ssh block driver to use libssh instead of libssh2. The libssh library has various advantages over libssh2: - easier API for authentication (for example for using ssh-agent) - easier API for known_hosts handling - supports newer types of keys

Re: [Qemu-block] [PATCH v5 33/42] blockdev: Fix active commit choice

2019-06-21 Thread Vladimir Sementsov-Ogievskiy
19.06.2019 18:59, Max Reitz wrote: > On 19.06.19 11:31, Vladimir Sementsov-Ogievskiy wrote: >> 13.06.2019 1:09, Max Reitz wrote: >>> We have to perform an active commit whenever the top node has a parent >>> that has taken the WRITE permission on it. >>> >>> Signed-off-by: Max Reitz >>> --- >>>

[Qemu-block] [PULL 8/8] iotests: Fix 205 for concurrent runs

2019-06-21 Thread Max Reitz
Tests should place their files into the test directory. This includes Unix sockets. 205 currently fails to do so, which prevents it from being run concurrently. Signed-off-by: Max Reitz Message-id: 20190618210238.9524-1-mre...@redhat.com Reviewed-by: Eric Blake Signed-off-by: Max Reitz ---

[Qemu-block] [PULL 6/8] vmdk: Add read-only support for seSparse snapshots

2019-06-21 Thread Max Reitz
From: Sam Eiderman Until ESXi 6.5 VMware used the vmfsSparse format for snapshots (VMDK3 in QEMU). This format was lacking in the following: * Grain directory (L1) and grain table (L2) entries were 32-bit, allowing access to only 2TB (slightly less) of data. * The grain size

[Qemu-block] [PULL 5/8] vmdk: Reduce the max bound for L1 table size

2019-06-21 Thread Max Reitz
From: Sam Eiderman 512M of L1 entries is a very loose bound, only 32M are required to store the maximal supported VMDK file size of 2TB. Fixed qemu-iotest 59# - now failure occures before on impossible L1 table size. Reviewed-by: Karl Heubaum Reviewed-by: Eyal Moscovici Reviewed-by: Liran

[Qemu-block] [PULL 4/8] vmdk: Fix comment regarding max l1_size coverage

2019-06-21 Thread Max Reitz
From: Sam Eiderman Commit b0651b8c246d ("vmdk: Move l1_size check into vmdk_add_extent") extended the l1_size check from VMDK4 to VMDK3 but did not update the default coverage in the moved comment. The previous vmdk4 calculation: (512 * 1024 * 1024) * 512(l2 entries) * 65536(grain) = 16PB

[Qemu-block] [PULL 2/8] blockdev: enable non-root nodes for transaction drive-backup source

2019-06-21 Thread Max Reitz
From: Vladimir Sementsov-Ogievskiy We forget to enable it for transaction .prepare, while it is already enabled in do_drive_backup since commit a2d665c1bc362 "blockdev: loosen restrictions on drive-backup source node" Signed-off-by: Vladimir Sementsov-Ogievskiy Message-id:

[Qemu-block] [PULL 3/8] iotest 134: test cluster-misaligned encrypted write

2019-06-21 Thread Max Reitz
From: Anton Nefedov COW (even empty/zero) areas require encryption too Signed-off-by: Anton Nefedov Reviewed-by: Eric Blake Reviewed-by: Max Reitz Reviewed-by: Alberto Garcia Message-id: 20190516143028.81155-1-anton.nefe...@virtuozzo.com Signed-off-by: Max Reitz --- tests/qemu-iotests/134

[Qemu-block] [PULL 1/8] nvme: do not advertise support for unsupported arbitration mechanism

2019-06-21 Thread Max Reitz
From: Klaus Birkelund Jensen The device mistakenly reports that the Weighted Round Robin with Urgent Priority Class arbitration mechanism is supported. It is not. Signed-off-by: Klaus Birkelund Jensen Message-id: 20190606092530.14206-1-kl...@birkelund.eu Acked-by: Maxim Levitsky

[Qemu-block] [PULL 0/8] Block patches

2019-06-21 Thread Max Reitz
The following changes since commit 33d609990621dea6c7d056c86f707b8811320ac1: Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging (2019-06-18 17:00:52 +0100) are available in the Git repository at: https://github.com/XanClic/qemu.git tags/pull-block-2019-06-21 for

Re: [Qemu-block] [PATCH v5 30/42] qemu-img: Use child access functions

2019-06-21 Thread Vladimir Sementsov-Ogievskiy
19.06.2019 18:49, Max Reitz wrote: > On 19.06.19 11:18, Vladimir Sementsov-Ogievskiy wrote: >> 13.06.2019 1:09, Max Reitz wrote: >>> This changes iotest 204's output, because blkdebug on top of a COW node >>> used to make qemu-img map disregard the rest of the backing chain (the >>> backing chain

Re: [Qemu-block] [PATCH 07/12] block/backup: add 'always' bitmap sync policy

2019-06-21 Thread Vladimir Sementsov-Ogievskiy
21.06.2019 15:59, Vladimir Sementsov-Ogievskiy wrote: > 21.06.2019 15:57, Vladimir Sementsov-Ogievskiy wrote: >> 20.06.2019 4:03, John Snow wrote: >>> This adds an "always" policy for bitmap synchronization. Regardless of if >>> the job succeeds or fails, the bitmap is *always* synchronized. This

Re: [Qemu-block] [PATCH 07/12] block/backup: add 'always' bitmap sync policy

2019-06-21 Thread Vladimir Sementsov-Ogievskiy
21.06.2019 15:57, Vladimir Sementsov-Ogievskiy wrote: > 20.06.2019 4:03, John Snow wrote: >> This adds an "always" policy for bitmap synchronization. Regardless of if >> the job succeeds or fails, the bitmap is *always* synchronized. This means >> that for backups that fail part-way through, the

Re: [Qemu-block] [PATCH 07/12] block/backup: add 'always' bitmap sync policy

2019-06-21 Thread Vladimir Sementsov-Ogievskiy
20.06.2019 4:03, John Snow wrote: > This adds an "always" policy for bitmap synchronization. Regardless of if > the job succeeds or fails, the bitmap is *always* synchronized. This means > that for backups that fail part-way through, the bitmap retains a record of > which sectors need to be copied

Re: [Qemu-block] [PATCH] iotests: Fix 205 for concurrent runs

2019-06-21 Thread Max Reitz
On 18.06.19 23:02, Max Reitz wrote: > Tests should place their files into the test directory. This includes > Unix sockets. 205 currently fails to do so, which prevents it from > being run concurrently. > > Signed-off-by: Max Reitz > --- > tests/qemu-iotests/205 | 2 +- > 1 file changed, 1

Re: [Qemu-block] [PATCH 06/12] block/dirty-bitmap: add bdrv_dirty_bitmap_claim

2019-06-21 Thread Vladimir Sementsov-Ogievskiy
20.06.2019 19:36, John Snow wrote: > > > On 6/20/19 12:02 PM, Max Reitz wrote: >> On 20.06.19 03:03, John Snow wrote: >>> This function can claim an hbitmap to replace its own current hbitmap. >>> In the case that the granularities do not match, it will use >>> hbitmap_merge to copy the bit data

Re: [Qemu-block] [PATCH 05/12] hbitmap: enable merging across granularities

2019-06-21 Thread Vladimir Sementsov-Ogievskiy
20.06.2019 19:47, Max Reitz wrote: > On 20.06.19 03:03, John Snow wrote: >> Signed-off-by: John Snow >> --- >> util/hbitmap.c | 22 +- >> 1 file changed, 21 insertions(+), 1 deletion(-) > > I wonder whether this could be used in > backup_incremental_init_copy_bitmap().

Re: [Qemu-block] [PATCH 05/12] hbitmap: enable merging across granularities

2019-06-21 Thread Vladimir Sementsov-Ogievskiy
20.06.2019 4:03, John Snow wrote: > Signed-off-by: John Snow > --- > util/hbitmap.c | 22 +- > 1 file changed, 21 insertions(+), 1 deletion(-) > > diff --git a/util/hbitmap.c b/util/hbitmap.c > index 45d1725daf..0d6724b7bc 100644 > --- a/util/hbitmap.c > +++

Re: [Qemu-block] [PATCH 02/12] block/backup: Add mirror sync mode 'bitmap'

2019-06-21 Thread Vladimir Sementsov-Ogievskiy
20.06.2019 4:03, John Snow wrote: > We don't need or want a new sync mode for simple differences in > semantics. Create a new mode simply named "BITMAP" that is designed to > make use of the new Bitmap Sync Mode field. > > Because the only bitmap mode is 'conditional', this adds no new >

Re: [Qemu-block] [Qemu-devel] [PATCH] blk: postpone request execution on a context protected with "drained section"

2019-06-21 Thread Vladimir Sementsov-Ogievskiy
21.06.2019 12:16, Kevin Wolf wrote: > Am 09.04.2019 um 12:01 hat Kevin Wolf geschrieben: >> Am 02.04.2019 um 10:35 hat Denis Plotnikov geschrieben: >>> On 13.03.2019 19:04, Kevin Wolf wrote: Am 14.12.2018 um 12:54 hat Denis Plotnikov geschrieben: > On 13.12.2018 15:20, Kevin Wolf wrote:

Re: [Qemu-block] [PATCH 2/3] block/accounting: introduce block size histogram

2019-06-21 Thread Vladimir Sementsov-Ogievskiy
20.06.2019 11:54, zhenwei pi wrote: > Introduce block size histogram statics for block devices. > > For read/write/flush operation type, the block size region > [0, +inf) is divided into subregions by several points. > It works like block latency histogram. > > Signed-off-by: zhenwei pi > --- >

Re: [Qemu-block] [PATCH 1/3] block/accounting: rename struct BlockLatencyHistogram

2019-06-21 Thread Vladimir Sementsov-Ogievskiy
20.06.2019 11:54, zhenwei pi wrote: > Rename struct BlockLatencyHistogram to BlockHistogram, and rename > related functions name. > make this struct and functions be common, they can be used widely. Hmm, we can go further, and make it just Histogram and move to separate file, as there is nothing

Re: [Qemu-block] [External Email] Re: [Qemu-devel] [PATCH 3/3] qapi: add block size histogram interface

2019-06-21 Thread Vladimir Sementsov-Ogievskiy
21.06.2019 4:52, zhenwei pi wrote: > On 6/20/19 10:03 PM, Eric Blake wrote: > >> On 6/20/19 3:54 AM, zhenwei pi wrote: >>> Set/Clear block size histograms through new command >>> x-block-size-histogram-set and show new statistics in >>> query-blockstats results. >>> >> I'm guessing this is

Re: [Qemu-block] [PATCH] blk: postpone request execution on a context protected with "drained section"

2019-06-21 Thread Kevin Wolf
Am 09.04.2019 um 12:01 hat Kevin Wolf geschrieben: > Am 02.04.2019 um 10:35 hat Denis Plotnikov geschrieben: > > On 13.03.2019 19:04, Kevin Wolf wrote: > > > Am 14.12.2018 um 12:54 hat Denis Plotnikov geschrieben: > > >> On 13.12.2018 15:20, Kevin Wolf wrote: > > >>> Am 13.12.2018 um 12:07 hat

Re: [Qemu-block] [PATCH] blockjob: drain all job nodes in block_job_drain

2019-06-21 Thread Vladimir Sementsov-Ogievskiy
19.06.2019 19:37, Max Reitz wrote: > On 19.06.19 16:44, Vladimir Sementsov-Ogievskiy wrote: >> Instead of draining additional nodes in each job code, let's do it in >> common block_job_drain, draining just all job's children. >> >> It's also a first step to finally get rid of blockjob->blk. >> >>

Re: [Qemu-block] [PATCH v7 1/3] block: include base when checking image chain for block allocation

2019-06-21 Thread Vladimir Sementsov-Ogievskiy
19.06.2019 22:27, Max Reitz wrote: > On 29.05.19 19:56, Andrey Shinkevich wrote: >> This patch is used in the 'block/stream: introduce a bottom node' >> that is following. Instead of the base node, the caller may pass >> the node that has the base as its backing image to the function >>

[Qemu-block] [PATCH V2 RESEND] block/replication.c: Fix crash issue after failover

2019-06-21 Thread Zhang Chen
From: Zhang Chen If we try to close replication after failover, it will crash here. So we need check the block job on active disk before cancel the job. Signed-off-by: Zhang Chen --- block/replication.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/block/replication.c

Re: [Qemu-block] [PATCH V2] block/replication.c: Fix crash issue after failover

2019-06-21 Thread Zhang, Chen
Please redirect to the V2 RESEND patch, this one not rebased on upstream code. Thanks Zhang Chen > -Original Message- > From: Zhang, Chen > Sent: Friday, June 21, 2019 1:52 PM > To: Xie Changlong ; Kevin Wolf > ; Max Reitz ; qemu-block bl...@nongnu.org>; qemu-dev > Cc: Zhang Chen ;

[Qemu-block] [PATCH V2] block/replication.c: Fix crash issue after failover

2019-06-21 Thread Zhang Chen
From: Zhang Chen If we try to close replication after failover, it will crash here. So we need check the block job on active disk before cancel the job. Signed-off-by: Zhang Chen --- block/replication.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/block/replication.c