Re: [Qemu-block] [Qemu-devel] [PATCH v2 01/18] qapi/block-core: Introduce BackupCommon

2019-07-05 Thread Markus Armbruster
John Snow writes: > On 7/5/19 10:14 AM, Markus Armbruster wrote: >> John Snow writes: >> >>> drive-backup and blockdev-backup have an awful lot of things in common >>> that are the same. Let's fix that. >>> >>> I don't deduplicate 'target', because the semantics actually did change >>> between

Re: [Qemu-block] [PATCH v2 RFC] qemu-nbd: Permit TLS with Unix sockets

2019-07-05 Thread Eric Blake
On 7/5/19 4:31 AM, Max Reitz wrote: > On 04.07.19 00:47, Eric Blake wrote: >> Although you generally won't use encryption with a Unix socket (after >> all, everything is local, so why waste the CPU power), there are >> situations in testsuites where Unix sockets are much nicer than TCP >> sockets.

[Qemu-block] [PATCH v3 17/18] iotests: add test 257 for bitmap-mode backups

2019-07-05 Thread John Snow
Signed-off-by: John Snow --- tests/qemu-iotests/257 | 409 +++ tests/qemu-iotests/257.out | 2199 tests/qemu-iotests/group |1 + 3 files changed, 2609 insertions(+) create mode 100755 tests/qemu-iotests/257 create mode 100644

[Qemu-block] [PATCH v3 18/18] block/backup: loosen restriction on readonly bitmaps

2019-07-05 Thread John Snow
With the "never" sync policy, we actually can utilize readonly bitmaps now. Loosen the check at the QMP level, and tighten it based on provided arguments down at the job creation level instead. Reviewed-by: Max Reitz Signed-off-by: John Snow --- block/backup.c | 6 ++ blockdev.c | 2 +-

[Qemu-block] [PATCH v3 16/18] iotests: Add virtio-scsi device helper

2019-07-05 Thread John Snow
Seems that it comes up enough. Reviewed-by: Max Reitz Signed-off-by: John Snow --- tests/qemu-iotests/040| 6 +- tests/qemu-iotests/093| 6 ++ tests/qemu-iotests/139| 7 ++- tests/qemu-iotests/238| 5 + tests/qemu-iotests/iotests.py | 4 5

[Qemu-block] [PATCH v3 09/18] block/dirty-bitmap: add bdrv_dirty_bitmap_merge_internal

2019-07-05 Thread John Snow
I'm surprised it didn't come up sooner, but sometimes we have a +busy bitmap as a source. This is dangerous from the QMP API, but if we are the owner that marked the bitmap busy, it's safe to merge it using it as a read only source. It is not safe in the general case to allow users to read from

[Qemu-block] [PATCH v3 08/18] hbitmap: enable merging across granularities

2019-07-05 Thread John Snow
Reviewed-by: Max Reitz Signed-off-by: John Snow --- util/hbitmap.c | 36 +++- 1 file changed, 35 insertions(+), 1 deletion(-) diff --git a/util/hbitmap.c b/util/hbitmap.c index 3b6acae42b..306bc4876d 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -777,7

[Qemu-block] [PATCH v3 14/18] iotests: teach run_job to cancel pending jobs

2019-07-05 Thread John Snow
run_job can cancel pending jobs to simulate failure. This lets us use the pending callback to issue test commands while the job is open, but then still have the job fail in the end. Reviewed-by: Max Reitz Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 22 -- 1

[Qemu-block] [PATCH v3 15/18] iotests: teach FilePath to produce multiple paths

2019-07-05 Thread John Snow
Use "FilePaths" instead of "FilePath" to request multiple files be cleaned up after we leave that object's scope. This is not crucial; but it saves a little typing. Reviewed-by: Max Reitz Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 34 -- 1

[Qemu-block] [PATCH v3 05/18] block/backup: Add mirror sync mode 'bitmap'

2019-07-05 Thread John Snow
We don't need or want a new sync mode for simple differences in semantics. Create a new mode simply named "BITMAP" that is designed to make use of the new Bitmap Sync Mode field. Because the only bitmap sync mode is 'on-success', this adds no new functionality to the backup job (yet). The old

[Qemu-block] [PATCH v3 13/18] iotests: add testing shim for script-style python tests

2019-07-05 Thread John Snow
Because the new-style python tests don't use the iotests.main() test launcher, we don't turn on the debugger logging for these scripts when invoked via ./check -d. Refactor the launcher shim into new and old style shims so that they share environmental configuration. Two cleanup notes: debug was

[Qemu-block] [PATCH v3 11/18] block/backup: upgrade copy_bitmap to BdrvDirtyBitmap

2019-07-05 Thread John Snow
This simplifies some interface matters; namely the initialization and (later) merging the manifest back into the sync_bitmap if it was provided. Signed-off-by: John Snow --- block/backup.c | 81 ++ 1 file changed, 42 insertions(+), 39 deletions(-)

[Qemu-block] [PATCH v3 10/18] block/dirty-bitmap: add bdrv_dirty_bitmap_get

2019-07-05 Thread John Snow
Add a public interface for get. While we're at it, rename "bdrv_get_dirty_bitmap_locked" to "bdrv_dirty_bitmap_get_locked". (There are more functions to rename to the bdrv_dirty_bitmap_VERB form, but they will wait until the conclusion of this series.) Reviewed-by: Max Reitz Signed-off-by: John

[Qemu-block] [PATCH v3 12/18] block/backup: add 'always' bitmap sync policy

2019-07-05 Thread John Snow
This adds an "always" policy for bitmap synchronization. Regardless of if the job succeeds or fails, the bitmap is *always* synchronized. This means that for backups that fail part-way through, the bitmap retains a record of which sectors need to be copied out to accomplish a new backup using the

[Qemu-block] [PATCH v3 04/18] qapi: add BitmapSyncMode enum

2019-07-05 Thread John Snow
Depending on what a user is trying to accomplish, there might be a few bitmap cleanup actions that occur when an operation is finished that could be useful. I am proposing three: - NEVER: The bitmap is never synchronized against what was copied. - ALWAYS: The bitmap is always synchronized, even

[Qemu-block] [PATCH v3 06/18] block/backup: add 'never' policy to bitmap sync mode

2019-07-05 Thread John Snow
This adds a "never" policy for bitmap synchronization. Regardless of if the job succeeds or fails, we never update the bitmap. This can be used to perform differential backups, or simply to avoid the job modifying a bitmap. Signed-off-by: John Snow --- block/backup.c | 7 +--

[Qemu-block] [PATCH v3 07/18] hbitmap: Fix merge when b is empty, and result is not an alias of a

2019-07-05 Thread John Snow
Nobody calls the function like this currently, but we neither prohibit or cope with this behavior. I decided to make the function cope with it. Reviewed-by: Max Reitz Signed-off-by: John Snow --- util/hbitmap.c | 13 ++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git

[Qemu-block] [PATCH v3 02/18] drive-backup: create do_backup_common

2019-07-05 Thread John Snow
Create a common core that comprises the actual meat of what the backup API boundary needs to do, and then switch drive-backup to use it. Signed-off-by: John Snow --- blockdev.c | 122 + 1 file changed, 67 insertions(+), 55 deletions(-) diff

[Qemu-block] [PATCH v3 01/18] qapi/block-core: Introduce BackupCommon

2019-07-05 Thread John Snow
drive-backup and blockdev-backup have an awful lot of things in common that are the same. Let's fix that. I don't deduplicate 'target', because the semantics actually did change between each structure. Leave that one alone so it can be documented separately. Reviewed-by: Max Reitz

[Qemu-block] [PATCH v3 03/18] blockdev-backup: utilize do_backup_common

2019-07-05 Thread John Snow
Signed-off-by: John Snow --- blockdev.c | 65 +- 1 file changed, 6 insertions(+), 59 deletions(-) diff --git a/blockdev.c b/blockdev.c index 5bc8ecd087..77365d8166 100644 --- a/blockdev.c +++ b/blockdev.c @@ -3624,78 +3624,25 @@ BlockJob

[Qemu-block] [PATCH v3 00/18] bitmaps: introduce 'bitmap' sync mode

2019-07-05 Thread John Snow
This series adds a new "BITMAP" sync mode that is meant to replace the existing "INCREMENTAL" sync mode. This mode can have its behavior modified by issuing any of three bitmap sync modes, passed as arguments to the job. The three bitmap sync modes are: - ON-SUCCESS: This is an alias for the old

Re: [Qemu-block] [PATCH v2 10/18] block/dirty-bitmap: add bdrv_dirty_bitmap_get

2019-07-05 Thread John Snow
On 7/4/19 1:01 PM, Max Reitz wrote: > On 03.07.19 23:55, John Snow wrote: >> Add a public interface for get. While we're at it, >> rename "bdrv_get_dirty_bitmap_locked" to "bdrv_dirty_bitmap_get_locked". >> >> (There are more functions to rename to the bdrv_dirty_bitmap_VERB form, >> but they

Re: [Qemu-block] [PATCH v2 02/18] drive-backup: create do_backup_common

2019-07-05 Thread John Snow
On 7/4/19 11:06 AM, Max Reitz wrote: > On 03.07.19 23:55, John Snow wrote: >> Create a common core that comprises the actual meat of what the backup API >> boundary needs to do, and then switch drive-backup to use it. >> >> Questions: >> - do_drive_backup now acquires and releases the

Re: [Qemu-block] [PATCH v2 02/18] drive-backup: create do_backup_common

2019-07-05 Thread John Snow
On 7/4/19 11:06 AM, Max Reitz wrote: > On 03.07.19 23:55, John Snow wrote: >> Create a common core that comprises the actual meat of what the backup API >> boundary needs to do, and then switch drive-backup to use it. >> >> Questions: >> - do_drive_backup now acquires and releases the

Re: [Qemu-block] [PATCH v2 12/18] block/backup: add 'always' bitmap sync policy

2019-07-05 Thread John Snow
On 7/4/19 2:05 PM, Max Reitz wrote: > On 04.07.19 19:43, Max Reitz wrote: >> On 03.07.19 23:55, John Snow wrote: >>> This adds an "always" policy for bitmap synchronization. Regardless of if >>> the job succeeds or fails, the bitmap is *always* synchronized. This means >>> that for backups that

Re: [Qemu-block] [PATCH v2 11/18] block/backup: upgrade copy_bitmap to BdrvDirtyBitmap

2019-07-05 Thread John Snow
On 7/4/19 1:29 PM, Max Reitz wrote: > On 03.07.19 23:55, John Snow wrote: >> This simplifies some interface matters; namely the initialization and >> (later) merging the manifest back into the sync_bitmap if it was >> provided. >> >> Signed-off-by: John Snow >> --- >> block/backup.c | 76

Re: [Qemu-block] [PATCH v3 2/9] hw/block/pflash_cfi01: Use the correct READ_ARRAY value

2019-07-05 Thread Laszlo Ersek
On 07/05/19 17:46, Philippe Mathieu-Daudé wrote: > In the "Read Array Flowchart" the command has a value of 0xFF. > > In the document [*] the "Read Array Flowchart", the READ_ARRAY > command has a value of 0xff. > > Use the correct value in the pflash model. > > There is no change of behavior

Re: [Qemu-block] [PATCH v2 09/18] block/dirty-bitmap: add bdrv_dirty_bitmap_merge_internal

2019-07-05 Thread John Snow
On 7/4/19 12:49 PM, Max Reitz wrote: > On 03.07.19 23:55, John Snow wrote: >> I'm surprised it didn't come up sooner, but sometimes we have a +busy >> bitmap as a source. This is dangerous from the QMP API, but if we are >> the owner that marked the bitmap busy, it's safe to merge it using it

Re: [Qemu-block] [Qemu-devel] [PATCH v2 04/18] qapi: add BitmapSyncMode enum

2019-07-05 Thread John Snow
On 7/5/19 10:18 AM, Markus Armbruster wrote: > John Snow writes: > >> Depending on what a user is trying to accomplish, there might be a few >> bitmap cleanup actions that occur when an operation is finished that >> could be useful. >> >> I am proposing three: >> - NEVER: The bitmap is never

Re: [Qemu-block] [Qemu-devel] [PATCH v2 01/18] qapi/block-core: Introduce BackupCommon

2019-07-05 Thread John Snow
On 7/5/19 10:14 AM, Markus Armbruster wrote: > John Snow writes: > >> drive-backup and blockdev-backup have an awful lot of things in common >> that are the same. Let's fix that. >> >> I don't deduplicate 'target', because the semantics actually did change >> between each structure. Leave

Re: [Qemu-block] [Qemu-devel] [PATCH 16/16] nvme: support multiple namespaces

2019-07-05 Thread Daniel P . Berrangé
On Fri, Jul 05, 2019 at 06:20:14PM +0200, Klaus Birkelund wrote: > On Fri, Jul 05, 2019 at 02:49:29PM +0100, Daniel P. Berrangé wrote: > > On Fri, Jul 05, 2019 at 03:36:17PM +0200, Klaus Birkelund wrote: > > > On Fri, Jul 05, 2019 at 09:23:33AM +0200, Klaus Birkelund Jensen wrote: > > > > This

Re: [Qemu-block] [Qemu-devel] [PATCH 16/16] nvme: support multiple namespaces

2019-07-05 Thread Klaus Birkelund
On Fri, Jul 05, 2019 at 02:49:29PM +0100, Daniel P. Berrangé wrote: > On Fri, Jul 05, 2019 at 03:36:17PM +0200, Klaus Birkelund wrote: > > On Fri, Jul 05, 2019 at 09:23:33AM +0200, Klaus Birkelund Jensen wrote: > > > This adds support for multiple namespaces by introducing a new 'nvme-ns' > > >

[Qemu-block] [PATCH-for-4.1] Revert "hw/block/pflash_cfi02: Reduce I/O accesses to 16-bit"

2019-07-05 Thread Philippe Mathieu-Daudé
Stephen Checkoway noticed commit 3ae0343db69 is incorrect. This commit state all parallel flashes are limited to 16-bit accesses, however the x32 configuration exists in some models, such the Cypress S29CL032J, which CFI Device Geometry Definition announces: CFI ADDR DATA 0x28,0x29 =

Re: [Qemu-block] [Qemu-devel] [PATCH for-4.1] qcow2: Allow -o compat=v3 during qemu-img amend

2019-07-05 Thread Eric Blake
On 7/5/19 10:53 AM, Eric Blake wrote: > On 7/5/19 10:28 AM, Eric Blake wrote: >> Commit b76b4f60 allowed '-o compat=v3' as an alias for the >> less-appealing '-o compat=1.1' for 'qemu-img create' since we want to >> use the QMP form as much as possible, but forgot to do likewise for >> qemu-img

Re: [Qemu-block] [Qemu-devel] [PATCH for-4.1] qcow2: Allow -o compat=v3 during qemu-img amend

2019-07-05 Thread Eric Blake
On 7/5/19 10:28 AM, Eric Blake wrote: > Commit b76b4f60 allowed '-o compat=v3' as an alias for the > less-appealing '-o compat=1.1' for 'qemu-img create' since we want to > use the QMP form as much as possible, but forgot to do likewise for > qemu-img amend. Also, it doesn't help that '-o help'

[Qemu-block] [PATCH v3 9/9] hw/block/pflash_cfi01: Hold the PRI table offset in a variable

2019-07-05 Thread Philippe Mathieu-Daudé
Manufacturers are allowed to move the PRI table, this is why the offset is queryable via fixed offsets 0x15/0x16. Add a variable to hold the offset, so it will be easier to later move the PRI table. Reviewed-by: Alistair Francis Regression-tested-by: Laszlo Ersek Signed-off-by: Philippe

[Qemu-block] [PATCH v3 8/9] hw/block/pflash_cfi01: Replace DPRINTF by qemu_log_mask(GUEST_ERROR)

2019-07-05 Thread Philippe Mathieu-Daudé
Reviewed-by: Alistair Francis Regression-tested-by: Laszlo Ersek Signed-off-by: Philippe Mathieu-Daudé --- hw/block/pflash_cfi01.c | 8 ++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/hw/block/pflash_cfi01.c b/hw/block/pflash_cfi01.c index ba00e52923..ab72af22a7 100644

[Qemu-block] [PATCH v3 7/9] hw/block/pflash_cfi01: Improve command comments

2019-07-05 Thread Philippe Mathieu-Daudé
Reviewed-by: Alistair Francis Regression-tested-by: Laszlo Ersek Signed-off-by: Philippe Mathieu-Daudé --- hw/block/pflash_cfi01.c | 16 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/hw/block/pflash_cfi01.c b/hw/block/pflash_cfi01.c index e097d9260d..ba00e52923

[Qemu-block] [PATCH v3 6/9] hw/block/pflash_cfi01: Simplify CFI_QUERY processing

2019-07-05 Thread Philippe Mathieu-Daudé
The current code does: if (write_cycle == 0) if (command == CFI_QUERY) break write_cycle += 1 last_command = command if (write_cycle == 1) if (last_command == CFI_QUERY) if (command == READ_ARRAY write_cycle = 0 last_command = READ_ARRAY Simplify by not increasing

[Qemu-block] [PATCH v3 5/9] hw/block/pflash_cfi01: Add the DeviceReset() handler

2019-07-05 Thread Philippe Mathieu-Daudé
A "system reset" sets the device state machine in READ_ARRAY mode and, after some delay, set the SR.7 READY bit. We do not model timings, so we set the SR.7 bit directly. The TYPE_DEVICE interface provides a DeviceReset handler. This pflash device is a subclass of TYPE_SYS_BUS_DEVICE (which is a

[Qemu-block] [PATCH v3 3/9] hw/block/pflash_cfi01: Extract pflash_mode_read_array()

2019-07-05 Thread Philippe Mathieu-Daudé
The same pattern is used when setting the flash in READ_ARRAY mode: - Set the state machine command to READ_ARRAY - Reset the write_cycle counter - Reset the memory region in ROMD Refactor the current code by extracting this pattern. It is used twice: - On a write access (on command failure,

[Qemu-block] [PATCH v3 0/9] hw/block/pflash_cfi01: Add DeviceReset() handler

2019-07-05 Thread Philippe Mathieu-Daudé
The pflash device lacks a reset() function. When a machine is resetted, the flash might be in an inconsistent state, leading to unexpected behavior: https://bugzilla.redhat.com/show_bug.cgi?id=1678713 Resolve this issue by adding a DeviceReset() handler. Fix also two minor issues, and clean a bit

[Qemu-block] [PATCH v3 4/9] hw/block/pflash_cfi01: Start state machine as READY to accept commands

2019-07-05 Thread Philippe Mathieu-Daudé
When the state machine is ready to accept command, the bit 7 of the status register (SR) is set to 1. The guest polls the status register and check this bit before writting command to the internal 'Write State Machine' (WSM). Set SR.7 bit to 1 when the device is created. There is no migration

[Qemu-block] [PATCH v3 2/9] hw/block/pflash_cfi01: Use the correct READ_ARRAY value

2019-07-05 Thread Philippe Mathieu-Daudé
In the "Read Array Flowchart" the command has a value of 0xFF. In the document [*] the "Read Array Flowchart", the READ_ARRAY command has a value of 0xff. Use the correct value in the pflash model. There is no change of behavior in the guest, because: - when the guest were sending 0xFF, the

[Qemu-block] [PATCH v3 1/9] hw/block/pflash_cfi01: Removed an unused timer

2019-07-05 Thread Philippe Mathieu-Daudé
The 'CFI02' NOR flash was introduced in commit 29133e9a0fff, with timing modelled. One year later, the CFI01 model was introduced (commit 05ee37ebf630) based on the CFI02 model. As noted in the header, "It does not support timings". 12 years later, we never had to model the device timings. Time to

[Qemu-block] [PATCH for-4.1] qcow2: Allow -o compat=v3 during qemu-img amend

2019-07-05 Thread Eric Blake
Commit b76b4f60 allowed '-o compat=v3' as an alias for the less-appealing '-o compat=1.1' for 'qemu-img create' since we want to use the QMP form as much as possible, but forgot to do likewise for qemu-img amend. Also, it doesn't help that '-o help' doesn't list our new preferred spellings.

Re: [Qemu-block] [QEMU-SECURITY] ide: fix assertion in ide_dma_cb() to prevent qemu DoS from quest

2019-07-05 Thread Alexander Popov
On 05.07.2019 17:07, Alexander Popov wrote: > This assertion was introduced in the commit a718978ed58a in July 2015. > It implies that the size of successful DMA transfers handled in > ide_dma_cb() should be multiple of 512 (the size of a sector). > > But guest systems can initiate DMA transfers

[Qemu-block] [QEMU-SECURITY] ide: fix assertion in ide_dma_cb() to prevent qemu DoS from quest

2019-07-05 Thread Alexander Popov
This assertion was introduced in the commit a718978ed58a in July 2015. It implies that the size of successful DMA transfers handled in ide_dma_cb() should be multiple of 512 (the size of a sector). But guest systems can initiate DMA transfers that don't fit this requirement. Let's improve the

Re: [Qemu-block] [Qemu-devel] [PATCH v2 04/18] qapi: add BitmapSyncMode enum

2019-07-05 Thread Markus Armbruster
John Snow writes: > Depending on what a user is trying to accomplish, there might be a few > bitmap cleanup actions that occur when an operation is finished that > could be useful. > > I am proposing three: > - NEVER: The bitmap is never synchronized against what was copied. > - ALWAYS: The

Re: [Qemu-block] [Qemu-devel] [PATCH v2 01/18] qapi/block-core: Introduce BackupCommon

2019-07-05 Thread Markus Armbruster
John Snow writes: > drive-backup and blockdev-backup have an awful lot of things in common > that are the same. Let's fix that. > > I don't deduplicate 'target', because the semantics actually did change > between each structure. Leave that one alone so it can be documented > separately. > >

Re: [Qemu-block] [PATCH v4] block/nvme: add support for discard

2019-07-05 Thread Max Reitz
On 03.07.19 18:07, Maxim Levitsky wrote: > Signed-off-by: Maxim Levitsky > --- > block/nvme.c | 81 ++ > block/trace-events | 2 ++ > 2 files changed, 83 insertions(+) > > diff --git a/block/nvme.c b/block/nvme.c > index 02e0846643..96a715dcc1

Re: [Qemu-block] [Qemu-devel] [PATCH 16/16] nvme: support multiple namespaces

2019-07-05 Thread Daniel P . Berrangé
On Fri, Jul 05, 2019 at 03:36:17PM +0200, Klaus Birkelund wrote: > On Fri, Jul 05, 2019 at 09:23:33AM +0200, Klaus Birkelund Jensen wrote: > > This adds support for multiple namespaces by introducing a new 'nvme-ns' > > device model. The nvme device creates a bus named from the device name > >

Re: [Qemu-block] [PATCH 16/16] nvme: support multiple namespaces

2019-07-05 Thread Klaus Birkelund
On Fri, Jul 05, 2019 at 09:23:33AM +0200, Klaus Birkelund Jensen wrote: > This adds support for multiple namespaces by introducing a new 'nvme-ns' > device model. The nvme device creates a bus named from the device name > ('id'). The nvme-ns devices then connect to this and registers > themselves

Re: [Qemu-block] [PATCH v3 5/6] block/nvme: add support for write zeros

2019-07-05 Thread Max Reitz
On 03.07.19 17:59, Maxim Levitsky wrote: > Signed-off-by: Maxim Levitsky > --- > block/nvme.c | 69 +++- > block/trace-events | 1 + > include/block/nvme.h | 19 +++- > 3 files changed, 87 insertions(+), 2 deletions(-) > > diff --git

Re: [Qemu-block] [PATCH v3 4/6] block/nvme: add support for image creation

2019-07-05 Thread Max Reitz
On 03.07.19 17:59, Maxim Levitsky wrote: > Tesed on a nvme device like that: > > # create preallocated qcow2 image > $ qemu-img create -f qcow2 nvme://:06:00.0/1 10G -o preallocation=metadata > Formatting 'nvme://:06:00.0/1', fmt=qcow2 size=10737418240 > cluster_size=65536

Re: [Qemu-block] [PATCH v3 3/6] block/nvme: support larger that 512 bytes sector devices

2019-07-05 Thread Max Reitz
On 03.07.19 17:59, Maxim Levitsky wrote: > Currently the driver hardcodes the sector size to 512, > and doesn't check the underlying device. Fix that. > > Also fail if underlying nvme device is formatted with metadata > as this needs special support. > > Signed-off-by: Maxim Levitsky > --- >

Re: [Qemu-block] [PATCH v3 2/6] block/nvme: fix doorbell stride

2019-07-05 Thread Max Reitz
On 05.07.19 13:09, Max Reitz wrote: > On 03.07.19 17:59, Maxim Levitsky wrote: >> Fix the math involving non standard doorbell stride >> >> Signed-off-by: Maxim Levitsky >> --- >> block/nvme.c | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/block/nvme.c b/block/nvme.c

Re: [Qemu-block] [PATCH v3 2/6] block/nvme: fix doorbell stride

2019-07-05 Thread Max Reitz
On 03.07.19 17:59, Maxim Levitsky wrote: > Fix the math involving non standard doorbell stride > > Signed-off-by: Maxim Levitsky > --- > block/nvme.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/block/nvme.c b/block/nvme.c > index 6d4e7f3d83..52798081b2 100644 > ---

Re: [Qemu-block] [PATCH v3 1/6] block/nvme: don't touch the completion entries

2019-07-05 Thread Max Reitz
On 03.07.19 17:59, Maxim Levitsky wrote: > Completion entries are meant to be only read by the host and written by the > device. > The driver is supposed to scan the completions from the last point where it > left, > and until it sees a completion with non flipped phase bit. (Disclaimer: This

Re: [Qemu-block] [PATCH v3] block/rbd: implement .bdrv_get_allocated_file_size callback

2019-07-05 Thread Stefano Garzarella
On Fri, Jul 05, 2019 at 11:58:43AM +0200, Max Reitz wrote: > On 05.07.19 11:32, Stefano Garzarella wrote: > > This patch allows 'qemu-img info' to show the 'disk size' for > > the RBD images that have the fast-diff feature enabled. > > > > If this feature is enabled, we use the

Re: [Qemu-block] [Qemu-devel] [PATCH v2 RFC] qemu-nbd: Permit TLS with Unix sockets

2019-07-05 Thread Daniel P . Berrangé
On Wed, Jul 03, 2019 at 05:47:07PM -0500, Eric Blake wrote: > +== check TLS works over Unix == > +image: nbd+unix://?socket=SOCKET > +file format: nbd > +virtual size: 64 MiB (67108864 bytes) > +disk size: unavailable > +image: nbd+unix://?socket=SOCKET > +file format: nbd > +virtual size: 64 MiB

Re: [Qemu-block] [Qemu-devel] [PATCH v2 RFC] qemu-nbd: Permit TLS with Unix sockets

2019-07-05 Thread Daniel P . Berrangé
On Fri, Jul 05, 2019 at 11:31:51AM +0200, Max Reitz wrote: > On 04.07.19 00:47, Eric Blake wrote: > > diff --git a/tests/qemu-iotests/233.out b/tests/qemu-iotests/233.out > > index 9b46284ab0de..b86bee020649 100644 > > --- a/tests/qemu-iotests/233.out > > +++ b/tests/qemu-iotests/233.out > >

Re: [Qemu-block] [Qemu-devel] [PATCH v2 RFC] qemu-nbd: Permit TLS with Unix sockets

2019-07-05 Thread Daniel P . Berrangé
On Wed, Jul 03, 2019 at 05:47:07PM -0500, Eric Blake wrote: > Although you generally won't use encryption with a Unix socket (after > all, everything is local, so why waste the CPU power), there are > situations in testsuites where Unix sockets are much nicer than TCP > sockets. Since nbdkit

[Qemu-block] [PATCH v7] qemu-io: add pattern file for write command

2019-07-05 Thread Denis Plotnikov
The patch allows to provide a pattern file for write command. There was no similar ability before. Signed-off-by: Denis Plotnikov --- v7: * fix variable naming * make code more readable * extend help for write command v6: * the pattern file is read once to reduce io v5: * file name

Re: [Qemu-block] [PATCH v3] block/rbd: implement .bdrv_get_allocated_file_size callback

2019-07-05 Thread Max Reitz
On 05.07.19 11:32, Stefano Garzarella wrote: > This patch allows 'qemu-img info' to show the 'disk size' for > the RBD images that have the fast-diff feature enabled. > > If this feature is enabled, we use the rbd_diff_iterate2() API > to calculate the allocated size for the image. > >

Re: [Qemu-block] [PATCH v2 RFC] qemu-nbd: Permit TLS with Unix sockets

2019-07-05 Thread Max Reitz
On 04.07.19 00:47, Eric Blake wrote: > Although you generally won't use encryption with a Unix socket (after > all, everything is local, so why waste the CPU power), there are > situations in testsuites where Unix sockets are much nicer than TCP > sockets. Since nbdkit allows encryption over both

Re: [Qemu-block] [Qemu-devel] [PATCH 00/16] nvme: support NVMe v1.3d, SGLs and multiple namespaces

2019-07-05 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20190705072333.17171-1-kl...@birkelund.eu/ Hi, This series failed the asan build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BEGIN === #!/bin/bash make

Re: [Qemu-block] [Qemu-devel] [PATCH 00/16] nvme: support NVMe v1.3d, SGLs and multiple namespaces

2019-07-05 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20190705072333.17171-1-kl...@birkelund.eu/ Hi, This series seems to have some coding style problems. See output below for more information: Type: series Subject: [Qemu-devel] [PATCH 00/16] nvme: support NVMe v1.3d, SGLs and multiple namespaces

Re: [Qemu-block] question:about introduce a new feature named “I/O hang”

2019-07-05 Thread Kevin Wolf
Am 04.07.2019 um 17:16 hat wangjie (P) geschrieben: > Hi, everybody: > > I developed a feature named "I/O hang",my intention is to solve the problem > like that: > If the backend storage media of VM disk is far-end storage like IPSAN or > FCSAN, storage net link will always disconnection and >

Re: [Qemu-block] [RFC,v1] Namespace Management Support

2019-07-05 Thread Klaus Birkelund
On Tue, Jul 02, 2019 at 10:39:36AM -0700, Matt Fitzpatrick wrote: > Adding namespace management support to the nvme device. Namespace creation > requires contiguous block space for a simple method of allocation. > > I wrote this a few years ago based on Keith's fork and nvmeqemu fork and > have

[Qemu-block] [PATCH 12/16] nvme: bump supported NVMe revision to 1.3d

2019-07-05 Thread Klaus Birkelund Jensen
Add the new Namespace Identification Descriptor List (CNS 03h) and track creation of queues to enable the controller to return Command Sequence Error if Set Features is called for Number of Queues after any queues have been created. Signed-off-by: Klaus Birkelund Jensen --- hw/block/nvme.c

[Qemu-block] [PATCH 14/16] nvme: support multiple block requests per request

2019-07-05 Thread Klaus Birkelund Jensen
Currently, the device only issues a single block backend request per NVMe request, but as we move towards supporting metadata (and discontiguous vector requests supported by OpenChannel 2.0) it will be required to issue multiple block backend requests per NVMe request. With this patch the NVMe

[Qemu-block] [PATCH 16/16] nvme: support multiple namespaces

2019-07-05 Thread Klaus Birkelund Jensen
This adds support for multiple namespaces by introducing a new 'nvme-ns' device model. The nvme device creates a bus named from the device name ('id'). The nvme-ns devices then connect to this and registers themselves with the nvme device. This changes how an nvme device is created. Example with

[Qemu-block] [PATCH 13/16] nvme: simplify dma/cmb mappings

2019-07-05 Thread Klaus Birkelund Jensen
Instead of handling both QSGs and IOVs in multiple places, simply use QSGs everywhere by assuming that the request does not involve the controller memory buffer (CMB). If the request is found to involve the CMB, convert the QSG to an IOV and issue the I/O. The QSG is converted to an IOV by the dma

[Qemu-block] [PATCH 09/16] nvme: support Asynchronous Event Request command

2019-07-05 Thread Klaus Birkelund Jensen
Required for compliance with NVMe revision 1.2.1. See NVM Express 1.2.1, Section 5.2 ("Asynchronous Event Request command"). Modified from Keith's qemu-nvme tree. Signed-off-by: Klaus Birkelund Jensen --- hw/block/nvme.c | 88 ++- hw/block/nvme.h

[Qemu-block] [PATCH 05/16] nvme: populate the mandatory subnqn and ver fields

2019-07-05 Thread Klaus Birkelund Jensen
Required for compliance with NVMe revision 1.2.1 or later. See NVM Express 1.2.1, Section 5.11 ("Identify command"), Figure 90 and Section 7.9 ("NVMe Qualified Names"). This also bumps the supported version to 1.2.1. Signed-off-by: Klaus Birkelund Jensen --- hw/block/nvme.c | 8 +++- 1

[Qemu-block] [PATCH 07/16] nvme: support Abort command

2019-07-05 Thread Klaus Birkelund Jensen
Required for compliance with NVMe revision 1.2.1. See NVM Express 1.2.1, Section 5.1 ("Abort command"). Extracted from Keith's qemu-nvme tree. Modified to only consider queued and not executing commands. Signed-off-by: Klaus Birkelund Jensen --- hw/block/nvme.c | 56

[Qemu-block] [PATCH 11/16] nvme: add missing mandatory Features

2019-07-05 Thread Klaus Birkelund Jensen
Add support for returning a resonable response to Get/Set Features of mandatory features. Signed-off-by: Klaus Birkelund Jensen --- hw/block/nvme.c | 49 --- hw/block/trace-events | 2 ++ include/block/nvme.h | 3 ++- 3 files changed, 50

[Qemu-block] [PATCH 15/16] nvme: support scatter gather lists

2019-07-05 Thread Klaus Birkelund Jensen
For now, support the Data Block, Segment and Last Segment descriptor types. See NVM Express 1.3d, Section 4.4 ("Scatter Gather List (SGL)"). Signed-off-by: Klaus Birkelund Jensen --- block/nvme.c | 18 +- hw/block/nvme.c | 390 +++---

[Qemu-block] [PATCH 06/16] nvme: support completion queue in cmb

2019-07-05 Thread Klaus Birkelund Jensen
While not particularly useful, allow completion queues in the controller memory buffer. Could be useful for testing. Signed-off-by: Klaus Birkelund Jensen --- hw/block/nvme.c | 16 +--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c

[Qemu-block] [PATCH 00/16] nvme: support NVMe v1.3d, SGLs and multiple namespaces

2019-07-05 Thread Klaus Birkelund Jensen
Matt Fitzpatrick's post ("[RFC,v1] Namespace Management Support") pushed me to finally get my head out of my a** and post this series. This is basically a follow-up to my previous series ("nvme: v1.3, sgls, metadata and new 'ocssd' device"), but I'm not tagging it as a v2 because the patches for

[Qemu-block] [PATCH 02/16] nvme: move device parameters to separate struct

2019-07-05 Thread Klaus Birkelund Jensen
Move device configuration parameters to separate struct to make it explicit what is configurable and what is set internally. Also, clean up some includes. Signed-off-by: Klaus Birkelund Jensen --- hw/block/nvme.c | 54 +++-- hw/block/nvme.h | 16

[Qemu-block] [PATCH 04/16] nvme: add missing fields in identify controller

2019-07-05 Thread Klaus Birkelund Jensen
Not used by the device model but added for completeness. See NVM Express 1.2.1, Section 5.11 ("Identify command"), Figure 90. Signed-off-by: Klaus Birkelund Jensen --- include/block/nvme.h | 34 +- 1 file changed, 29 insertions(+), 5 deletions(-) diff --git

[Qemu-block] [PATCH 10/16] nvme: support Get Log Page command

2019-07-05 Thread Klaus Birkelund Jensen
Add support for the Get Log Page command and stub/dumb implementations of the mandatory Error Information, SMART/Health Information and Firmware Slot Information log pages. Required for compliance with NVMe revision 1.2.1. See NVM Express 1.2.1, Section 5.10 ("Get Log Page command").

[Qemu-block] [PATCH 03/16] nvme: fix lpa field

2019-07-05 Thread Klaus Birkelund Jensen
The Log Page Attributes in the Identify Controller structure indicates that the controller supports the SMART / Health Information log page on a per namespace basis. It does not, given that neither this log page or the Get Log Page command is implemented. Signed-off-by: Klaus Birkelund Jensen

[Qemu-block] [PATCH 01/16] nvme: simplify namespace code

2019-07-05 Thread Klaus Birkelund Jensen
The device model currently only supports a single namespace and also specifically sets num_namespaces to 1. Take this into account and simplify the code. Signed-off-by: Klaus Birkelund Jensen --- hw/block/nvme.c | 26 +++--- hw/block/nvme.h | 2 +- 2 files changed, 8

[Qemu-block] [PATCH 08/16] nvme: refactor device realization

2019-07-05 Thread Klaus Birkelund Jensen
Signed-off-by: Klaus Birkelund Jensen --- hw/block/nvme.c | 196 ++-- hw/block/nvme.h | 11 +++ 2 files changed, 152 insertions(+), 55 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index 4b9ff51868c0..eb6af6508e2d 100644 ---