Re: [Qemu-devel] [PULL 00/41] Block layer patches

[Peter Maydell]

On Wed, 12 Dec 2018 at 13:27, Kevin Wolf  wrote:
>
> The following changes since commit bb9bf94b3e8926553290bc9a7cb84315af422086:
>
>   Merge remote-tracking branch 
> 'remotes/ehabkost/tags/machine-next-pull-request' into staging (2018-12-11 
> 19:18:58 +)
>
> are available in the Git repository at:
>
>   git://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to 14a8882c0d6660684a0ec7e90cc75d4f38b07f0d:
>
>   iotests: make 235 work on s390 (and others) (2018-12-12 12:22:31 +0100)
>
> 
> Block layer patches:
>
> - qcow2: Decompression worker threads
> - dmg: lzfse compression support
> - file-posix: Simplify delegation to worker thread
> - Don't pass flags to bdrv_reopen_queue()
> - iotests: make 235 work on s390 (and others)


Hi; I get a new warning on FreeBSD, NetBSD, OpenBSD and OSX:
block/file-posix.c:1155:12: warning: 'handle_aiocb_ioctl' defined but
not used [-Wunused-function]
 static int handle_aiocb_ioctl(void *opaque)
^

This function is used only from code guarded by #ifdef __linux__
but the function definition itself is not ifdeffed.

thanks
-- PMM

[Qemu-devel] [PULL 00/41] Block layer patches

[Kevin Wolf]

The following changes since commit bb9bf94b3e8926553290bc9a7cb84315af422086:

  Merge remote-tracking branch 
'remotes/ehabkost/tags/machine-next-pull-request' into staging (2018-12-11 
19:18:58 +)

are available in the Git repository at:

  git://repo.or.cz/qemu/kevin.git tags/for-upstream

for you to fetch changes up to 14a8882c0d6660684a0ec7e90cc75d4f38b07f0d:

  iotests: make 235 work on s390 (and others) (2018-12-12 12:22:31 +0100)


Block layer patches:

- qcow2: Decompression worker threads
- dmg: lzfse compression support
- file-posix: Simplify delegation to worker thread
- Don't pass flags to bdrv_reopen_queue()
- iotests: make 235 work on s390 (and others)


Alberto Garcia (15):
  block: Add bdrv_reopen_set_read_only()
  block: Use bdrv_reopen_set_read_only() in bdrv_backing_update_filename()
  block: Use bdrv_reopen_set_read_only() in commit_start/complete()
  block: Use bdrv_reopen_set_read_only() in bdrv_commit()
  block: Use bdrv_reopen_set_read_only() in stream_start/complete()
  block: Use bdrv_reopen_set_read_only() in qmp_change_backing_file()
  block: Use bdrv_reopen_set_read_only() in external_snapshot_commit()
  block: Use bdrv_reopen_set_read_only() in the mirror driver
  block: Drop bdrv_reopen()
  qemu-io: Put flag changes in the options QDict in reopen_f()
  block: Clean up reopen_backing_file() in block/replication.c
  block: Remove flags parameter from bdrv_reopen_queue()
  block: Stop passing flags to bdrv_reopen_queue_child()
  block: Remove assertions from update_flags_from_options()
  block: Assert that flags are up-to-date in bdrv_reopen_prepare()

Christian Borntraeger (1):
  iotests: make 235 work on s390 (and others)

Julio Faracco (4):
  block: adding lzfse decompressing support as a module.
  configure: adding support to lzfse library.
  dmg: including dmg-lzfse module inside dmg block driver.
  dmg: exchanging hardcoded dmg UDIF block types to enum.

Kevin Wolf (12):
  file-posix: Reorganise RawPosixAIOData
  file-posix: Factor out raw_thread_pool_submit()
  file-posix: Avoid aio_worker() for QEMU_AIO_TRUNCATE
  file-posix: Avoid aio_worker() for QEMU_AIO_COPY_RANGE
  file-posix: Avoid aio_worker() for QEMU_AIO_WRITE_ZEROES
  file-posix: Avoid aio_worker() for QEMU_AIO_DISCARD
  file-posix: Avoid aio_worker() for QEMU_AIO_FLUSH
  file-posix: Move read/write operation logic out of aio_worker()
  file-posix: Avoid aio_worker() for QEMU_AIO_READ/WRITE
  file-posix: Remove paio_submit_co()
  file-posix: Switch to .bdrv_co_ioctl
  file-posix: Avoid aio_worker() for QEMU_AIO_IOCTL

Vladimir Sementsov-Ogievskiy (9):
  block/replication: drop extra synchronization
  block/backup: drop unused synchronization interface
  qcow2: use Z_OK instead of 0 for deflateInit2 return code check
  qcow2: make more generic interface for qcow2_compress
  qcow2: move decompression from qcow2-cluster.c to qcow2.c
  qcow2: refactor decompress_buffer
  qcow2: use byte-based read in qcow2_decompress_cluster
  qcow2: aio support for compressed cluster read
  qcow2: do decompression in threads

 configure|  31 
 block/dmg.h  |   3 +
 block/qcow2.h|   4 -
 include/block/block.h|   6 +-
 include/block/block_backup.h |  13 --
 include/scsi/pr-manager.h|   8 +-
 block.c  |  89 +-
 block/backup.c   |  38 +
 block/commit.c   |  23 +--
 block/dmg-lzfse.c|  49 ++
 block/dmg.c  |  65 ++--
 block/file-posix.c   | 380 ++-
 block/mirror.c   |  19 ++-
 block/qcow2-cluster.c|  70 
 block/qcow2.c| 170 ---
 block/replication.c  |  67 +++-
 block/stream.c   |  20 +--
 blockdev.c   |  11 +-
 qemu-io-cmds.c   |  29 +++-
 scsi/pr-manager.c|  21 +--
 block/Makefile.objs  |   2 +
 scsi/trace-events|   2 +-
 tests/qemu-iotests/133   |  18 ++
 tests/qemu-iotests/133.out   |  15 ++
 tests/qemu-iotests/235   |   4 +-
 25 files changed, 658 insertions(+), 499 deletions(-)
 create mode 100644 block/dmg-lzfse.c

Re: [Qemu-devel] [PULL 00/41] Block layer patches

[Kevin Wolf]

Am 15.03.2018 um 18:55 hat John Snow geschrieben:
> 
> 
> On 03/15/2018 12:56 PM, Kevin Wolf wrote:
> > Am 15.03.2018 um 17:42 hat Peter Maydell geschrieben:
> >> On 13 March 2018 at 16:17, Kevin Wolf  wrote:
> >>> The following changes since commit 
> >>> 22ef7ba8e8ce7fef297549b3defcac333742b804:
> >>>
> >>>   Merge remote-tracking branch 'remotes/famz/tags/staging-pull-request' 
> >>> into staging (2018-03-13 11:42:45 +)
> >>>
> >>> are available in the git repository at:
> >>>
> >>>   git://repo.or.cz/qemu/kevin.git tags/for-upstream
> >>>
> >>> for you to fetch changes up to be6c885842efded81a20f4ca24f0d4e123a80c00:
> >>>
> >>>   block/mirror: change the semantic of 'force' of block-job-cancel 
> >>> (2018-03-13 16:54:47 +0100)
> >>>
> >>> 
> >>> Block layer patches
> >>>
> >>> 
> >>
> >> I get a compile failure here on some hosts:
> >>
> >> /home/pm215/qemu/blockjob.c: In function ‘block_job_txn_apply.isra.8’:
> >> /home/pm215/qemu/blockjob.c:524:5: error: ‘rc’ may be used
> >> uninitialized in this function [-Werror=maybe-uninitialized]
> >>  return rc;
> >>  ^
> >>
> >> I guess the compiler can't always figure out whether there is
> >> guaranteed to be at least one thing in the list ?
> > 
> > I think so.
> > 
> > John, I'll just modify your patch 'blockjobs: add prepare callback' to
> > initialise rc = 0 in this function and send a v2 pull request.
> > 
> > Kevin
> > 
> 
> Oh, interesting. I guess technically the list COULD be empty and that'd
> be perfectly valid. I wonder why my compiler doesn't complain?
> 
> Anyway, initializing to zero seems like the correct behavior, thanks.

You only call block_job_txn_apply() in the context of completing a
specific job, which has to be in its transaction, so I don't think the
list can actually be empty.

Not sure how the compiler is able to infer that, though.

Kevin

Re: [Qemu-devel] [PULL 00/41] Block layer patches

[John Snow]

On 03/15/2018 12:56 PM, Kevin Wolf wrote:
> Am 15.03.2018 um 17:42 hat Peter Maydell geschrieben:
>> On 13 March 2018 at 16:17, Kevin Wolf  wrote:
>>> The following changes since commit 22ef7ba8e8ce7fef297549b3defcac333742b804:
>>>
>>>   Merge remote-tracking branch 'remotes/famz/tags/staging-pull-request' 
>>> into staging (2018-03-13 11:42:45 +)
>>>
>>> are available in the git repository at:
>>>
>>>   git://repo.or.cz/qemu/kevin.git tags/for-upstream
>>>
>>> for you to fetch changes up to be6c885842efded81a20f4ca24f0d4e123a80c00:
>>>
>>>   block/mirror: change the semantic of 'force' of block-job-cancel 
>>> (2018-03-13 16:54:47 +0100)
>>>
>>> 
>>> Block layer patches
>>>
>>> 
>>
>> I get a compile failure here on some hosts:
>>
>> /home/pm215/qemu/blockjob.c: In function ‘block_job_txn_apply.isra.8’:
>> /home/pm215/qemu/blockjob.c:524:5: error: ‘rc’ may be used
>> uninitialized in this function [-Werror=maybe-uninitialized]
>>  return rc;
>>  ^
>>
>> I guess the compiler can't always figure out whether there is
>> guaranteed to be at least one thing in the list ?
> 
> I think so.
> 
> John, I'll just modify your patch 'blockjobs: add prepare callback' to
> initialise rc = 0 in this function and send a v2 pull request.
> 
> Kevin
> 

Oh, interesting. I guess technically the list COULD be empty and that'd
be perfectly valid. I wonder why my compiler doesn't complain?

Anyway, initializing to zero seems like the correct behavior, thanks.

--js

Re: [Qemu-devel] [PULL 00/41] Block layer patches

[Kevin Wolf]

Am 15.03.2018 um 17:42 hat Peter Maydell geschrieben:
> On 13 March 2018 at 16:17, Kevin Wolf  wrote:
> > The following changes since commit 22ef7ba8e8ce7fef297549b3defcac333742b804:
> >
> >   Merge remote-tracking branch 'remotes/famz/tags/staging-pull-request' 
> > into staging (2018-03-13 11:42:45 +)
> >
> > are available in the git repository at:
> >
> >   git://repo.or.cz/qemu/kevin.git tags/for-upstream
> >
> > for you to fetch changes up to be6c885842efded81a20f4ca24f0d4e123a80c00:
> >
> >   block/mirror: change the semantic of 'force' of block-job-cancel 
> > (2018-03-13 16:54:47 +0100)
> >
> > 
> > Block layer patches
> >
> > 
> 
> I get a compile failure here on some hosts:
> 
> /home/pm215/qemu/blockjob.c: In function ‘block_job_txn_apply.isra.8’:
> /home/pm215/qemu/blockjob.c:524:5: error: ‘rc’ may be used
> uninitialized in this function [-Werror=maybe-uninitialized]
>  return rc;
>  ^
> 
> I guess the compiler can't always figure out whether there is
> guaranteed to be at least one thing in the list ?

I think so.

John, I'll just modify your patch 'blockjobs: add prepare callback' to
initialise rc = 0 in this function and send a v2 pull request.

Kevin

Re: [Qemu-devel] [PULL 00/41] Block layer patches

[Peter Maydell]

On 13 March 2018 at 16:17, Kevin Wolf  wrote:
> The following changes since commit 22ef7ba8e8ce7fef297549b3defcac333742b804:
>
>   Merge remote-tracking branch 'remotes/famz/tags/staging-pull-request' into 
> staging (2018-03-13 11:42:45 +)
>
> are available in the git repository at:
>
>   git://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to be6c885842efded81a20f4ca24f0d4e123a80c00:
>
>   block/mirror: change the semantic of 'force' of block-job-cancel 
> (2018-03-13 16:54:47 +0100)
>
> 
> Block layer patches
>
> 

I get a compile failure here on some hosts:

/home/pm215/qemu/blockjob.c: In function ‘block_job_txn_apply.isra.8’:
/home/pm215/qemu/blockjob.c:524:5: error: ‘rc’ may be used
uninitialized in this function [-Werror=maybe-uninitialized]
 return rc;
 ^

I guess the compiler can't always figure out whether there is
guaranteed to be at least one thing in the list ?

thanks
-- PMM

Re: [Qemu-devel] [PULL 00/41] Block layer patches

[no-reply]

Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: 20180313161803.1814-1-kw...@redhat.com
Subject: [Qemu-devel] [PULL 00/41] Block layer patches

=== TEST SCRIPT BEGIN ===
#!/bin/bash

BASE=base
n=1
total=$(git log --oneline $BASE.. | wc -l)
failed=0

git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram

commits="$(git log --format=%H --reverse $BASE..)"
for c in $commits; do
echo "Checking PATCH $n/$total: $(git log -n 1 --format=%s $c)..."
if ! git show $c --format=email | ./scripts/checkpatch.pl --mailback -; then
failed=1
echo
fi
n=$((n+1))
done

exit $failed
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 t [tag update]patchew/20180307082512.14203-1-be...@igalia.com -> 
patchew/20180307082512.14203-1-be...@igalia.com
 t [tag update]
patchew/20180313153458.26822-1-peter.mayd...@linaro.org -> 
patchew/20180313153458.26822-1-peter.mayd...@linaro.org
 * [new tag]   patchew/20180313161803.1814-1-kw...@redhat.com -> 
patchew/20180313161803.1814-1-kw...@redhat.com
Switched to a new branch 'test'
73ceee78e6 block/mirror: change the semantic of 'force' of block-job-cancel
7363482c0a vpc: Require aligned size in .bdrv_co_create
2882d5ed9b vpc: Support .bdrv_co_create
fc6a997d9c vhdx: Support .bdrv_co_create
20ef05f192 vdi: Make comments consistent with other drivers
459ee653e4 qed: Support .bdrv_co_create
8bba4791b7 qcow: Support .bdrv_co_create
f64c119db2 qemu-iotests: Enable write tests for parallels
2080c0a1ab parallels: Support .bdrv_co_create
9faa105c59 iotests: Add regression test for commit base locking
6a296d9cfe block: Fix flags in reopen queue
781f48c549 vdi: Implement .bdrv_co_create
4cab0e18bb vdi: Move file creation to vdi_co_create_opts
891969da22 vdi: Pull option parsing from vdi_co_create
49280fb721 qemu-iotests: Test luks QMP image creation
0a4d72fa21 luks: Catch integer overflow for huge sizes
6fda8b9a38 luks: Turn invalid assertion into check
c88dc7ac6e luks: Support .bdrv_co_create
681d5dff50 luks: Create block_crypto_co_create_generic()
d641340c5a luks: Separate image file creation from formatting
8cae2fd8e8 tests/test-blockjob: test cancellations
1390b7c37d iotests: test manual job dismissal
1ad1823194 blockjobs: Expose manual property
55441ac858 blockjobs: add block-job-finalize
4a6e1bfbb0 blockjobs: add PENDING status and event
ca394bb9c1 blockjobs: add waiting status
92017bd151 blockjobs: add prepare callback
ef89eb33ad blockjobs: add block_job_txn_apply function
225d9d25ba blockjobs: add commit, abort, clean helpers
2de3034128 blockjobs: ensure abort is called for cancelled jobs
37ef0263ce blockjobs: add block_job_dismiss
b59095a50b blockjobs: add NULL state
4c49f9fa27 blockjobs: add CONCLUDED state
7a4f169154 blockjobs: add ABORTING state
ac30b7288c blockjobs: add block_job_verb permission table
068f7c2061 iotests: add pause_wait
8c386101bf blockjobs: add state transition table
d8d8ffcb3d blockjobs: add status enum
99b5fa3cf0 Blockjobs: documentation touchup
318dc73f7e blockjobs: model single jobs as transactions
020497053e blockjobs: fix set-speed kick

=== OUTPUT BEGIN ===
Checking PATCH 1/41: blockjobs: fix set-speed kick...
Checking PATCH 2/41: blockjobs: model single jobs as transactions...
Checking PATCH 3/41: Blockjobs: documentation touchup...
Checking PATCH 4/41: blockjobs: add status enum...
Checking PATCH 5/41: blockjobs: add state transition table...
ERROR: space prohibited before open square bracket '['
#82: FILE: blockjob.c:48:
+/* U: */ [BLOCK_JOB_STATUS_UNDEFINED] = {0, 1, 0, 0, 0, 0},

ERROR: space prohibited before open square bracket '['
#83: FILE: blockjob.c:49:
+/* C: */ [BLOCK_JOB_STATUS_CREATED]   = {0, 0, 1, 0, 0, 0},

ERROR: space prohibited before open square bracket '['
#84: FILE: blockjob.c:50:
+/* R: */ [BLOCK_JOB_STATUS_RUNNING]   = {0, 0, 0, 1, 1, 0},

ERROR: space prohibited before open square bracket '['
#85: FILE: blockjob.c:51:
+/* P: */ [BLOCK_JOB_STATUS_PAUSED]= {0, 0, 1, 0, 0, 0},

ERROR: space prohibited before open square bracket '['
#86: FILE: blockjob.c:52:
+/* Y: */ [BLOCK_JOB_STATUS_READY] = {0, 0, 0, 0, 0, 1},

ERROR: space prohibited before open square bracket '['
#87: FILE: blockjob.c:53:
+/* S: */ [BLOCK_JOB_STATUS_STANDBY]   = {0, 0, 0, 0, 1, 0},

total: 6 errors, 0 warnings, 88 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

Checking PATCH 6/41: iotests: add pause_wait...
Checking PATCH 7/41: blockjobs: add block_job_verb permission table...
Checking PATCH 8/41: blockjobs: add ABORTING state...
ERROR: space prohibited before open square bracket '['
#65: F

[Qemu-devel] [PULL 00/41] Block layer patches

[Kevin Wolf]

The following changes since commit 22ef7ba8e8ce7fef297549b3defcac333742b804:

  Merge remote-tracking branch 'remotes/famz/tags/staging-pull-request' into 
staging (2018-03-13 11:42:45 +)

are available in the git repository at:

  git://repo.or.cz/qemu/kevin.git tags/for-upstream

for you to fetch changes up to be6c885842efded81a20f4ca24f0d4e123a80c00:

  block/mirror: change the semantic of 'force' of block-job-cancel (2018-03-13 
16:54:47 +0100)


Block layer patches


Fam Zheng (2):
  block: Fix flags in reopen queue
  iotests: Add regression test for commit base locking

John Snow (21):
  blockjobs: fix set-speed kick
  blockjobs: model single jobs as transactions
  Blockjobs: documentation touchup
  blockjobs: add status enum
  blockjobs: add state transition table
  iotests: add pause_wait
  blockjobs: add block_job_verb permission table
  blockjobs: add ABORTING state
  blockjobs: add CONCLUDED state
  blockjobs: add NULL state
  blockjobs: add block_job_dismiss
  blockjobs: ensure abort is called for cancelled jobs
  blockjobs: add commit, abort, clean helpers
  blockjobs: add block_job_txn_apply function
  blockjobs: add prepare callback
  blockjobs: add waiting status
  blockjobs: add PENDING status and event
  blockjobs: add block-job-finalize
  blockjobs: Expose manual property
  iotests: test manual job dismissal
  tests/test-blockjob: test cancellations

Kevin Wolf (14):
  luks: Separate image file creation from formatting
  luks: Create block_crypto_co_create_generic()
  luks: Support .bdrv_co_create
  luks: Turn invalid assertion into check
  luks: Catch integer overflow for huge sizes
  qemu-iotests: Test luks QMP image creation
  parallels: Support .bdrv_co_create
  qemu-iotests: Enable write tests for parallels
  qcow: Support .bdrv_co_create
  qed: Support .bdrv_co_create
  vdi: Make comments consistent with other drivers
  vhdx: Support .bdrv_co_create
  vpc: Support .bdrv_co_create
  vpc: Require aligned size in .bdrv_co_create

Liang Li (1):
  block/mirror: change the semantic of 'force' of block-job-cancel

Max Reitz (3):
  vdi: Pull option parsing from vdi_co_create
  vdi: Move file creation to vdi_co_create_opts
  vdi: Implement .bdrv_co_create

 qapi/block-core.json  | 363 --
 include/block/blockjob.h  |  71 -
 include/block/blockjob_int.h  |  17 +-
 block.c   |   8 +
 block/backup.c|   5 +-
 block/commit.c|   2 +-
 block/crypto.c| 150 -
 block/mirror.c|  12 +-
 block/parallels.c | 199 +--
 block/qcow.c  | 196 +++
 block/qed.c   | 204 
 block/stream.c|   2 +-
 block/vdi.c   | 147 +
 block/vhdx.c  | 216 +++--
 block/vpc.c   | 241 +---
 blockdev.c|  71 +++--
 blockjob.c| 358 +++--
 tests/test-bdrv-drain.c   |   5 +-
 tests/test-blockjob-txn.c |  27 ++--
 tests/test-blockjob.c | 233 ++-
 block/trace-events|   7 +
 hmp-commands.hx   |   3 +-
 tests/qemu-iotests/030|   6 +-
 tests/qemu-iotests/055|  17 +-
 tests/qemu-iotests/056| 187 ++
 tests/qemu-iotests/056.out|   4 +-
 tests/qemu-iotests/109.out|  24 +--
 tests/qemu-iotests/153|  12 ++
 tests/qemu-iotests/153.out|   5 +
 tests/qemu-iotests/181|   2 +-
 tests/qemu-iotests/209| 210 
 tests/qemu-iotests/209.out| 136 
 tests/qemu-iotests/check  |   1 -
 tests/qemu-iotests/common.rc  |   2 +-
 tests/qemu-iotests/group  |   1 +
 tests/qemu-iotests/iotests.py |  12 +-
 36 files changed, 2642 insertions(+), 514 deletions(-)
 create mode 100755 tests/qemu-iotests/209
 create mode 100644 tests/qemu-iotests/209.out