Re: [RFC PATCH v3 00/27] Add subcluster allocation to qcow2

2020-02-22 Thread Alberto Garcia
On Fri 21 Feb 2020 06:10:52 PM CET, Max Reitz wrote: > So now I wonder on what your plans are after this series. Apart from some fixes here and there, there are some things that I would live to solve: - I'm not 100% happy with the separation between QCow2ClusterType and QCow2SubclusterType.

Re: [PATCH v3 0/3] Dump QCOW2 metadata

2020-02-22 Thread Eric Blake
On 2/20/20 6:28 AM, Kevin Wolf wrote: Am 20.02.2020 um 12:58 hat Max Reitz geschrieben: On 14.01.20 09:22, Andrey Shinkevich wrote: The information about QCOW2 metadata allocations in an image ELF-file is helpful for finding issues with the image data integrity. Sorry that I’m replying only

Re: [PULL 24/31] fuzz: support for fork-based fuzzing.

2020-02-22 Thread Eric Blake
On 2/22/20 2:50 AM, Stefan Hajnoczi wrote: From: Alexander Bulekov fork() is a simple way to ensure that state does not leak in between fuzzing runs. Unfortunately, the fuzzer mutation engine relies on bitmaps which contain coverage information for each fuzzing run, and these bitmaps should be

[PATCH 3/3] qemu-img: Deprecate use of -b without -F

2020-02-22 Thread Eric Blake
Creating an image that requires format probing of the backing image is inherently unsafe (we've had several CVEs over the years based on probes leaking information to the guest on a subsequent boot). If our probing algorithm ever changes, or if other tools like libvirt determine a different probe

[PATCH 0/3] Tighten qemu-img rules on missing backing format

2020-02-22 Thread Eric Blake
In the past, we have had CVEs caused by qemu probing one image type when an image started out as another but the guest was able to modify content. The solution to those CVEs was to encode backing format information into qcow2, to ensure that once we make a decision, we don't have to probe any

[PATCH 2/3] block: Add support to warn on backing file change without format

2020-02-22 Thread Eric Blake
For now, this is a mechanical addition; all callers pass false. But the next patch will use it to improve 'qemu-img rebase -u' when selecting a backing file with no format. Signed-off-by: Eric Blake --- block.c | 14 +++--- block/qcow2.c | 2 +- block/stream.c

Re: [PULL 00/31] Block patches

2020-02-22 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200222085030.1760640-1-stefa...@redhat.com/ Hi, This series seems to have some coding style problems. See output below for more information: Subject: [PULL 00/31] Block patches Message-id: 20200222085030.1760640-1-stefa...@redhat.com Type: series ===

[PULL 30/31] fuzz: add virtio-scsi fuzz target

2020-02-22 Thread Stefan Hajnoczi
From: Alexander Bulekov The virtio-scsi fuzz target sets up and fuzzes the available virtio-scsi queues. After an element is placed on a queue, the fuzzer can select whether to perform a kick, or continue adding elements. Signed-off-by: Alexander Bulekov Reviewed-by: Darren Kenny Message-id:

[PULL 24/31] fuzz: support for fork-based fuzzing.

2020-02-22 Thread Stefan Hajnoczi
From: Alexander Bulekov fork() is a simple way to ensure that state does not leak in between fuzzing runs. Unfortunately, the fuzzer mutation engine relies on bitmaps which contain coverage information for each fuzzing run, and these bitmaps should be copied from the child to the parent(where

[PULL 20/31] libqos: move useful qos-test funcs to qos_external

2020-02-22 Thread Stefan Hajnoczi
From: Alexander Bulekov The moved functions are not specific to qos-test and might be useful elsewhere. For example the virtual-device fuzzer makes use of them for qos-assisted fuzz-targets. Signed-off-by: Alexander Bulekov Reviewed-by: Stefan Hajnoczi Reviewed-by: Philippe Mathieu-Daudé

[PULL 29/31] fuzz: add virtio-net fuzz target

2020-02-22 Thread Stefan Hajnoczi
From: Alexander Bulekov The virtio-net fuzz target feeds inputs to all three virtio-net virtqueues, and uses forking to avoid leaking state between fuzz runs. Signed-off-by: Alexander Bulekov Reviewed-by: Stefan Hajnoczi Reviewed-by: Darren Kenny Message-id:

[PULL 31/31] fuzz: add documentation to docs/devel/

2020-02-22 Thread Stefan Hajnoczi
From: Alexander Bulekov Signed-off-by: Alexander Bulekov Reviewed-by: Stefan Hajnoczi Reviewed-by: Darren Kenny Message-id: 20200220041118.23264-23-alx...@bu.edu Signed-off-by: Stefan Hajnoczi --- docs/devel/fuzzing.txt | 116 + 1 file changed, 116

[PULL 26/31] fuzz: add target/fuzz makefile rules

2020-02-22 Thread Stefan Hajnoczi
From: Alexander Bulekov Signed-off-by: Alexander Bulekov Reviewed-by: Darren Kenny Reviewed-by: Stefan Hajnoczi Message-id: 20200220041118.23264-18-alx...@bu.edu Signed-off-by: Stefan Hajnoczi --- Makefile| 15 ++- Makefile.target | 16 2 files changed,

[PULL 27/31] fuzz: add configure flag --enable-fuzzing

2020-02-22 Thread Stefan Hajnoczi
From: Alexander Bulekov Signed-off-by: Alexander Bulekov Reviewed-by: Stefan Hajnoczi Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Darren Kenny Message-id: 20200220041118.23264-19-alx...@bu.edu Signed-off-by: Stefan Hajnoczi --- configure | 39 +++ 1

[PULL 25/31] fuzz: add support for qos-assisted fuzz targets

2020-02-22 Thread Stefan Hajnoczi
From: Alexander Bulekov Signed-off-by: Alexander Bulekov Reviewed-by: Stefan Hajnoczi Reviewed-by: Darren Kenny Message-id: 20200220041118.23264-17-alx...@bu.edu Signed-off-by: Stefan Hajnoczi --- tests/qtest/fuzz/Makefile.include | 2 + tests/qtest/fuzz/qos_fuzz.c | 234

[PULL 22/31] exec: keep ram block across fork when using qtest

2020-02-22 Thread Stefan Hajnoczi
From: Alexander Bulekov Ram blocks were marked MADV_DONTFORK breaking fuzzing-tests which execute each test-input in a forked process. Signed-off-by: Alexander Bulekov Reviewed-by: Stefan Hajnoczi Reviewed-by: Darren Kenny Message-id: 20200220041118.23264-14-alx...@bu.edu Signed-off-by:

[PULL 28/31] fuzz: add i440fx fuzz targets

2020-02-22 Thread Stefan Hajnoczi
From: Alexander Bulekov These three targets should simply fuzz reads/writes to a couple ioports, but they mostly serve as examples of different ways to write targets. They demonstrate using qtest and qos for fuzzing, as well as using rebooting and forking to reset state, or not resetting it at

[PULL 09/31] aio-posix: make AioHandler dispatch O(1) with epoll

2020-02-22 Thread Stefan Hajnoczi
File descriptor monitoring is O(1) with epoll(7), but aio_dispatch_handlers() still scans all AioHandlers instead of dispatching just those that are ready. This makes aio_poll() O(n) with respect to the total number of registered handlers. Add a local ready_list to aio_poll() so that each nested

[PULL 18/31] libqos: rename i2c_send and i2c_recv

2020-02-22 Thread Stefan Hajnoczi
From: Alexander Bulekov The names i2c_send and i2c_recv collide with functions defined in hw/i2c/core.c. This causes an error when linking against libqos and softmmu simultaneously (for example when using qtest inproc). Rename the libqos functions to avoid this. Signed-off-by: Alexander Bulekov

[PULL 19/31] libqos: split qos-test and libqos makefile vars

2020-02-22 Thread Stefan Hajnoczi
From: Alexander Bulekov Most qos-related objects were specified in the qos-test-obj-y variable. qos-test-obj-y also included qos-test.o which defines a main(). This made it difficult to repurpose qos-test-obj-y to link anything beside tests/qos-test against libqos. This change separates objects

[PULL 21/31] fuzz: add fuzzer skeleton

2020-02-22 Thread Stefan Hajnoczi
From: Alexander Bulekov tests/fuzz/fuzz.c serves as the entry point for the virtual-device fuzzer. Namely, libfuzzer invokes the LLVMFuzzerInitialize and LLVMFuzzerTestOneInput functions, both of which are defined in this file. This change adds a "FuzzTarget" struct, along with the

[PULL 16/31] libqtest: make bufwrite rely on the TransportOps

2020-02-22 Thread Stefan Hajnoczi
From: Alexander Bulekov When using qtest "in-process" communication, qtest_sendf directly calls a function in the server (qtest.c). Previously, bufwrite used socket_send, which bypasses the TransportOps enabling the call into qtest.c. This change replaces the socket_send calls with ops->send,

[PULL 08/31] aio-posix: make AioHandler deletion O(1)

2020-02-22 Thread Stefan Hajnoczi
It is not necessary to scan all AioHandlers for deletion. Keep a list of deleted handlers instead of scanning the full list of all handlers. The AioHandler->deleted field can be dropped. Let's check if the handler has been inserted into the deleted list instead. Add a new QLIST_IS_INSERTED()

[PULL 14/31] qtest: add qtest_server_send abstraction

2020-02-22 Thread Stefan Hajnoczi
From: Alexander Bulekov qtest_server_send is a function pointer specifying the handler used to transmit data to the qtest client. In the standard configuration, this calls the CharBackend handler, but now it is possible for other types of handlers, e.g direct-function calls if the qtest client

[PULL 23/31] main: keep rcu_atfork callback enabled for qtest

2020-02-22 Thread Stefan Hajnoczi
From: Alexander Bulekov The qtest-based fuzzer makes use of forking to reset-state between tests. Keep the callback enabled, so the call_rcu thread gets created within the child process. Signed-off-by: Alexander Bulekov Reviewed-by: Darren Kenny Acked-by: Stefan Hajnoczi Message-id:

[PULL 06/31] aio-posix: don't pass ns timeout to epoll_wait()

2020-02-22 Thread Stefan Hajnoczi
Don't pass the nanosecond timeout into epoll_wait(), which expects milliseconds. The epoll_wait() timeout value does not matter if qemu_poll_ns() determined that the poll fd is ready, but passing a value in the wrong units is still ugly. Pass a 0 timeout to epoll_wait() instead. Signed-off-by:

[PULL 17/31] qtest: add in-process incoming command handler

2020-02-22 Thread Stefan Hajnoczi
From: Alexander Bulekov The handler allows a qtest client to send commands to the server by directly calling a function, rather than using a file/CharBackend Signed-off-by: Alexander Bulekov Reviewed-by: Stefan Hajnoczi Reviewed-by: Darren Kenny Message-id:

[PULL 05/31] aio-posix: fix use after leaving scope in aio_poll()

2020-02-22 Thread Stefan Hajnoczi
epoll_handler is a stack variable and must not be accessed after it goes out of scope: if (aio_epoll_check_poll(ctx, pollfds, npfd, timeout)) { AioHandler epoll_handler; ... add_pollfd(_handler); ret = aio_epoll(ctx, pollfds, npfd, timeout); }

[PULL 12/31] module: check module wasn't already initialized

2020-02-22 Thread Stefan Hajnoczi
From: Alexander Bulekov The virtual-device fuzzer must initialize QOM, prior to running vl:qemu_init, so that it can use the qos_graph to identify the arguments required to initialize a guest for libqos-assisted fuzzing. This change prevents errors when vl:qemu_init tries to (re)initialize the

[PULL 15/31] libqtest: add a layer of abstraction to send/recv

2020-02-22 Thread Stefan Hajnoczi
From: Alexander Bulekov This makes it simple to swap the transport functions for qtest commands to and from the qtest client. For example, now it is possible to directly pass qtest commands to a server handler that exists within the same process, without the standard way of writing to a file

[PULL 04/31] util/async: make bh_aio_poll() O(1)

2020-02-22 Thread Stefan Hajnoczi
The ctx->first_bh list contains all created BHs, including those that are not scheduled. The list is iterated by the event loop and therefore has O(n) time complexity with respected to the number of created BHs. Rewrite BHs so that only scheduled or deleted BHs are enqueued. Only BHs that

[PULL 13/31] fuzz: add FUZZ_TARGET module type

2020-02-22 Thread Stefan Hajnoczi
From: Alexander Bulekov Signed-off-by: Alexander Bulekov Reviewed-by: Stefan Hajnoczi Reviewed-by: Darren Kenny Message-id: 20200220041118.23264-5-alx...@bu.edu Signed-off-by: Stefan Hajnoczi --- include/qemu/module.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git

[PULL 11/31] softmmu: split off vl.c:main() into main.c

2020-02-22 Thread Stefan Hajnoczi
From: Alexander Bulekov A program might rely on functions implemented in vl.c, but implement its own main(). By placing main into a separate source file, there are no complaints about duplicate main()s when linking against vl.o. For example, the virtual-device fuzzer uses a main() provided by

[PULL 10/31] softmmu: move vl.c to softmmu/

2020-02-22 Thread Stefan Hajnoczi
From: Alexander Bulekov Move vl.c to a separate directory, similar to linux-user/ Update the chechpatch and get_maintainer scripts, since they relied on /vl.c for top_of_tree checks. Signed-off-by: Alexander Bulekov Reviewed-by: Darren Kenny Message-id: 20200220041118.23264-2-alx...@bu.edu

[PULL 03/31] rcu_queue: add QSLIST functions

2020-02-22 Thread Stefan Hajnoczi
From: Paolo Bonzini QSLIST is the only family of lists for which we do not have RCU-friendly accessors, add them. Signed-off-by: Paolo Bonzini Reviewed-by: Stefan Hajnoczi Message-id: 20200220103828.24525-1-pbonz...@redhat.com Signed-off-by: Stefan Hajnoczi --- include/qemu/queue.h |

[PULL 07/31] qemu/queue.h: add QLIST_SAFE_REMOVE()

2020-02-22 Thread Stefan Hajnoczi
QLIST_REMOVE() assumes the element is in a list. It also leaves the element's linked list pointers dangling. Introduce a safe version of QLIST_REMOVE() and convert open-coded instances of this pattern. Signed-off-by: Stefan Hajnoczi Reviewed-by: Sergio Lopez Message-id:

[PULL 01/31] virtio: increase virtqueue size for virtio-scsi and virtio-blk

2020-02-22 Thread Stefan Hajnoczi
From: Denis Plotnikov The goal is to reduce the amount of requests issued by a guest on 1M reads/writes. This rises the performance up to 4% on that kind of disk access pattern. The maximum chunk size to be used for the guest disk accessing is limited with seg_max parameter, which represents

[PULL 00/31] Block patches

2020-02-22 Thread Stefan Hajnoczi
The following changes since commit 9ac5df20f51fabcba0d902025df4bd7ea987c158: Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20200221-1' into staging (2020-02-21 16:18:38 +) are available in the Git repository at: https://github.com/stefanha/qemu.git

[PULL 02/31] aio-posix: avoid reacquiring rcu_read_lock() when polling

2020-02-22 Thread Stefan Hajnoczi
The first rcu_read_lock/unlock() is expensive. Nested calls are cheap. This optimization increases IOPS from 73k to 162k with a Linux guest that has 2 virtio-blk,num-queues=1 and 99 virtio-blk,num-queues=32 devices. Signed-off-by: Stefan Hajnoczi Reviewed-by: Paolo Bonzini Message-id:

Re: [PATCH v7 01/11] qapi/error: add (Error **errp) cleaning APIs

2020-02-22 Thread Markus Armbruster
Vladimir Sementsov-Ogievskiy writes: > 21.02.2020 19:34, Markus Armbruster wrote: >> Vladimir Sementsov-Ogievskiy writes: >> >>> 21.02.2020 10:38, Markus Armbruster wrote: Vladimir Sementsov-Ogievskiy writes: > Add functions to clean Error **errp: call corresponding Error *err