Markus Armbruster writes:
> "Daniel P. Berrange" writes:
>
>> Instead of requiring all callers to go through the mutli-step
>
> multi-step
>
>> process of turning QemuOpts into a suitable QObject for visiting,
>> add a new constructor that encapsulates this logic. This will
>> allow QObjectInput
"Daniel P. Berrange" writes:
> If given an option string such as
>
> size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind
>
> the qemu_opts_to_qdict() method will currently overwrite
> the values for repeated option keys, so only the last
> value is in the returned dict:
>
> size=QString("102
On 13/10/2016 02:49, John Snow wrote:
> As context to everyone else as to why I'm going down the rabbit hole of
> trying to remove external references to AioContext at all, see
> https://lists.gnu.org/archive/html/qemu-devel/2016-10/msg00795.html
>
> On 10/07/2016 03:49 AM, Paolo Bonzini wrote:
Markus Armbruster writes:
> "Daniel P. Berrange" writes:
>
>> If given an option string such as
>>
>> size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind
>>
>> the qemu_opts_to_qdict() method will currently overwrite
>> the values for repeated option keys, so only the last
>> value is in the re
Am 28.09.2016 um 22:55 hat Max Reitz geschrieben:
> Signed-off-by: Max Reitz
Reviewed-by: Kevin Wolf
Am 28.09.2016 um 22:55 hat Max Reitz geschrieben:
> Currently, a port that is passed along with a UNIX socket path is
> silently ignored. That is not exactly ideal, it should be an error
> instead.
>
> Signed-off-by: Max Reitz
Reviewed-by: Kevin Wolf
Am 28.09.2016 um 22:55 hat Max Reitz geschrieben:
> Instead of not emitting the port in nbd_refresh_filename(), just set it
> to the default if the user did not specify it. This makes the logic a
> bit simpler.
>
> Signed-off-by: Max Reitz
Reviewed-by: Kevin Wolf
Am 28.09.2016 um 22:55 hat Max Reitz geschrieben:
> Instead of inlining this nice macro (i.e. resorting to
> qdict_put_obj(..., QOBJECT(...))), use it.
>
> Signed-off-by: Max Reitz
Reviewed-by: Kevin Wolf
Am 28.09.2016 um 22:55 hat Max Reitz geschrieben:
> Add a new option "address" to the NBD block driver which accepts a
> SocketAddress.
>
> "path", "host" and "port" are still supported as legacy options and are
> mapped to their corresponding SocketAddress representation.
>
> Signed-off-by: Max
Am 28.09.2016 um 22:55 hat Max Reitz geschrieben:
> Right now, we have four possible options that conflict with specifying
> an NBD filename, and a future patch will add another one ("address").
> This future option is a nested QDict that is flattened at this point,
> requiring us to test each opti
On 12.10.2016 21:21, Max Reitz wrote:
On 11.10.2016 14:09, Vladimir Sementsov-Ogievskiy wrote:
On 01.10.2016 17:46, Max Reitz wrote:
On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
Add dirty bitmap extension as specified in docs/specs/qcow2.txt.
For now, just mirror extension header i
Cc: Kevin for discussion of QemuOpts dotted key convention
"Daniel P. Berrange" writes:
> Currently qdict_crumple requires a totally flat QDict as its
> input. i.e. all values in the QDict must be scalar types.
>
> In order to have backwards compatibility with the OptsVisitor,
> qemu_opt_to_qdic
Am 28.09.2016 um 22:55 hat Max Reitz geschrieben:
> Drop the use of legacy options in favor of the SocketAddress
> representation, even for internal use (i.e. for storing the result of
> the filename parsing).
>
> Signed-off-by: Max Reitz
Reviewed-by: Kevin Wolf
Am 28.09.2016 um 22:55 hat Max Reitz geschrieben:
> Signed-off-by: Max Reitz
Reviewed-by: Kevin Wolf
Am 28.09.2016 um 22:55 hat Max Reitz geschrieben:
> Signed-off-by: Max Reitz
> ---
> tests/qemu-iotests/iotests.py | 8
> 1 file changed, 8 insertions(+)
>
> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
> index 3329bc1..5a2678f 100644
> --- a/tests/qemu-iot
Am 28.09.2016 um 22:56 hat Max Reitz geschrieben:
> This gives us more freedom about the fd that is passed to qemu, allowing
> us to e.g. pass sockets.
>
> Signed-off-by: Max Reitz
Reviewed-by: Kevin Wolf
Am 28.09.2016 um 22:56 hat Max Reitz geschrieben:
> By adding an optional suffix to the files used for communication with a
> VM, we can launch multiple VM instances concurrently.
>
> Signed-off-by: Max Reitz
Reviewed-by: Kevin Wolf
Am 28.09.2016 um 22:56 hat Max Reitz geschrieben:
> Signed-off-by: Max Reitz
> ---
> tests/qemu-iotests/147 | 201
> +
> tests/qemu-iotests/147.out | 5 ++
> tests/qemu-iotests/group | 1 +
> 3 files changed, 207 insertions(+)
> create mode
Am 13.10.2016 um 14:35 hat Markus Armbruster geschrieben:
> Cc: Kevin for discussion of QemuOpts dotted key convention
>
> "Daniel P. Berrange" writes:
>
> > Currently qdict_crumple requires a totally flat QDict as its
> > input. i.e. all values in the QDict must be scalar types.
> >
> > In orde
On 07.10.2016 22:24, Max Reitz wrote:
On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
Realize block bitmap stroing interface, to allow qcow2 images store
[snip]
+uint64_t end = MIN(bm_size, sector + dsc);
+uint64_t write_size =
+bdrv_dirty_bitmap_serializ
Without this change, there is a race condition in tests/test-replication.
Depending on how fast the failover job (active commit) runs, there is a
chance of two bad things happening:
1) replication_done can be called after the secondary has been closed
and hence when the BDRVReplicationState is not
Ensure that there are no changes between the last check to
bdrv_get_dirty_count and the switch to the target.
There is already a bdrv_drained_end call, we only need to ensure
that bdrv_drained_begin is not called twice.
Cc: qemu-sta...@nongnu.org
Signed-off-by: Paolo Bonzini
---
block/mirror.c
This is required to decouple block jobs from running in an
AioContext. With multiqueue block devices, a BlockDriverState
does not really belong to a single AioContext.
The solution is to first wait until all I/O operations are
complete; then loop in the main thread for the block job to
complete e
From: Fam Zheng
The "need_check_timer" is used to clear the "NEED_CHECK" flag in the
image header after a grace period once metadata update has finished. To
comply with the bdrv_drain semantics, we should make sure it remains
deleted once .bdrv_drain is called.
The change to qed_need_check_timer
Unlike tracked_requests, this field also counts throttled requests,
and remains non-zero if an AIO operation needs a BH to be "really"
completed.
With this change, it is no longer necessary to have a dummy
BdrvTrackedRequest for requests that are never serialising, and
it is no longer necessary to
These will make it possible to use nfs_get_allocated_file_size on
a file that is not in the main AioContext.
Signed-off-by: Paolo Bonzini
---
block/nfs.c | 47 +--
1 file changed, 29 insertions(+), 18 deletions(-)
diff --git a/block/nfs.c b/block/nfs.
aio_poll is not thread safe; for example bdrv_drain can hang if
the last in-flight I/O operation is completed in the I/O thread after
the main thread has checked bs->in_flight.
The bug remains latent as long as all of it is called within
aio_context_acquire/aio_context_release, but this will chang
We want the BDS event loop to run exclusively in the iothread that
owns the BDS's AioContext. This function and macro provides the
synchronization between the two event loops.
Signed-off-by: Paolo Bonzini
---
block/block-backend.c | 7 +--
block/io.c| 47 +++
bdrv_requests_pending is checking children to also wait until internal
requests (such as metadata writes) have completed. However, checking
children is in general overkill. Children requests can be of two kinds:
- requests caused by an operation on bs, e.g. a bdrv_aio_write to bs
causing a write
nfs_set_events only needs to be called once before entering the
while loop; afterwards, nfs_process_read and nfs_process_write
take care of it.
Signed-off-by: Paolo Bonzini
---
block/nfs.c | 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/block/nfs.c b/block/nfs.c
index
This patch reorganizes aio_poll callers to establish new rules for
dataplane locking. The idea is that I/O operations on a dataplane
BDS (i.e. one where the AioContext is not the main one) do not call
aio_poll anymore. Instead, they wait for the operation to end in the
other I/O thread, at which
It is simpler and a bit faster, and QEMU does not need the contention
callbacks (and thus the fairness) anymore.
Reviewed-by: Fam Zheng
Signed-off-by: Paolo Bonzini
---
async.c | 8 ++---
include/block/aio.h | 3 +-
include/qemu/rfifolock.h | 54 -
After the next patch bdrv_drain_all will have to be called without holding any
AioContext. Prepare to do this by adding an AioContext argument to
bdrv_reopen_multiple.
Signed-off-by: Paolo Bonzini
---
block.c | 4 ++--
block/commit.c| 2 +-
block/replication.c | 3 ++-
i
This is the first step towards having fine-grained critical sections in
dataplane threads, which will resolve lock ordering problems between
address_space_* functions (which need the BQL when doing MMIO, even
after we complete RCU-based dispatch) and the AioContext.
Because AioContext does not use
GRecMutex is new in glib 2.32, so we cannot use it. Introduce
a recursive mutex in qemu-thread instead, which will be used
instead of RFifoLock.
Reviewed-by: Fam Zheng
Signed-off-by: Paolo Bonzini
---
include/qemu/thread-posix.h | 6 ++
include/qemu/thread-win32.h | 10 ++
include
These ensure that bdrv_poll_while will exit for a BlockDriverState
that is not in the main AioContext.
Signed-off-by: Paolo Bonzini
---
block/sheepdog.c | 67
1 file changed, 38 insertions(+), 29 deletions(-)
diff --git a/block/sheepdog.c
This will be used by bdrv_poll_while (and thus by bdrv_drain)
to choose how to wait for I/O completion.
Reviewed-by: Fam Zheng
Signed-off-by: Paolo Bonzini
---
include/block/aio.h | 15 +++
iothread.c | 9 +
stubs/Makefile.objs | 1 +
stubs/iothread.c| 8
This will be needed in the next patch to retrieve the AioContext.
Signed-off-by: Paolo Bonzini
---
block/replication.c | 21 -
1 file changed, 12 insertions(+), 9 deletions(-)
diff --git a/block/replication.c b/block/replication.c
index 5231a00..af47479 100644
--- a/block/re
Soon bdrv_drain will not call aio_poll itself on iothreads. If block
devices are left hanging off the iothread's AioContext, there will be no
one to do I/O for those poor devices.
Signed-off-by: Paolo Bonzini
---
iothread.c | 10 ++
1 file changed, 10 insertions(+)
diff --git a/iothrea
On 10/01/2016 09:37 AM, Max Reitz wrote:
On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
v7:
https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=refs%2Ftags%2Fqcow2-bitmap-v7
based on block-next (https://github.com/XanClic/qemu/commits/block-next)
It should be noted that (a
On Tue, Oct 11, 2016 at 1:07 PM, Ashijeet Acharya
wrote:
> Add InetSocketAddress compatibility to SSH driver.
>
> Add a new option "server" to the SSH block driver which then accepts
> a InetSocketAddress.
>
> "host" and "port" are supported as legacy options and are mapped to
> their InetSocketAd
We have both 'struct NBDRequest' and 'struct nbd_request'; making
it confusing to see which does what. Furthermore, we want to
rename nbd_request to align with our normal CamelCase naming
conventions. So, rename the struct which is used to associate
the data received during request callbacks, whi
The NBD protocol allows servers to advertise a human-readable
description alongside an export name during NBD_OPT_LIST. Add
an option to pass through the user's string to the NBD client.
Doing this also makes it easier to test commit 200650d4, which
is the client counterpart of receiving the desc
The NBD Protocol allows us to send human-readable messages
along with any NBD_REP_ERR error during option negotiation;
make use of this fact for clients that know what to do with
our message.
Signed-off-by: Eric Blake
---
v6: tweak comments, fix indentation
v5: don't leak 'msg'
v4: new patch
---
Current upstream NBD documents that requests have a 16-bit flags,
followed by a 16-bit type integer; although older versions mentioned
only a 32-bit field with masking to find flags. Since the protocol
is in network order (big-endian over the wire), the ABI is unchanged;
but dealing with the flags
It's better to use consistent capitalization of the namespace
used for NBD functions; we have more instances of NBD* than
Nbd*.
Signed-off-by: Eric Blake
---
v6: new patch
---
block/nbd-client.h | 6 +++---
block/nbd-client.c | 26 +-
block/nbd.c| 4 ++--
3 fil
Our coding convention prefers CamelCase names, and we already
have other existing structs with NBDFoo naming. Let's be
consistent, before later patches add even more structs.
Signed-off-by: Eric Blake
---
v6: new patch
---
block/nbd-client.h | 2 +-
include/block/nbd.h | 10 ++
block/
Rather than open-coding each option request, it's easier to
have common helper functions do the work. That in turn requires
having convenient packed types for handling option requests
and replies.
Signed-off-by: Eric Blake
---
v6: comment and formatting tweaks
v5: no change
v4: rebase
v3: rebas
The NBD spec says that a client should send NBD_OPT_ABORT
rather than just dropping the connection, if the client doesn't
like something the server sent during option negotiation. This
is a best-effort attempt only, and can only be done in places
where we know the server is still in sync with what
v5 was here, but missed 2.7 freeze:
https://lists.gnu.org/archive/html/qemu-devel/2016-07/msg04053.html
Since then, I've rebased the series, and the bulk of the changes
were to use consistent NBDFoo CamelCase naming, as well as to
improve the commit messages for questions raised on v5.
Also avail
Rather than open-coding NBD_REP_SERVER, reuse the code we
already have by adding a length parameter. Additionally,
the refactoring will make adding NBD_OPT_GO in a later patch
easier.
Signed-off-by: Eric Blake
---
v6: improve (and add) function comments
v5: no change
v4: no change
v3: rebase to
Upstream NBD protocol recently added the ability to efficiently
write zeroes without having to send the zeroes over the wire,
along with a flag to control whether the client wants to allow
a hole.
Note that when it comes to requiring full allocation, vs.
permitting optimizations, the NBD spec inte
The server has a nice helper function nbd_negotiate_drop_sync()
which lets it easily ignore fluff from the client (such as the
payload to an unknown option request). We can't quite make it
common, since it depends on nbd_negotiate_read() which handles
coroutine magic, but we can copy the idea into
Upstream NBD protocol recently added the ability to efficiently
write zeroes without having to send the zeroes over the wire,
along with a flag to control whether the client wants a hole.
The generic block code takes care of falling back to the obvious
write of lots of zeroes if we return -ENOTSUP
NBD commit 6d34500b clarified how clients and servers are supposed
to behave before closing a connection. It added NBD_REP_ERR_SHUTDOWN
(for the server to announce it is about to go away during option
haggling, so the client should quit sending NBD_OPT_* other than
NBD_OPT_ABORT) and ESHUTDOWN (for
Since we know that the maximum name we are willing to accept
is small enough to stack-allocate, rework the iteration over
NBD_OPT_LIST responses to reuse a stack buffer rather than
allocating every time. Furthermore, we don't even have to
allocate if we know the server's length doesn't match what
The NBD Protocol allows the server and client to mutually agree
on a shorter handshake (omit the 124 bytes of reserved 0), via
the server advertising NBD_FLAG_NO_ZEROES and the client
acknowledging with NBD_FLAG_C_NO_ZEROES (only possible in
newstyle, whether or not it is fixed newstyle). It doesn
From: Fam Zheng
Upon each bit toggle, the corresponding bit in the meta bitmap will be
set.
Signed-off-by: Fam Zheng
[Amended text inline. --js]
Reviewed-by: Max Reitz
Signed-off-by: John Snow
---
include/qemu/hbitmap.h | 21 +++
util/hbitmap.c | 69 +
From: Fam Zheng
The added group of operations enables tracking of the changed bits in
the dirty bitmap.
Signed-off-by: Fam Zheng
Reviewed-by: Max Reitz
Signed-off-by: John Snow
---
block/dirty-bitmap.c | 52
include/block/dirty-bitmap.h |
From: Vladimir Sementsov-Ogievskiy
Functions to serialize / deserialize(restore) HBitmap. HBitmap should be
saved to linear sequence of bits independently of endianness and bitmap
array element (unsigned long) size. Therefore Little Endian is chosen.
These functions are appropriate for dirty bit
From: Fam Zheng
For dirty bitmap users to get the size and the name of a
BdrvDirtyBitmap.
Signed-off-by: Fam Zheng
Reviewed-by: John Snow
Reviewed-by: Max Reitz
Signed-off-by: John Snow
---
block/dirty-bitmap.c | 10 ++
include/block/dirty-bitmap.h | 2 ++
2 files changed,
Key:
[] : patches are identical
[] : number of functional differences between upstream/downstream patch
[down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively
001/10:[] [--] 'block: Hide HBitmap in block dirty bitmap interface'
From: Fam Zheng
We use a loop over bs->dirty_bitmaps to make sure the caller is
only releasing a bitmap owned by bs. Let's also assert that in this case
the caller is releasing a bitmap that does exist.
Signed-off-by: Fam Zheng
Reviewed-by: Max Reitz
Signed-off-by: John Snow
---
block/dirty-
From: Vladimir Sementsov-Ogievskiy
Several functions to provide necessary access to BdrvDirtyBitmap for
block-migration.c
Signed-off-by: Vladimir Sementsov-Ogievskiy
[Add the "finish" parameters. - Fam]
Signed-off-by: Fam Zheng
Reviewed-by: John Snow
Reviewed-by: Max Reitz
Signed-off-by: Jo
From: Fam Zheng
HBitmap is an implementation detail of block dirty bitmap that should be hidden
from users. Introduce a BdrvDirtyBitmapIter to encapsulate the underlying
HBitmapIter.
A small difference in the interface is, before, an HBitmapIter is initialized
in place, now the new BdrvDirtyBitm
From: Fam Zheng
Signed-off-by: Fam Zheng
Reviewed-by: John Snow
Reviewed-by: Max Reitz
Signed-off-by: John Snow
---
tests/test-hbitmap.c | 116 +++
1 file changed, 116 insertions(+)
diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c
inde
From: Fam Zheng
Callers can create an iterator of meta bitmap with
bdrv_dirty_meta_iter_new(), then use the bdrv_dirty_iter_* operations on
it. Meta iterators are also counted by bitmap->active_iterators.
Also add a couple of functions to retrieve granularity and count.
Signed-off-by: Fam Zheng
From: Fam Zheng
Signed-off-by: Fam Zheng
[Fixed minor constant issue. --js]
Signed-off-by: John Snow
Signed-off-by: John Snow
---
tests/test-hbitmap.c | 156 +++
1 file changed, 156 insertions(+)
diff --git a/tests/test-hbitmap.c b/tests/test-
If jobs are not created directly by the user, do not allow them to be
seen by the user/management utility. At the moment, 'internal' jobs are
those that do not have an ID. As of this patch it is impossible to
create such jobs.
Signed-off-by: John Snow
---
blockdev.c | 17 ++
(Trivial)
Fix wrong function names in documentation.
Signed-off-by: John Snow
---
include/block/blockjob_int.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h
index 8eced19..10ebb38 100644
--- a/include/block/bl
This is a follow-up to patches 1-6 of:
[PATCH v2 00/11] blockjobs: Fix transactional race condition
That series started trying to refactor blockjobs with the goal of
internalizing BlockJob state as a side effect of having gone through
the effort of figuring out which commands were "safe" to call o
Add the ability to create jobs without an ID.
Signed-off-by: John Snow
---
block/backup.c| 2 +-
block/commit.c| 2 +-
block/mirror.c| 3 ++-
block/stream.c| 2 +-
blockjob.c| 25 -
include/block/blockjob
There's no reason to leave this to blockdev; we can do it in blockjobs
directly and get rid of an extra callback for most users.
All non-internal events, even those created outside of QMP, will
consistently emit events.
Signed-off-by: John Snow
---
block/commit.c| 8
block
To make it a little more obvious which functions are intended to be
public interface and which are intended to be for use only by jobs
themselves, split the interface into "public" and "private" files.
Convert blockjobs (e.g. block/backup) to using the private interface.
Leave blockdev and others
Bubble up the internal interface to commit and backup jobs, then switch
replication tasks over to using this methodology.
Signed-off-by: John Snow
---
block/backup.c| 3 ++-
block/mirror.c| 21 ++---
block/replication.c | 14 +++---
blockdev
BlockJobs will begin hiding their state in preparation for some
refactorings anyway, so let's internalize the user_pause mechanism
instead of leaving it to callers to correctly manage.
Signed-off-by: John Snow
---
blockdev.c | 12 +---
blockjob.c | 22
Hi,
Your series failed automatic build test. Please find the testing commands and
their output below. If you have docker installed, you can probably reproduce it
locally.
Message-id: 1476392335-9256-1-git-send-email-ebl...@redhat.com
Type: series
Subject: [Qemu-devel] [PATCH v6 00/15] nbd: effici
On 10/13/2016 07:00 PM,
no-re...@ec2-52-6-146-230.compute-1.amazonaws.com wrote:
> Hi,
>
> Your series failed automatic build test. Please find the testing commands and
> their output below. If you have docker installed, you can probably reproduce
> it
> locally.
>
> /tmp/qemu-test/src/nbd/clie
Hi,
Your series failed automatic build test. Please find the testing commands and
their output below. If you have docker installed, you can probably reproduce it
locally.
Subject: [Qemu-devel] [PATCH v4 00/12] qapi: Allow blockdev-add for NBD
Type: series
Message-id: 20160928205602.17275-1-mre...
Hi,
Your series seems to have some coding style problems. See output below for
more information:
Subject: [Qemu-devel] [PATCH v6 00/15] nbd: efficient write zeroes
Type: series
Message-id: 1476392335-9256-1-git-send-email-ebl...@redhat.com
=== TEST SCRIPT BEGIN ===
#!/bin/bash
BASE=base
n=1
tot
Hi,
Your series seems to have some coding style problems. See output below for
more information:
Type: series
Message-id: 1476399422-8028-1-git-send-email-js...@redhat.com
Subject: [Qemu-devel] [PATCH 0/7] blockjobs: preliminary refactoring work, Pt 1
=== TEST SCRIPT BEGIN ===
#!/bin/bash
BASE=
81 matches
Mail list logo