Re: [Qemu-block] [Qemu-devel] [RFC] transactions: add transaction-wide property

[Markus Armbruster]

John Snow  writes:

> On 10/16/2015 08:23 AM, Stefan Hajnoczi wrote:
>> On Mon, Oct 12, 2015 at 12:50:20PM -0400, John Snow wrote:
>>> Ping -- any consensus on how we should implement the "do-or-die"
>>> argument for transactions that start block jobs? :)
>>>
>>> This patch may look a little hokey in how it boxes arguments, but I can
>>> re-do it on top of Eric Blake's very official way of boxing arguments,
>>> when the QAPI dust settles.
>> 
>> I don't understand what you are trying to do after staring at the email
>> for 5 minutes.  Maybe the other reviewers hit the same problem and
>> haven't responded.
>> 
>> What is the problem you're trying to solve?
>> 
>> Stefan
>> 
>
> Sorry...
>
> What I am trying to do is to add the transactional blocker property to
> the *transaction* command and not as an argument to each individual action.
>
> There was some discussion on this so I wanted to just send an RFC to
> show what I had in mind.

Was it the discussion on @transactional-cancel?  I'm on record
supporting it per transaction rather than per action:
Message-ID: <87mvwd8k9q@blackfin.pond.sub.org>
http://lists.gnu.org/archive/html/qemu-devel/2015-09/msg05948.html

> This series applies on top of Fam's latest series and moves the
> arguments from each action to a transaction-wide property.

Re: [Qemu-block] [PATCH v2 3/3] virtio-blk: switch off scsi-passthrough by default

[Cornelia Huck]

On Mon, 19 Oct 2015 14:57:31 +0300
"Michael S. Tsirkin"  wrote:

> On Mon, Oct 19, 2015 at 01:53:50PM +0200, Cornelia Huck wrote:
> > On Sun, 18 Oct 2015 10:59:59 +0300
> > "Michael S. Tsirkin"  wrote:
> > 
> > > On Fri, Oct 16, 2015 at 01:07:28PM +0200, Christian Borntraeger wrote:
> > 
> > > > Lets keep this patch as is to have scsi=off as default for virtio 1.0
> > > > 
> > > > (some iotests do fail because of this)
> > > > 
> > > > Christian
> > > 
> > > What fails, exactly?
> > 
> > For example, testcase 068 (on s390x, since we default virtio-1 to on):
> 
> I see, thanks.
> So it's just the assertion that we have in code that fires.
> Sure, scsi must be off by default before virtio 1 is on.

And -hda always assumes defaults...

Re: [Qemu-block] [PATCH v6 1/4] Add new block driver interface to add/delete a BDS's child

[Alberto Garcia]

On Fri 16 Oct 2015 10:57:43 AM CEST, Wen Congyang  wrote:
> In some cases, we want to take a quorum child offline, and take
> another child online.
>
> Signed-off-by: Wen Congyang 
> Signed-off-by: zhanghailiang 
> Signed-off-by: Gonglei 
> Reviewed-by: Eric Blake 
> ---

Reviewed-by: Alberto Garcia 

Berto

Re: [Qemu-block] [Qemu-devel] Question about odd snapshot behaviour

[Kevin Wolf]

[ CC: qemu-block ]

Am 19.10.2015 um 07:27 hat Sam Bobroff geschrieben:
> Hi all,
> 
> While working through the QEMU code that saves and loads snapshots, I've
> noticed some confusing behaviour when using named VM snapshots that may need 
> to
> be fixed somehow. This is what I've noticed:
> 
> * If I create two empty qcow2 filesystems: fs1.qcow2 and fs2.qcow2:
>   * e.g. qemu-image create -f qcow2 fs1.qcow2 10G
>   * e.g. qemu-image create -f qcow2 fs2.qcow2 10G
> * Then boot the guest with the above filesystems as hda and hdb (my guest's 
> rootfs
>   is an initramfs, not either of these):
>   * e.g. qemu-system-ppc64 ... -hda fs1.qcow2 -hdb fs2.qcow2
> * In the QEMU monitor, create a blockdev snapshot on hd2 called "foo":
>   e.g. snapshot_blkdev_internal scsi0-hd1 foo
> * Check fs2 on the host command line:
>   e.g. qemu-img snapshot -l fs2.qcow2
>   Snapshot list:
>   IDTAG VM SIZEDATE   VM CLOCK
>   1 foo   0 2015-10-19 15:56:29   00:00:20.969
> * In the QEMU monitor, save the vm state as "bar":
>   e.g. savevm bar
> * Check the snapshot on fs1:
>   e.g. info snapshots
>   IDTAG VM SIZEDATE   VM CLOCK
>   1 bar215M 2015-10-19 04:57:13   00:01:05.589
> * Check fs2 again on the host command line:
>   e.g. qemu-img snapshot -l fs2.qcow2
>   Snapshot list:
>   IDTAG VM SIZEDATE   VM CLOCK
>   1 foo   0 2015-10-19 15:56:29   00:00:20.969
>   2 bar   0 2015-10-19 15:57:13   00:01:05.589
> 
> * Now the fun part: overwrite the bar snapshot on fs1 by ID:
>   * savevm 1
> * Examine the results from the monitor:
>   * e.g. info snapshots
>   There is no suitable snapshot available
> * Examine fs1 and fs2 from the command line:
>   * e.g. qemu-img snapshot -l fs1.qcow2 
>   Snapshot list:
>   IDTAG VM SIZEDATE   VM CLOCK
>   1 bar215M 2015-10-19 16:00:45   00:04:36.065
>   * e.g. qemu-img snapshot -l fs2.qcow2 
>   Snapshot list:
>   IDTAG VM SIZEDATE   VM CLOCK
>   2 bar   0 2015-10-19 15:57:13   00:01:05.589
>   3 bar   0 2015-10-19 16:00:45   00:04:36.065

I must admit that I wouldn't have been able to predict this result. :-)

Generally, internal snapshots with multiple disks are somewhat tricky
because we have individual names and IDs in every image file, and each
file can individually be snapshotted, but need to create a consistent
view from it.

> So what seems odd is:
> 
> * QEMU has lost the bar snapshot on fs1, although qemu-img still shows
>   it and it re-appears in QEMU if the guest is started with only
>   hd1.qcow attached.

This one I think is a feature: A snapshot can only be loaded if it's
present on all image files. Loading the snapshot only on some disks and
leaving the other disks alone is likely to result in an inconsistent
state.

> * QEMU has deleted the snapshot named "foo" on hd2, even though we issued
>   "savevm 1" and the old snapshot (on hd1) with ID 1 was named "bar". This
>   surprised me.

You mean we should check whether a snapshot identified by an ID has the
same name on all image files before it's considered valid? That might be
a reasonable constraint.

What I would have expected is that the old snapshot with ID 1 indeed
gets deleted, but a new one with ID 1 is created again. I'm not sure if
I would have expected "foo" or "bar" as its name, I could argue either
way.

> * QEMU has created a snapshot on hd2 with a duplicate name ("bar") and if we
>   issue a "loadvm bar" QEMU loads the snapshot with ID 1 on hd1 and the first
>   snapshot with the name "bar" on hd2 which is the one with ID 2. That is not
>   the snapshot that matches the vm state (which is the one with ID 3) and
>   presumably can lead to filesystem corruption on hd2.

Yup, this doesn't seem very nice. I'm not sure whether we can forbid
duplicate names, though, without breaking compatibility in an
unacceptable way.

On the other hand, IDs and names don't really mix well and make it too
easy to shoot yourself in the foot. Maybe it's bad enough to justify
even some incompatibilities.

> Do these seem like bugs? Should I try to find a way to fix some or all of 
> them?
> 
> (Note: I checked the launchpad bug list but didn't see these listed.)

I'm not sure whether to call them bugs, but there's definitely a safety
check or two missing. Let's discuss what the exact behaviour should be,
and then you can go ahead and write the fixes to get that behaviour.

Kevin

Re: [Qemu-block] [PATCH 0/3] Add 'blockdev-del' command

[Kevin Wolf]

Am 13.10.2015 um 15:48 hat Alberto Garcia geschrieben:
> Here's my first attempt at the 'blockdev-del' command.
> 
> This series goes on top of Max's "BlockBackend and media" v6:
> 
> https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg02810.html
> 
> With Max's code, 'blockdev-add' can now create a BlockDriverState with
> or without a BlockBackend (depending on whether the 'id' parameter is
> passed).
> 
> Therefore, 'blockdev-del' can be used to delete a backend and/or a
> BDS.
> 
> The way it works is simple: it receives a single 'device' parameter
> which can refer to a block backend or a node name.
> 
>  - If it's a block backend, it will delete it along with its attached
>BDS (if any).
> 
>  - If it's a node name, it will delete the BDS along with the backend
>it is attached to (if any).
> 
>  - If you're wondering whether it's possible to delete the BDS but not
>the backend it is attached to, there is already eject or
>blockdev-remove-medium for that.
> 
> If either the backend or the BDS are being used (refcount > 1, or if
> the BDS has any parents) the command fails.

I've been thinking a bit about the creation and deletion of
BlockBackends a bit last week, and honestly it still feels a bit messy
(or maybe I just don't fully understand it yet). Anyway, the cover
letter of this series seems to be a good place for trying to write it
down.

So we seem to have two criteria to distinguish BDSes:

1. Does/Doesn't have a BlockBackend on top.
   In the future, multiple BlockBackends are possible, too.

2. Has/Hasn't been created explicitly by the user.
   Only if it has, the monitor owns a reference.

In addition, we also have BlockBackends without a BDS attached. All BBs
are explicitly created by the user. This leads us to the following
cases:

1. BB without BDS
2. BB attached to BDS without monitor reference
3. BB attached to BDS with monitor reference
4. BDS without BB and without monitor reference
5. BDS without BB and with monitor reference

The question I've been thinking about for each of the cases is: Can we
create this with blockdev-add today, and what should blockdev-del do?
Another interesting question is: Can we move between the cases, e.g. by
adding a monitor reference retroactively, or do we even want to do that?

Let me start with the first two questions for all cases:

1. As far as I know, it's currently not possible to directly create a BB
   without a BDS (or before Max' series with an empty BDS). You have to
   create a BB with an opened BDS and then eject that. Oops.

   blockdev-del is easy: It deletes the BB and nothing else.

2. This is the normal case, easy to produce with blockdev-add.
   The two options for deleting something are removing the BDS while
   keeping the BB (this is already covered by 'eject') and dropping the
   BB (probably makes sense to use 'blockdev-del' here). There is no
   monitor reference to the BDS anyway, so blockdev-dev can't delete
   that.

   Dropping the BB automatically also drops the BDS in simple cases. In
   more complicated cases where the BDS has got more references later,
   it might still exist. Then the blockdev-del wasn't the exact opposite
   operation of blockdev-add.

   Is this a problem for a user/management tool that wants to keep track
   of which image files are still in use?

3. A BDS with a monitor reference can be created (with some patches) by
   using blockdev-add with a node-name, but no id. Directly attaching it
   to a BB is tricky today, even though I'm sure you could do it with
   some clever use of block jobs... With 1. fixed (i.e. you can create
   an empty BB), insert-medium should do the job.

   Here we have even more choices for deleting something: Delete the BB,
   but leave the BDS alone; delete the BDS and leave an empty BB behind
   (this is different from eject because eject would drop the connection
   between BB and BDS, but both would continue to exist!); delete both
   at once.

   We had two separate blockdev-add commands for the BDS and the BB, so
   it makes sense to require two separate blockdev-del commands as well.
   In order to keep blockdev-add/del symmetrical, we would have to make
   blockdev-del of a node-name delete the BDS and blockdev-del of an id
   delete the BB.

4. That's the typical bs->file. Created by nested blockdev-add
   descriptions. blockdev-del errors out, there is no monitor reference
   to drop, and there is also no BB that the request could be used for.

5. Created by a top-level blockdev-add with node-name and no id (like 3.
   but without attaching it to a BB afterwards). blockdev-del can only
   mean deleting the BDS.

If I compare this with the semantics described in the cover letter, I
must say that I see some problems, especially with case 3.

I haven't thought about it enough yet, but it seems to me that we can't
do the BDS/BB aliasing with blockdev-del, but need to interpret
node-name as BDS and id as BB. Perhaps we also shouldn't use a single

Re: [Qemu-block] [PATCH v2 3/3] virtio-blk: switch off scsi-passthrough by default

[Cornelia Huck]

On Sun, 18 Oct 2015 10:59:59 +0300
"Michael S. Tsirkin"  wrote:

> On Fri, Oct 16, 2015 at 01:07:28PM +0200, Christian Borntraeger wrote:

> > Lets keep this patch as is to have scsi=off as default for virtio 1.0
> > 
> > (some iotests do fail because of this)
> > 
> > Christian
> 
> What fails, exactly?

For example, testcase 068 (on s390x, since we default virtio-1 to on):

--- /data/git/yyy/qemu/tests/qemu-iotests/068.out   2015-03-09 
12:32:35.245444052 +0100
+++ 068.out.bad 2015-10-19 13:48:00.023772655 +0200
@@ -3,9 +3,8 @@
 === Saving and reloading a VM state to/from a qcow2 image ===
 
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=131072
+qemu-system-s390x: -hda 
/data/git/yyy/qemu/build/tests/qemu-iotests/scratch/t.qcow2: Please set 
scsi=off for virtio-blk devices in order to use virtio 1.0
 QEMU X.Y.Z monitor - type 'help' for more information
-(qemu) savevm 0
-(qemu) quit
+(qemu) qemu-system-s390x: -hda 
/data/git/yyy/qemu/build/tests/qemu-iotests/scratch/t.qcow2: Please set 
scsi=off for virtio-blk devices in order to use virtio 1.0
 QEMU X.Y.Z monitor - type 'help' for more information
-(qemu) quit
-*** done
+(qemu) *** done

Re: [Qemu-block] [PATCH v2 3/3] virtio-blk: switch off scsi-passthrough by default

[Michael S. Tsirkin]

On Mon, Oct 19, 2015 at 01:53:50PM +0200, Cornelia Huck wrote:
> On Sun, 18 Oct 2015 10:59:59 +0300
> "Michael S. Tsirkin"  wrote:
> 
> > On Fri, Oct 16, 2015 at 01:07:28PM +0200, Christian Borntraeger wrote:
> 
> > > Lets keep this patch as is to have scsi=off as default for virtio 1.0
> > > 
> > > (some iotests do fail because of this)
> > > 
> > > Christian
> > 
> > What fails, exactly?
> 
> For example, testcase 068 (on s390x, since we default virtio-1 to on):

I see, thanks.
So it's just the assertion that we have in code that fires.
Sure, scsi must be off by default before virtio 1 is on.


> --- /data/git/yyy/qemu/tests/qemu-iotests/068.out 2015-03-09 
> 12:32:35.245444052 +0100
> +++ 068.out.bad   2015-10-19 13:48:00.023772655 +0200
> @@ -3,9 +3,8 @@
>  === Saving and reloading a VM state to/from a qcow2 image ===
>  
>  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=131072
> +qemu-system-s390x: -hda 
> /data/git/yyy/qemu/build/tests/qemu-iotests/scratch/t.qcow2: Please set 
> scsi=off for virtio-blk devices in order to use virtio 1.0
>  QEMU X.Y.Z monitor - type 'help' for more information
> -(qemu) savevm 0
> -(qemu) quit
> +(qemu) qemu-system-s390x: -hda 
> /data/git/yyy/qemu/build/tests/qemu-iotests/scratch/t.qcow2: Please set 
> scsi=off for virtio-blk devices in order to use virtio 1.0
>  QEMU X.Y.Z monitor - type 'help' for more information
> -(qemu) quit
> -*** done
> +(qemu) *** done

Re: [Qemu-block] [PATCH 2/3] block: Add 'blockdev-del' QMP command

[Alberto Garcia]

On Sat 17 Oct 2015 08:06:19 PM CEST, Max Reitz wrote:
>> +if (blk_get_refcnt(blk) > 1 ||
>> +(bs && (bs->refcnt > 1 || !QLIST_EMPTY(>parents {
>> +error_setg(errp, "Block device %s is being used", device);
>> +goto out;
>> +}
>
> I can't think of a way off the top of my head that a BB with a refcount
> of one or a BDS with a refcount of one without any parents would not be
> monitor-owned, but I don't quite like assuming it just from the refcount
> and the parent list anyway.

I agree, I would also like to have a clearer way to do it.

I'll try to go over all possible ways to create a BDS and see other
cases that I might have missed. At the very list I would like to extend
the iotest and make it as comprehensive as possible.

> The two patches which are significant for this patch are:
> - "blockdev: Keep track of monitor-owned BDS" from the first series
> - "blockdev: Add list of monitor-owned BlockBackends" from the second
>
> Explaining what they do and why they are relevant to this patch doesn't
> really seem necessary, but anyway: They add one list for the
> monitor-owned BDSs and one for the monitor-owned BBs.

Yeah, those would be handy indeed.

> It's up to you. It probably works just how you implemented it, and
> considering the pace of the "BB and media" series (and other series of
> mine in the past year), getting those two series merged may be a
> matter of decades. So it's probably best to do it like this for now
> and I'll fix it up then...

I think I'll first try to go over all possible cases and see what I
find.

Thanks,

Berto

Re: [Qemu-block] [PATCH v6 18/39] block: Add BlockBackendRootState

[Kevin Wolf]

Am 12.10.2015 um 22:00 hat Max Reitz geschrieben:
> This structure will store some of the state of the root BDS if the BDS
> tree is removed, so that state can be restored once a new BDS tree is
> inserted.
> 
> Signed-off-by: Max Reitz 

Random thought, not directly related to this series:

We currently don't have a way to create just a BlockBackend with no
medium. If we were to add that, would we want that to be just like a
missing -drive file=... option, or would it actually make sense to
specify the BlockBackendRootState?

If we want to do that, we might actually have found a reason why the
'options' key makes sense in blockdev-add. We could make it a union
where you either describe a BlockBackendRootState or a BDS.

Another related question is whether we want to separate out BB options
(which would otherwise be shared between the BBRS and BDS variants) into
their own dict. This dict could be optional and that would be the way to
specify whether you want a BB on top or not. Candidates for this dict
are id/rerror/werror.

There are more fields that exist in both the BDS and BBRS variants, but
don't really belong to the BB (i.e. they end up in the real BBRS and not
in the BB, and are only valid as long as no BDS is attached). These
would not be moved to the BB options dict.

Any opinions?

Kevin

Re: [Qemu-block] [PATCH 0/3] Add 'blockdev-del' command

[Max Reitz]

On 19.10.2015 13:27, Kevin Wolf wrote:
> Am 13.10.2015 um 15:48 hat Alberto Garcia geschrieben:
>> Here's my first attempt at the 'blockdev-del' command.
>>
>> This series goes on top of Max's "BlockBackend and media" v6:
>>
>> https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg02810.html
>>
>> With Max's code, 'blockdev-add' can now create a BlockDriverState with
>> or without a BlockBackend (depending on whether the 'id' parameter is
>> passed).
>>
>> Therefore, 'blockdev-del' can be used to delete a backend and/or a
>> BDS.
>>
>> The way it works is simple: it receives a single 'device' parameter
>> which can refer to a block backend or a node name.
>>
>>  - If it's a block backend, it will delete it along with its attached
>>BDS (if any).
>>
>>  - If it's a node name, it will delete the BDS along with the backend
>>it is attached to (if any).
>>
>>  - If you're wondering whether it's possible to delete the BDS but not
>>the backend it is attached to, there is already eject or
>>blockdev-remove-medium for that.
>>
>> If either the backend or the BDS are being used (refcount > 1, or if
>> the BDS has any parents) the command fails.
> 
> I've been thinking a bit about the creation and deletion of
> BlockBackends a bit last week, and honestly it still feels a bit messy
> (or maybe I just don't fully understand it yet). Anyway, the cover
> letter of this series seems to be a good place for trying to write it
> down.
> 
> So we seem to have two criteria to distinguish BDSes:
> 
> 1. Does/Doesn't have a BlockBackend on top.
>In the future, multiple BlockBackends are possible, too.
> 
> 2. Has/Hasn't been created explicitly by the user.
>Only if it has, the monitor owns a reference.
> 
> In addition, we also have BlockBackends without a BDS attached. All BBs
> are explicitly created by the user. This leads us to the following
> cases:
> 
> 1. BB without BDS
> 2. BB attached to BDS without monitor reference
> 3. BB attached to BDS with monitor reference
> 4. BDS without BB and without monitor reference
> 5. BDS without BB and with monitor reference
> 
> The question I've been thinking about for each of the cases is: Can we
> create this with blockdev-add today, and what should blockdev-del do?
> Another interesting question is: Can we move between the cases, e.g. by
> adding a monitor reference retroactively, or do we even want to do that?
> 
> Let me start with the first two questions for all cases:
> 
> 1. As far as I know, it's currently not possible to directly create a BB
>without a BDS (or before Max' series with an empty BDS). You have to
>create a BB with an opened BDS and then eject that. Oops.
> 
>blockdev-del is easy: It deletes the BB and nothing else.
> 
> 2. This is the normal case, easy to produce with blockdev-add.
>The two options for deleting something are removing the BDS while
>keeping the BB (this is already covered by 'eject') and dropping the
>BB (probably makes sense to use 'blockdev-del' here). There is no
>monitor reference to the BDS anyway, so blockdev-dev can't delete
>that.
> 
>Dropping the BB automatically also drops the BDS in simple cases. In
>more complicated cases where the BDS has got more references later,
>it might still exist. Then the blockdev-del wasn't the exact opposite
>operation of blockdev-add.
> 
>Is this a problem for a user/management tool that wants to keep track
>of which image files are still in use?
> 
> 3. A BDS with a monitor reference can be created (with some patches) by
>using blockdev-add with a node-name, but no id. Directly attaching it
>to a BB is tricky today, even though I'm sure you could do it with
>some clever use of block jobs... With 1. fixed (i.e. you can create
>an empty BB), insert-medium should do the job.
> 
>Here we have even more choices for deleting something: Delete the BB,
>but leave the BDS alone; delete the BDS and leave an empty BB behind
>(this is different from eject because eject would drop the connection
>between BB and BDS, but both would continue to exist!); delete both
>at once.
> 
>We had two separate blockdev-add commands for the BDS and the BB, so
>it makes sense to require two separate blockdev-del commands as well.
>In order to keep blockdev-add/del symmetrical, we would have to make
>blockdev-del of a node-name delete the BDS and blockdev-del of an id
>delete the BB.

And keep in mind that in this case the monitor has two references, one
to the BB and one to the BDS, so in case these are the only "external"
references, the BB will have a refcount of 1 and the BDS a refcount of
2. Therefore, without any magic, you'll need two blockdev-del
invocations anyway, one for the BB and one for the BDS.

As Berto said, with patch 2 as it is, either blockdev-del will fail, and
you need to call remove-medium/eject first, which seems fine to me.

Max

> 4. That's the typical bs->file. 

Re: [Qemu-block] [PATCH v6 18/39] block: Add BlockBackendRootState

[Kevin Wolf]

Am 19.10.2015 um 16:32 hat Max Reitz geschrieben:
> On 19.10.2015 16:18, Kevin Wolf wrote:
> > Am 12.10.2015 um 22:00 hat Max Reitz geschrieben:
> >> This structure will store some of the state of the root BDS if the BDS
> >> tree is removed, so that state can be restored once a new BDS tree is
> >> inserted.
> >>
> >> Signed-off-by: Max Reitz 
> > 
> > Random thought, not directly related to this series:
> > 
> > We currently don't have a way to create just a BlockBackend with no
> > medium. If we were to add that, would we want that to be just like a
> > missing -drive file=... option, or would it actually make sense to
> > specify the BlockBackendRootState?
> 
> We don't? -drive if=none,id=foo works just fine for me. Issuing qemu-io
> foo "read 0 512" then prints "read failed: " + strerror(ENOMEDIUM) to
> stderr.

Sorry, I meant "no blockdev-add way", i.e. hotplug an empty BB in QMP.

> > If we want to do that, we might actually have found a reason why the
> > 'options' key makes sense in blockdev-add. We could make it a union
> > where you either describe a BlockBackendRootState or a BDS.
> 
> I really wouldn't want to use the BBRS for anything but legacy support
> (i.e. change inheriting the options of the last medium)...

Fair enough. Then I guess it was a random stupid thought.

> > Another related question is whether we want to separate out BB options
> > (which would otherwise be shared between the BBRS and BDS variants) into
> > their own dict. This dict could be optional and that would be the way to
> > specify whether you want a BB on top or not. Candidates for this dict
> > are id/rerror/werror.
> > 
> > There are more fields that exist in both the BDS and BBRS variants, but
> > don't really belong to the BB (i.e. they end up in the real BBRS and not
> > in the BB, and are only valid as long as no BDS is attached). These
> > would not be moved to the BB options dict.
> > 
> > Any opinions?
> 
> Not on the latter part.

Pity.

Kevin


pgp0MNvJfQJnr.pgp
Description: PGP signature

[Qemu-block] [PATCH 00/17] Framework for securely passing secrets to QEMU

[Daniel P. Berrange]

There are a variety of places where QEMU needs to have access
to passwords, encryption keys or similar kinds of secrets.

 - VNC / SPICE user passwords
 - Curl block http / proxy passwords
 - RBD auth password
 - iSCSI CHAP password
 - x509 private key password
 - QCow/QCow2 encryption key

QEMU has a variety of ways of dealing with this problem, some
good, some ugly, some bad.

 - The RBD block driver accepts the password in plaintext
   via a private RBD config option. This is a pending CVE

https://security-tracker.debian.org/tracker/CVE-2015-5160

 - The iSCSI driver accepts the password in plaintext as
   a block driver option. This is the same as the RBD CVE
   essentially, just a QEMU option, rather than a librbd
   option

 - The VNC / SPICE servers only accept the passwords via
   the QEMU monitor. This is secure, but it inconvenient
   for adhoc developer usage where security of CLI args
   does not matter.

 - QCow/QCow2 encryption keys can be provided by the monitor
   but this is not available for qemu-img, qemu-io and
   qemu-nbd. QEMU falls back to doing interactive
   console prompting to get keys.

 - x509 privte key passwords are not supported at all by
   QEMU which forces users to store their key in plaintext
   on their host FS.

 - The CURL driver doesn't support HTTP auth at all
   currently.

It is obvious there there is a wide variety of functionality
in QEMU that needs access to "secrets". This need will only
grow over time. We need to stop having everyone invent their
own dangerous wheels and provide a standard mechanism for
securely passing secrets to QEMU.

To this end, this series introduces a QCryptoSecret object
class with short name "secret". All the places which needs
passwords/keys are then converted to get their via this
API, except VNC/SPICE which are a future exercise.

Example usage for creating secrets...

Direct password, insecure, for ad-hoc developer testing only

  $QEMU -object secret,id=sec0,data=letmein

Indirect password via a file, good for production

  echo -n "letmein" > mypasswd.txt
  $QEMU -object secret,id=sec0,file=mypasswd.txt

The file based approach supports file descriptor passing,
so mgmt apps can use that to dynamically add passwords to
running QEMU.

There is a better way though, which is to use an encrypted
secret. The intent here is that a mgmt app (like libvirt)
will generate a random AES-256 key for each virtual machine
it starts (saved in eg /var/run/libvirt/qemu/$GUEST.key)
It can then use the direct password passing, but encrypt
the data.

  $QEMU \
-object 
secret,id=secmaster0,file=/var/run/libvirt/qemu/$GUEST.key,format=base64 \
-object secret,id=sec0,data=[base64 ciphertext],\
   keyid=secmaster0,iv=[base64 initialization vector]

This means that the mgmt app does not need to worry about
file descriptor passing at all. It can just use regular
object properties, safe in the knowledge that the data is
protected by a secret AES key shared only between QEMU
and the mgmt app.

Use of encrypted secrets is not restricted to directly
provided inline data. If the secret is stored in an
external file, that can be encrypted too

  $QEMU \
-object 
secret,id=secmaster0,file=/var/run/libvirt/qemu/$GUEST.key,format=base64 \
-object secret,id=sec0,file=/some/secret/file.aes,\
   keyid=secmaster0,iv=[base64 initialization vector]



Example usage for referencing secrets...

CURL:

  $QEMU -object secret,id=sec0... \
 -drive driver=http,url=http://example.com/someimg.qcow2,\
  user=dan,passwordid=sec0

  $QEMU -object secret,id=sec0... -object secret,id=sec1 \
 -drive driver=http,url=http://example.com/someimg.qcow2,\
  user=dan,passwordid=sec0,proxyuser=dan,passwordid=sec1

iSCSI:

  $QEMU -object secret,id=sec0... \
 -drive driver=iscsi,url=iscsi://example.com/target-foo/lun1,\
 user=dan,passwordid=sec0

RBD:

  $QEMU -object secret,id=sec0... \
 -drive driver=rbd,file=rbd:pool/image:id=myname,\
 auth-supported-cephx,authsecret=sec0

QCow/Qcow2 encryption

  $QEMU -object secret,id=sec0... \
 -drive file=someimage.qcow2,keyid=sec0


Finally, this extends qemu-img, qemu-nbd and qemu-io. All of
them gain a new '--object' parameter which provides the same
functionality as '-object' with QEMU system emulators. This
lets us create QCryptoSecret object instances in those tools

The tools also then get support for a new '--source IMG-OPTS'
parameter to allow a full set of image options to be specified,
instead of just separate hardcoded args for format + filename
which they currently permit. This is probably the area I am
least sure of. I struggled to understand what the "best
practice" is for turning a QemuOpts into something you can
use to instantiate block backends. So I may well have not
done the right thing.

Towards the end I rip out the current encryption key handling
from the block layer so all the hairy code for dealing
with encrypted 

[Qemu-block] [PATCH 04/17] curl: add support for HTTP authentication parameters

[Daniel P. Berrange]

If connecting to a web server which has authentication
turned on, QEMU gets a 401 as curl has not been configured
with any authentication credentials.

This adds 4 new parameters to the curl block driver
options, username, passwordid, proxyusername and
proxypasswordid.

 $QEMU \
 -object secret,id=sec0,filename=/home/berrange/example.pw \
 -object secret,id=sec1,filename=/home/berrange/proxy.pw \
 -drive driver=http,url=http://example.com/some.img,\
 username=dan,passwordid=sec0,\
 proxyusername=dan,passwordid=sec1

Of course it is possible to use the same secret for both the
proxy & server passwords if desired.

Signed-off-by: Daniel P. Berrange 
---
 block/curl.c | 66 
 1 file changed, 66 insertions(+)

diff --git a/block/curl.c b/block/curl.c
index 032cc8a..5ad5704 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -26,6 +26,7 @@
 #include "block/block_int.h"
 #include "qapi/qmp/qbool.h"
 #include "qapi/qmp/qstring.h"
+#include "crypto/secret.h"
 #include 
 
 // #define DEBUG_CURL
@@ -77,6 +78,10 @@ static CURLMcode __curl_multi_socket_action(CURLM 
*multi_handle,
 #define CURL_BLOCK_OPT_SSLVERIFY "sslverify"
 #define CURL_BLOCK_OPT_TIMEOUT "timeout"
 #define CURL_BLOCK_OPT_COOKIE"cookie"
+#define CURL_BLOCK_OPT_USERNAME "username"
+#define CURL_BLOCK_OPT_PASSWORDID "passwordid"
+#define CURL_BLOCK_OPT_PROXY_USERNAME "proxyusername"
+#define CURL_BLOCK_OPT_PROXY_PASSWORDID "proxypasswordid"
 
 struct BDRVCURLState;
 
@@ -119,6 +124,10 @@ typedef struct BDRVCURLState {
 char *cookie;
 bool accept_range;
 AioContext *aio_context;
+char *username;
+char *password;
+char *proxyusername;
+char *proxypassword;
 } BDRVCURLState;
 
 static void curl_clean_state(CURLState *s);
@@ -416,6 +425,21 @@ static CURLState *curl_init_state(BlockDriverState *bs, 
BDRVCURLState *s)
 curl_easy_setopt(state->curl, CURLOPT_ERRORBUFFER, state->errmsg);
 curl_easy_setopt(state->curl, CURLOPT_FAILONERROR, 1);
 
+if (s->username) {
+curl_easy_setopt(state->curl, CURLOPT_USERNAME, s->username);
+}
+if (s->password) {
+curl_easy_setopt(state->curl, CURLOPT_PASSWORD, s->password);
+}
+if (s->proxyusername) {
+curl_easy_setopt(state->curl,
+ CURLOPT_PROXYUSERNAME, s->proxyusername);
+}
+if (s->proxypassword) {
+curl_easy_setopt(state->curl,
+ CURLOPT_PROXYPASSWORD, s->proxypassword);
+}
+
 /* Restrict supported protocols to avoid security issues in the more
  * obscure protocols.  For example, do not allow POP3/SMTP/IMAP see
  * CVE-2013-0249.
@@ -522,10 +546,31 @@ static QemuOptsList runtime_opts = {
 .type = QEMU_OPT_STRING,
 .help = "Pass the cookie or list of cookies with each request"
 },
+{
+.name = CURL_BLOCK_OPT_USERNAME,
+.type = QEMU_OPT_STRING,
+.help = "Username for HTTP auth"
+},
+{
+.name = CURL_BLOCK_OPT_PASSWORDID,
+.type = QEMU_OPT_STRING,
+.help = "ID of secret used as password for HTTP auth",
+},
+{
+.name = CURL_BLOCK_OPT_PROXY_USERNAME,
+.type = QEMU_OPT_STRING,
+.help = "Username for HTTP proxy auth"
+},
+{
+.name = CURL_BLOCK_OPT_PROXY_PASSWORDID,
+.type = QEMU_OPT_STRING,
+.help = "ID of secret used as password for HTTP proxy auth",
+},
 { /* end of list */ }
 },
 };
 
+
 static int curl_open(BlockDriverState *bs, QDict *options, int flags,
  Error **errp)
 {
@@ -536,6 +581,7 @@ static int curl_open(BlockDriverState *bs, QDict *options, 
int flags,
 const char *file;
 const char *cookie;
 double d;
+const char *passwordid;
 
 static int inited = 0;
 
@@ -577,6 +623,26 @@ static int curl_open(BlockDriverState *bs, QDict *options, 
int flags,
 goto out_noclean;
 }
 
+s->username = g_strdup(qemu_opt_get(opts, CURL_BLOCK_OPT_USERNAME));
+passwordid = qemu_opt_get(opts, CURL_BLOCK_OPT_PASSWORDID);
+
+if (passwordid) {
+s->password = qcrypto_secret_lookup_as_utf8(passwordid, errp);
+if (!s->password) {
+goto out_noclean;
+}
+}
+
+s->proxyusername = g_strdup(
+qemu_opt_get(opts, CURL_BLOCK_OPT_PROXY_USERNAME));
+passwordid = qemu_opt_get(opts, CURL_BLOCK_OPT_PROXY_PASSWORDID);
+if (passwordid) {
+s->proxypassword = qcrypto_secret_lookup_as_utf8(passwordid, errp);
+if (!s->proxypassword) {
+goto out_noclean;
+}
+}
+
 if (!inited) {
 curl_global_init(CURL_GLOBAL_ALL);
 inited = 1;
-- 
2.4.3

[Qemu-block] [PATCH 03/17] rbd: add support for getting password from QCryptoSecret object

[Daniel P. Berrange]

Currently RBD passwords must be provided on the command line
via

  $QEMU -drive file=rbd:pool/image:id=myname:\
key=QVFDVm41aE82SHpGQWhBQXEwTkN2OGp0SmNJY0UrSE9CbE1RMUE=:\
auth_supported=cephx

This is insecure because the key is visible in the OS process
listing.

This adds support for an 'authsecret' parameter in the RBD
parameters that can be used with the QCryptoSecret object to
provide the password via a file:

  echo "QVFDVm41aE82SHpGQWhBQXEwTkN2OGp0SmNJY0UrSE9CbE1RMUE=" > poolkey.b64
  $QEMU -object secret,id=secret0,file=poolkey.b64,format=base64 \
  -drive file=rbd:pool/image:id=myname:\
  auth_supported=cephx,authsecret=secret0

Signed-off-by: Daniel P. Berrange 
---
 block/rbd.c | 42 ++
 1 file changed, 42 insertions(+)

diff --git a/block/rbd.c b/block/rbd.c
index a60a19d..0acf777 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -16,6 +16,7 @@
 #include "qemu-common.h"
 #include "qemu/error-report.h"
 #include "block/block_int.h"
+#include "crypto/secret.h"
 
 #include 
 
@@ -228,6 +229,23 @@ static char *qemu_rbd_parse_clientname(const char *conf, 
char *clientname)
 return NULL;
 }
 
+
+static int qemu_rbd_set_auth(rados_t cluster, const char *secretid,
+ Error **errp)
+{
+gchar *secret = qcrypto_secret_lookup_as_base64(secretid,
+errp);
+if (!secret) {
+return -1;
+}
+
+rados_conf_set(cluster, "key", secret);
+g_free(secret);
+
+return 0;
+}
+
+
 static int qemu_rbd_set_conf(rados_t cluster, const char *conf,
  bool only_read_conf_file,
  Error **errp)
@@ -299,10 +317,13 @@ static int qemu_rbd_create(const char *filename, QemuOpts 
*opts, Error **errp)
 char conf[RBD_MAX_CONF_SIZE];
 char clientname_buf[RBD_MAX_CONF_SIZE];
 char *clientname;
+const char *secretid;
 rados_t cluster;
 rados_ioctx_t io_ctx;
 int ret;
 
+secretid = qemu_opt_get(opts, "authsecret");
+
 if (qemu_rbd_parsename(filename, pool, sizeof(pool),
snap_buf, sizeof(snap_buf),
name, sizeof(name),
@@ -350,6 +371,11 @@ static int qemu_rbd_create(const char *filename, QemuOpts 
*opts, Error **errp)
 return -EIO;
 }
 
+if (qemu_rbd_set_auth(cluster, secretid, errp) < 0) {
+rados_shutdown(cluster);
+return -EIO;
+}
+
 if (rados_connect(cluster) < 0) {
 error_setg(errp, "error connecting");
 rados_shutdown(cluster);
@@ -423,6 +449,11 @@ static QemuOptsList runtime_opts = {
 .type = QEMU_OPT_STRING,
 .help = "Specification of the rbd image",
 },
+{
+.name = "authsecret",
+.type = QEMU_OPT_STRING,
+.help = "ID of secret providing the password",
+},
 { /* end of list */ }
 },
 };
@@ -436,6 +467,7 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict 
*options, int flags,
 char conf[RBD_MAX_CONF_SIZE];
 char clientname_buf[RBD_MAX_CONF_SIZE];
 char *clientname;
+const char *secretid;
 QemuOpts *opts;
 Error *local_err = NULL;
 const char *filename;
@@ -450,6 +482,7 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict 
*options, int flags,
 }
 
 filename = qemu_opt_get(opts, "filename");
+secretid = qemu_opt_get(opts, "authsecret");
 
 if (qemu_rbd_parsename(filename, pool, sizeof(pool),
snap_buf, sizeof(snap_buf),
@@ -488,6 +521,10 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict 
*options, int flags,
 }
 }
 
+if (qemu_rbd_set_auth(s->cluster, secretid, errp) < 0) {
+goto failed_shutdown;
+}
+
 /*
  * Fallback to more conservative semantics if setting cache
  * options fails. Ignore errors from setting rbd_cache because the
@@ -919,6 +956,11 @@ static QemuOptsList qemu_rbd_create_opts = {
 .type = QEMU_OPT_SIZE,
 .help = "RBD object size"
 },
+{
+.name = "authsecret",
+.type = QEMU_OPT_STRING,
+.help = "ID of secret providing the password",
+},
 { /* end of list */ }
 }
 };
-- 
2.4.3

Re: [Qemu-block] [PATCH v6 18/39] block: Add BlockBackendRootState

[Max Reitz]

On 19.10.2015 16:18, Kevin Wolf wrote:
> Am 12.10.2015 um 22:00 hat Max Reitz geschrieben:
>> This structure will store some of the state of the root BDS if the BDS
>> tree is removed, so that state can be restored once a new BDS tree is
>> inserted.
>>
>> Signed-off-by: Max Reitz 
> 
> Random thought, not directly related to this series:
> 
> We currently don't have a way to create just a BlockBackend with no
> medium. If we were to add that, would we want that to be just like a
> missing -drive file=... option, or would it actually make sense to
> specify the BlockBackendRootState?

We don't? -drive if=none,id=foo works just fine for me. Issuing qemu-io
foo "read 0 512" then prints "read failed: " + strerror(ENOMEDIUM) to
stderr.

> If we want to do that, we might actually have found a reason why the
> 'options' key makes sense in blockdev-add. We could make it a union
> where you either describe a BlockBackendRootState or a BDS.

I really wouldn't want to use the BBRS for anything but legacy support
(i.e. change inheriting the options of the last medium)...

> Another related question is whether we want to separate out BB options
> (which would otherwise be shared between the BBRS and BDS variants) into
> their own dict. This dict could be optional and that would be the way to
> specify whether you want a BB on top or not. Candidates for this dict
> are id/rerror/werror.
> 
> There are more fields that exist in both the BDS and BBRS variants, but
> don't really belong to the BB (i.e. they end up in the real BBRS and not
> in the BB, and are only valid as long as no BDS is attached). These
> would not be moved to the BB options dict.
> 
> Any opinions?

Not on the latter part.

Max



signature.asc
Description: OpenPGP digital signature

[Qemu-block] [PATCH 17/17] block: remove support for writing to qcow/qcow2 encrypted images

[Daniel P. Berrange]

Refuse to open a qcow/qcow2 image with encryption if write
access has been requested. To enable historic data to be
liberated support for reading images is retained, as it
does not pose an unreasonable support burden now that the
new key handling infrastructure is inplace.

Signed-off-by: Daniel P. Berrange 
---
 block.c|  7 ---
 block/qcow.c   |  6 ++
 block/qcow2.c  |  6 ++
 tests/qemu-iotests/087.out | 24 
 tests/qemu-iotests/134 |  8 
 tests/qemu-iotests/134.out | 36 +---
 6 files changed, 29 insertions(+), 58 deletions(-)

diff --git a/block.c b/block.c
index bb8cb96..afb4003 100644
--- a/block.c
+++ b/block.c
@@ -923,13 +923,6 @@ static int bdrv_open_common(BlockDriverState *bs, 
BdrvChild *file,
 goto free_and_fail;
 }
 
-if (bs->encrypted) {
-error_report("Encrypted images are deprecated");
-error_printf("Support for them will be removed in a future release.\n"
- "You can use 'qemu-img convert' to convert your image"
- " to an unencrypted one.\n");
-}
-
 ret = refresh_total_sectors(bs, bs->total_sectors);
 if (ret < 0) {
 error_setg_errno(errp, -ret, "Could not refresh total sector count");
diff --git a/block/qcow.c b/block/qcow.c
index ccf6de1..1914e5e 100644
--- a/block/qcow.c
+++ b/block/qcow.c
@@ -224,6 +224,12 @@ static int qcow_open(BlockDriverState *bs, QDict *options, 
int flags,
 }
 s->crypt_method_header = header.crypt_method;
 if (s->crypt_method_header) {
+if (flags & BDRV_O_RDWR) {
+error_setg(errp,
+   "Writing of encrypted qcow images is no longer 
supported");
+ret = -ENOSYS;
+goto fail;
+}
 bs->encrypted = 1;
 }
 if (!(flags & BDRV_O_NO_IO) &&
diff --git a/block/qcow2.c b/block/qcow2.c
index c93df92..a2559f8 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -1028,6 +1028,12 @@ static int qcow2_open(BlockDriverState *bs, QDict 
*options, int flags,
 }
 s->crypt_method_header = header.crypt_method;
 if (s->crypt_method_header) {
+if (flags & BDRV_O_RDWR) {
+error_setg(errp,
+   "Writing of encrypted qcow2 images is no longer 
supported");
+ret = -ENOSYS;
+goto fail;
+}
 bs->encrypted = 1;
 }
 
diff --git a/tests/qemu-iotests/087.out b/tests/qemu-iotests/087.out
index 50ce50d..0cd656a 100644
--- a/tests/qemu-iotests/087.out
+++ b/tests/qemu-iotests/087.out
@@ -38,21 +38,13 @@ QMP_VERSION
 
 === Encrypted image ===
 
-qemu-img: Encrypted images are deprecated
-Support for them will be removed in a future release.
-You can use 'qemu-img convert' to convert your image to an unencrypted one.
-qemu-img: Encrypted images are deprecated
-Support for them will be removed in a future release.
-You can use 'qemu-img convert' to convert your image to an unencrypted one.
+qemu-img: TEST_DIR/t.IMGFMT: Writing of encrypted IMGFMT images is no longer 
supported
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on
 Testing: -S
 QMP_VERSION
 {"return": {}}
 {"return": {}}
-Encrypted images are deprecated
-Support for them will be removed in a future release.
-You can use 'qemu-img convert' to convert your image to an unencrypted one.
-{"return": {}}
+{"error": {"class": "GenericError", "desc": "Writing of encrypted qcow2 images 
is no longer supported"}}
 {"return": {}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": 
"SHUTDOWN"}
 
@@ -60,22 +52,14 @@ Testing:
 QMP_VERSION
 {"return": {}}
 {"return": {}}
-Encrypted images are deprecated
-Support for them will be removed in a future release.
-You can use 'qemu-img convert' to convert your image to an unencrypted one.
-{"return": {}}
+{"error": {"class": "GenericError", "desc": "Writing of encrypted qcow2 images 
is no longer supported"}}
 {"return": {}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": 
"SHUTDOWN"}
 
 
 === Missing driver ===
 
-qemu-img: Encrypted images are deprecated
-Support for them will be removed in a future release.
-You can use 'qemu-img convert' to convert your image to an unencrypted one.
-qemu-img: Encrypted images are deprecated
-Support for them will be removed in a future release.
-You can use 'qemu-img convert' to convert your image to an unencrypted one.
+qemu-img: TEST_DIR/t.IMGFMT: Writing of encrypted IMGFMT images is no longer 
supported
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on
 Testing: -S
 QMP_VERSION
diff --git a/tests/qemu-iotests/134 b/tests/qemu-iotests/134
index a247473..5b094bc 100755
--- a/tests/qemu-iotests/134
+++ b/tests/qemu-iotests/134
@@ -55,19 +55,19 @@ TEST_IMG="driver=qcow2,file=$TEST_IMG,keyid=sec0"
 
 echo
 echo "== reading whole image =="
-$QEMU_IO --object 

Re: [Qemu-block] [PATCH 01/17] crypto: add QCryptoSecret object class for password/key handling

[Paolo Bonzini]

On 19/10/2015 17:09, Daniel P. Berrange wrote:
> +
> +switch (secret->format) {
> +case QCRYPTO_SECRET_FORMAT_UTF8:
> +if (!g_utf8_validate(input, strlen(input), NULL)) {
> +error_setg(errp,
> +   "Data from secret %s is not valid UTF-8",
> +   secretid);
> +goto cleanup;
> +}
> +output = input;
> +input = NULL;
> +break;

Why validate secrets as UTF-8?  In other words why have "utf8" instead
of "binary" as a possible QCryptoSecretFormat?

Paolo

[Qemu-block] [PATCH 02/17] crypto: add support for loading encrypted x509 keys

[Daniel P. Berrange]

Make use of the QCryptoSecret object to support loading of
encrypted x509 keys. The optional 'passwordid' parameter
to the tls-creds-x509 object type, provides the ID of a
secret object instance that holds the decryption password
for the PEM file.

 # echo "123456" > mypasswd.txt
 # $QEMU \
-object secret,id=sec0,filename=mypasswd.txt \
-object tls-creds-x509,passwordid=sec0,id=creds0,\
dir=/home/berrange/.pki/qemu,endpoint=server \
-vnc :1,tls-creds=creds0

This requires QEMU to be linked to GNUTLS >= 3.1.11. If
GNUTLS is too old an error will be reported if an attempt
is made to pass a decryption password.

Signed-off-by: Daniel P. Berrange 
---
 crypto/tlscredsx509.c | 47 +++
 include/crypto/tlscredsx509.h |  1 +
 qemu-options.hx   |  8 +++-
 3 files changed, 55 insertions(+), 1 deletion(-)

diff --git a/crypto/tlscredsx509.c b/crypto/tlscredsx509.c
index dc46bc4..637465f 100644
--- a/crypto/tlscredsx509.c
+++ b/crypto/tlscredsx509.c
@@ -20,6 +20,7 @@
 
 #include "crypto/tlscredsx509.h"
 #include "crypto/tlscredspriv.h"
+#include "crypto/secret.h"
 #include "qom/object_interfaces.h"
 #include "trace.h"
 
@@ -605,9 +606,29 @@ qcrypto_tls_creds_x509_load(QCryptoTLSCredsX509 *creds,
 }
 
 if (cert != NULL && key != NULL) {
+#if GNUTLS_VERSION_NUMBER >= 0x030111
+char *password = NULL;
+if (creds->passwordid) {
+password = qcrypto_secret_lookup_as_utf8(creds->passwordid,
+ errp);
+if (!password) {
+goto cleanup;
+}
+}
+ret = gnutls_certificate_set_x509_key_file2(creds->data,
+cert, key,
+GNUTLS_X509_FMT_PEM,
+password,
+0);
+#else /* GNUTLS_VERSION_NUMBER < 0x030111 */
+if (creds->passwordid) {
+error_setg(errp, "PKCS8 decryption requires GNUTLS >= 3.1.11");
+goto cleanup;
+}
 ret = gnutls_certificate_set_x509_key_file(creds->data,
cert, key,
GNUTLS_X509_FMT_PEM);
+#endif /* GNUTLS_VERSION_NUMBER < 0x030111 */
 if (ret < 0) {
 error_setg(errp, "Cannot load certificate '%s' & key '%s': %s",
cert, key, gnutls_strerror(ret));
@@ -731,6 +752,27 @@ qcrypto_tls_creds_x509_prop_set_sanity(Object *obj,
 }
 
 
+static void
+qcrypto_tls_creds_x509_prop_set_passwordid(Object *obj,
+   const char *value,
+   Error **errp G_GNUC_UNUSED)
+{
+QCryptoTLSCredsX509 *creds = QCRYPTO_TLS_CREDS_X509(obj);
+
+creds->passwordid = g_strdup(value);
+}
+
+
+static char *
+qcrypto_tls_creds_x509_prop_get_passwordid(Object *obj,
+   Error **errp G_GNUC_UNUSED)
+{
+QCryptoTLSCredsX509 *creds = QCRYPTO_TLS_CREDS_X509(obj);
+
+return g_strdup(creds->passwordid);
+}
+
+
 static bool
 qcrypto_tls_creds_x509_prop_get_sanity(Object *obj,
Error **errp G_GNUC_UNUSED)
@@ -763,6 +805,10 @@ qcrypto_tls_creds_x509_init(Object *obj)
  qcrypto_tls_creds_x509_prop_get_sanity,
  qcrypto_tls_creds_x509_prop_set_sanity,
  NULL);
+object_property_add_str(obj, "passwordid",
+qcrypto_tls_creds_x509_prop_get_passwordid,
+qcrypto_tls_creds_x509_prop_set_passwordid,
+NULL);
 }
 
 
@@ -771,6 +817,7 @@ qcrypto_tls_creds_x509_finalize(Object *obj)
 {
 QCryptoTLSCredsX509 *creds = QCRYPTO_TLS_CREDS_X509(obj);
 
+g_free(creds->passwordid);
 qcrypto_tls_creds_x509_unload(creds);
 }
 
diff --git a/include/crypto/tlscredsx509.h b/include/crypto/tlscredsx509.h
index b9785fd..25796d7 100644
--- a/include/crypto/tlscredsx509.h
+++ b/include/crypto/tlscredsx509.h
@@ -101,6 +101,7 @@ struct QCryptoTLSCredsX509 {
 gnutls_certificate_credentials_t data;
 #endif
 bool sanityCheck;
+char *passwordid;
 };
 
 
diff --git a/qemu-options.hx b/qemu-options.hx
index 831213b..c43893a 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -3616,7 +3616,7 @@ expensive operation that consumes random pool entropy, so 
it is
 recommended that a persistent set of parameters be generated
 upfront and saved.
 
-@item -object 
tls-creds-x509,id=@var{id},endpoint=@var{endpoint},dir=@var{/path/to/cred/dir},verify-peer=@var{on|off}
+@item -object 
tls-creds-x509,id=@var{id},endpoint=@var{endpoint},dir=@var{/path/to/cred/dir},verify-peer=@var{on|off},passwordid=@var{id}
 
 Creates a TLS 

[Qemu-block] [PATCH 09/17] qemu-img: add support for --object command line arg

[Daniel P. Berrange]

Allow creation of user creatable object types with qemu-img
via a --object command line arg. This will be used to supply
passwords and/or encryption keys to the various block driver
backends via the recently added 'secret' object type.

 # echo -n letmein > mypasswd.txt
 # qemu-img info --object secret,id=sec0,file=mypasswd.txt \
  ...other info args...

Signed-off-by: Daniel P. Berrange 
---
 qemu-img-cmds.hx |  44 
 qemu-img.c   | 300 +--
 qemu-img.texi|   8 ++
 3 files changed, 322 insertions(+), 30 deletions(-)

diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index 9567774..5bb1de7 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -10,68 +10,68 @@ STEXI
 ETEXI
 
 DEF("check", img_check,
-"check [-q] [-f fmt] [--output=ofmt] [-r [leaks | all]] [-T src_cache] 
filename")
+"check [-q] [--object objectdef] [-f fmt] [--output=ofmt] [-r [leaks | 
all]] [-T src_cache] filename")
 STEXI
-@item check [-q] [-f @var{fmt}] [--output=@var{ofmt}] [-r [leaks | all]] [-T 
@var{src_cache}] @var{filename}
+@item check [--object objectdef] [-q] [-f @var{fmt}] [--output=@var{ofmt}] [-r 
[leaks | all]] [-T @var{src_cache}] @var{filename}
 ETEXI
 
 DEF("create", img_create,
-"create [-q] [-f fmt] [-o options] filename [size]")
+"create [-q] [--object objectdef] [-f fmt] [-o options] filename [size]")
 STEXI
-@item create [-q] [-f @var{fmt}] [-o @var{options}] @var{filename} [@var{size}]
+@item create [--object objectdef] [-q] [-f @var{fmt}] [-o @var{options}] 
@var{filename} [@var{size}]
 ETEXI
 
 DEF("commit", img_commit,
-"commit [-q] [-f fmt] [-t cache] [-b base] [-d] [-p] filename")
+"commit [-q] [--object objectdef] [-f fmt] [-t cache] [-b base] [-d] [-p] 
filename")
 STEXI
-@item commit [-q] [-f @var{fmt}] [-t @var{cache}] [-b @var{base}] [-d] [-p] 
@var{filename}
+@item commit [--object objectdef] [-q] [-f @var{fmt}] [-t @var{cache}] [-b 
@var{base}] [-d] [-p] @var{filename}
 ETEXI
 
 DEF("compare", img_compare,
-"compare [-f fmt] [-F fmt] [-T src_cache] [-p] [-q] [-s] filename1 
filename2")
+"compare [--object objectdef] [-f fmt] [-F fmt] [-T src_cache] [-p] [-q] 
[-s] filename1 filename2")
 STEXI
-@item compare [-f @var{fmt}] [-F @var{fmt}] [-T @var{src_cache}] [-p] [-q] 
[-s] @var{filename1} @var{filename2}
+@item compare [--object objectdef] [-f @var{fmt}] [-F @var{fmt}] [-T 
@var{src_cache}] [-p] [-q] [-s] @var{filename1} @var{filename2}
 ETEXI
 
 DEF("convert", img_convert,
-"convert [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O 
output_fmt] [-o options] [-s snapshot_id_or_name] [-l snapshot_param] [-S 
sparse_size] filename [filename2 [...]] output_filename")
+"convert [--object objectdef] [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T 
src_cache] [-O output_fmt] [-o options] [-s snapshot_id_or_name] [-l 
snapshot_param] [-S sparse_size] filename [filename2 [...]] output_filename")
 STEXI
-@item convert [-c] [-p] [-q] [-n] [-f @var{fmt}] [-t @var{cache}] [-T 
@var{src_cache}] [-O @var{output_fmt}] [-o @var{options}] [-s 
@var{snapshot_id_or_name}] [-l @var{snapshot_param}] [-S @var{sparse_size}] 
@var{filename} [@var{filename2} [...]] @var{output_filename}
+@item convert [--object objectdef] [-c] [-p] [-q] [-n] [-f @var{fmt}] [-t 
@var{cache}] [-T @var{src_cache}] [-O @var{output_fmt}] [-o @var{options}] [-s 
@var{snapshot_id_or_name}] [-l @var{snapshot_param}] [-S @var{sparse_size}] 
@var{filename} [@var{filename2} [...]] @var{output_filename}
 ETEXI
 
 DEF("info", img_info,
-"info [-f fmt] [--output=ofmt] [--backing-chain] filename")
+"info [--object objectdef] [-f fmt] [--output=ofmt] [--backing-chain] 
filename")
 STEXI
-@item info [-f @var{fmt}] [--output=@var{ofmt}] [--backing-chain] 
@var{filename}
+@item info [--object objectdef] [-f @var{fmt}] [--output=@var{ofmt}] 
[--backing-chain] @var{filename}
 ETEXI
 
 DEF("map", img_map,
-"map [-f fmt] [--output=ofmt] filename")
+"map [--object objectdef] [-f fmt] [--output=ofmt] filename")
 STEXI
-@item map [-f @var{fmt}] [--output=@var{ofmt}] @var{filename}
+@item map [--object objectdef] [-f @var{fmt}] [--output=@var{ofmt}] 
@var{filename}
 ETEXI
 
 DEF("snapshot", img_snapshot,
-"snapshot [-q] [-l | -a snapshot | -c snapshot | -d snapshot] filename")
+"snapshot [--object objectdef] [-q] [-l | -a snapshot | -c snapshot | -d 
snapshot] filename")
 STEXI
-@item snapshot [-q] [-l | -a @var{snapshot} | -c @var{snapshot} | -d 
@var{snapshot}] @var{filename}
+@item snapshot [--object objectdef] [-q] [-l | -a @var{snapshot} | -c 
@var{snapshot} | -d @var{snapshot}] @var{filename}
 ETEXI
 
 DEF("rebase", img_rebase,
-"rebase [-q] [-f fmt] [-t cache] [-T src_cache] [-p] [-u] -b backing_file 
[-F backing_fmt] filename")
+"rebase [--object objectdef] [-q] [-f fmt] [-t cache] [-T src_cache] [-p] 
[-u] -b backing_file [-F backing_fmt] filename")
 STEXI
-@item rebase [-q] [-f @var{fmt}] [-t 

[Qemu-block] [PATCH 12/17] qemu-io: allow specifying image as a set of options args

[Daniel P. Berrange]

Currently qemu-io allows an image filename to be passed on the
command line, but does not have a way to set any options except
the format eg

 qemu-io https://127.0.0.1/images/centos7.iso
 qemu-io /home/berrange/demo.qcow2

This adds a --source arg (that is mutually exclusive with a
positional filename arg and -f arg) that accepts a full option
string, as well as the original syntax eg

 qemu-io --source driver=http,url=https://127.0.0.1/images,sslverify=off
 qemu-io --source https://127.0.0.1/images/centos7.iso
 qemu-io --source file=/home/berrange/demo.qcow2
 qemu-io --source /home/berrange/demo.qcow2

Signed-off-by: Daniel P. Berrange 
---
 qemu-io.c | 37 -
 1 file changed, 36 insertions(+), 1 deletion(-)

diff --git a/qemu-io.c b/qemu-io.c
index cf1dac6..fc7f81b 100644
--- a/qemu-io.c
+++ b/qemu-io.c
@@ -373,6 +373,7 @@ static void reenable_tty_echo(void)
 
 enum {
 OPTION_OBJECT = 258,
+OPTION_SOURCE = 259,
 };
 
 static QemuOptsList qemu_object_opts = {
@@ -436,6 +437,16 @@ out:
 return 0;
 }
 
+static QemuOptsList file_opts = {
+.name = "file",
+.implied_opt_name = "file",
+.head = QTAILQ_HEAD_INITIALIZER(file_opts.head),
+.desc = {
+/* no elements => accept any params */
+{ /* end of list */ }
+},
+};
+
 int main(int argc, char **argv)
 {
 int readonly = 0;
@@ -455,6 +466,7 @@ int main(int argc, char **argv)
 { "cache", 1, NULL, 't' },
 { "trace", 1, NULL, 'T' },
 { "object", 1, NULL, OPTION_OBJECT },
+{ "source", 1, NULL, OPTION_SOURCE },
 { NULL, 0, NULL, 0 }
 };
 int c;
@@ -531,6 +543,12 @@ int main(int argc, char **argv)
 exit(1);
 }
 break;
+case OPTION_SOURCE:
+if (!qemu_opts_parse_noisily(_opts, optarg, false)) {
+qemu_opts_reset(_opts);
+return 0;
+}
+break;
 default:
 usage(progname);
 exit(1);
@@ -572,7 +590,24 @@ int main(int argc, char **argv)
 flags |= BDRV_O_RDWR;
 }
 
-if ((argc - optind) == 1) {
+qopts = qemu_opts_find(_opts, NULL);
+if (qopts) {
+char *file;
+if (opts) {
+error_report("--source and -f are mutually exclusive");
+exit(1);
+}
+if ((argc - optind) == 1) {
+error_report("--source and filename are mutually exclusive");
+exit(1);
+}
+file = g_strdup(qemu_opt_get(qopts, "file"));
+qemu_opt_unset(qopts, "file");
+opts = qemu_opts_to_qdict(qopts, NULL);
+qemu_opts_reset(_opts);
+openfile(file, flags, opts);
+g_free(file);
+} else if ((argc - optind) == 1) {
 openfile(argv[optind], flags, opts);
 }
 command_loop();
-- 
2.4.3

Re: [Qemu-block] [PATCH 0/3] Add 'blockdev-del' command

[Kevin Wolf]

Am 19.10.2015 um 16:15 hat Alberto Garcia geschrieben:
> On Mon 19 Oct 2015 01:27:45 PM CEST, Kevin Wolf wrote:
> > I've been thinking a bit about the creation and deletion of
> > BlockBackends a bit last week, and honestly it still feels a bit messy
> > (or maybe I just don't fully understand it yet). Anyway, the cover
> > letter of this series seems to be a good place for trying to write it
> > down.
> 
> Thanks for the summary!
> 
> > So we seem to have two criteria to distinguish BDSes:
> >
> > 1. Does/Doesn't have a BlockBackend on top.
> >In the future, multiple BlockBackends are possible, too.
> 
> One single BDS attached to multiple BlockBackends at the same time?
> What's the use case?

For having multiple users of the BDS, e.g. a guest device and an NBD
server.

Or sometimes maybe just because it's the logical thing to happen:
Imagine an image B with a backing file A. Both have a BlockBackend on
them. Do a live commit of B into A. On completion, the BDS B is deleted
and both BBs point to A.

> > 1. BB without BDS
> > 2. BB attached to BDS without monitor reference
> > 3. BB attached to BDS with monitor reference
> > 4. BDS without BB and without monitor reference
> > 5. BDS without BB and with monitor reference
> 
> I would say that the rule that we should follow is: if the outcome is
> not clear for a particular case, then the command fails. We should try
> not to guess what the user wants. blockdev-del should only delete
> something when there's only one sane alternative.
> 
> > Let me start with the first two questions for all cases:
> >
> > 1. As far as I know, it's currently not possible to directly create a BB
> >without a BDS (or before Max' series with an empty BDS). You have to
> >create a BB with an opened BDS and then eject that. Oops.
> >
> >blockdev-del is easy: It deletes the BB and nothing else.
> 
> I agree.
> 
> > 2. This is the normal case, easy to produce with blockdev-add.
> >The two options for deleting something are removing the BDS while
> >keeping the BB (this is already covered by 'eject') and dropping the
> >BB (probably makes sense to use 'blockdev-del' here). There is no
> >monitor reference to the BDS anyway, so blockdev-dev can't delete
> >that.
> >
> >Dropping the BB automatically also drops the BDS in simple cases. In
> >more complicated cases where the BDS has got more references later,
> >it might still exist. Then the blockdev-del wasn't the exact opposite
> >operation of blockdev-add.
> >
> >Is this a problem for a user/management tool that wants to keep track
> >of which image files are still in use?
> 
> I would say that if the BDS has additional references then
> 'blockdev-del' should fail and do nothing (what the current patch does).

Sorry for not having read the patches yet. I tried to figure out what
the right thing to do is before I could start reviewing whether the
patches do the thing right.

I agree with your solution, which addresses my question.

> > 3. A BDS with a monitor reference can be created (with some patches) by
> >using blockdev-add with a node-name, but no id. Directly attaching it
> >to a BB is tricky today, even though I'm sure you could do it with
> >some clever use of block jobs... With 1. fixed (i.e. you can create
> >an empty BB), insert-medium should do the job.
> >
> >Here we have even more choices for deleting something: Delete the BB,
> >but leave the BDS alone; delete the BDS and leave an empty BB behind
> >(this is different from eject because eject would drop the connection
> >between BB and BDS, but both would continue to exist!); delete both
> >at once.
> >
> >We had two separate blockdev-add commands for the BDS and the BB, so
> >it makes sense to require two separate blockdev-del commands as well.
> >In order to keep blockdev-add/del symmetrical, we would have to make
> >blockdev-del of a node-name delete the BDS and blockdev-del of an id
> >delete the BB.
> 
> This is a tricky case, thanks for pointing it out. I would also go for
> the conservative option here: if you create a BDS with blockdev-add and
> then attach it to a BlockBackend, you're adding one additional reference
> (via blk_insert_bs()). Therefore blockdev-del would fail like in case 2.
> 
> You need to eject the BDS first in order to decide which one you want to
> delete.

Okay, so is the following your generalised rule?

If device refers to a BDS without a BB or to a BB without a BDS, then
just delete it. Otherwise, try to treat one BDS with one BB as a unit
and remove that unit. They can only be treated as a unit if the BDS is
only referenced by the BB (i.e. not by the monitor either). If they
can't be treated as a unit, error out.

I think that's safe and allows using the usual aliasing (using the BB
name or the node-name of its BDS means the same thing), which is good
for consistency.

My only concern with this is whether it isn't too 

Re: [Qemu-block] [PATCH 01/17] crypto: add QCryptoSecret object class for password/key handling

[Daniel P. Berrange]

On Mon, Oct 19, 2015 at 05:18:56PM +0200, Paolo Bonzini wrote:
> 
> 
> On 19/10/2015 17:09, Daniel P. Berrange wrote:
> > +
> > +switch (secret->format) {
> > +case QCRYPTO_SECRET_FORMAT_UTF8:
> > +if (!g_utf8_validate(input, strlen(input), NULL)) {
> > +error_setg(errp,
> > +   "Data from secret %s is not valid UTF-8",
> > +   secretid);
> > +goto cleanup;
> > +}
> > +output = input;
> > +input = NULL;
> > +break;
> 
> Why validate secrets as UTF-8?  In other words why have "utf8" instead
> of "binary" as a possible QCryptoSecretFormat?

JSON doesn't accept arbitrary 8-bit binary data, so the alternative
'base64' is effectively providing binary data facility. Having to
use base64 for plain passwords is rather tedious though, so allowing
utf8 is a much more developer friendly approach for people using QEMU
without a mgmt tool like libvirt.

NB, this dual-format utf8-or-base64 approach matches the approach used
in QEMU guest agent for the guest-file-read/write commands for the same
reason.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

[Qemu-block] [PATCH v7 02/39] block: Set BDRV_O_INCOMING in bdrv_fill_options()

[Max Reitz]

This flag should not be set for the root BDS only, but for any BDS that
is being created while incoming migration is pending, so setting it is
moved from blockdev_init() to bdrv_fill_options().

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Kevin Wolf 
Reviewed-by: Alberto Garcia 
---
 block.c| 4 
 blockdev.c | 4 
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/block.c b/block.c
index 09f2a75..4fa2057 100644
--- a/block.c
+++ b/block.c
@@ -1081,6 +1081,10 @@ static int bdrv_fill_options(QDict **options, const char 
**pfilename,
 }
 }
 
+if (runstate_check(RUN_STATE_INMIGRATE)) {
+*flags |= BDRV_O_INCOMING;
+}
+
 return 0;
 }
 
diff --git a/blockdev.c b/blockdev.c
index 8141b6b..27398b1 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -537,10 +537,6 @@ static BlockBackend *blockdev_init(const char *file, QDict 
*bs_opts,
 bdrv_flags |= BDRV_O_COPY_ON_READ;
 }
 
-if (runstate_check(RUN_STATE_INMIGRATE)) {
-bdrv_flags |= BDRV_O_INCOMING;
-}
-
 bdrv_flags |= ro ? 0 : BDRV_O_RDWR;
 
 blk = blk_new_open(qemu_opts_id(opts), file, NULL, bs_opts, bdrv_flags,
-- 
2.6.1

[Qemu-block] [PATCH v7 05/39] block: Make bdrv_is_inserted() return a bool

[Max Reitz]

Make bdrv_is_inserted(), blk_is_inserted(), and the callback
BlockDriver.bdrv_is_inserted() return a bool.

Suggested-by: Eric Blake 
Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Alberto Garcia 
Reviewed-by: Kevin Wolf 
---
 block.c| 12 +++-
 block/block-backend.c  |  2 +-
 block/raw-posix.c  |  8 +++-
 block/raw_bsd.c|  2 +-
 include/block/block.h  |  2 +-
 include/block/block_int.h  |  2 +-
 include/sysemu/block-backend.h |  2 +-
 7 files changed, 15 insertions(+), 15 deletions(-)

diff --git a/block.c b/block.c
index 4fa2057..2a77169 100644
--- a/block.c
+++ b/block.c
@@ -3140,14 +3140,16 @@ void bdrv_invalidate_cache_all(Error **errp)
 /**
  * Return TRUE if the media is present
  */
-int bdrv_is_inserted(BlockDriverState *bs)
+bool bdrv_is_inserted(BlockDriverState *bs)
 {
 BlockDriver *drv = bs->drv;
 
-if (!drv)
-return 0;
-if (!drv->bdrv_is_inserted)
-return 1;
+if (!drv) {
+return false;
+}
+if (!drv->bdrv_is_inserted) {
+return true;
+}
 return drv->bdrv_is_inserted(bs);
 }
 
diff --git a/block/block-backend.c b/block/block-backend.c
index 2256551..1db002c 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -769,7 +769,7 @@ void blk_invalidate_cache(BlockBackend *blk, Error **errp)
 bdrv_invalidate_cache(blk->bs, errp);
 }
 
-int blk_is_inserted(BlockBackend *blk)
+bool blk_is_inserted(BlockBackend *blk)
 {
 return bdrv_is_inserted(blk->bs);
 }
diff --git a/block/raw-posix.c b/block/raw-posix.c
index 2e3db44..918c756 100644
--- a/block/raw-posix.c
+++ b/block/raw-posix.c
@@ -2398,15 +2398,13 @@ out:
 return prio;
 }
 
-static int cdrom_is_inserted(BlockDriverState *bs)
+static bool cdrom_is_inserted(BlockDriverState *bs)
 {
 BDRVRawState *s = bs->opaque;
 int ret;
 
 ret = ioctl(s->fd, CDROM_DRIVE_STATUS, CDSL_CURRENT);
-if (ret == CDS_DISC_OK)
-return 1;
-return 0;
+return ret == CDS_DISC_OK;
 }
 
 static void cdrom_eject(BlockDriverState *bs, bool eject_flag)
@@ -2532,7 +2530,7 @@ static int cdrom_reopen(BlockDriverState *bs)
 return 0;
 }
 
-static int cdrom_is_inserted(BlockDriverState *bs)
+static bool cdrom_is_inserted(BlockDriverState *bs)
 {
 return raw_getlength(bs) > 0;
 }
diff --git a/block/raw_bsd.c b/block/raw_bsd.c
index 63ee911..3c7b413 100644
--- a/block/raw_bsd.c
+++ b/block/raw_bsd.c
@@ -154,7 +154,7 @@ static int raw_truncate(BlockDriverState *bs, int64_t 
offset)
 return bdrv_truncate(bs->file->bs, offset);
 }
 
-static int raw_is_inserted(BlockDriverState *bs)
+static bool raw_is_inserted(BlockDriverState *bs)
 {
 return bdrv_is_inserted(bs->file->bs);
 }
diff --git a/include/block/block.h b/include/block/block.h
index 6d70eb4..db860ef 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -399,7 +399,7 @@ int bdrv_is_read_only(BlockDriverState *bs);
 int bdrv_is_sg(BlockDriverState *bs);
 int bdrv_enable_write_cache(BlockDriverState *bs);
 void bdrv_set_enable_write_cache(BlockDriverState *bs, bool wce);
-int bdrv_is_inserted(BlockDriverState *bs);
+bool bdrv_is_inserted(BlockDriverState *bs);
 int bdrv_media_changed(BlockDriverState *bs);
 void bdrv_lock_medium(BlockDriverState *bs, bool locked);
 void bdrv_eject(BlockDriverState *bs, bool eject_flag);
diff --git a/include/block/block_int.h b/include/block/block_int.h
index c0e6513..40d40df 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -212,7 +212,7 @@ struct BlockDriver {
 const char *backing_file, const char *backing_fmt);
 
 /* removable device specific */
-int (*bdrv_is_inserted)(BlockDriverState *bs);
+bool (*bdrv_is_inserted)(BlockDriverState *bs);
 int (*bdrv_media_changed)(BlockDriverState *bs);
 void (*bdrv_eject)(BlockDriverState *bs, bool eject_flag);
 void (*bdrv_lock_medium)(BlockDriverState *bs, bool locked);
diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h
index 8fc960f..8f2bf10 100644
--- a/include/sysemu/block-backend.h
+++ b/include/sysemu/block-backend.h
@@ -130,7 +130,7 @@ int blk_is_sg(BlockBackend *blk);
 int blk_enable_write_cache(BlockBackend *blk);
 void blk_set_enable_write_cache(BlockBackend *blk, bool wce);
 void blk_invalidate_cache(BlockBackend *blk, Error **errp);
-int blk_is_inserted(BlockBackend *blk);
+bool blk_is_inserted(BlockBackend *blk);
 void blk_lock_medium(BlockBackend *blk, bool locked);
 void blk_eject(BlockBackend *blk, bool eject_flag);
 int blk_get_flags(BlockBackend *blk);
-- 
2.6.1

[Qemu-block] [PATCH v7 38/39] hmp: Add read-only-mode option to change command

[Max Reitz]

Expose the new read-only-mode option of 'blockdev-change-medium' for the
'change' HMP command.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
---
 hmp-commands.hx | 20 +---
 hmp.c   | 22 +-
 2 files changed, 38 insertions(+), 4 deletions(-)

diff --git a/hmp-commands.hx b/hmp-commands.hx
index 3a4ae39..814ea86 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -194,8 +194,8 @@ ETEXI
 
 {
 .name   = "change",
-.args_type  = "device:B,target:F,arg:s?",
-.params = "device filename [format]",
+.args_type  = "device:B,target:F,arg:s?,read-only-mode:s?",
+.params = "device filename [format [read-only-mode]]",
 .help   = "change a removable medium, optional format",
 .mhandler.cmd = hmp_change,
 },
@@ -206,7 +206,7 @@ STEXI
 Change the configuration of a device.
 
 @table @option
-@item change @var{diskdevice} @var{filename} [@var{format}]
+@item change @var{diskdevice} @var{filename} [@var{format} 
[@var{read-only-mode}]]
 Change the medium for a removable disk device to point to @var{filename}. eg
 
 @example
@@ -215,6 +215,20 @@ Change the medium for a removable disk device to point to 
@var{filename}. eg
 
 @var{format} is optional.
 
+@var{read-only-mode} may be used to change the read-only status of the device.
+It accepts the following values:
+
+@table @var
+@item retain
+Retains the current status; this is the default.
+
+@item read-only
+Makes the device read-only.
+
+@item read-write
+Makes the device writable.
+@end table
+
 @item change vnc @var{display},@var{options}
 Change the configuration of the VNC server. The valid syntax for @var{display}
 and @var{options} are described at @ref{sec_invocation}. eg
diff --git a/hmp.c b/hmp.c
index 9e6b7e5..28caa7d 100644
--- a/hmp.c
+++ b/hmp.c
@@ -27,6 +27,7 @@
 #include "qapi/opts-visitor.h"
 #include "qapi/qmp/qerror.h"
 #include "qapi/string-output-visitor.h"
+#include "qapi/util.h"
 #include "qapi-visit.h"
 #include "ui/console.h"
 #include "block/qapi.h"
@@ -1336,9 +1337,16 @@ void hmp_change(Monitor *mon, const QDict *qdict)
 const char *device = qdict_get_str(qdict, "device");
 const char *target = qdict_get_str(qdict, "target");
 const char *arg = qdict_get_try_str(qdict, "arg");
+const char *read_only = qdict_get_try_str(qdict, "read-only-mode");
+BlockdevChangeReadOnlyMode read_only_mode = 0;
 Error *err = NULL;
 
 if (strcmp(device, "vnc") == 0) {
+if (read_only) {
+monitor_printf(mon,
+   "Parameter 'read-only-mode' is invalid for VNC");
+return;
+}
 if (strcmp(target, "passwd") == 0 ||
 strcmp(target, "password") == 0) {
 if (!arg) {
@@ -1348,7 +1356,19 @@ void hmp_change(Monitor *mon, const QDict *qdict)
 }
 qmp_change("vnc", target, !!arg, arg, );
 } else {
-qmp_blockdev_change_medium(device, target, !!arg, arg, false, 0, );
+if (read_only) {
+read_only_mode =
+qapi_enum_parse(BlockdevChangeReadOnlyMode_lookup,
+read_only, BLOCKDEV_CHANGE_READ_ONLY_MODE_MAX,
+BLOCKDEV_CHANGE_READ_ONLY_MODE_RETAIN, );
+if (err) {
+hmp_handle_error(mon, );
+return;
+}
+}
+
+qmp_blockdev_change_medium(device, target, !!arg, arg,
+   !!read_only, read_only_mode, );
 if (err &&
 error_get_class(err) == ERROR_CLASS_DEVICE_ENCRYPTED) {
 error_free(err);
-- 
2.6.1

[Qemu-block] [PATCH v7 10/39] hw/block/fdc: Implement tray status

[Max Reitz]

The tray of an FDD is open iff there is no medium inserted (there are
only two states for an FDD: "medium inserted" or "no medium inserted").

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Kevin Wolf 
---
 hw/block/fdc.c   | 20 
 tests/fdc-test.c |  4 +---
 2 files changed, 17 insertions(+), 7 deletions(-)

diff --git a/hw/block/fdc.c b/hw/block/fdc.c
index 6686a72..4292ece 100644
--- a/hw/block/fdc.c
+++ b/hw/block/fdc.c
@@ -192,6 +192,8 @@ typedef struct FDrive {
 uint8_t ro;   /* Is read-only   */
 uint8_t media_changed;/* Is media changed   */
 uint8_t media_rate;   /* Data rate of medium*/
+
+bool media_inserted;  /* Is there a medium in the tray */
 } FDrive;
 
 static void fd_init(FDrive *drv)
@@ -261,7 +263,7 @@ static int fd_seek(FDrive *drv, uint8_t head, uint8_t 
track, uint8_t sect,
 #endif
 drv->head = head;
 if (drv->track != track) {
-if (drv->blk != NULL && blk_is_inserted(drv->blk)) {
+if (drv->media_inserted) {
 drv->media_changed = 0;
 }
 ret = 1;
@@ -270,7 +272,7 @@ static int fd_seek(FDrive *drv, uint8_t head, uint8_t 
track, uint8_t sect,
 drv->sect = sect;
 }
 
-if (drv->blk == NULL || !blk_is_inserted(drv->blk)) {
+if (!drv->media_inserted) {
 ret = 2;
 }
 
@@ -296,7 +298,7 @@ static void fd_revalidate(FDrive *drv)
 ro = blk_is_read_only(drv->blk);
 pick_geometry(drv->blk, _heads, _track,
   _sect, drv->drive, , );
-if (!blk_is_inserted(drv->blk)) {
+if (!drv->media_inserted) {
 FLOPPY_DPRINTF("No disk in drive\n");
 } else {
 FLOPPY_DPRINTF("Floppy disk (%d h %d t %d s) %s\n", nb_heads,
@@ -692,7 +694,7 @@ static bool fdrive_media_changed_needed(void *opaque)
 {
 FDrive *drive = opaque;
 
-return (drive->blk != NULL && drive->media_changed != 1);
+return (drive->media_inserted && drive->media_changed != 1);
 }
 
 static const VMStateDescription vmstate_fdrive_media_changed = {
@@ -2184,12 +2186,21 @@ static void fdctrl_change_cb(void *opaque, bool load)
 {
 FDrive *drive = opaque;
 
+drive->media_inserted = load && drive->blk && blk_is_inserted(drive->blk);
+
 drive->media_changed = 1;
 fd_revalidate(drive);
 }
 
+static bool fdctrl_is_tray_open(void *opaque)
+{
+FDrive *drive = opaque;
+return !drive->media_inserted;
+}
+
 static const BlockDevOps fdctrl_block_ops = {
 .change_media_cb = fdctrl_change_cb,
+.is_tray_open = fdctrl_is_tray_open,
 };
 
 /* Init functions */
@@ -2217,6 +2228,7 @@ static void fdctrl_connect_drives(FDCtrl *fdctrl, Error 
**errp)
 fdctrl_change_cb(drive, 0);
 if (drive->blk) {
 blk_set_dev_ops(drive->blk, _block_ops, drive);
+drive->media_inserted = blk_is_inserted(drive->blk);
 }
 }
 }
diff --git a/tests/fdc-test.c b/tests/fdc-test.c
index 416394f..b5a4696 100644
--- a/tests/fdc-test.c
+++ b/tests/fdc-test.c
@@ -304,9 +304,7 @@ static void test_media_insert(void)
 qmp_discard_response("{'execute':'change', 'arguments':{"
  " 'device':'floppy0', 'target': %s, 'arg': 'raw' }}",
  test_image);
-qmp_discard_response(""); /* ignore event
- (FIXME open -> open transition?!) */
-qmp_discard_response(""); /* ignore event */
+qmp_discard_response(""); /* ignore event (open -> close) */
 
 dir = inb(FLOPPY_BASE + reg_dir);
 assert_bit_set(dir, DSKCHG);
-- 
2.6.1

[Qemu-block] [PATCH v7 36/39] hmp: Use blockdev-change-medium for change command

[Max Reitz]

Use separate code paths for the two overloaded functions of the 'change'
HMP command, and invoke the 'blockdev-change-medium' QMP command if used
on a block device (by calling qmp_blockdev_change_medium()).

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
---
 hmp.c | 27 +++
 1 file changed, 15 insertions(+), 12 deletions(-)

diff --git a/hmp.c b/hmp.c
index 5048eee..b91821b 100644
--- a/hmp.c
+++ b/hmp.c
@@ -1338,22 +1338,25 @@ void hmp_change(Monitor *mon, const QDict *qdict)
 const char *arg = qdict_get_try_str(qdict, "arg");
 Error *err = NULL;
 
-if (strcmp(device, "vnc") == 0 &&
-(strcmp(target, "passwd") == 0 ||
- strcmp(target, "password") == 0)) {
-if (!arg) {
-monitor_read_password(mon, hmp_change_read_arg, NULL);
+if (strcmp(device, "vnc") == 0) {
+if (strcmp(target, "passwd") == 0 ||
+strcmp(target, "password") == 0) {
+if (!arg) {
+monitor_read_password(mon, hmp_change_read_arg, NULL);
+return;
+}
+}
+qmp_change("vnc", target, !!arg, arg, );
+} else {
+qmp_blockdev_change_medium(device, target, !!arg, arg, );
+if (err &&
+error_get_class(err) == ERROR_CLASS_DEVICE_ENCRYPTED) {
+error_free(err);
+monitor_read_block_device_key(mon, device, NULL, NULL);
 return;
 }
 }
 
-qmp_change(device, target, !!arg, arg, );
-if (err &&
-error_get_class(err) == ERROR_CLASS_DEVICE_ENCRYPTED) {
-error_free(err);
-monitor_read_block_device_key(mon, device, NULL, NULL);
-return;
-}
 hmp_handle_error(mon, );
 }
 
-- 
2.6.1

Re: [Qemu-block] [PATCH 01/17] crypto: add QCryptoSecret object class for password/key handling

[Daniel P. Berrange]

On Mon, Oct 19, 2015 at 05:40:08PM +0200, Paolo Bonzini wrote:
> 
> 
> On 19/10/2015 17:24, Daniel P. Berrange wrote:
> > JSON doesn't accept arbitrary 8-bit binary data, so the alternative
> > 'base64' is effectively providing binary data facility. Having to
> > use base64 for plain passwords is rather tedious though, so allowing
> > utf8 is a much more developer friendly approach for people using QEMU
> > without a mgmt tool like libvirt.
> > 
> > NB, this dual-format utf8-or-base64 approach matches the approach used
> > in QEMU guest agent for the guest-file-read/write commands for the same
> > reason.
> 
> The difference is that guest-file-read/write have the payload in JSON;
> for file-based secrets the payload is not JSON.

For non-file based secrets though, the payload *is* in the JSON,
and per the cover letter, I actually anticipate passing all
secrets inline in the JSON and only using the file backend for
loading the initial master key. This avoids the need to do
file handle passing and/or create lots of temporary files, when
hotplugging resources.

> So I think that "binary" (which is the default anyway) would fit all the
> usecases (direct over JSON, file-based, direct over command line).
> Direct over JSON would be limited to valid UTF-8, but that's just a
> limitation of the transport.

I don't think that's actually an acceptable limitation - I want the
inline data passing to be fully usable for non-UTF-8 data too.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

[Qemu-block] [PATCH 13/17] qemu-nbd: allow specifying image as a set of options args

[Daniel P. Berrange]

Currently qemu-nbd allows an image filename to be passed on the
command line, but does not have a way to set any options except
the format eg

   qemu-nbd https://127.0.0.1/images/centos7.iso
   qemu-nbd /home/berrange/demo.qcow2

This adds a --source arg (that is mutually exclusive with a
positional filename arg and -f arg) that accepts a full option
string, as well as the original syntax eg

   qemu-nbd --source driver=http,url=https://127.0.0.1/images,sslverify=off
   qemu-nbd --source https://127.0.0.1/images/centos7.iso
   qemu-nbd --source file=/home/berrange/demo.qcow2
   qemu-nbd --source /home/berrange/demo.qcow2

Signed-off-by: Daniel P. Berrange 
---
 qemu-nbd.c | 57 ++---
 1 file changed, 50 insertions(+), 7 deletions(-)

diff --git a/qemu-nbd.c b/qemu-nbd.c
index 7eca167..1100c75 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -49,6 +49,7 @@
 #define QEMU_NBD_OPT_DISCARD   3
 #define QEMU_NBD_OPT_DETECT_ZEROES 4
 #define QEMU_NBD_OPT_OBJECT5
+#define QEMU_NBD_OPT_SOURCE6
 
 static NBDExport *exp;
 static int verbose;
@@ -387,6 +388,16 @@ static SocketAddress *nbd_build_socket_address(const char 
*sockpath,
 }
 
 
+static QemuOptsList file_opts = {
+.name = "file",
+.implied_opt_name = "file",
+.head = QTAILQ_HEAD_INITIALIZER(file_opts.head),
+.desc = {
+/* no elements => accept any params */
+{ /* end of list */ }
+},
+};
+
 static QemuOptsList qemu_object_opts = {
 .name = "object",
 .implied_opt_name = "qom-type",
@@ -486,6 +497,7 @@ int main(int argc, char **argv)
 { "persistent", 0, NULL, 't' },
 { "verbose", 0, NULL, 'v' },
 { "object", 1, NULL, QEMU_NBD_OPT_OBJECT },
+{ "source", 1, NULL, QEMU_NBD_OPT_SOURCE },
 { NULL, 0, NULL, 0 }
 };
 int ch;
@@ -657,13 +669,23 @@ int main(int argc, char **argv)
 exit(1);
 }
 break;
+case QEMU_NBD_OPT_SOURCE:
+if (srcpath) {
+errx(EXIT_FAILURE, "--source can only be used once");
+}
+if (!qemu_opts_parse_noisily(_opts, optarg, true)) {
+qemu_opts_reset(_opts);
+exit(EXIT_FAILURE);
+}
+srcpath = optarg;
+break;
 case '?':
 errx(EXIT_FAILURE, "Try `%s --help' for more information.",
  argv[0]);
 }
 }
 
-if ((argc - optind) != 1) {
+if ((argc - optind) > 1) {
 errx(EXIT_FAILURE, "Invalid number of argument.\n"
  "Try `%s --help' for more information.",
  argv[0]);
@@ -757,15 +779,36 @@ int main(int argc, char **argv)
 bdrv_init();
 atexit(bdrv_close_all);
 
-if (fmt) {
-options = qdict_new();
-qdict_put(options, "driver", qstring_from_str(fmt));
+if (srcpath) {
+char *file = NULL;
+opts = qemu_opts_find(_opts, NULL);
+if (fmt) {
+errx(EXIT_FAILURE, "--source and -f are mutually exclusive");
+}
+if ((argc - optind) > 1) {
+errx(EXIT_FAILURE, "--source and filename are mutually exclusive");
+}
+file = g_strdup(qemu_opt_get(opts, "file"));
+qemu_opt_unset(opts, "file");
+options = qemu_opts_to_qdict(opts, NULL);
+qemu_opts_reset(_opts);
+blk = blk_new_open("hda", file, NULL, options, flags, _err);
+g_free(file);
+} else {
+if (fmt) {
+options = qdict_new();
+qdict_put(options, "driver", qstring_from_str(fmt));
+}
+if ((argc - optind) != 1) {
+errx(EXIT_FAILURE, "one of --source or filename are expected");
+}
+
+srcpath = argv[optind];
+blk = blk_new_open("hda", srcpath, NULL, options, flags, _err);
 }
 
-srcpath = argv[optind];
-blk = blk_new_open("hda", srcpath, NULL, options, flags, _err);
 if (!blk) {
-errx(EXIT_FAILURE, "Failed to blk_new_open '%s': %s", argv[optind],
+errx(EXIT_FAILURE, "Failed to blk_new_open '%s': %s", srcpath,
  error_get_pretty(local_err));
 }
 bs = blk_bs(blk);
-- 
2.4.3

[Qemu-block] [PATCH 11/17] qemu-io: add support for --object command line arg

[Daniel P. Berrange]

Allow creation of user creatable object types with qemu-io
via a --object command line arg. This will be used to supply
passwords and/or encryption keys to the various block driver
backends via the recently added 'secret' object type.

 # echo -n letmein > mypasswd.txt
 # qemu-io --object secret,id=sec0,file=mypasswd.txt \
  ...other args...

Signed-off-by: Daniel P. Berrange 
---
 qemu-io.c | 87 +++
 1 file changed, 87 insertions(+)

diff --git a/qemu-io.c b/qemu-io.c
index 269f17c..cf1dac6 100644
--- a/qemu-io.c
+++ b/qemu-io.c
@@ -21,6 +21,8 @@
 #include "qemu/config-file.h"
 #include "qemu/readline.h"
 #include "qapi/qmp/qstring.h"
+#include "qapi/opts-visitor.h"
+#include "qom/object_interfaces.h"
 #include "sysemu/block-backend.h"
 #include "block/block_int.h"
 #include "trace/control.h"
@@ -205,6 +207,9 @@ static void usage(const char *name)
 "Usage: %s [-h] [-V] [-rsnm] [-f FMT] [-c STRING] ... [file]\n"
 "QEMU Disk exerciser\n"
 "\n"
+"  --object OBJECTDEF   define a object such as 'secret' for\n"
+"   providing passwords and/or encryption\n"
+"   keys\n"
 "  -c, --cmd STRING execute command with its arguments\n"
 "   from the given string\n"
 "  -f, --format FMT specifies the block driver to use\n"
@@ -366,6 +371,71 @@ static void reenable_tty_echo(void)
 qemu_set_tty_echo(STDIN_FILENO, true);
 }
 
+enum {
+OPTION_OBJECT = 258,
+};
+
+static QemuOptsList qemu_object_opts = {
+.name = "object",
+.implied_opt_name = "qom-type",
+.head = QTAILQ_HEAD_INITIALIZER(qemu_object_opts.head),
+.desc = {
+{ }
+},
+};
+
+static int object_create(void *opaque, QemuOpts *opts, Error **errp)
+{
+Error *err = NULL;
+char *type = NULL;
+char *id = NULL;
+void *dummy = NULL;
+OptsVisitor *ov;
+QDict *pdict;
+
+ov = opts_visitor_new(opts);
+pdict = qemu_opts_to_qdict(opts, NULL);
+
+visit_start_struct(opts_get_visitor(ov), , NULL, NULL, 0, );
+if (err) {
+goto out;
+}
+
+qdict_del(pdict, "qom-type");
+visit_type_str(opts_get_visitor(ov), , "qom-type", );
+if (err) {
+goto out;
+}
+
+qdict_del(pdict, "id");
+visit_type_str(opts_get_visitor(ov), , "id", );
+if (err) {
+goto out;
+}
+
+user_creatable_add(type, id, pdict, opts_get_visitor(ov), );
+if (err) {
+goto out;
+}
+visit_end_struct(opts_get_visitor(ov), );
+if (err) {
+user_creatable_del(id, NULL);
+}
+
+out:
+opts_visitor_cleanup(ov);
+
+QDECREF(pdict);
+g_free(id);
+g_free(type);
+g_free(dummy);
+if (err) {
+error_report_err(err);
+return -1;
+}
+return 0;
+}
+
 int main(int argc, char **argv)
 {
 int readonly = 0;
@@ -384,6 +454,7 @@ int main(int argc, char **argv)
 { "discard", 1, NULL, 'd' },
 { "cache", 1, NULL, 't' },
 { "trace", 1, NULL, 'T' },
+{ "object", 1, NULL, OPTION_OBJECT },
 { NULL, 0, NULL, 0 }
 };
 int c;
@@ -391,6 +462,7 @@ int main(int argc, char **argv)
 int flags = BDRV_O_UNMAP;
 Error *local_error = NULL;
 QDict *opts = NULL;
+QemuOpts *qopts = NULL;
 
 #ifdef CONFIG_POSIX
 signal(SIGPIPE, SIG_IGN);
@@ -399,6 +471,8 @@ int main(int argc, char **argv)
 progname = basename(argv[0]);
 qemu_init_exec_dir(argv[0]);
 
+module_call_init(MODULE_INIT_QOM);
+qemu_add_opts(_object_opts);
 bdrv_init();
 
 while ((c = getopt_long(argc, argv, sopt, lopt, _index)) != -1) {
@@ -450,6 +524,13 @@ int main(int argc, char **argv)
 case 'h':
 usage(progname);
 exit(0);
+case OPTION_OBJECT:
+qopts = qemu_opts_parse_noisily(qemu_find_opts("object"),
+optarg, true);
+if (!qopts) {
+exit(1);
+}
+break;
 default:
 usage(progname);
 exit(1);
@@ -466,6 +547,12 @@ int main(int argc, char **argv)
 exit(1);
 }
 
+if (qemu_opts_foreach(qemu_find_opts("object"),
+  object_create,
+  NULL, NULL)) {
+exit(1);
+}
+
 /* initialize commands */
 qemuio_add_command(_cmd);
 qemuio_add_command(_cmd);
-- 
2.4.3

[Qemu-block] [PATCH 01/17] crypto: add QCryptoSecret object class for password/key handling

[Daniel P. Berrange]

Introduce a new QCryptoSecret object class which will be used
for providing passwords and keys to other objects which need
sensitive credentials.

The new object can provide secret values directly as properties,
or indirectly via a file. The latter includes support for file
descriptor passing syntax on UNIX platforms. Ordinarily passing
secret values directly as properties is insecure, since they
are visible in process listings, or in log files showing the
CLI args / QMP commands. It is possible to use AES-256-CBC to
encrypt the secret values though, in which case all that is
visible is the ciphertext.  For adhoc developer testing though,
it is fine to provide the secrets directly without encryption
so this is not explicitly forbidden.

The anticipated scenario is that libvirtd will create a random
master key per QEMU instance (eg /var/run/libvirt/qemu/$VMNAME.key)
and will use that key to encrypt all passwords it provides to
QEMU via '-object secret,'.  This avoids the need for libvirt
(or other mgmt apps) to worry about file descriptor passing.

It also makes life easier for people who are scripting the
management of QEMU, for whom FD passing is significantly more
complex.

Providing data inline (insecure, only for adhoc dev tetsing)

  $QEMU -object secret,id=sec0,data=letmein

Providing data indirectly

  echo -n "letmein" > mypasswd.txt
  $QEMU -object secret,id=sec0,file=mypasswd.txt

Providing binary data

  $QEMU -object secret,id=sec0,file=mykey.bin,format=base64

Providing data with encyption

  $QEMU -object secret,id=master0,file=mykey.bin,format=base64 \
-object secret,id=sec0,data=[base64 ciphertext],\
   keyid=master0,iv=[base64 IV],format=utf8

Note that 'format' here refers to the format of the decrypted
data, which is independant of the ciphertext which must always
be in base64.

More examples are shown in the updated docs.

Signed-off-by: Daniel P. Berrange 
---
 crypto/Makefile.objs   |   1 +
 crypto/secret.c| 513 +
 include/crypto/secret.h| 139 
 qapi/crypto.json   |  14 ++
 qemu-options.hx|  76 +++
 tests/.gitignore   |   1 +
 tests/Makefile |   2 +
 tests/test-crypto-secret.c | 440 ++
 8 files changed, 1186 insertions(+)
 create mode 100644 crypto/secret.c
 create mode 100644 include/crypto/secret.h
 create mode 100644 tests/test-crypto-secret.c

diff --git a/crypto/Makefile.objs b/crypto/Makefile.objs
index b2a0e0b..a3135f1 100644
--- a/crypto/Makefile.objs
+++ b/crypto/Makefile.objs
@@ -7,6 +7,7 @@ crypto-obj-y += tlscreds.o
 crypto-obj-y += tlscredsanon.o
 crypto-obj-y += tlscredsx509.o
 crypto-obj-y += tlssession.o
+crypto-obj-y += secret.o
 
 # Let the userspace emulators avoid linking gnutls/etc
 crypto-aes-obj-y = aes.o
diff --git a/crypto/secret.c b/crypto/secret.c
new file mode 100644
index 000..e1ee4fa
--- /dev/null
+++ b/crypto/secret.c
@@ -0,0 +1,513 @@
+/*
+ * QEMU crypto secret support
+ *
+ * Copyright (c) 2015 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see .
+ *
+ */
+
+#include "crypto/secret.h"
+#include "crypto/cipher.h"
+#include "qom/object_interfaces.h"
+#include "trace.h"
+
+static void
+qcrypto_secret_prop_set_loaded(Object *obj,
+   bool value,
+   Error **errp)
+{
+QCryptoSecret *secret = QCRYPTO_SECRET(obj);
+int fd;
+gchar *data = NULL;
+size_t offset = 0;
+size_t length = 0;
+
+if (value) {
+if (secret->file) {
+if (secret->data) {
+error_setg(errp,
+   "'file' and 'data' are mutually exclusive");
+return;
+}
+fd = qemu_open(secret->file, O_RDONLY);
+if (fd < 0) {
+error_setg_errno(errp, errno,
+ "Unable to open %s", secret->file);
+return;
+}
+while (length < (1024 * 1024)) { /* Limit secrets to 1 MB */
+if ((length - offset) < 1024) {
+length += 1024;
+data = g_renew(gchar, data, length);
+}
+ssize_t ret = read(fd, data + offset, length - offset);
+

[Qemu-block] [PATCH v7 06/39] block: Add blk_is_available()

[Max Reitz]

blk_is_available() returns true iff the BDS is inserted (which means
blk_bs() is not NULL and bdrv_is_inserted() returns true) and if the
tray of the guest device is closed.

blk_is_inserted() is changed to return true only if blk_bs() is not
NULL.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Alberto Garcia 
Reviewed-by: Kevin Wolf 
---
 block/block-backend.c  | 7 ++-
 include/sysemu/block-backend.h | 1 +
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/block/block-backend.c b/block/block-backend.c
index 1db002c..74642dc 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -771,7 +771,12 @@ void blk_invalidate_cache(BlockBackend *blk, Error **errp)
 
 bool blk_is_inserted(BlockBackend *blk)
 {
-return bdrv_is_inserted(blk->bs);
+return blk->bs && bdrv_is_inserted(blk->bs);
+}
+
+bool blk_is_available(BlockBackend *blk)
+{
+return blk_is_inserted(blk) && !blk_dev_is_tray_open(blk);
 }
 
 void blk_lock_medium(BlockBackend *blk, bool locked)
diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h
index 8f2bf10..1e19d1b 100644
--- a/include/sysemu/block-backend.h
+++ b/include/sysemu/block-backend.h
@@ -131,6 +131,7 @@ int blk_enable_write_cache(BlockBackend *blk);
 void blk_set_enable_write_cache(BlockBackend *blk, bool wce);
 void blk_invalidate_cache(BlockBackend *blk, Error **errp);
 bool blk_is_inserted(BlockBackend *blk);
+bool blk_is_available(BlockBackend *blk);
 void blk_lock_medium(BlockBackend *blk, bool locked);
 void blk_eject(BlockBackend *blk, bool eject_flag);
 int blk_get_flags(BlockBackend *blk);
-- 
2.6.1

[Qemu-block] [PATCH v7 03/39] blockdev: Allow creation of BDS trees without BB

[Max Reitz]

If the "id" field is missing from the options given to blockdev-add,
just omit the BlockBackend and create the BlockDriverState tree alone.

However, if "id" is missing, "node-name" must be specified; otherwise,
the BDS tree would no longer be accessible.

Many BDS options which are not parsed by bdrv_open() (like caching)
cannot be specified for these BB-less BDS trees yet. A future patch will
remove this limitation.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Kevin Wolf 
Reviewed-by: Alberto Garcia 
---
 blockdev.c | 44 +++-
 qapi/block-core.json   | 13 +
 tests/qemu-iotests/087 |  2 +-
 tests/qemu-iotests/087.out |  4 ++--
 4 files changed, 43 insertions(+), 20 deletions(-)

diff --git a/blockdev.c b/blockdev.c
index 27398b1..0785557 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3026,17 +3026,12 @@ out:
 void qmp_blockdev_add(BlockdevOptions *options, Error **errp)
 {
 QmpOutputVisitor *ov = qmp_output_visitor_new();
-BlockBackend *blk;
+BlockDriverState *bs;
+BlockBackend *blk = NULL;
 QObject *obj;
 QDict *qdict;
 Error *local_err = NULL;
 
-/* Require an ID in the top level */
-if (!options->has_id) {
-error_setg(errp, "Block device needs an ID");
-goto fail;
-}
-
 /* TODO Sort it out in raw-posix and drive_new(): Reject aio=native with
  * cache.direct=false instead of silently switching to aio=threads, except
  * when called from drive_new().
@@ -3064,14 +3059,37 @@ void qmp_blockdev_add(BlockdevOptions *options, Error 
**errp)
 
 qdict_flatten(qdict);
 
-blk = blockdev_init(NULL, qdict, _err);
-if (local_err) {
-error_propagate(errp, local_err);
-goto fail;
+if (options->has_id) {
+blk = blockdev_init(NULL, qdict, _err);
+if (local_err) {
+error_propagate(errp, local_err);
+goto fail;
+}
+
+bs = blk_bs(blk);
+} else {
+int ret;
+
+if (!qdict_get_try_str(qdict, "node-name")) {
+error_setg(errp, "'id' and/or 'node-name' need to be specified for 
"
+   "the root node");
+goto fail;
+}
+
+bs = NULL;
+ret = bdrv_open(, NULL, NULL, qdict, BDRV_O_RDWR | BDRV_O_CACHE_WB,
+errp);
+if (ret < 0) {
+goto fail;
+}
 }
 
-if (bdrv_key_required(blk_bs(blk))) {
-blk_unref(blk);
+if (bs && bdrv_key_required(bs)) {
+if (blk) {
+blk_unref(blk);
+} else {
+bdrv_unref(bs);
+}
 error_setg(errp, "blockdev-add doesn't support encrypted devices");
 goto fail;
 }
diff --git a/qapi/block-core.json b/qapi/block-core.json
index c042561..425fdab 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1393,9 +1393,12 @@
 #
 # @driver:block driver name
 # @id:#optional id by which the new block device can be referred 
to.
-# This is a required option on the top level of blockdev-add, 
and
-# currently not allowed on any other level.
-# @node-name: #optional the name of a block driver state node (Since 2.0)
+# This option is only allowed on the top level of blockdev-add.
+# A BlockBackend will be created by blockdev-add if and only if
+# this option is given.
+# @node-name: #optional the name of a block driver state node (Since 2.0).
+# This option is required on the top level of blockdev-add if
+# the @id option is not given there.
 # @discard:   #optional discard-related options (default: ignore)
 # @cache: #optional cache-related options
 # @aio:   #optional AIO backend (default: threads)
@@ -1859,7 +1862,9 @@
 ##
 # @blockdev-add:
 #
-# Creates a new block device.
+# Creates a new block device. If the @id option is given at the top level, a
+# BlockBackend will be created; otherwise, @node-name is mandatory at the top
+# level and no BlockBackend will be created.
 #
 # This command is still a work in progress.  It doesn't support all
 # block drivers, it lacks a matching blockdev-del, and more.  Stay
diff --git a/tests/qemu-iotests/087 b/tests/qemu-iotests/087
index 8694749..af44299 100755
--- a/tests/qemu-iotests/087
+++ b/tests/qemu-iotests/087
@@ -54,7 +54,7 @@ size=128M
 _make_test_img $size
 
 echo
-echo === Missing ID ===
+echo === Missing ID and node-name ===
 echo
 
 run_qemu <

[Qemu-block] [PATCH v7 00/39] blockdev: BlockBackend and media

[Max Reitz]

This series reworks a lot regarding BlockBackend and media. Basically,
it allows empty BlockBackends, that is BBs without a BDS tree.

Before this series, empty drives are represented by a BlockBackend with
an empty BDS attached to it (a BDS with a NULL driver). However, now we
have BlockBackends, thus an empty drive should be represented by a
BlockBackend without any BDS tree attached to it. This is what this
series does.


Quick and early summary for the v7 changes:
- Rebased
- Addressed Kevin's comments for v6


Justification for each of the patches and their order:

-- Preparation before _is_inserted() patches --
 1: Patch 9 will not take care not to break host floppy support, so that
support needs to be removed first.
 2: Needed for patch 3, so that blockdev-added BDSs without a BB still
get the BDRV_O_INCOMING flag set.
 3: Needed for patch 4. Patch 26 is a follow-up after BDS-less BBs are
allowed.
 4: bdrv_close_all() is broken ("block: Rework bdrv_close_all()"). Patch
7 will break iotest 071 (actually, just make the problem apparent).
So this patch is required to work around the issue.
(with "the issue" being that bdrv_close_all() does not unref() the
BDSs it is closing, but just force-closes everything, even if the
BDS may still be in use somewhere)

-- _is_inserted() patches --
 5: General clean-up work, nice to have before patch 7 (and goes in tune
with patch 6).
 6: Using the same BB as a guest device means that the data read from
there should be exactly the same. Opening the guest tray should
therefore result in no data being readable. This is what we then
need this function for.
 7: General clean-up work (in the _is_inserted() area).
 8: General clean-up work (in the _is_inserted() area).
 9: General clean-up work (also regarding _is_inserted()).
10: Required so inserting a floppy will not result in the tray being
reported as closed (you need to "push in" the floppy first, using
blockdev-close-tray). It's here in the "_is_inserted() patches area"
because I feel like that's a very related topic.

-- Support for BDS-less BBs --
11: Preparation for BDS-less BBs
12: Preparation for BDS-less BBs
13: Preparation for BDS-less BBs (BB properties should be in the BB, and
not in the root BDS)
14: Patch 15 removes BlockAcctStats from the BDS, but wr_highest_sector
is BDS-dependent, so it needs to stay here
15: Preparation for BDS-less BBs (BB properties should be in the BB, and
not in the root BDS)
16: Preparation for BDS-less BBs (BB properties should be in the BB, and
not in the root BDS)
17: Needed for patch 18
18: Preparation for BDS-less BBs (Removing a BDS tree should retain some
properties for legacy reasons, which must therefore be stored in the
BB(RS))
19: Preparation for BDS-less BBs
20: Preparation for BDS-less BBs
21: Preparation for BDS-less BBs
22: Ability to add BDS trees to empty BBs ("inserting a medium")
23: Preparation for BDS-less BBs (needs patch 22)
24: One goal of this series, and fixes the "opening tray" event for
empty drives when shutting down qemu
25: Needed for patch 26
26: Completion of what patch 3 begun
27: Ability to detach BDS trees from BBs

-- "Atomic" QMP tray operations --
28: blockdev-open-tray
29: blockdev-close-tray
30: blockdev-remove-medium
31: blockdev-insert-medium

-- Reimplementation of change/eject --
32: eject
33: change
34: Clean-up patch

-- New QMP blockdev-change-medium command --
35: New QMP command
36: Use for HMP change command
37: Add flag to that command for changing the read-only access mode
(which was my original intention for this series)
38: Same flag for HMP

-- Tests --
39: iotests are always nice, so here is one


v7:
- Patch 4: Rebase conflict
- Patch 23: [Kevin]
  - Drop blk_is_available() check in drive_backup_prepare(); the same
check is done by qmp_drive_backup() already
  - eject_device(): If there is no BDS attached to the BB, return
immediately without doing anything
  - Simplify qmp_bdrv_open_encrypted()
  - Make last error path in qmp_change_blockdev() more explicit
  - Move the contents of two consecutive if (bs [&& ...]) blocks into a
single if (bs) block
- Patch 24: Drop the unnecessary and now actually wrong block which sets
  BDRV_O_INCOMING [Kevin]
- Patch 25:
  - Make parameters for extract_common_blockdev_options() optional; we
won't need the throttle options in bds_tree_init()
  - Reorder the parameters (throttling group and throttle config should
be together, because that makes sense)
  - Rebase conflict (CONFIG_LINUX_AIO dropped)
- Patch 26: Do not allow throttle configuration for BB-less BDS trees
  [Kevin]
- Patch 33: Rebase conflicts due to changes to patch 23


git-backport-diff against v6:

Key:
[] : patches are identical
[] : number of functional differences between upstream/downstream patch
[down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, 

[Qemu-block] [PATCH v7 04/39] iotests: Only create BB if necessary

[Max Reitz]

Tests 071 and 081 test giving references in blockdev-add. It is not
necessary to create a BlockBackend here, so omit it.

While at it, fix up some blockdev-add invocations in the vicinity
(s/raw/$IMGFMT/ in 081, drop the format BDS for blkverify's raw child in
071).

Signed-off-by: Max Reitz 
---
 tests/qemu-iotests/071 | 54 ++
 tests/qemu-iotests/071.out | 12 +++
 tests/qemu-iotests/081 | 18 +---
 tests/qemu-iotests/081.out |  5 +++--
 4 files changed, 71 insertions(+), 18 deletions(-)

diff --git a/tests/qemu-iotests/071 b/tests/qemu-iotests/071
index 9eaa49b..92ab991 100755
--- a/tests/qemu-iotests/071
+++ b/tests/qemu-iotests/071
@@ -104,11 +104,20 @@ echo
 echo "=== Testing blkdebug on existing block device ==="
 echo
 
-run_qemu -drive "file=$TEST_IMG,format=raw,if=none,id=drive0" <

[Qemu-block] [PATCH v7 08/39] block/raw_bsd: Drop raw_is_inserted()

[Max Reitz]

With the new automatically-recursive implementation of
bdrv_is_inserted() checking by default whether all the children of a BDS
are inserted, we can drop raw's own implementation.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Kevin Wolf 
Reviewed-by: Alberto Garcia 
---
 block/raw_bsd.c | 6 --
 1 file changed, 6 deletions(-)

diff --git a/block/raw_bsd.c b/block/raw_bsd.c
index 3c7b413..0aded31 100644
--- a/block/raw_bsd.c
+++ b/block/raw_bsd.c
@@ -154,11 +154,6 @@ static int raw_truncate(BlockDriverState *bs, int64_t 
offset)
 return bdrv_truncate(bs->file->bs, offset);
 }
 
-static bool raw_is_inserted(BlockDriverState *bs)
-{
-return bdrv_is_inserted(bs->file->bs);
-}
-
 static int raw_media_changed(BlockDriverState *bs)
 {
 return bdrv_media_changed(bs->file->bs);
@@ -264,7 +259,6 @@ BlockDriver bdrv_raw = {
 .bdrv_refresh_limits  = _refresh_limits,
 .bdrv_probe_blocksizes = _probe_blocksizes,
 .bdrv_probe_geometry  = _probe_geometry,
-.bdrv_is_inserted = _is_inserted,
 .bdrv_media_changed   = _media_changed,
 .bdrv_eject   = _eject,
 .bdrv_lock_medium = _lock_medium,
-- 
2.6.1

[Qemu-block] [PATCH 06/17] qcow: add a 'keyid' parameter to qcow options

[Daniel P. Berrange]

Add a 'keyid' parameter that refers to the ID of a
QCryptoSecret instance that provides the encryption key.
eg

 $QEMU \
-object secret,id=sec0,filename=/home/berrange/encrypted.pw \
-drive file=/home/berrange/encrypted.qcow,keyid=sec0

Signed-off-by: Daniel P. Berrange 
---
 block/qcow.c | 94 +++-
 qapi/block-core.json | 17 +-
 2 files changed, 87 insertions(+), 24 deletions(-)

diff --git a/block/qcow.c b/block/qcow.c
index 635085e..719ed7c 100644
--- a/block/qcow.c
+++ b/block/qcow.c
@@ -27,6 +27,7 @@
 #include 
 #include "qapi/qmp/qerror.h"
 #include "crypto/cipher.h"
+#include "crypto/secret.h"
 #include "migration/migration.h"
 
 /**/
@@ -40,6 +41,8 @@
 
 #define QCOW_OFLAG_COMPRESSED (1LL << 63)
 
+#define QCOW_OPT_KEY_ID "keyid"
+
 typedef struct QCowHeader {
 uint32_t magic;
 uint32_t version;
@@ -92,6 +95,43 @@ static int qcow_probe(const uint8_t *buf, int buf_size, 
const char *filename)
 return 0;
 }
 
+static QCryptoCipher *qcow_get_cipher_from_key(const char *key,
+   Error **errp)
+{
+uint8_t keybuf[16];
+int len, i;
+
+memset(keybuf, 0, 16);
+len = strlen(key);
+if (len > 16) {
+len = 16;
+}
+/* XXX: we could compress the chars to 7 bits to increase
+   entropy */
+for (i = 0; i < len; i++) {
+keybuf[i] = key[i];
+}
+
+return qcrypto_cipher_new(
+QCRYPTO_CIPHER_ALG_AES_128,
+QCRYPTO_CIPHER_MODE_CBC,
+keybuf, G_N_ELEMENTS(keybuf),
+errp);
+}
+
+static QemuOptsList qcow_runtime_opts = {
+.name = "qcow",
+.head = QTAILQ_HEAD_INITIALIZER(qcow_runtime_opts.head),
+.desc = {
+{
+.name = QCOW_OPT_KEY_ID,
+.type = QEMU_OPT_STRING,
+.help = "ID of the secret that provides the encryption key",
+},
+{ /* end of list */ }
+},
+};
+
 static int qcow_open(BlockDriverState *bs, QDict *options, int flags,
  Error **errp)
 {
@@ -99,6 +139,10 @@ static int qcow_open(BlockDriverState *bs, QDict *options, 
int flags,
 unsigned int len, i, shift;
 int ret;
 QCowHeader header;
+QemuOpts *opts = NULL;
+const char *keyid;
+char *key;
+Error *local_err = NULL;
 
 ret = bdrv_pread(bs->file->bs, 0, , sizeof(header));
 if (ret < 0) {
@@ -147,6 +191,32 @@ static int qcow_open(BlockDriverState *bs, QDict *options, 
int flags,
 goto fail;
 }
 
+opts = qemu_opts_create(_runtime_opts, NULL, 0, _abort);
+qemu_opts_absorb_qdict(opts, options, _err);
+if (local_err) {
+error_propagate(errp, local_err);
+ret = -EINVAL;
+goto fail;
+}
+
+keyid = qemu_opt_get(opts, QCOW_OPT_KEY_ID);
+if (keyid) {
+key = qcrypto_secret_lookup_as_utf8(keyid,
+errp);
+if (!key) {
+ret = -ENOENT;
+goto fail;
+}
+
+s->cipher = qcow_get_cipher_from_key(key,
+ errp);
+g_free(key);
+if (!s->cipher) {
+ret = -ENOSYS;
+goto fail;
+}
+}
+
 if (header.crypt_method > QCOW_CRYPT_AES) {
 error_setg(errp, "invalid encryption method in qcow header");
 ret = -EINVAL;
@@ -261,33 +331,11 @@ static int qcow_reopen_prepare(BDRVReopenState *state,
 static int qcow_set_key(BlockDriverState *bs, const char *key)
 {
 BDRVQcowState *s = bs->opaque;
-uint8_t keybuf[16];
-int len, i;
-Error *err;
 
-memset(keybuf, 0, 16);
-len = strlen(key);
-if (len > 16)
-len = 16;
-/* XXX: we could compress the chars to 7 bits to increase
-   entropy */
-for(i = 0;i < len;i++) {
-keybuf[i] = key[i];
-}
 assert(bs->encrypted);
-
 qcrypto_cipher_free(s->cipher);
-s->cipher = qcrypto_cipher_new(
-QCRYPTO_CIPHER_ALG_AES_128,
-QCRYPTO_CIPHER_MODE_CBC,
-keybuf, G_N_ELEMENTS(keybuf),
-);
-
+s->cipher = qcow_get_cipher_from_key(key, NULL);
 if (!s->cipher) {
-/* XXX would be nice if errors in this method could
- * be properly propagate to the caller. Would need
- * the bdrv_set_key() API signature to be fixed. */
-error_free(err);
 return -1;
 }
 return 0;
diff --git a/qapi/block-core.json b/qapi/block-core.json
index bb2189e..513fe93 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1562,6 +1562,21 @@
 'mode':  'Qcow2OverlapCheckMode' } }
 
 ##
+# @BlockdevOptionsQcow
+#
+# Driver specific block device options for qcow.
+#
+# @keyid: #optional ID of the "secret" object providing the
+# AES decryption key.
+#
+# Since: 2.5
+##
+{ 'struct': 'BlockdevOptionsQcow',
+  'base': 

[Qemu-block] [PATCH v7 28/39] blockdev: Add blockdev-open-tray

[Max Reitz]

Signed-off-by: Max Reitz 
---
 blockdev.c   | 49 +
 qapi/block-core.json | 23 +++
 qmp-commands.hx  | 39 +++
 3 files changed, 111 insertions(+)

diff --git a/blockdev.c b/blockdev.c
index 35e5719..aa68c36 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2049,6 +2049,55 @@ out:
 aio_context_release(aio_context);
 }
 
+void qmp_blockdev_open_tray(const char *device, bool has_force, bool force,
+Error **errp)
+{
+BlockBackend *blk;
+BlockDriverState *bs;
+AioContext *aio_context = NULL;
+
+if (!has_force) {
+force = false;
+}
+
+blk = blk_by_name(device);
+if (!blk) {
+error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
+  "Device '%s' not found", device);
+return;
+}
+
+if (!blk_dev_has_removable_media(blk)) {
+error_setg(errp, "Device '%s' is not removable", device);
+return;
+}
+
+if (blk_dev_is_tray_open(blk)) {
+return;
+}
+
+bs = blk_bs(blk);
+if (bs) {
+aio_context = bdrv_get_aio_context(bs);
+aio_context_acquire(aio_context);
+
+if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_EJECT, errp)) {
+goto out;
+}
+}
+
+if (blk_dev_is_medium_locked(blk)) {
+blk_dev_eject_request(blk, force);
+} else {
+blk_dev_change_media_cb(blk, false);
+}
+
+out:
+if (aio_context) {
+aio_context_release(aio_context);
+}
+}
+
 /* throttling disk I/O limits */
 void qmp_block_set_io_throttle(const char *device, int64_t bps, int64_t bps_rd,
int64_t bps_wr,
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 425fdab..b9b4a24 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1876,6 +1876,29 @@
 ##
 { 'command': 'blockdev-add', 'data': { 'options': 'BlockdevOptions' } }
 
+##
+# @blockdev-open-tray:
+#
+# Opens a block device's tray. If there is a block driver state tree inserted 
as
+# a medium, it will become inaccessible to the guest (but it will remain
+# associated to the block device, so closing the tray will make it accessible
+# again).
+#
+# If the tray was already open before, this will be a no-op.
+#
+# @device: block device name
+#
+# @force:  #optional if false (the default), an eject request will be sent to
+#  the guest if it has locked the tray (and the tray will not be opened
+#  immediately); if true, the tray will be opened regardless of whether
+#  it is locked
+#
+# Since: 2.5
+##
+{ 'command': 'blockdev-open-tray',
+  'data': { 'device': 'str',
+'*force': 'bool' } }
+
 
 ##
 # @BlockErrorAction
diff --git a/qmp-commands.hx b/qmp-commands.hx
index d7cf0ff..595aee2 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -3936,6 +3936,45 @@ Example (2):
 EQMP
 
 {
+.name   = "blockdev-open-tray",
+.args_type  = "device:s,force:b?",
+.mhandler.cmd_new = qmp_marshal_blockdev_open_tray,
+},
+
+SQMP
+blockdev-open-tray
+--
+
+Opens a block device's tray. If there is a block driver state tree inserted as 
a
+medium, it will become inaccessible to the guest (but it will remain associated
+to the block device, so closing the tray will make it accessible again).
+
+If the tray was already open before, this will be a no-op.
+
+Arguments:
+
+- "device": block device name (json-string)
+- "force": if false (the default), an eject request will be sent to the guest 
if
+   it has locked the tray (and the tray will not be opened 
immediately);
+   if true, the tray will be opened regardless of whether it is locked
+   (json-bool, optional)
+
+Example:
+
+-> { "execute": "blockdev-open-tray",
+ "arguments": { "device": "ide1-cd0" } }
+
+<- { "timestamp": { "seconds": 1418751016,
+"microseconds": 716996 },
+ "event": "DEVICE_TRAY_MOVED",
+ "data": { "device": "ide1-cd0",
+   "tray-open": true } }
+
+<- { "return": {} }
+
+EQMP
+
+{
 .name   = "query-named-block-nodes",
 .args_type  = "",
 .mhandler.cmd_new = qmp_marshal_query_named_block_nodes,
-- 
2.6.1

[Qemu-block] [PATCH v7 11/39] hw/usb-storage: Check whether BB is inserted

[Max Reitz]

Only call bdrv_add_key() on the BlockDriverState if it is not NULL.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Alberto Garcia 
Reviewed-by: Kevin Wolf 
---
 hw/usb/dev-storage.c | 30 --
 1 file changed, 16 insertions(+), 14 deletions(-)

diff --git a/hw/usb/dev-storage.c b/hw/usb/dev-storage.c
index 9a4e7dc..597d8fd 100644
--- a/hw/usb/dev-storage.c
+++ b/hw/usb/dev-storage.c
@@ -613,20 +613,22 @@ static void usb_msd_realize_storage(USBDevice *dev, Error 
**errp)
 return;
 }
 
-bdrv_add_key(blk_bs(blk), NULL, );
-if (err) {
-if (monitor_cur_is_qmp()) {
-error_propagate(errp, err);
-return;
-}
-error_free(err);
-err = NULL;
-if (cur_mon) {
-monitor_read_bdrv_key_start(cur_mon, blk_bs(blk),
-usb_msd_password_cb, s);
-s->dev.auto_attach = 0;
-} else {
-autostart = 0;
+if (blk_bs(blk)) {
+bdrv_add_key(blk_bs(blk), NULL, );
+if (err) {
+if (monitor_cur_is_qmp()) {
+error_propagate(errp, err);
+return;
+}
+error_free(err);
+err = NULL;
+if (cur_mon) {
+monitor_read_bdrv_key_start(cur_mon, blk_bs(blk),
+usb_msd_password_cb, s);
+s->dev.auto_attach = 0;
+} else {
+autostart = 0;
+}
 }
 }
 
-- 
2.6.1

[Qemu-block] [PATCH v7 14/39] block: Remove wr_highest_sector from BlockAcctStats

[Max Reitz]

BlockAcctStats contains statistics about the data transferred from and
to the device; wr_highest_sector does not fit in with the rest.

Furthermore, those statistics are supposed to be specific for a certain
device and not necessarily for a BDS (see the comment above
bdrv_get_stats()); on the other hand, wr_highest_sector may be a rather
important information to know for each BDS. When BlockAcctStats is
finally removed from the BDS, we will want to keep wr_highest_sector in
the BDS.

Finally, wr_highest_sector is renamed to wr_highest_offset and given the
appropriate meaning. Externally, it is represented as an offset so there
is no point in doing something different internally. Its definition is
changed to match that in qapi/block-core.json which is "the offset after
the greatest byte written to". Doing so should not cause any harm since
if external programs tried to calculate the volume usage by
(wr_highest_offset + 512) / volume_size, after this patch they will just
assume the volume to be full slightly earlier than before.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Alberto Garcia 
Reviewed-by: Kevin Wolf 
---
 block/accounting.c | 8 
 block/io.c | 4 +++-
 block/qapi.c   | 4 ++--
 include/block/accounting.h | 3 ---
 include/block/block_int.h  | 3 +++
 qmp-commands.hx| 4 ++--
 6 files changed, 10 insertions(+), 16 deletions(-)

diff --git a/block/accounting.c b/block/accounting.c
index 01d594f..a423560 100644
--- a/block/accounting.c
+++ b/block/accounting.c
@@ -47,14 +47,6 @@ void block_acct_done(BlockAcctStats *stats, BlockAcctCookie 
*cookie)
 }
 
 
-void block_acct_highest_sector(BlockAcctStats *stats, int64_t sector_num,
-   unsigned int nb_sectors)
-{
-if (stats->wr_highest_sector < sector_num + nb_sectors - 1) {
-stats->wr_highest_sector = sector_num + nb_sectors - 1;
-}
-}
-
 void block_acct_merge_done(BlockAcctStats *stats, enum BlockAcctType type,
   int num_requests)
 {
diff --git a/block/io.c b/block/io.c
index 5311473..b80044b 100644
--- a/block/io.c
+++ b/block/io.c
@@ -1151,7 +1151,9 @@ static int coroutine_fn 
bdrv_aligned_pwritev(BlockDriverState *bs,
 
 bdrv_set_dirty(bs, sector_num, nb_sectors);
 
-block_acct_highest_sector(>stats, sector_num, nb_sectors);
+if (bs->wr_highest_offset < offset + bytes) {
+bs->wr_highest_offset = offset + bytes;
+}
 
 if (ret >= 0) {
 bs->total_sectors = MAX(bs->total_sectors, sector_num + nb_sectors);
diff --git a/block/qapi.c b/block/qapi.c
index 355ba32..0360126 100644
--- a/block/qapi.c
+++ b/block/qapi.c
@@ -350,13 +350,13 @@ static BlockStats *bdrv_query_stats(const 
BlockDriverState *bs,
 s->stats->wr_operations = bs->stats.nr_ops[BLOCK_ACCT_WRITE];
 s->stats->rd_merged = bs->stats.merged[BLOCK_ACCT_READ];
 s->stats->wr_merged = bs->stats.merged[BLOCK_ACCT_WRITE];
-s->stats->wr_highest_offset =
-bs->stats.wr_highest_sector * BDRV_SECTOR_SIZE;
 s->stats->flush_operations = bs->stats.nr_ops[BLOCK_ACCT_FLUSH];
 s->stats->wr_total_time_ns = bs->stats.total_time_ns[BLOCK_ACCT_WRITE];
 s->stats->rd_total_time_ns = bs->stats.total_time_ns[BLOCK_ACCT_READ];
 s->stats->flush_total_time_ns = bs->stats.total_time_ns[BLOCK_ACCT_FLUSH];
 
+s->stats->wr_highest_offset = bs->wr_highest_offset;
+
 if (bs->file) {
 s->has_parent = true;
 s->parent = bdrv_query_stats(bs->file->bs, query_backing);
diff --git a/include/block/accounting.h b/include/block/accounting.h
index 4c406cf..66637cd 100644
--- a/include/block/accounting.h
+++ b/include/block/accounting.h
@@ -40,7 +40,6 @@ typedef struct BlockAcctStats {
 uint64_t nr_ops[BLOCK_MAX_IOTYPE];
 uint64_t total_time_ns[BLOCK_MAX_IOTYPE];
 uint64_t merged[BLOCK_MAX_IOTYPE];
-uint64_t wr_highest_sector;
 } BlockAcctStats;
 
 typedef struct BlockAcctCookie {
@@ -52,8 +51,6 @@ typedef struct BlockAcctCookie {
 void block_acct_start(BlockAcctStats *stats, BlockAcctCookie *cookie,
   int64_t bytes, enum BlockAcctType type);
 void block_acct_done(BlockAcctStats *stats, BlockAcctCookie *cookie);
-void block_acct_highest_sector(BlockAcctStats *stats, int64_t sector_num,
-   unsigned int nb_sectors);
 void block_acct_merge_done(BlockAcctStats *stats, enum BlockAcctType type,
int num_requests);
 
diff --git a/include/block/block_int.h b/include/block/block_int.h
index e79d8c0..b8e1c59 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -402,6 +402,9 @@ struct BlockDriverState {
 /* I/O stats (display with "info blockstats"). */
 BlockAcctStats stats;
 
+/* Offset after the highest byte written to */
+uint64_t wr_highest_offset;
+
 /* I/O Limits */
 BlockLimits bl;
 
diff --git 

[Qemu-block] [PATCH v7 09/39] block: Invoke change media CB before NULLing drv

[Max Reitz]

In order to handle host device passthrough, some guest device models
may call blk_is_inserted() to check whether the medium is inserted on
the host, when checking the guest tray status.

This tray status is inquired by blk_dev_change_media_cb(); because
bdrv_is_inserted() (invoked by blk_is_inserted()) always returns false
for BDS with drv set to NULL, blk_dev_change_media_cb() should therefore
be called before drv is set to NULL.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Alberto Garcia 
Reviewed-by: Kevin Wolf 
---
 block.c | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/block.c b/block.c
index 0f23a98..59a2a62 100644
--- a/block.c
+++ b/block.c
@@ -1912,6 +1912,10 @@ void bdrv_close(BlockDriverState *bs)
 bdrv_drain(bs); /* in case flush left pending I/O */
 notifier_list_notify(>close_notifiers, bs);
 
+if (bs->blk) {
+blk_dev_change_media_cb(bs->blk, false);
+}
+
 if (bs->drv) {
 BdrvChild *child, *next;
 
@@ -1950,10 +1954,6 @@ void bdrv_close(BlockDriverState *bs)
 bs->full_open_options = NULL;
 }
 
-if (bs->blk) {
-blk_dev_change_media_cb(bs->blk, false);
-}
-
 QLIST_FOREACH_SAFE(ban, >aio_notifiers, list, ban_next) {
 g_free(ban);
 }
-- 
2.6.1

[Qemu-block] [PATCH v7 15/39] block: Move BlockAcctStats into BlockBackend

[Max Reitz]

As the comment above bdrv_get_stats() says, BlockAcctStats is something
which belongs to the device instead of each BlockDriverState. This patch
therefore moves it into the BlockBackend.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Alberto Garcia 
Reviewed-by: Kevin Wolf 
---
 block.c   | 11 ---
 block/block-backend.c |  5 -
 block/io.c|  6 +-
 block/qapi.c  | 24 ++--
 include/block/block.h |  2 --
 include/block/block_int.h |  3 ---
 6 files changed, 23 insertions(+), 28 deletions(-)

diff --git a/block.c b/block.c
index 2f69ec7..40be84b 100644
--- a/block.c
+++ b/block.c
@@ -4153,14 +4153,3 @@ void bdrv_refresh_filename(BlockDriverState *bs)
 QDECREF(json);
 }
 }
-
-/* This accessor function purpose is to allow the device models to access the
- * BlockAcctStats structure embedded inside a BlockDriverState without being
- * aware of the BlockDriverState structure layout.
- * It will go away when the BlockAcctStats structure will be moved inside
- * the device models.
- */
-BlockAcctStats *bdrv_get_stats(BlockDriverState *bs)
-{
-return >stats;
-}
diff --git a/block/block-backend.c b/block/block-backend.c
index 7bc2eb1..a52037b 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -34,6 +34,9 @@ struct BlockBackend {
 
 /* the block size for which the guest device expects atomicity */
 int guest_block_size;
+
+/* I/O stats (display with "info blockstats"). */
+BlockAcctStats stats;
 };
 
 typedef struct BlockBackendAIOCB {
@@ -892,7 +895,7 @@ void blk_io_unplug(BlockBackend *blk)
 
 BlockAcctStats *blk_get_stats(BlockBackend *blk)
 {
-return bdrv_get_stats(blk->bs);
+return >stats;
 }
 
 void *blk_aio_get(const AIOCBInfo *aiocb_info, BlockBackend *blk,
diff --git a/block/io.c b/block/io.c
index b80044b..2fd7a1d 100644
--- a/block/io.c
+++ b/block/io.c
@@ -23,6 +23,7 @@
  */
 
 #include "trace.h"
+#include "sysemu/block-backend.h"
 #include "block/blockjob.h"
 #include "block/block_int.h"
 #include "block/throttle-groups.h"
@@ -1905,7 +1906,10 @@ static int multiwrite_merge(BlockDriverState *bs, 
BlockRequest *reqs,
 }
 }
 
-block_acct_merge_done(>stats, BLOCK_ACCT_WRITE, num_reqs - outidx - 1);
+if (bs->blk) {
+block_acct_merge_done(blk_get_stats(bs->blk), BLOCK_ACCT_WRITE,
+  num_reqs - outidx - 1);
+}
 
 return outidx + 1;
 }
diff --git a/block/qapi.c b/block/qapi.c
index 0360126..7c8209b 100644
--- a/block/qapi.c
+++ b/block/qapi.c
@@ -344,16 +344,20 @@ static BlockStats *bdrv_query_stats(const 
BlockDriverState *bs,
 }
 
 s->stats = g_malloc0(sizeof(*s->stats));
-s->stats->rd_bytes = bs->stats.nr_bytes[BLOCK_ACCT_READ];
-s->stats->wr_bytes = bs->stats.nr_bytes[BLOCK_ACCT_WRITE];
-s->stats->rd_operations = bs->stats.nr_ops[BLOCK_ACCT_READ];
-s->stats->wr_operations = bs->stats.nr_ops[BLOCK_ACCT_WRITE];
-s->stats->rd_merged = bs->stats.merged[BLOCK_ACCT_READ];
-s->stats->wr_merged = bs->stats.merged[BLOCK_ACCT_WRITE];
-s->stats->flush_operations = bs->stats.nr_ops[BLOCK_ACCT_FLUSH];
-s->stats->wr_total_time_ns = bs->stats.total_time_ns[BLOCK_ACCT_WRITE];
-s->stats->rd_total_time_ns = bs->stats.total_time_ns[BLOCK_ACCT_READ];
-s->stats->flush_total_time_ns = bs->stats.total_time_ns[BLOCK_ACCT_FLUSH];
+if (bs->blk) {
+BlockAcctStats *stats = blk_get_stats(bs->blk);
+
+s->stats->rd_bytes = stats->nr_bytes[BLOCK_ACCT_READ];
+s->stats->wr_bytes = stats->nr_bytes[BLOCK_ACCT_WRITE];
+s->stats->rd_operations = stats->nr_ops[BLOCK_ACCT_READ];
+s->stats->wr_operations = stats->nr_ops[BLOCK_ACCT_WRITE];
+s->stats->rd_merged = stats->merged[BLOCK_ACCT_READ];
+s->stats->wr_merged = stats->merged[BLOCK_ACCT_WRITE];
+s->stats->flush_operations = stats->nr_ops[BLOCK_ACCT_FLUSH];
+s->stats->wr_total_time_ns = stats->total_time_ns[BLOCK_ACCT_WRITE];
+s->stats->rd_total_time_ns = stats->total_time_ns[BLOCK_ACCT_READ];
+s->stats->flush_total_time_ns = stats->total_time_ns[BLOCK_ACCT_FLUSH];
+}
 
 s->stats->wr_highest_offset = bs->wr_highest_offset;
 
diff --git a/include/block/block.h b/include/block/block.h
index 0f40d9f..f1f6300 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -621,6 +621,4 @@ void bdrv_io_plug(BlockDriverState *bs);
 void bdrv_io_unplug(BlockDriverState *bs);
 void bdrv_flush_io_queue(BlockDriverState *bs);
 
-BlockAcctStats *bdrv_get_stats(BlockDriverState *bs);
-
 #endif
diff --git a/include/block/block_int.h b/include/block/block_int.h
index b8e1c59..f9c7ec5 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -399,9 +399,6 @@ struct BlockDriverState {
 unsigned   pending_reqs[2];
 QLIST_ENTRY(BlockDriverState) 

[Qemu-block] [PATCH v7 16/39] block: Move I/O status and error actions into BB

[Max Reitz]

These options are only relevant for the user of a whole BDS tree (like a
guest device or a block job) and should thus be moved into the
BlockBackend.

Signed-off-by: Max Reitz 
---
 block.c| 125 -
 block/backup.c |  17 --
 block/block-backend.c  | 116 --
 block/commit.c |   3 +-
 block/mirror.c |  17 --
 block/qapi.c   |   4 +-
 block/stream.c |   3 +-
 blockdev.c |   6 +-
 blockjob.c |   5 +-
 include/block/block.h  |  11 
 include/block/block_int.h  |   6 --
 include/sysemu/block-backend.h |   7 +++
 qmp.c  |   6 +-
 13 files changed, 158 insertions(+), 168 deletions(-)

diff --git a/block.c b/block.c
index 40be84b..6f239a4 100644
--- a/block.c
+++ b/block.c
@@ -257,7 +257,6 @@ BlockDriverState *bdrv_new(void)
 for (i = 0; i < BLOCK_OP_TYPE_MAX; i++) {
 QLIST_INIT(>op_blockers[i]);
 }
-bdrv_iostatus_disable(bs);
 notifier_list_init(>close_notifiers);
 notifier_with_return_list_init(>before_write_notifiers);
 qemu_co_queue_init(>throttled_reqs[0]);
@@ -2005,14 +2004,6 @@ static void bdrv_move_feature_fields(BlockDriverState 
*bs_dest,
 
 bs_dest->enable_write_cache = bs_src->enable_write_cache;
 
-/* r/w error */
-bs_dest->on_read_error  = bs_src->on_read_error;
-bs_dest->on_write_error = bs_src->on_write_error;
-
-/* i/o status */
-bs_dest->iostatus_enabled   = bs_src->iostatus_enabled;
-bs_dest->iostatus   = bs_src->iostatus;
-
 /* dirty bitmap */
 bs_dest->dirty_bitmaps  = bs_src->dirty_bitmaps;
 }
@@ -2499,82 +2490,6 @@ void bdrv_get_geometry(BlockDriverState *bs, uint64_t 
*nb_sectors_ptr)
 *nb_sectors_ptr = nb_sectors < 0 ? 0 : nb_sectors;
 }
 
-void bdrv_set_on_error(BlockDriverState *bs, BlockdevOnError on_read_error,
-   BlockdevOnError on_write_error)
-{
-bs->on_read_error = on_read_error;
-bs->on_write_error = on_write_error;
-}
-
-BlockdevOnError bdrv_get_on_error(BlockDriverState *bs, bool is_read)
-{
-return is_read ? bs->on_read_error : bs->on_write_error;
-}
-
-BlockErrorAction bdrv_get_error_action(BlockDriverState *bs, bool is_read, int 
error)
-{
-BlockdevOnError on_err = is_read ? bs->on_read_error : bs->on_write_error;
-
-switch (on_err) {
-case BLOCKDEV_ON_ERROR_ENOSPC:
-return (error == ENOSPC) ?
-   BLOCK_ERROR_ACTION_STOP : BLOCK_ERROR_ACTION_REPORT;
-case BLOCKDEV_ON_ERROR_STOP:
-return BLOCK_ERROR_ACTION_STOP;
-case BLOCKDEV_ON_ERROR_REPORT:
-return BLOCK_ERROR_ACTION_REPORT;
-case BLOCKDEV_ON_ERROR_IGNORE:
-return BLOCK_ERROR_ACTION_IGNORE;
-default:
-abort();
-}
-}
-
-static void send_qmp_error_event(BlockDriverState *bs,
- BlockErrorAction action,
- bool is_read, int error)
-{
-IoOperationType optype;
-
-optype = is_read ? IO_OPERATION_TYPE_READ : IO_OPERATION_TYPE_WRITE;
-qapi_event_send_block_io_error(bdrv_get_device_name(bs), optype, action,
-   bdrv_iostatus_is_enabled(bs),
-   error == ENOSPC, strerror(error),
-   _abort);
-}
-
-/* This is done by device models because, while the block layer knows
- * about the error, it does not know whether an operation comes from
- * the device or the block layer (from a job, for example).
- */
-void bdrv_error_action(BlockDriverState *bs, BlockErrorAction action,
-   bool is_read, int error)
-{
-assert(error >= 0);
-
-if (action == BLOCK_ERROR_ACTION_STOP) {
-/* First set the iostatus, so that "info block" returns an iostatus
- * that matches the events raised so far (an additional error iostatus
- * is fine, but not a lost one).
- */
-bdrv_iostatus_set_err(bs, error);
-
-/* Then raise the request to stop the VM and the event.
- * qemu_system_vmstop_request_prepare has two effects.  First,
- * it ensures that the STOP event always comes after the
- * BLOCK_IO_ERROR event.  Second, it ensures that even if management
- * can observe the STOP event and do a "cont" before the STOP
- * event is issued, the VM will not stop.  In this case, vm_start()
- * also ensures that the STOP/RESUME pair of events is emitted.
- */
-qemu_system_vmstop_request_prepare();
-send_qmp_error_event(bs, action, is_read, error);
-qemu_system_vmstop_request(RUN_STATE_IO_ERROR);
-} else {
-send_qmp_error_event(bs, action, is_read, error);
-}
-}
-
 int bdrv_is_read_only(BlockDriverState *bs)
 {
 return bs->read_only;
@@ -3602,46 

[Qemu-block] [PATCH 14/17] qemu-img: allow specifying image as a set of options args

[Daniel P. Berrange]

Currently qemu-img allows an image filename to be passed on the
command line, but does not have a way to set any options except
the format eg

   qemu-img info https://127.0.0.1/images/centos7.iso

This adds a --source arg (that is mutually exclusive with a
positional filename arg and -f arg) that accepts a full option
string, as well as the original syntax eg

   qemu-img info driver=http,url=https://127.0.0.1/images,sslverify=off

Signed-off-by: Daniel P. Berrange 
---
 include/qemu/option.h |   1 +
 qemu-img.c| 474 ++
 util/qemu-option.c|   6 +
 3 files changed, 407 insertions(+), 74 deletions(-)

diff --git a/include/qemu/option.h b/include/qemu/option.h
index 71f5f27..caf5a3b 100644
--- a/include/qemu/option.h
+++ b/include/qemu/option.h
@@ -104,6 +104,7 @@ int qemu_opt_foreach(QemuOpts *opts, qemu_opt_loopfunc 
func, void *opaque,
  Error **errp);
 
 QemuOpts *qemu_opts_find(QemuOptsList *list, const char *id);
+QemuOpts *qemu_opts_next(QemuOpts *opts);
 QemuOpts *qemu_opts_create(QemuOptsList *list, const char *id,
int fail_if_exists, Error **errp);
 void qemu_opts_reset(QemuOptsList *list);
diff --git a/qemu-img.c b/qemu-img.c
index 1578e0b..30869fe 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -38,6 +38,7 @@
 #include "block/blockjob.h"
 #include "block/qapi.h"
 #include 
+#include 
 
 #define QEMU_IMG_VERSION "qemu-img version " QEMU_VERSION QEMU_PKGVERSION \
   ", Copyright (c) 2004-2008 Fabrice Bellard\n"
@@ -51,6 +52,8 @@ enum {
 OPTION_OUTPUT = 256,
 OPTION_BACKING_CHAIN = 257,
 OPTION_OBJECT = 258,
+OPTION_SOURCE = 259,
+OPTION_TARGET = 260,
 };
 
 typedef enum OutputFormat {
@@ -172,6 +175,16 @@ static QemuOptsList qemu_object_opts = {
 },
 };
 
+static QemuOptsList qemu_source_opts = {
+.name = "source",
+.implied_opt_name = "file",
+.head = QTAILQ_HEAD_INITIALIZER(qemu_source_opts.head),
+.desc = {
+{ }
+},
+};
+
+
 static int object_create(void *opaque, QemuOpts *opts, Error **errp)
 {
 Error *err = NULL;
@@ -266,9 +279,31 @@ static int print_block_option_help(const char *filename, 
const char *fmt)
 return 0;
 }
 
-static BlockBackend *img_open(const char *id, const char *filename,
-  const char *fmt, int flags,
-  bool require_io, bool quiet)
+static BlockBackend *img_open_opts(const char *id,
+   QemuOpts *opts, int flags)
+{
+QDict *options;
+Error *local_err = NULL;
+char *file = NULL;
+BlockBackend *blk;
+file = g_strdup(qemu_opt_get(opts, "file"));
+qemu_opt_unset(opts, "file");
+options = qemu_opts_to_qdict(opts, NULL);
+blk = blk_new_open(id, file, NULL, options, flags, _err);
+if (!blk) {
+error_report("Could not open '%s': %s", file ? file : "",
+ error_get_pretty(local_err));
+g_free(file);
+error_free(local_err);
+return NULL;
+}
+g_free(file);
+return blk;
+}
+
+static BlockBackend *img_open_file(const char *id, const char *filename,
+   const char *fmt, int flags,
+   bool require_io, bool quiet)
 {
 BlockBackend *blk;
 BlockDriverState *bs;
@@ -576,7 +611,7 @@ static int img_check(int argc, char **argv)
 {
 int c, ret;
 OutputFormat output_format = OFORMAT_HUMAN;
-const char *filename, *fmt, *output, *cache;
+const char *filename = NULL, *fmt, *output, *cache;
 BlockBackend *blk;
 BlockDriverState *bs;
 int fix = 0;
@@ -596,6 +631,7 @@ static int img_check(int argc, char **argv)
 {"repair", required_argument, 0, 'r'},
 {"output", required_argument, 0, OPTION_OUTPUT},
 {"object", required_argument, 0, OPTION_OBJECT},
+{"source", required_argument, 0, OPTION_SOURCE},
 {0, 0, 0, 0}
 };
 c = getopt_long(argc, argv, "hf:r:T:q",
@@ -639,12 +675,29 @@ static int img_check(int argc, char **argv)
 exit(1);
 }
 break;
+case OPTION_SOURCE:
+if (filename) {
+errx(EXIT_FAILURE, "--source can only be specified once");
+}
+filename = optarg;
+opts = qemu_opts_parse_noisily(qemu_find_opts("source"),
+   optarg, true);
+if (!opts) {
+exit(1);
+}
+break;
 }
 }
-if (optind != argc - 1) {
-error_exit("Expecting one image file name");
+if (filename) {
+if (optind != argc) {
+error_exit("--source and filename are mutually exclusive");
+}
+} else {
+if (optind != argc - 1) {
+error_exit("Expecting one image file name");
+}
+

[Qemu-block] [PATCH v7 19/39] block: Make some BB functions fall back to BBRS

[Max Reitz]

If there is no BDS tree attached to a BlockBackend, functions that can
do so should fall back to the BlockBackendRootState structure.

Signed-off-by: Max Reitz 
Reviewed-by: Kevin Wolf 
---
 block/block-backend.c | 28 
 1 file changed, 24 insertions(+), 4 deletions(-)

diff --git a/block/block-backend.c b/block/block-backend.c
index 6a3f0c7..d790870 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -872,7 +872,11 @@ void blk_error_action(BlockBackend *blk, BlockErrorAction 
action,
 
 int blk_is_read_only(BlockBackend *blk)
 {
-return bdrv_is_read_only(blk->bs);
+if (blk->bs) {
+return bdrv_is_read_only(blk->bs);
+} else {
+return blk->root_state.read_only;
+}
 }
 
 int blk_is_sg(BlockBackend *blk)
@@ -882,12 +886,24 @@ int blk_is_sg(BlockBackend *blk)
 
 int blk_enable_write_cache(BlockBackend *blk)
 {
-return bdrv_enable_write_cache(blk->bs);
+if (blk->bs) {
+return bdrv_enable_write_cache(blk->bs);
+} else {
+return !!(blk->root_state.open_flags & BDRV_O_CACHE_WB);
+}
 }
 
 void blk_set_enable_write_cache(BlockBackend *blk, bool wce)
 {
-bdrv_set_enable_write_cache(blk->bs, wce);
+if (blk->bs) {
+bdrv_set_enable_write_cache(blk->bs, wce);
+} else {
+if (wce) {
+blk->root_state.open_flags |= BDRV_O_CACHE_WB;
+} else {
+blk->root_state.open_flags &= ~BDRV_O_CACHE_WB;
+}
+}
 }
 
 void blk_invalidate_cache(BlockBackend *blk, Error **errp)
@@ -917,7 +933,11 @@ void blk_eject(BlockBackend *blk, bool eject_flag)
 
 int blk_get_flags(BlockBackend *blk)
 {
-return bdrv_get_flags(blk->bs);
+if (blk->bs) {
+return bdrv_get_flags(blk->bs);
+} else {
+return blk->root_state.open_flags;
+}
 }
 
 int blk_get_max_transfer_length(BlockBackend *blk)
-- 
2.6.1

[Qemu-block] [PATCH v7 12/39] block: Fix BB AIOCB AioContext without BDS

[Max Reitz]

Fix the BlockBackend's AIOCB AioContext for aborting AIO in case there
is no BDS. If there is no implementation of AIOCBInfo::get_aio_context()
the AioContext is derived from the BDS the AIOCB belongs to. If that BDS
is NULL (because it has been removed from the BB) this will not work.

This patch makes blk_get_aio_context() fall back to the main loop
context if the BDS pointer is NULL and implements
AIOCBInfo::get_aio_context() (blk_aiocb_get_aio_context()) which invokes
blk_get_aio_context().

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Alberto Garcia 
Reviewed-by: Kevin Wolf 
---
 block/block-backend.c | 17 -
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/block/block-backend.c b/block/block-backend.c
index 74642dc..c7e0f7b 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -18,6 +18,8 @@
 /* Number of coroutines to reserve per attached device model */
 #define COROUTINE_POOL_RESERVATION 64
 
+static AioContext *blk_aiocb_get_aio_context(BlockAIOCB *acb);
+
 struct BlockBackend {
 char *name;
 int refcnt;
@@ -34,10 +36,12 @@ struct BlockBackend {
 typedef struct BlockBackendAIOCB {
 BlockAIOCB common;
 QEMUBH *bh;
+BlockBackend *blk;
 int ret;
 } BlockBackendAIOCB;
 
 static const AIOCBInfo block_backend_aiocb_info = {
+.get_aio_context = blk_aiocb_get_aio_context,
 .aiocb_size = sizeof(BlockBackendAIOCB),
 };
 
@@ -558,6 +562,7 @@ static BlockAIOCB *abort_aio_request(BlockBackend *blk, 
BlockCompletionFunc *cb,
 QEMUBH *bh;
 
 acb = blk_aio_get(_backend_aiocb_info, blk, cb, opaque);
+acb->blk = blk;
 acb->ret = ret;
 
 bh = aio_bh_new(blk_get_aio_context(blk), error_callback_bh, acb);
@@ -831,7 +836,17 @@ void blk_op_unblock_all(BlockBackend *blk, Error *reason)
 
 AioContext *blk_get_aio_context(BlockBackend *blk)
 {
-return bdrv_get_aio_context(blk->bs);
+if (blk->bs) {
+return bdrv_get_aio_context(blk->bs);
+} else {
+return qemu_get_aio_context();
+}
+}
+
+static AioContext *blk_aiocb_get_aio_context(BlockAIOCB *acb)
+{
+BlockBackendAIOCB *blk_acb = DO_UPCAST(BlockBackendAIOCB, common, acb);
+return blk_get_aio_context(blk_acb->blk);
 }
 
 void blk_set_aio_context(BlockBackend *blk, AioContext *new_context)
-- 
2.6.1

[Qemu-block] [PATCH v7 34/39] block: Inquire tray state before tray-moved events

[Max Reitz]

blk_dev_change_media_cb() is called for all potential tray movements;
however, it is possible to request closing the tray but nothing actually
happening (on a floppy disk drive without a medium).

Thus, the actual tray status should be inquired before sending a
tray-moved event (and an event should be sent whenever the status
changed).

Checking @load is now superfluous; it was necessary because it was
possible to change a medium without having explicitly opened the tray
and closed it again (or it might have been possible, at least). This is
no longer possible, though.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
---
 block/block-backend.c | 17 +++--
 1 file changed, 7 insertions(+), 10 deletions(-)

diff --git a/block/block-backend.c b/block/block-backend.c
index eb7409c..10e4d71 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -429,18 +429,15 @@ void blk_set_dev_ops(BlockBackend *blk, const BlockDevOps 
*ops,
 void blk_dev_change_media_cb(BlockBackend *blk, bool load)
 {
 if (blk->dev_ops && blk->dev_ops->change_media_cb) {
-bool tray_was_closed = !blk_dev_is_tray_open(blk);
+bool tray_was_open, tray_is_open;
 
+tray_was_open = blk_dev_is_tray_open(blk);
 blk->dev_ops->change_media_cb(blk->dev_opaque, load);
-if (tray_was_closed) {
-/* tray open */
-qapi_event_send_device_tray_moved(blk_name(blk),
-  true, _abort);
-}
-if (load) {
-/* tray close */
-qapi_event_send_device_tray_moved(blk_name(blk),
-  false, _abort);
+tray_is_open = blk_dev_is_tray_open(blk);
+
+if (tray_was_open != tray_is_open) {
+qapi_event_send_device_tray_moved(blk_name(blk), tray_is_open,
+  _abort);
 }
 }
 }
-- 
2.6.1

Re: [Qemu-block] [PATCH 01/17] crypto: add QCryptoSecret object class for password/key handling

[Paolo Bonzini]

On 19/10/2015 17:24, Daniel P. Berrange wrote:
> JSON doesn't accept arbitrary 8-bit binary data, so the alternative
> 'base64' is effectively providing binary data facility. Having to
> use base64 for plain passwords is rather tedious though, so allowing
> utf8 is a much more developer friendly approach for people using QEMU
> without a mgmt tool like libvirt.
> 
> NB, this dual-format utf8-or-base64 approach matches the approach used
> in QEMU guest agent for the guest-file-read/write commands for the same
> reason.

The difference is that guest-file-read/write have the payload in JSON;
for file-based secrets the payload is not JSON.

So I think that "binary" (which is the default anyway) would fit all the
usecases (direct over JSON, file-based, direct over command line).
Direct over JSON would be limited to valid UTF-8, but that's just a
limitation of the transport.

Paolo

[Qemu-block] [PATCH 16/17] block: remove all encryption handling APIs

[Daniel P. Berrange]

Now that all encryption keys must be provided upfront via
the QCryptoSecret API and associated block driver properties
there is no need for any explicit encryption handling APIs
in the block layer. Encryption can be handled transparently
within the block driver. We only retain an API for querying
whether an image is encrypted or not, since that is a
potentially useful piece of metadata to report to the user.

Signed-off-by: Daniel P. Berrange 
---
 block.c| 81 ++
 block/qapi.c   |  2 +-
 block/qcow.c   | 19 ---
 block/qcow2.c  | 19 ---
 blockdev.c | 69 +--
 include/block/block.h  |  4 ---
 tests/qemu-iotests/087.out |  4 +--
 7 files changed, 21 insertions(+), 177 deletions(-)

diff --git a/block.c b/block.c
index 09f2a75..bb8cb96 100644
--- a/block.c
+++ b/block.c
@@ -1542,17 +1542,8 @@ static int bdrv_open_inherit(BlockDriverState **pbs, 
const char *filename,
 goto close_and_fail;
 }
 
-if (!bdrv_key_required(bs)) {
-if (bs->blk) {
-blk_dev_change_media_cb(bs->blk, true);
-}
-} else if (!runstate_check(RUN_STATE_PRELAUNCH)
-   && !runstate_check(RUN_STATE_INMIGRATE)
-   && !runstate_check(RUN_STATE_PAUSED)) { /* HACK */
-error_setg(errp,
-   "Guest must be stopped for opening of encrypted image");
-ret = -EBUSY;
-goto close_and_fail;
+if (bs->blk) {
+blk_dev_change_media_cb(bs->blk, true);
 }
 
 QDECREF(options);
@@ -2608,74 +2599,6 @@ int bdrv_is_encrypted(BlockDriverState *bs)
 return bs->encrypted;
 }
 
-int bdrv_key_required(BlockDriverState *bs)
-{
-BdrvChild *backing = bs->backing;
-
-if (backing && backing->bs->encrypted && !backing->bs->valid_key) {
-return 1;
-}
-return (bs->encrypted && !bs->valid_key);
-}
-
-int bdrv_set_key(BlockDriverState *bs, const char *key)
-{
-int ret;
-if (bs->backing && bs->backing->bs->encrypted) {
-ret = bdrv_set_key(bs->backing->bs, key);
-if (ret < 0)
-return ret;
-if (!bs->encrypted)
-return 0;
-}
-if (!bs->encrypted) {
-return -EINVAL;
-} else if (!bs->drv || !bs->drv->bdrv_set_key) {
-return -ENOMEDIUM;
-}
-ret = bs->drv->bdrv_set_key(bs, key);
-if (ret < 0) {
-bs->valid_key = 0;
-} else if (!bs->valid_key) {
-bs->valid_key = 1;
-if (bs->blk) {
-/* call the change callback now, we skipped it on open */
-blk_dev_change_media_cb(bs->blk, true);
-}
-}
-return ret;
-}
-
-/*
- * Provide an encryption key for @bs.
- * If @key is non-null:
- * If @bs is not encrypted, fail.
- * Else if the key is invalid, fail.
- * Else set @bs's key to @key, replacing the existing key, if any.
- * If @key is null:
- * If @bs is encrypted and still lacks a key, fail.
- * Else do nothing.
- * On failure, store an error object through @errp if non-null.
- */
-void bdrv_add_key(BlockDriverState *bs, const char *key, Error **errp)
-{
-if (key) {
-if (!bdrv_is_encrypted(bs)) {
-error_setg(errp, "Node '%s' is not encrypted",
-  bdrv_get_device_or_node_name(bs));
-} else if (bdrv_set_key(bs, key) < 0) {
-error_setg(errp, QERR_INVALID_PASSWORD);
-}
-} else {
-if (bdrv_key_required(bs)) {
-error_set(errp, ERROR_CLASS_DEVICE_ENCRYPTED,
-  "'%s' (%s) is encrypted",
-  bdrv_get_device_or_node_name(bs),
-  bdrv_get_encrypted_filename(bs));
-}
-}
-}
-
 const char *bdrv_get_format_name(BlockDriverState *bs)
 {
 return bs->drv ? bs->drv->format_name : NULL;
diff --git a/block/qapi.c b/block/qapi.c
index 355ba32..a38258f 100644
--- a/block/qapi.c
+++ b/block/qapi.c
@@ -42,7 +42,7 @@ BlockDeviceInfo *bdrv_block_device_info(BlockDriverState *bs, 
Error **errp)
 info->ro = bs->read_only;
 info->drv= g_strdup(bs->drv->format_name);
 info->encrypted  = bs->encrypted;
-info->encryption_key_missing = bdrv_key_required(bs);
+info->encryption_key_missing = false;
 
 info->cache = g_new(BlockdevCacheInfo, 1);
 *info->cache = (BlockdevCacheInfo) {
diff --git a/block/qcow.c b/block/qcow.c
index ff80ef5..ccf6de1 100644
--- a/block/qcow.c
+++ b/block/qcow.c
@@ -222,11 +222,6 @@ static int qcow_open(BlockDriverState *bs, QDict *options, 
int flags,
 ret = -EINVAL;
 goto fail;
 }
-if (!qcrypto_cipher_supports(QCRYPTO_CIPHER_ALG_AES_128)) {
-error_setg(errp, "AES cipher not available");
-ret = -EINVAL;
-goto fail;
-}
 s->crypt_method_header = header.crypt_method;
 if 

[Qemu-block] [PATCH 08/17] qom: add user_creatable_add & user_creatable_del methods

[Daniel P. Berrange]

The QMP monitor code has two helper methods object_add
and qmp_object_del that are called from several places
in the code (QMP, HMP and main emulator startup).

We soon need to use this code from qemu-img, qemu-io
and qemu-nbd too, but don't want those to depend on
the monitor.

To avoid this, move object_add to user_creatable_add
an qmp_object_del to user_creatable_del, in the
object_interfaces.c file

Signed-off-by: Daniel P. Berrange 
---
 hmp.c   | 11 --
 include/monitor/monitor.h   |  3 --
 include/qom/object_interfaces.h | 31 +
 qmp.c   | 75 
 qom/object_interfaces.c | 76 +
 vl.c|  8 +++--
 6 files changed, 127 insertions(+), 77 deletions(-)

diff --git a/hmp.c b/hmp.c
index 5048eee..409d05d 100644
--- a/hmp.c
+++ b/hmp.c
@@ -28,6 +28,7 @@
 #include "qapi/qmp/qerror.h"
 #include "qapi/string-output-visitor.h"
 #include "qapi-visit.h"
+#include "qom/object_interfaces.h"
 #include "ui/console.h"
 #include "block/qapi.h"
 #include "qemu-io.h"
@@ -1630,6 +1631,7 @@ void hmp_object_add(Monitor *mon, const QDict *qdict)
 void *dummy = NULL;
 OptsVisitor *ov;
 QDict *pdict;
+Object *obj = NULL;
 
 opts = qemu_opts_from_qdict(qemu_find_opts("object"), qdict, );
 if (err) {
@@ -1656,12 +1658,12 @@ void hmp_object_add(Monitor *mon, const QDict *qdict)
 goto out_end;
 }
 
-object_add(type, id, pdict, opts_get_visitor(ov), );
+obj = user_creatable_add(type, id, pdict, opts_get_visitor(ov), );
 
 out_end:
 visit_end_struct(opts_get_visitor(ov), _end);
 if (!err && err_end) {
-qmp_object_del(id, NULL);
+user_creatable_del(id, NULL);
 }
 error_propagate(, err_end);
 out_clean:
@@ -1672,6 +1674,9 @@ out_clean:
 g_free(id);
 g_free(type);
 g_free(dummy);
+if (obj) {
+object_unref(obj);
+}
 
 out:
 hmp_handle_error(mon, );
@@ -1904,7 +1909,7 @@ void hmp_object_del(Monitor *mon, const QDict *qdict)
 const char *id = qdict_get_str(qdict, "id");
 Error *err = NULL;
 
-qmp_object_del(id, );
+user_creatable_del(id, );
 hmp_handle_error(mon, );
 }
 
diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
index 91b95ae..aa0f373 100644
--- a/include/monitor/monitor.h
+++ b/include/monitor/monitor.h
@@ -43,9 +43,6 @@ void monitor_read_command(Monitor *mon, int show_prompt);
 int monitor_read_password(Monitor *mon, ReadLineFunc *readline_func,
   void *opaque);
 
-void object_add(const char *type, const char *id, const QDict *qdict,
-Visitor *v, Error **errp);
-
 AddfdInfo *monitor_fdset_add_fd(int fd, bool has_fdset_id, int64_t fdset_id,
 bool has_opaque, const char *opaque,
 Error **errp);
diff --git a/include/qom/object_interfaces.h b/include/qom/object_interfaces.h
index 283ae0d..3e2afeb 100644
--- a/include/qom/object_interfaces.h
+++ b/include/qom/object_interfaces.h
@@ -2,6 +2,8 @@
 #define OBJECT_INTERFACES_H
 
 #include "qom/object.h"
+#include "qapi/qmp/qdict.h"
+#include "qapi/visitor.h"
 
 #define TYPE_USER_CREATABLE "user-creatable"
 
@@ -72,4 +74,33 @@ void user_creatable_complete(Object *obj, Error **errp);
  * from implements USER_CREATABLE interface.
  */
 bool user_creatable_can_be_deleted(UserCreatable *uc, Error **errp);
+
+/**
+ * user_creatable_add:
+ * @type: the object type name
+ * @id: the unique ID for the object
+ * @qdict: the object parameters
+ * @v: the visitor
+ * @errp: if an error occurs, a pointer to an area to store the error
+ *
+ * Create an instance of the user creatable object @type, placing
+ * it in the object composition tree with name @id, initializing
+ * it with properties from @qdict
+ *
+ * Returns: the newly created object or NULL on error
+ */
+Object *user_creatable_add(const char *type, const char *id,
+   const QDict *qdict,
+   Visitor *v, Error **errp);
+
+/**
+ * user_creatable_del:
+ * @id: the unique ID for the object
+ * @errp: if an error occurs, a pointer to an area to store the error
+ *
+ * Delete an instance of the user creatable object identified
+ * by @id.
+ */
+void user_creatable_del(const char *id, Error **errp);
+
 #endif
diff --git a/qmp.c b/qmp.c
index d9ecede..feea847 100644
--- a/qmp.c
+++ b/qmp.c
@@ -613,65 +613,13 @@ void qmp_add_client(const char *protocol, const char 
*fdname,
 close(fd);
 }
 
-void object_add(const char *type, const char *id, const QDict *qdict,
-Visitor *v, Error **errp)
-{
-Object *obj;
-ObjectClass *klass;
-const QDictEntry *e;
-Error *local_err = NULL;
-
-klass = object_class_by_name(type);
-if (!klass) {
-error_setg(errp, "invalid object type: %s", type);
-return;
-}
-
-   

[Qemu-block] [PATCH v7 01/39] block: Remove host floppy support

[Max Reitz]

It has been deprecated as of 2.3, so we can now remove it.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Kevin Wolf 
---
 block/raw-posix.c| 222 ++-
 qapi/block-core.json |   9 +--
 2 files changed, 9 insertions(+), 222 deletions(-)

diff --git a/block/raw-posix.c b/block/raw-posix.c
index 3a527f0..2e3db44 100644
--- a/block/raw-posix.c
+++ b/block/raw-posix.c
@@ -127,11 +127,6 @@ do { \
 
 #define FTYPE_FILE   0
 #define FTYPE_CD 1
-#define FTYPE_FD 2
-
-/* if the FD is not accessed during that time (in ns), we try to
-   reopen it to see if the disk has been changed */
-#define FD_OPEN_TIMEOUT (10)
 
 #define MAX_BLOCKSIZE  4096
 
@@ -141,13 +136,6 @@ typedef struct BDRVRawState {
 int open_flags;
 size_t buf_align;
 
-#if defined(__linux__)
-/* linux floppy specific */
-int64_t fd_open_time;
-int64_t fd_error_time;
-int fd_got_error;
-int fd_media_changed;
-#endif
 #ifdef CONFIG_LINUX_AIO
 int use_aio;
 void *aio_ctx;
@@ -635,7 +623,7 @@ static int raw_reopen_prepare(BDRVReopenState *state,
 }
 #endif
 
-if (s->type == FTYPE_FD || s->type == FTYPE_CD) {
+if (s->type == FTYPE_CD) {
 raw_s->open_flags |= O_NONBLOCK;
 }
 
@@ -2187,47 +2175,6 @@ static int hdev_open(BlockDriverState *bs, QDict 
*options, int flags,
 }
 
 #if defined(__linux__)
-/* Note: we do not have a reliable method to detect if the floppy is
-   present. The current method is to try to open the floppy at every
-   I/O and to keep it opened during a few hundreds of ms. */
-static int fd_open(BlockDriverState *bs)
-{
-BDRVRawState *s = bs->opaque;
-int last_media_present;
-
-if (s->type != FTYPE_FD)
-return 0;
-last_media_present = (s->fd >= 0);
-if (s->fd >= 0 &&
-(qemu_clock_get_ns(QEMU_CLOCK_REALTIME) - s->fd_open_time) >= 
FD_OPEN_TIMEOUT) {
-qemu_close(s->fd);
-s->fd = -1;
-DPRINTF("Floppy closed\n");
-}
-if (s->fd < 0) {
-if (s->fd_got_error &&
-(qemu_clock_get_ns(QEMU_CLOCK_REALTIME) - s->fd_error_time) < 
FD_OPEN_TIMEOUT) {
-DPRINTF("No floppy (open delayed)\n");
-return -EIO;
-}
-s->fd = qemu_open(bs->filename, s->open_flags & ~O_NONBLOCK);
-if (s->fd < 0) {
-s->fd_error_time = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
-s->fd_got_error = 1;
-if (last_media_present)
-s->fd_media_changed = 1;
-DPRINTF("No floppy\n");
-return -EIO;
-}
-DPRINTF("Floppy opened\n");
-}
-if (!last_media_present)
-s->fd_media_changed = 1;
-s->fd_open_time = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
-s->fd_got_error = 0;
-return 0;
-}
-
 static int hdev_ioctl(BlockDriverState *bs, unsigned long int req, void *buf)
 {
 BDRVRawState *s = bs->opaque;
@@ -2256,8 +2203,8 @@ static BlockAIOCB *hdev_aio_ioctl(BlockDriverState *bs,
 pool = aio_get_thread_pool(bdrv_get_aio_context(bs));
 return thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque);
 }
+#endif /* linux */
 
-#elif defined(__FreeBSD__) || defined(__FreeBSD_kernel__)
 static int fd_open(BlockDriverState *bs)
 {
 BDRVRawState *s = bs->opaque;
@@ -2267,14 +2214,6 @@ static int fd_open(BlockDriverState *bs)
 return 0;
 return -EIO;
 }
-#else /* !linux && !FreeBSD */
-
-static int fd_open(BlockDriverState *bs)
-{
-return 0;
-}
-
-#endif /* !linux && !FreeBSD */
 
 static coroutine_fn BlockAIOCB *hdev_aio_discard(BlockDriverState *bs,
 int64_t sector_num, int nb_sectors,
@@ -2318,14 +2257,13 @@ static int hdev_create(const char *filename, QemuOpts 
*opts,
 int64_t total_size = 0;
 bool has_prefix;
 
-/* This function is used by all three protocol block drivers and therefore
- * any of these three prefixes may be given.
+/* This function is used by both protocol block drivers and therefore 
either
+ * of these prefixes may be given.
  * The return value has to be stored somewhere, otherwise this is an error
  * due to -Werror=unused-value. */
 has_prefix =
 strstart(filename, "host_device:", ) ||
-strstart(filename, "host_cdrom:" , ) ||
-strstart(filename, "host_floppy:", );
+strstart(filename, "host_cdrom:" , );
 
 (void)has_prefix;
 
@@ -2405,155 +2343,6 @@ static BlockDriver bdrv_host_device = {
 #endif
 };
 
-#ifdef __linux__
-static void floppy_parse_filename(const char *filename, QDict *options,
-  Error **errp)
-{
-/* The prefix is optional, just as for "file". */
-strstart(filename, "host_floppy:", );
-
-qdict_put_obj(options, "filename", QOBJECT(qstring_from_str(filename)));
-}
-
-static int floppy_open(BlockDriverState *bs, QDict *options, int flags,
-   Error **errp)
-{
-

[Qemu-block] [PATCH v7 07/39] block: Make bdrv_is_inserted() recursive

[Max Reitz]

If bdrv_is_inserted() is called on the top level BDS, it should make
sure all nodes in the BDS tree are actually inserted.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Kevin Wolf 
---
 block.c | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/block.c b/block.c
index 2a77169..0f23a98 100644
--- a/block.c
+++ b/block.c
@@ -3143,14 +3143,20 @@ void bdrv_invalidate_cache_all(Error **errp)
 bool bdrv_is_inserted(BlockDriverState *bs)
 {
 BlockDriver *drv = bs->drv;
+BdrvChild *child;
 
 if (!drv) {
 return false;
 }
-if (!drv->bdrv_is_inserted) {
-return true;
+if (drv->bdrv_is_inserted) {
+return drv->bdrv_is_inserted(bs);
 }
-return drv->bdrv_is_inserted(bs);
+QLIST_FOREACH(child, >children, next) {
+if (!bdrv_is_inserted(child->bs)) {
+return false;
+}
+}
+return true;
 }
 
 /**
-- 
2.6.1

[Qemu-block] [PATCH v7 13/39] block: Move guest_block_size into BlockBackend

[Max Reitz]

guest_block_size is a guest device property so it should be moved into
the interface between block layer and guest devices, which is the
BlockBackend.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Alberto Garcia 
Reviewed-by: Kevin Wolf 
---
 block.c   | 7 ---
 block/block-backend.c | 7 +--
 include/block/block.h | 1 -
 include/block/block_int.h | 3 ---
 4 files changed, 5 insertions(+), 13 deletions(-)

diff --git a/block.c b/block.c
index 59a2a62..2f69ec7 100644
--- a/block.c
+++ b/block.c
@@ -857,7 +857,6 @@ static int bdrv_open_common(BlockDriverState *bs, BdrvChild 
*file,
 goto fail_opts;
 }
 
-bs->guest_block_size = 512;
 bs->request_alignment = 512;
 bs->zero_beyond_eof = true;
 open_flags = bdrv_open_flags(bs, flags);
@@ -2002,7 +2001,6 @@ static void bdrv_move_feature_fields(BlockDriverState 
*bs_dest,
 /* move some fields that need to stay attached to the device */
 
 /* dev info */
-bs_dest->guest_block_size   = bs_src->guest_block_size;
 bs_dest->copy_on_read   = bs_src->copy_on_read;
 
 bs_dest->enable_write_cache = bs_src->enable_write_cache;
@@ -3207,11 +3205,6 @@ void bdrv_lock_medium(BlockDriverState *bs, bool locked)
 }
 }
 
-void bdrv_set_guest_block_size(BlockDriverState *bs, int align)
-{
-bs->guest_block_size = align;
-}
-
 BdrvDirtyBitmap *bdrv_find_dirty_bitmap(BlockDriverState *bs, const char *name)
 {
 BdrvDirtyBitmap *bm;
diff --git a/block/block-backend.c b/block/block-backend.c
index c7e0f7b..7bc2eb1 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -31,6 +31,9 @@ struct BlockBackend {
 /* TODO change to DeviceState when all users are qdevified */
 const BlockDevOps *dev_ops;
 void *dev_opaque;
+
+/* the block size for which the guest device expects atomicity */
+int guest_block_size;
 };
 
 typedef struct BlockBackendAIOCB {
@@ -351,7 +354,7 @@ void blk_detach_dev(BlockBackend *blk, void *dev)
 blk->dev = NULL;
 blk->dev_ops = NULL;
 blk->dev_opaque = NULL;
-bdrv_set_guest_block_size(blk->bs, 512);
+blk->guest_block_size = 512;
 blk_unref(blk);
 }
 
@@ -806,7 +809,7 @@ int blk_get_max_transfer_length(BlockBackend *blk)
 
 void blk_set_guest_block_size(BlockBackend *blk, int align)
 {
-bdrv_set_guest_block_size(blk->bs, align);
+blk->guest_block_size = align;
 }
 
 void *blk_blockalign(BlockBackend *blk, size_t size)
diff --git a/include/block/block.h b/include/block/block.h
index db860ef..0f40d9f 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -466,7 +466,6 @@ void bdrv_img_create(const char *filename, const char *fmt,
 size_t bdrv_min_mem_align(BlockDriverState *bs);
 /* Returns optimal alignment in bytes for bounce buffer */
 size_t bdrv_opt_mem_align(BlockDriverState *bs);
-void bdrv_set_guest_block_size(BlockDriverState *bs, int align);
 void *qemu_blockalign(BlockDriverState *bs, size_t size);
 void *qemu_blockalign0(BlockDriverState *bs, size_t size);
 void *qemu_try_blockalign(BlockDriverState *bs, size_t size);
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 40d40df..e79d8c0 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -411,9 +411,6 @@ struct BlockDriverState {
 /* Alignment requirement for offset/length of I/O requests */
 unsigned int request_alignment;
 
-/* the block size for which the guest device expects atomicity */
-int guest_block_size;
-
 /* do we need to tell the quest if we have a volatile write cache? */
 int enable_write_cache;
 
-- 
2.6.1

[Qemu-block] [PATCH v7 21/39] block: Prepare remaining BB functions for NULL BDS

[Max Reitz]

There are several BlockBackend functions which, in theory, cannot fail.
This patch makes them cope with the BlockDriverState pointer being NULL
by making them fall back to some default action like ignoring the value
in setters and returning the default in getters.

Signed-off-by: Max Reitz 
Reviewed-by: Kevin Wolf 
---
 block/block-backend.c | 72 +++
 1 file changed, 56 insertions(+), 16 deletions(-)

diff --git a/block/block-backend.c b/block/block-backend.c
index 2779c22..a5c58c5 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -677,7 +677,11 @@ int64_t blk_getlength(BlockBackend *blk)
 
 void blk_get_geometry(BlockBackend *blk, uint64_t *nb_sectors_ptr)
 {
-bdrv_get_geometry(blk->bs, nb_sectors_ptr);
+if (!blk->bs) {
+*nb_sectors_ptr = 0;
+} else {
+bdrv_get_geometry(blk->bs, nb_sectors_ptr);
+}
 }
 
 int64_t blk_nb_sectors(BlockBackend *blk)
@@ -813,7 +817,9 @@ int blk_flush_all(void)
 
 void blk_drain(BlockBackend *blk)
 {
-bdrv_drain(blk->bs);
+if (blk->bs) {
+bdrv_drain(blk->bs);
+}
 }
 
 void blk_drain_all(void)
@@ -909,6 +915,10 @@ int blk_is_read_only(BlockBackend *blk)
 
 int blk_is_sg(BlockBackend *blk)
 {
+if (!blk->bs) {
+return 0;
+}
+
 return bdrv_is_sg(blk->bs);
 }
 
@@ -956,12 +966,16 @@ bool blk_is_available(BlockBackend *blk)
 
 void blk_lock_medium(BlockBackend *blk, bool locked)
 {
-bdrv_lock_medium(blk->bs, locked);
+if (blk->bs) {
+bdrv_lock_medium(blk->bs, locked);
+}
 }
 
 void blk_eject(BlockBackend *blk, bool eject_flag)
 {
-bdrv_eject(blk->bs, eject_flag);
+if (blk->bs) {
+bdrv_eject(blk->bs, eject_flag);
+}
 }
 
 int blk_get_flags(BlockBackend *blk)
@@ -975,7 +989,11 @@ int blk_get_flags(BlockBackend *blk)
 
 int blk_get_max_transfer_length(BlockBackend *blk)
 {
-return blk->bs->bl.max_transfer_length;
+if (blk->bs) {
+return blk->bs->bl.max_transfer_length;
+} else {
+return 0;
+}
 }
 
 void blk_set_guest_block_size(BlockBackend *blk, int align)
@@ -990,22 +1008,32 @@ void *blk_blockalign(BlockBackend *blk, size_t size)
 
 bool blk_op_is_blocked(BlockBackend *blk, BlockOpType op, Error **errp)
 {
+if (!blk->bs) {
+return false;
+}
+
 return bdrv_op_is_blocked(blk->bs, op, errp);
 }
 
 void blk_op_unblock(BlockBackend *blk, BlockOpType op, Error *reason)
 {
-bdrv_op_unblock(blk->bs, op, reason);
+if (blk->bs) {
+bdrv_op_unblock(blk->bs, op, reason);
+}
 }
 
 void blk_op_block_all(BlockBackend *blk, Error *reason)
 {
-bdrv_op_block_all(blk->bs, reason);
+if (blk->bs) {
+bdrv_op_block_all(blk->bs, reason);
+}
 }
 
 void blk_op_unblock_all(BlockBackend *blk, Error *reason)
 {
-bdrv_op_unblock_all(blk->bs, reason);
+if (blk->bs) {
+bdrv_op_unblock_all(blk->bs, reason);
+}
 }
 
 AioContext *blk_get_aio_context(BlockBackend *blk)
@@ -1025,15 +1053,19 @@ static AioContext *blk_aiocb_get_aio_context(BlockAIOCB 
*acb)
 
 void blk_set_aio_context(BlockBackend *blk, AioContext *new_context)
 {
-bdrv_set_aio_context(blk->bs, new_context);
+if (blk->bs) {
+bdrv_set_aio_context(blk->bs, new_context);
+}
 }
 
 void blk_add_aio_context_notifier(BlockBackend *blk,
 void (*attached_aio_context)(AioContext *new_context, void *opaque),
 void (*detach_aio_context)(void *opaque), void *opaque)
 {
-bdrv_add_aio_context_notifier(blk->bs, attached_aio_context,
-  detach_aio_context, opaque);
+if (blk->bs) {
+bdrv_add_aio_context_notifier(blk->bs, attached_aio_context,
+  detach_aio_context, opaque);
+}
 }
 
 void blk_remove_aio_context_notifier(BlockBackend *blk,
@@ -1042,23 +1074,31 @@ void blk_remove_aio_context_notifier(BlockBackend *blk,
  void (*detach_aio_context)(void *),
  void *opaque)
 {
-bdrv_remove_aio_context_notifier(blk->bs, attached_aio_context,
- detach_aio_context, opaque);
+if (blk->bs) {
+bdrv_remove_aio_context_notifier(blk->bs, attached_aio_context,
+ detach_aio_context, opaque);
+}
 }
 
 void blk_add_close_notifier(BlockBackend *blk, Notifier *notify)
 {
-bdrv_add_close_notifier(blk->bs, notify);
+if (blk->bs) {
+bdrv_add_close_notifier(blk->bs, notify);
+}
 }
 
 void blk_io_plug(BlockBackend *blk)
 {
-bdrv_io_plug(blk->bs);
+if (blk->bs) {
+bdrv_io_plug(blk->bs);
+}
 }
 
 void blk_io_unplug(BlockBackend *blk)
 {
-bdrv_io_unplug(blk->bs);
+if (blk->bs) {
+bdrv_io_unplug(blk->bs);
+}
 }
 
 BlockAcctStats *blk_get_stats(BlockBackend *blk)
-- 
2.6.1

[Qemu-block] [PATCH v7 17/39] block/throttle-groups: Make incref/decref public

[Max Reitz]

Throttle groups are not necessarily referenced by BDSs alone; a later
patch will essentially allow BBs to reference them, too. Make the
ref/unref functions public so that reference can be properly accounted
for.

Their interface is slightly adjusted in that they return and take a
ThrottleState pointer, respectively, instead of a ThrottleGroup pointer.
Functionally, they are equivalent, but since ThrottleGroup is not meant
to be used outside of block/throttle-groups.c, ThrottleState is easier
to handle.

Signed-off-by: Max Reitz 
Reviewed-by: Kevin Wolf 
---
 block/throttle-groups.c | 19 +++
 include/block/throttle-groups.h |  3 +++
 2 files changed, 14 insertions(+), 8 deletions(-)

diff --git a/block/throttle-groups.c b/block/throttle-groups.c
index 1abc6fc..20cb216 100644
--- a/block/throttle-groups.c
+++ b/block/throttle-groups.c
@@ -76,9 +76,9 @@ static QTAILQ_HEAD(, ThrottleGroup) throttle_groups =
  * created.
  *
  * @name: the name of the ThrottleGroup
- * @ret:  the ThrottleGroup
+ * @ret:  the ThrottleState member of the ThrottleGroup
  */
-static ThrottleGroup *throttle_group_incref(const char *name)
+ThrottleState *throttle_group_incref(const char *name)
 {
 ThrottleGroup *tg = NULL;
 ThrottleGroup *iter;
@@ -108,7 +108,7 @@ static ThrottleGroup *throttle_group_incref(const char 
*name)
 
 qemu_mutex_unlock(_groups_lock);
 
-return tg;
+return >ts;
 }
 
 /* Decrease the reference count of a ThrottleGroup.
@@ -116,10 +116,12 @@ static ThrottleGroup *throttle_group_incref(const char 
*name)
  * When the reference count reaches zero the ThrottleGroup is
  * destroyed.
  *
- * @tg:  The ThrottleGroup to unref
+ * @ts:  The ThrottleGroup to unref, given by its ThrottleState member
  */
-static void throttle_group_unref(ThrottleGroup *tg)
+void throttle_group_unref(ThrottleState *ts)
 {
+ThrottleGroup *tg = container_of(ts, ThrottleGroup, ts);
+
 qemu_mutex_lock(_groups_lock);
 if (--tg->refcount == 0) {
 QTAILQ_REMOVE(_groups, tg, list);
@@ -401,7 +403,8 @@ static void write_timer_cb(void *opaque)
 void throttle_group_register_bs(BlockDriverState *bs, const char *groupname)
 {
 int i;
-ThrottleGroup *tg = throttle_group_incref(groupname);
+ThrottleState *ts = throttle_group_incref(groupname);
+ThrottleGroup *tg = container_of(ts, ThrottleGroup, ts);
 int clock_type = QEMU_CLOCK_REALTIME;
 
 if (qtest_enabled()) {
@@ -409,7 +412,7 @@ void throttle_group_register_bs(BlockDriverState *bs, const 
char *groupname)
 clock_type = QEMU_CLOCK_VIRTUAL;
 }
 
-bs->throttle_state = >ts;
+bs->throttle_state = ts;
 
 qemu_mutex_lock(>lock);
 /* If the ThrottleGroup is new set this BlockDriverState as the token */
@@ -461,7 +464,7 @@ void throttle_group_unregister_bs(BlockDriverState *bs)
 throttle_timers_destroy(>throttle_timers);
 qemu_mutex_unlock(>lock);
 
-throttle_group_unref(tg);
+throttle_group_unref(>ts);
 bs->throttle_state = NULL;
 }
 
diff --git a/include/block/throttle-groups.h b/include/block/throttle-groups.h
index fab113f..f3b75b3 100644
--- a/include/block/throttle-groups.h
+++ b/include/block/throttle-groups.h
@@ -30,6 +30,9 @@
 
 const char *throttle_group_get_name(BlockDriverState *bs);
 
+ThrottleState *throttle_group_incref(const char *name);
+void throttle_group_unref(ThrottleState *ts);
+
 void throttle_group_config(BlockDriverState *bs, ThrottleConfig *cfg);
 void throttle_group_get_config(BlockDriverState *bs, ThrottleConfig *cfg);
 
-- 
2.6.1

[Qemu-block] [PATCH v7 35/39] qmp: Introduce blockdev-change-medium

[Max Reitz]

Introduce a new QMP command 'blockdev-change-medium' which is intended
to replace the 'change' command for block devices. The existing function
qmp_change_blockdev() is accordingly renamed to
qmp_blockdev_change_medium().

Signed-off-by: Max Reitz 
---
 blockdev.c|  7 ---
 include/sysemu/blockdev.h |  2 --
 qapi-schema.json  |  6 --
 qapi/block-core.json  | 23 +++
 qmp-commands.hx   | 31 +++
 qmp.c |  2 +-
 6 files changed, 63 insertions(+), 8 deletions(-)

diff --git a/blockdev.c b/blockdev.c
index 50e5e74..db789e2 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2094,8 +2094,9 @@ void qmp_blockdev_insert_medium(const char *device, const 
char *node_name,
 qmp_blockdev_insert_anon_medium(device, bs, errp);
 }
 
-void qmp_change_blockdev(const char *device, const char *filename,
- const char *format, Error **errp)
+void qmp_blockdev_change_medium(const char *device, const char *filename,
+bool has_format, const char *format,
+Error **errp)
 {
 BlockBackend *blk;
 BlockBackendRootState *blk_rs;
@@ -2119,7 +2120,7 @@ void qmp_change_blockdev(const char *device, const char 
*filename,
 bdrv_flags = blk_rs->read_only ? 0 : BDRV_O_RDWR;
 bdrv_flags |= blk_rs->open_flags & ~BDRV_O_RDWR;
 
-if (format) {
+if (has_format) {
 options = qdict_new();
 qdict_put(options, "driver", qstring_from_str(format));
 }
diff --git a/include/sysemu/blockdev.h b/include/sysemu/blockdev.h
index a00be94..b06a060 100644
--- a/include/sysemu/blockdev.h
+++ b/include/sysemu/blockdev.h
@@ -63,8 +63,6 @@ DriveInfo *drive_new(QemuOpts *arg, BlockInterfaceType 
block_default_type);
 
 /* device-hotplug */
 
-void qmp_change_blockdev(const char *device, const char *filename,
- const char *format, Error **errp);
 void hmp_commit(Monitor *mon, const QDict *qdict);
 void hmp_drive_del(Monitor *mon, const QDict *qdict);
 #endif
diff --git a/qapi-schema.json b/qapi-schema.json
index 702b7b5..058b8ec 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -1842,8 +1842,10 @@
 #  device's password.  The behavior of reads and writes to the block
 #  device between when these calls are executed is undefined.
 #
-# Notes:  It is strongly recommended that this interface is not used especially
-# for changing block devices.
+# Notes:  This interface is deprecated, and it is strongly recommended that you
+# avoid using it.  For changing block devices, use
+# blockdev-change-medium; for changing VNC parameters, use
+# change-vnc-password.
 #
 # Since: 0.14.0
 ##
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 81a1f19..b8cc18a 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1949,6 +1949,29 @@
 
 
 ##
+# @blockdev-change-medium:
+#
+# Changes the medium inserted into a block device by ejecting the current 
medium
+# and loading a new image file which is inserted as the new medium (this 
command
+# combines blockdev-open-tray, blockdev-remove-medium, blockdev-insert-medium
+# and blockdev-close-tray).
+#
+# @device:  block device name
+#
+# @filename:filename of the new image to be loaded
+#
+# @format:  #optional, format to open the new image with (defaults to
+#   the probed format)
+#
+# Since: 2.5
+##
+{ 'command': 'blockdev-change-medium',
+  'data': { 'device': 'str',
+'filename': 'str',
+'*format': 'str' } }
+
+
+##
 # @BlockErrorAction
 #
 # An enumeration of action that has been taken when a DISK I/O occurs
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 16d7e2a..8643d95 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -4154,6 +4154,37 @@ Example:
 EQMP
 
 {
+.name   = "blockdev-change-medium",
+.args_type  = "device:B,filename:F,format:s?",
+.mhandler.cmd_new = qmp_marshal_blockdev_change_medium,
+},
+
+SQMP
+blockdev-change-medium
+--
+
+Changes the medium inserted into a block device by ejecting the current medium
+and loading a new image file which is inserted as the new medium.
+
+Arguments:
+
+- "device": device name (json-string)
+- "filename": filename of the new image (json-string)
+- "format": format of the new image (json-string, optional)
+
+Examples:
+
+1. Change a removable medium
+
+-> { "execute": "blockdev-change-medium",
+ "arguments": { "device": "ide1-cd0",
+"filename": "/srv/images/Fedora-12-x86_64-DVD.iso",
+"format": "raw" } }
+<- { "return": {} }
+
+EQMP
+
+{
 .name   = "query-memdev",
 .args_type  = "",
 .mhandler.cmd_new = qmp_marshal_query_memdev,
diff --git a/qmp.c b/qmp.c
index ff54e5a..4e44f98 100644
--- a/qmp.c
+++ b/qmp.c
@@ 

[Qemu-block] [PATCH v7 39/39] iotests: Add test for change-related QMP commands

[Max Reitz]

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
---
 tests/qemu-iotests/118 | 638 +
 tests/qemu-iotests/118.out |   5 +
 tests/qemu-iotests/group   |   1 +
 3 files changed, 644 insertions(+)
 create mode 100755 tests/qemu-iotests/118
 create mode 100644 tests/qemu-iotests/118.out

diff --git a/tests/qemu-iotests/118 b/tests/qemu-iotests/118
new file mode 100755
index 000..915e439
--- /dev/null
+++ b/tests/qemu-iotests/118
@@ -0,0 +1,638 @@
+#!/usr/bin/env python
+#
+# Test case for the QMP 'change' command and all other associated
+# commands
+#
+# Copyright (C) 2015 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+#
+
+import os
+import stat
+import time
+import iotests
+from iotests import qemu_img
+
+old_img = os.path.join(iotests.test_dir, 'test0.img')
+new_img = os.path.join(iotests.test_dir, 'test1.img')
+
+class ChangeBaseClass(iotests.QMPTestCase):
+has_opened = False
+has_closed = False
+
+def process_events(self):
+for event in self.vm.get_qmp_events(wait=False):
+if (event['event'] == 'DEVICE_TRAY_MOVED' and
+event['data']['device'] == 'drive0'):
+if event['data']['tray-open'] == False:
+self.has_closed = True
+else:
+self.has_opened = True
+
+def wait_for_open(self):
+timeout = time.clock() + 3
+while not self.has_opened and time.clock() < timeout:
+self.process_events()
+if not self.has_opened:
+self.fail('Timeout while waiting for the tray to open')
+
+def wait_for_close(self):
+timeout = time.clock() + 3
+while not self.has_closed and time.clock() < timeout:
+self.process_events()
+if not self.has_opened:
+self.fail('Timeout while waiting for the tray to close')
+
+class GeneralChangeTestsBaseClass(ChangeBaseClass):
+def test_change(self):
+result = self.vm.qmp('change', device='drive0', target=new_img,
+   arg=iotests.imgfmt)
+self.assert_qmp(result, 'return', {})
+
+self.wait_for_open()
+self.wait_for_close()
+
+result = self.vm.qmp('query-block')
+self.assert_qmp(result, 'return[0]/tray_open', False)
+self.assert_qmp(result, 'return[0]/inserted/image/filename', new_img)
+
+def test_blockdev_change_medium(self):
+result = self.vm.qmp('blockdev-change-medium', device='drive0',
+   filename=new_img,
+   format=iotests.imgfmt)
+self.assert_qmp(result, 'return', {})
+
+self.wait_for_open()
+self.wait_for_close()
+
+result = self.vm.qmp('query-block')
+self.assert_qmp(result, 'return[0]/tray_open', False)
+self.assert_qmp(result, 'return[0]/inserted/image/filename', new_img)
+
+def test_eject(self):
+result = self.vm.qmp('eject', device='drive0', force=True)
+self.assert_qmp(result, 'return', {})
+
+self.wait_for_open()
+
+result = self.vm.qmp('query-block')
+self.assert_qmp(result, 'return[0]/tray_open', True)
+self.assert_qmp_absent(result, 'return[0]/inserted')
+
+def test_tray_eject_change(self):
+result = self.vm.qmp('eject', device='drive0', force=True)
+self.assert_qmp(result, 'return', {})
+
+self.wait_for_open()
+
+result = self.vm.qmp('query-block')
+self.assert_qmp(result, 'return[0]/tray_open', True)
+self.assert_qmp_absent(result, 'return[0]/inserted')
+
+result = self.vm.qmp('blockdev-change-medium', device='drive0',
+   filename=new_img,
+   format=iotests.imgfmt)
+self.assert_qmp(result, 'return', {})
+
+self.wait_for_close()
+
+result = self.vm.qmp('query-block')
+self.assert_qmp(result, 'return[0]/tray_open', False)
+self.assert_qmp(result, 'return[0]/inserted/image/filename', new_img)
+
+def test_tray_open_close(self):
+result = self.vm.qmp('blockdev-open-tray', device='drive0', force=True)
+self.assert_qmp(result, 'return', {})
+
+   

[Qemu-block] [PATCH v7 33/39] blockdev: Implement change with basic operations

[Max Reitz]

Implement 'change' on block devices by calling blockdev-open-tray,
blockdev-remove-medium, blockdev-insert-medium (a variation of that
which does not need a node-name) and blockdev-close-tray.

Signed-off-by: Max Reitz 
---
 blockdev.c | 184 +
 1 file changed, 74 insertions(+), 110 deletions(-)

diff --git a/blockdev.c b/blockdev.c
index 0481686..50e5e74 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1901,44 +1901,6 @@ exit:
 }
 }
 
-
-static void eject_device(BlockBackend *blk, int force, Error **errp)
-{
-BlockDriverState *bs = blk_bs(blk);
-AioContext *aio_context;
-
-if (!bs) {
-/* No medium inserted, so there is nothing to do */
-return;
-}
-
-aio_context = bdrv_get_aio_context(bs);
-aio_context_acquire(aio_context);
-
-if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_EJECT, errp)) {
-goto out;
-}
-if (!blk_dev_has_removable_media(blk)) {
-error_setg(errp, "Device '%s' is not removable",
-   bdrv_get_device_name(bs));
-goto out;
-}
-
-if (blk_dev_is_medium_locked(blk) && !blk_dev_is_tray_open(blk)) {
-blk_dev_eject_request(blk, force);
-if (!force) {
-error_setg(errp, "Device '%s' is locked",
-   bdrv_get_device_name(bs));
-goto out;
-}
-}
-
-bdrv_close(bs);
-
-out:
-aio_context_release(aio_context);
-}
-
 void qmp_eject(const char *device, bool has_force, bool force, Error **errp)
 {
 Error *local_err = NULL;
@@ -1976,78 +1938,6 @@ void qmp_block_passwd(bool has_device, const char 
*device,
 aio_context_release(aio_context);
 }
 
-/* Assumes AioContext is held */
-static void qmp_bdrv_open_encrypted(BlockDriverState **pbs,
-const char *filename,
-int bdrv_flags, const char *format,
-const char *password, Error **errp)
-{
-Error *local_err = NULL;
-QDict *options = NULL;
-int ret;
-
-if (format) {
-options = qdict_new();
-qdict_put(options, "driver", qstring_from_str(format));
-}
-
-ret = bdrv_open(pbs, filename, NULL, options, bdrv_flags, _err);
-if (ret < 0) {
-error_propagate(errp, local_err);
-return;
-}
-
-bdrv_add_key(*pbs, password, errp);
-}
-
-void qmp_change_blockdev(const char *device, const char *filename,
- const char *format, Error **errp)
-{
-BlockBackend *blk;
-BlockDriverState *bs;
-AioContext *aio_context;
-int bdrv_flags;
-bool new_bs;
-Error *err = NULL;
-
-blk = blk_by_name(device);
-if (!blk) {
-error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
-  "Device '%s' not found", device);
-return;
-}
-bs = blk_bs(blk);
-new_bs = !bs;
-
-aio_context = blk_get_aio_context(blk);
-aio_context_acquire(aio_context);
-
-eject_device(blk, 0, );
-if (err) {
-error_propagate(errp, err);
-goto out;
-}
-
-bdrv_flags = blk_is_read_only(blk) ? 0 : BDRV_O_RDWR;
-bdrv_flags |= blk_get_root_state(blk)->open_flags & ~BDRV_O_RDWR;
-
-qmp_bdrv_open_encrypted(, filename, bdrv_flags, format, NULL, );
-if (err) {
-error_propagate(errp, err);
-goto out;
-}
-
-if (new_bs) {
-blk_insert_bs(blk, bs);
-/* Has been sent automatically by bdrv_open() if blk_bs(blk) was not
- * NULL */
-blk_dev_change_media_cb(blk, true);
-}
-
-out:
-aio_context_release(aio_context);
-}
-
 void qmp_blockdev_open_tray(const char *device, bool has_force, bool force,
 Error **errp)
 {
@@ -2204,6 +2094,80 @@ void qmp_blockdev_insert_medium(const char *device, 
const char *node_name,
 qmp_blockdev_insert_anon_medium(device, bs, errp);
 }
 
+void qmp_change_blockdev(const char *device, const char *filename,
+ const char *format, Error **errp)
+{
+BlockBackend *blk;
+BlockBackendRootState *blk_rs;
+BlockDriverState *medium_bs = NULL;
+int bdrv_flags, ret;
+QDict *options = NULL;
+Error *err = NULL;
+
+blk = blk_by_name(device);
+if (!blk) {
+error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
+  "Device '%s' not found", device);
+goto fail;
+}
+
+if (blk_bs(blk)) {
+blk_update_root_state(blk);
+}
+
+blk_rs = blk_get_root_state(blk);
+bdrv_flags = blk_rs->read_only ? 0 : BDRV_O_RDWR;
+bdrv_flags |= blk_rs->open_flags & ~BDRV_O_RDWR;
+
+if (format) {
+options = qdict_new();
+qdict_put(options, "driver", qstring_from_str(format));
+}
+
+assert(!medium_bs);
+ret = bdrv_open(_bs, filename, NULL, options, bdrv_flags, errp);
+if (ret < 0) {
+goto fail;
+}
+
+medium_bs->detect_zeroes = blk_rs->detect_zeroes;
+if 

[Qemu-block] [PATCH v7 25/39] blockdev: Pull out blockdev option extraction

[Max Reitz]

Extract some of the blockdev option extraction code from blockdev_init()
into its own function. This simplifies blockdev_init() and will allow
reusing the code in a different function added in a follow-up patch.

Signed-off-by: Max Reitz 
---
 blockdev.c | 213 ++---
 1 file changed, 119 insertions(+), 94 deletions(-)

diff --git a/blockdev.c b/blockdev.c
index faa5218..dd4885f 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -350,25 +350,134 @@ static bool check_throttle_config(ThrottleConfig *cfg, 
Error **errp)
 
 typedef enum { MEDIA_DISK, MEDIA_CDROM } DriveMediaType;
 
+/* All parameters but @opts are optional and may be set to NULL. */
+static void extract_common_blockdev_options(QemuOpts *opts, int *bdrv_flags,
+const char **throttling_group, ThrottleConfig *throttle_cfg,
+BlockdevDetectZeroesOptions *detect_zeroes, Error **errp)
+{
+const char *discard;
+Error *local_error = NULL;
+const char *aio;
+
+if (bdrv_flags) {
+if (!qemu_opt_get_bool(opts, "read-only", false)) {
+*bdrv_flags |= BDRV_O_RDWR;
+}
+if (qemu_opt_get_bool(opts, "copy-on-read", false)) {
+*bdrv_flags |= BDRV_O_COPY_ON_READ;
+}
+
+if ((discard = qemu_opt_get(opts, "discard")) != NULL) {
+if (bdrv_parse_discard_flags(discard, bdrv_flags) != 0) {
+error_setg(errp, "Invalid discard option");
+return;
+}
+}
+
+if (qemu_opt_get_bool(opts, BDRV_OPT_CACHE_WB, true)) {
+*bdrv_flags |= BDRV_O_CACHE_WB;
+}
+if (qemu_opt_get_bool(opts, BDRV_OPT_CACHE_DIRECT, false)) {
+*bdrv_flags |= BDRV_O_NOCACHE;
+}
+if (qemu_opt_get_bool(opts, BDRV_OPT_CACHE_NO_FLUSH, false)) {
+*bdrv_flags |= BDRV_O_NO_FLUSH;
+}
+
+if ((aio = qemu_opt_get(opts, "aio")) != NULL) {
+if (!strcmp(aio, "native")) {
+*bdrv_flags |= BDRV_O_NATIVE_AIO;
+} else if (!strcmp(aio, "threads")) {
+/* this is the default */
+} else {
+   error_setg(errp, "invalid aio option");
+   return;
+}
+}
+}
+
+/* disk I/O throttling */
+if (throttling_group) {
+*throttling_group = qemu_opt_get(opts, "throttling.group");
+}
+
+if (throttle_cfg) {
+memset(throttle_cfg, 0, sizeof(*throttle_cfg));
+throttle_cfg->buckets[THROTTLE_BPS_TOTAL].avg =
+qemu_opt_get_number(opts, "throttling.bps-total", 0);
+throttle_cfg->buckets[THROTTLE_BPS_READ].avg  =
+qemu_opt_get_number(opts, "throttling.bps-read", 0);
+throttle_cfg->buckets[THROTTLE_BPS_WRITE].avg =
+qemu_opt_get_number(opts, "throttling.bps-write", 0);
+throttle_cfg->buckets[THROTTLE_OPS_TOTAL].avg =
+qemu_opt_get_number(opts, "throttling.iops-total", 0);
+throttle_cfg->buckets[THROTTLE_OPS_READ].avg =
+qemu_opt_get_number(opts, "throttling.iops-read", 0);
+throttle_cfg->buckets[THROTTLE_OPS_WRITE].avg =
+qemu_opt_get_number(opts, "throttling.iops-write", 0);
+
+throttle_cfg->buckets[THROTTLE_BPS_TOTAL].max =
+qemu_opt_get_number(opts, "throttling.bps-total-max", 0);
+throttle_cfg->buckets[THROTTLE_BPS_READ].max  =
+qemu_opt_get_number(opts, "throttling.bps-read-max", 0);
+throttle_cfg->buckets[THROTTLE_BPS_WRITE].max =
+qemu_opt_get_number(opts, "throttling.bps-write-max", 0);
+throttle_cfg->buckets[THROTTLE_OPS_TOTAL].max =
+qemu_opt_get_number(opts, "throttling.iops-total-max", 0);
+throttle_cfg->buckets[THROTTLE_OPS_READ].max =
+qemu_opt_get_number(opts, "throttling.iops-read-max", 0);
+throttle_cfg->buckets[THROTTLE_OPS_WRITE].max =
+qemu_opt_get_number(opts, "throttling.iops-write-max", 0);
+
+throttle_cfg->op_size =
+qemu_opt_get_number(opts, "throttling.iops-size", 0);
+
+if (!check_throttle_config(throttle_cfg, errp)) {
+return;
+}
+}
+
+if (detect_zeroes) {
+*detect_zeroes =
+qapi_enum_parse(BlockdevDetectZeroesOptions_lookup,
+qemu_opt_get(opts, "detect-zeroes"),
+BLOCKDEV_DETECT_ZEROES_OPTIONS_MAX,
+BLOCKDEV_DETECT_ZEROES_OPTIONS_OFF,
+_error);
+if (local_error) {
+error_propagate(errp, local_error);
+return;
+}
+
+if (bdrv_flags &&
+*detect_zeroes == BLOCKDEV_DETECT_ZEROES_OPTIONS_UNMAP &&
+!(*bdrv_flags & BDRV_O_UNMAP))
+{
+error_setg(errp, "setting detect-zeroes to unmap is not allowed "
+ "without setting discard operation to unmap");
+  

[Qemu-block] [PATCH v7 29/39] blockdev: Add blockdev-close-tray

[Max Reitz]

Signed-off-by: Max Reitz 
---
 blockdev.c   | 23 +++
 qapi/block-core.json | 16 
 qmp-commands.hx  | 35 +++
 3 files changed, 74 insertions(+)

diff --git a/blockdev.c b/blockdev.c
index aa68c36..743e5ca 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2098,6 +2098,29 @@ out:
 }
 }
 
+void qmp_blockdev_close_tray(const char *device, Error **errp)
+{
+BlockBackend *blk;
+
+blk = blk_by_name(device);
+if (!blk) {
+error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
+  "Device '%s' not found", device);
+return;
+}
+
+if (!blk_dev_has_removable_media(blk)) {
+error_setg(errp, "Device '%s' is not removable", device);
+return;
+}
+
+if (!blk_dev_is_tray_open(blk)) {
+return;
+}
+
+blk_dev_change_media_cb(blk, true);
+}
+
 /* throttling disk I/O limits */
 void qmp_block_set_io_throttle(const char *device, int64_t bps, int64_t bps_rd,
int64_t bps_wr,
diff --git a/qapi/block-core.json b/qapi/block-core.json
index b9b4a24..1a51829 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1899,6 +1899,22 @@
   'data': { 'device': 'str',
 '*force': 'bool' } }
 
+##
+# @blockdev-close-tray:
+#
+# Closes a block device's tray. If there is a block driver state tree 
associated
+# with the block device (which is currently ejected), that tree will be loaded
+# as the medium.
+#
+# If the tray was already closed before, this will be a no-op.
+#
+# @device: block device name
+#
+# Since: 2.5
+##
+{ 'command': 'blockdev-close-tray',
+  'data': { 'device': 'str' } }
+
 
 ##
 # @BlockErrorAction
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 595aee2..70241ac 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -3975,6 +3975,41 @@ Example:
 EQMP
 
 {
+.name   = "blockdev-close-tray",
+.args_type  = "device:s",
+.mhandler.cmd_new = qmp_marshal_blockdev_close_tray,
+},
+
+SQMP
+blockdev-close-tray
+---
+
+Closes a block device's tray. If there is a block driver state tree associated
+with the block device (which is currently ejected), that tree will be loaded as
+the medium.
+
+If the tray was already closed before, this will be a no-op.
+
+Arguments:
+
+- "device": block device name (json-string)
+
+Example:
+
+-> { "execute": "blockdev-close-tray",
+ "arguments": { "device": "ide1-cd0" } }
+
+<- { "timestamp": { "seconds": 1418751345,
+"microseconds": 272147 },
+ "event": "DEVICE_TRAY_MOVED",
+ "data": { "device": "ide1-cd0",
+   "tray-open": false } }
+
+<- { "return": {} }
+
+EQMP
+
+{
 .name   = "query-named-block-nodes",
 .args_type  = "",
 .mhandler.cmd_new = qmp_marshal_query_named_block_nodes,
-- 
2.6.1

[Qemu-block] [PATCH v7 32/39] blockdev: Implement eject with basic operations

[Max Reitz]

Implement 'eject' by calling blockdev-open-tray and
blockdev-remove-medium.

Signed-off-by: Max Reitz 
---
 blockdev.c | 11 +--
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/blockdev.c b/blockdev.c
index a4c278f..0481686 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1941,16 +1941,15 @@ out:
 
 void qmp_eject(const char *device, bool has_force, bool force, Error **errp)
 {
-BlockBackend *blk;
+Error *local_err = NULL;
 
-blk = blk_by_name(device);
-if (!blk) {
-error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
-  "Device '%s' not found", device);
+qmp_blockdev_open_tray(device, has_force, force, _err);
+if (local_err) {
+error_propagate(errp, local_err);
 return;
 }
 
-eject_device(blk, force, errp);
+qmp_blockdev_remove_medium(device, errp);
 }
 
 void qmp_block_passwd(bool has_device, const char *device,
-- 
2.6.1

[Qemu-block] [PATCH v7 27/39] block: Add blk_remove_bs()

[Max Reitz]

This function removes the BlockDriverState associated with the given
BlockBackend from that BB and sets the BDS pointer in the BB to NULL.

Signed-off-by: Max Reitz 
---
 block/block-backend.c  | 12 
 include/sysemu/block-backend.h |  1 +
 2 files changed, 13 insertions(+)

diff --git a/block/block-backend.c b/block/block-backend.c
index 19fdaae..eb7409c 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -334,6 +334,18 @@ void blk_hide_on_behalf_of_hmp_drive_del(BlockBackend *blk)
 }
 
 /*
+ * Disassociates the currently associated BlockDriverState from @blk.
+ */
+void blk_remove_bs(BlockBackend *blk)
+{
+blk_update_root_state(blk);
+
+bdrv_unref(blk->bs);
+blk->bs->blk = NULL;
+blk->bs = NULL;
+}
+
+/*
  * Associates a new BlockDriverState with @blk.
  */
 void blk_insert_bs(BlockBackend *blk, BlockDriverState *bs)
diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h
index 9306a52..14a6d32 100644
--- a/include/sysemu/block-backend.h
+++ b/include/sysemu/block-backend.h
@@ -72,6 +72,7 @@ BlockBackend *blk_by_name(const char *name);
 BlockBackend *blk_next(BlockBackend *blk);
 
 BlockDriverState *blk_bs(BlockBackend *blk);
+void blk_remove_bs(BlockBackend *blk);
 void blk_insert_bs(BlockBackend *blk, BlockDriverState *bs);
 
 void blk_hide_on_behalf_of_hmp_drive_del(BlockBackend *blk);
-- 
2.6.1

[Qemu-block] [PATCH 07/17] qcow2: add a 'keyid' parameter to qcow2 options

[Daniel P. Berrange]

Add a 'keyid' parameter that refers to the ID of a
QCryptoSecret instance that provides the encryption key.

$QEMU \
-object secret,id=sec0,filename=/home/berrange/encrypted.pw \
-drive file=/home/berrange/encrypted.qcow2,keyid=sec0

Signed-off-by: Daniel P. Berrange 
---
 block/qcow2.c| 80 +---
 block/qcow2.h|  1 +
 qapi/block-core.json |  8 --
 3 files changed, 64 insertions(+), 25 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index bacc4f2..3b108b0 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -34,6 +34,7 @@
 #include "qapi-event.h"
 #include "trace.h"
 #include "qemu/option_int.h"
+#include "crypto/secret.h"
 
 /*
   Differences with QCOW:
@@ -472,6 +473,11 @@ static QemuOptsList qcow2_runtime_opts = {
 .type = QEMU_OPT_NUMBER,
 .help = "Clean unused cache entries after this time (in seconds)",
 },
+{
+.name = QCOW2_OPT_KEY_ID,
+.type = QEMU_OPT_STRING,
+.help = "ID of the secret that provides the encryption key",
+},
 { /* end of list */ }
 },
 };
@@ -589,6 +595,31 @@ static void read_cache_sizes(BlockDriverState *bs, 
QemuOpts *opts,
 }
 }
 
+static QCryptoCipher *qcow2_get_cipher_from_key(const char *key,
+Error **errp)
+{
+uint8_t keybuf[16];
+int len, i;
+
+memset(keybuf, 0, 16);
+len = strlen(key);
+if (len > 16) {
+len = 16;
+}
+/* XXX: we could compress the chars to 7 bits to increase
+   entropy */
+for (i = 0; i < len; i++) {
+keybuf[i] = key[i];
+}
+
+return qcrypto_cipher_new(
+QCRYPTO_CIPHER_ALG_AES_128,
+QCRYPTO_CIPHER_MODE_CBC,
+keybuf, G_N_ELEMENTS(keybuf),
+errp);
+}
+
+
 typedef struct Qcow2ReopenState {
 Qcow2Cache *l2_table_cache;
 Qcow2Cache *refcount_block_cache;
@@ -596,6 +627,7 @@ typedef struct Qcow2ReopenState {
 int overlap_check;
 bool discard_passthrough[QCOW2_DISCARD_MAX];
 uint64_t cache_clean_interval;
+QCryptoCipher *cipher;
 } Qcow2ReopenState;
 
 static int qcow2_update_options_prepare(BlockDriverState *bs,
@@ -611,6 +643,8 @@ static int qcow2_update_options_prepare(BlockDriverState 
*bs,
 int i;
 Error *local_err = NULL;
 int ret;
+const char *keyid;
+char *key;
 
 opts = qemu_opts_create(_runtime_opts, NULL, 0, _abort);
 qemu_opts_absorb_qdict(opts, options, _err);
@@ -754,6 +788,24 @@ static int qcow2_update_options_prepare(BlockDriverState 
*bs,
 r->discard_passthrough[QCOW2_DISCARD_OTHER] =
 qemu_opt_get_bool(opts, QCOW2_OPT_DISCARD_OTHER, false);
 
+keyid = qemu_opt_get(opts, QCOW2_OPT_KEY_ID);
+if (keyid) {
+key = qcrypto_secret_lookup_as_utf8(keyid,
+errp);
+if (!key) {
+ret = -ENOENT;
+goto fail;
+}
+
+r->cipher = qcow2_get_cipher_from_key(key,
+  errp);
+g_free(key);
+if (!r->cipher) {
+ret = -ENOSYS;
+goto fail;
+}
+}
+
 ret = 0;
 fail:
 qemu_opts_del(opts);
@@ -788,6 +840,9 @@ static void qcow2_update_options_commit(BlockDriverState 
*bs,
 s->cache_clean_interval = r->cache_clean_interval;
 cache_clean_timer_init(bs, bdrv_get_aio_context(bs));
 }
+
+qcrypto_cipher_free(s->cipher);
+s->cipher = r->cipher;
 }
 
 static void qcow2_update_options_abort(BlockDriverState *bs,
@@ -799,6 +854,7 @@ static void qcow2_update_options_abort(BlockDriverState *bs,
 if (r->refcount_block_cache) {
 qcow2_cache_destroy(bs, r->refcount_block_cache);
 }
+qcrypto_cipher_free(r->cipher);
 }
 
 static int qcow2_update_options(BlockDriverState *bs, QDict *options,
@@ -1202,33 +1258,11 @@ static void qcow2_refresh_limits(BlockDriverState *bs, 
Error **errp)
 static int qcow2_set_key(BlockDriverState *bs, const char *key)
 {
 BDRVQcow2State *s = bs->opaque;
-uint8_t keybuf[16];
-int len, i;
-Error *err = NULL;
 
-memset(keybuf, 0, 16);
-len = strlen(key);
-if (len > 16)
-len = 16;
-/* XXX: we could compress the chars to 7 bits to increase
-   entropy */
-for(i = 0;i < len;i++) {
-keybuf[i] = key[i];
-}
 assert(bs->encrypted);
-
 qcrypto_cipher_free(s->cipher);
-s->cipher = qcrypto_cipher_new(
-QCRYPTO_CIPHER_ALG_AES_128,
-QCRYPTO_CIPHER_MODE_CBC,
-keybuf, G_N_ELEMENTS(keybuf),
-);
-
+s->cipher = qcow2_get_cipher_from_key(key, NULL);
 if (!s->cipher) {
-/* XXX would be nice if errors in this method could
- * be properly propagate to the caller. Would need
- * the bdrv_set_key() API signature to be fixed. */
-error_free(err);
 return -1;
 }
 return 0;
diff --git 

Re: [Qemu-block] [Qemu-devel] [PATCH 00/17] Framework for securely passing secrets to QEMU

[Alex Bennée]

Daniel P. Berrange  writes:

> There are a variety of places where QEMU needs to have access
> to passwords, encryption keys or similar kinds of secrets.
>

>
> Example usage for creating secrets...
>
> Direct password, insecure, for ad-hoc developer testing only
>
>   $QEMU -object secret,id=sec0,data=letmein
>
> Indirect password via a file, good for production
>
>   echo -n "letmein" > mypasswd.txt
>   $QEMU -object secret,id=sec0,file=mypasswd.txt
>
> The file based approach supports file descriptor passing,
> so mgmt apps can use that to dynamically add passwords to
> running QEMU.
>
> There is a better way though, which is to use an encrypted
> secret. The intent here is that a mgmt app (like libvirt)
> will generate a random AES-256 key for each virtual machine
> it starts (saved in eg /var/run/libvirt/qemu/$GUEST.key)
> It can then use the direct password passing, but encrypt
> the data.
>
>   $QEMU \
> -object 
> secret,id=secmaster0,file=/var/run/libvirt/qemu/$GUEST.key,format=base64 \
> -object secret,id=sec0,data=[base64 ciphertext],\
>keyid=secmaster0,iv=[base64 initialization vector]
>
> This means that the mgmt app does not need to worry about
> file descriptor passing at all. It can just use regular
> object properties, safe in the knowledge that the data is
> protected by a secret AES key shared only between QEMU
> and the mgmt app.
>
> Use of encrypted secrets is not restricted to directly
> provided inline data. If the secret is stored in an
> external file, that can be encrypted too
>
>   $QEMU \
> -object 
> secret,id=secmaster0,file=/var/run/libvirt/qemu/$GUEST.key,format=base64 \
> -object secret,id=sec0,file=/some/secret/file.aes,\
>keyid=secmaster0,iv=[base64 initialization vector]
>
>
>
> Example usage for referencing secrets...
>
> CURL:
>
>   $QEMU -object secret,id=sec0... \
>  -drive driver=http,url=http://example.com/someimg.qcow2,\
>   user=dan,passwordid=sec0
>
>   $QEMU -object secret,id=sec0... -object secret,id=sec1 \
>  -drive driver=http,url=http://example.com/someimg.qcow2,\
>   user=dan,passwordid=sec0,proxyuser=dan,passwordid=sec1
>
> iSCSI:
>
>   $QEMU -object secret,id=sec0... \
>  -drive driver=iscsi,url=iscsi://example.com/target-foo/lun1,\
>  user=dan,passwordid=sec0
>
> RBD:
>
>   $QEMU -object secret,id=sec0... \
>  -drive driver=rbd,file=rbd:pool/image:id=myname,\
>  auth-supported-cephx,authsecret=sec0
>
> QCow/Qcow2 encryption
>
>   $QEMU -object secret,id=sec0... \
>  -drive file=someimage.qcow2,keyid=sec0

So one use case which I don't see here but do on other programs that
need to access secrets is calling an external program and reading its
stdout. This is simpler than having to mess around with passing FDs
although there may be security concerns having QEMU create new tasks on
the system.


-- 
Alex Bennée

[Qemu-block] [PATCH 15/17] block: rip out all traces of password prompting

[Daniel P. Berrange]

Now that qcow & qcow2 are wired up to get encryption keys
via the QCryptoSecret object, all traces of code which
had to deal with prompting for passwords can be ripped
out.

When the image is initially opened, the encryption key
must be available immediately, or an error will be
reported.

$ qemu-system-x86_64 -drive file=secret.qcow2
qemu-system-x86_64: -drive file=secret.qcow2: Image is encrypted but no secret 
is provided

Users must provide the secret with -object

$ echo 123456 > mypasswd.txt
qemu-system-x86_64 -drive file=secret.qcow2,keysecret=sec0 -object 
secret,file=mypasswd.txt,id=sec0

The BDRV_O_NO_IO flag allows this error to be skipped,
for use when 'qemu-img info' wants to open the file
to query the headers, but not perform any actual I/O
operations.

Signed-off-by: Daniel P. Berrange 
---
 block/qcow.c |  7 +
 block/qcow2.c| 13 +++--
 hmp.c| 31 -
 hw/usb/dev-storage.c | 32 -
 include/block/block.h|  1 +
 include/monitor/monitor.h|  7 -
 include/qemu/osdep.h |  2 --
 monitor.c| 65 ---
 qemu-img.c   | 46 --
 qemu-io.c| 21 --
 qmp.c|  8 --
 tests/qemu-iotests/087   | 20 ++
 tests/qemu-iotests/087.out   |  2 ++
 tests/qemu-iotests/134   | 17 
 tests/qemu-iotests/134.out   |  8 --
 tests/qemu-iotests/common.rc |  4 +--
 util/oslib-posix.c   | 66 
 util/oslib-win32.c   | 24 
 18 files changed, 72 insertions(+), 302 deletions(-)

diff --git a/block/qcow.c b/block/qcow.c
index 719ed7c..ff80ef5 100644
--- a/block/qcow.c
+++ b/block/qcow.c
@@ -231,6 +231,13 @@ static int qcow_open(BlockDriverState *bs, QDict *options, 
int flags,
 if (s->crypt_method_header) {
 bs->encrypted = 1;
 }
+if (!(flags & BDRV_O_NO_IO) &&
+bs->encrypted && !s->cipher) {
+error_setg(errp, "Image is encrypted but no secret is provided");
+ret = -EINVAL;
+goto fail;
+}
+
 s->cluster_bits = header.cluster_bits;
 s->cluster_size = 1 << s->cluster_bits;
 s->cluster_sectors = 1 << (s->cluster_bits - 9);
diff --git a/block/qcow2.c b/block/qcow2.c
index 3b108b0..6ca2b25 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -1136,6 +1136,13 @@ static int qcow2_open(BlockDriverState *bs, QDict 
*options, int flags,
 goto fail;
 }
 
+if (!(flags & BDRV_O_NO_IO) &&
+bs->encrypted && !s->cipher) {
+error_setg(errp, "Image is encrypted but no secret is provided");
+ret = -EINVAL;
+goto fail;
+}
+
 s->cluster_cache = g_malloc(s->cluster_size);
 /* one more sector for decompressed data alignment */
 s->cluster_data = qemu_try_blockalign(bs->file->bs, QCOW_MAX_CRYPT_CLUSTERS
@@ -2207,7 +2214,8 @@ static int qcow2_create2(const char *filename, int64_t 
total_size,
 options = qdict_new();
 qdict_put(options, "driver", qstring_from_str("qcow2"));
 ret = bdrv_open(, filename, NULL, options,
-BDRV_O_RDWR | BDRV_O_CACHE_WB | BDRV_O_NO_FLUSH,
+BDRV_O_RDWR | BDRV_O_CACHE_WB | BDRV_O_NO_FLUSH |
+BDRV_O_NO_IO,
 _err);
 if (ret < 0) {
 error_propagate(errp, local_err);
@@ -2261,7 +2269,8 @@ static int qcow2_create2(const char *filename, int64_t 
total_size,
 options = qdict_new();
 qdict_put(options, "driver", qstring_from_str("qcow2"));
 ret = bdrv_open(, filename, NULL, options,
-BDRV_O_RDWR | BDRV_O_CACHE_WB | BDRV_O_NO_BACKING,
+BDRV_O_RDWR | BDRV_O_CACHE_WB | BDRV_O_NO_BACKING |
+BDRV_O_NO_IO,
 _err);
 if (local_err) {
 error_propagate(errp, local_err);
diff --git a/hmp.c b/hmp.c
index 409d05d..8036642 100644
--- a/hmp.c
+++ b/hmp.c
@@ -959,37 +959,12 @@ void hmp_ringbuf_read(Monitor *mon, const QDict *qdict)
 g_free(data);
 }
 
-static void hmp_cont_cb(void *opaque, int err)
-{
-if (!err) {
-qmp_cont(NULL);
-}
-}
-
-static bool key_is_missing(const BlockInfo *bdev)
-{
-return (bdev->inserted && bdev->inserted->encryption_key_missing);
-}
-
 void hmp_cont(Monitor *mon, const QDict *qdict)
 {
-BlockInfoList *bdev_list, *bdev;
 Error *err = NULL;
 
-bdev_list = qmp_query_block(NULL);
-for (bdev = bdev_list; bdev; bdev = bdev->next) {
-if (key_is_missing(bdev->value)) {
-monitor_read_block_device_key(mon, bdev->value->device,
-  hmp_cont_cb, NULL);
-goto out;
-}
-}
-
 qmp_cont();
 hmp_handle_error(mon, );
-
-out:
-qapi_free_BlockInfoList(bdev_list);
 }
 
 void 

[Qemu-block] [PATCH v7 37/39] blockdev: read-only-mode for blockdev-change-medium

[Max Reitz]

Add an option to qmp_blockdev_change_medium() which allows changing the
read-only status of the block device whose medium is changed.

Some drives do not have a inherently fixed read-only status; for
instance, floppy disks can be set read-only or writable independently of
the drive. Some users may find it useful to be able to therefore change
the read-only status of a block device when changing the medium.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
---
 blockdev.c   | 25 -
 hmp.c|  2 +-
 qapi/block-core.json | 24 +++-
 qmp-commands.hx  | 24 +++-
 qmp.c|  3 ++-
 5 files changed, 73 insertions(+), 5 deletions(-)

diff --git a/blockdev.c b/blockdev.c
index db789e2..b79b0a6 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2096,6 +2096,8 @@ void qmp_blockdev_insert_medium(const char *device, const 
char *node_name,
 
 void qmp_blockdev_change_medium(const char *device, const char *filename,
 bool has_format, const char *format,
+bool has_read_only,
+BlockdevChangeReadOnlyMode read_only,
 Error **errp)
 {
 BlockBackend *blk;
@@ -2117,7 +2119,28 @@ void qmp_blockdev_change_medium(const char *device, 
const char *filename,
 }
 
 blk_rs = blk_get_root_state(blk);
-bdrv_flags = blk_rs->read_only ? 0 : BDRV_O_RDWR;
+
+if (!has_read_only) {
+read_only = BLOCKDEV_CHANGE_READ_ONLY_MODE_RETAIN;
+}
+
+switch (read_only) {
+case BLOCKDEV_CHANGE_READ_ONLY_MODE_RETAIN:
+bdrv_flags = blk_rs->read_only ? 0 : BDRV_O_RDWR;
+break;
+
+case BLOCKDEV_CHANGE_READ_ONLY_MODE_READ_ONLY:
+bdrv_flags = 0;
+break;
+
+case BLOCKDEV_CHANGE_READ_ONLY_MODE_READ_WRITE:
+bdrv_flags = BDRV_O_RDWR;
+break;
+
+default:
+abort();
+}
+
 bdrv_flags |= blk_rs->open_flags & ~BDRV_O_RDWR;
 
 if (has_format) {
diff --git a/hmp.c b/hmp.c
index b91821b..9e6b7e5 100644
--- a/hmp.c
+++ b/hmp.c
@@ -1348,7 +1348,7 @@ void hmp_change(Monitor *mon, const QDict *qdict)
 }
 qmp_change("vnc", target, !!arg, arg, );
 } else {
-qmp_blockdev_change_medium(device, target, !!arg, arg, );
+qmp_blockdev_change_medium(device, target, !!arg, arg, false, 0, );
 if (err &&
 error_get_class(err) == ERROR_CLASS_DEVICE_ENCRYPTED) {
 error_free(err);
diff --git a/qapi/block-core.json b/qapi/block-core.json
index b8cc18a..5f12af7 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1949,6 +1949,24 @@
 
 
 ##
+# @BlockdevChangeReadOnlyMode:
+#
+# Specifies the new read-only mode of a block device subject to the
+# @blockdev-change-medium command.
+#
+# @retain:  Retains the current read-only mode
+#
+# @read-only:   Makes the device read-only
+#
+# @read-write:  Makes the device writable
+#
+# Since: 2.3
+##
+{ 'enum': 'BlockdevChangeReadOnlyMode',
+  'data': ['retain', 'read-only', 'read-write'] }
+
+
+##
 # @blockdev-change-medium:
 #
 # Changes the medium inserted into a block device by ejecting the current 
medium
@@ -1963,12 +1981,16 @@
 # @format:  #optional, format to open the new image with (defaults to
 #   the probed format)
 #
+# @read-only-mode:  #optional, change the read-only mode of the device; 
defaults
+#   to 'retain'
+#
 # Since: 2.5
 ##
 { 'command': 'blockdev-change-medium',
   'data': { 'device': 'str',
 'filename': 'str',
-'*format': 'str' } }
+'*format': 'str',
+'*read-only-mode': 'BlockdevChangeReadOnlyMode' } }
 
 
 ##
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 8643d95..ed6f4ad 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -4155,7 +4155,7 @@ EQMP
 
 {
 .name   = "blockdev-change-medium",
-.args_type  = "device:B,filename:F,format:s?",
+.args_type  = "device:B,filename:F,format:s?,read-only-mode:s?",
 .mhandler.cmd_new = qmp_marshal_blockdev_change_medium,
 },
 
@@ -4171,6 +4171,8 @@ Arguments:
 - "device": device name (json-string)
 - "filename": filename of the new image (json-string)
 - "format": format of the new image (json-string, optional)
+- "read-only-mode": new read-only mode (json-string, optional)
+  - Possible values: "retain" (default), "read-only", "read-write"
 
 Examples:
 
@@ -4182,6 +4184,26 @@ Examples:
 "format": "raw" } }
 <- { "return": {} }
 
+2. Load a read-only medium into a writable drive
+
+-> { "execute": "blockdev-change-medium",
+ "arguments": { "device": "isa-fd0",
+"filename": "/srv/images/ro.img",
+"format": "raw",
+"read-only-mode": "retain" } }
+
+<- { "error":
+ { 

[Qemu-block] [PATCH v5 1/6] block: Change bdrv_get_encrypted_filename()

[Max Reitz]

Instead of returning a pointer to the filename, g_strdup() it. This will
become necessary once we do not have BlockDriverState.filename anymore.

Signed-off-by: Max Reitz 
---
 block.c   | 17 ++---
 include/block/block.h |  2 +-
 monitor.c |  5 -
 3 files changed, 15 insertions(+), 9 deletions(-)

diff --git a/block.c b/block.c
index f4dec11..60f9450 100644
--- a/block.c
+++ b/block.c
@@ -2585,10 +2585,12 @@ void bdrv_add_key(BlockDriverState *bs, const char 
*key, Error **errp)
 }
 } else {
 if (bdrv_key_required(bs)) {
+char *enc_filename = bdrv_get_encrypted_filename(bs);
 error_set(errp, ERROR_CLASS_DEVICE_ENCRYPTED,
   "'%s' (%s) is encrypted",
   bdrv_get_device_or_node_name(bs),
-  bdrv_get_encrypted_filename(bs));
+  enc_filename ?: "");
+g_free(enc_filename);
 }
 }
 }
@@ -2810,14 +2812,15 @@ bool bdrv_can_write_zeroes_with_unmap(BlockDriverState 
*bs)
 return false;
 }
 
-const char *bdrv_get_encrypted_filename(BlockDriverState *bs)
+char *bdrv_get_encrypted_filename(BlockDriverState *bs)
 {
-if (bs->backing && bs->backing->bs->encrypted)
-return bs->backing_file;
-else if (bs->encrypted)
-return bs->filename;
-else
+if (bs->backing && bs->backing->bs->encrypted) {
+return g_strdup(bs->backing_file);
+} else if (bs->encrypted) {
+return g_strdup(bs->filename);
+} else {
 return NULL;
+}
 }
 
 void bdrv_get_backing_filename(BlockDriverState *bs,
diff --git a/include/block/block.h b/include/block/block.h
index 28d903c..340c00f 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -421,7 +421,7 @@ void bdrv_round_to_clusters(BlockDriverState *bs,
 int64_t *cluster_sector_num,
 int *cluster_nb_sectors);
 
-const char *bdrv_get_encrypted_filename(BlockDriverState *bs);
+char *bdrv_get_encrypted_filename(BlockDriverState *bs);
 void bdrv_get_backing_filename(BlockDriverState *bs,
char *filename, int filename_size);
 void bdrv_get_full_backing_filename(BlockDriverState *bs,
diff --git a/monitor.c b/monitor.c
index 301a143..63b884c 100644
--- a/monitor.c
+++ b/monitor.c
@@ -4117,10 +4117,13 @@ int monitor_read_bdrv_key_start(Monitor *mon, 
BlockDriverState *bs,
 BlockCompletionFunc *completion_cb,
 void *opaque)
 {
+char *enc_filename;
 int err;
 
+enc_filename = bdrv_get_encrypted_filename(bs);
 monitor_printf(mon, "%s (%s) is encrypted.\n", bdrv_get_device_name(bs),
-   bdrv_get_encrypted_filename(bs));
+   enc_filename ?: "");
+g_free(enc_filename);
 
 mon->password_completion_cb = completion_cb;
 mon->password_opaque = opaque;
-- 
2.6.1

[Qemu-block] [PATCH v5 0/6] block: Drop BDS.filename

[Max Reitz]

*** This series is based on v7 of my ***
*** "blockdev: BlockBackend and media" series ***


The BDS filename field is generally only used when opening disk images
or emitting error or warning messages, the only exception to this rule
is the map command of qemu-img. However, using exact_filename there
instead should not be a problem. Therefore, we can drop the filename
field from the BlockDriverState and use a function instead which builds
the filename from scratch when called.

This is slower than reading a static char array but the problem of that
static array is that it may become obsolete due to changes in any
BlockDriverState or in the BDS graph. Using a function which rebuilds
the filename every time it is called resolves this problem.

The disadvantage of worse performance is negligible, on the other hand.
After patch 2 of this series, which replaces some queries of
BDS.filename by reads from somewhere else (mostly BDS.exact_filename),
the filename field is only used when a disk image is opened or some
message should be emitted, both of which cases do not suffer from the
performance hit.


http://lists.nongnu.org/archive/html/qemu-devel/2015-06/msg07025.html
gives an example of how to achieve an outdated filename, so we do need
this series. We can technically fix it differently (by appropriately
invoking bdrv_refresh_filename()), but that is pretty difficult™. I find
this series simpler and it it's something we wanted to do anyway.

Regarding the fear that this might change current behavior, especially
for libvirt which used filenames to identify nodes in the BDS graph:
Since bdrv_open() already calls bdrv_refresh_filename() today, and
apparently everything works fine, this series will most likely not break
anything. The only thing that will change is if the BDS graph is
dynamically reconfigured at runtime, and in that case it's most likely a
bug fix. Also, I don't think libvirt supports such cases already.


tl;dr: This series is a bug fix and I don't think it'll break legacy
management applications relying on the filename to identify a BDS; as
long as they are not trying to continue that practice with more advanced
configurations (e.g. with Quorum).


v5: (Rebased and addressed Eric's comments)
- Patch 1: Do not pstrcpy() into a caller-supplied buffer, but
  g_strdup() instead [Eric]
- Patch 2: Rebase conflicts
- Patch 3: Drop bdrv_filename_alloc(), instead bdrv_filename() will
  allocate a buffer if NULL is specified
- Patch 4: Use bdrv_filename() instead of bdrv_filename_alloc()
- Patch 5:
  - Rebase conflicts
  - Use bdrv_filename() instead of bdrv_filename_alloc()


git-backport-diff against v4:

Key:
[] : patches are identical
[] : number of functional differences between upstream/downstream patch
[down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/6:[0029] [FC] 'block: Change bdrv_get_encrypted_filename()'
002/6:[0004] [FC] 'block: Avoid BlockDriverState.filename'
003/6:[0040] [FC] 'block: Add bdrv_filename()'
004/6:[0002] [FC] 'qemu-img: Use bdrv_filename() for map'
005/6:[0093] [FC] 'block: Drop BlockDriverState.filename'
006/6:[] [--] 'iotests: Test changed Quorum filename'


Max Reitz (6):
  block: Change bdrv_get_encrypted_filename()
  block: Avoid BlockDriverState.filename
  block: Add bdrv_filename()
  qemu-img: Use bdrv_filename() for map
  block: Drop BlockDriverState.filename
  iotests: Test changed Quorum filename

 block.c   | 105 +-
 block/commit.c|   4 +-
 block/gluster.c   |   2 +-
 block/mirror.c|  16 +--
 block/qapi.c  |   4 +-
 block/raw-posix.c |   8 ++--
 block/raw-win32.c |   4 +-
 block/raw_bsd.c   |   4 +-
 block/vhdx-log.c  |   5 ++-
 block/vmdk.c  |  22 +++---
 block/vpc.c   |   7 +++-
 blockdev.c|  25 ---
 include/block/block.h |   3 +-
 include/block/block_int.h |   1 -
 monitor.c |   5 ++-
 qemu-img.c|  14 ++-
 tests/qemu-iotests/041|  17 ++--
 17 files changed, 179 insertions(+), 67 deletions(-)

-- 
2.6.1

[Qemu-block] [PATCH v5 2/6] block: Avoid BlockDriverState.filename

[Max Reitz]

In places which directly pass a filename to the OS, we should not use
the filename field at all but exact_filename instead (although the
former currently equals the latter if that is set).

In raw_open_common(), we do not need to access BDS.filename because we
already have a local variable pointing to the filename.

Signed-off-by: Max Reitz 
---
 block.c   | 5 +++--
 block/gluster.c   | 2 +-
 block/raw-posix.c | 8 
 block/raw-win32.c | 4 ++--
 4 files changed, 10 insertions(+), 9 deletions(-)

diff --git a/block.c b/block.c
index 60f9450..0f08d7c 100644
--- a/block.c
+++ b/block.c
@@ -913,8 +913,9 @@ static int bdrv_open_common(BlockDriverState *bs, BdrvChild 
*file,
 if (ret < 0) {
 if (local_err) {
 error_propagate(errp, local_err);
-} else if (bs->filename[0]) {
-error_setg_errno(errp, -ret, "Could not open '%s'", bs->filename);
+} else if (bs->exact_filename[0]) {
+error_setg_errno(errp, -ret, "Could not open '%s'",
+ bs->exact_filename);
 } else {
 error_setg_errno(errp, -ret, "Could not open image");
 }
diff --git a/block/gluster.c b/block/gluster.c
index 1eb3a8c..176682b 100644
--- a/block/gluster.c
+++ b/block/gluster.c
@@ -358,7 +358,7 @@ static int qemu_gluster_reopen_prepare(BDRVReopenState 
*state,
 
 gconf = g_new0(GlusterConf, 1);
 
-reop_s->glfs = qemu_gluster_init(gconf, state->bs->filename, errp);
+reop_s->glfs = qemu_gluster_init(gconf, state->bs->exact_filename, errp);
 if (reop_s->glfs == NULL) {
 ret = -errno;
 goto exit;
diff --git a/block/raw-posix.c b/block/raw-posix.c
index 918c756..9cb7ebd 100644
--- a/block/raw-posix.c
+++ b/block/raw-posix.c
@@ -505,7 +505,7 @@ static int raw_open_common(BlockDriverState *bs, QDict 
*options,
  "specified. Falling back to aio=threads.\n"
  " This will become an error condition in "
  "future QEMU versions.\n",
- bs->filename);
+ filename);
 }
 #else
 if (bdrv_flags & BDRV_O_NATIVE_AIO) {
@@ -667,7 +667,7 @@ static int raw_reopen_prepare(BDRVReopenState *state,
 
 /* If we cannot use fcntl, or fcntl failed, fall back to qemu_open() */
 if (raw_s->fd == -1) {
-const char *normalized_filename = state->bs->filename;
+const char *normalized_filename = state->bs->exact_filename;
 ret = raw_normalize_devicepath(_filename);
 if (ret < 0) {
 error_setg_errno(errp, -ret, "Could not normalize device path");
@@ -2099,7 +2099,7 @@ static bool hdev_is_sg(BlockDriverState *bs)
 struct sg_scsi_id scsiid;
 int sg_version;
 
-if (stat(bs->filename, ) >= 0 && S_ISCHR(st.st_mode) &&
+if (stat(bs->exact_filename, ) >= 0 && S_ISCHR(st.st_mode) &&
 !bdrv_ioctl(bs, SG_GET_VERSION_NUM, _version) &&
 !bdrv_ioctl(bs, SG_GET_SCSI_ID, )) {
 DPRINTF("SG device found: type=%d, version=%d\n",
@@ -2518,7 +2518,7 @@ static int cdrom_reopen(BlockDriverState *bs)
  */
 if (s->fd >= 0)
 qemu_close(s->fd);
-fd = qemu_open(bs->filename, s->open_flags, 0644);
+fd = qemu_open(bs->exact_filename, s->open_flags, 0644);
 if (fd < 0) {
 s->fd = -1;
 return -EIO;
diff --git a/block/raw-win32.c b/block/raw-win32.c
index 2d0907a..e2bbe24 100644
--- a/block/raw-win32.c
+++ b/block/raw-win32.c
@@ -417,7 +417,7 @@ static void raw_close(BlockDriverState *bs)
 
 CloseHandle(s->hfile);
 if (bs->open_flags & BDRV_O_TEMPORARY) {
-unlink(bs->filename);
+unlink(bs->exact_filename);
 }
 }
 
@@ -485,7 +485,7 @@ static int64_t raw_get_allocated_file_size(BlockDriverState 
*bs)
   DWORD * high);
 get_compressed_t get_compressed;
 struct _stati64 st;
-const char *filename = bs->filename;
+const char *filename = bs->exact_filename;
 /* WinNT support GetCompressedFileSize to determine allocate size */
 get_compressed =
 (get_compressed_t) GetProcAddress(GetModuleHandle("kernel32"),
-- 
2.6.1

Re: [Qemu-block] [PATCH v5 6/6] iotests: Test changed Quorum filename

[Eric Blake]

On 10/19/2015 12:49 PM, Max Reitz wrote:
> After drive-mirror replacing a Quorum child, the filename of the Quorum
> BDS should reflect the change. This patch replaces the existing test for
> whether the operation did actually exchange the BDS (which simply tested
> whether the new BDS existed) by a test which examines the children list
> contained in the Quorum BDS's filename as returned by query-block.
> 
> As a nice side effect of confirming that the new BDS actually belongs to
> the Quorum BDS, this checks whether the filename was properly updated.
> 
> Signed-off-by: Max Reitz 
> ---
>  tests/qemu-iotests/041 | 17 +
>  1 file changed, 13 insertions(+), 4 deletions(-)
> 

Reviewed-by: Eric Blake 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-block] [PATCH v5 3/6] block: Add bdrv_filename()

[Eric Blake]

On 10/19/2015 12:49 PM, Max Reitz wrote:
> Split the part which actually refreshes the BlockDriverState.filename
> field off of bdrv_refresh_filename() into a more generic function
> bdrv_filename(), which first calls bdrv_refresh_filename() and then
> stores a qemu-usable filename in the given buffer instead of
> BlockDriverState.filename.
> 
> Since bdrv_refresh_filename() therefore no longer refreshes that field,
> some calls to that function have to be replaced by calls to
> bdrv_filename() "manually" refreshing the BDS filename field (this is
> only temporary).
> 
> Signed-off-by: Max Reitz 
> ---
>  block.c   | 43 ---
>  block/blkverify.c |  3 ++-
>  block/quorum.c|  3 ++-
>  include/block/block.h |  1 +
>  4 files changed, 41 insertions(+), 9 deletions(-)
> 

> +
> +/* First refreshes exact_filename and full_open_options by calling
> + * bdrv_refresh_filename(). Then, if exact_filename is set, it is copied into
> + * the target buffer. Otherwise, full_open_options is converted to a JSON
> + * object, prefixed with "json:" (for use through the JSON pseudo protocol) 
> and
> + * put there.
> + *
> + * If @sz > 0, the string put into the buffer will always be null-terminated.

Maybe add a warning:

...always be null-terminated, even if this truncates the result.

Reviewed-by: Eric Blake 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-block] [Qemu-devel] Question about xen disk unplug support for ahci missed in qemu

[Laszlo Ersek]

On 10/16/15 21:09, Laszlo Ersek wrote:
> On 10/16/15 13:34, Fabio Fantoni wrote:
>> Il 16/10/2015 12:47, Stefano Stabellini ha scritto:
>>> On Fri, 16 Oct 2015, Fabio Fantoni wrote:
 Il 16/10/2015 12:13, Anthony PERARD ha scritto:
> On Fri, Oct 16, 2015 at 10:32:44AM +0200, Fabio Fantoni wrote:
>> Il 15/10/2015 20:02, Anthony PERARD ha scritto:
>>> On Thu, Oct 15, 2015 at 06:27:17PM +0200, Fabio Fantoni wrote:
 Il 14/10/2015 13:06, Stefano Stabellini ha scritto:
> I would suggest Fabio to avoid AHCI disks altogether and just use
> OVMF
> with PV disks only and Anthony's patch to libxl to avoid creating
> any
> IDE disks: http://marc.info/?l=xen-devel=144482080812353.
>
> Would that work for you?
 Thanks for the advice, I tried it:
 https://github.com/Fantu/Xen/commits/rebase/m2r-testing-4.6

 I installed W10 pro 64 bit with ide disk, installed the win pv
 drivers
 and
 after changed to xvdX instead hdX, is the only change needed, right?
 Initial boot is ok (ovmf part about pv disks seems ok) but windows
 boot
 fails with problem with pv drivers.
 In attachment full qemu log with xen_platform trace and domU's xl
 cfg.

 Someone have windows domUs with ovmf and pv disks only working?
 If yes
 can
 tell me the difference to understand what can be the problem please?
>>> When I worked on the PV disk implementation in OVMF, I was able to
>>> boot
>>> a Windows 8 with pv disk only.
>>>
>>> I don't have access to the guest configuration I was using, but I
>>> think
>>> one
>>> difference would be the viridian setting, I'm pretty sure I did
>>> not set
>>> it.
>>>
>> I tried with viridian disabled but did the same thing, looking
>> cdrom as
>> latest thing before xenbug trace in qemu log I tried also to remove
>> it but
>> also in this case don't boot correctly, full qemu log in attachment.
>> I don't know if is a ovmf thing to improve (like what seems in
>> Laszlo and
>> Kevin mails) or xen winpv drivers unexpected case, have you tried also
>> with
>> latest winpv builds? (for exclude regression)
> No, I did not tried the latest winpv drivers.
>
> Sorry I can help much more that that. When I install this win8 guest
> tried
> to boot it with pv drivers only, that was more than a year ago. I
> have not
> check if it's still working. (Also I can not try anything more recent,
> right now.)
>
 I did many other tests, retrying with ide first boot working but show pv
 devices not working, I did another reboot (with ide) and pv devices was
 working, after I retried with pv (xvdX) and boot correctly.
 After other tests I found that with empty cdrom device (required for xl
 cd-insert/cd-eject) boot stop at start (tianocore image), same result
 with ide
 instead.
  From xl cfg:
 disk=['/mnt/vm/disks/W10UEFI.disk1.cow-sn1,qcow2,xvda,rw',',raw,xvdb,ro,cdrom']

 With seabios domU boot also with empty cdrom.
 In qemu log I found only these that can be related:
> xen be: qdisk-51728: error: Could not open image: No such file or
> directory
> xen be: qdisk-51728: initialise() failed
 And latest xl dmesg line is:
> (d1) Invoking OVMF ...
 If you need more informations/test tell me and I'll post them.
>>> Are you saying that without any cdrom drives, it works correctly?
>> Yes, I did also another test to be sure, starting with ide, installing
>> windows, the pv drivers, rebooting 2 times (with one at boot of time
>> boot with ide only and without net and disks pv drivers working) and
>> after rebooting with pv disks (xvdX) works.
>> With cdrom not empty (with iso) works, with empty not, tried with both
>> ide (hdX) and pv (xvdX).
>> Empty cdrom not working with ovmf I suppose is ovmf bug or inexpected case.
>> About major of winpv drivers problem at boot I suppose can be solved
>> improving ovmf and winpv driver removing bad hybrid thing actually, but
>> I have too low knowledge to be sure.
>> About the problem of pv start after install that requiring at least 2
>> reboot can be also a windows 10 problem (only a suppose).
>>
>> About empty cdrom with ovmf can be solved please?
>>
> 
> Sorry, I find your problem report impenetrable. :( Please slow down and
> try to spend time on punctuation at least.
> 
> For me to make heads or tails of this, I'll need the following:
> 
> - The debug output of an OVMF binary built with the DEBUG_VERBOSE bit
> (0x0040) enabled in PcdDebugPrintErrorLevel, in addition to the
> default setting.
> 
> - Preferably, I'll need two logs, one for the "working" case, and
> another for the "non-working" case.
> 
> - A description of the virtual hardware (disks etc) that is
> understandable to someone who 

Re: [Qemu-block] [PATCH v5 1/6] block: Change bdrv_get_encrypted_filename()

[Eric Blake]

On 10/19/2015 12:49 PM, Max Reitz wrote:
> Instead of returning a pointer to the filename, g_strdup() it. This will
> become necessary once we do not have BlockDriverState.filename anymore.
> 
> Signed-off-by: Max Reitz 
> ---
>  block.c   | 17 ++---
>  include/block/block.h |  2 +-
>  monitor.c |  5 -
>  3 files changed, 15 insertions(+), 9 deletions(-)

Reviewed-by: Eric Blake 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-block] [PATCH v7 22/39] block: Add blk_insert_bs()

[Max Reitz]

This function associates the given BlockDriverState with the given
BlockBackend.

Signed-off-by: Max Reitz 
Reviewed-by: Kevin Wolf 
---
 block/block-backend.c  | 11 +++
 include/sysemu/block-backend.h |  1 +
 2 files changed, 12 insertions(+)

diff --git a/block/block-backend.c b/block/block-backend.c
index a5c58c5..19fdaae 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -334,6 +334,17 @@ void blk_hide_on_behalf_of_hmp_drive_del(BlockBackend *blk)
 }
 
 /*
+ * Associates a new BlockDriverState with @blk.
+ */
+void blk_insert_bs(BlockBackend *blk, BlockDriverState *bs)
+{
+assert(!blk->bs && !bs->blk);
+bdrv_ref(bs);
+blk->bs = bs;
+bs->blk = blk;
+}
+
+/*
  * Attach device model @dev to @blk.
  * Return 0 on success, -EBUSY when a device model is attached already.
  */
diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h
index 52e35a1..9306a52 100644
--- a/include/sysemu/block-backend.h
+++ b/include/sysemu/block-backend.h
@@ -72,6 +72,7 @@ BlockBackend *blk_by_name(const char *name);
 BlockBackend *blk_next(BlockBackend *blk);
 
 BlockDriverState *blk_bs(BlockBackend *blk);
+void blk_insert_bs(BlockBackend *blk, BlockDriverState *bs);
 
 void blk_hide_on_behalf_of_hmp_drive_del(BlockBackend *blk);
 
-- 
2.6.1

[Qemu-block] [PATCH v5 6/6] iotests: Test changed Quorum filename

[Max Reitz]

After drive-mirror replacing a Quorum child, the filename of the Quorum
BDS should reflect the change. This patch replaces the existing test for
whether the operation did actually exchange the BDS (which simply tested
whether the new BDS existed) by a test which examines the children list
contained in the Quorum BDS's filename as returned by query-block.

As a nice side effect of confirming that the new BDS actually belongs to
the Quorum BDS, this checks whether the filename was properly updated.

Signed-off-by: Max Reitz 
---
 tests/qemu-iotests/041 | 17 +
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041
index 05b5962..6703882 100755
--- a/tests/qemu-iotests/041
+++ b/tests/qemu-iotests/041
@@ -18,6 +18,7 @@
 # along with this program.  If not, see .
 #
 
+import json
 import time
 import os
 import iotests
@@ -920,10 +921,18 @@ class TestRepairQuorum(iotests.QMPTestCase):
 self.assert_qmp(result, 'return', {})
 
 self.complete_and_wait(drive="quorum0")
-result = self.vm.qmp('query-named-block-nodes')
-self.assert_qmp(result, 'return[0]/file', quorum_repair_img)
-# TODO: a better test requiring some QEMU infrastructure will be added
-#   to check that this file is really driven by quorum
+
+result = self.vm.qmp('query-block')
+for blockdev in result['return']:
+if blockdev['device'] == 'quorum0':
+filename = blockdev['inserted']['image']['filename']
+self.assertEquals(filename[:5], 'json:')
+children = json.loads(filename[5:])['children']
+self.assertEquals(children[0]['file']['filename'], quorum_img1)
+self.assertEquals(children[1]['file']['filename'],
+  quorum_repair_img)
+self.assertEquals(children[2]['file']['filename'], quorum_img3)
+
 self.vm.shutdown()
 
 if __name__ == '__main__':
-- 
2.6.1

[Qemu-block] [PATCH v5 3/6] block: Add bdrv_filename()

[Max Reitz]

Split the part which actually refreshes the BlockDriverState.filename
field off of bdrv_refresh_filename() into a more generic function
bdrv_filename(), which first calls bdrv_refresh_filename() and then
stores a qemu-usable filename in the given buffer instead of
BlockDriverState.filename.

Since bdrv_refresh_filename() therefore no longer refreshes that field,
some calls to that function have to be replaced by calls to
bdrv_filename() "manually" refreshing the BDS filename field (this is
only temporary).

Signed-off-by: Max Reitz 
---
 block.c   | 43 ---
 block/blkverify.c |  3 ++-
 block/quorum.c|  3 ++-
 include/block/block.h |  1 +
 4 files changed, 41 insertions(+), 9 deletions(-)

diff --git a/block.c b/block.c
index 0f08d7c..1061d80 100644
--- a/block.c
+++ b/block.c
@@ -1527,7 +1527,7 @@ static int bdrv_open_inherit(BlockDriverState **pbs, 
const char *filename,
 }
 }
 
-bdrv_refresh_filename(bs);
+bdrv_filename(bs, bs->filename, sizeof(bs->filename));
 
 /* Check if any unknown options were used */
 if (options && (qdict_size(options) != 0)) {
@@ -3937,9 +3937,6 @@ static bool append_open_options(QDict *d, 
BlockDriverState *bs)
  *  - full_open_options: Options which, when given when opening a block device
  *   (without a filename), result in a BDS (mostly)
  *   equalling the given one
- *  - filename: If exact_filename is set, it is copied here. Otherwise,
- *  full_open_options is converted to a JSON object, prefixed with
- *  "json:" (for use through the JSON pseudo protocol) and put 
here.
  */
 void bdrv_refresh_filename(BlockDriverState *bs)
 {
@@ -4027,13 +4024,45 @@ void bdrv_refresh_filename(BlockDriverState *bs)
 
 bs->full_open_options = opts;
 }
+}
+
+/* First refreshes exact_filename and full_open_options by calling
+ * bdrv_refresh_filename(). Then, if exact_filename is set, it is copied into
+ * the target buffer. Otherwise, full_open_options is converted to a JSON
+ * object, prefixed with "json:" (for use through the JSON pseudo protocol) and
+ * put there.
+ *
+ * If @sz > 0, the string put into the buffer will always be null-terminated.
+ *
+ * If @dest is NULL, @sz is ignored and a new buffer will be allocated which is
+ * large enough to hold the filename and the trailing '\0'. This buffer is then
+ * returned and has to be freed by the caller when it is no longer needed.
+ *
+ * Returns @dest if it is not NULL, and the newly allocated buffer otherwise.
+ */
+char *bdrv_filename(BlockDriverState *bs, char *dest, size_t sz)
+{
+bdrv_refresh_filename(bs);
+
+if (sz > INT_MAX) {
+sz = INT_MAX;
+}
 
 if (bs->exact_filename[0]) {
-pstrcpy(bs->filename, sizeof(bs->filename), bs->exact_filename);
+if (dest) {
+pstrcpy(dest, sz, bs->exact_filename);
+} else {
+dest = g_strdup(bs->exact_filename);
+}
 } else if (bs->full_open_options) {
 QString *json = qobject_to_json(QOBJECT(bs->full_open_options));
-snprintf(bs->filename, sizeof(bs->filename), "json:%s",
- qstring_get_str(json));
+if (dest) {
+snprintf(dest, sz, "json:%s", qstring_get_str(json));
+} else {
+dest = g_strdup_printf("json:%s", qstring_get_str(json));
+}
 QDECREF(json);
 }
+
+return dest;
 }
diff --git a/block/blkverify.c b/block/blkverify.c
index c5f8e8d..3af37f8 100644
--- a/block/blkverify.c
+++ b/block/blkverify.c
@@ -312,7 +312,8 @@ static void blkverify_refresh_filename(BlockDriverState *bs)
 BDRVBlkverifyState *s = bs->opaque;
 
 /* bs->file->bs has already been refreshed */
-bdrv_refresh_filename(s->test_file->bs);
+bdrv_filename(s->test_file->bs, s->test_file->bs->filename,
+  sizeof(s->test_file->bs->filename));
 
 if (bs->file->bs->full_open_options
 && s->test_file->bs->full_open_options)
diff --git a/block/quorum.c b/block/quorum.c
index b9ba028..16ec188 100644
--- a/block/quorum.c
+++ b/block/quorum.c
@@ -1005,7 +1005,8 @@ static void quorum_refresh_filename(BlockDriverState *bs)
 int i;
 
 for (i = 0; i < s->num_children; i++) {
-bdrv_refresh_filename(s->children[i]->bs);
+bdrv_filename(s->children[i]->bs, s->children[i]->bs->filename,
+  sizeof(s->children[i]->bs->filename));
 if (!s->children[i]->bs->full_open_options) {
 return;
 }
diff --git a/include/block/block.h b/include/block/block.h
index 340c00f..9273e62 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -263,6 +263,7 @@ BlockDriverState *bdrv_find_backing_image(BlockDriverState 
*bs,
 const char *backing_file);
 int bdrv_get_backing_file_depth(BlockDriverState *bs);
 void bdrv_refresh_filename(BlockDriverState *bs);
+char 

Re: [Qemu-block] [Qemu-devel] [PATCH 00/17] Framework for securely passing secrets to QEMU

[Daniel P. Berrange]

On Mon, Oct 19, 2015 at 05:05:58PM +0100, Alex Bennée wrote:
> Daniel P. Berrange  writes:
> 
> > There are a variety of places where QEMU needs to have access
> > to passwords, encryption keys or similar kinds of secrets.
> >
> 
> >
> > Example usage for creating secrets...
> >
> > Direct password, insecure, for ad-hoc developer testing only
> >
> >   $QEMU -object secret,id=sec0,data=letmein
> >
> > Indirect password via a file, good for production
> >
> >   echo -n "letmein" > mypasswd.txt
> >   $QEMU -object secret,id=sec0,file=mypasswd.txt
> >
> > The file based approach supports file descriptor passing,
> > so mgmt apps can use that to dynamically add passwords to
> > running QEMU.
> >
> > There is a better way though, which is to use an encrypted
> > secret. The intent here is that a mgmt app (like libvirt)
> > will generate a random AES-256 key for each virtual machine
> > it starts (saved in eg /var/run/libvirt/qemu/$GUEST.key)
> > It can then use the direct password passing, but encrypt
> > the data.
> >
> >   $QEMU \
> > -object 
> > secret,id=secmaster0,file=/var/run/libvirt/qemu/$GUEST.key,format=base64 \
> > -object secret,id=sec0,data=[base64 ciphertext],\
> >keyid=secmaster0,iv=[base64 initialization vector]
> >
> > This means that the mgmt app does not need to worry about
> > file descriptor passing at all. It can just use regular
> > object properties, safe in the knowledge that the data is
> > protected by a secret AES key shared only between QEMU
> > and the mgmt app.
> >
> > Use of encrypted secrets is not restricted to directly
> > provided inline data. If the secret is stored in an
> > external file, that can be encrypted too
> >
> >   $QEMU \
> > -object 
> > secret,id=secmaster0,file=/var/run/libvirt/qemu/$GUEST.key,format=base64 \
> > -object secret,id=sec0,file=/some/secret/file.aes,\
> >keyid=secmaster0,iv=[base64 initialization vector]
> >
> >
> >
> > Example usage for referencing secrets...
> >
> > CURL:
> >
> >   $QEMU -object secret,id=sec0... \
> >  -drive driver=http,url=http://example.com/someimg.qcow2,\
> >   user=dan,passwordid=sec0
> >
> >   $QEMU -object secret,id=sec0... -object secret,id=sec1 \
> >  -drive driver=http,url=http://example.com/someimg.qcow2,\
> >   user=dan,passwordid=sec0,proxyuser=dan,passwordid=sec1
> >
> > iSCSI:
> >
> >   $QEMU -object secret,id=sec0... \
> >  -drive driver=iscsi,url=iscsi://example.com/target-foo/lun1,\
> >  user=dan,passwordid=sec0
> >
> > RBD:
> >
> >   $QEMU -object secret,id=sec0... \
> >  -drive driver=rbd,file=rbd:pool/image:id=myname,\
> >  auth-supported-cephx,authsecret=sec0
> >
> > QCow/Qcow2 encryption
> >
> >   $QEMU -object secret,id=sec0... \
> >  -drive file=someimage.qcow2,keyid=sec0
> 
> So one use case which I don't see here but do on other programs that
> need to access secrets is calling an external program and reading its
> stdout. This is simpler than having to mess around with passing FDs
> although there may be security concerns having QEMU create new tasks on
> the system.

Spawning external programs is a rather heavyweight approach. I can see
it being useful if you were locked into an existing API/syntax which
you couldn't modify, but we don't have that restriction here. I'm also
not a huge fan of having QEMU spawn a program that potentially has
access to a large number of passwords that QEMU should not be able
to access. I think passing in the required passwords explicitly as
done here is a better approach.

As noted earlier, FD passing is supported, but I don't think it is
going to be commonly needed or used. At least libvirt would not use
it, as providing the passwords directly, with encryption, is a more
straightforward approach to implement.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

[Qemu-block] [PATCH v5 5/6] block: Drop BlockDriverState.filename

[Max Reitz]

That field is now only used during initialization of BlockDriverStates
(opening images) and for error or warning messages. Performance is not
that much of an issue here, so we can drop the field and replace its use
by a call to bdrv_filename(). By doing so we can ensure the result
always to be recent, whereas the contents of BlockDriverState.filename
may have been obsoleted by manipulations of single BlockDriverStates or
of the BDS graph.

The users of the BDS filename field were changed as follows:
- copying the filename into another buffer is trivially replaced by
  using bdrv_filename() instead of the copy function
- strdup() on the filename is replaced by a call to
  bdrv_filename(filename, NULL, 0)
- bdrv_filename(bs, bs->filename, sizeof(bs->filename)) is replaced by
  bdrv_refresh_filename(bs)
  (these were introduced by the patch before the previous patch)
- anywhere else bdrv_filename(..., NULL, 0) is used, any access to
  BlockDriverState.filename is then replaced by the buffer returned, and
  it is freed when it is no longer needed

Signed-off-by: Max Reitz 
---
 block.c   | 46 ++
 block/blkverify.c |  3 +--
 block/commit.c|  4 +++-
 block/mirror.c| 16 
 block/qapi.c  |  4 ++--
 block/quorum.c|  3 +--
 block/raw_bsd.c   |  4 +++-
 block/vhdx-log.c  |  5 -
 block/vmdk.c  | 22 --
 block/vpc.c   |  7 +--
 blockdev.c| 25 +++--
 include/block/block_int.h |  1 -
 12 files changed, 96 insertions(+), 44 deletions(-)

diff --git a/block.c b/block.c
index 1061d80..a17e95f 100644
--- a/block.c
+++ b/block.c
@@ -226,10 +226,10 @@ void bdrv_get_full_backing_filename_from_filename(const 
char *backed,
 void bdrv_get_full_backing_filename(BlockDriverState *bs, char *dest, size_t 
sz,
 Error **errp)
 {
-char *backed = bs->exact_filename[0] ? bs->exact_filename : bs->filename;
-
+char *backed = bdrv_filename(bs, NULL, 0);
 bdrv_get_full_backing_filename_from_filename(backed, bs->backing_file,
  dest, sz, errp);
+g_free(backed);
 }
 
 void bdrv_register(BlockDriver *bdrv)
@@ -817,6 +817,7 @@ static int bdrv_open_common(BlockDriverState *bs, BdrvChild 
*file,
 QDict *options, int flags, BlockDriver *drv, Error **errp)
 {
 int ret, open_flags;
+char *filename_buffer = NULL;
 const char *filename;
 const char *node_name = NULL;
 QemuOpts *opts;
@@ -827,7 +828,8 @@ static int bdrv_open_common(BlockDriverState *bs, BdrvChild 
*file,
 assert(options != NULL && bs->options != options);
 
 if (file != NULL) {
-filename = file->bs->filename;
+filename_buffer = bdrv_filename(file->bs, NULL, 0);
+filename = filename_buffer;
 } else {
 filename = qdict_get_try_str(options, "filename");
 }
@@ -835,7 +837,8 @@ static int bdrv_open_common(BlockDriverState *bs, BdrvChild 
*file,
 if (drv->bdrv_needs_filename && !filename) {
 error_setg(errp, "The '%s' block driver requires a file name",
drv->format_name);
-return -EINVAL;
+ret = -EINVAL;
+goto fail_filename;
 }
 
 trace_bdrv_open_common(bs, filename ?: "", flags, drv->format_name);
@@ -883,11 +886,10 @@ static int bdrv_open_common(BlockDriverState *bs, 
BdrvChild *file,
 }
 
 if (filename != NULL) {
-pstrcpy(bs->filename, sizeof(bs->filename), filename);
+pstrcpy(bs->exact_filename, sizeof(bs->exact_filename), filename);
 } else {
-bs->filename[0] = '\0';
+bs->exact_filename[0] = '\0';
 }
-pstrcpy(bs->exact_filename, sizeof(bs->exact_filename), bs->filename);
 
 bs->drv = drv;
 bs->opaque = g_malloc0(drv->instance_size);
@@ -947,6 +949,7 @@ static int bdrv_open_common(BlockDriverState *bs, BdrvChild 
*file,
 assert((bs->request_alignment != 0) || bdrv_is_sg(bs));
 
 qemu_opts_del(opts);
+g_free(filename_buffer);
 return 0;
 
 free_and_fail:
@@ -956,6 +959,8 @@ free_and_fail:
 bs->drv = NULL;
 fail_opts:
 qemu_opts_del(opts);
+fail_filename:
+g_free(filename_buffer);
 return ret;
 }
 
@@ -1155,7 +1160,7 @@ void bdrv_set_backing_hd(BlockDriverState *bs, 
BlockDriverState *backing_hd)
 }
 bs->backing = bdrv_attach_child(bs, backing_hd, _backing);
 bs->open_flags &= ~BDRV_O_NO_BACKING;
-pstrcpy(bs->backing_file, sizeof(bs->backing_file), backing_hd->filename);
+bdrv_filename(backing_hd, bs->backing_file, sizeof(bs->backing_file));
 pstrcpy(bs->backing_format, sizeof(bs->backing_format),
 backing_hd->drv ? backing_hd->drv->format_name : "");
 
@@ -1527,7 +1532,7 @@ static int bdrv_open_inherit(BlockDriverState **pbs, 
const char *filename,
 }
 }
 
-

[Qemu-block] [PATCH v5 4/6] qemu-img: Use bdrv_filename() for map

[Max Reitz]

Replaces bs->filename by the result of bdrv_filename() in the
qemu-img map subcommand. Since that value is queried relatively often,
however, it should be cached.

Signed-off-by: Max Reitz 
---
 qemu-img.c | 14 +-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/qemu-img.c b/qemu-img.c
index 3025776..a44c77a 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -2142,6 +2142,7 @@ typedef struct MapEntry {
 int64_t length;
 int64_t offset;
 BlockDriverState *bs;
+char *bs_filename;
 } MapEntry;
 
 static void dump_map_entry(OutputFormat output_format, MapEntry *e,
@@ -2156,7 +2157,7 @@ static void dump_map_entry(OutputFormat output_format, 
MapEntry *e,
 }
 if ((e->flags & (BDRV_BLOCK_DATA|BDRV_BLOCK_ZERO)) == BDRV_BLOCK_DATA) 
{
 printf("%#-16"PRIx64"%#-16"PRIx64"%#-16"PRIx64"%s\n",
-   e->start, e->length, e->offset, e->bs->filename);
+   e->start, e->length, e->offset, e->bs_filename);
 }
 /* This format ignores the distinction between 0, ZERO and ZERO|DATA.
  * Modify the flags here to allow more coalescing.
@@ -2306,6 +2307,12 @@ static int img_map(int argc, char **argv)
 goto out;
 }
 
+if (next.bs == curr.bs) {
+next.bs_filename = curr.bs_filename;
+} else {
+next.bs_filename = bdrv_filename(next.bs, NULL, 0);
+}
+
 if (curr.length != 0 && curr.flags == next.flags &&
 curr.depth == next.depth &&
 ((curr.flags & BDRV_BLOCK_OFFSET_VALID) == 0 ||
@@ -2317,10 +2324,15 @@ static int img_map(int argc, char **argv)
 if (curr.length > 0) {
 dump_map_entry(output_format, , );
 }
+
+if (curr.bs != next.bs) {
+g_free(curr.bs_filename);
+}
 curr = next;
 }
 
 dump_map_entry(output_format, , NULL);
+g_free(curr.bs_filename);
 
 out:
 blk_unref(blk);
-- 
2.6.1

[Qemu-block] [PATCH 10/17] qemu-nbd: add support for --object command line arg

[Daniel P. Berrange]

Allow creation of user creatable object types with qemu-nbd
via a --object command line arg. This will be used to supply
passwords and/or encryption keys to the various block driver
backends via the recently added 'secret' object type.

 # echo -n letmein > mypasswd.txt
 # qemu-nbd --object secret,id=sec0,file=mypasswd.txt \
  ...other nbd args...

Signed-off-by: Daniel P. Berrange 
---
 qemu-nbd.c| 85 +++
 qemu-nbd.texi |  7 +
 2 files changed, 92 insertions(+)

diff --git a/qemu-nbd.c b/qemu-nbd.c
index 422a607..7eca167 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -23,9 +23,12 @@
 #include "qemu/main-loop.h"
 #include "qemu/sockets.h"
 #include "qemu/error-report.h"
+#include "qemu/config-file.h"
 #include "block/snapshot.h"
 #include "qapi/util.h"
 #include "qapi/qmp/qstring.h"
+#include "qapi/opts-visitor.h"
+#include "qom/object_interfaces.h"
 
 #include 
 #include 
@@ -45,6 +48,7 @@
 #define QEMU_NBD_OPT_AIO   2
 #define QEMU_NBD_OPT_DISCARD   3
 #define QEMU_NBD_OPT_DETECT_ZEROES 4
+#define QEMU_NBD_OPT_OBJECT5
 
 static NBDExport *exp;
 static int verbose;
@@ -78,6 +82,9 @@ static void usage(const char *name)
 "  -o, --offset=OFFSET   offset into the image\n"
 "  -P, --partition=NUM   only expose partition NUM\n"
 "\n"
+"General purpose options:\n"
+"  --object type,id=ID,...   define an object such as 'secret' for providing\n"
+"passwords and/or encryption keys\n"
 #ifdef __linux__
 "Kernel NBD client support:\n"
 "  -c, --connect=DEV connect FILE to the local NBD device DEV\n"
@@ -380,6 +387,67 @@ static SocketAddress *nbd_build_socket_address(const char 
*sockpath,
 }
 
 
+static QemuOptsList qemu_object_opts = {
+.name = "object",
+.implied_opt_name = "qom-type",
+.head = QTAILQ_HEAD_INITIALIZER(qemu_object_opts.head),
+.desc = {
+{ }
+},
+};
+
+static int object_create(void *opaque, QemuOpts *opts, Error **errp)
+{
+Error *err = NULL;
+char *type = NULL;
+char *id = NULL;
+void *dummy = NULL;
+OptsVisitor *ov;
+QDict *pdict;
+
+ov = opts_visitor_new(opts);
+pdict = qemu_opts_to_qdict(opts, NULL);
+
+visit_start_struct(opts_get_visitor(ov), , NULL, NULL, 0, );
+if (err) {
+goto out;
+}
+
+qdict_del(pdict, "qom-type");
+visit_type_str(opts_get_visitor(ov), , "qom-type", );
+if (err) {
+goto out;
+}
+
+qdict_del(pdict, "id");
+visit_type_str(opts_get_visitor(ov), , "id", );
+if (err) {
+goto out;
+}
+
+user_creatable_add(type, id, pdict, opts_get_visitor(ov), );
+if (err) {
+goto out;
+}
+visit_end_struct(opts_get_visitor(ov), );
+if (err) {
+user_creatable_del(id, NULL);
+}
+
+out:
+opts_visitor_cleanup(ov);
+
+QDECREF(pdict);
+g_free(id);
+g_free(type);
+g_free(dummy);
+if (err) {
+error_report_err(err);
+return -1;
+}
+return 0;
+}
+
 int main(int argc, char **argv)
 {
 BlockBackend *blk;
@@ -417,6 +485,7 @@ int main(int argc, char **argv)
 { "format", 1, NULL, 'f' },
 { "persistent", 0, NULL, 't' },
 { "verbose", 0, NULL, 'v' },
+{ "object", 1, NULL, QEMU_NBD_OPT_OBJECT },
 { NULL, 0, NULL, 0 }
 };
 int ch;
@@ -434,6 +503,7 @@ int main(int argc, char **argv)
 Error *local_err = NULL;
 BlockdevDetectZeroesOptions detect_zeroes = 
BLOCKDEV_DETECT_ZEROES_OPTIONS_OFF;
 QDict *options = NULL;
+QemuOpts *opts;
 
 /* The client thread uses SIGTERM to interrupt the server.  A signal
  * handler ensures that "qemu-nbd -v -c" exits with a nice status code.
@@ -442,6 +512,8 @@ int main(int argc, char **argv)
 memset(_sigterm, 0, sizeof(sa_sigterm));
 sa_sigterm.sa_handler = termsig_handler;
 sigaction(SIGTERM, _sigterm, NULL);
+module_call_init(MODULE_INIT_QOM);
+qemu_add_opts(_object_opts);
 qemu_init_exec_dir(argv[0]);
 
 while ((ch = getopt_long(argc, argv, sopt, lopt, _ind)) != -1) {
@@ -578,6 +650,13 @@ int main(int argc, char **argv)
 usage(argv[0]);
 exit(0);
 break;
+case QEMU_NBD_OPT_OBJECT:
+opts = qemu_opts_parse_noisily(qemu_find_opts("object"),
+   optarg, true);
+if (!opts) {
+exit(1);
+}
+break;
 case '?':
 errx(EXIT_FAILURE, "Try `%s --help' for more information.",
  argv[0]);
@@ -590,6 +669,12 @@ int main(int argc, char **argv)
  argv[0]);
 }
 
+if (qemu_opts_foreach(qemu_find_opts("object"),
+  object_create,
+  NULL, NULL)) {
+exit(1);
+}
+
 if (disconnect) {
 fd = open(argv[optind], O_RDWR);
 if (fd < 0) {
diff --git a/qemu-nbd.texi 

[Qemu-block] [PATCH v7 26/39] blockdev: Allow more options for BB-less BDS tree

[Max Reitz]

Most of the options which blockdev_init() parses for both the
BlockBackend and the root BDS are valid for just the root BDS as well
(e.g. read-only). This patch allows specifying these options even if not
creating a BlockBackend.

Signed-off-by: Max Reitz 
---
 blockdev.c | 97 ++
 1 file changed, 91 insertions(+), 6 deletions(-)

diff --git a/blockdev.c b/blockdev.c
index dd4885f..35e5719 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -614,6 +614,54 @@ err_no_opts:
 return NULL;
 }
 
+static QemuOptsList qemu_root_bds_opts;
+
+/* Takes the ownership of bs_opts */
+static BlockDriverState *bds_tree_init(QDict *bs_opts, Error **errp)
+{
+BlockDriverState *bs;
+QemuOpts *opts;
+Error *local_error = NULL;
+BlockdevDetectZeroesOptions detect_zeroes;
+int ret;
+int bdrv_flags = 0;
+
+opts = qemu_opts_create(_root_bds_opts, NULL, 1, errp);
+if (!opts) {
+goto fail;
+}
+
+qemu_opts_absorb_qdict(opts, bs_opts, _error);
+if (local_error) {
+error_propagate(errp, local_error);
+goto fail;
+}
+
+extract_common_blockdev_options(opts, _flags, NULL, NULL,
+_zeroes, _error);
+if (local_error) {
+error_propagate(errp, local_error);
+goto fail;
+}
+
+bs = NULL;
+ret = bdrv_open(, NULL, NULL, bs_opts, bdrv_flags, errp);
+if (ret < 0) {
+goto fail_no_bs_opts;
+}
+
+bs->detect_zeroes = detect_zeroes;
+
+fail_no_bs_opts:
+qemu_opts_del(opts);
+return bs;
+
+fail:
+qemu_opts_del(opts);
+QDECREF(bs_opts);
+return NULL;
+}
+
 static void qemu_opt_rename(QemuOpts *opts, const char *from, const char *to,
 Error **errp)
 {
@@ -3171,18 +3219,14 @@ void qmp_blockdev_add(BlockdevOptions *options, Error 
**errp)
 
 bs = blk_bs(blk);
 } else {
-int ret;
-
 if (!qdict_get_try_str(qdict, "node-name")) {
 error_setg(errp, "'id' and/or 'node-name' need to be specified for 
"
"the root node");
 goto fail;
 }
 
-bs = NULL;
-ret = bdrv_open(, NULL, NULL, qdict, BDRV_O_RDWR | BDRV_O_CACHE_WB,
-errp);
-if (ret < 0) {
+bs = bds_tree_init(qdict, errp);
+if (!bs) {
 goto fail;
 }
 }
@@ -3337,6 +3381,47 @@ QemuOptsList qemu_common_drive_opts = {
 },
 };
 
+static QemuOptsList qemu_root_bds_opts = {
+.name = "root-bds",
+.head = QTAILQ_HEAD_INITIALIZER(qemu_common_drive_opts.head),
+.desc = {
+{
+.name = "discard",
+.type = QEMU_OPT_STRING,
+.help = "discard operation (ignore/off, unmap/on)",
+},{
+.name = "cache.writeback",
+.type = QEMU_OPT_BOOL,
+.help = "enables writeback mode for any caches",
+},{
+.name = "cache.direct",
+.type = QEMU_OPT_BOOL,
+.help = "enables use of O_DIRECT (bypass the host page cache)",
+},{
+.name = "cache.no-flush",
+.type = QEMU_OPT_BOOL,
+.help = "ignore any flush requests for the device",
+},{
+.name = "aio",
+.type = QEMU_OPT_STRING,
+.help = "host AIO implementation (threads, native)",
+},{
+.name = "read-only",
+.type = QEMU_OPT_BOOL,
+.help = "open drive file as read-only",
+},{
+.name = "copy-on-read",
+.type = QEMU_OPT_BOOL,
+.help = "copy read data from backing file into image file",
+},{
+.name = "detect-zeroes",
+.type = QEMU_OPT_STRING,
+.help = "try to optimize zero writes (off, on, unmap)",
+},
+{ /* end of list */ }
+},
+};
+
 QemuOptsList qemu_drive_opts = {
 .name = "drive",
 .head = QTAILQ_HEAD_INITIALIZER(qemu_drive_opts.head),
-- 
2.6.1

Re: [Qemu-block] [Qemu-devel] [PATCH 07/17] qcow2: add a 'keyid' parameter to qcow2 options

[Eric Blake]

On 10/19/2015 09:09 AM, Daniel P. Berrange wrote:
> Add a 'keyid' parameter that refers to the ID of a
> QCryptoSecret instance that provides the encryption key.
> 
> $QEMU \
> -object secret,id=sec0,filename=/home/berrange/encrypted.pw \
> -drive file=/home/berrange/encrypted.qcow2,keyid=sec0
> 
> Signed-off-by: Daniel P. Berrange 
> ---
>  block/qcow2.c| 80 
> +---
>  block/qcow2.h|  1 +
>  qapi/block-core.json |  8 --
>  3 files changed, 64 insertions(+), 25 deletions(-)
> 

> +++ b/qapi/block-core.json
> @@ -1567,7 +1567,7 @@
>  # Driver specific block device options for qcow.
>  #
>  # @keyid: #optional ID of the "secret" object providing the
> -# AES decryption key.
> +# AES decryption key (since 2.5)

Looks like this line...

>  #
>  # Since: 2.5
>  ##
> @@ -1611,6 +1611,9 @@
>  # caches. The interval is in seconds. The default 
> value
>  # is 0 and it disables this feature (since 2.5)
>  #
> +# @keyid: #optional ID of the "secret" object providing the
> +# AES decryption key.

...and this line should be swapped.

> +#
>  # Since: 1.7

(For qcow, the entire struct is new so @keyid doesn't need versioning;
for qcow2, the struct existed since 1.7 and we are extending it in 2.5)

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-block] [Qemu-devel] [PATCH 03/17] rbd: add support for getting password from QCryptoSecret object

[Josh Durgin]

On 10/19/2015 08:09 AM, Daniel P. Berrange wrote:

Currently RBD passwords must be provided on the command line
via

   $QEMU -drive file=rbd:pool/image:id=myname:\
 key=QVFDVm41aE82SHpGQWhBQXEwTkN2OGp0SmNJY0UrSE9CbE1RMUE=:\
 auth_supported=cephx

This is insecure because the key is visible in the OS process
listing.

This adds support for an 'authsecret' parameter in the RBD
parameters that can be used with the QCryptoSecret object to
provide the password via a file:

   echo "QVFDVm41aE82SHpGQWhBQXEwTkN2OGp0SmNJY0UrSE9CbE1RMUE=" > poolkey.b64
   $QEMU -object secret,id=secret0,file=poolkey.b64,format=base64 \
   -drive file=rbd:pool/image:id=myname:\
   auth_supported=cephx,authsecret=secret0

Signed-off-by: Daniel P. Berrange 
---


Looks good in general, thanks for fixing this! Just one thing to fix
below.


  block/rbd.c | 42 ++
  1 file changed, 42 insertions(+)

diff --git a/block/rbd.c b/block/rbd.c
index a60a19d..0acf777 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -16,6 +16,7 @@
  #include "qemu-common.h"
  #include "qemu/error-report.h"
  #include "block/block_int.h"
+#include "crypto/secret.h"

  #include 

@@ -228,6 +229,23 @@ static char *qemu_rbd_parse_clientname(const char *conf, 
char *clientname)
  return NULL;
  }

+
+static int qemu_rbd_set_auth(rados_t cluster, const char *secretid,
+ Error **errp)
+{
+gchar *secret = qcrypto_secret_lookup_as_base64(secretid,
+errp);
+if (!secret) {
+return -1;
+}


It looks like this fails if no authsecret is provided. Ceph auth can be
disabled, so it seems like we should skip the qemu_rbd_set_auth() calls
in this case.


+
+rados_conf_set(cluster, "key", secret);
+g_free(secret);
+
+return 0;
+}
+
+
  static int qemu_rbd_set_conf(rados_t cluster, const char *conf,
   bool only_read_conf_file,
   Error **errp)
@@ -299,10 +317,13 @@ static int qemu_rbd_create(const char *filename, QemuOpts 
*opts, Error **errp)
  char conf[RBD_MAX_CONF_SIZE];
  char clientname_buf[RBD_MAX_CONF_SIZE];
  char *clientname;
+const char *secretid;
  rados_t cluster;
  rados_ioctx_t io_ctx;
  int ret;

+secretid = qemu_opt_get(opts, "authsecret");
+
  if (qemu_rbd_parsename(filename, pool, sizeof(pool),
 snap_buf, sizeof(snap_buf),
 name, sizeof(name),
@@ -350,6 +371,11 @@ static int qemu_rbd_create(const char *filename, QemuOpts 
*opts, Error **errp)
  return -EIO;
  }

+if (qemu_rbd_set_auth(cluster, secretid, errp) < 0) {
+rados_shutdown(cluster);
+return -EIO;
+}
+
  if (rados_connect(cluster) < 0) {
  error_setg(errp, "error connecting");
  rados_shutdown(cluster);
@@ -423,6 +449,11 @@ static QemuOptsList runtime_opts = {
  .type = QEMU_OPT_STRING,
  .help = "Specification of the rbd image",
  },
+{
+.name = "authsecret",
+.type = QEMU_OPT_STRING,
+.help = "ID of secret providing the password",
+},
  { /* end of list */ }
  },
  };
@@ -436,6 +467,7 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict 
*options, int flags,
  char conf[RBD_MAX_CONF_SIZE];
  char clientname_buf[RBD_MAX_CONF_SIZE];
  char *clientname;
+const char *secretid;
  QemuOpts *opts;
  Error *local_err = NULL;
  const char *filename;
@@ -450,6 +482,7 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict 
*options, int flags,
  }

  filename = qemu_opt_get(opts, "filename");
+secretid = qemu_opt_get(opts, "authsecret");

  if (qemu_rbd_parsename(filename, pool, sizeof(pool),
 snap_buf, sizeof(snap_buf),
@@ -488,6 +521,10 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict 
*options, int flags,
  }
  }

+if (qemu_rbd_set_auth(s->cluster, secretid, errp) < 0) {
+goto failed_shutdown;
+}
+
  /*
   * Fallback to more conservative semantics if setting cache
   * options fails. Ignore errors from setting rbd_cache because the
@@ -919,6 +956,11 @@ static QemuOptsList qemu_rbd_create_opts = {
  .type = QEMU_OPT_SIZE,
  .help = "RBD object size"
  },
+{
+.name = "authsecret",
+.type = QEMU_OPT_STRING,
+.help = "ID of secret providing the password",
+},
  { /* end of list */ }
  }
  };

[Qemu-block] [PATCH v4 04/12] aio: introduce aio_{disable, enable}_external

[Fam Zheng]

Signed-off-by: Fam Zheng 
---
 aio-posix.c |  3 ++-
 aio-win32.c |  3 ++-
 include/block/aio.h | 37 +
 3 files changed, 41 insertions(+), 2 deletions(-)

diff --git a/aio-posix.c b/aio-posix.c
index f0f9122..0467f23 100644
--- a/aio-posix.c
+++ b/aio-posix.c
@@ -261,7 +261,8 @@ bool aio_poll(AioContext *ctx, bool blocking)
 
 /* fill pollfds */
 QLIST_FOREACH(node, >aio_handlers, node) {
-if (!node->deleted && node->pfd.events) {
+if (!node->deleted && node->pfd.events
+&& aio_node_check(ctx, node->is_external)) {
 add_pollfd(node);
 }
 }
diff --git a/aio-win32.c b/aio-win32.c
index 3110d85..43c4c79 100644
--- a/aio-win32.c
+++ b/aio-win32.c
@@ -309,7 +309,8 @@ bool aio_poll(AioContext *ctx, bool blocking)
 /* fill fd sets */
 count = 0;
 QLIST_FOREACH(node, >aio_handlers, node) {
-if (!node->deleted && node->io_notify) {
+if (!node->deleted && node->io_notify
+&& aio_node_check(ctx, node->is_external)) {
 events[count++] = event_notifier_get_handle(node->e);
 }
 }
diff --git a/include/block/aio.h b/include/block/aio.h
index 12f1141..80151d1 100644
--- a/include/block/aio.h
+++ b/include/block/aio.h
@@ -122,6 +122,8 @@ struct AioContext {
 
 /* TimerLists for calling timers - one per clock type */
 QEMUTimerListGroup tlg;
+
+int external_disable_cnt;
 };
 
 /**
@@ -375,4 +377,39 @@ static inline void aio_timer_init(AioContext *ctx,
  */
 int64_t aio_compute_timeout(AioContext *ctx);
 
+/**
+ * aio_disable_external:
+ * @ctx: the aio context
+ *
+ * Disable the furthur processing of clients.
+ */
+static inline void aio_disable_external(AioContext *ctx)
+{
+atomic_inc(>external_disable_cnt);
+}
+
+/**
+ * aio_enable_external:
+ * @ctx: the aio context
+ *
+ * Disable the processing of external clients.
+ */
+static inline void aio_enable_external(AioContext *ctx)
+{
+atomic_dec(>external_disable_cnt);
+}
+
+/**
+ * aio_node_check:
+ * @ctx: the aio context
+ * @is_external: Whether or not the checked node is an external event source.
+ *
+ * Check if the node's is_external flag is okey to be polled by the ctx at this
+ * moment. True means green light.
+ */
+static inline bool aio_node_check(AioContext *ctx, bool is_external)
+{
+return !is_external || !atomic_read(>external_disable_cnt);
+}
+
 #endif
-- 
2.4.3

[Qemu-block] [PATCH v4 11/12] qed: Implement .bdrv_drain

[Fam Zheng]

The "need_check_timer" is used to clear the "NEED_CHECK" flag in the
image header after a grace period once metadata update has finished. In
compliance to the bdrv_drain semantics we should make sure it remains
deleted once .bdrv_drain is called.

Call the qed_need_check_timer_cb manually to update the header
immediately.

Signed-off-by: Fam Zheng 
---
 block/qed.c | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/block/qed.c b/block/qed.c
index 5ea05d4..e9dcb4d 100644
--- a/block/qed.c
+++ b/block/qed.c
@@ -375,6 +375,12 @@ static void bdrv_qed_attach_aio_context(BlockDriverState 
*bs,
 }
 }
 
+static void bdrv_qed_drain(BlockDriverState *bs)
+{
+qed_cancel_need_check_timer(bs->opaque);
+qed_need_check_timer_cb(bs->opaque);
+}
+
 static int bdrv_qed_open(BlockDriverState *bs, QDict *options, int flags,
  Error **errp)
 {
@@ -1676,6 +1682,7 @@ static BlockDriver bdrv_qed = {
 .bdrv_check   = bdrv_qed_check,
 .bdrv_detach_aio_context  = bdrv_qed_detach_aio_context,
 .bdrv_attach_aio_context  = bdrv_qed_attach_aio_context,
+.bdrv_drain   = bdrv_qed_drain,
 };
 
 static void bdrv_qed_init(void)
-- 
2.4.3

[Qemu-block] [PATCH v4 05/12] block: Introduce "drained begin/end" API

[Fam Zheng]

The semantics is that after bdrv_drained_begin(bs), bs will not get new external
requests until the matching bdrv_drained_end(bs).

Signed-off-by: Fam Zheng 
---
 block/io.c| 17 +
 include/block/block.h | 19 +++
 include/block/block_int.h |  2 ++
 3 files changed, 38 insertions(+)

diff --git a/block/io.c b/block/io.c
index 5311473..630c1aa 100644
--- a/block/io.c
+++ b/block/io.c
@@ -2618,3 +2618,20 @@ void bdrv_flush_io_queue(BlockDriverState *bs)
 }
 bdrv_start_throttled_reqs(bs);
 }
+
+void bdrv_drained_begin(BlockDriverState *bs)
+{
+if (!bs->quiesce_counter++) {
+aio_disable_external(bdrv_get_aio_context(bs));
+}
+bdrv_drain(bs);
+}
+
+void bdrv_drained_end(BlockDriverState *bs)
+{
+assert(bs->quiesce_counter > 0);
+if (--bs->quiesce_counter > 0) {
+return;
+}
+aio_enable_external(bdrv_get_aio_context(bs));
+}
diff --git a/include/block/block.h b/include/block/block.h
index 6d70eb4..72ea9f4 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -624,4 +624,23 @@ void bdrv_flush_io_queue(BlockDriverState *bs);
 
 BlockAcctStats *bdrv_get_stats(BlockDriverState *bs);
 
+/**
+ * bdrv_drained_begin:
+ *
+ * Begin a quiesced section for exclusive access to the BDS, by disabling
+ * external request sources including NBD server and device model. Note that
+ * this doesn't block timers or coroutines from submitting more requests, which
+ * means block_job_pause is still necessary.
+ *
+ * This function can be recursive.
+ */
+void bdrv_drained_begin(BlockDriverState *bs);
+
+/**
+ * bdrv_drained_end:
+ *
+ * End a quiescent section started by bdrv_drained_begin().
+ */
+void bdrv_drained_end(BlockDriverState *bs);
+
 #endif
diff --git a/include/block/block_int.h b/include/block/block_int.h
index c0e6513..45dcabe 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -456,6 +456,8 @@ struct BlockDriverState {
 /* threshold limit for writes, in bytes. "High water mark". */
 uint64_t write_threshold_offset;
 NotifierWithReturn write_threshold_notifier;
+
+int quiesce_counter;
 };
 
 static inline BlockDriverState *backing_bs(BlockDriverState *bs)
-- 
2.4.3

[Qemu-block] [PATCH v4 06/12] block: Add "drained begin/end" for transactional external snapshot

[Fam Zheng]

This ensures the atomicity of the transaction by avoiding processing of
external requests such as those from ioeventfd.

Signed-off-by: Fam Zheng 
---
 blockdev.c | 11 +--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/blockdev.c b/blockdev.c
index 8141b6b..fc63c3d 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1477,6 +1477,7 @@ static void external_snapshot_prepare(BlkTransactionState 
*common,
 /* Acquire AioContext now so any threads operating on old_bs stop */
 state->aio_context = bdrv_get_aio_context(state->old_bs);
 aio_context_acquire(state->aio_context);
+bdrv_drained_begin(state->old_bs);
 
 if (!bdrv_is_inserted(state->old_bs)) {
 error_setg(errp, QERR_DEVICE_HAS_NO_MEDIUM, device);
@@ -1546,8 +1547,6 @@ static void external_snapshot_commit(BlkTransactionState 
*common)
  * don't want to abort all of them if one of them fails the reopen */
 bdrv_reopen(state->old_bs, state->old_bs->open_flags & ~BDRV_O_RDWR,
 NULL);
-
-aio_context_release(state->aio_context);
 }
 
 static void external_snapshot_abort(BlkTransactionState *common)
@@ -1557,7 +1556,14 @@ static void external_snapshot_abort(BlkTransactionState 
*common)
 if (state->new_bs) {
 bdrv_unref(state->new_bs);
 }
+}
+
+static void external_snapshot_clean(BlkTransactionState *common)
+{
+ExternalSnapshotState *state =
+ DO_UPCAST(ExternalSnapshotState, common, common);
 if (state->aio_context) {
+bdrv_drained_end(state->old_bs);
 aio_context_release(state->aio_context);
 }
 }
@@ -1722,6 +1728,7 @@ static const BdrvActionOps actions[] = {
 .prepare  = external_snapshot_prepare,
 .commit   = external_snapshot_commit,
 .abort = external_snapshot_abort,
+.clean = external_snapshot_clean,
 },
 [TRANSACTION_ACTION_KIND_DRIVE_BACKUP] = {
 .instance_size = sizeof(DriveBackupState),
-- 
2.4.3

[Qemu-block] [PATCH v4 02/12] nbd: Mark fd handlers client type as "external"

[Fam Zheng]

So we could distinguish it from internal used fds, thus avoid handling
unwanted events in nested aio polls.

Signed-off-by: Fam Zheng 
---
 nbd.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/nbd.c b/nbd.c
index fbc66be..dab1ebb 100644
--- a/nbd.c
+++ b/nbd.c
@@ -1446,7 +1446,7 @@ static void nbd_set_handlers(NBDClient *client)
 {
 if (client->exp && client->exp->ctx) {
 aio_set_fd_handler(client->exp->ctx, client->sock,
-   false,
+   true,
client->can_read ? nbd_read : NULL,
client->send_coroutine ? nbd_restart_write : NULL,
client);
@@ -1457,7 +1457,7 @@ static void nbd_unset_handlers(NBDClient *client)
 {
 if (client->exp && client->exp->ctx) {
 aio_set_fd_handler(client->exp->ctx, client->sock,
-   false, NULL, NULL, NULL);
+   true, NULL, NULL, NULL);
 }
 }
 
-- 
2.4.3

[Qemu-block] [PATCH v4 01/12] aio: Add "is_external" flag for event handlers

[Fam Zheng]

All callers pass in false, and the real external ones will switch to
true in coming patches.

Signed-off-by: Fam Zheng 
---
 aio-posix.c |  6 -
 aio-win32.c |  5 
 async.c |  3 ++-
 block/curl.c| 14 +-
 block/iscsi.c   |  9 +++
 block/linux-aio.c   |  5 ++--
 block/nbd-client.c  | 10 ---
 block/nfs.c | 17 +---
 block/sheepdog.c| 38 ++-
 block/ssh.c |  5 ++--
 block/win32-aio.c   |  5 ++--
 hw/block/dataplane/virtio-blk.c |  6 +++--
 hw/scsi/virtio-scsi-dataplane.c | 24 +++--
 include/block/aio.h |  2 ++
 iohandler.c |  3 ++-
 nbd.c   |  4 ++-
 tests/test-aio.c| 58 +++--
 17 files changed, 130 insertions(+), 84 deletions(-)

diff --git a/aio-posix.c b/aio-posix.c
index d477033..f0f9122 100644
--- a/aio-posix.c
+++ b/aio-posix.c
@@ -25,6 +25,7 @@ struct AioHandler
 IOHandler *io_write;
 int deleted;
 void *opaque;
+bool is_external;
 QLIST_ENTRY(AioHandler) node;
 };
 
@@ -43,6 +44,7 @@ static AioHandler *find_aio_handler(AioContext *ctx, int fd)
 
 void aio_set_fd_handler(AioContext *ctx,
 int fd,
+bool is_external,
 IOHandler *io_read,
 IOHandler *io_write,
 void *opaque)
@@ -82,6 +84,7 @@ void aio_set_fd_handler(AioContext *ctx,
 node->io_read = io_read;
 node->io_write = io_write;
 node->opaque = opaque;
+node->is_external = is_external;
 
 node->pfd.events = (io_read ? G_IO_IN | G_IO_HUP | G_IO_ERR : 0);
 node->pfd.events |= (io_write ? G_IO_OUT | G_IO_ERR : 0);
@@ -92,10 +95,11 @@ void aio_set_fd_handler(AioContext *ctx,
 
 void aio_set_event_notifier(AioContext *ctx,
 EventNotifier *notifier,
+bool is_external,
 EventNotifierHandler *io_read)
 {
 aio_set_fd_handler(ctx, event_notifier_get_fd(notifier),
-   (IOHandler *)io_read, NULL, notifier);
+   is_external, (IOHandler *)io_read, NULL, notifier);
 }
 
 bool aio_prepare(AioContext *ctx)
diff --git a/aio-win32.c b/aio-win32.c
index 50a6867..3110d85 100644
--- a/aio-win32.c
+++ b/aio-win32.c
@@ -28,11 +28,13 @@ struct AioHandler {
 GPollFD pfd;
 int deleted;
 void *opaque;
+bool is_external;
 QLIST_ENTRY(AioHandler) node;
 };
 
 void aio_set_fd_handler(AioContext *ctx,
 int fd,
+bool is_external,
 IOHandler *io_read,
 IOHandler *io_write,
 void *opaque)
@@ -86,6 +88,7 @@ void aio_set_fd_handler(AioContext *ctx,
 node->opaque = opaque;
 node->io_read = io_read;
 node->io_write = io_write;
+node->is_external = is_external;
 
 event = event_notifier_get_handle(>notifier);
 WSAEventSelect(node->pfd.fd, event,
@@ -98,6 +101,7 @@ void aio_set_fd_handler(AioContext *ctx,
 
 void aio_set_event_notifier(AioContext *ctx,
 EventNotifier *e,
+bool is_external,
 EventNotifierHandler *io_notify)
 {
 AioHandler *node;
@@ -133,6 +137,7 @@ void aio_set_event_notifier(AioContext *ctx,
 node->e = e;
 node->pfd.fd = (uintptr_t)event_notifier_get_handle(e);
 node->pfd.events = G_IO_IN;
+node->is_external = is_external;
 QLIST_INSERT_HEAD(>aio_handlers, node, node);
 
 g_source_add_poll(>source, >pfd);
diff --git a/async.c b/async.c
index efce14b..bdc64a3 100644
--- a/async.c
+++ b/async.c
@@ -247,7 +247,7 @@ aio_ctx_finalize(GSource *source)
 }
 qemu_mutex_unlock(>bh_lock);
 
-aio_set_event_notifier(ctx, >notifier, NULL);
+aio_set_event_notifier(ctx, >notifier, false, NULL);
 event_notifier_cleanup(>notifier);
 rfifolock_destroy(>lock);
 qemu_mutex_destroy(>bh_lock);
@@ -329,6 +329,7 @@ AioContext *aio_context_new(Error **errp)
 }
 g_source_set_can_recurse(>source, true);
 aio_set_event_notifier(ctx, >notifier,
+   false,
(EventNotifierHandler *)
event_notifier_dummy_cb);
 ctx->thread_pool = NULL;
diff --git a/block/curl.c b/block/curl.c
index 032cc8a..8994182 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -154,18 +154,20 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int 
action,
 DPRINTF("CURL (AIO): Sock action %d on fd %d\n", action, fd);
 switch (action) {
 case CURL_POLL_IN:
- 

Re: [Qemu-block] [Qemu-devel] [RFC] transactions: add transaction-wide property

[Fam Zheng]

On Mon, 10/19 09:27, Markus Armbruster wrote:
> John Snow  writes:
> 
> > On 10/16/2015 08:23 AM, Stefan Hajnoczi wrote:
> >> On Mon, Oct 12, 2015 at 12:50:20PM -0400, John Snow wrote:
> >>> Ping -- any consensus on how we should implement the "do-or-die"
> >>> argument for transactions that start block jobs? :)
> >>>
> >>> This patch may look a little hokey in how it boxes arguments, but I can
> >>> re-do it on top of Eric Blake's very official way of boxing arguments,
> >>> when the QAPI dust settles.
> >> 
> >> I don't understand what you are trying to do after staring at the email
> >> for 5 minutes.  Maybe the other reviewers hit the same problem and
> >> haven't responded.
> >> 
> >> What is the problem you're trying to solve?
> >> 
> >> Stefan
> >> 
> >
> > Sorry...
> >
> > What I am trying to do is to add the transactional blocker property to
> > the *transaction* command and not as an argument to each individual action.
> >
> > There was some discussion on this so I wanted to just send an RFC to
> > show what I had in mind.
> 
> Was it the discussion on @transactional-cancel?  I'm on record
> supporting it per transaction rather than per action:
> Message-ID: <87mvwd8k9q@blackfin.pond.sub.org>
> http://lists.gnu.org/archive/html/qemu-devel/2015-09/msg05948.html

I prefer we start with a per-transaction flag as in this patch. Any
fine-grained arguments could be added in the future if it turns out to be
useful.

I'll take a look at the implementation later.

Fam

> 
> > This series applies on top of Fam's latest series and moves the
> > arguments from each action to a transaction-wide property.

[Qemu-block] [PATCH v4 03/12] dataplane: Mark host notifiers' client type as "external"

[Fam Zheng]

They will be excluded by type in the nested event loops in block layer,
so that unwanted events won't be processed there.

Signed-off-by: Fam Zheng 
---
 hw/block/dataplane/virtio-blk.c |  5 ++---
 hw/scsi/virtio-scsi-dataplane.c | 18 --
 2 files changed, 10 insertions(+), 13 deletions(-)

diff --git a/hw/block/dataplane/virtio-blk.c b/hw/block/dataplane/virtio-blk.c
index f8716bc..c42ddeb 100644
--- a/hw/block/dataplane/virtio-blk.c
+++ b/hw/block/dataplane/virtio-blk.c
@@ -283,7 +283,7 @@ void virtio_blk_data_plane_start(VirtIOBlockDataPlane *s)
 
 /* Get this show started by hooking up our callbacks */
 aio_context_acquire(s->ctx);
-aio_set_event_notifier(s->ctx, >host_notifier, false,
+aio_set_event_notifier(s->ctx, >host_notifier, true,
handle_notify);
 aio_context_release(s->ctx);
 return;
@@ -320,8 +320,7 @@ void virtio_blk_data_plane_stop(VirtIOBlockDataPlane *s)
 aio_context_acquire(s->ctx);
 
 /* Stop notifications for new requests from guest */
-aio_set_event_notifier(s->ctx, >host_notifier, false,
-   NULL);
+aio_set_event_notifier(s->ctx, >host_notifier, true, NULL);
 
 /* Drain and switch bs back to the QEMU main loop */
 blk_set_aio_context(s->conf->conf.blk, qemu_get_aio_context());
diff --git a/hw/scsi/virtio-scsi-dataplane.c b/hw/scsi/virtio-scsi-dataplane.c
index 21f51df..0d8d71e 100644
--- a/hw/scsi/virtio-scsi-dataplane.c
+++ b/hw/scsi/virtio-scsi-dataplane.c
@@ -60,8 +60,7 @@ static VirtIOSCSIVring *virtio_scsi_vring_init(VirtIOSCSI *s,
 r = g_new(VirtIOSCSIVring, 1);
 r->host_notifier = *virtio_queue_get_host_notifier(vq);
 r->guest_notifier = *virtio_queue_get_guest_notifier(vq);
-aio_set_event_notifier(s->ctx, >host_notifier, false,
-   handler);
+aio_set_event_notifier(s->ctx, >host_notifier, true, handler);
 
 r->parent = s;
 
@@ -72,8 +71,7 @@ static VirtIOSCSIVring *virtio_scsi_vring_init(VirtIOSCSI *s,
 return r;
 
 fail_vring:
-aio_set_event_notifier(s->ctx, >host_notifier, false,
-   NULL);
+aio_set_event_notifier(s->ctx, >host_notifier, true, NULL);
 k->set_host_notifier(qbus->parent, n, false);
 g_free(r);
 return NULL;
@@ -165,16 +163,16 @@ static void virtio_scsi_clear_aio(VirtIOSCSI *s)
 
 if (s->ctrl_vring) {
 aio_set_event_notifier(s->ctx, >ctrl_vring->host_notifier,
-   false, NULL);
+   true, NULL);
 }
 if (s->event_vring) {
 aio_set_event_notifier(s->ctx, >event_vring->host_notifier,
-   false, NULL);
+   true, NULL);
 }
 if (s->cmd_vrings) {
 for (i = 0; i < vs->conf.num_queues && s->cmd_vrings[i]; i++) {
 aio_set_event_notifier(s->ctx, >cmd_vrings[i]->host_notifier,
-   false, NULL);
+   true, NULL);
 }
 }
 }
@@ -296,12 +294,12 @@ void virtio_scsi_dataplane_stop(VirtIOSCSI *s)
 aio_context_acquire(s->ctx);
 
 aio_set_event_notifier(s->ctx, >ctrl_vring->host_notifier,
-   false, NULL);
+   true, NULL);
 aio_set_event_notifier(s->ctx, >event_vring->host_notifier,
-   false, NULL);
+   true, NULL);
 for (i = 0; i < vs->conf.num_queues; i++) {
 aio_set_event_notifier(s->ctx, >cmd_vrings[i]->host_notifier,
-   false, NULL);
+   true, NULL);
 }
 
 blk_drain_all(); /* ensure there are no in-flight requests */
-- 
2.4.3

[Qemu-block] [PATCH v4 00/12] block: Protect nested event loop with bdrv_drained_begin and bdrv_drained_end

[Fam Zheng]

v4: Rebase on to master so fix the "bdrv_move_feature_fields" issue.

v3: Call bdrv_drain unconditionally in bdrv_drained_begin.
Document the internal I/O implications between bdrv_drain_begin and end.

The nested aio_poll()'s in block layer has a bug that new r/w requests from
ioeventfds and nbd exports are processed, which might break the caller's
semantics (qmp_transaction) or even pointers (bdrv_reopen).


Fam Zheng (12):
  aio: Add "is_external" flag for event handlers
  nbd: Mark fd handlers client type as "external"
  dataplane: Mark host notifiers' client type as "external"
  aio: introduce aio_{disable,enable}_external
  block: Introduce "drained begin/end" API
  block: Add "drained begin/end" for transactional external snapshot
  block: Add "drained begin/end" for transactional backup
  block: Add "drained begin/end" for transactional blockdev-backup
  block: Add "drained begin/end" for internal snapshot
  block: Introduce BlockDriver.bdrv_drain callback
  qed: Implement .bdrv_drain
  tests: Add test case for aio_disable_external

 aio-posix.c |  9 -
 aio-win32.c |  8 +++-
 async.c |  3 +-
 block/curl.c| 14 ---
 block/io.c  | 23 +++-
 block/iscsi.c   |  9 ++---
 block/linux-aio.c   |  5 ++-
 block/nbd-client.c  | 10 +++--
 block/nfs.c | 17 -
 block/qed.c |  7 
 block/sheepdog.c| 38 ---
 block/ssh.c |  5 ++-
 block/win32-aio.c   |  5 ++-
 blockdev.c  | 27 +++---
 hw/block/dataplane/virtio-blk.c |  5 ++-
 hw/scsi/virtio-scsi-dataplane.c | 22 +++
 include/block/aio.h | 39 
 include/block/block.h   | 24 
 include/block/block_int.h   |  8 
 iohandler.c |  3 +-
 nbd.c   |  4 +-
 tests/test-aio.c| 82 -
 22 files changed, 274 insertions(+), 93 deletions(-)

-- 
2.4.3

[Qemu-block] [PATCH v4 10/12] block: Introduce BlockDriver.bdrv_drain callback

[Fam Zheng]

Drivers can have internal request sources that generate IO, like the
need_check_timer in QED. Since we want quiesced periods that contain
nested event loops in block layer, we need to have a way to disable such
event sources.

Block drivers must implement the "bdrv_drain" callback if it has any
internal sources that can generate I/O activity, like a timer or a
worker thread (even in a library) that can schedule QEMUBH in an
asynchronous callback.

Update the comments of bdrv_drain and bdrv_drained_begin accordingly.

Signed-off-by: Fam Zheng 
---
 block/io.c| 6 +-
 include/block/block.h | 9 +++--
 include/block/block_int.h | 6 ++
 3 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/block/io.c b/block/io.c
index 630c1aa..90a8119 100644
--- a/block/io.c
+++ b/block/io.c
@@ -234,7 +234,8 @@ bool bdrv_requests_pending(BlockDriverState *bs)
 }
 
 /*
- * Wait for pending requests to complete on a single BlockDriverState subtree
+ * Wait for pending requests to complete on a single BlockDriverState subtree,
+ * and suspend block driver's internal I/O until next request arrives.
  *
  * Note that unlike bdrv_drain_all(), the caller must hold the BlockDriverState
  * AioContext.
@@ -247,6 +248,9 @@ void bdrv_drain(BlockDriverState *bs)
 {
 bool busy = true;
 
+if (bs->drv && bs->drv->bdrv_drain) {
+bs->drv->bdrv_drain(bs);
+}
 while (busy) {
 /* Keep iterating */
  bdrv_flush_io_queue(bs);
diff --git a/include/block/block.h b/include/block/block.h
index 72ea9f4..c499c17 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -629,8 +629,13 @@ BlockAcctStats *bdrv_get_stats(BlockDriverState *bs);
  *
  * Begin a quiesced section for exclusive access to the BDS, by disabling
  * external request sources including NBD server and device model. Note that
- * this doesn't block timers or coroutines from submitting more requests, which
- * means block_job_pause is still necessary.
+ * this doesn't prevent timers or coroutines from submitting more requests,
+ * which means block_job_pause is still necessary.
+ *
+ * If new I/O requests are submitted after bdrv_drained_begin is called before
+ * bdrv_drained_end, more internal I/O might be going on after the request has
+ * been completed. If you don't want this, you have to issue another bdrv_drain
+ * or use a nested bdrv_drained_begin/end section.
  *
  * This function can be recursive.
  */
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 45dcabe..260d743 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -287,6 +287,12 @@ struct BlockDriver {
  */
 int (*bdrv_probe_geometry)(BlockDriverState *bs, HDGeometry *geo);
 
+/**
+ * Drain and stop any internal sources of requests in the driver, and
+ * remain so until next I/O callback (e.g. bdrv_co_writev) is called.
+ */
+void (*bdrv_drain)(BlockDriverState *bs);
+
 QLIST_ENTRY(BlockDriver) list;
 };
 
-- 
2.4.3

[Qemu-block] [PATCH v7 24/39] blockdev: Do not create BDS for empty drive

[Max Reitz]

Do not use "rudimentary" BDSs for empty drives any longer (for
freshly created drives).

After a follow-up patch, empty drives will generally use a NULL BDS, not
only the freshly created drives.

Signed-off-by: Max Reitz 
---
 blockdev.c | 68 --
 1 file changed, 40 insertions(+), 28 deletions(-)

diff --git a/blockdev.c b/blockdev.c
index c9271d2..faa5218 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -512,16 +512,40 @@ static BlockBackend *blockdev_init(const char *file, 
QDict *bs_opts,
 goto early_err;
 }
 
+if (snapshot) {
+/* always use cache=unsafe with snapshot */
+bdrv_flags &= ~BDRV_O_CACHE_MASK;
+bdrv_flags |= (BDRV_O_SNAPSHOT|BDRV_O_CACHE_WB|BDRV_O_NO_FLUSH);
+}
+
+if (copy_on_read) {
+bdrv_flags |= BDRV_O_COPY_ON_READ;
+}
+
+bdrv_flags |= ro ? 0 : BDRV_O_RDWR;
+
 /* init */
 if ((!file || !*file) && !has_driver_specific_opts) {
-blk = blk_new_with_bs(qemu_opts_id(opts), errp);
+BlockBackendRootState *blk_rs;
+
+blk = blk_new(qemu_opts_id(opts), errp);
 if (!blk) {
 goto early_err;
 }
 
-bs = blk_bs(blk);
-bs->open_flags = snapshot ? BDRV_O_SNAPSHOT : 0;
-bs->read_only = ro;
+blk_rs = blk_get_root_state(blk);
+blk_rs->open_flags= bdrv_flags;
+blk_rs->read_only = ro;
+blk_rs->detect_zeroes = detect_zeroes;
+
+if (throttle_enabled()) {
+if (!throttling_group) {
+throttling_group = blk_name(blk);
+}
+blk_rs->throttle_group = g_strdup(throttling_group);
+blk_rs->throttle_state = throttle_group_incref(throttling_group);
+blk_rs->throttle_state->cfg = cfg;
+}
 
 QDECREF(bs_opts);
 } else {
@@ -529,42 +553,30 @@ static BlockBackend *blockdev_init(const char *file, 
QDict *bs_opts,
 file = NULL;
 }
 
-if (snapshot) {
-/* always use cache=unsafe with snapshot */
-bdrv_flags &= ~BDRV_O_CACHE_MASK;
-bdrv_flags |= (BDRV_O_SNAPSHOT|BDRV_O_CACHE_WB|BDRV_O_NO_FLUSH);
-}
-
-if (copy_on_read) {
-bdrv_flags |= BDRV_O_COPY_ON_READ;
-}
-
-bdrv_flags |= ro ? 0 : BDRV_O_RDWR;
-
 blk = blk_new_open(qemu_opts_id(opts), file, NULL, bs_opts, bdrv_flags,
errp);
 if (!blk) {
 goto err_no_bs_opts;
 }
 bs = blk_bs(blk);
-}
 
-bs->detect_zeroes = detect_zeroes;
+bs->detect_zeroes = detect_zeroes;
 
-blk_set_on_error(blk, on_read_error, on_write_error);
+/* disk I/O throttling */
+if (throttle_enabled()) {
+if (!throttling_group) {
+throttling_group = blk_name(blk);
+}
+bdrv_io_limits_enable(bs, throttling_group);
+bdrv_set_io_limits(bs, );
+}
 
-/* disk I/O throttling */
-if (throttle_enabled()) {
-if (!throttling_group) {
-throttling_group = blk_name(blk);
+if (bdrv_key_required(bs)) {
+autostart = 0;
 }
-bdrv_io_limits_enable(bs, throttling_group);
-bdrv_set_io_limits(bs, );
 }
 
-if (bdrv_key_required(bs)) {
-autostart = 0;
-}
+blk_set_on_error(blk, on_read_error, on_write_error);
 
 err_no_bs_opts:
 qemu_opts_del(opts);
-- 
2.6.1

[Qemu-block] [PATCH v7 20/39] block: Fail requests to empty BlockBackend

[Max Reitz]

If there is no BlockDriverState in a BlockBackend or if the tray of the
guest device is open, fail all requests (where that is possible) with
-ENOMEDIUM.

The reason the status of the guest device is taken into account is
because once the guest device's tray is opened, any request on the same
BlockBackend as the guest uses should fail. If the BDS tree is supposed
to be usable even after ejecting it from the guest, a different
BlockBackend must be used.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Kevin Wolf 
---
 block/block-backend.c | 55 ++-
 1 file changed, 54 insertions(+), 1 deletion(-)

diff --git a/block/block-backend.c b/block/block-backend.c
index d790870..2779c22 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -529,7 +529,7 @@ static int blk_check_byte_request(BlockBackend *blk, 
int64_t offset,
 return -EIO;
 }
 
-if (!blk_is_inserted(blk)) {
+if (!blk_is_available(blk)) {
 return -ENOMEDIUM;
 }
 
@@ -668,6 +668,10 @@ int blk_pwrite(BlockBackend *blk, int64_t offset, const 
void *buf, int count)
 
 int64_t blk_getlength(BlockBackend *blk)
 {
+if (!blk_is_available(blk)) {
+return -ENOMEDIUM;
+}
+
 return bdrv_getlength(blk->bs);
 }
 
@@ -678,6 +682,10 @@ void blk_get_geometry(BlockBackend *blk, uint64_t 
*nb_sectors_ptr)
 
 int64_t blk_nb_sectors(BlockBackend *blk)
 {
+if (!blk_is_available(blk)) {
+return -ENOMEDIUM;
+}
+
 return bdrv_nb_sectors(blk->bs);
 }
 
@@ -708,6 +716,10 @@ BlockAIOCB *blk_aio_writev(BlockBackend *blk, int64_t 
sector_num,
 BlockAIOCB *blk_aio_flush(BlockBackend *blk,
   BlockCompletionFunc *cb, void *opaque)
 {
+if (!blk_is_available(blk)) {
+return abort_aio_request(blk, cb, opaque, -ENOMEDIUM);
+}
+
 return bdrv_aio_flush(blk->bs, cb, opaque);
 }
 
@@ -749,12 +761,20 @@ int blk_aio_multiwrite(BlockBackend *blk, BlockRequest 
*reqs, int num_reqs)
 
 int blk_ioctl(BlockBackend *blk, unsigned long int req, void *buf)
 {
+if (!blk_is_available(blk)) {
+return -ENOMEDIUM;
+}
+
 return bdrv_ioctl(blk->bs, req, buf);
 }
 
 BlockAIOCB *blk_aio_ioctl(BlockBackend *blk, unsigned long int req, void *buf,
   BlockCompletionFunc *cb, void *opaque)
 {
+if (!blk_is_available(blk)) {
+return abort_aio_request(blk, cb, opaque, -ENOMEDIUM);
+}
+
 return bdrv_aio_ioctl(blk->bs, req, buf, cb, opaque);
 }
 
@@ -770,11 +790,19 @@ int blk_co_discard(BlockBackend *blk, int64_t sector_num, 
int nb_sectors)
 
 int blk_co_flush(BlockBackend *blk)
 {
+if (!blk_is_available(blk)) {
+return -ENOMEDIUM;
+}
+
 return bdrv_co_flush(blk->bs);
 }
 
 int blk_flush(BlockBackend *blk)
 {
+if (!blk_is_available(blk)) {
+return -ENOMEDIUM;
+}
+
 return bdrv_flush(blk->bs);
 }
 
@@ -908,6 +936,11 @@ void blk_set_enable_write_cache(BlockBackend *blk, bool 
wce)
 
 void blk_invalidate_cache(BlockBackend *blk, Error **errp)
 {
+if (!blk->bs) {
+error_setg(errp, "Device '%s' has no medium", blk->name);
+return;
+}
+
 bdrv_invalidate_cache(blk->bs, errp);
 }
 
@@ -1063,6 +1096,10 @@ int blk_write_compressed(BlockBackend *blk, int64_t 
sector_num,
 
 int blk_truncate(BlockBackend *blk, int64_t offset)
 {
+if (!blk_is_available(blk)) {
+return -ENOMEDIUM;
+}
+
 return bdrv_truncate(blk->bs, offset);
 }
 
@@ -1079,21 +1116,37 @@ int blk_discard(BlockBackend *blk, int64_t sector_num, 
int nb_sectors)
 int blk_save_vmstate(BlockBackend *blk, const uint8_t *buf,
  int64_t pos, int size)
 {
+if (!blk_is_available(blk)) {
+return -ENOMEDIUM;
+}
+
 return bdrv_save_vmstate(blk->bs, buf, pos, size);
 }
 
 int blk_load_vmstate(BlockBackend *blk, uint8_t *buf, int64_t pos, int size)
 {
+if (!blk_is_available(blk)) {
+return -ENOMEDIUM;
+}
+
 return bdrv_load_vmstate(blk->bs, buf, pos, size);
 }
 
 int blk_probe_blocksizes(BlockBackend *blk, BlockSizes *bsz)
 {
+if (!blk_is_available(blk)) {
+return -ENOMEDIUM;
+}
+
 return bdrv_probe_blocksizes(blk->bs, bsz);
 }
 
 int blk_probe_geometry(BlockBackend *blk, HDGeometry *geo)
 {
+if (!blk_is_available(blk)) {
+return -ENOMEDIUM;
+}
+
 return bdrv_probe_geometry(blk->bs, geo);
 }
 
-- 
2.6.1

[Qemu-block] [PATCH v7 18/39] block: Add BlockBackendRootState

[Max Reitz]

This structure will store some of the state of the root BDS if the BDS
tree is removed, so that state can be restored once a new BDS tree is
inserted.

Signed-off-by: Max Reitz 
Reviewed-by: Kevin Wolf 
---
 block/block-backend.c  | 40 
 include/block/block_int.h  | 10 ++
 include/qemu/typedefs.h|  1 +
 include/sysemu/block-backend.h |  2 ++
 4 files changed, 53 insertions(+)

diff --git a/block/block-backend.c b/block/block-backend.c
index 2708ad1..6a3f0c7 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -13,6 +13,7 @@
 #include "sysemu/block-backend.h"
 #include "block/block_int.h"
 #include "block/blockjob.h"
+#include "block/throttle-groups.h"
 #include "sysemu/blockdev.h"
 #include "sysemu/sysemu.h"
 #include "qapi-event.h"
@@ -37,6 +38,10 @@ struct BlockBackend {
 /* the block size for which the guest device expects atomicity */
 int guest_block_size;
 
+/* If the BDS tree is removed, some of its options are stored here (which
+ * can be used to restore those options in the new BDS on insert) */
+BlockBackendRootState root_state;
+
 /* I/O stats (display with "info blockstats"). */
 BlockAcctStats stats;
 
@@ -161,6 +166,10 @@ static void blk_delete(BlockBackend *blk)
 bdrv_unref(blk->bs);
 blk->bs = NULL;
 }
+if (blk->root_state.throttle_state) {
+g_free(blk->root_state.throttle_group);
+throttle_group_unref(blk->root_state.throttle_state);
+}
 /* Avoid double-remove after blk_hide_on_behalf_of_hmp_drive_del() */
 if (blk->name[0]) {
 QTAILQ_REMOVE(_backends, blk, link);
@@ -1067,3 +1076,34 @@ int blk_probe_geometry(BlockBackend *blk, HDGeometry 
*geo)
 {
 return bdrv_probe_geometry(blk->bs, geo);
 }
+
+/*
+ * Updates the BlockBackendRootState object with data from the currently
+ * attached BlockDriverState.
+ */
+void blk_update_root_state(BlockBackend *blk)
+{
+assert(blk->bs);
+
+blk->root_state.open_flags= blk->bs->open_flags;
+blk->root_state.read_only = blk->bs->read_only;
+blk->root_state.detect_zeroes = blk->bs->detect_zeroes;
+
+if (blk->root_state.throttle_group) {
+g_free(blk->root_state.throttle_group);
+throttle_group_unref(blk->root_state.throttle_state);
+}
+if (blk->bs->throttle_state) {
+const char *name = throttle_group_get_name(blk->bs);
+blk->root_state.throttle_group = g_strdup(name);
+blk->root_state.throttle_state = throttle_group_incref(name);
+} else {
+blk->root_state.throttle_group = NULL;
+blk->root_state.throttle_state = NULL;
+}
+}
+
+BlockBackendRootState *blk_get_root_state(BlockBackend *blk)
+{
+return >root_state;
+}
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 009d6ea..e472a03 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -26,6 +26,7 @@
 
 #include "block/accounting.h"
 #include "block/block.h"
+#include "block/throttle-groups.h"
 #include "qemu/option.h"
 #include "qemu/queue.h"
 #include "block/coroutine.h"
@@ -449,6 +450,15 @@ struct BlockDriverState {
 NotifierWithReturn write_threshold_notifier;
 };
 
+struct BlockBackendRootState {
+int open_flags;
+bool read_only;
+BlockdevDetectZeroesOptions detect_zeroes;
+
+char *throttle_group;
+ThrottleState *throttle_state;
+};
+
 static inline BlockDriverState *backing_bs(BlockDriverState *bs)
 {
 return bs->backing ? bs->backing->bs : NULL;
diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
index d4a8f7a..d961362 100644
--- a/include/qemu/typedefs.h
+++ b/include/qemu/typedefs.h
@@ -11,6 +11,7 @@ typedef struct AddressSpace AddressSpace;
 typedef struct AioContext AioContext;
 typedef struct AudioState AudioState;
 typedef struct BlockBackend BlockBackend;
+typedef struct BlockBackendRootState BlockBackendRootState;
 typedef struct BlockDriverState BlockDriverState;
 typedef struct BusClass BusClass;
 typedef struct BusState BusState;
diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h
index eafcef0..52e35a1 100644
--- a/include/sysemu/block-backend.h
+++ b/include/sysemu/block-backend.h
@@ -163,6 +163,8 @@ void blk_add_close_notifier(BlockBackend *blk, Notifier 
*notify);
 void blk_io_plug(BlockBackend *blk);
 void blk_io_unplug(BlockBackend *blk);
 BlockAcctStats *blk_get_stats(BlockBackend *blk);
+BlockBackendRootState *blk_get_root_state(BlockBackend *blk);
+void blk_update_root_state(BlockBackend *blk);
 
 void *blk_aio_get(const AIOCBInfo *aiocb_info, BlockBackend *blk,
   BlockCompletionFunc *cb, void *opaque);
-- 
2.6.1

[Qemu-block] [PATCH v4 08/12] block: Add "drained begin/end" for transactional blockdev-backup

[Fam Zheng]

Similar to the previous patch, make sure that external events are not
dispatched during transaction operations.

Signed-off-by: Fam Zheng 
---
 blockdev.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/blockdev.c b/blockdev.c
index e3e68e6..6ab98e3 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1678,6 +1678,8 @@ static void blockdev_backup_prepare(BlkTransactionState 
*common, Error **errp)
 return;
 }
 aio_context_acquire(state->aio_context);
+state->bs = bs;
+bdrv_drained_begin(bs);
 
 qmp_blockdev_backup(backup->device, backup->target,
 backup->sync,
@@ -1690,7 +1692,6 @@ static void blockdev_backup_prepare(BlkTransactionState 
*common, Error **errp)
 return;
 }
 
-state->bs = bs;
 state->job = state->bs->job;
 }
 
@@ -1710,6 +1711,7 @@ static void blockdev_backup_clean(BlkTransactionState 
*common)
 BlockdevBackupState *state = DO_UPCAST(BlockdevBackupState, common, 
common);
 
 if (state->aio_context) {
+bdrv_drained_end(state->bs);
 aio_context_release(state->aio_context);
 }
 }
-- 
2.4.3

[Qemu-block] [PATCH v4 09/12] block: Add "drained begin/end" for internal snapshot

[Fam Zheng]

This ensures the atomicity of the transaction by avoiding processing of
external requests such as those from ioeventfd.

state->bs is assigned right after bdrv_drained_begin. Because it was
used as the flag for deletion or not in abort, now we need a separate
flag - InternalSnapshotState.created.

Signed-off-by: Fam Zheng 
---
 blockdev.c | 8 ++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/blockdev.c b/blockdev.c
index 6ab98e3..8635f77 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1278,6 +1278,7 @@ typedef struct InternalSnapshotState {
 BlockDriverState *bs;
 AioContext *aio_context;
 QEMUSnapshotInfo sn;
+bool created;
 } InternalSnapshotState;
 
 static void internal_snapshot_prepare(BlkTransactionState *common,
@@ -1316,6 +1317,8 @@ static void internal_snapshot_prepare(BlkTransactionState 
*common,
 /* AioContext is released in .clean() */
 state->aio_context = bdrv_get_aio_context(bs);
 aio_context_acquire(state->aio_context);
+bdrv_drained_begin(bs);
+state->bs = bs;
 
 if (!bdrv_is_inserted(bs)) {
 error_setg(errp, QERR_DEVICE_HAS_NO_MEDIUM, device);
@@ -1373,7 +1376,7 @@ static void internal_snapshot_prepare(BlkTransactionState 
*common,
 }
 
 /* 4. succeed, mark a snapshot is created */
-state->bs = bs;
+state->created = true;
 }
 
 static void internal_snapshot_abort(BlkTransactionState *common)
@@ -1384,7 +1387,7 @@ static void internal_snapshot_abort(BlkTransactionState 
*common)
 QEMUSnapshotInfo *sn = >sn;
 Error *local_error = NULL;
 
-if (!bs) {
+if (!state->created) {
 return;
 }
 
@@ -1405,6 +1408,7 @@ static void internal_snapshot_clean(BlkTransactionState 
*common)
  common, common);
 
 if (state->aio_context) {
+bdrv_drained_end(state->bs);
 aio_context_release(state->aio_context);
 }
 }
-- 
2.4.3

[Qemu-block] [PATCH v4 07/12] block: Add "drained begin/end" for transactional backup

[Fam Zheng]

This ensures the atomicity of the transaction by avoiding processing of
external requests such as those from ioeventfd.

Move the assignment to state->bs up right after bdrv_drained_begin, so
that we can use it in the clean callback. The abort callback will still
check bs->job and state->job, so it's OK.

Signed-off-by: Fam Zheng 
---
 blockdev.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/blockdev.c b/blockdev.c
index fc63c3d..e3e68e6 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1597,6 +1597,8 @@ static void drive_backup_prepare(BlkTransactionState 
*common, Error **errp)
 /* AioContext is released in .clean() */
 state->aio_context = bdrv_get_aio_context(bs);
 aio_context_acquire(state->aio_context);
+bdrv_drained_begin(bs);
+state->bs = bs;
 
 qmp_drive_backup(backup->device, backup->target,
  backup->has_format, backup->format,
@@ -1612,7 +1614,6 @@ static void drive_backup_prepare(BlkTransactionState 
*common, Error **errp)
 return;
 }
 
-state->bs = bs;
 state->job = state->bs->job;
 }
 
@@ -1632,6 +1633,7 @@ static void drive_backup_clean(BlkTransactionState 
*common)
 DriveBackupState *state = DO_UPCAST(DriveBackupState, common, common);
 
 if (state->aio_context) {
+bdrv_drained_end(state->bs);
 aio_context_release(state->aio_context);
 }
 }
-- 
2.4.3

[Qemu-block] [PATCH v7 23/39] block: Prepare for NULL BDS

[Max Reitz]

blk_bs() will not necessarily return a non-NULL value any more (unless
blk_is_available() is true or it can be assumed to otherwise, e.g.
because it is called immediately after a successful blk_new_with_bs() or
blk_new_open()).

Signed-off-by: Max Reitz 
---
 block.c |   5 ++
 block/qapi.c|   4 +-
 blockdev.c  | 204 ++--
 hw/block/xen_disk.c |   4 +-
 migration/block.c   |   5 ++
 monitor.c   |   4 ++
 6 files changed, 154 insertions(+), 72 deletions(-)

diff --git a/block.c b/block.c
index 6f239a4..f4dec11 100644
--- a/block.c
+++ b/block.c
@@ -2683,6 +2683,11 @@ BlockDriverState *bdrv_lookup_bs(const char *device,
 blk = blk_by_name(device);
 
 if (blk) {
+if (!blk_bs(blk)) {
+error_setg(errp, "Device '%s' has no medium", device);
+return NULL;
+}
+
 return blk_bs(blk);
 }
 }
diff --git a/block/qapi.c b/block/qapi.c
index 3b46f97..ec0f513 100644
--- a/block/qapi.c
+++ b/block/qapi.c
@@ -306,12 +306,12 @@ static void bdrv_query_info(BlockBackend *blk, BlockInfo 
**p_info,
 info->io_status = blk_iostatus(blk);
 }
 
-if (!QLIST_EMPTY(>dirty_bitmaps)) {
+if (bs && !QLIST_EMPTY(>dirty_bitmaps)) {
 info->has_dirty_bitmaps = true;
 info->dirty_bitmaps = bdrv_query_dirty_bitmaps(bs);
 }
 
-if (bs->drv) {
+if (bs && bs->drv) {
 info->has_inserted = true;
 info->inserted = bdrv_block_device_info(bs, errp);
 if (info->inserted == NULL) {
diff --git a/blockdev.c b/blockdev.c
index 433c4e2..c9271d2 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -124,14 +124,16 @@ void blockdev_mark_auto_del(BlockBackend *blk)
 return;
 }
 
-aio_context = bdrv_get_aio_context(bs);
-aio_context_acquire(aio_context);
+if (bs) {
+aio_context = bdrv_get_aio_context(bs);
+aio_context_acquire(aio_context);
 
-if (bs->job) {
-block_job_cancel(bs->job);
-}
+if (bs->job) {
+block_job_cancel(bs->job);
+}
 
-aio_context_release(aio_context);
+aio_context_release(aio_context);
+}
 
 dinfo->auto_del = 1;
 }
@@ -229,8 +231,8 @@ bool drive_check_orphaned(void)
 dinfo->type != IF_NONE) {
 fprintf(stderr, "Warning: Orphaned drive without device: "
 "id=%s,file=%s,if=%s,bus=%d,unit=%d\n",
-blk_name(blk), blk_bs(blk)->filename, if_name[dinfo->type],
-dinfo->bus, dinfo->unit);
+blk_name(blk), blk_bs(blk) ? blk_bs(blk)->filename : "",
+if_name[dinfo->type], dinfo->bus, dinfo->unit);
 rs = true;
 }
 }
@@ -1038,6 +1040,10 @@ void hmp_commit(Monitor *mon, const QDict *qdict)
 monitor_printf(mon, "Device '%s' not found\n", device);
 return;
 }
+if (!blk_is_available(blk)) {
+monitor_printf(mon, "Device '%s' has no medium\n", device);
+return;
+}
 ret = bdrv_commit(blk_bs(blk));
 }
 if (ret < 0) {
@@ -1117,7 +1123,9 @@ SnapshotInfo 
*qmp_blockdev_snapshot_delete_internal_sync(const char *device,
   "Device '%s' not found", device);
 return NULL;
 }
-bs = blk_bs(blk);
+
+aio_context = blk_get_aio_context(blk);
+aio_context_acquire(aio_context);
 
 if (!has_id) {
 id = NULL;
@@ -1129,11 +1137,14 @@ SnapshotInfo 
*qmp_blockdev_snapshot_delete_internal_sync(const char *device,
 
 if (!id && !name) {
 error_setg(errp, "Name or id must be provided");
-return NULL;
+goto out_aio_context;
 }
 
-aio_context = bdrv_get_aio_context(bs);
-aio_context_acquire(aio_context);
+if (!blk_is_available(blk)) {
+error_setg(errp, "Device '%s' has no medium", device);
+goto out_aio_context;
+}
+bs = blk_bs(blk);
 
 if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_INTERNAL_SNAPSHOT_DELETE, errp)) {
 goto out_aio_context;
@@ -1307,16 +1318,16 @@ static void 
internal_snapshot_prepare(BlkTransactionState *common,
   "Device '%s' not found", device);
 return;
 }
-bs = blk_bs(blk);
 
 /* AioContext is released in .clean() */
-state->aio_context = bdrv_get_aio_context(bs);
+state->aio_context = blk_get_aio_context(blk);
 aio_context_acquire(state->aio_context);
 
-if (!bdrv_is_inserted(bs)) {
+if (!blk_is_available(blk)) {
 error_setg(errp, QERR_DEVICE_HAS_NO_MEDIUM, device);
 return;
 }
+bs = blk_bs(blk);
 
 if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_INTERNAL_SNAPSHOT, errp)) {
 return;
@@ -1568,7 +1579,6 @@ typedef struct DriveBackupState {
 static void drive_backup_prepare(BlkTransactionState *common, Error **errp)
 {
 DriveBackupState *state = DO_UPCAST(DriveBackupState,