Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Denis V. Lunev]

On 03/10/2016 08:40 PM, Paolo Bonzini wrote:


On 10/03/2016 18:37, Stefan Hajnoczi wrote:

I suggest giving the new NBD command a "type" argument:
0 - SCSI mapped/anchored values according to SCSI Get LBA Status
1 - Dirty/clean, useful for incremental backup and other blocking tracking cases

This way we don't impinge on SCSI semantics and the command can be
used for both traditional logical block provisioning and dirty bitmap
info.

When the NBD export is started in QEMU you can optionally associate it
with a bitmap.  This bitmap is used to provide type=1 (dirty/clean)
status information.

Good idea.  The precedent in NBD is to use bits 16..31 of the command
for flags, so it could go there.

Paolo

like this too.

Den

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Paolo Bonzini]

On 10/03/2016 18:37, Stefan Hajnoczi wrote:
> I suggest giving the new NBD command a "type" argument:
> 0 - SCSI mapped/anchored values according to SCSI Get LBA Status
> 1 - Dirty/clean, useful for incremental backup and other blocking tracking 
> cases
> 
> This way we don't impinge on SCSI semantics and the command can be
> used for both traditional logical block provisioning and dirty bitmap
> info.
> 
> When the NBD export is started in QEMU you can optionally associate it
> with a bitmap.  This bitmap is used to provide type=1 (dirty/clean)
> status information.

Good idea.  The precedent in NBD is to use bits 16..31 of the command
for flags, so it could go there.

Paolo

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Stefan Hajnoczi]

On Mon, Feb 29, 2016 at 9:42 AM, Paolo Bonzini  wrote:
>
>
> On 29/02/2016 09:54, Paolo Bonzini wrote:
>>
>>
>> On 29/02/2016 09:14, Markus Armbruster wrote:
>>> I completely agree with you that Get LBA Status cannot just reflect the
>>> top layer.  But that's not what I meant to propose.  Let me try to
>>> explain myself more clearly.
>>>
>>> Consider a QCOW2 image D (for delta) with a backing file B (for base).
>>> If you open it normally, you see "D over B".  Get LBA Status should
>>> certainly claim the "deallocated" state only for blocks that are
>>> allocated neither in D nor B.
>>>
>>> However, you can also open D *without* its backing file.  Then you see
>>> "D over nothing".  Here, get LBA Status should claim "deallocated" state
>>> for anything not allocated in D.
>>
>> Ok, this makes more sense.
>>
>> The question then is whether to implement this NBD server inside QEMU,
>> or outside it as a separate process to which QEMU "pushes" blocks as in
>> the existing backup job.  I would prefer the latter, so that it is
>> possible to implement various APIs (get block status, but also VMware or
>> Parallels or whatever).
>>
>> Basically the same points made in
>> https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01969.html
>> still apply.
>
> Talked a bit to Fam now and I noted Denis's observation that QEMU would
> still use the backup block job, plus the NBD server as in Fam's
> fleecing.  Then the NBD server is already the push->pull adapter.  It's
> a bit clearer now.
>
> Opening D without backing file still feels a bit weird, because the NBD
> server would provide wrong data for clean blocks.  I would think that a
> "stupid" backup software could always ignore the get LBA status command
> and get a full backup.  Is this a requirement or not, and if not, why?
>
> I don't have any particular opinion against an NBD get LBA status
> command that returns deallocated/allocated _and_ clean/dirty.  But
> reusing one as the other feels like the kind of hack that seems clever
> and that you regret down the road.

I suggest giving the new NBD command a "type" argument:
0 - SCSI mapped/anchored values according to SCSI Get LBA Status
1 - Dirty/clean, useful for incremental backup and other blocking tracking cases

This way we don't impinge on SCSI semantics and the command can be
used for both traditional logical block provisioning and dirty bitmap
info.

When the NBD export is started in QEMU you can optionally associate it
with a bitmap.  This bitmap is used to provide type=1 (dirty/clean)
status information.

Stefan

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Markus Armbruster]

Markus Armbruster  writes:

[...]
> Consider a QCOW2 image D (for delta) with a backing file B (for base).
> If you open it normally, you see "D over B".  Get LBA Status should
> certainly claim the "deallocated" state only for blocks that are
> allocated neither in D nor B.
>
> However, you can also open D *without* its backing file.  Then you see
> "D over nothing".  Here, get LBA Status should claim "deallocated" state
> for anything not allocated in D.
>
> My proposal is to expose a "just the dirty blocks" view of a block
> backend similarly: it's a *separate* backend that contains *only* the
> dirty blocks.  Attempts to read a clean block behave exactly like
> reading an unmapped block from any other thinly provisioned backend
> (QCOW2 gives you zeroes, if I remember correctly).  I think it's only
> natural to make Get LBA Status claim "deallocated" for exactly the clean
> blocks then.

Regarding implementation: perhaps we can have a "bitmap filter" block
driver that masks out blocks based on a bitmap.  Read-only, at least
until somebody comes up with sane semantics for writing, and a use case
:)

Digression: masking out blocks based on a bitmap is is vaguely similar
to how a COW block driver replaces blocks based on delta.  What makes
such a COW block driver a *format* is persistence.  Same for bitmap
filters.  The question whether we want a bitmap filter *format* is being
discussed elsewhere.  Even if the answer should be "no", we could still
have a non-persistent filter if it's useful.

[...]

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Fam Zheng]

On Mon, 02/29 10:42, Paolo Bonzini wrote:
> 
> 
> On 29/02/2016 09:54, Paolo Bonzini wrote:
> > 
> > 
> > On 29/02/2016 09:14, Markus Armbruster wrote:
> >> I completely agree with you that Get LBA Status cannot just reflect the
> >> top layer.  But that's not what I meant to propose.  Let me try to
> >> explain myself more clearly.
> >>
> >> Consider a QCOW2 image D (for delta) with a backing file B (for base).
> >> If you open it normally, you see "D over B".  Get LBA Status should
> >> certainly claim the "deallocated" state only for blocks that are
> >> allocated neither in D nor B.
> >>
> >> However, you can also open D *without* its backing file.  Then you see
> >> "D over nothing".  Here, get LBA Status should claim "deallocated" state
> >> for anything not allocated in D.
> > 
> > Ok, this makes more sense.
> > 
> > The question then is whether to implement this NBD server inside QEMU,
> > or outside it as a separate process to which QEMU "pushes" blocks as in
> > the existing backup job.  I would prefer the latter, so that it is
> > possible to implement various APIs (get block status, but also VMware or
> > Parallels or whatever).
> > 
> > Basically the same points made in
> > https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01969.html
> > still apply.
> 
> Talked a bit to Fam now and I noted Denis's observation that QEMU would
> still use the backup block job, plus the NBD server as in Fam's
> fleecing.  Then the NBD server is already the push->pull adapter.  It's
> a bit clearer now.
> 
> Opening D without backing file still feels a bit weird, because the NBD
> server would provide wrong data for clean blocks.  I would think that a
> "stupid" backup software could always ignore the get LBA status command
> and get a full backup.  Is this a requirement or not, and if not, why?

Can we return -EIO instead of zero for unallocated/clean sectors?

> 
> I don't have any particular opinion against an NBD get LBA status
> command that returns deallocated/allocated _and_ clean/dirty.  But
> reusing one as the other feels like the kind of hack that seems clever
> and that you regret down the road.

Yes, I second this, my preference is also leaning towards separate states for
allocation and dirtiness in "Get LBA State" command.

> 
> I'm not sure whether the clean/dirty status is data plane or control
> plane either.  I don't think the terms have a well-defined meaning in
> terms of storage.  In the networking world, stuff like routing protocols
> (OSPF, BGP, etc.) is control plane, and based on this analogy dirty
> bitmaps seem like control plane to me.
> 
> So I wouldn't rule out QMP-based export of the dirty bitmap---either
> directly or optimized through a separate socket as in
> http://article.gmane.org/gmane.comp.emulators.qemu/397083.  Dirty
> bitmaps should also compress well, so perhaps gzip+base64 over JSON
> might work as well.  I'm not saying we certainly won't regret it, but it
> seems "less different".  Don't really know how to put it better...

One more side point here: a backup archive is useless until it has fully
completed, so in this particular use case, push or pull model doesn't matter in
sending the dirty bitmap.  But while we are at it, probably just be future
proof and do a pull iterface to allow more potential use cases? For example,
during data inspecting (a fleecing client is checking whether a particular file
in the guest file system has changed), only certain ranges in the dirty bitmap
is interesting.

Fam

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Paolo Bonzini]

On 29/02/2016 09:54, Paolo Bonzini wrote:
> 
> 
> On 29/02/2016 09:14, Markus Armbruster wrote:
>> I completely agree with you that Get LBA Status cannot just reflect the
>> top layer.  But that's not what I meant to propose.  Let me try to
>> explain myself more clearly.
>>
>> Consider a QCOW2 image D (for delta) with a backing file B (for base).
>> If you open it normally, you see "D over B".  Get LBA Status should
>> certainly claim the "deallocated" state only for blocks that are
>> allocated neither in D nor B.
>>
>> However, you can also open D *without* its backing file.  Then you see
>> "D over nothing".  Here, get LBA Status should claim "deallocated" state
>> for anything not allocated in D.
> 
> Ok, this makes more sense.
> 
> The question then is whether to implement this NBD server inside QEMU,
> or outside it as a separate process to which QEMU "pushes" blocks as in
> the existing backup job.  I would prefer the latter, so that it is
> possible to implement various APIs (get block status, but also VMware or
> Parallels or whatever).
> 
> Basically the same points made in
> https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01969.html
> still apply.

Talked a bit to Fam now and I noted Denis's observation that QEMU would
still use the backup block job, plus the NBD server as in Fam's
fleecing.  Then the NBD server is already the push->pull adapter.  It's
a bit clearer now.

Opening D without backing file still feels a bit weird, because the NBD
server would provide wrong data for clean blocks.  I would think that a
"stupid" backup software could always ignore the get LBA status command
and get a full backup.  Is this a requirement or not, and if not, why?

I don't have any particular opinion against an NBD get LBA status
command that returns deallocated/allocated _and_ clean/dirty.  But
reusing one as the other feels like the kind of hack that seems clever
and that you regret down the road.

I'm not sure whether the clean/dirty status is data plane or control
plane either.  I don't think the terms have a well-defined meaning in
terms of storage.  In the networking world, stuff like routing protocols
(OSPF, BGP, etc.) is control plane, and based on this analogy dirty
bitmaps seem like control plane to me.

So I wouldn't rule out QMP-based export of the dirty bitmap---either
directly or optimized through a separate socket as in
http://article.gmane.org/gmane.comp.emulators.qemu/397083.  Dirty
bitmaps should also compress well, so perhaps gzip+base64 over JSON
might work as well.  I'm not saying we certainly won't regret it, but it
seems "less different".  Don't really know how to put it better...

Paolo

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Paolo Bonzini]

On 29/02/2016 09:14, Markus Armbruster wrote:
> I completely agree with you that Get LBA Status cannot just reflect the
> top layer.  But that's not what I meant to propose.  Let me try to
> explain myself more clearly.
> 
> Consider a QCOW2 image D (for delta) with a backing file B (for base).
> If you open it normally, you see "D over B".  Get LBA Status should
> certainly claim the "deallocated" state only for blocks that are
> allocated neither in D nor B.
> 
> However, you can also open D *without* its backing file.  Then you see
> "D over nothing".  Here, get LBA Status should claim "deallocated" state
> for anything not allocated in D.

Ok, this makes more sense.

The question then is whether to implement this NBD server inside QEMU,
or outside it as a separate process to which QEMU "pushes" blocks as in
the existing backup job.  I would prefer the latter, so that it is
possible to implement various APIs (get block status, but also VMware or
Parallels or whatever).

Basically the same points made in
https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01969.html
still apply.

Paolo

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Markus Armbruster]

Paolo Bonzini  writes:

> On 19/02/2016 09:51, Markus Armbruster wrote:
>>> Is it an abuse to "Get LBA Status" to return dirty information? Because in 
>>> SCSI
>>> the command reports "mapped", "allocated" and "anchored" statuses. Does that
>>> mean NBD will use a different status set?
>> 
>> Perhaps some conceptual gymnastics can get us to standard semantics.
>> 
>> Incremental backup wants to copy out an image's "dirty" blocks.
>> 
>> We can view this as a bitmap telling us which of the image's blocks are
>> dirty.
>> 
>> An alternative view would be base image + dirty delta image.  In the the
>> dirty delta, exactly the dirty blocks are allocated.  The delta image
>> may be conceptual.
>
> I see a problem here. On one hand I agree that the "GET LBA STATUS" is
> a natural extension to the NBD protocol.
>
> On the other hand, the Get LBA Status command in SCSI reflects the
> state over the whole chain, not only the top element.  It is the
> equivalent of bdrv_get_block_status_above(bs, NULL, ...), rather than
> bdrv_get_block_status(bs, ...).  My understanding is that the dirty
> block information would require the latter, especially in the
> "conceptual delta image" model that Markus describes above.
>
> Having NBD implement subtly different semantics compared to SCSI is a
> bad idea in my opinion.

I completely agree with you that Get LBA Status cannot just reflect the
top layer.  But that's not what I meant to propose.  Let me try to
explain myself more clearly.

Consider a QCOW2 image D (for delta) with a backing file B (for base).
If you open it normally, you see "D over B".  Get LBA Status should
certainly claim the "deallocated" state only for blocks that are
allocated neither in D nor B.

However, you can also open D *without* its backing file.  Then you see
"D over nothing".  Here, get LBA Status should claim "deallocated" state
for anything not allocated in D.

My proposal is to expose a "just the dirty blocks" view of a block
backend similarly: it's a *separate* backend that contains *only* the
dirty blocks.  Attempts to read a clean block behave exactly like
reading an unmapped block from any other thinly provisioned backend
(QCOW2 gives you zeroes, if I remember correctly).  I think it's only
natural to make Get LBA Status claim "deallocated" for exactly the clean
blocks then.

> Of course if we call it "READ DIRTY BLOCKS" that would work, but I
> don't think such a command would be something that it makes sense to
> have in the general purpose NBD spec, so you would need a mechanism
> for vendor-specific extensions.
>
> Paolo
>
>> Now, consider the dirty delta *without* its backing image.  You can find
>> its allocated blocks with Get LBA Status.  As usual, you can read even
>> unallocated blocks, see SBC3 table 9.
>> 
>> If we NBD-export exactly that, we can use standard semantics, can't we?

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Fam Zheng]

On Fri, 02/26 23:40, Denis V. Lunev wrote:
> On 02/26/2016 10:55 PM, Paolo Bonzini wrote:
> >
> >On 19/02/2016 09:51, Markus Armbruster wrote:
> >>>Is it an abuse to "Get LBA Status" to return dirty information? Because in 
> >>>SCSI
> >>>the command reports "mapped", "allocated" and "anchored" statuses. Does 
> >>>that
> >>>mean NBD will use a different status set?
> >>Perhaps some conceptual gymnastics can get us to standard semantics.
> >>
> >>Incremental backup wants to copy out an image's "dirty" blocks.
> >>
> >>We can view this as a bitmap telling us which of the image's blocks are
> >>dirty.
> >>
> >>An alternative view would be base image + dirty delta image.  In the the
> >>dirty delta, exactly the dirty blocks are allocated.  The delta image
> >>may be conceptual.
> >I see a problem here. On one hand I agree that the "GET LBA STATUS" is
> >a natural extension to the NBD protocol.
> >
> >On the other hand, the Get LBA Status command in SCSI reflects the
> >state over the whole chain, not only the top element.  It is the
> >equivalent of bdrv_get_block_status_above(bs, NULL, ...), rather than
> >bdrv_get_block_status(bs, ...).  My understanding is that the dirty
> >block information would require the latter, especially in the
> >"conceptual delta image" model that Markus describes above.
> >
> >Having NBD implement subtly different semantics compared to SCSI is a
> >bad idea in my opinion.
> >
> >Of course if we call it "READ DIRTY BLOCKS" that would work, but I
> >don't think such a command would be something that it makes sense to
> >have in the general purpose NBD spec, so you would need a mechanism
> >for vendor-specific extensions.
> >
> >Paolo
> >
> In general, the idea to bind DIRTY BITMAP to GET STATUS command is not that
> bad. First of all, NBD has no relation to the SCSI at all thus we are not
> bound to the SCSI protocol. This is good thing.
> 
> Next, it is generally good to query state of the data block before reading
> to reduce amount of transfers over the network. This is useful even for a
> full backup operation to avoid zero block transfers.
> 
> Thus, if we have created special NBD share, we can provide STATUS bitmap
> in the way we want, f.e. with the proposed semantics. This will not be a
> violation of the protocol.

Paolo described what I was trying to say, but way clearer then what I said. :)
And I agree a dedicated bitmap status seems better here.

Fam

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[John Snow]

On 02/26/2016 03:03 PM, Paolo Bonzini wrote:
> 
> 
> On 26/02/2016 20:55, Paolo Bonzini wrote:
>>
>>
>> On 19/02/2016 09:51, Markus Armbruster wrote:
 Is it an abuse to "Get LBA Status" to return dirty information? Because in 
 SCSI
 the command reports "mapped", "allocated" and "anchored" statuses. Does 
 that
 mean NBD will use a different status set?
>>>
>>> Perhaps some conceptual gymnastics can get us to standard semantics.
>>>
>>> Incremental backup wants to copy out an image's "dirty" blocks.
>>>
>>> We can view this as a bitmap telling us which of the image's blocks are
>>> dirty.
>>>
>>> An alternative view would be base image + dirty delta image.  In the the
>>> dirty delta, exactly the dirty blocks are allocated.  The delta image
>>> may be conceptual.
>>
>> I see a problem here. On one hand I agree that the "GET LBA STATUS" is
>> a natural extension to the NBD protocol.
>>
>> On the other hand, the Get LBA Status command in SCSI reflects the
>> state over the whole chain, not only the top element.  It is the
>> equivalent of bdrv_get_block_status_above(bs, NULL, ...), rather than
>> bdrv_get_block_status(bs, ...).  My understanding is that the dirty
>> block information would require the latter, especially in the
>> "conceptual delta image" model that Markus describes above.
>>

This is unfortunate.

>> Having NBD implement subtly different semantics compared to SCSI is a
>> bad idea in my opinion.
>>
>> Of course if we call it "READ DIRTY BLOCKS" that would work, but I
>> don't think such a command would be something that it makes sense to
>> have in the general purpose NBD spec, so you would need a mechanism
>> for vendor-specific extensions.
> 
> Trying to be constructive: shall we instead have a simple mini-protocol
> with commands like "read bitmap slice", "clear bitmap slice", "query
> next 0 bit", "query next 1 bit"?
> 
> Not necessarily QMP, just a little socket thing.  It could be JSON or
> ASCII or binary.  It sucks to implement a new protocol, but perhaps it
> could be something compatible with VMware or Parallels.
> 
> Sorry if this has already been proposed, I'm late to the game and I only
> read this subthread.
> 
> Paolo
> 

The solution I proposed prior to the LBA status idea (which I did like,
pending its suitability which looks... compromised, now. sigh.) was a
simple binary protocol.

Something akin to this:

{ "execute": "dirty-bitmap-export",
  "arguments": {
"node": "drive0",
"bitmap": "bitmap0",
"uri": "tcp:127.0.0.1:31337"
  }
}

We'll open up a socket to the URI provided, and push data akin to the
following:

Bytes 0-3: "QBMP" (Qemu Bit Map Protocol)
Bytes 4-11: number of dirty bits (As a lazy checksum)
Bytes 12-19: granularity of bits
Bytes 20+: raw bitmap data
EOF

A simple, non-glorious format that just gets the data out there. Then
the client has the info it needs to poll dirty blocks as necessary.


The other suggestion is to literally fetch the bitmap over NBD. How
would this work -- would the export just be sized to the number of bytes
of the bitmap data, and the client queries it?

I can't help but feel like this is slightly hacky, but I can't think of
any truly solid reasons against it either.

--js

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Denis V. Lunev]

On 02/26/2016 10:55 PM, Paolo Bonzini wrote:


On 19/02/2016 09:51, Markus Armbruster wrote:

Is it an abuse to "Get LBA Status" to return dirty information? Because in SCSI
the command reports "mapped", "allocated" and "anchored" statuses. Does that
mean NBD will use a different status set?

Perhaps some conceptual gymnastics can get us to standard semantics.

Incremental backup wants to copy out an image's "dirty" blocks.

We can view this as a bitmap telling us which of the image's blocks are
dirty.

An alternative view would be base image + dirty delta image.  In the the
dirty delta, exactly the dirty blocks are allocated.  The delta image
may be conceptual.

I see a problem here. On one hand I agree that the "GET LBA STATUS" is
a natural extension to the NBD protocol.

On the other hand, the Get LBA Status command in SCSI reflects the
state over the whole chain, not only the top element.  It is the
equivalent of bdrv_get_block_status_above(bs, NULL, ...), rather than
bdrv_get_block_status(bs, ...).  My understanding is that the dirty
block information would require the latter, especially in the
"conceptual delta image" model that Markus describes above.

Having NBD implement subtly different semantics compared to SCSI is a
bad idea in my opinion.

Of course if we call it "READ DIRTY BLOCKS" that would work, but I
don't think such a command would be something that it makes sense to
have in the general purpose NBD spec, so you would need a mechanism
for vendor-specific extensions.

Paolo


In general, the idea to bind DIRTY BITMAP to GET STATUS command is not that
bad. First of all, NBD has no relation to the SCSI at all thus we are not
bound to the SCSI protocol. This is good thing.

Next, it is generally good to query state of the data block before reading
to reduce amount of transfers over the network. This is useful even for a
full backup operation to avoid zero block transfers.

Thus, if we have created special NBD share, we can provide STATUS bitmap
in the way we want, f.e. with the proposed semantics. This will not be a
violation of the protocol.

Den

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Denis V. Lunev]

On 02/26/2016 11:03 PM, Paolo Bonzini wrote:


On 26/02/2016 20:55, Paolo Bonzini wrote:


On 19/02/2016 09:51, Markus Armbruster wrote:

Is it an abuse to "Get LBA Status" to return dirty information? Because in SCSI
the command reports "mapped", "allocated" and "anchored" statuses. Does that
mean NBD will use a different status set?

Perhaps some conceptual gymnastics can get us to standard semantics.

Incremental backup wants to copy out an image's "dirty" blocks.

We can view this as a bitmap telling us which of the image's blocks are
dirty.

An alternative view would be base image + dirty delta image.  In the the
dirty delta, exactly the dirty blocks are allocated.  The delta image
may be conceptual.

I see a problem here. On one hand I agree that the "GET LBA STATUS" is
a natural extension to the NBD protocol.

On the other hand, the Get LBA Status command in SCSI reflects the
state over the whole chain, not only the top element.  It is the
equivalent of bdrv_get_block_status_above(bs, NULL, ...), rather than
bdrv_get_block_status(bs, ...).  My understanding is that the dirty
block information would require the latter, especially in the
"conceptual delta image" model that Markus describes above.

Having NBD implement subtly different semantics compared to SCSI is a
bad idea in my opinion.

Of course if we call it "READ DIRTY BLOCKS" that would work, but I
don't think such a command would be something that it makes sense to
have in the general purpose NBD spec, so you would need a mechanism
for vendor-specific extensions.

Trying to be constructive: shall we instead have a simple mini-protocol
with commands like "read bitmap slice", "clear bitmap slice", "query
next 0 bit", "query next 1 bit"?

Not necessarily QMP, just a little socket thing.  It could be JSON or
ASCII or binary.  It sucks to implement a new protocol, but perhaps it
could be something compatible with VMware or Parallels.

Sorry if this has already been proposed, I'm late to the game and I only
read this subthread.

Paolo

there was a proposal in the thread to export bitmap itself as NBD
block device. We have also discuss to transfer entire bitmap over
QMP but this idea is not that good according to Eric/Markus.

Den

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Paolo Bonzini]

On 26/02/2016 20:55, Paolo Bonzini wrote:
> 
> 
> On 19/02/2016 09:51, Markus Armbruster wrote:
>>> Is it an abuse to "Get LBA Status" to return dirty information? Because in 
>>> SCSI
>>> the command reports "mapped", "allocated" and "anchored" statuses. Does that
>>> mean NBD will use a different status set?
>>
>> Perhaps some conceptual gymnastics can get us to standard semantics.
>>
>> Incremental backup wants to copy out an image's "dirty" blocks.
>>
>> We can view this as a bitmap telling us which of the image's blocks are
>> dirty.
>>
>> An alternative view would be base image + dirty delta image.  In the the
>> dirty delta, exactly the dirty blocks are allocated.  The delta image
>> may be conceptual.
> 
> I see a problem here. On one hand I agree that the "GET LBA STATUS" is
> a natural extension to the NBD protocol.
> 
> On the other hand, the Get LBA Status command in SCSI reflects the
> state over the whole chain, not only the top element.  It is the
> equivalent of bdrv_get_block_status_above(bs, NULL, ...), rather than
> bdrv_get_block_status(bs, ...).  My understanding is that the dirty
> block information would require the latter, especially in the
> "conceptual delta image" model that Markus describes above.
> 
> Having NBD implement subtly different semantics compared to SCSI is a
> bad idea in my opinion.
> 
> Of course if we call it "READ DIRTY BLOCKS" that would work, but I
> don't think such a command would be something that it makes sense to
> have in the general purpose NBD spec, so you would need a mechanism
> for vendor-specific extensions.

Trying to be constructive: shall we instead have a simple mini-protocol
with commands like "read bitmap slice", "clear bitmap slice", "query
next 0 bit", "query next 1 bit"?

Not necessarily QMP, just a little socket thing.  It could be JSON or
ASCII or binary.  It sucks to implement a new protocol, but perhaps it
could be something compatible with VMware or Parallels.

Sorry if this has already been proposed, I'm late to the game and I only
read this subthread.

Paolo

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Paolo Bonzini]

On 19/02/2016 09:51, Markus Armbruster wrote:
>> Is it an abuse to "Get LBA Status" to return dirty information? Because in 
>> SCSI
>> the command reports "mapped", "allocated" and "anchored" statuses. Does that
>> mean NBD will use a different status set?
> 
> Perhaps some conceptual gymnastics can get us to standard semantics.
> 
> Incremental backup wants to copy out an image's "dirty" blocks.
> 
> We can view this as a bitmap telling us which of the image's blocks are
> dirty.
> 
> An alternative view would be base image + dirty delta image.  In the the
> dirty delta, exactly the dirty blocks are allocated.  The delta image
> may be conceptual.

I see a problem here. On one hand I agree that the "GET LBA STATUS" is
a natural extension to the NBD protocol.

On the other hand, the Get LBA Status command in SCSI reflects the
state over the whole chain, not only the top element.  It is the
equivalent of bdrv_get_block_status_above(bs, NULL, ...), rather than
bdrv_get_block_status(bs, ...).  My understanding is that the dirty
block information would require the latter, especially in the
"conceptual delta image" model that Markus describes above.

Having NBD implement subtly different semantics compared to SCSI is a
bad idea in my opinion.

Of course if we call it "READ DIRTY BLOCKS" that would work, but I
don't think such a command would be something that it makes sense to
have in the general purpose NBD spec, so you would need a mechanism
for vendor-specific extensions.

Paolo

> Now, consider the dirty delta *without* its backing image.  You can find
> its allocated blocks with Get LBA Status.  As usual, you can read even
> unallocated blocks, see SBC3 table 9.
> 
> If we NBD-export exactly that, we can use standard semantics, can't we?

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[John Snow]

On 02/19/2016 03:51 AM, Markus Armbruster wrote:
> Fam Zheng  writes:
> 
>> On Thu, 02/18 16:41, Stefan Hajnoczi wrote:
>>> On Thu, Feb 18, 2016 at 12:11:14PM +, Daniel P. Berrange wrote:
 On Wed, Feb 17, 2016 at 08:47:11PM +0300, Vladimir Sementsov-Ogievskiy 
 wrote:
> On 16.02.2016 20:09, Stefan Hajnoczi wrote:
>> On Wed, Feb 10, 2016 at 10:10:04AM +, Stefan Hajnoczi wrote:
>>> On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:
 On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:
> On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
>> On 02/03/2016 11:14 AM, Fam Zheng wrote:
>>> On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
 Hi all.

 These series which aims to add external backup api. This is needed 
 to allow
 backup software use our dirty bitmaps.

 Vmware and Parallels Cloud Server have this feature.
>>> What is the advantage of this appraoch over "drive-backup 
>>> sync=incremental
>>> ..."?
>> This will allow third-party vendors to backup QEMU VMs into
>> their own formats or to the cloud etc.
> As an example, there is a third-party backup format called VMA from
> Proxmox.  A few years ago I posted a proof-of-concept external backup
> tool in Python:
>
> https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html
>
> It takes a full backup using drive-backup NBD (plus RAM/device state)
> but the same can be done with incremental backups.
>
> Does this NBD approach meet your requirements?
>
> Stefan
 for us we should somehow provide implementation of
 calls posted by Vladimir. They are available in Parallels Server
 version 6 and should be available in the next QEMU based
 release using "Parallels SDK to libvirt" convertor. The problem
 for us is that this old approach is used in the other side
 of the product - in containers implementation while this
 SDK is a universal access tool to both things.
>>> Point taken.  I think many other backup applications will expect a
>>> similar API, so it's pragmatic to provide something compatible.
>> Kevin Wolf and Daniel Berrange proposed an elegant way to avoid the
>> concerns about the QMP monitor:
>>
>> Previously I described incremental backup in "push" mode (already
>> supported today with drive-backup).  QEMU connects to a remote NBD
>> server and writes out the contents of all dirty blocks:
>>
>>   QEMU ---Write dirty blocks--> Backup appliance (server)
>>
>> This doesn't lend itself well to existing backup applications that
>> expect to iterate the dirty bitmap manually.
>>
>> Let's add a "pull" mode where the connection of the NBD connection is
>> reversed.  The backup application connects to QEMU's NBD server (image
>> fleecing).  The NBD protocol is extended to support the SCSI Get LBA
>> Status command for querying block provisioning information.  Now the
>> backup application can use Get LBA Status to fetch the dirty block
>> information from QEMU.
>>
>>   QEMU (server) <--Get LBA Status or Read dirty blocks-- Backup appliance
>>
>> The dirty block information goes over the same NBD connection used to
>> read the contents of the dirty blocks.  The QMP monitor is not used to
>> transfer dirty block information.
>>
>> It may be necessary to extend the nbd-server-add command so that a
>> bitmap name can be passed.  This bitmap will be used to answer Get LBA
>> Status queries instead of using on bdrv_co_get_block_status().  This
>> would be necessary if image fleecing (point in time snapshot) is used.
>>
>> Stefan
>
> There are no such commands in nbd spec here:
>
> https://github.com/yoe/nbd/blob/master/doc/proto.md
>
>
> So, I'm not sure, that adding something qemu-specific to this external
> protocol will be simple or even true way. Is Qemu already extending 
> original
> nbd?

 No, we don't do any QEMU specific extensions. The point of the approach
 Stefan suggests here though, is that it is *not* an inherantly 
 QEMU-specific
 concept, it is relevant to any NBD server implementation.

 For example, consider you were using a regular NBD server to export a
 sparse LVM volume. This proposed extension would be relevant to such
 a use case. As such this proposed extension is something that is likely
 to be acceptable for the generic NBD specification.
>>>
>>> Yes, Get LBA Status could be useful for non-QEMU NBD users too.
>>>
>>> NBD already supports a TRIM command so the ability to query the
>>> allocation status is a natural feature to add.
>>
>> Is it 

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Markus Armbruster]

Fam Zheng  writes:

> On Thu, 02/18 16:41, Stefan Hajnoczi wrote:
>> On Thu, Feb 18, 2016 at 12:11:14PM +, Daniel P. Berrange wrote:
>> > On Wed, Feb 17, 2016 at 08:47:11PM +0300, Vladimir Sementsov-Ogievskiy 
>> > wrote:
>> > > On 16.02.2016 20:09, Stefan Hajnoczi wrote:
>> > > >On Wed, Feb 10, 2016 at 10:10:04AM +, Stefan Hajnoczi wrote:
>> > > >>On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:
>> > > >>>On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:
>> > > On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
>> > > >On 02/03/2016 11:14 AM, Fam Zheng wrote:
>> > > >>On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
>> > > >>>Hi all.
>> > > >>>
>> > > >>>These series which aims to add external backup api. This is 
>> > > >>>needed to allow
>> > > >>>backup software use our dirty bitmaps.
>> > > >>>
>> > > >>>Vmware and Parallels Cloud Server have this feature.
>> > > >>What is the advantage of this appraoch over "drive-backup 
>> > > >>sync=incremental
>> > > >>..."?
>> > > >This will allow third-party vendors to backup QEMU VMs into
>> > > >their own formats or to the cloud etc.
>> > > As an example, there is a third-party backup format called VMA from
>> > > Proxmox.  A few years ago I posted a proof-of-concept external backup
>> > > tool in Python:
>> > > 
>> > > https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html
>> > > 
>> > > It takes a full backup using drive-backup NBD (plus RAM/device state)
>> > > but the same can be done with incremental backups.
>> > > 
>> > > Does this NBD approach meet your requirements?
>> > > 
>> > > Stefan
>> > > >>>for us we should somehow provide implementation of
>> > > >>>calls posted by Vladimir. They are available in Parallels Server
>> > > >>>version 6 and should be available in the next QEMU based
>> > > >>>release using "Parallels SDK to libvirt" convertor. The problem
>> > > >>>for us is that this old approach is used in the other side
>> > > >>>of the product - in containers implementation while this
>> > > >>>SDK is a universal access tool to both things.
>> > > >>Point taken.  I think many other backup applications will expect a
>> > > >>similar API, so it's pragmatic to provide something compatible.
>> > > >Kevin Wolf and Daniel Berrange proposed an elegant way to avoid the
>> > > >concerns about the QMP monitor:
>> > > >
>> > > >Previously I described incremental backup in "push" mode (already
>> > > >supported today with drive-backup).  QEMU connects to a remote NBD
>> > > >server and writes out the contents of all dirty blocks:
>> > > >
>> > > >   QEMU ---Write dirty blocks--> Backup appliance (server)
>> > > >
>> > > >This doesn't lend itself well to existing backup applications that
>> > > >expect to iterate the dirty bitmap manually.
>> > > >
>> > > >Let's add a "pull" mode where the connection of the NBD connection is
>> > > >reversed.  The backup application connects to QEMU's NBD server (image
>> > > >fleecing).  The NBD protocol is extended to support the SCSI Get LBA
>> > > >Status command for querying block provisioning information.  Now the
>> > > >backup application can use Get LBA Status to fetch the dirty block
>> > > >information from QEMU.
>> > > >
>> > > >   QEMU (server) <--Get LBA Status or Read dirty blocks-- Backup 
>> > > > appliance
>> > > >
>> > > >The dirty block information goes over the same NBD connection used to
>> > > >read the contents of the dirty blocks.  The QMP monitor is not used to
>> > > >transfer dirty block information.
>> > > >
>> > > >It may be necessary to extend the nbd-server-add command so that a
>> > > >bitmap name can be passed.  This bitmap will be used to answer Get LBA
>> > > >Status queries instead of using on bdrv_co_get_block_status().  This
>> > > >would be necessary if image fleecing (point in time snapshot) is used.
>> > > >
>> > > >Stefan
>> > > 
>> > > There are no such commands in nbd spec here:
>> > > 
>> > > https://github.com/yoe/nbd/blob/master/doc/proto.md
>> > > 
>> > > 
>> > > So, I'm not sure, that adding something qemu-specific to this external
>> > > protocol will be simple or even true way. Is Qemu already extending 
>> > > original
>> > > nbd?
>> > 
>> > No, we don't do any QEMU specific extensions. The point of the approach
>> > Stefan suggests here though, is that it is *not* an inherantly 
>> > QEMU-specific
>> > concept, it is relevant to any NBD server implementation.
>> > 
>> > For example, consider you were using a regular NBD server to export a
>> > sparse LVM volume. This proposed extension would be relevant to such
>> > a use case. As such this proposed extension is something that is likely
>> > to be acceptable for the generic NBD specification.
>> 
>> Yes, Get LBA Status could be useful for non-QEMU NBD users too.
>> 
>> NBD already supports a TRIM command so the ability to query the
>> 

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Fam Zheng]

On Thu, 02/18 16:41, Stefan Hajnoczi wrote:
> On Thu, Feb 18, 2016 at 12:11:14PM +, Daniel P. Berrange wrote:
> > On Wed, Feb 17, 2016 at 08:47:11PM +0300, Vladimir Sementsov-Ogievskiy 
> > wrote:
> > > On 16.02.2016 20:09, Stefan Hajnoczi wrote:
> > > >On Wed, Feb 10, 2016 at 10:10:04AM +, Stefan Hajnoczi wrote:
> > > >>On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:
> > > >>>On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:
> > > On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
> > > >On 02/03/2016 11:14 AM, Fam Zheng wrote:
> > > >>On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
> > > >>>Hi all.
> > > >>>
> > > >>>These series which aims to add external backup api. This is needed 
> > > >>>to allow
> > > >>>backup software use our dirty bitmaps.
> > > >>>
> > > >>>Vmware and Parallels Cloud Server have this feature.
> > > >>What is the advantage of this appraoch over "drive-backup 
> > > >>sync=incremental
> > > >>..."?
> > > >This will allow third-party vendors to backup QEMU VMs into
> > > >their own formats or to the cloud etc.
> > > As an example, there is a third-party backup format called VMA from
> > > Proxmox.  A few years ago I posted a proof-of-concept external backup
> > > tool in Python:
> > > 
> > > https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html
> > > 
> > > It takes a full backup using drive-backup NBD (plus RAM/device state)
> > > but the same can be done with incremental backups.
> > > 
> > > Does this NBD approach meet your requirements?
> > > 
> > > Stefan
> > > >>>for us we should somehow provide implementation of
> > > >>>calls posted by Vladimir. They are available in Parallels Server
> > > >>>version 6 and should be available in the next QEMU based
> > > >>>release using "Parallels SDK to libvirt" convertor. The problem
> > > >>>for us is that this old approach is used in the other side
> > > >>>of the product - in containers implementation while this
> > > >>>SDK is a universal access tool to both things.
> > > >>Point taken.  I think many other backup applications will expect a
> > > >>similar API, so it's pragmatic to provide something compatible.
> > > >Kevin Wolf and Daniel Berrange proposed an elegant way to avoid the
> > > >concerns about the QMP monitor:
> > > >
> > > >Previously I described incremental backup in "push" mode (already
> > > >supported today with drive-backup).  QEMU connects to a remote NBD
> > > >server and writes out the contents of all dirty blocks:
> > > >
> > > >   QEMU ---Write dirty blocks--> Backup appliance (server)
> > > >
> > > >This doesn't lend itself well to existing backup applications that
> > > >expect to iterate the dirty bitmap manually.
> > > >
> > > >Let's add a "pull" mode where the connection of the NBD connection is
> > > >reversed.  The backup application connects to QEMU's NBD server (image
> > > >fleecing).  The NBD protocol is extended to support the SCSI Get LBA
> > > >Status command for querying block provisioning information.  Now the
> > > >backup application can use Get LBA Status to fetch the dirty block
> > > >information from QEMU.
> > > >
> > > >   QEMU (server) <--Get LBA Status or Read dirty blocks-- Backup 
> > > > appliance
> > > >
> > > >The dirty block information goes over the same NBD connection used to
> > > >read the contents of the dirty blocks.  The QMP monitor is not used to
> > > >transfer dirty block information.
> > > >
> > > >It may be necessary to extend the nbd-server-add command so that a
> > > >bitmap name can be passed.  This bitmap will be used to answer Get LBA
> > > >Status queries instead of using on bdrv_co_get_block_status().  This
> > > >would be necessary if image fleecing (point in time snapshot) is used.
> > > >
> > > >Stefan
> > > 
> > > There are no such commands in nbd spec here:
> > > 
> > > https://github.com/yoe/nbd/blob/master/doc/proto.md
> > > 
> > > 
> > > So, I'm not sure, that adding something qemu-specific to this external
> > > protocol will be simple or even true way. Is Qemu already extending 
> > > original
> > > nbd?
> > 
> > No, we don't do any QEMU specific extensions. The point of the approach
> > Stefan suggests here though, is that it is *not* an inherantly QEMU-specific
> > concept, it is relevant to any NBD server implementation.
> > 
> > For example, consider you were using a regular NBD server to export a
> > sparse LVM volume. This proposed extension would be relevant to such
> > a use case. As such this proposed extension is something that is likely
> > to be acceptable for the generic NBD specification.
> 
> Yes, Get LBA Status could be useful for non-QEMU NBD users too.
> 
> NBD already supports a TRIM command so the ability to query the
> allocation status is a natural feature to add.

Is it an abuse to "Get LBA Status" to return dirty information? Because in SCSI
the command reports 

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Markus Armbruster]

Stefan Hajnoczi  writes:

> On Tue, Feb 16, 2016 at 08:20:44PM +0300, Denis V. Lunev wrote:
>> the idea seems feasible at the first glance. I'll need to check
>> the amount of the overhead of this approach. If we'll need
>> to perform an additional request for each 
>> block - the overhead is too much.
>
> Regarding overhead, thankfully it's not one-block-at-a-time :).
>
>> Do you have the link to the spec/implementation?
>
> You can reference SCSI Block Commands (SBC-3) 5.4 GET LBA STATUS
> command.  The filename is sbc3r25.pdf and it's available from t10.org
> although other sites seem to host copies too.
>
> The approach is similar to the QMP command you have proposed.  Input
> parameters are "starting LBA" and "allocation length".  Outputs are a
> list of "status descriptors" which consists of an LBA, number of blocks,
> and a provisioning status field (allocated/unallocated/etc).
>
> This means a single call can retrieve information for a whole range of
> the disk.  It's slightly more efficient than QMP since the data is
> binary and not JSON text.

Moreover, it avoids abusing control-plane QMP as data plane.

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Stefan Hajnoczi]

On Thu, Feb 18, 2016 at 12:11:14PM +, Daniel P. Berrange wrote:
> On Wed, Feb 17, 2016 at 08:47:11PM +0300, Vladimir Sementsov-Ogievskiy wrote:
> > On 16.02.2016 20:09, Stefan Hajnoczi wrote:
> > >On Wed, Feb 10, 2016 at 10:10:04AM +, Stefan Hajnoczi wrote:
> > >>On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:
> > >>>On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:
> > On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
> > >On 02/03/2016 11:14 AM, Fam Zheng wrote:
> > >>On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
> > >>>Hi all.
> > >>>
> > >>>These series which aims to add external backup api. This is needed 
> > >>>to allow
> > >>>backup software use our dirty bitmaps.
> > >>>
> > >>>Vmware and Parallels Cloud Server have this feature.
> > >>What is the advantage of this appraoch over "drive-backup 
> > >>sync=incremental
> > >>..."?
> > >This will allow third-party vendors to backup QEMU VMs into
> > >their own formats or to the cloud etc.
> > As an example, there is a third-party backup format called VMA from
> > Proxmox.  A few years ago I posted a proof-of-concept external backup
> > tool in Python:
> > 
> > https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html
> > 
> > It takes a full backup using drive-backup NBD (plus RAM/device state)
> > but the same can be done with incremental backups.
> > 
> > Does this NBD approach meet your requirements?
> > 
> > Stefan
> > >>>for us we should somehow provide implementation of
> > >>>calls posted by Vladimir. They are available in Parallels Server
> > >>>version 6 and should be available in the next QEMU based
> > >>>release using "Parallels SDK to libvirt" convertor. The problem
> > >>>for us is that this old approach is used in the other side
> > >>>of the product - in containers implementation while this
> > >>>SDK is a universal access tool to both things.
> > >>Point taken.  I think many other backup applications will expect a
> > >>similar API, so it's pragmatic to provide something compatible.
> > >Kevin Wolf and Daniel Berrange proposed an elegant way to avoid the
> > >concerns about the QMP monitor:
> > >
> > >Previously I described incremental backup in "push" mode (already
> > >supported today with drive-backup).  QEMU connects to a remote NBD
> > >server and writes out the contents of all dirty blocks:
> > >
> > >   QEMU ---Write dirty blocks--> Backup appliance (server)
> > >
> > >This doesn't lend itself well to existing backup applications that
> > >expect to iterate the dirty bitmap manually.
> > >
> > >Let's add a "pull" mode where the connection of the NBD connection is
> > >reversed.  The backup application connects to QEMU's NBD server (image
> > >fleecing).  The NBD protocol is extended to support the SCSI Get LBA
> > >Status command for querying block provisioning information.  Now the
> > >backup application can use Get LBA Status to fetch the dirty block
> > >information from QEMU.
> > >
> > >   QEMU (server) <--Get LBA Status or Read dirty blocks-- Backup appliance
> > >
> > >The dirty block information goes over the same NBD connection used to
> > >read the contents of the dirty blocks.  The QMP monitor is not used to
> > >transfer dirty block information.
> > >
> > >It may be necessary to extend the nbd-server-add command so that a
> > >bitmap name can be passed.  This bitmap will be used to answer Get LBA
> > >Status queries instead of using on bdrv_co_get_block_status().  This
> > >would be necessary if image fleecing (point in time snapshot) is used.
> > >
> > >Stefan
> > 
> > There are no such commands in nbd spec here:
> > 
> > https://github.com/yoe/nbd/blob/master/doc/proto.md
> > 
> > 
> > So, I'm not sure, that adding something qemu-specific to this external
> > protocol will be simple or even true way. Is Qemu already extending original
> > nbd?
> 
> No, we don't do any QEMU specific extensions. The point of the approach
> Stefan suggests here though, is that it is *not* an inherantly QEMU-specific
> concept, it is relevant to any NBD server implementation.
> 
> For example, consider you were using a regular NBD server to export a
> sparse LVM volume. This proposed extension would be relevant to such
> a use case. As such this proposed extension is something that is likely
> to be acceptable for the generic NBD specification.

Yes, Get LBA Status could be useful for non-QEMU NBD users too.

NBD already supports a TRIM command so the ability to query the
allocation status is a natural feature to add.

Stefan


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Stefan Hajnoczi]

On Tue, Feb 16, 2016 at 08:20:44PM +0300, Denis V. Lunev wrote:
> the idea seems feasible at the first glance. I'll need to check
> the amount of the overhead of this approach. If we'll need
> to perform an additional request for each 
> block - the overhead is too much.

Regarding overhead, thankfully it's not one-block-at-a-time :).

> Do you have the link to the spec/implementation?

You can reference SCSI Block Commands (SBC-3) 5.4 GET LBA STATUS
command.  The filename is sbc3r25.pdf and it's available from t10.org
although other sites seem to host copies too.

The approach is similar to the QMP command you have proposed.  Input
parameters are "starting LBA" and "allocation length".  Outputs are a
list of "status descriptors" which consists of an LBA, number of blocks,
and a provisioning status field (allocated/unallocated/etc).

This means a single call can retrieve information for a whole range of
the disk.  It's slightly more efficient than QMP since the data is
binary and not JSON text.

Stefan


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Daniel P. Berrange]

On Wed, Feb 17, 2016 at 08:47:11PM +0300, Vladimir Sementsov-Ogievskiy wrote:
> On 16.02.2016 20:09, Stefan Hajnoczi wrote:
> >On Wed, Feb 10, 2016 at 10:10:04AM +, Stefan Hajnoczi wrote:
> >>On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:
> >>>On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:
> On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
> >On 02/03/2016 11:14 AM, Fam Zheng wrote:
> >>On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
> >>>Hi all.
> >>>
> >>>These series which aims to add external backup api. This is needed to 
> >>>allow
> >>>backup software use our dirty bitmaps.
> >>>
> >>>Vmware and Parallels Cloud Server have this feature.
> >>What is the advantage of this appraoch over "drive-backup 
> >>sync=incremental
> >>..."?
> >This will allow third-party vendors to backup QEMU VMs into
> >their own formats or to the cloud etc.
> As an example, there is a third-party backup format called VMA from
> Proxmox.  A few years ago I posted a proof-of-concept external backup
> tool in Python:
> 
> https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html
> 
> It takes a full backup using drive-backup NBD (plus RAM/device state)
> but the same can be done with incremental backups.
> 
> Does this NBD approach meet your requirements?
> 
> Stefan
> >>>for us we should somehow provide implementation of
> >>>calls posted by Vladimir. They are available in Parallels Server
> >>>version 6 and should be available in the next QEMU based
> >>>release using "Parallels SDK to libvirt" convertor. The problem
> >>>for us is that this old approach is used in the other side
> >>>of the product - in containers implementation while this
> >>>SDK is a universal access tool to both things.
> >>Point taken.  I think many other backup applications will expect a
> >>similar API, so it's pragmatic to provide something compatible.
> >Kevin Wolf and Daniel Berrange proposed an elegant way to avoid the
> >concerns about the QMP monitor:
> >
> >Previously I described incremental backup in "push" mode (already
> >supported today with drive-backup).  QEMU connects to a remote NBD
> >server and writes out the contents of all dirty blocks:
> >
> >   QEMU ---Write dirty blocks--> Backup appliance (server)
> >
> >This doesn't lend itself well to existing backup applications that
> >expect to iterate the dirty bitmap manually.
> >
> >Let's add a "pull" mode where the connection of the NBD connection is
> >reversed.  The backup application connects to QEMU's NBD server (image
> >fleecing).  The NBD protocol is extended to support the SCSI Get LBA
> >Status command for querying block provisioning information.  Now the
> >backup application can use Get LBA Status to fetch the dirty block
> >information from QEMU.
> >
> >   QEMU (server) <--Get LBA Status or Read dirty blocks-- Backup appliance
> >
> >The dirty block information goes over the same NBD connection used to
> >read the contents of the dirty blocks.  The QMP monitor is not used to
> >transfer dirty block information.
> >
> >It may be necessary to extend the nbd-server-add command so that a
> >bitmap name can be passed.  This bitmap will be used to answer Get LBA
> >Status queries instead of using on bdrv_co_get_block_status().  This
> >would be necessary if image fleecing (point in time snapshot) is used.
> >
> >Stefan
> 
> There are no such commands in nbd spec here:
> 
> https://github.com/yoe/nbd/blob/master/doc/proto.md
> 
> 
> So, I'm not sure, that adding something qemu-specific to this external
> protocol will be simple or even true way. Is Qemu already extending original
> nbd?

No, we don't do any QEMU specific extensions. The point of the approach
Stefan suggests here though, is that it is *not* an inherantly QEMU-specific
concept, it is relevant to any NBD server implementation.

For example, consider you were using a regular NBD server to export a
sparse LVM volume. This proposed extension would be relevant to such
a use case. As such this proposed extension is something that is likely
to be acceptable for the generic NBD specification.

> What about exporting bitmap as separate nbd entity? Just implement block
> driver, which will read from bitmap? If consider only read access and
> disabled (or frozen) bitmaps it should be simple enough.

IMHO that's not a desirable idea, since that *is* creating a QEMU specific
solution to the problem.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Fam Zheng]

On Wed, 02/17 20:47, Vladimir Sementsov-Ogievskiy wrote:
> What about exporting bitmap as separate nbd entity? Just implement
> block driver, which will read from bitmap? If consider only read
> access and disabled (or frozen) bitmaps it should be simple enough.

Yes, I think this idea also makes sense. The driver could implement .bdrv_read
by memcpy from the bitmap.

Fam

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Vladimir Sementsov-Ogievskiy]

On 16.02.2016 20:09, Stefan Hajnoczi wrote:

On Wed, Feb 10, 2016 at 10:10:04AM +, Stefan Hajnoczi wrote:

On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:

On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:

On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:

On 02/03/2016 11:14 AM, Fam Zheng wrote:

On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:

Hi all.

These series which aims to add external backup api. This is needed to allow
backup software use our dirty bitmaps.

Vmware and Parallels Cloud Server have this feature.

What is the advantage of this appraoch over "drive-backup sync=incremental
..."?

This will allow third-party vendors to backup QEMU VMs into
their own formats or to the cloud etc.

As an example, there is a third-party backup format called VMA from
Proxmox.  A few years ago I posted a proof-of-concept external backup
tool in Python:

https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html

It takes a full backup using drive-backup NBD (plus RAM/device state)
but the same can be done with incremental backups.

Does this NBD approach meet your requirements?

Stefan

for us we should somehow provide implementation of
calls posted by Vladimir. They are available in Parallels Server
version 6 and should be available in the next QEMU based
release using "Parallels SDK to libvirt" convertor. The problem
for us is that this old approach is used in the other side
of the product - in containers implementation while this
SDK is a universal access tool to both things.

Point taken.  I think many other backup applications will expect a
similar API, so it's pragmatic to provide something compatible.

Kevin Wolf and Daniel Berrange proposed an elegant way to avoid the
concerns about the QMP monitor:

Previously I described incremental backup in "push" mode (already
supported today with drive-backup).  QEMU connects to a remote NBD
server and writes out the contents of all dirty blocks:

   QEMU ---Write dirty blocks--> Backup appliance (server)

This doesn't lend itself well to existing backup applications that
expect to iterate the dirty bitmap manually.

Let's add a "pull" mode where the connection of the NBD connection is
reversed.  The backup application connects to QEMU's NBD server (image
fleecing).  The NBD protocol is extended to support the SCSI Get LBA
Status command for querying block provisioning information.  Now the
backup application can use Get LBA Status to fetch the dirty block
information from QEMU.

   QEMU (server) <--Get LBA Status or Read dirty blocks-- Backup appliance

The dirty block information goes over the same NBD connection used to
read the contents of the dirty blocks.  The QMP monitor is not used to
transfer dirty block information.

It may be necessary to extend the nbd-server-add command so that a
bitmap name can be passed.  This bitmap will be used to answer Get LBA
Status queries instead of using on bdrv_co_get_block_status().  This
would be necessary if image fleecing (point in time snapshot) is used.

Stefan


There are no such commands in nbd spec here:

https://github.com/yoe/nbd/blob/master/doc/proto.md


So, I'm not sure, that adding something qemu-specific to this external 
protocol will be simple or even true way. Is Qemu already extending 
original nbd?


What about exporting bitmap as separate nbd entity? Just implement block 
driver, which will read from bitmap? If consider only read access and 
disabled (or frozen) bitmaps it should be simple enough.


--
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Denis V. Lunev]

On 02/16/2016 08:09 PM, Stefan Hajnoczi wrote:

On Wed, Feb 10, 2016 at 10:10:04AM +, Stefan Hajnoczi wrote:

On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:

On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:

On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:

On 02/03/2016 11:14 AM, Fam Zheng wrote:

On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:

Hi all.

These series which aims to add external backup api. This is needed to allow
backup software use our dirty bitmaps.

Vmware and Parallels Cloud Server have this feature.

What is the advantage of this appraoch over "drive-backup sync=incremental
..."?

This will allow third-party vendors to backup QEMU VMs into
their own formats or to the cloud etc.

As an example, there is a third-party backup format called VMA from
Proxmox.  A few years ago I posted a proof-of-concept external backup
tool in Python:

https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html

It takes a full backup using drive-backup NBD (plus RAM/device state)
but the same can be done with incremental backups.

Does this NBD approach meet your requirements?

Stefan

for us we should somehow provide implementation of
calls posted by Vladimir. They are available in Parallels Server
version 6 and should be available in the next QEMU based
release using "Parallels SDK to libvirt" convertor. The problem
for us is that this old approach is used in the other side
of the product - in containers implementation while this
SDK is a universal access tool to both things.

Point taken.  I think many other backup applications will expect a
similar API, so it's pragmatic to provide something compatible.

Kevin Wolf and Daniel Berrange proposed an elegant way to avoid the
concerns about the QMP monitor:

Previously I described incremental backup in "push" mode (already
supported today with drive-backup).  QEMU connects to a remote NBD
server and writes out the contents of all dirty blocks:

   QEMU ---Write dirty blocks--> Backup appliance (server)

This doesn't lend itself well to existing backup applications that
expect to iterate the dirty bitmap manually.

Let's add a "pull" mode where the connection of the NBD connection is
reversed.  The backup application connects to QEMU's NBD server (image
fleecing).  The NBD protocol is extended to support the SCSI Get LBA
Status command for querying block provisioning information.  Now the
backup application can use Get LBA Status to fetch the dirty block
information from QEMU.

   QEMU (server) <--Get LBA Status or Read dirty blocks-- Backup appliance

The dirty block information goes over the same NBD connection used to
read the contents of the dirty blocks.  The QMP monitor is not used to
transfer dirty block information.

It may be necessary to extend the nbd-server-add command so that a
bitmap name can be passed.  This bitmap will be used to answer Get LBA
Status queries instead of using on bdrv_co_get_block_status().  This
would be necessary if image fleecing (point in time snapshot) is used.

Stefan

the idea seems feasible at the first glance. I'll need to check
the amount of the overhead of this approach. If we'll need
to perform an additional request for each 
block - the overhead is too much.

Do you have the link to the spec/implementation?

Den

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Vladimir Sementsov-Ogievskiy]

On 16.02.2016 20:09, Stefan Hajnoczi wrote:

On Wed, Feb 10, 2016 at 10:10:04AM +, Stefan Hajnoczi wrote:

On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:

On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:

On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:

On 02/03/2016 11:14 AM, Fam Zheng wrote:

On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:

Hi all.

These series which aims to add external backup api. This is needed to allow
backup software use our dirty bitmaps.

Vmware and Parallels Cloud Server have this feature.

What is the advantage of this appraoch over "drive-backup sync=incremental
..."?

This will allow third-party vendors to backup QEMU VMs into
their own formats or to the cloud etc.

As an example, there is a third-party backup format called VMA from
Proxmox.  A few years ago I posted a proof-of-concept external backup
tool in Python:

https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html

It takes a full backup using drive-backup NBD (plus RAM/device state)
but the same can be done with incremental backups.

Does this NBD approach meet your requirements?

Stefan

for us we should somehow provide implementation of
calls posted by Vladimir. They are available in Parallels Server
version 6 and should be available in the next QEMU based
release using "Parallels SDK to libvirt" convertor. The problem
for us is that this old approach is used in the other side
of the product - in containers implementation while this
SDK is a universal access tool to both things.

Point taken.  I think many other backup applications will expect a
similar API, so it's pragmatic to provide something compatible.

Kevin Wolf and Daniel Berrange proposed an elegant way to avoid the
concerns about the QMP monitor:

Previously I described incremental backup in "push" mode (already
supported today with drive-backup).  QEMU connects to a remote NBD
server and writes out the contents of all dirty blocks:

   QEMU ---Write dirty blocks--> Backup appliance (server)

This doesn't lend itself well to existing backup applications that
expect to iterate the dirty bitmap manually.

Let's add a "pull" mode where the connection of the NBD connection is
reversed.  The backup application connects to QEMU's NBD server (image
fleecing).  The NBD protocol is extended to support the SCSI Get LBA
Status command for querying block provisioning information.  Now the
backup application can use Get LBA Status to fetch the dirty block
information from QEMU.

   QEMU (server) <--Get LBA Status or Read dirty blocks-- Backup appliance

The dirty block information goes over the same NBD connection used to
read the contents of the dirty blocks.  The QMP monitor is not used to
transfer dirty block information.

It may be necessary to extend the nbd-server-add command so that a
bitmap name can be passed.  This bitmap will be used to answer Get LBA
Status queries instead of using on bdrv_co_get_block_status().  This
would be necessary if image fleecing (point in time snapshot) is used.

Stefan


Looks much easier than implement nbd-server for dirty bitmap, thanks.

--
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Stefan Hajnoczi]

On Wed, Feb 10, 2016 at 10:10:04AM +, Stefan Hajnoczi wrote:
> On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:
> > On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:
> > >On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
> > >>On 02/03/2016 11:14 AM, Fam Zheng wrote:
> > >>>On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
> > Hi all.
> > 
> > These series which aims to add external backup api. This is needed to 
> > allow
> > backup software use our dirty bitmaps.
> > 
> > Vmware and Parallels Cloud Server have this feature.
> > >>>What is the advantage of this appraoch over "drive-backup 
> > >>>sync=incremental
> > >>>..."?
> > >>This will allow third-party vendors to backup QEMU VMs into
> > >>their own formats or to the cloud etc.
> > >As an example, there is a third-party backup format called VMA from
> > >Proxmox.  A few years ago I posted a proof-of-concept external backup
> > >tool in Python:
> > >
> > >https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html
> > >
> > >It takes a full backup using drive-backup NBD (plus RAM/device state)
> > >but the same can be done with incremental backups.
> > >
> > >Does this NBD approach meet your requirements?
> > >
> > >Stefan
> > for us we should somehow provide implementation of
> > calls posted by Vladimir. They are available in Parallels Server
> > version 6 and should be available in the next QEMU based
> > release using "Parallels SDK to libvirt" convertor. The problem
> > for us is that this old approach is used in the other side
> > of the product - in containers implementation while this
> > SDK is a universal access tool to both things.
> 
> Point taken.  I think many other backup applications will expect a
> similar API, so it's pragmatic to provide something compatible.

Kevin Wolf and Daniel Berrange proposed an elegant way to avoid the
concerns about the QMP monitor:

Previously I described incremental backup in "push" mode (already
supported today with drive-backup).  QEMU connects to a remote NBD
server and writes out the contents of all dirty blocks:

  QEMU ---Write dirty blocks--> Backup appliance (server)

This doesn't lend itself well to existing backup applications that
expect to iterate the dirty bitmap manually.

Let's add a "pull" mode where the connection of the NBD connection is
reversed.  The backup application connects to QEMU's NBD server (image
fleecing).  The NBD protocol is extended to support the SCSI Get LBA
Status command for querying block provisioning information.  Now the
backup application can use Get LBA Status to fetch the dirty block
information from QEMU.

  QEMU (server) <--Get LBA Status or Read dirty blocks-- Backup appliance

The dirty block information goes over the same NBD connection used to
read the contents of the dirty blocks.  The QMP monitor is not used to
transfer dirty block information.

It may be necessary to extend the nbd-server-add command so that a
bitmap name can be passed.  This bitmap will be used to answer Get LBA
Status queries instead of using on bdrv_co_get_block_status().  This
would be necessary if image fleecing (point in time snapshot) is used.

Stefan


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Denis V. Lunev]

On 02/09/2016 10:25 PM, Denis V. Lunev wrote:

On 02/09/2016 09:12 PM, John Snow wrote:


On 02/09/2016 11:58 AM, Denis V. Lunev wrote:

On 02/09/2016 07:49 PM, John Snow wrote:

On 02/09/2016 09:37 AM, Denis V. Lunev wrote:

On 02/09/2016 05:21 PM, Stefan Hajnoczi wrote:

On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:

On 02/03/2016 11:14 AM, Fam Zheng wrote:

On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:

Hi all.

These series which aims to add external backup api. This is 
needed

to allow
backup software use our dirty bitmaps.

Vmware and Parallels Cloud Server have this feature.

What is the advantage of this appraoch over "drive-backup
sync=incremental
..."?

This will allow third-party vendors to backup QEMU VMs into
their own formats or to the cloud etc.

Backup software can implement NBD to receive the incremental blocks
from
QEMU.  Use drive-backup sync=incremental and the backup appliance as
the
NBD target.

It's more complicated to add this QMP command flow to backup 
software

than to implement NBD.

Stefan

it can, but this is a matter of problem due to the nature of
how this software is implemented. Usually it is written
in a semi-standard way and it uses "plugins" to actually
collect the data, i.e. the code is written in standard
interface/real implementation pattern and interfaces are
basically the same.

With this standard approach backup software is working
as an active side of the process, i.e. it performs operations
and controls the flow.

This means that "non-standard" QEMU technology will be
pain here.

Den

Am I to understand that for e.g. VMWare the backup appliance is
literally reading the disk image from storage directly while the VM is
running?

I'd be a bit surprised if that were true.

I think that backup software is asking alive VM about the data.

My biggest concern here is that there is not a safe way, today, to 
read

from a disk image atomically while the VM is running. I think that'd
take a lot of work to do and you'll not find a lot of support in
implementing it.

Of course, while the VM is paused/off is a different story, but for 
now

I still feel like NBD is the right answer for getting block data from
QEMU.

What am I missing?

--js

In general, in Parallels Server the backup was created using the
following approach:
- create external snapshot. In this case the base image (backing store
in QEMU terminology)
   will be READ-ONLY and could be safely read by any entity
- backup that read-only disk image (any way)
- merge snapshots


I see.


In this process backup software is active while PCS is passive.

PCS?

Parallels Cloud Server. Sorry for abbreviation :(



With QEMU the approach looks the same to me:
- start a backup
- ask QEMU to give a data to be backuped (using NBD server in QEMU
   with Fam's image fleecing)
- finish backup

Important bit here is that dirty bitmap should be provided
by QEMU by request. This dirty bitmap will be read-only at that
moment, current active dirty bitmap should be replaced with
new at backup start operation.

Den

I don't have any problems providing the bitmap data through an external
API, but the part I want to be 100% clear on before I ACK it is the API
portion where we allow an external client to split or merge bitmaps
externally -- that's functionality you don't need if you query the data
from QEMU itself.

That is fine :)

OOPS. This could be mistake!

The client should not perform operations on the bitmap manually,
but it should instruct QEMU to do this job for him. Bitmap child
should be created by request of backup software and its existence
should also be controlled by the backup software. This could be either
hidden or not, but the state should be controlled.

Lets again discuss the sequence:
1. call guest-fsfreeze-freeze to make consistent backup. This means
that the guest ensures that all its journals are committed and
there are no pending writes. Filesystem is in a good state.

2. start backup. At this moment original dirty bitmap child should
  be created to track new writes. There are no writes from the
  guest thanks to step (1). Thus we could avoid to move QEMU
  to VM_PAUSED state to perform the operation. This requires to
  2.1 start image fleecing
  2.2 create bitmap child (successor)

3. call guest-fsfreeze-thaw to unfreeze the guest

4. get dirty bitmap (parent). It is read-only and could be safely extracted

5. collect data from QEMU using NBD server inside QEMU (image
fleecing)

6a. finish backup successfully. This implies the following ops:
   6a.1  remove fleecing image
   6a.2 drop old dirty (parent) bitmap (abdicate)

6b. finish backup unsuccessfully:
   6b.1 remove fleecing image
   6b.2(a) merge child to parent (reclaim)
or alternatively
   6b.2(b) drop both child and parent dirty bitmaps and stop tracking
to facilitate new full backup next round

Constraints:
- step (4) is slow. We do not want to do it before step (3)
- paired operations aka 

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Stefan Hajnoczi]

On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:
> On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:
> >On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
> >>On 02/03/2016 11:14 AM, Fam Zheng wrote:
> >>>On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
> Hi all.
> 
> These series which aims to add external backup api. This is needed to 
> allow
> backup software use our dirty bitmaps.
> 
> Vmware and Parallels Cloud Server have this feature.
> >>>What is the advantage of this appraoch over "drive-backup sync=incremental
> >>>..."?
> >>This will allow third-party vendors to backup QEMU VMs into
> >>their own formats or to the cloud etc.
> >As an example, there is a third-party backup format called VMA from
> >Proxmox.  A few years ago I posted a proof-of-concept external backup
> >tool in Python:
> >
> >https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html
> >
> >It takes a full backup using drive-backup NBD (plus RAM/device state)
> >but the same can be done with incremental backups.
> >
> >Does this NBD approach meet your requirements?
> >
> >Stefan
> for us we should somehow provide implementation of
> calls posted by Vladimir. They are available in Parallels Server
> version 6 and should be available in the next QEMU based
> release using "Parallels SDK to libvirt" convertor. The problem
> for us is that this old approach is used in the other side
> of the product - in containers implementation while this
> SDK is a universal access tool to both things.

Point taken.  I think many other backup applications will expect a
similar API, so it's pragmatic to provide something compatible.

Stefan


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Denis V. Lunev]

On 02/09/2016 09:12 PM, John Snow wrote:


On 02/09/2016 11:58 AM, Denis V. Lunev wrote:

On 02/09/2016 07:49 PM, John Snow wrote:

On 02/09/2016 09:37 AM, Denis V. Lunev wrote:

On 02/09/2016 05:21 PM, Stefan Hajnoczi wrote:

On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:

On 02/03/2016 11:14 AM, Fam Zheng wrote:

On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:

Hi all.

These series which aims to add external backup api. This is needed
to allow
backup software use our dirty bitmaps.

Vmware and Parallels Cloud Server have this feature.

What is the advantage of this appraoch over "drive-backup
sync=incremental
..."?

This will allow third-party vendors to backup QEMU VMs into
their own formats or to the cloud etc.

Backup software can implement NBD to receive the incremental blocks
from
QEMU.  Use drive-backup sync=incremental and the backup appliance as
the
NBD target.

It's more complicated to add this QMP command flow to backup software
than to implement NBD.

Stefan

it can, but this is a matter of problem due to the nature of
how this software is implemented. Usually it is written
in a semi-standard way and it uses "plugins" to actually
collect the data, i.e. the code is written in standard
interface/real implementation pattern and interfaces are
basically the same.

With this standard approach backup software is working
as an active side of the process, i.e. it performs operations
and controls the flow.

This means that "non-standard" QEMU technology will be
pain here.

Den

Am I to understand that for e.g. VMWare the backup appliance is
literally reading the disk image from storage directly while the VM is
running?

I'd be a bit surprised if that were true.

I think that backup software is asking alive VM about the data.


My biggest concern here is that there is not a safe way, today, to read
from a disk image atomically while the VM is running. I think that'd
take a lot of work to do and you'll not find a lot of support in
implementing it.

Of course, while the VM is paused/off is a different story, but for now
I still feel like NBD is the right answer for getting block data from
QEMU.

What am I missing?

--js

In general, in Parallels Server the backup was created using the
following approach:
- create external snapshot. In this case the base image (backing store
in QEMU terminology)
   will be READ-ONLY and could be safely read by any entity
- backup that read-only disk image (any way)
- merge snapshots


I see.


In this process backup software is active while PCS is passive.

PCS?

Parallels Cloud Server. Sorry for abbreviation :(



With QEMU the approach looks the same to me:
- start a backup
- ask QEMU to give a data to be backuped (using NBD server in QEMU
   with Fam's image fleecing)
- finish backup

Important bit here is that dirty bitmap should be provided
by QEMU by request. This dirty bitmap will be read-only at that
moment, current active dirty bitmap should be replaced with
new at backup start operation.

Den

I don't have any problems providing the bitmap data through an external
API, but the part I want to be 100% clear on before I ACK it is the API
portion where we allow an external client to split or merge bitmaps
externally -- that's functionality you don't need if you query the data
from QEMU itself.

That is fine :)


If using NBD (either through the incremental backup tool or image
fleecing) that is completely fine, too.

My only reservations are over making direct reads to image files outside
of QEMU while the VM is running.


Perfect :) We are on the same page. I would not
going to go something unsafe is this very sensitive
area.

Den

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Denis V. Lunev]

On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:

On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:

On 02/03/2016 11:14 AM, Fam Zheng wrote:

On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:

Hi all.

These series which aims to add external backup api. This is needed to allow
backup software use our dirty bitmaps.

Vmware and Parallels Cloud Server have this feature.

What is the advantage of this appraoch over "drive-backup sync=incremental
..."?

This will allow third-party vendors to backup QEMU VMs into
their own formats or to the cloud etc.

As an example, there is a third-party backup format called VMA from
Proxmox.  A few years ago I posted a proof-of-concept external backup
tool in Python:

https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html

It takes a full backup using drive-backup NBD (plus RAM/device state)
but the same can be done with incremental backups.

Does this NBD approach meet your requirements?

Stefan

for us we should somehow provide implementation of
calls posted by Vladimir. They are available in Parallels Server
version 6 and should be available in the next QEMU based
release using "Parallels SDK to libvirt" convertor. The problem
for us is that this old approach is used in the other side
of the product - in containers implementation while this
SDK is a universal access tool to both things.

Den

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Denis V. Lunev]

On 02/09/2016 05:21 PM, Stefan Hajnoczi wrote:

On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:

On 02/03/2016 11:14 AM, Fam Zheng wrote:

On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:

Hi all.

These series which aims to add external backup api. This is needed to allow
backup software use our dirty bitmaps.

Vmware and Parallels Cloud Server have this feature.

What is the advantage of this appraoch over "drive-backup sync=incremental
..."?

This will allow third-party vendors to backup QEMU VMs into
their own formats or to the cloud etc.

Backup software can implement NBD to receive the incremental blocks from
QEMU.  Use drive-backup sync=incremental and the backup appliance as the
NBD target.

It's more complicated to add this QMP command flow to backup software
than to implement NBD.

Stefan

it can, but this is a matter of problem due to the nature of
how this software is implemented. Usually it is written
in a semi-standard way and it uses "plugins" to actually
collect the data, i.e. the code is written in standard
interface/real implementation pattern and interfaces are
basically the same.

With this standard approach backup software is working
as an active side of the process, i.e. it performs operations
and controls the flow.

This means that "non-standard" QEMU technology will be
pain here.

Den

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Stefan Hajnoczi]

On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
> On 02/03/2016 11:14 AM, Fam Zheng wrote:
> >On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
> >>Hi all.
> >>
> >>These series which aims to add external backup api. This is needed to allow
> >>backup software use our dirty bitmaps.
> >>
> >>Vmware and Parallels Cloud Server have this feature.
> >What is the advantage of this appraoch over "drive-backup sync=incremental
> >..."?
> 
> This will allow third-party vendors to backup QEMU VMs into
> their own formats or to the cloud etc.

Backup software can implement NBD to receive the incremental blocks from
QEMU.  Use drive-backup sync=incremental and the backup appliance as the
NBD target.

It's more complicated to add this QMP command flow to backup software
than to implement NBD.

Stefan


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Stefan Hajnoczi]

On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
> On 02/03/2016 11:14 AM, Fam Zheng wrote:
> >On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
> >>Hi all.
> >>
> >>These series which aims to add external backup api. This is needed to allow
> >>backup software use our dirty bitmaps.
> >>
> >>Vmware and Parallels Cloud Server have this feature.
> >What is the advantage of this appraoch over "drive-backup sync=incremental
> >..."?
> 
> This will allow third-party vendors to backup QEMU VMs into
> their own formats or to the cloud etc.

As an example, there is a third-party backup format called VMA from
Proxmox.  A few years ago I posted a proof-of-concept external backup
tool in Python:

https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html

It takes a full backup using drive-backup NBD (plus RAM/device state)
but the same can be done with incremental backups.

Does this NBD approach meet your requirements?

Stefan


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Denis V. Lunev]

On 02/09/2016 07:49 PM, John Snow wrote:


On 02/09/2016 09:37 AM, Denis V. Lunev wrote:

On 02/09/2016 05:21 PM, Stefan Hajnoczi wrote:

On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:

On 02/03/2016 11:14 AM, Fam Zheng wrote:

On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:

Hi all.

These series which aims to add external backup api. This is needed
to allow
backup software use our dirty bitmaps.

Vmware and Parallels Cloud Server have this feature.

What is the advantage of this appraoch over "drive-backup
sync=incremental
..."?

This will allow third-party vendors to backup QEMU VMs into
their own formats or to the cloud etc.

Backup software can implement NBD to receive the incremental blocks from
QEMU.  Use drive-backup sync=incremental and the backup appliance as the
NBD target.

It's more complicated to add this QMP command flow to backup software
than to implement NBD.

Stefan

it can, but this is a matter of problem due to the nature of
how this software is implemented. Usually it is written
in a semi-standard way and it uses "plugins" to actually
collect the data, i.e. the code is written in standard
interface/real implementation pattern and interfaces are
basically the same.

With this standard approach backup software is working
as an active side of the process, i.e. it performs operations
and controls the flow.

This means that "non-standard" QEMU technology will be
pain here.

Den

Am I to understand that for e.g. VMWare the backup appliance is
literally reading the disk image from storage directly while the VM is
running?

I'd be a bit surprised if that were true.

I think that backup software is asking alive VM about the data.


My biggest concern here is that there is not a safe way, today, to read
from a disk image atomically while the VM is running. I think that'd
take a lot of work to do and you'll not find a lot of support in
implementing it.

Of course, while the VM is paused/off is a different story, but for now
I still feel like NBD is the right answer for getting block data from QEMU.

What am I missing?

--js

In general, in Parallels Server the backup was created using the
following approach:
- create external snapshot. In this case the base image (backing store 
in QEMU terminology)

  will be READ-ONLY and could be safely read by any entity
- backup that read-only disk image (any way)
- merge snapshots

In this process backup software is active while PCS is passive.
With QEMU the approach looks the same to me:
- start a backup
- ask QEMU to give a data to be backuped (using NBD server in QEMU
  with Fam's image fleecing)
- finish backup

Important bit here is that dirty bitmap should be provided
by QEMU by request. This dirty bitmap will be read-only at that
moment, current active dirty bitmap should be replaced with
new at backup start operation.

Den

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[John Snow]

On 02/09/2016 09:37 AM, Denis V. Lunev wrote:
> On 02/09/2016 05:21 PM, Stefan Hajnoczi wrote:
>> On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
>>> On 02/03/2016 11:14 AM, Fam Zheng wrote:
 On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
> Hi all.
>
> These series which aims to add external backup api. This is needed
> to allow
> backup software use our dirty bitmaps.
>
> Vmware and Parallels Cloud Server have this feature.
 What is the advantage of this appraoch over "drive-backup
 sync=incremental
 ..."?
>>> This will allow third-party vendors to backup QEMU VMs into
>>> their own formats or to the cloud etc.
>> Backup software can implement NBD to receive the incremental blocks from
>> QEMU.  Use drive-backup sync=incremental and the backup appliance as the
>> NBD target.
>>
>> It's more complicated to add this QMP command flow to backup software
>> than to implement NBD.
>>
>> Stefan
> it can, but this is a matter of problem due to the nature of
> how this software is implemented. Usually it is written
> in a semi-standard way and it uses "plugins" to actually
> collect the data, i.e. the code is written in standard
> interface/real implementation pattern and interfaces are
> basically the same.
> 
> With this standard approach backup software is working
> as an active side of the process, i.e. it performs operations
> and controls the flow.
> 
> This means that "non-standard" QEMU technology will be
> pain here.
> 
> Den

Am I to understand that for e.g. VMWare the backup appliance is
literally reading the disk image from storage directly while the VM is
running?

I'd be a bit surprised if that were true.

My biggest concern here is that there is not a safe way, today, to read
from a disk image atomically while the VM is running. I think that'd
take a lot of work to do and you'll not find a lot of support in
implementing it.

Of course, while the VM is paused/off is a different story, but for now
I still feel like NBD is the right answer for getting block data from QEMU.

What am I missing?

--js

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[John Snow]

On 02/09/2016 11:58 AM, Denis V. Lunev wrote:
> On 02/09/2016 07:49 PM, John Snow wrote:
>>
>> On 02/09/2016 09:37 AM, Denis V. Lunev wrote:
>>> On 02/09/2016 05:21 PM, Stefan Hajnoczi wrote:
 On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
> On 02/03/2016 11:14 AM, Fam Zheng wrote:
>> On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
>>> Hi all.
>>>
>>> These series which aims to add external backup api. This is needed
>>> to allow
>>> backup software use our dirty bitmaps.
>>>
>>> Vmware and Parallels Cloud Server have this feature.
>> What is the advantage of this appraoch over "drive-backup
>> sync=incremental
>> ..."?
> This will allow third-party vendors to backup QEMU VMs into
> their own formats or to the cloud etc.
 Backup software can implement NBD to receive the incremental blocks
 from
 QEMU.  Use drive-backup sync=incremental and the backup appliance as
 the
 NBD target.

 It's more complicated to add this QMP command flow to backup software
 than to implement NBD.

 Stefan
>>> it can, but this is a matter of problem due to the nature of
>>> how this software is implemented. Usually it is written
>>> in a semi-standard way and it uses "plugins" to actually
>>> collect the data, i.e. the code is written in standard
>>> interface/real implementation pattern and interfaces are
>>> basically the same.
>>>
>>> With this standard approach backup software is working
>>> as an active side of the process, i.e. it performs operations
>>> and controls the flow.
>>>
>>> This means that "non-standard" QEMU technology will be
>>> pain here.
>>>
>>> Den
>> Am I to understand that for e.g. VMWare the backup appliance is
>> literally reading the disk image from storage directly while the VM is
>> running?
>>
>> I'd be a bit surprised if that were true.
> I think that backup software is asking alive VM about the data.
> 
>> My biggest concern here is that there is not a safe way, today, to read
>> from a disk image atomically while the VM is running. I think that'd
>> take a lot of work to do and you'll not find a lot of support in
>> implementing it.
>>
>> Of course, while the VM is paused/off is a different story, but for now
>> I still feel like NBD is the right answer for getting block data from
>> QEMU.
>>
>> What am I missing?
>>
>> --js
> In general, in Parallels Server the backup was created using the
> following approach:
> - create external snapshot. In this case the base image (backing store
> in QEMU terminology)
>   will be READ-ONLY and could be safely read by any entity
> - backup that read-only disk image (any way)
> - merge snapshots
> 

I see.

> In this process backup software is active while PCS is passive.

PCS?

> With QEMU the approach looks the same to me:
> - start a backup
> - ask QEMU to give a data to be backuped (using NBD server in QEMU
>   with Fam's image fleecing)
> - finish backup
> 
> Important bit here is that dirty bitmap should be provided
> by QEMU by request. This dirty bitmap will be read-only at that
> moment, current active dirty bitmap should be replaced with
> new at backup start operation.
> 
> Den

I don't have any problems providing the bitmap data through an external
API, but the part I want to be 100% clear on before I ACK it is the API
portion where we allow an external client to split or merge bitmaps
externally -- that's functionality you don't need if you query the data
from QEMU itself.

If using NBD (either through the incremental backup tool or image
fleecing) that is completely fine, too.

My only reservations are over making direct reads to image files outside
of QEMU while the VM is running.

--js

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Denis V. Lunev]

On 02/03/2016 11:14 AM, Fam Zheng wrote:

On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:

Hi all.

These series which aims to add external backup api. This is needed to allow
backup software use our dirty bitmaps.

Vmware and Parallels Cloud Server have this feature.

What is the advantage of this appraoch over "drive-backup sync=incremental
..."?


This will allow third-party vendors to backup QEMU VMs into
their own formats or to the cloud etc.

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Fam Zheng]

On Fri, 02/05 11:28, Denis V. Lunev wrote:
> On 02/03/2016 11:14 AM, Fam Zheng wrote:
> >On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
> >>Hi all.
> >>
> >>These series which aims to add external backup api. This is needed to allow
> >>backup software use our dirty bitmaps.
> >>
> >>Vmware and Parallels Cloud Server have this feature.
> >What is the advantage of this appraoch over "drive-backup sync=incremental
> >..."?
> 
> This will allow third-party vendors to backup QEMU VMs into
> their own formats or to the cloud etc.

Yes, I see, thanks for explaining.

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Fam Zheng]

On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
> Hi all.
> 
> These series which aims to add external backup api. This is needed to allow
> backup software use our dirty bitmaps.
> 
> Vmware and Parallels Cloud Server have this feature.

What is the advantage of this appraoch over "drive-backup sync=incremental
..."?

> 
> There are three things are done:
> - add query-block-dirty-bitmap-ranges qmp command
> - add qmp commands for dirty-bitmap functions: create_successor, abdicate,
>   reclaim.
> - make create-successor command transaction-able
> 
> Then, external backup should be done like this:
> 
> 1.  qmp transaction {
> external-snapshot
> bitmap-create-successor
> }
> 
> 2.  qmp query frozen bitmap, not acquiring aio context.

What do you do with the query response, read the snapshot image with an
external tool?

Fam

> 
> 3.  do external backup, using snapshot and bitmap
> 
> 
> 4.  if (success backup)
> qmp bitmap-abdicate
> else
> qmp bitmap-reclaime
> 
> 5.  qmp merge snapshot
> 
> 
> v2: a lot of additions and changes, no sense to compare with v1
> 
> 
> Vladimir Sementsov-Ogievskiy (6):
>   block dirty bitmap: add next_zero function
>   qmp: add query-block-dirty-bitmap-ranges
>   iotests: test query-block-dirty-bitmap-ranges
>   qapi: add qmp commands for some dirty bitmap functions
>   qapi: make block-dirty-bitmap-create-successor transaction-able
>   iotests: test external backup api
> 
>  block/dirty-bitmap.c |  60 ++
>  blockdev.c   | 113 +
>  include/block/dirty-bitmap.h |   9 
>  include/qemu/hbitmap.h   |   8 +++
>  qapi-schema.json |   4 +-
>  qapi/block-core.json |  90 +
>  qmp-commands.hx  | 118 
> +++
>  tests/qemu-iotests/150   |  88 
>  tests/qemu-iotests/150.out   |  21 
>  tests/qemu-iotests/151   |  77 
>  tests/qemu-iotests/151.out   |   7 +++
>  tests/qemu-iotests/group |   2 +
>  util/hbitmap.c   |  26 ++
>  13 files changed, 622 insertions(+), 1 deletion(-)
>  create mode 100755 tests/qemu-iotests/150
>  create mode 100644 tests/qemu-iotests/150.out
>  create mode 100755 tests/qemu-iotests/151
>  create mode 100644 tests/qemu-iotests/151.out
> 
> -- 
> 1.8.3.1
> 

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Fam Zheng]

On Wed, 02/03 13:57, Vladimir Sementsov-Ogievskiy wrote:
> On 03.02.2016 11:14, Fam Zheng wrote:
> >On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
> >>Hi all.
> >>
> >>These series which aims to add external backup api. This is needed to allow
> >>backup software use our dirty bitmaps.
> >>
> >>Vmware and Parallels Cloud Server have this feature.
> >What is the advantage of this appraoch over "drive-backup sync=incremental
> >..."?
> 
> Hmm, you are asking about advantage of external backup over
> internal? It depends on external backup tool.

I'm asking why the tool can't use drive-backup to achieve what you want. They
seem very comparable.

Fam

> 
> >
> >>There are three things are done:
> >>- add query-block-dirty-bitmap-ranges qmp command
> >>- add qmp commands for dirty-bitmap functions: create_successor, abdicate,
> >>   reclaim.
> >>- make create-successor command transaction-able
> >>
> >>Then, external backup should be done like this:
> >>
> >>1.  qmp transaction {
> >> external-snapshot
> >> bitmap-create-successor
> >> }
> >>
> >>2.  qmp query frozen bitmap, not acquiring aio context.
> >What do you do with the query response, read the snapshot image with an
> >external tool?
> 
> Yes. Alternatively, image fleecing may be used instead of snapshot.
> 
> >
> >Fam
> >
> >>3.  do external backup, using snapshot and bitmap
> >>
> >>
> >>4.  if (success backup)
> >> qmp bitmap-abdicate
> >> else
> >> qmp bitmap-reclaime
> >>
> >>5.  qmp merge snapshot
> >>
> >>
> >>v2: a lot of additions and changes, no sense to compare with v1
> >>
> >>
> >>Vladimir Sementsov-Ogievskiy (6):
> >>   block dirty bitmap: add next_zero function
> >>   qmp: add query-block-dirty-bitmap-ranges
> >>   iotests: test query-block-dirty-bitmap-ranges
> >>   qapi: add qmp commands for some dirty bitmap functions
> >>   qapi: make block-dirty-bitmap-create-successor transaction-able
> >>   iotests: test external backup api
> >>
> >>  block/dirty-bitmap.c |  60 ++
> >>  blockdev.c   | 113 
> >> +
> >>  include/block/dirty-bitmap.h |   9 
> >>  include/qemu/hbitmap.h   |   8 +++
> >>  qapi-schema.json |   4 +-
> >>  qapi/block-core.json |  90 +
> >>  qmp-commands.hx  | 118 
> >> +++
> >>  tests/qemu-iotests/150   |  88 
> >>  tests/qemu-iotests/150.out   |  21 
> >>  tests/qemu-iotests/151   |  77 
> >>  tests/qemu-iotests/151.out   |   7 +++
> >>  tests/qemu-iotests/group |   2 +
> >>  util/hbitmap.c   |  26 ++
> >>  13 files changed, 622 insertions(+), 1 deletion(-)
> >>  create mode 100755 tests/qemu-iotests/150
> >>  create mode 100644 tests/qemu-iotests/150.out
> >>  create mode 100755 tests/qemu-iotests/151
> >>  create mode 100644 tests/qemu-iotests/151.out
> >>
> >>-- 
> >>1.8.3.1
> >>
> 
> 
> -- 
> Best regards,
> Vladimir
> 

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Vladimir Sementsov-Ogievskiy]

On 03.02.2016 14:02, Fam Zheng wrote:

On Wed, 02/03 13:57, Vladimir Sementsov-Ogievskiy wrote:

On 03.02.2016 11:14, Fam Zheng wrote:

On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:

Hi all.

These series which aims to add external backup api. This is needed to allow
backup software use our dirty bitmaps.

Vmware and Parallels Cloud Server have this feature.

What is the advantage of this appraoch over "drive-backup sync=incremental
..."?

Hmm, you are asking about advantage of external backup over
internal? It depends on external backup tool.

I'm asking why the tool can't use drive-backup to achieve what you want. They
seem very comparable.


drive-backup has limited range of maintained by qemu targets.. Third 
party tool uses its own internal targets. If we use drvie-backup there 
will be two copies instead of one: from qemu to drive-backup produced 
backup and then to internal target in third-party tool.




Fam


There are three things are done:
- add query-block-dirty-bitmap-ranges qmp command
- add qmp commands for dirty-bitmap functions: create_successor, abdicate,
   reclaim.
- make create-successor command transaction-able

Then, external backup should be done like this:

1.  qmp transaction {
 external-snapshot
 bitmap-create-successor
 }

2.  qmp query frozen bitmap, not acquiring aio context.

What do you do with the query response, read the snapshot image with an
external tool?

Yes. Alternatively, image fleecing may be used instead of snapshot.


Fam


3.  do external backup, using snapshot and bitmap


4.  if (success backup)
 qmp bitmap-abdicate
 else
 qmp bitmap-reclaime

5.  qmp merge snapshot


v2: a lot of additions and changes, no sense to compare with v1


Vladimir Sementsov-Ogievskiy (6):
   block dirty bitmap: add next_zero function
   qmp: add query-block-dirty-bitmap-ranges
   iotests: test query-block-dirty-bitmap-ranges
   qapi: add qmp commands for some dirty bitmap functions
   qapi: make block-dirty-bitmap-create-successor transaction-able
   iotests: test external backup api

  block/dirty-bitmap.c |  60 ++
  blockdev.c   | 113 +
  include/block/dirty-bitmap.h |   9 
  include/qemu/hbitmap.h   |   8 +++
  qapi-schema.json |   4 +-
  qapi/block-core.json |  90 +
  qmp-commands.hx  | 118 +++
  tests/qemu-iotests/150   |  88 
  tests/qemu-iotests/150.out   |  21 
  tests/qemu-iotests/151   |  77 
  tests/qemu-iotests/151.out   |   7 +++
  tests/qemu-iotests/group |   2 +
  util/hbitmap.c   |  26 ++
  13 files changed, 622 insertions(+), 1 deletion(-)
  create mode 100755 tests/qemu-iotests/150
  create mode 100644 tests/qemu-iotests/150.out
  create mode 100755 tests/qemu-iotests/151
  create mode 100644 tests/qemu-iotests/151.out

--
1.8.3.1



--
Best regards,
Vladimir




--
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Vladimir Sementsov-Ogievskiy]

On 03.02.2016 11:14, Fam Zheng wrote:

On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:

Hi all.

These series which aims to add external backup api. This is needed to allow
backup software use our dirty bitmaps.

Vmware and Parallels Cloud Server have this feature.

What is the advantage of this appraoch over "drive-backup sync=incremental
..."?


Hmm, you are asking about advantage of external backup over internal? It 
depends on external backup tool.





There are three things are done:
- add query-block-dirty-bitmap-ranges qmp command
- add qmp commands for dirty-bitmap functions: create_successor, abdicate,
   reclaim.
- make create-successor command transaction-able

Then, external backup should be done like this:

1.  qmp transaction {
 external-snapshot
 bitmap-create-successor
 }

2.  qmp query frozen bitmap, not acquiring aio context.

What do you do with the query response, read the snapshot image with an
external tool?


Yes. Alternatively, image fleecing may be used instead of snapshot.



Fam


3.  do external backup, using snapshot and bitmap


4.  if (success backup)
 qmp bitmap-abdicate
 else
 qmp bitmap-reclaime

5.  qmp merge snapshot


v2: a lot of additions and changes, no sense to compare with v1


Vladimir Sementsov-Ogievskiy (6):
   block dirty bitmap: add next_zero function
   qmp: add query-block-dirty-bitmap-ranges
   iotests: test query-block-dirty-bitmap-ranges
   qapi: add qmp commands for some dirty bitmap functions
   qapi: make block-dirty-bitmap-create-successor transaction-able
   iotests: test external backup api

  block/dirty-bitmap.c |  60 ++
  blockdev.c   | 113 +
  include/block/dirty-bitmap.h |   9 
  include/qemu/hbitmap.h   |   8 +++
  qapi-schema.json |   4 +-
  qapi/block-core.json |  90 +
  qmp-commands.hx  | 118 +++
  tests/qemu-iotests/150   |  88 
  tests/qemu-iotests/150.out   |  21 
  tests/qemu-iotests/151   |  77 
  tests/qemu-iotests/151.out   |   7 +++
  tests/qemu-iotests/group |   2 +
  util/hbitmap.c   |  26 ++
  13 files changed, 622 insertions(+), 1 deletion(-)
  create mode 100755 tests/qemu-iotests/150
  create mode 100644 tests/qemu-iotests/150.out
  create mode 100755 tests/qemu-iotests/151
  create mode 100644 tests/qemu-iotests/151.out

--
1.8.3.1




--
Best regards,
Vladimir

[Qemu-devel] [PATCH v2 0/6] external backup api

[Vladimir Sementsov-Ogievskiy]

Hi all.

These series which aims to add external backup api. This is needed to allow
backup software use our dirty bitmaps.

Vmware and Parallels Cloud Server have this feature.

There are three things are done:
- add query-block-dirty-bitmap-ranges qmp command
- add qmp commands for dirty-bitmap functions: create_successor, abdicate,
  reclaim.
- make create-successor command transaction-able

Then, external backup should be done like this:

1.  qmp transaction {
external-snapshot
bitmap-create-successor
}

2.  qmp query frozen bitmap, not acquiring aio context.

3.  do external backup, using snapshot and bitmap


4.  if (success backup)
qmp bitmap-abdicate
else
qmp bitmap-reclaime

5.  qmp merge snapshot


v2: a lot of additions and changes, no sense to compare with v1


Vladimir Sementsov-Ogievskiy (6):
  block dirty bitmap: add next_zero function
  qmp: add query-block-dirty-bitmap-ranges
  iotests: test query-block-dirty-bitmap-ranges
  qapi: add qmp commands for some dirty bitmap functions
  qapi: make block-dirty-bitmap-create-successor transaction-able
  iotests: test external backup api

 block/dirty-bitmap.c |  60 ++
 blockdev.c   | 113 +
 include/block/dirty-bitmap.h |   9 
 include/qemu/hbitmap.h   |   8 +++
 qapi-schema.json |   4 +-
 qapi/block-core.json |  90 +
 qmp-commands.hx  | 118 +++
 tests/qemu-iotests/150   |  88 
 tests/qemu-iotests/150.out   |  21 
 tests/qemu-iotests/151   |  77 
 tests/qemu-iotests/151.out   |   7 +++
 tests/qemu-iotests/group |   2 +
 util/hbitmap.c   |  26 ++
 13 files changed, 622 insertions(+), 1 deletion(-)
 create mode 100755 tests/qemu-iotests/150
 create mode 100644 tests/qemu-iotests/150.out
 create mode 100755 tests/qemu-iotests/151
 create mode 100644 tests/qemu-iotests/151.out

-- 
1.8.3.1