Re: [Qemu-block] [Qemu-devel] [PATCH v11 0/9] hw/m68k: add Apple Machintosh Quadra 800 machine

[no-reply]

Patchew URL: https://patchew.org/QEMU/20190910193347.16000-1-laur...@vivier.eu/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Subject: [Qemu-devel] [PATCH v11 0/9] hw/m68k: add Apple Machintosh Quadra 800 
machine
Message-id: 20190910193347.16000-1-laur...@vivier.eu
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
a82c9f9 hw/m68k: define Macintosh Quadra 800
79df5b3 hw/m68k: add a dummy SWIM floppy controller
6cf208a hw/m68k: add Nubus support for macfb video card
5f38935 hw/m68k: add Nubus support
be3903a hw/m68k: add macfb video card
7543084 hw/m68k: implement ADB bus support for via
126fd3e hw/m68k: add via support
b1b5f5f dp8393x: manage big endian bus
81c722f esp: add pseudo-DMA as used by Macintosh

=== OUTPUT BEGIN ===
1/9 Checking commit 81c722fdfd31 (esp: add pseudo-DMA as used by Macintosh)
2/9 Checking commit b1b5f5f329a2 (dp8393x: manage big endian bus)
3/9 Checking commit 126fd3e27620 (hw/m68k: add via support)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#77: 
new file mode 100644

ERROR: space prohibited after that '&&' (ctx:WxW)
#426: FILE: hw/misc/mac_via.c:345:
+if (!(v1s->last_b & VIA1B_vRTCClk) && (s->b & VIA1B_vRTCClk)) {
^

total: 1 errors, 1 warnings, 867 lines checked

Patch 3/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

4/9 Checking commit 7543084b151c (hw/m68k: implement ADB bus support for via)
5/9 Checking commit be3903ae6e87 (hw/m68k: add macfb video card)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#68: 
new file mode 100644

total: 0 errors, 1 warnings, 518 lines checked

Patch 5/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
6/9 Checking commit 5f389351a991 (hw/m68k: add Nubus support)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#62: 
new file mode 100644

total: 0 errors, 1 warnings, 532 lines checked

Patch 6/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
7/9 Checking commit 6cf208a8e54b (hw/m68k: add Nubus support for macfb video 
card)
8/9 Checking commit 79df5b3a854c (hw/m68k: add a dummy SWIM floppy controller)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#53: 
new file mode 100644

total: 0 errors, 1 warnings, 591 lines checked

Patch 8/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
9/9 Checking commit a82c9f9fafdc (hw/m68k: define Macintosh Quadra 800)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#70: 
new file mode 100644

total: 0 errors, 1 warnings, 518 lines checked

Patch 9/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20190910193347.16000-1-laur...@vivier.eu/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

Re: [Qemu-block] [PATCH v6 23/42] blockdev: Use CAF in external_snapshot_prepare()

[Max Reitz]

On 10.09.19 17:02, Kevin Wolf wrote:
> Am 09.08.2019 um 18:13 hat Max Reitz geschrieben:
>> This allows us to differentiate between filters and nodes with COW
>> backing files: Filters cannot be used as overlays at all (for this
>> function).
>>
>> Signed-off-by: Max Reitz 
>> Reviewed-by: Vladimir Sementsov-Ogievskiy 
> 
> Didn't we occasionally advertise blockdev-snapshot as the way to insert
> filters on top at runtime?

I can only remember advertising for it as the only graph manipulation
tool we had, and maybe saying “We’d want something like
blockdev-snapshot for filters, too”.

Max

> Though it seems it has always only worked for
> filters that use bs->backing, among which I think there aren't any
> user-creatable ones. So we're probably good.
> 
> Kevin
> 
>>  blockdev.c | 7 ++-
>>  1 file changed, 6 insertions(+), 1 deletion(-)
>>
>> diff --git a/blockdev.c b/blockdev.c
>> index 29c6c6044a..c540802127 100644
>> --- a/blockdev.c
>> +++ b/blockdev.c
>> @@ -1664,7 +1664,12 @@ static void external_snapshot_prepare(BlkActionState 
>> *common,
>>  goto out;
>>  }
>>  
>> -if (state->new_bs->backing != NULL) {
>> +if (state->new_bs->drv->is_filter) {
>> +error_setg(errp, "Filters cannot be used as overlays");
>> +goto out;
>> +}
>> +
>> +if (bdrv_filtered_cow_child(state->new_bs)) {
>>  error_setg(errp, "The overlay already has a backing image");
>>  goto out;
>>  }
>> -- 
>> 2.21.0
>>




signature.asc
Description: OpenPGP digital signature

Re: [Qemu-block] [PATCH v6 22/42] block: Fix bdrv_get_allocated_file_size's fallback

[Max Reitz]

On 10.09.19 16:52, Kevin Wolf wrote:
> Am 09.08.2019 um 18:13 hat Max Reitz geschrieben:
>> If the driver does not implement bdrv_get_allocated_file_size(), we
>> should fall back to cumulating the allocated size of all non-COW
>> children instead of just bs->file.
>>
>> Suggested-by: Vladimir Sementsov-Ogievskiy 
>> Signed-off-by: Max Reitz 
> 
> This smells like an overgeneralisation, but if we want to count all vmdk
> extents, the qcow2 external data file, etc. it's an improvement anyway.
> A driver that has a child that should not be counted must just remember
> to implement the callback.
> 
> Let me think of an example... How about quorum, for a change? :-)
> Or the second blkverify child.
> 
> Or eventually the block job filter nodes.

I actually think it makes sense for all of these nodes to report the sum
of all of their children’s allocated sizes.

If a quorum node has three children with allocated sizes of 3 MB, 1 MB,
and 2 MB, respectively (totally possible if some have explicit zeroes
and others don’t; it may also depend on the protocol, the filesystem,
etc.), then I think it makes most sense to report indeed 6 MB for the
quorum subtree as a whole.  What would you report?  3 MB?

> Ehm... Maybe I should just take back what I said first. It almost feels
> like it would be better if qcow2 and vmdk explicitly used a handler that
> counts all children (could still be a generic one in block.c) rather
> than having to remember to disable the functionality everywhere where we
> don't want to have it.

I don’t, because everywhere we don’t want this functionality, we still
need to choose a child.  This has to be done by the driver anyway.

Max

> And please adjust the comment for bdrv_get_allocated_file_size(), it
> only talks about a single file as if trees didn't exist. Actually, it
> doesn't even seem so easy to define. Maybe primary node + storage nodes?
> Then vmdk needs to expose its extents as storage nodes (plural!), but
> in the long run that might be needed anyway.



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-block] [Qemu-devel] [PATCH v11 0/9] hw/m68k: add Apple Machintosh Quadra 800 machine

[no-reply]

Patchew URL: https://patchew.org/QEMU/20190910193347.16000-1-laur...@vivier.eu/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Subject: [Qemu-devel] [PATCH v11 0/9] hw/m68k: add Apple Machintosh Quadra 800 
machine
Message-id: 20190910193347.16000-1-laur...@vivier.eu
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 * [new tag] patchew/20190911034302.29108-1-and...@aj.id.au -> 
patchew/20190911034302.29108-1-and...@aj.id.au
Switched to a new branch 'test'
ba04957 hw/m68k: define Macintosh Quadra 800
99060cf hw/m68k: add a dummy SWIM floppy controller
d1012bd hw/m68k: add Nubus support for macfb video card
771de64 hw/m68k: add Nubus support
664a0d4 hw/m68k: add macfb video card
d763a5e hw/m68k: implement ADB bus support for via
c4005b8 hw/m68k: add via support
7032706 dp8393x: manage big endian bus
9ad093b esp: add pseudo-DMA as used by Macintosh

=== OUTPUT BEGIN ===
1/9 Checking commit 9ad093b6ab18 (esp: add pseudo-DMA as used by Macintosh)
2/9 Checking commit 703270689bab (dp8393x: manage big endian bus)
3/9 Checking commit c4005b89095c (hw/m68k: add via support)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#77: 
new file mode 100644

ERROR: space prohibited after that '&&' (ctx:WxW)
#426: FILE: hw/misc/mac_via.c:345:
+if (!(v1s->last_b & VIA1B_vRTCClk) && (s->b & VIA1B_vRTCClk)) {
^

total: 1 errors, 1 warnings, 867 lines checked

Patch 3/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

4/9 Checking commit d763a5ee0676 (hw/m68k: implement ADB bus support for via)
5/9 Checking commit 664a0d473afd (hw/m68k: add macfb video card)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#68: 
new file mode 100644

total: 0 errors, 1 warnings, 518 lines checked

Patch 5/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
6/9 Checking commit 771de64f4aa2 (hw/m68k: add Nubus support)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#62: 
new file mode 100644

total: 0 errors, 1 warnings, 532 lines checked

Patch 6/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
7/9 Checking commit d1012bd3f111 (hw/m68k: add Nubus support for macfb video 
card)
8/9 Checking commit 99060cf0bb1b (hw/m68k: add a dummy SWIM floppy controller)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#53: 
new file mode 100644

total: 0 errors, 1 warnings, 591 lines checked

Patch 8/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
9/9 Checking commit ba04957f42dd (hw/m68k: define Macintosh Quadra 800)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#70: 
new file mode 100644

total: 0 errors, 1 warnings, 518 lines checked

Patch 9/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20190910193347.16000-1-laur...@vivier.eu/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

Re: [Qemu-block] [Qemu-devel] [PATCH v11 0/9] hw/m68k: add Apple Machintosh Quadra 800 machine

[no-reply]

Patchew URL: https://patchew.org/QEMU/20190910193347.16000-1-laur...@vivier.eu/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Subject: [Qemu-devel] [PATCH v11 0/9] hw/m68k: add Apple Machintosh Quadra 800 
machine
Message-id: 20190910193347.16000-1-laur...@vivier.eu
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
161e84d hw/m68k: define Macintosh Quadra 800
8e014ae hw/m68k: add a dummy SWIM floppy controller
48b413f hw/m68k: add Nubus support for macfb video card
c85a9d0 hw/m68k: add Nubus support
2a9c811 hw/m68k: add macfb video card
dd6f485 hw/m68k: implement ADB bus support for via
962fb19 hw/m68k: add via support
bf2b5cd dp8393x: manage big endian bus
b9f0b34 esp: add pseudo-DMA as used by Macintosh

=== OUTPUT BEGIN ===
1/9 Checking commit b9f0b34536a6 (esp: add pseudo-DMA as used by Macintosh)
2/9 Checking commit bf2b5cdc0edd (dp8393x: manage big endian bus)
3/9 Checking commit 962fb196b521 (hw/m68k: add via support)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#77: 
new file mode 100644

ERROR: space prohibited after that '&&' (ctx:WxW)
#426: FILE: hw/misc/mac_via.c:345:
+if (!(v1s->last_b & VIA1B_vRTCClk) && (s->b & VIA1B_vRTCClk)) {
^

total: 1 errors, 1 warnings, 867 lines checked

Patch 3/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

4/9 Checking commit dd6f4858fa7b (hw/m68k: implement ADB bus support for via)
5/9 Checking commit 2a9c811632ea (hw/m68k: add macfb video card)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#68: 
new file mode 100644

total: 0 errors, 1 warnings, 518 lines checked

Patch 5/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
6/9 Checking commit c85a9d0d90a0 (hw/m68k: add Nubus support)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#62: 
new file mode 100644

total: 0 errors, 1 warnings, 532 lines checked

Patch 6/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
7/9 Checking commit 48b413fb01cc (hw/m68k: add Nubus support for macfb video 
card)
8/9 Checking commit 8e014aea3937 (hw/m68k: add a dummy SWIM floppy controller)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#53: 
new file mode 100644

total: 0 errors, 1 warnings, 591 lines checked

Patch 8/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
9/9 Checking commit 161e84d328bc (hw/m68k: define Macintosh Quadra 800)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#70: 
new file mode 100644

total: 0 errors, 1 warnings, 518 lines checked

Patch 9/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20190910193347.16000-1-laur...@vivier.eu/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/7] block/curl: Fix hang and potential crash

[John Snow]

On 9/10/19 8:41 AM, Max Reitz wrote:
> Hi,
> 
> As reported in https://bugzilla.redhat.com/show_bug.cgi?id=1740193, our
> curl block driver can spontaneously hang.  This becomes visible e.g.
> when reading compressed qcow2 images:
> 
> $ qemu-img convert -p -O raw -n \
>   https://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img \
>   null-co://
> 
> (Hangs at 74.21 %, usually.)
> 
> A more direct way is:
> 
> $ qemu-img bench -f raw http://download.qemu.org/qemu-4.1.0.tar.xz \
> -d 1 -S 524288 -c 2
> 
> (Which simply performs two requests, and the second one hangs.  You can
> use any HTTP resource (probably FTP, too) you’d like that is at least
> 1 MB in size.)
> 
> It turns out that this is because cURL 7.59.0 has added a protective
> feature against some misuse we had in our code: curl_multi_add_handle()
> must not be called from within a cURL callback, but in some cases we
> did.  As of 7.59.0, this fails, our new request is not registered and
> the I/O request stalls.  This is fixed by patch 6.
> 
> Patch 7 makes us check for curl_multi_add_handle()’s return code,
> because if we had done that before, debugging would have been much
> simpler.
> 
> 
> On the way to fixing it, I had a look over the whole cURL code and found
> a suspicious QLIST_FOREACH_SAFE() loop that actually does not seem very
> safe at all.  I think this may lead to crashes, although I have never
> seen any myself.  https://bugzilla.redhat.com/show_bug.cgi?id=1744602#c5
> shows one in exactly the function in question, so I think it actually is
> a problem.
> 
> This is fixed by patch 5, patches 1, 2, and 4 prepare for it.
> 
> (Patch 3 is kind of a misc patch that should ensure that we always end
> up calling curl_multi_check_completion() whenever a request might have
> been completed.)
> 
> 
> v2:
> - Patch 2: Remove the socket from the list only add the end of the
>function (yielding a nicer 5+/5- diff stat)
> - Patch 3: Added
> - Patch 4: Rebased on patch 3, and s/socket/ready_socket/ in one place
> - Patch 5: Rebased on the changed patch 4
> 
> 
> git-backport-diff against v1:
> 
> Key:
> [] : patches are identical
> [] : number of functional differences between upstream/downstream patch
> [down] : patch is downstream-only
> The flags [FC] indicate (F)unctional and (C)ontextual differences, 
> respectively
> 
> 001/7:[] [--] 'curl: Keep pointer to the CURLState in CURLSocket'
> 002/7:[0007] [FC] 'curl: Keep *socket until the end of curl_sock_cb()'
> 003/7:[down] 'curl: Check completion in curl_multi_do()'
> 004/7:[0019] [FC] 'curl: Pass CURLSocket to curl_multi_{do,read}()'
> 005/7:[0002] [FC] 'curl: Report only ready sockets'
> 006/7:[] [--] 'curl: Handle success in multi_check_completion'
> 007/7:[] [--] 'curl: Check curl_multi_add_handle()'s return code'
> 
> 
> Max Reitz (7):
>   curl: Keep pointer to the CURLState in CURLSocket
>   curl: Keep *socket until the end of curl_sock_cb()
>   curl: Check completion in curl_multi_do()
>   curl: Pass CURLSocket to curl_multi_do()
>   curl: Report only ready sockets
>   curl: Handle success in multi_check_completion
>   curl: Check curl_multi_add_handle()'s return code
> 
>  block/curl.c | 133 +++
>  1 file changed, 59 insertions(+), 74 deletions(-)
> 

And for 4-7:

Reviewed-by: John Snow 

Re: [Qemu-block] [PATCH v2 3/7] curl: Check completion in curl_multi_do()

[John Snow]

On 9/10/19 12:11 PM, Maxim Levitsky wrote:
> On Tue, 2019-09-10 at 14:41 +0200, Max Reitz wrote:
>> While it is more likely that transfers complete after some file
>> descriptor has data ready to read, we probably should not rely on it.
>> Better be safe than sorry and call curl_multi_check_completion() in
>> curl_multi_do(), too, just like it is done in curl_multi_read().
>>
>> With this change, curl_multi_do() and curl_multi_read() are actually the
>> same, so drop curl_multi_read() and use curl_multi_do() as the sole FD
>> handler.
> 
> I understand the reasoning, but I still a bit worry that this
> could paper over some bug/race in the future.
> If curl asks us only to deal with write, that would mean
> that it doesn't expect any data to be received.
> 
> Do you by a chance have an example, of this patch
> affecting the code? Maybe when a unexpected error reply
> is received from the server?
> 
> I don't really know the CURL library, so I probably missed
> something important.
> 
> Other than that,
> Reviewed-by: Maxim Levitsky 
> 

In this case, it's because I had some doubts in V1 about when we call
the post-completion cleanup code. It didn't look obvious at a glance.

This just makes it simpler.

I don't think that by checking *more* things we're going to paper over
some race condition -- it's the opposite. If anything, this will explode
sooner rather than later.

So I think it's OK, naturally.

Reviewed-by: John Snow 

Re: [Qemu-block] [Qemu-devel] [PATCH v11 0/9] hw/m68k: add Apple Machintosh Quadra 800 machine

[no-reply]

Patchew URL: https://patchew.org/QEMU/20190910193347.16000-1-laur...@vivier.eu/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Subject: [Qemu-devel] [PATCH v11 0/9] hw/m68k: add Apple Machintosh Quadra 800 
machine
Message-id: 20190910193347.16000-1-laur...@vivier.eu
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
6c8a056 hw/m68k: define Macintosh Quadra 800
0a4fe5f hw/m68k: add a dummy SWIM floppy controller
b2f7c98 hw/m68k: add Nubus support for macfb video card
5925088 hw/m68k: add Nubus support
9d7f87b hw/m68k: add macfb video card
57d3c1b hw/m68k: implement ADB bus support for via
79d3ec3 hw/m68k: add via support
3cd0a65 dp8393x: manage big endian bus
98468c1 esp: add pseudo-DMA as used by Macintosh

=== OUTPUT BEGIN ===
1/9 Checking commit 98468c173c48 (esp: add pseudo-DMA as used by Macintosh)
2/9 Checking commit 3cd0a6527565 (dp8393x: manage big endian bus)
3/9 Checking commit 79d3ec37b19d (hw/m68k: add via support)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#77: 
new file mode 100644

ERROR: space prohibited after that '&&' (ctx:WxW)
#426: FILE: hw/misc/mac_via.c:345:
+if (!(v1s->last_b & VIA1B_vRTCClk) && (s->b & VIA1B_vRTCClk)) {
^

total: 1 errors, 1 warnings, 867 lines checked

Patch 3/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

4/9 Checking commit 57d3c1b68d1b (hw/m68k: implement ADB bus support for via)
5/9 Checking commit 9d7f87b197f7 (hw/m68k: add macfb video card)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#68: 
new file mode 100644

total: 0 errors, 1 warnings, 518 lines checked

Patch 5/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
6/9 Checking commit 59250888e2e4 (hw/m68k: add Nubus support)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#62: 
new file mode 100644

total: 0 errors, 1 warnings, 532 lines checked

Patch 6/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
7/9 Checking commit b2f7c9845fe3 (hw/m68k: add Nubus support for macfb video 
card)
8/9 Checking commit 0a4fe5f04b97 (hw/m68k: add a dummy SWIM floppy controller)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#53: 
new file mode 100644

total: 0 errors, 1 warnings, 591 lines checked

Patch 8/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
9/9 Checking commit 6c8a0561eda1 (hw/m68k: define Macintosh Quadra 800)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#70: 
new file mode 100644

total: 0 errors, 1 warnings, 518 lines checked

Patch 9/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20190910193347.16000-1-laur...@vivier.eu/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

Re: [Qemu-block] [Qemu-devel] [PATCH v2 2/7] curl: Keep *socket until the end of curl_sock_cb()

[John Snow]

On 9/10/19 8:41 AM, Max Reitz wrote:
> This does not really change anything, but it makes the code a bit easier
> to follow once we use @socket as the opaque pointer for
> aio_set_fd_handler().
> 
> Cc: qemu-sta...@nongnu.org
> Signed-off-by: Max Reitz 

Reviewed-by: John Snow 

Re: [Qemu-block] [Qemu-devel] [PATCH 0/3] proper locking on bitmap add/remove paths

[no-reply]

Patchew URL: 
https://patchew.org/QEMU/20190910162724.79574-1-vsement...@virtuozzo.com/



Hi,

This series failed the docker-quick@centos7 build test. Please find the testing 
commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
=== TEST SCRIPT END ===

libudev   no
default devices   yes

warning: Python 2 support is deprecated
warning: Python 3 will be required for building future versions of QEMU

NOTE: cross-compilers enabled:  'cc'
  GEN x86_64-softmmu/config-devices.mak.tmp
---
  CC  block/qed-cluster.o
  CC  block/qed-check.o
/tmp/qemu-test/src/block/qcow2-bitmap.c: In function 
'qcow2_co_remove_persistent_dirty_bitmap':
/tmp/qemu-test/src/block/qcow2-bitmap.c:502:8: error: 'bm' may be used 
uninitialized in this function [-Werror=maybe-uninitialized]
 if (bm == NULL) {
^
/tmp/qemu-test/src/block/qcow2-bitmap.c:1413:18: note: 'bm' was declared here


The full log is available at
http://patchew.org/logs/20190910162724.79574-1-vsement...@virtuozzo.com/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

[Qemu-block] [PATCH v11 6/9] hw/m68k: add Nubus support

[Laurent Vivier]

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
Reviewed-by: Thomas Huth 
---
 MAINTAINERS |   2 +
 hw/Kconfig  |   1 +
 hw/Makefile.objs|   1 +
 hw/m68k/Kconfig |   1 +
 hw/nubus/Kconfig|   2 +
 hw/nubus/Makefile.objs  |   4 +
 hw/nubus/mac-nubus-bridge.c |  45 ++
 hw/nubus/nubus-bridge.c |  34 +
 hw/nubus/nubus-bus.c| 111 ++
 hw/nubus/nubus-device.c | 215 
 include/hw/nubus/mac-nubus-bridge.h |  24 
 include/hw/nubus/nubus.h|  69 +
 12 files changed, 509 insertions(+)
 create mode 100644 hw/nubus/Kconfig
 create mode 100644 hw/nubus/Makefile.objs
 create mode 100644 hw/nubus/mac-nubus-bridge.c
 create mode 100644 hw/nubus/nubus-bridge.c
 create mode 100644 hw/nubus/nubus-bus.c
 create mode 100644 hw/nubus/nubus-device.c
 create mode 100644 include/hw/nubus/mac-nubus-bridge.h
 create mode 100644 include/hw/nubus/nubus.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 6b5fc50aef..4f6b2b037a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -922,8 +922,10 @@ M: Laurent Vivier 
 S: Maintained
 F: hw/misc/mac_via.c
 F: hw/display/macfb.c
+F: hw/nubus/*
 F: include/hw/misc/mac_via.h
 F: include/hw/display/macfb.h
+F: include/hw/nubus/*
 
 MicroBlaze Machines
 ---
diff --git a/hw/Kconfig b/hw/Kconfig
index b45db3c813..0501a55315 100644
--- a/hw/Kconfig
+++ b/hw/Kconfig
@@ -21,6 +21,7 @@ source isa/Kconfig
 source mem/Kconfig
 source misc/Kconfig
 source net/Kconfig
+source nubus/Kconfig
 source nvram/Kconfig
 source pci-bridge/Kconfig
 source pci-host/Kconfig
diff --git a/hw/Makefile.objs b/hw/Makefile.objs
index ece6cc3755..457b95e28d 100644
--- a/hw/Makefile.objs
+++ b/hw/Makefile.objs
@@ -37,6 +37,7 @@ devices-dirs-y += virtio/
 devices-dirs-y += watchdog/
 devices-dirs-y += xen/
 devices-dirs-$(CONFIG_MEM_DEVICE) += mem/
+devices-dirs-$(CONFIG_NUBUS) += nubus/
 devices-dirs-y += semihosting/
 devices-dirs-y += smbios/
 endif
diff --git a/hw/m68k/Kconfig b/hw/m68k/Kconfig
index 58c39ec1a9..9133919bb8 100644
--- a/hw/m68k/Kconfig
+++ b/hw/m68k/Kconfig
@@ -17,3 +17,4 @@ config Q800
 bool
 select MAC_VIA
 select MACFB
+select NUBUS
diff --git a/hw/nubus/Kconfig b/hw/nubus/Kconfig
new file mode 100644
index 00..8fb8b22189
--- /dev/null
+++ b/hw/nubus/Kconfig
@@ -0,0 +1,2 @@
+config NUBUS
+bool
diff --git a/hw/nubus/Makefile.objs b/hw/nubus/Makefile.objs
new file mode 100644
index 00..135ba7878d
--- /dev/null
+++ b/hw/nubus/Makefile.objs
@@ -0,0 +1,4 @@
+common-obj-y += nubus-device.o
+common-obj-y += nubus-bus.o
+common-obj-y += nubus-bridge.o
+common-obj-$(CONFIG_Q800) += mac-nubus-bridge.o
diff --git a/hw/nubus/mac-nubus-bridge.c b/hw/nubus/mac-nubus-bridge.c
new file mode 100644
index 00..7c329300b8
--- /dev/null
+++ b/hw/nubus/mac-nubus-bridge.c
@@ -0,0 +1,45 @@
+/*
+ *  Copyright (c) 2013-2018 Laurent Vivier 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "hw/sysbus.h"
+#include "hw/nubus/mac-nubus-bridge.h"
+
+
+static void mac_nubus_bridge_init(Object *obj)
+{
+MacNubusState *s = MAC_NUBUS_BRIDGE(obj);
+SysBusDevice *sbd = SYS_BUS_DEVICE(obj);
+
+s->bus = NUBUS_BUS(qbus_create(TYPE_NUBUS_BUS, DEVICE(s), NULL));
+
+sysbus_init_mmio(sbd, &s->bus->super_slot_io);
+sysbus_init_mmio(sbd, &s->bus->slot_io);
+}
+
+static void mac_nubus_bridge_class_init(ObjectClass *klass, void *data)
+{
+DeviceClass *dc = DEVICE_CLASS(klass);
+
+dc->desc = "Nubus bridge";
+}
+
+static const TypeInfo mac_nubus_bridge_info = {
+.name  = TYPE_MAC_NUBUS_BRIDGE,
+.parent= TYPE_NUBUS_BRIDGE,
+.instance_init = mac_nubus_bridge_init,
+.instance_size = sizeof(MacNubusState),
+.class_init= mac_nubus_bridge_class_init,
+};
+
+static void mac_nubus_bridge_register_types(void)
+{
+type_register_static(&mac_nubus_bridge_info);
+}
+
+type_init(mac_nubus_bridge_register_types)
diff --git a/hw/nubus/nubus-bridge.c b/hw/nubus/nubus-bridge.c
new file mode 100644
index 00..cd8c6a91eb
--- /dev/null
+++ b/hw/nubus/nubus-bridge.c
@@ -0,0 +1,34 @@
+/*
+ * QEMU Macintosh Nubus
+ *
+ * Copyright (c) 2013-2018 Laurent Vivier 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "hw/sysbus.h"
+#include "hw/nubus/nubus.h"
+
+static void nubus_bridge_class_init(ObjectClass *klass, void *data)
+{
+DeviceClass *dc = DEVICE_CLASS(klass);
+
+dc->fw_name = "nubus";
+}
+
+static const TypeInfo nubus_bridge_info = {
+.name  = TYPE_NUBUS_BRIDGE,
+.parent= TYPE

[Qemu-block] [PATCH v11 0/9] hw/m68k: add Apple Machintosh Quadra 800 machine

[Laurent Vivier]

I'm rebasing some of these patches for seven years now,
too many years...

if you want to test the machine, I'm sorry, it doesn't boot
a MacROM, but you can boot a linux kernel from the command line.

You can install your own disk using debian-installer, with:

...
-M q800 \
-serial none -serial mon:stdio \
-m 1000M \
-net nic,model=dp83932,addr=09:00:07:12:34:57 \
-append "console=ttyS0 vga=off" \
-kernel vmlinux-4.16.0-1-m68k \
-initrd initrd.gz \
-drive file=debian-10.0-m68k-NETINST-1.iso,media=cdrom \
-drive file=m68k.qcow2,format=qcow2 \
-nographic

If you use a graphic adapter instead of "-nographic", you can use "-g" to set 
the
size of the display (I use "-g 1600x800x24").

You can get the ISO from:

https://cdimage.debian.org/cdimage/ports/10.0/m68k/iso-cd/debian-10.0-m68k-NETINST-1.iso

and extract the kernel and initrd.gz:

guestfish --add debian-10.0-m68k-NETINST-1.iso --ro \
  --mount /dev/sda:/ <<_EOF_
copy-out /install/cdrom/initrd.gz .
copy-out /install/kernels/vmlinux-4.16.0-1-m68k .
_EOF_

The mirror to use is: http://ftp.ports.debian.org/debian-ports/
when it fails, continue without boot loader.

In the same way, you can extract the kernel and the initramfs from the qcow2
image to use it with "-kernel" and "-initrd":

guestfish --add m68k.qcow2 --mount /dev/sda2:/ <<_EOF_
copy-out /boot/vmlinux-4.16.0-1-m68k .
copy-out /boot/initrd.img-4.16.0-1-m68k .
_EOF_

and boot with:

   ...
   -append "root=/dev/sda2 rw console=ttyS0 console=tty \
   -kernel vmlinux-4.16.0-1-m68k \
   -initrd initrd.img-4.16.0-1-m68k

NOTE: DHCP doesn't work but you can assign a static IP address.
  We need some patches for dp8393x that are not ready to be merged.
  See http://patchwork.ozlabs.org/patch/927020/
  http://patchwork.ozlabs.org/patch/927030/
  http://patchwork.ozlabs.org/patch/927026/

v11: Add VMState to migrate ESP PDMA

 The new VMState structures cannot be tested because m68k is not
 migratable and then Q800 is not either.
 I've tested the ESP VMState is not broken by the change
 with 'migrate "exec:cat > mig"' with qemu-system-sparc and
 I have compared the result with/without the patch with
 scripts/analyze-migrate.py: files desc.json are identical.

v10: Add SWIM VMState and reset function
 Add MacVIA VMState
 rework Kconfig

v9: Fix comments format
rebase on top of NeXTcube

v8: rebase (new blk_new(), add "qemu-common.h")
update bootinfo information and license
add some braces
Rename Q800IRQState to GLUEState:
it's more like a Logic Unit than an IRQ controller,
and Apple calls it "GLUE" (Mark: I prefer to keep it
like this for the moment, in the future this part
need to be reworked, we have to review the IRQ levels
and to wire NUBUS IRQ. The implementation is really trivial
for the moment and we will move it to QOM in the future)

v7: rebase and port to Kconfig
move IRQ controller back to q800.c (we don't need an object for this)
update log message for ESP changes and add some g_assert()
re-order patches: put esp, escc and dp8393x first

v6: Rebase onto git master (this now includes the m68k EXCP_ILLEGAL fix required
  for this patchset to boot)
Add Hervé's R-B tags
Drop ASC (Apple Sound Chip) device since the Linux driver is broken and
  it is not required for a successful boot
Remove extra esp_raise_irq() from ESP pseudo-DMA patch (Hervé)
Remove "return" from unimplemented write functions and instead add a
  "read only" comment (Hervé)
Rename MAX_FD to SWIM_MAX_FD in SWIM floppy controller patch to prevent
  potential conflicts with other files (Hervé)

v5: Rebase onto git master
Add Philippe's R-B to patch 10
Include the command line to boot a Linux kernel under the q800 machine in 
the
commit message for patch 11 (Philippe)
Fix up comments in hw/misc/mac_via.c (Thomas)
Add asserts to VIA ADB support to prevent potential buffer overflows 
(Thomas)
Move macfb surface/resolution checks to realise and remove hw_error (Thomas)
Move macfb draw_line functions inline and remove macfb-template.h (Mark)
Use guest address rather than source pointer in draw_line functions - this 
brings
  macfb in line with the VGA device and can prevent a potential buffer 
overflow
Use g_strdup_printf() for memory region names in NuBus devices instead of
  hardcoded length char arrays (Thomas)
Move NuBus QOM types from patch 7 to patch 8 (spotted by Thomas)
Move CONFIG_COLDFIRE sections together in hw/m68k/Makefile.objs (Thomas)
Remove obsolete comment from q800.c in patch 11 (Thomas)

v4: Drop RFC from subject prefix as this is getting close to final
Rebased onto master (fixing ESP, rom_ptr() conflicts)
Reworked q800.c based upon Thomas' comments about cpu_init() and
  qemu_check_nic_model()
Address Thomas' comments on using error_report() instead of hw

[Qemu-block] [PATCH v11 7/9] hw/m68k: add Nubus support for macfb video card

[Laurent Vivier]

From: Mark Cave-Ayland 

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
Reviewed-by: Hervé Poussineau 
---
 hw/display/Kconfig |  1 +
 hw/display/macfb.c | 56 ++
 include/hw/display/macfb.h | 21 ++
 3 files changed, 78 insertions(+)

diff --git a/hw/display/Kconfig b/hw/display/Kconfig
index 32e8d29003..c500d1fc6d 100644
--- a/hw/display/Kconfig
+++ b/hw/display/Kconfig
@@ -136,3 +136,4 @@ config ATI_VGA
 config MACFB
 bool
 select FRAMEBUFFER
+depends on NUBUS
diff --git a/hw/display/macfb.c b/hw/display/macfb.c
index d7c5ef296e..f4fa8e3206 100644
--- a/hw/display/macfb.c
+++ b/hw/display/macfb.c
@@ -15,6 +15,7 @@
 #include "hw/sysbus.h"
 #include "ui/console.h"
 #include "ui/pixel_ops.h"
+#include "hw/nubus/nubus.h"
 #include "hw/display/macfb.h"
 #include "qapi/error.h"
 #include "hw/qdev-properties.h"
@@ -382,12 +383,38 @@ static void macfb_sysbus_realize(DeviceState *dev, Error 
**errp)
 sysbus_init_mmio(SYS_BUS_DEVICE(s), &ms->mem_vram);
 }
 
+const uint8_t macfb_rom[] = {
+255, 0, 0, 0,
+};
+
+static void macfb_nubus_realize(DeviceState *dev, Error **errp)
+{
+NubusDevice *nd = NUBUS_DEVICE(dev);
+MacfbNubusState *s = NUBUS_MACFB(dev);
+MacfbNubusDeviceClass *ndc = MACFB_NUBUS_GET_CLASS(dev);
+MacfbState *ms = &s->macfb;
+
+ndc->parent_realize(dev, errp);
+
+macfb_common_realize(dev, ms, errp);
+memory_region_add_subregion(&nd->slot_mem, DAFB_BASE, &ms->mem_ctrl);
+memory_region_add_subregion(&nd->slot_mem, VIDEO_BASE, &ms->mem_vram);
+
+nubus_register_rom(nd, macfb_rom, sizeof(macfb_rom), 1, 9, 0xf);
+}
+
 static void macfb_sysbus_reset(DeviceState *d)
 {
 MacfbSysBusState *s = MACFB(d);
 macfb_reset(&s->macfb);
 }
 
+static void macfb_nubus_reset(DeviceState *d)
+{
+MacfbNubusState *s = NUBUS_MACFB(d);
+macfb_reset(&s->macfb);
+}
+
 static Property macfb_sysbus_properties[] = {
 DEFINE_PROP_UINT32("width", MacfbSysBusState, macfb.width, 640),
 DEFINE_PROP_UINT32("height", MacfbSysBusState, macfb.height, 480),
@@ -395,6 +422,13 @@ static Property macfb_sysbus_properties[] = {
 DEFINE_PROP_END_OF_LIST(),
 };
 
+static Property macfb_nubus_properties[] = {
+DEFINE_PROP_UINT32("width", MacfbNubusState, macfb.width, 640),
+DEFINE_PROP_UINT32("height", MacfbNubusState, macfb.height, 480),
+DEFINE_PROP_UINT8("depth", MacfbNubusState, macfb.depth, 8),
+DEFINE_PROP_END_OF_LIST(),
+};
+
 static void macfb_sysbus_class_init(ObjectClass *klass, void *data)
 {
 DeviceClass *dc = DEVICE_CLASS(klass);
@@ -406,6 +440,19 @@ static void macfb_sysbus_class_init(ObjectClass *klass, 
void *data)
 dc->props = macfb_sysbus_properties;
 }
 
+static void macfb_nubus_class_init(ObjectClass *klass, void *data)
+{
+DeviceClass *dc = DEVICE_CLASS(klass);
+MacfbNubusDeviceClass *ndc = MACFB_NUBUS_DEVICE_CLASS(klass);
+
+device_class_set_parent_realize(dc, macfb_nubus_realize,
+&ndc->parent_realize);
+dc->desc = "Nubus Macintosh framebuffer";
+dc->reset = macfb_nubus_reset;
+dc->vmsd = &vmstate_macfb;
+dc->props = macfb_nubus_properties;
+}
+
 static TypeInfo macfb_sysbus_info = {
 .name  = TYPE_MACFB,
 .parent= TYPE_SYS_BUS_DEVICE,
@@ -413,9 +460,18 @@ static TypeInfo macfb_sysbus_info = {
 .class_init= macfb_sysbus_class_init,
 };
 
+static TypeInfo macfb_nubus_info = {
+.name  = TYPE_NUBUS_MACFB,
+.parent= TYPE_NUBUS_DEVICE,
+.instance_size = sizeof(MacfbNubusState),
+.class_init= macfb_nubus_class_init,
+.class_size= sizeof(MacfbNubusDeviceClass),
+};
+
 static void macfb_register_types(void)
 {
 type_register_static(&macfb_sysbus_info);
+type_register_static(&macfb_nubus_info);
 }
 
 type_init(macfb_register_types)
diff --git a/include/hw/display/macfb.h b/include/hw/display/macfb.h
index 3fe2592735..26367ae2c4 100644
--- a/include/hw/display/macfb.h
+++ b/include/hw/display/macfb.h
@@ -40,4 +40,25 @@ typedef struct {
 MacfbState macfb;
 } MacfbSysBusState;
 
+#define MACFB_NUBUS_DEVICE_CLASS(class) \
+OBJECT_CLASS_CHECK(MacfbNubusDeviceClass, (class), TYPE_NUBUS_MACFB)
+#define MACFB_NUBUS_GET_CLASS(obj) \
+OBJECT_GET_CLASS(MacfbNubusDeviceClass, (obj), TYPE_NUBUS_MACFB)
+
+typedef struct MacfbNubusDeviceClass {
+DeviceClass parent_class;
+
+DeviceRealize parent_realize;
+} MacfbNubusDeviceClass;
+
+#define TYPE_NUBUS_MACFB "nubus-macfb"
+#define NUBUS_MACFB(obj) \
+OBJECT_CHECK(MacfbNubusState, (obj), TYPE_NUBUS_MACFB)
+
+typedef struct {
+NubusDevice busdev;
+
+MacfbState macfb;
+} MacfbNubusState;
+
 #endif
-- 
2.21.0

[Qemu-block] [PATCH v11 8/9] hw/m68k: add a dummy SWIM floppy controller

[Laurent Vivier]

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
Reviewed-by: Hervé Poussineau 
---
 MAINTAINERS |   2 +
 hw/block/Kconfig|   3 +
 hw/block/Makefile.objs  |   1 +
 hw/block/swim.c | 487 
 hw/m68k/Kconfig |   1 +
 include/hw/block/swim.h |  76 +++
 6 files changed, 570 insertions(+)
 create mode 100644 hw/block/swim.c
 create mode 100644 include/hw/block/swim.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 4f6b2b037a..f85f11d83c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -923,9 +923,11 @@ S: Maintained
 F: hw/misc/mac_via.c
 F: hw/display/macfb.c
 F: hw/nubus/*
+F: hw/block/swim.c
 F: include/hw/misc/mac_via.h
 F: include/hw/display/macfb.h
 F: include/hw/nubus/*
+F: include/hw/block/swim.h
 
 MicroBlaze Machines
 ---
diff --git a/hw/block/Kconfig b/hw/block/Kconfig
index df96dc5dcc..2d17f481ad 100644
--- a/hw/block/Kconfig
+++ b/hw/block/Kconfig
@@ -37,3 +37,6 @@ config VHOST_USER_BLK
 # Only PCI devices are provided for now
 default y if VIRTIO_PCI
 depends on VIRTIO && VHOST_USER && LINUX
+
+config SWIM
+bool
diff --git a/hw/block/Makefile.objs b/hw/block/Makefile.objs
index f5f643f0cc..28c2495a00 100644
--- a/hw/block/Makefile.objs
+++ b/hw/block/Makefile.objs
@@ -8,6 +8,7 @@ common-obj-$(CONFIG_XEN) += xen-block.o
 common-obj-$(CONFIG_ECC) += ecc.o
 common-obj-$(CONFIG_ONENAND) += onenand.o
 common-obj-$(CONFIG_NVME_PCI) += nvme.o
+common-obj-$(CONFIG_SWIM) += swim.o
 
 obj-$(CONFIG_SH4) += tc58128.o
 
diff --git a/hw/block/swim.c b/hw/block/swim.c
new file mode 100644
index 00..80addcea9d
--- /dev/null
+++ b/hw/block/swim.c
@@ -0,0 +1,487 @@
+/*
+ * QEMU Macintosh floppy disk controller emulator (SWIM)
+ *
+ * Copyright (c) 2014-2018 Laurent Vivier 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.  See
+ * the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/main-loop.h"
+#include "qapi/error.h"
+#include "sysemu/block-backend.h"
+#include "hw/sysbus.h"
+#include "migration/vmstate.h"
+#include "hw/block/block.h"
+#include "hw/block/swim.h"
+#include "hw/qdev-properties.h"
+
+/* IWM registers */
+
+#define IWM_PH0L0
+#define IWM_PH0H1
+#define IWM_PH1L2
+#define IWM_PH1H3
+#define IWM_PH2L4
+#define IWM_PH2H5
+#define IWM_PH3L6
+#define IWM_PH3H7
+#define IWM_MTROFF  8
+#define IWM_MTRON   9
+#define IWM_INTDRIVE10
+#define IWM_EXTDRIVE11
+#define IWM_Q6L 12
+#define IWM_Q6H 13
+#define IWM_Q7L 14
+#define IWM_Q7H 15
+
+/* SWIM registers */
+
+#define SWIM_WRITE_DATA 0
+#define SWIM_WRITE_MARK 1
+#define SWIM_WRITE_CRC  2
+#define SWIM_WRITE_PARAMETER3
+#define SWIM_WRITE_PHASE4
+#define SWIM_WRITE_SETUP5
+#define SWIM_WRITE_MODE06
+#define SWIM_WRITE_MODE17
+
+#define SWIM_READ_DATA  8
+#define SWIM_READ_MARK  9
+#define SWIM_READ_ERROR 10
+#define SWIM_READ_PARAMETER 11
+#define SWIM_READ_PHASE 12
+#define SWIM_READ_SETUP 13
+#define SWIM_READ_STATUS14
+#define SWIM_READ_HANDSHAKE 15
+
+#define REG_SHIFT   9
+
+#define SWIM_MODE_IWM  0
+#define SWIM_MODE_SWIM 1
+
+/* bits in phase register */
+
+#define SWIM_SEEK_NEGATIVE   0x074
+#define SWIM_STEP0x071
+#define SWIM_MOTOR_ON0x072
+#define SWIM_MOTOR_OFF   0x076
+#define SWIM_INDEX   0x073
+#define SWIM_EJECT   0x077
+#define SWIM_SETMFM  0x171
+#define SWIM_SETGCR  0x175
+#define SWIM_RELAX   0x033
+#define SWIM_LSTRB   0x008
+#define SWIM_CA_MASK 0x077
+
+/* Select values for swim_select and swim_readbit */
+
+#define SWIM_READ_DATA_0 0x074
+#define SWIM_TWOMEG_DRIVE0x075
+#define SWIM_SINGLE_SIDED0x076
+#define SWIM_DRIVE_PRESENT   0x077
+#define SWIM_DISK_IN 0x170
+#define SWIM_WRITE_PROT  0x171
+#define SWIM_TRACK_ZERO  0x172
+#define SWIM_TACHO   0x173
+#define SWIM_READ_DATA_1 0x174
+#define SWIM_MFM_MODE0x175
+#define SWIM_SEEK_COMPLETE   0x176
+#define SWIM_ONEMEG_MEDIA0x177
+
+/* Bits in handshake register */
+
+#define SWIM_MARK_BYTE   0x01
+#define SWIM_CRC_ZERO0x02
+#define SWIM_RDDATA  0x04
+#define SWIM_SENSE   0x08
+#define SWIM_MOTEN   0x10
+#define SWIM_ERROR   0x20
+#define SWIM_DAT2BYTE0x40
+#define SWIM_DAT1BYTE0x80
+
+/* bits in setup register */
+
+#define SWIM_S_INV_WDATA 0x01
+#define SWIM_S_3_5_SELECT0x02
+#define SWIM_S_GCR   0x04
+#define SWIM_S_FCLK_DIV2 0x08
+#define SWIM_S_ERROR_CORR0x10
+#define SWIM_S_IBM_DR

[Qemu-block] [PATCH v11 3/9] hw/m68k: add via support

[Laurent Vivier]

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
Reviewed-by: Hervé Poussineau 
---
 MAINTAINERS  |   6 +
 default-configs/m68k-softmmu.mak |   1 +
 hw/m68k/Kconfig  |   4 +
 hw/misc/Kconfig  |   4 +
 hw/misc/Makefile.objs|   1 +
 hw/misc/mac_via.c| 722 +++
 include/hw/misc/mac_via.h| 107 +
 7 files changed, 845 insertions(+)
 create mode 100644 hw/misc/mac_via.c
 create mode 100644 include/hw/misc/mac_via.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 50eaf005f4..b01826ba39 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -917,6 +917,12 @@ F: hw/m68k/next-*.c
 F: hw/display/next-fb.c
 F: include/hw/m68k/next-cube.h
 
+q800
+M: Laurent Vivier 
+S: Maintained
+F: hw/misc/mac_via.c
+F: include/hw/misc/mac_via.h
+
 MicroBlaze Machines
 ---
 petalogix_s3adsp1800
diff --git a/default-configs/m68k-softmmu.mak b/default-configs/m68k-softmmu.mak
index d67ab8b96d..6629fd2aa3 100644
--- a/default-configs/m68k-softmmu.mak
+++ b/default-configs/m68k-softmmu.mak
@@ -7,3 +7,4 @@ CONFIG_SEMIHOSTING=y
 CONFIG_AN5206=y
 CONFIG_MCF5208=y
 CONFIG_NEXTCUBE=y
+CONFIG_Q800=y
diff --git a/hw/m68k/Kconfig b/hw/m68k/Kconfig
index a74fac5abd..22a357609c 100644
--- a/hw/m68k/Kconfig
+++ b/hw/m68k/Kconfig
@@ -12,3 +12,7 @@ config NEXTCUBE
 bool
 select FRAMEBUFFER
 select ESCC
+
+config Q800
+bool
+select MAC_VIA
diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig
index 51754bb47c..18a5dc9c09 100644
--- a/hw/misc/Kconfig
+++ b/hw/misc/Kconfig
@@ -120,4 +120,8 @@ config AUX
 config UNIMP
 bool
 
+config MAC_VIA
+bool
+select MOS6522
+
 source macio/Kconfig
diff --git a/hw/misc/Makefile.objs b/hw/misc/Makefile.objs
index e4aad707fb..c4836dd5c3 100644
--- a/hw/misc/Makefile.objs
+++ b/hw/misc/Makefile.objs
@@ -76,6 +76,7 @@ obj-$(CONFIG_PVPANIC) += pvpanic.o
 obj-$(CONFIG_AUX) += auxbus.o
 obj-$(CONFIG_ASPEED_SOC) += aspeed_xdma.o
 obj-$(CONFIG_ASPEED_SOC) += aspeed_scu.o aspeed_sdmc.o
+obj-$(CONFIG_MAC_VIA) += mac_via.o
 obj-$(CONFIG_MSF2) += msf2-sysreg.o
 obj-$(CONFIG_NRF51_SOC) += nrf51_rng.o
 
diff --git a/hw/misc/mac_via.c b/hw/misc/mac_via.c
new file mode 100644
index 00..a052259613
--- /dev/null
+++ b/hw/misc/mac_via.c
@@ -0,0 +1,722 @@
+/*
+ * QEMU m68k Macintosh VIA device support
+ *
+ * Copyright (c) 2011-2018 Laurent Vivier
+ * Copyright (c) 2018 Mark Cave-Ayland
+ *
+ * Some parts from hw/misc/macio/cuda.c
+ *
+ * Copyright (c) 2004-2007 Fabrice Bellard
+ * Copyright (c) 2007 Jocelyn Mayer
+ *
+ * some parts from linux-2.6.29, arch/m68k/include/asm/mac_via.h
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu-common.h"
+#include "migration/vmstate.h"
+#include "hw/sysbus.h"
+#include "hw/irq.h"
+#include "qemu/timer.h"
+#include "hw/misc/mac_via.h"
+#include "hw/misc/mos6522.h"
+#include "hw/input/adb.h"
+#include "sysemu/runstate.h"
+#include "qapi/error.h"
+#include "qemu/cutils.h"
+
+
+/*
+ * VIAs: There are two in every machine,
+ */
+
+#define VIA_SIZE (0x2000)
+
+/*
+ * Not all of these are true post MacII I think.
+ * CSA: probably the ones CHRP marks as 'unused' change purposes
+ * when the IWM becomes the SWIM.
+ * http://www.rs6000.ibm.com/resource/technology/chrpio/via5.mak.html
+ * ftp://ftp.austin.ibm.com/pub/technology/spec/chrp/inwork/CHRP_IORef_1.0.pdf
+ *
+ * also, http://developer.apple.com/technotes/hw/hw_09.html claims the
+ * following changes for IIfx:
+ * VIA1A_vSccWrReq not available and that VIA1A_vSync has moved to an IOP.
+ * Also, "All of the functionality of VIA2 has been moved to other chips".
+ */
+
+#define VIA1A_vSccWrReq 0x80   /*
+* SCC write. (input)
+* [CHRP] SCC WREQ: Reflects the state of the
+* Wait/Request pins from the SCC.
+* [Macintosh Family Hardware]
+* as CHRP on SE/30,II,IIx,IIcx,IIci.
+* on IIfx, "0 means an active request"
+*/
+#define VIA1A_vRev8 0x40   /*
+* Revision 8 board ???
+* [CHRP] En WaitReqB: Lets the WaitReq_L
+* signal from port B of the SCC appear on
+* the PA7 input pin. Output.
+* [Macintosh Family] On the SE/30, this
+* is the bit to flip screen buffers.
+* 0=alternate, 1=main.
+* on II,IIx,IIcx,IIci,IIfx this is a bit
+* for Rev ID. 0=II,IIx, 1=IIcx,IIci,IIfx
+*/
+#define VIA

[Qemu-block] [PATCH v11 9/9] hw/m68k: define Macintosh Quadra 800

[Laurent Vivier]

If you want to test the machine, it doesn't yet boot a MacROM, but you can
boot a linux kernel from the command line.

You can install your own disk using debian-installer with:

./qemu-system-m68k \
-M q800 \
-serial none -serial mon:stdio \
-m 1000M -drive file=m68k.qcow2,format=qcow2 \
-net nic,model=dp83932,addr=09:00:07:12:34:57 \
-append "console=ttyS0 vga=off" \
-kernel vmlinux-4.15.0-2-m68k \
-initrd initrd.gz \
-drive file=debian-9.0-m68k-NETINST-1.iso \
-drive file=m68k.qcow2,format=qcow2 \
-nographic

If you use a graphic adapter instead of "-nographic", you can use "-g" to set 
the
size of the display (I use "-g 1600x800x24").

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
---
 MAINTAINERS   |   2 +
 hw/m68k/Kconfig   |   3 +
 hw/m68k/Makefile.objs |   1 +
 hw/m68k/bootinfo.h| 114 +
 hw/m68k/q800.c| 382 ++
 5 files changed, 502 insertions(+)
 create mode 100644 hw/m68k/bootinfo.h
 create mode 100644 hw/m68k/q800.c

diff --git a/MAINTAINERS b/MAINTAINERS
index f85f11d83c..e6d37acb84 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -920,10 +920,12 @@ F: include/hw/m68k/next-cube.h
 q800
 M: Laurent Vivier 
 S: Maintained
+F: hw/m68k/q800.c
 F: hw/misc/mac_via.c
 F: hw/display/macfb.c
 F: hw/nubus/*
 F: hw/block/swim.c
+F: hw/m68k/bootinfo.h
 F: include/hw/misc/mac_via.h
 F: include/hw/display/macfb.h
 F: include/hw/nubus/*
diff --git a/hw/m68k/Kconfig b/hw/m68k/Kconfig
index 7aa830327c..c663920f8f 100644
--- a/hw/m68k/Kconfig
+++ b/hw/m68k/Kconfig
@@ -19,3 +19,6 @@ config Q800
 select MACFB
 select NUBUS
 select SWIM
+select ESCC
+select ESP
+select DP8393X
diff --git a/hw/m68k/Makefile.objs b/hw/m68k/Makefile.objs
index f25854730d..b2c9e5ab12 100644
--- a/hw/m68k/Makefile.objs
+++ b/hw/m68k/Makefile.objs
@@ -1,3 +1,4 @@
 obj-$(CONFIG_AN5206) += an5206.o mcf5206.o
 obj-$(CONFIG_MCF5208) += mcf5208.o mcf_intc.o
 obj-$(CONFIG_NEXTCUBE) += next-kbd.o next-cube.o
+obj-$(CONFIG_Q800) += q800.o
diff --git a/hw/m68k/bootinfo.h b/hw/m68k/bootinfo.h
new file mode 100644
index 00..5f8ded2686
--- /dev/null
+++ b/hw/m68k/bootinfo.h
@@ -0,0 +1,114 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note
+ *
+ * Bootinfo tags from linux bootinfo.h and bootinfo-mac.h:
+ * This is an easily parsable and extendable structure containing all
+ * information to be passed from the bootstrap to the kernel
+ *
+ * This structure is copied right after the kernel by the bootstrap
+ * routine.
+ */
+
+#ifndef HW_M68K_BOOTINFO_H
+#define HW_M68K_BOOTINFO_H
+struct bi_record {
+uint16_t tag;/* tag ID */
+uint16_t size;   /* size of record */
+uint32_t data[0];/* data */
+};
+
+/* machine independent tags */
+
+#define BI_LAST 0x /* last record */
+#define BI_MACHTYPE 0x0001 /* machine type (u_long) */
+#define BI_CPUTYPE  0x0002 /* cpu type (u_long) */
+#define BI_FPUTYPE  0x0003 /* fpu type (u_long) */
+#define BI_MMUTYPE  0x0004 /* mmu type (u_long) */
+#define BI_MEMCHUNK 0x0005 /* memory chunk address and size */
+   /* (struct mem_info) */
+#define BI_RAMDISK  0x0006 /* ramdisk address and size */
+   /* (struct mem_info) */
+#define BI_COMMAND_LINE 0x0007 /* kernel command line parameters */
+   /* (string) */
+
+/*  Macintosh-specific tags (all u_long) */
+
+#define BI_MAC_MODEL0x8000  /* Mac Gestalt ID (model type) */
+#define BI_MAC_VADDR0x8001  /* Mac video base address */
+#define BI_MAC_VDEPTH   0x8002  /* Mac video depth */
+#define BI_MAC_VROW 0x8003  /* Mac video rowbytes */
+#define BI_MAC_VDIM 0x8004  /* Mac video dimensions */
+#define BI_MAC_VLOGICAL 0x8005  /* Mac video logical base */
+#define BI_MAC_SCCBASE  0x8006  /* Mac SCC base address */
+#define BI_MAC_BTIME0x8007  /* Mac boot time */
+#define BI_MAC_GMTBIAS  0x8008  /* Mac GMT timezone offset */
+#define BI_MAC_MEMSIZE  0x8009  /* Mac RAM size (sanity check) */
+#define BI_MAC_CPUID0x800a  /* Mac CPU type (sanity check) */
+#define BI_MAC_ROMBASE  0x800b  /* Mac system ROM base address */
+
+/*  Macintosh hardware profile data */
+
+#define BI_MAC_VIA1BASE 0x8010  /* Mac VIA1 base address (always present) */
+#define BI_MAC_VIA2BASE 0x8011  /* Mac VIA2 base address (type varies) */
+#define BI_MAC_VIA2TYPE 0x8012  /* Mac VIA2 type (VIA, RBV, OSS) */
+#define BI_MAC_ADBTYPE  0x8013  /* Mac ADB interface type */
+#define BI_MAC_ASCBASE  0x8014  /* Mac Apple Sound Chip base address */
+#define BI_MAC_SCSI5380 0x8015  /* Mac NCR 5380 SCSI (base address, multi) */
+#define BI_MAC_SCSIDMA  0x8016  /* Mac SCSI DMA (base address) */
+#define BI_MAC_SCSI5396 0x8017  /* Mac NCR 53C96 SCSI (base address, multi) */
+#define BI_MAC_IDETYPE  0x8018  /* Mac IDE interface type */
+#d

[Qemu-block] [PATCH v11 5/9] hw/m68k: add macfb video card

[Laurent Vivier]

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
Reviewed-by: Hervé Poussineau 
Reviewed-by: Thomas Huth 
---
 MAINTAINERS|   2 +
 arch_init.c|   4 +
 hw/display/Kconfig |   4 +
 hw/display/Makefile.objs   |   1 +
 hw/display/macfb.c | 421 +
 hw/m68k/Kconfig|   1 +
 include/hw/display/macfb.h |  43 
 qemu-options.hx|   2 +-
 vl.c   |   3 +-
 9 files changed, 479 insertions(+), 2 deletions(-)
 create mode 100644 hw/display/macfb.c
 create mode 100644 include/hw/display/macfb.h

diff --git a/MAINTAINERS b/MAINTAINERS
index b01826ba39..6b5fc50aef 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -921,7 +921,9 @@ q800
 M: Laurent Vivier 
 S: Maintained
 F: hw/misc/mac_via.c
+F: hw/display/macfb.c
 F: include/hw/misc/mac_via.h
+F: include/hw/display/macfb.h
 
 MicroBlaze Machines
 ---
diff --git a/arch_init.c b/arch_init.c
index 0a1531124c..705d0b94ad 100644
--- a/arch_init.c
+++ b/arch_init.c
@@ -38,6 +38,10 @@
 int graphic_width = 1024;
 int graphic_height = 768;
 int graphic_depth = 8;
+#elif defined(TARGET_M68K)
+int graphic_width = 800;
+int graphic_height = 600;
+int graphic_depth = 8;
 #else
 int graphic_width = 800;
 int graphic_height = 600;
diff --git a/hw/display/Kconfig b/hw/display/Kconfig
index cbdf7b1a67..32e8d29003 100644
--- a/hw/display/Kconfig
+++ b/hw/display/Kconfig
@@ -132,3 +132,7 @@ config ATI_VGA
 select VGA
 select BITBANG_I2C
 select DDC
+
+config MACFB
+bool
+select FRAMEBUFFER
diff --git a/hw/display/Makefile.objs b/hw/display/Makefile.objs
index 5a4066383b..f2182e3bef 100644
--- a/hw/display/Makefile.objs
+++ b/hw/display/Makefile.objs
@@ -26,6 +26,7 @@ common-obj-$(CONFIG_EXYNOS4) += exynos4210_fimd.o
 common-obj-$(CONFIG_FRAMEBUFFER) += framebuffer.o
 obj-$(CONFIG_MILKYMIST) += milkymist-vgafb.o
 common-obj-$(CONFIG_ZAURUS) += tc6393xb.o
+common-obj-$(CONFIG_MACFB) += macfb.o
 
 obj-$(CONFIG_MILKYMIST_TMU2) += milkymist-tmu2.o
 milkymist-tmu2.o-cflags := $(X11_CFLAGS) $(OPENGL_CFLAGS)
diff --git a/hw/display/macfb.c b/hw/display/macfb.c
new file mode 100644
index 00..d7c5ef296e
--- /dev/null
+++ b/hw/display/macfb.c
@@ -0,0 +1,421 @@
+/*
+ * QEMU Motorola 680x0 Macintosh Video Card Emulation
+ * Copyright (c) 2012-2018 Laurent Vivier
+ *
+ * some parts from QEMU G364 framebuffer Emulator.
+ * Copyright (c) 2007-2011 Herve Poussineau
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/units.h"
+#include "hw/sysbus.h"
+#include "ui/console.h"
+#include "ui/pixel_ops.h"
+#include "hw/display/macfb.h"
+#include "qapi/error.h"
+#include "hw/qdev-properties.h"
+#include "migration/vmstate.h"
+
+#define VIDEO_BASE 0x1000
+#define DAFB_BASE  0x0080
+
+#define MACFB_PAGE_SIZE 4096
+#define MACFB_VRAM_SIZE (4 * MiB)
+
+#define DAFB_RESET  0x200
+#define DAFB_LUT0x213
+
+
+typedef void macfb_draw_line_func(MacfbState *s, uint8_t *d, uint32_t addr,
+  int width);
+
+static inline uint8_t macfb_read_byte(MacfbState *s, uint32_t addr)
+{
+return s->vram[addr & s->vram_bit_mask];
+}
+
+/* 1-bit color */
+static void macfb_draw_line1(MacfbState *s, uint8_t *d, uint32_t addr,
+ int width)
+{
+uint8_t r, g, b;
+int x;
+
+for (x = 0; x < width; x++) {
+int bit = x & 7;
+int idx = (macfb_read_byte(s, addr) >> (7 - bit)) & 1;
+r = g = b  = ((1 - idx) << 7);
+addr += (bit == 7);
+
+*(uint32_t *)d = rgb_to_pixel32(r, g, b);
+d += 4;
+}
+}
+
+/* 2-bit color */
+static void macfb_draw_line2(MacfbState *s, uint8_t *d, uint32_t addr,
+ int width)
+{
+uint8_t r, g, b;
+int x;
+
+for (x = 0; x < width; x++) {
+int bit = (x & 3);
+int idx = (macfb_read_byte(s, addr) >> ((3 - bit) << 1)) & 3;
+r = s->color_palette[idx * 3];
+g = s->color_palette[idx * 3 + 1];
+b = s->color_palette[idx * 3 + 2];
+addr += (bit == 3);
+
+*(uint32_t *)d = rgb_to_pixel32(r, g, b);
+d += 4;
+}
+}
+
+/* 4-bit color */
+static void macfb_draw_line4(MacfbState *s, uint8_t *d, uint32_t addr,
+ int width)
+{
+uint8_t r, g, b;
+int x;
+
+for (x = 0; x < width; x++) {
+int bit = x & 1;
+int idx = (macfb_read_byte(s, addr) >> ((1 - bit) << 2)) & 15;
+r = s->color_palette[idx * 3];
+g = s->color_palette[idx * 3 + 1];
+b = s->color_palette[idx * 3 + 2];
+addr += (bit == 1);
+
+*(uint32_t *)d = rgb_to_pixel32(r, g, b);
+d += 4;
+}
+}
+
+/* 8-bit color */
+static void macfb_draw_line8(MacfbState *s, uint8_

[Qemu-block] [PATCH v11 4/9] hw/m68k: implement ADB bus support for via

[Laurent Vivier]

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
Reviewed-by: Hervé Poussineau 
Reviewed-by: Thomas Huth 
---
 hw/misc/Kconfig   |   1 +
 hw/misc/mac_via.c | 198 +-
 include/hw/misc/mac_via.h |   7 ++
 3 files changed, 205 insertions(+), 1 deletion(-)

diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig
index 18a5dc9c09..2164646553 100644
--- a/hw/misc/Kconfig
+++ b/hw/misc/Kconfig
@@ -123,5 +123,6 @@ config UNIMP
 config MAC_VIA
 bool
 select MOS6522
+select ADB
 
 source macio/Kconfig
diff --git a/hw/misc/mac_via.c b/hw/misc/mac_via.c
index a052259613..2f54db9e1e 100644
--- a/hw/misc/mac_via.c
+++ b/hw/misc/mac_via.c
@@ -264,10 +264,16 @@
  * Table 19-10 ADB transaction states
  */
 
+#define ADB_STATE_NEW   0
+#define ADB_STATE_EVEN  1
+#define ADB_STATE_ODD   2
+#define ADB_STATE_IDLE  3
+
 #define VIA1B_vADB_StateMask(VIA1B_vADBS1 | VIA1B_vADBS2)
 #define VIA1B_vADB_StateShift   4
 
 #define VIA_TIMER_FREQ (783360)
+#define VIA_ADB_POLL_FREQ 50 /* XXX: not real */
 
 /* VIA returns time offset from Jan 1, 1904, not 1970 */
 #define RTC_OFFSET 2082844800
@@ -449,6 +455,181 @@ static void via1_rtc_update(MacVIAState *m)
 }
 }
 
+static int adb_via_poll(MacVIAState *s, int state, uint8_t *data)
+{
+if (state != ADB_STATE_IDLE) {
+return 0;
+}
+
+if (s->adb_data_in_size < s->adb_data_in_index) {
+return 0;
+}
+
+if (s->adb_data_out_index != 0) {
+return 0;
+}
+
+s->adb_data_in_index = 0;
+s->adb_data_out_index = 0;
+s->adb_data_in_size = adb_poll(&s->adb_bus, s->adb_data_in, 0x);
+
+if (s->adb_data_in_size) {
+*data = s->adb_data_in[s->adb_data_in_index++];
+qemu_irq_raise(s->adb_data_ready);
+}
+
+return s->adb_data_in_size;
+}
+
+static int adb_via_send(MacVIAState *s, int state, uint8_t data)
+{
+switch (state) {
+case ADB_STATE_NEW:
+s->adb_data_out_index = 0;
+break;
+case ADB_STATE_EVEN:
+if ((s->adb_data_out_index & 1) == 0) {
+return 0;
+}
+break;
+case ADB_STATE_ODD:
+if (s->adb_data_out_index & 1) {
+return 0;
+}
+break;
+case ADB_STATE_IDLE:
+return 0;
+}
+
+assert(s->adb_data_out_index < sizeof(s->adb_data_out) - 1);
+
+s->adb_data_out[s->adb_data_out_index++] = data;
+qemu_irq_raise(s->adb_data_ready);
+return 1;
+}
+
+static int adb_via_receive(MacVIAState *s, int state, uint8_t *data)
+{
+switch (state) {
+case ADB_STATE_NEW:
+return 0;
+
+case ADB_STATE_EVEN:
+if (s->adb_data_in_size <= 0) {
+qemu_irq_raise(s->adb_data_ready);
+return 0;
+}
+
+if (s->adb_data_in_index >= s->adb_data_in_size) {
+*data = 0;
+qemu_irq_raise(s->adb_data_ready);
+return 1;
+}
+
+if ((s->adb_data_in_index & 1) == 0) {
+return 0;
+}
+
+break;
+
+case ADB_STATE_ODD:
+if (s->adb_data_in_size <= 0) {
+qemu_irq_raise(s->adb_data_ready);
+return 0;
+}
+
+if (s->adb_data_in_index >= s->adb_data_in_size) {
+*data = 0;
+qemu_irq_raise(s->adb_data_ready);
+return 1;
+}
+
+if (s->adb_data_in_index & 1) {
+return 0;
+}
+
+break;
+
+case ADB_STATE_IDLE:
+if (s->adb_data_out_index == 0) {
+return 0;
+}
+
+s->adb_data_in_size = adb_request(&s->adb_bus, s->adb_data_in,
+  s->adb_data_out,
+  s->adb_data_out_index);
+s->adb_data_out_index = 0;
+s->adb_data_in_index = 0;
+if (s->adb_data_in_size < 0) {
+*data = 0xff;
+qemu_irq_raise(s->adb_data_ready);
+return -1;
+}
+
+if (s->adb_data_in_size == 0) {
+return 0;
+}
+
+break;
+}
+
+assert(s->adb_data_in_index < sizeof(s->adb_data_in) - 1);
+
+*data = s->adb_data_in[s->adb_data_in_index++];
+qemu_irq_raise(s->adb_data_ready);
+if (*data == 0xff || *data == 0) {
+return 0;
+}
+return 1;
+}
+
+static void via1_adb_update(MacVIAState *m)
+{
+MOS6522Q800VIA1State *v1s = MOS6522_Q800_VIA1(&m->mos6522_via1);
+MOS6522State *s = MOS6522(v1s);
+int state;
+int ret;
+
+state = (s->b & VIA1B_vADB_StateMask) >> VIA1B_vADB_StateShift;
+
+if (s->acr & VIA1ACR_vShiftOut) {
+/* output mode */
+ret = adb_via_send(m, state, s->sr);
+if (ret > 0) {
+s->b &= ~VIA1B_vADBInt;
+} else {
+s->b |= VIA1B_vADBInt;
+}
+} else {
+/* input mode */
+ret = adb_via_receive(m, state, &s->sr);
+if (ret > 0 && s->sr != 

[Qemu-block] [PATCH v11 1/9] esp: add pseudo-DMA as used by Macintosh

[Laurent Vivier]

There is no DMA in Quadra 800, so the CPU reads/writes the data from the
PDMA register (offset 0x100, ESP_PDMA in hw/m68k/q800.c) and copies them
to/from the memory.

There is a nice assembly loop in the kernel to do that, see
linux/drivers/scsi/mac_esp.c:MAC_ESP_PDMA_LOOP().

The start of the transfer is triggered by the DREQ interrupt (see linux
mac_esp_send_pdma_cmd()), the CPU polls on the IRQ flag to start the
transfer after a SCSI command has been sent (in Quadra 800 it goes
through the VIA2, the via2-irq line and the vIFR register)

The Macintosh hardware includes hardware handshaking to prevent the CPU
from reading invalid data or writing data faster than the peripheral
device can accept it.

This is the "blind mode", and from the doc:
"Approximate maximum SCSI transfer rates within a blocks are 1.4 MB per
second for blind transfers in the Macintosh II"

Some references can be found in:
  Apple Macintosh Family Hardware Reference, ISBN 0-201-19255-1
  Guide to the Macintosh Family Hardware, ISBN-0-201-52405-8

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
---
 hw/scsi/esp.c | 338 ++
 include/hw/scsi/esp.h |  15 ++
 2 files changed, 324 insertions(+), 29 deletions(-)

diff --git a/hw/scsi/esp.c b/hw/scsi/esp.c
index 841d79b60e..90b40c4cb5 100644
--- a/hw/scsi/esp.c
+++ b/hw/scsi/esp.c
@@ -38,6 +38,8 @@
  * 
http://www.ibiblio.org/pub/historic-linux/early-ports/Sparc/NCR/NCR89C100.txt
  * and
  * http://www.ibiblio.org/pub/historic-linux/early-ports/Sparc/NCR/NCR53C9X.txt
+ *
+ * On Macintosh Quadra it is a NCR53C96.
  */
 
 static void esp_raise_irq(ESPState *s)
@@ -58,6 +60,16 @@ static void esp_lower_irq(ESPState *s)
 }
 }
 
+static void esp_raise_drq(ESPState *s)
+{
+qemu_irq_raise(s->irq_data);
+}
+
+static void esp_lower_drq(ESPState *s)
+{
+qemu_irq_lower(s->irq_data);
+}
+
 void esp_dma_enable(ESPState *s, int irq, int level)
 {
 if (level) {
@@ -84,29 +96,35 @@ void esp_request_cancelled(SCSIRequest *req)
 }
 }
 
-static uint32_t get_cmd(ESPState *s, uint8_t *buf, uint8_t buflen)
+static void set_pdma(ESPState *s, enum pdma_origin_id origin,
+ uint32_t index, uint32_t len)
+{
+s->pdma_origin = origin;
+s->pdma_start = index;
+s->pdma_cur = index;
+s->pdma_len = len;
+}
+
+static uint8_t *get_pdma_buf(ESPState *s)
+{
+switch (s->pdma_origin) {
+case PDMA:
+return s->pdma_buf;
+case TI:
+return s->ti_buf;
+case CMD:
+return s->cmdbuf;
+case ASYNC:
+return s->async_buf;
+}
+return NULL;
+}
+
+static int get_cmd_cb(ESPState *s)
 {
-uint32_t dmalen;
 int target;
 
 target = s->wregs[ESP_WBUSID] & BUSID_DID;
-if (s->dma) {
-dmalen = s->rregs[ESP_TCLO];
-dmalen |= s->rregs[ESP_TCMID] << 8;
-dmalen |= s->rregs[ESP_TCHI] << 16;
-if (dmalen > buflen) {
-return 0;
-}
-s->dma_memory_read(s->dma_opaque, buf, dmalen);
-} else {
-dmalen = s->ti_size;
-if (dmalen > TI_BUFSZ) {
-return 0;
-}
-memcpy(buf, s->ti_buf, dmalen);
-buf[0] = buf[2] >> 5;
-}
-trace_esp_get_cmd(dmalen, target);
 
 s->ti_size = 0;
 s->ti_rptr = 0;
@@ -125,8 +143,46 @@ static uint32_t get_cmd(ESPState *s, uint8_t *buf, uint8_t 
buflen)
 s->rregs[ESP_RINTR] = INTR_DC;
 s->rregs[ESP_RSEQ] = SEQ_0;
 esp_raise_irq(s);
+return -1;
+}
+return 0;
+}
+
+static uint32_t get_cmd(ESPState *s, uint8_t *buf, uint8_t buflen)
+{
+int target;
+uint32_t dmalen;
+
+target = s->wregs[ESP_WBUSID] & BUSID_DID;
+if (s->dma) {
+dmalen = s->rregs[ESP_TCLO];
+dmalen |= s->rregs[ESP_TCMID] << 8;
+dmalen |= s->rregs[ESP_TCHI] << 16;
+if (dmalen > buflen) {
+return 0;
+}
+if (s->dma_memory_read) {
+s->dma_memory_read(s->dma_opaque, buf, dmalen);
+} else {
+memcpy(s->pdma_buf, buf, dmalen);
+set_pdma(s, PDMA, 0, dmalen);
+esp_raise_drq(s);
+return 0;
+}
+} else {
+dmalen = s->ti_size;
+if (dmalen > TI_BUFSZ) {
+return 0;
+}
+memcpy(buf, s->ti_buf, dmalen);
+buf[0] = buf[2] >> 5;
+}
+trace_esp_get_cmd(dmalen, target);
+
+if (get_cmd_cb(s) < 0) {
 return 0;
 }
+
 return dmalen;
 }
 
@@ -165,6 +221,16 @@ static void do_cmd(ESPState *s, uint8_t *buf)
 do_busid_cmd(s, &buf[1], busid);
 }
 
+static void satn_pdma_cb(ESPState *s)
+{
+if (get_cmd_cb(s) < 0) {
+return;
+}
+if (s->pdma_cur != s->pdma_start) {
+do_cmd(s, get_pdma_buf(s) + s->pdma_start);
+}
+}
+
 static void handle_satn(ESPState *s)
 {
 uint8_t buf[32];
@@ -174,11 +240,22 @@ static void handle_satn(ESPState *s)
 s->dma_cb = handle_sa

[Qemu-block] [PATCH v11 2/9] dp8393x: manage big endian bus

[Laurent Vivier]

This is needed by Quadra 800, this card can run on little-endian
or big-endian bus.

Signed-off-by: Laurent Vivier 
Tested-by: Hervé Poussineau 
Reviewed-by: Philippe Mathieu-Daudé 
Reviewed-by: Hervé Poussineau 
---
 hw/net/dp8393x.c | 88 +++-
 1 file changed, 57 insertions(+), 31 deletions(-)

diff --git a/hw/net/dp8393x.c b/hw/net/dp8393x.c
index a5678e11fa..693e244ce6 100644
--- a/hw/net/dp8393x.c
+++ b/hw/net/dp8393x.c
@@ -153,6 +153,7 @@ typedef struct dp8393xState {
 
 /* Hardware */
 uint8_t it_shift;
+bool big_endian;
 qemu_irq irq;
 #ifdef DEBUG_SONIC
 int irq_level;
@@ -223,6 +224,29 @@ static uint32_t dp8393x_wt(dp8393xState *s)
 return s->regs[SONIC_WT1] << 16 | s->regs[SONIC_WT0];
 }
 
+static uint16_t dp8393x_get(dp8393xState *s, int width, uint16_t *base,
+int offset)
+{
+uint16_t val;
+
+if (s->big_endian) {
+val = be16_to_cpu(base[offset * width + width - 1]);
+} else {
+val = le16_to_cpu(base[offset * width]);
+}
+return val;
+}
+
+static void dp8393x_put(dp8393xState *s, int width, uint16_t *base, int offset,
+uint16_t val)
+{
+if (s->big_endian) {
+base[offset * width + width - 1] = cpu_to_be16(val);
+} else {
+base[offset * width] = cpu_to_le16(val);
+}
+}
+
 static void dp8393x_update_irq(dp8393xState *s)
 {
 int level = (s->regs[SONIC_IMR] & s->regs[SONIC_ISR]) ? 1 : 0;
@@ -254,12 +278,12 @@ static void dp8393x_do_load_cam(dp8393xState *s)
 /* Fill current entry */
 address_space_rw(&s->as, dp8393x_cdp(s),
 MEMTXATTRS_UNSPECIFIED, (uint8_t *)data, size, 0);
-s->cam[index][0] = data[1 * width] & 0xff;
-s->cam[index][1] = data[1 * width] >> 8;
-s->cam[index][2] = data[2 * width] & 0xff;
-s->cam[index][3] = data[2 * width] >> 8;
-s->cam[index][4] = data[3 * width] & 0xff;
-s->cam[index][5] = data[3 * width] >> 8;
+s->cam[index][0] = dp8393x_get(s, width, data, 1) & 0xff;
+s->cam[index][1] = dp8393x_get(s, width, data, 1) >> 8;
+s->cam[index][2] = dp8393x_get(s, width, data, 2) & 0xff;
+s->cam[index][3] = dp8393x_get(s, width, data, 2) >> 8;
+s->cam[index][4] = dp8393x_get(s, width, data, 3) & 0xff;
+s->cam[index][5] = dp8393x_get(s, width, data, 3) >> 8;
 DPRINTF("load cam[%d] with %02x%02x%02x%02x%02x%02x\n", index,
 s->cam[index][0], s->cam[index][1], s->cam[index][2],
 s->cam[index][3], s->cam[index][4], s->cam[index][5]);
@@ -272,7 +296,7 @@ static void dp8393x_do_load_cam(dp8393xState *s)
 /* Read CAM enable */
 address_space_rw(&s->as, dp8393x_cdp(s),
 MEMTXATTRS_UNSPECIFIED, (uint8_t *)data, size, 0);
-s->regs[SONIC_CE] = data[0 * width];
+s->regs[SONIC_CE] = dp8393x_get(s, width, data, 0);
 DPRINTF("load cam done. cam enable mask 0x%04x\n", s->regs[SONIC_CE]);
 
 /* Done */
@@ -293,10 +317,10 @@ static void dp8393x_do_read_rra(dp8393xState *s)
 MEMTXATTRS_UNSPECIFIED, (uint8_t *)data, size, 0);
 
 /* Update SONIC registers */
-s->regs[SONIC_CRBA0] = data[0 * width];
-s->regs[SONIC_CRBA1] = data[1 * width];
-s->regs[SONIC_RBWC0] = data[2 * width];
-s->regs[SONIC_RBWC1] = data[3 * width];
+s->regs[SONIC_CRBA0] = dp8393x_get(s, width, data, 0);
+s->regs[SONIC_CRBA1] = dp8393x_get(s, width, data, 1);
+s->regs[SONIC_RBWC0] = dp8393x_get(s, width, data, 2);
+s->regs[SONIC_RBWC1] = dp8393x_get(s, width, data, 3);
 DPRINTF("CRBA0/1: 0x%04x/0x%04x, RBWC0/1: 0x%04x/0x%04x\n",
 s->regs[SONIC_CRBA0], s->regs[SONIC_CRBA1],
 s->regs[SONIC_RBWC0], s->regs[SONIC_RBWC1]);
@@ -411,12 +435,12 @@ static void dp8393x_do_transmit_packets(dp8393xState *s)
 tx_len = 0;
 
 /* Update registers */
-s->regs[SONIC_TCR] = data[0 * width] & 0xf000;
-s->regs[SONIC_TPS] = data[1 * width];
-s->regs[SONIC_TFC] = data[2 * width];
-s->regs[SONIC_TSA0] = data[3 * width];
-s->regs[SONIC_TSA1] = data[4 * width];
-s->regs[SONIC_TFS] = data[5 * width];
+s->regs[SONIC_TCR] = dp8393x_get(s, width, data, 0) & 0xf000;
+s->regs[SONIC_TPS] = dp8393x_get(s, width, data, 1);
+s->regs[SONIC_TFC] = dp8393x_get(s, width, data, 2);
+s->regs[SONIC_TSA0] = dp8393x_get(s, width, data, 3);
+s->regs[SONIC_TSA1] = dp8393x_get(s, width, data, 4);
+s->regs[SONIC_TFS] = dp8393x_get(s, width, data, 5);
 
 /* Handle programmable interrupt */
 if (s->regs[SONIC_TCR] & SONIC_TCR_PINT) {
@@ -442,9 +466,9 @@ static void dp8393x_do_transmit_packets(dp8393xState *s)
 address_space_rw(&s->as,
 dp8393x_ttda(s) + sizeof(uint16_t) * (4 + 3 * i) * width,
 MEMTXATTRS_UNSPECIFIED, (uint8_t *)data, size, 0);
-s->regs[SONIC_TSA0

Re: [Qemu-block] [PATCH 0/7] Move qtests to a separate folder

[Eric Blake]

On 9/10/19 1:58 PM, Thomas Huth wrote:
> Our "tests" directory is very overcrowded - we store the qtests,
> unit test and other files there. That makes it difficult to
> determine which file belongs to each test subsystem, and the
> wildcards in the MAINTAINERS file are inaccurate, too.
> 
> Let's clean up this mess. The first patches disentangle some
> dependencies, and the last three patches then move the qtests
> and libqos (which is a subsystem of the qtests) to a new folder
> called "tests/qtest/".
> 

I'd also welcome a rename of tests/qemu-iotests to tests/iotests.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-block] [PATCH 0/3] proper locking on bitmap add/remove paths

[Vladimir Sementsov-Ogievskiy]

We need to lock qcow2 mutex on accessing in-image metadata, especially
on updating this metadata. Let's implement it.

Vladimir Sementsov-Ogievskiy (3):
  block: move bdrv_can_store_new_dirty_bitmap to block/dirty-bitmap.c
  block/dirty-bitmap: return int from
bdrv_remove_persistent_dirty_bitmap
  block/qcow2: proper locking on bitmap add/remove paths

 block/qcow2.h|  14 ++---
 include/block/block_int.h|  14 ++---
 include/block/dirty-bitmap.h |   5 +-
 block.c  |  22 ---
 block/dirty-bitmap.c | 119 +--
 block/qcow2-bitmap.c |  36 +++
 block/qcow2.c|   5 +-
 blockdev.c   |  28 +++--
 8 files changed, 163 insertions(+), 80 deletions(-)

-- 
2.18.0

[Qemu-block] [PATCH 3/3] block/qcow2: proper locking on bitmap add/remove paths

[Vladimir Sementsov-Ogievskiy]

qmp_block_dirty_bitmap_add and do_block_dirty_bitmap_remove do acquire
aio context since 0a6c86d024c52b. But this is not enough: we also must
lock qcow2 mutex when access in-image metadata. Especially it concerns
freeing qcow2 clusters.

To achieve this, move qcow2_can_store_new_dirty_bitmap and
qcow2_remove_persistent_dirty_bitmap to coroutine context.

Since we work in coroutines in correct aio context, we don't need
context acquiring in blockdev.c anymore, drop it.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block/qcow2.h |  11 ++--
 include/block/block_int.h |  10 ++--
 block/dirty-bitmap.c  | 102 +++---
 block/qcow2-bitmap.c  |  22 +---
 block/qcow2.c |   5 +-
 blockdev.c|  27 +++---
 6 files changed, 130 insertions(+), 47 deletions(-)

diff --git a/block/qcow2.h b/block/qcow2.h
index 99ee88f802..27e20c9b4a 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -743,12 +743,13 @@ int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error 
**errp);
 int qcow2_truncate_bitmaps_check(BlockDriverState *bs, Error **errp);
 void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp);
 int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error **errp);
-bool qcow2_can_store_new_dirty_bitmap(BlockDriverState *bs,
-  const char *name,
-  uint32_t granularity,
-  Error **errp);
-int qcow2_remove_persistent_dirty_bitmap(BlockDriverState *bs, const char 
*name,
+bool qcow2_co_can_store_new_dirty_bitmap(BlockDriverState *bs,
+ const char *name,
+ uint32_t granularity,
  Error **errp);
+int qcow2_co_remove_persistent_dirty_bitmap(BlockDriverState *bs,
+const char *name,
+Error **errp);
 
 ssize_t coroutine_fn
 qcow2_co_compress(BlockDriverState *bs, void *dest, size_t dest_size,
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 503ac9e3cd..1e54486ad1 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -552,13 +552,13 @@ struct BlockDriver {
  * field of BlockDirtyBitmap's in case of success.
  */
 int (*bdrv_reopen_bitmaps_rw)(BlockDriverState *bs, Error **errp);
-bool (*bdrv_can_store_new_dirty_bitmap)(BlockDriverState *bs,
-const char *name,
-uint32_t granularity,
-Error **errp);
-int (*bdrv_remove_persistent_dirty_bitmap)(BlockDriverState *bs,
+bool (*bdrv_co_can_store_new_dirty_bitmap)(BlockDriverState *bs,
const char *name,
+   uint32_t granularity,
Error **errp);
+int (*bdrv_co_remove_persistent_dirty_bitmap)(BlockDriverState *bs,
+  const char *name,
+  Error **errp);
 
 /**
  * Register/unregister a buffer for I/O. For example, when the driver is
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index a52b83b619..f50c682308 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -26,6 +26,7 @@
 #include "trace.h"
 #include "block/block_int.h"
 #include "block/blockjob.h"
+#include "qemu/main-loop.h"
 
 struct BdrvDirtyBitmap {
 QemuMutex *mutex;
@@ -455,18 +456,59 @@ void bdrv_release_named_dirty_bitmaps(BlockDriverState 
*bs)
  * not fail.
  * This function doesn't release corresponding BdrvDirtyBitmap.
  */
-int bdrv_remove_persistent_dirty_bitmap(BlockDriverState *bs, const char *name,
-Error **errp)
+static int coroutine_fn
+bdrv_co_remove_persistent_dirty_bitmap(BlockDriverState *bs, const char *name,
+   Error **errp)
 {
-if (bs->drv && bs->drv->bdrv_remove_persistent_dirty_bitmap) {
-return bs->drv->bdrv_remove_persistent_dirty_bitmap(bs, name, errp);
+if (bs->drv && bs->drv->bdrv_co_remove_persistent_dirty_bitmap) {
+return bs->drv->bdrv_co_remove_persistent_dirty_bitmap(bs, name, errp);
 }
 
 return -ENOTSUP;
 }
 
-bool bdrv_can_store_new_dirty_bitmap(BlockDriverState *bs, const char *name,
- uint32_t granularity, Error **errp)
+typedef struct BdrvRemovePersistentDirtyBitmapCo {
+BlockDriverState *bs;
+const char *name;
+Error **errp;
+int ret;
+} BdrvRemovePersistentDirtyBitmapCo;
+
+static void coroutine_fn
+bdrv_co_remove_persistent_dirty_bitmap_entry(void *opaque)
+{
+BdrvRemovePersistentDirtyBitmapCo *s = opaque;
+
+s->ret = bdrv_co_remove_persistent_dirty_bit

[Qemu-block] [PATCH 2/3] block/dirty-bitmap: return int from bdrv_remove_persistent_dirty_bitmap

[Vladimir Sementsov-Ogievskiy]

It's more comfortable to not deal with local_err.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block/qcow2.h|  5 ++---
 include/block/block_int.h|  6 +++---
 include/block/dirty-bitmap.h |  5 ++---
 block/dirty-bitmap.c |  9 +
 block/qcow2-bitmap.c | 20 +++-
 blockdev.c   |  7 +++
 6 files changed, 26 insertions(+), 26 deletions(-)

diff --git a/block/qcow2.h b/block/qcow2.h
index 998bcdaef1..99ee88f802 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -747,9 +747,8 @@ bool qcow2_can_store_new_dirty_bitmap(BlockDriverState *bs,
   const char *name,
   uint32_t granularity,
   Error **errp);
-void qcow2_remove_persistent_dirty_bitmap(BlockDriverState *bs,
-  const char *name,
-  Error **errp);
+int qcow2_remove_persistent_dirty_bitmap(BlockDriverState *bs, const char 
*name,
+ Error **errp);
 
 ssize_t coroutine_fn
 qcow2_co_compress(BlockDriverState *bs, void *dest, size_t dest_size,
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 0422acdf1c..503ac9e3cd 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -556,9 +556,9 @@ struct BlockDriver {
 const char *name,
 uint32_t granularity,
 Error **errp);
-void (*bdrv_remove_persistent_dirty_bitmap)(BlockDriverState *bs,
-const char *name,
-Error **errp);
+int (*bdrv_remove_persistent_dirty_bitmap)(BlockDriverState *bs,
+   const char *name,
+   Error **errp);
 
 /**
  * Register/unregister a buffer for I/O. For example, when the driver is
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 4b4b731b46..07503b03b5 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -37,9 +37,8 @@ int bdrv_dirty_bitmap_check(const BdrvDirtyBitmap *bitmap, 
uint32_t flags,
 Error **errp);
 void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap);
 void bdrv_release_named_dirty_bitmaps(BlockDriverState *bs);
-void bdrv_remove_persistent_dirty_bitmap(BlockDriverState *bs,
- const char *name,
- Error **errp);
+int bdrv_remove_persistent_dirty_bitmap(BlockDriverState *bs, const char *name,
+Error **errp);
 void bdrv_disable_dirty_bitmap(BdrvDirtyBitmap *bitmap);
 void bdrv_enable_dirty_bitmap(BdrvDirtyBitmap *bitmap);
 void bdrv_enable_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap);
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 8f42015db9..a52b83b619 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -455,13 +455,14 @@ void bdrv_release_named_dirty_bitmaps(BlockDriverState 
*bs)
  * not fail.
  * This function doesn't release corresponding BdrvDirtyBitmap.
  */
-void bdrv_remove_persistent_dirty_bitmap(BlockDriverState *bs,
- const char *name,
- Error **errp)
+int bdrv_remove_persistent_dirty_bitmap(BlockDriverState *bs, const char *name,
+Error **errp)
 {
 if (bs->drv && bs->drv->bdrv_remove_persistent_dirty_bitmap) {
-bs->drv->bdrv_remove_persistent_dirty_bitmap(bs, name, errp);
+return bs->drv->bdrv_remove_persistent_dirty_bitmap(bs, name, errp);
 }
+
+return -ENOTSUP;
 }
 
 bool bdrv_can_store_new_dirty_bitmap(BlockDriverState *bs, const char *name,
diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index b2487101ed..1aaedb3b55 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -1404,11 +1404,10 @@ static Qcow2Bitmap *find_bitmap_by_name(Qcow2BitmapList 
*bm_list,
 return NULL;
 }
 
-void qcow2_remove_persistent_dirty_bitmap(BlockDriverState *bs,
-  const char *name,
-  Error **errp)
+int qcow2_remove_persistent_dirty_bitmap(BlockDriverState *bs, const char 
*name,
+ Error **errp)
 {
-int ret;
+int ret = 0;
 BDRVQcow2State *s = bs->opaque;
 Qcow2Bitmap *bm;
 Qcow2BitmapList *bm_list;
@@ -1416,18 +1415,19 @@ void 
qcow2_remove_persistent_dirty_bitmap(BlockDriverState *bs,
 if (s->nb_bitmaps == 0) {
 /* Absence of the bitmap is not an error: see explanation above
  * bdrv_remove_persistent_dirty_bitmap() definition. */
-retur

[Qemu-block] [PATCH 1/3] block: move bdrv_can_store_new_dirty_bitmap to block/dirty-bitmap.c

[Vladimir Sementsov-Ogievskiy]

block/dirty-bitmap.c seems to be more appropriate for it and
bdrv_remove_persistent_dirty_bitmap already in it.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c  | 22 --
 block/dirty-bitmap.c | 22 ++
 2 files changed, 22 insertions(+), 22 deletions(-)

diff --git a/block.c b/block.c
index 5944124845..bea03cfcc9 100644
--- a/block.c
+++ b/block.c
@@ -6555,25 +6555,3 @@ void bdrv_del_child(BlockDriverState *parent_bs, 
BdrvChild *child, Error **errp)
 
 parent_bs->drv->bdrv_del_child(parent_bs, child, errp);
 }
-
-bool bdrv_can_store_new_dirty_bitmap(BlockDriverState *bs, const char *name,
- uint32_t granularity, Error **errp)
-{
-BlockDriver *drv = bs->drv;
-
-if (!drv) {
-error_setg_errno(errp, ENOMEDIUM,
- "Can't store persistent bitmaps to %s",
- bdrv_get_device_or_node_name(bs));
-return false;
-}
-
-if (!drv->bdrv_can_store_new_dirty_bitmap) {
-error_setg_errno(errp, ENOTSUP,
- "Can't store persistent bitmaps to %s",
- bdrv_get_device_or_node_name(bs));
-return false;
-}
-
-return drv->bdrv_can_store_new_dirty_bitmap(bs, name, granularity, errp);
-}
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 134e0c9a0c..8f42015db9 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -464,6 +464,28 @@ void bdrv_remove_persistent_dirty_bitmap(BlockDriverState 
*bs,
 }
 }
 
+bool bdrv_can_store_new_dirty_bitmap(BlockDriverState *bs, const char *name,
+ uint32_t granularity, Error **errp)
+{
+BlockDriver *drv = bs->drv;
+
+if (!drv) {
+error_setg_errno(errp, ENOMEDIUM,
+ "Can't store persistent bitmaps to %s",
+ bdrv_get_device_or_node_name(bs));
+return false;
+}
+
+if (!drv->bdrv_can_store_new_dirty_bitmap) {
+error_setg_errno(errp, ENOTSUP,
+ "Can't store persistent bitmaps to %s",
+ bdrv_get_device_or_node_name(bs));
+return false;
+}
+
+return drv->bdrv_can_store_new_dirty_bitmap(bs, name, granularity, errp);
+}
+
 void bdrv_disable_dirty_bitmap(BdrvDirtyBitmap *bitmap)
 {
 bdrv_dirty_bitmap_lock(bitmap);
-- 
2.18.0

Re: [Qemu-block] [PATCH v2 5/7] curl: Report only ready sockets

[Maxim Levitsky]

On Tue, 2019-09-10 at 14:41 +0200, Max Reitz wrote:
> Instead of reporting all sockets to cURL, only report the one that has
> caused curl_multi_do_locked() to be called.  This lets us get rid of the
> QLIST_FOREACH_SAFE() list, which was actually wrong: SAFE foreaches are
> only safe when the current element is removed in each iteration.  If it
> possible for the list to be concurrently modified, we cannot guarantee
> that only the current element will be removed.  Therefore, we must not
> use QLIST_FOREACH_SAFE() here.
> 
> Fixes: ff5ca1664af85b24a4180d595ea6873fd3deac57
> Cc: qemu-sta...@nongnu.org
> Signed-off-by: Max Reitz 
> ---
>  block/curl.c | 17 ++---
>  1 file changed, 6 insertions(+), 11 deletions(-)
> 
> diff --git a/block/curl.c b/block/curl.c
> index cf2686218d..fd70f1ebc4 100644
> --- a/block/curl.c
> +++ b/block/curl.c
> @@ -392,24 +392,19 @@ static void curl_multi_check_completion(BDRVCURLState 
> *s)
>  }
>  
>  /* Called with s->mutex held.  */
> -static void curl_multi_do_locked(CURLSocket *ready_socket)
> +static void curl_multi_do_locked(CURLSocket *socket)
Here you revert the variable name change you had in previous commit

>  {
> -CURLSocket *socket, *next_socket;
> -CURLState *s = ready_socket->state;
> +BDRVCURLState *s = socket->state->s;
>  int running;
>  int r;
>  
> -if (!s->s->multi) {
> +if (!s->multi) {
>  return;
>  }
>  
> -/* Need to use _SAFE because curl_multi_socket_action() may trigger
> - * curl_sock_cb() which might modify this list */
> -QLIST_FOREACH_SAFE(socket, &s->sockets, next, next_socket) {
> -do {
> -r = curl_multi_socket_action(s->s->multi, socket->fd, 0, 
> &running);
> -} while (r == CURLM_CALL_MULTI_PERFORM);
> -}
> +do {
> +r = curl_multi_socket_action(s->multi, socket->fd, 0, &running);
> +} while (r == CURLM_CALL_MULTI_PERFORM);
>  }
>  
>  static void curl_multi_do(void *arg)

Other than that nitpick,
Reviewed-by: Maxim Levitsky 

Best regards,
Maxim Levitsky

Re: [Qemu-block] [PATCH v2 7/7] curl: Check curl_multi_add_handle()'s return code

[Maxim Levitsky]

On Tue, 2019-09-10 at 14:41 +0200, Max Reitz wrote:
> If we had done that all along, debugging would have been much simpler.
> (Also, I/O errors are better than hangs.)
> 
> Signed-off-by: Max Reitz 
> ---
>  block/curl.c | 8 +++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/block/curl.c b/block/curl.c
> index c343c7ed3d..f86299378e 100644
> --- a/block/curl.c
> +++ b/block/curl.c
> @@ -882,7 +882,13 @@ static void curl_setup_preadv(BlockDriverState *bs, 
> CURLAIOCB *acb)
>  trace_curl_setup_preadv(acb->bytes, start, state->range);
>  curl_easy_setopt(state->curl, CURLOPT_RANGE, state->range);
>  
> -curl_multi_add_handle(s->multi, state->curl);
> +if (curl_multi_add_handle(s->multi, state->curl) != CURLM_OK) {
> +state->acb[0] = NULL;
> +acb->ret = -EIO;
> +
> +curl_clean_state(state);
> +goto out;
> +}
>  
>  /* Tell curl it needs to kick things off */
>  curl_multi_socket_action(s->multi, CURL_SOCKET_TIMEOUT, 0, &running);

Checking the return values is always a very good idea.
I would myself make this patch #1 in the series, since it doesn't
depend on others and it itself a bugfix.
But this is my style, so I don't mind if you leave this as is.

Reviewed-by: Maxim Levitsky 

Best regards,
Maxim Levitsky

Re: [Qemu-block] [PATCH v2 4/7] curl: Pass CURLSocket to curl_multi_do()

[Maxim Levitsky]

On Tue, 2019-09-10 at 14:41 +0200, Max Reitz wrote:
> curl_multi_do_locked() currently marks all sockets as ready.  That is
> not only inefficient, but in fact unsafe (the loop is).  A follow-up
> patch will change that, but to do so, curl_multi_do_locked() needs to
> know exactly which socket is ready; and that is accomplished by this
> patch here.
> 
> Cc: qemu-sta...@nongnu.org
> Signed-off-by: Max Reitz 
> ---
>  block/curl.c | 20 +++-
>  1 file changed, 11 insertions(+), 9 deletions(-)
> 
> diff --git a/block/curl.c b/block/curl.c
> index 5838afef99..cf2686218d 100644
> --- a/block/curl.c
> +++ b/block/curl.c
> @@ -185,15 +185,15 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, 
> int action,
>  switch (action) {
>  case CURL_POLL_IN:
>  aio_set_fd_handler(s->aio_context, fd, false,
> -   curl_multi_do, NULL, NULL, state);
> +   curl_multi_do, NULL, NULL, socket);
>  break;
>  case CURL_POLL_OUT:
>  aio_set_fd_handler(s->aio_context, fd, false,
> -   NULL, curl_multi_do, NULL, state);
> +   NULL, curl_multi_do, NULL, socket);
>  break;
>  case CURL_POLL_INOUT:
>  aio_set_fd_handler(s->aio_context, fd, false,
> -   curl_multi_do, curl_multi_do, NULL, state);
> +   curl_multi_do, curl_multi_do, NULL, socket);
>  break;
>  case CURL_POLL_REMOVE:
>  aio_set_fd_handler(s->aio_context, fd, false,
> @@ -392,9 +392,10 @@ static void curl_multi_check_completion(BDRVCURLState *s)
>  }
>  
>  /* Called with s->mutex held.  */
> -static void curl_multi_do_locked(CURLState *s)
> +static void curl_multi_do_locked(CURLSocket *ready_socket)
>  {
>  CURLSocket *socket, *next_socket;
> +CURLState *s = ready_socket->state;
>  int running;
>  int r;
>  
> @@ -413,12 +414,13 @@ static void curl_multi_do_locked(CURLState *s)
>  
>  static void curl_multi_do(void *arg)
>  {
> -CURLState *s = (CURLState *)arg;
> +CURLSocket *socket = arg;
> +BDRVCURLState *s = socket->state->s;
>  
> -qemu_mutex_lock(&s->s->mutex);
> -curl_multi_do_locked(s);
> -curl_multi_check_completion(s->s);
> -qemu_mutex_unlock(&s->s->mutex);
> +qemu_mutex_lock(&s->mutex);
> +curl_multi_do_locked(socket);
> +curl_multi_check_completion(s);
> +qemu_mutex_unlock(&s->mutex);
>  }
>  
>  static void curl_multi_timeout_do(void *arg)

Reviewed-by: Maxim Levitsky 
Best regards,
Maxim Levitsky

Re: [Qemu-block] [PATCH v2 6/7] curl: Handle success in multi_check_completion

[Maxim Levitsky]

On Tue, 2019-09-10 at 14:41 +0200, Max Reitz wrote:
> Background: As of cURL 7.59.0, it verifies that several functions are
> not called from within a callback.  Among these functions is
> curl_multi_add_handle().
> 
> curl_read_cb() is a callback from cURL and not a coroutine.  Waking up
> acb->co will lead to entering it then and there, which means the current
> request will settle and the caller (if it runs in the same coroutine)
> may then issue the next request.  In such a case, we will enter
> curl_setup_preadv() effectively from within curl_read_cb().
> 
> Calling curl_multi_add_handle() will then fail and the new request will
> not be processed.
> 
> Fix this by not letting curl_read_cb() wake up acb->co.  Instead, leave
> the whole business of settling the AIOCB objects to
> curl_multi_check_completion() (which is called from our timer callback
> and our FD handler, so not from any cURL callbacks).
> 
> Reported-by: Natalie Gavrielov 
> Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1740193
> Cc: qemu-sta...@nongnu.org
> Signed-off-by: Max Reitz 
> ---
>  block/curl.c | 69 ++--
>  1 file changed, 29 insertions(+), 40 deletions(-)
> 
> diff --git a/block/curl.c b/block/curl.c
> index fd70f1ebc4..c343c7ed3d 100644
> --- a/block/curl.c
> +++ b/block/curl.c
> @@ -229,7 +229,6 @@ static size_t curl_read_cb(void *ptr, size_t size, size_t 
> nmemb, void *opaque)
>  {
>  CURLState *s = ((CURLState*)opaque);
>  size_t realsize = size * nmemb;
> -int i;
>  
>  trace_curl_read_cb(realsize);
>  
> @@ -245,32 +244,6 @@ static size_t curl_read_cb(void *ptr, size_t size, 
> size_t nmemb, void *opaque)
>  memcpy(s->orig_buf + s->buf_off, ptr, realsize);
>  s->buf_off += realsize;
>  
> -for(i=0; i -CURLAIOCB *acb = s->acb[i];
> -
> -if (!acb)
> -continue;
> -
> -if ((s->buf_off >= acb->end)) {
> -size_t request_length = acb->bytes;
> -
> -qemu_iovec_from_buf(acb->qiov, 0, s->orig_buf + acb->start,
> -acb->end - acb->start);
> -
> -if (acb->end - acb->start < request_length) {
> -size_t offset = acb->end - acb->start;
> -qemu_iovec_memset(acb->qiov, offset, 0,
> -  request_length - offset);
> -}
> -
> -acb->ret = 0;
> -s->acb[i] = NULL;
> -qemu_mutex_unlock(&s->s->mutex);
> -aio_co_wake(acb->co);
> -qemu_mutex_lock(&s->s->mutex);
> -}
> -}
> -
>  read_end:
>  /* curl will error out if we do not return this value */
>  return size * nmemb;
> @@ -351,13 +324,14 @@ static void curl_multi_check_completion(BDRVCURLState 
> *s)
>  break;
>  
>  if (msg->msg == CURLMSG_DONE) {
> +int i;
>  CURLState *state = NULL;
> +bool error = msg->data.result != CURLE_OK;
> +
>  curl_easy_getinfo(msg->easy_handle, CURLINFO_PRIVATE,
>(char **)&state);
>  
> -/* ACBs for successful messages get completed in curl_read_cb */
> -if (msg->data.result != CURLE_OK) {
> -int i;
> +if (error) {
>  static int errcount = 100;
>  
>  /* Don't lose the original error message from curl, since
> @@ -369,20 +343,35 @@ static void curl_multi_check_completion(BDRVCURLState 
> *s)
>  error_report("curl: further errors suppressed");
>  }
>  }
> +}
>  
> -for (i = 0; i < CURL_NUM_ACB; i++) {
> -CURLAIOCB *acb = state->acb[i];
> +for (i = 0; i < CURL_NUM_ACB; i++) {
> +CURLAIOCB *acb = state->acb[i];
>  
> -if (acb == NULL) {
> -continue;
> -}
> +if (acb == NULL) {
> +continue;
> +}
> +
> +if (!error) {
> +/* Assert that we have read all data */
> +assert(state->buf_off >= acb->end);
> +
> +qemu_iovec_from_buf(acb->qiov, 0,
> +state->orig_buf + acb->start,
> +acb->end - acb->start);
>  
> -acb->ret = -EIO;
> -state->acb[i] = NULL;
> -qemu_mutex_unlock(&s->mutex);
> -aio_co_wake(acb->co);
> -qemu_mutex_lock(&s->mutex);
> +if (acb->end - acb->start < acb->bytes) {
> +size_t offset = acb->end - acb->start;
> +qemu_iovec_memset(acb->qiov, offset, 0,
> +  acb->bytes - offset);
> +}
Original code was memsetting the tail of

Re: [Qemu-block] [PATCH v2 3/7] curl: Check completion in curl_multi_do()

[Maxim Levitsky]

On Tue, 2019-09-10 at 14:41 +0200, Max Reitz wrote:
> While it is more likely that transfers complete after some file
> descriptor has data ready to read, we probably should not rely on it.
> Better be safe than sorry and call curl_multi_check_completion() in
> curl_multi_do(), too, just like it is done in curl_multi_read().
> 
> With this change, curl_multi_do() and curl_multi_read() are actually the
> same, so drop curl_multi_read() and use curl_multi_do() as the sole FD
> handler.

I understand the reasoning, but I still a bit worry that this
could paper over some bug/race in the future.
If curl asks us only to deal with write, that would mean
that it doesn't expect any data to be received.

Do you by a chance have an example, of this patch
affecting the code? Maybe when a unexpected error reply
is received from the server?

I don't really know the CURL library, so I probably missed
something important.

Other than that,
Reviewed-by: Maxim Levitsky 


Best regards,
Maxim Levitsky


> 
> Signed-off-by: Max Reitz 
> ---
>  block/curl.c | 14 ++
>  1 file changed, 2 insertions(+), 12 deletions(-)
> 
> diff --git a/block/curl.c b/block/curl.c
> index 95d7b77dc0..5838afef99 100644
> --- a/block/curl.c
> +++ b/block/curl.c
> @@ -139,7 +139,6 @@ typedef struct BDRVCURLState {
>  
>  static void curl_clean_state(CURLState *s);
>  static void curl_multi_do(void *arg);
> -static void curl_multi_read(void *arg);
>  
>  #ifdef NEED_CURL_TIMER_CALLBACK
>  /* Called from curl_multi_do_locked, with s->mutex held.  */
> @@ -186,7 +185,7 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int 
> action,
>  switch (action) {
>  case CURL_POLL_IN:
>  aio_set_fd_handler(s->aio_context, fd, false,
> -   curl_multi_read, NULL, NULL, state);
> +   curl_multi_do, NULL, NULL, state);
>  break;
>  case CURL_POLL_OUT:
>  aio_set_fd_handler(s->aio_context, fd, false,
> @@ -194,7 +193,7 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int 
> action,
>  break;
>  case CURL_POLL_INOUT:
>  aio_set_fd_handler(s->aio_context, fd, false,
> -   curl_multi_read, curl_multi_do, NULL, state);
> +   curl_multi_do, curl_multi_do, NULL, state);
>  break;
>  case CURL_POLL_REMOVE:
>  aio_set_fd_handler(s->aio_context, fd, false,
> @@ -416,15 +415,6 @@ static void curl_multi_do(void *arg)
>  {
>  CURLState *s = (CURLState *)arg;
>  
> -qemu_mutex_lock(&s->s->mutex);
> -curl_multi_do_locked(s);
> -qemu_mutex_unlock(&s->s->mutex);
> -}
> -
> -static void curl_multi_read(void *arg)
> -{
> -CURLState *s = (CURLState *)arg;
> -
>  qemu_mutex_lock(&s->s->mutex);
>  curl_multi_do_locked(s);
>  curl_multi_check_completion(s->s);

Re: [Qemu-block] [PATCH v2 2/7] curl: Keep *socket until the end of curl_sock_cb()

[Maxim Levitsky]

On Tue, 2019-09-10 at 14:41 +0200, Max Reitz wrote:
> This does not really change anything, but it makes the code a bit easier
> to follow once we use @socket as the opaque pointer for
> aio_set_fd_handler().
> 
> Cc: qemu-sta...@nongnu.org
> Signed-off-by: Max Reitz 
> ---
>  block/curl.c | 10 +-
>  1 file changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/block/curl.c b/block/curl.c
> index 92dc2f630e..95d7b77dc0 100644
> --- a/block/curl.c
> +++ b/block/curl.c
> @@ -172,10 +172,6 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, 
> int action,
>  
>  QLIST_FOREACH(socket, &state->sockets, next) {
>  if (socket->fd == fd) {
> -if (action == CURL_POLL_REMOVE) {
> -QLIST_REMOVE(socket, next);
> -g_free(socket);
> -}
>  break;
>  }
>  }
> @@ -185,7 +181,6 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int 
> action,
>  socket->state = state;
>  QLIST_INSERT_HEAD(&state->sockets, socket, next);
>  }
> -socket = NULL;
>  
>  trace_curl_sock_cb(action, (int)fd);
>  switch (action) {
> @@ -207,6 +202,11 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, 
> int action,
>  break;
>  }
>  
> +if (action == CURL_POLL_REMOVE) {
> +QLIST_REMOVE(socket, next);
> +g_free(socket);
> +}
> +
>  return 0;
>  }
>  
Reviewed-by: Maxim Levitsky 

Best regards,
Maxim Levitsky

Re: [Qemu-block] [PATCH v2 1/7] curl: Keep pointer to the CURLState in CURLSocket

[Maxim Levitsky]

On Tue, 2019-09-10 at 14:41 +0200, Max Reitz wrote:
> A follow-up patch will make curl_multi_do() and curl_multi_read() take a
> CURLSocket instead of the CURLState.  They still need the latter,
> though, so add a pointer to it to the former.
> 
> Cc: qemu-sta...@nongnu.org
> Signed-off-by: Max Reitz 
> Reviewed-by: John Snow 
> ---
>  block/curl.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/block/curl.c b/block/curl.c
> index d4c8e94f3e..92dc2f630e 100644
> --- a/block/curl.c
> +++ b/block/curl.c
> @@ -80,6 +80,7 @@ static CURLMcode __curl_multi_socket_action(CURLM 
> *multi_handle,
>  #define CURL_BLOCK_OPT_TIMEOUT_DEFAULT 5
>  
>  struct BDRVCURLState;
> +struct CURLState;
>  
>  static bool libcurl_initialized;
>  
> @@ -97,6 +98,7 @@ typedef struct CURLAIOCB {
>  
>  typedef struct CURLSocket {
>  int fd;
> +struct CURLState *state;
>  QLIST_ENTRY(CURLSocket) next;
>  } CURLSocket;
>  
> @@ -180,6 +182,7 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int 
> action,
>  if (!socket) {
>  socket = g_new0(CURLSocket, 1);
>  socket->fd = fd;
> +socket->state = state;
>  QLIST_INSERT_HEAD(&state->sockets, socket, next);
>  }
>  socket = NULL;

Reviewed-by: Maxim Levitsky 

Best regards,
Maxim Levitsky

[Qemu-block] [PATCH V2 1/2] block/nfs: tear down aio before nfs_close

[Peter Lieven]

nfs_close is a sync call from libnfs and has its own event
handler polling on the nfs FD. Avoid that both QEMU and libnfs
are intefering here.

CC: qemu-sta...@nongnu.org
Signed-off-by: Peter Lieven 
---
 block/nfs.c | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/block/nfs.c b/block/nfs.c
index 0ec50953e4..2c98508275 100644
--- a/block/nfs.c
+++ b/block/nfs.c
@@ -390,12 +390,14 @@ static void nfs_attach_aio_context(BlockDriverState *bs,
 static void nfs_client_close(NFSClient *client)
 {
 if (client->context) {
+qemu_mutex_lock(&client->mutex);
+aio_set_fd_handler(client->aio_context, nfs_get_fd(client->context),
+   false, NULL, NULL, NULL, NULL);
+qemu_mutex_unlock(&client->mutex);
 if (client->fh) {
 nfs_close(client->context, client->fh);
 client->fh = NULL;
 }
-aio_set_fd_handler(client->aio_context, nfs_get_fd(client->context),
-   false, NULL, NULL, NULL, NULL);
 nfs_destroy_context(client->context);
 client->context = NULL;
 }
-- 
2.17.1

[Qemu-block] [PATCH V2 0/2] add support for nfs_umount

[Peter Lieven]

add support for NFSv3 umount call. V2 adds a patch that fixes
the order of the aio teardown. The addition of the NFS umount
call unmasked that bug.

Peter Lieven (2):
  block/nfs: tear down aio before nfs_close
  block/nfs: add support for nfs_umount

 block/nfs.c | 9 +++--
 1 file changed, 7 insertions(+), 2 deletions(-)

-- 
2.17.1

[Qemu-block] [PATCH V2 2/2] block/nfs: add support for nfs_umount

[Peter Lieven]

libnfs recently added support for unmounting. Add support
in Qemu too.

Signed-off-by: Peter Lieven 
---
 block/nfs.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/block/nfs.c b/block/nfs.c
index 2c98508275..f39acfdb28 100644
--- a/block/nfs.c
+++ b/block/nfs.c
@@ -398,6 +398,9 @@ static void nfs_client_close(NFSClient *client)
 nfs_close(client->context, client->fh);
 client->fh = NULL;
 }
+#ifdef LIBNFS_FEATURE_UMOUNT
+nfs_umount(client->context);
+#endif
 nfs_destroy_context(client->context);
 client->context = NULL;
 }
-- 
2.17.1

Re: [Qemu-block] [Qemu-devel] [PATCH v10 0/9] hw/m68k: add Apple Machintosh Quadra 800 machine

[no-reply]

Patchew URL: https://patchew.org/QEMU/20190910113323.17324-1-laur...@vivier.eu/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Subject: [Qemu-devel] [PATCH v10 0/9] hw/m68k: add Apple Machintosh Quadra 800 
machine
Message-id: 20190910113323.17324-1-laur...@vivier.eu
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Switched to a new branch 'test'
e61ee52 hw/m68k: define Macintosh Quadra 800
175ab91 hw/m68k: add a dummy SWIM floppy controller
5866720 hw/m68k: add Nubus support for macfb video card
4f30fb5 hw/m68k: add Nubus support
fa6e847 hw/m68k: add macfb video card
363af58 hw/m68k: implement ADB bus support for via
3c0580d hw/m68k: add via support
06648a2 dp8393x: manage big endian bus
0ba9d85 esp: add pseudo-DMA as used by Macintosh

=== OUTPUT BEGIN ===
1/9 Checking commit 0ba9d85aee70 (esp: add pseudo-DMA as used by Macintosh)
2/9 Checking commit 06648a2f9d42 (dp8393x: manage big endian bus)
3/9 Checking commit 3c0580dfb93d (hw/m68k: add via support)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#77: 
new file mode 100644

ERROR: space prohibited after that '&&' (ctx:WxW)
#426: FILE: hw/misc/mac_via.c:345:
+if (!(v1s->last_b & VIA1B_vRTCClk) && (s->b & VIA1B_vRTCClk)) {
^

total: 1 errors, 1 warnings, 867 lines checked

Patch 3/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

4/9 Checking commit 363af58aecd1 (hw/m68k: implement ADB bus support for via)
5/9 Checking commit fa6e847440b3 (hw/m68k: add macfb video card)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#68: 
new file mode 100644

total: 0 errors, 1 warnings, 518 lines checked

Patch 5/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
6/9 Checking commit 4f30fb5f428a (hw/m68k: add Nubus support)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#62: 
new file mode 100644

total: 0 errors, 1 warnings, 532 lines checked

Patch 6/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
7/9 Checking commit 5866720d4275 (hw/m68k: add Nubus support for macfb video 
card)
8/9 Checking commit 175ab9145112 (hw/m68k: add a dummy SWIM floppy controller)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#53: 
new file mode 100644

total: 0 errors, 1 warnings, 591 lines checked

Patch 8/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
9/9 Checking commit e61ee524314d (hw/m68k: define Macintosh Quadra 800)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#70: 
new file mode 100644

total: 0 errors, 1 warnings, 518 lines checked

Patch 9/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20190910113323.17324-1-laur...@vivier.eu/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

[Qemu-block] [PATCH V4] block/vhdx: add check for truncated image files

[Peter Lieven]

qemu is currently not able to detect truncated vhdx image files.
Add a basic check if all allocated blocks are reachable at open and
report all errors during bdrv_co_check.

Signed-off-by: Peter Lieven 
---
V4: - allow partial last blocks [Kevin]
- report offsets in error messages [Kevin]
- check for start and end offset after eof

V3: - check for bdrv_getlength failure [Kevin]
- use uint32_t for i [Kevin]
- check for BAT entry overflow [Kevin]
- break on !errcnt in second check

V2: - add error reporting [Kevin]
- use bdrv_getlength instead of bdrv_get_allocated_file_size [Kevin]
- factor out BAT entry check and add error reporting for region
  overlaps
- already check on vhdx_open

 block/vhdx.c | 120 +++
 1 file changed, 103 insertions(+), 17 deletions(-)

diff --git a/block/vhdx.c b/block/vhdx.c
index 6a09d0a55c..371f226286 100644
--- a/block/vhdx.c
+++ b/block/vhdx.c
@@ -24,6 +24,7 @@
 #include "qemu/option.h"
 #include "qemu/crc32c.h"
 #include "qemu/bswap.h"
+#include "qemu/error-report.h"
 #include "vhdx.h"
 #include "migration/blocker.h"
 #include "qemu/uuid.h"
@@ -235,6 +236,9 @@ static int vhdx_region_check(BDRVVHDXState *s, uint64_t 
start, uint64_t length)
 end = start + length;
 QLIST_FOREACH(r, &s->regions, entries) {
 if (!((start >= r->end) || (end <= r->start))) {
+error_report("VHDX region %" PRIu64 "-%" PRIu64 " overlaps with "
+ "region %" PRIu64 "-%." PRIu64, start, end, r->start,
+ r->end);
 ret = -EINVAL;
 goto exit;
 }
@@ -877,6 +881,95 @@ static void vhdx_calc_bat_entries(BDRVVHDXState *s)
 
 }
 
+static int vhdx_check_bat_entries(BlockDriverState *bs, int *errcnt)
+{
+BDRVVHDXState *s = bs->opaque;
+int64_t image_file_size = bdrv_getlength(bs->file->bs);
+uint64_t payblocks = s->chunk_ratio;
+uint64_t i;
+int ret = 0;
+
+if (image_file_size < 0) {
+error_report("Could not determinate VHDX image file size.");
+return image_file_size;
+}
+
+for (i = 0; i < s->bat_entries; i++) {
+if ((s->bat[i] & VHDX_BAT_STATE_BIT_MASK) ==
+PAYLOAD_BLOCK_FULLY_PRESENT) {
+uint64_t offset = s->bat[i] & VHDX_BAT_FILE_OFF_MASK;
+/*
+ * Allow that the last block exists only partially. The VHDX spec
+ * states that the image file can only grow in blocksize 
increments,
+ * but QEMU created images with partial last blocks in the past.
+ */
+uint32_t block_length = MIN(s->block_size,
+bs->total_sectors * BDRV_SECTOR_SIZE - i * s->block_size);
+/*
+ * Check for BAT entry overflow.
+ */
+if (offset > INT64_MAX - s->block_size) {
+error_report("VHDX BAT entry %" PRIu64 " offset overflow.", i);
+ret = -EINVAL;
+if (!errcnt) {
+break;
+}
+(*errcnt)++;
+}
+/*
+ * Check if fully allocated BAT entries do not reside after
+ * end of the image file.
+ */
+if (offset >= image_file_size) {
+error_report("VHDX BAT entry %" PRIu64 " start offset %" PRIu64
+ " points after end of file (%" PRIi64 "). Image"
+ " has probably been truncated.",
+ i, offset, image_file_size);
+ret = -EINVAL;
+if (!errcnt) {
+break;
+}
+(*errcnt)++;
+} else if (offset + block_length > image_file_size) {
+error_report("VHDX BAT entry %" PRIu64 " end offset %" PRIu64
+ " points after end of file (%" PRIi64 "). Image"
+ " has probably been truncated.",
+ i, offset + block_length - 1, image_file_size);
+ret = -EINVAL;
+if (!errcnt) {
+break;
+}
+(*errcnt)++;
+}
+
+/*
+ * verify populated BAT field file offsets against
+ * region table and log entries
+ */
+if (payblocks--) {
+/* payload bat entries */
+int ret2;
+ret2 = vhdx_region_check(s, offset, s->block_size);
+if (ret2 < 0) {
+ret = -EINVAL;
+if (!errcnt) {
+break;
+}
+(*errcnt)++;
+}
+} else {
+payblocks = s->chunk_ratio;
+/*
+ * Once differencing files are supported, verify sector bitmap
+ * blocks here
+ 

Re: [Qemu-block] [PATCH v9 3/9] block: add empty account cookie type

[Anton Nefedov]

On 9/9/2019 5:54 PM, Alberto Garcia wrote:
> On Fri 06 Sep 2019 06:01:14 PM CEST, Anton Nefedov wrote:
>> This adds some protection from accounting uninitialized cookie.
>> That is, block_acct_failed/done without previous block_acct_start;
>> in that case, cookie probably holds values from previous operation.
>>
>> (Note: it might also be uninitialized holding garbage value and there
>> is still "< BLOCK_MAX_IOTYPE" assertion for that.  So
>> block_acct_failed/done without previous block_acct_start should be
>> used with caution.)
>>
>> Currently this is particularly useful in ide code where it's hard to
>> keep track whether the request started accounting or not. For example,
>> trim requests do the accounting separately.
> 
> Sorry if I'm understanding it wrong, but it sounds like you know that
> there's a bug in the ide code (where you call block_acct_done() without
> having it initialized it first), and the purpose of the this patch is to
> hide the bug ?
> 

hi,

not really; in the existing code, I can't see block_acct_done() without
block_acct_start(), but there might be double-accounting though;
e.g. ide_atapi_cmd_read_dma_cb(): it can account the same operation
twice like
   ide_handle_rw_error();
   goto eot;
   block_acct_failed();

The patch should solve it.

The commit message is misleading, sorry. I'll change to:

 > Each block_acct_done/failed call is designed to correspond to a
 > previous block_acct_start call, which initializes the stats cookie.
 > However sometimes it is not the case, e.g. some error paths might
 > report the same cookie twice because it is hard to accurately track if
 > the cookie was reported yet or not.

 > This patch cleans the cookie after report.
 > (Note: block_acct_failed/done without a previous block_acct_start at
 > all should be avoided. Uninitialized cookie might hold a garbage value
 > and there is still "< BLOCK_MAX_IOTYPE" assertion for that)

 > It will be particularly useful in ide code where it's hard to
 > keep track whether the request done its accounting or not: in the
 > following patch of the series, trim requests will do the accounting
 > separately.

/Anton

Re: [Qemu-block] [Qemu-devel] [PATCH 2/2] block/nvme: add support for discard

[Maxim Levitsky]

On Tue, 2019-09-10 at 16:49 +0200, Paolo Bonzini wrote:
> On 09/09/19 19:03, John Snow wrote:
> > 
> > 
> > On 9/9/19 5:25 AM, Max Reitz wrote:
> > > On 05.09.19 19:27, John Snow wrote:
> > > 
> > > [...]
> > > 
> > > > You also probably require review (or at least an ACK) from Keith Busch
> > > > who maintains this file.
> > > 
> > > Keith actually maintains the NVMe guest device; technically, Fam is the
> > > NVMe block driver maintainer.
> > 
> > W h o o p s. Thanks for correcting me.
> > 
> > Well, if it's Fam -- he seems a little busier lately -- it's probably
> > not so crucial to gate on his approval. I thought it'd be nice to at
> > least get an ACK from someone who has used this module before, because I
> > haven't -- I was just giving some style review to help push it along.
> > 
> > (On that note, if you felt like my style review was wrong or isn't worth
> > doing -- it is always perfectly fair to just say so, along with some
> > reason as to why you won't -- that way patches won't rot on the list
> > when people may have gotten the impression that a V2 is warranted.)
Absolutely not, your review was fine! I just was/is a bit lazy to send next 
version of the patches
before I get some kind of indication if anything else is needed for this to be 
merged,
since the module doesn't have currently an active maintainer.


> 
> Looks good to me with the changes you pointed out (especially res30;
> leaving out the unused macros is not so important).

All right, I'll send an updated version of those two patches soon.

Best regards,
Maxim Levitsky

Re: [Qemu-block] [PATCH v6 22/42] block: Fix bdrv_get_allocated_file_size's fallback

[Kevin Wolf]

Am 09.08.2019 um 18:13 hat Max Reitz geschrieben:
> If the driver does not implement bdrv_get_allocated_file_size(), we
> should fall back to cumulating the allocated size of all non-COW
> children instead of just bs->file.
> 
> Suggested-by: Vladimir Sementsov-Ogievskiy 
> Signed-off-by: Max Reitz 

This smells like an overgeneralisation, but if we want to count all vmdk
extents, the qcow2 external data file, etc. it's an improvement anyway.
A driver that has a child that should not be counted must just remember
to implement the callback.

Let me think of an example... How about quorum, for a change? :-)
Or the second blkverify child.

Or eventually the block job filter nodes.

Ehm... Maybe I should just take back what I said first. It almost feels
like it would be better if qcow2 and vmdk explicitly used a handler that
counts all children (could still be a generic one in block.c) rather
than having to remember to disable the functionality everywhere where we
don't want to have it.

And please adjust the comment for bdrv_get_allocated_file_size(), it
only talks about a single file as if trees didn't exist. Actually, it
doesn't even seem so easy to define. Maybe primary node + storage nodes?
Then vmdk needs to expose its extents as storage nodes (plural!), but
in the long run that might be needed anyway.

Kevin

Re: [Qemu-block] [PATCH v6 23/42] blockdev: Use CAF in external_snapshot_prepare()

[Kevin Wolf]

Am 09.08.2019 um 18:13 hat Max Reitz geschrieben:
> This allows us to differentiate between filters and nodes with COW
> backing files: Filters cannot be used as overlays at all (for this
> function).
> 
> Signed-off-by: Max Reitz 
> Reviewed-by: Vladimir Sementsov-Ogievskiy 

Didn't we occasionally advertise blockdev-snapshot as the way to insert
filters on top at runtime? Though it seems it has always only worked for
filters that use bs->backing, among which I think there aren't any
user-creatable ones. So we're probably good.

Kevin

>  blockdev.c | 7 ++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/blockdev.c b/blockdev.c
> index 29c6c6044a..c540802127 100644
> --- a/blockdev.c
> +++ b/blockdev.c
> @@ -1664,7 +1664,12 @@ static void external_snapshot_prepare(BlkActionState 
> *common,
>  goto out;
>  }
>  
> -if (state->new_bs->backing != NULL) {
> +if (state->new_bs->drv->is_filter) {
> +error_setg(errp, "Filters cannot be used as overlays");
> +goto out;
> +}
> +
> +if (bdrv_filtered_cow_child(state->new_bs)) {
>  error_setg(errp, "The overlay already has a backing image");
>  goto out;
>  }
> -- 
> 2.21.0
> 

Re: [Qemu-block] [Qemu-devel] [PATCH 2/2] block/nvme: add support for discard

[Paolo Bonzini]

On 09/09/19 19:03, John Snow wrote:
> 
> 
> On 9/9/19 5:25 AM, Max Reitz wrote:
>> On 05.09.19 19:27, John Snow wrote:
>>
>> [...]
>>
>>> You also probably require review (or at least an ACK) from Keith Busch
>>> who maintains this file.
>>
>> Keith actually maintains the NVMe guest device; technically, Fam is the
>> NVMe block driver maintainer.
> 
> W h o o p s. Thanks for correcting me.
> 
> Well, if it's Fam -- he seems a little busier lately -- it's probably
> not so crucial to gate on his approval. I thought it'd be nice to at
> least get an ACK from someone who has used this module before, because I
> haven't -- I was just giving some style review to help push it along.
> 
> (On that note, if you felt like my style review was wrong or isn't worth
> doing -- it is always perfectly fair to just say so, along with some
> reason as to why you won't -- that way patches won't rot on the list
> when people may have gotten the impression that a V2 is warranted.)

Looks good to me with the changes you pointed out (especially res30;
leaving out the unused macros is not so important).

Paolo

Re: [Qemu-block] [Qemu-devel] [PATCH] ahci: enable pci bus master MemoryRegion before loading ahci engines

[John Snow]

On 9/10/19 3:20 AM, Andy wrote:
> Hi John,
> 
> Sorry I'm re-sending this mail due to format issue in the last one.
> 

No problem at all. Thank you for the detailed logs, it's really helpful.

> This issue can only be reproduced on Windows 10.
> I've observed and compared the behavior of Windows 10 and Windows 7.
> It seems Windows 7 wouldn't disable the PCI_COMMAND_MASTER flag
> when disabling ahci devices. That's why this issue won't happen on Win7.
> 
> Here's the trace log on both guest OS, on disabling and re-engaging SATA
> disk:
> 
> Windows 10, disabling SATA disk:
> 
> ahci_port_write ahci(0x7f6da4eb1400)[0]: port write [reg:PxCI] @ 0x38:
> 0x0004
> handle_cmd_fis_dump ahci(0x7f6da4eb1400)[0]: FIS:
> 0x00: 27 80 ea 00 00 00 00 a0 00 00 00 00 00 00 00 00
> 0x10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 
> ahci_cmd_done ahci(0x7f6da4eb1400)[0]: cmd done
> ahci_port_write ahci(0x7f6da4eb1400)[0]: port write [reg:PxIS] @ 0x10:
> 0x0001
> ahci_mem_write_host ahci(0x7f6da4eb1400) write4 [reg:IS] @ 0x8:
> 0x0001
> ahci_port_write ahci(0x7f6da4eb1400)[0]: port write [reg:PxCI] @ 0x38:
> 0x0008
> handle_cmd_fis_dump ahci(0x7f6da4eb1400)[0]: FIS:
> 0x00: 27 80 e0 00 00 00 00 a0 00 00 00 00 00 00 00 00
> 0x10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 
> ahci_cmd_done ahci(0x7f6da4eb1400)[0]: cmd done
> ahci_port_write ahci(0x7f6da4eb1400)[0]: port write [reg:PxIS] @ 0x10:
> 0x0001
> ahci_mem_write_host ahci(0x7f6da4eb1400) write4 [reg:IS] @ 0x8:
> 0x0001
> ahci_mem_write_host ahci(0x7f6da4eb1400) write4 [reg:GHC] @ 0x4:
> 0x8000
> pci_cfg_write ich9-ahci 26:0 @0x4 <- 0x507
> pci_cfg_write ich9-ahci 26:0 @0x82 <- 0x80
> pci_cfg_write ich9-ahci 26:0 @0x4 <- 0x500
> pci_update_mappings_del d=0x7f6da4eb0b20 00:1a.0 4,0xc0a0+0x20
> pci_update_mappings_del d=0x7f6da4eb0b20 00:1a.0 5,0xfebf1000+0x1000

! It doesn't look like windows changes the PxCMD registers at all, it's
trying to clear AHCI Enable (AE) instead.

GHC's bits are:

0: HBA Reset (HR), RW1
1: Interrupt Enable (IE), RW
2: MSI Revert to Single Message (MSRM), Read Only
31: AHCI Enable (AE)


It looks like it's trying to disable the AHCI device in this manner, but
I'm not sure that makes sense; it's not really a spin-down or sleep command.

Our implementation for AHCI sets HOST_CAP_AHCI, which is CAP.SAM in the
spec -- "Supports AHCI-mode Only" -- which means that AHCI Enable is
supposed to be RO and set to '1'. I'm not sure what Windows is trying to
do here -- it might assume that this is a true-blue ICH9 and it can
switch off the AHCI engine with a quirk.

It's not immediately clear to me what QEMU should do when it sees this
behavior.

> ---
> 
> Windows 10, re-engaging SATA disk:
> 
> pci_cfg_write ich9-ahci 26:0 @0x14 <- 0x0
> pci_cfg_write ich9-ahci 26:0 @0x18 <- 0x0
> pci_cfg_write ich9-ahci 26:0 @0x1c <- 0x0
> pci_cfg_write ich9-ahci 26:0 @0x20 <- 0xc0a0
> pci_cfg_write ich9-ahci 26:0 @0x24 <- 0xfebf1000
> pci_cfg_write ich9-ahci 26:0 @0x30 <- 0x0
> pci_cfg_write ich9-ahci 26:0 @0x3c <- 0x0
> pci_cfg_write ich9-ahci 26:0 @0xc <- 0x0
> pci_cfg_write ich9-ahci 26:0 @0xd <- 0x0
> pci_cfg_write ich9-ahci 26:0 @0x4 <- 0x500
> pci_cfg_write ich9-ahci 26:0 @0x4 <- 0x507
> pci_update_mappings_add d=0x7f6da4eb0b20 00:1a.0 4,0xc0a0+0x20
> pci_update_mappings_add d=0x7f6da4eb0b20 00:1a.0 5,0xfebf1000+0x1000
> pci_cfg_write ich9-ahci 26:0 @0x6 <- 0xf900
> pci_cfg_write ich9-ahci 26:0 @0x4 <- 0x507
> pci_cfg_write ich9-ahci 26:0 @0x82 <- 0x80
> pci_cfg_write ich9-ahci 26:0 @0x84 <- 0xfee0100c
> pci_cfg_write ich9-ahci 26:0 @0x88 <- 0x0
> pci_cfg_write ich9-ahci 26:0 @0x8c <- 0x49a1
> pci_cfg_write ich9-ahci 26:0 @0x82 <- 0x81
> ahci_mem_write_host ahci(0x7f6da4eb1400) write4 [reg:GHC] @ 0x4:
> 0x8002

Not sure what that 8 is. Some vendor-specific thing that the real ICH9
might implement.

> ahci_port_write ahci(0x7f6da4eb1400)[1]: port write [reg:PxCLB] @ 0x0:
> 0x7fe9f000
> ahci_port_write ahci(0x7f6da4eb1400)[1]: port write [reg:PxCLBU] @ 0x4:
> 0x
> ahci_port_write ahci(0x7f6da4eb1400)[1]: port write [reg:PxFB] @ 0x8:
> 0x7fe9f400
> ahci_port_write ahci(0x7f6da4eb1400)[1]: port write [reg:PxFBU] @ 0xc:
> 0x
> ahci_port_write ahci(0x7

Re: [Qemu-block] [PATCH v2 1/3] block/qcow2: refactoring of threaded encryption code

[Maxim Levitsky]

On Tue, 2019-09-10 at 14:17 +, Vladimir Sementsov-Ogievskiy wrote:
> 10.09.2019 15:31, Maxim Levitsky wrote:
> > On Sat, 2019-09-07 at 19:08 +, Vladimir Sementsov-Ogievskiy wrote:
> > > 06.09.2019 22:57, Maxim Levitsky wrote:
> > > > This commit tries to clarify few function arguments,
> > > > and add comments describing the encrypt/decrypt interface
> > > > 
> > > > Signed-off-by: Maxim Levitsky 
> > > > ---
> > > >block/qcow2-cluster.c | 10 +++
> > > >block/qcow2-threads.c | 61 
> > > > ++-
> > > >2 files changed, 53 insertions(+), 18 deletions(-)
> > > > 
> > > > diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
> > > > index f09cc992af..1989b423da 100644
> > > > --- a/block/qcow2-cluster.c
> > > > +++ b/block/qcow2-cluster.c
> > > > @@ -463,8 +463,8 @@ static int coroutine_fn 
> > > > do_perform_cow_read(BlockDriverState *bs,
> > > >}
> > > >
> > > >static bool coroutine_fn do_perform_cow_encrypt(BlockDriverState *bs,
> > > > -uint64_t 
> > > > src_cluster_offset,
> > > > -uint64_t 
> > > > cluster_offset,
> > > > +uint64_t 
> > > > guest_cluster_offset,
> > > > +uint64_t 
> > > > host_cluster_offset,
> > > >unsigned 
> > > > offset_in_cluster,
> > > >uint8_t *buffer,
> > > >unsigned bytes)
> > > > @@ -474,8 +474,8 @@ static bool coroutine_fn 
> > > > do_perform_cow_encrypt(BlockDriverState *bs,
> > > >assert((offset_in_cluster & ~BDRV_SECTOR_MASK) == 0);
> > > >assert((bytes & ~BDRV_SECTOR_MASK) == 0);
> > > >assert(s->crypto);
> > > > -if (qcow2_co_encrypt(bs, cluster_offset,
> > > > - src_cluster_offset + offset_in_cluster,
> > > > +if (qcow2_co_encrypt(bs, host_cluster_offset,
> > > > + guest_cluster_offset + offset_in_cluster,
> > > > buffer, bytes) < 0) {
> > > >return false;
> > > >}
> > > > @@ -496,7 +496,7 @@ static int coroutine_fn 
> > > > do_perform_cow_write(BlockDriverState *bs,
> > > >}
> > > >
> > > >ret = qcow2_pre_write_overlap_check(bs, 0,
> > > > -cluster_offset + offset_in_cluster, qiov->size, true);
> > > > +  cluster_offset + offset_in_cluster, qiov->size, true);
> > > 
> > > 
> > > Hmm, unrelated hunk.
> > 
> > I was asked to do this to fix coding style, so that wrapped line,
> > is 4 characters shifted to the right.
> 
> AFAIS, Eric asked only about qcow2_co_encdec calls and definition.. It's OK 
> to fix style in code
> you touch in you patch anyway, but no reason to fix style somewhere else, it 
> dirties patch for
> no reason, making it more difficult (a bit, but still) to review and more 
> difficult to backport..
All right, then I'll drop that change. I kind of agree with this, but I also 
didn't mind doing these fixes
either.

> 
> > 
> > > 
> > > >if (ret < 0) {
> > > >return ret;
> > > >}
> > > > diff --git a/block/qcow2-threads.c b/block/qcow2-threads.c
> > > > index 3b1e63fe41..c3cda0c6a5 100644
> > > > --- a/block/qcow2-threads.c
> > > > +++ b/block/qcow2-threads.c
> > > > @@ -234,15 +234,19 @@ static int qcow2_encdec_pool_func(void *opaque)
> > > >}
> > > >
> > > >static int coroutine_fn
> > > > -qcow2_co_encdec(BlockDriverState *bs, uint64_t file_cluster_offset,
> > > > -  uint64_t offset, void *buf, size_t len, 
> > > > Qcow2EncDecFunc func)
> > > > +qcow2_co_encdec(BlockDriverState *bs, uint64_t host_cluster_offset,
> > > > +uint64_t guest_offset, void *buf, size_t len,
> > > > +Qcow2EncDecFunc func)
> > > >{
> > > >BDRVQcow2State *s = bs->opaque;
> > > > +
> > > > +uint64_t offset = s->crypt_physical_offset ?
> > > > +host_cluster_offset + offset_into_cluster(s, guest_offset) :
> > > > +guest_offset;
> > > > +
> > > >Qcow2EncDecData arg = {
> > > >.block = s->crypto,
> > > > -.offset = s->crypt_physical_offset ?
> > > > -  file_cluster_offset + offset_into_cluster(s, 
> > > > offset) :
> > > > -  offset,
> > > > +.offset = offset,
> > > >.buf = buf,
> > > >.len = len,
> > > >.func = func,
> > > > @@ -251,18 +255,49 @@ qcow2_co_encdec(BlockDriverState *bs, uint64_t 
> > > > file_cluster_offset,
> > > >return qcow2_co_process(bs, qcow2_encdec_pool_func, &arg);
> > > >}
> > > >
> > > > +
> > > > +/*
> > > > + * qcow2_co_encrypt()
> > > > + *
> > > > + * Encrypts one or more contiguous alig

Re: [Qemu-block] [PATCH v2 1/3] block/qcow2: refactoring of threaded encryption code

[Vladimir Sementsov-Ogievskiy]

10.09.2019 15:31, Maxim Levitsky wrote:
> On Sat, 2019-09-07 at 19:08 +, Vladimir Sementsov-Ogievskiy wrote:
>> 06.09.2019 22:57, Maxim Levitsky wrote:
>>> This commit tries to clarify few function arguments,
>>> and add comments describing the encrypt/decrypt interface
>>>
>>> Signed-off-by: Maxim Levitsky 
>>> ---
>>>block/qcow2-cluster.c | 10 +++
>>>block/qcow2-threads.c | 61 ++-
>>>2 files changed, 53 insertions(+), 18 deletions(-)
>>>
>>> diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
>>> index f09cc992af..1989b423da 100644
>>> --- a/block/qcow2-cluster.c
>>> +++ b/block/qcow2-cluster.c
>>> @@ -463,8 +463,8 @@ static int coroutine_fn 
>>> do_perform_cow_read(BlockDriverState *bs,
>>>}
>>>
>>>static bool coroutine_fn do_perform_cow_encrypt(BlockDriverState *bs,
>>> -uint64_t 
>>> src_cluster_offset,
>>> -uint64_t cluster_offset,
>>> +uint64_t 
>>> guest_cluster_offset,
>>> +uint64_t 
>>> host_cluster_offset,
>>>unsigned 
>>> offset_in_cluster,
>>>uint8_t *buffer,
>>>unsigned bytes)
>>> @@ -474,8 +474,8 @@ static bool coroutine_fn 
>>> do_perform_cow_encrypt(BlockDriverState *bs,
>>>assert((offset_in_cluster & ~BDRV_SECTOR_MASK) == 0);
>>>assert((bytes & ~BDRV_SECTOR_MASK) == 0);
>>>assert(s->crypto);
>>> -if (qcow2_co_encrypt(bs, cluster_offset,
>>> - src_cluster_offset + offset_in_cluster,
>>> +if (qcow2_co_encrypt(bs, host_cluster_offset,
>>> + guest_cluster_offset + offset_in_cluster,
>>> buffer, bytes) < 0) {
>>>return false;
>>>}
>>> @@ -496,7 +496,7 @@ static int coroutine_fn 
>>> do_perform_cow_write(BlockDriverState *bs,
>>>}
>>>
>>>ret = qcow2_pre_write_overlap_check(bs, 0,
>>> -cluster_offset + offset_in_cluster, qiov->size, true);
>>> +  cluster_offset + offset_in_cluster, qiov->size, true);
>>
>>
>> Hmm, unrelated hunk.
> 
> I was asked to do this to fix coding style, so that wrapped line,
> is 4 characters shifted to the right.

AFAIS, Eric asked only about qcow2_co_encdec calls and definition.. It's OK to 
fix style in code
you touch in you patch anyway, but no reason to fix style somewhere else, it 
dirties patch for
no reason, making it more difficult (a bit, but still) to review and more 
difficult to backport..

> 
>>
>>>if (ret < 0) {
>>>return ret;
>>>}
>>> diff --git a/block/qcow2-threads.c b/block/qcow2-threads.c
>>> index 3b1e63fe41..c3cda0c6a5 100644
>>> --- a/block/qcow2-threads.c
>>> +++ b/block/qcow2-threads.c
>>> @@ -234,15 +234,19 @@ static int qcow2_encdec_pool_func(void *opaque)
>>>}
>>>
>>>static int coroutine_fn
>>> -qcow2_co_encdec(BlockDriverState *bs, uint64_t file_cluster_offset,
>>> -  uint64_t offset, void *buf, size_t len, Qcow2EncDecFunc 
>>> func)
>>> +qcow2_co_encdec(BlockDriverState *bs, uint64_t host_cluster_offset,
>>> +uint64_t guest_offset, void *buf, size_t len,
>>> +Qcow2EncDecFunc func)
>>>{
>>>BDRVQcow2State *s = bs->opaque;
>>> +
>>> +uint64_t offset = s->crypt_physical_offset ?
>>> +host_cluster_offset + offset_into_cluster(s, guest_offset) :
>>> +guest_offset;
>>> +
>>>Qcow2EncDecData arg = {
>>>.block = s->crypto,
>>> -.offset = s->crypt_physical_offset ?
>>> -  file_cluster_offset + offset_into_cluster(s, offset) 
>>> :
>>> -  offset,
>>> +.offset = offset,
>>>.buf = buf,
>>>.len = len,
>>>.func = func,
>>> @@ -251,18 +255,49 @@ qcow2_co_encdec(BlockDriverState *bs, uint64_t 
>>> file_cluster_offset,
>>>return qcow2_co_process(bs, qcow2_encdec_pool_func, &arg);
>>>}
>>>
>>> +
>>> +/*
>>> + * qcow2_co_encrypt()
>>> + *
>>> + * Encrypts one or more contiguous aligned sectors
>>> + *
>>> + * @host_cluster_offset - on disk offset of the first cluster in which
>>> + * the encrypted data will be written
>>
>>
>> It's not quite right, it's not on disk, but on .file child of qcow2 node, 
>> which
>> may be any other format or protocol node.. So, I called it 
>> file_cluster_offset.
>> But I'm OK with new naming anyway. And it may be better for encryption 
>> related
>> logic..
> 
> Yes, the .file is the underlying storage for both qcow2 metadata and the data,
> and it is unlikely be another qcow2 file. Usually it will be a raw file,
> accessed with some protocol.
> I will change 

Re: [Qemu-block] [Qemu-devel] [PATCH] ahci: enable pci bus master MemoryRegion before loading ahci engines

[John Snow]

On 9/10/19 9:58 AM, Michael S. Tsirkin wrote:
> On Tue, Sep 10, 2019 at 09:50:41AM -0400, John Snow wrote:
>>
>>
>> On 9/10/19 3:04 AM, Michael S. Tsirkin wrote:
>>> On Tue, Sep 10, 2019 at 01:18:37AM +0800, andychiu wrote:
 If Windows 10 guests have enabled 'turn off hard disk after idle'
 option in power settings, and the guest has a SATA disk plugged in,
 the SATA disk will be turned off after a specified idle time.
 If the guest is live migrated or saved/loaded with its SATA disk
 turned off, the following error will occur:

 qemu-system-x86_64: AHCI: Failed to start FIS receive engine: bad FIS 
 receive buffer address
 qemu-system-x86_64: Failed to load ich9_ahci:ahci
 qemu-system-x86_64: error while loading state for instance 0x0 of device 
 ':00:1a.0/ich9_ahci'
 qemu-system-x86_64: load of migration failed: Operation not permitted

 Observation from trace logs shows that a while after Windows 10 turns off
 a SATA disk (IDE disks don't have the following behavior),
 it will disable the PCI_COMMAND_MASTER flag of the pci device containing
 the ahci device. When the the disk is turning back on,
 the PCI_COMMAND_MASTER flag will be restored first.
 But if the guest is migrated or saved/loaded while the disk is off,
 the post_load callback of ahci device, ahci_state_post_load(), will fail
 at ahci_cond_start_engines() if the MemoryRegion
 pci_dev->bus_master_enable_region is not enabled, with pci_dev pointing
 to the PCIDevice struct containing the ahci device.

 This patch enables pci_dev->bus_master_enable_region before calling
 ahci_cond_start_engines() in ahci_state_post_load(), and restore the
 MemoryRegion to its original state afterwards.

 Signed-off-by: andychiu 
>>>
>>> Poking at PCI device internals like this seems fragile.  And force
>>> enabling bus master can lead to unpleasantness like corrupting guest
>>> memory, unhandled interrupts, etc.  E.g. it's quite reasonable,
>>> spec-wise, for the guest to move thing in memory around while bus
>>> mastering is off.
>>>
>>> Can you teach ahci that region being disabled
>>> during migration is ok, and recover from it?
>>
>> That's what I'm wondering.
>>
>> I could try to just disable the FIS RX engine if the mapping fails, but
>> that will require a change to guest visible state.
>>
>> My hunch, though, is that when windows re-enables the device it will
>> need to re-program the address registers anyway, so it might cope well
>> with the FIS RX bit getting switched off.
>>
>> (I'm wondering if it isn't a mistake that QEMU is trying to re-map this
>> address in the first place. Is it legal that the PCI device has pci bus
>> master disabled but we've held on to a mapping?
> 
> If you are poking at guest memory when bus master is off, then most likely 
> yes.
> 
>> Should there be some
>> callback where AHCI knows to invalidate mappings at that point...?)
> 
> ATM the callback is the config write, you check
> proxy->pci_dev.config[PCI_COMMAND] & PCI_COMMAND_MASTER
> and if disabled invalidate the mapping.
> 
> virtio at least has code that pokes at
> proxy->pci_dev.config[PCI_COMMAND] too, I'm quite
> open to a function along the lines of
> pci_is_bus_master_enabled()
> that will do this.
> 

Well, that's not a callback. I don't think it's right to check the
PCI_COMMAND register *every* time AHCI does anything at all to see if
its mappings are still valid.

AHCI makes a mapping *once* when FIS RX is turned on, and it unmaps it
when it's turned off. It assumes it remains valid that whole time. When
we migrate, it checks to see if it was running, and performs the
mappings again to re-boot the state machine.

What I'm asking is; what are the implications of a guest disabling
PCI_COMMAND_MASTER? (I don't know PCI as well as you do.)

What should that mean for the AHCI state machine?

Does this *necessarily* invalidate the mappings?
(In which case -- it's an error that AHCI held on to them after Windows
disabled the card, even if AHCI isn't being engaged by the guest
anymore. Essentially, we were turned off but didn't clean up a dangling
pointer, but we need the event that tells us to clean the dangling mapping.)

Re: [Qemu-block] [PATCH v2] blockjob: update nodes head while removing all bdrv

[Sergio Lopez]

Max Reitz  writes:

> On 10.09.19 15:36, Sergio Lopez wrote:
>> block_job_remove_all_bdrv() iterates through job->nodes, calling
>> bdrv_root_unref_child() for each entry. The call to the latter may
>> reach child_job_[can_]set_aio_ctx(), which will also attempt to
>> traverse job->nodes, potentially finding entries that where freed
>> on previous iterations.
>> 
>> To avoid this situation, update job->nodes head on each iteration to
>> ensure that already freed entries are no longer linked to the list.
>> 
>> RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1746631
>> Signed-off-by: Sergio Lopez 
>> ---
>> Changelog
>> 
>> v2:
>>  - Avoid leaking job->nodes (thanks Max Reitz)
>> ---
>>  blockjob.c | 12 ++--
>>  1 file changed, 10 insertions(+), 2 deletions(-)
>> ---
>> diff --git a/blockjob.c b/blockjob.c
>> index 6e32d1a0c0..ffda6dd1e4 100644
>> --- a/blockjob.c
>> +++ b/blockjob.c
>> @@ -187,13 +187,21 @@ static const BdrvChildRole child_job = {
>>  
>>  void block_job_remove_all_bdrv(BlockJob *job)
>>  {
>> -GSList *l;
>> +GSList *l, *orig_nodes;
>> +
>> +orig_nodes = job->nodes;
>>  for (l = job->nodes; l; l = l->next) {
>>  BdrvChild *c = l->data;
>>  bdrv_op_unblock_all(c->bs, job->blocker);
>>  bdrv_root_unref_child(c);
>> +/*
>> + * The call above may reach child_job_[can_]set_aio_ctx(), which 
>> will
>> + * also traverse job->nodes, so update the head here to make sure it
>> + * doesn't attempt to process an already freed BdrvChild.
>> + */
>> +job->nodes = l->next;
>>  }
>> -g_slist_free(job->nodes);
>> +g_slist_free(orig_nodes);
>>  job->nodes = NULL;
>
> Hm, this assignment is now a no-op.
>
> I think I’d just rewrite the whole function in the following fashion:
>
> orig_nodes = job->nodes;
> while (job->nodes) {
> BdrvChild *c = job->nodes->data;
> [...]
> job->nodes = job->nodes->next;
> }
> g_slist_free(orig_nodes);
>
> What do you think?
>

As this is the first time I was touching this code, I was trying to keep
the changes minimal, but I definitely prefer to rewrite the function as
you suggest.

Should I send a v3, or do you want to send a patch yourself? I don't
really mind either, just want to get this fixed ASAP :-)

Thanks Max,
Sergio.


signature.asc
Description: PGP signature

Re: [Qemu-block] [Qemu-devel] [PATCH] ahci: enable pci bus master MemoryRegion before loading ahci engines

[Michael S. Tsirkin]

On Tue, Sep 10, 2019 at 09:50:41AM -0400, John Snow wrote:
> 
> 
> On 9/10/19 3:04 AM, Michael S. Tsirkin wrote:
> > On Tue, Sep 10, 2019 at 01:18:37AM +0800, andychiu wrote:
> >> If Windows 10 guests have enabled 'turn off hard disk after idle'
> >> option in power settings, and the guest has a SATA disk plugged in,
> >> the SATA disk will be turned off after a specified idle time.
> >> If the guest is live migrated or saved/loaded with its SATA disk
> >> turned off, the following error will occur:
> >>
> >> qemu-system-x86_64: AHCI: Failed to start FIS receive engine: bad FIS 
> >> receive buffer address
> >> qemu-system-x86_64: Failed to load ich9_ahci:ahci
> >> qemu-system-x86_64: error while loading state for instance 0x0 of device 
> >> ':00:1a.0/ich9_ahci'
> >> qemu-system-x86_64: load of migration failed: Operation not permitted
> >>
> >> Observation from trace logs shows that a while after Windows 10 turns off
> >> a SATA disk (IDE disks don't have the following behavior),
> >> it will disable the PCI_COMMAND_MASTER flag of the pci device containing
> >> the ahci device. When the the disk is turning back on,
> >> the PCI_COMMAND_MASTER flag will be restored first.
> >> But if the guest is migrated or saved/loaded while the disk is off,
> >> the post_load callback of ahci device, ahci_state_post_load(), will fail
> >> at ahci_cond_start_engines() if the MemoryRegion
> >> pci_dev->bus_master_enable_region is not enabled, with pci_dev pointing
> >> to the PCIDevice struct containing the ahci device.
> >>
> >> This patch enables pci_dev->bus_master_enable_region before calling
> >> ahci_cond_start_engines() in ahci_state_post_load(), and restore the
> >> MemoryRegion to its original state afterwards.
> >>
> >> Signed-off-by: andychiu 
> > 
> > Poking at PCI device internals like this seems fragile.  And force
> > enabling bus master can lead to unpleasantness like corrupting guest
> > memory, unhandled interrupts, etc.  E.g. it's quite reasonable,
> > spec-wise, for the guest to move thing in memory around while bus
> > mastering is off.
> > 
> > Can you teach ahci that region being disabled
> > during migration is ok, and recover from it?
> 
> That's what I'm wondering.
> 
> I could try to just disable the FIS RX engine if the mapping fails, but
> that will require a change to guest visible state.
> 
> My hunch, though, is that when windows re-enables the device it will
> need to re-program the address registers anyway, so it might cope well
> with the FIS RX bit getting switched off.
> 
> (I'm wondering if it isn't a mistake that QEMU is trying to re-map this
> address in the first place. Is it legal that the PCI device has pci bus
> master disabled but we've held on to a mapping?

If you are poking at guest memory when bus master is off, then most likely yes.

> Should there be some
> callback where AHCI knows to invalidate mappings at that point...?)

ATM the callback is the config write, you check
proxy->pci_dev.config[PCI_COMMAND] & PCI_COMMAND_MASTER
and if disabled invalidate the mapping.

virtio at least has code that pokes at
proxy->pci_dev.config[PCI_COMMAND] too, I'm quite
open to a function along the lines of
pci_is_bus_master_enabled()
that will do this.

-- 
MST

Re: [Qemu-block] [PATCH v2] blockjob: update nodes head while removing all bdrv

[Max Reitz]

On 10.09.19 15:36, Sergio Lopez wrote:
> block_job_remove_all_bdrv() iterates through job->nodes, calling
> bdrv_root_unref_child() for each entry. The call to the latter may
> reach child_job_[can_]set_aio_ctx(), which will also attempt to
> traverse job->nodes, potentially finding entries that where freed
> on previous iterations.
> 
> To avoid this situation, update job->nodes head on each iteration to
> ensure that already freed entries are no longer linked to the list.
> 
> RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1746631
> Signed-off-by: Sergio Lopez 
> ---
> Changelog
> 
> v2:
>  - Avoid leaking job->nodes (thanks Max Reitz)
> ---
>  blockjob.c | 12 ++--
>  1 file changed, 10 insertions(+), 2 deletions(-)
> ---
> diff --git a/blockjob.c b/blockjob.c
> index 6e32d1a0c0..ffda6dd1e4 100644
> --- a/blockjob.c
> +++ b/blockjob.c
> @@ -187,13 +187,21 @@ static const BdrvChildRole child_job = {
>  
>  void block_job_remove_all_bdrv(BlockJob *job)
>  {
> -GSList *l;
> +GSList *l, *orig_nodes;
> +
> +orig_nodes = job->nodes;
>  for (l = job->nodes; l; l = l->next) {
>  BdrvChild *c = l->data;
>  bdrv_op_unblock_all(c->bs, job->blocker);
>  bdrv_root_unref_child(c);
> +/*
> + * The call above may reach child_job_[can_]set_aio_ctx(), which will
> + * also traverse job->nodes, so update the head here to make sure it
> + * doesn't attempt to process an already freed BdrvChild.
> + */
> +job->nodes = l->next;
>  }
> -g_slist_free(job->nodes);
> +g_slist_free(orig_nodes);
>  job->nodes = NULL;

Hm, this assignment is now a no-op.

I think I’d just rewrite the whole function in the following fashion:

orig_nodes = job->nodes;
while (job->nodes) {
BdrvChild *c = job->nodes->data;
[...]
job->nodes = job->nodes->next;
}
g_slist_free(orig_nodes);

What do you think?

>  }
>  
> 




signature.asc
Description: OpenPGP digital signature

Re: [Qemu-block] [Qemu-devel] [PATCH] ahci: enable pci bus master MemoryRegion before loading ahci engines

[John Snow]

On 9/10/19 3:04 AM, Michael S. Tsirkin wrote:
> On Tue, Sep 10, 2019 at 01:18:37AM +0800, andychiu wrote:
>> If Windows 10 guests have enabled 'turn off hard disk after idle'
>> option in power settings, and the guest has a SATA disk plugged in,
>> the SATA disk will be turned off after a specified idle time.
>> If the guest is live migrated or saved/loaded with its SATA disk
>> turned off, the following error will occur:
>>
>> qemu-system-x86_64: AHCI: Failed to start FIS receive engine: bad FIS 
>> receive buffer address
>> qemu-system-x86_64: Failed to load ich9_ahci:ahci
>> qemu-system-x86_64: error while loading state for instance 0x0 of device 
>> ':00:1a.0/ich9_ahci'
>> qemu-system-x86_64: load of migration failed: Operation not permitted
>>
>> Observation from trace logs shows that a while after Windows 10 turns off
>> a SATA disk (IDE disks don't have the following behavior),
>> it will disable the PCI_COMMAND_MASTER flag of the pci device containing
>> the ahci device. When the the disk is turning back on,
>> the PCI_COMMAND_MASTER flag will be restored first.
>> But if the guest is migrated or saved/loaded while the disk is off,
>> the post_load callback of ahci device, ahci_state_post_load(), will fail
>> at ahci_cond_start_engines() if the MemoryRegion
>> pci_dev->bus_master_enable_region is not enabled, with pci_dev pointing
>> to the PCIDevice struct containing the ahci device.
>>
>> This patch enables pci_dev->bus_master_enable_region before calling
>> ahci_cond_start_engines() in ahci_state_post_load(), and restore the
>> MemoryRegion to its original state afterwards.
>>
>> Signed-off-by: andychiu 
> 
> Poking at PCI device internals like this seems fragile.  And force
> enabling bus master can lead to unpleasantness like corrupting guest
> memory, unhandled interrupts, etc.  E.g. it's quite reasonable,
> spec-wise, for the guest to move thing in memory around while bus
> mastering is off.
> 
> Can you teach ahci that region being disabled
> during migration is ok, and recover from it?

That's what I'm wondering.

I could try to just disable the FIS RX engine if the mapping fails, but
that will require a change to guest visible state.

My hunch, though, is that when windows re-enables the device it will
need to re-program the address registers anyway, so it might cope well
with the FIS RX bit getting switched off.

(I'm wondering if it isn't a mistake that QEMU is trying to re-map this
address in the first place. Is it legal that the PCI device has pci bus
master disabled but we've held on to a mapping? Should there be some
callback where AHCI knows to invalidate mappings at that point...?)

[Qemu-block] [PATCH v2] blockjob: update nodes head while removing all bdrv

[Sergio Lopez]

block_job_remove_all_bdrv() iterates through job->nodes, calling
bdrv_root_unref_child() for each entry. The call to the latter may
reach child_job_[can_]set_aio_ctx(), which will also attempt to
traverse job->nodes, potentially finding entries that where freed
on previous iterations.

To avoid this situation, update job->nodes head on each iteration to
ensure that already freed entries are no longer linked to the list.

RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1746631
Signed-off-by: Sergio Lopez 
---
Changelog

v2:
 - Avoid leaking job->nodes (thanks Max Reitz)
---
 blockjob.c | 12 ++--
 1 file changed, 10 insertions(+), 2 deletions(-)
---
diff --git a/blockjob.c b/blockjob.c
index 6e32d1a0c0..ffda6dd1e4 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -187,13 +187,21 @@ static const BdrvChildRole child_job = {
 
 void block_job_remove_all_bdrv(BlockJob *job)
 {
-GSList *l;
+GSList *l, *orig_nodes;
+
+orig_nodes = job->nodes;
 for (l = job->nodes; l; l = l->next) {
 BdrvChild *c = l->data;
 bdrv_op_unblock_all(c->bs, job->blocker);
 bdrv_root_unref_child(c);
+/*
+ * The call above may reach child_job_[can_]set_aio_ctx(), which will
+ * also traverse job->nodes, so update the head here to make sure it
+ * doesn't attempt to process an already freed BdrvChild.
+ */
+job->nodes = l->next;
 }
-g_slist_free(job->nodes);
+g_slist_free(orig_nodes);
 job->nodes = NULL;
 }
 
-- 
2.21.0

Re: [Qemu-block] [Qemu-devel] [PATCH] block/backup: install notifier during creation

[John Snow]

On 9/10/19 4:19 AM, Stefan Hajnoczi wrote:
> On Wed, Aug 21, 2019 at 04:01:52PM -0400, John Snow wrote:
>>
>>
>> On 8/21/19 10:41 AM, Vladimir Sementsov-Ogievskiy wrote:
>>> 09.08.2019 23:13, John Snow wrote:
 Backup jobs may yield prior to installing their handler, because of the
 job_co_entry shim which guarantees that a job won't begin work until
 we are ready to start an entire transaction.

 Unfortunately, this makes proving correctness about transactional
 points-in-time for backup hard to reason about. Make it explicitly clear
 by moving the handler registration to creation time, and changing the
 write notifier to a no-op until the job is started.

 Reported-by: Vladimir Sementsov-Ogievskiy 
 Signed-off-by: John Snow 
 ---
   block/backup.c | 32 +++-
   include/qemu/job.h |  5 +
   job.c  |  2 +-
   3 files changed, 29 insertions(+), 10 deletions(-)

 diff --git a/block/backup.c b/block/backup.c
 index 07d751aea4..4df5b95415 100644
 --- a/block/backup.c
 +++ b/block/backup.c
 @@ -344,6 +344,13 @@ static int coroutine_fn backup_before_write_notify(
   assert(QEMU_IS_ALIGNED(req->offset, BDRV_SECTOR_SIZE));
   assert(QEMU_IS_ALIGNED(req->bytes, BDRV_SECTOR_SIZE));
   
 +/* The handler is installed at creation time; the actual point-in-time
 + * starts at job_start(). Transactions guarantee those two points are
 + * the same point in time. */
 +if (!job_started(&job->common.job)) {
 +return 0;
 +}
>>>
>>> Hmm, sorry if it is a stupid question, I'm not good in multiprocessing and 
>>> in
>>> Qemu iothreads..
>>>
>>> job_started just reads job->co. If bs runs in iothread, and therefore 
>>> write-notifier
>>> is in iothread, when job_start is called from main thread.. Is it 
>>> guaranteed that
>>> write-notifier will see job->co variable change early enough to not miss 
>>> guest write?
>>> Should not job->co be volatile for example or something like this?
>>>
>>> If not think about this patch looks good for me.
>>>
>>
>> You know, it's a really good question.
>> So good, in fact, that I have no idea.
>>
>> ¯\_(ツ)_/¯
>>
>> I'm fairly certain that IO will not come in until the .clean phase of a
>> qmp_transaction, because bdrv_drained_begin(bs) is called during
>> .prepare, and we activate the handler (by starting the job) in .commit.
>> We do not end the drained section until .clean.
>>
>> I'm not fully clear on what threading guarantees we have otherwise,
>> though; is it possible that "Thread A" would somehow lift the bdrv_drain
>> on an IO thread ("Thread B") and, after that, "Thread B" would somehow
>> still be able to see an outdated version of job->co that was set by
>> "Thread A"?
>>
>> I doubt it; but I can't prove it.
> 
> In the qmp_backup() case (not qmp_transaction()) there is:
> 
>   void qmp_drive_backup(DriveBackup *arg, Error **errp)
>   {
> 
>   BlockJob *job;
>   job = do_drive_backup(arg, NULL, errp);
>   if (job) {
>   job_start(&job->job);
>   }
>   }
> 
> job_start() is called without any thread synchronization, which is
> usually fine because the coroutine doesn't run until job_start() calls
> aio_co_enter().
> 
> Now that the before write notifier has been installed early, there is
> indeed a race between job_start() and the write notifier accessing
> job->co from an IOThread.
> 
> The write before notifier might see job->co != NULL before job_start()
> has finished.  This could lead to issues if job_*() APIs are invoked by
> the write notifier and access an in-between job state.
> 

I see. I think in this case, as long as it sees != NULL, that the
notifier is actually safe to run. I agree that this might be confusing
to verify and could bite us in the future. The worry we had, too, is
more the opposite: will it see NULL for too long? We want to make sure
that it is registering as true *before the first yield*.

> A safer approach is to set a BackupBlockJob variable at the beginning of
> backup_run() and check it from the before write notifier.
> 

That's too late, for reasons below.

> That said, I don't understand the benefit of this patch and IMO it makes
> the code harder to understand because now we need to think about the
> created but not started state too.
> 
> Stefan
> 

It's always possible I've hyped myself up into believing there's a
problem where there isn't one, but the fear is this:

The point in time from a QMP transaction covers the job creation and the
job start, but when we start the job it will actually yield before we
get to backup_run -- and there is no guarantee that the handler will get
installed synchronously, so the point in time ends before the handler
activates.

The yield occurs in job_co_entry as an intentional feature of forcing a
yield and pause point at run time -- so it's harder to write a job that
accidentally hog

Re: [Qemu-block] [PATCH v6 04/42] block: Add child access functions

[Kevin Wolf]

Am 10.09.2019 um 14:59 hat Max Reitz geschrieben:
> On 10.09.19 14:48, Kevin Wolf wrote:
> > Am 10.09.2019 um 13:36 hat Max Reitz geschrieben:
> >> On 10.09.19 12:47, Kevin Wolf wrote:
> >>> Am 10.09.2019 um 11:14 hat Max Reitz geschrieben:
>  Maybe we should stop declaring Quorum a filter and then rename the
>  bdrv_recurse_is_first_non_filter() to, I don’t know,
>  bdrv_recurse_can_be_replaced_by_mirror()?
> >>>
> >>> Why not.
> >>
> >> It feels difficult to do in this series because this is a whole new can
> >> of worms.
> >>
> >> In patch 35, I actually replace the mirror use case by
> >> is_filtered_child().  So it looks to me as if that should not be done,
> >> because I should instead fix bdrv_recurse_is_first_non_filter() (and
> >> rename it), because quorum does allow replacing its children by mirror,
> >> even if it does not act as a filter for them.
> >>
> >> OTOH, there are other users of bdrv_is_first_non_filter().  Those are
> >> qmp_block_resize() and external_snapshot_prepare(), who throw an error
> >> if that returns false.
> >>
> >> I think that’s just wrong.  First of all, I don’t even know why we have
> >> that restriction anymore (I can imagine why it used to make sense before
> >> the permission system).  qmp_block_resize() should always work as long
> >> as it can get BLK_PERM_RESIZE; and I don’t know why the parents of some
> >> node would care if you take a snapshot of their child.
> > 
> > Hm, doesn't it make sense in a way for qmp_block_resize() at least? It
> > means that you can't resize just a filter, but you need to resize the
> > image that actually provides the data for the filter.
> 
> Filters generally implement .bdrv_truncate() by passing it through, so
> it should be fine.

Good point.

Then checking bdrv_is_first_non_filter() probably just forbids the only
command that would actually work correctly (resizing the top-level
filter).

Kevin


signature.asc
Description: PGP signature

Re: [Qemu-block] [PATCH v6 20/42] block/snapshot: Fix fallback

[Max Reitz]

On 10.09.19 14:49, Kevin Wolf wrote:
> Am 10.09.2019 um 14:04 hat Max Reitz geschrieben:
>> On 10.09.19 13:56, Kevin Wolf wrote:
>>> Am 09.08.2019 um 18:13 hat Max Reitz geschrieben:
 If the top node's driver does not provide snapshot functionality and we
 want to fall back to a node down the chain, we need to snapshot all
 non-COW children.  For simplicity's sake, just do not fall back if there
 is more than one such child.

 bdrv_snapshot_goto() becomes a bit weird because we may have to redirect
 the actual child pointer, so it only works if the fallback child is
 bs->file or bs->backing (and then we have to find out which it is).

 Suggested-by: Vladimir Sementsov-Ogievskiy 
 Signed-off-by: Max Reitz 
 ---
  block/snapshot.c | 100 +--
  1 file changed, 79 insertions(+), 21 deletions(-)

 diff --git a/block/snapshot.c b/block/snapshot.c
 index f2f48f926a..35403c167f 100644
 --- a/block/snapshot.c
 +++ b/block/snapshot.c
 @@ -146,6 +146,32 @@ bool 
 bdrv_snapshot_find_by_id_and_name(BlockDriverState *bs,
  return ret;
  }
  
 +/**
 + * Return the child BDS to which we can fall back if the given BDS
 + * does not support snapshots.
 + * Return NULL if there is no BDS to (safely) fall back to.
 + */
 +static BlockDriverState *bdrv_snapshot_fallback(BlockDriverState *bs)
 +{
 +BlockDriverState *child_bs = NULL;
 +BdrvChild *child;
 +
 +QLIST_FOREACH(child, &bs->children, next) {
 +if (child == bdrv_filtered_cow_child(bs)) {
 +/* Ignore: COW children need not be included in snapshots */
 +continue;
 +}
 +
 +if (child_bs) {
 +/* Cannot fall back to a single child if there are multiple */
 +return NULL;
 +}
 +child_bs = child->bs;
 +}
 +
 +return child_bs;
 +}
>>>
>>> Why do we return child->bs here when bdrv_snapshot_goto() then needs to
>>> reconstruct what the associated BdrvChild was? Wouldn't it make more
>>> sense to return BdrvChild** from here and maybe have a small wrapper for
>>> the other functions that only need a BDS?
>>
>> What would you return instead?  &child doesn’t work.
> 
> Oops, brain fart. :-)
> 
>> We could limit ourselves to bs->file and bs->backing.  It just seemed
>> like a bit of an artificial limit to me, because we only really have it
>> for bdrv_snapshot_goto().
> 
> Hm, but then, what use is supporting other children for creating a
> snapshot when you can't load it any more afterwards?

Well, the snapshot is still there, it’s just on a different node.  So in
theory, you could take a snapshot in a live VM (where the snapshotting
node is not at the top), and then later revert to it with qemu-img (by
accessing the file with the snapshot directly).

Though in practice this is just a fallback anyway, and I don’t think we
currently have anything where it would make sense to fall through some
node to any child but .file or .backing.  So why not.  It would be
shorter, too.  (Well, plus the short comment why looking at .file and
.backing is sufficient.)

Max



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-block] [PATCH v6 04/42] block: Add child access functions

[Max Reitz]

On 10.09.19 14:48, Kevin Wolf wrote:
> Am 10.09.2019 um 13:36 hat Max Reitz geschrieben:
>> On 10.09.19 12:47, Kevin Wolf wrote:
>>> Am 10.09.2019 um 11:14 hat Max Reitz geschrieben:
 Maybe we should stop declaring Quorum a filter and then rename the
 bdrv_recurse_is_first_non_filter() to, I don’t know,
 bdrv_recurse_can_be_replaced_by_mirror()?
>>>
>>> Why not.
>>
>> It feels difficult to do in this series because this is a whole new can
>> of worms.
>>
>> In patch 35, I actually replace the mirror use case by
>> is_filtered_child().  So it looks to me as if that should not be done,
>> because I should instead fix bdrv_recurse_is_first_non_filter() (and
>> rename it), because quorum does allow replacing its children by mirror,
>> even if it does not act as a filter for them.
>>
>> OTOH, there are other users of bdrv_is_first_non_filter().  Those are
>> qmp_block_resize() and external_snapshot_prepare(), who throw an error
>> if that returns false.
>>
>> I think that’s just wrong.  First of all, I don’t even know why we have
>> that restriction anymore (I can imagine why it used to make sense before
>> the permission system).  qmp_block_resize() should always work as long
>> as it can get BLK_PERM_RESIZE; and I don’t know why the parents of some
>> node would care if you take a snapshot of their child.
> 
> Hm, doesn't it make sense in a way for qmp_block_resize() at least? It
> means that you can't resize just a filter, but you need to resize the
> image that actually provides the data for the filter.

Filters generally implement .bdrv_truncate() by passing it through, so
it should be fine.

> Of course, there is no reason for it to be the _first_ non-filter as
> long as BLK_PERM_RESIZE is shared, but just some non-filter node.
> 
> Two more random observations:
> 
> * quorum uses bdrv_filter_default_perms(), which allows BLK_PERM_RESIZE.
>   I think this is wrong and quorum should make sure that all children are
>   always the same size because otherwise it can't tell what its own size
>   is. (Or vote on size...? :-/) Probably not a problem in practice as
>   long as we check bdrv_is_first_non_filter().

(“Quorum is broken” seems to be a recurring observation.)

I agree, it shouldn’t share that permission.

> * child_file and child_backing don't implement .resize. So if you resize
>   a non-top-level image, parents (in particular filters) don't get their
>   size adjusted. This is probably a bug, too, but one that isn't
>   prevented by bdrv_is_first_non_filter() and should be visible today.

Hm. :-/

The good news is that I can try to fix this independently of this series.

[...]

>> We have come to two results, as far as I can see:
>>
>> First, naming COW backing nodes “COW filtered children” clashes with our
>> existing use of ”filter”.  There is no point in forcing the ”filter”
>> label on everything.  We can just keep calling (R/W) filters filters and
>> COW backing children COW children.  The names are succinct enough.
>>
>> In some cases, we don’t care whether something is a COW or filtered
>> child, in such a case a caller can be bothered to use the slightly
>> longer bdrv_cow_or_filtered_child().
> 
> Aye.
> 
>> Second, most of the time we want a filter node to have a clear and
>> unique path to go down.  This is the important property of filters: That
>> you can skip them and go to the node that actually has the data.
>>
>> Quorum breaks this by having multiple children, and nobody knows which
>> of them has the data we will see on the next read operation.
>>
>> All “filters” who could have multiple children would have this problem.
>>  Hence a filter must always have a single unique data child.  I think.
> 
> I agree, and this is the condition that I mentioned somewhere above, but
> failed to actually find guaranteed somewhere. We should probably make
> this explicit.
> 
> Of course, quorum and similar things intend all their children to
> provide the same data, but the whole point of the driver is that this is
> not always guaranteed, so they aren't actually filters.

OK, great, I’ll get cracking then.

Max



signature.asc
Description: OpenPGP digital signature

[Qemu-block] [PATCH v2 6/7] curl: Handle success in multi_check_completion

[Max Reitz]

Background: As of cURL 7.59.0, it verifies that several functions are
not called from within a callback.  Among these functions is
curl_multi_add_handle().

curl_read_cb() is a callback from cURL and not a coroutine.  Waking up
acb->co will lead to entering it then and there, which means the current
request will settle and the caller (if it runs in the same coroutine)
may then issue the next request.  In such a case, we will enter
curl_setup_preadv() effectively from within curl_read_cb().

Calling curl_multi_add_handle() will then fail and the new request will
not be processed.

Fix this by not letting curl_read_cb() wake up acb->co.  Instead, leave
the whole business of settling the AIOCB objects to
curl_multi_check_completion() (which is called from our timer callback
and our FD handler, so not from any cURL callbacks).

Reported-by: Natalie Gavrielov 
Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1740193
Cc: qemu-sta...@nongnu.org
Signed-off-by: Max Reitz 
---
 block/curl.c | 69 ++--
 1 file changed, 29 insertions(+), 40 deletions(-)

diff --git a/block/curl.c b/block/curl.c
index fd70f1ebc4..c343c7ed3d 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -229,7 +229,6 @@ static size_t curl_read_cb(void *ptr, size_t size, size_t 
nmemb, void *opaque)
 {
 CURLState *s = ((CURLState*)opaque);
 size_t realsize = size * nmemb;
-int i;
 
 trace_curl_read_cb(realsize);
 
@@ -245,32 +244,6 @@ static size_t curl_read_cb(void *ptr, size_t size, size_t 
nmemb, void *opaque)
 memcpy(s->orig_buf + s->buf_off, ptr, realsize);
 s->buf_off += realsize;
 
-for(i=0; iacb[i];
-
-if (!acb)
-continue;
-
-if ((s->buf_off >= acb->end)) {
-size_t request_length = acb->bytes;
-
-qemu_iovec_from_buf(acb->qiov, 0, s->orig_buf + acb->start,
-acb->end - acb->start);
-
-if (acb->end - acb->start < request_length) {
-size_t offset = acb->end - acb->start;
-qemu_iovec_memset(acb->qiov, offset, 0,
-  request_length - offset);
-}
-
-acb->ret = 0;
-s->acb[i] = NULL;
-qemu_mutex_unlock(&s->s->mutex);
-aio_co_wake(acb->co);
-qemu_mutex_lock(&s->s->mutex);
-}
-}
-
 read_end:
 /* curl will error out if we do not return this value */
 return size * nmemb;
@@ -351,13 +324,14 @@ static void curl_multi_check_completion(BDRVCURLState *s)
 break;
 
 if (msg->msg == CURLMSG_DONE) {
+int i;
 CURLState *state = NULL;
+bool error = msg->data.result != CURLE_OK;
+
 curl_easy_getinfo(msg->easy_handle, CURLINFO_PRIVATE,
   (char **)&state);
 
-/* ACBs for successful messages get completed in curl_read_cb */
-if (msg->data.result != CURLE_OK) {
-int i;
+if (error) {
 static int errcount = 100;
 
 /* Don't lose the original error message from curl, since
@@ -369,20 +343,35 @@ static void curl_multi_check_completion(BDRVCURLState *s)
 error_report("curl: further errors suppressed");
 }
 }
+}
 
-for (i = 0; i < CURL_NUM_ACB; i++) {
-CURLAIOCB *acb = state->acb[i];
+for (i = 0; i < CURL_NUM_ACB; i++) {
+CURLAIOCB *acb = state->acb[i];
 
-if (acb == NULL) {
-continue;
-}
+if (acb == NULL) {
+continue;
+}
+
+if (!error) {
+/* Assert that we have read all data */
+assert(state->buf_off >= acb->end);
+
+qemu_iovec_from_buf(acb->qiov, 0,
+state->orig_buf + acb->start,
+acb->end - acb->start);
 
-acb->ret = -EIO;
-state->acb[i] = NULL;
-qemu_mutex_unlock(&s->mutex);
-aio_co_wake(acb->co);
-qemu_mutex_lock(&s->mutex);
+if (acb->end - acb->start < acb->bytes) {
+size_t offset = acb->end - acb->start;
+qemu_iovec_memset(acb->qiov, offset, 0,
+  acb->bytes - offset);
+}
 }
+
+acb->ret = error ? -EIO : 0;
+state->acb[i] = NULL;
+qemu_mutex_unlock(&s->mutex);
+aio_co_wake(acb->co);
+qemu_mutex_lock(&s->mutex);
 }
 
 curl_clean_state(state);
-- 
2.21.0

Re: [Qemu-block] [PATCH v6 20/42] block/snapshot: Fix fallback

[Kevin Wolf]

Am 10.09.2019 um 14:04 hat Max Reitz geschrieben:
> On 10.09.19 13:56, Kevin Wolf wrote:
> > Am 09.08.2019 um 18:13 hat Max Reitz geschrieben:
> >> If the top node's driver does not provide snapshot functionality and we
> >> want to fall back to a node down the chain, we need to snapshot all
> >> non-COW children.  For simplicity's sake, just do not fall back if there
> >> is more than one such child.
> >>
> >> bdrv_snapshot_goto() becomes a bit weird because we may have to redirect
> >> the actual child pointer, so it only works if the fallback child is
> >> bs->file or bs->backing (and then we have to find out which it is).
> >>
> >> Suggested-by: Vladimir Sementsov-Ogievskiy 
> >> Signed-off-by: Max Reitz 
> >> ---
> >>  block/snapshot.c | 100 +--
> >>  1 file changed, 79 insertions(+), 21 deletions(-)
> >>
> >> diff --git a/block/snapshot.c b/block/snapshot.c
> >> index f2f48f926a..35403c167f 100644
> >> --- a/block/snapshot.c
> >> +++ b/block/snapshot.c
> >> @@ -146,6 +146,32 @@ bool 
> >> bdrv_snapshot_find_by_id_and_name(BlockDriverState *bs,
> >>  return ret;
> >>  }
> >>  
> >> +/**
> >> + * Return the child BDS to which we can fall back if the given BDS
> >> + * does not support snapshots.
> >> + * Return NULL if there is no BDS to (safely) fall back to.
> >> + */
> >> +static BlockDriverState *bdrv_snapshot_fallback(BlockDriverState *bs)
> >> +{
> >> +BlockDriverState *child_bs = NULL;
> >> +BdrvChild *child;
> >> +
> >> +QLIST_FOREACH(child, &bs->children, next) {
> >> +if (child == bdrv_filtered_cow_child(bs)) {
> >> +/* Ignore: COW children need not be included in snapshots */
> >> +continue;
> >> +}
> >> +
> >> +if (child_bs) {
> >> +/* Cannot fall back to a single child if there are multiple */
> >> +return NULL;
> >> +}
> >> +child_bs = child->bs;
> >> +}
> >> +
> >> +return child_bs;
> >> +}
> > 
> > Why do we return child->bs here when bdrv_snapshot_goto() then needs to
> > reconstruct what the associated BdrvChild was? Wouldn't it make more
> > sense to return BdrvChild** from here and maybe have a small wrapper for
> > the other functions that only need a BDS?
> 
> What would you return instead?  &child doesn’t work.

Oops, brain fart. :-)

> We could limit ourselves to bs->file and bs->backing.  It just seemed
> like a bit of an artificial limit to me, because we only really have it
> for bdrv_snapshot_goto().

Hm, but then, what use is supporting other children for creating a
snapshot when you can't load it any more afterwards?

Kevin


signature.asc
Description: PGP signature

[Qemu-block] [PATCH v2 7/7] curl: Check curl_multi_add_handle()'s return code

[Max Reitz]

If we had done that all along, debugging would have been much simpler.
(Also, I/O errors are better than hangs.)

Signed-off-by: Max Reitz 
---
 block/curl.c | 8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/block/curl.c b/block/curl.c
index c343c7ed3d..f86299378e 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -882,7 +882,13 @@ static void curl_setup_preadv(BlockDriverState *bs, 
CURLAIOCB *acb)
 trace_curl_setup_preadv(acb->bytes, start, state->range);
 curl_easy_setopt(state->curl, CURLOPT_RANGE, state->range);
 
-curl_multi_add_handle(s->multi, state->curl);
+if (curl_multi_add_handle(s->multi, state->curl) != CURLM_OK) {
+state->acb[0] = NULL;
+acb->ret = -EIO;
+
+curl_clean_state(state);
+goto out;
+}
 
 /* Tell curl it needs to kick things off */
 curl_multi_socket_action(s->multi, CURL_SOCKET_TIMEOUT, 0, &running);
-- 
2.21.0

Re: [Qemu-block] [PATCH v6 04/42] block: Add child access functions

[Kevin Wolf]

Am 10.09.2019 um 13:36 hat Max Reitz geschrieben:
> On 10.09.19 12:47, Kevin Wolf wrote:
> > Am 10.09.2019 um 11:14 hat Max Reitz geschrieben:
> >> Maybe we should stop declaring Quorum a filter and then rename the
> >> bdrv_recurse_is_first_non_filter() to, I don’t know,
> >> bdrv_recurse_can_be_replaced_by_mirror()?
> > 
> > Why not.
> 
> It feels difficult to do in this series because this is a whole new can
> of worms.
> 
> In patch 35, I actually replace the mirror use case by
> is_filtered_child().  So it looks to me as if that should not be done,
> because I should instead fix bdrv_recurse_is_first_non_filter() (and
> rename it), because quorum does allow replacing its children by mirror,
> even if it does not act as a filter for them.
> 
> OTOH, there are other users of bdrv_is_first_non_filter().  Those are
> qmp_block_resize() and external_snapshot_prepare(), who throw an error
> if that returns false.
> 
> I think that’s just wrong.  First of all, I don’t even know why we have
> that restriction anymore (I can imagine why it used to make sense before
> the permission system).  qmp_block_resize() should always work as long
> as it can get BLK_PERM_RESIZE; and I don’t know why the parents of some
> node would care if you take a snapshot of their child.

Hm, doesn't it make sense in a way for qmp_block_resize() at least? It
means that you can't resize just a filter, but you need to resize the
image that actually provides the data for the filter.

Of course, there is no reason for it to be the _first_ non-filter as
long as BLK_PERM_RESIZE is shared, but just some non-filter node.

Two more random observations:

* quorum uses bdrv_filter_default_perms(), which allows BLK_PERM_RESIZE.
  I think this is wrong and quorum should make sure that all children are
  always the same size because otherwise it can't tell what its own size
  is. (Or vote on size...? :-/) Probably not a problem in practice as
  long as we check bdrv_is_first_non_filter().

* child_file and child_backing don't implement .resize. So if you resize
  a non-top-level image, parents (in particular filters) don't get their
  size adjusted. This is probably a bug, too, but one that isn't
  prevented by bdrv_is_first_non_filter() and should be visible today.

> >>> Maybe the documentation of bdrv_filtered_child() needs to be rephrased?
> >>>
> >>> Going back to qcow2, it's really not much different as it has multiple
> >>> (two) filtered children, too.
> >>
> >> Well, it doesn’t.  It isn’t an R/W filter.
> > 
> > What do I have to look at to see whether something is an R/W filter or
> > not? qcow2 matches your criteria for an R/W filter.
> 
> No.  Some qcow2 nodes match the criteria.  But not all, which makes the
> qcow2 driver not a filter driver.
> 
> > You say that it's not useful, so it's not an R/W filter anyway. But
> > where in the code could I get this information?
> 
> “Where in the code”?  Do you want to add a comment to every BlockDriver
> structure on why it does or doesn’t set .is_filter?

Never mind, I just didn't understand that .is_filter is the thing that
defines a R/W filter. In fact, I didn't really understand what
.is_filter was supposed to mean at all because I was so confused. For
some reason I was sure it had to mean any kind of filter, but that
assumption just didn't match up with its use at all.

> > Specficially, according to your definition, qcow2 filters both the
> > backing file (COW filter) and the external data file (R/W filter).
> 
>  Not wrong.  But the same question as for raw arises: Is there any use to
>  declaring qcow2 an R/W filter driver just because it fits the definition?
> >>>
> >>> Wait, where is there even a place where this could be declared?
> >>>
> >>> The once thing I see that a driver even can declare is drv->is_filter,
> >>> which is about the whole driver and not about nodes. It is false for
> >>> qcow2.
> >>
> >> That’s correct.  But that’s not a fundamental problem, of course, we
> >> could make it a per-BDS attribute if that made sense.
> > 
> > I was thinking per-child, actually, because you declare one BdrvChild
> > filtered and another not filtered.
> 
> Why don’t you say so from the start then?

Yes, I wrote "nodes", thought "child nodes" and should have said
"children" because edges are not nodes. My bad, sorry.

> (Sorry, but honestly about 30 % of this discussion to me feels like
> you’re playing games with me.  Please don’t take this the wrong way, I
> mean it very neutrally.  It’s just that I feel like I’m explaining
> things to you that you very much know, but you just want me to say them.
>  And that feels unproductive and sometimes indeed frustrating.)

No, certainly not. If my mails seemed confusing or pointless, it just
shows how thoroughly confused I was.

> One thing is that this wouldn’t make the quorum case any easier because
> it actually doesn’t know for which children it acts as a filter and for
> which it doesn’t.
> 
> > But by now I think mos

[Qemu-block] [PATCH v2 2/7] curl: Keep *socket until the end of curl_sock_cb()

[Max Reitz]

This does not really change anything, but it makes the code a bit easier
to follow once we use @socket as the opaque pointer for
aio_set_fd_handler().

Cc: qemu-sta...@nongnu.org
Signed-off-by: Max Reitz 
---
 block/curl.c | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/block/curl.c b/block/curl.c
index 92dc2f630e..95d7b77dc0 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -172,10 +172,6 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int 
action,
 
 QLIST_FOREACH(socket, &state->sockets, next) {
 if (socket->fd == fd) {
-if (action == CURL_POLL_REMOVE) {
-QLIST_REMOVE(socket, next);
-g_free(socket);
-}
 break;
 }
 }
@@ -185,7 +181,6 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int 
action,
 socket->state = state;
 QLIST_INSERT_HEAD(&state->sockets, socket, next);
 }
-socket = NULL;
 
 trace_curl_sock_cb(action, (int)fd);
 switch (action) {
@@ -207,6 +202,11 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int 
action,
 break;
 }
 
+if (action == CURL_POLL_REMOVE) {
+QLIST_REMOVE(socket, next);
+g_free(socket);
+}
+
 return 0;
 }
 
-- 
2.21.0

[Qemu-block] [PATCH v2 4/7] curl: Pass CURLSocket to curl_multi_do()

[Max Reitz]

curl_multi_do_locked() currently marks all sockets as ready.  That is
not only inefficient, but in fact unsafe (the loop is).  A follow-up
patch will change that, but to do so, curl_multi_do_locked() needs to
know exactly which socket is ready; and that is accomplished by this
patch here.

Cc: qemu-sta...@nongnu.org
Signed-off-by: Max Reitz 
---
 block/curl.c | 20 +++-
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/block/curl.c b/block/curl.c
index 5838afef99..cf2686218d 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -185,15 +185,15 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int 
action,
 switch (action) {
 case CURL_POLL_IN:
 aio_set_fd_handler(s->aio_context, fd, false,
-   curl_multi_do, NULL, NULL, state);
+   curl_multi_do, NULL, NULL, socket);
 break;
 case CURL_POLL_OUT:
 aio_set_fd_handler(s->aio_context, fd, false,
-   NULL, curl_multi_do, NULL, state);
+   NULL, curl_multi_do, NULL, socket);
 break;
 case CURL_POLL_INOUT:
 aio_set_fd_handler(s->aio_context, fd, false,
-   curl_multi_do, curl_multi_do, NULL, state);
+   curl_multi_do, curl_multi_do, NULL, socket);
 break;
 case CURL_POLL_REMOVE:
 aio_set_fd_handler(s->aio_context, fd, false,
@@ -392,9 +392,10 @@ static void curl_multi_check_completion(BDRVCURLState *s)
 }
 
 /* Called with s->mutex held.  */
-static void curl_multi_do_locked(CURLState *s)
+static void curl_multi_do_locked(CURLSocket *ready_socket)
 {
 CURLSocket *socket, *next_socket;
+CURLState *s = ready_socket->state;
 int running;
 int r;
 
@@ -413,12 +414,13 @@ static void curl_multi_do_locked(CURLState *s)
 
 static void curl_multi_do(void *arg)
 {
-CURLState *s = (CURLState *)arg;
+CURLSocket *socket = arg;
+BDRVCURLState *s = socket->state->s;
 
-qemu_mutex_lock(&s->s->mutex);
-curl_multi_do_locked(s);
-curl_multi_check_completion(s->s);
-qemu_mutex_unlock(&s->s->mutex);
+qemu_mutex_lock(&s->mutex);
+curl_multi_do_locked(socket);
+curl_multi_check_completion(s);
+qemu_mutex_unlock(&s->mutex);
 }
 
 static void curl_multi_timeout_do(void *arg)
-- 
2.21.0

[Qemu-block] [PATCH v2 5/7] curl: Report only ready sockets

[Max Reitz]

Instead of reporting all sockets to cURL, only report the one that has
caused curl_multi_do_locked() to be called.  This lets us get rid of the
QLIST_FOREACH_SAFE() list, which was actually wrong: SAFE foreaches are
only safe when the current element is removed in each iteration.  If it
possible for the list to be concurrently modified, we cannot guarantee
that only the current element will be removed.  Therefore, we must not
use QLIST_FOREACH_SAFE() here.

Fixes: ff5ca1664af85b24a4180d595ea6873fd3deac57
Cc: qemu-sta...@nongnu.org
Signed-off-by: Max Reitz 
---
 block/curl.c | 17 ++---
 1 file changed, 6 insertions(+), 11 deletions(-)

diff --git a/block/curl.c b/block/curl.c
index cf2686218d..fd70f1ebc4 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -392,24 +392,19 @@ static void curl_multi_check_completion(BDRVCURLState *s)
 }
 
 /* Called with s->mutex held.  */
-static void curl_multi_do_locked(CURLSocket *ready_socket)
+static void curl_multi_do_locked(CURLSocket *socket)
 {
-CURLSocket *socket, *next_socket;
-CURLState *s = ready_socket->state;
+BDRVCURLState *s = socket->state->s;
 int running;
 int r;
 
-if (!s->s->multi) {
+if (!s->multi) {
 return;
 }
 
-/* Need to use _SAFE because curl_multi_socket_action() may trigger
- * curl_sock_cb() which might modify this list */
-QLIST_FOREACH_SAFE(socket, &s->sockets, next, next_socket) {
-do {
-r = curl_multi_socket_action(s->s->multi, socket->fd, 0, &running);
-} while (r == CURLM_CALL_MULTI_PERFORM);
-}
+do {
+r = curl_multi_socket_action(s->multi, socket->fd, 0, &running);
+} while (r == CURLM_CALL_MULTI_PERFORM);
 }
 
 static void curl_multi_do(void *arg)
-- 
2.21.0

[Qemu-block] [PATCH v2 3/7] curl: Check completion in curl_multi_do()

[Max Reitz]

While it is more likely that transfers complete after some file
descriptor has data ready to read, we probably should not rely on it.
Better be safe than sorry and call curl_multi_check_completion() in
curl_multi_do(), too, just like it is done in curl_multi_read().

With this change, curl_multi_do() and curl_multi_read() are actually the
same, so drop curl_multi_read() and use curl_multi_do() as the sole FD
handler.

Signed-off-by: Max Reitz 
---
 block/curl.c | 14 ++
 1 file changed, 2 insertions(+), 12 deletions(-)

diff --git a/block/curl.c b/block/curl.c
index 95d7b77dc0..5838afef99 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -139,7 +139,6 @@ typedef struct BDRVCURLState {
 
 static void curl_clean_state(CURLState *s);
 static void curl_multi_do(void *arg);
-static void curl_multi_read(void *arg);
 
 #ifdef NEED_CURL_TIMER_CALLBACK
 /* Called from curl_multi_do_locked, with s->mutex held.  */
@@ -186,7 +185,7 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int 
action,
 switch (action) {
 case CURL_POLL_IN:
 aio_set_fd_handler(s->aio_context, fd, false,
-   curl_multi_read, NULL, NULL, state);
+   curl_multi_do, NULL, NULL, state);
 break;
 case CURL_POLL_OUT:
 aio_set_fd_handler(s->aio_context, fd, false,
@@ -194,7 +193,7 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int 
action,
 break;
 case CURL_POLL_INOUT:
 aio_set_fd_handler(s->aio_context, fd, false,
-   curl_multi_read, curl_multi_do, NULL, state);
+   curl_multi_do, curl_multi_do, NULL, state);
 break;
 case CURL_POLL_REMOVE:
 aio_set_fd_handler(s->aio_context, fd, false,
@@ -416,15 +415,6 @@ static void curl_multi_do(void *arg)
 {
 CURLState *s = (CURLState *)arg;
 
-qemu_mutex_lock(&s->s->mutex);
-curl_multi_do_locked(s);
-qemu_mutex_unlock(&s->s->mutex);
-}
-
-static void curl_multi_read(void *arg)
-{
-CURLState *s = (CURLState *)arg;
-
 qemu_mutex_lock(&s->s->mutex);
 curl_multi_do_locked(s);
 curl_multi_check_completion(s->s);
-- 
2.21.0

[Qemu-block] [PATCH v2 0/7] block/curl: Fix hang and potential crash

[Max Reitz]

Hi,

As reported in https://bugzilla.redhat.com/show_bug.cgi?id=1740193, our
curl block driver can spontaneously hang.  This becomes visible e.g.
when reading compressed qcow2 images:

$ qemu-img convert -p -O raw -n \
  https://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img \
  null-co://

(Hangs at 74.21 %, usually.)

A more direct way is:

$ qemu-img bench -f raw http://download.qemu.org/qemu-4.1.0.tar.xz \
-d 1 -S 524288 -c 2

(Which simply performs two requests, and the second one hangs.  You can
use any HTTP resource (probably FTP, too) you’d like that is at least
1 MB in size.)

It turns out that this is because cURL 7.59.0 has added a protective
feature against some misuse we had in our code: curl_multi_add_handle()
must not be called from within a cURL callback, but in some cases we
did.  As of 7.59.0, this fails, our new request is not registered and
the I/O request stalls.  This is fixed by patch 6.

Patch 7 makes us check for curl_multi_add_handle()’s return code,
because if we had done that before, debugging would have been much
simpler.


On the way to fixing it, I had a look over the whole cURL code and found
a suspicious QLIST_FOREACH_SAFE() loop that actually does not seem very
safe at all.  I think this may lead to crashes, although I have never
seen any myself.  https://bugzilla.redhat.com/show_bug.cgi?id=1744602#c5
shows one in exactly the function in question, so I think it actually is
a problem.

This is fixed by patch 5, patches 1, 2, and 4 prepare for it.

(Patch 3 is kind of a misc patch that should ensure that we always end
up calling curl_multi_check_completion() whenever a request might have
been completed.)


v2:
- Patch 2: Remove the socket from the list only add the end of the
   function (yielding a nicer 5+/5- diff stat)
- Patch 3: Added
- Patch 4: Rebased on patch 3, and s/socket/ready_socket/ in one place
- Patch 5: Rebased on the changed patch 4


git-backport-diff against v1:

Key:
[] : patches are identical
[] : number of functional differences between upstream/downstream patch
[down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/7:[] [--] 'curl: Keep pointer to the CURLState in CURLSocket'
002/7:[0007] [FC] 'curl: Keep *socket until the end of curl_sock_cb()'
003/7:[down] 'curl: Check completion in curl_multi_do()'
004/7:[0019] [FC] 'curl: Pass CURLSocket to curl_multi_{do,read}()'
005/7:[0002] [FC] 'curl: Report only ready sockets'
006/7:[] [--] 'curl: Handle success in multi_check_completion'
007/7:[] [--] 'curl: Check curl_multi_add_handle()'s return code'


Max Reitz (7):
  curl: Keep pointer to the CURLState in CURLSocket
  curl: Keep *socket until the end of curl_sock_cb()
  curl: Check completion in curl_multi_do()
  curl: Pass CURLSocket to curl_multi_do()
  curl: Report only ready sockets
  curl: Handle success in multi_check_completion
  curl: Check curl_multi_add_handle()'s return code

 block/curl.c | 133 +++
 1 file changed, 59 insertions(+), 74 deletions(-)

-- 
2.21.0

[Qemu-block] [PATCH v2 1/7] curl: Keep pointer to the CURLState in CURLSocket

[Max Reitz]

A follow-up patch will make curl_multi_do() and curl_multi_read() take a
CURLSocket instead of the CURLState.  They still need the latter,
though, so add a pointer to it to the former.

Cc: qemu-sta...@nongnu.org
Signed-off-by: Max Reitz 
Reviewed-by: John Snow 
---
 block/curl.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/block/curl.c b/block/curl.c
index d4c8e94f3e..92dc2f630e 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -80,6 +80,7 @@ static CURLMcode __curl_multi_socket_action(CURLM 
*multi_handle,
 #define CURL_BLOCK_OPT_TIMEOUT_DEFAULT 5
 
 struct BDRVCURLState;
+struct CURLState;
 
 static bool libcurl_initialized;
 
@@ -97,6 +98,7 @@ typedef struct CURLAIOCB {
 
 typedef struct CURLSocket {
 int fd;
+struct CURLState *state;
 QLIST_ENTRY(CURLSocket) next;
 } CURLSocket;
 
@@ -180,6 +182,7 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int 
action,
 if (!socket) {
 socket = g_new0(CURLSocket, 1);
 socket->fd = fd;
+socket->state = state;
 QLIST_INSERT_HEAD(&state->sockets, socket, next);
 }
 socket = NULL;
-- 
2.21.0

Re: [Qemu-block] [PATCH v2 1/3] block/qcow2: refactoring of threaded encryption code

[Maxim Levitsky]

On Sat, 2019-09-07 at 19:08 +, Vladimir Sementsov-Ogievskiy wrote:
> 06.09.2019 22:57, Maxim Levitsky wrote:
> > This commit tries to clarify few function arguments,
> > and add comments describing the encrypt/decrypt interface
> > 
> > Signed-off-by: Maxim Levitsky 
> > ---
> >   block/qcow2-cluster.c | 10 +++
> >   block/qcow2-threads.c | 61 ++-
> >   2 files changed, 53 insertions(+), 18 deletions(-)
> > 
> > diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
> > index f09cc992af..1989b423da 100644
> > --- a/block/qcow2-cluster.c
> > +++ b/block/qcow2-cluster.c
> > @@ -463,8 +463,8 @@ static int coroutine_fn 
> > do_perform_cow_read(BlockDriverState *bs,
> >   }
> >   
> >   static bool coroutine_fn do_perform_cow_encrypt(BlockDriverState *bs,
> > -uint64_t 
> > src_cluster_offset,
> > -uint64_t cluster_offset,
> > +uint64_t 
> > guest_cluster_offset,
> > +uint64_t 
> > host_cluster_offset,
> >   unsigned 
> > offset_in_cluster,
> >   uint8_t *buffer,
> >   unsigned bytes)
> > @@ -474,8 +474,8 @@ static bool coroutine_fn 
> > do_perform_cow_encrypt(BlockDriverState *bs,
> >   assert((offset_in_cluster & ~BDRV_SECTOR_MASK) == 0);
> >   assert((bytes & ~BDRV_SECTOR_MASK) == 0);
> >   assert(s->crypto);
> > -if (qcow2_co_encrypt(bs, cluster_offset,
> > - src_cluster_offset + offset_in_cluster,
> > +if (qcow2_co_encrypt(bs, host_cluster_offset,
> > + guest_cluster_offset + offset_in_cluster,
> >buffer, bytes) < 0) {
> >   return false;
> >   }
> > @@ -496,7 +496,7 @@ static int coroutine_fn 
> > do_perform_cow_write(BlockDriverState *bs,
> >   }
> >   
> >   ret = qcow2_pre_write_overlap_check(bs, 0,
> > -cluster_offset + offset_in_cluster, qiov->size, true);
> > +  cluster_offset + offset_in_cluster, qiov->size, true);
> 
> 
> Hmm, unrelated hunk.

I was asked to do this to fix coding style, so that wrapped line,
is 4 characters shifted to the right.

> 
> >   if (ret < 0) {
> >   return ret;
> >   }
> > diff --git a/block/qcow2-threads.c b/block/qcow2-threads.c
> > index 3b1e63fe41..c3cda0c6a5 100644
> > --- a/block/qcow2-threads.c
> > +++ b/block/qcow2-threads.c
> > @@ -234,15 +234,19 @@ static int qcow2_encdec_pool_func(void *opaque)
> >   }
> >   
> >   static int coroutine_fn
> > -qcow2_co_encdec(BlockDriverState *bs, uint64_t file_cluster_offset,
> > -  uint64_t offset, void *buf, size_t len, Qcow2EncDecFunc 
> > func)
> > +qcow2_co_encdec(BlockDriverState *bs, uint64_t host_cluster_offset,
> > +uint64_t guest_offset, void *buf, size_t len,
> > +Qcow2EncDecFunc func)
> >   {
> >   BDRVQcow2State *s = bs->opaque;
> > +
> > +uint64_t offset = s->crypt_physical_offset ?
> > +host_cluster_offset + offset_into_cluster(s, guest_offset) :
> > +guest_offset;
> > +
> >   Qcow2EncDecData arg = {
> >   .block = s->crypto,
> > -.offset = s->crypt_physical_offset ?
> > -  file_cluster_offset + offset_into_cluster(s, offset) 
> > :
> > -  offset,
> > +.offset = offset,
> >   .buf = buf,
> >   .len = len,
> >   .func = func,
> > @@ -251,18 +255,49 @@ qcow2_co_encdec(BlockDriverState *bs, uint64_t 
> > file_cluster_offset,
> >   return qcow2_co_process(bs, qcow2_encdec_pool_func, &arg);
> >   }
> >   
> > +
> > +/*
> > + * qcow2_co_encrypt()
> > + *
> > + * Encrypts one or more contiguous aligned sectors
> > + *
> > + * @host_cluster_offset - on disk offset of the first cluster in which
> > + * the encrypted data will be written
> 
> 
> It's not quite right, it's not on disk, but on .file child of qcow2 node, 
> which
> may be any other format or protocol node.. So, I called it 
> file_cluster_offset.
> But I'm OK with new naming anyway. And it may be better for encryption related
> logic..

Yes, the .file is the underlying storage for both qcow2 metadata and the data,
and it is unlikely be another qcow2 file. Usually it will be a raw file,
accessed with some protocol.
I will change the wording to not include the 'disk' word though.


To be really honest, the best naming here would be one that follows the virtual 
memory concepts.
A virtual block/cluster address and a physical block/cluster address.
However we talked with Kevin recently and I also studied quite a lot of qcow2 
code,
and the usual convention is guest cluster offset and host cluster offset,
and often guest off

Re: [Qemu-block] [PATCH v6 20/42] block/snapshot: Fix fallback

[Max Reitz]

On 10.09.19 13:56, Kevin Wolf wrote:
> Am 09.08.2019 um 18:13 hat Max Reitz geschrieben:
>> If the top node's driver does not provide snapshot functionality and we
>> want to fall back to a node down the chain, we need to snapshot all
>> non-COW children.  For simplicity's sake, just do not fall back if there
>> is more than one such child.
>>
>> bdrv_snapshot_goto() becomes a bit weird because we may have to redirect
>> the actual child pointer, so it only works if the fallback child is
>> bs->file or bs->backing (and then we have to find out which it is).
>>
>> Suggested-by: Vladimir Sementsov-Ogievskiy 
>> Signed-off-by: Max Reitz 
>> ---
>>  block/snapshot.c | 100 +--
>>  1 file changed, 79 insertions(+), 21 deletions(-)
>>
>> diff --git a/block/snapshot.c b/block/snapshot.c
>> index f2f48f926a..35403c167f 100644
>> --- a/block/snapshot.c
>> +++ b/block/snapshot.c
>> @@ -146,6 +146,32 @@ bool bdrv_snapshot_find_by_id_and_name(BlockDriverState 
>> *bs,
>>  return ret;
>>  }
>>  
>> +/**
>> + * Return the child BDS to which we can fall back if the given BDS
>> + * does not support snapshots.
>> + * Return NULL if there is no BDS to (safely) fall back to.
>> + */
>> +static BlockDriverState *bdrv_snapshot_fallback(BlockDriverState *bs)
>> +{
>> +BlockDriverState *child_bs = NULL;
>> +BdrvChild *child;
>> +
>> +QLIST_FOREACH(child, &bs->children, next) {
>> +if (child == bdrv_filtered_cow_child(bs)) {
>> +/* Ignore: COW children need not be included in snapshots */
>> +continue;
>> +}
>> +
>> +if (child_bs) {
>> +/* Cannot fall back to a single child if there are multiple */
>> +return NULL;
>> +}
>> +child_bs = child->bs;
>> +}
>> +
>> +return child_bs;
>> +}
> 
> Why do we return child->bs here when bdrv_snapshot_goto() then needs to
> reconstruct what the associated BdrvChild was? Wouldn't it make more
> sense to return BdrvChild** from here and maybe have a small wrapper for
> the other functions that only need a BDS?

What would you return instead?  &child doesn’t work.

We could limit ourselves to bs->file and bs->backing.  It just seemed
like a bit of an artificial limit to me, because we only really have it
for bdrv_snapshot_goto().

Max



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-block] [PATCH v6 20/42] block/snapshot: Fix fallback

[Kevin Wolf]

Am 09.08.2019 um 18:13 hat Max Reitz geschrieben:
> If the top node's driver does not provide snapshot functionality and we
> want to fall back to a node down the chain, we need to snapshot all
> non-COW children.  For simplicity's sake, just do not fall back if there
> is more than one such child.
> 
> bdrv_snapshot_goto() becomes a bit weird because we may have to redirect
> the actual child pointer, so it only works if the fallback child is
> bs->file or bs->backing (and then we have to find out which it is).
> 
> Suggested-by: Vladimir Sementsov-Ogievskiy 
> Signed-off-by: Max Reitz 
> ---
>  block/snapshot.c | 100 +--
>  1 file changed, 79 insertions(+), 21 deletions(-)
> 
> diff --git a/block/snapshot.c b/block/snapshot.c
> index f2f48f926a..35403c167f 100644
> --- a/block/snapshot.c
> +++ b/block/snapshot.c
> @@ -146,6 +146,32 @@ bool bdrv_snapshot_find_by_id_and_name(BlockDriverState 
> *bs,
>  return ret;
>  }
>  
> +/**
> + * Return the child BDS to which we can fall back if the given BDS
> + * does not support snapshots.
> + * Return NULL if there is no BDS to (safely) fall back to.
> + */
> +static BlockDriverState *bdrv_snapshot_fallback(BlockDriverState *bs)
> +{
> +BlockDriverState *child_bs = NULL;
> +BdrvChild *child;
> +
> +QLIST_FOREACH(child, &bs->children, next) {
> +if (child == bdrv_filtered_cow_child(bs)) {
> +/* Ignore: COW children need not be included in snapshots */
> +continue;
> +}
> +
> +if (child_bs) {
> +/* Cannot fall back to a single child if there are multiple */
> +return NULL;
> +}
> +child_bs = child->bs;
> +}
> +
> +return child_bs;
> +}

Why do we return child->bs here when bdrv_snapshot_goto() then needs to
reconstruct what the associated BdrvChild was? Wouldn't it make more
sense to return BdrvChild** from here and maybe have a small wrapper for
the other functions that only need a BDS?

Kevin

Re: [Qemu-block] [PATCH V3] block/vhdx: add check for truncated image files

[Peter Lieven]

Am 10.09.19 um 13:15 schrieb Kevin Wolf:

Am 05.09.2019 um 12:02 hat Peter Lieven geschrieben:

Am 04.09.19 um 16:09 schrieb Kevin Wolf:

Am 03.09.2019 um 15:35 hat Peter Lieven geschrieben:

qemu is currently not able to detect truncated vhdx image files.
Add a basic check if all allocated blocks are reachable at open and
report all errors during bdrv_co_check.

Signed-off-by: Peter Lieven 
---
V3: - check for bdrv_getlength failure [Kevin]
  - use uint32_t for i [Kevin]
  - check for BAT entry overflow [Kevin]
  - break on !errcnt in second check

V2: - add error reporting [Kevin]
  - use bdrv_getlength instead of bdrv_get_allocated_file_size [Kevin]
  - factor out BAT entry check and add error reporting for region
overlaps
  - already check on vhdx_open

Something still seems to be wrong with this patch:

  213  fail   [15:50:13] [15:50:14]  (last: 2s)output 
mismatch (see 213.out.bad)
  --- /home/kwolf/source/qemu/tests/qemu-iotests/213.out  2019-06-28 
14:19:50.065797707 +0200
  +++ /home/kwolf/source/qemu/tests/qemu-iotests/213.out.bad  
2019-09-04 15:50:14.582053976 +0200
  @@ -46,10 +46,8 @@
   {"execute": "job-dismiss", "arguments": {"id": "job0"}}
   {"return": {}}

  -image: TEST_IMG
  -file format: IMGFMT
  -virtual size: 32 MiB (33554432 bytes)
  -cluster_size: 268435456
  +qemu-img: VHDX BAT entry 0 offset points after end of file. Image has 
probably been truncated.
  +qemu-img: Could not open 'TEST_IMG': Could not open 'TEST_IMG': Invalid 
argument

   === Invalid BlockdevRef ===

I can reproduce this manually with the following qemu-img invocations.
It seems all three options must be given to reproduce the error:

  $ ./qemu-img create -f vhdx -o 
block_size=268435456,subformat=fixed,block_state_zero=off /tmp/test.vhdx 32M
  Formatting '/tmp/test.vhdx', fmt=vhdx size=33554432 log_size=1048576 
block_size=268435456 subformat=fixed block_state_zero=off
  $ ./qemu-img info /tmp/test.vhdx
  qemu-img: VHDX BAT entry 0 offset points after end of file. Image has 
probably been truncated.
  qemu-img: Could not open '/tmp/test.vhdx': Could not open 
'/tmp/test.vhdx': Invalid argument

If I add the offsets to the error message (would probably nice to have),
I get:

  qemu-img: VHDX BAT entry 0 offset 8388608 points after end of file 
(41943040). Image has probably been truncated.

So it seems that the file is large enough to hold 32M + metadata, but we
don't increase the file size to hold a full block (256M). Is this a
problem in the way we create images or are partial blocks at the end
expected?

Kevin


A short look into the VHDX spec [1] seems to suggest that a VHDX File
can only grow in Block increments.

See page 8 in the definition of blocks: "Allocation of new space for a
virtual hard disk that supports dynamic growth of the virtual hard
disk file is done in fixes size units defined as blocks."

Then I guess we need to fix the creation of VHDX images before we can
apply this patch because otherwise qemu-iotests fails.

Hm... And probably ignore the error for a partial final block anyway to
maintain compatibility with images created by older QEMU versions.



I would change the check to not fail for partial blocks at the end

of the image if the available bytes match the filesize.


I will send an update.


Peter

Re: [Qemu-block] [PATCH] blockjob: update nodes head while removing all bdrv

[Sergio Lopez]

Max Reitz  writes:

> On 10.09.19 13:07, Sergio Lopez wrote:
>> block_job_remove_all_bdrv() iterates through job->nodes, calling
>> bdrv_root_unref_child() for each entry. The call to the latter may
>> reach child_job_[can_]set_aio_ctx(), which will also attempt to
>> traverse job->nodes, potentially finding entries that where freed
>> on previous iterations.
>> 
>> To avoid this situation, update job->nodes head on each iteration to
>> ensure that already freed entries are no longer linked to the list.
>> 
>> RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1746631
>> Signed-off-by: Sergio Lopez 
>> ---
>>  blockjob.c | 6 ++
>>  1 file changed, 6 insertions(+)
>> 
>> diff --git a/blockjob.c b/blockjob.c
>> index 6e32d1a0c0..7b1551d981 100644
>> --- a/blockjob.c
>> +++ b/blockjob.c
>> @@ -192,6 +192,12 @@ void block_job_remove_all_bdrv(BlockJob *job)
>>  BdrvChild *c = l->data;
>>  bdrv_op_unblock_all(c->bs, job->blocker);
>>  bdrv_root_unref_child(c);
>> +/*
>> + * The call above may reach child_job_[can_]set_aio_ctx(), which 
>> will
>> + * also traverse job->nodes, so update the head here to make sure it
>> + * doesn't attempt to process an already freed BdrvChild.
>> + */
>> +job->nodes = l->next;
>>  }
>>  g_slist_free(job->nodes);
>
> But this will leak the whole list.

Ouch. This is what happens when you rush up things. I'll send a v2 ASAP.

Thanks,
Sergio.


signature.asc
Description: PGP signature

[Qemu-block] [PATCH v10 8/9] hw/m68k: add a dummy SWIM floppy controller

[Laurent Vivier]

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
Reviewed-by: Hervé Poussineau 
---
 MAINTAINERS |   2 +
 hw/block/Kconfig|   3 +
 hw/block/Makefile.objs  |   1 +
 hw/block/swim.c | 487 
 hw/m68k/Kconfig |   1 +
 include/hw/block/swim.h |  76 +++
 6 files changed, 570 insertions(+)
 create mode 100644 hw/block/swim.c
 create mode 100644 include/hw/block/swim.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 4f6b2b037a..f85f11d83c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -923,9 +923,11 @@ S: Maintained
 F: hw/misc/mac_via.c
 F: hw/display/macfb.c
 F: hw/nubus/*
+F: hw/block/swim.c
 F: include/hw/misc/mac_via.h
 F: include/hw/display/macfb.h
 F: include/hw/nubus/*
+F: include/hw/block/swim.h
 
 MicroBlaze Machines
 ---
diff --git a/hw/block/Kconfig b/hw/block/Kconfig
index df96dc5dcc..2d17f481ad 100644
--- a/hw/block/Kconfig
+++ b/hw/block/Kconfig
@@ -37,3 +37,6 @@ config VHOST_USER_BLK
 # Only PCI devices are provided for now
 default y if VIRTIO_PCI
 depends on VIRTIO && VHOST_USER && LINUX
+
+config SWIM
+bool
diff --git a/hw/block/Makefile.objs b/hw/block/Makefile.objs
index f5f643f0cc..28c2495a00 100644
--- a/hw/block/Makefile.objs
+++ b/hw/block/Makefile.objs
@@ -8,6 +8,7 @@ common-obj-$(CONFIG_XEN) += xen-block.o
 common-obj-$(CONFIG_ECC) += ecc.o
 common-obj-$(CONFIG_ONENAND) += onenand.o
 common-obj-$(CONFIG_NVME_PCI) += nvme.o
+common-obj-$(CONFIG_SWIM) += swim.o
 
 obj-$(CONFIG_SH4) += tc58128.o
 
diff --git a/hw/block/swim.c b/hw/block/swim.c
new file mode 100644
index 00..80addcea9d
--- /dev/null
+++ b/hw/block/swim.c
@@ -0,0 +1,487 @@
+/*
+ * QEMU Macintosh floppy disk controller emulator (SWIM)
+ *
+ * Copyright (c) 2014-2018 Laurent Vivier 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.  See
+ * the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/main-loop.h"
+#include "qapi/error.h"
+#include "sysemu/block-backend.h"
+#include "hw/sysbus.h"
+#include "migration/vmstate.h"
+#include "hw/block/block.h"
+#include "hw/block/swim.h"
+#include "hw/qdev-properties.h"
+
+/* IWM registers */
+
+#define IWM_PH0L0
+#define IWM_PH0H1
+#define IWM_PH1L2
+#define IWM_PH1H3
+#define IWM_PH2L4
+#define IWM_PH2H5
+#define IWM_PH3L6
+#define IWM_PH3H7
+#define IWM_MTROFF  8
+#define IWM_MTRON   9
+#define IWM_INTDRIVE10
+#define IWM_EXTDRIVE11
+#define IWM_Q6L 12
+#define IWM_Q6H 13
+#define IWM_Q7L 14
+#define IWM_Q7H 15
+
+/* SWIM registers */
+
+#define SWIM_WRITE_DATA 0
+#define SWIM_WRITE_MARK 1
+#define SWIM_WRITE_CRC  2
+#define SWIM_WRITE_PARAMETER3
+#define SWIM_WRITE_PHASE4
+#define SWIM_WRITE_SETUP5
+#define SWIM_WRITE_MODE06
+#define SWIM_WRITE_MODE17
+
+#define SWIM_READ_DATA  8
+#define SWIM_READ_MARK  9
+#define SWIM_READ_ERROR 10
+#define SWIM_READ_PARAMETER 11
+#define SWIM_READ_PHASE 12
+#define SWIM_READ_SETUP 13
+#define SWIM_READ_STATUS14
+#define SWIM_READ_HANDSHAKE 15
+
+#define REG_SHIFT   9
+
+#define SWIM_MODE_IWM  0
+#define SWIM_MODE_SWIM 1
+
+/* bits in phase register */
+
+#define SWIM_SEEK_NEGATIVE   0x074
+#define SWIM_STEP0x071
+#define SWIM_MOTOR_ON0x072
+#define SWIM_MOTOR_OFF   0x076
+#define SWIM_INDEX   0x073
+#define SWIM_EJECT   0x077
+#define SWIM_SETMFM  0x171
+#define SWIM_SETGCR  0x175
+#define SWIM_RELAX   0x033
+#define SWIM_LSTRB   0x008
+#define SWIM_CA_MASK 0x077
+
+/* Select values for swim_select and swim_readbit */
+
+#define SWIM_READ_DATA_0 0x074
+#define SWIM_TWOMEG_DRIVE0x075
+#define SWIM_SINGLE_SIDED0x076
+#define SWIM_DRIVE_PRESENT   0x077
+#define SWIM_DISK_IN 0x170
+#define SWIM_WRITE_PROT  0x171
+#define SWIM_TRACK_ZERO  0x172
+#define SWIM_TACHO   0x173
+#define SWIM_READ_DATA_1 0x174
+#define SWIM_MFM_MODE0x175
+#define SWIM_SEEK_COMPLETE   0x176
+#define SWIM_ONEMEG_MEDIA0x177
+
+/* Bits in handshake register */
+
+#define SWIM_MARK_BYTE   0x01
+#define SWIM_CRC_ZERO0x02
+#define SWIM_RDDATA  0x04
+#define SWIM_SENSE   0x08
+#define SWIM_MOTEN   0x10
+#define SWIM_ERROR   0x20
+#define SWIM_DAT2BYTE0x40
+#define SWIM_DAT1BYTE0x80
+
+/* bits in setup register */
+
+#define SWIM_S_INV_WDATA 0x01
+#define SWIM_S_3_5_SELECT0x02
+#define SWIM_S_GCR   0x04
+#define SWIM_S_FCLK_DIV2 0x08
+#define SWIM_S_ERROR_CORR0x10
+#define SWIM_S_IBM_DR

Re: [Qemu-block] [PATCH v6 04/42] block: Add child access functions

[Max Reitz]

On 10.09.19 12:47, Kevin Wolf wrote:
> Am 10.09.2019 um 11:14 hat Max Reitz geschrieben:
>> On 09.09.19 18:13, Kevin Wolf wrote:
>>> Am 09.09.2019 um 16:04 hat Max Reitz geschrieben:
 On 09.09.19 11:36, Kevin Wolf wrote:
> Am 09.09.2019 um 09:56 hat Max Reitz geschrieben:
>> On 04.09.19 18:16, Kevin Wolf wrote:
>>> Am 09.08.2019 um 18:13 hat Max Reitz geschrieben:
 There are BDS children that the general block layer code can access,
 namely bs->file and bs->backing.  Since the introduction of filters and
 external data files, their meaning is not quite clear.  bs->backing can
 be a COW source, or it can be an R/W-filtered child; bs->file can be an
 R/W-filtered child, it can be data and metadata storage, or it can be
 just metadata storage.

 This overloading really is not helpful.  This patch adds function that
 retrieve the correct child for each exact purpose.  Later patches in
 this series will make use of them.  Doing so will allow us to handle
 filter nodes and external data files in a meaningful way.

 Signed-off-by: Max Reitz 
 Reviewed-by: Vladimir Sementsov-Ogievskiy 
>>>
>>> Each time I look at this patch, I'm confused by the function names.
>>> Maybe I should just ask what the idea there was, or more specifically:
>>> What does the "filtered" in "filtered child" really mean?
>>>
>>> Apparently any child of a filter node is "filtered" (which makes sense),
>>
>> It isn’t, filters can have non-filter children.  For example, backup-top
>> could have the source as a filtered child and the target as a non-filter
>> child.
>
> Hm, okay, makes sense. I had a definition in mind that says that filter
> nodes only have a single child node. Is it that a filter may have only a
> single _filtered_ child node?

 Well, there’s Quorum...
>>>
>>> Ah, nice, quorum sets is_filter = true even though it neither fulfulls
>>> the conditions for it before this series, nor the changed conditions
>>> after this series.
>>>
>>> So either quorum lies and isn't actually a filter driver, or our
>>> definition in the documentation of is_filter is wrong.
>>
>> You could say it lies because in FIFO mode it clearly isn’t a filter for
>> all of its children.
>>
>> There is a reason for lying, though, which is
>> bdrv_recurse_is_first_non_filter(), which is necessary to use the whole
>> to_replace mirror stuff.
> 
> Hm, actually, now that you mention bdrv_recurse_is_first_non_filter(),
> quorum was the first driver to declare itself a filter, so strictly
> speaking, if there is an inconsistency, it's the other uses that are
> abusing the field...
> 
>> (You mirror from a quorum with a failed child and then replace the
>> failed child.  mirror needs to ensure that there are only R/W filters
>> between the child and the mirror source so that replacing it will not
>> suddenly change any visible data.  Which is actually a lie for quorum,
>> because the child is clearly broken and thus precisely doesn’t show the
>> same data...)
>>
>> Maybe we should stop declaring Quorum a filter and then rename the
>> bdrv_recurse_is_first_non_filter() to, I don’t know,
>> bdrv_recurse_can_be_replaced_by_mirror()?
> 
> Why not.

It feels difficult to do in this series because this is a whole new can
of worms.

In patch 35, I actually replace the mirror use case by
is_filtered_child().  So it looks to me as if that should not be done,
because I should instead fix bdrv_recurse_is_first_non_filter() (and
rename it), because quorum does allow replacing its children by mirror,
even if it does not act as a filter for them.

OTOH, there are other users of bdrv_is_first_non_filter().  Those are
qmp_block_resize() and external_snapshot_prepare(), who throw an error
if that returns false.

I think that’s just wrong.  First of all, I don’t even know why we have
that restriction anymore (I can imagine why it used to make sense before
the permission system).  qmp_block_resize() should always work as long
as it can get BLK_PERM_RESIZE; and I don’t know why the parents of some
node would care if you take a snapshot of their child.

>>> but also bs->backing of a qcow2 image, while bs->file of qcow2 isn't.
>>> raw doesn't have any "filtered" child. What's the system behind this?
>>
>> “filtered” means: If the parent node returns data from this child, it
>> won’t modify it, neither its content nor its position.  COW and R/W
>> filters differ in how they handle writes; R/W filters pass them through
>> to the filtered child, COW filters copy them off to some other child
>> node (and then the filtered child’s data will no longer be visible at
>> that location).
>
> But there is no reason why a node couldn't fulfill this condition for
> more than one child node. bdrv_filtered_child() isn't well-defined then.
> Technically, the descr

Re: [Qemu-block] [PATCH] blockjob: update nodes head while removing all bdrv

[Max Reitz]

On 10.09.19 13:07, Sergio Lopez wrote:
> block_job_remove_all_bdrv() iterates through job->nodes, calling
> bdrv_root_unref_child() for each entry. The call to the latter may
> reach child_job_[can_]set_aio_ctx(), which will also attempt to
> traverse job->nodes, potentially finding entries that where freed
> on previous iterations.
> 
> To avoid this situation, update job->nodes head on each iteration to
> ensure that already freed entries are no longer linked to the list.
> 
> RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1746631
> Signed-off-by: Sergio Lopez 
> ---
>  blockjob.c | 6 ++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/blockjob.c b/blockjob.c
> index 6e32d1a0c0..7b1551d981 100644
> --- a/blockjob.c
> +++ b/blockjob.c
> @@ -192,6 +192,12 @@ void block_job_remove_all_bdrv(BlockJob *job)
>  BdrvChild *c = l->data;
>  bdrv_op_unblock_all(c->bs, job->blocker);
>  bdrv_root_unref_child(c);
> +/*
> + * The call above may reach child_job_[can_]set_aio_ctx(), which will
> + * also traverse job->nodes, so update the head here to make sure it
> + * doesn't attempt to process an already freed BdrvChild.
> + */
> +job->nodes = l->next;
>  }
>  g_slist_free(job->nodes);

But this will leak the whole list.

Max

>  job->nodes = NULL;
> 




signature.asc
Description: OpenPGP digital signature

[Qemu-block] [PATCH v10 4/9] hw/m68k: implement ADB bus support for via

[Laurent Vivier]

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
Reviewed-by: Hervé Poussineau 
Reviewed-by: Thomas Huth 
---
 hw/misc/Kconfig   |   1 +
 hw/misc/mac_via.c | 198 +-
 include/hw/misc/mac_via.h |   7 ++
 3 files changed, 205 insertions(+), 1 deletion(-)

diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig
index 18a5dc9c09..2164646553 100644
--- a/hw/misc/Kconfig
+++ b/hw/misc/Kconfig
@@ -123,5 +123,6 @@ config UNIMP
 config MAC_VIA
 bool
 select MOS6522
+select ADB
 
 source macio/Kconfig
diff --git a/hw/misc/mac_via.c b/hw/misc/mac_via.c
index a052259613..2f54db9e1e 100644
--- a/hw/misc/mac_via.c
+++ b/hw/misc/mac_via.c
@@ -264,10 +264,16 @@
  * Table 19-10 ADB transaction states
  */
 
+#define ADB_STATE_NEW   0
+#define ADB_STATE_EVEN  1
+#define ADB_STATE_ODD   2
+#define ADB_STATE_IDLE  3
+
 #define VIA1B_vADB_StateMask(VIA1B_vADBS1 | VIA1B_vADBS2)
 #define VIA1B_vADB_StateShift   4
 
 #define VIA_TIMER_FREQ (783360)
+#define VIA_ADB_POLL_FREQ 50 /* XXX: not real */
 
 /* VIA returns time offset from Jan 1, 1904, not 1970 */
 #define RTC_OFFSET 2082844800
@@ -449,6 +455,181 @@ static void via1_rtc_update(MacVIAState *m)
 }
 }
 
+static int adb_via_poll(MacVIAState *s, int state, uint8_t *data)
+{
+if (state != ADB_STATE_IDLE) {
+return 0;
+}
+
+if (s->adb_data_in_size < s->adb_data_in_index) {
+return 0;
+}
+
+if (s->adb_data_out_index != 0) {
+return 0;
+}
+
+s->adb_data_in_index = 0;
+s->adb_data_out_index = 0;
+s->adb_data_in_size = adb_poll(&s->adb_bus, s->adb_data_in, 0x);
+
+if (s->adb_data_in_size) {
+*data = s->adb_data_in[s->adb_data_in_index++];
+qemu_irq_raise(s->adb_data_ready);
+}
+
+return s->adb_data_in_size;
+}
+
+static int adb_via_send(MacVIAState *s, int state, uint8_t data)
+{
+switch (state) {
+case ADB_STATE_NEW:
+s->adb_data_out_index = 0;
+break;
+case ADB_STATE_EVEN:
+if ((s->adb_data_out_index & 1) == 0) {
+return 0;
+}
+break;
+case ADB_STATE_ODD:
+if (s->adb_data_out_index & 1) {
+return 0;
+}
+break;
+case ADB_STATE_IDLE:
+return 0;
+}
+
+assert(s->adb_data_out_index < sizeof(s->adb_data_out) - 1);
+
+s->adb_data_out[s->adb_data_out_index++] = data;
+qemu_irq_raise(s->adb_data_ready);
+return 1;
+}
+
+static int adb_via_receive(MacVIAState *s, int state, uint8_t *data)
+{
+switch (state) {
+case ADB_STATE_NEW:
+return 0;
+
+case ADB_STATE_EVEN:
+if (s->adb_data_in_size <= 0) {
+qemu_irq_raise(s->adb_data_ready);
+return 0;
+}
+
+if (s->adb_data_in_index >= s->adb_data_in_size) {
+*data = 0;
+qemu_irq_raise(s->adb_data_ready);
+return 1;
+}
+
+if ((s->adb_data_in_index & 1) == 0) {
+return 0;
+}
+
+break;
+
+case ADB_STATE_ODD:
+if (s->adb_data_in_size <= 0) {
+qemu_irq_raise(s->adb_data_ready);
+return 0;
+}
+
+if (s->adb_data_in_index >= s->adb_data_in_size) {
+*data = 0;
+qemu_irq_raise(s->adb_data_ready);
+return 1;
+}
+
+if (s->adb_data_in_index & 1) {
+return 0;
+}
+
+break;
+
+case ADB_STATE_IDLE:
+if (s->adb_data_out_index == 0) {
+return 0;
+}
+
+s->adb_data_in_size = adb_request(&s->adb_bus, s->adb_data_in,
+  s->adb_data_out,
+  s->adb_data_out_index);
+s->adb_data_out_index = 0;
+s->adb_data_in_index = 0;
+if (s->adb_data_in_size < 0) {
+*data = 0xff;
+qemu_irq_raise(s->adb_data_ready);
+return -1;
+}
+
+if (s->adb_data_in_size == 0) {
+return 0;
+}
+
+break;
+}
+
+assert(s->adb_data_in_index < sizeof(s->adb_data_in) - 1);
+
+*data = s->adb_data_in[s->adb_data_in_index++];
+qemu_irq_raise(s->adb_data_ready);
+if (*data == 0xff || *data == 0) {
+return 0;
+}
+return 1;
+}
+
+static void via1_adb_update(MacVIAState *m)
+{
+MOS6522Q800VIA1State *v1s = MOS6522_Q800_VIA1(&m->mos6522_via1);
+MOS6522State *s = MOS6522(v1s);
+int state;
+int ret;
+
+state = (s->b & VIA1B_vADB_StateMask) >> VIA1B_vADB_StateShift;
+
+if (s->acr & VIA1ACR_vShiftOut) {
+/* output mode */
+ret = adb_via_send(m, state, s->sr);
+if (ret > 0) {
+s->b &= ~VIA1B_vADBInt;
+} else {
+s->b |= VIA1B_vADBInt;
+}
+} else {
+/* input mode */
+ret = adb_via_receive(m, state, &s->sr);
+if (ret > 0 && s->sr != 

[Qemu-block] [PATCH v10 7/9] hw/m68k: add Nubus support for macfb video card

[Laurent Vivier]

From: Mark Cave-Ayland 

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
Reviewed-by: Hervé Poussineau 
---
 hw/display/Kconfig |  1 +
 hw/display/macfb.c | 56 ++
 include/hw/display/macfb.h | 21 ++
 3 files changed, 78 insertions(+)

diff --git a/hw/display/Kconfig b/hw/display/Kconfig
index 32e8d29003..c500d1fc6d 100644
--- a/hw/display/Kconfig
+++ b/hw/display/Kconfig
@@ -136,3 +136,4 @@ config ATI_VGA
 config MACFB
 bool
 select FRAMEBUFFER
+depends on NUBUS
diff --git a/hw/display/macfb.c b/hw/display/macfb.c
index d7c5ef296e..f4fa8e3206 100644
--- a/hw/display/macfb.c
+++ b/hw/display/macfb.c
@@ -15,6 +15,7 @@
 #include "hw/sysbus.h"
 #include "ui/console.h"
 #include "ui/pixel_ops.h"
+#include "hw/nubus/nubus.h"
 #include "hw/display/macfb.h"
 #include "qapi/error.h"
 #include "hw/qdev-properties.h"
@@ -382,12 +383,38 @@ static void macfb_sysbus_realize(DeviceState *dev, Error 
**errp)
 sysbus_init_mmio(SYS_BUS_DEVICE(s), &ms->mem_vram);
 }
 
+const uint8_t macfb_rom[] = {
+255, 0, 0, 0,
+};
+
+static void macfb_nubus_realize(DeviceState *dev, Error **errp)
+{
+NubusDevice *nd = NUBUS_DEVICE(dev);
+MacfbNubusState *s = NUBUS_MACFB(dev);
+MacfbNubusDeviceClass *ndc = MACFB_NUBUS_GET_CLASS(dev);
+MacfbState *ms = &s->macfb;
+
+ndc->parent_realize(dev, errp);
+
+macfb_common_realize(dev, ms, errp);
+memory_region_add_subregion(&nd->slot_mem, DAFB_BASE, &ms->mem_ctrl);
+memory_region_add_subregion(&nd->slot_mem, VIDEO_BASE, &ms->mem_vram);
+
+nubus_register_rom(nd, macfb_rom, sizeof(macfb_rom), 1, 9, 0xf);
+}
+
 static void macfb_sysbus_reset(DeviceState *d)
 {
 MacfbSysBusState *s = MACFB(d);
 macfb_reset(&s->macfb);
 }
 
+static void macfb_nubus_reset(DeviceState *d)
+{
+MacfbNubusState *s = NUBUS_MACFB(d);
+macfb_reset(&s->macfb);
+}
+
 static Property macfb_sysbus_properties[] = {
 DEFINE_PROP_UINT32("width", MacfbSysBusState, macfb.width, 640),
 DEFINE_PROP_UINT32("height", MacfbSysBusState, macfb.height, 480),
@@ -395,6 +422,13 @@ static Property macfb_sysbus_properties[] = {
 DEFINE_PROP_END_OF_LIST(),
 };
 
+static Property macfb_nubus_properties[] = {
+DEFINE_PROP_UINT32("width", MacfbNubusState, macfb.width, 640),
+DEFINE_PROP_UINT32("height", MacfbNubusState, macfb.height, 480),
+DEFINE_PROP_UINT8("depth", MacfbNubusState, macfb.depth, 8),
+DEFINE_PROP_END_OF_LIST(),
+};
+
 static void macfb_sysbus_class_init(ObjectClass *klass, void *data)
 {
 DeviceClass *dc = DEVICE_CLASS(klass);
@@ -406,6 +440,19 @@ static void macfb_sysbus_class_init(ObjectClass *klass, 
void *data)
 dc->props = macfb_sysbus_properties;
 }
 
+static void macfb_nubus_class_init(ObjectClass *klass, void *data)
+{
+DeviceClass *dc = DEVICE_CLASS(klass);
+MacfbNubusDeviceClass *ndc = MACFB_NUBUS_DEVICE_CLASS(klass);
+
+device_class_set_parent_realize(dc, macfb_nubus_realize,
+&ndc->parent_realize);
+dc->desc = "Nubus Macintosh framebuffer";
+dc->reset = macfb_nubus_reset;
+dc->vmsd = &vmstate_macfb;
+dc->props = macfb_nubus_properties;
+}
+
 static TypeInfo macfb_sysbus_info = {
 .name  = TYPE_MACFB,
 .parent= TYPE_SYS_BUS_DEVICE,
@@ -413,9 +460,18 @@ static TypeInfo macfb_sysbus_info = {
 .class_init= macfb_sysbus_class_init,
 };
 
+static TypeInfo macfb_nubus_info = {
+.name  = TYPE_NUBUS_MACFB,
+.parent= TYPE_NUBUS_DEVICE,
+.instance_size = sizeof(MacfbNubusState),
+.class_init= macfb_nubus_class_init,
+.class_size= sizeof(MacfbNubusDeviceClass),
+};
+
 static void macfb_register_types(void)
 {
 type_register_static(&macfb_sysbus_info);
+type_register_static(&macfb_nubus_info);
 }
 
 type_init(macfb_register_types)
diff --git a/include/hw/display/macfb.h b/include/hw/display/macfb.h
index 3fe2592735..26367ae2c4 100644
--- a/include/hw/display/macfb.h
+++ b/include/hw/display/macfb.h
@@ -40,4 +40,25 @@ typedef struct {
 MacfbState macfb;
 } MacfbSysBusState;
 
+#define MACFB_NUBUS_DEVICE_CLASS(class) \
+OBJECT_CLASS_CHECK(MacfbNubusDeviceClass, (class), TYPE_NUBUS_MACFB)
+#define MACFB_NUBUS_GET_CLASS(obj) \
+OBJECT_GET_CLASS(MacfbNubusDeviceClass, (obj), TYPE_NUBUS_MACFB)
+
+typedef struct MacfbNubusDeviceClass {
+DeviceClass parent_class;
+
+DeviceRealize parent_realize;
+} MacfbNubusDeviceClass;
+
+#define TYPE_NUBUS_MACFB "nubus-macfb"
+#define NUBUS_MACFB(obj) \
+OBJECT_CHECK(MacfbNubusState, (obj), TYPE_NUBUS_MACFB)
+
+typedef struct {
+NubusDevice busdev;
+
+MacfbState macfb;
+} MacfbNubusState;
+
 #endif
-- 
2.21.0

[Qemu-block] [PATCH v10 5/9] hw/m68k: add macfb video card

[Laurent Vivier]

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
Reviewed-by: Hervé Poussineau 
Reviewed-by: Thomas Huth 
---
 MAINTAINERS|   2 +
 arch_init.c|   4 +
 hw/display/Kconfig |   4 +
 hw/display/Makefile.objs   |   1 +
 hw/display/macfb.c | 421 +
 hw/m68k/Kconfig|   1 +
 include/hw/display/macfb.h |  43 
 qemu-options.hx|   2 +-
 vl.c   |   3 +-
 9 files changed, 479 insertions(+), 2 deletions(-)
 create mode 100644 hw/display/macfb.c
 create mode 100644 include/hw/display/macfb.h

diff --git a/MAINTAINERS b/MAINTAINERS
index b01826ba39..6b5fc50aef 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -921,7 +921,9 @@ q800
 M: Laurent Vivier 
 S: Maintained
 F: hw/misc/mac_via.c
+F: hw/display/macfb.c
 F: include/hw/misc/mac_via.h
+F: include/hw/display/macfb.h
 
 MicroBlaze Machines
 ---
diff --git a/arch_init.c b/arch_init.c
index 0a1531124c..705d0b94ad 100644
--- a/arch_init.c
+++ b/arch_init.c
@@ -38,6 +38,10 @@
 int graphic_width = 1024;
 int graphic_height = 768;
 int graphic_depth = 8;
+#elif defined(TARGET_M68K)
+int graphic_width = 800;
+int graphic_height = 600;
+int graphic_depth = 8;
 #else
 int graphic_width = 800;
 int graphic_height = 600;
diff --git a/hw/display/Kconfig b/hw/display/Kconfig
index cbdf7b1a67..32e8d29003 100644
--- a/hw/display/Kconfig
+++ b/hw/display/Kconfig
@@ -132,3 +132,7 @@ config ATI_VGA
 select VGA
 select BITBANG_I2C
 select DDC
+
+config MACFB
+bool
+select FRAMEBUFFER
diff --git a/hw/display/Makefile.objs b/hw/display/Makefile.objs
index 5a4066383b..f2182e3bef 100644
--- a/hw/display/Makefile.objs
+++ b/hw/display/Makefile.objs
@@ -26,6 +26,7 @@ common-obj-$(CONFIG_EXYNOS4) += exynos4210_fimd.o
 common-obj-$(CONFIG_FRAMEBUFFER) += framebuffer.o
 obj-$(CONFIG_MILKYMIST) += milkymist-vgafb.o
 common-obj-$(CONFIG_ZAURUS) += tc6393xb.o
+common-obj-$(CONFIG_MACFB) += macfb.o
 
 obj-$(CONFIG_MILKYMIST_TMU2) += milkymist-tmu2.o
 milkymist-tmu2.o-cflags := $(X11_CFLAGS) $(OPENGL_CFLAGS)
diff --git a/hw/display/macfb.c b/hw/display/macfb.c
new file mode 100644
index 00..d7c5ef296e
--- /dev/null
+++ b/hw/display/macfb.c
@@ -0,0 +1,421 @@
+/*
+ * QEMU Motorola 680x0 Macintosh Video Card Emulation
+ * Copyright (c) 2012-2018 Laurent Vivier
+ *
+ * some parts from QEMU G364 framebuffer Emulator.
+ * Copyright (c) 2007-2011 Herve Poussineau
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/units.h"
+#include "hw/sysbus.h"
+#include "ui/console.h"
+#include "ui/pixel_ops.h"
+#include "hw/display/macfb.h"
+#include "qapi/error.h"
+#include "hw/qdev-properties.h"
+#include "migration/vmstate.h"
+
+#define VIDEO_BASE 0x1000
+#define DAFB_BASE  0x0080
+
+#define MACFB_PAGE_SIZE 4096
+#define MACFB_VRAM_SIZE (4 * MiB)
+
+#define DAFB_RESET  0x200
+#define DAFB_LUT0x213
+
+
+typedef void macfb_draw_line_func(MacfbState *s, uint8_t *d, uint32_t addr,
+  int width);
+
+static inline uint8_t macfb_read_byte(MacfbState *s, uint32_t addr)
+{
+return s->vram[addr & s->vram_bit_mask];
+}
+
+/* 1-bit color */
+static void macfb_draw_line1(MacfbState *s, uint8_t *d, uint32_t addr,
+ int width)
+{
+uint8_t r, g, b;
+int x;
+
+for (x = 0; x < width; x++) {
+int bit = x & 7;
+int idx = (macfb_read_byte(s, addr) >> (7 - bit)) & 1;
+r = g = b  = ((1 - idx) << 7);
+addr += (bit == 7);
+
+*(uint32_t *)d = rgb_to_pixel32(r, g, b);
+d += 4;
+}
+}
+
+/* 2-bit color */
+static void macfb_draw_line2(MacfbState *s, uint8_t *d, uint32_t addr,
+ int width)
+{
+uint8_t r, g, b;
+int x;
+
+for (x = 0; x < width; x++) {
+int bit = (x & 3);
+int idx = (macfb_read_byte(s, addr) >> ((3 - bit) << 1)) & 3;
+r = s->color_palette[idx * 3];
+g = s->color_palette[idx * 3 + 1];
+b = s->color_palette[idx * 3 + 2];
+addr += (bit == 3);
+
+*(uint32_t *)d = rgb_to_pixel32(r, g, b);
+d += 4;
+}
+}
+
+/* 4-bit color */
+static void macfb_draw_line4(MacfbState *s, uint8_t *d, uint32_t addr,
+ int width)
+{
+uint8_t r, g, b;
+int x;
+
+for (x = 0; x < width; x++) {
+int bit = x & 1;
+int idx = (macfb_read_byte(s, addr) >> ((1 - bit) << 2)) & 15;
+r = s->color_palette[idx * 3];
+g = s->color_palette[idx * 3 + 1];
+b = s->color_palette[idx * 3 + 2];
+addr += (bit == 1);
+
+*(uint32_t *)d = rgb_to_pixel32(r, g, b);
+d += 4;
+}
+}
+
+/* 8-bit color */
+static void macfb_draw_line8(MacfbState *s, uint8_

[Qemu-block] [PATCH v10 2/9] dp8393x: manage big endian bus

[Laurent Vivier]

This is needed by Quadra 800, this card can run on little-endian
or big-endian bus.

Signed-off-by: Laurent Vivier 
Tested-by: Hervé Poussineau 
Reviewed-by: Philippe Mathieu-Daudé 
Reviewed-by: Hervé Poussineau 
---
 hw/net/dp8393x.c | 88 +++-
 1 file changed, 57 insertions(+), 31 deletions(-)

diff --git a/hw/net/dp8393x.c b/hw/net/dp8393x.c
index a5678e11fa..693e244ce6 100644
--- a/hw/net/dp8393x.c
+++ b/hw/net/dp8393x.c
@@ -153,6 +153,7 @@ typedef struct dp8393xState {
 
 /* Hardware */
 uint8_t it_shift;
+bool big_endian;
 qemu_irq irq;
 #ifdef DEBUG_SONIC
 int irq_level;
@@ -223,6 +224,29 @@ static uint32_t dp8393x_wt(dp8393xState *s)
 return s->regs[SONIC_WT1] << 16 | s->regs[SONIC_WT0];
 }
 
+static uint16_t dp8393x_get(dp8393xState *s, int width, uint16_t *base,
+int offset)
+{
+uint16_t val;
+
+if (s->big_endian) {
+val = be16_to_cpu(base[offset * width + width - 1]);
+} else {
+val = le16_to_cpu(base[offset * width]);
+}
+return val;
+}
+
+static void dp8393x_put(dp8393xState *s, int width, uint16_t *base, int offset,
+uint16_t val)
+{
+if (s->big_endian) {
+base[offset * width + width - 1] = cpu_to_be16(val);
+} else {
+base[offset * width] = cpu_to_le16(val);
+}
+}
+
 static void dp8393x_update_irq(dp8393xState *s)
 {
 int level = (s->regs[SONIC_IMR] & s->regs[SONIC_ISR]) ? 1 : 0;
@@ -254,12 +278,12 @@ static void dp8393x_do_load_cam(dp8393xState *s)
 /* Fill current entry */
 address_space_rw(&s->as, dp8393x_cdp(s),
 MEMTXATTRS_UNSPECIFIED, (uint8_t *)data, size, 0);
-s->cam[index][0] = data[1 * width] & 0xff;
-s->cam[index][1] = data[1 * width] >> 8;
-s->cam[index][2] = data[2 * width] & 0xff;
-s->cam[index][3] = data[2 * width] >> 8;
-s->cam[index][4] = data[3 * width] & 0xff;
-s->cam[index][5] = data[3 * width] >> 8;
+s->cam[index][0] = dp8393x_get(s, width, data, 1) & 0xff;
+s->cam[index][1] = dp8393x_get(s, width, data, 1) >> 8;
+s->cam[index][2] = dp8393x_get(s, width, data, 2) & 0xff;
+s->cam[index][3] = dp8393x_get(s, width, data, 2) >> 8;
+s->cam[index][4] = dp8393x_get(s, width, data, 3) & 0xff;
+s->cam[index][5] = dp8393x_get(s, width, data, 3) >> 8;
 DPRINTF("load cam[%d] with %02x%02x%02x%02x%02x%02x\n", index,
 s->cam[index][0], s->cam[index][1], s->cam[index][2],
 s->cam[index][3], s->cam[index][4], s->cam[index][5]);
@@ -272,7 +296,7 @@ static void dp8393x_do_load_cam(dp8393xState *s)
 /* Read CAM enable */
 address_space_rw(&s->as, dp8393x_cdp(s),
 MEMTXATTRS_UNSPECIFIED, (uint8_t *)data, size, 0);
-s->regs[SONIC_CE] = data[0 * width];
+s->regs[SONIC_CE] = dp8393x_get(s, width, data, 0);
 DPRINTF("load cam done. cam enable mask 0x%04x\n", s->regs[SONIC_CE]);
 
 /* Done */
@@ -293,10 +317,10 @@ static void dp8393x_do_read_rra(dp8393xState *s)
 MEMTXATTRS_UNSPECIFIED, (uint8_t *)data, size, 0);
 
 /* Update SONIC registers */
-s->regs[SONIC_CRBA0] = data[0 * width];
-s->regs[SONIC_CRBA1] = data[1 * width];
-s->regs[SONIC_RBWC0] = data[2 * width];
-s->regs[SONIC_RBWC1] = data[3 * width];
+s->regs[SONIC_CRBA0] = dp8393x_get(s, width, data, 0);
+s->regs[SONIC_CRBA1] = dp8393x_get(s, width, data, 1);
+s->regs[SONIC_RBWC0] = dp8393x_get(s, width, data, 2);
+s->regs[SONIC_RBWC1] = dp8393x_get(s, width, data, 3);
 DPRINTF("CRBA0/1: 0x%04x/0x%04x, RBWC0/1: 0x%04x/0x%04x\n",
 s->regs[SONIC_CRBA0], s->regs[SONIC_CRBA1],
 s->regs[SONIC_RBWC0], s->regs[SONIC_RBWC1]);
@@ -411,12 +435,12 @@ static void dp8393x_do_transmit_packets(dp8393xState *s)
 tx_len = 0;
 
 /* Update registers */
-s->regs[SONIC_TCR] = data[0 * width] & 0xf000;
-s->regs[SONIC_TPS] = data[1 * width];
-s->regs[SONIC_TFC] = data[2 * width];
-s->regs[SONIC_TSA0] = data[3 * width];
-s->regs[SONIC_TSA1] = data[4 * width];
-s->regs[SONIC_TFS] = data[5 * width];
+s->regs[SONIC_TCR] = dp8393x_get(s, width, data, 0) & 0xf000;
+s->regs[SONIC_TPS] = dp8393x_get(s, width, data, 1);
+s->regs[SONIC_TFC] = dp8393x_get(s, width, data, 2);
+s->regs[SONIC_TSA0] = dp8393x_get(s, width, data, 3);
+s->regs[SONIC_TSA1] = dp8393x_get(s, width, data, 4);
+s->regs[SONIC_TFS] = dp8393x_get(s, width, data, 5);
 
 /* Handle programmable interrupt */
 if (s->regs[SONIC_TCR] & SONIC_TCR_PINT) {
@@ -442,9 +466,9 @@ static void dp8393x_do_transmit_packets(dp8393xState *s)
 address_space_rw(&s->as,
 dp8393x_ttda(s) + sizeof(uint16_t) * (4 + 3 * i) * width,
 MEMTXATTRS_UNSPECIFIED, (uint8_t *)data, size, 0);
-s->regs[SONIC_TSA0

[Qemu-block] [PATCH v10 9/9] hw/m68k: define Macintosh Quadra 800

[Laurent Vivier]

If you want to test the machine, it doesn't yet boot a MacROM, but you can
boot a linux kernel from the command line.

You can install your own disk using debian-installer with:

./qemu-system-m68k \
-M q800 \
-serial none -serial mon:stdio \
-m 1000M -drive file=m68k.qcow2,format=qcow2 \
-net nic,model=dp83932,addr=09:00:07:12:34:57 \
-append "console=ttyS0 vga=off" \
-kernel vmlinux-4.15.0-2-m68k \
-initrd initrd.gz \
-drive file=debian-9.0-m68k-NETINST-1.iso \
-drive file=m68k.qcow2,format=qcow2 \
-nographic

If you use a graphic adapter instead of "-nographic", you can use "-g" to set 
the
size of the display (I use "-g 1600x800x24").

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
---
 MAINTAINERS   |   2 +
 hw/m68k/Kconfig   |   3 +
 hw/m68k/Makefile.objs |   1 +
 hw/m68k/bootinfo.h| 114 +
 hw/m68k/q800.c| 382 ++
 5 files changed, 502 insertions(+)
 create mode 100644 hw/m68k/bootinfo.h
 create mode 100644 hw/m68k/q800.c

diff --git a/MAINTAINERS b/MAINTAINERS
index f85f11d83c..e6d37acb84 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -920,10 +920,12 @@ F: include/hw/m68k/next-cube.h
 q800
 M: Laurent Vivier 
 S: Maintained
+F: hw/m68k/q800.c
 F: hw/misc/mac_via.c
 F: hw/display/macfb.c
 F: hw/nubus/*
 F: hw/block/swim.c
+F: hw/m68k/bootinfo.h
 F: include/hw/misc/mac_via.h
 F: include/hw/display/macfb.h
 F: include/hw/nubus/*
diff --git a/hw/m68k/Kconfig b/hw/m68k/Kconfig
index 7aa830327c..c663920f8f 100644
--- a/hw/m68k/Kconfig
+++ b/hw/m68k/Kconfig
@@ -19,3 +19,6 @@ config Q800
 select MACFB
 select NUBUS
 select SWIM
+select ESCC
+select ESP
+select DP8393X
diff --git a/hw/m68k/Makefile.objs b/hw/m68k/Makefile.objs
index f25854730d..b2c9e5ab12 100644
--- a/hw/m68k/Makefile.objs
+++ b/hw/m68k/Makefile.objs
@@ -1,3 +1,4 @@
 obj-$(CONFIG_AN5206) += an5206.o mcf5206.o
 obj-$(CONFIG_MCF5208) += mcf5208.o mcf_intc.o
 obj-$(CONFIG_NEXTCUBE) += next-kbd.o next-cube.o
+obj-$(CONFIG_Q800) += q800.o
diff --git a/hw/m68k/bootinfo.h b/hw/m68k/bootinfo.h
new file mode 100644
index 00..5f8ded2686
--- /dev/null
+++ b/hw/m68k/bootinfo.h
@@ -0,0 +1,114 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note
+ *
+ * Bootinfo tags from linux bootinfo.h and bootinfo-mac.h:
+ * This is an easily parsable and extendable structure containing all
+ * information to be passed from the bootstrap to the kernel
+ *
+ * This structure is copied right after the kernel by the bootstrap
+ * routine.
+ */
+
+#ifndef HW_M68K_BOOTINFO_H
+#define HW_M68K_BOOTINFO_H
+struct bi_record {
+uint16_t tag;/* tag ID */
+uint16_t size;   /* size of record */
+uint32_t data[0];/* data */
+};
+
+/* machine independent tags */
+
+#define BI_LAST 0x /* last record */
+#define BI_MACHTYPE 0x0001 /* machine type (u_long) */
+#define BI_CPUTYPE  0x0002 /* cpu type (u_long) */
+#define BI_FPUTYPE  0x0003 /* fpu type (u_long) */
+#define BI_MMUTYPE  0x0004 /* mmu type (u_long) */
+#define BI_MEMCHUNK 0x0005 /* memory chunk address and size */
+   /* (struct mem_info) */
+#define BI_RAMDISK  0x0006 /* ramdisk address and size */
+   /* (struct mem_info) */
+#define BI_COMMAND_LINE 0x0007 /* kernel command line parameters */
+   /* (string) */
+
+/*  Macintosh-specific tags (all u_long) */
+
+#define BI_MAC_MODEL0x8000  /* Mac Gestalt ID (model type) */
+#define BI_MAC_VADDR0x8001  /* Mac video base address */
+#define BI_MAC_VDEPTH   0x8002  /* Mac video depth */
+#define BI_MAC_VROW 0x8003  /* Mac video rowbytes */
+#define BI_MAC_VDIM 0x8004  /* Mac video dimensions */
+#define BI_MAC_VLOGICAL 0x8005  /* Mac video logical base */
+#define BI_MAC_SCCBASE  0x8006  /* Mac SCC base address */
+#define BI_MAC_BTIME0x8007  /* Mac boot time */
+#define BI_MAC_GMTBIAS  0x8008  /* Mac GMT timezone offset */
+#define BI_MAC_MEMSIZE  0x8009  /* Mac RAM size (sanity check) */
+#define BI_MAC_CPUID0x800a  /* Mac CPU type (sanity check) */
+#define BI_MAC_ROMBASE  0x800b  /* Mac system ROM base address */
+
+/*  Macintosh hardware profile data */
+
+#define BI_MAC_VIA1BASE 0x8010  /* Mac VIA1 base address (always present) */
+#define BI_MAC_VIA2BASE 0x8011  /* Mac VIA2 base address (type varies) */
+#define BI_MAC_VIA2TYPE 0x8012  /* Mac VIA2 type (VIA, RBV, OSS) */
+#define BI_MAC_ADBTYPE  0x8013  /* Mac ADB interface type */
+#define BI_MAC_ASCBASE  0x8014  /* Mac Apple Sound Chip base address */
+#define BI_MAC_SCSI5380 0x8015  /* Mac NCR 5380 SCSI (base address, multi) */
+#define BI_MAC_SCSIDMA  0x8016  /* Mac SCSI DMA (base address) */
+#define BI_MAC_SCSI5396 0x8017  /* Mac NCR 53C96 SCSI (base address, multi) */
+#define BI_MAC_IDETYPE  0x8018  /* Mac IDE interface type */
+#d

[Qemu-block] [PATCH v10 0/9] hw/m68k: add Apple Machintosh Quadra 800 machine

[Laurent Vivier]

I'm rebasing some of these patches for seven years now,
too many years...

if you want to test the machine, I'm sorry, it doesn't boot
a MacROM, but you can boot a linux kernel from the command line.

You can install your own disk using debian-installer, with:

...
-M q800 \
-serial none -serial mon:stdio \
-m 1000M \
-net nic,model=dp83932,addr=09:00:07:12:34:57 \
-append "console=ttyS0 vga=off" \
-kernel vmlinux-4.16.0-1-m68k \
-initrd initrd.gz \
-drive file=debian-10.0-m68k-NETINST-1.iso,media=cdrom \
-drive file=m68k.qcow2,format=qcow2 \
-nographic

If you use a graphic adapter instead of "-nographic", you can use "-g" to set 
the
size of the display (I use "-g 1600x800x24").

You can get the ISO from:

https://cdimage.debian.org/cdimage/ports/10.0/m68k/iso-cd/debian-10.0-m68k-NETINST-1.iso

and extract the kernel and initrd.gz:

guestfish --add debian-10.0-m68k-NETINST-1.iso --ro \
  --mount /dev/sda:/ <<_EOF_
copy-out /install/cdrom/initrd.gz .
copy-out /install/kernels/vmlinux-4.16.0-1-m68k .
_EOF_

The mirror to use is: http://ftp.ports.debian.org/debian-ports/
when it fails, continue without boot loader.

In the same way, you can extract the kernel and the initramfs from the qcow2
image to use it with "-kernel" and "-initrd":

guestfish --add m68k.qcow2 --mount /dev/sda2:/ <<_EOF_
copy-out /boot/vmlinux-4.16.0-1-m68k .
copy-out /boot/initrd.img-4.16.0-1-m68k .
_EOF_

and boot with:

   ...
   -append "root=/dev/sda2 rw console=ttyS0 console=tty \
   -kernel vmlinux-4.16.0-1-m68k \
   -initrd initrd.img-4.16.0-1-m68k

NOTE: DHCP doesn't work but you can assign a static IP address.
  We need some patches for dp8393x that are not ready to be merged.
  See http://patchwork.ozlabs.org/patch/927020/
  http://patchwork.ozlabs.org/patch/927030/
  http://patchwork.ozlabs.org/patch/927026/

v10: Add SWIM VMState and reset function
 Add MacVIA VMState
 rework Kconfig

v9: Fix comments format
rebase on top of NeXTcube

v8: rebase (new blk_new(), add "qemu-common.h")
update bootinfo information and license
add some braces
Rename Q800IRQState to GLUEState:
it's more like a Logic Unit than an IRQ controller,
and Apple calls it "GLUE" (Mark: I prefer to keep it
like this for the moment, in the future this part
need to be reworked, we have to review the IRQ levels
and to wire NUBUS IRQ. The implementation is really trivial
for the moment and we will move it to QOM in the future)

v7: rebase and port to Kconfig
move IRQ controller back to q800.c (we don't need an object for this)
update log message for ESP changes and add some g_assert()
re-order patches: put esp, escc and dp8393x first

v6: Rebase onto git master (this now includes the m68k EXCP_ILLEGAL fix required
  for this patchset to boot)
Add Hervé's R-B tags
Drop ASC (Apple Sound Chip) device since the Linux driver is broken and
  it is not required for a successful boot
Remove extra esp_raise_irq() from ESP pseudo-DMA patch (Hervé)
Remove "return" from unimplemented write functions and instead add a
  "read only" comment (Hervé)
Rename MAX_FD to SWIM_MAX_FD in SWIM floppy controller patch to prevent
  potential conflicts with other files (Hervé)

v5: Rebase onto git master
Add Philippe's R-B to patch 10
Include the command line to boot a Linux kernel under the q800 machine in 
the
commit message for patch 11 (Philippe)
Fix up comments in hw/misc/mac_via.c (Thomas)
Add asserts to VIA ADB support to prevent potential buffer overflows 
(Thomas)
Move macfb surface/resolution checks to realise and remove hw_error (Thomas)
Move macfb draw_line functions inline and remove macfb-template.h (Mark)
Use guest address rather than source pointer in draw_line functions - this 
brings
  macfb in line with the VGA device and can prevent a potential buffer 
overflow
Use g_strdup_printf() for memory region names in NuBus devices instead of
  hardcoded length char arrays (Thomas)
Move NuBus QOM types from patch 7 to patch 8 (spotted by Thomas)
Move CONFIG_COLDFIRE sections together in hw/m68k/Makefile.objs (Thomas)
Remove obsolete comment from q800.c in patch 11 (Thomas)

v4: Drop RFC from subject prefix as this is getting close to final
Rebased onto master (fixing ESP, rom_ptr() conflicts)
Reworked q800.c based upon Thomas' comments about cpu_init() and
  qemu_check_nic_model()
Address Thomas' comments on using error_report() instead of hw_error()
Change the NuBus memory regions from DEVICE_NATIVE_ENDIAN to
  DEVICE_BIG_ENDIAN
Split macfb Nubus support into separate commit
Change VMSTATE_BUFFER_UNSAFE() to VMSTATE_UINT8_ARRAY() in macfb.c as
  suggested by David
Remove dummy Apple Sound Chip migration state as pointed out by David
Keep VIA ADB state and buffers in the mac_via device rather than adding

[Qemu-block] [PATCH v10 3/9] hw/m68k: add via support

[Laurent Vivier]

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
Reviewed-by: Hervé Poussineau 
---
 MAINTAINERS  |   6 +
 default-configs/m68k-softmmu.mak |   1 +
 hw/m68k/Kconfig  |   4 +
 hw/misc/Kconfig  |   4 +
 hw/misc/Makefile.objs|   1 +
 hw/misc/mac_via.c| 722 +++
 include/hw/misc/mac_via.h| 107 +
 7 files changed, 845 insertions(+)
 create mode 100644 hw/misc/mac_via.c
 create mode 100644 include/hw/misc/mac_via.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 50eaf005f4..b01826ba39 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -917,6 +917,12 @@ F: hw/m68k/next-*.c
 F: hw/display/next-fb.c
 F: include/hw/m68k/next-cube.h
 
+q800
+M: Laurent Vivier 
+S: Maintained
+F: hw/misc/mac_via.c
+F: include/hw/misc/mac_via.h
+
 MicroBlaze Machines
 ---
 petalogix_s3adsp1800
diff --git a/default-configs/m68k-softmmu.mak b/default-configs/m68k-softmmu.mak
index d67ab8b96d..6629fd2aa3 100644
--- a/default-configs/m68k-softmmu.mak
+++ b/default-configs/m68k-softmmu.mak
@@ -7,3 +7,4 @@ CONFIG_SEMIHOSTING=y
 CONFIG_AN5206=y
 CONFIG_MCF5208=y
 CONFIG_NEXTCUBE=y
+CONFIG_Q800=y
diff --git a/hw/m68k/Kconfig b/hw/m68k/Kconfig
index a74fac5abd..22a357609c 100644
--- a/hw/m68k/Kconfig
+++ b/hw/m68k/Kconfig
@@ -12,3 +12,7 @@ config NEXTCUBE
 bool
 select FRAMEBUFFER
 select ESCC
+
+config Q800
+bool
+select MAC_VIA
diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig
index 51754bb47c..18a5dc9c09 100644
--- a/hw/misc/Kconfig
+++ b/hw/misc/Kconfig
@@ -120,4 +120,8 @@ config AUX
 config UNIMP
 bool
 
+config MAC_VIA
+bool
+select MOS6522
+
 source macio/Kconfig
diff --git a/hw/misc/Makefile.objs b/hw/misc/Makefile.objs
index e4aad707fb..c4836dd5c3 100644
--- a/hw/misc/Makefile.objs
+++ b/hw/misc/Makefile.objs
@@ -76,6 +76,7 @@ obj-$(CONFIG_PVPANIC) += pvpanic.o
 obj-$(CONFIG_AUX) += auxbus.o
 obj-$(CONFIG_ASPEED_SOC) += aspeed_xdma.o
 obj-$(CONFIG_ASPEED_SOC) += aspeed_scu.o aspeed_sdmc.o
+obj-$(CONFIG_MAC_VIA) += mac_via.o
 obj-$(CONFIG_MSF2) += msf2-sysreg.o
 obj-$(CONFIG_NRF51_SOC) += nrf51_rng.o
 
diff --git a/hw/misc/mac_via.c b/hw/misc/mac_via.c
new file mode 100644
index 00..a052259613
--- /dev/null
+++ b/hw/misc/mac_via.c
@@ -0,0 +1,722 @@
+/*
+ * QEMU m68k Macintosh VIA device support
+ *
+ * Copyright (c) 2011-2018 Laurent Vivier
+ * Copyright (c) 2018 Mark Cave-Ayland
+ *
+ * Some parts from hw/misc/macio/cuda.c
+ *
+ * Copyright (c) 2004-2007 Fabrice Bellard
+ * Copyright (c) 2007 Jocelyn Mayer
+ *
+ * some parts from linux-2.6.29, arch/m68k/include/asm/mac_via.h
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu-common.h"
+#include "migration/vmstate.h"
+#include "hw/sysbus.h"
+#include "hw/irq.h"
+#include "qemu/timer.h"
+#include "hw/misc/mac_via.h"
+#include "hw/misc/mos6522.h"
+#include "hw/input/adb.h"
+#include "sysemu/runstate.h"
+#include "qapi/error.h"
+#include "qemu/cutils.h"
+
+
+/*
+ * VIAs: There are two in every machine,
+ */
+
+#define VIA_SIZE (0x2000)
+
+/*
+ * Not all of these are true post MacII I think.
+ * CSA: probably the ones CHRP marks as 'unused' change purposes
+ * when the IWM becomes the SWIM.
+ * http://www.rs6000.ibm.com/resource/technology/chrpio/via5.mak.html
+ * ftp://ftp.austin.ibm.com/pub/technology/spec/chrp/inwork/CHRP_IORef_1.0.pdf
+ *
+ * also, http://developer.apple.com/technotes/hw/hw_09.html claims the
+ * following changes for IIfx:
+ * VIA1A_vSccWrReq not available and that VIA1A_vSync has moved to an IOP.
+ * Also, "All of the functionality of VIA2 has been moved to other chips".
+ */
+
+#define VIA1A_vSccWrReq 0x80   /*
+* SCC write. (input)
+* [CHRP] SCC WREQ: Reflects the state of the
+* Wait/Request pins from the SCC.
+* [Macintosh Family Hardware]
+* as CHRP on SE/30,II,IIx,IIcx,IIci.
+* on IIfx, "0 means an active request"
+*/
+#define VIA1A_vRev8 0x40   /*
+* Revision 8 board ???
+* [CHRP] En WaitReqB: Lets the WaitReq_L
+* signal from port B of the SCC appear on
+* the PA7 input pin. Output.
+* [Macintosh Family] On the SE/30, this
+* is the bit to flip screen buffers.
+* 0=alternate, 1=main.
+* on II,IIx,IIcx,IIci,IIfx this is a bit
+* for Rev ID. 0=II,IIx, 1=IIcx,IIci,IIfx
+*/
+#define VIA

[Qemu-block] [PATCH v10 6/9] hw/m68k: add Nubus support

[Laurent Vivier]

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
Reviewed-by: Thomas Huth 
---
 MAINTAINERS |   2 +
 hw/Kconfig  |   1 +
 hw/Makefile.objs|   1 +
 hw/m68k/Kconfig |   1 +
 hw/nubus/Kconfig|   2 +
 hw/nubus/Makefile.objs  |   4 +
 hw/nubus/mac-nubus-bridge.c |  45 ++
 hw/nubus/nubus-bridge.c |  34 +
 hw/nubus/nubus-bus.c| 111 ++
 hw/nubus/nubus-device.c | 215 
 include/hw/nubus/mac-nubus-bridge.h |  24 
 include/hw/nubus/nubus.h|  69 +
 12 files changed, 509 insertions(+)
 create mode 100644 hw/nubus/Kconfig
 create mode 100644 hw/nubus/Makefile.objs
 create mode 100644 hw/nubus/mac-nubus-bridge.c
 create mode 100644 hw/nubus/nubus-bridge.c
 create mode 100644 hw/nubus/nubus-bus.c
 create mode 100644 hw/nubus/nubus-device.c
 create mode 100644 include/hw/nubus/mac-nubus-bridge.h
 create mode 100644 include/hw/nubus/nubus.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 6b5fc50aef..4f6b2b037a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -922,8 +922,10 @@ M: Laurent Vivier 
 S: Maintained
 F: hw/misc/mac_via.c
 F: hw/display/macfb.c
+F: hw/nubus/*
 F: include/hw/misc/mac_via.h
 F: include/hw/display/macfb.h
+F: include/hw/nubus/*
 
 MicroBlaze Machines
 ---
diff --git a/hw/Kconfig b/hw/Kconfig
index b45db3c813..0501a55315 100644
--- a/hw/Kconfig
+++ b/hw/Kconfig
@@ -21,6 +21,7 @@ source isa/Kconfig
 source mem/Kconfig
 source misc/Kconfig
 source net/Kconfig
+source nubus/Kconfig
 source nvram/Kconfig
 source pci-bridge/Kconfig
 source pci-host/Kconfig
diff --git a/hw/Makefile.objs b/hw/Makefile.objs
index ece6cc3755..457b95e28d 100644
--- a/hw/Makefile.objs
+++ b/hw/Makefile.objs
@@ -37,6 +37,7 @@ devices-dirs-y += virtio/
 devices-dirs-y += watchdog/
 devices-dirs-y += xen/
 devices-dirs-$(CONFIG_MEM_DEVICE) += mem/
+devices-dirs-$(CONFIG_NUBUS) += nubus/
 devices-dirs-y += semihosting/
 devices-dirs-y += smbios/
 endif
diff --git a/hw/m68k/Kconfig b/hw/m68k/Kconfig
index 58c39ec1a9..9133919bb8 100644
--- a/hw/m68k/Kconfig
+++ b/hw/m68k/Kconfig
@@ -17,3 +17,4 @@ config Q800
 bool
 select MAC_VIA
 select MACFB
+select NUBUS
diff --git a/hw/nubus/Kconfig b/hw/nubus/Kconfig
new file mode 100644
index 00..8fb8b22189
--- /dev/null
+++ b/hw/nubus/Kconfig
@@ -0,0 +1,2 @@
+config NUBUS
+bool
diff --git a/hw/nubus/Makefile.objs b/hw/nubus/Makefile.objs
new file mode 100644
index 00..135ba7878d
--- /dev/null
+++ b/hw/nubus/Makefile.objs
@@ -0,0 +1,4 @@
+common-obj-y += nubus-device.o
+common-obj-y += nubus-bus.o
+common-obj-y += nubus-bridge.o
+common-obj-$(CONFIG_Q800) += mac-nubus-bridge.o
diff --git a/hw/nubus/mac-nubus-bridge.c b/hw/nubus/mac-nubus-bridge.c
new file mode 100644
index 00..7c329300b8
--- /dev/null
+++ b/hw/nubus/mac-nubus-bridge.c
@@ -0,0 +1,45 @@
+/*
+ *  Copyright (c) 2013-2018 Laurent Vivier 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "hw/sysbus.h"
+#include "hw/nubus/mac-nubus-bridge.h"
+
+
+static void mac_nubus_bridge_init(Object *obj)
+{
+MacNubusState *s = MAC_NUBUS_BRIDGE(obj);
+SysBusDevice *sbd = SYS_BUS_DEVICE(obj);
+
+s->bus = NUBUS_BUS(qbus_create(TYPE_NUBUS_BUS, DEVICE(s), NULL));
+
+sysbus_init_mmio(sbd, &s->bus->super_slot_io);
+sysbus_init_mmio(sbd, &s->bus->slot_io);
+}
+
+static void mac_nubus_bridge_class_init(ObjectClass *klass, void *data)
+{
+DeviceClass *dc = DEVICE_CLASS(klass);
+
+dc->desc = "Nubus bridge";
+}
+
+static const TypeInfo mac_nubus_bridge_info = {
+.name  = TYPE_MAC_NUBUS_BRIDGE,
+.parent= TYPE_NUBUS_BRIDGE,
+.instance_init = mac_nubus_bridge_init,
+.instance_size = sizeof(MacNubusState),
+.class_init= mac_nubus_bridge_class_init,
+};
+
+static void mac_nubus_bridge_register_types(void)
+{
+type_register_static(&mac_nubus_bridge_info);
+}
+
+type_init(mac_nubus_bridge_register_types)
diff --git a/hw/nubus/nubus-bridge.c b/hw/nubus/nubus-bridge.c
new file mode 100644
index 00..cd8c6a91eb
--- /dev/null
+++ b/hw/nubus/nubus-bridge.c
@@ -0,0 +1,34 @@
+/*
+ * QEMU Macintosh Nubus
+ *
+ * Copyright (c) 2013-2018 Laurent Vivier 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "hw/sysbus.h"
+#include "hw/nubus/nubus.h"
+
+static void nubus_bridge_class_init(ObjectClass *klass, void *data)
+{
+DeviceClass *dc = DEVICE_CLASS(klass);
+
+dc->fw_name = "nubus";
+}
+
+static const TypeInfo nubus_bridge_info = {
+.name  = TYPE_NUBUS_BRIDGE,
+.parent= TYPE

[Qemu-block] [PATCH v10 1/9] esp: add pseudo-DMA as used by Macintosh

[Laurent Vivier]

There is no DMA in Quadra 800, so the CPU reads/writes the data from the
PDMA register (offset 0x100, ESP_PDMA in hw/m68k/q800.c) and copies them
to/from the memory.

There is a nice assembly loop in the kernel to do that, see
linux/drivers/scsi/mac_esp.c:MAC_ESP_PDMA_LOOP().

The start of the transfer is triggered by the DREQ interrupt (see linux
mac_esp_send_pdma_cmd()), the CPU polls on the IRQ flag to start the
transfer after a SCSI command has been sent (in Quadra 800 it goes
through the VIA2, the via2-irq line and the vIFR register)

The Macintosh hardware includes hardware handshaking to prevent the CPU
from reading invalid data or writing data faster than the peripheral
device can accept it.

This is the "blind mode", and from the doc:
"Approximate maximum SCSI transfer rates within a blocks are 1.4 MB per
second for blind transfers in the Macintosh II"

Some references can be found in:
  Apple Macintosh Family Hardware Reference, ISBN 0-201-19255-1
  Guide to the Macintosh Family Hardware, ISBN-0-201-52405-8

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
---
 hw/scsi/esp.c | 296 +-
 include/hw/scsi/esp.h |   7 +
 2 files changed, 274 insertions(+), 29 deletions(-)

diff --git a/hw/scsi/esp.c b/hw/scsi/esp.c
index 841d79b60e..ffbeee9f3c 100644
--- a/hw/scsi/esp.c
+++ b/hw/scsi/esp.c
@@ -38,6 +38,8 @@
  * 
http://www.ibiblio.org/pub/historic-linux/early-ports/Sparc/NCR/NCR89C100.txt
  * and
  * http://www.ibiblio.org/pub/historic-linux/early-ports/Sparc/NCR/NCR53C9X.txt
+ *
+ * On Macintosh Quadra it is a NCR53C96.
  */
 
 static void esp_raise_irq(ESPState *s)
@@ -58,6 +60,16 @@ static void esp_lower_irq(ESPState *s)
 }
 }
 
+static void esp_raise_drq(ESPState *s)
+{
+qemu_irq_raise(s->irq_data);
+}
+
+static void esp_lower_drq(ESPState *s)
+{
+qemu_irq_lower(s->irq_data);
+}
+
 void esp_dma_enable(ESPState *s, int irq, int level)
 {
 if (level) {
@@ -84,29 +96,11 @@ void esp_request_cancelled(SCSIRequest *req)
 }
 }
 
-static uint32_t get_cmd(ESPState *s, uint8_t *buf, uint8_t buflen)
+static int get_cmd_cb(ESPState *s)
 {
-uint32_t dmalen;
 int target;
 
 target = s->wregs[ESP_WBUSID] & BUSID_DID;
-if (s->dma) {
-dmalen = s->rregs[ESP_TCLO];
-dmalen |= s->rregs[ESP_TCMID] << 8;
-dmalen |= s->rregs[ESP_TCHI] << 16;
-if (dmalen > buflen) {
-return 0;
-}
-s->dma_memory_read(s->dma_opaque, buf, dmalen);
-} else {
-dmalen = s->ti_size;
-if (dmalen > TI_BUFSZ) {
-return 0;
-}
-memcpy(buf, s->ti_buf, dmalen);
-buf[0] = buf[2] >> 5;
-}
-trace_esp_get_cmd(dmalen, target);
 
 s->ti_size = 0;
 s->ti_rptr = 0;
@@ -125,8 +119,48 @@ static uint32_t get_cmd(ESPState *s, uint8_t *buf, uint8_t 
buflen)
 s->rregs[ESP_RINTR] = INTR_DC;
 s->rregs[ESP_RSEQ] = SEQ_0;
 esp_raise_irq(s);
+return -1;
+}
+return 0;
+}
+
+static uint32_t get_cmd(ESPState *s, uint8_t *buf, uint8_t buflen)
+{
+int target;
+uint32_t dmalen;
+
+target = s->wregs[ESP_WBUSID] & BUSID_DID;
+if (s->dma) {
+dmalen = s->rregs[ESP_TCLO];
+dmalen |= s->rregs[ESP_TCMID] << 8;
+dmalen |= s->rregs[ESP_TCHI] << 16;
+if (dmalen > buflen) {
+return 0;
+}
+if (s->dma_memory_read) {
+s->dma_memory_read(s->dma_opaque, buf, dmalen);
+} else {
+memcpy(s->pdma_buf, buf, dmalen);
+s->pdma_len = dmalen;
+s->pdma_start = s->pdma_buf;
+s->pdma_cur = s->pdma_buf;
+esp_raise_drq(s);
+return 0;
+}
+} else {
+dmalen = s->ti_size;
+if (dmalen > TI_BUFSZ) {
+return 0;
+}
+memcpy(buf, s->ti_buf, dmalen);
+buf[0] = buf[2] >> 5;
+}
+trace_esp_get_cmd(dmalen, target);
+
+if (get_cmd_cb(s) < 0) {
 return 0;
 }
+
 return dmalen;
 }
 
@@ -165,6 +199,16 @@ static void do_cmd(ESPState *s, uint8_t *buf)
 do_busid_cmd(s, &buf[1], busid);
 }
 
+static void satn_pdma_cb(ESPState *s)
+{
+if (get_cmd_cb(s) < 0) {
+return;
+}
+if (s->pdma_cur != s->pdma_start) {
+do_cmd(s, s->pdma_start);
+}
+}
+
 static void handle_satn(ESPState *s)
 {
 uint8_t buf[32];
@@ -174,11 +218,22 @@ static void handle_satn(ESPState *s)
 s->dma_cb = handle_satn;
 return;
 }
+s->pdma_cb = satn_pdma_cb;
 len = get_cmd(s, buf, sizeof(buf));
 if (len)
 do_cmd(s, buf);
 }
 
+static void s_without_satn_pdma_cb(ESPState *s)
+{
+if (get_cmd_cb(s) < 0) {
+return;
+}
+if (s->pdma_cur != s->pdma_start) {
+do_busid_cmd(s, s->pdma_start, 0);
+}
+}
+
 static void handle_s_without_atn(ESPState *s)
 {
 uint8_t buf[32];
@@ -188,18 +243,36 @@ static void hand

Re: [Qemu-block] [PATCH V3] block/vhdx: add check for truncated image files

[Kevin Wolf]

Am 05.09.2019 um 12:02 hat Peter Lieven geschrieben:
> Am 04.09.19 um 16:09 schrieb Kevin Wolf:
> > Am 03.09.2019 um 15:35 hat Peter Lieven geschrieben:
> > > qemu is currently not able to detect truncated vhdx image files.
> > > Add a basic check if all allocated blocks are reachable at open and
> > > report all errors during bdrv_co_check.
> > > 
> > > Signed-off-by: Peter Lieven 
> > > ---
> > > V3: - check for bdrv_getlength failure [Kevin]
> > >  - use uint32_t for i [Kevin]
> > >  - check for BAT entry overflow [Kevin]
> > >  - break on !errcnt in second check
> > > 
> > > V2: - add error reporting [Kevin]
> > >  - use bdrv_getlength instead of bdrv_get_allocated_file_size [Kevin]
> > >  - factor out BAT entry check and add error reporting for region
> > >overlaps
> > >  - already check on vhdx_open
> > Something still seems to be wrong with this patch:
> > 
> >  213  fail   [15:50:13] [15:50:14]  (last: 2s)output 
> > mismatch (see 213.out.bad)
> >  --- /home/kwolf/source/qemu/tests/qemu-iotests/213.out  2019-06-28 
> > 14:19:50.065797707 +0200
> >  +++ /home/kwolf/source/qemu/tests/qemu-iotests/213.out.bad  
> > 2019-09-04 15:50:14.582053976 +0200
> >  @@ -46,10 +46,8 @@
> >   {"execute": "job-dismiss", "arguments": {"id": "job0"}}
> >   {"return": {}}
> > 
> >  -image: TEST_IMG
> >  -file format: IMGFMT
> >  -virtual size: 32 MiB (33554432 bytes)
> >  -cluster_size: 268435456
> >  +qemu-img: VHDX BAT entry 0 offset points after end of file. Image has 
> > probably been truncated.
> >  +qemu-img: Could not open 'TEST_IMG': Could not open 'TEST_IMG': 
> > Invalid argument
> > 
> >   === Invalid BlockdevRef ===
> > 
> > I can reproduce this manually with the following qemu-img invocations.
> > It seems all three options must be given to reproduce the error:
> > 
> >  $ ./qemu-img create -f vhdx -o 
> > block_size=268435456,subformat=fixed,block_state_zero=off /tmp/test.vhdx 32M
> >  Formatting '/tmp/test.vhdx', fmt=vhdx size=33554432 log_size=1048576 
> > block_size=268435456 subformat=fixed block_state_zero=off
> >  $ ./qemu-img info /tmp/test.vhdx
> >  qemu-img: VHDX BAT entry 0 offset points after end of file. Image has 
> > probably been truncated.
> >  qemu-img: Could not open '/tmp/test.vhdx': Could not open 
> > '/tmp/test.vhdx': Invalid argument
> > 
> > If I add the offsets to the error message (would probably nice to have),
> > I get:
> > 
> >  qemu-img: VHDX BAT entry 0 offset 8388608 points after end of file 
> > (41943040). Image has probably been truncated.
> > 
> > So it seems that the file is large enough to hold 32M + metadata, but we
> > don't increase the file size to hold a full block (256M). Is this a
> > problem in the way we create images or are partial blocks at the end
> > expected?
> > 
> > Kevin
> 
> 
> A short look into the VHDX spec [1] seems to suggest that a VHDX File
> can only grow in Block increments.
> 
> See page 8 in the definition of blocks: "Allocation of new space for a
> virtual hard disk that supports dynamic growth of the virtual hard
> disk file is done in fixes size units defined as blocks."

Then I guess we need to fix the creation of VHDX images before we can
apply this patch because otherwise qemu-iotests fails.

Hm... And probably ignore the error for a partial final block anyway to
maintain compatibility with images created by older QEMU versions.

Kevin

Re: [Qemu-block] [Qemu-stable] [Qemu-devel] [PATCH 2/3] block/qcow2: fix the corruption when rebasing luks encrypted files

[Maxim Levitsky]

On Mon, 2019-09-09 at 12:56 +0200, Kevin Wolf wrote:
> Am 06.09.2019 um 21:17 hat Eric Blake geschrieben:
> > > -assert((offset_in_cluster & ~BDRV_SECTOR_MASK) == 0);
> > > +assert((guest_offset & ~BDRV_SECTOR_MASK) == 0);
> > > +assert((host_offset & ~BDRV_SECTOR_MASK) == 0);
> > >  assert((bytes & ~BDRV_SECTOR_MASK) == 0);
> > 
> > Pre-existing, but we could use QEMU_IS_ALIGNED(x, BDRV_SECTOR_SIZE) for
> > slightly more legibility than open-coding the bit operation.
> > 
> > Neat trick about power-of-2 alignment checks:
> > 
> > assert(QEMU_IS_ALIGNED(offset_in_cluster | guest_offset |
> >host_offset | bytes, BDRV_SECTOR_SIZE));
> > 
> > gives the same result in one assertion.  (I've used it elsewhere in the
> > code base, but I'm not opposed to one assert per variable if you think
> > batching is too dense.)
> 
> A possible downside of this is that if a user reports an assertion
> failure, you can't tell any more which of the variables ended up in a
> bad state.
> 
> If you're lucky, you can still tell in gdb at least if the bug is
> reproducible, but I wouldn't be surprised if in release builds, half of
> the variables were actually optimised away, so that even this wouldn't
> work.
Agreed. I guess I'll keep the separate asserts anyway after all, even though
I prefer shorter code.


Best regards,
Maxim Levitsky

[Qemu-block] [PATCH] blockjob: update nodes head while removing all bdrv

[Sergio Lopez]

block_job_remove_all_bdrv() iterates through job->nodes, calling
bdrv_root_unref_child() for each entry. The call to the latter may
reach child_job_[can_]set_aio_ctx(), which will also attempt to
traverse job->nodes, potentially finding entries that where freed
on previous iterations.

To avoid this situation, update job->nodes head on each iteration to
ensure that already freed entries are no longer linked to the list.

RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1746631
Signed-off-by: Sergio Lopez 
---
 blockjob.c | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/blockjob.c b/blockjob.c
index 6e32d1a0c0..7b1551d981 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -192,6 +192,12 @@ void block_job_remove_all_bdrv(BlockJob *job)
 BdrvChild *c = l->data;
 bdrv_op_unblock_all(c->bs, job->blocker);
 bdrv_root_unref_child(c);
+/*
+ * The call above may reach child_job_[can_]set_aio_ctx(), which will
+ * also traverse job->nodes, so update the head here to make sure it
+ * doesn't attempt to process an already freed BdrvChild.
+ */
+job->nodes = l->next;
 }
 g_slist_free(job->nodes);
 job->nodes = NULL;
-- 
2.21.0

Re: [Qemu-block] [PATCH v2 3/3] qemu-iotests: Add test for bz #1745922

[Maxim Levitsky]

On Mon, 2019-09-09 at 11:35 +0100, Daniel P. Berrangé wrote:
> On Fri, Sep 06, 2019 at 10:57:50PM +0300, Maxim Levitsky wrote:
> > Signed-off-by: Maxim Levitsky 
> > ---
> >  tests/qemu-iotests/263 | 75 ++
> >  tests/qemu-iotests/263.out | 19 ++
> >  tests/qemu-iotests/group   |  1 +
> >  3 files changed, 95 insertions(+)
> >  create mode 100755 tests/qemu-iotests/263
> >  create mode 100644 tests/qemu-iotests/263.out
> > 
> > diff --git a/tests/qemu-iotests/263 b/tests/qemu-iotests/263
> > new file mode 100755
> > index 00..36951ff7b4
> > --- /dev/null
> > +++ b/tests/qemu-iotests/263
> > @@ -0,0 +1,75 @@
> > +#!/usr/bin/env bash
> > +#
> > +# Test encrypted write that crosses cluster boundary of two unallocated 
> > clusters
> > +# Based on 188
> > +#
> > +# Copyright (C) 2019 Red Hat, Inc.
> > +#
> > +# This program is free software; you can redistribute it and/or modify
> > +# it under the terms of the GNU General Public License as published by
> > +# the Free Software Foundation; either version 2 of the License, or
> > +# (at your option) any later version.
> > +#
> > +# This program is distributed in the hope that it will be useful,
> > +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +# GNU General Public License for more details.
> > +#
> > +# You should have received a copy of the GNU General Public License
> > +# along with this program.  If not, see .
> > +#
> > +
> > +# creator
> > +owner=mlevi...@redhat.com
> > +
> > +seq=`basename $0`
> > +echo "QA output created by $seq"
> > +
> > +status=1   # failure is the default!
> > +
> > +_cleanup()
> > +{
> > +   _cleanup_test_img
> > +}
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +# get standard environment, filters and checks
> > +. ./common.rc
> > +. ./common.filter
> > +
> > +_supported_fmt qcow2
> > +_supported_proto generic
> > +_supported_os Linux
> > +
> > +
> > +size=1M
> > +
> > +SECRET="secret,id=sec0,data=astrochicken"
> > +
> > +_make_test_img --object $SECRET -o 
> > "encrypt.format=luks,encrypt.key-secret=sec0,encrypt.iter-time=10,cluster_size=64K"
> >  $size
> > +
> > +IMGSPEC="driver=$IMGFMT,encrypt.key-secret=sec0,file.filename=$TEST_IMG"
> > +
> > +QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT
> > +
> > +echo
> > +echo "== reading the whole image =="
> > +$QEMU_IO --object $SECRET -c "read -P 0 0 $size" --image-opts $IMGSPEC | 
> > _filter_qemu_io | _filter_testdir
> > +
> > +echo
> > +echo "== write two 512 byte sectors on a cluster boundary =="
> > +$QEMU_IO --object $SECRET -c "write -P 0xAA 0xFE00 0x400" --image-opts 
> > $IMGSPEC | _filter_qemu_io | _filter_testdir
> > +
> > +echo
> > +echo "== verify that the rest of the image is not changed =="
> > +$QEMU_IO --object $SECRET -c "read -P 0x00 0x0 0xFE00" --image-opts 
> > $IMGSPEC | _filter_qemu_io | _filter_testdir
> > +$QEMU_IO --object $SECRET -c "read -P 0xAA 0x0FE00 0x400" --image-opts 
> > $IMGSPEC | _filter_qemu_io | _filter_testdir
> > +$QEMU_IO --object $SECRET -c "read -P 0x00 0x10200 0xEFE00" --image-opts 
> > $IMGSPEC | _filter_qemu_io | _filter_testdir
> 
> This tests LUKS encryption, but the code you'r changing/fixing also used
> for the traditionl qcow2 encryption. The difference in IV handling for
> these two methods is what made this code confusing, so I'd like to see
> that the test also covers traditional qcow2 encryption.
This is very good idea. Done.

> 
> Also can you confirm that the test succeeds when run on a qemu
> built against 8c1ecb590497b0349c550607db923972b37f6963  (the change
> immediately before Vladimir's threading series) ?
Yes, the test fails with this commit. It also fails on master and works
with my fix (both encryption case).

> 
> 
> Regards,
> Daniel


Best regards,
Maxim Levitsky

Re: [Qemu-block] [PATCH] tests/qemu-iotests/check: Replace "tests" with "iotests" in final status text

[Max Reitz]

On 06.09.19 13:39, Thomas Huth wrote:
> When running "make check -j8" or something similar, the iotests are
> running in parallel with the other tests. So when they are printing
> out "Passed all xx tests" or a similar status message at the end,
> it might not be quite clear that this message belongs to the iotests,
> since the output might be mixed with the other tests. Thus change the
> word "tests" here to "iotests" instead to avoid confusion.
> 
> Signed-off-by: Thomas Huth 
> ---
>  tests/qemu-iotests/check | 8 
>  1 file changed, 4 insertions(+), 4 deletions(-)

Thanks, applied to my block branch:

https://git.xanclic.moe/XanClic/qemu/commits/branch/block

Max



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-block] [PATCH] tests/Makefile: Do not print the name of the check-block.sh shell script

[Max Reitz]

On 10.09.19 12:55, Thomas Huth wrote:
> On 10/09/2019 12.53, Max Reitz wrote:
>> On 06.09.19 13:35, Thomas Huth wrote:
>>> The check script is already printing out which iotest is currently
>>> running, so printing out the name of the check-block.sh shell script
>>> looks superfluous here.
>>>
>>> Signed-off-by: Thomas Huth 
>>> ---
>>>  tests/Makefile.include | 2 +-
>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> Reviewed-by: Max Reitz 
>>
>> (Not sure which tree you’d like this to go through...)
> 
> I think either your block tree or trivial would be appropriate?

OK then, I’ve taken it. :-)

Max



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-block] [PATCH] tests/Makefile: Do not print the name of the check-block.sh shell script

[Thomas Huth]

On 10/09/2019 12.53, Max Reitz wrote:
> On 06.09.19 13:35, Thomas Huth wrote:
>> The check script is already printing out which iotest is currently
>> running, so printing out the name of the check-block.sh shell script
>> looks superfluous here.
>>
>> Signed-off-by: Thomas Huth 
>> ---
>>  tests/Makefile.include | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> Reviewed-by: Max Reitz 
> 
> (Not sure which tree you’d like this to go through...)

I think either your block tree or trivial would be appropriate?

 Thomas



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-block] [PATCH] tests/Makefile: Do not print the name of the check-block.sh shell script

[Max Reitz]

On 06.09.19 13:35, Thomas Huth wrote:
> The check script is already printing out which iotest is currently
> running, so printing out the name of the check-block.sh shell script
> looks superfluous here.
> 
> Signed-off-by: Thomas Huth 
> ---
>  tests/Makefile.include | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Reviewed-by: Max Reitz 

(Not sure which tree you’d like this to go through...)



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-block] [PATCH v2 0/2] Alignment checks cleanup

[Max Reitz]

On 27.08.19 20:59, Nir Soffer wrote:
> While working on 4k support, I noticed that there is lot of code using
> BDRV_SECTOR_SIZE (512) for checking alignment. I wonder how this can work with
> 4k storage.
> 
> Lets start by cleaning up to make the code easier to understand:
> - Use QEMU_IS_ALIGNED macro to check alignment
> - Remove unneeded masks based on BDRV_SECTOR_SIZE
> 
> Nir Soffer (2):
>   block: Use QEMU_IS_ALIGNED
>   block: Remove unused masks
> 
>  block/bochs.c | 4 ++--
>  block/cloop.c | 4 ++--
>  block/dmg.c   | 4 ++--
>  block/io.c| 8 
>  block/qcow2-cluster.c | 4 ++--
>  block/qcow2.c | 4 ++--
>  block/vvfat.c | 8 
>  include/block/block.h | 2 --
>  migration/block.c | 2 +-
>  qemu-img.c| 2 +-
>  10 files changed, 20 insertions(+), 22 deletions(-)

Thanks, applied to my block branch:

https://git.xanclic.moe/XanClic/qemu/commits/branch/block


(I think you should check the setting of git config’s user.email in your
qemu directory.  Even if you send the patches through a gmail address
(which I don’t quite know why you’re doing that, but it isn’t like that
really matters), the patches should then still contain a “From: ” that
shows the actual author as set by user.name and user.email.)

Max



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-block] [PATCH v6 04/42] block: Add child access functions

[Kevin Wolf]

Am 10.09.2019 um 11:14 hat Max Reitz geschrieben:
> On 09.09.19 18:13, Kevin Wolf wrote:
> > Am 09.09.2019 um 16:04 hat Max Reitz geschrieben:
> >> On 09.09.19 11:36, Kevin Wolf wrote:
> >>> Am 09.09.2019 um 09:56 hat Max Reitz geschrieben:
>  On 04.09.19 18:16, Kevin Wolf wrote:
> > Am 09.08.2019 um 18:13 hat Max Reitz geschrieben:
> >> There are BDS children that the general block layer code can access,
> >> namely bs->file and bs->backing.  Since the introduction of filters and
> >> external data files, their meaning is not quite clear.  bs->backing can
> >> be a COW source, or it can be an R/W-filtered child; bs->file can be an
> >> R/W-filtered child, it can be data and metadata storage, or it can be
> >> just metadata storage.
> >>
> >> This overloading really is not helpful.  This patch adds function that
> >> retrieve the correct child for each exact purpose.  Later patches in
> >> this series will make use of them.  Doing so will allow us to handle
> >> filter nodes and external data files in a meaningful way.
> >>
> >> Signed-off-by: Max Reitz 
> >> Reviewed-by: Vladimir Sementsov-Ogievskiy 
> >
> > Each time I look at this patch, I'm confused by the function names.
> > Maybe I should just ask what the idea there was, or more specifically:
> > What does the "filtered" in "filtered child" really mean?
> >
> > Apparently any child of a filter node is "filtered" (which makes sense),
> 
>  It isn’t, filters can have non-filter children.  For example, backup-top
>  could have the source as a filtered child and the target as a non-filter
>  child.
> >>>
> >>> Hm, okay, makes sense. I had a definition in mind that says that filter
> >>> nodes only have a single child node. Is it that a filter may have only a
> >>> single _filtered_ child node?
> >>
> >> Well, there’s Quorum...
> > 
> > Ah, nice, quorum sets is_filter = true even though it neither fulfulls
> > the conditions for it before this series, nor the changed conditions
> > after this series.
> > 
> > So either quorum lies and isn't actually a filter driver, or our
> > definition in the documentation of is_filter is wrong.
> 
> You could say it lies because in FIFO mode it clearly isn’t a filter for
> all of its children.
> 
> There is a reason for lying, though, which is
> bdrv_recurse_is_first_non_filter(), which is necessary to use the whole
> to_replace mirror stuff.

Hm, actually, now that you mention bdrv_recurse_is_first_non_filter(),
quorum was the first driver to declare itself a filter, so strictly
speaking, if there is an inconsistency, it's the other uses that are
abusing the field...

> (You mirror from a quorum with a failed child and then replace the
> failed child.  mirror needs to ensure that there are only R/W filters
> between the child and the mirror source so that replacing it will not
> suddenly change any visible data.  Which is actually a lie for quorum,
> because the child is clearly broken and thus precisely doesn’t show the
> same data...)
> 
> Maybe we should stop declaring Quorum a filter and then rename the
> bdrv_recurse_is_first_non_filter() to, I don’t know,
> bdrv_recurse_can_be_replaced_by_mirror()?

Why not.

> > but also bs->backing of a qcow2 image, while bs->file of qcow2 isn't.
> > raw doesn't have any "filtered" child. What's the system behind this?
> 
>  “filtered” means: If the parent node returns data from this child, it
>  won’t modify it, neither its content nor its position.  COW and R/W
>  filters differ in how they handle writes; R/W filters pass them through
>  to the filtered child, COW filters copy them off to some other child
>  node (and then the filtered child’s data will no longer be visible at
>  that location).
> >>>
> >>> But there is no reason why a node couldn't fulfill this condition for
> >>> more than one child node. bdrv_filtered_child() isn't well-defined then.
> >>> Technically, the description "Return any filtered child" is correct
> >>> because "any" can be interpreted as "an arbitrary", but obviously that
> >>> makes the function useless.
> >>
> >> Which is why it currently returns NULL for Quorum.
> > 
> > Which is about the only possible choice that breaks the contract...
> > 
> >  * Return any filtered child, independently of how it reacts to write
> 
> I don’t know if you’re serious about this proposition, because I don’t
> know whether that could be useful in any way. :-?

Huh? This is just quoting the contract from your code?

> >  * accesses and whether data is copied onto this BDS through COR.
> 
> I meant the contract as “Return the single filtered child there is, or NULL”

Then that should probably be spelt out in the contract. Probably even
explicitly "NULL if there is either no filtered child or multiple
filtered children".

> > Maybe the documentation of bdrv_filtered_child() needs to be rephrased?
> > 
> > Going back to qcow2,

Re: [Qemu-block] [PATCH 0/7] block: Generic file creation fallback

[Maxim Levitsky]

On Tue, 2019-09-10 at 11:16 +0200, Max Reitz wrote:
> On 05.09.19 15:30, Maxim Levitsky wrote:
> > On Fri, 2019-07-12 at 19:35 +0200, Max Reitz wrote:
> > > Hi,
> > > 
> > > Kevin commented on my RFC, so I got what an RFC wants, and he didn’t
> > > object to the creation fallback part.  So I suppose I can go down that
> > > route at least.  (Which was actually the more important part of the
> > > series.)
> > > 
> > > So as in the RFC, this series adds a fallback path for creating files
> > > (on the protocol layer) if the protocol driver does not support file
> > > creation, but the file already exists.
> > > 
> > 
> > Hi!
> > Do you have any update on this patch series by a chance?
> 
> Unfortunately, no.  I was on PTO, and before that, there was just too
> much else going on.  (And frankly, there’s still too much else going on.)

No problem!


Best regards,
Maxim Levitsky

Re: [Qemu-block] [Qemu-devel] IOTEST 162

[Maxim Levitsky]

On Mon, 2019-09-09 at 13:24 -0400, John Snow wrote:
> 
> On 9/6/19 1:25 PM, Maxim Levitsky wrote:
> > Hi!
> > 
> > I just had a very fun rabbit hole dive, and I want to share it with you.
> > 
> > I notice for some time that iotest 162 fails with that:
> > 
> > -qemu-img: Could not open 'json:{"driver": "nbd", "host": 42}': Failed to 
> > connect socket: Invalid argument
> > +qemu-img: Could not open 'json:{"driver": "nbd", "host": 42}': Failed to 
> > connect socket:
> > Connection timed out
> > 
> > 
> > I didn't bother to report it yet as it started failing more or less when 
> > qemu 4.2 merge window opened,
> > so I thought that maybe something got broken, or maybe something broken in 
> > my environment 
> > (ahem, AF_UNIX path is too long, ahem)
> > 
> > I just didn't had enough time to pay attention to this. Until today.
> > 
> > So I asked Kevin Wolf today, just by the way why I see for some time that
> > iotest 162 fails. for me.
> > 
> > 
> > He told me that it works for him.
> > So I look at the test and I see that it just does
> > 
> > qemu-img info 'json:{"driver": "nbd", "host": 42}'
> > 
> > Supposedly it should fail because 42 is not quoted, and it 
> > doesn't look like a valid host name.
> > 
> > I try with disto's qemu-img (2.11) and I still see that I get the 
> > 'Connection timed out'
> > 
> > Then I ask him to try on his system with '42' quoted and it still passes.
> > 
> > All right - this means that this 42 is parsed. He also mentions that he 
> > uses fedora 29 and I still
> > use fedora 28. So I start a VM with fedora 30, and yep, there it does work 
> > correctly.
> > 
> > All right, that 42 must be parsed as a host name. Yep. 'ping 42' on my 
> > machine tries to ping 0.0.0.42,
> > and in VM exits immediately with 'Invalid argument'
> > All right, lets ping 0.0.0.42. Maybe something in hostname parsing changed, 
> > maybe something parses this on DNS level?
> > Nope, ping 0.0.0.42 works on my machine, fails with invalid argument in 
> > VM...
> > 
> > All right lets strace it
> > 
> > connect(5, {sa_family=AF_INET, sin_port=htons(1025), 
> > sin_addr=inet_addr("0.0.0.42")}, 16) = -1 EINVAL (Invalid argument)
> > 
> > Same thing passes on my machine
> > 
> > Hmm... this is something in the kernel. Maybe something in iptables/etc. I 
> > don't yet know that
> > area that well to be honest.
> > 
> > So I googled a bit and look what I found:
> > 
> > https://lwn.net/Articles/791086/
> > 
> > And yes, while my machine runs fedora 28, it as usual runs vanilla git 
> > master
> > kernel, which I compile from time to time. Currently I am on 5.3.0-rc4+.
> > 
> > So I must say that nothing less but kernel 5.3.0, breaks iotest 162.
> > 
> > I''l prepare a patch soon to fix this.
> 
> I wonder if I am seeing some related problems, though I am running
> 5.2.11 right now and not 5.3.x:
> 
> 162 is failing in this way:
> 
> -qemu-img: Could not open 'json:{"driver": "ssh", "host": "localhost",
> "port": "0", "path": "/foo"}': Failed to connect socket: Connection refused
> -qemu-img: Could not open 'driver=ssh,host=localhost,port=0,path=/foo':
> Failed to connect socket: Connection refused
> +qemu-img: Could not open 'json:{"driver": "ssh", "host": "localhost",
> "port": "0", "path": "/foo"}': Failed to connect socket: Connection
> timed out
> +qemu-img: Could not open 'driver=ssh,host=localhost,port=0,path=/foo':
> Failed to connect socket: Connection timed out
>  qemu-img: Could not open 'json:{"driver": "ssh", "host": "localhost",
> "port": 0.42, "path": "/foo"}': Parameter 'port' expects a number
>  qemu-img: Could not open
> 'driver=ssh,host=localhost,port=0.42,path=/foo': Parameter 'port'
> expects a number

This must be the same issue, although here it fails like that:


[mlevitsk@maximlenovopc ~/qemu/tests/qemu-iotests]$./check -qcow2 162
QEMU  -- 
"/home/mlevitsk/qemu/tests/qemu-iotests/../../x86_64-softmmu/qemu-system-x86_64"
 -nodefaults -display none -machine accel=qtest
QEMU_IMG  -- "/home/mlevitsk/qemu/tests/qemu-iotests/../../qemu-img" 
QEMU_IO   -- "/home/mlevitsk/qemu/tests/qemu-iotests/../../qemu-io"  
--cache writeback -f qcow2
QEMU_NBD  -- "/home/mlevitsk/qemu/tests/qemu-iotests/../../qemu-nbd" 
IMGFMT-- qcow2 (compat=1.1)
IMGPROTO  -- file
PLATFORM  -- Linux/x86_64 maximlenovopc 5.3.0-rc4+
TEST_DIR  -- /home/mlevitsk/qemu/tests/qemu-iotests/scratch
SOCKET_SCM_HELPER -- /home/mlevitsk/qemu/tests/qemu-iotests/socket_scm_helper

162  fail   [12:25:07] [12:27:18]output mismatch 
(see 162.out.bad)
--- /home/mlevitsk/USERSPACE/qemu/src/tests/qemu-iotests/162.out
2019-04-07 16:37:54.656311574 +0300
+++ /home/mlevitsk/qemu/tests/qemu-iotests/162.out.bad  2019-09-10 
12:27:18.032903571 +0300
@@ -1,7 +1,7 @@
 QA output created by 162
 
 === NBD ===
-qemu-img: Could not open 'json:{"driver": "nbd", "host": 42}': Failed to 
connect socket: Invalid argument
+qemu-img: Could not open 'json:{"driver": "nbd",

[Qemu-block] [PATCH v11 14/14] block/backup: use backup-top instead of write notifiers

[Vladimir Sementsov-Ogievskiy]

Drop write notifiers and use filter node instead.

= Changes =

1. Add filter-node-name argument for backup qmp api. We have to do it
in this commit, as 257 needs to be fixed.

2. There are no more write notifiers here, so is_write_notifier
parameter is dropped from block-copy paths.

3. Intersecting requests handling changed, now synchronization between
backup-top, backup and guest writes are all done in block/block-copy.c
and works as follows:

On copy operation, we work only with dirty areas. If bits are dirty it
means that there are no requests intersecting with this area. We clear
dirty bits and take bdrv range lock (bdrv_co_try_lock) on this area to
prevent further operations from interaction with guest (only with
guest, as neither backup nor backup-top will touch non-dirty area). If
copy-operation failed we set dirty bits back together with releasing
the lock.

The actual difference with old scheme is that on guest writes we
don't lock the whole region but only dirty-parts, and to be more
precise: only dirty-part we are currently operate on. In old scheme
guest write to non-dirty area (which may be safely ignored by backup)
may wait for intersecting request, touching some other area which is
dirty.

4. To sync with in-flight requests at job finish we now have drained
removing of the filter, we don't need rw-lock.

= Notes =

Note the consequence of three objects appearing: backup-top, backup job
and block-copy-state:

1. We want to insert backup-top before job creation, to behave similar
with mirror and commit, where job is started upon filter.

2. We also have to create block-copy-state after filter injection, as
we don't want its source child be replaced by filter. Instead we want
to keep BCS.source to be real source node, as we want to use
bdrv_co_try_lock in CBW operations and it can't be used on filter, as
on filter we already have in-flight (write) request from upper layer.

So, we firstly create inject backup-top, then create job and BCS. BCS
is the latest just to not create extra variable for it. Finally we set
bcs for backup-top filter.

= Iotest changes =

56: op-blocker doesn't shoot now, as we set it on source, but then
check on filter, when trying to start second backup.
To keep the test we instead can catch another collision: both jobs will
get 'drive0' job-id, as job-id parameter is unspecified. To prevent
interleaving with file-posix locks (as they are dependent on config)
let's use another target for second backup.

Also, it's obvious now that we'd like to drop this op-blocker at all
and add a test-case for two backups from one node (to different
destinations) actually works. But not in these series.

257: The test wants to emulate guest write during backup. They should
go to filter node, not to original source node, of course. Therefore we
need to specify filter node name and use it.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Max Reitz 
---
 qapi/block-core.json   |   8 +-
 include/block/block-copy.h |  10 +-
 include/block/block_int.h  |   1 +
 block/backup-top.c |  14 +-
 block/backup.c | 112 +++---
 block/block-copy.c |  45 --
 block/replication.c|   2 +-
 blockdev.c |   1 +
 tests/qemu-iotests/056 |   8 +-
 tests/qemu-iotests/257 |   7 +-
 tests/qemu-iotests/257.out | 306 ++---
 11 files changed, 237 insertions(+), 277 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index e6edd641f1..b5cd00c361 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1391,6 +1391,11 @@
 #list without user intervention.
 #Defaults to true. (Since 2.12)
 #
+# @filter-node-name: the node name that should be assigned to the
+#filter driver that the backup job inserts into the graph
+#above node specified by @drive. If this option is not 
given,
+#a node name is autogenerated. (Since: 4.2)
+#
 # Note: @on-source-error and @on-target-error only affect background
 # I/O.  If an error occurs during a guest write request, the device's
 # rerror/werror actions will be used.
@@ -1404,7 +1409,8 @@
 '*compress': 'bool',
 '*on-source-error': 'BlockdevOnError',
 '*on-target-error': 'BlockdevOnError',
-'*auto-finalize': 'bool', '*auto-dismiss': 'bool' } }
+'*auto-finalize': 'bool', '*auto-dismiss': 'bool',
+'*filter-node-name': 'str' } }
 
 ##
 # @DriveBackup:
diff --git a/include/block/block-copy.h b/include/block/block-copy.h
index 54f90d0c9a..bd813280f0 100644
--- a/include/block/block-copy.h
+++ b/include/block/block-copy.h
@@ -20,6 +20,14 @@
 typedef void (*ProgressBytesCallbackFunc)(int64_t bytes, void *opaque);
 typedef void (*ProgressResetCallbackFunc)(void *opaque);
 typedef struct BlockCopyState {
+/*
+ * block-copy shares WRITE permission on source, but it relies on user to
+   

[Qemu-block] [PATCH v11 00/14] backup-top filter driver for backup

[Vladimir Sementsov-Ogievskiy]

Hi all!

These series introduce backup-top driver. It's a filter-node, which
do copy-before-write operation. Mirror uses filter-node for handling
guest writes, let's move to filter-node (from write-notifiers) for
backup too.

v11: based on Kevin's block branch

02,03: Add Max's r-b
04: - improve comments
- rebase on dropped backup_drain
- s/job/bcs/ in trace events
05: one hunk dropped due to rebasing
on dropped backup_drain, still,
keep Max's r-b
06: rebased on 04 changes, keep Max's r-b
08,13,14: add Max's r-b

Based-on: git://repo.or.cz/qemu/kevin.git block

Vladimir Sementsov-Ogievskiy (14):
  block/backup: fix backup_cow_with_offload for last cluster
  block/backup: split shareable copying part from backup_do_cow
  block/backup: improve comment about image fleecing
  block/backup: introduce BlockCopyState
  block/backup: fix block-comment style
  block: move block_copy from block/backup.c to separate file
  block: teach bdrv_debug_breakpoint skip filters with backing
  iotests: prepare 124 and 257 bitmap querying for backup-top filter
  iotests: 257: drop unused Drive.device field
  iotests: 257: drop device_add
  block/io: refactor wait_serialising_requests
  block: add lock/unlock range functions
  block: introduce backup-top filter driver
  block/backup: use backup-top instead of write notifiers

 qapi/block-core.json  |   8 +-
 block/backup-top.h|  37 ++
 include/block/block-copy.h|  84 
 include/block/block_int.h |   5 +
 block.c   |  34 +-
 block/backup-top.c| 240 
 block/backup.c| 440 -
 block/block-copy.c| 337 
 block/io.c|  68 +++-
 block/replication.c   |   2 +-
 blockdev.c|   1 +
 block/Makefile.objs   |   3 +
 block/trace-events|  14 +-
 tests/qemu-iotests/056|   8 +-
 tests/qemu-iotests/124|  83 ++--
 tests/qemu-iotests/257|  91 ++---
 tests/qemu-iotests/257.out| 714 ++
 tests/qemu-iotests/iotests.py |  27 ++
 18 files changed, 1278 insertions(+), 918 deletions(-)
 create mode 100644 block/backup-top.h
 create mode 100644 include/block/block-copy.h
 create mode 100644 block/backup-top.c
 create mode 100644 block/block-copy.c

-- 
2.18.0

[Qemu-block] [PATCH v11 10/14] iotests: 257: drop device_add

[Vladimir Sementsov-Ogievskiy]

SCSI devices are unused in test, drop them.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Max Reitz 
---
 tests/qemu-iotests/257 |  8 ---
 tests/qemu-iotests/257.out | 44 --
 2 files changed, 52 deletions(-)

diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
index 5d77202157..de8b45f094 100755
--- a/tests/qemu-iotests/257
+++ b/tests/qemu-iotests/257
@@ -325,12 +325,6 @@ def test_bitmap_sync(bsync_mode, msync_mode='bitmap', 
failure=None):
node_name=drive0.node,
driver=drive0.fmt,
file=file_config)
-# Use share-rw to allow writes directly to the node;
-# The anonymous block-backend for this configuration prevents us
-# from using HMP's qemu-io commands to address the device.
-vm.qmp_log("device_add", id='device0',
-   drive=drive0.node, driver="scsi-hd",
-   share_rw=True)
 log('')
 
 # 0 - Writes and Reference Backup
@@ -467,8 +461,6 @@ def test_backup_api():
node_name=drive0.node,
driver=drive0.fmt,
file=file_config)
-vm.qmp_log("device_add", id='device0',
-   drive=drive0.node, driver="scsi-hd")
 log('')
 
 target0 = Drive(backup_path, vm=vm)
diff --git a/tests/qemu-iotests/257.out b/tests/qemu-iotests/257.out
index c9b4b68232..ec7e25877b 100644
--- a/tests/qemu-iotests/257.out
+++ b/tests/qemu-iotests/257.out
@@ -5,8 +5,6 @@
 
 {"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": 
{"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
 {"return": {}}
-{"execute": "device_add", "arguments": {"drive": "drive0", "driver": 
"scsi-hd", "id": "device0", "share-rw": true}}
-{"return": {}}
 
 --- Write #0 ---
 
@@ -267,8 +265,6 @@ qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" 
==> Identical, OK!
 
 {"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": 
{"driver": "blkdebug", "image": {"driver": "file", "filename": 
"TEST_DIR/PID-img"}, "inject-error": [{"errno": 5, "event": "read_aio", 
"immediately": false, "once": true, "state": 3}], "set-state": [{"event": 
"flush_to_disk", "new-state": 2, "state": 1}, {"event": "read_aio", 
"new-state": 3, "state": 2}]}, "node-name": "drive0"}}
 {"return": {}}
-{"execute": "device_add", "arguments": {"drive": "drive0", "driver": 
"scsi-hd", "id": "device0", "share-rw": true}}
-{"return": {}}
 
 --- Write #0 ---
 
@@ -480,8 +476,6 @@ qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" 
==> Identical, OK!
 
 {"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": 
{"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
 {"return": {}}
-{"execute": "device_add", "arguments": {"drive": "drive0", "driver": 
"scsi-hd", "id": "device0", "share-rw": true}}
-{"return": {}}
 
 --- Write #0 ---
 
@@ -742,8 +736,6 @@ qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" 
==> Identical, OK!
 
 {"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": 
{"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
 {"return": {}}
-{"execute": "device_add", "arguments": {"drive": "drive0", "driver": 
"scsi-hd", "id": "device0", "share-rw": true}}
-{"return": {}}
 
 --- Write #0 ---
 
@@ -1004,8 +996,6 @@ qemu_img compare "TEST_DIR/PID-img" 
"TEST_DIR/PID-fbackup2" ==> Identical, OK!
 
 {"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": 
{"driver": "blkdebug", "image": {"driver": "file", "filename": 
"TEST_DIR/PID-img"}, "inject-error": [{"errno": 5, "event": "read_aio", 
"immediately": false, "once": true, "state": 3}], "set-state": [{"event": 
"flush_to_disk", "new-state": 2, "state": 1}, {"event": "read_aio", 
"new-state": 3, "state": 2}]}, "node-name": "drive0"}}
 {"return": {}}
-{"execute": "device_add", "arguments": {"drive": "drive0", "driver": 
"scsi-hd", "id": "device0", "share-rw": true}}
-{"return": {}}
 
 --- Write #0 ---
 
@@ -1217,8 +1207,6 @@ qemu_img compare "TEST_DIR/PID-img" 
"TEST_DIR/PID-fbackup2" ==> Identical, OK!
 
 {"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": 
{"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
 {"return": {}}
-{"execute": "device_add", "arguments": {"drive": "drive0", "driver": 
"scsi-hd", "id": "device0", "share-rw": true}}
-{"return": {}}
 
 --- Write #0 ---
 
@@ -1479,8 +1467,6 @@ qemu_img compare "TEST_DIR/PID-img" 
"TEST_DIR/PID-fbackup2" ==> Identical, OK!
 
 {"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": 
{"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
 {"return": {}}
-{"execute": "device_add", "arguments": {"drive": "drive0", "driver": 
"scsi-hd", "id": "device0", "share-rw": true}}
-{"return": {}}
 
 --- Write #0 ---
 
@@ -1741,8 +1727,6 @@ qemu_img compare "TEST_DIR/PID-img" 
"TEST_DIR/P

[Qemu-block] [PATCH v11 13/14] block: introduce backup-top filter driver

[Vladimir Sementsov-Ogievskiy]

Backup-top filter caches write operations and does copy-before-write
operations.

The driver will be used in backup instead of write-notifiers.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Max Reitz 
---
 block/backup-top.h  |  37 +++
 block/backup-top.c  | 244 
 block/Makefile.objs |   2 +
 3 files changed, 283 insertions(+)
 create mode 100644 block/backup-top.h
 create mode 100644 block/backup-top.c

diff --git a/block/backup-top.h b/block/backup-top.h
new file mode 100644
index 00..67de7a9133
--- /dev/null
+++ b/block/backup-top.h
@@ -0,0 +1,37 @@
+/*
+ * backup-top filter driver
+ *
+ * The driver performs Copy-Before-Write (CBW) operation: it is injected above
+ * some node, and before each write it copies _old_ data to the target node.
+ *
+ * Copyright (c) 2018-2019 Virtuozzo International GmbH.
+ *
+ * Author:
+ *  Sementsov-Ogievskiy Vladimir 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see .
+ */
+
+#ifndef BACKUP_TOP_H
+#define BACKUP_TOP_H
+
+#include "block/block_int.h"
+
+BlockDriverState *bdrv_backup_top_append(BlockDriverState *source,
+ const char *filter_node_name,
+ Error **errp);
+void bdrv_backup_top_set_bcs(BlockDriverState *bs, BlockCopyState *copy_state);
+void bdrv_backup_top_drop(BlockDriverState *bs);
+
+#endif /* BACKUP_TOP_H */
diff --git a/block/backup-top.c b/block/backup-top.c
new file mode 100644
index 00..0991b64759
--- /dev/null
+++ b/block/backup-top.c
@@ -0,0 +1,244 @@
+/*
+ * backup-top filter driver
+ *
+ * The driver performs Copy-Before-Write (CBW) operation: it is injected above
+ * some node, and before each write it copies _old_ data to the target node.
+ *
+ * Copyright (c) 2018-2019 Virtuozzo International GmbH.
+ *
+ * Author:
+ *  Sementsov-Ogievskiy Vladimir 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see .
+ */
+
+#include "qemu/osdep.h"
+
+#include "sysemu/block-backend.h"
+#include "qemu/cutils.h"
+#include "qapi/error.h"
+#include "block/block_int.h"
+#include "block/qdict.h"
+#include "block/block-copy.h"
+
+#include "block/backup-top.h"
+
+typedef struct BDRVBackupTopState {
+BlockCopyState *bcs;
+bool active;
+} BDRVBackupTopState;
+
+static coroutine_fn int backup_top_co_preadv(
+BlockDriverState *bs, uint64_t offset, uint64_t bytes,
+QEMUIOVector *qiov, int flags)
+{
+return bdrv_co_preadv(bs->backing, offset, bytes, qiov, flags);
+}
+
+static coroutine_fn int backup_top_cbw(BlockDriverState *bs, uint64_t offset,
+   uint64_t bytes)
+{
+/*
+ * Here we'd like to use block_copy(), but it needs some additional
+ * synchronization mechanism to prevent intersecting guest writes during
+ * copy operation. The will appear in further commit (it should be done
+ * together with moving backup to using of backup-top and to the same
+ * synchronization mechanism), and for now it is a TODO.
+ */
+
+abort();
+}
+
+static int coroutine_fn backup_top_co_pdiscard(BlockDriverState *bs,
+   int64_t offset, int bytes)
+{
+int ret = backup_top_cbw(bs, offset, bytes);
+if (ret < 0) {
+return ret;
+}
+
+return bdrv_co_pdiscard(bs->backing, offset, bytes);
+}
+
+static int coroutine_fn backup_top_co_pwrite_zeroes(BlockDriverState *bs,
+int64_t offset, int bytes, BdrvRequestFlags flags)
+{
+int ret = backup_top_cbw(bs, offset, bytes);
+if (ret < 0) {
+return ret;
+}
+
+return bdrv_co_pwrite_zeroes(bs->backing, offset, bytes, flags);
+}
+
+static coroutine_fn int backup_top_co_pwritev(BlockDriverState *bs,
+

[Qemu-block] [PATCH v11 04/14] block/backup: introduce BlockCopyState

[Vladimir Sementsov-Ogievskiy]

Split copying code part from backup to "block-copy", including separate
state structure and function renaming. This is needed to share it with
backup-top filter driver in further commits.

Notes:

1. As BlockCopyState keeps own BlockBackend objects, remaining
job->common.blk users only use it to get bs by blk_bs() call, so clear
job->commen.blk permissions set in block_job_create and add
job->source_bs to be used instead of blk_bs(job->common.blk), to keep
it more clear which bs we use when introduce backup-top filter in
further commit.

2. Rename s/initializing_bitmap/skip_unallocated/ to sound a bit better
as interface to BlockCopyState

3. Split is not very clean: there left some duplicated fields, backup
code uses some BlockCopyState fields directly, let's postpone it for
further improvements and keep this comment simpler for review.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block/backup.c | 356 -
 block/trace-events |  12 +-
 2 files changed, 231 insertions(+), 137 deletions(-)

diff --git a/block/backup.c b/block/backup.c
index 56a4bae61e..e5bcfe7177 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -35,12 +35,52 @@ typedef struct CowRequest {
 CoQueue wait_queue; /* coroutines blocked on this request */
 } CowRequest;
 
+typedef void (*ProgressBytesCallbackFunc)(int64_t bytes, void *opaque);
+typedef void (*ProgressResetCallbackFunc)(void *opaque);
+typedef struct BlockCopyState {
+BlockBackend *source;
+BlockBackend *target;
+BdrvDirtyBitmap *copy_bitmap;
+int64_t cluster_size;
+bool use_copy_range;
+int64_t copy_range_size;
+uint64_t len;
+
+BdrvRequestFlags write_flags;
+
+/*
+ * skip_unallocated:
+ *
+ * Used by sync=top jobs, which first scan the source node for unallocated
+ * areas and clear them in the copy_bitmap.  During this process, the 
bitmap
+ * is thus not fully initialized: It may still have bits set for areas that
+ * are unallocated and should actually not be copied.
+ *
+ * This is indicated by skip_unallocated.
+ *
+ * In this case, block_copy() will query the source’s allocation status,
+ * skip unallocated regions, clear them in the copy_bitmap, and invoke
+ * block_copy_reset_unallocated() every time it does.
+ */
+bool skip_unallocated;
+
+/* progress_bytes_callback: called when some copying progress is done. */
+ProgressBytesCallbackFunc progress_bytes_callback;
+
+/*
+ * progress_reset_callback: called when some bytes reset from copy_bitmap
+ * (see @skip_unallocated above). The callee is assumed to recalculate how
+ * many bytes remain based on the dirty bit count of copy_bitmap.
+ */
+ProgressResetCallbackFunc progress_reset_callback;
+void *progress_opaque;
+} BlockCopyState;
+
 typedef struct BackupBlockJob {
 BlockJob common;
-BlockBackend *target;
+BlockDriverState *source_bs;
 
 BdrvDirtyBitmap *sync_bitmap;
-BdrvDirtyBitmap *copy_bitmap;
 
 MirrorSyncMode sync_mode;
 BitmapSyncMode bitmap_mode;
@@ -53,11 +93,7 @@ typedef struct BackupBlockJob {
 NotifierWithReturn before_write;
 QLIST_HEAD(, CowRequest) inflight_reqs;
 
-bool use_copy_range;
-int64_t copy_range_size;
-
-BdrvRequestFlags write_flags;
-bool initializing_bitmap;
+BlockCopyState *bcs;
 } BackupBlockJob;
 
 static const BlockJobDriver backup_job_driver;
@@ -99,9 +135,88 @@ static void cow_request_end(CowRequest *req)
 qemu_co_queue_restart_all(&req->wait_queue);
 }
 
+static void block_copy_state_free(BlockCopyState *s)
+{
+if (!s) {
+return;
+}
+
+bdrv_release_dirty_bitmap(blk_bs(s->source), s->copy_bitmap);
+blk_unref(s->source);
+blk_unref(s->target);
+g_free(s);
+}
+
+static BlockCopyState *block_copy_state_new(
+BlockDriverState *source, BlockDriverState *target,
+int64_t cluster_size, BdrvRequestFlags write_flags,
+ProgressBytesCallbackFunc progress_bytes_callback,
+ProgressResetCallbackFunc progress_reset_callback,
+void *progress_opaque, Error **errp)
+{
+BlockCopyState *s;
+int ret;
+uint64_t no_resize = BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE |
+ BLK_PERM_WRITE_UNCHANGED | BLK_PERM_GRAPH_MOD;
+BdrvDirtyBitmap *copy_bitmap;
+
+copy_bitmap = bdrv_create_dirty_bitmap(source, cluster_size, NULL, errp);
+if (!copy_bitmap) {
+return NULL;
+}
+bdrv_disable_dirty_bitmap(copy_bitmap);
+
+s = g_new(BlockCopyState, 1);
+*s = (BlockCopyState) {
+.source = blk_new(bdrv_get_aio_context(source),
+  BLK_PERM_CONSISTENT_READ, no_resize),
+.target = blk_new(bdrv_get_aio_context(target),
+  BLK_PERM_WRITE, no_resize),
+.copy_bitmap = copy_bitmap,
+.cluster_size = cluster_size,
+.len = bdrv_dirty_bitmap_size(copy_bitmap),
+.write

[Qemu-block] [PATCH v11 06/14] block: move block_copy from block/backup.c to separate file

[Vladimir Sementsov-Ogievskiy]

Split block_copy to separate file, to be cleanly shared with backup-top
filter driver in further commits.

It's a clean movement, the only change is drop "static" from interface
functions.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Max Reitz 
---
 include/block/block-copy.h |  76 
 block/backup.c | 346 +
 block/block-copy.c | 324 ++
 block/Makefile.objs|   1 +
 block/trace-events |   2 +
 5 files changed, 404 insertions(+), 345 deletions(-)
 create mode 100644 include/block/block-copy.h
 create mode 100644 block/block-copy.c

diff --git a/include/block/block-copy.h b/include/block/block-copy.h
new file mode 100644
index 00..54f90d0c9a
--- /dev/null
+++ b/include/block/block-copy.h
@@ -0,0 +1,76 @@
+/*
+ * block_copy API
+ *
+ * Copyright (C) 2013 Proxmox Server Solutions
+ * Copyright (c) 2019 Virtuozzo International GmbH.
+ *
+ * Authors:
+ *  Dietmar Maurer (diet...@proxmox.com)
+ *  Vladimir Sementsov-Ogievskiy 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef BLOCK_COPY_H
+#define BLOCK_COPY_H
+
+#include "block/block.h"
+
+typedef void (*ProgressBytesCallbackFunc)(int64_t bytes, void *opaque);
+typedef void (*ProgressResetCallbackFunc)(void *opaque);
+typedef struct BlockCopyState {
+BlockBackend *source;
+BlockBackend *target;
+BdrvDirtyBitmap *copy_bitmap;
+int64_t cluster_size;
+bool use_copy_range;
+int64_t copy_range_size;
+uint64_t len;
+
+BdrvRequestFlags write_flags;
+
+/*
+ * skip_unallocated:
+ *
+ * Used by sync=top jobs, which first scan the source node for unallocated
+ * areas and clear them in the copy_bitmap.  During this process, the 
bitmap
+ * is thus not fully initialized: It may still have bits set for areas that
+ * are unallocated and should actually not be copied.
+ *
+ * This is indicated by skip_unallocated.
+ *
+ * In this case, block_copy() will query the source’s allocation status,
+ * skip unallocated regions, clear them in the copy_bitmap, and invoke
+ * block_copy_reset_unallocated() every time it does.
+ */
+bool skip_unallocated;
+
+/* progress_bytes_callback: called when some copying progress is done. */
+ProgressBytesCallbackFunc progress_bytes_callback;
+
+/*
+ * progress_reset_callback: called when some bytes reset from copy_bitmap
+ * (see @skip_unallocated above). The callee is assumed to recalculate how
+ * many bytes remain based on the dirty bit count of copy_bitmap.
+ */
+ProgressResetCallbackFunc progress_reset_callback;
+void *progress_opaque;
+} BlockCopyState;
+
+BlockCopyState *block_copy_state_new(
+BlockDriverState *source, BlockDriverState *target,
+int64_t cluster_size, BdrvRequestFlags write_flags,
+ProgressBytesCallbackFunc progress_bytes_callback,
+ProgressResetCallbackFunc progress_reset_callback,
+void *progress_opaque, Error **errp);
+
+void block_copy_state_free(BlockCopyState *s);
+
+int64_t block_copy_reset_unallocated(BlockCopyState *s,
+ int64_t offset, int64_t *count);
+
+int coroutine_fn block_copy(BlockCopyState *s, int64_t start, uint64_t bytes,
+bool *error_is_read, bool is_write_notifier);
+
+#endif /* BLOCK_COPY_H */
diff --git a/block/backup.c b/block/backup.c
index 7a99ff14aa..4613b8c88d 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -18,6 +18,7 @@
 #include "block/block_int.h"
 #include "block/blockjob_int.h"
 #include "block/block_backup.h"
+#include "block/block-copy.h"
 #include "qapi/error.h"
 #include "qapi/qmp/qerror.h"
 #include "qemu/ratelimit.h"
@@ -35,47 +36,6 @@ typedef struct CowRequest {
 CoQueue wait_queue; /* coroutines blocked on this request */
 } CowRequest;
 
-typedef void (*ProgressBytesCallbackFunc)(int64_t bytes, void *opaque);
-typedef void (*ProgressResetCallbackFunc)(void *opaque);
-typedef struct BlockCopyState {
-BlockBackend *source;
-BlockBackend *target;
-BdrvDirtyBitmap *copy_bitmap;
-int64_t cluster_size;
-bool use_copy_range;
-int64_t copy_range_size;
-uint64_t len;
-
-BdrvRequestFlags write_flags;
-
-/*
- * skip_unallocated:
- *
- * Used by sync=top jobs, which first scan the source node for unallocated
- * areas and clear them in the copy_bitmap.  During this process, the 
bitmap
- * is thus not fully initialized: It may still have bits set for areas that
- * are unallocated and should actually not be copied.
- *
- * This is indicated by skip_unallocated.
- *
- * In this case, block_copy() will query the source’s allocation status,
- * skip unallocated regions, clear them in the copy_bitmap, and invoke
- * block_copy_reset_unallocated() every