date:20190409

John Snow  writes:

> Presently we use hxtool and a .hx format to generate a few things like
> the qemu_img subcommand dispatch table, the qemu_img help() display output,
> and a help output in qemu-img.texi.
>
> Unfortunately, this means that this information is duplicated in at least
> three places:
>
> (1) in qemu-img-cmds.hx as a human readable string
> (2) in qemu-img-cmds.hx as a texi string
> (3) in qemu-img.texi again, as a texi string
>
> We can do a little better, though: all of these sources can be generated
> from a single json file. Add a new small tool ("pxtool") that can do this.
>
> This tool can at least handle generating (1) and (2) from the same source
> without needing to reduplicate that information. Deduplicating (3) happens
> in the next patch.
>
> Notes:
>  - The json format was chosen to be very "stupid", and the command line
>documentation is being kept one-per-line to make future diffs easier
>to read.
>  - It's easy enough to generate the human-readable output from the texi
>output by removing '@var{foo}' with 'foo'.
>  - The qemu-img command dispatch always calls img_cmdname, so we don't
>need to store this information separately, either.
>  - The need for DEF() macros could be removed as well, but I left it in
>to create a minimally disruptive patch.
> Signed-off-by: John Snow 

We got just five .hx:

qemu-img.cmds.hxkilled off by this patch
qemu-options.hx CLI QAPIfication should kill this off
hw/audio/pl041.hx   misnamed, not actually food for hxtool
hmp-commands.hx no exit strategy
hmp-commands-info.hxfor these two, yet

CLI QAPIfication got stuck in the back-burner, and as long as that's the
case, it's not an alternative to your patches.

Re: [Qemu-devel] [PATCH for-4.0-maybe] device_tree: Fix integer overflowing in load_device_tree()

On 4/10/19 7:28 AM, Markus Armbruster wrote:
> Philippe Mathieu-Daudé  writes:
>> On 4/9/19 7:40 PM, Markus Armbruster wrote:
>>> If the value of get_image_size() exceeds INT_MAX / 2 - 1, the
>>> computation of @dt_size overflows to a negative number, which then
>>> gets converted to a very large size_t for g_malloc0() and
>>> load_image_size().  In the (fortunately improbable) case g_malloc0()
>>> succeeds and load_image_size() survives, we'd assign the negative
>>> number to *sizep.  What that would do to the callers I can't say, but
>>> it's unlikely to be good.
>>>
>>> Fix by rejecting images whose size would overflow.
>>>
>>> Signed-off-by: Markus Armbruster 
>>> ---
>>>  device_tree.c | 4 
>>>  1 file changed, 4 insertions(+)
>>>
>>> diff --git a/device_tree.c b/device_tree.c
>>> index 296278e12a..f8b46b3c73 100644
>>> --- a/device_tree.c
>>> +++ b/device_tree.c
>>> @@ -84,6 +84,10 @@ void *load_device_tree(const char *filename_path, int 
>>> *sizep)
>>>   filename_path);
>>>  goto fail;
>>>  }
>>> +if (dt_size > INT_MAX / 2 - 1) {
>>
>> We should avoid magic number duplication.
>> That said, this patch looks safe.
>>
>> Reviewed-by: Philippe Mathieu-Daudé 
> 
> Thanks!
> 
>> BTW how did you figure that out?
> 
> Downstream handling of upstream commit da885fe1ee8 led me to the
> function.  I spotted dt_size = get_image_size(filename_path).
> Experience has taught me to check the left hand side's type.  Bad.  Then
> I saw how dt_size gets increased.  Worse.

So you genuinely neglected to mention Kurtis Miller then :)

>>> +error_report("Device tree file '%s' is too large", filename_path);
>>> +goto fail;
>>> +}
>>>  
>>>  /* Expand to 2x size to give enough room for manipulation.  */
>>>  dt_size += 1;
>>>

Re: [Qemu-devel] [PATCH] qemu-img: fix .hx and .texi disparity

John Snow  writes:

> It turns out that having options listed in three places continues to be
> a bad idea. I'm still toying with the idea of an improved infrastructure
> here, but in the meantime, another bandaid.
>
> There are three locations:
> (1) .hx file, formatted as texi
> (2) .hx file, formatted as human readable.
> (3) .texi file, as section headers, formatted as texi.
>
> You can compare the two summaries within the .hx file like so:
>
> Human-readable command summaries:
> `./qemu-img --help | grep 'Command syntax' -A14`
> Detokenized texi command summaries:
> `grep "@item" qemu-img-cmds.hx | sed -E 's|@var\{([^\}]*?)\}|\1|g'`
>
> You can compare the two separate texi summaries like so:
>
> Texi command summaries:
> `grep "@item" qemu-img-cmds.hx"`
> Texi command headers:
> grep -E "@item.*@var" qemu-img.texi | tail -14
>
> Signed-off-by: John Snow 

Extra points for explaining how you found the issues in detail
sufficient for reproducing.

Nice to have in 4.0.

Reviewed-by: Markus Armbruster

[Qemu-devel] [PATCH] roms: List and describe the Makefile 'clean' rule

Signed-off-by: Philippe Mathieu-Daudé 
---
 roms/Makefile | 10 --
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/roms/Makefile b/roms/Makefile
index 1ff78b63bb3..f55c4a2d3bb 100644
--- a/roms/Makefile
+++ b/roms/Makefile
@@ -61,6 +61,8 @@ default:
@echo "  skiboot-- update skiboot.lid"
@echo "  u-boot.e500-- update u-boot.e500"
@echo "  u-boot.sam460  -- update u-boot.sam460"
+   @echo "  clean  -- delete the files generated by the previous" \
+ "build targets"
 
 bios: build-seabios-config-seabios-128k build-seabios-config-seabios-256k
cp seabios/builds/seabios-128k/bios.bin ../pc-bios/bios.bin
@@ -102,7 +104,7 @@ pxe-rom-%: build-pxe-roms
 
 efirom: $(patsubst %,efi-rom-%,$(pxerom_variants))
 
-efi-rom-%: build-pxe-roms build-efi-roms $(EDK2_EFIROM)
+efi-rom-%: build-pxe-roms build-efi-roms edk2-basetools
$(EDK2_EFIROM) -f "0x$(VID)" -i "0x$(DID)" -l 0x02 \
-b ipxe/src/bin/$(VID)$(DID).rom \
-ec ipxe/src/bin-i386-efi/$(VID)$(DID).efidrv \
@@ -131,7 +133,7 @@ build-efi-roms: build-pxe-roms
 #EDK2_BASETOOLS_LDFLAGS='...' \
 #efirom
 #
-$(EDK2_EFIROM):
+edk2-basetools:
$(MAKE) -C edk2/BaseTools \
EXTRA_OPTFLAGS='$(EDK2_BASETOOLS_OPTFLAGS)' \
EXTRA_LDFLAGS='$(EDK2_BASETOOLS_LDFLAGS)'
@@ -156,6 +158,9 @@ skiboot:
$(MAKE) -C skiboot CROSS=$(powerpc64_cross_prefix)
cp skiboot/skiboot.lid ../pc-bios/skiboot.lid
 
+efi: edk2-basetools
+   $(MAKE) -f Makefile.edk2
+
 clean:
rm -rf seabios/.config seabios/out seabios/builds
$(MAKE) -C sgabios clean
@@ -166,3 +171,4 @@ clean:
rm -rf u-boot/build.e500
$(MAKE) -C u-boot-sam460ex distclean
$(MAKE) -C skiboot clean
+   $(MAKE) -f Makefile.edk2 clean
-- 
2.20.1

Re: [Qemu-devel] [PATCH for-4.0-maybe] device_tree: Fix integer overflowing in load_device_tree()

Peter Maydell  writes:

> On Tue, 9 Apr 2019 at 21:15, Alistair Francis  wrote:
>>
>> On Tue, Apr 9, 2019 at 1:08 PM Peter Maydell  
>> wrote:
>> >
>> > On Wed, 10 Apr 2019 at 00:40, Markus Armbruster  wrote:
>> > >
>> > > If the value of get_image_size() exceeds INT_MAX / 2 - 1, the
>> > > computation of @dt_size overflows to a negative number, which then
>> > > gets converted to a very large size_t for g_malloc0() and
>> > > load_image_size().  In the (fortunately improbable) case g_malloc0()
>> > > succeeds and load_image_size() survives, we'd assign the negative
>> > > number to *sizep.  What that would do to the callers I can't say, but
>> > > it's unlikely to be good.
>> > >
>> > > Fix by rejecting images whose size would overflow.
>> > >
>> > > Signed-off-by: Markus Armbruster 
>> >
>> > I think this patch is missing some attributions for the
>> > security researchers who found the issue initially.
>> > PJP's patch for this from a couple of weeks back has a
>> > reported-by credit:
>> > https://patchew.org/QEMU/20190322073555.20889-1-ppan...@redhat.com/

Uh, I missed that thread.  Thanks for doing my homework for me!

>> It seems like from that discussion that this patch is the correct approach.
>>
>> I can add the attributions and send a PR for 4.0. I'll send it by EOD
>> unless anyone has any objections.
>
> Thanks. I think given it's 21:30 here I'm going to postpone
> tagging rc3 til tomorrow (mid-afternoon UK time). I'm still
> hoping we can avoid an rc4...

Want me to look for a few more integer overflows today?  ;-P

Re: [Qemu-devel] [PATCH for-4.0-maybe] device_tree: Fix integer overflowing in load_device_tree()

Philippe Mathieu-Daudé  writes:

> On 4/9/19 7:40 PM, Markus Armbruster wrote:
>> If the value of get_image_size() exceeds INT_MAX / 2 - 1, the
>> computation of @dt_size overflows to a negative number, which then
>> gets converted to a very large size_t for g_malloc0() and
>> load_image_size().  In the (fortunately improbable) case g_malloc0()
>> succeeds and load_image_size() survives, we'd assign the negative
>> number to *sizep.  What that would do to the callers I can't say, but
>> it's unlikely to be good.
>> 
>> Fix by rejecting images whose size would overflow.
>> 
>> Signed-off-by: Markus Armbruster 
>> ---
>>  device_tree.c | 4 
>>  1 file changed, 4 insertions(+)
>> 
>> diff --git a/device_tree.c b/device_tree.c
>> index 296278e12a..f8b46b3c73 100644
>> --- a/device_tree.c
>> +++ b/device_tree.c
>> @@ -84,6 +84,10 @@ void *load_device_tree(const char *filename_path, int 
>> *sizep)
>>   filename_path);
>>  goto fail;
>>  }
>> +if (dt_size > INT_MAX / 2 - 1) {
>
> We should avoid magic number duplication.
> That said, this patch looks safe.
>
> Reviewed-by: Philippe Mathieu-Daudé 

Thanks!

> BTW how did you figure that out?

Downstream handling of upstream commit da885fe1ee8 led me to the
function.  I spotted dt_size = get_image_size(filename_path).
Experience has taught me to check the left hand side's type.  Bad.  Then
I saw how dt_size gets increased.  Worse.

>> +error_report("Device tree file '%s' is too large", filename_path);
>> +goto fail;
>> +}
>>  
>>  /* Expand to 2x size to give enough room for manipulation.  */
>>  dt_size += 1;
>>

Re: [Qemu-devel] [PATCH for-4.0-maybe] device_tree: Fix integer overflowing in load_device_tree()

On 4/9/19 10:08 PM, Peter Maydell wrote:
> On Wed, 10 Apr 2019 at 00:40, Markus Armbruster  wrote:
>>
>> If the value of get_image_size() exceeds INT_MAX / 2 - 1, the
>> computation of @dt_size overflows to a negative number, which then
>> gets converted to a very large size_t for g_malloc0() and
>> load_image_size().  In the (fortunately improbable) case g_malloc0()
>> succeeds and load_image_size() survives, we'd assign the negative
>> number to *sizep.  What that would do to the callers I can't say, but
>> it's unlikely to be good.
>>
>> Fix by rejecting images whose size would overflow.
>>
>> Signed-off-by: Markus Armbruster 
> 
> I think this patch is missing some attributions for the
> security researchers who found the issue initially.
> PJP's patch for this from a couple of weeks back has a
> reported-by credit:
> https://patchew.org/QEMU/20190322073555.20889-1-ppan...@redhat.com/

Oh I missed that thread while on PTO. This answers my "how did you
figure that out?" question :)

Thanks,

Phil.

Re: [Qemu-devel] [PATCH for-4.1 v4 07/12] roms: build edk2 firmware binaries and variable store templates

On 4/10/19 1:00 AM, Laszlo Ersek wrote:
> Add the "efi" target to "Makefile".
> 
> Introduce "Makefile.edk2" for building and cleaning the firmware images
> and varstore templates.
> 
> Collect the common bits from the recipes in the helper script
> "edk2-build.sh".
> 
> Signed-off-by: Laszlo Ersek 
> Reviewed-by: Michal Privoznik 
> Reviewed-by: Philippe Mathieu-Daudé 
> Reviewed-by: Michael S. Tsirkin 

Tested-by: Philippe Mathieu-Daudé 

> ---
> 
> Notes:
> v4:
> 
> - no change
> 
> v3:
> 
> - pick up Michal's R-b
> 
> - pick up Phil's R-b
> 
> - pick up Michael's R-b
> 
> - compress FD files with bzip2 rather than xz, so that decompression at
>   "make install" time succeed on older build OSes too [Peter]
> 
> - do not pick up Phil's T-b, consequently
> 
> - do not pick up Igor's T-b for the same reason
> 
> v2:
> 
> - drop comma after copyright year, in both new files [Eric]
> 
> - define the SHELL macro as /bin/bash near the top of "Makefile.edk2",
>   so that various bashisms (e.g. the "source" built-in, and the ==
>   operator of "[") work even if /bin/sh isn't bash [Phil, Eric]
> 
> - rework "Makefile.edk2" to produce xz-compressed flash device files
>   [Dan, Michael, Phil]:
> 
>   - add implicit rule for compression,
> 
>   - mark uncompressed FD files as intermediate,
> 
>   - factor out the "flashdevs" macro for sharing between the "all" and
> ".INTERMEDIATE" targets
> 
> - due to said rework above, do not pick up Phil's R-b / T-b, and
>   Michal's and Michael's R-b's
> 
>  roms/Makefile  |   5 +
>  roms/Makefile.edk2 | 148 
>  roms/edk2-build.sh |  55 
>  3 files changed, 208 insertions(+)
> 
> diff --git a/roms/Makefile b/roms/Makefile
> index 93c3d467be14..0ce84a45ad57 100644
> --- a/roms/Makefile
> +++ b/roms/Makefile
> @@ -61,6 +61,7 @@ default:
>   @echo "  skiboot-- update skiboot.lid"
>   @echo "  u-boot.e500-- update u-boot.e500"
>   @echo "  u-boot.sam460  -- update u-boot.sam460"
> + @echo "  efi-- update UEFI (edk2) platform firmware"
>  
>  bios: build-seabios-config-seabios-128k build-seabios-config-seabios-256k
>   cp seabios/builds/seabios-128k/bios.bin ../pc-bios/bios.bin
> @@ -156,6 +157,9 @@ skiboot:
>   $(MAKE) -C skiboot CROSS=$(powerpc64_cross_prefix)
>   cp skiboot/skiboot.lid ../pc-bios/skiboot.lid
>  
> +efi: edk2-basetools
> + $(MAKE) -f Makefile.edk2
> +
>  clean:
>   rm -rf seabios/.config seabios/out seabios/builds
>   $(MAKE) -C sgabios clean
> @@ -166,3 +170,4 @@ clean:
>   rm -rf u-boot/build.e500
>   $(MAKE) -C u-boot-sam460ex distclean
>   $(MAKE) -C skiboot clean
> + $(MAKE) -f Makefile.edk2 clean
> diff --git a/roms/Makefile.edk2 b/roms/Makefile.edk2
> new file mode 100644
> index ..822c547fec64
> --- /dev/null
> +++ b/roms/Makefile.edk2
> @@ -0,0 +1,148 @@
> +# Makefile for building firmware binaries and variable store templates for a
> +# number of virtual platforms in edk2.
> +#
> +# Copyright (C) 2019 Red Hat, Inc.
> +#
> +# This program and the accompanying materials are licensed and made available
> +# under the terms and conditions of the BSD License that accompanies this
> +# distribution. The full text of the license may be found at
> +# .
> +#
> +# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, 
> WITHOUT
> +# WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
> +
> +SHELL = /bin/bash
> +
> +toolchain = $(shell source ./edk2-funcs.sh && qemu_edk2_get_toolchain $(1))
> +
> +licenses := \
> + edk2/License.txt \
> + edk2/OvmfPkg/License.txt \
> + edk2/CryptoPkg/Library/OpensslLib/openssl/LICENSE
> +
> +# The "edk2-arm-vars.fd" varstore template is suitable for aarch64 as well.
> +# Similarly, the "edk2-i386-vars.fd" varstore template is suitable for x86_64
> +# as well, independently of "secure" too.
> +flashdevs := \
> + aarch64-code \
> + arm-code \
> + i386-code \
> + i386-secure-code \
> + x86_64-code \
> + x86_64-secure-code \
> + \
> + arm-vars \
> + i386-vars
> +
> +all: $(foreach flashdev,$(flashdevs),../pc-bios/edk2-$(flashdev).fd.bz2) \
> + ../pc-bios/edk2-licenses.txt
> +
> +../pc-bios/edk2-%.fd.bz2: ../pc-bios/edk2-%.fd
> + bzip2 -9 -c $< > $@
> +
> +# When the build completes, we need not keep the uncompressed flash device
> +# files.
> +.INTERMEDIATE: $(foreach 
> flashdev,$(flashdevs),../pc-bios/edk2-$(flashdev).fd)
> +
> +submodules:
> + cd edk2 && git submodule update --init --force
> +
> +# See notes on the ".NOTPARALLEL" target and the "+" indicator in
> +# "tests/uefi-test-tools/Makefile".
> +.NOTPARALLEL:
> +
> +../pc-bios/edk2-aarch64-code.fd: submodules
> + +./edk2-build.sh \
> +

Re: [Qemu-devel] [PATCH 0/4] Header cleanups

Peter Maydell  writes:

> On Tue, 9 Apr 2019 at 17:42, Aruna Jayasena  wrote:
>>
>> Yeah this is a stand alone patch.
>
> OK, thanks -- just wanted to check there wasn't a missing
> bit. I think this could go in via the qemu-trivial tree
> so I have cc'd them.
>
> We're in the middle of a release at the moment, but we
> should reopen for development in a couple of weeks. If
> this patch still hasn't gone anywhere by a week or so after
> that then please follow up with a 'ping' to check it hasn't
> been forgotten by accident.

Hopefully, qemu-trivial won't wait for the next development cycle to
begin just to queue patches for it.

Regardless, patch submitters should watch their patches and inquire when
they don't get up within a reasonable time frame.  Normally, waiting for
two weeks feels about right.  Perhaps give it an extra week or so when
the release circus is in town (like now).

Re: [Qemu-devel] [PATCH for-4.1 v4 09/12] pc-bios: document the edk2 firmware images; add firmware descriptors

On 4/10/19 1:00 AM, Laszlo Ersek wrote:
> Update the README file with information on the images added previously,
> and provide firmware descriptor documents that conform to
> "docs/interop/firmware.json".
> 
> Signed-off-by: Laszlo Ersek 
> Reviewed-by: Michal Privoznik 
> Reviewed-by: Michael S. Tsirkin 
> Tested-by: Igor Mammedov 

Reviewed-by: Philippe Mathieu-Daudé 

> ---
> 
> Notes:
> v4:
> 
> - no change
> 
> v3:
> 
> - refer to "edk2-*.fd.bz2" rather than "edk2-*.fd.xz" in the README
>   [Peter]
> 
> - pick up Igor's T-b still
> 
> v2:
> 
> - refer to "edk2-*.fd.xz" rather than "edk2-*.fd" in the README hunk
>   [Dan, Michael, Phil]
> 
> - pick up Michal's and Michael's R-b's
> 
>  pc-bios/descriptors/50-edk2-i386-secure.json   | 34 +++
>  pc-bios/descriptors/50-edk2-x86_64-secure.json | 35 
>  pc-bios/descriptors/60-edk2-aarch64.json   | 31 +
>  pc-bios/descriptors/60-edk2-arm.json   | 31 +
>  pc-bios/descriptors/60-edk2-i386.json  | 33 ++
>  pc-bios/descriptors/60-edk2-x86_64.json| 34 +++
>  pc-bios/README | 11 ++
>  7 files changed, 209 insertions(+)
> 
> diff --git a/pc-bios/descriptors/50-edk2-i386-secure.json 
> b/pc-bios/descriptors/50-edk2-i386-secure.json
> new file mode 100644
> index ..d7108c1da05a
> --- /dev/null
> +++ b/pc-bios/descriptors/50-edk2-i386-secure.json
> @@ -0,0 +1,34 @@
> +{
> +"description": "UEFI firmware for i386, with Secure Boot and SMM",
> +"interface-types": [
> +"uefi"
> +],
> +"mapping": {
> +"device": "flash",
> +"executable": {
> +"filename": "@DATADIR@/edk2-i386-secure-code.fd",
> +"format": "raw"
> +},
> +"nvram-template": {
> +"filename": "@DATADIR@/edk2-i386-vars.fd",
> +"format": "raw"
> +}
> +},
> +"targets": [
> +{
> +"architecture": "i386",
> +"machines": [
> +"pc-q35-*"
> +]
> +}
> +],
> +"features": [
> +"acpi-s3",
> +"requires-smm",
> +"secure-boot",
> +"verbose-dynamic"
> +],
> +"tags": [
> +
> +]
> +}
> diff --git a/pc-bios/descriptors/50-edk2-x86_64-secure.json 
> b/pc-bios/descriptors/50-edk2-x86_64-secure.json
> new file mode 100644
> index ..387eb356236b
> --- /dev/null
> +++ b/pc-bios/descriptors/50-edk2-x86_64-secure.json
> @@ -0,0 +1,35 @@
> +{
> +"description": "UEFI firmware for x86_64, with Secure Boot and SMM",
> +"interface-types": [
> +"uefi"
> +],
> +"mapping": {
> +"device": "flash",
> +"executable": {
> +"filename": "@DATADIR@/edk2-x86_64-secure-code.fd",
> +"format": "raw"
> +},
> +"nvram-template": {
> +"filename": "@DATADIR@/edk2-i386-vars.fd",
> +"format": "raw"
> +}
> +},
> +"targets": [
> +{
> +"architecture": "x86_64",
> +"machines": [
> +"pc-q35-*"
> +]
> +}
> +],
> +"features": [
> +"acpi-s3",
> +"amd-sev",
> +"requires-smm",
> +"secure-boot",
> +"verbose-dynamic"
> +],
> +"tags": [
> +
> +]
> +}
> diff --git a/pc-bios/descriptors/60-edk2-aarch64.json 
> b/pc-bios/descriptors/60-edk2-aarch64.json
> new file mode 100644
> index ..800a21bda691
> --- /dev/null
> +++ b/pc-bios/descriptors/60-edk2-aarch64.json
> @@ -0,0 +1,31 @@
> +{
> +"description": "UEFI firmware for aarch64",
> +"interface-types": [
> +"uefi"
> +],
> +"mapping": {
> +"device": "flash",
> +"executable": {
> +"filename": "@DATADIR@/edk2-aarch64-code.fd",
> +"format": "raw"
> +},
> +"nvram-template": {
> +"filename": "@DATADIR@/edk2-arm-vars.fd",
> +"format": "raw"
> +}
> +},
> +"targets": [
> +{
> +"architecture": "aarch64",
> +"machines": [
> +"virt-*"
> +]
> +}
> +],
> +"features": [
> +"verbose-static"
> +],
> +"tags": [
> +
> +]
> +}
> diff --git a/pc-bios/descriptors/60-edk2-arm.json 
> b/pc-bios/descriptors/60-edk2-arm.json
> new file mode 100644
> index ..d5f1bba6cc82
> --- /dev/null
> +++ b/pc-bios/descriptors/60-edk2-arm.json
> @@ -0,0 +1,31 @@
> +{
> +"description": "UEFI firmware for arm",
> +"interface-types": [
> +"uefi"
> +],
> +"mapping": {
> +"device": "flash",
> +"executable": {
> +"filename": "@DATADIR@/edk2-arm-code.fd",
> +"format": "raw"
> +},
> +"nvram-template": {
> +"filename":

Re: [Qemu-devel] [PATCH 12/15] qemu-print: New qemu_printf(), qemu_vprintf() etc.

"Dr. David Alan Gilbert"  writes:

> * Markus Armbruster (arm...@redhat.com) wrote:
>> We commonly want to print to the current monitor if we have one, else
>> to stdout/stderr.  For stderr, have error_printf().  For stdout, all
>> we have is monitor_vfprintf(), which is rather unwieldy.  We often
>> print to stderr just because error_printf() is easier.
>> 
>> New qemu_printf() and qemu_vprintf() do exactly what's needed.  The
>> next commits will put them to use.
>> 
>> Cc: Dr. David Alan Gilbert 
>> Signed-off-by: Markus Armbruster 
>
> OK, it seems a little odd not to have the fprintf as well, but OK.

Wait for my next series touching this :)

> Reviewed-by: Dr. David Alan Gilbert 

Thanks!

Re: [Qemu-devel] [PATCH for-4.1 v4 06/12] roms/Makefile: replace the $(EDK2_EFIROM) target with "edk2-basetools"

On 4/10/19 1:00 AM, Laszlo Ersek wrote:
> We don't (can't) have a recipe for building just $(EDK2_EFIROM);
> therefore, while we call the target $(EDK2_EFIROM), we actually build all
> of the edk2 BaseTools. Rename the target to edk2-basetools, and update the
> iPXE prerequisite accordingly. This will let other targets depend on
> "edk2-basetools", where an $(EDK2_EFIROM) pre-requisite would be
> misleading.
> 
> Signed-off-by: Laszlo Ersek 

Reviewed-by: Philippe Mathieu-Daudé 
Tested-by: Philippe Mathieu-Daudé 

> ---
> 
> Notes:
> v4:
> 
> - rebase: resolve conflicts with
> 
>   - commit d912e795e029 ("roms: Rename the EFIROM variable to avoid
> clashing with iPXE", 2019-04-09)
> 
>   - commit 1cab464136b4 ("roms: Allow passing configure options to the
> EDK2 build tools", 2019-04-09)
> 
> - refresh commit message accordingly: replace EFIROM references with
>   EDK2_EFIROM references
> 
> - drop all previous feedback tags, consequently :(
> 
>   - Phil's R-b / T-b
>   - Michal's R-b
>   - Michael's R-b
>   - Igor's T-b
> 
> v3:
> 
> - pick up Igor's T-b
> 
> v2:
> 
> - pick up Phil's R-b / T-b
> 
> - pick up Michal's and Michael's R-b's
> 
>  roms/Makefile | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/roms/Makefile b/roms/Makefile
> index 1ff78b63bb3a..93c3d467be14 100644
> --- a/roms/Makefile
> +++ b/roms/Makefile
> @@ -102,7 +102,7 @@ pxe-rom-%: build-pxe-roms
>  
>  efirom: $(patsubst %,efi-rom-%,$(pxerom_variants))
>  
> -efi-rom-%: build-pxe-roms build-efi-roms $(EDK2_EFIROM)
> +efi-rom-%: build-pxe-roms build-efi-roms edk2-basetools
>   $(EDK2_EFIROM) -f "0x$(VID)" -i "0x$(DID)" -l 0x02 \
>   -b ipxe/src/bin/$(VID)$(DID).rom \
>   -ec ipxe/src/bin-i386-efi/$(VID)$(DID).efidrv \
> @@ -131,7 +131,7 @@ build-efi-roms: build-pxe-roms
>  #EDK2_BASETOOLS_LDFLAGS='...' \
>  #efirom
>  #
> -$(EDK2_EFIROM):
> +edk2-basetools:
>   $(MAKE) -C edk2/BaseTools \
>   EXTRA_OPTFLAGS='$(EDK2_BASETOOLS_OPTFLAGS)' \
>   EXTRA_LDFLAGS='$(EDK2_BASETOOLS_LDFLAGS)'
>

[Qemu-devel] [PATCH for-4.1] roms: Correct the EDK2_BASETOOLS_OPTFLAGS variable description

In commit 1cab464136b4 we incorrectly described the
EDK2_BASETOOLS_OPTFLAGS can pass CPPFLAGS and CFLAGS
options to the EDK2 build tools, but it only expands
the CFLAGS (not to the CPPFLAGS).
Update the description to be more accurate.

Reported-by: Laszlo Ersek 
Signed-off-by: Philippe Mathieu-Daudé 
---
 roms/Makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/roms/Makefile b/roms/Makefile
index 1ff78b63bb3..d7fd6025e7c 100644
--- a/roms/Makefile
+++ b/roms/Makefile
@@ -120,8 +120,8 @@ build-efi-roms: build-pxe-roms
$(patsubst %,bin-i386-efi/%.efidrv,$(pxerom_targets)) \
$(patsubst %,bin-x86_64-efi/%.efidrv,$(pxerom_targets))
 
-# Build scripts can pass compiler/linker flags to the EDK2 build tools
-# via the EDK2_BASETOOLS_OPTFLAGS (CPPFLAGS and CFLAGS) and
+# Build scripts can pass compiler/linker flags to the EDK2
+# build  tools via the EDK2_BASETOOLS_OPTFLAGS (CFLAGS) and
 # EDK2_BASETOOLS_LDFLAGS (LDFLAGS) environment variables.
 #
 # Example:
-- 
2.20.1

[Qemu-devel] [PATCH v5 3/6] libnvdimm: add dax_dev sync flag

This patch adds 'DAXDEV_SYNC' flag which is set
for nd_region doing synchronous flush. This later
is used to disable MAP_SYNC functionality for
ext4 & xfs filesystem for devices don't support
synchronous flush.

Signed-off-by: Pankaj Gupta 
---
 drivers/dax/bus.c|  2 +-
 drivers/dax/super.c  | 13 -
 drivers/md/dm.c  |  2 +-
 drivers/nvdimm/pmem.c|  3 ++-
 drivers/nvdimm/region_devs.c |  7 +++
 include/linux/dax.h  |  9 +++--
 include/linux/libnvdimm.h|  1 +
 7 files changed, 31 insertions(+), 6 deletions(-)

diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c
index 2109cfe80219..431bf7d2a7f9 100644
--- a/drivers/dax/bus.c
+++ b/drivers/dax/bus.c
@@ -388,7 +388,7 @@ struct dev_dax *__devm_create_dev_dax(struct dax_region 
*dax_region, int id,
 * No 'host' or dax_operations since there is no access to this
 * device outside of mmap of the resulting character device.
 */
-   dax_dev = alloc_dax(dev_dax, NULL, NULL);
+   dax_dev = alloc_dax(dev_dax, NULL, NULL, true);
if (!dax_dev)
goto err;
 
diff --git a/drivers/dax/super.c b/drivers/dax/super.c
index 0a339b85133e..bd6509308d05 100644
--- a/drivers/dax/super.c
+++ b/drivers/dax/super.c
@@ -186,6 +186,8 @@ enum dax_device_flags {
DAXDEV_ALIVE,
/* gate whether dax_flush() calls the low level flush routine */
DAXDEV_WRITE_CACHE,
+   /* flag to check if device supports synchronous flush */
+   DAXDEV_SYNC,
 };
 
 /**
@@ -354,6 +356,12 @@ bool dax_write_cache_enabled(struct dax_device *dax_dev)
 }
 EXPORT_SYMBOL_GPL(dax_write_cache_enabled);
 
+bool dax_synchronous(struct dax_device *dax_dev)
+{
+   return test_bit(DAXDEV_SYNC, _dev->flags);
+}
+EXPORT_SYMBOL_GPL(dax_synchronous);
+
 bool dax_alive(struct dax_device *dax_dev)
 {
lockdep_assert_held(_srcu);
@@ -511,7 +519,7 @@ static void dax_add_host(struct dax_device *dax_dev, const 
char *host)
 }
 
 struct dax_device *alloc_dax(void *private, const char *__host,
-   const struct dax_operations *ops)
+   const struct dax_operations *ops, bool sync)
 {
struct dax_device *dax_dev;
const char *host;
@@ -534,6 +542,9 @@ struct dax_device *alloc_dax(void *private, const char 
*__host,
dax_add_host(dax_dev, host);
dax_dev->ops = ops;
dax_dev->private = private;
+   if (sync)
+   set_bit(DAXDEV_SYNC, _dev->flags);
+
return dax_dev;
 
  err_dev:
diff --git a/drivers/md/dm.c b/drivers/md/dm.c
index 68d24056d0b1..534e12ca6329 100644
--- a/drivers/md/dm.c
+++ b/drivers/md/dm.c
@@ -1965,7 +1965,7 @@ static struct mapped_device *alloc_dev(int minor)
sprintf(md->disk->disk_name, "dm-%d", minor);
 
if (IS_ENABLED(CONFIG_DAX_DRIVER)) {
-   dax_dev = alloc_dax(md, md->disk->disk_name, _dax_ops);
+   dax_dev = alloc_dax(md, md->disk->disk_name, _dax_ops, true);
if (!dax_dev)
goto bad;
}
diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c
index 5a5b3ea4d073..78f71ba0e7cf 100644
--- a/drivers/nvdimm/pmem.c
+++ b/drivers/nvdimm/pmem.c
@@ -466,7 +466,8 @@ static int pmem_attach_disk(struct device *dev,
nvdimm_badblocks_populate(nd_region, >bb, _res);
disk->bb = >bb;
 
-   dax_dev = alloc_dax(pmem, disk->disk_name, _dax_ops);
+   dax_dev = alloc_dax(pmem, disk->disk_name, _dax_ops,
+   is_nvdimm_sync(nd_region));
 
if (!dax_dev) {
put_disk(disk);
diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c
index fb1041ab32a6..8c7aa047fe2b 100644
--- a/drivers/nvdimm/region_devs.c
+++ b/drivers/nvdimm/region_devs.c
@@ -1231,6 +1231,13 @@ int nvdimm_has_cache(struct nd_region *nd_region)
 }
 EXPORT_SYMBOL_GPL(nvdimm_has_cache);
 
+bool is_nvdimm_sync(struct nd_region *nd_region)
+{
+   return is_nd_pmem(_region->dev) &&
+   !test_bit(ND_REGION_ASYNC, _region->flags);
+}
+EXPORT_SYMBOL_GPL(is_nvdimm_sync);
+
 struct conflict_context {
struct nd_region *nd_region;
resource_size_t start, size;
diff --git a/include/linux/dax.h b/include/linux/dax.h
index 0dd316a74a29..b896706a5ee9 100644
--- a/include/linux/dax.h
+++ b/include/linux/dax.h
@@ -32,18 +32,19 @@ extern struct attribute_group dax_attribute_group;
 #if IS_ENABLED(CONFIG_DAX)
 struct dax_device *dax_get_by_host(const char *host);
 struct dax_device *alloc_dax(void *private, const char *host,
-   const struct dax_operations *ops);
+   const struct dax_operations *ops, bool sync);
 void put_dax(struct dax_device *dax_dev);
 void kill_dax(struct dax_device *dax_dev);
 void dax_write_cache(struct dax_device *dax_dev, bool wc);
 bool dax_write_cache_enabled(struct dax_device *dax_dev);
+bool dax_synchronous(struct dax_device *dax_dev);
 #else
 static inline struct dax_device

[Qemu-devel] [PATCH v5 6/6] xfs: disable map_sync for async flush

Dont support 'MAP_SYNC' with non-DAX files and DAX files
with asynchronous dax_device. Virtio pmem provides
asynchronous host page cache flush mechanism. We don't
support 'MAP_SYNC' with virtio pmem and xfs.

Signed-off-by: Pankaj Gupta 
---
 fs/xfs/xfs_file.c | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index 1f2e2845eb76..0e59be018511 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -1196,11 +1196,13 @@ xfs_file_mmap(
struct file *filp,
struct vm_area_struct *vma)
 {
-   /*
-* We don't support synchronous mappings for non-DAX files. At least
-* until someone comes with a sensible use case.
+   struct dax_device *dax_dev = xfs_find_daxdev_for_inode
+   (file_inode(filp));
+
+   /* We don't support synchronous mappings for non-DAX files and
+* for DAX files if underneath dax_device is not synchronous.
 */
-   if (!IS_DAX(file_inode(filp)) && (vma->vm_flags & VM_SYNC))
+   if (!daxdev_mapping_supported(vma, dax_dev))
return -EOPNOTSUPP;
 
file_accessed(filp);
-- 
2.20.1

[Qemu-devel] [PATCH v5 2/5] virtio-pmem: Add virtio pmem driver

This patch adds virtio-pmem driver for KVM guest.

Guest reads the persistent memory range information from
Qemu over VIRTIO and registers it on nvdimm_bus. It also
creates a nd_region object with the persistent memory
range information so that existing 'nvdimm/pmem' driver
can reserve this into system memory map. This way
'virtio-pmem' driver uses existing functionality of pmem
driver to register persistent memory compatible for DAX
capable filesystems.

This also provides function to perform guest flush over
VIRTIO from 'pmem' driver when userspace performs flush
on DAX memory range.

Signed-off-by: Pankaj Gupta 
---
 drivers/nvdimm/virtio_pmem.c |  88 ++
 drivers/virtio/Kconfig   |  10 +++
 drivers/virtio/Makefile  |   1 +
 drivers/virtio/pmem.c| 124 +++
 include/linux/virtio_pmem.h  |  60 +++
 include/uapi/linux/virtio_ids.h  |   1 +
 include/uapi/linux/virtio_pmem.h |  10 +++
 7 files changed, 294 insertions(+)
 create mode 100644 drivers/nvdimm/virtio_pmem.c
 create mode 100644 drivers/virtio/pmem.c
 create mode 100644 include/linux/virtio_pmem.h
 create mode 100644 include/uapi/linux/virtio_pmem.h

diff --git a/drivers/nvdimm/virtio_pmem.c b/drivers/nvdimm/virtio_pmem.c
new file mode 100644
index ..01044cc2f3a3
--- /dev/null
+++ b/drivers/nvdimm/virtio_pmem.c
@@ -0,0 +1,88 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * virtio_pmem.c: Virtio pmem Driver
+ *
+ * Discovers persistent memory range information
+ * from host and provides a virtio based flushing
+ * interface.
+ */
+#include 
+#include "nd.h"
+
+ /* The interrupt handler */
+void host_ack(struct virtqueue *vq)
+{
+   unsigned int len;
+   unsigned long flags;
+   struct virtio_pmem_request *req, *req_buf;
+   struct virtio_pmem *vpmem = vq->vdev->priv;
+
+   spin_lock_irqsave(>pmem_lock, flags);
+   while ((req = virtqueue_get_buf(vq, )) != NULL) {
+   req->done = true;
+   wake_up(>host_acked);
+
+   if (!list_empty(>req_list)) {
+   req_buf = list_first_entry(>req_list,
+   struct virtio_pmem_request, list);
+   list_del(>req_list);
+   req_buf->wq_buf_avail = true;
+   wake_up(_buf->wq_buf);
+   }
+   }
+   spin_unlock_irqrestore(>pmem_lock, flags);
+}
+EXPORT_SYMBOL_GPL(host_ack);
+
+ /* The request submission function */
+int virtio_pmem_flush(struct nd_region *nd_region)
+{
+   int err;
+   unsigned long flags;
+   struct scatterlist *sgs[2], sg, ret;
+   struct virtio_device *vdev = nd_region->provider_data;
+   struct virtio_pmem *vpmem = vdev->priv;
+   struct virtio_pmem_request *req;
+
+   might_sleep();
+   req = kmalloc(sizeof(*req), GFP_KERNEL);
+   if (!req)
+   return -ENOMEM;
+
+   req->done = req->wq_buf_avail = false;
+   strcpy(req->name, "FLUSH");
+   init_waitqueue_head(>host_acked);
+   init_waitqueue_head(>wq_buf);
+   sg_init_one(, req->name, strlen(req->name));
+   sgs[0] = 
+   sg_init_one(, >ret, sizeof(req->ret));
+   sgs[1] = 
+
+   spin_lock_irqsave(>pmem_lock, flags);
+   err = virtqueue_add_sgs(vpmem->req_vq, sgs, 1, 1, req, GFP_ATOMIC);
+   if (err) {
+   dev_err(>dev, "failed to send command to virtio pmem 
device\n");
+
+   list_add_tail(>req_list, >list);
+   spin_unlock_irqrestore(>pmem_lock, flags);
+
+   /* When host has read buffer, this completes via host_ack */
+   wait_event(req->wq_buf, req->wq_buf_avail);
+   spin_lock_irqsave(>pmem_lock, flags);
+   }
+   err = virtqueue_kick(vpmem->req_vq);
+   spin_unlock_irqrestore(>pmem_lock, flags);
+
+   if (!err) {
+   err = -EIO;
+   goto ret;
+   }
+   /* When host has read buffer, this completes via host_ack */
+   wait_event(req->host_acked, req->done);
+   err = req->ret;
+ret:
+   kfree(req);
+   return err;
+};
+EXPORT_SYMBOL_GPL(virtio_pmem_flush);
+MODULE_LICENSE("GPL");
diff --git a/drivers/virtio/Kconfig b/drivers/virtio/Kconfig
index 35897649c24f..9f634a2ed638 100644
--- a/drivers/virtio/Kconfig
+++ b/drivers/virtio/Kconfig
@@ -42,6 +42,16 @@ config VIRTIO_PCI_LEGACY
 
  If unsure, say Y.
 
+config VIRTIO_PMEM
+   tristate "Support for virtio pmem driver"
+   depends on VIRTIO
+   depends on LIBNVDIMM
+   help
+   This driver provides support for virtio based flushing interface
+   for persistent memory range.
+
+   If unsure, say M.
+
 config VIRTIO_BALLOON
tristate "Virtio balloon driver"
depends on VIRTIO
diff --git a/drivers/virtio/Makefile b/drivers/virtio/Makefile
index 3a2b5c5dcf46..143ce91eabe9 100644
--- a/drivers/virtio/Makefile
+++

[Qemu-devel] [PATCH v5 0/6] virtio pmem driver

 This patch series has implementation for "virtio pmem". 
 "virtio pmem" is fake persistent memory(nvdimm) in guest 
 which allows to bypass the guest page cache. This also
 implements a VIRTIO based asynchronous flush mechanism.  
 
 Sharing guest kernel driver in this patchset with the 
 changes suggested in v4. Tested with Qemu side device 
 emulation [6] for virtio-pmem. 

 We have incorporated all the suggestions in V4. Documented 
 the impact of possible page cache side channel attacks with 
 suggested countermeasures.
 
 Details of project idea for 'virtio pmem' flushing interface 
 is shared [3] & [4].

 Implementation is divided into two parts:
 New virtio pmem guest driver and qemu code changes for new 
 virtio pmem paravirtualized device.

1. Guest virtio-pmem kernel driver
-
   - Reads persistent memory range from paravirt device and 
 registers with 'nvdimm_bus'.  
   - 'nvdimm/pmem' driver uses this information to allocate 
 persistent memory region and setup filesystem operations 
 to the allocated memory. 
   - virtio pmem driver implements asynchronous flushing 
 interface to flush from guest to host.

2. Qemu virtio-pmem device
-
   - Creates virtio pmem device and exposes a memory range to 
 KVM guest. 
   - At host side this is file backed memory which acts as 
 persistent memory. 
   - Qemu side flush uses aio thread pool API's and virtio 
 for asynchronous guest multi request handling. 

   David Hildenbrand CCed also posted a modified version[7] of 
   qemu virtio-pmem code based on updated Qemu memory device API. 

 Virtio-pmem security implications and countermeasures:
 -

 In previous posting of kernel driver, there was discussion [9]
 on possible implications of page cache side channel attacks with 
 virtio pmem. After thorough analysis of details of known side 
 channel attacks, below are the suggestions:

 - Depends entirely on how host backing image file is mapped 
   into guest address space. 

 - virtio-pmem device emulation, by default shared mapping is used
   to map host backing file. It is recommended to use separate
   backing file at host side for every guest. This will prevent
   any possibility of executing common code from multiple guests
   and any chance of inferring guest local data based based on 
   execution time.

 - If backing file is required to be shared among multiple guests 
   it is recommended to don't support host page cache eviction 
   commands from the guest driver. This will avoid any possibility
   of inferring guest local data or host data from another guest. 

 - Proposed device specification [8] for virtio-pmem device with 
   details of possible security implications and suggested 
   countermeasures for device emulation.

 Virtio-pmem errors handling:
 
  Checked behaviour of virtio-pmem for below types of errors
  Need suggestions on expected behaviour for handling these errors?

  - Hardware Errors: Uncorrectable recoverable Errors: 
  a] virtio-pmem: 
- As per current logic if error page belongs to Qemu process, 
  host MCE handler isolates(hwpoison) that page and send SIGBUS. 
  Qemu SIGBUS handler injects exception to KVM guest. 
- KVM guest then isolates the page and send SIGBUS to guest 
  userspace process which has mapped the page. 
  
  b] Existing implementation for ACPI pmem driver: 
- Handles such errors with MCE notifier and creates a list 
  of bad blocks. Read/direct access DAX operation return EIO 
  if accessed memory page fall in bad block list.
- It also starts backgound scrubbing.  
- Similar functionality can be reused in virtio-pmem with MCE 
  notifier but without scrubbing(no ACPI/ARS)? Need inputs to 
  confirm if this behaviour is ok or needs any change?

Changes from PATCH v4: [1]
- Factor out MAP_SYNC supported functionality to a common helper
[Dave, Darrick, Jan]
- Comment, indentation and virtqueue_kick failure handle - Yuval Shaia

Changes from PATCH v3: [2]
- Use generic dax_synchronous() helper to check for DAXDEV_SYNC 
  flag - [Dan, Darrick, Jan]
- Add 'is_nvdimm_async' function
- Document page cache side channel attacks implications & 
  countermeasures - [Dave Chinner, Michael]

Changes from PATCH v2: 
- Disable MAP_SYNC for ext4 & XFS filesystems - [Dan] 
- Use name 'virtio pmem' in place of 'fake dax' 

Changes from PATCH v1: 
- 0-day build test for build dependency on libnvdimm 

 Changes suggested by - [Dan Williams]
- Split the driver into two parts virtio & pmem  
- Move queuing of async block request to block layer
- Add "sync" parameter in nvdimm_flush function
- Use indirect call for nvdimm_flush
- Don’t move declarations to common global header e.g nd.h
- nvdimm_flush() return 0 or -EIO if it fails
- Teach nsio_rw_bytes() that the

[Qemu-devel] [PATCH v5 5/6] ext4: disable map_sync for async flush

Dont support 'MAP_SYNC' with non-DAX files and DAX files
with asynchronous dax_device. Virtio pmem provides 
asynchronous host page cache flush mechanism. We don't
support 'MAP_SYNC' with virtio pmem and ext4.

Signed-off-by: Pankaj Gupta 
---
 fs/ext4/file.c | 11 ++-
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/fs/ext4/file.c b/fs/ext4/file.c
index 69d65d49837b..4b2ccaf1932e 100644
--- a/fs/ext4/file.c
+++ b/fs/ext4/file.c
@@ -360,15 +360,16 @@ static const struct vm_operations_struct ext4_file_vm_ops 
= {
 static int ext4_file_mmap(struct file *file, struct vm_area_struct *vma)
 {
struct inode *inode = file->f_mapping->host;
+   struct ext4_sb_info *sbi = EXT4_SB(inode->i_sb);
+   struct dax_device *dax_dev = sbi->s_daxdev;
 
-   if (unlikely(ext4_forced_shutdown(EXT4_SB(inode->i_sb
+   if (unlikely(ext4_forced_shutdown(sbi)))
return -EIO;
 
-   /*
-* We don't support synchronous mappings for non-DAX files. At least
-* until someone comes with a sensible use case.
+   /* We don't support synchronous mappings for non-DAX files and
+* for DAX files if underneath dax_device is not synchronous.
 */
-   if (!IS_DAX(file_inode(file)) && (vma->vm_flags & VM_SYNC))
+   if (!daxdev_mapping_supported(vma, dax_dev))
return -EOPNOTSUPP;
 
file_accessed(file);
-- 
2.20.1

[Qemu-devel] [PATCH v5 4/6] dax: check synchronous mapping is supported

This patch introduces 'daxdev_mapping_supported' helper
which checks if 'MAP_SYNC' is supported with filesystem
mapping. It also checks if corresponding dax_device is
synchronous. Virtio pmem device is asynchronous and
does not not support VM_SYNC. 

Suggested-by: Jan Kara 
Signed-off-by: Pankaj Gupta 
---
 include/linux/dax.h | 23 +++
 1 file changed, 23 insertions(+)

diff --git a/include/linux/dax.h b/include/linux/dax.h
index b896706a5ee9..4a2a60ffec86 100644
--- a/include/linux/dax.h
+++ b/include/linux/dax.h
@@ -38,6 +38,24 @@ void kill_dax(struct dax_device *dax_dev);
 void dax_write_cache(struct dax_device *dax_dev, bool wc);
 bool dax_write_cache_enabled(struct dax_device *dax_dev);
 bool dax_synchronous(struct dax_device *dax_dev);
+
+/*
+ * Callers check if synchronous mapping is enabled for DAX file
+ * and attached dax device is also synchronous.
+ *
+ * dax_synchronous function verifies if dax device is synchronous.
+ * Currently, only virtio pmem device supports asynchronous device
+ * flush.
+ */
+static inline bool daxdev_mapping_supported(struct vm_area_struct *vma,
+   struct dax_device *dax_dev)
+{
+   if (!(vma->vm_flags & VM_SYNC))
+   return true;
+   if (!IS_DAX(file_inode(vma->vm_file)))
+   return false;
+   return dax_synchronous(dax_dev);
+}
 #else
 static inline struct dax_device *dax_get_by_host(const char *host)
 {
@@ -69,6 +87,11 @@ static inline bool dax_synchronous(struct dax_device 
*dax_dev)
 {
return true;
 }
+static inline bool daxdev_mapping_supported(struct vm_area_struct *vma,
+   struct dax_device *dax_dev)
+{
+   return true;
+}
 #endif
 
 struct writeback_control;
-- 
2.20.1

[Qemu-devel] [PATCH v5 1/6] libnvdimm: nd_region flush callback support

This patch adds functionality to perform flush from guest
to host over VIRTIO. We are registering a callback based
on 'nd_region' type. virtio_pmem driver requires this special
flush function. For rest of the region types we are registering
existing flush function. Report error returned by host fsync
failure to userspace.

This also handles asynchronous flush requests from the block layer
by creating a child bio and chaining it with parent bio.

Signed-off-by: Pankaj Gupta 
---
 drivers/acpi/nfit/core.c |  4 ++--
 drivers/nvdimm/claim.c   |  6 --
 drivers/nvdimm/nd.h  |  1 +
 drivers/nvdimm/pmem.c| 14 -
 drivers/nvdimm/region_devs.c | 38 ++--
 include/linux/libnvdimm.h|  8 +++-
 6 files changed, 59 insertions(+), 12 deletions(-)

diff --git a/drivers/acpi/nfit/core.c b/drivers/acpi/nfit/core.c
index 5a389a4f4f65..567017a2190e 100644
--- a/drivers/acpi/nfit/core.c
+++ b/drivers/acpi/nfit/core.c
@@ -2434,7 +2434,7 @@ static void write_blk_ctl(struct nfit_blk *nfit_blk, 
unsigned int bw,
offset = to_interleave_offset(offset, mmio);
 
writeq(cmd, mmio->addr.base + offset);
-   nvdimm_flush(nfit_blk->nd_region);
+   nvdimm_flush(nfit_blk->nd_region, NULL, false);
 
if (nfit_blk->dimm_flags & NFIT_BLK_DCR_LATCH)
readq(mmio->addr.base + offset);
@@ -2483,7 +2483,7 @@ static int acpi_nfit_blk_single_io(struct nfit_blk 
*nfit_blk,
}
 
if (rw)
-   nvdimm_flush(nfit_blk->nd_region);
+   nvdimm_flush(nfit_blk->nd_region, NULL, false);
 
rc = read_blk_stat(nfit_blk, lane) ? -EIO : 0;
return rc;
diff --git a/drivers/nvdimm/claim.c b/drivers/nvdimm/claim.c
index fb667bf469c7..a1dfa066786b 100644
--- a/drivers/nvdimm/claim.c
+++ b/drivers/nvdimm/claim.c
@@ -263,7 +263,7 @@ static int nsio_rw_bytes(struct nd_namespace_common *ndns,
struct nd_namespace_io *nsio = to_nd_namespace_io(>dev);
unsigned int sz_align = ALIGN(size + (offset & (512 - 1)), 512);
sector_t sector = offset >> 9;
-   int rc = 0;
+   int rc = 0, ret = 0;
 
if (unlikely(!size))
return 0;
@@ -301,7 +301,9 @@ static int nsio_rw_bytes(struct nd_namespace_common *ndns,
}
 
memcpy_flushcache(nsio->addr + offset, buf, size);
-   nvdimm_flush(to_nd_region(ndns->dev.parent));
+   ret = nvdimm_flush(to_nd_region(ndns->dev.parent), NULL, false);
+   if (ret)
+   rc = ret;
 
return rc;
 }
diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
index a5ac3b240293..916cd6c5451a 100644
--- a/drivers/nvdimm/nd.h
+++ b/drivers/nvdimm/nd.h
@@ -159,6 +159,7 @@ struct nd_region {
struct badblocks bb;
struct nd_interleave_set *nd_set;
struct nd_percpu_lane __percpu *lane;
+   int (*flush)(struct nd_region *nd_region);
struct nd_mapping mapping[0];
 };
 
diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c
index bc2f700feef8..5a5b3ea4d073 100644
--- a/drivers/nvdimm/pmem.c
+++ b/drivers/nvdimm/pmem.c
@@ -192,6 +192,7 @@ static blk_status_t pmem_do_bvec(struct pmem_device *pmem, 
struct page *page,
 
 static blk_qc_t pmem_make_request(struct request_queue *q, struct bio *bio)
 {
+   int ret = 0;
blk_status_t rc = 0;
bool do_acct;
unsigned long start;
@@ -201,7 +202,7 @@ static blk_qc_t pmem_make_request(struct request_queue *q, 
struct bio *bio)
struct nd_region *nd_region = to_region(pmem);
 
if (bio->bi_opf & REQ_PREFLUSH)
-   nvdimm_flush(nd_region);
+   ret = nvdimm_flush(nd_region, bio, true);
 
do_acct = nd_iostat_start(bio, );
bio_for_each_segment(bvec, bio, iter) {
@@ -216,7 +217,10 @@ static blk_qc_t pmem_make_request(struct request_queue *q, 
struct bio *bio)
nd_iostat_end(bio, start);
 
if (bio->bi_opf & REQ_FUA)
-   nvdimm_flush(nd_region);
+   ret = nvdimm_flush(nd_region, bio, true);
+
+   if (ret)
+   bio->bi_status = errno_to_blk_status(ret);
 
bio_endio(bio);
return BLK_QC_T_NONE;
@@ -463,13 +467,13 @@ static int pmem_attach_disk(struct device *dev,
disk->bb = >bb;
 
dax_dev = alloc_dax(pmem, disk->disk_name, _dax_ops);
+
if (!dax_dev) {
put_disk(disk);
return -ENOMEM;
}
dax_write_cache(dax_dev, nvdimm_has_cache(nd_region));
pmem->dax_dev = dax_dev;
-
gendev = disk_to_dev(disk);
gendev->groups = pmem_attribute_groups;
 
@@ -527,14 +531,14 @@ static int nd_pmem_remove(struct device *dev)
sysfs_put(pmem->bb_state);
pmem->bb_state = NULL;
}
-   nvdimm_flush(to_nd_region(dev->parent));
+   nvdimm_flush(to_nd_region(dev->parent), NULL, false);
 
return 0;
 }
 
 static void nd_pmem_shutdown(struct device *dev)
 {
-

[Qemu-devel] [Bug 1824053] [NEW] Qemu-img convert appears to be stuck on aarch64 host with low probability

2019-04-09 Thread 贞贵李

Public bug reported:

Hi,  I found a problem that qemu-img convert appears to be stuck on
aarch64 host with low probability.

The convert command  line is  "qemu-img convert -f qcow2 -O raw
disk.qcow2 disk.raw ".

The bt is below:

Thread 2 (Thread 0x4b776e50 (LWP 27215)):
#0  0x4a3f2994 in sigtimedwait () from /lib64/libc.so.6
#1  0x4a39c60c in sigwait () from /lib64/libpthread.so.0
#2  0xaae82610 in sigwait_compat (opaque=0xc5163b00) at 
util/compatfd.c:37
#3  0xaae85038 in qemu_thread_start (args=args@entry=0xc5163b90) at 
util/qemu_thread_posix.c:496
#4  0x4a3918bc in start_thread () from /lib64/libpthread.so.0
#5  0x4a492b2c in thread_start () from /lib64/libc.so.6

Thread 1 (Thread 0x4b573370 (LWP 27214)):
#0  0x4a489020 in ppoll () from /lib64/libc.so.6
#1  0xaadaefc0 in ppoll (__ss=0x0, __timeout=0x0, __nfds=, __fds=) at /usr/include/bits/poll2.h:77
#2  qemu_poll_ns (fds=, nfds=, timeout=) at qemu_timer.c:391
#3  0xaadae014 in os_host_main_loop_wait (timeout=) at 
main_loop.c:272
#4  0xaadae190 in main_loop_wait (nonblocking=) at 
main_loop.c:534
#5  0xaad97be0 in convert_do_copy (s=0xdc32eb48) at qemu-img.c:1923
#6  0xaada2d70 in img_convert (argc=, argv=) at qemu-img.c:2414
#7  0xaad99ac4 in main (argc=7, argv=) at qemu-img.c:5305


The problem seems to be very similar to the phenomenon described by this patch 
(https://resources.ovirt.org/pub/ovirt-4.1/src/qemu-kvm-ev/0025-aio_notify-force-main-loop-wakeup-with-SIGIO-aarch64.patch),
 

which force main loop wakeup with SIGIO.  But this patch was reverted by
the patch (http://ovirt.repo.nfrance.com/src/qemu-kvm-ev/kvm-Revert-
aio_notify-force-main-loop-wakeup-with-SIGIO-.patch).

The problem still seems to exist in aarch64 host. The qemu version I used is 
2.8.1. The host version is 4.19.28-1.2.108.aarch64.
 Do you have any solutions to fix it?  Thanks for your reply !

** Affects: qemu
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1824053

Title:
  Qemu-img convert appears to be stuck on aarch64 host with low
  probability

Status in QEMU:
  New

Bug description:
  Hi,  I found a problem that qemu-img convert appears to be stuck on
  aarch64 host with low probability.

  The convert command  line is  "qemu-img convert -f qcow2 -O raw
  disk.qcow2 disk.raw ".

  The bt is below:

  Thread 2 (Thread 0x4b776e50 (LWP 27215)):
  #0  0x4a3f2994 in sigtimedwait () from /lib64/libc.so.6
  #1  0x4a39c60c in sigwait () from /lib64/libpthread.so.0
  #2  0xaae82610 in sigwait_compat (opaque=0xc5163b00) at 
util/compatfd.c:37
  #3  0xaae85038 in qemu_thread_start (args=args@entry=0xc5163b90) 
at util/qemu_thread_posix.c:496
  #4  0x4a3918bc in start_thread () from /lib64/libpthread.so.0
  #5  0x4a492b2c in thread_start () from /lib64/libc.so.6

  Thread 1 (Thread 0x4b573370 (LWP 27214)):
  #0  0x4a489020 in ppoll () from /lib64/libc.so.6
  #1  0xaadaefc0 in ppoll (__ss=0x0, __timeout=0x0, __nfds=, __fds=) at /usr/include/bits/poll2.h:77
  #2  qemu_poll_ns (fds=, nfds=, 
timeout=) at qemu_timer.c:391
  #3  0xaadae014 in os_host_main_loop_wait (timeout=) at 
main_loop.c:272
  #4  0xaadae190 in main_loop_wait (nonblocking=) at 
main_loop.c:534
  #5  0xaad97be0 in convert_do_copy (s=0xdc32eb48) at 
qemu-img.c:1923
  #6  0xaada2d70 in img_convert (argc=, argv=) at qemu-img.c:2414
  #7  0xaad99ac4 in main (argc=7, argv=) at 
qemu-img.c:5305

  
  The problem seems to be very similar to the phenomenon described by this 
patch 
(https://resources.ovirt.org/pub/ovirt-4.1/src/qemu-kvm-ev/0025-aio_notify-force-main-loop-wakeup-with-SIGIO-aarch64.patch),
 

  which force main loop wakeup with SIGIO.  But this patch was reverted
  by the patch (http://ovirt.repo.nfrance.com/src/qemu-kvm-ev/kvm-
  Revert-aio_notify-force-main-loop-wakeup-with-SIGIO-.patch).

  The problem still seems to exist in aarch64 host. The qemu version I used is 
2.8.1. The host version is 4.19.28-1.2.108.aarch64.
   Do you have any solutions to fix it?  Thanks for your reply !

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1824053/+subscriptions

Re: [Qemu-devel] [PATCH v6 48/49] linux-user: Split out ioctl

On 4/9/19 1:30 PM, Richard Henderson wrote:
> On 4/9/19 1:15 PM, Richard Henderson wrote:
>> On 2/13/19 3:46 AM, Laurent Vivier wrote:
>>> In fact it happens with sid, with ppc64 too, but not with m68k and sh4.
>>>
>>> And only with "unshare --pid --fork".
>>>
>>> Without "unshare", the result is:
>>>
>>>   Unsupported ioctl: cmd=0x80047476
>>>   bash: cannot set terminal process group (11507): Function not implemented
>>>   bash: no job control in this shell
>>>   Unsupported ioctl: cmd=0x80087467
>>>   Unsupported ioctl: cmd=0x80087467
>>>   Unsupported ioctl: cmd=0x802c7415
>>
>> I'm just now getting back to this, and cannot replicate the issue
> 
> ... Oh nevermind.  I should be expecting no "unsupported" at all, since that's
> what we get with master.

The cause is that "cmd" accidentally changed from "int" in the do_ioctl
parameter list to "abi_ulong" as a local variable in impl_ioctl.  This caused a
table lookup failure on any ioctl with bit 31 set, for 64-bit guests.


r~

[Qemu-devel] [PATCH RFC 3/3] qemu-img.texi: use macros for command summaries

Use a macro to recall this text instead of re-entering it manually.

Modify the pxtool script to create macros for each subcommand, then
alter the qemu-img.texi file to recall these macros.

Signed-off-by: John Snow 
---
 qemu-img.texi | 28 ++--
 scripts/pxtool.py |  4 
 2 files changed, 18 insertions(+), 14 deletions(-)

diff --git a/qemu-img.texi b/qemu-img.texi
index 724f244ba1..c21a97e934 100644
--- a/qemu-img.texi
+++ b/qemu-img.texi
@@ -197,12 +197,12 @@ Command description:
 
 @table @option
 
-@item amend [--object @var{objectdef}] [--image-opts] [-p] [-q] [-f @var{fmt}] 
[-t @var{cache}] -o @var{options} @var{filename}
+@qemuimgcmdamend
 
 Amends the image format specific @var{options} for the image file
 @var{filename}. Not all file formats support this operation.
 
-@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] 
[--flush-interval=@var{flush_interval}] [-n] [--no-drain] [-o @var{offset}] 
[--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-S @var{step_size}] [-t 
@var{cache}] [-w] [-U] @var{filename}
+@qemuimgcmdbench
 
 Run a simple sequential I/O benchmark on the specified image. If @code{-w} is
 specified, a write test is performed, otherwise a read test is performed.
@@ -226,7 +226,7 @@ specified as well.
 For write tests, by default a buffer filled with zeros is written. This can be
 overridden with a pattern byte specified by @var{pattern}.
 
-@item check [--object @var{objectdef}] [--image-opts] [-q] [-f @var{fmt}] 
[--output=@var{ofmt}] [-r [leaks | all]] [-T @var{src_cache}] [-U] 
@var{filename}
+@qemuimgcmdcheck
 
 Perform a consistency check on the disk image @var{filename}. The command can
 output in the format @var{ofmt} which is either @code{human} or @code{json}.
@@ -262,7 +262,7 @@ If @code{-r} is specified, exit codes representing the 
image state refer to the
 state after (the attempt at) repairing it. That is, a successful @code{-r all}
 will yield the exit code 0, independently of the image state before.
 
-@item commit [--object @var{objectdef}] [--image-opts] [-q] [-f @var{fmt}] [-t 
@var{cache}] [-b @var{base}] [-d] [-p] @var{filename}
+@qemuimgcmdcommit
 
 Commit the changes recorded in @var{filename} in its base image or backing 
file.
 If the backing file is smaller than the snapshot, then the backing file will be
@@ -284,7 +284,7 @@ all images between @var{base} and the top image will be 
invalid and may return
 garbage data when read. For this reason, @code{-b} implies @code{-d} (so that
 the top image stays valid).
 
-@item compare [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [-F 
@var{fmt}] [-T @var{src_cache}] [-p] [-q] [-s] [-U] @var{filename1} 
@var{filename2}
+@qemuimgcmdcompare
 
 Check if two images have the same content. You can compare images with
 different format or settings.
@@ -325,7 +325,7 @@ Error on reading data
 
 @end table
 
-@item convert [--object @var{objectdef}] [--image-opts] [--target-image-opts] 
[-U] [-C] [-c] [-p] [-q] [-n] [-f @var{fmt}] [-t @var{cache}] [-T 
@var{src_cache}] [-O @var{output_fmt}] [-B @var{backing_file}] [-o 
@var{options}] [-l @var{snapshot_param}] [-S @var{sparse_size}] [-m 
@var{num_coroutines}] [-W] @var{filename} [@var{filename2} [...]] 
@var{output_filename}
+@qemuimgcmdconvert
 
 Convert the disk image @var{filename} or a snapshot @var{snapshot_param}
 to disk image @var{output_filename} using format @var{output_fmt}. It can be 
optionally compressed (@code{-c}
@@ -366,7 +366,7 @@ creating compressed images.
 @var{num_coroutines} specifies how many coroutines work in parallel during
 the convert process (defaults to 8).
 
-@item create [--object @var{objectdef}] [-q] [-f @var{fmt}] [-b 
@var{backing_file}] [-F @var{backing_fmt}] [-u] [-o @var{options}] 
@var{filename} [@var{size}]
+@qemuimgcmdcreate
 
 Create the new disk image @var{filename} of size @var{size} and format
 @var{fmt}. Depending on the file format, you can add one or more @var{options}
@@ -390,7 +390,7 @@ way.
 The size can also be specified using the @var{size} option with @code{-o},
 it doesn't need to be specified separately in this case.
 
-@item dd [--image-opts] [-U] [-f @var{fmt}] [-O @var{output_fmt}] 
[bs=@var{block_size}] [count=@var{blocks}] [skip=@var{blocks}] if=@var{input} 
of=@var{output}
+@qemuimgcmddd
 
 Dd copies from @var{input} file to @var{output} file converting it from
 @var{fmt} format to @var{output_fmt} format.
@@ -401,7 +401,7 @@ dd will stop reading input after reading @var{blocks} input 
blocks.
 
 The size syntax is similar to dd(1)'s size syntax.
 
-@item info [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] 
[--output=@var{ofmt}] [--backing-chain] [-U] @var{filename}
+@qemuimgcmdinfo
 
 Give information about the disk image @var{filename}. Use it in
 particular to know the size reserved on disk which can be different
@@ -424,7 +424,7 @@ To enumerate information about each disk image in the above 
chain, starting from
 qemu-img info --backing-chain

[Qemu-devel] [PATCH RFC 2/3] pxtool: Add new qemu-img command info generation tool

Presently we use hxtool and a .hx format to generate a few things like
the qemu_img subcommand dispatch table, the qemu_img help() display output,
and a help output in qemu-img.texi.

Unfortunately, this means that this information is duplicated in at least
three places:

(1) in qemu-img-cmds.hx as a human readable string
(2) in qemu-img-cmds.hx as a texi string
(3) in qemu-img.texi again, as a texi string

We can do a little better, though: all of these sources can be generated
from a single json file. Add a new small tool ("pxtool") that can do this.

This tool can at least handle generating (1) and (2) from the same source
without needing to reduplicate that information. Deduplicating (3) happens
in the next patch.

Notes:
 - The json format was chosen to be very "stupid", and the command line
   documentation is being kept one-per-line to make future diffs easier
   to read.
 - It's easy enough to generate the human-readable output from the texi
   output by removing '@var{foo}' with 'foo'.
 - The qemu-img command dispatch always calls img_cmdname, so we don't
   need to store this information separately, either.
 - The need for DEF() macros could be removed as well, but I left it in
   to create a minimally disruptive patch.
Signed-off-by: John Snow 
---
 Makefile   |   8 +--
 qemu-img-cmds.json | 165 +
 qemu-img-cmds.hx   | 102 
 scripts/pxtool.py  |  49 ++
 4 files changed, 218 insertions(+), 106 deletions(-)
 create mode 100644 qemu-img-cmds.json
 delete mode 100644 qemu-img-cmds.hx
 create mode 100755 scripts/pxtool.py

diff --git a/Makefile b/Makefile
index 04a0d45050..d203bb90dc 100644
--- a/Makefile
+++ b/Makefile
@@ -524,8 +524,8 @@ ifdef CONFIG_MPATH
 scsi/qemu-pr-helper$(EXESUF): LIBS += -ludev -lmultipath -lmpathpersist
 endif
 
-qemu-img-cmds.h: $(SRC_PATH)/qemu-img-cmds.hx $(SRC_PATH)/scripts/hxtool
-   $(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -h < $< > 
$@,"GEN","$@")
+qemu-img-cmds.h: $(SRC_PATH)/qemu-img-cmds.json $(SRC_PATH)/scripts/pxtool.py
+   $(call quiet-command,$(PYTOHN) $(SRC_PATH)/scripts/pxtool.py --macros 
$< > $@,"GEN","$@")
 
 qemu-ga$(EXESUF): LIBS = $(LIBS_QGA)
 qemu-ga$(EXESUF): QEMU_CFLAGS += -I qga/qapi-generated
@@ -918,8 +918,8 @@ qemu-monitor.texi: $(SRC_PATH)/hmp-commands.hx 
$(SRC_PATH)/scripts/hxtool
 qemu-monitor-info.texi: $(SRC_PATH)/hmp-commands-info.hx 
$(SRC_PATH)/scripts/hxtool
$(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -t < $< > 
$@,"GEN","$@")
 
-qemu-img-cmds.texi: $(SRC_PATH)/qemu-img-cmds.hx $(SRC_PATH)/scripts/hxtool
-   $(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -t < $< > 
$@,"GEN","$@")
+qemu-img-cmds.texi: $(SRC_PATH)/qemu-img-cmds.json 
$(SRC_PATH)/scripts/pxtool.py
+   $(call quiet-command,$(PYTON) $(SRC_PATH)/scripts/pxtool.py --texi $< > 
$@,"GEN","$@")
 
 docs/interop/qemu-qmp-qapi.texi: qapi/qapi-doc.texi
@cp -p $< $@
diff --git a/qemu-img-cmds.json b/qemu-img-cmds.json
new file mode 100644
index 00..35d96f92a3
--- /dev/null
+++ b/qemu-img-cmds.json
@@ -0,0 +1,165 @@
+{
+"amend": [
+"[--object @var{objectdef}]",
+"[--image-opts]",
+"[-p]",
+"[-q]",
+"[-f @var{fmt}]",
+"[-t @var{cache}]",
+"-o @var{options}",
+"@var{filename}"
+],
+"bench": [
+"[-c @var{count}]",
+"[-d @var{depth}]",
+"[-f @var{fmt}]",
+"[--flush-interval=@var{flush_interval}]",
+"[-n]",
+"[--no-drain]",
+"[-o @var{offset}]",
+"[--pattern=@var{pattern}]",
+"[-q]",
+"[-s @var{buffer_size}]",
+"[-S @var{step_size}]",
+"[-t @var{cache}]",
+"[-w]",
+"[-U]",
+"@var{filename}"
+],
+"check": [
+"[--object @var{objectdef}]",
+"[--image-opts]",
+"[-q]",
+"[-f @var{fmt}]",
+"[--output=@var{ofmt}]",
+"[-r [leaks | all]]",
+"[-T @var{src_cache}]",
+"[-U]",
+"@var{filename}"
+],
+"commit": [
+"[--object @var{objectdef}]",
+"[--image-opts]",
+"[-q]",
+"[-f @var{fmt}]",
+"[-t @var{cache}]",
+"[-b @var{base}]",
+"[-d]",
+"[-p]",
+"@var{filename}"
+],
+"compare": [
+"[--object @var{objectdef}]",
+"[--image-opts]",
+"[-f @var{fmt}]",
+"[-F @var{fmt}]",
+"[-T @var{src_cache}]",
+"[-p]",
+"[-q]",
+"[-s]",
+"[-U]",
+"@var{filename1}",
+"@var{filename2}"
+],
+"convert": [
+"[--object @var{objectdef}]",
+"[--image-opts]",
+"[--target-image-opts]",
+"[-U]",
+"[-C]",
+"[-c]",
+"[-p]",
+"[-q]",
+"[-n]",
+"[-f @var{fmt}]",
+"[-t @var{cache}]",
+"[-T @var{src_cache}]",
+"[-O

[Qemu-devel] [PATCH RFC 1/3] qemu-img: fix .hx and .texi disparity

It turns out that having options listed in three places continues to be
a bad idea. I'm still toying with the idea of an improved infrastructure
here, but in the meantime, another bandaid.

There are three locations:
(1) .hx file, formatted as texi
(2) .hx file, formatted as human readable.
(3) .texi file, as section headers, formatted as texi.

You can compare the two summaries within the .hx file like so:

Human-readable command summaries:
`./qemu-img --help | grep 'Command syntax' -A14`
Detokenized texi command summaries:
`grep "@item" qemu-img-cmds.hx | sed -E 's|@var\{([^\}]*?)\}|\1|g'`

You can compare the two separate texi summaries like so:

Texi command summaries:
`grep "@item" qemu-img-cmds.hx"`
Texi command headers:
grep -E "@item.*@var" qemu-img.texi | tail -14

Signed-off-by: John Snow 
---
 qemu-img.texi| 4 ++--
 qemu-img-cmds.hx | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/qemu-img.texi b/qemu-img.texi
index 3b6710a580..724f244ba1 100644
--- a/qemu-img.texi
+++ b/qemu-img.texi
@@ -197,7 +197,7 @@ Command description:
 
 @table @option
 
-@item amend [--object @var{objectdef}] [--image-opts] [-p] [-p] [-f @var{fmt}] 
[-t @var{cache}] -o @var{options} @var{filename}
+@item amend [--object @var{objectdef}] [--image-opts] [-p] [-q] [-f @var{fmt}] 
[-t @var{cache}] -o @var{options} @var{filename}
 
 Amends the image format specific @var{options} for the image file
 @var{filename}. Not all file formats support this operation.
@@ -424,7 +424,7 @@ To enumerate information about each disk image in the above 
chain, starting from
 qemu-img info --backing-chain snap2.qcow2
 @end example
 
-@item map [-f @var{fmt}] [--output=@var{ofmt}] @var{filename}
+@item map [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] 
[--output=@var{ofmt}] [-U] @var{filename}
 
 Dump the metadata of image @var{filename} and its backing file chain.
 In particular, this commands dumps the allocation state of every sector
diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index 1526f327a5..4b47f7495d 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -46,7 +46,7 @@ ETEXI
 DEF("convert", img_convert,
 "convert [--object objectdef] [--image-opts] [--target-image-opts] [-U] 
[-C] [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-B 
backing_file] [-o options] [-l snapshot_param] [-S sparse_size] [-m 
num_coroutines] [-W] filename [filename2 [...]] output_filename")
 STEXI
-@item convert [--object @var{objectdef}] [--image-opts] [--target-image-opts] 
[-U] [-c] [-p] [-q] [-n] [-f @var{fmt}] [-t @var{cache}] [-T @var{src_cache}] 
[-O @var{output_fmt}] [-B @var{backing_file}] [-o @var{options}] [-l 
@var{snapshot_param}] [-S @var{sparse_size}] [-m @var{num_coroutines}] [-W] 
@var{filename} [@var{filename2} [...]] @var{output_filename}
+@item convert [--object @var{objectdef}] [--image-opts] [--target-image-opts] 
[-U] [-C] [-c] [-p] [-q] [-n] [-f @var{fmt}] [-t @var{cache}] [-T 
@var{src_cache}] [-O @var{output_fmt}] [-B @var{backing_file}] [-o 
@var{options}] [-l @var{snapshot_param}] [-S @var{sparse_size}] [-m 
@var{num_coroutines}] [-W] @var{filename} [@var{filename2} [...]] 
@var{output_filename}
 ETEXI
 
 DEF("create", img_create,
-- 
2.17.2

[Qemu-devel] [PATCH RFC 0/3] qemu-img: remove command documentation duplication

This might hopefully cut down on the doc duplication/mismatching
until I can devise something more comprehensive.

Ideally, I'd like to redo all of the documentation for qemu-img
nearly from scratch; with a parser generator that helps generate
the documentation as well so they'll never get out of date.

That's probably quite a ways off, and maybe there are other
structural changes we want to make with respect to sphinx and
other build tools, so I am sending something very minimal instead.

This ought to be functionally identical down to the last char.

(I've re-included patch one which I have sent to the list separately,
purely as a dependency if you want to apply these patches.)

John Snow (3):
  qemu-img: fix .hx and .texi disparity
  pxtool: Add new qemu-img command info generation tool
  qemu-img.texi: use macros for command summaries

 qemu-img.texi  |  28 
 Makefile   |   8 +--
 qemu-img-cmds.json | 165 +
 qemu-img-cmds.hx   | 102 
 scripts/pxtool.py  |  53 +++
 5 files changed, 236 insertions(+), 120 deletions(-)
 create mode 100644 qemu-img-cmds.json
 delete mode 100644 qemu-img-cmds.hx
 create mode 100755 scripts/pxtool.py

-- 
2.17.2

Re: [Qemu-devel] [PATCH for-4.1] q35: acpi: do not create dummy MCFG table

2019-04-09 Thread Wei Yang

On Tue, Apr 09, 2019 at 05:00:37PM +0200, Igor Mammedov wrote:
>Dummy table (with signature "QEMU") creation came from original SeaBIOS
>codebase. And QEMU would have to keep it around if there were Q35 machine
>that depended on keeping ACPI tables blob constant size. Luckily there
>were no versioned Q35 machine types before commit:
>  (since 2.3) a1666142db acpi-build: make ROMs RAM blocks resizeable
>which obsoleted need to keep ACPI tables blob the same size on 
>source/destination.
>

I am not sure getting the "versioned Q35" correctly. Seems originally there is
q35-1.4?

commit bf3caa3dc17552b323cec6831301a22cfc98ecd5
Author: Paolo Bonzini 
Date:   Fri Feb 8 14:06:15 2013 +0100

pc: add compatibility machine types for 1.4

Adds both pc-i440fx-1.4 and pc-q35-1.4.



-- 
Wei Yang
Help you, Help me

[Qemu-devel] [RHEL-8.1 virt 1/2] memory: Fix the memory region type assignment order

2019-04-09 Thread Gary R Hook

BZ: 1667249
Branch: rhel-8.1.0
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1667249
Upstream Status: 4.0.0-rc1
Build Info: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=20980582
Conflicts: None

commit 2ddb89b00f947f785c9ca6742f28f954e3b75e62
Author: Singh, Brijesh 
Date:   Mon Feb 4 22:23:39 2019 +

memory: Fix the memory region type assignment order

Currently, a callback registered through the RAMBlock notifier
is not able to get the memory region type (i.e callback is not
able to use memory_region_is_ram_device function). This is
because mr->ram assignment happens _after_ the memory is allocated
whereas the callback is executed during allocation.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1667249
Suggested-by: Alex Williamson 
Cc: Paolo Bonzini 
Reviewed-by: Alex Williamson 
Signed-off-by: Brijesh Singh 
Message-Id: <20190204222322.26766-2-brijesh.si...@amd.com>
Signed-off-by: Paolo Bonzini 

Cc: Paolo Bonzini  
Cc: qemu-devel@nongnu.org 
---
 memory.c | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/memory.c b/memory.c
index 4974f972d5..04ff5e9108 100644
--- a/memory.c
+++ b/memory.c
@@ -1631,10 +1631,17 @@ void memory_region_init_ram_device_ptr(MemoryRegion *mr,
uint64_t size,
void *ptr)
 {
-memory_region_init_ram_ptr(mr, owner, name, size, ptr);
+memory_region_init(mr, owner, name, size);
+mr->ram = true;
+mr->terminates = true;
 mr->ram_device = true;
 mr->ops = _device_mem_ops;
 mr->opaque = mr;
+mr->destructor = memory_region_destructor_ram;
+mr->dirty_log_mask = tcg_enabled() ? (1 << DIRTY_MEMORY_CODE) : 0;
+/* qemu_ram_alloc_from_ptr cannot fail with ptr != NULL.  */
+assert(ptr != NULL);
+mr->ram_block = qemu_ram_alloc_from_ptr(size, ptr, mr, _fatal);
 }
 
 void memory_region_init_alias(MemoryRegion *mr,
-- 
2.18.1

[Qemu-devel] [RHEL-8.1 virt 2/2] target/i386: sev: Do not pin the ram device memory region

2019-04-09 Thread Gary R Hook

BZ: 1667249
Branch: rhel-8.1.0
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1667249
Upstream Status: 4.0.0-rc1
Build Info: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=20980582
Conflicts: None

commit cedc0ad539afbbb669dba9e73dfad2915bc1c25b
Author: Singh, Brijesh 
Date:   Mon Feb 4 22:23:40 2019 +

target/i386: sev: Do not pin the ram device memory region

The RAM device presents a memory region that should be handled
as an IO region and should not be pinned.

In the case of the vfio-pci, RAM device represents a MMIO BAR
and the memory region is not backed by pages hence
KVM_MEMORY_ENCRYPT_REG_REGION fails to lock the memory range.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1667249
Cc: Alex Williamson 
Cc: Paolo Bonzini 
Signed-off-by: Brijesh Singh 
Message-Id: <20190204222322.26766-3-brijesh.si...@amd.com>
Signed-off-by: Paolo Bonzini 

Cc: Paolo Bonzini  
Cc: Richard Henderson  
Cc: Eduardo Habkost  
Cc: qemu-devel@nongnu.org 
---
 target/i386/sev.c | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/target/i386/sev.c b/target/i386/sev.c
index 2395171acf..b8009b001a 100644
--- a/target/i386/sev.c
+++ b/target/i386/sev.c
@@ -130,6 +130,17 @@ sev_ram_block_added(RAMBlockNotifier *n, void *host, 
size_t size)
 {
 int r;
 struct kvm_enc_region range;
+ram_addr_t offset;
+MemoryRegion *mr;
+
+/*
+ * The RAM device presents a memory region that should be treated
+ * as IO region and should not be pinned.
+ */
+mr = memory_region_from_host(host, );
+if (mr && memory_region_is_ram_device(mr)) {
+   return;
+}
 
 range.addr = (__u64)(unsigned long)host;
 range.size = size;
-- 
2.18.1

[Qemu-devel] [RHEL-8.1 virt 0/2] Enable SEV VM to boot with assigned PCI device

2019-04-09 Thread Gary R Hook

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1667249

On an AMD SEV enabled host with an SEV enabled guest, attaching an
assigned device to the VM results in a failure to start the VM:

qemu-kvm: -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.2,addr=0x0: 
sev_ram_block_added: failed to register region (0x7fd96e6bb000+0x2) error 
'Cannot allocate memory'

In this example the assigned device is a simple Intel 82574L NIC:

01:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
Subsystem: Intel Corporation Gigabit CT Desktop Adapter
Flags: bus master, fast devsel, latency 0, IRQ 89, NUMA node 0
Memory at fb9c (32-bit, non-prefetchable) [size=128K]
Memory at fb90 (32-bit, non-prefetchable) [size=512K]

Note that the error indicates the region as (base+size) where a size
of 0x2 is 128K, which matches that of BAR0 for the device.
dmesg on the host also reports:

SVM: SEV: Failure locking 32 pages.

SEV guests make use of the RAMBlock notifier in QEMU to add page
pinnings for SEV; the kernel side of the call only knows how to pin
pages with get_user_pages(), and this currently faults on non-page
backed mappings (e.g. the mmap of an MMIO BAR).

To resolve this failure, change the order of the memory region type
assignment and avoid pinning device memory regions.

Cc: "Danilo C. L. de Paula" 
Cc: Eduardo Habkost 
Cc: Paolo Bonzini 
Cc: qemu-devel@nongnu.org
Cc: Richard Henderson 

Danilo C. L. de Paula (2):
  redhat: branching qemu-kvm to rhel-8.1.0
  redhat: renaming branch to rhel-8.1.0

Gary R Hook (2):
  Subject: memory: Fix the memory region type assignment order
  Subject: target/i386: sev: Do not pin the ram device memory region

 .gitpublish   |  6 +++---
 memory.c  |  9 -
 target/i386/sev.c | 11 +++
 3 files changed, 22 insertions(+), 4 deletions(-)

-- 
2.18.1

Re: [Qemu-devel] [PATCH for-4.1 2/2] spapr: Drop duplicate code in LSI mapping

2019-04-09 Thread David Gibson

On Mon, Apr 08, 2019 at 05:01:37PM +0200, Greg Kurz wrote:
> On Mon, 8 Apr 2019 13:40:54 +1000
> David Gibson  wrote:
> 
> > On Fri, Apr 05, 2019 at 06:30:48PM +0200, Greg Kurz wrote:
> > > LSI mapping in spapr currently open-codes standard PCI swizzling. It thus
> > > duplicates the code of pci_swizzle_map_irq_fn().
> > > 
> > > Expose the swizzling formula so that it can be used with a slot number
> > > when building the device tree. Simply drop pci_spapr_map_irq() and call
> > > pci_swizzle_map_irq_fn() instead.
> > > 
> > > Signed-off-by: Greg Kurz   
> > 
> > Applied to ppc-for-4.1, thanks.
> > 
> 
> Thanks ! I just realize that I had come up with a better title, at least
> from my POV, but I forgot to update before posting...
> 
> spapr: Drop duplicate PCI swizzle code

Sure, that is a better title and the 4.1 queue won't be sent off for a
bit, so I've updated that.

> No big deal if you keep the current title of course :)



> 
> 
> > > ---
> > >  hw/pci/pci.c |2 +-
> > >  hw/ppc/spapr_pci.c   |   24 
> > >  include/hw/pci/pci.h |4 
> > >  3 files changed, 9 insertions(+), 21 deletions(-)
> > > 
> > > diff --git a/hw/pci/pci.c b/hw/pci/pci.c
> > > index 35451c1e9987..bb52bafd8d1c 100644
> > > --- a/hw/pci/pci.c
> > > +++ b/hw/pci/pci.c
> > > @@ -1532,7 +1532,7 @@ void pci_device_set_intx_routing_notifier(PCIDevice 
> > > *dev,
> > >   */
> > >  int pci_swizzle_map_irq_fn(PCIDevice *pci_dev, int pin)
> > >  {
> > > -return (pin + PCI_SLOT(pci_dev->devfn)) % PCI_NUM_PINS;
> > > +return pci_swizzle(PCI_SLOT(pci_dev->devfn), pin);
> > >  }
> > >  
> > >  /***/
> > > diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> > > index c70688a0dc23..26850d0b94f4 100644
> > > --- a/hw/ppc/spapr_pci.c
> > > +++ b/hw/ppc/spapr_pci.c
> > > @@ -719,26 +719,10 @@ param_error_exit:
> > >  rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
> > >  }
> > >  
> > > -static int pci_spapr_swizzle(int slot, int pin)
> > > -{
> > > -return (slot + pin) % PCI_NUM_PINS;
> > > -}
> > > -
> > > -static int pci_spapr_map_irq(PCIDevice *pci_dev, int irq_num)
> > > -{
> > > -/*
> > > - * Here we need to convert pci_dev + irq_num to some unique value
> > > - * which is less than number of IRQs on the specific bus (4).  We
> > > - * use standard PCI swizzling, that is (slot number + pin number)
> > > - * % 4.
> > > - */
> > > -return pci_spapr_swizzle(PCI_SLOT(pci_dev->devfn), irq_num);
> > > -}
> > > -
> > >  static void pci_spapr_set_irq(void *opaque, int irq_num, int level)
> > >  {
> > >  /*
> > > - * Here we use the number returned by pci_spapr_map_irq to find a
> > > + * Here we use the number returned by pci_swizzle_map_irq_fn to find 
> > > a
> > >   * corresponding qemu_irq.
> > >   */
> > >  SpaprPhbState *phb = opaque;
> > > @@ -1744,7 +1728,7 @@ static void spapr_phb_realize(DeviceState *dev, 
> > > Error **errp)
> > >  >iowindow);
> > >  
> > >  bus = pci_register_root_bus(dev, NULL,
> > > -pci_spapr_set_irq, pci_spapr_map_irq, 
> > > sphb,
> > > +pci_spapr_set_irq, 
> > > pci_swizzle_map_irq_fn, sphb,
> > >  >memspace, >iospace,
> > >  PCI_DEVFN(0, 0), PCI_NUM_PINS, 
> > > TYPE_PCI_BUS);
> > >  phb->bus = bus;
> > > @@ -2236,14 +2220,14 @@ int spapr_populate_pci_dt(SpaprPhbState *phb, 
> > > uint32_t intc_phandle, void *fdt,
> > >  }
> > >  
> > >  /* Build the interrupt-map, this must matches what is done
> > > - * in pci_spapr_map_irq
> > > + * in pci_swizzle_map_irq_fn
> > >   */
> > >  _FDT(fdt_setprop(fdt, bus_off, "interrupt-map-mask",
> > >   _map_mask, sizeof(interrupt_map_mask)));
> > >  for (i = 0; i < PCI_SLOT_MAX; i++) {
> > >  for (j = 0; j < PCI_NUM_PINS; j++) {
> > >  uint32_t *irqmap = interrupt_map[i*PCI_NUM_PINS + j];
> > > -int lsi_num = pci_spapr_swizzle(i, j);
> > > +int lsi_num = pci_swizzle(i, j);
> > >  
> > >  irqmap[0] = cpu_to_be32(b_d(i)|b_fff(0));
> > >  irqmap[1] = 0;
> > > diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h
> > > index d87f5f93e9cd..033b2145d98c 100644
> > > --- a/include/hw/pci/pci.h
> > > +++ b/include/hw/pci/pci.h
> > > @@ -411,6 +411,10 @@ void pci_bus_irqs(PCIBus *bus, pci_set_irq_fn 
> > > set_irq, pci_map_irq_fn map_irq,
> > >  void pci_bus_irqs_cleanup(PCIBus *bus);
> > >  int pci_bus_get_irq_level(PCIBus *bus, int irq_num);
> > >  /* 0 <= pin <= 3 0 = INTA, 1 = INTB, 2 = INTC, 3 = INTD */
> > > +static inline int pci_swizzle(int slot, int pin)
> > > +{
> > > +return (slot + pin) % PCI_NUM_PINS;
> > > +}
> > >  int pci_swizzle_map_irq_fn(PCIDevice *pci_dev, int pin);
> > >  PCIBus *pci_register_root_bus(DeviceState

Re: [Qemu-devel] [PATCH for-4.0-maybe] device_tree: Fix integer overflowing in load_device_tree()

2019-04-09 Thread David Gibson

On Tue, Apr 09, 2019 at 08:59:55PM +0200, Philippe Mathieu-Daudé wrote:
> On 4/9/19 7:40 PM, Markus Armbruster wrote:
> > If the value of get_image_size() exceeds INT_MAX / 2 - 1, the
> > computation of @dt_size overflows to a negative number, which then
> > gets converted to a very large size_t for g_malloc0() and
> > load_image_size().  In the (fortunately improbable) case g_malloc0()
> > succeeds and load_image_size() survives, we'd assign the negative
> > number to *sizep.  What that would do to the callers I can't say, but
> > it's unlikely to be good.
> > 
> > Fix by rejecting images whose size would overflow.
> > 
> > Signed-off-by: Markus Armbruster 
> > ---
> >  device_tree.c | 4 
> >  1 file changed, 4 insertions(+)
> > 
> > diff --git a/device_tree.c b/device_tree.c
> > index 296278e12a..f8b46b3c73 100644
> > --- a/device_tree.c
> > +++ b/device_tree.c
> > @@ -84,6 +84,10 @@ void *load_device_tree(const char *filename_path, int 
> > *sizep)
> >   filename_path);
> >  goto fail;
> >  }
> > +if (dt_size > INT_MAX / 2 - 1) {
> 
> We should avoid magic number duplication.
> That said, this patch looks safe.
> 
> Reviewed-by: Philippe Mathieu-Daudé 

As Philippe says, the test condition is kinda ugly and I hope we can
refine it in future.  But since it fixes a real problem for now,

Reviewed-by: David Gibson 

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH for-4.1] hw/arm/smmuv3: Remove SMMUNotifierNode

2019-04-09 Thread Peter Xu

On Tue, Apr 09, 2019 at 06:02:19PM +0200, Eric Auger wrote:
> The SMMUNotifierNode struct is not necessary and brings extra
> complexity so let's remove it. We now directly track the SMMUDevices
> which have registered IOMMU MR notifiers.
> 
> This is inspired from the same transformation on intel-iommu
> done in commit b4a4ba0d68f50f218ee3957b6638dbee32a5eeef
> ("intel-iommu: remove IntelIOMMUNotifierNode")
> 
> Signed-off-by: Eric Auger 

Reviewed-by: Peter Xu 

-- 
Peter Xu

[Qemu-devel] [PULL 0/1] device-tree queue

The following changes since commit f151f8aca5cf5da24f6eb743a55a2233091ae532:

  migration/ram.c: Fix use-after-free in multifd_recv_unfill_packet() 
(2019-04-09 20:46:34 +0100)

are available in the Git repository at:

  g...@github.com:alistair23/qemu.git tags/pull-device-tree-20190409-1

for you to fetch changes up to 065e6298a75164b4347682b63381dbe752c2b156:

  device_tree: Fix integer overflowing in load_device_tree() (2019-04-09 
16:35:40 -0700)


Single device tree fix for 4.0

A single patch to avoid an overflow when loading device trees.


Markus Armbruster (1):
  device_tree: Fix integer overflowing in load_device_tree()

 device_tree.c | 4 
 1 file changed, 4 insertions(+)

[Qemu-devel] [PULL 1/1] device_tree: Fix integer overflowing in load_device_tree()

From: Markus Armbruster 

If the value of get_image_size() exceeds INT_MAX / 2 - 1, the
computation of @dt_size overflows to a negative number, which then
gets converted to a very large size_t for g_malloc0() and
load_image_size().  In the (fortunately improbable) case g_malloc0()
succeeds and load_image_size() survives, we'd assign the negative
number to *sizep.  What that would do to the callers I can't say, but
it's unlikely to be good.

Fix by rejecting images whose size would overflow.

Reported-by: Kurtis Miller 
Signed-off-by: Markus Armbruster 
Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Alistair Francis 
Message-Id: <20190409174018.25798-1-arm...@redhat.com>
---
 device_tree.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/device_tree.c b/device_tree.c
index 296278e12a..f8b46b3c73 100644
--- a/device_tree.c
+++ b/device_tree.c
@@ -84,6 +84,10 @@ void *load_device_tree(const char *filename_path, int *sizep)
  filename_path);
 goto fail;
 }
+if (dt_size > INT_MAX / 2 - 1) {
+error_report("Device tree file '%s' is too large", filename_path);
+goto fail;
+}
 
 /* Expand to 2x size to give enough room for manipulation.  */
 dt_size += 1;
-- 
2.21.0

Re: [Qemu-devel] [PATCH v4 1/6] vfio-ccw: make it safe to access channel programs

2019-04-09 Thread Halil Pasic

On Mon, 8 Apr 2019 19:07:47 +0200
Cornelia Huck  wrote:

> On Mon, 8 Apr 2019 13:02:12 -0400
> Farhan Ali  wrote:
> 
> > On 03/01/2019 04:38 AM, Cornelia Huck wrote:
> > > When we get a solicited interrupt, the start function may have
> > > been cleared by a csch, but we still have a channel program
> > > structure allocated. Make it safe to call the cp accessors in
> > > any case, so we can call them unconditionally.
> > > 
> > > While at it, also make sure that functions called from other parts
> > > of the code return gracefully if the channel program structure
> > > has not been initialized (even though that is a bug in the caller).
> > > 
> > > Reviewed-by: Eric Farman
> > > Signed-off-by: Cornelia Huck
> > > ---  
> > 
> > Hi Connie,
> > 
> > My series of fixes for vfio-ccw depends on this patch as I would like to 
> > call cp_free unconditionally :) (I had developed my code on top of your 
> > patches).
> > 
> > Could we pick this patch up as well when/if you pick up my patch series? 
> > I am in the process of sending out a v2.
> > 
> > Regarding this patch we could merge it as a stand alone patch, separate 
> > from this series. And also the patch LGTM
> > 
> > Reviewed-by: Farhan Ali 
> 
> Actually, I wanted to ask how people felt about merging this whole
> series for the next release :) It would be one thing less on my plate...
> 

Sorry I was not able to spend any significant amount of time on this
lately.

Gave the combined set (this + Farhans fio-ccw fixes for kernel
stacktraces v2) it a bit of smoke testing after some minor adjustments
to make it compile:

--- a/drivers/s390/cio/vfio_ccw_ops.c
+++ b/drivers/s390/cio/vfio_ccw_ops.c
@@ -13,6 +13,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "vfio_ccw_private.h"


I'm just running fio on a pass-through DASD and on some virto-blk disks
in parallel. My QEMU is today's vfio-ccw-caps from your repo.

I see stuff like this:
qemu-git: vfio-ccw: wirte I/O region failed with errno=16[1811/7332/0 iops] 
[eta 26m:34s]
[Thread 0x3ff75890910 (LWP 43803) exited]/7932KB/0KB /s] [1930/7932/0 iops] 
[eta 26m:33s]
[Thread 0x3ff6b7b7910 (LWP 43800) exited]/8030KB/0KB /s] [2031/8030/0 iops] 
[eta 26m:32s]
dasd-eckd 0.0.1234: An error occurred in the DASD device driver, reason=14 
caa27abe
INFO: task kworker/u6:1:26 blocked for more than 122 seconds.ps] [eta 
23m:26s]eta 23m:25s]
  Not tainted 5.1.0-rc3-00217-g6ab18dc #598
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kworker/u6:1D026  2 0x
Workqueue: writeback wb_workfn (flush-94:0)
Call Trace:
([<00ae23f2>] __schedule+0x4fa/0xc98)
 [<00ae2bda>] schedule+0x4a/0xb0 
 [<001b30ec>] io_schedule+0x2c/0x50 
 [<0071cc9c>] blk_mq_get_tag+0x1bc/0x310 
 [<0071571c>] blk_mq_get_request+0x1a4/0x4a8 
 [<00719d38>] blk_mq_make_request+0x100/0x728 
 [<0070aa0a>] generic_make_request+0x26a/0x478 
 [<0070ac76>] submit_bio+0x5e/0x178 
 [<004cfa2c>] ext4_io_submit+0x74/0x88 
 [<004cfd32>] ext4_bio_write_page+0x2d2/0x4c8 
 [<004aa5b4>] mpage_submit_page+0x74/0xa8 
 [<004aa676>] mpage_process_page_bufs+0x8e/0x1b8 
 [<004aa9bc>] mpage_prepare_extent_to_map+0x21c/0x390 
 [<004b063c>] ext4_writepages+0x4bc/0x11a0 
 [<0032ef7a>] do_writepages+0x3a/0xf0 
 [<00416226>] __writeback_single_inode+0x86/0x7a0 
 [<00417154>] writeback_sb_inodes+0x2cc/0x550 
 [<00417476>] __writeback_inodes_wb+0x9e/0xe8 
 [<004179e0>] wb_writeback+0x468/0x598 
 [<00418780>] wb_workfn+0x3b8/0x710 
 [<00199322>] process_one_work+0x25a/0x668 
 [<0019977a>] worker_thread+0x4a/0x428 
 [<001a1ae8>] kthread+0x150/0x170 
 [<00aeadda>] kernel_thread_starter+0x6/0xc 
 [<00aeadd4>] kernel_thread_starter+0x0/0xc 
4 locks held by kworker/u6:1/26:
 #0: 792cf224 ((wq_completion)writeback){+.+.}, at: 
process_one_work+0x19c/0x668
 #1: 9888c0e5 ((work_completion)(&(>dwork)->work)){+.+.}, at: 
process_one_work+0x19c/0x668
 #2: 2bfb76f0 (>s_umount_key#29){}, at: 
trylock_super+0x2e/0xa8
 #3: ff47fe1d (>s_journal_flag_rwsem){.+.+}, at: 
do_writepages+0x3a/0xf0


Since I haven't had the time to keep up lately, I will just trust Eric
and Farhan on whether this should be merged or not. From a quick look at
the code, and a quick stroll through my remaining memories, I think, there
are a couple of things, that I myself would try to solve differently. But
that is not a valid reason to hold this up.

I would like to spare the hustle of revisiting my old comments for everyone.
From the stability and utility perspective I'm pretty convinced we are
better off than without the patches in question.

TLDR:
If it is good enough for Eric and Farhan, I have no objections against merging.

Regards,
Halil

[Qemu-devel] [PATCH] migration: savevm: fix error code with migration blockers

2019-04-09 Thread Cole Robinson

The only caller that checks the error code is looking for != 0,
so returning false is incorrect.

Fixes: 5aaac467938 "migration: savevm: consult migration blockers"

Signed-off-by: Cole Robinson 
---
 migration/savevm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/migration/savevm.c b/migration/savevm.c
index 34bcad3807..a3dae4cf02 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -2544,7 +2544,7 @@ int save_snapshot(const char *name, Error **errp)
 AioContext *aio_context;
 
 if (migration_is_blocked(errp)) {
-return false;
+return ret;
 }
 
 if (!replay_can_snapshot()) {
-- 
2.21.0

Re: [Qemu-devel] [PATCH v6 48/49] linux-user: Split out ioctl

On 4/9/19 1:15 PM, Richard Henderson wrote:
> On 2/13/19 3:46 AM, Laurent Vivier wrote:
>> In fact it happens with sid, with ppc64 too, but not with m68k and sh4.
>>
>> And only with "unshare --pid --fork".
>>
>> Without "unshare", the result is:
>>
>>   Unsupported ioctl: cmd=0x80047476
>>   bash: cannot set terminal process group (11507): Function not implemented
>>   bash: no job control in this shell
>>   Unsupported ioctl: cmd=0x80087467
>>   Unsupported ioctl: cmd=0x80087467
>>   Unsupported ioctl: cmd=0x802c7415
> 
> I'm just now getting back to this, and cannot replicate the issue

... Oh nevermind.  I should be expecting no "unsupported" at all, since that's
what we get with master.


r~

Re: [Qemu-devel] [PATCH v6 48/49] linux-user: Split out ioctl

On 2/13/19 3:46 AM, Laurent Vivier wrote:
> On 13/02/2019 14:09, Laurent Vivier wrote:
>> Hi,
>>
>> this one is really a cut'n'paste but it introduces a problem with 
>> qemu-alpha, I don't know how/why:
>>
>>   $ sudo unshare --ipc --uts --pid --fork --kill-child --mount chroot 
>> chroot/alpha/sid/
>>   Unsupported ioctl: cmd=0x80047476
>>
>> And then it hangs.
>>
>> I'll try to debug.
> 
> In fact it happens with sid, with ppc64 too, but not with m68k and sh4.
> 
> And only with "unshare --pid --fork".
> 
> Without "unshare", the result is:
> 
>   Unsupported ioctl: cmd=0x80047476
>   bash: cannot set terminal process group (11507): Function not implemented
>   bash: no job control in this shell
>   Unsupported ioctl: cmd=0x80087467
>   Unsupported ioctl: cmd=0x80087467
>   Unsupported ioctl: cmd=0x802c7415

I'm just now getting back to this, and cannot replicate the issue, at least
rebased onto master.  With or without unshare, I see your "without unshare"
results.

The rebase onto master is at

  https://github.com/rth7680/qemu.git lu-split-6


r~

Re: [Qemu-devel] [PATCH for-4.1 v4 00/12] bundle edk2 platform firmware with QEMU

2019-04-09 Thread Michael S. Tsirkin

On Wed, Apr 10, 2019 at 01:00:10AM +0200, Laszlo Ersek wrote:
> Repo:   https://github.com/lersek/qemu.git
> Branch: edk2_build_v4
> 
> Posting a v4 is necessary because patch #6 needs
> - manual conflict resolution against some commits between v4.0.0-rc2 and
>   v4.0.0-rc3,
> - and corresponding commit message updates.
> 
> These are noted in more detail on the patch itself:
> 
>   [PATCH for-4.1 v4 06/12]
>   roms/Makefile: replace the $(EDK2_EFIROM) target with "edk2-basetools"
> 
> Due to this update, I had to drop all previous feedback tags on this
> patch. I plan to send a pullreq as soon as 4.0 is released, so please
> re-review / retest the above patch (and *only* the above patch) soon.
> (The notes on the patch list the tags individually that I've now
> dropped.)

Acked-by: Michael S. Tsirkin 

I also think pull requests for that is a good idea.
Firmware is pretty independent of the rest of qemu.

> Version 3, that is:
> 
>   [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with 
> QEMU
> 
> was posted at:
> 
>   https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg06148.html
>   http://mid.mail-archive.com/20190321113408.19929-1-lersek@redhat.com
> 
> Thanks,
> Laszlo
> 
> Laszlo Ersek (12):
>   roms: lift "edk2-funcs.sh" from "tests/uefi-test-tools/build.sh"
>   roms/edk2-funcs.sh: require gcc-4.8+ for building i386 and x86_64
>   tests/uefi-test-tools/build.sh: work around TianoCore#1607
>   roms/edk2: advance to tag edk2-stable201903
>   roms/edk2-funcs.sh: add the qemu_edk2_get_thread_count() function
>   roms/Makefile: replace the $(EDK2_EFIROM) target with "edk2-basetools"
>   roms: build edk2 firmware binaries and variable store templates
>   pc-bios: add edk2 firmware binaries and variable store templates
>   pc-bios: document the edk2 firmware images; add firmware descriptors
>   tests: add missing dependency to build QTEST_QEMU_BINARY, round 2
>   Makefile: install the edk2 firmware images and their descriptors
>   MAINTAINERS: add the "EDK2 Firmware" subsystem
> 
>  .gitignore |   1 +
>  MAINTAINERS|  12 +
>  Makefile   |  29 ++-
>  configure  |   1 +
>  pc-bios/README |  11 +
>  pc-bios/descriptors/50-edk2-i386-secure.json   |  34 +++
>  pc-bios/descriptors/50-edk2-x86_64-secure.json |  35 +++
>  pc-bios/descriptors/60-edk2-aarch64.json   |  31 +++
>  pc-bios/descriptors/60-edk2-arm.json   |  31 +++
>  pc-bios/descriptors/60-edk2-i386.json  |  33 +++
>  pc-bios/descriptors/60-edk2-x86_64.json|  34 +++
>  pc-bios/edk2-aarch64-code.fd.bz2   | Bin 0 -> 1177603 bytes
>  pc-bios/edk2-arm-code.fd.bz2   | Bin 0 -> 1173662 bytes
>  pc-bios/edk2-arm-vars.fd.bz2   | Bin 0 -> 263 bytes
>  pc-bios/edk2-i386-code.fd.bz2  | Bin 0 -> 1688659 bytes
>  pc-bios/edk2-i386-secure-code.fd.bz2   | Bin 0 -> 1881979 bytes
>  pc-bios/edk2-i386-vars.fd.bz2  | Bin 0 -> 190 bytes
>  pc-bios/edk2-licenses.txt  | 209 
>  pc-bios/edk2-x86_64-code.fd.bz2| Bin 0 -> 1669280 bytes
>  pc-bios/edk2-x86_64-secure-code.fd.bz2 | Bin 0 -> 1901210 bytes
>  roms/Makefile  |   9 +-
>  roms/Makefile.edk2 | 148 
>  roms/edk2  |   2 +-
>  roms/edk2-build.sh |  55 +
>  roms/edk2-funcs.sh | 253 
>  tests/Makefile.include |   2 +-
>  tests/uefi-test-tools/build.sh | 100 +---
>  27 files changed, 934 insertions(+), 96 deletions(-)
>  create mode 100644 pc-bios/descriptors/50-edk2-i386-secure.json
>  create mode 100644 pc-bios/descriptors/50-edk2-x86_64-secure.json
>  create mode 100644 pc-bios/descriptors/60-edk2-aarch64.json
>  create mode 100644 pc-bios/descriptors/60-edk2-arm.json
>  create mode 100644 pc-bios/descriptors/60-edk2-i386.json
>  create mode 100644 pc-bios/descriptors/60-edk2-x86_64.json
>  create mode 100644 pc-bios/edk2-aarch64-code.fd.bz2
>  create mode 100644 pc-bios/edk2-arm-code.fd.bz2
>  create mode 100644 pc-bios/edk2-arm-vars.fd.bz2
>  create mode 100644 pc-bios/edk2-i386-code.fd.bz2
>  create mode 100644 pc-bios/edk2-i386-secure-code.fd.bz2
>  create mode 100644 pc-bios/edk2-i386-vars.fd.bz2
>  create mode 100644 pc-bios/edk2-licenses.txt
>  create mode 100644 pc-bios/edk2-x86_64-code.fd.bz2
>  create mode 100644 pc-bios/edk2-x86_64-secure-code.fd.bz2
>  create mode 100644 roms/Makefile.edk2
>  create mode 100755 roms/edk2-build.sh
>  create mode 100644 roms/edk2-funcs.sh
> 
> -- 
> 2.19.1.3.g30247aa5d201

[Qemu-devel] [PATCH for-4.1 v4 12/12] MAINTAINERS: add the "EDK2 Firmware" subsystem

We now have the edk2 submodule, somewhat elaborate build helpers for it,
and even a UEFI application written against edk2 whose genuine home is the
QEMU repository. Add the "EDK2 Firmware" subsystem such that all relevant
pathnames be covered.

Suggested-by: Daniel P. Berrangé 
Suggested-by: Michael S. Tsirkin 
Suggested-by: Philippe Mathieu-Daudé 
Signed-off-by: Laszlo Ersek 
Reviewed-by: Michal Privoznik 
Reviewed-by: Philippe Mathieu-Daudé 
Tested-by: Philippe Mathieu-Daudé 
Tested-by: Igor Mammedov 
Reviewed-by: Michael S. Tsirkin 
---

Notes:
v4:

- no change

v3:

- pick up Michal's R-b

- pick up Phil's R-b/T-b

- pick up Igor's T-b

- pick up Michael's R-b

v2:

- new patch [Dan, Michael, Phil]

- compared to the suggestions, upgrade the maintenance status ("S") from
  Maintained to Supported

 MAINTAINERS | 12 
 1 file changed, 12 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 56139ac8ab05..d7505e074a0d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -,6 +,18 @@ F: include/hw/i2c/smbus_master.h
 F: include/hw/i2c/smbus_slave.h
 F: include/hw/i2c/smbus_eeprom.h
 
+EDK2 Firmware
+M: Laszlo Ersek 
+M: Philippe Mathieu-Daudé 
+S: Supported
+F: pc-bios/descriptors/??-edk2-*.json
+F: pc-bios/edk2-*
+F: roms/Makefile.edk2
+F: roms/edk2
+F: roms/edk2-*
+F: tests/data/uefi-boot-images/
+F: tests/uefi-test-tools/
+
 Usermode Emulation
 --
 Overall
-- 
2.19.1.3.g30247aa5d201

[Qemu-devel] [PATCH for-4.1 v4 11/12] Makefile: install the edk2 firmware images and their descriptors

Decompress and install the edk2 firmware blobs as part of "make install",
unless blob installation was disabled with configure's "--disable-blobs"
option.

Additionally, decompress the blobs as a pre-requisite for building softmmu
binaries -- this is helpful for both "make check" and other ad-hoc tests
one might want to run in the build directory.

Signed-off-by: Laszlo Ersek 
Reviewed-by: Michal Privoznik 
Reviewed-by: Philippe Mathieu-Daudé 
Reviewed-by: Michael S. Tsirkin 
---

Notes:
v4:

- no change

v3:

- pick up Michal's R-b

- pick up Phil's R-b

- pick up Michael's R-b

- Decompress fd.bz2 files with bzip2 rather than fd.xz files with xz, so
  that decompression at "make install" time succeed on older build OSes
  too [Peter]. Note that "BUNZIP2" matches the name of the actual
  command "bunzip2" (i.e. it is sensible), and that it consists of 7
  characters, satisfying the quiet-command limit.

- do not pick up Phil's T-b, consequently

- do not pick up Igor's T-b for the same reason

v2:

- adapt to tracking the edk2 flash device files in compressed form [Dan,
  Michael, Phil]

- do not pick up Michal's and Michael's R-b's due to the above change

 configure  |  1 +
 Makefile   | 29 +++-
 .gitignore |  1 +
 3 files changed, 30 insertions(+), 1 deletion(-)

diff --git a/configure b/configure
index 1c563a70276a..e1ad87b697e0 100755
--- a/configure
+++ b/configure
@@ -7892,6 +7892,7 @@ for bios_file in \
 $source_path/pc-bios/*.img \
 $source_path/pc-bios/openbios-* \
 $source_path/pc-bios/u-boot.* \
+$source_path/pc-bios/edk2-*.fd.bz2 \
 $source_path/pc-bios/palcode-*
 do
 LINKS="$LINKS pc-bios/$(basename $bios_file)"
diff --git a/Makefile b/Makefile
index 04a0d4505085..626a04d305cc 100644
--- a/Makefile
+++ b/Makefile
@@ -296,6 +296,10 @@ ui/input-keymap-%.c: $(KEYCODEMAP_GEN) $(KEYCODEMAP_CSV) 
$(SRC_PATH)/ui/Makefile
 $(KEYCODEMAP_GEN): .git-submodule-status
 $(KEYCODEMAP_CSV): .git-submodule-status
 
+edk2-decompressed = $(basename $(wildcard pc-bios/edk2-*.fd.bz2))
+pc-bios/edk2-%.fd: pc-bios/edk2-%.fd.bz2
+   $(call quiet-command,bzip2 -d -c $< > $@,"BUNZIP2",$<)
+
 # Don't try to regenerate Makefile or configure
 # We don't generate any of them
 Makefile: ;
@@ -445,6 +449,7 @@ $(SOFTMMU_SUBDIR_RULES): $(block-obj-y)
 $(SOFTMMU_SUBDIR_RULES): $(crypto-obj-y)
 $(SOFTMMU_SUBDIR_RULES): $(io-obj-y)
 $(SOFTMMU_SUBDIR_RULES): config-all-devices.mak
+$(SOFTMMU_SUBDIR_RULES): $(edk2-decompressed)
 
 subdir-%:
$(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) -C $* V="$(V)" 
TARGET_DIR="$*/" all,)
@@ -633,6 +638,7 @@ clean:
! -path ./roms/edk2/ArmPkg/Library/GccLto/liblto-arm.a \
! -path ./roms/edk2/BaseTools/Source/Python/UPT/Dll/sqlite3.dll 
\
-exec rm {} +
+   rm -f $(edk2-decompressed)
rm -f $(filter-out %.tlb,$(TOOLS)) $(HELPERS-y) qemu-ga TAGS cscope.* 
*.pod *~ */*~
rm -f fsdev/*.pod scsi/*.pod
rm -f qemu-img-cmds.h
@@ -723,9 +729,14 @@ spapr-rtas.bin slof.bin skiboot.lid \
 palcode-clipper \
 u-boot.e500 u-boot-sam460-20100605.bin \
 qemu_vga.ndrv \
+edk2-licenses.txt \
 hppa-firmware.img
+
+DESCS=50-edk2-i386-secure.json 50-edk2-x86_64-secure.json \
+60-edk2-aarch64.json 60-edk2-arm.json 60-edk2-i386.json 60-edk2-x86_64.json
 else
 BLOBS=
+DESCS=
 endif
 
 # Note that we manually filter-out the non-Sphinx documentation which
@@ -786,7 +797,8 @@ endif
 
 ICON_SIZES=16x16 24x24 32x32 48x48 64x64 128x128 256x256 512x512
 
-install: all $(if $(BUILD_DOCS),install-doc) install-datadir 
install-localstatedir
+install: all $(if $(BUILD_DOCS),install-doc) install-datadir 
install-localstatedir \
+   $(if $(INSTALL_BLOBS),$(edk2-decompressed))
 ifneq ($(TOOLS),)
$(call install-prog,$(subst 
qemu-ga,qemu-ga$(EXESUF),$(TOOLS)),$(DESTDIR)$(bindir))
 endif
@@ -808,6 +820,21 @@ ifneq ($(BLOBS),)
set -e; for x in $(BLOBS); do \
$(INSTALL_DATA) $(SRC_PATH)/pc-bios/$$x 
"$(DESTDIR)$(qemu_datadir)"; \
done
+endif
+ifdef INSTALL_BLOBS
+   set -e; for x in $(edk2-decompressed); do \
+   $(INSTALL_DATA) $$x "$(DESTDIR)$(qemu_datadir)"; \
+   done
+endif
+ifneq ($(DESCS),)
+   $(INSTALL_DIR) "$(DESTDIR)$(qemu_datadir)/firmware"
+   set -e; tmpf=$$(mktemp); trap 'rm -f -- "$$tmpf"' EXIT; \
+   for x in $(DESCS); do \
+   sed -e 's,@DATADIR@,$(DESTDIR)$(qemu_datadir),' \
+   "$(SRC_PATH)/pc-bios/descriptors/$$x" > "$$tmpf"; \
+   $(INSTALL_DATA) "$$tmpf" \
+   "$(DESTDIR)$(qemu_datadir)/firmware/$$x"; \
+   done
 endif
for s in $(ICON_SIZES); do \
mkdir -p "$(DESTDIR)/$(qemu_icondir)/hicolor/$${s}/apps"; \
diff --git a/.gitignore b/.gitignore
index 8f782218044e..fd6e6c38c7b9 100644
--- a/.gitignore
+++

[Qemu-devel] [PATCH for-4.1 v4 08/12] pc-bios: add edk2 firmware binaries and variable store templates

Add the files built by the last patch: (compressed) binaries, and the
cumulative license text that covers them.

Signed-off-by: Laszlo Ersek 
Reviewed-by: Michal Privoznik 
Reviewed-by: Philippe Mathieu-Daudé 
Reviewed-by: Michael S. Tsirkin 
---

Notes:
v4:

- no change

v3:

- pick up Michal's R-b

- pick up Phil's R-b

- pick up Michael's R-b

- Compress FD files with bzip2 rather than xz, so that decompression at
  "make install" time succeed on older build OSes too. The total size
  grows from 9,394,072 bytes to 9,492,846. [Peter]

- do not pick up Igor's T-b therefore

v2:

- capture the compressed build outputs of the last patch; slightly
  update the commit message [Dan, Michael, Phil]

- consequently, do not pick up Michal's and Michael's R-b's

 pc-bios/edk2-licenses.txt  | 209 
 pc-bios/edk2-aarch64-code.fd.bz2   | Bin 0 -> 1177603 bytes
 pc-bios/edk2-arm-code.fd.bz2   | Bin 0 -> 1173662 bytes
 pc-bios/edk2-arm-vars.fd.bz2   | Bin 0 -> 263 bytes
 pc-bios/edk2-i386-code.fd.bz2  | Bin 0 -> 1688659 bytes
 pc-bios/edk2-i386-secure-code.fd.bz2   | Bin 0 -> 1881979 bytes
 pc-bios/edk2-i386-vars.fd.bz2  | Bin 0 -> 190 bytes
 pc-bios/edk2-x86_64-code.fd.bz2| Bin 0 -> 1669280 bytes
 pc-bios/edk2-x86_64-secure-code.fd.bz2 | Bin 0 -> 1901210 bytes
 9 files changed, 209 insertions(+)

diff --git a/pc-bios/edk2-licenses.txt b/pc-bios/edk2-licenses.txt
new file mode 100644
index ..8bdb1abc993e
--- /dev/null
+++ b/pc-bios/edk2-licenses.txt
@@ -0,0 +1,209 @@
+==> edk2/License.txt <==
+Copyright (c) 2004 - 2016, Intel Corporation. All rights reserved.
+Copyright (c) 2008 - 2010, Apple Inc. All rights reserved.
+Copyright (c) 2011 - 2015, ARM Limited. All rights reserved.
+Copyright (c) 2014 - 2015, Linaro Limited. All rights reserved.
+Copyright (c) 2013 - 2015, Red Hat, Inc.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions
+are met:
+
+* Redistributions of source code must retain the above copyright
+  notice, this list of conditions and the following disclaimer.
+* Redistributions in binary form must reproduce the above copyright
+  notice, this list of conditions and the following disclaimer in
+  the documentation and/or other materials provided with the
+  distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
+INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
+BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
+ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGE.
+
+==> edk2/OvmfPkg/License.txt <==
+Copyright (c) 2012, Intel Corporation. All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions
+are met:
+
+* Redistributions of source code must retain the above copyright
+  notice, this list of conditions and the following disclaimer.
+* Redistributions in binary form must reproduce the above copyright
+  notice, this list of conditions and the following disclaimer in
+  the documentation and/or other materials provided with the
+  distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
+INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
+BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
+ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGE.
+
+
+Some files are subject to the following license, the MIT license. Those files
+are located in:
+- OvmfPkg/Include/IndustryStandard/Xen/
+- OvmfPkg/XenBusDxe/
+- OvmfPkg/XenPvBlkDxe/
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction,

[Qemu-devel] [PATCH for-4.1 v4 10/12] tests: add missing dependency to build QTEST_QEMU_BINARY, round 2

In commit b94b330e2333 ("tests: add missing dependency to build
QTEST_QEMU_BINARY", 2017-07-31), Phil fixed the dependency list of make
target "check-qtest-%". Namely, the recipe would set QTEST_QEMU_BINARY to
the softmmu emulator for the emulation target, but the prerequisites
didn't include the emulator.

The same issue affects the "check-report-qtest-%.tap" make target, which
is the other make target whose recipe sets QTEST_QEMU_BINARY:

> $ make -j4 check-report-qtest-aarch64.tap
>   TAP check-report-qtest-aarch64.tap
> sh: /.../aarch64-softmmu/qemu-system-aarch64: No such file or directory

Apply Phil's fix to this make target too.

Signed-off-by: Laszlo Ersek 
Reviewed-by: Michal Privoznik 
Tested-by: Igor Mammedov 
Reviewed-by: Michael S. Tsirkin 
---

Notes:
v4:

- no change

v3:

- pick up Michal's R-b

- pick up Igor's T-b

- pick up Michael's R-b

v2:

- new patch, relied upon by the next patch

 tests/Makefile.include | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 36fc73fef55a..e2432d5e7712 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -912,7 +912,7 @@ check-speed: $(check-speed-y)
 
 # gtester tests with TAP output
 
-$(patsubst %, check-report-qtest-%.tap, $(QTEST_TARGETS)): 
check-report-qtest-%.tap: $(check-qtest-y)
+$(patsubst %, check-report-qtest-%.tap, $(QTEST_TARGETS)): 
check-report-qtest-%.tap: subdir-%-softmmu $(check-qtest-y)
$(call do_test_tap, $(check-qtest-$*-y) $(check-qtest-generic-y), \
  QTEST_QEMU_BINARY=$*-softmmu/qemu-system-$* \
  QTEST_QEMU_IMG=qemu-img$(EXESUF))
-- 
2.19.1.3.g30247aa5d201

[Qemu-devel] [PATCH for-4.1 v4 03/12] tests/uefi-test-tools/build.sh: work around TianoCore#1607

The edk2-stabe201903 release introduced Python3 support to edk2's
BaseTools; however the Python3 enablement breaks in a corner case (which
is nevertheless supported by the edk2 community), namely the in-module
parallelization that we utilize.

This is tracked under
. For now, work
around the issue (in advance) by forcing Python2. (The workaround is a
no-op before we move to edk2-stabe201903 in the roms/edk2 submodule.)

Signed-off-by: Laszlo Ersek 
Reviewed-by: Philippe Mathieu-Daudé 
Tested-by: Philippe Mathieu-Daudé 
Reviewed-by: Michal Privoznik 
Reviewed-by: Michael S. Tsirkin 
Tested-by: Igor Mammedov 
---

Notes:
v4:

- no change

v3:

- pick up Igor's T-b

v2:

- pick up Phil's R-b / T-b

- pick up Michal's and Michael's R-b's

 tests/uefi-test-tools/build.sh | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/tests/uefi-test-tools/build.sh b/tests/uefi-test-tools/build.sh
index e2b52c855c39..8aa7935c43bb 100755
--- a/tests/uefi-test-tools/build.sh
+++ b/tests/uefi-test-tools/build.sh
@@ -29,6 +29,9 @@ export PACKAGES_PATH=$(realpath -- "$edk2_dir")
 export WORKSPACE=$PWD
 mkdir -p Conf
 
+# Work around .
+export PYTHON_COMMAND=python2
+
 # Source "edksetup.sh" carefully.
 set +e +u +C
 source "$PACKAGES_PATH/edksetup.sh"
-- 
2.19.1.3.g30247aa5d201

[Qemu-devel] [PATCH for-4.1 v4 07/12] roms: build edk2 firmware binaries and variable store templates

Add the "efi" target to "Makefile".

Introduce "Makefile.edk2" for building and cleaning the firmware images
and varstore templates.

Collect the common bits from the recipes in the helper script
"edk2-build.sh".

Signed-off-by: Laszlo Ersek 
Reviewed-by: Michal Privoznik 
Reviewed-by: Philippe Mathieu-Daudé 
Reviewed-by: Michael S. Tsirkin 
---

Notes:
v4:

- no change

v3:

- pick up Michal's R-b

- pick up Phil's R-b

- pick up Michael's R-b

- compress FD files with bzip2 rather than xz, so that decompression at
  "make install" time succeed on older build OSes too [Peter]

- do not pick up Phil's T-b, consequently

- do not pick up Igor's T-b for the same reason

v2:

- drop comma after copyright year, in both new files [Eric]

- define the SHELL macro as /bin/bash near the top of "Makefile.edk2",
  so that various bashisms (e.g. the "source" built-in, and the ==
  operator of "[") work even if /bin/sh isn't bash [Phil, Eric]

- rework "Makefile.edk2" to produce xz-compressed flash device files
  [Dan, Michael, Phil]:

  - add implicit rule for compression,

  - mark uncompressed FD files as intermediate,

  - factor out the "flashdevs" macro for sharing between the "all" and
".INTERMEDIATE" targets

- due to said rework above, do not pick up Phil's R-b / T-b, and
  Michal's and Michael's R-b's

 roms/Makefile  |   5 +
 roms/Makefile.edk2 | 148 
 roms/edk2-build.sh |  55 
 3 files changed, 208 insertions(+)

diff --git a/roms/Makefile b/roms/Makefile
index 93c3d467be14..0ce84a45ad57 100644
--- a/roms/Makefile
+++ b/roms/Makefile
@@ -61,6 +61,7 @@ default:
@echo "  skiboot-- update skiboot.lid"
@echo "  u-boot.e500-- update u-boot.e500"
@echo "  u-boot.sam460  -- update u-boot.sam460"
+   @echo "  efi-- update UEFI (edk2) platform firmware"
 
 bios: build-seabios-config-seabios-128k build-seabios-config-seabios-256k
cp seabios/builds/seabios-128k/bios.bin ../pc-bios/bios.bin
@@ -156,6 +157,9 @@ skiboot:
$(MAKE) -C skiboot CROSS=$(powerpc64_cross_prefix)
cp skiboot/skiboot.lid ../pc-bios/skiboot.lid
 
+efi: edk2-basetools
+   $(MAKE) -f Makefile.edk2
+
 clean:
rm -rf seabios/.config seabios/out seabios/builds
$(MAKE) -C sgabios clean
@@ -166,3 +170,4 @@ clean:
rm -rf u-boot/build.e500
$(MAKE) -C u-boot-sam460ex distclean
$(MAKE) -C skiboot clean
+   $(MAKE) -f Makefile.edk2 clean
diff --git a/roms/Makefile.edk2 b/roms/Makefile.edk2
new file mode 100644
index ..822c547fec64
--- /dev/null
+++ b/roms/Makefile.edk2
@@ -0,0 +1,148 @@
+# Makefile for building firmware binaries and variable store templates for a
+# number of virtual platforms in edk2.
+#
+# Copyright (C) 2019 Red Hat, Inc.
+#
+# This program and the accompanying materials are licensed and made available
+# under the terms and conditions of the BSD License that accompanies this
+# distribution. The full text of the license may be found at
+# .
+#
+# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, WITHOUT
+# WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
+
+SHELL = /bin/bash
+
+toolchain = $(shell source ./edk2-funcs.sh && qemu_edk2_get_toolchain $(1))
+
+licenses := \
+   edk2/License.txt \
+   edk2/OvmfPkg/License.txt \
+   edk2/CryptoPkg/Library/OpensslLib/openssl/LICENSE
+
+# The "edk2-arm-vars.fd" varstore template is suitable for aarch64 as well.
+# Similarly, the "edk2-i386-vars.fd" varstore template is suitable for x86_64
+# as well, independently of "secure" too.
+flashdevs := \
+   aarch64-code \
+   arm-code \
+   i386-code \
+   i386-secure-code \
+   x86_64-code \
+   x86_64-secure-code \
+   \
+   arm-vars \
+   i386-vars
+
+all: $(foreach flashdev,$(flashdevs),../pc-bios/edk2-$(flashdev).fd.bz2) \
+   ../pc-bios/edk2-licenses.txt
+
+../pc-bios/edk2-%.fd.bz2: ../pc-bios/edk2-%.fd
+   bzip2 -9 -c $< > $@
+
+# When the build completes, we need not keep the uncompressed flash device
+# files.
+.INTERMEDIATE: $(foreach flashdev,$(flashdevs),../pc-bios/edk2-$(flashdev).fd)
+
+submodules:
+   cd edk2 && git submodule update --init --force
+
+# See notes on the ".NOTPARALLEL" target and the "+" indicator in
+# "tests/uefi-test-tools/Makefile".
+.NOTPARALLEL:
+
+../pc-bios/edk2-aarch64-code.fd: submodules
+   +./edk2-build.sh \
+   aarch64 \
+   --arch=AARCH64 \
+   --platform=ArmVirtPkg/ArmVirtQemu.dsc \
+   -D NETWORK_IP6_ENABLE \
+   -D HTTP_BOOT_ENABLE
+   cp edk2/Build/ArmVirtQemu-AARCH64/DEBUG_$(call 
toolchain,aarch64)/FV/QEMU_EFI.fd \
+   $@
+   truncate --size=64M $@
+

[Qemu-devel] [PATCH for-4.1 v4 04/12] roms/edk2: advance to tag edk2-stable201903

Update the roms/edk2 submodule hash from edk2-stable201811 to
edk2-stable201903. The release notes are available at
.

$ git shortlog edk2-stable201811..edk2-stable201903

Achin Gupta (9):
  ArmPkg: Add PCDs needed for MM communication driver.
  ArmPkg/Drivers: Add EFI_MM_COMMUNICATION_PROTOCOL DXE driver.
  ArmPkg/Include: Add MM interface SVC return codes.
  ArmPkg/ArmMmuLib: Add MMU Library suitable for use in S-EL0.
  StandaloneMmPkg: Add missing dependency on PL011UartClockLib
  StandaloneMmPkg: Enforce alignment check for AArch64
  StandaloneMmPkg: Zero data structure explicitly
  StandaloneMmPkg: Replace dependency on ArmMmuLib
  StandaloneMmPkg: Update dependency on PeCoffExtraActionLib

Albecki, Mateusz (1):
  MdeModulePkg/SdMmcPciHcDxe Fix eMMC HS400 switch sequence

Alex James (2):
  StdLib/sys/termios: Define cc_t as unsigned
  StdLib/Environs: Avoid infinite recursion in _Exit

Antoine Coeur (5):
  ArmVirtPkg: Fix various typos
  CryptoPkg: Fix various typos
  CorebootPayloadPkg: Fix various typos
  CorebootModulePkg: Fix various typos
  BaseTools: Various typo

Ard Biesheuvel (116):
  MdePkg/BaseIoLibIntrinsicArmVirt ARM: avoid double word loads and stores
  ArmPkg/ArmGicDxe ARM: fix encoding for GICv3 interrupt acknowledge
  ArmPlatformPkg: clear frame pointer in startup code
  ArmVirtPkg/PrePi: clear frame pointer in startup code
  ArmPkg/ArmSmcPsciResetSystemLib: add missing call to ExitBootServices()
  ArmPkg: remove now unused BsdLib.h
  ArmPlatformPkg/NorFlashDxe: prepare for devicepath format change
  ArmPlatformPkg/NorFlashDxe: use one GUID plus index to identify flash 
banks
  ArmVirtPkg/FdtClientDxe: take DT node 'status' properties into account
  ArmVirtPkg/NorFlashQemuLib: discover NOR flash banks dynamically
  ArmPlatformPkg/NorFlashPlatformLib: remove unused Guid member from struct
  ArmPkg/ArmPkg.dsc: move ArmMmuStandaloneMmLib.inf to AARCH64 section
  EmbeddedPkg/TemplateSec: remove unused module
  EmbeddedPkg/PrePiHobLib: drop CreateHobList() from library
  ArmVirtPkg/FdtPciHostBridgeLib: map ECAM and I/O spaces in GCD memory map
  ArmVirtPkg/QemuVirtMemInfoLib: remove 1:1 mapping of top of PA range
  MdePkg/ProcessorBind.h AARCH64: limit MAX_ADDRESS to 48 bits
  ArmPkg/ArmLib: add support for reading the max physical address space size
  ArmVirtPkg/XenVirtMemInfoLib: refactor reading of the PA space size
  ArmPkg/ArmMmuLib: take the CPU supported maximum PA space into account
  ArmPkg/CpuPei: base GCD memory space size on CPU's PA range
  ArmPlatformPkg/PrePi: base GCD memory space size on CPU's PA range
  ArmVirtPkg/PrePi: base GCD memory space size on CPU's PA range
  BeagleBoardPkg/PrePi: base GCD memory space size on CPU's PA range
  ArmPlatformPkg/PlatformPei: drop unused PCD references
  EmbeddedPkg/PrePiLib: drop unused PCD reference
  ArmVirtPkg: drop PcdPrePiCpuMemorySize assignments from all platforms
  EmbeddedPkg/EmbeddedPkg.dec: drop PcdPrePiCpuMemorySize declarations
  ArmPkg/ArmMmuLib ARM: handle unmapped section in GetMemoryRegion()
  ArmPkg/ArmMmuLib ARM: handle unmapped sections when updating permissions
  ArmVirtPkg/NorFlashQemuLib: disregard our primary FV
  ArmVirtPkg/QemuVirtMemInfoLib: trim the MMIO region mapping
  BaseTools/CommonLib: avoid using 'native' word size in IP address handling
  BaseTools/CommonLib: use explicit 64-bit type in Strtoi()
  BaseTools/DevicePath: use explicit 64-bit number parsing routines
  BaseTools/CommonLib: add definition of MAX_UINT32
  BaseTools/DevicePath: use MAX_UINT32 as default device path max size
  BaseTools/CommonLib: get rid of 'native' type string parsing routines
  BaseTools/CommonLib: drop definition of MAX_UINTN
  BaseTools/CommonLib: drop the use of MAX_ADDRESS
  Revert "MdePkg/ProcessorBind.h AARCH64: limit MAX_ADDRESS to 48 bits"
  MdeModulePkg/FileExplorerLib: avoid packed struct for program data
  BaseTools/tools_def AARCH64 RELEASE: move GCC49/GGC5 to 4 KB alignment
  ArmVirtPkg/ArmVirtQemuKernel ARM: make some PCD settings apply to ARM
  ArmVirtPkg/PrePiUniCoreRelocatable CLANG38: work around build issues
  BaseTools/GenFw ARM: don't permit R_ARM_GOT_PREL relocations
  MdePkg/BaseMemoryLibOptDxe ARM: add missing function annotations
  BaseTools/tools_def ARM CLANG35: work around -mno-movt option name change
  ArmVirtPkg/PrePi ARM CLANG35: drop incompatible command line option
  ArmVirtPkg/ArmVirt.dsc.inc: define TcpIoLib resolution unconditionally
  ArmPkg: remove redundant _ARM_PLATFORM_FLAGS overrides
  EmbeddedPkg: remove GdbDebugAgent library
  BaseTools/tools_def ARM: emit PIC veneers
  ArmPkg/DefaultExceptionHandlerLib ARM: avoid endless loop in

[Qemu-devel] [PATCH for-4.1 v4 09/12] pc-bios: document the edk2 firmware images; add firmware descriptors

Update the README file with information on the images added previously,
and provide firmware descriptor documents that conform to
"docs/interop/firmware.json".

Signed-off-by: Laszlo Ersek 
Reviewed-by: Michal Privoznik 
Reviewed-by: Michael S. Tsirkin 
Tested-by: Igor Mammedov 
---

Notes:
v4:

- no change

v3:

- refer to "edk2-*.fd.bz2" rather than "edk2-*.fd.xz" in the README
  [Peter]

- pick up Igor's T-b still

v2:

- refer to "edk2-*.fd.xz" rather than "edk2-*.fd" in the README hunk
  [Dan, Michael, Phil]

- pick up Michal's and Michael's R-b's

 pc-bios/descriptors/50-edk2-i386-secure.json   | 34 +++
 pc-bios/descriptors/50-edk2-x86_64-secure.json | 35 
 pc-bios/descriptors/60-edk2-aarch64.json   | 31 +
 pc-bios/descriptors/60-edk2-arm.json   | 31 +
 pc-bios/descriptors/60-edk2-i386.json  | 33 ++
 pc-bios/descriptors/60-edk2-x86_64.json| 34 +++
 pc-bios/README | 11 ++
 7 files changed, 209 insertions(+)

diff --git a/pc-bios/descriptors/50-edk2-i386-secure.json 
b/pc-bios/descriptors/50-edk2-i386-secure.json
new file mode 100644
index ..d7108c1da05a
--- /dev/null
+++ b/pc-bios/descriptors/50-edk2-i386-secure.json
@@ -0,0 +1,34 @@
+{
+"description": "UEFI firmware for i386, with Secure Boot and SMM",
+"interface-types": [
+"uefi"
+],
+"mapping": {
+"device": "flash",
+"executable": {
+"filename": "@DATADIR@/edk2-i386-secure-code.fd",
+"format": "raw"
+},
+"nvram-template": {
+"filename": "@DATADIR@/edk2-i386-vars.fd",
+"format": "raw"
+}
+},
+"targets": [
+{
+"architecture": "i386",
+"machines": [
+"pc-q35-*"
+]
+}
+],
+"features": [
+"acpi-s3",
+"requires-smm",
+"secure-boot",
+"verbose-dynamic"
+],
+"tags": [
+
+]
+}
diff --git a/pc-bios/descriptors/50-edk2-x86_64-secure.json 
b/pc-bios/descriptors/50-edk2-x86_64-secure.json
new file mode 100644
index ..387eb356236b
--- /dev/null
+++ b/pc-bios/descriptors/50-edk2-x86_64-secure.json
@@ -0,0 +1,35 @@
+{
+"description": "UEFI firmware for x86_64, with Secure Boot and SMM",
+"interface-types": [
+"uefi"
+],
+"mapping": {
+"device": "flash",
+"executable": {
+"filename": "@DATADIR@/edk2-x86_64-secure-code.fd",
+"format": "raw"
+},
+"nvram-template": {
+"filename": "@DATADIR@/edk2-i386-vars.fd",
+"format": "raw"
+}
+},
+"targets": [
+{
+"architecture": "x86_64",
+"machines": [
+"pc-q35-*"
+]
+}
+],
+"features": [
+"acpi-s3",
+"amd-sev",
+"requires-smm",
+"secure-boot",
+"verbose-dynamic"
+],
+"tags": [
+
+]
+}
diff --git a/pc-bios/descriptors/60-edk2-aarch64.json 
b/pc-bios/descriptors/60-edk2-aarch64.json
new file mode 100644
index ..800a21bda691
--- /dev/null
+++ b/pc-bios/descriptors/60-edk2-aarch64.json
@@ -0,0 +1,31 @@
+{
+"description": "UEFI firmware for aarch64",
+"interface-types": [
+"uefi"
+],
+"mapping": {
+"device": "flash",
+"executable": {
+"filename": "@DATADIR@/edk2-aarch64-code.fd",
+"format": "raw"
+},
+"nvram-template": {
+"filename": "@DATADIR@/edk2-arm-vars.fd",
+"format": "raw"
+}
+},
+"targets": [
+{
+"architecture": "aarch64",
+"machines": [
+"virt-*"
+]
+}
+],
+"features": [
+"verbose-static"
+],
+"tags": [
+
+]
+}
diff --git a/pc-bios/descriptors/60-edk2-arm.json 
b/pc-bios/descriptors/60-edk2-arm.json
new file mode 100644
index ..d5f1bba6cc82
--- /dev/null
+++ b/pc-bios/descriptors/60-edk2-arm.json
@@ -0,0 +1,31 @@
+{
+"description": "UEFI firmware for arm",
+"interface-types": [
+"uefi"
+],
+"mapping": {
+"device": "flash",
+"executable": {
+"filename": "@DATADIR@/edk2-arm-code.fd",
+"format": "raw"
+},
+"nvram-template": {
+"filename": "@DATADIR@/edk2-arm-vars.fd",
+"format": "raw"
+}
+},
+"targets": [
+{
+"architecture": "arm",
+"machines": [
+"virt-*"
+]
+}
+],
+"features": [
+"verbose-static"
+],
+"tags": [
+
+]
+}
diff --git a/pc-bios/descriptors/60-edk2-i386.json 
b/pc-bios/descriptors/60-edk2-i386.json
new file mode 100644
index

[Qemu-devel] [PATCH for-4.1 v4 02/12] roms/edk2-funcs.sh: require gcc-4.8+ for building i386 and x86_64

Adapt the qemu_edk2_get_toolchain() function in "roms/edk2-funcs.sh" in
advance to edk2 commit 8d7cdfae8cb8 ("OvmfPkg: require GCC48 or later",
2019-01-08), which is part of the "edk2-stable201903" tag.

Signed-off-by: Laszlo Ersek 
Reviewed-by: Philippe Mathieu-Daudé 
Tested-by: Philippe Mathieu-Daudé 
Reviewed-by: Michal Privoznik 
Reviewed-by: Michael S. Tsirkin 
Tested-by: Igor Mammedov 
---

Notes:
v4:

- no change

v3:

- pick up Igor's T-b

v2:

- pick up Phil's R-b / T-b

- pick up Michal's and Michael's R-b's

 roms/edk2-funcs.sh | 14 +-
 1 file changed, 1 insertion(+), 13 deletions(-)

diff --git a/roms/edk2-funcs.sh b/roms/edk2-funcs.sh
index 8930479dcb44..d1cb1e4a111e 100644
--- a/roms/edk2-funcs.sh
+++ b/roms/edk2-funcs.sh
@@ -149,23 +149,11 @@ qemu_edk2_get_toolchain()
   # Run "git-blame" on "OvmfPkg/build.sh" in edk2 for more information on
   # the mapping below.
   case "$gcc_version" in
-([1-3].*|4.[0-3].*)
+([1-3].*|4.[0-7].*)
   printf '%s: unsupported gcc version "%s"\n' \
 "$program_name" "$gcc_version" >&2
   return 1
   ;;
-(4.4.*)
-  printf 'GCC44\n'
-  ;;
-(4.5.*)
-  printf 'GCC45\n'
-  ;;
-(4.6.*)
-  printf 'GCC46\n'
-  ;;
-(4.7.*)
-  printf 'GCC47\n'
-  ;;
 (4.8.*)
   printf 'GCC48\n'
   ;;
-- 
2.19.1.3.g30247aa5d201

[Qemu-devel] [PATCH for-4.1 v4 06/12] roms/Makefile: replace the $(EDK2_EFIROM) target with "edk2-basetools"

We don't (can't) have a recipe for building just $(EDK2_EFIROM);
therefore, while we call the target $(EDK2_EFIROM), we actually build all
of the edk2 BaseTools. Rename the target to edk2-basetools, and update the
iPXE prerequisite accordingly. This will let other targets depend on
"edk2-basetools", where an $(EDK2_EFIROM) pre-requisite would be
misleading.

Signed-off-by: Laszlo Ersek 
---

Notes:
v4:

- rebase: resolve conflicts with

  - commit d912e795e029 ("roms: Rename the EFIROM variable to avoid
clashing with iPXE", 2019-04-09)

  - commit 1cab464136b4 ("roms: Allow passing configure options to the
EDK2 build tools", 2019-04-09)

- refresh commit message accordingly: replace EFIROM references with
  EDK2_EFIROM references

- drop all previous feedback tags, consequently :(

  - Phil's R-b / T-b
  - Michal's R-b
  - Michael's R-b
  - Igor's T-b

v3:

- pick up Igor's T-b

v2:

- pick up Phil's R-b / T-b

- pick up Michal's and Michael's R-b's

 roms/Makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/roms/Makefile b/roms/Makefile
index 1ff78b63bb3a..93c3d467be14 100644
--- a/roms/Makefile
+++ b/roms/Makefile
@@ -102,7 +102,7 @@ pxe-rom-%: build-pxe-roms
 
 efirom: $(patsubst %,efi-rom-%,$(pxerom_variants))
 
-efi-rom-%: build-pxe-roms build-efi-roms $(EDK2_EFIROM)
+efi-rom-%: build-pxe-roms build-efi-roms edk2-basetools
$(EDK2_EFIROM) -f "0x$(VID)" -i "0x$(DID)" -l 0x02 \
-b ipxe/src/bin/$(VID)$(DID).rom \
-ec ipxe/src/bin-i386-efi/$(VID)$(DID).efidrv \
@@ -131,7 +131,7 @@ build-efi-roms: build-pxe-roms
 #EDK2_BASETOOLS_LDFLAGS='...' \
 #efirom
 #
-$(EDK2_EFIROM):
+edk2-basetools:
$(MAKE) -C edk2/BaseTools \
EXTRA_OPTFLAGS='$(EDK2_BASETOOLS_OPTFLAGS)' \
EXTRA_LDFLAGS='$(EDK2_BASETOOLS_LDFLAGS)'
-- 
2.19.1.3.g30247aa5d201

[Qemu-devel] [PATCH for-4.1 v4 00/12] bundle edk2 platform firmware with QEMU

Repo:   https://github.com/lersek/qemu.git
Branch: edk2_build_v4

Posting a v4 is necessary because patch #6 needs
- manual conflict resolution against some commits between v4.0.0-rc2 and
  v4.0.0-rc3,
- and corresponding commit message updates.

These are noted in more detail on the patch itself:

  [PATCH for-4.1 v4 06/12]
  roms/Makefile: replace the $(EDK2_EFIROM) target with "edk2-basetools"

Due to this update, I had to drop all previous feedback tags on this
patch. I plan to send a pullreq as soon as 4.0 is released, so please
re-review / retest the above patch (and *only* the above patch) soon.
(The notes on the patch list the tags individually that I've now
dropped.)

Version 3, that is:

  [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

was posted at:

  https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg06148.html
  http://mid.mail-archive.com/20190321113408.19929-1-lersek@redhat.com

Thanks,
Laszlo

Laszlo Ersek (12):
  roms: lift "edk2-funcs.sh" from "tests/uefi-test-tools/build.sh"
  roms/edk2-funcs.sh: require gcc-4.8+ for building i386 and x86_64
  tests/uefi-test-tools/build.sh: work around TianoCore#1607
  roms/edk2: advance to tag edk2-stable201903
  roms/edk2-funcs.sh: add the qemu_edk2_get_thread_count() function
  roms/Makefile: replace the $(EDK2_EFIROM) target with "edk2-basetools"
  roms: build edk2 firmware binaries and variable store templates
  pc-bios: add edk2 firmware binaries and variable store templates
  pc-bios: document the edk2 firmware images; add firmware descriptors
  tests: add missing dependency to build QTEST_QEMU_BINARY, round 2
  Makefile: install the edk2 firmware images and their descriptors
  MAINTAINERS: add the "EDK2 Firmware" subsystem

 .gitignore |   1 +
 MAINTAINERS|  12 +
 Makefile   |  29 ++-
 configure  |   1 +
 pc-bios/README |  11 +
 pc-bios/descriptors/50-edk2-i386-secure.json   |  34 +++
 pc-bios/descriptors/50-edk2-x86_64-secure.json |  35 +++
 pc-bios/descriptors/60-edk2-aarch64.json   |  31 +++
 pc-bios/descriptors/60-edk2-arm.json   |  31 +++
 pc-bios/descriptors/60-edk2-i386.json  |  33 +++
 pc-bios/descriptors/60-edk2-x86_64.json|  34 +++
 pc-bios/edk2-aarch64-code.fd.bz2   | Bin 0 -> 1177603 bytes
 pc-bios/edk2-arm-code.fd.bz2   | Bin 0 -> 1173662 bytes
 pc-bios/edk2-arm-vars.fd.bz2   | Bin 0 -> 263 bytes
 pc-bios/edk2-i386-code.fd.bz2  | Bin 0 -> 1688659 bytes
 pc-bios/edk2-i386-secure-code.fd.bz2   | Bin 0 -> 1881979 bytes
 pc-bios/edk2-i386-vars.fd.bz2  | Bin 0 -> 190 bytes
 pc-bios/edk2-licenses.txt  | 209 
 pc-bios/edk2-x86_64-code.fd.bz2| Bin 0 -> 1669280 bytes
 pc-bios/edk2-x86_64-secure-code.fd.bz2 | Bin 0 -> 1901210 bytes
 roms/Makefile  |   9 +-
 roms/Makefile.edk2 | 148 
 roms/edk2  |   2 +-
 roms/edk2-build.sh |  55 +
 roms/edk2-funcs.sh | 253 
 tests/Makefile.include |   2 +-
 tests/uefi-test-tools/build.sh | 100 +---
 27 files changed, 934 insertions(+), 96 deletions(-)
 create mode 100644 pc-bios/descriptors/50-edk2-i386-secure.json
 create mode 100644 pc-bios/descriptors/50-edk2-x86_64-secure.json
 create mode 100644 pc-bios/descriptors/60-edk2-aarch64.json
 create mode 100644 pc-bios/descriptors/60-edk2-arm.json
 create mode 100644 pc-bios/descriptors/60-edk2-i386.json
 create mode 100644 pc-bios/descriptors/60-edk2-x86_64.json
 create mode 100644 pc-bios/edk2-aarch64-code.fd.bz2
 create mode 100644 pc-bios/edk2-arm-code.fd.bz2
 create mode 100644 pc-bios/edk2-arm-vars.fd.bz2
 create mode 100644 pc-bios/edk2-i386-code.fd.bz2
 create mode 100644 pc-bios/edk2-i386-secure-code.fd.bz2
 create mode 100644 pc-bios/edk2-i386-vars.fd.bz2
 create mode 100644 pc-bios/edk2-licenses.txt
 create mode 100644 pc-bios/edk2-x86_64-code.fd.bz2
 create mode 100644 pc-bios/edk2-x86_64-secure-code.fd.bz2
 create mode 100644 roms/Makefile.edk2
 create mode 100755 roms/edk2-build.sh
 create mode 100644 roms/edk2-funcs.sh

-- 
2.19.1.3.g30247aa5d201

[Qemu-devel] [PATCH for-4.1 v4 01/12] roms: lift "edk2-funcs.sh" from "tests/uefi-test-tools/build.sh"

Extract the dense logic for architecture and toolchain massaging from
"tests/uefi-test-tools/build.sh", to a set of small functions. We'll reuse
these functions for building full platform firmware images.

Signed-off-by: Laszlo Ersek 
Reviewed-by: Philippe Mathieu-Daudé 
Tested-by: Philippe Mathieu-Daudé 
Reviewed-by: Michal Privoznik 
Reviewed-by: Michael S. Tsirkin 
Tested-by: Igor Mammedov 
---

Notes:
v4:

- no change

v3:

- pick up Igor's T-b

v2:

- Pick up Phil's R-b and T-b. (I hope that's okay, after the discussion
  about cross-building on Ubuntu and/or with Linaro toolchains!)

- drop comma after copyright year, in the new file [Eric]

- clarify in the leading comment of "edk2-funcs.sh" that the file
  requires bash [Phil, Eric]

- pick up Michal's and Michael's R-b's

 roms/edk2-funcs.sh | 240 
 tests/uefi-test-tools/build.sh |  97 +---
 2 files changed, 246 insertions(+), 91 deletions(-)

diff --git a/roms/edk2-funcs.sh b/roms/edk2-funcs.sh
new file mode 100644
index ..8930479dcb44
--- /dev/null
+++ b/roms/edk2-funcs.sh
@@ -0,0 +1,240 @@
+# Shell script that defines functions for determining some environmental
+# characteristics for the edk2 "build" utility.
+#
+# This script is meant to be sourced, in a bash environment.
+#
+# Copyright (C) 2019 Red Hat, Inc.
+#
+# This program and the accompanying materials are licensed and made available
+# under the terms and conditions of the BSD License that accompanies this
+# distribution. The full text of the license may be found at
+# .
+#
+# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, WITHOUT
+# WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
+
+
+# Verify whether the QEMU system emulation target is supported by the UEFI spec
+# and edk2. Print a message to the standard error, and return with nonzero
+# status, if verification fails.
+#
+# Parameters:
+#   $1: QEMU system emulation target
+qemu_edk2_verify_arch()
+{
+  local emulation_target="$1"
+  local program_name=$(basename -- "$0")
+
+  case "$emulation_target" in
+(arm|aarch64|i386|x86_64)
+  ;;
+(*)
+  printf '%s: unknown/unsupported QEMU system emulation target "%s"\n' \
+"$program_name" "$emulation_target" >&2
+  return 1
+  ;;
+  esac
+}
+
+
+# Translate the QEMU system emulation target to the edk2 architecture
+# identifier. Print the result to the standard output.
+#
+# Parameters:
+#   $1: QEMU system emulation target
+qemu_edk2_get_arch()
+{
+  local emulation_target="$1"
+
+  if ! qemu_edk2_verify_arch "$emulation_target"; then
+return 1
+  fi
+
+  case "$emulation_target" in
+(arm)
+  printf 'ARM\n'
+  ;;
+(aarch64)
+  printf 'AARCH64\n'
+  ;;
+(i386)
+  printf 'IA32\n'
+  ;;
+(x86_64)
+  printf 'X64\n'
+  ;;
+  esac
+}
+
+
+# Translate the QEMU system emulation target to the gcc cross-compilation
+# architecture identifier. Print the result to the standard output.
+#
+# Parameters:
+#   $1: QEMU system emulation target
+qemu_edk2_get_gcc_arch()
+{
+  local emulation_target="$1"
+
+  if ! qemu_edk2_verify_arch "$emulation_target"; then
+return 1
+  fi
+
+  case "$emulation_target" in
+(arm|aarch64|x86_64)
+  printf '%s\n' "$emulation_target"
+  ;;
+(i386)
+  printf 'i686\n'
+  ;;
+  esac
+}
+
+
+# Determine the gcc cross-compiler prefix (if any) for use with the edk2
+# toolchain. Print the result to the standard output.
+#
+# Parameters:
+#   $1: QEMU system emulation target
+qemu_edk2_get_cross_prefix()
+{
+  local emulation_target="$1"
+  local gcc_arch
+  local host_arch
+
+  if ! gcc_arch=$(qemu_edk2_get_gcc_arch "$emulation_target"); then
+return 1
+  fi
+
+  host_arch=$(uname -m)
+
+  if [ "$gcc_arch" == "$host_arch" ] ||
+ ( [ "$gcc_arch" == i686 ] && [ "$host_arch" == x86_64 ] ); then
+# no cross-compiler needed
+:
+  else
+printf '%s-linux-gnu-\n' "$gcc_arch"
+  fi
+}
+
+
+# Determine the edk2 toolchain tag for the QEMU system emulation target. Print
+# the result to the standard output. Print a message to the standard error, and
+# return with nonzero status, if the (conditional) gcc version check fails.
+#
+# Parameters:
+#   $1: QEMU system emulation target
+qemu_edk2_get_toolchain()
+{
+  local emulation_target="$1"
+  local program_name=$(basename -- "$0")
+  local cross_prefix
+  local gcc_version
+
+  if ! qemu_edk2_verify_arch "$emulation_target"; then
+return 1
+  fi
+
+  case "$emulation_target" in
+(arm|aarch64)
+  printf 'GCC5\n'
+  ;;
+
+(i386|x86_64)
+  if ! cross_prefix=$(qemu_edk2_get_cross_prefix "$emulation_target"); then
+return 1
+  fi
+
+  gcc_version=$("${cross_prefix}gcc" -v 2>&1 | tail -1 | awk '{print $3}')
+  # Run "git-blame" on "OvmfPkg/build.sh" in edk2

[Qemu-devel] [PATCH for-4.1 v4 05/12] roms/edk2-funcs.sh: add the qemu_edk2_get_thread_count() function

The edk2 "build" utility natively supports building modules (that is, INF
files) in parallel. The feature is not useful when building a single
module (with the "-m" option), but it is useful for platform firmware
builds (which include many modules). Add a function that determines the
"-n" option argument for "build", from the MAKEFLAGS variable (i.e. based
on the presence of a make job server).

Signed-off-by: Laszlo Ersek 
Reviewed-by: Philippe Mathieu-Daudé 
Tested-by: Philippe Mathieu-Daudé 
Reviewed-by: Michal Privoznik 
Reviewed-by: Michael S. Tsirkin 
Tested-by: Igor Mammedov 
---

Notes:
v4:

- no change

v3:

- pick up Igor's T-b

v2:

- pick up Phil's R-b / T-b

- pick up Michal's and Michael's R-b's

 roms/edk2-funcs.sh | 25 
 1 file changed, 25 insertions(+)

diff --git a/roms/edk2-funcs.sh b/roms/edk2-funcs.sh
index d1cb1e4a111e..a9fae7ee891b 100644
--- a/roms/edk2-funcs.sh
+++ b/roms/edk2-funcs.sh
@@ -226,3 +226,28 @@ qemu_edk2_set_cross_env()
 
   eval "export $cross_prefix_var=\$cross_prefix"
 }
+
+
+# Determine the "-n" option argument (that is, the number of modules to build
+# in parallel) for the edk2 "build" utility. Print the result to the standard
+# output.
+#
+# Parameters:
+#   $1: the value of the MAKEFLAGS variable
+qemu_edk2_get_thread_count()
+{
+  local makeflags="$1"
+
+  if [[ "$makeflags" == *--jobserver-auth=* ]] ||
+ [[ "$makeflags" == *--jobserver-fds=* ]]; then
+# If there is a job server, allow the edk2 "build" utility to parallelize
+# as many module builds as there are logical CPUs in the system. The "make"
+# instances forked by "build" are supposed to limit themselves through the
+# job server. The zero value below causes the edk2 "build" utility to fetch
+# the logical CPU count with Python's multiprocessing.cpu_count() method.
+printf '0\n'
+  else
+# Build a single module at a time.
+printf '1\n'
+  fi
+}
-- 
2.19.1.3.g30247aa5d201

[Qemu-devel] [PATCH] qemu-img: fix .hx and .texi disparity

It turns out that having options listed in three places continues to be
a bad idea. I'm still toying with the idea of an improved infrastructure
here, but in the meantime, another bandaid.

There are three locations:
(1) .hx file, formatted as texi
(2) .hx file, formatted as human readable.
(3) .texi file, as section headers, formatted as texi.

You can compare the two summaries within the .hx file like so:

Human-readable command summaries:
`./qemu-img --help | grep 'Command syntax' -A14`
Detokenized texi command summaries:
`grep "@item" qemu-img-cmds.hx | sed -E 's|@var\{([^\}]*?)\}|\1|g'`

You can compare the two separate texi summaries like so:

Texi command summaries:
`grep "@item" qemu-img-cmds.hx"`
Texi command headers:
grep -E "@item.*@var" qemu-img.texi | tail -14

Signed-off-by: John Snow 
---
 qemu-img.texi| 4 ++--
 qemu-img-cmds.hx | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/qemu-img.texi b/qemu-img.texi
index 3b6710a580..724f244ba1 100644
--- a/qemu-img.texi
+++ b/qemu-img.texi
@@ -197,7 +197,7 @@ Command description:
 
 @table @option
 
-@item amend [--object @var{objectdef}] [--image-opts] [-p] [-p] [-f @var{fmt}] 
[-t @var{cache}] -o @var{options} @var{filename}
+@item amend [--object @var{objectdef}] [--image-opts] [-p] [-q] [-f @var{fmt}] 
[-t @var{cache}] -o @var{options} @var{filename}
 
 Amends the image format specific @var{options} for the image file
 @var{filename}. Not all file formats support this operation.
@@ -424,7 +424,7 @@ To enumerate information about each disk image in the above 
chain, starting from
 qemu-img info --backing-chain snap2.qcow2
 @end example
 
-@item map [-f @var{fmt}] [--output=@var{ofmt}] @var{filename}
+@item map [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] 
[--output=@var{ofmt}] [-U] @var{filename}
 
 Dump the metadata of image @var{filename} and its backing file chain.
 In particular, this commands dumps the allocation state of every sector
diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index 1526f327a5..4b47f7495d 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -46,7 +46,7 @@ ETEXI
 DEF("convert", img_convert,
 "convert [--object objectdef] [--image-opts] [--target-image-opts] [-U] 
[-C] [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-B 
backing_file] [-o options] [-l snapshot_param] [-S sparse_size] [-m 
num_coroutines] [-W] filename [filename2 [...]] output_filename")
 STEXI
-@item convert [--object @var{objectdef}] [--image-opts] [--target-image-opts] 
[-U] [-c] [-p] [-q] [-n] [-f @var{fmt}] [-t @var{cache}] [-T @var{src_cache}] 
[-O @var{output_fmt}] [-B @var{backing_file}] [-o @var{options}] [-l 
@var{snapshot_param}] [-S @var{sparse_size}] [-m @var{num_coroutines}] [-W] 
@var{filename} [@var{filename2} [...]] @var{output_filename}
+@item convert [--object @var{objectdef}] [--image-opts] [--target-image-opts] 
[-U] [-C] [-c] [-p] [-q] [-n] [-f @var{fmt}] [-t @var{cache}] [-T 
@var{src_cache}] [-O @var{output_fmt}] [-B @var{backing_file}] [-o 
@var{options}] [-l @var{snapshot_param}] [-S @var{sparse_size}] [-m 
@var{num_coroutines}] [-W] @var{filename} [@var{filename2} [...]] 
@var{output_filename}
 ETEXI
 
 DEF("create", img_create,
-- 
2.17.2

Re: [Qemu-devel] [PATCH] configure: Automatically fall back to TCI on non-release architectures

On 4/9/19 9:46 AM, Stefan Weil wrote:
> * Calling conventions. The current implementation works on many hosts,
> but for example not on Sparc. A fix would require simple calling
> conventions for all helper functions (for example stack based argument
> passing, can this be enforced?), or it needs to know the signature of
> each helper function at runtime. I'm afraid that fixing this would
> require much work. A runtime test whether calls of helper functions work
> correctly could be implemented easily and could abort the program
> execution when calls fail to pass the right arguments. Would such a
> runtime test help a little bit?

In the rewrite of tci that I proposed some years ago,
I used libffi for this.

Those patches could probably be recovered...

https://patchwork.ozlabs.org/patch/348528/
https://patchwork.ozlabs.org/patch/348527/


r~

[Qemu-devel] [Bug 1815252] Re: virtio-9p-pci passthrough fsync hang

2019-04-09 Thread john

Damn, I just ran into this bug again! Then realized I'd already reported
it :(

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1815252

Title:
  virtio-9p-pci passthrough fsync hang

Status in QEMU:
  New

Bug description:
  Tested against QEMU: e47f81b617684c4546af286d307b69014a83538a

  and $qemu-system-x86_64 --version
  QEMU emulator version 2.11.1(Debian 1:2.11+dfsg-1ubuntu7.9)
  Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers

  + exec sudo qemu-system-x86_64 -enable-kvm -bios /usr/share/ovmf/OVMF.fd -cpu 
host -smp sockets=1,cpus=4,cores=2 -m 2048 -vga none -nographic -chardev 
socket,id=chrtpm0,path=/tmp/clrwifi/swtpm2/tpm-sock -tpmdev 
emulator,id=tpm0,chardev=chrtpm0 -device tpm-tis,tpmdev=tpm0 -device 
virtio-rng-pci -netdev user,id=mynet0,hostfwd=tcp::10022-:22 -device 
virtio-net-pci,netdev=mynet0 -kernel /home/.../Documents/c/iwd/tools/bzImage 
-append 'console=ttyS0,115200n8 quiet kvm-intel.nested=1 init=/usr/bin/bash 
initcall_debug tsc=reliable no_timer_check noreplace-smp cryptomgr.notests 
rootfstype=9p root=/dev/root rootflags=trans=virtio,version=9p2000.u rw' -fsdev 
local,id=fsdev-root,path=mnt,security_model=passthrough -device 
virtio-9p-pci,fsdev=fsdev-root,mount_tag=/dev/root
  [0.00] Linux version 4.19.0-rc2 (...) (gcc version 7.3.0 (Ubuntu 
7.3.0-27ubuntu1~18.04)) #10 SMP Fri Feb 8 13:55:20 PST 2019
  [0.00] Command line: console=ttyS0,115200n8 quiet kvm-intel.nested=1 
init=/usr/bin/bash initcall_debug tsc=reliable no_timer_check noreplace-smp 
cryptomgr.notests rootfstype=9p root=/dev/root 
rootflags=trans=virtio,version=9p2000.u rw
  [0.00] KERNEL supported cpus:
  [0.00]   Intel GenuineIntel
  [0.00] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point 
registers'
  [0.00] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
  [0.00] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
  [0.00] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
  [0.00] x86/fpu: Enabled xstate features 0x7, context size is 832 
bytes, using 'standard' format.
  [0.00] BIOS-provided physical RAM map:
  [0.00] BIOS-e820: [mem 0x-0x0009] usable
  [0.00] BIOS-e820: [mem 0x0010-0x007f] usable
  [0.00] BIOS-e820: [mem 0x0080-0x00807fff] ACPI NVS
  [0.00] BIOS-e820: [mem 0x00808000-0x0080] usable
  [0.00] BIOS-e820: [mem 0x0081-0x008f] ACPI NVS
  [0.00] BIOS-e820: [mem 0x0090-0x7e87efff] usable
  [0.00] BIOS-e820: [mem 0x7e87f000-0x7e888fff] ACPI NVS
  [0.00] BIOS-e820: [mem 0x7e889000-0x7e889fff] ACPI 
data
  [0.00] BIOS-e820: [mem 0x7e88a000-0x7e88bfff] usable
  [0.00] BIOS-e820: [mem 0x7e88c000-0x7e88] ACPI NVS
  [0.00] BIOS-e820: [mem 0x7e89-0x7e8a9fff] reserved
  [0.00] BIOS-e820: [mem 0x7e8aa000-0x7e8b9fff] ACPI NVS
  [0.00] BIOS-e820: [mem 0x7e8ba000-0x7e8bafff] reserved
  [0.00] BIOS-e820: [mem 0x7e8bb000-0x7e9e5fff] usable
  [0.00] BIOS-e820: [mem 0x7e9e6000-0x7e9edfff] ACPI NVS
  [0.00] BIOS-e820: [mem 0x7e9ee000-0x7eb1afff] reserved
  [0.00] BIOS-e820: [mem 0x7eb1b000-0x7fb9afff] usable
  [0.00] BIOS-e820: [mem 0x7fb9b000-0x7fbf2fff] reserved
  [0.00] BIOS-e820: [mem 0x7fbf3000-0x7fbfafff] ACPI 
data
  [0.00] BIOS-e820: [mem 0x7fbfb000-0x7fbfefff] ACPI NVS
  [0.00] BIOS-e820: [mem 0x7fbff000-0x7ff3] usable
  [0.00] BIOS-e820: [mem 0x7ff4-0x7ff5] reserved
  [0.00] BIOS-e820: [mem 0x7ff6-0x7fff] ACPI NVS
  [0.352469] acpiphp_ibm: ibm_acpiphp_init: acpi_walk_namespace failed
  Started bpfilter
  bash: cannot set terminal process group (-1): Inappropriate ioctl for device
  bash: no job control in this shell
  grep: /proc/cpuinfo: No such file or directory
  grep: /proc/cpuinfo: No such file or directory
  root@clr / # passwd
  ...
  openat(AT_FDCWD, "/etc/nshadow", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 4
  umask(077)  = 077
  openat(AT_FDCWD, "/etc/shadow", O_RDONLY|O_CREAT|O_CLOEXEC, 000) = 5
  fcntl(5, F_GETFL)   = 0x8000 (flags O_RDONLY|O_LARGEFILE)
  fstat(5, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
  fchown(4, 0, 0) = 0
  fchmod(4, 0100644)  = 0
  lseek(5, 0, SEEK_CUR)   = 0
  fstat(5, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
  read(5, "",

Re: [Qemu-devel] [PATCH for-4.0-maybe] device_tree: Fix integer overflowing in load_device_tree()

On Tue, 9 Apr 2019 at 21:15, Alistair Francis  wrote:
>
> On Tue, Apr 9, 2019 at 1:08 PM Peter Maydell  wrote:
> >
> > On Wed, 10 Apr 2019 at 00:40, Markus Armbruster  wrote:
> > >
> > > If the value of get_image_size() exceeds INT_MAX / 2 - 1, the
> > > computation of @dt_size overflows to a negative number, which then
> > > gets converted to a very large size_t for g_malloc0() and
> > > load_image_size().  In the (fortunately improbable) case g_malloc0()
> > > succeeds and load_image_size() survives, we'd assign the negative
> > > number to *sizep.  What that would do to the callers I can't say, but
> > > it's unlikely to be good.
> > >
> > > Fix by rejecting images whose size would overflow.
> > >
> > > Signed-off-by: Markus Armbruster 
> >
> > I think this patch is missing some attributions for the
> > security researchers who found the issue initially.
> > PJP's patch for this from a couple of weeks back has a
> > reported-by credit:
> > https://patchew.org/QEMU/20190322073555.20889-1-ppan...@redhat.com/
>
> It seems like from that discussion that this patch is the correct approach.
>
> I can add the attributions and send a PR for 4.0. I'll send it by EOD
> unless anyone has any objections.

Thanks. I think given it's 21:30 here I'm going to postpone
tagging rc3 til tomorrow (mid-afternoon UK time). I'm still
hoping we can avoid an rc4...

thanks
-- PMM

Re: [Qemu-devel] [Bug 1814352] Re: SIOCGIFNAME takes a struct ifreq not an integer

On Wed, 10 Apr 2019 at 01:26, Erik Kline <1814...@bugs.launchpad.net> wrote:
> Sure.  Looking at HEAD, and even the surrounding the lines, I think I
> should have tried STRUCT_short_ifreq instead of STRUCT_int_ifreq, though
> I'm not sure what the real difference would be.

The multiple STRUCT_*_ifreq are working around the fact that
our MK_STRUCT infrastructure can't handle unions. The struct
ifreq is a char array followed by a union whose members are
various different types. You should use the STRUCT_*_ifreq
corresponding to whatever type the union field used by this
particular ioctl is. For SIOCGIFNAME the ifr_ifindex is read,
and that's an int, so you want STRUCT_int_ifreq. (If you used
the 'short' version by mistake this would probably break for the
case of big-endian guest and little-endian host or vice-versa
because we'd swap the wrong amount of data.)

thanks
-- PMM

Re: [Qemu-devel] [PATCH for-4.0-maybe] device_tree: Fix integer overflowing in load_device_tree()

On Tue, Apr 9, 2019 at 1:08 PM Peter Maydell  wrote:
>
> On Wed, 10 Apr 2019 at 00:40, Markus Armbruster  wrote:
> >
> > If the value of get_image_size() exceeds INT_MAX / 2 - 1, the
> > computation of @dt_size overflows to a negative number, which then
> > gets converted to a very large size_t for g_malloc0() and
> > load_image_size().  In the (fortunately improbable) case g_malloc0()
> > succeeds and load_image_size() survives, we'd assign the negative
> > number to *sizep.  What that would do to the callers I can't say, but
> > it's unlikely to be good.
> >
> > Fix by rejecting images whose size would overflow.
> >
> > Signed-off-by: Markus Armbruster 
>
> I think this patch is missing some attributions for the
> security researchers who found the issue initially.
> PJP's patch for this from a couple of weeks back has a
> reported-by credit:
> https://patchew.org/QEMU/20190322073555.20889-1-ppan...@redhat.com/

It seems like from that discussion that this patch is the correct approach.

I can add the attributions and send a PR for 4.0. I'll send it by EOD
unless anyone has any objections.

Alistair

>
> thanks
> -- PMM
>

Re: [Qemu-devel] [PATCH for-4.0-maybe] device_tree: Fix integer overflowing in load_device_tree()

On Wed, 10 Apr 2019 at 00:40, Markus Armbruster  wrote:
>
> If the value of get_image_size() exceeds INT_MAX / 2 - 1, the
> computation of @dt_size overflows to a negative number, which then
> gets converted to a very large size_t for g_malloc0() and
> load_image_size().  In the (fortunately improbable) case g_malloc0()
> succeeds and load_image_size() survives, we'd assign the negative
> number to *sizep.  What that would do to the callers I can't say, but
> it's unlikely to be good.
>
> Fix by rejecting images whose size would overflow.
>
> Signed-off-by: Markus Armbruster 

I think this patch is missing some attributions for the
security researchers who found the issue initially.
PJP's patch for this from a couple of weeks back has a
reported-by credit:
https://patchew.org/QEMU/20190322073555.20889-1-ppan...@redhat.com/

thanks
-- PMM

Re: [Qemu-devel] Misaligned memory writes on ARM and other runtime errors

On Wed, 10 Apr 2019 at 02:12, Stefan Weil  wrote:
>
> When QEMU for ARM is compiled with sanitizers
> (--extra-cflags=-fsanitize=address,undefined), I see runtime errors for
> qemu-arm which indicate misaligned memory writes:
>
> /qemu/linux-user/fd-trans.c:1389:18: runtime error: store to misaligned
> address 0x42ff4b34 for type 'uint64_t', which requires 8 byte alignment
>
> The full sanitizer report is available from
> https://qemu.weilnetz.de/doc/sanitizer.txt. Some of the reported runtime
> errors can easily be fixed, for example by replacing "1 << 31"
> (currently 188 matches) by "1U << 31" (currently only 70 matches).

The shift "errors" are not in fact errors -- we compile with -fwrapv
to request a dialect of C where left shift of signed integers is
well-defined and has 2s complement semantics. Some versions
of the clang sanitizers don't correctly handle that flag as meaning
that those shifts should no longer generate sanitizer warnings.
If you're using a clang without that bug fix you can suppress the
warnings by adding "-fno-sanitize=shift-base" to the --extra-cflags
you pass to QEMU's configure.

The misaligned accesses are definitely bugs. The linux-user code
unfortunately has quite a lot of code in its data-structure conversion
code that is pretty fast and loose about accessing fields in
target-layout data structures.

thanks
-- PMM

Re: [Qemu-devel] [PATCH] configure: Automatically fall back to TCI on non-release architectures

2019-04-09 Thread Stefan Weil

On 05.04.19 11:16, Philippe Mathieu-Daudé wrote:
> On 4/5/19 11:02 AM, Daniel P. Berrangé wrote:
>> On Fri, Apr 05, 2019 at 10:47:54AM +0200, Philippe Mathieu-Daudé wrote:
>> Do the various crashes that you illustrate in that cover letter
>> still exist today ? If so, 2 years of continued brokenness with no
>> fixes would reinforce the the view that it is time to remove TCI
>> from the codebase.
> 
> Or find a maintainer and add tests...

Thank you for CC'ing me. I could not spend much of my free time for QEMU
last year and typically will miss important messages on the list unless
my address is explicitly given. Nevertheless I still feel responsible
for TCI, and I am also listed as maintainer in MAINTAINERS.

In the past there was only limited use of TCI (as far as I know), and
the current implementation worked for many applications, for example to
count the frequency of TCG opcodes.

The known problems with the current implementation are

* Misaligned memory accesses. This should be easy to fix. I already sent
a patch (currently discussed) which fixed user mode emulation on ARM for me.

* Calling conventions. The current implementation works on many hosts,
but for example not on Sparc. A fix would require simple calling
conventions for all helper functions (for example stack based argument
passing, can this be enforced?), or it needs to know the signature of
each helper function at runtime. I'm afraid that fixing this would
require much work. A runtime test whether calls of helper functions work
correctly could be implemented easily and could abort the program
execution when calls fail to pass the right arguments. Would such a
runtime test help a little bit?

* Performance. It can be improved a bit by implementing more TCG opcodes
for the interpreter, but of course TCI is slower than TCG (which is
slower than KVM).

Cheers
Stefan

[Qemu-devel] [PATCH] linux-user: Support the SIOCSIFPFLAGS and SIOCGIFPFLAGS ioctls

2019-04-09 Thread Aleksandar Markovic

From: Neng Chen 

Add support for setting and getting extended (private) flags
of a network device via SIOCSIFPFLAGS and SIOCGIFPFLAGS ioctls.

The ioctl numeric values are platform-independent and determined
by the file include/uapi/linux/sockios.h in Linux kernel source
code.

The ioctls set and get field ifr_flags of type short in the
structure ifreq. Such functionality in QEMU is achieved using
MK_STRUCT() and MK_PTR() macros with an appropriate argument.

Signed-off-by: Neng Chen 
Signed-off-by: Aleksandar Markovic 
---
 linux-user/ioctls.h   | 2 ++
 linux-user/syscall_defs.h | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/linux-user/ioctls.h b/linux-user/ioctls.h
index ae89516..11829a1 100644
--- a/linux-user/ioctls.h
+++ b/linux-user/ioctls.h
@@ -206,6 +206,8 @@
   IOCTL(SIOCADDMULTI, IOC_W, MK_PTR(MK_STRUCT(STRUCT_sockaddr_ifreq)))
   IOCTL(SIOCDELMULTI, IOC_W, MK_PTR(MK_STRUCT(STRUCT_sockaddr_ifreq)))
   IOCTL(SIOCGIFINDEX, IOC_W | IOC_R, MK_PTR(MK_STRUCT(STRUCT_int_ifreq)))
+  IOCTL(SIOCSIFPFLAGS, IOC_W, MK_PTR(MK_STRUCT(STRUCT_short_ifreq)))
+  IOCTL(SIOCGIFPFLAGS, IOC_W | IOC_R, MK_PTR(MK_STRUCT(STRUCT_short_ifreq)))
   IOCTL(SIOCSIFLINK, 0, TYPE_NULL)
   IOCTL_SPECIAL(SIOCGIFCONF, IOC_W | IOC_R, do_ioctl_ifconf,
 MK_PTR(MK_STRUCT(STRUCT_ifconf)))
diff --git a/linux-user/syscall_defs.h b/linux-user/syscall_defs.h
index 12c8407..060b854 100644
--- a/linux-user/syscall_defs.h
+++ b/linux-user/syscall_defs.h
@@ -777,6 +777,8 @@ struct target_pollfd {
 #define TARGET_SIOCADDMULTI0x8931  /* Multicast address lists  
*/
 #define TARGET_SIOCDELMULTI0x8932
 #define TARGET_SIOCGIFINDEX0x8933
+#define TARGET_SIOCSIFPFLAGS   0x8934  /* set extended flags  
*/
+#define TARGET_SIOCGIFPFLAGS   0x8935  /* get extended flags  
*/
 
 /* Bridging control calls */
 #define TARGET_SIOCGIFBR   0x8940  /* Bridging support 
*/
-- 
2.7.4

Re: [Qemu-devel] [PATCH for-4.0] migration/ram.c: Fix use-after-free in multifd_recv_unfill_packet()

On Tue, 9 Apr 2019 at 22:42, Juan Quintela  wrote:
>
> Peter Maydell  wrote:
> > Coverity points out (CID 1400442) that in this code:
> >
> > if (packet->pages_alloc > p->pages->allocated) {
> > multifd_pages_clear(p->pages);
> > multifd_pages_init(packet->pages_alloc);
> > }
> >
> > we free p->pages in multifd_pages_clear() but continue to
> > use it in the following code. We also leak memory, because
> > multifd_pages_init() returns the pointer to a new MultiFDPages_t
> > struct but we are ignoring its return value.
> >
> > Fix both of these bugs by adding the missing assignment of
> > the newly created struct to p->pages.
> >
> > Signed-off-by: Peter Maydell 
> > ---
>
> ouch,
>
> good catch.
>
> Reviewed-by: Juan Quintela 

Thanks for the quick review. Applied to master for rc3.

-- PMM

[Qemu-devel] Misaligned memory writes on ARM and other runtime errors

2019-04-09 Thread Stefan Weil

When QEMU for ARM is compiled with sanitizers
(--extra-cflags=-fsanitize=address,undefined), I see runtime errors for
qemu-arm which indicate misaligned memory writes:

/qemu/linux-user/fd-trans.c:1389:18: runtime error: store to misaligned
address 0x42ff4b34 for type 'uint64_t', which requires 8 byte alignment

The full sanitizer report is available from
https://qemu.weilnetz.de/doc/sanitizer.txt. Some of the reported runtime
errors can easily be fixed, for example by replacing "1 << 31"
(currently 188 matches) by "1U << 31" (currently only 70 matches).

The test results are from running "arm-linux-user/qemu-arm
/usr/bin/scite" on Ubuntu ARM, but running other binaries with qemu-arm
shows similar runtime errors.

Regards,
Stefan Weil

Re: [Qemu-devel] [PATCH v4 00/20] monitor: add asynchronous command type

2019-04-09 Thread no-reply

Patchew URL: 
https://patchew.org/QEMU/20190409161009.6322-1-marcandre.lur...@redhat.com/



Hi,

This series failed the docker-mingw@fedora build test. Please find the testing 
commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
time make docker-test-mingw@fedora SHOW_ENV=1 J=14 NETWORK=1
=== TEST SCRIPT END ===




The full log is available at
http://patchew.org/logs/20190409161009.6322-1-marcandre.lur...@redhat.com/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

Re: [Qemu-devel] [PATCH 0/9] tcg: Add tcg_gen_extract2_{i32,i64}

2019-04-09 Thread David Hildenbrand

On 09.04.19 21:02, Richard Henderson wrote:
> On 4/9/19 8:53 AM, David Hildenbrand wrote:
>> On 07.03.19 15:41, Richard Henderson wrote:
>>> The primary motivator here is usage within s390x,
>>> but (as with any good primitive) the opcode has
>>> applications outside that.
>>
>> I am planning to send the next big part of vector instruction support
>> (Vector Integer Instructions) soon, that would make use of these patches.
> 
> I'll be sure to get them into the first pull after 4.1 opens.
> 

Nice, I'll be posting the next series, referencing this series (might
take some days as the amount of instructions is getting out of hand).
Thanks!


-- 

Thanks,

David / dhildenb

Re: [Qemu-devel] [PATCH v4 00/20] monitor: add asynchronous command type

2019-04-09 Thread no-reply

Patchew URL: 
https://patchew.org/QEMU/20190409161009.6322-1-marcandre.lur...@redhat.com/



Hi,

This series failed the asan build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1
=== TEST SCRIPT END ===




The full log is available at
http://patchew.org/logs/20190409161009.6322-1-marcandre.lur...@redhat.com/testing.asan/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

Re: [Qemu-devel] [PATCH 0/9] tcg: Add tcg_gen_extract2_{i32,i64}

On 4/9/19 8:53 AM, David Hildenbrand wrote:
> On 07.03.19 15:41, Richard Henderson wrote:
>> The primary motivator here is usage within s390x,
>> but (as with any good primitive) the opcode has
>> applications outside that.
> 
> I am planning to send the next big part of vector instruction support
> (Vector Integer Instructions) soon, that would make use of these patches.

I'll be sure to get them into the first pull after 4.1 opens.


r~

Re: [Qemu-devel] [PATCH 0/9] tcg: Add tcg_gen_extract2_{i32,i64}

2019-04-09 Thread David Hildenbrand

On 07.03.19 15:41, Richard Henderson wrote:
> The primary motivator here is usage within s390x,
> but (as with any good primitive) the opcode has
> applications outside that.

I am planning to send the next big part of vector instruction support
(Vector Integer Instructions) soon, that would make use of these patches.

-- 

Thanks,

David / dhildenb

Re: [Qemu-devel] [PATCH for-4.0-maybe] device_tree: Fix integer overflowing in load_device_tree()

On 4/9/19 7:40 PM, Markus Armbruster wrote:
> If the value of get_image_size() exceeds INT_MAX / 2 - 1, the
> computation of @dt_size overflows to a negative number, which then
> gets converted to a very large size_t for g_malloc0() and
> load_image_size().  In the (fortunately improbable) case g_malloc0()
> succeeds and load_image_size() survives, we'd assign the negative
> number to *sizep.  What that would do to the callers I can't say, but
> it's unlikely to be good.
> 
> Fix by rejecting images whose size would overflow.
> 
> Signed-off-by: Markus Armbruster 
> ---
>  device_tree.c | 4 
>  1 file changed, 4 insertions(+)
> 
> diff --git a/device_tree.c b/device_tree.c
> index 296278e12a..f8b46b3c73 100644
> --- a/device_tree.c
> +++ b/device_tree.c
> @@ -84,6 +84,10 @@ void *load_device_tree(const char *filename_path, int 
> *sizep)
>   filename_path);
>  goto fail;
>  }
> +if (dt_size > INT_MAX / 2 - 1) {

We should avoid magic number duplication.
That said, this patch looks safe.

Reviewed-by: Philippe Mathieu-Daudé 

BTW how did you figure that out?

> +error_report("Device tree file '%s' is too large", filename_path);
> +goto fail;
> +}
>  
>  /* Expand to 2x size to give enough room for manipulation.  */
>  dt_size += 1;
>

Re: [Qemu-devel] [PATCH] docs: replace min-glib with fedora

On 4/9/19 5:44 PM, Marc-André Lureau wrote:
> min-glib.docker was removed in commit
> e7b3af81597db1a6b55f2c15d030d703c6b2c6ac ("glib: bump min required
> glib library version to 2.40").
> 
> Cc: Daniel P. Berrangé 
> Signed-off-by: Marc-André Lureau 

Reviewed-by: Philippe Mathieu-Daudé 

> ---
>  docs/devel/testing.rst | 8 
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
> index 60f897d915..cec7d56c57 100644
> --- a/docs/devel/testing.rst
> +++ b/docs/devel/testing.rst
> @@ -317,17 +317,17 @@ the container, with parameters defined by the make 
> target):
>  
>  .. code::
>  
> -  make docker-test-build@min-glib
> +  make docker-test-build@fedora
>  
> -This will create a container instance using the ``min-glib`` image (the image
> +This will create a container instance using the ``fedora`` image (the image
>  is downloaded and initialized automatically), in which the ``test-build`` job
>  is executed.
>  
>  Images
>  --
>  
> -Along with many other images, the ``min-glib`` image is defined in a 
> Dockerfile
> -in ``tests/docker/dockefiles/``, called ``min-glib.docker``. ``make docker``
> +Along with many other images, the ``fedora`` image is defined in a Dockerfile
> +in ``tests/docker/dockefiles/``, called ``fedora.docker``. ``make docker``
>  command will list all the available images.
>  
>  To add a new image, simply create a new ``.docker`` file under the
>

Re: [Qemu-devel] [PATCH for-4.0] migration/ram.c: Fix use-after-free in multifd_recv_unfill_packet()

On 4/9/19 5:18 PM, Peter Maydell wrote:
> Coverity points out (CID 1400442) that in this code:
> 
> if (packet->pages_alloc > p->pages->allocated) {
> multifd_pages_clear(p->pages);
> multifd_pages_init(packet->pages_alloc);
> }
> 
> we free p->pages in multifd_pages_clear() but continue to
> use it in the following code. We also leak memory, because
> multifd_pages_init() returns the pointer to a new MultiFDPages_t
> struct but we are ignoring its return value.
> 
> Fix both of these bugs by adding the missing assignment of
> the newly created struct to p->pages.
> 
> Signed-off-by: Peter Maydell 

Reviewed-by: Philippe Mathieu-Daudé 

> ---
> I don't know anything about the multifd code, but this seems like
> the obvious fix based on looking at what the clear and init
> functions are doing. I have only run 'make check' on this,
> so review and testing definitely in order. I think we should
> really put this into 4.0, which means ideally I'd like to
> commit it to master today or tomorrow, though...
> ---
>  migration/ram.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/migration/ram.c b/migration/ram.c
> index f68bffc..1ca9ba77b6a 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -851,7 +851,7 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams 
> *p, Error **errp)
>   */
>  if (packet->pages_alloc > p->pages->allocated) {
>  multifd_pages_clear(p->pages);
> -multifd_pages_init(packet->pages_alloc);
> +p->pages = multifd_pages_init(packet->pages_alloc);
>  }
>  
>  p->pages->used = be32_to_cpu(packet->pages_used);
>

Re: [Qemu-devel] [PULL v3 0/5] Misc patches for QEMU 5.0-rc3

On 4/9/19 7:25 PM, Peter Maydell wrote:
> On Tue, 9 Apr 2019 at 17:36, Paolo Bonzini  wrote:
>>
>> The following changes since commit 248987f92cfda8305d6d44ced23e173e62a8bc0e:
>>
>>   Merge remote-tracking branch 
>> 'remotes/vivier2/tags/linux-user-for-4.0-pull-request' into staging 
>> (2019-04-09 10:02:30 +0100)
>>
>> are available in the git repository at:
>>
>>
>>   git://github.com/bonzini/qemu.git tags/for-upstream
>>
>> for you to fetch changes up to 3e20c81ed87cb15cd04f929b075e244e0758641a:
>>
>>   tests: Make check-block a phony target (2019-04-09 18:34:21 +0200)
>>
>> 
>> * fixes for Alpine and SuSE
>> * fix crash when hot-plugging nvdimm on older machine types
>>
> 
> Applied, thanks.
> 
> Please update the changelog at https://wiki.qemu.org/ChangeLog/4.0
> for any user-visible changes.

The subject says 5.0 <:O) Hopefully Peter applied it for 4.0 :)

Thanks both for this PR,

Phil.

Re: [Qemu-devel] [PATCH 15/15] monitor: Simplify how -device/device_add print help

2019-04-09 Thread Dr. David Alan Gilbert

* Markus Armbruster (arm...@redhat.com) wrote:
> Commit a95db58f210 added monitor_vfprintf() as an error_printf()
> generalized from stderr to arbitrary streams, then used it wrapped in
> helper out_printf() to print -device/device_add help to stdout.  Use
> qemu_printf() instead, and delete monitor_vfprintf() and out_printf().
> 
> Cc: Dr. David Alan Gilbert 
> Signed-off-by: Markus Armbruster 

(the 'monitor_vprintf vs monitor_vfprintf took a minute to notice)

Reviewed-by: Dr. David Alan Gilbert 

> ---
>  include/monitor/monitor.h |  3 ---
>  monitor.c | 16 
>  qdev-monitor.c| 36 ++--
>  3 files changed, 18 insertions(+), 37 deletions(-)
> 
> diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
> index e4c3717454..316a168c41 100644
> --- a/include/monitor/monitor.h
> +++ b/include/monitor/monitor.h
> @@ -48,7 +48,4 @@ int monitor_fdset_dup_fd_add(int64_t fdset_id, int dup_fd);
>  void monitor_fdset_dup_fd_remove(int dup_fd);
>  int monitor_fdset_dup_fd_find(int dup_fd);
>  
> -int monitor_vfprintf(FILE *stream,
> - const char *fmt, va_list ap) GCC_FMT_ATTR(2, 0);
> -
>  #endif /* MONITOR_H */
> diff --git a/monitor.c b/monitor.c
> index 7b4a78d798..10be8bdb86 100644
> --- a/monitor.c
> +++ b/monitor.c
> @@ -4541,23 +4541,15 @@ static void monitor_readline_flush(void *opaque)
>  monitor_flush(opaque);
>  }
>  
> -/*
> - * Print to current monitor if we have one, else to stream.
> - */
> -int monitor_vfprintf(FILE *stream, const char *fmt, va_list ap)
> -{
> -if (cur_mon && !monitor_cur_is_qmp()) {
> -return monitor_vprintf(cur_mon, fmt, ap);
> -}
> -return vfprintf(stream, fmt, ap);
> -}
> -
>  /*
>   * Print to current monitor if we have one, else to stderr.
>   */
>  int error_vprintf(const char *fmt, va_list ap)
>  {
> -return monitor_vfprintf(stderr, fmt, ap);
> +if (cur_mon && !monitor_cur_is_qmp()) {
> +return monitor_vprintf(cur_mon, fmt, ap);
> +}
> +return vfprintf(stderr, fmt, ap);
>  }
>  
>  int error_vprintf_unless_qmp(const char *fmt, va_list ap)
> diff --git a/qdev-monitor.c b/qdev-monitor.c
> index d4320986a2..373b9ad445 100644
> --- a/qdev-monitor.c
> +++ b/qdev-monitor.c
> @@ -31,6 +31,7 @@
>  #include "qemu/error-report.h"
>  #include "qemu/help_option.h"
>  #include "qemu/option.h"
> +#include "qemu/qemu-print.h"
>  #include "sysemu/block-backend.h"
>  #include "migration/misc.h"
>  
> @@ -104,31 +105,22 @@ static bool qdev_class_has_alias(DeviceClass *dc)
>  return (qdev_class_get_alias(dc) != NULL);
>  }
>  
> -static void out_printf(const char *fmt, ...)
> -{
> -va_list ap;
> -
> -va_start(ap, fmt);
> -monitor_vfprintf(stdout, fmt, ap);
> -va_end(ap);
> -}
> -
>  static void qdev_print_devinfo(DeviceClass *dc)
>  {
> -out_printf("name \"%s\"", object_class_get_name(OBJECT_CLASS(dc)));
> +qemu_printf("name \"%s\"", object_class_get_name(OBJECT_CLASS(dc)));
>  if (dc->bus_type) {
> -out_printf(", bus %s", dc->bus_type);
> +qemu_printf(", bus %s", dc->bus_type);
>  }
>  if (qdev_class_has_alias(dc)) {
> -out_printf(", alias \"%s\"", qdev_class_get_alias(dc));
> +qemu_printf(", alias \"%s\"", qdev_class_get_alias(dc));
>  }
>  if (dc->desc) {
> -out_printf(", desc \"%s\"", dc->desc);
> +qemu_printf(", desc \"%s\"", dc->desc);
>  }
>  if (!dc->user_creatable) {
> -out_printf(", no-user");
> +qemu_printf(", no-user");
>  }
> -out_printf("\n");
> +qemu_printf("\n");
>  }
>  
>  static void qdev_print_devinfos(bool show_no_user)
> @@ -164,7 +156,7 @@ static void qdev_print_devinfos(bool show_no_user)
>  continue;
>  }
>  if (!cat_printed) {
> -out_printf("%s%s devices:\n", i ? "\n" : "", cat_name[i]);
> +qemu_printf("%s%s devices:\n", i ? "\n" : "", cat_name[i]);
>  cat_printed = true;
>  }
>  qdev_print_devinfo(dc);
> @@ -286,20 +278,20 @@ int qdev_device_help(QemuOpts *opts)
>  }
>  
>  if (prop_list) {
> -out_printf("%s options:\n", driver);
> +qemu_printf("%s options:\n", driver);
>  } else {
> -out_printf("There are no options for %s.\n", driver);
> +qemu_printf("There are no options for %s.\n", driver);
>  }
>  for (prop = prop_list; prop; prop = prop->next) {
>  int len;
> -out_printf("  %s=<%s>%n", prop->value->name, prop->value->type, 
> );
> +qemu_printf("  %s=<%s>%n", prop->value->name, prop->value->type, 
> );
>  if (prop->value->has_description) {
>  if (len < 24) {
> -out_printf("%*s", 24 - len, "");
> +qemu_printf("%*s", 24 - len, "");
>  }
> -out_printf(" - %s\n", prop->value->description);
> +qemu_printf(" - %s\n",

Re: [Qemu-devel] [Bug 1814352] Re: SIOCGIFNAME takes a struct ifreq not an integer

2019-04-09 Thread Erik Kline

Sure.  Looking at HEAD, and even the surrounding the lines, I think I
should have tried STRUCT_short_ifreq instead of STRUCT_int_ifreq, though
I'm not sure what the real difference would be.

I'll try to test internally with the _short_ version and if that works send
that.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1814352

Title:
  SIOCGIFNAME takes a struct ifreq not an integer

Status in QEMU:
  Confirmed

Bug description:
  The ioctl SIOCGIFNAME takes a pointer to a struct ifreq, not an
  integer.  This leads to if_indextoname() not correctly returning
  interface names (well, not if they're longer than 4 characters
  including the trailing NULL ;-).

  This is observed on v3.1.0.

  The following one-line patch will be sent to the qemu-devel mailing
  list:

  """
  diff --git a/linux-user/ioctls.h b/linux-user/ioctls.h
  index ae8951625f..37501f575c 100644
  --- a/linux-user/ioctls.h
  +++ b/linux-user/ioctls.h
  @@ -178,7 +178,7 @@
   #endif /* CONFIG_USBFS */
   
 IOCTL(SIOCATMARK, IOC_R, MK_PTR(TYPE_INT))
  -  IOCTL(SIOCGIFNAME, IOC_RW, MK_PTR(TYPE_INT))
  +  IOCTL(SIOCGIFNAME, IOC_RW, MK_PTR(MK_STRUCT(STRUCT_int_ifreq)))
 IOCTL(SIOCGIFFLAGS, IOC_W | IOC_R, MK_PTR(MK_STRUCT(STRUCT_short_ifreq)))
 IOCTL(SIOCSIFFLAGS, IOC_W, MK_PTR(MK_STRUCT(STRUCT_short_ifreq)))
 IOCTL(SIOCGIFADDR, IOC_W | IOC_R, MK_PTR(MK_STRUCT(STRUCT_sockaddr_ifreq)))
  """

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1814352/+subscriptions

[Qemu-devel] [Bug 1823998] [NEW] qemu-system-aarch64: support kernels bigger than 128MiB

2019-04-09 Thread Mark Rutland

Public bug reported:

Presently QEMU reserves up to 128MiB of space for an arm64 Linux kernel,
placing the initrd following this, and the dtb following the initrd.

This is not sufficient for some debug configurations of the kernel,
which can be larger than 128MiB. Depending on the relative size of the
kernel Image and unpopulated BSS, the dtb (or kernel) will be clobbered
by the other, resulting in a silent boot failure.

Since v3.17, the kernel Image header exposes a field called image_size,
which describes the entire size of the kernel (including unpopulated
sections such as the BSS) as a 64-bit little-endian value. For kernels
prior to v3.17, this field is zero. This is documented at:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/arm64/booting.txt?h=v5.0#n68

It would be great if QEMU could take the image_size field into account
when placing the initrd and dtb to avoid overlap with the kernel.

** Affects: qemu
Importance: Undecided
Status: New

** Tags: feature-request

--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1823998

Title:
qemu-system-aarch64: support kernels bigger than 128MiB

Status in QEMU:
New

Bug description:
Presently QEMU reserves up to 128MiB of space for an arm64 Linux
kernel, placing the initrd following this, and the dtb following the
initrd.

This is not sufficient for some debug configurations of the kernel,
which can be larger than 128MiB. Depending on the relative size of the
kernel Image and unpopulated BSS, the dtb (or kernel) will be
clobbered by the other, resulting in a silent boot failure.

Since v3.17, the kernel Image header exposes a field called
image_size, which describes the entire size of the kernel (including
unpopulated sections such as the BSS) as a 64-bit little-endian value.
For kernels prior to v3.17, this field is zero. This is documented at:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/arm64/booting.txt?h=v5.0#n68

It would be great if QEMU could take the image_size field into account
when placing the initrd and dtb to avoid overlap with the kernel.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1823998/+subscriptions

Re: [Qemu-devel] [PATCH 12/15] qemu-print: New qemu_printf(), qemu_vprintf() etc.

2019-04-09 Thread Dr. David Alan Gilbert

* Markus Armbruster (arm...@redhat.com) wrote:
> We commonly want to print to the current monitor if we have one, else
> to stdout/stderr.  For stderr, have error_printf().  For stdout, all
> we have is monitor_vfprintf(), which is rather unwieldy.  We often
> print to stderr just because error_printf() is easier.
> 
> New qemu_printf() and qemu_vprintf() do exactly what's needed.  The
> next commits will put them to use.
> 
> Cc: Dr. David Alan Gilbert 
> Signed-off-by: Markus Armbruster 

OK, it seems a little odd not to have the fprintf as well, but OK.

Reviewed-by: Dr. David Alan Gilbert 

> ---
>  MAINTAINERS   |  2 ++
>  include/qemu/qemu-print.h | 19 ++
>  stubs/monitor.c   |  5 +
>  util/Makefile.objs|  1 +
>  util/qemu-print.c | 42 +++
>  5 files changed, 69 insertions(+)
>  create mode 100644 include/qemu/qemu-print.h
>  create mode 100644 util/qemu-print.c
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 56139ac8ab..1aa19dc4ef 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1895,6 +1895,8 @@ F: hmp.[ch]
>  F: hmp-commands*.hx
>  F: include/monitor/hmp-target.h
>  F: tests/test-hmp.c
> +F: include/qemu/qemu-print.h
> +F: util/qemu-print.c
>  
>  Network device backends
>  M: Jason Wang 
> diff --git a/include/qemu/qemu-print.h b/include/qemu/qemu-print.h
> new file mode 100644
> index 00..8fed32bf42
> --- /dev/null
> +++ b/include/qemu/qemu-print.h
> @@ -0,0 +1,19 @@
> +/*
> + * Print to stream or current monitor
> + *
> + * Copyright (C) 2019 Red Hat Inc.
> + *
> + * Authors:
> + *  Markus Armbruster ,
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + */
> +
> +#ifndef QEMU_PRINT_H
> +#define QEMU_PRINT_H
> +
> +int qemu_vprintf(const char *fmt, va_list ap) GCC_FMT_ATTR(1, 0);
> +int qemu_printf(const char *fmt, ...) GCC_FMT_ATTR(1, 2);
> +
> +#endif
> diff --git a/stubs/monitor.c b/stubs/monitor.c
> index b57fe6c32f..b2ea975e40 100644
> --- a/stubs/monitor.c
> +++ b/stubs/monitor.c
> @@ -6,6 +6,11 @@
>  
>  __thread Monitor *cur_mon;
>  
> +int monitor_vprintf(Monitor *mon, const char *fmt, va_list ap)
> +{
> +abort();
> +}
> +
>  int monitor_get_fd(Monitor *mon, const char *name, Error **errp)
>  {
>  error_setg(errp, "only QEMU supports file descriptor passing");
> diff --git a/util/Makefile.objs b/util/Makefile.objs
> index 835fcd69e2..9206878dec 100644
> --- a/util/Makefile.objs
> +++ b/util/Makefile.objs
> @@ -23,6 +23,7 @@ util-obj-y += bitmap.o bitops.o hbitmap.o
>  util-obj-y += fifo8.o
>  util-obj-y += cacheinfo.o
>  util-obj-y += error.o qemu-error.o
> +util-obj-y += qemu-print.o
>  util-obj-y += id.o
>  util-obj-y += iov.o qemu-config.o qemu-sockets.o uri.o notify.o
>  util-obj-y += qemu-option.o qemu-progress.o
> diff --git a/util/qemu-print.c b/util/qemu-print.c
> new file mode 100644
> index 00..86f9417af8
> --- /dev/null
> +++ b/util/qemu-print.c
> @@ -0,0 +1,42 @@
> +/*
> + * Print to stream or current monitor
> + *
> + * Copyright (C) 2019 Red Hat Inc.
> + *
> + * Authors:
> + *  Markus Armbruster ,
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + */
> +
> +#include "qemu/osdep.h"
> +#include "monitor/monitor.h"
> +#include "qemu/qemu-print.h"
> +
> +/*
> + * Print like vprintf().
> + * Print to current monitor if we have one, else to stdout.
> + */
> +int qemu_vprintf(const char *fmt, va_list ap)
> +{
> +if (cur_mon) {
> +return monitor_vprintf(cur_mon, fmt, ap);
> +}
> +return vprintf(fmt, ap);
> +}
> +
> +/*
> + * Print like printf().
> + * Print to current monitor if we have one, else to stdout.
> + */
> +int qemu_printf(const char *fmt, ...)
> +{
> +va_list ap;
> +int ret;
> +
> +va_start(ap, fmt);
> +ret = qemu_vprintf(fmt, ap);
> +va_end(ap);
> +return ret;
> +}
> -- 
> 2.17.2
> 
--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH 00/10] block/pflash_cfi02: Implement missing AMD pflash functionality

2019-04-09 Thread Stephen Checkoway

On Apr 9, 2019, at 12:15, Philippe Mathieu-Daudé  wrote:

> Since you did changes in the CFI table, I think we should add a tests
> verifying the table is correctly generated for all you FlashConfig entries.

That's a good idea. Some of the values are essentially arbitrary (e.g., how 
long an erase typically takes) but the CFI values that indicate support for 
erase/suspend and erase regions at least should be tested. I'll take a closer 
look at what all I changed (it has been a few months since I wrote the code). 
Are there any in particular you think I should be sure to test? Do you want me 
to add those tests to the appropriate patches in the series or would you like 
an additional patch that adds those tests? (The later is easier to do as 
modifying the earlier patches in the series are likely to cause conflicts with 
later patches that I'd need to resolve, but I can do either.)

> I suppose you can not share the firmware binary. Can you share these
> unlock addresses? Maybe once I reviewed carefully your series I might
> ask you the (pflash) trace event output.

I probably cannot share the binary but the unlock addresses are 0x1554 and 
0x2AA8. This is for two interleaved x8/x16 chips in x16 mode so shifting right 
by 2 (1 for the interleaving and 1 because it's an x16 chip) gives 0x555 and 
0xAAA. The appropriate (word) unlock addresses for the chips are 0x555 and 
0x2AA which is what you get if you restrict to 11 bits.

My guess is that the firmware authors took the byte unlock addresses (0xAAA and 
0x555), shifted left by 2, discovered that this didn't work, tried the unlock 
addresses in the opposite order, and found that that worked due to the chips 
only looking at 11 bits.

Looking at hw/arm/musicpal.c, I see that it is using unlock addresses 0x 
and 0x2AAA. My guess is this reflects a bug in some firmware and it should be 
using 0x555 and 0x2AA. I haven't seen any AMD pflash chips that used other 
unlock addresses, but I've only looked at about a dozen part sheets and I'm not 
sure what chips the musicpal is actually using.

-- 
Stephen Checkoway

[Qemu-devel] [PATCH for-4.0-maybe] device_tree: Fix integer overflowing in load_device_tree()

If the value of get_image_size() exceeds INT_MAX / 2 - 1, the
computation of @dt_size overflows to a negative number, which then
gets converted to a very large size_t for g_malloc0() and
load_image_size().  In the (fortunately improbable) case g_malloc0()
succeeds and load_image_size() survives, we'd assign the negative
number to *sizep.  What that would do to the callers I can't say, but
it's unlikely to be good.

Fix by rejecting images whose size would overflow.

Signed-off-by: Markus Armbruster 
---
 device_tree.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/device_tree.c b/device_tree.c
index 296278e12a..f8b46b3c73 100644
--- a/device_tree.c
+++ b/device_tree.c
@@ -84,6 +84,10 @@ void *load_device_tree(const char *filename_path, int *sizep)
  filename_path);
 goto fail;
 }
+if (dt_size > INT_MAX / 2 - 1) {
+error_report("Device tree file '%s' is too large", filename_path);
+goto fail;
+}
 
 /* Expand to 2x size to give enough room for manipulation.  */
 dt_size += 1;
-- 
2.17.2

Re: [Qemu-devel] How to correctly use more than 2 floppy drives?




On 4/9/19 7:38 AM, Philippe Mathieu-Daudé wrote:
> On 4/8/19 9:30 PM, John Snow wrote:
>> On 4/8/19 1:38 AM, Markus Armbruster wrote:
>>> Hervé Poussineau  writes:
>>>
 Le 05/04/2019 à 12:29, Philippe Mathieu-Daudé a écrit :
> Hi,
>
> I am trying to understand the possible values for the MAX_FD variable
> used by the floppy controller model (hw/block/fdc.c).
> 
>> Out of curiosity, why?
> 
> Cleaning Super I/O chipset I figured some code uses arrays of 2 elements
> while other use MAX_FD. If we want to have a build-configurable MAX_FD
> we can't simply replace "2" -> "MAX_FD" to clean the codebase, we need
> to correct various places, and fix migration.
> If we agree that MAX_FD is strictly 2, then we can clean the codebase
> and remove the MAX_FD != 2 cases (point 3/ below).

OK! I just get nervous when people start poking around floppy disks :)

It's "maintained" but really I just keep it on life support. If anyone
were to claim responsibility for it I would happily let them. As you can
guess, floppy disks are not a huge business priority for my employer and
I am not personally passionate about them, so...

I just keep it alive out of kindness and, for a time, support for
loading virtio drivers on Windows XP VMs, which is a use case that Red
Hat doesn't care about too much anymore.

>>
>> I think I'd rather have MAX_FD set to 2 and a cleaner codebase than a
>> half-working implementation for 4.
>>
>> Or, does it actually work with four? I think if Hervé wants to preserve
>> this feature it should be formalized as a device property and made to
>> work with migration ... or I am content to remove it.
> 
> I understand Hervé doesn't want to preserve his attempt ("no emulated
> board took advantage of it"), and the work is safe in the git history if
> someone want to restore it (in a way that doesn't break migration).
> 
> Since we all seems to agree, I prepared a series to remove it and will
> send it soon (although there is no rush).
> 
> Thanks,
> 
> Phil.
> 

Sounds good, thank you!

Re: [Qemu-devel] [PULL v3 0/5] Misc patches for QEMU 5.0-rc3

On Tue, 9 Apr 2019 at 17:36, Paolo Bonzini  wrote:
>
> The following changes since commit 248987f92cfda8305d6d44ced23e173e62a8bc0e:
>
>   Merge remote-tracking branch 
> 'remotes/vivier2/tags/linux-user-for-4.0-pull-request' into staging 
> (2019-04-09 10:02:30 +0100)
>
> are available in the git repository at:
>
>
>   git://github.com/bonzini/qemu.git tags/for-upstream
>
> for you to fetch changes up to 3e20c81ed87cb15cd04f929b075e244e0758641a:
>
>   tests: Make check-block a phony target (2019-04-09 18:34:21 +0200)
>
> 
> * fixes for Alpine and SuSE
> * fix crash when hot-plugging nvdimm on older machine types
>

Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/4.0
for any user-visible changes.

-- PMM

Re: [Qemu-devel] [PATCH for 4.1 v2 2/6] target/riscv: Fall back to generating a RISC-V CPU

On Thu, Apr 4, 2019 at 5:35 AM Ian Campbell  wrote:
>
> On Fri, 2019-03-29 at 22:39 +, Alistair Francis wrote:
> > +for (i = 0; i < strlen(riscv_cpu); i++) {
> > +if (i == 0 && riscv_cpu[i] == 'r' &&
> > +riscv_cpu[i + 1] == 'v') {
>
> Dpes something somewhere else enforce a minimum length or can
> `riscv_cpu[i + 1]` be past the end of the string?
>
> Similarly some more times below and also for the `i += 2` bits.

Good point, a valid string must be at least 5 characters so I have
added a check before we start the parsing.

Alistair

>
> > +/* Starts with "rv" */
> > +i += 2;
> > +if (riscv_cpu[i] == '3' && riscv_cpu[i + 1] == '2') {
> > +i += 2;
> > +valid = true;
> > +rvxlen = RV32;
> > +}
> > +if (riscv_cpu[i] == '6' && riscv_cpu[i + 1] == '4') {
> > +i += 2;
> > +valid = true;
> > +rvxlen = RV64;
> > +}
> > +}
>
> Ian.
>

Re: [Qemu-devel] [PATCH] tci: Fix some unaligned memory accesses

On Tue, 9 Apr 2019 at 18:04, Stefan Weil  wrote:
>
> On 09.04.19 08:58, Richard Henderson wrote:
> > On 4/8/19 8:04 PM, Stefan Weil wrote:
> >>  static tcg_target_ulong tci_read_i(uint8_t **tb_ptr)
> >>  {
> >> -tcg_target_ulong value = *(tcg_target_ulong *)(*tb_ptr);
> >> +tcg_target_ulong value;
> >
> > Ideally these would use the helpers from "qemu/bswap.h", ldl_he_p(), etc.

> That would require adding a helper for tcg_target_ulong to qemu/bswap.h.
> Or tci.c would need conditional code for reading a tcg_target_ulong.
>
> Those helpers in qemu/bswap.h are also a little bit strange:
>
> - Why does lduw_he_p return an int instead of an uint16_t?
> - Why does ldsw_he_p return an int instead of an int16_t?
> - Why does ldl_he_p return an int instead of an int32_t?

Some of these are for historical reasons (ie their API was
designed in a world where the callers were typically throwing
values around in "int"s rather than carefully using exact
right-sized types). If you want to assign the results to a
uint16_t/int16_t/int32_t this will work fine (and no cast is
required).

> - Should ldl_he_p be renamed into ldsl_he_p?
>   And why is ldul_he_p missing?

The assumption is that for 32-bit accesses there is no need
to care about the signedness of the load because the result
is going into a 32-bit variable anyway and so there is no
extension of any kind to be done.

> - Should ldq_he_p be renamed into lduq_he_p?
>   And why is ldsq_he_p missing?

Similarly here the value is being returned as a 64-bit
quantity, so there is no question of whether it should
be sign or zero extended as it isn't being extended at all.

> Using the helpers might require nasty type casts to avoid compiler
> warnings because of signed / unsigned and size mismatches.
>
> Aren't the few memcpy statements in the TCI helpers much more direct and
> understandable?

In general, no. "memcpy is a way to do an efficient possibly-unaligned
access" is an obscure bit of knowledge. We deliberately abstract this
away into these helper functions which provide APIs which are more
clearly "we need to perform a possibly-unaligned load of a specified
endianness". (It also means that if it turns out that we would
rather use __builtin_memcpy() rather than memcpy(), as we've just
discovered we need to, then there's only one file to change rather
than many.)

thanks
-- PMM

Re: [Qemu-devel] [PATCH 0/4] Header cleanups

Thanks!

On Tue, Apr 9, 2019 at 10:36 PM, Thomas Huth  wrote:

> On 09/04/2019 17.56, Aruna Jayasena wrote:
> > Removed unwanted includes from cpu-common.h
> > This task was under https://wiki.qemu.org/Contribute/BiteSizedTasks
> >
> > Signed-off-by: Aruna Jayasena 
> >
> > ---
> >  include/exec/cpu-common.h | 2 --
> >  1 file changed, 2 deletions(-)
> >
> > diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h
> > index cef8b88a2a..a3594f3f50 100644
> > --- a/include/exec/cpu-common.h
> > +++ b/include/exec/cpu-common.h
> > @@ -7,8 +7,6 @@
> >  #include "exec/hwaddr.h"
> >  #endif
> >
> > -#include "qemu/bswap.h"
> > -#include "qemu/queue.h"
> >  #include "qemu/fprintf-fn.h"
> >
> >  /**
> >
>
> Reviewed-by: Thomas Huth 
>
-- 
*Aruna* Jayasena
Embedded Systems and Computer Science Engineering Undergraduate
University of Moratuwa
*✆ **+94719872152 *
✉  ⌨  in

Re: [Qemu-devel] [PATCH 0/4] Header cleanups

2019-04-09 Thread Thomas Huth

On 09/04/2019 17.56, Aruna Jayasena wrote:
> Removed unwanted includes from cpu-common.h
> This task was under https://wiki.qemu.org/Contribute/BiteSizedTasks
> 
> Signed-off-by: Aruna Jayasena 
> 
> ---
>  include/exec/cpu-common.h | 2 --
>  1 file changed, 2 deletions(-)
> 
> diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h
> index cef8b88a2a..a3594f3f50 100644
> --- a/include/exec/cpu-common.h
> +++ b/include/exec/cpu-common.h
> @@ -7,8 +7,6 @@
>  #include "exec/hwaddr.h"
>  #endif
>  
> -#include "qemu/bswap.h"
> -#include "qemu/queue.h"
>  #include "qemu/fprintf-fn.h"
>  
>  /**
> 

Reviewed-by: Thomas Huth

Re: [Qemu-devel] [PATCH] tci: Fix some unaligned memory accesses

2019-04-09 Thread Stefan Weil

On 09.04.19 08:58, Richard Henderson wrote:
> On 4/8/19 8:04 PM, Stefan Weil wrote:
>>  static tcg_target_ulong tci_read_i(uint8_t **tb_ptr)
>>  {
>> -tcg_target_ulong value = *(tcg_target_ulong *)(*tb_ptr);
>> +tcg_target_ulong value;
> 
> Ideally these would use the helpers from "qemu/bswap.h", ldl_he_p(), etc.
> 
> r~

That would require adding a helper for tcg_target_ulong to qemu/bswap.h.
Or tci.c would need conditional code for reading a tcg_target_ulong.

Those helpers in qemu/bswap.h are also a little bit strange:

- Why does lduw_he_p return an int instead of an uint16_t?
- Why does ldsw_he_p return an int instead of an int16_t?
- Why does ldl_he_p return an int instead of an int32_t?
- Should ldl_he_p be renamed into ldsl_he_p?
  And why is ldul_he_p missing?
- Should ldq_he_p be renamed into lduq_he_p?
  And why is ldsq_he_p missing?

Using the helpers might require nasty type casts to avoid compiler
warnings because of signed / unsigned and size mismatches.

Aren't the few memcpy statements in the TCI helpers much more direct and
understandable?

Regards
Stefan

Re: [Qemu-devel] [PATCH 0/4] Header cleanups

Ok, thanks

On Tue, Apr 9, 2019 at 10:20 PM, Peter Maydell 
wrote:

> On Tue, 9 Apr 2019 at 17:42, Aruna Jayasena 
> wrote:
> >
> > Yeah this is a stand alone patch.
>
> OK, thanks -- just wanted to check there wasn't a missing
> bit. I think this could go in via the qemu-trivial tree
> so I have cc'd them.
>
> We're in the middle of a release at the moment, but we
> should reopen for development in a couple of weeks. If
> this patch still hasn't gone anywhere by a week or so after
> that then please follow up with a 'ping' to check it hasn't
> been forgotten by accident.
>
> > On Tue, Apr 9, 2019 at 10:10 PM, Peter Maydell 
> wrote:
> >> On Tue, 9 Apr 2019 at 16:58, Aruna Jayasena 
> wrote:
> >> >
> >> > Removed unwanted includes from cpu-common.h
> >> > This task was under https://wiki.qemu.org/Contribute/BiteSizedTasks
> >> >
> >> > Signed-off-by: Aruna Jayasena 
> >> >
> >> > ---
> >> >  include/exec/cpu-common.h | 2 --
> >> >  1 file changed, 2 deletions(-)
> >> >
> >> > diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h
> >> > index cef8b88a2a..a3594f3f50 100644
> >> > --- a/include/exec/cpu-common.h
> >> > +++ b/include/exec/cpu-common.h
> >> > @@ -7,8 +7,6 @@
> >> >  #include "exec/hwaddr.h"
> >> >  #endif
> >> >
> >> > -#include "qemu/bswap.h"
> >> > -#include "qemu/queue.h"
> >> >  #include "qemu/fprintf-fn.h"
> >> >
> >>
> >> I was surprised that no .c files are implicitly relying
> >> on these includes, but apparently none are, so we can
> >> safely remove them.
> >>
> >> A couple of notes:
> >>  * this doesn't need to go to qemu-stable as it's just a
> >>code cleanup
> >>  * did something go wrong with your sending of this patch?
> >>Its subject is "PATCH 0/4" which would usually indicate
> >>a cover letter for a four-patch series, but this seems
> >>to be a stand-alone patch.
> >>
> >> Reviewed-by: Peter Maydell 
>
> thanks
> -- PMM
>
-- 
*Aruna* Jayasena
Embedded Systems and Computer Science Engineering Undergraduate
University of Moratuwa
*✆ **+94719872152 *
✉  ⌨  in

Re: [Qemu-devel] [PATCH 0/4] Header cleanups

On Tue, 9 Apr 2019 at 17:42, Aruna Jayasena  wrote:
>
> Yeah this is a stand alone patch.

OK, thanks -- just wanted to check there wasn't a missing
bit. I think this could go in via the qemu-trivial tree
so I have cc'd them.

We're in the middle of a release at the moment, but we
should reopen for development in a couple of weeks. If
this patch still hasn't gone anywhere by a week or so after
that then please follow up with a 'ping' to check it hasn't
been forgotten by accident.

> On Tue, Apr 9, 2019 at 10:10 PM, Peter Maydell  
> wrote:
>> On Tue, 9 Apr 2019 at 16:58, Aruna Jayasena  wrote:
>> >
>> > Removed unwanted includes from cpu-common.h
>> > This task was under https://wiki.qemu.org/Contribute/BiteSizedTasks
>> >
>> > Signed-off-by: Aruna Jayasena 
>> >
>> > ---
>> >  include/exec/cpu-common.h | 2 --
>> >  1 file changed, 2 deletions(-)
>> >
>> > diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h
>> > index cef8b88a2a..a3594f3f50 100644
>> > --- a/include/exec/cpu-common.h
>> > +++ b/include/exec/cpu-common.h
>> > @@ -7,8 +7,6 @@
>> >  #include "exec/hwaddr.h"
>> >  #endif
>> >
>> > -#include "qemu/bswap.h"
>> > -#include "qemu/queue.h"
>> >  #include "qemu/fprintf-fn.h"
>> >
>>
>> I was surprised that no .c files are implicitly relying
>> on these includes, but apparently none are, so we can
>> safely remove them.
>>
>> A couple of notes:
>>  * this doesn't need to go to qemu-stable as it's just a
>>code cleanup
>>  * did something go wrong with your sending of this patch?
>>Its subject is "PATCH 0/4" which would usually indicate
>>a cover letter for a four-patch series, but this seems
>>to be a stand-alone patch.
>>
>> Reviewed-by: Peter Maydell 

thanks
-- PMM

Re: [Qemu-devel] [PATCH 0/4] Header cleanups

Thank you

On Tue, Apr 9, 2019 at 10:12 PM, Aruna Jayasena 
wrote:

> Yeah this is a stand alone patch.
>
> On Tue, Apr 9, 2019 at 10:10 PM, Peter Maydell 
> wrote:
>
>> On Tue, 9 Apr 2019 at 16:58, Aruna Jayasena 
>> wrote:
>> >
>> > Removed unwanted includes from cpu-common.h
>> > This task was under https://wiki.qemu.org/Contribute/BiteSizedTasks
>> >
>> > Signed-off-by: Aruna Jayasena 
>> >
>> > ---
>> >  include/exec/cpu-common.h | 2 --
>> >  1 file changed, 2 deletions(-)
>> >
>> > diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h
>> > index cef8b88a2a..a3594f3f50 100644
>> > --- a/include/exec/cpu-common.h
>> > +++ b/include/exec/cpu-common.h
>> > @@ -7,8 +7,6 @@
>> >  #include "exec/hwaddr.h"
>> >  #endif
>> >
>> > -#include "qemu/bswap.h"
>> > -#include "qemu/queue.h"
>> >  #include "qemu/fprintf-fn.h"
>> >
>>
>> I was surprised that no .c files are implicitly relying
>> on these includes, but apparently none are, so we can
>> safely remove them.
>>
>> A couple of notes:
>>  * this doesn't need to go to qemu-stable as it's just a
>>code cleanup
>>  * did something go wrong with your sending of this patch?
>>Its subject is "PATCH 0/4" which would usually indicate
>>a cover letter for a four-patch series, but this seems
>>to be a stand-alone patch.
>>
>> Reviewed-by: Peter Maydell 
>>
>> thanks
>> -- PMM
>>
> --
> *Aruna* Jayasena
> Embedded Systems and Computer Science Engineering Undergraduate
> University of Moratuwa
> *✆ **+94719872152 *
> ✉  ⌨  in
> 
>
>
> --
*Aruna* Jayasena
Embedded Systems and Computer Science Engineering Undergraduate
University of Moratuwa
*✆ **+94719872152 *
✉  ⌨  in

Re: [Qemu-devel] [PATCH 0/4] Header cleanups

Yeah this is a stand alone patch.

On Tue, Apr 9, 2019 at 10:10 PM, Peter Maydell 
wrote:

> On Tue, 9 Apr 2019 at 16:58, Aruna Jayasena 
> wrote:
> >
> > Removed unwanted includes from cpu-common.h
> > This task was under https://wiki.qemu.org/Contribute/BiteSizedTasks
> >
> > Signed-off-by: Aruna Jayasena 
> >
> > ---
> >  include/exec/cpu-common.h | 2 --
> >  1 file changed, 2 deletions(-)
> >
> > diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h
> > index cef8b88a2a..a3594f3f50 100644
> > --- a/include/exec/cpu-common.h
> > +++ b/include/exec/cpu-common.h
> > @@ -7,8 +7,6 @@
> >  #include "exec/hwaddr.h"
> >  #endif
> >
> > -#include "qemu/bswap.h"
> > -#include "qemu/queue.h"
> >  #include "qemu/fprintf-fn.h"
> >
>
> I was surprised that no .c files are implicitly relying
> on these includes, but apparently none are, so we can
> safely remove them.
>
> A couple of notes:
>  * this doesn't need to go to qemu-stable as it's just a
>code cleanup
>  * did something go wrong with your sending of this patch?
>Its subject is "PATCH 0/4" which would usually indicate
>a cover letter for a four-patch series, but this seems
>to be a stand-alone patch.
>
> Reviewed-by: Peter Maydell 
>
> thanks
> -- PMM
>
-- 
*Aruna* Jayasena
Embedded Systems and Computer Science Engineering Undergraduate
University of Moratuwa
*✆ **+94719872152 *
✉  ⌨  in

[Qemu-devel] [PATCH] xen-block: support feature-large-sector-size

2019-04-09 Thread Paul Durrant

A recent Xen commit [1] clarified the semantics of sector based quantities
used in the blkif protocol such that it is now safe to create a xen-block
device with a logical_block_size != 512, as long as the device only
connects to a frontend advertizing 'feature-large-block-size'.

This patch modifies xen-block accordingly. It also uses a stack variable
for the BlockBackend in xen_block_realize() to avoid repeated dereferencing
of the BlockConf pointer, and changes the parameters of
xen_block_dataplane_create() so that the BlockBackend pointer and sector
size are passed expicitly rather than implicitly via the BlockConf.

These modifications have been tested against a recent Windows PV XENVBD
driver [2] using a xen-disk device with a 4kB logical block size.

[1] 
http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=67e1c050e36b2c9900cca83618e56189effbad98
[2] https://winpvdrvbuild.xenproject.org:8080/job/XENVBD-master/126

Signed-off-by: Paul Durrant 
---
Cc: Stefano Stabellini 
Cc: Anthony Perard 
Cc: Stefan Hajnoczi 
Cc: Kevin Wolf 
Cc: Max Reitz 
---
 hw/block/dataplane/xen-block.c | 25 --
 hw/block/dataplane/xen-block.h |  3 ++-
 hw/block/xen-block.c   | 38 +-
 3 files changed, 40 insertions(+), 26 deletions(-)

diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
index 908bd27bbd..50094a886b 100644
--- a/hw/block/dataplane/xen-block.c
+++ b/hw/block/dataplane/xen-block.c
@@ -58,6 +58,7 @@ struct XenBlockDataPlane {
 int requests_inflight;
 unsigned int max_requests;
 BlockBackend *blk;
+unsigned int sector_size;
 QEMUBH *bh;
 IOThread *iothread;
 AioContext *ctx;
@@ -167,7 +168,7 @@ static int xen_block_parse_request(XenBlockRequest *request)
 goto err;
 }
 
-request->start = request->req.sector_number * XEN_BLKIF_SECTOR_SIZE;
+request->start = request->req.sector_number * dataplane->sector_size;
 for (i = 0; i < request->req.nr_segments; i++) {
 if (i == BLKIF_MAX_SEGMENTS_PER_REQUEST) {
 error_report("error: nr_segments too big");
@@ -177,14 +178,14 @@ static int xen_block_parse_request(XenBlockRequest 
*request)
 error_report("error: first > last sector");
 goto err;
 }
-if (request->req.seg[i].last_sect * XEN_BLKIF_SECTOR_SIZE >=
+if (request->req.seg[i].last_sect * dataplane->sector_size >=
 XC_PAGE_SIZE) {
 error_report("error: page crossing");
 goto err;
 }
 
 len = (request->req.seg[i].last_sect -
-   request->req.seg[i].first_sect + 1) * XEN_BLKIF_SECTOR_SIZE;
+   request->req.seg[i].first_sect + 1) * dataplane->sector_size;
 request->size += len;
 }
 if (request->start + request->size > blk_getlength(dataplane->blk)) {
@@ -218,17 +219,17 @@ static int xen_block_copy_request(XenBlockRequest 
*request)
 if (to_domain) {
 segs[i].dest.foreign.ref = request->req.seg[i].gref;
 segs[i].dest.foreign.offset = request->req.seg[i].first_sect *
-XEN_BLKIF_SECTOR_SIZE;
+dataplane->sector_size;
 segs[i].source.virt = virt;
 } else {
 segs[i].source.foreign.ref = request->req.seg[i].gref;
 segs[i].source.foreign.offset = request->req.seg[i].first_sect *
-XEN_BLKIF_SECTOR_SIZE;
+dataplane->sector_size;
 segs[i].dest.virt = virt;
 }
 segs[i].len = (request->req.seg[i].last_sect -
request->req.seg[i].first_sect + 1) *
-  XEN_BLKIF_SECTOR_SIZE;
+  dataplane->sector_size;
 virt += segs[i].len;
 }
 
@@ -338,12 +339,12 @@ static bool xen_block_split_discard(XenBlockRequest 
*request,
 
 /* Wrap around, or overflowing byte limit? */
 if (sec_start + sec_count < sec_count ||
-sec_start + sec_count > INT64_MAX / XEN_BLKIF_SECTOR_SIZE) {
+sec_start + sec_count > INT64_MAX / dataplane->sector_size) {
 return false;
 }
 
-byte_offset = sec_start * XEN_BLKIF_SECTOR_SIZE;
-byte_remaining = sec_count * XEN_BLKIF_SECTOR_SIZE;
+byte_offset = sec_start * dataplane->sector_size;
+byte_remaining = sec_count * dataplane->sector_size;
 
 do {
 byte_chunk = byte_remaining > BDRV_REQUEST_MAX_BYTES ?
@@ -626,13 +627,15 @@ static bool xen_block_dataplane_event(void *opaque)
 }
 
 XenBlockDataPlane *xen_block_dataplane_create(XenDevice *xendev,
-  BlockConf *conf,
+  BlockBackend *blk,
+  unsigned int sector_size,
   IOThread *iothread)
 {
 XenBlockDataPlane *dataplane = g_new0(XenBlockDataPlane, 1);
 
 dataplane->xendev = xendev;
-dataplane->blk =

Re: [Qemu-devel] [PATCH 0/4] Header cleanups

On Tue, 9 Apr 2019 at 16:58, Aruna Jayasena  wrote:
>
> Removed unwanted includes from cpu-common.h
> This task was under https://wiki.qemu.org/Contribute/BiteSizedTasks
>
> Signed-off-by: Aruna Jayasena 
>
> ---
>  include/exec/cpu-common.h | 2 --
>  1 file changed, 2 deletions(-)
>
> diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h
> index cef8b88a2a..a3594f3f50 100644
> --- a/include/exec/cpu-common.h
> +++ b/include/exec/cpu-common.h
> @@ -7,8 +7,6 @@
>  #include "exec/hwaddr.h"
>  #endif
>
> -#include "qemu/bswap.h"
> -#include "qemu/queue.h"
>  #include "qemu/fprintf-fn.h"
>

I was surprised that no .c files are implicitly relying
on these includes, but apparently none are, so we can
safely remove them.

A couple of notes:
 * this doesn't need to go to qemu-stable as it's just a
   code cleanup
 * did something go wrong with your sending of this patch?
   Its subject is "PATCH 0/4" which would usually indicate
   a cover letter for a four-patch series, but this seems
   to be a stand-alone patch.

Reviewed-by: Peter Maydell 

thanks
-- PMM

[Qemu-devel] [PULL 5/5] tests: Make check-block a phony target

From: Markus Armbruster 

Fixes: b93b63f574c "test makefile overhaul"

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Message-Id: <20190319072104.32591-1-arm...@redhat.com>
Signed-off-by: Paolo Bonzini 
---
 tests/Makefile.include | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 6b904d7..36fc73f 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -1164,7 +1164,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR)
 
 # Consolidated targets
 
-.PHONY: check-qapi-schema check-qtest check-unit check check-clean
+.PHONY: check-block check-qapi-schema check-qtest check-unit check check-clean
 check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y)) 
check-tests/qapi-schema/doc-good.texi
 check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
 check-block: $(patsubst %,check-%, $(check-block-y))
-- 
1.8.3.1

[Qemu-devel] [PULL v3 0/5] Misc patches for QEMU 5.0-rc3

The following changes since commit 248987f92cfda8305d6d44ced23e173e62a8bc0e:

  Merge remote-tracking branch 
'remotes/vivier2/tags/linux-user-for-4.0-pull-request' into staging (2019-04-09 
10:02:30 +0100)

are available in the git repository at:


  git://github.com/bonzini/qemu.git tags/for-upstream

for you to fetch changes up to 3e20c81ed87cb15cd04f929b075e244e0758641a:

  tests: Make check-block a phony target (2019-04-09 18:34:21 +0200)


* fixes for Alpine and SuSE
* fix crash when hot-plugging nvdimm on older machine types


Markus Armbruster (1):
  tests: Make check-block a phony target

Peter Maydell (1):
  include/qemu/bswap.h: Use __builtin_memcpy() in accessor functions

Philippe Mathieu-Daudé (2):
  roms: Rename the EFIROM variable to avoid clashing with iPXE
  roms: Allow passing configure options to the EDK2 build tools

Thomas Huth (1):
  hw/i386/pc: Fix crash when hot-plugging nvdimm on older machine types

 hw/i386/pc.c   |  9 +++--
 include/qemu/bswap.h   | 26 --
 roms/Makefile  | 23 ++-
 tests/Makefile.include |  2 +-
 4 files changed, 42 insertions(+), 18 deletions(-)
-- 
1.8.3.1

[Qemu-devel] [PULL 3/5] include/qemu/bswap.h: Use __builtin_memcpy() in accessor functions

From: Peter Maydell 

In the accessor functions ld*_he_p() and st*_he_p() we use memcpy()
to perform a load or store to a pointer which might not be aligned
for the size of the type. We rely on the compiler to optimize this
memcpy() into an efficient load or store instruction where possible.
This is required for good performance, but at the moment it is also
required for correct operation, because some users of these functions
require that the access is atomic if the pointer is aligned, which
will only be the case if the compiler has optimized out the memcpy().
(The particular example where we discovered this is the virtio
vring_avail_idx() which calls virtio_lduw_phys_cached() which
eventually ends up calling lduw_he_p().)

Unfortunately some compile environments, such as the fortify-source
setup used in Alpine Linux, define memcpy() to a wrapper function
in a way that inhibits this compiler optimization.

The correct long-term fix here is to add a set of functions for
doing atomic accesses into AddressSpaces (and to other relevant
families of accessor functions like the virtio_*_phys_cached()
ones), and make sure that callsites which want atomic behaviour
use the correct functions.

In the meantime, switch to using __builtin_memcpy() in the
bswap.h accessor functions. This will make us robust against things
like this fortify library in the short term. In the longer term
it will mean that we don't end up with these functions being really
badly-performing even if the semantics of the out-of-line memcpy()
are correct.

Reported-by: Fernando Casas Schössow 
Signed-off-by: Peter Maydell 
Reviewed-by: Richard Henderson 
Message-Id: <20190318112938.8298-1-peter.mayd...@linaro.org>
Signed-off-by: Paolo Bonzini 
---
 include/qemu/bswap.h | 26 --
 1 file changed, 16 insertions(+), 10 deletions(-)

diff --git a/include/qemu/bswap.h b/include/qemu/bswap.h
index 5a70f78..2a9f3fe 100644
--- a/include/qemu/bswap.h
+++ b/include/qemu/bswap.h
@@ -316,51 +316,57 @@ static inline void stb_p(void *ptr, uint8_t v)
 *(uint8_t *)ptr = v;
 }
 
-/* Any compiler worth its salt will turn these memcpy into native unaligned
-   operations.  Thus we don't need to play games with packed attributes, or
-   inline byte-by-byte stores.  */
+/*
+ * Any compiler worth its salt will turn these memcpy into native unaligned
+ * operations.  Thus we don't need to play games with packed attributes, or
+ * inline byte-by-byte stores.
+ * Some compilation environments (eg some fortify-source implementations)
+ * may intercept memcpy() in a way that defeats the compiler optimization,
+ * though, so we use __builtin_memcpy() to give ourselves the best chance
+ * of good performance.
+ */
 
 static inline int lduw_he_p(const void *ptr)
 {
 uint16_t r;
-memcpy(, ptr, sizeof(r));
+__builtin_memcpy(, ptr, sizeof(r));
 return r;
 }
 
 static inline int ldsw_he_p(const void *ptr)
 {
 int16_t r;
-memcpy(, ptr, sizeof(r));
+__builtin_memcpy(, ptr, sizeof(r));
 return r;
 }
 
 static inline void stw_he_p(void *ptr, uint16_t v)
 {
-memcpy(ptr, , sizeof(v));
+__builtin_memcpy(ptr, , sizeof(v));
 }
 
 static inline int ldl_he_p(const void *ptr)
 {
 int32_t r;
-memcpy(, ptr, sizeof(r));
+__builtin_memcpy(, ptr, sizeof(r));
 return r;
 }
 
 static inline void stl_he_p(void *ptr, uint32_t v)
 {
-memcpy(ptr, , sizeof(v));
+__builtin_memcpy(ptr, , sizeof(v));
 }
 
 static inline uint64_t ldq_he_p(const void *ptr)
 {
 uint64_t r;
-memcpy(, ptr, sizeof(r));
+__builtin_memcpy(, ptr, sizeof(r));
 return r;
 }
 
 static inline void stq_he_p(void *ptr, uint64_t v)
 {
-memcpy(ptr, , sizeof(v));
+__builtin_memcpy(ptr, , sizeof(v));
 }
 
 static inline int lduw_le_p(const void *ptr)
-- 
1.8.3.1

[Qemu-devel] [PULL 1/5] roms: Rename the EFIROM variable to avoid clashing with iPXE