Re: [PATCH] hw/arm/aspeed: Map the UART5 device unconditionally

[Andrew Jeffery]

On Fri, 18 Sep 2020, at 02:33, Cédric Le Goater wrote:
> On 9/17/20 6:57 PM, Philippe Mathieu-Daudé wrote:
> > On 9/16/20 7:51 AM, Cédric Le Goater wrote:
> >> On 9/15/20 7:23 PM, Philippe Mathieu-Daudé wrote:
> >>> ping?
> >>
> >> It's reviewed : 
> >>
> >>   
> >> http://patchwork.ozlabs.org/project/qemu-devel/patch/20200905212415.760452-1-f4...@amsat.org/
> >>
> > 
> > Yes I know :) This is part of my routine to check if a
> > patch hasn't been confirmed to be queued 1 week after the
> > last review, to ping the maintainer (because some
> > automatically flush patches older than 1month in their
> > mailbox).
> 
> ooh. That's brutal.
> 
> >> I will send a PR when I have more patches.
> > 
> > Ah OK. I didn't know you would keep merging the Aspeed
> > patches. Since this was a single patch, I thought it would
> > go via the usual qemu-arm queue from Peter.
> 
> sure. It could also. Fine with me. I have only three for the
> moment. 
> 
> > No rush, I just wanted to be sure the patch was not lost.
> > Also, once a patch is queued, I understand it is the
> > maintainer responsibility to keep rebasing the patch
> > queued.
> 
> yes. I know. I have been taking care of Andrew's ADC patches 
> since 2017 ... cough cough :)

Agh!

Re: [PATCH v2 11/13] block/export: convert vhost-user-blk server to block export API

[Markus Armbruster]

Stefan Hajnoczi  writes:

> Use the new QAPI block exports API instead of defining our own QOM
> objects.
>
> This is a large change because the lifecycle of VuBlockDev needs to
> follow BlockExportDriver. QOM properties are replaced by QAPI options
> objects.
>
> VuBlockDev is renamed VuBlkExport and contains a BlockExport field.
> Several fields can be dropped since BlockExport already has equivalents.
>
> The file names and meson build integration will be adjusted in a future
> patch. libvhost-user should probably be built as a static library that
> is linked into QEMU instead of as a .c file that results in duplicate
> compilation.
>
> The new command-line syntax is:
>
>   $ qemu-storage-daemon \
>   --blockdev file,node-name=drive0,filename=test.img \
>   --export 
> vhost-user-blk,node-name=drive0,id=export0,unix-socket=/tmp/vhost-user-blk.sock
>
> Note that unix-socket is optional because we may wish to accept chardevs
> too in the future.
>
> Signed-off-by: Stefan Hajnoczi 
> ---
> v2:
>  * Replace str unix-socket with SocketAddress addr to match NBD and
>support file descriptor passing
>  * Make addr mandatory [Markus]
>  * Update vhost-user-blk-test.c to use --export syntax
> ---
>  qapi/block-export.json   |  21 +-
>  block/export/vhost-user-blk-server.h |  23 +-
>  block/export/export.c|   8 +-
>  block/export/vhost-user-blk-server.c | 452 +++
>  tests/qtest/vhost-user-blk-test.c|   2 +-
>  util/vhost-user-server.c |  10 +-
>  block/export/meson.build |   1 +
>  block/meson.build|   1 -
>  8 files changed, 158 insertions(+), 360 deletions(-)
>
> diff --git a/qapi/block-export.json b/qapi/block-export.json
> index ace0d66e17..2e44625bb1 100644
> --- a/qapi/block-export.json
> +++ b/qapi/block-export.json
> @@ -84,6 +84,21 @@
>'data': { '*name': 'str', '*description': 'str',
>  '*bitmap': 'str' } }
>  
> +##
> +# @BlockExportOptionsVhostUserBlk:
> +#
> +# A vhost-user-blk block export.
> +#
> +# @addr: The vhost-user socket on which to listen. Both 'unix' and 'fd'
> +#SocketAddress types are supported. Passed fds must be UNIX domain
> +#sockets.

"addr.type must be 'unix' or 'fd'" is not visible in introspection.
Awkward.  Practical problem only if other addresses ever become
available here.  Is that possible?

> +# @logical-block-size: Logical block size in bytes. Defaults to 512 bytes.
> +#
> +# Since: 5.2
> +##
> +{ 'struct': 'BlockExportOptionsVhostUserBlk',
> +  'data': { 'addr': 'SocketAddress', '*logical-block-size': 'size' } }
> +
>  ##
>  # @NbdServerAddOptions:
>  #
> @@ -180,11 +195,12 @@
>  # An enumeration of block export types
>  #
>  # @nbd: NBD export
> +# @vhost-user-blk: vhost-user-blk export (since 5.2)
>  #
>  # Since: 4.2
>  ##
>  { 'enum': 'BlockExportType',
> -  'data': [ 'nbd' ] }
> +  'data': [ 'nbd', 'vhost-user-blk' ] }
>  
>  ##
>  # @BlockExportOptions:
> @@ -213,7 +229,8 @@
>  '*writethrough': 'bool' },
>'discriminator': 'type',
>'data': {
> -  'nbd': 'BlockExportOptionsNbd'
> +  'nbd': 'BlockExportOptionsNbd',
> +  'vhost-user-blk': 'BlockExportOptionsVhostUserBlk'
> } }
>  
>  ##
[...]

Re: [PATCH v5 09/14] hw/block/nvme: Support Zoned Namespace Command Set

[Klaus Jensen]

On Sep 28 12:42, Klaus Jensen wrote:
> On Sep 28 11:35, Dmitry Fomichev wrote:
> > The emulation code has been changed to advertise NVM Command Set when
> > "zoned" device property is not set (default) and Zoned Namespace
> > Command Set otherwise.
> > 
> > Handlers for three new NVMe commands introduced in Zoned Namespace
> > Command Set specification are added, namely for Zone Management
> > Receive, Zone Management Send and Zone Append.
> > 
> > Device initialization code has been extended to create a proper
> > configuration for zoned operation using device properties.
> > 
> > Read/Write command handler is modified to only allow writes at the
> > write pointer if the namespace is zoned. For Zone Append command,
> > writes implicitly happen at the write pointer and the starting write
> > pointer value is returned as the result of the command. Write Zeroes
> > handler is modified to add zoned checks that are identical to those
> > done as a part of Write flow.
> > 
> > The code to support for Zone Descriptor Extensions is not included in
> > this commit and ZDES 0 is always reported. A later commit in this
> > series will add ZDE support.
> > 
> > This commit doesn't yet include checks for active and open zone
> > limits. It is assumed that there are no limits on either active or
> > open zones.
> > 
> 
> I think the fill_pattern feature stands separate, so it would be nice to
> extract that to a patch on its own.
> 

Please disregard this.

Since the fill_pattern feature is tightly bound to reading in zones, it
doesnt really make sense to extract it.


signature.asc
Description: PGP signature

network buffering in fault tolerance

[Shivam Mehra]

I came across this documentation with source code for providing network
buffering to applications
https://www.nfradead.org/~tgr/libnl/doc/api/route_2qdisc_2plug_8c_source.html
.
This network-buffering helps output-commit problem when providing fault
tolerance to virtual machines. The output is buffered until an
acknowledgement arrives from the backup VM and then released to the
external world. So that backup and primary VMs seem consistent externally.
Initially developed for XEN VMM to provide fault tolerance to VMs and I
think it's now available for QEMU too.

Where does the script reside which does network-buffering for checkpoints?
and what are the commands to make this happen?

I want to do this network-buffering for packets originating from an
application. Is it possible to do it in the same way as above? Does it do
any damage to the host kernel? Can I get  a simple working example for this?

Re: [PATCH 16/16] qapi/expr.py: Use an expression checker dispatch table

[John Snow]

On 9/26/20 7:31 AM, Helio Loureiro wrote:



On Fri, Sep 25, 2020, 16:16 John Snow > wrote:


On 9/25/20 2:03 AM, Helio Loureiro wrote:
 > Hi,
 >
 > I would replace the word variable "kind" by "category".
 >

Hi, welcome to the list!


Tkz!


For patch reviews, we try to reply in-line, below the original post.


I realized that later.  It has been more than 20 years I don't use this 
formating.  But if I intend to join the pack, I need to follow the pack.




(For more information on the QAPI Schema Language that we are parsing
and validating here, see docs/devel/qapi-code-gen.txt if you are
curious. Ultimately it is a JSON-like format that permits multiple
objects per document and allows comments. We use these structures to
generate types and command interfaces for our API protocol, QMP.)


Based on that I would suggest 'type_ref' instead to match the 
definitions over there and since word 'type' itself is reserved.




One of the unsolvable problems in computer science is naming things: 
"TYPE-REF" also has a specific meaning in QAPI, as it is the name of one 
of the BNF grammar tokens we use.


So I might suggest (if "kind" is too ambiguous), that I might use 
"statement_type" or "expression_type" if that helps clarify things.


--js

Re: [PATCH] hw/arm: Restrict APEI tables generation to the 'virt' machine

[Dongjiu Geng]

On 2020/9/29 20:56, Philippe Mathieu-Daudé wrote:
> As only the Virt machine uses the RAS Virtualization feature (see
> commit 2afa8c8519: "hw/arm/virt: Introduce a RAS machine option"),
> restrict the APEI tables generation code to the virt machine.

APEI is a generic feature for X86 and arm64.  X86 platform also can use it, 
although currently it was mainly
used by ARM64. what is reason that we restrict APEI tables generation to the 
'virt' machine?

> 
> Fixes: aa16508f1d ("ACPI: Build related register address fields via hardware 
> error fw_cfg blob")
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
> Cc: Laszlo Ersek 
> Cc: Xiang Zheng 
> Cc: Jonathan Cameron 
> Cc: Igor Mammedov 
> Cc: Dongjiu Geng 
> Cc: Michael S. Tsirkin 
> ---
>  default-configs/arm-softmmu.mak | 1 -
>  hw/arm/Kconfig  | 1 +
>  2 files changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/default-configs/arm-softmmu.mak b/default-configs/arm-softmmu.mak
> index 9a94ebd0be..08a32123b4 100644
> --- a/default-configs/arm-softmmu.mak
> +++ b/default-configs/arm-softmmu.mak
> @@ -43,4 +43,3 @@ CONFIG_FSL_IMX7=y
>  CONFIG_FSL_IMX6UL=y
>  CONFIG_SEMIHOSTING=y
>  CONFIG_ALLWINNER_H3=y
> -CONFIG_ACPI_APEI=y
> diff --git a/hw/arm/Kconfig b/hw/arm/Kconfig
> index f303c6bead..7d040827af 100644
> --- a/hw/arm/Kconfig
> +++ b/hw/arm/Kconfig
> @@ -26,6 +26,7 @@ config ARM_VIRT
>  select ACPI_MEMORY_HOTPLUG
>  select ACPI_HW_REDUCED
>  select ACPI_NVDIMM
> +select ACPI_APEI
>  
>  config CHEETAH
>  bool
> 

Re: [PATCH 00/14] qapi: static typing conversion, pt3

[John Snow]

On 9/22/20 5:17 PM, John Snow wrote:

based-on: <20200922211313.4082880-1-js...@redhat.com>
   [PATCH 00/16] qapi: static typing conversion, pt2

Hi, this series adds static type hints to the QAPI module.
This is part three!

Part 3: https://gitlab.com/jsnow/qemu/-/tree/python-qapi-cleanup-pt3
Everything: https://gitlab.com/jsnow/qemu/-/tree/python-qapi-cleanup-pt6



This entire series is now deprecated, because of Peter Maydell's work 
removing doc.py!


There will not be a replacement. The series now goes from part 1, 2, ... 
4, 5, 6.


--js

[PATCH v4 43/46] qapi/types.py: remove one-letter variables

[John Snow]

"John, if pylint told you to jump off a bridge, would you?"
Hey, if it looked like fun, I might.

Now that this file is clean, enable pylint checks on this file.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
---
 scripts/qapi/pylintrc |  1 -
 scripts/qapi/types.py | 29 +++--
 2 files changed, 15 insertions(+), 15 deletions(-)

diff --git a/scripts/qapi/pylintrc b/scripts/qapi/pylintrc
index 8badcb11cda..b3c4cf46dbf 100644
--- a/scripts/qapi/pylintrc
+++ b/scripts/qapi/pylintrc
@@ -6,7 +6,6 @@ ignore-patterns=error.py,
 expr.py,
 parser.py,
 schema.py,
-types.py,
 visit.py,
 
 
diff --git a/scripts/qapi/types.py b/scripts/qapi/types.py
index 766822feb3a..9d1e79d503d 100644
--- a/scripts/qapi/types.py
+++ b/scripts/qapi/types.py
@@ -49,14 +49,14 @@ def gen_enum_lookup(name: str,
 .array = (const char *const[]) {
 ''',
 c_name=c_name(name))
-for m in members:
-ret += gen_if(m.ifcond)
-index = c_enum_const(name, m.name, prefix)
+for member in members:
+ret += gen_if(member.ifcond)
+index = c_enum_const(name, member.name, prefix)
 ret += mcgen('''
 [%(index)s] = "%(name)s",
 ''',
- index=index, name=m.name)
-ret += gen_endif(m.ifcond)
+ index=index, name=member.name)
+ret += gen_endif(member.ifcond)
 
 ret += mcgen('''
 },
@@ -79,13 +79,13 @@ def gen_enum(name: str,
 ''',
 c_name=c_name(name))
 
-for m in enum_members:
-ret += gen_if(m.ifcond)
+for member in enum_members:
+ret += gen_if(member.ifcond)
 ret += mcgen('''
 %(c_enum)s,
 ''',
- c_enum=c_enum_const(name, m.name, prefix))
-ret += gen_endif(m.ifcond)
+ c_enum=c_enum_const(name, member.name, prefix))
+ret += gen_endif(member.ifcond)
 
 ret += mcgen('''
 } %(c_name)s;
@@ -148,11 +148,12 @@ def gen_object(name: str, ifcond: List[str],
 objects_seen.add(name)
 
 ret = ''
-if variants:
-for v in variants.variants:
-if isinstance(v.type, QAPISchemaObjectType):
-ret += gen_object(v.type.name, v.type.ifcond, v.type.base,
-  v.type.local_members, v.type.variants)
+for variant in variants.variants if variants else ():
+obj = variant.type
+if not isinstance(obj, QAPISchemaObjectType):
+continue
+ret += gen_object(obj.name, obj.ifcond, obj.base,
+  obj.local_members, obj.variants)
 
 ret += mcgen('''
 
-- 
2.26.2

[PATCH v4 45/46] qapi/visit.py: remove unused parameters from gen_visit_object

[John Snow]

And this fixes the pylint report for this file, so make sure we check
this in the future, too.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
Tested-by: Cleber Rosa 
---
 scripts/qapi/pylintrc | 1 -
 scripts/qapi/visit.py | 4 ++--
 2 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/scripts/qapi/pylintrc b/scripts/qapi/pylintrc
index b3c4cf46dbf..b9e077a1642 100644
--- a/scripts/qapi/pylintrc
+++ b/scripts/qapi/pylintrc
@@ -6,7 +6,6 @@ ignore-patterns=error.py,
 expr.py,
 parser.py,
 schema.py,
-visit.py,
 
 
 [MESSAGES CONTROL]
diff --git a/scripts/qapi/visit.py b/scripts/qapi/visit.py
index 4f11fd325b8..e54694e23db 100644
--- a/scripts/qapi/visit.py
+++ b/scripts/qapi/visit.py
@@ -250,7 +250,7 @@ def gen_visit_alternate(name, variants):
 return ret
 
 
-def gen_visit_object(name, base, members, variants):
+def gen_visit_object(name):
 return mcgen('''
 
 bool visit_type_%(c_name)s
@@ -343,7 +343,7 @@ def visit_object_type(self, name, info, ifcond, features,
 if not name.startswith('q_'):
 # only explicit types need an allocating visit
 self._genh.add(gen_visit_decl(name))
-self._genc.add(gen_visit_object(name, base, members, variants))
+self._genc.add(gen_visit_object(name))
 
 def visit_alternate_type(self, name, info, ifcond, features, variants):
 with ifcontext(ifcond, self._genh, self._genc):
-- 
2.26.2

[PATCH v4 38/46] qapi/introspect.py: add _gen_features helper

[John Snow]

_make_tree might receive a dict or some other type. Adding features
information should arguably be performed by the caller at such a time
when we know the type of the object and don't have to re-interrogate it.

Signed-off-by: John Snow 
---
 scripts/qapi/introspect.py | 19 ---
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
index f7de3953391..5cbdc7414bd 100644
--- a/scripts/qapi/introspect.py
+++ b/scripts/qapi/introspect.py
@@ -52,16 +52,12 @@
 
 
 def _make_tree(obj: Union[_DObject, str], ifcond: List[str],
-   features: List[QAPISchemaFeature],
extra: Optional[Extra] = None
) -> Union[TreeNode, AnnotatedNode]:
 if extra is None:
 extra = {}
 if ifcond:
 extra['if'] = ifcond
-if features:
-assert isinstance(obj, dict)
-obj['features'] = [(f.name, {'if': f.ifcond}) for f in features]
 if extra:
 return (obj, extra)
 return obj
@@ -197,6 +193,11 @@ def _use_type(self, typ: QAPISchemaType) -> str:
 return '[' + self._use_type(typ.element_type) + ']'
 return self._name(typ.name)
 
+@classmethod
+def _gen_features(cls,
+  features: List[QAPISchemaFeature]) -> List[TreeNode]:
+return [_make_tree(f.name, f.ifcond) for f in features]
+
 def _gen_tree(self, name: str, mtype: str, obj: _DObject,
   ifcond: List[str],
   features: Optional[List[QAPISchemaFeature]]) -> None:
@@ -209,7 +210,9 @@ def _gen_tree(self, name: str, mtype: str, obj: _DObject,
 name = self._name(name)
 obj['name'] = name
 obj['meta-type'] = mtype
-self._trees.append(_make_tree(obj, ifcond, features, extra))
+if features:
+obj['features'] = self._gen_features(features)
+self._trees.append(_make_tree(obj, ifcond, extra))
 
 def _gen_member(self,
 member: QAPISchemaObjectTypeMember) -> TreeNode:
@@ -219,7 +222,9 @@ def _gen_member(self,
 }
 if member.optional:
 obj['default'] = None
-return _make_tree(obj, member.ifcond, member.features)
+if member.features:
+obj['features'] = self._gen_features(member.features)
+return _make_tree(obj, member.ifcond)
 
 def _gen_variants(self, tag_name: str,
   variants: List[QAPISchemaVariant]) -> _DObject:
@@ -231,7 +236,7 @@ def _gen_variant(self, variant: QAPISchemaVariant) -> 
TreeNode:
 'case': variant.name,
 'type': self._use_type(variant.type)
 }
-return _make_tree(obj, variant.ifcond, None)
+return _make_tree(obj, variant.ifcond)
 
 def visit_builtin_type(self, name: str, info: Optional[QAPISourceInfo],
json_type: str) -> None:
-- 
2.26.2

Re: [PATCH] pci: check bus pointer before dereference

[P J P]

[+Paolo, +Fam Zheng - for scsi]

+-- On Mon, 28 Sep 2020, P J P wrote --+
| +-- On Wed, 16 Sep 2020, Peter Maydell wrote --+
| | On Wed, 16 Sep 2020 at 07:28, P J P  wrote:
| | > -> 
https://ruhr-uni-bochum.sciebo.de/s/NNWP2GfwzYKeKwE?path=%2Flsi_nullptr1
| | > ==1183858==Hint: address points to the zero page.
| | > #0 pci_change_irq_level hw/pci/pci.c:259
| | > #1 pci_irq_handler hw/pci/pci.c:1445
| | > #2 pci_set_irq hw/pci/pci.c:1463
| | > #3 lsi_set_irq hw/scsi/lsi53c895a.c:488
| | > #4 lsi_update_irq hw/scsi/lsi53c895a.c:523
| | > #5 lsi_script_scsi_interrupt hw/scsi/lsi53c895a.c:554
| | > #6 lsi_execute_script hw/scsi/lsi53c895a.c:1149
| | > #7 lsi_reg_writeb hw/scsi/lsi53c895a.c:1984
| | > #8 lsi_io_write hw/scsi/lsi53c895a.c:2146
| ...
| | Generally we don't bother to assert() that pointers that shouldn't be NULL 
| | really are NULL immediately before dereferencing them, because the 
| | dereference provides an equally easy-to-debug crash to the assert, and so 
| | the assert doesn't provide anything extra. assert()ing that a pointer is 
| | non-NULL is more useful if it is done in a place that identifies the 
problem 
| | at an earlier and easier-to-debug point in execution rather than at a later 
| | point which is distantly removed from the place where the bogus pointer was 
| | introduced.
| 
| * The NULL dereference above occurs because the 'pci_dev->qdev->parent_bus' 
|   address gets overwritten (with 0x0) during scsi 'Memory Move' operation in
| 
|  ../hw/scsi/lsi53c895a.c
|   #define LSI_BUF_SIZE 4096
| 
| lsi_mmio_write
|  lsi_reg_writeb
|   lsi_execute_script
|static void lsi_memcpy(LSIState *s, ... int count=12MB)
|{
|   int n;
|   uint8_t buf[LSI_BUF_SIZE];
| 
|   while (count) {
| n = (count > LSI_BUF_SIZE) ? LSI_BUF_SIZE : count;
| lsi_mem_read(s, src, buf, n);  <== read from DMA memory
| lsi_mem_write(s, dest, buf, n);<== write to I/O memory
| src += n;
| dest += n;
| count -= n;
|  }
|}
| -> 
https://www.manualslib.com/manual/1407578/Lsi-Lsi53c895a.html?page=254#manual
| 
| * Above loop moves data between DMA memory to i/o address space.
| 
| * Going through the manual above, it seems 'Memory Move' can move upto 16MB 
of 
|   data between memory spaces.
| 
| * I tried to see a suitable fix, but couldn't get one.
| 
|   - Limiting 'count' value does not seem right, as allowed value is upto 16MB.
| 
|   - Manual above talks about moving data via 'dma_buf'. But it doesn't seem 
to 
| be used here.
| 
| * During above loop, 'dest' address moves past its 'MemoryRegion mr' and 
|   overwrites the adjacent 'mr' memory area, overwritting 'parent_bus' value.
| 
| Any thoughts/hints please...?

@Paolo, @Fam...wdyt?

Thank you.
--
Prasad J Pandit / Red Hat Product Security Team
8685 545E B54C 486B C6EB 271E E285 8B5A F050 DE8D

[PATCH v4 37/46] qapi/instrospect.py: add preliminary type hint annotations

[John Snow]

From: Eduardo Habkost 

The typing of _make_tree and friends is a bit involved, but it can be
done with some stubbed out types and a bit of elbow grease. The
forthcoming patches attempt to make some simplifications, but having the
type hints in advance can aid in review.

Signed-off-by: Eduardo Habkost 
Signed-off-by: John Snow 
---
 scripts/qapi/introspect.py | 133 +++--
 scripts/qapi/mypy.ini  |   5 --
 scripts/qapi/schema.py |   2 +-
 3 files changed, 98 insertions(+), 42 deletions(-)

diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
index 83140f2c564..f7de3953391 100644
--- a/scripts/qapi/introspect.py
+++ b/scripts/qapi/introspect.py
@@ -10,6 +10,15 @@
 See the COPYING file in the top-level directory.
 """
 
+from typing import (
+Dict,
+List,
+Optional,
+Sequence,
+Tuple,
+Union,
+)
+
 from .common import (
 c_name,
 gen_endif,
@@ -18,13 +27,34 @@
 )
 from .gen import QAPISchemaMonolithicCVisitor
 from .schema import (
+QAPISchema,
 QAPISchemaArrayType,
 QAPISchemaBuiltinType,
+QAPISchemaEntity,
+QAPISchemaEnumMember,
+QAPISchemaFeature,
+QAPISchemaObjectType,
+QAPISchemaObjectTypeMember,
 QAPISchemaType,
+QAPISchemaVariant,
+QAPISchemaVariants,
 )
+from .source import QAPISourceInfo
 
 
-def _make_tree(obj, ifcond, features, extra=None):
+# The correct type for TreeNode is actually:
+# Union[AnnotatedNode, List[TreeNode], Dict[str, TreeNode], str, bool]
+# but mypy does not support recursive types yet.
+TreeNode = object
+_DObject = Dict[str, object]
+Extra = Dict[str, object]
+AnnotatedNode = Tuple[TreeNode, Extra]
+
+
+def _make_tree(obj: Union[_DObject, str], ifcond: List[str],
+   features: List[QAPISchemaFeature],
+   extra: Optional[Extra] = None
+   ) -> Union[TreeNode, AnnotatedNode]:
 if extra is None:
 extra = {}
 if ifcond:
@@ -37,9 +67,11 @@ def _make_tree(obj, ifcond, features, extra=None):
 return obj
 
 
-def _tree_to_qlit(obj, level=0, suppress_first_indent=False):
+def _tree_to_qlit(obj: TreeNode,
+  level: int = 0,
+  suppress_first_indent: bool = False) -> str:
 
-def indent(level):
+def indent(level: int) -> str:
 return level * 4 * ' '
 
 if isinstance(obj, tuple):
@@ -89,21 +121,20 @@ def indent(level):
 return ret
 
 
-def to_c_string(string):
+def to_c_string(string: str) -> str:
 return '"' + string.replace('\\', r'\\').replace('"', r'\"') + '"'
 
 
 class QAPISchemaGenIntrospectVisitor(QAPISchemaMonolithicCVisitor):
-
-def __init__(self, prefix, unmask):
+def __init__(self, prefix: str, unmask: bool):
 super().__init__(
 prefix, 'qapi-introspect',
 ' * QAPI/QMP schema introspection', __doc__)
 self._unmask = unmask
-self._schema = None
-self._trees = []
-self._used_types = []
-self._name_map = {}
+self._schema: Optional[QAPISchema] = None
+self._trees: List[TreeNode] = []
+self._used_types: List[QAPISchemaType] = []
+self._name_map: Dict[str, str] = {}
 self._genc.add(mcgen('''
 #include "qemu/osdep.h"
 #include "%(prefix)sqapi-introspect.h"
@@ -111,10 +142,10 @@ def __init__(self, prefix, unmask):
 ''',
  prefix=prefix))
 
-def visit_begin(self, schema):
+def visit_begin(self, schema: QAPISchema) -> None:
 self._schema = schema
 
-def visit_end(self):
+def visit_end(self) -> None:
 # visit the types that are actually used
 for typ in self._used_types:
 typ.visit(self)
@@ -136,18 +167,18 @@ def visit_end(self):
 self._used_types = []
 self._name_map = {}
 
-def visit_needed(self, entity):
+def visit_needed(self, entity: QAPISchemaEntity) -> bool:
 # Ignore types on first pass; visit_end() will pick up used types
 return not isinstance(entity, QAPISchemaType)
 
-def _name(self, name):
+def _name(self, name: str) -> str:
 if self._unmask:
 return name
 if name not in self._name_map:
 self._name_map[name] = '%d' % len(self._name_map)
 return self._name_map[name]
 
-def _use_type(self, typ):
+def _use_type(self, typ: QAPISchemaType) -> str:
 # Map the various integer types to plain int
 if typ.json_type() == 'int':
 typ = self._schema.lookup_type('int')
@@ -166,8 +197,10 @@ def _use_type(self, typ):
 return '[' + self._use_type(typ.element_type) + ']'
 return self._name(typ.name)
 
-def _gen_tree(self, name, mtype, obj, ifcond, features):
-extra = None
+def _gen_tree(self, name: str, mtype: str, obj: _DObject,
+  ifcond: List[str],
+  features: Optional[List[QAPISchemaFeature]]) -> None:
+extra: Extra = None
 if mtype not 

[PATCH v4 35/46] qapi/gen.py: delint with pylint

[John Snow]

'fp' and 'fd' are self-evident in context, add them to the list of OK
names.

_top and _bottom also need to stay standard methods because some users
override the method and need to use `self`. Tell pylint to shush.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
---
 scripts/qapi/gen.py   | 2 ++
 scripts/qapi/pylintrc | 5 +++--
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/scripts/qapi/gen.py b/scripts/qapi/gen.py
index 2ad96e396e1..fc793552036 100644
--- a/scripts/qapi/gen.py
+++ b/scripts/qapi/gen.py
@@ -53,9 +53,11 @@ def get_content(self) -> str:
 return self._top() + self._preamble + self._body + self._bottom()
 
 def _top(self) -> str:
+# pylint: disable=no-self-use
 return ''
 
 def _bottom(self) -> str:
+# pylint: disable=no-self-use
 return ''
 
 def write(self, output_dir: str) -> None:
diff --git a/scripts/qapi/pylintrc b/scripts/qapi/pylintrc
index d840b150313..8badcb11cda 100644
--- a/scripts/qapi/pylintrc
+++ b/scripts/qapi/pylintrc
@@ -4,7 +4,6 @@
 # The regex matches against base names, not paths.
 ignore-patterns=error.py,
 expr.py,
-gen.py,
 parser.py,
 schema.py,
 types.py,
@@ -45,7 +44,9 @@ good-names=i,
k,
ex,
Run,
-   _
+   _,
+   fp,  # fp = open(...)
+   fd,  # fd = os.open(...)
 
 [VARIABLES]
 
-- 
2.26.2

[PATCH v4 39/46] qapi/introspect.py: Unify return type of _make_tree()

[John Snow]

Returning a *something* or a Tuple of *something* is hard to accurately
type. Let's just always return a tuple for structural consistency.

Instances of the 'TreeNode' type can be replaced with the slightly more
specific 'AnnotatedNode' type.

Signed-off-by: John Snow 
---
 scripts/qapi/introspect.py | 15 +++
 1 file changed, 7 insertions(+), 8 deletions(-)

diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
index 5cbdc7414bd..1c3ba41f1dc 100644
--- a/scripts/qapi/introspect.py
+++ b/scripts/qapi/introspect.py
@@ -53,14 +53,12 @@
 
 def _make_tree(obj: Union[_DObject, str], ifcond: List[str],
extra: Optional[Extra] = None
-   ) -> Union[TreeNode, AnnotatedNode]:
+   ) -> AnnotatedNode:
 if extra is None:
 extra = {}
 if ifcond:
 extra['if'] = ifcond
-if extra:
-return (obj, extra)
-return obj
+return (obj, extra)
 
 
 def _tree_to_qlit(obj: TreeNode,
@@ -128,7 +126,7 @@ def __init__(self, prefix: str, unmask: bool):
 ' * QAPI/QMP schema introspection', __doc__)
 self._unmask = unmask
 self._schema: Optional[QAPISchema] = None
-self._trees: List[TreeNode] = []
+self._trees: List[AnnotatedNode] = []
 self._used_types: List[QAPISchemaType] = []
 self._name_map: Dict[str, str] = {}
 self._genc.add(mcgen('''
@@ -195,7 +193,8 @@ def _use_type(self, typ: QAPISchemaType) -> str:
 
 @classmethod
 def _gen_features(cls,
-  features: List[QAPISchemaFeature]) -> List[TreeNode]:
+  features: List[QAPISchemaFeature]
+  ) -> List[AnnotatedNode]:
 return [_make_tree(f.name, f.ifcond) for f in features]
 
 def _gen_tree(self, name: str, mtype: str, obj: _DObject,
@@ -215,7 +214,7 @@ def _gen_tree(self, name: str, mtype: str, obj: _DObject,
 self._trees.append(_make_tree(obj, ifcond, extra))
 
 def _gen_member(self,
-member: QAPISchemaObjectTypeMember) -> TreeNode:
+member: QAPISchemaObjectTypeMember) -> AnnotatedNode:
 obj: _DObject = {
 'name': member.name,
 'type': self._use_type(member.type)
@@ -231,7 +230,7 @@ def _gen_variants(self, tag_name: str,
 return {'tag': tag_name,
 'variants': [self._gen_variant(v) for v in variants]}
 
-def _gen_variant(self, variant: QAPISchemaVariant) -> TreeNode:
+def _gen_variant(self, variant: QAPISchemaVariant) -> AnnotatedNode:
 obj: _DObject = {
 'case': variant.name,
 'type': self._use_type(variant.type)
-- 
2.26.2

[PATCH v4 44/46] qapi/visit.py: assert tag_member contains a QAPISchemaEnumType

[John Snow]

This is true by design, but not presently able to be expressed in the
type system. An assertion helps mypy understand our constraints.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
---
 scripts/qapi/visit.py | 12 +++-
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/scripts/qapi/visit.py b/scripts/qapi/visit.py
index 14f30c228b7..4f11fd325b8 100644
--- a/scripts/qapi/visit.py
+++ b/scripts/qapi/visit.py
@@ -22,7 +22,7 @@
 mcgen,
 )
 from .gen import QAPISchemaModularCVisitor, ifcontext
-from .schema import QAPISchemaObjectType
+from .schema import QAPISchemaEnumType, QAPISchemaObjectType
 
 
 def gen_visit_decl(name, scalar=False):
@@ -84,15 +84,17 @@ def gen_visit_object_members(name, base, members, variants):
 ret += gen_endif(memb.ifcond)
 
 if variants:
+tag_member = variants.tag_member
+assert isinstance(tag_member.type, QAPISchemaEnumType)
+
 ret += mcgen('''
 switch (obj->%(c_name)s) {
 ''',
- c_name=c_name(variants.tag_member.name))
+ c_name=c_name(tag_member.name))
 
 for var in variants.variants:
-case_str = c_enum_const(variants.tag_member.type.name,
-var.name,
-variants.tag_member.type.prefix)
+case_str = c_enum_const(tag_member.type.name, var.name,
+tag_member.type.prefix)
 ret += gen_if(var.ifcond)
 if var.type.name == 'q_empty':
 # valid variant and nothing to do
-- 
2.26.2

[PATCH v4 29/46] qapi/source.py: delint with pylint

[John Snow]

Shush an error and leave a hint for future cleanups when we're allowed
to use Python 3.7+.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
Tested-by: Cleber Rosa 
---
 scripts/qapi/pylintrc  | 1 -
 scripts/qapi/source.py | 3 +++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/scripts/qapi/pylintrc b/scripts/qapi/pylintrc
index 507f15537ab..d840b150313 100644
--- a/scripts/qapi/pylintrc
+++ b/scripts/qapi/pylintrc
@@ -7,7 +7,6 @@ ignore-patterns=error.py,
 gen.py,
 parser.py,
 schema.py,
-source.py,
 types.py,
 visit.py,
 
diff --git a/scripts/qapi/source.py b/scripts/qapi/source.py
index 1cc6a5b82dc..ba991d798fe 100644
--- a/scripts/qapi/source.py
+++ b/scripts/qapi/source.py
@@ -15,6 +15,9 @@
 
 
 class QAPISchemaPragma:
+# Replace with @dataclass in Python 3.7+
+# pylint: disable=too-few-public-methods
+
 def __init__(self) -> None:
 # Are documentation comments required?
 self.doc_required = False
-- 
2.26.2

[PATCH v4 42/46] qapi/types.py: add type hint annotations

[John Snow]

Annotations do not change runtime behavior.
This commit *only* adds annotations.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
---
 scripts/qapi/mypy.ini |  5 ---
 scripts/qapi/types.py | 86 ---
 2 files changed, 64 insertions(+), 27 deletions(-)

diff --git a/scripts/qapi/mypy.ini b/scripts/qapi/mypy.ini
index 5e5c305062e..eeb697c487c 100644
--- a/scripts/qapi/mypy.ini
+++ b/scripts/qapi/mypy.ini
@@ -24,11 +24,6 @@ disallow_untyped_defs = False
 disallow_incomplete_defs = False
 check_untyped_defs = False
 
-[mypy-qapi.types]
-disallow_untyped_defs = False
-disallow_incomplete_defs = False
-check_untyped_defs = False
-
 [mypy-qapi.visit]
 disallow_untyped_defs = False
 disallow_incomplete_defs = False
diff --git a/scripts/qapi/types.py b/scripts/qapi/types.py
index 53b47f9e58a..766822feb3a 100644
--- a/scripts/qapi/types.py
+++ b/scripts/qapi/types.py
@@ -13,6 +13,8 @@
 # See the COPYING file in the top-level directory.
 """
 
+from typing import List, Optional
+
 from .common import (
 c_enum_const,
 c_name,
@@ -21,7 +23,16 @@
 mcgen,
 )
 from .gen import QAPISchemaModularCVisitor, ifcontext
-from .schema import QAPISchemaEnumMember, QAPISchemaObjectType
+from .schema import (
+QAPISchema,
+QAPISchemaEnumMember,
+QAPISchemaFeature,
+QAPISchemaObjectType,
+QAPISchemaObjectTypeMember,
+QAPISchemaType,
+QAPISchemaVariants,
+)
+from .source import QAPISourceInfo
 
 
 # variants must be emitted before their container; track what has already
@@ -29,7 +40,9 @@
 objects_seen = set()
 
 
-def gen_enum_lookup(name, members, prefix=None):
+def gen_enum_lookup(name: str,
+members: List[QAPISchemaEnumMember],
+prefix: Optional[str] = None) -> str:
 ret = mcgen('''
 
 const QEnumLookup %(c_name)s_lookup = {
@@ -54,7 +67,9 @@ def gen_enum_lookup(name, members, prefix=None):
 return ret
 
 
-def gen_enum(name, members, prefix=None):
+def gen_enum(name: str,
+ members: List[QAPISchemaEnumMember],
+ prefix: Optional[str] = None) -> str:
 # append automatically generated _MAX value
 enum_members = members + [QAPISchemaEnumMember('_MAX', None)]
 
@@ -88,7 +103,7 @@ def gen_enum(name, members, prefix=None):
 return ret
 
 
-def gen_fwd_object_or_array(name):
+def gen_fwd_object_or_array(name: str) -> str:
 return mcgen('''
 
 typedef struct %(c_name)s %(c_name)s;
@@ -96,7 +111,7 @@ def gen_fwd_object_or_array(name):
  c_name=c_name(name))
 
 
-def gen_array(name, element_type):
+def gen_array(name: str, element_type: QAPISchemaType) -> str:
 return mcgen('''
 
 struct %(c_name)s {
@@ -107,7 +122,7 @@ def gen_array(name, element_type):
  c_name=c_name(name), c_type=element_type.c_type())
 
 
-def gen_struct_members(members):
+def gen_struct_members(members: List[QAPISchemaObjectTypeMember]) -> str:
 ret = ''
 for memb in members:
 ret += gen_if(memb.ifcond)
@@ -124,7 +139,10 @@ def gen_struct_members(members):
 return ret
 
 
-def gen_object(name, ifcond, base, members, variants):
+def gen_object(name: str, ifcond: List[str],
+   base: Optional[QAPISchemaObjectType],
+   members: List[QAPISchemaObjectTypeMember],
+   variants: Optional[QAPISchemaVariants]) -> str:
 if name in objects_seen:
 return ''
 objects_seen.add(name)
@@ -178,7 +196,7 @@ def gen_object(name, ifcond, base, members, variants):
 return ret
 
 
-def gen_upcast(name, base):
+def gen_upcast(name: str, base: QAPISchemaObjectType) -> str:
 # C makes const-correctness ugly.  We have to cast away const to let
 # this function work for both const and non-const obj.
 return mcgen('''
@@ -191,7 +209,7 @@ def gen_upcast(name, base):
  c_name=c_name(name), base=base.c_name())
 
 
-def gen_variants(variants):
+def gen_variants(variants: QAPISchemaVariants) -> str:
 ret = mcgen('''
 union { /* union tag is @%(c_name)s */
 ''',
@@ -215,7 +233,7 @@ def gen_variants(variants):
 return ret
 
 
-def gen_type_cleanup_decl(name):
+def gen_type_cleanup_decl(name: str) -> str:
 ret = mcgen('''
 
 void qapi_free_%(c_name)s(%(c_name)s *obj);
@@ -225,7 +243,7 @@ def gen_type_cleanup_decl(name):
 return ret
 
 
-def gen_type_cleanup(name):
+def gen_type_cleanup(name: str) -> str:
 ret = mcgen('''
 
 void qapi_free_%(c_name)s(%(c_name)s *obj)
@@ -247,12 +265,12 @@ def gen_type_cleanup(name):
 
 class QAPISchemaGenTypeVisitor(QAPISchemaModularCVisitor):
 
-def __init__(self, prefix):
+def __init__(self, prefix: str):
 super().__init__(
 prefix, 'qapi-types', ' * Schema-defined QAPI types',
 ' * Built-in QAPI types', __doc__)
 
-def _begin_system_module(self, name):
+def _begin_system_module(self, name: None) -> None:
 self._genc.preamble_add(mcgen('''
 #include 

[PATCH v4 41/46] qapi/introspect.py: create a typed 'Node' data structure

[John Snow]

This replaces _make_tree with Node.__init__(), effectively. By creating
it as a generic container, we can more accurately describe the exact
nature of this particular Node.

Signed-off-by: John Snow 
---
 scripts/qapi/introspect.py | 77 +++---
 1 file changed, 38 insertions(+), 39 deletions(-)

diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
index 43b6ba5df1f..86286e755ca 100644
--- a/scripts/qapi/introspect.py
+++ b/scripts/qapi/introspect.py
@@ -12,11 +12,12 @@
 
 from typing import (
 Dict,
+Generic,
+Iterable,
 List,
 Optional,
 Sequence,
-Tuple,
-Union,
+TypeVar,
 )
 
 from .common import (
@@ -43,42 +44,42 @@
 
 
 # The correct type for TreeNode is actually:
-# Union[AnnotatedNode, List[TreeNode], Dict[str, TreeNode], str, bool]
+# Union[Node[TreeNode], List[TreeNode], Dict[str, TreeNode], str, bool]
 # but mypy does not support recursive types yet.
 TreeNode = object
+_NodeType = TypeVar('_NodeType', bound=TreeNode)
 _DObject = Dict[str, object]
-Extra = Dict[str, object]
-AnnotatedNode = Tuple[TreeNode, Extra]
 
 
-def _make_tree(obj: Union[_DObject, str], ifcond: List[str],
-   comment: Optional[str] = None) -> AnnotatedNode:
-extra: Extra = {
-'if': ifcond,
-'comment': comment,
-}
-return (obj, extra)
+class Node(Generic[_NodeType]):
+"""
+Node generally contains a SchemaInfo-like type (as a dict),
+But it also used to wrap comments/ifconds around leaf value types.
+"""
+# Remove after 3.7 adds @dataclass:
+# pylint: disable=too-few-public-methods
+def __init__(self, data: _NodeType, ifcond: Iterable[str],
+ comment: Optional[str] = None):
+self.data = data
+self.comment: Optional[str] = comment
+self.ifcond: Sequence[str] = tuple(ifcond)
 
 
-def _tree_to_qlit(obj: TreeNode,
-  level: int = 0,
+def _tree_to_qlit(obj: TreeNode, level: int = 0,
   suppress_first_indent: bool = False) -> str:
 
 def indent(level: int) -> str:
 return level * 4 * ' '
 
-if isinstance(obj, tuple):
-ifobj, extra = obj
-ifcond = extra.get('if')
-comment = extra.get('comment')
+if isinstance(obj, Node):
 ret = ''
-if comment:
-ret += indent(level) + '/* %s */\n' % comment
-if ifcond:
-ret += gen_if(ifcond)
-ret += _tree_to_qlit(ifobj, level)
-if ifcond:
-ret += '\n' + gen_endif(ifcond)
+if obj.comment:
+ret += indent(level) + '/* %s */\n' % obj.comment
+if obj.ifcond:
+ret += gen_if(obj.ifcond)
+ret += _tree_to_qlit(obj.data, level)
+if obj.ifcond:
+ret += '\n' + gen_endif(obj.ifcond)
 return ret
 
 ret = ''
@@ -125,7 +126,7 @@ def __init__(self, prefix: str, unmask: bool):
 ' * QAPI/QMP schema introspection', __doc__)
 self._unmask = unmask
 self._schema: Optional[QAPISchema] = None
-self._trees: List[AnnotatedNode] = []
+self._trees: List[Node[_DObject]] = []
 self._used_types: List[QAPISchemaType] = []
 self._name_map: Dict[str, str] = {}
 self._genc.add(mcgen('''
@@ -192,9 +193,8 @@ def _use_type(self, typ: QAPISchemaType) -> str:
 
 @classmethod
 def _gen_features(cls,
-  features: List[QAPISchemaFeature]
-  ) -> List[AnnotatedNode]:
-return [_make_tree(f.name, f.ifcond) for f in features]
+  features: List[QAPISchemaFeature]) -> List[Node[str]]:
+return [Node(f.name, f.ifcond) for f in features]
 
 def _gen_tree(self, name: str, mtype: str, obj: _DObject,
   ifcond: List[str],
@@ -210,10 +210,10 @@ def _gen_tree(self, name: str, mtype: str, obj: _DObject,
 obj['meta-type'] = mtype
 if features:
 obj['features'] = self._gen_features(features)
-self._trees.append(_make_tree(obj, ifcond, comment))
+self._trees.append(Node(obj, ifcond, comment))
 
 def _gen_member(self,
-member: QAPISchemaObjectTypeMember) -> AnnotatedNode:
+member: QAPISchemaObjectTypeMember) -> Node[_DObject]:
 obj: _DObject = {
 'name': member.name,
 'type': self._use_type(member.type)
@@ -222,19 +222,19 @@ def _gen_member(self,
 obj['default'] = None
 if member.features:
 obj['features'] = self._gen_features(member.features)
-return _make_tree(obj, member.ifcond)
+return Node(obj, member.ifcond)
 
 def _gen_variants(self, tag_name: str,
   variants: List[QAPISchemaVariant]) -> _DObject:
 return {'tag': tag_name,
 'variants': [self._gen_variant(v) for v in variants]}
 
-def _gen_variant(self, variant: QAPISchemaVariant) -> AnnotatedNode:
+

[PATCH v4 30/46] qapi/gen.py: Fix edge-case of _is_user_module

[John Snow]

The edge case is that if the name is '', this expression returns a
string instead of a bool, which violates our declared type.

Signed-off-by: John Snow 
Reviewed-by: Cleber Rosa 
---
 scripts/qapi/gen.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/qapi/gen.py b/scripts/qapi/gen.py
index b8a6fe8a5c3..58cc1d8e734 100644
--- a/scripts/qapi/gen.py
+++ b/scripts/qapi/gen.py
@@ -243,7 +243,7 @@ def __init__(self, prefix, what, user_blurb, builtin_blurb, 
pydoc):
 
 @staticmethod
 def _is_user_module(name):
-return name and not name.startswith('./')
+return bool(name and not name.startswith('./'))
 
 @staticmethod
 def _is_builtin_module(name):
-- 
2.26.2

[PATCH v4 24/46] qapi/events.py: Move comments into docstrings

[John Snow]

Clarify them while we're here.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
---
 scripts/qapi/events.py | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/scripts/qapi/events.py b/scripts/qapi/events.py
index 57e0939e963..599f3d1f564 100644
--- a/scripts/qapi/events.py
+++ b/scripts/qapi/events.py
@@ -44,8 +44,12 @@ def gen_event_send_decl(name: str,
  proto=build_event_send_proto(name, arg_type, boxed))
 
 
-# Declare and initialize an object 'qapi' using parameters from build_params()
 def gen_param_var(typ: QAPISchemaObjectType) -> str:
+"""
+Generate a struct variable holding the event parameters.
+
+Initialize it with the function arguments defined in `gen_event_send`.
+"""
 assert not typ.variants
 ret = mcgen('''
 %(c_name)s param = {
-- 
2.26.2

[PATCH v4 40/46] qapi/introspect.py: replace 'extra' dict with 'comment' argument

[John Snow]

This is only used to pass in a dictionary with a comment already set, so
skip the runaround and just accept the comment.

Signed-off-by: John Snow 
---
 scripts/qapi/introspect.py | 17 -
 1 file changed, 8 insertions(+), 9 deletions(-)

diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
index 1c3ba41f1dc..43b6ba5df1f 100644
--- a/scripts/qapi/introspect.py
+++ b/scripts/qapi/introspect.py
@@ -52,12 +52,11 @@
 
 
 def _make_tree(obj: Union[_DObject, str], ifcond: List[str],
-   extra: Optional[Extra] = None
-   ) -> AnnotatedNode:
-if extra is None:
-extra = {}
-if ifcond:
-extra['if'] = ifcond
+   comment: Optional[str] = None) -> AnnotatedNode:
+extra: Extra = {
+'if': ifcond,
+'comment': comment,
+}
 return (obj, extra)
 
 
@@ -200,18 +199,18 @@ def _gen_features(cls,
 def _gen_tree(self, name: str, mtype: str, obj: _DObject,
   ifcond: List[str],
   features: Optional[List[QAPISchemaFeature]]) -> None:
-extra: Extra = None
+comment: Optional[str] = None
 if mtype not in ('command', 'event', 'builtin', 'array'):
 if not self._unmask:
 # Output a comment to make it easy to map masked names
 # back to the source when reading the generated output.
-extra = {'comment': '"%s" = %s' % (self._name(name), name)}
+comment = f'"{self._name(name)}" = {name}'
 name = self._name(name)
 obj['name'] = name
 obj['meta-type'] = mtype
 if features:
 obj['features'] = self._gen_features(features)
-self._trees.append(_make_tree(obj, ifcond, extra))
+self._trees.append(_make_tree(obj, ifcond, comment))
 
 def _gen_member(self,
 member: QAPISchemaObjectTypeMember) -> AnnotatedNode:
-- 
2.26.2

[PATCH v4 25/46] qapi/commands.py: Don't re-bind to variable of different type

[John Snow]

Mypy isn't a fan of rebinding a variable with a new data type.
It's easy enough to avoid.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
---
 scripts/qapi/commands.py | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/scripts/qapi/commands.py b/scripts/qapi/commands.py
index f67393f8713..586774a23c7 100644
--- a/scripts/qapi/commands.py
+++ b/scripts/qapi/commands.py
@@ -195,14 +195,12 @@ def gen_register_command(name, success_response, 
allow_oob, allow_preconfig):
 if not options:
 options = ['QCO_NO_OPTIONS']
 
-options = " | ".join(options)
-
 ret = mcgen('''
 qmp_register_command(cmds, "%(name)s",
  qmp_marshal_%(c_name)s, %(opts)s);
 ''',
 name=name, c_name=c_name(name),
-opts=options)
+opts=" | ".join(options))
 return ret
 
 
-- 
2.26.2

[PATCH v4 21/46] qapi/common.py: move build_params into gen.py

[John Snow]

Including it in common.py creates a circular import dependency; schema
relies on common, but common.build_params requires a type annotation
from schema. To type this properly, it needs to be moved outside the
cycle.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
---
 scripts/qapi/commands.py |  9 +++--
 scripts/qapi/common.py   | 23 ---
 scripts/qapi/events.py   |  9 ++---
 scripts/qapi/gen.py  | 31 +--
 4 files changed, 38 insertions(+), 34 deletions(-)

diff --git a/scripts/qapi/commands.py b/scripts/qapi/commands.py
index 5dc2f5a9fa8..f67393f8713 100644
--- a/scripts/qapi/commands.py
+++ b/scripts/qapi/commands.py
@@ -13,8 +13,13 @@
 See the COPYING file in the top-level directory.
 """
 
-from .common import build_params, c_name, mcgen
-from .gen import QAPIGenCCode, QAPISchemaModularCVisitor, ifcontext
+from .common import c_name, mcgen
+from .gen import (
+QAPIGenCCode,
+QAPISchemaModularCVisitor,
+build_params,
+ifcontext,
+)
 
 
 def gen_command_decl(name, arg_type, boxed, ret_type):
diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index 0ef38ea5fe0..9ab0685cc51 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -209,26 +209,3 @@ def gen_endif(ifcond: Sequence[str]) -> str:
 #endif /* %(cond)s */
 ''', cond=ifc)
 return ret
-
-
-def build_params(arg_type,
- boxed: bool,
- extra: Optional[str] = None) -> str:
-ret = ''
-sep = ''
-if boxed:
-assert arg_type
-ret += '%s arg' % arg_type.c_param_type()
-sep = ', '
-elif arg_type:
-assert not arg_type.variants
-for memb in arg_type.members:
-ret += sep
-sep = ', '
-if memb.optional:
-ret += 'bool has_%s, ' % c_name(memb.name)
-ret += '%s %s' % (memb.type.c_param_type(),
-  c_name(memb.name))
-if extra:
-ret += sep + extra
-return ret if ret else 'void'
diff --git a/scripts/qapi/events.py b/scripts/qapi/events.py
index 6b3afa14d72..f840a62ed92 100644
--- a/scripts/qapi/events.py
+++ b/scripts/qapi/events.py
@@ -12,13 +12,8 @@
 See the COPYING file in the top-level directory.
 """
 
-from .common import (
-build_params,
-c_enum_const,
-c_name,
-mcgen,
-)
-from .gen import QAPISchemaModularCVisitor, ifcontext
+from .common import c_enum_const, c_name, mcgen
+from .gen import QAPISchemaModularCVisitor, build_params, ifcontext
 from .schema import QAPISchemaEnumMember
 from .types import gen_enum, gen_enum_lookup
 
diff --git a/scripts/qapi/gen.py b/scripts/qapi/gen.py
index 61b3c53b180..b8a6fe8a5c3 100644
--- a/scripts/qapi/gen.py
+++ b/scripts/qapi/gen.py
@@ -2,9 +2,11 @@
 #
 # QAPI code generation
 #
-# Copyright (c) 2018-2019 Red Hat Inc.
+# Copyright IBM, Corp. 2011
+# Copyright (c) 2013-2019 Red Hat Inc.
 #
 # Authors:
+#  Anthony Liguori 
 #  Markus Armbruster 
 #  Marc-André Lureau 
 #
@@ -15,16 +17,18 @@
 import errno
 import os
 import re
+from typing import Optional
 
 from .common import (
 c_fname,
+c_name,
 gen_endif,
 gen_if,
 guardend,
 guardstart,
 mcgen,
 )
-from .schema import QAPISchemaVisitor
+from .schema import QAPISchemaObjectType, QAPISchemaVisitor
 
 
 class QAPIGen:
@@ -90,6 +94,29 @@ def _wrap_ifcond(ifcond, before, after):
 return out
 
 
+def build_params(arg_type: Optional[QAPISchemaObjectType],
+ boxed: bool,
+ extra: Optional[str] = None) -> str:
+ret = ''
+sep = ''
+if boxed:
+assert arg_type
+ret += '%s arg' % arg_type.c_param_type()
+sep = ', '
+elif arg_type:
+assert not arg_type.variants
+for memb in arg_type.members:
+ret += sep
+sep = ', '
+if memb.optional:
+ret += 'bool has_%s, ' % c_name(memb.name)
+ret += '%s %s' % (memb.type.c_param_type(),
+  c_name(memb.name))
+if extra:
+ret += sep + extra
+return ret if ret else 'void'
+
+
 class QAPIGenCCode(QAPIGen):
 
 def __init__(self, fname):
-- 
2.26.2

[PATCH v4 27/46] qapi/commands.py: enable checking with mypy

[John Snow]

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
---
 scripts/qapi/mypy.ini | 5 -
 1 file changed, 5 deletions(-)

diff --git a/scripts/qapi/mypy.ini b/scripts/qapi/mypy.ini
index 5df11e53fd1..8ab9ac52cc4 100644
--- a/scripts/qapi/mypy.ini
+++ b/scripts/qapi/mypy.ini
@@ -4,11 +4,6 @@ strict_optional = False
 disallow_untyped_calls = False
 python_version = 3.6
 
-[mypy-qapi.commands]
-disallow_untyped_defs = False
-disallow_incomplete_defs = False
-check_untyped_defs = False
-
 [mypy-qapi.error]
 disallow_untyped_defs = False
 disallow_incomplete_defs = False
-- 
2.26.2

[PATCH v4 36/46] qapi/introspect.py: assert obj is a dict when features are given

[John Snow]

This is necessary to keep mypy passing in the next patch when we add
preliminary type hints. It will be removed shortly.

Signed-off-by: John Snow 
---
 scripts/qapi/introspect.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
index 31acd2f230a..83140f2c564 100644
--- a/scripts/qapi/introspect.py
+++ b/scripts/qapi/introspect.py
@@ -30,6 +30,7 @@ def _make_tree(obj, ifcond, features, extra=None):
 if ifcond:
 extra['if'] = ifcond
 if features:
+assert isinstance(obj, dict)
 obj['features'] = [(f.name, {'if': f.ifcond}) for f in features]
 if extra:
 return (obj, extra)
-- 
2.26.2

Re: Contributor wanting to get started with simple contributions

[John Snow]

On 9/2/20 12:38 PM, Rohit Shinde wrote:

Hey John,

I wanted to follow up on this, in case you missed my previous email :)

Thanks,
Rohit.



Sorry Rohit, very buried in my work and haven't been doing a good job 
with my inbox.


Right now, I am working on converting the QAPI parser module to the 
strictly typed mypy subset:


https://lists.gnu.org/archive/html/qemu-devel/2020-09/msg09136.html

There are six parts in all, that is part one. I am posting a v4 right 
now as I write this.



Work that remains to be done after these six parts:

- Get ./python/qemu passing under mypy/pylint/flake8 again. I have old 
patches for this that were on-list prior to the 5.1 release.


- Move ./scripts/qapi to ./python/qemu/qapi/

- Move ./python/qemu to ./python/qemu/core/

- Create a "make check" style script that will run 
mypy/pylint/flake8/isort on all the code in ./python. (I have patches 
for this, too.)


- Start investigating python scripts in ./scripts and consider moving 
them to ./python/qemu/tools, fixing them up to pass 
mypy/flake8/pylint/isort (etc) as I go. There are no existing patches to 
do this yet.



Other work I am doing:

- Investigating the use of Pydantic to replace ./scripts/qapi/expr.py
- Investigating a YAML format for the QAPI parser
- Building a JSON-SCHEMA output format for the QAPI generator


On Sat, Aug 29, 2020 at 1:14 AM Rohit Shinde 
mailto:rohit.shinde12...@gmail.com>> wrote:


Hey John,

Sorry to bother you! I just wanted to know if you had any thoughts
on the mail I sent.

Is there anything I can pick up right now with regards to the Python
package? You mentioned that linting is something that needs to be
completed before we go ahead, so maybe I can start with that?

Thanks,
Rohit.

[PATCH v4 19/46] qapi/common.py: add type hint annotations

[John Snow]

Annotations do not change runtime behavior.
This commit *only* adds annotations.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
---
 scripts/qapi/common.py | 27 ---
 1 file changed, 16 insertions(+), 11 deletions(-)

diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index 338adedef4f..74a2c001ed9 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -12,6 +12,7 @@
 # See the COPYING file in the top-level directory.
 
 import re
+from typing import Optional, Sequence
 
 
 EATSPACE = '\033EATSPACE.'
@@ -22,7 +23,7 @@
 # ENUMName -> ENUM_NAME, EnumName1 -> ENUM_NAME1
 # ENUM_NAME -> ENUM_NAME, ENUM_NAME1 -> ENUM_NAME1, ENUM_Name2 -> ENUM_NAME2
 # ENUM24_Name -> ENUM24_NAME
-def camel_to_upper(value):
+def camel_to_upper(value: str) -> str:
 c_fun_str = c_name(value, False)
 if value.isupper():
 return c_fun_str
@@ -41,7 +42,9 @@ def camel_to_upper(value):
 return new_name.lstrip('_').upper()
 
 
-def c_enum_const(type_name, const_name, prefix=None):
+def c_enum_const(type_name: str,
+ const_name: str,
+ prefix: Optional[str] = None) -> str:
 if prefix is not None:
 type_name = prefix
 return camel_to_upper(type_name) + '_' + c_name(const_name, False).upper()
@@ -56,7 +59,7 @@ def c_enum_const(type_name, const_name, prefix=None):
 # into substrings of a generated C function name.
 # '__a.b_c' -> '__a_b_c', 'x-foo' -> 'x_foo'
 # protect=True: 'int' -> 'q_int'; protect=False: 'int' -> 'int'
-def c_name(name, protect=True):
+def c_name(name: str, protect: bool = True) -> str:
 # ANSI X3J11/88-090, 3.1.1
 c89_words = set(['auto', 'break', 'case', 'char', 'const', 'continue',
  'default', 'do', 'double', 'else', 'enum', 'extern',
@@ -131,24 +134,24 @@ def decrease(self, amount: int = 4) -> None:
 
 # Generate @code with @kwds interpolated.
 # Obey indent, and strip EATSPACE.
-def cgen(code, **kwds):
+def cgen(code: str, **kwds: object) -> str:
 raw = code % kwds
 if indent:
 raw = re.sub(r'^(?!(#|$))', str(indent), raw, flags=re.MULTILINE)
 return re.sub(re.escape(EATSPACE) + r' *', '', raw)
 
 
-def mcgen(code, **kwds):
+def mcgen(code: str, **kwds: object) -> str:
 if code[0] == '\n':
 code = code[1:]
 return cgen(code, **kwds)
 
 
-def c_fname(filename):
+def c_fname(filename: str) -> str:
 return re.sub(r'[^A-Za-z0-9_]', '_', filename)
 
 
-def guardstart(name):
+def guardstart(name: str) -> str:
 return mcgen('''
 #ifndef %(name)s
 #define %(name)s
@@ -157,7 +160,7 @@ def guardstart(name):
  name=c_fname(name).upper())
 
 
-def guardend(name):
+def guardend(name: str) -> str:
 return mcgen('''
 
 #endif /* %(name)s */
@@ -165,7 +168,7 @@ def guardend(name):
  name=c_fname(name).upper())
 
 
-def gen_if(ifcond):
+def gen_if(ifcond: Sequence[str]) -> str:
 ret = ''
 for ifc in ifcond:
 ret += mcgen('''
@@ -174,7 +177,7 @@ def gen_if(ifcond):
 return ret
 
 
-def gen_endif(ifcond):
+def gen_endif(ifcond: Sequence[str]) -> str:
 ret = ''
 for ifc in reversed(ifcond):
 ret += mcgen('''
@@ -183,7 +186,9 @@ def gen_endif(ifcond):
 return ret
 
 
-def build_params(arg_type, boxed, extra=None):
+def build_params(arg_type,
+ boxed: bool,
+ extra: Optional[str] = None) -> str:
 ret = ''
 sep = ''
 if boxed:
-- 
2.26.2

[PATCH v4 26/46] qapi/commands.py: add type hint annotations

[John Snow]

Annotations do not change runtime behavior.
This commit *only* adds annotations.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
---
 scripts/qapi/commands.py | 71 ++--
 1 file changed, 54 insertions(+), 17 deletions(-)

diff --git a/scripts/qapi/commands.py b/scripts/qapi/commands.py
index 586774a23c7..24f347a3f12 100644
--- a/scripts/qapi/commands.py
+++ b/scripts/qapi/commands.py
@@ -13,16 +13,34 @@
 See the COPYING file in the top-level directory.
 """
 
+from typing import (
+Dict,
+List,
+Optional,
+Set,
+)
+
 from .common import c_name, mcgen
 from .gen import (
+QAPIGenC,
 QAPIGenCCode,
 QAPISchemaModularCVisitor,
 build_params,
 ifcontext,
 )
+from .schema import (
+QAPISchema,
+QAPISchemaFeature,
+QAPISchemaObjectType,
+QAPISchemaType,
+)
+from .source import QAPISourceInfo
 
 
-def gen_command_decl(name, arg_type, boxed, ret_type):
+def gen_command_decl(name: str,
+ arg_type: Optional[QAPISchemaObjectType],
+ boxed: bool,
+ ret_type: Optional[QAPISchemaType]) -> str:
 return mcgen('''
 %(c_type)s qmp_%(c_name)s(%(params)s);
 ''',
@@ -31,7 +49,10 @@ def gen_command_decl(name, arg_type, boxed, ret_type):
  params=build_params(arg_type, boxed, 'Error **errp'))
 
 
-def gen_call(name, arg_type, boxed, ret_type):
+def gen_call(name: str,
+ arg_type: Optional[QAPISchemaObjectType],
+ boxed: bool,
+ ret_type: Optional[QAPISchemaType]) -> str:
 ret = ''
 
 argstr = ''
@@ -67,7 +88,7 @@ def gen_call(name, arg_type, boxed, ret_type):
 return ret
 
 
-def gen_marshal_output(ret_type):
+def gen_marshal_output(ret_type: QAPISchemaType) -> str:
 return mcgen('''
 
 static void qmp_marshal_output_%(c_name)s(%(c_type)s ret_in,
@@ -88,19 +109,22 @@ def gen_marshal_output(ret_type):
  c_type=ret_type.c_type(), c_name=ret_type.c_name())
 
 
-def build_marshal_proto(name):
+def build_marshal_proto(name: str) -> str:
 return ('void qmp_marshal_%s(QDict *args, QObject **ret, Error **errp)'
 % c_name(name))
 
 
-def gen_marshal_decl(name):
+def gen_marshal_decl(name: str) -> str:
 return mcgen('''
 %(proto)s;
 ''',
  proto=build_marshal_proto(name))
 
 
-def gen_marshal(name, arg_type, boxed, ret_type):
+def gen_marshal(name: str,
+arg_type: Optional[QAPISchemaObjectType],
+boxed: bool,
+ret_type: Optional[QAPISchemaType]) -> str:
 have_args = boxed or (arg_type and not arg_type.is_empty())
 
 ret = mcgen('''
@@ -182,7 +206,10 @@ def gen_marshal(name, arg_type, boxed, ret_type):
 return ret
 
 
-def gen_register_command(name, success_response, allow_oob, allow_preconfig):
+def gen_register_command(name: str,
+ success_response: bool,
+ allow_oob: bool,
+ allow_preconfig: bool) -> str:
 options = []
 
 if not success_response:
@@ -204,7 +231,7 @@ def gen_register_command(name, success_response, allow_oob, 
allow_preconfig):
 return ret
 
 
-def gen_registry(registry, prefix):
+def gen_registry(registry: str, prefix: str) -> str:
 ret = mcgen('''
 
 void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds)
@@ -221,15 +248,14 @@ def gen_registry(registry, prefix):
 
 
 class QAPISchemaGenCommandVisitor(QAPISchemaModularCVisitor):
-
-def __init__(self, prefix):
+def __init__(self, prefix: str):
 super().__init__(
 prefix, 'qapi-commands',
 ' * Schema-defined QAPI/QMP commands', None, __doc__)
 self._regy = QAPIGenCCode(None)
-self._visited_ret_types = {}
+self._visited_ret_types: Dict[QAPIGenC, Set[QAPISchemaType]] = {}
 
-def _begin_user_module(self, name):
+def _begin_user_module(self, name: str) -> None:
 self._visited_ret_types[self._genc] = set()
 commands = self._module_basename('qapi-commands', name)
 types = self._module_basename('qapi-types', name)
@@ -253,7 +279,7 @@ def _begin_user_module(self, name):
 ''',
  types=types))
 
-def visit_end(self):
+def visit_end(self) -> None:
 self._add_system_module('init', ' * QAPI Commands initialization')
 self._genh.add(mcgen('''
 #include "qapi/qmp/dispatch.h"
@@ -269,9 +295,18 @@ def visit_end(self):
   prefix=self._prefix))
 self._genc.add(gen_registry(self._regy.get_content(), self._prefix))
 
-def visit_command(self, name, info, ifcond, features,
-  arg_type, ret_type, gen, success_response, boxed,
-  allow_oob, allow_preconfig):
+def visit_command(self,
+  name: str,
+  info: QAPISourceInfo,
+  ifcond: List[str],
+  

[PATCH v4 16/46] qapi/common.py: delint with pylint

[John Snow]

At this point, that just means using a consistent strategy for constant names.
constants get UPPER_CASE and names not used externally get a leading underscore.

As a preference, while renaming constants to be UPPERCASE, move them to
the head of the file. Generally, it's nice to be able to audit the code
that runs on import in one central place.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
---
 scripts/qapi/common.py | 18 --
 scripts/qapi/schema.py | 14 +++---
 2 files changed, 15 insertions(+), 17 deletions(-)

diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index b35318b72cf..a417b6029c8 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -14,6 +14,11 @@
 import re
 
 
+EATSPACE = '\033EATSPACE.'
+POINTER_SUFFIX = ' *' + EATSPACE
+_C_NAME_TRANS = str.maketrans('.-', '__')
+
+
 # ENUMName -> ENUM_NAME, EnumName1 -> ENUM_NAME1
 # ENUM_NAME -> ENUM_NAME, ENUM_NAME1 -> ENUM_NAME1, ENUM_Name2 -> ENUM_NAME2
 # ENUM24_Name -> ENUM24_NAME
@@ -42,9 +47,6 @@ def c_enum_const(type_name, const_name, prefix=None):
 return camel_to_upper(type_name) + '_' + c_name(const_name, False).upper()
 
 
-c_name_trans = str.maketrans('.-', '__')
-
-
 # Map @name to a valid C identifier.
 # If @protect, avoid returning certain ticklish identifiers (like
 # C keywords) by prepending 'q_'.
@@ -82,17 +84,13 @@ def c_name(name, protect=True):
  'not_eq', 'or', 'or_eq', 'xor', 'xor_eq'])
 # namespace pollution:
 polluted_words = set(['unix', 'errno', 'mips', 'sparc', 'i386'])
-name = name.translate(c_name_trans)
+name = name.translate(_C_NAME_TRANS)
 if protect and (name in c89_words | c99_words | c11_words | gcc_words
 | cpp_words | polluted_words):
 return 'q_' + name
 return name
 
 
-eatspace = '\033EATSPACE.'
-pointer_suffix = ' *' + eatspace
-
-
 class Indentation:
 """
 Indentation level management.
@@ -132,12 +130,12 @@ def decrease(self, amount: int = 4) -> None:
 
 
 # Generate @code with @kwds interpolated.
-# Obey indent, and strip eatspace.
+# Obey indent, and strip EATSPACE.
 def cgen(code, **kwds):
 raw = code % kwds
 if indent:
 raw = re.sub(r'^(?!(#|$))', str(indent), raw, flags=re.MULTILINE)
-return re.sub(re.escape(eatspace) + r' *', '', raw)
+return re.sub(re.escape(EATSPACE) + r' *', '', raw)
 
 
 def mcgen(code, **kwds):
diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
index cfc52e1ae44..74c6b96d391 100644
--- a/scripts/qapi/schema.py
+++ b/scripts/qapi/schema.py
@@ -18,7 +18,7 @@
 import os
 import re
 
-from .common import c_name, pointer_suffix
+from .common import POINTER_SUFFIX, c_name
 from .error import QAPIError, QAPISemError
 from .expr import check_exprs
 from .parser import QAPISchemaParser
@@ -309,7 +309,7 @@ def is_implicit(self):
 return True
 
 def c_type(self):
-return c_name(self.name) + pointer_suffix
+return c_name(self.name) + POINTER_SUFFIX
 
 def json_type(self):
 return 'array'
@@ -430,7 +430,7 @@ def c_name(self):
 
 def c_type(self):
 assert not self.is_implicit()
-return c_name(self.name) + pointer_suffix
+return c_name(self.name) + POINTER_SUFFIX
 
 def c_unboxed_type(self):
 return c_name(self.name)
@@ -504,7 +504,7 @@ def connect_doc(self, doc=None):
 v.connect_doc(doc)
 
 def c_type(self):
-return c_name(self.name) + pointer_suffix
+return c_name(self.name) + POINTER_SUFFIX
 
 def json_type(self):
 return 'value'
@@ -896,7 +896,7 @@ def _def_builtin_type(self, name, json_type, c_type):
 self._make_array_type(name, None)
 
 def _def_predefineds(self):
-for t in [('str','string',  'char' + pointer_suffix),
+for t in [('str','string',  'char' + POINTER_SUFFIX),
   ('number', 'number',  'double'),
   ('int','int', 'int64_t'),
   ('int8',   'int', 'int8_t'),
@@ -909,8 +909,8 @@ def _def_predefineds(self):
   ('uint64', 'int', 'uint64_t'),
   ('size',   'int', 'uint64_t'),
   ('bool',   'boolean', 'bool'),
-  ('any','value',   'QObject' + pointer_suffix),
-  ('null',   'null','QNull' + pointer_suffix)]:
+  ('any','value',   'QObject' + POINTER_SUFFIX),
+  ('null',   'null','QNull' + POINTER_SUFFIX)]:
 self._def_builtin_type(*t)
 self.the_empty_object_type = QAPISchemaObjectType(
 'q_empty', None, None, None, None, None, [], None)
-- 
2.26.2

[PATCH v4 34/46] qapi/gen.py: update write() to be more idiomatic

[John Snow]

Make the file handling here just a tiny bit more idiomatic.
(I realize this is heavily subjective.)

Use exist_ok=True for os.makedirs and remove the exception,
use fdopen() to wrap the file descriptor in a File-like object,
and use a context manager for managing the file pointer.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
---
 scripts/qapi/gen.py | 25 +++--
 1 file changed, 11 insertions(+), 14 deletions(-)

diff --git a/scripts/qapi/gen.py b/scripts/qapi/gen.py
index 5694a00062c..2ad96e396e1 100644
--- a/scripts/qapi/gen.py
+++ b/scripts/qapi/gen.py
@@ -14,7 +14,6 @@
 # See the COPYING file in the top-level directory.
 
 from contextlib import contextmanager
-import errno
 import os
 import re
 from typing import (
@@ -67,21 +66,19 @@ def write(self, output_dir: str) -> None:
 return
 pathname = os.path.join(output_dir, self.fname)
 odir = os.path.dirname(pathname)
+
 if odir:
-try:
-os.makedirs(odir)
-except os.error as e:
-if e.errno != errno.EEXIST:
-raise
+os.makedirs(odir, exist_ok=True)
+
+# use os.open for O_CREAT to create and read a non-existant file
 fd = os.open(pathname, os.O_RDWR | os.O_CREAT, 0o666)
-f = open(fd, 'r+', encoding='utf-8')
-text = self.get_content()
-oldtext = f.read(len(text) + 1)
-if text != oldtext:
-f.seek(0)
-f.truncate(0)
-f.write(text)
-f.close()
+with os.fdopen(fd, 'r+', encoding='utf-8') as fp:
+text = self.get_content()
+oldtext = fp.read(len(text) + 1)
+if text != oldtext:
+fp.seek(0)
+fp.truncate(0)
+fp.write(text)
 
 
 def _wrap_ifcond(ifcond: List[str], before: str, after: str) -> str:
-- 
2.26.2

[PATCH v4 33/46] qapi/gen.py: Remove unused parameter

[John Snow]

module_basename doesn't use the 'what' argument, so remove it.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
---
 scripts/qapi/gen.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/qapi/gen.py b/scripts/qapi/gen.py
index 4c6bba940ed..5694a00062c 100644
--- a/scripts/qapi/gen.py
+++ b/scripts/qapi/gen.py
@@ -259,7 +259,7 @@ def _is_user_module(name: Optional[str]) -> bool:
 def _is_builtin_module(name: Optional[str]) -> bool:
 return not name
 
-def _module_dirname(self, what: str, name: Optional[str]) -> str:
+def _module_dirname(self, name: Optional[str]) -> str:
 if self._is_user_module(name):
 return os.path.dirname(name)
 return ''
@@ -277,7 +277,7 @@ def _module_basename(self, what: str, name: Optional[str]) 
-> str:
 return ret
 
 def _module_filename(self, what: str, name: Optional[str]) -> str:
-return os.path.join(self._module_dirname(what, name),
+return os.path.join(self._module_dirname(name),
 self._module_basename(what, name))
 
 def _add_module(self, name: Optional[str], blurb: str) -> None:
-- 
2.26.2

[PATCH v4 23/46] qapi/events.py: add type hint annotations

[John Snow]

Annotations do not change runtime behavior.
This commit *only* adds annotations.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
---
 scripts/qapi/events.py | 46 --
 scripts/qapi/mypy.ini  |  5 -
 2 files changed, 35 insertions(+), 16 deletions(-)

diff --git a/scripts/qapi/events.py b/scripts/qapi/events.py
index f840a62ed92..57e0939e963 100644
--- a/scripts/qapi/events.py
+++ b/scripts/qapi/events.py
@@ -12,19 +12,31 @@
 See the COPYING file in the top-level directory.
 """
 
+from typing import List
+
 from .common import c_enum_const, c_name, mcgen
 from .gen import QAPISchemaModularCVisitor, build_params, ifcontext
-from .schema import QAPISchemaEnumMember
+from .schema import (
+QAPISchema,
+QAPISchemaEnumMember,
+QAPISchemaFeature,
+QAPISchemaObjectType,
+)
+from .source import QAPISourceInfo
 from .types import gen_enum, gen_enum_lookup
 
 
-def build_event_send_proto(name, arg_type, boxed):
+def build_event_send_proto(name: str,
+   arg_type: QAPISchemaObjectType,
+   boxed: bool) -> str:
 return 'void qapi_event_send_%(c_name)s(%(param)s)' % {
 'c_name': c_name(name.lower()),
 'param': build_params(arg_type, boxed)}
 
 
-def gen_event_send_decl(name, arg_type, boxed):
+def gen_event_send_decl(name: str,
+arg_type: QAPISchemaObjectType,
+boxed: bool) -> str:
 return mcgen('''
 
 %(proto)s;
@@ -33,7 +45,7 @@ def gen_event_send_decl(name, arg_type, boxed):
 
 
 # Declare and initialize an object 'qapi' using parameters from build_params()
-def gen_param_var(typ):
+def gen_param_var(typ: QAPISchemaObjectType) -> str:
 assert not typ.variants
 ret = mcgen('''
 %(c_name)s param = {
@@ -61,7 +73,11 @@ def gen_param_var(typ):
 return ret
 
 
-def gen_event_send(name, arg_type, boxed, event_enum_name, event_emit):
+def gen_event_send(name: str,
+   arg_type: QAPISchemaObjectType,
+   boxed: bool,
+   event_enum_name: str,
+   event_emit: str) -> str:
 # FIXME: Our declaration of local variables (and of 'errp' in the
 # parameter list) can collide with exploded members of the event's
 # data type passed in as parameters.  If this collision ever hits in
@@ -137,15 +153,15 @@ def gen_event_send(name, arg_type, boxed, 
event_enum_name, event_emit):
 
 class QAPISchemaGenEventVisitor(QAPISchemaModularCVisitor):
 
-def __init__(self, prefix):
+def __init__(self, prefix: str):
 super().__init__(
 prefix, 'qapi-events',
 ' * Schema-defined QAPI/QMP events', None, __doc__)
 self._event_enum_name = c_name(prefix + 'QAPIEvent', protect=False)
-self._event_enum_members = []
+self._event_enum_members: List[QAPISchemaEnumMember] = []
 self._event_emit_name = c_name(prefix + 'qapi_event_emit')
 
-def _begin_user_module(self, name):
+def _begin_user_module(self, name: str) -> None:
 events = self._module_basename('qapi-events', name)
 types = self._module_basename('qapi-types', name)
 visit = self._module_basename('qapi-visit', name)
@@ -168,7 +184,7 @@ def _begin_user_module(self, name):
 ''',
  types=types))
 
-def visit_end(self):
+def visit_end(self) -> None:
 self._add_system_module('emit', ' * QAPI Events emission')
 self._genc.preamble_add(mcgen('''
 #include "qemu/osdep.h"
@@ -189,7 +205,13 @@ def visit_end(self):
  event_emit=self._event_emit_name,
  event_enum=self._event_enum_name))
 
-def visit_event(self, name, info, ifcond, features, arg_type, boxed):
+def visit_event(self,
+name: str,
+info: QAPISourceInfo,
+ifcond: List[str],
+features: List[QAPISchemaFeature],
+arg_type: QAPISchemaObjectType,
+boxed: bool) -> None:
 with ifcontext(ifcond, self._genh, self._genc):
 self._genh.add(gen_event_send_decl(name, arg_type, boxed))
 self._genc.add(gen_event_send(name, arg_type, boxed,
@@ -200,7 +222,9 @@ def visit_event(self, name, info, ifcond, features, 
arg_type, boxed):
 self._event_enum_members.append(QAPISchemaEnumMember(name, None))
 
 
-def gen_events(schema, output_dir, prefix):
+def gen_events(schema: QAPISchema,
+   output_dir: str,
+   prefix: str) -> None:
 vis = QAPISchemaGenEventVisitor(prefix)
 schema.visit(vis)
 vis.write(output_dir)
diff --git a/scripts/qapi/mypy.ini b/scripts/qapi/mypy.ini
index 00fac15dc6e..5df11e53fd1 100644
--- a/scripts/qapi/mypy.ini
+++ b/scripts/qapi/mypy.ini
@@ -14,11 +14,6 @@ disallow_untyped_defs = False
 disallow_incomplete_defs = False
 check_untyped_defs = 

[PATCH v4 14/46] qapi/common.py: Remove python compatibility workaround

[John Snow]

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
---
 scripts/qapi/common.py | 5 +
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index ba35abea478..cee63eb95c7 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -119,10 +119,7 @@ def cgen(code, **kwds):
 raw = code % kwds
 if indent_level:
 indent = genindent(indent_level)
-# re.subn() lacks flags support before Python 2.7, use re.compile()
-raw = re.subn(re.compile(r'^(?!(#|$))', re.MULTILINE),
-  indent, raw)
-raw = raw[0]
+raw = re.sub(r'^(?!(#|$))', indent, raw, flags=re.MULTILINE)
 return re.sub(re.escape(eatspace) + r' *', '', raw)
 
 
-- 
2.26.2

[PATCH v4 32/46] qapi/gen.py: Enable checking with mypy

[John Snow]

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
Tested-by: Cleber Rosa 
---
 scripts/qapi/mypy.ini | 5 -
 1 file changed, 5 deletions(-)

diff --git a/scripts/qapi/mypy.ini b/scripts/qapi/mypy.ini
index 1b8555dfa39..c6960ff2dbd 100644
--- a/scripts/qapi/mypy.ini
+++ b/scripts/qapi/mypy.ini
@@ -14,11 +14,6 @@ disallow_untyped_defs = False
 disallow_incomplete_defs = False
 check_untyped_defs = False
 
-[mypy-qapi.gen]
-disallow_untyped_defs = False
-disallow_incomplete_defs = False
-check_untyped_defs = False
-
 [mypy-qapi.introspect]
 disallow_untyped_defs = False
 disallow_incomplete_defs = False
-- 
2.26.2

[PATCH v4 22/46] qapi: establish mypy type-checking baseline

[John Snow]

Fix two very minor issues, and then establish a mypy type-checking
baseline.

Like pylint, this should be run from the folder above:

 > mypy --config-file=qapi/mypy.ini qapi/

This is designed and tested for mypy 0.770 or greater.

Signed-off-by: John Snow 
Tested-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
Tested-by: Cleber Rosa 
---
 scripts/qapi/mypy.ini  | 60 ++
 scripts/qapi/schema.py |  3 ++-
 2 files changed, 62 insertions(+), 1 deletion(-)
 create mode 100644 scripts/qapi/mypy.ini

diff --git a/scripts/qapi/mypy.ini b/scripts/qapi/mypy.ini
new file mode 100644
index 000..00fac15dc6e
--- /dev/null
+++ b/scripts/qapi/mypy.ini
@@ -0,0 +1,60 @@
+[mypy]
+strict = True
+strict_optional = False
+disallow_untyped_calls = False
+python_version = 3.6
+
+[mypy-qapi.commands]
+disallow_untyped_defs = False
+disallow_incomplete_defs = False
+check_untyped_defs = False
+
+[mypy-qapi.error]
+disallow_untyped_defs = False
+disallow_incomplete_defs = False
+check_untyped_defs = False
+
+[mypy-qapi.events]
+disallow_untyped_defs = False
+disallow_incomplete_defs = False
+check_untyped_defs = False
+
+[mypy-qapi.expr]
+disallow_untyped_defs = False
+disallow_incomplete_defs = False
+check_untyped_defs = False
+
+[mypy-qapi.gen]
+disallow_untyped_defs = False
+disallow_incomplete_defs = False
+check_untyped_defs = False
+
+[mypy-qapi.introspect]
+disallow_untyped_defs = False
+disallow_incomplete_defs = False
+check_untyped_defs = False
+
+[mypy-qapi.parser]
+disallow_untyped_defs = False
+disallow_incomplete_defs = False
+check_untyped_defs = False
+
+[mypy-qapi.schema]
+disallow_untyped_defs = False
+disallow_incomplete_defs = False
+check_untyped_defs = False
+
+[mypy-qapi.source]
+disallow_untyped_defs = False
+disallow_incomplete_defs = False
+check_untyped_defs = False
+
+[mypy-qapi.types]
+disallow_untyped_defs = False
+disallow_incomplete_defs = False
+check_untyped_defs = False
+
+[mypy-qapi.visit]
+disallow_untyped_defs = False
+disallow_incomplete_defs = False
+check_untyped_defs = False
diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
index 74c6b96d391..483b4b68dff 100644
--- a/scripts/qapi/schema.py
+++ b/scripts/qapi/schema.py
@@ -17,6 +17,7 @@
 from collections import OrderedDict
 import os
 import re
+from typing import Optional
 
 from .common import POINTER_SUFFIX, c_name
 from .error import QAPIError, QAPISemError
@@ -25,7 +26,7 @@
 
 
 class QAPISchemaEntity:
-meta = None
+meta: Optional[str] = None
 
 def __init__(self, name, info, doc, ifcond=None, features=None):
 assert name is None or isinstance(name, str)
-- 
2.26.2

[PATCH v4 12/46] qapi: delint using flake8

[John Snow]

Petty style guide fixes and line length enforcement.  Not a big win, not
a big loss, but flake8 passes 100% on the qapi module, which gives us an
easy baseline to enforce hereafter.

A note on the flake8 exception: flake8 will warn on *any* bare except,
but pylint's is context-aware and will suppress the warning if you
re-raise the exception.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
---
 scripts/qapi/.flake8 |  2 ++
 scripts/qapi/commands.py |  3 ++-
 scripts/qapi/schema.py   |  8 +---
 scripts/qapi/visit.py| 16 +++-
 4 files changed, 20 insertions(+), 9 deletions(-)
 create mode 100644 scripts/qapi/.flake8

diff --git a/scripts/qapi/.flake8 b/scripts/qapi/.flake8
new file mode 100644
index 000..6b158c68b84
--- /dev/null
+++ b/scripts/qapi/.flake8
@@ -0,0 +1,2 @@
+[flake8]
+extend-ignore = E722  # Prefer pylint's bare-except checks to flake8's
diff --git a/scripts/qapi/commands.py b/scripts/qapi/commands.py
index 64ed5278f93..5dc2f5a9fa8 100644
--- a/scripts/qapi/commands.py
+++ b/scripts/qapi/commands.py
@@ -65,7 +65,8 @@ def gen_call(name, arg_type, boxed, ret_type):
 def gen_marshal_output(ret_type):
 return mcgen('''
 
-static void qmp_marshal_output_%(c_name)s(%(c_type)s ret_in, QObject 
**ret_out, Error **errp)
+static void qmp_marshal_output_%(c_name)s(%(c_type)s ret_in,
+QObject **ret_out, Error **errp)
 {
 Visitor *v;
 
diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
index 093f7a38d88..cfc52e1ae44 100644
--- a/scripts/qapi/schema.py
+++ b/scripts/qapi/schema.py
@@ -536,7 +536,7 @@ def set_defined_in(self, name):
 v.set_defined_in(name)
 
 def check(self, schema, seen):
-if not self.tag_member: # flat union
+if not self.tag_member:  # flat union
 self.tag_member = seen.get(c_name(self._tag_name))
 base = "'base'"
 # Pointing to the base type when not implicit would be
@@ -821,7 +821,7 @@ def __init__(self, fname):
 self._entity_dict = {}
 self._module_dict = OrderedDict()
 self._schema_dir = os.path.dirname(fname)
-self._make_module(None) # built-ins
+self._make_module(None)  # built-ins
 self._make_module(fname)
 self._predefining = True
 self._def_predefineds()
@@ -965,7 +965,9 @@ def _make_implicit_object_type(self, name, info, ifcond, 
role, members):
 # But it's not tight: the disjunction need not imply it.  We
 # may end up compiling useless wrapper types.
 # TODO kill simple unions or implement the disjunction
-assert (ifcond or []) == typ._ifcond # pylint: 
disable=protected-access
+
+# pylint: disable=protected-access
+assert (ifcond or []) == typ._ifcond
 else:
 self._def_entity(QAPISchemaObjectType(
 name, info, None, ifcond, None, None, members, None))
diff --git a/scripts/qapi/visit.py b/scripts/qapi/visit.py
index ea277e7704b..808410d6f1b 100644
--- a/scripts/qapi/visit.py
+++ b/scripts/qapi/visit.py
@@ -31,7 +31,9 @@ def gen_visit_decl(name, scalar=False):
 if not scalar:
 c_type += '*'
 return mcgen('''
-bool visit_type_%(c_name)s(Visitor *v, const char *name, %(c_type)sobj, Error 
**errp);
+
+bool visit_type_%(c_name)s(Visitor *v, const char *name,
+ %(c_type)sobj, Error **errp);
 ''',
  c_name=c_name(name), c_type=c_type)
 
@@ -125,7 +127,8 @@ def gen_visit_object_members(name, base, members, variants):
 def gen_visit_list(name, element_type):
 return mcgen('''
 
-bool visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, 
Error **errp)
+bool visit_type_%(c_name)s(Visitor *v, const char *name,
+ %(c_name)s **obj, Error **errp)
 {
 bool ok = false;
 %(c_name)s *tail;
@@ -158,7 +161,8 @@ def gen_visit_list(name, element_type):
 def gen_visit_enum(name):
 return mcgen('''
 
-bool visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s *obj, 
Error **errp)
+bool visit_type_%(c_name)s(Visitor *v, const char *name,
+ %(c_name)s *obj, Error **errp)
 {
 int value = *obj;
 bool ok = visit_type_enum(v, name, , &%(c_name)s_lookup, errp);
@@ -172,7 +176,8 @@ def gen_visit_enum(name):
 def gen_visit_alternate(name, variants):
 ret = mcgen('''
 
-bool visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, 
Error **errp)
+bool visit_type_%(c_name)s
+(Visitor *v, const char *name, %(c_name)s **obj, Error **errp)
 {
 bool ok = false;
 
@@ -247,7 +252,8 @@ def gen_visit_alternate(name, variants):
 def gen_visit_object(name, base, members, variants):
 return mcgen('''
 
-bool visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, 
Error **errp)
+bool visit_type_%(c_name)s
+(Visitor *v, const char *name, %(c_name)s **obj, Error **errp)
 {
 bool ok = false;
 
-- 

[PATCH v4 31/46] qapi/gen.py: add type hint annotations

[John Snow]

Annotations do not change runtime behavior.
This commit *only* adds annotations.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
---
 scripts/qapi/gen.py | 104 
 1 file changed, 57 insertions(+), 47 deletions(-)

diff --git a/scripts/qapi/gen.py b/scripts/qapi/gen.py
index 58cc1d8e734..4c6bba940ed 100644
--- a/scripts/qapi/gen.py
+++ b/scripts/qapi/gen.py
@@ -17,7 +17,13 @@
 import errno
 import os
 import re
-from typing import Optional
+from typing import (
+Dict,
+Iterator,
+List,
+Optional,
+Tuple,
+)
 
 from .common import (
 c_fname,
@@ -29,31 +35,31 @@
 mcgen,
 )
 from .schema import QAPISchemaObjectType, QAPISchemaVisitor
+from .source import QAPISourceInfo
 
 
 class QAPIGen:
-
-def __init__(self, fname):
+def __init__(self, fname: Optional[str]):
 self.fname = fname
 self._preamble = ''
 self._body = ''
 
-def preamble_add(self, text):
+def preamble_add(self, text: str) -> None:
 self._preamble += text
 
-def add(self, text):
+def add(self, text: str) -> None:
 self._body += text
 
-def get_content(self):
+def get_content(self) -> str:
 return self._top() + self._preamble + self._body + self._bottom()
 
-def _top(self):
+def _top(self) -> str:
 return ''
 
-def _bottom(self):
+def _bottom(self) -> str:
 return ''
 
-def write(self, output_dir):
+def write(self, output_dir: str) -> None:
 # Include paths starting with ../ are used to reuse modules of the main
 # schema in specialised schemas. Don't overwrite the files that are
 # already generated for the main schema.
@@ -78,7 +84,7 @@ def write(self, output_dir):
 f.close()
 
 
-def _wrap_ifcond(ifcond, before, after):
+def _wrap_ifcond(ifcond: List[str], before: str, after: str) -> str:
 if before == after:
 return after   # suppress empty #if ... #endif
 
@@ -118,40 +124,38 @@ def build_params(arg_type: Optional[QAPISchemaObjectType],
 
 
 class QAPIGenCCode(QAPIGen):
-
-def __init__(self, fname):
+def __init__(self, fname: Optional[str]):
 super().__init__(fname)
-self._start_if = None
+self._start_if: Optional[Tuple[List[str], str, str]] = None
 
-def start_if(self, ifcond):
+def start_if(self, ifcond: List[str]) -> None:
 assert self._start_if is None
 self._start_if = (ifcond, self._body, self._preamble)
 
-def end_if(self):
+def end_if(self) -> None:
 assert self._start_if
 self._wrap_ifcond()
 self._start_if = None
 
-def _wrap_ifcond(self):
+def _wrap_ifcond(self) -> None:
 self._body = _wrap_ifcond(self._start_if[0],
   self._start_if[1], self._body)
 self._preamble = _wrap_ifcond(self._start_if[0],
   self._start_if[2], self._preamble)
 
-def get_content(self):
+def get_content(self) -> str:
 assert self._start_if is None
 return super().get_content()
 
 
 class QAPIGenC(QAPIGenCCode):
-
-def __init__(self, fname, blurb, pydoc):
+def __init__(self, fname: str, blurb: str, pydoc: str):
 super().__init__(fname)
 self._blurb = blurb
 self._copyright = '\n * '.join(re.findall(r'^Copyright .*', pydoc,
   re.MULTILINE))
 
-def _top(self):
+def _top(self) -> str:
 return mcgen('''
 /* AUTOMATICALLY GENERATED, DO NOT MODIFY */
 
@@ -167,7 +171,7 @@ def _top(self):
 ''',
  blurb=self._blurb, copyright=self._copyright)
 
-def _bottom(self):
+def _bottom(self) -> str:
 return mcgen('''
 
 /* Dummy declaration to prevent empty .o file */
@@ -177,16 +181,15 @@ def _bottom(self):
 
 
 class QAPIGenH(QAPIGenC):
-
-def _top(self):
+def _top(self) -> str:
 return super()._top() + guardstart(self.fname)
 
-def _bottom(self):
+def _bottom(self) -> str:
 return guardend(self.fname)
 
 
 @contextmanager
-def ifcontext(ifcond, *args):
+def ifcontext(ifcond: List[str], *args: QAPIGenCCode) -> Iterator[None]:
 """
 A 'with' statement context manager that wraps with `start_if` and `end_if`.
 
@@ -214,8 +217,11 @@ def ifcontext(ifcond, *args):
 
 
 class QAPISchemaMonolithicCVisitor(QAPISchemaVisitor):
-
-def __init__(self, prefix, what, blurb, pydoc):
+def __init__(self,
+ prefix: str,
+ what: str,
+ blurb: str,
+ pydoc: str):
 self._prefix = prefix
 self._what = what
 self._genc = QAPIGenC(self._prefix + self._what + '.c',
@@ -223,38 +229,42 @@ def __init__(self, prefix, what, blurb, pydoc):
 self._genh = QAPIGenH(self._prefix + self._what + '.h',
   blurb, pydoc)
 
-def write(self, 

[PATCH v4 46/46] qapi/visit.py: add type hint annotations

[John Snow]

Annotations do not change runtime behavior.
This commit *only* adds annotations.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
Tested-by: Cleber Rosa 
---
 scripts/qapi/mypy.ini |  5 ---
 scripts/qapi/visit.py | 73 +--
 2 files changed, 56 insertions(+), 22 deletions(-)

diff --git a/scripts/qapi/mypy.ini b/scripts/qapi/mypy.ini
index eeb697c487c..c0f2a58306d 100644
--- a/scripts/qapi/mypy.ini
+++ b/scripts/qapi/mypy.ini
@@ -23,8 +23,3 @@ check_untyped_defs = False
 disallow_untyped_defs = False
 disallow_incomplete_defs = False
 check_untyped_defs = False
-
-[mypy-qapi.visit]
-disallow_untyped_defs = False
-disallow_incomplete_defs = False
-check_untyped_defs = False
diff --git a/scripts/qapi/visit.py b/scripts/qapi/visit.py
index e54694e23db..14d4f0b261f 100644
--- a/scripts/qapi/visit.py
+++ b/scripts/qapi/visit.py
@@ -13,6 +13,8 @@
 See the COPYING file in the top-level directory.
 """
 
+from typing import List, Optional
+
 from .common import (
 c_enum_const,
 c_name,
@@ -22,10 +24,20 @@
 mcgen,
 )
 from .gen import QAPISchemaModularCVisitor, ifcontext
-from .schema import QAPISchemaEnumType, QAPISchemaObjectType
+from .schema import (
+QAPISchema,
+QAPISchemaEnumMember,
+QAPISchemaEnumType,
+QAPISchemaFeature,
+QAPISchemaObjectType,
+QAPISchemaObjectTypeMember,
+QAPISchemaType,
+QAPISchemaVariants,
+)
+from .source import QAPISourceInfo
 
 
-def gen_visit_decl(name, scalar=False):
+def gen_visit_decl(name: str, scalar: bool = False) -> str:
 c_type = c_name(name) + ' *'
 if not scalar:
 c_type += '*'
@@ -37,7 +49,7 @@ def gen_visit_decl(name, scalar=False):
  c_name=c_name(name), c_type=c_type)
 
 
-def gen_visit_members_decl(name):
+def gen_visit_members_decl(name: str) -> str:
 return mcgen('''
 
 bool visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp);
@@ -45,7 +57,10 @@ def gen_visit_members_decl(name):
  c_name=c_name(name))
 
 
-def gen_visit_object_members(name, base, members, variants):
+def gen_visit_object_members(name: str,
+ base: Optional[QAPISchemaObjectType],
+ members: List[QAPISchemaObjectTypeMember],
+ variants: Optional[QAPISchemaVariants]) -> str:
 ret = mcgen('''
 
 bool visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp)
@@ -125,7 +140,7 @@ def gen_visit_object_members(name, base, members, variants):
 return ret
 
 
-def gen_visit_list(name, element_type):
+def gen_visit_list(name: str, element_type: QAPISchemaType) -> str:
 return mcgen('''
 
 bool visit_type_%(c_name)s(Visitor *v, const char *name,
@@ -159,7 +174,7 @@ def gen_visit_list(name, element_type):
  c_name=c_name(name), c_elt_type=element_type.c_name())
 
 
-def gen_visit_enum(name):
+def gen_visit_enum(name: str) -> str:
 return mcgen('''
 
 bool visit_type_%(c_name)s(Visitor *v, const char *name,
@@ -174,7 +189,7 @@ def gen_visit_enum(name):
  c_name=c_name(name))
 
 
-def gen_visit_alternate(name, variants):
+def gen_visit_alternate(name: str, variants: QAPISchemaVariants) -> str:
 ret = mcgen('''
 
 bool visit_type_%(c_name)s
@@ -250,7 +265,7 @@ def gen_visit_alternate(name, variants):
 return ret
 
 
-def gen_visit_object(name):
+def gen_visit_object(name: str) -> str:
 return mcgen('''
 
 bool visit_type_%(c_name)s
@@ -285,12 +300,12 @@ def gen_visit_object(name):
 
 class QAPISchemaGenVisitVisitor(QAPISchemaModularCVisitor):
 
-def __init__(self, prefix):
+def __init__(self, prefix: str):
 super().__init__(
 prefix, 'qapi-visit', ' * Schema-defined QAPI visitors',
 ' * Built-in QAPI visitors', __doc__)
 
-def _begin_system_module(self, name):
+def _begin_system_module(self, name: None) -> None:
 self._genc.preamble_add(mcgen('''
 #include "qemu/osdep.h"
 #include "qapi/error.h"
@@ -302,7 +317,7 @@ def _begin_system_module(self, name):
 
 '''))
 
-def _begin_user_module(self, name):
+def _begin_user_module(self, name: str) -> None:
 types = self._module_basename('qapi-types', name)
 visit = self._module_basename('qapi-visit', name)
 self._genc.preamble_add(mcgen('''
@@ -319,18 +334,34 @@ def _begin_user_module(self, name):
 ''',
   types=types))
 
-def visit_enum_type(self, name, info, ifcond, features, members, prefix):
+def visit_enum_type(self,
+name: str,
+info: QAPISourceInfo,
+ifcond: List[str],
+features: List[QAPISchemaFeature],
+members: List[QAPISchemaEnumMember],
+prefix: Optional[str]) -> None:
 with ifcontext(ifcond, self._genh, self._genc):
 

[PATCH v4 18/46] qapi/common.py: check with pylint

[John Snow]

Remove qapi/common.py from the pylintrc ignore list.

Signed-off-by: John Snow 
Reviewed-by: Cleber Rosa 
Tested-by: Cleber Rosa 
Tested-by: Eduardo Habkost 
---
 scripts/qapi/pylintrc | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/scripts/qapi/pylintrc b/scripts/qapi/pylintrc
index 76d54c30f85..507f15537ab 100644
--- a/scripts/qapi/pylintrc
+++ b/scripts/qapi/pylintrc
@@ -2,8 +2,7 @@
 
 # Add files or directories matching the regex patterns to the ignore list.
 # The regex matches against base names, not paths.
-ignore-patterns=common.py,
-error.py,
+ignore-patterns=error.py,
 expr.py,
 gen.py,
 parser.py,
-- 
2.26.2

[PATCH v4 10/46] qapi: Remove wildcard includes

[John Snow]

Wildcard includes become hard to manage when refactoring and dealing
with circular dependencies with strictly typed mypy.

flake8 also flags each one as a warning, as it is not smart enough to
know which names exist in the imported file.

Remove them and include things explicitly by name instead.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
---
 scripts/qapi/commands.py   |  2 +-
 scripts/qapi/events.py |  7 ++-
 scripts/qapi/gen.py| 12 +---
 scripts/qapi/introspect.py |  7 ++-
 scripts/qapi/types.py  |  8 +++-
 scripts/qapi/visit.py  | 10 +-
 6 files changed, 38 insertions(+), 8 deletions(-)

diff --git a/scripts/qapi/commands.py b/scripts/qapi/commands.py
index ce5926146a4..64ed5278f93 100644
--- a/scripts/qapi/commands.py
+++ b/scripts/qapi/commands.py
@@ -13,7 +13,7 @@
 See the COPYING file in the top-level directory.
 """
 
-from .common import *
+from .common import build_params, c_name, mcgen
 from .gen import QAPIGenCCode, QAPISchemaModularCVisitor, ifcontext
 
 
diff --git a/scripts/qapi/events.py b/scripts/qapi/events.py
index 04672724388..6b3afa14d72 100644
--- a/scripts/qapi/events.py
+++ b/scripts/qapi/events.py
@@ -12,7 +12,12 @@
 See the COPYING file in the top-level directory.
 """
 
-from .common import *
+from .common import (
+build_params,
+c_enum_const,
+c_name,
+mcgen,
+)
 from .gen import QAPISchemaModularCVisitor, ifcontext
 from .schema import QAPISchemaEnumMember
 from .types import gen_enum, gen_enum_lookup
diff --git a/scripts/qapi/gen.py b/scripts/qapi/gen.py
index 14d584680dc..61b3c53b180 100644
--- a/scripts/qapi/gen.py
+++ b/scripts/qapi/gen.py
@@ -11,13 +11,19 @@
 # This work is licensed under the terms of the GNU GPL, version 2.
 # See the COPYING file in the top-level directory.
 
-
+from contextlib import contextmanager
 import errno
 import os
 import re
-from contextlib import contextmanager
 
-from .common import *
+from .common import (
+c_fname,
+gen_endif,
+gen_if,
+guardend,
+guardstart,
+mcgen,
+)
 from .schema import QAPISchemaVisitor
 
 
diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
index 2a34cd1e8ea..b036fcf9ce7 100644
--- a/scripts/qapi/introspect.py
+++ b/scripts/qapi/introspect.py
@@ -10,7 +10,12 @@
 See the COPYING file in the top-level directory.
 """
 
-from .common import *
+from .common import (
+c_name,
+gen_endif,
+gen_if,
+mcgen,
+)
 from .gen import QAPISchemaMonolithicCVisitor
 from .schema import (QAPISchemaArrayType, QAPISchemaBuiltinType,
  QAPISchemaType)
diff --git a/scripts/qapi/types.py b/scripts/qapi/types.py
index ca9a5aacb39..53b47f9e58a 100644
--- a/scripts/qapi/types.py
+++ b/scripts/qapi/types.py
@@ -13,7 +13,13 @@
 # See the COPYING file in the top-level directory.
 """
 
-from .common import *
+from .common import (
+c_enum_const,
+c_name,
+gen_endif,
+gen_if,
+mcgen,
+)
 from .gen import QAPISchemaModularCVisitor, ifcontext
 from .schema import QAPISchemaEnumMember, QAPISchemaObjectType
 
diff --git a/scripts/qapi/visit.py b/scripts/qapi/visit.py
index 7850f6e8480..ea277e7704b 100644
--- a/scripts/qapi/visit.py
+++ b/scripts/qapi/visit.py
@@ -13,7 +13,15 @@
 See the COPYING file in the top-level directory.
 """
 
-from .common import *
+from .common import (
+c_enum_const,
+c_name,
+gen_endif,
+gen_if,
+mcgen,
+pop_indent,
+push_indent,
+)
 from .gen import QAPISchemaModularCVisitor, ifcontext
 from .schema import QAPISchemaObjectType
 
-- 
2.26.2

[PATCH v4 15/46] qapi/common.py: Add indent manager

[John Snow]

Code style tools really dislike the use of global keywords, because it
generally involves re-binding the name at runtime which can have strange
effects depending on when and how that global name is referenced in
other modules.

Make a little indent level manager instead.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
---
 scripts/qapi/common.py | 49 --
 scripts/qapi/visit.py  |  7 +++---
 2 files changed, 36 insertions(+), 20 deletions(-)

diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index cee63eb95c7..b35318b72cf 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -93,33 +93,50 @@ def c_name(name, protect=True):
 pointer_suffix = ' *' + eatspace
 
 
-def genindent(count):
-ret = ''
-for _ in range(count):
-ret += ' '
-return ret
+class Indentation:
+"""
+Indentation level management.
 
+:param initial: Initial number of spaces, default 0.
+"""
+def __init__(self, initial: int = 0) -> None:
+self._level = initial
 
-indent_level = 0
+def __int__(self) -> int:
+return self._level
 
+def __repr__(self) -> str:
+return "{}({:d})".format(type(self).__name__, self._level)
 
-def push_indent(indent_amount=4):
-global indent_level
-indent_level += indent_amount
+def __str__(self) -> str:
+"""Return the current indentation as a string of spaces."""
+return ' ' * self._level
 
+def __bool__(self) -> bool:
+"""True when there is a non-zero indentation."""
+return bool(self._level)
 
-def pop_indent(indent_amount=4):
-global indent_level
-indent_level -= indent_amount
+def increase(self, amount: int = 4) -> None:
+"""Increase the indentation level by ``amount``, default 4."""
+self._level += amount
+
+def decrease(self, amount: int = 4) -> None:
+"""Decrease the indentation level by ``amount``, default 4."""
+if self._level < amount:
+raise ArithmeticError(
+f"Can't remove {amount:d} spaces from {self!r}")
+self._level -= amount
+
+
+indent = Indentation()
 
 
 # Generate @code with @kwds interpolated.
-# Obey indent_level, and strip eatspace.
+# Obey indent, and strip eatspace.
 def cgen(code, **kwds):
 raw = code % kwds
-if indent_level:
-indent = genindent(indent_level)
-raw = re.sub(r'^(?!(#|$))', indent, raw, flags=re.MULTILINE)
+if indent:
+raw = re.sub(r'^(?!(#|$))', str(indent), raw, flags=re.MULTILINE)
 return re.sub(re.escape(eatspace) + r' *', '', raw)
 
 
diff --git a/scripts/qapi/visit.py b/scripts/qapi/visit.py
index 808410d6f1b..14f30c228b7 100644
--- a/scripts/qapi/visit.py
+++ b/scripts/qapi/visit.py
@@ -18,9 +18,8 @@
 c_name,
 gen_endif,
 gen_if,
+indent,
 mcgen,
-pop_indent,
-push_indent,
 )
 from .gen import QAPISchemaModularCVisitor, ifcontext
 from .schema import QAPISchemaObjectType
@@ -69,7 +68,7 @@ def gen_visit_object_members(name, base, members, variants):
 if (visit_optional(v, "%(name)s", >has_%(c_name)s)) {
 ''',
  name=memb.name, c_name=c_name(memb.name))
-push_indent()
+indent.increase()
 ret += mcgen('''
 if (!visit_type_%(c_type)s(v, "%(name)s", >%(c_name)s, errp)) {
 return false;
@@ -78,7 +77,7 @@ def gen_visit_object_members(name, base, members, variants):
  c_type=memb.type.c_name(), name=memb.name,
  c_name=c_name(memb.name))
 if memb.optional:
-pop_indent()
+indent.decrease()
 ret += mcgen('''
 }
 ''')
-- 
2.26.2

[PATCH v4 17/46] qapi/common.py: Replace one-letter 'c' variable

[John Snow]

Signed-off-by: John Snow 
Reviewed-by: Cleber Rosa 
Reviewed-by: Eduardo Habkost 
---
 scripts/qapi/common.py | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index a417b6029c8..338adedef4f 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -30,14 +30,14 @@ def camel_to_upper(value):
 new_name = ''
 length = len(c_fun_str)
 for i in range(length):
-c = c_fun_str[i]
-# When c is upper and no '_' appears before, do more checks
-if c.isupper() and (i > 0) and c_fun_str[i - 1] != '_':
+char = c_fun_str[i]
+# When char is upper case and no '_' appears before, do more checks
+if char.isupper() and (i > 0) and c_fun_str[i - 1] != '_':
 if i < length - 1 and c_fun_str[i + 1].islower():
 new_name += '_'
 elif c_fun_str[i - 1].isdigit():
 new_name += '_'
-new_name += c
+new_name += char
 return new_name.lstrip('_').upper()
 
 
-- 
2.26.2

[PATCH v4 06/46] qapi-gen: Separate arg-parsing from generation

[John Snow]

This is a minor re-work of the entrypoint script. It isolates a
generate() method from the actual command-line mechanism.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
Tested-by: Cleber Rosa 
---
 scripts/qapi-gen.py | 85 +
 1 file changed, 62 insertions(+), 23 deletions(-)

diff --git a/scripts/qapi-gen.py b/scripts/qapi-gen.py
index 541e8c1f55d..117b396a595 100644
--- a/scripts/qapi-gen.py
+++ b/scripts/qapi-gen.py
@@ -1,30 +1,77 @@
 #!/usr/bin/env python3
-# QAPI generator
-#
+
 # This work is licensed under the terms of the GNU GPL, version 2 or later.
 # See the COPYING file in the top-level directory.
 
+"""
+QAPI Generator
+
+This script is the main entry point for generating C code from the QAPI schema.
+"""
 
 import argparse
 import re
 import sys
 
 from qapi.commands import gen_commands
+from qapi.error import QAPIError
 from qapi.events import gen_events
 from qapi.introspect import gen_introspect
-from qapi.schema import QAPIError, QAPISchema
+from qapi.schema import QAPISchema
 from qapi.types import gen_types
 from qapi.visit import gen_visit
 
 
-def main(argv):
+DEFAULT_OUTPUT_DIR = ''
+DEFAULT_PREFIX = ''
+
+
+def generate(schema_file: str,
+ output_dir: str,
+ prefix: str,
+ unmask: bool = False,
+ builtins: bool = False) -> None:
+"""
+generate uses a given schema to produce C code in the target directory.
+
+:param schema_file: The primary QAPI schema file.
+:param output_dir: The output directory to store generated code.
+:param prefix: Optional C-code prefix for symbol names.
+:param unmask: Expose non-ABI names through introspection?
+:param builtins: Generate code for built-in types?
+
+:raise QAPIError: On failures.
+"""
+match = re.match(r'([A-Za-z_.-][A-Za-z0-9_.-]*)?', prefix)
+if match.end() != len(prefix):
+msg = "funny character '{:s}' in prefix '{:s}'".format(
+prefix[match.end()], prefix)
+raise QAPIError('', None, msg)
+
+schema = QAPISchema(schema_file)
+gen_types(schema, output_dir, prefix, builtins)
+gen_visit(schema, output_dir, prefix, builtins)
+gen_commands(schema, output_dir, prefix)
+gen_events(schema, output_dir, prefix)
+gen_introspect(schema, output_dir, prefix, unmask)
+
+
+def main() -> int:
+"""
+gapi-gen shell script entrypoint.
+Expects arguments via sys.argv, see --help for details.
+
+:return: int, 0 on success, 1 on failure.
+"""
 parser = argparse.ArgumentParser(
 description='Generate code from a QAPI schema')
 parser.add_argument('-b', '--builtins', action='store_true',
 help="generate code for built-in types")
-parser.add_argument('-o', '--output-dir', action='store', default='',
+parser.add_argument('-o', '--output-dir', action='store',
+default=DEFAULT_OUTPUT_DIR,
 help="write output to directory OUTPUT_DIR")
-parser.add_argument('-p', '--prefix', action='store', default='',
+parser.add_argument('-p', '--prefix', action='store',
+default=DEFAULT_PREFIX,
 help="prefix for symbols")
 parser.add_argument('-u', '--unmask-non-abi-names', action='store_true',
 dest='unmask',
@@ -32,25 +79,17 @@ def main(argv):
 parser.add_argument('schema', action='store')
 args = parser.parse_args()
 
-match = re.match(r'([A-Za-z_.-][A-Za-z0-9_.-]*)?', args.prefix)
-if match.end() != len(args.prefix):
-print("%s: 'funny character '%s' in argument of --prefix"
-  % (sys.argv[0], args.prefix[match.end()]),
-  file=sys.stderr)
-sys.exit(1)
-
 try:
-schema = QAPISchema(args.schema)
+generate(args.schema,
+ output_dir=args.output_dir,
+ prefix=args.prefix,
+ unmask=args.unmask,
+ builtins=args.builtins)
 except QAPIError as err:
-print(err, file=sys.stderr)
-exit(1)
-
-gen_types(schema, args.output_dir, args.prefix, args.builtins)
-gen_visit(schema, args.output_dir, args.prefix, args.builtins)
-gen_commands(schema, args.output_dir, args.prefix)
-gen_events(schema, args.output_dir, args.prefix)
-gen_introspect(schema, args.output_dir, args.prefix, args.unmask)
+print(f"{sys.argv[0]}: {str(err)}", file=sys.stderr)
+return 1
+return 0
 
 
 if __name__ == '__main__':
-main(sys.argv)
+sys.exit(main())
-- 
2.26.2

[PATCH v4 28/46] qapi/source.py: add type hint annotations

[John Snow]

Annotations do not change runtime behavior.
This commit *only* adds annotations.

A note on typing of __init__: mypy requires init functions with no
parameters to document a return type of None to be considered fully
typed. In the case when there are input parameters, None may be omitted.

Since __init__ may never return any value, it is preferred to omit the
return annotation whenever possible.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
Tested-by: Cleber Rosa 
---
 scripts/qapi/mypy.ini  |  5 -
 scripts/qapi/source.py | 31 ++-
 2 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/scripts/qapi/mypy.ini b/scripts/qapi/mypy.ini
index 8ab9ac52cc4..1b8555dfa39 100644
--- a/scripts/qapi/mypy.ini
+++ b/scripts/qapi/mypy.ini
@@ -34,11 +34,6 @@ disallow_untyped_defs = False
 disallow_incomplete_defs = False
 check_untyped_defs = False
 
-[mypy-qapi.source]
-disallow_untyped_defs = False
-disallow_incomplete_defs = False
-check_untyped_defs = False
-
 [mypy-qapi.types]
 disallow_untyped_defs = False
 disallow_incomplete_defs = False
diff --git a/scripts/qapi/source.py b/scripts/qapi/source.py
index e97b9a8e15e..1cc6a5b82dc 100644
--- a/scripts/qapi/source.py
+++ b/scripts/qapi/source.py
@@ -11,37 +11,42 @@
 
 import copy
 import sys
+from typing import List, Optional, TypeVar
 
 
 class QAPISchemaPragma:
-def __init__(self):
+def __init__(self) -> None:
 # Are documentation comments required?
 self.doc_required = False
 # Whitelist of commands allowed to return a non-dictionary
-self.returns_whitelist = []
+self.returns_whitelist: List[str] = []
 # Whitelist of entities allowed to violate case conventions
-self.name_case_whitelist = []
+self.name_case_whitelist: List[str] = []
 
 
 class QAPISourceInfo:
-def __init__(self, fname, line, parent):
+T = TypeVar('T', bound='QAPISourceInfo')
+
+def __init__(self: T, fname: str, line: int, parent: Optional[T]):
 self.fname = fname
 self.line = line
 self.parent = parent
-self.pragma = parent.pragma if parent else QAPISchemaPragma()
-self.defn_meta = None
-self.defn_name = None
+self.pragma: QAPISchemaPragma = (
+parent.pragma if parent else QAPISchemaPragma()
+)
+self.defn_meta: Optional[str] = None
+self.defn_name: Optional[str] = None
 
-def set_defn(self, meta, name):
+def set_defn(self, meta: str, name: str) -> None:
 self.defn_meta = meta
 self.defn_name = name
 
-def next_line(self):
+def next_line(self: T) -> T:
 info = copy.copy(self)
 info.line += 1
 return info
 
-def loc(self):
+def loc(self) -> str:
 if self.fname is None:
 return sys.argv[0]
 ret = self.fname
@@ -49,13 +54,13 @@ def loc(self):
 ret += ':%d' % self.line
 return ret
 
-def in_defn(self):
+def in_defn(self) -> str:
 if self.defn_name:
 return "%s: In %s '%s':\n" % (self.fname,
   self.defn_meta, self.defn_name)
 return ''
 
-def include_path(self):
+def include_path(self) -> str:
 ret = ''
 parent = self.parent
 while parent:
@@ -63,5 +68,5 @@ def include_path(self):
 parent = parent.parent
 return ret
 
-def __str__(self):
+def __str__(self) -> str:
 return self.include_path() + self.in_defn() + self.loc()
-- 
2.26.2

[PATCH v4 11/46] qapi: enforce import order/styling with isort

[John Snow]

While we're mucking around with imports, we might as well formalize the
style we use. Let's use isort to do it for us.

lines_after_imports=2: Use two lines after imports, to match PEP8's
desire to have "two lines before and after" class definitions, which are
likely to start immediately after imports.

force_sort_within_sections: Intermingles "from x" and "import x" style
statements, such that sorting is always performed strictly on the module
name itself.

force_grid_wrap=4: Four or more imports from a single module will force
the one-per-line style that's more git-friendly. This will generally
happen for 'typing' imports.

multi_line_output=3: Uses the one-per-line indented style for long
imports.

include_trailing_comma: Adds a comma to the last import in a group,
which makes git conflicts nicer to deal with, generally.

line_length: 72 is chosen to match PEP8's "docstrings and comments" line
length limit. If you have a single line import that exceeds 72
characters, your names are too long!

Suggested-by: Cleber Rosa 
Signed-off-by: John Snow 
Reviewed-by: Cleber Rosa 
Tested-by: Cleber Rosa 
---
 scripts/qapi/.isort.cfg| 7 +++
 scripts/qapi/expr.py   | 3 ++-
 scripts/qapi/introspect.py | 7 +--
 scripts/qapi/parser.py | 2 +-
 scripts/qapi/schema.py | 2 +-
 5 files changed, 16 insertions(+), 5 deletions(-)
 create mode 100644 scripts/qapi/.isort.cfg

diff --git a/scripts/qapi/.isort.cfg b/scripts/qapi/.isort.cfg
new file mode 100644
index 000..6d0fd6cc0d3
--- /dev/null
+++ b/scripts/qapi/.isort.cfg
@@ -0,0 +1,7 @@
+[settings]
+force_grid_wrap=4
+force_sort_within_sections=True
+include_trailing_comma=True
+line_length=72
+lines_after_imports=2
+multi_line_output=3
\ No newline at end of file
diff --git a/scripts/qapi/expr.py b/scripts/qapi/expr.py
index 03b31ecfc19..e73b65b6a7e 100644
--- a/scripts/qapi/expr.py
+++ b/scripts/qapi/expr.py
@@ -14,8 +14,9 @@
 # This work is licensed under the terms of the GNU GPL, version 2.
 # See the COPYING file in the top-level directory.
 
-import re
 from collections import OrderedDict
+import re
+
 from .common import c_name
 from .error import QAPISemError
 
diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
index b036fcf9ce7..31acd2f230a 100644
--- a/scripts/qapi/introspect.py
+++ b/scripts/qapi/introspect.py
@@ -17,8 +17,11 @@
 mcgen,
 )
 from .gen import QAPISchemaMonolithicCVisitor
-from .schema import (QAPISchemaArrayType, QAPISchemaBuiltinType,
- QAPISchemaType)
+from .schema import (
+QAPISchemaArrayType,
+QAPISchemaBuiltinType,
+QAPISchemaType,
+)
 
 
 def _make_tree(obj, ifcond, features, extra=None):
diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index a9388eaf765..a6081a0c5d4 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -14,9 +14,9 @@
 # This work is licensed under the terms of the GNU GPL, version 2.
 # See the COPYING file in the top-level directory.
 
+from collections import OrderedDict
 import os
 import re
-from collections import OrderedDict
 
 from .error import QAPIParseError, QAPISemError
 from .source import QAPISourceInfo
diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
index a835ee6fde3..093f7a38d88 100644
--- a/scripts/qapi/schema.py
+++ b/scripts/qapi/schema.py
@@ -14,9 +14,9 @@
 
 # TODO catching name collisions in generated code would be nice
 
+from collections import OrderedDict
 import os
 import re
-from collections import OrderedDict
 
 from .common import c_name, pointer_suffix
 from .error import QAPIError, QAPISemError
-- 
2.26.2

[PATCH v4 20/46] qapi/common.py: Convert comments into docstrings, and elaborate

[John Snow]

As docstrings, they'll show up in documentation and IDE help.

The docstring style being targeted is the Sphinx documentation
style. Sphinx uses an extension of ReST with "domains". We use the
(implicit) Python domain, which supports a number of custom "info
fields". Those info fields are documented here:
https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#info-field-lists

Primarily, we use `:param X: descr`, `:return[s]: descr`, and `:raise[s]
Z: when`. Everything else is the Sphinx dialect of ReST.

(No, nothing checks or enforces this style that I am aware of. Sphinx
either chokes or succeeds, but does not enforce a standard of what is
otherwise inside the docstring. Pycharm does highlight when your param
fields are not aligned with the actual fields present. It does not
highlight missing return or exception statements. There is no existing
style guide I am aware of that covers a standard for a minimally
acceptable docstring. I am debating writing one.)

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
---
 scripts/qapi/common.py | 53 +++---
 1 file changed, 39 insertions(+), 14 deletions(-)

diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index 74a2c001ed9..0ef38ea5fe0 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -15,15 +15,24 @@
 from typing import Optional, Sequence
 
 
+#: Sentinel value that causes all space to its right to be removed.
 EATSPACE = '\033EATSPACE.'
 POINTER_SUFFIX = ' *' + EATSPACE
 _C_NAME_TRANS = str.maketrans('.-', '__')
 
 
-# ENUMName -> ENUM_NAME, EnumName1 -> ENUM_NAME1
-# ENUM_NAME -> ENUM_NAME, ENUM_NAME1 -> ENUM_NAME1, ENUM_Name2 -> ENUM_NAME2
-# ENUM24_Name -> ENUM24_NAME
 def camel_to_upper(value: str) -> str:
+"""
+Converts CamelCase to CAMEL_CASE.
+
+Examples:
+  ENUMName -> ENUM_NAME
+  EnumName1 -> ENUM_NAME1
+  ENUM_NAME -> ENUM_NAME
+  ENUM_NAME1 -> ENUM_NAME1
+  ENUM_Name2 -> ENUM_NAME2
+  ENUM24_Name -> ENUM24_NAME
+"""
 c_fun_str = c_name(value, False)
 if value.isupper():
 return c_fun_str
@@ -45,21 +54,33 @@ def camel_to_upper(value: str) -> str:
 def c_enum_const(type_name: str,
  const_name: str,
  prefix: Optional[str] = None) -> str:
+"""
+Generate a C enumeration constant name.
+
+:param type_name: The name of the enumeration.
+:param const_name: The name of this constant.
+:param prefix: Optional, prefix that overrides the type_name.
+"""
 if prefix is not None:
 type_name = prefix
 return camel_to_upper(type_name) + '_' + c_name(const_name, False).upper()
 
 
-# Map @name to a valid C identifier.
-# If @protect, avoid returning certain ticklish identifiers (like
-# C keywords) by prepending 'q_'.
-#
-# Used for converting 'name' from a 'name':'type' qapi definition
-# into a generated struct member, as well as converting type names
-# into substrings of a generated C function name.
-# '__a.b_c' -> '__a_b_c', 'x-foo' -> 'x_foo'
-# protect=True: 'int' -> 'q_int'; protect=False: 'int' -> 'int'
 def c_name(name: str, protect: bool = True) -> str:
+"""
+Map ``name`` to a valid C identifier.
+
+Used for converting 'name' from a 'name':'type' qapi definition
+into a generated struct member, as well as converting type names
+into substrings of a generated C function name.
+
+'__a.b_c' -> '__a_b_c', 'x-foo' -> 'x_foo'
+protect=True: 'int' -> 'q_int'; protect=False: 'int' -> 'int'
+
+:param name: The name to map.
+:param protect: If true, avoid returning certain ticklish identifiers
+(like C keywords) by prepending ``q_``.
+"""
 # ANSI X3J11/88-090, 3.1.1
 c89_words = set(['auto', 'break', 'case', 'char', 'const', 'continue',
  'default', 'do', 'double', 'else', 'enum', 'extern',
@@ -129,12 +150,16 @@ def decrease(self, amount: int = 4) -> None:
 self._level -= amount
 
 
+#: Global, current indent level for code generation.
 indent = Indentation()
 
 
-# Generate @code with @kwds interpolated.
-# Obey indent, and strip EATSPACE.
 def cgen(code: str, **kwds: object) -> str:
+"""
+Generate ``code`` with ``kwds`` interpolated.
+
+Obey `indent`, and strip `EATSPACE`.
+"""
 raw = code % kwds
 if indent:
 raw = re.sub(r'^(?!(#|$))', str(indent), raw, flags=re.MULTILINE)
-- 
2.26.2

[PATCH v4 13/46] qapi: add pylintrc

[John Snow]

Using `pylint --generate-rcfile > pylintrc`, generate a skeleton
pylintrc file. Sections that are not presently relevant (by the end of
this series) are removed leaving just the empty section as a search
engine / documentation hint to future authors.

I am targeting pylint 2.6.0. In the future (and hopefully before 5.2 is
released), I aim to have gitlab CI running the specific targeted
versions of pylint, mypy, flake8, etc in a job.

2.5.x will work if you additionally pass --disable=bad-whitespace.
This warning was removed from 2.6.x, for lack of consistent support.

Right now, quite a few modules are ignored as they are known to fail as
of this commit. modules will be removed from the known-bad list
throughout this and following series as they are repaired.

Note: Normally, pylintrc would go in the folder above the module, but as
that folder is shared by many things, it is going inside the module
folder (for now). Due to a bug in pylint 2.5+, pylint does not
correctly recognize when it is being run from "inside" a package, and
must be run *outside* of the package.

Therefore, to run it, you must:

 > pylint scripts/qapi/ --rcfile=scripts/qapi/pylintrc

Signed-off-by: John Snow 
Tested-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
Tested-by: Cleber Rosa 
---
 scripts/qapi/pylintrc | 73 +++
 1 file changed, 73 insertions(+)
 create mode 100644 scripts/qapi/pylintrc

diff --git a/scripts/qapi/pylintrc b/scripts/qapi/pylintrc
new file mode 100644
index 000..76d54c30f85
--- /dev/null
+++ b/scripts/qapi/pylintrc
@@ -0,0 +1,73 @@
+[MASTER]
+
+# Add files or directories matching the regex patterns to the ignore list.
+# The regex matches against base names, not paths.
+ignore-patterns=common.py,
+error.py,
+expr.py,
+gen.py,
+parser.py,
+schema.py,
+source.py,
+types.py,
+visit.py,
+
+
+[MESSAGES CONTROL]
+
+# Disable the message, report, category or checker with the given id(s). You
+# can either give multiple identifiers separated by comma (,) or put this
+# option multiple times (only on the command line, not in the configuration
+# file where it should appear only once). You can also use "--disable=all" to
+# disable everything first and then reenable specific checks. For example, if
+# you want to run only the similarities checker, you can use "--disable=all
+# --enable=similarities". If you want to run only the classes checker, but have
+# no Warning level messages displayed, use "--disable=all --enable=classes
+# --disable=W".
+disable=fixme,
+missing-docstring,
+too-many-arguments,
+too-many-branches,
+too-many-statements,
+too-many-instance-attributes,
+
+[REPORTS]
+
+[REFACTORING]
+
+[MISCELLANEOUS]
+
+[LOGGING]
+
+[BASIC]
+
+# Good variable names which should always be accepted, separated by a comma.
+good-names=i,
+   j,
+   k,
+   ex,
+   Run,
+   _
+
+[VARIABLES]
+
+[STRING]
+
+[SPELLING]
+
+[FORMAT]
+
+[SIMILARITIES]
+
+# Ignore import statements themselves when computing similarities.
+ignore-imports=yes
+
+[TYPECHECK]
+
+[CLASSES]
+
+[IMPORTS]
+
+[DESIGN]
+
+[EXCEPTIONS]
-- 
2.26.2

[PATCH v4 07/46] qapi: move generator entrypoint into module

[John Snow]

As part of delinting and adding type hints to the QAPI generator, it's
helpful for the entrypoint to be part of the package, only leaving a
very tiny entrypoint shim outside of the module.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
Tested-by: Cleber Rosa 
---
 scripts/qapi-gen.py  | 88 +++
 scripts/qapi/main.py | 89 
 2 files changed, 95 insertions(+), 82 deletions(-)
 create mode 100644 scripts/qapi/main.py

diff --git a/scripts/qapi-gen.py b/scripts/qapi-gen.py
index 117b396a595..f3518d29a54 100644
--- a/scripts/qapi-gen.py
+++ b/scripts/qapi-gen.py
@@ -4,92 +4,16 @@
 # See the COPYING file in the top-level directory.
 
 """
-QAPI Generator
+QAPI code generation execution shim.
 
-This script is the main entry point for generating C code from the QAPI schema.
+This standalone script exists primarily to facilitate the running of the QAPI
+code generator without needing to install the python module to the current
+execution environment.
 """
 
-import argparse
-import re
 import sys
 
-from qapi.commands import gen_commands
-from qapi.error import QAPIError
-from qapi.events import gen_events
-from qapi.introspect import gen_introspect
-from qapi.schema import QAPISchema
-from qapi.types import gen_types
-from qapi.visit import gen_visit
-
-
-DEFAULT_OUTPUT_DIR = ''
-DEFAULT_PREFIX = ''
-
-
-def generate(schema_file: str,
- output_dir: str,
- prefix: str,
- unmask: bool = False,
- builtins: bool = False) -> None:
-"""
-generate uses a given schema to produce C code in the target directory.
-
-:param schema_file: The primary QAPI schema file.
-:param output_dir: The output directory to store generated code.
-:param prefix: Optional C-code prefix for symbol names.
-:param unmask: Expose non-ABI names through introspection?
-:param builtins: Generate code for built-in types?
-
-:raise QAPIError: On failures.
-"""
-match = re.match(r'([A-Za-z_.-][A-Za-z0-9_.-]*)?', prefix)
-if match.end() != len(prefix):
-msg = "funny character '{:s}' in prefix '{:s}'".format(
-prefix[match.end()], prefix)
-raise QAPIError('', None, msg)
-
-schema = QAPISchema(schema_file)
-gen_types(schema, output_dir, prefix, builtins)
-gen_visit(schema, output_dir, prefix, builtins)
-gen_commands(schema, output_dir, prefix)
-gen_events(schema, output_dir, prefix)
-gen_introspect(schema, output_dir, prefix, unmask)
-
-
-def main() -> int:
-"""
-gapi-gen shell script entrypoint.
-Expects arguments via sys.argv, see --help for details.
-
-:return: int, 0 on success, 1 on failure.
-"""
-parser = argparse.ArgumentParser(
-description='Generate code from a QAPI schema')
-parser.add_argument('-b', '--builtins', action='store_true',
-help="generate code for built-in types")
-parser.add_argument('-o', '--output-dir', action='store',
-default=DEFAULT_OUTPUT_DIR,
-help="write output to directory OUTPUT_DIR")
-parser.add_argument('-p', '--prefix', action='store',
-default=DEFAULT_PREFIX,
-help="prefix for symbols")
-parser.add_argument('-u', '--unmask-non-abi-names', action='store_true',
-dest='unmask',
-help="expose non-ABI names in introspection")
-parser.add_argument('schema', action='store')
-args = parser.parse_args()
-
-try:
-generate(args.schema,
- output_dir=args.output_dir,
- prefix=args.prefix,
- unmask=args.unmask,
- builtins=args.builtins)
-except QAPIError as err:
-print(f"{sys.argv[0]}: {str(err)}", file=sys.stderr)
-return 1
-return 0
-
+from qapi import main
 
 if __name__ == '__main__':
-sys.exit(main())
+sys.exit(main.main())
diff --git a/scripts/qapi/main.py b/scripts/qapi/main.py
new file mode 100644
index 000..9210a0e1a80
--- /dev/null
+++ b/scripts/qapi/main.py
@@ -0,0 +1,89 @@
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+
+"""
+QAPI Generator
+
+This is the main entry point for generating C code from the QAPI schema.
+"""
+
+import argparse
+import re
+import sys
+
+from qapi.commands import gen_commands
+from qapi.error import QAPIError
+from qapi.events import gen_events
+from qapi.introspect import gen_introspect
+from qapi.schema import QAPISchema
+from qapi.types import gen_types
+from qapi.visit import gen_visit
+
+
+DEFAULT_OUTPUT_DIR = ''
+DEFAULT_PREFIX = ''
+
+
+def generate(schema_file: str,
+ output_dir: str,
+ prefix: str,
+ unmask: bool = False,
+ builtins: bool = False) -> None:
+"""

[PATCH v4 08/46] [DO-NOT-MERGE] docs: add scripts/qapi/main to python manual

[John Snow]

Signed-off-by: John Snow 
---
 docs/devel/python/qapi.main.rst | 7 +++
 docs/devel/python/qapi.rst  | 1 +
 2 files changed, 8 insertions(+)
 create mode 100644 docs/devel/python/qapi.main.rst

diff --git a/docs/devel/python/qapi.main.rst b/docs/devel/python/qapi.main.rst
new file mode 100644
index 000..1255fcda633
--- /dev/null
+++ b/docs/devel/python/qapi.main.rst
@@ -0,0 +1,7 @@
+qapi.main module
+
+
+.. automodule:: qapi.main
+   :members:
+   :undoc-members:
+   :show-inheritance:
diff --git a/docs/devel/python/qapi.rst b/docs/devel/python/qapi.rst
index cfeb759d763..c762019aad3 100644
--- a/docs/devel/python/qapi.rst
+++ b/docs/devel/python/qapi.rst
@@ -18,6 +18,7 @@ Submodules
qapi.expr
qapi.gen
qapi.introspect
+   qapi.main
qapi.parser
qapi.schema
qapi.source
-- 
2.26.2

[PATCH v4 05/46] [DO-NOT-MERGE] docs: enable sphinx-autodoc for scripts/qapi

[John Snow]

Signed-off-by: John Snow 
---
 docs/conf.py  |  3 ++-
 docs/devel/index.rst  |  1 +
 docs/devel/python/index.rst   |  7 +++
 docs/devel/python/qapi.commands.rst   |  7 +++
 docs/devel/python/qapi.common.rst |  7 +++
 docs/devel/python/qapi.error.rst  |  7 +++
 docs/devel/python/qapi.events.rst |  7 +++
 docs/devel/python/qapi.expr.rst   |  7 +++
 docs/devel/python/qapi.gen.rst|  7 +++
 docs/devel/python/qapi.introspect.rst |  7 +++
 docs/devel/python/qapi.parser.rst |  8 
 docs/devel/python/qapi.rst| 25 +
 docs/devel/python/qapi.schema.rst |  7 +++
 docs/devel/python/qapi.source.rst |  7 +++
 docs/devel/python/qapi.types.rst  |  7 +++
 docs/devel/python/qapi.visit.rst  |  7 +++
 16 files changed, 120 insertions(+), 1 deletion(-)
 create mode 100644 docs/devel/python/index.rst
 create mode 100644 docs/devel/python/qapi.commands.rst
 create mode 100644 docs/devel/python/qapi.common.rst
 create mode 100644 docs/devel/python/qapi.error.rst
 create mode 100644 docs/devel/python/qapi.events.rst
 create mode 100644 docs/devel/python/qapi.expr.rst
 create mode 100644 docs/devel/python/qapi.gen.rst
 create mode 100644 docs/devel/python/qapi.introspect.rst
 create mode 100644 docs/devel/python/qapi.parser.rst
 create mode 100644 docs/devel/python/qapi.rst
 create mode 100644 docs/devel/python/qapi.schema.rst
 create mode 100644 docs/devel/python/qapi.source.rst
 create mode 100644 docs/devel/python/qapi.types.rst
 create mode 100644 docs/devel/python/qapi.visit.rst

diff --git a/docs/conf.py b/docs/conf.py
index 49599e38314..1e428a4bda8 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -70,7 +70,8 @@
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-extensions = ['kerneldoc', 'qmp_lexer', 'hxtool', 'depfile', 'qapidoc']
+extensions = ['kerneldoc', 'qmp_lexer', 'hxtool',
+  'depfile', 'qapidoc', 'sphinx.ext.autodoc']
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
diff --git a/docs/devel/index.rst b/docs/devel/index.rst
index 04773ce076b..04726ca7870 100644
--- a/docs/devel/index.rst
+++ b/docs/devel/index.rst
@@ -31,3 +31,4 @@ Contents:
reset
s390-dasd-ipl
clocks
+   python/index
diff --git a/docs/devel/python/index.rst b/docs/devel/python/index.rst
new file mode 100644
index 000..31c470154b3
--- /dev/null
+++ b/docs/devel/python/index.rst
@@ -0,0 +1,7 @@
+qapi
+
+
+.. toctree::
+   :maxdepth: 4
+
+   qapi
diff --git a/docs/devel/python/qapi.commands.rst 
b/docs/devel/python/qapi.commands.rst
new file mode 100644
index 000..018f7b08a9c
--- /dev/null
+++ b/docs/devel/python/qapi.commands.rst
@@ -0,0 +1,7 @@
+qapi.commands module
+
+
+.. automodule:: qapi.commands
+   :members:
+   :undoc-members:
+   :show-inheritance:
diff --git a/docs/devel/python/qapi.common.rst 
b/docs/devel/python/qapi.common.rst
new file mode 100644
index 000..128a90d74be
--- /dev/null
+++ b/docs/devel/python/qapi.common.rst
@@ -0,0 +1,7 @@
+qapi.common module
+==
+
+.. automodule:: qapi.common
+   :members:
+   :undoc-members:
+   :show-inheritance:
diff --git a/docs/devel/python/qapi.error.rst b/docs/devel/python/qapi.error.rst
new file mode 100644
index 000..980e32b63de
--- /dev/null
+++ b/docs/devel/python/qapi.error.rst
@@ -0,0 +1,7 @@
+qapi.error module
+=
+
+.. automodule:: qapi.error
+   :members:
+   :undoc-members:
+   :show-inheritance:
diff --git a/docs/devel/python/qapi.events.rst 
b/docs/devel/python/qapi.events.rst
new file mode 100644
index 000..1fce85b044e
--- /dev/null
+++ b/docs/devel/python/qapi.events.rst
@@ -0,0 +1,7 @@
+qapi.events module
+==
+
+.. automodule:: qapi.events
+   :members:
+   :undoc-members:
+   :show-inheritance:
diff --git a/docs/devel/python/qapi.expr.rst b/docs/devel/python/qapi.expr.rst
new file mode 100644
index 000..0660270629c
--- /dev/null
+++ b/docs/devel/python/qapi.expr.rst
@@ -0,0 +1,7 @@
+qapi.expr module
+
+
+.. automodule:: qapi.expr
+   :members:
+   :undoc-members:
+   :show-inheritance:
diff --git a/docs/devel/python/qapi.gen.rst b/docs/devel/python/qapi.gen.rst
new file mode 100644
index 000..7b495fd4bf2
--- /dev/null
+++ b/docs/devel/python/qapi.gen.rst
@@ -0,0 +1,7 @@
+qapi.gen module
+===
+
+.. automodule:: qapi.gen
+   :members:
+   :undoc-members:
+   :show-inheritance:
diff --git a/docs/devel/python/qapi.introspect.rst 
b/docs/devel/python/qapi.introspect.rst
new file mode 100644
index 000..f65ebfccd1b
--- /dev/null
+++ b/docs/devel/python/qapi.introspect.rst
@@ -0,0 +1,7 @@
+qapi.introspect module
+==
+
+.. automodule:: qapi.introspect
+ 

[PATCH v4 01/46] [DO-NOT-MERGE] docs: replace single backtick (`) with double-backtick (``)

[John Snow]

The single backtick in ReST is the "default role". Currently, Sphinx's
default role is called "content". Sphinx suggests you can use the "Any"
role instead to turn any single-backtick enclosed item into a
cross-reference.

Before we do that, though, we'll need to turn all existing usages of the
"content" role to inline verbatim markup by using double backticks
instead.

Signed-off-by: John Snow 
---
 docs/devel/build-system.rst| 118 -
 docs/devel/migration.rst   |  59 +++--
 docs/devel/tcg-plugins.rst |  14 +--
 docs/devel/testing.rst |   2 +-
 docs/interop/live-block-operations.rst |   4 +-
 docs/system/arm/cpu-features.rst   | 110 +++
 docs/system/arm/nuvoton.rst|   2 +-
 docs/system/s390x/protvirt.rst |  10 +--
 qapi/block-core.json   |   4 +-
 9 files changed, 163 insertions(+), 160 deletions(-)

diff --git a/docs/devel/build-system.rst b/docs/devel/build-system.rst
index 08e85c69e1f..a4711cd4e1a 100644
--- a/docs/devel/build-system.rst
+++ b/docs/devel/build-system.rst
@@ -53,14 +53,14 @@ following tasks:
  - Add a Meson build option to meson_options.txt.
 
  - Add support to the command line arg parser to handle any new
-   `--enable-XXX`/`--disable-XXX` flags required by the feature.
+   ``--enable-XXX``/``--disable-XXX`` flags required by the feature.
 
  - Add information to the help output message to report on the new
feature flag.
 
  - Add code to perform the actual feature check.
 
- - Add code to include the feature status in `config-host.h`
+ - Add code to include the feature status in ``config-host.h``
 
  - Add code to print out the feature status in the configure summary
upon completion.
@@ -116,51 +116,51 @@ Helper functions
 The configure script provides a variety of helper functions to assist
 developers in checking for system features:
 
-`do_cc $ARGS...`
+``do_cc $ARGS...``
Attempt to run the system C compiler passing it $ARGS...
 
-`do_cxx $ARGS...`
+``do_cxx $ARGS...``
Attempt to run the system C++ compiler passing it $ARGS...
 
-`compile_object $CFLAGS`
+``compile_object $CFLAGS``
Attempt to compile a test program with the system C compiler using
$CFLAGS. The test program must have been previously written to a file
-   called $TMPC.  The replacement in Meson is the compiler object `cc`,
-   which has methods such as `cc.compiles()`,
-   `cc.check_header()`, `cc.has_function()`.
+   called $TMPC.  The replacement in Meson is the compiler object ``cc``,
+   which has methods such as ``cc.compiles()``,
+   ``cc.check_header()``, ``cc.has_function()``.
 
-`compile_prog $CFLAGS $LDFLAGS`
+``compile_prog $CFLAGS $LDFLAGS``
Attempt to compile a test program with the system C compiler using
$CFLAGS and link it with the system linker using $LDFLAGS. The test
program must have been previously written to a file called $TMPC.
-   The replacement in Meson is `cc.find_library()` and `cc.links()`.
+   The replacement in Meson is ``cc.find_library()`` and ``cc.links()``.
 
-`has $COMMAND`
+``has $COMMAND``
Determine if $COMMAND exists in the current environment, either as a
shell builtin, or executable binary, returning 0 on success.  The
-   replacement in Meson is `find_program()`.
+   replacement in Meson is ``find_program()``.
 
-`check_define $NAME`
+``check_define $NAME``
Determine if the macro $NAME is defined by the system C compiler
 
-`check_include $NAME`
+``check_include $NAME``
Determine if the include $NAME file is available to the system C
-   compiler.  The replacement in Meson is `cc.has_header()`.
+   compiler.  The replacement in Meson is ``cc.has_header()``.
 
-`write_c_skeleton`
+``write_c_skeleton``
Write a minimal C program main() function to the temporary file
indicated by $TMPC
 
-`feature_not_found $NAME $REMEDY`
+``feature_not_found $NAME $REMEDY``
Print a message to stderr that the feature $NAME was not available
on the system, suggesting the user try $REMEDY to address the
problem.
 
-`error_exit $MESSAGE $MORE...`
+``error_exit $MESSAGE $MORE...``
Print $MESSAGE to stderr, followed by $MORE... and then exit from the
configure script with non-zero status
 
-`query_pkg_config $ARGS...`
+``query_pkg_config $ARGS...``
Run pkg-config passing it $ARGS. If QEMU is doing a static build,
then --static will be automatically added to $ARGS
 
@@ -194,8 +194,8 @@ to list the files and their dependency on various 
configuration
 symbols.
 
 Various subsystems that are common to both tools and emulators have
-their own sourceset, for example `block_ss` for the block device subsystem,
-`chardev_ss` for the character device subsystem, etc.  These sourcesets
+their own sourceset, for example ``block_ss`` for the block device subsystem,
+``chardev_ss`` for the character device subsystem, etc.  These sourcesets
 are then turned into static libraries as 

[PATCH v4 09/46] qapi: Prefer explicit relative imports

[John Snow]

All of the QAPI include statements are changed to be package-aware, as
explicit relative imports.

A quirk of Python packages is that the name of the package exists only
*outside* of the package. This means that to a module inside of the qapi
folder, there is inherently no such thing as the "qapi" package. The
reason these imports work is because the "qapi" package exists in the
context of the caller -- the execution shim, where sys.path includes a
directory that has a 'qapi' folder in it.

When we write "from qapi import sibling", we are NOT referencing the folder
'qapi', but rather "any package named qapi in sys.path". If you should
so happen to have a 'qapi' package in your path, it will use *that*
package.

When we write "from .sibling import foo", we always reference explicitly
our sibling module; guaranteeing consistency in *where* we are importing
these modules from.

This can be useful when working with virtual environments and packages
in development mode. In development mode, a package is installed as a
series of symlinks that forwards to your same source files. The problem
arises because code quality checkers will follow "import qapi.x" to the
"installed" version instead of the sibling file and -- even though they
are the same file -- they have different module paths, and this causes
cyclic import problems, false positive type mismatch errors, and more.

It can also be useful when dealing with hierarchical packages, e.g. if
we allow qemu.core.qmp, qemu.qapi.parser, etc.

Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Cleber Rosa 
---
 scripts/qapi/commands.py   |  4 ++--
 scripts/qapi/events.py |  8 
 scripts/qapi/expr.py   |  4 ++--
 scripts/qapi/gen.py|  4 ++--
 scripts/qapi/introspect.py |  8 
 scripts/qapi/main.py   | 14 +++---
 scripts/qapi/parser.py |  4 ++--
 scripts/qapi/schema.py |  8 
 scripts/qapi/types.py  |  6 +++---
 scripts/qapi/visit.py  |  6 +++---
 10 files changed, 33 insertions(+), 33 deletions(-)

diff --git a/scripts/qapi/commands.py b/scripts/qapi/commands.py
index 3cf9e1110b2..ce5926146a4 100644
--- a/scripts/qapi/commands.py
+++ b/scripts/qapi/commands.py
@@ -13,8 +13,8 @@
 See the COPYING file in the top-level directory.
 """
 
-from qapi.common import *
-from qapi.gen import QAPIGenCCode, QAPISchemaModularCVisitor, ifcontext
+from .common import *
+from .gen import QAPIGenCCode, QAPISchemaModularCVisitor, ifcontext
 
 
 def gen_command_decl(name, arg_type, boxed, ret_type):
diff --git a/scripts/qapi/events.py b/scripts/qapi/events.py
index b544af5a1ce..04672724388 100644
--- a/scripts/qapi/events.py
+++ b/scripts/qapi/events.py
@@ -12,10 +12,10 @@
 See the COPYING file in the top-level directory.
 """
 
-from qapi.common import *
-from qapi.gen import QAPISchemaModularCVisitor, ifcontext
-from qapi.schema import QAPISchemaEnumMember
-from qapi.types import gen_enum, gen_enum_lookup
+from .common import *
+from .gen import QAPISchemaModularCVisitor, ifcontext
+from .schema import QAPISchemaEnumMember
+from .types import gen_enum, gen_enum_lookup
 
 
 def build_event_send_proto(name, arg_type, boxed):
diff --git a/scripts/qapi/expr.py b/scripts/qapi/expr.py
index 2942520399a..03b31ecfc19 100644
--- a/scripts/qapi/expr.py
+++ b/scripts/qapi/expr.py
@@ -16,8 +16,8 @@
 
 import re
 from collections import OrderedDict
-from qapi.common import c_name
-from qapi.error import QAPISemError
+from .common import c_name
+from .error import QAPISemError
 
 
 # Names must be letters, numbers, -, and _.  They must start with letter,
diff --git a/scripts/qapi/gen.py b/scripts/qapi/gen.py
index fc19b2aeb9b..14d584680dc 100644
--- a/scripts/qapi/gen.py
+++ b/scripts/qapi/gen.py
@@ -17,8 +17,8 @@
 import re
 from contextlib import contextmanager
 
-from qapi.common import *
-from qapi.schema import QAPISchemaVisitor
+from .common import *
+from .schema import QAPISchemaVisitor
 
 
 class QAPIGen:
diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
index 23652be8102..2a34cd1e8ea 100644
--- a/scripts/qapi/introspect.py
+++ b/scripts/qapi/introspect.py
@@ -10,10 +10,10 @@
 See the COPYING file in the top-level directory.
 """
 
-from qapi.common import *
-from qapi.gen import QAPISchemaMonolithicCVisitor
-from qapi.schema import (QAPISchemaArrayType, QAPISchemaBuiltinType,
- QAPISchemaType)
+from .common import *
+from .gen import QAPISchemaMonolithicCVisitor
+from .schema import (QAPISchemaArrayType, QAPISchemaBuiltinType,
+ QAPISchemaType)
 
 
 def _make_tree(obj, ifcond, features, extra=None):
diff --git a/scripts/qapi/main.py b/scripts/qapi/main.py
index 9210a0e1a80..1b168a9771a 100644
--- a/scripts/qapi/main.py
+++ b/scripts/qapi/main.py
@@ -11,13 +11,13 @@
 import re
 import sys
 
-from qapi.commands import gen_commands
-from qapi.error import QAPIError
-from qapi.events import gen_events
-from qapi.introspect import gen_introspect

[PATCH v4 03/46] [DO-NOT-MERGE] docs/sphinx: change default role to "any"

[John Snow]

Signed-off-by: John Snow 
---
 docs/conf.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/docs/conf.py b/docs/conf.py
index 606f623211d..49599e38314 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -84,6 +84,9 @@
 # The master toctree document.
 master_doc = 'index'
 
+# Interpret `this` to be a cross-reference to "anything".
+default_role = 'any'
+
 # General information about the project.
 project = u'QEMU'
 copyright = u'2020, The QEMU Project Developers'
-- 
2.26.2

[PATCH v4 02/46] docs: repair broken references

[John Snow]

In two different places, we are not making a cross-reference to some
resource correctly.

Signed-off-by: John Snow 
---
 docs/devel/multi-thread-tcg.rst | 2 +-
 docs/devel/testing.rst  | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/docs/devel/multi-thread-tcg.rst b/docs/devel/multi-thread-tcg.rst
index 21483870dbc..92a9eba13c9 100644
--- a/docs/devel/multi-thread-tcg.rst
+++ b/docs/devel/multi-thread-tcg.rst
@@ -267,7 +267,7 @@ of view of external observers (e.g. another processor 
core). They can
 apply to any memory operations as well as just loads or stores.
 
 The Linux kernel has an excellent `write-up
-`
+`_
 on the various forms of memory barrier and the guarantees they can
 provide.
 
diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 666c4d72408..dfeda820482 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -953,7 +953,7 @@ compiler flags are needed to build for a given target.
 If you have the ability to run containers as the user you can also
 take advantage of the build systems "Docker" support. It will then use
 containers to build any test case for an enabled guest where there is
-no system compiler available. See :ref: `_docker-ref` for details.
+no system compiler available. See :ref:`docker-ref` for details.
 
 Running subset of tests
 ---
-- 
2.26.2

[PATCH v4 04/46] qapi: modify docstrings to be sphinx-compatible

[John Snow]

I did not say "sphinx beautiful", just "sphinx compatible". They will
not throw errors when parsed and interpreted as ReST.

Signed-off-by: John Snow 
---
 scripts/qapi/gen.py| 6 --
 scripts/qapi/parser.py | 9 +
 2 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/scripts/qapi/gen.py b/scripts/qapi/gen.py
index ca66c82b5b8..fc19b2aeb9b 100644
--- a/scripts/qapi/gen.py
+++ b/scripts/qapi/gen.py
@@ -154,9 +154,11 @@ def _bottom(self):
 
 @contextmanager
 def ifcontext(ifcond, *args):
-"""A 'with' statement context manager to wrap with start_if()/end_if()
+"""
+A 'with' statement context manager that wraps with `start_if` and `end_if`.
 
-*args: any number of QAPIGenCCode
+:param ifcond: List of conditionals
+:param args: any number of `QAPIGenCCode`.
 
 Example::
 
diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index 9d1a3e2eea9..02983979965 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -381,10 +381,11 @@ def append(self, line):
 
 The way that the line is dealt with depends on which part of
 the documentation we're parsing right now:
-* The body section: ._append_line is ._append_body_line
-* An argument section: ._append_line is ._append_args_line
-* A features section: ._append_line is ._append_features_line
-* An additional section: ._append_line is ._append_various_line
+
+ * The body section: ._append_line is ._append_body_line
+ * An argument section: ._append_line is ._append_args_line
+ * A features section: ._append_line is ._append_features_line
+ * An additional section: ._append_line is ._append_various_line
 """
 line = line[1:]
 if not line:
-- 
2.26.2

[PATCH v4 00/46] qapi: static typing conversion, pt1

[John Snow]

Based-on: <20200925162316.21205-1-peter.mayd...@linaro.org>

Hi, this series adds static type hints to the QAPI module.
This is part one!

Part 1: https://gitlab.com/jsnow/qemu/-/tree/python-qapi-cleanup-pt1
Everything: https://gitlab.com/jsnow/qemu/-/tree/python-qapi-cleanup-pt6

- Requires Python 3.6+
- Requires mypy 0.770 or newer (for type analysis only)
- Requires pylint 2.6.0 or newer (for lint checking only)

In general, this series tackles the cleanup of one individual QAPI
module at a time. Once it passes pylint or mypy checks, those checks are
enabled for that file.

Type hints are added in patches that add *only* type hints and change no
other behavior. Any necessary changes to behavior to accommodate typing
are split out into their own tiny patches.

Notes:

- `make sphinxdocs` should work on every commit. It begins to include
  docstring content after the DO-NOT-MERGE patch 5.

- After patch 11, `isort -c` should pass 100% on this and every
  future commit.

- After patch 12, `flake8 qapi/` should pass 100% on this and every
  future commit.

- After patch 13, `pylint --rcfile=3Dqapi/pylintrc qapi/` should pass 100%
  on this and every future commit.

- After patch 22, `mypy --config-file=3Dqapi/mypy.ini qapi/` should pass
  100% on this and every future commit.

V4:
 - Rebase on Peter Maydell's work;
  - 05: Context differences from Peter Maydell's work
  - 06: Remove qapi.doc
  - 07: Remove qapi.doc, remove superfluous "if match"
  - 09: Remove qapi.doc
  - 13: remove qapi.doc
  - 18: remove qapi.doc
  - 22: remove qapi.doc
  - 31: remove QAPIGenDoc changes

 - Minor adjustments from list feedback;
  - 01: More backticks for QAPI json files, now that they are in Sphinx-exe
  - 02: Use :ref: role for the `docker-ref` cross-reference
  - 04: Remove doc.py work changes; add references for start_if/end_if
  - 10: Changes to accommodate isort
  - 11: Added lines_after_imports=3D2
  - 16: isort whitespace changes
  - 24: Take Markus's docstring phrasing
  - 34: add comment explaining os.open()
  - 37: isort differences

Status:

(This is my stgit summary with reviewer tags visible.)

[01] do-not-merge-docs-replace  #
[02] docs-repair-broken-references  #
[03] docs-sphinx-change-default #
[04] qapi-modify-docstrings-to-be   #
[05] do-not-merge-docs-enable   #
[06] qapi-gen-separate-arg-parsing  # [SOB] JS [RB] CR,EH [TB] CR
[07] qapi-move-generator-entrypoint # [SOB] JS [RB] CR,EH [TB] CR
[08] do-not-merge-more-apidoc   #
[09] qapi-prefer-explicit-relative  # [SOB] JS [RB] CR,EH
[10] qapi-remove-wildcard-includes  # [SOB] JS [RB] CR,EH
[11] qapi-enforce-import-order  # [SOB] JS [RB] CR [TB] CR
[12] qapi-delint-using-flake8   # [SOB] JS [RB] CR,EH
[13] qapi-add-pylintrc  # [SOB] JS [TB] CR,EH [RB] CR
[14] qapi-common-py-remove-python   # [SOB] JS [RB] CR,EH
[15] qapi-common-add-indent-manager # [SOB] JS [RB] CR,EH
[16] qapi-common-py-delint-with # [SOB] JS [RB] CR,EH
[17] replace-c-by-char  # [SOB] JS [RB] CR,EH
[18] qapi-common-py-check-with  # [SOB] JS [RB] CR [TB] CR,EH
[19] qapi-common-py-add-notational  # [SOB] JS [RB] CR,EH
[20] qapi-common-move-comments-into # [SOB] JS [RB] CR,EH
[21] qapi-split-build_params-into   # [SOB] JS [RB] CR,EH
[22] qapi-establish-mypy-type   # [SOB] JS [TB] CR,EH [RB] CR
[23] qapi-events-py-add-notational  # [SOB] JS [RB] CR,EH
[24] qapi-events-move-comments-into # [SOB] JS [RB] CR,EH
[25] qapi-commands-py-don-t-re-bind # [SOB] JS [RB] CR,EH
[26] qapi-commands-py-add   # [SOB] JS [RB] CR,EH
[27] qapi-commands-py-enable# [SOB] JS [RB] CR,EH
[28] qapi-source-py-add-notational  # [SOB] JS [RB] CR,EH [TB] CR
[29] qapi-source-py-delint-with # [SOB] JS [RB] CR,EH [TB] CR
[30] qapi-gen-py-fix-edge-case-of   # [SOB] JS [RB] CR
[31] qapi-gen-py-add-notational # [SOB] JS [RB] CR,EH
[32] qapi-gen-py-enable-checking# [SOB] JS [RB] CR,EH [TB] CR
[33] qapi-gen-py-remove-unused  # [SOB] JS [RB] CR,EH
[34] qapi-gen-py-update-write-to-be # [SOB] JS [RB] CR,EH
[35] qapi-gen-py-delint-with-pylint # [SOB] JS [RB] CR,EH
[36] qapi-introspect-py-assert-obj  #
[37] qapi-introspect-py-create-a# [SOB] EH,JS
[38] qapi-introspect-py-add #
[39] qapi-introspect-py-unify   #
[40] qapi-introspect-py-replace #
[41] qapi-introspect-py-create-a-0  #
[42] qapi-types-py-add-type-hint# [SOB] JS [RB] CR,EH
[43] qapi-types-py-remove-one   # [SOB] JS [RB] CR,EH
[44] qapi-visit-py-assert   # [SOB] JS [RB] CR,EH
[45] qapi-visit-py-remove-unused# [SOB] JS [RB] CR,EH [TB] CR
[46] qapi-visit-py-add-notational   # [SOB] JS [RB] CR,EH [TB] CR

Changelog:

(Patches without changes elided)

001/46:[0004] [FC] '[DO-NOT-MERGE] docs: replace single backtick (`) with dou=
ble-backtick (``)'
002/46:[0002] [FC] 'docs: repair broken references'
004/46:[0014] [FC] 'qapi: modify docstrings to be sphinx-compatible'
005/46:[0015] [FC] '[DO-NOT-MERGE] docs: enable sphinx-autodoc for scripts/qa=

Re: [PATCH v2] hw/ide: check null block before _cancel_dma_sync

[P J P]

+-- On Tue, 29 Sep 2020, Li Qiang wrote --+
| P J P  于2020年9月29日周二 下午2:22写道：
| > +-- On Fri, 18 Sep 2020, Li Qiang wrote --+
| > | P J P  于2020年9月18日周五 下午6:26写道：
| > | > +-- On Fri, 18 Sep 2020, Li Qiang wrote --+
| > | > | Update v2: use an assert() call
| > | > |   
->https://lists.nongnu.org/archive/html/qemu-devel/2020-08/msg08336.html
| > |
| > | In 'ide_ioport_write' the guest can set 'bus->unit' to 0 or 1 by issue
| > | 'ATA_IOPORT_WR_DEVICE_HEAD'. So this case the guest can set the active 
ifs.
| > | If the guest set this to 1.
| > |
| > | Then in 'idebus_active_if' will return 'IDEBus.ifs[1]' and thus the 
's->blk'
| > | will be NULL.
| >
| > Right, guest does select the drive via
| >
| >   portio_write
| >->ide_ioport_write
| >   case ATA_IOPORT_WR_DEVICE_HEAD:
| >   /* FIXME: HOB readback uses bit 7 */
| >   bus->ifs[0].select = (val & ~0x10) | 0xa0;
| >   bus->ifs[1].select = (val | 0x10) | 0xa0;
| >   /* select drive */
| >   bus->unit = (val >> 4) & 1; <== set bus->unit=0x1
| >   break;
| >
| >
| > | So from your (Peter's) saying, we need to check the value in
| > | 'ATA_IOPORT_WR_DEVICE_HEAD' handler. To say if the guest
| > | set a valid 'bus->unit'. This can also work I think.
| >
| > Yes, with the following fix, an assert(3) in ide_cancel_dma_sync fails.
| >
| > ===
| > diff --git a/hw/ide/core.c b/hw/ide/core.c
| > index f76f7e5234..cb55cc8b0f 100644
| > --- a/hw/ide/core.c
| > +++ b/hw/ide/core.c
| > @@ -1300,7 +1300,11 @@ void ide_ioport_write(void *opaque, uint32_t addr,
| > uint_)
| >  bus->ifs[0].select = (val & ~0x10) | 0xa0;
| >  bus->ifs[1].select = (val | 0x10) | 0xa0;
| >  /* select drive */
| > +uint8_t bu = bus->unit;
| >  bus->unit = (val >> 4) & 1;
| > +if (!bus->ifs[bus->unit].blk) {
| > +bus->unit = bu;
| > +}
| >  break;
| >  default:
| >
| > qemu-system-x86_64: ../hw/ide/core.c:724: ide_cancel_dma_sync: Assertion 
`s->bus->dma->aiocb == NULL' failed.
| > Aborted (core dumped)
| 
| This is what I am worried, in the 'ide_ioport_write' set the 'bus->unit'. It 
| also change the 'buf->ifs[0].select'. Also there maybe some other corner 
| case that causes some inconsistent. And if we choice this method we need to 
| deep into the more ahci-spec to know how things really going.
| 
| > ===
| >
| > | As we the 'ide_exec_cmd' and other functions in 'hw/ide/core.c' check the
| > | 's->blk' directly. I think we just check it in 'ide_cancel_dma_sync' is
| > | enough and also this is more consistent with the other functions.
| > | 'ide_cancel_dma_sync' is also called by 'cmd_device_reset' which is one of
| > | the 'ide_cmd_table' handler.
| >
| >   Yes, I'm okay with either approach. Earlier patch v1 checks 's->blk' in
| > ide_cancel_dma_sync().
| 
| I prefer the 'check the s->blk in the beginning of ide_cancel_dma_sync' 
method.
| Some little different with your earlier patch.
| 
| Anyway, let the maintainer do the choices.
| 

@John ...wdyt?

Thank you.
--
Prasad J Pandit / Red Hat Product Security Team
8685 545E B54C 486B C6EB 271E E285 8B5A F050 DE8D

Re: [RFC PATCH v4 00/29] Hexagon patch series

[Brad Smith]

On 9/29/2020 1:01 PM, Philippe Mathieu-Daudé wrote:

On 9/29/20 5:53 PM, Taylor Simpson wrote:

-Original Message-
From: Philippe Mathieu-Daudé  On
Behalf Of Philippe Mathieu-Daudé
Sent: Tuesday, September 29, 2020 6:22 AM
To: Taylor Simpson ; qemu-devel@nongnu.org
Cc: a...@rev.ng; riku.voi...@iki.fi; richard.hender...@linaro.org;
laur...@vivier.eu; aleksandar.m.m...@gmail.com
Subject: Re: [RFC PATCH v4 00/29] Hexagon patch series

cc1: all warnings being treated as errors
make: *** [Makefile.ninja:638:
libqemu-hexagon-linux-user.fa.p/target_hexagon_decode.c.o] Error 1

$ gcc --version
gcc (GCC) 10.2.1 20200723 (Red Hat 10.2.1-1)


Thanks for all your feedback.  I really appreciate it and will make the changes 
you mentioned.

No problem, I also appreciate the effort you did to address all
of the previous issues :)


I'm using an older GCC that doesn't have these errors.  Is this the version of 
GCC that is recommended (mandated?) for building qemu?

QEMU aims to support the 2 latest releases of supported distributions.
 From time to time a brave developer look at the different versions
packaged and make some cleanup in the code base. It used to be tedious,
now that repology.org exists it is a bit easier.

The last effort is from Thomas, see commit efc6c070aca:

 The supported distributions use the following version
 of GCC:

   RHEL-7: 4.8.5
   Debian (Stretch): 6.3.0
   Debian (Jessie): 4.8.4
   OpenBSD (ports): 4.9.4


OpenBSD as of 6.6 uses GCC 8.3.0 for our ports-gcc.


   FreeBSD (ports): 8.2.0
   OpenSUSE Leap 15: 7.3.1
   Ubuntu (Xenial): 5.3.1
   macOS (Homebrew): 8.2.0

 So we can safely assume GCC 4.8 these days.

FreeBSD and OpenBSD nowadays use Clang for the (system) compiler.


This is the "mandated" compiler version.


QEMU has some CI jobs, see:
https://wiki.qemu.org/Testing/CI

You can use most of them by opening GitLab and Travis/Cirrus
(for GitHub, which you already use).

GitLab will become our "gating CI" soon, so your series is
expected to pass all the GitLab jobs. IIRC running the tests
is as easy as register and push your branch to your account.


PS  You were right about Richard recommending const.  It's already on my TODO 
list from his review 


=)

Regards,

Phil.


Thanks,
Taylor

[Bug 1849644] Autopkgtest regression report (qemu/1:4.2-3ubuntu6.7)

[Ubuntu SRU Bot]

All autopkgtests for the newly accepted qemu (1:4.2-3ubuntu6.7) for focal have 
finished running.
The following regressions have been reported in tests triggered by the package:

casper/1.445.1 (amd64)


Please visit the excuses page listed below and investigate the failures, 
proceeding afterwards as per the StableReleaseUpdates policy regarding 
autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-
migration/focal/update_excuses.html#qemu

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1849644

Title:
  QEMU VNC websocket proxy requires non-standard 'binary' subprotocol

Status in QEMU:
  Fix Released
Status in qemu package in Ubuntu:
  Fix Released
Status in qemu source package in Focal:
  Fix Committed

Bug description:
  [Impact]

   * The exact details of the protocol/subprotocal was slightly unclear
 between the projects. So qemu ended up insisting on "binary" being
 used but newer noVNC clients no more used it.

   * Qemu got fixed in 5.0 to be more tolerant and accept an empty sub-
 protocol as well. This SRU backports that fix to Focal.

  [Test Case]

   * Without the fix the following will "Failed to connect", but with
  the fix it will work.

  $ sudo apt install qemu-system-x86
  # will only boot into a failing bootloader, but that is enough
  $ qemu-system-x86_64 -vnc :0,websocket
  # We need version 1.2 or later, so use the snap
  $ snap install novnc
  $ novnc --vnc localhost:5700
  Connect browser to http://:6080/vnc.html
  Click "Connect"

   * Cross check with an older noVNC (e.g. the one in Focal) if the 
 connectivity still works on those as well

 - Reminders when switching between the noVNC implementations
   - always refresh the browser with all clear ctrl+alt+f5
   - start/stop the snapped one via snap.novnc.novncsvc.service

  [Regression Potential]

   * This is exclusive to the functionality of noVNC, so regressions would 
 have to be expected in there. The tests try to exercise the basics, but
 e.g. Openstack would be a major product using 

  [Other Info]
   
   * The noVNC in Focal itself does not yet have the offending change, but
 we want the qemu in focal to be connecteable from ~any type of client


  ---


  
  When running a machine using "-vnc" and the "websocket" option QEMU seems to 
require the subprotocol called 'binary'. This subprotocol does not exist in the 
WebSocket specification. In fact it has never existed in the spec, in one of 
the very early drafts of WebSockets it was briefly mentioned but it never made 
it to a final version.

  When the WebSocket server requires a non-standard subprotocol any
  WebSocket client that works correctly won't be able to connect.

  One example of such a client is noVNC, it tells the server that it
  doesn't want to use any subprotocol. QEMU's WebSocket proxy doesn't
  let noVNC connect. If noVNC is modified to ask for 'binary' it will
  work, this is, however, incorrect behavior.

  Looking at the code in "io/channel-websock.c" it seems it's quite
  hard-coded to binary:

  Look at line 58 and 433 here:
  https://git.qemu.org/?p=qemu.git;a=blob;f=io/channel-websock.c

  This code has to be made more dynamic, and shouldn't require binary.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1849644/+subscriptions

Re: [Qemu-devel PATCH v2] target/i386: add "-cpu,lbr-fmt=*" support to enable guest LBR

[Like Xu]

Hi Eduardo,

On 2020/9/30 1:38, Eduardo Habkost wrote:

(CCing the people from the thread, as kvm_exact_match_flags would
be useful for INTEL_PT_IP_LIP)

On Tue, Sep 29, 2020 at 02:12:17PM +0800, Like Xu wrote:

The last branch recording (LBR) is a performance monitor unit (PMU)
feature on Intel processors that records a running trace of the most
recent branches taken by the processor in the LBR stack. The QEMU
could configure whether it's enabled or not for each guest via CLI.

The LBR feature would be enabled on the guest if:
- the KVM is enabled and the PMU is enabled and,
- the msr-based-feature IA32_PERF_CAPABILITIES is supporterd on KVM and,
- the supported returned value for lbr_fmt from this msr is not zero and,
- the requested guest vcpu model does support FEAT_1_ECX.CPUID_EXT_PDCM,
- the configured lbr-fmt value is the same as the host lbr_fmt value.

Cc: Eduardo Habkost 
Cc: Paolo Bonzini 
Signed-off-by: Like Xu 


The approach below looks better, thanks!  Only one problem below,
with a few suggestions and questions:


---
  target/i386/cpu.c | 16 
  target/i386/cpu.h | 10 ++
  2 files changed, 26 insertions(+)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 3ffd877dd5..b10344be01 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -6461,6 +6461,13 @@ static void x86_cpu_filter_features(X86CPU *cpu, bool 
verbose)
  x86_cpu_get_supported_feature_word(w, false);
  uint64_t requested_features = env->features[w];
  uint64_t unavailable_features = requested_features & ~host_feat;
+if (w == FEAT_PERF_CAPABILITIES &&
+(requested_features & PERF_CAP_LBR_FMT)) {
+if ((host_feat & PERF_CAP_LBR_FMT) !=
+(requested_features & PERF_CAP_LBR_FMT)) {
+unavailable_features |= PERF_CAP_LBR_FMT;
+}
+}


This looks correct, but needs to be conditional on kvm_enabled().

I also have a suggestion: instead of hardcoding the
PERF_CAPABILITIES rules in this loop, this could become a
FeatureWordInfo field.  It would be very useful for other
features like intel-pt, where we need some bits to match the host
too.


The idea looks good to me.



Could you please check if the following patch works?

Signed-off-by: Eduardo Habkost 
---
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index b10344be010..d4107dcc026 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -704,6 +704,8 @@ typedef struct FeatureWordInfo {
  uint64_t migratable_flags; /* Feature flags known to be migratable */
  /* Features that shouldn't be auto-enabled by "-cpu host" */
  uint64_t no_autoenable_flags;
+/* Bits that must match host exactly when using KVM */
+uint64_t kvm_exact_match_flags;
  } FeatureWordInfo;
  
  static FeatureWordInfo feature_word_info[FEATURE_WORDS] = {

@@ -1143,6 +1145,11 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] 
= {
  .msr = {
  .index = MSR_IA32_PERF_CAPABILITIES,
  },
+/*
+ * KVM is not able to emulate a VCPU with LBR_FMT different
+ * from the host, so LBR_FMT must match the host exactly.
+ */
+.kvm_exact_match_flags = PERF_CAP_LBR_FMT,
  },
  
  [FEAT_VMX_PROCBASED_CTLS] = {

@@ -6457,16 +6464,15 @@ static void x86_cpu_filter_features(X86CPU *cpu, bool 
verbose)
  }
  
  for (w = 0; w < FEATURE_WORDS; w++) {

+FeatureWordInfo *fi = _word_info[w];
  uint64_t host_feat =
  x86_cpu_get_supported_feature_word(w, false);
  uint64_t requested_features = env->features[w];
  uint64_t unavailable_features = requested_features & ~host_feat;
-if (w == FEAT_PERF_CAPABILITIES &&
-(requested_features & PERF_CAP_LBR_FMT)) {
-if ((host_feat & PERF_CAP_LBR_FMT) !=
-(requested_features & PERF_CAP_LBR_FMT)) {
-unavailable_features |= PERF_CAP_LBR_FMT;
-}
+if (kvm_enabled()) {
+uint64_t mismatches = (requested_features ^ host_feat) &
+  fi->kvm_exact_match_flags;
+mark_unavailable_features(cpu, w, mismatches, "feature doesn't match 
host");
  }
  mark_unavailable_features(cpu, w, unavailable_features, prefix);
  }
   
---


I may refine this part in this way:

for (w = 0; w < FEATURE_WORDS; w++) {
FeatureWordInfo *fi = _word_info[w];
uint64_t match_flags = fi->kvm_exact_match_flags;
uint64_t host_feat =
x86_cpu_get_supported_feature_word(w, false);
uint64_t requested_features = env->features[w];
uint64_t unavailable_features = requested_features & ~host_feat;
if (kvm_enabled() && match_flags) {
uint64_t mismatches = (requested_features & match_flags) &&
(requested_features ^ host_feat) & match_flags;
mark_unavailable_features(cpu, w, 

Re: [External] Re: [PULL 26/26] virtiofsd: Add -o allow_direct_io|no_allow_direct_io options

[Jiachen Zhang]

On Wed, Sep 30, 2020 at 5:53 AM Vivek Goyal  wrote:

> On Fri, Sep 25, 2020 at 01:06:55PM +0100, Dr. David Alan Gilbert (git)
> wrote:
> > From: Jiachen Zhang 
> >
> > Due to the commit 65da4539803373ec4eec97ffc49ee90083e56efd, the O_DIRECT
> > open flag of guest applications will be discarded by virtiofsd. While
> > this behavior makes it consistent with the virtio-9p scheme when guest
> > applications use direct I/O, we no longer have any chance to bypass the
> > host page cache.
> >
> > Therefore, we add a flag 'allow_direct_io' to lo_data. If '-o
> >  no_allow_direct_io' option is added, or none of '-o allow_direct_io' or
> >  '-o no_allow_direct_io' is added, the 'allow_direct_io' will be set to
> >  0, and virtiofsd discards O_DIRECT as before. If '-o allow_direct_io'
> > is added to the starting command-line, 'allow_direct_io' will be set to
> > 1, so that the O_DIRECT flags will be retained and host page cache can
> > be bypassed.
>
> Hi Jiachen,
>
> Curious that in what cases you want to bypass host page cache.
>
> Thanks
> Vivek
>

Hi Vivek,

Some apps like DBMS may allocate their own file cache in userspace, so they
may want to bypass kernel page cache by using O_DIRECT. When these apps are
running in guest and access virtio-fs files, we'd better obey these needs.
This can also eliminate the host memory usage.

Another case is when we perform file I/O benchmarks on different storage
devices (like HDD and SSD), it's not that fair to introduce ram caches. By
using "cache=none" with "allow_direct_io", we can bypass guest and host page
caches, and access the storage devices directly.

Best wishes,
Jiachen


> >
> > Signed-off-by: Jiachen Zhang 
> > Reviewed-by: Stefan Hajnoczi 
> > Message-Id: <20200824105957.61265-1-zhangjiachen.jay...@bytedance.com>
> > Signed-off-by: Dr. David Alan Gilbert 
> > ---
> >  tools/virtiofsd/helper.c |  4 
> >  tools/virtiofsd/passthrough_ll.c | 20 ++--
> >  2 files changed, 18 insertions(+), 6 deletions(-)
> >
> > diff --git a/tools/virtiofsd/helper.c b/tools/virtiofsd/helper.c
> > index 7bc5d7dc5a..85770d63f1 100644
> > --- a/tools/virtiofsd/helper.c
> > +++ b/tools/virtiofsd/helper.c
> > @@ -178,6 +178,10 @@ void fuse_cmdline_help(void)
> > "   (0 leaves rlimit
> unchanged)\n"
> > "   default: min(100,
> fs.file-max - 16384)\n"
> > "if the current
> rlimit is lower\n"
> > +   "-o allow_direct_io|no_allow_direct_io\n"
> > +   "   retain/discard O_DIRECT
> flags passed down\n"
> > +   "   to virtiofsd from guest
> applications.\n"
> > +   "   default:
> no_allow_direct_io\n"
> > );
> >  }
> >
> > diff --git a/tools/virtiofsd/passthrough_ll.c
> b/tools/virtiofsd/passthrough_ll.c
> > index 784330e0e4..0b229ebd57 100644
> > --- a/tools/virtiofsd/passthrough_ll.c
> > +++ b/tools/virtiofsd/passthrough_ll.c
> > @@ -151,6 +151,7 @@ struct lo_data {
> >  int timeout_set;
> >  int readdirplus_set;
> >  int readdirplus_clear;
> > +int allow_direct_io;
> >  struct lo_inode root;
> >  GHashTable *inodes; /* protected by lo->mutex */
> >  struct lo_map ino_map; /* protected by lo->mutex */
> > @@ -179,6 +180,8 @@ static const struct fuse_opt lo_opts[] = {
> >  { "cache=always", offsetof(struct lo_data, cache), CACHE_ALWAYS },
> >  { "readdirplus", offsetof(struct lo_data, readdirplus_set), 1 },
> >  { "no_readdirplus", offsetof(struct lo_data, readdirplus_clear), 1
> },
> > +{ "allow_direct_io", offsetof(struct lo_data, allow_direct_io), 1 },
> > +{ "no_allow_direct_io", offsetof(struct lo_data, allow_direct_io),
> 0 },
> >  FUSE_OPT_END
> >  };
> >  static bool use_syslog = false;
> > @@ -1516,7 +1519,8 @@ static void lo_releasedir(fuse_req_t req,
> fuse_ino_t ino,
> >  fuse_reply_err(req, 0);
> >  }
> >
> > -static void update_open_flags(int writeback, struct fuse_file_info *fi)
> > +static void update_open_flags(int writeback, int allow_direct_io,
> > +  struct fuse_file_info *fi)
> >  {
> >  /*
> >   * With writeback cache, kernel may send read requests even
> > @@ -1541,10 +1545,13 @@ static void update_open_flags(int writeback,
> struct fuse_file_info *fi)
> >
> >  /*
> >   * O_DIRECT in guest should not necessarily mean bypassing page
> > - * cache on host as well. If somebody needs that behavior, it
> > - * probably should be a configuration knob in daemon.
> > + * cache on host as well. Therefore, we discard it by default
> > + * ('-o no_allow_direct_io'). If somebody needs that behavior,
> > + * the '-o allow_direct_io' option should be set.
> >   */
> > -fi->flags &= ~O_DIRECT;
> > +if (!allow_direct_io) {
> > +fi->flags &= ~O_DIRECT;
> > +  

Re: [RFC 0/3] QEMU as IPMI BMC emulator

[Corey Minyard]

On Tue, Sep 29, 2020 at 06:05:16PM -0700, Havard Skinnemoen wrote:
> On Tue, Sep 29, 2020 at 10:46 AM Corey Minyard  wrote:
> >
> > On Mon, Sep 28, 2020 at 05:39:13PM -0700, Havard Skinnemoen via wrote:
> > > This series briefly documents the existing IPMI device support for main
> > > processor emulation, and goes on to propose a similar device structure to
> > > emulate IPMI responder devices in BMC machines. This would allow a qemu
> > > instance running BMC firmware to serve as an external BMC for a qemu 
> > > instance
> > > running server software.
> > >
> > > RFC only at this point because the series does not include actual code to
> > > implement this. I'd appreciate some initial feedback on
> > >
> > > 1. Whether anyone else is interested in something like this.
> >
> > Though I've had this idea once or twice, I'm not working on real BMCs,
> > so I didn't really pursue anything.  It's a good idea, I think, for the
> > BMC developers, and possibly for system developers trying to do
> > integration testing between BMCs and system software.
> >
> > You will need to tie in to more emulation than just the BMC side of the
> > system interface registers.  You will also need to tie into GPIOs or
> > whatnot for things like host reset.
> 
> That is true. The OpenIPMI protocol seems to handle at least some of
> that, so it should be just a matter of adding a few GPIO inputs
> (power, reset, ATTN, ...) to the ipmi-host-extern device.
> 
> I should add some more details about this to the doc.
> 
> > Power handling is going to be a bit weird.  The OpenIPMI emulator
> > starts/stops qemu based upon power control.  It might be possible to do
> > the same thing in this sort of emulator.
> 
> Hmm, yeah, I guess we can't kill/restart qemu from within qemu itself.
> But perhaps stopping all CPUs and doing a full system reset might be a
> good enough approximation for power-off?

This might be argument for keeping them in separate qemu processes.  But
it would be really cool if you could start up a single qemu and have a
BMC built-in running it's own code.  I'm sure something could be done to
simulate a power off.  If you can stop the CPUs, that's probably good
enough.

-corey

> 
> > You may need extensions to the protocol, and that's fine.  I can't think
> > of any at the moment, but you never know.
> 
> True.
> 
> > > 2. Completeness (i.e. anything that could be explained in more detail in 
> > > the
> > >docs).
> >
> > It's certainly a good start.  The second patch would be useful right
> > now.  There are more details, of course, but I think that's covered in
> > the man page under the various devices.
> 
> Thanks, I might send the second patch separately in the next round.
> 
> Havard
> 
> > Thanks,
> >
> > -corey
> >
> > > 3. Naming, and whether 'specs' is the right place to put this.
> > > 4. Whether it's OK to enable the blockdiag sphinx extension (if not, I'll 
> > > just
> > >toss the block diagrams and turn the docs into walls of text).
> > >
> > > If this seems reasonable, I'll start working with one of my team mates on
> > > implementing the common part, as well as the Nuvoton-specific responder 
> > > device.
> > > Possibly also an Aspeed device.
> > >
> > > Havard Skinnemoen (3):
> > >   docs: enable sphinx blockdiag extension
> > >   docs/specs: IPMI device emulation: main processor
> > >   docs/specs: IPMI device emulation: BMC
> > >
> > >  docs/conf.py |   5 +-
> > >  docs/specs/index.rst |   1 +
> > >  docs/specs/ipmi.rst  | 183 +++
> > >  3 files changed, 188 insertions(+), 1 deletion(-)
> > >  create mode 100644 docs/specs/ipmi.rst
> > >
> > > --
> > > 2.28.0.709.gb0816b6eb0-goog
> > >
> > >

Re: [PATCH 1/4] linux-user: update syscall_nr.h to Linux 5.9-rc7

[Alistair Francis]

On Tue, Sep 29, 2020 at 5:34 PM Laurent Vivier  wrote:
>
> Update gensyscalls.sh not to generate an empty line at the end of the file
>
> And then automatically update syscall_nr.h running scripts/gensyscalls.sh
>
> Signed-off-by: Laurent Vivier 

For the RISC-V part:

Reviewed-by: Alistair Francis 

Alistair

> ---
>  linux-user/aarch64/syscall_nr.h  | 7 +--
>  linux-user/nios2/syscall_nr.h| 7 +--
>  linux-user/openrisc/syscall_nr.h | 8 ++--
>  linux-user/riscv/syscall32_nr.h  | 8 +++-
>  linux-user/riscv/syscall64_nr.h  | 8 +++-
>  scripts/gensyscalls.sh   | 3 +--
>  6 files changed, 31 insertions(+), 10 deletions(-)
>
> diff --git a/linux-user/aarch64/syscall_nr.h b/linux-user/aarch64/syscall_nr.h
> index 85de000b2490..6fd5b331e780 100644
> --- a/linux-user/aarch64/syscall_nr.h
> +++ b/linux-user/aarch64/syscall_nr.h
> @@ -298,7 +298,10 @@
>  #define TARGET_NR_fspick 433
>  #define TARGET_NR_pidfd_open 434
>  #define TARGET_NR_clone3 435
> -#define TARGET_NR_syscalls 436
> +#define TARGET_NR_close_range 436
> +#define TARGET_NR_openat2 437
> +#define TARGET_NR_pidfd_getfd 438
> +#define TARGET_NR_faccessat2 439
> +#define TARGET_NR_syscalls 440
>
>  #endif /* LINUX_USER_AARCH64_SYSCALL_NR_H */
> -
> diff --git a/linux-user/nios2/syscall_nr.h b/linux-user/nios2/syscall_nr.h
> index 32d485dc9ae8..e37f40179bf3 100644
> --- a/linux-user/nios2/syscall_nr.h
> +++ b/linux-user/nios2/syscall_nr.h
> @@ -318,7 +318,10 @@
>  #define TARGET_NR_fsmount 432
>  #define TARGET_NR_fspick 433
>  #define TARGET_NR_pidfd_open 434
> -#define TARGET_NR_syscalls 436
> +#define TARGET_NR_close_range 436
> +#define TARGET_NR_openat2 437
> +#define TARGET_NR_pidfd_getfd 438
> +#define TARGET_NR_faccessat2 439
> +#define TARGET_NR_syscalls 440
>
>  #endif /* LINUX_USER_NIOS2_SYSCALL_NR_H */
> -
> diff --git a/linux-user/openrisc/syscall_nr.h 
> b/linux-user/openrisc/syscall_nr.h
> index 340383beb2c6..a8fc0295109a 100644
> --- a/linux-user/openrisc/syscall_nr.h
> +++ b/linux-user/openrisc/syscall_nr.h
> @@ -318,7 +318,11 @@
>  #define TARGET_NR_fsmount 432
>  #define TARGET_NR_fspick 433
>  #define TARGET_NR_pidfd_open 434
> -#define TARGET_NR_syscalls 436
> +#define TARGET_NR_clone3 435
> +#define TARGET_NR_close_range 436
> +#define TARGET_NR_openat2 437
> +#define TARGET_NR_pidfd_getfd 438
> +#define TARGET_NR_faccessat2 439
> +#define TARGET_NR_syscalls 440
>
>  #endif /* LINUX_USER_OPENRISC_SYSCALL_NR_H */
> -
> diff --git a/linux-user/riscv/syscall32_nr.h b/linux-user/riscv/syscall32_nr.h
> index 4fef73e954da..079b804daef5 100644
> --- a/linux-user/riscv/syscall32_nr.h
> +++ b/linux-user/riscv/syscall32_nr.h
> @@ -1,5 +1,7 @@
>  /*
>   * This file contains the system call numbers.
> + * Do not modify.
> + * This file is generated by scripts/gensyscalls.sh
>   */
>  #ifndef LINUX_USER_RISCV_SYSCALL32_NR_H
>  #define LINUX_USER_RISCV_SYSCALL32_NR_H
> @@ -290,6 +292,10 @@
>  #define TARGET_NR_fspick 433
>  #define TARGET_NR_pidfd_open 434
>  #define TARGET_NR_clone3 435
> -#define TARGET_NR_syscalls 436
> +#define TARGET_NR_close_range 436
> +#define TARGET_NR_openat2 437
> +#define TARGET_NR_pidfd_getfd 438
> +#define TARGET_NR_faccessat2 439
> +#define TARGET_NR_syscalls 440
>
>  #endif /* LINUX_USER_RISCV_SYSCALL32_NR_H */
> diff --git a/linux-user/riscv/syscall64_nr.h b/linux-user/riscv/syscall64_nr.h
> index cc82f3244f55..d54224ccec64 100644
> --- a/linux-user/riscv/syscall64_nr.h
> +++ b/linux-user/riscv/syscall64_nr.h
> @@ -1,5 +1,7 @@
>  /*
>   * This file contains the system call numbers.
> + * Do not modify.
> + * This file is generated by scripts/gensyscalls.sh
>   */
>  #ifndef LINUX_USER_RISCV_SYSCALL64_NR_H
>  #define LINUX_USER_RISCV_SYSCALL64_NR_H
> @@ -296,6 +298,10 @@
>  #define TARGET_NR_fspick 433
>  #define TARGET_NR_pidfd_open 434
>  #define TARGET_NR_clone3 435
> -#define TARGET_NR_syscalls 436
> +#define TARGET_NR_close_range 436
> +#define TARGET_NR_openat2 437
> +#define TARGET_NR_pidfd_getfd 438
> +#define TARGET_NR_faccessat2 439
> +#define TARGET_NR_syscalls 440
>
>  #endif /* LINUX_USER_RISCV_SYSCALL64_NR_H */
> diff --git a/scripts/gensyscalls.sh b/scripts/gensyscalls.sh
> index b7b8456f6312..bba9fb052c47 100755
> --- a/scripts/gensyscalls.sh
> +++ b/scripts/gensyscalls.sh
> @@ -86,8 +86,7 @@ generate_syscall_nr()
>  read_includes $arch $bits | filter_defines | rename_defines | \
>  evaluate_values | sort -n -k 3
>  echo
> -echo "#endif /* ${guard} */"
> -echo) > "$file"
> +echo "#endif /* ${guard} */") > "$file"
>  }
>
>  mkdir "$TMP/asm"
> --
> 2.26.2
>
>

Re: [RFC 0/3] QEMU as IPMI BMC emulator

[Havard Skinnemoen]

On Tue, Sep 29, 2020 at 10:46 AM Corey Minyard  wrote:
>
> On Mon, Sep 28, 2020 at 05:39:13PM -0700, Havard Skinnemoen via wrote:
> > This series briefly documents the existing IPMI device support for main
> > processor emulation, and goes on to propose a similar device structure to
> > emulate IPMI responder devices in BMC machines. This would allow a qemu
> > instance running BMC firmware to serve as an external BMC for a qemu 
> > instance
> > running server software.
> >
> > RFC only at this point because the series does not include actual code to
> > implement this. I'd appreciate some initial feedback on
> >
> > 1. Whether anyone else is interested in something like this.
>
> Though I've had this idea once or twice, I'm not working on real BMCs,
> so I didn't really pursue anything.  It's a good idea, I think, for the
> BMC developers, and possibly for system developers trying to do
> integration testing between BMCs and system software.
>
> You will need to tie in to more emulation than just the BMC side of the
> system interface registers.  You will also need to tie into GPIOs or
> whatnot for things like host reset.

That is true. The OpenIPMI protocol seems to handle at least some of
that, so it should be just a matter of adding a few GPIO inputs
(power, reset, ATTN, ...) to the ipmi-host-extern device.

I should add some more details about this to the doc.

> Power handling is going to be a bit weird.  The OpenIPMI emulator
> starts/stops qemu based upon power control.  It might be possible to do
> the same thing in this sort of emulator.

Hmm, yeah, I guess we can't kill/restart qemu from within qemu itself.
But perhaps stopping all CPUs and doing a full system reset might be a
good enough approximation for power-off?

> You may need extensions to the protocol, and that's fine.  I can't think
> of any at the moment, but you never know.

True.

> > 2. Completeness (i.e. anything that could be explained in more detail in the
> >docs).
>
> It's certainly a good start.  The second patch would be useful right
> now.  There are more details, of course, but I think that's covered in
> the man page under the various devices.

Thanks, I might send the second patch separately in the next round.

Havard

> Thanks,
>
> -corey
>
> > 3. Naming, and whether 'specs' is the right place to put this.
> > 4. Whether it's OK to enable the blockdiag sphinx extension (if not, I'll 
> > just
> >toss the block diagrams and turn the docs into walls of text).
> >
> > If this seems reasonable, I'll start working with one of my team mates on
> > implementing the common part, as well as the Nuvoton-specific responder 
> > device.
> > Possibly also an Aspeed device.
> >
> > Havard Skinnemoen (3):
> >   docs: enable sphinx blockdiag extension
> >   docs/specs: IPMI device emulation: main processor
> >   docs/specs: IPMI device emulation: BMC
> >
> >  docs/conf.py |   5 +-
> >  docs/specs/index.rst |   1 +
> >  docs/specs/ipmi.rst  | 183 +++
> >  3 files changed, 188 insertions(+), 1 deletion(-)
> >  create mode 100644 docs/specs/ipmi.rst
> >
> > --
> > 2.28.0.709.gb0816b6eb0-goog
> >
> >

[PATCH 0/4] linux-user: update syscall_nr headers to Linux 5.9-rc7

[Laurent Vivier]

All the changes have been done using:

scripts/gensyscalls.sh, scripts/update-mips-syscall-args.sh and
scripts/update-syscalltbl.sh

I've checked syscall_nr.h generated from syscall.tbl are
always correctly generated (there have been massive changes in
x86 targets).

The series also removes _sysctl from syscall.c as it has been
removed from the kernel (and we didn't emulate it correctly)

Laurent Vivier (4):
  linux-user: update syscall_nr.h to Linux 5.9-rc7
  linux-user: update mips/syscall-args-o32.c.inc to Linux 5.9-rc7
  linux-user: update syscall.tbl to Linux 5.9-rc7
  linux-user: remove _sysctl

 linux-user/aarch64/syscall_nr.h|   7 +-
 linux-user/alpha/syscall.tbl   |   4 +-
 linux-user/arm/syscall.tbl |   4 +-
 linux-user/hppa/syscall.tbl|   8 +-
 linux-user/i386/syscall_32.tbl | 820 +
 linux-user/m68k/syscall.tbl|   4 +-
 linux-user/microblaze/syscall.tbl  |   4 +-
 linux-user/mips/syscall-args-o32.c.inc |   4 +
 linux-user/mips/syscall_o32.tbl|   8 +-
 linux-user/mips64/syscall_n32.tbl  |   8 +-
 linux-user/mips64/syscall_n64.tbl  |   4 +-
 linux-user/nios2/syscall_nr.h  |   7 +-
 linux-user/openrisc/syscall_nr.h   |   8 +-
 linux-user/ppc/syscall.tbl |  30 +-
 linux-user/riscv/syscall32_nr.h|   8 +-
 linux-user/riscv/syscall64_nr.h|   8 +-
 linux-user/s390x/syscall.tbl   |   8 +-
 linux-user/sh4/syscall.tbl |   4 +-
 linux-user/sparc/syscall.tbl   |   8 +-
 linux-user/sparc64/syscall.tbl |   8 +-
 linux-user/syscall.c   |   6 -
 linux-user/x86_64/syscall_64.tbl   | 742 +++---
 linux-user/xtensa/syscall.tbl  |   4 +-
 scripts/gensyscalls.sh |   3 +-
 24 files changed, 889 insertions(+), 830 deletions(-)

-- 
2.26.2

[PATCH 3/4] linux-user: update syscall.tbl to Linux 5.9-rc7

[Laurent Vivier]

Updated running scripts/update-syscalltbl.sh

Signed-off-by: Laurent Vivier 
---
 linux-user/alpha/syscall.tbl  |   4 +-
 linux-user/arm/syscall.tbl|   4 +-
 linux-user/hppa/syscall.tbl   |   8 +-
 linux-user/i386/syscall_32.tbl| 820 +++---
 linux-user/m68k/syscall.tbl   |   4 +-
 linux-user/microblaze/syscall.tbl |   4 +-
 linux-user/mips/syscall_o32.tbl   |   8 +-
 linux-user/mips64/syscall_n32.tbl |   8 +-
 linux-user/mips64/syscall_n64.tbl |   4 +-
 linux-user/ppc/syscall.tbl|  30 +-
 linux-user/s390x/syscall.tbl  |   8 +-
 linux-user/sh4/syscall.tbl|   4 +-
 linux-user/sparc/syscall.tbl  |   8 +-
 linux-user/sparc64/syscall.tbl|   8 +-
 linux-user/x86_64/syscall_64.tbl  | 742 +--
 linux-user/xtensa/syscall.tbl |   4 +-
 16 files changed, 854 insertions(+), 814 deletions(-)

diff --git a/linux-user/alpha/syscall.tbl b/linux-user/alpha/syscall.tbl
index 36d42da7466a..ec8bed9e7b75 100644
--- a/linux-user/alpha/syscall.tbl
+++ b/linux-user/alpha/syscall.tbl
@@ -249,7 +249,7 @@
 316common  mlockallsys_mlockall
 317common  munlockall  sys_munlockall
 318common  sysinfo sys_sysinfo
-319common  _sysctl sys_sysctl
+319common  _sysctl sys_ni_syscall
 # 320 was sys_idle
 321common  oldumount   sys_oldumount
 322common  swapon  sys_swapon
@@ -475,5 +475,7 @@
 543common  fspick  sys_fspick
 544common  pidfd_open  sys_pidfd_open
 # 545 reserved for clone3
+546common  close_range sys_close_range
 547common  openat2 sys_openat2
 548common  pidfd_getfd sys_pidfd_getfd
+549common  faccessat2  sys_faccessat2
diff --git a/linux-user/arm/syscall.tbl b/linux-user/arm/syscall.tbl
index 4d1cf74a2caa..171077cbf419 100644
--- a/linux-user/arm/syscall.tbl
+++ b/linux-user/arm/syscall.tbl
@@ -162,7 +162,7 @@
 146common  writev  sys_writev
 147common  getsid  sys_getsid
 148common  fdatasync   sys_fdatasync
-149common  _sysctl sys_sysctl
+149common  _sysctl sys_ni_syscall
 150common  mlock   sys_mlock
 151common  munlock sys_munlock
 152common  mlockallsys_mlockall
@@ -449,5 +449,7 @@
 433common  fspick  sys_fspick
 434common  pidfd_open  sys_pidfd_open
 435common  clone3  sys_clone3
+436common  close_range sys_close_range
 437common  openat2 sys_openat2
 438common  pidfd_getfd sys_pidfd_getfd
+439common  faccessat2  sys_faccessat2
diff --git a/linux-user/hppa/syscall.tbl b/linux-user/hppa/syscall.tbl
index 52a15f5cd130..def64d221cd4 100644
--- a/linux-user/hppa/syscall.tbl
+++ b/linux-user/hppa/syscall.tbl
@@ -163,7 +163,7 @@
 146common  writev  sys_writev  
compat_sys_writev
 147common  getsid  sys_getsid
 148common  fdatasync   sys_fdatasync
-149common  _sysctl sys_sysctl  
compat_sys_sysctl
+149common  _sysctl sys_ni_syscall
 150common  mlock   sys_mlock
 151common  munlock sys_munlock
 152common  mlockallsys_mlockall
@@ -198,8 +198,8 @@
 178common  rt_sigqueueinfo sys_rt_sigqueueinfo 
compat_sys_rt_sigqueueinfo
 179common  rt_sigsuspend   sys_rt_sigsuspend   
compat_sys_rt_sigsuspend
 180common  chown   sys_chown
-181common  setsockopt  sys_setsockopt  
compat_sys_setsockopt
-182common  getsockopt  sys_getsockopt  
compat_sys_getsockopt
+181common  setsockopt  sys_setsockopt  
sys_setsockopt
+182common  getsockopt  sys_getsockopt  
sys_getsockopt
 183common  sendmsg sys_sendmsg 
compat_sys_sendmsg
 184common  recvmsg sys_recvmsg 
compat_sys_recvmsg
 185common  semop   sys_semop
@@ -433,5 +433,7 @@
 433common  fspick  sys_fspick
 434common  pidfd_open  sys_pidfd_open
 435common  clone3  sys_clone3_wrapper
+436common  close_range sys_close_range
 437common  openat2 sys_openat2
 438common  pidfd_getfd sys_pidfd_getfd
+439

[PATCH 1/4] linux-user: update syscall_nr.h to Linux 5.9-rc7

[Laurent Vivier]

Update gensyscalls.sh not to generate an empty line at the end of the file

And then automatically update syscall_nr.h running scripts/gensyscalls.sh

Signed-off-by: Laurent Vivier 
---
 linux-user/aarch64/syscall_nr.h  | 7 +--
 linux-user/nios2/syscall_nr.h| 7 +--
 linux-user/openrisc/syscall_nr.h | 8 ++--
 linux-user/riscv/syscall32_nr.h  | 8 +++-
 linux-user/riscv/syscall64_nr.h  | 8 +++-
 scripts/gensyscalls.sh   | 3 +--
 6 files changed, 31 insertions(+), 10 deletions(-)

diff --git a/linux-user/aarch64/syscall_nr.h b/linux-user/aarch64/syscall_nr.h
index 85de000b2490..6fd5b331e780 100644
--- a/linux-user/aarch64/syscall_nr.h
+++ b/linux-user/aarch64/syscall_nr.h
@@ -298,7 +298,10 @@
 #define TARGET_NR_fspick 433
 #define TARGET_NR_pidfd_open 434
 #define TARGET_NR_clone3 435
-#define TARGET_NR_syscalls 436
+#define TARGET_NR_close_range 436
+#define TARGET_NR_openat2 437
+#define TARGET_NR_pidfd_getfd 438
+#define TARGET_NR_faccessat2 439
+#define TARGET_NR_syscalls 440
 
 #endif /* LINUX_USER_AARCH64_SYSCALL_NR_H */
-
diff --git a/linux-user/nios2/syscall_nr.h b/linux-user/nios2/syscall_nr.h
index 32d485dc9ae8..e37f40179bf3 100644
--- a/linux-user/nios2/syscall_nr.h
+++ b/linux-user/nios2/syscall_nr.h
@@ -318,7 +318,10 @@
 #define TARGET_NR_fsmount 432
 #define TARGET_NR_fspick 433
 #define TARGET_NR_pidfd_open 434
-#define TARGET_NR_syscalls 436
+#define TARGET_NR_close_range 436
+#define TARGET_NR_openat2 437
+#define TARGET_NR_pidfd_getfd 438
+#define TARGET_NR_faccessat2 439
+#define TARGET_NR_syscalls 440
 
 #endif /* LINUX_USER_NIOS2_SYSCALL_NR_H */
-
diff --git a/linux-user/openrisc/syscall_nr.h b/linux-user/openrisc/syscall_nr.h
index 340383beb2c6..a8fc0295109a 100644
--- a/linux-user/openrisc/syscall_nr.h
+++ b/linux-user/openrisc/syscall_nr.h
@@ -318,7 +318,11 @@
 #define TARGET_NR_fsmount 432
 #define TARGET_NR_fspick 433
 #define TARGET_NR_pidfd_open 434
-#define TARGET_NR_syscalls 436
+#define TARGET_NR_clone3 435
+#define TARGET_NR_close_range 436
+#define TARGET_NR_openat2 437
+#define TARGET_NR_pidfd_getfd 438
+#define TARGET_NR_faccessat2 439
+#define TARGET_NR_syscalls 440
 
 #endif /* LINUX_USER_OPENRISC_SYSCALL_NR_H */
-
diff --git a/linux-user/riscv/syscall32_nr.h b/linux-user/riscv/syscall32_nr.h
index 4fef73e954da..079b804daef5 100644
--- a/linux-user/riscv/syscall32_nr.h
+++ b/linux-user/riscv/syscall32_nr.h
@@ -1,5 +1,7 @@
 /*
  * This file contains the system call numbers.
+ * Do not modify.
+ * This file is generated by scripts/gensyscalls.sh
  */
 #ifndef LINUX_USER_RISCV_SYSCALL32_NR_H
 #define LINUX_USER_RISCV_SYSCALL32_NR_H
@@ -290,6 +292,10 @@
 #define TARGET_NR_fspick 433
 #define TARGET_NR_pidfd_open 434
 #define TARGET_NR_clone3 435
-#define TARGET_NR_syscalls 436
+#define TARGET_NR_close_range 436
+#define TARGET_NR_openat2 437
+#define TARGET_NR_pidfd_getfd 438
+#define TARGET_NR_faccessat2 439
+#define TARGET_NR_syscalls 440
 
 #endif /* LINUX_USER_RISCV_SYSCALL32_NR_H */
diff --git a/linux-user/riscv/syscall64_nr.h b/linux-user/riscv/syscall64_nr.h
index cc82f3244f55..d54224ccec64 100644
--- a/linux-user/riscv/syscall64_nr.h
+++ b/linux-user/riscv/syscall64_nr.h
@@ -1,5 +1,7 @@
 /*
  * This file contains the system call numbers.
+ * Do not modify.
+ * This file is generated by scripts/gensyscalls.sh
  */
 #ifndef LINUX_USER_RISCV_SYSCALL64_NR_H
 #define LINUX_USER_RISCV_SYSCALL64_NR_H
@@ -296,6 +298,10 @@
 #define TARGET_NR_fspick 433
 #define TARGET_NR_pidfd_open 434
 #define TARGET_NR_clone3 435
-#define TARGET_NR_syscalls 436
+#define TARGET_NR_close_range 436
+#define TARGET_NR_openat2 437
+#define TARGET_NR_pidfd_getfd 438
+#define TARGET_NR_faccessat2 439
+#define TARGET_NR_syscalls 440
 
 #endif /* LINUX_USER_RISCV_SYSCALL64_NR_H */
diff --git a/scripts/gensyscalls.sh b/scripts/gensyscalls.sh
index b7b8456f6312..bba9fb052c47 100755
--- a/scripts/gensyscalls.sh
+++ b/scripts/gensyscalls.sh
@@ -86,8 +86,7 @@ generate_syscall_nr()
 read_includes $arch $bits | filter_defines | rename_defines | \
 evaluate_values | sort -n -k 3
 echo
-echo "#endif /* ${guard} */"
-echo) > "$file"
+echo "#endif /* ${guard} */") > "$file"
 }
 
 mkdir "$TMP/asm"
-- 
2.26.2

[PATCH 2/2] hw/block/m25p80: Fix nonvolatile-cfg property default value

[Joe Komlodi]

The nvcfg registers are all 1s, unless previously modified.

Signed-off-by: Joe Komlodi 
---
 hw/block/m25p80.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/block/m25p80.c b/hw/block/m25p80.c
index 43830c9..69c88d4 100644
--- a/hw/block/m25p80.c
+++ b/hw/block/m25p80.c
@@ -1334,7 +1334,7 @@ static int m25p80_pre_save(void *opaque)
 
 static Property m25p80_properties[] = {
 /* This is default value for Micron flash */
-DEFINE_PROP_UINT32("nonvolatile-cfg", Flash, nonvolatile_cfg, 0x8FFF),
+DEFINE_PROP_UINT32("nonvolatile-cfg", Flash, nonvolatile_cfg, 0x),
 DEFINE_PROP_UINT8("spansion-cr1nv", Flash, spansion_cr1nv, 0x0),
 DEFINE_PROP_UINT8("spansion-cr2nv", Flash, spansion_cr2nv, 0x8),
 DEFINE_PROP_UINT8("spansion-cr3nv", Flash, spansion_cr3nv, 0x2),
-- 
2.7.4

[PATCH 4/4] linux-user: remove _sysctl

[Laurent Vivier]

It has been removed from linux since

  61a47c1ad3a4 ("sysctl: Remove the sysctl system call")

It's a good news because it was not really supported by qemu.

Signed-off-by: Laurent Vivier 
---
 linux-user/syscall.c | 6 --
 1 file changed, 6 deletions(-)

diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index 897d20c076ce..0cfc95088fa4 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -10524,12 +10524,6 @@ static abi_long do_syscall1(void *cpu_env, int num, 
abi_long arg1,
 #if defined(TARGET_NR_fdatasync) /* Not on alpha (osf_datasync ?) */
 case TARGET_NR_fdatasync:
 return get_errno(fdatasync(arg1));
-#endif
-#ifdef TARGET_NR__sysctl
-case TARGET_NR__sysctl:
-/* We don't implement this, but ENOTDIR is always a safe
-   return value. */
-return -TARGET_ENOTDIR;
 #endif
 case TARGET_NR_sched_getaffinity:
 {
-- 
2.26.2

[PATCH 2/4] linux-user: update mips/syscall-args-o32.c.inc to Linux 5.9-rc7

[Laurent Vivier]

Updated running scripts/update-mips-syscall-args.sh

Signed-off-by: Laurent Vivier 
---
 linux-user/mips/syscall-args-o32.c.inc | 4 
 1 file changed, 4 insertions(+)

diff --git a/linux-user/mips/syscall-args-o32.c.inc 
b/linux-user/mips/syscall-args-o32.c.inc
index 0ad35857b4e4..92ee4f921ec7 100644
--- a/linux-user/mips/syscall-args-o32.c.inc
+++ b/linux-user/mips/syscall-args-o32.c.inc
@@ -434,3 +434,7 @@
 [ 433] = 3, /* fspick */
 [ 434] = 2, /* pidfd_open */
 [ 435] = 2, /* clone3 */
+[ 436] = 3, /* close_range */
+[ 437] = 4, /* openat2 */
+[ 438] = 3, /* pidfd_getfd */
+[ 439] = 4, /* faccessat2 */
-- 
2.26.2

[PATCH 1/2] hw/block/m25p80: Fix Numonyx dummy cycle register behavior

[Joe Komlodi]

Numonyx chips determine the number of cycles to wait based on bits 7:4 in the
volatile configuration register.

However, if these bits are 0x0 or 0xF, the number of dummy cycles to wait is
10 on a QIOR or QIOR4 command, or 8 on any other currently supported
fast read command. [1]

[1] http://www.micron.com/-/media/client/global/documents/products/
data-sheet/nor-flash/serial-nor/n25q/n25q_512mb_1_8v_65nm.pdf

Page 22 note 2, and page 30 notes 5 and 10.

Signed-off-by: Joe Komlodi 
---
 hw/block/m25p80.c | 26 +++---
 1 file changed, 23 insertions(+), 3 deletions(-)

diff --git a/hw/block/m25p80.c b/hw/block/m25p80.c
index 483925f..43830c9 100644
--- a/hw/block/m25p80.c
+++ b/hw/block/m25p80.c
@@ -820,6 +820,26 @@ static void reset_memory(Flash *s)
 trace_m25p80_reset_done(s);
 }
 
+static uint8_t numonyx_fast_read_num_dummies(Flash *s)
+{
+uint8_t cycle_count;
+uint8_t num_dummies;
+assert(get_man(s) == MAN_NUMONYX);
+
+cycle_count = extract32(s->volatile_cfg, 4, 4);
+if (cycle_count == 0x0 || cycle_count == 0x0F) {
+if (s->cmd_in_progress == QIOR || s->cmd_in_progress == QIOR4) {
+num_dummies = 10;
+} else {
+num_dummies = 8;
+}
+} else {
+num_dummies = cycle_count;
+}
+
+return num_dummies;
+}
+
 static void decode_fast_read_cmd(Flash *s)
 {
 s->needed_bytes = get_addr_length(s);
@@ -829,7 +849,7 @@ static void decode_fast_read_cmd(Flash *s)
 s->needed_bytes += 8;
 break;
 case MAN_NUMONYX:
-s->needed_bytes += extract32(s->volatile_cfg, 4, 4);
+s->needed_bytes += numonyx_fast_read_num_dummies(s);
 break;
 case MAN_MACRONIX:
 if (extract32(s->volatile_cfg, 6, 2) == 1) {
@@ -868,7 +888,7 @@ static void decode_dio_read_cmd(Flash *s)
 );
 break;
 case MAN_NUMONYX:
-s->needed_bytes += extract32(s->volatile_cfg, 4, 4);
+s->needed_bytes += numonyx_fast_read_num_dummies(s);
 break;
 case MAN_MACRONIX:
 switch (extract32(s->volatile_cfg, 6, 2)) {
@@ -908,7 +928,7 @@ static void decode_qio_read_cmd(Flash *s)
 );
 break;
 case MAN_NUMONYX:
-s->needed_bytes += extract32(s->volatile_cfg, 4, 4);
+s->needed_bytes += numonyx_fast_read_num_dummies(s);
 break;
 case MAN_MACRONIX:
 switch (extract32(s->volatile_cfg, 6, 2)) {
-- 
2.7.4

[PATCH 0/2] hw/block/m25p80: Fix Numonyx flash dummy cycle register behavior

[Joe Komlodi]

Hi all,

This series addresses a couple issues with dummy cycle counts with Numonyx
flashes.

The first patch fixes the behavior of the dummy cycle register so it's closer to
how hardware behaves.
As a consequence, it also corrects the amount of dummy cycles QIOR and QIOR4
commands need by default.

The second patch changes the default value of the nvcfg register so it
matches what would be in hardware from the factory.

Thanks!
Joe

Joe Komlodi (2):
  hw/block/m25p80: Fix Numonyx dummy cycle register behavior
  hw/block/m25p80: Fix nonvolatile-cfg property default value

 hw/block/m25p80.c | 28 
 1 file changed, 24 insertions(+), 4 deletions(-)

-- 
2.7.4

[Bug 1821595] Re: Failed to emulate MMIO access with EmulatorReturnStatus: 2

[Fernando Luiz]

Hi. I didnt build Qemu but I downloaded the last version 5.1.0 from
https://qemu.weilnetz.de/w64/ and the error "Failed to emulate MMIO
access with EmulatorReturnStatus: 2" happens when I use -drive if=pflash
or -pflash with -accel whpx.

"qemu-system-x86_64.exe -pflash d:\macWIN\OVMF_CODE.fd -accel whpx
Windows Hypervisor Platform accelerator is operational
qemu-system-x86_64.exe: WHPX: Failed to emulate MMIO access with 
EmulatorReturnStatus: 2
qemu-system-x86_64.exe: WHPX: Failed to exec a virtual processor"


If I remove pflash and change to -bios "ovmf" the machine works.
qemu-system-x86_64.exe -bios d:\macWIN\OVMF_CODE.fd -accel whpx

If I maintain pflash but remove -accel whpx the machine works too.
qemu-system-x86_64.exe -pflash d:\macWIN\OVMF_CODE.fd

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1821595

Title:
  Failed to emulate MMIO access with EmulatorReturnStatus: 2

Status in QEMU:
  New

Bug description:
  I have compiled qemu with enable-whpx parameter for Hyper-V Platform API in 
Mingw64 . When I tried run with Windows 7 iso file I have faced issue with the 
following problem: 
  qemu-system-x86_64.exe: WHPX: Failed to emulate MMIO access with 
EmulatorReturnStatus: 2
  qemu-system-x86_64.exe: WHPX: Failed to exec a virtual processor

  
  configuration directives:

  ../configure --target-list=x86_64-softmmu,i386-softmmu --enable-lzo\
   --enable-bzip2 --enable-tools --enable-sdl --enable-gtk --enable-hax\
   --enable-vdi --enable-qcow1 --enable-whpx --disable-capstone\
   --disable-werror --disable-stack-protector --prefix="../../QEMU-bin"

  
  Qemu command line:
  qemu-system-x86_64.exe -m 1024 -cdrom 
"C:\Users\vmcs\Documents\en_windows_7_home_premium_with_sp1_x86_dvd_u_676701.iso"
 -display sdl -machine q35 -accel whpx

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1821595/+subscriptions

[PATCH] qom: fix objects with improper parent type

[Sergey Nizovtsev]

Some objects accidentally inherit ObjectClass instead of Object.
They compile silently but may crash after downcasting.

In this patch, we introduce a coccinelle script to find broken
declarations and fix them manually with proper base type.

Signed-off-by: Sergey Nizovtsev 
---
 scripts/coccinelle/qobject-parent-type.cocci | 26 
 include/hw/acpi/vmgenid.h|  2 +-
 include/hw/misc/vmcoreinfo.h |  2 +-
 include/net/can_host.h   |  2 +-
 MAINTAINERS  |  1 +
 5 files changed, 30 insertions(+), 3 deletions(-)
 create mode 100644 scripts/coccinelle/qobject-parent-type.cocci

diff --git a/scripts/coccinelle/qobject-parent-type.cocci
b/scripts/coccinelle/qobject-parent-type.cocci
new file mode 100644
index 00..9afb3edd97
--- /dev/null
+++ b/scripts/coccinelle/qobject-parent-type.cocci
@@ -0,0 +1,26 @@
+// Highlight object declarations that don't look like object class but
+// accidentally inherit from it.
+
+@match@
+identifier obj_t, fld;
+type parent_t =~ ".*Class$";
+@@
+struct obj_t {
+parent_t fld;
+...
+};
+
+@script:python filter depends on match@
+obj_t << match.obj_t;
+@@
+is_class_obj = obj_t.endswith('Class')
+cocci.include_match(not is_class_obj)
+
+@replacement depends on filter@
+identifier match.obj_t, match.fld;
+type match.parent_t;
+@@
+struct obj_t {
+*   parent_t fld;
+...
+};
diff --git a/include/hw/acpi/vmgenid.h b/include/hw/acpi/vmgenid.h
index d50fbacb8e..cb4ad37fc5 100644
--- a/include/hw/acpi/vmgenid.h
+++ b/include/hw/acpi/vmgenid.h
@@ -19,7 +19,7 @@
 OBJECT_DECLARE_SIMPLE_TYPE(VmGenIdState, VMGENID)

 struct VmGenIdState {
-DeviceClass parent_obj;
+DeviceState parent_obj;
 QemuUUID guid;/* The 128-bit GUID seen by the guest */
 uint8_t vmgenid_addr_le[8];   /* Address of the GUID (little-endian) */
 };
diff --git a/include/hw/misc/vmcoreinfo.h b/include/hw/misc/vmcoreinfo.h
index ebada6617a..0b7b55d400 100644
--- a/include/hw/misc/vmcoreinfo.h
+++ b/include/hw/misc/vmcoreinfo.h
@@ -24,7 +24,7 @@ DECLARE_INSTANCE_CHECKER(VMCoreInfoState, VMCOREINFO,
 typedef struct fw_cfg_vmcoreinfo FWCfgVMCoreInfo;

 struct VMCoreInfoState {
-DeviceClass parent_obj;
+DeviceState parent_obj;

 bool has_vmcoreinfo;
 FWCfgVMCoreInfo vmcoreinfo;
diff --git a/include/net/can_host.h b/include/net/can_host.h
index 4e3ce3f954..caab71bdda 100644
--- a/include/net/can_host.h
+++ b/include/net/can_host.h
@@ -35,7 +35,7 @@
 OBJECT_DECLARE_TYPE(CanHostState, CanHostClass, CAN_HOST)

 struct CanHostState {
-ObjectClass oc;
+Object oc;

 CanBusState *bus;
 CanBusClientState bus_client;
diff --git a/MAINTAINERS b/MAINTAINERS
index 5eed1e692b..2160b8196a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2402,6 +2402,7 @@ F: qobject/
 F: include/qapi/qmp/
 X: include/qapi/qmp/dispatch.h
 F: scripts/coccinelle/qobject.cocci
+F: scripts/coccinelle/qobject-parent-type.cocci
 F: tests/check-qdict.c
 F: tests/check-qjson.c
 F: tests/check-qlist.c
-- 
2.28.0

Re: [PATCH v2 4/8] linux-user: Add IPv6 options to do_print_sockopt()

[Laurent Vivier]

Le 11/08/2020 à 09:09, Shu-Chun Weng a écrit :
> Signed-off-by: Shu-Chun Weng 
> ---
> v1 -> v2:
>   New: Add all IPV6 options to do_print_sockopt(), including the newly 
> supported
>   IPV6_ADDR_PREFERENCES.
> 
>  linux-user/strace.c | 108 
>  1 file changed, 108 insertions(+)
> 
> diff --git a/linux-user/strace.c b/linux-user/strace.c
> index 854b54a2ad..089fb3968e 100644
> --- a/linux-user/strace.c
> +++ b/linux-user/strace.c
> @@ -6,6 +6,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>  #include 
>  #include 
>  #include 
> @@ -2307,6 +2308,113 @@ print_optint:
>  break;
>  }
>  break;
> +case SOL_IPV6:
> +qemu_log("SOL_IPV6,");
> +switch (optname) {
> +case IPV6_MTU_DISCOVER:
> +qemu_log("IPV6_MTU_DISCOVER,");
> +goto print_optint;
> +case IPV6_MTU:
> +qemu_log("IPV6_MTU,");
> +goto print_optint;
> +case IPV6_V6ONLY:
> +qemu_log("IPV6_V6ONLY,");
> +goto print_optint;
> +case IPV6_RECVPKTINFO:
> +qemu_log("IPV6_RECVPKTINFO,");
> +goto print_optint;
> +case IPV6_UNICAST_HOPS:
> +qemu_log("IPV6_UNICAST_HOPS,");
> +goto print_optint;
> +case IPV6_MULTICAST_HOPS:
> +qemu_log("IPV6_MULTICAST_HOPS,");
> +goto print_optint;
> +case IPV6_MULTICAST_LOOP:
> +qemu_log("IPV6_MULTICAST_LOOP,");
> +goto print_optint;
> +case IPV6_RECVERR:
> +qemu_log("IPV6_RECVERR,");
> +goto print_optint;
> +case IPV6_RECVHOPLIMIT:
> +qemu_log("IPV6_RECVHOPLIMIT,");
> +goto print_optint;
> +case IPV6_2292HOPLIMIT:
> +qemu_log("IPV6_2292HOPLIMIT,");
> +goto print_optint;
> +case IPV6_CHECKSUM:
> +qemu_log("IPV6_CHECKSUM,");
> +goto print_optint;
> +case IPV6_ADDRFORM:
> +qemu_log("IPV6_ADDRFORM,");
> +goto print_optint;
> +case IPV6_2292PKTINFO:
> +qemu_log("IPV6_2292PKTINFO,");
> +goto print_optint;
> +case IPV6_RECVTCLASS:
> +qemu_log("IPV6_RECVTCLASS,");
> +goto print_optint;
> +case IPV6_RECVRTHDR:
> +qemu_log("IPV6_RECVRTHDR,");
> +goto print_optint;
> +case IPV6_2292RTHDR:
> +qemu_log("IPV6_2292RTHDR,");
> +goto print_optint;
> +case IPV6_RECVHOPOPTS:
> +qemu_log("IPV6_RECVHOPOPTS,");
> +goto print_optint;
> +case IPV6_2292HOPOPTS:
> +qemu_log("IPV6_2292HOPOPTS,");
> +goto print_optint;
> +case IPV6_RECVDSTOPTS:
> +qemu_log("IPV6_RECVDSTOPTS,");
> +goto print_optint;
> +case IPV6_2292DSTOPTS:
> +qemu_log("IPV6_2292DSTOPTS,");
> +goto print_optint;
> +case IPV6_TCLASS:
> +qemu_log("IPV6_TCLASS,");
> +goto print_optint;
> +case IPV6_ADDR_PREFERENCES:
> +qemu_log("IPV6_ADDR_PREFERENCES,");
> +goto print_optint;
> +#ifdef IPV6_RECVPATHMTU
> +case IPV6_RECVPATHMTU:
> +qemu_log("IPV6_RECVPATHMTU,");
> +goto print_optint;
> +#endif
> +#ifdef IPV6_TRANSPARENT
> +case IPV6_TRANSPARENT:
> +qemu_log("IPV6_TRANSPARENT,");
> +goto print_optint;
> +#endif
> +#ifdef IPV6_FREEBIND
> +case IPV6_FREEBIND:
> +qemu_log("IPV6_FREEBIND,");
> +goto print_optint;
> +#endif
> +#ifdef IPV6_RECVORIGDSTADDR
> +case IPV6_RECVORIGDSTADDR:
> +qemu_log("IPV6_RECVORIGDSTADDR,");
> +goto print_optint;
> +#endif
> +case IPV6_PKTINFO:
> +qemu_log("IPV6_PKTINFO,");
> +print_pointer(optval, 0);
> +break;
> +case IPV6_ADD_MEMBERSHIP:
> +qemu_log("IPV6_ADD_MEMBERSHIP,");
> +print_pointer(optval, 0);
> +break;
> +case IPV6_DROP_MEMBERSHIP:
> +qemu_log("IPV6_DROP_MEMBERSHIP,");
> +print_pointer(optval, 0);
> +break;
> +default:
> +print_raw_param(TARGET_ABI_FMT_ld, optname, 0);
> +print_pointer(optval, 0);
> +break;
> +}
> +break;
>  default:
>  print_raw_param(TARGET_ABI_FMT_ld, level, 0);
>  print_raw_param(TARGET_ABI_FMT_ld, optname, 0);
> 

Reviewed-by: Laurent Vivier 

Re: [PATCH v4] linux-user: Add most IFTUN ioctls

[Laurent Vivier]

Le 29/09/2020 à 03:48, Shu-Chun Weng via a écrit :
> The three options handling `struct sock_fprog` (TUNATTACHFILTER,
> TUNDETACHFILTER, and TUNGETFILTER) are not implemented. Linux kernel
> keeps a user space pointer in them which we cannot correctly handle.
> 
> Signed-off-by: Josh Kunz 
> Signed-off-by: Shu-Chun Weng 
> ---
> v2->v3:
>   IOCTL_SPECIAL(TUNSETTXFILTER) type changed to MK_PTR(TYPE_PTRVOID) for 
> strace
>   to display the raw pointer.
> 
>   Updated do_ioctl_TUNSETTXFILTER for correct usages of unlock_user() and
>   offsetof().
> 
> v3->v4:
>   IOCTL_SPECIAL(TUNSETTXFILTER) corrected to TYPE_PTRVOID.
> 
>  linux-user/ioctls.h   | 46 +++
>  linux-user/syscall.c  | 38 
>  linux-user/syscall_defs.h | 32 +++
>  3 files changed, 116 insertions(+)
> 
...
> diff --git a/linux-user/syscall.c b/linux-user/syscall.c
> index 945fc25279..1c955bc675 100644
> --- a/linux-user/syscall.c
> +++ b/linux-user/syscall.c
> @@ -56,6 +56,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>  #include 
>  #include 
>  #ifdef CONFIG_TIMERFD
> @@ -5415,6 +5416,43 @@ static abi_long do_ioctl_drm(const IOCTLEntry *ie, 
> uint8_t *buf_temp,
>  
>  #endif
>  
> +static abi_long do_ioctl_TUNSETTXFILTER(const IOCTLEntry *ie, uint8_t 
> *buf_temp,
> +int fd, int cmd, abi_long arg)
> +{
> +struct tun_filter *filter = (struct tun_filter *)buf_temp;
> +struct tun_filter *target_filter;
> +char *target_addr;
> +
> +assert(ie->access == IOC_W);
> +
> +target_filter = lock_user(VERIFY_READ, arg, sizeof(*target_filter), 1);
> +if (!target_filter) {
> +return -TARGET_EFAULT;
> +}
> +filter->flags = tswap16(target_filter->flags);
> +filter->count = tswap16(target_filter->count);
> +unlock_user(target_filter, arg, 0);
> +
> +if (filter->count) {
> +if (offsetof(struct tun_filter, addr) + filter->count * ETH_ALEN >
> +MAX_STRUCT_SIZE) {
> +return -TARGET_EFAULT;
> +}
> +
> +target_addr = lock_user(VERIFY_READ,
> +arg + offsetof(struct tun_filter, addr),
> +filter->count * ETH_ALEN, 1);
> +if (!target_addr) {
> +return -TARGET_EFAULT;
> +}
> +memcpy(filter->addr, target_addr, filter->count * ETH_ALEN);
> +unlock_user(target_addr, arg + offsetof(struct tun_filter, addr),
> +filter->count * ETH_ALEN);

As we don't modify target_addr memory content (locked with VERIFY_READ),
we can replace "filter->count * ETH_ALEN" by 0.

With that modified:

Reviewed-by: Laurent Vivier 

[PATCH v2 14/14] hw/block/nvme: allow open to close transitions by controller

[Klaus Jensen]

From: Klaus Jensen 

Allow the controller to release open resources by transitioning
implicitly and explicitly opened zones to closed. This is done using a
naive "least recently opened" strategy.

Signed-off-by: Klaus Jensen 
---
 hw/block/nvme-ns.h|   5 ++
 hw/block/nvme-ns.c|   5 ++
 hw/block/nvme.c   | 105 +++---
 hw/block/trace-events |   5 ++
 4 files changed, 103 insertions(+), 17 deletions(-)

diff --git a/hw/block/nvme-ns.h b/hw/block/nvme-ns.h
index ff34cd37af7d..491a77f3ae2f 100644
--- a/hw/block/nvme-ns.h
+++ b/hw/block/nvme-ns.h
@@ -62,6 +62,8 @@ typedef struct NvmeZone {
 uint8_t*zde;
 
 uint64_t wp_staging;
+
+QTAILQ_ENTRY(NvmeZone) lru_entry;
 } NvmeZone;
 
 typedef struct NvmeNamespace {
@@ -101,6 +103,9 @@ typedef struct NvmeNamespace {
 struct {
 uint32_t open;
 uint32_t active;
+
+QTAILQ_HEAD(, NvmeZone) lru_open;
+QTAILQ_HEAD(, NvmeZone) lru_active;
 } resources;
 } zns;
 } NvmeNamespace;
diff --git a/hw/block/nvme-ns.c b/hw/block/nvme-ns.c
index 9584fbb3f62d..26c9f846417a 100644
--- a/hw/block/nvme-ns.c
+++ b/hw/block/nvme-ns.c
@@ -225,6 +225,9 @@ void nvme_ns_zns_init_zone_state(NvmeNamespace *ns)
 ns->zns.resources.open = ns->params.zns.mor != 0x ?
 ns->params.zns.mor + 1 : ns->zns.num_zones;
 
+QTAILQ_INIT(>zns.resources.lru_open);
+QTAILQ_INIT(>zns.resources.lru_active);
+
 for (int i = 0; i < ns->zns.num_zones; i++) {
 NvmeZone *zone = >zns.zones[i];
 zone->zd = >zns.zd[i];
@@ -248,6 +251,8 @@ void nvme_ns_zns_init_zone_state(NvmeNamespace *ns)
 
 if (ns->zns.resources.active) {
 ns->zns.resources.active--;
+QTAILQ_INSERT_TAIL(>zns.resources.lru_active, zone,
+   lru_entry);
 continue;
 }
 
diff --git a/hw/block/nvme.c b/hw/block/nvme.c
index fc5b119e3f35..34093f33ad1a 100644
--- a/hw/block/nvme.c
+++ b/hw/block/nvme.c
@@ -1209,12 +1209,61 @@ static inline void nvme_zone_reset_wp(NvmeZone *zone)
 zone->wp_staging = nvme_zslba(zone);
 }
 
-static uint16_t nvme_zrm_transition(NvmeNamespace *ns, NvmeZone *zone,
-NvmeZoneState to)
+static uint16_t nvme_zrm_transition(NvmeCtrl *n, NvmeNamespace *ns,
+NvmeZone *zone, NvmeZoneState to,
+NvmeRequest *req);
+
+static uint16_t nvme_zrm_release_open(NvmeCtrl *n, NvmeNamespace *ns,
+  NvmeRequest *req)
+{
+NvmeZone *candidate;
+NvmeZoneState zs;
+uint16_t status;
+
+trace_pci_nvme_zone_zrm_release_open(nvme_cid(req), ns->params.nsid);
+
+QTAILQ_FOREACH(candidate, >zns.resources.lru_open, lru_entry) {
+zs = nvme_zs(candidate);
+
+trace_pci_nvme_zone_zrm_candidate(nvme_cid(req), ns->params.nsid,
+  nvme_zslba(candidate),
+  nvme_wp(candidate), zs);
+
+/* skip explicitly opened zones */
+if (zs == NVME_ZS_ZSEO) {
+continue;
+}
+
+/* the zone cannot be closed if it is currently writing */
+if (candidate->wp_staging != nvme_wp(candidate)) {
+continue;
+}
+
+status = nvme_zrm_transition(n, ns, candidate, NVME_ZS_ZSC, req);
+if (status) {
+return status;
+}
+
+if (nvme_zns_commit_zone(ns, candidate) < 0) {
+return NVME_INTERNAL_DEV_ERROR;
+}
+
+return NVME_SUCCESS;
+}
+
+return NVME_TOO_MANY_OPEN_ZONES;
+}
+
+static uint16_t nvme_zrm_transition(NvmeCtrl *n, NvmeNamespace *ns,
+NvmeZone *zone, NvmeZoneState to,
+NvmeRequest *req)
 {
 NvmeZoneState from = nvme_zs(zone);
+uint16_t status;
+
+trace_pci_nvme_zone_zrm_transition(nvme_cid(req), ns->params.nsid,
+   nvme_zslba(zone), nvme_zs(zone), to);
 
-/* fast path */
 if (from == to) {
 return NVME_SUCCESS;
 }
@@ -1229,25 +1278,32 @@ static uint16_t nvme_zrm_transition(NvmeNamespace *ns, 
NvmeZone *zone,
 
 case NVME_ZS_ZSC:
 if (!ns->zns.resources.active) {
+trace_pci_nvme_err_too_many_active_zones(nvme_cid(req));
 return NVME_TOO_MANY_ACTIVE_ZONES;
 }
 
 ns->zns.resources.active--;
+QTAILQ_INSERT_TAIL(>zns.resources.lru_active, zone, lru_entry);
 
 break;
 
 case NVME_ZS_ZSIO:
 case NVME_ZS_ZSEO:
 if (!ns->zns.resources.active) {
+trace_pci_nvme_err_too_many_active_zones(nvme_cid(req));
 return NVME_TOO_MANY_ACTIVE_ZONES;
 }
 
 if (!ns->zns.resources.open) {
-return 

[PATCH] qom: fix objects with improper parent type

[Sergey Nizovtsev]

Some objects accidentally inherit ObjectClass instead of Object.
They compile silently but may crash after downcasting.

In this patch, we introduce a coccinelle script to find broken
declarations and fix them manually with proper base type.

Signed-off-by: Sergey Nizovtsev 
---
 scripts/coccinelle/qobject-parent-type.cocci | 26 
 include/hw/acpi/vmgenid.h|  2 +-
 include/hw/misc/vmcoreinfo.h |  2 +-
 include/net/can_host.h   |  2 +-
 MAINTAINERS  |  1 +
 5 files changed, 30 insertions(+), 3 deletions(-)
 create mode 100644 scripts/coccinelle/qobject-parent-type.cocci

diff --git a/scripts/coccinelle/qobject-parent-type.cocci
b/scripts/coccinelle/qobject-parent-type.cocci
new file mode 100644
index 00..9afb3edd97
--- /dev/null
+++ b/scripts/coccinelle/qobject-parent-type.cocci
@@ -0,0 +1,26 @@
+// Highlight object declarations that don't look like object class but
+// accidentally inherit from it.
+
+@match@
+identifier obj_t, fld;
+type parent_t =~ ".*Class$";
+@@
+struct obj_t {
+parent_t fld;
+...
+};
+
+@script:python filter depends on match@
+obj_t << match.obj_t;
+@@
+is_class_obj = obj_t.endswith('Class')
+cocci.include_match(not is_class_obj)
+
+@replacement depends on filter@
+identifier match.obj_t, match.fld;
+type match.parent_t;
+@@
+struct obj_t {
+*   parent_t fld;
+...
+};
diff --git a/include/hw/acpi/vmgenid.h b/include/hw/acpi/vmgenid.h
index d50fbacb8e..cb4ad37fc5 100644
--- a/include/hw/acpi/vmgenid.h
+++ b/include/hw/acpi/vmgenid.h
@@ -19,7 +19,7 @@
 OBJECT_DECLARE_SIMPLE_TYPE(VmGenIdState, VMGENID)

 struct VmGenIdState {
-DeviceClass parent_obj;
+DeviceState parent_obj;
 QemuUUID guid;/* The 128-bit GUID seen by the guest */
 uint8_t vmgenid_addr_le[8];   /* Address of the GUID (little-endian) */
 };
diff --git a/include/hw/misc/vmcoreinfo.h b/include/hw/misc/vmcoreinfo.h
index ebada6617a..0b7b55d400 100644
--- a/include/hw/misc/vmcoreinfo.h
+++ b/include/hw/misc/vmcoreinfo.h
@@ -24,7 +24,7 @@ DECLARE_INSTANCE_CHECKER(VMCoreInfoState, VMCOREINFO,
 typedef struct fw_cfg_vmcoreinfo FWCfgVMCoreInfo;

 struct VMCoreInfoState {
-DeviceClass parent_obj;
+DeviceState parent_obj;

 bool has_vmcoreinfo;
 FWCfgVMCoreInfo vmcoreinfo;
diff --git a/include/net/can_host.h b/include/net/can_host.h
index 4e3ce3f954..caab71bdda 100644
--- a/include/net/can_host.h
+++ b/include/net/can_host.h
@@ -35,7 +35,7 @@
 OBJECT_DECLARE_TYPE(CanHostState, CanHostClass, CAN_HOST)

 struct CanHostState {
-ObjectClass oc;
+Object oc;

 CanBusState *bus;
 CanBusClientState bus_client;
diff --git a/MAINTAINERS b/MAINTAINERS
index 5eed1e692b..2160b8196a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2402,6 +2402,7 @@ F: qobject/
 F: include/qapi/qmp/
 X: include/qapi/qmp/dispatch.h
 F: scripts/coccinelle/qobject.cocci
+F: scripts/coccinelle/qobject-parent-type.cocci
 F: tests/check-qdict.c
 F: tests/check-qjson.c
 F: tests/check-qlist.c
-- 
2.28.0

[PATCH v2 11/14] hw/block/nvme: add the zone management send command

[Klaus Jensen]

From: Klaus Jensen 

Add the Zone Management Send command.

Signed-off-by: Klaus Jensen 
---
 hw/block/nvme.h   |   1 +
 include/block/nvme.h  |  29 +++
 hw/block/nvme.c   | 552 --
 hw/block/trace-events |  11 +
 4 files changed, 574 insertions(+), 19 deletions(-)

diff --git a/hw/block/nvme.h b/hw/block/nvme.h
index 523eef0bcad8..c704663e0a3e 100644
--- a/hw/block/nvme.h
+++ b/hw/block/nvme.h
@@ -71,6 +71,7 @@ static inline const char *nvme_io_opc_str(uint8_t opc)
 case NVME_CMD_WRITE:return "NVME_NVM_CMD_WRITE";
 case NVME_CMD_READ: return "NVME_NVM_CMD_READ";
 case NVME_CMD_WRITE_ZEROES: return "NVME_NVM_CMD_WRITE_ZEROES";
+case NVME_CMD_ZONE_MGMT_SEND:   return "NVME_ZONED_CMD_ZONE_MGMT_SEND";
 case NVME_CMD_ZONE_MGMT_RECV:   return "NVME_ZONED_CMD_ZONE_MGMT_RECV";
 default:return "NVME_NVM_CMD_UNKNOWN";
 }
diff --git a/include/block/nvme.h b/include/block/nvme.h
index 9bacf48ee9e9..967b42eb5da7 100644
--- a/include/block/nvme.h
+++ b/include/block/nvme.h
@@ -481,6 +481,7 @@ enum NvmeIoCommands {
 NVME_CMD_COMPARE= 0x05,
 NVME_CMD_WRITE_ZEROES   = 0x08,
 NVME_CMD_DSM= 0x09,
+NVME_CMD_ZONE_MGMT_SEND = 0x79,
 NVME_CMD_ZONE_MGMT_RECV = 0x7a,
 };
 
@@ -594,6 +595,32 @@ enum {
 NVME_RW_PRINFO_PRCHK_REF= 1 << 10,
 };
 
+typedef struct QEMU_PACKED NvmeZoneManagementSendCmd {
+uint8_t opcode;
+uint8_t flags;
+uint16_tcid;
+uint32_tnsid;
+uint32_trsvd8[4];
+NvmeCmdDptr dptr;
+uint64_tslba;
+uint32_trsvd48;
+uint8_t zsa;
+uint8_t zsflags;
+uint16_trsvd54;
+uint32_trsvd56[2];
+} NvmeZoneManagementSendCmd;
+
+#define NVME_CMD_ZONE_MGMT_SEND_SELECT_ALL(zsflags) ((zsflags) & 0x1)
+
+typedef enum NvmeZoneManagementSendAction {
+NVME_CMD_ZONE_MGMT_SEND_CLOSE   = 0x1,
+NVME_CMD_ZONE_MGMT_SEND_FINISH  = 0x2,
+NVME_CMD_ZONE_MGMT_SEND_OPEN= 0x3,
+NVME_CMD_ZONE_MGMT_SEND_RESET   = 0x4,
+NVME_CMD_ZONE_MGMT_SEND_OFFLINE = 0x5,
+NVME_CMD_ZONE_MGMT_SEND_SET_ZDE = 0x10,
+} NvmeZoneManagementSendAction;
+
 typedef struct QEMU_PACKED NvmeZoneManagementRecvCmd {
 uint8_t opcode;
 uint8_t flags;
@@ -748,6 +775,7 @@ enum NvmeStatusCodes {
 NVME_ZONE_IS_READ_ONLY  = 0x01ba,
 NVME_ZONE_IS_OFFLINE= 0x01bb,
 NVME_ZONE_INVALID_WRITE = 0x01bc,
+NVME_INVALID_ZONE_STATE_TRANSITION = 0x01bf,
 NVME_WRITE_FAULT= 0x0280,
 NVME_UNRECOVERED_READ   = 0x0281,
 NVME_E2E_GUARD_ERROR= 0x0282,
@@ -1207,6 +1235,7 @@ static inline void _nvme_check_size(void)
 QEMU_BUILD_BUG_ON(sizeof(NvmeIdentify) != 64);
 QEMU_BUILD_BUG_ON(sizeof(NvmeRwCmd) != 64);
 QEMU_BUILD_BUG_ON(sizeof(NvmeDsmCmd) != 64);
+QEMU_BUILD_BUG_ON(sizeof(NvmeZoneManagementSendCmd) != 64);
 QEMU_BUILD_BUG_ON(sizeof(NvmeZoneManagementRecvCmd) != 64);
 QEMU_BUILD_BUG_ON(sizeof(NvmeRangeType) != 64);
 QEMU_BUILD_BUG_ON(sizeof(NvmeErrorLog) != 64);
diff --git a/hw/block/nvme.c b/hw/block/nvme.c
index 1e6c57752769..5c109cab58e8 100644
--- a/hw/block/nvme.c
+++ b/hw/block/nvme.c
@@ -164,6 +164,8 @@ static const NvmeEffectsLog nvme_effects[NVME_IOCS_MAX] = {
 .iocs = {
 NVME_EFFECTS_NVM_INITIALIZER,
 [NVME_CMD_ZONE_MGMT_RECV] = NVME_EFFECTS_CSUPP,
+[NVME_CMD_ZONE_MGMT_SEND] = NVME_EFFECTS_CSUPP |
+NVME_EFFECTS_LBCC,
 },
 },
 };
@@ -1064,21 +1066,20 @@ static inline uint16_t nvme_check_dulbe(NvmeNamespace 
*ns, uint64_t slba,
 return NVME_SUCCESS;
 }
 
-static int nvme_allocate(NvmeNamespace *ns, uint64_t slba, uint32_t nlb)
+static int __nvme_allocate(NvmeNamespace *ns, uint64_t slba, uint32_t nlb,
+   bool deallocate)
 {
 int nlongs, idx;
 int64_t offset;
 unsigned long *map, *src;
 int ret;
 
-if (!(ns->pstate.blk && nvme_check_dulbe(ns, slba, nlb))) {
-return 0;
+if (deallocate) {
+bitmap_clear(ns->pstate.utilization.map, slba, nlb);
+} else {
+bitmap_set(ns->pstate.utilization.map, slba, nlb);
 }
 
-trace_pci_nvme_allocate(nvme_nsid(ns), slba, nlb);
-
-bitmap_set(ns->pstate.utilization.map, slba, nlb);
-
 /*
  * The bitmap is an array of unsigned longs, so calculate the index given
  * the size of a long.
@@ -1123,6 +1124,28 @@ static int nvme_allocate(NvmeNamespace *ns, uint64_t 
slba, uint32_t nlb)
 return ret;
 }
 
+static int nvme_allocate(NvmeNamespace *ns, uint64_t slba, uint32_t nlb)
+{
+if (!(ns->pstate.blk && nvme_check_dulbe(ns, slba, nlb))) {
+return 0;
+}
+
+trace_pci_nvme_allocate(nvme_nsid(ns), slba, nlb);
+
+return __nvme_allocate(ns, slba, nlb, false /* deallocate */);
+}
+
+static int nvme_deallocate(NvmeNamespace *ns, uint64_t 

[PATCH v2 12/14] hw/block/nvme: add the zone append command

[Klaus Jensen]

From: Klaus Jensen 

Add the Zone Append command.

Signed-off-by: Klaus Jensen 
---
 hw/block/nvme.h   |  6 
 include/block/nvme.h  |  7 +
 hw/block/nvme.c   | 71 +++
 hw/block/trace-events |  1 +
 4 files changed, 79 insertions(+), 6 deletions(-)

diff --git a/hw/block/nvme.h b/hw/block/nvme.h
index c704663e0a3e..8cd2d936548e 100644
--- a/hw/block/nvme.h
+++ b/hw/block/nvme.h
@@ -16,6 +16,10 @@ typedef struct NvmeParams {
 uint32_t aer_max_queued;
 uint8_t  mdts;
 bool use_intel_id;
+
+struct {
+uint8_t zasl;
+} zns;
 } NvmeParams;
 
 typedef struct NvmeAsyncEvent {
@@ -41,6 +45,7 @@ static inline bool nvme_req_is_write(NvmeRequest *req)
 switch (req->cmd.opcode) {
 case NVME_CMD_WRITE:
 case NVME_CMD_WRITE_ZEROES:
+case NVME_CMD_ZONE_APPEND:
 return true;
 default:
 return false;
@@ -73,6 +78,7 @@ static inline const char *nvme_io_opc_str(uint8_t opc)
 case NVME_CMD_WRITE_ZEROES: return "NVME_NVM_CMD_WRITE_ZEROES";
 case NVME_CMD_ZONE_MGMT_SEND:   return "NVME_ZONED_CMD_ZONE_MGMT_SEND";
 case NVME_CMD_ZONE_MGMT_RECV:   return "NVME_ZONED_CMD_ZONE_MGMT_RECV";
+case NVME_CMD_ZONE_APPEND:  return "NVME_ZONED_CMD_ZONE_APPEND";
 default:return "NVME_NVM_CMD_UNKNOWN";
 }
 }
diff --git a/include/block/nvme.h b/include/block/nvme.h
index 967b42eb5da7..5f8914f594f4 100644
--- a/include/block/nvme.h
+++ b/include/block/nvme.h
@@ -483,6 +483,7 @@ enum NvmeIoCommands {
 NVME_CMD_DSM= 0x09,
 NVME_CMD_ZONE_MGMT_SEND = 0x79,
 NVME_CMD_ZONE_MGMT_RECV = 0x7a,
+NVME_CMD_ZONE_APPEND= 0x7d,
 };
 
 typedef struct QEMU_PACKED NvmeDeleteQ {
@@ -1018,6 +1019,11 @@ enum NvmeIdCtrlLpa {
 NVME_LPA_EXTENDED = 1 << 2,
 };
 
+typedef struct QEMU_PACKED NvmeIdCtrlZns {
+uint8_t zasl;
+uint8_t rsvd1[4095];
+} NvmeIdCtrlZns;
+
 #define NVME_CTRL_SQES_MIN(sqes) ((sqes) & 0xf)
 #define NVME_CTRL_SQES_MAX(sqes) (((sqes) >> 4) & 0xf)
 #define NVME_CTRL_CQES_MIN(cqes) ((cqes) & 0xf)
@@ -1242,6 +1248,7 @@ static inline void _nvme_check_size(void)
 QEMU_BUILD_BUG_ON(sizeof(NvmeFwSlotInfoLog) != 512);
 QEMU_BUILD_BUG_ON(sizeof(NvmeSmartLog) != 512);
 QEMU_BUILD_BUG_ON(sizeof(NvmeIdCtrl) != 4096);
+QEMU_BUILD_BUG_ON(sizeof(NvmeIdCtrlZns) != 4096);
 QEMU_BUILD_BUG_ON(sizeof(NvmeIdNsNvm) != 4096);
 QEMU_BUILD_BUG_ON(sizeof(NvmeIdNsZns) != 4096);
 QEMU_BUILD_BUG_ON(sizeof(NvmeSglDescriptor) != 16);
diff --git a/hw/block/nvme.c b/hw/block/nvme.c
index 5c109cab58e8..a891ee284d1d 100644
--- a/hw/block/nvme.c
+++ b/hw/block/nvme.c
@@ -166,6 +166,8 @@ static const NvmeEffectsLog nvme_effects[NVME_IOCS_MAX] = {
 [NVME_CMD_ZONE_MGMT_RECV] = NVME_EFFECTS_CSUPP,
 [NVME_CMD_ZONE_MGMT_SEND] = NVME_EFFECTS_CSUPP |
 NVME_EFFECTS_LBCC,
+[NVME_CMD_ZONE_APPEND]= NVME_EFFECTS_CSUPP |
+NVME_EFFECTS_LBCC,
 },
 },
 };
@@ -1041,6 +1043,21 @@ static inline uint16_t nvme_check_mdts(NvmeCtrl *n, 
size_t len)
 return NVME_SUCCESS;
 }
 
+static inline uint16_t nvme_check_zasl(NvmeCtrl *n, size_t len)
+{
+uint8_t zasl = n->params.zns.zasl;
+
+if (!zasl) {
+return nvme_check_mdts(n, len);
+}
+
+if (len > n->page_size << zasl) {
+return NVME_INVALID_FIELD | NVME_DNR;
+}
+
+return NVME_SUCCESS;
+}
+
 static inline uint16_t nvme_check_bounds(NvmeCtrl *n, NvmeNamespace *ns,
  uint64_t slba, uint32_t nlb)
 {
@@ -1410,6 +1427,7 @@ static uint16_t nvme_do_aio(BlockBackend *blk, int64_t 
offset, size_t len,
 break;
 
 case NVME_CMD_WRITE:
+case NVME_CMD_ZONE_APPEND:
 is_write = true;
 
 /* fallthrough */
@@ -1945,12 +1963,40 @@ static uint16_t nvme_rwz(NvmeCtrl *n, NvmeRequest *req)
 uint32_t nlb = (uint32_t)le16_to_cpu(rw->nlb) + 1;
 size_t len = nvme_l2b(ns, nlb);
 
-bool is_write = nvme_req_is_write(req);
+bool is_append, is_write = nvme_req_is_write(req);
 uint16_t status;
 
 trace_pci_nvme_rwz(nvme_cid(req), nvme_io_opc_str(rw->opcode),
nvme_nsid(ns), nlb, len, slba);
 
+if (req->cmd.opcode == NVME_CMD_ZONE_APPEND) {
+uint64_t wp;
+is_append = true;
+
+zone = nvme_ns_get_zone(ns, slba);
+if (!zone) {
+trace_pci_nvme_err_invalid_zone(nvme_cid(req), slba);
+status = NVME_INVALID_FIELD | NVME_DNR;
+goto invalid;
+}
+
+wp = zone->wp_staging;
+
+if (slba != nvme_zslba(zone)) {
+trace_pci_nvme_err_invalid_zslba(nvme_cid(req), slba);
+return NVME_INVALID_FIELD | NVME_DNR;
+}
+
+status = nvme_check_zasl(n, len);
+if (status) {
+trace_pci_nvme_err_zasl(nvme_cid(req), len);
+   

[PATCH] qom: fix objects with improper parent type

[Sergey Nizovtsev]

Some objects accidentally inherit ObjectClass instead of Object.
They compile silently but may crash after downcasting.

In this patch, we introduce a coccinelle script to find broken
declarations and fix them manually with proper base type.

Signed-off-by: Sergey Nizovtsev 
---
 scripts/coccinelle/qobject-parent-type.cocci | 26 
 include/hw/acpi/vmgenid.h|  2 +-
 include/hw/misc/vmcoreinfo.h |  2 +-
 include/net/can_host.h   |  2 +-
 MAINTAINERS  |  1 +
 5 files changed, 30 insertions(+), 3 deletions(-)
 create mode 100644 scripts/coccinelle/qobject-parent-type.cocci

diff --git a/scripts/coccinelle/qobject-parent-type.cocci
b/scripts/coccinelle/qobject-parent-type.cocci
new file mode 100644
index 00..9afb3edd97
--- /dev/null
+++ b/scripts/coccinelle/qobject-parent-type.cocci
@@ -0,0 +1,26 @@
+// Highlight object declarations that don't look like object class but
+// accidentally inherit from it.
+
+@match@
+identifier obj_t, fld;
+type parent_t =~ ".*Class$";
+@@
+struct obj_t {
+parent_t fld;
+...
+};
+
+@script:python filter depends on match@
+obj_t << match.obj_t;
+@@
+is_class_obj = obj_t.endswith('Class')
+cocci.include_match(not is_class_obj)
+
+@replacement depends on filter@
+identifier match.obj_t, match.fld;
+type match.parent_t;
+@@
+struct obj_t {
+*   parent_t fld;
+...
+};
diff --git a/include/hw/acpi/vmgenid.h b/include/hw/acpi/vmgenid.h
index d50fbacb8e..cb4ad37fc5 100644
--- a/include/hw/acpi/vmgenid.h
+++ b/include/hw/acpi/vmgenid.h
@@ -19,7 +19,7 @@
 OBJECT_DECLARE_SIMPLE_TYPE(VmGenIdState, VMGENID)

 struct VmGenIdState {
-DeviceClass parent_obj;
+DeviceState parent_obj;
 QemuUUID guid;/* The 128-bit GUID seen by the guest */
 uint8_t vmgenid_addr_le[8];   /* Address of the GUID (little-endian) */
 };
diff --git a/include/hw/misc/vmcoreinfo.h b/include/hw/misc/vmcoreinfo.h
index ebada6617a..0b7b55d400 100644
--- a/include/hw/misc/vmcoreinfo.h
+++ b/include/hw/misc/vmcoreinfo.h
@@ -24,7 +24,7 @@ DECLARE_INSTANCE_CHECKER(VMCoreInfoState, VMCOREINFO,
 typedef struct fw_cfg_vmcoreinfo FWCfgVMCoreInfo;

 struct VMCoreInfoState {
-DeviceClass parent_obj;
+DeviceState parent_obj;

 bool has_vmcoreinfo;
 FWCfgVMCoreInfo vmcoreinfo;
diff --git a/include/net/can_host.h b/include/net/can_host.h
index 4e3ce3f954..caab71bdda 100644
--- a/include/net/can_host.h
+++ b/include/net/can_host.h
@@ -35,7 +35,7 @@
 OBJECT_DECLARE_TYPE(CanHostState, CanHostClass, CAN_HOST)

 struct CanHostState {
-ObjectClass oc;
+Object oc;

 CanBusState *bus;
 CanBusClientState bus_client;
diff --git a/MAINTAINERS b/MAINTAINERS
index 5eed1e692b..2160b8196a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2402,6 +2402,7 @@ F: qobject/
 F: include/qapi/qmp/
 X: include/qapi/qmp/dispatch.h
 F: scripts/coccinelle/qobject.cocci
+F: scripts/coccinelle/qobject-parent-type.cocci
 F: tests/check-qdict.c
 F: tests/check-qjson.c
 F: tests/check-qlist.c
-- 
2.28.0

[PATCH v2 05/14] hw/block/nvme: consolidate read, write and write zeroes

[Klaus Jensen]

From: Klaus Jensen 

Consolidate the read/write and write zeroes functions.

Signed-off-by: Klaus Jensen 
---
 hw/block/nvme.h   | 11 
 include/block/nvme.h  |  2 ++
 hw/block/nvme.c   | 65 +++
 hw/block/trace-events |  3 +-
 4 files changed, 37 insertions(+), 44 deletions(-)

diff --git a/hw/block/nvme.h b/hw/block/nvme.h
index e080a2318a50..ccf52ac7bb82 100644
--- a/hw/block/nvme.h
+++ b/hw/block/nvme.h
@@ -36,6 +36,17 @@ typedef struct NvmeRequest {
 QTAILQ_ENTRY(NvmeRequest)entry;
 } NvmeRequest;
 
+static inline bool nvme_req_is_write(NvmeRequest *req)
+{
+switch (req->cmd.opcode) {
+case NVME_CMD_WRITE:
+case NVME_CMD_WRITE_ZEROES:
+return true;
+default:
+return false;
+}
+}
+
 static inline const char *nvme_adm_opc_str(uint8_t opc)
 {
 switch (opc) {
diff --git a/include/block/nvme.h b/include/block/nvme.h
index 7a30cf285ae0..999b4f8ae0d4 100644
--- a/include/block/nvme.h
+++ b/include/block/nvme.h
@@ -438,6 +438,8 @@ typedef struct QEMU_PACKED NvmeCmd {
 uint32_tcdw15;
 } NvmeCmd;
 
+#define NVME_CMD_OPCODE_DATA_TRANSFER_MASK 0x3
+
 #define NVME_CMD_FLAGS_FUSE(flags) (flags & 0x3)
 #define NVME_CMD_FLAGS_PSDT(flags) ((flags >> 6) & 0x3)
 
diff --git a/hw/block/nvme.c b/hw/block/nvme.c
index 27af2f0b38d5..795c7e7c529f 100644
--- a/hw/block/nvme.c
+++ b/hw/block/nvme.c
@@ -997,48 +997,19 @@ static uint16_t nvme_flush(NvmeCtrl *n, NvmeRequest *req)
 return nvme_do_aio(ns->blkconf.blk, 0, 0, req);
 }
 
-static uint16_t nvme_write_zeroes(NvmeCtrl *n, NvmeRequest *req)
+static uint16_t nvme_rwz(NvmeCtrl *n, NvmeRequest *req)
 {
 NvmeRwCmd *rw = (NvmeRwCmd *)>cmd;
 NvmeNamespace *ns = req->ns;
+
 uint64_t slba = le64_to_cpu(rw->slba);
 uint32_t nlb = (uint32_t)le16_to_cpu(rw->nlb) + 1;
-uint64_t offset = nvme_l2b(ns, slba);
-uint32_t count = nvme_l2b(ns, nlb);
+size_t len = nvme_l2b(ns, nlb);
+
 uint16_t status;
 
-trace_pci_nvme_write_zeroes(nvme_cid(req), nvme_nsid(ns), slba, nlb);
-
-status = nvme_check_bounds(n, ns, slba, nlb);
-if (status) {
-trace_pci_nvme_err_invalid_lba_range(slba, nlb, ns->id_ns.nsze);
-return status;
-}
-
-return nvme_do_aio(ns->blkconf.blk, offset, count, req);
-}
-
-static uint16_t nvme_rw(NvmeCtrl *n, NvmeRequest *req)
-{
-NvmeRwCmd *rw = (NvmeRwCmd *)>cmd;
-NvmeNamespace *ns = req->ns;
-uint32_t nlb = (uint32_t)le16_to_cpu(rw->nlb) + 1;
-uint64_t slba = le64_to_cpu(rw->slba);
-
-uint64_t data_size = nvme_l2b(ns, nlb);
-uint64_t data_offset = nvme_l2b(ns, slba);
-enum BlockAcctType acct = req->cmd.opcode == NVME_CMD_WRITE ?
-BLOCK_ACCT_WRITE : BLOCK_ACCT_READ;
-uint16_t status;
-
-trace_pci_nvme_rw(nvme_cid(req), nvme_io_opc_str(rw->opcode),
-  nvme_nsid(ns), nlb, data_size, slba);
-
-status = nvme_check_mdts(n, data_size);
-if (status) {
-trace_pci_nvme_err_mdts(nvme_cid(req), data_size);
-goto invalid;
-}
+trace_pci_nvme_rwz(nvme_cid(req), nvme_io_opc_str(rw->opcode),
+   nvme_nsid(ns), nlb, len, slba);
 
 status = nvme_check_bounds(n, ns, slba, nlb);
 if (status) {
@@ -1046,15 +1017,26 @@ static uint16_t nvme_rw(NvmeCtrl *n, NvmeRequest *req)
 goto invalid;
 }
 
-status = nvme_map_dptr(n, data_size, req);
-if (status) {
-goto invalid;
+if (req->cmd.opcode & NVME_CMD_OPCODE_DATA_TRANSFER_MASK) {
+status = nvme_check_mdts(n, len);
+if (status) {
+trace_pci_nvme_err_mdts(nvme_cid(req), len);
+goto invalid;
+}
+
+status = nvme_map_dptr(n, len, req);
+if (status) {
+goto invalid;
+}
 }
 
-return nvme_do_aio(ns->blkconf.blk, data_offset, data_size, req);
+return nvme_do_aio(ns->blkconf.blk, nvme_l2b(ns, slba), len, req);
 
 invalid:
-block_acct_invalid(blk_get_stats(ns->blkconf.blk), acct);
+block_acct_invalid(blk_get_stats(ns->blkconf.blk),
+   nvme_req_is_write(req) ? BLOCK_ACCT_WRITE :
+   BLOCK_ACCT_READ);
+
 return status;
 }
 
@@ -1082,10 +1064,9 @@ static uint16_t nvme_io_cmd(NvmeCtrl *n, NvmeRequest 
*req)
 case NVME_CMD_FLUSH:
 return nvme_flush(n, req);
 case NVME_CMD_WRITE_ZEROES:
-return nvme_write_zeroes(n, req);
 case NVME_CMD_WRITE:
 case NVME_CMD_READ:
-return nvme_rw(n, req);
+return nvme_rwz(n, req);
 default:
 trace_pci_nvme_err_invalid_opc(req->cmd.opcode);
 return NVME_INVALID_OPCODE | NVME_DNR;
diff --git a/hw/block/trace-events b/hw/block/trace-events
index 9e7507c5abde..b18056c49836 100644
--- a/hw/block/trace-events
+++ b/hw/block/trace-events
@@ -40,9 +40,8 @@ pci_nvme_map_prp(uint64_t trans_len, uint32_t len, uint64_t 
prp1, uint64_t prp2,
 pci_nvme_map_sgl(uint16_t cid, uint8_t typ, uint64_t len) "cid 

[PATCH v2 13/14] hw/block/nvme: track and enforce zone resources

[Klaus Jensen]

From: Klaus Jensen 

Track number of open/active resources.

Signed-off-by: Klaus Jensen 
---
 docs/specs/nvme.txt  |  7 +
 hw/block/nvme-ns.h   |  7 +
 include/block/nvme.h |  2 ++
 hw/block/nvme-ns.c   | 25 +++--
 hw/block/nvme.c  | 67 +++-
 5 files changed, 104 insertions(+), 4 deletions(-)

diff --git a/docs/specs/nvme.txt b/docs/specs/nvme.txt
index b23e59dd3075..e3810843cd2d 100644
--- a/docs/specs/nvme.txt
+++ b/docs/specs/nvme.txt
@@ -42,6 +42,13 @@ nvme-ns Options
  zns.zcap; if the zone capacity is a power of two, the zone size will be
  set to that, otherwise it will default to the next power of two.
 
+  `zns.mar`; Specifies the number of active resources available. This is a 0s
+ based value.
+
+  `zns.mor`; Specifies the number of open resources available. This is a 0s
+ based value.
+
+
 Reference Specifications
 
 
diff --git a/hw/block/nvme-ns.h b/hw/block/nvme-ns.h
index 82cb0b0bce82..ff34cd37af7d 100644
--- a/hw/block/nvme-ns.h
+++ b/hw/block/nvme-ns.h
@@ -52,6 +52,8 @@ typedef struct NvmeNamespaceParams {
 uint64_t zcap;
 uint64_t zsze;
 uint8_t  zdes;
+uint32_t mar;
+uint32_t mor;
 } zns;
 } NvmeNamespaceParams;
 
@@ -95,6 +97,11 @@ typedef struct NvmeNamespace {
 NvmeZone   *zones;
 NvmeZoneDescriptor *zd;
 uint8_t*zde;
+
+struct {
+uint32_t open;
+uint32_t active;
+} resources;
 } zns;
 } NvmeNamespace;
 
diff --git a/include/block/nvme.h b/include/block/nvme.h
index 5f8914f594f4..d51f397e7ff1 100644
--- a/include/block/nvme.h
+++ b/include/block/nvme.h
@@ -776,6 +776,8 @@ enum NvmeStatusCodes {
 NVME_ZONE_IS_READ_ONLY  = 0x01ba,
 NVME_ZONE_IS_OFFLINE= 0x01bb,
 NVME_ZONE_INVALID_WRITE = 0x01bc,
+NVME_TOO_MANY_ACTIVE_ZONES  = 0x01bd,
+NVME_TOO_MANY_OPEN_ZONES= 0x01be,
 NVME_INVALID_ZONE_STATE_TRANSITION = 0x01bf,
 NVME_WRITE_FAULT= 0x0280,
 NVME_UNRECOVERED_READ   = 0x0281,
diff --git a/hw/block/nvme-ns.c b/hw/block/nvme-ns.c
index bc49c7f2674f..9584fbb3f62d 100644
--- a/hw/block/nvme-ns.c
+++ b/hw/block/nvme-ns.c
@@ -119,8 +119,8 @@ static void nvme_ns_init_zoned(NvmeNamespace *ns)
 ns->zns.zde = g_malloc0_n(ns->zns.num_zones, nvme_ns_zdes_bytes(ns));
 }
 
-id_ns_zns->mar = 0x;
-id_ns_zns->mor = 0x;
+id_ns_zns->mar = cpu_to_le32(ns->params.zns.mar);
+id_ns_zns->mor = cpu_to_le32(ns->params.zns.mor);
 }
 
 static void nvme_ns_init(NvmeNamespace *ns)
@@ -220,6 +220,11 @@ static int nvme_ns_pstate_init(NvmeNamespace *ns, Error 
**errp)
 
 void nvme_ns_zns_init_zone_state(NvmeNamespace *ns)
 {
+ns->zns.resources.active = ns->params.zns.mar != 0x ?
+ns->params.zns.mar + 1 : ns->zns.num_zones;
+ns->zns.resources.open = ns->params.zns.mor != 0x ?
+ns->params.zns.mor + 1 : ns->zns.num_zones;
+
 for (int i = 0; i < ns->zns.num_zones; i++) {
 NvmeZone *zone = >zns.zones[i];
 zone->zd = >zns.zd[i];
@@ -238,9 +243,15 @@ void nvme_ns_zns_init_zone_state(NvmeNamespace *ns)
 if (nvme_wp(zone) == nvme_zslba(zone) &&
 !(zone->zd->za & NVME_ZA_ZDEV)) {
 nvme_zs_set(zone, NVME_ZS_ZSE);
+continue;
 }
 
-continue;
+if (ns->zns.resources.active) {
+ns->zns.resources.active--;
+continue;
+}
+
+/* fallthrough */
 
 case NVME_ZS_ZSIO:
 case NVME_ZS_ZSEO:
@@ -462,6 +473,12 @@ static int nvme_ns_check_constraints(NvmeNamespace *ns, 
Error **errp)
 return -1;
 }
 
+if (ns->params.zns.mor > ns->params.zns.mar) {
+error_setg(errp, "maximum open resources (zns.mor) must be less "
+   "than or equal to maximum active resources (zns.mar)");
+return -1;
+}
+
 break;
 
 default:
@@ -547,6 +564,8 @@ static Property nvme_ns_props[] = {
 DEFINE_PROP_UINT64("zns.zcap", NvmeNamespace, params.zns.zcap, 0),
 DEFINE_PROP_UINT64("zns.zsze", NvmeNamespace, params.zns.zsze, 0),
 DEFINE_PROP_UINT8("zns.zdes", NvmeNamespace, params.zns.zdes, 0),
+DEFINE_PROP_UINT32("zns.mar", NvmeNamespace, params.zns.mar, 0x),
+DEFINE_PROP_UINT32("zns.mor", NvmeNamespace, params.zns.mor, 0x),
 DEFINE_PROP_END_OF_LIST(),
 };
 
diff --git a/hw/block/nvme.c b/hw/block/nvme.c
index a891ee284d1d..fc5b119e3f35 100644
--- a/hw/block/nvme.c
+++ b/hw/block/nvme.c
@@ -1221,6 +1221,40 @@ static uint16_t nvme_zrm_transition(NvmeNamespace *ns, 
NvmeZone *zone,
 
 switch (from) {
 case NVME_ZS_ZSE:
+switch (to) {
+case NVME_ZS_ZSF:
+case NVME_ZS_ZSRO:
+case NVME_ZS_ZSO:
+break;
+
+case NVME_ZS_ZSC:
+   

[PATCH v2 10/14] hw/block/nvme: add the zone management receive command

[Klaus Jensen]

From: Klaus Jensen 

Add the Zone Management Receive command.

Signed-off-by: Klaus Jensen 
---
 hw/block/nvme-ns.h|  11 +++-
 hw/block/nvme.h   |   1 +
 include/block/nvme.h  |  46 ++
 hw/block/nvme-ns.c|  49 ---
 hw/block/nvme.c   | 135 ++
 hw/block/trace-events |   1 +
 6 files changed, 233 insertions(+), 10 deletions(-)

diff --git a/hw/block/nvme-ns.h b/hw/block/nvme-ns.h
index e16f50dc4bb8..82cb0b0bce82 100644
--- a/hw/block/nvme-ns.h
+++ b/hw/block/nvme-ns.h
@@ -37,9 +37,10 @@ typedef struct NvmePstateHeader {
 struct {
 uint64_t zcap;
 uint64_t zsze;
+uint8_t  zdes;
 } QEMU_PACKED zns;
 
-uint8_t  rsvd3088[1008];
+uint8_t  rsvd3089[1007];
 } QEMU_PACKED NvmePstateHeader;
 
 typedef struct NvmeNamespaceParams {
@@ -50,11 +51,13 @@ typedef struct NvmeNamespaceParams {
 struct {
 uint64_t zcap;
 uint64_t zsze;
+uint8_t  zdes;
 } zns;
 } NvmeNamespaceParams;
 
 typedef struct NvmeZone {
 NvmeZoneDescriptor *zd;
+uint8_t*zde;
 
 uint64_t wp_staging;
 } NvmeZone;
@@ -91,6 +94,7 @@ typedef struct NvmeNamespace {
 
 NvmeZone   *zones;
 NvmeZoneDescriptor *zd;
+uint8_t*zde;
 } zns;
 } NvmeNamespace;
 
@@ -183,6 +187,11 @@ static inline void nvme_zs_set(NvmeZone *zone, 
NvmeZoneState zs)
 zone->zd->zs = zs << 4;
 }
 
+static inline size_t nvme_ns_zdes_bytes(NvmeNamespace *ns)
+{
+return ns->params.zns.zdes << 6;
+}
+
 static inline bool nvme_ns_zone_wp_valid(NvmeZone *zone)
 {
 switch (nvme_zs(zone)) {
diff --git a/hw/block/nvme.h b/hw/block/nvme.h
index f66ed9ab7eff..523eef0bcad8 100644
--- a/hw/block/nvme.h
+++ b/hw/block/nvme.h
@@ -71,6 +71,7 @@ static inline const char *nvme_io_opc_str(uint8_t opc)
 case NVME_CMD_WRITE:return "NVME_NVM_CMD_WRITE";
 case NVME_CMD_READ: return "NVME_NVM_CMD_READ";
 case NVME_CMD_WRITE_ZEROES: return "NVME_NVM_CMD_WRITE_ZEROES";
+case NVME_CMD_ZONE_MGMT_RECV:   return "NVME_ZONED_CMD_ZONE_MGMT_RECV";
 default:return "NVME_NVM_CMD_UNKNOWN";
 }
 }
diff --git a/include/block/nvme.h b/include/block/nvme.h
index 2e523c9d97b4..9bacf48ee9e9 100644
--- a/include/block/nvme.h
+++ b/include/block/nvme.h
@@ -481,6 +481,7 @@ enum NvmeIoCommands {
 NVME_CMD_COMPARE= 0x05,
 NVME_CMD_WRITE_ZEROES   = 0x08,
 NVME_CMD_DSM= 0x09,
+NVME_CMD_ZONE_MGMT_RECV = 0x7a,
 };
 
 typedef struct QEMU_PACKED NvmeDeleteQ {
@@ -593,6 +594,44 @@ enum {
 NVME_RW_PRINFO_PRCHK_REF= 1 << 10,
 };
 
+typedef struct QEMU_PACKED NvmeZoneManagementRecvCmd {
+uint8_t opcode;
+uint8_t flags;
+uint16_tcid;
+uint32_tnsid;
+uint8_t rsvd8[16];
+NvmeCmdDptr dptr;
+uint64_tslba;
+uint32_tnumdw;
+uint8_t zra;
+uint8_t zrasp;
+uint8_t zrasf;
+uint8_t rsvd55[9];
+} NvmeZoneManagementRecvCmd;
+
+typedef enum NvmeZoneManagementRecvAction {
+NVME_CMD_ZONE_MGMT_RECV_REPORT_ZONES  = 0x0,
+NVME_CMD_ZONE_MGMT_RECV_EXTENDED_REPORT_ZONES = 0x1,
+} NvmeZoneManagementRecvAction;
+
+typedef enum NvmeZoneManagementRecvActionSpecificField {
+NVME_CMD_ZONE_MGMT_RECV_LIST_ALL  = 0x0,
+NVME_CMD_ZONE_MGMT_RECV_LIST_ZSE  = 0x1,
+NVME_CMD_ZONE_MGMT_RECV_LIST_ZSIO = 0x2,
+NVME_CMD_ZONE_MGMT_RECV_LIST_ZSEO = 0x3,
+NVME_CMD_ZONE_MGMT_RECV_LIST_ZSC  = 0x4,
+NVME_CMD_ZONE_MGMT_RECV_LIST_ZSF  = 0x5,
+NVME_CMD_ZONE_MGMT_RECV_LIST_ZSRO = 0x6,
+NVME_CMD_ZONE_MGMT_RECV_LIST_ZSO  = 0x7,
+} NvmeZoneManagementRecvActionSpecificField;
+
+#define NVME_CMD_ZONE_MGMT_RECEIVE_PARTIAL 0x1
+
+typedef struct QEMU_PACKED NvmeZoneReportHeader {
+uint64_t num_zones;
+uint8_t  rsvd[56];
+} NvmeZoneReportHeader;
+
 typedef struct QEMU_PACKED NvmeDsmCmd {
 uint8_t opcode;
 uint8_t flags;
@@ -812,6 +851,12 @@ typedef struct QEMU_PACKED NvmeZoneDescriptor {
 uint8_t  rsvd32[32];
 } NvmeZoneDescriptor;
 
+#define NVME_ZA_ZDEV (1 << 7)
+
+#define NVME_ZA_SET(za, attrs)   ((za) |= (attrs))
+#define NVME_ZA_CLEAR(za, attrs) ((za) &= ~(attrs))
+#define NVME_ZA_CLEAR_ALL(za)((za) = 0x0)
+
 enum NvmeSmartWarn {
 NVME_SMART_SPARE  = 1 << 0,
 NVME_SMART_TEMPERATURE= 1 << 1,
@@ -1162,6 +1207,7 @@ static inline void _nvme_check_size(void)
 QEMU_BUILD_BUG_ON(sizeof(NvmeIdentify) != 64);
 QEMU_BUILD_BUG_ON(sizeof(NvmeRwCmd) != 64);
 QEMU_BUILD_BUG_ON(sizeof(NvmeDsmCmd) != 64);
+QEMU_BUILD_BUG_ON(sizeof(NvmeZoneManagementRecvCmd) != 64);
 QEMU_BUILD_BUG_ON(sizeof(NvmeRangeType) != 64);
 QEMU_BUILD_BUG_ON(sizeof(NvmeErrorLog) != 64);
 QEMU_BUILD_BUG_ON(sizeof(NvmeFwSlotInfoLog) != 512);
diff --git a/hw/block/nvme-ns.c b/hw/block/nvme-ns.c
index ba32ea6d1326..bc49c7f2674f 100644
--- 

[PATCH v2 08/14] hw/block/nvme: support namespace types

[Klaus Jensen]

From: Klaus Jensen 

Implement support for TP 4056 ("Namespace Types"). This adds the 'iocs'
(I/O Command Set) device parameter to the nvme-ns device.

Signed-off-by: Klaus Jensen 
---
 docs/specs/nvme.txt   |   3 +
 hw/block/nvme-ns.h|  14 ++-
 hw/block/nvme.h   |   3 +
 include/block/nvme.h  |  58 +--
 block/nvme.c  |   4 +-
 hw/block/nvme-ns.c|  29 +-
 hw/block/nvme.c   | 228 ++
 hw/block/trace-events |   6 +-
 8 files changed, 285 insertions(+), 60 deletions(-)

diff --git a/docs/specs/nvme.txt b/docs/specs/nvme.txt
index 6d00ac064998..a13c7a5dbe86 100644
--- a/docs/specs/nvme.txt
+++ b/docs/specs/nvme.txt
@@ -12,6 +12,9 @@ nvme-ns Options
  namespace. It is specified in terms of a power of two. Only values between
  9 and 12 (both inclusive) are supported.
 
+  `iocs`; The "I/O Command Set" associated with the namespace. E.g. 0x0 for the
+ NVM Command Set (the default), or 0x2 for the Zoned Namespace Command Set.
+
   `pstate`; This parameter specifies another blockdev to be used for storing
  persistent state such as logical block allocation tracking. Adding this
  parameter enables various optional features of the device.
diff --git a/hw/block/nvme-ns.h b/hw/block/nvme-ns.h
index 0ad83910dde9..d39fa79fa682 100644
--- a/hw/block/nvme-ns.h
+++ b/hw/block/nvme-ns.h
@@ -29,12 +29,14 @@ typedef struct NvmePstateHeader {
 int64_t  blk_len;
 
 uint8_t  lbads;
+uint8_t  iocs;
 
-uint8_t  rsvd17[4079];
+uint8_t  rsvd18[4078];
 } QEMU_PACKED NvmePstateHeader;
 
 typedef struct NvmeNamespaceParams {
 uint32_t nsid;
+uint8_t  iocs;
 uint8_t  lbads;
 } NvmeNamespaceParams;
 
@@ -43,7 +45,8 @@ typedef struct NvmeNamespace {
 BlockConfblkconf;
 int32_t  bootindex;
 int64_t  size;
-NvmeIdNs id_ns;
+uint8_t  iocs;
+void *id_ns[NVME_IOCS_MAX];
 
 struct {
 BlockBackend *blk;
@@ -70,9 +73,14 @@ static inline uint32_t nvme_nsid(NvmeNamespace *ns)
 return -1;
 }
 
+static inline NvmeIdNsNvm *nvme_ns_id_nvm(NvmeNamespace *ns)
+{
+return ns->id_ns[NVME_IOCS_NVM];
+}
+
 static inline NvmeLBAF *nvme_ns_lbaf(NvmeNamespace *ns)
 {
-NvmeIdNs *id_ns = >id_ns;
+NvmeIdNsNvm *id_ns = nvme_ns_id_nvm(ns);
 return _ns->lbaf[NVME_ID_NS_FLBAS_INDEX(id_ns->flbas)];
 }
 
diff --git a/hw/block/nvme.h b/hw/block/nvme.h
index ccf52ac7bb82..f66ed9ab7eff 100644
--- a/hw/block/nvme.h
+++ b/hw/block/nvme.h
@@ -123,6 +123,7 @@ typedef struct NvmeFeatureVal {
 };
 uint32_tasync_config;
 uint32_tvwc;
+uint32_tiocsci;
 } NvmeFeatureVal;
 
 typedef struct NvmeCtrl {
@@ -150,6 +151,7 @@ typedef struct NvmeCtrl {
 uint64_ttimestamp_set_qemu_clock_ms;/* QEMU clock time */
 uint64_tstarttime_ms;
 uint16_ttemperature;
+uint64_tiocscs[512];
 
 HostMemoryBackend *pmrdev;
 
@@ -165,6 +167,7 @@ typedef struct NvmeCtrl {
 NvmeSQueue  admin_sq;
 NvmeCQueue  admin_cq;
 NvmeIdCtrl  id_ctrl;
+void*id_ctrl_iocss[NVME_IOCS_MAX];
 NvmeFeatureVal  features;
 } NvmeCtrl;
 
diff --git a/include/block/nvme.h b/include/block/nvme.h
index 5a5e19f6bedc..792fccf8c81f 100644
--- a/include/block/nvme.h
+++ b/include/block/nvme.h
@@ -82,6 +82,11 @@ enum NvmeCapMask {
 #define NVME_CAP_SET_PMRS(cap, val) (cap |= (uint64_t)(val & CAP_PMR_MASK)\
 << CAP_PMR_SHIFT)
 
+enum NvmeCapCss {
+NVME_CAP_CSS_NVM = 1 << 0,
+NVME_CAP_CSS_CSI = 1 << 6,
+};
+
 enum NvmeCcShift {
 CC_EN_SHIFT = 0,
 CC_CSS_SHIFT= 4,
@@ -112,6 +117,7 @@ enum NvmeCcMask {
 
 enum NvmeCcCss {
 NVME_CC_CSS_NVM= 0x0,
+NVME_CC_CSS_ALL= 0x6,
 NVME_CC_CSS_ADMIN_ONLY = 0x7,
 };
 
@@ -383,6 +389,11 @@ enum NvmePmrmscMask {
 #define NVME_PMRMSC_SET_CBA(pmrmsc, val)   \
 (pmrmsc |= (uint64_t)(val & PMRMSC_CBA_MASK) << PMRMSC_CBA_SHIFT)
 
+enum NvmeCommandSet {
+NVME_IOCS_NVM = 0x0,
+NVME_IOCS_MAX = 0x1,
+};
+
 enum NvmeSglDescriptorType {
 NVME_SGL_DESCR_TYPE_DATA_BLOCK  = 0x0,
 NVME_SGL_DESCR_TYPE_BIT_BUCKET  = 0x1,
@@ -531,8 +542,13 @@ typedef struct QEMU_PACKED NvmeIdentify {
 uint64_trsvd2[2];
 uint64_tprp1;
 uint64_tprp2;
-uint32_tcns;
-uint32_trsvd11[5];
+uint8_t cns;
+uint8_t rsvd3;
+uint16_tcntid;
+uint16_tnvmsetid;
+uint8_t rsvd4;
+uint8_t csi;
+uint32_trsvd11[4];
 } NvmeIdentify;
 
 typedef struct QEMU_PACKED NvmeRwCmd {
@@ -624,8 +640,15 @@ typedef struct QEMU_PACKED NvmeAerResult {
 } NvmeAerResult;
 
 typedef struct QEMU_PACKED NvmeCqe {
-uint32_tresult;
-uint32_trsvd;
+union {
+struct {
+uint32_tdw0;
+uint32_tdw1;
+};
+
+uint64_t qw0;
+};
+
 uint16_tsq_head;
 

[PATCH v2 07/14] hw/block/nvme: add commands supported and effects log page

[Klaus Jensen]

From: Gollu Appalanaidu 

This is to support for the Commands Supported and Effects log page. See
NVM Express Spec 1.3d, sec. 5.14.1.5 ("Commands Supported and Effects")

Signed-off-by: Gollu Appalanaidu 
Signed-off-by: Klaus Jensen 
---
 include/block/nvme.h | 21 +
 hw/block/nvme.c  | 75 +++-
 2 files changed, 95 insertions(+), 1 deletion(-)

diff --git a/include/block/nvme.h b/include/block/nvme.h
index abd49d371e63..5a5e19f6bedc 100644
--- a/include/block/nvme.h
+++ b/include/block/nvme.h
@@ -734,6 +734,24 @@ typedef struct QEMU_PACKED NvmeSmartLog {
 uint8_t reserved2[320];
 } NvmeSmartLog;
 
+typedef struct QEMU_PACKED NvmeEffectsLog {
+uint32_t acs[256];
+uint32_t iocs[256];
+uint8_t  rsvd2048[2048];
+} NvmeEffectsLog;
+
+enum {
+NVME_EFFECTS_CSUPP  = 1 <<  0,
+NVME_EFFECTS_LBCC   = 1 <<  1,
+NVME_EFFECTS_NCC= 1 <<  2,
+NVME_EFFECTS_NIC= 1 <<  3,
+NVME_EFFECTS_CCC= 1 <<  4,
+NVME_EFFECTS_CSE_SINGLE = 1 << 16,
+NVME_EFFECTS_CSE_MULTI  = 1 << 17,
+NVME_EFFECTS_CSE_MASK   = 3 << 16,
+NVME_EFFECTS_UUID_SEL   = 1 << 19,
+};
+
 enum NvmeSmartWarn {
 NVME_SMART_SPARE  = 1 << 0,
 NVME_SMART_TEMPERATURE= 1 << 1,
@@ -746,6 +764,7 @@ enum NvmeLogIdentifier {
 NVME_LOG_ERROR_INFO = 0x01,
 NVME_LOG_SMART_INFO = 0x02,
 NVME_LOG_FW_SLOT_INFO   = 0x03,
+NVME_LOG_EFFECTS= 0x05,
 };
 
 typedef struct QEMU_PACKED NvmePSD {
@@ -857,6 +876,7 @@ enum NvmeIdCtrlFrmw {
 };
 
 enum NvmeIdCtrlLpa {
+NVME_LPA_EFFECTS_LOG  = 1 << 1,
 NVME_LPA_EXTENDED = 1 << 2,
 };
 
@@ -1064,5 +1084,6 @@ static inline void _nvme_check_size(void)
 QEMU_BUILD_BUG_ON(sizeof(NvmeIdNs) != 4096);
 QEMU_BUILD_BUG_ON(sizeof(NvmeSglDescriptor) != 16);
 QEMU_BUILD_BUG_ON(sizeof(NvmeIdNsDescr) != 4);
+QEMU_BUILD_BUG_ON(sizeof(NvmeEffectsLog) != 4096);
 }
 #endif
diff --git a/hw/block/nvme.c b/hw/block/nvme.c
index be5a0a7dfa09..0bc19a1e3688 100644
--- a/hw/block/nvme.c
+++ b/hw/block/nvme.c
@@ -81,6 +81,7 @@
 #define NVME_TEMPERATURE_WARNING 0x157
 #define NVME_TEMPERATURE_CRITICAL 0x175
 #define NVME_NUM_FW_SLOTS 1
+#define NVME_MAX_ADM_IO_CMDS 0xFF
 
 #define NVME_GUEST_ERR(trace, fmt, ...) \
 do { \
@@ -112,6 +113,46 @@ static const uint32_t nvme_feature_cap[NVME_FID_MAX] = {
 [NVME_TIMESTAMP]= NVME_FEAT_CAP_CHANGE,
 };
 
+#define NVME_EFFECTS_ADMIN_INITIALIZER \
+[NVME_ADM_CMD_DELETE_SQ]= NVME_EFFECTS_CSUPP,  \
+[NVME_ADM_CMD_CREATE_SQ]= NVME_EFFECTS_CSUPP,  \
+[NVME_ADM_CMD_GET_LOG_PAGE] = NVME_EFFECTS_CSUPP,  \
+[NVME_ADM_CMD_DELETE_CQ]= NVME_EFFECTS_CSUPP,  \
+[NVME_ADM_CMD_CREATE_CQ]= NVME_EFFECTS_CSUPP,  \
+[NVME_ADM_CMD_IDENTIFY] = NVME_EFFECTS_CSUPP,  \
+[NVME_ADM_CMD_ABORT]= NVME_EFFECTS_CSUPP,  \
+[NVME_ADM_CMD_SET_FEATURES] = NVME_EFFECTS_CSUPP | \
+  NVME_EFFECTS_CCC |   \
+  NVME_EFFECTS_NIC |   \
+  NVME_EFFECTS_NCC,\
+[NVME_ADM_CMD_GET_FEATURES] = NVME_EFFECTS_CSUPP,  \
+[NVME_ADM_CMD_ASYNC_EV_REQ] = NVME_EFFECTS_CSUPP
+
+#define NVME_EFFECTS_NVM_INITIALIZER   \
+[NVME_CMD_FLUSH]= NVME_EFFECTS_CSUPP | \
+  NVME_EFFECTS_LBCC,   \
+[NVME_CMD_WRITE]= NVME_EFFECTS_CSUPP | \
+  NVME_EFFECTS_LBCC,   \
+[NVME_CMD_READ] = NVME_EFFECTS_CSUPP,  \
+[NVME_CMD_WRITE_ZEROES] = NVME_EFFECTS_CSUPP | \
+  NVME_EFFECTS_LBCC
+
+static const NvmeEffectsLog nvme_effects_admin_only = {
+.acs = {
+NVME_EFFECTS_ADMIN_INITIALIZER,
+},
+};
+
+static const NvmeEffectsLog nvme_effects = {
+.acs = {
+NVME_EFFECTS_ADMIN_INITIALIZER,
+},
+
+.iocs = {
+NVME_EFFECTS_NVM_INITIALIZER,
+},
+};
+
 static void nvme_process_sq(void *opaque);
 
 static uint16_t nvme_cid(NvmeRequest *req)
@@ -1382,6 +1423,36 @@ static uint16_t nvme_error_info(NvmeCtrl *n, uint8_t 
rae, uint32_t buf_len,
 DMA_DIRECTION_FROM_DEVICE, req);
 }
 
+static uint16_t nvme_effects_log(NvmeCtrl *n, uint32_t buf_len, uint64_t off,
+ NvmeRequest *req)
+{
+const NvmeEffectsLog *effects;
+
+uint32_t trans_len;
+
+if (off > sizeof(NvmeEffectsLog)) {
+return NVME_INVALID_FIELD | NVME_DNR;
+}
+
+switch (NVME_CC_CSS(n->bar.cc)) {
+case NVME_CC_CSS_ADMIN_ONLY:
+effects = _effects_admin_only;
+break;
+
+case NVME_CC_CSS_NVM:
+effects = _effects;
+break;
+
+default:
+return NVME_INTERNAL_DEV_ERROR | NVME_DNR;
+}
+
+trans_len = MIN(sizeof(NvmeEffectsLog) - off, buf_len);
+
+return nvme_dma(n, (uint8_t *)effects + off, 

[PATCH v2 04/14] hw/block/nvme: reject io commands if only admin command set selected

[Klaus Jensen]

From: Klaus Jensen 

If the host sets CC.CSS to 111b, all commands submitted to I/O queues
should be completed with status Invalid Command Opcode.

Note that this is technically a v1.4 feature, but it does not hurt to
implement before we finally bump the reported version implemented.

Signed-off-by: Klaus Jensen 
---
 include/block/nvme.h | 5 +
 hw/block/nvme.c  | 4 
 2 files changed, 9 insertions(+)

diff --git a/include/block/nvme.h b/include/block/nvme.h
index 58647bcdad0b..7a30cf285ae0 100644
--- a/include/block/nvme.h
+++ b/include/block/nvme.h
@@ -110,6 +110,11 @@ enum NvmeCcMask {
 #define NVME_CC_IOSQES(cc) ((cc >> CC_IOSQES_SHIFT) & CC_IOSQES_MASK)
 #define NVME_CC_IOCQES(cc) ((cc >> CC_IOCQES_SHIFT) & CC_IOCQES_MASK)
 
+enum NvmeCcCss {
+NVME_CC_CSS_NVM= 0x0,
+NVME_CC_CSS_ADMIN_ONLY = 0x7,
+};
+
 enum NvmeCstsShift {
 CSTS_RDY_SHIFT  = 0,
 CSTS_CFS_SHIFT  = 1,
diff --git a/hw/block/nvme.c b/hw/block/nvme.c
index 758f58c88026..27af2f0b38d5 100644
--- a/hw/block/nvme.c
+++ b/hw/block/nvme.c
@@ -1065,6 +1065,10 @@ static uint16_t nvme_io_cmd(NvmeCtrl *n, NvmeRequest 
*req)
 trace_pci_nvme_io_cmd(nvme_cid(req), nsid, nvme_sqid(req),
   req->cmd.opcode, nvme_io_opc_str(req->cmd.opcode));
 
+if (NVME_CC_CSS(n->bar.cc) == NVME_CC_CSS_ADMIN_ONLY) {
+return NVME_INVALID_OPCODE | NVME_DNR;
+}
+
 if (!nvme_nsid_valid(n, nsid)) {
 return NVME_INVALID_NSID | NVME_DNR;
 }
-- 
2.28.0

[PATCH v2 09/14] hw/block/nvme: add basic read/write for zoned namespaces

[Klaus Jensen]

From: Klaus Jensen 

This adds basic read and write for zoned namespaces.

A zoned namespace is created by setting the iocs namespace parameter to
0x2 and specifying the zns.zcap parameter (zone capacity) in number of
logical blocks per zone. If a zone size (zns.zsze) is not specified, the
namespace device will set the zone size to be the next power of two and
fit in as many zones as possible on the underlying namespace blockdev.
This behavior is not required by the specification, but ensures that the
device can be initialized by the Linux kernel nvme driver, which
requires a power of two zone size.

If the namespace has an associated 'pstate' blockdev it will be used to
store the zone states persistently. Only zone state changes are
persisted, that is, zone write pointer updates are only persistent if
the zone is explicitly closed. On boot up, any zones that were in an
Opened state will be transitioned to Full.

Signed-off-by: Klaus Jensen 
---
 docs/specs/nvme.txt   |   7 ++
 hw/block/nvme-ns.h| 116 +++-
 include/block/nvme.h  |  59 ++-
 hw/block/nvme-ns.c| 184 +++-
 hw/block/nvme.c   | 239 +-
 hw/block/trace-events |   8 ++
 6 files changed, 604 insertions(+), 9 deletions(-)

diff --git a/docs/specs/nvme.txt b/docs/specs/nvme.txt
index a13c7a5dbe86..b23e59dd3075 100644
--- a/docs/specs/nvme.txt
+++ b/docs/specs/nvme.txt
@@ -34,6 +34,13 @@ nvme-ns Options
  values (such as lbads) differs from those specified for the nvme-ns
  device an error is produced.
 
+  `zns.zcap`; If `iocs` is 0x2, this specifies the zone capacity. It is
+ specified in units of logical blocks.
+
+  `zns.zsze`; If `iocs` is 0x2, this specifies the zone size. It is specified
+ in units of the logical blocks. If not specified, the value depends on
+ zns.zcap; if the zone capacity is a power of two, the zone size will be
+ set to that, otherwise it will default to the next power of two.
 
 Reference Specifications
 
diff --git a/hw/block/nvme-ns.h b/hw/block/nvme-ns.h
index d39fa79fa682..e16f50dc4bb8 100644
--- a/hw/block/nvme-ns.h
+++ b/hw/block/nvme-ns.h
@@ -31,15 +31,34 @@ typedef struct NvmePstateHeader {
 uint8_t  lbads;
 uint8_t  iocs;
 
-uint8_t  rsvd18[4078];
+uint8_t  rsvd18[3054];
+
+/* offset 0xc00 */
+struct {
+uint64_t zcap;
+uint64_t zsze;
+} QEMU_PACKED zns;
+
+uint8_t  rsvd3088[1008];
 } QEMU_PACKED NvmePstateHeader;
 
 typedef struct NvmeNamespaceParams {
 uint32_t nsid;
 uint8_t  iocs;
 uint8_t  lbads;
+
+struct {
+uint64_t zcap;
+uint64_t zsze;
+} zns;
 } NvmeNamespaceParams;
 
+typedef struct NvmeZone {
+NvmeZoneDescriptor *zd;
+
+uint64_t wp_staging;
+} NvmeZone;
+
 typedef struct NvmeNamespace {
 DeviceState  parent_obj;
 BlockConfblkconf;
@@ -55,6 +74,10 @@ typedef struct NvmeNamespace {
 unsigned long *map;
 int64_t   offset;
 } utilization;
+
+struct {
+int64_t offset;
+} zns;
 } pstate;
 
 NvmeNamespaceParams params;
@@ -62,8 +85,20 @@ typedef struct NvmeNamespace {
 struct {
 uint32_t err_rec;
 } features;
+
+struct {
+int num_zones;
+
+NvmeZone   *zones;
+NvmeZoneDescriptor *zd;
+} zns;
 } NvmeNamespace;
 
+static inline bool nvme_ns_zoned(NvmeNamespace *ns)
+{
+return ns->iocs == NVME_IOCS_ZONED;
+}
+
 static inline uint32_t nvme_nsid(NvmeNamespace *ns)
 {
 if (ns) {
@@ -78,17 +113,39 @@ static inline NvmeIdNsNvm *nvme_ns_id_nvm(NvmeNamespace 
*ns)
 return ns->id_ns[NVME_IOCS_NVM];
 }
 
+static inline NvmeIdNsZns *nvme_ns_id_zoned(NvmeNamespace *ns)
+{
+return ns->id_ns[NVME_IOCS_ZONED];
+}
+
 static inline NvmeLBAF *nvme_ns_lbaf(NvmeNamespace *ns)
 {
 NvmeIdNsNvm *id_ns = nvme_ns_id_nvm(ns);
 return _ns->lbaf[NVME_ID_NS_FLBAS_INDEX(id_ns->flbas)];
 }
 
+static inline NvmeLBAFE *nvme_ns_lbafe(NvmeNamespace *ns)
+{
+NvmeIdNsNvm *id_ns = nvme_ns_id_nvm(ns);
+NvmeIdNsZns *id_ns_zns = nvme_ns_id_zoned(ns);
+return _ns_zns->lbafe[NVME_ID_NS_FLBAS_INDEX(id_ns->flbas)];
+}
+
 static inline uint8_t nvme_ns_lbads(NvmeNamespace *ns)
 {
 return nvme_ns_lbaf(ns)->ds;
 }
 
+static inline uint64_t nvme_ns_zsze(NvmeNamespace *ns)
+{
+return nvme_ns_lbafe(ns)->zsze;
+}
+
+static inline uint64_t nvme_ns_zsze_bytes(NvmeNamespace *ns)
+{
+return nvme_ns_zsze(ns) << nvme_ns_lbads(ns);
+}
+
 /* calculate the number of LBAs that the namespace can accomodate */
 static inline uint64_t nvme_ns_nlbas(NvmeNamespace *ns)
 {
@@ -101,12 +158,69 @@ static inline size_t nvme_l2b(NvmeNamespace *ns, uint64_t 
lba)
 return lba << nvme_ns_lbads(ns);
 }
 
+static inline int nvme_ns_zone_idx(NvmeNamespace *ns, uint64_t lba)
+{
+return lba / nvme_ns_zsze(ns);
+}
+
+static inline NvmeZone 

[PATCH v2 06/14] hw/block/nvme: add support for dulbe and block utilization tracking

[Klaus Jensen]

From: Klaus Jensen 

This adds support for reporting the Deallocated or Unwritten Logical
Block error (DULBE). This requires tracking the allocated/deallocated
status of all logical blocks.

Introduce a bitmap that does this. The bitmap is persisted on the new
'pstate' blockdev that is associated with a namespace. If no such drive
is attached, the controller will not indicate support for DULBE.

Signed-off-by: Klaus Jensen 
---
 docs/specs/nvme.txt   |  19 +
 hw/block/nvme-ns.h|  32 
 include/block/nvme.h  |   5 ++
 hw/block/nvme-ns.c| 186 ++
 hw/block/nvme.c   | 130 -
 hw/block/trace-events |   2 +
 6 files changed, 371 insertions(+), 3 deletions(-)

diff --git a/docs/specs/nvme.txt b/docs/specs/nvme.txt
index 438ca50d698c..6d00ac064998 100644
--- a/docs/specs/nvme.txt
+++ b/docs/specs/nvme.txt
@@ -12,6 +12,25 @@ nvme-ns Options
  namespace. It is specified in terms of a power of two. Only values between
  9 and 12 (both inclusive) are supported.
 
+  `pstate`; This parameter specifies another blockdev to be used for storing
+ persistent state such as logical block allocation tracking. Adding this
+ parameter enables various optional features of the device.
+
+   -drive id=pstate,file=pstate.img,format=raw
+   -device nvme-ns,pstate=pstate,...
+
+ To reset (or initialize) state, the blockdev image should be of zero size:
+
+   qemu-img create -f raw pstate.img 0
+
+ The image will be intialized with a file format header and truncated to
+ the required size.
+
+ If the pstate given is of non-zero size, it will be assumed to contain
+ previous saved state and will be checked for consistency. If any stored
+ values (such as lbads) differs from those specified for the nvme-ns
+ device an error is produced.
+
 
 Reference Specifications
 
diff --git a/hw/block/nvme-ns.h b/hw/block/nvme-ns.h
index 78b0d1a00672..0ad83910dde9 100644
--- a/hw/block/nvme-ns.h
+++ b/hw/block/nvme-ns.h
@@ -19,6 +19,20 @@
 #define NVME_NS(obj) \
 OBJECT_CHECK(NvmeNamespace, (obj), TYPE_NVME_NS)
 
+#define NVME_PSTATE_MAGIC ((0x00 << 24) | ('S' << 16) | ('P' << 8) | 'N')
+#define NVME_PSTATE_V1 1
+
+typedef struct NvmePstateHeader {
+uint32_t magic;
+uint32_t version;
+
+int64_t  blk_len;
+
+uint8_t  lbads;
+
+uint8_t  rsvd17[4079];
+} QEMU_PACKED NvmePstateHeader;
+
 typedef struct NvmeNamespaceParams {
 uint32_t nsid;
 uint8_t  lbads;
@@ -31,7 +45,20 @@ typedef struct NvmeNamespace {
 int64_t  size;
 NvmeIdNs id_ns;
 
+struct {
+BlockBackend *blk;
+
+struct {
+unsigned long *map;
+int64_t   offset;
+} utilization;
+} pstate;
+
 NvmeNamespaceParams params;
+
+struct {
+uint32_t err_rec;
+} features;
 } NvmeNamespace;
 
 static inline uint32_t nvme_nsid(NvmeNamespace *ns)
@@ -72,4 +99,9 @@ int nvme_ns_setup(NvmeCtrl *n, NvmeNamespace *ns, Error 
**errp);
 void nvme_ns_drain(NvmeNamespace *ns);
 void nvme_ns_flush(NvmeNamespace *ns);
 
+static inline void _nvme_ns_check_size(void)
+{
+QEMU_BUILD_BUG_ON(sizeof(NvmePstateHeader) != 4096);
+}
+
 #endif /* NVME_NS_H */
diff --git a/include/block/nvme.h b/include/block/nvme.h
index 999b4f8ae0d4..abd49d371e63 100644
--- a/include/block/nvme.h
+++ b/include/block/nvme.h
@@ -683,6 +683,7 @@ enum NvmeStatusCodes {
 NVME_E2E_REF_ERROR  = 0x0284,
 NVME_CMP_FAILURE= 0x0285,
 NVME_ACCESS_DENIED  = 0x0286,
+NVME_DULB   = 0x0287,
 NVME_MORE   = 0x2000,
 NVME_DNR= 0x4000,
 NVME_NO_COMPLETE= 0x,
@@ -898,6 +899,9 @@ enum NvmeIdCtrlLpa {
 #define NVME_AEC_NS_ATTR(aec)   ((aec >> 8) & 0x1)
 #define NVME_AEC_FW_ACTIVATION(aec) ((aec >> 9) & 0x1)
 
+#define NVME_ERR_REC_TLER(err_rec)  (err_rec & 0x)
+#define NVME_ERR_REC_DULBE(err_rec) (err_rec & 0x1)
+
 enum NvmeFeatureIds {
 NVME_ARBITRATION= 0x1,
 NVME_POWER_MANAGEMENT   = 0x2,
@@ -1018,6 +1022,7 @@ enum NvmeNsIdentifierType {
 
 
 #define NVME_ID_NS_NSFEAT_THIN(nsfeat)  ((nsfeat & 0x1))
+#define NVME_ID_NS_NSFEAT_DULBE(nsfeat) ((nsfeat >> 2) & 0x1)
 #define NVME_ID_NS_FLBAS_EXTENDED(flbas)((flbas >> 4) & 0x1)
 #define NVME_ID_NS_FLBAS_INDEX(flbas)   ((flbas & 0xf))
 #define NVME_ID_NS_MC_SEPARATE(mc)  ((mc >> 1) & 0x1)
diff --git a/hw/block/nvme-ns.c b/hw/block/nvme-ns.c
index 576c7486f45b..5e24b1a5dacd 100644
--- a/hw/block/nvme-ns.c
+++ b/hw/block/nvme-ns.c
@@ -25,9 +25,36 @@
 #include "hw/qdev-properties.h"
 #include "hw/qdev-core.h"
 
+#include "trace.h"
+
 #include "nvme.h"
 #include "nvme-ns.h"
 
+static int nvme_blk_truncate(BlockBackend *blk, size_t len, Error **errp)
+{
+int ret;
+uint64_t perm, shared_perm;
+
+blk_get_perm(blk, , _perm);
+
+

[PATCH v2 02/14] hw/block/nvme: add trace event for requests with non-zero status code

[Klaus Jensen]

From: Klaus Jensen 

If a command results in a non-zero status code, trace it.

Signed-off-by: Klaus Jensen 
---
 hw/block/nvme.c   | 6 ++
 hw/block/trace-events | 1 +
 2 files changed, 7 insertions(+)

diff --git a/hw/block/nvme.c b/hw/block/nvme.c
index 84b6b516fa7b..3cbc3c7b75b1 100644
--- a/hw/block/nvme.c
+++ b/hw/block/nvme.c
@@ -777,6 +777,12 @@ static void nvme_enqueue_req_completion(NvmeCQueue *cq, 
NvmeRequest *req)
 assert(cq->cqid == req->sq->cqid);
 trace_pci_nvme_enqueue_req_completion(nvme_cid(req), cq->cqid,
   req->status);
+
+if (req->status) {
+trace_pci_nvme_err_req_status(nvme_cid(req), nvme_nsid(req->ns),
+  req->status, req->cmd.opcode);
+}
+
 QTAILQ_REMOVE(>sq->out_req_list, req, entry);
 QTAILQ_INSERT_TAIL(>req_list, req, entry);
 timer_mod(cq->timer, qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) + 500);
diff --git a/hw/block/trace-events b/hw/block/trace-events
index 5a239b80bf36..9e7507c5abde 100644
--- a/hw/block/trace-events
+++ b/hw/block/trace-events
@@ -89,6 +89,7 @@ pci_nvme_mmio_shutdown_cleared(void) "shutdown bit cleared"
 
 # nvme traces for error conditions
 pci_nvme_err_mdts(uint16_t cid, size_t len) "cid %"PRIu16" len %zu"
+pci_nvme_err_req_status(uint16_t cid, uint32_t nsid, uint16_t status, uint8_t 
opc) "cid %"PRIu16" nsid %"PRIu32" status 0x%"PRIx16" opc 0x%"PRIx8""
 pci_nvme_err_addr_read(uint64_t addr) "addr 0x%"PRIx64""
 pci_nvme_err_addr_write(uint64_t addr) "addr 0x%"PRIx64""
 pci_nvme_err_cfs(void) "controller fatal status"
-- 
2.28.0

[PATCH v2 01/14] hw/block/nvme: add nsid to get/setfeat trace events

[Klaus Jensen]

From: Klaus Jensen 

Include the namespace id in the pci_nvme_{get,set}feat trace events.

Signed-off-by: Klaus Jensen 
---
 hw/block/nvme.c   | 4 ++--
 hw/block/trace-events | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/hw/block/nvme.c b/hw/block/nvme.c
index da8344f196a8..84b6b516fa7b 100644
--- a/hw/block/nvme.c
+++ b/hw/block/nvme.c
@@ -1660,7 +1660,7 @@ static uint16_t nvme_get_feature(NvmeCtrl *n, NvmeRequest 
*req)
 [NVME_ARBITRATION] = NVME_ARB_AB_NOLIMIT,
 };
 
-trace_pci_nvme_getfeat(nvme_cid(req), fid, sel, dw11);
+trace_pci_nvme_getfeat(nvme_cid(req), nsid, fid, sel, dw11);
 
 if (!nvme_feature_support[fid]) {
 return NVME_INVALID_FIELD | NVME_DNR;
@@ -1798,7 +1798,7 @@ static uint16_t nvme_set_feature(NvmeCtrl *n, NvmeRequest 
*req)
 uint8_t fid = NVME_GETSETFEAT_FID(dw10);
 uint8_t save = NVME_SETFEAT_SAVE(dw10);
 
-trace_pci_nvme_setfeat(nvme_cid(req), fid, save, dw11);
+trace_pci_nvme_setfeat(nvme_cid(req), nsid, fid, save, dw11);
 
 if (save) {
 return NVME_FID_NOT_SAVEABLE | NVME_DNR;
diff --git a/hw/block/trace-events b/hw/block/trace-events
index 446cca08e9ab..5a239b80bf36 100644
--- a/hw/block/trace-events
+++ b/hw/block/trace-events
@@ -53,8 +53,8 @@ pci_nvme_identify_ns(uint32_t ns) "nsid %"PRIu32""
 pci_nvme_identify_nslist(uint32_t ns) "nsid %"PRIu32""
 pci_nvme_identify_ns_descr_list(uint32_t ns) "nsid %"PRIu32""
 pci_nvme_get_log(uint16_t cid, uint8_t lid, uint8_t lsp, uint8_t rae, uint32_t 
len, uint64_t off) "cid %"PRIu16" lid 0x%"PRIx8" lsp 0x%"PRIx8" rae 0x%"PRIx8" 
len %"PRIu32" off %"PRIu64""
-pci_nvme_getfeat(uint16_t cid, uint8_t fid, uint8_t sel, uint32_t cdw11) "cid 
%"PRIu16" fid 0x%"PRIx8" sel 0x%"PRIx8" cdw11 0x%"PRIx32""
-pci_nvme_setfeat(uint16_t cid, uint8_t fid, uint8_t save, uint32_t cdw11) "cid 
%"PRIu16" fid 0x%"PRIx8" save 0x%"PRIx8" cdw11 0x%"PRIx32""
+pci_nvme_getfeat(uint16_t cid, uint32_t nsid, uint8_t fid, uint8_t sel, 
uint32_t cdw11) "cid %"PRIu16" nsid 0x%"PRIx32" fid 0x%"PRIx8" sel 0x%"PRIx8" 
cdw11 0x%"PRIx32""
+pci_nvme_setfeat(uint16_t cid, uint32_t nsid, uint8_t fid, uint8_t save, 
uint32_t cdw11) "cid %"PRIu16" nsid 0x%"PRIx32" fid 0x%"PRIx8" save 0x%"PRIx8" 
cdw11 0x%"PRIx32""
 pci_nvme_getfeat_vwcache(const char* result) "get feature volatile write 
cache, result=%s"
 pci_nvme_getfeat_numq(int result) "get feature number of queues, result=%d"
 pci_nvme_setfeat_numq(int reqcq, int reqsq, int gotcq, int gotsq) "requested 
cq_count=%d sq_count=%d, responding with cq_count=%d sq_count=%d"
-- 
2.28.0

[PATCH v2 03/14] hw/block/nvme: make lba data size configurable

[Klaus Jensen]

From: Klaus Jensen 

Allos the LBA data size (lbads) to be set between 9 and 12.

Signed-off-by: Klaus Jensen 
Acked-by: Keith Busch 
Reviewed-by: Maxim Levitsky 
Reviewed-by: Philippe Mathieu-Daudé 
---
 docs/specs/nvme.txt | 11 ++-
 hw/block/nvme-ns.h  |  1 +
 hw/block/nvme-ns.c  |  8 +++-
 hw/block/nvme.c |  1 +
 4 files changed, 19 insertions(+), 2 deletions(-)

diff --git a/docs/specs/nvme.txt b/docs/specs/nvme.txt
index 56d393884e7a..438ca50d698c 100644
--- a/docs/specs/nvme.txt
+++ b/docs/specs/nvme.txt
@@ -1,7 +1,16 @@
 NVM Express Controller
 ==
 
-The nvme device (-device nvme) emulates an NVM Express Controller.
+The nvme device (-device nvme) emulates an NVM Express Controller. It is used
+together with nvme-ns devices (-device nvme-ns) which emulates an NVM Express
+Namespace.
+
+nvme-ns Options
+---
+
+  `lbads`; The "LBA Data Size (LBADS)" indicates the LBA data size used by the
+ namespace. It is specified in terms of a power of two. Only values between
+ 9 and 12 (both inclusive) are supported.
 
 
 Reference Specifications
diff --git a/hw/block/nvme-ns.h b/hw/block/nvme-ns.h
index 83734f4606e1..78b0d1a00672 100644
--- a/hw/block/nvme-ns.h
+++ b/hw/block/nvme-ns.h
@@ -21,6 +21,7 @@
 
 typedef struct NvmeNamespaceParams {
 uint32_t nsid;
+uint8_t  lbads;
 } NvmeNamespaceParams;
 
 typedef struct NvmeNamespace {
diff --git a/hw/block/nvme-ns.c b/hw/block/nvme-ns.c
index 2ba0263ddaca..576c7486f45b 100644
--- a/hw/block/nvme-ns.c
+++ b/hw/block/nvme-ns.c
@@ -36,7 +36,7 @@ static void nvme_ns_init(NvmeNamespace *ns)
 ns->id_ns.dlfeat = 0x9;
 }
 
-id_ns->lbaf[0].ds = BDRV_SECTOR_BITS;
+id_ns->lbaf[0].ds = ns->params.lbads;
 
 id_ns->nsze = cpu_to_le64(nvme_ns_nlbas(ns));
 
@@ -77,6 +77,11 @@ static int nvme_ns_check_constraints(NvmeNamespace *ns, 
Error **errp)
 return -1;
 }
 
+if (ns->params.lbads < 9 || ns->params.lbads > 12) {
+error_setg(errp, "unsupported lbads (supported: 9-12)");
+return -1;
+}
+
 return 0;
 }
 
@@ -125,6 +130,7 @@ static void nvme_ns_realize(DeviceState *dev, Error **errp)
 static Property nvme_ns_props[] = {
 DEFINE_BLOCK_PROPERTIES(NvmeNamespace, blkconf),
 DEFINE_PROP_UINT32("nsid", NvmeNamespace, params.nsid, 0),
+DEFINE_PROP_UINT8("lbads", NvmeNamespace, params.lbads, BDRV_SECTOR_BITS),
 DEFINE_PROP_END_OF_LIST(),
 };
 
diff --git a/hw/block/nvme.c b/hw/block/nvme.c
index 3cbc3c7b75b1..758f58c88026 100644
--- a/hw/block/nvme.c
+++ b/hw/block/nvme.c
@@ -2812,6 +2812,7 @@ static void nvme_realize(PCIDevice *pci_dev, Error **errp)
 if (n->namespace.blkconf.blk) {
 ns = >namespace;
 ns->params.nsid = 1;
+ns->params.lbads = BDRV_SECTOR_BITS;
 
 if (nvme_ns_setup(n, ns, errp)) {
 return;
-- 
2.28.0

[PATCH v2 00/14] hw/block/nvme: zoned namespace command set

[Klaus Jensen]

From: Klaus Jensen 

So, things escalated a bit.

I'm removing the old cover letter. I don't think we need it anymore - you all
know the reason I keep posting versions of this implementation ;)

Based-on: <20200922084533.1273962-1-...@irrelevant.dk>

Changes for v2
~~

  * "hw/block/nvme: add support for dulbe and block utilization tracking"
- Factor out pstate init/load into separate functions.

- Fixed a stupid off-by-1 bug that would trigger when resetting the
  last zone.

- I added a more formalized pstate file format that includes a
  header. This is pretty similar to what is done in Dmitry's series,
  but with fewer fields. The device parameters for nvme-ns are still
  the "authoritative" ones, so if any parameters that influence LBA
  size, number of zones, etc. do not match, an error indicating the
  discrepancy will be produced. IIRC, Dmitry's version does the
  same.

  It is set up such that newer versions can load pstate files from
  older versions. The file format header is not unlike a standard
  nvme datastructure with reserved areas. This means that when
  adding new command sets that require persistent state, it is not
  needed to bump the version number, unless the header has to change
  dramatically.  This is demonstrated when the zoned namespace
  command set support is added in "hw/block/nvme: add basic
  read/write for zoned namespaces".

  * "hw/block/nvme: add basic read/write for zoned namespaces"
- The device will now transition all opened zones to Closed on
  "normal shutdown". You can force the "transition to Full" behavior
  by killing QEMU from the monitor.

  * "hw/block/nvme: add the zone append command"
- Slightly reordered the logic so a LBA Out of Range error is
  returned before Invalid Field in Command for normal read/write
  commands.

  * "hw/block/nvme: support zone active excursions"
- Dropped. Optional and non-critical.

  * "hw/block/nvme: support reset/finish recommended limits"
- Dropped. Optional and non-critical.

Gollu Appalanaidu (1):
  hw/block/nvme: add commands supported and effects log page

Klaus Jensen (13):
  hw/block/nvme: add nsid to get/setfeat trace events
  hw/block/nvme: add trace event for requests with non-zero status code
  hw/block/nvme: make lba data size configurable
  hw/block/nvme: reject io commands if only admin command set selected
  hw/block/nvme: consolidate read, write and write zeroes
  hw/block/nvme: add support for dulbe and block utilization tracking
  hw/block/nvme: support namespace types
  hw/block/nvme: add basic read/write for zoned namespaces
  hw/block/nvme: add the zone management receive command
  hw/block/nvme: add the zone management send command
  hw/block/nvme: add the zone append command
  hw/block/nvme: track and enforce zone resources
  hw/block/nvme: allow open to close transitions by controller

 docs/specs/nvme.txt   |   47 +-
 hw/block/nvme-ns.h|  180 -
 hw/block/nvme.h   |   22 +
 include/block/nvme.h  |  230 +-
 block/nvme.c  |4 +-
 hw/block/nvme-ns.c|  456 +++-
 hw/block/nvme.c   | 1560 +++--
 hw/block/trace-events |   42 +-
 8 files changed, 2443 insertions(+), 98 deletions(-)

-- 
2.28.0

[Bug 1897481] Re: qemu crashes with VGA pass-through, e-GPU, nvidia 1060

[Alex Williamson]

Please attach output from `dmesg` and `sudo lspci -vvv`, both from the
host.  Laptops typically don't provide sufficient resources for GPUs
attached like this, so my guess is that we're trying to add a quirk on
top of a BAR that isn't mapped.  If that's the case, the following host
kernel options might help: pci=realloc,assign-busses,nocrs

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1897481

Title:
  qemu crashes with VGA pass-through, e-GPU, nvidia 1060

Status in QEMU:
  New

Bug description:
  I try to pass-through nvidia 1060 6gb card, which is connected via
  ExpressCard (EXP-GDC converter).

  I can successfully run my virtual machine without pass-through, but
  when I try to add the devices, qemu crashes.

  The coredump contains:

  Stack trace of thread 3289311:
  #0  0x00614c49 memory_region_update_container_subregions 
(qemu-system-x86_64 + 0x214c49)
  #1  0x005c0e8c vfio_probe_nvidia_bar0_quirk (qemu-system-x86_64 + 
0x1c0e8c)
  #2  0x005bcec0 vfio_realize (qemu-system-x86_64 + 0x1bcec0)
  #3  0x0079b423 pci_qdev_realize (qemu-system-x86_64 + 0x39b423)
  #4  0x006facda device_set_realized (qemu-system-x86_64 + 0x2facda)
  #5  0x00887e57 property_set_bool (qemu-system-x86_64 + 0x487e57)
  #6  0x0088ac48 object_property_set (qemu-system-x86_64 + 0x48ac48)
  #7  0x0088d1d2 object_property_set_qobject (qemu-system-x86_64 + 
0x48d1d2)
  #8  0x0088b1f7 object_property_set_bool (qemu-system-x86_64 + 
0x48b1f7)
  #9  0x00693785 qdev_device_add (qemu-system-x86_64 + 0x293785)
  #10 0x0061aad0 device_init_func (qemu-system-x86_64 + 0x21aad0)
  #11 0x0098c87b qemu_opts_foreach (qemu-system-x86_64 + 0x58c87b)
  #12 0x006211cb qemu_init (qemu-system-x86_64 + 0x2211cb)
  #13 0x005002aa main (qemu-system-x86_64 + 0x1002aa)
  #14 0x7fce8af21152 __libc_start_main (libc.so.6 + 0x28152)
  #15 0x0050087e _start (qemu-system-x86_64 + 0x10087e)

  The whole running command is pretty long, since I use libvirt to
  manage my machines:

  LC_ALL=C \
  PATH=/usr/local/sbin:/usr/local/bin:/usr/bin \
  HOME=/var/lib/libvirt/qemu/domain-2-Win10 \
  XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-2-Win10/.local/share \
  XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-2-Win10/.cache \
  XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-2-Win10/.config \
  QEMU_AUDIO_DRV=spice \
  /usr/bin/qemu-system-x86_64 \
  -name guest=Win10,debug-threads=on \
  -S \
  -blockdev 
'{"driver":"file","filename":"/usr/share/edk2-ovmf/x64/OVMF_CODE.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}'
 \
  -blockdev 
'{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}'
 \
  -blockdev 
'{"driver":"file","filename":"/var/lib/libvirt/qemu/nvram/Win10_VARS.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}'
 \
  -blockdev 
'{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}'
 \
  -machine 
pc-q35-5.1,accel=kvm,usb=off,vmport=off,dump-guest-core=off,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format
 \
  -cpu host,migratable=on,hv-time,hv-relaxed,hv-vapic,hv-spinlocks=0x1fff \
  -m 8192 \
  -overcommit mem-lock=off \
  -smp 2,sockets=2,cores=1,threads=1 \
  -uuid 7043c77b-4903-4527-8089-9679d9a17fee \
  -no-user-config \
  -nodefaults \
  -chardev stdio,mux=on,id=charmonitor \
  -mon chardev=charmonitor,id=monitor,mode=control \
  -rtc base=localtime,driftfix=slew \
  -global kvm-pit.lost_tick_policy=delay \
  -no-hpet \
  -no-shutdown \
  -global ICH9-LPC.disable_s3=1 \
  -global ICH9-LPC.disable_s4=1 \
  -boot strict=on \
  -device 
pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2
 \
  -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 \
  -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 \
  -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 \
  -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 \
  -device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 \
  -device qemu-xhci,p2=15,p3=15,id=usb,bus=pci.2,addr=0x0 \
  -device virtio-serial-pci,id=virtio-serial0,bus=pci.3,addr=0x0 \
  -blockdev '{"driver":"file","filename":"/home/sergiy/VirtualBox 
VMs/win4games.img","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}'
 \
  -blockdev 
'{"node-name":"libvirt-2-format","read-only":false,"driver":"raw","file":"libvirt-2-storage"}'
 \
  -device ide-hd,bus=ide.0,drive=libvirt-2-format,id=sata0-0-0,bootindex=1 \
  -blockdev 
'{"driver":"file","filename":"/home/sergiy/Downloads/Win10_2004_Ukrainian_x64.iso","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}'
 \
  -blockdev 

Re: [PATCH v3 07/18] hw/block/nvme: add support for the get log page command

[Keith Busch]

On Wed, Sep 30, 2020 at 12:42:48AM +0200, Klaus Jensen wrote:
> On Sep 29 15:34, Keith Busch wrote:
> > Yeah, looks safe as-is, but we're missing out on returning the spec
> > required 'Invalid Field'.
> 
> I can't see where it says that we should do that? Invalid Field in
> Command if offset is *greater* than the size of the log page.
> 
> Some dynamic log pages have side-effects of being read, so while this is
> a super wierd way of specifying that we want nothing returned, I think
> it is valid?

Eh, when spec says "size of the log page", I assume they're using the
"zeroes based" definition for size as aligned with the NUMD field. So
512 is bigger than the sizeof the smart log occupying bytes 0-511.

But I guess there's room to see it the other way, so maybe it is a
way to request a no data log.

[PATCH v4 12/12] .travis.yml: Add a KVM-only Aarch64 job

[Philippe Mathieu-Daudé]

Add a job to build QEMU on Aarch64 with TCG disabled, so
this configuration won't bitrot over time.

We explicitly modify default-configs/aarch64-softmmu.mak to
only select the 'virt' and 'SBSA-REF' machines.

Signed-off-by: Philippe Mathieu-Daudé 
---
Job ran for 7 min 30 sec
https://travis-ci.org/github/philmd/qemu/jobs/731428859
---
 .travis.yml | 35 +++
 1 file changed, 35 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index c75221dca3..cad65cf181 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -402,6 +402,41 @@ jobs:
 - CONFIG="--disable-containers --target-list=${MAIN_SOFTMMU_TARGETS}"
 - UNRELIABLE=true
 
+- name: "[aarch64] GCC (disable-tcg)"
+  arch: arm64
+  dist: bionic
+  addons:
+apt_packages:
+  - libaio-dev
+  - libattr1-dev
+  - libcap-ng-dev
+  - libgcrypt20-dev
+  - libgnutls28-dev
+  - libiscsi-dev
+  - liblttng-ust-dev
+  - libnfs-dev
+  - libnss3-dev
+  - libpixman-1-dev
+  - libpng-dev
+  - librados-dev
+  - libseccomp-dev
+  - liburcu-dev
+  - libusb-1.0-0-dev
+  - libvdeplug-dev
+  - libvte-2.91-dev
+  # Tests dependencies
+  - genisoimage
+  env:
+- CONFIG="--disable-containers --disable-tcg --enable-kvm 
--disable-tools"
+- TEST_CMD="make check-unit"
+- CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-aarch64"
+  before_script:
+# Only use the 'virt' and 'sbsa-ref' machine which don't need TCG.
+- echo CONFIG_ARM_VIRT=y > default-configs/aarch64-softmmu.mak
+- echo CONFIG_SBSA_REF=y >> default-configs/aarch64-softmmu.mak
+- mkdir -p ${BUILD_DIR} && cd ${BUILD_DIR}
+- ${SRC_DIR}/configure ${BASE_CONFIG} ${CONFIG} || { cat config.log && 
exit 1; }
+
 - name: "[ppc64] GCC check-tcg"
   arch: ppc64le
   dist: xenial
-- 
2.26.2

[PATCH v4 10/12] target/arm: Do not build TCG objects when TCG is off

[Philippe Mathieu-Daudé]

From: Samuel Ortiz 

We can now safely turn all TCG dependent build off when CONFIG_TCG is
off. This allows building ARM binaries with --disable-tcg.

Signed-off-by: Samuel Ortiz 
[PMD: Heavily rebased during almost 2 years then finally rewritten =) ]
Signed-off-by: Philippe Mathieu-Daudé 
---
 target/arm/meson.build | 24 +---
 1 file changed, 17 insertions(+), 7 deletions(-)

diff --git a/target/arm/meson.build b/target/arm/meson.build
index f6a88297a8..9b7727d4bb 100644
--- a/target/arm/meson.build
+++ b/target/arm/meson.build
@@ -16,24 +16,29 @@ arm_ss = ss.source_set()
 arm_ss.add(gen)
 arm_ss.add(files(
   'cpu.c',
-  'crypto_helper.c',
-  'debug_helper.c',
   'gdbstub.c',
   'helper.c',
+  'vfp_helper.c',
+))
+
+arm_tcg_ss = ss.source_set()
+arm_tcg_ss.add(files(
+  'arm-semi.c',
+  'cpu_tcg.c',
+  'crypto_helper.c',
+  'debug_helper.c',
   'iwmmxt_helper.c',
   'neon_helper.c',
   'op_helper.c',
   'tlb_helper.c',
   'translate.c',
   'vec_helper.c',
-  'vfp_helper.c',
-  'cpu_tcg.c',
 ))
-arm_ss.add(when: 'CONFIG_ARM_V7M', if_true: files('m_helper.c'), if_false: 
files('m_helper-stub.c'))
+
+arm_tcg_ss.add(when: 'CONFIG_ARM_V7M', if_true: files('m_helper.c'), if_false: 
files('m_helper-stub.c'))
 
 arm_ss.add(zlib)
 
-arm_ss.add(when: 'CONFIG_TCG', if_true: files('arm-semi.c'))
 arm_ss.add(when: 'CONFIG_TCG', if_false: files('m_helper-stub.c'))
 
 arm_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c', 'kvm64.c'), if_false: 
files('kvm-stub.c'))
@@ -41,6 +46,9 @@ arm_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c', 
'kvm64.c'), if_false: fil
 arm_ss.add(when: 'TARGET_AARCH64', if_true: files(
   'cpu64.c',
   'gdbstub64.c',
+))
+
+arm_tcg_ss.add(when: 'TARGET_AARCH64', if_true: files(
   'helper-a64.c',
   'mte_helper.c',
   'pauth_helper.c',
@@ -49,14 +57,16 @@ arm_ss.add(when: 'TARGET_AARCH64', if_true: files(
   'translate-sve.c',
 ))
 
+arm_ss.add_all(when: 'CONFIG_TCG', if_true: arm_tcg_ss)
+
 arm_softmmu_ss = ss.source_set()
 arm_softmmu_ss.add(files(
   'arch_dump.c',
   'arm-powerctl.c',
   'machine.c',
   'monitor.c',
-  'psci.c',
 ))
+arm_softmmu_ss.add(when: 'CONFIG_TCG', if_true: files('psci.c'))
 
 target_arch += {'arm': arm_ss}
 target_softmmu_arch += {'arm': arm_softmmu_ss}
-- 
2.26.2

[PATCH] tests/acceptance: Add a 'virt_kvm' test using the GICv3

[Philippe Mathieu-Daudé]

The current 'virt_kvm' test is restricted to GICv2, but can also
work with a GICv3. Duplicate it but add a GICv3 test which can be
tested on some hardware.

Noticed while running:

 $ avocado --show=app run -t machine:virt tests/acceptance/
 ...
 (2/6) tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_kvm: ERROR: 
Unexpected empty reply from server (1.82 s)

The job.log content is:

  L0351 DEBUG| Output: 'qemu-system-aarch64: host does not support in-kernel 
GICv2 emulation\n'

With this patch:

 $ avocado --show=app run -t device:gicv3 tests/acceptance/
 (1/1) tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_kvm_gicv3: 
PASS (55.10 s)

Signed-off-by: Philippe Mathieu-Daudé 
---
 tests/acceptance/boot_linux.py | 17 -
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/tests/acceptance/boot_linux.py b/tests/acceptance/boot_linux.py
index 0055dc7cee..c743e231f4 100644
--- a/tests/acceptance/boot_linux.py
+++ b/tests/acceptance/boot_linux.py
@@ -182,10 +182,11 @@ def test_virt_tcg(self):
 self.add_common_args()
 self.launch_and_wait()
 
-def test_virt_kvm(self):
+def test_virt_kvm_gicv2(self):
 """
 :avocado: tags=accel:kvm
 :avocado: tags=cpu:host
+:avocado: tags=device:gicv2
 """
 if not kvm_available(self.arch, self.qemu_bin):
 self.cancel(KVM_NOT_AVAILABLE)
@@ -195,6 +196,20 @@ def test_virt_kvm(self):
 self.add_common_args()
 self.launch_and_wait()
 
+def test_virt_kvm_gicv3(self):
+"""
+:avocado: tags=accel:kvm
+:avocado: tags=cpu:host
+:avocado: tags=device:gicv3
+"""
+if not kvm_available(self.arch, self.qemu_bin):
+self.cancel(KVM_NOT_AVAILABLE)
+self.vm.add_args("-accel", "kvm")
+self.vm.add_args("-cpu", "host")
+self.vm.add_args("-machine", "virt,gic-version=3")
+self.add_common_args()
+self.launch_and_wait()
+
 
 class BootLinuxPPC64(BootLinux):
 """
-- 
2.26.2

[PATCH v4 11/12] target/arm: Reorder meson.build rules

[Philippe Mathieu-Daudé]

Reorder the rules to make this file easier to modify.

Signed-off-by: Philippe Mathieu-Daudé 
---
I prefer to not squash that with the previous patch,
as it makes it harder to review.
---
 target/arm/meson.build | 22 ++
 1 file changed, 10 insertions(+), 12 deletions(-)

diff --git a/target/arm/meson.build b/target/arm/meson.build
index 9b7727d4bb..341af8f2ca 100644
--- a/target/arm/meson.build
+++ b/target/arm/meson.build
@@ -14,6 +14,7 @@ gen = [
 
 arm_ss = ss.source_set()
 arm_ss.add(gen)
+arm_ss.add(zlib)
 arm_ss.add(files(
   'cpu.c',
   'gdbstub.c',
@@ -21,6 +22,13 @@ arm_ss.add(files(
   'vfp_helper.c',
 ))
 
+arm_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c', 'kvm64.c'), if_false: 
files('kvm-stub.c'))
+
+arm_ss.add(when: 'TARGET_AARCH64', if_true: files(
+  'cpu64.c',
+  'gdbstub64.c',
+))
+
 arm_tcg_ss = ss.source_set()
 arm_tcg_ss.add(files(
   'arm-semi.c',
@@ -35,26 +43,16 @@ arm_tcg_ss.add(files(
   'vec_helper.c',
 ))
 
-arm_tcg_ss.add(when: 'CONFIG_ARM_V7M', if_true: files('m_helper.c'), if_false: 
files('m_helper-stub.c'))
-
-arm_ss.add(zlib)
-
 arm_ss.add(when: 'CONFIG_TCG', if_false: files('m_helper-stub.c'))
-
-arm_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c', 'kvm64.c'), if_false: 
files('kvm-stub.c'))
-
-arm_ss.add(when: 'TARGET_AARCH64', if_true: files(
-  'cpu64.c',
-  'gdbstub64.c',
-))
+arm_tcg_ss.add(when: 'CONFIG_ARM_V7M', if_true: files('m_helper.c'), if_false: 
files('m_helper-stub.c'))
 
 arm_tcg_ss.add(when: 'TARGET_AARCH64', if_true: files(
   'helper-a64.c',
   'mte_helper.c',
   'pauth_helper.c',
-  'sve_helper.c',
   'translate-a64.c',
   'translate-sve.c',
+  'sve_helper.c',
 ))
 
 arm_ss.add_all(when: 'CONFIG_TCG', if_true: arm_tcg_ss)
-- 
2.26.2

[PATCH v4 06/12] target/arm: Restrict ARMv6 cpus to TCG accel

[Philippe Mathieu-Daudé]

KVM requires a cpu based on (at least) the ARMv7 architecture.

Only enable the following ARMv6 CPUs when TCG is available:

  - ARM1136
  - ARM1176
  - ARM11MPCore
  - Cortex-M0

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/arm/Kconfig | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/hw/arm/Kconfig b/hw/arm/Kconfig
index d2876b2c8b..e01eb55bc0 100644
--- a/hw/arm/Kconfig
+++ b/hw/arm/Kconfig
@@ -6,6 +6,10 @@ config ARM_V5
 bool
 select TCG
 
+config ARM_V6
+bool
+select TCG
+
 config ARM_VIRT
 bool
 imply PCI_DEVICES
@@ -123,6 +127,7 @@ config NETDUINOPLUS2
 
 config NSERIES
 bool
+select ARM_V6
 select OMAP
 select TMP105   # tempature sensor
 select BLIZZARD # LCD/TV controller
@@ -391,6 +396,7 @@ config FSL_IMX25
 
 config FSL_IMX31
 bool
+select ARM_V6
 select SERIAL
 select IMX
 select IMX_I2C
-- 
2.26.2