Re: [PATCH 4/5] docs/system/target-sparc: Improve the Sparc documentation

2024-04-18 Thread Mark Cave-Ayland 

On 19/04/2024 05:59, Thomas Huth wrote:


On 18/04/2024 22.27, Mark Cave-Ayland wrote:

On 07/03/2024 17:43, Thomas Huth wrote:


Add some words about how to enable or disable boolean features,
and remove the note about a Linux kernel being available on the
QEMU website (they have been removed long ago already).

Signed-off-by: Thomas Huth 
---
  docs/system/target-sparc.rst | 8 ++--
  1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/docs/system/target-sparc.rst b/docs/system/target-sparc.rst
index 9ec8c90c14..9f418b9d3e 100644
--- a/docs/system/target-sparc.rst
+++ b/docs/system/target-sparc.rst
@@ -27,6 +27,11 @@ architecture machines:
  The emulation is somewhat complete. SMP up to 16 CPUs is supported, but
  Linux limits the number of usable CPUs to 4.
+The list of available CPUs can be viewed by starting QEMU with ``-cpu help``.
+Optional boolean features can be added with a "+" in front of the feature name,
+or disabled with a "-" in front of the name, for example
+``-cpu TI-SuperSparc-II,+float128``.
+
  QEMU emulates the following sun4m peripherals:
  -  IOMMU
@@ -55,8 +60,7 @@ OpenBIOS is a free (GPL v2) portable firmware implementation. 
The goal

  is to implement a 100% IEEE 1275-1994 (referred to as Open Firmware)
  compliant firmware.
-A sample Linux 2.6 series kernel and ram disk image are available on the
-QEMU web site. There are still issues with NetBSD and OpenBSD, but most
+There are still issues with NetBSD and OpenBSD, but most
  kernel versions work. Please note that currently older Solaris kernels
  don't work probably due to interface issues between OpenBIOS and
  Solaris.


Just curious as to what current issues exist with NetBSD and OpenBSD? At least both 
my NetBSD and OpenBSD test images survive a casual boot test here with latest git.


Maybe it's also about a long fixed bug ... shall I remove that sentence while 
I'm at it?


Yeah, I don't believe that has been true about NetBSD/OpenBSD for quite some time :) 
Please do keep the wording about Solaris since I get asked that often (and it's a 
task that will be quite difficult and time-consuming to do).



ATB,

Mark.

Re: Deprecation/removal of nios2 target support

2024-04-18 Thread Marek Vasut 

On 4/18/24 8:41 PM, Arnd Bergmann wrote:

On Thu, Apr 18, 2024, at 17:44, Joseph Myers wrote:

On Wed, 17 Apr 2024, Sandra Loosemore wrote:


Therefore I'd like to mark Nios II as obsolete in GCC 14 now, and remove
support from all toolchain components after the release is made.  I'm not sure
there is an established process for obsoleting/removing support in other
components; besides binutils, GDB, and GLIBC, there's QEMU, newlib/libgloss,
and the Linux kernel.  But, we need to get the ball rolling somewhere.


CC:ing Arnd Bergmann regarding the obsolescence in the Linux kernel.


We have not yet marked nios2 as deprecated in the kernel, but that
is mostly because the implementation does not get in the way too
much and Dinh Nguyen is still around as a maintainer and merging
bugfixes.

Almost all nios2 kernel changes I see in the past decade have been
done blindly without testing on hardware, either for treewide
changes, or by code inspection. The only notable exceptions I could
find are from Andreas Oetken and Bernd Weiberg at Siemens and
from Marek Vasut (all added to Cc in case they have something to add).


I might still have the 10M50 board, but it wasn't powered for a very 
long time.

Re: [PATCH 4/5] docs/system/target-sparc: Improve the Sparc documentation

2024-04-18 Thread Thomas Huth 

On 18/04/2024 22.27, Mark Cave-Ayland wrote:

On 07/03/2024 17:43, Thomas Huth wrote:


Add some words about how to enable or disable boolean features,
and remove the note about a Linux kernel being available on the
QEMU website (they have been removed long ago already).

Signed-off-by: Thomas Huth 
---
  docs/system/target-sparc.rst | 8 ++--
  1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/docs/system/target-sparc.rst b/docs/system/target-sparc.rst
index 9ec8c90c14..9f418b9d3e 100644
--- a/docs/system/target-sparc.rst
+++ b/docs/system/target-sparc.rst
@@ -27,6 +27,11 @@ architecture machines:
  The emulation is somewhat complete. SMP up to 16 CPUs is supported, but
  Linux limits the number of usable CPUs to 4.
+The list of available CPUs can be viewed by starting QEMU with ``-cpu 
help``.
+Optional boolean features can be added with a "+" in front of the feature 
name,

+or disabled with a "-" in front of the name, for example
+``-cpu TI-SuperSparc-II,+float128``.
+
  QEMU emulates the following sun4m peripherals:
  -  IOMMU
@@ -55,8 +60,7 @@ OpenBIOS is a free (GPL v2) portable firmware 
implementation. The goal

  is to implement a 100% IEEE 1275-1994 (referred to as Open Firmware)
  compliant firmware.
-A sample Linux 2.6 series kernel and ram disk image are available on the
-QEMU web site. There are still issues with NetBSD and OpenBSD, but most
+There are still issues with NetBSD and OpenBSD, but most
  kernel versions work. Please note that currently older Solaris kernels
  don't work probably due to interface issues between OpenBIOS and
  Solaris.


Just curious as to what current issues exist with NetBSD and OpenBSD? At 
least both my NetBSD and OpenBSD test images survive a casual boot test here 
with latest git.


Maybe it's also about a long fixed bug ... shall I remove that sentence 
while I'm at it?


 Thomas

[PATCH 21/27] docs/qapi-domain: RFC patch - add malformed field list entries

2024-04-18 Thread John Snow 
This patch demonstrates what happens when you mess up a field list
entry. The next patch adds a safeguard against this.

Signed-off-by: John Snow 
---
 docs/qapi/index.rst | 4 
 1 file changed, 4 insertions(+)

diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
index c68e2044890..5bb1c37a5ed 100644
--- a/docs/qapi/index.rst
+++ b/docs/qapi/index.rst
@@ -129,6 +129,10 @@ Explicit cross-referencing syntax for QAPI modules is 
available with
   argument values. It's very temperamental.
:return SomeTypeName: An esoteric collection of mystical nonsense to
   both confound and delight.
+   :arg: this is malformed.
+   :memb: this is malformed and unrecognized.
+   :choice type name: This is unrecognized.
+   :errors FooError: Also malformed.
 
Field lists can appear anywhere in the directive block, but any field
list entries in the same list block that are recognized as special
-- 
2.44.0

[PATCH 11/27] docs/qapi-domain: add "Errors:" field lists

2024-04-18 Thread John Snow 
``:error type: descr`` can now be used to document error conditions,
naming the type of error object and a description of when the error is
surfaced.

Like the previous Arguments patch, this patch does not apply any special
QAPI syntax highlighting or cross-referencing for the types, but this
can be adjusted in the future if desired.

(At present, I have no commits that add such highlighting. Sphinx also
does not appear to support Grouped fields with optional (or no)
parameters, so the ability to exclude error types is currently not
supported. If you omit the type, Sphinx treats it as a regular field
list and doesn't apply the special Grouping postprocessing to it.)

Signed-off-by: John Snow 
---
 docs/qapi/index.rst| 4 
 docs/sphinx/qapi-domain.py | 6 ++
 2 files changed, 10 insertions(+)

diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
index a570c37abb2..004d02e0437 100644
--- a/docs/qapi/index.rst
+++ b/docs/qapi/index.rst
@@ -98,6 +98,10 @@ Explicit cross-referencing syntax for QAPI modules is 
available with
:feat unstable: More than unstable, this command doesn't even exist!
:arg no-descr:
:feat hallucination: This command is a figment of your imagination.
+   :error CommandNotFound: When you try to use this command, because it
+  isn't real.
+   :error GenericError: If the system decides it doesn't like the
+  argument values. It's very temperamental.
 
Field lists can appear anywhere in the directive block, but any field
list entries in the same list block that are recognized as special
diff --git a/docs/sphinx/qapi-domain.py b/docs/sphinx/qapi-domain.py
index c0dc6482204..1f0b168fa2c 100644
--- a/docs/sphinx/qapi-domain.py
+++ b/docs/sphinx/qapi-domain.py
@@ -273,6 +273,12 @@ class QAPICommand(QAPIObject):
 names=("arg",),
 can_collapse=True,
 ),
+GroupedField(
+"error",
+label=_("Errors"),
+names=("error",),
+can_collapse=True,
+),
 ]
 )
 
-- 
2.44.0

[PATCH 20/27] docs/qapi-domain: add :ifcond: directive option

2024-04-18 Thread John Snow 
Add a special :ifcond: option that allows us to annotate the
definition-level conditionals.

RFC: This patch renders IFCOND information in two places, because I'm
undecided about how to style this information. One option is in the
signature bar, and another option is in an eye-catch, like :deprecated:
or :unstable:.

A benefit to having this be a directive option is that we can put it in
the signature bar, the QAPI index, etc. However, if we merely want it in
the content section, a directive would work just as well,
e.g. ".. qapi:ifcond:: CONFIG_LINUX".

(Though, having it be in the same containing box as the unstable/ifcond
boxes might require some extra fiddling/post-processing to
achieve. Generally, the less docutils tree muddling I have to do, the
happier I am.)

The syntax of the argument is currently undefined, but it is possible to
parse it back down into constituent parts to avoid applying literal
formatting to "AND" or "&&" or whichever syntax we formalize. (Or, in
the future, applying cross-reference links to the config values for
additional reading on some of those build options. Not for this series.)

"Vote now on your phones!"

Signed-off-by: Harmonie Snow 
Signed-off-by: John Snow 
---
 docs/qapi/index.rst|  1 +
 docs/sphinx-static/theme_overrides.css | 13 +
 docs/sphinx/qapi-domain.py | 37 --
 3 files changed, 49 insertions(+), 2 deletions(-)

diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
index 6350791a3ed..c68e2044890 100644
--- a/docs/qapi/index.rst
+++ b/docs/qapi/index.rst
@@ -97,6 +97,7 @@ Explicit cross-referencing syntax for QAPI modules is 
available with
:since: 13.37
:deprecated:
:unstable:
+   :ifcond: CONFIG_LINUX
 
This is a fake command, it's not real. It can't hurt you.
 
diff --git a/docs/sphinx-static/theme_overrides.css 
b/docs/sphinx-static/theme_overrides.css
index acdf11675db..b239a762a9e 100644
--- a/docs/sphinx-static/theme_overrides.css
+++ b/docs/sphinx-static/theme_overrides.css
@@ -188,3 +188,16 @@ div[class^="highlight"] pre {
 .qapi-deprecated::before {
 content: '⚠️ ';
 }
+
+.qapi-ifcond::before {
+/* gaze ye into the crystal ball to determine feature availability */
+content: ' ';
+}
+
+.qapi-ifcond {
+background-color: #f9f5ff;
+border: solid #dac2ff 6px;
+padding: 8px;
+border-radius: 15px;
+margin: 5px;
+}
diff --git a/docs/sphinx/qapi-domain.py b/docs/sphinx/qapi-domain.py
index 76157476e02..e44db10488f 100644
--- a/docs/sphinx/qapi-domain.py
+++ b/docs/sphinx/qapi-domain.py
@@ -15,6 +15,7 @@
 NamedTuple,
 Optional,
 Tuple,
+Union,
 cast,
 )
 
@@ -150,6 +151,7 @@ class QAPIObject(ObjectDescription[Signature]):
 "module": directives.unchanged,  # Override contextual module name
 # These are QAPI originals:
 "since": since_validator,
+"ifcond": directives.unchanged,
 "deprecated": directives.flag,
 "unstable": directives.flag,
 }
@@ -176,6 +178,20 @@ def get_signature_suffix(self, sig: str) -> 
List[nodes.Node]:
 """Returns a suffix to put after the object name in the signature."""
 ret: List[nodes.Node] = []
 
+if "ifcond" in self.options:
+ret += [
+addnodes.desc_sig_space(),
+nodes.inline(
+self.options["ifcond"],
+"",
+nodes.Text("["),
+nodes.literal("", "#if"),
+addnodes.desc_sig_space(),
+nodes.literal(self.options["ifcond"], 
self.options["ifcond"]),
+nodes.Text("]"),
+),
+]
+
 if "since" in self.options:
 ret += [
 addnodes.desc_sig_space(),
@@ -257,9 +273,14 @@ def _add_infopips(self, contentnode: 
addnodes.desc_content) -> None:
 infopips = nodes.container()
 infopips.attributes["classes"].append("qapi-infopips")
 
-def _add_pip(source: str, content: str, classname: str) -> None:
+def _add_pip(
+source: str, content: Union[str, List[nodes.Node]], classname: str
+) -> None:
 node = nodes.container(source)
-node.append(nodes.Text(content))
+if isinstance(content, str):
+node.append(nodes.Text(content))
+else:
+node.extend(content)
 node.attributes["classes"].extend(["qapi-infopip", classname])
 infopips.append(node)
 
@@ -277,6 +298,18 @@ def _add_pip(source: str, content: str, classname: str) -> 
None:
 "qapi-unstable",
 )
 
+if self.options.get("ifcond", ""):
+ifcond = self.options["ifcond"]
+_add_pip(
+f":ifcond: {ifcond}",
+[
+nodes.emphasis("", "Availability"),
+nodes.Text(":

[PATCH 13/27] docs/qapi-domain: add qapi:enum directive

2024-04-18 Thread John Snow 
Add the .. qapi:enum:: directive, object, and :qapi:enum:`name`
cross-reference role.

Add the :value name: field list for documenting Enum values.

Of note, also introduce a new "type" role that is intended to be used by
other QAPI object directives to cross-reference arbitrary QAPI type
names, but will exclude commands, events, and modules from
consideration.

Signed-off-by: John Snow 
---
 docs/qapi/index.rst| 14 ++
 docs/sphinx/qapi-domain.py | 24 
 2 files changed, 38 insertions(+)

diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
index 39fe4dd2dae..cf794e6e739 100644
--- a/docs/qapi/index.rst
+++ b/docs/qapi/index.rst
@@ -113,3 +113,17 @@ Explicit cross-referencing syntax for QAPI modules is 
available with
At the moment, the order of grouped sections is based on the order in
which each group was encountered. This example will render Arguments
first, and then Features; but the order can be any that you choose.
+
+.. qapi:enum:: BitmapSyncMode
+   :since: 4.2
+
+   An enumeration of possible behaviors for the synchronization of a
+   bitmap when used for data copy operations.
+
+   :value on-success: The bitmap is only synced when the operation is
+  successful. This is the behavior always used for
+  ``INCREMENTAL`` backups.
+   :value never: The bitmap is never synchronized with the operation, and
+  is treated solely as a read-only manifest of blocks to copy.
+   :value always: The bitmap is always synchronized with the operation,
+  regardless of whether or not the operation was successful.
diff --git a/docs/sphinx/qapi-domain.py b/docs/sphinx/qapi-domain.py
index 5d44dba6cd3..6759c39290d 100644
--- a/docs/sphinx/qapi-domain.py
+++ b/docs/sphinx/qapi-domain.py
@@ -289,6 +289,22 @@ class QAPICommand(QAPIObject):
 )
 
 
+class QAPIEnum(QAPIObject):
+"""Description of a QAPI Enum."""
+
+doc_field_types = QAPIObject.doc_field_types.copy()
+doc_field_types.extend(
+[
+GroupedField(
+"value",
+label=_("Values"),
+names=("value",),
+can_collapse=True,
+)
+]
+)
+
+
 class QAPIModule(SphinxDirective):
 """
 Directive to mark description of a new module.
@@ -434,9 +450,14 @@ class QAPIDomain(Domain):
 # This table associates cross-reference object types (key) with an
 # ObjType instance, which defines the valid cross-reference roles
 # for each object type.
+#
+# e.g., the :qapi:type: cross-reference role can refer to enum,
+# struct, union, or alternate objects; but :qapi:obj: can refer to
+# anything. Each object also gets its own targeted cross-reference role.
 object_types: Dict[str, ObjType] = {
 "module": ObjType(_("module"), "mod", "obj"),
 "command": ObjType(_("command"), "cmd", "obj"),
+"enum": ObjType(_("enum"), "enum", "obj", "type"),
 }
 
 # Each of these provides a ReST directive,
@@ -444,6 +465,7 @@ class QAPIDomain(Domain):
 directives = {
 "module": QAPIModule,
 "command": QAPICommand,
+"enum": QAPIEnum,
 }
 
 # These are all cross-reference roles; e.g.
@@ -452,6 +474,8 @@ class QAPIDomain(Domain):
 roles = {
 "mod": QAPIXRefRole(),
 "cmd": QAPIXRefRole(),
+"enum": QAPIXRefRole(),
+"type": QAPIXRefRole(),  # reference any data type (excludes modules, 
commands, events)
 "obj": QAPIXRefRole(),  # reference *any* type of QAPI object.
 }
 
-- 
2.44.0

[PATCH 14/27] docs/qapi-domain: add qapi:alternate directive

2024-04-18 Thread John Snow 
Add the .. qapi:alternate:: directive, object, and qapi:alt:`name`
cross-reference role.

Add the "Choices:" field list for describing alternate choices. Like
other field lists that reference QAPI types, a forthcoming commit will
add cross-referencing support to this field.

RFC: In the future, it would be nice to directly inline Alternates as
part of the type information in the containing object (i.e. directly in
arguments/members) - but that's a task for another series. For now, the
branch "names" are documented just like qapidoc.py does, even though
this information is superfluous for user documentation. Room for future
improvement, but not now.

Signed-off-by: John Snow 
---
 docs/qapi/index.rst|  7 +++
 docs/sphinx/qapi-domain.py | 19 +++
 2 files changed, 26 insertions(+)

diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
index cf794e6e739..9bfe4d9f454 100644
--- a/docs/qapi/index.rst
+++ b/docs/qapi/index.rst
@@ -127,3 +127,10 @@ Explicit cross-referencing syntax for QAPI modules is 
available with
   is treated solely as a read-only manifest of blocks to copy.
:value always: The bitmap is always synchronized with the operation,
   regardless of whether or not the operation was successful.
+
+.. qapi:alternate:: BlockDirtyBitmapOrStr
+   :since: 4.1
+
+   :choice str local: name of the bitmap, attached to the same node as
+  target bitmap.
+   :choice BlockDirtyBitmap external: bitmap with specified node
diff --git a/docs/sphinx/qapi-domain.py b/docs/sphinx/qapi-domain.py
index 6759c39290d..c6eb6324594 100644
--- a/docs/sphinx/qapi-domain.py
+++ b/docs/sphinx/qapi-domain.py
@@ -305,6 +305,22 @@ class QAPIEnum(QAPIObject):
 )
 
 
+class QAPIAlternate(QAPIObject):
+"""Description of a QAPI Alternate."""
+
+doc_field_types = QAPIObject.doc_field_types.copy()
+doc_field_types.extend(
+[
+TypedField(
+"choice",
+label=_("Choices"),
+names=("choice",),
+can_collapse=True,
+),
+]
+)
+
+
 class QAPIModule(SphinxDirective):
 """
 Directive to mark description of a new module.
@@ -458,6 +474,7 @@ class QAPIDomain(Domain):
 "module": ObjType(_("module"), "mod", "obj"),
 "command": ObjType(_("command"), "cmd", "obj"),
 "enum": ObjType(_("enum"), "enum", "obj", "type"),
+"alternate": ObjType(_("alternate"), "alt", "obj", "type"),
 }
 
 # Each of these provides a ReST directive,
@@ -466,6 +483,7 @@ class QAPIDomain(Domain):
 "module": QAPIModule,
 "command": QAPICommand,
 "enum": QAPIEnum,
+"alternate": QAPIAlternate,
 }
 
 # These are all cross-reference roles; e.g.
@@ -475,6 +493,7 @@ class QAPIDomain(Domain):
 "mod": QAPIXRefRole(),
 "cmd": QAPIXRefRole(),
 "enum": QAPIXRefRole(),
+"alt": QAPIXRefRole(),
 "type": QAPIXRefRole(),  # reference any data type (excludes modules, 
commands, events)
 "obj": QAPIXRefRole(),  # reference *any* type of QAPI object.
 }
-- 
2.44.0

[PATCH 08/27] docs/qapi-domain: add :since: directive option

2024-04-18 Thread John Snow 
Add a little special markup for registering "Since:" information. Adding
it as an option instead of generic content lets us hoist the information
into the Signature bar, optionally put it in the index, etc.

Signed-off-by: John Snow 
---
 docs/qapi/index.rst|  1 +
 docs/sphinx/qapi-domain.py | 27 +--
 2 files changed, 26 insertions(+), 2 deletions(-)

diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
index 5516f762a24..33b9349a3ee 100644
--- a/docs/qapi/index.rst
+++ b/docs/qapi/index.rst
@@ -54,6 +54,7 @@ Explicit cross-referencing syntax for QAPI modules is 
available with
 
 
 .. qapi:command:: example-command
+   :since: 42.0
 
This directive creates a QAPI command named `example-command` that
appears in both the `genindex` and the `qapi-index`. As of this
diff --git a/docs/sphinx/qapi-domain.py b/docs/sphinx/qapi-domain.py
index 2c1e60290d9..38a50318d08 100644
--- a/docs/sphinx/qapi-domain.py
+++ b/docs/sphinx/qapi-domain.py
@@ -4,6 +4,7 @@
 
 from __future__ import annotations
 
+import re
 from typing import (
 TYPE_CHECKING,
 Any,
@@ -86,6 +87,18 @@ def process_link(
 return title, target
 
 
+def since_validator(param: str) -> str:
+"""
+Validate the `:since: X.Y` option field.
+"""
+match = re.match(r"[0-9]+\.[0-9]+", param)
+if not match:
+raise ValueError(
+f":since: requires a version number in X.Y format; not {param!r}"
+)
+return param
+
+
 def _nested_parse(directive: SphinxDirective, content_node: Element) -> None:
 """
 This helper preserves error parsing context across sphinx versions.
@@ -127,6 +140,8 @@ class QAPIObject(ObjectDescription[Signature]):
 {
 # Borrowed from the Python domain:
 "module": directives.unchanged,  # Override contextual module name
+# These are QAPI originals:
+"since": since_validator,
 }
 )
 
@@ -138,9 +153,17 @@ def get_signature_prefix(self, sig: str) -> 
List[nodes.Node]:
 addnodes.desc_sig_space(),
 ]
 
-def get_signature_suffix(self, sig: str) -> list[nodes.Node]:
+def get_signature_suffix(self, sig: str) -> List[nodes.Node]:
 """Returns a suffix to put after the object name in the signature."""
-return []
+ret: List[nodes.Node] = []
+
+if "since" in self.options:
+ret += [
+addnodes.desc_sig_space(),
+addnodes.desc_sig_element("", f"(Since: 
{self.options['since']})"),
+]
+
+return ret
 
 def handle_signature(self, sig: str, signode: desc_signature) -> Signature:
 """
-- 
2.44.0

[PATCH 24/27] docs/qapi-domain: add type cross-refs to field lists

2024-04-18 Thread John Snow 
This commit, finally, adds cross-referencing support to various field
lists; modeled tightly after Sphinx's own Python domain code.

Cross-referencing support is added to type names provided to :arg:,
:memb:, :returns: and :choice:.

:feat:, :error: and :value:, which do not take type names, do not
support this syntax.

The general syntax is simple:

:arg TypeName ArgName: Lorem Ipsum ...

The domain will transform TypeName into :qapi:type:`TypeName` in this
basic case, and also apply the ``literal`` decoration to indicate that
this is a type cross-reference.

For Optional arguments, the special "?" suffix is used. Because "*" has
special meaning in ReST that would cause parsing errors, we elect to use
"?" instead. The special syntax processing in QAPIXrefMixin strips this
character from the end of any type name argument and will append ",
Optional" to the rendered output, applying the cross-reference only to
the actual type name.

The intent here is that the actual syntax in doc-blocks need not change;
but e.g. qapidoc.py will need to process and transform "@arg foo lorem
ipsum" into ":arg type? foo: lorem ipsum" based on the schema
information. Therefore, nobody should ever actually witness this
intermediate syntax unless they are writing manual documentation or the
doc transmogrifier breaks.

For array arguments, type names can similarly be surrounded by "[]",
which are stripped off and then re-appended outside of the
cross-reference.

Note: The mixin pattern here (borrowed from Sphinx) confuses mypy
because it cannot tell that it will be mixed into a descendent of
Field. Doing that instead causes more errors, because many versions of
Sphinx erroneously did not mark various arguments as Optional, so we're
a bit hosed either way. Do the simpler thing.

Signed-off-by: John Snow 
---
 docs/qapi/index.rst|  34 
 docs/sphinx/qapi-domain.py | 110 +++--
 2 files changed, 138 insertions(+), 6 deletions(-)

diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
index 8352a27d4a5..6e85ea5280d 100644
--- a/docs/qapi/index.rst
+++ b/docs/qapi/index.rst
@@ -105,6 +105,11 @@ Explicit cross-referencing syntax for QAPI modules is 
available with
:arg str bar: Another normal parameter description.
:arg baz: Missing a type.
:arg no-descr:
+   :arg int? oof: Testing optional argument parsing.
+   :arg [XDbgBlockGraphNode] rab: Testing array argument parsing.
+   :arg [BitmapSyncMode]? zab: Testing optional array argument parsing,
+  even though Markus said this should never happen. I believe him,
+  but I didn't *forbid* the syntax either.
:arg BitmapSyncMode discrim: How about branches in commands?
 
.. qapi:branch:: discrim on-success
@@ -261,3 +266,32 @@ Explicit cross-referencing syntax for QAPI modules is 
available with
 
   :memb str key-secret: ID of a QCryptoSecret object providing a
  passphrase for unlocking the encryption
+
+.. qapi:command:: x-debug-query-block-graph
+   :since: 4.0
+   :unstable:
+
+   Get the block graph.
+
+   :feat unstable: This command is meant for debugging.
+   :return XDbgBlockGraph: lorem ipsum ...
+
+.. qapi:struct:: XDbgBlockGraph
+   :since: 4.0
+
+   Block Graph - list of nodes and list of edges.
+
+   :memb [XDbgBlockGraphNode] nodes:
+   :memb [XDbgBlockGraphEdge] edges:
+
+.. qapi:struct:: XDbgBlockGraphNode
+   :since: 4.0
+
+   :memb uint64 id: Block graph node identifier.  This @id is generated only 
for
+  x-debug-query-block-graph and does not relate to any other
+  identifiers in Qemu.
+   :memb XDbgBlockGraphNodeType type: Type of graph node.  Can be one of
+  block-backend, block-job or block-driver-state.
+   :memb str name: Human readable name of the node.  Corresponds to
+  node-name for block-driver-state nodes; is not guaranteed to be
+  unique in the whole graph (with block-jobs and block-backends).
diff --git a/docs/sphinx/qapi-domain.py b/docs/sphinx/qapi-domain.py
index bf8bb933345..074453193ce 100644
--- a/docs/sphinx/qapi-domain.py
+++ b/docs/sphinx/qapi-domain.py
@@ -50,11 +50,12 @@
 
 if TYPE_CHECKING:
 from docutils.nodes import Element, Node
+from docutils.parsers.rst.states import Inliner
 
 from sphinx.application import Sphinx
 from sphinx.builders import Builder
 from sphinx.environment import BuildEnvironment
-from sphinx.util.typing import OptionSpec
+from sphinx.util.typing import OptionSpec, TextlikeNode
 
 logger = logging.getLogger(__name__)
 
@@ -68,6 +69,90 @@ class ObjectEntry(NamedTuple):
 aliased: bool
 
 
+class QAPIXrefMixin:
+def make_xref(
+self,
+rolename: str,
+domain: str,
+target: str,
+innernode: type[TextlikeNode] = nodes.literal,
+contnode: Optional[Node] = None,
+env: Optional[BuildEnvironment] = None,
+inliner: Optional[Inliner] = None,
+location: Optional[Node] = None,
+) -> Node:
+result =

[PATCH 04/27] docs/qapi-domain: add QAPI index

2024-04-18 Thread John Snow 
Use the QAPI object registry to generate a special index just for QAPI
definitions. The index can show entries both by definition type and
alphabetically.

The index can be linked from anywhere in the QEMU manual by using
`qapi-index`.

Signed-off-by: John Snow 
---
 docs/qapi/index.rst|  6 +++-
 docs/sphinx/qapi-domain.py | 66 +++---
 2 files changed, 66 insertions(+), 6 deletions(-)

diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
index 880fd17c709..051dc6b3a37 100644
--- a/docs/qapi/index.rst
+++ b/docs/qapi/index.rst
@@ -9,7 +9,8 @@ QAPI Domain Test
doesn't create a cross-reference target and it isn't added to the
index.
 
-   Check out the `genindex` for proof that foo-module is not present.
+   Check out the `genindex` or the `qapi-index` for proof that
+   foo-module is not present.
 
 .. qapi:module:: bar-module
:no-typesetting:
@@ -36,3 +37,6 @@ QAPI Domain Test
 
The ``block-core`` module will have two entries in the `genindex`,
under both "block-core" and "QAPI module".
+
+   Modules will also be reported in the `qapi-index`, under the Modules
+   category and in the alphabetical categories that follow.
diff --git a/docs/sphinx/qapi-domain.py b/docs/sphinx/qapi-domain.py
index ab80fd5f634..65409786119 100644
--- a/docs/sphinx/qapi-domain.py
+++ b/docs/sphinx/qapi-domain.py
@@ -12,6 +12,7 @@
 Iterable,
 List,
 NamedTuple,
+Optional,
 Tuple,
 cast,
 )
@@ -20,7 +21,12 @@
 from docutils.parsers.rst import directives
 
 from sphinx import addnodes
-from sphinx.domains import Domain, ObjType
+from sphinx.domains import (
+Domain,
+Index,
+IndexEntry,
+ObjType,
+)
 from sphinx.locale import _, __
 from sphinx.util import logging
 from sphinx.util.docutils import SphinxDirective, switch_source_input
@@ -74,9 +80,10 @@ class QAPIModule(SphinxDirective):
 a pass-through for the content body. Named section titles are
 allowed in the content body.
 
-Use this directive to associate subsequent definitions with the
-module they are defined in for purposes of search and QAPI index
-organization.
+Use this directive to create entries for the QAPI module in the
+global index and the qapi index; as well as to associate subsequent
+definitions with the module they are defined in for purposes of
+search and QAPI index organization.
 
 :arg: The name of the module.
 :opt no-index: Don't add cross-reference targets or index entries.
@@ -155,6 +162,52 @@ def run(self) -> List[Node]:
 return ret
 
 
+class QAPIIndex(Index):
+"""
+Index subclass to provide the QAPI definition index.
+"""
+
+name = "index"
+localname = _("QAPI Index")
+shortname = _("QAPI Index")
+
+def generate(
+self,
+docnames: Optional[Iterable[str]] = None,
+) -> Tuple[List[Tuple[str, List[IndexEntry]]], bool]:
+assert isinstance(self.domain, QAPIDomain)
+content: Dict[str, List[IndexEntry]] = {}
+collapse = False
+
+# list of all object (name, ObjectEntry) pairs, sorted by name
+objects = sorted(self.domain.objects.items(), key=lambda x: 
x[0].lower())
+
+for objname, obj in objects:
+if docnames and obj.docname not in docnames:
+continue
+
+# Strip the module name out:
+objname = objname.split(".")[-1]
+
+# Add an alphabetical entry:
+entries = content.setdefault(objname[0].upper(), [])
+entries.append(
+IndexEntry(objname, 0, obj.docname, obj.node_id, obj.objtype, 
"", "")
+)
+
+# Add a categorical entry:
+category = obj.objtype.title() + "s"
+entries = content.setdefault(category, [])
+entries.append(IndexEntry(objname, 0, obj.docname, obj.node_id, 
"", "", ""))
+
+# alphabetically sort categories; type names first, ABC entries last.
+sorted_content = sorted(
+content.items(),
+key=lambda x: (len(x[0]) == 1, x[0]),
+)
+return sorted_content, collapse
+
+
 class QAPIDomain(Domain):
 """QAPI language domain."""
 
@@ -182,7 +235,10 @@ class QAPIDomain(Domain):
 "objects": {},  # fullname -> ObjectEntry
 }
 
-indices = []
+# Index pages to generate; each entry is an Index class.
+indices = [
+QAPIIndex,
+]
 
 @property
 def objects(self) -> Dict[str, ObjectEntry]:
-- 
2.44.0

[PATCH 02/27] docs/qapi-domain: add qapi:module directive

2024-04-18 Thread John Snow 
This adds a qapi:module directive, which just notes the current module
being documented and performs a nested parse of the content block, if
present.

This code is based pretty heavily on Sphinx's PyModule directive, but
with the modindex functionality excised.

This commit also adds the _nested_parse helper, which adds cross-version
compatibility for nested parsing while preserving proper line context
information.

For example:

.. qapi:module:: block-core

   Hello, and welcome to block-core!
   =

   lorem ipsum, dolor sit amet ...

(For RFC purposes, this commit also adds a test document that
demonstrates the functionality-so-far to allow reviewers to easily test
and experiment with each commit. The eventual submission for inclusion
will remove this playground file.)

Signed-off-by: John Snow 
---
 docs/index.rst |   1 +
 docs/qapi/index.rst|  38 +++
 docs/sphinx/qapi-domain.py | 128 -
 3 files changed, 166 insertions(+), 1 deletion(-)
 create mode 100644 docs/qapi/index.rst

diff --git a/docs/index.rst b/docs/index.rst
index 0b9ee9901d9..11c18c598a8 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -18,3 +18,4 @@ Welcome to QEMU's documentation!
interop/index
specs/index
devel/index
+   qapi/index
diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
new file mode 100644
index 000..880fd17c709
--- /dev/null
+++ b/docs/qapi/index.rst
@@ -0,0 +1,38 @@
+
+QAPI Domain Test
+
+
+.. qapi:module:: foo-module
+   :no-index:
+
+   This starts a hypothetical module named ``foo-module``, but it
+   doesn't create a cross-reference target and it isn't added to the
+   index.
+
+   Check out the `genindex` for proof that foo-module is not present.
+
+.. qapi:module:: bar-module
+   :no-typesetting:
+
+   This starts a hypothetical module named ``bar-module``, but the
+   contents of the body here will not be rendered in the
+   output. However, any link targets created here or in nested
+   directives will be preserved and functional.
+
+   Check out the `genindex` for proof that bar-module is present in two
+   places! (under both "bar-module" and "QAPI module".)
+
+.. qapi:module:: block-core
+
+   Block core (VM unrelated)
+   =
+
+   This starts the documentation section for the ``block-core`` module.
+   All documentation objects that follow belong to the block-core module
+   until another ``qapi:module:`` directive is encountered.
+
+   This directive does not create an entry in the sidebar or the TOC
+   *unless* you create a nested section title within the directive.
+
+   The ``block-core`` module will have two entries in the `genindex`,
+   under both "block-core" and "QAPI module".
diff --git a/docs/sphinx/qapi-domain.py b/docs/sphinx/qapi-domain.py
index 163b9ff21c3..7c5e4407bc1 100644
--- a/docs/sphinx/qapi-domain.py
+++ b/docs/sphinx/qapi-domain.py
@@ -7,21 +7,141 @@
 from typing import (
 TYPE_CHECKING,
 Any,
+ClassVar,
 Dict,
+Iterable,
 List,
 Tuple,
+cast,
 )
 
+from docutils import nodes
+from docutils.parsers.rst import directives
+
+from sphinx import addnodes
 from sphinx.domains import Domain, ObjType
 from sphinx.util import logging
+from sphinx.util.docutils import SphinxDirective, switch_source_input
+from sphinx.util.nodes import make_id, nested_parse_with_titles
 
 
 if TYPE_CHECKING:
+from docutils.nodes import Element, Node
+
 from sphinx.application import Sphinx
+from sphinx.util.typing import OptionSpec
 
 logger = logging.getLogger(__name__)
 
 
+def _nested_parse(directive: SphinxDirective, content_node: Element) -> None:
+"""
+This helper preserves error parsing context across sphinx versions.
+"""
+
+# necessary so that the child nodes get the right source/line set
+content_node.document = directive.state.document
+
+try:
+# Modern sphinx (6.2.0+) supports proper offsetting for
+# nested parse error context management
+nested_parse_with_titles(
+directive.state,
+directive.content,
+content_node,
+content_offset=directive.content_offset,  # type: ignore[call-arg]
+)
+except TypeError:
+# No content_offset argument. Fall back to SSI method.
+with switch_source_input(directive.state, directive.content):
+nested_parse_with_titles(directive.state, directive.content, 
content_node)
+
+
+class QAPIModule(SphinxDirective):
+"""
+Directive to mark description of a new module.
+
+This directive doesn't generate any special formatting, and is just
+a pass-through for the content body. Named section titles are
+allowed in the content body.
+
+Use this directive to associate subsequent definitions with the
+module they are defined in for purposes of search and QAPI index
+organization.
+
+:arg: The name

[PATCH 25/27] docs/qapi-domain: implement error context reporting fix

2024-04-18 Thread John Snow 
Sphinx 5.3.0 to Sphinx 6.2.0 has a bug where nested content in an
ObjectDescription content block has its error position reported
incorrectly due to an oversight when they added nested section support
to this directive.

(This bug is present in Sphinx's own Python and C domains; test it
yourself by creating a py:func directive and creating a syntax error in
the directive's content block.)

To avoid overriding and re-implementing the entirety of the run()
method, a workaround is employed where we parse the content block
ourselves in before_content(), then null the content block to make
Sphinx's own parsing a no-op. Then, in transform_content (which occurs
after Sphinx's nested parse), we simply swap our own parsed content tree
back in for Sphinx's.

It appears a little tricky, but it's the nicest solution I can find.

Signed-off-by: John Snow 
---
 docs/sphinx/qapi-domain.py | 22 ++
 1 file changed, 22 insertions(+)

diff --git a/docs/sphinx/qapi-domain.py b/docs/sphinx/qapi-domain.py
index 074453193ce..7d8911c635f 100644
--- a/docs/sphinx/qapi-domain.py
+++ b/docs/sphinx/qapi-domain.py
@@ -21,7 +21,9 @@
 
 from docutils import nodes
 from docutils.parsers.rst import directives
+from docutils.statemachine import StringList
 
+import sphinx
 from sphinx import addnodes
 from sphinx.addnodes import desc_signature, pending_xref
 from sphinx.directives import ObjectDescription
@@ -470,7 +472,27 @@ def _validate_field(self, field: nodes.field) -> None:
 )
 logger.warning(msg, location=field)
 
+def before_content(self) -> None:
+# Work around a sphinx bug and parse the content ourselves.
+self._temp_content = self.content
+self._temp_offset = self.content_offset
+self._temp_node = None
+
+if sphinx.version_info[:3] >= (5, 3, 0) and sphinx.version_info[:3] < 
(6, 2, 0):
+self._temp_node = addnodes.desc_content()
+self.state.nested_parse(self.content, self.content_offset, 
self._temp_node)
+# Sphinx will try to parse the content block itself,
+# Give it nothingness to parse instead.
+self.content = StringList()
+self.content_offset = 0
+
 def transform_content(self, contentnode: addnodes.desc_content) -> None:
+# Sphinx workaround: Inject our parsed content and restore state.
+if self._temp_node:
+contentnode += self._temp_node.children
+self.content = self._temp_content
+self.content_offset = self._temp_offset
+
 self._add_infopips(contentnode)
 self._merge_adjoining_field_lists(contentnode)
 
-- 
2.44.0

[PATCH 17/27] docs/qapi-domain: add qapi:union and qapi:branch directives

2024-04-18 Thread John Snow 
Adds the .. qapi:union:: directive, object, and :qapi:union:`name`
cross-referencing role.

In order to support discriminated branches of unions, a new qapi:branch
directive is created whose only purpose is to create a dynamically named
field list section based on the name of the branch key and value.

Because the label for these branch sections is dynamically generated,
this patch allows the use of either :memb: or :arg: in branches; they
are simply aliases and do not do anything different.

(This is to allow the directive to be used for either Commands or Unions
as needed; i.e. for commands whose argument type is a discriminated
union.)

For example:

.. qapi:union:: foo-union

   :memb foo-enum key: Discriminator field
   :memb foo-type fieldname: Some base type field that is always present

   .. qapi:branch:: key value1

  :memb type name: lorem ipsum ...

   .. qapi:branch:: key value2

  :memb type2 name2: dolor sit amet ...

In order to support this syntax, the root QAPIObject class needs to
perform some post-processing on field lists to merge adjecent field
lists. because each branch directive will return a separate field list,
and we want to combine them into one unified list for proper rendering.

NOTE: Technically, the branch directive will allow you to put arbitrary
ReST besides field lists inside of it, but it almost certainly won't do
anything you'd consider useful. I don't recommend it, but
programmatically forbidding it is expensive. Recommendation: "Don't!"

... and now ...

RFC: The branches abuse the field list system to create new categories
of field list entries that are dynamically generated. At the moment, I
do not have code to apply appropriate formatting to these headings, but
would like to apply ``literal`` syntax at a minimum to the enum values.

The other idea I have is to create a "blank" members value that doesn't
add a "title" (field label) on the left column and instead adds a
horizontal info bar into the arguments list in the right, but the
problem with this approach is that I won't be able to get rid of the "*
..." list bullet syntax, so it may look odd. Otherwise, a benefit to
this syntax is that it would allow us to write a longer explanation,
e.g. "When ``{key}`` is ``{value}``".

It's possible I could split the argument list into several nested lists,
but I haven't experimented in that direction yet - it's also likely that
Sphinx will want to append a ":" even to blank section titles, so this
may prove tricky to do within the constraints of Sphinx's existing Field
list system. (I believe the HTML formatter is responsible for appending
the colons.)

I'm open to suggestions, but wrestling with Sphinx, html and css is time
consuming (for me, at least), so ultimately, I'm afraid I must say:
***PATCHES WELCOME***.

Signed-off-by: John Snow 
---
 docs/qapi/index.rst| 37 ++
 docs/sphinx/qapi-domain.py | 98 +-
 2 files changed, 134 insertions(+), 1 deletion(-)

diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
index b07e6e9e2e3..d37ceac497f 100644
--- a/docs/qapi/index.rst
+++ b/docs/qapi/index.rst
@@ -103,6 +103,18 @@ Explicit cross-referencing syntax for QAPI modules is 
available with
:arg baz: Missing a type.
:feat unstable: More than unstable, this command doesn't even exist!
:arg no-descr:
+   :arg BitmapSyncMode discrim: How about branches in commands?
+
+   .. qapi:branch:: discrim on-success
+
+  :arg str foobar: This is an argument that belongs to a tagged union 
branch.
+  :arg int? foobaz: This is another argument belonging to the same branch.
+
+   .. qapi:branch:: discrim never
+
+  :arg str barfoo: This is an argument that belongs to a *different* 
tagged union branch.
+  :arg int64 zzxyz: And this is another argument belonging to that same 
branch.
+
:feat hallucination: This command is a figment of your imagination.
:error CommandNotFound: When you try to use this command, because it
   isn't real.
@@ -190,3 +202,28 @@ Explicit cross-referencing syntax for QAPI modules is 
available with
  operations.  0 means unlimited.  If max-chunk is non-zero then it
  should not be less than job cluster size which is calculated as
  maximum of target image cluster size and 64k.  Default 0.
+
+.. qapi:union:: RbdEncryptionOptions
+   :since: 6.1
+
+   :memb RbdImageEncryptionFormat format: Encryption format.
+   :memb RbdEncryptionOptions? parent: Parent image encryption options
+  (for cloned images).  Can be left unspecified if this cloned image
+  is encrypted using the same format and secret as its parent image
+  (i.e. not explicitly formatted) or if its parent image is not
+  encrypted.  (Since 8.0)
+
+   .. qapi:branch:: format luks
+
+  :memb str key-secret: ID of a QCryptoSecret object providing a
+ passphrase for unlocking the encryption
+
+   .. qapi:branch:: format luks2
+
+  :memb str key-secret: ID of a

[PATCH 18/27] docs/qapi-domain: add :deprecated: directive option

2024-04-18 Thread John Snow 
Although "deprecated" is a feature (and *will* appear in the features
list), add a special :deprecated: option to generate an eye-catch that
makes this information very hard to miss.

(The intent is to modify qapidoc.py to add this option whenever it
detects that the features list attached to a definition contains the
"deprecated" entry.)

-

RFC: Technically, this object-level option is un-needed and could be
replaced with a standard content-level directive that e.g. qapidoc.py
could insert at the beginning of the content block. I've done it here as
an option to demonstrate how it would be possible to do.

It's a matter of taste for "where" we feel like implementing it.

One benefit of doing it this way is that we can create a single
containing box to set CSS style options controlling the flow of multiple
infoboxes. The other way to achieve that would be to create a directive
that allows us to set multiple options instead, e.g.:

.. qapi:infoboxes:: deprecated unstable

or possibly:

.. qapi:infoboxes::
   :deprecated:
   :unstable:

For now, I've left these as top-level QAPI object options. "Hey, it works."

P.S., I outsourced the CSS ;)

Signed-off-by: Harmonie Snow 
Signed-off-by: John Snow 
---
 docs/qapi/index.rst|  4 
 docs/sphinx-static/theme_overrides.css | 25 +
 docs/sphinx/qapi-domain.py | 24 
 3 files changed, 53 insertions(+)

diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
index d37ceac497f..b9a69f6bd17 100644
--- a/docs/qapi/index.rst
+++ b/docs/qapi/index.rst
@@ -95,6 +95,7 @@ Explicit cross-referencing syntax for QAPI modules is 
available with
 
 .. qapi:command:: fake-command
:since: 13.37
+   :deprecated:
 
This is a fake command, it's not real. It can't hurt you.
 
@@ -116,6 +117,9 @@ Explicit cross-referencing syntax for QAPI modules is 
available with
   :arg int64 zzxyz: And this is another argument belonging to that same 
branch.
 
:feat hallucination: This command is a figment of your imagination.
+   :feat deprecated: Although this command is fake, you should know that
+  it's also deprecated. That's great news! Maybe it will go away and
+  stop haunting you someday.
:error CommandNotFound: When you try to use this command, because it
   isn't real.
:error GenericError: If the system decides it doesn't like the
diff --git a/docs/sphinx-static/theme_overrides.css 
b/docs/sphinx-static/theme_overrides.css
index c70ef951286..97b8c1c60e6 100644
--- a/docs/sphinx-static/theme_overrides.css
+++ b/docs/sphinx-static/theme_overrides.css
@@ -159,3 +159,28 @@ div[class^="highlight"] pre {
 color: inherit;
 }
 }
+
+/* QAPI domain theming */
+
+.qapi-infopips {
+margin-bottom: 1em;
+}
+
+.qapi-infopip {
+display: inline-block;
+padding: 0em 0.5em 0em 0.5em;
+margin: 0.25em;
+}
+
+.qapi-deprecated {
+background-color: #fffef5;
+border: solid #fff176 6px;
+font-weight: bold;
+padding: 8px;
+border-radius: 15px;
+margin: 5px;
+}
+
+.qapi-deprecated::before {
+content: '⚠️ ';
+}
diff --git a/docs/sphinx/qapi-domain.py b/docs/sphinx/qapi-domain.py
index f0094300c03..065ad501960 100644
--- a/docs/sphinx/qapi-domain.py
+++ b/docs/sphinx/qapi-domain.py
@@ -150,6 +150,7 @@ class QAPIObject(ObjectDescription[Signature]):
 "module": directives.unchanged,  # Override contextual module name
 # These are QAPI originals:
 "since": since_validator,
+"deprecated": directives.flag,
 }
 )
 
@@ -249,6 +250,28 @@ def add_target_and_index(
 ("single", indextext, node_id, "", None)
 )
 
+def _add_infopips(self, contentnode: addnodes.desc_content) -> None:
+# Add various eye-catches and things that go below the signature
+# bar, but precede the user-defined content.
+infopips = nodes.container()
+infopips.attributes["classes"].append("qapi-infopips")
+
+def _add_pip(source: str, content: str, classname: str) -> None:
+node = nodes.container(source)
+node.append(nodes.Text(content))
+node.attributes["classes"].extend(["qapi-infopip", classname])
+infopips.append(node)
+
+if "deprecated" in self.options:
+_add_pip(
+":deprecated:",
+f"This {self.objtype} is deprecated.",
+"qapi-deprecated",
+)
+
+if infopips.children:
+contentnode.insert(0, infopips)
+
 def _merge_adjoining_field_lists(self, contentnode: addnodes.desc_content) 
-> None:
 # Take any adjacent field lists and glue them together into
 # one list for further processing by Sphinx. This is done so
@@ -271,6 +294,7 @@ def _merge_adjoining_field_lists(self, contentnode: 
addnodes.desc_content) -> No
 contentnode.remove(child)
 
 def transform_content(self,

[PATCH 07/27] docs/qapi-domain: add qapi:command directive

2024-04-18 Thread John Snow 
This commit adds a generic QAPIObject class for use in documenting
various QAPI entities in the Sphinx ecosystem.

It also adds a stubbed version of QAPICommand that utilizes the
QAPIObject class; along with the qapi:command directive, the
:qapi:cmd: cross-reference role, and the "command" object type in the
QAPI object registry.

They don't do anything *particularly* interesting yet, but that will
come in forthcoming commits.

Note: some versions of mypy get a little confused over the difference
between class and instance variables; because sphinx's ObjectDescription
does not declare option_spec as a ClassVar (even though it's obvious
that it is), mypy may produce this error:

qapi-domain.py:125: error: Cannot override instance variable (previously
declared on base class "ObjectDescription") with class variable [misc]

I can't control that; so silence the error with a pragma.

Signed-off-by: John Snow 
---
 docs/qapi/index.rst|  34 ++
 docs/sphinx/qapi-domain.py | 132 -
 2 files changed, 165 insertions(+), 1 deletion(-)

diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
index e2223d5f363..5516f762a24 100644
--- a/docs/qapi/index.rst
+++ b/docs/qapi/index.rst
@@ -51,3 +51,37 @@ the actual output of that directive was suppressed. Here's a 
link to
 Explicit cross-referencing syntax for QAPI modules is available with
 ``:qapi:mod:`foo```, here's a link to :qapi:mod:`bar-module` and one to
 :qapi:mod:`block-core`.
+
+
+.. qapi:command:: example-command
+
+   This directive creates a QAPI command named `example-command` that
+   appears in both the `genindex` and the `qapi-index`. As of this
+   commit, there aren't any special arguments or options you can give to
+   this directive, it merely parses its content block and handles the
+   TOC/index/xref book-keeping.
+
+   Unlike the QAPI module directive, this directive *does* add a TOC
+   entry by default.
+
+   This object can be referenced in *quite a few ways*:
+
+   * ```example-command``` => `example-command`
+   * ```block-core.example-command``` => `block-core.example-command`
+   * ``:qapi:cmd:`example-command``` => :qapi:cmd:`example-command`
+   * ``:qapi:cmd:`block-core.example-command``` => 
:qapi:cmd:`block-core.example-command`
+   * ``:qapi:cmd:`~example-command``` => :qapi:cmd:`~example-command`
+   * ``:qapi:cmd:`~block-core.example-command``` => 
:qapi:cmd:`~block-core.example-command`
+   * ``:qapi:obj:`example-command``` => :qapi:obj:`example-command`
+   * ``:qapi:obj:`block-core.example-command``` => 
:qapi:obj:`block-core.example-command`
+   * ``:qapi:obj:`~example-command``` => :qapi:obj:`~example-command`
+   * ``:qapi:obj:`~block-core.example-command``` => 
:qapi:obj:`~block-core.example-command`
+
+   As of Sphinx v7.2.6, there are a few sphinx-standard options this
+   directive has:
+
+   * ``:no-index:`` or ``:noindex:`` Don't add to the `genindex` nor
+ the `qapi-index`; do not register for cross-references.
+   * ``:no-index-entry:`` or ``:noindexentry:``
+   * ``:no-contents-entry:`` or ``:nocontentsentry:``
+   * ``:no-typesetting:``
diff --git a/docs/sphinx/qapi-domain.py b/docs/sphinx/qapi-domain.py
index d28ac1cb9d8..2c1e60290d9 100644
--- a/docs/sphinx/qapi-domain.py
+++ b/docs/sphinx/qapi-domain.py
@@ -21,7 +21,8 @@
 from docutils.parsers.rst import directives
 
 from sphinx import addnodes
-from sphinx.addnodes import pending_xref
+from sphinx.addnodes import desc_signature, pending_xref
+from sphinx.directives import ObjectDescription
 from sphinx.domains import (
 Domain,
 Index,
@@ -108,6 +109,132 @@ def _nested_parse(directive: SphinxDirective, 
content_node: Element) -> None:
 nested_parse_with_titles(directive.state, directive.content, 
content_node)
 
 
+# Alias for the return of handle_signature(), which is used in several places.
+# (In the Python domain, this is Tuple[str, str] instead.)
+Signature = str
+
+
+class QAPIObject(ObjectDescription[Signature]):
+"""
+Description of a generic QAPI object.
+
+It's not used directly, but is instead subclassed by specific directives.
+"""
+
+# Inherit some standard options from Sphinx's ObjectDescription
+option_spec: OptionSpec = ObjectDescription.option_spec.copy()  # 
type:ignore[misc]
+option_spec.update(
+{
+# Borrowed from the Python domain:
+"module": directives.unchanged,  # Override contextual module name
+}
+)
+
+def get_signature_prefix(self, sig: str) -> List[nodes.Node]:
+"""Returns a prefix to put before the object name in the signature."""
+assert self.objtype
+return [
+addnodes.desc_sig_keyword("", self.objtype.title()),
+addnodes.desc_sig_space(),
+]
+
+def get_signature_suffix(self, sig: str) -> list[nodes.Node]:
+"""Returns a suffix to put after the object name in the signature."""
+return []
+
+def

[PATCH 16/27] docs/qapi-domain: add qapi:struct directive

2024-04-18 Thread John Snow 
Adds the .. qapi:struct:: directive, object, and :qapi:struct:`name`
cross-referencing role.

As per usual, QAPI cross-referencing for types in the member field list
will be added in a forthcoming commit.

RFC Note: The "?" syntax sneaks into the example document again. Please
ignore that for now.

Signed-off-by: John Snow 
---
 docs/qapi/index.rst| 16 
 docs/sphinx/qapi-domain.py |  9 +
 2 files changed, 25 insertions(+)

diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
index d81bccfb06a..b07e6e9e2e3 100644
--- a/docs/qapi/index.rst
+++ b/docs/qapi/index.rst
@@ -174,3 +174,19 @@ Explicit cross-referencing syntax for QAPI modules is 
available with
  "microseconds": 959568
}
  }
+
+.. qapi:struct:: BackupPerf
+   :since: 6.0
+
+   Optional parameters for backup.  These parameters don't affect
+   functionality, but may significantly affect performance.
+
+   :memb bool? use-copy-range: Use copy offloading.  Default false.
+   :memb int? max-workers: Maximum number of parallel requests for the
+  sustained background copying process.  Doesn't influence
+  copy-before-write operations.  Default 64.
+   :memb int64? max-chunk: Maximum request length for the sustained
+ background copying process.  Doesn't influence copy-before-write
+ operations.  0 means unlimited.  If max-chunk is non-zero then it
+ should not be less than job cluster size which is calculated as
+ maximum of target image cluster size and 64k.  Default 0.
diff --git a/docs/sphinx/qapi-domain.py b/docs/sphinx/qapi-domain.py
index 74dc578b3c7..b46faeaceef 100644
--- a/docs/sphinx/qapi-domain.py
+++ b/docs/sphinx/qapi-domain.py
@@ -343,6 +343,12 @@ class QAPIEvent(QAPIObjectWithMembers):
 pass
 
 
+class QAPIStruct(QAPIObjectWithMembers):
+"""Description of a QAPI Struct."""
+
+pass
+
+
 class QAPIModule(SphinxDirective):
 """
 Directive to mark description of a new module.
@@ -497,6 +503,7 @@ class QAPIDomain(Domain):
 "command": ObjType(_("command"), "cmd", "obj"),
 "event": ObjType(_("event"), "event", "obj"),
 "enum": ObjType(_("enum"), "enum", "obj", "type"),
+"struct": ObjType(_("struct"), "struct", "obj", "type"),
 "alternate": ObjType(_("alternate"), "alt", "obj", "type"),
 }
 
@@ -507,6 +514,7 @@ class QAPIDomain(Domain):
 "command": QAPICommand,
 "event": QAPIEvent,
 "enum": QAPIEnum,
+"struct": QAPIStruct,
 "alternate": QAPIAlternate,
 }
 
@@ -518,6 +526,7 @@ class QAPIDomain(Domain):
 "cmd": QAPIXRefRole(),
 "event": QAPIXRefRole(),
 "enum": QAPIXRefRole(),
+"struct": QAPIXRefRole(),
 "alt": QAPIXRefRole(),
 "type": QAPIXRefRole(),  # reference any data type (excludes modules, 
commands, events)
 "obj": QAPIXRefRole(),  # reference *any* type of QAPI object.
-- 
2.44.0

[PATCH 15/27] docs/qapi-domain: add qapi:event directive

2024-04-18 Thread John Snow 
Adds the .. qapi:event:: directive, object, and :qapi:event:`name`
cross-referencing role.

Adds the :memb type name: field list syntax for documenting event data
members. As this syntax and phrasing will be shared with Structs and
Unions as well, add the field list definition to a shared abstract
class.

As per usual, QAPI cross-referencing for types in the member field list
will be added in a forthcoming commit.

NOTE 1: The "str?" type annotation syntax sneaks into this commit in the
demonstration doc. It isn't processed yet, so just ignore it for
now. The non-RFC version of this series won't include the sandbox doc,
so that inconsistency will sort itself out later. (Skip ahead to the
commit that adds XRef support for TypedField and GroupedField lists to
learn what the deal is there and leave feedback on that syntax.)

NOTE 2: We already have a QMP lexer, turn it on for example blocks in this
sample demo.

Signed-off-by: John Snow 
---
 docs/qapi/index.rst| 40 ++
 docs/sphinx/qapi-domain.py | 25 
 2 files changed, 65 insertions(+)

diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
index 9bfe4d9f454..d81bccfb06a 100644
--- a/docs/qapi/index.rst
+++ b/docs/qapi/index.rst
@@ -2,6 +2,12 @@
 QAPI Domain Test
 
 
+.. this sets the code-highlighting language to QMP for this *document*.
+   I wonder if I can set a domain default...?
+
+.. highlight:: QMP
+
+
 .. qapi:module:: foo-module
:no-index:
 
@@ -134,3 +140,37 @@ Explicit cross-referencing syntax for QAPI modules is 
available with
:choice str local: name of the bitmap, attached to the same node as
   target bitmap.
:choice BlockDirtyBitmap external: bitmap with specified node
+
+.. qapi:event:: BLOCK_JOB_COMPLETED
+   :since: 1.1
+
+   Emitted when a block job has completed.
+
+   :memb JobType type: job type
+   :memb str device: The job identifier. Originally the device name but
+  other values are allowed since QEMU 2.7
+   :memb int len: maximum progress value
+   :memb int offset: current progress value. On success this is equal to
+  len. On failure this is less than len
+   :memb int speed: rate limit, bytes per second
+   :memb str? error: error message. Only present on failure. This field
+  contains a human-readable error message. There are no semantics
+  other than that streaming has failed and clients should not try to
+  interpret the error string
+
+   Example::
+
+ <- {
+   "event": "BLOCK_JOB_COMPLETED",
+   "data": {
+ "type": "stream",
+ "device": "virtio-disk0",
+ "len": 10737418240,
+ "offset": 10737418240,
+ "speed": 0
+   },
+   "timestamp": {
+ "seconds": 1267061043,
+ "microseconds": 959568
+   }
+ }
diff --git a/docs/sphinx/qapi-domain.py b/docs/sphinx/qapi-domain.py
index c6eb6324594..74dc578b3c7 100644
--- a/docs/sphinx/qapi-domain.py
+++ b/docs/sphinx/qapi-domain.py
@@ -321,6 +321,28 @@ class QAPIAlternate(QAPIObject):
 )
 
 
+class QAPIObjectWithMembers(QAPIObject):
+"""Base class for Events/Structs/Unions"""
+
+doc_field_types = QAPIObject.doc_field_types.copy()
+doc_field_types.extend(
+[
+TypedField(
+"member",
+label=_("Members"),
+names=("memb",),
+can_collapse=True,
+),
+]
+)
+
+
+class QAPIEvent(QAPIObjectWithMembers):
+"""Description of a QAPI Event."""
+
+pass
+
+
 class QAPIModule(SphinxDirective):
 """
 Directive to mark description of a new module.
@@ -473,6 +495,7 @@ class QAPIDomain(Domain):
 object_types: Dict[str, ObjType] = {
 "module": ObjType(_("module"), "mod", "obj"),
 "command": ObjType(_("command"), "cmd", "obj"),
+"event": ObjType(_("event"), "event", "obj"),
 "enum": ObjType(_("enum"), "enum", "obj", "type"),
 "alternate": ObjType(_("alternate"), "alt", "obj", "type"),
 }
@@ -482,6 +505,7 @@ class QAPIDomain(Domain):
 directives = {
 "module": QAPIModule,
 "command": QAPICommand,
+"event": QAPIEvent,
 "enum": QAPIEnum,
 "alternate": QAPIAlternate,
 }
@@ -492,6 +516,7 @@ class QAPIDomain(Domain):
 roles = {
 "mod": QAPIXRefRole(),
 "cmd": QAPIXRefRole(),
+"event": QAPIXRefRole(),
 "enum": QAPIXRefRole(),
 "alt": QAPIXRefRole(),
 "type": QAPIXRefRole(),  # reference any data type (excludes modules, 
commands, events)
-- 
2.44.0

[PATCH 19/27] docs/qapi-domain: add :unstable: directive option

2024-04-18 Thread John Snow 
Although "unstable" is a feature (and *will* appear in the features
list), add a special :unstable: option to generate an eye-catch that
makes this information very hard to miss.

(The intent is to modify qapidoc.py to add this option whenever it
detects that the features list attached to a definition contains the
"unstable" entry.)

RFC: Same comments as last patch.

Signed-off-by: Harmonie Snow 
Signed-off-by: John Snow 
---
 docs/qapi/index.rst| 4 +++-
 docs/sphinx-static/theme_overrides.css | 6 +-
 docs/sphinx/qapi-domain.py | 8 
 3 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
index b9a69f6bd17..6350791a3ed 100644
--- a/docs/qapi/index.rst
+++ b/docs/qapi/index.rst
@@ -96,13 +96,13 @@ Explicit cross-referencing syntax for QAPI modules is 
available with
 .. qapi:command:: fake-command
:since: 13.37
:deprecated:
+   :unstable:
 
This is a fake command, it's not real. It can't hurt you.
 
:arg int foo: normal parameter documentation.
:arg str bar: Another normal parameter description.
:arg baz: Missing a type.
-   :feat unstable: More than unstable, this command doesn't even exist!
:arg no-descr:
:arg BitmapSyncMode discrim: How about branches in commands?
 
@@ -120,6 +120,8 @@ Explicit cross-referencing syntax for QAPI modules is 
available with
:feat deprecated: Although this command is fake, you should know that
   it's also deprecated. That's great news! Maybe it will go away and
   stop haunting you someday.
+   :feat unstable: This command, as a figment of your imagination, is
+  highly unstable and should not be relied upon.
:error CommandNotFound: When you try to use this command, because it
   isn't real.
:error GenericError: If the system decides it doesn't like the
diff --git a/docs/sphinx-static/theme_overrides.css 
b/docs/sphinx-static/theme_overrides.css
index 97b8c1c60e6..acdf11675db 100644
--- a/docs/sphinx-static/theme_overrides.css
+++ b/docs/sphinx-static/theme_overrides.css
@@ -172,7 +172,7 @@ div[class^="highlight"] pre {
 margin: 0.25em;
 }
 
-.qapi-deprecated {
+.qapi-deprecated,.qapi-unstable {
 background-color: #fffef5;
 border: solid #fff176 6px;
 font-weight: bold;
@@ -181,6 +181,10 @@ div[class^="highlight"] pre {
 margin: 5px;
 }
 
+.qapi-unstable::before {
+content: ' ';
+}
+
 .qapi-deprecated::before {
 content: '⚠️ ';
 }
diff --git a/docs/sphinx/qapi-domain.py b/docs/sphinx/qapi-domain.py
index 065ad501960..76157476e02 100644
--- a/docs/sphinx/qapi-domain.py
+++ b/docs/sphinx/qapi-domain.py
@@ -151,6 +151,7 @@ class QAPIObject(ObjectDescription[Signature]):
 # These are QAPI originals:
 "since": since_validator,
 "deprecated": directives.flag,
+"unstable": directives.flag,
 }
 )
 
@@ -269,6 +270,13 @@ def _add_pip(source: str, content: str, classname: str) -> 
None:
 "qapi-deprecated",
 )
 
+if "unstable" in self.options:
+_add_pip(
+":unstable:",
+f"This {self.objtype} is unstable/experimental.",
+"qapi-unstable",
+)
+
 if infopips.children:
 contentnode.insert(0, infopips)
 
-- 
2.44.0

[PATCH 10/27] docs/qapi-domain: add "Features:" field lists

2024-04-18 Thread John Snow 
Add support for Features field lists. There is no QAPI-specific
functionality here, but this could be changed if desired (if we wanted
the feature names to link somewhere, for instance.)

This feature list doesn't have any restrictions, so it can be used to
document object-wide features or per-member features as deemed
appropriate. It's essentially free-form text.

Signed-off-by: John Snow 
---
 docs/qapi/index.rst|  6 ++
 docs/sphinx/qapi-domain.py | 11 ++-
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
index 197587bbc81..a570c37abb2 100644
--- a/docs/qapi/index.rst
+++ b/docs/qapi/index.rst
@@ -95,9 +95,15 @@ Explicit cross-referencing syntax for QAPI modules is 
available with
:arg int foo: normal parameter documentation.
:arg str bar: Another normal parameter description.
:arg baz: Missing a type.
+   :feat unstable: More than unstable, this command doesn't even exist!
:arg no-descr:
+   :feat hallucination: This command is a figment of your imagination.
 
Field lists can appear anywhere in the directive block, but any field
list entries in the same list block that are recognized as special
("arg") will be reformatted and grouped accordingly for rendered
output.
+
+   At the moment, the order of grouped sections is based on the order in
+   which each group was encountered. This example will render Arguments
+   first, and then Features; but the order can be any that you choose.
diff --git a/docs/sphinx/qapi-domain.py b/docs/sphinx/qapi-domain.py
index 853bd91b7a8..c0dc6482204 100644
--- a/docs/sphinx/qapi-domain.py
+++ b/docs/sphinx/qapi-domain.py
@@ -33,7 +33,7 @@
 from sphinx.locale import _, __
 from sphinx.roles import XRefRole
 from sphinx.util import logging
-from sphinx.util.docfields import TypedField
+from sphinx.util.docfields import GroupedField, TypedField
 from sphinx.util.docutils import SphinxDirective, switch_source_input
 from sphinx.util.nodes import (
 make_id,
@@ -146,6 +146,15 @@ class QAPIObject(ObjectDescription[Signature]):
 }
 )
 
+doc_field_types = [
+GroupedField(
+"feature",
+label=_("Features"),
+names=("feat",),
+can_collapse=True,
+),
+]
+
 def get_signature_prefix(self, sig: str) -> List[nodes.Node]:
 """Returns a prefix to put before the object name in the signature."""
 assert self.objtype
-- 
2.44.0

[PATCH 06/27] docs/qapi-domain: add QAPI xref roles

2024-04-18 Thread John Snow 
Add domain-specific cross-reference syntax. As of this commit, that
means new :qapi:mod:`block-core` and :qapi:obj:`block-core` referencing
syntax.

:mod: will only find modules, but :obj: will find anything registered to
the QAPI domain. (In forthcoming commits, this means commands, events,
enums, etc.)

Creating the cross-references is powered by the QAPIXRefRole class;
resolving them is handled by QAPIDomain.resolve_xref().

QAPIXrefRole is copied almost verbatim from Sphinx's own
PyXrefRole. PyXrefRole (and QAPIXrefRole) adds two features over the
base class:

(1) Creating a cross-reference with e.g. :py:class:`~class.name`
instructs sphinx to omit the fully qualified parts of the resolved name
from the actual link text. This may be useful in the future if we add
namespaces to QAPI documentation, e.g. :qapi:cmd:`~qsd.blockdev-backup`
could link to the QSD-specific documentation for blockdev-backup while
omitting that prefix from the link text.

(2) Prefixing the link target with a "." changes the search behavior to
prefer locally-scoped items first.

I think both of these are worth keeping to help manage future namespace
issues between QEMU, QSD and QGA; but it's possible it's extraneous. It
may possibly be worth keeping just to keep feature parity with Sphinx's
other domains; e.g. "principle of least surprise". Dunno.

Signed-off-by: John Snow 
---
 docs/qapi/index.rst|  4 +++
 docs/sphinx/qapi-domain.py | 67 +-
 2 files changed, 70 insertions(+), 1 deletion(-)

diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
index 39ad405fd93..e2223d5f363 100644
--- a/docs/qapi/index.rst
+++ b/docs/qapi/index.rst
@@ -47,3 +47,7 @@ cross-referencing syntax. Here's a link to `bar-module`, even 
though
 the actual output of that directive was suppressed. Here's a link to
 `block-core`. A link to ```foo-module``` won't resolve because of the
 ``:no-index:`` option we used for that directive.
+
+Explicit cross-referencing syntax for QAPI modules is available with
+``:qapi:mod:`foo```, here's a link to :qapi:mod:`bar-module` and one to
+:qapi:mod:`block-core`.
diff --git a/docs/sphinx/qapi-domain.py b/docs/sphinx/qapi-domain.py
index 4758451ff0e..d28ac1cb9d8 100644
--- a/docs/sphinx/qapi-domain.py
+++ b/docs/sphinx/qapi-domain.py
@@ -29,6 +29,7 @@
 ObjType,
 )
 from sphinx.locale import _, __
+from sphinx.roles import XRefRole
 from sphinx.util import logging
 from sphinx.util.docutils import SphinxDirective, switch_source_input
 from sphinx.util.nodes import (
@@ -56,6 +57,34 @@ class ObjectEntry(NamedTuple):
 aliased: bool
 
 
+class QAPIXRefRole(XRefRole):
+def process_link(
+self,
+env: BuildEnvironment,
+refnode: Element,
+has_explicit_title: bool,
+title: str,
+target: str,
+) -> tuple[str, str]:
+refnode["qapi:module"] = env.ref_context.get("qapi:module")
+if not has_explicit_title:
+title = title.lstrip(".")  # only has a meaning for the target
+target = target.lstrip("~")  # only has a meaning for the title
+# if the first character is a tilde, don't display the module
+# parts of the contents
+if title[0:1] == "~":
+title = title[1:]
+dot = title.rfind(".")
+if dot != -1:
+title = title[dot + 1 :]
+# if the first character is a dot, search more specific namespaces 
first
+# else search builtins first
+if target[0:1] == ".":
+target = target[1:]
+refnode["refspecific"] = True
+return title, target
+
+
 def _nested_parse(directive: SphinxDirective, content_node: Element) -> None:
 """
 This helper preserves error parsing context across sphinx versions.
@@ -234,7 +263,13 @@ class QAPIDomain(Domain):
 "module": QAPIModule,
 }
 
-roles = {}
+# These are all cross-reference roles; e.g.
+# :qapi:cmd:`query-block`. The keys correlate to the names used in
+# the object_types table values above.
+roles = {
+"mod": QAPIXRefRole(),
+"obj": QAPIXRefRole(),  # reference *any* type of QAPI object.
+}
 
 # Moved into the data property at runtime;
 # this is the internal index of reference-able objects.
@@ -363,6 +398,36 @@ def find_obj(
 matches = [m for m in matches if not m[1].aliased]
 return matches
 
+def resolve_xref(
+self,
+env: BuildEnvironment,
+fromdocname: str,
+builder: Builder,
+type: str,
+target: str,
+node: pending_xref,
+contnode: Element,
+) -> Element | None:
+modname = node.get("qapi:module")
+matches = self.find_obj(modname, target, type)
+multiple_matches = len(matches) > 1
+
+if not matches:
+return None
+elif multiple_matches:
+logger.warning(
+__("more than

[PATCH 27/27] docs/qapi-domain: add CSS styling

2024-04-18 Thread John Snow 
From: Harmonie Snow 

Improve the general look and feel of generated QAPI docs.

Attempt to limit line lengths to offer a more comfortable measure on
maximized windows, and improve some margin and spacing for field lists.

Signed-off-by: Harmonie Snow 
Signed-off-by: John Snow 
---
 docs/sphinx-static/theme_overrides.css | 50 --
 1 file changed, 48 insertions(+), 2 deletions(-)

diff --git a/docs/sphinx-static/theme_overrides.css 
b/docs/sphinx-static/theme_overrides.css
index b239a762a9e..9a58cf86b5c 100644
--- a/docs/sphinx-static/theme_overrides.css
+++ b/docs/sphinx-static/theme_overrides.css
@@ -18,8 +18,8 @@ h1, h2, .rst-content .toctree-wrapper p.caption, h3, h4, h5, 
h6, legend {
 
 .rst-content dl:not(.docutils) dt {
 border-top: none;
-border-left: solid 3px #ccc;
-background-color: #f0f0f0;
+border-left: solid 5px #bcc6d2;
+background-color: #eaedf1;
 color: black;
 }
 
@@ -162,6 +162,18 @@ div[class^="highlight"] pre {
 
 /* QAPI domain theming */
 
+/* most content in a qapi object definition should not eclipse about
+   80ch, but nested field lists are explicitly exempt due to their
+   two-column nature */
+.qapi dd *:not(dl) {
+max-width: 80ch;
+}
+
+/* but the content column itself should still be less than ~80ch. */
+.qapi .field-list dd {
+max-width: 80ch;
+}
+
 .qapi-infopips {
 margin-bottom: 1em;
 }
@@ -201,3 +213,37 @@ div[class^="highlight"] pre {
 border-radius: 15px;
 margin: 5px;
 }
+
+/* code blocks */
+.qapi div[class^="highlight"] {
+width: fit-content;
+background-color: #fffafd;
+border: 2px solid #ffe1f3;
+}
+
+/* note, warning, etc. */
+.qapi .admonition {
+width: fit-content;
+}
+
+/* pad the top of the field-list so the text doesn't start directly at
+   the top border; primarily for the field list labels, but adjust the
+   field bodies as well for parity. */
+dl.field-list > dt:first-of-type, dl.field-list > dd:first-of-type {
+padding-top: 0.3em;
+}
+
+dl.field-list > dt:last-of-type, dl.field-list > dd:last-of-type {
+padding-bottom: 0.3em;
+}
+
+/* pad the field list labels so they don't crash into the border */
+dl.field-list > dt {
+padding-left: 0.5em;
+padding-right: 0.5em;
+}
+
+/* Add a little padding between field list sections */
+dl.field-list > dd:not(:last-child) {
+padding-bottom: 1em;
+}
-- 
2.44.0

[PATCH 23/27] docs/qapi-domain: RFC patch - delete malformed field lists

2024-04-18 Thread John Snow 
Cleanup of the last patch to fix the build before closing out this RFC
series.

Signed-off-by: John Snow 
---
 docs/qapi/index.rst | 4 
 1 file changed, 4 deletions(-)

diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
index ef58dfc4bcd..8352a27d4a5 100644
--- a/docs/qapi/index.rst
+++ b/docs/qapi/index.rst
@@ -129,10 +129,6 @@ Explicit cross-referencing syntax for QAPI modules is 
available with
   argument values. It's very temperamental.
:return SomeTypeName: An esoteric collection of mystical nonsense to
   both confound and delight.
-   :arg: this is malformed.
-   :memb: this is malformed and unrecognized.
-   :choice type name: This is unrecognized.
-   :errors FooError: Also malformed.
:example: This isn't a "semantic" field, but it's been added to the
   allowed field names list. you can use whatever field names you'd
   like; but to prevent accidental typos, there is an allow list of
-- 
2.44.0

[PATCH 01/27] docs/sphinx: create QAPI domain extension stub

2024-04-18 Thread John Snow 
It doesn't really do anything yet, we'll get to it brick-by-brick in the
forthcoming commits to keep the series breezy and the git history
informative.

Signed-off-by: John Snow 
---
 docs/conf.py   |  3 ++-
 docs/sphinx/qapi-domain.py | 50 ++
 2 files changed, 52 insertions(+), 1 deletion(-)
 create mode 100644 docs/sphinx/qapi-domain.py

diff --git a/docs/conf.py b/docs/conf.py
index aae0304ac6e..b15665d956d 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -61,7 +61,8 @@
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-extensions = ['kerneldoc', 'qmp_lexer', 'hxtool', 'depfile', 'qapidoc']
+extensions = ['kerneldoc', 'hxtool', 'depfile',
+  'qapidoc', 'qapi-domain', 'qmp_lexer']
 
 if sphinx.version_info[:3] > (4, 0, 0):
 tags.add('sphinx4')
diff --git a/docs/sphinx/qapi-domain.py b/docs/sphinx/qapi-domain.py
new file mode 100644
index 000..163b9ff21c3
--- /dev/null
+++ b/docs/sphinx/qapi-domain.py
@@ -0,0 +1,50 @@
+"""
+QAPI domain extension.
+"""
+
+from __future__ import annotations
+
+from typing import (
+TYPE_CHECKING,
+Any,
+Dict,
+List,
+Tuple,
+)
+
+from sphinx.domains import Domain, ObjType
+from sphinx.util import logging
+
+
+if TYPE_CHECKING:
+from sphinx.application import Sphinx
+
+logger = logging.getLogger(__name__)
+
+
+class QAPIDomain(Domain):
+"""QAPI language domain."""
+
+name = "qapi"
+label = "QAPI"
+
+object_types: Dict[str, ObjType] = {}
+directives = {}
+roles = {}
+initial_data: Dict[str, Dict[str, Tuple[Any]]] = {}
+indices = []
+
+def merge_domaindata(self, docnames: List[str], otherdata: Dict[str, Any]) 
-> None:
+pass
+
+
+def setup(app: Sphinx) -> Dict[str, Any]:
+app.setup_extension("sphinx.directives")
+app.add_domain(QAPIDomain)
+
+return {
+"version": "1.0",
+"env_version": 1,
+"parallel_read_safe": True,
+"parallel_write_safe": True,
+}
-- 
2.44.0

[PATCH 22/27] docs/qapi-domain: add warnings for malformed field lists

2024-04-18 Thread John Snow 
Normally, Sphinx will silently fall back to its standard field list
processing if it doesn't match one of your defined fields. A lot of the
time, that's not what we want - we want to be warned if we goof
something up.

For instance, the canonical argument field list form is:

:arg type name: descr

This form is captured by Sphinx and transformed so that the field label
will become "Arguments:". It's possible to omit the type name and descr
and still have it be processed correctly. However, if you omit the type
name, Sphinx no longer recognizes it:

:arg: this is not recognized.

This will turn into an arbitrary field list entry whose label is "Arg:",
and it otherwise silently fails. You may also see failures for doing
things like using :values: instead of :value:, or :errors: instead of
:error:, and so on. It's also case sensitive, and easy to trip up.

Add a validator that guarantees all field list entries that are the
direct child of an ObjectDescription use only recognized forms of field
lists, and emit a warning (treated as error by default in most build
configurations) whenever we detect one that is goofed up.

However, there's still benefit to allowing arbitrary fields -- they are
after all not a Sphinx invention, but perfectly normal docutils
syntax. Create an allow list for known spellings we don't mind letting
through, but warn against anything else.

-

RFC: Yes, this patch breaks the build! I thought it'd be helpful for you
to see the error checking in action. The next commit drops the erroneous
fields.

Signed-off-by: John Snow 
---
 docs/conf.py   | 12 
 docs/qapi/index.rst| 27 +
 docs/sphinx/qapi-domain.py | 59 ++
 3 files changed, 98 insertions(+)

diff --git a/docs/conf.py b/docs/conf.py
index b15665d956d..7a7d7005780 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -148,6 +148,18 @@
 with open(os.path.join(qemu_docdir, 'defs.rst.inc')) as f:
 rst_epilog += f.read()
 
+
+# Normally, the QAPI domain is picky about what field lists you use to
+# describe a QAPI entity. If you'd like to use arbitrary additional
+# fields in source documentation, add them here.
+qapi_allowed_fields = {
+"example",
+"note",
+"see also",
+"TODO",
+}
+
+
 # -- Options for HTML output --
 
 # The theme to use for HTML and HTML Help pages.  See the documentation for
diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
index 5bb1c37a5ed..ef58dfc4bcd 100644
--- a/docs/qapi/index.rst
+++ b/docs/qapi/index.rst
@@ -133,6 +133,33 @@ Explicit cross-referencing syntax for QAPI modules is 
available with
:memb: this is malformed and unrecognized.
:choice type name: This is unrecognized.
:errors FooError: Also malformed.
+   :example: This isn't a "semantic" field, but it's been added to the
+  allowed field names list. you can use whatever field names you'd
+  like; but to prevent accidental typos, there is an allow list of
+  "arbitrary" section names.
+
+  You can nestle code-blocks in here, too, by using the ``::``
+  syntax::
+
+ -> { [ "bidirectional QMP example" ] }
+ <- { [ "hello world!"] }
+
+  Or use explicit ``.. code-block:: QMP`` syntax, but it must start
+  on its own line with a blank line both before and after the
+  directive to render correctly:
+
+  .. code-block:: QMP
+
+ -> "Hello friend!"
+
+  Note that the QMP highlighter is merely garden-variety JSON, but
+  with the addition of ``->``, ``<-`` and ``...`` symbols to help
+  denote bidirectionality and elided segments. Eduardo Habkost and I
+  wrote this lexer many moons ago to support the
+  :doc:`/interop/bitmaps` documentation.
+   :see also: This is also not a "semantic" field. The only limit is
+  your imagination and what you can convince others to let you check
+  into conf.py.
 
Field lists can appear anywhere in the directive block, but any field
list entries in the same list block that are recognized as special
diff --git a/docs/sphinx/qapi-domain.py b/docs/sphinx/qapi-domain.py
index e44db10488f..bf8bb933345 100644
--- a/docs/sphinx/qapi-domain.py
+++ b/docs/sphinx/qapi-domain.py
@@ -334,10 +334,63 @@ def _merge_adjoining_field_lists(self, contentnode: 
addnodes.desc_content) -> No
 for child in delete_queue:
 contentnode.remove(child)
 
+def _validate_field(self, field: nodes.field) -> None:
+field_name = field.children[0]
+assert isinstance(field_name, nodes.field_name)
+allowed_fields = set(self.env.app.config.qapi_allowed_fields)
+
+field_label = field_name.astext()
+if re.match(r"\[\S+ = \S+\]", field_label) or field_label in 
allowed_fields:
+# okie-dokey. branch entry or known good allowed name.
+return
+
+try:
+# split into field type and argument
+field_type,

[PATCH 09/27] docs/qapi-domain: add "Arguments:" field lists

2024-04-18 Thread John Snow 
This adds special rendering for Sphinx's typed field lists.

This patch does not add any QAPI-aware markup, rendering, or
cross-referencing for the type names, yet. That feature requires a
subclass to TypedField which will happen in its own commit quite a bit
later in this series; after all the basic fields and objects have been
established first.

The syntax for this field is:

:arg type name: description
   description cont'd

You can omit the type or the description, but you cannot omit the name
-- if you do so, it degenerates into a "normal field list" entry, and
probably isn't what you want.

Signed-off-by: John Snow 
---
 docs/qapi/index.rst| 15 +++
 docs/sphinx/qapi-domain.py | 14 --
 2 files changed, 27 insertions(+), 2 deletions(-)

diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
index 33b9349a3ee..197587bbc81 100644
--- a/docs/qapi/index.rst
+++ b/docs/qapi/index.rst
@@ -86,3 +86,18 @@ Explicit cross-referencing syntax for QAPI modules is 
available with
* ``:no-index-entry:`` or ``:noindexentry:``
* ``:no-contents-entry:`` or ``:nocontentsentry:``
* ``:no-typesetting:``
+
+.. qapi:command:: fake-command
+   :since: 13.37
+
+   This is a fake command, it's not real. It can't hurt you.
+
+   :arg int foo: normal parameter documentation.
+   :arg str bar: Another normal parameter description.
+   :arg baz: Missing a type.
+   :arg no-descr:
+
+   Field lists can appear anywhere in the directive block, but any field
+   list entries in the same list block that are recognized as special
+   ("arg") will be reformatted and grouped accordingly for rendered
+   output.
diff --git a/docs/sphinx/qapi-domain.py b/docs/sphinx/qapi-domain.py
index 38a50318d08..853bd91b7a8 100644
--- a/docs/sphinx/qapi-domain.py
+++ b/docs/sphinx/qapi-domain.py
@@ -33,6 +33,7 @@
 from sphinx.locale import _, __
 from sphinx.roles import XRefRole
 from sphinx.util import logging
+from sphinx.util.docfields import TypedField
 from sphinx.util.docutils import SphinxDirective, switch_source_input
 from sphinx.util.nodes import (
 make_id,
@@ -254,8 +255,17 @@ def _toc_entry_name(self, sig_node: desc_signature) -> str:
 class QAPICommand(QAPIObject):
 """Description of a QAPI Command."""
 
-# Nothing unique for now! Changed in later commits O:-)
-pass
+doc_field_types = QAPIObject.doc_field_types.copy()
+doc_field_types.extend(
+[
+TypedField(
+"argument",
+label=_("Arguments"),
+names=("arg",),
+can_collapse=True,
+),
+]
+)
 
 
 class QAPIModule(SphinxDirective):
-- 
2.44.0

[PATCH 26/27] docs/qapi-domain: RFC patch - Add one last sample command

2024-04-18 Thread John Snow 
Just to have a bit more to look at in the generated doc, here's a fairly
complex command with a lot of bells and whistles.

Signed-off-by: John Snow 
---
 docs/qapi/index.rst | 67 +
 1 file changed, 67 insertions(+)

diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
index 6e85ea5280d..4562e755d21 100644
--- a/docs/qapi/index.rst
+++ b/docs/qapi/index.rst
@@ -267,6 +267,73 @@ Explicit cross-referencing syntax for QAPI modules is 
available with
   :memb str key-secret: ID of a QCryptoSecret object providing a
  passphrase for unlocking the encryption
 
+.. qapi:command:: blockdev-backup
+   :since: 4.2
+
+   Start a point-in-time copy of a block device to a new
+   destination. The status of ongoing blockdev-backup operations can be
+   checked with query-block-jobs where the BlockJobInfo.type field has
+   the value ‘backup’. The operation can be stopped before it has
+   completed using the block-job-cancel command.
+
+   :arg str target:
+  the device name or node-name of the backup target node.
+   :arg str? job-id:
+  identifier for the newly-created block job. If omitted, the device
+  name will be used. (Since 2.7)
+   :arg str device:
+  the device name or node-name of a root node which should be copied.
+   :arg MirrorSyncMode sync:
+  what parts of the disk image should be copied to the destination
+  (all the disk, only the sectors allocated in the topmost image,
+  from a dirty bitmap, or only new I/O).
+   :arg int? speed:
+  the maximum speed, in bytes per second. The default is 0, for unlimited.
+   :arg str? bitmap:
+  The name of a dirty bitmap to use. Must be present if sync is
+  ``bitmap`` or ``incremental``. Can be present if sync is ``full`` or
+  ``top``. Must not be present otherwise. (Since 2.4 (drive-backup),
+  3.1 (blockdev-backup))
+   :arg BitmapSyncMode? bitmap-mode:
+  Specifies the type of data the bitmap should contain after the
+  operation concludes. Must be present if a bitmap was provided,
+  Must NOT be present otherwise. (Since 4.2)
+   :arg bool? compress:
+  true to compress data, if the target format supports it. (default:
+  false) (since 2.8)
+   :arg BlockdevOnError? on-source-error:
+  the action to take on an error on the source, default
+  ``report``. ``stop`` and ``enospc`` can only be used if the block device
+  supports io-status (see BlockInfo).
+   :arg BlockdevOnError? on-target-error:
+  the action to take on an error on the target, default ``report`` (no
+  limitations, since this applies to a different block device than
+  device).
+   :arg bool? auto-finalize:
+  When false, this job will wait in a ``PENDING`` state after it has
+  finished its work, waiting for block-job-finalize before making
+  any block graph changes. When true, this job will automatically
+  perform its abort or commit actions. Defaults to true. (Since
+  2.12)
+   :arg bool? auto-dismiss:
+  When false, this job will wait in a ``CONCLUDED`` state after it has
+  completely ceased all work, and awaits block-job-dismiss. When
+  true, this job will automatically disappear from the query list
+  without user intervention. Defaults to true. (Since 2.12)
+   :arg str? filter-node-name:
+  the node name that should be assigned to the filter driver that
+  the backup job inserts into the graph above node specified by
+  drive. If this option is not given, a node name is
+  autogenerated. (Since: 4.2)
+   :arg BackupPerf? x-perf:
+  Performance options. (Since 6.0)
+   :feat unstable: Member ``x-perf`` is experimental.
+   :error DeviceNotFound: if ``device`` is not a valid block device.
+
+   .. note:: ``on-source-error`` and ``on-target-error only`` affect
+ background I/O. If an error occurs during a guest write
+ request, the device's rerror/werror actions will be used.
+
 .. qapi:command:: x-debug-query-block-graph
:since: 4.0
:unstable:
-- 
2.44.0

[PATCH 03/27] docs/qapi-module: add QAPI domain object registry

2024-04-18 Thread John Snow 
This is the first step towards QAPI domain cross-references and a QAPI
reference index.

For now, just create the object registry and amend the qapi:module
directive to use that registry. Update the merge_domaindata method now
that we have actual data we may need to merge.

RFC: Much of this code is adapted directly from sphinx.domains.python;
it is unclear to me if there ever would be a collision on merge, hence
the assertion. I haven't been able to trigger it or find better code to
use as a template, so probably I'll just leave the assertion in there.

Signed-off-by: John Snow 
---
 docs/sphinx/qapi-domain.py | 76 --
 1 file changed, 73 insertions(+), 3 deletions(-)

diff --git a/docs/sphinx/qapi-domain.py b/docs/sphinx/qapi-domain.py
index 7c5e4407bc1..ab80fd5f634 100644
--- a/docs/sphinx/qapi-domain.py
+++ b/docs/sphinx/qapi-domain.py
@@ -11,6 +11,7 @@
 Dict,
 Iterable,
 List,
+NamedTuple,
 Tuple,
 cast,
 )
@@ -20,6 +21,7 @@
 
 from sphinx import addnodes
 from sphinx.domains import Domain, ObjType
+from sphinx.locale import _, __
 from sphinx.util import logging
 from sphinx.util.docutils import SphinxDirective, switch_source_input
 from sphinx.util.nodes import make_id, nested_parse_with_titles
@@ -34,6 +36,13 @@
 logger = logging.getLogger(__name__)
 
 
+class ObjectEntry(NamedTuple):
+docname: str
+node_id: str
+objtype: str
+aliased: bool
+
+
 def _nested_parse(directive: SphinxDirective, content_node: Element) -> None:
 """
 This helper preserves error parsing context across sphinx versions.
@@ -101,6 +110,7 @@ class QAPIModule(SphinxDirective):
 }
 
 def run(self) -> List[Node]:
+domain = cast(QAPIDomain, self.env.get_domain("qapi"))
 modname = self.arguments[0].strip()
 no_index = "no-index" in self.options or "noindex" in self.options
 
@@ -113,11 +123,14 @@ def run(self) -> List[Node]:
 inode = addnodes.index(entries=[])
 
 if not no_index:
+# note module to the domain
 node_id = make_id(self.env, self.state.document, "module", modname)
 target = nodes.target("", "", ids=[node_id], ismod=True)
 self.set_source_info(target)
 self.state.document.note_explicit_target(target)
 
+domain.note_object(modname, "module", node_id, location=target)
+
 indextext = f"QAPI module; {modname}"
 inode = addnodes.index(
 entries=[
@@ -148,7 +161,12 @@ class QAPIDomain(Domain):
 name = "qapi"
 label = "QAPI"
 
-object_types: Dict[str, ObjType] = {}
+# This table associates cross-reference object types (key) with an
+# ObjType instance, which defines the valid cross-reference roles
+# for each object type.
+object_types: Dict[str, ObjType] = {
+"module": ObjType(_("module"), "mod", "obj"),
+}
 
 # Each of these provides a ReST directive,
 # e.g. .. qapi:module:: block-core
@@ -157,11 +175,63 @@ class QAPIDomain(Domain):
 }
 
 roles = {}
-initial_data: Dict[str, Dict[str, Tuple[Any]]] = {}
+
+# Moved into the data property at runtime;
+# this is the internal index of reference-able objects.
+initial_data: Dict[str, Dict[str, Tuple[Any]]] = {
+"objects": {},  # fullname -> ObjectEntry
+}
+
 indices = []
 
+@property
+def objects(self) -> Dict[str, ObjectEntry]:
+return self.data.setdefault("objects", {})  # type: 
ignore[no-any-return]
+
+def note_object(
+self,
+name: str,
+objtype: str,
+node_id: str,
+aliased: bool = False,
+location: Any = None,
+) -> None:
+"""Note a QAPI object for cross reference."""
+if name in self.objects:
+other = self.objects[name]
+if other.aliased and aliased is False:
+# The original definition found. Override it!
+pass
+elif other.aliased is False and aliased:
+# The original definition is already registered.
+return
+else:
+# duplicated
+logger.warning(
+__(
+"duplicate object description of %s, "
+"other instance in %s, use :no-index: for one of them"
+),
+name,
+other.docname,
+location=location,
+)
+self.objects[name] = ObjectEntry(self.env.docname, node_id, objtype, 
aliased)
+
+def clear_doc(self, docname: str) -> None:
+for fullname, obj in list(self.objects.items()):
+if obj.docname == docname:
+del self.objects[fullname]
+
 def merge_domaindata(self, docnames: List[str], otherdata: Dict[str, Any]) 
-> None:
-pass
+for fullname, obj in otherdata["objects"].items():
+if obj.docname

[PATCH 00/27] Add qapi-domain Sphinx extension

2024-04-18 Thread John Snow 
This series adds a new qapi-domain extension for Sphinx, which adds a
series of custom directives for documenting QAPI definitions.

GitLab CI: https://gitlab.com/jsnow/qemu/-/pipelines/1259566476

(Link to a demo HTML page at the end of this cover letter, but I want
you to read the cover letter first to explain what you're seeing.)

This adds a new QAPI Index page, cross-references for QMP commands,
events, and data types, and improves the aesthetics of the QAPI/QMP
documentation.

This series adds only the new ReST syntax, *not* the autogenerator. The
ReST syntax used in this series is, in general, not intended for anyone
to actually write by hand. This mimics how Sphinx's own autodoc
extension generates Python domain directives, which are then re-parsed
to produce the final result.

I have prototyped such a generator, but it isn't ready for inclusion
yet. (Rest assured: error context reporting is preserved down to the
line, even in generated ReST. There is no loss in usability for this
approach. It will likely either supplant qapidoc.py or heavily alter
it.) The generator requires only extremely minor changes to
scripts/qapi/parser.py to preserve nested indentation and provide more
accurate line information. It is less invasive than you may
fear. Relying on a secondary ReST parse phase eliminates much of the
complexity of qapidoc.py. Sleep soundly.

The purpose of sending this series in its current form is largely to
solicit feedback on general aesthetics, layout, and features. Sphinx is
a wily beast, and feedback at this stage will dictate how and where
certain features are implemented.

A goal for this syntax (and the generator) is to fully in-line all
command/event/object members, inherited or local, boxed or not, branched
or not. This should provide a boon to using these docs as a reference,
because users will not have to grep around the page looking for various
types, branches, or inherited members. Any arguments types will be
hyperlinked to their definition, further aiding usability. Commands can
be hotlinked from anywhere else in the manual, and will provide a
complete reference directly on the first screenful of information.

(Okay, maybe two screenfuls for commands with a ton of
arguments... There's only so much I can do!)

This RFC series includes a "sandbox" .rst document that showcases the
features of this domain by writing QAPI directives by hand; this
document will be removed from the series before final inclusion. It's
here to serve as a convenient test-bed for y'all to give feedback.

All that said, here's the sandbox document fully rendered:
https://jsnow.gitlab.io/qemu/qapi/index.html

And here's the new QAPI index page created by that sandbox document:
https://jsnow.gitlab.io/qemu/qapi-index.html

Known issues / points of interest:

- The formatting upsets checkpatch. The default line length for the
  "black" formatting tool is a little long. I'll fix it next spin.

- I did my best to preserve cross-version compatibility, but some
  problems have crept in here and there. This series may require
  Sphinx>= 4.0, like the dbus extension. Notably, the Ubuntu build fails
  on Gitlab CI currently. The next version will text against more Sphinx
  versions more rigorously. Sphinx version 5.3.0 and above should work
  perfectly.

- There's a bug in Sphinx itself that may manifest in your testing,
  concerning reported error locations. There's a patch in this series
  that fixes it, but it's later in the series. If you run into the bug
  while testing with this series, try applying that patch first.

- QAPI 'namespaces' aren't yet handled. i.e., there's nothing to
  distinguish entities between QMP, QGA and QSD yet. That feature will
  be added to a future version of this patchset (Likely when the
  generator is ready for inclusion: without it, references will clash
  and the index will gripe about duplicated entries.)

- Per-member features and ifcond are not yet accounted for; though
  definition-scoped features and ifconds are. Please feel free to
  suggest how you'd like to see those represented.

- Inlining all members means there is some ambiguity on what to do with
  doc block sections on inlined entities; features and members have an
  obvious home - body, since, and misc sections are not as obvious on
  how to handle. This will feature more prominently in the generator
  series.

- Some features could be implemented in either the QAPI domain syntax
  *or* the generator, or some combination of both. Depending on
  aesthetic feedback, this may influence where those features should be
  implemented.

- The formatting and aesthetics of branches are a little up in the air,
  see the qapi:union patch for more details.

- It's late, maybe other things. Happy Friday!

Hope you like it!

--js

Harmonie Snow (1):
  docs/qapi-domain: add CSS styling

John Snow (26):
  docs/sphinx: create QAPI domain extension stub
  docs/qapi-domain: add qapi:module directive
  docs/qapi-module: add QAPI domain

[PATCH 05/27] docs/qapi-domain: add resolve_any_xref()

2024-04-18 Thread John Snow 
Add the ability to resolve cross-references using the `any`
cross-reference syntax. Adding QAPI-specific cross-reference roles will
be added in a forthcoming commit, and will share the same find_obj()
helper.

(There's less code needed for the generic cross-reference resolver, so
it comes first in this series.)

Once again, this code is based very heavily on sphinx.domains.python.

Signed-off-by: John Snow 
---
 docs/qapi/index.rst|  7 +++
 docs/sphinx/qapi-domain.py | 95 +-
 2 files changed, 101 insertions(+), 1 deletion(-)

diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
index 051dc6b3a37..39ad405fd93 100644
--- a/docs/qapi/index.rst
+++ b/docs/qapi/index.rst
@@ -40,3 +40,10 @@ QAPI Domain Test
 
Modules will also be reported in the `qapi-index`, under the Modules
category and in the alphabetical categories that follow.
+
+
+QAPI modules can now be cross-referenced using the ```any```
+cross-referencing syntax. Here's a link to `bar-module`, even though
+the actual output of that directive was suppressed. Here's a link to
+`block-core`. A link to ```foo-module``` won't resolve because of the
+``:no-index:`` option we used for that directive.
diff --git a/docs/sphinx/qapi-domain.py b/docs/sphinx/qapi-domain.py
index 65409786119..4758451ff0e 100644
--- a/docs/sphinx/qapi-domain.py
+++ b/docs/sphinx/qapi-domain.py
@@ -21,6 +21,7 @@
 from docutils.parsers.rst import directives
 
 from sphinx import addnodes
+from sphinx.addnodes import pending_xref
 from sphinx.domains import (
 Domain,
 Index,
@@ -30,13 +31,19 @@
 from sphinx.locale import _, __
 from sphinx.util import logging
 from sphinx.util.docutils import SphinxDirective, switch_source_input
-from sphinx.util.nodes import make_id, nested_parse_with_titles
+from sphinx.util.nodes import (
+make_id,
+make_refnode,
+nested_parse_with_titles,
+)
 
 
 if TYPE_CHECKING:
 from docutils.nodes import Element, Node
 
 from sphinx.application import Sphinx
+from sphinx.builders import Builder
+from sphinx.environment import BuildEnvironment
 from sphinx.util.typing import OptionSpec
 
 logger = logging.getLogger(__name__)
@@ -289,6 +296,92 @@ def merge_domaindata(self, docnames: List[str], otherdata: 
Dict[str, Any]) -> No
 ), f"!?!? collision on merge? {fullname=} {obj=} 
{self.objects[fullname]=}"
 self.objects[fullname] = obj
 
+def find_obj(
+self, modname: str, name: str, type: Optional[str]
+) -> list[tuple[str, ObjectEntry]]:
+"""
+Find a QAPI object for "name", perhaps using the given module.
+
+Returns a list of (name, object entry) tuples.
+
+:param modname: The current module context (if any!)
+under which we are searching.
+:param name: The name of the x-ref to resolve;
+ may or may not include a leading module.
+:param type: The role name of the x-ref we're resolving, if provided.
+ (This is absent for "any" lookups.)
+"""
+if not name:
+return []
+
+names: list[str] = []
+matches: list[tuple[str, ObjectEntry]] = []
+
+fullname = name
+if "." in fullname:
+# We're searching for a fully qualified reference;
+# ignore the contextual module.
+pass
+elif modname:
+# We're searching for something from somewhere;
+# try searching the current module first.
+# e.g. :qapi:cmd:`query-block` or `query-block` is being searched.
+fullname = f"{modname}.{name}"
+
+if type is None:
+# type isn't specified, this is a generic xref.
+# search *all* qapi-specific object types.
+objtypes: Optional[List[str]] = list(self.object_types)
+else:
+# type is specified and will be a role (e.g. obj, mod, cmd)
+# convert this to eligible object types (e.g. command, module)
+# using the QAPIDomain.object_types table.
+objtypes = self.objtypes_for_role(type)
+
+# Either we should have been given no type, or the type we were
+# given should correspond to at least one real actual object
+# type.
+assert objtypes
+
+if name in self.objects and self.objects[name].objtype in objtypes:
+names = [name]
+elif fullname in self.objects and self.objects[fullname].objtype in 
objtypes:
+names = [fullname]
+else:
+# exact match wasn't found; e.g. we are searching for
+# `query-block` from a different (or no) module.
+searchname = "." + name
+names = [
+oname
+for oname in self.objects
+if oname.endswith(searchname)
+and self.objects[oname].objtype in objtypes
+]
+
+matches = [(oname,

[PATCH 12/27] docs/qapi-domain: add "Returns:" field lists

2024-04-18 Thread John Snow 
Add "Returns:" field list syntax to QAPI Commands.

Like "Arguments:" and "Errors:", the type name isn't currently processed
for cross-referencing, but this will be addressed in a forthcoming
commit.

This patch adds "errors" as a GroupedField, which means that multiple
return values can be annotated - this is only done because Sphinx does
not seemingly (Maybe I missed it?) support mandatory type arguments to
Ungrouped fields. Because we want to cross-reference this type
information later, we want to make the type argument mandatory. As a
result, you can technically add multiple :return: fields, though I'm not
aware of any circumstance in which you'd need or want
to. Recommendation: "Don't do that, then."

Since this field describes an action/event instead of describing a list
of nouns (arguments, features, errors), I added both the imperative and
indicative forms (:return: and :returns:) to allow doc writers to use
whichever mood "feels right" in the source document. The rendered output
will always use the "Returns:" label, however.

I'm sure you'll let me know how you feel about that. O:-)

Signed-off-by: John Snow 
---
 docs/qapi/index.rst| 2 ++
 docs/sphinx/qapi-domain.py | 6 ++
 2 files changed, 8 insertions(+)

diff --git a/docs/qapi/index.rst b/docs/qapi/index.rst
index 004d02e0437..39fe4dd2dae 100644
--- a/docs/qapi/index.rst
+++ b/docs/qapi/index.rst
@@ -102,6 +102,8 @@ Explicit cross-referencing syntax for QAPI modules is 
available with
   isn't real.
:error GenericError: If the system decides it doesn't like the
   argument values. It's very temperamental.
+   :return SomeTypeName: An esoteric collection of mystical nonsense to
+  both confound and delight.
 
Field lists can appear anywhere in the directive block, but any field
list entries in the same list block that are recognized as special
diff --git a/docs/sphinx/qapi-domain.py b/docs/sphinx/qapi-domain.py
index 1f0b168fa2c..5d44dba6cd3 100644
--- a/docs/sphinx/qapi-domain.py
+++ b/docs/sphinx/qapi-domain.py
@@ -279,6 +279,12 @@ class QAPICommand(QAPIObject):
 names=("error",),
 can_collapse=True,
 ),
+GroupedField(
+"returnvalue",
+label=_("Returns"),
+names=("return", "returns"),
+can_collapse=True,
+),
 ]
 )
 
-- 
2.44.0

Re: [PATCH 22/24] exec: Remove 'exec/tswap.h' from 'exec/cpu-all.h'

2024-04-18 Thread Harsh Prateek Bora 




On 4/19/24 00:55, Philippe Mathieu-Daudé wrote:

"exec/cpu-all.h" doesn't require "exec/tswap.h". Remove it,
including it in the sources when required.

Signed-off-by: Philippe Mathieu-Daudé 
---
  hw/xtensa/bootparam.h   | 1 +
  include/exec/cpu-all.h  | 1 -
  accel/tcg/translator.c  | 1 +
  hw/arm/boot.c   | 1 +
  hw/arm/npcm7xx.c| 1 +
  hw/mips/fuloong2e.c | 1 +
  hw/mips/malta.c | 1 +
  hw/ppc/sam460ex.c   | 1 +
  hw/ppc/spapr.c  | 1 +


For spapr:
Reviewed-by: Harsh Prateek Bora 


  hw/ppc/virtex_ml507.c   | 1 +
  hw/sh4/r2d.c| 1 +
  target/arm/gdbstub.c| 1 +
  target/xtensa/xtensa-semi.c | 1 +
  13 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/hw/xtensa/bootparam.h b/hw/xtensa/bootparam.h
index f57ff850bc..e1d47b503c 100644
--- a/hw/xtensa/bootparam.h
+++ b/hw/xtensa/bootparam.h
@@ -1,6 +1,7 @@
  #ifndef HW_XTENSA_BOOTPARAM_H
  #define HW_XTENSA_BOOTPARAM_H
  
+#include "exec/tswap.h"

  #include "exec/cpu-common.h"
  
  #define BP_TAG_COMMAND_LINE 0x1001  /* command line (0-terminated string)*/

diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index 554b937ddb..cfbf51822c 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -21,7 +21,6 @@
  
  #include "exec/cpu-common.h"

  #include "exec/memory.h"
-#include "exec/tswap.h"
  #include "hw/core/cpu.h"
  
  /* some important defines:

diff --git a/accel/tcg/translator.c b/accel/tcg/translator.c
index 6832e55135..85950377d9 100644
--- a/accel/tcg/translator.c
+++ b/accel/tcg/translator.c
@@ -12,6 +12,7 @@
  #include "qemu/error-report.h"
  #include "exec/exec-all.h"
  #include "exec/translator.h"
+#include "exec/tswap.h"
  #include "exec/cpu_ldst.h"
  #include "exec/plugin-gen.h"
  #include "tcg/tcg-op-common.h"
diff --git a/hw/arm/boot.c b/hw/arm/boot.c
index 84ea6a807a..93945a1a15 100644
--- a/hw/arm/boot.c
+++ b/hw/arm/boot.c
@@ -22,6 +22,7 @@
  #include "sysemu/reset.h"
  #include "hw/loader.h"
  #include "elf.h"
+#include "exec/tswap.h"
  #include "sysemu/device_tree.h"
  #include "qemu/config-file.h"
  #include "qemu/option.h"
diff --git a/hw/arm/npcm7xx.c b/hw/arm/npcm7xx.c
index cc68b5d8f1..1ef303415b 100644
--- a/hw/arm/npcm7xx.c
+++ b/hw/arm/npcm7xx.c
@@ -27,6 +27,7 @@
  #include "qemu/units.h"
  #include "sysemu/sysemu.h"
  #include "target/arm/cpu-qom.h"
+#include "exec/tswap.h"
  
  /*

   * This covers the whole MMIO space. We'll use this to catch any MMIO accesses
diff --git a/hw/mips/fuloong2e.c b/hw/mips/fuloong2e.c
index a45aac368c..1d0613a76f 100644
--- a/hw/mips/fuloong2e.c
+++ b/hw/mips/fuloong2e.c
@@ -40,6 +40,7 @@
  #include "sysemu/reset.h"
  #include "sysemu/sysemu.h"
  #include "qemu/error-report.h"
+#include "exec/tswap.h"
  
  #define ENVP_PADDR  0x2000

  #define ENVP_VADDR  cpu_mips_phys_to_kseg0(NULL, ENVP_PADDR)
diff --git a/hw/mips/malta.c b/hw/mips/malta.c
index af74008c82..3dca0f100c 100644
--- a/hw/mips/malta.c
+++ b/hw/mips/malta.c
@@ -56,6 +56,7 @@
  #include "semihosting/semihost.h"
  #include "hw/mips/cps.h"
  #include "hw/qdev-clock.h"
+#include "exec/tswap.h"
  #include "target/mips/internal.h"
  #include "trace.h"
  #include "cpu.h"
diff --git a/hw/ppc/sam460ex.c b/hw/ppc/sam460ex.c
index d42b677898..abc02f0817 100644
--- a/hw/ppc/sam460ex.c
+++ b/hw/ppc/sam460ex.c
@@ -24,6 +24,7 @@
  #include "hw/loader.h"
  #include "elf.h"
  #include "exec/memory.h"
+#include "exec/tswap.h"
  #include "ppc440.h"
  #include "hw/pci-host/ppc4xx.h"
  #include "hw/block/flash.h"
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index e9bc97fee0..b4b1f43983 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -74,6 +74,7 @@
  #include "hw/virtio/virtio-scsi.h"
  #include "hw/virtio/vhost-scsi-common.h"
  
+#include "exec/tswap.h"

  #include "exec/ram_addr.h"
  #include "hw/usb.h"
  #include "qemu/config-file.h"
diff --git a/hw/ppc/virtex_ml507.c b/hw/ppc/virtex_ml507.c
index d02f330650..fd23afebf5 100644
--- a/hw/ppc/virtex_ml507.c
+++ b/hw/ppc/virtex_ml507.c
@@ -38,6 +38,7 @@
  #include "qapi/error.h"
  #include "qemu/error-report.h"
  #include "qemu/option.h"
+#include "exec/tswap.h"
  
  #include "hw/intc/ppc-uic.h"

  #include "hw/ppc/ppc.h"
diff --git a/hw/sh4/r2d.c b/hw/sh4/r2d.c
index e5ac6751bd..5f4420f534 100644
--- a/hw/sh4/r2d.c
+++ b/hw/sh4/r2d.c
@@ -43,6 +43,7 @@
  #include "hw/loader.h"
  #include "hw/usb.h"
  #include "hw/block/flash.h"
+#include "exec/tswap.h"
  
  #define FLASH_BASE 0x

  #define FLASH_SIZE (16 * MiB)
diff --git a/target/arm/gdbstub.c b/target/arm/gdbstub.c
index a3bb73cfa7..f2b001afdd 100644
--- a/target/arm/gdbstub.c
+++ b/target/arm/gdbstub.c
@@ -20,6 +20,7 @@
  #include "qemu/osdep.h"
  #include "cpu.h"
  #include "exec/gdbstub.h"
+#include "exec/tswap.h"
  #include "gdbstub/helpers.h"
  #include "sysemu/tcg.h"
  #include "internals.h"
diff --git a/target/xtensa/xtensa-semi.c b/target/xtensa/xtensa-semi.c
index

[RFC QEMU PATCH v6 0/1] Support device passthrough when dom0 is PVH on Xen

2024-04-18 Thread Jiqian Chen 
Hi All,
This is v6 series to support passthrough on Xen when dom0 is PVH.
v5->v6 changes:
* Due to changes in the implementation of obtaining gsi in the kernel and Xen. 
Change to use xc_physdev_gsi_from_irq, instead of gsi sysfs.

Best regards,
Jiqian Chen


v4->v5 changes:
* Add review by Stefano


v3->v4 changes:
* Add gsi into struct XenHostPCIDevice and use gsi number that read from gsi 
sysfs
  if it exists, if there is no gsi sysfs, still use irq.


v2->v3 changes:
* Du to changes in the implementation of the second patch on kernel side(that 
adds
  a new sysfs for gsi instead of a new syscall), so read gsi number from the 
sysfs of gsi.


Below is the description of v2 cover letter:
This patch is the v2 of the implementation of passthrough when dom0 is PVH on 
Xen.
Issues we encountered:
1. failed to map pirq for gsi
Problem: qemu will call xc_physdev_map_pirq() to map a passthrough 
device\u2019s gsi to pirq in
function xen_pt_realize(). But failed.

Reason: According to the implement of xc_physdev_map_pirq(), it needs gsi 
instead of irq,
but qemu pass irq to it and treat irq as gsi, it is got from file
/sys/bus/pci/devices/:xx:xx.x/irq in function xen_host_pci_device_get(). 
But actually
the gsi number is not equal with irq. On PVH dom0, when it allocates irq for a 
gsi in
function acpi_register_gsi_ioapic(), allocation is dynamic, and follow the 
principle of
applying first, distributing first. And if you debug the kernel codes
(see function __irq_alloc_descs), you will find the irq number is allocated 
from small to
large by order, but the applying gsi number is not, gsi 38 may come before gsi 
28, that
causes gsi 38 get a smaller irq number than gsi 28, and then gsi != irq.

Solution: we can record the relation between gsi and irq, then when 
userspace(qemu) want
to use gsi, we can do a translation. The third patch of kernel(xen/privcmd: Add 
new syscall
to get gsi from irq) records all the relations in acpi_register_gsi_xen_pvh() 
when dom0
initialize pci devices, and provide a syscall for userspace to get the gsi from 
irq. The
third patch of xen(tools: Add new function to get gsi from irq) add a new 
function
xc_physdev_gsi_from_irq() to call the new syscall added on kernel side.
And then userspace can use that function to get gsi. Then xc_physdev_map_pirq() 
will success.

This v2 on qemu side is the same as the v1 
(qemu 
https://lore.kernel.org/xen-devel/20230312092244.451465-19-ray.hu...@amd.com/), 
just call
xc_physdev_gsi_from_irq() to get gsi from irq.

Jiqian Chen (1):
  xen/pci: get gsi from irq for passthrough devices

 hw/xen/xen-host-pci-device.c | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

-- 
2.34.1

[RFC QEMU PATCH v6 1/1] xen/pci: get gsi from irq for passthrough devices

2024-04-18 Thread Jiqian Chen 
In PVH dom0, it uses the linux local interrupt mechanism,
when it allocs irq for a gsi, it is dynamic, and follow
the principle of applying first, distributing first. And
the irq number is alloced from small to large, but the
applying gsi number is not, may gsi 38 comes before gsi
28, that causes the irq number is not equal with the gsi
number. And when passthrough a device, qemu wants to use
gsi to map pirq, xen_pt_realize->xc_physdev_map_pirq, but
the gsi number is got from file
/sys/bus/pci/devices//irq in current code, so it
will fail when mapping.

Translate irq to gsi by using new function supported by
Xen tools.

Signed-off-by: Huang Rui 
Signed-off-by: Jiqian Chen 
---
 hw/xen/xen-host-pci-device.c | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/hw/xen/xen-host-pci-device.c b/hw/xen/xen-host-pci-device.c
index 8c6e9a1716a2..5e9aa9679e3e 100644
--- a/hw/xen/xen-host-pci-device.c
+++ b/hw/xen/xen-host-pci-device.c
@@ -10,6 +10,7 @@
 #include "qapi/error.h"
 #include "qemu/cutils.h"
 #include "xen-host-pci-device.h"
+#include "hw/xen/xen_native.h"
 
 #define XEN_HOST_PCI_MAX_EXT_CAP \
 ((PCIE_CONFIG_SPACE_SIZE - PCI_CONFIG_SPACE_SIZE) / (PCI_CAP_SIZEOF + 4))
@@ -368,7 +369,11 @@ void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t 
domain,
 if (*errp) {
 goto error;
 }
-d->irq = v;
+d->irq = xc_physdev_gsi_from_irq(xen_xc, v);
+/* if fail to get gsi, fallback to irq */
+if (d->irq == -1) {
+d->irq = v;
+}
 
 xen_host_pci_get_hex_value(d, "class", , errp);
 if (*errp) {
-- 
2.34.1

回复：[PATCH] tcg: Fix the overflow in indexing tcg_ctx->temps

2024-04-18 Thread 姜智伟 
> On 4/18/24 03:27, Zhiwei Jiang wrote:
> > Sometimes, when the address of the passed TCGTemp *ts variable is the same 
> > as tcg_ctx,
> 
> Pardon?  When would TCGTemp *ts == TCGContext *tcg_ctx?
> 
> 
> > the index calculated in the temp_idx function, i.e., ts - tcg_ctx->temps,
> > can result in a particularly large value, causing overflow in the 
> > subsequent array access.
> 
> Or, assert:
> 
> size_t temp_idx(TCGTemp *ts)
> {
>      ptrdiff_t n = ts - tcg_ctx->temps;
>      assert(n >= 0 && n < tcg_ctx->nb_temps);
>      return n;
> }
> 
> >   static inline TCGTemp *tcgv_i32_temp(TCGv_i32 v)
> >   {
> > -    return (void *)tcg_ctx + (uintptr_t)v;
> > +    return (void *)tcg_ctx->temps + (uintptr_t)v;
> >   }
> 
> This will generate 0 for the first temp, which will test as NULL.

Hi Richard:
You can reproduce this issue on the latest upstream QEMU version. Using the 
RISC-V QEMU version, 
if we compile a test program with the first instruction being '.insn r 0xf, 2, 
0, x0, x0, x0',that is a RISC-V CBO instruction, 
qemu will crash with a segmentation fault upon execution.

When the first instruction in the program is a CBO instruction,  temp_idx in 
init_ts_info func returns a very large value, 
causing the subsequent test_bit function to access out-of-bounds memory.

static void init_ts_info(OptContext *ctx, TCGTemp *ts)
{
size_t idx = temp_idx(ts);
TempOptInfo *ti;

if (test_bit(idx, ctx->temps_used.l)) {
return;
}
...
I can fix this segmentation fault by applying the modification above, 
and it seems more logical in terms of code logic to match the allocation and 
indexing of TCGTemp.

Ths

[PATCH v7 04/12] hw/mem/cxl_type3: Add support to create DC regions to type3 memory devices

2024-04-18 Thread nifan . cxl 
From: Fan Ni 

With the change, when setting up memory for type3 memory device, we can
create DC regions.
A property 'num-dc-regions' is added to ct3_props to allow users to pass the
number of DC regions to create. To make it easier, other region parameters
like region base, length, and block size are hard coded. If needed,
these parameters can be added easily.

With the change, we can create DC regions with proper kernel side
support like below:

region=$(cat /sys/bus/cxl/devices/decoder0.0/create_dc_region)
echo $region > /sys/bus/cxl/devices/decoder0.0/create_dc_region
echo 256 > /sys/bus/cxl/devices/$region/interleave_granularity
echo 1 > /sys/bus/cxl/devices/$region/interleave_ways

echo "dc0" >/sys/bus/cxl/devices/decoder2.0/mode
echo 0x4000 >/sys/bus/cxl/devices/decoder2.0/dpa_size

echo 0x4000 > /sys/bus/cxl/devices/$region/size
echo  "decoder2.0" > /sys/bus/cxl/devices/$region/target0
echo 1 > /sys/bus/cxl/devices/$region/commit
echo $region > /sys/bus/cxl/drivers/cxl_region/bind

Reviewed-by: Jonathan Cameron 
Signed-off-by: Fan Ni 
---
 hw/mem/cxl_type3.c | 49 ++
 1 file changed, 49 insertions(+)

diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 5d6d3ab87d..5ceed0ab4c 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -30,6 +30,7 @@
 #include "hw/pci/msix.h"
 
 #define DWORD_BYTE 4
+#define CXL_CAPACITY_MULTIPLIER   (256 * MiB)
 
 /* Default CDAT entries for a memory region */
 enum {
@@ -567,6 +568,46 @@ static void ct3d_reg_write(void *opaque, hwaddr offset, 
uint64_t value,
 }
 }
 
+/*
+ * TODO: dc region configuration will be updated once host backend and address
+ * space support is added for DCD.
+ */
+static bool cxl_create_dc_regions(CXLType3Dev *ct3d, Error **errp)
+{
+int i;
+uint64_t region_base = 0;
+uint64_t region_len =  2 * GiB;
+uint64_t decode_len = 2 * GiB;
+uint64_t blk_size = 2 * MiB;
+CXLDCRegion *region;
+MemoryRegion *mr;
+
+if (ct3d->hostvmem) {
+mr = host_memory_backend_get_memory(ct3d->hostvmem);
+region_base += memory_region_size(mr);
+}
+if (ct3d->hostpmem) {
+mr = host_memory_backend_get_memory(ct3d->hostpmem);
+region_base += memory_region_size(mr);
+}
+assert(region_base % CXL_CAPACITY_MULTIPLIER == 0);
+
+for (i = 0, region = >dc.regions[0];
+ i < ct3d->dc.num_regions;
+ i++, region++, region_base += region_len) {
+*region = (CXLDCRegion) {
+.base = region_base,
+.decode_len = decode_len,
+.len = region_len,
+.block_size = blk_size,
+/* dsmad_handle set when creating CDAT table entries */
+.flags = 0,
+};
+}
+
+return true;
+}
+
 static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
 {
 DeviceState *ds = DEVICE(ct3d);
@@ -635,6 +676,13 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error 
**errp)
 g_free(p_name);
 }
 
+if (ct3d->dc.num_regions > 0) {
+if (!cxl_create_dc_regions(ct3d, errp)) {
+error_setg(errp, "setup DC regions failed");
+return false;
+}
+}
+
 return true;
 }
 
@@ -931,6 +979,7 @@ static Property ct3_props[] = {
  HostMemoryBackend *),
 DEFINE_PROP_UINT64("sn", CXLType3Dev, sn, UI64_NULL),
 DEFINE_PROP_STRING("cdat", CXLType3Dev, cxl_cstate.cdat.filename),
+DEFINE_PROP_UINT8("num-dc-regions", CXLType3Dev, dc.num_regions, 0),
 DEFINE_PROP_END_OF_LIST(),
 };
 
-- 
2.43.0

[PATCH v7 10/12] hw/mem/cxl_type3: Add DPA range validation for accesses to DC regions

2024-04-18 Thread nifan . cxl 
From: Fan Ni 

All DPA ranges in the DC regions are invalid to access until an extent
covering the range has been successfully accepted by the host. A bitmap
is added to each region to record whether a DC block in the region has
been backed by a DC extent. Each bit in the bitmap represents a DC block.
When a DC extent is accepted, all the bits representing the blocks in the
extent are set, which will be cleared when the extent is released.

Reviewed-by: Jonathan Cameron 
Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c  |  3 ++
 hw/mem/cxl_type3.c  | 76 +
 include/hw/cxl/cxl_device.h |  7 
 3 files changed, 86 insertions(+)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 3569902e9e..57f1ce9cce 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -1655,6 +1655,7 @@ static CXLRetCode cmd_dcd_add_dyn_cap_rsp(const struct 
cxl_cmd *cmd,
 
 cxl_insert_extent_to_extent_list(extent_list, dpa, len, NULL, 0);
 ct3d->dc.total_extent_count += 1;
+ct3_set_region_block_backed(ct3d, dpa, len);
 }
 /* Remove the first extent group in the pending list*/
 cxl_extent_group_list_delete_front(>dc.extents_pending);
@@ -1813,10 +1814,12 @@ static CXLRetCode cmd_dcd_release_dyn_cap(const struct 
cxl_cmd *cmd,
  * list and update the extent count;
  */
 QTAILQ_FOREACH_SAFE(ent, >dc.extents, node, ent_next) {
+ct3_clear_region_block_backed(ct3d, ent->start_dpa, ent->len);
 cxl_remove_extent_from_extent_list(>dc.extents, ent);
 }
 copy_extent_list(>dc.extents, _list);
 QTAILQ_FOREACH_SAFE(ent, _list, node, ent_next) {
+ct3_set_region_block_backed(ct3d, ent->start_dpa, ent->len);
 cxl_remove_extent_from_extent_list(_list, ent);
 }
 ct3d->dc.total_extent_count = updated_list_size;
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index e892b3de7b..a3e1a5de25 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -665,6 +665,7 @@ static bool cxl_create_dc_regions(CXLType3Dev *ct3d, Error 
**errp)
 .flags = 0,
 };
 ct3d->dc.total_capacity += region->len;
+region->blk_bitmap = bitmap_new(region->len / region->block_size);
 }
 QTAILQ_INIT(>dc.extents);
 QTAILQ_INIT(>dc.extents_pending);
@@ -676,6 +677,8 @@ static void cxl_destroy_dc_regions(CXLType3Dev *ct3d)
 {
 CXLDCExtent *ent, *ent_next;
 CXLDCExtentGroup *group, *group_next;
+int i;
+CXLDCRegion *region;
 
 QTAILQ_FOREACH_SAFE(ent, >dc.extents, node, ent_next) {
 cxl_remove_extent_from_extent_list(>dc.extents, ent);
@@ -688,6 +691,11 @@ static void cxl_destroy_dc_regions(CXLType3Dev *ct3d)
 }
 g_free(group);
 }
+
+for (i = 0; i < ct3d->dc.num_regions; i++) {
+region = >dc.regions[i];
+g_free(region->blk_bitmap);
+}
 }
 
 static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
@@ -920,6 +928,70 @@ static void ct3_exit(PCIDevice *pci_dev)
 }
 }
 
+/*
+ * Mark the DPA range [dpa, dap + len - 1] to be backed and accessible. This
+ * happens when a DC extent is added and accepted by the host.
+ */
+void ct3_set_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa,
+ uint64_t len)
+{
+CXLDCRegion *region;
+
+region = cxl_find_dc_region(ct3d, dpa, len);
+if (!region) {
+return;
+}
+
+bitmap_set(region->blk_bitmap, (dpa - region->base) / region->block_size,
+   len / region->block_size);
+}
+
+/*
+ * Check whether the DPA range [dpa, dpa + len - 1] is backed with DC extents.
+ * Used when validating read/write to dc regions
+ */
+bool ct3_test_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa,
+  uint64_t len)
+{
+CXLDCRegion *region;
+uint64_t nbits;
+long nr;
+
+region = cxl_find_dc_region(ct3d, dpa, len);
+if (!region) {
+return false;
+}
+
+nr = (dpa - region->base) / region->block_size;
+nbits = DIV_ROUND_UP(len, region->block_size);
+/*
+ * if bits between [dpa, dpa + len) are all 1s, meaning the DPA range is
+ * backed with DC extents, return true; else return false.
+ */
+return find_next_zero_bit(region->blk_bitmap, nr + nbits, nr) == nr + 
nbits;
+}
+
+/*
+ * Mark the DPA range [dpa, dap + len - 1] to be unbacked and inaccessible.
+ * This happens when a dc extent is released by the host.
+ */
+void ct3_clear_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa,
+   uint64_t len)
+{
+CXLDCRegion *region;
+uint64_t nbits;
+long nr;
+
+region = cxl_find_dc_region(ct3d, dpa, len);
+if (!region) {
+return;
+}
+
+nr = (dpa - region->base) / region->block_size;
+nbits = len / region->block_size;
+bitmap_clear(region->blk_bitmap, nr, nbits);
+}
+
 static bool cxl_type3_dpa(CXLType3Dev *ct3d, hwaddr host_addr,

[PATCH v7 05/12] hw/mem/cxl-type3: Refactor ct3_build_cdat_entries_for_mr to take mr size instead of mr as argument

2024-04-18 Thread nifan . cxl 
From: Fan Ni 

The function ct3_build_cdat_entries_for_mr only uses size of the passed
memory region argument, refactor the function definition to make the passed
arguments more specific.

Reviewed-by: Jonathan Cameron 
Signed-off-by: Fan Ni 
---
 hw/mem/cxl_type3.c | 15 +--
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 5ceed0ab4c..a1fe268560 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -44,7 +44,7 @@ enum {
 };
 
 static void ct3_build_cdat_entries_for_mr(CDATSubHeader **cdat_table,
-  int dsmad_handle, MemoryRegion *mr,
+  int dsmad_handle, uint64_t size,
   bool is_pmem, uint64_t dpa_base)
 {
 CDATDsmas *dsmas;
@@ -63,7 +63,7 @@ static void ct3_build_cdat_entries_for_mr(CDATSubHeader 
**cdat_table,
 .DSMADhandle = dsmad_handle,
 .flags = is_pmem ? CDAT_DSMAS_FLAG_NV : 0,
 .DPA_base = dpa_base,
-.DPA_length = memory_region_size(mr),
+.DPA_length = size,
 };
 
 /* For now, no memory side cache, plausiblish numbers */
@@ -132,7 +132,7 @@ static void ct3_build_cdat_entries_for_mr(CDATSubHeader 
**cdat_table,
  */
 .EFI_memory_type_attr = is_pmem ? 2 : 1,
 .DPA_offset = 0,
-.DPA_length = memory_region_size(mr),
+.DPA_length = size,
 };
 
 /* Header always at start of structure */
@@ -149,6 +149,7 @@ static int ct3_build_cdat_table(CDATSubHeader 
***cdat_table, void *priv)
 g_autofree CDATSubHeader **table = NULL;
 CXLType3Dev *ct3d = priv;
 MemoryRegion *volatile_mr = NULL, *nonvolatile_mr = NULL;
+uint64_t vmr_size = 0, pmr_size = 0;
 int dsmad_handle = 0;
 int cur_ent = 0;
 int len = 0;
@@ -163,6 +164,7 @@ static int ct3_build_cdat_table(CDATSubHeader 
***cdat_table, void *priv)
 return -EINVAL;
 }
 len += CT3_CDAT_NUM_ENTRIES;
+vmr_size = memory_region_size(volatile_mr);
 }
 
 if (ct3d->hostpmem) {
@@ -171,21 +173,22 @@ static int ct3_build_cdat_table(CDATSubHeader 
***cdat_table, void *priv)
 return -EINVAL;
 }
 len += CT3_CDAT_NUM_ENTRIES;
+pmr_size = memory_region_size(nonvolatile_mr);
 }
 
 table = g_malloc0(len * sizeof(*table));
 
 /* Now fill them in */
 if (volatile_mr) {
-ct3_build_cdat_entries_for_mr(table, dsmad_handle++, volatile_mr,
+ct3_build_cdat_entries_for_mr(table, dsmad_handle++, vmr_size,
   false, 0);
 cur_ent = CT3_CDAT_NUM_ENTRIES;
 }
 
 if (nonvolatile_mr) {
-uint64_t base = volatile_mr ? memory_region_size(volatile_mr) : 0;
+uint64_t base = vmr_size;
 ct3_build_cdat_entries_for_mr(&(table[cur_ent]), dsmad_handle++,
-  nonvolatile_mr, true, base);
+  pmr_size, true, base);
 cur_ent += CT3_CDAT_NUM_ENTRIES;
 }
 assert(len == cur_ent);
-- 
2.43.0

[PATCH v7 11/12] hw/cxl/cxl-mailbox-utils: Add superset extent release mailbox support

2024-04-18 Thread nifan . cxl 
From: Fan Ni 

With the change, we extend the extent release mailbox command processing
to allow more flexible release. As long as the DPA range of the extent to
release is covered by accepted extent(s) in the device, the release can be
performed.

Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c | 21 -
 1 file changed, 8 insertions(+), 13 deletions(-)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 57f1ce9cce..89f0ab8116 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -1704,6 +1704,13 @@ static CXLRetCode 
cxl_dc_extent_release_dry_run(CXLType3Dev *ct3d,
 dpa = in->updated_entries[i].start_dpa;
 len = in->updated_entries[i].len;
 
+/* Check if the DPA range is not fully backed with valid extents */
+if (!ct3_test_region_block_backed(ct3d, dpa, len)) {
+ret = CXL_MBOX_INVALID_PA;
+goto free_and_exit;
+}
+
+/* After this point, extent overflow is the only error can happen */
 while (len > 0) {
 QTAILQ_FOREACH(ent, updated_list, node) {
 range_init_nofail(, ent->start_dpa, ent->len);
@@ -1718,14 +1725,7 @@ static CXLRetCode 
cxl_dc_extent_release_dry_run(CXLType3Dev *ct3d,
 if (range_contains(, dpa + len - 1)) {
 len2 = ent_start_dpa + ent_len - dpa - len;
 } else {
-/*
- * TODO: we reject the attempt to remove an extent
- * that overlaps with multiple extents in the device
- * for now. We will allow it once superset release
- * support is added.
- */
-ret = CXL_MBOX_INVALID_PA;
-goto free_and_exit;
+dpa = ent_start_dpa + ent_len;
 }
 len_done = ent_len - len1 - len2;
 
@@ -1752,14 +1752,9 @@ static CXLRetCode 
cxl_dc_extent_release_dry_run(CXLType3Dev *ct3d,
 }
 
 len -= len_done;
-/* len == 0 here until superset release is added */
 break;
 }
 }
-if (len) {
-ret = CXL_MBOX_INVALID_PA;
-goto free_and_exit;
-}
 }
 }
 free_and_exit:
-- 
2.43.0

[PATCH v7 12/12] hw/mem/cxl_type3: Allow to release extent superset in QMP interface

2024-04-18 Thread nifan . cxl 
From: Fan Ni 

Before the change, the QMP interface used for add/release DC extents
only allows to release an extent whose DPA range is contained by a single
accepted extent in the device.

With the change, we relax the constraints.  As long as the DPA range of
the extent is covered by accepted extents, we allow the release.

Reviewed-by: Jonathan Cameron 
Signed-off-by: Fan Ni 
---
 hw/mem/cxl_type3.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index a3e1a5de25..9e725647f1 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -1941,7 +1941,7 @@ static void 
qmp_cxl_process_dynamic_capacity_prescriptive(const char *path,
"cannot release extent with pending DPA range");
 return;
 }
-if (!cxl_extents_contains_dpa_range(>dc.extents, dpa, len)) {
+if (!ct3_test_region_block_backed(dcd, dpa, len)) {
 error_setg(errp,
"cannot release extent with non-existing DPA 
range");
 return;
-- 
2.43.0

[PATCH v7 07/12] hw/mem/cxl_type3: Add DC extent list representative and get DC extent list mailbox support

2024-04-18 Thread nifan . cxl 
From: Fan Ni 

Add dynamic capacity extent list representative to the definition of
CXLType3Dev and implement get DC extent list mailbox command per
CXL.spec.3.1:.8.2.9.9.9.2.

Reviewed-by: Jonathan Cameron 
Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c  | 73 -
 hw/mem/cxl_type3.c  |  1 +
 include/hw/cxl/cxl_device.h | 22 +++
 3 files changed, 95 insertions(+), 1 deletion(-)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 831cef0567..1915959015 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -84,6 +84,7 @@ enum {
 #define CLEAR_POISON   0x2
 DCD_CONFIG  = 0x48,
 #define GET_DC_CONFIG  0x0
+#define GET_DYN_CAP_EXT_LIST   0x1
 PHYSICAL_SWITCH = 0x51,
 #define IDENTIFY_SWITCH_DEVICE  0x0
 #define GET_PHYSICAL_PORT_STATE 0x1
@@ -1322,7 +1323,8 @@ static CXLRetCode cmd_dcd_get_dyn_cap_config(const struct 
cxl_cmd *cmd,
  * to use.
  */
 stl_le_p(_out->num_extents_supported, CXL_NUM_EXTENTS_SUPPORTED);
-stl_le_p(_out->num_extents_available, CXL_NUM_EXTENTS_SUPPORTED);
+stl_le_p(_out->num_extents_available, CXL_NUM_EXTENTS_SUPPORTED -
+ ct3d->dc.total_extent_count);
 stl_le_p(_out->num_tags_supported, CXL_NUM_TAGS_SUPPORTED);
 stl_le_p(_out->num_tags_available, CXL_NUM_TAGS_SUPPORTED);
 
@@ -1330,6 +1332,72 @@ static CXLRetCode cmd_dcd_get_dyn_cap_config(const 
struct cxl_cmd *cmd,
 return CXL_MBOX_SUCCESS;
 }
 
+/*
+ * CXL r3.1 section 8.2.9.9.9.2:
+ * Get Dynamic Capacity Extent List (Opcode 4801h)
+ */
+static CXLRetCode cmd_dcd_get_dyn_cap_ext_list(const struct cxl_cmd *cmd,
+   uint8_t *payload_in,
+   size_t len_in,
+   uint8_t *payload_out,
+   size_t *len_out,
+   CXLCCI *cci)
+{
+CXLType3Dev *ct3d = CXL_TYPE3(cci->d);
+struct {
+uint32_t extent_cnt;
+uint32_t start_extent_id;
+} QEMU_PACKED *in = (void *)payload_in;
+struct {
+uint32_t count;
+uint32_t total_extents;
+uint32_t generation_num;
+uint8_t rsvd[4];
+CXLDCExtentRaw records[];
+} QEMU_PACKED *out = (void *)payload_out;
+uint32_t start_extent_id = in->start_extent_id;
+CXLDCExtentList *extent_list = >dc.extents;
+uint16_t record_count = 0, i = 0, record_done = 0;
+uint16_t out_pl_len, size;
+CXLDCExtent *ent;
+
+if (start_extent_id > ct3d->dc.total_extent_count) {
+return CXL_MBOX_INVALID_INPUT;
+}
+
+record_count = MIN(in->extent_cnt,
+   ct3d->dc.total_extent_count - start_extent_id);
+size = CXL_MAILBOX_MAX_PAYLOAD_SIZE - sizeof(*out);
+record_count = MIN(record_count, size / sizeof(out->records[0]));
+out_pl_len = sizeof(*out) + record_count * sizeof(out->records[0]);
+
+stl_le_p(>count, record_count);
+stl_le_p(>total_extents, ct3d->dc.total_extent_count);
+stl_le_p(>generation_num, ct3d->dc.ext_list_gen_seq);
+
+if (record_count > 0) {
+CXLDCExtentRaw *out_rec = >records[record_done];
+
+QTAILQ_FOREACH(ent, extent_list, node) {
+if (i++ < start_extent_id) {
+continue;
+}
+stq_le_p(_rec->start_dpa, ent->start_dpa);
+stq_le_p(_rec->len, ent->len);
+memcpy(_rec->tag, ent->tag, 0x10);
+stw_le_p(_rec->shared_seq, ent->shared_seq);
+
+record_done++;
+if (record_done == record_count) {
+break;
+}
+}
+}
+
+*len_out = out_pl_len;
+return CXL_MBOX_SUCCESS;
+}
+
 #define IMMEDIATE_CONFIG_CHANGE (1 << 1)
 #define IMMEDIATE_DATA_CHANGE (1 << 2)
 #define IMMEDIATE_POLICY_CHANGE (1 << 3)
@@ -1377,6 +1445,9 @@ static const struct cxl_cmd cxl_cmd_set[256][256] = {
 static const struct cxl_cmd cxl_cmd_set_dcd[256][256] = {
 [DCD_CONFIG][GET_DC_CONFIG] = { "DCD_GET_DC_CONFIG",
 cmd_dcd_get_dyn_cap_config, 2, 0 },
+[DCD_CONFIG][GET_DYN_CAP_EXT_LIST] = {
+"DCD_GET_DYNAMIC_CAPACITY_EXTENT_LIST", cmd_dcd_get_dyn_cap_ext_list,
+8, 0 },
 };
 
 static const struct cxl_cmd cxl_cmd_set_sw[256][256] = {
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index ac87398089..9fffeae613 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -666,6 +666,7 @@ static bool cxl_create_dc_regions(CXLType3Dev *ct3d, Error 
**errp)
 };
 ct3d->dc.total_capacity += region->len;
 }
+QTAILQ_INIT(>dc.extents);
 
 return true;
 }
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index c2c3df0d2a..6aec6ac983 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -424,6 +424,25 @@ typedef QLIST_HEAD(,

[PATCH v7 08/12] hw/cxl/cxl-mailbox-utils: Add mailbox commands to support add/release dynamic capacity response

2024-04-18 Thread nifan . cxl 
From: Fan Ni 

Per CXL spec 3.1, two mailbox commands are implemented:
Add Dynamic Capacity Response (Opcode 4802h) 8.2.9.9.9.3, and
Release Dynamic Capacity (Opcode 4803h) 8.2.9.9.9.4.

For the process of the above two commands, we use two-pass approach.
Pass 1: Check whether the input payload is valid or not; if not, skip
Pass 2 and return mailbox process error.
Pass 2: Do the real work--add or release extents, respectively.

Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c  | 394 
 hw/mem/cxl_type3.c  |  11 +
 include/hw/cxl/cxl_device.h |   4 +
 3 files changed, 409 insertions(+)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 1915959015..9d54e10cd4 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -19,6 +19,7 @@
 #include "qemu/units.h"
 #include "qemu/uuid.h"
 #include "sysemu/hostmem.h"
+#include "qemu/range.h"
 
 #define CXL_CAPACITY_MULTIPLIER   (256 * MiB)
 #define CXL_DC_EVENT_LOG_SIZE 8
@@ -85,6 +86,8 @@ enum {
 DCD_CONFIG  = 0x48,
 #define GET_DC_CONFIG  0x0
 #define GET_DYN_CAP_EXT_LIST   0x1
+#define ADD_DYN_CAP_RSP0x2
+#define RELEASE_DYN_CAP0x3
 PHYSICAL_SWITCH = 0x51,
 #define IDENTIFY_SWITCH_DEVICE  0x0
 #define GET_PHYSICAL_PORT_STATE 0x1
@@ -1398,6 +1401,391 @@ static CXLRetCode cmd_dcd_get_dyn_cap_ext_list(const 
struct cxl_cmd *cmd,
 return CXL_MBOX_SUCCESS;
 }
 
+/*
+ * Check whether any bit between addr[nr, nr+size) is set,
+ * return true if any bit is set, otherwise return false
+ */
+static bool test_any_bits_set(const unsigned long *addr, unsigned long nr,
+  unsigned long size)
+{
+unsigned long res = find_next_bit(addr, size + nr, nr);
+
+return res < nr + size;
+}
+
+CXLDCRegion *cxl_find_dc_region(CXLType3Dev *ct3d, uint64_t dpa, uint64_t len)
+{
+int i;
+CXLDCRegion *region = >dc.regions[0];
+
+if (dpa < region->base ||
+dpa >= region->base + ct3d->dc.total_capacity) {
+return NULL;
+}
+
+/*
+ * CXL r3.1 section 9.13.3: Dynamic Capacity Device (DCD)
+ *
+ * Regions are used in increasing-DPA order, with Region 0 being used for
+ * the lowest DPA of Dynamic Capacity and Region 7 for the highest DPA.
+ * So check from the last region to find where the dpa belongs. Extents 
that
+ * cross multiple regions are not allowed.
+ */
+for (i = ct3d->dc.num_regions - 1; i >= 0; i--) {
+region = >dc.regions[i];
+if (dpa >= region->base) {
+if (dpa + len > region->base + region->len) {
+return NULL;
+}
+return region;
+}
+}
+
+return NULL;
+}
+
+static void cxl_insert_extent_to_extent_list(CXLDCExtentList *list,
+ uint64_t dpa,
+ uint64_t len,
+ uint8_t *tag,
+ uint16_t shared_seq)
+{
+CXLDCExtent *extent;
+
+extent = g_new0(CXLDCExtent, 1);
+extent->start_dpa = dpa;
+extent->len = len;
+if (tag) {
+memcpy(extent->tag, tag, 0x10);
+}
+extent->shared_seq = shared_seq;
+
+QTAILQ_INSERT_TAIL(list, extent, node);
+}
+
+void cxl_remove_extent_from_extent_list(CXLDCExtentList *list,
+CXLDCExtent *extent)
+{
+QTAILQ_REMOVE(list, extent, node);
+g_free(extent);
+}
+
+/*
+ * CXL r3.1 Table 8-168: Add Dynamic Capacity Response Input Payload
+ * CXL r3.1 Table 8-170: Release Dynamic Capacity Input Payload
+ */
+typedef struct CXLUpdateDCExtentListInPl {
+uint32_t num_entries_updated;
+uint8_t flags;
+uint8_t rsvd[3];
+/* CXL r3.1 Table 8-169: Updated Extent */
+struct {
+uint64_t start_dpa;
+uint64_t len;
+uint8_t rsvd[8];
+} QEMU_PACKED updated_entries[];
+} QEMU_PACKED CXLUpdateDCExtentListInPl;
+
+/*
+ * For the extents in the extent list to operate, check whether they are valid
+ * 1. The extent should be in the range of a valid DC region;
+ * 2. The extent should not cross multiple regions;
+ * 3. The start DPA and the length of the extent should align with the block
+ * size of the region;
+ * 4. The address range of multiple extents in the list should not overlap.
+ */
+static CXLRetCode cxl_detect_malformed_extent_list(CXLType3Dev *ct3d,
+const CXLUpdateDCExtentListInPl *in)
+{
+uint64_t min_block_size = UINT64_MAX;
+CXLDCRegion *region;
+CXLDCRegion *lastregion = >dc.regions[ct3d->dc.num_regions - 1];
+g_autofree unsigned long *blk_bitmap = NULL;
+uint64_t dpa, len;
+uint32_t i;
+
+for (i = 0; i < ct3d->dc.num_regions; i++) {
+region = >dc.regions[i];
+min_block_size = MIN(min_block_size, region->block_size);
+}
+
+blk_bitmap =

[PATCH v7 03/12] include/hw/cxl/cxl_device: Rename mem_size as static_mem_size for type3 memory devices

2024-04-18 Thread nifan . cxl 
From: Fan Ni 

Rename mem_size as static_mem_size for type3 memdev to cover static RAM and
pmem capacity, preparing for the introduction of dynamic capacity to support
dynamic capacity devices.

Reviewed-by: Jonathan Cameron 
Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c  | 4 ++--
 hw/mem/cxl_type3.c  | 8 
 include/hw/cxl/cxl_device.h | 2 +-
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 49c7944d93..0f2ad58a14 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -803,7 +803,7 @@ static CXLRetCode cmd_identify_memory_device(const struct 
cxl_cmd *cmd,
 snprintf(id->fw_revision, 0x10, "BWFW VERSION %02d", 0);
 
 stq_le_p(>total_capacity,
- cxl_dstate->mem_size / CXL_CAPACITY_MULTIPLIER);
+ cxl_dstate->static_mem_size / CXL_CAPACITY_MULTIPLIER);
 stq_le_p(>persistent_capacity,
  cxl_dstate->pmem_size / CXL_CAPACITY_MULTIPLIER);
 stq_le_p(>volatile_capacity,
@@ -1179,7 +1179,7 @@ static CXLRetCode cmd_media_clear_poison(const struct 
cxl_cmd *cmd,
 struct clear_poison_pl *in = (void *)payload_in;
 
 dpa = ldq_le_p(>dpa);
-if (dpa + CXL_CACHE_LINE_SIZE > cxl_dstate->mem_size) {
+if (dpa + CXL_CACHE_LINE_SIZE > cxl_dstate->static_mem_size) {
 return CXL_MBOX_INVALID_PA;
 }
 
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index b0a7e9f11b..5d6d3ab87d 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -608,7 +608,7 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error 
**errp)
 }
 address_space_init(>hostvmem_as, vmr, v_name);
 ct3d->cxl_dstate.vmem_size = memory_region_size(vmr);
-ct3d->cxl_dstate.mem_size += memory_region_size(vmr);
+ct3d->cxl_dstate.static_mem_size += memory_region_size(vmr);
 g_free(v_name);
 }
 
@@ -631,7 +631,7 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error 
**errp)
 }
 address_space_init(>hostpmem_as, pmr, p_name);
 ct3d->cxl_dstate.pmem_size = memory_region_size(pmr);
-ct3d->cxl_dstate.mem_size += memory_region_size(pmr);
+ct3d->cxl_dstate.static_mem_size += memory_region_size(pmr);
 g_free(p_name);
 }
 
@@ -838,7 +838,7 @@ static int cxl_type3_hpa_to_as_and_dpa(CXLType3Dev *ct3d,
 return -EINVAL;
 }
 
-if (*dpa_offset > ct3d->cxl_dstate.mem_size) {
+if (*dpa_offset > ct3d->cxl_dstate.static_mem_size) {
 return -EINVAL;
 }
 
@@ -1011,7 +1011,7 @@ static bool set_cacheline(CXLType3Dev *ct3d, uint64_t 
dpa_offset, uint8_t *data)
 return false;
 }
 
-if (dpa_offset + CXL_CACHE_LINE_SIZE > ct3d->cxl_dstate.mem_size) {
+if (dpa_offset + CXL_CACHE_LINE_SIZE > ct3d->cxl_dstate.static_mem_size) {
 return false;
 }
 
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index e839370266..f7f56b44e3 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -234,7 +234,7 @@ typedef struct cxl_device_state {
 } timestamp;
 
 /* memory region size, HDM */
-uint64_t mem_size;
+uint64_t static_mem_size;
 uint64_t pmem_size;
 uint64_t vmem_size;
 
-- 
2.43.0

[PATCH v7 06/12] hw/mem/cxl_type3: Add host backend and address space handling for DC regions

2024-04-18 Thread nifan . cxl 
From: Fan Ni 

Add (file/memory backed) host backend for DCD. All the dynamic capacity
regions will share a single, large enough host backend. Set up address
space for DC regions to support read/write operations to dynamic capacity
for DCD.

With the change, the following support is added:
1. Add a new property to type3 device "volatile-dc-memdev" to point to host
   memory backend for dynamic capacity. Currently, all DC regions share one
   host backend;
2. Add namespace for dynamic capacity for read/write support;
3. Create cdat entries for each dynamic capacity region.

Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c  |  16 ++--
 hw/mem/cxl_type3.c  | 172 +---
 include/hw/cxl/cxl_device.h |   8 ++
 3 files changed, 160 insertions(+), 36 deletions(-)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 0f2ad58a14..831cef0567 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -622,7 +622,8 @@ static CXLRetCode cmd_firmware_update_get_info(const struct 
cxl_cmd *cmd,
size_t *len_out,
CXLCCI *cci)
 {
-CXLDeviceState *cxl_dstate = _TYPE3(cci->d)->cxl_dstate;
+CXLType3Dev *ct3d = CXL_TYPE3(cci->d);
+CXLDeviceState *cxl_dstate = >cxl_dstate;
 struct {
 uint8_t slots_supported;
 uint8_t slot_info;
@@ -636,7 +637,8 @@ static CXLRetCode cmd_firmware_update_get_info(const struct 
cxl_cmd *cmd,
 QEMU_BUILD_BUG_ON(sizeof(*fw_info) != 0x50);
 
 if ((cxl_dstate->vmem_size < CXL_CAPACITY_MULTIPLIER) ||
-(cxl_dstate->pmem_size < CXL_CAPACITY_MULTIPLIER)) {
+(cxl_dstate->pmem_size < CXL_CAPACITY_MULTIPLIER) ||
+(ct3d->dc.total_capacity < CXL_CAPACITY_MULTIPLIER)) {
 return CXL_MBOX_INTERNAL_ERROR;
 }
 
@@ -793,7 +795,8 @@ static CXLRetCode cmd_identify_memory_device(const struct 
cxl_cmd *cmd,
 CXLDeviceState *cxl_dstate = >cxl_dstate;
 
 if ((!QEMU_IS_ALIGNED(cxl_dstate->vmem_size, CXL_CAPACITY_MULTIPLIER)) ||
-(!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER))) {
+(!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER)) ||
+(!QEMU_IS_ALIGNED(ct3d->dc.total_capacity, CXL_CAPACITY_MULTIPLIER))) {
 return CXL_MBOX_INTERNAL_ERROR;
 }
 
@@ -835,9 +838,11 @@ static CXLRetCode cmd_ccls_get_partition_info(const struct 
cxl_cmd *cmd,
 uint64_t next_pmem;
 } QEMU_PACKED *part_info = (void *)payload_out;
 QEMU_BUILD_BUG_ON(sizeof(*part_info) != 0x20);
+CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
 
 if ((!QEMU_IS_ALIGNED(cxl_dstate->vmem_size, CXL_CAPACITY_MULTIPLIER)) ||
-(!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER))) {
+(!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER)) ||
+(!QEMU_IS_ALIGNED(ct3d->dc.total_capacity, CXL_CAPACITY_MULTIPLIER))) {
 return CXL_MBOX_INTERNAL_ERROR;
 }
 
@@ -1179,7 +1184,8 @@ static CXLRetCode cmd_media_clear_poison(const struct 
cxl_cmd *cmd,
 struct clear_poison_pl *in = (void *)payload_in;
 
 dpa = ldq_le_p(>dpa);
-if (dpa + CXL_CACHE_LINE_SIZE > cxl_dstate->static_mem_size) {
+if (dpa + CXL_CACHE_LINE_SIZE > cxl_dstate->static_mem_size +
+ct3d->dc.total_capacity) {
 return CXL_MBOX_INVALID_PA;
 }
 
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index a1fe268560..ac87398089 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -45,7 +45,8 @@ enum {
 
 static void ct3_build_cdat_entries_for_mr(CDATSubHeader **cdat_table,
   int dsmad_handle, uint64_t size,
-  bool is_pmem, uint64_t dpa_base)
+  bool is_pmem, bool is_dynamic,
+  uint64_t dpa_base)
 {
 CDATDsmas *dsmas;
 CDATDslbis *dslbis0;
@@ -61,7 +62,8 @@ static void ct3_build_cdat_entries_for_mr(CDATSubHeader 
**cdat_table,
 .length = sizeof(*dsmas),
 },
 .DSMADhandle = dsmad_handle,
-.flags = is_pmem ? CDAT_DSMAS_FLAG_NV : 0,
+.flags = (is_pmem ? CDAT_DSMAS_FLAG_NV : 0) |
+ (is_dynamic ? CDAT_DSMAS_FLAG_DYNAMIC_CAP : 0),
 .DPA_base = dpa_base,
 .DPA_length = size,
 };
@@ -149,12 +151,13 @@ static int ct3_build_cdat_table(CDATSubHeader 
***cdat_table, void *priv)
 g_autofree CDATSubHeader **table = NULL;
 CXLType3Dev *ct3d = priv;
 MemoryRegion *volatile_mr = NULL, *nonvolatile_mr = NULL;
+MemoryRegion *dc_mr = NULL;
 uint64_t vmr_size = 0, pmr_size = 0;
 int dsmad_handle = 0;
 int cur_ent = 0;
 int len = 0;
 
-if (!ct3d->hostpmem && !ct3d->hostvmem) {
+if (!ct3d->hostpmem && !ct3d->hostvmem && !ct3d->dc.num_regions) {
 return 0;
 }
 
@@ -176,21

[PATCH v7 09/12] hw/cxl/events: Add qmp interfaces to add/release dynamic capacity extents

2024-04-18 Thread nifan . cxl 
From: Fan Ni 

To simulate FM functionalities for initiating Dynamic Capacity Add
(Opcode 5604h) and Dynamic Capacity Release (Opcode 5605h) as in CXL spec
r3.1 7.6.7.6.5 and 7.6.7.6.6, we implemented two QMP interfaces to issue
add/release dynamic capacity extents requests.

With the change, we allow to release an extent only when its DPA range
is contained by a single accepted extent in the device. That is to say,
extent superset release is not supported yet.

1. Add dynamic capacity extents:

For example, the command to add two continuous extents (each 128MiB long)
to region 0 (starting at DPA offset 0) looks like below:

{ "execute": "qmp_capabilities" }

{ "execute": "cxl-add-dynamic-capacity",
  "arguments": {
  "path": "/machine/peripheral/cxl-dcd0",
  "hid": 0,
  "selection-policy": 2,
  "region-id": 0,
  "tag": "",
  "extents": [
  {
  "offset": 0,
  "len": 134217728
  },
  {
  "offset": 134217728,
  "len": 134217728
  }
  ]
  }
}

2. Release dynamic capacity extents:

For example, the command to release an extent of size 128MiB from region 0
(DPA offset 128MiB) looks like below:

{ "execute": "cxl-release-dynamic-capacity",
  "arguments": {
  "path": "/machine/peripheral/cxl-dcd0",
  "hid": 0,
  "flags": 1,
  "region-id": 0,
  "tag": "",
  "extents": [
  {
  "offset": 134217728,
  "len": 134217728
  }
  ]
  }
}

Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c  |  62 +--
 hw/mem/cxl_type3.c  | 311 +++-
 hw/mem/cxl_type3_stubs.c|  20 +++
 include/hw/cxl/cxl_device.h |  22 +++
 include/hw/cxl/cxl_events.h |  18 +++
 qapi/cxl.json   |  69 
 6 files changed, 489 insertions(+), 13 deletions(-)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 9d54e10cd4..3569902e9e 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -1405,7 +1405,7 @@ static CXLRetCode cmd_dcd_get_dyn_cap_ext_list(const 
struct cxl_cmd *cmd,
  * Check whether any bit between addr[nr, nr+size) is set,
  * return true if any bit is set, otherwise return false
  */
-static bool test_any_bits_set(const unsigned long *addr, unsigned long nr,
+bool test_any_bits_set(const unsigned long *addr, unsigned long nr,
   unsigned long size)
 {
 unsigned long res = find_next_bit(addr, size + nr, nr);
@@ -1444,7 +1444,7 @@ CXLDCRegion *cxl_find_dc_region(CXLType3Dev *ct3d, 
uint64_t dpa, uint64_t len)
 return NULL;
 }
 
-static void cxl_insert_extent_to_extent_list(CXLDCExtentList *list,
+void cxl_insert_extent_to_extent_list(CXLDCExtentList *list,
  uint64_t dpa,
  uint64_t len,
  uint8_t *tag,
@@ -1470,6 +1470,44 @@ void cxl_remove_extent_from_extent_list(CXLDCExtentList 
*list,
 g_free(extent);
 }
 
+/*
+ * Add a new extent to the extent "group" if group exists;
+ * otherwise, create a new group
+ * Return value: return the group where the extent is inserted.
+ */
+CXLDCExtentGroup *cxl_insert_extent_to_extent_group(CXLDCExtentGroup *group,
+uint64_t dpa,
+uint64_t len,
+uint8_t *tag,
+uint16_t shared_seq)
+{
+if (!group) {
+group = g_new0(CXLDCExtentGroup, 1);
+QTAILQ_INIT(>list);
+}
+cxl_insert_extent_to_extent_list(>list, dpa, len,
+ tag, shared_seq);
+return group;
+}
+
+void cxl_extent_group_list_insert_tail(CXLDCExtentGroupList *list,
+   CXLDCExtentGroup *group)
+{
+QTAILQ_INSERT_TAIL(list, group, node);
+}
+
+void cxl_extent_group_list_delete_front(CXLDCExtentGroupList *list)
+{
+CXLDCExtent *ent, *ent_next;
+CXLDCExtentGroup *group = QTAILQ_FIRST(list);
+
+QTAILQ_REMOVE(list, group, node);
+QTAILQ_FOREACH_SAFE(ent, >list, node, ent_next) {
+cxl_remove_extent_from_extent_list(>list, ent);
+}
+g_free(group);
+}
+
 /*
  * CXL r3.1 Table 8-168: Add Dynamic Capacity Response Input Payload
  * CXL r3.1 Table 8-170: Release Dynamic Capacity Input Payload
@@ -1541,6 +1579,7 @@ static CXLRetCode 
cxl_dcd_add_dyn_cap_rsp_dry_run(CXLType3Dev *ct3d,
 {
 uint32_t i;
 CXLDCExtent *ent;
+CXLDCExtentGroup *ext_group;
 uint64_t dpa, len;
 Range range1, range2;
 
@@ -1551,9 +1590,13 @@ static CXLRetCode 
cxl_dcd_add_dyn_cap_rsp_dry_run(CXLType3Dev *ct3d,
 range_init_nofail(, dpa, len);
 
 /*
- * TODO: once the pending extent list is added, check against
- * the list will be added here.
+ * The host-accepted DPA range must be contained by

[PATCH v7 02/12] hw/cxl/cxl-mailbox-utils: Add dynamic capacity region representative and mailbox command support

2024-04-18 Thread nifan . cxl 
From: Fan Ni 

Per cxl spec r3.1, add dynamic capacity region representative based on
Table 8-165 and extend the cxl type3 device definition to include DC region
information. Also, based on info in 8.2.9.9.9.1, add 'Get Dynamic Capacity
Configuration' mailbox support.

Note: we store region decode length as byte-wise length on the device, which
should be divided by 256 * MiB before being returned to the host
for "Get Dynamic Capacity Configuration" mailbox command per
specification.

Reviewed-by: Jonathan Cameron 
Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c  | 96 +
 include/hw/cxl/cxl_device.h | 16 +++
 2 files changed, 112 insertions(+)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index ba1d9901df..49c7944d93 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -22,6 +22,8 @@
 
 #define CXL_CAPACITY_MULTIPLIER   (256 * MiB)
 #define CXL_DC_EVENT_LOG_SIZE 8
+#define CXL_NUM_EXTENTS_SUPPORTED 512
+#define CXL_NUM_TAGS_SUPPORTED 0
 
 /*
  * How to add a new command, example. The command set FOO, with cmd BAR.
@@ -80,6 +82,8 @@ enum {
 #define GET_POISON_LIST0x0
 #define INJECT_POISON  0x1
 #define CLEAR_POISON   0x2
+DCD_CONFIG  = 0x48,
+#define GET_DC_CONFIG  0x0
 PHYSICAL_SWITCH = 0x51,
 #define IDENTIFY_SWITCH_DEVICE  0x0
 #define GET_PHYSICAL_PORT_STATE 0x1
@@ -1238,6 +1242,88 @@ static CXLRetCode cmd_media_clear_poison(const struct 
cxl_cmd *cmd,
 return CXL_MBOX_SUCCESS;
 }
 
+/*
+ * CXL r3.1 section 8.2.9.9.9.1: Get Dynamic Capacity Configuration
+ * (Opcode: 4800h)
+ */
+static CXLRetCode cmd_dcd_get_dyn_cap_config(const struct cxl_cmd *cmd,
+ uint8_t *payload_in,
+ size_t len_in,
+ uint8_t *payload_out,
+ size_t *len_out,
+ CXLCCI *cci)
+{
+CXLType3Dev *ct3d = CXL_TYPE3(cci->d);
+struct {
+uint8_t region_cnt;
+uint8_t start_rid;
+} QEMU_PACKED *in = (void *)payload_in;
+struct {
+uint8_t num_regions;
+uint8_t regions_returned;
+uint8_t rsvd1[6];
+struct {
+uint64_t base;
+uint64_t decode_len;
+uint64_t region_len;
+uint64_t block_size;
+uint32_t dsmadhandle;
+uint8_t flags;
+uint8_t rsvd2[3];
+} QEMU_PACKED records[];
+} QEMU_PACKED *out = (void *)payload_out;
+struct {
+uint32_t num_extents_supported;
+uint32_t num_extents_available;
+uint32_t num_tags_supported;
+uint32_t num_tags_available;
+} QEMU_PACKED *extra_out;
+uint16_t record_count;
+uint16_t i;
+uint16_t out_pl_len;
+uint8_t start_rid;
+
+start_rid = in->start_rid;
+if (start_rid >= ct3d->dc.num_regions) {
+return CXL_MBOX_INVALID_INPUT;
+}
+
+record_count = MIN(ct3d->dc.num_regions - in->start_rid, in->region_cnt);
+
+out_pl_len = sizeof(*out) + record_count * sizeof(out->records[0]);
+extra_out = (void *)(payload_out + out_pl_len);
+out_pl_len += sizeof(*extra_out);
+assert(out_pl_len <= CXL_MAILBOX_MAX_PAYLOAD_SIZE);
+
+out->num_regions = ct3d->dc.num_regions;
+out->regions_returned = record_count;
+for (i = 0; i < record_count; i++) {
+stq_le_p(>records[i].base,
+ ct3d->dc.regions[start_rid + i].base);
+stq_le_p(>records[i].decode_len,
+ ct3d->dc.regions[start_rid + i].decode_len /
+ CXL_CAPACITY_MULTIPLIER);
+stq_le_p(>records[i].region_len,
+ ct3d->dc.regions[start_rid + i].len);
+stq_le_p(>records[i].block_size,
+ ct3d->dc.regions[start_rid + i].block_size);
+stl_le_p(>records[i].dsmadhandle,
+ ct3d->dc.regions[start_rid + i].dsmadhandle);
+out->records[i].flags = ct3d->dc.regions[start_rid + i].flags;
+}
+/*
+ * TODO: Assign values once extents and tags are introduced
+ * to use.
+ */
+stl_le_p(_out->num_extents_supported, CXL_NUM_EXTENTS_SUPPORTED);
+stl_le_p(_out->num_extents_available, CXL_NUM_EXTENTS_SUPPORTED);
+stl_le_p(_out->num_tags_supported, CXL_NUM_TAGS_SUPPORTED);
+stl_le_p(_out->num_tags_available, CXL_NUM_TAGS_SUPPORTED);
+
+*len_out = out_pl_len;
+return CXL_MBOX_SUCCESS;
+}
+
 #define IMMEDIATE_CONFIG_CHANGE (1 << 1)
 #define IMMEDIATE_DATA_CHANGE (1 << 2)
 #define IMMEDIATE_POLICY_CHANGE (1 << 3)
@@ -1282,6 +1368,11 @@ static const struct cxl_cmd cxl_cmd_set[256][256] = {
 cmd_media_clear_poison, 72, 0 },
 };
 
+static const struct cxl_cmd cxl_cmd_set_dcd[256][256] = {
+[DCD_CONFIG][GET_DC_CONFIG] = { "DCD_GET_DC_CONFIG",
+

[PATCH v7 01/12] hw/cxl/cxl-mailbox-utils: Add dc_event_log_size field to output payload of identify memory device command

2024-04-18 Thread nifan . cxl 
From: Fan Ni 

Based on CXL spec r3.1 Table 8-127 (Identify Memory Device Output
Payload), dynamic capacity event log size should be part of
output of the Identify command.
Add dc_event_log_size to the output payload for the host to get the info.

Reviewed-by: Jonathan Cameron 
Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 4bcd727f4c..ba1d9901df 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -21,6 +21,7 @@
 #include "sysemu/hostmem.h"
 
 #define CXL_CAPACITY_MULTIPLIER   (256 * MiB)
+#define CXL_DC_EVENT_LOG_SIZE 8
 
 /*
  * How to add a new command, example. The command set FOO, with cmd BAR.
@@ -780,8 +781,9 @@ static CXLRetCode cmd_identify_memory_device(const struct 
cxl_cmd *cmd,
 uint16_t inject_poison_limit;
 uint8_t poison_caps;
 uint8_t qos_telemetry_caps;
+uint16_t dc_event_log_size;
 } QEMU_PACKED *id;
-QEMU_BUILD_BUG_ON(sizeof(*id) != 0x43);
+QEMU_BUILD_BUG_ON(sizeof(*id) != 0x45);
 CXLType3Dev *ct3d = CXL_TYPE3(cci->d);
 CXLType3Class *cvc = CXL_TYPE3_GET_CLASS(ct3d);
 CXLDeviceState *cxl_dstate = >cxl_dstate;
@@ -807,6 +809,7 @@ static CXLRetCode cmd_identify_memory_device(const struct 
cxl_cmd *cmd,
 st24_le_p(id->poison_list_max_mer, 256);
 /* No limit - so limited by main poison record limit */
 stw_le_p(>inject_poison_limit, 0);
+stw_le_p(>dc_event_log_size, CXL_DC_EVENT_LOG_SIZE);
 
 *len_out = sizeof(*id);
 return CXL_MBOX_SUCCESS;
-- 
2.43.0

[PATCH v7 00/12] Enabling DCD emulation support in Qemu

2024-04-18 Thread nifan . cxl 
A git tree of this series can be found here (with one extra commit on top
for printing out accepted/pending extent list): 
https://github.com/moking/qemu/tree/dcd-v7

v6->v7:

1. Fixed the dvsec range register issue mentioned in the the cover letter in v6.
   Only relevant bits are set to mark the device ready (Patch 6). (Jonathan)
2. Moved the if statement in cxl_setup_memory from Patch 6 to Patch 4. 
(Jonathan)
3. Used MIN instead of if statement to get record_count in Patch 7. (Jonathan)
4. Added "Reviewed-by" tag to Patch 7.
5. Modified cxl_dc_extent_release_dry_run so the updated extent list can be
   reused in cmd_dcd_release_dyn_cap to simplify the process in Patch 8. 
(Jørgen) 
6. Added comments to indicate further "TODO" items in cmd_dcd_add_dyn_cap_rsp.
(Jonathan)
7. Avoided irrelevant code reformat in Patch 8. (Jonathan)
8. Modified QMP interfaces for adding/releasing DC extents to allow passing
   tags, selection policy, flags in the interface. (Jonathan, Gregory)
9. Redesigned the pending list so extents in the same requests are grouped
together. A new data structure is introduced to represent "extent group"
in pending list.  (Jonathan)
10. Added support in QMP interface for "More" flag. 
11. Check "Forced removal" flag for release request and not let it pass through.
12. Removed the dynamic capacity log type from CxlEventLog definition in 
cxl.json
   to avoid the side effect it may introduce to inject error to DC event log.
   (Jonathan)
13. Hard coded the event log type to dynamic capacity event log in QMP
interfaces. (Jonathan)
14. Adding space in between "-1]". (Jonathan)
15. Some minor comment fixes.

The code is tested with similar setup and has passed similar tests as listed
in the cover letter of v5[1] and v6[2].
Also, the code is tested with the latest DCD kernel patchset[3].

[1] Qemu DCD patchset v5: 
https://lore.kernel.org/linux-cxl/20240304194331.1586191-1-nifan@gmail.com/T/#t
[2] Qemu DCD patchset v6: 
https://lore.kernel.org/linux-cxl/20240325190339.696686-1-nifan@gmail.com/T/#t
[3] DCD kernel patches: 
https://lore.kernel.org/linux-cxl/20240324-dcd-type2-upstream-v1-0-b7b00d623...@intel.com/T/#m11c571e21c4fe17c7d04ec5c2c7bc7cbf2cd07e3


Fan Ni (12):
  hw/cxl/cxl-mailbox-utils: Add dc_event_log_size field to output
payload of identify memory device command
  hw/cxl/cxl-mailbox-utils: Add dynamic capacity region representative
and mailbox command support
  include/hw/cxl/cxl_device: Rename mem_size as static_mem_size for
type3 memory devices
  hw/mem/cxl_type3: Add support to create DC regions to type3 memory
devices
  hw/mem/cxl-type3: Refactor ct3_build_cdat_entries_for_mr to take mr
size instead of mr as argument
  hw/mem/cxl_type3: Add host backend and address space handling for DC
regions
  hw/mem/cxl_type3: Add DC extent list representative and get DC extent
list mailbox support
  hw/cxl/cxl-mailbox-utils: Add mailbox commands to support add/release
dynamic capacity response
  hw/cxl/events: Add qmp interfaces to add/release dynamic capacity
extents
  hw/mem/cxl_type3: Add DPA range validation for accesses to DC regions
  hw/cxl/cxl-mailbox-utils: Add superset extent release mailbox support
  hw/mem/cxl_type3: Allow to release extent superset in QMP interface

 hw/cxl/cxl-mailbox-utils.c  | 620 ++-
 hw/mem/cxl_type3.c  | 633 +---
 hw/mem/cxl_type3_stubs.c|  20 ++
 include/hw/cxl/cxl_device.h |  81 -
 include/hw/cxl/cxl_events.h |  18 +
 qapi/cxl.json   |  69 
 6 files changed, 1396 insertions(+), 45 deletions(-)

-- 
2.43.0

Re: [PATCH 1/2] libvhost-user: Fix pointer arithmetic in indirect read

2024-04-18 Thread Raphael Norwitz 
The change looks right to me. As is, it looks like the code is
skipping over descriptors when the intent should be to bounce data
into a single descriptor.

I agree the variable rename should go in as a separate change.

On Thu, Apr 18, 2024 at 6:56 AM Daniel P. Berrangé  wrote:
>
> On Sat, Jan 13, 2024 at 04:27:40AM +0300, Temir Zharaspayev wrote:
> > When zero-copy usage of indirect descriptors buffer table isn't
> > possible, library gather scattered memory chunks in a local copy.
> > This commit fixes the issue with pointer arithmetic for the local copy
> > buffer.
> >
> > Signed-off-by: Temir Zharaspayev 
> > ---
> >  subprojects/libvhost-user/libvhost-user.c | 11 ++-
> >  1 file changed, 6 insertions(+), 5 deletions(-)
> >
> > diff --git a/subprojects/libvhost-user/libvhost-user.c 
> > b/subprojects/libvhost-user/libvhost-user.c
> > index 6684057370..e952c098a3 100644
> > --- a/subprojects/libvhost-user/libvhost-user.c
> > +++ b/subprojects/libvhost-user/libvhost-user.c
> > @@ -2307,7 +2307,7 @@ static int
> >  virtqueue_read_indirect_desc(VuDev *dev, struct vring_desc *desc,
> >   uint64_t addr, size_t len)
> >  {
> > -struct vring_desc *ori_desc;
> > +uint8_t *src_cursor, *dst_cursor;
> >  uint64_t read_len;
> >
> >  if (len > (VIRTQUEUE_MAX_SIZE * sizeof(struct vring_desc))) {
> > @@ -2318,17 +2318,18 @@ virtqueue_read_indirect_desc(VuDev *dev, struct 
> > vring_desc *desc,
> >  return -1;
> >  }
> >
> > +dst_cursor = (uint8_t *) desc;

Nit - no space on cast

> >  while (len) {
> >  read_len = len;
> > -ori_desc = vu_gpa_to_va(dev, _len, addr);
> > -if (!ori_desc) {
> > +src_cursor = vu_gpa_to_va(dev, _len, addr);
> > +if (!src_cursor) {
> >  return -1;
> >  }
> >
> > -memcpy(desc, ori_desc, read_len);
> > +memcpy(dst_cursor, src_cursor, read_len);
> >  len -= read_len;
> >  addr += read_len;
> > -desc += read_len;
> > +dst_cursor += read_len;
>
> The ori_desc -> src_cursor changes don't look to have any functional
> effect. Having that change present obscures the functional part of
> the patch, which is this line. FWIW, it is generally preferrable to
> not mix functional and non-functional changes in the same patch
>
> It now interprets 'read_len' as the number of bytes to increment the
> address by, rather than incrementing by the number of elements of
> size 'sizeof(struct vring_desc)'.
>
> I don't know enough about this area of QEMU code to say which
> semantics were desired, so I'll defer to the Michael as maintainer
> to give a formal review.
>
>
> With regards,
> Daniel
> --
> |: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org -o-https://fstop138.berrange.com :|
> |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
>
>

[PATCH v7 4/5] ui/console: Introduce qemu_dmabuf_new() and free() helpers

2024-04-18 Thread dongwon . kim 
From: Dongwon Kim 

This commit introduces utility functions for the creation and deallocation
of QemuDmaBuf instances. Additionally, it updates all relevant sections
of the codebase to utilize these new utility functions.

v7: remove prefix, "dpy_gl_" from all helpers
qemu_dmabuf_free() returns without doing anything if input is null
(Daniel P. Berrangé )
call G_DEFINE_AUTOPTR_CLEANUP_FUNC for qemu_dmabuf_free()
(Daniel P. Berrangé )

Suggested-by: Marc-André Lureau 
Cc: Philippe Mathieu-Daudé 
Cc: Daniel P. Berrangé 
Cc: Vivek Kasireddy 
Signed-off-by: Dongwon Kim 
---
 include/hw/vfio/vfio-common.h   |  2 +-
 include/hw/virtio/virtio-gpu.h  |  4 ++--
 include/ui/console.h|  9 +
 hw/display/vhost-user-gpu.c | 32 -
 hw/display/virtio-gpu-udmabuf.c | 24 +-
 hw/vfio/display.c   | 26 +++-
 ui/console.c| 36 +
 ui/dbus-listener.c  | 28 +++--
 8 files changed, 99 insertions(+), 62 deletions(-)

diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h
index b9da6c08ef..d66e27db02 100644
--- a/include/hw/vfio/vfio-common.h
+++ b/include/hw/vfio/vfio-common.h
@@ -148,7 +148,7 @@ typedef struct VFIOGroup {
 } VFIOGroup;
 
 typedef struct VFIODMABuf {
-QemuDmaBuf buf;
+QemuDmaBuf *buf;
 uint32_t pos_x, pos_y, pos_updates;
 uint32_t hot_x, hot_y, hot_updates;
 int dmabuf_id;
diff --git a/include/hw/virtio/virtio-gpu.h b/include/hw/virtio/virtio-gpu.h
index ed44cdad6b..56d6e821bf 100644
--- a/include/hw/virtio/virtio-gpu.h
+++ b/include/hw/virtio/virtio-gpu.h
@@ -169,7 +169,7 @@ struct VirtIOGPUBaseClass {
 DEFINE_PROP_UINT32("yres", _state, _conf.yres, 800)
 
 typedef struct VGPUDMABuf {
-QemuDmaBuf buf;
+QemuDmaBuf *buf;
 uint32_t scanout_id;
 QTAILQ_ENTRY(VGPUDMABuf) next;
 } VGPUDMABuf;
@@ -238,7 +238,7 @@ struct VhostUserGPU {
 VhostUserBackend *vhost;
 int vhost_gpu_fd; /* closed by the chardev */
 CharBackend vhost_chr;
-QemuDmaBuf dmabuf[VIRTIO_GPU_MAX_SCANOUTS];
+QemuDmaBuf *dmabuf[VIRTIO_GPU_MAX_SCANOUTS];
 bool backend_blocked;
 };
 
diff --git a/include/ui/console.h b/include/ui/console.h
index 92a7d0d468..b47135bc2f 100644
--- a/include/ui/console.h
+++ b/include/ui/console.h
@@ -358,6 +358,15 @@ void dpy_gl_cursor_dmabuf(QemuConsole *con, QemuDmaBuf 
*dmabuf,
   bool have_hot, uint32_t hot_x, uint32_t hot_y);
 void dpy_gl_cursor_position(QemuConsole *con,
 uint32_t pos_x, uint32_t pos_y);
+QemuDmaBuf *qemu_dmabuf_new(uint32_t width, uint32_t height,
+   uint32_t stride, uint32_t x,
+   uint32_t y, uint32_t backing_width,
+   uint32_t backing_height, uint32_t fourcc,
+   uint64_t modifier, int32_t dmabuf_fd,
+   bool allow_fences, bool y0_top);
+void qemu_dmabuf_free(QemuDmaBuf *dmabuf);
+
+G_DEFINE_AUTOPTR_CLEANUP_FUNC(QemuDmaBuf, qemu_dmabuf_free);
 
 int32_t qemu_dmabuf_get_fd(QemuDmaBuf *dmabuf);
 uint32_t qemu_dmabuf_get_width(QemuDmaBuf *dmabuf);
diff --git a/hw/display/vhost-user-gpu.c b/hw/display/vhost-user-gpu.c
index ea9a6c5d10..7f8cf38647 100644
--- a/hw/display/vhost-user-gpu.c
+++ b/hw/display/vhost-user-gpu.c
@@ -250,6 +250,7 @@ vhost_user_gpu_handle_display(VhostUserGPU *g, 
VhostUserGpuMsg *msg)
 VhostUserGpuDMABUFScanout *m = >payload.dmabuf_scanout;
 int fd = qemu_chr_fe_get_msgfd(>vhost_chr);
 int old_fd;
+uint64_t modifier = 0;
 QemuDmaBuf *dmabuf;
 
 if (m->scanout_id >= g->parent_obj.conf.max_outputs) {
@@ -262,31 +263,34 @@ vhost_user_gpu_handle_display(VhostUserGPU *g, 
VhostUserGpuMsg *msg)
 
 g->parent_obj.enable = 1;
 con = g->parent_obj.scanout[m->scanout_id].con;
-dmabuf = >dmabuf[m->scanout_id];
-old_fd = qemu_dmabuf_get_fd(dmabuf);
-if (old_fd >= 0) {
-close(old_fd);
-dmabuf->fd = -1;
+dmabuf = g->dmabuf[m->scanout_id];
+if (dmabuf) {
+old_fd = qemu_dmabuf_get_fd(dmabuf);
+if (old_fd >= 0) {
+close(old_fd);
+qemu_dmabuf_set_fd(dmabuf, -1);
+}
 }
 dpy_gl_release_dmabuf(con, dmabuf);
+g_clear_pointer(, qemu_dmabuf_free);
 if (fd == -1) {
 dpy_gl_scanout_disable(con);
 break;
 }
-*dmabuf = (QemuDmaBuf) {
-.fd = fd,
-.width = m->fd_width,
-.height = m->fd_height,
-.stride = m->fd_stride,
-.fourcc = m->fd_drm_fourcc,
-.y0_top = m->fd_flags & VIRTIO_GPU_RESOURCE_FLAG_Y_0_TOP,
-};
+
 if (msg->request == VHOST_USER_GPU_DMABUF_SCANOUT2) {

[PATCH v7 5/5] ui/dmabuf: New dmabuf.c and dmabuf.h for QemuDmaBuf struct and helpers

2024-04-18 Thread dongwon . kim 
From: Dongwon Kim 

Create new header and source files to encapsulate QemuDmaBuf struct
and its data for privatization.

Suggested-by: Marc-André Lureau 
Cc: Philippe Mathieu-Daudé 
Cc: Daniel P. Berrangé 
Cc: Vivek Kasireddy 
Signed-off-by: Dongwon Kim 
---
 include/ui/console.h |  52 +-
 include/ui/dmabuf.h  |  38 
 ui/console.c | 179 +-
 ui/dmabuf.c  | 223 +++
 ui/meson.build   |   1 +
 5 files changed, 264 insertions(+), 229 deletions(-)
 create mode 100644 include/ui/dmabuf.h
 create mode 100644 ui/dmabuf.c

diff --git a/include/ui/console.h b/include/ui/console.h
index b47135bc2f..5e5c4b6951 100644
--- a/include/ui/console.h
+++ b/include/ui/console.h
@@ -7,6 +7,7 @@
 #include "qapi/qapi-types-ui.h"
 #include "ui/input.h"
 #include "ui/surface.h"
+#include "ui/dmabuf.h"
 
 #define TYPE_QEMU_CONSOLE "qemu-console"
 OBJECT_DECLARE_TYPE(QemuConsole, QemuConsoleClass, QEMU_CONSOLE)
@@ -185,25 +186,6 @@ struct QEMUGLParams {
 int minor_ver;
 };
 
-typedef struct QemuDmaBuf {
-int   fd;
-uint32_t  width;
-uint32_t  height;
-uint32_t  stride;
-uint32_t  fourcc;
-uint64_t  modifier;
-uint32_t  texture;
-uint32_t  x;
-uint32_t  y;
-uint32_t  backing_width;
-uint32_t  backing_height;
-bool  y0_top;
-void  *sync;
-int   fence_fd;
-bool  allow_fences;
-bool  draw_submitted;
-} QemuDmaBuf;
-
 enum display_scanout {
 SCANOUT_NONE,
 SCANOUT_SURFACE,
@@ -358,42 +340,10 @@ void dpy_gl_cursor_dmabuf(QemuConsole *con, QemuDmaBuf 
*dmabuf,
   bool have_hot, uint32_t hot_x, uint32_t hot_y);
 void dpy_gl_cursor_position(QemuConsole *con,
 uint32_t pos_x, uint32_t pos_y);
-QemuDmaBuf *qemu_dmabuf_new(uint32_t width, uint32_t height,
-   uint32_t stride, uint32_t x,
-   uint32_t y, uint32_t backing_width,
-   uint32_t backing_height, uint32_t fourcc,
-   uint64_t modifier, int32_t dmabuf_fd,
-   bool allow_fences, bool y0_top);
-void qemu_dmabuf_free(QemuDmaBuf *dmabuf);
-
-G_DEFINE_AUTOPTR_CLEANUP_FUNC(QemuDmaBuf, qemu_dmabuf_free);
-
-int32_t qemu_dmabuf_get_fd(QemuDmaBuf *dmabuf);
-uint32_t qemu_dmabuf_get_width(QemuDmaBuf *dmabuf);
-uint32_t qemu_dmabuf_get_height(QemuDmaBuf *dmabuf);
-uint32_t qemu_dmabuf_get_stride(QemuDmaBuf *dmabuf);
-uint32_t qemu_dmabuf_get_fourcc(QemuDmaBuf *dmabuf);
-uint64_t qemu_dmabuf_get_modifier(QemuDmaBuf *dmabuf);
-uint32_t qemu_dmabuf_get_texture(QemuDmaBuf *dmabuf);
-uint32_t qemu_dmabuf_get_x(QemuDmaBuf *dmabuf);
-uint32_t qemu_dmabuf_get_y(QemuDmaBuf *dmabuf);
-uint32_t qemu_dmabuf_get_backing_width(QemuDmaBuf *dmabuf);
-uint32_t qemu_dmabuf_get_backing_height(QemuDmaBuf *dmabuf);
-bool qemu_dmabuf_get_y0_top(QemuDmaBuf *dmabuf);
-void *qemu_dmabuf_get_sync(QemuDmaBuf *dmabuf);
-int32_t qemu_dmabuf_get_fence_fd(QemuDmaBuf *dmabuf);
-bool qemu_dmabuf_get_allow_fences(QemuDmaBuf *dmabuf);
-bool qemu_dmabuf_get_draw_submitted(QemuDmaBuf *dmabuf);
-void qemu_dmabuf_set_texture(QemuDmaBuf *dmabuf, uint32_t texture);
-void qemu_dmabuf_set_fence_fd(QemuDmaBuf *dmabuf, int32_t fence_fd);
-void qemu_dmabuf_set_sync(QemuDmaBuf *dmabuf, void *sync);
-void qemu_dmabuf_set_draw_submitted(QemuDmaBuf *dmabuf, bool draw_submitted);
-void qemu_dmabuf_set_fd(QemuDmaBuf *dmabuf, int32_t fd);
 void dpy_gl_release_dmabuf(QemuConsole *con,
QemuDmaBuf *dmabuf);
 void dpy_gl_update(QemuConsole *con,
uint32_t x, uint32_t y, uint32_t w, uint32_t h);
-
 QEMUGLContext dpy_gl_ctx_create(QemuConsole *con,
 QEMUGLParams *params);
 void dpy_gl_ctx_destroy(QemuConsole *con, QEMUGLContext ctx);
diff --git a/include/ui/dmabuf.h b/include/ui/dmabuf.h
new file mode 100644
index 00..8de33cd1ab
--- /dev/null
+++ b/include/ui/dmabuf.h
@@ -0,0 +1,38 @@
+#ifndef DMABUF_H
+#define DMABUF_H
+
+typedef struct QemuDmaBuf QemuDmaBuf;
+
+QemuDmaBuf *qemu_dmabuf_new(uint32_t width, uint32_t height,
+   uint32_t stride, uint32_t x,
+   uint32_t y, uint32_t backing_width,
+   uint32_t backing_height, uint32_t fourcc,
+   uint64_t modifier, int32_t dmabuf_fd,
+   bool allow_fences, bool y0_top);
+void qemu_dmabuf_free(QemuDmaBuf *dmabuf);
+
+G_DEFINE_AUTOPTR_CLEANUP_FUNC(QemuDmaBuf, qemu_dmabuf_free);
+
+int32_t qemu_dmabuf_get_fd(QemuDmaBuf *dmabuf);
+uint32_t qemu_dmabuf_get_width(QemuDmaBuf *dmabuf);
+uint32_t qemu_dmabuf_get_height(QemuDmaBuf *dmabuf);
+uint32_t qemu_dmabuf_get_stride(QemuDmaBuf *dmabuf);
+uint32_t qemu_dmabuf_get_fourcc(QemuDmaBuf *dmabuf);
+uint64_t

[PATCH v7 2/5] ui/console: Introduce qemu_dmabuf_get_..() helpers

2024-04-18 Thread dongwon . kim 
From: Dongwon Kim 

This commit introduces qemu_dmabuf_get_... helpers to extract
specific fields from the QemuDmaBuf struct. It also updates all instances
where fields within the QemuDmaBuf struct are directly accessed, replacing
them with calls to these new helper functions.

v6: fix typos in helper names in ui/spice-display.c

v7: removed prefix, "dpy_gl_" from all helpers

Suggested-by: Marc-André Lureau 
Reviewed-by: Marc-André Lureau 
Cc: Philippe Mathieu-Daudé 
Cc: Daniel P. Berrangé 
Cc: Vivek Kasireddy 
Signed-off-by: Dongwon Kim 
---
 include/ui/console.h|  17 +
 hw/display/vhost-user-gpu.c |   6 +-
 hw/display/virtio-gpu-udmabuf.c |   7 +-
 hw/vfio/display.c   |  15 +++--
 ui/console.c| 116 +++-
 ui/dbus-console.c   |   9 ++-
 ui/dbus-listener.c  |  43 +++-
 ui/egl-headless.c   |  23 +--
 ui/egl-helpers.c|  47 +++--
 ui/gtk-egl.c|  48 -
 ui/gtk-gl-area.c|  37 ++
 ui/gtk.c|   6 +-
 ui/spice-display.c  |  50 --
 13 files changed, 316 insertions(+), 108 deletions(-)

diff --git a/include/ui/console.h b/include/ui/console.h
index 0bc7a00ac0..d845f67182 100644
--- a/include/ui/console.h
+++ b/include/ui/console.h
@@ -358,6 +358,23 @@ void dpy_gl_cursor_dmabuf(QemuConsole *con, QemuDmaBuf 
*dmabuf,
   bool have_hot, uint32_t hot_x, uint32_t hot_y);
 void dpy_gl_cursor_position(QemuConsole *con,
 uint32_t pos_x, uint32_t pos_y);
+
+int32_t qemu_dmabuf_get_fd(QemuDmaBuf *dmabuf);
+uint32_t qemu_dmabuf_get_width(QemuDmaBuf *dmabuf);
+uint32_t qemu_dmabuf_get_height(QemuDmaBuf *dmabuf);
+uint32_t qemu_dmabuf_get_stride(QemuDmaBuf *dmabuf);
+uint32_t qemu_dmabuf_get_fourcc(QemuDmaBuf *dmabuf);
+uint64_t qemu_dmabuf_get_modifier(QemuDmaBuf *dmabuf);
+uint32_t qemu_dmabuf_get_texture(QemuDmaBuf *dmabuf);
+uint32_t qemu_dmabuf_get_x(QemuDmaBuf *dmabuf);
+uint32_t qemu_dmabuf_get_y(QemuDmaBuf *dmabuf);
+uint32_t qemu_dmabuf_get_backing_width(QemuDmaBuf *dmabuf);
+uint32_t qemu_dmabuf_get_backing_height(QemuDmaBuf *dmabuf);
+bool qemu_dmabuf_get_y0_top(QemuDmaBuf *dmabuf);
+void *qemu_dmabuf_get_sync(QemuDmaBuf *dmabuf);
+int32_t qemu_dmabuf_get_fence_fd(QemuDmaBuf *dmabuf);
+bool qemu_dmabuf_get_allow_fences(QemuDmaBuf *dmabuf);
+bool qemu_dmabuf_get_draw_submitted(QemuDmaBuf *dmabuf);
 void dpy_gl_release_dmabuf(QemuConsole *con,
QemuDmaBuf *dmabuf);
 void dpy_gl_update(QemuConsole *con,
diff --git a/hw/display/vhost-user-gpu.c b/hw/display/vhost-user-gpu.c
index 709c8a02a1..ea9a6c5d10 100644
--- a/hw/display/vhost-user-gpu.c
+++ b/hw/display/vhost-user-gpu.c
@@ -249,6 +249,7 @@ vhost_user_gpu_handle_display(VhostUserGPU *g, 
VhostUserGpuMsg *msg)
 case VHOST_USER_GPU_DMABUF_SCANOUT: {
 VhostUserGpuDMABUFScanout *m = >payload.dmabuf_scanout;
 int fd = qemu_chr_fe_get_msgfd(>vhost_chr);
+int old_fd;
 QemuDmaBuf *dmabuf;
 
 if (m->scanout_id >= g->parent_obj.conf.max_outputs) {
@@ -262,8 +263,9 @@ vhost_user_gpu_handle_display(VhostUserGPU *g, 
VhostUserGpuMsg *msg)
 g->parent_obj.enable = 1;
 con = g->parent_obj.scanout[m->scanout_id].con;
 dmabuf = >dmabuf[m->scanout_id];
-if (dmabuf->fd >= 0) {
-close(dmabuf->fd);
+old_fd = qemu_dmabuf_get_fd(dmabuf);
+if (old_fd >= 0) {
+close(old_fd);
 dmabuf->fd = -1;
 }
 dpy_gl_release_dmabuf(con, dmabuf);
diff --git a/hw/display/virtio-gpu-udmabuf.c b/hw/display/virtio-gpu-udmabuf.c
index d51184d658..c90eba281e 100644
--- a/hw/display/virtio-gpu-udmabuf.c
+++ b/hw/display/virtio-gpu-udmabuf.c
@@ -206,6 +206,7 @@ int virtio_gpu_update_dmabuf(VirtIOGPU *g,
 {
 struct virtio_gpu_scanout *scanout = >parent_obj.scanout[scanout_id];
 VGPUDMABuf *new_primary, *old_primary = NULL;
+uint32_t width, height;
 
 new_primary = virtio_gpu_create_dmabuf(g, scanout_id, res, fb, r);
 if (!new_primary) {
@@ -216,10 +217,10 @@ int virtio_gpu_update_dmabuf(VirtIOGPU *g,
 old_primary = g->dmabuf.primary[scanout_id];
 }
 
+width = qemu_dmabuf_get_width(_primary->buf);
+height = qemu_dmabuf_get_height(_primary->buf);
 g->dmabuf.primary[scanout_id] = new_primary;
-qemu_console_resize(scanout->con,
-new_primary->buf.width,
-new_primary->buf.height);
+qemu_console_resize(scanout->con, width, height);
 dpy_gl_scanout_dmabuf(scanout->con, _primary->buf);
 
 if (old_primary) {
diff --git a/hw/vfio/display.c b/hw/vfio/display.c
index 1aa440c663..4861c8161d 100644
--- a/hw/vfio/display.c
+++ b/hw/vfio/display.c
@@ -259,9 +259,13 @@ static VFIODMABuf *vfio_display_get_dmabuf(VFIOPCIDevice 
*vdev,
 
 static void

[PATCH v7 3/5] ui/console: Introduce qemu_dmabuf_set_..() helpers

2024-04-18 Thread dongwon . kim 
From: Dongwon Kim 

To enhance security in accessing the QemuDmaBuf struct, new helper
functions for setting specific fields within the struct were introduced.
And all occurrences where these fields were previously set directly
have been updated to utilize these helper functions.

v7: removed prefix, "dpy_gl_" from all helpers

Suggested-by: Marc-André Lureau 
Cc: Philippe Mathieu-Daudé 
Cc: Daniel P. Berrangé 
Cc: Vivek Kasireddy 
Signed-off-by: Dongwon Kim 
---
 include/ui/console.h |  5 +
 ui/console.c | 30 ++
 ui/egl-helpers.c | 16 +---
 ui/gtk-egl.c |  4 ++--
 ui/gtk-gl-area.c |  4 ++--
 ui/gtk.c |  6 +++---
 6 files changed, 51 insertions(+), 14 deletions(-)

diff --git a/include/ui/console.h b/include/ui/console.h
index d845f67182..92a7d0d468 100644
--- a/include/ui/console.h
+++ b/include/ui/console.h
@@ -375,6 +375,11 @@ void *qemu_dmabuf_get_sync(QemuDmaBuf *dmabuf);
 int32_t qemu_dmabuf_get_fence_fd(QemuDmaBuf *dmabuf);
 bool qemu_dmabuf_get_allow_fences(QemuDmaBuf *dmabuf);
 bool qemu_dmabuf_get_draw_submitted(QemuDmaBuf *dmabuf);
+void qemu_dmabuf_set_texture(QemuDmaBuf *dmabuf, uint32_t texture);
+void qemu_dmabuf_set_fence_fd(QemuDmaBuf *dmabuf, int32_t fence_fd);
+void qemu_dmabuf_set_sync(QemuDmaBuf *dmabuf, void *sync);
+void qemu_dmabuf_set_draw_submitted(QemuDmaBuf *dmabuf, bool draw_submitted);
+void qemu_dmabuf_set_fd(QemuDmaBuf *dmabuf, int32_t fd);
 void dpy_gl_release_dmabuf(QemuConsole *con,
QemuDmaBuf *dmabuf);
 void dpy_gl_update(QemuConsole *con,
diff --git a/ui/console.c b/ui/console.c
index 1bba3324d5..bd16ea7a4a 100644
--- a/ui/console.c
+++ b/ui/console.c
@@ -1244,6 +1244,36 @@ bool qemu_dmabuf_get_draw_submitted(QemuDmaBuf *dmabuf)
 return dmabuf->draw_submitted;
 }
 
+void qemu_dmabuf_set_texture(QemuDmaBuf *dmabuf, uint32_t texture)
+{
+assert(dmabuf != NULL);
+dmabuf->texture = texture;
+}
+
+void qemu_dmabuf_set_fence_fd(QemuDmaBuf *dmabuf, int32_t fence_fd)
+{
+assert(dmabuf != NULL);
+dmabuf->fence_fd = fence_fd;
+}
+
+void qemu_dmabuf_set_sync(QemuDmaBuf *dmabuf, void *sync)
+{
+assert(dmabuf != NULL);
+dmabuf->sync = sync;
+}
+
+void qemu_dmabuf_set_draw_submitted(QemuDmaBuf *dmabuf, bool draw_submitted)
+{
+assert(dmabuf != NULL);
+dmabuf->draw_submitted = draw_submitted;
+}
+
+void qemu_dmabuf_set_fd(QemuDmaBuf *dmabuf, int32_t fd)
+{
+assert(dmabuf != NULL);
+dmabuf->fd = fd;
+}
+
 void dpy_gl_release_dmabuf(QemuConsole *con,
   QemuDmaBuf *dmabuf)
 {
diff --git a/ui/egl-helpers.c b/ui/egl-helpers.c
index 3f96e63d25..99b2ebbe23 100644
--- a/ui/egl-helpers.c
+++ b/ui/egl-helpers.c
@@ -348,8 +348,8 @@ void egl_dmabuf_import_texture(QemuDmaBuf *dmabuf)
 return;
 }
 
-glGenTextures(1, >texture);
-texture = qemu_dmabuf_get_texture(dmabuf);
+glGenTextures(1, );
+qemu_dmabuf_set_texture(dmabuf, texture);
 glBindTexture(GL_TEXTURE_2D, texture);
 glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
 glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
@@ -368,7 +368,7 @@ void egl_dmabuf_release_texture(QemuDmaBuf *dmabuf)
 }
 
 glDeleteTextures(1, );
-dmabuf->texture = 0;
+qemu_dmabuf_set_texture(dmabuf, 0);
 }
 
 void egl_dmabuf_create_sync(QemuDmaBuf *dmabuf)
@@ -382,7 +382,7 @@ void egl_dmabuf_create_sync(QemuDmaBuf *dmabuf)
 sync = eglCreateSyncKHR(qemu_egl_display,
 EGL_SYNC_NATIVE_FENCE_ANDROID, NULL);
 if (sync != EGL_NO_SYNC_KHR) {
-dmabuf->sync = sync;
+qemu_dmabuf_set_sync(dmabuf, sync);
 }
 }
 }
@@ -390,12 +390,14 @@ void egl_dmabuf_create_sync(QemuDmaBuf *dmabuf)
 void egl_dmabuf_create_fence(QemuDmaBuf *dmabuf)
 {
 void *sync = qemu_dmabuf_get_sync(dmabuf);
+int fence_fd;
 
 if (sync) {
-dmabuf->fence_fd = eglDupNativeFenceFDANDROID(qemu_egl_display,
-  sync);
+fence_fd = eglDupNativeFenceFDANDROID(qemu_egl_display,
+  sync);
+qemu_dmabuf_set_fence_fd(dmabuf, fence_fd);
 eglDestroySyncKHR(qemu_egl_display, sync);
-dmabuf->sync = NULL;
+qemu_dmabuf_set_sync(dmabuf, NULL);
 }
 }
 
diff --git a/ui/gtk-egl.c b/ui/gtk-egl.c
index 7a45daefa1..ec0bf45482 100644
--- a/ui/gtk-egl.c
+++ b/ui/gtk-egl.c
@@ -87,7 +87,7 @@ void gd_egl_draw(VirtualConsole *vc)
 if (!qemu_dmabuf_get_draw_submitted(dmabuf)) {
 return;
 } else {
-dmabuf->draw_submitted = false;
+qemu_dmabuf_set_draw_submitted(dmabuf, false);
 }
 }
 #endif
@@ -381,7 +381,7 @@ void gd_egl_flush(DisplayChangeListener *dcl,
 if (vc->gfx.guest_fb.dmabuf &&
 !qemu_dmabuf_get_draw_submitted(vc->gfx.guest_fb.dmabuf)) {

[PATCH v7 0/5] ui/console: Private QemuDmaBuf struct

2024-04-18 Thread dongwon . kim 
From: Dongwon Kim 

This series introduces privacy enhancements to the QemuDmaBuf struct
and its contained data to bolster security. it accomplishes this by
introducing of helper functions for allocating, deallocating, and
accessing individual fields within the struct and replacing all direct
references to individual fields in the struct with methods using helpers
throughout the codebase.

This change was made based on a suggestion from Marc-André Lureau


(Resumitting same patch series with this new cover-leter)

v6: fixed some typos in patch -
ui/console: Introduce dpy_gl_qemu_dmabuf_get_..() helpers)

v7: included minor fix (ui/gtk: Check if fence_fd is equal to or greater than 0)
(Marc-André Lureau )

migrated all helpers and QemuDmaBuf struct into dmabuf.c and their 
prototypes
to dmabuf.h for better encapsulation (ui/dmabuf: New dmabuf.c and 
dmabuf.h..)
(Daniel P. Berrangé  and
 Marc-André Lureau )

removed 'dpy_gl' from all helpers' names
Defined autoptr clean up function for QemuDmaBuf*
(Daniel P. Berrangé )

Minor corrections

Dongwon Kim (5):
  ui/gtk: Check if fence_fd is equal to or greater than 0
  ui/console: Introduce qemu_dmabuf_get_..() helpers
  ui/console: Introduce qemu_dmabuf_set_..() helpers
  ui/console: Introduce qemu_dmabuf_new() and free() helpers
  ui/dmabuf: New dmabuf.c and dmabuf.h for QemuDmaBuf struct and helpers

 include/hw/vfio/vfio-common.h   |   2 +-
 include/hw/virtio/virtio-gpu.h  |   4 +-
 include/ui/console.h|  21 +--
 include/ui/dmabuf.h |  38 ++
 hw/display/vhost-user-gpu.c |  32 +++--
 hw/display/virtio-gpu-udmabuf.c |  27 ++--
 hw/vfio/display.c   |  35 ++---
 ui/console.c|   5 +-
 ui/dbus-console.c   |   9 +-
 ui/dbus-listener.c  |  71 +-
 ui/dmabuf.c | 223 
 ui/egl-headless.c   |  23 +++-
 ui/egl-helpers.c|  59 +
 ui/gtk-egl.c|  52 +---
 ui/gtk-gl-area.c|  41 --
 ui/gtk.c|  12 +-
 ui/spice-display.c  |  50 ---
 ui/meson.build  |   1 +
 18 files changed, 512 insertions(+), 193 deletions(-)
 create mode 100644 include/ui/dmabuf.h
 create mode 100644 ui/dmabuf.c

-- 
2.34.1

[PATCH v7 1/5] ui/gtk: Check if fence_fd is equal to or greater than 0

2024-04-18 Thread dongwon . kim 
From: Dongwon Kim 

'fence_fd' needs to be validated always before being referenced
And the passing condition should include '== 0' as 0 is a valid
value for the file descriptor.

Suggested-by: Marc-André Lureau 
Cc: Philippe Mathieu-Daudé 
Cc: Daniel P. Berrangé 
Cc: Vivek Kasireddy 
Signed-off-by: Dongwon Kim 
---
 ui/gtk-egl.c |  2 +-
 ui/gtk-gl-area.c |  2 +-
 ui/gtk.c | 10 ++
 3 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/ui/gtk-egl.c b/ui/gtk-egl.c
index 3af5ac5bcf..955234429d 100644
--- a/ui/gtk-egl.c
+++ b/ui/gtk-egl.c
@@ -99,7 +99,7 @@ void gd_egl_draw(VirtualConsole *vc)
 #ifdef CONFIG_GBM
 if (dmabuf) {
 egl_dmabuf_create_fence(dmabuf);
-if (dmabuf->fence_fd > 0) {
+if (dmabuf->fence_fd >= 0) {
 qemu_set_fd_handler(dmabuf->fence_fd, gd_hw_gl_flushed, NULL, 
vc);
 return;
 }
diff --git a/ui/gtk-gl-area.c b/ui/gtk-gl-area.c
index 52dcac161e..7fffd0544e 100644
--- a/ui/gtk-gl-area.c
+++ b/ui/gtk-gl-area.c
@@ -86,7 +86,7 @@ void gd_gl_area_draw(VirtualConsole *vc)
 #ifdef CONFIG_GBM
 if (dmabuf) {
 egl_dmabuf_create_fence(dmabuf);
-if (dmabuf->fence_fd > 0) {
+if (dmabuf->fence_fd >= 0) {
 qemu_set_fd_handler(dmabuf->fence_fd, gd_hw_gl_flushed, NULL, 
vc);
 return;
 }
diff --git a/ui/gtk.c b/ui/gtk.c
index 810d7fc796..7819a86321 100644
--- a/ui/gtk.c
+++ b/ui/gtk.c
@@ -597,10 +597,12 @@ void gd_hw_gl_flushed(void *vcon)
 VirtualConsole *vc = vcon;
 QemuDmaBuf *dmabuf = vc->gfx.guest_fb.dmabuf;
 
-qemu_set_fd_handler(dmabuf->fence_fd, NULL, NULL, NULL);
-close(dmabuf->fence_fd);
-dmabuf->fence_fd = -1;
-graphic_hw_gl_block(vc->gfx.dcl.con, false);
+if (dmabuf->fence_fd >= 0) {
+qemu_set_fd_handler(dmabuf->fence_fd, NULL, NULL, NULL);
+close(dmabuf->fence_fd);
+dmabuf->fence_fd = -1;
+graphic_hw_gl_block(vc->gfx.dcl.con, false);
+}
 }
 
 /** DisplayState Callbacks (opengl version) **/
-- 
2.34.1

[PATCH] hw/sparc64: set iommu_platform=on for virtio devices attached to the sun4u machine

2024-04-18 Thread Mark Cave-Ayland 
The sun4u machine has an IOMMU and therefore it is possible to program it such
that the virtio-device IOVA does not map directly to the CPU physical address.

This is not a problem with Linux which always maps the IOVA directly to the CPU
physical address, however it is required for the NetBSD virtio driver where this
is not the case.

Set the sun4u machine defaults for all virtio devices so that disable-legacy=on
and iommu_platform=on to ensure a default configuration will allow virtio
devices to function correctly on both Linux and NetBSD.

Signed-off-by: Mark Cave-Ayland 
---
 hw/sparc64/sun4u.c | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/hw/sparc64/sun4u.c b/hw/sparc64/sun4u.c
index cff6d5abaf..4ece1ac1ff 100644
--- a/hw/sparc64/sun4u.c
+++ b/hw/sparc64/sun4u.c
@@ -793,6 +793,12 @@ static void sun4v_init(MachineState *machine)
 sun4uv_init(get_system_memory(), machine, [1]);
 }
 
+static GlobalProperty hw_compat_sparc64[] = {
+{ "virtio-pci", "disable-legacy", "on", .optional = true },
+{ "virtio-device", "iommu_platform", "on" },
+};
+static const size_t hw_compat_sparc64_len = G_N_ELEMENTS(hw_compat_sparc64);
+
 static void sun4u_class_init(ObjectClass *oc, void *data)
 {
 MachineClass *mc = MACHINE_CLASS(oc);
@@ -810,6 +816,7 @@ static void sun4u_class_init(ObjectClass *oc, void *data)
 mc->default_nic = "sunhme";
 mc->no_parallel = !module_object_class_by_name(TYPE_ISA_PARALLEL);
 fwc->get_dev_path = sun4u_fw_dev_path;
+compat_props_add(mc->compat_props, hw_compat_sparc64, 
hw_compat_sparc64_len);
 }
 
 static const TypeInfo sun4u_type = {
-- 
2.39.2

Re: [RFC 1/2] iova_tree: add an id member to DMAMap

2024-04-18 Thread Si-Wei Liu 




On 4/10/2024 3:03 AM, Eugenio Pérez wrote:

IOVA tree is also used to track the mappings of virtio-net shadow
virtqueue.  This mappings may not match with the GPA->HVA ones.

This causes a problem when overlapped regions (different GPA but same
translated HVA) exists in the tree, as looking them by HVA will return
them twice.  To solve this, create an id member so we can assign unique
identifiers (GPA) to the maps.

Signed-off-by: Eugenio Pérez 
---
  include/qemu/iova-tree.h | 5 +++--
  util/iova-tree.c | 3 ++-
  2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/include/qemu/iova-tree.h b/include/qemu/iova-tree.h
index 2a10a7052e..34ee230e7d 100644
--- a/include/qemu/iova-tree.h
+++ b/include/qemu/iova-tree.h
@@ -36,6 +36,7 @@ typedef struct DMAMap {
  hwaddr iova;
  hwaddr translated_addr;
  hwaddr size;/* Inclusive */
+uint64_t id;
  IOMMUAccessFlags perm;
  } QEMU_PACKED DMAMap;
  typedef gboolean (*iova_tree_iterator)(DMAMap *map);
@@ -100,8 +101,8 @@ const DMAMap *iova_tree_find(const IOVATree *tree, const 
DMAMap *map);
   * @map: the mapping to search
   *
   * Search for a mapping in the iova tree that translated_addr overlaps with 
the
- * mapping range specified.  Only the first found mapping will be
- * returned.
+ * mapping range specified and map->id is equal.  Only the first found
+ * mapping will be returned.
   *
   * Return: DMAMap pointer if found, or NULL if not found.  Note that
   * the returned DMAMap pointer is maintained internally.  User should
diff --git a/util/iova-tree.c b/util/iova-tree.c
index 536789797e..0863e0a3b8 100644
--- a/util/iova-tree.c
+++ b/util/iova-tree.c
@@ -97,7 +97,8 @@ static gboolean iova_tree_find_address_iterator(gpointer key, 
gpointer value,
  
  needle = args->needle;

  if (map->translated_addr + map->size < needle->translated_addr ||
-needle->translated_addr + needle->size < map->translated_addr) {
+needle->translated_addr + needle->size < map->translated_addr ||
+needle->id != map->id) {


It looks this iterator can also be invoked by SVQ from 
vhost_svq_translate_addr() -> iova_tree_find_iova(), where guest GPA 
space will be searched on without passing in the ID (GPA), and exact 
match for the same GPA range is not actually needed unlike the mapping 
removal case. Could we create an API variant, for the SVQ lookup case 
specifically? Or alternatively, add a special flag, say skip_id_match to 
DMAMap, and the id match check may look like below:


(!needle->skip_id_match && needle->id != map->id)

I think vhost_svq_translate_addr() could just call the API variant or 
pass DMAmap with skip_id_match set to true to svq_iova_tree_find_iova().


Thanks,
-Siwei

  return false;
  }

Re: [PATCH] hw/core/clock: always iterate through childs in clock_propagate_period

2024-04-18 Thread Raphael Poggi 
Hi Philippe,

Le jeu. 18 avr. 2024 à 20:43, Philippe Mathieu-Daudé
 a écrit :
>
> Hi Raphael,
>
> On 18/4/24 21:16, Raphael Poggi wrote:
> > When dealing with few clocks depending with each others, sometimes
> > we might only want to update the multiplier/diviser on a specific clock
> > (cf clockB in drawing below) and call "clock_propagate(clockA)" to
> > update the childs period according to the potential new multiplier/diviser 
> > values.
> >
> > ++ ++  ++
> > | clockA | --> | clockB |  --> | clockC |
> > ++ ++  ++
> >
> > The actual code would not allow that because, since we cannot call
> > "clock_propagate" directly on a child, it would exit on the
> > first child has the period has not changed for clockB, only clockC is
>
> Typo "as the period has not changed"?

That's a typo indeed, thanks!

>
> Why can't you call clock_propagate() on a child?

There is an assert "assert(clk->source == NULL);" in clock_propagate().
If I am not wrong, clk->source is set when the clock has a parent.

>
> > impacted in our example.
> >
> > Signed-off-by: Raphael Poggi 
> > ---
> >   hw/core/clock.c | 3 ++-
> >   1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/hw/core/clock.c b/hw/core/clock.c
> > index a19c7db7df..85421f8b55 100644
> > --- a/hw/core/clock.c
> > +++ b/hw/core/clock.c
> > @@ -101,8 +101,9 @@ static void clock_propagate_period(Clock *clk, bool 
> > call_callbacks)
> >   if (call_callbacks) {
> >   clock_call_callback(child, ClockUpdate);
> >   }
> > -clock_propagate_period(child, call_callbacks);
> >   }
> > +
> > +clock_propagate_period(child, call_callbacks);
> >   }
> >   }
> >
>

Re: [PATCH 0/5] Sparc CPU naming and help text improvements

2024-04-18 Thread Mark Cave-Ayland 

On 18/04/2024 21:08, Mark Cave-Ayland wrote:


On 15/04/2024 08:26, Thomas Huth wrote:


On 07/03/2024 18.43, Thomas Huth wrote:

The Sparc CPU naming and the corresponding help text is somewhat
confusing for the users. We should avoid spaces in the Names and
provide clear information to the users what can be passed to the
"-cpu" option.
While we're at it, also remove the "+" from two of the CPU names
since this character is now not allowed in device names anymore
(and was worked around with an ugly hack in qom/object.c so far).

Thomas Huth (5):
   target/sparc/cpu: Rename the CPU models with a "+" in their names
   target/sparc/cpu: Avoid spaces by default in the CPU names
   target/sparc/cpu: Improve the CPU help text
   docs/system/target-sparc: Improve the Sparc documentation
   docs/about: Deprecate the old "UltraSparc" CPU names that contain a
 "+"


Ping!

Mark, Aryom, could you please comment on this patch series, too?

Thanks,
  Thomas


Done! I didn't realise that it was only patches 1 and 2 that were still outstanding 
until I tested the series, so I've replied to those separately.


Oops actually that's not quite right: looks like my git-am failed on patch 3 because 
it was already applied, but the remaining 2 patches are still relevant. I've just 
replied to those too.



ATB,

Mark.

Re: [PATCH 5/5] docs/about: Deprecate the old "UltraSparc" CPU names that contain a "+"

2024-04-18 Thread Mark Cave-Ayland 

On 07/03/2024 17:43, Thomas Huth wrote:


For consistency we should drop the names with a "+" in it in the
long run.

Signed-off-by: Thomas Huth 
---
  docs/about/deprecated.rst | 9 +
  1 file changed, 9 insertions(+)

diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst
index 8565644da6..7058341f8f 100644
--- a/docs/about/deprecated.rst
+++ b/docs/about/deprecated.rst
@@ -202,6 +202,15 @@ in the QEMU object model anymore. ``power5+``, 
``power5+_v2.1``,
  an alias, but for consistency these will get removed in a future
  release, too. Use ``power5p_v2.1`` and ``power7p_v2.1`` instead.
  
+``Sun-UltraSparc-IIIi+`` and ``Sun-UltraSparc-IV+`` CPU names (since 9.0)

+'
+
+The character "+" in device (and thus also CPU) names is not allowed
+in the QEMU object model anymore. ``Sun-UltraSparc-IIIi+`` and
+``Sun-UltraSparc-IV+`` are currently still supported via a workaround,
+but for consistency these will get removed in a future release, too.
+Use ``Sun-UltraSparc-IIIip`` and ``Sun-UltraSparc-IVp`` instead.
+
  CRIS CPU architecture (since 9.0)
  '


See my previous comment about the CPU names, otherwise:

Reviewed-by: Mark Cave-Ayland 


ATB,

Mark.

Re: [PATCH 4/5] docs/system/target-sparc: Improve the Sparc documentation

2024-04-18 Thread Mark Cave-Ayland 

On 07/03/2024 17:43, Thomas Huth wrote:


Add some words about how to enable or disable boolean features,
and remove the note about a Linux kernel being available on the
QEMU website (they have been removed long ago already).

Signed-off-by: Thomas Huth 
---
  docs/system/target-sparc.rst | 8 ++--
  1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/docs/system/target-sparc.rst b/docs/system/target-sparc.rst
index 9ec8c90c14..9f418b9d3e 100644
--- a/docs/system/target-sparc.rst
+++ b/docs/system/target-sparc.rst
@@ -27,6 +27,11 @@ architecture machines:
  The emulation is somewhat complete. SMP up to 16 CPUs is supported, but
  Linux limits the number of usable CPUs to 4.
  
+The list of available CPUs can be viewed by starting QEMU with ``-cpu help``.

+Optional boolean features can be added with a "+" in front of the feature name,
+or disabled with a "-" in front of the name, for example
+``-cpu TI-SuperSparc-II,+float128``.
+
  QEMU emulates the following sun4m peripherals:
  
  -  IOMMU

@@ -55,8 +60,7 @@ OpenBIOS is a free (GPL v2) portable firmware implementation. 
The goal
  is to implement a 100% IEEE 1275-1994 (referred to as Open Firmware)
  compliant firmware.
  
-A sample Linux 2.6 series kernel and ram disk image are available on the

-QEMU web site. There are still issues with NetBSD and OpenBSD, but most
+There are still issues with NetBSD and OpenBSD, but most
  kernel versions work. Please note that currently older Solaris kernels
  don't work probably due to interface issues between OpenBIOS and
  Solaris.


Just curious as to what current issues exist with NetBSD and OpenBSD? At least both 
my NetBSD and OpenBSD test images survive a casual boot test here with latest git.



ATB,

Mark.

Re: [PATCH 0/5] Sparc CPU naming and help text improvements

2024-04-18 Thread Mark Cave-Ayland 

On 15/04/2024 08:26, Thomas Huth wrote:


On 07/03/2024 18.43, Thomas Huth wrote:

The Sparc CPU naming and the corresponding help text is somewhat
confusing for the users. We should avoid spaces in the Names and
provide clear information to the users what can be passed to the
"-cpu" option.
While we're at it, also remove the "+" from two of the CPU names
since this character is now not allowed in device names anymore
(and was worked around with an ugly hack in qom/object.c so far).

Thomas Huth (5):
   target/sparc/cpu: Rename the CPU models with a "+" in their names
   target/sparc/cpu: Avoid spaces by default in the CPU names
   target/sparc/cpu: Improve the CPU help text
   docs/system/target-sparc: Improve the Sparc documentation
   docs/about: Deprecate the old "UltraSparc" CPU names that contain a
 "+"


Ping!

Mark, Aryom, could you please comment on this patch series, too?

Thanks,
  Thomas


Done! I didn't realise that it was only patches 1 and 2 that were still outstanding 
until I tested the series, so I've replied to those separately.



ATB,

Mark.

[Stable-7.2.11 45/59] hw/display/virtio-gpu: Protect from DMA re-entrancy bugs

2024-04-18 Thread Michael Tokarev 
From: Philippe Mathieu-Daudé 

Replace qemu_bh_new_guarded() by virtio_bh_new_guarded()
so the bus and device use the same guard. Otherwise the
DMA-reentrancy protection can be bypassed:

  $ cat << EOF | qemu-system-i386 -display none -nodefaults \
  -machine q35,accel=qtest \
  -m 512M \
  -device virtio-gpu \
  -qtest stdio
  outl 0xcf8 0x8820
  outl 0xcfc 0xe0004000
  outl 0xcf8 0x8804
  outw 0xcfc 0x06
  write 0xe0004030 0x4 0x024000e0
  write 0xe0004028 0x1 0xff
  write 0xe0004020 0x4 0x9300
  write 0xe000401c 0x1 0x01
  write 0x101 0x1 0x04
  write 0x103 0x1 0x1c
  write 0x9301c8 0x1 0x18
  write 0x105 0x1 0x1c
  write 0x107 0x1 0x1c
  write 0x109 0x1 0x1c
  write 0x10b 0x1 0x00
  write 0x10d 0x1 0x00
  write 0x10f 0x1 0x00
  write 0x111 0x1 0x00
  write 0x113 0x1 0x00
  write 0x115 0x1 0x00
  write 0x117 0x1 0x00
  write 0x119 0x1 0x00
  write 0x11b 0x1 0x00
  write 0x11d 0x1 0x00
  write 0x11f 0x1 0x00
  write 0x121 0x1 0x00
  write 0x123 0x1 0x00
  write 0x125 0x1 0x00
  write 0x127 0x1 0x00
  write 0x129 0x1 0x00
  write 0x12b 0x1 0x00
  write 0x12d 0x1 0x00
  write 0x12f 0x1 0x00
  write 0x131 0x1 0x00
  write 0x133 0x1 0x00
  write 0x135 0x1 0x00
  write 0x137 0x1 0x00
  write 0x139 0x1 0x00
  write 0xe0007003 0x1 0x00
  EOF
  ...
  =
  ==276099==ERROR: AddressSanitizer: heap-use-after-free on address 
0x60d11178
  at pc 0x562cc3b736c7 bp 0x7ffed49dee60 sp 0x7ffed49dee58
  READ of size 8 at 0x60d11178 thread T0
  #0 0x562cc3b736c6 in virtio_gpu_ctrl_response 
hw/display/virtio-gpu.c:180:42
  #1 0x562cc3b7c40b in virtio_gpu_ctrl_response_nodata 
hw/display/virtio-gpu.c:192:5
  #2 0x562cc3b7c40b in virtio_gpu_simple_process_cmd 
hw/display/virtio-gpu.c:1015:13
  #3 0x562cc3b82873 in virtio_gpu_process_cmdq 
hw/display/virtio-gpu.c:1050:9
  #4 0x562cc4a85514 in aio_bh_call util/async.c:169:5
  #5 0x562cc4a85c52 in aio_bh_poll util/async.c:216:13
  #6 0x562cc4a1a79b in aio_dispatch util/aio-posix.c:423:5
  #7 0x562cc4a8a2da in aio_ctx_dispatch util/async.c:358:5
  #8 0x7f36840547a8 in g_main_context_dispatch 
(/lib/x86_64-linux-gnu/libglib-2.0.so.0+0x547a8)
  #9 0x562cc4a8b753 in glib_pollfds_poll util/main-loop.c:290:9
  #10 0x562cc4a8b753 in os_host_main_loop_wait util/main-loop.c:313:5
  #11 0x562cc4a8b753 in main_loop_wait util/main-loop.c:592:11
  #12 0x562cc3938186 in qemu_main_loop system/runstate.c:782:9
  #13 0x562cc43b7af5 in qemu_default_main system/main.c:37:14
  #14 0x7f3683a6c189 in __libc_start_call_main 
csu/../sysdeps/nptl/libc_start_call_main.h:58:16
  #15 0x7f3683a6c244 in __libc_start_main csu/../csu/libc-start.c:381:3
  #16 0x562cc2a58ac0 in _start (qemu-system-i386+0x231bac0)

  0x60d11178 is located 56 bytes inside of 136-byte region 
[0x60d11140,0x60d111c8)
  freed by thread T0 here:
  #0 0x562cc2adb662 in __interceptor_free (qemu-system-i386+0x239e662)
  #1 0x562cc3b86b21 in virtio_gpu_reset hw/display/virtio-gpu.c:1524:9
  #2 0x562cc416e20e in virtio_reset hw/virtio/virtio.c:2145:9
  #3 0x562cc37c5644 in virtio_pci_reset hw/virtio/virtio-pci.c:2249:5
  #4 0x562cc4233758 in memory_region_write_accessor system/memory.c:497:5
  #5 0x562cc4232eea in access_with_adjusted_size system/memory.c:573:18

  previously allocated by thread T0 here:
  #0 0x562cc2adb90e in malloc (qemu-system-i386+0x239e90e)
  #1 0x7f368405a678 in g_malloc 
(/lib/x86_64-linux-gnu/libglib-2.0.so.0+0x5a678)
  #2 0x562cc4163ffc in virtqueue_split_pop hw/virtio/virtio.c:1612:12
  #3 0x562cc4163ffc in virtqueue_pop hw/virtio/virtio.c:1783:16
  #4 0x562cc3b91a95 in virtio_gpu_handle_ctrl 
hw/display/virtio-gpu.c:1112:15
  #5 0x562cc4a85514 in aio_bh_call util/async.c:169:5
  #6 0x562cc4a85c52 in aio_bh_poll util/async.c:216:13
  #7 0x562cc4a1a79b in aio_dispatch util/aio-posix.c:423:5

  SUMMARY: AddressSanitizer: heap-use-after-free hw/display/virtio-gpu.c:180:42 
in virtio_gpu_ctrl_response

With this change, the same reproducer triggers:

  qemu-system-i386: warning: Blocked re-entrant IO on MemoryRegion: 
virtio-pci-common-virtio-gpu at addr: 0x6

Fixes: CVE-2024-3446
Cc: qemu-sta...@nongnu.org
Reported-by: Alexander Bulekov 
Reported-by: Yongkang Jia 
Reported-by: Xiao Lei 
Reported-by: Yiming Tao 
Buglink: https://bugs.launchpad.net/qemu/+bug/1888606
Reviewed-by: Gerd Hoffmann 
Acked-by: Michael S. Tsirkin 
Signed-off-by: Philippe Mathieu-Daudé 
Reviewed-by: Michael S. Tsirkin 
Message-Id: <20240409105537.18308-3-phi...@linaro.org>
(cherry picked from commit ba28e0ff4d95b56dc334aac2730ab3651ffc3132)
Signed-off-by: Michael Tokarev 
(Mjt: context fixup in hw/display/virtio-gpu.c:virtio_gpu_device_realize()
 due to missing v8.1.0-rc2-69-ga41e2d97f92b
 "virtio-gpu:

[Stable-7.2.11 50/59] hw/block/nand: Have blk_load() take unsigned offset and return boolean

2024-04-18 Thread Michael Tokarev 
From: Philippe Mathieu-Daudé 

Negative offset is meaningless, use unsigned type.
Return a boolean value indicating success.

Reviewed-by: Richard Henderson 
Reviewed-by: Kevin Wolf 
Signed-off-by: Philippe Mathieu-Daudé 
Message-Id: <20240409135944.24997-3-phi...@linaro.org>
(cherry picked from commit 2e3e09b368001f7eaeeca7a9b49cb1f0c9092d85)
Signed-off-by: Michael Tokarev 

diff --git a/hw/block/nand.c b/hw/block/nand.c
index 4e3d7fb065..81b2bb804d 100644
--- a/hw/block/nand.c
+++ b/hw/block/nand.c
@@ -84,7 +84,11 @@ struct NANDFlashState {
 
 void (*blk_write)(NANDFlashState *s);
 void (*blk_erase)(NANDFlashState *s);
-void (*blk_load)(NANDFlashState *s, uint64_t addr, int offset);
+/*
+ * Returns %true when block containing (@addr + @offset) is
+ * successfully loaded, otherwise %false.
+ */
+bool (*blk_load)(NANDFlashState *s, uint64_t addr, unsigned offset);
 
 uint32_t ioaddr_vmstate;
 };
@@ -772,11 +776,11 @@ static void glue(nand_blk_erase_, 
NAND_PAGE_SIZE)(NANDFlashState *s)
 }
 }
 
-static void glue(nand_blk_load_, NAND_PAGE_SIZE)(NANDFlashState *s,
-uint64_t addr, int offset)
+static bool glue(nand_blk_load_, NAND_PAGE_SIZE)(NANDFlashState *s,
+ uint64_t addr, unsigned 
offset)
 {
 if (PAGE(addr) >= s->pages) {
-return;
+return false;
 }
 
 if (s->blk) {
@@ -804,6 +808,8 @@ static void glue(nand_blk_load_, 
NAND_PAGE_SIZE)(NANDFlashState *s,
 offset, NAND_PAGE_SIZE + OOB_SIZE - offset);
 s->ioaddr = s->io;
 }
+
+return true;
 }
 
 static void glue(nand_init_, NAND_PAGE_SIZE)(NANDFlashState *s)
-- 
2.39.2

[RFC] linux-newbie thread on dev setup using Qemu

2024-04-18 Thread Josh Marshall 
Hello all,

I started a thread a thread at
https://lore.kernel.org/linux-newbie/CAFkJGRcg+ThJ-xUve0=WorChW=-6prelhxem8ywtwzwpkht...@mail.gmail.com/T/#m563ab52dd2bfe7ac8d85f838f866a1a9d1a6d0a9
.  It is using Qemu, but both Linux and Qemu and rather complex pieces
of software.  I was wondering if someone could chime in and tell if
there would be any better ways to use Qemu in this situation or make
Linux and Qemu work better together since the kernel is being compiled
for such a specific situation.

[Stable-7.2.11 56/59] hw/sd/sdhci: Do not update TRNMOD when Command Inhibit (DAT) is set

2024-04-18 Thread Michael Tokarev 
From: Philippe Mathieu-Daudé 

Per "SD Host Controller Standard Specification Version 3.00":

  * 2.2.5 Transfer Mode Register (Offset 00Ch)

Writes to this register shall be ignored when the Command
Inhibit (DAT) in the Present State register is 1.

Do not update the TRNMOD register when Command Inhibit (DAT)
bit is set to avoid the present-status register going out of
sync, leading to malicious guest using DMA mode and overflowing
the FIFO buffer:

  $ cat << EOF | qemu-system-i386 \
 -display none -nographic -nodefaults \
 -machine accel=qtest -m 512M \
 -device sdhci-pci,sd-spec-version=3 \
 -device sd-card,drive=mydrive \
 -drive 
if=none,index=0,file=null-co://,format=raw,id=mydrive \
 -qtest stdio
  outl 0xcf8 0x80001013
  outl 0xcfc 0x91
  outl 0xcf8 0x80001001
  outl 0xcfc 0x0600
  write 0x912c 0x1 0x05
  write 0x9158 0x1 0x16
  write 0x9105 0x1 0x04
  write 0x9128 0x1 0x08
  write 0x16 0x1 0x21
  write 0x19 0x1 0x20
  write 0x910c 0x1 0x01
  write 0x910e 0x1 0x20
  write 0x910f 0x1 0x00
  write 0x910c 0x1 0x00
  write 0x9120 0x1 0x00
  EOF

Stack trace (part):
=
==89993==ERROR: AddressSanitizer: heap-buffer-overflow on address
0x61529900 at pc 0x55d5f885700d bp 0x7ffc1e1e9470 sp 0x7ffc1e1e9468
WRITE of size 1 at 0x61529900 thread T0
#0 0x55d5f885700c in sdhci_write_dataport hw/sd/sdhci.c:564:39
#1 0x55d5f8849150 in sdhci_write hw/sd/sdhci.c:1223:13
#2 0x55d5fa01db63 in memory_region_write_accessor system/memory.c:497:5
#3 0x55d5fa01d245 in access_with_adjusted_size system/memory.c:573:18
#4 0x55d5fa01b1a9 in memory_region_dispatch_write system/memory.c:1521:16
#5 0x55d5fa09f5c9 in flatview_write_continue system/physmem.c:2711:23
#6 0x55d5fa08f78b in flatview_write system/physmem.c:2753:12
#7 0x55d5fa08f258 in address_space_write system/physmem.c:2860:18
...
0x61529900 is located 0 bytes to the right of 512-byte region
[0x61529700,0x61529900) allocated by thread T0 here:
#0 0x55d5f7237b27 in __interceptor_calloc
#1 0x7f9e36dd4c50 in g_malloc0
#2 0x55d5f88672f7 in sdhci_pci_realize hw/sd/sdhci-pci.c:36:5
#3 0x55d5f844b582 in pci_qdev_realize hw/pci/pci.c:2092:9
#4 0x55d5fa2ee74b in device_set_realized hw/core/qdev.c:510:13
#5 0x55d5fa325bfb in property_set_bool qom/object.c:2358:5
#6 0x55d5fa31ea45 in object_property_set qom/object.c:1472:5
#7 0x55d5fa332509 in object_property_set_qobject om/qom-qobject.c:28:10
#8 0x55d5fa31f6ed in object_property_set_bool qom/object.c:1541:15
#9 0x55d5fa2e2948 in qdev_realize hw/core/qdev.c:292:12
#10 0x55d5f8eed3f1 in qdev_device_add_from_qdict 
system/qdev-monitor.c:719:10
#11 0x55d5f8eef7ff in qdev_device_add system/qdev-monitor.c:738:11
#12 0x55d5f8f211f0 in device_init_func system/vl.c:1200:11
#13 0x55d5fad0877d in qemu_opts_foreach util/qemu-option.c:1135:14
#14 0x55d5f8f0df9c in qemu_create_cli_devices system/vl.c:2638:5
#15 0x55d5f8f0db24 in qmp_x_exit_preconfig system/vl.c:2706:5
#16 0x55d5f8f14dc0 in qemu_init system/vl.c:3737:9
...
SUMMARY: AddressSanitizer: heap-buffer-overflow hw/sd/sdhci.c:564:39
in sdhci_write_dataport

Add assertions to ensure the fifo_buffer[] is not overflowed by
malicious accesses to the Buffer Data Port register.

Fixes: CVE-2024-3447
Cc: qemu-sta...@nongnu.org
Fixes: d7dfca0807 ("hw/sdhci: introduce standard SD host controller")
Buglink: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=58813
Reported-by: Alexander Bulekov 
Reported-by: Chuhong Yuan 
Signed-off-by: Peter Maydell 
Message-Id: 
Signed-off-by: Philippe Mathieu-Daudé 
Message-Id: <20240409145524.27913-1-phi...@linaro.org>
(cherry picked from commit 9e4b27ca6bf4974f169bbca7f3dca117b1208b6f)
Signed-off-by: Michael Tokarev 

diff --git a/hw/sd/sdhci.c b/hw/sd/sdhci.c
index ef60badc6b..abd503d168 100644
--- a/hw/sd/sdhci.c
+++ b/hw/sd/sdhci.c
@@ -473,6 +473,7 @@ static uint32_t sdhci_read_dataport(SDHCIState *s, unsigned 
size)
 }
 
 for (i = 0; i < size; i++) {
+assert(s->data_count < s->buf_maxsz);
 value |= s->fifo_buffer[s->data_count] << i * 8;
 s->data_count++;
 /* check if we've read all valid data (blksize bytes) from buffer */
@@ -561,6 +562,7 @@ static void sdhci_write_dataport(SDHCIState *s, uint32_t 
value, unsigned size)
 }
 
 for (i = 0; i < size; i++) {
+assert(s->data_count < s->buf_maxsz);
 s->fifo_buffer[s->data_count] = value & 0xFF;
 s->data_count++;
 value >>= 8;
@@ -1208,6 +1210,12 @@ sdhci_write(void *opaque, hwaddr offset, uint64_t val, 
unsigned size)
 if (!(s->capareg & R_SDHC_CAPAB_SDMA_MASK)) {
 value &= ~SDHC_TRNS_DMA;
 }
+
+/* TRNMOD writes are inhibited while

[Stable-7.2.11 52/59] hw/misc/applesmc: Fix memory leak in reset() handler

2024-04-18 Thread Michael Tokarev 
From: Philippe Mathieu-Daudé 

AppleSMCData is allocated with g_new0() in applesmc_add_key():
release it with g_free().

Leaked since commit 1ddda5cd36 ("AppleSMC device emulation").

Cc: qemu-sta...@nongnu.org
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2272
Reported-by: Zheyu Ma 
Signed-off-by: Philippe Mathieu-Daudé 
Reviewed-by: Peter Maydell 
Message-Id: <20240408095217.57239-3-phi...@linaro.org>
(cherry picked from commit fc09ff2979defdcf8d00c2db94022d5d610e36ba)
Signed-off-by: Michael Tokarev 

diff --git a/hw/misc/applesmc.c b/hw/misc/applesmc.c
index 5f9c742e50..80642efc57 100644
--- a/hw/misc/applesmc.c
+++ b/hw/misc/applesmc.c
@@ -273,6 +273,7 @@ static void qdev_applesmc_isa_reset(DeviceState *dev)
 /* Remove existing entries */
 QLIST_FOREACH_SAFE(d, >data_def, node, next) {
 QLIST_REMOVE(d, node);
+g_free(d);
 }
 s->status = 0x00;
 s->status_1e = 0x00;
-- 
2.39.2

[Stable-7.2.11 49/59] hw/block/nand: Factor nand_load_iolen() method out

2024-04-18 Thread Michael Tokarev 
From: Philippe Mathieu-Daudé 

Reviewed-by: Richard Henderson 
Reviewed-by: Kevin Wolf 
Signed-off-by: Philippe Mathieu-Daudé 
Message-Id: <20240409135944.24997-2-phi...@linaro.org>
(cherry picked from commit 7a86544f286d8af4fa5251101c1026ddae92cc3d)
Signed-off-by: Michael Tokarev 

diff --git a/hw/block/nand.c b/hw/block/nand.c
index 1aee1cb2b1..4e3d7fb065 100644
--- a/hw/block/nand.c
+++ b/hw/block/nand.c
@@ -243,9 +243,28 @@ static inline void nand_pushio_byte(NANDFlashState *s, 
uint8_t value)
 }
 }
 
+/*
+ * nand_load_block: Load block containing (s->addr + @offset).
+ * Returns length of data available at @offset in this block.
+ */
+static unsigned nand_load_block(NANDFlashState *s, unsigned offset)
+{
+unsigned iolen;
+
+s->blk_load(s, s->addr, offset);
+
+iolen = (1 << s->page_shift);
+if (s->gnd) {
+iolen += 1 << s->oob_shift;
+}
+assert(offset <= iolen);
+iolen -= offset;
+
+return iolen;
+}
+
 static void nand_command(NANDFlashState *s)
 {
-unsigned int offset;
 switch (s->cmd) {
 case NAND_CMD_READ0:
 s->iolen = 0;
@@ -271,12 +290,7 @@ static void nand_command(NANDFlashState *s)
 case NAND_CMD_NOSERIALREAD2:
 if (!(nand_flash_ids[s->chip_id].options & NAND_SAMSUNG_LP))
 break;
-offset = s->addr & ((1 << s->addr_shift) - 1);
-s->blk_load(s, s->addr, offset);
-if (s->gnd)
-s->iolen = (1 << s->page_shift) - offset;
-else
-s->iolen = (1 << s->page_shift) + (1 << s->oob_shift) - offset;
+s->iolen = nand_load_block(s, s->addr & ((1 << s->addr_shift) - 1));
 break;
 
 case NAND_CMD_RESET:
@@ -597,12 +611,7 @@ uint32_t nand_getio(DeviceState *dev)
 if (!s->iolen && s->cmd == NAND_CMD_READ0) {
 offset = (int) (s->addr & ((1 << s->addr_shift) - 1)) + s->offset;
 s->offset = 0;
-
-s->blk_load(s, s->addr, offset);
-if (s->gnd)
-s->iolen = (1 << s->page_shift) - offset;
-else
-s->iolen = (1 << s->page_shift) + (1 << s->oob_shift) - offset;
+s->iolen = nand_load_block(s, offset);
 }
 
 if (s->ce || s->iolen <= 0) {
-- 
2.39.2

[Stable-7.2.11 54/59] hw/net/lan9118: Fix overflow in MIL TX FIFO

2024-04-18 Thread Michael Tokarev 
From: Philippe Mathieu-Daudé 

When the MAC Interface Layer (MIL) transmit FIFO is full,
truncate the packet, and raise the Transmitter Error (TXE)
flag.

Broken since model introduction in commit 2a42499017
("LAN9118 emulation").

When using the reproducer from
https://gitlab.com/qemu-project/qemu/-/issues/2267 we get:

  hw/net/lan9118.c:798:17: runtime error:
  index 2048 out of bounds for type 'uint8_t[2048]' (aka 'unsigned char[2048]')
    #0 0x563ec9a057b1 in tx_fifo_push hw/net/lan9118.c:798:43
    #1 0x563ec99fbb28 in lan9118_writel hw/net/lan9118.c:1042:9
    #2 0x563ec99f2de2 in lan9118_16bit_mode_write hw/net/lan9118.c:1205:9
    #3 0x563ecbf78013 in memory_region_write_accessor system/memory.c:497:5
    #4 0x563ecbf776f5 in access_with_adjusted_size system/memory.c:573:18
    #5 0x563ecbf75643 in memory_region_dispatch_write system/memory.c:1521:16
    #6 0x563ecc01bade in flatview_write_continue_step system/physmem.c:2713:18
    #7 0x563ecc01b374 in flatview_write_continue system/physmem.c:2743:19
    #8 0x563ecbff1c9b in flatview_write system/physmem.c:2774:12
    #9 0x563ecbff1768 in address_space_write system/physmem.c:2894:18
...

[*] LAN9118 DS2266B.pdf, Table 5.3.3 "INTERRUPT STATUS REGISTER"

Cc: qemu-sta...@nongnu.org
Reported-by: Will Lester
Reported-by: Chuhong Yuan 
Suggested-by: Peter Maydell 
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2267
Signed-off-by: Philippe Mathieu-Daudé 
Reviewed-by: Peter Maydell 
Message-Id: <20240409133801.23503-3-phi...@linaro.org>
(cherry picked from commit ad766d603f39888309cfb1433ba2de1d0e9e4f58)
Signed-off-by: Michael Tokarev 

diff --git a/hw/net/lan9118.c b/hw/net/lan9118.c
index 00a6d82efb..bf81c84984 100644
--- a/hw/net/lan9118.c
+++ b/hw/net/lan9118.c
@@ -798,8 +798,22 @@ static void tx_fifo_push(lan9118_state *s, uint32_t val)
 /* Documentation is somewhat unclear on the ordering of bytes
in FIFO words.  Empirical results show it to be little-endian.
*/
-/* TODO: FIFO overflow checking.  */
 while (n--) {
+if (s->txp->len == MIL_TXFIFO_SIZE) {
+/*
+ * No more space in the FIFO. The datasheet is not
+ * precise about this case. We choose what is easiest
+ * to model: the packet is truncated, and TXE is raised.
+ *
+ * Note, it could be a fragmented packet, but we currently
+ * do not handle that (see earlier TX_B case).
+ */
+qemu_log_mask(LOG_GUEST_ERROR,
+  "MIL TX FIFO overrun, discarding %u 
byte%s\n",
+  n, n > 1 ? "s" : "");
+s->int_sts |= TXE_INT;
+break;
+}
 s->txp->data[s->txp->len] = val & 0xff;
 s->txp->len++;
 val >>= 8;
-- 
2.39.2

Re: [PATCH 1/5] target/sparc/cpu: Rename the CPU models with a "+" in their names

2024-04-18 Thread Mark Cave-Ayland 

On 07/03/2024 17:43, Thomas Huth wrote:


Commit b447378e12 ("qom/object: Limit type names to alphanumerical ...")
cut down the amount of allowed characters for QOM types to a saner set.
The "+" character was not meant to be included in this set, so we had
to add a hack there to still allow the legacy names of POWER and Sparc64
CPUs. However, instead of putting such a hack in the common QOM code,
there is a much better place to do this: The sparc_cpu_class_by_name()
function which is used to look up the names of all Sparc CPUs.
Thus let's finally get rid of the "+" in the Sparc CPU names, and provide
backward compatibility for the old names via some simple checks in the
sparc_cpu_class_by_name() function.

Signed-off-by: Thomas Huth 
---
  qom/object.c   |  8 
  target/sparc/cpu.c | 14 --
  2 files changed, 12 insertions(+), 10 deletions(-)

diff --git a/qom/object.c b/qom/object.c
index d4a001cf41..759e194972 100644
--- a/qom/object.c
+++ b/qom/object.c
@@ -158,14 +158,6 @@ static bool type_name_is_valid(const char *name)
  "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
  "0123456789-_.");
  
-/* Allow some legacy names with '+' in it for compatibility reasons */

-if (name[plen] == '+') {
-if (plen >= 17 && g_str_has_prefix(name, "Sun-UltraSparc-I")) {
-/* Allow "Sun-UltraSparc-IV+" and "Sun-UltraSparc-IIIi+" */
-return true;
-}
-}
-
  return plen == slen;
  }
  
diff --git a/target/sparc/cpu.c b/target/sparc/cpu.c

index 313ebc4c11..651e49bfeb 100644
--- a/target/sparc/cpu.c
+++ b/target/sparc/cpu.c
@@ -316,7 +316,7 @@ static const sparc_def_t sparc_defs[] = {
  .features = CPU_DEFAULT_FEATURES,
  },
  {
-.name = "Sun UltraSparc IV+",
+.name = "Sun UltraSparc IVp",
  .iu_version = ((0x3eULL << 48) | (0x19ULL << 32) | (0x22ULL << 24)),
  .fpu_version = 0x,
  .mmu_version = mmu_us_12,
@@ -325,7 +325,7 @@ static const sparc_def_t sparc_defs[] = {
  .features = CPU_DEFAULT_FEATURES | CPU_FEATURE_CMT,
  },
  {
-.name = "Sun UltraSparc IIIi+",
+.name = "Sun UltraSparc IIIip",
  .iu_version = ((0x3eULL << 48) | (0x22ULL << 32) | (0ULL << 24)),
  .fpu_version = 0x,
  .mmu_version = mmu_us_3,
@@ -767,6 +767,16 @@ static ObjectClass *sparc_cpu_class_by_name(const char 
*cpu_model)
  char *typename;
  
  typename = sparc_cpu_type_name(cpu_model);

+
+/* Fix up legacy names with '+' in it */
+if (g_str_equal(typename, SPARC_CPU_TYPE_NAME("Sun-UltraSparc-IV+"))) {
+g_free(typename);
+typename = g_strdup(SPARC_CPU_TYPE_NAME("Sun-UltraSparc-IVp"));
+} else if (g_str_equal(typename, 
SPARC_CPU_TYPE_NAME("Sun-UltraSparc-IIIi+"))) {
+g_free(typename);
+typename = g_strdup(SPARC_CPU_TYPE_NAME("Sun-UltraSparc-IIIip"));
+}
+
  oc = object_class_by_name(typename);
  g_free(typename);
  return oc;


I've seen some references in Sun documentation to processors in the form "UltraSparc 
IIIi plus" so I'd be inclined to use that form for the new type names e.g. 
"UltraSparc-IIIi-plus".


Otherwise looks good to me, thanks for having a look at this!

Reviewed-by: Mark Cave-Ayland 


ATB,

Mark.

Re: [PATCH 2/5] target/sparc/cpu: Avoid spaces by default in the CPU names

2024-04-18 Thread Mark Cave-Ayland 

On 07/03/2024 17:43, Thomas Huth wrote:


The output of "-cpu help" is currently rather confusing to the users:
It is not clear which part of the output defines the CPU names since
the CPU names contain white spaces (which we later have to convert
into dashes internally) For example:

Sparc TI UltraSparc II IU 001700112000 FPU  MMU  NWINS 8

At a first glance, should the name for -cpu be "Sparc TI Ultrasparc II"
or "TI UltraSparc II IU" here? Both would be wrong, the right guess is
"TI UltraSparc II" only. Let's start cleaning up this mess by using
dashes instead of white spaces for the CPU names, like we're doing it
internally later (and like we're doing it in most other targets of QEMU).
Note that it is still possible to pass the CPU names with spaces to the
"-cpu" option, since sparc_cpu_type_name() still translates those to "-".

Buglink: https://gitlab.com/qemu-project/qemu/-/issues/2141
Signed-off-by: Thomas Huth 
---
  target/sparc/cpu.c | 56 +++---
  1 file changed, 28 insertions(+), 28 deletions(-)

diff --git a/target/sparc/cpu.c b/target/sparc/cpu.c
index 651e49bfeb..ae30cded22 100644
--- a/target/sparc/cpu.c
+++ b/target/sparc/cpu.c
@@ -208,7 +208,7 @@ void cpu_sparc_set_id(CPUSPARCState *env, unsigned int cpu)
  static const sparc_def_t sparc_defs[] = {
  #ifdef TARGET_SPARC64
  {
-.name = "Fujitsu Sparc64",
+.name = "Fujitsu-Sparc64",
  .iu_version = ((0x04ULL << 48) | (0x02ULL << 32) | (0ULL << 24)),
  .fpu_version = 0x,
  .mmu_version = mmu_us_12,
@@ -217,7 +217,7 @@ static const sparc_def_t sparc_defs[] = {
  .features = CPU_DEFAULT_FEATURES,
  },
  {
-.name = "Fujitsu Sparc64 III",
+.name = "Fujitsu-Sparc64-III",
  .iu_version = ((0x04ULL << 48) | (0x03ULL << 32) | (0ULL << 24)),
  .fpu_version = 0x,
  .mmu_version = mmu_us_12,
@@ -226,7 +226,7 @@ static const sparc_def_t sparc_defs[] = {
  .features = CPU_DEFAULT_FEATURES,
  },
  {
-.name = "Fujitsu Sparc64 IV",
+.name = "Fujitsu-Sparc64-IV",
  .iu_version = ((0x04ULL << 48) | (0x04ULL << 32) | (0ULL << 24)),
  .fpu_version = 0x,
  .mmu_version = mmu_us_12,
@@ -235,7 +235,7 @@ static const sparc_def_t sparc_defs[] = {
  .features = CPU_DEFAULT_FEATURES,
  },
  {
-.name = "Fujitsu Sparc64 V",
+.name = "Fujitsu-Sparc64-V",
  .iu_version = ((0x04ULL << 48) | (0x05ULL << 32) | (0x51ULL << 24)),
  .fpu_version = 0x,
  .mmu_version = mmu_us_12,
@@ -244,7 +244,7 @@ static const sparc_def_t sparc_defs[] = {
  .features = CPU_DEFAULT_FEATURES,
  },
  {
-.name = "TI UltraSparc I",
+.name = "TI-UltraSparc-I",
  .iu_version = ((0x17ULL << 48) | (0x10ULL << 32) | (0x40ULL << 24)),
  .fpu_version = 0x,
  .mmu_version = mmu_us_12,
@@ -253,7 +253,7 @@ static const sparc_def_t sparc_defs[] = {
  .features = CPU_DEFAULT_FEATURES,
  },
  {
-.name = "TI UltraSparc II",
+.name = "TI-UltraSparc-II",
  .iu_version = ((0x17ULL << 48) | (0x11ULL << 32) | (0x20ULL << 24)),
  .fpu_version = 0x,
  .mmu_version = mmu_us_12,
@@ -262,7 +262,7 @@ static const sparc_def_t sparc_defs[] = {
  .features = CPU_DEFAULT_FEATURES,
  },
  {
-.name = "TI UltraSparc IIi",
+.name = "TI-UltraSparc-IIi",
  .iu_version = ((0x17ULL << 48) | (0x12ULL << 32) | (0x91ULL << 24)),
  .fpu_version = 0x,
  .mmu_version = mmu_us_12,
@@ -271,7 +271,7 @@ static const sparc_def_t sparc_defs[] = {
  .features = CPU_DEFAULT_FEATURES,
  },
  {
-.name = "TI UltraSparc IIe",
+.name = "TI-UltraSparc-IIe",
  .iu_version = ((0x17ULL << 48) | (0x13ULL << 32) | (0x14ULL << 24)),
  .fpu_version = 0x,
  .mmu_version = mmu_us_12,
@@ -280,7 +280,7 @@ static const sparc_def_t sparc_defs[] = {
  .features = CPU_DEFAULT_FEATURES,
  },
  {
-.name = "Sun UltraSparc III",
+.name = "Sun-UltraSparc-III",
  .iu_version = ((0x3eULL << 48) | (0x14ULL << 32) | (0x34ULL << 24)),
  .fpu_version = 0x,
  .mmu_version = mmu_us_12,
@@ -289,7 +289,7 @@ static const sparc_def_t sparc_defs[] = {
  .features = CPU_DEFAULT_FEATURES,
  },
  {
-.name = "Sun UltraSparc III Cu",
+.name = "Sun-UltraSparc-III-Cu",
  .iu_version = ((0x3eULL << 48) | (0x15ULL << 32) | (0x41ULL << 24)),
  .fpu_version = 0x,
  .mmu_version = mmu_us_3,
@@ -298,7 +298,7 @@ static const sparc_def_t sparc_defs[] = {
  .features = CPU_DEFAULT_FEATURES,
  },
  {
-.name = "Sun UltraSparc IIIi",
+.name = "Sun-UltraSparc-IIIi",
  .iu_version =

[Stable-7.2.11 57/59] target/sh4: add missing CHECK_NOT_DELAY_SLOT

2024-04-18 Thread Michael Tokarev 
From: Zack Buhman 

CHECK_NOT_DELAY_SLOT is correctly applied to the branch-related
instructions, but not to the PC-relative mov* instructions.

I verified the existence of an illegal slot exception on a SH7091 when
any of these instructions are attempted inside a delay slot.

This also matches the behavior described in the SH-4 ISA manual.

Signed-off-by: Zack Buhman 
Reviewed-by: Richard Henderson 
Message-Id: <20240407150705.5965-1-z...@buhman.org>
Signed-off-by: Richard Henderson 
Reviewd-by: Yoshinori Sato 
(cherry picked from commit b754cb2dcde26a7bc8a9d17bb6900a0ac0dd38e2)
Signed-off-by: Michael Tokarev 
(Mjt: trivial context (whitespace before comments) fixup)

diff --git a/target/sh4/translate.c b/target/sh4/translate.c
index 7db3468b01..8d6eae7ddf 100644
--- a/target/sh4/translate.c
+++ b/target/sh4/translate.c
@@ -528,6 +528,7 @@ static void _decode_opc(DisasContext * ctx)
tcg_gen_movi_i32(REG(B11_8), B7_0s);
return;
 case 0x9000:   /* mov.w @(disp,PC),Rn */
+CHECK_NOT_DELAY_SLOT
{
 TCGv addr = tcg_const_i32(ctx->base.pc_next + 4 + B7_0 * 2);
 tcg_gen_qemu_ld_i32(REG(B11_8), addr, ctx->memidx, MO_TESW);
@@ -535,6 +536,7 @@ static void _decode_opc(DisasContext * ctx)
}
return;
 case 0xd000:   /* mov.l @(disp,PC),Rn */
+CHECK_NOT_DELAY_SLOT
{
 TCGv addr = tcg_const_i32((ctx->base.pc_next + 4 + B7_0 * 4) & ~3);
 tcg_gen_qemu_ld_i32(REG(B11_8), addr, ctx->memidx, MO_TESL);
@@ -1295,6 +1297,7 @@ static void _decode_opc(DisasContext * ctx)
}
return;
 case 0xc700:   /* mova @(disp,PC),R0 */
+CHECK_NOT_DELAY_SLOT
 tcg_gen_movi_i32(REG(0), ((ctx->base.pc_next & 0xfffc) +
   4 + B7_0 * 4) & ~3);
return;
-- 
2.39.2

[Stable-7.2.11 48/59] qemu-options: Fix CXL Fixed Memory Window interleave-granularity typo

2024-04-18 Thread Michael Tokarev 
From: Yuquan Wang 

Fix the unit typo of interleave-granularity of CXL Fixed Memory
Window in qemu-option.hx.

Fixes: 03b39fcf64 ("hw/cxl: Make the CFMW a machine parameter.")
Signed-off-by: Yuquan Wang wangyuquan1...@phytium.com.cn
Message-ID: <20240407083539.1488172-2-wangyuquan1...@phytium.com.cn>
[PMD: Reworded]
Signed-off-by: Philippe Mathieu-Daudé 
(cherry picked from commit aa88f99c87c0e5d195d6d96190374650553ea61f)
Signed-off-by: Michael Tokarev 

diff --git a/qemu-options.hx b/qemu-options.hx
index 7f798ce47e..2c00ceac83 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -149,14 +149,14 @@ SRST
 platform and configuration dependent.
 
 ``interleave-granularity=granularity`` sets the granularity of
-interleave. Default 256KiB. Only 256KiB, 512KiB, 1024KiB, 2048KiB
-4096KiB, 8192KiB and 16384KiB granularities supported.
+interleave. Default 256 (bytes). Only 256, 512, 1k, 2k,
+4k, 8k and 16k granularities supported.
 
 Example:
 
 ::
 
--machine 
cxl-fmw.0.targets.0=cxl.0,cxl-fmw.0.targets.1=cxl.1,cxl-fmw.0.size=128G,cxl-fmw.0.interleave-granularity=512k
+-machine 
cxl-fmw.0.targets.0=cxl.0,cxl-fmw.0.targets.1=cxl.1,cxl-fmw.0.size=128G,cxl-fmw.0.interleave-granularity=512
 ERST
 
 DEF("M", HAS_ARG, QEMU_OPTION_M,
-- 
2.39.2

[Stable-7.2.11 53/59] backends/cryptodev: Do not abort for invalid session ID

2024-04-18 Thread Michael Tokarev 
From: Philippe Mathieu-Daudé 

Instead of aborting when a session ID is invalid,
return VIRTIO_CRYPTO_INVSESS ("Invalid session id").

Reproduced using:

  $ cat << EOF | qemu-system-i386 -display none \
 -machine q35,accel=qtest -m 512M -nodefaults \
 -object cryptodev-backend-builtin,id=cryptodev0 \
 -device virtio-crypto-pci,id=crypto0,cryptodev=cryptodev0 \
 -qtest stdio
  outl 0xcf8 0x8804
  outw 0xcfc 0x06
  outl 0xcf8 0x8820
  outl 0xcfc 0xe0008000
  write 0x10800e 0x1 0x01
  write 0xe0008016 0x1 0x01
  write 0xe0008020 0x4 0x00801000
  write 0xe0008028 0x4 0x00c01000
  write 0xe000801c 0x1 0x01
  write 0x11 0x1 0x05
  write 0x110001 0x1 0x04
  write 0x108002 0x1 0x11
  write 0x108008 0x1 0x48
  write 0x10800c 0x1 0x01
  write 0x108018 0x1 0x10
  write 0x10801c 0x1 0x02
  write 0x10c002 0x1 0x01
  write 0xe000b005 0x1 0x00
  EOF
  Assertion failed: (session_id < MAX_NUM_SESSIONS && 
builtin->sessions[session_id]),
  function cryptodev_builtin_close_session, file cryptodev-builtin.c, line 430.

Cc: qemu-sta...@nongnu.org
Reported-by: Zheyu Ma 
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2274
Signed-off-by: Philippe Mathieu-Daudé 
Reviewed-by: zhenwei pi 
Message-Id: <20240409094757.9127-1-phi...@linaro.org>
(cherry picked from commit eaf2bd29538d039df80bb4b1584de33a61312bc6)
Signed-off-by: Michael Tokarev 

diff --git a/backends/cryptodev-builtin.c b/backends/cryptodev-builtin.c
index cda6ca3b71..2e792be756 100644
--- a/backends/cryptodev-builtin.c
+++ b/backends/cryptodev-builtin.c
@@ -416,7 +416,9 @@ static int cryptodev_builtin_close_session(
   CRYPTODEV_BACKEND_BUILTIN(backend);
 CryptoDevBackendBuiltinSession *session;
 
-assert(session_id < MAX_NUM_SESSIONS && builtin->sessions[session_id]);
+if (session_id >= MAX_NUM_SESSIONS || !builtin->sessions[session_id]) {
+return -VIRTIO_CRYPTO_INVSESS;
+}
 
 session = builtin->sessions[session_id];
 if (session->cipher) {
-- 
2.39.2

[Stable-7.2.11 51/59] hw/block/nand: Fix out-of-bound access in NAND block buffer

2024-04-18 Thread Michael Tokarev 
From: Philippe Mathieu-Daudé 

nand_command() and nand_getio() don't check @offset points
into the block, nor the available data length (s->iolen) is
not negative.

In order to fix:

- check the offset is in range in nand_blk_load_NAND_PAGE_SIZE(),
- do not set @iolen if blk_load() failed.

Reproducer:

  $ cat << EOF | qemu-system-arm -machine tosa \
 -monitor none -serial none \
 -display none -qtest stdio
  write 0x1111 0x1 0xca
  write 0x1104 0x1 0x47
  write 0x1000ca04 0x1 0xd7
  write 0x1000ca01 0x1 0xe0
  write 0x1000ca04 0x1 0x71
  write 0x1000ca00 0x1 0x50
  write 0x1000ca04 0x1 0xd7
  read 0x1000ca02 0x1
  write 0x1000ca01 0x1 0x10
  EOF

=
==15750==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x61f00de0
 at pc 0x560e61557210 bp 0x7ffcfc4a59f0 sp 0x7ffcfc4a59e8
READ of size 1 at 0x61f00de0 thread T0
#0 0x560e6155720f in mem_and hw/block/nand.c:101:20
#1 0x560e6155ac9c in nand_blk_write_512 hw/block/nand.c:663:9
#2 0x560e61544200 in nand_command hw/block/nand.c:293:13
#3 0x560e6153cc83 in nand_setio hw/block/nand.c:520:13
#4 0x560e61a0a69e in tc6393xb_nand_writeb hw/display/tc6393xb.c:380:13
#5 0x560e619f9bf7 in tc6393xb_writeb hw/display/tc6393xb.c:524:9
#6 0x560e647c7d03 in memory_region_write_accessor softmmu/memory.c:492:5
#7 0x560e647c7641 in access_with_adjusted_size softmmu/memory.c:554:18
#8 0x560e647c5f66 in memory_region_dispatch_write softmmu/memory.c:1514:16
#9 0x560e6485409e in flatview_write_continue softmmu/physmem.c:2825:23
#10 0x560e648421eb in flatview_write softmmu/physmem.c:2867:12
#11 0x560e64841ca8 in address_space_write softmmu/physmem.c:2963:18
#12 0x560e61170162 in qemu_writeb tests/qtest/videzzo/videzzo_qemu.c:1080:5
#13 0x560e6116eef7 in dispatch_mmio_write 
tests/qtest/videzzo/videzzo_qemu.c:1227:28

0x61f00de0 is located 0 bytes to the right of 3424-byte region 
[0x61f00080,0x61f00de0)
allocated by thread T0 here:
#0 0x560e611276cf in malloc 
/root/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:145:3
#1 0x7f7959a87e98 in g_malloc 
(/lib/x86_64-linux-gnu/libglib-2.0.so.0+0x57e98)
#2 0x560e64b98871 in object_new qom/object.c:749:12
#3 0x560e64b5d1a1 in qdev_new hw/core/qdev.c:153:19
#4 0x560e61547ea5 in nand_init hw/block/nand.c:639:11
#5 0x560e619f8772 in tc6393xb_init hw/display/tc6393xb.c:558:16
#6 0x560e6390bad2 in tosa_init hw/arm/tosa.c:250:12

SUMMARY: AddressSanitizer: heap-buffer-overflow hw/block/nand.c:101:20 in 
mem_and
==15750==ABORTING

Broken since introduction in commit 3e3d5815cb ("NAND Flash memory
emulation and ECC calculation helpers for use by NAND controllers").

Cc: qemu-sta...@nongnu.org
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1445
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1446
Reported-by: Qiang Liu 
Reviewed-by: Richard Henderson 
Reviewed-by: Kevin Wolf 
Signed-off-by: Philippe Mathieu-Daudé 
Message-Id: <20240409135944.24997-4-phi...@linaro.org>
(cherry picked from commit d39fdfff348fdf00173b7a58e935328a64db7d28)
Signed-off-by: Michael Tokarev 

diff --git a/hw/block/nand.c b/hw/block/nand.c
index 81b2bb804d..d994bfe372 100644
--- a/hw/block/nand.c
+++ b/hw/block/nand.c
@@ -255,7 +255,9 @@ static unsigned nand_load_block(NANDFlashState *s, unsigned 
offset)
 {
 unsigned iolen;
 
-s->blk_load(s, s->addr, offset);
+if (!s->blk_load(s, s->addr, offset)) {
+return 0;
+}
 
 iolen = (1 << s->page_shift);
 if (s->gnd) {
@@ -783,6 +785,10 @@ static bool glue(nand_blk_load_, 
NAND_PAGE_SIZE)(NANDFlashState *s,
 return false;
 }
 
+if (offset > NAND_PAGE_SIZE + OOB_SIZE) {
+return false;
+}
+
 if (s->blk) {
 if (s->mem_oob) {
 if (blk_pread(s->blk, SECTOR(addr) << BDRV_SECTOR_BITS,
-- 
2.39.2

[Stable-7.2.11 58/59] ppc/spapr: Introduce SPAPR_IRQ_NR_IPIS to refer IRQ range for CPU IPIs.

2024-04-18 Thread Michael Tokarev 
From: Harsh Prateek Bora 

spapr_irq_init currently uses existing macro SPAPR_XIRQ_BASE to refer to
the range of CPU IPIs during initialization of nr-irqs property.
It is more appropriate to have its own define which can be further
reused as appropriate for correct interpretation.

Suggested-by: Cedric Le Goater 
Reviewed-by: Cédric Le Goater 
Tested-by: Kowshik Jois 
Signed-off-by: Harsh Prateek Bora 
Signed-off-by: Nicholas Piggin 
(cherry picked from commit 2df5c1f5b014126595a26c6797089d284a3b211c)
Signed-off-by: Michael Tokarev 

diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c
index a0d1e1298e..97b2fc42ab 100644
--- a/hw/ppc/spapr_irq.c
+++ b/hw/ppc/spapr_irq.c
@@ -23,6 +23,8 @@
 
 #include "trace.h"
 
+QEMU_BUILD_BUG_ON(SPAPR_IRQ_NR_IPIS > SPAPR_XIRQ_BASE);
+
 static const TypeInfo spapr_intc_info = {
 .name = TYPE_SPAPR_INTC,
 .parent = TYPE_INTERFACE,
@@ -329,7 +331,7 @@ void spapr_irq_init(SpaprMachineState *spapr, Error **errp)
 int i;
 
 dev = qdev_new(TYPE_SPAPR_XIVE);
-qdev_prop_set_uint32(dev, "nr-irqs", smc->nr_xirqs + SPAPR_XIRQ_BASE);
+qdev_prop_set_uint32(dev, "nr-irqs", smc->nr_xirqs + 
SPAPR_IRQ_NR_IPIS);
 /*
  * 8 XIVE END structures per CPU. One for each available
  * priority
@@ -356,7 +358,7 @@ void spapr_irq_init(SpaprMachineState *spapr, Error **errp)
 }
 
 spapr->qirqs = qemu_allocate_irqs(spapr_set_irq, spapr,
-  smc->nr_xirqs + SPAPR_XIRQ_BASE);
+  smc->nr_xirqs + SPAPR_IRQ_NR_IPIS);
 
 /*
  * Mostly we don't actually need this until reset, except that not
diff --git a/include/hw/ppc/spapr_irq.h b/include/hw/ppc/spapr_irq.h
index c22a72c9e2..4fd2d5853d 100644
--- a/include/hw/ppc/spapr_irq.h
+++ b/include/hw/ppc/spapr_irq.h
@@ -14,9 +14,21 @@
 #include "qom/object.h"
 
 /*
- * IRQ range offsets per device type
+ * The XIVE IRQ backend uses the same layout as the XICS backend but
+ * covers the full range of the IRQ number space. The IRQ numbers for
+ * the CPU IPIs are allocated at the bottom of this space, below 4K,
+ * to preserve compatibility with XICS which does not use that range.
+ */
+
+/*
+ * CPU IPI range (XIVE only)
  */
 #define SPAPR_IRQ_IPI0x0
+#define SPAPR_IRQ_NR_IPIS0x1000
+
+/*
+ * IRQ range offsets per device type
+ */
 
 #define SPAPR_XIRQ_BASE  XICS_IRQ_BASE /* 0x1000 */
 #define SPAPR_IRQ_EPOW   (SPAPR_XIRQ_BASE + 0x)
-- 
2.39.2

[Stable-7.2.11 42/59] tcg/optimize: Do not attempt to constant fold neg_vec

2024-04-18 Thread Michael Tokarev 
From: Richard Henderson 

Split out the tail of fold_neg to fold_neg_no_const so that we
can avoid attempting to constant fold vector negate.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2150
Signed-off-by: Richard Henderson 
(cherry picked from commit e25fe886b89a396bae5847520b70c148587d490a)
Signed-off-by: Michael Tokarev 
(Mjt: context fixup in tests/tcg/aarch64/Makefile.target)

diff --git a/tcg/optimize.c b/tcg/optimize.c
index b6f6436c74..100b75efd8 100644
--- a/tcg/optimize.c
+++ b/tcg/optimize.c
@@ -1634,16 +1634,10 @@ static bool fold_nand(OptContext *ctx, TCGOp *op)
 return false;
 }
 
-static bool fold_neg(OptContext *ctx, TCGOp *op)
+static bool fold_neg_no_const(OptContext *ctx, TCGOp *op)
 {
-uint64_t z_mask;
-
-if (fold_const1(ctx, op)) {
-return true;
-}
-
 /* Set to 1 all bits to the left of the rightmost.  */
-z_mask = arg_info(op->args[1])->z_mask;
+uint64_t z_mask = arg_info(op->args[1])->z_mask;
 ctx->z_mask = -(z_mask & -z_mask);
 
 /*
@@ -1654,6 +1648,11 @@ static bool fold_neg(OptContext *ctx, TCGOp *op)
 return true;
 }
 
+static bool fold_neg(OptContext *ctx, TCGOp *op)
+{
+return fold_const1(ctx, op) || fold_neg_no_const(ctx, op);
+}
+
 static bool fold_nor(OptContext *ctx, TCGOp *op)
 {
 if (fold_const2_commutative(ctx, op) ||
@@ -1949,7 +1948,7 @@ static bool fold_sub_to_neg(OptContext *ctx, TCGOp *op)
 if (have_neg) {
 op->opc = neg_op;
 op->args[1] = op->args[2];
-return fold_neg(ctx, op);
+return fold_neg_no_const(ctx, op);
 }
 return false;
 }
diff --git a/tests/tcg/aarch64/Makefile.target 
b/tests/tcg/aarch64/Makefile.target
index 474f61bc30..bd29446835 100644
--- a/tests/tcg/aarch64/Makefile.target
+++ b/tests/tcg/aarch64/Makefile.target
@@ -10,7 +10,7 @@ VPATH += $(AARCH64_SRC)
 
 # Base architecture tests
 AARCH64_TESTS=fcvt pcalign-a64
-AARCH64_TESTS += test-2248
+AARCH64_TESTS += test-2248 test-2150
 
 fcvt: LDFLAGS+=-lm
 
diff --git a/tests/tcg/aarch64/test-2150.c b/tests/tcg/aarch64/test-2150.c
new file mode 100644
index 00..fb86c11958
--- /dev/null
+++ b/tests/tcg/aarch64/test-2150.c
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/* See https://gitlab.com/qemu-project/qemu/-/issues/2150 */
+
+int main()
+{
+asm volatile(
+"movi v6.4s, #1\n"
+"movi v7.4s, #0\n"
+"sub  v6.2d, v7.2d, v6.2d\n"
+: : : "v6", "v7");
+return 0;
+}
-- 
2.39.2

[Stable-7.2.11 44/59] hw/virtio: Introduce virtio_bh_new_guarded() helper

2024-04-18 Thread Michael Tokarev 
From: Philippe Mathieu-Daudé 

Introduce virtio_bh_new_guarded(), similar to qemu_bh_new_guarded()
but using the transport memory guard, instead of the device one
(there can only be one virtio device per virtio bus).

Inspired-by: Gerd Hoffmann 
Reviewed-by: Gerd Hoffmann 
Acked-by: Michael S. Tsirkin 
Signed-off-by: Philippe Mathieu-Daudé 
Reviewed-by: Michael S. Tsirkin 
Message-Id: <20240409105537.18308-2-phi...@linaro.org>
(cherry picked from commit ec0504b989ca61e03636384d3602b7bf07ffe4da)
Signed-off-by: Michael Tokarev 
(Mjt: trivial #include context fixup in include/hw/virtio/virtio.h)

diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index e4f8ed1e63..4a35d7cb0c 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -5029,3 +5029,13 @@ static void virtio_register_types(void)
 }
 
 type_init(virtio_register_types)
+
+QEMUBH *virtio_bh_new_guarded_full(DeviceState *dev,
+   QEMUBHFunc *cb, void *opaque,
+   const char *name)
+{
+DeviceState *transport = qdev_get_parent_bus(dev)->parent;
+
+return qemu_bh_new_full(cb, opaque, name,
+>mem_reentrancy_guard);
+}
diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index 96a56430a6..c1a7c9bd3b 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -23,6 +23,7 @@
 #include "standard-headers/linux/virtio_ring.h"
 #include "qom/object.h"
 #include "hw/virtio/vhost.h"
+#include "block/aio.h"
 
 /*
  * A guest should never accept this. It implies negotiation is broken
@@ -463,4 +464,10 @@ static inline bool virtio_device_disabled(VirtIODevice 
*vdev)
 bool virtio_legacy_allowed(VirtIODevice *vdev);
 bool virtio_legacy_check_disabled(VirtIODevice *vdev);
 
+QEMUBH *virtio_bh_new_guarded_full(DeviceState *dev,
+   QEMUBHFunc *cb, void *opaque,
+   const char *name);
+#define virtio_bh_new_guarded(dev, cb, opaque) \
+virtio_bh_new_guarded_full((dev), (cb), (opaque), (stringify(cb)))
+
 #endif
-- 
2.39.2

[Stable-7.2.11 55/59] hw/net/lan9118: Replace magic '2048' value by MIL_TXFIFO_SIZE definition

2024-04-18 Thread Michael Tokarev 
From: Philippe Mathieu-Daudé 

The magic 2048 is explained in the LAN9211 datasheet (DS2414A)
in chapter 1.4, "10/100 Ethernet MAC":

  The MAC Interface Layer (MIL), within the MAC, contains a
  2K Byte transmit and a 128 Byte receive FIFO which is separate
  from the TX and RX FIFOs. [...]

Note, the use of the constant in lan9118_receive() reveals that
our implementation is using the same buffer for both tx and rx.

Signed-off-by: Philippe Mathieu-Daudé 
Reviewed-by: Peter Maydell 
Message-Id: <20240409133801.23503-2-phi...@linaro.org>
(cherry picked from commit a45223467e4e185fff1c76a6483784fa379ded77)
Signed-off-by: Michael Tokarev 

diff --git a/hw/net/lan9118.c b/hw/net/lan9118.c
index bf81c84984..f269d72d9e 100644
--- a/hw/net/lan9118.c
+++ b/hw/net/lan9118.c
@@ -155,6 +155,12 @@ do { fprintf(stderr, "lan9118: error: " fmt , ## 
__VA_ARGS__);} while (0)
 
 #define GPT_TIMER_EN0x2000
 
+/*
+ * The MAC Interface Layer (MIL), within the MAC, contains a 2K Byte transmit
+ * and a 128 Byte receive FIFO which is separate from the TX and RX FIFOs.
+ */
+#define MIL_TXFIFO_SIZE 2048
+
 enum tx_state {
 TX_IDLE,
 TX_B,
@@ -171,7 +177,7 @@ typedef struct {
 int32_t pad;
 int32_t fifo_used;
 int32_t len;
-uint8_t data[2048];
+uint8_t data[MIL_TXFIFO_SIZE];
 } LAN9118Packet;
 
 static const VMStateDescription vmstate_lan9118_packet = {
@@ -187,7 +193,7 @@ static const VMStateDescription vmstate_lan9118_packet = {
 VMSTATE_INT32(pad, LAN9118Packet),
 VMSTATE_INT32(fifo_used, LAN9118Packet),
 VMSTATE_INT32(len, LAN9118Packet),
-VMSTATE_UINT8_ARRAY(data, LAN9118Packet, 2048),
+VMSTATE_UINT8_ARRAY(data, LAN9118Packet, MIL_TXFIFO_SIZE),
 VMSTATE_END_OF_LIST()
 }
 };
@@ -549,7 +555,7 @@ static ssize_t lan9118_receive(NetClientState *nc, const 
uint8_t *buf,
 return -1;
 }
 
-if (size >= 2048 || size < 14) {
+if (size >= MIL_TXFIFO_SIZE || size < 14) {
 return -1;
 }
 
-- 
2.39.2

Re: [PATCH RFC 00/26] Multifd  device state transfer support with VFIO consumer

2024-04-18 Thread Peter Xu 
On Thu, Apr 18, 2024 at 08:14:15PM +0200, Maciej S. Szmigiero wrote:
> On 18.04.2024 12:39, Daniel P. Berrangé wrote:
> > On Thu, Apr 18, 2024 at 11:50:12AM +0200, Maciej S. Szmigiero wrote:
> > > On 17.04.2024 18:35, Daniel P. Berrangé wrote:
> > > > On Wed, Apr 17, 2024 at 02:11:37PM +0200, Maciej S. Szmigiero wrote:
> > > > > On 17.04.2024 10:36, Daniel P. Berrangé wrote:
> > > > > > On Tue, Apr 16, 2024 at 04:42:39PM +0200, Maciej S. Szmigiero wrote:
> > > > > > > From: "Maciej S. Szmigiero" 
> (..)
> > > > > > That said, the idea of reserving channels specifically for VFIO 
> > > > > > doesn't
> > > > > > make a whole lot of sense to me either.
> > > > > > 
> > > > > > Once we've done the RAM transfer, and are in the switchover phase
> > > > > > doing device state transfer, all the multifd channels are idle.
> > > > > > We should just use all those channels to transfer the device state,
> > > > > > in parallel.  Reserving channels just guarantees many idle channels
> > > > > > during RAM transfer, and further idle channels during vmstate
> > > > > > transfer.
> > > > > > 
> > > > > > IMHO it is more flexible to just use all available multifd channel
> > > > > > resources all the time.
> > > > > 
> > > > > The reason for having dedicated device state channels is that they
> > > > > provide lower downtime in my tests.
> > > > > 
> > > > > With either 15 or 11 mixed multifd channels (no dedicated device state
> > > > > channels) I get a downtime of about 1250 msec.
> > > > > 
> > > > > Comparing that with 15 total multifd channels / 4 dedicated device
> > > > > state channels that give downtime of about 1100 ms it means that using
> > > > > dedicated channels gets about 14% downtime improvement.
> > > > 
> > > > Hmm, can you clarify. /when/ is the VFIO vmstate transfer taking
> > > > place ? Is is transferred concurrently with the RAM ? I had thought
> > > > this series still has the RAM transfer iterations running first,
> > > > and then the VFIO VMstate at the end, simply making use of multifd
> > > > channels for parallelism of the end phase. your reply though makes
> > > > me question my interpretation though.
> > > > 
> > > > Let me try to illustrate channel flow in various scenarios, time
> > > > flowing left to right:
> > > > 
> > > > 1. serialized RAM, then serialized VM state  (ie historical migration)
> > > > 
> > > > main: | Init | RAM iter 1 | RAM iter 2 | ... | RAM iter N | VM 
> > > > State |
> > > > 
> > > > 
> > > > 2. parallel RAM, then serialized VM state (ie today's multifd)
> > > > 
> > > > main: | Init || VM 
> > > > state |
> > > > multifd1:| RAM iter 1 | RAM iter 2 | ... | RAM iter N |
> > > > multifd2:| RAM iter 1 | RAM iter 2 | ... | RAM iter N |
> > > > multifd3:| RAM iter 1 | RAM iter 2 | ... | RAM iter N |
> > > > 
> > > > 
> > > > 3. parallel RAM, then parallel VM state
> > > > 
> > > > main: | Init || VM 
> > > > state |
> > > > multifd1:| RAM iter 1 | RAM iter 2 | ... | RAM iter N |
> > > > multifd2:| RAM iter 1 | RAM iter 2 | ... | RAM iter N |
> > > > multifd3:| RAM iter 1 | RAM iter 2 | ... | RAM iter N |
> > > > multifd4: | 
> > > > VFIO VM state |
> > > > multifd5: | 
> > > > VFIO VM state |
> > > > 
> > > > 
> > > > 4. parallel RAM and VFIO VM state, then remaining VM state
> > > > 
> > > > main: | Init || VM 
> > > > state |
> > > > multifd1:| RAM iter 1 | RAM iter 2 | ... | RAM iter N |
> > > > multifd2:| RAM iter 1 | RAM iter 2 | ... | RAM iter N |
> > > > multifd3:| RAM iter 1 | RAM iter 2 | ... | RAM iter N |
> > > > multifd4:| VFIO VM state
> > > >  |
> > > > multifd5:| VFIO VM state
> > > >  |
> > > > 
> > > > 
> > > > I thought this series was implementing approx (3), but are you actually
> > > > implementing (4), or something else entirely ?
> > > 
> > > You are right that this series operation is approximately implementing
> > > the schema described as numer 3 in your diagrams.
> > 
> > > However, there are some additional details worth mentioning:
> > > * There's some but relatively small amount of VFIO data being
> > > transferred from the "save_live_iterate" SaveVMHandler while the VM is
> > > still running.
> > > 
> > > This is still happening via the main migration channel.
> > > Parallelizing this transfer in the future might make sense too,
> > > although obviously this doesn't impact the downtime.
> > > 
> > > * After the VM is stopped and downtime starts the main (~ 400 MiB)
> > > VFIO device state gets transferred via multifd channels.
> > > 
> > > However, these

[Stable-7.2.11 59/59] ppc/spapr: Initialize max_cpus limit to SPAPR_IRQ_NR_IPIS.

2024-04-18 Thread Michael Tokarev 
From: Harsh Prateek Bora 

Initialize the machine specific max_cpus limit as per the maximum range
of CPU IPIs available. Keeping between 4096 to 8192 will throw IRQ not
free error due to XIVE/XICS limitation and keeping beyond 8192 will hit
assert in tcg_region_init or spapr_xive_claim_irq.

Logs:

Without patch fix:

[root@host build]# qemu-system-ppc64 -accel tcg -smp 10,maxcpus=4097
qemu-system-ppc64: IRQ 4096 is not free
[root@host build]#

On LPAR:
[root@host build]# qemu-system-ppc64 -accel tcg -smp 10,maxcpus=8193
**
ERROR:../tcg/region.c:774:tcg_region_init: assertion failed:
(region_size >= 2 * page_size)
Bail out! ERROR:../tcg/region.c:774:tcg_region_init: assertion failed:
(region_size >= 2 * page_size)
Aborted (core dumped)
[root@host build]#

On x86:
[root@host build]# qemu-system-ppc64 -accel tcg -smp 10,maxcpus=8193
qemu-system-ppc64: ../hw/intc/spapr_xive.c:596: spapr_xive_claim_irq:
Assertion `lisn < xive->nr_irqs' failed.
Aborted (core dumped)
[root@host build]#

With patch fix:
[root@host build]# qemu-system-ppc64 -accel tcg -smp 10,maxcpus=4097
qemu-system-ppc64: Invalid SMP CPUs 4097. The max CPUs supported by
machine 'pseries-8.2' is 4096
[root@host build]#

Reported-by: Kowshik Jois 
Tested-by: Kowshik Jois 
Reviewed-by: Cédric Le Goater 
Signed-off-by: Harsh Prateek Bora 
Signed-off-by: Nicholas Piggin 
(cherry picked from commit c4f91d7b7be76c47015521ab0109c6e998a369b0)
Signed-off-by: Michael Tokarev 

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 66b414d2e9..9e860f5047 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -4602,13 +4602,10 @@ static void spapr_machine_class_init(ObjectClass *oc, 
void *data)
 mc->block_default_type = IF_SCSI;
 
 /*
- * Setting max_cpus to INT32_MAX. Both KVM and TCG max_cpus values
- * should be limited by the host capability instead of hardcoded.
- * max_cpus for KVM guests will be checked in kvm_init(), and TCG
- * guests are welcome to have as many CPUs as the host are capable
- * of emulate.
+ * While KVM determines max cpus in kvm_init() using kvm_max_vcpus(),
+ * In TCG the limit is restricted by the range of CPU IPIs available.
  */
-mc->max_cpus = INT32_MAX;
+mc->max_cpus = SPAPR_IRQ_NR_IPIS;
 
 mc->no_parallel = 1;
 mc->default_boot_order = "";
-- 
2.39.2

[Stable-7.2.11 46/59] hw/char/virtio-serial-bus: Protect from DMA re-entrancy bugs

2024-04-18 Thread Michael Tokarev 
From: Philippe Mathieu-Daudé 

Replace qemu_bh_new_guarded() by virtio_bh_new_guarded()
so the bus and device use the same guard. Otherwise the
DMA-reentrancy protection can be bypassed.

Fixes: CVE-2024-3446
Cc: qemu-sta...@nongnu.org
Suggested-by: Alexander Bulekov 
Reviewed-by: Gerd Hoffmann 
Acked-by: Michael S. Tsirkin 
Signed-off-by: Philippe Mathieu-Daudé 
Reviewed-by: Michael S. Tsirkin 
Message-Id: <20240409105537.18308-4-phi...@linaro.org>
(cherry picked from commit b4295bff25f7b50de1d9cc94a9c6effd40056bca)
Signed-off-by: Michael Tokarev 

diff --git a/hw/char/virtio-serial-bus.c b/hw/char/virtio-serial-bus.c
index dd619f0731..1221fb7f15 100644
--- a/hw/char/virtio-serial-bus.c
+++ b/hw/char/virtio-serial-bus.c
@@ -985,8 +985,7 @@ static void virtser_port_device_realize(DeviceState *dev, 
Error **errp)
 return;
 }
 
-port->bh = qemu_bh_new_guarded(flush_queued_data_bh, port,
-   >mem_reentrancy_guard);
+port->bh = virtio_bh_new_guarded(dev, flush_queued_data_bh, port);
 port->elem = NULL;
 }
 
-- 
2.39.2

[Stable-7.2.11 43/59] linux-user: Fix waitid return of siginfo_t and rusage

2024-04-18 Thread Michael Tokarev 
From: Richard Henderson 

The copy back to siginfo_t should be conditional only on arg3,
not the specific values that might have been written.
The copy back to rusage was missing entirely.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2262
Signed-off-by: Richard Henderson 
Tested-by: Alex Fan 
Reviewed-by: Philippe Mathieu-Daudé 
(cherry picked from commit f0907ff4cae743f1a4ef3d0a55a047029eed06ff)
Signed-off-by: Michael Tokarev 

diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index aead0f6ac9..41017b0df2 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -8759,14 +8759,24 @@ static abi_long do_syscall1(CPUArchState *cpu_env, int 
num, abi_long arg1,
 #ifdef TARGET_NR_waitid
 case TARGET_NR_waitid:
 {
+struct rusage ru;
 siginfo_t info;
-info.si_pid = 0;
-ret = get_errno(safe_waitid(arg1, arg2, , arg4, NULL));
-if (!is_error(ret) && arg3 && info.si_pid != 0) {
-if (!(p = lock_user(VERIFY_WRITE, arg3, 
sizeof(target_siginfo_t), 0)))
+
+ret = get_errno(safe_waitid(arg1, arg2, (arg3 ?  : NULL),
+arg4, (arg5 ?  : NULL)));
+if (!is_error(ret)) {
+if (arg3) {
+p = lock_user(VERIFY_WRITE, arg3,
+  sizeof(target_siginfo_t), 0);
+if (!p) {
+return -TARGET_EFAULT;
+}
+host_to_target_siginfo(p, );
+unlock_user(p, arg3, sizeof(target_siginfo_t));
+}
+if (arg5 && host_to_target_rusage(arg5, )) {
 return -TARGET_EFAULT;
-host_to_target_siginfo(p, );
-unlock_user(p, arg3, sizeof(target_siginfo_t));
+}
 }
 }
 return ret;
-- 
2.39.2

[Stable-7.2.11 47/59] hw/virtio/virtio-crypto: Protect from DMA re-entrancy bugs

2024-04-18 Thread Michael Tokarev 
From: Philippe Mathieu-Daudé 

Replace qemu_bh_new_guarded() by virtio_bh_new_guarded()
so the bus and device use the same guard. Otherwise the
DMA-reentrancy protection can be bypassed.

Fixes: CVE-2024-3446
Cc: qemu-sta...@nongnu.org
Suggested-by: Alexander Bulekov 
Reviewed-by: Gerd Hoffmann 
Acked-by: Michael S. Tsirkin 
Signed-off-by: Philippe Mathieu-Daudé 
Reviewed-by: Michael S. Tsirkin 
Message-Id: <20240409105537.18308-5-phi...@linaro.org>
(cherry picked from commit f4729ec39ad97a42ceaa7b5697f84f440ea6e5dc)
Signed-off-by: Michael Tokarev 

diff --git a/hw/virtio/virtio-crypto.c b/hw/virtio/virtio-crypto.c
index b2e0646d9a..ce995c66d8 100644
--- a/hw/virtio/virtio-crypto.c
+++ b/hw/virtio/virtio-crypto.c
@@ -1057,8 +1057,8 @@ static void virtio_crypto_device_realize(DeviceState 
*dev, Error **errp)
 vcrypto->vqs[i].dataq =
  virtio_add_queue(vdev, 1024, virtio_crypto_handle_dataq_bh);
 vcrypto->vqs[i].dataq_bh =
- qemu_bh_new_guarded(virtio_crypto_dataq_bh, >vqs[i],
- >mem_reentrancy_guard);
+ virtio_bh_new_guarded(dev, virtio_crypto_dataq_bh,
+   >vqs[i]);
 vcrypto->vqs[i].vcrypto = vcrypto;
 }
 
-- 
2.39.2

[Stable-7.2.11 v2 00/59] Patch Round-up for stable 7.2.11, freeze on 2024-04-20

2024-04-18 Thread Michael Tokarev 
The following patches are queued for QEMU stable v7.2.11:

  https://gitlab.com/qemu-project/qemu/-/commits/staging-7.2

Patch freeze is 2024-04-20, and the release is planned for 2024-04-22:

  https://wiki.qemu.org/Planning/7.2

Please respond here or CC qemu-sta...@nongnu.org on any additional patches
you think should (or shouldn't) be included in the release.

The changes which are staging for inclusion, with the original commit hash
from master branch, are given below the bottom line.

Thanks!

/mjt

--
01* 9ea920dc2825 Daniel P. Berrangé:
   gitlab: update FreeBSD Cirrus CI image to 13.3
02* f5af80271aad David Parsons:
   ui/cocoa: Fix window clipping on macOS 14
03* bc6bd20ee353 Zhuojia Shen:
   target/arm: align exposed ID registers with Linux
04* 3dc2afeab296 Peter Maydell:
   tests/tcg/aarch64/sysregs.c: Use S syntax for id_aa64zfr0_el1 and 
   id_aa64smfr0_el1
05* 1f51573f7925 Richard Henderson:
   target/arm: Fix SME full tile indexing
06* fd7f95f23d6f Peter Maydell:
   hw/rtc/sun4v-rtc: Relicense to GPLv2-or-later
07* 012b170173bc Dmitrii Gavrilov:
   system/qdev-monitor: move drain_call_rcu call under if (!dev) in 
   qmp_device_add()
08* a9198b3132d8 Sven Schnelle:
   hw/scsi/lsi53c895a: stop script on phase mismatch
09* 8b09b7fe4708 Sven Schnelle:
   hw/scsi/lsi53c895a: add missing decrement of reentrancy counter
10* 9876359990dd Sven Schnelle:
   hw/scsi/lsi53c895a: add timer to scripts processing
11* 9bc9e9511944 Michael Tokarev:
   make-release: switch to .xz format by default
12* 4cadf1023498 Laurent Vivier:
   e1000e: fix link state on resume
13* 6a5287ce8047 Nick Briggs:
   Avoid unaligned fetch in ladr_match()
14* 784fd35387e9 Klaus Jensen:
   hw/nvme: clean up confusing use of errp/local_err
15* 973f76cf7743 Klaus Jensen:
   hw/nvme: cleanup error reporting in nvme_init_pci()
16* 4f0a4a3d5854 Minwoo Im:
   hw/nvme: separate 'serial' property for VFs
17* ee7bda4d38cd Klaus Jensen:
   hw/nvme: generalize the mbar size helper
18* fa905f65c554 Klaus Jensen:
   hw/nvme: add machine compatibility parameter to enable msix exclusive bar
19* 31180dbdca28 Akihiko Odaki:
   pcie: Introduce pcie_sriov_num_vfs
20* 91bb64a8d201 Akihiko Odaki:
   hw/nvme: Use pcie_sriov_num_vfs()
21* 6081b4243cd6 Akihiko Odaki:
   pcie_sriov: Validate NumVFs
22* 74e2845c5f95 Jonathan Cameron:
   hmat acpi: Fix out of bounds access due to missing use of indirection
23* 2e128776dc56 Cédric Le Goater:
   migration: Skip only empty block devices
24* c45f8f1aef35 Thomas Huth:
   tests/unit: Bump test-aio-multithread test timeout to 2 minutes
25* e1b363e328d5 Thomas Huth:
   tests/unit: Bump test-crypto-block test timeout to 5 minutes
26* 63b18312d14a Kevin Wolf:
   tests/unit: Bump test-replication timeout to 60 seconds
27* 55f7c6a5f2bd Peter Maydell:
   tests: Raise timeouts for bufferiszero and crypto-tlscredsx509
28* 5f97afe2543f Paolo Bonzini:
   target/i386: introduce function to query MMU indices
29* 90f641531c78 Paolo Bonzini:
   target/i386: use separate MMU indexes for 32-bit accesses
30* 2cc68629a6fc Paolo Bonzini:
   target/i386: fix direction of "32-bit MMU" test
31* 7fd226b04746 Tao Su:
   target/i386: Revert monitor_puts() in do_inject_x86_mce()
32* 1590154ee437 Song Gao:
   target/loongarch: Fix qemu-system-loongarch64 assert failed with the 
   option '-d int'
33* 7c7a9f578e4f Lorenz Brun:
   hw/scsi/scsi-generic: Fix io_timeout property not applying
34* a158c63b3ba1 Yao Xingtao:
   monitor/hmp-cmds-target: Append a space in error message in gpa2hva()
35* 1c188fc8cbff Akihiko Odaki:
   virtio-net: Fix vhost virtqueue notifiers for RSS
36* 2911e9b95f3b Richard Henderson:
   tcg/optimize: Fix sign_mask for logical right-shift
37* 4a3aa11e1fb2 Richard Henderson:
   target/hppa: Clear psw_n for BE on use_nullify_skip path
38* 1d2f2b35bc86 Michael Tokarev:
   gitlab-ci/cirrus: switch from 'master' to 'latest'
39* 44e25fbc1900 Peter Maydell:
   hw/intc/arm_gicv3: ICC_HPPIR* return SPURIOUS if int group is disabled
40* 4c54f5bc8e1d Yajun Wu:
   hw/net/virtio-net: fix qemu set used ring flag even vhost started
41* 2d9a31b3c273 Wafer:
   hw/virtio: Fix packed virtqueue flush used_idx
42 e25fe886b89a Richard Henderson:
   tcg/optimize: Do not attempt to constant fold neg_vec
43 f0907ff4cae7 Richard Henderson:
   linux-user: Fix waitid return of siginfo_t and rusage
44 ec0504b989ca Philippe Mathieu-Daudé:
   hw/virtio: Introduce virtio_bh_new_guarded() helper
45 ba28e0ff4d95 Philippe Mathieu-Daudé:
   hw/display/virtio-gpu: Protect from DMA re-entrancy bugs
46 b4295bff25f7 Philippe Mathieu-Daudé:
   hw/char/virtio-serial-bus: Protect from DMA re-entrancy bugs
47 f4729ec39ad9 Philippe Mathieu-Daudé:
   hw/virtio/virtio-crypto: Protect from DMA re-entrancy bugs
48 aa88f99c87c0 Yuquan Wang:
   qemu-options: Fix CXL Fixed Memory Window interleave-granularity typo
49 7a86544f286d Philippe Mathieu-Daudé:
   hw/block/nand: Factor nand_load_iolen() method out
50 2e3e09b36800 Philippe

Re: [PATCH] hw/core/clock: always iterate through childs in clock_propagate_period

2024-04-18 Thread Philippe Mathieu-Daudé 

Hi Raphael,

On 18/4/24 21:16, Raphael Poggi wrote:

When dealing with few clocks depending with each others, sometimes
we might only want to update the multiplier/diviser on a specific clock
(cf clockB in drawing below) and call "clock_propagate(clockA)" to
update the childs period according to the potential new multiplier/diviser 
values.

++ ++  ++
| clockA | --> | clockB |  --> | clockC |
++ ++  ++

The actual code would not allow that because, since we cannot call
"clock_propagate" directly on a child, it would exit on the
first child has the period has not changed for clockB, only clockC is


Typo "as the period has not changed"?

Why can't you call clock_propagate() on a child?


impacted in our example.

Signed-off-by: Raphael Poggi 
---
  hw/core/clock.c | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/hw/core/clock.c b/hw/core/clock.c
index a19c7db7df..85421f8b55 100644
--- a/hw/core/clock.c
+++ b/hw/core/clock.c
@@ -101,8 +101,9 @@ static void clock_propagate_period(Clock *clk, bool 
call_callbacks)
  if (call_callbacks) {
  clock_call_callback(child, ClockUpdate);
  }
-clock_propagate_period(child, call_callbacks);
  }
+
+clock_propagate_period(child, call_callbacks);
  }
  }

Re: [PATCH 00/24] include/exec: Rework (part 2)

2024-04-18 Thread Philippe Mathieu-Daudé 

On 18/4/24 21:24, Philippe Mathieu-Daudé wrote:

Reordering exec/, second round. Mostly splitting:
- TCG LDST API (to "exec/cpu_ldst.h")
- CPU TLB APIs (to "exec/cputlb.h")
from "exec/cpu-all.h" and "exec/exec-all.h", and
reducing inclusions of "hw/core/cpu.h".

Based-on: <20240417182806.69446-1-phi...@linaro.org>
or https://gitlab.com/philmd/qemu/-/commits/exec-next


And this series is available as
https://gitlab.com/philmd/qemu/-/commits/exec-rework-part2/


Philippe Mathieu-Daudé (24):
   exec: Declare MMUAccessType type in 'mmu-access-type.h' header
   exec: Declare CPUBreakpoint/CPUWatchpoint type in 'breakpoint.h'
 header
   hw/core: Avoid including the full 'hw/core/cpu.h' in 'tcg-cpu-ops.h'
   exec: Restrict TCG specific declarations of 'cputlb.h'
   exec: Restrict 'cpu_ldst.h' to TCG accelerator
   exec: Have guest_addr_valid() methods take abi_ptr/size_t arguments
   exec: Un-inline tlb_vaddr_to_host() and declare it in 'exec/cputlb.h'
   physmem: Move TCG CPU IOTLB methods around
   physmem: Restrict TCG CPU IOTLB code to TCG accel
   exec: Reduce tlb_set_dirty() declaration scope
   exec: Move tlb_reset_dirty*() declarations to 'exec/cputlb.h'
   exec: Move CPUTLBEntry helpers to cputlb.c
   target/sparc: Replace abi_ulong by uint32_t for TARGET_ABI32
   exec: Rename 'exec/user/guest-base.h' as 'user/guest-base.h'
   exec: Restrict inclusion of 'user/guest-base.h'
   exec: Move GUEST_ADDR_MAX definition to 'cpu_ldst.h'
   exec: Include missing 'qemu/log-for-trace.h' header in 'exec/log.h'
   plugins: Include missing 'qemu/bitmap.h' header
   gdbstub: Avoid including 'cpu.h' in 'gdbstub/helpers.h'
   hw/xtensa: Include missing 'exec/cpu-common.h' in 'bootparam.h'
   plugins: Un-inline qemu_plugin_disable_mem_helpers()
   exec: Remove 'exec/tswap.h' from 'exec/cpu-all.h'
   exec: Remove 'disas/disas.h' from 'exec/log.h'
   exec: Remove unnecessary inclusions of 'hw/core/cpu.h'

[PATCH 16/24] exec: Move GUEST_ADDR_MAX definition to 'cpu_ldst.h'

2024-04-18 Thread Philippe Mathieu-Daudé 
GUEST_ADDR_MAX is only used in accel/tcg/user-exec.c
and "exec/cpu_ldst.h". Move it from "exec/cpu-all.h"
to "exec/cpu_ldst.h" so we can avoid including
"exec/cpu-all.h" in accel/tcg/user-exec.c.

Signed-off-by: Philippe Mathieu-Daudé 
---
 include/exec/cpu-all.h  | 34 +-
 include/exec/cpu_ldst.h | 30 ++
 accel/tcg/cpu-exec.c|  1 -
 3 files changed, 31 insertions(+), 34 deletions(-)

diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index e75ec13cd0..554b937ddb 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -63,39 +63,7 @@
 
 /* MMU memory access macros */
 
-#if defined(CONFIG_USER_ONLY)
-#include "exec/user/abitypes.h"
-
-/*
- * If non-zero, the guest virtual address space is a contiguous subset
- * of the host virtual address space, i.e. '-R reserved_va' is in effect
- * either from the command-line or by default.  The value is the last
- * byte of the guest address space e.g. UINT32_MAX.
- *
- * If zero, the host and guest virtual address spaces are intermingled.
- */
-extern unsigned long reserved_va;
-
-/*
- * Limit the guest addresses as best we can.
- *
- * When not using -R reserved_va, we cannot really limit the guest
- * to less address space than the host.  For 32-bit guests, this
- * acts as a sanity check that we're not giving the guest an address
- * that it cannot even represent.  For 64-bit guests... the address
- * might not be what the real kernel would give, but it is at least
- * representable in the guest.
- *
- * TODO: Improve address allocation to avoid this problem, and to
- * avoid setting bits at the top of guest addresses that might need
- * to be used for tags.
- */
-#define GUEST_ADDR_MAX_ \
-((MIN_CONST(TARGET_VIRT_ADDR_SPACE_BITS, TARGET_ABI_BITS) <= 32) ?  \
- UINT32_MAX : ~0ul)
-#define GUEST_ADDR_MAX(reserved_va ? : GUEST_ADDR_MAX_)
-
-#else
+#if !defined(CONFIG_USER_ONLY)
 
 #include "exec/hwaddr.h"
 
diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h
index 6d47d3114a..07649a01c1 100644
--- a/include/exec/cpu_ldst.h
+++ b/include/exec/cpu_ldst.h
@@ -73,8 +73,38 @@
 
 #if defined(CONFIG_USER_ONLY)
 
+#include "user/abitypes.h"
 #include "user/guest-base.h"
 
+/*
+ * If non-zero, the guest virtual address space is a contiguous subset
+ * of the host virtual address space, i.e. '-R reserved_va' is in effect
+ * either from the command-line or by default.  The value is the last
+ * byte of the guest address space e.g. UINT32_MAX.
+ *
+ * If zero, the host and guest virtual address spaces are intermingled.
+ */
+extern unsigned long reserved_va;
+
+/*
+ * Limit the guest addresses as best we can.
+ *
+ * When not using -R reserved_va, we cannot really limit the guest
+ * to less address space than the host.  For 32-bit guests, this
+ * acts as a sanity check that we're not giving the guest an address
+ * that it cannot even represent.  For 64-bit guests... the address
+ * might not be what the real kernel would give, but it is at least
+ * representable in the guest.
+ *
+ * TODO: Improve address allocation to avoid this problem, and to
+ * avoid setting bits at the top of guest addresses that might need
+ * to be used for tags.
+ */
+#define GUEST_ADDR_MAX_ \
+((MIN_CONST(TARGET_VIRT_ADDR_SPACE_BITS, TARGET_ABI_BITS) <= 32) ?  \
+ UINT32_MAX : ~0ul)
+#define GUEST_ADDR_MAX(reserved_va ? : GUEST_ADDR_MAX_)
+
 #ifndef TARGET_TAGGED_ADDRESSES
 static inline abi_ptr cpu_untagged_addr(CPUState *cs, abi_ptr x)
 {
diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index 225e5fbd3e..7bfd558027 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -31,7 +31,6 @@
 #include "exec/log.h"
 #include "qemu/main-loop.h"
 #include "sysemu/cpus.h"
-#include "exec/cpu-all.h"
 #include "sysemu/cpu-timers.h"
 #include "exec/replay-core.h"
 #include "sysemu/tcg.h"
-- 
2.41.0

[PATCH 23/24] exec: Remove 'disas/disas.h' from 'exec/log.h'

2024-04-18 Thread Philippe Mathieu-Daudé 
"exec/log.h" doesn't require "disas/disas.h". Remove it,
including it in the sources when required.

Signed-off-by: Philippe Mathieu-Daudé 
---
 include/exec/log.h   | 1 -
 target/avr/translate.c   | 1 +
 target/hexagon/translate.c   | 1 +
 target/loongarch/tcg/translate.c | 1 +
 target/rx/translate.c| 1 +
 tcg/tcg.c| 1 +
 6 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/include/exec/log.h b/include/exec/log.h
index e0ff778a10..ee33981aab 100644
--- a/include/exec/log.h
+++ b/include/exec/log.h
@@ -4,7 +4,6 @@
 #include "qemu/log.h"
 #include "qemu/log-for-trace.h"
 #include "hw/core/cpu.h"
-#include "disas/disas.h"
 
 /* cpu_dump_state() logging functions: */
 /**
diff --git a/target/avr/translate.c b/target/avr/translate.c
index 87e2bd5ef1..b2005f3cbe 100644
--- a/target/avr/translate.c
+++ b/target/avr/translate.c
@@ -29,6 +29,7 @@
 #include "exec/helper-gen.h"
 #include "exec/log.h"
 #include "exec/translator.h"
+#include "disas/disas.h"
 
 #define HELPER_H "helper.h"
 #include "exec/helper-info.c.inc"
diff --git a/target/hexagon/translate.c b/target/hexagon/translate.c
index 47a870f42d..6bec38404e 100644
--- a/target/hexagon/translate.c
+++ b/target/hexagon/translate.c
@@ -25,6 +25,7 @@
 #include "exec/translation-block.h"
 #include "exec/cpu_ldst.h"
 #include "exec/log.h"
+#include "disas/disas.h"
 #include "internal.h"
 #include "attribs.h"
 #include "insn.h"
diff --git a/target/loongarch/tcg/translate.c b/target/loongarch/tcg/translate.c
index 7567712655..ae87a68c69 100644
--- a/target/loongarch/tcg/translate.c
+++ b/target/loongarch/tcg/translate.c
@@ -14,6 +14,7 @@
 #include "exec/helper-proto.h"
 #include "exec/helper-gen.h"
 #include "exec/log.h"
+#include "disas/disas.h"
 #include "qemu/qemu-print.h"
 #include "fpu/softfloat.h"
 #include "translate.h"
diff --git a/target/rx/translate.c b/target/rx/translate.c
index f6e9e0ec90..4259ad6d74 100644
--- a/target/rx/translate.c
+++ b/target/rx/translate.c
@@ -27,6 +27,7 @@
 #include "exec/helper-gen.h"
 #include "exec/translator.h"
 #include "exec/log.h"
+#include "disas/disas.h"
 
 #define HELPER_H "helper.h"
 #include "exec/helper-info.c.inc"
diff --git a/tcg/tcg.c b/tcg/tcg.c
index 6a32656cd4..e7e0ab6846 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -51,6 +51,7 @@
 #endif
 
 #include "elf.h"
+#include "disas/disas.h"
 #include "exec/log.h"
 #include "tcg/tcg-ldst.h"
 #include "tcg/tcg-temp-internal.h"
-- 
2.41.0

[PATCH 17/24] exec: Include missing 'qemu/log-for-trace.h' header in 'exec/log.h'

2024-04-18 Thread Philippe Mathieu-Daudé 
"exec/log.h" accesses the qemu_loglevel variable,
which is declared in "qemu/log-for-trace.h".

Signed-off-by: Philippe Mathieu-Daudé 
---
 include/exec/log.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/include/exec/log.h b/include/exec/log.h
index 4a7375a45f..e0ff778a10 100644
--- a/include/exec/log.h
+++ b/include/exec/log.h
@@ -2,6 +2,7 @@
 #define QEMU_EXEC_LOG_H
 
 #include "qemu/log.h"
+#include "qemu/log-for-trace.h"
 #include "hw/core/cpu.h"
 #include "disas/disas.h"
 
-- 
2.41.0

[PATCH 18/24] plugins: Include missing 'qemu/bitmap.h' header

2024-04-18 Thread Philippe Mathieu-Daudé 
"qemu/plugin.h" uses DECLARE_BITMAP(), which is
declared in "qemu/bitmap.h".

Signed-off-by: Philippe Mathieu-Daudé 
---
 include/qemu/plugin.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/include/qemu/plugin.h b/include/qemu/plugin.h
index 12a96cea2a..41db748eda 100644
--- a/include/qemu/plugin.h
+++ b/include/qemu/plugin.h
@@ -13,6 +13,7 @@
 #include "qemu/queue.h"
 #include "qemu/option.h"
 #include "qemu/plugin-event.h"
+#include "qemu/bitmap.h"
 #include "exec/memopidx.h"
 #include "hw/core/cpu.h"
 
-- 
2.41.0

[PATCH 21/24] plugins: Un-inline qemu_plugin_disable_mem_helpers()

2024-04-18 Thread Philippe Mathieu-Daudé 
"qemu/plugin.h" only include the huge "hw/core/cpu.h"
because qemu_plugin_disable_mem_helpers() accesses
CPUState::plugin_mem_cbs. In order to avoid including
it, un-inline qemu_plugin_disable_mem_helpers().

Signed-off-by: Philippe Mathieu-Daudé 
---
 include/qemu/plugin.h | 6 +-
 plugins/core.c| 5 +
 2 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/include/qemu/plugin.h b/include/qemu/plugin.h
index 41db748eda..28467c4a28 100644
--- a/include/qemu/plugin.h
+++ b/include/qemu/plugin.h
@@ -15,7 +15,6 @@
 #include "qemu/plugin-event.h"
 #include "qemu/bitmap.h"
 #include "exec/memopidx.h"
-#include "hw/core/cpu.h"
 
 /*
  * Option parsing/processing.
@@ -227,10 +226,7 @@ void qemu_plugin_atexit_cb(void);
 
 void qemu_plugin_add_dyn_cb_arr(GArray *arr);
 
-static inline void qemu_plugin_disable_mem_helpers(CPUState *cpu)
-{
-cpu->plugin_mem_cbs = NULL;
-}
+void qemu_plugin_disable_mem_helpers(CPUState *cpu);
 
 /**
  * qemu_plugin_user_exit(): clean-up callbacks before calling exit callbacks
diff --git a/plugins/core.c b/plugins/core.c
index 11ca20e626..b25092abb5 100644
--- a/plugins/core.c
+++ b/plugins/core.c
@@ -493,6 +493,11 @@ void exec_inline_op(struct qemu_plugin_dyn_cb *cb, int 
cpu_index)
 }
 }
 
+void qemu_plugin_disable_mem_helpers(CPUState *cpu)
+{
+cpu->plugin_mem_cbs = NULL;
+}
+
 void qemu_plugin_vcpu_mem_cb(CPUState *cpu, uint64_t vaddr,
  MemOpIdx oi, enum qemu_plugin_mem_rw rw)
 {
-- 
2.41.0

[PATCH 22/24] exec: Remove 'exec/tswap.h' from 'exec/cpu-all.h'

2024-04-18 Thread Philippe Mathieu-Daudé 
"exec/cpu-all.h" doesn't require "exec/tswap.h". Remove it,
including it in the sources when required.

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/xtensa/bootparam.h   | 1 +
 include/exec/cpu-all.h  | 1 -
 accel/tcg/translator.c  | 1 +
 hw/arm/boot.c   | 1 +
 hw/arm/npcm7xx.c| 1 +
 hw/mips/fuloong2e.c | 1 +
 hw/mips/malta.c | 1 +
 hw/ppc/sam460ex.c   | 1 +
 hw/ppc/spapr.c  | 1 +
 hw/ppc/virtex_ml507.c   | 1 +
 hw/sh4/r2d.c| 1 +
 target/arm/gdbstub.c| 1 +
 target/xtensa/xtensa-semi.c | 1 +
 13 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/hw/xtensa/bootparam.h b/hw/xtensa/bootparam.h
index f57ff850bc..e1d47b503c 100644
--- a/hw/xtensa/bootparam.h
+++ b/hw/xtensa/bootparam.h
@@ -1,6 +1,7 @@
 #ifndef HW_XTENSA_BOOTPARAM_H
 #define HW_XTENSA_BOOTPARAM_H
 
+#include "exec/tswap.h"
 #include "exec/cpu-common.h"
 
 #define BP_TAG_COMMAND_LINE 0x1001  /* command line (0-terminated string)*/
diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index 554b937ddb..cfbf51822c 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -21,7 +21,6 @@
 
 #include "exec/cpu-common.h"
 #include "exec/memory.h"
-#include "exec/tswap.h"
 #include "hw/core/cpu.h"
 
 /* some important defines:
diff --git a/accel/tcg/translator.c b/accel/tcg/translator.c
index 6832e55135..85950377d9 100644
--- a/accel/tcg/translator.c
+++ b/accel/tcg/translator.c
@@ -12,6 +12,7 @@
 #include "qemu/error-report.h"
 #include "exec/exec-all.h"
 #include "exec/translator.h"
+#include "exec/tswap.h"
 #include "exec/cpu_ldst.h"
 #include "exec/plugin-gen.h"
 #include "tcg/tcg-op-common.h"
diff --git a/hw/arm/boot.c b/hw/arm/boot.c
index 84ea6a807a..93945a1a15 100644
--- a/hw/arm/boot.c
+++ b/hw/arm/boot.c
@@ -22,6 +22,7 @@
 #include "sysemu/reset.h"
 #include "hw/loader.h"
 #include "elf.h"
+#include "exec/tswap.h"
 #include "sysemu/device_tree.h"
 #include "qemu/config-file.h"
 #include "qemu/option.h"
diff --git a/hw/arm/npcm7xx.c b/hw/arm/npcm7xx.c
index cc68b5d8f1..1ef303415b 100644
--- a/hw/arm/npcm7xx.c
+++ b/hw/arm/npcm7xx.c
@@ -27,6 +27,7 @@
 #include "qemu/units.h"
 #include "sysemu/sysemu.h"
 #include "target/arm/cpu-qom.h"
+#include "exec/tswap.h"
 
 /*
  * This covers the whole MMIO space. We'll use this to catch any MMIO accesses
diff --git a/hw/mips/fuloong2e.c b/hw/mips/fuloong2e.c
index a45aac368c..1d0613a76f 100644
--- a/hw/mips/fuloong2e.c
+++ b/hw/mips/fuloong2e.c
@@ -40,6 +40,7 @@
 #include "sysemu/reset.h"
 #include "sysemu/sysemu.h"
 #include "qemu/error-report.h"
+#include "exec/tswap.h"
 
 #define ENVP_PADDR  0x2000
 #define ENVP_VADDR  cpu_mips_phys_to_kseg0(NULL, ENVP_PADDR)
diff --git a/hw/mips/malta.c b/hw/mips/malta.c
index af74008c82..3dca0f100c 100644
--- a/hw/mips/malta.c
+++ b/hw/mips/malta.c
@@ -56,6 +56,7 @@
 #include "semihosting/semihost.h"
 #include "hw/mips/cps.h"
 #include "hw/qdev-clock.h"
+#include "exec/tswap.h"
 #include "target/mips/internal.h"
 #include "trace.h"
 #include "cpu.h"
diff --git a/hw/ppc/sam460ex.c b/hw/ppc/sam460ex.c
index d42b677898..abc02f0817 100644
--- a/hw/ppc/sam460ex.c
+++ b/hw/ppc/sam460ex.c
@@ -24,6 +24,7 @@
 #include "hw/loader.h"
 #include "elf.h"
 #include "exec/memory.h"
+#include "exec/tswap.h"
 #include "ppc440.h"
 #include "hw/pci-host/ppc4xx.h"
 #include "hw/block/flash.h"
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index e9bc97fee0..b4b1f43983 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -74,6 +74,7 @@
 #include "hw/virtio/virtio-scsi.h"
 #include "hw/virtio/vhost-scsi-common.h"
 
+#include "exec/tswap.h"
 #include "exec/ram_addr.h"
 #include "hw/usb.h"
 #include "qemu/config-file.h"
diff --git a/hw/ppc/virtex_ml507.c b/hw/ppc/virtex_ml507.c
index d02f330650..fd23afebf5 100644
--- a/hw/ppc/virtex_ml507.c
+++ b/hw/ppc/virtex_ml507.c
@@ -38,6 +38,7 @@
 #include "qapi/error.h"
 #include "qemu/error-report.h"
 #include "qemu/option.h"
+#include "exec/tswap.h"
 
 #include "hw/intc/ppc-uic.h"
 #include "hw/ppc/ppc.h"
diff --git a/hw/sh4/r2d.c b/hw/sh4/r2d.c
index e5ac6751bd..5f4420f534 100644
--- a/hw/sh4/r2d.c
+++ b/hw/sh4/r2d.c
@@ -43,6 +43,7 @@
 #include "hw/loader.h"
 #include "hw/usb.h"
 #include "hw/block/flash.h"
+#include "exec/tswap.h"
 
 #define FLASH_BASE 0x
 #define FLASH_SIZE (16 * MiB)
diff --git a/target/arm/gdbstub.c b/target/arm/gdbstub.c
index a3bb73cfa7..f2b001afdd 100644
--- a/target/arm/gdbstub.c
+++ b/target/arm/gdbstub.c
@@ -20,6 +20,7 @@
 #include "qemu/osdep.h"
 #include "cpu.h"
 #include "exec/gdbstub.h"
+#include "exec/tswap.h"
 #include "gdbstub/helpers.h"
 #include "sysemu/tcg.h"
 #include "internals.h"
diff --git a/target/xtensa/xtensa-semi.c b/target/xtensa/xtensa-semi.c
index fa21b7e11f..dbc42d1587 100644
--- a/target/xtensa/xtensa-semi.c
+++ b/target/xtensa/xtensa-semi.c
@@ -29,6 +29,7 @@
 #include "cpu.h"
 #include "chardev/char-fe.h"
 #include "exec/helper-proto.h"

[PATCH 12/24] exec: Move CPUTLBEntry helpers to cputlb.c

2024-04-18 Thread Philippe Mathieu-Daudé 
The following CPUTLBEntry helpers are only used in accel/tcg/cputlb.c:
  - tlb_index()
  - tlb_entry()
  - tlb_read_idx()
  - tlb_addr_write()

Move them to this file, allowing to remove the huge "cpu.h" header
inclusion from "exec/cpu_ldst.h".

Signed-off-by: Philippe Mathieu-Daudé 
---
 include/exec/cpu_ldst.h | 55 -
 accel/tcg/cputlb.c  | 51 ++
 2 files changed, 51 insertions(+), 55 deletions(-)

diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h
index 7032949dba..2c5a0a5c81 100644
--- a/include/exec/cpu_ldst.h
+++ b/include/exec/cpu_ldst.h
@@ -70,7 +70,6 @@
 #include "exec/abi_ptr.h"
 #include "exec/mmu-access-type.h"
 #include "qemu/int128.h"
-#include "cpu.h"
 
 #if defined(CONFIG_USER_ONLY)
 
@@ -294,60 +293,6 @@ Int128 cpu_atomic_cmpxchgo_be_mmu(CPUArchState *env, 
abi_ptr addr,
   Int128 cmpv, Int128 newv,
   MemOpIdx oi, uintptr_t retaddr);
 
-#if !defined(CONFIG_USER_ONLY)
-
-#include "tcg/oversized-guest.h"
-
-static inline uint64_t tlb_read_idx(const CPUTLBEntry *entry,
-MMUAccessType access_type)
-{
-/* Do not rearrange the CPUTLBEntry structure members. */
-QEMU_BUILD_BUG_ON(offsetof(CPUTLBEntry, addr_read) !=
-  MMU_DATA_LOAD * sizeof(uint64_t));
-QEMU_BUILD_BUG_ON(offsetof(CPUTLBEntry, addr_write) !=
-  MMU_DATA_STORE * sizeof(uint64_t));
-QEMU_BUILD_BUG_ON(offsetof(CPUTLBEntry, addr_code) !=
-  MMU_INST_FETCH * sizeof(uint64_t));
-
-#if TARGET_LONG_BITS == 32
-/* Use qatomic_read, in case of addr_write; only care about low bits. */
-const uint32_t *ptr = (uint32_t *)>addr_idx[access_type];
-ptr += HOST_BIG_ENDIAN;
-return qatomic_read(ptr);
-#else
-const uint64_t *ptr = >addr_idx[access_type];
-# if TCG_OVERSIZED_GUEST
-return *ptr;
-# else
-/* ofs might correspond to .addr_write, so use qatomic_read */
-return qatomic_read(ptr);
-# endif
-#endif
-}
-
-static inline uint64_t tlb_addr_write(const CPUTLBEntry *entry)
-{
-return tlb_read_idx(entry, MMU_DATA_STORE);
-}
-
-/* Find the TLB index corresponding to the mmu_idx + address pair.  */
-static inline uintptr_t tlb_index(CPUState *cpu, uintptr_t mmu_idx,
-  vaddr addr)
-{
-uintptr_t size_mask = cpu->neg.tlb.f[mmu_idx].mask >> CPU_TLB_ENTRY_BITS;
-
-return (addr >> TARGET_PAGE_BITS) & size_mask;
-}
-
-/* Find the TLB entry corresponding to the mmu_idx + address pair.  */
-static inline CPUTLBEntry *tlb_entry(CPUState *cpu, uintptr_t mmu_idx,
- vaddr addr)
-{
-return >neg.tlb.f[mmu_idx].table[tlb_index(cpu, mmu_idx, addr)];
-}
-
-#endif /* !defined(CONFIG_USER_ONLY) */
-
 #if TARGET_BIG_ENDIAN
 # define cpu_lduw_datacpu_lduw_be_data
 # define cpu_ldsw_datacpu_ldsw_be_data
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index e16d02a62c..953c437ba9 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -27,6 +27,9 @@
 #include "exec/tb-flush.h"
 #include "exec/memory-internal.h"
 #include "exec/ram_addr.h"
+#include "exec/mmu-access-type.h"
+#include "exec/tlb-common.h"
+#include "exec/vaddr.h"
 #include "tcg/tcg.h"
 #include "qemu/error-report.h"
 #include "exec/log.h"
@@ -95,6 +98,54 @@ static inline size_t sizeof_tlb(CPUTLBDescFast *fast)
 return fast->mask + (1 << CPU_TLB_ENTRY_BITS);
 }
 
+static inline uint64_t tlb_read_idx(const CPUTLBEntry *entry,
+MMUAccessType access_type)
+{
+/* Do not rearrange the CPUTLBEntry structure members. */
+QEMU_BUILD_BUG_ON(offsetof(CPUTLBEntry, addr_read) !=
+  MMU_DATA_LOAD * sizeof(uint64_t));
+QEMU_BUILD_BUG_ON(offsetof(CPUTLBEntry, addr_write) !=
+  MMU_DATA_STORE * sizeof(uint64_t));
+QEMU_BUILD_BUG_ON(offsetof(CPUTLBEntry, addr_code) !=
+  MMU_INST_FETCH * sizeof(uint64_t));
+
+#if TARGET_LONG_BITS == 32
+/* Use qatomic_read, in case of addr_write; only care about low bits. */
+const uint32_t *ptr = (uint32_t *)>addr_idx[access_type];
+ptr += HOST_BIG_ENDIAN;
+return qatomic_read(ptr);
+#else
+const uint64_t *ptr = >addr_idx[access_type];
+# if TCG_OVERSIZED_GUEST
+return *ptr;
+# else
+/* ofs might correspond to .addr_write, so use qatomic_read */
+return qatomic_read(ptr);
+# endif
+#endif
+}
+
+static inline uint64_t tlb_addr_write(const CPUTLBEntry *entry)
+{
+return tlb_read_idx(entry, MMU_DATA_STORE);
+}
+
+/* Find the TLB index corresponding to the mmu_idx + address pair.  */
+static inline uintptr_t tlb_index(CPUState *cpu, uintptr_t mmu_idx,
+  vaddr addr)
+{
+uintptr_t size_mask = cpu->neg.tlb.f[mmu_idx].mask >> CPU_TLB_ENTRY_BITS;
+
+return (addr >> TARGET_PAGE_BITS) & size_mask;
+}
+
+/* Find

[PATCH 10/24] exec: Reduce tlb_set_dirty() declaration scope

2024-04-18 Thread Philippe Mathieu-Daudé 
tlb_set_dirty() is only used in accel/tcg/cputlb.c,
where it is defined. Declare it statically, removing
the stub.

Signed-off-by: Philippe Mathieu-Daudé 
---
 include/exec/exec-all.h | 1 -
 accel/stubs/tcg-stub.c  | 4 
 accel/tcg/cputlb.c  | 2 +-
 3 files changed, 1 insertion(+), 6 deletions(-)

diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index e4cff08db4..778c82ba8e 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -641,7 +641,6 @@ static inline void mmap_unlock(void) {}
 #define WITH_MMAP_LOCK_GUARD()
 
 void tlb_reset_dirty(CPUState *cpu, ram_addr_t start1, ram_addr_t length);
-void tlb_set_dirty(CPUState *cpu, vaddr addr);
 void tlb_reset_dirty_range_all(ram_addr_t start, ram_addr_t length);
 
 #endif
diff --git a/accel/stubs/tcg-stub.c b/accel/stubs/tcg-stub.c
index 8a496a2a6f..dd890d6cf6 100644
--- a/accel/stubs/tcg-stub.c
+++ b/accel/stubs/tcg-stub.c
@@ -18,10 +18,6 @@ void tb_flush(CPUState *cpu)
 {
 }
 
-void tlb_set_dirty(CPUState *cpu, vaddr vaddr)
-{
-}
-
 int probe_access_flags(CPUArchState *env, vaddr addr, int size,
MMUAccessType access_type, int mmu_idx,
bool nonfault, void **phost, uintptr_t retaddr)
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index 93b1ca810b..e16d02a62c 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -1039,7 +1039,7 @@ static inline void tlb_set_dirty1_locked(CPUTLBEntry 
*tlb_entry,
 
 /* update the TLB corresponding to virtual page vaddr
so that it is no longer dirty */
-void tlb_set_dirty(CPUState *cpu, vaddr addr)
+static void tlb_set_dirty(CPUState *cpu, vaddr addr)
 {
 int mmu_idx;
 
-- 
2.41.0

[PATCH 19/24] gdbstub: Avoid including 'cpu.h' in 'gdbstub/helpers.h'

2024-04-18 Thread Philippe Mathieu-Daudé 
We only need the "exec/tswap.h" and "cpu-param.h" headers.
Only include "cpu.h" in the target gdbstub.c source files.

Signed-off-by: Philippe Mathieu-Daudé 
---
 include/gdbstub/helpers.h | 3 ++-
 target/avr/gdbstub.c  | 1 +
 target/tricore/gdbstub.c  | 1 +
 3 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/include/gdbstub/helpers.h b/include/gdbstub/helpers.h
index 6277a858a1..26140ef1ac 100644
--- a/include/gdbstub/helpers.h
+++ b/include/gdbstub/helpers.h
@@ -16,7 +16,8 @@
 #error "gdbstub helpers should only be included by target specific code"
 #endif
 
-#include "cpu.h"
+#include "exec/tswap.h"
+#include "cpu-param.h"
 
 /*
  * The GDB remote protocol transfers values in target byte order. As
diff --git a/target/avr/gdbstub.c b/target/avr/gdbstub.c
index 22bf4e..d6d3c1479b 100644
--- a/target/avr/gdbstub.c
+++ b/target/avr/gdbstub.c
@@ -20,6 +20,7 @@
 
 #include "qemu/osdep.h"
 #include "gdbstub/helpers.h"
+#include "cpu.h"
 
 int avr_cpu_gdb_read_register(CPUState *cs, GByteArray *mem_buf, int n)
 {
diff --git a/target/tricore/gdbstub.c b/target/tricore/gdbstub.c
index f9309c5e27..29a70051ff 100644
--- a/target/tricore/gdbstub.c
+++ b/target/tricore/gdbstub.c
@@ -19,6 +19,7 @@
 
 #include "qemu/osdep.h"
 #include "gdbstub/helpers.h"
+#include "cpu.h"
 
 
 #define LCX_REGNUM 32
-- 
2.41.0

[PATCH 24/24] exec: Remove unnecessary inclusions of 'hw/core/cpu.h'

2024-04-18 Thread Philippe Mathieu-Daudé 
When "hw/core/cpu.h" is not required, remove it.

Signed-off-by: Philippe Mathieu-Daudé 
---
 include/exec/cpu-all.h| 1 -
 include/exec/cpu-defs.h   | 1 -
 include/hw/boards.h   | 1 -
 include/hw/ppc/openpic.h  | 1 -
 include/sysemu/hw_accel.h | 1 -
 5 files changed, 5 deletions(-)

diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index cfbf51822c..03963442a0 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -21,7 +21,6 @@
 
 #include "exec/cpu-common.h"
 #include "exec/memory.h"
-#include "hw/core/cpu.h"
 
 /* some important defines:
  *
diff --git a/include/exec/cpu-defs.h b/include/exec/cpu-defs.h
index 0dbef3010c..2b88db71a8 100644
--- a/include/exec/cpu-defs.h
+++ b/include/exec/cpu-defs.h
@@ -29,7 +29,6 @@
 #include "exec/hwaddr.h"
 #endif
 #include "exec/memattrs.h"
-#include "hw/core/cpu.h"
 
 #include "cpu-param.h"
 
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 8b8f6d5c00..2daa23082f 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -9,7 +9,6 @@
 #include "qapi/qapi-types-machine.h"
 #include "qemu/module.h"
 #include "qom/object.h"
-#include "hw/core/cpu.h"
 
 #define TYPE_MACHINE_SUFFIX "-machine"
 
diff --git a/include/hw/ppc/openpic.h b/include/hw/ppc/openpic.h
index 9c6af8e207..8fe3a5902f 100644
--- a/include/hw/ppc/openpic.h
+++ b/include/hw/ppc/openpic.h
@@ -2,7 +2,6 @@
 #define OPENPIC_H
 
 #include "hw/sysbus.h"
-#include "hw/core/cpu.h"
 #include "qom/object.h"
 
 #define MAX_CPU 32
diff --git a/include/sysemu/hw_accel.h b/include/sysemu/hw_accel.h
index c71b77e71f..6aca3c6178 100644
--- a/include/sysemu/hw_accel.h
+++ b/include/sysemu/hw_accel.h
@@ -11,7 +11,6 @@
 #ifndef QEMU_HW_ACCEL_H
 #define QEMU_HW_ACCEL_H
 
-#include "hw/core/cpu.h"
 #include "sysemu/kvm.h"
 #include "sysemu/hvf.h"
 #include "sysemu/whpx.h"
-- 
2.41.0

Re: [PATCH 02/24] exec: Declare CPUBreakpoint/CPUWatchpoint type in 'breakpoint.h' header

2024-04-18 Thread Philippe Mathieu-Daudé 

On 18/4/24 21:25, Philippe Mathieu-Daudé wrote:

The CPUBreakpoint and CPUWatchpoint structures are declared
in "hw/core/cpu.h", which contains declarations related to
CPUState and CPUClass. Some source files only require the
BP/WP definitions and don't need to pull in all CPU* API.
In order to simplify, create a new "exec/breakpoint.h" header.

Signed-off-by: Philippe Mathieu-Daudé 
---
  include/exec/breakpoint.h | 23 +++
  include/hw/core/cpu.h | 16 +---
  target/arm/internals.h|  1 +
  target/ppc/internal.h |  1 +
  target/riscv/debug.h  |  2 ++
  5 files changed, 28 insertions(+), 15 deletions(-)
  create mode 100644 include/exec/breakpoint.h

diff --git a/include/exec/breakpoint.h b/include/exec/breakpoint.h
new file mode 100644
index 00..45038263e3
--- /dev/null
+++ b/include/exec/breakpoint.h
@@ -0,0 +1,23 @@


Oops, forgot:

+/*
+ * QEMU breakpoint & watchpoint definitions
+ *
+ * Copyright (c) 2012 SUSE LINUX Products GmbH
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */


+#ifndef EXEC_BREAKPOINT_H
+#define EXEC_BREAKPOINT_H
+
+#include "qemu/queue.h"
+#include "exec/vaddr.h"
+#include "exec/memattrs.h" //MemTxAttrs


Also forgot to remove that comment.


+
+typedef struct CPUBreakpoint {
+vaddr pc;
+int flags; /* BP_* */
+QTAILQ_ENTRY(CPUBreakpoint) entry;
+} CPUBreakpoint;
+
+typedef struct CPUWatchpoint {
+vaddr vaddr;
+vaddr len;
+vaddr hitaddr;
+MemTxAttrs hitattrs;
+int flags; /* BP_* */
+QTAILQ_ENTRY(CPUWatchpoint) entry;
+} CPUWatchpoint;
+
+#endif

[PATCH 14/24] exec: Rename 'exec/user/guest-base.h' as 'user/guest-base.h'

2024-04-18 Thread Philippe Mathieu-Daudé 
The include/user/ directory contains the user-emulation
specific headers. Move guest-base.h there too.

Signed-off-by: Philippe Mathieu-Daudé 
---
 include/exec/cpu-all.h   | 2 +-
 include/{exec => }/user/guest-base.h | 4 ++--
 tcg/tcg.c| 2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)
 rename include/{exec => }/user/guest-base.h (72%)

diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index 78848f018c..027f19e052 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -65,7 +65,7 @@
 
 #if defined(CONFIG_USER_ONLY)
 #include "exec/user/abitypes.h"
-#include "exec/user/guest-base.h"
+#include "user/guest-base.h"
 
 extern bool have_guest_base;
 
diff --git a/include/exec/user/guest-base.h b/include/user/guest-base.h
similarity index 72%
rename from include/exec/user/guest-base.h
rename to include/user/guest-base.h
index afe2ab7fbb..1e42bca5db 100644
--- a/include/exec/user/guest-base.h
+++ b/include/user/guest-base.h
@@ -4,8 +4,8 @@
  *  Copyright (c) 2003 Fabrice Bellard
  */
 
-#ifndef EXEC_USER_GUEST_BASE_H
-#define EXEC_USER_GUEST_BASE_H
+#ifndef USER_GUEST_BASE_H
+#define USER_GUEST_BASE_H
 
 extern uintptr_t guest_base;
 
diff --git a/tcg/tcg.c b/tcg/tcg.c
index 0c0bb9d169..6a32656cd4 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -57,7 +57,7 @@
 #include "tcg-internal.h"
 #include "tcg/perf.h"
 #ifdef CONFIG_USER_ONLY
-#include "exec/user/guest-base.h"
+#include "user/guest-base.h"
 #endif
 
 /* Forward declarations for functions declared in tcg-target.c.inc and
-- 
2.41.0

[PATCH 20/24] hw/xtensa: Include missing 'exec/cpu-common.h' in 'bootparam.h'

2024-04-18 Thread Philippe Mathieu-Daudé 
cpu_physical_memory_write() is declared in "exec/cpu-common.h".

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/xtensa/bootparam.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/hw/xtensa/bootparam.h b/hw/xtensa/bootparam.h
index ade7891ec5..f57ff850bc 100644
--- a/hw/xtensa/bootparam.h
+++ b/hw/xtensa/bootparam.h
@@ -1,6 +1,8 @@
 #ifndef HW_XTENSA_BOOTPARAM_H
 #define HW_XTENSA_BOOTPARAM_H
 
+#include "exec/cpu-common.h"
+
 #define BP_TAG_COMMAND_LINE 0x1001  /* command line (0-terminated string)*/
 #define BP_TAG_INITRD   0x1002  /* ramdisk addr and size (bp_meminfo) 
*/
 #define BP_TAG_MEMORY   0x1003  /* memory addr and size (bp_meminfo) */
-- 
2.41.0

  1   2   3   >