Re: [PATCH v3 0/5] Generate x86 cpu features

2024-03-11 Thread Tim Wiederhake
On Tue, 2024-03-05 at 14:17 +, Daniel P. Berrangé wrote:
> > On Tue, Feb 06, 2024 at 02:47:34PM +0100, Tim Wiederhake wrote:
> > > > Synchronizing the list of cpu features and models with qemu is
> > > > a
> > > > recurring
> > > > task in libvirt. For x86, this is done by reading
> > > > qom-list-properties for
> > > > max-x86_64-cpu and manually filtering out everthing that does
> > > > not
> > > > look like
> > > > a feature name, as well as parsing target/i386/cpu.c for cpu
> > > > models.
> > > > 
> > > > This is a flawed, tedious and error-prone procedure. Ideally,
> > > > qemu
> > > > and libvirt would query a common source for cpu feature and
> > > > model
> > > > related information. Meanwhile, converting this information
> > > > into an
> > > > easier
> > > > to parse format would help libvirt a lot.
> > > > 
> > > > This patch series converts the cpu feature information present
> > > > in
> > > > target/i386/cpu.c (`feature_word_info`) into a yaml file and
> > > > adds a
> > > > script to generate the c code from this data.
> > 
> > Looking at this fresh, I'm left wondering why I didn't suggested
> > using 'QMP' to expose this information when reviewing the earlier
> > versions. I see Igor did indeed suggest this:
> > 
> >   
> > https://lists.nongnu.org/archive/html/qemu-devel/2023-09/msg03905.html
> > 
> > Your commentry that "qom-list-properties" doesn't distinguish
> > between CPU features and other random QOM properties is bang
> > on the money.
> > 
> > I think what this highlights, is that 'qom-list-properties'
> > is a very poor design/fit for the problem that management apps
> > need to solve in this regard.
> > 
> > Libvirt should not need to manually exclude non-feature properties
> > like 'check' 'enforce' 'migratable' etc.
> > 
> > QEMU already has this knowledge, as IIUC, 'query-cpu-model-
> > expansion'
> > can distinguish this:
> > 
> > query-cpu-model-expansion type=static model={'name':'Nehalem'}
> > {
> >     "return": {
> >     "model": {
> >     "name": "base",
> >     "props": {
> >     "3dnow": false,
> >     ...snip...
> >     "xtpr": false
> >     }
> >     }
> >     }
> > }
> > 
> > We still have the problem that we're not exposing the CPUID/MSR
> > leafs/register bits. So query-cpu-model-expansion isn't a fit
> > for the problem.
> > 
> > Rather than try to design something super general purpose, I'd
> > suggest we take a short cut and design something entirley x86
> > specific, and simply mark the QMP command as "unstable"
> > eg a 'x-query-x86-cpu-model-features', and then basically
> > report all the information libvirt needs there.
> > 
> > This is functionally equivalent to what you expose in the YAML
> > file, while still using QEMU's formal 'QMP' API mechanism, so
> > we avoid inventing a new API concept via YAML.
> > 
> > I think this would avoid need to have a code generator refactor
> > the CPU definitions too. We just need to expose the values of
> > the existing CPUID_xxx constants against each register.
> > 
> > 
> > 
> > With regards,
> > Daniel

Thank you for your feedback.

I do not see the patches and your proposed x-query-x86-cpu-model-
features QMP command being mutually exclusive.

In fact, I'd advocate for merging this patches still, as they provide a
solution (albeit not through QMP) already whereas the QMP command would
still need to be written. Additionally, there are more benefits to the
generate-code approach, as the code generator can be extended to also
generate the feature bits "#define CPUID_* (1U << ...)" in cpu.h,
removing one more source of errors. And with the generated
`feature_word_info` structure being virtually identical to the current
version, I see no downsides: If the generator does become obsolete in
the future, simply remove the python script and the yaml file, and all
that is left is the original feature_word_info code, but better
formatted.

Regards,
Tim




Re: [PATCH v3 0/5] Generate x86 cpu features

2024-03-04 Thread Tim Wiederhake
ping

On Tue, 2024-02-06 at 14:47 +0100, Tim Wiederhake wrote:
> Synchronizing the list of cpu features and models with qemu is a
> recurring
> task in libvirt. For x86, this is done by reading qom-list-properties
> for
> max-x86_64-cpu and manually filtering out everthing that does not
> look like
> a feature name, as well as parsing target/i386/cpu.c for cpu models.
> 
> This is a flawed, tedious and error-prone procedure. Ideally, qemu
> and libvirt would query a common source for cpu feature and model
> related information. Meanwhile, converting this information into an
> easier
> to parse format would help libvirt a lot.
> 
> This patch series converts the cpu feature information present in
> target/i386/cpu.c (`feature_word_info`) into a yaml file and adds a
> script to generate the c code from this data.
> 
> v1:
> https://lists.nongnu.org/archive/html/qemu-devel/2023-08/msg02005.html
> v2:
> https://lists.nongnu.org/archive/html/qemu-devel/2023-09/msg01773.html
> 
> Changes since v2:
> * Rebased on top of current master
> * Removed all "drive-by" changes to feature names ("vmx-ept-uc",
> "vmx-ept-wb",
>   "kvmclock", "vmx-invept-single-context", and
>   "vmx-invept-single-context-noglobals") as these needed further
> discussion.
> * Changes to the generator script reduce the changes in formatting to
> the
>   current feature_word_info even further to address the concern about
> code
>   legibility. See Patch 5, "target/i386: Generate
> feature_word_info.c.inc" for
>   all non-whitespace changes.
> 
> Tim Wiederhake (5):
>   target/i386: Split out feature_word_info
>   target/i386: Translate feature_word_info to yaml
>   target/i386: Remove comments from feature_word_info.c.inc
>   target/i386: Fix feature_word_info.c.inc formatting
>   target/i386: Generate feature_word_info.c.inc
> 
>  target/i386/cpu.c   | 679 +-
>  target/i386/feature_word_info.c.inc | 710
> 
>  target/i386/feature_word_info.py    |  71 +++
>  target/i386/feature_word_info.yaml  | 701
> +++
>  4 files changed, 1483 insertions(+), 678 deletions(-)
>  create mode 100644 target/i386/feature_word_info.c.inc
>  create mode 100755 target/i386/feature_word_info.py
>  create mode 100644 target/i386/feature_word_info.yaml
> 




[PATCH v3 1/5] target/i386: Split out feature_word_info

2024-02-06 Thread Tim Wiederhake
The isolated part will be generated by a script.

Signed-off-by: Tim Wiederhake 
---
 target/i386/cpu.c   | 679 +---
 target/i386/feature_word_info.c.inc | 678 +++
 2 files changed, 679 insertions(+), 678 deletions(-)
 create mode 100644 target/i386/feature_word_info.c.inc

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index ef46755a50..e87bce1970 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -763,684 +763,7 @@ void x86_cpu_vendor_words2str(char *dst, uint32_t vendor1,
 #define TCG_8000_0008_EBX  (CPUID_8000_0008_EBX_XSAVEERPTR | \
   CPUID_8000_0008_EBX_WBNOINVD | CPUID_8000_0008_EBX_KERNEL_FEATURES)
 
-FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
-[FEAT_1_EDX] = {
-.type = CPUID_FEATURE_WORD,
-.feat_names = {
-"fpu", "vme", "de", "pse",
-"tsc", "msr", "pae", "mce",
-"cx8", "apic", NULL, "sep",
-"mtrr", "pge", "mca", "cmov",
-"pat", "pse36", "pn" /* Intel psn */, "clflush" /* Intel clfsh */,
-NULL, "ds" /* Intel dts */, "acpi", "mmx",
-"fxsr", "sse", "sse2", "ss",
-"ht" /* Intel htt */, "tm", "ia64", "pbe",
-},
-.cpuid = {.eax = 1, .reg = R_EDX, },
-.tcg_features = TCG_FEATURES,
-.no_autoenable_flags = CPUID_HT,
-},
-[FEAT_1_ECX] = {
-.type = CPUID_FEATURE_WORD,
-.feat_names = {
-"pni" /* Intel,AMD sse3 */, "pclmulqdq", "dtes64", "monitor",
-"ds-cpl", "vmx", "smx", "est",
-"tm2", "ssse3", "cid", NULL,
-"fma", "cx16", "xtpr", "pdcm",
-NULL, "pcid", "dca", "sse4.1",
-"sse4.2", "x2apic", "movbe", "popcnt",
-"tsc-deadline", "aes", "xsave", NULL /* osxsave */,
-"avx", "f16c", "rdrand", "hypervisor",
-},
-.cpuid = { .eax = 1, .reg = R_ECX, },
-.tcg_features = TCG_EXT_FEATURES,
-},
-/* Feature names that are already defined on feature_name[] but
- * are set on CPUID[8000_0001].EDX on AMD CPUs don't have their
- * names on feat_names below. They are copied automatically
- * to features[FEAT_8000_0001_EDX] if and only if CPU vendor is AMD.
- */
-[FEAT_8000_0001_EDX] = {
-.type = CPUID_FEATURE_WORD,
-.feat_names = {
-NULL /* fpu */, NULL /* vme */, NULL /* de */, NULL /* pse */,
-NULL /* tsc */, NULL /* msr */, NULL /* pae */, NULL /* mce */,
-NULL /* cx8 */, NULL /* apic */, NULL, "syscall",
-NULL /* mtrr */, NULL /* pge */, NULL /* mca */, NULL /* cmov */,
-NULL /* pat */, NULL /* pse36 */, NULL, NULL /* Linux mp */,
-"nx", NULL, "mmxext", NULL /* mmx */,
-NULL /* fxsr */, "fxsr-opt", "pdpe1gb", "rdtscp",
-NULL, "lm", "3dnowext", "3dnow",
-},
-.cpuid = { .eax = 0x8001, .reg = R_EDX, },
-.tcg_features = TCG_EXT2_FEATURES,
-},
-[FEAT_8000_0001_ECX] = {
-.type = CPUID_FEATURE_WORD,
-.feat_names = {
-"lahf-lm", "cmp-legacy", "svm", "extapic",
-"cr8legacy", "abm", "sse4a", "misalignsse",
-"3dnowprefetch", "osvw", "ibs", "xop",
-"skinit", "wdt", NULL, "lwp",
-"fma4", "tce", NULL, "nodeid-msr",
-NULL, "tbm", "topoext", "perfctr-core",
-"perfctr-nb", NULL, NULL, NULL,
-NULL, NULL, NULL, NULL,
-},
-.cpuid = { .eax = 0x8001, .reg = R_ECX, },
-.tcg_features = TCG_EXT3_FEATURES,
-/*
- * TOPOEXT is always allowed but can't be enabled blindly by
- * "-cpu host", as it requires consistent cache topology info
- * to be provided so it doesn't confuse guests.
- */
-.no_autoenable_flags = CPUID_EXT3_TOPOEXT,
-},
-[FEAT_C000_0001_EDX] = {
-.type = CPUID_FEATURE_WORD,
-.feat_names = {
-NULL, NULL, "xstore", "xstore-en",
-NULL, NUL

[PATCH v3 4/5] target/i386: Fix feature_word_info.c.inc formatting

2024-02-06 Thread Tim Wiederhake
Make the formatting of the file more regular. This reduces the
diff to the generated version.

Signed-off-by: Tim Wiederhake 
---
 target/i386/feature_word_info.c.inc | 136 ++--
 1 file changed, 86 insertions(+), 50 deletions(-)

diff --git a/target/i386/feature_word_info.c.inc 
b/target/i386/feature_word_info.c.inc
index 152d4b45a9..4beae01918 100644
--- a/target/i386/feature_word_info.c.inc
+++ b/target/i386/feature_word_info.c.inc
@@ -11,7 +11,10 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 "fxsr", "sse", "sse2", "ss",
 "ht", "tm", "ia64", "pbe",
 },
-.cpuid = {.eax = 1, .reg = R_EDX, },
+.cpuid = {
+.eax = 1,
+.reg = R_EDX,
+},
 .tcg_features = TCG_FEATURES,
 .no_autoenable_flags = CPUID_HT,
 },
@@ -27,7 +30,10 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 "tsc-deadline", "aes", "xsave", NULL,
 "avx", "f16c", "rdrand", "hypervisor",
 },
-.cpuid = { .eax = 1, .reg = R_ECX, },
+.cpuid = {
+.eax = 1,
+.reg = R_ECX,
+},
 .tcg_features = TCG_EXT_FEATURES,
 },
 [FEAT_8000_0001_EDX] = {
@@ -42,7 +48,10 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 NULL, "fxsr-opt", "pdpe1gb", "rdtscp",
 NULL, "lm", "3dnowext", "3dnow",
 },
-.cpuid = { .eax = 0x8001, .reg = R_EDX, },
+.cpuid = {
+.eax = 0x8001,
+.reg = R_EDX,
+},
 .tcg_features = TCG_EXT2_FEATURES,
 },
 [FEAT_8000_0001_ECX] = {
@@ -57,7 +66,10 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 "perfctr-nb", NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
 },
-.cpuid = { .eax = 0x8001, .reg = R_ECX, },
+.cpuid = {
+.eax = 0x8001,
+.reg = R_ECX,
+},
 .tcg_features = TCG_EXT3_FEATURES,
 .no_autoenable_flags = CPUID_EXT3_TOPOEXT,
 },
@@ -73,7 +85,10 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 NULL, NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
 },
-.cpuid = { .eax = 0xC001, .reg = R_EDX, },
+.cpuid = {
+.eax = 0xC001,
+.reg = R_EDX,
+},
 .tcg_features = TCG_EXT4_FEATURES,
 },
 [FEAT_KVM] = {
@@ -88,7 +103,10 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 "kvmclock-stable-bit", NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
 },
-.cpuid = { .eax = KVM_CPUID_FEATURES, .reg = R_EAX, },
+.cpuid = {
+.eax = KVM_CPUID_FEATURES,
+.reg = R_EAX,
+},
 .tcg_features = TCG_KVM_FEATURES,
 },
 [FEAT_KVM_HINTS] = {
@@ -103,7 +121,10 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 NULL, NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
 },
-.cpuid = { .eax = KVM_CPUID_FEATURES, .reg = R_EDX, },
+.cpuid = {
+.eax = KVM_CPUID_FEATURES,
+.reg = R_EDX,
+},
 .tcg_features = TCG_KVM_FEATURES,
 .no_autoenable_flags = ~0U,
 },
@@ -111,7 +132,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 .type = CPUID_FEATURE_WORD,
 .feat_names = {
 "npt", "lbrv", "svm-lock", "nrip-save",
-"tsc-scale", "vmcb-clean",  "flushbyasid", "decodeassists",
+"tsc-scale", "vmcb-clean", "flushbyasid", "decodeassists",
 NULL, NULL, "pause-filter", NULL,
 "pfthreshold", "avic", NULL, "v-vmsave-vmload",
 "vgif", NULL, NULL, NULL,
@@ -119,7 +140,10 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 NULL, "vnmi", NULL, NULL,
 "svme-addr-chk", NULL, NULL, NULL,
 },
-.cpuid = { .eax = 0x800A, .reg = R_EDX, },
+.cpuid = {
+.eax = 0x800A,
+.reg = R_EDX,
+},
 .tcg_features = TCG_SVM_FEATURES,
 },
 [FEAT_7_0_EBX] = {
@@ -136,7 +160,8 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 },
 .cpuid = {
 .eax = 7,
-.needs_ecx = true, .ecx = 0,
+.needs_ecx = true,
+.ecx = 0,
 .reg = R_EBX,
 },
 .tcg_features = TCG_7_0_EBX_FEATURES,
@@ -155,7 +180,8 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 },
 .cpuid = {
 

[PATCH v3 2/5] target/i386: Translate feature_word_info to yaml

2024-02-06 Thread Tim Wiederhake
This is the data file that will be used to generate the C code.
All information, including the comments, is preserved.

Signed-off-by: Tim Wiederhake 
---
 target/i386/feature_word_info.yaml | 699 +
 1 file changed, 699 insertions(+)
 create mode 100644 target/i386/feature_word_info.yaml

diff --git a/target/i386/feature_word_info.yaml 
b/target/i386/feature_word_info.yaml
new file mode 100644
index 00..7a8ca3f051
--- /dev/null
+++ b/target/i386/feature_word_info.yaml
@@ -0,0 +1,699 @@
+- index: FEAT_1_EDX
+  cpuid:
+eax: '1'
+reg: R_EDX
+  feat_names:
+0: fpu
+1: vme
+2: de
+3: pse
+4: tsc
+5: msr
+6: pae
+7: mce
+8: cx8
+9: apic
+11: sep
+12: mtrr
+13: pge
+14: mca
+15: cmov
+16: pat
+17: pse36
+18: pn  # Intel psn
+19: clflush # Intel clfsh
+21: ds  # Intel dts
+22: acpi
+23: mmx
+24: fxsr
+25: sse
+26: sse2
+27: ss
+28: ht  # Intel htt
+29: tm
+30: ia64
+31: pbe
+  no_autoenable_flags: CPUID_HT
+  tcg_features: TCG_FEATURES
+  type: CPUID_FEATURE_WORD
+
+- index: FEAT_1_ECX
+  cpuid:
+eax: '1'
+reg: R_ECX
+  feat_names:
+0: pni  # Intel,AMD sse3
+1: pclmulqdq
+2: dtes64
+3: monitor
+4: ds-cpl
+5: vmx
+6: smx
+7: est
+8: tm2
+9: ssse3
+10: cid
+12: fma
+13: cx16
+14: xtpr
+15: pdcm
+17: pcid
+18: dca
+19: sse4.1
+20: sse4.2
+21: x2apic
+22: movbe
+23: popcnt
+24: tsc-deadline
+25: aes
+26: xsave
+# 27: osxsave
+28: avx
+29: f16c
+30: rdrand
+31: hypervisor
+  tcg_features: TCG_EXT_FEATURES
+  type: CPUID_FEATURE_WORD
+
+# Feature names that are already defined on feature_name[] but are set on
+# CPUID[8000_0001].EDX on AMD CPUs don't have their names on feat_names below.
+# They are copied automatically to features[FEAT_8000_0001_EDX] if and only if
+# CPU vendor is AMD.
+- index: FEAT_8000_0001_EDX
+  cpuid:
+eax: '0x8001'
+reg: R_EDX
+  feat_names:
+# 0: fpu
+# 1: vme
+# 2: de
+# 3: pse
+# 4: tsc
+# 5: msr
+# 6: pae
+# 7: mce
+# 8: cx8
+# 9: apic
+11: syscall
+# 12: mtrr
+# 13: pge
+# 14: mca
+# 15: cmov
+# 16: pat
+# 17: pse36
+# 19: Linux mp
+20: nx
+22: mmxext
+# 23: mmx
+# 24: fxsr
+25: fxsr-opt
+26: pdpe1gb
+27: rdtscp
+29: lm
+30: 3dnowext
+31: 3dnow
+  tcg_features: TCG_EXT2_FEATURES
+  type: CPUID_FEATURE_WORD
+
+- index: FEAT_8000_0001_ECX
+  cpuid:
+eax: '0x8001'
+reg: R_ECX
+  feat_names:
+0: lahf-lm
+1: cmp-legacy
+2: svm
+3: extapic
+4: cr8legacy
+5: abm
+6: sse4a
+7: misalignsse
+8: 3dnowprefetch
+9: osvw
+10: ibs
+11: xop
+12: skinit
+13: wdt
+15: lwp
+16: fma4
+17: tce
+19: nodeid-msr
+21: tbm
+22: topoext
+23: perfctr-core
+24: perfctr-nb
+  # TOPOEXT is always allowed but can't be enabled blindly by "-cpu host", as
+  # it requires consistent cache topology info to be provided so it doesn't
+  # confuse guests.
+  no_autoenable_flags: CPUID_EXT3_TOPOEXT
+  tcg_features: TCG_EXT3_FEATURES
+  type: CPUID_FEATURE_WORD
+
+- index: FEAT_C000_0001_EDX
+  cpuid:
+eax: '0xC001'
+reg: R_EDX
+  feat_names:
+2: xstore
+3: xstore-en
+6: xcrypt
+7: xcrypt-en
+8: ace2
+9: ace2-en
+10: phe
+11: phe-en
+12: pmm
+13: pmm-en
+  tcg_features: TCG_EXT4_FEATURES
+  type: CPUID_FEATURE_WORD
+
+- index: FEAT_KVM
+  cpuid:
+eax: KVM_CPUID_FEATURES
+reg: R_EAX
+  feat_names:
+0: kvmclock
+1: kvm-nopiodelay
+2: kvm-mmu
+3: kvmclock
+4: kvm-asyncpf
+5: kvm-steal-time
+6: kvm-pv-eoi
+7: kvm-pv-unhalt
+9: kvm-pv-tlb-flush
+11: kvm-pv-ipi
+12: kvm-poll-control
+13: kvm-pv-sched-yield
+14: kvm-asyncpf-int
+15: kvm-msi-ext-dest-id
+24: kvmclock-stable-bit
+  tcg_features: TCG_KVM_FEATURES
+  type: CPUID_FEATURE_WORD
+
+- index: FEAT_KVM_HINTS
+  cpuid:
+eax: KVM_CPUID_FEATURES
+reg: R_EDX
+  feat_names:
+0: kvm-hint-dedicated
+  # KVM hints aren't auto-enabled by -cpu host, they need to be explicitly
+  # enabled in the command-line.
+  no_autoenable_flags: ~0U
+  tcg_features: TCG_KVM_FEATURES
+  type: CPUID_FEATURE_WORD
+
+- index: FEAT_SVM
+  cpuid:
+eax: '0x800A'
+reg: R_EDX
+  feat_names:
+0: npt
+1: lbrv
+2: svm-lock
+3: nrip-save
+4: tsc-scale
+5: vmcb-clean
+6: flushbyasid
+7: decodeassists
+10: pause-filter
+12: pfthreshold
+13: avic
+15: v-vmsave-vmload
+16: vgif
+25: vnmi
+28: svme-addr-chk
+  tcg_features: TCG_SVM_FEATURES
+  type: CPUID_FEATURE_WORD
+
+- index: FEAT_7_0_EBX
+  cpuid:
+eax: '7'
+ecx: '0'
+needs_ecx: 'true'
+

[PATCH v3 5/5] target/i386: Generate feature_word_info.c.inc

2024-02-06 Thread Tim Wiederhake
Signed-off-by: Tim Wiederhake 
---
 target/i386/feature_word_info.c.inc | 30 ++--
 target/i386/feature_word_info.py| 71 +
 target/i386/feature_word_info.yaml  |  2 +
 3 files changed, 99 insertions(+), 4 deletions(-)
 create mode 100755 target/i386/feature_word_info.py

diff --git a/target/i386/feature_word_info.c.inc 
b/target/i386/feature_word_info.c.inc
index 4beae01918..b4bece08fe 100644
--- a/target/i386/feature_word_info.c.inc
+++ b/target/i386/feature_word_info.c.inc
@@ -1,3 +1,5 @@
+/* This file is generated by feature_word_info.py. */
+
 FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 [FEAT_1_EDX] = {
 .type = CPUID_FEATURE_WORD,
@@ -587,9 +589,22 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 [FEAT_VMX_BASIC] = {
 .type = MSR_FEATURE_WORD,
 .feat_names = {
-[54] = "vmx-ins-outs",
-[55] = "vmx-true-ctls",
-[56] = "vmx-any-errcode",
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, "vmx-ins-outs", "vmx-true-ctls",
+"vmx-any-errcode", NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
 },
 .msr = {
 .index = MSR_IA32_VMX_BASIC,
@@ -599,7 +614,14 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 [FEAT_VMX_VMFUNC] = {
 .type = MSR_FEATURE_WORD,
 .feat_names = {
-[0] = "vmx-eptp-switching",
+"vmx-eptp-switching", NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
 },
 .msr = {
 .index = MSR_IA32_VMX_VMFUNC,
diff --git a/target/i386/feature_word_info.py b/target/i386/feature_word_info.py
new file mode 100755
index 00..c9aa00900c
--- /dev/null
+++ b/target/i386/feature_word_info.py
@@ -0,0 +1,71 @@
+#!/bin/env python3
+
+import os
+import yaml
+
+
+def write_feature_word(f, data):
+f.write("[{}] = {{\n".format(data["index"]))
+f.write(".type = {},\n".format(data["type"]))
+
+if "feat_names" in data:
+keys = data.get("feat_names").keys()
+if not keys or max(keys) < 32:
+highest = 32
+else:
+highest = 64
+
+f.write(".feat_names = {\n")
+for i in range(0, highest, 4):
+names = [data.get("feat_names").get(j) for j in range(i, i + 4)]
+names = ["NULL" if not name else f"\"{name}\"" for name in names]
+display = ", ".join(names)
+if len(display) < 90:
+f.write("{},\n".format(display))
+else:
+f.write("{}, {},\n".format(names[0], names[1]))
+f.write("{}, {},\n".format(names[2], names[3]))
+f.write("},\n")
+if "cpuid" in data:
+cpuid = data["cpuid"]
+f.write(".cpuid = {\n")
+f.write(".eax = {},\n".format(cpuid["eax"]))
+if "ecx" in cpuid:
+f.write(".needs_ecx = true,\n")
+f.write(".ecx = {},\n".format(cpuid["ecx"]))
+f.write(".reg = {},\n".format(cpuid["reg"]))
+f.write("},\n")
+if "msr" in data:
+f.write(".msr = {\n")
+f.write(".index = {},\n".format(data["msr"]))
+f.write("},\n")
+if "tcg_features" in data:
+f.write(".tcg_features = {},\n".format(data["tcg_features"]))
+if "unmigratable_flags" in data:
+f.write(".unmigratable_flags = {},\n".format(
+data["unmigratable_flags"]))
+if "migratable_flags" in data:
+f.write(".migratable_flags = {},\n".format(
+data["migratable_flags"]))
+if "no_autoenable_flags" in data:
+f.write(" 

[PATCH v3 3/5] target/i386: Remove comments from feature_word_info.c.inc

2024-02-06 Thread Tim Wiederhake
The comments are preserved in the yaml file.

Signed-off-by: Tim Wiederhake 
---
 target/i386/feature_word_info.c.inc | 56 -
 1 file changed, 15 insertions(+), 41 deletions(-)

diff --git a/target/i386/feature_word_info.c.inc 
b/target/i386/feature_word_info.c.inc
index 4c6a1613ae..152d4b45a9 100644
--- a/target/i386/feature_word_info.c.inc
+++ b/target/i386/feature_word_info.c.inc
@@ -6,10 +6,10 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 "tsc", "msr", "pae", "mce",
 "cx8", "apic", NULL, "sep",
 "mtrr", "pge", "mca", "cmov",
-"pat", "pse36", "pn" /* Intel psn */, "clflush" /* Intel clfsh */,
-NULL, "ds" /* Intel dts */, "acpi", "mmx",
+"pat", "pse36", "pn", "clflush",
+NULL, "ds", "acpi", "mmx",
 "fxsr", "sse", "sse2", "ss",
-"ht" /* Intel htt */, "tm", "ia64", "pbe",
+"ht", "tm", "ia64", "pbe",
 },
 .cpuid = {.eax = 1, .reg = R_EDX, },
 .tcg_features = TCG_FEATURES,
@@ -18,33 +18,28 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 [FEAT_1_ECX] = {
 .type = CPUID_FEATURE_WORD,
 .feat_names = {
-"pni" /* Intel,AMD sse3 */, "pclmulqdq", "dtes64", "monitor",
+"pni", "pclmulqdq", "dtes64", "monitor",
 "ds-cpl", "vmx", "smx", "est",
 "tm2", "ssse3", "cid", NULL,
 "fma", "cx16", "xtpr", "pdcm",
 NULL, "pcid", "dca", "sse4.1",
 "sse4.2", "x2apic", "movbe", "popcnt",
-"tsc-deadline", "aes", "xsave", NULL /* osxsave */,
+"tsc-deadline", "aes", "xsave", NULL,
 "avx", "f16c", "rdrand", "hypervisor",
 },
 .cpuid = { .eax = 1, .reg = R_ECX, },
 .tcg_features = TCG_EXT_FEATURES,
 },
-/* Feature names that are already defined on feature_name[] but
- * are set on CPUID[8000_0001].EDX on AMD CPUs don't have their
- * names on feat_names below. They are copied automatically
- * to features[FEAT_8000_0001_EDX] if and only if CPU vendor is AMD.
- */
 [FEAT_8000_0001_EDX] = {
 .type = CPUID_FEATURE_WORD,
 .feat_names = {
-NULL /* fpu */, NULL /* vme */, NULL /* de */, NULL /* pse */,
-NULL /* tsc */, NULL /* msr */, NULL /* pae */, NULL /* mce */,
-NULL /* cx8 */, NULL /* apic */, NULL, "syscall",
-NULL /* mtrr */, NULL /* pge */, NULL /* mca */, NULL /* cmov */,
-NULL /* pat */, NULL /* pse36 */, NULL, NULL /* Linux mp */,
-"nx", NULL, "mmxext", NULL /* mmx */,
-NULL /* fxsr */, "fxsr-opt", "pdpe1gb", "rdtscp",
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, "syscall",
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+"nx", NULL, "mmxext", NULL,
+NULL, "fxsr-opt", "pdpe1gb", "rdtscp",
 NULL, "lm", "3dnowext", "3dnow",
 },
 .cpuid = { .eax = 0x8001, .reg = R_EDX, },
@@ -64,11 +59,6 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 },
 .cpuid = { .eax = 0x8001, .reg = R_ECX, },
 .tcg_features = TCG_EXT3_FEATURES,
-/*
- * TOPOEXT is always allowed but can't be enabled blindly by
- * "-cpu host", as it requires consistent cache topology info
- * to be provided so it doesn't confuse guests.
- */
 .no_autoenable_flags = CPUID_EXT3_TOPOEXT,
 },
 [FEAT_C000_0001_EDX] = {
@@ -115,10 +105,6 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 },
 .cpuid = { .eax = KVM_CPUID_FEATURES, .reg = R_EDX, },
 .tcg_features = TCG_KVM_FEATURES,
-/*
- * KVM hints aren't auto-enabled by -cpu host, they need to be
- * explicitly enabled in the command-line.
- */
 .no_autoenable_flags = ~0U,
 },
 [FEAT_SVM] = {
@@ -159,7 +145,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 .type 

[PATCH v3 0/5] Generate x86 cpu features

2024-02-06 Thread Tim Wiederhake
Synchronizing the list of cpu features and models with qemu is a recurring
task in libvirt. For x86, this is done by reading qom-list-properties for
max-x86_64-cpu and manually filtering out everthing that does not look like
a feature name, as well as parsing target/i386/cpu.c for cpu models.

This is a flawed, tedious and error-prone procedure. Ideally, qemu
and libvirt would query a common source for cpu feature and model
related information. Meanwhile, converting this information into an easier
to parse format would help libvirt a lot.

This patch series converts the cpu feature information present in
target/i386/cpu.c (`feature_word_info`) into a yaml file and adds a
script to generate the c code from this data.

v1: https://lists.nongnu.org/archive/html/qemu-devel/2023-08/msg02005.html
v2: https://lists.nongnu.org/archive/html/qemu-devel/2023-09/msg01773.html

Changes since v2:
* Rebased on top of current master
* Removed all "drive-by" changes to feature names ("vmx-ept-uc", "vmx-ept-wb",
  "kvmclock", "vmx-invept-single-context", and
  "vmx-invept-single-context-noglobals") as these needed further discussion.
* Changes to the generator script reduce the changes in formatting to the
  current feature_word_info even further to address the concern about code
  legibility. See Patch 5, "target/i386: Generate feature_word_info.c.inc" for
  all non-whitespace changes.

Tim Wiederhake (5):
  target/i386: Split out feature_word_info
  target/i386: Translate feature_word_info to yaml
  target/i386: Remove comments from feature_word_info.c.inc
  target/i386: Fix feature_word_info.c.inc formatting
  target/i386: Generate feature_word_info.c.inc

 target/i386/cpu.c   | 679 +-
 target/i386/feature_word_info.c.inc | 710 
 target/i386/feature_word_info.py|  71 +++
 target/i386/feature_word_info.yaml  | 701 +++
 4 files changed, 1483 insertions(+), 678 deletions(-)
 create mode 100644 target/i386/feature_word_info.c.inc
 create mode 100755 target/i386/feature_word_info.py
 create mode 100644 target/i386/feature_word_info.yaml

-- 
2.43.0




Re: [PATCH v2 00/10] Generate x86 cpu features

2023-09-15 Thread Tim Wiederhake
On Mon, 2023-09-11 at 13:26 +0200, Igor Mammedov wrote:
> On Fri, 8 Sep 2023 17:55:12 +0100
> Daniel P. Berrangé  wrote:
> 
> > On Fri, Sep 08, 2023 at 04:48:46PM +0200, Igor Mammedov wrote:
> > > On Fri,  8 Sep 2023 14:45:24 +0200
> > > Tim Wiederhake  wrote:
> > >   
> > > > Synchronizing the list of cpu features and models with qemu is
> > > > a recurring
> > > > task in libvirt. For x86, this is done by reading qom-list-
> > > > properties for
> > > > max-x86_64-cpu and manually filtering out everthing that does
> > > > not look like
> > > > a feature name, as well as parsing target/i386/cpu.c for cpu
> > > > models.  
> > > 
> > > modulo fixing typos/name conflicts in 1st 3 patches,
> > > 
> > > I don't think it's a great idea for libvirt (or any other user)
> > > to parse
> > > QEMU source (whether it's C code or yaml) or other way around for
> > > users
> > > to influence QEMU internals.  
> > 
> > NB It isn't for libvirt to parse at runtime, rather it is for
> > libvirt
> > maintainers to consume during dev, so libvirt keeps in sync with
> > QEMU
> > features.
> 
> As QEMU dev, I'm not fond of code generators as sometimes they make
> difficult for me to read, and on to of that inventing new 'language'
> to describe features that works on some cases only (not everything
> is described in feature array, and for non x86 properties mostly
> coded in initfn/realizefn).
> (I'd dislike it less if it were part of QMP schema as it gets us
> closer to processing '-device' with QMP machinery).
> 
> why not use existing QMP interface there as well (or alter it if
> it's not sufficient)?
> 
> 
I understand your concern regarding the legibility of generated code.
If you have a look at patches 6 - 9, you will see that the only changes
introduced are white space changes, harmonizing the usage of trailing
commas, breaking lines, and filling the "feat_names" field fully. That,
if anything, makes the code more readable in my opinion.

The format of the yaml file is chosen to mimic struct FeatureWordInfo
as closely as possible, it is effectively a 1:1 mapping.

Regarding using the QMP interface: That is what libvirt is currently
doing, see [1] if you are interested. The problem with that approach is
that the response to

{
"execute": "qom-list-properties",
"arguments": { "typename": "max-x86_64-cpu"}
}

does not distinguish between cpu features and other information about
that cpu model, thus forcing libvirt to maintain a list of non-
features. Examples of such items are "check", "enforce", "start-
powered-off", or "migratable".

Additionally, the response does not include information about the cpuid
leaf, register and bit, or msr register respectively. This has to be
supplemented manually.

If there is a way to query for this information that I am not aware of,
please let me know.

Ultimately I would love to see libvirt stop querying qemu for x86 cpu
feature information but instead have both qemu and libvirt query a
common source / database for this kind of information, for all
architectures, not only x86.

Regards,
Tim

[1]
https://gitlab.com/libvirt/libvirt/-/blob/master/src/cpu_map/sync_qemu_features_i386.py




Re: [PATCH v2 03/10] target/i386: Fix duplicated feature name in FEAT_KVM

2023-09-15 Thread Tim Wiederhake
On Fri, 2023-09-08 at 16:21 +0200, Igor Mammedov wrote:
> On Fri,  8 Sep 2023 14:45:27 +0200
> Tim Wiederhake  wrote:
> 
> > The mistake became apparent as there were two features with the
> > same name
> > in this cpuid leaf. The names are now in line with the
> > documentation from
> > https://kernel.org/doc/html/latest/virt/kvm/x86/cpuid.html
> 
> I'd describe what duplication breaks and it's effects.
> 
> and also why it's considered that it's safe to change names here.
> 
This is my first contribution to qemu. I do not know whether it is safe
to change the names, nor how to find out I'm afraid. I ran `make test`,
if that is what you mean.

Please let me know what you need me to do here.

In the meantime, I will separate the patches that rename / fix feature
names from the remaining patch set, and split the changes up as
requested.

Regards,
Tim

> > Fixes: 642258c6c7 ("kvm: add kvmclock to its second bit")
> > Signed-off-by: Tim Wiederhake 
> > ---
> >  target/i386/cpu.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> > index f10d343935..f0fedf4b88 100644
> > --- a/target/i386/cpu.c
> > +++ b/target/i386/cpu.c
> > @@ -852,7 +852,7 @@ FeatureWordInfo
> > feature_word_info[FEATURE_WORDS] = {
> >  [FEAT_KVM] = {
> >  .type = CPUID_FEATURE_WORD,
> >  .feat_names = {
> > -    "kvmclock", "kvm-nopiodelay", "kvm-mmu", "kvmclock",
> > +    "kvmclock", "kvm-nopiodelay", "kvm-mmu", "kvmclock2",
> >  "kvm-asyncpf", "kvm-steal-time", "kvm-pv-eoi", "kvm-
> > pv-unhalt",
> >  NULL, "kvm-pv-tlb-flush", NULL, "kvm-pv-ipi",
> >  "kvm-poll-control", "kvm-pv-sched-yield", "kvm-
> > asyncpf-int", "kvm-msi-ext-dest-id",
> 




[PATCH v2 05/10] target/i386: Translate feature_word_info to yaml

2023-09-08 Thread Tim Wiederhake
This is the data file that will be used to generate the C code.
All information, including the comments, is preserved.

Signed-off-by: Tim Wiederhake 
---
 target/i386/feature_word_info.yaml | 695 +
 1 file changed, 695 insertions(+)
 create mode 100644 target/i386/feature_word_info.yaml

diff --git a/target/i386/feature_word_info.yaml 
b/target/i386/feature_word_info.yaml
new file mode 100644
index 00..cd6cdc8053
--- /dev/null
+++ b/target/i386/feature_word_info.yaml
@@ -0,0 +1,695 @@
+- index: FEAT_1_EDX
+  cpuid:
+eax: '1'
+reg: R_EDX
+  feat_names:
+0: fpu
+1: vme
+2: de
+3: pse
+4: tsc
+5: msr
+6: pae
+7: mce
+8: cx8
+9: apic
+11: sep
+12: mtrr
+13: pge
+14: mca
+15: cmov
+16: pat
+17: pse36
+18: pn  # Intel psn
+19: clflush # Intel clfsh
+21: ds  # Intel dts
+22: acpi
+23: mmx
+24: fxsr
+25: sse
+26: sse2
+27: ss
+28: ht  # Intel htt
+29: tm
+30: ia64
+31: pbe
+  tcg_features: TCG_FEATURES
+  type: CPUID_FEATURE_WORD
+
+- index: FEAT_1_ECX
+  cpuid:
+eax: '1'
+reg: R_ECX
+  feat_names:
+0: pni  # Intel,AMD sse3
+1: pclmulqdq
+2: dtes64
+3: monitor
+4: ds-cpl
+5: vmx
+6: smx
+7: est
+8: tm2
+9: ssse3
+10: cid
+12: fma
+13: cx16
+14: xtpr
+15: pdcm
+17: pcid
+18: dca
+19: sse4.1
+20: sse4.2
+21: x2apic
+22: movbe
+23: popcnt
+24: tsc-deadline
+25: aes
+26: xsave
+# 27: osxsave
+28: avx
+29: f16c
+30: rdrand
+31: hypervisor
+  tcg_features: TCG_EXT_FEATURES
+  type: CPUID_FEATURE_WORD
+
+# Feature names that are already defined on feature_name[] but are set on
+# CPUID[8000_0001].EDX on AMD CPUs don't have their names on feat_names below.
+# They are copied automatically to features[FEAT_8000_0001_EDX] if and only if
+# CPU vendor is AMD.
+- index: FEAT_8000_0001_EDX
+  cpuid:
+eax: '0x8001'
+reg: R_EDX
+  feat_names:
+# 0: fpu
+# 1: vme
+# 2: de
+# 3: pse
+# 4: tsc
+# 5: msr
+# 6: pae
+# 7: mce
+# 8: cx8
+# 9: apic
+11: syscall
+# 12: mtrr
+# 13: pge
+# 14: mca
+# 15: cmov
+# 16: pat
+# 17: pse36
+# 19: Linux mp
+20: nx
+22: mmxext
+# 23: mmx
+# 24: fxsr
+25: fxsr-opt
+26: pdpe1gb
+27: rdtscp
+29: lm
+30: 3dnowext
+31: 3dnow
+  tcg_features: TCG_EXT2_FEATURES
+  type: CPUID_FEATURE_WORD
+
+- index: FEAT_8000_0001_ECX
+  cpuid:
+eax: '0x8001'
+reg: R_ECX
+  feat_names:
+0: lahf-lm
+1: cmp-legacy
+2: svm
+3: extapic
+4: cr8legacy
+5: abm
+6: sse4a
+7: misalignsse
+8: 3dnowprefetch
+9: osvw
+10: ibs
+11: xop
+12: skinit
+13: wdt
+15: lwp
+16: fma4
+17: tce
+19: nodeid-msr
+21: tbm
+22: topoext
+23: perfctr-core
+24: perfctr-nb
+  # TOPOEXT is always allowed but can't be enabled blindly by "-cpu host", as
+  # it requires consistent cache topology info to be provided so it doesn't
+  # confuse guests.
+  no_autoenable_flags: CPUID_EXT3_TOPOEXT
+  tcg_features: TCG_EXT3_FEATURES
+  type: CPUID_FEATURE_WORD
+
+- index: FEAT_C000_0001_EDX
+  cpuid:
+eax: '0xC001'
+reg: R_EDX
+  feat_names:
+2: xstore
+3: xstore-en
+6: xcrypt
+7: xcrypt-en
+8: ace2
+9: ace2-en
+10: phe
+11: phe-en
+12: pmm
+13: pmm-en
+  tcg_features: TCG_EXT4_FEATURES
+  type: CPUID_FEATURE_WORD
+
+- index: FEAT_KVM
+  cpuid:
+eax: KVM_CPUID_FEATURES
+reg: R_EAX
+  feat_names:
+0: kvmclock
+1: kvm-nopiodelay
+2: kvm-mmu
+3: kvmclock2
+4: kvm-asyncpf
+5: kvm-steal-time
+6: kvm-pv-eoi
+7: kvm-pv-unhalt
+9: kvm-pv-tlb-flush
+11: kvm-pv-ipi
+12: kvm-poll-control
+13: kvm-pv-sched-yield
+14: kvm-asyncpf-int
+15: kvm-msi-ext-dest-id
+24: kvmclock-stable-bit
+  tcg_features: TCG_KVM_FEATURES
+  type: CPUID_FEATURE_WORD
+
+- index: FEAT_KVM_HINTS
+  cpuid:
+eax: KVM_CPUID_FEATURES
+reg: R_EDX
+  feat_names:
+0: kvm-hint-dedicated
+  # KVM hints aren't auto-enabled by -cpu host, they need to be explicitly
+  # enabled in the command-line.
+  no_autoenable_flags: ~0U
+  tcg_features: TCG_KVM_FEATURES
+  type: CPUID_FEATURE_WORD
+
+- index: FEAT_SVM
+  cpuid:
+eax: '0x800A'
+reg: R_EDX
+  feat_names:
+0: npt
+1: lbrv
+2: svm-lock
+3: nrip-save
+4: tsc-scale
+5: vmcb-clean
+6: flushbyasid
+7: decodeassists
+10: pause-filter
+12: pfthreshold
+13: avic
+15: v-vmsave-vmload
+16: vgif
+25: vnmi
+28: svme-addr-chk
+  tcg_features: TCG_SVM_FEATURES
+  type: CPUID_FEATURE_WORD
+
+- index: FEAT_7_0_EBX
+  cpuid:
+eax: '7'
+ecx: '0'
+needs_ecx: 'true'
+reg: R_EBX
+  feat_names:
+0: fsgsb

[PATCH v2 10/10] target/i386: Autogenerate feature_word_info.c.inc

2023-09-08 Thread Tim Wiederhake
This introduces no semantic changes to the file.

Signed-off-by: Tim Wiederhake 
---
 target/i386/feature_word_info.c.inc |  2 +
 target/i386/feature_word_info.py| 62 +
 target/i386/feature_word_info.yaml  |  2 +
 3 files changed, 66 insertions(+)
 create mode 100755 target/i386/feature_word_info.py

diff --git a/target/i386/feature_word_info.c.inc 
b/target/i386/feature_word_info.c.inc
index 5bac4aaeba..fb699528a6 100644
--- a/target/i386/feature_word_info.c.inc
+++ b/target/i386/feature_word_info.c.inc
@@ -1,3 +1,5 @@
+/* This file is generated by feature_word_info.py. */
+
 FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 [FEAT_1_EDX] = {
 .type = CPUID_FEATURE_WORD,
diff --git a/target/i386/feature_word_info.py b/target/i386/feature_word_info.py
new file mode 100755
index 00..e5773c8dbf
--- /dev/null
+++ b/target/i386/feature_word_info.py
@@ -0,0 +1,62 @@
+#!/bin/env python3
+
+import os
+import yaml
+
+
+def write_feature_word(f, data):
+f.write("[{}] = {{\n".format(data["index"]))
+f.write(".type = {},\n".format(data["type"]))
+f.write(".feat_names = {\n")
+for index in range(64):
+name = data.get("feat_names", {}).get(index)
+name = "NULL" if name is None else "\"" + name + "\""
+if index % 4 == 0:
+f.write(" " * 11)
+f.write(" " + str(name) + ",")
+if index % 4 == 3:
+f.write("\n")
+f.write("},\n")
+if "cpuid" in data:
+cpuid = data["cpuid"]
+f.write(".cpuid = {\n")
+f.write(".eax = {},\n".format(cpuid["eax"]))
+if "ecx" in cpuid:
+f.write(".needs_ecx = true,\n")
+f.write(".ecx = {},\n".format(cpuid["ecx"]))
+f.write(".reg = {},\n".format(cpuid["reg"]))
+f.write("},\n")
+if "msr" in data:
+f.write(".msr = {\n")
+f.write(".index = {},\n".format(data["msr"]))
+f.write("},\n")
+if "tcg_features" in data:
+f.write(".tcg_features = {},\n".format(data["tcg_features"]))
+if "unmigratable_flags" in data:
+f.write(".unmigratable_flags = {},\n".format(
+data["unmigratable_flags"]))
+if "migratable_flags" in data:
+f.write(".migratable_flags = {},\n".format(
+data["migratable_flags"]))
+if "no_autoenable_flags" in data:
+f.write(".no_autoenable_flags = {},\n".format(
+data["no_autoenable_flags"]))
+f.write("},\n")
+
+
+def main():
+dirname = os.path.dirname(__file__)
+
+with open(os.path.join(dirname, "feature_word_info.yaml"), "tr") as f:
+feature_words = yaml.safe_load(f)
+
+with open(os.path.join(dirname, "feature_word_info.c.inc"), "tw") as f:
+f.write("/* This file is generated by feature_word_info.py. */\n\n")
+f.write("FeatureWordInfo feature_word_info[FEATURE_WORDS] = {\n")
+for feature_word in feature_words:
+write_feature_word(f, feature_word)
+f.write("};\n")
+
+
+if __name__ == "__main__":
+main()
diff --git a/target/i386/feature_word_info.yaml 
b/target/i386/feature_word_info.yaml
index cd6cdc8053..79914a0ece 100644
--- a/target/i386/feature_word_info.yaml
+++ b/target/i386/feature_word_info.yaml
@@ -1,3 +1,5 @@
+# Run `feature_word_info.py` when you make changes to this file.
+
 - index: FEAT_1_EDX
   cpuid:
 eax: '1'
-- 
2.39.2




[PATCH v2 08/10] target/i386: Format feature_word_info.c.inc: Unfold cpuid member

2023-09-08 Thread Tim Wiederhake
Having a consistent formatting minimizes the diff to the generated
code.

Signed-off-by: Tim Wiederhake 
---
 target/i386/feature_word_info.c.inc | 101 +---
 1 file changed, 75 insertions(+), 26 deletions(-)

diff --git a/target/i386/feature_word_info.c.inc 
b/target/i386/feature_word_info.c.inc
index 1f28c3f66e..c154a2b0cf 100644
--- a/target/i386/feature_word_info.c.inc
+++ b/target/i386/feature_word_info.c.inc
@@ -19,7 +19,10 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 NULL, NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
 },
-.cpuid = {.eax = 1, .reg = R_EDX, },
+.cpuid = {
+.eax = 1,
+.reg = R_EDX,
+},
 .tcg_features = TCG_FEATURES,
 },
 [FEAT_1_ECX] = {
@@ -42,7 +45,10 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 NULL, NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
 },
-.cpuid = { .eax = 1, .reg = R_ECX, },
+.cpuid = {
+.eax = 1,
+.reg = R_ECX,
+},
 .tcg_features = TCG_EXT_FEATURES,
 },
 [FEAT_8000_0001_EDX] = {
@@ -65,7 +71,10 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 NULL, NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
 },
-.cpuid = { .eax = 0x8001, .reg = R_EDX, },
+.cpuid = {
+.eax = 0x8001,
+.reg = R_EDX,
+},
 .tcg_features = TCG_EXT2_FEATURES,
 },
 [FEAT_8000_0001_ECX] = {
@@ -88,7 +97,10 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 NULL, NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
 },
-.cpuid = { .eax = 0x8001, .reg = R_ECX, },
+.cpuid = {
+.eax = 0x8001,
+.reg = R_ECX,
+},
 .tcg_features = TCG_EXT3_FEATURES,
 .no_autoenable_flags = CPUID_EXT3_TOPOEXT,
 },
@@ -112,7 +124,10 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 NULL, NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
 },
-.cpuid = { .eax = 0xC001, .reg = R_EDX, },
+.cpuid = {
+.eax = 0xC001,
+.reg = R_EDX,
+},
 .tcg_features = TCG_EXT4_FEATURES,
 },
 [FEAT_KVM] = {
@@ -135,7 +150,10 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 NULL, NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
 },
-.cpuid = { .eax = KVM_CPUID_FEATURES, .reg = R_EAX, },
+.cpuid = {
+.eax = KVM_CPUID_FEATURES,
+.reg = R_EAX,
+},
 .tcg_features = TCG_KVM_FEATURES,
 },
 [FEAT_KVM_HINTS] = {
@@ -158,7 +176,10 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 NULL, NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
 },
-.cpuid = { .eax = KVM_CPUID_FEATURES, .reg = R_EDX, },
+.cpuid = {
+.eax = KVM_CPUID_FEATURES,
+.reg = R_EDX,
+},
 .tcg_features = TCG_KVM_FEATURES,
 .no_autoenable_flags = ~0U,
 },
@@ -182,7 +203,10 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 NULL, NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
 },
-.cpuid = { .eax = 0x800A, .reg = R_EDX, },
+.cpuid = {
+.eax = 0x800A,
+.reg = R_EDX,
+},
 .tcg_features = TCG_SVM_FEATURES,
 },
 [FEAT_7_0_EBX] = {
@@ -207,7 +231,8 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 },
 .cpuid = {
 .eax = 7,
-.needs_ecx = true, .ecx = 0,
+.needs_ecx = true,
+.ecx = 0,
 .reg = R_EBX,
 },
 .tcg_features = TCG_7_0_EBX_FEATURES,
@@ -234,7 +259,8 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 },
 .cpuid = {
 .eax = 7,
-.needs_ecx = true, .ecx = 0,
+.needs_ecx = true,
+.ecx = 0,
 .reg = R_ECX,
 },
 .tcg_features = TCG_7_0_ECX_FEATURES,
@@ -261,7 +287,8 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 },
 .cpuid = {
 .eax = 7,
-.needs_ecx = true, .ecx = 0,
+.needs_ecx = true,
+.ecx = 0,
 .reg = R_EDX,
 },
 .tcg_features = TCG_7_0_EDX_FEATURES,
@@ -288,7 +315,8 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 },
 .cpuid = {
 .eax = 7,
-.needs_ecx = true, .ecx = 1,
+.needs_ecx = true,
+.ecx = 1,
 .reg = R_EAX,
 },
 .tcg_features = TCG_7_1_EAX_FEATURES,
@@ -315,7 +343,8 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 },
 .cpuid = {
 .eax = 7,
-.needs_ecx = true, .ecx = 1,
+.needs_ecx = true,
+.ecx

[PATCH v2 06/10] target/i386: Format feature_word_info.c.inc: Remove comments

2023-09-08 Thread Tim Wiederhake
The comments are preserved in the yaml file.

Signed-off-by: Tim Wiederhake 
---
 target/i386/feature_word_info.c.inc | 56 -
 1 file changed, 15 insertions(+), 41 deletions(-)

diff --git a/target/i386/feature_word_info.c.inc 
b/target/i386/feature_word_info.c.inc
index d4462d7b1f..1d7eae9130 100644
--- a/target/i386/feature_word_info.c.inc
+++ b/target/i386/feature_word_info.c.inc
@@ -6,10 +6,10 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 "tsc", "msr", "pae", "mce",
 "cx8", "apic", NULL, "sep",
 "mtrr", "pge", "mca", "cmov",
-"pat", "pse36", "pn" /* Intel psn */, "clflush" /* Intel clfsh */,
-NULL, "ds" /* Intel dts */, "acpi", "mmx",
+"pat", "pse36", "pn", "clflush",
+NULL, "ds", "acpi", "mmx",
 "fxsr", "sse", "sse2", "ss",
-"ht" /* Intel htt */, "tm", "ia64", "pbe",
+"ht", "tm", "ia64", "pbe",
 },
 .cpuid = {.eax = 1, .reg = R_EDX, },
 .tcg_features = TCG_FEATURES,
@@ -17,33 +17,28 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 [FEAT_1_ECX] = {
 .type = CPUID_FEATURE_WORD,
 .feat_names = {
-"pni" /* Intel,AMD sse3 */, "pclmulqdq", "dtes64", "monitor",
+"pni", "pclmulqdq", "dtes64", "monitor",
 "ds-cpl", "vmx", "smx", "est",
 "tm2", "ssse3", "cid", NULL,
 "fma", "cx16", "xtpr", "pdcm",
 NULL, "pcid", "dca", "sse4.1",
 "sse4.2", "x2apic", "movbe", "popcnt",
-"tsc-deadline", "aes", "xsave", NULL /* osxsave */,
+"tsc-deadline", "aes", "xsave", NULL,
 "avx", "f16c", "rdrand", "hypervisor",
 },
 .cpuid = { .eax = 1, .reg = R_ECX, },
 .tcg_features = TCG_EXT_FEATURES,
 },
-/* Feature names that are already defined on feature_name[] but
- * are set on CPUID[8000_0001].EDX on AMD CPUs don't have their
- * names on feat_names below. They are copied automatically
- * to features[FEAT_8000_0001_EDX] if and only if CPU vendor is AMD.
- */
 [FEAT_8000_0001_EDX] = {
 .type = CPUID_FEATURE_WORD,
 .feat_names = {
-NULL /* fpu */, NULL /* vme */, NULL /* de */, NULL /* pse */,
-NULL /* tsc */, NULL /* msr */, NULL /* pae */, NULL /* mce */,
-NULL /* cx8 */, NULL /* apic */, NULL, "syscall",
-NULL /* mtrr */, NULL /* pge */, NULL /* mca */, NULL /* cmov */,
-NULL /* pat */, NULL /* pse36 */, NULL, NULL /* Linux mp */,
-"nx", NULL, "mmxext", NULL /* mmx */,
-NULL /* fxsr */, "fxsr-opt", "pdpe1gb", "rdtscp",
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, "syscall",
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+"nx", NULL, "mmxext", NULL,
+NULL, "fxsr-opt", "pdpe1gb", "rdtscp",
 NULL, "lm", "3dnowext", "3dnow",
 },
 .cpuid = { .eax = 0x8001, .reg = R_EDX, },
@@ -63,11 +58,6 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 },
 .cpuid = { .eax = 0x8001, .reg = R_ECX, },
 .tcg_features = TCG_EXT3_FEATURES,
-/*
- * TOPOEXT is always allowed but can't be enabled blindly by
- * "-cpu host", as it requires consistent cache topology info
- * to be provided so it doesn't confuse guests.
- */
 .no_autoenable_flags = CPUID_EXT3_TOPOEXT,
 },
 [FEAT_C000_0001_EDX] = {
@@ -114,10 +104,6 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 },
 .cpuid = { .eax = KVM_CPUID_FEATURES, .reg = R_EDX, },
 .tcg_features = TCG_KVM_FEATURES,
-/*
- * KVM hints aren't auto-enabled by -cpu host, they need to be
- * explicitly enabled in the command-line.
- */
 .no_autoenable_flags = ~0U,
 },
 [FEAT_SVM] = {
@@ -158,7 +144,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 .type 

[PATCH v2 07/10] target/i386: Format feature_word_info.c.inc: Fill out feat_names

2023-09-08 Thread Tim Wiederhake
This member is an array of length 64. Initializing all entries in this
array allows for some simplification in the generating code.

Signed-off-by: Tim Wiederhake 
---
 target/i386/feature_word_info.c.inc | 355 +++-
 1 file changed, 351 insertions(+), 4 deletions(-)

diff --git a/target/i386/feature_word_info.c.inc 
b/target/i386/feature_word_info.c.inc
index 1d7eae9130..1f28c3f66e 100644
--- a/target/i386/feature_word_info.c.inc
+++ b/target/i386/feature_word_info.c.inc
@@ -10,6 +10,14 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 NULL, "ds", "acpi", "mmx",
 "fxsr", "sse", "sse2", "ss",
 "ht", "tm", "ia64", "pbe",
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
 },
 .cpuid = {.eax = 1, .reg = R_EDX, },
 .tcg_features = TCG_FEATURES,
@@ -25,6 +33,14 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 "sse4.2", "x2apic", "movbe", "popcnt",
 "tsc-deadline", "aes", "xsave", NULL,
 "avx", "f16c", "rdrand", "hypervisor",
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
 },
 .cpuid = { .eax = 1, .reg = R_ECX, },
 .tcg_features = TCG_EXT_FEATURES,
@@ -40,6 +56,14 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 "nx", NULL, "mmxext", NULL,
 NULL, "fxsr-opt", "pdpe1gb", "rdtscp",
 NULL, "lm", "3dnowext", "3dnow",
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
 },
 .cpuid = { .eax = 0x8001, .reg = R_EDX, },
 .tcg_features = TCG_EXT2_FEATURES,
@@ -55,6 +79,14 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 NULL, "tbm", "topoext", "perfctr-core",
 "perfctr-nb", NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
 },
 .cpuid = { .eax = 0x8001, .reg = R_ECX, },
 .tcg_features = TCG_EXT3_FEATURES,
@@ -71,6 +103,14 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 NULL, NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
 },
 .cpuid = { .eax = 0xC001, .reg = R_EDX, },
 .tcg_features = TCG_EXT4_FEATURES,
@@ -86,6 +126,14 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 NULL, NULL, NULL, NULL,
 "kvmclock-stable-bit", NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
 },
 .cpuid = { .eax = KVM_CPUID_FEATURES, .reg = R_EAX, },
 .tcg_features = TCG_KVM_FEATURES,
@@ -101,6 +149,14 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 NULL, NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
  

[PATCH v2 09/10] target/i386: Format feature_word_info.c.inc: Whitespaces and trailing commas

2023-09-08 Thread Tim Wiederhake
Having a consistent formatting minimizes the diff to the generated
code.

Signed-off-by: Tim Wiederhake 
---
 target/i386/feature_word_info.c.inc | 44 +
 1 file changed, 13 insertions(+), 31 deletions(-)

diff --git a/target/i386/feature_word_info.c.inc 
b/target/i386/feature_word_info.c.inc
index c154a2b0cf..5bac4aaeba 100644
--- a/target/i386/feature_word_info.c.inc
+++ b/target/i386/feature_word_info.c.inc
@@ -187,7 +187,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 .type = CPUID_FEATURE_WORD,
 .feat_names = {
 "npt", "lbrv", "svm-lock", "nrip-save",
-"tsc-scale", "vmcb-clean",  "flushbyasid", "decodeassists",
+"tsc-scale", "vmcb-clean", "flushbyasid", "decodeassists",
 NULL, NULL, "pause-filter", NULL,
 "pfthreshold", "avic", NULL, "v-vmsave-vmload",
 "vgif", NULL, NULL, NULL,
@@ -593,10 +593,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 .reg = R_EAX,
 },
 .tcg_features = ~0U,
-.migratable_flags = XSTATE_FP_MASK | XSTATE_SSE_MASK |
-XSTATE_YMM_MASK | XSTATE_BNDREGS_MASK | XSTATE_BNDCSR_MASK |
-XSTATE_OPMASK_MASK | XSTATE_ZMM_Hi256_MASK | XSTATE_Hi16_ZMM_MASK |
-XSTATE_PKRU_MASK,
+.migratable_flags = XSTATE_FP_MASK | XSTATE_SSE_MASK | XSTATE_YMM_MASK 
| XSTATE_BNDREGS_MASK | XSTATE_BNDCSR_MASK | XSTATE_OPMASK_MASK | 
XSTATE_ZMM_Hi256_MASK | XSTATE_Hi16_ZMM_MASK | XSTATE_PKRU_MASK,
 },
 [FEAT_XSAVE_XCR0_HI] = {
 .type = CPUID_FEATURE_WORD,
@@ -699,7 +696,6 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 .index = MSR_IA32_PERF_CAPABILITIES,
 },
 },
-
 [FEAT_VMX_PROCBASED_CTLS] = {
 .type = MSR_FEATURE_WORD,
 .feat_names = {
@@ -722,9 +718,8 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 },
 .msr = {
 .index = MSR_IA32_VMX_TRUE_PROCBASED_CTLS,
-}
+},
 },
-
 [FEAT_VMX_SECONDARY_CTLS] = {
 .type = MSR_FEATURE_WORD,
 .feat_names = {
@@ -747,9 +742,8 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 },
 .msr = {
 .index = MSR_IA32_VMX_PROCBASED_CTLS2,
-}
+},
 },
-
 [FEAT_VMX_PINBASED_CTLS] = {
 .type = MSR_FEATURE_WORD,
 .feat_names = {
@@ -772,9 +766,8 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 },
 .msr = {
 .index = MSR_IA32_VMX_TRUE_PINBASED_CTLS,
-}
+},
 },
-
 [FEAT_VMX_EXIT_CTLS] = {
 .type = MSR_FEATURE_WORD,
 .feat_names = {
@@ -783,8 +776,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 NULL, NULL, NULL, NULL,
 "vmx-exit-load-perf-global-ctrl", NULL, NULL, "vmx-exit-ack-intr",
 NULL, NULL, "vmx-exit-save-pat", "vmx-exit-load-pat",
-"vmx-exit-save-efer", "vmx-exit-load-efer",
-"vmx-exit-save-preemption-timer", "vmx-exit-clear-bndcfgs",
+"vmx-exit-save-efer", "vmx-exit-load-efer", 
"vmx-exit-save-preemption-timer", "vmx-exit-clear-bndcfgs",
 NULL, "vmx-exit-clear-rtit-ctl", NULL, NULL,
 NULL, "vmx-exit-load-pkrs", NULL, NULL,
 NULL, NULL, NULL, NULL,
@@ -798,9 +790,8 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 },
 .msr = {
 .index = MSR_IA32_VMX_TRUE_EXIT_CTLS,
-}
+},
 },
-
 [FEAT_VMX_ENTRY_CTLS] = {
 .type = MSR_FEATURE_WORD,
 .feat_names = {
@@ -823,9 +814,8 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 },
 .msr = {
 .index = MSR_IA32_VMX_TRUE_ENTRY_CTLS,
-}
+},
 },
-
 [FEAT_VMX_MISC] = {
 .type = MSR_FEATURE_WORD,
 .feat_names = {
@@ -848,9 +838,8 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 },
 .msr = {
 .index = MSR_IA32_VMX_MISC,
-}
+},
 },
-
 [FEAT_VMX_EPT_VPID_CAPS] = {
 .type = MSR_FEATURE_WORD,
 .feat_names = {
@@ -864,8 +853,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 NULL, NULL, NULL, NULL,
 "vmx-invvpid", NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
-"vmx-invvpid-single-addr", "vmx-invvpid-single-context",
-"vmx-invvpid-all-context", 
"vmx-invvpid-single-context-noglobals",
+"vmx-invvpid-single-addr", "vmx-invvpid-single-context", 
"vmx-invvpid-all-c

[PATCH v2 04/10] target/i386: Split out feature_word_info

2023-09-08 Thread Tim Wiederhake
The isolated part will be generated by a script.

Signed-off-by: Tim Wiederhake 
---
 target/i386/cpu.c   | 677 +---
 target/i386/feature_word_info.c.inc | 676 +++
 2 files changed, 677 insertions(+), 676 deletions(-)
 create mode 100644 target/i386/feature_word_info.c.inc

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index f0fedf4b88..23ed2fb09c 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -762,682 +762,7 @@ void x86_cpu_vendor_words2str(char *dst, uint32_t vendor1,
 #define TCG_8000_0008_EBX  (CPUID_8000_0008_EBX_XSAVEERPTR | \
   CPUID_8000_0008_EBX_WBNOINVD | CPUID_8000_0008_EBX_KERNEL_FEATURES)
 
-FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
-[FEAT_1_EDX] = {
-.type = CPUID_FEATURE_WORD,
-.feat_names = {
-"fpu", "vme", "de", "pse",
-"tsc", "msr", "pae", "mce",
-"cx8", "apic", NULL, "sep",
-"mtrr", "pge", "mca", "cmov",
-"pat", "pse36", "pn" /* Intel psn */, "clflush" /* Intel clfsh */,
-NULL, "ds" /* Intel dts */, "acpi", "mmx",
-"fxsr", "sse", "sse2", "ss",
-"ht" /* Intel htt */, "tm", "ia64", "pbe",
-},
-.cpuid = {.eax = 1, .reg = R_EDX, },
-.tcg_features = TCG_FEATURES,
-},
-[FEAT_1_ECX] = {
-.type = CPUID_FEATURE_WORD,
-.feat_names = {
-"pni" /* Intel,AMD sse3 */, "pclmulqdq", "dtes64", "monitor",
-"ds-cpl", "vmx", "smx", "est",
-"tm2", "ssse3", "cid", NULL,
-"fma", "cx16", "xtpr", "pdcm",
-NULL, "pcid", "dca", "sse4.1",
-"sse4.2", "x2apic", "movbe", "popcnt",
-"tsc-deadline", "aes", "xsave", NULL /* osxsave */,
-"avx", "f16c", "rdrand", "hypervisor",
-},
-.cpuid = { .eax = 1, .reg = R_ECX, },
-.tcg_features = TCG_EXT_FEATURES,
-},
-/* Feature names that are already defined on feature_name[] but
- * are set on CPUID[8000_0001].EDX on AMD CPUs don't have their
- * names on feat_names below. They are copied automatically
- * to features[FEAT_8000_0001_EDX] if and only if CPU vendor is AMD.
- */
-[FEAT_8000_0001_EDX] = {
-.type = CPUID_FEATURE_WORD,
-.feat_names = {
-NULL /* fpu */, NULL /* vme */, NULL /* de */, NULL /* pse */,
-NULL /* tsc */, NULL /* msr */, NULL /* pae */, NULL /* mce */,
-NULL /* cx8 */, NULL /* apic */, NULL, "syscall",
-NULL /* mtrr */, NULL /* pge */, NULL /* mca */, NULL /* cmov */,
-NULL /* pat */, NULL /* pse36 */, NULL, NULL /* Linux mp */,
-"nx", NULL, "mmxext", NULL /* mmx */,
-NULL /* fxsr */, "fxsr-opt", "pdpe1gb", "rdtscp",
-NULL, "lm", "3dnowext", "3dnow",
-},
-.cpuid = { .eax = 0x8001, .reg = R_EDX, },
-.tcg_features = TCG_EXT2_FEATURES,
-},
-[FEAT_8000_0001_ECX] = {
-.type = CPUID_FEATURE_WORD,
-.feat_names = {
-"lahf-lm", "cmp-legacy", "svm", "extapic",
-"cr8legacy", "abm", "sse4a", "misalignsse",
-"3dnowprefetch", "osvw", "ibs", "xop",
-"skinit", "wdt", NULL, "lwp",
-"fma4", "tce", NULL, "nodeid-msr",
-NULL, "tbm", "topoext", "perfctr-core",
-"perfctr-nb", NULL, NULL, NULL,
-NULL, NULL, NULL, NULL,
-},
-.cpuid = { .eax = 0x8001, .reg = R_ECX, },
-.tcg_features = TCG_EXT3_FEATURES,
-/*
- * TOPOEXT is always allowed but can't be enabled blindly by
- * "-cpu host", as it requires consistent cache topology info
- * to be provided so it doesn't confuse guests.
- */
-.no_autoenable_flags = CPUID_EXT3_TOPOEXT,
-},
-[FEAT_C000_0001_EDX] = {
-.type = CPUID_FEATURE_WORD,
-.feat_names = {
-NULL, NULL, "xstore", "xstore-en",
-NULL, NULL, "xcrypt", "xcrypt-en&

[PATCH v2 00/10] Generate x86 cpu features

2023-09-08 Thread Tim Wiederhake
Synchronizing the list of cpu features and models with qemu is a recurring
task in libvirt. For x86, this is done by reading qom-list-properties for
max-x86_64-cpu and manually filtering out everthing that does not look like
a feature name, as well as parsing target/i386/cpu.c for cpu models.

This is a flawed, tedious and error-prone procedure. Ideally, qemu
and libvirt would query a common source for cpu feature and model
related information. Meanwhile, converting this information into an easier
to parse format would help libvirt a lot.

This patch series converts the cpu feature information present in
target/i386/cpu.c (`feature_word_info`) into a yaml file and adds a
script to generate the c code from this data.

A patch set to convert the cpu model data (`builtin_x86_defs`) in the
same way will follow.

v1: https://lists.nongnu.org/archive/html/qemu-devel/2023-08/msg02005.html

Changes since v1:
* Incorporated changes from
  https://lists.nongnu.org/archive/html/qemu-devel/2023-08/msg04241.html.
* Changed data format from xml to yaml, as proposed in
  https://lists.nongnu.org/archive/html/qemu-devel/2023-09/msg01033.html.
  Using json has some drawbacks, see
  https://lists.nongnu.org/archive/html/qemu-devel/2023-08/msg03384.html.
* Rebased on top of current master. Features added in the meantime:
  amx-complex and vmx-enable-user-wait-pause
* Split up the reformatting of feature_word_info.c.inc to make it easier
  to review. These patches could be squashed together before merging.

Tim Wiederhake (10):
  target/i386: Add missing feature names in FEAT_VMX_EPT_VPID_CAPS
  target/i386: Fix feature names in FEAT_VMX_EPT_VPID_CAPS
  target/i386: Fix duplicated feature name in FEAT_KVM
  target/i386: Split out feature_word_info
  target/i386: Translate feature_word_info to yaml
  target/i386: Format feature_word_info.c.inc: Remove comments
  target/i386: Format feature_word_info.c.inc: feat_names
  target/i386: Format feature_word_info.c.inc: Unfold cpuid member
  target/i386: Format feature_word_info.c.inc: Whitespaces and trailing
commas
  target/i386: Autogenerate feature_word_info.c.inc

 target/i386/cpu.c   |  677 +-
 target/i386/feature_word_info.c.inc | 1030 +++
 target/i386/feature_word_info.py|   62 ++
 target/i386/feature_word_info.yaml  |  697 ++
 4 files changed, 1790 insertions(+), 676 deletions(-)
 create mode 100644 target/i386/feature_word_info.c.inc
 create mode 100755 target/i386/feature_word_info.py
 create mode 100644 target/i386/feature_word_info.yaml

-- 
2.39.2




[PATCH v2 02/10] target/i386: Fix feature names in FEAT_VMX_EPT_VPID_CAPS

2023-09-08 Thread Tim Wiederhake
Fix a copy-paste-mistake in the FEAT_VMX_EPT_VIPD_CAPS cpuid leaf.
The mistake became apparent as there were two features with the same name
in this cpuid leaf. The names are now in line with SDM volume 3, appendix A,
section 10.

Fixes: 20a78b02d3 ("target/i386: add VMX features")
Signed-off-by: Tim Wiederhake 
---
 target/i386/cpu.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 7c2c48ac06..f10d343935 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1322,8 +1322,8 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 NULL, NULL, NULL, NULL,
 "vmx-invvpid", NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
-"vmx-invvpid-single-addr", "vmx-invept-single-context",
-"vmx-invvpid-all-context", 
"vmx-invept-single-context-noglobals",
+"vmx-invvpid-single-addr", "vmx-invvpid-single-context",
+"vmx-invvpid-all-context", 
"vmx-invvpid-single-context-noglobals",
 NULL, NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
-- 
2.39.2




[PATCH v2 03/10] target/i386: Fix duplicated feature name in FEAT_KVM

2023-09-08 Thread Tim Wiederhake
The mistake became apparent as there were two features with the same name
in this cpuid leaf. The names are now in line with the documentation from
https://kernel.org/doc/html/latest/virt/kvm/x86/cpuid.html

Fixes: 642258c6c7 ("kvm: add kvmclock to its second bit")
Signed-off-by: Tim Wiederhake 
---
 target/i386/cpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index f10d343935..f0fedf4b88 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -852,7 +852,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 [FEAT_KVM] = {
 .type = CPUID_FEATURE_WORD,
 .feat_names = {
-"kvmclock", "kvm-nopiodelay", "kvm-mmu", "kvmclock",
+"kvmclock", "kvm-nopiodelay", "kvm-mmu", "kvmclock2",
 "kvm-asyncpf", "kvm-steal-time", "kvm-pv-eoi", "kvm-pv-unhalt",
 NULL, "kvm-pv-tlb-flush", NULL, "kvm-pv-ipi",
 "kvm-poll-control", "kvm-pv-sched-yield", "kvm-asyncpf-int", 
"kvm-msi-ext-dest-id",
-- 
2.39.2




[PATCH v2 01/10] target/i386: Add missing feature names in FEAT_VMX_EPT_VPID_CAPS

2023-09-08 Thread Tim Wiederhake
Add the missing feature names for two bits in the FEAT_VMX_EPT_VPID_CAPS
cpuid leaf. "vmx-ept-uc" is currently unused, but "vmx-ept-wb" is enabled
for multiple cpu models.

Signed-off-by: Tim Wiederhake 
---
 target/i386/cpu.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 24ee67b42d..7c2c48ac06 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1314,8 +1314,8 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 .feat_names = {
 "vmx-ept-execonly", NULL, NULL, NULL,
 NULL, NULL, "vmx-page-walk-4", "vmx-page-walk-5",
-NULL, NULL, NULL, NULL,
-NULL, NULL, NULL, NULL,
+"vmx-ept-uc", NULL, NULL, NULL,
+NULL, NULL, "vmx-ept-wb", NULL,
 "vmx-ept-2mb", "vmx-ept-1gb", NULL, NULL,
 "vmx-invept", "vmx-eptad", "vmx-ept-advanced-exitinfo", NULL,
 NULL, "vmx-invept-single-context", "vmx-invept-all-context", NULL,
-- 
2.39.2




Re: [PATCH 3/3] target/i386: Fix duplicated feature name in FEAT_KVM

2023-08-25 Thread Tim Wiederhake
On Thu, 2023-08-24 at 17:12 +0200, Philippe Mathieu-Daudé wrote:
> On 24/8/23 15:57, Tim Wiederhake wrote:
> > The mistake became apparent as there were two features with the
> > same name
> > in this cpuid leaf. The names are now in line with the
> > documentation from
> > https://kernel.org/doc/html/latest/virt/kvm/x86/cpuid.html
> > 
> 
> Fixes: 642258c6c7 ("kvm: add kvmclock to its second bit")
> ?
> 
Right, added that locally. Thanks!

> > Signed-off-by: Tim Wiederhake 
> > ---
> >   target/i386/cpu.c | 2 +-
> >   1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> > index 0b74d80371..ceb291f8a8 100644
> > --- a/target/i386/cpu.c
> > +++ b/target/i386/cpu.c
> > @@ -852,7 +852,7 @@ FeatureWordInfo
> > feature_word_info[FEATURE_WORDS] = {
> >   [FEAT_KVM] = {
> >   .type = CPUID_FEATURE_WORD,
> >   .feat_names = {
> > -    "kvmclock", "kvm-nopiodelay", "kvm-mmu", "kvmclock",
> > +    "kvmclock", "kvm-nopiodelay", "kvm-mmu", "kvmclock2",
> >   "kvm-asyncpf", "kvm-steal-time", "kvm-pv-eoi", "kvm-
> > pv-unhalt",
> >   NULL, "kvm-pv-tlb-flush", NULL, "kvm-pv-ipi",
> >   "kvm-poll-control", "kvm-pv-sched-yield", "kvm-
> > asyncpf-int", "kvm-msi-ext-dest-id",
> 




[PATCH 2/3] target/i386: Fix feature names in FEAT_VMX_EPT_VPID_CAPS

2023-08-24 Thread Tim Wiederhake
Fix a copy-paste-mistake in the FEAT_VMX_EPT_VIPD_CAPS cpuid leaf.
The mistake became apparent as there were two features with the same name
in this cpuid leaf. The names are now in line with SDM volume 3, appendix A,
section 10.

Fixes: 20a78b02d3 ("target/i386: add VMX features")
Signed-off-by: Tim Wiederhake 
---
 target/i386/cpu.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index e6b8c62b92..0b74d80371 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1322,8 +1322,8 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 NULL, NULL, NULL, NULL,
 "vmx-invvpid", NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
-"vmx-invvpid-single-addr", "vmx-invept-single-context",
-"vmx-invvpid-all-context", 
"vmx-invept-single-context-noglobals",
+"vmx-invvpid-single-addr", "vmx-invvpid-single-context",
+"vmx-invvpid-all-context", 
"vmx-invvpid-single-context-noglobals",
 NULL, NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
-- 
2.39.2




[PATCH 1/3] target/i386: Add missing feature names in FEAT_VMX_EPT_VPID_CAPS

2023-08-24 Thread Tim Wiederhake
Add the missing feature names for two bits in the FEAT_VMX_EPT_VPID_CAPS
cpuid leaf. "vmx-ept-uc" is currently unused, but "vmx-ept-wb" is enabled
for multiple cpu models.

Signed-off-by: Tim Wiederhake 
---
 target/i386/cpu.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 97ad229d8b..e6b8c62b92 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1314,8 +1314,8 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 .feat_names = {
 "vmx-ept-execonly", NULL, NULL, NULL,
 NULL, NULL, "vmx-page-walk-4", "vmx-page-walk-5",
-NULL, NULL, NULL, NULL,
-NULL, NULL, NULL, NULL,
+"vmx-ept-uc", NULL, NULL, NULL,
+NULL, NULL, "vmx-ept-wb", NULL,
 "vmx-ept-2mb", "vmx-ept-1gb", NULL, NULL,
 "vmx-invept", "vmx-eptad", "vmx-ept-advanced-exitinfo", NULL,
 NULL, "vmx-invept-single-context", "vmx-invept-all-context", NULL,
-- 
2.39.2




[PATCH 0/3] Fix some feature names for i386

2023-08-24 Thread Tim Wiederhake
Some feature names were missing, wrong, or duplicated in the
feature_word_info table. See individual commits for details.

This introduces some merge conflicts for
https://lists.nongnu.org/archive/html/qemu-devel/2023-08/msg02005.html
but that series might need to be reworked anyway.

Regards,
Tim

Tim Wiederhake (3):
  target/i386: Add missing feature names in FEAT_VMX_EPT_VPID_CAPS
  target/i386: Fix feature names in FEAT_VMX_EPT_VPID_CAPS
  target/i386: Fix duplicated feature name in FEAT_KVM

 target/i386/cpu.c | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

-- 
2.39.2




[PATCH 3/3] target/i386: Fix duplicated feature name in FEAT_KVM

2023-08-24 Thread Tim Wiederhake
The mistake became apparent as there were two features with the same name
in this cpuid leaf. The names are now in line with the documentation from
https://kernel.org/doc/html/latest/virt/kvm/x86/cpuid.html

Signed-off-by: Tim Wiederhake 
---
 target/i386/cpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 0b74d80371..ceb291f8a8 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -852,7 +852,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 [FEAT_KVM] = {
 .type = CPUID_FEATURE_WORD,
 .feat_names = {
-"kvmclock", "kvm-nopiodelay", "kvm-mmu", "kvmclock",
+"kvmclock", "kvm-nopiodelay", "kvm-mmu", "kvmclock2",
 "kvm-asyncpf", "kvm-steal-time", "kvm-pv-eoi", "kvm-pv-unhalt",
 NULL, "kvm-pv-tlb-flush", NULL, "kvm-pv-ipi",
 "kvm-poll-control", "kvm-pv-sched-yield", "kvm-asyncpf-int", 
"kvm-msi-ext-dest-id",
-- 
2.39.2




Re: [PATCH 2/4] target/i386: Translate feature_word_info to xml

2023-08-21 Thread Tim Wiederhake
On Thu, 2023-08-17 at 12:07 +0100, Daniel P. Berrangé wrote:
> On Fri, Aug 11, 2023 at 03:50:09PM +0200, Tim Wiederhake wrote:
> > This is the data file that will be used to generate the C code.
> > All information, including the comments, is preserved.
> > 
> > Signed-off-by: Tim Wiederhake 
> > ---
> >  target/i386/feature_word_info.xml | 1607
> > +
> >  1 file changed, 1607 insertions(+)
> >  create mode 100644 target/i386/feature_word_info.xml
> > 
> > diff --git a/target/i386/feature_word_info.xml
> > b/target/i386/feature_word_info.xml
> > new file mode 100644
> > index 00..ff741b9f5a
> > --- /dev/null
> > +++ b/target/i386/feature_word_info.xml
> > @@ -0,0 +1,1607 @@
> > +
> 
> I think adding data formats based on XML is pretty undesirable
> for QEMU. AFAIK, the only place we've used XML is where we
> needed to have interoperability with an external tool.
> 
> Can we not just do this using JSON instead, which would avoid
> the need to write python parsing code as we can directly load
> it into a python object.
> 
> With regards,
> Daniel

I thought of json as well, but that has some drawbacks over xml here:

Integer values, e.g. "eax=0x8008", would need to be stored in
either decimal form or as a string. Both solutions are not desirable in
my opinion.

Additionally, preserving the comments is not as straight forward.
Comments in json are a non-standard extension and not understood by all
parsers, e.g. python's json module. This would require some regex-
preprocessing that comes with its own problems.

Regards,
Tim




[PATCH 2/4] target/i386: Translate feature_word_info to xml

2023-08-11 Thread Tim Wiederhake
This is the data file that will be used to generate the C code.
All information, including the comments, is preserved.

Signed-off-by: Tim Wiederhake 
---
 target/i386/feature_word_info.xml | 1607 +
 1 file changed, 1607 insertions(+)
 create mode 100644 target/i386/feature_word_info.xml

diff --git a/target/i386/feature_word_info.xml 
b/target/i386/feature_word_info.xml
new file mode 100644
index 00..ff741b9f5a
--- /dev/null
+++ b/target/i386/feature_word_info.xml
@@ -0,0 +1,1607 @@
+
+
+CPUID_FEATURE_WORD
+
+"fpu"
+"vme"
+"de"
+"pse"
+"tsc"
+"msr"
+"pae"
+"mce"
+"cx8"
+"apic"
+
+"sep"
+"mtrr"
+"pge"
+"mca"
+"cmov"
+"pat"
+"pse36"
+"pn"
+"clflush"
+
+"ds"
+"acpi"
+"mmx"
+"fxsr"
+"sse"
+"sse2"
+"ss"
+"ht"
+"tm"
+"ia64"
+"pbe"
+
+
+1
+R_EDX
+
+TCG_FEATURES
+
+
+CPUID_FEATURE_WORD
+
+"pni"
+"pclmulqdq"
+"dtes64"
+"monitor"
+"ds-cpl"
+"vmx"
+"smx"
+"est"
+"tm2"
+"ssse3"
+"cid"
+
+"fma"
+"cx16"
+"xtpr"
+"pdcm"
+
+"pcid"
+"dca"
+"sse4.1"
+"sse4.2"
+"x2apic"
+"movbe"
+"popcnt"
+"tsc-deadline"
+"aes"
+"xsave"
+
+"avx"
+"f16c"
+"rdrand"
+"hypervisor"
+
+
+1
+R_ECX
+
+TCG_EXT_FEATURES
+
+
+
+CPUID_FEATURE_WORD
+
+
+
+
+
+
+
+
+
+
+
+
+"syscall"
+
+
+
+
+
+
+
+
+"nx"
+
+"mmxext"
+
+
+"fxsr-opt"
+"pdpe1gb"
+"rdtscp"
+
+"lm"
+"3dnowext"
+"3dnow"
+
+
+0x8001
+R_EDX
+
+TCG_EXT2_FEATURES
+
+
+CPUID_FEATURE_WORD
+
+"lahf-lm"
+"cmp-legacy"
+"svm"
+"extapic"
+"cr8legacy"
+"abm"
+"sse4a"
+"misalignsse"
+"3dnowprefetch"
+"osvw"
+"ibs"
+"xop"
+"skinit"
+"wdt"
+
+"lwp"
+"fma4"
+"tce"
+
+"nodeid-msr"
+
+"tbm"
+"topoext"
+"perfctr-core"
+"perfctr-nb"
+
+
+
+
+
+
+
+
+
+0x8001
+R_ECX
+
+TCG_EXT3_FEATURES
+
+CPUID_EXT3_TOPOEXT
+
+
+CPUID_FEATURE_WORD
+
+
+
+"xstore"
+"xstore-en"
+
+
+"xcrypt"
+"xcrypt-en"
+"ace2"
+"ace2-en"
+"phe"
+"phe-en"
+"pmm"
+ 

[PATCH 4/4] target/i386: Autogenerate feature_word_info.c.inc

2023-08-11 Thread Tim Wiederhake
This introduces no semantic changes to the file.

Signed-off-by: Tim Wiederhake 
---
 target/i386/feature_word_info.c.inc |   2 +
 target/i386/feature_word_info.py| 110 
 target/i386/feature_word_info.xml   |   3 +
 3 files changed, 115 insertions(+)
 create mode 100755 target/i386/feature_word_info.py

diff --git a/target/i386/feature_word_info.c.inc 
b/target/i386/feature_word_info.c.inc
index 040c3c4e56..b8e77ab7e5 100644
--- a/target/i386/feature_word_info.c.inc
+++ b/target/i386/feature_word_info.c.inc
@@ -1,3 +1,5 @@
+/* This file is autogenerated by feature_word_info.py. */
+
 FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 [FEAT_1_EDX] = {
 .type = CPUID_FEATURE_WORD,
diff --git a/target/i386/feature_word_info.py b/target/i386/feature_word_info.py
new file mode 100755
index 00..95f3931aa0
--- /dev/null
+++ b/target/i386/feature_word_info.py
@@ -0,0 +1,110 @@
+#!/bin/env python3
+
+import lxml.etree
+import os
+
+
+class FeatureWord:
+def __init__(self, xml):
+self.index = xml.values()[0]
+for node in xml:
+if node.tag == "cpuid":
+self.cpuid = dict()
+for child in node:
+self.cpuid[child.tag] = child.text
+elif node.tag == "feat_names":
+self.feat_names = [child.text for child in node]
+else:
+setattr(self, node.tag, node.text)
+
+
+def write_feat_names(f, data):
+f.write(".feat_names = {\n")
+for index, name in enumerate(data):
+if name is None:
+name = "NULL"
+if index % 4 == 0:
+f.write(" " * 11)
+f.write(" " + str(name) + ",")
+if index % 4 == 3:
+f.write("\n")
+f.write("},\n")
+
+
+def write_cpuid(f, data):
+f.write(".cpuid = {\n")
+f.write(".eax = {},\n".format(data["eax"]))
+if "ecx" in data:
+f.write(".needs_ecx = true,\n")
+f.write(".ecx = {},\n".format(data["ecx"]))
+f.write(".reg = {},\n".format(data["reg"]))
+f.write("},\n")
+
+
+def write_msr(f, data):
+f.write(".msr = {\n")
+f.write(".index = {},\n".format(data))
+f.write("},\n")
+
+
+def write_tcg_features(f, data):
+f.write(".tcg_features = {},\n".format(data))
+
+
+def write_unmigratable_flags(f, data):
+f.write(".unmigratable_flags = {},\n".format(data))
+
+
+def write_migratable_flags(f, data):
+f.write(".migratable_flags = {},\n".format(data))
+
+
+def write_no_autoenable_flags(f, data):
+f.write(".no_autoenable_flags = {},\n".format(data))
+
+
+def write_feature_word(f, data):
+f.write("[{}] = {{\n".format(data.index))
+f.write(".type = {},\n".format(data.type))
+if hasattr(data, "feat_names"):
+write_feat_names(f, data.feat_names)
+if hasattr(data, "cpuid"):
+write_cpuid(f, data.cpuid)
+if hasattr(data, "msr"):
+write_msr(f, data.msr)
+if hasattr(data, "tcg_features"):
+write_tcg_features(f, data.tcg_features)
+if hasattr(data, "unmigratable_flags"):
+write_unmigratable_flags(f, data.unmigratable_flags)
+if hasattr(data, "migratable_flags"):
+write_migratable_flags(f, data.migratable_flags)
+if hasattr(data, "no_autoenable_flags"):
+write_no_autoenable_flags(f, data.no_autoenable_flags)
+f.write("},\n")
+
+
+def write_feature_words(f, data):
+f.write("/* This file is autogenerated by feature_word_info.py. */\n\n")
+f.write("FeatureWordInfo feature_word_info[FEATURE_WORDS] = {\n")
+for feature_word in data:
+write_feature_word(f, feature_word)
+f.write("};\n")
+
+
+def main():
+dirname = os.path.dirname(__file__)
+ifile = os.path.join(dirname, "feature_word_info.xml")
+ofile = os.path.join(dirname, "feature_word_info.c.inc")
+
+parser = lxml.etree.XMLParser(remove_comments=True, remove_blank_text=True)
+with open(ifile, "tr") as f:
+doc = lxml.etree.parse(f, parser=parser)
+
+feature_words = [FeatureWord(node) for node in doc.getroot()]
+
+with open(ofile, "tw") as f:
+write_feature_words(f, feature_words)
+
+
+if __name__ == "__main__":
+main()
diff --git a/target/i386/feature_word_info.xml 
b/target/i386/feature_word_info.xml
index ff741b9f5a..662b8b1dfc 100644
--- a/target/i386/feature_word_info.xml
+++ b/target/i386/feature_word_info.xml
@@ -1,3 +1,6 @@
+
 
 
 CPUID_FEATURE_WORD
-- 
2.39.2




[PATCH 1/4] target/i386: Split out feature_word_info

2023-08-11 Thread Tim Wiederhake
The isolated part will be generated by a script.

Signed-off-by: Tim Wiederhake 
---
 target/i386/cpu.c   | 677 +---
 target/i386/feature_word_info.c.inc | 676 +++
 2 files changed, 677 insertions(+), 676 deletions(-)
 create mode 100644 target/i386/feature_word_info.c.inc

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 97ad229d8b..d44a5b300e 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -762,682 +762,7 @@ void x86_cpu_vendor_words2str(char *dst, uint32_t vendor1,
 #define TCG_8000_0008_EBX  (CPUID_8000_0008_EBX_XSAVEERPTR | \
   CPUID_8000_0008_EBX_WBNOINVD | CPUID_8000_0008_EBX_KERNEL_FEATURES)
 
-FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
-[FEAT_1_EDX] = {
-.type = CPUID_FEATURE_WORD,
-.feat_names = {
-"fpu", "vme", "de", "pse",
-"tsc", "msr", "pae", "mce",
-"cx8", "apic", NULL, "sep",
-"mtrr", "pge", "mca", "cmov",
-"pat", "pse36", "pn" /* Intel psn */, "clflush" /* Intel clfsh */,
-NULL, "ds" /* Intel dts */, "acpi", "mmx",
-"fxsr", "sse", "sse2", "ss",
-"ht" /* Intel htt */, "tm", "ia64", "pbe",
-},
-.cpuid = {.eax = 1, .reg = R_EDX, },
-.tcg_features = TCG_FEATURES,
-},
-[FEAT_1_ECX] = {
-.type = CPUID_FEATURE_WORD,
-.feat_names = {
-"pni" /* Intel,AMD sse3 */, "pclmulqdq", "dtes64", "monitor",
-"ds-cpl", "vmx", "smx", "est",
-"tm2", "ssse3", "cid", NULL,
-"fma", "cx16", "xtpr", "pdcm",
-NULL, "pcid", "dca", "sse4.1",
-"sse4.2", "x2apic", "movbe", "popcnt",
-"tsc-deadline", "aes", "xsave", NULL /* osxsave */,
-"avx", "f16c", "rdrand", "hypervisor",
-},
-.cpuid = { .eax = 1, .reg = R_ECX, },
-.tcg_features = TCG_EXT_FEATURES,
-},
-/* Feature names that are already defined on feature_name[] but
- * are set on CPUID[8000_0001].EDX on AMD CPUs don't have their
- * names on feat_names below. They are copied automatically
- * to features[FEAT_8000_0001_EDX] if and only if CPU vendor is AMD.
- */
-[FEAT_8000_0001_EDX] = {
-.type = CPUID_FEATURE_WORD,
-.feat_names = {
-NULL /* fpu */, NULL /* vme */, NULL /* de */, NULL /* pse */,
-NULL /* tsc */, NULL /* msr */, NULL /* pae */, NULL /* mce */,
-NULL /* cx8 */, NULL /* apic */, NULL, "syscall",
-NULL /* mtrr */, NULL /* pge */, NULL /* mca */, NULL /* cmov */,
-NULL /* pat */, NULL /* pse36 */, NULL, NULL /* Linux mp */,
-"nx", NULL, "mmxext", NULL /* mmx */,
-NULL /* fxsr */, "fxsr-opt", "pdpe1gb", "rdtscp",
-NULL, "lm", "3dnowext", "3dnow",
-},
-.cpuid = { .eax = 0x8001, .reg = R_EDX, },
-.tcg_features = TCG_EXT2_FEATURES,
-},
-[FEAT_8000_0001_ECX] = {
-.type = CPUID_FEATURE_WORD,
-.feat_names = {
-"lahf-lm", "cmp-legacy", "svm", "extapic",
-"cr8legacy", "abm", "sse4a", "misalignsse",
-"3dnowprefetch", "osvw", "ibs", "xop",
-"skinit", "wdt", NULL, "lwp",
-"fma4", "tce", NULL, "nodeid-msr",
-NULL, "tbm", "topoext", "perfctr-core",
-"perfctr-nb", NULL, NULL, NULL,
-NULL, NULL, NULL, NULL,
-},
-.cpuid = { .eax = 0x8001, .reg = R_ECX, },
-.tcg_features = TCG_EXT3_FEATURES,
-/*
- * TOPOEXT is always allowed but can't be enabled blindly by
- * "-cpu host", as it requires consistent cache topology info
- * to be provided so it doesn't confuse guests.
- */
-.no_autoenable_flags = CPUID_EXT3_TOPOEXT,
-},
-[FEAT_C000_0001_EDX] = {
-.type = CPUID_FEATURE_WORD,
-.feat_names = {
-NULL, NULL, "xstore", "xstore-en",
-NULL, NULL, "xcrypt", "xcrypt-en

[PATCH 3/4] target/i386: Format feature_word_info.c.inc

2023-08-11 Thread Tim Wiederhake
Harmonize the formatting: Use trailing commas, fix indentation and
empty line usage, define "cpuid" fields one per line, unwind index-
assignment in arrays, and remove comments. The information in the
comments is preserved in the xml file.

Signed-off-by: Tim Wiederhake 
---
 target/i386/feature_word_info.c.inc | 230 
 1 file changed, 128 insertions(+), 102 deletions(-)

diff --git a/target/i386/feature_word_info.c.inc 
b/target/i386/feature_word_info.c.inc
index bba7975eab..040c3c4e56 100644
--- a/target/i386/feature_word_info.c.inc
+++ b/target/i386/feature_word_info.c.inc
@@ -6,47 +6,51 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 "tsc", "msr", "pae", "mce",
 "cx8", "apic", NULL, "sep",
 "mtrr", "pge", "mca", "cmov",
-"pat", "pse36", "pn" /* Intel psn */, "clflush" /* Intel clfsh */,
-NULL, "ds" /* Intel dts */, "acpi", "mmx",
+"pat", "pse36", "pn", "clflush",
+NULL, "ds", "acpi", "mmx",
 "fxsr", "sse", "sse2", "ss",
-"ht" /* Intel htt */, "tm", "ia64", "pbe",
+"ht", "tm", "ia64", "pbe",
+},
+.cpuid = {
+.eax = 1,
+.reg = R_EDX,
 },
-.cpuid = {.eax = 1, .reg = R_EDX, },
 .tcg_features = TCG_FEATURES,
 },
 [FEAT_1_ECX] = {
 .type = CPUID_FEATURE_WORD,
 .feat_names = {
-"pni" /* Intel,AMD sse3 */, "pclmulqdq", "dtes64", "monitor",
+"pni", "pclmulqdq", "dtes64", "monitor",
 "ds-cpl", "vmx", "smx", "est",
 "tm2", "ssse3", "cid", NULL,
 "fma", "cx16", "xtpr", "pdcm",
 NULL, "pcid", "dca", "sse4.1",
 "sse4.2", "x2apic", "movbe", "popcnt",
-"tsc-deadline", "aes", "xsave", NULL /* osxsave */,
+"tsc-deadline", "aes", "xsave", NULL,
 "avx", "f16c", "rdrand", "hypervisor",
 },
-.cpuid = { .eax = 1, .reg = R_ECX, },
+.cpuid = {
+.eax = 1,
+.reg = R_ECX,
+},
 .tcg_features = TCG_EXT_FEATURES,
 },
-/* Feature names that are already defined on feature_name[] but
- * are set on CPUID[8000_0001].EDX on AMD CPUs don't have their
- * names on feat_names below. They are copied automatically
- * to features[FEAT_8000_0001_EDX] if and only if CPU vendor is AMD.
- */
 [FEAT_8000_0001_EDX] = {
 .type = CPUID_FEATURE_WORD,
 .feat_names = {
-NULL /* fpu */, NULL /* vme */, NULL /* de */, NULL /* pse */,
-NULL /* tsc */, NULL /* msr */, NULL /* pae */, NULL /* mce */,
-NULL /* cx8 */, NULL /* apic */, NULL, "syscall",
-NULL /* mtrr */, NULL /* pge */, NULL /* mca */, NULL /* cmov */,
-NULL /* pat */, NULL /* pse36 */, NULL, NULL /* Linux mp */,
-"nx", NULL, "mmxext", NULL /* mmx */,
-NULL /* fxsr */, "fxsr-opt", "pdpe1gb", "rdtscp",
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, "syscall",
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+"nx", NULL, "mmxext", NULL,
+NULL, "fxsr-opt", "pdpe1gb", "rdtscp",
 NULL, "lm", "3dnowext", "3dnow",
 },
-.cpuid = { .eax = 0x8001, .reg = R_EDX, },
+.cpuid = {
+.eax = 0x8001,
+.reg = R_EDX,
+},
 .tcg_features = TCG_EXT2_FEATURES,
 },
 [FEAT_8000_0001_ECX] = {
@@ -61,13 +65,11 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 "perfctr-nb", NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
 },
-.cpuid = { .eax = 0x8001, .reg = R_ECX, },
+.cpuid = {
+.eax = 0x8001,
+.reg = R_ECX,
+},
 .tcg_features = TCG_EXT3_FEATURES,
-/*
- * TOPOEXT is always allowed but can't be enabled blindly by
- * "-cpu host", as it requires consistent cache to

[PATCH 0/4] Generate x86 cpu features

2023-08-11 Thread Tim Wiederhake
Synchronizing the list of cpu features and models with qemu is a recurring
task in libvirt. For x86, this is done by reading qom-list-properties for
max-x86_64-cpu and manually filtering out everthing that does not look like
a feature name, as well as parsing target/i386/cpu.c for cpu models.

This is a flawed, tedious and error-prone procedure. Ideally, qemu
and libvirt would query a common source for cpu feature and model
related information. Meanwhile, converting this information into an easier
to parse format would help libvirt a lot.

This patch series converts the cpu feature information present in
target/i386/cpu.c (`feature_word_info`) into an xml file and adds a
script to generate the c code from this xml.

A patch set to convert the cpu model data (`builtin_x86_defs`) in the
same way will follow.

Tim Wiederhake (4):
  target/i386: Split out feature_word_info
  target/i386: Translate feature_word_info to xml
  target/i386: Format feature_word_info.c.inc
  target/i386: Autogenerate feature_word_info.c.inc

 target/i386/cpu.c   |  677 +--
 target/i386/feature_word_info.c.inc |  704 
 target/i386/feature_word_info.py|  110 ++
 target/i386/feature_word_info.xml   | 1610 +++
 4 files changed, 2425 insertions(+), 676 deletions(-)
 create mode 100644 target/i386/feature_word_info.c.inc
 create mode 100755 target/i386/feature_word_info.py
 create mode 100644 target/i386/feature_word_info.xml

-- 
2.39.2




[PATCH] i386/cpu: Fix Icelake Server model number

2020-12-02 Thread Tim Wiederhake
See arch/x86/include/asm/intel-family.h in the Kernel:
  #define INTEL_FAM6_ICELAKE_X  0x6A

Signed-off-by: Tim Wiederhake 
---
 target/i386/cpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 5a8c96072e..67e3f92f98 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -3381,7 +3381,7 @@ static X86CPUDefinition builtin_x86_defs[] = {
 .level = 0xd,
 .vendor = CPUID_VENDOR_INTEL,
 .family = 6,
-.model = 134,
+.model = 106,
 .stepping = 0,
 .features[FEAT_1_EDX] =
 CPUID_VME | CPUID_SSE2 | CPUID_SSE | CPUID_FXSR | CPUID_MMX |
-- 
2.26.2