Re: [Qemu-devel] [PATCH qemu] loader: Trace loaded images

[Alexey Kardashevskiy]

On 17/06/2019 14:56, Philippe Mathieu-Daudé wrote:
> On 6/17/19 3:25 AM, Alexey Kardashevskiy wrote:
>> On 14/06/2019 19:33, Stefan Hajnoczi wrote:
>>> On Fri, Jun 14, 2019 at 10:13:04AM +1000, Alexey Kardashevskiy wrote:


 On 13/06/2019 23:08, Philippe Mathieu-Daudé wrote:
> Hi Alexey,
>
> On 6/13/19 7:09 AM, Alexey Kardashevskiy wrote:
>> This adds a trace point which prints every loaded image. This includes
>> bios/firmware/kernel/initradmdisk/pcirom.
>>
>> Signed-off-by: Alexey Kardashevskiy 
>> ---
>>
>> The example for a pseries guest:
>>
>> loader_write_rom slof.bin: @0x0 size=0xe22e0 ROM=0
>> loader_write_rom phdr #0: /home/aik/t/vml4120le: @0x40 
>> size=0x13df000 ROM=0
>> loader_write_rom /home/aik/t/le.cpio: @0x1ad size=0x9463a00 ROM=0
>
> I find the "ROM=0" part confuse, maybe you can change to "ROM:false".

 How? I mean I can do that in the code as rom->isrom?"true":"false" and
 make trace point accept "%s" but it is quite ugly and others seem to
 just use %d for bool.
>>>
>>> Yes, %d is the convention for bool.  Perhaps you can name it "is_rom"
>>> instead of "ROM".  That way the name communicates that this is a boolean
>>> value.
>>
>> It is quite obvious though that it is boolean even as "ROM" (what else
>> can that be realistically?) and there does not seem to be a convention
>> about xxx:N vs is_xxx:N. And personally I find longer lines worse for
>> limited width screens (I run multiple qemus in tiled tmux). Whose tree
>> is this going to? Let's ask that person :)
> 
> Personally I find 'is_rom' clearer. I read 2 addresses, then my first
> reaction was to parse it as another address. But it is also true we now
> enforce traced hex values with '0x' prefix, so your 'ROM' is unlikely an
> address. Tiled tmux is an acceptable argument. Anyway you already got my
> R-b.
> 
> Tree: the PPC tree is likely to get it merged quicker than the MISC tree.


There is nothing specific about PPC though so I guess it is the MISC
tree, who does maintain that?


-- 
Alexey

[Qemu-devel] [PATCH v4 1/3] target/i386: Add CPUID.1F generation support for multi-dies PCMachine

[Like Xu]

The CPUID.1F as Intel V2 Extended Topology Enumeration Leaf would be
exposed if guests want to emulate multiple software-visible die within
each package. Per Intel's SDM, the 0x1f is a superset of 0xb, thus they
can be generated by almost same code as 0xb except die_offset setting.

If the number of dies per package is greater than 1, the cpuid_min_level
would be adjusted to 0x1f regardless of whether the host supports CPUID.1F.
Likewise, the CPUID.1F wouldn't be exposed if env->nr_dies < 2.

Suggested-by: Eduardo Habkost 
Signed-off-by: Like Xu 
---
 target/i386/cpu.c | 41 +
 target/i386/cpu.h |  1 +
 target/i386/kvm.c | 12 
 3 files changed, 54 insertions(+)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 88908a6373..efcbe6a2b2 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -4439,6 +4439,42 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, 
uint32_t count,
 *ecx |= CPUID_TOPOLOGY_LEVEL_INVALID;
 }
 
+assert(!(*eax & ~0x1f));
+*ebx &= 0x; /* The count doesn't need to be reliable. */
+break;
+case 0x1F:
+/* V2 Extended Topology Enumeration Leaf */
+if (env->nr_dies < 2) {
+*eax = *ebx = *ecx = *edx = 0;
+break;
+}
+
+*ecx = count & 0xff;
+*edx = cpu->apic_id;
+switch (count) {
+case 0:
+*eax = apicid_core_offset(env->nr_dies, cs->nr_cores,
+cs->nr_threads);
+*ebx = cs->nr_threads;
+*ecx |= CPUID_TOPOLOGY_LEVEL_SMT;
+break;
+case 1:
+*eax = apicid_die_offset(env->nr_dies, cs->nr_cores,
+   cs->nr_threads);
+*ebx = cs->nr_cores * cs->nr_threads;
+*ecx |= CPUID_TOPOLOGY_LEVEL_CORE;
+break;
+case 2:
+*eax = apicid_pkg_offset(env->nr_dies, cs->nr_cores,
+   cs->nr_threads);
+*ebx = env->nr_dies * cs->nr_cores * cs->nr_threads;
+*ecx |= CPUID_TOPOLOGY_LEVEL_DIE;
+break;
+default:
+*eax = 0;
+*ebx = 0;
+*ecx |= CPUID_TOPOLOGY_LEVEL_INVALID;
+}
 assert(!(*eax & ~0x1f));
 *ebx &= 0x; /* The count doesn't need to be reliable. */
 break;
@@ -5116,6 +5152,11 @@ static void x86_cpu_expand_features(X86CPU *cpu, Error 
**errp)
 x86_cpu_adjust_level(cpu, >env.cpuid_min_level, 0x14);
 }
 
+/* CPU topology with multi-dies support requires CPUID[0x1F] */
+if (env->nr_dies > 1) {
+x86_cpu_adjust_level(cpu, >cpuid_min_level, 0x1F);
+}
+
 /* SVM requires CPUID[0x800A] */
 if (env->features[FEAT_8000_0001_ECX] & CPUID_EXT3_SVM) {
 x86_cpu_adjust_level(cpu, >cpuid_min_xlevel, 0x800A);
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 46dd81f6b7..eec6e4b7b7 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -726,6 +726,7 @@ typedef uint32_t FeatureWordArray[FEATURE_WORDS];
 #define CPUID_TOPOLOGY_LEVEL_INVALID  (0U << 8)
 #define CPUID_TOPOLOGY_LEVEL_SMT  (1U << 8)
 #define CPUID_TOPOLOGY_LEVEL_CORE (2U << 8)
+#define CPUID_TOPOLOGY_LEVEL_DIE  (5U << 8)
 
 /* MSR Feature Bits */
 #define MSR_ARCH_CAP_RDCL_NO(1U << 0)
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 6899061b4e..5deb4248ac 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1080,6 +1080,10 @@ int kvm_arch_init_vcpu(CPUState *cs)
 }
 break;
 }
+case 0x1f:
+if (env->nr_dies < 2) {
+break;
+}
 case 4:
 case 0xb:
 case 0xd:
@@ -1087,6 +1091,11 @@ int kvm_arch_init_vcpu(CPUState *cs)
 if (i == 0xd && j == 64) {
 break;
 }
+
+if (i == 0x1f && j == 64) {
+break;
+}
+
 c->function = i;
 c->flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX;
 c->index = j;
@@ -1098,6 +1107,9 @@ int kvm_arch_init_vcpu(CPUState *cs)
 if (i == 0xb && !(c->ecx & 0xff00)) {
 break;
 }
+if (i == 0x1f && !(c->ecx & 0xff00)) {
+break;
+}
 if (i == 0xd && c->eax == 0) {
 continue;
 }
-- 
2.21.0

[Qemu-devel] [PATCH v4 3/3] vl.c: Add -smp, dies=* command line support and update doc

[Like Xu]

For PC target, users could configure the number of dies per one package
via command line with this patch, such as "-smp dies=2,cores=4".

The parsing rules of new cpu-topology model obey the same restrictions/logic
as the legacy socket/core/thread model especially on missing values computing.

Signed-off-by: Like Xu 
---
 hw/i386/pc.c| 30 +-
 qemu-options.hx | 17 +
 vl.c|  3 +++
 3 files changed, 29 insertions(+), 21 deletions(-)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 092bd10d4d..2ed1b3f8de 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1548,9 +1548,12 @@ static void pc_new_cpu(PCMachineState *pcms, int64_t 
apic_id, Error **errp)
  */
 void pc_smp_parse(MachineState *ms, QemuOpts *opts)
 {
+PCMachineState *pcms = PC_MACHINE(ms);
+
 if (opts) {
 unsigned cpus= qemu_opt_get_number(opts, "cpus", 0);
 unsigned sockets = qemu_opt_get_number(opts, "sockets", 0);
+unsigned dies = qemu_opt_get_number(opts, "dies", 1);
 unsigned cores   = qemu_opt_get_number(opts, "cores", 0);
 unsigned threads = qemu_opt_get_number(opts, "threads", 0);
 
@@ -1560,24 +1563,24 @@ void pc_smp_parse(MachineState *ms, QemuOpts *opts)
 threads = threads > 0 ? threads : 1;
 if (cpus == 0) {
 sockets = sockets > 0 ? sockets : 1;
-cpus = cores * threads * sockets;
+cpus = cores * threads * dies * sockets;
 } else {
 ms->smp.max_cpus =
 qemu_opt_get_number(opts, "maxcpus", cpus);
-sockets = ms->smp.max_cpus / (cores * threads);
+sockets = ms->smp.max_cpus / (cores * threads * dies);
 }
 } else if (cores == 0) {
 threads = threads > 0 ? threads : 1;
-cores = cpus / (sockets * threads);
+cores = cpus / (sockets * dies * threads);
 cores = cores > 0 ? cores : 1;
 } else if (threads == 0) {
-threads = cpus / (cores * sockets);
+threads = cpus / (cores * dies * sockets);
 threads = threads > 0 ? threads : 1;
-} else if (sockets * cores * threads < cpus) {
+} else if (sockets * dies * cores * threads < cpus) {
 error_report("cpu topology: "
- "sockets (%u) * cores (%u) * threads (%u) < "
+ "sockets (%u) * dies (%u) * cores (%u) * threads (%u) 
< "
  "smp_cpus (%u)",
- sockets, cores, threads, cpus);
+ sockets, dies, cores, threads, cpus);
 exit(1);
 }
 
@@ -1589,26 +1592,27 @@ void pc_smp_parse(MachineState *ms, QemuOpts *opts)
 exit(1);
 }
 
-if (sockets * cores * threads > ms->smp.max_cpus) {
+if (sockets * dies * cores * threads > ms->smp.max_cpus) {
 error_report("cpu topology: "
- "sockets (%u) * cores (%u) * threads (%u) > "
+ "sockets (%u) * dies (%u) * cores (%u) * threads (%u) 
> "
  "maxcpus (%u)",
- sockets, cores, threads,
+ sockets, dies, cores, threads,
  ms->smp.max_cpus);
 exit(1);
 }
 
-if (sockets * cores * threads != ms->smp.max_cpus) {
+if (sockets * dies * cores * threads != ms->smp.max_cpus) {
 warn_report("Invalid CPU topology deprecated: "
-"sockets (%u) * cores (%u) * threads (%u) "
+"sockets (%u) * dies (%u) * cores (%u) * threads (%u) "
 "!= maxcpus (%u)",
-sockets, cores, threads,
+sockets, dies, cores, threads,
 ms->smp.max_cpus);
 }
 
 ms->smp.cpus = cpus;
 ms->smp.cores = cores;
 ms->smp.threads = threads;
+pcms->smp_dies = dies;
 }
 
 if (ms->smp.cpus > 1) {
diff --git a/qemu-options.hx b/qemu-options.hx
index 0d8beb4afd..a5b314a448 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -138,25 +138,26 @@ no incompatible TCG features have been enabled (e.g. 
icount/replay).
 ETEXI
 
 DEF("smp", HAS_ARG, QEMU_OPTION_smp,
-"-smp 
[cpus=]n[,maxcpus=cpus][,cores=cores][,threads=threads][,sockets=sockets]\n"
+"-smp 
[cpus=]n[,maxcpus=cpus][,cores=cores][,threads=threads][,dies=dies][,sockets=sockets]\n"
 "set the number of CPUs to 'n' [default=1]\n"
 "maxcpus= maximum number of total cpus, including\n"
 "offline CPUs for hotplug, etc\n"
-"cores= number of CPU cores on one socket\n"
+"cores= number of CPU cores on one socket (for PC, it's on 
one die)\n"
 "threads= number of threads on one CPU core\n"
+"

Re: [Qemu-devel] [SeaBIOS] [QEMU] [PATCH v4 0/8] Add Qemu to SeaBIOS LCHS interface

[Gerd Hoffmann]

> === OUTPUT BEGIN ===
> 1/8 Checking commit cf2168a66dbd (block: Refactor macros - fix tabbing)
> ERROR: Macros with complex values should be enclosed in parenthesis
> #55: FILE: include/hw/block/block.h:65:
> +#define DEFINE_BLOCK_CHS_PROPERTIES(_state, _conf)  \
> +DEFINE_PROP_UINT32("cyls", _state, _conf.cyls, 0),  \
> +DEFINE_PROP_UINT32("heads", _state, _conf.heads, 0),\
>  DEFINE_PROP_UINT32("secs", _state, _conf.secs, 0)

False positive.

> 8/8 Checking commit 6664d7d898d5 (hd-geo-test: Add tests for lchs override)
> WARNING: line over 80 characters
> #242: FILE: tests/hd-geo-test.c:579:
> + (char *)r->dev_path, &(r->chs.c), &(r->chs.h), 
> &(r->chs.s));

Should be fixed.

> WARNING: Block comments use a leading /* on a separate line
> #640: FILE: tests/hd-geo-test.c:995:
> +   "skipping hd-geo/override/* tests");

False positive too.

cheers,
  Gerd

[Qemu-devel] [PATCH v4 0/3] Introduce cpu die topology and enable CPUID.1F for i386

[Like Xu]

This patch series introduces a new cpu topolgy 'die' for PCMachine,
which extends virtual cpu topology to the socket/die/core/thread model,
allowing the setting of dies number per one socket via -smp qemu command.

For i386, it upgrades APIC-IDs generation and reversion functions with a
new exposed leaf called CPUID.1F, which is a preferred superset to leaf 0BH.
The CPUID.1F spec is on the latest Inetl SDM, 3-190 Vol 2A.

Guest system could discover multi-die/package topology through CPUID.1F.
and its benefit is primarily for _reporting_ of the guest CPU topology.
The guest kernel with multi-die/package support have no impact on its
cache topology, NUMA topology, Linux scheduler, or system performance.

==changelog==

v4:

- base commit: 22fa84da on github.com/ehabkost/qemu.git:machine-next
- refine comments for pc_smp_parse()
- remove the use of cpu->enable_cpuid_0x1f
- apply new logic for cpuid_min_level adjustment and drop the legacy one
- refine the way of MachineState casting in pc_smp_parse()
- [QUEUED] move test_topo_bits to the previous patch for bisectability

v3: https://patchwork.kernel.org/cover/10989013/

- add a MachineClass::smp_parse function pointer
- place the PC-specific function inside hw/i386/pc.c
- introduce die_id in a separate patch with default value 0
- set env->nr_dies in pc_new_cpu() and pc_cpu_pre_plug()
- fix a circular dependency between target/i386/cpu.c and hw/i386/pc.c
- fix cpu->die_id check in pc_cpu_pre_plug()
- Based on "[PATCH v3 00/10] Refactor cpu topo into machine properties"
- Rebase to commit 219dca61ebf41625831d4f96a720852baf44b762

v2: https://patchwork.kernel.org/cover/10953191/

- Enable cpu die-level topolgy only for PCMachine and X86CPU
- Minimize cpuid.0.eax to the setting value actually used by guest
- Update cmd line -smps docs for die-level configurations
- Refactoring topo-bit tests for x86_apicid_from_cpu_idx() with nr_dies
- Based on "[PATCH v3 00/10] Refactor cpu topo into machine properties"
- Rebase to commit 2259637b95bef3116cc262459271de08e038cc66

v1: https://patchwork.kernel.org/cover/10876667/

Like Xu (3):
  target/i386: Add CPUID.1F generation support for multi-dies PCMachine
  machine: Refactor smp_parse() in vl.c as MachineClass::smp_parse()
  vl.c: Add -smp, dies=* command line support and update doc

 hw/core/machine.c| 76 
 hw/i386/pc.c | 83 
 include/hw/boards.h  |  5 +++
 include/hw/i386/pc.h |  1 +
 qemu-options.hx  | 17 -
 target/i386/cpu.c| 41 ++
 target/i386/cpu.h|  1 +
 target/i386/kvm.c| 12 +++
 vl.c | 78 +++--
 9 files changed, 233 insertions(+), 81 deletions(-)

-- 
2.21.0

Re: [Qemu-devel] [QEMU] [PATCH v4 7/8] bootdevice: FW_CFG interface for LCHS values

[Gerd Hoffmann]

On Wed, Jun 19, 2019 at 12:29:04PM +0300, Sam Eiderman wrote:
> Using fw_cfg, supply logical CHS values directly from QEMU to the BIOS.
> 
> Non-standard logical geometries break under QEMU.
> 
> A virtual disk which contains an operating system which depends on
> logical geometries (consistent values being reported from BIOS INT13
> AH=08) will most likely break under QEMU/SeaBIOS if it has non-standard
> logical geometries - for example 56 SPT (sectors per track).
> No matter what QEMU will report - SeaBIOS, for large enough disks - will
> use LBA translation, which will report 63 SPT instead.
> 
> In addition we cannot force SeaBIOS to rely on physical geometries at
> all. A virtio-blk-pci virtual disk with 255 phyiscal heads cannot
> report more than 16 physical heads when moved to an IDE controller,
> since the ATA spec allows a maximum of 16 heads - this is an artifact of
> virtualization.
> 
> By supplying the logical geometries directly we are able to support such
> "exotic" disks.
> 
> We serialize this information in a similar way to the "bootorder"
> interface.
> The new fw_cfg entry is "bios-geometry".
> 
> Reviewed-by: Karl Heubaum 
> Reviewed-by: Arbel Moshe 
> Signed-off-by: Sam Eiderman 
> ---
>  bootdevice.c| 32 
>  hw/nvram/fw_cfg.c   | 14 +++---
>  include/sysemu/sysemu.h |  1 +
>  3 files changed, 44 insertions(+), 3 deletions(-)
> 
> diff --git a/bootdevice.c b/bootdevice.c
> index 2b12fb85a4..b034ad7bdc 100644
> --- a/bootdevice.c
> +++ b/bootdevice.c
> @@ -405,3 +405,35 @@ void del_boot_device_lchs(DeviceState *dev, const char 
> *suffix)
>  }
>  }
>  }
> +
> +/* Serialized as: (device name\0 + lchs struct) x devices */

Comment is outdated.

> +if (!mc->legacy_fw_cfg_order) {
> +buf = get_boot_devices_lchs_list();
> +ptr = fw_cfg_modify_file(s, "bios-geometry", (uint8_t *)buf, len);

Can fw_cfg_modify_file handle buf == NULL?

cheers,
  Gerd

[Qemu-devel] [PATCH v4 2/3] machine: Refactor smp_parse() in vl.c as MachineClass::smp_parse()

[Like Xu]

To make smp_parse() more flexible and expansive, a smp_parse function
pointer is added to MachineClass that machine types could override.

The generic smp_parse() code in vl.c is moved to hw/core/machine.c, and
become the default implementation of MachineClass::smp_parse. A PC-specific
function called pc_smp_parse() has been added to hw/i386/pc.c, which in
this patch changes nothing against the default one .

Suggested-by: Eduardo Habkost 
Signed-off-by: Like Xu 
Reviewed-by: Eduardo Habkost 
---
 hw/core/machine.c| 76 ++
 hw/i386/pc.c | 79 
 include/hw/boards.h  |  5 +++
 include/hw/i386/pc.h |  1 +
 vl.c | 75 ++---
 5 files changed, 163 insertions(+), 73 deletions(-)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index 8b8d263afe..36a838f1cb 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -11,6 +11,9 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/option.h"
+#include "qapi/qmp/qerror.h"
+#include "sysemu/replay.h"
 #include "qemu/units.h"
 #include "hw/boards.h"
 #include "qapi/error.h"
@@ -728,6 +731,78 @@ void machine_set_cpu_numa_node(MachineState *machine,
 }
 }
 
+static void smp_parse(MachineState *ms, QemuOpts *opts)
+{
+if (opts) {
+unsigned cpus= qemu_opt_get_number(opts, "cpus", 0);
+unsigned sockets = qemu_opt_get_number(opts, "sockets", 0);
+unsigned cores   = qemu_opt_get_number(opts, "cores", 0);
+unsigned threads = qemu_opt_get_number(opts, "threads", 0);
+
+/* compute missing values, prefer sockets over cores over threads */
+if (cpus == 0 || sockets == 0) {
+cores = cores > 0 ? cores : 1;
+threads = threads > 0 ? threads : 1;
+if (cpus == 0) {
+sockets = sockets > 0 ? sockets : 1;
+cpus = cores * threads * sockets;
+} else {
+ms->smp.max_cpus =
+qemu_opt_get_number(opts, "maxcpus", cpus);
+sockets = ms->smp.max_cpus / (cores * threads);
+}
+} else if (cores == 0) {
+threads = threads > 0 ? threads : 1;
+cores = cpus / (sockets * threads);
+cores = cores > 0 ? cores : 1;
+} else if (threads == 0) {
+threads = cpus / (cores * sockets);
+threads = threads > 0 ? threads : 1;
+} else if (sockets * cores * threads < cpus) {
+error_report("cpu topology: "
+ "sockets (%u) * cores (%u) * threads (%u) < "
+ "smp_cpus (%u)",
+ sockets, cores, threads, cpus);
+exit(1);
+}
+
+ms->smp.max_cpus =
+qemu_opt_get_number(opts, "maxcpus", cpus);
+
+if (ms->smp.max_cpus < cpus) {
+error_report("maxcpus must be equal to or greater than smp");
+exit(1);
+}
+
+if (sockets * cores * threads > ms->smp.max_cpus) {
+error_report("cpu topology: "
+ "sockets (%u) * cores (%u) * threads (%u) > "
+ "maxcpus (%u)",
+ sockets, cores, threads,
+ ms->smp.max_cpus);
+exit(1);
+}
+
+if (sockets * cores * threads != ms->smp.max_cpus) {
+warn_report("Invalid CPU topology deprecated: "
+"sockets (%u) * cores (%u) * threads (%u) "
+"!= maxcpus (%u)",
+sockets, cores, threads,
+ms->smp.max_cpus);
+}
+
+ms->smp.cpus = cpus;
+ms->smp.cores = cores;
+ms->smp.threads = threads;
+}
+
+if (ms->smp.cpus > 1) {
+Error *blocker = NULL;
+error_setg(, QERR_REPLAY_NOT_SUPPORTED, "smp");
+replay_add_blocker(blocker);
+}
+}
+
 static void machine_class_init(ObjectClass *oc, void *data)
 {
 MachineClass *mc = MACHINE_CLASS(oc);
@@ -735,6 +810,7 @@ static void machine_class_init(ObjectClass *oc, void *data)
 /* Default 128 MB as guest ram size */
 mc->default_ram_size = 128 * MiB;
 mc->rom_file_has_mr = true;
+mc->smp_parse = smp_parse;
 
 /* numa node memory size aligned on 8MB by default.
  * On Linux, each node's border has to be 8MB aligned
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 7b8c9caed6..092bd10d4d 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -79,6 +79,8 @@
 #include "hw/i386/intel_iommu.h"
 #include "hw/net/ne2000-isa.h"
 #include "standard-headers/asm-x86/bootparam.h"
+#include "sysemu/replay.h"
+#include "qapi/qmp/qerror.h"
 
 /* debug PC/ISA interrupts */
 //#define DEBUG_IRQ
@@ -1540,6 +1542,82 @@ static void pc_new_cpu(PCMachineState *pcms, int64_t 
apic_id, Error **errp)
 error_propagate(errp, local_err);
 }
 
+/*
+ * This function is very similar to 

Re: [Qemu-devel] [SeaBIOS] [PATCH v3 4/4] geometry: Apply LCHS values for boot devices

[Gerd Hoffmann]

> +static int
> +overriden_lchs_supplied(struct drive_s *drive)
> +{
> +return drive->lchs.cylinder || drive->lchs.head || drive->lchs.sector;
> +}

> +case TRANSLATION_MACHINE:

Hmm, why this name?  Doesn't look intuitive to me.

> +desc = "overriden";

I'd name that "host-supplied" or "fw-cfg".

> +cylinders = drive->lchs.cylinder;
> +heads = drive->lchs.head;
> +if (heads > 255)
> +heads = 255;

I suggest to move these sanity checks to overriden_lchs_supplied(), then
ignore the override altogether when heads or sectors is out of range
instead of trying to fixup things.

The other patches look all sane to me.

cheers,
  Gerd

Re: [Qemu-devel] [PATCH] memory: do not do out of bound notification

[Yan Zhao]

On Thu, Jun 20, 2019 at 12:02:30PM +0800, Peter Xu wrote:
> On Wed, Jun 19, 2019 at 03:17:41PM +0200, Auger Eric wrote:
> > Hi Yan,
> > 
> > [+ Peter]
> > 
> > On 6/19/19 10:49 AM, Yan Zhao wrote:
> > > even if an entry overlaps with notifier's range, should not map/unmap
> > > out of bound part in the entry.
> > 
> > I don't think the patch was based on the master as the trace at the very
> > end if not part of the upstream code.
> > > 
> > > This would cause problem in below case:
> > > 1. initially there are two notifiers with ranges
> > > 0-0xfedf, 0xfef0-0x,
> > > IOVAs from 0x3c00 - 0x3c1f is in shadow page table.
> > > 
> > > 2. in vfio, memory_region_register_iommu_notifier() is followed by
> > > memory_region_iommu_replay(), which will first call address space unmap,
> > > and walk and add back all entries in vtd shadow page table. e.g.
> > > (1) for notifier 0-0xfedf,
> > > IOVAs from 0 - 0x get unmapped,
> > > and IOVAs from 0x3c00 - 0x3c1f get mapped
> > 
> > While the patch looks sensible, the issue is the notifier scope used in
> > vtd_address_space_unmap is not a valid mask (ctpop64(size) != 1). Then
> > the size is recomputed (either using n = 64 - clz64(size) for the 1st
> > notifier or n = s->aw_bits for the 2d) and also the entry (especially
> > for the 2d notifier where it becomes 0) to get a proper alignment.
> > 
> > vtd_page_walk sends notifications per block or page (with valid
> > addr_mask) so stays within the notifier.
> > 
> > Modifying the entry->iova/addr_mask again in memory_region_notify_one
> > leads to unaligned start address / addr_mask. I don't think we want that.
> > 
> > Can't we modity the vtd_address_space_unmap() implementation to split
> > the invalidation in smaller chunks instead?
> 
> Seems workable, to be explicit - we can even cut it into chunks with
> different size to be efficient.  Like, this range:
> 
>   0x0e00_ - 0x1__ (size 0xf200_)
> 
> can be one of this:
> 
>   0x0e00 - 0x1000_ (size 0x0200_)
> 
> plus one of this:
> 
>   0x1000_ - 0x1__ (size 0xf000_)
> 
> Yan, could you help explain the issue better on how to reproduce and
> what's the error when the problem occurs?  For example, is that
> happened when a device hot-plugged into an existing VFIO container
> (with some mapped IOVAs)?  Did you get host DMA errors later on?
> 
> Thanks,
> 
> -- 
> Peter Xu

Hi Peter
it happens when there's an RMRR region in my guest iommu driver.
if not adding this range check, IOVAs in this region would be unmapped and DMA
faults are met in host.

Thanks
Yan

Re: [Qemu-devel] [PATCH] memory: do not do out of bound notification

[Peter Xu]

On Wed, Jun 19, 2019 at 03:17:41PM +0200, Auger Eric wrote:
> Hi Yan,
> 
> [+ Peter]
> 
> On 6/19/19 10:49 AM, Yan Zhao wrote:
> > even if an entry overlaps with notifier's range, should not map/unmap
> > out of bound part in the entry.
> 
> I don't think the patch was based on the master as the trace at the very
> end if not part of the upstream code.
> > 
> > This would cause problem in below case:
> > 1. initially there are two notifiers with ranges
> > 0-0xfedf, 0xfef0-0x,
> > IOVAs from 0x3c00 - 0x3c1f is in shadow page table.
> > 
> > 2. in vfio, memory_region_register_iommu_notifier() is followed by
> > memory_region_iommu_replay(), which will first call address space unmap,
> > and walk and add back all entries in vtd shadow page table. e.g.
> > (1) for notifier 0-0xfedf,
> > IOVAs from 0 - 0x get unmapped,
> > and IOVAs from 0x3c00 - 0x3c1f get mapped
> 
> While the patch looks sensible, the issue is the notifier scope used in
> vtd_address_space_unmap is not a valid mask (ctpop64(size) != 1). Then
> the size is recomputed (either using n = 64 - clz64(size) for the 1st
> notifier or n = s->aw_bits for the 2d) and also the entry (especially
> for the 2d notifier where it becomes 0) to get a proper alignment.
> 
> vtd_page_walk sends notifications per block or page (with valid
> addr_mask) so stays within the notifier.
> 
> Modifying the entry->iova/addr_mask again in memory_region_notify_one
> leads to unaligned start address / addr_mask. I don't think we want that.
> 
> Can't we modity the vtd_address_space_unmap() implementation to split
> the invalidation in smaller chunks instead?

Seems workable, to be explicit - we can even cut it into chunks with
different size to be efficient.  Like, this range:

  0x0e00_ - 0x1__ (size 0xf200_)

can be one of this:

  0x0e00 - 0x1000_ (size 0x0200_)

plus one of this:

  0x1000_ - 0x1__ (size 0xf000_)

Yan, could you help explain the issue better on how to reproduce and
what's the error when the problem occurs?  For example, is that
happened when a device hot-plugged into an existing VFIO container
(with some mapped IOVAs)?  Did you get host DMA errors later on?

Thanks,

-- 
Peter Xu

Re: [Qemu-devel] [PATCH v3 7/9] target/i386: Support multi-dies when host doesn't support CPUID.1F

[Eduardo Habkost]

On Thu, Jun 20, 2019 at 10:03:07AM +0800, Like Xu wrote:
> On 2019/6/20 7:36, Eduardo Habkost wrote:
> > On Wed, Jun 19, 2019 at 04:15:46PM -0300, Eduardo Habkost wrote:
> > > On Wed, Jun 12, 2019 at 04:41:02PM +0800, Like Xu wrote:
> > > > In guest CPUID generation process, the cpuid_min_level would be 
> > > > adjusted to
> > > > the maximum passed value for basic CPUID configuration and it should 
> > > > not be
> > > > restricted by the limited value returned from cpu_x86_cpuid(). After 
> > > > the basic
> > > > cpu_x86_cpuid() loop is finished, the cpuid_0_entry.eax needs to be 
> > > > configured
> > > > again by the last adjusted cpuid_min_level value.
> > > > 
> > > > If a user wants to expose CPUID.1F by passing dies > 1 for any reason 
> > > > without
> > > > host support, a per-cpu smp topology warning will appear but it's not 
> > > > blocked.
> > > > 
> > > > Signed-off-by: Like Xu 
> > > 
> > > This code doesn't look at host CPUID at all, as far as I can see.
> > > Isn't it simpler to just make cpuid_x86_cpuid() return the
> > > correct data?
> > 
> > I suggest the following change instead.
> > 
> > Signed-off-by: Eduardo Habkost 
> 
> Hi Eduardo,
> 
> Your code is more reasonable and concise than mine on this
> so let's not break cpuid_x86_cpuid().
> 
> I'll remove the use of enable_cpuid_0x1f in next version, and should I
> resend the patch series "Refactor cpu topo into machine properties" because
> rebase-fix may distract you ?

"Refactor cpu topo" and patches 1-4 of this series are already
queued on my machine-next branch.  You can send the next version
of the series using that branch as base:

  https://github.com/ehabkost/qemu.git machine-next

-- 
Eduardo

Re: [Qemu-devel] [PATCH] x86/cpu: use FeatureWordArray to define filtered_features

[Eduardo Habkost]

On Thu, Jun 20, 2019 at 10:37:46AM +0800, Wei Yang wrote:
> Use the same definition as features/user_features in CPUX86State.
> 
> Signed-off-by: Wei Yang 

Queued, thanks.

-- 
Eduardo

Re: [Qemu-devel] [PATCH] migration: Improve accuracy of vCPU throttling with per-vCPU timers

[Peter Xu]

On Wed, Jun 19, 2019 at 03:23:28PM +, Cosmin Marin wrote:
> 
> 
> On 19/06/2019, 02:35, "Peter Xu"  wrote:
> 
> On Tue, Jun 18, 2019 at 04:52:09PM +, Cosmin Marin wrote:
> > 
> > 
> > On 18/06/2019, 15:51, "Peter Xu"  wrote:
> > 
> > On Tue, Jun 18, 2019 at 12:25:43PM +, Cosmin Marin wrote:
> > >   Hi Peter,
> > > 
> > >   thanks for reviewing the patch. Indeed, I agree that it's 
> almost impossible to determine which solution it's better from the 
> scalability perspective. However, I feel that using per-vCPU timers is the 
> only way for ensuring correctness of the throttling ratio.
> > 
> > The thing is that your patch actually contains two changes:
> > 
> > 1. use N timers instead of one.
> > 
> > 2. remove throttle_thread_scheduled check, so we do the throttle
> >always
> > 
> > Here what I'm worried is that _maybe_ the 2nd item is the one that
> > really helped.
> > 
> > C: The removal of *throttle_thread_scheduled* is a consequence 
> of the per-vCPU model only. In this model, each of the vCPUs schedules work 
> just for itself (as part of the timer's firing callback) - there's no global 
> point of control - therefore, the variable isn't helpful for scheduling 
> anymore.
> > 
> > Note that there is a side effect that we might queue more than one
> > work on one specific cpu if we queue it too fast, but it does not
> > block us from trying it out to identify which item (1 or 2 or both)
> > really helped here.  Then if we think that (queuing too much) is an
> > issue then we can discuss on how to fix it since current patch will
> > have this problem as well.
> > 
中央党史> > C: I believe that in the per-vCPU timer implementation we 
cannot queue more than one piece of work because, here, the vCPU queues work 
for itself and that happens only when the timer fires - so, the two "states" - 
scheduling and sleeping - are mutually exclusive running from the same thread 
context. 
> 
> I think this is the place where I'm in question with - I don't think
> they are using the same context.  IMO the timer will always be run in
> the main thread no matter you use per-cpu timer or not, however the
> sleeping part should be run on per-cpu.
> 
> A simple way to verify it would be: break at cpu_throttle_timer_tick()
> to see which thread it is running in.
> 
>   You're absolutely right, it was indeed a confusion I made (I've run a 
> test in which I printed the thread IDs to confirm as well). However, I 
> believe that there are two contributing factors preventing the scheduling of 
> more than one piece of work: 
>   - the timer's firing period is always greater than the vCPU's 
> sleeping interval, therefore the timer won't fire while a vCPU is sleeping 
> and as a consequence no additional work is scheduled (as long as the start of 
> the sleeping time does not "move to the right" towards the next firing of the 
> timer)

I suspect the timer could still fire during vcpu sleeping.  The old
code have had that problem from the very beginning and that's why we
have had the throttle_thread_scheduled, AFAICT.  Meanwhile I cannot
see why per-cpu timer could help to avoid this.

The problem is async_run_on_cpu() will only queue the work onto the
CPU, but it never guarantees that when the work will be scheduled on
the CPU.  The delay should be unpredictable.

>   - the timer's callback schedules work for one vCPU only (simple 
> & fast) preventing additional delays between work executions on different 
> vCPUs or potential overlapping of timer firing with vCPU sleeps 

Splitting the single timer into per-cpu timers doesn't help at all IMO
because you'll need to call async_run_on_cpu() as many times as
before.  Although you'll be with different timer contexts, but you are
still with the _same_ main thread context for all these timers so you
should even need more time to schedule these timers in total.  With
that, it seems to me that it's even more overhead and it could bring
more delays comparing to the old code rather than helping anything.

If you can schedule these timers on separate threads, then I would
agree. But I don't see how it could happen easily.

> 
> > > 
> > >   It's a bit unclear to me how the throttling ratio inconsistency 
> can be fixed by using a single timer even avoiding the conditional timer 
> re-arming.  Could you provide more details about the use of a single timer ?
> > 
> > C: I feel like in this case it will sleep too much running into 
> a problem similar to the one solved by 90bb0c0; under heavy throttling more 
> than one work item may be scheduled.
> 
> Right.  So I feel like we need a solution that will avoid this problem
> but at the same time keep 

[Qemu-devel] [PATCH] x86/cpu: use FeatureWordArray to define filtered_features

[Wei Yang]

Use the same definition as features/user_features in CPUX86State.

Signed-off-by: Wei Yang 
---
 target/i386/cpu.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 0732e059ec..fd5ff13817 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1413,7 +1413,7 @@ struct X86CPU {
 } mwait;
 
 /* Features that were filtered out because of missing host capabilities */
-uint32_t filtered_features[FEATURE_WORDS];
+FeatureWordArray filtered_features;
 
 /* Enable PMU CPUID bits. This can't be enabled by default yet because
  * it doesn't have ABI stability guarantees, as it passes all PMU CPUID
-- 
2.19.1

Re: [Qemu-devel] [PATCH v2 11/21] aspeed/timer: Ensure positive muldiv delta

[Andrew Jeffery]

On Wed, 19 Jun 2019, at 02:24, Cédric Le Goater wrote:
> From: Christian Svensson 
> 
> If the host decrements the counter register that results in a negative
> delta. This is then passed to muldiv64 which only handles unsigned
> numbers resulting in bogus results.
> 
> This fix ensures the delta being operated on is positive.
> 
> Test case: kexec a kernel using aspeed_timer and it will freeze on the
> second bootup when the kernel initializes the timer. With this patch
> that no longer happens and the timer appears to run OK.
> 
> Signed-off-by: Christian Svensson 
> Signed-off-by: Cédric Le Goater 

Reviewed-by: Andrew Jeffery 

> ---
>  hw/timer/aspeed_timer.c | 6 +-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/timer/aspeed_timer.c b/hw/timer/aspeed_timer.c
> index 6501fa0768e7..1f0f1886fb2a 100644
> --- a/hw/timer/aspeed_timer.c
> +++ b/hw/timer/aspeed_timer.c
> @@ -275,7 +275,11 @@ static void 
> aspeed_timer_set_value(AspeedTimerCtrlState *s, int timer, int reg,
>  int64_t delta = (int64_t) value - (int64_t) 
> calculate_ticks(t, now);
>  uint32_t rate = calculate_rate(t);
>  
> -t->start += muldiv64(delta, NANOSECONDS_PER_SECOND, rate);
> +if (delta >= 0) {
> +t->start += muldiv64(delta, NANOSECONDS_PER_SECOND, rate);
> +} else {
> +t->start -= muldiv64(-delta, NANOSECONDS_PER_SECOND, rate);
> +}
>  aspeed_timer_mod(t);
>  }
>  break;
> -- 
> 2.21.0
> 
>

Re: [Qemu-devel] [PATCH v3 7/9] target/i386: Support multi-dies when host doesn't support CPUID.1F

[Like Xu]

On 2019/6/20 7:36, Eduardo Habkost wrote:

On Wed, Jun 19, 2019 at 04:15:46PM -0300, Eduardo Habkost wrote:

On Wed, Jun 12, 2019 at 04:41:02PM +0800, Like Xu wrote:

In guest CPUID generation process, the cpuid_min_level would be adjusted to
the maximum passed value for basic CPUID configuration and it should not be
restricted by the limited value returned from cpu_x86_cpuid(). After the basic
cpu_x86_cpuid() loop is finished, the cpuid_0_entry.eax needs to be configured
again by the last adjusted cpuid_min_level value.

If a user wants to expose CPUID.1F by passing dies > 1 for any reason without
host support, a per-cpu smp topology warning will appear but it's not blocked.

Signed-off-by: Like Xu 


This code doesn't look at host CPUID at all, as far as I can see.
Isn't it simpler to just make cpuid_x86_cpuid() return the
correct data?


I suggest the following change instead.

Signed-off-by: Eduardo Habkost 


Hi Eduardo,

Your code is more reasonable and concise than mine on this
so let's not break cpuid_x86_cpuid().

I'll remove the use of enable_cpuid_0x1f in next version, and should I 
resend the patch series "Refactor cpu topo into machine properties" 
because rebase-fix may distract you ?



---
  target/i386/cpu.c | 4 
  1 file changed, 4 insertions(+)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 6db38e145b..d05a224092 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -5152,6 +5152,10 @@ static void x86_cpu_expand_features(X86CPU *cpu, Error 
**errp)
  x86_cpu_adjust_level(cpu, >env.cpuid_min_level, 0x14);
  }
  
+if (env->nr_dies > 1) {

+x86_cpu_adjust_level(cpu, >cpuid_min_level, 0x1F);
+}
+
  /* SVM requires CPUID[0x800A] */
  if (env->features[FEAT_8000_0001_ECX] & CPUID_EXT3_SVM) {
  x86_cpu_adjust_level(cpu, >cpuid_min_xlevel, 0x800A);

Re: [Qemu-devel] [PATCH] memory: do not do out of bound notification

[Yan Zhao]

hi Eric,
Thanks for your reply.

On Wed, Jun 19, 2019 at 09:17:41PM +0800, Auger Eric wrote:
> Hi Yan,
> 
> [+ Peter]
> 
> On 6/19/19 10:49 AM, Yan Zhao wrote:
> > even if an entry overlaps with notifier's range, should not map/unmap
> > out of bound part in the entry.
> 
> I don't think the patch was based on the master as the trace at the very
> end if not part of the upstream code.
> > 
It's indeed based on the latest master branch. but I added a debug log
and forgot to remove that before sending out the patch. sorry for that :)


> > This would cause problem in below case:
> > 1. initially there are two notifiers with ranges
> > 0-0xfedf, 0xfef0-0x,
> > IOVAs from 0x3c00 - 0x3c1f is in shadow page table.
> > 
> > 2. in vfio, memory_region_register_iommu_notifier() is followed by
> > memory_region_iommu_replay(), which will first call address space unmap,
> > and walk and add back all entries in vtd shadow page table. e.g.
> > (1) for notifier 0-0xfedf,
> > IOVAs from 0 - 0x get unmapped,
> > and IOVAs from 0x3c00 - 0x3c1f get mapped
> 
> While the patch looks sensible, the issue is the notifier scope used in
> vtd_address_space_unmap is not a valid mask (ctpop64(size) != 1). Then
> the size is recomputed (either using n = 64 - clz64(size) for the 1st
> notifier or n = s->aw_bits for the 2d) and also the entry (especially
> for the 2d notifier where it becomes 0) to get a proper alignment.
>
maybe the size is calculated right, but 0 for the 2d notifier is because
this line below ?
 entry.iova = n->start & ~(size - 1);

> vtd_page_walk sends notifications per block or page (with valid
> addr_mask) so stays within the notifier.
> 
> Modifying the entry->iova/addr_mask again in memory_region_notify_one
> leads to unaligned start address / addr_mask. I don't think we want that.
>
if the notifier's start and end is aligned, and entry->iova/addr_mask is
aligned before modification,  then after modification, the start addr
/addr_mask are still aligned ?

> Can't we modity the vtd_address_space_unmap() implementation to split
> the invalidation in smaller chunks instead?
>
as this is an API, it cannot reply on the caller to ensure the entry is
within its address range. Do you think it's reasonable?

Thanks
Yan


> Thanks
> 
> Eric
> 
> 
> > (2) for notifier 0xfef0-0x
> > IOVAs from 0 - 0x7f get unmapped,> but IOVAs from 
> > 0x3c00 - 0x3c1f cannot get mapped back.
> > 
> > Signed-off-by: Yan Zhao 
> > ---
> >  memory.c | 8 
> >  1 file changed, 8 insertions(+)
> > 
> > diff --git a/memory.c b/memory.c
> > index 07c8315..a6b9da6 100644
> > --- a/memory.c
> > +++ b/memory.c
> > @@ -1948,6 +1948,14 @@ void memory_region_notify_one(IOMMUNotifier 
> > *notifier,
> >  return;
> >  }
> >  
> > +if (entry->iova < notifier->start) {
> > +entry->iova = notifier->start;
> > +}
> > +
> > +if (entry->iova + entry->addr_mask > notifier->end) {
> > +entry->addr_mask = notifier->end - entry->iova;> +}
> > +
> >  if (entry->perm & IOMMU_RW) {
> >  printf("map %lx %lx\n", entry->iova, entry->iova + 
> > entry->addr_mask);
> >  request_flags = IOMMU_NOTIFIER_MAP;
> 
> > 

[Qemu-devel] [PATCH 01/12] qapi: add BitmapSyncMode enum

[John Snow]

Depending on what a user is trying to accomplish, there might be a few
bitmap cleanup actions that occur when an operation is finished that
could be useful.

I am proposing three:
- NEVER: The bitmap is never synchronized against what was copied.
- ALWAYS: The bitmap is always synchronized, even on failures.
- CONDITIONAL: The bitmap is synchronized only on success.

The existing incremental backup modes use 'conditional' semantics,
so add just that one for right now.

Signed-off-by: John Snow 
---
 qapi/block-core.json | 14 ++
 1 file changed, 14 insertions(+)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 0d43d4f37c..caf28a71a0 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1134,6 +1134,20 @@
 { 'enum': 'MirrorSyncMode',
   'data': ['top', 'full', 'none', 'incremental'] }
 
+##
+# @BitmapSyncMode:
+#
+# An enumeration of possible behaviors for the synchronization of a bitmap
+# when used for data copy operations.
+#
+# @conditional: The bitmap is only synchronized when the operation is 
successul.
+#   This is useful for Incremental semantics.
+#
+# Since: 4.1
+##
+{ 'enum': 'BitmapSyncMode',
+  'data': ['conditional'] }
+
 ##
 # @MirrorCopyMode:
 #
-- 
2.21.0

[Qemu-devel] [PATCH 02/12] block/backup: Add mirror sync mode 'bitmap'

[John Snow]

We don't need or want a new sync mode for simple differences in
semantics.  Create a new mode simply named "BITMAP" that is designed to
make use of the new Bitmap Sync Mode field.

Because the only bitmap mode is 'conditional', this adds no new
functionality to the backup job (yet). The old incremental backup mode
is maintained as a syntactic sugar for sync=bitmap, mode=conditional.

Add all of the plumbing necessary to support this new instruction.

Signed-off-by: John Snow 
---
 qapi/block-core.json  | 30 ++
 include/block/block_int.h |  6 +-
 block/backup.c| 35 ---
 block/mirror.c|  6 --
 block/replication.c   |  2 +-
 blockdev.c|  8 ++--
 6 files changed, 66 insertions(+), 21 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index caf28a71a0..6d05ad8f47 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1127,12 +1127,15 @@
 #
 # @none: only copy data written from now on
 #
-# @incremental: only copy data described by the dirty bitmap. Since: 2.4
+# @incremental: only copy data described by the dirty bitmap. (since: 2.4)
+#
+# @bitmap: only copy data described by the dirty bitmap. (since: 4.1)
+#  Behavior on completion is determined by the BitmapSyncMode.
 #
 # Since: 1.3
 ##
 { 'enum': 'MirrorSyncMode',
-  'data': ['top', 'full', 'none', 'incremental'] }
+  'data': ['top', 'full', 'none', 'incremental', 'bitmap'] }
 
 ##
 # @BitmapSyncMode:
@@ -1352,10 +1355,14 @@
 #
 # @speed: the maximum speed, in bytes per second
 #
-# @bitmap: the name of dirty bitmap if sync is "incremental".
-#  Must be present if sync is "incremental", must NOT be present
+# @bitmap: the name of dirty bitmap if sync is "bitmap".
+#  Must be present if sync is "bitmap", must NOT be present
 #  otherwise. (Since 2.4)
 #
+# @bitmap-mode: Specifies the type of data the bitmap should contain after
+#   the operation concludes. Must be present if sync is "bitmap".
+#   Must NOT be present otherwise. (Since 4.1)
+#
 # @compress: true to compress data, if the target format supports it.
 #(default: false) (since 2.8)
 #
@@ -1390,7 +1397,8 @@
   'data': { '*job-id': 'str', 'device': 'str', 'target': 'str',
 '*format': 'str', 'sync': 'MirrorSyncMode',
 '*mode': 'NewImageMode', '*speed': 'int',
-'*bitmap': 'str', '*compress': 'bool',
+'*bitmap': 'str', '*bitmap-mode': 'BitmapSyncMode',
+'*compress': 'bool',
 '*on-source-error': 'BlockdevOnError',
 '*on-target-error': 'BlockdevOnError',
 '*auto-finalize': 'bool', '*auto-dismiss': 'bool' } }
@@ -1412,10 +1420,14 @@
 # @speed: the maximum speed, in bytes per second. The default is 0,
 # for unlimited.
 #
-# @bitmap: the name of dirty bitmap if sync is "incremental".
-#  Must be present if sync is "incremental", must NOT be present
+# @bitmap: the name of dirty bitmap if sync is "bitmap".
+#  Must be present if sync is "bitmap", must NOT be present
 #  otherwise. (Since 3.1)
 #
+# @bitmap-mode: Specifies the type of data the bitmap should contain after
+#   the operation concludes. Must be present if sync is "bitmap".
+#   Must NOT be present otherwise. (Since 4.1)
+#
 # @compress: true to compress data, if the target format supports it.
 #(default: false) (since 2.8)
 #
@@ -1449,7 +1461,9 @@
 { 'struct': 'BlockdevBackup',
   'data': { '*job-id': 'str', 'device': 'str', 'target': 'str',
 'sync': 'MirrorSyncMode', '*speed': 'int',
-'*bitmap': 'str', '*compress': 'bool',
+'*bitmap': 'str',
+'*bitmap-mode': 'BitmapSyncMode',
+'*compress': 'bool',
 '*on-source-error': 'BlockdevOnError',
 '*on-target-error': 'BlockdevOnError',
 '*auto-finalize': 'bool', '*auto-dismiss': 'bool' } }
diff --git a/include/block/block_int.h b/include/block/block_int.h
index d6415b53c1..89370c1b9b 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -1132,7 +1132,9 @@ void mirror_start(const char *job_id, BlockDriverState 
*bs,
  * @target: Block device to write to.
  * @speed: The maximum speed, in bytes per second, or 0 for unlimited.
  * @sync_mode: What parts of the disk image should be copied to the 
destination.
- * @sync_bitmap: The dirty bitmap if sync_mode is MIRROR_SYNC_MODE_INCREMENTAL.
+ * @sync_bitmap: The dirty bitmap if sync_mode is 'bitmap' or 'incremental'
+ * @has_bitmap_mode: true if @bitmap_sync carries a meaningful value.
+ * @bitmap_mode: The bitmap synchronization policy to use.
  * @on_source_error: The action to take upon error reading from the source.
  * @on_target_error: The action to take upon error writing to the target.
  * @creation_flags: Flags that control the behavior of the Job lifetime.
@@ 

[Qemu-devel] [PATCH 11/12] iotests: add test 257 for bitmap-mode backups

[John Snow]

Signed-off-by: John Snow 
---
 tests/qemu-iotests/257 |  412 +++
 tests/qemu-iotests/257.out | 2199 
 tests/qemu-iotests/group   |1 +
 3 files changed, 2612 insertions(+)
 create mode 100755 tests/qemu-iotests/257
 create mode 100644 tests/qemu-iotests/257.out

diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
new file mode 100755
index 00..5f7f388504
--- /dev/null
+++ b/tests/qemu-iotests/257
@@ -0,0 +1,412 @@
+#!/usr/bin/env python
+#
+# Test bitmap-sync backups (incremental, differential, and partials)
+#
+# Copyright (c) 2019 John Snow for Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+#
+# owner=js...@redhat.com
+
+from collections import namedtuple
+import math
+import os
+
+import iotests
+from iotests import log, qemu_img
+
+SIZE = 64 * 1024 * 1024
+GRANULARITY = 64 * 1024
+
+Pattern = namedtuple('Pattern', ['byte', 'offset', 'size'])
+def mkpattern(byte, offset, size=GRANULARITY):
+"""Constructor for Pattern() with default size"""
+return Pattern(byte, offset, size)
+
+class PatternGroup:
+"""Grouping of Pattern objects. Initialize with an iterable of Patterns."""
+def __init__(self, patterns):
+self.patterns = patterns
+
+def bits(self, granularity):
+"""Calculate the unique bits dirtied by this pattern grouping"""
+res = set()
+for pattern in self.patterns:
+lower = math.floor(pattern.offset / granularity)
+upper = math.floor((pattern.offset + pattern.size - 1) / 
granularity)
+res = res | set(range(lower, upper + 1))
+return res
+
+GROUPS = [
+PatternGroup([
+# Batch 0: 4 clusters
+mkpattern('0x49', 0x000),
+mkpattern('0x6c', 0x010),   # 1M
+mkpattern('0x6f', 0x200),   # 32M
+mkpattern('0x76', 0x3ff)]), # 64M - 64K
+PatternGroup([
+# Batch 1: 6 clusters (3 new)
+mkpattern('0x65', 0x000),   # Full overwrite
+mkpattern('0x77', 0x00f8000),   # Partial-left (1M-32K)
+mkpattern('0x72', 0x2008000),   # Partial-right (32M+32K)
+mkpattern('0x69', 0x3fe)]), # Adjacent-left (64M - 128K)
+PatternGroup([
+# Batch 2: 7 clusters (3 new)
+mkpattern('0x74', 0x001),   # Adjacent-right
+mkpattern('0x69', 0x00e8000),   # Partial-left  (1M-96K)
+mkpattern('0x6e', 0x2018000),   # Partial-right (32M+96K)
+mkpattern('0x67', 0x3fe,
+  2*GRANULARITY)]), # Overwrite [(64M-128K)-64M)
+PatternGroup([
+# Batch 3: 8 clusters (5 new)
+# Carefully chosen such that nothing re-dirties the one cluster
+# that copies out successfully before failure in Group #1.
+mkpattern('0xaa', 0x001,
+  3*GRANULARITY),   # Overwrite and 2x Adjacent-right
+mkpattern('0xbb', 0x00d8000),   # Partial-left (1M-160K)
+mkpattern('0xcc', 0x2028000),   # Partial-right (32M+160K)
+mkpattern('0xdd', 0x3fc)]), # New; leaving a gap to the right
+]
+
+class Drive:
+"""Represents, vaguely, a drive attached to a VM.
+Includes format, graph, and device information."""
+
+def __init__(self, path, vm=None):
+self.path = path
+self.vm = vm
+self.fmt = None
+self.size = None
+self.node = None
+self.device = None
+
+@property
+def name(self):
+return self.node or self.device
+
+def img_create(self, fmt, size):
+self.fmt = fmt
+self.size = size
+iotests.qemu_img_create('-f', self.fmt, self.path, str(self.size))
+
+def create_target(self, name, fmt, size):
+basename = os.path.basename(self.path)
+file_node_name = "file_{}".format(basename)
+vm = self.vm
+
+log(vm.command('blockdev-create', job_id='bdc-file-job',
+   options={
+   'driver': 'file',
+   'filename': self.path,
+   'size': 0,
+   }))
+vm.run_job('bdc-file-job')
+log(vm.command('blockdev-add', driver='file',
+   node_name=file_node_name, filename=self.path))
+
+log(vm.command('blockdev-create', job_id='bdc-fmt-job',
+   options={
+   

[Qemu-devel] [PATCH 00/12] bitmaps: introduce 'bitmap' sync mode

[John Snow]

This series adds a new "BITMAP" sync mode that is meant to replace the
existing "INCREMENTAL" sync mode.

This mode can be changed by issuing any of three bitmap sync modes;
passed as arguments to the job.

The three bitmap sync modes are:
- CONDITIONAL: This is an alias for the old incremental mode. The bitmap is
   conditionally synchronized based on the return code of the job
   upon completion.
- NEVER: This is, effectively, the differential backup mode. It never clears
 the bitmap, as the name suggests.
- ALWAYS: Here is the new, exciting thing. The bitmap is always synchronized,
  even on failure. On success, this is identical to incremental, but
  on failure it clears only the bits that were copied successfully.
  This can be used to "resume" incremental backups from later points
  in times.

I wrote this series by accident on my way to implement incremental mode
for mirror, but this happened first -- the problem is that Mirror mode
uses its existing modes in a very particular way; and this was the best
way to add bitmap support into the mirror job properly.

In this series:
patches 1-2: Adds CONDITIONAL
patch 3: Adds NEVER
Patches 4-7: Adds ALWAYS.
Patches 8-11: Adds iotest 257
Patch 12: Minor permission loosening for RO bitmaps.

Next up:
 - Add these modes to Mirror. (Done*, but need tests.)
 - Allow the use of bitmaps and bitmap sync modes with non-BITMAP modes;
   This will allow for resumable/re-tryable full backups.

John Snow (12):
  qapi: add BitmapSyncMode enum
  block/backup: Add mirror sync mode 'bitmap'
  block/backup: add 'never' policy to bitmap sync mode
  hbitmap: Fix merge when b is empty, and result is not an alias of a
  hbitmap: enable merging across granularities
  block/dirty-bitmap: add bdrv_dirty_bitmap_claim
  block/backup: add 'always' bitmap sync policy
  iotests: add testing shim for script-style python tests
  iotests: teach run_job to cancel pending jobs
  iotests: teach FilePath to produce multiple paths
  iotests: add test 257 for bitmap-mode backups
  block/backup: loosen restriction on readonly bitmaps

 qapi/block-core.json  |   51 +-
 include/block/block_int.h |7 +-
 include/qemu/hbitmap.h|8 +
 block/backup.c|   52 +-
 block/dirty-bitmap.c  |   14 +
 block/mirror.c|6 +-
 block/replication.c   |2 +-
 blockdev.c|   12 +-
 util/hbitmap.c|   36 +-
 tests/qemu-iotests/257|  412 ++
 tests/qemu-iotests/257.out| 2199 +
 tests/qemu-iotests/group  |1 +
 tests/qemu-iotests/iotests.py |   83 +-
 13 files changed, 2830 insertions(+), 53 deletions(-)
 create mode 100755 tests/qemu-iotests/257
 create mode 100644 tests/qemu-iotests/257.out

-- 
2.21.0

[Qemu-devel] [PATCH 05/12] hbitmap: enable merging across granularities

[John Snow]

Signed-off-by: John Snow 
---
 util/hbitmap.c | 22 +-
 1 file changed, 21 insertions(+), 1 deletion(-)

diff --git a/util/hbitmap.c b/util/hbitmap.c
index 45d1725daf..0d6724b7bc 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -777,7 +777,17 @@ void hbitmap_truncate(HBitmap *hb, uint64_t size)
 
 bool hbitmap_can_merge(const HBitmap *a, const HBitmap *b)
 {
-return (a->size == b->size) && (a->granularity == b->granularity);
+return (a->size == b->size);
+}
+
+static void hbitmap_sparse_merge(HBitmap *dst, const HBitmap *src)
+{
+uint64_t offset = 0;
+uint64_t count = src->orig_size;
+
+while (hbitmap_next_dirty_area(src, , )) {
+hbitmap_set(dst, offset, count);
+}
 }
 
 /**
@@ -804,6 +814,16 @@ bool hbitmap_merge(const HBitmap *a, const HBitmap *b, 
HBitmap *result)
 return true;
 }
 
+if (a->size != b->size) {
+if (a != result) {
+hbitmap_sparse_merge(result, a);
+}
+if (b != result) {
+hbitmap_sparse_merge(result, b);
+}
+return true;
+}
+
 /* This merge is O(size), as BITS_PER_LONG and HBITMAP_LEVELS are constant.
  * It may be possible to improve running times for sparsely populated maps
  * by using hbitmap_iter_next, but this is suboptimal for dense maps.
-- 
2.21.0

[Qemu-devel] [PATCH 06/12] block/dirty-bitmap: add bdrv_dirty_bitmap_claim

[John Snow]

This function can claim an hbitmap to replace its own current hbitmap.
In the case that the granularities do not match, it will use
hbitmap_merge to copy the bit data instead.

Signed-off-by: John Snow 
---
 include/block/block_int.h |  1 +
 include/qemu/hbitmap.h|  8 
 block/dirty-bitmap.c  | 14 ++
 util/hbitmap.c|  5 +
 4 files changed, 28 insertions(+)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index 89370c1b9b..7348ea8e78 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -1240,6 +1240,7 @@ void bdrv_set_dirty(BlockDriverState *bs, int64_t offset, 
int64_t bytes);
 
 void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap **out);
 void bdrv_restore_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap *backup);
+void bdrv_dirty_bitmap_claim(BdrvDirtyBitmap *bitmap, HBitmap **hbitmap);
 
 void bdrv_inc_in_flight(BlockDriverState *bs);
 void bdrv_dec_in_flight(BlockDriverState *bs);
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index 4afbe6292e..c74519042a 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -82,6 +82,14 @@ void hbitmap_truncate(HBitmap *hb, uint64_t size);
  */
 bool hbitmap_merge(const HBitmap *a, const HBitmap *b, HBitmap *result);
 
+/**
+ * hbitmap_same_conf:
+ *
+ * Compares the configuration for HBitmaps A and B.
+ * Return true if they are identical, false otherwise.
+ */
+bool hbitmap_same_conf(const HBitmap *a, const HBitmap *b);
+
 /**
  * hbitmap_can_merge:
  *
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 95a9c2a5d8..15c857e445 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -632,6 +632,20 @@ void bdrv_restore_dirty_bitmap(BdrvDirtyBitmap *bitmap, 
HBitmap *backup)
 hbitmap_free(tmp);
 }
 
+/* claim ownership of an hbitmap */
+void bdrv_dirty_bitmap_claim(BdrvDirtyBitmap *bitmap, HBitmap **hbitmap)
+{
+if (hbitmap_same_conf(bitmap->bitmap, *hbitmap)) {
+bdrv_restore_dirty_bitmap(bitmap, *hbitmap);
+} else {
+assert(hbitmap_can_merge(bitmap->bitmap, *hbitmap));
+bdrv_clear_dirty_bitmap(bitmap, NULL);
+hbitmap_merge(bitmap->bitmap, *hbitmap, bitmap->bitmap);
+hbitmap_free(*hbitmap);
+}
+*hbitmap = NULL;
+}
+
 uint64_t bdrv_dirty_bitmap_serialization_size(const BdrvDirtyBitmap *bitmap,
   uint64_t offset, uint64_t bytes)
 {
diff --git a/util/hbitmap.c b/util/hbitmap.c
index 0d6724b7bc..a2abd425b5 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -775,6 +775,11 @@ void hbitmap_truncate(HBitmap *hb, uint64_t size)
 }
 }
 
+bool hbitmap_same_conf(const HBitmap *a, const HBitmap *b)
+{
+return (a->size == b->size) && (a->granularity == b->granularity);
+}
+
 bool hbitmap_can_merge(const HBitmap *a, const HBitmap *b)
 {
 return (a->size == b->size);
-- 
2.21.0

[Qemu-devel] [PATCH 08/12] iotests: add testing shim for script-style python tests

[John Snow]

Because the new-style python tests don't use the iotests.main() test
launcher, we don't turn on the debugger logging for these scripts
when invoked via ./check -d.

Refactor the launcher shim into new and old style shims so that they
share environmental configuration.

Two cleanup notes: debug was not actually used as a global, and there
was no reason to create a class in an inner scope just to achieve
default variables; we can simply create an instance of the runner with
the values we want instead.

Signed-off-by: John Snow 
---
 tests/qemu-iotests/iotests.py | 40 +++
 1 file changed, 26 insertions(+), 14 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 3ecef5bc90..fcad957d63 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -61,7 +61,6 @@ cachemode = os.environ.get('CACHEMODE')
 qemu_default_machine = os.environ.get('QEMU_DEFAULT_MACHINE')
 
 socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper')
-debug = False
 
 luks_default_secret_object = 'secret,id=keysec0,data=' + \
  os.environ.get('IMGKEYSECRET', '')
@@ -834,11 +833,22 @@ def skip_if_unsupported(required_formats=[], 
read_only=False):
 return func_wrapper
 return skip_test_decorator
 
-def main(supported_fmts=[], supported_oses=['linux'], supported_cache_modes=[],
- unsupported_fmts=[]):
-'''Run tests'''
+def execute_unittest(output, verbosity, debug):
+runner = unittest.TextTestRunner(stream=output, descriptions=True,
+ verbosity=verbosity)
+try:
+# unittest.main() will use sys.exit(); so expect a SystemExit
+# exception
+unittest.main(testRunner=runner)
+finally:
+if not debug:
+sys.stderr.write(re.sub(r'Ran (\d+) tests? in [\d.]+s',
+r'Ran \1 tests', output.getvalue()))
 
-global debug
+def execute_test(test_function=None,
+ supported_fmts=[], supported_oses=['linux'],
+ supported_cache_modes=[], unsupported_fmts=[]):
+"""Run either unittest or script-style tests."""
 
 # We are using TEST_DIR and QEMU_DEFAULT_MACHINE as proxies to
 # indicate that we're not being run via "check". There may be
@@ -870,13 +880,15 @@ def main(supported_fmts=[], supported_oses=['linux'], 
supported_cache_modes=[],
 
 logging.basicConfig(level=(logging.DEBUG if debug else logging.WARN))
 
-class MyTestRunner(unittest.TextTestRunner):
-def __init__(self, stream=output, descriptions=True, 
verbosity=verbosity):
-unittest.TextTestRunner.__init__(self, stream, descriptions, 
verbosity)
+if not test_function:
+execute_unittest(output, verbosity, debug)
+else:
+test_function()
 
-# unittest.main() will use sys.exit() so expect a SystemExit exception
-try:
-unittest.main(testRunner=MyTestRunner)
-finally:
-if not debug:
-sys.stderr.write(re.sub(r'Ran (\d+) tests? in [\d.]+s', r'Ran \1 
tests', output.getvalue()))
+def script_main(test_function, *args, **kwargs):
+"""Run script-style tests outside of the unittest framework"""
+execute_test(test_function, *args, **kwargs)
+
+def main(*args, **kwargs):
+"""Run tests using the unittest framework"""
+execute_test(None, *args, **kwargs)
-- 
2.21.0

[Qemu-devel] [PATCH 09/12] iotests: teach run_job to cancel pending jobs

[John Snow]

run_job can cancel pending jobs to simulate failure. This lets us use
the pending callback to issue test commands while the job is open, but
then still have the job fail in the end.

Signed-off-by: John Snow 
---
 tests/qemu-iotests/iotests.py | 22 --
 1 file changed, 20 insertions(+), 2 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index fcad957d63..c544659ecb 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -541,7 +541,22 @@ class VM(qtest.QEMUQtestMachine):
 
 # Returns None on success, and an error string on failure
 def run_job(self, job, auto_finalize=True, auto_dismiss=False,
-pre_finalize=None, wait=60.0):
+pre_finalize=None, cancel=False, wait=60.0):
+"""
+run_job moves a job from creation through to dismissal.
+
+:param job: String. ID of recently-launched job
+:param auto_finalize: Bool. True if the job was launched with
+  auto_finalize. Defaults to True.
+:param auto_dismiss: Bool. True if the job was launched with
+ auto_dismiss=True. Defaults to False.
+:param pre_finalize: Callback. A callable that takes no arguments to be
+ invoked prior to issuing job-finalize, if any.
+:param cancel: Bool. When true, cancels the job after the pre_finalize
+   callback.
+:param wait: Float. Timeout value specifying how long to wait for any
+ event, in seconds. Defaults to 60.0.
+"""
 match_device = {'data': {'device': job}}
 match_id = {'data': {'id': job}}
 events = [
@@ -568,7 +583,10 @@ class VM(qtest.QEMUQtestMachine):
 elif status == 'pending' and not auto_finalize:
 if pre_finalize:
 pre_finalize()
-self.qmp_log('job-finalize', id=job)
+if cancel:
+self.qmp_log('job-cancel', id=job)
+else:
+self.qmp_log('job-finalize', id=job)
 elif status == 'concluded' and not auto_dismiss:
 self.qmp_log('job-dismiss', id=job)
 elif status == 'null':
-- 
2.21.0

[Qemu-devel] [PATCH 07/12] block/backup: add 'always' bitmap sync policy

[John Snow]

This adds an "always" policy for bitmap synchronization. Regardless of if
the job succeeds or fails, the bitmap is *always* synchronized. This means
that for backups that fail part-way through, the bitmap retains a record of
which sectors need to be copied out to accomplish a new backup using the
old, partial result.

In effect, this allows us to "resume" a failed backup; however the new backup
will be from the new point in time, so it isn't a "resume" as much as it is
an "incremental retry." This can be useful in the case of extremely large
backups that fail considerably through the operation and we'd like to not waste
the work that was already performed.

Signed-off-by: John Snow 
---
 qapi/block-core.json |  5 -
 block/backup.c   | 10 ++
 2 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 0332dcaabc..58d267f1f5 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1143,6 +1143,9 @@
 # An enumeration of possible behaviors for the synchronization of a bitmap
 # when used for data copy operations.
 #
+# @always: The bitmap is always synchronized with remaining blocks to copy,
+#  whether or not the operation has completed successfully or not.
+#
 # @conditional: The bitmap is only synchronized when the operation is 
successul.
 #   This is useful for Incremental semantics.
 #
@@ -1153,7 +1156,7 @@
 # Since: 4.1
 ##
 { 'enum': 'BitmapSyncMode',
-  'data': ['conditional', 'never'] }
+  'data': ['always', 'conditional', 'never'] }
 
 ##
 # @MirrorCopyMode:
diff --git a/block/backup.c b/block/backup.c
index 627f724b68..beb2078696 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -266,15 +266,17 @@ static void backup_cleanup_sync_bitmap(BackupBlockJob 
*job, int ret)
 BlockDriverState *bs = blk_bs(job->common.blk);
 
 if (ret < 0 || job->bitmap_mode == BITMAP_SYNC_MODE_NEVER) {
-/* Failure, or we don't want to synchronize the bitmap.
- * Merge the successor back into the parent, delete nothing. */
+/* Failure, or we don't want to synchronize the bitmap. */
+if (job->bitmap_mode == BITMAP_SYNC_MODE_ALWAYS) {
+bdrv_dirty_bitmap_claim(job->sync_bitmap, >copy_bitmap);
+}
+/* Merge the successor back into the parent. */
 bm = bdrv_reclaim_dirty_bitmap(bs, job->sync_bitmap, NULL);
-assert(bm);
 } else {
 /* Everything is fine, delete this bitmap and install the backup. */
 bm = bdrv_dirty_bitmap_abdicate(bs, job->sync_bitmap, NULL);
-assert(bm);
 }
+assert(bm);
 }
 
 static void backup_commit(Job *job)
-- 
2.21.0

[Qemu-devel] [PATCH 03/12] block/backup: add 'never' policy to bitmap sync mode

[John Snow]

This adds a "never" policy for bitmap synchronization. Regardless of if
the job succeeds or fails, we never update the bitmap. This can be used
to perform differential backups, or simply to avoid the job modifying a
bitmap.

Signed-off-by: John Snow 
---
 qapi/block-core.json | 6 +-
 block/backup.c   | 5 +++--
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 6d05ad8f47..0332dcaabc 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1146,10 +1146,14 @@
 # @conditional: The bitmap is only synchronized when the operation is 
successul.
 #   This is useful for Incremental semantics.
 #
+# @never: The bitmap is never synchronized with the operation, and is
+# treated solely as a manifest of blocks to copy.
+# This is useful for Differential semantics.
+#
 # Since: 4.1
 ##
 { 'enum': 'BitmapSyncMode',
-  'data': ['conditional'] }
+  'data': ['conditional', 'never'] }
 
 ##
 # @MirrorCopyMode:
diff --git a/block/backup.c b/block/backup.c
index c4f83d4ef7..627f724b68 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -265,8 +265,9 @@ static void backup_cleanup_sync_bitmap(BackupBlockJob *job, 
int ret)
 BdrvDirtyBitmap *bm;
 BlockDriverState *bs = blk_bs(job->common.blk);
 
-if (ret < 0) {
-/* Merge the successor back into the parent, delete nothing. */
+if (ret < 0 || job->bitmap_mode == BITMAP_SYNC_MODE_NEVER) {
+/* Failure, or we don't want to synchronize the bitmap.
+ * Merge the successor back into the parent, delete nothing. */
 bm = bdrv_reclaim_dirty_bitmap(bs, job->sync_bitmap, NULL);
 assert(bm);
 } else {
-- 
2.21.0

[Qemu-devel] [PATCH 04/12] hbitmap: Fix merge when b is empty, and result is not an alias of a

[John Snow]

Nobody calls the function like this currently, but we neither prohibit
or cope with this behavior. I decided to make the function cope with it.

Signed-off-by: John Snow 
---
 util/hbitmap.c | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/util/hbitmap.c b/util/hbitmap.c
index 7905212a8b..45d1725daf 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -781,8 +781,9 @@ bool hbitmap_can_merge(const HBitmap *a, const HBitmap *b)
 }
 
 /**
- * Given HBitmaps A and B, let A := A (BITOR) B.
- * Bitmap B will not be modified.
+ * Given HBitmaps A and B, let R := A (BITOR) B.
+ * Bitmaps A and B will not be modified,
+ * except when bitmap R is an alias of A or B.
  *
  * @return true if the merge was successful,
  * false if it was not attempted.
@@ -797,7 +798,9 @@ bool hbitmap_merge(const HBitmap *a, const HBitmap *b, 
HBitmap *result)
 }
 assert(hbitmap_can_merge(b, result));
 
-if (hbitmap_count(b) == 0) {
+if ((!hbitmap_count(a) && result == b) ||
+(!hbitmap_count(b) && result == a) ||
+(!hbitmap_count(a) && !hbitmap_count(b))) {
 return true;
 }
 
-- 
2.21.0

[Qemu-devel] [PATCH 10/12] iotests: teach FilePath to produce multiple paths

[John Snow]

Use "FilePaths" instead of "FilePath" to request multiple files be
cleaned up after we leave that object's scope.

This is not crucial; but it saves a little typing.

Signed-off-by: John Snow 
---
 tests/qemu-iotests/iotests.py | 21 +++--
 1 file changed, 15 insertions(+), 6 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index c544659ecb..b938fa9719 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -359,7 +359,7 @@ class Timeout:
 raise Exception(self.errmsg)
 
 
-class FilePath(object):
+class FilePaths(object):
 '''An auto-generated filename that cleans itself up.
 
 Use this context manager to generate filenames and ensure that the file
@@ -369,20 +369,29 @@ class FilePath(object):
 qemu_img('create', img_path, '1G')
 # migration_sock_path is automatically deleted
 '''
-def __init__(self, name):
-filename = '{0}-{1}'.format(os.getpid(), name)
-self.path = os.path.join(test_dir, filename)
+def __init__(self, names):
+self.paths = []
+for name in names:
+filename = '{0}-{1}'.format(os.getpid(), name)
+self.paths.append(os.path.join(test_dir, filename))
 
 def __enter__(self):
-return self.path
+return self.paths
 
 def __exit__(self, exc_type, exc_val, exc_tb):
 try:
-os.remove(self.path)
+for path in self.paths:
+os.remove(path)
 except OSError:
 pass
 return False
 
+class FilePath(FilePaths):
+def __init__(self, name):
+super(FilePath, self).__init__([name])
+
+def __enter__(self):
+return self.paths[0]
 
 def file_path_remover():
 for path in reversed(file_path_remover.paths):
-- 
2.21.0

[Qemu-devel] [PATCH 12/12] block/backup: loosen restriction on readonly bitmaps

[John Snow]

With the "never" sync policy, we actually can utilize readonly bitmaps
now. Loosen the check at the QMP level, and tighten it based on
provided arguments down at the job creation level instead.

Signed-off-by: John Snow 
---
 block/backup.c | 6 ++
 blockdev.c | 4 ++--
 2 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/block/backup.c b/block/backup.c
index beb2078696..ead9a049e2 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -614,6 +614,12 @@ BlockJob *backup_job_create(const char *job_id, 
BlockDriverState *bs,
 return NULL;
 }
 
+/* If we need to write to this bitmap, check that we can: */
+if (bitmap_mode != BITMAP_SYNC_MODE_NEVER &&
+bdrv_dirty_bitmap_check(sync_bitmap, BDRV_BITMAP_DEFAULT, errp)) {
+return NULL;
+}
+
 /* Create a new bitmap, and freeze/disable this one. */
 if (bdrv_dirty_bitmap_create_successor(bs, sync_bitmap, errp) < 0) {
 return NULL;
diff --git a/blockdev.c b/blockdev.c
index 7abbd6bbf2..173a6b85c6 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3560,7 +3560,7 @@ static BlockJob *do_drive_backup(DriveBackup *backup, 
JobTxn *txn,
 error_setg(errp, "Bitmap '%s' could not be found", backup->bitmap);
 goto unref;
 }
-if (bdrv_dirty_bitmap_check(bmap, BDRV_BITMAP_DEFAULT, errp)) {
+if (bdrv_dirty_bitmap_check(bmap, BDRV_BITMAP_ALLOW_RO, errp)) {
 goto unref;
 }
 }
@@ -3667,7 +3667,7 @@ BlockJob *do_blockdev_backup(BlockdevBackup *backup, 
JobTxn *txn,
 error_setg(errp, "Bitmap '%s' could not be found", backup->bitmap);
 goto out;
 }
-if (bdrv_dirty_bitmap_check(bmap, BDRV_BITMAP_DEFAULT, errp)) {
+if (bdrv_dirty_bitmap_check(bmap, BDRV_BITMAP_ALLOW_RO, errp)) {
 goto out;
 }
 }
-- 
2.21.0

Re: [Qemu-devel] [QEMU PATCH v4 10/10] target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities

[Maran Wilson]

On 6/19/2019 9:21 AM, Liran Alon wrote:

Previous commits have added support for migration of nested virtualization
workloads. This was done by utilising two new KVM capabilities:
KVM_CAP_NESTED_STATE and KVM_CAP_EXCEPTION_PAYLOAD. Both which are
required in order to correctly migrate such workloads.

Therefore, change code to add a migration blocker for vCPUs exposed with
Intel VMX or AMD SVM in case one of these kernel capabilities is
missing.

Signed-off-by: Liran Alon 
---
  target/i386/kvm.c | 9 +++--
  1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 99480a52ad33..a3d0fbed3b35 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1313,9 +1313,14 @@ int kvm_arch_init_vcpu(CPUState *cs)
!!(c->ecx & CPUID_EXT_SMX);
  }
  
-if (cpu_has_nested_virt(env) && !nested_virt_mig_blocker) {

+if (cpu_has_nested_virt(env) && !nested_virt_mig_blocker &&
+((kvm_max_nested_state_length() <= 0) || !has_exception_payload)) {
  error_setg(_virt_mig_blocker,
-   "Nested virtualization does not support live migration 
yet");
+   "Kernel do not provide required capabilities for "


s/do/does/

And with that change:

Reviewed-by: Maran Wilson 

Thanks,
-Maran



+   "nested virtualization migration. "
+   "(CAP_NESTED_STATE=%d, CAP_EXCEPTION_PAYLOAD=%d)",
+   kvm_max_nested_state_length() > 0,
+   has_exception_payload);
  r = migrate_add_blocker(nested_virt_mig_blocker, _err);
  if (local_err) {
  error_report_err(local_err);

Re: [Qemu-devel] [PATCH 15/17] dump: Move the code to dump/

[Marc-André Lureau]

On Wed, Jun 19, 2019 at 10:10 PM Markus Armbruster  wrote:
>
> Cc: Marc-André Lureau 
> Signed-off-by: Markus Armbruster 


Reviewed-by: Marc-André Lureau 


> ---
>  MAINTAINERS   | 2 +-
>  Makefile.target   | 3 +--
>  dump/Makefile.objs| 2 ++
>  dump.c => dump/dump.c | 0
>  win_dump.c => dump/win_dump.c | 0
>  win_dump.h => dump/win_dump.h | 0
>  6 files changed, 4 insertions(+), 3 deletions(-)
>  create mode 100644 dump/Makefile.objs
>  rename dump.c => dump/dump.c (100%)
>  rename win_dump.c => dump/win_dump.c (100%)
>  rename win_dump.h => dump/win_dump.h (100%)
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index ea110e99ca..a3fdda015f 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1850,7 +1850,7 @@ F: include/sysemu/device_tree.h
>  Dump
>  S: Supported
>  M: Marc-André Lureau 
> -F: dump.c
> +F: dump/dump.c
>  F: hw/misc/vmcoreinfo.c
>  F: include/hw/misc/vmcoreinfo.h
>  F: include/sysemu/dump-arch.h
> diff --git a/Makefile.target b/Makefile.target
> index 167ae2174e..a6919e0caf 100644
> --- a/Makefile.target
> +++ b/Makefile.target
> @@ -150,13 +150,12 @@ endif #CONFIG_BSD_USER
>  ifdef CONFIG_SOFTMMU
>  obj-y += arch_init.o cpus.o gdbstub.o balloon.o ioport.o
>  obj-y += qtest.o
> +obj-y += dump/
>  obj-y += hw/
>  obj-y += monitor/
>  obj-y += qapi/
>  obj-y += memory.o
>  obj-y += memory_mapping.o
> -obj-y += dump.o
> -obj-$(TARGET_X86_64) += win_dump.o
>  obj-y += migration/ram.o
>  LIBS := $(libs_softmmu) $(LIBS)
>
> diff --git a/dump/Makefile.objs b/dump/Makefile.objs
> new file mode 100644
> index 00..ea6b074967
> --- /dev/null
> +++ b/dump/Makefile.objs
> @@ -0,0 +1,2 @@
> +obj-y += dump.o
> +obj-$(TARGET_X86_64) += win_dump.o
> diff --git a/dump.c b/dump/dump.c
> similarity index 100%
> rename from dump.c
> rename to dump/dump.c
> diff --git a/win_dump.c b/dump/win_dump.c
> similarity index 100%
> rename from win_dump.c
> rename to dump/win_dump.c
> diff --git a/win_dump.h b/dump/win_dump.h
> similarity index 100%
> rename from win_dump.h
> rename to dump/win_dump.h
> --
> 2.21.0
>

Re: [Qemu-devel] [PATCH v3 7/9] target/i386: Support multi-dies when host doesn't support CPUID.1F

[Eduardo Habkost]

On Wed, Jun 19, 2019 at 04:15:46PM -0300, Eduardo Habkost wrote:
> On Wed, Jun 12, 2019 at 04:41:02PM +0800, Like Xu wrote:
> > In guest CPUID generation process, the cpuid_min_level would be adjusted to
> > the maximum passed value for basic CPUID configuration and it should not be
> > restricted by the limited value returned from cpu_x86_cpuid(). After the 
> > basic
> > cpu_x86_cpuid() loop is finished, the cpuid_0_entry.eax needs to be 
> > configured
> > again by the last adjusted cpuid_min_level value.
> > 
> > If a user wants to expose CPUID.1F by passing dies > 1 for any reason 
> > without
> > host support, a per-cpu smp topology warning will appear but it's not 
> > blocked.
> > 
> > Signed-off-by: Like Xu 
> 
> This code doesn't look at host CPUID at all, as far as I can see.
> Isn't it simpler to just make cpuid_x86_cpuid() return the
> correct data?

I suggest the following change instead.

Signed-off-by: Eduardo Habkost 
---
 target/i386/cpu.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 6db38e145b..d05a224092 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -5152,6 +5152,10 @@ static void x86_cpu_expand_features(X86CPU *cpu, Error 
**errp)
 x86_cpu_adjust_level(cpu, >env.cpuid_min_level, 0x14);
 }
 
+if (env->nr_dies > 1) {
+x86_cpu_adjust_level(cpu, >cpuid_min_level, 0x1F);
+}
+
 /* SVM requires CPUID[0x800A] */
 if (env->features[FEAT_8000_0001_ECX] & CPUID_EXT3_SVM) {
 x86_cpu_adjust_level(cpu, >cpuid_min_xlevel, 0x800A);
-- 
2.18.0.rc1.1.g3f1ff2140

Re: [Qemu-devel] [PATCH 16/17] MAINTAINERS: Add Windows dump to section "Dump"

[Marc-André Lureau]

On Wed, Jun 19, 2019 at 10:10 PM Markus Armbruster  wrote:
>
> Commit 2da91b54fe9 "dump: add Windows dump format to
> dump-guest-memory" neglected to update MAINTAINERS.  Do it now.
>
> Cc: Marc-André Lureau 
> Signed-off-by: Markus Armbruster 

argh, I guess I will CC relevant people ;)

Reviewed-by: Marc-André Lureau 


> ---
>  MAINTAINERS | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index a3fdda015f..327ad80ae4 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1850,9 +1850,10 @@ F: include/sysemu/device_tree.h
>  Dump
>  S: Supported
>  M: Marc-André Lureau 
> -F: dump/dump.c
> +F: dump/
>  F: hw/misc/vmcoreinfo.c
>  F: include/hw/misc/vmcoreinfo.h
> +F: include/qemu/win_dump_defs
>  F: include/sysemu/dump-arch.h
>  F: include/sysemu/dump.h
>  F: qapi/dump.json
> --
> 2.21.0
>

Re: [Qemu-devel] [PATCH 14/17] qapi: Split dump.json off misc.json

[Marc-André Lureau]

On Wed, Jun 19, 2019 at 10:10 PM Markus Armbruster  wrote:
>
> Move commands dump-guest-memory, query-dump,
> query-dump-guest-memory-capability with their types from misc.json to
> new dump.json.  Add dump.json to MAINTAINERS section "Dump".
>
> Cc: Marc-André Lureau 
> Signed-off-by: Markus Armbruster 

Reviewed-by: Marc-André Lureau 

> ---
>  MAINTAINERS   |   1 +
>  dump.c|   4 +-
>  include/sysemu/dump.h |   2 +-
>  monitor/hmp-cmds.c|   1 +
>  qapi/Makefile.objs|   2 +-
>  qapi/dump.json| 200 ++
>  qapi/misc.json| 192 
>  qapi/qapi-schema.json |   1 +
>  8 files changed, 207 insertions(+), 196 deletions(-)
>  create mode 100644 qapi/dump.json
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 89e8c01cf3..ea110e99ca 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1855,6 +1855,7 @@ F: hw/misc/vmcoreinfo.c
>  F: include/hw/misc/vmcoreinfo.h
>  F: include/sysemu/dump-arch.h
>  F: include/sysemu/dump.h
> +F: qapi/dump.json
>  F: scripts/dump-guest-memory.py
>  F: stubs/dump.c
>
> diff --git a/dump.c b/dump.c
> index e99554628c..c7b2301652 100644
> --- a/dump.c
> +++ b/dump.c
> @@ -24,8 +24,8 @@
>  #include "sysemu/memory_mapping.h"
>  #include "sysemu/cpus.h"
>  #include "qapi/error.h"
> -#include "qapi/qapi-commands-misc.h"
> -#include "qapi/qapi-events-misc.h"
> +#include "qapi/qapi-commands-dump.h"
> +#include "qapi/qapi-events-dump.h"
>  #include "qapi/qmp/qerror.h"
>  #include "qemu/error-report.h"
>  #include "hw/misc/vmcoreinfo.h"
> diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
> index d824bc0941..250143cb5a 100644
> --- a/include/sysemu/dump.h
> +++ b/include/sysemu/dump.h
> @@ -14,7 +14,7 @@
>  #ifndef DUMP_H
>  #define DUMP_H
>
> -#include "qapi/qapi-types-misc.h"
> +#include "qapi/qapi-types-dump.h"
>
>  #define MAKEDUMPFILE_SIGNATURE  "makedumpfile"
>  #define MAX_SIZE_MDF_HEADER (4096) /* max size of 
> makedumpfile_header */
> diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
> index ea8ae2966e..18ffeb7017 100644
> --- a/monitor/hmp-cmds.c
> +++ b/monitor/hmp-cmds.c
> @@ -31,6 +31,7 @@
>  #include "qapi/qapi-builtin-visit.h"
>  #include "qapi/qapi-commands-block.h"
>  #include "qapi/qapi-commands-char.h"
> +#include "qapi/qapi-commands-dump.h"
>  #include "qapi/qapi-commands-migration.h"
>  #include "qapi/qapi-commands-misc.h"
>  #include "qapi/qapi-commands-net.h"
> diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
> index c0be6fcd3e..c5a29e86e2 100644
> --- a/qapi/Makefile.objs
> +++ b/qapi/Makefile.objs
> @@ -6,7 +6,7 @@ util-obj-y += qmp-event.o
>  util-obj-y += qapi-util.o
>
>  QAPI_COMMON_MODULES = audio authz block-core block char common crypto
> -QAPI_COMMON_MODULES += introspect job machine migration misc net
> +QAPI_COMMON_MODULES += dump introspect job machine migration misc net
>  QAPI_COMMON_MODULES += qdev qom rdma rocker run-state sockets tpm
>  QAPI_COMMON_MODULES += trace transaction ui
>  QAPI_TARGET_MODULES = machine-target misc-target
> diff --git a/qapi/dump.json b/qapi/dump.json
> new file mode 100644
> index 00..2b35409a7b
> --- /dev/null
> +++ b/qapi/dump.json
> @@ -0,0 +1,200 @@
> +# -*- Mode: Python -*-
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or later.
> +# See the COPYING file in the top-level directory.
> +
> +##
> +# = Dump guest memory
> +##
> +
> +##
> +# @DumpGuestMemoryFormat:
> +#
> +# An enumeration of guest-memory-dump's format.
> +#
> +# @elf: elf format
> +#
> +# @kdump-zlib: kdump-compressed format with zlib-compressed
> +#
> +# @kdump-lzo: kdump-compressed format with lzo-compressed
> +#
> +# @kdump-snappy: kdump-compressed format with snappy-compressed
> +#
> +# @win-dmp: Windows full crashdump format,
> +#   can be used instead of ELF converting (since 2.13)
> +#
> +# Since: 2.0
> +##
> +{ 'enum': 'DumpGuestMemoryFormat',
> +  'data': [ 'elf', 'kdump-zlib', 'kdump-lzo', 'kdump-snappy', 'win-dmp' ] }
> +
> +##
> +# @dump-guest-memory:
> +#
> +# Dump guest's memory to vmcore. It is a synchronous operation that can take
> +# very long depending on the amount of guest memory.
> +#
> +# @paging: if true, do paging to get guest's memory mapping. This allows
> +#  using gdb to process the core file.
> +#
> +#  IMPORTANT: this option can make QEMU allocate several gigabytes
> +# of RAM. This can happen for a large guest, or a
> +# malicious guest pretending to be large.
> +#
> +#  Also, paging=true has the following limitations:
> +#
> +# 1. The guest may be in a catastrophic state or can have 
> corrupted
> +#memory, which cannot be trusted
> +# 2. The guest can be in real-mode even if paging is enabled. For
> +#example, the guest uses ACPI to sleep, and ACPI sleep state
> +#goes in real-mode
> +# 3. 

Re: [Qemu-devel] [PULL 0/6] update edk2 submodule & binaries to edk2-stable201905

[Laszlo Ersek]

On 06/17/19 12:54, Peter Maydell wrote:
> On Fri, 14 Jun 2019 at 21:25, Laszlo Ersek  wrote:
>>
>> The following changes since commit f3d0bec9f80e4ed7796fffa834ba0a53f2094f7f:
>>
>>   Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-06-14' 
>> into staging (2019-06-14 14:46:13 +0100)
>>
>> are available in the Git repository at:
>>
>>   https://github.com/lersek/qemu.git tags/edk2-pull-2019-06-14
>>
>> for you to fetch changes up to 541617cad3445fdc6735e9e5752e1f698e337737:
>>
>>   pc-bios: update the README file with edk2-stable201905 information 
>> (2019-06-14 21:48:00 +0200)
>>
>> 
>> edk2-stable201905 was released on 2019-06-06:
>>
>>   https://github.com/tianocore/edk2/releases/tag/edk2-stable201905
>>
>> Advance QEMU's edk2 submodule to edk2-stable201905, and rebuild the
>> firmware binaries. This should be the edk2 release that goes into QEMU
>> 4.1.
>>
>> Launchpad: https://bugs.launchpad.net/qemu/+bug/1831477
> 
> 
> Applied, thanks.

Thanks!

> 
> Please update the changelog at https://wiki.qemu.org/ChangeLog/4.1
> for any user-visible changes.

Done.

Laszlo

Re: [Qemu-devel] [PATCH v3 6/9] i386/cpu: Add CPUID.1F generation support for multi-dies PCMachine

[Eduardo Habkost]

I've just noticed one thing I don't understand:

On Wed, Jun 12, 2019 at 04:41:01PM +0800, Like Xu wrote:
> The CPUID.1F as Intel V2 Extended Topology Enumeration Leaf would be
> exposed if guests want to emulate multiple software-visible die within
> each package. Per Intel's SDM, the 0x1f is a superset of 0xb, thus they
> can be generated by almost same code as 0xb except die_offset setting.
> 
> If the number of dies per package is less than 2, the qemu will not
> expose CPUID.1F regardless of whether the host supports CPUID.1F.
> 
> Signed-off-by: Like Xu 
> ---
>  target/i386/cpu.c | 37 +
>  target/i386/cpu.h |  4 
>  target/i386/kvm.c | 12 
>  3 files changed, 53 insertions(+)
> 
> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> index 09e20a2c3b..127aff74a6 100644
> --- a/target/i386/cpu.c
> +++ b/target/i386/cpu.c
> @@ -4437,6 +4437,42 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, 
> uint32_t count,
>  *ecx |= CPUID_TOPOLOGY_LEVEL_INVALID;
>  }
>  
> +assert(!(*eax & ~0x1f));
> +*ebx &= 0x; /* The count doesn't need to be reliable. */
> +break;
> +case 0x1F:
> +/* V2 Extended Topology Enumeration Leaf */
> +if (env->nr_dies < 2 || !cpu->enable_cpuid_0x1f) {
> +*eax = *ebx = *ecx = *edx = 0;

Why exactly do you need cpu->enable_cpuid_0x1f?  When would it
make sense to set dies > 1 but disable CPUID.1F?


> +break;
> +}
[...]

-- 
Eduardo

Re: [Qemu-devel] [PATCH v8 00/10] hw/m68k: add Apple Machintosh Quadra 800 machine

[no-reply]

Patchew URL: https://patchew.org/QEMU/20190619221933.1981-1-laur...@vivier.eu/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Subject: [Qemu-devel] [PATCH v8 00/10] hw/m68k: add Apple Machintosh Quadra 800 
machine
Type: series
Message-id: 20190619221933.1981-1-laur...@vivier.eu

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

From https://github.com/patchew-project/qemu
 * [new tag]   patchew/20190619221933.1981-1-laur...@vivier.eu -> 
patchew/20190619221933.1981-1-laur...@vivier.eu
Switched to a new branch 'test'
d87fb4249f hw/m68k: define Macintosh Quadra 800
65f79545a0 hw/m68k: add a dummy SWIM floppy controller
7c656d0722 hw/m68k: add Nubus support for macfb video card
be7b2607e8 hw/m68k: add Nubus support
55b5d6c451 hw/m68k: add macfb video card
8194119b9c hw/m68k: implement ADB bus support for via
9f35eabcdf hw/m68k: add via support
996fa6f5f1 dp8393x: manage big endian bus
1c2a722f31 esp: add pseudo-DMA as used by Macintosh
0bbfb84e62 escc: introduce a selector for the register bit

=== OUTPUT BEGIN ===
1/10 Checking commit 0bbfb84e625c (escc: introduce a selector for the register 
bit)
2/10 Checking commit 1c2a722f314f (esp: add pseudo-DMA as used by Macintosh)
3/10 Checking commit 996fa6f5f117 (dp8393x: manage big endian bus)
4/10 Checking commit 9f35eabcdf9a (hw/m68k: add via support)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#39: 
new file mode 100644

WARNING: Block comments use a leading /* on a separate line
#92: FILE: hw/misc/mac_via.c:49:
+#define VIA1A_vSccWrReq 0x80   /* SCC write. (input)

WARNING: Block comments use a leading /* on a separate line
#99: FILE: hw/misc/mac_via.c:56:
+#define VIA1A_vRev8 0x40   /* Revision 8 board ???

WARNING: Block comments use a leading /* on a separate line
#109: FILE: hw/misc/mac_via.c:66:
+#define VIA1A_vHeadSel  0x20   /* Head select for IWM.

WARNING: Block comments use a leading /* on a separate line
#114: FILE: hw/misc/mac_via.c:71:
+#define VIA1A_vOverlay  0x10   /* [Macintosh Family] On SE/30,II,IIx,IIcx

WARNING: Block comments use a leading /* on a separate line
#128: FILE: hw/misc/mac_via.c:85:
+#define VIA1A_vSync 0x08   /* [CHRP] Sync Modem: modem clock select:

WARNING: Block comments use a leading /* on a separate line
#136: FILE: hw/misc/mac_via.c:93:
+/* Macintosh Family Hardware sez: bits 0-2 of VIA1A are volume control

WARNING: Block comments use a leading /* on a separate line
#151: FILE: hw/misc/mac_via.c:108:
+/* Info on VIA1B is from Macintosh Family Hardware & MkLinux.

WARNING: Block comments use a trailing */ on a separate line
#152: FILE: hw/misc/mac_via.c:109:
+ * CHRP offers no info. */

WARNING: Block comments use a leading /* on a separate line
#153: FILE: hw/misc/mac_via.c:110:
+#define VIA1B_vSound   0x80/* Sound enable (for compatibility with

WARNING: Block comments use a trailing */ on a separate line
#156: FILE: hw/misc/mac_via.c:113:
+* 0=error, 1=OK. */

WARNING: Block comments use a leading /* on a separate line
#157: FILE: hw/misc/mac_via.c:114:
+#define VIA1B_vMystery 0x40/* On IIci, parity enable. 0=enabled,1=disabled

WARNING: Block comments use a trailing */ on a separate line
#160: FILE: hw/misc/mac_via.c:117:
+* as a slot $E interrupt. */

WARNING: Block comments use a leading /* on a separate line
#187: FILE: hw/misc/mac_via.c:144:
+/* RAM size bits decoded as follows:

WARNING: Block comments use a leading /* on a separate line
#199: FILE: hw/misc/mac_via.c:156:
+#define VIA2B_vVBL0x80/* VBL output to VIA1 (60.15Hz) driven by

WARNING: Block comments use a trailing */ on a separate line
#202: FILE: hw/misc/mac_via.c:159:
+   * [MkLinux] RBV_PARODD: 1=odd,0=even. */

WARNING: Block comments use a leading /* on a separate line
#203: FILE: hw/misc/mac_via.c:160:
+#define VIA2B_vSndJck 0x40/* External sound jack status.

WARNING: Block comments use a trailing */ on a separate line
#204: FILE: hw/misc/mac_via.c:161:
+   * 0=plug is inserted.  On SE/30, always 0 */

WARNING: Block comments use a leading /* on a separate line
#207: FILE: hw/misc/mac_via.c:164:
+#define VIA2B_vMode32 0x08/* 24/32bit switch - doubles as cache flush

WARNING: Block comments use a leading /* on a separate line
#215: FILE: hw/misc/mac_via.c:172:
+#define VIA2B_vPower  0x04   /* Power off, 0=shut off power.

WARNING: Block comments use a leading /* on a separate line
#218: FILE: hw/misc/mac_via.c:175:
+#define VIA2B_vBusLk  0x02   /* Lock NuBus transactions, 0=locked.

WARNING: Block comments use a leading /* on a separate line
#221: FILE: hw/misc/mac_via.c:178:
+#define 

[Qemu-devel] [PATCH v8 04/10] hw/m68k: add via support

[Laurent Vivier]

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
Reviewed-by: Hervé Poussineau 
---
 hw/misc/Kconfig   |   4 +
 hw/misc/Makefile.objs |   1 +
 hw/misc/mac_via.c | 667 ++
 include/hw/misc/mac_via.h | 107 ++
 4 files changed, 779 insertions(+)
 create mode 100644 hw/misc/mac_via.c
 create mode 100644 include/hw/misc/mac_via.h

diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig
index 385e1b0cec..87312c3abb 100644
--- a/hw/misc/Kconfig
+++ b/hw/misc/Kconfig
@@ -117,4 +117,8 @@ config AUX
 bool
 select I2C
 
+config MAC_VIA
+bool
+select MOS6522
+
 source macio/Kconfig
diff --git a/hw/misc/Makefile.objs b/hw/misc/Makefile.objs
index 77b9df9796..ef0d0d5d9b 100644
--- a/hw/misc/Makefile.objs
+++ b/hw/misc/Makefile.objs
@@ -75,6 +75,7 @@ obj-$(CONFIG_ARMSSE_MHU) += armsse-mhu.o
 obj-$(CONFIG_PVPANIC) += pvpanic.o
 obj-$(CONFIG_AUX) += auxbus.o
 obj-$(CONFIG_ASPEED_SOC) += aspeed_scu.o aspeed_sdmc.o
+obj-$(CONFIG_MAC_VIA) += mac_via.o
 obj-$(CONFIG_MSF2) += msf2-sysreg.o
 obj-$(CONFIG_NRF51_SOC) += nrf51_rng.o
 
diff --git a/hw/misc/mac_via.c b/hw/misc/mac_via.c
new file mode 100644
index 00..baa87c2879
--- /dev/null
+++ b/hw/misc/mac_via.c
@@ -0,0 +1,667 @@
+/*
+ * QEMU m68k Macintosh VIA device support
+ *
+ * Copyright (c) 2011-2018 Laurent Vivier
+ * Copyright (c) 2018 Mark Cave-Ayland
+ *
+ * Some parts from hw/misc/macio/cuda.c
+ *
+ * Copyright (c) 2004-2007 Fabrice Bellard
+ * Copyright (c) 2007 Jocelyn Mayer
+ *
+ * some parts from linux-2.6.29, arch/m68k/include/asm/mac_via.h
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu-common.h"
+#include "hw/sysbus.h"
+#include "qemu/timer.h"
+#include "hw/misc/mac_via.h"
+#include "hw/misc/mos6522.h"
+#include "hw/input/adb.h"
+#include "sysemu/sysemu.h"
+#include "qapi/error.h"
+#include "qemu/cutils.h"
+
+
+/*
+ * VIAs: There are two in every machine,
+ */
+
+#define VIA_SIZE (0x2000)
+
+/*
+ * Not all of these are true post MacII I think.
+ * CSA: probably the ones CHRP marks as 'unused' change purposes
+ * when the IWM becomes the SWIM.
+ * http://www.rs6000.ibm.com/resource/technology/chrpio/via5.mak.html
+ * ftp://ftp.austin.ibm.com/pub/technology/spec/chrp/inwork/CHRP_IORef_1.0.pdf
+ *
+ * also, http://developer.apple.com/technotes/hw/hw_09.html claims the
+ * following changes for IIfx:
+ * VIA1A_vSccWrReq not available and that VIA1A_vSync has moved to an IOP.
+ * Also, "All of the functionality of VIA2 has been moved to other chips".
+ */
+
+#define VIA1A_vSccWrReq 0x80   /* SCC write. (input)
+* [CHRP] SCC WREQ: Reflects the state of the
+* Wait/Request pins from the SCC.
+* [Macintosh Family Hardware]
+* as CHRP on SE/30,II,IIx,IIcx,IIci.
+* on IIfx, "0 means an active request"
+*/
+#define VIA1A_vRev8 0x40   /* Revision 8 board ???
+* [CHRP] En WaitReqB: Lets the WaitReq_L
+* signal from port B of the SCC appear on
+* the PA7 input pin. Output.
+* [Macintosh Family] On the SE/30, this
+* is the bit to flip screen buffers.
+* 0=alternate, 1=main.
+* on II,IIx,IIcx,IIci,IIfx this is a bit
+* for Rev ID. 0=II,IIx, 1=IIcx,IIci,IIfx
+*/
+#define VIA1A_vHeadSel  0x20   /* Head select for IWM.
+* [CHRP] unused.
+* [Macintosh Family] "Floppy disk
+* state-control line SEL" on all but IIfx
+*/
+#define VIA1A_vOverlay  0x10   /* [Macintosh Family] On SE/30,II,IIx,IIcx
+* this bit enables the "Overlay" address
+* map in the address decoders as it is on
+* reset for mapping the ROM over the reset
+* vector. 1=use overlay map.
+* On the IIci,IIfx it is another bit of the
+* CPU ID: 0=normal IIci, 1=IIci with parity
+* feature or IIfx.
+* [CHRP] En WaitReqA: Lets the WaitReq_L
+* signal from port A of the SCC appear
+* on the PA7 input pin (CHRP). Output.
+* [MkLinux] "Drive Select"
+*  (with 0x20 being 'disk head select')
+  

[Qemu-devel] [PATCH v8 09/10] hw/m68k: add a dummy SWIM floppy controller

[Laurent Vivier]

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
Reviewed-by: Hervé Poussineau 
---
 hw/block/Kconfig|   3 +
 hw/block/Makefile.objs  |   1 +
 hw/block/swim.c | 415 
 include/hw/block/swim.h |  76 
 4 files changed, 495 insertions(+)
 create mode 100644 hw/block/swim.c
 create mode 100644 include/hw/block/swim.h

diff --git a/hw/block/Kconfig b/hw/block/Kconfig
index df96dc5dcc..2d17f481ad 100644
--- a/hw/block/Kconfig
+++ b/hw/block/Kconfig
@@ -37,3 +37,6 @@ config VHOST_USER_BLK
 # Only PCI devices are provided for now
 default y if VIRTIO_PCI
 depends on VIRTIO && VHOST_USER && LINUX
+
+config SWIM
+bool
diff --git a/hw/block/Makefile.objs b/hw/block/Makefile.objs
index f5f643f0cc..28c2495a00 100644
--- a/hw/block/Makefile.objs
+++ b/hw/block/Makefile.objs
@@ -8,6 +8,7 @@ common-obj-$(CONFIG_XEN) += xen-block.o
 common-obj-$(CONFIG_ECC) += ecc.o
 common-obj-$(CONFIG_ONENAND) += onenand.o
 common-obj-$(CONFIG_NVME_PCI) += nvme.o
+common-obj-$(CONFIG_SWIM) += swim.o
 
 obj-$(CONFIG_SH4) += tc58128.o
 
diff --git a/hw/block/swim.c b/hw/block/swim.c
new file mode 100644
index 00..0f48e46d58
--- /dev/null
+++ b/hw/block/swim.c
@@ -0,0 +1,415 @@
+/*
+ * QEMU Macintosh floppy disk controller emulator (SWIM)
+ *
+ * Copyright (c) 2014-2018 Laurent Vivier 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.  See
+ * the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "sysemu/block-backend.h"
+#include "hw/sysbus.h"
+#include "hw/block/block.h"
+#include "hw/block/swim.h"
+
+/* IWM registers */
+
+#define IWM_PH0L0
+#define IWM_PH0H1
+#define IWM_PH1L2
+#define IWM_PH1H3
+#define IWM_PH2L4
+#define IWM_PH2H5
+#define IWM_PH3L6
+#define IWM_PH3H7
+#define IWM_MTROFF  8
+#define IWM_MTRON   9
+#define IWM_INTDRIVE10
+#define IWM_EXTDRIVE11
+#define IWM_Q6L 12
+#define IWM_Q6H 13
+#define IWM_Q7L 14
+#define IWM_Q7H 15
+
+/* SWIM registers */
+
+#define SWIM_WRITE_DATA 0
+#define SWIM_WRITE_MARK 1
+#define SWIM_WRITE_CRC  2
+#define SWIM_WRITE_PARAMETER3
+#define SWIM_WRITE_PHASE4
+#define SWIM_WRITE_SETUP5
+#define SWIM_WRITE_MODE06
+#define SWIM_WRITE_MODE17
+
+#define SWIM_READ_DATA  8
+#define SWIM_READ_MARK  9
+#define SWIM_READ_ERROR 10
+#define SWIM_READ_PARAMETER 11
+#define SWIM_READ_PHASE 12
+#define SWIM_READ_SETUP 13
+#define SWIM_READ_STATUS14
+#define SWIM_READ_HANDSHAKE 15
+
+#define REG_SHIFT   9
+
+#define SWIM_MODE_IWM  0
+#define SWIM_MODE_SWIM 1
+
+/* bits in phase register */
+
+#define SWIM_SEEK_NEGATIVE   0x074
+#define SWIM_STEP0x071
+#define SWIM_MOTOR_ON0x072
+#define SWIM_MOTOR_OFF   0x076
+#define SWIM_INDEX   0x073
+#define SWIM_EJECT   0x077
+#define SWIM_SETMFM  0x171
+#define SWIM_SETGCR  0x175
+#define SWIM_RELAX   0x033
+#define SWIM_LSTRB   0x008
+#define SWIM_CA_MASK 0x077
+
+/* Select values for swim_select and swim_readbit */
+
+#define SWIM_READ_DATA_0 0x074
+#define SWIM_TWOMEG_DRIVE0x075
+#define SWIM_SINGLE_SIDED0x076
+#define SWIM_DRIVE_PRESENT   0x077
+#define SWIM_DISK_IN 0x170
+#define SWIM_WRITE_PROT  0x171
+#define SWIM_TRACK_ZERO  0x172
+#define SWIM_TACHO   0x173
+#define SWIM_READ_DATA_1 0x174
+#define SWIM_MFM_MODE0x175
+#define SWIM_SEEK_COMPLETE   0x176
+#define SWIM_ONEMEG_MEDIA0x177
+
+/* Bits in handshake register */
+
+#define SWIM_MARK_BYTE   0x01
+#define SWIM_CRC_ZERO0x02
+#define SWIM_RDDATA  0x04
+#define SWIM_SENSE   0x08
+#define SWIM_MOTEN   0x10
+#define SWIM_ERROR   0x20
+#define SWIM_DAT2BYTE0x40
+#define SWIM_DAT1BYTE0x80
+
+/* bits in setup register */
+
+#define SWIM_S_INV_WDATA 0x01
+#define SWIM_S_3_5_SELECT0x02
+#define SWIM_S_GCR   0x04
+#define SWIM_S_FCLK_DIV2 0x08
+#define SWIM_S_ERROR_CORR0x10
+#define SWIM_S_IBM_DRIVE 0x20
+#define SWIM_S_GCR_WRITE 0x40
+#define SWIM_S_TIMEOUT   0x80
+
+/* bits in mode register */
+
+#define SWIM_CLFIFO  0x01
+#define SWIM_ENBL1   0x02
+#define SWIM_ENBL2   0x04
+#define SWIM_ACTION  0x08
+#define SWIM_WRITE_MODE  0x10
+#define SWIM_HEDSEL  0x20
+#define SWIM_MOTON   0x80
+
+static void swim_change_cb(void *opaque, bool load, Error **errp)
+{
+FDrive *drive = opaque;
+
+if (!load) {
+blk_set_perm(drive->blk, 0, BLK_PERM_ALL, 

[Qemu-devel] [PATCH v8 06/10] hw/m68k: add macfb video card

[Laurent Vivier]

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
Reviewed-by: Hervé Poussineau 
Reviewed-by: Thomas Huth 
---
 arch_init.c|   4 +
 hw/display/Kconfig |   3 +
 hw/display/Makefile.objs   |   1 +
 hw/display/macfb.c | 419 +
 include/hw/display/macfb.h |  43 
 qemu-options.hx|   2 +-
 vl.c   |   3 +-
 7 files changed, 473 insertions(+), 2 deletions(-)
 create mode 100644 hw/display/macfb.c
 create mode 100644 include/hw/display/macfb.h

diff --git a/arch_init.c b/arch_init.c
index 74b0708634..9e04cfb581 100644
--- a/arch_init.c
+++ b/arch_init.c
@@ -38,6 +38,10 @@
 int graphic_width = 1024;
 int graphic_height = 768;
 int graphic_depth = 8;
+#elif defined(TARGET_M68K)
+int graphic_width = 800;
+int graphic_height = 600;
+int graphic_depth = 8;
 #else
 int graphic_width = 800;
 int graphic_height = 600;
diff --git a/hw/display/Kconfig b/hw/display/Kconfig
index 910dccb2f7..035cd85bcf 100644
--- a/hw/display/Kconfig
+++ b/hw/display/Kconfig
@@ -130,3 +130,6 @@ config ATI_VGA
 default y if PCI_DEVICES
 depends on PCI
 select VGA
+
+config MACFB
+bool
diff --git a/hw/display/Makefile.objs b/hw/display/Makefile.objs
index a64998fc7b..2a26709e9e 100644
--- a/hw/display/Makefile.objs
+++ b/hw/display/Makefile.objs
@@ -26,6 +26,7 @@ common-obj-$(CONFIG_EXYNOS4) += exynos4210_fimd.o
 common-obj-$(CONFIG_FRAMEBUFFER) += framebuffer.o
 obj-$(CONFIG_MILKYMIST) += milkymist-vgafb.o
 common-obj-$(CONFIG_ZAURUS) += tc6393xb.o
+common-obj-$(CONFIG_MACFB) += macfb.o
 
 obj-$(CONFIG_MILKYMIST_TMU2) += milkymist-tmu2.o
 milkymist-tmu2.o-cflags := $(X11_CFLAGS) $(OPENGL_CFLAGS)
diff --git a/hw/display/macfb.c b/hw/display/macfb.c
new file mode 100644
index 00..61ba2e9e15
--- /dev/null
+++ b/hw/display/macfb.c
@@ -0,0 +1,419 @@
+/*
+ * QEMU Motorola 680x0 Macintosh Video Card Emulation
+ * Copyright (c) 2012-2018 Laurent Vivier
+ *
+ * some parts from QEMU G364 framebuffer Emulator.
+ * Copyright (c) 2007-2011 Herve Poussineau
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/units.h"
+#include "hw/sysbus.h"
+#include "ui/console.h"
+#include "ui/pixel_ops.h"
+#include "hw/display/macfb.h"
+#include "qapi/error.h"
+
+#define VIDEO_BASE 0x1000
+#define DAFB_BASE  0x0080
+
+#define MACFB_PAGE_SIZE 4096
+#define MACFB_VRAM_SIZE (4 * MiB)
+
+#define DAFB_RESET  0x200
+#define DAFB_LUT0x213
+
+
+typedef void macfb_draw_line_func(MacfbState *s, uint8_t *d, uint32_t addr,
+  int width);
+
+static inline uint8_t macfb_read_byte(MacfbState *s, uint32_t addr)
+{
+return s->vram[addr & s->vram_bit_mask];
+}
+
+/* 1-bit color */
+static void macfb_draw_line1(MacfbState *s, uint8_t *d, uint32_t addr,
+ int width)
+{
+uint8_t r, g, b;
+int x;
+
+for (x = 0; x < width; x++) {
+int bit = x & 7;
+int idx = (macfb_read_byte(s, addr) >> (7 - bit)) & 1;
+r = g = b  = ((1 - idx) << 7);
+addr += (bit == 7);
+
+*(uint32_t *)d = rgb_to_pixel32(r, g, b);
+d += 4;
+}
+}
+
+/* 2-bit color */
+static void macfb_draw_line2(MacfbState *s, uint8_t *d, uint32_t addr,
+ int width)
+{
+uint8_t r, g, b;
+int x;
+
+for (x = 0; x < width; x++) {
+int bit = (x & 3);
+int idx = (macfb_read_byte(s, addr) >> ((3 - bit) << 1)) & 3;
+r = s->color_palette[idx * 3];
+g = s->color_palette[idx * 3 + 1];
+b = s->color_palette[idx * 3 + 2];
+addr += (bit == 3);
+
+*(uint32_t *)d = rgb_to_pixel32(r, g, b);
+d += 4;
+}
+}
+
+/* 4-bit color */
+static void macfb_draw_line4(MacfbState *s, uint8_t *d, uint32_t addr,
+ int width)
+{
+uint8_t r, g, b;
+int x;
+
+for (x = 0; x < width; x++) {
+int bit = x & 1;
+int idx = (macfb_read_byte(s, addr) >> ((1 - bit) << 2)) & 15;
+r = s->color_palette[idx * 3];
+g = s->color_palette[idx * 3 + 1];
+b = s->color_palette[idx * 3 + 2];
+addr += (bit == 1);
+
+*(uint32_t *)d = rgb_to_pixel32(r, g, b);
+d += 4;
+}
+}
+
+/* 8-bit color */
+static void macfb_draw_line8(MacfbState *s, uint8_t *d, uint32_t addr,
+ int width)
+{
+uint8_t r, g, b;
+int x;
+
+for (x = 0; x < width; x++) {
+r = s->color_palette[macfb_read_byte(s, addr) * 3];
+g = s->color_palette[macfb_read_byte(s, addr) * 3 + 1];
+b = s->color_palette[macfb_read_byte(s, addr) * 3 + 2];
+addr++;
+
+*(uint32_t *)d = rgb_to_pixel32(r, g, b);
+d += 4;
+}
+}
+
+/* 16-bit color */
+static void 

[Qemu-devel] [PATCH v8 03/10] dp8393x: manage big endian bus

[Laurent Vivier]

This is needed by Quadra 800, this card can run on little-endian
or big-endian bus.

Signed-off-by: Laurent Vivier 
Tested-by: Hervé Poussineau 
Reviewed-by: Philippe Mathieu-Daudé 
Reviewed-by: Hervé Poussineau 
---
 hw/net/dp8393x.c | 88 +++-
 1 file changed, 57 insertions(+), 31 deletions(-)

diff --git a/hw/net/dp8393x.c b/hw/net/dp8393x.c
index bdb0b3b2c2..b014c015c6 100644
--- a/hw/net/dp8393x.c
+++ b/hw/net/dp8393x.c
@@ -150,6 +150,7 @@ typedef struct dp8393xState {
 
 /* Hardware */
 uint8_t it_shift;
+bool big_endian;
 qemu_irq irq;
 #ifdef DEBUG_SONIC
 int irq_level;
@@ -220,6 +221,29 @@ static uint32_t dp8393x_wt(dp8393xState *s)
 return s->regs[SONIC_WT1] << 16 | s->regs[SONIC_WT0];
 }
 
+static uint16_t dp8393x_get(dp8393xState *s, int width, uint16_t *base,
+int offset)
+{
+uint16_t val;
+
+if (s->big_endian) {
+val = be16_to_cpu(base[offset * width + width - 1]);
+} else {
+val = le16_to_cpu(base[offset * width]);
+}
+return val;
+}
+
+static void dp8393x_put(dp8393xState *s, int width, uint16_t *base, int offset,
+uint16_t val)
+{
+if (s->big_endian) {
+base[offset * width + width - 1] = cpu_to_be16(val);
+} else {
+base[offset * width] = cpu_to_le16(val);
+}
+}
+
 static void dp8393x_update_irq(dp8393xState *s)
 {
 int level = (s->regs[SONIC_IMR] & s->regs[SONIC_ISR]) ? 1 : 0;
@@ -251,12 +275,12 @@ static void dp8393x_do_load_cam(dp8393xState *s)
 /* Fill current entry */
 address_space_rw(>as, dp8393x_cdp(s),
 MEMTXATTRS_UNSPECIFIED, (uint8_t *)data, size, 0);
-s->cam[index][0] = data[1 * width] & 0xff;
-s->cam[index][1] = data[1 * width] >> 8;
-s->cam[index][2] = data[2 * width] & 0xff;
-s->cam[index][3] = data[2 * width] >> 8;
-s->cam[index][4] = data[3 * width] & 0xff;
-s->cam[index][5] = data[3 * width] >> 8;
+s->cam[index][0] = dp8393x_get(s, width, data, 1) & 0xff;
+s->cam[index][1] = dp8393x_get(s, width, data, 1) >> 8;
+s->cam[index][2] = dp8393x_get(s, width, data, 2) & 0xff;
+s->cam[index][3] = dp8393x_get(s, width, data, 2) >> 8;
+s->cam[index][4] = dp8393x_get(s, width, data, 3) & 0xff;
+s->cam[index][5] = dp8393x_get(s, width, data, 3) >> 8;
 DPRINTF("load cam[%d] with %02x%02x%02x%02x%02x%02x\n", index,
 s->cam[index][0], s->cam[index][1], s->cam[index][2],
 s->cam[index][3], s->cam[index][4], s->cam[index][5]);
@@ -269,7 +293,7 @@ static void dp8393x_do_load_cam(dp8393xState *s)
 /* Read CAM enable */
 address_space_rw(>as, dp8393x_cdp(s),
 MEMTXATTRS_UNSPECIFIED, (uint8_t *)data, size, 0);
-s->regs[SONIC_CE] = data[0 * width];
+s->regs[SONIC_CE] = dp8393x_get(s, width, data, 0);
 DPRINTF("load cam done. cam enable mask 0x%04x\n", s->regs[SONIC_CE]);
 
 /* Done */
@@ -290,10 +314,10 @@ static void dp8393x_do_read_rra(dp8393xState *s)
 MEMTXATTRS_UNSPECIFIED, (uint8_t *)data, size, 0);
 
 /* Update SONIC registers */
-s->regs[SONIC_CRBA0] = data[0 * width];
-s->regs[SONIC_CRBA1] = data[1 * width];
-s->regs[SONIC_RBWC0] = data[2 * width];
-s->regs[SONIC_RBWC1] = data[3 * width];
+s->regs[SONIC_CRBA0] = dp8393x_get(s, width, data, 0);
+s->regs[SONIC_CRBA1] = dp8393x_get(s, width, data, 1);
+s->regs[SONIC_RBWC0] = dp8393x_get(s, width, data, 2);
+s->regs[SONIC_RBWC1] = dp8393x_get(s, width, data, 3);
 DPRINTF("CRBA0/1: 0x%04x/0x%04x, RBWC0/1: 0x%04x/0x%04x\n",
 s->regs[SONIC_CRBA0], s->regs[SONIC_CRBA1],
 s->regs[SONIC_RBWC0], s->regs[SONIC_RBWC1]);
@@ -408,12 +432,12 @@ static void dp8393x_do_transmit_packets(dp8393xState *s)
 tx_len = 0;
 
 /* Update registers */
-s->regs[SONIC_TCR] = data[0 * width] & 0xf000;
-s->regs[SONIC_TPS] = data[1 * width];
-s->regs[SONIC_TFC] = data[2 * width];
-s->regs[SONIC_TSA0] = data[3 * width];
-s->regs[SONIC_TSA1] = data[4 * width];
-s->regs[SONIC_TFS] = data[5 * width];
+s->regs[SONIC_TCR] = dp8393x_get(s, width, data, 0) & 0xf000;
+s->regs[SONIC_TPS] = dp8393x_get(s, width, data, 1);
+s->regs[SONIC_TFC] = dp8393x_get(s, width, data, 2);
+s->regs[SONIC_TSA0] = dp8393x_get(s, width, data, 3);
+s->regs[SONIC_TSA1] = dp8393x_get(s, width, data, 4);
+s->regs[SONIC_TFS] = dp8393x_get(s, width, data, 5);
 
 /* Handle programmable interrupt */
 if (s->regs[SONIC_TCR] & SONIC_TCR_PINT) {
@@ -439,9 +463,9 @@ static void dp8393x_do_transmit_packets(dp8393xState *s)
 address_space_rw(>as,
 dp8393x_ttda(s) + sizeof(uint16_t) * (4 + 3 * i) * width,
 MEMTXATTRS_UNSPECIFIED, (uint8_t *)data, size, 0);
-s->regs[SONIC_TSA0] = 

[Qemu-devel] [PATCH v8 02/10] esp: add pseudo-DMA as used by Macintosh

[Laurent Vivier]

There is no DMA in Quadra 800, so the CPU reads/writes the data from the
PDMA register (offset 0x100, ESP_PDMA in hw/m68k/q800.c) and copies them
to/from the memory.

There is a nice assembly loop in the kernel to do that, see
linux/drivers/scsi/mac_esp.c:MAC_ESP_PDMA_LOOP().

The start of the transfer is triggered by the DREQ interrupt (see linux
mac_esp_send_pdma_cmd()), the CPU polls on the IRQ flag to start the
transfer after a SCSI command has been sent (in Quadra 800 it goes
through the VIA2, the via2-irq line and the vIFR register)

The Macintosh hardware includes hardware handshaking to prevent the CPU
from reading invalid data or writing data faster than the peripheral
device can accept it.

This is the "blind mode", and from the doc:
"Approximate maximum SCSI transfer rates within a blocks are 1.4 MB per
second for blind transfers in the Macintosh II"

Some references can be found in:
  Apple Macintosh Family Hardware Reference, ISBN 0-201-19255-1
  Guide to the Macintosh Family Hardware, ISBN-0-201-52405-8

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
---
 hw/scsi/esp.c | 296 +-
 include/hw/scsi/esp.h |   7 +
 2 files changed, 274 insertions(+), 29 deletions(-)

diff --git a/hw/scsi/esp.c b/hw/scsi/esp.c
index 7508d035ca..edacb0bd9f 100644
--- a/hw/scsi/esp.c
+++ b/hw/scsi/esp.c
@@ -36,6 +36,8 @@
  * 
http://www.ibiblio.org/pub/historic-linux/early-ports/Sparc/NCR/NCR89C100.txt
  * and
  * http://www.ibiblio.org/pub/historic-linux/early-ports/Sparc/NCR/NCR53C9X.txt
+ *
+ * On Macintosh Quadra it is a NCR53C96.
  */
 
 static void esp_raise_irq(ESPState *s)
@@ -56,6 +58,16 @@ static void esp_lower_irq(ESPState *s)
 }
 }
 
+static void esp_raise_drq(ESPState *s)
+{
+qemu_irq_raise(s->irq_data);
+}
+
+static void esp_lower_drq(ESPState *s)
+{
+qemu_irq_lower(s->irq_data);
+}
+
 void esp_dma_enable(ESPState *s, int irq, int level)
 {
 if (level) {
@@ -82,29 +94,11 @@ void esp_request_cancelled(SCSIRequest *req)
 }
 }
 
-static uint32_t get_cmd(ESPState *s, uint8_t *buf, uint8_t buflen)
+static int get_cmd_cb(ESPState *s)
 {
-uint32_t dmalen;
 int target;
 
 target = s->wregs[ESP_WBUSID] & BUSID_DID;
-if (s->dma) {
-dmalen = s->rregs[ESP_TCLO];
-dmalen |= s->rregs[ESP_TCMID] << 8;
-dmalen |= s->rregs[ESP_TCHI] << 16;
-if (dmalen > buflen) {
-return 0;
-}
-s->dma_memory_read(s->dma_opaque, buf, dmalen);
-} else {
-dmalen = s->ti_size;
-if (dmalen > TI_BUFSZ) {
-return 0;
-}
-memcpy(buf, s->ti_buf, dmalen);
-buf[0] = buf[2] >> 5;
-}
-trace_esp_get_cmd(dmalen, target);
 
 s->ti_size = 0;
 s->ti_rptr = 0;
@@ -123,8 +117,48 @@ static uint32_t get_cmd(ESPState *s, uint8_t *buf, uint8_t 
buflen)
 s->rregs[ESP_RINTR] = INTR_DC;
 s->rregs[ESP_RSEQ] = SEQ_0;
 esp_raise_irq(s);
+return -1;
+}
+return 0;
+}
+
+static uint32_t get_cmd(ESPState *s, uint8_t *buf, uint8_t buflen)
+{
+int target;
+uint32_t dmalen;
+
+target = s->wregs[ESP_WBUSID] & BUSID_DID;
+if (s->dma) {
+dmalen = s->rregs[ESP_TCLO];
+dmalen |= s->rregs[ESP_TCMID] << 8;
+dmalen |= s->rregs[ESP_TCHI] << 16;
+if (dmalen > buflen) {
+return 0;
+}
+if (s->dma_memory_read) {
+s->dma_memory_read(s->dma_opaque, buf, dmalen);
+} else {
+memcpy(s->pdma_buf, buf, dmalen);
+s->pdma_len = dmalen;
+s->pdma_start = s->pdma_buf;
+s->pdma_cur = s->pdma_buf;
+esp_raise_drq(s);
+return 0;
+}
+} else {
+dmalen = s->ti_size;
+if (dmalen > TI_BUFSZ) {
+return 0;
+}
+memcpy(buf, s->ti_buf, dmalen);
+buf[0] = buf[2] >> 5;
+}
+trace_esp_get_cmd(dmalen, target);
+
+if (get_cmd_cb(s) < 0) {
 return 0;
 }
+
 return dmalen;
 }
 
@@ -163,6 +197,16 @@ static void do_cmd(ESPState *s, uint8_t *buf)
 do_busid_cmd(s, [1], busid);
 }
 
+static void satn_pdma_cb(ESPState *s)
+{
+if (get_cmd_cb(s) < 0) {
+return;
+}
+if (s->pdma_cur != s->pdma_start) {
+do_cmd(s, s->pdma_start);
+}
+}
+
 static void handle_satn(ESPState *s)
 {
 uint8_t buf[32];
@@ -172,11 +216,22 @@ static void handle_satn(ESPState *s)
 s->dma_cb = handle_satn;
 return;
 }
+s->pdma_cb = satn_pdma_cb;
 len = get_cmd(s, buf, sizeof(buf));
 if (len)
 do_cmd(s, buf);
 }
 
+static void s_without_satn_pdma_cb(ESPState *s)
+{
+if (get_cmd_cb(s) < 0) {
+return;
+}
+if (s->pdma_cur != s->pdma_start) {
+do_busid_cmd(s, s->pdma_start, 0);
+}
+}
+
 static void handle_s_without_atn(ESPState *s)
 {
 uint8_t buf[32];
@@ -186,18 +241,36 @@ static void 

[Qemu-devel] [PATCH v8 05/10] hw/m68k: implement ADB bus support for via

[Laurent Vivier]

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
Reviewed-by: Hervé Poussineau 
Reviewed-by: Thomas Huth 
---
 hw/misc/mac_via.c | 190 ++
 include/hw/misc/mac_via.h |   7 ++
 2 files changed, 197 insertions(+)

diff --git a/hw/misc/mac_via.c b/hw/misc/mac_via.c
index baa87c2879..4249632a57 100644
--- a/hw/misc/mac_via.c
+++ b/hw/misc/mac_via.c
@@ -238,10 +238,16 @@
  * Table 19-10 ADB transaction states
  */
 
+#define ADB_STATE_NEW   0
+#define ADB_STATE_EVEN  1
+#define ADB_STATE_ODD   2
+#define ADB_STATE_IDLE  3
+
 #define VIA1B_vADB_StateMask(VIA1B_vADBS1 | VIA1B_vADBS2)
 #define VIA1B_vADB_StateShift   4
 
 #define VIA_TIMER_FREQ (783360)
+#define VIA_ADB_POLL_FREQ 50 /* XXX: not real */
 
 /* VIA returns time offset from Jan 1, 1904, not 1970 */
 #define RTC_OFFSET 2082844800
@@ -423,6 +429,181 @@ static void via1_rtc_update(MacVIAState *m)
 }
 }
 
+static int adb_via_poll(MacVIAState *s, int state, uint8_t *data)
+{
+if (state != ADB_STATE_IDLE) {
+return 0;
+}
+
+if (s->adb_data_in_size < s->adb_data_in_index) {
+return 0;
+}
+
+if (s->adb_data_out_index != 0) {
+return 0;
+}
+
+s->adb_data_in_index = 0;
+s->adb_data_out_index = 0;
+s->adb_data_in_size = adb_poll(>adb_bus, s->adb_data_in, 0x);
+
+if (s->adb_data_in_size) {
+*data = s->adb_data_in[s->adb_data_in_index++];
+qemu_irq_raise(s->adb_data_ready);
+}
+
+return s->adb_data_in_size;
+}
+
+static int adb_via_send(MacVIAState *s, int state, uint8_t data)
+{
+switch (state) {
+case ADB_STATE_NEW:
+s->adb_data_out_index = 0;
+break;
+case ADB_STATE_EVEN:
+if ((s->adb_data_out_index & 1) == 0) {
+return 0;
+}
+break;
+case ADB_STATE_ODD:
+if (s->adb_data_out_index & 1) {
+return 0;
+}
+break;
+case ADB_STATE_IDLE:
+return 0;
+}
+
+assert(s->adb_data_out_index < sizeof(s->adb_data_out) - 1);
+
+s->adb_data_out[s->adb_data_out_index++] = data;
+qemu_irq_raise(s->adb_data_ready);
+return 1;
+}
+
+static int adb_via_receive(MacVIAState *s, int state, uint8_t *data)
+{
+switch (state) {
+case ADB_STATE_NEW:
+return 0;
+
+case ADB_STATE_EVEN:
+if (s->adb_data_in_size <= 0) {
+qemu_irq_raise(s->adb_data_ready);
+return 0;
+}
+
+if (s->adb_data_in_index >= s->adb_data_in_size) {
+*data = 0;
+qemu_irq_raise(s->adb_data_ready);
+return 1;
+}
+
+if ((s->adb_data_in_index & 1) == 0) {
+return 0;
+}
+
+break;
+
+case ADB_STATE_ODD:
+if (s->adb_data_in_size <= 0) {
+qemu_irq_raise(s->adb_data_ready);
+return 0;
+}
+
+if (s->adb_data_in_index >= s->adb_data_in_size) {
+*data = 0;
+qemu_irq_raise(s->adb_data_ready);
+return 1;
+}
+
+if (s->adb_data_in_index & 1) {
+return 0;
+}
+
+break;
+
+case ADB_STATE_IDLE:
+if (s->adb_data_out_index == 0) {
+return 0;
+}
+
+s->adb_data_in_size = adb_request(>adb_bus, s->adb_data_in,
+  s->adb_data_out,
+  s->adb_data_out_index);
+s->adb_data_out_index = 0;
+s->adb_data_in_index = 0;
+if (s->adb_data_in_size < 0) {
+*data = 0xff;
+qemu_irq_raise(s->adb_data_ready);
+return -1;
+}
+
+if (s->adb_data_in_size == 0) {
+return 0;
+}
+
+break;
+}
+
+assert(s->adb_data_in_index < sizeof(s->adb_data_in) - 1);
+
+*data = s->adb_data_in[s->adb_data_in_index++];
+qemu_irq_raise(s->adb_data_ready);
+if (*data == 0xff || *data == 0) {
+return 0;
+}
+return 1;
+}
+
+static void via1_adb_update(MacVIAState *m)
+{
+MOS6522Q800VIA1State *v1s = MOS6522_Q800_VIA1(>mos6522_via1);
+MOS6522State *s = MOS6522(v1s);
+int state;
+int ret;
+
+state = (s->b & VIA1B_vADB_StateMask) >> VIA1B_vADB_StateShift;
+
+if (s->acr & VIA1ACR_vShiftOut) {
+/* output mode */
+ret = adb_via_send(m, state, s->sr);
+if (ret > 0) {
+s->b &= ~VIA1B_vADBInt;
+} else {
+s->b |= VIA1B_vADBInt;
+}
+} else {
+/* input mode */
+ret = adb_via_receive(m, state, >sr);
+if (ret > 0 && s->sr != 0xff) {
+s->b &= ~VIA1B_vADBInt;
+} else {
+s->b |= VIA1B_vADBInt;
+}
+}
+}
+
+static void via_adb_poll(void *opaque)
+{
+MacVIAState *m = opaque;
+MOS6522Q800VIA1State *v1s = MOS6522_Q800_VIA1(>mos6522_via1);
+MOS6522State *s = MOS6522(v1s);
+int 

[Qemu-devel] [PATCH v8 08/10] hw/m68k: add Nubus support for macfb video card

[Laurent Vivier]

From: Mark Cave-Ayland 

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
Reviewed-by: Hervé Poussineau 
---
 hw/display/Kconfig |  1 +
 hw/display/macfb.c | 56 ++
 include/hw/display/macfb.h | 21 ++
 3 files changed, 78 insertions(+)

diff --git a/hw/display/Kconfig b/hw/display/Kconfig
index 035cd85bcf..95096d3c82 100644
--- a/hw/display/Kconfig
+++ b/hw/display/Kconfig
@@ -133,3 +133,4 @@ config ATI_VGA
 
 config MACFB
 bool
+depends on NUBUS
diff --git a/hw/display/macfb.c b/hw/display/macfb.c
index 61ba2e9e15..458e268d86 100644
--- a/hw/display/macfb.c
+++ b/hw/display/macfb.c
@@ -15,6 +15,7 @@
 #include "hw/sysbus.h"
 #include "ui/console.h"
 #include "ui/pixel_ops.h"
+#include "hw/nubus/nubus.h"
 #include "hw/display/macfb.h"
 #include "qapi/error.h"
 
@@ -380,12 +381,38 @@ static void macfb_sysbus_realize(DeviceState *dev, Error 
**errp)
 sysbus_init_mmio(SYS_BUS_DEVICE(s), >mem_vram);
 }
 
+const uint8_t macfb_rom[] = {
+255, 0, 0, 0,
+};
+
+static void macfb_nubus_realize(DeviceState *dev, Error **errp)
+{
+NubusDevice *nd = NUBUS_DEVICE(dev);
+MacfbNubusState *s = NUBUS_MACFB(dev);
+MacfbNubusDeviceClass *ndc = MACFB_NUBUS_GET_CLASS(dev);
+MacfbState *ms = >macfb;
+
+ndc->parent_realize(dev, errp);
+
+macfb_common_realize(dev, ms, errp);
+memory_region_add_subregion(>slot_mem, DAFB_BASE, >mem_ctrl);
+memory_region_add_subregion(>slot_mem, VIDEO_BASE, >mem_vram);
+
+nubus_register_rom(nd, macfb_rom, sizeof(macfb_rom), 1, 9, 0xf);
+}
+
 static void macfb_sysbus_reset(DeviceState *d)
 {
 MacfbSysBusState *s = MACFB(d);
 macfb_reset(>macfb);
 }
 
+static void macfb_nubus_reset(DeviceState *d)
+{
+MacfbNubusState *s = NUBUS_MACFB(d);
+macfb_reset(>macfb);
+}
+
 static Property macfb_sysbus_properties[] = {
 DEFINE_PROP_UINT32("width", MacfbSysBusState, macfb.width, 640),
 DEFINE_PROP_UINT32("height", MacfbSysBusState, macfb.height, 480),
@@ -393,6 +420,13 @@ static Property macfb_sysbus_properties[] = {
 DEFINE_PROP_END_OF_LIST(),
 };
 
+static Property macfb_nubus_properties[] = {
+DEFINE_PROP_UINT32("width", MacfbNubusState, macfb.width, 640),
+DEFINE_PROP_UINT32("height", MacfbNubusState, macfb.height, 480),
+DEFINE_PROP_UINT8("depth", MacfbNubusState, macfb.depth, 8),
+DEFINE_PROP_END_OF_LIST(),
+};
+
 static void macfb_sysbus_class_init(ObjectClass *klass, void *data)
 {
 DeviceClass *dc = DEVICE_CLASS(klass);
@@ -404,6 +438,19 @@ static void macfb_sysbus_class_init(ObjectClass *klass, 
void *data)
 dc->props = macfb_sysbus_properties;
 }
 
+static void macfb_nubus_class_init(ObjectClass *klass, void *data)
+{
+DeviceClass *dc = DEVICE_CLASS(klass);
+MacfbNubusDeviceClass *ndc = MACFB_NUBUS_DEVICE_CLASS(klass);
+
+device_class_set_parent_realize(dc, macfb_nubus_realize,
+>parent_realize);
+dc->desc = "Nubus Macintosh framebuffer";
+dc->reset = macfb_nubus_reset;
+dc->vmsd = _macfb;
+dc->props = macfb_nubus_properties;
+}
+
 static TypeInfo macfb_sysbus_info = {
 .name  = TYPE_MACFB,
 .parent= TYPE_SYS_BUS_DEVICE,
@@ -411,9 +458,18 @@ static TypeInfo macfb_sysbus_info = {
 .class_init= macfb_sysbus_class_init,
 };
 
+static TypeInfo macfb_nubus_info = {
+.name  = TYPE_NUBUS_MACFB,
+.parent= TYPE_NUBUS_DEVICE,
+.instance_size = sizeof(MacfbNubusState),
+.class_init= macfb_nubus_class_init,
+.class_size= sizeof(MacfbNubusDeviceClass),
+};
+
 static void macfb_register_types(void)
 {
 type_register_static(_sysbus_info);
+type_register_static(_nubus_info);
 }
 
 type_init(macfb_register_types)
diff --git a/include/hw/display/macfb.h b/include/hw/display/macfb.h
index 3fe2592735..26367ae2c4 100644
--- a/include/hw/display/macfb.h
+++ b/include/hw/display/macfb.h
@@ -40,4 +40,25 @@ typedef struct {
 MacfbState macfb;
 } MacfbSysBusState;
 
+#define MACFB_NUBUS_DEVICE_CLASS(class) \
+OBJECT_CLASS_CHECK(MacfbNubusDeviceClass, (class), TYPE_NUBUS_MACFB)
+#define MACFB_NUBUS_GET_CLASS(obj) \
+OBJECT_GET_CLASS(MacfbNubusDeviceClass, (obj), TYPE_NUBUS_MACFB)
+
+typedef struct MacfbNubusDeviceClass {
+DeviceClass parent_class;
+
+DeviceRealize parent_realize;
+} MacfbNubusDeviceClass;
+
+#define TYPE_NUBUS_MACFB "nubus-macfb"
+#define NUBUS_MACFB(obj) \
+OBJECT_CHECK(MacfbNubusState, (obj), TYPE_NUBUS_MACFB)
+
+typedef struct {
+NubusDevice busdev;
+
+MacfbState macfb;
+} MacfbNubusState;
+
 #endif
-- 
2.21.0

[Qemu-devel] [PATCH v8 07/10] hw/m68k: add Nubus support

[Laurent Vivier]

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
Reviewed-by: Thomas Huth 
---
 hw/Kconfig  |   1 +
 hw/Makefile.objs|   1 +
 hw/nubus/Kconfig|   2 +
 hw/nubus/Makefile.objs  |   4 +
 hw/nubus/mac-nubus-bridge.c |  45 ++
 hw/nubus/nubus-bridge.c |  34 +
 hw/nubus/nubus-bus.c| 111 ++
 hw/nubus/nubus-device.c | 215 
 include/hw/nubus/mac-nubus-bridge.h |  24 
 include/hw/nubus/nubus.h|  69 +
 10 files changed, 506 insertions(+)
 create mode 100644 hw/nubus/Kconfig
 create mode 100644 hw/nubus/Makefile.objs
 create mode 100644 hw/nubus/mac-nubus-bridge.c
 create mode 100644 hw/nubus/nubus-bridge.c
 create mode 100644 hw/nubus/nubus-bus.c
 create mode 100644 hw/nubus/nubus-device.c
 create mode 100644 include/hw/nubus/mac-nubus-bridge.h
 create mode 100644 include/hw/nubus/nubus.h

diff --git a/hw/Kconfig b/hw/Kconfig
index 195f541e50..c34e0aa99a 100644
--- a/hw/Kconfig
+++ b/hw/Kconfig
@@ -21,6 +21,7 @@ source isa/Kconfig
 source mem/Kconfig
 source misc/Kconfig
 source net/Kconfig
+source nubus/Kconfig
 source nvram/Kconfig
 source pci-bridge/Kconfig
 source pci-host/Kconfig
diff --git a/hw/Makefile.objs b/hw/Makefile.objs
index d770926ba9..686808dd17 100644
--- a/hw/Makefile.objs
+++ b/hw/Makefile.objs
@@ -36,6 +36,7 @@ devices-dirs-$(CONFIG_SOFTMMU) += watchdog/
 devices-dirs-$(CONFIG_SOFTMMU) += xen/
 devices-dirs-$(CONFIG_MEM_DEVICE) += mem/
 devices-dirs-$(CONFIG_SOFTMMU) += smbios/
+devices-dirs-$(CONFIG_NUBUS) += nubus/
 devices-dirs-y += semihosting/
 devices-dirs-y += core/
 common-obj-y += $(devices-dirs-y)
diff --git a/hw/nubus/Kconfig b/hw/nubus/Kconfig
new file mode 100644
index 00..8fb8b22189
--- /dev/null
+++ b/hw/nubus/Kconfig
@@ -0,0 +1,2 @@
+config NUBUS
+bool
diff --git a/hw/nubus/Makefile.objs b/hw/nubus/Makefile.objs
new file mode 100644
index 00..135ba7878d
--- /dev/null
+++ b/hw/nubus/Makefile.objs
@@ -0,0 +1,4 @@
+common-obj-y += nubus-device.o
+common-obj-y += nubus-bus.o
+common-obj-y += nubus-bridge.o
+common-obj-$(CONFIG_Q800) += mac-nubus-bridge.o
diff --git a/hw/nubus/mac-nubus-bridge.c b/hw/nubus/mac-nubus-bridge.c
new file mode 100644
index 00..7c329300b8
--- /dev/null
+++ b/hw/nubus/mac-nubus-bridge.c
@@ -0,0 +1,45 @@
+/*
+ *  Copyright (c) 2013-2018 Laurent Vivier 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "hw/sysbus.h"
+#include "hw/nubus/mac-nubus-bridge.h"
+
+
+static void mac_nubus_bridge_init(Object *obj)
+{
+MacNubusState *s = MAC_NUBUS_BRIDGE(obj);
+SysBusDevice *sbd = SYS_BUS_DEVICE(obj);
+
+s->bus = NUBUS_BUS(qbus_create(TYPE_NUBUS_BUS, DEVICE(s), NULL));
+
+sysbus_init_mmio(sbd, >bus->super_slot_io);
+sysbus_init_mmio(sbd, >bus->slot_io);
+}
+
+static void mac_nubus_bridge_class_init(ObjectClass *klass, void *data)
+{
+DeviceClass *dc = DEVICE_CLASS(klass);
+
+dc->desc = "Nubus bridge";
+}
+
+static const TypeInfo mac_nubus_bridge_info = {
+.name  = TYPE_MAC_NUBUS_BRIDGE,
+.parent= TYPE_NUBUS_BRIDGE,
+.instance_init = mac_nubus_bridge_init,
+.instance_size = sizeof(MacNubusState),
+.class_init= mac_nubus_bridge_class_init,
+};
+
+static void mac_nubus_bridge_register_types(void)
+{
+type_register_static(_nubus_bridge_info);
+}
+
+type_init(mac_nubus_bridge_register_types)
diff --git a/hw/nubus/nubus-bridge.c b/hw/nubus/nubus-bridge.c
new file mode 100644
index 00..cd8c6a91eb
--- /dev/null
+++ b/hw/nubus/nubus-bridge.c
@@ -0,0 +1,34 @@
+/*
+ * QEMU Macintosh Nubus
+ *
+ * Copyright (c) 2013-2018 Laurent Vivier 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "hw/sysbus.h"
+#include "hw/nubus/nubus.h"
+
+static void nubus_bridge_class_init(ObjectClass *klass, void *data)
+{
+DeviceClass *dc = DEVICE_CLASS(klass);
+
+dc->fw_name = "nubus";
+}
+
+static const TypeInfo nubus_bridge_info = {
+.name  = TYPE_NUBUS_BRIDGE,
+.parent= TYPE_SYS_BUS_DEVICE,
+.instance_size = sizeof(SysBusDevice),
+.class_init= nubus_bridge_class_init,
+};
+
+static void nubus_register_types(void)
+{
+type_register_static(_bridge_info);
+}
+
+type_init(nubus_register_types)
diff --git a/hw/nubus/nubus-bus.c b/hw/nubus/nubus-bus.c
new file mode 100644
index 00..942a6d5342
--- /dev/null
+++ b/hw/nubus/nubus-bus.c
@@ -0,0 +1,111 @@
+/*
+ * QEMU Macintosh Nubus
+ *
+ * Copyright (c) 2013-2018 Laurent Vivier 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level 

[Qemu-devel] [PATCH v8 00/10] hw/m68k: add Apple Machintosh Quadra 800 machine

[Laurent Vivier]

I'm rebasing some of these patches for seven years now,
too many years...

if you want to test the machine, I'm sorry, it doesn't boot
a MacROM, but you can boot a linux kernel from the command line.

You can install your own disk using debian-installer, with:

...
-M q800 \
-serial none -serial mon:stdio \
-m 1000M \
-net nic,model=dp83932,addr=09:00:07:12:34:57 \
-append "console=ttyS0 vga=off" \
-kernel vmlinux-4.16.0-1-m68k \
-initrd initrd.gz \
-drive file=debian-10.0-m68k-NETINST-1.iso,media=cdrom \
-drive file=m68k.qcow2,format=qcow2 \
-nographic

If you use a graphic adapter instead of "-nographic", you can use "-g" to set 
the
size of the display (I use "-g 1600x800x24").

You can get the ISO from:

https://cdimage.debian.org/cdimage/ports/10.0/m68k/iso-cd/debian-10.0-m68k-NETINST-1.iso

and extract the kernel and initrd.gz:

guestfish --add debian-10.0-m68k-NETINST-1.iso --ro \
  --mount /dev/sda:/ <<_EOF_
copy-out /install/cdrom/initrd.gz .
copy-out /install/kernels/vmlinux-4.16.0-1-m68k .
_EOF_

The mirror to use is: http://ftp.ports.debian.org/debian-ports/
when it fails, continue without boot loader.

In the same way, you can extract the kernel and the initramfs from the qcow2
image to use it with "-kernel" and "-initrd":

guestfish --add m68k.qcow2 --mount /dev/sda2:/ <<_EOF_
copy-out /boot/vmlinux-4.16.0-1-m68k .
copy-out /boot/initrd.img-4.16.0-1-m68k .
_EOF_

and boot with:

   ...
   -append "root=/dev/sda2 rw console=ttyS0 console=tty \
   -kernel vmlinux-4.16.0-1-m68k \
   -initrd initrd.img-4.16.0-1-m68k

NOTE: DHCP doesn't work but you can assign a static IP address.
  We need some patches for dp8393x that are not ready to be merged.
  See http://patchwork.ozlabs.org/patch/927020/
  http://patchwork.ozlabs.org/patch/927030/
  http://patchwork.ozlabs.org/patch/927026/

v8: rebase (new blk_new(), add "qemu-common.h")
update bootinfo information and license
add some braces
Rename Q800IRQState to GLUEState:
it's more like a Logic Unit than an IRQ controller,
and Apple calls it "GLUE" (Mark: I prefer to keep it
like this for the moment, in the future this part
need to be reworked, we have to review the IRQ levels
and to wire NUBUS IRQ. The implementation is really trivial
for the moment and we will move it to QOM in the future)

v7: rebase and port to Kconfig
move IRQ controller back to q800.c (we don't need an object for this)
update log message for ESP changes and add some g_assert()
re-order patches: put esp, escc and dp8393x first

v6: Rebase onto git master (this now includes the m68k EXCP_ILLEGAL fix required
  for this patchset to boot)
Add Hervé's R-B tags
Drop ASC (Apple Sound Chip) device since the Linux driver is broken and
  it is not required for a successful boot
Remove extra esp_raise_irq() from ESP pseudo-DMA patch (Hervé)
Remove "return" from unimplemented write functions and instead add a
  "read only" comment (Hervé)
Rename MAX_FD to SWIM_MAX_FD in SWIM floppy controller patch to prevent
  potential conflicts with other files (Hervé)

v5: Rebase onto git master
Add Philippe's R-B to patch 10
Include the command line to boot a Linux kernel under the q800 machine in 
the
commit message for patch 11 (Philippe)
Fix up comments in hw/misc/mac_via.c (Thomas)
Add asserts to VIA ADB support to prevent potential buffer overflows 
(Thomas)
Move macfb surface/resolution checks to realise and remove hw_error (Thomas)
Move macfb draw_line functions inline and remove macfb-template.h (Mark)
Use guest address rather than source pointer in draw_line functions - this 
brings
  macfb in line with the VGA device and can prevent a potential buffer 
overflow
Use g_strdup_printf() for memory region names in NuBus devices instead of
  hardcoded length char arrays (Thomas)
Move NuBus QOM types from patch 7 to patch 8 (spotted by Thomas)
Move CONFIG_COLDFIRE sections together in hw/m68k/Makefile.objs (Thomas)
Remove obsolete comment from q800.c in patch 11 (Thomas)

v4: Drop RFC from subject prefix as this is getting close to final
Rebased onto master (fixing ESP, rom_ptr() conflicts)
Reworked q800.c based upon Thomas' comments about cpu_init() and
  qemu_check_nic_model()
Address Thomas' comments on using error_report() instead of hw_error()
Change the NuBus memory regions from DEVICE_NATIVE_ENDIAN to
  DEVICE_BIG_ENDIAN
Split macfb Nubus support into separate commit
Change VMSTATE_BUFFER_UNSAFE() to VMSTATE_UINT8_ARRAY() in macfb.c as
  suggested by David
Remove dummy Apple Sound Chip migration state as pointed out by David
Keep VIA ADB state and buffers in the mac_via device rather than adding
  to existing ADBState (this matches the pattern used in the PPC CUDA/PMU
  VIAs)
Remove blacklisting for q800 machine from "make 

[Qemu-devel] [PATCH v8 10/10] hw/m68k: define Macintosh Quadra 800

[Laurent Vivier]

If you want to test the machine, it doesn't yet boot a MacROM, but you can
boot a linux kernel from the command line.

You can install your own disk using debian-installer with:

./qemu-system-m68k \
-M q800 \
-serial none -serial mon:stdio \
-m 1000M -drive file=m68k.qcow2,format=qcow2 \
-net nic,model=dp83932,addr=09:00:07:12:34:57 \
-append "console=ttyS0 vga=off" \
-kernel vmlinux-4.15.0-2-m68k \
-initrd initrd.gz \
-drive file=debian-9.0-m68k-NETINST-1.iso \
-drive file=m68k.qcow2,format=qcow2 \
-nographic

If you use a graphic adapter instead of "-nographic", you can use "-g" to set 
the
size of the display (I use "-g 1600x800x24").

Co-developed-by: Mark Cave-Ayland 
Signed-off-by: Mark Cave-Ayland 
Signed-off-by: Laurent Vivier 
---
 MAINTAINERS  |  14 ++
 default-configs/m68k-softmmu.mak |   1 +
 hw/m68k/Kconfig  |  12 +
 hw/m68k/Makefile.objs|   1 +
 hw/m68k/bootinfo.h   | 114 ++
 hw/m68k/q800.c   | 377 +++
 6 files changed, 519 insertions(+)
 create mode 100644 hw/m68k/bootinfo.h
 create mode 100644 hw/m68k/q800.c

diff --git a/MAINTAINERS b/MAINTAINERS
index d32c5c2313..aa00525d37 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -902,6 +902,20 @@ F: hw/char/mcf_uart.c
 F: hw/net/mcf_fec.c
 F: include/hw/m68k/mcf*.h
 
+q800
+M: Laurent Vivier 
+S: Maintained
+F: hw/block/swim.c
+F: hw/m68k/bootinfo.h
+F: hw/display/macfb.c
+F: hw/m68k/q800.c
+F: hw/misc/mac_via.c
+F: hw/nubus/*
+F: include/hw/block/swim.h
+F: include/hw/display/macfb.h
+F: include/hw/misc/mac_via.h
+F: include/hw/nubus/*
+
 MicroBlaze Machines
 ---
 petalogix_s3adsp1800
diff --git a/default-configs/m68k-softmmu.mak b/default-configs/m68k-softmmu.mak
index 4049a8f2ba..0fbae017ca 100644
--- a/default-configs/m68k-softmmu.mak
+++ b/default-configs/m68k-softmmu.mak
@@ -6,3 +6,4 @@ CONFIG_SEMIHOSTING=y
 #
 CONFIG_AN5206=y
 CONFIG_MCF5208=y
+CONFIG_Q800=y
diff --git a/hw/m68k/Kconfig b/hw/m68k/Kconfig
index 49ef0b3f6d..ffa8e48fd8 100644
--- a/hw/m68k/Kconfig
+++ b/hw/m68k/Kconfig
@@ -7,3 +7,15 @@ config MCF5208
 bool
 select COLDFIRE
 select PTIMER
+
+config Q800
+bool
+select FRAMEBUFFER
+select ADB
+select MAC_VIA
+select ESCC
+select ESP
+select MACFB
+select NUBUS
+select DP8393X
+select SWIM
diff --git a/hw/m68k/Makefile.objs b/hw/m68k/Makefile.objs
index 482f8477b4..cfd13fae53 100644
--- a/hw/m68k/Makefile.objs
+++ b/hw/m68k/Makefile.objs
@@ -1,2 +1,3 @@
 obj-$(CONFIG_AN5206) += an5206.o mcf5206.o
 obj-$(CONFIG_MCF5208) += mcf5208.o mcf_intc.o
+obj-$(CONFIG_Q800) += q800.o
diff --git a/hw/m68k/bootinfo.h b/hw/m68k/bootinfo.h
new file mode 100644
index 00..5f8ded2686
--- /dev/null
+++ b/hw/m68k/bootinfo.h
@@ -0,0 +1,114 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note
+ *
+ * Bootinfo tags from linux bootinfo.h and bootinfo-mac.h:
+ * This is an easily parsable and extendable structure containing all
+ * information to be passed from the bootstrap to the kernel
+ *
+ * This structure is copied right after the kernel by the bootstrap
+ * routine.
+ */
+
+#ifndef HW_M68K_BOOTINFO_H
+#define HW_M68K_BOOTINFO_H
+struct bi_record {
+uint16_t tag;/* tag ID */
+uint16_t size;   /* size of record */
+uint32_t data[0];/* data */
+};
+
+/* machine independent tags */
+
+#define BI_LAST 0x /* last record */
+#define BI_MACHTYPE 0x0001 /* machine type (u_long) */
+#define BI_CPUTYPE  0x0002 /* cpu type (u_long) */
+#define BI_FPUTYPE  0x0003 /* fpu type (u_long) */
+#define BI_MMUTYPE  0x0004 /* mmu type (u_long) */
+#define BI_MEMCHUNK 0x0005 /* memory chunk address and size */
+   /* (struct mem_info) */
+#define BI_RAMDISK  0x0006 /* ramdisk address and size */
+   /* (struct mem_info) */
+#define BI_COMMAND_LINE 0x0007 /* kernel command line parameters */
+   /* (string) */
+
+/*  Macintosh-specific tags (all u_long) */
+
+#define BI_MAC_MODEL0x8000  /* Mac Gestalt ID (model type) */
+#define BI_MAC_VADDR0x8001  /* Mac video base address */
+#define BI_MAC_VDEPTH   0x8002  /* Mac video depth */
+#define BI_MAC_VROW 0x8003  /* Mac video rowbytes */
+#define BI_MAC_VDIM 0x8004  /* Mac video dimensions */
+#define BI_MAC_VLOGICAL 0x8005  /* Mac video logical base */
+#define BI_MAC_SCCBASE  0x8006  /* Mac SCC base address */
+#define BI_MAC_BTIME0x8007  /* Mac boot time */
+#define BI_MAC_GMTBIAS  0x8008  /* Mac GMT timezone offset */
+#define BI_MAC_MEMSIZE  0x8009  /* Mac RAM size (sanity check) */
+#define BI_MAC_CPUID0x800a  /* Mac CPU type (sanity check) */
+#define BI_MAC_ROMBASE  0x800b  /* Mac system ROM base address */
+
+/*  Macintosh hardware profile data */
+
+#define BI_MAC_VIA1BASE 0x8010  /* Mac VIA1 base address 

[Qemu-devel] [PATCH v8 01/10] escc: introduce a selector for the register bit

[Laurent Vivier]

On Sparc and PowerMac, the bit 0 of the address
selects the register type (control or data) and
bit 1 selects the channel (B or A).

On m68k Macintosh, the bit 0 selects the channel and
bit 1 the register type.

This patch introduces a new parameter (bit_swap) to
the device interface to indicate bits usage must
be swapped between registers and channels.

For the moment all the machines use the bit 0,
but this change will be needed to emulate Quadra 800.

Signed-off-by: Laurent Vivier 
Reviewed-by: Hervé Poussineau 
Reviewed-by: Thomas Huth 
---
 hw/char/escc.c | 30 --
 include/hw/char/escc.h |  1 +
 2 files changed, 25 insertions(+), 6 deletions(-)

diff --git a/hw/char/escc.c b/hw/char/escc.c
index 8ddbb4be4f..2748bd62c3 100644
--- a/hw/char/escc.c
+++ b/hw/char/escc.c
@@ -43,14 +43,21 @@
  * mouse and keyboard ports don't implement all functions and they are
  * only asynchronous. There is no DMA.
  *
- * Z85C30 is also used on PowerMacs. There are some small differences
- * between Sparc version (sunzilog) and PowerMac (pmac):
+ * Z85C30 is also used on PowerMacs and m68k Macs.
+ *
+ * There are some small differences between Sparc version (sunzilog)
+ * and PowerMac (pmac):
  *  Offset between control and data registers
  *  There is some kind of lockup bug, but we can ignore it
  *  CTS is inverted
  *  DMA on pmac using DBDMA chip
  *  pmac can do IRDA and faster rates, sunzilog can only do 38400
  *  pmac baud rate generator clock is 3.6864 MHz, sunzilog 4.9152 MHz
+ *
+ * Linux driver for m68k Macs is the same as for PowerMac (pmac_zilog),
+ * but registers are grouped by type and not by channel:
+ * channel is selected by bit 0 of the address (instead of bit 1)
+ * and register is selected by bit 1 of the address (instead of bit 0).
  */
 
 /*
@@ -170,6 +177,16 @@ static void handle_kbd_command(ESCCChannelState *s, int 
val);
 static int serial_can_receive(void *opaque);
 static void serial_receive_byte(ESCCChannelState *s, int ch);
 
+static int reg_shift(ESCCState *s)
+{
+return s->bit_swap ? s->it_shift + 1 : s->it_shift;
+}
+
+static int chn_shift(ESCCState *s)
+{
+return s->bit_swap ? s->it_shift : s->it_shift + 1;
+}
+
 static void clear_queue(void *opaque)
 {
 ESCCChannelState *s = opaque;
@@ -434,8 +451,8 @@ static void escc_mem_write(void *opaque, hwaddr addr,
 int newreg, channel;
 
 val &= 0xff;
-saddr = (addr >> serial->it_shift) & 1;
-channel = (addr >> (serial->it_shift + 1)) & 1;
+saddr = (addr >> reg_shift(serial)) & 1;
+channel = (addr >> chn_shift(serial)) & 1;
 s = >chn[channel];
 switch (saddr) {
 case SERIAL_CTRL:
@@ -545,8 +562,8 @@ static uint64_t escc_mem_read(void *opaque, hwaddr addr,
 uint32_t ret;
 int channel;
 
-saddr = (addr >> serial->it_shift) & 1;
-channel = (addr >> (serial->it_shift + 1)) & 1;
+saddr = (addr >> reg_shift(serial)) & 1;
+channel = (addr >> chn_shift(serial)) & 1;
 s = >chn[channel];
 switch (saddr) {
 case SERIAL_CTRL:
@@ -830,6 +847,7 @@ static void escc_realize(DeviceState *dev, Error **errp)
 static Property escc_properties[] = {
 DEFINE_PROP_UINT32("frequency", ESCCState, frequency,   0),
 DEFINE_PROP_UINT32("it_shift",  ESCCState, it_shift,0),
+DEFINE_PROP_BOOL("bit_swap",ESCCState, bit_swap,false),
 DEFINE_PROP_UINT32("disabled",  ESCCState, disabled,0),
 DEFINE_PROP_UINT32("chnBtype",  ESCCState, chn[0].type, 0),
 DEFINE_PROP_UINT32("chnAtype",  ESCCState, chn[1].type, 0),
diff --git a/include/hw/char/escc.h b/include/hw/char/escc.h
index 42aca83611..8762f61c14 100644
--- a/include/hw/char/escc.h
+++ b/include/hw/char/escc.h
@@ -50,6 +50,7 @@ typedef struct ESCCState {
 
 struct ESCCChannelState chn[2];
 uint32_t it_shift;
+bool bit_swap;
 MemoryRegion mmio;
 uint32_t disabled;
 uint32_t frequency;
-- 
2.21.0

Re: [Qemu-devel] [RFC v1 3/5] hw/riscv: Extend the kernel loading support

[Alistair Francis]

On Wed, Jun 19, 2019 at 2:01 PM Alistair Francis  wrote:
>
> On Wed, Jun 19, 2019 at 8:16 AM Bin Meng  wrote:
> >
> > On Wed, Jun 19, 2019 at 8:56 AM Alistair Francis
> >  wrote:
> > >
> > > Extend the RISC-V kernel loader to support uImage and Image files.
> > > A Linux kernel can now be booted with:
> > >
> > > qemu-system-riscv64 -machine virt -bios fw_jump.elf -kernel Image
> > >
> > > Signed-off-by: Alistair Francis 
> > > ---
> > >  hw/riscv/boot.c | 19 ++-
> > >  1 file changed, 14 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/hw/riscv/boot.c b/hw/riscv/boot.c
> > > index 392ca0cb2e..7f68035a3f 100644
> > > --- a/hw/riscv/boot.c
> > > +++ b/hw/riscv/boot.c
> > > @@ -67,13 +67,22 @@ target_ulong riscv_load_kernel(MachineState *machine,
> > >  uint64_t kernel_entry, kernel_high;
> > >
> > >  if (load_elf(kernel_filename, NULL, kernel_translate, machine,
> > > - _entry, NULL, _high,
> > > - 0, EM_RISCV, 1, 0) < 0) {
> > > -error_report("could not load kernel '%s'", kernel_filename);
> > > -exit(1);
> > > + _entry, NULL, _high, 0, EM_RISCV, 1, 0) > 
> > > 0) {
> > > +return kernel_entry;
> > > +}
> > > +
> > > +if (load_uimage_as(kernel_filename, _entry, NULL, NULL,
> > > +   kernel_translate, machine, NULL) > 0) {
> >
> > We should not set the 'kernel_translate' here for uImage.
> >
> > In fact, the whole kernel_translate() is not necessary.
>
> I have removed the kernel_translate() function. I tested loading
> uImage files though and they don't seem to work as the image is loaded
> at the wrong address.
>
> I have removed uImage loading support from this series. We can look at
> it in the future if we decide we want it.

Ah, my mistake, it looks like my uImage wasn't built correctly. I'll
keep this in.

Alistair

>
> Alistair
>
> >
> > > +return kernel_entry;
> > > +}
> > > +
> >
> > Regards,
> > Bin

Re: [Qemu-devel] [QEMU PATCH v4 06/10] linux-headers: i386: Modify struct kvm_nested_state to have explicit fields for data

[Maran Wilson]

On 6/19/2019 9:21 AM, Liran Alon wrote:

Improve the KVM_{GET,SET}_NESTED_STATE structs by detailing the format
of VMX nested state data in a struct.

In order to avoid changing the ioctl values of
KVM_{GET,SET}_NESTED_STATE, there is a need to preserve
sizeof(struct kvm_nested_state). This is done by defining the data
struct as "data.vmx[0]". It was the most elegant way I found to
preserve struct size while still keeping struct readable and easy to
maintain. It does have a misfortunate side-effect that now it has to be
accessed as "data.vmx[0]" rather than just "data.vmx".

Because we are already modifying these structs, I also modified the
following:
* Define the "format" field values as macros.
* Rename vmcs_pa to vmcs12_pa for better readability.

Signed-off-by: Liran Alon 
---
  linux-headers/asm-x86/kvm.h | 33 ++---
  1 file changed, 22 insertions(+), 11 deletions(-)

diff --git a/linux-headers/asm-x86/kvm.h b/linux-headers/asm-x86/kvm.h
index 7a0e64ccd6ff..6e7dd792e448 100644
--- a/linux-headers/asm-x86/kvm.h
+++ b/linux-headers/asm-x86/kvm.h
@@ -383,16 +383,26 @@ struct kvm_sync_regs {
  #define KVM_X86_QUIRK_LAPIC_MMIO_HOLE (1 << 2)
  #define KVM_X86_QUIRK_OUT_7E_INC_RIP  (1 << 3)
  
+#define KVM_STATE_NESTED_FORMAT_VMX	0

+#define KVM_STATE_NESTED_FORMAT_SVM1
+
  #define KVM_STATE_NESTED_GUEST_MODE   0x0001
  #define KVM_STATE_NESTED_RUN_PENDING  0x0002
  #define KVM_STATE_NESTED_EVMCS0x0004
  
+#define KVM_STATE_NESTED_VMX_VMCS_SIZE	0x1000

+
  #define KVM_STATE_NESTED_SMM_GUEST_MODE   0x0001
  #define KVM_STATE_NESTED_SMM_VMXON0x0002
  
-struct kvm_vmx_nested_state {

+struct kvm_vmx_nested_state_data {
+   __u8 vmcs12[KVM_STATE_NESTED_VMX_VMCS_SIZE];
+   __u8 shadow_vmcs12[KVM_STATE_NESTED_VMX_VMCS_SIZE];
+};
+
+struct kvm_vmx_nested_state_hdr {
__u64 vmxon_pa;
-   __u64 vmcs_pa;
+   __u64 vmcs12_pa;
  
  	struct {

__u16 flags;
@@ -401,24 +411,25 @@ struct kvm_vmx_nested_state {
  
  /* for KVM_CAP_NESTED_STATE */

  struct kvm_nested_state {
-   /* KVM_STATE_* flags */
__u16 flags;
-
-   /* 0 for VMX, 1 for SVM.  */
__u16 format;
-
-   /* 128 for SVM, 128 + VMCS size for VMX.  */
__u32 size;
  
  	union {

-   /* VMXON, VMCS */
-   struct kvm_vmx_nested_state vmx;
+   struct kvm_vmx_nested_state_hdr vmx;
  
  		/* Pad the header to 128 bytes.  */

__u8 pad[120];
-   };
+   } hdr;
  
-	__u8 data[0];

+   /*
+* Define data region as 0 bytes to preserve backwards-compatability
+* to old definition of kvm_nested_state in order to avoid changing
+* KVM_{GET,PUT}_NESTED_STATE ioctl values.
+*/
+   union {
+   struct kvm_vmx_nested_state_data vmx[0];
+   } data;
  };
  
  #endif /* _ASM_X86_KVM_H */


Reviewed-by: Maran Wilson 

Thanks,
-Maran

Re: [Qemu-devel] [RFC v1 3/5] hw/riscv: Extend the kernel loading support

[Alistair Francis]

On Wed, Jun 19, 2019 at 8:16 AM Bin Meng  wrote:
>
> On Wed, Jun 19, 2019 at 8:56 AM Alistair Francis
>  wrote:
> >
> > Extend the RISC-V kernel loader to support uImage and Image files.
> > A Linux kernel can now be booted with:
> >
> > qemu-system-riscv64 -machine virt -bios fw_jump.elf -kernel Image
> >
> > Signed-off-by: Alistair Francis 
> > ---
> >  hw/riscv/boot.c | 19 ++-
> >  1 file changed, 14 insertions(+), 5 deletions(-)
> >
> > diff --git a/hw/riscv/boot.c b/hw/riscv/boot.c
> > index 392ca0cb2e..7f68035a3f 100644
> > --- a/hw/riscv/boot.c
> > +++ b/hw/riscv/boot.c
> > @@ -67,13 +67,22 @@ target_ulong riscv_load_kernel(MachineState *machine,
> >  uint64_t kernel_entry, kernel_high;
> >
> >  if (load_elf(kernel_filename, NULL, kernel_translate, machine,
> > - _entry, NULL, _high,
> > - 0, EM_RISCV, 1, 0) < 0) {
> > -error_report("could not load kernel '%s'", kernel_filename);
> > -exit(1);
> > + _entry, NULL, _high, 0, EM_RISCV, 1, 0) > 
> > 0) {
> > +return kernel_entry;
> > +}
> > +
> > +if (load_uimage_as(kernel_filename, _entry, NULL, NULL,
> > +   kernel_translate, machine, NULL) > 0) {
>
> We should not set the 'kernel_translate' here for uImage.
>
> In fact, the whole kernel_translate() is not necessary.

I have removed the kernel_translate() function. I tested loading
uImage files though and they don't seem to work as the image is loaded
at the wrong address.

I have removed uImage loading support from this series. We can look at
it in the future if we decide we want it.

Alistair

>
> > +return kernel_entry;
> > +}
> > +
>
> Regards,
> Bin

Re: [Qemu-devel] [Qemu-riscv] [RFC v1 2/5] hw/riscv: Add support for loading a firmware

[Alistair Francis]

On Wed, Jun 19, 2019 at 8:30 AM Bin Meng  wrote:
>
> Hi,
>
> On Wed, Jun 19, 2019 at 11:26 PM Jonathan Behrens  wrote:
> >
> > I was actually just writing up the same thing.  Shifting all the addresses 
> > in the ELF file by 2 or 4MB is somewhat surprising behavior, especially 
> > because this will apply to segments that are mapped even at much higher 
> > addresses. If you want a segment aligned to a 1GB superpage boundary you 
> > now need to place it slightly below so that it will be bumped up to the 
> > right place. Further, ANDing all addresses with 0x7fff makes it 
> > impossible to map anything beyond the first 2GB of RAM.
> >
>
> Yes, current kernel_translate() logic is tightly coupled to the kernel
> entry VA, and if we link kernel at some other address it will just
> fail.

I thought this was required but it looks like it isn't. I have remove
the kernel_translate() function.

>
> > Jonathan
> >
> > On Wed, Jun 19, 2019 at 11:16 AM Bin Meng  wrote:
> >>
> >> On Wed, Jun 19, 2019 at 8:53 AM Alistair Francis
> >>  wrote:
> >> >
> >> > Add support for loading a firmware file for the virt machine and the
> >> > SiFive U. This can be run with the following command:
> >> >
> >> > qemu-system-riscv64 -machine virt -bios fw_jump.elf -kernel vmlinux
> >> >
> >> > Signed-off-by: Alistair Francis 
> >> > ---
> >> >  hw/riscv/boot.c | 41 +++--
> >> >  hw/riscv/sifive_e.c |  2 +-
> >> >  hw/riscv/sifive_u.c |  6 +-
> >> >  hw/riscv/spike.c|  6 +++---
> >> >  hw/riscv/virt.c |  7 ++-
> >> >  include/hw/riscv/boot.h |  4 +++-
> >> >  6 files changed, 57 insertions(+), 9 deletions(-)
> >> >
> >> > diff --git a/hw/riscv/boot.c b/hw/riscv/boot.c
> >> > index 62f94aaf8a..392ca0cb2e 100644
> >> > --- a/hw/riscv/boot.c
> >> > +++ b/hw/riscv/boot.c
> >> > @@ -23,13 +23,50 @@
> >> >  #include "exec/cpu-defs.h"
> >> >  #include "hw/loader.h"
> >> >  #include "hw/riscv/boot.h"
> >> > +#include "hw/boards.h"
> >> >  #include "elf.h"
> >> >
> >> > -target_ulong riscv_load_kernel(const char *kernel_filename)
> >> > +#if defined(TARGET_RISCV32)
> >> > +# define KERNEL_BOOT_ADDRESS 0x8040
> >> > +#else
> >> > +# define KERNEL_BOOT_ADDRESS 0x8020
> >> > +#endif
> >> > +
> >> > +static uint64_t kernel_translate(void *opaque, uint64_t addr)
> >> > +{
> >> > +MachineState *machine = opaque;
> >> > +
> >> > +/*
> >> > + * If the user specified a firmware move the kernel to the offset
> >> > + * start address.
> >> > + */
> >>
> >> Why?

Removed.

> >>
> >> > +if (machine->firmware) {
> >> > +return (addr & 0x7fff) + KERNEL_BOOT_ADDRESS;
> >>
> >> So with both "-bios" and "-kernel", the kernel address will be moved
> >> to another address other than 0x8020 (for 64-bit). This does not
> >> look good to me.
> >>
>
> So why not simply return KERNEL_BOOT_ADDRESS in kernel_translate()?

That's what I am doing now.

Alistair

>
> Regards,
> Bin

Re: [Qemu-devel] [QEMU PATCH v4 01/10] target/i386: kvm: Delete VMX migration blocker on vCPU init failure

[Maran Wilson]

On 6/19/2019 1:33 PM, Liran Alon wrote:

On 19 Jun 2019, at 23:30, Maran Wilson  wrote:

On 6/19/2019 9:21 AM, Liran Alon wrote:

Commit d98f26073beb ("target/i386: kvm: add VMX migration blocker")
added migration blocker for vCPU exposed with Intel VMX because QEMU
doesn't yet contain code to support migration of nested virtualization
workloads.

However, that commit missed adding deletion of the migration blocker in
case init of vCPU failed. Similar to invtsc_mig_blocker. This commit fix
that issue.

Fixes: d98f26073beb ("target/i386: kvm: add VMX migration blocker")
Signed-off-by: Liran Alon 
---
  target/i386/kvm.c | 7 +--
  1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 3b29ce5c0d08..7aa7914a498c 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -940,7 +940,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
r = kvm_arch_set_tsc_khz(cs);
  if (r < 0) {
-goto fail;
+return r;
  }
/* vcpu's TSC frequency is either specified by user, or following
@@ -1295,7 +1295,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
  if (local_err) {
  error_report_err(local_err);
  error_free(invtsc_mig_blocker);
-return r;
+goto fail2;
  }
  }
  }
@@ -1346,6 +1346,9 @@ int kvm_arch_init_vcpu(CPUState *cs)
 fail:
  migrate_del_blocker(invtsc_mig_blocker);
+ fail2:
+migrate_del_blocker(vmx_mig_blocker);
+

At the risk of being a bit pedantic...

Your changes don't introduce this problem, but they do make it worse -- Since 
[vmx|invtsc]_mig_blocker are both global in scope, isn't it possible you end up deleting one or 
both valid blockers that were created by a previous invocation of kvm_arch_init_vcpu() ?  Seems to 
me that you would need something like an additional pair of local boolean variables named 
created_[vmx|invtsc]_mig_blocker and condition the calls to migrate_del_blocker() accordingly. On 
the positive side, that would simplify some of the logic around when and if it's ok to jump to 
"fail" (and you wouldn't need the "fail2").

Thanks,
-Maran

I actually thought about this as-well when I encountered this issue.
In general one can argue that every vCPU should introduce it’s own migration 
blocker on init (if required) and remove it’s own migration blocker on deletion 
(on vCPU destroy).
On 99% of the time, all of this shouldn’t matter as all vCPUs of a given VM 
have exactly the same properties.


The example I was thinking about is a VM that is created with a bunch of 
vCPUs -- all of which require installation of the blocker(s). Then at 
some point in the future, a failed CPU hotplug attempt wipes out all the 
pre-existing blockers and leaves the VM exposed.


But I agree that the problem wasn't introduced by this patch series and 
so it is reasonable to argue that you shouldn't have to fix it here. As 
such:


Reviewed-by: Maran Wilson 

Thanks,
-Maran


Anyway, I decided that this is entirely not relevant for this patch-series and 
therefore if this should be addressed further, let it be another unrelated 
patch-series.
QEMU have too many issues to fix all at once :P. I need to filter.

-Liran


  return r;
  }
  

Re: [Qemu-devel] [QEMU PATCH v4 01/10] target/i386: kvm: Delete VMX migration blocker on vCPU init failure

[Liran Alon]

> On 19 Jun 2019, at 23:30, Maran Wilson  wrote:
> 
> On 6/19/2019 9:21 AM, Liran Alon wrote:
>> Commit d98f26073beb ("target/i386: kvm: add VMX migration blocker")
>> added migration blocker for vCPU exposed with Intel VMX because QEMU
>> doesn't yet contain code to support migration of nested virtualization
>> workloads.
>> 
>> However, that commit missed adding deletion of the migration blocker in
>> case init of vCPU failed. Similar to invtsc_mig_blocker. This commit fix
>> that issue.
>> 
>> Fixes: d98f26073beb ("target/i386: kvm: add VMX migration blocker")
>> Signed-off-by: Liran Alon 
>> ---
>>  target/i386/kvm.c | 7 +--
>>  1 file changed, 5 insertions(+), 2 deletions(-)
>> 
>> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
>> index 3b29ce5c0d08..7aa7914a498c 100644
>> --- a/target/i386/kvm.c
>> +++ b/target/i386/kvm.c
>> @@ -940,7 +940,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
>>r = kvm_arch_set_tsc_khz(cs);
>>  if (r < 0) {
>> -goto fail;
>> +return r;
>>  }
>>/* vcpu's TSC frequency is either specified by user, or following
>> @@ -1295,7 +1295,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
>>  if (local_err) {
>>  error_report_err(local_err);
>>  error_free(invtsc_mig_blocker);
>> -return r;
>> +goto fail2;
>>  }
>>  }
>>  }
>> @@ -1346,6 +1346,9 @@ int kvm_arch_init_vcpu(CPUState *cs)
>> fail:
>>  migrate_del_blocker(invtsc_mig_blocker);
>> + fail2:
>> +migrate_del_blocker(vmx_mig_blocker);
>> +
> 
> At the risk of being a bit pedantic...
> 
> Your changes don't introduce this problem, but they do make it worse -- Since 
> [vmx|invtsc]_mig_blocker are both global in scope, isn't it possible you end 
> up deleting one or both valid blockers that were created by a previous 
> invocation of kvm_arch_init_vcpu() ?  Seems to me that you would need 
> something like an additional pair of local boolean variables named 
> created_[vmx|invtsc]_mig_blocker and condition the calls to 
> migrate_del_blocker() accordingly. On the positive side, that would simplify 
> some of the logic around when and if it's ok to jump to "fail" (and you 
> wouldn't need the "fail2").
> 
> Thanks,
> -Maran

I actually thought about this as-well when I encountered this issue.
In general one can argue that every vCPU should introduce it’s own migration 
blocker on init (if required) and remove it’s own migration blocker on deletion 
(on vCPU destroy).
On 99% of the time, all of this shouldn’t matter as all vCPUs of a given VM 
have exactly the same properties.
Anyway, I decided that this is entirely not relevant for this patch-series and 
therefore if this should be addressed further, let it be another unrelated 
patch-series.
QEMU have too many issues to fix all at once :P. I need to filter.

-Liran

> 
>>  return r;
>>  }
>>  

Re: [Qemu-devel] [PATCH 01/17] MAINTAINERS: new maintainers for QOM

[Andreas Färber]

Am 19.06.19 um 22:10 schrieb Markus Armbruster:
> From: Paolo Bonzini 
> 
> QOM is not a particularly active subsystem now: 51 commits in two years.
> But, we need active maintainers to review and merge patches, and Git
> shows the following top committers taking on QOM:
> 
> Markus Armbruster 
> Eduardo Habkost 
> Paolo Bonzini 
> Marc-André Lureau 
> Eric Blake 
> 
> I volunteer myself, and also volunteer Eduardo and Daniel as reviewers
> since they understand the code well.
> 
> Cc: Andreas Färber 
> Cc: Daniel P. Berrange 
> Cc: Eduardo Habkost 
> Suggested-by: Markus Armbruster 
> Signed-off-by: Paolo Bonzini 
> Message-Id: <20190607113737.13473-1-pbonz...@redhat.com>
> Signed-off-by: Markus Armbruster 

I'm pretty sure I gave an Acked-by that's missing above?

Regards,
Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG Nürnberg)

Re: [Qemu-devel] [PATCH v1 2/4] disas/riscv: Disassemble reserved compressed encodings as illegal

[Alistair Francis]

On Fri, Jun 14, 2019 at 2:18 AM Palmer Dabbelt  wrote:
>
> On Fri, 17 May 2019 15:11:01 PDT (-0700), Alistair Francis wrote:
> > From: Michael Clark 
> >
> > Due to the design of the disassembler, the immediate is not
> > known during decoding of the opcode; so to handle compressed
> > encodings with reserved immediate values (non-zero), we need
> > to add an additional check during decompression to match
> > reserved encodings with zero immediates and translate them
> > into the illegal instruction.
> >
> > The following compressed opcodes have reserved encodings with
> > zero immediates: c.addi4spn, c.addi, c.lui, c.addi16sp, c.srli,
> > c.srai, c.andi and c.slli
> >
> > Signed-off-by: Michael Clark 
> > Signed-off-by: Alistair Francis 
> > ---
> >  disas/riscv.c | 51 ++-
> >  1 file changed, 34 insertions(+), 17 deletions(-)
> >
> > diff --git a/disas/riscv.c b/disas/riscv.c
> > index 59a9b0437a..3ab4586f0a 100644
> > --- a/disas/riscv.c
> > +++ b/disas/riscv.c
> > @@ -504,14 +504,19 @@ typedef struct {
> >  const rvc_constraint *constraints;
> >  } rv_comp_data;
> >
> > +enum {
> > +rvcd_imm_nz = 0x1
> > +};
> > +
> >  typedef struct {
> >  const char * const name;
> >  const rv_codec codec;
> >  const char * const format;
> >  const rv_comp_data *pseudo;
> > -const int decomp_rv32;
> > -const int decomp_rv64;
> > -const int decomp_rv128;
> > +const short decomp_rv32;
> > +const short decomp_rv64;
> > +const short decomp_rv128;
> > +const short decomp_data;
> >  } rv_opcode_data;
> >
> >  /* register names */
> > @@ -1011,7 +1016,7 @@ const rv_opcode_data opcode_data[] = {
> >  { "fcvt.q.lu", rv_codec_r_m, rv_fmt_rm_frd_rs1, NULL, 0, 0, 0 },
> >  { "fmv.x.q", rv_codec_r, rv_fmt_rd_frs1, NULL, 0, 0, 0 },
> >  { "fmv.q.x", rv_codec_r, rv_fmt_frd_rs1, NULL, 0, 0, 0 },
> > -{ "c.addi4spn", rv_codec_ciw_4spn, rv_fmt_rd_rs1_imm, NULL, 
> > rv_op_addi, rv_op_addi, rv_op_addi },
> > +{ "c.addi4spn", rv_codec_ciw_4spn, rv_fmt_rd_rs1_imm, NULL, 
> > rv_op_addi, rv_op_addi, rv_op_addi, rvcd_imm_nz },
> >  { "c.fld", rv_codec_cl_ld, rv_fmt_frd_offset_rs1, NULL, rv_op_fld, 
> > rv_op_fld, 0 },
> >  { "c.lw", rv_codec_cl_lw, rv_fmt_rd_offset_rs1, NULL, rv_op_lw, 
> > rv_op_lw, rv_op_lw },
> >  { "c.flw", rv_codec_cl_lw, rv_fmt_frd_offset_rs1, NULL, rv_op_flw, 0, 
> > 0 },
> > @@ -1019,14 +1024,14 @@ const rv_opcode_data opcode_data[] = {
> >  { "c.sw", rv_codec_cs_sw, rv_fmt_rs2_offset_rs1, NULL, rv_op_sw, 
> > rv_op_sw, rv_op_sw },
> >  { "c.fsw", rv_codec_cs_sw, rv_fmt_frs2_offset_rs1, NULL, rv_op_fsw, 0, 
> > 0 },
> >  { "c.nop", rv_codec_ci_none, rv_fmt_none, NULL, rv_op_addi, 
> > rv_op_addi, rv_op_addi },
> > -{ "c.addi", rv_codec_ci, rv_fmt_rd_rs1_imm, NULL, rv_op_addi, 
> > rv_op_addi, rv_op_addi },
> > +{ "c.addi", rv_codec_ci, rv_fmt_rd_rs1_imm, NULL, rv_op_addi, 
> > rv_op_addi, rv_op_addi, rvcd_imm_nz },
> >  { "c.jal", rv_codec_cj_jal, rv_fmt_rd_offset, NULL, rv_op_jal, 0, 0 },
> >  { "c.li", rv_codec_ci_li, rv_fmt_rd_rs1_imm, NULL, rv_op_addi, 
> > rv_op_addi, rv_op_addi },
> > -{ "c.addi16sp", rv_codec_ci_16sp, rv_fmt_rd_rs1_imm, NULL, rv_op_addi, 
> > rv_op_addi, rv_op_addi },
> > -{ "c.lui", rv_codec_ci_lui, rv_fmt_rd_imm, NULL, rv_op_lui, rv_op_lui, 
> > rv_op_lui },
> > -{ "c.srli", rv_codec_cb_sh6, rv_fmt_rd_rs1_imm, NULL, rv_op_srli, 
> > rv_op_srli, rv_op_srli },
> > -{ "c.srai", rv_codec_cb_sh6, rv_fmt_rd_rs1_imm, NULL, rv_op_srai, 
> > rv_op_srai, rv_op_srai },
> > -{ "c.andi", rv_codec_cb_imm, rv_fmt_rd_rs1_imm, NULL, rv_op_andi, 
> > rv_op_andi, rv_op_andi },
> > +{ "c.addi16sp", rv_codec_ci_16sp, rv_fmt_rd_rs1_imm, NULL, rv_op_addi, 
> > rv_op_addi, rv_op_addi, rvcd_imm_nz },
> > +{ "c.lui", rv_codec_ci_lui, rv_fmt_rd_imm, NULL, rv_op_lui, rv_op_lui, 
> > rv_op_lui, rvcd_imm_nz },
> > +{ "c.srli", rv_codec_cb_sh6, rv_fmt_rd_rs1_imm, NULL, rv_op_srli, 
> > rv_op_srli, rv_op_srli, rvcd_imm_nz },
> > +{ "c.srai", rv_codec_cb_sh6, rv_fmt_rd_rs1_imm, NULL, rv_op_srai, 
> > rv_op_srai, rv_op_srai, rvcd_imm_nz },
> > +{ "c.andi", rv_codec_cb_imm, rv_fmt_rd_rs1_imm, NULL, rv_op_andi, 
> > rv_op_andi, rv_op_andi, rvcd_imm_nz },
>
> Unless I'm missing something, c.andi can have a zero immediate.

Yeah, I'll remove that.

Alistair

>
> >  { "c.sub", rv_codec_cs, rv_fmt_rd_rs1_rs2, NULL, rv_op_sub, rv_op_sub, 
> > rv_op_sub },
> >  { "c.xor", rv_codec_cs, rv_fmt_rd_rs1_rs2, NULL, rv_op_xor, rv_op_xor, 
> > rv_op_xor },
> >  { "c.or", rv_codec_cs, rv_fmt_rd_rs1_rs2, NULL, rv_op_or, rv_op_or, 
> > rv_op_or },
> > @@ -1036,7 +1041,7 @@ const rv_opcode_data opcode_data[] = {
> >  { "c.j", rv_codec_cj, rv_fmt_rd_offset, NULL, rv_op_jal, rv_op_jal, 
> > rv_op_jal },
> >  { "c.beqz", rv_codec_cb, rv_fmt_rs1_rs2_offset, NULL, rv_op_beq, 
> > rv_op_beq, rv_op_beq },
> >  { "c.bnez", 

Re: [Qemu-devel] [QEMU PATCH v4 01/10] target/i386: kvm: Delete VMX migration blocker on vCPU init failure

[Maran Wilson]

On 6/19/2019 9:21 AM, Liran Alon wrote:

Commit d98f26073beb ("target/i386: kvm: add VMX migration blocker")
added migration blocker for vCPU exposed with Intel VMX because QEMU
doesn't yet contain code to support migration of nested virtualization
workloads.

However, that commit missed adding deletion of the migration blocker in
case init of vCPU failed. Similar to invtsc_mig_blocker. This commit fix
that issue.

Fixes: d98f26073beb ("target/i386: kvm: add VMX migration blocker")
Signed-off-by: Liran Alon 
---
  target/i386/kvm.c | 7 +--
  1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 3b29ce5c0d08..7aa7914a498c 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -940,7 +940,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
  
  r = kvm_arch_set_tsc_khz(cs);

  if (r < 0) {
-goto fail;
+return r;
  }
  
  /* vcpu's TSC frequency is either specified by user, or following

@@ -1295,7 +1295,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
  if (local_err) {
  error_report_err(local_err);
  error_free(invtsc_mig_blocker);
-return r;
+goto fail2;
  }
  }
  }
@@ -1346,6 +1346,9 @@ int kvm_arch_init_vcpu(CPUState *cs)
  
   fail:

  migrate_del_blocker(invtsc_mig_blocker);
+ fail2:
+migrate_del_blocker(vmx_mig_blocker);
+


At the risk of being a bit pedantic...

Your changes don't introduce this problem, but they do make it worse -- 
Since [vmx|invtsc]_mig_blocker are both global in scope, isn't it 
possible you end up deleting one or both valid blockers that were 
created by a previous invocation of kvm_arch_init_vcpu() ?  Seems to me 
that you would need something like an additional pair of local boolean 
variables named created_[vmx|invtsc]_mig_blocker and condition the calls 
to migrate_del_blocker() accordingly. On the positive side, that would 
simplify some of the logic around when and if it's ok to jump to "fail" 
(and you wouldn't need the "fail2").


Thanks,
-Maran


  return r;
  }
  

Re: [Qemu-devel] [PATCH 17/17] dump: Move HMP command handlers to dump/

[Eric Blake]

On 6/19/19 3:10 PM, Markus Armbruster wrote:
> Move the HMP handlers related to qapi/dump.json to
> dimp/dump-hmp-cmds.c, where they are covered by MAINTAINERS section

s/dimp/dump/

> "Dump", just like qapi/dump.json.
> 
> Cc: Marc-André Lureau 
> Cc: "Dr. David Alan Gilbert" 
> Signed-off-by: Markus Armbruster 
> ---

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH 12/17] qapi: Split machine-target.json off target.json and misc.json

[Eric Blake]

On 6/19/19 3:10 PM, Markus Armbruster wrote:
> Move commands query-cpu-definitions, query-cpu-model-baseline,
> query-cpu-model-comparison, and query-cpu-model-expansion with their
> types from target.json to machine-target.json.  Also move typee

s/typee/types/

> CpuModelInfo, CpuModelExpansionType, and CpuModelCompareResult from
> misc.json there.  Add machine-target.json to MAINTAINERS section
> "Machine core".
> 
> Cc: Eduardo Habkost 
> Cc: Marcel Apfelbaum 
> Signed-off-by: Markus Armbruster 
> ---

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-devel] [PATCH 10/17] hw/core: Collect QMP command handlers in hw/core/

[Markus Armbruster]

The handlers for qapi/machine.json's QMP commands are spread over
cpus.c, hw/core/numa.c, monitor/misc.c, monitor/qmp-cmds.c, and vl.c.
Move them all to new hw/core/machine-qmp-cmds.c, where they are
covered by MAINTAINERS section "Machine core", just like
qapi/machine.json.

Cc: Eduardo Habkost 
Cc: Marcel Apfelbaum 
Signed-off-by: Markus Armbruster 
---
 MAINTAINERS|   1 +
 cpus.c | 188 -
 hw/core/Makefile.objs  |   1 +
 hw/core/machine-qmp-cmds.c | 328 +
 hw/core/numa.c |  62 ---
 include/sysemu/numa.h  |   2 +
 monitor/misc.c |  13 --
 monitor/qmp-cmds.c |  12 --
 vl.c   |  45 -
 9 files changed, 332 insertions(+), 320 deletions(-)
 create mode 100644 hw/core/machine-qmp-cmds.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 96a72a999a..0e6d0861ab 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1265,6 +1265,7 @@ Machine core
 M: Eduardo Habkost 
 M: Marcel Apfelbaum 
 S: Supported
+F: hw/core/machine-qmp-cmds.c
 F: hw/core/machine.c
 F: hw/core/null-machine.c
 F: hw/core/numa.c
diff --git a/cpus.c b/cpus.c
index f5e677b2c4..1c4bc06e4c 100644
--- a/cpus.c
+++ b/cpus.c
@@ -25,16 +25,13 @@
 #include "qemu/osdep.h"
 #include "qemu-common.h"
 #include "qemu/config-file.h"
-#include "cpu.h"
 #include "monitor/monitor.h"
 #include "qapi/error.h"
-#include "qapi/qapi-commands-machine.h"
 #include "qapi/qapi-commands-misc.h"
 #include "qapi/qapi-events-run-state.h"
 #include "qapi/qmp/qerror.h"
 #include "qemu/error-report.h"
 #include "qemu/qemu-print.h"
-#include "sysemu/sysemu.h"
 #include "sysemu/tcg.h"
 #include "sysemu/block-backend.h"
 #include "exec/gdbstub.h"
@@ -57,7 +54,6 @@
 #include "tcg.h"
 #include "hw/nmi.h"
 #include "sysemu/replay.h"
-#include "hw/boards.h"
 
 #ifdef CONFIG_LINUX
 
@@ -2202,190 +2198,6 @@ void list_cpus(const char *optarg)
 #endif
 }
 
-CpuInfoList *qmp_query_cpus(Error **errp)
-{
-MachineState *ms = MACHINE(qdev_get_machine());
-MachineClass *mc = MACHINE_GET_CLASS(ms);
-CpuInfoList *head = NULL, *cur_item = NULL;
-CPUState *cpu;
-
-CPU_FOREACH(cpu) {
-CpuInfoList *info;
-#if defined(TARGET_I386)
-X86CPU *x86_cpu = X86_CPU(cpu);
-CPUX86State *env = _cpu->env;
-#elif defined(TARGET_PPC)
-PowerPCCPU *ppc_cpu = POWERPC_CPU(cpu);
-CPUPPCState *env = _cpu->env;
-#elif defined(TARGET_SPARC)
-SPARCCPU *sparc_cpu = SPARC_CPU(cpu);
-CPUSPARCState *env = _cpu->env;
-#elif defined(TARGET_RISCV)
-RISCVCPU *riscv_cpu = RISCV_CPU(cpu);
-CPURISCVState *env = _cpu->env;
-#elif defined(TARGET_MIPS)
-MIPSCPU *mips_cpu = MIPS_CPU(cpu);
-CPUMIPSState *env = _cpu->env;
-#elif defined(TARGET_TRICORE)
-TriCoreCPU *tricore_cpu = TRICORE_CPU(cpu);
-CPUTriCoreState *env = _cpu->env;
-#elif defined(TARGET_S390X)
-S390CPU *s390_cpu = S390_CPU(cpu);
-CPUS390XState *env = _cpu->env;
-#endif
-
-cpu_synchronize_state(cpu);
-
-info = g_malloc0(sizeof(*info));
-info->value = g_malloc0(sizeof(*info->value));
-info->value->CPU = cpu->cpu_index;
-info->value->current = (cpu == first_cpu);
-info->value->halted = cpu->halted;
-info->value->qom_path = object_get_canonical_path(OBJECT(cpu));
-info->value->thread_id = cpu->thread_id;
-#if defined(TARGET_I386)
-info->value->arch = CPU_INFO_ARCH_X86;
-info->value->u.x86.pc = env->eip + env->segs[R_CS].base;
-#elif defined(TARGET_PPC)
-info->value->arch = CPU_INFO_ARCH_PPC;
-info->value->u.ppc.nip = env->nip;
-#elif defined(TARGET_SPARC)
-info->value->arch = CPU_INFO_ARCH_SPARC;
-info->value->u.q_sparc.pc = env->pc;
-info->value->u.q_sparc.npc = env->npc;
-#elif defined(TARGET_MIPS)
-info->value->arch = CPU_INFO_ARCH_MIPS;
-info->value->u.q_mips.PC = env->active_tc.PC;
-#elif defined(TARGET_TRICORE)
-info->value->arch = CPU_INFO_ARCH_TRICORE;
-info->value->u.tricore.PC = env->PC;
-#elif defined(TARGET_S390X)
-info->value->arch = CPU_INFO_ARCH_S390;
-info->value->u.s390.cpu_state = env->cpu_state;
-#elif defined(TARGET_RISCV)
-info->value->arch = CPU_INFO_ARCH_RISCV;
-info->value->u.riscv.pc = env->pc;
-#else
-info->value->arch = CPU_INFO_ARCH_OTHER;
-#endif
-info->value->has_props = !!mc->cpu_index_to_instance_props;
-if (info->value->has_props) {
-CpuInstanceProperties *props;
-props = g_malloc0(sizeof(*props));
-*props = mc->cpu_index_to_instance_props(ms, cpu->cpu_index);
-info->value->props = props;
-}
-
-/* XXX: waiting for the qapi to support GSList */
-if (!cur_item) {
-head = cur_item = info;
-} else {
-cur_item->next = info;
-cur_item = info;
-

[Qemu-devel] [PATCH 06/17] qom: Move HMP command handlers to qom/

[Markus Armbruster]

There are just two HMP commands related to QOM: qom-list and qom-set.
Move their handlers from monitor/hmp-cmds.c to new qom/qom-hmp-cmds.c,
where they are covered by MAINTAINERS section QOM.

Cc: Paolo Bonzini 
Cc: "Daniel P. Berrange" 
Cc: Eduardo Habkost 
Cc: "Dr. David Alan Gilbert" 
Signed-off-by: Markus Armbruster 
---
 include/monitor/hmp.h |  2 ++
 monitor/hmp-cmds.c| 50 +-
 qom/Makefile.objs |  2 +-
 qom/qom-hmp-cmds.c| 62 +++
 4 files changed, 66 insertions(+), 50 deletions(-)
 create mode 100644 qom/qom-hmp-cmds.c

diff --git a/include/monitor/hmp.h b/include/monitor/hmp.h
index 1d095d5837..f46ccdaa35 100644
--- a/include/monitor/hmp.h
+++ b/include/monitor/hmp.h
@@ -16,6 +16,8 @@
 
 #include "qemu/readline.h"
 
+void hmp_handle_error(Monitor *mon, Error **errp);
+
 void hmp_info_name(Monitor *mon, const QDict *qdict);
 void hmp_info_version(Monitor *mon, const QDict *qdict);
 void hmp_info_kvm(Monitor *mon, const QDict *qdict);
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
index d94ab7563e..5641036dc3 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
@@ -60,7 +60,7 @@
 #include 
 #endif
 
-static void hmp_handle_error(Monitor *mon, Error **errp)
+void hmp_handle_error(Monitor *mon, Error **errp)
 {
 assert(errp);
 if (*errp) {
@@ -2714,54 +2714,6 @@ void hmp_info_iothreads(Monitor *mon, const QDict *qdict)
 qapi_free_IOThreadInfoList(info_list);
 }
 
-void hmp_qom_list(Monitor *mon, const QDict *qdict)
-{
-const char *path = qdict_get_try_str(qdict, "path");
-ObjectPropertyInfoList *list;
-Error *err = NULL;
-
-if (path == NULL) {
-monitor_printf(mon, "/\n");
-return;
-}
-
-list = qmp_qom_list(path, );
-if (err == NULL) {
-ObjectPropertyInfoList *start = list;
-while (list != NULL) {
-ObjectPropertyInfo *value = list->value;
-
-monitor_printf(mon, "%s (%s)\n",
-   value->name, value->type);
-list = list->next;
-}
-qapi_free_ObjectPropertyInfoList(start);
-}
-hmp_handle_error(mon, );
-}
-
-void hmp_qom_set(Monitor *mon, const QDict *qdict)
-{
-const char *path = qdict_get_str(qdict, "path");
-const char *property = qdict_get_str(qdict, "property");
-const char *value = qdict_get_str(qdict, "value");
-Error *err = NULL;
-bool ambiguous = false;
-Object *obj;
-
-obj = object_resolve_path(path, );
-if (obj == NULL) {
-error_set(, ERROR_CLASS_DEVICE_NOT_FOUND,
-  "Device '%s' not found", path);
-} else {
-if (ambiguous) {
-monitor_printf(mon, "Warning: Path '%s' is ambiguous\n", path);
-}
-object_property_parse(obj, value, property, );
-}
-hmp_handle_error(mon, );
-}
-
 void hmp_rocker(Monitor *mon, const QDict *qdict)
 {
 const char *name = qdict_get_str(qdict, "name");
diff --git a/qom/Makefile.objs b/qom/Makefile.objs
index 5fb43b842c..aae478fc21 100644
--- a/qom/Makefile.objs
+++ b/qom/Makefile.objs
@@ -2,4 +2,4 @@ qom-obj-y = object.o container.o qom-qobject.o
 qom-obj-y += object_interfaces.o
 
 common-obj-y = cpu.o
-common-obj-$(CONFIG_SOFTMMU) += qom-qmp-cmds.o
+common-obj-$(CONFIG_SOFTMMU) += qom-hmp-cmds.o qom-qmp-cmds.o
diff --git a/qom/qom-hmp-cmds.c b/qom/qom-hmp-cmds.c
new file mode 100644
index 00..2028a21052
--- /dev/null
+++ b/qom/qom-hmp-cmds.c
@@ -0,0 +1,62 @@
+/*
+ * HMP commands related to QOM
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later.  See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "monitor/hmp.h"
+#include "qapi/qapi-commands-qom.h"
+#include "qapi/qmp/qdict.h"
+#include "monitor/monitor.h"
+#include "qom/object.h"
+#include "qapi/error.h"
+
+void hmp_qom_list(Monitor *mon, const QDict *qdict)
+{
+const char *path = qdict_get_try_str(qdict, "path");
+ObjectPropertyInfoList *list;
+Error *err = NULL;
+
+if (path == NULL) {
+monitor_printf(mon, "/\n");
+return;
+}
+
+list = qmp_qom_list(path, );
+if (err == NULL) {
+ObjectPropertyInfoList *start = list;
+while (list != NULL) {
+ObjectPropertyInfo *value = list->value;
+
+monitor_printf(mon, "%s (%s)\n",
+   value->name, value->type);
+list = list->next;
+}
+qapi_free_ObjectPropertyInfoList(start);
+}
+hmp_handle_error(mon, );
+}
+
+void hmp_qom_set(Monitor *mon, const QDict *qdict)
+{
+const char *path = qdict_get_str(qdict, "path");
+const char *property = qdict_get_str(qdict, "property");
+const char *value = qdict_get_str(qdict, "value");
+Error *err = NULL;
+bool ambiguous = false;
+Object *obj;
+
+obj = object_resolve_path(path, );
+if (obj == NULL) {
+error_set(, ERROR_CLASS_DEVICE_NOT_FOUND,

[Qemu-devel] [PATCH 12/17] qapi: Split machine-target.json off target.json and misc.json

[Markus Armbruster]

Move commands query-cpu-definitions, query-cpu-model-baseline,
query-cpu-model-comparison, and query-cpu-model-expansion with their
types from target.json to machine-target.json.  Also move typee
CpuModelInfo, CpuModelExpansionType, and CpuModelCompareResult from
misc.json there.  Add machine-target.json to MAINTAINERS section
"Machine core".

Cc: Eduardo Habkost 
Cc: Marcel Apfelbaum 
Signed-off-by: Markus Armbruster 
---
 MAINTAINERS |   1 +
 qapi/Makefile.objs  |   2 +-
 qapi/machine-target.json| 318 
 qapi/misc.json  |  74 
 qapi/qapi-schema.json   |   1 +
 qapi/target.json| 242 
 target/arm/helper.c |   2 +-
 target/i386/cpu.c   |   2 +-
 target/mips/helper.c|   2 +-
 target/ppc/translate_init.inc.c |   2 +-
 target/s390x/cpu_models.c   |   2 +-
 11 files changed, 326 insertions(+), 322 deletions(-)
 create mode 100644 qapi/machine-target.json

diff --git a/MAINTAINERS b/MAINTAINERS
index 0e6d0861ab..89e8c01cf3 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1271,6 +1271,7 @@ F: hw/core/null-machine.c
 F: hw/core/numa.c
 F: hw/cpu/cluster.c
 F: qapi/machine.json
+F: qapi/machine-target.json
 F: qom/cpu.c
 F: include/hw/boards.h
 F: include/hw/cpu/cluster.h
diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
index 01dced01aa..4e87bef6e1 100644
--- a/qapi/Makefile.objs
+++ b/qapi/Makefile.objs
@@ -9,7 +9,7 @@ QAPI_COMMON_MODULES = audio authz block-core block char common 
crypto
 QAPI_COMMON_MODULES += introspect job machine migration misc net
 QAPI_COMMON_MODULES += qdev qom rdma rocker run-state sockets tpm
 QAPI_COMMON_MODULES += trace transaction ui
-QAPI_TARGET_MODULES = target
+QAPI_TARGET_MODULES = machine-target target
 QAPI_MODULES = $(QAPI_COMMON_MODULES) $(QAPI_TARGET_MODULES)
 
 util-obj-y += qapi-builtin-types.o
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
new file mode 100644
index 00..5d7480f6ab
--- /dev/null
+++ b/qapi/machine-target.json
@@ -0,0 +1,318 @@
+# -*- Mode: Python -*-
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+
+##
+# @CpuModelInfo:
+#
+# Virtual CPU model.
+#
+# A CPU model consists of the name of a CPU definition, to which
+# delta changes are applied (e.g. features added/removed). Most magic values
+# that an architecture might require should be hidden behind the name.
+# However, if required, architectures can expose relevant properties.
+#
+# @name: the name of the CPU definition the model is based on
+# @props: a dictionary of QOM properties to be applied
+#
+# Since: 2.8.0
+##
+{ 'struct': 'CpuModelInfo',
+  'data': { 'name': 'str',
+'*props': 'any' } }
+
+##
+# @CpuModelExpansionType:
+#
+# An enumeration of CPU model expansion types.
+#
+# @static: Expand to a static CPU model, a combination of a static base
+#  model name and property delta changes. As the static base model will
+#  never change, the expanded CPU model will be the same, independent 
of
+#  QEMU version, machine type, machine options, and accelerator 
options.
+#  Therefore, the resulting model can be used by tooling without having
+#  to specify a compatibility machine - e.g. when displaying the "host"
+#  model. The @static CPU models are migration-safe.
+
+# @full: Expand all properties. The produced model is not guaranteed to be
+#migration-safe, but allows tooling to get an insight and work with
+#model details.
+#
+# Note: When a non-migration-safe CPU model is expanded in static mode, some
+# features enabled by the CPU model may be omitted, because they can't be
+# implemented by a static CPU model definition (e.g. cache info passthrough and
+# PMU passthrough in x86). If you need an accurate representation of the
+# features enabled by a non-migration-safe CPU model, use @full. If you need a
+# static representation that will keep ABI compatibility even when changing 
QEMU
+# version or machine-type, use @static (but keep in mind that some features may
+# be omitted).
+#
+# Since: 2.8.0
+##
+{ 'enum': 'CpuModelExpansionType',
+  'data': [ 'static', 'full' ] }
+
+
+##
+# @CpuModelCompareResult:
+#
+# An enumeration of CPU model comparison results. The result is usually
+# calculated using e.g. CPU features or CPU generations.
+#
+# @incompatible: If model A is incompatible to model B, model A is not
+#guaranteed to run where model B runs and the other way around.
+#
+# @identical: If model A is identical to model B, model A is guaranteed to run
+# where model B runs and the other way around.
+#
+# @superset: If model A is a superset of model B, model B is guaranteed to run
+#where model A runs. There are no guarantees about the other way.
+#
+# @subset: If model A is a subset of 

[Qemu-devel] [PATCH 14/17] qapi: Split dump.json off misc.json

[Markus Armbruster]

Move commands dump-guest-memory, query-dump,
query-dump-guest-memory-capability with their types from misc.json to
new dump.json.  Add dump.json to MAINTAINERS section "Dump".

Cc: Marc-André Lureau 
Signed-off-by: Markus Armbruster 
---
 MAINTAINERS   |   1 +
 dump.c|   4 +-
 include/sysemu/dump.h |   2 +-
 monitor/hmp-cmds.c|   1 +
 qapi/Makefile.objs|   2 +-
 qapi/dump.json| 200 ++
 qapi/misc.json| 192 
 qapi/qapi-schema.json |   1 +
 8 files changed, 207 insertions(+), 196 deletions(-)
 create mode 100644 qapi/dump.json

diff --git a/MAINTAINERS b/MAINTAINERS
index 89e8c01cf3..ea110e99ca 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1855,6 +1855,7 @@ F: hw/misc/vmcoreinfo.c
 F: include/hw/misc/vmcoreinfo.h
 F: include/sysemu/dump-arch.h
 F: include/sysemu/dump.h
+F: qapi/dump.json
 F: scripts/dump-guest-memory.py
 F: stubs/dump.c
 
diff --git a/dump.c b/dump.c
index e99554628c..c7b2301652 100644
--- a/dump.c
+++ b/dump.c
@@ -24,8 +24,8 @@
 #include "sysemu/memory_mapping.h"
 #include "sysemu/cpus.h"
 #include "qapi/error.h"
-#include "qapi/qapi-commands-misc.h"
-#include "qapi/qapi-events-misc.h"
+#include "qapi/qapi-commands-dump.h"
+#include "qapi/qapi-events-dump.h"
 #include "qapi/qmp/qerror.h"
 #include "qemu/error-report.h"
 #include "hw/misc/vmcoreinfo.h"
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index d824bc0941..250143cb5a 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -14,7 +14,7 @@
 #ifndef DUMP_H
 #define DUMP_H
 
-#include "qapi/qapi-types-misc.h"
+#include "qapi/qapi-types-dump.h"
 
 #define MAKEDUMPFILE_SIGNATURE  "makedumpfile"
 #define MAX_SIZE_MDF_HEADER (4096) /* max size of makedumpfile_header 
*/
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
index ea8ae2966e..18ffeb7017 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
@@ -31,6 +31,7 @@
 #include "qapi/qapi-builtin-visit.h"
 #include "qapi/qapi-commands-block.h"
 #include "qapi/qapi-commands-char.h"
+#include "qapi/qapi-commands-dump.h"
 #include "qapi/qapi-commands-migration.h"
 #include "qapi/qapi-commands-misc.h"
 #include "qapi/qapi-commands-net.h"
diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
index c0be6fcd3e..c5a29e86e2 100644
--- a/qapi/Makefile.objs
+++ b/qapi/Makefile.objs
@@ -6,7 +6,7 @@ util-obj-y += qmp-event.o
 util-obj-y += qapi-util.o
 
 QAPI_COMMON_MODULES = audio authz block-core block char common crypto
-QAPI_COMMON_MODULES += introspect job machine migration misc net
+QAPI_COMMON_MODULES += dump introspect job machine migration misc net
 QAPI_COMMON_MODULES += qdev qom rdma rocker run-state sockets tpm
 QAPI_COMMON_MODULES += trace transaction ui
 QAPI_TARGET_MODULES = machine-target misc-target
diff --git a/qapi/dump.json b/qapi/dump.json
new file mode 100644
index 00..2b35409a7b
--- /dev/null
+++ b/qapi/dump.json
@@ -0,0 +1,200 @@
+# -*- Mode: Python -*-
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+
+##
+# = Dump guest memory
+##
+
+##
+# @DumpGuestMemoryFormat:
+#
+# An enumeration of guest-memory-dump's format.
+#
+# @elf: elf format
+#
+# @kdump-zlib: kdump-compressed format with zlib-compressed
+#
+# @kdump-lzo: kdump-compressed format with lzo-compressed
+#
+# @kdump-snappy: kdump-compressed format with snappy-compressed
+#
+# @win-dmp: Windows full crashdump format,
+#   can be used instead of ELF converting (since 2.13)
+#
+# Since: 2.0
+##
+{ 'enum': 'DumpGuestMemoryFormat',
+  'data': [ 'elf', 'kdump-zlib', 'kdump-lzo', 'kdump-snappy', 'win-dmp' ] }
+
+##
+# @dump-guest-memory:
+#
+# Dump guest's memory to vmcore. It is a synchronous operation that can take
+# very long depending on the amount of guest memory.
+#
+# @paging: if true, do paging to get guest's memory mapping. This allows
+#  using gdb to process the core file.
+#
+#  IMPORTANT: this option can make QEMU allocate several gigabytes
+# of RAM. This can happen for a large guest, or a
+# malicious guest pretending to be large.
+#
+#  Also, paging=true has the following limitations:
+#
+# 1. The guest may be in a catastrophic state or can have corrupted
+#memory, which cannot be trusted
+# 2. The guest can be in real-mode even if paging is enabled. For
+#example, the guest uses ACPI to sleep, and ACPI sleep state
+#goes in real-mode
+# 3. Currently only supported on i386 and x86_64.
+#
+# @protocol: the filename or file descriptor of the vmcore. The supported
+#protocols are:
+#
+#1. file: the protocol starts with "file:", and the following
+#   string is the file's path.
+#2. fd: the protocol starts with "fd:", and the following string
+#   

[Qemu-devel] [PATCH 08/17] qapi: Split machine.json off misc.json

[Markus Armbruster]

Move commands cpu-add, query-cpus, query-cpus-fast,
query-current-machine, query-hotpluggable-cpus, query-machines,
query-memdev, and set-numa-node with their types from misc.json to new
machine.json.  Also move types X86CPURegister32 and
X86CPUFeatureWordInfo.  Add machine.json to MAINTAINERS section
"Machine core".

Cc: Eduardo Habkost 
Cc: Marcel Apfelbaum 
Signed-off-by: Markus Armbruster 
---
 MAINTAINERS  |   1 +
 cpus.c   |   1 +
 include/hw/boards.h  |   1 +
 include/sysemu/hostmem.h |   2 +-
 monitor/hmp-cmds.c   |   1 +
 monitor/qmp-cmds.c   |   1 +
 numa.c   |   4 +-
 qapi/Makefile.objs   |   2 +-
 qapi/machine.json| 697 +++
 qapi/misc.json   | 687 --
 qapi/qapi-schema.json|   1 +
 target/i386/cpu.c|   2 +-
 target/s390x/cpu.c   |   2 +-
 target/s390x/sigp.c  |   2 +-
 vl.c |   2 +-
 15 files changed, 711 insertions(+), 695 deletions(-)
 create mode 100644 qapi/machine.json

diff --git a/MAINTAINERS b/MAINTAINERS
index f18fdd..9b488c5151 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1269,6 +1269,7 @@ F: hw/core/machine.c
 F: hw/core/null-machine.c
 F: hw/cpu/cluster.c
 F: numa.c
+F: qapi/machine.json
 F: qom/cpu.c
 F: include/hw/boards.h
 F: include/hw/cpu/cluster.h
diff --git a/cpus.c b/cpus.c
index dde3b7b981..f5e677b2c4 100644
--- a/cpus.c
+++ b/cpus.c
@@ -28,6 +28,7 @@
 #include "cpu.h"
 #include "monitor/monitor.h"
 #include "qapi/error.h"
+#include "qapi/qapi-commands-machine.h"
 #include "qapi/qapi-commands-misc.h"
 #include "qapi/qapi-events-run-state.h"
 #include "qapi/qmp/qerror.h"
diff --git a/include/hw/boards.h b/include/hw/boards.h
index b7362af3f1..0c1cc510b5 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -6,6 +6,7 @@
 #include "sysemu/blockdev.h"
 #include "sysemu/accel.h"
 #include "hw/qdev.h"
+#include "qapi/qapi-types-machine.h"
 #include "qemu/module.h"
 #include "qom/object.h"
 #include "qom/cpu.h"
diff --git a/include/sysemu/hostmem.h b/include/sysemu/hostmem.h
index a023b372a4..92fa0e458c 100644
--- a/include/sysemu/hostmem.h
+++ b/include/sysemu/hostmem.h
@@ -14,7 +14,7 @@
 #define SYSEMU_HOSTMEM_H
 
 #include "sysemu/sysemu.h" /* for MAX_NODES */
-#include "qapi/qapi-types-misc.h"
+#include "qapi/qapi-types-machine.h"
 #include "qom/object.h"
 #include "exec/memory.h"
 #include "qemu/bitmap.h"
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
index 5641036dc3..789f763938 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
@@ -31,6 +31,7 @@
 #include "qapi/qapi-builtin-visit.h"
 #include "qapi/qapi-commands-block.h"
 #include "qapi/qapi-commands-char.h"
+#include "qapi/qapi-commands-machine.h"
 #include "qapi/qapi-commands-migration.h"
 #include "qapi/qapi-commands-misc.h"
 #include "qapi/qapi-commands-net.h"
diff --git a/monitor/qmp-cmds.c b/monitor/qmp-cmds.c
index 884eeaab80..58b54db033 100644
--- a/monitor/qmp-cmds.c
+++ b/monitor/qmp-cmds.c
@@ -31,6 +31,7 @@
 #include "sysemu/block-backend.h"
 #include "qapi/error.h"
 #include "qapi/qapi-commands-block-core.h"
+#include "qapi/qapi-commands-machine.h"
 #include "qapi/qapi-commands-misc.h"
 #include "qapi/qapi-commands-ui.h"
 #include "qapi/qmp/qerror.h"
diff --git a/numa.c b/numa.c
index 955ec0c830..6f85407c46 100644
--- a/numa.c
+++ b/numa.c
@@ -31,8 +31,8 @@
 #include "qemu/error-report.h"
 #include "qapi/error.h"
 #include "qapi/opts-visitor.h"
-#include "qapi/qapi-commands-misc.h"
-#include "qapi/qapi-visit-misc.h"
+#include "qapi/qapi-commands-machine.h"
+#include "qapi/qapi-visit-machine.h"
 #include "hw/boards.h"
 #include "sysemu/hostmem.h"
 #include "hw/mem/pc-dimm.h"
diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
index 40b1dcffc4..01dced01aa 100644
--- a/qapi/Makefile.objs
+++ b/qapi/Makefile.objs
@@ -6,7 +6,7 @@ util-obj-y += qmp-event.o
 util-obj-y += qapi-util.o
 
 QAPI_COMMON_MODULES = audio authz block-core block char common crypto
-QAPI_COMMON_MODULES += introspect job migration misc net
+QAPI_COMMON_MODULES += introspect job machine migration misc net
 QAPI_COMMON_MODULES += qdev qom rdma rocker run-state sockets tpm
 QAPI_COMMON_MODULES += trace transaction ui
 QAPI_TARGET_MODULES = target
diff --git a/qapi/machine.json b/qapi/machine.json
new file mode 100644
index 00..81849acb3a
--- /dev/null
+++ b/qapi/machine.json
@@ -0,0 +1,697 @@
+# -*- Mode: Python -*-
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+
+##
+# = Machines
+##
+
+{ 'include': 'common.json' }
+
+##
+# @CpuInfoArch:
+#
+# An enumeration of cpu types that enable additional information during
+# @query-cpus and @query-cpus-fast.
+#
+# @s390: since 2.12
+#
+# @riscv: since 2.12
+#
+# Since: 2.6
+##
+{ 'enum': 'CpuInfoArch',
+  'data': ['x86', 'sparc', 'ppc', 'mips', 'tricore', 's390', 'riscv', 'other' 
] }
+
+##
+# @CpuInfo:

[Qemu-devel] [PATCH 17/17] dump: Move HMP command handlers to dump/

[Markus Armbruster]

Move the HMP handlers related to qapi/dump.json to
dimp/dump-hmp-cmds.c, where they are covered by MAINTAINERS section
"Dump", just like qapi/dump.json.

Cc: Marc-André Lureau 
Cc: "Dr. David Alan Gilbert" 
Signed-off-by: Markus Armbruster 
---
 Makefile.objs|  1 +
 dump/Makefile.objs   |  1 +
 dump/dump-hmp-cmds.c | 88 
 monitor/hmp-cmds.c   | 76 --
 4 files changed, 90 insertions(+), 76 deletions(-)
 create mode 100644 dump/dump-hmp-cmds.c

diff --git a/Makefile.objs b/Makefile.objs
index 7494d6143b..c93d731047 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -45,6 +45,7 @@ io-obj-y = io/
 ifeq ($(CONFIG_SOFTMMU),y)
 common-obj-y = blockdev.o blockdev-nbd.o block/
 common-obj-y += bootdevice.o iothread.o
+common-obj-y += dump/
 common-obj-y += job-qmp.o
 common-obj-y += monitor/
 common-obj-y += net/
diff --git a/dump/Makefile.objs b/dump/Makefile.objs
index ea6b074967..d2a5db3b81 100644
--- a/dump/Makefile.objs
+++ b/dump/Makefile.objs
@@ -1,2 +1,3 @@
 obj-y += dump.o
+common-obj-y += dump-hmp-cmds.o
 obj-$(TARGET_X86_64) += win_dump.o
diff --git a/dump/dump-hmp-cmds.c b/dump/dump-hmp-cmds.c
new file mode 100644
index 00..3dbf44372c
--- /dev/null
+++ b/dump/dump-hmp-cmds.c
@@ -0,0 +1,88 @@
+/*
+ * Human Monitor Interface commands
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "monitor/hmp.h"
+#include "monitor/monitor.h"
+#include "qapi/error.h"
+#include "qapi/qapi-commands-dump.h"
+#include "qapi/qmp/qdict.h"
+
+void hmp_dump_guest_memory(Monitor *mon, const QDict *qdict)
+{
+Error *err = NULL;
+bool win_dmp = qdict_get_try_bool(qdict, "windmp", false);
+bool paging = qdict_get_try_bool(qdict, "paging", false);
+bool zlib = qdict_get_try_bool(qdict, "zlib", false);
+bool lzo = qdict_get_try_bool(qdict, "lzo", false);
+bool snappy = qdict_get_try_bool(qdict, "snappy", false);
+const char *file = qdict_get_str(qdict, "filename");
+bool has_begin = qdict_haskey(qdict, "begin");
+bool has_length = qdict_haskey(qdict, "length");
+bool has_detach = qdict_haskey(qdict, "detach");
+int64_t begin = 0;
+int64_t length = 0;
+bool detach = false;
+enum DumpGuestMemoryFormat dump_format = DUMP_GUEST_MEMORY_FORMAT_ELF;
+char *prot;
+
+if (zlib + lzo + snappy + win_dmp > 1) {
+error_setg(, "only one of '-z|-l|-s|-w' can be set");
+hmp_handle_error(mon, );
+return;
+}
+
+if (win_dmp) {
+dump_format = DUMP_GUEST_MEMORY_FORMAT_WIN_DMP;
+}
+
+if (zlib) {
+dump_format = DUMP_GUEST_MEMORY_FORMAT_KDUMP_ZLIB;
+}
+
+if (lzo) {
+dump_format = DUMP_GUEST_MEMORY_FORMAT_KDUMP_LZO;
+}
+
+if (snappy) {
+dump_format = DUMP_GUEST_MEMORY_FORMAT_KDUMP_SNAPPY;
+}
+
+if (has_begin) {
+begin = qdict_get_int(qdict, "begin");
+}
+if (has_length) {
+length = qdict_get_int(qdict, "length");
+}
+if (has_detach) {
+detach = qdict_get_bool(qdict, "detach");
+}
+
+prot = g_strconcat("file:", file, NULL);
+
+qmp_dump_guest_memory(paging, prot, true, detach, has_begin, begin,
+  has_length, length, true, dump_format, );
+hmp_handle_error(mon, );
+g_free(prot);
+}
+
+void hmp_info_dump(Monitor *mon, const QDict *qdict)
+{
+DumpQueryResult *result = qmp_query_dump(NULL);
+
+assert(result && result->status < DUMP_STATUS__MAX);
+monitor_printf(mon, "Status: %s\n", DumpStatus_str(result->status));
+
+if (result->status == DUMP_STATUS_ACTIVE) {
+float percent = 0;
+assert(result->total != 0);
+percent = 100.0 * result->completed / result->total;
+monitor_printf(mon, "Finished: %.2f %%\n", percent);
+}
+
+qapi_free_DumpQueryResult(result);
+}
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
index 18ffeb7017..dc12ae6129 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
@@ -31,7 +31,6 @@
 #include "qapi/qapi-builtin-visit.h"
 #include "qapi/qapi-commands-block.h"
 #include "qapi/qapi-commands-char.h"
-#include "qapi/qapi-commands-dump.h"
 #include "qapi/qapi-commands-migration.h"
 #include "qapi/qapi-commands-misc.h"
 #include "qapi/qapi-commands-net.h"
@@ -2160,64 +2159,6 @@ void hmp_device_del(Monitor *mon, const QDict *qdict)
 hmp_handle_error(mon, );
 }
 
-void hmp_dump_guest_memory(Monitor *mon, const QDict *qdict)
-{
-Error *err = NULL;
-bool win_dmp = qdict_get_try_bool(qdict, "windmp", false);
-bool paging = qdict_get_try_bool(qdict, "paging", false);
-bool zlib = qdict_get_try_bool(qdict, "zlib", false);
-bool lzo = qdict_get_try_bool(qdict, "lzo", false);
-bool snappy = qdict_get_try_bool(qdict, "snappy", false);
-const char *file = qdict_get_str(qdict, "filename");
-bool has_begin 

[Qemu-devel] [PATCH 04/17] qapi: Split qom.json and qdev.json off misc.json

[Markus Armbruster]

Move commands object-add, object-del, qom-get, qom-list,
qom-list-properties, qom-list-types, and qom-set with their types from
misc.json to new qom.json.  Add qom.json to MAINTAINERS section QOM.

Move commands device-list-properties, device_add, device-del, and
event DEVICE_DELETED from misc.json to new qdev.json.  qdev.json
remains uncovered by MAINTAINERS, like the rest of qdev.

Cc: Paolo Bonzini 
Cc: "Daniel P. Berrange" 
Cc: Eduardo Habkost 
Signed-off-by: Markus Armbruster 
---
 MAINTAINERS   |   1 +
 hw/block/xen-block.c  |   2 +-
 hw/core/qdev.c|   2 +-
 monitor/hmp-cmds.c|   1 +
 monitor/qmp-cmds.c|   1 +
 qapi/Makefile.objs|   5 +-
 qapi/misc.json| 351 --
 qapi/qapi-schema.json |   2 +
 qapi/qdev.json| 125 +++
 qapi/qom.json | 244 +
 qdev-monitor.c|   2 +-
 11 files changed, 380 insertions(+), 356 deletions(-)
 create mode 100644 qapi/qdev.json
 create mode 100644 qapi/qom.json

diff --git a/MAINTAINERS b/MAINTAINERS
index 63390eadb5..1aef0afaf7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2035,6 +2035,7 @@ R: Eduardo Habkost 
 S: Supported
 F: include/qom/
 X: include/qom/cpu.h
+F: qapi/qom.json
 F: qom/
 X: qom/cpu.c
 F: tests/check-qom-interface.c
diff --git a/hw/block/xen-block.c b/hw/block/xen-block.c
index 4de537aef4..654b79f921 100644
--- a/hw/block/xen-block.c
+++ b/hw/block/xen-block.c
@@ -11,7 +11,7 @@
 #include "qemu/option.h"
 #include "qapi/error.h"
 #include "qapi/qapi-commands-block-core.h"
-#include "qapi/qapi-commands-misc.h"
+#include "qapi/qapi-commands-qom.h"
 #include "qapi/qapi-visit-block-core.h"
 #include "qapi/qobject-input-visitor.h"
 #include "qapi/visitor.h"
diff --git a/hw/core/qdev.c b/hw/core/qdev.c
index f9b6efe509..94ebc0a4a1 100644
--- a/hw/core/qdev.c
+++ b/hw/core/qdev.c
@@ -29,7 +29,7 @@
 #include "hw/qdev.h"
 #include "sysemu/sysemu.h"
 #include "qapi/error.h"
-#include "qapi/qapi-events-misc.h"
+#include "qapi/qapi-events-qdev.h"
 #include "qapi/qmp/qerror.h"
 #include "qapi/visitor.h"
 #include "qemu/error-report.h"
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
index 5082fcd016..d94ab7563e 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
@@ -34,6 +34,7 @@
 #include "qapi/qapi-commands-migration.h"
 #include "qapi/qapi-commands-misc.h"
 #include "qapi/qapi-commands-net.h"
+#include "qapi/qapi-commands-qdev.h"
 #include "qapi/qapi-commands-rocker.h"
 #include "qapi/qapi-commands-run-state.h"
 #include "qapi/qapi-commands-tpm.h"
diff --git a/monitor/qmp-cmds.c b/monitor/qmp-cmds.c
index f1b1e4f08b..ae7f201dc0 100644
--- a/monitor/qmp-cmds.c
+++ b/monitor/qmp-cmds.c
@@ -34,6 +34,7 @@
 #include "qapi/error.h"
 #include "qapi/qapi-commands-block-core.h"
 #include "qapi/qapi-commands-misc.h"
+#include "qapi/qapi-commands-qdev.h"
 #include "qapi/qapi-commands-ui.h"
 #include "qapi/qmp/qdict.h"
 #include "qapi/qmp/qerror.h"
diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
index 729e5185c5..40b1dcffc4 100644
--- a/qapi/Makefile.objs
+++ b/qapi/Makefile.objs
@@ -6,8 +6,9 @@ util-obj-y += qmp-event.o
 util-obj-y += qapi-util.o
 
 QAPI_COMMON_MODULES = audio authz block-core block char common crypto
-QAPI_COMMON_MODULES += introspect job migration misc net rdma rocker
-QAPI_COMMON_MODULES += run-state sockets tpm trace transaction ui
+QAPI_COMMON_MODULES += introspect job migration misc net
+QAPI_COMMON_MODULES += qdev qom rdma rocker run-state sockets tpm
+QAPI_COMMON_MODULES += trace transaction ui
 QAPI_TARGET_MODULES = target
 QAPI_MODULES = $(QAPI_COMMON_MODULES) $(QAPI_TARGET_MODULES)
 
diff --git a/qapi/misc.json b/qapi/misc.json
index dc4cf9da20..5fbda70ad8 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -1342,140 +1342,6 @@
   'data': {'command-line': 'str', '*cpu-index': 'int'},
   'returns': 'str' }
 
-##
-# @ObjectPropertyInfo:
-#
-# @name: the name of the property
-#
-# @type: the type of the property.  This will typically come in one of four
-#forms:
-#
-#1) A primitive type such as 'u8', 'u16', 'bool', 'str', or 'double'.
-#   These types are mapped to the appropriate JSON type.
-#
-#2) A child type in the form 'child' where subtype is a qdev
-#   device type name.  Child properties create the composition tree.
-#
-#3) A link type in the form 'link' where subtype is a qdev
-#   device type name.  Link properties form the device model graph.
-#
-# @description: if specified, the description of the property.
-#
-# Since: 1.2
-##
-{ 'struct': 'ObjectPropertyInfo',
-  'data': { 'name': 'str', 'type': 'str', '*description': 'str' } }
-
-##
-# @qom-list:
-#
-# This command will list any properties of a object given a path in the object
-# model.
-#
-# @path: the path within the object model.  See @qom-get for a description of
-#this parameter.
-#
-# Returns: a list of @ObjectPropertyInfo that describe the properties of the
-# 

[Qemu-devel] [PATCH 13/17] qapi: Rename target.json to misc-target.json

[Markus Armbruster]

Signed-off-by: Markus Armbruster 
---
 hw/ppc/spapr_rtc.c | 2 +-
 hw/s390x/s390-skeys.c  | 2 +-
 hw/timer/mc146818rtc.c | 4 ++--
 qapi/Makefile.objs | 2 +-
 qapi/{target.json => misc-target.json} | 4 
 qapi/qapi-schema.json  | 2 +-
 target/arm/monitor.c   | 2 +-
 target/i386/sev_i386.h | 2 +-
 8 files changed, 8 insertions(+), 12 deletions(-)
 rename qapi/{target.json => misc-target.json} (99%)

diff --git a/hw/ppc/spapr_rtc.c b/hw/ppc/spapr_rtc.c
index af1ef30a53..6cf0113b34 100644
--- a/hw/ppc/spapr_rtc.c
+++ b/hw/ppc/spapr_rtc.c
@@ -32,7 +32,7 @@
 #include "sysemu/sysemu.h"
 #include "hw/ppc/spapr.h"
 #include "qapi/error.h"
-#include "qapi/qapi-events-target.h"
+#include "qapi/qapi-events-misc-target.h"
 #include "qemu/cutils.h"
 #include "qemu/module.h"
 
diff --git a/hw/s390x/s390-skeys.c b/hw/s390x/s390-skeys.c
index daac936698..e5bd92c0c7 100644
--- a/hw/s390x/s390-skeys.c
+++ b/hw/s390x/s390-skeys.c
@@ -14,7 +14,7 @@
 #include "hw/boards.h"
 #include "hw/s390x/storage-keys.h"
 #include "qapi/error.h"
-#include "qapi/qapi-commands-target.h"
+#include "qapi/qapi-commands-misc-target.h"
 #include "qapi/qmp/qdict.h"
 #include "qemu/error-report.h"
 #include "sysemu/kvm.h"
diff --git a/hw/timer/mc146818rtc.c b/hw/timer/mc146818rtc.c
index 0d79e000d2..ce4550b6f2 100644
--- a/hw/timer/mc146818rtc.c
+++ b/hw/timer/mc146818rtc.c
@@ -33,8 +33,8 @@
 #include "sysemu/replay.h"
 #include "hw/timer/mc146818rtc.h"
 #include "qapi/error.h"
-#include "qapi/qapi-commands-target.h"
-#include "qapi/qapi-events-target.h"
+#include "qapi/qapi-commands-misc-target.h"
+#include "qapi/qapi-events-misc-target.h"
 #include "qapi/visitor.h"
 #include "exec/address-spaces.h"
 
diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
index 4e87bef6e1..c0be6fcd3e 100644
--- a/qapi/Makefile.objs
+++ b/qapi/Makefile.objs
@@ -9,7 +9,7 @@ QAPI_COMMON_MODULES = audio authz block-core block char common 
crypto
 QAPI_COMMON_MODULES += introspect job machine migration misc net
 QAPI_COMMON_MODULES += qdev qom rdma rocker run-state sockets tpm
 QAPI_COMMON_MODULES += trace transaction ui
-QAPI_TARGET_MODULES = machine-target target
+QAPI_TARGET_MODULES = machine-target misc-target
 QAPI_MODULES = $(QAPI_COMMON_MODULES) $(QAPI_TARGET_MODULES)
 
 util-obj-y += qapi-builtin-types.o
diff --git a/qapi/target.json b/qapi/misc-target.json
similarity index 99%
rename from qapi/target.json
rename to qapi/misc-target.json
index e0a73581d8..a00fd821eb 100644
--- a/qapi/target.json
+++ b/qapi/misc-target.json
@@ -1,10 +1,6 @@
 # -*- Mode: Python -*-
 #
 
-##
-# = Target-specific commands & events
-##
-
 ##
 # @RTC_CHANGE:
 #
diff --git a/qapi/qapi-schema.json b/qapi/qapi-schema.json
index 8dc82fde31..bcfac85074 100644
--- a/qapi/qapi-schema.json
+++ b/qapi/qapi-schema.json
@@ -102,5 +102,5 @@
 { 'include': 'machine.json' }
 { 'include': 'machine-target.json' }
 { 'include': 'misc.json' }
-{ 'include': 'target.json' }
+{ 'include': 'misc-target.json' }
 { 'include': 'audio.json' }
diff --git a/target/arm/monitor.c b/target/arm/monitor.c
index 41b32b94b2..6ec6dd04ac 100644
--- a/target/arm/monitor.c
+++ b/target/arm/monitor.c
@@ -23,7 +23,7 @@
 #include "qemu/osdep.h"
 #include "hw/boards.h"
 #include "kvm_arm.h"
-#include "qapi/qapi-commands-target.h"
+#include "qapi/qapi-commands-misc-target.h"
 
 static GICCapability *gic_cap_new(int version)
 {
diff --git a/target/i386/sev_i386.h b/target/i386/sev_i386.h
index c0f9373beb..55313441ae 100644
--- a/target/i386/sev_i386.h
+++ b/target/i386/sev_i386.h
@@ -19,7 +19,7 @@
 #include "sysemu/kvm.h"
 #include "sysemu/sev.h"
 #include "qemu/error-report.h"
-#include "qapi/qapi-commands-target.h"
+#include "qapi/qapi-commands-misc-target.h"
 
 #define SEV_POLICY_NODBG0x1
 #define SEV_POLICY_NOKS 0x2
-- 
2.21.0

[Qemu-devel] [PATCH 03/17] hmp: Move hmp.h to include/monitor/

[Markus Armbruster]

Cc: "Dr. David Alan Gilbert" 
Signed-off-by: Markus Armbruster 
---
 hmp.h => include/monitor/hmp.h | 0
 monitor/hmp-cmds.c | 2 +-
 monitor/misc.c | 2 +-
 target/i386/monitor.c  | 2 +-
 target/nios2/monitor.c | 2 +-
 target/ppc/monitor.c   | 2 +-
 target/sh4/monitor.c   | 2 +-
 target/sparc/monitor.c | 2 +-
 target/xtensa/monitor.c| 2 +-
 9 files changed, 8 insertions(+), 8 deletions(-)
 rename hmp.h => include/monitor/hmp.h (100%)

diff --git a/hmp.h b/include/monitor/hmp.h
similarity index 100%
rename from hmp.h
rename to include/monitor/hmp.h
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
index c283dde0e9..5082fcd016 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
@@ -14,7 +14,7 @@
  */
 
 #include "qemu/osdep.h"
-#include "hmp.h"
+#include "monitor/hmp.h"
 #include "net/net.h"
 #include "net/eth.h"
 #include "chardev/char.h"
diff --git a/monitor/misc.c b/monitor/misc.c
index bf9faceb86..2fb6896e84 100644
--- a/monitor/misc.c
+++ b/monitor/misc.c
@@ -56,13 +56,13 @@
 #include "qom/object_interfaces.h"
 #include "trace/control.h"
 #include "monitor/hmp-target.h"
+#include "monitor/hmp.h"
 #ifdef CONFIG_TRACE_SIMPLE
 #include "trace/simple.h"
 #endif
 #include "exec/memory.h"
 #include "exec/exec-all.h"
 #include "qemu/option.h"
-#include "hmp.h"
 #include "qemu/thread.h"
 #include "block/qapi.h"
 #include "qapi/qapi-commands.h"
diff --git a/target/i386/monitor.c b/target/i386/monitor.c
index 56e2dbece7..1f3b532fc2 100644
--- a/target/i386/monitor.c
+++ b/target/i386/monitor.c
@@ -26,11 +26,11 @@
 #include "cpu.h"
 #include "monitor/monitor.h"
 #include "monitor/hmp-target.h"
+#include "monitor/hmp.h"
 #include "qapi/qmp/qdict.h"
 #include "hw/i386/pc.h"
 #include "sysemu/kvm.h"
 #include "sysemu/sev.h"
-#include "hmp.h"
 #include "qapi/error.h"
 #include "sev_i386.h"
 #include "qapi/qapi-commands-misc.h"
diff --git a/target/nios2/monitor.c b/target/nios2/monitor.c
index d5e3393716..6646836df5 100644
--- a/target/nios2/monitor.c
+++ b/target/nios2/monitor.c
@@ -25,7 +25,7 @@
 #include "cpu.h"
 #include "monitor/monitor.h"
 #include "monitor/hmp-target.h"
-#include "hmp.h"
+#include "monitor/hmp.h"
 
 void hmp_info_tlb(Monitor *mon, const QDict *qdict)
 {
diff --git a/target/ppc/monitor.c b/target/ppc/monitor.c
index 7f8360d903..a5a177d717 100644
--- a/target/ppc/monitor.c
+++ b/target/ppc/monitor.c
@@ -27,7 +27,7 @@
 #include "monitor/monitor.h"
 #include "qemu/ctype.h"
 #include "monitor/hmp-target.h"
-#include "hmp.h"
+#include "monitor/hmp.h"
 
 static target_long monitor_get_ccr(const struct MonitorDef *md, int val)
 {
diff --git a/target/sh4/monitor.c b/target/sh4/monitor.c
index 4c7f36c9cc..918a5ccfc6 100644
--- a/target/sh4/monitor.c
+++ b/target/sh4/monitor.c
@@ -25,7 +25,7 @@
 #include "cpu.h"
 #include "monitor/monitor.h"
 #include "monitor/hmp-target.h"
-#include "hmp.h"
+#include "monitor/hmp.h"
 
 static void print_tlb(Monitor *mon, int idx, tlb_t *tlb)
 {
diff --git a/target/sparc/monitor.c b/target/sparc/monitor.c
index 3ec3b51a3d..a7ea287cbc 100644
--- a/target/sparc/monitor.c
+++ b/target/sparc/monitor.c
@@ -25,7 +25,7 @@
 #include "cpu.h"
 #include "monitor/monitor.h"
 #include "monitor/hmp-target.h"
-#include "hmp.h"
+#include "monitor/hmp.h"
 
 
 void hmp_info_tlb(Monitor *mon, const QDict *qdict)
diff --git a/target/xtensa/monitor.c b/target/xtensa/monitor.c
index cf7957bb63..608173c238 100644
--- a/target/xtensa/monitor.c
+++ b/target/xtensa/monitor.c
@@ -25,7 +25,7 @@
 #include "cpu.h"
 #include "monitor/monitor.h"
 #include "monitor/hmp-target.h"
-#include "hmp.h"
+#include "monitor/hmp.h"
 
 void hmp_info_tlb(Monitor *mon, const QDict *qdict)
 {
-- 
2.21.0

[Qemu-devel] [PATCH 05/17] qom: Move QMP command handlers to qom/

[Markus Armbruster]

The handlers for qapi/qom.json's QMP commands are in
monitor/qmp-cmds.c.  Move them to new qom/qom-qmp-cmds.c, where they
are covered by MAINTAINERS section QOM, just like qapi/qom.json.

Move along qmp_device_list_properties() even though it's specified in
qapi/qdev.json, because it's so similar to qmp_qom_list_properties().

Cc: Paolo Bonzini 
Cc: "Daniel P. Berrange" 
Cc: Eduardo Habkost 
Signed-off-by: Markus Armbruster 
---
 monitor/qmp-cmds.c | 303 --
 qom/Makefile.objs  |   1 +
 qom/qom-qmp-cmds.c | 323 +
 3 files changed, 324 insertions(+), 303 deletions(-)
 create mode 100644 qom/qom-qmp-cmds.c

diff --git a/monitor/qmp-cmds.c b/monitor/qmp-cmds.c
index ae7f201dc0..884eeaab80 100644
--- a/monitor/qmp-cmds.c
+++ b/monitor/qmp-cmds.c
@@ -27,20 +27,14 @@
 #include "ui/vnc.h"
 #include "sysemu/kvm.h"
 #include "sysemu/arch_init.h"
-#include "hw/qdev.h"
 #include "sysemu/blockdev.h"
 #include "sysemu/block-backend.h"
-#include "qom/qom-qobject.h"
 #include "qapi/error.h"
 #include "qapi/qapi-commands-block-core.h"
 #include "qapi/qapi-commands-misc.h"
-#include "qapi/qapi-commands-qdev.h"
 #include "qapi/qapi-commands-ui.h"
-#include "qapi/qmp/qdict.h"
 #include "qapi/qmp/qerror.h"
-#include "qapi/qobject-input-visitor.h"
 #include "hw/boards.h"
-#include "qom/object_interfaces.h"
 #include "hw/mem/memory-device.h"
 #include "hw/acpi/acpi_dev_interface.h"
 
@@ -197,69 +191,6 @@ void qmp_system_wakeup(Error **errp)
 qemu_system_wakeup_request(QEMU_WAKEUP_REASON_OTHER, errp);
 }
 
-ObjectPropertyInfoList *qmp_qom_list(const char *path, Error **errp)
-{
-Object *obj;
-bool ambiguous = false;
-ObjectPropertyInfoList *props = NULL;
-ObjectProperty *prop;
-ObjectPropertyIterator iter;
-
-obj = object_resolve_path(path, );
-if (obj == NULL) {
-if (ambiguous) {
-error_setg(errp, "Path '%s' is ambiguous", path);
-} else {
-error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
-  "Device '%s' not found", path);
-}
-return NULL;
-}
-
-object_property_iter_init(, obj);
-while ((prop = object_property_iter_next())) {
-ObjectPropertyInfoList *entry = g_malloc0(sizeof(*entry));
-
-entry->value = g_malloc0(sizeof(ObjectPropertyInfo));
-entry->next = props;
-props = entry;
-
-entry->value->name = g_strdup(prop->name);
-entry->value->type = g_strdup(prop->type);
-}
-
-return props;
-}
-
-void qmp_qom_set(const char *path, const char *property, QObject *value,
- Error **errp)
-{
-Object *obj;
-
-obj = object_resolve_path(path, NULL);
-if (!obj) {
-error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
-  "Device '%s' not found", path);
-return;
-}
-
-object_property_set_qobject(obj, value, property, errp);
-}
-
-QObject *qmp_qom_get(const char *path, const char *property, Error **errp)
-{
-Object *obj;
-
-obj = object_resolve_path(path, NULL);
-if (!obj) {
-error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
-  "Device '%s' not found", path);
-return NULL;
-}
-
-return object_property_get_qobject(obj, property, errp);
-}
-
 void qmp_set_password(const char *protocol, const char *password,
   bool has_connected, const char *connected, Error **errp)
 {
@@ -408,208 +339,6 @@ void qmp_change(const char *device, const char *target,
 }
 }
 
-static void qom_list_types_tramp(ObjectClass *klass, void *data)
-{
-ObjectTypeInfoList *e, **pret = data;
-ObjectTypeInfo *info;
-ObjectClass *parent = object_class_get_parent(klass);
-
-info = g_malloc0(sizeof(*info));
-info->name = g_strdup(object_class_get_name(klass));
-info->has_abstract = info->abstract = object_class_is_abstract(klass);
-if (parent) {
-info->has_parent = true;
-info->parent = g_strdup(object_class_get_name(parent));
-}
-
-e = g_malloc0(sizeof(*e));
-e->value = info;
-e->next = *pret;
-*pret = e;
-}
-
-ObjectTypeInfoList *qmp_qom_list_types(bool has_implements,
-   const char *implements,
-   bool has_abstract,
-   bool abstract,
-   Error **errp)
-{
-ObjectTypeInfoList *ret = NULL;
-
-object_class_foreach(qom_list_types_tramp, implements, abstract, );
-
-return ret;
-}
-
-/* Return a DevicePropertyInfo for a qdev property.
- *
- * If a qdev property with the given name does not exist, use the given default
- * type.  If the qdev property info should not be shown, return NULL.
- *
- * The caller must free the return value.
- */
-static ObjectPropertyInfo *make_device_property_info(ObjectClass *klass,
-  const char *name,
-   

[Qemu-devel] [PATCH 11/17] hw/core: Collect HMP command handlers in hw/core/

[Markus Armbruster]

Move the HMP handlers related to qapi/machine.json to
hw/core/machine-hmp-cmds.c, where they are covered by MAINTAINERS
section "Machine core", just like qapi/machine.json.

Cc: Eduardo Habkost 
Cc: Marcel Apfelbaum 
Cc: "Dr. David Alan Gilbert" 
Signed-off-by: Markus Armbruster 
---
 hw/core/Makefile.objs  |   1 +
 hw/core/machine-hmp-cmds.c | 164 +
 include/monitor/hmp.h  |   1 +
 monitor/hmp-cmds.c | 111 -
 monitor/misc.c |  32 +---
 5 files changed, 168 insertions(+), 141 deletions(-)
 create mode 100644 hw/core/machine-hmp-cmds.c

diff --git a/hw/core/Makefile.objs b/hw/core/Makefile.objs
index 99e7abe982..585b734358 100644
--- a/hw/core/Makefile.objs
+++ b/hw/core/Makefile.objs
@@ -25,3 +25,4 @@ common-obj-$(CONFIG_SOFTMMU) += null-machine.o
 
 obj-$(CONFIG_SOFTMMU) += machine-qmp-cmds.o
 obj-$(CONFIG_SOFTMMU) += numa.o
+common-obj-$(CONFIG_SOFTMMU) += machine-hmp-cmds.o
diff --git a/hw/core/machine-hmp-cmds.c b/hw/core/machine-hmp-cmds.c
new file mode 100644
index 00..7fa6075f1e
--- /dev/null
+++ b/hw/core/machine-hmp-cmds.c
@@ -0,0 +1,164 @@
+/*
+ * HMP commands related to machines and CPUs
+ *
+ * Copyright IBM, Corp. 2011
+ *
+ * Authors:
+ *  Anthony Liguori   
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.  See
+ * the COPYING file in the top-level directory.
+ *
+ * Contributions after 2012-01-13 are licensed under the terms of the
+ * GNU GPL, version 2 or (at your option) any later version.
+ */
+
+#include "qemu/osdep.h"
+#include "monitor/hmp.h"
+#include "monitor/monitor.h"
+#include "qapi/error.h"
+#include "qapi/qapi-builtin-visit.h"
+#include "qapi/qapi-commands-machine.h"
+#include "qapi/qmp/qdict.h"
+#include "qapi/string-output-visitor.h"
+#include "qemu/error-report.h"
+#include "sysemu/numa.h"
+
+void hmp_info_cpus(Monitor *mon, const QDict *qdict)
+{
+CpuInfoFastList *cpu_list, *cpu;
+
+cpu_list = qmp_query_cpus_fast(NULL);
+
+for (cpu = cpu_list; cpu; cpu = cpu->next) {
+int active = ' ';
+
+if (cpu->value->cpu_index == monitor_get_cpu_index()) {
+active = '*';
+}
+
+monitor_printf(mon, "%c CPU #%" PRId64 ":", active,
+   cpu->value->cpu_index);
+monitor_printf(mon, " thread_id=%" PRId64 "\n", cpu->value->thread_id);
+}
+
+qapi_free_CpuInfoFastList(cpu_list);
+}
+
+void hmp_cpu_add(Monitor *mon, const QDict *qdict)
+{
+int cpuid;
+Error *err = NULL;
+
+error_report("cpu_add is deprecated, please use device_add instead");
+
+cpuid = qdict_get_int(qdict, "id");
+qmp_cpu_add(cpuid, );
+hmp_handle_error(mon, );
+}
+
+void hmp_hotpluggable_cpus(Monitor *mon, const QDict *qdict)
+{
+Error *err = NULL;
+HotpluggableCPUList *l = qmp_query_hotpluggable_cpus();
+HotpluggableCPUList *saved = l;
+CpuInstanceProperties *c;
+
+if (err != NULL) {
+hmp_handle_error(mon, );
+return;
+}
+
+monitor_printf(mon, "Hotpluggable CPUs:\n");
+while (l) {
+monitor_printf(mon, "  type: \"%s\"\n", l->value->type);
+monitor_printf(mon, "  vcpus_count: \"%" PRIu64 "\"\n",
+   l->value->vcpus_count);
+if (l->value->has_qom_path) {
+monitor_printf(mon, "  qom_path: \"%s\"\n", l->value->qom_path);
+}
+
+c = l->value->props;
+monitor_printf(mon, "  CPUInstance Properties:\n");
+if (c->has_node_id) {
+monitor_printf(mon, "node-id: \"%" PRIu64 "\"\n", c->node_id);
+}
+if (c->has_socket_id) {
+monitor_printf(mon, "socket-id: \"%" PRIu64 "\"\n", 
c->socket_id);
+}
+if (c->has_core_id) {
+monitor_printf(mon, "core-id: \"%" PRIu64 "\"\n", c->core_id);
+}
+if (c->has_thread_id) {
+monitor_printf(mon, "thread-id: \"%" PRIu64 "\"\n", 
c->thread_id);
+}
+
+l = l->next;
+}
+
+qapi_free_HotpluggableCPUList(saved);
+}
+
+void hmp_info_memdev(Monitor *mon, const QDict *qdict)
+{
+Error *err = NULL;
+MemdevList *memdev_list = qmp_query_memdev();
+MemdevList *m = memdev_list;
+Visitor *v;
+char *str;
+
+while (m) {
+v = string_output_visitor_new(false, );
+visit_type_uint16List(v, NULL, >value->host_nodes, NULL);
+monitor_printf(mon, "memory backend: %s\n", m->value->id);
+monitor_printf(mon, "  size:  %" PRId64 "\n", m->value->size);
+monitor_printf(mon, "  merge: %s\n",
+   m->value->merge ? "true" : "false");
+monitor_printf(mon, "  dump: %s\n",
+   m->value->dump ? "true" : "false");
+monitor_printf(mon, "  prealloc: %s\n",
+   m->value->prealloc ? "true" : "false");
+monitor_printf(mon, "  policy: %s\n",
+   HostMemPolicy_str(m->value->policy));
+

[Qemu-devel] [PATCH 07/17] MAINTAINERS: Merge sections CPU, NUMA into Machine core

[Markus Armbruster]

Suggested-by: Paolo Bonzini 
Cc: Eduardo Habkost 
Cc: Marcel Apfelbaum 
Signed-off-by: Markus Armbruster 
---
 MAINTAINERS | 16 
 1 file changed, 4 insertions(+), 12 deletions(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index 1aef0afaf7..f18fdd 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1268,8 +1268,12 @@ S: Supported
 F: hw/core/machine.c
 F: hw/core/null-machine.c
 F: hw/cpu/cluster.c
+F: numa.c
+F: qom/cpu.c
 F: include/hw/boards.h
 F: include/hw/cpu/cluster.h
+F: include/qom/cpu.h
+F: include/sysemu/numa.h
 T: git https://github.com/ehabkost/qemu.git machine-next
 
 Xtensa Machines
@@ -1833,11 +1837,6 @@ M: Markus Armbruster 
 S: Supported
 F: scripts/coverity-model.c
 
-CPU
-S: Supported
-F: qom/cpu.c
-F: include/qom/cpu.h
-
 Device Tree
 M: Alistair Francis 
 R: David Gibson 
@@ -1945,13 +1944,6 @@ W: http://info.iet.unipi.it/~luigi/netmap/
 S: Maintained
 F: net/netmap.c
 
-NUMA
-M: Eduardo Habkost 
-S: Maintained
-F: numa.c
-F: include/sysemu/numa.h
-T: git https://github.com/ehabkost/qemu.git machine-next
-
 Host Memory Backends
 M: Eduardo Habkost 
 M: Igor Mammedov 
-- 
2.21.0

[Qemu-devel] [PATCH 00/17] Move QOM, qdev, machine core and dump code

[Markus Armbruster]

Split qom.json, qdev.json, machine.json, machine-target.json and
dump.json off misc.json, so MAINTAINERS can cover them properly.

Move their QMP and HMP code similarly.

Move some code out of the the tree's root directory to better places.

Markus Armbruster (16):
  Makefile: Don't add monitor/ twice to common-obj-y
  hmp: Move hmp.h to include/monitor/
  qapi: Split qom.json and qdev.json off misc.json
  qom: Move QMP command handlers to qom/
  qom: Move HMP command handlers to qom/
  MAINTAINERS: Merge sections CPU, NUMA into Machine core
  qapi: Split machine.json off misc.json
  hw/core: Move numa.c to hw/core/
  hw/core: Collect QMP command handlers in hw/core/
  hw/core: Collect HMP command handlers in hw/core/
  qapi: Split machine-target.json off target.json and misc.json
  qapi: Rename target.json to misc-target.json
  qapi: Split dump.json off misc.json
  dump: Move the code to dump/
  MAINTAINERS: Add Windows dump to section "Dump"
  dump: Move HMP command handlers to dump/

Paolo Bonzini (1):
  MAINTAINERS: new maintainers for QOM

 MAINTAINERS   |   29 +-
 Makefile.objs |2 +-
 Makefile.target   |5 +-
 cpus.c|  187 ---
 dump/Makefile.objs|3 +
 dump/dump-hmp-cmds.c  |   88 ++
 dump.c => dump/dump.c |4 +-
 win_dump.c => dump/win_dump.c |0
 win_dump.h => dump/win_dump.h |0
 hw/block/xen-block.c  |2 +-
 hw/core/Makefile.objs |4 +
 hw/core/machine-hmp-cmds.c|  164 +++
 hw/core/machine-qmp-cmds.c|  328 ++
 numa.c => hw/core/numa.c  |   64 +-
 hw/core/qdev.c|2 +-
 hw/ppc/spapr_rtc.c|2 +-
 hw/s390x/s390-skeys.c |2 +-
 hw/timer/mc146818rtc.c|4 +-
 include/hw/boards.h   |1 +
 hmp.h => include/monitor/hmp.h|3 +
 include/sysemu/dump.h |2 +-
 include/sysemu/hostmem.h  |2 +-
 include/sysemu/numa.h |2 +
 monitor/hmp-cmds.c|  238 +---
 monitor/misc.c|   47 +-
 monitor/qmp-cmds.c|  315 +
 qapi/Makefile.objs|7 +-
 qapi/dump.json|  200 
 qapi/{target.json => machine-target.json} |  304 +
 qapi/machine.json |  697 +++
 qapi/misc-target.json |  268 +
 qapi/misc.json| 1304 -
 qapi/qapi-schema.json |7 +-
 qapi/qdev.json|  125 ++
 qapi/qom.json |  244 
 qdev-monitor.c|2 +-
 qom/Makefile.objs |1 +
 qom/qom-hmp-cmds.c|   62 +
 qom/qom-qmp-cmds.c|  323 +
 target/arm/helper.c   |2 +-
 target/arm/monitor.c  |2 +-
 target/i386/cpu.c |4 +-
 target/i386/monitor.c |2 +-
 target/i386/sev_i386.h|2 +-
 target/mips/helper.c  |2 +-
 target/nios2/monitor.c|2 +-
 target/ppc/monitor.c  |2 +-
 target/ppc/translate_init.inc.c   |2 +-
 target/s390x/cpu.c|2 +-
 target/s390x/cpu_models.c |2 +-
 target/s390x/sigp.c   |2 +-
 target/sh4/monitor.c  |2 +-
 target/sparc/monitor.c|2 +-
 target/xtensa/monitor.c   |2 +-
 vl.c  |   45 -
 55 files changed, 2629 insertions(+), 2492 deletions(-)
 create mode 100644 dump/Makefile.objs
 create mode 100644 dump/dump-hmp-cmds.c
 rename dump.c => dump/dump.c (99%)
 rename win_dump.c => dump/win_dump.c (100%)
 rename win_dump.h => dump/win_dump.h (100%)
 create mode 100644 hw/core/machine-hmp-cmds.c
 create mode 100644 hw/core/machine-qmp-cmds.c
 rename numa.c => hw/core/numa.c (90%)
 rename hmp.h => include/monitor/hmp.h (98%)
 create mode 100644 qapi/dump.json
 rename qapi/{target.json => machine-target.json} (61%)
 create mode 100644 qapi/machine.json
 create mode 100644 qapi/misc-target.json
 create mode 100644 qapi/qdev.json
 create mode 100644 qapi/qom.json
 create mode 100644 qom/qom-hmp-cmds.c
 create mode 100644 qom/qom-qmp-cmds.c

-- 
2.21.0

[Qemu-devel] [PATCH 09/17] hw/core: Move numa.c to hw/core/

[Markus Armbruster]

Cc: Eduardo Habkost 
Cc: Marcel Apfelbaum 
Signed-off-by: Markus Armbruster 
---
 MAINTAINERS  | 2 +-
 Makefile.target  | 2 +-
 hw/core/Makefile.objs| 2 ++
 numa.c => hw/core/numa.c | 0
 4 files changed, 4 insertions(+), 2 deletions(-)
 rename numa.c => hw/core/numa.c (100%)

diff --git a/MAINTAINERS b/MAINTAINERS
index 9b488c5151..96a72a999a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1267,8 +1267,8 @@ M: Marcel Apfelbaum 
 S: Supported
 F: hw/core/machine.c
 F: hw/core/null-machine.c
+F: hw/core/numa.c
 F: hw/cpu/cluster.c
-F: numa.c
 F: qapi/machine.json
 F: qom/cpu.c
 F: include/hw/boards.h
diff --git a/Makefile.target b/Makefile.target
index 72c267f7dc..167ae2174e 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -148,7 +148,7 @@ endif #CONFIG_BSD_USER
 #
 # System emulator target
 ifdef CONFIG_SOFTMMU
-obj-y += arch_init.o cpus.o gdbstub.o balloon.o ioport.o numa.o
+obj-y += arch_init.o cpus.o gdbstub.o balloon.o ioport.o
 obj-y += qtest.o
 obj-y += hw/
 obj-y += monitor/
diff --git a/hw/core/Makefile.objs b/hw/core/Makefile.objs
index a799c83815..e3a8307be6 100644
--- a/hw/core/Makefile.objs
+++ b/hw/core/Makefile.objs
@@ -22,3 +22,5 @@ common-obj-$(CONFIG_SOFTMMU) += split-irq.o
 common-obj-$(CONFIG_PLATFORM_BUS) += platform-bus.o
 common-obj-$(CONFIG_SOFTMMU) += generic-loader.o
 common-obj-$(CONFIG_SOFTMMU) += null-machine.o
+
+obj-$(CONFIG_SOFTMMU) += numa.o
diff --git a/numa.c b/hw/core/numa.c
similarity index 100%
rename from numa.c
rename to hw/core/numa.c
-- 
2.21.0

[Qemu-devel] [PATCH 15/17] dump: Move the code to dump/

[Markus Armbruster]

Cc: Marc-André Lureau 
Signed-off-by: Markus Armbruster 
---
 MAINTAINERS   | 2 +-
 Makefile.target   | 3 +--
 dump/Makefile.objs| 2 ++
 dump.c => dump/dump.c | 0
 win_dump.c => dump/win_dump.c | 0
 win_dump.h => dump/win_dump.h | 0
 6 files changed, 4 insertions(+), 3 deletions(-)
 create mode 100644 dump/Makefile.objs
 rename dump.c => dump/dump.c (100%)
 rename win_dump.c => dump/win_dump.c (100%)
 rename win_dump.h => dump/win_dump.h (100%)

diff --git a/MAINTAINERS b/MAINTAINERS
index ea110e99ca..a3fdda015f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1850,7 +1850,7 @@ F: include/sysemu/device_tree.h
 Dump
 S: Supported
 M: Marc-André Lureau 
-F: dump.c
+F: dump/dump.c
 F: hw/misc/vmcoreinfo.c
 F: include/hw/misc/vmcoreinfo.h
 F: include/sysemu/dump-arch.h
diff --git a/Makefile.target b/Makefile.target
index 167ae2174e..a6919e0caf 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -150,13 +150,12 @@ endif #CONFIG_BSD_USER
 ifdef CONFIG_SOFTMMU
 obj-y += arch_init.o cpus.o gdbstub.o balloon.o ioport.o
 obj-y += qtest.o
+obj-y += dump/
 obj-y += hw/
 obj-y += monitor/
 obj-y += qapi/
 obj-y += memory.o
 obj-y += memory_mapping.o
-obj-y += dump.o
-obj-$(TARGET_X86_64) += win_dump.o
 obj-y += migration/ram.o
 LIBS := $(libs_softmmu) $(LIBS)
 
diff --git a/dump/Makefile.objs b/dump/Makefile.objs
new file mode 100644
index 00..ea6b074967
--- /dev/null
+++ b/dump/Makefile.objs
@@ -0,0 +1,2 @@
+obj-y += dump.o
+obj-$(TARGET_X86_64) += win_dump.o
diff --git a/dump.c b/dump/dump.c
similarity index 100%
rename from dump.c
rename to dump/dump.c
diff --git a/win_dump.c b/dump/win_dump.c
similarity index 100%
rename from win_dump.c
rename to dump/win_dump.c
diff --git a/win_dump.h b/dump/win_dump.h
similarity index 100%
rename from win_dump.h
rename to dump/win_dump.h
-- 
2.21.0

[Qemu-devel] [PATCH 16/17] MAINTAINERS: Add Windows dump to section "Dump"

[Markus Armbruster]

Commit 2da91b54fe9 "dump: add Windows dump format to
dump-guest-memory" neglected to update MAINTAINERS.  Do it now.

Cc: Marc-André Lureau 
Signed-off-by: Markus Armbruster 
---
 MAINTAINERS | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index a3fdda015f..327ad80ae4 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1850,9 +1850,10 @@ F: include/sysemu/device_tree.h
 Dump
 S: Supported
 M: Marc-André Lureau 
-F: dump/dump.c
+F: dump/
 F: hw/misc/vmcoreinfo.c
 F: include/hw/misc/vmcoreinfo.h
+F: include/qemu/win_dump_defs
 F: include/sysemu/dump-arch.h
 F: include/sysemu/dump.h
 F: qapi/dump.json
-- 
2.21.0

[Qemu-devel] [PATCH 01/17] MAINTAINERS: new maintainers for QOM

[Markus Armbruster]

From: Paolo Bonzini 

QOM is not a particularly active subsystem now: 51 commits in two years.
But, we need active maintainers to review and merge patches, and Git
shows the following top committers taking on QOM:

Markus Armbruster 
Eduardo Habkost 
Paolo Bonzini 
Marc-André Lureau 
Eric Blake 

I volunteer myself, and also volunteer Eduardo and Daniel as reviewers
since they understand the code well.

Cc: Andreas Färber 
Cc: Daniel P. Berrange 
Cc: Eduardo Habkost 
Suggested-by: Markus Armbruster 
Signed-off-by: Paolo Bonzini 
Message-Id: <20190607113737.13473-1-pbonz...@redhat.com>
Signed-off-by: Markus Armbruster 
---
 MAINTAINERS | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index d32c5c2313..63390eadb5 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2029,9 +2029,10 @@ F: docs/interop/qemu-ga-ref.texi
 T: git https://github.com/mdroth/qemu.git qga
 
 QOM
-M: Andreas Färber 
+M: Paolo Bonzini 
+R: Daniel P. Berrange 
+R: Eduardo Habkost 
 S: Supported
-T: git https://github.com/afaerber/qemu-cpu.git qom-next
 F: include/qom/
 X: include/qom/cpu.h
 F: qom/
-- 
2.21.0

[Qemu-devel] [PATCH 02/17] Makefile: Don't add monitor/ twice to common-obj-y

[Markus Armbruster]

Both commit f1b3ccfaa68 "monitor: Move {hmp, qmp}.c to monitor/{hmp,
qmp}-cmds.c" and commit 7e3c0deab1b "monitor: Split out monitor/qmp.c"
added monitor/ to common-obj-y ifeq ($(CONFIG_SOFTMMU),y).  Revert the
second addition.

Signed-off-by: Markus Armbruster 
---
 Makefile.objs | 1 -
 1 file changed, 1 deletion(-)

diff --git a/Makefile.objs b/Makefile.objs
index 658cfc9d9f..7494d6143b 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -85,7 +85,6 @@ common-obj-$(CONFIG_FDT) += device_tree.o
 # qapi
 
 common-obj-y += qapi/
-common-obj-y += monitor/
 endif
 
 ###
-- 
2.21.0

[Qemu-devel] [PATCH v1 16/17] .travis.yml: default the --disable-system build to --static

[Alex Bennée]

It's fairly common to build qemu-user binaries with --static linking
so the binary can be copied around without libraries. Enable --static
in the default qemu-user build to cover this.

There are other qemu-user builds that use dynamic linking so they
should catch any problems there.

Signed-off-by: Alex Bennée 
---
 .travis.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index aeb9b211cd..9750dc905c 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -80,7 +80,7 @@ script:
 matrix:
   include:
 - env:
-- CONFIG="--disable-system"
+- CONFIG="--disable-system --static"
 
 
 # we split the system builds as it takes a while to build them all
-- 
2.20.1

[Qemu-devel] [PATCH v1 12/17] tests/vm: freebsd autoinstall, using serial console

[Alex Bennée]

From: Gerd Hoffmann 

Instead of fetching the prebuilt image from patchew download the install
iso and prepare the image locally.  Install to disk, using the serial
console.  Create qemu user, configure ssh login.  Install packages
needed for qemu builds.

Note that freebsd package downloads are delivered as non-cachable
content, so I had to configure squid with "ignore-no-store
ignore-private ignore-reload" for pkgmir.geo.freebsd.org to make the
caching actually work.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Philippe Mathieu-Daudé 
Tested-by: Philippe Mathieu-Daudé 
Tested-by: Thomas Huth 
Message-Id: <20190617043858.8290-9-kra...@redhat.com>
---
 tests/vm/freebsd | 182 ---
 1 file changed, 172 insertions(+), 10 deletions(-)

diff --git a/tests/vm/freebsd b/tests/vm/freebsd
index 091be1a065..2a19461a90 100755
--- a/tests/vm/freebsd
+++ b/tests/vm/freebsd
@@ -2,41 +2,203 @@
 #
 # FreeBSD VM image
 #
-# Copyright 2017 Red Hat Inc.
+# Copyright 2017-2019 Red Hat Inc.
 #
 # Authors:
 #  Fam Zheng 
+#  Gerd Hoffmann 
 #
 # This code is licensed under the GPL version 2 or later.  See
 # the COPYING file in the top-level directory.
 #
 
 import os
+import re
 import sys
+import time
+import socket
 import subprocess
 import basevm
 
 class FreeBSDVM(basevm.BaseVM):
 name = "freebsd"
 arch = "x86_64"
+
+link = 
"https://download.freebsd.org/ftp/releases/ISO-IMAGES/12.0/FreeBSD-12.0-RELEASE-amd64-disc1.iso.xz;
+csum = "1d40015bea89d05b8bd13e2ed80c40b522a9ec1abd8e7c8b80954fb485fb99db"
+size = "20G"
+pkgs = [
+# build tools
+"git",
+"pkgconf",
+"bzip2",
+
+# gnu tools
+"bash",
+"gmake",
+"gsed",
+"flex", "bison",
+
+# libs: crypto
+"gnutls",
+
+# libs: images
+"jpeg-turbo",
+"png",
+
+# libs: ui
+"sdl2",
+"gtk3",
+"libxkbcommon",
+
+# libs: opengl
+"libepoxy",
+"mesa-libs",
+]
+
 BUILD_SCRIPT = """
 set -e;
-rm -rf /var/tmp/qemu-test.*
-cd $(mktemp -d /var/tmp/qemu-test.XX);
+rm -rf /home/qemu/qemu-test.*
+cd $(mktemp -d /home/qemu/qemu-test.XX);
+mkdir src build; cd src;
 tar -xf /dev/vtbd1;
-./configure {configure_opts};
+cd ../build
+../src/configure --python=python3.6 {configure_opts};
 gmake --output-sync -j{jobs} {target} {verbose};
 """
 
+def console_boot_serial(self):
+self.console_wait_send("Autoboot", "3")
+self.console_wait_send("OK", "set console=comconsole\n")
+self.console_wait_send("OK", "boot\n")
+
 def build_image(self, img):
-cimg = 
self._download_with_cache("http://download.patchew.org/freebsd-11.1-amd64.img.xz;,
-
sha256sum='adcb771549b37bc63826c501f05121a206ed3d9f55f49145908f7e1432d65891')
-img_tmp_xz = img + ".tmp.xz"
+self.print_step("Downloading install iso")
+cimg = self._download_with_cache(self.link, sha256sum=self.csum)
 img_tmp = img + ".tmp"
-sys.stderr.write("Extracting the image...\n")
-subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
-subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
+iso = img + ".install.iso"
+iso_xz = iso + ".xz"
+
+self.print_step("Preparing iso and disk image")
+subprocess.check_call(["cp", "-f", cimg, iso_xz])
+subprocess.check_call(["xz", "-dvf", iso_xz])
+subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
+   img_tmp, self.size])
+
+self.print_step("Booting installer")
+self.boot(img_tmp, extra_args = [
+"-bios", "pc-bios/bios-256k.bin",
+"-machine", "graphics=off",
+"-cdrom", iso
+])
+self.console_init()
+self.console_boot_serial()
+self.console_wait_send("Console type",  "xterm\n")
+
+# pre-install configuration
+self.console_wait_send("Welcome",   "\n")
+self.console_wait_send("Keymap Selection",  "\n")
+self.console_wait_send("Set Hostname",  "freebsd\n")
+self.console_wait_send("Distribution Select",   "\n")
+self.console_wait_send("Partitioning",  "\n")
+self.console_wait_send("Partition", "\n")
+self.console_wait_send("Scheme","\n")
+self.console_wait_send("Editor","f")
+self.console_wait_send("Confirmation",  "c")
+
+self.print_step("Installation started now, this will take a while")
+
+# post-install configuration
+self.console_wait("New Password:")
+self.console_send("%s\n" % self.ROOT_PASS)
+self.console_wait("Retype New Password:")
+self.console_send("%s\n" % self.ROOT_PASS)
+
+

[Qemu-devel] [PATCH v1 15/17] tests/vm: ubuntu.i386: apt proxy setup

[Alex Bennée]

From: Gerd Hoffmann 

Configure apt proxy so package downloads
can be cached and can pass firewalls.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Philippe Mathieu-Daudé 
Message-Id: <20190617043858.8290-12-kra...@redhat.com>
---
 tests/vm/ubuntu.i386 | 4 
 1 file changed, 4 insertions(+)

diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
index 3ea459ee20..38f740eabf 100755
--- a/tests/vm/ubuntu.i386
+++ b/tests/vm/ubuntu.i386
@@ -51,6 +51,10 @@ class UbuntuX86VM(basevm.BaseVM):
   "ssh-authorized-keys:\n",
   "- %s\n" % basevm.SSH_PUB_KEY,
   "locale: en_US.UTF-8\n"])
+proxy = os.environ.get("http_proxy")
+if not proxy is None:
+udata.writelines(["apt:\n",
+  "  proxy: %s" % proxy])
 udata.close()
 subprocess.check_call(["genisoimage", "-output", "cloud-init.iso",
"-volid", "cidata", "-joliet", "-rock",
-- 
2.20.1

[Qemu-devel] [PATCH v1 17/17] .travis.yml: force a brew update for MacOS builds

[Alex Bennée]

It looks like the Travis image package databases are out of date
causing the build to error with:

  Error: Your Homebrew is outdated. Please run `brew update`.
  Error: Kernel.exit

Signed-off-by: Alex Bennée 
---
 .travis.yml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.travis.yml b/.travis.yml
index 9750dc905c..a891ce2485 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -43,6 +43,7 @@ addons:
   - glib
   - pixman
   - gnu-sed
+update: true
 
 
 # The channel name "irc.oftc.net#qemu" is encrypted against qemu/qemu
-- 
2.20.1

[Qemu-devel] [PATCH v1 11/17] tests/vm: openbsd autoinstall, using serial console

[Alex Bennée]

From: Gerd Hoffmann 

Instead of fetching the prebuilt image from patchew download the install
iso and prepare the image locally.  Install to disk, using the serial
console.  Create qemu user, configure ssh login.  Install packages
needed for qemu builds.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Philippe Mathieu-Daudé 
Tested-by: Philippe Mathieu-Daudé 
Message-Id: <20190617043858.8290-8-kra...@redhat.com>
[AJB: added tags]
Signed-off-by: Alex Bennée 
---
 tests/vm/openbsd | 161 +++
 1 file changed, 148 insertions(+), 13 deletions(-)

diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index 28c7d25e29..b92c39f89a 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -2,10 +2,11 @@
 #
 # OpenBSD VM image
 #
-# Copyright 2017 Red Hat Inc.
+# Copyright 2017-2019 Red Hat Inc.
 #
 # Authors:
 #  Fam Zheng 
+#  Gerd Hoffmann 
 #
 # This code is licensed under the GPL version 2 or later.  See
 # the COPYING file in the top-level directory.
@@ -13,32 +14,166 @@
 
 import os
 import sys
+import socket
 import subprocess
 import basevm
 
 class OpenBSDVM(basevm.BaseVM):
 name = "openbsd"
 arch = "x86_64"
+
+link = "https://cdn.openbsd.org/pub/OpenBSD/6.5/amd64/install65.iso;
+csum = "38d1f8cadd502f1c27bf05c5abde6cc505dd28f3f34f8a941048ff9a54f9f608"
+size = "20G"
+pkgs = [
+# tools
+"git",
+"pkgconf",
+"bzip2", "xz",
+
+# gnu tools
+"bash",
+"gmake",
+"gsed",
+"bison",
+
+# libs: usb
+"libusb1",
+
+# libs: crypto
+"gnutls",
+
+# libs: images
+"jpeg",
+"png",
+
+   # libs: ui
+"sdl2",
+"gtk+3",
+"libxkbcommon",
+]
+
 BUILD_SCRIPT = """
 set -e;
-rm -rf /var/tmp/qemu-test.*
-cd $(mktemp -d /var/tmp/qemu-test.XX);
+rm -rf /home/qemu/qemu-test.*
+cd $(mktemp -d /home/qemu/qemu-test.XX);
+mkdir src build; cd src;
 tar -xf /dev/rsd1c;
-./configure --cc=x86_64-unknown-openbsd6.1-gcc-4.9.4 
--python=python2.7 {configure_opts};
-gmake --output-sync -j{jobs} {verbose};
-# XXX: "gmake check" seems to always hang or fail
-#gmake --output-sync -j{jobs} check {verbose};
+cd ../build
+../src/configure --cc=cc --python=python3 {configure_opts};
+gmake --output-sync -j{jobs} {target} {verbose};
 """
+poweroff = "halt -p"
 
 def build_image(self, img):
-cimg = 
self._download_with_cache("http://download.patchew.org/openbsd-6.1-amd64.img.xz;,
-
sha256sum='8c6cedc483e602cfee5e04f0406c64eb99138495e8ca580bc0293bcf0640c1bf')
-img_tmp_xz = img + ".tmp.xz"
+self.print_step("Downloading install iso")
+cimg = self._download_with_cache(self.link, sha256sum=self.csum)
 img_tmp = img + ".tmp"
-sys.stderr.write("Extracting the image...\n")
-subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
-subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
+iso = img + ".install.iso"
+
+self.print_step("Preparing iso and disk image")
+subprocess.check_call(["cp", "-f", cimg, iso])
+subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
+   img_tmp, self.size])
+
+self.print_step("Booting installer")
+self.boot(img_tmp, extra_args = [
+"-bios", "pc-bios/bios-256k.bin",
+"-machine", "graphics=off",
+"-cdrom", iso
+])
+self.console_init()
+self.console_wait_send("boot>", "set tty com0\n")
+self.console_wait_send("boot>", "\n")
+
+# pre-install configuration
+self.console_wait_send("(I)nstall",   "i\n")
+self.console_wait_send("Terminal type",   "xterm\n")
+self.console_wait_send("System hostname", "openbsd\n")
+self.console_wait_send("Which network interface", "vio0\n")
+self.console_wait_send("IPv4 address","dhcp\n")
+self.console_wait_send("IPv6 address","none\n")
+self.console_wait_send("Which network interface", "done\n")
+self.console_wait_send("DNS domain name", "localnet\n")
+self.console_wait("Password for root account")
+self.console_send("%s\n" % self.ROOT_PASS)
+self.console_wait("Password for root account")
+self.console_send("%s\n" % self.ROOT_PASS)
+self.console_wait_send("Start sshd(8)",   "yes\n")
+self.console_wait_send("X Window System", "\n")
+self.console_wait_send("xenodm",  "\n")
+self.console_wait_send("console to com0", "\n")
+self.console_wait_send("Which speed", "\n")
+
+self.console_wait("Setup a user")
+self.console_send("%s\n" % self.GUEST_USER)
+

[Qemu-devel] [PATCH v1 08/17] tests/vm: proper guest shutdown

[Alex Bennée]

From: Gerd Hoffmann 

When not running in snapshot mode ask the guest to poweroff and wait for
this to finish instead of simply quitting qemu, so the guest can flush
pending updates to disk.

Signed-off-by: Gerd Hoffmann 
Tested-by: Thomas Huth 
Reviewed-by: Philippe Mathieu-Daudé 
Tested-by: Philippe Mathieu-Daudé 
Message-Id: <20190617043858.8290-5-kra...@redhat.com>
[AJB: added tags]
Signed-off-by: Alex Bennée 
---
 tests/vm/basevm.py | 22 +++---
 1 file changed, 15 insertions(+), 7 deletions(-)

diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 395eefaec9..f27178f3c7 100755
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -51,6 +51,8 @@ class BaseVM(object):
 name = "#base"
 # The guest architecture, to be overridden by subclasses
 arch = "#arch"
+# command to halt the guest, can be overridden by subclasses
+poweroff = "poweroff"
 def __init__(self, debug=False, vcpus=None):
 self._guest = None
 self._tmpdir = os.path.realpath(tempfile.mkdtemp(prefix="vm-test-",
@@ -202,6 +204,10 @@ class BaseVM(object):
 def wait(self):
 self._guest.wait()
 
+def graceful_shutdown(self):
+self.ssh_root(self.poweroff)
+self._guest.wait()
+
 def qmp(self, *args, **kwargs):
 return self._guest.qmp(*args, **kwargs)
 
@@ -278,11 +284,13 @@ def main(vmcls):
 traceback.print_exc()
 return 2
 
-if args.interactive:
-if vm.ssh(*cmd) == 0:
-return 0
+exitcode = 0
+if vm.ssh(*cmd) != 0:
+exitcode = 3
+if exitcode != 0 and args.interactive:
 vm.ssh()
-return 3
-else:
-if vm.ssh(*cmd) != 0:
-return 3
+
+if not args.snapshot:
+vm.graceful_shutdown()
+
+return exitcode
-- 
2.20.1

[Qemu-devel] [PATCH v1 07/17] tests/vm: run test builds on snapshot

[Alex Bennée]

From: Gerd Hoffmann 

The build script doesn't shutdown the guest VMs properly,
which results in filesystem corruption and guest boot
failures sooner or later.

Use the --snapshot to run builds on a snapshot,
That way killing the VM doesn't corrupt the base image.

Signed-off-by: Gerd Hoffmann 
Tested-by: Thomas Huth 
Acked-by: Ed Maste 
Reviewed-by: Philippe Mathieu-Daudé 
Tested-by: Philippe Mathieu-Daudé 
Message-Id: <20190617043858.8290-4-kra...@redhat.com>
[AJB: added tags]
Signed-off-by: Alex Bennée 
---
 tests/vm/Makefile.include | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index c59411bee0..276b870216 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -57,6 +57,7 @@ vm-build-%: $(IMAGES_DIR)/%.img
$(if $(V),--verbose) \
--image "$<" \
$(if $(BUILD_TARGET),--build-target $(BUILD_TARGET)) \
+   --snapshot \
--build-qemu $(SRC_PATH) -- \
$(if $(TARGET_LIST),--target-list=$(TARGET_LIST)) \
$(if $(EXTRA_CONFIGURE_OPTS),$(EXTRA_CONFIGURE_OPTS)), \
-- 
2.20.1

[Qemu-devel] [PATCH v1 13/17] tests/vm: netbsd autoinstall, using serial console

[Alex Bennée]

From: Gerd Hoffmann 

Instead of fetching the prebuilt image from patchew download the install
iso and prepare the image locally.  Install to disk, using the serial
console.  Create qemu user, configure ssh login.  Install packages
needed for qemu builds.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Kamil Rytarowski 
Tested-by: Thomas Huth 
Reviewed-by: Philippe Mathieu-Daudé 
Tested-by: Philippe Mathieu-Daudé 
Message-Id: <20190617043858.8290-10-kra...@redhat.com>
[AJB: added tags]
Signed-off-by: Alex Bennée 
---
 tests/vm/netbsd | 189 +---
 1 file changed, 179 insertions(+), 10 deletions(-)

diff --git a/tests/vm/netbsd b/tests/vm/netbsd
index ee9eaeab50..be59a2c1da 100755
--- a/tests/vm/netbsd
+++ b/tests/vm/netbsd
@@ -2,10 +2,11 @@
 #
 # NetBSD VM image
 #
-# Copyright 2017 Red Hat Inc.
+# Copyright 2017-2019 Red Hat Inc.
 #
 # Authors:
 #  Fam Zheng 
+#  Gerd Hoffmann 
 #
 # This code is licensed under the GPL version 2 or later.  See
 # the COPYING file in the top-level directory.
@@ -13,30 +14,198 @@
 
 import os
 import sys
+import time
 import subprocess
 import basevm
 
 class NetBSDVM(basevm.BaseVM):
 name = "netbsd"
 arch = "x86_64"
+
+link = 
"https://cdn.netbsd.org/pub/NetBSD/NetBSD-8.0/images/NetBSD-8.0-amd64.iso;
+size = "20G"
+pkgs = [
+# tools
+"git-base",
+"pkgconf",
+"xz",
+"python37",
+
+# gnu tools
+"bash",
+"gmake",
+"gsed",
+"flex", "bison",
+
+# libs: crypto
+"gnutls",
+
+# libs: images
+"jpeg",
+"png",
+
+   # libs: ui
+"SDL2",
+"gtk3+",
+"libxkbcommon",
+]
+
 BUILD_SCRIPT = """
 set -e;
-rm -rf /var/tmp/qemu-test.*
-cd $(mktemp -d /var/tmp/qemu-test.XX);
+rm -rf /home/qemu/qemu-test.*
+cd $(mktemp -d /home/qemu/qemu-test.XX);
+mkdir src build; cd src;
 tar -xf /dev/rld1a;
-./configure --python=python2.7 {configure_opts};
+cd ../build
+../src/configure --python=python3.7 --disable-opengl {configure_opts};
 gmake --output-sync -j{jobs} {target} {verbose};
 """
+poweroff = "/sbin/poweroff"
 
 def build_image(self, img):
-cimg = 
self._download_with_cache("http://download.patchew.org/netbsd-7.1-amd64.img.xz;,
- 
sha256sum='b633d565b0eac3d02015cd0c81440bd8a7a8df8512615ac1ee05d318be015732')
-img_tmp_xz = img + ".tmp.xz"
+cimg = self._download_with_cache(self.link)
 img_tmp = img + ".tmp"
-sys.stderr.write("Extracting the image...\n")
-subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
-subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
+iso = img + ".install.iso"
+
+self.print_step("Preparing iso and disk image")
+subprocess.check_call(["cp", "-f", cimg, iso])
+subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
+   img_tmp, self.size])
+
+self.print_step("Booting installer")
+self.boot(img_tmp, extra_args = [
+"-bios", "pc-bios/bios-256k.bin",
+"-machine", "graphics=off",
+"-cdrom", iso
+])
+self.console_init()
+self.console_wait("Primary Bootstrap")
+
+# serial console boot menu output doesn't work for some
+# reason, so we have to fly blind ...
+for char in list("5consdev com0\n"):
+time.sleep(0.2)
+self.console_send(char)
+self.console_wait("")
+self.console_wait_send("> ", "boot\n")
+
+self.console_wait_send("Terminal type","xterm\n")
+self.console_wait_send("a: Installation messages", "a\n")
+self.console_wait_send("b: US-English","b\n")
+self.console_wait_send("a: Install NetBSD","a\n")
+self.console_wait("Shall we continue?")
+self.console_wait_send("b: Yes",   "b\n")
+
+self.console_wait_send("a: ld0",   "a\n")
+self.console_wait_send("a: This is the correct",   "a\n")
+self.console_wait_send("b: Use the entire disk",   "b\n")
+self.console_wait("NetBSD bootcode")
+self.console_wait_send("a: Yes",   "a\n")
+self.console_wait_send("b: Use existing part", "b\n")
+self.console_wait_send("x: Partition sizes ok","x\n")
+self.console_wait_send("for your NetBSD disk", "\n")
+self.console_wait("Shall we continue?")
+self.console_wait_send("b: Yes",   "b\n")
+
+self.console_wait_send("b: Use serial port com0",  "b\n")
+self.console_wait_send("f: Set serial baud rate",  "f\n")
+self.console_wait_send("a: 9600",  "a\n")
+self.console_wait_send("x: Exit",  

[Qemu-devel] [PATCH v1 09/17] tests/vm: add vm-boot-{ssh, serial}- targets

[Alex Bennée]

From: Gerd Hoffmann 

For testing/troubleshooting convenience.

make vm-boot-serial-
  Boot guest, with the serial console on stdio.

make vm-boot-ssh-
  Boot guest, login via ssh.

Signed-off-by: Gerd Hoffmann 
Tested-by: Thomas Huth 
Reviewed-by: Philippe Mathieu-Daudé 
Tested-by: Philippe Mathieu-Daudé 
Message-Id: <20190617043858.8290-6-kra...@redhat.com>
[AJB: added tags]
Signed-off-by: Alex Bennée 
---
 tests/vm/Makefile.include | 19 +++
 1 file changed, 19 insertions(+)

diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index 276b870216..a61c236b8e 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -20,6 +20,10 @@ vm-test:
@echo "  vm-build-all- Build QEMU in all VMs"
@echo "  vm-clean-all- Clean up VM images"
@echo
+   @echo "For trouble-shooting:"
+   @echo "  vm-boot-serial-  - Boot guest, serial console 
on stdio"
+   @echo "  vm-boot-ssh- - Boot guest and login via ssh"
+   @echo
@echo "Special variables:"
@echo "BUILD_TARGET=foo  - Override the build target"
@echo "TARGET_LIST=a,b,c - Override target list in 
builds"
@@ -63,3 +67,18 @@ vm-build-%: $(IMAGES_DIR)/%.img
$(if $(EXTRA_CONFIGURE_OPTS),$(EXTRA_CONFIGURE_OPTS)), \
"  VM-BUILD $*")
 
+vm-boot-serial-%: $(IMAGES_DIR)/%.img
+   qemu-system-x86_64 -enable-kvm -m 4G -smp 2 -nographic \
+   -drive if=none,id=vblk,cache=writeback,file="$<" \
+   -netdev user,id=vnet \
+   -device virtio-blk-pci,drive=vblk \
+   -device virtio-net-pci,netdev=vnet \
+   || true
+
+vm-boot-ssh-%: $(IMAGES_DIR)/%.img
+   $(call quiet-command, \
+   $(SRC_PATH)/tests/vm/$* \
+   --image "$<" \
+   --interactive \
+   false, \
+   "  VM-BOOT-SSH $*") || true
-- 
2.20.1

[Qemu-devel] [PATCH v1 14/17] tests/vm: fedora autoinstall, using serial console

[Alex Bennée]

From: Gerd Hoffmann 

Download the install iso and prepare the image locally.  Install to
disk, using the serial console.  Create qemu user, configure ssh login.
Install packages needed for qemu builds.

Yes, we have docker images for fedora.  But for trouble-shooting it
might be helpful to have a vm too.  When vm builds fail you can use
it to figure whenever the vm setup or the guest os is the problem.

Signed-off-by: Gerd Hoffmann 
Tested-by: Thomas Huth 
Message-Id: <20190617043858.8290-11-kra...@redhat.com>
---
 tests/vm/Makefile.include |   3 +-
 tests/vm/basevm.py|   9 +-
 tests/vm/fedora   | 189 ++
 3 files changed, 199 insertions(+), 2 deletions(-)
 create mode 100755 tests/vm/fedora

diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index a61c236b8e..809b80e2e5 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -2,7 +2,7 @@
 
 .PHONY: vm-build-all vm-clean-all
 
-IMAGES := ubuntu.i386 freebsd netbsd openbsd centos
+IMAGES := ubuntu.i386 freebsd netbsd openbsd centos fedora
 IMAGES_DIR := $(HOME)/.cache/qemu-vm/images
 IMAGE_FILES := $(patsubst %, $(IMAGES_DIR)/%.img, $(IMAGES))
 
@@ -16,6 +16,7 @@ vm-test:
@echo "  vm-build-netbsd - Build QEMU in NetBSD VM"
@echo "  vm-build-openbsd- Build QEMU in OpenBSD VM"
@echo "  vm-build-centos - Build QEMU in CentOS VM, 
with Docker"
+   @echo "  vm-build-fedora - Build QEMU in Fedora VM"
@echo ""
@echo "  vm-build-all- Build QEMU in all VMs"
@echo "  vm-clean-all- Clean up VM images"
diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 592a344773..3d717da498 100755
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -206,7 +206,7 @@ class BaseVM(object):
 # log console line
 sys.stderr.write("con recv: %s\n" % line)
 
-def console_wait(self, expect):
+def console_wait(self, expect, expectalt = None):
 vm = self._guest
 output = ""
 while True:
@@ -215,6 +215,8 @@ class BaseVM(object):
 except socket.timeout:
 sys.stderr.write("console: *** read timeout ***\n")
 sys.stderr.write("console: waiting for: '%s'\n" % expect)
+if not expectalt is None:
+sys.stderr.write("console: waiting for: '%s' (alt)\n" % 
expectalt)
 sys.stderr.write("console: line buffer:\n")
 sys.stderr.write("\n")
 self.console_log(output.rstrip())
@@ -223,6 +225,8 @@ class BaseVM(object):
 output += chars.decode("latin1")
 if expect in output:
 break
+if not expectalt is None and expectalt in output:
+break
 if "\r" in output or "\n" in output:
 lines = re.split("[\r\n]", output)
 output = lines.pop()
@@ -230,6 +234,9 @@ class BaseVM(object):
 self.console_log("\n".join(lines))
 if self.debug:
 self.console_log(output)
+if not expectalt is None and expectalt in output:
+return False
+return True
 
 def console_send(self, command):
 vm = self._guest
diff --git a/tests/vm/fedora b/tests/vm/fedora
new file mode 100755
index 00..e8fa5bf0d2
--- /dev/null
+++ b/tests/vm/fedora
@@ -0,0 +1,189 @@
+#!/usr/bin/env python
+#
+# Fedora VM image
+#
+# Copyright 2019 Red Hat Inc.
+#
+# Authors:
+#  Gerd Hoffmann 
+#
+# This code is licensed under the GPL version 2 or later.  See
+# the COPYING file in the top-level directory.
+#
+
+import os
+import re
+import sys
+import time
+import socket
+import subprocess
+import basevm
+
+class FedoraVM(basevm.BaseVM):
+name = "fedora"
+arch = "x86_64"
+
+base = "http://dl.fedoraproject.org/pub/fedora/linux/releases/30/;
+link = base + "Server/x86_64/iso/Fedora-Server-netinst-x86_64-30-1.2.iso"
+repo = base + "Server/x86_64/os/"
+full = base + "Everything/x86_64/os/"
+csum = "5e4eac4566d8c572bfb3bcf54b7d6c82006ec3c6c882a2c9235c6d3494d7b100"
+size = "20G"
+pkgs = [
+# tools
+'git-core',
+'flex', 'bison',
+'gcc', 'binutils', 'make',
+
+# perl
+'perl-Test-Harness',
+
+# libs: usb
+'"pkgconfig(libusb-1.0)"',
+'"pkgconfig(libusbredirparser-0.5)"',
+
+# libs: crypto
+'"pkgconfig(gnutls)"',
+
+# libs: ui
+'"pkgconfig(sdl2)"',
+'"pkgconfig(gtk+-3.0)"',
+'"pkgconfig(ncursesw)"',
+
+# libs: audio
+'"pkgconfig(libpulse)"',
+'"pkgconfig(alsa)"',
+]
+
+BUILD_SCRIPT = """
+set -e;
+rm -rf /home/qemu/qemu-test.*
+cd $(mktemp -d /home/qemu/qemu-test.XX);
+mkdir src build; cd src;
+tar -xf /dev/vdb;
+cd ../build
+ 

[Qemu-devel] [PATCH v1 03/17] tests/vm: pin ubuntu.i386 image

[Alex Bennée]

From: Cleber Rosa 

It's a good practice to always have the same components used in tests.
According to:

   https://cloud-images.ubuntu.com/releases/16.04/

New images are released from time to time, and the "release/"
directory points to the latest release.  Let's pin to the latest
available version, and while at it, set a hash for verification.

Signed-off-by: Cleber Rosa 
Message-Id: <20190613130718.3763-4-cr...@redhat.com>
Signed-off-by: Alex Bennée 
---
 tests/vm/ubuntu.i386 | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
index 12867b193f..7017e6d388 100755
--- a/tests/vm/ubuntu.i386
+++ b/tests/vm/ubuntu.i386
@@ -61,7 +61,9 @@ class UbuntuX86VM(basevm.BaseVM):
 return os.path.join(cidir, "cloud-init.iso")
 
 def build_image(self, img):
-cimg = 
self._download_with_cache("https://cloud-images.ubuntu.com/releases/16.04/release/ubuntu-16.04-server-cloudimg-i386-disk1.img;)
+cimg = self._download_with_cache(
+
"https://cloud-images.ubuntu.com/releases/16.04/release-20190605/ubuntu-16.04-server-cloudimg-i386-disk1.img;,
+
sha256sum="e30091144c73483822b7c27193e9d47346dd1064229da577c3fedcf943f7cfcc")
 img_tmp = img + ".tmp"
 subprocess.check_call(["cp", "-f", cimg, img_tmp])
 subprocess.check_call(["qemu-img", "resize", img_tmp, "50G"])
-- 
2.20.1

[Qemu-devel] [PATCH v1 02/17] tests/vm: avoid image presence check and removal

[Alex Bennée]

From: Cleber Rosa 

Python's os.rename() will silently replace an existing file,
so there's no need for the extra check and removal.

Reference: https://docs.python.org/3/library/os.html#os.rename
Signed-off-by: Cleber Rosa 
Message-Id: <20190613130718.3763-3-cr...@redhat.com>
Signed-off-by: Alex Bennée 
---
 tests/vm/centos  | 2 --
 tests/vm/freebsd | 2 --
 tests/vm/netbsd  | 2 --
 tests/vm/openbsd | 2 --
 tests/vm/ubuntu.i386 | 2 --
 5 files changed, 10 deletions(-)

diff --git a/tests/vm/centos b/tests/vm/centos
index b00b46a8dc..53976f1c4c 100755
--- a/tests/vm/centos
+++ b/tests/vm/centos
@@ -77,8 +77,6 @@ class CentosVM(basevm.BaseVM):
 self.ssh_root_check("systemctl enable docker")
 self.ssh_root("poweroff")
 self.wait()
-if os.path.exists(img):
-os.remove(img)
 os.rename(img_tmp, img)
 return 0
 
diff --git a/tests/vm/freebsd b/tests/vm/freebsd
index 5575c23a6f..091be1a065 100755
--- a/tests/vm/freebsd
+++ b/tests/vm/freebsd
@@ -36,8 +36,6 @@ class FreeBSDVM(basevm.BaseVM):
 sys.stderr.write("Extracting the image...\n")
 subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
 subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
-if os.path.exists(img):
-os.remove(img)
 os.rename(img_tmp, img)
 
 if __name__ == "__main__":
diff --git a/tests/vm/netbsd b/tests/vm/netbsd
index d0508f4465..ee9eaeab50 100755
--- a/tests/vm/netbsd
+++ b/tests/vm/netbsd
@@ -36,8 +36,6 @@ class NetBSDVM(basevm.BaseVM):
 sys.stderr.write("Extracting the image...\n")
 subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
 subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
-if os.path.exists(img):
-os.remove(img)
 os.rename(img_tmp, img)
 
 if __name__ == "__main__":
diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index 87ec982489..28c7d25e29 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -38,8 +38,6 @@ class OpenBSDVM(basevm.BaseVM):
 sys.stderr.write("Extracting the image...\n")
 subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
 subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
-if os.path.exists(img):
-os.remove(img)
 os.rename(img_tmp, img)
 
 if __name__ == "__main__":
diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
index a22d137e76..12867b193f 100755
--- a/tests/vm/ubuntu.i386
+++ b/tests/vm/ubuntu.i386
@@ -80,8 +80,6 @@ class UbuntuX86VM(basevm.BaseVM):
 self.ssh_root_check("apt-get install -y libfdt-dev flex bison")
 self.ssh_root("poweroff")
 self.wait()
-if os.path.exists(img):
-os.remove(img)
 os.rename(img_tmp, img)
 return 0
 
-- 
2.20.1

[Qemu-devel] [PATCH v1 05/17] tests/vm: send proxy environment variables over ssh

[Alex Bennée]

From: Gerd Hoffmann 

Packages are fetched via proxy that way, if configured on the host.
That might be required to pass firewalls, and it allows to route
package downloads through a caching proxy server.

Needs AcceptEnv setup in sshd_config on the guest side to work.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Philippe Mathieu-Daudé 
Tested-by: Thomas Huth 
Message-Id: <20190617043858.8290-2-kra...@redhat.com>
---
 tests/vm/basevm.py | 9 +
 1 file changed, 9 insertions(+)

diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 4847549592..5e30bac661 100755
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -38,6 +38,13 @@ class BaseVM(object):
 GUEST_PASS = "qemupass"
 ROOT_PASS = "qemupass"
 
+envvars = [
+"https_proxy",
+"http_proxy",
+"ftp_proxy",
+"no_proxy",
+]
+
 # The script to run in the guest that builds QEMU
 BUILD_SCRIPT = ""
 # The guest name, to be overridden by subclasses
@@ -106,6 +113,8 @@ class BaseVM(object):
"-o", "UserKnownHostsFile=" + os.devnull,
"-o", "ConnectTimeout=1",
"-p", self.ssh_port, "-i", self._ssh_key_file]
+for var in self.envvars:
+ssh_cmd += ['-o', "SendEnv=%s" % var ]
 if interactive:
 ssh_cmd += ['-t']
 assert not isinstance(cmd, str)
-- 
2.20.1

[Qemu-devel] [PATCH v1 04/17] tests/vm: add source repos on ubuntu.i386

[Alex Bennée]

From: Cleber Rosa 

Possibly because of different behavior on the newly update
cloud-image, trying to run 'apt-get build-dep' results in:

   E: You must put some 'source' URIs in your sources.list

This enables all source repos (even though some are not
needed) for simplicity sake.

Signed-off-by: Cleber Rosa 
Message-Id: <20190613130718.3763-5-cr...@redhat.com>
Signed-off-by: Alex Bennée 
---
 tests/vm/ubuntu.i386 | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
index 7017e6d388..3ea459ee20 100755
--- a/tests/vm/ubuntu.i386
+++ b/tests/vm/ubuntu.i386
@@ -77,6 +77,7 @@ class UbuntuX86VM(basevm.BaseVM):
 time.sleep(5)
 self.wait_ssh()
 # The previous update sometimes doesn't survive a reboot, so do it 
again
+self.ssh_root_check("sed -ie s/^#\ deb-src/deb-src/g 
/etc/apt/sources.list")
 self.ssh_root_check("apt-get update")
 self.ssh_root_check("apt-get build-dep -y qemu")
 self.ssh_root_check("apt-get install -y libfdt-dev flex bison")
-- 
2.20.1

[Qemu-devel] [PATCH v1 06/17] tests/vm: use ssh with pty unconditionally

[Alex Bennée]

From: Gerd Hoffmann 

Allways ask ssh to run with a pseudo terminal.
Not having a terminal causes problems now and then.

Signed-off-by: Gerd Hoffmann 
Tested-by: Thomas Huth 
Reviewed-by: Philippe Mathieu-Daudé 
Tested-by: Philippe Mathieu-Daudé 
Message-Id: <20190617043858.8290-3-kra...@redhat.com>
[AJB: added PMD's r-b/t-b tags]
Signed-off-by: Alex Bennée 
---
 tests/vm/basevm.py | 13 -
 1 file changed, 4 insertions(+), 9 deletions(-)

diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 5e30bac661..395eefaec9 100755
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -107,16 +107,14 @@ class BaseVM(object):
 os.rename(fname + ".download", fname)
 return fname
 
-def _ssh_do(self, user, cmd, check, interactive=False):
-ssh_cmd = ["ssh", "-q",
+def _ssh_do(self, user, cmd, check):
+ssh_cmd = ["ssh", "-q", "-t",
"-o", "StrictHostKeyChecking=no",
"-o", "UserKnownHostsFile=" + os.devnull,
"-o", "ConnectTimeout=1",
"-p", self.ssh_port, "-i", self._ssh_key_file]
 for var in self.envvars:
 ssh_cmd += ['-o', "SendEnv=%s" % var ]
-if interactive:
-ssh_cmd += ['-t']
 assert not isinstance(cmd, str)
 ssh_cmd += ["%s@127.0.0.1" % user] + list(cmd)
 logging.debug("ssh_cmd: %s", " ".join(ssh_cmd))
@@ -128,9 +126,6 @@ class BaseVM(object):
 def ssh(self, *cmd):
 return self._ssh_do(self.GUEST_USER, cmd, False)
 
-def ssh_interactive(self, *cmd):
-return self._ssh_do(self.GUEST_USER, cmd, False, True)
-
 def ssh_root(self, *cmd):
 return self._ssh_do("root", cmd, False)
 
@@ -284,9 +279,9 @@ def main(vmcls):
 return 2
 
 if args.interactive:
-if vm.ssh_interactive(*cmd) == 0:
+if vm.ssh(*cmd) == 0:
 return 0
-vm.ssh_interactive()
+vm.ssh()
 return 3
 else:
 if vm.ssh(*cmd) != 0:
-- 
2.20.1

[Qemu-devel] [PATCH v1 00/17] testing/next (Travis fixes, more tests/vm)

[Alex Bennée]

Hi,

This is the current state of testing/next which has fixes for the
MacOS Travis failures. We also have the return of the bsd-over-serial
autosetup patches which I've tested on the previously failing
platform. The one thing I've not included is adding check-block to the
default set of check tests. The blocker for that is it still bails on
some iotests when the source tree filesystem (e.g. ZFS) doesn't
support features it needs for some tests. It seems that they should
skip a bit more gracefully:

  Not run: 045 059 064 070 075 077 078 081 084 088 092 094 101 106 113 116 119 
123 128 131 146 148 160 162 171 175 211 212 213 221 225 231 233 237 239 241 253
  Some cases not run in: 139
  Failures: 102 243 250
  Failed 3 of 138 tests

I'll try and send the PR for this out by the end of the week.

The following patches need review
 patch 0014/tests vm fedora autoinstall using serial console.patch
 patch 0016/.travis.yml default the disable system build to s.patch
 patch 0017/.travis.yml force a brew update for MacOS builds.patch

Alex Bennée (2):
  .travis.yml: default the --disable-system build to --static
  .travis.yml: force a brew update for MacOS builds

Cleber Rosa (4):
  tests/vm: avoid extra compressed image copy
  tests/vm: avoid image presence check and removal
  tests/vm: pin ubuntu.i386 image
  tests/vm: add source repos on ubuntu.i386

Gerd Hoffmann (11):
  tests/vm: send proxy environment variables over ssh
  tests/vm: use ssh with pty unconditionally
  tests/vm: run test builds on snapshot
  tests/vm: proper guest shutdown
  tests/vm: add vm-boot-{ssh,serial}- targets
  tests/vm: serial console support helpers
  tests/vm: openbsd autoinstall, using serial console
  tests/vm: freebsd autoinstall, using serial console
  tests/vm: netbsd autoinstall, using serial console
  tests/vm: fedora autoinstall, using serial console
  tests/vm: ubuntu.i386: apt proxy setup

 .travis.yml   |   3 +-
 tests/vm/Makefile.include |  23 -
 tests/vm/basevm.py| 135 +++
 tests/vm/centos   |   6 +-
 tests/vm/fedora   | 189 ++
 tests/vm/freebsd  | 180 ++--
 tests/vm/netbsd   | 187 +++--
 tests/vm/openbsd  | 159 +---
 tests/vm/ubuntu.i386  |  11 ++-
 9 files changed, 833 insertions(+), 60 deletions(-)
 create mode 100755 tests/vm/fedora

-- 
2.20.1

[Qemu-devel] [PATCH v1 01/17] tests/vm: avoid extra compressed image copy

[Alex Bennée]

From: Cleber Rosa 

The image copy is only really needed because xz doesn't know to
properly decompress a file not named properly.  Instead of
decompressing to stdout, and having to rely on a shell, let's just
create a link instead of copying the file.

Signed-off-by: Cleber Rosa 
Message-Id: <20190613130718.3763-2-cr...@redhat.com>
Signed-off-by: Alex Bennée 
---
 tests/vm/centos  | 4 ++--
 tests/vm/freebsd | 4 ++--
 tests/vm/netbsd  | 4 ++--
 tests/vm/openbsd | 4 ++--
 4 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/tests/vm/centos b/tests/vm/centos
index 7417b50af4..b00b46a8dc 100755
--- a/tests/vm/centos
+++ b/tests/vm/centos
@@ -66,8 +66,8 @@ class CentosVM(basevm.BaseVM):
 cimg = 
self._download_with_cache("https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-1802.qcow2.xz;)
 img_tmp = img + ".tmp"
 sys.stderr.write("Extracting the image...\n")
-subprocess.check_call(["cp", "-f", cimg, img_tmp + ".xz"])
-subprocess.check_call(["xz", "-dvf", img_tmp + ".xz"])
+subprocess.check_call(["ln", "-f", cimg, img_tmp + ".xz"])
+subprocess.check_call(["xz", "--keep", "-dvf", img_tmp + ".xz"])
 subprocess.check_call(["qemu-img", "resize", img_tmp, "50G"])
 self.boot(img_tmp, extra_args = ["-cdrom", self._gen_cloud_init_iso()])
 self.wait_ssh()
diff --git a/tests/vm/freebsd b/tests/vm/freebsd
index b0066017a6..5575c23a6f 100755
--- a/tests/vm/freebsd
+++ b/tests/vm/freebsd
@@ -34,8 +34,8 @@ class FreeBSDVM(basevm.BaseVM):
 img_tmp_xz = img + ".tmp.xz"
 img_tmp = img + ".tmp"
 sys.stderr.write("Extracting the image...\n")
-subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
-subprocess.check_call(["xz", "-dvf", img_tmp_xz])
+subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
+subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
 if os.path.exists(img):
 os.remove(img)
 os.rename(img_tmp, img)
diff --git a/tests/vm/netbsd b/tests/vm/netbsd
index 4c6624ea5e..d0508f4465 100755
--- a/tests/vm/netbsd
+++ b/tests/vm/netbsd
@@ -34,8 +34,8 @@ class NetBSDVM(basevm.BaseVM):
 img_tmp_xz = img + ".tmp.xz"
 img_tmp = img + ".tmp"
 sys.stderr.write("Extracting the image...\n")
-subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
-subprocess.check_call(["xz", "-dvf", img_tmp_xz])
+subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
+subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
 if os.path.exists(img):
 os.remove(img)
 os.rename(img_tmp, img)
diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index 2105c01a26..87ec982489 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -36,8 +36,8 @@ class OpenBSDVM(basevm.BaseVM):
 img_tmp_xz = img + ".tmp.xz"
 img_tmp = img + ".tmp"
 sys.stderr.write("Extracting the image...\n")
-subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
-subprocess.check_call(["xz", "-dvf", img_tmp_xz])
+subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
+subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
 if os.path.exists(img):
 os.remove(img)
 os.rename(img_tmp, img)
-- 
2.20.1

[Qemu-devel] [PATCH v1 10/17] tests/vm: serial console support helpers

[Alex Bennée]

From: Gerd Hoffmann 

Add a bunch of helpers to talk to the guest using the
serial console.

Also drop the hard-coded -serial parameter for the vm
so QEMUMachine.set_console() actually works.

Signed-off-by: Gerd Hoffmann 
Tested-by: Thomas Huth 
Reviewed-by: Philippe Mathieu-Daudé 
Tested-by: Philippe Mathieu-Daudé 
Message-Id: <20190617043858.8290-7-kra...@redhat.com>
[AJB: added tags]
Signed-off-by: Alex Bennée 
---
 tests/vm/basevm.py | 86 --
 1 file changed, 83 insertions(+), 3 deletions(-)

diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index f27178f3c7..592a344773 100755
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -2,10 +2,11 @@
 #
 # VM testing base class
 #
-# Copyright 2017 Red Hat Inc.
+# Copyright 2017-2019 Red Hat Inc.
 #
 # Authors:
 #  Fam Zheng 
+#  Gerd Hoffmann 
 #
 # This code is licensed under the GPL version 2 or later.  See
 # the COPYING file in the top-level directory.
@@ -13,7 +14,9 @@
 
 from __future__ import print_function
 import os
+import re
 import sys
+import socket
 import logging
 import time
 import datetime
@@ -79,8 +82,7 @@ class BaseVM(object):
 "-cpu", "max",
 "-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22",
 "-device", "virtio-net-pci,netdev=vnet",
-"-vnc", "127.0.0.1:0,to=20",
-"-serial", "file:%s" % os.path.join(self._tmpdir, "serial.out")]
+"-vnc", "127.0.0.1:0,to=20"]
 if vcpus and vcpus > 1:
 self._args += ["-smp", "%d" % vcpus]
 if kvm_available(self.arch):
@@ -162,6 +164,8 @@ class BaseVM(object):
 logging.debug("QEMU args: %s", " ".join(args))
 qemu_bin = os.environ.get("QEMU", "qemu-system-" + self.arch)
 guest = QEMUMachine(binary=qemu_bin, args=args)
+guest.set_machine('pc')
+guest.set_console()
 try:
 guest.launch()
 except:
@@ -184,6 +188,82 @@ class BaseVM(object):
 raise Exception("Cannot find ssh port from 'info usernet':\n%s" % \
 usernet_info)
 
+def console_init(self, timeout = 120):
+vm = self._guest
+vm.console_socket.settimeout(timeout)
+
+def console_log(self, text):
+for line in re.split("[\r\n]", text):
+# filter out terminal escape sequences
+line = re.sub("\x1b\[[0-9;?]*[a-zA-Z]", "", line)
+line = re.sub("\x1b\([0-9;?]*[a-zA-Z]", "", line)
+# replace unprintable chars
+line = re.sub("\x1b", "", line)
+line = re.sub("[\x00-\x1f]", ".", line)
+line = re.sub("[\x80-\xff]", ".", line)
+if line == "":
+continue
+# log console line
+sys.stderr.write("con recv: %s\n" % line)
+
+def console_wait(self, expect):
+vm = self._guest
+output = ""
+while True:
+try:
+chars = vm.console_socket.recv(1)
+except socket.timeout:
+sys.stderr.write("console: *** read timeout ***\n")
+sys.stderr.write("console: waiting for: '%s'\n" % expect)
+sys.stderr.write("console: line buffer:\n")
+sys.stderr.write("\n")
+self.console_log(output.rstrip())
+sys.stderr.write("\n")
+raise
+output += chars.decode("latin1")
+if expect in output:
+break
+if "\r" in output or "\n" in output:
+lines = re.split("[\r\n]", output)
+output = lines.pop()
+if self.debug:
+self.console_log("\n".join(lines))
+if self.debug:
+self.console_log(output)
+
+def console_send(self, command):
+vm = self._guest
+if self.debug:
+logline = re.sub("\n", "", command)
+logline = re.sub("[\x00-\x1f]", ".", logline)
+sys.stderr.write("con send: %s\n" % logline)
+for char in list(command):
+vm.console_socket.send(char.encode("utf-8"))
+time.sleep(0.01)
+
+def console_wait_send(self, wait, command):
+self.console_wait(wait)
+self.console_send(command)
+
+def console_ssh_init(self, prompt, user, pw):
+sshkey_cmd = "echo '%s' > .ssh/authorized_keys\n" % 
SSH_PUB_KEY.rstrip()
+self.console_wait_send("login:","%s\n" % user)
+self.console_wait_send("Password:", "%s\n" % pw)
+self.console_wait_send(prompt,  "mkdir .ssh\n")
+self.console_wait_send(prompt,  sshkey_cmd)
+self.console_wait_send(prompt,  "chmod 755 .ssh\n")
+self.console_wait_send(prompt,  "chmod 644 .ssh/authorized_keys\n")
+
+def console_sshd_config(self, prompt):
+self.console_wait(prompt)
+self.console_send("echo 'PermitRootLogin yes' >> 
/etc/ssh/sshd_config\n")
+for var in self.envvars:
+ 

Re: [Qemu-devel] [PATCH 1/2] vl: Drain before (block) job cancel when quitting

[Max Reitz]

On 14.06.19 11:22, Vladimir Sementsov-Ogievskiy wrote:
> 13.06.2019 19:03, Max Reitz wrote:
>> [re-adding the original CCs, why not]
>>
>> On 13.06.19 16:30, Vladimir Sementsov-Ogievskiy wrote:
>>> 13.06.2019 17:21, Max Reitz wrote:
 On 13.06.19 16:19, Vladimir Sementsov-Ogievskiy wrote:
> 13.06.2019 1:08, Max Reitz wrote:
>> If the main loop cancels all block jobs while the block layer is not
>> drained, this cancelling may not happen instantaneously.  We can start a
>> drained section before vm_shutdown(), which entails another
>> bdrv_drain_all(); this nested bdrv_drain_all() will thus be a no-op,
>> basically.
>>
>> We do not have to end the drained section, because we actually do not
>> want any requests to happen from this point on.
>>
>> Signed-off-by: Max Reitz 
>> ---
>> I don't know whether it actually makes sense to never end this drained
>> section.  It makes sense to me.  Please correct me if I'm wrong.
>> ---
>> vl.c | 11 +++
>> 1 file changed, 11 insertions(+)
>>
>> diff --git a/vl.c b/vl.c
>> index cd1fbc4cdc..3f8b3f74f5 100644
>> --- a/vl.c
>> +++ b/vl.c
>> @@ -4538,6 +4538,17 @@ int main(int argc, char **argv, char **envp)
>>  */
>> migration_shutdown();
>> 
>> +/*
>> + * We must cancel all block jobs while the block layer is drained,
>> + * or cancelling will be affected by throttling and thus may block
>> + * for an extended period of time.
>> + * vm_shutdown() will bdrv_drain_all(), so we may as well include
>> + * it in the drained section.
>> + * We do not need to end this section, because we do not want any
>> + * requests happening from here on anyway.
>> + */
>> +bdrv_drain_all_begin();
>> +
>> /* No more vcpu or device emulation activity beyond this point */
>> vm_shutdown();
>> 
>>
>
> So, actually, the problem is that we may wait for job requests twice:
> on drain and then on cancel.

 We don’t wait on drain.  When the throttle node is drained, it will
 ignore throttling (as noted in the cover letter).

 We do wait when cancelling a job while the throttle node isn’t drained,
 though.  That’s the problem.
>>>
>>> Ah, understand now.
>>>
>>> Is it safe to drain_begin before stopping cpus? We may finish up then with 
>>> some queued
>>> somewhere IO requests..
>>
>> Hm...  Aren’t guest devices prohibited from issuing requests to the
>> block layer while their respective block device is drained?
> 
> It's at least a buggy place, I remember Denis Plotnikov sent patch to fix it 
> and had a huge
> discussion with Kevin.
> And here it is:
> https://lists.gnu.org/archive/html/qemu-devel/2018-12/msg00732.html

Ah, I even have that in my inbox...  The latest reply I see came in April:

https://lists.nongnu.org/archive/html/qemu-block/2019-04/msg00243.html

Where Kevin asked for an RFC patch in the current state.

I’m not sure whether I should work around a potential bug here, if we
can agree that it is a bug, and if it isn’t clear whether this place
would actually be affected.

Max



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v2] iotest 134: test cluster-misaligned encrypted write

[Max Reitz]

On 16.05.19 16:30, Anton Nefedov wrote:
> COW (even empty/zero) areas require encryption too
> 
> Signed-off-by: Anton Nefedov 
> Reviewed-by: Eric Blake 
> Reviewed-by: Max Reitz 
> Reviewed-by: Alberto Garcia 
> ---
> 
> ..apparently v1 ended up in a weird base64 that would not easily git-am.
> Resending.
> 
> used to be a part of 'qcow2: cluster space preallocation' series
> http://lists.nongnu.org/archive/html/qemu-devel/2019-01/msg02769.html
> 
> ---
>  tests/qemu-iotests/134 |  9 +
>  tests/qemu-iotests/134.out | 10 ++
>  2 files changed, 19 insertions(+)
> 

Thanks, applied to my block branch:

https://git.xanclic.moe/XanClic/qemu/commits/branch/block

Max



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v7 0/3] block/stream: get rid of the base

[Max Reitz]

On 29.05.19 19:56, Andrey Shinkevich wrote:
> This series introduces a bottom intermediate node that eliminates the
> dependency on the base that may change while stream job is running.
> It happens when stream/commit parallel jobs are running on the same
> backing chain. The base node of the stream job may be a top node of
> the parallel commit job and can change before the stream job is
> completed. We avoid that dependency by introducing the bottom node.
> 
> v7: [resend by Andrey]
>   01: assert(intermediate) was inserted before the call to
>   bdrv_is_allocated() in the intermediate node loop of the
>   bdrv_is_allocated_above() as suggested by Max.
>   02: The change of the intermediate node loop in the stream_start() was
>   rolled back to its original design and the reassignment of the base
>   node pointer was added as Vladimir and Max suggested. The relevant
>   comment was amended.
> 
> v6: [resend by Vladimir]
>   01: improve comment in block/io.c, suggested by Alberto
> 
> v5: [resend by Vladimir]
>   01: use comment wording in block/io.c suggested by Alberto
> 
> v4:
> trace_stream_start reverted to the base.
> bdrv_is_allocated_above_inclusive() deleted and the new parameter
> 'bool include_base' was added to the bdrv_is_allocated_above().
> 
> Andrey Shinkevich (3):
>   block: include base when checking image chain for block allocation
>   block/stream: refactor stream_run: drop goto
>   block/stream: introduce a bottom node
> 
>  block/commit.c |  2 +-
>  block/io.c | 21 +--
>  block/mirror.c |  2 +-
>  block/replication.c|  2 +-
>  block/stream.c | 56 
> --
>  include/block/block.h  |  3 ++-
>  tests/qemu-iotests/245 |  4 ++--
>  7 files changed, 49 insertions(+), 41 deletions(-)

Reviewed-by: Max Reitz 

Just needs some simple changes to patch 1 to rebase it on 863cc78f1b3
and c8bb23cbdbe.

Max



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v7 1/3] block: include base when checking image chain for block allocation

[Max Reitz]

On 29.05.19 19:56, Andrey Shinkevich wrote:
> This patch is used in the 'block/stream: introduce a bottom node'
> that is following. Instead of the base node, the caller may pass
> the node that has the base as its backing image to the function
> bdrv_is_allocated_above() with a new parameter include_base = true
> and get rid of the dependency on the base that may change during
> commit/stream parallel jobs. Now, if the specified base is not
> found in the backing image chain, the QEMU will abort.
> 
> Suggested-by: Vladimir Sementsov-Ogievskiy 
> Signed-off-by: Andrey Shinkevich 
> Reviewed-by: Vladimir Sementsov-Ogievskiy 
> Reviewed-by: Alberto Garcia 
> ---
>  block/commit.c|  2 +-
>  block/io.c| 21 +++--
>  block/mirror.c|  2 +-
>  block/replication.c   |  2 +-
>  block/stream.c|  2 +-
>  include/block/block.h |  3 ++-
>  6 files changed, 21 insertions(+), 11 deletions(-)

This needs the following hunk squashed in so it still compiles:

(I can do that, if you agree.)

diff --git a/block/qcow2.c b/block/qcow2.c
index 9396d490d5..2a59eb27fe 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2148,7 +2148,8 @@ static bool is_unallocated(BlockDriverState *bs,
int64_t offset, int64_t bytes)
 {
 int64_t nr;
 return !bytes ||
-(!bdrv_is_allocated_above(bs, NULL, offset, bytes, ) && nr
== bytes);
+(!bdrv_is_allocated_above(bs, NULL, false, offset, bytes, ) &&
+ nr == bytes);
 }

 static bool is_zero_cow(BlockDriverState *bs, QCowL2Meta *m)
diff --git a/qemu-img.c b/qemu-img.c
index 158b3a505f..79983772de 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -3518,7 +3518,7 @@ static int img_rebase(int argc, char **argv)
  * to take action
  */
 ret = bdrv_is_allocated_above(backing_bs(bs),
prefix_chain_bs,
-  offset, n, );
+  false, offset, n, );
 if (ret < 0) {
 error_report("error while reading image metadata: %s",
  strerror(-ret));



signature.asc
Description: OpenPGP digital signature