Re: [Qemu-devel] [RFC v4 00/27] vSMMUv3/pSMMUv3 2 stage VFIO integration

[Auger Eric]

Hi Zhangfei,

On 7/11/19 3:53 AM, Zhangfei Gao wrote:
> On Mon, May 27, 2019 at 7:44 PM Eric Auger  wrote:
>>
>> Up to now vSMMUv3 has not been integrated with VFIO. VFIO
>> integration requires to program the physical IOMMU consistently
>> with the guest mappings. However, as opposed to VTD, SMMUv3 has
>> no "Caching Mode" which allows easy trapping of guest mappings.
>> This means the vSMMUV3 cannot use the same VFIO integration as VTD.
>>
>> However SMMUv3 has 2 translation stages. This was devised with
>> virtualization use case in mind where stage 1 is "owned" by the
>> guest whereas the host uses stage 2 for VM isolation.
>>
>> This series sets up this nested translation stage. It only works
>> if there is one physical SMMUv3 used along with QEMU vSMMUv3 (in
>> other words, it does not work if there is a physical SMMUv2).
>>
>> The series uses a new kernel user API [1], still under definition.
>>
>> - We force the host to use stage 2 instead of stage 1, when we
>>   detect a vSMMUV3 is behind a VFIO device. For a VFIO device
>>   without any virtual IOMMU, we still use stage 1 as many existing
>>   SMMUs expect this behavior.
>> - We introduce new IOTLB "config" notifiers, requested to notify
>>   changes in the config of a given iommu memory region. So now
>>   we have notifiers for IOTLB changes and config changes.
>> - vSMMUv3 calls config notifiers when STE (Stream Table Entries)
>>   are updated by the guest.
>> - We implement a specific UNMAP notifier that conveys guest
>>   IOTLB invalidations to the host
>> - We implement a new MAP notifiers only used for MSI IOVAs so
>>   that the host can build a nested stage translation for MSI IOVAs
>> - As the legacy MAP notifier is not called anymore, we must make
>>   sure stage 2 mappings are set. This is achieved through another
>>   memory listener.
>> - Physical SMMUs faults are reported to the guest via en eventfd
>>   mechanism and reinjected into this latter.
>>
>> Note: The first patch is a code cleanup and was sent separately.
>>
>> Best Regards
>>
>> Eric
>>
>> This series can be found at:
>> https://github.com/eauger/qemu/tree/v4.0.0-2stage-rfcv4
>>
>> Compatible with kernel series:
>> [PATCH v8 00/29] SMMUv3 Nested Stage Setup
>> (https://lkml.org/lkml/2019/5/26/95)
>>
> 
> Have tested vfio mode in qemu on arm64 platform.
> 
> Tested-by: Zhangfei Gao 
> qemu: https://github.com/eauger/qemu/tree/v4.0.0-2stage-rfcv4
> kernel: https://github.com/eauger/linux/tree/v5.2-rc1-2stage-v8

Your testing is really appreciated.

Both kernel and QEMU series will be respinned. I am currently waiting
for 5.3 kernel window as it will resolve some dependencies on the fault
reporting APIs. My focus is to get the updated kernel series reviewed
and tested and then refine the QEMU integration accordingly.

Thanks

Eric


> 

Re: [Qemu-devel] [RFC v2 PATCH 3/3] spapr: Add Hcalls to support PAPR NVDIMM device

[Shivaprasad G Bhat]

Thanks for the comments Fabiano.


On 05/22/2019 11:38 PM, Fabiano Rosas wrote:

Shivaprasad G Bhat  writes:

+

+ddc = NVDIMM_GET_CLASS(nvdimm);
+ddc->read_label_data(nvdimm, [0], numBytesToRead, offset);
+
+return H_SUCCESS;
+}
+
+
+static target_ulong h_scm_write_metadata(PowerPCCPU *cpu,
+ SpaprMachineState *spapr,
+ target_ulong opcode,
+ target_ulong *args)
+{
+uint32_t drc_index = args[0];
+uint64_t offset = args[1];
+uint64_t data = args[2];
+int8_t numBytesToWrite = args[3];

Likewise.


This is supposed to be uint64_t like used in other places.
Will fix it in the next version.

Rest of the comments I think David has already answered the rational
behind the usage to avoid integer overflow.




+SpaprDrc *drc = spapr_drc_by_index(drc_index);
+NVDIMMDevice *nvdimm = NULL;
+DeviceState *dev = NULL;



Regards,
Shivaprasad

Re: [Qemu-devel] [PATCH v1] x86: Intel AVX512_BF16 feature enabling

[no-reply]

Patchew URL: 
https://patchew.org/QEMU/1562823509-13072-1-git-send-email-jing2@linux.intel.com/



Hi,

This series failed build test on s390x host. Please find the details below.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
# Testing script will be invoked under the git checkout with
# HEAD pointing to a commit that has the patches applied on top of "base"
# branch
set -e

echo
echo "=== ENV ==="
env

echo
echo "=== PACKAGES ==="
rpm -qa

echo
echo "=== UNAME ==="
uname -a

CC=$HOME/bin/cc
INSTALL=$PWD/install
BUILD=$PWD/build
mkdir -p $BUILD $INSTALL
SRC=$PWD
cd $BUILD
$SRC/configure --cc=$CC --prefix=$INSTALL
make -j4
# XXX: we need reliable clean up
# make check -j4 V=1
make install
=== TEST SCRIPT END ===

  CC  aarch64-softmmu/hw/arm/virt-acpi-build.o
/usr/bin/ld: target/i386/cpu.o: in function `cpu_x86_cpuid':
/var/tmp/patchew-tester-tmp-f9kile96/src/target/i386/cpu.c:4317: undefined 
reference to `kvm_arch_get_supported_cpuid'
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:208: qemu-system-i386] Error 1
make: *** [Makefile:472: i386-softmmu/all] Error 2
make: *** Waiting for unfinished jobs


The full log is available at
http://patchew.org/logs/1562823509-13072-1-git-send-email-jing2@linux.intel.com/testing.s390x/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

Re: [Qemu-devel] [RFC v2 PATCH 3/3] spapr: Add Hcalls to support PAPR NVDIMM device

[Shivaprasad G Bhat]

On 05/14/2019 10:08 AM, David Gibson wrote:

On Mon, May 13, 2019 at 04:28:36AM -0500, Shivaprasad G Bhat wrote:

This patch implements few of the necessary hcalls for the nvdimm support.

PAPR semantics is such that each NVDIMM device is comprising of multiple
SCM(Storage Class Memory) blocks. The guest requests the hypervisor to bind
each of the SCM blocks of the NVDIMM device using hcalls. There can be
SCM block unbind requests in case of driver errors or unplug(not supported now)
use cases. The NVDIMM label read/writes are done through hcalls.

Since each virtual NVDIMM device is divided into multiple SCM blocks, the bind,
unbind, and queries using hcalls on those blocks can come independently. This
doesn't fit well into the qemu device semantics, where the map/unmap are done
at the (whole)device/object level granularity. The patch doesnt actually
bind/unbind on hcalls but let it happen at the object_add/del phase itself
instead.

The guest kernel makes bind/unbind requests for the virtual NVDIMM device at the
region level granularity. Without interleaving, each virtual NVDIMM device is
presented as separate region. There is no way to configure the virtual NVDIMM
interleaving for the guests today. So, there is no way a partial bind/unbind
request can come for the vNVDIMM in a hcall for a subset of SCM blocks of a
virtual NVDIMM. Hence it is safe to do bind/unbind everything during the
object_add/del.

Signed-off-by: Shivaprasad G Bhat 
---
  hw/ppc/spapr_hcall.c   |  202 
  include/hw/ppc/spapr.h |7 +-
  2 files changed, 208 insertions(+), 1 deletion(-)

diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c
index 6c16d2b120..b6e7d04dcf 100644
--- a/hw/ppc/spapr_hcall.c
+++ b/hw/ppc/spapr_hcall.c
@@ -3,11 +3,13 @@
  #include "sysemu/hw_accel.h"
  #include "sysemu/sysemu.h"
  #include "qemu/log.h"
+#include "qemu/range.h"
  #include "qemu/error-report.h"
  #include "cpu.h"
  #include "exec/exec-all.h"
  #include "helper_regs.h"
  #include "hw/ppc/spapr.h"
+#include "hw/ppc/spapr_drc.h"
  #include "hw/ppc/spapr_cpu_core.h"
  #include "mmu-hash64.h"
  #include "cpu-models.h"
@@ -16,6 +18,7 @@
  #include "hw/ppc/spapr_ovec.h"
  #include "mmu-book3s-v3.h"
  #include "hw/mem/memory-device.h"
+#include "hw/mem/nvdimm.h"
  
  static bool has_spr(PowerPCCPU *cpu, int spr)

  {
@@ -1795,6 +1798,199 @@ static target_ulong h_update_dt(PowerPCCPU *cpu, 
SpaprMachineState *spapr,
  return H_SUCCESS;
  }
  
+static target_ulong h_scm_read_metadata(PowerPCCPU *cpu,

+SpaprMachineState *spapr,
+target_ulong opcode,
+target_ulong *args)
+{
+uint32_t drc_index = args[0];
+uint64_t offset = args[1];
+uint64_t numBytesToRead = args[2];
+SpaprDrc *drc = spapr_drc_by_index(drc_index);
+NVDIMMDevice *nvdimm = NULL;
+NVDIMMClass *ddc = NULL;
+
+if (drc && spapr_drc_type(drc) != SPAPR_DR_CONNECTOR_TYPE_PMEM) {
+return H_PARAMETER;
+}
+
+if (numBytesToRead != 1 && numBytesToRead != 2 &&
+numBytesToRead != 4 && numBytesToRead != 8) {
+return H_P3;
+}
+
+nvdimm = NVDIMM(drc->dev);
+if ((offset + numBytesToRead < offset) ||
+(nvdimm->label_size < numBytesToRead + offset)) {
+return H_P2;
+}
+
+ddc = NVDIMM_GET_CLASS(nvdimm);
+ddc->read_label_data(nvdimm, [0], numBytesToRead, offset);

I'm pretty sure this will need some sort of byteswap.  args[0] is
effectively in host native order (since it maps to a register, not
memory).  But the guest will have to assume some byteorder (BE, I'm
guessing, because PAPR) in order to map that register to an in-memory
byteorder.  So, you'll need to compensate for that here.


You are right, I have figured out the required changes working with the 
kernel

team. Will post it in the next version.




+return H_SUCCESS;
+}
+
+
+static target_ulong h_scm_write_metadata(PowerPCCPU *cpu,
+ SpaprMachineState *spapr,
+ target_ulong opcode,
+ target_ulong *args)
+{
+uint32_t drc_index = args[0];
+uint64_t offset = args[1];
+uint64_t data = args[2];
+int8_t numBytesToWrite = args[3];
+SpaprDrc *drc = spapr_drc_by_index(drc_index);
+NVDIMMDevice *nvdimm = NULL;
+DeviceState *dev = NULL;
+NVDIMMClass *ddc = NULL;
+
+if (drc && spapr_drc_type(drc) != SPAPR_DR_CONNECTOR_TYPE_PMEM) {
+return H_PARAMETER;
+}
+
+if (numBytesToWrite != 1 && numBytesToWrite != 2 &&
+numBytesToWrite != 4 && numBytesToWrite != 8) {
+return H_P4;
+}
+
+dev = drc->dev;
+nvdimm = NVDIMM(dev);
+if ((nvdimm->label_size < numBytesToWrite + offset) ||
+(offset + numBytesToWrite < offset)) {
+return H_P2;
+}
+
+ddc = 

[Qemu-devel] [PATCH v1] x86: Intel AVX512_BF16 feature enabling

[Jing Liu]

Intel CooperLake cpu adds AVX512_BF16 instruction, defining as
CPUID.(EAX=7,ECX=1):EAX[bit 05].

The release spec link as follows,
https://software.intel.com/sites/default/files/managed/c5/15/\
architecture-instruction-set-extensions-programming-reference.pdf

Signed-off-by: Jing Liu 
---
 target/i386/cpu.c | 29 -
 target/i386/cpu.h |  3 +++
 target/i386/kvm.c |  3 ++-
 3 files changed, 33 insertions(+), 2 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index c1ab86d..de3daf5 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -767,6 +767,7 @@ static void x86_cpu_vendor_words2str(char *dst, uint32_t 
vendor1,
   /* CPUID_7_0_ECX_OSPKE is dynamic */ \
   CPUID_7_0_ECX_LA57)
 #define TCG_7_0_EDX_FEATURES 0
+#define TCG_7_1_EAX_FEATURES 0
 #define TCG_APM_FEATURES 0
 #define TCG_6_EAX_FEATURES CPUID_6_EAX_ARAT
 #define TCG_XSAVE_FEATURES (CPUID_XSAVE_XSAVEOPT | CPUID_XSAVE_XGETBV1)
@@ -1092,6 +1093,25 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] 
= {
 },
 .tcg_features = TCG_7_0_EDX_FEATURES,
 },
+[FEAT_7_1_EAX] = {
+.type = CPUID_FEATURE_WORD,
+.feat_names = {
+NULL, NULL, NULL, NULL,
+NULL, "avx512-bf16", NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+NULL, NULL, NULL, NULL,
+},
+.cpuid = {
+.eax = 7,
+.needs_ecx = true, .ecx = 1,
+.reg = R_EAX,
+},
+.tcg_features = TCG_7_1_EAX_FEATURES,
+},
 [FEAT_8000_0007_EDX] = {
 .type = CPUID_FEATURE_WORD,
 .feat_names = {
@@ -4344,13 +4364,20 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, 
uint32_t count,
 case 7:
 /* Structured Extended Feature Flags Enumeration Leaf */
 if (count == 0) {
-*eax = 0; /* Maximum ECX value for sub-leaves */
+/* Maximum ECX value for sub-leaves */
+*eax = kvm_arch_get_supported_cpuid(cs->kvm_state, 0x7,
+count, R_EAX);
 *ebx = env->features[FEAT_7_0_EBX]; /* Feature flags */
 *ecx = env->features[FEAT_7_0_ECX]; /* Feature flags */
 if ((*ecx & CPUID_7_0_ECX_PKU) && env->cr[4] & CR4_PKE_MASK) {
 *ecx |= CPUID_7_0_ECX_OSPKE;
 }
 *edx = env->features[FEAT_7_0_EDX]; /* Feature flags */
+} else if (count == 1) {
+*eax = env->features[FEAT_7_1_EAX];
+*ebx = 0;
+*ecx = 0;
+*edx = 0;
 } else {
 *eax = 0;
 *ebx = 0;
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index bd06523..40594a1 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -488,6 +488,7 @@ typedef enum FeatureWord {
 FEAT_7_0_EBX,   /* CPUID[EAX=7,ECX=0].EBX */
 FEAT_7_0_ECX,   /* CPUID[EAX=7,ECX=0].ECX */
 FEAT_7_0_EDX,   /* CPUID[EAX=7,ECX=0].EDX */
+FEAT_7_1_EAX,   /* CPUID[EAX=7,ECX=1].EAX */
 FEAT_8000_0001_EDX, /* CPUID[8000_0001].EDX */
 FEAT_8000_0001_ECX, /* CPUID[8000_0001].ECX */
 FEAT_8000_0007_EDX, /* CPUID[8000_0007].EDX */
@@ -699,6 +700,8 @@ typedef uint32_t FeatureWordArray[FEATURE_WORDS];
 #define CPUID_7_0_EDX_ARCH_CAPABILITIES (1U << 29)  /*Arch Capabilities*/
 #define CPUID_7_0_EDX_SPEC_CTRL_SSBD  (1U << 31) /* Speculative Store Bypass 
Disable */
 
+#define CPUID_7_1_EAX_AVX512_BF16 (1U << 5) /* AVX512 BFloat16 Instruction */
+
 #define CPUID_8000_0008_EBX_WBNOINVD  (1U << 9)  /* Write back and
  
do not invalidate cache */
 #define CPUID_8000_0008_EBX_IBPB(1U << 12) /* Indirect Branch Prediction 
Barrier */
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 3b29ce5..977aaa5 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1110,6 +1110,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
 c = _data.entries[cpuid_i++];
 }
 break;
+case 0x7:
 case 0x14: {
 uint32_t times;
 
@@ -1122,7 +1123,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
 for (j = 1; j <= times; ++j) {
 if (cpuid_i == KVM_MAX_CPUID_ENTRIES) {
 fprintf(stderr, "cpuid_data is full, no space for "
-"cpuid(eax:0x14,ecx:0x%x)\n", j);
+"cpuid(eax:0x%x,ecx:0x%x)\n", i, j);
 abort();
 }
 c = _data.entries[cpuid_i++];
-- 
1.8.3.1

[Qemu-devel] qemu compilation failure with nettle 3.5.1

[Amol Surati]

Hi,

The qemu upstream (at commit 6df2cdf44a at the moment) fails to compile
with nettle 3.5.1. It seems that Nettle has deprecated a few parts of
its API.

A workaround is to provide --disable-nettle during qemu configuration.

A portion of the error log:

error: ‘nettle_aes_encrypt’ is deprecated
error: ‘nettle_aes_decrypt’ is deprecated
error: ‘nettle_aes_encrypt’ is deprecated
error: ‘nettle_aes_decrypt’ is deprecated
error: ‘nettle_aes_set_encrypt_key’ is deprecated
error: ‘nettle_aes_set_decrypt_key’ is deprecated
error: ‘nettle_aes_set_encrypt_key’ is deprecated
error: ‘nettle_aes_set_decrypt_key’ is deprecated
error: ‘nettle_aes_set_encrypt_key’ is deprecated
error: ‘nettle_aes_set_decrypt_key’ is deprecated


The declaration of one of the functions from nettle's aes.h:

void
aes_encrypt(const struct aes_ctx *ctx,
size_t length, uint8_t *dst,
const uint8_t *src) _NETTLE_ATTRIBUTE_DEPRECATED;

Thanks,
Amol

[Qemu-devel] [Bug 1836136] [NEW] u-boot: any plans to update u-boot to v2019.07

[Rebecca Chang Swee Fun]

Public bug reported:

Just want to pulse about the plan to update u-boot binary to latest
v2019.07.

** Affects: qemu
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1836136

Title:
  u-boot: any plans to update u-boot to v2019.07

Status in QEMU:
  New

Bug description:
  Just want to pulse about the plan to update u-boot binary to latest
  v2019.07.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1836136/+subscriptions

Re: [Qemu-devel] [PATCH v0] ppc/spapr: Support reboot of secure pseries guest

[Bharata B Rao]

On Thu, Jul 11, 2019 at 11:31:00AM +1000, David Gibson wrote:
> On Wed, Jul 10, 2019 at 11:36:21AM +0530, Bharata B Rao wrote:
> > A pseries guest can be run as a secure guest on Ultravisor-enabled
> > POWER platforms. When such a secure guest is reset, we need to
> > release/reset a few resources both on ultravisor and hypervisor side.
> > This is achieved by invoking this new ioctl KVM_PPC_SVM_OFF from the
> > machine reset path.
> > 
> > As part of this ioctl, the secure guest is essentially transitioned
> > back to normal mode so that it can reboot like a regular guest and
> > become secure again.
> > 
> > This ioctl has no effect when invoked for a normal guest.
> > 
> > Signed-off-by: Bharata B Rao 
> > ---
> > * The ioctl implementation in the kernel can be found as part of this 
> > patchset:
> >   https://www.spinics.net/lists/linux-mm/msg184366.html
> > * Updated linux-headers/linux/kvm.h here for completeness as the
> >   definition of KVM_PPC_SVM_OFF isn't yet part of upstream kernel.
> 
> The qemu change looks good to me.  To actually merge this, the support
> will need to go upstream in the kernel first, then we'll need an
> update-kernel-headers as a separate patch.

Sure, Thanks. Will post again once ioctl support is included in the
kernel.

Regards,
Bharata.

Re: [Qemu-devel] [PATCH v7 00/13] Add migration support for VFIO device

[Yan Zhao]

Hi Kirti,
There are still unaddressed comments to your patches v4.
Would you mind addressing them?

1. should we register two migration interfaces simultaneously
(https://lists.gnu.org/archive/html/qemu-devel/2019-06/msg04750.html)
2. in each save iteration, how much data is to be saved
(https://lists.gnu.org/archive/html/qemu-devel/2019-06/msg04683.html)
3. do we need extra interface to get data for device state only
(https://lists.gnu.org/archive/html/qemu-devel/2019-06/msg04812.html)
4. definition of dirty page copied_pfn
(https://lists.gnu.org/archive/html/qemu-devel/2019-06/msg05592.html)

Also, I'm glad to see that you updated code by following my comments below,
but please don't forget to reply my comments next time:)
https://lists.gnu.org/archive/html/qemu-devel/2019-06/msg05357.html
https://lists.gnu.org/archive/html/qemu-devel/2019-06/msg06454.html

Thanks
Yan

On Tue, Jul 09, 2019 at 05:49:07PM +0800, Kirti Wankhede wrote:
> Add migration support for VFIO device
> 
> This Patch set include patches as below:
> - Define KABI for VFIO device for migration support.
> - Added save and restore functions for PCI configuration space
> - Generic migration functionality for VFIO device.
>   * This patch set adds functionality only for PCI devices, but can be
> extended to other VFIO devices.
>   * Added all the basic functions required for pre-copy, stop-and-copy and
> resume phases of migration.
>   * Added state change notifier and from that notifier function, VFIO
> device's state changed is conveyed to VFIO device driver.
>   * During save setup phase and resume/load setup phase, migration region
> is queried and is used to read/write VFIO device data.
>   * .save_live_pending and .save_live_iterate are implemented to use QEMU's
> functionality of iteration during pre-copy phase.
>   * In .save_live_complete_precopy, that is in stop-and-copy phase,
> iteration to read data from VFIO device driver is implemented till pending
> bytes returned by driver are not zero.
>   * Added function to get dirty pages bitmap for the pages which are used by
> driver.
> - Add vfio_listerner_log_sync to mark dirty pages.
> - Make VFIO PCI device migration capable. If migration region is not provided 
> by
>   driver, migration is blocked.
> 
> Below is the flow of state change for live migration where states in brackets
> represent VM state, migration state and VFIO device state as:
> (VM state, MIGRATION_STATUS, VFIO_DEVICE_STATE)
> 
> Live migration save path:
> QEMU normal running state
> (RUNNING, _NONE, _RUNNING)
> |
> migrate_init spawns migration_thread.
> (RUNNING, _SETUP, _RUNNING|_SAVING)
> Migration thread then calls each device's .save_setup()
> |
> (RUNNING, _ACTIVE, _RUNNING|_SAVING)
> If device is active, get pending bytes by .save_live_pending()
> if pending bytes >= threshold_size,  call save_live_iterate()
> Data of VFIO device for pre-copy phase is copied.
> Iterate till pending bytes converge and are less than threshold
> |
> On migration completion, vCPUs stops and calls .save_live_complete_precopy
> for each active device. VFIO device is then transitioned in
>  _SAVING state.
> (FINISH_MIGRATE, _DEVICE, _SAVING)
> For VFIO device, iterate in  .save_live_complete_precopy  until
> pending data is 0.
> (FINISH_MIGRATE, _DEVICE, _STOPPED)
> |
> (FINISH_MIGRATE, _COMPLETED, STOPPED)
> Migraton thread schedule cleanup bottom half and exit
> 
> Live migration resume path:
> Incomming migration calls .load_setup for each device
> (RESTORE_VM, _ACTIVE, STOPPED)
> |
> For each device, .load_state is called for that device section data
> |
> At the end, called .load_cleanup for each device and vCPUs are started.
> |
> (RUNNING, _NONE, _RUNNING)
> 
> Note that:
> - Migration post copy is not supported.
> 
> v6 -> v7:
> - Fix build failures.
> 
> v5 -> v6:
> - Fix build failure.
> 
> v4 -> v5:
> - Added decriptive comment about the sequence of access of members of 
> structure
>   vfio_device_migration_info to be followed based on Alex's suggestion
> - Updated get dirty pages sequence.
> - As per Cornelia Huck's suggestion, added callbacks to VFIODeviceOps to
>   get_object, save_config and load_config.
> - Fixed multiple nit picks.
> - Tested live migration with multiple vfio device assigned to a VM.
> 
> v3 -> v4:
> - Added one more bit for _RESUMING flag to be set explicitly.
> - data_offset field is read-only for user space application.
> - data_size is read for every iteration before reading data from migration, 
> that
>   is removed assumption that data will be till end of migration region.
> - If vendor driver supports mappable sparsed region, map those region during
>   setup state of 

Re: [Qemu-devel] [PATCH v25 5/7] target/avr: Add limited support for USART and 16 bit timer peripherals

[Michael Rolnik]

Hi Philippe.

Thanks for you review.
Please explain how to link a device.

Regards,
Michael

On Wed, Jul 10, 2019 at 1:06 PM Philippe Mathieu-Daudé 
wrote:

> Hi Michael, Sarah.
>
> Sorry for reviewing this series that late.
>
> On 7/9/19 11:36 PM, Michael Rolnik wrote:
> > From: Sarah Harris 
> >
> > These were designed to facilitate testing but should provide enough
> function to be useful in other contexts.
> > Only a subset of the functions of each peripheral is implemented, mainly
> due to the lack of a standard way to handle electrical connections (like
> GPIO pins).
> >
> > Signed-off-by: Michael Rolnik 
> > ---
> >  hw/char/Kconfig|   3 +
> >  hw/char/Makefile.objs  |   1 +
> >  hw/char/avr_usart.c| 316 ++
> >  hw/timer/Kconfig   |   3 +
> >  hw/timer/Makefile.objs |   1 +
> >  hw/timer/avr_timer16.c | 587 +
> >  include/hw/char/avr_usart.h|  99 ++
> >  include/hw/timer/avr_timer16.h |  99 ++
> >  8 files changed, 1109 insertions(+)
> >  create mode 100644 hw/char/avr_usart.c
> >  create mode 100644 hw/timer/avr_timer16.c
> >  create mode 100644 include/hw/char/avr_usart.h
> >  create mode 100644 include/hw/timer/avr_timer16.h
> >
> > diff --git a/hw/char/Kconfig b/hw/char/Kconfig
> > index 40e7a8b8bb..331b20983f 100644
> > --- a/hw/char/Kconfig
> > +++ b/hw/char/Kconfig
> > @@ -46,3 +46,6 @@ config SCLPCONSOLE
> >
> >  config TERMINAL3270
> >  bool
> > +
> > +config AVR_USART
> > +bool
> > diff --git a/hw/char/Makefile.objs b/hw/char/Makefile.objs
> > index 02d8a66925..09ed50f1d0 100644
> > --- a/hw/char/Makefile.objs
> > +++ b/hw/char/Makefile.objs
> > @@ -21,6 +21,7 @@ obj-$(CONFIG_PSERIES) += spapr_vty.o
> >  obj-$(CONFIG_DIGIC) += digic-uart.o
> >  obj-$(CONFIG_STM32F2XX_USART) += stm32f2xx_usart.o
> >  obj-$(CONFIG_RASPI) += bcm2835_aux.o
> > +obj-$(CONFIG_AVR_USART) += avr_usart.o
> >
> >  common-obj-$(CONFIG_CMSDK_APB_UART) += cmsdk-apb-uart.o
> >  common-obj-$(CONFIG_ETRAXFS) += etraxfs_ser.o
> > diff --git a/hw/char/avr_usart.c b/hw/char/avr_usart.c
> > new file mode 100644
> > index 00..26c711336b
> > --- /dev/null
> > +++ b/hw/char/avr_usart.c
> > @@ -0,0 +1,316 @@
> > +/*
> > + * AVR USART
> > + *
> > + * Copyright (c) 2018 University of Kent
> > + * Author: Sarah Harris
> > + *
> > + * Permission is hereby granted, free of charge, to any person
> obtaining a copy
> > + * of this software and associated documentation files (the
> "Software"), to deal
> > + * in the Software without restriction, including without limitation
> the rights
> > + * to use, copy, modify, merge, publish, distribute, sublicense, and/or
> sell
> > + * copies of the Software, and to permit persons to whom the Software is
> > + * furnished to do so, subject to the following conditions:
> > + *
> > + * The above copyright notice and this permission notice shall be
> included in
> > + * all copies or substantial portions of the Software.
> > + *
> > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
> EXPRESS OR
> > + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> MERCHANTABILITY,
> > + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
> SHALL
> > + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
> OTHER
> > + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
> ARISING FROM,
> > + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
> DEALINGS IN
> > + * THE SOFTWARE.
> > + */
> > +
> > +#include "qemu/osdep.h"
> > +#include "hw/char/avr_usart.h"
> > +#include "qemu/log.h"
> > +
> > +static int avr_usart_can_receive(void *opaque)
> > +{
> > +AVRUsartState *usart = opaque;
> > +
> > +if (usart->data_valid || !(usart->csrb & USART_CSRB_RXEN)) {
> > +return 0;
> > +}
> > +return 1;
> > +}
> > +
> > +static void avr_usart_receive(void *opaque, const uint8_t *buffer, int
> size)
> > +{
> > +AVRUsartState *usart = opaque;
> > +assert(size == 1);
> > +assert(!usart->data_valid);
> > +usart->data = buffer[0];
> > +usart->data_valid = true;
> > +usart->csra |= USART_CSRA_RXC;
> > +if (usart->csrb & USART_CSRB_RXCIE) {
> > +qemu_set_irq(usart->rxc_irq, 1);
> > +}
> > +}
> > +
> > +static void update_char_mask(AVRUsartState *usart)
> > +{
> > +uint8_t mode = ((usart->csrc & USART_CSRC_CSZ0) ? 1 : 0) |
> > +((usart->csrc & USART_CSRC_CSZ1) ? 2 : 0) |
> > +((usart->csrb & USART_CSRB_CSZ2) ? 4 : 0);
> > +switch (mode) {
> > +case 0:
> > +usart->char_mask = 0b1;
> > +break;
> > +case 1:
> > +usart->char_mask = 0b11;
> > +break;
> > +case 2:
> > +usart->char_mask = 0b111;
> > +break;
> > +case 3:
> > +usart->char_mask = 0b;
> > +break;
> > +case 4:
> > +/* Fallthrough. */
> > +case 5:
> > +   

Re: [Qemu-devel] [PATCH v0] ppc/spapr: Support reboot of secure pseries guest

[David Gibson]

On Wed, Jul 10, 2019 at 11:36:21AM +0530, Bharata B Rao wrote:
> A pseries guest can be run as a secure guest on Ultravisor-enabled
> POWER platforms. When such a secure guest is reset, we need to
> release/reset a few resources both on ultravisor and hypervisor side.
> This is achieved by invoking this new ioctl KVM_PPC_SVM_OFF from the
> machine reset path.
> 
> As part of this ioctl, the secure guest is essentially transitioned
> back to normal mode so that it can reboot like a regular guest and
> become secure again.
> 
> This ioctl has no effect when invoked for a normal guest.
> 
> Signed-off-by: Bharata B Rao 
> ---
> * The ioctl implementation in the kernel can be found as part of this 
> patchset:
>   https://www.spinics.net/lists/linux-mm/msg184366.html
> * Updated linux-headers/linux/kvm.h here for completeness as the
>   definition of KVM_PPC_SVM_OFF isn't yet part of upstream kernel.

The qemu change looks good to me.  To actually merge this, the support
will need to go upstream in the kernel first, then we'll need an
update-kernel-headers as a separate patch.

> 
>  hw/ppc/spapr.c| 1 +
>  linux-headers/linux/kvm.h | 1 +
>  target/ppc/kvm.c  | 7 +++
>  target/ppc/kvm_ppc.h  | 6 ++
>  4 files changed, 15 insertions(+)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 821f0d4a49..6abf71f159 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -1709,6 +1709,7 @@ static void spapr_machine_reset(MachineState *machine)
>  void *fdt;
>  int rc;
>  
> +kvmppc_svm_off();
>  spapr_caps_apply(spapr);
>  
>  first_ppc_cpu = POWERPC_CPU(first_cpu);
> diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
> index c8423e760c..9603fef9bf 100644
> --- a/linux-headers/linux/kvm.h
> +++ b/linux-headers/linux/kvm.h
> @@ -1327,6 +1327,7 @@ struct kvm_s390_ucas_mapping {
>  #define KVM_PPC_GET_RMMU_INFO  _IOW(KVMIO,  0xb0, struct 
> kvm_ppc_rmmu_info)
>  /* Available with KVM_CAP_PPC_GET_CPU_CHAR */
>  #define KVM_PPC_GET_CPU_CHAR   _IOR(KVMIO,  0xb1, struct kvm_ppc_cpu_char)
> +#define KVM_PPC_SVM_OFF   _IO(KVMIO, 0xb2)
>  
>  /* ioctl for vm fd */
>  #define KVM_CREATE_DEVICE  _IOWR(KVMIO,  0xe0, struct kvm_create_device)
> diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
> index 8a06d3171e..079d83ce6c 100644
> --- a/target/ppc/kvm.c
> +++ b/target/ppc/kvm.c
> @@ -2953,3 +2953,10 @@ void kvmppc_set_reg_tb_offset(PowerPCCPU *cpu, int64_t 
> tb_offset)
>  kvm_set_one_reg(cs, KVM_REG_PPC_TB_OFFSET, _offset);
>  }
>  }
> +
> +int kvmppc_svm_off(void)
> +{
> +KVMState *s = KVM_STATE(current_machine->accelerator);
> +
> +return kvm_vm_ioctl(s, KVM_PPC_SVM_OFF);
> +}
> diff --git a/target/ppc/kvm_ppc.h b/target/ppc/kvm_ppc.h
> index 98bd7d5da6..0fd80e1100 100644
> --- a/target/ppc/kvm_ppc.h
> +++ b/target/ppc/kvm_ppc.h
> @@ -37,6 +37,7 @@ int kvmppc_booke_watchdog_enable(PowerPCCPU *cpu);
>  target_ulong kvmppc_configure_v3_mmu(PowerPCCPU *cpu,
>   bool radix, bool gtse,
>   uint64_t proc_tbl);
> +int kvmppc_svm_off(void);
>  #ifndef CONFIG_USER_ONLY
>  bool kvmppc_spapr_use_multitce(void);
>  int kvmppc_spapr_enable_inkernel_multitce(void);
> @@ -201,6 +202,11 @@ static inline target_ulong 
> kvmppc_configure_v3_mmu(PowerPCCPU *cpu,
>  return 0;
>  }
>  
> +static inline int kvmppc_svm_off(void)
> +{
> + return 0;
> +}
> +
>  static inline void kvmppc_set_reg_ppc_online(PowerPCCPU *cpu,
>   unsigned int online)
>  {

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] [Qemu-ppc] [PULL 41/44] spapr: change default interrupt mode to 'dual'

[David Gibson]

On Wed, Jul 10, 2019 at 06:26:09PM +0200, Laurent Vivier wrote:
> On 29/05/2019 08:50, David Gibson wrote:
> > From: Cédric Le Goater 
> > 
> > Now that XIVE support is complete (QEMU emulated and KVM devices),
> > change the pseries machine to advertise both interrupt modes: XICS
> > (P7/P8) and XIVE (P9).
> > 
> > The machine default interrupt modes depends on the version. Current
> > settings are:
> > 
> > pseries   default interrupt mode
> > 
> > 4.1   dual
> > 4.0   xics
> > 3.1   xics
> > 3.0   legacy xics (different IRQ number space layout)
> > 
> > Signed-off-by: Cédric Le Goater 
> > Message-Id: <20190522074016.10521-3-...@kaod.org>
> > Reviewed-by: Greg Kurz 
> > Signed-off-by: David Gibson 
> > ---
> >  hw/ppc/spapr.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index 39e698e9b0..4fd16b43f0 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -4352,7 +4352,7 @@ static void spapr_machine_class_init(ObjectClass *oc, 
> > void *data)
> >  smc->default_caps.caps[SPAPR_CAP_LARGE_DECREMENTER] = SPAPR_CAP_ON;
> >  smc->default_caps.caps[SPAPR_CAP_CCF_ASSIST] = SPAPR_CAP_OFF;
> >  spapr_caps_add_properties(smc, _abort);
> > -smc->irq = _irq_xics;
> > +smc->irq = _irq_dual;
> >  smc->dr_phb_enabled = true;
> >  }
> >  
> > @@ -4430,6 +4430,7 @@ static void 
> > spapr_machine_4_0_class_options(MachineClass *mc)
> >  spapr_machine_4_1_class_options(mc);
> >  compat_props_add(mc->compat_props, hw_compat_4_0, hw_compat_4_0_len);
> >  smc->phb_placement = phb_placement_4_0;
> > +smc->irq = _irq_xics;
> >  }
> >  
> >  DEFINE_SPAPR_MACHINE(4_0, "4.0", false);
> > 
> 
> This patch breaks the '-no-reboot' parameter (I think the "dual" mode
> breaks the -no-reboot parameter)
> 
> After grub loads the kernel and starts it, the kernel aborts:
> 
> OF stdout device is: /vdevice/vty@7100
> Preparing to boot Linux version 4.18.0-112.el8.ppc64le
> (mockbu...@ppc-061.build.eng.bos.redhat.com) (gcc version 8.3.1 20190507
> (Red Hat 8.3.1-4) (GCC)) #1 SMP Fri Jul 5 11:21:28 UTC 2019
> Detected machine type: 0101
> command line: BOOT_IMAGE=/vmlinuz-4.18.0-112.el8.ppc64le
> root=/dev/mapper/rhel_ibm--p8--kvm--03--guest--02-root ro
> crashkernel=auto rd.lvm.lv=rhel_ibm-p8-kvm-03-guest-02/root
> rd.lvm.lv=rhel_ibm-p8-kvm-03-guest-02/swap
> Max number of cores passed to firmware: 256 (NR_CPUS = 2048)
> Calling ibm,client-architecture-support...[lvivier@localhost ~]$
> 
> I bisected to this patch, and then after I understood the problem is
> with the -no-reboot parameter as the machine is reset by the CAS
> negotiation... and the -no-reboot prevents this reset.
> 
> I don't know if it's a real problem or not.

Ah, bother.  I didn't think of the interaction between the CAS reboot
and -no-reboot.  I guess that's more reason to work out a way to do
the xics/xive switch without a full reset.  People were already not
thrilled with the extra reboots here.

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH v0] ppc/spapr: Support reboot of secure pseries guest

[David Gibson]

On Tue, Jul 09, 2019 at 11:12:13PM -0700, no-re...@patchew.org wrote:
> Patchew URL: 
> https://patchew.org/QEMU/20190710060621.16430-1-bhar...@linux.ibm.com/
> 
> 
> 
> Hi,
> 
> This series seems to have some coding style problems. See output below for
> more information:
> 
> Message-id: 20190710060621.16430-1-bhar...@linux.ibm.com
> Type: series
> Subject: [Qemu-devel] [PATCH v0] ppc/spapr: Support reboot of secure pseries 
> guest
> 
> === TEST SCRIPT BEGIN ===
> #!/bin/bash
> git rev-parse base > /dev/null || exit 0
> git config --local diff.renamelimit 0
> git config --local diff.renames True
> git config --local diff.algorithm histogram
> ./scripts/checkpatch.pl --mailback base..
> === TEST SCRIPT END ===
> 
> From https://github.com/patchew-project/qemu
>  * [new tag]   
> patchew/20190710060621.16430-1-bhar...@linux.ibm.com -> 
> patchew/20190710060621.16430-1-bhar...@linux.ibm.com
> Switched to a new branch 'test'
> 636a27f80f ppc/spapr: Support reboot of secure pseries guest
> 
> === OUTPUT BEGIN ===
> ERROR: code indent should never use tabs
> #79: FILE: target/ppc/kvm_ppc.h:207:
> +^Ireturn 0;$

Ah, yeah, fix this too, please.

> total: 1 errors, 0 warnings, 42 lines checked
> 
> Commit 636a27f80f0d (ppc/spapr: Support reboot of secure pseries guest) has 
> style problems, please review.  If any of these errors
> are false positives report them to the maintainer, see
> CHECKPATCH in MAINTAINERS.
> === OUTPUT END ===
> 
> Test command exited with code: 1
> 
> 
> The full log is available at
> http://patchew.org/logs/20190710060621.16430-1-bhar...@linux.ibm.com/testing.checkpatch/?type=message.
> ---
> Email generated automatically by Patchew [https://patchew.org/].
> Please send your feedback to patchew-de...@redhat.com

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH] tests/boot_linux_console: add a test for riscv64 + virt

[Chih-Min Chao]

On Thu, Jul 11, 2019 at 4:23 AM Cleber Rosa  wrote:

> On Wed, Jul 10, 2019 at 10:50:23AM -0700, Chih-Min Chao wrote:
> > Similar to the mips + malta test, it boots a Linux kernel on a virt
> > board and verify the serial is working.  Also, it relies on the serial
> > device set by the machine itself.
> >
> > If riscv64 is a target being built, "make check-acceptance" will
> > automatically include this test by the use of the "arch:riscv64" tags.
> >
> > Alternatively, this test can be run using:
> >
> >   $ avocado run -t arch:riscv64 tests/acceptance
> >
> > Signed-off-by: Chih-Min Chao 
> > ---
> >  tests/acceptance/boot_linux_console.py | 40
> ++
> >  1 file changed, 40 insertions(+)
> >
> > diff --git a/tests/acceptance/boot_linux_console.py
> b/tests/acceptance/boot_linux_console.py
> > index 3215950..bbc6b06 100644
> > --- a/tests/acceptance/boot_linux_console.py
> > +++ b/tests/acceptance/boot_linux_console.py
> > @@ -354,3 +354,43 @@ class BootLinuxConsole(Test):
> >  self.vm.launch()
> >  console_pattern = 'Kernel command line: %s' %
> kernel_command_line
> >  self.wait_for_console_pattern(console_pattern)
> > +
> > +def test_riscv64_virt(self):
> > +"""
> > +:avocado: tags=arch:riscv64
> > +:avocado: tags=machine:virt
> > +"""
> > +
> > +kernel_url = ('https://github.com/chihminchao/test-binary/raw/'
> > +
> '0b7787305d9e40815c05a805266cc74ff356239e/qemu/riscv64/'
> > +  'bbl_w_kernel.gz')
> > +kernel_hash = 'c7f6cc7967975ad42dc61ee0535db01c9cbd0968'
> > +kernel_path_gz = self.fetch_asset(kernel_url,
> asset_hash=kernel_hash)
> > +kernel_path = self.workdir + "bbl_w_kernel"
> > +
> > +with gzip.open(kernel_path_gz, 'rb') as f_in:
> > +with open(kernel_path, 'wb') as f_out:
> > +shutil.copyfileobj(f_in, f_out)
>
> There are currently two patterns for extracting a gzipped file
> in this test.  So, this is not a must, but maybe you'd prefer:
>

  The suggestion is good and avocado is an awesome testing framework.

---
> diff --git a/tests/acceptance/boot_linux_console.py
> b/tests/acceptance/boot_linux_console.py
> index bbc6b0683f..9f819e20e1 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -365,12 +365,8 @@ class BootLinuxConsole(Test):
>
>  '0b7787305d9e40815c05a805266cc74ff356239e/qemu/riscv64/'
>'bbl_w_kernel.gz')
>  kernel_hash = 'c7f6cc7967975ad42dc61ee0535db01c9cbd0968'
> -kernel_path_gz = self.fetch_asset(kernel_url,
> asset_hash=kernel_hash)
> -kernel_path = self.workdir + "bbl_w_kernel"
> -
> -with gzip.open(kernel_path_gz, 'rb') as f_in:
> -with open(kernel_path, 'wb') as f_out:
> -shutil.copyfileobj(f_in, f_out)
> +kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
> +uncompressed_kernel = archive.uncompress(kernel_path,
> self.workdir)
>
>  initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
>'8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/'
> @@ -382,7 +378,7 @@ class BootLinuxConsole(Test):
>  self.vm.set_console()
>  kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
> + 'console=ttyS0 noreboot')
> -self.vm.add_args('-kernel', kernel_path,
> +self.vm.add_args('-kernel', uncompressed_kernel,
>   '-initrd', initrd_path,
>   '-append', kernel_command_line)
>  self.vm.launch()
> ---
>
> > +
> > +initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
> > +  '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/'
> > +  'riscv64/rootfs.cpio.gz')
> > +initrd_hash = 'f4867d263754961b6f626cdcdc0cb334c47e3b49'
> > +initrd_path = self.fetch_asset(initrd_url,
> asset_hash=initrd_hash)
> > +
> > +self.vm.set_machine('virt')
> > +self.vm.set_console()
> > +kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
> > +   + 'console=ttyS0 noreboot')
> > +self.vm.add_args('-kernel', kernel_path,
> > + '-initrd', initrd_path,
> > + '-append', kernel_command_line)
> > +self.vm.launch()
> > +self.wait_for_console_pattern('Boot successful.')
> > +
> > +self.exec_command_and_wait_for_pattern('cat /proc/cpuinfo',
> > +   'isa')
> > +self.exec_command_and_wait_for_pattern('uname -a',
> > +   'sifive')
> > +self.exec_command_and_wait_for_pattern('reboot',
> > +   'reboot: Restarting
> system')
> > --
> > 2.7.4
> >
> >
>
> It'd be nice to also add riscv64 

Re: [Qemu-devel] [PATCH] tests/boot_linux_console: add a test for riscv64 + virt

[Chih-Min Chao]

On Thu, Jul 11, 2019 at 2:53 AM Alistair Francis 
wrote:

> On Wed, Jul 10, 2019 at 10:51 AM Chih-Min Chao 
> wrote:
> >
> > Similar to the mips + malta test, it boots a Linux kernel on a virt
> > board and verify the serial is working.  Also, it relies on the serial
> > device set by the machine itself.
> >
> > If riscv64 is a target being built, "make check-acceptance" will
> > automatically include this test by the use of the "arch:riscv64" tags.
> >
> > Alternatively, this test can be run using:
> >
> >   $ avocado run -t arch:riscv64 tests/acceptance
> >
> > Signed-off-by: Chih-Min Chao 
>
> Awesome! Thanks for the test case. This will help a lot with RISC-V
> regressions in QEMU.
>
> > ---
> >  tests/acceptance/boot_linux_console.py | 40
> ++
> >  1 file changed, 40 insertions(+)
> >
> > diff --git a/tests/acceptance/boot_linux_console.py
> b/tests/acceptance/boot_linux_console.py
> > index 3215950..bbc6b06 100644
> > --- a/tests/acceptance/boot_linux_console.py
> > +++ b/tests/acceptance/boot_linux_console.py
> > @@ -354,3 +354,43 @@ class BootLinuxConsole(Test):
> >  self.vm.launch()
> >  console_pattern = 'Kernel command line: %s' %
> kernel_command_line
> >  self.wait_for_console_pattern(console_pattern)
> > +
> > +def test_riscv64_virt(self):
> > +"""
> > +:avocado: tags=arch:riscv64
> > +:avocado: tags=machine:virt
> > +"""
> > +
> > +kernel_url = ('https://github.com/chihminchao/test-binary/raw/'
>
> These images need to be built with a standard build flow. Having them
> built from SiFive's custom scripts will make debugging problems in the
> future impossible. I'm also a little worried here about GPL
> violations, I'm not sure if it's enough to just point to a script SHA
> to meet GPL source disclosure. I know companies have huge headaches
> meeting GPL requirements so this seems too easy.
>

   I am not very familiar with this kind of binary and source license
conflict.
   Is it ok if I write a simple script with BSD license to build  kernel
and image from sifive's linux/buildroot repo and commit the script to my
test-binary repo ?

>
> > +
> '0b7787305d9e40815c05a805266cc74ff356239e/qemu/riscv64/'
> > +  'bbl_w_kernel.gz')
>
> Don't use BBL, most people use OpenSBI now which is what we should be
> testing with.
>
   I will try to move to OpenSBI in next version. My environment is based
freedom-u-sdk and It still relays  on BBL. So ..:P

>
> > +kernel_hash = 'c7f6cc7967975ad42dc61ee0535db01c9cbd0968'
> > +kernel_path_gz = self.fetch_asset(kernel_url,
> asset_hash=kernel_hash)
> > +kernel_path = self.workdir + "bbl_w_kernel"
> > +
> > +with gzip.open(kernel_path_gz, 'rb') as f_in:
> > +with open(kernel_path, 'wb') as f_out:
> > +shutil.copyfileobj(f_in, f_out)
> > +
> > +initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
> > +  '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/'
> > +  'riscv64/rootfs.cpio.gz')
>
> Same comment about build tools.
>
>
> Alistair
>
> Got it

> > +initrd_hash = 'f4867d263754961b6f626cdcdc0cb334c47e3b49'
> > +initrd_path = self.fetch_asset(initrd_url,
> asset_hash=initrd_hash)
> > +
> > +self.vm.set_machine('virt')
> > +self.vm.set_console()
> > +kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
> > +   + 'console=ttyS0 noreboot')
> > +self.vm.add_args('-kernel', kernel_path,
> > + '-initrd', initrd_path,
> > + '-append', kernel_command_line)
> > +self.vm.launch()
> > +self.wait_for_console_pattern('Boot successful.')
> > +
> > +self.exec_command_and_wait_for_pattern('cat /proc/cpuinfo',
> > +   'isa')
> > +self.exec_command_and_wait_for_pattern('uname -a',
> > +   'sifive')
> > +self.exec_command_and_wait_for_pattern('reboot',
> > +   'reboot: Restarting
> system')
> > --
> > 2.7.4
> >
> >
>

Re: [Qemu-devel] [RFC v4 00/27] vSMMUv3/pSMMUv3 2 stage VFIO integration

[Zhangfei Gao]

On Mon, May 27, 2019 at 7:44 PM Eric Auger  wrote:
>
> Up to now vSMMUv3 has not been integrated with VFIO. VFIO
> integration requires to program the physical IOMMU consistently
> with the guest mappings. However, as opposed to VTD, SMMUv3 has
> no "Caching Mode" which allows easy trapping of guest mappings.
> This means the vSMMUV3 cannot use the same VFIO integration as VTD.
>
> However SMMUv3 has 2 translation stages. This was devised with
> virtualization use case in mind where stage 1 is "owned" by the
> guest whereas the host uses stage 2 for VM isolation.
>
> This series sets up this nested translation stage. It only works
> if there is one physical SMMUv3 used along with QEMU vSMMUv3 (in
> other words, it does not work if there is a physical SMMUv2).
>
> The series uses a new kernel user API [1], still under definition.
>
> - We force the host to use stage 2 instead of stage 1, when we
>   detect a vSMMUV3 is behind a VFIO device. For a VFIO device
>   without any virtual IOMMU, we still use stage 1 as many existing
>   SMMUs expect this behavior.
> - We introduce new IOTLB "config" notifiers, requested to notify
>   changes in the config of a given iommu memory region. So now
>   we have notifiers for IOTLB changes and config changes.
> - vSMMUv3 calls config notifiers when STE (Stream Table Entries)
>   are updated by the guest.
> - We implement a specific UNMAP notifier that conveys guest
>   IOTLB invalidations to the host
> - We implement a new MAP notifiers only used for MSI IOVAs so
>   that the host can build a nested stage translation for MSI IOVAs
> - As the legacy MAP notifier is not called anymore, we must make
>   sure stage 2 mappings are set. This is achieved through another
>   memory listener.
> - Physical SMMUs faults are reported to the guest via en eventfd
>   mechanism and reinjected into this latter.
>
> Note: The first patch is a code cleanup and was sent separately.
>
> Best Regards
>
> Eric
>
> This series can be found at:
> https://github.com/eauger/qemu/tree/v4.0.0-2stage-rfcv4
>
> Compatible with kernel series:
> [PATCH v8 00/29] SMMUv3 Nested Stage Setup
> (https://lkml.org/lkml/2019/5/26/95)
>

Have tested vfio mode in qemu on arm64 platform.

Tested-by: Zhangfei Gao 
qemu: https://github.com/eauger/qemu/tree/v4.0.0-2stage-rfcv4
kernel: https://github.com/eauger/linux/tree/v5.2-rc1-2stage-v8

Re: [Qemu-devel] [PATCH v4 17/18] iotests: add test 257 for bitmap-mode backups

[John Snow]

On 7/9/19 7:25 PM, John Snow wrote:
> Signed-off-by: John Snow 
> ---
>  tests/qemu-iotests/257 |  416 +++
>  tests/qemu-iotests/257.out | 2247 
>  tests/qemu-iotests/group   |1 +
>  3 files changed, 2664 insertions(+)
>  create mode 100755 tests/qemu-iotests/257
>  create mode 100644 tests/qemu-iotests/257.out
> 
> diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
> new file mode 100755
> index 00..75a651c7c3
> --- /dev/null
> +++ b/tests/qemu-iotests/257
> @@ -0,0 +1,416 @@
> +#!/usr/bin/env python
> +#
> +# Test bitmap-sync backups (incremental, differential, and partials)
> +#
> +# Copyright (c) 2019 John Snow for Red Hat, Inc.
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; either version 2 of the License, or
> +# (at your option) any later version.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program.  If not, see .
> +#
> +# owner=js...@redhat.com
> +
> +from collections import namedtuple
> +import math
> +import os
> +
> +import iotests
> +from iotests import log, qemu_img
> +
> +SIZE = 64 * 1024 * 1024
> +GRANULARITY = 64 * 1024
> +
> +Pattern = namedtuple('Pattern', ['byte', 'offset', 'size'])
> +def mkpattern(byte, offset, size=GRANULARITY):
> +"""Constructor for Pattern() with default size"""
> +return Pattern(byte, offset, size)
> +
> +class PatternGroup:
> +"""Grouping of Pattern objects. Initialize with an iterable of 
> Patterns."""
> +def __init__(self, patterns):
> +self.patterns = patterns
> +
> +def bits(self, granularity):
> +"""Calculate the unique bits dirtied by this pattern grouping"""
> +res = set()
> +for pattern in self.patterns:
> +lower = math.floor(pattern.offset / granularity)
> +upper = math.floor((pattern.offset + pattern.size - 1) / 
> granularity)
> +res = res | set(range(lower, upper + 1))
> +return res
> +

Squashing in:

-lower = math.floor(pattern.offset / granularity)
-upper = math.floor((pattern.offset + pattern.size - 1) /
granularity)
+lower = pattern.offset // granularity
+upper = (pattern.offset + pattern.size - 1) // granularity

To quiet the Python2 beast.


--js

Re: [Qemu-devel] [RFC v1 06/18] intel_iommu: support virtual command emulation and pasid request

[Peter Xu]

On Wed, Jul 10, 2019 at 11:51:17AM +, Liu, Yi L wrote:

[...]

> > > +s->vcrsp = 1;
> > > +vtd_set_quad_raw(s, DMAR_VCRSP_REG,
> > > + ((uint64_t) s->vcrsp));
> > 
> > Do we really need to emulate the "In Progress" like this?  The vcpu is
> > blocked here after all, and AFAICT all the rest of vcpus should not
> > access these registers because obviously these registers cannot be
> > accessed concurrently...
> 
> Other vcpus should poll the IP bit before submitting vcmds. As IP bit
> is set, other vcpus will not access these bits. but if not, they may submit
> new vcmds, while we only have 1 response register, that is not we
> support. That's why we need to set IP bit.

I still don't think another CPU can use this register even if it
polled with IP==0...  The reason is simply as you described - we only
have one pair of VCMD/VRSPD registers so IMHO the guest IOMMU driver
must have a lock (probably a mutex) to guarantee sequential access of
these registers otherwise race can happen.

> 
> > 
> > I think the IP bit is useful when some new vcmd would take plenty of
> > time so that we can do the long vcmds in async way.  However here it
> > seems not the case?
> 
> no, so far, it is synchronize way. As mentioned above, IP bit is to ensure
> only one vcmd is handled for a time. Other vcpus won't be able to submit
> vcmds before IP is cleared.

[...]

> > > @@ -192,6 +198,7 @@
> > >  #define VTD_ECAP_SRS(1ULL << 31)
> > >  #define VTD_ECAP_PASID  (1ULL << 40)
> > >  #define VTD_ECAP_SMTS   (1ULL << 43)
> > > +#define VTD_ECAP_VCS(1ULL << 44)
> > >  #define VTD_ECAP_SLTS   (1ULL << 46)
> > >  #define VTD_ECAP_FLTS   (1ULL << 47)
> > >
> > > @@ -314,6 +321,29 @@ typedef enum VTDFaultReason {
> > >
> > >  #define VTD_CONTEXT_CACHE_GEN_MAX   0xUL
> > >
> > > +/* VCCAP_REG */
> > > +#define VTD_VCCAP_PAS   (1UL << 0)
> > > +#define VTD_MIN_HPASID  200
> > 
> > Comment this value a bit?
> 
> The basic idea is to let hypervisor to set a range for available PASIDs for
> VMs. One of the reasons is PASID #0 is reserved by RID_PASID usage.
> We have no idea how many reserved PASIDs in future, so here just a
> evaluated value. Honestly, set it as "1" is enough at current stage.

That'll be a very nice initial comment for that (I mean, put it into
the patch, of course :).

Regards,

-- 
Peter Xu

Re: [Qemu-devel] Problems building and installing qemu v4.1.0-rc1 in single step

[Guenter Roeck]

On Wed, Jul 10, 2019 at 03:01:53PM -0700, Guenter Roeck wrote:
> Hi,
> 
> when trying to run "make -j30 install" from a clean tree on v4.1.0-rc0, I get
> _lots_ of undefined symbol errors.
> 
> If I run "make -j30" followed by "make -j30 install", make succeeds, but then
> I get linker errors such as the following when running "make -j30 install".
> 
> /usr/bin/ld: final link failed: File truncated
> /usr/bin/ld: BFD (GNU Binutils for Ubuntu) 2.26.1 internal error,
>   aborting at ../../bfd/merge.c:905 in _bfd_merged_section_offset
> 
> Running "make -j30" followed by "make install" succeeds.
> 

Correction: This doesn't always work either. Sometimes I still get a linker
error. If that happens, another round of "make; make install" succeeds.

Guenter

> This looks like "make install" may have bad dependencies. Has anyone else
> experienced this problem ?
> 
> Thanks,
> Guenter

Re: [Qemu-devel] [PATCH v2 2/2] hw/riscv: Load OpenSBI as the default firmware

[Guenter Roeck]

On Wed, Jul 10, 2019 at 05:14:07PM -0700, Alistair Francis wrote:
> If the user hasn't specified a firmware to load (with -bios) or
> specified no bios (with -bios none) then load OpenSBI by default. This
> allows users to boot a RISC-V kernel with just -kernel.
> 
> Signed-off-by: Alistair Francis 
> Reviewed-by: Bin Meng 
> Tested-by: Bin Meng 

Reviewed-by: Guenter Roeck 
Tested-by: Guenter Roeck 

> ---
>  hw/riscv/boot.c | 54 +
>  hw/riscv/sifive_u.c |  7 +++---
>  hw/riscv/virt.c | 11 ++---
>  include/hw/riscv/boot.h |  3 +++
>  qemu-deprecated.texi| 20 +++
>  5 files changed, 89 insertions(+), 6 deletions(-)
> 
> diff --git a/hw/riscv/boot.c b/hw/riscv/boot.c
> index ff023f42d0..5dee63011b 100644
> --- a/hw/riscv/boot.c
> +++ b/hw/riscv/boot.c
> @@ -18,6 +18,7 @@
>   */
>  
>  #include "qemu/osdep.h"
> +#include "qemu-common.h"
>  #include "qemu/units.h"
>  #include "qemu/error-report.h"
>  #include "exec/cpu-defs.h"
> @@ -32,6 +33,59 @@
>  # define KERNEL_BOOT_ADDRESS 0x8020
>  #endif
>  
> +void riscv_find_and_load_firmware(MachineState *machine,
> +  const char *default_machine_firmware,
> +  hwaddr firmware_load_addr)
> +{
> +char *firmware_filename;
> +
> +if (!machine->firmware) {
> +/*
> + * The user didn't specify -bios.
> + * At the moment we default to loading nothing when this hapens.
> + * In the future this defaul will change to loading the prebuilt
> + * OpenSBI firmware. Let's warn the user and then continue.
> +*/
> +warn_report("No -bios option specified. Not loading a firmware.");
> +warn_report("This default will change in QEMU 4.3. Please use the " \
> +"-bios option to aviod breakages when this happens.");
> +warn_report("See QEMU's deprecation documentation for details");
> +return;
> +}
> +
> +if (!strcmp(machine->firmware, "default")) {
> +/*
> + * The user has specified "-bios default". That means we are going to
> + * load the OpenSBI binary included in the QEMU source.
> + *
> + * We can't load the binary by default as it will break existing 
> users
> + * as users are already loading their own firmware.
> + *
> + * Let's try to get everyone to specify the -bios option at all 
> times,
> + * so then in the future we can make "-bios default" the default 
> option
> + * if no -bios option is set without breaking anything.
> + */
> +firmware_filename = qemu_find_file(QEMU_FILE_TYPE_BIOS,
> +   default_machine_firmware);
> +if (firmware_filename == NULL) {
> +error_report("Unable to load the default RISC-V firmware \"%s\"",
> + default_machine_firmware);
> +exit(1);
> +}
> +} else {
> +firmware_filename = machine->firmware;
> +}
> +
> +if (strcmp(firmware_filename, "none")) {
> +/* If not "none" load the firmware */
> +riscv_load_firmware(firmware_filename, firmware_load_addr);
> +}
> +
> +if (!strcmp(machine->firmware, "default")) {
> +g_free(firmware_filename);
> +}
> +}
> +
>  target_ulong riscv_load_firmware(const char *firmware_filename,
>   hwaddr firmware_load_addr)
>  {
> diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c
> index ca53a9290d..71b8083c05 100644
> --- a/hw/riscv/sifive_u.c
> +++ b/hw/riscv/sifive_u.c
> @@ -49,6 +49,8 @@
>  
>  #include 
>  
> +#define BIOS_FILENAME "opensbi-riscv64-sifive_u-fw_jump.bin"
> +
>  static const struct MemmapEntry {
>  hwaddr base;
>  hwaddr size;
> @@ -269,9 +271,8 @@ static void riscv_sifive_u_init(MachineState *machine)
>  /* create device tree */
>  create_fdt(s, memmap, machine->ram_size, machine->kernel_cmdline);
>  
> -if (machine->firmware) {
> -riscv_load_firmware(machine->firmware, memmap[SIFIVE_U_DRAM].base);
> -}
> +riscv_find_and_load_firmware(machine, BIOS_FILENAME,
> + memmap[SIFIVE_U_DRAM].base);
>  
>  if (machine->kernel_filename) {
>  riscv_load_kernel(machine->kernel_filename);
> diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
> index ecdc77d728..25faf3b417 100644
> --- a/hw/riscv/virt.c
> +++ b/hw/riscv/virt.c
> @@ -44,6 +44,12 @@
>  
>  #include 
>  
> +#if defined(TARGET_RISCV32)
> +# define BIOS_FILENAME "opensbi-riscv32-virt-fw_jump.bin"
> +#else
> +# define BIOS_FILENAME "opensbi-riscv64-virt-fw_jump.bin"
> +#endif
> +
>  static const struct MemmapEntry {
>  hwaddr base;
>  hwaddr size;
> @@ -399,9 +405,8 @@ static void riscv_virt_board_init(MachineState *machine)
>  memory_region_add_subregion(system_memory, memmap[VIRT_MROM].base,
>  

Re: [Qemu-devel] [PATCH v2 1/2] roms: Add OpenSBI version 0.4

[Guenter Roeck]

On Wed, Jul 10, 2019 at 05:14:04PM -0700, Alistair Francis wrote:
> Add OpenSBI version 0.4 as a git submodule and as a prebult binary.
> 
> OpenSBI (https://github.com/riscv/opensbi) aims to provide an open-source
> reference implementation of the RISC-V Supervisor Binary Interface (SBI)
> specifications for platform-specific firmwares executing in M-mode. For all
> supported platforms, OpenSBI provides several runtime firmware examples.
> These example firmwares can be used to replace the legacy riscv-pk bootloader
> and enable the use of well-known bootloaders such as U-Boot.
> 
> OpenSBI is distributed under the terms of the BSD 2-clause license
> ("Simplified BSD License" or "FreeBSD License", SPDX: BSD-2-Clause). OpenSBI
> source code also contains code reused from other projects desribed here:
> https://github.com/riscv/opensbi/blob/master/ThirdPartyNotices.md.
> 
> In this case all of the code we are using from OpenSBI is BSD 2-clause
> as we aren't using the Kendryte code (Apache-2.0) with QEMU and libfdt
> is dual licensed as BSD 2-clause (and GPL-2.0+). OpenSBI isn't being
> linked with QEMU either it is just being included with QEMU.
> 
> Signed-off-by: Alistair Francis 
> Reviewed-by: Bin Meng 

Tested-by: Guenter Roeck 

> ---
>  .gitmodules  |   3 ++
>  LICENSE  |  21 
>  Makefile |   5 +-
>  pc-bios/README   |  11 +
>  pc-bios/opensbi-riscv32-virt-fw_jump.bin | Bin 0 -> 36888 bytes
>  pc-bios/opensbi-riscv64-sifive_u-fw_jump.bin | Bin 0 -> 40968 bytes
>  pc-bios/opensbi-riscv64-virt-fw_jump.bin | Bin 0 -> 40968 bytes
>  roms/Makefile|  48 ++-
>  roms/opensbi |   1 +
>  9 files changed, 68 insertions(+), 21 deletions(-)
>  create mode 100755 pc-bios/opensbi-riscv32-virt-fw_jump.bin
>  create mode 100755 pc-bios/opensbi-riscv64-sifive_u-fw_jump.bin
>  create mode 100755 pc-bios/opensbi-riscv64-virt-fw_jump.bin
>  create mode 16 roms/opensbi
> 
> diff --git a/.gitmodules b/.gitmodules
> index 2857eec763..7a10e72e09 100644
> --- a/.gitmodules
> +++ b/.gitmodules
> @@ -55,3 +55,6 @@
>  [submodule "slirp"]
>   path = slirp
>   url = https://git.qemu.org/git/libslirp.git
> +[submodule "roms/opensbi"]
> + path = roms/opensbi
> + url = https://github.com/riscv/opensbi.git
> diff --git a/LICENSE b/LICENSE
> index 0e0b4b9553..9389ba614f 100644
> --- a/LICENSE
> +++ b/LICENSE
> @@ -1,13 +1,18 @@
> -The following points clarify the QEMU license:
> +The QEMU distribution includes both the QEMU emulator and
> +various firmware files.  These are separate programs that are
> +distributed together for our users' convenience, and they have
> +separate licenses.
>  
> -1) QEMU as a whole is released under the GNU General Public License,
> -version 2.
> +The following points clarify the license of the QEMU emulator:
>  
> -2) Parts of QEMU have specific licenses which are compatible with the
> -GNU General Public License, version 2. Hence each source file contains
> -its own licensing information.  Source files with no licensing information
> -are released under the GNU General Public License, version 2 or (at your
> -option) any later version.
> +1) The QEMU emulator as a whole is released under the GNU General
> +Public License, version 2.
> +
> +2) Parts of the QEMU emulator have specific licenses which are compatible
> +with the GNU General Public License, version 2. Hence each source file
> +contains its own licensing information.  Source files with no licensing
> +information are released under the GNU General Public License, version
> +2 or (at your option) any later version.
>  
>  As of July 2013, contributions under version 2 of the GNU General Public
>  License (and no later version) are only accepted for the following files
> diff --git a/Makefile b/Makefile
> index 1fcbaed62c..b44e6ae5ad 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -769,7 +769,10 @@ palcode-clipper \
>  u-boot.e500 u-boot-sam460-20100605.bin \
>  qemu_vga.ndrv \
>  edk2-licenses.txt \
> -hppa-firmware.img
> +hppa-firmware.img \
> +opensbi-riscv32-virt-fw_jump.bin \
> +opensbi-riscv64-sifive_u-fw_jump.bin opensbi-riscv64-virt-fw_jump.bin
> +
>  
>  DESCS=50-edk2-i386-secure.json 50-edk2-x86_64-secure.json \
>  60-edk2-aarch64.json 60-edk2-arm.json 60-edk2-i386.json 60-edk2-x86_64.json
> diff --git a/pc-bios/README b/pc-bios/README
> index 180795a55b..eacb97043d 100644
> --- a/pc-bios/README
> +++ b/pc-bios/README
> @@ -63,3 +63,14 @@
>ARM. Licensing information is given in "edk2-licenses.txt". The image files
>are described by the JSON documents in the "pc-bios/descriptors" directory,
>which conform to the "docs/interop/firmware.json" schema.
> +
> +- OpenSBI (https://github.com/riscv/opensbi) aims to provide an open-source
> +  reference implementation of the RISC-V 

Re: [Qemu-devel] [RFC v1 04/18] intel_iommu: add "sm_model" option

[Peter Xu]

On Wed, Jul 10, 2019 at 12:14:44PM +, Liu, Yi L wrote:
> > From: Peter Xu [mailto:zh...@redhat.com]
> > Sent: Tuesday, July 9, 2019 10:16 AM
> > To: Liu, Yi L 
> > Subject: Re: [RFC v1 04/18] intel_iommu: add "sm_model" option
> > 
> > On Fri, Jul 05, 2019 at 07:01:37PM +0800, Liu Yi L wrote:
> > > Intel VT-d 3.0 introduces scalable mode, and it has a bunch of
> > > capabilities related to scalable mode translation, thus there
> > > are multiple combinations. While this vIOMMU implementation
> > > wants simplify it for user by providing typical combinations.
> > > User could config it by "sm_model" option. The usage is as
> > > below:
> > >
> > > "-device intel-iommu,x-scalable-mode=on,sm_model=["legacy"|"scalable"]"
> > 
> > Is it a requirement to split into two parameters, instead of just
> > exposing everything about scalable mode when x-scalable-mode is set?
> 
> yes, it is. Scalable mode has multiple capabilities. And we want to support
> the most typical combinations to simplify software. e.g. current scalable mode
> vIOMMU exposes only 2nd level translation to guest, and guest IOVA support
> is via shadowing guest 2nd level page table. We have plan to move IOVA from
> 2nd level page table to 1st level page table, thus guest IOVA can be supported
> with nested translation. And this also addresses the co-existence issue of 
> guest
> SVA and guest IOVA. So in future we will have scalable mode vIOMMU expose
> 1st level translation only. To differentiate this config with current vIOMMU,
> we need an extra option to control it. But yes, it is still scalable mode 
> vIOMMU.
> just has different capability exposed to guest.

I see.  Thanks for explaining.

> 
> BTW. do you know if I can add sub-options under "x-scalable-mode"? I think
> that may demonstrate the dependency better.

I'm not an expert of that, but I think at least we can make it a
string parameter depends on what you prefer, then we can do
"x-scalable-mode=legacy|modern".  Or keep this would be fine too.

> 
> > >
> > >  - "legacy": gives support for SL page table
> > >  - "scalable": gives support for FL page table, pasid, virtual command
> > >  - default to be "legacy" if "x-scalable-mode=on while no sm_model is
> > >configured
> > >
> > > Cc: Kevin Tian 
> > > Cc: Jacob Pan 
> > > Cc: Peter Xu 
> > > Cc: Yi Sun 
> > > Signed-off-by: Liu Yi L 
> > > Signed-off-by: Yi Sun 
> > > ---
> > >  hw/i386/intel_iommu.c  | 28 +++-
> > >  hw/i386/intel_iommu_internal.h |  2 ++
> > >  include/hw/i386/intel_iommu.h  |  1 +
> > >  3 files changed, 30 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
> > > index 44b1231..3160a05 100644
> > > --- a/hw/i386/intel_iommu.c
> > > +++ b/hw/i386/intel_iommu.c
> > > @@ -3014,6 +3014,7 @@ static Property vtd_properties[] = {
> > >  DEFINE_PROP_BOOL("caching-mode", IntelIOMMUState, caching_mode,
> > FALSE),
> > >  DEFINE_PROP_BOOL("x-scalable-mode", IntelIOMMUState, scalable_mode,
> > FALSE),
> > >  DEFINE_PROP_BOOL("dma-drain", IntelIOMMUState, dma_drain, true),
> > > +DEFINE_PROP_STRING("sm_model", IntelIOMMUState, sm_model),
> > 
> > Can do 's/-/_/' to follow the rest if we need it.
> 
> Do you mean sub-options after "x-scalable-mode"?

No, I only mean "sm-model". :)

Regards,

-- 
Peter Xu

Re: [Qemu-devel] [PATCH v2 1/2] roms: Add OpenSBI version 0.4

[Guenter Roeck]

On Wed, Jul 10, 2019 at 05:14:04PM -0700, Alistair Francis wrote:
> Add OpenSBI version 0.4 as a git submodule and as a prebult binary.
> 
> OpenSBI (https://github.com/riscv/opensbi) aims to provide an open-source
> reference implementation of the RISC-V Supervisor Binary Interface (SBI)
> specifications for platform-specific firmwares executing in M-mode. For all
> supported platforms, OpenSBI provides several runtime firmware examples.
> These example firmwares can be used to replace the legacy riscv-pk bootloader
> and enable the use of well-known bootloaders such as U-Boot.
> 
> OpenSBI is distributed under the terms of the BSD 2-clause license
> ("Simplified BSD License" or "FreeBSD License", SPDX: BSD-2-Clause). OpenSBI
> source code also contains code reused from other projects desribed here:
> https://github.com/riscv/opensbi/blob/master/ThirdPartyNotices.md.
> 
> In this case all of the code we are using from OpenSBI is BSD 2-clause
> as we aren't using the Kendryte code (Apache-2.0) with QEMU and libfdt
> is dual licensed as BSD 2-clause (and GPL-2.0+). OpenSBI isn't being
> linked with QEMU either it is just being included with QEMU.
> 
> Signed-off-by: Alistair Francis 
> Reviewed-by: Bin Meng 

Reviewed-by: Guenter Roeck 

> ---
>  .gitmodules  |   3 ++
>  LICENSE  |  21 
>  Makefile |   5 +-
>  pc-bios/README   |  11 +
>  pc-bios/opensbi-riscv32-virt-fw_jump.bin | Bin 0 -> 36888 bytes
>  pc-bios/opensbi-riscv64-sifive_u-fw_jump.bin | Bin 0 -> 40968 bytes
>  pc-bios/opensbi-riscv64-virt-fw_jump.bin | Bin 0 -> 40968 bytes
>  roms/Makefile|  48 ++-
>  roms/opensbi |   1 +
>  9 files changed, 68 insertions(+), 21 deletions(-)
>  create mode 100755 pc-bios/opensbi-riscv32-virt-fw_jump.bin
>  create mode 100755 pc-bios/opensbi-riscv64-sifive_u-fw_jump.bin
>  create mode 100755 pc-bios/opensbi-riscv64-virt-fw_jump.bin
>  create mode 16 roms/opensbi
> 
> diff --git a/.gitmodules b/.gitmodules
> index 2857eec763..7a10e72e09 100644
> --- a/.gitmodules
> +++ b/.gitmodules
> @@ -55,3 +55,6 @@
>  [submodule "slirp"]
>   path = slirp
>   url = https://git.qemu.org/git/libslirp.git
> +[submodule "roms/opensbi"]
> + path = roms/opensbi
> + url = https://github.com/riscv/opensbi.git
> diff --git a/LICENSE b/LICENSE
> index 0e0b4b9553..9389ba614f 100644
> --- a/LICENSE
> +++ b/LICENSE
> @@ -1,13 +1,18 @@
> -The following points clarify the QEMU license:
> +The QEMU distribution includes both the QEMU emulator and
> +various firmware files.  These are separate programs that are
> +distributed together for our users' convenience, and they have
> +separate licenses.
>  
> -1) QEMU as a whole is released under the GNU General Public License,
> -version 2.
> +The following points clarify the license of the QEMU emulator:
>  
> -2) Parts of QEMU have specific licenses which are compatible with the
> -GNU General Public License, version 2. Hence each source file contains
> -its own licensing information.  Source files with no licensing information
> -are released under the GNU General Public License, version 2 or (at your
> -option) any later version.
> +1) The QEMU emulator as a whole is released under the GNU General
> +Public License, version 2.
> +
> +2) Parts of the QEMU emulator have specific licenses which are compatible
> +with the GNU General Public License, version 2. Hence each source file
> +contains its own licensing information.  Source files with no licensing
> +information are released under the GNU General Public License, version
> +2 or (at your option) any later version.
>  
>  As of July 2013, contributions under version 2 of the GNU General Public
>  License (and no later version) are only accepted for the following files
> diff --git a/Makefile b/Makefile
> index 1fcbaed62c..b44e6ae5ad 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -769,7 +769,10 @@ palcode-clipper \
>  u-boot.e500 u-boot-sam460-20100605.bin \
>  qemu_vga.ndrv \
>  edk2-licenses.txt \
> -hppa-firmware.img
> +hppa-firmware.img \
> +opensbi-riscv32-virt-fw_jump.bin \
> +opensbi-riscv64-sifive_u-fw_jump.bin opensbi-riscv64-virt-fw_jump.bin
> +
>  
>  DESCS=50-edk2-i386-secure.json 50-edk2-x86_64-secure.json \
>  60-edk2-aarch64.json 60-edk2-arm.json 60-edk2-i386.json 60-edk2-x86_64.json
> diff --git a/pc-bios/README b/pc-bios/README
> index 180795a55b..eacb97043d 100644
> --- a/pc-bios/README
> +++ b/pc-bios/README
> @@ -63,3 +63,14 @@
>ARM. Licensing information is given in "edk2-licenses.txt". The image files
>are described by the JSON documents in the "pc-bios/descriptors" directory,
>which conform to the "docs/interop/firmware.json" schema.
> +
> +- OpenSBI (https://github.com/riscv/opensbi) aims to provide an open-source
> +  reference implementation of the RISC-V 

Re: [Qemu-devel] [PATCH 0/4] virtio: handle zoned backing devices

[Dmitry Fomichev]

On Wed, 2019-07-10 at 23:09 +0200, Kevin Wolf wrote:
> Am 10.07.2019 um 13:33 hat Paolo Bonzini geschrieben:
> > On 10/07/19 13:02, Kevin Wolf wrote:
> > > Hm... Actually, file-posix implements .bdrv_check_perm and could just
> > > refuse attaching a parent there if it doesn't request a specific
> > > permission like BLK_PERM_SUPPORT_ZONED. That should give us the
> > > whitelist semantics through existing infrastructure.
> > 
> > I'd like Dmitry to have something more precise to base his work on.  The
> > permissions system is really complicated and I never really wrapped my
> > head around it, so I need your help.
> > 
> > IIUC, blkconf_apply_backend_options would grow a new argument (like
> > "resizable") and that argument would add BLK_PERM_SUPPORT_ZONED to the
> > perm that blkconf_apply_backend_options passes to blk_set_perm.  On the
> > other side raw_check_perm would say something like
> > 
> > if (is_zoned(s) && !(perm & BLK_PERM_SUPPORT_ZONED)) {
> > error_setg();
> > return -ENOTSUP;
> > }
> > 
> > Is this correct?
> 
> Yes, I think this is how you'd best implement it.
> 
> > In addition, BLK_PERM_SUPPORT_ZONED would have to be a shared
> > permission, since it's possible to assign the same block device to
> > multiple scsi-block devices.  So BLK_PERM_SUPPORT_ZONED would be added
> > unconditionally to shared_perm.
> 
> Right, this part shows that we're kind of abusing the permission system
> here. I think unconditionally adding BLK_PERM_SUPPORT_ZONED to the set
> of shared permissions could probably happen centrally in
> bdrv_child_perm().
> 
> > ps: I have always thought that shared_perm is expressed the wrong way
> > and should have been "denied_perm".  How hard would it be to change that
> > now?
> 
> I'm not sure it would be better. It is shared_perm because that means
> that the default is restrictive (error mode: operation refused, clearly
> reported, easy to fix) rather than permissive (error mode: image
> corruption, hard to figure out where we were too permissive). Basically,
> whitelist instead of blacklist, once again.
> 
> But if we did indeed decide to change it, the only way to find out how
> hard it is would be doing it. I suspect not too hard in principle, but
> making sure that we converted all callers and don't introduce wrong
> callers later through (silent) merge conflicts is the more concerning
> part.
> 
> Kevin

Thanks for the feedback, the permissions based approach indeed looks
cleaner. I am looking into modifying the patchset to do the check in
bdrv_check_perm and will send a V2.

Paolo,
WRT to Host Aware drives, these MAY work, but we don't have any of these
available for testing and are not able to verify which drivers do work
with them and which do not. This is the reason for not letting them pass
thru. If you prefer, I can enable passing HA drives in V2.

Dmitry

[Qemu-devel] [PATCH v2 1/2] roms: Add OpenSBI version 0.4

[Alistair Francis]

Add OpenSBI version 0.4 as a git submodule and as a prebult binary.

OpenSBI (https://github.com/riscv/opensbi) aims to provide an open-source
reference implementation of the RISC-V Supervisor Binary Interface (SBI)
specifications for platform-specific firmwares executing in M-mode. For all
supported platforms, OpenSBI provides several runtime firmware examples.
These example firmwares can be used to replace the legacy riscv-pk bootloader
and enable the use of well-known bootloaders such as U-Boot.

OpenSBI is distributed under the terms of the BSD 2-clause license
("Simplified BSD License" or "FreeBSD License", SPDX: BSD-2-Clause). OpenSBI
source code also contains code reused from other projects desribed here:
https://github.com/riscv/opensbi/blob/master/ThirdPartyNotices.md.

In this case all of the code we are using from OpenSBI is BSD 2-clause
as we aren't using the Kendryte code (Apache-2.0) with QEMU and libfdt
is dual licensed as BSD 2-clause (and GPL-2.0+). OpenSBI isn't being
linked with QEMU either it is just being included with QEMU.

Signed-off-by: Alistair Francis 
Reviewed-by: Bin Meng 
---
 .gitmodules  |   3 ++
 LICENSE  |  21 
 Makefile |   5 +-
 pc-bios/README   |  11 +
 pc-bios/opensbi-riscv32-virt-fw_jump.bin | Bin 0 -> 36888 bytes
 pc-bios/opensbi-riscv64-sifive_u-fw_jump.bin | Bin 0 -> 40968 bytes
 pc-bios/opensbi-riscv64-virt-fw_jump.bin | Bin 0 -> 40968 bytes
 roms/Makefile|  48 ++-
 roms/opensbi |   1 +
 9 files changed, 68 insertions(+), 21 deletions(-)
 create mode 100755 pc-bios/opensbi-riscv32-virt-fw_jump.bin
 create mode 100755 pc-bios/opensbi-riscv64-sifive_u-fw_jump.bin
 create mode 100755 pc-bios/opensbi-riscv64-virt-fw_jump.bin
 create mode 16 roms/opensbi

diff --git a/.gitmodules b/.gitmodules
index 2857eec763..7a10e72e09 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -55,3 +55,6 @@
 [submodule "slirp"]
path = slirp
url = https://git.qemu.org/git/libslirp.git
+[submodule "roms/opensbi"]
+   path = roms/opensbi
+   url = https://github.com/riscv/opensbi.git
diff --git a/LICENSE b/LICENSE
index 0e0b4b9553..9389ba614f 100644
--- a/LICENSE
+++ b/LICENSE
@@ -1,13 +1,18 @@
-The following points clarify the QEMU license:
+The QEMU distribution includes both the QEMU emulator and
+various firmware files.  These are separate programs that are
+distributed together for our users' convenience, and they have
+separate licenses.
 
-1) QEMU as a whole is released under the GNU General Public License,
-version 2.
+The following points clarify the license of the QEMU emulator:
 
-2) Parts of QEMU have specific licenses which are compatible with the
-GNU General Public License, version 2. Hence each source file contains
-its own licensing information.  Source files with no licensing information
-are released under the GNU General Public License, version 2 or (at your
-option) any later version.
+1) The QEMU emulator as a whole is released under the GNU General
+Public License, version 2.
+
+2) Parts of the QEMU emulator have specific licenses which are compatible
+with the GNU General Public License, version 2. Hence each source file
+contains its own licensing information.  Source files with no licensing
+information are released under the GNU General Public License, version
+2 or (at your option) any later version.
 
 As of July 2013, contributions under version 2 of the GNU General Public
 License (and no later version) are only accepted for the following files
diff --git a/Makefile b/Makefile
index 1fcbaed62c..b44e6ae5ad 100644
--- a/Makefile
+++ b/Makefile
@@ -769,7 +769,10 @@ palcode-clipper \
 u-boot.e500 u-boot-sam460-20100605.bin \
 qemu_vga.ndrv \
 edk2-licenses.txt \
-hppa-firmware.img
+hppa-firmware.img \
+opensbi-riscv32-virt-fw_jump.bin \
+opensbi-riscv64-sifive_u-fw_jump.bin opensbi-riscv64-virt-fw_jump.bin
+
 
 DESCS=50-edk2-i386-secure.json 50-edk2-x86_64-secure.json \
 60-edk2-aarch64.json 60-edk2-arm.json 60-edk2-i386.json 60-edk2-x86_64.json
diff --git a/pc-bios/README b/pc-bios/README
index 180795a55b..eacb97043d 100644
--- a/pc-bios/README
+++ b/pc-bios/README
@@ -63,3 +63,14 @@
   ARM. Licensing information is given in "edk2-licenses.txt". The image files
   are described by the JSON documents in the "pc-bios/descriptors" directory,
   which conform to the "docs/interop/firmware.json" schema.
+
+- OpenSBI (https://github.com/riscv/opensbi) aims to provide an open-source
+  reference implementation of the RISC-V Supervisor Binary Interface (SBI)
+  specifications for platform-specific firmwares executing in M-mode. For all
+  supported platforms, OpenSBI provides several runtime firmware examples.
+  These example firmwares can be used to replace the legacy riscv-pk bootloader
+  and enable the use of 

[Qemu-devel] [PATCH v2 2/2] hw/riscv: Load OpenSBI as the default firmware

[Alistair Francis]

If the user hasn't specified a firmware to load (with -bios) or
specified no bios (with -bios none) then load OpenSBI by default. This
allows users to boot a RISC-V kernel with just -kernel.

Signed-off-by: Alistair Francis 
Reviewed-by: Bin Meng 
Tested-by: Bin Meng 
---
 hw/riscv/boot.c | 54 +
 hw/riscv/sifive_u.c |  7 +++---
 hw/riscv/virt.c | 11 ++---
 include/hw/riscv/boot.h |  3 +++
 qemu-deprecated.texi| 20 +++
 5 files changed, 89 insertions(+), 6 deletions(-)

diff --git a/hw/riscv/boot.c b/hw/riscv/boot.c
index ff023f42d0..5dee63011b 100644
--- a/hw/riscv/boot.c
+++ b/hw/riscv/boot.c
@@ -18,6 +18,7 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu-common.h"
 #include "qemu/units.h"
 #include "qemu/error-report.h"
 #include "exec/cpu-defs.h"
@@ -32,6 +33,59 @@
 # define KERNEL_BOOT_ADDRESS 0x8020
 #endif
 
+void riscv_find_and_load_firmware(MachineState *machine,
+  const char *default_machine_firmware,
+  hwaddr firmware_load_addr)
+{
+char *firmware_filename;
+
+if (!machine->firmware) {
+/*
+ * The user didn't specify -bios.
+ * At the moment we default to loading nothing when this hapens.
+ * In the future this defaul will change to loading the prebuilt
+ * OpenSBI firmware. Let's warn the user and then continue.
+*/
+warn_report("No -bios option specified. Not loading a firmware.");
+warn_report("This default will change in QEMU 4.3. Please use the " \
+"-bios option to aviod breakages when this happens.");
+warn_report("See QEMU's deprecation documentation for details");
+return;
+}
+
+if (!strcmp(machine->firmware, "default")) {
+/*
+ * The user has specified "-bios default". That means we are going to
+ * load the OpenSBI binary included in the QEMU source.
+ *
+ * We can't load the binary by default as it will break existing users
+ * as users are already loading their own firmware.
+ *
+ * Let's try to get everyone to specify the -bios option at all times,
+ * so then in the future we can make "-bios default" the default option
+ * if no -bios option is set without breaking anything.
+ */
+firmware_filename = qemu_find_file(QEMU_FILE_TYPE_BIOS,
+   default_machine_firmware);
+if (firmware_filename == NULL) {
+error_report("Unable to load the default RISC-V firmware \"%s\"",
+ default_machine_firmware);
+exit(1);
+}
+} else {
+firmware_filename = machine->firmware;
+}
+
+if (strcmp(firmware_filename, "none")) {
+/* If not "none" load the firmware */
+riscv_load_firmware(firmware_filename, firmware_load_addr);
+}
+
+if (!strcmp(machine->firmware, "default")) {
+g_free(firmware_filename);
+}
+}
+
 target_ulong riscv_load_firmware(const char *firmware_filename,
  hwaddr firmware_load_addr)
 {
diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c
index ca53a9290d..71b8083c05 100644
--- a/hw/riscv/sifive_u.c
+++ b/hw/riscv/sifive_u.c
@@ -49,6 +49,8 @@
 
 #include 
 
+#define BIOS_FILENAME "opensbi-riscv64-sifive_u-fw_jump.bin"
+
 static const struct MemmapEntry {
 hwaddr base;
 hwaddr size;
@@ -269,9 +271,8 @@ static void riscv_sifive_u_init(MachineState *machine)
 /* create device tree */
 create_fdt(s, memmap, machine->ram_size, machine->kernel_cmdline);
 
-if (machine->firmware) {
-riscv_load_firmware(machine->firmware, memmap[SIFIVE_U_DRAM].base);
-}
+riscv_find_and_load_firmware(machine, BIOS_FILENAME,
+ memmap[SIFIVE_U_DRAM].base);
 
 if (machine->kernel_filename) {
 riscv_load_kernel(machine->kernel_filename);
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index ecdc77d728..25faf3b417 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -44,6 +44,12 @@
 
 #include 
 
+#if defined(TARGET_RISCV32)
+# define BIOS_FILENAME "opensbi-riscv32-virt-fw_jump.bin"
+#else
+# define BIOS_FILENAME "opensbi-riscv64-virt-fw_jump.bin"
+#endif
+
 static const struct MemmapEntry {
 hwaddr base;
 hwaddr size;
@@ -399,9 +405,8 @@ static void riscv_virt_board_init(MachineState *machine)
 memory_region_add_subregion(system_memory, memmap[VIRT_MROM].base,
 mask_rom);
 
-if (machine->firmware) {
-riscv_load_firmware(machine->firmware, memmap[VIRT_DRAM].base);
-}
+riscv_find_and_load_firmware(machine, BIOS_FILENAME,
+ memmap[VIRT_DRAM].base);
 
 if (machine->kernel_filename) {
 uint64_t kernel_entry = riscv_load_kernel(machine->kernel_filename);
diff --git a/include/hw/riscv/boot.h 

[Qemu-devel] [PATCH v2 0/2] RISC-V: Add default OpenSBI ROM

[Alistair Francis]

This series includes the OpenSBI firmware for QEMU RISC-V users.

To avoid breakages we have not changed the default behaviour of QEMU.
The plan is to change the default though, which is why an entry to the
qemu-deprecated.texi file has been added as well as a new warning.

After this series QEMU 4.1 has three options:
 1. ``-bios none`` - This is the current default behavior if no -bios option
  is included. QEMU will not automatically load any firmware. It is up
  to the user to load all the images they need.
 2. ``-bios default`` - In a future QEMU release this will become the default
  behaviour if no -bios option is specified. This option will load the
  default OpenSBI firmware automatically. The firmware is included with
  the QEMU release and no user interaction is required. All a user needs
  to do is specify the kernel they want to boot with the -kernel option
 3. ``-bios `` - Tells QEMU to load the specified file as the firmwrae.

All users should transition to using a -bios option. We can start
updating all documentation after the release of 4.1.

At the end of this series and the transition period we are in the good
place of no longer requiring users to build firmware to boot a kernel.
Instead users can just run QEMU with the -kernel option and everything
will work. They can also override the firmware with their own using
the -bios option. Using "-bios none" will result in no firmware being
loaded (as it is today).

The commits can be found here: 
https://github.com/alistair23/qemu/tree/mainline/alistair/opensbi.next

v2:
 - Error and exit if we can't load the default firmware


Alistair Francis (2):
  roms: Add OpenSBI version 0.4
  hw/riscv: Load OpenSBI as the default firmware

 .gitmodules  |   3 ++
 LICENSE  |  21 +---
 Makefile |   5 +-
 hw/riscv/boot.c  |  54 +++
 hw/riscv/sifive_u.c  |   7 +--
 hw/riscv/virt.c  |  11 ++--
 include/hw/riscv/boot.h  |   3 ++
 pc-bios/README   |  11 
 pc-bios/opensbi-riscv32-virt-fw_jump.bin | Bin 0 -> 36888 bytes
 pc-bios/opensbi-riscv64-sifive_u-fw_jump.bin | Bin 0 -> 40968 bytes
 pc-bios/opensbi-riscv64-virt-fw_jump.bin | Bin 0 -> 40968 bytes
 qemu-deprecated.texi |  20 +++
 roms/Makefile|  48 -
 roms/opensbi |   1 +
 14 files changed, 157 insertions(+), 27 deletions(-)
 create mode 100755 pc-bios/opensbi-riscv32-virt-fw_jump.bin
 create mode 100755 pc-bios/opensbi-riscv64-sifive_u-fw_jump.bin
 create mode 100755 pc-bios/opensbi-riscv64-virt-fw_jump.bin
 create mode 16 roms/opensbi

-- 
2.22.0

Re: [Qemu-devel] [Qemu-riscv] [PATCH 2/2] riscv: sifive_u: Update the plic hart config to support multicore

[Alistair Francis]

On Mon, Jul 8, 2019 at 9:32 AM Fabien Chouteau  wrote:
>
> Hi Bin,
>
> Thanks for this patch.
>
> I know I am very late to the game but I have a comment here.
>
> On 17/05/2019 17:51, Bin Meng wrote:
> > +/* create PLIC hart topology configuration string */
> > +plic_hart_config_len = (strlen(SIFIVE_U_PLIC_HART_CONFIG) + 1) * 
> > smp_cpus;
> > +plic_hart_config = g_malloc0(plic_hart_config_len);
> > +for (i = 0; i < smp_cpus; i++) {
> > +if (i != 0) {
> > +strncat(plic_hart_config, ",", plic_hart_config_len);
> > +}
> > +strncat(plic_hart_config, SIFIVE_U_PLIC_HART_CONFIG,
> > +plic_hart_config_len);
> > +plic_hart_config_len -= (strlen(SIFIVE_U_PLIC_HART_CONFIG) + 1);
> > +}
> > +
>
> This will create up to 4 MS PLIC devices. However on the Unleashed FU540 the 
> PLICs are M,MS,MS,MS,MS because of the monitor hart #0.
>
> This means a different memory layout than the real hardware.
>
> For instance address 0x0C00_2080 will be hart #0 S-Mode interrupt enables in 
> QEMU, instead of #1 M-Mode interrupt enables for the real hardware.
>
> To fix this I suggest to change this loop to:
>
> for (i = 0; i < smp_cpus; i++) {
> if (i != 0) {
> strncat(plic_hart_config, "," SIFIVE_U_PLIC_HART_CONFIG,
> plic_hart_config_len);
> } else {
> strncat(plic_hart_config, "M", plic_hart_config_len);
> }
> plic_hart_config_len -= (strlen(SIFIVE_U_PLIC_HART_CONFIG) + 1);
> }
>
> This will make hart #0 PLIC in M mode and the others in MS.
>
> What do you think?

I think I understand what you mean, in which case I also think you are
correct. Do you want to send a patch and we can discuss there?

Alistair

>
> Best regards,
>

Re: [Qemu-devel] [PATCH v1 2/2] hw/riscv: Load OpenSBI as the default firmware

[Alistair Francis]

On Wed, Jul 10, 2019 at 3:46 PM Guenter Roeck  wrote:
>
> On Mon, Jul 08, 2019 at 11:49:40AM -0700, Alistair Francis wrote:
> > If the user hasn't specified a firmware to load (with -bios) or
> > specified no bios (with -bios none) then load OpenSBI by default. This
> > allows users to boot a RISC-V kernel with just -kernel.
> >
> > Signed-off-by: Alistair Francis 
> > Reviewed-by: Bin Meng 
> > Tested-by: Bin Meng 
> > ---
> >  hw/riscv/boot.c | 49 +
> >  hw/riscv/sifive_u.c |  7 +++---
> >  hw/riscv/virt.c | 11 ++---
> >  include/hw/riscv/boot.h |  3 +++
> >  qemu-deprecated.texi| 20 +
> >  5 files changed, 84 insertions(+), 6 deletions(-)
> >
> > diff --git a/hw/riscv/boot.c b/hw/riscv/boot.c
> > index ff023f42d0..c7d72f682f 100644
> > --- a/hw/riscv/boot.c
> > +++ b/hw/riscv/boot.c
> > @@ -18,6 +18,7 @@
> >   */
> >
> >  #include "qemu/osdep.h"
> > +#include "qemu-common.h"
> >  #include "qemu/units.h"
> >  #include "qemu/error-report.h"
> >  #include "exec/cpu-defs.h"
> > @@ -32,6 +33,54 @@
> >  # define KERNEL_BOOT_ADDRESS 0x8020
> >  #endif
> >
> > +void riscv_find_and_load_firmware(MachineState *machine,
> > +  const char *default_machine_firmware,
> > +  hwaddr firmware_load_addr)
> > +{
> > +char *firmware_filename;
> > +
> > +if (!machine->firmware) {
> > +/*
> > + * The user didn't specify -bios.
> > + * At the moment we default to loading nothing when this hapens.
> > + * In the future this defaul will change to loading the prebuilt
> > + * OpenSBI firmware. Let's warn the user and then continue.
> > +*/
> > +warn_report("No -bios option specified. Not loading a firmware.");
> > +warn_report("This default will change in QEMU 4.3. Please use the 
> > " \
> > +"-bios option to aviod breakages when this happens.");
> > +warn_report("See QEMU's deprecation documentation for details");
> > +return;
> > +}
> > +
> > +if (!strcmp(machine->firmware, "default")) {
> > +/*
> > + * The user has specified "-bios default". That means we are going 
> > to
> > + * load the OpenSBI binary included in the QEMU source.
> > + *
> > + * We can't load the binary by default as it will break existing 
> > users
> > + * as users are already loading their own firmware.
> > + *
> > + * Let's try to get everyone to specify the -bios option at all 
> > times,
> > + * so then in the future we can make "-bios default" the default 
> > option
> > + * if no -bios option is set without breaking anything.
> > + */
> > +firmware_filename = qemu_find_file(QEMU_FILE_TYPE_BIOS,
> > +   default_machine_firmware);
>
> This can return NULL if the file is not found (such as after a bad install) 
> ...
>
> > +} else {
> > +firmware_filename = machine->firmware;
> > +}
> > +
> > +if (strcmp(firmware_filename, "none")) {
>
> ... which will then crash here.

Yep, you are right. I'll send a v2

Alistair

>
> > +/* If not "none" load the firmware */
> > +riscv_load_firmware(firmware_filename, firmware_load_addr);
> > +}
> > +
> > +if (!strcmp(machine->firmware, "default")) {
> > +g_free(firmware_filename);
> > +}
> > +}
> > +
> >  target_ulong riscv_load_firmware(const char *firmware_filename,
> >   hwaddr firmware_load_addr)
> >  {
> > diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c
> > index ca53a9290d..71b8083c05 100644
> > --- a/hw/riscv/sifive_u.c
> > +++ b/hw/riscv/sifive_u.c
> > @@ -49,6 +49,8 @@
> >
> >  #include 
> >
> > +#define BIOS_FILENAME "opensbi-riscv64-sifive_u-fw_jump.bin"
> > +
> >  static const struct MemmapEntry {
> >  hwaddr base;
> >  hwaddr size;
> > @@ -269,9 +271,8 @@ static void riscv_sifive_u_init(MachineState *machine)
> >  /* create device tree */
> >  create_fdt(s, memmap, machine->ram_size, machine->kernel_cmdline);
> >
> > -if (machine->firmware) {
> > -riscv_load_firmware(machine->firmware, memmap[SIFIVE_U_DRAM].base);
> > -}
> > +riscv_find_and_load_firmware(machine, BIOS_FILENAME,
> > + memmap[SIFIVE_U_DRAM].base);
> >
> >  if (machine->kernel_filename) {
> >  riscv_load_kernel(machine->kernel_filename);
> > diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
> > index ecdc77d728..25faf3b417 100644
> > --- a/hw/riscv/virt.c
> > +++ b/hw/riscv/virt.c
> > @@ -44,6 +44,12 @@
> >
> >  #include 
> >
> > +#if defined(TARGET_RISCV32)
> > +# define BIOS_FILENAME "opensbi-riscv32-virt-fw_jump.bin"
> > +#else
> > +# define BIOS_FILENAME "opensbi-riscv64-virt-fw_jump.bin"
> > +#endif
> > +
> >  static const struct MemmapEntry {
> >  hwaddr base;
> 

[Qemu-devel] [PATCH] configure: fix sdl detection using sdl2-config

[Carlo Marcelo Arenas Belón]

If SDL2 is requested but pkg-config doesn't have a module for it
configure should fallback to use sdl*-config, but wasn't able to
because and old variable (from SDL) was being used by mistake.

Correct the variable name and complete other related changes so
there are no more references to the old SDL.

Fixes: 0015ca5cbabe ("ui: remove support for SDL1.2 in favour of SDL2")
Signed-off-by: Carlo Marcelo Arenas Belón 
---
 configure | 9 -
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/configure b/configure
index 4983c8b533..0f88ba98a6 100755
--- a/configure
+++ b/configure
@@ -3016,15 +3016,15 @@ fi
 ##
 # SDL probe
 
-# Look for sdl configuration program (pkg-config or sdl-config).  Try
-# sdl-config even without cross prefix, and favour pkg-config over sdl-config.
+# Look for sdl configuration program (pkg-config or sdl2-config).  Try
+# sdl2-config even without cross prefix, and favour pkg-config over 
sdl2-config.
 
 sdl_probe ()
 {
   if $pkg_config sdl2 --exists; then
 sdlconfig="$pkg_config sdl2"
 sdlversion=$($sdlconfig --modversion 2>/dev/null)
-  elif has ${sdl_config}; then
+  elif has "$sdl2_config"; then
 sdlconfig="$sdl2_config"
 sdlversion=$($sdlconfig --version)
   else
@@ -3035,7 +3035,7 @@ sdl_probe ()
 # no need to do the rest
 return
   fi
-  if test -n "$cross_prefix" && test "$(basename "$sdlconfig")" = sdl-config; 
then
+  if test -n "$cross_prefix" && test "$(basename "$sdlconfig")" = sdl2-config; 
then
 echo warning: using "\"$sdlconfig\"" to detect cross-compiled sdl >&2
   fi
 
@@ -8019,7 +8019,6 @@ preserve_env PKG_CONFIG
 preserve_env PKG_CONFIG_LIBDIR
 preserve_env PKG_CONFIG_PATH
 preserve_env PYTHON
-preserve_env SDL_CONFIG
 preserve_env SDL2_CONFIG
 preserve_env SMBD
 preserve_env STRIP
-- 
2.22.0

Re: [Qemu-devel] [PATCH v1 2/2] hw/riscv: Load OpenSBI as the default firmware

[Guenter Roeck]

On Mon, Jul 08, 2019 at 11:49:40AM -0700, Alistair Francis wrote:
> If the user hasn't specified a firmware to load (with -bios) or
> specified no bios (with -bios none) then load OpenSBI by default. This
> allows users to boot a RISC-V kernel with just -kernel.
> 
> Signed-off-by: Alistair Francis 
> Reviewed-by: Bin Meng 
> Tested-by: Bin Meng 
> ---
>  hw/riscv/boot.c | 49 +
>  hw/riscv/sifive_u.c |  7 +++---
>  hw/riscv/virt.c | 11 ++---
>  include/hw/riscv/boot.h |  3 +++
>  qemu-deprecated.texi| 20 +
>  5 files changed, 84 insertions(+), 6 deletions(-)
> 
> diff --git a/hw/riscv/boot.c b/hw/riscv/boot.c
> index ff023f42d0..c7d72f682f 100644
> --- a/hw/riscv/boot.c
> +++ b/hw/riscv/boot.c
> @@ -18,6 +18,7 @@
>   */
>  
>  #include "qemu/osdep.h"
> +#include "qemu-common.h"
>  #include "qemu/units.h"
>  #include "qemu/error-report.h"
>  #include "exec/cpu-defs.h"
> @@ -32,6 +33,54 @@
>  # define KERNEL_BOOT_ADDRESS 0x8020
>  #endif
>  
> +void riscv_find_and_load_firmware(MachineState *machine,
> +  const char *default_machine_firmware,
> +  hwaddr firmware_load_addr)
> +{
> +char *firmware_filename;
> +
> +if (!machine->firmware) {
> +/*
> + * The user didn't specify -bios.
> + * At the moment we default to loading nothing when this hapens.
> + * In the future this defaul will change to loading the prebuilt
> + * OpenSBI firmware. Let's warn the user and then continue.
> +*/
> +warn_report("No -bios option specified. Not loading a firmware.");
> +warn_report("This default will change in QEMU 4.3. Please use the " \
> +"-bios option to aviod breakages when this happens.");
> +warn_report("See QEMU's deprecation documentation for details");
> +return;
> +}
> +
> +if (!strcmp(machine->firmware, "default")) {
> +/*
> + * The user has specified "-bios default". That means we are going to
> + * load the OpenSBI binary included in the QEMU source.
> + *
> + * We can't load the binary by default as it will break existing 
> users
> + * as users are already loading their own firmware.
> + *
> + * Let's try to get everyone to specify the -bios option at all 
> times,
> + * so then in the future we can make "-bios default" the default 
> option
> + * if no -bios option is set without breaking anything.
> + */
> +firmware_filename = qemu_find_file(QEMU_FILE_TYPE_BIOS,
> +   default_machine_firmware);

This can return NULL if the file is not found (such as after a bad install) ...

> +} else {
> +firmware_filename = machine->firmware;
> +}
> +
> +if (strcmp(firmware_filename, "none")) {

... which will then crash here.

> +/* If not "none" load the firmware */
> +riscv_load_firmware(firmware_filename, firmware_load_addr);
> +}
> +
> +if (!strcmp(machine->firmware, "default")) {
> +g_free(firmware_filename);
> +}
> +}
> +
>  target_ulong riscv_load_firmware(const char *firmware_filename,
>   hwaddr firmware_load_addr)
>  {
> diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c
> index ca53a9290d..71b8083c05 100644
> --- a/hw/riscv/sifive_u.c
> +++ b/hw/riscv/sifive_u.c
> @@ -49,6 +49,8 @@
>  
>  #include 
>  
> +#define BIOS_FILENAME "opensbi-riscv64-sifive_u-fw_jump.bin"
> +
>  static const struct MemmapEntry {
>  hwaddr base;
>  hwaddr size;
> @@ -269,9 +271,8 @@ static void riscv_sifive_u_init(MachineState *machine)
>  /* create device tree */
>  create_fdt(s, memmap, machine->ram_size, machine->kernel_cmdline);
>  
> -if (machine->firmware) {
> -riscv_load_firmware(machine->firmware, memmap[SIFIVE_U_DRAM].base);
> -}
> +riscv_find_and_load_firmware(machine, BIOS_FILENAME,
> + memmap[SIFIVE_U_DRAM].base);
>  
>  if (machine->kernel_filename) {
>  riscv_load_kernel(machine->kernel_filename);
> diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
> index ecdc77d728..25faf3b417 100644
> --- a/hw/riscv/virt.c
> +++ b/hw/riscv/virt.c
> @@ -44,6 +44,12 @@
>  
>  #include 
>  
> +#if defined(TARGET_RISCV32)
> +# define BIOS_FILENAME "opensbi-riscv32-virt-fw_jump.bin"
> +#else
> +# define BIOS_FILENAME "opensbi-riscv64-virt-fw_jump.bin"
> +#endif
> +
>  static const struct MemmapEntry {
>  hwaddr base;
>  hwaddr size;
> @@ -399,9 +405,8 @@ static void riscv_virt_board_init(MachineState *machine)
>  memory_region_add_subregion(system_memory, memmap[VIRT_MROM].base,
>  mask_rom);
>  
> -if (machine->firmware) {
> -riscv_load_firmware(machine->firmware, memmap[VIRT_DRAM].base);
> -}
> +

[Qemu-devel] Problems building and installing qemu v4.1.0-rc1 in single step

[Guenter Roeck]

Hi,

when trying to run "make -j30 install" from a clean tree on v4.1.0-rc0, I get
_lots_ of undefined symbol errors.

If I run "make -j30" followed by "make -j30 install", make succeeds, but then
I get linker errors such as the following when running "make -j30 install".

/usr/bin/ld: final link failed: File truncated
/usr/bin/ld: BFD (GNU Binutils for Ubuntu) 2.26.1 internal error,
aborting at ../../bfd/merge.c:905 in _bfd_merged_section_offset

Running "make -j30" followed by "make install" succeeds.

This looks like "make install" may have bad dependencies. Has anyone else
experienced this problem ?

Thanks,
Guenter

Re: [Qemu-devel] [PATCH] LUKS: support preallocation in qemu-img

[Max Reitz]

On 10.07.19 23:24, Max Reitz wrote:
> On 10.07.19 19:03, Maxim Levitsky wrote:
>> preallocation=off and preallocation=metadata
>> both allocate luks header only, and preallocation=falloc/full
>> is passed to underlying file, with the given image size.
>>
>> Note that the actual preallocated size is a bit smaller due
>> to luks header.
> 
> Couldn’t you just preallocate it after creating the crypto header so
> qcrypto_block_get_payload_offset(crypto->block) + size is the actual
> file size?
> 
>> Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1534951
>>
>> Signed-off-by: Maxim Levitsky 
>> ---
>>  block/crypto.c | 28 ++--
>>  1 file changed, 26 insertions(+), 2 deletions(-)
> 
> Hm.  I would expect a preallocated image to read 0.  But if you just
> pass this through to the protocol layer, it won’t read 0.
> 
> (In fact, I don’t even quite see the point of having LUKS as an own
> format still.  It was useful when qcow2 didn’t have LUKS support, but
> now it does, so...  I suppose everyone using the LUKS format should
> actually be using qcow2 with LUKS?)

Kevin just pointed out to me that our LUKS format is compatible to the
actual layout cryptsetup uses.  OK, that is an important use case.

Hm.  Unfortunately, that doesn’t really necessitate preallocation.

Well, whatever.  If it’s simple enough, that shouldn’t stop us from
implementing preallocation anyway.


Now I found that qapi/block-core.json defines PreallocMode’s falloc and
full values as follows:

> # @falloc: like @full preallocation but allocate disk space by
> #  posix_fallocate() rather than writing zeros.
> # @full: preallocate all data by writing zeros to device to ensure disk
> #space is really available. @full preallocation also sets up
> #metadata correctly.

So it isn’t just me who expects these to pre-initialize the image to 0.
 Hm, although...  I suppose @falloc technically does not specify whether
the data reads as zeroes.  I kind of find it to be implied, but, well...

Max



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v2 00/12] block: Fixes for concurrent block jobs

[Max Reitz]

Ping

I suppose I’ll apply the patches I had in v1 if I don’t get reviews for
the new patches, because some fixes are better than none.

(And probably patch 2, because it’s obvious.)

Max

On 03.07.19 19:28, Max Reitz wrote:
> This is a v2 to “block: Add BDS.never_freeze”.
> 
> It depends on my “block: Delay poll when ending drained sections”
> series:
> 
> Depends-on: <20190619152603.5937-1-mre...@redhat.com>
> 
> 
> It turned out that if you run 030 (or just the new test_overlapping_5
> case) sufficiently often, it breaks; which is why I’m hesitant to just
> merge the “add never_freeze” series as it is.
> 
> There are several reasons for why this test case breaks, I hope patches
> 3 to 6 fix them.  Patch 12 adds a test that is much more reliable than
> test_overlapping_5 at detecting the problems fixed by at least patches 4
> to 6.  (I think that 3 doesn’t really need a test.)
> 
> I’m sure there are other ways to see these problems, but well, coming
> from 030, concurrent commit/stream jobs are how I reproduced them.
> Hence the same of this series.
> 
> Patch 2 is for something I encountered on the way.  Patch 11 tests it.
> 
> 
> v2:
> - Added a bunch of more patches.
> 
> 
> git backport-diff against v1:
> 
> Key:
> [] : patches are identical
> [] : number of functional differences between upstream/downstream patch
> [down] : patch is downstream-only
> The flags [FC] indicate (F)unctional and (C)ontextual differences, 
> respectively
> 
> 001/12:[] [--] 'block: Add BDS.never_freeze'
> 002/12:[down] 'block/stream: Fix error path'
> 003/12:[down] 'block/stream: Swap backing file change order'
> 004/12:[down] 'block: Keep subtree drained in drop_intermediate'
> 005/12:[down] 'block: Reduce (un)drains when replacing a child'
> 006/12:[down] 'block: Deep-clear inherits_from'
> 007/12:[] [--] 'iotests: Fix throttling in 030'
> 008/12:[] [--] 'iotests: Compare error messages in 030'
> 009/12:[] [--] 'iotests: Add @use_log to VM.run_job()'
> 010/12:[] [--] 'iotests: Add new case to 030'
> 011/12:[down] 'iotests: Add read-only test case to 030'
> 012/12:[down] 'iotests: Add test for concurrent stream/commit'
> 
> 
> 
> Max Reitz (12):
>   block: Add BDS.never_freeze
>   block/stream: Fix error path
>   block/stream: Swap backing file change order
>   block: Keep subtree drained in drop_intermediate
>   block: Reduce (un)drains when replacing a child
>   block: Deep-clear inherits_from
>   iotests: Fix throttling in 030
>   iotests: Compare error messages in 030
>   iotests: Add @use_log to VM.run_job()
>   iotests: Add new case to 030
>   iotests: Add read-only test case to 030
>   iotests: Add test for concurrent stream/commit
> 
>  include/block/block_int.h |   3 +
>  block.c   |  93 +--
>  block/commit.c|   4 +
>  block/mirror.c|   4 +
>  block/stream.c|   4 +-
>  tests/qemu-iotests/030| 150 +--
>  tests/qemu-iotests/030.out|   4 +-
>  tests/qemu-iotests/258| 163 ++
>  tests/qemu-iotests/258.out|  33 +++
>  tests/qemu-iotests/group  |   1 +
>  tests/qemu-iotests/iotests.py |  18 ++--
>  11 files changed, 413 insertions(+), 64 deletions(-)
>  create mode 100755 tests/qemu-iotests/258
>  create mode 100644 tests/qemu-iotests/258.out
> 




signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH] LUKS: support preallocation in qemu-img

[Max Reitz]

On 10.07.19 19:03, Maxim Levitsky wrote:
> preallocation=off and preallocation=metadata
> both allocate luks header only, and preallocation=falloc/full
> is passed to underlying file, with the given image size.
> 
> Note that the actual preallocated size is a bit smaller due
> to luks header.

Couldn’t you just preallocate it after creating the crypto header so
qcrypto_block_get_payload_offset(crypto->block) + size is the actual
file size?

> Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1534951
> 
> Signed-off-by: Maxim Levitsky 
> ---
>  block/crypto.c | 28 ++--
>  1 file changed, 26 insertions(+), 2 deletions(-)

Hm.  I would expect a preallocated image to read 0.  But if you just
pass this through to the protocol layer, it won’t read 0.

(In fact, I don’t even quite see the point of having LUKS as an own
format still.  It was useful when qcow2 didn’t have LUKS support, but
now it does, so...  I suppose everyone using the LUKS format should
actually be using qcow2 with LUKS?)

Max



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH 0/4] virtio: handle zoned backing devices

[Kevin Wolf]

Am 10.07.2019 um 13:33 hat Paolo Bonzini geschrieben:
> On 10/07/19 13:02, Kevin Wolf wrote:
> > Hm... Actually, file-posix implements .bdrv_check_perm and could just
> > refuse attaching a parent there if it doesn't request a specific
> > permission like BLK_PERM_SUPPORT_ZONED. That should give us the
> > whitelist semantics through existing infrastructure.
> 
> I'd like Dmitry to have something more precise to base his work on.  The
> permissions system is really complicated and I never really wrapped my
> head around it, so I need your help.
> 
> IIUC, blkconf_apply_backend_options would grow a new argument (like
> "resizable") and that argument would add BLK_PERM_SUPPORT_ZONED to the
> perm that blkconf_apply_backend_options passes to blk_set_perm.  On the
> other side raw_check_perm would say something like
> 
> if (is_zoned(s) && !(perm & BLK_PERM_SUPPORT_ZONED)) {
> error_setg();
> return -ENOTSUP;
> }
> 
> Is this correct?

Yes, I think this is how you'd best implement it.

> In addition, BLK_PERM_SUPPORT_ZONED would have to be a shared
> permission, since it's possible to assign the same block device to
> multiple scsi-block devices.  So BLK_PERM_SUPPORT_ZONED would be added
> unconditionally to shared_perm.

Right, this part shows that we're kind of abusing the permission system
here. I think unconditionally adding BLK_PERM_SUPPORT_ZONED to the set
of shared permissions could probably happen centrally in
bdrv_child_perm().

> ps: I have always thought that shared_perm is expressed the wrong way
> and should have been "denied_perm".  How hard would it be to change that
> now?

I'm not sure it would be better. It is shared_perm because that means
that the default is restrictive (error mode: operation refused, clearly
reported, easy to fix) rather than permissive (error mode: image
corruption, hard to figure out where we were too permissive). Basically,
whitelist instead of blacklist, once again.

But if we did indeed decide to change it, the only way to find out how
hard it is would be doing it. I suspect not too hard in principle, but
making sure that we converted all callers and don't introduce wrong
callers later through (silent) merge conflicts is the more concerning
part.

Kevin

Re: [Qemu-devel] [PATCH] nvme: Set number of queues later in nvme_init()

[Max Reitz]

On 10.07.19 16:57, Michal Privoznik wrote:
> When creating the admin queue in nvme_init() the variable that
> holds the number of queues created is modified before actual
> queue creation. This is a problem because if creating the queue
> fails then the variable is left in inconsistent state. This was
> actually observed when I tried to hotplug a nvme disk. The
> control got to nvme_file_open() which called nvme_init() which
> failed and thus nvme_close() was called which in turn called
> nvme_free_queue_pair() with queue being NULL. This lead to an
> instant crash:
> 
>   #0  0x55d9507ec211 in nvme_free_queue_pair (bs=0x55d952ddb880, q=0x0) 
> at block/nvme.c:164
>   #1  0x55d9507ee180 in nvme_close (bs=0x55d952ddb880) at block/nvme.c:729
>   #2  0x55d9507ee3d5 in nvme_file_open (bs=0x55d952ddb880, 
> options=0x55d952bb1410, flags=147456, errp=0x7ffd8e19e200) at block/nvme.c:781
>   #3  0x55d9507629f3 in bdrv_open_driver (bs=0x55d952ddb880, 
> drv=0x55d95109c1e0 , node_name=0x0, options=0x55d952bb1410, 
> open_flags=147456, errp=0x7ffd8e19e310) at block.c:1291
>   #4  0x55d9507633d6 in bdrv_open_common (bs=0x55d952ddb880, file=0x0, 
> options=0x55d952bb1410, errp=0x7ffd8e19e310) at block.c:1551
>   #5  0x55d950766881 in bdrv_open_inherit (filename=0x0, reference=0x0, 
> options=0x55d952bb1410, flags=32768, parent=0x55d9538ce420, 
> child_role=0x55d950eaade0 , errp=0x7ffd8e19e510) at block.c:3063
>   #6  0x55d950765ae4 in bdrv_open_child_bs (filename=0x0, 
> options=0x55d9541cdff0, bdref_key=0x55d950af33aa "file", 
> parent=0x55d9538ce420, child_role=0x55d950eaade0 , 
> allow_none=true, errp=0x7ffd8e19e510) at block.c:2712
>   #7  0x55d950766633 in bdrv_open_inherit (filename=0x0, reference=0x0, 
> options=0x55d9541cdff0, flags=0, parent=0x0, child_role=0x0, 
> errp=0x7ffd8e19e908) at block.c:3011
>   #8  0x55d950766dba in bdrv_open (filename=0x0, reference=0x0, 
> options=0x55d953d00390, flags=0, errp=0x7ffd8e19e908) at block.c:3156
>   #9  0x55d9507cb635 in blk_new_open (filename=0x0, reference=0x0, 
> options=0x55d953d00390, flags=0, errp=0x7ffd8e19e908) at 
> block/block-backend.c:389
>   #10 0x55d950465ec5 in blockdev_init (file=0x0, bs_opts=0x55d953d00390, 
> errp=0x7ffd8e19e908) at blockdev.c:602
> 
> Signed-off-by: Michal Privoznik 
> ---
>  block/nvme.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Thanks, applied to my block branch:

https://git.xanclic.moe/XanClic/qemu/commits/branch/block

Max



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v2 00/13] Add SEV guest live migration support

[no-reply]

Patchew URL: 
https://patchew.org/QEMU/20190710202219.25939-1-brijesh.si...@amd.com/



Hi,

This series failed build test on s390x host. Please find the details below.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
# Testing script will be invoked under the git checkout with
# HEAD pointing to a commit that has the patches applied on top of "base"
# branch
set -e

echo
echo "=== ENV ==="
env

echo
echo "=== PACKAGES ==="
rpm -qa

echo
echo "=== UNAME ==="
uname -a

CC=$HOME/bin/cc
INSTALL=$PWD/install
BUILD=$PWD/build
mkdir -p $BUILD $INSTALL
SRC=$PWD
cd $BUILD
$SRC/configure --cc=$CC --prefix=$INSTALL
make -j4
# XXX: we need reliable clean up
# make check -j4 V=1
make install
=== TEST SCRIPT END ===

  CC  alpha-softmmu/hw/scsi/virtio-scsi-dataplane.o
/usr/bin/ld: qapi/qapi-commands-misc-target.o: in function 
`qmp_marshal_migrate_set_sev_info':
/var/tmp/patchew-tester-tmp-iinjo1bi/src/build/qapi/qapi-commands-misc-target.c:363:
 undefined reference to `qmp_migrate_set_sev_info'
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:208: qemu-system-cris] Error 1
make: *** [Makefile:472: cris-softmmu/all] Error 2
make: *** Waiting for unfinished jobs
---
  CC  aarch64-softmmu/hw/misc/bcm2835_mbox.o
/usr/bin/ld: qapi/qapi-commands-misc-target.o: in function 
`qmp_marshal_migrate_set_sev_info':
/var/tmp/patchew-tester-tmp-iinjo1bi/src/build/qapi/qapi-commands-misc-target.c:363:
 undefined reference to `qmp_migrate_set_sev_info'
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:208: qemu-system-alpha] Error 1
make: *** [Makefile:472: alpha-softmmu/all] Error 2
  CC  aarch64-softmmu/hw/misc/bcm2835_property.o
---
  CC  arm-softmmu/target/arm/debug_helper.o
/usr/bin/ld: qapi/qapi-commands-misc-target.o: in function 
`qmp_marshal_migrate_set_sev_info':
/var/tmp/patchew-tester-tmp-iinjo1bi/src/build/qapi/qapi-commands-misc-target.c:363:
 undefined reference to `qmp_migrate_set_sev_info'
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:208: qemu-system-aarch64] Error 1
make: *** [Makefile:472: aarch64-softmmu/all] Error 2
  GEN arm-softmmu/target/arm/decode-vfp.inc.c
---
  LINKarm-softmmu/qemu-system-arm
/usr/bin/ld: qapi/qapi-commands-misc-target.o: in function 
`qmp_marshal_migrate_set_sev_info':
/var/tmp/patchew-tester-tmp-iinjo1bi/src/build/qapi/qapi-commands-misc-target.c:363:
 undefined reference to `qmp_migrate_set_sev_info'
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:208: qemu-system-arm] Error 1
make: *** [Makefile:472: arm-softmmu/all] Error 2
=== OUTPUT END ===


The full log is available at
http://patchew.org/logs/20190710202219.25939-1-brijesh.si...@amd.com/testing.s390x/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

Re: [Qemu-devel] [PATCH 6/8] block/backup: issue progress updates for skipped regions

[Max Reitz]

On 10.07.19 22:47, John Snow wrote:
> 
> 
> On 7/10/19 4:30 PM, Max Reitz wrote:
>> On 10.07.19 20:20, John Snow wrote:
>>>
>>>
>>> On 7/10/19 12:36 PM, Max Reitz wrote:
 On 10.07.19 03:05, John Snow wrote:
> The way bitmap backups work is by starting at 75% if it needs
> to copy just 25% of the disk.

 Although there is this comment:

> /* TODO job_progress_set_remaining() would make more sense */

 So...

> The way sync=top currently works, however, is to start at 0% and then
> never update the progress if it doesn't copy a region. If it needs to
> copy 25% of the disk, we'll finish at 25%.
>
> Update the progress when we skip regions.

 Wouldn’t it be more correct to decrease the job length?

 Max

>>>
>>> Admittedly I have precisely no idea. Maybe? As far as I understand it,
>>> we guarantee only:
>>>
>>> (1) Progress monotonically increases
>>> (2) Upon completion, progress will equal the total work estimate.
>>> [Trying to fix that to be true here.]
>>>
>>> This means we can do stuff like:
>>>
>>> - Total work estimate can increase or decrease arbitrarily
>>> - Neither value has to mean anything in particular
>>>
>>>
>>> Bitmap sync works by artificially increasing progress for NOP regions,
>>> seen in init_copy_bitmap.
>>
>> Yes, and it has a TODO comment that says it should be done differently.
>>
>>> Full sync also tends to increase progress regardless of it actually did
>>> a copy or not; offload support also counts as progress here. So if you
>>> full sync an empty image, you'll see it increasing the progress as it
>>> doesn't actually do anything.
>>>
>>> Top sync is the odd one out, which just omits progress for regions it skips.
>>>
>>> My only motivation here was to make them consistent. Can I do it the
>>> other way? Yeah, probably. Is one way better than the other? I
>>> legitimately have no idea. I guess whoever wrote the last comment felt
>>> that it should all be the other way instead. Why'd they not do that?
>>
>> If you look at the commit (05df8a6a2b4), I suppose it was because that
>> commit simply did not intend to change behavior.  It just touched that
>> piece of code and noted that maybe there should be a follow-up commit to
>> change it.
>>
>> But yeah, whatever.
>>
>> Reviewed-by: Max Reitz 
>>
> 
> I mean. I'll make it consistent either way, but I actually don't know
> which way I should make it go. I just think that all the modes should
> work the same if we can help it.
> 
> Flip a coin?

If you’d flip a coin, I can say that I’d find it a bit more meaningful
to reduce the length.  Especially if you change sync=top to calculate
beforehand how much we need to copy, so the length isn’t even reduced
while the job is running.

And it’s easier to remove the TODO comment this way.  If we decide
against it, you’d have to remove the TODO comment aso (because we
decided against it), but you’d need to justify it in the commit message.
 And we all know that writing explanations and documentation is the
hardest thing of all.

Max



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v2 00/13] Add SEV guest live migration support

[no-reply]

Patchew URL: 
https://patchew.org/QEMU/20190710202219.25939-1-brijesh.si...@amd.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Subject: [Qemu-devel] [PATCH v2 00/13] Add SEV guest live migration support
Type: series
Message-id: 20190710202219.25939-1-brijesh.si...@amd.com

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Switched to a new branch 'test'
9eb9639 target/i386: sev: remove migration blocker
00c1826 migration: add support to migrate page encryption bitmap
cc5d459 kvm: introduce high-level API to migrate the page encryption bitmap
744b933 target/i386: sev: add support to load incoming encrypted page
0365253 target/i386: sev: add support to encrypt the outgoing page
b616e11 misc.json: add migrate-set-sev-info command
6dbc25d target/i386: sev: do not create launch context for an incoming guest
467cdc0 doc: update AMD SEV to include Live migration flow
a02bd6f doc: update AMD SEV API spec web link
26fee5c kvm: add support to sync the page encryption state bitmap
ec80b9c migration/ram: add support to send encrypted pages
f09bf4e kvm: introduce high-level API to support encrypted page migration
9b1e5ae linux-headers: update kernel header to include SEV migration commands

=== OUTPUT BEGIN ===
1/13 Checking commit 9b1e5aef53f4 (linux-headers: update kernel header to 
include SEV migration commands)
2/13 Checking commit f09bf4e4d9e0 (kvm: introduce high-level API to support 
encrypted page migration)
WARNING: line over 80 characters
#45: FILE: accel/kvm/kvm-all.c:177:
+return 
kvm_state->memcrypt_save_outgoing_page(kvm_state->memcrypt_handle,

WARNING: line over 80 characters
#56: FILE: accel/kvm/kvm-all.c:188:
+return 
kvm_state->memcrypt_load_incoming_page(kvm_state->memcrypt_handle,

total: 0 errors, 2 warnings, 96 lines checked

Patch 2/13 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
3/13 Checking commit ec80b9c1d59e (migration/ram: add support to send encrypted 
pages)
4/13 Checking commit 26fee5cf1cef (kvm: add support to sync the page encryption 
state bitmap)
WARNING: Block comments use a leading /* on a separate line
#40: FILE: accel/kvm/kvm-all.c:520:
+ /*HOST_LONG_BITS*/ 64) / 8;

WARNING: line over 80 characters
#110: FILE: include/exec/ram_addr.h:336:
+src = 
atomic_rcu_read(_list.dirty_memory[DIRTY_MEMORY_ENCRYPTED])->blocks;

WARNING: line over 80 characters
#124: FILE: include/exec/ram_addr.h:350:
+static inline void cpu_physical_memory_set_dirty_enc_lebitmap(unsigned long 
*bitmap,

WARNING: line over 80 characters
#141: FILE: include/exec/ram_addr.h:387:
+atomic_xchg([DIRTY_MEMORY_ENCRYPTED][idx][offset], 
temp);

WARNING: line over 80 characters
#143: FILE: include/exec/ram_addr.h:389:
+atomic_or([DIRTY_MEMORY_MIGRATION][idx][offset], 
temp);

WARNING: line over 80 characters
#146: FILE: include/exec/ram_addr.h:392:
+atomic_or([DIRTY_MEMORY_CODE][idx][offset], 
temp);

WARNING: Block comments use a leading /* on a separate line
#156: FILE: include/exec/ram_addr.h:414:
+/* If its encrypted bitmap update, then we need to copy the bitmap

WARNING: line over 80 characters
#160: FILE: include/exec/ram_addr.h:418:
+cpu_physical_memory_set_encrypted_range(start + i * 
TARGET_PAGE_SIZE,

WARNING: line over 80 characters
#161: FILE: include/exec/ram_addr.h:419:
+TARGET_PAGE_SIZE * 
hpratio,

WARNING: line over 80 characters
#174: FILE: include/exec/ram_addr.h:440:
+static inline void cpu_physical_memory_set_encrypted_lebitmap(unsigned long 
*bitmap,

WARNING: line over 80 characters
#178: FILE: include/exec/ram_addr.h:444:
+return cpu_physical_memory_set_dirty_enc_lebitmap(bitmap, start, pages, 
true);

WARNING: line over 80 characters
#185: FILE: include/exec/ram_addr.h:451:
+return cpu_physical_memory_set_dirty_enc_lebitmap(bitmap, start, pages, 
false);

WARNING: line over 80 characters
#195: FILE: include/exec/ram_addr.h:473:
+cpu_physical_memory_test_and_clear_dirty(start, length, 
DIRTY_MEMORY_ENCRYPTED);

WARNING: line over 80 characters
#220: FILE: include/exec/ram_addr.h:559:
+src = 
atomic_rcu_read(_list.dirty_memory[DIRTY_MEMORY_ENCRYPTED])->blocks;

total: 0 errors, 14 warnings, 320 lines checked

Patch 4/13 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
5/13 Checking commit a02bd6ff047c (doc: update AMD SEV API spec web link)
6/13 Checking commit 467cdc0f381c (doc: update AMD SEV to include Live 
migration flow)
7/13 Checking 

Re: [Qemu-devel] [PATCH 6/8] block/backup: issue progress updates for skipped regions

[John Snow]

On 7/10/19 4:30 PM, Max Reitz wrote:
> On 10.07.19 20:20, John Snow wrote:
>>
>>
>> On 7/10/19 12:36 PM, Max Reitz wrote:
>>> On 10.07.19 03:05, John Snow wrote:
 The way bitmap backups work is by starting at 75% if it needs
 to copy just 25% of the disk.
>>>
>>> Although there is this comment:
>>>
 /* TODO job_progress_set_remaining() would make more sense */
>>>
>>> So...
>>>
 The way sync=top currently works, however, is to start at 0% and then
 never update the progress if it doesn't copy a region. If it needs to
 copy 25% of the disk, we'll finish at 25%.

 Update the progress when we skip regions.
>>>
>>> Wouldn’t it be more correct to decrease the job length?
>>>
>>> Max
>>>
>>
>> Admittedly I have precisely no idea. Maybe? As far as I understand it,
>> we guarantee only:
>>
>> (1) Progress monotonically increases
>> (2) Upon completion, progress will equal the total work estimate.
>> [Trying to fix that to be true here.]
>>
>> This means we can do stuff like:
>>
>> - Total work estimate can increase or decrease arbitrarily
>> - Neither value has to mean anything in particular
>>
>>
>> Bitmap sync works by artificially increasing progress for NOP regions,
>> seen in init_copy_bitmap.
> 
> Yes, and it has a TODO comment that says it should be done differently.
> 
>> Full sync also tends to increase progress regardless of it actually did
>> a copy or not; offload support also counts as progress here. So if you
>> full sync an empty image, you'll see it increasing the progress as it
>> doesn't actually do anything.
>>
>> Top sync is the odd one out, which just omits progress for regions it skips.
>>
>> My only motivation here was to make them consistent. Can I do it the
>> other way? Yeah, probably. Is one way better than the other? I
>> legitimately have no idea. I guess whoever wrote the last comment felt
>> that it should all be the other way instead. Why'd they not do that?
> 
> If you look at the commit (05df8a6a2b4), I suppose it was because that
> commit simply did not intend to change behavior.  It just touched that
> piece of code and noted that maybe there should be a follow-up commit to
> change it.
> 
> But yeah, whatever.
> 
> Reviewed-by: Max Reitz 
> 

I mean. I'll make it consistent either way, but I actually don't know
which way I should make it go. I just think that all the modes should
work the same if we can help it.

Flip a coin?

--js

Re: [Qemu-devel] [PATCH 8/8] iotests/257: test traditional sync modes

[Max Reitz]

On 10.07.19 21:00, John Snow wrote:
> 
> 
> On 7/10/19 1:14 PM, Max Reitz wrote:
>> On 10.07.19 03:05, John Snow wrote:
>>> Signed-off-by: John Snow 
>>> ---
>>>  tests/qemu-iotests/257 |   31 +
>>>  tests/qemu-iotests/257.out | 3089 
>>>  2 files changed, 3120 insertions(+)
>>
>> Oof.
>>
> 
> Yeah, it's... a lot of test output. We probably shouldn't count the
> reference test output against any kind of SLOC metrics.
> 
>>> diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
>>> index de8707cb19..8de1c4da19 100755
>>> --- a/tests/qemu-iotests/257
>>> +++ b/tests/qemu-iotests/257
>>
>> [...]
>>
>>> @@ -410,6 +416,11 @@ def test_bitmap_sync(bsync_mode, msync_mode='bitmap', 
>>> failure=None):
>>>  if bsync_mode == 'always' and failure == 'intermediate':
>>>  # We manage to copy one sector (one bit) before the error.
>>>  ebitmap.clear_bit(ebitmap.first_bit)
>>> +if msync_mode in ('full', 'top'):
>>> +# These modes return all bits set except what was 
>>> copied/skipped
>>
>> Hm.  How useful is bitmap support for 'top' then, anyway?  That means
>> that if you want to resume a top backup, you always have to resume it
>> like it was a full backup.  Which sounds kind of useless.
>>
>> Max
>>
> 
> Good point!
> 
> I think this can be fixed by doing an initialization pass of the
> copy_bitmap when sync=top to set only the allocated regions in the bitmap.
> 
> This means that the write notifier won't copy out regions that are
> written to that weren't already in the top layer. I believe this is
> actually a bugfix; the data we'd copy out in such cases is actually in
> the backing layer and shouldn't be copied with sync=top.

Now that you mention it...  I didn’t realize that.  Yes, you’re right.

> So this would have two effects:
> (1) sync=top gets a little more judicious about what it copies out on
> sync=top, and
> (2) the bitmap return value is more meaningful again.
> 
> This doesn't touch sync=none at all, which needs more invasive fixes if
> we wanted it to have useful bitmap return values (it needs to
> differentiate the idea between must-copy and can-copy, and I still don't
> know if this is worthwhile to do, so until I hear otherwise, I'm not gonna.)

No, I’m with you on that one.

Max

>>> +fail_bit = ebitmap.first_bit
>>> +ebitmap.clear()
>>> +ebitmap.dirty_bits(range(fail_bit, SIZE // GRANULARITY))
>>>  ebitmap.compare(get_bitmap(bitmaps, drive0.device, 'bitmap0'))
>>>  
>>>  # 2 - Writes and Reference Backup
>> [...]
>>




signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH 7/8] block/backup: support bitmap sync modes for non-bitmap backups

[Max Reitz]

On 10.07.19 20:32, John Snow wrote:
> 
> 
> On 7/10/19 12:48 PM, Max Reitz wrote:
>> On 10.07.19 03:05, John Snow wrote:
>>> Accept bitmaps and sync policies for the other backup modes.
>>> This allows us to do things like create a bitmap synced to a full backup
>>> without a transaction, or start a resumable backup process.
>>>
>>> Some combinations don't make sense, though:
>>>
>>> - NEVER policy combined with any non-BITMAP mode doesn't do anything,
>>>   because the bitmap isn't used for input or output.
>>>   It's harmless, but is almost certainly never what the user wanted.
>>>
>>> - sync=NONE is more questionable. It can't use on-success because this
>>>   job never completes with success anyway, and the resulting artifact
>>>   of 'always' is suspect: because we start with a full bitmap and only
>>>   copy out segments that get written to, the final output bitmap will
>>>   always be ... a fully set bitmap.
>>>
>>>   Maybe there's contexts in which bitmaps make sense for sync=none,
>>>   but not without more severe changes to the current job, and omitting
>>>   it here doesn't prevent us from adding it later.
>>>
>>> Signed-off-by: John Snow 
>>> ---
>>>  block/backup.c   |  8 +---
>>>  blockdev.c   | 22 ++
>>>  qapi/block-core.json |  6 --
>>>  3 files changed, 27 insertions(+), 9 deletions(-)
>>
>> [...]
>>
>>> diff --git a/blockdev.c b/blockdev.c
>>> index f0b7da53b0..bc152f8e0d 100644
>>> --- a/blockdev.c
>>> +++ b/blockdev.c
>>
>> [...]
>>
>>> +if (!backup->has_bitmap && backup->has_bitmap_mode) {
>>> +error_setg(errp, "Cannot specify Bitmap sync mode without a 
>>> bitmap");
>>
>> Any reason for capitalizing the first “Bitmap”?
>>
>> With a reason or it fixed:
>>
>> Reviewed-by: Max Reitz 
>>
> 
> Hanging around Germans too much?

You should know then that the korrekt way to write it would be:

„Specifying Binarydigitmapsynchronizationmode without a Binarydigitmap
is absolutely verboten!“

> Actually, I can explain why: because a "bitmap" is a generic term, but
> whenever I capitalize it as "Bitmap" I am referring to a Block Dirty
> Bitmap which is a specific sort of thing. I do this unconsciously.
> 
> But in that case, I should actually be consistent in the interface (and
> docstrings and docs and error strings) and always call it that specific
> thing, which I don't.
> 
> "bitmap" is probably more correct for now, but I ought to go through all
> the interface and make it consistent in some way or another.
> 
> 
> (Actually: I'd like to see if I can't rename the "BdrvDirtyBitmap"
> object to something more generic and shorter so I can rename many of the
> functions we have something shorter.

Well, BDB is free.  That path has worked fine for BlockDriverStates.

Or what I said on IRC, but you know.

> Because the structure is "BdrvDirtyBitmap", there's some confusion when
> we name functions like bdrv_dirty_bitmap_{verb} because it's not clear
> if this is a bdrv function that does something with dirty bitmaps, or if
> it's a "BdrvDirtyBitmap" function that does something with that object.
> 
> I guess that seems like a subtle point, but it's why the naming
> conventions in dirty-bitmap.c are all over the place. I think at one
> point, the idea was that:
> 
> bdrv_{verb}_dirty_bitmap was an action applied to a BDS that happened to
> do something with dirty bitmaps. bdrv_dirty_bitmap_{verb} was an action
> that applied to a "BdrvDirtyBitmap". Crystal clear and not confusing at
> all, right?

I just thought people named their functions whatever they felt like at
the time.

> It'd be nice to have functions that operate on a node be named
> bdrv_dbitmap_foo() and functions that operate on the bitmap structure
> itself named just dbitmap_bar().
> 
> Would it be okay if I named them such a thing, I wonder?
> 
> we have "bitmap" and "hbitmap" already, I could do something like
> "dbitmap" for "dirty bitmap" or some such. Kind of an arbitrary change I
> admit, but I'm itching to do a big spring cleaning in dirty-bitmap.c
> right after this series is done.)

HBitmaps are generally used to track dirty areas, so I’d find this a
misnomer.  BDBitmap would be OK.  The “block” part should be in there
somewhere.

Max



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH] tests/boot_linux_console: add a test for riscv64 + virt

[Cleber Rosa]

On Wed, Jul 10, 2019 at 10:50:23AM -0700, Chih-Min Chao wrote:
> Similar to the mips + malta test, it boots a Linux kernel on a virt
> board and verify the serial is working.  Also, it relies on the serial
> device set by the machine itself.
> 
> If riscv64 is a target being built, "make check-acceptance" will
> automatically include this test by the use of the "arch:riscv64" tags.
> 
> Alternatively, this test can be run using:
> 
>   $ avocado run -t arch:riscv64 tests/acceptance
> 
> Signed-off-by: Chih-Min Chao 
> ---
>  tests/acceptance/boot_linux_console.py | 40 
> ++
>  1 file changed, 40 insertions(+)
> 
> diff --git a/tests/acceptance/boot_linux_console.py 
> b/tests/acceptance/boot_linux_console.py
> index 3215950..bbc6b06 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -354,3 +354,43 @@ class BootLinuxConsole(Test):
>  self.vm.launch()
>  console_pattern = 'Kernel command line: %s' % kernel_command_line
>  self.wait_for_console_pattern(console_pattern)
> +
> +def test_riscv64_virt(self):
> +"""
> +:avocado: tags=arch:riscv64
> +:avocado: tags=machine:virt
> +"""
> +
> +kernel_url = ('https://github.com/chihminchao/test-binary/raw/'
> +  
> '0b7787305d9e40815c05a805266cc74ff356239e/qemu/riscv64/'
> +  'bbl_w_kernel.gz')
> +kernel_hash = 'c7f6cc7967975ad42dc61ee0535db01c9cbd0968'
> +kernel_path_gz = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
> +kernel_path = self.workdir + "bbl_w_kernel"
> +
> +with gzip.open(kernel_path_gz, 'rb') as f_in:
> +with open(kernel_path, 'wb') as f_out:
> +shutil.copyfileobj(f_in, f_out)

There are currently two patterns for extracting a gzipped file
in this test.  So, this is not a must, but maybe you'd prefer:

---
diff --git a/tests/acceptance/boot_linux_console.py 
b/tests/acceptance/boot_linux_console.py
index bbc6b0683f..9f819e20e1 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -365,12 +365,8 @@ class BootLinuxConsole(Test):
   '0b7787305d9e40815c05a805266cc74ff356239e/qemu/riscv64/'
   'bbl_w_kernel.gz')
 kernel_hash = 'c7f6cc7967975ad42dc61ee0535db01c9cbd0968'
-kernel_path_gz = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
-kernel_path = self.workdir + "bbl_w_kernel"
-
-with gzip.open(kernel_path_gz, 'rb') as f_in:
-with open(kernel_path, 'wb') as f_out:
-shutil.copyfileobj(f_in, f_out)
+kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+uncompressed_kernel = archive.uncompress(kernel_path, self.workdir)
 
 initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
   '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/'
@@ -382,7 +378,7 @@ class BootLinuxConsole(Test):
 self.vm.set_console()
 kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
+ 'console=ttyS0 noreboot')
-self.vm.add_args('-kernel', kernel_path,
+self.vm.add_args('-kernel', uncompressed_kernel,
  '-initrd', initrd_path,
  '-append', kernel_command_line)
 self.vm.launch()
---

> +
> +initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
> +  '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/'
> +  'riscv64/rootfs.cpio.gz')
> +initrd_hash = 'f4867d263754961b6f626cdcdc0cb334c47e3b49'
> +initrd_path = self.fetch_asset(initrd_url, asset_hash=initrd_hash)
> +
> +self.vm.set_machine('virt')
> +self.vm.set_console()
> +kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
> +   + 'console=ttyS0 noreboot')
> +self.vm.add_args('-kernel', kernel_path,
> + '-initrd', initrd_path,
> + '-append', kernel_command_line)
> +self.vm.launch()
> +self.wait_for_console_pattern('Boot successful.')
> +
> +self.exec_command_and_wait_for_pattern('cat /proc/cpuinfo',
> +   'isa')
> +self.exec_command_and_wait_for_pattern('uname -a',
> +   'sifive')
> +self.exec_command_and_wait_for_pattern('reboot',
> +   'reboot: Restarting system')
> -- 
> 2.7.4
> 
> 

It'd be nice to also add riscv64 to the target list in .travis.yaml
"acceptance tests" job.

Regards and many thanks for this contribution!
- Cleber.

[Qemu-devel] [PATCH v2 12/13] migration: add support to migrate page encryption bitmap

[Singh, Brijesh]

When memory encryption is enabled, the hypervisor maintains a page
encryption bitmap which is referred by hypervisor during migratoin to check
if page is private or shared. The bitmap is built during the VM bootup and
must be migrated to the target host so that hypervisor on target host can
use it for future migration. The KVM_{SET,GET}_PAGE_ENC_BITMAP can be used
to get and set the bitmap for a given gfn range.

Signed-off-by: Brijesh Singh 
---
 accel/kvm/kvm-all.c  |  4 +++
 migration/ram.c  | 11 +++
 target/i386/sev.c| 67 
 target/i386/trace-events |  2 ++
 4 files changed, 84 insertions(+)

diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 442b1af36e..9e23088a94 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -1831,6 +1831,10 @@ static int kvm_init(MachineState *ms)
 kvm_state->memcrypt_encrypt_data = sev_encrypt_data;
 kvm_state->memcrypt_save_outgoing_page = sev_save_outgoing_page;
 kvm_state->memcrypt_load_incoming_page = sev_load_incoming_page;
+kvm_state->memcrypt_load_incoming_page_enc_bitmap =
+sev_load_incoming_page_enc_bitmap;
+kvm_state->memcrypt_save_outgoing_page_enc_bitmap =
+sev_save_outgoing_page_enc_bitmap;
 }
 
 ret = kvm_arch_init(ms, s);
diff --git a/migration/ram.c b/migration/ram.c
index d179867e1b..3a4bdf3c03 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -78,6 +78,7 @@
 /* 0x80 is reserved in migration.h start with 0x100 next */
 #define RAM_SAVE_FLAG_COMPRESS_PAGE0x100
 #define RAM_SAVE_FLAG_ENCRYPTED_PAGE   0x200
+#define RAM_SAVE_FLAG_PAGE_ENCRYPTED_BITMAP   0x400 /* used in 
target/i386/sev.c */
 
 static inline bool is_zero_range(uint8_t *p, uint64_t size)
 {
@@ -3595,6 +3596,10 @@ static int ram_save_complete(QEMUFile *f, void *opaque)
 flush_compressed_data(rs);
 ram_control_after_iterate(f, RAM_CONTROL_FINISH);
 
+if (kvm_memcrypt_enabled()) {
+ret = kvm_memcrypt_save_outgoing_page_enc_bitmap(f);
+}
+
 rcu_read_unlock();
 
 multifd_send_sync_main();
@@ -4469,6 +4474,12 @@ static int ram_load(QEMUFile *f, void *opaque, int 
version_id)
 ret = -EINVAL;
 }
 break;
+case RAM_SAVE_FLAG_PAGE_ENCRYPTED_BITMAP:
+if (kvm_memcrypt_load_incoming_page_enc_bitmap(f)) {
+error_report("Failed to load page enc bitmap");
+ret = -EINVAL;
+}
+break;
 case RAM_SAVE_FLAG_EOS:
 /* normal exit */
 multifd_recv_sync_main();
diff --git a/target/i386/sev.c b/target/i386/sev.c
index 09a62d6f88..93c6a90806 100644
--- a/target/i386/sev.c
+++ b/target/i386/sev.c
@@ -63,6 +63,7 @@ static const char *const sev_fw_errlist[] = {
 };
 
 #define SEV_FW_MAX_ERROR  ARRAY_SIZE(sev_fw_errlist)
+#define RAM_SAVE_FLAG_PAGE_ENCRYPTED_BITMAP   0x400
 
 static int
 sev_ioctl(int fd, int cmd, void *data, int *error)
@@ -1189,6 +1190,72 @@ int sev_load_incoming_page(void *handle, QEMUFile *f, 
uint8_t *ptr)
 return sev_receive_update_data(f, ptr);
 }
 
+#define ALIGN(x, y)  (((x)+(y)-1) & ~((y)-1))
+
+int sev_load_incoming_page_enc_bitmap(void *handle, QEMUFile *f)
+{
+void *bmap;
+unsigned long npages;
+unsigned long bmap_size, base_gpa;
+struct kvm_page_enc_bitmap e = {};
+
+base_gpa = qemu_get_be64(f);
+npages = qemu_get_be64(f);
+bmap_size = qemu_get_be64(f);
+
+bmap = g_malloc0(bmap_size);
+qemu_get_buffer(f, (uint8_t *)bmap, bmap_size);
+
+trace_kvm_sev_load_page_enc_bitmap(base_gpa, npages << TARGET_PAGE_BITS);
+
+e.start_gfn = base_gpa >> TARGET_PAGE_BITS;
+e.num_pages = npages;
+e.enc_bitmap = bmap;
+if (kvm_vm_ioctl(kvm_state, KVM_SET_PAGE_ENC_BITMAP, ) == -1) {
+error_report("KVM_SET_PAGE_ENC_BITMAP ioctl failed %d", errno);
+g_free(bmap);
+return 1;
+}
+
+g_free(bmap);
+
+return 0;
+}
+
+int sev_save_outgoing_page_enc_bitmap(void *handle, QEMUFile *f,
+  unsigned long start, uint64_t length)
+{
+uint64_t size;
+struct kvm_page_enc_bitmap e = {};
+
+if (!length) {
+return 0;
+}
+
+size = ALIGN((length >> TARGET_PAGE_BITS), /*HOST_LONG_BITS*/ 64) / 8;
+e.enc_bitmap = g_malloc0(size);
+e.start_gfn = start >> TARGET_PAGE_BITS;
+e.num_pages = length >> TARGET_PAGE_BITS;
+
+trace_kvm_sev_save_page_enc_bitmap(start, length);
+
+if (kvm_vm_ioctl(kvm_state, KVM_GET_PAGE_ENC_BITMAP, ) == -1) {
+error_report("%s: KVM_GET_PAGE_ENC_BITMAP ioctl failed %d",
+__func__, errno);
+g_free(e.enc_bitmap);
+return 1;
+}
+
+qemu_put_be64(f, RAM_SAVE_FLAG_PAGE_ENCRYPTED_BITMAP);
+qemu_put_be64(f, start);
+qemu_put_be64(f, e.num_pages);
+qemu_put_be64(f, size);
+qemu_put_buffer(f, (uint8_t *)e.enc_bitmap, size);
+
+

[Qemu-devel] [PATCH v2 11/13] kvm: introduce high-level API to migrate the page encryption bitmap

[Singh, Brijesh]

Encrypted VMs have concept of private and shared memory. The private
memory is encrypted with the guest-specific key, while shared memory
may be encrypted with hyperivosr key. The guest OS uses a hypercall
to notify the page encryption state to the hypervisor. The hypervisor
maintain a bitmap of page encryption state. This bitmap should be
migrated to ensure that target machine can function correctly.

Signed-off-by: Brijesh Singh 
---
 accel/kvm/kvm-all.c| 37 +
 accel/kvm/sev-stub.c   | 11 +++
 accel/stubs/kvm-stub.c | 10 ++
 include/sysemu/kvm.h   | 13 +
 include/sysemu/sev.h   |  3 +++
 5 files changed, 74 insertions(+)

diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 7f94dba6f9..442b1af36e 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -114,6 +114,9 @@ struct KVMState
 uint8_t *ptr, uint32_t sz, uint64_t *bytes_sent);
 int (*memcrypt_load_incoming_page)(void *ehandle, QEMUFile *f,
 uint8_t *ptr);
+int (*memcrypt_load_incoming_page_enc_bitmap)(void *ehandle, QEMUFile *f);
+int (*memcrypt_save_outgoing_page_enc_bitmap)(void *ehandle, QEMUFile *f,
+uint64_t start, uint64_t length);
 };
 
 KVMState *kvm_state;
@@ -192,6 +195,40 @@ int kvm_memcrypt_load_incoming_page(QEMUFile *f, uint8_t 
*ptr)
 return 1;
 }
 
+int kvm_memcrypt_load_incoming_page_enc_bitmap(QEMUFile *f)
+{
+if (kvm_state->memcrypt_handle &&
+kvm_state->memcrypt_load_incoming_page_enc_bitmap) {
+return kvm_state->memcrypt_load_incoming_page_enc_bitmap(
+kvm_state->memcrypt_handle, f);
+}
+
+return 1;
+}
+
+int kvm_memcrypt_save_outgoing_page_enc_bitmap(QEMUFile *f)
+{
+KVMMemoryListener *kml = _state->memory_listener;
+KVMState *s = kvm_state;
+int ret = 1, i;
+
+if (s->memcrypt_handle &&
+s->memcrypt_save_outgoing_page_enc_bitmap) {
+
+/* iterate through all the registered slots and send the bitmap */
+for (i = 0; i < s->nr_slots; i++) {
+KVMSlot *mem = >slots[i];
+ret = s->memcrypt_save_outgoing_page_enc_bitmap(s->memcrypt_handle,
+f, mem->start_addr, mem->memory_size);
+if (ret) {
+return 1;
+}
+}
+}
+
+return ret;
+}
+
 static KVMSlot *kvm_get_free_slot(KVMMemoryListener *kml)
 {
 KVMState *s = kvm_state;
diff --git a/accel/kvm/sev-stub.c b/accel/kvm/sev-stub.c
index c12a8e005e..7acd7211e6 100644
--- a/accel/kvm/sev-stub.c
+++ b/accel/kvm/sev-stub.c
@@ -35,3 +35,14 @@ int sev_load_incoming_page(void *handle, QEMUFile *f, 
uint8_t *ptr)
 {
 return 1;
 }
+
+int sev_load_incoming_page_enc_bitmap(void *handle, QEMUFile *f)
+{
+return 1;
+}
+
+int sev_save_outgoing_page_enc_bitmap(void *handle, QEMUFile *f,
+  uint64_t start, uint64_t length)
+{
+return 1;
+}
diff --git a/accel/stubs/kvm-stub.c b/accel/stubs/kvm-stub.c
index e14b879531..ae607787e7 100644
--- a/accel/stubs/kvm-stub.c
+++ b/accel/stubs/kvm-stub.c
@@ -125,6 +125,16 @@ int kvm_memcrypt_load_incoming_page(QEMUFile *f, uint8_t 
*ptr)
 return 1;
 }
 
+int kvm_memcrypt_load_incoming_page_enc_bitmap(QEMUFile *f)
+{
+return 1;
+}
+
+int kvm_memcrypt_save_outgoing_page_enc_bitmap(QEMUFile *f)
+{
+return 1;
+}
+
 
 #ifndef CONFIG_USER_ONLY
 int kvm_irqchip_add_msi_route(KVMState *s, int vector, PCIDevice *dev)
diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index bb6bcc143c..8aa06b4462 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -260,6 +260,19 @@ int kvm_memcrypt_save_outgoing_page(QEMUFile *f, uint8_t 
*ptr, uint32_t size,
  */
 int kvm_memcrypt_load_incoming_page(QEMUFile *f, uint8_t *ptr);
 
+/**
+ * kvm_memcrypt_load_incoming_page_enc_bitmap: read the page encryption bitmap
+ * from the socket and pass it to the hypervisor.
+ */
+int kvm_memcrypt_load_incoming_page_enc_bitmap(QEMUFile *f);
+
+/**
+ * kvm_memcrypt_save_outgoing_page_enc_bitmap: write the page encryption bitmap
+ * on socket.
+ */
+int kvm_memcrypt_save_outgoing_page_enc_bitmap(QEMUFile *f);
+
+
 #ifdef NEED_CPU_H
 #include "cpu.h"
 
diff --git a/include/sysemu/sev.h b/include/sysemu/sev.h
index 752a71b1c0..e08886ca33 100644
--- a/include/sysemu/sev.h
+++ b/include/sysemu/sev.h
@@ -21,4 +21,7 @@ int sev_encrypt_data(void *handle, uint8_t *ptr, uint64_t 
len);
 int sev_save_outgoing_page(void *handle, QEMUFile *f, uint8_t *ptr,
uint32_t size, uint64_t *bytes_sent);
 int sev_load_incoming_page(void *handle, QEMUFile *f, uint8_t *ptr);
+int sev_load_incoming_page_enc_bitmap(void *handle, QEMUFile *f);
+int sev_save_outgoing_page_enc_bitmap(void *handle, QEMUFile *f,
+  uint64_t start, uint64_t length);
 #endif
-- 
2.17.1

[Qemu-devel] [PATCH v2 10/13] target/i386: sev: add support to load incoming encrypted page

[Singh, Brijesh]

The sev_load_incoming_page() provide the implementation to read the
incoming guest private pages from the socket and load it into the guest
memory. The routines uses the RECEIVE_START command to create the
incoming encryption context on the first call then uses the
RECEIEVE_UPDATE_DATA command to load the encrypted pages into the guest
memory. After migration is completed, we issue the RECEIVE_FINISH command
to transition the SEV guest to the runnable state so that it can be
executed.

Signed-off-by: Brijesh Singh 
---
 accel/kvm/kvm-all.c  |   1 +
 target/i386/sev.c| 126 ++-
 target/i386/trace-events |   3 +
 3 files changed, 129 insertions(+), 1 deletion(-)

diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index a9fb447248..7f94dba6f9 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -1793,6 +1793,7 @@ static int kvm_init(MachineState *ms)
 
 kvm_state->memcrypt_encrypt_data = sev_encrypt_data;
 kvm_state->memcrypt_save_outgoing_page = sev_save_outgoing_page;
+kvm_state->memcrypt_load_incoming_page = sev_load_incoming_page;
 }
 
 ret = kvm_arch_init(ms, s);
diff --git a/target/i386/sev.c b/target/i386/sev.c
index 28b36c8035..09a62d6f88 100644
--- a/target/i386/sev.c
+++ b/target/i386/sev.c
@@ -708,13 +708,34 @@ sev_launch_finish(SEVState *s)
 }
 }
 
+static int
+sev_receive_finish(SEVState *s)
+{
+int error, ret = 1;
+
+trace_kvm_sev_receive_finish();
+ret = sev_ioctl(s->sev_fd, KVM_SEV_RECEIVE_FINISH, 0, );
+if (ret) {
+error_report("%s: RECEIVE_FINISH ret=%d fw_error=%d '%s'",
+__func__, ret, error, fw_error_to_str(error));
+goto err;
+}
+
+sev_set_guest_state(SEV_STATE_RUNNING);
+err:
+return ret;
+}
+
+
 static void
 sev_vm_state_change(void *opaque, int running, RunState state)
 {
 SEVState *s = opaque;
 
 if (running) {
-if (!sev_check_state(SEV_STATE_RUNNING)) {
+if (sev_check_state(SEV_STATE_RECEIVE_UPDATE)) {
+sev_receive_finish(s);
+} else if (!sev_check_state(SEV_STATE_RUNNING)) {
 sev_launch_finish(s);
 }
 }
@@ -1065,6 +1086,109 @@ int sev_save_outgoing_page(void *handle, QEMUFile *f, 
uint8_t *ptr,
 return sev_send_update_data(s, f, ptr, sz, bytes_sent);
 }
 
+static int
+sev_receive_start(QSevGuestInfo *sev, QEMUFile *f)
+{
+int ret = 1;
+int fw_error;
+struct kvm_sev_receive_start *start;
+gchar *session = NULL, *pdh_cert = NULL;
+
+start = g_new0(struct kvm_sev_receive_start, 1);
+
+/* get SEV guest handle */
+start->handle = object_property_get_int(OBJECT(sev), "handle",
+_abort);
+
+/* get the source policy */
+start->policy = qemu_get_be32(f);
+
+/* get source PDH key */
+start->pdh_len = qemu_get_be32(f);
+pdh_cert = g_new(gchar, start->pdh_len);
+qemu_get_buffer(f, (uint8_t *)pdh_cert, start->pdh_len);
+start->pdh_uaddr = (unsigned long)pdh_cert;
+
+/* get source session data */
+start->session_len = qemu_get_be32(f);
+session = g_new(gchar, start->session_len);
+qemu_get_buffer(f, (uint8_t *)session, start->session_len);
+start->session_uaddr = (unsigned long)session;
+
+trace_kvm_sev_receive_start(start->policy, session, pdh_cert);
+
+ret = sev_ioctl(sev_state->sev_fd, KVM_SEV_RECEIVE_START, start, 
_error);
+if (ret < 0) {
+error_report("Error RECEIVE_START ret=%d fw_error=%d '%s'",
+ret, fw_error, fw_error_to_str(fw_error));
+goto err;
+}
+
+object_property_set_int(OBJECT(sev), start->handle, "handle", 
_abort);
+sev_set_guest_state(SEV_STATE_RECEIVE_UPDATE);
+err:
+g_free(start);
+g_free(session);
+g_free(pdh_cert);
+
+return ret;
+}
+
+static int sev_receive_update_data(QEMUFile *f, uint8_t *ptr)
+{
+int ret = 1, fw_error = 0;
+gchar *hdr = NULL, *trans = NULL;
+struct kvm_sev_receive_update_data *update;
+
+update = g_new0(struct kvm_sev_receive_update_data, 1);
+
+/* get packet header */
+update->hdr_len = qemu_get_be32(f);
+hdr = g_new(gchar, update->hdr_len);
+qemu_get_buffer(f, (uint8_t *)hdr, update->hdr_len);
+update->hdr_uaddr = (unsigned long)hdr;
+
+/* get transport buffer */
+update->trans_len = qemu_get_be32(f);
+trans = g_new(gchar, update->trans_len);
+update->trans_uaddr = (unsigned long)trans;
+qemu_get_buffer(f, (uint8_t *)update->trans_uaddr, update->trans_len);
+
+update->guest_uaddr = (unsigned long) ptr;
+update->guest_len = update->trans_len;
+
+trace_kvm_sev_receive_update_data(trans, ptr, update->guest_len,
+hdr, update->hdr_len);
+
+ret = sev_ioctl(sev_state->sev_fd, KVM_SEV_RECEIVE_UPDATE_DATA,
+update, _error);
+if (ret) {
+error_report("Error RECEIVE_UPDATE_DATA ret=%d fw_error=%d '%s'",
+ret, fw_error, fw_error_to_str(fw_error));
+  

Re: [Qemu-devel] [PATCH 6/8] block/backup: issue progress updates for skipped regions

[Max Reitz]

On 10.07.19 20:20, John Snow wrote:
> 
> 
> On 7/10/19 12:36 PM, Max Reitz wrote:
>> On 10.07.19 03:05, John Snow wrote:
>>> The way bitmap backups work is by starting at 75% if it needs
>>> to copy just 25% of the disk.
>>
>> Although there is this comment:
>>
>>> /* TODO job_progress_set_remaining() would make more sense */
>>
>> So...
>>
>>> The way sync=top currently works, however, is to start at 0% and then
>>> never update the progress if it doesn't copy a region. If it needs to
>>> copy 25% of the disk, we'll finish at 25%.
>>>
>>> Update the progress when we skip regions.
>>
>> Wouldn’t it be more correct to decrease the job length?
>>
>> Max
>>
> 
> Admittedly I have precisely no idea. Maybe? As far as I understand it,
> we guarantee only:
> 
> (1) Progress monotonically increases
> (2) Upon completion, progress will equal the total work estimate.
> [Trying to fix that to be true here.]
> 
> This means we can do stuff like:
> 
> - Total work estimate can increase or decrease arbitrarily
> - Neither value has to mean anything in particular
> 
> 
> Bitmap sync works by artificially increasing progress for NOP regions,
> seen in init_copy_bitmap.

Yes, and it has a TODO comment that says it should be done differently.

> Full sync also tends to increase progress regardless of it actually did
> a copy or not; offload support also counts as progress here. So if you
> full sync an empty image, you'll see it increasing the progress as it
> doesn't actually do anything.
> 
> Top sync is the odd one out, which just omits progress for regions it skips.
> 
> My only motivation here was to make them consistent. Can I do it the
> other way? Yeah, probably. Is one way better than the other? I
> legitimately have no idea. I guess whoever wrote the last comment felt
> that it should all be the other way instead. Why'd they not do that?

If you look at the commit (05df8a6a2b4), I suppose it was because that
commit simply did not intend to change behavior.  It just touched that
piece of code and noted that maybe there should be a follow-up commit to
change it.

But yeah, whatever.

Reviewed-by: Max Reitz 



signature.asc
Description: OpenPGP digital signature

[Qemu-devel] [PATCH v2 08/13] misc.json: add migrate-set-sev-info command

[Singh, Brijesh]

The command can be used by the hypervisor to specify the target Platform
Diffie-Hellman key (PDH) and certificate chain before starting the SEV
guest migration. The values passed through the command will be used while
creating the outgoing encryption context.

Signed-off-by: Brijesh Singh 
---
 qapi/misc-target.json  | 18 ++
 target/i386/monitor.c  | 10 ++
 target/i386/sev-stub.c |  5 +
 target/i386/sev.c  | 11 +++
 target/i386/sev_i386.h |  9 -
 5 files changed, 52 insertions(+), 1 deletion(-)

diff --git a/qapi/misc-target.json b/qapi/misc-target.json
index a00fd821eb..938dcaea14 100644
--- a/qapi/misc-target.json
+++ b/qapi/misc-target.json
@@ -266,3 +266,21 @@
 ##
 { 'command': 'query-gic-capabilities', 'returns': ['GICCapability'],
   'if': 'defined(TARGET_ARM)' }
+
+##
+# @migrate-set-sev-info:
+#
+# The command is used to provide the target host information used during the
+# SEV guest.
+#
+# @pdh the target host platform diffie-hellman key encoded in base64
+#
+# @plat-cert the target host platform certificate chain encoded in base64
+#
+# @amd-cert AMD certificate chain which include ASK and OCA encoded in base64
+#
+# Since 4.2
+#
+##
+{ 'command': 'migrate-set-sev-info',
+  'data': { 'pdh': 'str', 'plat-cert': 'str', 'amd-cert' : 'str' }}
diff --git a/target/i386/monitor.c b/target/i386/monitor.c
index 1f3b532fc2..4a5f50fb45 100644
--- a/target/i386/monitor.c
+++ b/target/i386/monitor.c
@@ -736,3 +736,13 @@ SevCapability *qmp_query_sev_capabilities(Error **errp)
 
 return data;
 }
+
+void qmp_migrate_set_sev_info(const char *pdh, const char *plat_cert,
+  const char *amd_cert, Error **errp)
+{
+if (sev_enabled()) {
+sev_set_migrate_info(pdh, plat_cert, amd_cert);
+} else {
+error_setg(errp, "SEV is not enabled");
+}
+}
diff --git a/target/i386/sev-stub.c b/target/i386/sev-stub.c
index e5ee13309c..173bfa6374 100644
--- a/target/i386/sev-stub.c
+++ b/target/i386/sev-stub.c
@@ -48,3 +48,8 @@ SevCapability *sev_get_capabilities(void)
 {
 return NULL;
 }
+
+void sev_set_migrate_info(const char *pdh, const char *plat_cert,
+  const char *amd_cert)
+{
+}
diff --git a/target/i386/sev.c b/target/i386/sev.c
index 49baf8fef0..6c902d0be8 100644
--- a/target/i386/sev.c
+++ b/target/i386/sev.c
@@ -825,6 +825,17 @@ sev_encrypt_data(void *handle, uint8_t *ptr, uint64_t len)
 return 0;
 }
 
+void sev_set_migrate_info(const char *pdh, const char *plat_cert,
+  const char *amd_cert)
+{
+SEVState *s = sev_state;
+
+s->remote_pdh = g_base64_decode(pdh, >remote_pdh_len);
+s->remote_plat_cert = g_base64_decode(plat_cert,
+  >remote_plat_cert_len);
+s->amd_cert = g_base64_decode(amd_cert, >amd_cert_len);
+}
+
 static void
 sev_register_types(void)
 {
diff --git a/target/i386/sev_i386.h b/target/i386/sev_i386.h
index 55313441ae..3f3449b346 100644
--- a/target/i386/sev_i386.h
+++ b/target/i386/sev_i386.h
@@ -39,7 +39,8 @@ extern uint32_t sev_get_cbit_position(void);
 extern uint32_t sev_get_reduced_phys_bits(void);
 extern char *sev_get_launch_measurement(void);
 extern SevCapability *sev_get_capabilities(void);
-
+extern void sev_set_migrate_info(const char *pdh, const char *plat_cert,
+ const char *amd_cert);
 typedef struct QSevGuestInfo QSevGuestInfo;
 typedef struct QSevGuestInfoClass QSevGuestInfoClass;
 
@@ -81,6 +82,12 @@ struct SEVState {
 int sev_fd;
 SevState state;
 gchar *measurement;
+guchar *remote_pdh;
+size_t remote_pdh_len;
+guchar *remote_plat_cert;
+size_t remote_plat_cert_len;
+guchar *amd_cert;
+size_t amd_cert_len;
 };
 
 typedef struct SEVState SEVState;
-- 
2.17.1

[Qemu-devel] [PATCH v2 04/13] kvm: add support to sync the page encryption state bitmap

[Singh, Brijesh]

The SEV VMs have concept of private and shared memory. The private memory
is encrypted with guest-specific key, while shared memory may be encrypted
with hyperivosr key. The KVM_GET_PAGE_ENC_BITMAP can be used to get a
bitmap indicating whether the guest page is private or shared. A private
page must be transmitted using the SEV migration commands.

Add a cpu_physical_memory_sync_encrypted_bitmap() which can be used to sync
the page encryption bitmap for a given memory region.

Signed-off-by: Brijesh Singh 
---
 accel/kvm/kvm-all.c |  38 ++
 include/exec/ram_addr.h | 161 ++--
 include/exec/ramlist.h  |   3 +-
 migration/ram.c |  28 ++-
 4 files changed, 222 insertions(+), 8 deletions(-)

diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 162a2d5085..c935e9366c 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -504,6 +504,37 @@ static int 
kvm_get_dirty_pages_log_range(MemoryRegionSection *section,
 
 #define ALIGN(x, y)  (((x)+(y)-1) & ~((y)-1))
 
+/* sync page_enc bitmap */
+static int kvm_sync_page_enc_bitmap(KVMMemoryListener *kml,
+MemoryRegionSection *section,
+KVMSlot *mem)
+{
+unsigned long size;
+KVMState *s = kvm_state;
+struct kvm_page_enc_bitmap e = {};
+ram_addr_t pages = int128_get64(section->size) / getpagesize();
+ram_addr_t start = section->offset_within_region +
+   memory_region_get_ram_addr(section->mr);
+
+size = ALIGN(((mem->memory_size) >> TARGET_PAGE_BITS),
+ /*HOST_LONG_BITS*/ 64) / 8;
+e.enc_bitmap = g_malloc0(size);
+e.start_gfn = mem->start_addr >> TARGET_PAGE_BITS;
+e.num_pages = pages;
+if (kvm_vm_ioctl(s, KVM_GET_PAGE_ENC_BITMAP, ) == -1) {
+DPRINTF("KVM_GET_PAGE_ENC_BITMAP ioctl failed %d\n", errno);
+g_free(e.enc_bitmap);
+return 1;
+}
+
+cpu_physical_memory_set_encrypted_lebitmap(e.enc_bitmap,
+   start, pages);
+
+g_free(e.enc_bitmap);
+
+return 0;
+}
+
 /**
  * kvm_physical_sync_dirty_bitmap - Grab dirty bitmap from kernel space
  * This function updates qemu's dirty bitmap using
@@ -553,6 +584,13 @@ static int 
kvm_physical_sync_dirty_bitmap(KVMMemoryListener *kml,
 }
 
 kvm_get_dirty_pages_log_range(section, d.dirty_bitmap);
+
+if (kvm_memcrypt_enabled() &&
+kvm_sync_page_enc_bitmap(kml, section, mem)) {
+g_free(d.dirty_bitmap);
+return -1;
+}
+
 g_free(d.dirty_bitmap);
 }
 
diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h
index f96777bb99..6fc6864194 100644
--- a/include/exec/ram_addr.h
+++ b/include/exec/ram_addr.h
@@ -51,6 +51,8 @@ struct RAMBlock {
 unsigned long *unsentmap;
 /* bitmap of already received pages in postcopy */
 unsigned long *receivedmap;
+/* bitmap of page encryption state for an encrypted guest */
+unsigned long *encbmap;
 };
 
 static inline bool offset_in_ramblock(RAMBlock *b, ram_addr_t offset)
@@ -314,9 +316,41 @@ static inline void 
cpu_physical_memory_set_dirty_range(ram_addr_t start,
 }
 
 #if !defined(_WIN32)
-static inline void cpu_physical_memory_set_dirty_lebitmap(unsigned long 
*bitmap,
+
+static inline void cpu_physical_memory_set_encrypted_range(ram_addr_t start,
+   ram_addr_t length,
+   unsigned long val)
+{
+unsigned long end, page;
+unsigned long * const *src;
+
+if (length == 0) {
+return;
+}
+
+end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
+page = start >> TARGET_PAGE_BITS;
+
+rcu_read_lock();
+
+src = 
atomic_rcu_read(_list.dirty_memory[DIRTY_MEMORY_ENCRYPTED])->blocks;
+
+while (page < end) {
+unsigned long idx = page / DIRTY_MEMORY_BLOCK_SIZE;
+unsigned long offset = page % DIRTY_MEMORY_BLOCK_SIZE;
+unsigned long num = MIN(end - page, DIRTY_MEMORY_BLOCK_SIZE - offset);
+
+atomic_xchg([idx][BIT_WORD(offset)], val);
+page += num;
+}
+
+rcu_read_unlock();
+}
+
+static inline void cpu_physical_memory_set_dirty_enc_lebitmap(unsigned long 
*bitmap,
   ram_addr_t start,
-  ram_addr_t pages)
+  ram_addr_t pages,
+  bool enc_map)
 {
 unsigned long i, j;
 unsigned long page_number, c;
@@ -349,10 +383,14 @@ static inline void 
cpu_physical_memory_set_dirty_lebitmap(unsigned long *bitmap,
 if (bitmap[k]) {
 unsigned long temp = leul_to_cpu(bitmap[k]);
 
-atomic_or([DIRTY_MEMORY_MIGRATION][idx][offset], temp);
-

[Qemu-devel] [PATCH v2 05/13] doc: update AMD SEV API spec web link

[Singh, Brijesh]

Signed-off-by: Brijesh Singh 
---
 docs/amd-memory-encryption.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/amd-memory-encryption.txt b/docs/amd-memory-encryption.txt
index 43bf3ee6a5..abb9a976f5 100644
--- a/docs/amd-memory-encryption.txt
+++ b/docs/amd-memory-encryption.txt
@@ -98,7 +98,7 @@ AMD Memory Encryption whitepaper:
 
http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2013/12/AMD_Memory_Encryption_Whitepaper_v7-Public.pdf
 
 Secure Encrypted Virtualization Key Management:
-[1] http://support.amd.com/TechDocs/55766_SEV-KM API_Specification.pdf
+[1] https://developer.amd.com/sev/ (Secure Encrypted Virtualization API)
 
 KVM Forum slides:
 
http://www.linux-kvm.org/images/7/74/02x08A-Thomas_Lendacky-AMDs_Virtualizatoin_Memory_Encryption_Technology.pdf
-- 
2.17.1

[Qemu-devel] [PATCH v2 06/13] doc: update AMD SEV to include Live migration flow

[Singh, Brijesh]

Signed-off-by: Brijesh Singh 
---
 docs/amd-memory-encryption.txt | 42 +-
 1 file changed, 41 insertions(+), 1 deletion(-)

diff --git a/docs/amd-memory-encryption.txt b/docs/amd-memory-encryption.txt
index abb9a976f5..374f4b0a94 100644
--- a/docs/amd-memory-encryption.txt
+++ b/docs/amd-memory-encryption.txt
@@ -89,7 +89,47 @@ TODO
 
 Live Migration
 
-TODO
+AMD SEV encrypts the memory of VMs and because a different key is used
+in each VM, the hypervisor will be unable to simply copy the
+ciphertext from one VM to another to migrate the VM. Instead the AMD SEV Key
+Management API provides sets of function which the hypervisor can use
+to package a guest page for migration, while maintaining the confidentiality
+provided by AMD SEV.
+
+SEV guest VMs have the concept of private and shared memory. The private
+memory is encrypted with the guest-specific key, while shared memory may
+be encrypted with the hypervisor key. The migration APIs provided by the
+SEV API spec should be used for migrating the private pages. The
+KVM_GET_PAGE_ENC_BITMAP ioctl can be used to get the guest page encryption
+bitmap. The bitmap can be used to check if the given guest page is
+private or shared.
+
+Before initiating the migration, we need to know the targets machine's public
+Diffie-Hellman key (PDH) and certificate chain. It can be retrieved
+with the 'query-sev-capabilities' QMP command or using the sev-tool. The
+migrate-set-sev-info object can be used to pass the target machine's PDH and
+certificate chain.
+
+e.g
+(QMP) migrate-sev-set-info pdh= plat-cert= \
+   amd-cert=
+(QMP) migrate tcp:0:
+
+
+During the migration flow, the SEND_START is called on the source hypervisor
+to create outgoing encryption context. The SEV guest policy dectates whether
+the certificate passed through the migrate-sev-set-info command will be
+validate. SEND_UPDATE_DATA is called to encrypt the guest private pages.
+After migration is completed, SEND_FINISH is called to destroy the encryption
+context and make the VM non-runnable to protect it against the cloning.
+
+On the target machine, RECEIVE_START is called first to create an
+incoming encryption context. The RECEIVE_UPDATE_DATA is called to copy
+the receieved encrypted page into guest memory. After migration has
+completed, RECEIVE_FINISH is called to make the VM runnable.
+
+For more information about the migration see SEV API Appendix A
+Usage flow (Live migration section).
 
 References
 -
-- 
2.17.1

[Qemu-devel] [PATCH v2 13/13] target/i386: sev: remove migration blocker

[Singh, Brijesh]

Signed-off-by: Brijesh Singh 
---
 target/i386/sev.c | 12 
 1 file changed, 12 deletions(-)

diff --git a/target/i386/sev.c b/target/i386/sev.c
index 93c6a90806..48336515a2 100644
--- a/target/i386/sev.c
+++ b/target/i386/sev.c
@@ -34,7 +34,6 @@
 #define DEFAULT_SEV_DEVICE  "/dev/sev"
 
 static SEVState *sev_state;
-static Error *sev_mig_blocker;
 
 static const char *const sev_fw_errlist[] = {
 "",
@@ -686,7 +685,6 @@ static void
 sev_launch_finish(SEVState *s)
 {
 int ret, error;
-Error *local_err = NULL;
 
 trace_kvm_sev_launch_finish();
 ret = sev_ioctl(sev_state->sev_fd, KVM_SEV_LAUNCH_FINISH, 0, );
@@ -697,16 +695,6 @@ sev_launch_finish(SEVState *s)
 }
 
 sev_set_guest_state(SEV_STATE_RUNNING);
-
-/* add migration blocker */
-error_setg(_mig_blocker,
-   "SEV: Migration is not implemented");
-ret = migrate_add_blocker(sev_mig_blocker, _err);
-if (local_err) {
-error_report_err(local_err);
-error_free(sev_mig_blocker);
-exit(1);
-}
 }
 
 static int
-- 
2.17.1

[Qemu-devel] [PATCH v2 09/13] target/i386: sev: add support to encrypt the outgoing page

[Singh, Brijesh]

The sev_save_outgoing_page() provide the implementation to encrypt the
guest private pages during the transit. The routines uses the SEND_START
command to create the outgoing encryption context on the first call then
uses the SEND_UPDATE_DATA command to encrypt the data before writing it
to the socket. While encrypting the data SEND_UPDATE_DATA produces some
metadata (e.g MAC, IV). The metadata is also sent to the target machine.
After migration is completed, we issue the SEND_FINISH command to transition
the SEV guest state from sending to unrunnable state.

Signed-off-by: Brijesh Singh 
---
 accel/kvm/kvm-all.c  |   1 +
 target/i386/sev.c| 229 +++
 target/i386/sev_i386.h   |   2 +
 target/i386/trace-events |   3 +
 4 files changed, 235 insertions(+)

diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index c935e9366c..a9fb447248 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -1792,6 +1792,7 @@ static int kvm_init(MachineState *ms)
 }
 
 kvm_state->memcrypt_encrypt_data = sev_encrypt_data;
+kvm_state->memcrypt_save_outgoing_page = sev_save_outgoing_page;
 }
 
 ret = kvm_arch_init(ms, s);
diff --git a/target/i386/sev.c b/target/i386/sev.c
index 6c902d0be8..28b36c8035 100644
--- a/target/i386/sev.c
+++ b/target/i386/sev.c
@@ -27,6 +27,8 @@
 #include "sysemu/sysemu.h"
 #include "trace.h"
 #include "migration/blocker.h"
+#include "migration/qemu-file.h"
+#include "migration/misc.h"
 
 #define DEFAULT_GUEST_POLICY0x1 /* disable debug */
 #define DEFAULT_SEV_DEVICE  "/dev/sev"
@@ -718,6 +720,39 @@ sev_vm_state_change(void *opaque, int running, RunState 
state)
 }
 }
 
+static void
+sev_send_finish(void)
+{
+int ret, error;
+
+trace_kvm_sev_send_finish();
+ret = sev_ioctl(sev_state->sev_fd, KVM_SEV_SEND_FINISH, 0, );
+if (ret) {
+error_report("%s: LAUNCH_FINISH ret=%d fw_error=%d '%s'",
+ __func__, ret, error, fw_error_to_str(error));
+}
+
+sev_set_guest_state(SEV_STATE_RUNNING);
+}
+
+static void
+sev_migration_state_notifier(Notifier *notifier, void *data)
+{
+MigrationState *s = data;
+
+if (migration_has_finished(s) ||
+migration_in_postcopy_after_devices(s) ||
+migration_has_failed(s)) {
+if (sev_check_state(SEV_STATE_SEND_UPDATE)) {
+sev_send_finish();
+}
+}
+}
+
+static Notifier sev_migration_state_notify = {
+.notify = sev_migration_state_notifier,
+};
+
 void *
 sev_guest_init(const char *id)
 {
@@ -804,6 +839,7 @@ sev_guest_init(const char *id)
 ram_block_notifier_add(_ram_notifier);
 qemu_add_machine_init_done_notifier(_machine_done_notify);
 qemu_add_vm_change_state_handler(sev_vm_state_change, s);
+add_migration_state_change_notifier(_migration_state_notify);
 
 return s;
 err:
@@ -836,6 +872,199 @@ void sev_set_migrate_info(const char *pdh, const char 
*plat_cert,
 s->amd_cert = g_base64_decode(amd_cert, >amd_cert_len);
 }
 
+static int
+sev_get_send_session_length(void)
+{
+int ret, fw_err = 0;
+struct kvm_sev_send_start *start;
+
+start = g_new0(struct kvm_sev_send_start, 1);
+
+ret = sev_ioctl(sev_state->sev_fd, KVM_SEV_SEND_START, start, _err);
+if (fw_err != SEV_RET_INVALID_LEN) {
+ret = -1;
+error_report("%s: failed to get session length ret=%d fw_error=%d 
'%s'",
+ __func__, ret, fw_err, fw_error_to_str(fw_err));
+goto err;
+}
+
+ret = start->session_len;
+err:
+g_free(start);
+return ret;
+}
+
+static int
+sev_send_start(SEVState *s, QEMUFile *f, uint64_t *bytes_sent)
+{
+gsize pdh_len = 0, plat_cert_len;
+int session_len, ret, fw_error;
+struct kvm_sev_send_start *start;
+guchar *pdh = NULL, *plat_cert = NULL, *session = NULL;
+
+if (!s->remote_pdh || !s->remote_plat_cert) {
+error_report("%s: missing remote PDH or PLAT_CERT", __func__);
+return 1;
+}
+
+start = g_new0(struct kvm_sev_send_start, 1);
+
+start->pdh_cert_uaddr = (unsigned long) s->remote_pdh;
+start->pdh_cert_len = s->remote_pdh_len;
+
+start->plat_cert_uaddr = (unsigned long)s->remote_plat_cert;
+start->plat_cert_len = s->remote_plat_cert_len;
+
+start->amd_cert_uaddr = (unsigned long)s->amd_cert;
+start->amd_cert_len = s->amd_cert_len;
+
+/* get the session length */
+session_len = sev_get_send_session_length();
+if (session_len < 0) {
+ret = 1;
+goto err;
+}
+
+session = g_new0(guchar, session_len);
+start->session_uaddr = (unsigned long)session;
+start->session_len = session_len;
+
+/* Get our PDH certificate */
+ret = sev_get_pdh_info(s->sev_fd, , _len,
+   _cert, _cert_len);
+if (ret) {
+error_report("Failed to get our PDH cert");
+goto err;
+}
+
+trace_kvm_sev_send_start(start->pdh_cert_uaddr, start->pdh_cert_len,
+  

[Qemu-devel] [PATCH v2 01/13] linux-headers: update kernel header to include SEV migration commands

[Singh, Brijesh]

Signed-off-by: Brijesh Singh 
---
 linux-headers/linux/kvm.h | 53 +++
 1 file changed, 53 insertions(+)

diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
index c8423e760c..2b0a2a97b8 100644
--- a/linux-headers/linux/kvm.h
+++ b/linux-headers/linux/kvm.h
@@ -492,6 +492,16 @@ struct kvm_dirty_log {
};
 };
 
+/* for KVM_GET_PAGE_ENC_BITMAP */
+struct kvm_page_enc_bitmap {
+__u64 start_gfn;
+__u64 num_pages;
+   union {
+   void *enc_bitmap; /* one bit per page */
+   __u64 padding2;
+   };
+};
+
 /* for KVM_CLEAR_DIRTY_LOG */
 struct kvm_clear_dirty_log {
__u32 slot;
@@ -1451,6 +1461,9 @@ struct kvm_enc_region {
 /* Available with KVM_CAP_ARM_SVE */
 #define KVM_ARM_VCPU_FINALIZE_IOW(KVMIO,  0xc2, int)
 
+#define KVM_GET_PAGE_ENC_BITMAP _IOW(KVMIO, 0xc2, struct 
kvm_page_enc_bitmap)
+#define KVM_SET_PAGE_ENC_BITMAP _IOW(KVMIO, 0xc3, struct 
kvm_page_enc_bitmap)
+
 /* Secure Encrypted Virtualization command */
 enum sev_cmd_id {
/* Guest initialization commands */
@@ -1531,6 +1544,46 @@ struct kvm_sev_dbg {
__u32 len;
 };
 
+struct kvm_sev_send_start {
+   __u32 policy;
+   __u64 pdh_cert_uaddr;
+   __u32 pdh_cert_len;
+   __u64 plat_cert_uaddr;
+   __u32 plat_cert_len;
+   __u64 amd_cert_uaddr;
+   __u32 amd_cert_len;
+   __u64 session_uaddr;
+   __u32 session_len;
+};
+
+struct kvm_sev_send_update_data {
+   __u64 hdr_uaddr;
+   __u32 hdr_len;
+   __u64 guest_uaddr;
+   __u32 guest_len;
+   __u64 trans_uaddr;
+   __u32 trans_len;
+};
+
+struct kvm_sev_receive_start {
+   __u32 handle;
+   __u32 policy;
+   __u64 pdh_uaddr;
+   __u32 pdh_len;
+   __u64 session_uaddr;
+   __u32 session_len;
+};
+
+struct kvm_sev_receive_update_data {
+   __u64 hdr_uaddr;
+   __u32 hdr_len;
+   __u64 guest_uaddr;
+   __u32 guest_len;
+   __u64 trans_uaddr;
+   __u32 trans_len;
+};
+
+
 #define KVM_DEV_ASSIGN_ENABLE_IOMMU(1 << 0)
 #define KVM_DEV_ASSIGN_PCI_2_3 (1 << 1)
 #define KVM_DEV_ASSIGN_MASK_INTX   (1 << 2)
-- 
2.17.1

[Qemu-devel] [PATCH v2 07/13] target/i386: sev: do not create launch context for an incoming guest

[Singh, Brijesh]

The LAUNCH_START is used for creating an encryption context to encrypt
newly created guest, for an incoming guest the RECEIVE_START should be
used.

Signed-off-by: Brijesh Singh 
---
 target/i386/sev.c | 14 ++
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/target/i386/sev.c b/target/i386/sev.c
index 6dbdc3cdf1..49baf8fef0 100644
--- a/target/i386/sev.c
+++ b/target/i386/sev.c
@@ -789,10 +789,16 @@ sev_guest_init(const char *id)
 goto err;
 }
 
-ret = sev_launch_start(s);
-if (ret) {
-error_report("%s: failed to create encryption context", __func__);
-goto err;
+/*
+ * The LAUNCH context is used for new guest, if its an incoming guest
+ * then RECEIVE context will be created after the connection is 
established.
+ */
+if (!runstate_check(RUN_STATE_INMIGRATE)) {
+ret = sev_launch_start(s);
+if (ret) {
+error_report("%s: failed to create encryption context", __func__);
+goto err;
+}
 }
 
 ram_block_notifier_add(_ram_notifier);
-- 
2.17.1

[Qemu-devel] [PATCH v2 03/13] migration/ram: add support to send encrypted pages

[Singh, Brijesh]

When memory encryption is enabled, the guest memory will be encrypted with
the guest specific key. The patch introduces RAM_SAVE_FLAG_ENCRYPTED_PAGE
flag to distinguish the encrypted data from plaintext. Encrypted pages
may need special handling. The kvm_memcrypt_save_outgoing_page() is used
by the sender to write the encrypted pages onto the socket, similarly the
kvm_memcrypt_load_incoming_page() is used by the target to read the
encrypted pages from the socket and load into the guest memory.

Signed-off-by: Brijesh Singh 
---
 migration/ram.c | 54 -
 1 file changed, 53 insertions(+), 1 deletion(-)

diff --git a/migration/ram.c b/migration/ram.c
index 908517fc2b..3c8977d508 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -57,6 +57,7 @@
 #include "qemu/uuid.h"
 #include "savevm.h"
 #include "qemu/iov.h"
+#include "sysemu/kvm.h"
 
 /***/
 /* ram save/restore */
@@ -76,6 +77,7 @@
 #define RAM_SAVE_FLAG_XBZRLE   0x40
 /* 0x80 is reserved in migration.h start with 0x100 next */
 #define RAM_SAVE_FLAG_COMPRESS_PAGE0x100
+#define RAM_SAVE_FLAG_ENCRYPTED_PAGE   0x200
 
 static inline bool is_zero_range(uint8_t *p, uint64_t size)
 {
@@ -460,6 +462,9 @@ static QemuCond decomp_done_cond;
 static bool do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock 
*block,
  ram_addr_t offset, uint8_t *source_buf);
 
+static int ram_save_encrypted_page(RAMState *rs, PageSearchStatus *pss,
+   bool last_stage);
+
 static void *do_data_compress(void *opaque)
 {
 CompressParam *param = opaque;
@@ -2006,6 +2011,36 @@ static int ram_save_multifd_page(RAMState *rs, RAMBlock 
*block,
 return 1;
 }
 
+/**
+ * ram_save_encrypted_page - send the given encrypted page to the stream
+ */
+static int ram_save_encrypted_page(RAMState *rs, PageSearchStatus *pss,
+   bool last_stage)
+{
+int ret;
+uint8_t *p;
+RAMBlock *block = pss->block;
+ram_addr_t offset = pss->page << TARGET_PAGE_BITS;
+uint64_t bytes_xmit;
+
+p = block->host + offset;
+
+ram_counters.transferred +=
+save_page_header(rs, rs->f, block,
+offset | RAM_SAVE_FLAG_ENCRYPTED_PAGE);
+
+ret = kvm_memcrypt_save_outgoing_page(rs->f, p,
+TARGET_PAGE_SIZE, _xmit);
+if (ret) {
+return -1;
+}
+
+ram_counters.transferred += bytes_xmit;
+ram_counters.normal++;
+
+return 1;
+}
+
 static bool do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock 
*block,
  ram_addr_t offset, uint8_t *source_buf)
 {
@@ -2450,6 +2485,16 @@ static int ram_save_target_page(RAMState *rs, 
PageSearchStatus *pss,
 return res;
 }
 
+/*
+ * If memory encryption is enabled then use memory encryption APIs
+ * to write the outgoing buffer to the wire. The encryption APIs
+ * will take care of accessing the guest memory and re-encrypt it
+ * for the transport purposes.
+ */
+ if (kvm_memcrypt_enabled()) {
+return ram_save_encrypted_page(rs, pss, last_stage);
+ }
+
 if (save_compress_page(rs, block, offset)) {
 return 1;
 }
@@ -4271,7 +4316,8 @@ static int ram_load(QEMUFile *f, void *opaque, int 
version_id)
 }
 
 if (flags & (RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE |
- RAM_SAVE_FLAG_COMPRESS_PAGE | RAM_SAVE_FLAG_XBZRLE)) {
+ RAM_SAVE_FLAG_COMPRESS_PAGE | RAM_SAVE_FLAG_XBZRLE |
+ RAM_SAVE_FLAG_ENCRYPTED_PAGE)) {
 RAMBlock *block = ram_block_from_stream(f, flags);
 
 /*
@@ -4391,6 +4437,12 @@ static int ram_load(QEMUFile *f, void *opaque, int 
version_id)
 break;
 }
 break;
+case RAM_SAVE_FLAG_ENCRYPTED_PAGE:
+if (kvm_memcrypt_load_incoming_page(f, host)) {
+error_report("Failed to encrypted incoming data");
+ret = -EINVAL;
+}
+break;
 case RAM_SAVE_FLAG_EOS:
 /* normal exit */
 multifd_recv_sync_main();
-- 
2.17.1

[Qemu-devel] [PATCH v2 02/13] kvm: introduce high-level API to support encrypted page migration

[Singh, Brijesh]

When memory encryption is enabled in VM, the guest pages will be
encrypted with the guest-specific key, to protect the confidentiality
of data in transit. To support the live migration we need to use
platform specific hooks to access the guest memory.

The kvm_memcrypt_save_outgoing_page() can be used by the sender to write
the encrypted pages and metadata associated with it on the socket.

The kvm_memcrypt_load_incoming_page() can be used by receiver to read the
incoming encrypted pages from the socket and load into the guest memory.

Signed-off-by: Brijesh Singh <>
---
 accel/kvm/kvm-all.c| 27 +++
 accel/kvm/sev-stub.c   | 11 +++
 accel/stubs/kvm-stub.c | 12 
 include/sysemu/kvm.h   | 12 
 include/sysemu/sev.h   |  3 +++
 5 files changed, 65 insertions(+)

diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 3d86ae5052..162a2d5085 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -110,6 +110,10 @@ struct KVMState
 /* memory encryption */
 void *memcrypt_handle;
 int (*memcrypt_encrypt_data)(void *handle, uint8_t *ptr, uint64_t len);
+int (*memcrypt_save_outgoing_page)(void *ehandle, QEMUFile *f,
+uint8_t *ptr, uint32_t sz, uint64_t *bytes_sent);
+int (*memcrypt_load_incoming_page)(void *ehandle, QEMUFile *f,
+uint8_t *ptr);
 };
 
 KVMState *kvm_state;
@@ -165,6 +169,29 @@ int kvm_memcrypt_encrypt_data(uint8_t *ptr, uint64_t len)
 return 1;
 }
 
+int kvm_memcrypt_save_outgoing_page(QEMUFile *f, uint8_t *ptr,
+uint32_t size, uint64_t *bytes_sent)
+{
+if (kvm_state->memcrypt_handle &&
+kvm_state->memcrypt_save_outgoing_page) {
+return 
kvm_state->memcrypt_save_outgoing_page(kvm_state->memcrypt_handle,
+f, ptr, size, bytes_sent);
+}
+
+return 1;
+}
+
+int kvm_memcrypt_load_incoming_page(QEMUFile *f, uint8_t *ptr)
+{
+if (kvm_state->memcrypt_handle &&
+kvm_state->memcrypt_load_incoming_page) {
+return 
kvm_state->memcrypt_load_incoming_page(kvm_state->memcrypt_handle,
+f, ptr);
+}
+
+return 1;
+}
+
 static KVMSlot *kvm_get_free_slot(KVMMemoryListener *kml)
 {
 KVMState *s = kvm_state;
diff --git a/accel/kvm/sev-stub.c b/accel/kvm/sev-stub.c
index 4f97452585..c12a8e005e 100644
--- a/accel/kvm/sev-stub.c
+++ b/accel/kvm/sev-stub.c
@@ -24,3 +24,14 @@ void *sev_guest_init(const char *id)
 {
 return NULL;
 }
+
+int sev_save_outgoing_page(void *handle, QEMUFile *f, uint8_t *ptr,
+   uint32_t size, uint64_t *bytes_sent)
+{
+return 1;
+}
+
+int sev_load_incoming_page(void *handle, QEMUFile *f, uint8_t *ptr)
+{
+return 1;
+}
diff --git a/accel/stubs/kvm-stub.c b/accel/stubs/kvm-stub.c
index 6feb66ed80..e14b879531 100644
--- a/accel/stubs/kvm-stub.c
+++ b/accel/stubs/kvm-stub.c
@@ -114,6 +114,18 @@ int kvm_memcrypt_encrypt_data(uint8_t *ptr, uint64_t len)
   return 1;
 }
 
+int kvm_memcrypt_save_outgoing_page(QEMUFile *f, uint8_t *ptr,
+uint32_t size, uint64_t *bytes_sent)
+{
+return 1;
+}
+
+int kvm_memcrypt_load_incoming_page(QEMUFile *f, uint8_t *ptr)
+{
+return 1;
+}
+
+
 #ifndef CONFIG_USER_ONLY
 int kvm_irqchip_add_msi_route(KVMState *s, int vector, PCIDevice *dev)
 {
diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index acd90aebb6..bb6bcc143c 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -247,6 +247,18 @@ bool kvm_memcrypt_enabled(void);
  */
 int kvm_memcrypt_encrypt_data(uint8_t *ptr, uint64_t len);
 
+/**
+ * kvm_memcrypt_save_outgoing_buffer - encrypt the outgoing buffer
+ * and write to the wire.
+ */
+int kvm_memcrypt_save_outgoing_page(QEMUFile *f, uint8_t *ptr, uint32_t size,
+uint64_t *bytes_sent);
+
+/**
+ * kvm_memcrypt_load_incoming_buffer - read the encrypt incoming buffer and 
copy
+ * the buffer into the guest memory space.
+ */
+int kvm_memcrypt_load_incoming_page(QEMUFile *f, uint8_t *ptr);
 
 #ifdef NEED_CPU_H
 #include "cpu.h"
diff --git a/include/sysemu/sev.h b/include/sysemu/sev.h
index 98c1ec8d38..752a71b1c0 100644
--- a/include/sysemu/sev.h
+++ b/include/sysemu/sev.h
@@ -18,4 +18,7 @@
 
 void *sev_guest_init(const char *id);
 int sev_encrypt_data(void *handle, uint8_t *ptr, uint64_t len);
+int sev_save_outgoing_page(void *handle, QEMUFile *f, uint8_t *ptr,
+   uint32_t size, uint64_t *bytes_sent);
+int sev_load_incoming_page(void *handle, QEMUFile *f, uint8_t *ptr);
 #endif
-- 
2.17.1

[Qemu-devel] [PATCH v2 00/13] Add SEV guest live migration support

[Singh, Brijesh]

AMD SEV encrypts the memory of VMs and because this encryption is done using
an address tweak, the hypervisor will not be able to simply copy ciphertext
between machines to migrate a VM. Instead the AMD SEV Key Management API
provides a set of functions which the hypervisor can use to package a
guest encrypted pages for migration, while maintaining the confidentiality
provided by AMD SEV.

The patch series add the support required in Qemu to perform the SEV
guest live migration. Before initiating the live migration a user
should use newly added 'migrate-set-sev-info' command to pass the
target machines certificate chain. See the docs/amd-memory-encryption.txt
for further details.

The patch series depends on kernel patches available here:
https://marc.info/?l=kvm=156278967226011=2

The complete tree with patch is available at:
https://github.com/codomania/qemu/tree/sev-migration-v2

Changes since v1:
 - use the dirty log sync APIs to also sync the page encryption bitmap
   when SEV is active.

Brijesh Singh (13):
  linux-headers: update kernel header to include SEV migration commands
  kvm: introduce high-level API to support encrypted page migration
  migration/ram: add support to send encrypted pages
  kvm: add support to sync the page encryption state bitmap
  doc: update AMD SEV API spec web link
  doc: update AMD SEV to include Live migration flow
  target/i386: sev: do not create launch context for an incoming guest
  misc.json: add migrate-set-sev-info command
  target/i386: sev: add support to encrypt the outgoing page
  target/i386: sev: add support to load incoming encrypted page
  kvm: introduce high-level API to migrate the page encryption bitmap
  migration: add support to migrate page encryption bitmap
  target/i386: sev: remove migration blocker

 accel/kvm/kvm-all.c| 108 
 accel/kvm/sev-stub.c   |  22 ++
 accel/stubs/kvm-stub.c |  22 ++
 docs/amd-memory-encryption.txt |  44 +++-
 include/exec/ram_addr.h| 161 +++-
 include/exec/ramlist.h |   3 +-
 include/sysemu/kvm.h   |  25 ++
 include/sysemu/sev.h   |   6 +
 linux-headers/linux/kvm.h  |  53 
 migration/ram.c|  91 ++-
 qapi/misc-target.json  |  18 ++
 target/i386/monitor.c  |  10 +
 target/i386/sev-stub.c |   5 +
 target/i386/sev.c  | 455 +++--
 target/i386/sev_i386.h |  11 +-
 target/i386/trace-events   |   8 +
 16 files changed, 1016 insertions(+), 26 deletions(-)

-- 
2.17.1

Re: [Qemu-devel] [PATCH 4/8] block/backup: hoist bitmap check into QMP interface

[Max Reitz]

On 10.07.19 19:57, John Snow wrote:
> 
> 
> On 7/10/19 12:11 PM, Max Reitz wrote:
>> On 10.07.19 03:05, John Snow wrote:
>>> This is nicer to do in the unified QMP interface that we have now,
>>> because it lets us use the right terminology back at the user.
>>>
>>> Signed-off-by: John Snow 
>>> ---
>>>  block/backup.c | 13 -
>>>  blockdev.c | 10 ++
>>>  2 files changed, 14 insertions(+), 9 deletions(-)
>>>
>>> diff --git a/block/backup.c b/block/backup.c
>>> index e2729cf6fa..a64b768e24 100644
>>> --- a/block/backup.c
>>> +++ b/block/backup.c
>>> @@ -566,6 +566,10 @@ BlockJob *backup_job_create(const char *job_id, 
>>> BlockDriverState *bs,
>>>  assert(bs);
>>>  assert(target);
>>>  
>>> +/* QMP interface protects us from these cases */
>>> +assert(sync_mode != MIRROR_SYNC_MODE_INCREMENTAL);
>>> +assert(sync_bitmap || sync_mode != MIRROR_SYNC_MODE_BITMAP);
>>
>> Implication would be a nice operator sometimes.
>>
>> ("assert(sync_mode == MIRROR_SYNC_MODE_BITMAP -> sync_bitmap)")
>>
>> (Can you do that in C++?  No, you can’t overload bool’s operators, right?)
>>
>> Reviewed-by: Max Reitz 
>>
> 
> Yes, I also find this assertion kind of hard to read personally, but it
> feels somewhat clunky to write:
> 
> if (antecedent) {
> assert(condition);
> }
> 
> I suppose we can also phrase this as:
> 
> assert(sync_mode == MIRROR_SYNC_MODE_BITMAP ? sync_bitmap : true);
> 
> Which might honestly be pretty good. Mind if I change it to this?

Looks weird (mostly unfamiliar), but I do not.

Max



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH 3/8] iotests/257: Refactor backup helpers

[Max Reitz]

On 10.07.19 19:52, John Snow wrote:
> 
> 
> On 7/10/19 12:04 PM, Max Reitz wrote:
>> On 10.07.19 03:05, John Snow wrote:
>>> This test needs support for non-bitmap backups and missing or
>>> unspecified bitmap sync modes, so rewrite the helpers to be a little
>>> more generic.
>>>
>>> Signed-off-by: John Snow 
>>> ---
>>>  tests/qemu-iotests/257 |  46 +
>>>  tests/qemu-iotests/257.out | 192 ++---
>>>  2 files changed, 124 insertions(+), 114 deletions(-)
>>>
>>> diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
>>> index 2ff4aa8695..2eb4f26c28 100755
>>> --- a/tests/qemu-iotests/257
>>> +++ b/tests/qemu-iotests/257
>>
>> [...]
>>
>>> -def bitmap_backup(drive, n, filepath, bitmap, bitmap_mode):
>>> -log("--- Bitmap Backup #{:d} ---\n".format(n))
>>> -target_id = "bitmap_target_{:d}".format(n)
>>> -job_id = "bitmap_backup_{:d}".format(n)
>>> +def backup(drive, n, filepath, bitmap, bitmap_mode, sync='bitmap'):
>>> +log("--- Test Backup #{:d} ---\n".format(n))
>>> +target_id = "backup_target_{:d}".format(n)
>>> +job_id = "backup_{:d}".format(n)
>>>  target_drive = Drive(filepath, vm=drive.vm)
>>>  
>>>  target_drive.create_target(target_id, drive.fmt, drive.size)
>>> -drive.vm.qmp_log("blockdev-backup", job_id=job_id, device=drive.name,
>>> - target=target_id, sync="bitmap",
>>> - bitmap_mode=bitmap_mode,
>>> - bitmap=bitmap,
>>> - auto_finalize=False)
>>> +
>>> +kwargs = {
>>> +'job_id': job_id,
>>> +'auto_finalize': False,
>>> +'bitmap': bitmap,
>>> +'bitmap_mode': bitmap_mode,
>>> +}
>>> +kwargs = {key: val for key, val in kwargs.items() if val is not None}
>>
>> I suppose this is to remove items that are None?
>>
>> Very cute, but why not just
>>
>>   kwargs = {
>> 'job_id': job_id,
>> 'auto_finalize': False,
>>   }
>>   if bitmap is not None:
>> kwargs['bitmap'] = bitmap
>> kwargs['bitmap_mode'] = bitmap_mode
>>
>> Exactly the same number of lines, but immediately makes it clear what’s
>> going on.  Not as cute, I admit.
>>
>> (Yes, I am indeed actively trying to train you not to write cute code.)
>>
> 
> It sneaks in. I genuinely struggle with understanding what other people
> will find readable; I have an authentically hard time reviewing other
> people's patches too. I'm earnestly not sure how I can help improve
> this, but I would like to.
> 
> I wasn't sure what the easiest way to avoid sending the "None" over the
> wire was, so I went with a general thing, but yes: it's because bitmap
> and bitmap_mode are set to None sometimes and I need to omit such keys.
> 
> In this case, though, I do test bitmap and bitmap_mode separately, so
> for the purposes of testing intentionally bad combinations you do need:
> 
> if bitmap is not None:
> kwargs['bitmap'] = bitmap
> if bitmap_mode is not None:
> kwargs['bitmap_mode'] = bitmap_mode
> 
> And I just looked at this and it did not spark joy; so I went with a
> generic filter to remove nulled keys. I admit it's /slightly/ cute and
> not immediately obvious why it needs to be done.
> 
> 
> This is even cuter, so maybe I am traveling in the wrong direction:
> 
> def backup(drive, n, filepath, sync, **kwargs):
> log("--- Test Backup #{:d} ---\n".format(n))
> target_id = "backup_target_{:d}".format(n)
> job_id = "backup_{:d}".format(n)
> target_drive = Drive(filepath, vm=drive.vm)
> 
> target_drive.create_target(target_id, drive.fmt, drive.size)
> kwargs.setdefault('auto_finalize', False)
> # Strip any arguments explicitly nulled by the caller:
> kwargs = {key: val for key, val in kwargs.items()
>   if val is not None}
> blockdev_backup(drive.vm, drive.name, target_id, sync, **kwargs)
> return job_id
> 
> It's quite a bit shorter and also makes backup() more flexible by
> omitting the bitmap and bitmap_mode arguments entirely, allowing the
> caller to override the auto_finalize default, etc. In this permutation,
> we don't know the full extent of kwargs so it makes sense to generically
> filter it.
> 
> Manually conditionally setting arguments is probably also fine.
> Do you still have a preference for the more static approach?

It’s OK with the comment.
Although it needs a kwargs.setdefault('job_id', job_id), too.

(Hm.  Shouldn’t 'auto-finalize' and 'job-id' work just fine?  It’s not
like the QMP scripts swap - and _.  They just replace _  by -.)

Max

>> The rest looks good to me:
>>
>> Reviewed-by: Max Reitz 
>>
> 
> Thanks for reviewing, as always!
> 




signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [RFC v2 2/2] hw/pvrdma: add live migration support

[Sukrit Bhatnagar]

On Mon, 8 Jul 2019 at 10:43, Yuval Shaia  wrote:
>
> On Sat, Jul 06, 2019 at 09:39:40AM +0530, Sukrit Bhatnagar wrote:
> > Use VMStateDescription for migrating device state. Currently,
>
> What do you mean by 'Currently'?

I meant that 'vmstate_pvrdma' will contain more fields later. This is
how it looks till now.

> > 'vmstate_pvrdma' describes the PCI and MSIX state for pvrdma and
> > 'vmstate_pvrdma_dsr_dma' describes a temporary state containing
> > some values obtained only after mapping to dsr in the source.
> > Since the dsr will not be available on dest until we map to the
> > dma address we had on source, these values cannot be migrated
> > directly.
> >
> > Add PVRDMAMigTmp to store this temporary state which consists of
> > dma addresses and ring page information. The 'parent' member is
> > used to refer to the device state (PVRDMADev) so that parent PCI
> > device object is accessible, which is needed to remap to DSR.
> >
> > pvrdma_dsr_dma_pre_save() saves the dsr state into this temporary
> > representation and pvrdma_dsr_dma_post_load() loads it back.
> > This load function also remaps to the dsr and and calls
> > load_dsr() for further map and ring init operations.
> >
> > Please note that this call to load_dsr() can be removed from the
> > migration flow and included in pvrdma_regs_write() to perform a
> > lazy load.
>
> The lazy load was suggested to overcome a potential problem with mapping to
> addresses while still in migration process. With that been solved i would
> suggest to drop the idea of lazy load.
>
> > As of now, migration will fail if there in an error in load_dsr().
> > Also, there might be a considerable amount of pages in the rings,
> > which will have dma map operations when the init functions are
> > called.
> > If this takes noticeable time, it might be better to have lazy
> > load instead.
>
> Yeah, make sense but i hope we will not get to this.
>
> >
> > Cc: Marcel Apfelbaum 
> > Cc: Yuval Shaia 
> > Signed-off-by: Sukrit Bhatnagar 
> > ---
> >  hw/rdma/vmw/pvrdma_main.c | 87 +++
> >  1 file changed, 87 insertions(+)
> >
> > diff --git a/hw/rdma/vmw/pvrdma_main.c b/hw/rdma/vmw/pvrdma_main.c
> > index 6c90db96f9..4a10bd2fc7 100644
> > --- a/hw/rdma/vmw/pvrdma_main.c
> > +++ b/hw/rdma/vmw/pvrdma_main.c
> > @@ -28,6 +28,7 @@
> >  #include "sysemu/sysemu.h"
> >  #include "monitor/monitor.h"
> >  #include "hw/rdma/rdma.h"
> > +#include "migration/register.h"
> >
> >  #include "../rdma_rm.h"
> >  #include "../rdma_backend.h"
> > @@ -593,6 +594,91 @@ static void pvrdma_shutdown_notifier(Notifier *n, void 
> > *opaque)
> >  pvrdma_fini(pci_dev);
> >  }
> >
> > +struct PVRDMAMigTmp {
> > +PVRDMADev *parent;
> > +uint64_t dma;
> > +uint64_t cmd_slot_dma;
> > +uint64_t resp_slot_dma;
> > +uint32_t cq_ring_pages_num_pages;
> > +uint64_t cq_ring_pages_pdir_dma;
> > +uint32_t async_ring_pages_num_pages;
> > +uint64_t async_ring_pages_pdir_dma;
> > +};
> > +
> > +static int pvrdma_dsr_dma_pre_save(void *opaque)
> > +{
> > +struct PVRDMAMigTmp *tmp = opaque;
>
> For me tmp sounds like a very bad name, if it is the convention then i can
> live with that, anyways suggesting something like mig_tmp_info or something
> like that.

It doesn't seem like a convention, but this is how it is named
in other places in the source. But sure, I'll change it to a more
meaningful name, if it is needed.

> > +DSRInfo *dsr_info = >parent->dsr_info;
>
> Can you shad some light on how the parent field is initialized with the
> pointer to the device object?

TL;DR
The trick is on line 567 in migration/vmstate-types.c in the
function 'put_tmp'.


Each VMStateDescription has one or more VMStateField values
which are used to define members of the device state for migration.

VMSTATE_WITH_TMP is a macro which defines a VMStateField for
those types of device state which need an intermediate representation
and cannot be migrated directly. In our case, this might apply to the
dma addresses and pdir information stored within dsr.

Each VMStateField has a corresponding VMStateInfo, which basically
is an interface declaring two functions, get and put. These functions are
used to describe how this certain type of VMStateField has to be
saved/loaded during migration.

VMSTATE_WITH_TMP has its VMStateInfo set to 'vmstate_info_tmp',
which assigns the functions 'put_tmp' and 'get_tmp' for save/load logic.

Lets consider state save during migration.
All the state saving work is done inside 'vmstate_save_state_v'.
It takes a VMStateDescription as 2nd argument and an opaque type
as 3rd argument.

When 'vmstate_save_state_v' is first called during migration, the
VMStateDescription argument is set to 'vmstate_pvrdma', and
opaque is set to our instance of PVRDMADev.

This function iteratively saves the VMStateFields; PCI then MSIX
then the WITH_TMP.
When it reaches WITH_TMP, it falls through the if-else conditions,
and finally reaches 

Re: [Qemu-devel] [PATCH 05/11] paaudio: do not move stream when sink/source name is specified

[Marc-André Lureau]

On Tue, Jul 9, 2019 at 10:53 PM Kővágó, Zoltán  wrote:
>
> Unless we disable stream moving, pulseaudio can easily move the stream
> on connect, effectively ignoring the source/sink specified by the user.
>
> Signed-off-by: Kővágó, Zoltán 

Reviewed-by: Marc-André Lureau 


> ---
>  audio/paaudio.c | 5 +
>  1 file changed, 5 insertions(+)
>
> diff --git a/audio/paaudio.c b/audio/paaudio.c
> index cc3a34c2ea..24d98b344a 100644
> --- a/audio/paaudio.c
> +++ b/audio/paaudio.c
> @@ -517,6 +517,11 @@ static pa_stream *qpa_simple_new (
>  #endif
>  | PA_STREAM_AUTO_TIMING_UPDATE;
>
> +if (dev) {
> +/* don't move the stream if the user specified a sink/source */
> +flags |= PA_STREAM_DONT_MOVE;
> +}
> +
>  if (dir == PA_STREAM_PLAYBACK) {
>  r = pa_stream_connect_playback(stream, dev, attr, flags, NULL, NULL);
>  } else {
> --
> 2.22.0
>
>


--
Marc-André Lureau

Re: [Qemu-devel] [PATCH 03/11] audio: add audiodev properties to frontends

[Marc-André Lureau]

On Tue, Jul 9, 2019 at 10:58 PM Kővágó, Zoltán  wrote:
>
> Finally add audiodev= options to audio frontends so users can specify
> which backend to use when multiple backends exist.  Not specifying an
> audiodev= option currently causes the first audiodev to be used, this is
> fixed in the next commit.
>
> Example usage: -audiodev pa,id=foo -device AC97,audiodev=foo
>
> Signed-off-by: Kővágó, Zoltán 

Reviewed-by: Marc-André Lureau 


> ---
>  audio/audio.h|  3 ++
>  include/hw/qdev-properties.h |  3 ++
>  hw/audio/ac97.c  |  1 +
>  hw/audio/adlib.c |  1 +
>  hw/audio/cs4231a.c   |  1 +
>  hw/audio/es1370.c|  7 +++-
>  hw/audio/gus.c   |  1 +
>  hw/audio/hda-codec.c |  1 +
>  hw/audio/milkymist-ac97.c|  6 
>  hw/audio/pcspk.c |  1 +
>  hw/audio/pl041.c |  1 +
>  hw/audio/sb16.c  |  1 +
>  hw/audio/wm8750.c|  6 
>  hw/core/qdev-properties-system.c | 57 
>  hw/usb/dev-audio.c   |  1 +
>  15 files changed, 90 insertions(+), 1 deletion(-)
>
> diff --git a/audio/audio.h b/audio/audio.h
> index ad2457f4de..c0722a5cda 100644
> --- a/audio/audio.h
> +++ b/audio/audio.h
> @@ -181,4 +181,7 @@ void audio_legacy_help(void);
>  AudioState *audio_state_by_name(const char *name);
>  const char *audio_get_id(QEMUSoundCard *card);
>
> +#define DEFINE_AUDIO_PROPERTIES(_s, _f) \
> +DEFINE_PROP_AUDIODEV("audiodev", _s, _f)
> +
>  #endif /* QEMU_AUDIO_H */
> diff --git a/include/hw/qdev-properties.h b/include/hw/qdev-properties.h
> index 1eae5ab056..de0b2c8423 100644
> --- a/include/hw/qdev-properties.h
> +++ b/include/hw/qdev-properties.h
> @@ -35,6 +35,7 @@ extern const PropertyInfo qdev_prop_blocksize;
>  extern const PropertyInfo qdev_prop_pci_host_devaddr;
>  extern const PropertyInfo qdev_prop_uuid;
>  extern const PropertyInfo qdev_prop_arraylen;
> +extern const PropertyInfo qdev_prop_audiodev;
>  extern const PropertyInfo qdev_prop_link;
>  extern const PropertyInfo qdev_prop_off_auto_pcibar;
>  extern const PropertyInfo qdev_prop_pcie_link_speed;
> @@ -236,6 +237,8 @@ extern const PropertyInfo qdev_prop_pcie_link_width;
>  + type_check(QemuUUID, typeof_field(_state, _field)),  \
>  .set_default = true,   \
>  }
> +#define DEFINE_PROP_AUDIODEV(_n, _s, _f) \
> +DEFINE_PROP(_n, _s, _f, qdev_prop_audiodev, QEMUSoundCard)
>
>  #define DEFINE_PROP_END_OF_LIST()   \
>  {}
> diff --git a/hw/audio/ac97.c b/hw/audio/ac97.c
> index fb98da2678..0d8e524233 100644
> --- a/hw/audio/ac97.c
> +++ b/hw/audio/ac97.c
> @@ -1409,6 +1409,7 @@ static int ac97_init (PCIBus *bus)
>  }
>
>  static Property ac97_properties[] = {
> +DEFINE_AUDIO_PROPERTIES(AC97LinkState, card),
>  DEFINE_PROP_UINT32 ("use_broken_id", AC97LinkState, use_broken_id, 0),
>  DEFINE_PROP_END_OF_LIST (),
>  };
> diff --git a/hw/audio/adlib.c b/hw/audio/adlib.c
> index 7dd9a89b89..df2e781788 100644
> --- a/hw/audio/adlib.c
> +++ b/hw/audio/adlib.c
> @@ -299,6 +299,7 @@ static void adlib_realizefn (DeviceState *dev, Error 
> **errp)
>  }
>
>  static Property adlib_properties[] = {
> +DEFINE_AUDIO_PROPERTIES(AdlibState, card),
>  DEFINE_PROP_UINT32 ("iobase",  AdlibState, port, 0x220),
>  DEFINE_PROP_UINT32 ("freq",AdlibState, freq,  44100),
>  DEFINE_PROP_END_OF_LIST (),
> diff --git a/hw/audio/cs4231a.c b/hw/audio/cs4231a.c
> index 7216b41cc1..e3ea830b47 100644
> --- a/hw/audio/cs4231a.c
> +++ b/hw/audio/cs4231a.c
> @@ -689,6 +689,7 @@ static int cs4231a_init (ISABus *bus)
>  }
>
>  static Property cs4231a_properties[] = {
> +DEFINE_AUDIO_PROPERTIES(CSState, card),
>  DEFINE_PROP_UINT32 ("iobase",  CSState, port, 0x534),
>  DEFINE_PROP_UINT32 ("irq", CSState, irq,  9),
>  DEFINE_PROP_UINT32 ("dma", CSState, dma,  3),
> diff --git a/hw/audio/es1370.c b/hw/audio/es1370.c
> index 260c142b70..7589671d20 100644
> --- a/hw/audio/es1370.c
> +++ b/hw/audio/es1370.c
> @@ -887,6 +887,11 @@ static int es1370_init (PCIBus *bus)
>  return 0;
>  }
>
> +static Property es1370_properties[] = {
> +DEFINE_AUDIO_PROPERTIES(ES1370State, card),
> +DEFINE_PROP_END_OF_LIST(),
> +};
> +
>  static void es1370_class_init (ObjectClass *klass, void *data)
>  {
>  DeviceClass *dc = DEVICE_CLASS (klass);
> @@ -903,6 +908,7 @@ static void es1370_class_init (ObjectClass *klass, void 
> *data)
>  dc->desc = "ENSONIQ AudioPCI ES1370";
>  dc->vmsd = _es1370;
>  dc->reset = es1370_on_reset;
> +dc->props = es1370_properties;
>  }
>
>  static const TypeInfo es1370_info = {
> @@ -923,4 +929,3 @@ static void es1370_register_types (void)
>  }
>
>  type_init (es1370_register_types)
> -
> diff --git a/hw/audio/gus.c b/hw/audio/gus.c
> index 9ab51631d9..566864bc9e 100644
> --- a/hw/audio/gus.c
> +++ 

Re: [Qemu-devel] [PATCH 02/11] audio: basic support for multi backend audio

[Marc-André Lureau]

On Tue, Jul 9, 2019 at 11:02 PM Kővágó, Zoltán  wrote:
>
> Audio functions no longer access glob_audio_state, instead they get an
> AudioState as a parameter.  This is required in order to support
> multiple backends.
>
> glob_audio_state is also gone, and replaced with a tailq so we can store
> more than one states.
>
> Signed-off-by: Kővágó, Zoltán 


I guess you could have made the vnc and wavcapture audiodev argument a
seperate commit. Or mention it in commit message.

Reviewed-by: Marc-André Lureau 




> ---
>  audio/audio.h  |  12 +++--
>  audio/audio_int.h  |   2 +
>  audio/audio_template.h |   2 +-
>  ui/vnc.h   |   2 +
>  audio/audio.c  | 102 +++--
>  audio/wavcapture.c |   6 +--
>  monitor/misc.c |  12 -
>  ui/vnc.c   |  15 +-
>  hmp-commands.hx|  11 +++--
>  qemu-options.hx|   5 ++
>  10 files changed, 131 insertions(+), 38 deletions(-)
>
> diff --git a/audio/audio.h b/audio/audio.h
> index 64b0f761bc..ad2457f4de 100644
> --- a/audio/audio.h
> +++ b/audio/audio.h
> @@ -78,8 +78,10 @@ typedef struct SWVoiceOut SWVoiceOut;
>  typedef struct CaptureVoiceOut CaptureVoiceOut;
>  typedef struct SWVoiceIn SWVoiceIn;
>
> +typedef struct AudioState AudioState;
>  typedef struct QEMUSoundCard {
>  char *name;
> +AudioState *state;
>  QLIST_ENTRY (QEMUSoundCard) entries;
>  } QEMUSoundCard;
>
> @@ -92,7 +94,8 @@ void AUD_log (const char *cap, const char *fmt, ...) 
> GCC_FMT_ATTR(2, 3);
>
>  void AUD_register_card (const char *name, QEMUSoundCard *card);
>  void AUD_remove_card (QEMUSoundCard *card);
> -CaptureVoiceOut *AUD_add_capture (
> +CaptureVoiceOut *AUD_add_capture(
> +AudioState *s,
>  struct audsettings *as,
>  struct audio_capture_ops *ops,
>  void *opaque
> @@ -160,8 +163,8 @@ static inline void *advance (void *p, int incr)
>  #define audio_MAX(a, b) ((a)<(b)?(b):(a))
>  #endif
>
> -int wav_start_capture (CaptureState *s, const char *path, int freq,
> -   int bits, int nchannels);
> +int wav_start_capture(AudioState *state, CaptureState *s, const char *path,
> +  int freq, int bits, int nchannels);
>
>  bool audio_is_cleaning_up(void);
>  void audio_cleanup(void);
> @@ -175,4 +178,7 @@ void audio_parse_option(const char *opt);
>  void audio_init_audiodevs(void);
>  void audio_legacy_help(void);
>
> +AudioState *audio_state_by_name(const char *name);
> +const char *audio_get_id(QEMUSoundCard *card);
> +
>  #endif /* QEMU_AUDIO_H */
> diff --git a/audio/audio_int.h b/audio/audio_int.h
> index 8164696b2c..9f01f6ad00 100644
> --- a/audio/audio_int.h
> +++ b/audio/audio_int.h
> @@ -196,6 +196,8 @@ typedef struct AudioState {
>
>  bool timer_running;
>  uint64_t timer_last;
> +
> +QTAILQ_ENTRY(AudioState) list;
>  } AudioState;
>
>  extern const struct mixeng_volume nominal_volume;
> diff --git a/audio/audio_template.h b/audio/audio_template.h
> index c721fed75d..54f07338e7 100644
> --- a/audio/audio_template.h
> +++ b/audio/audio_template.h
> @@ -428,7 +428,7 @@ SW *glue (AUD_open_, TYPE) (
>  struct audsettings *as
>  )
>  {
> -AudioState *s = _audio_state;
> +AudioState *s = card->state;
>  AudiodevPerDirectionOptions *pdo = glue(audio_get_pdo_, TYPE)(s->dev);
>
>  if (audio_bug(__func__, !card || !name || !callback_fn || !as)) {
> diff --git a/ui/vnc.h b/ui/vnc.h
> index 2f84db3142..6f54653455 100644
> --- a/ui/vnc.h
> +++ b/ui/vnc.h
> @@ -183,6 +183,8 @@ struct VncDisplay
>  #ifdef CONFIG_VNC_SASL
>  VncDisplaySASL sasl;
>  #endif
> +
> +AudioState *audio_state;
>  };
>
>  typedef struct VncTight {
> diff --git a/audio/audio.c b/audio/audio.c
> index 8d2f580788..e9dd7c8b32 100644
> --- a/audio/audio.c
> +++ b/audio/audio.c
> @@ -87,7 +87,8 @@ audio_driver *audio_driver_lookup(const char *name)
>  return NULL;
>  }
>
> -static AudioState glob_audio_state;
> +static QTAILQ_HEAD(AudioStateHead, AudioState) audio_states =
> +QTAILQ_HEAD_INITIALIZER(audio_states);
>
>  const struct mixeng_volume nominal_volume = {
>  .mute = 0,
> @@ -1236,11 +1237,14 @@ static void audio_run_capture (AudioState *s)
>
>  void audio_run (const char *msg)
>  {
> -AudioState *s = _audio_state;
> +AudioState *s;
> +
> +QTAILQ_FOREACH(s, _states, list) {
> +audio_run_out(s);
> +audio_run_in(s);
> +audio_run_capture(s);
> +}
>
> -audio_run_out (s);
> -audio_run_in (s);
> -audio_run_capture (s);
>  #ifdef DEBUG_POLL
>  {
>  static double prevtime;
> @@ -1304,13 +1308,11 @@ bool audio_is_cleaning_up(void)
>  return is_cleaning_up;
>  }
>
> -void audio_cleanup(void)
> +static void free_audio_state(AudioState *s)
>  {
> -AudioState *s = _audio_state;
>  HWVoiceOut *hwo, *hwon;
>  HWVoiceIn *hwi, *hwin;
>
> -is_cleaning_up = true;
>  QLIST_FOREACH_SAFE(hwo, >hw_head_out, entries, hwon) {
>  

Re: [Qemu-devel] [PATCH 07/11] audio: remove audio_MIN, audio_MAX

[Marc-André Lureau]

On Tue, Jul 9, 2019 at 11:08 PM Kővágó, Zoltán  wrote:
>
> There's already a MIN and MAX macro in include/qemu/osdep.h, use them
> instead.
>
> Signed-off-by: Kővágó, Zoltán 


Reviewed-by: Marc-André Lureau 


> ---
>  audio/audio.h | 17 -
>  audio/alsaaudio.c |  6 +++---
>  audio/audio.c | 20 ++--
>  audio/coreaudio.c |  2 +-
>  audio/dsoundaudio.c   |  2 +-
>  audio/noaudio.c   | 10 +-
>  audio/ossaudio.c  |  6 +++---
>  audio/paaudio.c   | 12 ++--
>  audio/sdlaudio.c  |  6 +++---
>  audio/spiceaudio.c| 10 +-
>  audio/wavaudio.c  |  4 ++--
>  hw/audio/ac97.c   | 10 +-
>  hw/audio/adlib.c  |  4 ++--
>  hw/audio/cs4231a.c|  4 ++--
>  hw/audio/es1370.c |  6 +++---
>  hw/audio/gus.c|  6 +++---
>  hw/audio/hda-codec.c  | 16 
>  hw/audio/milkymist-ac97.c |  8 
>  hw/audio/pcspk.c  |  2 +-
>  hw/audio/sb16.c   |  2 +-
>  hw/audio/wm8750.c |  4 ++--
>  21 files changed, 70 insertions(+), 87 deletions(-)
>
> diff --git a/audio/audio.h b/audio/audio.h
> index c0722a5cda..4a95758516 100644
> --- a/audio/audio.h
> +++ b/audio/audio.h
> @@ -146,23 +146,6 @@ static inline void *advance (void *p, int incr)
>  return (d + incr);
>  }
>
> -#ifdef __GNUC__
> -#define audio_MIN(a, b) ( __extension__ ({  \
> -__typeof (a) ta = a;\
> -__typeof (b) tb = b;\
> -((ta)>(tb)?(tb):(ta));  \
> -}))
> -
> -#define audio_MAX(a, b) ( __extension__ ({  \
> -__typeof (a) ta = a;\
> -__typeof (b) tb = b;\
> -((ta)<(tb)?(tb):(ta));  \
> -}))
> -#else
> -#define audio_MIN(a, b) ((a)>(b)?(b):(a))
> -#define audio_MAX(a, b) ((a)<(b)?(b):(a))
> -#endif
> -
>  int wav_start_capture(AudioState *state, CaptureState *s, const char *path,
>int freq, int bits, int nchannels);
>
> diff --git a/audio/alsaaudio.c b/audio/alsaaudio.c
> index 3745c823ad..6b9e0f06af 100644
> --- a/audio/alsaaudio.c
> +++ b/audio/alsaaudio.c
> @@ -634,7 +634,7 @@ static void alsa_write_pending (ALSAVoiceOut *alsa)
>
>  while (alsa->pending) {
>  int left_till_end_samples = hw->samples - alsa->wpos;
> -int len = audio_MIN (alsa->pending, left_till_end_samples);
> +int len = MIN (alsa->pending, left_till_end_samples);
>  char *src = advance (alsa->pcm_buf, alsa->wpos << hw->info.shift);
>
>  while (len) {
> @@ -697,7 +697,7 @@ static int alsa_run_out (HWVoiceOut *hw, int live)
>  return 0;
>  }
>
> -decr = audio_MIN (live, avail);
> +decr = MIN (live, avail);
>  decr = audio_pcm_hw_clip_out (hw, alsa->pcm_buf, decr, alsa->pending);
>  alsa->pending += decr;
>  alsa_write_pending (alsa);
> @@ -915,7 +915,7 @@ static int alsa_run_in (HWVoiceIn *hw)
>  }
>  }
>
> -decr = audio_MIN (dead, avail);
> +decr = MIN (dead, avail);
>  if (!decr) {
>  return 0;
>  }
> diff --git a/audio/audio.c b/audio/audio.c
> index 82dd0e3e13..6bf30ac9b3 100644
> --- a/audio/audio.c
> +++ b/audio/audio.c
> @@ -533,7 +533,7 @@ static int audio_pcm_hw_find_min_in (HWVoiceIn *hw)
>
>  for (sw = hw->sw_head.lh_first; sw; sw = sw->entries.le_next) {
>  if (sw->active) {
> -m = audio_MIN (m, sw->total_hw_samples_acquired);
> +m = MIN (m, sw->total_hw_samples_acquired);
>  }
>  }
>  return m;
> @@ -553,14 +553,14 @@ int audio_pcm_hw_clip_out (HWVoiceOut *hw, void 
> *pcm_buf,
> int live, int pending)
>  {
>  int left = hw->samples - pending;
> -int len = audio_MIN (left, live);
> +int len = MIN (left, live);
>  int clipped = 0;
>
>  while (len) {
>  struct st_sample *src = hw->mix_buf + hw->rpos;
>  uint8_t *dst = advance (pcm_buf, hw->rpos << hw->info.shift);
>  int samples_till_end_of_buf = hw->samples - hw->rpos;
> -int samples_to_clip = audio_MIN (len, samples_till_end_of_buf);
> +int samples_to_clip = MIN (len, samples_till_end_of_buf);
>
>  hw->clip (dst, src, samples_to_clip);
>
> @@ -614,7 +614,7 @@ int audio_pcm_sw_read (SWVoiceIn *sw, void *buf, int size)
>  }
>
>  swlim = (live * sw->ratio) >> 32;
> -swlim = audio_MIN (swlim, samples);
> +swlim = MIN (swlim, samples);
>
>  while (swlim) {
>  src = hw->conv_buf + rpos;
> @@ -662,7 +662,7 @@ static int audio_pcm_hw_find_min_out (HWVoiceOut *hw, int 
> *nb_livep)
>
>  for (sw = hw->sw_head.lh_first; sw; sw = sw->entries.le_next) {
>  if (sw->active || !sw->empty) {
> -m = audio_MIN (m, sw->total_hw_samples_mixed);
> +m = MIN (m, sw->total_hw_samples_mixed);
>  nb_live += 1;
>   

Re: [Qemu-devel] [PATCH 01/11] audio: reduce glob_audio_state usage

[Marc-André Lureau]

On Tue, Jul 9, 2019 at 10:57 PM Kővágó, Zoltán  wrote:
>
> Remove glob_audio_state from functions, where possible without breaking
> the API.  This means that most static functions in audio.c now take an
> AudioState pointer instead of implicitly using glob_audio_state.  Also
> included a pointer in SWVoice*, HWVoice* structs, so that functions
> dealing them can know the audio state without having to pass it around
> separately.
>
> This is required in order to support multiple simultaneous audio
> backends (added in a later commit).
>
> Signed-off-by: Kővágó, Zoltán 

Reviewed-by: Marc-André Lureau 


> ---
>  audio/audio_int.h  |  8 ++
>  audio/audio_template.h | 46 
>  audio/audio.c  | 59 +++---
>  3 files changed, 57 insertions(+), 56 deletions(-)
>
> diff --git a/audio/audio_int.h b/audio/audio_int.h
> index 3f14842709..8164696b2c 100644
> --- a/audio/audio_int.h
> +++ b/audio/audio_int.h
> @@ -49,9 +49,11 @@ struct audio_pcm_info {
>  int swap_endianness;
>  };
>
> +typedef struct AudioState AudioState;
>  typedef struct SWVoiceCap SWVoiceCap;
>
>  typedef struct HWVoiceOut {
> +AudioState *s;
>  int enabled;
>  int poll_mode;
>  int pending_disable;
> @@ -73,6 +75,7 @@ typedef struct HWVoiceOut {
>  } HWVoiceOut;
>
>  typedef struct HWVoiceIn {
> +AudioState *s;
>  int enabled;
>  int poll_mode;
>  struct audio_pcm_info info;
> @@ -94,6 +97,7 @@ typedef struct HWVoiceIn {
>
>  struct SWVoiceOut {
>  QEMUSoundCard *card;
> +AudioState *s;
>  struct audio_pcm_info info;
>  t_sample *conv;
>  int64_t ratio;
> @@ -111,6 +115,7 @@ struct SWVoiceOut {
>
>  struct SWVoiceIn {
>  QEMUSoundCard *card;
> +AudioState *s;
>  int active;
>  struct audio_pcm_info info;
>  int64_t ratio;
> @@ -188,6 +193,9 @@ typedef struct AudioState {
>  int nb_hw_voices_in;
>  int vm_running;
>  int64_t period_ticks;
> +
> +bool timer_running;
> +uint64_t timer_last;
>  } AudioState;
>
>  extern const struct mixeng_volume nominal_volume;
> diff --git a/audio/audio_template.h b/audio/audio_template.h
> index 1232bb54db..c721fed75d 100644
> --- a/audio/audio_template.h
> +++ b/audio/audio_template.h
> @@ -36,9 +36,9 @@
>  #define HWBUF hw->conv_buf
>  #endif
>
> -static void glue (audio_init_nb_voices_, TYPE) (struct audio_driver *drv)
> +static void glue(audio_init_nb_voices_, TYPE)(AudioState *s,
> +  struct audio_driver *drv)
>  {
> -AudioState *s = _audio_state;
>  int max_voices = glue (drv->max_voices_, TYPE);
>  int voice_size = glue (drv->voice_size_, TYPE);
>
> @@ -183,8 +183,8 @@ static void glue (audio_pcm_hw_del_sw_, TYPE) (SW *sw)
>
>  static void glue (audio_pcm_hw_gc_, TYPE) (HW **hwp)
>  {
> -AudioState *s = _audio_state;
>  HW *hw = *hwp;
> +AudioState *s = hw->s;
>
>  if (!hw->sw_head.lh_first) {
>  #ifdef DAC
> @@ -199,15 +199,14 @@ static void glue (audio_pcm_hw_gc_, TYPE) (HW **hwp)
>  }
>  }
>
> -static HW *glue (audio_pcm_hw_find_any_, TYPE) (HW *hw)
> +static HW *glue(audio_pcm_hw_find_any_, TYPE)(AudioState *s, HW *hw)
>  {
> -AudioState *s = _audio_state;
>  return hw ? hw->entries.le_next : glue (s->hw_head_, TYPE).lh_first;
>  }
>
> -static HW *glue (audio_pcm_hw_find_any_enabled_, TYPE) (HW *hw)
> +static HW *glue(audio_pcm_hw_find_any_enabled_, TYPE)(AudioState *s, HW *hw)
>  {
> -while ((hw = glue (audio_pcm_hw_find_any_, TYPE) (hw))) {
> +while ((hw = glue(audio_pcm_hw_find_any_, TYPE)(s, hw))) {
>  if (hw->enabled) {
>  return hw;
>  }
> @@ -215,12 +214,10 @@ static HW *glue (audio_pcm_hw_find_any_enabled_, TYPE) 
> (HW *hw)
>  return NULL;
>  }
>
> -static HW *glue (audio_pcm_hw_find_specific_, TYPE) (
> -HW *hw,
> -struct audsettings *as
> -)
> +static HW *glue(audio_pcm_hw_find_specific_, TYPE)(AudioState *s, HW *hw,
> +   struct audsettings *as)
>  {
> -while ((hw = glue (audio_pcm_hw_find_any_, TYPE) (hw))) {
> +while ((hw = glue(audio_pcm_hw_find_any_, TYPE)(s, hw))) {
>  if (audio_pcm_info_eq (>info, as)) {
>  return hw;
>  }
> @@ -228,10 +225,10 @@ static HW *glue (audio_pcm_hw_find_specific_, TYPE) (
>  return NULL;
>  }
>
> -static HW *glue (audio_pcm_hw_add_new_, TYPE) (struct audsettings *as)
> +static HW *glue(audio_pcm_hw_add_new_, TYPE)(AudioState *s,
> + struct audsettings *as)
>  {
>  HW *hw;
> -AudioState *s = _audio_state;
>  struct audio_driver *drv = s->drv;
>
>  if (!glue (s->nb_hw_voices_, TYPE)) {
> @@ -255,6 +252,7 @@ static HW *glue (audio_pcm_hw_add_new_, TYPE) (struct 
> audsettings *as)
>  return NULL;
>  }
>
> +hw->s = s;
>  hw->pcm_ops = drv->pcm_ops;
>  hw->ctl_caps = drv->ctl_caps;
>

Re: [Qemu-devel] [PATCH 06/11] paaudio: properly disconnect streams in fini_*

[Marc-André Lureau]

On Tue, Jul 9, 2019 at 10:53 PM Kővágó, Zoltán  wrote:
>
> Currently this needs a workaround due to bug #247 in pulseaudio.
>
> Signed-off-by: Kővágó, Zoltán 
> ---
>  audio/paaudio.c | 25 +++--
>  1 file changed, 23 insertions(+), 2 deletions(-)
>
> diff --git a/audio/paaudio.c b/audio/paaudio.c
> index 24d98b344a..490bcd770e 100644
> --- a/audio/paaudio.c
> +++ b/audio/paaudio.c
> @@ -685,6 +685,27 @@ static int qpa_init_in(HWVoiceIn *hw, struct audsettings 
> *as, void *drv_opaque)
>  return -1;
>  }
>
> +static void qpa_simple_disconnect(PAConnection *c, pa_stream *stream)
> +{
> +int err;
> +
> +pa_threaded_mainloop_lock(c->mainloop);
> +/*
> + * wait until actually connects. workaround pa bug #247
> + * https://gitlab.freedesktop.org/pulseaudio/pulseaudio/issues/247
> + */
> +while (pa_stream_get_state(stream) == PA_STREAM_CREATING) {
> +pa_threaded_mainloop_wait(c->mainloop);
> +}
> +
> +err = pa_stream_disconnect(stream);
> +if (err != 0) {
> +dolog("Failed to dissconnect! err=%d\n", err);

disconnect

Reviewed-by: Marc-André Lureau 


> +}
> +pa_stream_unref(stream);
> +pa_threaded_mainloop_unlock(c->mainloop);
> +}
> +
>  static void qpa_fini_out (HWVoiceOut *hw)
>  {
>  void *ret;
> @@ -696,7 +717,7 @@ static void qpa_fini_out (HWVoiceOut *hw)
>  audio_pt_join(>pt, , __func__);
>
>  if (pa->stream) {
> -pa_stream_unref (pa->stream);
> +qpa_simple_disconnect(pa->g->conn, pa->stream);
>  pa->stream = NULL;
>  }
>
> @@ -716,7 +737,7 @@ static void qpa_fini_in (HWVoiceIn *hw)
>  audio_pt_join(>pt, , __func__);
>
>  if (pa->stream) {
> -pa_stream_unref (pa->stream);
> +qpa_simple_disconnect(pa->g->conn, pa->stream);
>  pa->stream = NULL;
>  }
>
> --
> 2.22.0
>
>


--
Marc-André Lureau

Re: [Qemu-devel] [PATCH 08/11] audio: do not run each backend in audio_run

[Marc-André Lureau]

On Tue, Jul 9, 2019 at 10:51 PM Kővágó, Zoltán  wrote:
>
> audio_run is called manually by alsa and oss backends when polling.
> In this case only the requesting backend should be run, not all of them.
>
> Signed-off-by: Kővágó, Zoltán 


Reviewed-by: Marc-André Lureau 


> ---
>  audio/audio_int.h |  2 +-
>  audio/alsaaudio.c |  7 +--
>  audio/audio.c | 14 +-
>  audio/ossaudio.c  | 12 ++--
>  4 files changed, 17 insertions(+), 18 deletions(-)
>
> diff --git a/audio/audio_int.h b/audio/audio_int.h
> index 9f01f6ad00..7e00c1332e 100644
> --- a/audio/audio_int.h
> +++ b/audio/audio_int.h
> @@ -221,7 +221,7 @@ int audio_pcm_hw_clip_out (HWVoiceOut *hw, void *pcm_buf,
>  int audio_bug (const char *funcname, int cond);
>  void *audio_calloc (const char *funcname, int nmemb, size_t size);
>
> -void audio_run (const char *msg);
> +void audio_run(AudioState *s, const char *msg);
>
>  #define VOICE_ENABLE 1
>  #define VOICE_DISABLE 2
> diff --git a/audio/alsaaudio.c b/audio/alsaaudio.c
> index 6b9e0f06af..3daa7c8f8f 100644
> --- a/audio/alsaaudio.c
> +++ b/audio/alsaaudio.c
> @@ -39,6 +39,7 @@ struct pollhlp {
>  struct pollfd *pfds;
>  int count;
>  int mask;
> +AudioState *s;
>  };
>
>  typedef struct ALSAVoiceOut {
> @@ -199,11 +200,11 @@ static void alsa_poll_handler (void *opaque)
>  break;
>
>  case SND_PCM_STATE_PREPARED:
> -audio_run ("alsa run (prepared)");
> +audio_run(hlp->s, "alsa run (prepared)");
>  break;
>
>  case SND_PCM_STATE_RUNNING:
> -audio_run ("alsa run (running)");
> +audio_run(hlp->s, "alsa run (running)");
>  break;
>
>  default:
> @@ -749,6 +750,7 @@ static int alsa_init_out(HWVoiceOut *hw, struct 
> audsettings *as,
>  return -1;
>  }
>
> +alsa->pollhlp.s = hw->s;
>  alsa->handle = handle;
>  alsa->dev = dev;
>  return 0;
> @@ -850,6 +852,7 @@ static int alsa_init_in(HWVoiceIn *hw, struct audsettings 
> *as, void *drv_opaque)
>  return -1;
>  }
>
> +alsa->pollhlp.s = hw->s;
>  alsa->handle = handle;
>  alsa->dev = dev;
>  return 0;
> diff --git a/audio/audio.c b/audio/audio.c
> index 6bf30ac9b3..d73cc086b6 100644
> --- a/audio/audio.c
> +++ b/audio/audio.c
> @@ -835,7 +835,7 @@ static void audio_timer (void *opaque)
>  }
>  s->timer_last = now;
>
> -audio_run("timer");
> +audio_run(s, "timer");
>  audio_reset_timer(s);
>  }
>
> @@ -1237,15 +1237,11 @@ static void audio_run_capture (AudioState *s)
>  }
>  }
>
> -void audio_run (const char *msg)
> +void audio_run(AudioState *s, const char *msg)
>  {
> -AudioState *s;
> -
> -QTAILQ_FOREACH(s, _states, list) {
> -audio_run_out(s);
> -audio_run_in(s);
> -audio_run_capture(s);
> -}
> +audio_run_out(s);
> +audio_run_in(s);
> +audio_run_capture(s);
>
>  #ifdef DEBUG_POLL
>  {
> diff --git a/audio/ossaudio.c b/audio/ossaudio.c
> index 29139ef1f5..456fba7480 100644
> --- a/audio/ossaudio.c
> +++ b/audio/ossaudio.c
> @@ -110,28 +110,28 @@ static void oss_anal_close (int *fdp)
>
>  static void oss_helper_poll_out (void *opaque)
>  {
> -(void) opaque;
> -audio_run ("oss_poll_out");
> +AudioState *s = opaque;
> +audio_run(s, "oss_poll_out");
>  }
>
>  static void oss_helper_poll_in (void *opaque)
>  {
> -(void) opaque;
> -audio_run ("oss_poll_in");
> +AudioState *s = opaque;
> +audio_run(s, "oss_poll_in");
>  }
>
>  static void oss_poll_out (HWVoiceOut *hw)
>  {
>  OSSVoiceOut *oss = (OSSVoiceOut *) hw;
>
> -qemu_set_fd_handler (oss->fd, NULL, oss_helper_poll_out, NULL);
> +qemu_set_fd_handler(oss->fd, NULL, oss_helper_poll_out, hw->s);
>  }
>
>  static void oss_poll_in (HWVoiceIn *hw)
>  {
>  OSSVoiceIn *oss = (OSSVoiceIn *) hw;
>
> -qemu_set_fd_handler (oss->fd, oss_helper_poll_in, NULL, NULL);
> +qemu_set_fd_handler(oss->fd, oss_helper_poll_in, NULL, hw->s);
>  }
>
>  static int oss_write (SWVoiceOut *sw, void *buf, int len)
> --
> 2.22.0
>
>


--
Marc-André Lureau

Re: [Qemu-devel] [PATCH 09/11] paaudio: fix playback glitches

[Marc-André Lureau]

On Tue, Jul 9, 2019 at 10:49 PM Kővágó, Zoltán  wrote:
>
> Pulseaudio normally assumes that when the server wants it, the client
> can generate the audio samples and send it right away.  Unfortunately
> this is not the case with QEMU -- it's up to the emulated system when
> does it generate the samples.  Buffering the samples and sending them
> from a background thread is just a workaround, that doesn't work too
> well.  Instead enable pa's compatibility support and let pa worry about
> the details.
>
> Signed-off-by: Kővágó, Zoltán 

Could you explain how this is related to PA_STREAM_ADJUST_LATENCY ?


> ---
>  audio/paaudio.c | 6 ++
>  1 file changed, 2 insertions(+), 4 deletions(-)
>
> diff --git a/audio/paaudio.c b/audio/paaudio.c
> index 9d46f11b0a..d320d2e453 100644
> --- a/audio/paaudio.c
> +++ b/audio/paaudio.c
> @@ -512,10 +512,8 @@ static pa_stream *qpa_simple_new (
>
>  flags =
>  PA_STREAM_INTERPOLATE_TIMING
> -#ifdef PA_STREAM_ADJUST_LATENCY
> -| PA_STREAM_ADJUST_LATENCY
> -#endif
> -| PA_STREAM_AUTO_TIMING_UPDATE;
> +| PA_STREAM_AUTO_TIMING_UPDATE
> +| PA_STREAM_EARLY_REQUESTS;
>
>  if (dev) {
>  /* don't move the stream if the user specified a sink/source */
> --
> 2.22.0
>
>


--
Marc-André Lureau

Re: [Qemu-devel] [PATCH 10/11] audio: remove read and write pcm_ops

[Marc-André Lureau]

On Tue, Jul 9, 2019 at 10:57 PM Kővágó, Zoltán  wrote:
>
> They just called audio_pcm_sw_read/write anyway, so it makes no sense
> to have them too.  (The noaudio's read is the only exception, but it
> should work with the generic code too.)

It works with the generic code, but wouldn't it be more expensive?

Perhaps there can be something in audio_pcm_sw_write() to skip the
work if noaudio is the backend?

>
> Signed-off-by: Kővágó, Zoltán 
> ---
>  audio/audio_int.h   |  5 -
>  audio/alsaaudio.c   | 12 
>  audio/audio.c   |  8 
>  audio/coreaudio.c   |  6 --
>  audio/dsoundaudio.c | 12 
>  audio/noaudio.c | 19 ---
>  audio/ossaudio.c| 12 
>  audio/paaudio.c | 12 
>  audio/sdlaudio.c|  6 --
>  audio/spiceaudio.c  | 12 
>  audio/wavaudio.c|  6 --
>  11 files changed, 4 insertions(+), 106 deletions(-)
>
> diff --git a/audio/audio_int.h b/audio/audio_int.h
> index 7e00c1332e..003b7ab8cc 100644
> --- a/audio/audio_int.h
> +++ b/audio/audio_int.h
> @@ -150,13 +150,11 @@ struct audio_pcm_ops {
>  int  (*init_out)(HWVoiceOut *hw, struct audsettings *as, void 
> *drv_opaque);
>  void (*fini_out)(HWVoiceOut *hw);
>  int  (*run_out) (HWVoiceOut *hw, int live);
> -int  (*write)   (SWVoiceOut *sw, void *buf, int size);
>  int  (*ctl_out) (HWVoiceOut *hw, int cmd, ...);
>
>  int  (*init_in) (HWVoiceIn *hw, struct audsettings *as, void 
> *drv_opaque);
>  void (*fini_in) (HWVoiceIn *hw);
>  int  (*run_in)  (HWVoiceIn *hw);
> -int  (*read)(SWVoiceIn *sw, void *buf, int size);
>  int  (*ctl_in)  (HWVoiceIn *hw, int cmd, ...);
>  };
>
> @@ -210,11 +208,8 @@ audio_driver *audio_driver_lookup(const char *name);
>  void audio_pcm_init_info (struct audio_pcm_info *info, struct audsettings 
> *as);
>  void audio_pcm_info_clear_buf (struct audio_pcm_info *info, void *buf, int 
> len);
>
> -int  audio_pcm_sw_write (SWVoiceOut *sw, void *buf, int len);
>  int  audio_pcm_hw_get_live_in (HWVoiceIn *hw);
>
> -int  audio_pcm_sw_read (SWVoiceIn *sw, void *buf, int len);
> -
>  int audio_pcm_hw_clip_out (HWVoiceOut *hw, void *pcm_buf,
> int live, int pending);
>
> diff --git a/audio/alsaaudio.c b/audio/alsaaudio.c
> index 3daa7c8f8f..e9e3a4819c 100644
> --- a/audio/alsaaudio.c
> +++ b/audio/alsaaudio.c
> @@ -270,11 +270,6 @@ static int alsa_poll_in (HWVoiceIn *hw)
>  return alsa_poll_helper (alsa->handle, >pollhlp, POLLIN);
>  }
>
> -static int alsa_write (SWVoiceOut *sw, void *buf, int len)
> -{
> -return audio_pcm_sw_write (sw, buf, len);
> -}
> -
>  static snd_pcm_format_t aud_to_alsafmt (AudioFormat fmt, int endianness)
>  {
>  switch (fmt) {
> @@ -988,11 +983,6 @@ static int alsa_run_in (HWVoiceIn *hw)
>  return read_samples;
>  }
>
> -static int alsa_read (SWVoiceIn *sw, void *buf, int size)
> -{
> -return audio_pcm_sw_read (sw, buf, size);
> -}
> -
>  static int alsa_ctl_in (HWVoiceIn *hw, int cmd, ...)
>  {
>  ALSAVoiceIn *alsa = (ALSAVoiceIn *) hw;
> @@ -1076,13 +1066,11 @@ static struct audio_pcm_ops alsa_pcm_ops = {
>  .init_out = alsa_init_out,
>  .fini_out = alsa_fini_out,
>  .run_out  = alsa_run_out,
> -.write= alsa_write,
>  .ctl_out  = alsa_ctl_out,
>
>  .init_in  = alsa_init_in,
>  .fini_in  = alsa_fini_in,
>  .run_in   = alsa_run_in,
> -.read = alsa_read,
>  .ctl_in   = alsa_ctl_in,
>  };
>
> diff --git a/audio/audio.c b/audio/audio.c
> index d73cc086b6..b79f56fe64 100644
> --- a/audio/audio.c
> +++ b/audio/audio.c
> @@ -594,7 +594,7 @@ static int audio_pcm_sw_get_rpos_in (SWVoiceIn *sw)
>  }
>  }
>
> -int audio_pcm_sw_read (SWVoiceIn *sw, void *buf, int size)
> +static int audio_pcm_sw_read(SWVoiceIn *sw, void *buf, int size)
>  {
>  HWVoiceIn *hw = sw->hw;
>  int samples, live, ret = 0, swlim, isamp, osamp, rpos, total = 0;
> @@ -696,7 +696,7 @@ static int audio_pcm_hw_get_live_out (HWVoiceOut *hw, int 
> *nb_live)
>  /*
>   * Soft voice (playback)
>   */
> -int audio_pcm_sw_write (SWVoiceOut *sw, void *buf, int size)
> +static int audio_pcm_sw_write(SWVoiceOut *sw, void *buf, int size)
>  {
>  int hwsamples, samples, isamp, osamp, wpos, live, dead, left, swlim, 
> blck;
>  int ret = 0, pos = 0, total = 0;
> @@ -854,7 +854,7 @@ int AUD_write (SWVoiceOut *sw, void *buf, int size)
>  return 0;
>  }
>
> -return sw->hw->pcm_ops->write(sw, buf, size);
> +return audio_pcm_sw_write(sw, buf, size);
>  }
>
>  int AUD_read (SWVoiceIn *sw, void *buf, int size)
> @@ -869,7 +869,7 @@ int AUD_read (SWVoiceIn *sw, void *buf, int size)
>  return 0;
>  }
>
> -return sw->hw->pcm_ops->read(sw, buf, size);
> +return audio_pcm_sw_read(sw, buf, size);
>  }
>
>  int AUD_get_buffer_size_out (SWVoiceOut *sw)
> diff --git a/audio/coreaudio.c b/audio/coreaudio.c
> index f0ab4014a8..091fe84a34 100644
> --- 

Re: [Qemu-devel] [PATCH 02/11] audio: basic support for multi backend audio

[Zoltán Kővágó]

On 2019-07-10 06:06, Markus Armbruster wrote:
> "Kővágó, Zoltán"  writes:
> 
>> Audio functions no longer access glob_audio_state, instead they get an
>> AudioState as a parameter.  This is required in order to support
>> multiple backends.
>>
>> glob_audio_state is also gone, and replaced with a tailq so we can store
>> more than one states.
>>
>> Signed-off-by: Kővágó, Zoltán 
>> ---
> [...]
>> diff --git a/hmp-commands.hx b/hmp-commands.hx
>> index bfa5681dd2..23196da3fe 100644
>> --- a/hmp-commands.hx
>> +++ b/hmp-commands.hx
>> @@ -819,16 +819,17 @@ ETEXI
>>  
>>  {
>>  .name   = "wavcapture",
>> -.args_type  = "path:F,freq:i?,bits:i?,nchannels:i?",
>> -.params = "path [frequency [bits [channels]]]",
>> +.args_type  = "path:F,freq:i?,bits:i?,nchannels:i?,audiodev:s?",
>> +.params = "path [frequency [bits [channels [audiodev",
>>  .help   = "capture audio to a wave file (default 
>> frequency=44100 bits=16 channels=2)",
>>  .cmd= hmp_wavcapture,
>>  },
>>  STEXI
>> -@item wavcapture @var{filename} [@var{frequency} [@var{bits} 
>> [@var{channels}]]]
>> +@item wavcapture @var{filename} [@var{frequency} [@var{bits} 
>> [@var{channels} [@var{audiodev}
>>  @findex wavcapture
>> -Capture audio into @var{filename}. Using sample rate @var{frequency}
>> -bits per sample @var{bits} and number of channels @var{channels}.
>> +Capture audio into @var{filename} from @var{audiodev}. Using sample rate
>> +@var{frequency} bits per sample @var{bits} and number of channels
>> +@var{channels}.
>>  
>>  Defaults:
>>  @itemize @minus
>@item Sample rate = 44100 Hz - CD quality
>@item Bits = 16
>@item Number of channels = 2 - Stereo
>@end itemize
>ETEXI
> 
> Defaults for the other optional arguments are listed here.  Why not for
> @audiodev?

There's no default listed because there's no default when you use the
-audiodev options, since there's no good default.  When you don't use
-audiodev, it'll use the implicitly created audiodev which doesn't have
a name, so it can't be specified.  But I agree that this situation
should be documented somehow.

> 
>> diff --git a/qemu-options.hx b/qemu-options.hx
>> index 9621e934c0..0111055aa4 100644
>> --- a/qemu-options.hx
>> +++ b/qemu-options.hx
>> @@ -1978,6 +1978,11 @@ can help the device and guest to keep up and not lose 
>> events in case
>>  events are arriving in bulk.  Possible causes for the latter are flaky
>>  network connections, or scripts for automated testing.
>>  
>> +@item audiodev=@var{audiodev}
>> +
>> +Use the specified @var{audiodev} when the VNC client requests audio
>> +transmission.
>> +
> 
> What's the default?

It's the same story as wav_capture.

Regards,
Zoltan

Re: [Qemu-devel] [PATCH 04/11] audio: audiodev= parameters no longer optional when -audiodev present

[Marc-André Lureau]

On Tue, Jul 9, 2019 at 11:00 PM Kővágó, Zoltán  wrote:
>
> This means you should probably stop using -soundhw (as it doesn't allow
> you to specify any options) and add the device manually with -device.
> The exception is pcspk, it's currently not possible to manually add it.
> To use it with audiodev, use something like this:
>
> -audiodev id=foo,... -global isa-pcspk.audiodev=foo -soundhw pcspk

Can you seperate the paaudio changes to ease review?

>
> Signed-off-by: Kővágó, Zoltán 
> ---
>  audio/audio.c   |  24 ++--
>  audio/paaudio.c | 329 +++-
>  2 files changed, 203 insertions(+), 150 deletions(-)
>
> diff --git a/audio/audio.c b/audio/audio.c
> index e9dd7c8b32..82dd0e3e13 100644
> --- a/audio/audio.c
> +++ b/audio/audio.c
> @@ -101,6 +101,8 @@ const struct mixeng_volume nominal_volume = {
>  #endif
>  };
>
> +static bool legacy_config = true;
> +
>  #ifdef AUDIO_IS_FLAWLESS_AND_NO_CHECKS_ARE_REQURIED
>  #error No its not
>  #else
> @@ -1392,7 +1394,7 @@ static AudiodevListEntry *audiodev_find(
>   * if dev == NULL => legacy implicit initialization, return the already 
> created
>   *   state or create a new one
>   */
> -static AudioState *audio_init(Audiodev *dev)
> +static AudioState *audio_init(Audiodev *dev, const char *name)
>  {
>  static bool atexit_registered;
>  size_t i;
> @@ -1406,12 +1408,13 @@ static AudioState *audio_init(Audiodev *dev)
>
>  if (dev) {
>  /* -audiodev option */
> +legacy_config = false;
>  drvname = AudiodevDriver_str(dev->driver);
>  } else if (!QTAILQ_EMPTY(_states)) {
> -/*
> - * todo: check for -audiodev once we have normal audiodev selection
> - * support
> - */
> +if (!legacy_config) {
> +dolog("You must specify an audiodev= for the device %s\n", name);
> +exit(1);
> +}
>  return QTAILQ_FIRST(_states);
>  } else {
>  /* legacy implicit initialization */
> @@ -1518,7 +1521,7 @@ void audio_free_audiodev_list(AudiodevListHead *head)
>  void AUD_register_card (const char *name, QEMUSoundCard *card)
>  {
>  if (!card->state) {
> -card->state = audio_init(NULL);
> +card->state = audio_init(NULL, name);
>  }
>
>  card->name = g_strdup (name);
> @@ -1544,8 +1547,11 @@ CaptureVoiceOut *AUD_add_capture(
>  struct capture_callback *cb;
>
>  if (!s) {
> -/* todo: remove when we have normal audiodev selection support */
> -s = audio_init(NULL);
> +if (!legacy_config) {
> +dolog("You must specify audiodev when trying to capture\n");
> +return NULL;
> +}
> +s = audio_init(NULL, NULL);
>  }
>
>  if (audio_validate_settings (as)) {
> @@ -1776,7 +1782,7 @@ void audio_init_audiodevs(void)
>  AudiodevListEntry *e;
>
>  QSIMPLEQ_FOREACH(e, , next) {
> -audio_init(e->dev);
> +audio_init(e->dev, NULL);
>  }
>  }
>
> diff --git a/audio/paaudio.c b/audio/paaudio.c
> index 5fc886bb33..cc3a34c2ea 100644
> --- a/audio/paaudio.c
> +++ b/audio/paaudio.c
> @@ -11,10 +11,21 @@
>  #include "audio_int.h"
>  #include "audio_pt_int.h"
>
> -typedef struct {
> -Audiodev *dev;
> +typedef struct PAConnection {
> +char *server;
> +int refcount;
> +QTAILQ_ENTRY(PAConnection) list;
> +
>  pa_threaded_mainloop *mainloop;
>  pa_context *context;
> +} PAConnection;
> +
> +static QTAILQ_HEAD(PAConnectionHead, PAConnection) pa_conns =
> +QTAILQ_HEAD_INITIALIZER(pa_conns);
> +
> +typedef struct {
> +Audiodev *dev;
> +PAConnection *conn;
>  } paaudio;
>
>  typedef struct {
> @@ -45,7 +56,7 @@ typedef struct {
>  int samples;
>  } PAVoiceIn;
>
> -static void qpa_audio_fini(void *opaque);
> +static void qpa_conn_fini(PAConnection *c);
>
>  static void GCC_FMT_ATTR (2, 3) qpa_logerr (int err, const char *fmt, ...)
>  {
> @@ -108,11 +119,11 @@ static inline int PA_STREAM_IS_GOOD(pa_stream_state_t x)
>
>  static int qpa_simple_read (PAVoiceIn *p, void *data, size_t length, int 
> *rerror)
>  {
> -paaudio *g = p->g;
> +PAConnection *c = p->g->conn;
>
> -pa_threaded_mainloop_lock (g->mainloop);
> +pa_threaded_mainloop_lock(c->mainloop);
>
> -CHECK_DEAD_GOTO (g, p->stream, rerror, unlock_and_fail);
> +CHECK_DEAD_GOTO(c, p->stream, rerror, unlock_and_fail);
>
>  while (length > 0) {
>  size_t l;
> @@ -121,11 +132,11 @@ static int qpa_simple_read (PAVoiceIn *p, void *data, 
> size_t length, int *rerror
>  int r;
>
>  r = pa_stream_peek (p->stream, >read_data, >read_length);
> -CHECK_SUCCESS_GOTO (g, rerror, r == 0, unlock_and_fail);
> +CHECK_SUCCESS_GOTO(c, rerror, r == 0, unlock_and_fail);
>
>  if (!p->read_data) {
> -pa_threaded_mainloop_wait (g->mainloop);
> -CHECK_DEAD_GOTO (g, p->stream, rerror, unlock_and_fail);
> +

[Qemu-devel] [PULL 3/3] docs/bitmaps: use QMP lexer instead of json

[John Snow]

The annotated style json we use in QMP documentation is not strict json
and depending on the version of Sphinx (2.0+) or Pygments installed,
might cause the build to fail.

Use the new QMP lexer.

Further, some versions of Sphinx can not apply custom lexers to "code"
directives and require the use of "code-block" directives instead, so
make that change at this time as well.

Tested under:
- Sphinx 1.3.6 and Pygments 2.4
- Sphinx 1.7.6 and Pygments 2.2 (Fedora 29 packages)
- Sphinx 2.0.1 and Pygments 2.4
- Sphinx 3.0.0+/f396b3a783 and Pygments 2.4 (From Sphinx git c4f44bdd)

Reported-by: Aarushi Mehta 
Signed-off-by: John Snow 
Reviewed-by: Eduardo Habkost 
Message-id: 20190603214653.29369-4-js...@redhat.com
Signed-off-by: John Snow 
---
 docs/interop/bitmaps.rst | 54 
 1 file changed, 27 insertions(+), 27 deletions(-)

diff --git a/docs/interop/bitmaps.rst b/docs/interop/bitmaps.rst
index c29ac4a854..c20bd37a79 100644
--- a/docs/interop/bitmaps.rst
+++ b/docs/interop/bitmaps.rst
@@ -199,7 +199,7 @@ persistence, and recording state can be adjusted at 
creation time.
 
  to create a new, actively recording persistent bitmap:
 
- .. code:: json
+ .. code-block:: QMP
 
   -> { "execute": "block-dirty-bitmap-add",
"arguments": {
@@ -220,7 +220,7 @@ persistence, and recording state can be adjusted at 
creation time.
  To create a new, disabled (``-recording``), transient bitmap that tracks
  changes in 32KiB segments:
 
- .. code:: json
+ .. code-block:: QMP
 
   -> { "execute": "block-dirty-bitmap-add",
"arguments": {
@@ -254,7 +254,7 @@ Deletes a bitmap. Bitmaps that are ``+busy`` cannot be 
removed.
 
  Remove a bitmap named ``bitmap0`` from node ``drive0``:
 
- .. code:: json
+ .. code-block:: QMP
 
   -> { "execute": "block-dirty-bitmap-remove",
"arguments": {
@@ -280,7 +280,7 @@ Clears all dirty bits from a bitmap. ``+busy`` bitmaps 
cannot be cleared.
 
  Clear all dirty bits from bitmap ``bitmap0`` on node ``drive0``:
 
- .. code:: json
+ .. code-block:: QMP
 
   -> { "execute": "block-dirty-bitmap-clear",
"arguments": {
@@ -309,7 +309,7 @@ begin being recorded. ``+busy`` bitmaps cannot be enabled.
 
  To set ``+recording`` on bitmap ``bitmap0`` on node ``drive0``:
 
- .. code:: json
+ .. code-block:: QMP
 
   -> { "execute": "block-dirty-bitmap-enable",
"arguments": {
@@ -347,7 +347,7 @@ writes to begin being ignored. ``+busy`` bitmaps cannot be 
disabled.
 
  To set ``-recording`` on bitmap ``bitmap0`` on node ``drive0``:
 
- .. code:: json
+ .. code-block:: QMP
 
   -> { "execute": "block-dirty-bitmap-disable",
"arguments": {
@@ -393,7 +393,7 @@ in any one source bitmap, the target bitmap will mark that 
segment dirty.
  ``drive0``. If ``new_bitmap`` was empty prior to this command, this achieves
  a copy.
 
- .. code:: json
+ .. code-block:: QMP
 
   -> { "execute": "block-dirty-bitmap-merge",
"arguments": {
@@ -424,7 +424,7 @@ attached to nodes serving as the root for guest devices.
  API. This result highlights a bitmap ``bitmap0`` attached to the root node of
  device ``drive0``.
 
- .. code:: json
+ .. code-block:: QMP
 
   -> {
"execute": "query-block",
@@ -562,7 +562,7 @@ new, empty bitmap that records writes from this point in 
time forward.
   destination. These writes will be recorded in the bitmap
   accordingly.
 
-.. code:: json
+.. code-block:: QMP
 
   -> {
"execute": "transaction",
@@ -650,7 +650,7 @@ Example: Resetting an Incremental Backup Anchor Point
 If we want to start a new backup chain with an existing bitmap, we can also
 use a transaction to reset the bitmap while making a new full backup:
 
-.. code:: json
+.. code-block:: QMP
 
   -> {
"execute": "transaction",
@@ -730,7 +730,7 @@ Example: First Incremental Backup
 
 #. Issue an incremental backup command:
 
-   .. code:: json
+   .. code-block:: QMP
 
 -> {
  "execute": "drive-backup",
@@ -788,7 +788,7 @@ Example: Second Incremental Backup
 #. Issue a new incremental backup command. The only difference here is that we
have changed the target image below.
 
-   .. code:: json
+   .. code-block:: QMP
 
 -> {
  "execute": "drive-backup",
@@ -869,7 +869,7 @@ image:
 #. Issue a new incremental backup command. Apart from the new destination
image, there is no difference from the last two examples.
 
-   .. code:: json
+   .. code-block:: QMP
 
 -> {
  "execute": "drive-backup",
@@ -932,7 +932,7 @@ point in time.
 
 #. Create a full (anchor) backup for each drive, with accompanying bitmaps:
 
-   .. code:: json
+   .. code-block:: QMP
 
 -> {
  "execute": "transaction",
@@ -1018,7 +1018,7 @@ point in time.
 
 #. Issue a multi-drive incremental push backup transaction:
 
-   .. code:: json
+   .. code-block:: QMP
 
 -> {
  "execute": "transaction",
@@ -1121,7 +1121,7 @@ described above. This example demonstrates the single-job 

[Qemu-devel] [PULL 0/3] Bitmaps patches

[John Snow]

The following changes since commit 6df2cdf44a82426f7a59dcb03f0dd2181ed7fdfa:

  Update version for v4.1.0-rc0 release (2019-07-09 17:21:53 +0100)

are available in the Git repository at:

  https://github.com/jnsnow/qemu.git tags/bitmaps-pull-request

for you to fetch changes up to a7786bfb0effe0b4b0fc61d8a8cd307c0b739ed7:

  docs/bitmaps: use QMP lexer instead of json (2019-07-10 15:08:07 -0400)


Pull request:
  This is a build fix.



John Snow (3):
  docs/interop/bitmaps.rst: Fix typos
  sphinx: add qmp_lexer
  docs/bitmaps: use QMP lexer instead of json

 docs/conf.py |  4 +--
 docs/interop/bitmaps.rst | 58 
 docs/sphinx/qmp_lexer.py | 43 +
 3 files changed, 74 insertions(+), 31 deletions(-)
 create mode 100644 docs/sphinx/qmp_lexer.py

-- 
2.21.0

[Qemu-devel] [PULL 2/3] sphinx: add qmp_lexer

[John Snow]

Sphinx, through Pygments, does not like annotated json examples very
much. In some versions of Sphinx (1.7), it will render the non-json
portions of code blocks in red, but in newer versions (2.0) it will
throw an exception and not highlight the block at all. Though we can
suppress this warning, it doesn't bring back highlighting on non-strict
json blocks.

We can alleviate this by creating a custom lexer for QMP examples that
allows us to properly highlight these examples in a robust way, keeping
our directionality and elision notations.

Signed-off-by: Eduardo Habkost 
Signed-off-by: John Snow 
Reported-by: Aarushi Mehta 
Reviewed-by: Peter Maydell 
Message-id: 20190603214653.29369-3-js...@redhat.com
Signed-off-by: John Snow 
---
 docs/conf.py |  4 ++--
 docs/sphinx/qmp_lexer.py | 43 
 2 files changed, 45 insertions(+), 2 deletions(-)
 create mode 100644 docs/sphinx/qmp_lexer.py

diff --git a/docs/conf.py b/docs/conf.py
index befbcc6c3e..e46b299b71 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -41,7 +41,7 @@ except NameError:
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use an absolute path starting from qemu_docdir.
 #
-# sys.path.insert(0, os.path.join(qemu_docdir, "my_subdir"))
+sys.path.insert(0, os.path.join(qemu_docdir, "sphinx"))
 
 
 # -- General configuration 
@@ -54,7 +54,7 @@ needs_sphinx = '1.3'
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-extensions = []
+extensions = ['qmp_lexer']
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
diff --git a/docs/sphinx/qmp_lexer.py b/docs/sphinx/qmp_lexer.py
new file mode 100644
index 00..f7e4c0e198
--- /dev/null
+++ b/docs/sphinx/qmp_lexer.py
@@ -0,0 +1,43 @@
+# QEMU Monitor Protocol Lexer Extension
+#
+# Copyright (C) 2019, Red Hat Inc.
+#
+# Authors:
+#  Eduardo Habkost 
+#  John Snow 
+#
+# This work is licensed under the terms of the GNU GPLv2 or later.
+# See the COPYING file in the top-level directory.
+"""qmp_lexer is a Sphinx extension that provides a QMP lexer for code 
blocks."""
+
+from pygments.lexer import RegexLexer, DelegatingLexer
+from pygments.lexers.data import JsonLexer
+from pygments import token
+from sphinx import errors
+
+class QMPExampleMarkersLexer(RegexLexer):
+"""
+QMPExampleMarkersLexer lexes QMP example annotations.
+This lexer adds support for directionality flow and elision indicators.
+"""
+tokens = {
+'root': [
+(r'-> ', token.Generic.Prompt),
+(r'<- ', token.Generic.Prompt),
+(r' ?\.{3} ?', token.Generic.Prompt),
+]
+}
+
+class QMPExampleLexer(DelegatingLexer):
+"""QMPExampleLexer lexes annotated QMP examples."""
+def __init__(self, **options):
+super(QMPExampleLexer, self).__init__(JsonLexer, 
QMPExampleMarkersLexer,
+  token.Error, **options)
+
+def setup(sphinx):
+"""For use by the Sphinx extensions API."""
+try:
+sphinx.require_sphinx('2.1')
+sphinx.add_lexer('QMP', QMPExampleLexer)
+except errors.VersionRequirementError:
+sphinx.add_lexer('QMP', QMPExampleLexer())
-- 
2.21.0

[Qemu-devel] [PULL 1/3] docs/interop/bitmaps.rst: Fix typos

[John Snow]

Pygments and Sphinx get pickier all the time; Sphinx 2.1+ now catches
these errors.

Signed-off-by: John Snow 
Reported-by: Aarushi Mehta 
Reviewed-by: Vladimir Sementsov-Ogievskiy 
Message-id: 20190603214653.29369-2-js...@redhat.com
Signed-off-by: John Snow 
---
 docs/interop/bitmaps.rst | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/docs/interop/bitmaps.rst b/docs/interop/bitmaps.rst
index 510e8809a9..c29ac4a854 100644
--- a/docs/interop/bitmaps.rst
+++ b/docs/interop/bitmaps.rst
@@ -399,7 +399,7 @@ in any one source bitmap, the target bitmap will mark that 
segment dirty.
"arguments": {
  "node": "drive0",
  "target": "new_bitmap",
- "bitmaps: [ "bitmap0" ]
+ "bitmaps": [ "bitmap0" ]
}
  }
 
@@ -1437,7 +1437,7 @@ applied:
.. code:: json
 
 <- {
- "timestamp": {...}
+ "timestamp": {...},
  "data": {
"device": "drive0",
"type": "backup",
-- 
2.21.0

Re: [Qemu-devel] [PATCH 8/8] iotests/257: test traditional sync modes

[John Snow]

On 7/10/19 1:14 PM, Max Reitz wrote:
> On 10.07.19 03:05, John Snow wrote:
>> Signed-off-by: John Snow 
>> ---
>>  tests/qemu-iotests/257 |   31 +
>>  tests/qemu-iotests/257.out | 3089 
>>  2 files changed, 3120 insertions(+)
> 
> Oof.
> 

Yeah, it's... a lot of test output. We probably shouldn't count the
reference test output against any kind of SLOC metrics.

>> diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
>> index de8707cb19..8de1c4da19 100755
>> --- a/tests/qemu-iotests/257
>> +++ b/tests/qemu-iotests/257
> 
> [...]
> 
>> @@ -410,6 +416,11 @@ def test_bitmap_sync(bsync_mode, msync_mode='bitmap', 
>> failure=None):
>>  if bsync_mode == 'always' and failure == 'intermediate':
>>  # We manage to copy one sector (one bit) before the error.
>>  ebitmap.clear_bit(ebitmap.first_bit)
>> +if msync_mode in ('full', 'top'):
>> +# These modes return all bits set except what was 
>> copied/skipped
> 
> Hm.  How useful is bitmap support for 'top' then, anyway?  That means
> that if you want to resume a top backup, you always have to resume it
> like it was a full backup.  Which sounds kind of useless.
> 
> Max
> 

Good point!

I think this can be fixed by doing an initialization pass of the
copy_bitmap when sync=top to set only the allocated regions in the bitmap.

This means that the write notifier won't copy out regions that are
written to that weren't already in the top layer. I believe this is
actually a bugfix; the data we'd copy out in such cases is actually in
the backing layer and shouldn't be copied with sync=top.

So this would have two effects:
(1) sync=top gets a little more judicious about what it copies out on
sync=top, and
(2) the bitmap return value is more meaningful again.

This doesn't touch sync=none at all, which needs more invasive fixes if
we wanted it to have useful bitmap return values (it needs to
differentiate the idea between must-copy and can-copy, and I still don't
know if this is worthwhile to do, so until I hear otherwise, I'm not gonna.)

>> +fail_bit = ebitmap.first_bit
>> +ebitmap.clear()
>> +ebitmap.dirty_bits(range(fail_bit, SIZE // GRANULARITY))
>>  ebitmap.compare(get_bitmap(bitmaps, drive0.device, 'bitmap0'))
>>  
>>  # 2 - Writes and Reference Backup
> [...]
> 

Re: [Qemu-devel] [PATCH] tests/boot_linux_console: add a test for riscv64 + virt

[Alistair Francis]

On Wed, Jul 10, 2019 at 10:51 AM Chih-Min Chao  wrote:
>
> Similar to the mips + malta test, it boots a Linux kernel on a virt
> board and verify the serial is working.  Also, it relies on the serial
> device set by the machine itself.
>
> If riscv64 is a target being built, "make check-acceptance" will
> automatically include this test by the use of the "arch:riscv64" tags.
>
> Alternatively, this test can be run using:
>
>   $ avocado run -t arch:riscv64 tests/acceptance
>
> Signed-off-by: Chih-Min Chao 

Awesome! Thanks for the test case. This will help a lot with RISC-V
regressions in QEMU.

> ---
>  tests/acceptance/boot_linux_console.py | 40 
> ++
>  1 file changed, 40 insertions(+)
>
> diff --git a/tests/acceptance/boot_linux_console.py 
> b/tests/acceptance/boot_linux_console.py
> index 3215950..bbc6b06 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -354,3 +354,43 @@ class BootLinuxConsole(Test):
>  self.vm.launch()
>  console_pattern = 'Kernel command line: %s' % kernel_command_line
>  self.wait_for_console_pattern(console_pattern)
> +
> +def test_riscv64_virt(self):
> +"""
> +:avocado: tags=arch:riscv64
> +:avocado: tags=machine:virt
> +"""
> +
> +kernel_url = ('https://github.com/chihminchao/test-binary/raw/'

These images need to be built with a standard build flow. Having them
built from SiFive's custom scripts will make debugging problems in the
future impossible. I'm also a little worried here about GPL
violations, I'm not sure if it's enough to just point to a script SHA
to meet GPL source disclosure. I know companies have huge headaches
meeting GPL requirements so this seems too easy.

> +  
> '0b7787305d9e40815c05a805266cc74ff356239e/qemu/riscv64/'
> +  'bbl_w_kernel.gz')

Don't use BBL, most people use OpenSBI now which is what we should be
testing with.

> +kernel_hash = 'c7f6cc7967975ad42dc61ee0535db01c9cbd0968'
> +kernel_path_gz = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
> +kernel_path = self.workdir + "bbl_w_kernel"
> +
> +with gzip.open(kernel_path_gz, 'rb') as f_in:
> +with open(kernel_path, 'wb') as f_out:
> +shutil.copyfileobj(f_in, f_out)
> +
> +initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
> +  '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/'
> +  'riscv64/rootfs.cpio.gz')

Same comment about build tools.


Alistair

> +initrd_hash = 'f4867d263754961b6f626cdcdc0cb334c47e3b49'
> +initrd_path = self.fetch_asset(initrd_url, asset_hash=initrd_hash)
> +
> +self.vm.set_machine('virt')
> +self.vm.set_console()
> +kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
> +   + 'console=ttyS0 noreboot')
> +self.vm.add_args('-kernel', kernel_path,
> + '-initrd', initrd_path,
> + '-append', kernel_command_line)
> +self.vm.launch()
> +self.wait_for_console_pattern('Boot successful.')
> +
> +self.exec_command_and_wait_for_pattern('cat /proc/cpuinfo',
> +   'isa')
> +self.exec_command_and_wait_for_pattern('uname -a',
> +   'sifive')
> +self.exec_command_and_wait_for_pattern('reboot',
> +   'reboot: Restarting system')
> --
> 2.7.4
>
>

Re: [Qemu-devel] [RISU RFC PATCH v2 06/14] x86.risu: add MMX instructions

[Jan Bobek]

On 7/3/19 6:01 PM, Peter Maydell wrote:
> On Mon, 1 Jul 2019 at 05:43, Jan Bobek  wrote:
>>
>> Add an x86 configuration file with all MMX instructions.
>>
>> Signed-off-by: Jan Bobek 
> 
>> --- /dev/null
>> +++ b/x86.risu
>> @@ -0,0 +1,96 @@
>> +###
>> +# Copyright (c) 2019 Linaro Limited
> 
> I'm guessing from your email address that this copyright line probably
> isn't right :-)

Haha indeed, I just copy-pasted it from the other files; the same goes for
the rest of the source files.

Any suggestions on what it should be? I'm not currently employed by
anyone (as Google keeps reminding us).

-Jan



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH 7/8] block/backup: support bitmap sync modes for non-bitmap backups

[John Snow]

On 7/10/19 12:48 PM, Max Reitz wrote:
> On 10.07.19 03:05, John Snow wrote:
>> Accept bitmaps and sync policies for the other backup modes.
>> This allows us to do things like create a bitmap synced to a full backup
>> without a transaction, or start a resumable backup process.
>>
>> Some combinations don't make sense, though:
>>
>> - NEVER policy combined with any non-BITMAP mode doesn't do anything,
>>   because the bitmap isn't used for input or output.
>>   It's harmless, but is almost certainly never what the user wanted.
>>
>> - sync=NONE is more questionable. It can't use on-success because this
>>   job never completes with success anyway, and the resulting artifact
>>   of 'always' is suspect: because we start with a full bitmap and only
>>   copy out segments that get written to, the final output bitmap will
>>   always be ... a fully set bitmap.
>>
>>   Maybe there's contexts in which bitmaps make sense for sync=none,
>>   but not without more severe changes to the current job, and omitting
>>   it here doesn't prevent us from adding it later.
>>
>> Signed-off-by: John Snow 
>> ---
>>  block/backup.c   |  8 +---
>>  blockdev.c   | 22 ++
>>  qapi/block-core.json |  6 --
>>  3 files changed, 27 insertions(+), 9 deletions(-)
> 
> [...]
> 
>> diff --git a/blockdev.c b/blockdev.c
>> index f0b7da53b0..bc152f8e0d 100644
>> --- a/blockdev.c
>> +++ b/blockdev.c
> 
> [...]
> 
>> +if (!backup->has_bitmap && backup->has_bitmap_mode) {
>> +error_setg(errp, "Cannot specify Bitmap sync mode without a 
>> bitmap");
> 
> Any reason for capitalizing the first “Bitmap”?
> 
> With a reason or it fixed:
> 
> Reviewed-by: Max Reitz 
> 

Hanging around Germans too much?

Actually, I can explain why: because a "bitmap" is a generic term, but
whenever I capitalize it as "Bitmap" I am referring to a Block Dirty
Bitmap which is a specific sort of thing. I do this unconsciously.

But in that case, I should actually be consistent in the interface (and
docstrings and docs and error strings) and always call it that specific
thing, which I don't.

"bitmap" is probably more correct for now, but I ought to go through all
the interface and make it consistent in some way or another.


(Actually: I'd like to see if I can't rename the "BdrvDirtyBitmap"
object to something more generic and shorter so I can rename many of the
functions we have something shorter.

Because the structure is "BdrvDirtyBitmap", there's some confusion when
we name functions like bdrv_dirty_bitmap_{verb} because it's not clear
if this is a bdrv function that does something with dirty bitmaps, or if
it's a "BdrvDirtyBitmap" function that does something with that object.

I guess that seems like a subtle point, but it's why the naming
conventions in dirty-bitmap.c are all over the place. I think at one
point, the idea was that:

bdrv_{verb}_dirty_bitmap was an action applied to a BDS that happened to
do something with dirty bitmaps. bdrv_dirty_bitmap_{verb} was an action
that applied to a "BdrvDirtyBitmap". Crystal clear and not confusing at
all, right?

It'd be nice to have functions that operate on a node be named
bdrv_dbitmap_foo() and functions that operate on the bitmap structure
itself named just dbitmap_bar().

Would it be okay if I named them such a thing, I wonder?

we have "bitmap" and "hbitmap" already, I could do something like
"dbitmap" for "dirty bitmap" or some such. Kind of an arbitrary change I
admit, but I'm itching to do a big spring cleaning in dirty-bitmap.c
right after this series is done.)

Re: [Qemu-devel] [RISU RFC PATCH v2 06/14] x86.risu: add MMX instructions

[Jan Bobek]

On 7/3/19 5:49 PM, Richard Henderson wrote:
> On 7/1/19 6:35 AM, Jan Bobek wrote:
>> +MOVQMMX  011 d 1110 !emit { rex(w => 1); modrm(mod 
>> => MOD_DIRECT, rm => ~REG_ESP); }
>> +MOVQ_memMMX  011 d 1110 !emit { rex(w => 1); modrm(mod 
>> => ~MOD_DIRECT); mem(size => 8); }
> 
> Oh, note that there are only 8 mmx registers, so the respective rex.{r,b} bit
> can't be set.

Actually, my CPU chewed it without choking even when the bits were
set, but it will taken care of in v3.

-Jan

Re: [Qemu-devel] [RISU RFC PATCH v2 06/14] x86.risu: add MMX instructions

[Jan Bobek]

On 7/3/19 5:35 PM, Richard Henderson wrote:
> On 7/1/19 6:35 AM, Jan Bobek wrote:
>> Add an x86 configuration file with all MMX instructions.
>>
>> Signed-off-by: Jan Bobek 
>> ---
>>  x86.risu | 96 
>>  1 file changed, 96 insertions(+)
>>  create mode 100644 x86.risu
> 
> Note that most of these MMX instructions affect the FPU, not the vector unit.
> We would want to extend risu again to handle this.  You'd also need to seed 
> the
> FPU with random data.
> 
> I was thinking for a moment that this is really beyond what you've signed up
> for, but on second thoughts it's not.  Decoding SSE is really tangled with
> decoding MMX, via the 0x66 prefix, and you'll want to be able to verify that
> you don't regress.

Honestly, I added MMX instructions just for completeness; I figured it can't
hurt, and you can always filter them out via command-line switches. You have
a point with the regression testing, though...

>> +# State Management Instructions
>> +EMMSMMX  01110111 !emit { }
> 
> I'm not sure this is really testable, because of the state change.  But we'll
> see what happens with the aforementioned dumping.
> 
>> +# Arithmetic Instructions
>> +PADDB   MMX  1100 !emit { modrm(); mem(size => 8); }
>> +PADDW   MMX  1101 !emit { modrm(); mem(size => 8); }
>> +PADDD   MMX  1110 !emit { modrm(); mem(size => 8); }
>> +PADDQ   MMX  11010100 !emit { modrm(); mem(size => 8); }

Not this one, at least according to the Intel docs:

NP 0F D4 /r: PADDQ mm, mm/m64  (MMX)
66 0F D4 /r: PADDQ xmm1, xmm2/m128 (SSE2)

The SSE2 version is added in a later patch.

-Jan



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [RISU RFC PATCH v2 04/14] risugen_x86: add module

[Jan Bobek]

On 7/3/19 12:11 PM, Richard Henderson wrote:
> On 7/1/19 6:35 AM, Jan Bobek wrote:
>> +sub write_mov_rr($$)
>> +{
>> +my ($r1, $r2) = @_;
>> +
>> +my %insn = (opcode => X86OP_MOV,
>> +modrm => {mod => MOD_DIRECT,
>> +  reg => ($r1 & 0x7),
>> +  rm => ($r2 & 0x7)});
>> +
>> +$insn{rex}{w} = 1 if $is_x86_64;
>> +$insn{rex}{r} = 1 if $r1 >= 8;
>> +$insn{rex}{b} = 1 if $r2 >= 8;
> 
> This is where maybe it's better to leave rex.[rb] to risugen_x86_asm, and just
> leave $modrm{reg} and $modrm{rm} as 4-bit quantities.

That's what I have in v3, stay tuned!

>> +sub write_mov_reg_imm($$)
>> +{
>> +my ($reg, $imm) = @_;
>> +my %insn;
>> +
>> +if (0 <= $imm && $imm <= 0x) {
> 
> Should include !$is_x86_64 here,
> 
>> +%insn = (opcode => {value => 0xB8 | ($reg & 0x7), len => 1},
>> + imm => {value => $imm, len => 4});
>> +} elsif (-0x8000 <= $imm && $imm <= 0x7fff) {
>> +%insn = (opcode => {value => 0xC7, len => 1},
>> + modrm => {mod => MOD_DIRECT,
>> +   reg => 0, rm => ($reg & 0x7)},
>> + imm => {value => $imm, len => 4});
>> +
>> +$insn{rex}{w} = 1 if $is_x86_64;
> 
> making this unconditional.

Doesn't B8 (without REX.W) work for x86_64, too? It zeroes the upper
part of the destination, so it's effectively zero-extending, and it's
one byte shorter than C7 (no ModR/M byte needed).

That being said, I moved most of this function to risugen_x86_asm and
included a bunch of comments regarding different cases, so it should
be easier to understand.

>> +sub write_random_ymmdata()
>> +{
>> +my $ymm_cnt = $is_x86_64 ? 16 : 8;
>> +my $ymm_len = 32;
>> +my $datalen = $ymm_cnt * $ymm_len;
>> +
>> +# Generate random data blob
>> +write_random_datablock($datalen);
>> +
>> +# Load the random data into YMM regs.
>> +for (my $ymm_reg = 0; $ymm_reg < $ymm_cnt; $ymm_reg++) {
>> +write_insn(vex => {l => VEX_L_256, p => VEX_P_DATA16,
>> +   r => !($ymm_reg >= 8)},
> 
> Again, vex.r should be handled in vex_encode.

As I said, there will be more high-level instruction-assembling
functions exported by risugen_x86_asm in v3, which take care of this.

>> +   opcode => X86OP_VMOVAPS,
>> +   modrm => {mod => MOD_INDIRECT_DISP32,
>> + reg => ($ymm_reg & 0x7),
>> + rm => REG_EAX},
>> +   disp => {value => $ymm_reg * $ymm_len,
>> +len => 4});
>> +}
> 
> So... this now generates code that cannot run without AVX2.
> 
> Which is probably fine for testing right now, since we do
> want to be able to notice effects of SSE/AVX insns on the
> high bits of the registers.
> 
> But we'll probably need to have the same --xsave=foo
> command-line option that we have for risu itself.
> 
> That would let you initialize only 16-bytes here, or
> for avx512 initialize 64-bytes, plus the k-registers.

Ah yes, indeed.

-Jan



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH 6/8] block/backup: issue progress updates for skipped regions

[John Snow]

On 7/10/19 12:36 PM, Max Reitz wrote:
> On 10.07.19 03:05, John Snow wrote:
>> The way bitmap backups work is by starting at 75% if it needs
>> to copy just 25% of the disk.
> 
> Although there is this comment:
> 
>> /* TODO job_progress_set_remaining() would make more sense */
> 
> So...
> 
>> The way sync=top currently works, however, is to start at 0% and then
>> never update the progress if it doesn't copy a region. If it needs to
>> copy 25% of the disk, we'll finish at 25%.
>>
>> Update the progress when we skip regions.
> 
> Wouldn’t it be more correct to decrease the job length?
> 
> Max
> 

Admittedly I have precisely no idea. Maybe? As far as I understand it,
we guarantee only:

(1) Progress monotonically increases
(2) Upon completion, progress will equal the total work estimate.
[Trying to fix that to be true here.]

This means we can do stuff like:

- Total work estimate can increase or decrease arbitrarily
- Neither value has to mean anything in particular


Bitmap sync works by artificially increasing progress for NOP regions,
seen in init_copy_bitmap.

Full sync also tends to increase progress regardless of it actually did
a copy or not; offload support also counts as progress here. So if you
full sync an empty image, you'll see it increasing the progress as it
doesn't actually do anything.

Top sync is the odd one out, which just omits progress for regions it skips.

My only motivation here was to make them consistent. Can I do it the
other way? Yeah, probably. Is one way better than the other? I
legitimately have no idea. I guess whoever wrote the last comment felt
that it should all be the other way instead. Why'd they not do that?

¯\_(ツ)_/¯

>> Signed-off-by: John Snow 
>> ---
>>  block/backup.c | 1 +
>>  1 file changed, 1 insertion(+)
>>
>> diff --git a/block/backup.c b/block/backup.c
>> index a64b768e24..38c4a688c6 100644
>> --- a/block/backup.c
>> +++ b/block/backup.c
>> @@ -417,6 +417,7 @@ static int coroutine_fn backup_loop(BackupBlockJob *job)
>>  if (job->sync_mode == MIRROR_SYNC_MODE_TOP &&
>>  bdrv_is_unallocated_range(bs, offset, job->cluster_size))
>>  {
>> +job_progress_update(>common.job, job->cluster_size);
>>  bdrv_reset_dirty_bitmap(job->copy_bitmap, offset,
>>  job->cluster_size);
>>  continue;
>>
> 
> 

Re: [Qemu-devel] RFC: Why does target/m68k RTE insn. use gen_exception

[Alex Bennée]

Lucien Murray-Pitts  writes:

>> On Wed, Jul 10, 2019 at 4:04 AM Richard Henderson <
> richard.hender...@linaro.org> wrote:
>
>> > I did have a suggestion.  It was fairly detailed.
>> > https://lists.gnu.org/archive/html/qemu-devel/2019-06/msg06522.html
>>
>> Your solution is elegant at about 10 lines that return getl_ilen(pc), but
> it seems the s390 has a far simpler
> instruction word format than the m68k.
>
> However then that got me to thinking, it seems that we can call a portion
> of the TCG system to disassemble a single instruction.
> TranslationBlock tb;
> tb.pc = env->pc;
> gen_intermediate_code(cs, , /* max isn */ 1);
> int ilen = tb.size;
> printf( "PC: %08x sz:%08x\n", env->pc, tb, ilen ) ;
>
> I am very new to TCG, so it does seem there is a lot of code in the
> translator_loop that appears to be interacting with the CPU model/state.
> Should I be worried about this, or is this a safe function to call outside
> of the translator core proper?

I would recommend against it - the time to do stuff like this would be
during translation phase where you can save the data. Don't re-invoke the
translator while trying to process an exception.

Is the instruction format that irregular that you can't do a simple
disassembly in a helper?

> (if everyone is too busy I can dig by myself but I think its going to take
> some time)
>
> Cheers,
> Luc


--
Alex Bennée

Re: [Qemu-devel] [PATCH] tcg/aarch64: Fix output of extract2 opcodes

[Aleksandar Markovic]

On Tue, Jul 9, 2019 at 8:56 PM Richard Henderson
 wrote:
>
> The aarch64 argument ordering for the operands is big-endian,
> whereas the tcg argument ordering is little-endian.  Use REG0
> so that we honor the rZ constraints.

Hello, Richard.

If endian and rZ constraints are unrelated problems, then I think the
commit message
should be:

"This patch fixes two problem:

- endianness: the aarch64 argument ordering for the operands is
big-endian, whereas the tcg argument ordering is little-endian.

- rZ constrains: REG0() macro should be applied to the affected
arguments."

One could argue that in this case the patch this should be actually two patches.
This is better because of bisectability. The number of line in the
patch doesn't matter.

If, on the other hand, endian and rZ constraints are related problems, then you
should explain how in the commit message.

In general, your commit messages appear too terse, and it is hard to establish
logical sense of the changes in question.

Would you be so kind to consider my opinion?

Regards,
Aleksandar

>
> Fixes: 464c2969d5d
> Reported-by: Peter Maydell 
> Signed-off-by: Richard Henderson 
> ---
>  tcg/aarch64/tcg-target.inc.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tcg/aarch64/tcg-target.inc.c b/tcg/aarch64/tcg-target.inc.c
> index b0f8106642..0713448bf5 100644
> --- a/tcg/aarch64/tcg-target.inc.c
> +++ b/tcg/aarch64/tcg-target.inc.c
> @@ -2226,7 +2226,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc,
>
>  case INDEX_op_extract2_i64:
>  case INDEX_op_extract2_i32:
> -tcg_out_extr(s, ext, a0, a1, a2, args[3]);
> +tcg_out_extr(s, ext, a0, REG0(2), REG0(1), args[3]);
>  break;
>
>  case INDEX_op_add2_i32:
> --
> 2.17.1
>
>

Re: [Qemu-devel] [RISU RFC PATCH v2 03/14] risugen_x86_emit: add module

[Jan Bobek]

On 7/3/19 11:47 AM, Richard Henderson wrote:
> On 7/1/19 6:35 AM, Jan Bobek wrote:
>> +sub parse_emitblock($$)
>> +{
>> +my ($rec, $insn) = @_;
>> +my $insnname = $rec->{name};
>> +my $opcode = $insn->{opcode}{value};
>> +
>> +$emit_opts = {};
>> +
>> +my $emitblock = $rec->{blocks}{"emit"};
>> +if (defined $emitblock) {
>> +eval_with_fields($insnname, $opcode, $rec, "emit", $emitblock);
>> +}
> 
> And if !defined?  Silently discard?
> 
> Is this just weirdness higher in the risugen stack,
> such that this might be called maybe_parse_emitblock?

If !defined, there _is_ no emit block, and we treat that as an empty
block. The caller gets an empty hash, and it's up to them to decide
what that means. I could rename it, but the difference doesn't seem
that important to me...?

-Jan



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [RISU RFC PATCH v2 02/14] risugen_x86_asm: add module

[Jan Bobek]

On 7/3/19 11:37 AM, Richard Henderson wrote:
> On 7/1/19 6:35 AM, Jan Bobek wrote:
>> +VEX_V_UNUSED => 0b,
> 
> I think perhaps this is a mistake.  Yes, that's what goes in the field, but
> what goes in the field is ~(logical_value).
> 
> While for general RISU-ish operation, ~(random_number) is just as random as
> +(random_number), the difference will be if we ever want to explicitly emit
> with this interface a specific vex instruction which also requires the 
> v-register.

See below.

>> +sub rex_encode(%)
>> +{
>> +my (%args) = @_;
>> +
>> +$args{w} = 0 unless defined $args{w};
>> +$args{r} = 0 unless defined $args{r};
>> +$args{x} = 0 unless defined $args{x};
>> +$args{b} = 0 unless defined $args{b};
>> +
>> +return (value => 0x40
>> +| (($args{w} ? 1 : 0) << 3)
>> +| (($args{r} ? 1 : 0) << 2)
>> +| (($args{x} ? 1 : 0) << 1)
>> +| ($args{b} ? 1 : 0),
>> +len => 1);
>> +}
> 
> Does
> 
>   (defined $args{w} && $args{w}) << 3
> 
> work?  That seems tidier to me than splitting these conditions.

It does, I will change it. Thanks!

>> +return (value => (0xC4 << 16)
>> +| (($args{r} ? 1 : 0) << 15)
>> +| (($args{x} ? 1 : 0) << 14)
>> +| (($args{b} ? 1 : 0) << 13)
> 
> Further down in vex_encode, and along the lines of VEX_V_UNUSED, this appears
> to be actively wrong, since these bits are encoded as inverses.  What this
> *really* means is that because of that, rex_encode and vex_encode will not
> encode the same registers for a given instruction.  Which really does feel
> bug-like, random inputs or no.

So, vex_encode, rex_encode and friends were meant to be really
low-level functions; they literally just encode the bits from what you
pass in, without any concern for what the fields even mean. In that
spirit, write_insn itself never did much of error-checking.

I have added quite a lot of code to risugen_x86_asm in v3; most
importantly, there are now asm_insn_* functions which are more
high-level, in that you pass in the logical values and they care of
error checks and encoding. I also removed write_insn and all the
encoding-related symbolic constants from the public interface of the
module.

-Jan



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH 5/8] iotests/257: test API failures

[John Snow]

On 7/10/19 12:22 PM, Max Reitz wrote:
> On 10.07.19 03:05, John Snow wrote:
>> Signed-off-by: John Snow 
>> ---
>>  tests/qemu-iotests/257 | 69 +++
>>  tests/qemu-iotests/257.out | 85 ++
>>  2 files changed, 154 insertions(+)
>>
>> diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
>> index 2eb4f26c28..de8707cb19 100755
>> --- a/tests/qemu-iotests/257
>> +++ b/tests/qemu-iotests/257
>> @@ -450,10 +450,79 @@ def test_bitmap_sync(bsync_mode, msync_mode='bitmap', 
>> failure=None):
>>  compare_images(img_path, fbackup2)
>>  log('')
>>  
>> +def test_backup_api():
>> +"""
>> +"""
> 
> Er, OK?
> 
> [...]
> 

Whops.

>> +for sync_mode, config in error_cases.items():
>> +log("-- Sync mode {:s} tests --\n".format(sync_mode))
>> +for bitmap, policies in config.items():
> 
> You might be interested in the fact that the iteration order is
> different for Python2.  Or maybe you aren’t.
> 

asdf. This is undoubtedly the worst thing about Python. I'll have to fix
this, because the ordering isn't guaranteed until 3.5 or some such and
we're only EOLing Python2.

--js

Re: [Qemu-devel] [PATCH 4/8] block/backup: hoist bitmap check into QMP interface

[John Snow]

On 7/10/19 12:11 PM, Max Reitz wrote:
> On 10.07.19 03:05, John Snow wrote:
>> This is nicer to do in the unified QMP interface that we have now,
>> because it lets us use the right terminology back at the user.
>>
>> Signed-off-by: John Snow 
>> ---
>>  block/backup.c | 13 -
>>  blockdev.c | 10 ++
>>  2 files changed, 14 insertions(+), 9 deletions(-)
>>
>> diff --git a/block/backup.c b/block/backup.c
>> index e2729cf6fa..a64b768e24 100644
>> --- a/block/backup.c
>> +++ b/block/backup.c
>> @@ -566,6 +566,10 @@ BlockJob *backup_job_create(const char *job_id, 
>> BlockDriverState *bs,
>>  assert(bs);
>>  assert(target);
>>  
>> +/* QMP interface protects us from these cases */
>> +assert(sync_mode != MIRROR_SYNC_MODE_INCREMENTAL);
>> +assert(sync_bitmap || sync_mode != MIRROR_SYNC_MODE_BITMAP);
> 
> Implication would be a nice operator sometimes.
> 
> ("assert(sync_mode == MIRROR_SYNC_MODE_BITMAP -> sync_bitmap)")
> 
> (Can you do that in C++?  No, you can’t overload bool’s operators, right?)
> 
> Reviewed-by: Max Reitz 
> 

Yes, I also find this assertion kind of hard to read personally, but it
feels somewhat clunky to write:

if (antecedent) {
assert(condition);
}

I suppose we can also phrase this as:

assert(sync_mode == MIRROR_SYNC_MODE_BITMAP ? sync_bitmap : true);

Which might honestly be pretty good. Mind if I change it to this?

--js

Re: [Qemu-devel] [PATCH 3/8] iotests/257: Refactor backup helpers

[John Snow]

On 7/10/19 12:04 PM, Max Reitz wrote:
> On 10.07.19 03:05, John Snow wrote:
>> This test needs support for non-bitmap backups and missing or
>> unspecified bitmap sync modes, so rewrite the helpers to be a little
>> more generic.
>>
>> Signed-off-by: John Snow 
>> ---
>>  tests/qemu-iotests/257 |  46 +
>>  tests/qemu-iotests/257.out | 192 ++---
>>  2 files changed, 124 insertions(+), 114 deletions(-)
>>
>> diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
>> index 2ff4aa8695..2eb4f26c28 100755
>> --- a/tests/qemu-iotests/257
>> +++ b/tests/qemu-iotests/257
> 
> [...]
> 
>> -def bitmap_backup(drive, n, filepath, bitmap, bitmap_mode):
>> -log("--- Bitmap Backup #{:d} ---\n".format(n))
>> -target_id = "bitmap_target_{:d}".format(n)
>> -job_id = "bitmap_backup_{:d}".format(n)
>> +def backup(drive, n, filepath, bitmap, bitmap_mode, sync='bitmap'):
>> +log("--- Test Backup #{:d} ---\n".format(n))
>> +target_id = "backup_target_{:d}".format(n)
>> +job_id = "backup_{:d}".format(n)
>>  target_drive = Drive(filepath, vm=drive.vm)
>>  
>>  target_drive.create_target(target_id, drive.fmt, drive.size)
>> -drive.vm.qmp_log("blockdev-backup", job_id=job_id, device=drive.name,
>> - target=target_id, sync="bitmap",
>> - bitmap_mode=bitmap_mode,
>> - bitmap=bitmap,
>> - auto_finalize=False)
>> +
>> +kwargs = {
>> +'job_id': job_id,
>> +'auto_finalize': False,
>> +'bitmap': bitmap,
>> +'bitmap_mode': bitmap_mode,
>> +}
>> +kwargs = {key: val for key, val in kwargs.items() if val is not None}
> 
> I suppose this is to remove items that are None?
> 
> Very cute, but why not just
> 
>   kwargs = {
> 'job_id': job_id,
> 'auto_finalize': False,
>   }
>   if bitmap is not None:
> kwargs['bitmap'] = bitmap
> kwargs['bitmap_mode'] = bitmap_mode
> 
> Exactly the same number of lines, but immediately makes it clear what’s
> going on.  Not as cute, I admit.
> 
> (Yes, I am indeed actively trying to train you not to write cute code.)
> 

It sneaks in. I genuinely struggle with understanding what other people
will find readable; I have an authentically hard time reviewing other
people's patches too. I'm earnestly not sure how I can help improve
this, but I would like to.

I wasn't sure what the easiest way to avoid sending the "None" over the
wire was, so I went with a general thing, but yes: it's because bitmap
and bitmap_mode are set to None sometimes and I need to omit such keys.

In this case, though, I do test bitmap and bitmap_mode separately, so
for the purposes of testing intentionally bad combinations you do need:

if bitmap is not None:
kwargs['bitmap'] = bitmap
if bitmap_mode is not None:
kwargs['bitmap_mode'] = bitmap_mode

And I just looked at this and it did not spark joy; so I went with a
generic filter to remove nulled keys. I admit it's /slightly/ cute and
not immediately obvious why it needs to be done.


This is even cuter, so maybe I am traveling in the wrong direction:

def backup(drive, n, filepath, sync, **kwargs):
log("--- Test Backup #{:d} ---\n".format(n))
target_id = "backup_target_{:d}".format(n)
job_id = "backup_{:d}".format(n)
target_drive = Drive(filepath, vm=drive.vm)

target_drive.create_target(target_id, drive.fmt, drive.size)
kwargs.setdefault('auto_finalize', False)
# Strip any arguments explicitly nulled by the caller:
kwargs = {key: val for key, val in kwargs.items()
  if val is not None}
blockdev_backup(drive.vm, drive.name, target_id, sync, **kwargs)
return job_id

It's quite a bit shorter and also makes backup() more flexible by
omitting the bitmap and bitmap_mode arguments entirely, allowing the
caller to override the auto_finalize default, etc. In this permutation,
we don't know the full extent of kwargs so it makes sense to generically
filter it.

Manually conditionally setting arguments is probably also fine.
Do you still have a preference for the more static approach?

> The rest looks good to me:
> 
> Reviewed-by: Max Reitz 
> 

Thanks for reviewing, as always!

Re: [Qemu-devel] RFC: Why does target/m68k RTE insn. use gen_exception

[Lucien Murray-Pitts]

> On Wed, Jul 10, 2019 at 4:04 AM Richard Henderson <
richard.hender...@linaro.org> wrote:

> > I did have a suggestion.  It was fairly detailed.
> > https://lists.gnu.org/archive/html/qemu-devel/2019-06/msg06522.html
>
> Your solution is elegant at about 10 lines that return getl_ilen(pc), but
it seems the s390 has a far simpler
instruction word format than the m68k.

However then that got me to thinking, it seems that we can call a portion
of the TCG system to disassemble a single instruction.
TranslationBlock tb;
tb.pc = env->pc;
gen_intermediate_code(cs, , /* max isn */ 1);
int ilen = tb.size;
printf( "PC: %08x sz:%08x\n", env->pc, tb, ilen ) ;

I am very new to TCG, so it does seem there is a lot of code in the
translator_loop that appears to be interacting with the CPU model/state.
Should I be worried about this, or is this a safe function to call outside
of the translator core proper?
(if everyone is too busy I can dig by myself but I think its going to take
some time)

Cheers,
Luc

[Qemu-devel] [PATCH] tests/boot_linux_console: add a test for riscv64 + virt

[Chih-Min Chao]

Similar to the mips + malta test, it boots a Linux kernel on a virt
board and verify the serial is working.  Also, it relies on the serial
device set by the machine itself.

If riscv64 is a target being built, "make check-acceptance" will
automatically include this test by the use of the "arch:riscv64" tags.

Alternatively, this test can be run using:

  $ avocado run -t arch:riscv64 tests/acceptance

Signed-off-by: Chih-Min Chao 
---
 tests/acceptance/boot_linux_console.py | 40 ++
 1 file changed, 40 insertions(+)

diff --git a/tests/acceptance/boot_linux_console.py 
b/tests/acceptance/boot_linux_console.py
index 3215950..bbc6b06 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -354,3 +354,43 @@ class BootLinuxConsole(Test):
 self.vm.launch()
 console_pattern = 'Kernel command line: %s' % kernel_command_line
 self.wait_for_console_pattern(console_pattern)
+
+def test_riscv64_virt(self):
+"""
+:avocado: tags=arch:riscv64
+:avocado: tags=machine:virt
+"""
+
+kernel_url = ('https://github.com/chihminchao/test-binary/raw/'
+  '0b7787305d9e40815c05a805266cc74ff356239e/qemu/riscv64/'
+  'bbl_w_kernel.gz')
+kernel_hash = 'c7f6cc7967975ad42dc61ee0535db01c9cbd0968'
+kernel_path_gz = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+kernel_path = self.workdir + "bbl_w_kernel"
+
+with gzip.open(kernel_path_gz, 'rb') as f_in:
+with open(kernel_path, 'wb') as f_out:
+shutil.copyfileobj(f_in, f_out)
+
+initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
+  '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/'
+  'riscv64/rootfs.cpio.gz')
+initrd_hash = 'f4867d263754961b6f626cdcdc0cb334c47e3b49'
+initrd_path = self.fetch_asset(initrd_url, asset_hash=initrd_hash)
+
+self.vm.set_machine('virt')
+self.vm.set_console()
+kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
+   + 'console=ttyS0 noreboot')
+self.vm.add_args('-kernel', kernel_path,
+ '-initrd', initrd_path,
+ '-append', kernel_command_line)
+self.vm.launch()
+self.wait_for_console_pattern('Boot successful.')
+
+self.exec_command_and_wait_for_pattern('cat /proc/cpuinfo',
+   'isa')
+self.exec_command_and_wait_for_pattern('uname -a',
+   'sifive')
+self.exec_command_and_wait_for_pattern('reboot',
+   'reboot: Restarting system')
-- 
2.7.4

Re: [Qemu-devel] [RISU RFC PATCH v2 01/14] risugen_common: add insnv, randint_constr, rand_fill

[Jan Bobek]

Hi Richard,

sorry for replying so late. I read your comments last week; as I
mentioned in our weekly update email, I ended up adding/removing quite
a lot since v2, so I wasn't 100% sure how much of it will remain
relevant.

Anyways,

On 7/3/19 11:22 AM, Richard Henderson wrote:
> On 7/1/19 6:35 AM, Jan Bobek wrote:
>> +while ($bitcur < $bitend) {
>> +my $format;
>> +my $bitlen;
>> +
>> +if ($bitcur + 64 <= $bitend) {
>> +$format = "Q";
>> +$bitlen = 64;
>> +} elsif ($bitcur + 32 <= $bitend) {
>> +$format = "L";
>> +$bitlen = 32;
>> +} elsif ($bitcur + 16 <= $bitend) {
>> +$format = "S";
>> +$bitlen = 16;
>> +} else {
>> +$format = "C";
>> +$bitlen = 8;
>> +}
>> +
>> +$format .= ($args{bigendian} ? ">" : "<") if $bitlen > 8;
> 
> It now occurs to me to wonder if it's worth simplifying this function to 
> always
> emit bytes, and thus take care of all of the endianness ourselves, since we're
> doing it anyway for larger/odd-sized hunks.

Good point. *facepalm*

I will include this change in v3.

-Jan



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH 1/8] iotests/257: add Pattern class

[John Snow]

On 7/10/19 12:26 PM, Max Reitz wrote:
> On 10.07.19 03:05, John Snow wrote:
>> Just kidding, this is easier to manage with a full class instead of a
>> namedtuple.
>>
>> Signed-off-by: John Snow 
>> ---
>>  tests/qemu-iotests/257 | 58 +++---
>>  1 file changed, 32 insertions(+), 26 deletions(-)
>>
>> diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
>> index 75a651c7c3..f576a35a5c 100755
>> --- a/tests/qemu-iotests/257
>> +++ b/tests/qemu-iotests/257
>> @@ -19,7 +19,6 @@
>>  #
>>  # owner=js...@redhat.com
>>  
>> -from collections import namedtuple
>>  import math
>>  import os
>>  
>> @@ -29,10 +28,18 @@ from iotests import log, qemu_img
>>  SIZE = 64 * 1024 * 1024
>>  GRANULARITY = 64 * 1024
>>  
>> -Pattern = namedtuple('Pattern', ['byte', 'offset', 'size'])
>> -def mkpattern(byte, offset, size=GRANULARITY):
>> -"""Constructor for Pattern() with default size"""
>> -return Pattern(byte, offset, size)
>> +
>> +class Pattern:
>> +def __init__(self, byte, offset, size=GRANULARITY):
>> +self.byte = byte
>> +self.offset = offset
>> +self.size = size
>> +
>> +def bits(self, granularity):
>> +lower = math.floor(self.offset / granularity)
>> +upper = math.floor((self.offset + self.size - 1) / granularity)
>> +return set(range(lower, upper + 1))
> 
> By the way, this doesn’t work with Python2 (pre-existing in your other
> series).  It complains that these are floats.
> 
> Now I don’t know whether you care but there is the fact that the
> expressions would be shorter if they were of the form x // y instead of
> math.floor(x / y).
> 
> Max
> 

Ah, crud; OK -- I'll play nice with python2 for a while longer. Thanks
for pointing this out.

Re: [Qemu-devel] [PATCH 2/8] iotests/257: add EmulatedBitmap class

[John Snow]

On 7/10/19 11:47 AM, Max Reitz wrote:
> On 10.07.19 03:05, John Snow wrote:
>> Represent a bitmap with an object that we can mark and clear bits in.
>> This makes it easier to manage partial writes when we don't write a
>> full group's worth of patterns before an error.
>>
>> Signed-off-by: John Snow 
>> ---
>>  tests/qemu-iotests/257 | 125 +
>>  1 file changed, 76 insertions(+), 49 deletions(-)
>>
>> diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
>> index f576a35a5c..2ff4aa8695 100755
>> --- a/tests/qemu-iotests/257
>> +++ b/tests/qemu-iotests/257
>> @@ -85,6 +85,60 @@ GROUPS = [
>>  Pattern('0xdd', 0x3fc)]), # New; leaving a gap to the right
>>  ]
>>  
>> +
>> +class EmulatedBitmap:
>> +def __init__(self, granularity=GRANULARITY):
>> +self._bits = set()
>> +self.groups = set()
>> +self.granularity = granularity
>> +
>> +def dirty_bits(self, bits):
>> +self._bits |= set(bits)
>> +
>> +def dirty_group(self, n):
>> +self.dirty_bits(GROUPS[n].bits(self.granularity))
>> +
>> +def clear(self):
>> +self._bits = set()
>> +
>> +def clear_bits(self, bits):
>> +self._bits = self._bits - set(bits)
> 
> Does -= not work here?
> 
> No real complaints.  Sorry.
> 
> Reviewed-by: Max Reitz 
> 

It's okay. I forget which shorthand operators Python has for which types
sometimes.

>>> a = {'a', 'b', 'c'}
>>> a -= {'c'}
>>> a
{'b', 'a'}

Well, apparently it does. I'll change it.

--js

[Qemu-devel] [Bug 1836078] Re: Regressions on arm-linux-gnueabihf target with some GCC tests

[Alex Bennée]

In the ieee6 test case it is attempting to write OFE, bit [10] which:

This bit is RW only if the implementation supports the trapping of
floating-point exceptions. In an implementation that does not support
floating-point exception trapping, this bit is RAZ/WI.

When this bit is RW, it applies only to floating-point operations.
Advanced SIMD operations always use untrapped floating-point exception
handling in AArch32 state

This might be a broken test.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1836078

Title:
  Regressions on arm-linux-gnueabihf target with some GCC tests

Status in QEMU:
  In Progress

Bug description:
  Hi,

  After trying qemu master:
  commit 474f3938d79ab36b9231c9ad3b5a9314c2aeacde
  Merge: 68d7ff0 14f5d87
  Author: Peter Maydell 
  Date: Fri Jun 21 15:40:50 2019 +0100

  even with the fix for https://bugs.launchpad.net/qemu/+bug/1834496,
  I've noticed several regressions compared to qemu-3.1 when running the GCC 
testsuite.
  I'm attaching a tarball containing several GCC tests (binaries), needed 
shared libs, and a short script to run all the tests.

  All tests used to pass w/o error, but with a recent qemu, all of them
  make qemu crash.

  This was noticed with GCC master configured with
  --target arm-none-linux-gnueabihf
  --with-cpu cortex-a57
  --with-fpu crypto-neon-fp-armv8

  Thanks

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1836078/+subscriptions

Re: [Qemu-devel] [PATCH] tcg/aarch64: Fix output of extract2 opcodes

[Aleksandar Markovic]

> Can you be less combative in your review comments, please?

Sure, I will.

Thanks, Aleksandar

[Qemu-devel] [PATCH for 4.1] target/arm: report ARMv8.2 FP16 for ARM -cpu max

[Alex Bennée]

When we converted to using feature bits in 602f6e42cfbf we missed out
the fact (dp && arm_dc_feature(s, ARM_FEATURE_V8)) was supported for
-cpu max configurations. This caused a regression in the GCC test
suite. Fix this by setting the appropriate FP16 bits in mvfr1.FPHP.

Fixes: https://bugs.launchpad.net/qemu/+bug/1836078
Signed-off-by: Alex Bennée 
---
 target/arm/cpu.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index e75a64a25a..0a0a202fe3 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -2452,6 +2452,10 @@ static void arm_max_initfn(Object *obj)
 t = FIELD_DP32(t, ID_ISAR6, SPECRES, 1);
 cpu->isar.id_isar6 = t;
 
+t = cpu->isar.mvfr1;
+t = FIELD_DP32(t, MVFR1, FPHP, 2); /* v8.2 FP16 */
+cpu->isar.mvfr1 = t;
+
 t = cpu->isar.mvfr2;
 t = FIELD_DP32(t, MVFR2, SIMDMISC, 3); /* SIMD MaxNum */
 t = FIELD_DP32(t, MVFR2, FPMISC, 4);   /* FP MaxNum */
-- 
2.20.1