Re: [Qemu-devel] [PATCH 3/4] RFC target/arm: Do not build pre-ARMv7 cpus when using KVM

[Thomas Huth]

On 29/08/2019 20.19, Philippe Mathieu-Daudé wrote:
> Hi Thomas,
> 
> On 8/23/19 4:28 PM, Thomas Huth wrote:
>> On 8/23/19 3:58 PM, Philippe Mathieu-Daudé wrote:
>>> A KVM-only build won't be able to run pre-ARMv7 cpus, disable them.
>>>
>>> If KVM is not enabled, they are enabled by default.
>> [...]
>>>  config CHEETAH
>>>  bool
>>> +select ARM_V4
>>>  select OMAP
>>>  select TSC210X
>>
>> Are you sure about the "enabled by default" ? There is not "default y"
>> here, is it?
> 
> What I mean is if you build with --disable-kvm, this selects
> --enable-tcg which provides the pre-ARMv7 cpus. So to make no changes, I
> also added:
> 
>   config ARM_V4
>   default y
> 
> Which include the "default y".

Well, so the ARM_V4 config switch is enabled by default. But where is
the CHEETAH config switch enabled now?

>> I think we should maybe rather rework the default-configs directory:
>> Rename the default to "config/default/" instead and then we can add
>> other subfolders with such special configurations, e.g. config/nemu/ or
>> config/lean-kvm/ or however you want to call it. Then add a new switch
>> to the configure script to be able to use the configs from such a
>> different folder.
> 
> OK so if someone wants a special config, he'd know the config values to
> select, so it is pointless/confusing to keep them commented.
> Are you suggesting to simply remove the default entries?
Certainly not! I meant to keep the current file (with everything
enabled) in config/default/, and to add another config file to
config/lean-kvm/ where the TCG-only boards are disabled. Then the user
can easily run "./configure --build-config-dir=config/lean-kvm/" to
enable these settings.

 Thomas

Re: [Qemu-devel] [PATCH v3 0/4] Introduce the microvm machine type

[Jing Liu]

Hi Sergio,

On 8/29/2019 11:46 PM, Sergio Lopez wrote:


Jing Liu  writes:


Hi Sergio,

The idea is interesting and I tried to launch a guest by your
guide but seems failed to me. I tried both legacy and normal modes,
but the vncviewer connected and told me that:
The vm has no graphic display device.
All the screen in vnc is just black.


The microvm machine type doesn't support any graphics device, so you
need to rely on the serial console.

Got it.




kernel config:
CONFIG_KVM_MMIO=y
CONFIG_VIRTIO_MMIO=y

I don't know if any specified kernel version/patch/config
is needed or anything I missed.
Could you kindly give some tips?


I'm testing it with upstream vanilla Linux. In addition to MMIO, you
need to add support for PVH (the next version of this patchset, v4, will
support booting from FW, so it'll be possible to use non-PVH ELF kernels
and bzImages too).

I've just uploaded a working kernel config here:

https://gist.github.com/slp/1060ba3aaf708584572ad4109f28c8f9


Thanks very much and this config is helpful to me.


As for the QEMU command line, something like this should do the trick:

./x86_64-softmmu/qemu-system-x86_64 -smp 1 -m 1g -enable-kvm -M microvm,legacy -kernel 
vmlinux -append "earlyprintk=ttyS0 console=ttyS0 reboot=k panic=1" -nodefaults 
-no-user-config -nographic -serial stdio

If this works, you can move to non-legacy mode with a virtio-console:

./x86_64-softmmu/qemu-system-x86_64 -smp 1 -m 1g -enable-kvm -M microvm -kernel vmlinux 
-append "console=hvc0 reboot=k panic=1" -nodefaults -no-user-config -nographic 
-serial pty -chardev stdio,id=virtiocon0,server -device virtio-serial-device -device 
virtconsole,chardev=virtiocon0


I tried the above two ways and it works now. Thanks!


If is still working, you can try adding some devices too:

./x86_64-softmmu/qemu-system-x86_64 -smp 1 -m 1g -enable-kvm -M microvm -kernel vmlinux 
-append "console=hvc0 reboot=k panic=1 root=/dev/vda" -nodefaults 
-no-user-config -nographic -serial pty -chardev stdio,id=virtiocon0,server -device 
virtio-serial-device -device virtconsole,chardev=virtiocon0 -netdev user,id=testnet 
-device virtio-net-device,netdev=testnet -drive 
id=test,file=alpine-rootfs-x86_64.raw,format=raw,if=none -device 
virtio-blk-device,drive=test


But I'm wondering why the image I used can not be found.
root=/dev/vda3 and the same image worked well on normal qemu/guest-
config bootup, but didn't work here. The details are,

-append "console=hvc0 reboot=k panic=1 root=/dev/vda3 rw rootfstype=ext4" \

[0.022784] Key type encrypted registered
[0.022988] VFS: Cannot open root device "vda3" or 
unknown-block(254,3): error -6
[0.023041] Please append a correct "root=" boot option; here are the 
available partitions:

[0.023089] fe00 8946688 vda
[0.023090]  driver: virtio_blk
[0.023143] Kernel panic - not syncing: VFS: Unable to mount root fs 
on unknown-block(254,3)

[0.023201] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.3.0-rc3 #23


BTW, root=/dev/vda is also tried and didn't work. The dmesg is a little 
different:


[0.028050] Key type encrypted registered
[0.028484] List of all partitions:
[0.028529] fe00 8946688 vda
[0.028529]  driver: virtio_blk
[0.028615] No filesystem could mount root, tried:
[0.028616]  ext4
[0.028670]
[0.028712] Kernel panic - not syncing: VFS: Unable to mount root fs 
on unknown-block(254,0)


I tried another ext4 img but still doesn't work.
Is there any limitation of blk image? Could I copy your image for simple
test?

Thanks in advance,
Jing


Sergio.


Thanks very much.
Jing




A QEMU instance with the microvm machine type can be invoked this way:

   - Normal mode:

qemu-system-x86_64 -M microvm -m 512m -smp 2 \
   -kernel vmlinux -append "console=hvc0 root=/dev/vda" \
   -nodefaults -no-user-config \
   -chardev pty,id=virtiocon0,server \
   -device virtio-serial-device \
   -device virtconsole,chardev=virtiocon0 \
   -drive id=test,file=test.img,format=raw,if=none \
   -device virtio-blk-device,drive=test \
   -netdev tap,id=tap0,script=no,downscript=no \
   -device virtio-net-device,netdev=tap0

   - Legacy mode:

qemu-system-x86_64 -M microvm,legacy -m 512m -smp 2 \
   -kernel vmlinux -append "console=ttyS0 root=/dev/vda" \
   -nodefaults -no-user-config \
   -drive id=test,file=test.img,format=raw,if=none \
   -device virtio-blk-device,drive=test \
   -netdev tap,id=tap0,script=no,downscript=no \
   -device virtio-net-device,netdev=tap0 \
   -serial stdio

Re: [Qemu-devel] [PATCH v2 4/8] exec: Factor out cpu_watchpoint_address_matches

[Richard Henderson]

On 8/29/19 10:20 AM, Philippe Mathieu-Daudé wrote:
>> +/* Avoid trapping reads of pages with a write breakpoint. */
>> +match = (prot & PAGE_READ ? BP_MEM_READ : 0)
>> +  | (prot & PAGE_WRITE ? BP_MEM_WRITE : 0);
> 
> Isn't it cheaper to do here:
> 
>if (!match) {
>return iotlb;
>}
> 
> or
> 
>if (match) {

Note that PROT_NONE pages never reach here; they always trap in tlb_fill.

The only way we can get match == 0 here is for the case of an execute-only
page.  Which is possible, but extremely unlikely.  Almost all targets merge the
text and rodata sections, which means that virtually all executable pages are
also readable.

(Although I must say that in this age of ROP-gadgets, leaving the rodata
section executable is probably a mistake, and tools should be updated to *not*
merge them.  That's still not necessarily execute-only for the text section,
but I don't see anything in principal that would prevent it.)


r~

Re: [Qemu-devel] [PATCH for-4.2 v10 03/15] virtio-iommu: Add skeleton

[Peter Xu]

On Thu, Aug 29, 2019 at 02:18:42PM +0200, Auger Eric wrote:
> Hi Peter,
> 
> First of all, please forgive me for the delay.
> On 8/15/19 3:54 PM, Peter Xu wrote:
> > On Tue, Jul 30, 2019 at 07:21:25PM +0200, Eric Auger wrote:
> >> +static void virtio_iommu_handle_command(VirtIODevice *vdev, VirtQueue *vq)
> >> +{
> >> +VirtIOIOMMU *s = VIRTIO_IOMMU(vdev);
> >> +struct virtio_iommu_req_head head;
> >> +struct virtio_iommu_req_tail tail;
> > 
> > [1]
> > 
> >> +VirtQueueElement *elem;
> >> +unsigned int iov_cnt;
> >> +struct iovec *iov;
> >> +size_t sz;
> >> +
> >> +for (;;) {
> >> +elem = virtqueue_pop(vq, sizeof(VirtQueueElement));
> >> +if (!elem) {
> >> +return;
> >> +}
> >> +
> >> +if (iov_size(elem->in_sg, elem->in_num) < sizeof(tail) ||
> >> +iov_size(elem->out_sg, elem->out_num) < sizeof(head)) {
> >> +virtio_error(vdev, "virtio-iommu bad head/tail size");
> >> +virtqueue_detach_element(vq, elem, 0);
> >> +g_free(elem);
> >> +break;
> >> +}
> >> +
> >> +iov_cnt = elem->out_num;
> >> +iov = g_memdup(elem->out_sg, sizeof(struct iovec) * 
> >> elem->out_num);
> > 
> > Could I ask why memdup is needed here?
> Indeed I don't think it is needed and besides iov is not freed!
> 
> I got inspired from hw/net/virtio-net.c. To be honest I don't get why
> the g_memdup is needed there either. The out_sg gets duplicated and
> commands work on the duplicated data and not in place.

Oh true, I found that it's because of calling of iov_discard_front().
Please have a look at 771b6ed37e3.  Though it seems to me that
virtio-iommu does not truncate iovs so it should not be needed.

> > 
> >> +sz = iov_to_buf(iov, iov_cnt, 0, , sizeof(head));
> >> +if (unlikely(sz != sizeof(head))) {
> >> +tail.status = VIRTIO_IOMMU_S_DEVERR;
> > 
> > Do you need to zero the reserved bits to make sure it won't contain
> > garbage?  Same question to below uses of tail.
> yes. I initialized tail.
> > 
> >> +goto out;
> >> +}
> >> +qemu_mutex_lock(>mutex);
> >> +switch (head.type) {
> >> +case VIRTIO_IOMMU_T_ATTACH:
> >> +tail.status = virtio_iommu_handle_attach(s, iov, iov_cnt);
> >> +break;
> >> +case VIRTIO_IOMMU_T_DETACH:
> >> +tail.status = virtio_iommu_handle_detach(s, iov, iov_cnt);
> >> +break;
> >> +case VIRTIO_IOMMU_T_MAP:
> >> +tail.status = virtio_iommu_handle_map(s, iov, iov_cnt);
> >> +break;
> >> +case VIRTIO_IOMMU_T_UNMAP:
> >> +tail.status = virtio_iommu_handle_unmap(s, iov, iov_cnt);
> >> +break;
> >> +default:
> >> +tail.status = VIRTIO_IOMMU_S_UNSUPP;
> >> +}
> >> +qemu_mutex_unlock(>mutex);
> >> +
> >> +out:
> >> +sz = iov_from_buf(elem->in_sg, elem->in_num, 0,
> >> +  , sizeof(tail));
> >> +assert(sz == sizeof(tail));
> >> +
> >> +virtqueue_push(vq, elem, sizeof(tail));
> > 
> > s/tail/head/ (though they are the same size)?
> That's unclear to me. Similarly when checking against virtio-net.c, the
> element is pushed back to the used ring and len is set to the size of
> the status with:
> 
> /*
>  * Control virtqueue data structures
>  *
>  * The control virtqueue expects a header in the first sg entry
>  * and an ack/status response in the last entry.  Data for the
>  * command goes in between.
>  */

I was referencing the balloon code when reading the patch, e.g.,
virtio_balloon_handle_output().  Though after I read more carefully I
see that other places are using it as you described.  Now I tend to
agree with you, because virtqueue_push() who calls
virtqueue_unmap_sg() used the len to unmap in_sg[] rather than
out_sg[].  So please ignore my previous comment.

(then I'm not sure whether the usage in the balloon code was correct
 now...)

> > 
> >> +virtio_notify(vdev, vq);
> >> +g_free(elem);
> >> +}
> >> +}
> > 
> > [...]
> > 
> >> +static void virtio_iommu_set_features(VirtIODevice *vdev, uint64_t val)
> >> +{
> >> +VirtIOIOMMU *dev = VIRTIO_IOMMU(vdev);
> >> +
> >> +dev->acked_features = val;
> >> +trace_virtio_iommu_set_features(dev->acked_features);
> >> +}
> >> +
> >> +static const VMStateDescription vmstate_virtio_iommu_device = {
> >> +.name = "virtio-iommu-device",
> >> +.unmigratable = 1,
> > 
> > Curious, is there explicit reason to not support migration from the
> > first version? :)
> The state is made of red black trees, lists. For the former there is no
> VMSTATE* ready. I am working on it but I think this should be handled
> separately

Fair enough.  Would you mind to add a similar comment above
unmigratable?

Thanks,

-- 
Peter Xu

Re: [Qemu-devel] [PATCH v2 2/8] exec: Factor out core logic of check_watchpoint()

[Richard Henderson]

On 8/29/19 10:26 AM, Philippe Mathieu-Daudé wrote:
>> -wp->hitaddr = vaddr;
>> +wp->hitaddr = MAX(addr, wp->vaddr);
> 
> When is addr > wp->vaddr?

Both the watchpoint and the access are arbitrary ranges.

  wp:[ 1000   - 1008 ]
  store: [ 1002 - 1004 ]

  wp:   [ 1004- 1008 ]
  store: [ 1000   - 1008 ]

The old code would, for the first case, return 1002 and not the 1000 of the
watch point, which seems reasonable.  For the second case, we would set 1000,
an address outside of the watchpoint.

David's change makes sure that the address signaled is inside the watchpoint.
I.e. leaving the first case unchanged and making the second  set 1004.

It seems very reasonable to me.


r~

Re: [Qemu-devel] [PATCH] RISCV: support riscv vector extension 0.7.1

[Alistair Francis]

On Thu, Aug 29, 2019 at 5:05 AM liuzhiwei  wrote:
>
> On 2019/8/29 上午5:34, Alistair Francis wrote:
> > On Wed, Aug 28, 2019 at 12:04 AM liuzhiwei  wrote:
> >> Change-Id: I3cf891bc400713b95f47ecca82b1bf773f3dcb25
> >> Signed-off-by: liuzhiwei 
> >> ---
> >>   fpu/softfloat.c |   119 +
> >>   include/fpu/softfloat.h | 4 +
> >>   linux-user/riscv/cpu_loop.c | 8 +-
> >>   target/riscv/Makefile.objs  | 2 +-
> >>   target/riscv/cpu.h  |30 +
> >>   target/riscv/cpu_bits.h |15 +
> >>   target/riscv/cpu_helper.c   | 7 +
> >>   target/riscv/csr.c  |65 +-
> >>   target/riscv/helper.h   |   354 +
> >>   target/riscv/insn32.decode  |   374 +-
> >>   target/riscv/insn_trans/trans_rvv.inc.c |   484 +
> >>   target/riscv/translate.c| 1 +
> >>   target/riscv/vector_helper.c| 26563 
> >> ++
> >>   13 files changed, 28017 insertions(+), 9 deletions(-)
> >>   create mode 100644 target/riscv/insn_trans/trans_rvv.inc.c
> >>   create mode 100644 target/riscv/vector_helper.c
> >>
> > Hello,
> >
> > Thanks for the patch!
> >
> > As others have pointed out you will need to split the patch up into
> > multiple smaller patches, otherwise it is too hard to review almost
> > 30,000 lines of code.
>
> Hi, Alistair
>
> I'm so sorry for the inconvenience. It will be a patch set with a cover
> letter in V2.

No worries.

>
> > Can you also include a cover letter with your patch series describing
> > how you are testing this? AFAIK vector extension support isn't in any
> > compiler so I'm assuming you are handwriting the assembly or have
> > toolchain patches. Either way it will help if you can share that so
> > others can test your implementation.
>
> Yes, it's handwriting assembly. The assembler in Binutils has support
> Vector extension.  First define an function test_vadd_vv_8 in assembly
> and then it can be called from a C program.
>
> The function is something like
>
> /* vadd.vv */
> TEST_FUNC(test_vadd_vv_8)
>  vsetvlit1, x0, e8, m2
>  vlb.v   v6, (a4)
>  vsb.v   v6, (a3)
>  vsetvlit1, a0, e8, m2
>  vlb.v   v0, (a1)
>  vlb.v   v2, (a2)
>  vadd.vv v4, v0, v2
>  vsb.v  v4, (a3)
> ret
>  .size   test_vadd_vv_8, .-test_vadd_vv_8

If possible it might be worth releasing the code that you are using for testing.

>
> It takes more time to test than to implement the instructions. Maybe
> there is some better test method or some forced test cases in QEMU.
> Could you give me some advice for testing?

Richard's idea of risu seems like a good option.

Thinking about it a bit more we are going to have other extensions in
the future that will need assembly testing so setting up a test
framework seems like a good idea. I am happy to help try and get this
going as well.

Alistair

>
> Best Regards,
>
> Zhiwei
>
> > Alex and Richard have kindly started the review. Once you have
> > addressed their comments and split this patch up into smaller patches
> > you can send a v2 and we can go from there.
> >
> > Once again thanks for doing this implementation for QEMU!
> >
> > Alistair
> >

Re: [Qemu-devel] [PATCH] migration: Do not re-read the clock on pre_save in case of paused guest

[Eduardo Habkost]

CCing Marcelo, who wrote kvm_update_clock() and
kvmclock_pre_save().

On Thu, Aug 29, 2019 at 06:07:11PM -0300, Maxiwell S. Garcia wrote:
> The clock move makes the guest knows about the paused time between the
> 'stop' and 'migrate' commands. This is an issue in an already-paused
> VM because some side effects, like process stalls, could happen
> after migration.
> 
> So, this patch checks the runstate of guest in the pre_save handler and
> do not re-reads the clock in case of paused state (cold migration).
> 
> Signed-off-by: Maxiwell S. Garcia 
> ---
>  hw/i386/kvm/clock.c | 15 +++
>  1 file changed, 11 insertions(+), 4 deletions(-)
> 
> diff --git a/hw/i386/kvm/clock.c b/hw/i386/kvm/clock.c
> index 80c133a724..2c59b6894b 100644
> --- a/hw/i386/kvm/clock.c
> +++ b/hw/i386/kvm/clock.c
> @@ -41,6 +41,9 @@ typedef struct KVMClockState {
>  uint64_t clock;
>  bool clock_valid;
>  
> +/* whether the 'clock' value was obtained in the 'paused' state */
> +bool runstate_paused;
> +
>  /* whether machine type supports reliable KVM_GET_CLOCK */
>  bool mach_use_reliable_get_clock;
>  
> @@ -202,6 +205,8 @@ static void kvmclock_vm_state_change(void *opaque, int 
> running,
>  return;
>  }
>  
> +s->runstate_paused = runstate_check(RUN_STATE_PAUSED);
> +
>  kvm_synchronize_all_tsc();
>  
>  kvm_update_clock(s);
> @@ -260,9 +265,9 @@ static int kvmclock_pre_load(void *opaque)
>  }
>  
>  /*
> - * When migrating, read the clock just before migration,
> - * so that the guest clock counts during the events
> - * between:
> + * When migrating a running guest, read the clock just
> + * before migration, so that the guest clock counts
> + * during the events between:
>   *
>   *  * vm_stop()
>   *  *
> @@ -277,7 +282,9 @@ static int kvmclock_pre_save(void *opaque)
>  {
>  KVMClockState *s = opaque;
>  
> -kvm_update_clock(s);
> +if (!s->runstate_paused) {
> +kvm_update_clock(s);
> +}
>  
>  return 0;
>  }
> -- 
> 2.20.1
> 

-- 
Eduardo

[Qemu-devel] [PATCH] migration: Do not re-read the clock on pre_save in case of paused guest

[Maxiwell S. Garcia]

The clock move makes the guest knows about the paused time between the
'stop' and 'migrate' commands. This is an issue in an already-paused
VM because some side effects, like process stalls, could happen
after migration.

So, this patch checks the runstate of guest in the pre_save handler and
do not re-reads the clock in case of paused state (cold migration).

Signed-off-by: Maxiwell S. Garcia 
---
 hw/i386/kvm/clock.c | 15 +++
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/hw/i386/kvm/clock.c b/hw/i386/kvm/clock.c
index 80c133a724..2c59b6894b 100644
--- a/hw/i386/kvm/clock.c
+++ b/hw/i386/kvm/clock.c
@@ -41,6 +41,9 @@ typedef struct KVMClockState {
 uint64_t clock;
 bool clock_valid;
 
+/* whether the 'clock' value was obtained in the 'paused' state */
+bool runstate_paused;
+
 /* whether machine type supports reliable KVM_GET_CLOCK */
 bool mach_use_reliable_get_clock;
 
@@ -202,6 +205,8 @@ static void kvmclock_vm_state_change(void *opaque, int 
running,
 return;
 }
 
+s->runstate_paused = runstate_check(RUN_STATE_PAUSED);
+
 kvm_synchronize_all_tsc();
 
 kvm_update_clock(s);
@@ -260,9 +265,9 @@ static int kvmclock_pre_load(void *opaque)
 }
 
 /*
- * When migrating, read the clock just before migration,
- * so that the guest clock counts during the events
- * between:
+ * When migrating a running guest, read the clock just
+ * before migration, so that the guest clock counts
+ * during the events between:
  *
  *  * vm_stop()
  *  *
@@ -277,7 +282,9 @@ static int kvmclock_pre_save(void *opaque)
 {
 KVMClockState *s = opaque;
 
-kvm_update_clock(s);
+if (!s->runstate_paused) {
+kvm_update_clock(s);
+}
 
 return 0;
 }
-- 
2.20.1

Re: [Qemu-devel] [PATCH 0/2] tests/acceptance: Update MIPS Malta ssh test

[Eduardo Habkost]

On Fri, Aug 02, 2019 at 05:35:56PM +0200, Aleksandar Markovic wrote:
> From: Aleksandar Markovic 
> 
> This little series improves linux_ssh_mips_malta.py, both in the sense
> of code organization and in the sense of quantity of executed tests.

Thanks!  I'm queueing it on python-next.  The changes suggested
by others can be implemented as follow up patches.


> 
> Aleksandar Markovic (2):
>   tests/acceptance: Refactor and improve reporting in
> linux_ssh_mips_malta.py
>   tests/acceptance: Add new test cases in linux_ssh_mips_malta.py
> 
>  tests/acceptance/linux_ssh_mips_malta.py | 81 
> ++--
>  1 file changed, 66 insertions(+), 15 deletions(-)
> 
> -- 
> 2.7.4
> 
> 

-- 
Eduardo

[Qemu-devel] [Bug 1841990] [NEW] instruction 'denbcdq' misbehaving

[Paul Clarke]

Public bug reported:

Instruction 'denbcdq' appears to have no effect.  Test case attached.

On ppc64le native:
--
gcc -g -O -mcpu=power9 bcdcfsq.c test-denbcdq.c -o test-denbcdq
$ ./test-denbcdq
0x
0x000c
0x2208
$ ./test-denbcdq 1
0x0001
0x001c
0x22080001
$ ./test-denbcdq $(seq 0 99)
0x0064
0x100c
0x22080080
--

With "qemu-ppc64le -cpu power9"
--
$ qemu-ppc64le -cpu power9 -L [...] ./test-denbcdq
0x
0x000c
0x000c
$ qemu-ppc64le -cpu power9 -L [...] ./test-denbcdq 1
0x0001
0x001c
0x001c
$ qemu-ppc64le -cpu power9 -L [...] ./test-denbcdq $(seq 100)
0x0064
0x100c
0x100c
--

I started looking at the code, but I got confused rather quickly.  Could
be related to endianness? I think denbcdq arrived on the scene before
little-endian was a big deal.  Maybe something to do with utilizing
implicit floating-point register pairs...  I don't think the right data
is getting to helper_denbcdq, which would point back to the gen_fprp_ptr
uses in dfp-impl.inc.c (GEN_DFP_T_FPR_I32_Rc).  (Maybe?)

** Affects: qemu
 Importance: Undecided
 Status: New

** Attachment added: "test case for using denbcdq"
   
https://bugs.launchpad.net/bugs/1841990/+attachment/5285701/+files/test-denbcdq.c

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1841990

Title:
  instruction 'denbcdq' misbehaving

Status in QEMU:
  New

Bug description:
  Instruction 'denbcdq' appears to have no effect.  Test case attached.

  On ppc64le native:
  --
  gcc -g -O -mcpu=power9 bcdcfsq.c test-denbcdq.c -o test-denbcdq
  $ ./test-denbcdq
  0x
  0x000c
  0x2208
  $ ./test-denbcdq 1
  0x0001
  0x001c
  0x22080001
  $ ./test-denbcdq $(seq 0 99)
  0x0064
  0x100c
  0x22080080
  --

  With "qemu-ppc64le -cpu power9"
  --
  $ qemu-ppc64le -cpu power9 -L [...] ./test-denbcdq
  0x
  0x000c
  0x000c
  $ qemu-ppc64le -cpu power9 -L [...] ./test-denbcdq 1
  0x0001
  0x001c
  0x001c
  $ qemu-ppc64le -cpu power9 -L [...] ./test-denbcdq $(seq 100)
  0x0064
  0x100c
  0x100c
  --

  I started looking at the code, but I got confused rather quickly.
  Could be related to endianness? I think denbcdq arrived on the scene
  before little-endian was a big deal.  Maybe something to do with
  utilizing implicit floating-point register pairs...  I don't think the
  right data is getting to helper_denbcdq, which would point back to the
  gen_fprp_ptr uses in dfp-impl.inc.c (GEN_DFP_T_FPR_I32_Rc).  (Maybe?)

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1841990/+subscriptions

Re: [Qemu-devel] [PATCH] target/arm: Free TCG temps in trans_VMOV_64_sp()

[Alex Bennée]

Aleksandar Markovic  writes:

> 27.08.2019. 14.20, "Peter Maydell"  је написао/ла:
>>
>> The function neon_store_reg32() doesn't free the TCG temp that it
>> is passed, so the caller must do that. We got this right in most
>> places but forgot to free the TCG temps in trans_VMOV_64_sp().
>>
>> Cc: qemu-sta...@nongnu.org
>> Signed-off-by: Peter Maydell 
>> ---
>
> Hello, Peter,
>
> I am just curious if you found this by manual code inspection, or perhaps
> using a tool?

I'm guessing that if you run code that exercises this while built with
--enable-tcg-debug then TCG's sanity checking complains about unfreed
temps.

>
> Yours,
> Aleksandar
>
>>  target/arm/translate-vfp.inc.c | 2 ++
>>  1 file changed, 2 insertions(+)
>>
>> diff --git a/target/arm/translate-vfp.inc.c
> b/target/arm/translate-vfp.inc.c
>> index 3e8ea80493b..9ae980bef63 100644
>> --- a/target/arm/translate-vfp.inc.c
>> +++ b/target/arm/translate-vfp.inc.c
>> @@ -880,8 +880,10 @@ static bool trans_VMOV_64_sp(DisasContext *s,
> arg_VMOV_64_sp *a)
>>  /* gpreg to fpreg */
>>  tmp = load_reg(s, a->rt);
>>  neon_store_reg32(tmp, a->vm);
>> +tcg_temp_free_i32(tmp);
>>  tmp = load_reg(s, a->rt2);
>>  neon_store_reg32(tmp, a->vm + 1);
>> +tcg_temp_free_i32(tmp);
>>  }
>>
>>  return true;
>> --
>> 2.20.1
>>
>>


--
Alex Bennée

Re: [Qemu-devel] [PATCH v3 1/4] iotests: add script_initialize

[John Snow]

On 8/29/19 2:58 PM, Philippe Mathieu-Daudé wrote:
> On 8/29/19 8:43 PM, John Snow wrote:
>> On 8/29/19 2:27 PM, Philippe Mathieu-Daudé wrote:
>>> This restrict test 208 to the Linux platform, is this OK?
>>>
>>> The rest looks good.
>>
>> I forgot about that, so good catch.
>>
>> I don't know. We seem to already restrict a LOT of tests to the Linux
>> platform. Does this one actually work on other platforms?
>>
>> Actually, I can't see any other supported oses/supported platforms calls
>> anywhere in iotests that don't just specify Linux or leave it the
>> default (...which is also linux.)
>>
>> There isn't a way to engage the old-style python unittest framework
>> without implying Linux; you have to manually override it if so.
> 
> Then let's use:
> 
> iotests.script_initialize(supported_oses=[])
> 
> and call it a day?
> 

*whines*

The even lazier thing to do is to do more work to come up with some
excuse to avoid re-spinning the series:

commit bc521696607c5348fcd8a9e57b408d0ac0dbe2f8
Author: Fam Zheng 
Date:   Sun Jan 4 09:53:52 2015 +0800

qemu-iotests: Add supported os parameter for python tests

If I understand correctly, qemu-iotests never meant to be portable. We
only support Linux for all the shell cases, but didn't specify it for
python tests. Now add this and default all the python tests as Linux
only. If we cares enough later, we can override the parameter in
individual cases.



I think it's only an oversight that this one test didn't specify a
platform. It's certainly the only one. If someone feels strongly that
other platforms are supported, let them send the patch.

--js

Re: [Qemu-devel] [PATCH v3 1/4] iotests: add script_initialize

[Philippe Mathieu-Daudé]

On 8/29/19 8:43 PM, John Snow wrote:
> On 8/29/19 2:27 PM, Philippe Mathieu-Daudé wrote:
>> This restrict test 208 to the Linux platform, is this OK?
>>
>> The rest looks good.
> 
> I forgot about that, so good catch.
> 
> I don't know. We seem to already restrict a LOT of tests to the Linux
> platform. Does this one actually work on other platforms?
> 
> Actually, I can't see any other supported oses/supported platforms calls
> anywhere in iotests that don't just specify Linux or leave it the
> default (...which is also linux.)
> 
> There isn't a way to engage the old-style python unittest framework
> without implying Linux; you have to manually override it if so.

Then let's use:

iotests.script_initialize(supported_oses=[])

and call it a day?

> 
> For new tests, MOST of them specified Linux in some way or another, as
> you saw.
> 
> So either:
> - 208 was an oversight, or
> - Many tests are accidentally limiting to Linux and could be loosened.

This can stay in your TODO for after this painful series.

> Which is it? Dunno. Guess I'll look at the VM tests to see if I can
> co-opt some of that... stay tuned?
> 
> --js
> 

Re: [Qemu-devel] [PATCH] target/arm: Free TCG temps in trans_VMOV_64_sp()

[Aleksandar Markovic]

27.08.2019. 14.20, "Peter Maydell"  је написао/ла:
>
> The function neon_store_reg32() doesn't free the TCG temp that it
> is passed, so the caller must do that. We got this right in most
> places but forgot to free the TCG temps in trans_VMOV_64_sp().
>
> Cc: qemu-sta...@nongnu.org
> Signed-off-by: Peter Maydell 
> ---

Hello, Peter,

I am just curious if you found this by manual code inspection, or perhaps
using a tool?

Yours,
Aleksandar

>  target/arm/translate-vfp.inc.c | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/target/arm/translate-vfp.inc.c
b/target/arm/translate-vfp.inc.c
> index 3e8ea80493b..9ae980bef63 100644
> --- a/target/arm/translate-vfp.inc.c
> +++ b/target/arm/translate-vfp.inc.c
> @@ -880,8 +880,10 @@ static bool trans_VMOV_64_sp(DisasContext *s,
arg_VMOV_64_sp *a)
>  /* gpreg to fpreg */
>  tmp = load_reg(s, a->rt);
>  neon_store_reg32(tmp, a->vm);
> +tcg_temp_free_i32(tmp);
>  tmp = load_reg(s, a->rt2);
>  neon_store_reg32(tmp, a->vm + 1);
> +tcg_temp_free_i32(tmp);
>  }
>
>  return true;
> --
> 2.20.1
>
>

Re: [Qemu-devel] [PATCH v3 4/4] iotests: use python logging for iotests.log()

[John Snow]

On 8/29/19 2:34 PM, Philippe Mathieu-Daudé wrote:
> On 8/21/19 1:52 AM, John Snow wrote:
>> We can turn logging on/off globally instead of per-function.
>>
>> Remove use_log from run_job, and use python logging to turn on
>> diffable output when we run through a script entry point.
>>
>> iotest 245 changes output order due to buffering reasons.
>> ---
>>  tests/qemu-iotests/030|  4 +--
>>  tests/qemu-iotests/245|  1 +
>>  tests/qemu-iotests/245.out| 24 +-
>>  tests/qemu-iotests/iotests.py | 47 +--
>>  4 files changed, 43 insertions(+), 33 deletions(-)
>>
>> diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030
>> index 1b69f318c6..a382cb430b 100755
>> --- a/tests/qemu-iotests/030
>> +++ b/tests/qemu-iotests/030
>> @@ -411,8 +411,8 @@ class TestParallelOps(iotests.QMPTestCase):
>>  result = self.vm.qmp('block-job-set-speed', device='drive0', 
>> speed=0)
>>  self.assert_qmp(result, 'return', {})
>>  
>> -self.vm.run_job(job='drive0', auto_dismiss=True, use_log=False)
>> -self.vm.run_job(job='node4', auto_dismiss=True, use_log=False)
>> +self.vm.run_job(job='drive0', auto_dismiss=True)
>> +self.vm.run_job(job='node4', auto_dismiss=True)
>>  self.assert_no_active_block_jobs()
>>  
>>  # Test a block-stream and a block-commit job in parallel
>> diff --git a/tests/qemu-iotests/245 b/tests/qemu-iotests/245
>> index bc1ceb9792..3bc29acb33 100644
>> --- a/tests/qemu-iotests/245
>> +++ b/tests/qemu-iotests/245
>> @@ -1000,4 +1000,5 @@ class TestBlockdevReopen(iotests.QMPTestCase):
>>  self.reopen(opts, {'backing': 'hd2'})
>>  
>>  if __name__ == '__main__':
>> +iotests.activate_logging()
>>  iotests.main(supported_fmts=["qcow2"])
> 
> Why not use:
> 
>iotests.script_main(iotests.main, supported_fmts=['qcow2')
> 

Well, that'd call iotests.execute_test twice and it'd perform setup
twice, too.

Usually, we want logging on for "script-style" tests, but we want
logging off for unittest-style ones. This test has opted to use both.

(Or more likely: just wanted to use run_job and just dealt with the
extramodal output.)

OK; we can turn on logging as we see fit.

--js

Re: [Qemu-devel] [PATCH v3 1/4] iotests: add script_initialize

[John Snow]

On 8/29/19 2:27 PM, Philippe Mathieu-Daudé wrote:
> This restrict test 208 to the Linux platform, is this OK?
> 
> The rest looks good.

I forgot about that, so good catch.

I don't know. We seem to already restrict a LOT of tests to the Linux
platform. Does this one actually work on other platforms?

Actually, I can't see any other supported oses/supported platforms calls
anywhere in iotests that don't just specify Linux or leave it the
default (...which is also linux.)

There isn't a way to engage the old-style python unittest framework
without implying Linux; you have to manually override it if so.

For new tests, MOST of them specified Linux in some way or another, as
you saw.

So either:
- 208 was an oversight, or
- Many tests are accidentally limiting to Linux and could be loosened.

Which is it? Dunno. Guess I'll look at the VM tests to see if I can
co-opt some of that... stay tuned?

--js

Re: [Qemu-devel] [PATCH] RISCV: support riscv vector extension 0.7.1

[Aleksandar Markovic]

29.08.2019. 15.02, "liuzhiwei"  је написао/ла:
>
>
> On 2019/8/29 上午3:20, Aleksandar Markovic wrote:
>>
>>
>>
>> > On Wed, Aug 28, 2019 at 9:04 AM liuzhiwei  wrote:
>>>
>>> Change-Id: I3cf891bc400713b95f47ecca82b1bf773f3dcb25
>>> Signed-off-by: liuzhiwei 
>>> ---
>>
>>
>> Such large patch and "Change-Id:
I3cf891bc400713b95f47ecca82b1bf773f3dcb25" is its entire commit message??
Horrible.
>
> Hi,  Aleksandar
>
> I am so sorry. A patch set with cover letter will be sent later.
>
> Best Regards,
>
> Zhiwei

OK, Zhiwei,

You'll soon get more used  to participating in open source, and write much
better patches.

Try to follow guidelines described at
https://wiki.qemu.org/Contribute/SubmitAPatch
Thanks,
Aleksandar



>>
>> Aleksandar
>>
>>>
>>>  fpu/softfloat.c |   119 +
>>>  include/fpu/softfloat.h | 4 +
>>>  linux-user/riscv/cpu_loop.c | 8 +-
>>>  target/riscv/Makefile.objs  | 2 +-
>>>  target/riscv/cpu.h  |30 +
>>>  target/riscv/cpu_bits.h |15 +
>>>  target/riscv/cpu_helper.c   | 7 +
>>>  target/riscv/csr.c  |65 +-
>>>  target/riscv/helper.h   |   354 +
>>>  target/riscv/insn32.decode  |   374 +-
>>>  target/riscv/insn_trans/trans_rvv.inc.c |   484 +
>>>  target/riscv/translate.c| 1 +
>>>  target/riscv/vector_helper.c| 26563
++
>>>  13 files changed, 28017 insertions(+), 9 deletions(-)
>>>  create mode 100644 target/riscv/insn_trans/trans_rvv.inc.c
>>>  create mode 100644 target/riscv/vector_helper.c
>>>
>>> diff --git a/fpu/softfloat.c b/fpu/softfloat.c
>>> index 2ba36ec..da155ea 100644
>>> --- a/fpu/softfloat.c
>>> +++ b/fpu/softfloat.c
>>> @@ -433,6 +433,16 @@ static inline int extractFloat16Exp(float16 a)
>>>  }
>>>
>>>
 /*
>>> +| Returns the sign bit of the half-precision floating-point value `a'.
>>>
+**/
>>> +
>>> +static inline flag extractFloat16Sign(float16 a)
>>> +{
>>> +return float16_val(a) >> 0xf;
>>> +}
>>> +
>>> +
>>>
+/*
>>>  | Returns the fraction bits of the single-precision floating-point
value `a'.
>>>
 **/
>>>
>>> @@ -4790,6 +4800,35 @@ int float32_eq(float32 a, float32 b,
float_status *status)
>>>  }
>>>
>>>
 /*
>>> +| Returns 1 if the half-precision floating-point value `a' is less than
>>> +| or equal to the corresponding value `b', and 0 otherwise.  The
invalid
>>> +| exception is raised if either operand is a NaN.  The comparison is
performed
>>> +| according to the IEC/IEEE Standard for Binary Floating-Point
Arithmetic.
>>>
+**/
>>> +
>>> +int float16_le(float16 a, float16 b, float_status *status)
>>> +{
>>> +flag aSign, bSign;
>>> +uint16_t av, bv;
>>> +a = float16_squash_input_denormal(a, status);
>>> +b = float16_squash_input_denormal(b, status);
>>> +
>>> +if (( ( extractFloat16Exp( a ) == 0x1F ) &&
extractFloat16Frac( a ) )
>>> + || ( ( extractFloat16Exp( b ) == 0x1F ) &&
extractFloat16Frac( b ) )
>>> +   ) {
>>> +float_raise(float_flag_invalid, status);
>>> +return 0;
>>> +}
>>> +aSign = extractFloat16Sign( a );
>>> +bSign = extractFloat16Sign( b );
>>> +av = float16_val(a);
>>> +bv = float16_val(b);
>>> +if ( aSign != bSign ) return aSign || ( (uint16_t) ( ( av | bv
)<<1 ) == 0 );
>>> +return ( av == bv ) || ( aSign ^ ( av < bv ) );
>>> +
>>> +}
>>> +
>>>
+/*
>>>  | Returns 1 if the single-precision floating-point value `a' is less
than
>>>  | or equal to the corresponding value `b', and 0 otherwise.  The
invalid
>>>  | exception is raised if either operand is a NaN.  The comparison is
performed
>>> @@ -4825,6 +4864,35 @@ int float32_le(float32 a, float32 b,
float_status *status)
>>>  | to the IEC/IEEE Standard for Binary Floating-Point Arithmetic.
>>>
 **/
>>>
>>> +int float16_lt(float16 a, float16 b, float_status *status)
>>> +{
>>> +flag aSign, bSign;
>>> +uint16_t av, bv;
>>> +a = float16_squash_input_denormal(a, status);
>>> +b = float16_squash_input_denormal(b, status);
>>> +
>>> +if (( ( extractFloat16Exp( a ) == 0x1F ) &&
extractFloat16Frac( a ) )
>>> + || ( ( extractFloat16Exp( b ) == 0x1F ) &&
extractFloat16Frac( b ) )
>>> +   ) {
>>> +float_raise(float_flag_invalid, status);
>>> +return 0;

Re: [Qemu-devel] [PATCH] i386/vmmouse: Properly reset state

[Philippe Mathieu-Daudé]

On 8/29/19 8:12 PM, Jan Kiszka wrote:
> On 29.08.19 20:00, Philippe Mathieu-Daudé wrote:
>> Hi Jan,
>>
>> On 8/27/19 9:49 PM, Eduardo Habkost wrote:
>>> On Sun, Aug 25, 2019 at 04:58:18PM +0200, Jan Kiszka wrote:
 On 21.07.19 10:58, Jan Kiszka wrote:
> From: Jan Kiszka 
>
> nb_queue was not zeroed so that we no longer delivered events if a
> previous guest left the device in an overflow state.
>
> The state of absolute does not matter as the next
> vmmouse_update_handler
> call will align it again.
>
> Signed-off-by: Jan Kiszka 
> ---
>    hw/i386/vmmouse.c | 1 +
>    1 file changed, 1 insertion(+)
>
> diff --git a/hw/i386/vmmouse.c b/hw/i386/vmmouse.c
> index 5d2d278be4..e335bd07da 100644
> --- a/hw/i386/vmmouse.c
> +++ b/hw/i386/vmmouse.c
> @@ -257,6 +257,7 @@ static void vmmouse_reset(DeviceState *d)
>    VMMouseState *s = VMMOUSE(d);
>
>    s->queue_size = VMMOUSE_QUEUE_SIZE;
> +    s->nb_queue = 0;
>>
>> Don't we also need to reset the status in case vmmouse_get_status() is
>> called directly after reset?
>>
>>   s->status = 0;
>>
> 
> Thanks for checking. We call vmmouse_disable() here, and that sets
> status to
> 0x anyway.

I missed that, you are correct :)

So:
Reviewed-by: Philippe Mathieu-Daudé 

>
>    vmmouse_disable(s);
>    }
> -- 
> 2.16.4
>
>

 Ping - or who is looking after this?
>>>
>>> Despite being in hw/i386, I think we can say vmmouse.c doesn't
>>> have a maintainer.  Last time someone changed vmmouse.c in a
>>> meaningful way (not just adapting to API changes or removing
>>> duplicate code) was in 2012.
>>>
>>
>> Well it does has a few:
>>
>> $ ./scripts/get_maintainer.pl -f hw/i386/vmmouse.c
>> "Michael S. Tsirkin"  (supporter:PC)
>> Marcel Apfelbaum  (supporter:PC)
>> Paolo Bonzini  (maintainer:X86 TCG CPUs)
>> Richard Henderson  (maintainer:X86 TCG CPUs)
>> Eduardo Habkost  (maintainer:X86 TCG CPUs)
>>
>> However the correct section should rather be "PC Chipset".
>>
>>> But the change makes sense to me, so:
>>>
>>> Reviewed-by: Eduardo Habkost 
>>>
>>> I'll queue it.
>>>

Re: [Qemu-devel] [PATCH v3 4/4] iotests: use python logging for iotests.log()

[Philippe Mathieu-Daudé]

On 8/21/19 1:52 AM, John Snow wrote:
> We can turn logging on/off globally instead of per-function.
> 
> Remove use_log from run_job, and use python logging to turn on
> diffable output when we run through a script entry point.
> 
> iotest 245 changes output order due to buffering reasons.
> ---
>  tests/qemu-iotests/030|  4 +--
>  tests/qemu-iotests/245|  1 +
>  tests/qemu-iotests/245.out| 24 +-
>  tests/qemu-iotests/iotests.py | 47 +--
>  4 files changed, 43 insertions(+), 33 deletions(-)
> 
> diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030
> index 1b69f318c6..a382cb430b 100755
> --- a/tests/qemu-iotests/030
> +++ b/tests/qemu-iotests/030
> @@ -411,8 +411,8 @@ class TestParallelOps(iotests.QMPTestCase):
>  result = self.vm.qmp('block-job-set-speed', device='drive0', speed=0)
>  self.assert_qmp(result, 'return', {})
>  
> -self.vm.run_job(job='drive0', auto_dismiss=True, use_log=False)
> -self.vm.run_job(job='node4', auto_dismiss=True, use_log=False)
> +self.vm.run_job(job='drive0', auto_dismiss=True)
> +self.vm.run_job(job='node4', auto_dismiss=True)
>  self.assert_no_active_block_jobs()
>  
>  # Test a block-stream and a block-commit job in parallel
> diff --git a/tests/qemu-iotests/245 b/tests/qemu-iotests/245
> index bc1ceb9792..3bc29acb33 100644
> --- a/tests/qemu-iotests/245
> +++ b/tests/qemu-iotests/245
> @@ -1000,4 +1000,5 @@ class TestBlockdevReopen(iotests.QMPTestCase):
>  self.reopen(opts, {'backing': 'hd2'})
>  
>  if __name__ == '__main__':
> +iotests.activate_logging()
>  iotests.main(supported_fmts=["qcow2"])

Why not use:

   iotests.script_main(iotests.main, supported_fmts=['qcow2')

> diff --git a/tests/qemu-iotests/245.out b/tests/qemu-iotests/245.out
> index a19de5214d..15c3630e92 100644
> --- a/tests/qemu-iotests/245.out
> +++ b/tests/qemu-iotests/245.out
> @@ -1,17 +1,17 @@
> +{"execute": "job-finalize", "arguments": {"id": "commit0"}}
> +{"return": {}}
> +{"data": {"id": "commit0", "type": "commit"}, "event": "BLOCK_JOB_PENDING", 
> "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
> +{"data": {"device": "commit0", "len": 3145728, "offset": 3145728, "speed": 
> 0, "type": "commit"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": 
> {"microseconds": "USECS", "seconds": "SECS"}}
> +{"execute": "job-finalize", "arguments": {"id": "stream0"}}
> +{"return": {}}
> +{"data": {"id": "stream0", "type": "stream"}, "event": "BLOCK_JOB_PENDING", 
> "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
> +{"data": {"device": "stream0", "len": 3145728, "offset": 3145728, "speed": 
> 0, "type": "stream"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": 
> {"microseconds": "USECS", "seconds": "SECS"}}
> +{"execute": "job-finalize", "arguments": {"id": "stream0"}}
> +{"return": {}}
> +{"data": {"id": "stream0", "type": "stream"}, "event": "BLOCK_JOB_PENDING", 
> "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
> +{"data": {"device": "stream0", "len": 3145728, "offset": 3145728, "speed": 
> 0, "type": "stream"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": 
> {"microseconds": "USECS", "seconds": "SECS"}}
>  ..
>  --
>  Ran 18 tests
>  
>  OK
> -{"execute": "job-finalize", "arguments": {"id": "commit0"}}
> -{"return": {}}
> -{"data": {"id": "commit0", "type": "commit"}, "event": "BLOCK_JOB_PENDING", 
> "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
> -{"data": {"device": "commit0", "len": 3145728, "offset": 3145728, "speed": 
> 0, "type": "commit"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": 
> {"microseconds": "USECS", "seconds": "SECS"}}
> -{"execute": "job-finalize", "arguments": {"id": "stream0"}}
> -{"return": {}}
> -{"data": {"id": "stream0", "type": "stream"}, "event": "BLOCK_JOB_PENDING", 
> "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
> -{"data": {"device": "stream0", "len": 3145728, "offset": 3145728, "speed": 
> 0, "type": "stream"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": 
> {"microseconds": "USECS", "seconds": "SECS"}}
> -{"execute": "job-finalize", "arguments": {"id": "stream0"}}
> -{"return": {}}
> -{"data": {"id": "stream0", "type": "stream"}, "event": "BLOCK_JOB_PENDING", 
> "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
> -{"data": {"device": "stream0", "len": 3145728, "offset": 3145728, "speed": 
> 0, "type": "stream"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": 
> {"microseconds": "USECS", "seconds": "SECS"}}
> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
> index 661d7f93bf..b97cc2fab2 100644
> --- a/tests/qemu-iotests/iotests.py
> +++ b/tests/qemu-iotests/iotests.py
> @@ -35,6 +35,13 @@ from collections import OrderedDict
>  sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 
> 'python'))
>  from qemu import qtest
>  
> +# Use 

Re: [Qemu-devel] [PATCH v3 2/4] iotest 258: use script_main

[Philippe Mathieu-Daudé]

On 8/21/19 1:52 AM, John Snow wrote:
> Since this one is nicely factored to use a single entry point,
> use script_main to run the tests.

With your Signed-off-by:
Reviewed-by: Philippe Mathieu-Daudé 

> ---
>  tests/qemu-iotests/258 | 8 ++--
>  1 file changed, 2 insertions(+), 6 deletions(-)
> 
> diff --git a/tests/qemu-iotests/258 b/tests/qemu-iotests/258
> index b84cf02254..1372522c7a 100755
> --- a/tests/qemu-iotests/258
> +++ b/tests/qemu-iotests/258
> @@ -23,11 +23,6 @@ import iotests
>  from iotests import log, qemu_img, qemu_io_silent, \
>  filter_qmp_testfiles, filter_qmp_imgfmt
>  
> -# Need backing file and change-backing-file support
> -iotests.verify_image_format(supported_fmts=['qcow2', 'qed'])
> -iotests.verify_platform(['linux'])
> -
> -
>  # Returns a node for blockdev-add
>  def node(node_name, path, backing=None, fmt=None, throttle=None):
>  if fmt is None:
> @@ -160,4 +155,5 @@ def main():
>  test_concurrent_finish(False)
>  
>  if __name__ == '__main__':
> -main()
> +# Need backing file and change-backing-file support
> +iotests.script_main(main, supported_fmts=['qcow2', 'qed'])
> 

Re: [Qemu-devel] [PATCH v3 3/4] iotests: add protocol support to initialization info

[Philippe Mathieu-Daudé]

On 8/21/19 1:52 AM, John Snow wrote:
> This will add supported_protocols and unsupported_protocols to all of
> iotests.main, iotests.script_main, and iotests.script_initialize.

With your Signed-off-by:
Reviewed-by: Philippe Mathieu-Daudé 

> ---
>  tests/qemu-iotests/207| 4 ++--
>  tests/qemu-iotests/210| 4 ++--
>  tests/qemu-iotests/211| 4 ++--
>  tests/qemu-iotests/212| 4 ++--
>  tests/qemu-iotests/213| 4 ++--
>  tests/qemu-iotests/iotests.py | 5 -
>  6 files changed, 14 insertions(+), 11 deletions(-)
> 
> diff --git a/tests/qemu-iotests/207 b/tests/qemu-iotests/207
> index ab9e3b6747..35d98f2736 100755
> --- a/tests/qemu-iotests/207
> +++ b/tests/qemu-iotests/207
> @@ -24,8 +24,8 @@ import iotests
>  import subprocess
>  import re
>  
> -iotests.script_initialize(supported_fmts=['raw'])
> -iotests.verify_protocol(supported=['ssh'])
> +iotests.script_initialize(supported_fmts=['raw'],
> +  supported_protocols=['ssh'])
>  
>  def filter_hash(qmsg):
>  def _filter(key, value):
> diff --git a/tests/qemu-iotests/210 b/tests/qemu-iotests/210
> index 5a7013cd34..d9fe780c07 100755
> --- a/tests/qemu-iotests/210
> +++ b/tests/qemu-iotests/210
> @@ -23,8 +23,8 @@
>  import iotests
>  from iotests import imgfmt
>  
> -iotests.script_initialize(supported_fmts=['luks'])
> -iotests.verify_protocol(supported=['file'])
> +iotests.script_initialize(supported_fmts=['luks'],
> +  supported_protocols=['file'])
>  
>  def blockdev_create(vm, options):
>  result = vm.qmp_log('blockdev-create', job_id='job0', options=options,
> diff --git a/tests/qemu-iotests/211 b/tests/qemu-iotests/211
> index 4d6aac497f..155fac4e87 100755
> --- a/tests/qemu-iotests/211
> +++ b/tests/qemu-iotests/211
> @@ -23,8 +23,8 @@
>  import iotests
>  from iotests import imgfmt
>  
> -iotests.script_initialize(supported_fmts=['vdi'])
> -iotests.verify_protocol(supported=['file'])
> +iotests.script_initialize(supported_fmts=['vdi'],
> +  supported_protocols=['file'])
>  
>  def blockdev_create(vm, options):
>  result = vm.qmp_log('blockdev-create', job_id='job0', options=options,
> diff --git a/tests/qemu-iotests/212 b/tests/qemu-iotests/212
> index ec35bceb11..67e5a1dbb5 100755
> --- a/tests/qemu-iotests/212
> +++ b/tests/qemu-iotests/212
> @@ -23,8 +23,8 @@
>  import iotests
>  from iotests import imgfmt
>  
> -iotests.script_initialize(supported_fmts=['parallels'])
> -iotests.verify_protocol(supported=['file'])
> +iotests.script_initialize(supported_fmts=['parallels'],
> +  supported_protocols=['file'])
>  
>  def blockdev_create(vm, options):
>  result = vm.qmp_log('blockdev-create', job_id='job0', options=options,
> diff --git a/tests/qemu-iotests/213 b/tests/qemu-iotests/213
> index 3d2c024375..23f387ab63 100755
> --- a/tests/qemu-iotests/213
> +++ b/tests/qemu-iotests/213
> @@ -23,8 +23,8 @@
>  import iotests
>  from iotests import imgfmt
>  
> -iotests.script_initialize(supported_fmts=['vhdx'])
> -iotests.verify_protocol(supported=['file'])
> +iotests.script_initialize(supported_fmts=['vhdx'],
> +  supported_protocols=['file'])
>  
>  def blockdev_create(vm, options):
>  result = vm.qmp_log('blockdev-create', job_id='job0', options=options,
> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
> index 2970d7304a..661d7f93bf 100644
> --- a/tests/qemu-iotests/iotests.py
> +++ b/tests/qemu-iotests/iotests.py
> @@ -925,7 +925,9 @@ def execute_unittest(debug=False):
>  def execute_setup_common(supported_fmts=[],
>   supported_oses=['linux'],
>   supported_cache_modes=[],
> - unsupported_fmts=[]):
> + unsupported_fmts=[],
> + supported_protocols=[],
> + unsupported_protocols=[]):
>  """
>  Perform necessary setup for either script-style or unittest-style tests.
>  """
> @@ -941,6 +943,7 @@ def execute_setup_common(supported_fmts=[],
>  verify_image_format(supported_fmts, unsupported_fmts)
>  verify_platform(supported_oses)
>  verify_cache_mode(supported_cache_modes)
> +verify_protocol(supported_protocols, unsupported_protocols)
>  
>  debug = '-d' in sys.argv
>  if debug:
> 

Re: [Qemu-devel] [PATCH v3 1/4] iotests: add script_initialize

[Philippe Mathieu-Daudé]

On 8/21/19 1:52 AM, John Snow wrote:
> Like script_main, but doesn't require a single point of entry.
> Replace all existing initialization sections with this drop-in replacement.
> 
> This brings debug support to all existing script-style iotests.
> 
> Note: supported_oses=['linux'] was omitted, as it is a default argument.
> ---
>  tests/qemu-iotests/149|  3 +-
>  tests/qemu-iotests/194|  3 +-
>  tests/qemu-iotests/202|  3 +-
>  tests/qemu-iotests/203|  3 +-
>  tests/qemu-iotests/206|  2 +-
>  tests/qemu-iotests/207|  2 +-
>  tests/qemu-iotests/208|  2 +-
>  tests/qemu-iotests/209|  2 +-
>  tests/qemu-iotests/210|  2 +-
>  tests/qemu-iotests/211|  2 +-
>  tests/qemu-iotests/212|  2 +-
>  tests/qemu-iotests/213|  2 +-
>  tests/qemu-iotests/216|  3 +-
>  tests/qemu-iotests/218|  2 +-
>  tests/qemu-iotests/219|  2 +-
>  tests/qemu-iotests/222|  5 ++-
>  tests/qemu-iotests/224|  3 +-
>  tests/qemu-iotests/228|  3 +-
>  tests/qemu-iotests/234|  3 +-
>  tests/qemu-iotests/235|  4 +--
>  tests/qemu-iotests/236|  2 +-
>  tests/qemu-iotests/237|  2 +-
>  tests/qemu-iotests/238|  2 ++
>  tests/qemu-iotests/242|  2 +-
>  tests/qemu-iotests/246|  2 +-
>  tests/qemu-iotests/248|  2 +-
>  tests/qemu-iotests/254|  2 +-
>  tests/qemu-iotests/255|  2 +-
>  tests/qemu-iotests/256|  2 +-
>  tests/qemu-iotests/262|  3 +-
>  tests/qemu-iotests/iotests.py | 58 +++
>  31 files changed, 71 insertions(+), 61 deletions(-)
> 
> diff --git a/tests/qemu-iotests/149 b/tests/qemu-iotests/149
> index 4f363f295f..9fa97966c5 100755
> --- a/tests/qemu-iotests/149
> +++ b/tests/qemu-iotests/149
> @@ -383,8 +383,7 @@ def test_once(config, qemu_img=False):
>  
>  
>  # Obviously we only work with the luks image format
> -iotests.verify_image_format(supported_fmts=['luks'])
> -iotests.verify_platform()
> +iotests.script_initialize(supported_fmts=['luks'])
>  
>  # We need sudo in order to run cryptsetup to create
>  # dm-crypt devices. This is safe to use on any
> diff --git a/tests/qemu-iotests/194 b/tests/qemu-iotests/194
> index d746ab1e21..c8aeb6d0e4 100755
> --- a/tests/qemu-iotests/194
> +++ b/tests/qemu-iotests/194
> @@ -21,8 +21,7 @@
>  
>  import iotests
>  
> -iotests.verify_image_format(supported_fmts=['qcow2', 'qed', 'raw'])
> -iotests.verify_platform(['linux'])
> +iotests.script_initialize(supported_fmts=['qcow2', 'qed', 'raw'])
>  
>  with iotests.FilePath('source.img') as source_img_path, \
>   iotests.FilePath('dest.img') as dest_img_path, \
> diff --git a/tests/qemu-iotests/202 b/tests/qemu-iotests/202
> index 581ca34d79..1271ac9459 100755
> --- a/tests/qemu-iotests/202
> +++ b/tests/qemu-iotests/202
> @@ -24,8 +24,7 @@
>  
>  import iotests
>  
> -iotests.verify_image_format(supported_fmts=['qcow2'])
> -iotests.verify_platform(['linux'])
> +iotests.script_initialize(supported_fmts=['qcow2'])
>  
>  with iotests.FilePath('disk0.img') as disk0_img_path, \
>   iotests.FilePath('disk1.img') as disk1_img_path, \
> diff --git a/tests/qemu-iotests/203 b/tests/qemu-iotests/203
> index 4874a1a0d8..c40fe231ea 100755
> --- a/tests/qemu-iotests/203
> +++ b/tests/qemu-iotests/203
> @@ -24,8 +24,7 @@
>  
>  import iotests
>  
> -iotests.verify_image_format(supported_fmts=['qcow2'])
> -iotests.verify_platform(['linux'])
> +iotests.script_initialize(supported_fmts=['qcow2'])
>  
>  with iotests.FilePath('disk0.img') as disk0_img_path, \
>   iotests.FilePath('disk1.img') as disk1_img_path, \
> diff --git a/tests/qemu-iotests/206 b/tests/qemu-iotests/206
> index 5bb738bf23..23ff2f624b 100755
> --- a/tests/qemu-iotests/206
> +++ b/tests/qemu-iotests/206
> @@ -23,7 +23,7 @@
>  import iotests
>  from iotests import imgfmt
>  
> -iotests.verify_image_format(supported_fmts=['qcow2'])
> +iotests.script_initialize(supported_fmts=['qcow2'])
>  
>  def blockdev_create(vm, options):
>  result = vm.qmp_log('blockdev-create',
> diff --git a/tests/qemu-iotests/207 b/tests/qemu-iotests/207
> index ec8c1d06f0..ab9e3b6747 100755
> --- a/tests/qemu-iotests/207
> +++ b/tests/qemu-iotests/207
> @@ -24,7 +24,7 @@ import iotests
>  import subprocess
>  import re
>  
> -iotests.verify_image_format(supported_fmts=['raw'])
> +iotests.script_initialize(supported_fmts=['raw'])
>  iotests.verify_protocol(supported=['ssh'])
>  
>  def filter_hash(qmsg):
> diff --git a/tests/qemu-iotests/208 b/tests/qemu-iotests/208
> index 1e202388dc..dfce6f9fe4 100755
> --- a/tests/qemu-iotests/208
> +++ b/tests/qemu-iotests/208
> @@ -22,7 +22,7 @@
>  
>  import iotests
>  
> -iotests.verify_image_format(supported_fmts=['generic'])
> +iotests.script_initialize(supported_fmts=['generic'])
>  
>  with iotests.FilePath('disk.img') as disk_img_path, \
>   iotests.FilePath('disk-snapshot.img') as 

Re: [Qemu-devel] [PATCH 0/2] tests/acceptance: Update MIPS Malta ssh test

[Aleksandar Markovic]

28.08.2019. 23.24, "Cleber Rosa"  је написао/ла:
>
> On Thu, Aug 22, 2019 at 07:59:07PM +0200, Aleksandar Markovic wrote:
> > 22.08.2019. 05.15, "Aleksandar Markovic" 
је
> > написао/ла:
> > >
> > >
> > > 21.08.2019. 23.00, "Eduardo Habkost"  је
написао/ла:
> > > >
> > > > On Wed, Aug 21, 2019 at 10:27:11PM +0200, Aleksandar Markovic wrote:
> > > > > 02.08.2019. 17.37, "Aleksandar Markovic" <
> > aleksandar.marko...@rt-rk.com> је
> > > > > написао/ла:
> > > > > >
> > > > > > From: Aleksandar Markovic 
> > > > > >
> > > > > > This little series improves linux_ssh_mips_malta.py, both in the
> > sense
> > > > > > of code organization and in the sense of quantity of executed
tests.
> > > > > >
> > > > >
> > > > > Hello, all.
> > > > >
> > > > > I am going to send a new version in few days, and I have a
question
> > for
> > > > > test team:
> > > > >
> > > > > Currently, the outcome of the script execition is either PASS:1
> > FAIL:0 or
> > > > > PASS:0 FAIL:1. But the test actually consists of several
subtests. Is
> > there
> > > > > any way that this single Python script considers these subtests as
> > separate
> > > > > tests (test cases), reporting something like PASS:12 FAIL:7? If
yes,
> > what
> > > > > would be the best way to achieve that?
> > > >
> > > > If you are talking about each test_*() method, they are already
> > > > treated like separate tests.  If you mean treating each
> > > > ssh_command_output_contains() call as a separate test, this might
> > > > be difficult.
> > > >
> > >
> > > Yes, I meant the latter one, individual code segments involving an
> > invocation of ssh_command_output_contains() instance being treated as
> > separate tests.
> > >
> >
> > Hello, Cleber,
> >
> > I am willing to rewamp python file structure if needed.
> >
> > The only thing I feel a little unconfortable is if I need to reboot the
> > virtual machine for each case of ssh_command_output_contains().
> >
>
> Hi Aleksandar,
>
> The short answer is that Avocado provides no way to report "subtest"
> statuses (as a formal concept), neither does the current
> "avocado_qemu" infrastructure allow for management of VMs across
> tests.  The later is an Avocado-VT feature, and it to be honest it
> brings a good deal of problems in itself, which we decided to avoid
> here.
>
> About the lack of subtests, we (the autotest project, then the Avocado
> project) found that this concept, to be well applied, need more than
> we could deal with initially.  For instance, Avocado has the concept
> of "pre_test" and "post_test" hooks, with that, should those be
> applied to subtests as well?  Also, there's support for capturing
> system information (a feature called sysinfo) before and after the
> tests... again, should it be applied to subtests?  Avocado also stores
> a well defined results directory, and we'd have to deal with something
> like that for subtests.  With regards to the variants feature, should
> they also be applied to subtests?  The list of questions goes on and
> on.
>
> The fact that one test should not be able (as much as possible) to
> influence another test also comes into play in our initial decision
> to avoid subtests.
>
> IMO, the best way to handle this is to either keep a separate logger
> with the test progress:
>
>
https://avocado-framework.readthedocs.io/en/71.0/WritingTests.html#advanced-logging-capabilities
>
> With a change similar to:
>
> ---
> diff --git a/tests/acceptance/linux_ssh_mips_malta.py
b/tests/acceptance/linux_ssh_mips_malta.py
> index 509ff929cf..0683586c35 100644
> --- a/tests/acceptance/linux_ssh_mips_malta.py
> +++ b/tests/acceptance/linux_ssh_mips_malta.py
> @@ -17,6 +17,7 @@ from avocado_qemu import Test
>  from avocado.utils import process
>  from avocado.utils import archive
>
> +progress_log = logging.getLogger("progress")
>
>  class LinuxSSH(Test):
>
> @@ -149,6 +150,7 @@ class LinuxSSH(Test):
>  stdout, _ = self.ssh_command(cmd)
>  for line in stdout:
>  if exp in line:
> +progress_log.info('Check successful for "%s"', cmd)
>  break
>  else:
>  self.fail('"%s" output does not contain "%s"' % (cmd, exp))
> ---
>
> You could run tests with:
>
>   $ ./tests/venv/bin/avocado --show=console,progress run
--store-logging-stream progress -- tests/acceptance/linux_ssh_mips_malta.py
>
> And at the same time:
>
>   $ tail -f ~/avocado/job-results/latest/progress.INFO
>   17:20:44 INFO | Check successful for "uname -a"
>   17:20:44 INFO | Check successful for "cat /proc/cpuinfo"
>   ...
>
> I hope this helps somehow.
>
> Best regards,
> - Cleber.
>

Thanks, Cleber, for your detailed response. I'll use whatever is available,
along the lines you highligted. I will most likely gradually modify this
test until I find the sweet spot where I am satisfied with test behavior
and reporting, but also everything fits well into Avocado framework.

Thanks again, both to you and Eduardo,
Aleksandar

> > Grateful in advance,
> 

Re: [Qemu-devel] [PATCH v2 0/4] docs: add docs about use of automatic cleanup functions

[no-reply]

Patchew URL: https://patchew.org/QEMU/20190829160710.8792-1-berra...@redhat.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Subject: [Qemu-devel] [PATCH v2 0/4] docs: add docs about use of automatic 
cleanup functions
Message-id: 20190829160710.8792-1-berra...@redhat.com

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
8800019 docs: split the CODING_STYLE doc into distinct groups
5a78ce5 docs: document use of automatic cleanup functions in glib
c94ac86 docs: merge HACKING.rst contents into CODING_STYLE.rst
e6aeca1 docs: convert README, CODING_STYLE and HACKING to RST syntax

=== OUTPUT BEGIN ===
Must be run from the top-level dir. of a kernel tree
=== OUTPUT END ===

Test command exited with code: 2


The full log is available at
http://patchew.org/logs/20190829160710.8792-1-berra...@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

Re: [Qemu-devel] [PATCH 3/4] RFC target/arm: Do not build pre-ARMv7 cpus when using KVM

[Philippe Mathieu-Daudé]

Hi Thomas,

On 8/23/19 4:28 PM, Thomas Huth wrote:
> On 8/23/19 3:58 PM, Philippe Mathieu-Daudé wrote:
>> A KVM-only build won't be able to run pre-ARMv7 cpus, disable them.
>>
>> If KVM is not enabled, they are enabled by default.
> [...]
>>  config CHEETAH
>>  bool
>> +select ARM_V4
>>  select OMAP
>>  select TSC210X
> 
> Are you sure about the "enabled by default" ? There is not "default y"
> here, is it?

What I mean is if you build with --disable-kvm, this selects
--enable-tcg which provides the pre-ARMv7 cpus. So to make no changes, I
also added:

  config ARM_V4
  default y

Which include the "default y".

> 
> Also I'm not sure whether it's such a good idea to always disable the
> config switches in default-configs/arm-softmmu.mak ... if somebody wants
> to build such a restricted QEMU, don't they have to maintain their own
> set of config files anyway?

Ah... I followed your example:

$ git show 9e5c2056d1e
commit 9e5c2056d1e80f344a0c412d7a3d847db1f4e034
Author: Thomas Huth 
Date:   Tue Jan 29 10:42:14 2019 +0100

s390x: express dependencies with Kconfig

Instead of hard-coding all config switches in the config file
default-configs/s390x-softmmu.mak, let's use the new Kconfig files
to express the necessary dependencies: The S390_CCW_VIRTIO config switch
for the "s390-ccw-virtio" machine now selects all non-optional devices.

And since we already have the VIRTIO_PCI and VIRTIO_MMIO config switches
for the other two virtio transports, this patch also introduces a new
config switch VIRTIO_CCW for the third, s390x-specific virtio transport,
so that all three virtio transports are now handled in the same way.

diff --git a/default-configs/s390x-softmmu.mak
b/default-configs/s390x-softmmu.mak
@@ -1,9 +1,13 @@
-CONFIG_PCI=y
-CONFIG_VIRTIO_PCI=y
-CONFIG_SCLPCONSOLE=y
-CONFIG_TERMINAL3270=y
-CONFIG_S390_FLIC=y
-CONFIG_WDT_DIAG288=y
+# Default configuration for s390x-softmmu
+
+# Uncomment the following lines to disable these optional devices:
+#
+#CONFIG_TERMINAL3270=n
+#CONFIG_VFIO_AP=n
+#CONFIG_VFIO_CCW=n
+#CONFIG_VIRTIO_PCI=n
+#CONFIG_WDT_DIAG288=n
+
+# Boards:
+#
 CONFIG_S390_CCW_VIRTIO=y
-CONFIG_VFIO_CCW=y
-CONFIG_VFIO_AP=y

OK now I see, I should have added your comment and use the opposite form
(because now these boards are all enabled) so 's/=y/=n' in my patch.

> I think we should maybe rather rework the default-configs directory:
> Rename the default to "config/default/" instead and then we can add
> other subfolders with such special configurations, e.g. config/nemu/ or
> config/lean-kvm/ or however you want to call it. Then add a new switch
> to the configure script to be able to use the configs from such a
> different folder.

OK so if someone wants a special config, he'd know the config values to
select, so it is pointless/confusing to keep them commented.
Are you suggesting to simply remove the default entries? Such:

-- >8 --
@@ -9,34 +9,33 @@ CONFIG_ARM_V7M=y
 CONFIG_ARM_VIRT=y
 CONFIG_CUBIEBOARD=y
 CONFIG_EXYNOS4=y
-CONFIG_HIGHBANK=y
-CONFIG_INTEGRATOR=y
 CONFIG_FSL_IMX31=y
-CONFIG_MUSICPAL=y
 CONFIG_MUSCA=y
-CONFIG_CHEETAH=y
-CONFIG_SX1=y
-CONFIG_NSERIES=y
 CONFIG_STELLARIS=y
 CONFIG_REALVIEW=y
-CONFIG_VERSATILE=y
 CONFIG_VEXPRESS=y
 CONFIG_ZYNQ=y
-CONFIG_MAINSTONE=y
-CONFIG_GUMSTIX=y
-CONFIG_SPITZ=y
-CONFIG_TOSA=y
-CONFIG_Z2=y
-CONFIG_COLLIE=y
-CONFIG_ASPEED_SOC=y
 CONFIG_NETDUINO2=y
 CONFIG_MPS2=y
 CONFIG_RASPI=y
-CONFIG_DIGIC=y
 CONFIG_SABRELITE=y
 CONFIG_EMCRAFT_SF2=y
-CONFIG_MICROBIT=y
-CONFIG_FSL_IMX25=y
 CONFIG_FSL_IMX7=y
 CONFIG_FSL_IMX6UL=y
 CONFIG_SEMIHOSTING=y
---

Thanks,

Phil.

Re: [Qemu-devel] [PATCH v5 2/2] hw/vfio/common: Fail on VFIO/HW nested paging detection

[Alex Williamson]

On Thu, 29 Aug 2019 11:01:41 +0200
Eric Auger  wrote:

> As of today, VFIO only works along with vIOMMU supporting
> caching mode. The SMMUv3 does not support this mode and
> requires HW nested paging to work properly with VFIO.
> 
> So any attempt to run a VFIO device protected by such IOMMU
> would prevent the assigned device from working and at the
> moment the guest does not even boot as the default
> memory_region_iommu_replay() implementation attempts to
> translate the whole address space and completely stalls
> the guest.

Why doesn't this stall an x86 guest?

I'm a bit confused about what this provides versus the flag_changed
notifier looking for IOMMU_NOTIFIER_MAP, which AIUI is the common
deficiency between VT-d w/o caching-mode and SMMUv3 w/o nested mode.
The iommu notifier is registered prior to calling iommu_replay, so it
seems we already have an opportunity to do something there.  Help me
understand why this is needed.  Thanks,

Alex

> 
> So let's fail on that case.
> 
> Signed-off-by: Eric Auger 
> 
> ---
> 
> v3 -> v4:
> - use IOMMU_ATTR_HW_NESTED_PAGING
> - do not abort anymore but jump to fail
> ---
>  hw/vfio/common.c | 10 ++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/hw/vfio/common.c b/hw/vfio/common.c
> index 3e03c495d8..e8c009d019 100644
> --- a/hw/vfio/common.c
> +++ b/hw/vfio/common.c
> @@ -606,9 +606,19 @@ static void vfio_listener_region_add(MemoryListener 
> *listener,
>  if (memory_region_is_iommu(section->mr)) {
>  VFIOGuestIOMMU *giommu;
>  IOMMUMemoryRegion *iommu_mr = IOMMU_MEMORY_REGION(section->mr);
> +bool nested;
>  int iommu_idx;
>  
>  trace_vfio_listener_region_add_iommu(iova, end);
> +
> +if (!memory_region_iommu_get_attr(iommu_mr,
> +  IOMMU_ATTR_NEED_HW_NESTED_PAGING,
> +  (void *)) && nested) {
> +error_report("VFIO/vIOMMU integration based on HW nested paging "
> + "is not yet supported");
> +ret = -EINVAL;
> +goto fail;
> +}
>  /*
>   * FIXME: For VFIO iommu types which have KVM acceleration to
>   * avoid bouncing all map/unmaps through qemu this way, this

Re: [Qemu-devel] [PATCH] i386/vmmouse: Properly reset state

[Jan Kiszka]

On 29.08.19 20:00, Philippe Mathieu-Daudé wrote:

Hi Jan,

On 8/27/19 9:49 PM, Eduardo Habkost wrote:

On Sun, Aug 25, 2019 at 04:58:18PM +0200, Jan Kiszka wrote:

On 21.07.19 10:58, Jan Kiszka wrote:

From: Jan Kiszka 

nb_queue was not zeroed so that we no longer delivered events if a
previous guest left the device in an overflow state.

The state of absolute does not matter as the next vmmouse_update_handler
call will align it again.

Signed-off-by: Jan Kiszka 
---
   hw/i386/vmmouse.c | 1 +
   1 file changed, 1 insertion(+)

diff --git a/hw/i386/vmmouse.c b/hw/i386/vmmouse.c
index 5d2d278be4..e335bd07da 100644
--- a/hw/i386/vmmouse.c
+++ b/hw/i386/vmmouse.c
@@ -257,6 +257,7 @@ static void vmmouse_reset(DeviceState *d)
   VMMouseState *s = VMMOUSE(d);

   s->queue_size = VMMOUSE_QUEUE_SIZE;
+s->nb_queue = 0;


Don't we also need to reset the status in case vmmouse_get_status() is
called directly after reset?

  s->status = 0;



Thanks for checking. We call vmmouse_disable() here, and that sets status to
0x anyway.

Jan


With it:
Reviewed-by: Philippe Mathieu-Daudé 



   vmmouse_disable(s);
   }
--
2.16.4




Ping - or who is looking after this?


Despite being in hw/i386, I think we can say vmmouse.c doesn't
have a maintainer.  Last time someone changed vmmouse.c in a
meaningful way (not just adapting to API changes or removing
duplicate code) was in 2012.



Well it does has a few:

$ ./scripts/get_maintainer.pl -f hw/i386/vmmouse.c
"Michael S. Tsirkin"  (supporter:PC)
Marcel Apfelbaum  (supporter:PC)
Paolo Bonzini  (maintainer:X86 TCG CPUs)
Richard Henderson  (maintainer:X86 TCG CPUs)
Eduardo Habkost  (maintainer:X86 TCG CPUs)

However the correct section should rather be "PC Chipset".


But the change makes sense to me, so:

Reviewed-by: Eduardo Habkost 

I'll queue it.

Re: [Qemu-devel] [PATCH 3/4] RFC target/arm: Do not build pre-ARMv7 cpus when using KVM

[Philippe Mathieu-Daudé]

On 8/23/19 7:12 PM, Richard Henderson wrote:
> On 8/23/19 6:58 AM, Philippe Mathieu-Daudé wrote:
>> A KVM-only build won't be able to run pre-ARMv7 cpus, disable them.
>>
>> If KVM is not enabled, they are enabled by default.
>>
>> Signed-off-by: Philippe Mathieu-Daudé 
>> ---
>> Sadly this does not work with --enable-tcg --enable-kvm dual config.
>> ---
> 
> Huh?  --enable-kvm does not imply --disable-tcg.

That would have made my life easier ;)

>> +config ARM_V4
>> +default y
>> +depends on !KVM
>> +bool
> 
> This should surely be "depends on TCG".

I restricted this to KVM because this is the only one I know and got
confirmation by Peter. I don't know about other accelerators but I'll
happily take your suggestion!

Thanks,

Phil.

Re: [Qemu-devel] [PATCH 1/4] target/arm: Restrict pre-ARMv7 cpus to TCG

[Philippe Mathieu-Daudé]

On 8/23/19 7:04 PM, Richard Henderson wrote:
> On 8/23/19 6:58 AM, Philippe Mathieu-Daudé wrote:
>> @@ -2535,6 +2544,7 @@ static const ARMCPUInfo arm_cpus[] = {
>>  { .name = "arm1176", .initfn = arm1176_initfn },
>>  { .name = "arm11mpcore", .initfn = arm11mpcore_initfn },
>>  { .name = "cortex-m0",   .initfn = cortex_m0_initfn,
>> +#endif
>>   .class_init = arm_v7m_class_init },
>>  { .name = "cortex-m3",   .initfn = cortex_m3_initfn,
>>   .class_init = arm_v7m_class_init },
> 
> Ifdef is misplaced.  This shouldn't even compile without CONFIG_TCG.

What a shame... I hope I messed this due to a failed rebase...

> Also, m-profile shouldn't work with kvm either, so I think the endif should go
> below cortex-m33.

Obviously.

Re: [Qemu-devel] [PATCH] i386/vmmouse: Properly reset state

[Philippe Mathieu-Daudé]

Hi Jan,

On 8/27/19 9:49 PM, Eduardo Habkost wrote:
> On Sun, Aug 25, 2019 at 04:58:18PM +0200, Jan Kiszka wrote:
>> On 21.07.19 10:58, Jan Kiszka wrote:
>>> From: Jan Kiszka 
>>>
>>> nb_queue was not zeroed so that we no longer delivered events if a
>>> previous guest left the device in an overflow state.
>>>
>>> The state of absolute does not matter as the next vmmouse_update_handler
>>> call will align it again.
>>>
>>> Signed-off-by: Jan Kiszka 
>>> ---
>>>   hw/i386/vmmouse.c | 1 +
>>>   1 file changed, 1 insertion(+)
>>>
>>> diff --git a/hw/i386/vmmouse.c b/hw/i386/vmmouse.c
>>> index 5d2d278be4..e335bd07da 100644
>>> --- a/hw/i386/vmmouse.c
>>> +++ b/hw/i386/vmmouse.c
>>> @@ -257,6 +257,7 @@ static void vmmouse_reset(DeviceState *d)
>>>   VMMouseState *s = VMMOUSE(d);
>>>
>>>   s->queue_size = VMMOUSE_QUEUE_SIZE;
>>> +s->nb_queue = 0;

Don't we also need to reset the status in case vmmouse_get_status() is
called directly after reset?

 s->status = 0;

With it:
Reviewed-by: Philippe Mathieu-Daudé 

>>>
>>>   vmmouse_disable(s);
>>>   }
>>> --
>>> 2.16.4
>>>
>>>
>>
>> Ping - or who is looking after this?
> 
> Despite being in hw/i386, I think we can say vmmouse.c doesn't
> have a maintainer.  Last time someone changed vmmouse.c in a
> meaningful way (not just adapting to API changes or removing
> duplicate code) was in 2012.
> 

Well it does has a few:

$ ./scripts/get_maintainer.pl -f hw/i386/vmmouse.c
"Michael S. Tsirkin"  (supporter:PC)
Marcel Apfelbaum  (supporter:PC)
Paolo Bonzini  (maintainer:X86 TCG CPUs)
Richard Henderson  (maintainer:X86 TCG CPUs)
Eduardo Habkost  (maintainer:X86 TCG CPUs)

However the correct section should rather be "PC Chipset".

> But the change makes sense to me, so:
> 
> Reviewed-by: Eduardo Habkost 
> 
> I'll queue it.
> 

Re: [Qemu-devel] [libvirt] [PATCH 2/2] qapi: deprecate implicit filters

[John Snow]

On 8/29/19 12:45 PM, Christophe de Dinechin wrote:
> 
> Markus Armbruster writes:
> 
>> Peter Krempa  writes:
>>
> [...]
>>> From my experience users report non-fatal messages mostly only if it is
>>> spamming the system log. One of instances are very unlikely to be
>>> noticed.
>>>
>>> In my experience it's better to notify us in libvirt of such change and
>>> we will try our best to fix it.
>>
>> How to best alert the layers above QEMU was one of the topic of the KVM
>> Forum 2018 BoF on deprecating stuff.  Minutes:
>>
>> Message-ID: <87mur0ls8o@dusky.pond.sub.org>
>> https://lists.nongnu.org/archive/html/qemu-devel/2018-10/msg05828.html
>>
>> Relevant part:
>>
>> * We need to communicate "you're using something that is deprecated".
>>   How?  Right now, we print a deprecation message.  Okay when humans use
>>   QEMU directly in a shell.  However, when QEMU sits at the bottom of a
>>   software stack, the message will likely end up in a log file that is
>>   effectively write-only.
>>
>>   - The one way to get people read log files is crashing their
>> application.  A command line option --future could make QEMU crash
>> right after printing a deprecation message.  This could help with
>> finding use of deprecated features in a testing environment.
>>
>>   - A less destructive way to grab people's attention is to make things
>> run really, really slow: have QEMU go to sleep for a while after
>> printing a deprecation message.
>>
>>   - We can also pass the buck to the next layer up: emit a QMP event.
>>
>> Sadly, by the time the next layer connects to QMP, plenty of stuff
>> already happened.  We'd have to buffer deprecation events somehow.
>>
>> What would libvirt do with such an event?  Log it, taint the domain,
>> emit a (libvirt) event to pass it on to the next layer up.
>>
>>   - A completely different idea is to have a configuratin linter.  To
>> support doing this at the libvirt level, QEMU could expose "is
>> deprecated" in interface introspection.  Feels feasible for QMP,
>> where we already have sufficiently expressive introspection.  For
>> CLI, we'd first have to provide that (but we want that anyway).
>>
>>   - We might also want to dispay deprecation messages in QEMU's GUI
>> somehow, or on serial consoles.
> 
> Sorry for catching up late, this mail thread happened during my PTO.
> 
> I remember bringing up at the time [1] that the correct solution needs
> to take into account usage models that vary from
> 
> - a workstation case, where displaying an error box is easy and
>   convenient,
> 
> - to local headless VMs where system-level notification would do the job
>   better, allowing us to leverage things like system-wide email notifications
> 
> - to large-scale collections of VMs managed by some layered product,
>   where the correct reporting would be through something like Insights,
>   i.e. you don't scan individual logs, you want something like "913 VMs
>   are using deprecated X"
> 
> To me, that implies that we need to have a clear division of roles, with
> a standard way to
> 
> a) produce the errors,
> b) propagate them,

I started replying to this thread to the other mail you sent; I think
this is going to be fairly involved. I wouldn't mind being proven wrong
though.

> c) consume them (at least up to libvirt)
> 
> Notice that this work has already been done for "real" errors,
> i.e. there is a real QAPI notion of "errors". AFAICT, warn_report does
> not connect to it, though, it goes through error_vprintf which is really
> just basic logging.
> 
> So would it make sense to:
> 
> 1. Add a deprecation_report() alongside warn_report()?
> 

Where's that get routed to? just an error_vprintf style situation?

> 2. Connect warn_report() and all the error_vprintf output to QAPI,
>e.g. using John's suggestion of adding the messages using some
>"warning" or "deprecated" tag?
> 

How do you correlate them?

> 3. Teach libvirt how to consume that new tag and pass it along?
> 

I think it's not libvirt's job to pass it along, exactly -- libvirt made
the decision for which features to engage in QEMU, not the end user.

If the user upgrades QEMU but not libvirt, it's not really anything they
have control over and they shouldn't be pestered with such things.

However, if libvirt accidentally released a version that engages
deprecated behavior (and were unaware of it), it'd be nice to get user
reports, surely?

Logging messages for libvirt might be the best that can be done there in
that case.


In contrast, power user tools like QMP libraries, qmp-shell and others
allow more direct and meaningful access to QMP, so those should report
deprecation messages to the user.

> 
> [1] https://lists.nongnu.org/archive/html/qemu-devel/2018-10/msg06131.html
> 
> 
> --
> Cheers,
> Christophe de Dinechin (IRC c3d)
> 

Re: [Qemu-devel] [PATCH v8 08/11] Adding tb_stats [start|pause|stop|filter] command to hmp.

[Vanderson Martins do Rosario]

Ops, this commit shouldn't exist. My mistake. The series still work though.
I will remove this on v9.


Vanderson M. Rosario


On Thu, Aug 29, 2019 at 2:35 PM vandersonmr  wrote:

> This allows controlling the collection of statistics.
> It is also possible to set the level of collection:
> all, jit, or exec.
>
> tb_stats filter allow to only collect statistics for the TB
> in the last_search list.
>
> The goal of this command is to allow the dynamic exploration
> of the TCG behavior and quality. Therefore, for now, a
> corresponding QMP command is not worthwhile.
>
> Acked-by: Dr. David Alan Gilbert 
> Signed-off-by: Vanderson M. do Rosario 
> ---
>  monitor/misc.c | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/monitor/misc.c b/monitor/misc.c
> index b389ca09a1..218263d29a 100644
> --- a/monitor/misc.c
> +++ b/monitor/misc.c
> @@ -74,6 +74,8 @@
>  #include "sysemu/cpus.h"
>  #include "qemu/cutils.h"
>  #include "tcg/tcg.h"
> +#include "exec/tb-stats.h"
> +#include "qemu-common.h"
>
>  #if defined(TARGET_S390X)
>  #include "hw/s390x/storage-keys.h"
> --
> 2.22.0
>
>

Re: [Qemu-devel] [PATCH] target/arm: Free TCG temps in trans_VMOV_64_sp()

[Philippe Mathieu-Daudé]

On 8/27/19 2:19 PM, Peter Maydell wrote:
> The function neon_store_reg32() doesn't free the TCG temp that it
> is passed, so the caller must do that. We got this right in most
> places but forgot to free the TCG temps in trans_VMOV_64_sp().
> 
> Cc: qemu-sta...@nongnu.org
> Signed-off-by: Peter Maydell 
> ---
>  target/arm/translate-vfp.inc.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/target/arm/translate-vfp.inc.c b/target/arm/translate-vfp.inc.c
> index 3e8ea80493b..9ae980bef63 100644
> --- a/target/arm/translate-vfp.inc.c
> +++ b/target/arm/translate-vfp.inc.c
> @@ -880,8 +880,10 @@ static bool trans_VMOV_64_sp(DisasContext *s, 
> arg_VMOV_64_sp *a)
>  /* gpreg to fpreg */
>  tmp = load_reg(s, a->rt);
>  neon_store_reg32(tmp, a->vm);
> +tcg_temp_free_i32(tmp);
>  tmp = load_reg(s, a->rt2);
>  neon_store_reg32(tmp, a->vm + 1);
> +tcg_temp_free_i32(tmp);
>  }
>  
>  return true;

Reviewed-by: Philippe Mathieu-Daudé 

Re: [Qemu-devel] [PATCH 2/2] qapi: deprecate implicit filters

[John Snow]

On 8/29/19 11:17 AM, Vladimir Sementsov-Ogievskiy wrote:
> Aren't you going to deprecate and drop it at some moment?

Libvirt's going to keep supporting older versions of QEMU in perpetuity.
Apparently, there are still some barriers (SD cards?) to using only
blockdev API for new versions, too:

>> "Note that even with new qemu, if an SD card is used blockdev will be
>> disabled."

It sounds like we need to facilitate libvirt's transfer to an
all-blockdev API for modern QEMU instances. Once we do that, we can
likely add an introspectable feature flag to commands that create
formerly-implicit nodes to tip off libvirt as to what behavior it can
expect.

(In practice, if libvirt sees the new flag, it knows it needs to rely
exclusively on blockdev API and that (likely) it should always provide a
node-name for the filter.)

I think it is reasonably clear that deprecating and re-implementing is
not something that will be compatible with libvirt's feature detection,
so we shouldn't do it.

I'm still not sold that this is worth the effort, but you and Max would
know best right now. I'll leave it to you two to sort out.

--js

Re: [Qemu-devel] [Virtio-fs] [PATCH] virtiofsd: add man page

[Liu Bo]

On Thu, Aug 29, 2019 at 11:41:33AM +0100, Stefan Hajnoczi wrote:

Looks good to me, thanks for working this out.

Reviewed-by: Liu Bo 

thanks,
-liubo
> Signed-off-by: Stefan Hajnoczi 
> ---
>  Makefile |  7 +++
>  contrib/virtiofsd/virtiofsd.texi | 85 
>  2 files changed, 92 insertions(+)
>  create mode 100644 contrib/virtiofsd/virtiofsd.texi
> 
> diff --git a/Makefile b/Makefile
> index a3dfdd6fa8..cc18025753 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -334,6 +334,9 @@ DOCS+=docs/qemu-cpu-models.7
>  ifdef CONFIG_VIRTFS
>  DOCS+=fsdev/virtfs-proxy-helper.1
>  endif
> +ifdef CONFIG_LINUX
> +DOCS+=contrib/virtiofsd/virtiofsd.1
> +endif
>  ifdef CONFIG_TRACE_SYSTEMTAP
>  DOCS+=scripts/qemu-trace-stap.1
>  endif
> @@ -834,6 +837,9 @@ ifdef CONFIG_VIRTFS
>   $(INSTALL_DIR) "$(DESTDIR)$(mandir)/man1"
>   $(INSTALL_DATA) fsdev/virtfs-proxy-helper.1 "$(DESTDIR)$(mandir)/man1"
>  endif
> +ifdef CONFIG_LINUX
> + $(INSTALL_DATA) contrib/virtiofsd.1 "$(DESTDIR)$(mandir)/man1"
> +endif
>  
>  install-datadir:
>   $(INSTALL_DIR) "$(DESTDIR)$(qemu_datadir)"
> @@ -1018,6 +1024,7 @@ qemu.1: qemu-doc.texi qemu-options.texi 
> qemu-monitor.texi qemu-monitor-info.texi
>  qemu.1: qemu-option-trace.texi
>  qemu-img.1: qemu-img.texi qemu-option-trace.texi qemu-img-cmds.texi
>  fsdev/virtfs-proxy-helper.1: fsdev/virtfs-proxy-helper.texi
> +contrib/virtiofsd/virtiofsd.1: contrib/virtiofsd/virtiofsd.texi
>  qemu-nbd.8: qemu-nbd.texi qemu-option-trace.texi
>  qemu-ga.8: qemu-ga.texi
>  docs/qemu-block-drivers.7: docs/qemu-block-drivers.texi
> diff --git a/contrib/virtiofsd/virtiofsd.texi 
> b/contrib/virtiofsd/virtiofsd.texi
> new file mode 100644
> index 00..eec7fbf4e6
> --- /dev/null
> +++ b/contrib/virtiofsd/virtiofsd.texi
> @@ -0,0 +1,85 @@
> +@example
> +@c man begin SYNOPSIS
> +@command{virtiofsd} [OPTION] 
> @option{--socket-path=}@var{path}|@option{--fd=}@var{fdnum} @option{-o 
> source=}@var{path}
> +@c man end
> +@end example
> +
> +@c man begin DESCRIPTION
> +
> +Share a host directory tree with a guest through a virtio-fs device.  This
> +program is a vhost-user backend that implements the virtio-fs device.  Each
> +virtio-fs device instance requires its own virtiofsd process.
> +
> +This program is designed to work with QEMU's @code{--device 
> vhost-user-fs-pci}
> +but should work with any virtual machine monitor (VMM) that supports
> +vhost-user.  See the EXAMPLES section below.
> +
> +This program must be run as the root user.  Upon startup the program will
> +switch into a new file system namespace with the shared directory tree as its
> +root.  This prevents "file system escapes" due to symlinks and other file
> +system objects that might lead to files outside the shared directory.  The
> +program also sandboxes itself using seccomp(2) to prevent ptrace(2) and other
> +vectors that could allow an attacker to compromise the system after gaining
> +control of the virtiofsd process.
> +
> +@c man end
> +
> +@c man begin OPTIONS
> +@table @option
> +@item -h, --help
> +Print help.
> +@item -V, --version
> +Print version.
> +@item -d, -o debug
> +Enable debug output.
> +@item --syslog
> +Print log messages to syslog instead of stderr.
> +@item -o log_level=@var{level}
> +Print only log messages matching @var{level} or more severe.  @var{level} is
> +one of @code{err}, @code{warn}, @code{info}, or @code{debug}.  The default is
> +@var{info}.
> +@item -o source=@var{path}
> +Share host directory tree located at @var{path}.  This option is required.
> +@item --socket-path=@var{path}, -o vhost_user_socket=@var{path}
> +Listen on vhost-user UNIX domain socket at @var{path}.
> +@item --fd=@var{fdnum}
> +Accept connections from vhost-user UNIX domain socket file descriptor 
> @var{fdnum}.  The file descriptor must already be listening for connections.
> +@item --thread-pool-size=@var{num}
> +Restrict the number of worker threads per request queue to @var{num}.  The 
> default is 64.
> +@item --cache=@code{none}|@code{auto}|@code{always}
> +Select the desired trade-off between coherency and performance.  @code{none}
> +forbids the FUSE client from caching to achieve best coherency at the cost of
> +performance.  @code{auto} acts similar to NFS with a 1 second metadata cache
> +timeout.  @code{always} sets a long cache lifetime at the expense of 
> coherency.
> +@item --writeback
> +Enable writeback cache, allowing the FUSE client to buffer and merge write 
> requests.
> +@end table
> +@c man end
> +
> +@c man begin EXAMPLES
> +Export @code{/var/lib/fs/vm001/} on vhost-user UNIX domain socket 
> @code{/var/run/vm001-vhost-fs.sock}:
> +
> +@example
> +host# virtiofsd --socket-path=/var/run/vm001-vhost-fs.sock -o 
> source=/var/lib/fs/vm001
> +host# qemu-system-x86_64 \
> +-chardev socket,id=char0,path=/var/run/vm001-vhost-fs.sock \
> +-device vhost-user-fs-pci,chardev=char0,tag=myfs \
> +-object 

[Qemu-devel] [PATCH v8 08/11] Adding tb_stats [start|pause|stop|filter] command to hmp.

[vandersonmr]

This allows controlling the collection of statistics.
It is also possible to set the level of collection:
all, jit, or exec.

tb_stats filter allow to only collect statistics for the TB
in the last_search list.

The goal of this command is to allow the dynamic exploration
of the TCG behavior and quality. Therefore, for now, a
corresponding QMP command is not worthwhile.

Acked-by: Dr. David Alan Gilbert 
Signed-off-by: Vanderson M. do Rosario 
---
 monitor/misc.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/monitor/misc.c b/monitor/misc.c
index b389ca09a1..218263d29a 100644
--- a/monitor/misc.c
+++ b/monitor/misc.c
@@ -74,6 +74,8 @@
 #include "sysemu/cpus.h"
 #include "qemu/cutils.h"
 #include "tcg/tcg.h"
+#include "exec/tb-stats.h"
+#include "qemu-common.h"
 
 #if defined(TARGET_S390X)
 #include "hw/s390x/storage-keys.h"
-- 
2.22.0

[Qemu-devel] [PATCH v8 09/11] Adding info [tb-list|tb|coverset] commands to HMP.

[vandersonmr]

These commands allow the exploration of TBs
generated by the TCG. Understand which one
hotter, with more guest/host instructions...
and examine their guest, host and IR code.

The goal of this command is to allow the dynamic exploration
of TCG behavior and code quality. Therefore, for now, a
corresponding QMP command is not worthwhile.

Example of output:

TB id:1 | phys:0x34d54 virt:0x00034d54 flags:0xf0
| exec:4828932/0 guest inst cov:16.38%
| trans:1 ints: g:3 op:82 op_opt:34 spills:3
| h/g (host bytes / guest insts): 90.64
| time to gen at 2.4GHz => code:3150.83(ns) IR:712.08(ns)
| targets: 0x00034d5e (id:11), 0x00034d0d (id:2)

TB id:2 | phys:0x34d0d virt:0x00034d0d flags:0xf0
| exec:4825842/0 guest inst cov:21.82%
| trans:1 ints: g:4 op:80 op_opt:38 spills:2
| h/g (host bytes / guest insts): 84.00
| time to gen at 2.4GHz => code:3362.92(ns) IR:793.75(ns)
| targets: 0x00034d19 (id:12), 0x00034d54 (id:1)

TB id:2 | phys:0x34d0d virt:0x00034d0d flags:0xf0
| exec:6956495/0  guest inst cov:21.82%
| trans:2 ints: g:2 op:40 op_opt:19 spills:1
| h/g (host bytes / guest insts): 84.00
| time to gen at 2.4GHz => code:3130.83(ns) IR:722.50(ns)
| targets: 0x00034d19 (id:12), 0x00034d54 (id:1)


IN:
0x00034d0d:  89 demovl %ebx, %esi
0x00034d0f:  26 8b 0e movl %es:(%esi), %ecx
0x00034d12:  26 f6 46 08 80   testb$0x80, %es:8(%esi)
0x00034d17:  75 3bjne  0x34d54

--

TB id:1 | phys:0x34d54 virt:0x00034d54 flags:0xf0
| exec:5202686/0 guest inst cov:11.28%
| trans:1 ints: g:3 op:82 op_opt:34 spills:3
| h/g (host bytes / guest insts): 90.64
| time to gen at 2.4GHz => code:2793.75(ns) IR:614.58(ns)
| targets: 0x00034d5e (id:3), 0x00034d0d (id:2)

TB id:2 | phys:0x34d0d virt:0x00034d0d flags:0xf0
| exec:5199468/0 guest inst cov:15.03%
| trans:1 ints: g:4 op:80 op_opt:38 spills:2
| h/g (host bytes / guest insts): 84.00
| time to gen at 2.4GHz => code:2958.75(ns) IR:719.58(ns)
| targets: 0x00034d19 (id:4), 0x00034d54 (id:1)

--
2 TBs to reach 25% of guest inst exec coverage
Total of guest insts exec: 138346727

--

Acked-by: Dr. David Alan Gilbert 
Signed-off-by: Vanderson M. do Rosario 
---
 accel/tcg/tb-stats.c | 421 ++-
 accel/tcg/translate-all.c|   2 +-
 disas.c  |  31 ++-
 hmp-commands-info.hx |  24 ++
 include/exec/tb-stats.h  |  45 +++-
 include/qemu/log-for-trace.h |   4 +
 include/qemu/log.h   |   2 +
 monitor/misc.c   |  74 ++
 util/log.c   |  52 -
 9 files changed, 634 insertions(+), 21 deletions(-)

diff --git a/accel/tcg/tb-stats.c b/accel/tcg/tb-stats.c
index 9959477fbb..d588c551c9 100644
--- a/accel/tcg/tb-stats.c
+++ b/accel/tcg/tb-stats.c
@@ -34,9 +34,35 @@
 
 /* only accessed in safe work */
 static GList *last_search;
-
+int id = 1; /* display_id increment counter */
 uint64_t dev_time;
 
+static TBStatistics *get_tbstats_by_id(int id)
+{
+GList *iter;
+
+for (iter = last_search; iter; iter = g_list_next(iter)) {
+TBStatistics *tbs = iter->data;
+if (tbs && tbs->display_id == id) {
+return tbs;
+break;
+}
+}
+return NULL;
+}
+
+static TBStatistics *get_tbstats_by_addr(target_ulong pc)
+{
+GList *iter;
+for (iter = last_search; iter; iter = g_list_next(iter)) {
+TBStatistics *tbs = iter->data;
+if (tbs && tbs->pc == pc) {
+return tbs;
+}
+}
+return NULL;
+}
+
 struct jit_profile_info {
 uint64_t translations;
 uint64_t aborted;
@@ -175,6 +201,7 @@ static void clean_tbstats(void)
 qht_destroy(_ctx.tb_stats);
 }
 
+
 void do_hmp_tbstats_safe(CPUState *cpu, run_on_cpu_data icmd)
 {
 struct TbstatsCommand *cmdinfo = icmd.host_ptr;
@@ -261,6 +288,398 @@ void init_tb_stats_htable_if_not(void)
 }
 }
 
+static void collect_tb_stats(void *p, uint32_t hash, void *userp)
+{
+last_search = g_list_prepend(last_search, p);
+}
+
+static void dump_tb_targets(TBStatistics *tbs)
+{
+if (tbs && tbs->tb) {
+TBStatistics *valid_target_tbstats[2] = {NULL, NULL};
+
+/*
+ * Check and ensure that targets' tbstats have a valid display_id and
+ * are in last_search list
+ */
+for (int jmp_id = 0; jmp_id < 2; jmp_id++) {
+qemu_spin_lock(>tb->jmp_lock);
+TranslationBlock *tb_dst =
+(TranslationBlock *) (atomic_read(>tb->jmp_dest[jmp_id]) 
& ~1);
+

[Qemu-devel] [PATCH v8 05/11] accel: adding TB_JIT_TIME and full replacing CONFIG_PROFILER

[vandersonmr]

Replace all others CONFIG_PROFILER statistics and migrate it to
TBStatistics system. However, TCGProfiler still exists and can
be use to store global statistics and times. All TB related
statistics goes to TBStatistics.

Signed-off-by: Vanderson M. do Rosario 
---
 accel/tcg/tb-stats.c  |  91 +++-
 accel/tcg/translate-all.c |  47 ---
 configure |   3 -
 cpus.c|  14 ++---
 include/exec/tb-stats.h   |  19 +-
 include/qemu/timer.h  |   5 +-
 monitor/misc.c|  28 ++---
 tcg/tcg.c | 124 +++---
 tcg/tcg.h |  10 +--
 vl.c  |   8 +--
 10 files changed, 156 insertions(+), 193 deletions(-)

diff --git a/accel/tcg/tb-stats.c b/accel/tcg/tb-stats.c
index 176da60e13..66abc97ad4 100644
--- a/accel/tcg/tb-stats.c
+++ b/accel/tcg/tb-stats.c
@@ -32,6 +32,8 @@
 
 #include "exec/tb-stats.h"
 
+uint64_t dev_time;
+
 struct jit_profile_info {
 uint64_t translations;
 uint64_t aborted;
@@ -43,6 +45,13 @@ struct jit_profile_info {
 uint64_t host;
 uint64_t guest;
 uint64_t search_data;
+
+uint64_t interm_time;
+uint64_t code_time;
+uint64_t restore_count;
+uint64_t restore_time;
+uint64_t opt_time;
+uint64_t la_time;
 };
 
 /* accumulate the statistics from all TBs */
@@ -64,6 +73,29 @@ static void collect_jit_profile_info(void *p, uint32_t hash, 
void *userp)
 jpi->host += tbs->code.out_len;
 jpi->guest += tbs->code.in_len;
 jpi->search_data += tbs->code.search_out_len;
+
+jpi->interm_time += stat_per_translation(tbs, time.interm);
+jpi->code_time += stat_per_translation(tbs, time.code);
+jpi->opt_time += stat_per_translation(tbs, time.opt);
+jpi->la_time += stat_per_translation(tbs, time.la);
+jpi->restore_time += tbs->time.restore;
+jpi->restore_count += tbs->time.restore_count;
+}
+
+void dump_jit_exec_time_info(uint64_t dev_time)
+{
+static uint64_t last_cpu_exec_time;
+uint64_t cpu_exec_time;
+uint64_t delta;
+
+cpu_exec_time = tcg_cpu_exec_time();
+delta = cpu_exec_time - last_cpu_exec_time;
+
+qemu_printf("async time  %" PRId64 " (%0.3f)\n",
+   dev_time, dev_time / (double) NANOSECONDS_PER_SECOND);
+qemu_printf("qemu time   %" PRId64 " (%0.3f)\n",
+   delta, delta / (double) NANOSECONDS_PER_SECOND);
+last_cpu_exec_time = cpu_exec_time;
 }
 
 /* dump JIT statisticis using TCGProfile and TBStats */
@@ -90,34 +122,39 @@ void dump_jit_profile_info(TCGProfile *s)
 qemu_printf("avg search data/TB  %0.1f\n",
 jpi->search_data / (double) jpi->translations);
 
+uint64_t tot = jpi->interm_time + jpi->code_time;
+
+qemu_printf("JIT cycles  %" PRId64 " (%0.3fs at 2.4 GHz)\n",
+tot, tot / 2.4e9);
+qemu_printf("  cycles/op   %0.1f\n",
+jpi->ops ? (double)tot / jpi->ops : 0);
+qemu_printf("  cycles/in byte  %0.1f\n",
+jpi->guest ? (double)tot / jpi->guest : 0);
+qemu_printf("  cycles/out byte %0.1f\n",
+jpi->host ? (double)tot / jpi->host : 0);
+qemu_printf("  cycles/search byte  %0.1f\n",
+jpi->search_data ? (double)tot / jpi->search_data : 0);
+if (tot == 0) {
+tot = 1;
+}
+
+qemu_printf("  gen_interm time %0.1f%%\n",
+(double)jpi->interm_time / tot * 100.0);
+qemu_printf("  gen_code time   %0.1f%%\n",
+(double)jpi->code_time / tot * 100.0);
+
+qemu_printf("optim./code time%0.1f%%\n",
+(double)jpi->opt_time / (jpi->code_time ? jpi->code_time : 1) 
* 100.0);
+qemu_printf("liveness/code time  %0.1f%%\n",
+(double)jpi->la_time / (jpi->code_time ? jpi->code_time : 1) * 
100.0);
+
+qemu_printf("cpu_restore count   %" PRId64 "\n", jpi->restore_count);
+qemu_printf("  avg cycles%0.1f\n",
+jpi->restore_count ? (double)jpi->restore_time / 
jpi->restore_count : 0);
+
 if (s) {
-int64_t tot = s->interm_time + s->code_time;
-qemu_printf("JIT cycles  %" PRId64 " (%0.3f s at 2.4 
GHz)\n",
-tot, tot / 2.4e9);
-qemu_printf("cycles/op   %0.1f\n",
-jpi->ops ? (double)tot / jpi->ops : 0);
-qemu_printf("cycles/in byte  %0.1f\n",
-jpi->guest ? (double)tot / jpi->guest : 0);
-qemu_printf("cycles/out byte %0.1f\n",
-jpi->host ? (double)tot / jpi->host : 0);
-qemu_printf("cycles/search byte %0.1f\n",
-jpi->search_data ? (double)tot / jpi->search_data : 0);
-if (tot == 0) {
-tot = 1;
-}
-qemu_printf("  

[Qemu-devel] [PATCH v8 07/11] monitor: adding tb_stats hmp command

[vandersonmr]

Adding tb_stats [start|pause|stop|filter] command to hmp.
This allows controlling the collection of statistics.
It is also possible to set the level of collection:
all, jit, or exec.

tb_stats filter allow to only collect statistics for the TB
in the last_search list.

The goal of this command is to allow the dynamic exploration
of the TCG behavior and quality. Therefore, for now, a
corresponding QMP command is not worthwhile.

Acked-by: Dr. David Alan Gilbert 
Signed-off-by: Vanderson M. do Rosario 
---
 accel/tcg/tb-stats.c  | 110 ++
 hmp-commands.hx   |  17 ++
 include/exec/tb-stats-flags.h |   1 +
 include/exec/tb-stats.h   |  11 
 monitor/misc.c|  47 +++
 vl.c  |   6 ++
 6 files changed, 192 insertions(+)

diff --git a/accel/tcg/tb-stats.c b/accel/tcg/tb-stats.c
index 66abc97ad4..9959477fbb 100644
--- a/accel/tcg/tb-stats.c
+++ b/accel/tcg/tb-stats.c
@@ -32,6 +32,9 @@
 
 #include "exec/tb-stats.h"
 
+/* only accessed in safe work */
+static GList *last_search;
+
 uint64_t dev_time;
 
 struct jit_profile_info {
@@ -160,6 +163,95 @@ void dump_jit_profile_info(TCGProfile *s)
 g_free(jpi);
 }
 
+static void free_tbstats(void *p, uint32_t hash, void *userp)
+{
+g_free(p);
+}
+
+static void clean_tbstats(void)
+{
+/* remove all tb_stats */
+qht_iter(_ctx.tb_stats, free_tbstats, NULL);
+qht_destroy(_ctx.tb_stats);
+}
+
+void do_hmp_tbstats_safe(CPUState *cpu, run_on_cpu_data icmd)
+{
+struct TbstatsCommand *cmdinfo = icmd.host_ptr;
+int cmd = cmdinfo->cmd;
+uint32_t level = cmdinfo->level;
+
+switch (cmd) {
+case START:
+if (tb_stats_collection_paused()) {
+set_tbstats_flags(level);
+} else {
+if (tb_stats_collection_enabled()) {
+qemu_printf("TB information already being recorded");
+return;
+}
+qht_init(_ctx.tb_stats, tb_stats_cmp, CODE_GEN_HTABLE_SIZE,
+QHT_MODE_AUTO_RESIZE);
+}
+
+set_default_tbstats_flag(level);
+enable_collect_tb_stats();
+tb_flush(cpu);
+break;
+case PAUSE:
+if (!tb_stats_collection_enabled()) {
+qemu_printf("TB information not being recorded");
+return;
+}
+
+/* Continue to create TBStatistic structures but stop collecting 
statistics */
+pause_collect_tb_stats();
+set_default_tbstats_flag(TB_NOTHING);
+set_tbstats_flags(TB_PAUSED);
+tb_flush(cpu);
+break;
+case STOP:
+if (!tb_stats_collection_enabled()) {
+qemu_printf("TB information not being recorded");
+return;
+}
+
+/* Dissalloc all TBStatistics structures and stop creating new ones */
+disable_collect_tb_stats();
+clean_tbstats();
+tb_flush(cpu);
+break;
+case FILTER:
+if (!tb_stats_collection_enabled()) {
+qemu_printf("TB information not being recorded");
+return;
+}
+if (!last_search) {
+qemu_printf("no search on record! execute info tbs before 
filtering!");
+return;
+}
+
+set_default_tbstats_flag(TB_NOTHING);
+
+/* Set all tbstats as paused, then return only the ones from 
last_search */
+pause_collect_tb_stats();
+set_tbstats_flags(TB_PAUSED);
+
+for (GList *iter = last_search; iter; iter = g_list_next(iter)) {
+TBStatistics *tbs = iter->data;
+tbs->stats_enabled = level;
+}
+
+tb_flush(cpu);
+
+break;
+default: /* INVALID */
+g_assert_not_reached();
+break;
+}
+
+g_free(cmdinfo);
+}
 
 void init_tb_stats_htable_if_not(void)
 {
@@ -195,6 +287,24 @@ bool tb_stats_collection_paused(void)
 return tcg_collect_tb_stats == TB_STATS_PAUSED;
 }
 
+static void reset_tbstats_flag(void *p, uint32_t hash, void *userp)
+{
+uint32_t flag = *((int *)userp);
+TBStatistics *tbs = p;
+tbs->stats_enabled = flag;
+}
+
+void set_default_tbstats_flag(uint32_t flag)
+{
+default_tbstats_flag = flag;
+}
+
+void set_tbstats_flags(uint32_t flag)
+{
+/* iterate over tbstats setting their flag as TB_NOTHING */
+qht_iter(_ctx.tb_stats, reset_tbstats_flag, );
+}
+
 uint32_t get_default_tbstats_flag(void)
 {
 return default_tbstats_flag;
diff --git a/hmp-commands.hx b/hmp-commands.hx
index cfcc044ce4..6cd2800378 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1886,6 +1886,23 @@ STEXI
 @findex qemu-io
 Executes a qemu-io command on the given block device.
 
+ETEXI
+#if defined(CONFIG_TCG)
+{
+.name   = "tb_stats",
+.args_type  = "command:s,level:s?",
+.params = "command [stats_level]",
+.help   = "Control tb statistics collection:"
+"tb_stats 

Re: [Qemu-devel] [PATCH 0/2] git.orderfile: Order Python/shell scripts before unordered files

[John Snow]

On 8/29/19 6:05 AM, Philippe Mathieu-Daudé wrote:
> This series update the git.orderfile to order Python and shell
> scripts before unordered files.
> This is particularly useful for changes in tests/qemu-iotests.
> 
> Regards,
> 
> Phil.
> 
> Philippe Mathieu-Daudé (2):
>   scripts/git.orderfile: Order Python files before unordered ones
>   scripts/git.orderfile: Order shell scripts before unordered files
> 
>  scripts/git.orderfile | 5 +
>  1 file changed, 5 insertions(+)
> 

ACK, not that you need more reviews for a trivial patch:

Reviewed-by: John Snow 

[Qemu-devel] [PATCH v8 10/11] monitor: adding new info cfg command

[vandersonmr]

Adding "info cfg id depth" commands to HMP.
This command allow the exploration a TB
neighbors by dumping [and opening] a .dot
file with the TB CFG neighbors colorized
by their hotness.

The goal of this command is to allow the dynamic exploration
of TCG behavior and code quality. Therefore, for now, a
corresponding QMP command is not worthwhile.

Signed-off-by: Vanderson M. do Rosario 
---
 accel/tcg/tb-stats.c| 164 
 hmp-commands-info.hx|   7 ++
 include/exec/tb-stats.h |   1 +
 monitor/misc.c  |  22 ++
 4 files changed, 194 insertions(+)

diff --git a/accel/tcg/tb-stats.c b/accel/tcg/tb-stats.c
index d588c551c9..fd2344c5d1 100644
--- a/accel/tcg/tb-stats.c
+++ b/accel/tcg/tb-stats.c
@@ -679,6 +679,170 @@ void dump_tb_info(int id, int log_mask, bool use_monitor)
 /* tbdi free'd by do_dump_tb_info_safe */
 }
 
+/* TB CFG xdot/dot dump implementation */
+#define MAX_CFG_NUM_NODES 1000
+static int cfg_tb_id;
+static GHashTable *cfg_nodes;
+static uint64_t root_count;
+
+static void fputs_jump(TBStatistics *from, TBStatistics *to, FILE *dot)
+{
+if (!from || !to) {
+return;
+}
+
+int *from_id = (int *) g_hash_table_lookup(cfg_nodes, from);
+int *to_id   = (int *) g_hash_table_lookup(cfg_nodes, to);
+
+if (!from_id || !to_id) {
+return;
+}
+
+fprintf(dot, "   node_%d -> node_%d;\n", *from_id, *to_id);
+}
+
+/* Hotness colors used in the CFG */
+#define HOT_RED1  0xFF000 /* RGB(255,0,0) */
+#define HOT_RED2  0xFF333 /* RGB(255,51,48)   */
+#define MILD_RED  0xFF666 /* RGB(255,102,96)  */
+#define WEAK_RED1 0xFF999 /* RGB(255,153,144) */
+#define WEAK_RED2 0xFFCCC /* RGB(255,204,192) */
+
+static void fputs_tbstats(TBStatistics *tbs, FILE *dot, int log_flags)
+{
+if (!tbs) {
+return;
+}
+
+uint32_t color = MILD_RED;
+uint64_t count = tbs->executions.normal;
+if (count > 1.6 * root_count) {
+color = HOT_RED1;
+} else if (count > 1.2 * root_count) {
+color = HOT_RED2;
+} else if (count < 0.4 * root_count) {
+color = WEAK_RED2;
+} else if (count < 0.8 * root_count) {
+color = WEAK_RED1;
+}
+
+GString *code_s = get_code_string(tbs, log_flags);
+
+for (int i = 0; i < code_s->len; i++) {
+if (code_s->str[i] == '\n') {
+code_s->str[i] = ' ';
+code_s = g_string_insert(code_s, i, "\\l");
+i += 2;
+}
+}
+
+fprintf(dot,
+"   node_%d [fillcolor=\"#%xFF\" shape=\"record\" "
+"label=\"TB %d\\l"
+"-\\l"
+"PC:\t0x"TARGET_FMT_lx"\\l"
+"exec count:\t%lu\\l"
+"\\l %s\"];\n",
+cfg_tb_id, color, cfg_tb_id, tbs->pc,
+tbs->executions.normal, code_s->str);
+
+int *id = g_new(int, 1);
+*id = cfg_tb_id;
+g_hash_table_insert(cfg_nodes, tbs, id);
+
+cfg_tb_id++;
+
+g_string_free(code_s, true);
+}
+
+static void fputs_preorder_walk(TBStatistics *tbs, int depth, FILE *dot, int 
log_flags)
+{
+if (tbs && depth > 0
+&& cfg_tb_id < MAX_CFG_NUM_NODES
+&& !g_hash_table_contains(cfg_nodes, tbs)) {
+
+fputs_tbstats(tbs, dot, log_flags);
+
+if (tbs->tb) {
+TranslationBlock *left_tb  = NULL;
+TranslationBlock *right_tb = NULL;
+if (tbs->tb->jmp_dest[0]) {
+left_tb = (TranslationBlock *) atomic_read(tbs->tb->jmp_dest);
+}
+if (tbs->tb->jmp_dest[1]) {
+right_tb = (TranslationBlock *) atomic_read(tbs->tb->jmp_dest 
+ 1);
+}
+
+if (left_tb) {
+fputs_preorder_walk(left_tb->tb_stats, depth - 1, dot, 
log_flags);
+fputs_jump(tbs, left_tb->tb_stats, dot);
+}
+if (right_tb) {
+fputs_preorder_walk(right_tb->tb_stats, depth - 1, dot, 
log_flags);
+fputs_jump(tbs, right_tb->tb_stats, dot);
+}
+}
+}
+}
+
+struct PreorderInfo {
+TBStatistics *tbs;
+int depth;
+int log_flags;
+};
+
+static void fputs_preorder_walk_safe(CPUState *cpu, run_on_cpu_data icmd)
+{
+struct PreorderInfo *info = icmd.host_ptr;
+
+GString *file_name = g_string_new(NULL);;
+g_string_printf(file_name, "/tmp/qemu-cfg-tb-%d-%d.dot", id, info->depth);
+FILE *dot = fopen(file_name->str, "w+");
+
+fputs(
+"digraph G {\n"
+"   mclimit=1.5;\n"
+"   rankdir=TD; ordering=out;\n"
+"   graph[fontsize=10 fontname=\"Verdana\"];\n"
+"   color=\"#efefef\";\n"
+"   node[shape=box style=filled fontsize=8 fontname=\"Verdana\" 
fillcolor=\"#efefef\"];\n"
+"   edge[fontsize=8 fontname=\"Verdana\"];\n"
+ , dot);
+
+cfg_nodes = g_hash_table_new(NULL, NULL);
+fputs_preorder_walk(info->tbs, info->depth, dot, info->log_flags);
+

[Qemu-devel] [PATCH v8 06/11] Adding -d tb_stats to control TBStatistics collection:

[vandersonmr]

 -d tb_stats[[,level=(+all+jit+exec+time)][,dump_limit=]]

"dump_limit" is used to limit the number of dumped TBStats in
linux-user mode.

[all+jit+exec+time] control the profilling level used
by the TBStats. Can be used as follow:

-d tb_stats
-d tb_stats,level=jit+time
-d tb_stats,dump_limit=15
...

Signed-off-by: Vanderson M. do Rosario 
---
 include/exec/gen-icount.h |  1 +
 include/exec/tb-stats-flags.h | 42 +++
 include/exec/tb-stats.h   | 18 +++
 include/qemu/log.h|  1 +
 util/log.c| 35 +
 5 files changed, 82 insertions(+), 15 deletions(-)
 create mode 100644 include/exec/tb-stats-flags.h

diff --git a/include/exec/gen-icount.h b/include/exec/gen-icount.h
index be006383b9..3987adfb0e 100644
--- a/include/exec/gen-icount.h
+++ b/include/exec/gen-icount.h
@@ -2,6 +2,7 @@
 #define GEN_ICOUNT_H
 
 #include "qemu/timer.h"
+#include "tb-stats-flags.h"
 
 /* Helpers for instruction counting code generation.  */
 
diff --git a/include/exec/tb-stats-flags.h b/include/exec/tb-stats-flags.h
new file mode 100644
index 00..c936ac1084
--- /dev/null
+++ b/include/exec/tb-stats-flags.h
@@ -0,0 +1,42 @@
+/*
+ * QEMU System Emulator, Code Quality Monitor System
+ *
+ * Copyright (c) 2019 Vanderson M. do Rosario 
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to 
deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+#ifndef TB_STATS_FLAGS
+#define TB_STATS_FLAGS
+
+enum TBStatsStatus {
+TB_STATS_DISABLED = 0,
+TB_STATS_RUNNING,
+TB_STATS_PAUSED,
+TB_STATS_STOPPED
+};
+
+#define TB_NOTHING(1 << 0)
+#define TB_EXEC_STATS (1 << 1)
+#define TB_JIT_STATS  (1 << 2)
+#define TB_JIT_TIME   (1 << 3)
+
+extern int tcg_collect_tb_stats;
+extern uint32_t default_tbstats_flag;
+
+#endif
diff --git a/include/exec/tb-stats.h b/include/exec/tb-stats.h
index 0b9a6e2f72..6a53bef31b 100644
--- a/include/exec/tb-stats.h
+++ b/include/exec/tb-stats.h
@@ -30,6 +30,8 @@
 #include "exec/tb-context.h"
 #include "tcg.h"
 
+#include "exec/tb-stats-flags.h"
+
 #define tb_stats_enabled(tb, JIT_STATS) \
 (tb && tb->tb_stats && (tb->tb_stats->stats_enabled & JIT_STATS))
 
@@ -98,26 +100,12 @@ bool tb_stats_cmp(const void *ap, const void *bp);
 
 void dump_jit_exec_time_info(uint64_t dev_time);
 
+void set_tbstats_flags(uint32_t flags);
 void init_tb_stats_htable_if_not(void);
 
 void dump_jit_profile_info(TCGProfile *s);
 
 /* TBStatistic collection controls */
-enum TBStatsStatus {
-TB_STATS_DISABLED = 0,
-TB_STATS_RUNNING,
-TB_STATS_PAUSED,
-TB_STATS_STOPPED
-};
-
-#define TB_NOTHING(1 << 0)
-#define TB_EXEC_STATS (1 << 1)
-#define TB_JIT_STATS  (1 << 2)
-#define TB_JIT_TIME   (1 << 3)
-
-extern int tcg_collect_tb_stats;
-extern uint32_t default_tbstats_flag;
-
 void enable_collect_tb_stats(void);
 void disable_collect_tb_stats(void);
 void pause_collect_tb_stats(void);
diff --git a/include/qemu/log.h b/include/qemu/log.h
index b097a6cae1..a8d1997cde 100644
--- a/include/qemu/log.h
+++ b/include/qemu/log.h
@@ -45,6 +45,7 @@ static inline bool qemu_log_separate(void)
 /* LOG_TRACE (1 << 15) is defined in log-for-trace.h */
 #define CPU_LOG_TB_OP_IND  (1 << 16)
 #define CPU_LOG_TB_FPU (1 << 17)
+#define CPU_LOG_TB_STATS   (1 << 18)
 
 /* Lock output for a series of related logs.  Since this is not needed
  * for a single qemu_log / qemu_log_mask / qemu_log_mask_and_addr, we
diff --git a/util/log.c b/util/log.c
index 29021a4584..c3805b331b 100644
--- a/util/log.c
+++ b/util/log.c
@@ -19,17 +19,20 @@
 
 #include "qemu/osdep.h"
 #include "qemu/log.h"
+#include "qemu/qemu-print.h"
 #include "qemu/range.h"
 #include "qemu/error-report.h"
 #include "qapi/error.h"
 #include "qemu/cutils.h"
 #include "trace/control.h"
+#include "exec/tb-stats-flags.h"
 
 static char *logfilename;
 FILE *qemu_logfile;
 int qemu_loglevel;
 static int log_append = 0;
 static GArray *debug_regions;
+int32_t max_num_hot_tbs_to_dump;
 
 

[Qemu-devel] [PATCH v8 11/11] linux-user: dumping hot TBs at the end of the execution

[vandersonmr]

dumps, in linux-user mode, the hottest TBs if -d tb_stats is used.

Example of output for the 3 hottest TBs:

TB id:1 | phys:0x34d54 virt:0x00034d54 flags:0xf0
| exec:4828932/0 guest inst cov:16.38%
| trans:1 ints: g:3 op:82 op_opt:34 spills:3
| h/g (host bytes / guest insts): 90.64
| time to gen at 2.4GHz => code:3150.83(ns) IR:712.08(ns)
| targets: 0x00034d5e (id:11), 0x00034d0d (id:2)

TB id:2 | phys:0x34d0d virt:0x00034d0d flags:0xf0
| exec:4825842/0 guest inst cov:21.82%
| trans:1 ints: g:4 op:80 op_opt:38 spills:2
| h/g (host bytes / guest insts): 84.00
| time to gen at 2.4GHz => code:3362.92(ns) IR:793.75(ns)
| targets: 0x00034d19 (id:12), 0x00034d54 (id:1)

TB id:3 | phys:0xec1c1 virt:0x000ec1c1 flags:0xb0
| exec:872032/0 guest inst cov:1.97%
| trans:1 ints: g:2 op:56 op_opt:26 spills:1
| h/g (host bytes / guest insts): 68.00
| time to gen at 2.4GHz => code:1692.08(ns) IR:473.75(ns)
| targets: 0x000ec1c5 (id:4), 0x000ec1cb (id:13)

Signed-off-by: Vanderson M. do Rosario 
---
 linux-user/exit.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/linux-user/exit.c b/linux-user/exit.c
index bdda720553..7226104959 100644
--- a/linux-user/exit.c
+++ b/linux-user/exit.c
@@ -28,6 +28,10 @@ extern void __gcov_dump(void);
 
 void preexit_cleanup(CPUArchState *env, int code)
 {
+if (tb_stats_collection_enabled()) {
+dump_tbs_info(max_num_hot_tbs_to_dump, SORT_BY_HOTNESS, false);
+}
+
 #ifdef TARGET_GPROF
 _mcleanup();
 #endif
-- 
2.22.0

[Qemu-devel] [PATCH v8 02/11] accel: collecting TB execution count

[vandersonmr]

If a TB has a TBS (TBStatistics) with the TB_EXEC_STATS
enabled, then we instrument the start code of this TB
to atomically count the number of times it is executed.
We count both the number of "normal" executions and atomic
executions of a TB.

The execution count of the TB is stored in its respective
TBS.

All TBStatistics are created by default with the flags from
default_tbstats_flag.

Signed-off-by: Vanderson M. do Rosario 
---
 accel/tcg/cpu-exec.c  |  4 
 accel/tcg/tb-stats.c  |  5 +
 accel/tcg/tcg-runtime.c   |  7 +++
 accel/tcg/tcg-runtime.h   |  2 ++
 accel/tcg/translate-all.c |  7 +++
 accel/tcg/translator.c|  1 +
 include/exec/gen-icount.h |  9 +
 include/exec/tb-stats.h   | 19 +++
 util/log.c|  1 +
 9 files changed, 55 insertions(+)

diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index 48272c781b..9b2b7bff80 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -251,6 +251,10 @@ void cpu_exec_step_atomic(CPUState *cpu)
 
 start_exclusive();
 
+if (tb_stats_enabled(tb, TB_EXEC_STATS)) {
+tb->tb_stats->executions.atomic++;
+}
+
 /* Since we got here, we know that parallel_cpus must be true.  */
 parallel_cpus = false;
 in_exclusive_region = true;
diff --git a/accel/tcg/tb-stats.c b/accel/tcg/tb-stats.c
index 948b107e68..1db81d83e7 100644
--- a/accel/tcg/tb-stats.c
+++ b/accel/tcg/tb-stats.c
@@ -61,3 +61,8 @@ bool tb_stats_collection_paused(void)
 {
 return tcg_collect_tb_stats == TB_STATS_PAUSED;
 }
+
+uint32_t get_default_tbstats_flag(void)
+{
+return default_tbstats_flag;
+}
diff --git a/accel/tcg/tcg-runtime.c b/accel/tcg/tcg-runtime.c
index 8a1e408e31..6f4aafba11 100644
--- a/accel/tcg/tcg-runtime.c
+++ b/accel/tcg/tcg-runtime.c
@@ -167,3 +167,10 @@ void HELPER(exit_atomic)(CPUArchState *env)
 {
 cpu_loop_exit_atomic(env_cpu(env), GETPC());
 }
+
+void HELPER(inc_exec_freq)(void *ptr)
+{
+TBStatistics *stats = (TBStatistics *) ptr;
+g_assert(stats);
+atomic_inc(>executions.normal);
+}
diff --git a/accel/tcg/tcg-runtime.h b/accel/tcg/tcg-runtime.h
index 4fa61b49b4..bf0b75dbe8 100644
--- a/accel/tcg/tcg-runtime.h
+++ b/accel/tcg/tcg-runtime.h
@@ -28,6 +28,8 @@ DEF_HELPER_FLAGS_1(lookup_tb_ptr, TCG_CALL_NO_WG_SE, ptr, env)
 
 DEF_HELPER_FLAGS_1(exit_atomic, TCG_CALL_NO_WG, noreturn, env)
 
+DEF_HELPER_FLAGS_1(inc_exec_freq, TCG_CALL_NO_RWG, void, ptr)
+
 #ifdef CONFIG_SOFTMMU
 
 DEF_HELPER_FLAGS_5(atomic_cmpxchgb, TCG_CALL_NO_WG,
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index b7bccacd3b..e72aeba682 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -1785,6 +1785,13 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
  */
 if (tb_stats_collection_enabled()) {
 tb->tb_stats = tb_get_stats(phys_pc, pc, cs_base, flags, tb);
+
+if (qemu_log_in_addr_range(tb->pc)) {
+uint32_t flag = get_default_tbstats_flag();
+if (flag & TB_EXEC_STATS) {
+tb->tb_stats->stats_enabled |= TB_EXEC_STATS;
+}
+}
 } else {
 tb->tb_stats = NULL;
 }
diff --git a/accel/tcg/translator.c b/accel/tcg/translator.c
index 70c66c538c..ec6bd829a0 100644
--- a/accel/tcg/translator.c
+++ b/accel/tcg/translator.c
@@ -46,6 +46,7 @@ void translator_loop(const TranslatorOps *ops, 
DisasContextBase *db,
 
 ops->init_disas_context(db, cpu);
 tcg_debug_assert(db->is_jmp == DISAS_NEXT);  /* no early exit */
+gen_tb_exec_count(tb);
 
 /* Reset the temp count so that we can identify leaks */
 tcg_clear_temp_count();
diff --git a/include/exec/gen-icount.h b/include/exec/gen-icount.h
index 822c43cfd3..be006383b9 100644
--- a/include/exec/gen-icount.h
+++ b/include/exec/gen-icount.h
@@ -32,6 +32,15 @@ static inline void gen_io_end(void)
 tcg_temp_free_i32(tmp);
 }
 
+static inline void gen_tb_exec_count(TranslationBlock *tb)
+{
+if (tb_stats_enabled(tb, TB_EXEC_STATS)) {
+TCGv_ptr ptr = tcg_const_ptr(tb->tb_stats);
+gen_helper_inc_exec_freq(ptr);
+tcg_temp_free_ptr(ptr);
+}
+}
+
 static inline void gen_tb_start(TranslationBlock *tb)
 {
 TCGv_i32 count, imm;
diff --git a/include/exec/tb-stats.h b/include/exec/tb-stats.h
index 898e05a36f..c4a8715400 100644
--- a/include/exec/tb-stats.h
+++ b/include/exec/tb-stats.h
@@ -30,6 +30,9 @@
 #include "exec/tb-context.h"
 #include "tcg.h"
 
+#define tb_stats_enabled(tb, JIT_STATS) \
+(tb && tb->tb_stats && (tb->tb_stats->stats_enabled & JIT_STATS))
+
 typedef struct TBStatistics TBStatistics;
 
 /*
@@ -46,6 +49,15 @@ struct TBStatistics {
 uint32_t flags;
 /* cs_base isn't included in the hash but we do check for matches */
 target_ulong cs_base;
+
+uint32_t stats_enabled;
+
+/* Execution stats */
+struct {
+unsigned long normal;
+unsigned long atomic;
+} executions;
+
 /* current TB 

[Qemu-devel] [PATCH v8 03/11] accel: collecting JIT statistics

[vandersonmr]

If a TB has a TBS (TBStatistics) with the TB_JIT_STATS
enabled then we collect statistics of its translation
processes and code translation.

Collecting the number of host instructions seems to be
not simple as it would imply in having to modify several
target source files. So, for now, we are only collecting
the size of the host gen code.

Reviewed-by: Alex Bennée 
Signed-off-by: Vanderson M. do Rosario 
---
 accel/tcg/translate-all.c | 15 ++-
 accel/tcg/translator.c|  4 
 include/exec/tb-stats.h   | 15 +++
 tcg/tcg.c | 23 +++
 tcg/tcg.h |  2 ++
 5 files changed, 58 insertions(+), 1 deletion(-)

diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index e72aeba682..fb2fe0fa1f 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -1705,6 +1705,7 @@ static TBStatistics *tb_get_stats(tb_page_addr_t phys_pc, 
target_ulong pc,
  * then just make the new TB point to the older TBStatistic
  */
 g_free(new_stats);
+((TBStatistics *) existing_stats)->tb = current_tb;
 return existing_stats;
 } else {
 return new_stats;
@@ -1785,13 +1786,18 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
  */
 if (tb_stats_collection_enabled()) {
 tb->tb_stats = tb_get_stats(phys_pc, pc, cs_base, flags, tb);
+uint32_t flag = get_default_tbstats_flag();
 
 if (qemu_log_in_addr_range(tb->pc)) {
-uint32_t flag = get_default_tbstats_flag();
 if (flag & TB_EXEC_STATS) {
 tb->tb_stats->stats_enabled |= TB_EXEC_STATS;
 }
 }
+
+if (flag & TB_JIT_STATS) {
+tb->tb_stats->stats_enabled |= TB_JIT_STATS;
+atomic_inc(>tb_stats->translations.total);
+}
 } else {
 tb->tb_stats = NULL;
 }
@@ -1869,6 +1875,10 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
 atomic_set(>search_out_len, prof->search_out_len + search_size);
 #endif
 
+if (tb_stats_enabled(tb, TB_JIT_STATS)) {
+atomic_add(>tb_stats->code.out_len, gen_code_size);
+}
+
 #ifdef DEBUG_DISAS
 if (qemu_loglevel_mask(CPU_LOG_TB_OUT_ASM) &&
 qemu_log_in_addr_range(tb->pc)) {
@@ -1926,6 +1936,9 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
 phys_page2 = -1;
 if ((pc & TARGET_PAGE_MASK) != virt_page2) {
 phys_page2 = get_page_addr_code(env, virt_page2);
+if (tb_stats_enabled(tb, TB_JIT_STATS)) {
+atomic_inc(>tb_stats->translations.spanning);
+}
 }
 /*
  * No explicit memory barrier is required -- tb_link_page() makes the
diff --git a/accel/tcg/translator.c b/accel/tcg/translator.c
index ec6bd829a0..9b2e248b09 100644
--- a/accel/tcg/translator.c
+++ b/accel/tcg/translator.c
@@ -116,6 +116,10 @@ void translator_loop(const TranslatorOps *ops, 
DisasContextBase *db,
 db->tb->size = db->pc_next - db->pc_first;
 db->tb->icount = db->num_insns;
 
+if (tb_stats_enabled(tb, TB_JIT_STATS)) {
+atomic_add(>tb->tb_stats->code.num_guest_inst, db->num_insns);
+}
+
 #ifdef DEBUG_DISAS
 if (qemu_loglevel_mask(CPU_LOG_TB_IN_ASM)
 && qemu_log_in_addr_range(db->pc_first)) {
diff --git a/include/exec/tb-stats.h b/include/exec/tb-stats.h
index c4a8715400..b68edd5d24 100644
--- a/include/exec/tb-stats.h
+++ b/include/exec/tb-stats.h
@@ -58,6 +58,20 @@ struct TBStatistics {
 unsigned long atomic;
 } executions;
 
+struct {
+unsigned num_guest_inst;
+unsigned num_tcg_ops;
+unsigned num_tcg_ops_opt;
+unsigned spills;
+unsigned out_len;
+} code;
+
+struct {
+unsigned long total;
+unsigned long uncached;
+unsigned long spanning;
+} translations;
+
 /* current TB linked to this TBStatistics */
 TranslationBlock *tb;
 };
@@ -71,6 +85,7 @@ enum TBStatsStatus { TB_STATS_RUNNING, TB_STATS_PAUSED, 
TB_STATS_STOPPED };
 
 #define TB_NOTHING(1 << 0)
 #define TB_EXEC_STATS (1 << 1)
+#define TB_JIT_STATS  (1 << 2)
 
 extern int tcg_collect_tb_stats;
 extern uint32_t default_tbstats_flag;
diff --git a/tcg/tcg.c b/tcg/tcg.c
index 0458eaec57..ae3e7a2217 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -3125,6 +3125,11 @@ static void temp_sync(TCGContext *s, TCGTemp *ts, 
TCGRegSet allocated_regs,
 case TEMP_VAL_REG:
 tcg_out_st(s, ts->type, ts->reg,
ts->mem_base->reg, ts->mem_offset);
+
+/* Count number of spills */
+if (tb_stats_enabled(s->current_tb, TB_JIT_STATS)) {
+atomic_inc(>current_tb->tb_stats->code.spills);
+}
 break;
 
 case TEMP_VAL_MEM:
@@ -3996,6 +4001,8 @@ int tcg_gen_code(TCGContext *s, TranslationBlock *tb)
 int i, num_insns;
 TCGOp *op;
 
+s->current_tb = tb;
+
 #ifdef CONFIG_PROFILER
 {
 int n = 0;
@@ -4027,6 +4034,14 @@ int tcg_gen_code(TCGContext 

[Qemu-devel] [PATCH v8 04/11] accel: replacing part of CONFIG_PROFILER with TBStats

[vandersonmr]

We add some of the statistics collected in the TCGProfiler
into the TBStats, having the statistics not only for the whole
emulation but for each TB. Then, we removed these stats
from TCGProfiler and reconstruct the information for the
"info jit" using the sum of all TBStats statistics.

The goal is to have one unique and better way of collecting
emulation statistics. Moreover, checking dynamiclly if the
profiling is enabled showed to have an insignificant impact
on the performance:
https://wiki.qemu.org/Internships/ProjectIdeas/TCGCodeQuality#Overheads.

Reviewed-by: Alex Bennée 
Signed-off-by: Vanderson M. do Rosario 
---
 accel/tcg/tb-stats.c  | 96 +++
 accel/tcg/translate-all.c |  8 +---
 include/exec/tb-stats.h   | 11 +
 tcg/tcg.c | 93 -
 tcg/tcg.h | 10 
 5 files changed, 119 insertions(+), 99 deletions(-)

diff --git a/accel/tcg/tb-stats.c b/accel/tcg/tb-stats.c
index 1db81d83e7..176da60e13 100644
--- a/accel/tcg/tb-stats.c
+++ b/accel/tcg/tb-stats.c
@@ -25,9 +25,105 @@
 #include "qemu/osdep.h"
 
 #include "disas/disas.h"
+#include "exec/exec-all.h"
+#include "tcg.h"
+
+#include "qemu/qemu-print.h"
 
 #include "exec/tb-stats.h"
 
+struct jit_profile_info {
+uint64_t translations;
+uint64_t aborted;
+uint64_t ops;
+unsigned ops_max;
+uint64_t del_ops;
+uint64_t temps;
+unsigned temps_max;
+uint64_t host;
+uint64_t guest;
+uint64_t search_data;
+};
+
+/* accumulate the statistics from all TBs */
+static void collect_jit_profile_info(void *p, uint32_t hash, void *userp)
+{
+struct jit_profile_info *jpi = userp;
+TBStatistics *tbs = p;
+
+jpi->translations += tbs->translations.total;
+jpi->ops += tbs->code.num_tcg_ops;
+if (stat_per_translation(tbs, code.num_tcg_ops) > jpi->ops_max) {
+jpi->ops_max = stat_per_translation(tbs, code.num_tcg_ops);
+}
+jpi->del_ops += tbs->code.deleted_ops;
+jpi->temps += tbs->code.temps;
+if (stat_per_translation(tbs, code.temps) > jpi->temps_max) {
+jpi->temps_max = stat_per_translation(tbs, code.temps);
+}
+jpi->host += tbs->code.out_len;
+jpi->guest += tbs->code.in_len;
+jpi->search_data += tbs->code.search_out_len;
+}
+
+/* dump JIT statisticis using TCGProfile and TBStats */
+void dump_jit_profile_info(TCGProfile *s)
+{
+if (!tb_stats_collection_enabled()) {
+return;
+}
+
+struct jit_profile_info *jpi = g_new0(struct jit_profile_info, 1);
+
+qht_iter(_ctx.tb_stats, collect_jit_profile_info, jpi);
+
+if (jpi->translations) {
+qemu_printf("translated TBs  %" PRId64 "\n", jpi->translations);
+qemu_printf("avg ops/TB  %0.1f max=%d\n",
+jpi->ops / (double) jpi->translations, jpi->ops_max);
+qemu_printf("deleted ops/TB  %0.2f\n",
+jpi->del_ops / (double) jpi->translations);
+qemu_printf("avg temps/TB%0.2f max=%d\n",
+jpi->temps / (double) jpi->translations, jpi->temps_max);
+qemu_printf("avg host code/TB%0.1f\n",
+jpi->host / (double) jpi->translations);
+qemu_printf("avg search data/TB  %0.1f\n",
+jpi->search_data / (double) jpi->translations);
+
+if (s) {
+int64_t tot = s->interm_time + s->code_time;
+qemu_printf("JIT cycles  %" PRId64 " (%0.3f s at 2.4 
GHz)\n",
+tot, tot / 2.4e9);
+qemu_printf("cycles/op   %0.1f\n",
+jpi->ops ? (double)tot / jpi->ops : 0);
+qemu_printf("cycles/in byte  %0.1f\n",
+jpi->guest ? (double)tot / jpi->guest : 0);
+qemu_printf("cycles/out byte %0.1f\n",
+jpi->host ? (double)tot / jpi->host : 0);
+qemu_printf("cycles/search byte %0.1f\n",
+jpi->search_data ? (double)tot / jpi->search_data : 0);
+if (tot == 0) {
+tot = 1;
+}
+qemu_printf("  gen_interm time   %0.1f%%\n",
+(double)s->interm_time / tot * 100.0);
+qemu_printf("  gen_code time %0.1f%%\n",
+(double)s->code_time / tot * 100.0);
+qemu_printf("optim./code time%0.1f%%\n",
+(double)s->opt_time / (s->code_time ? s->code_time : 1)
+* 100.0);
+qemu_printf("liveness/code time  %0.1f%%\n",
+(double)s->la_time / (s->code_time ? s->code_time : 1) * 
100.0);
+qemu_printf("cpu_restore count   %" PRId64 "\n",
+s->restore_count);
+qemu_printf("  avg cycles%0.1f\n",
+s->restore_count ? (double)s->restore_time / 
s->restore_count : 0);
+}
+}
+g_free(jpi);
+}
+
+
 void 

[Qemu-devel] [PATCH v8 01/11] accel: introducing TBStatistics structure

[vandersonmr]

To store statistics for each TB, we created a TBStatistics structure
which is linked with the TBs. TBStatistics can stay alive after
tb_flush and be relinked to a regenerated TB. So the statistics can
be accumulated even through flushes.

The goal is to have all present and future qemu/tcg statistics and
meta-data stored in this new structure.

Reviewed-by: Alex Bennée 
Signed-off-by: Vanderson M. do Rosario 
---
 accel/tcg/Makefile.objs   |  2 +-
 accel/tcg/tb-stats.c  | 63 
 accel/tcg/translate-all.c | 57 +
 include/exec/exec-all.h   | 15 +++--
 include/exec/tb-context.h | 12 +++
 include/exec/tb-hash.h|  7 
 include/exec/tb-stats.h   | 67 +++
 util/log.c|  2 ++
 8 files changed, 213 insertions(+), 12 deletions(-)
 create mode 100644 accel/tcg/tb-stats.c
 create mode 100644 include/exec/tb-stats.h

diff --git a/accel/tcg/Makefile.objs b/accel/tcg/Makefile.objs
index d381a02f34..49ffe81b5d 100644
--- a/accel/tcg/Makefile.objs
+++ b/accel/tcg/Makefile.objs
@@ -2,7 +2,7 @@ obj-$(CONFIG_SOFTMMU) += tcg-all.o
 obj-$(CONFIG_SOFTMMU) += cputlb.o
 obj-y += tcg-runtime.o tcg-runtime-gvec.o
 obj-y += cpu-exec.o cpu-exec-common.o translate-all.o
-obj-y += translator.o
+obj-y += translator.o tb-stats.o
 
 obj-$(CONFIG_USER_ONLY) += user-exec.o
 obj-$(call lnot,$(CONFIG_SOFTMMU)) += user-exec-stub.o
diff --git a/accel/tcg/tb-stats.c b/accel/tcg/tb-stats.c
new file mode 100644
index 00..948b107e68
--- /dev/null
+++ b/accel/tcg/tb-stats.c
@@ -0,0 +1,63 @@
+/*
+ * QEMU System Emulator, Code Quality Monitor System
+ *
+ * Copyright (c) 2019 Vanderson M. do Rosario 
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to 
deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu/osdep.h"
+
+#include "disas/disas.h"
+
+#include "exec/tb-stats.h"
+
+void init_tb_stats_htable_if_not(void)
+{
+if (tb_stats_collection_enabled() && !tb_ctx.tb_stats.map) {
+qht_init(_ctx.tb_stats, tb_stats_cmp,
+CODE_GEN_HTABLE_SIZE, QHT_MODE_AUTO_RESIZE);
+}
+}
+
+void enable_collect_tb_stats(void)
+{
+init_tb_stats_htable_if_not();
+tcg_collect_tb_stats = TB_STATS_RUNNING;
+}
+
+void disable_collect_tb_stats(void)
+{
+tcg_collect_tb_stats = TB_STATS_PAUSED;
+}
+
+void pause_collect_tb_stats(void)
+{
+tcg_collect_tb_stats = TB_STATS_STOPPED;
+}
+
+bool tb_stats_collection_enabled(void)
+{
+return tcg_collect_tb_stats == TB_STATS_RUNNING;
+}
+
+bool tb_stats_collection_paused(void)
+{
+return tcg_collect_tb_stats == TB_STATS_PAUSED;
+}
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index 5d1e08b169..b7bccacd3b 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -1118,6 +1118,23 @@ static inline void code_gen_alloc(size_t tb_size)
 }
 }
 
+/*
+ * This is the more or less the same compare as tb_cmp(), but the
+ * data persists over tb_flush. We also aggregate the various
+ * variations of cflags under one record and ignore the details of
+ * page overlap (although we can count it).
+ */
+bool tb_stats_cmp(const void *ap, const void *bp)
+{
+const TBStatistics *a = ap;
+const TBStatistics *b = bp;
+
+return a->phys_pc == b->phys_pc &&
+a->pc == b->pc &&
+a->cs_base == b->cs_base &&
+a->flags == b->flags;
+}
+
 static bool tb_cmp(const void *ap, const void *bp)
 {
 const TranslationBlock *a = ap;
@@ -1137,6 +1154,7 @@ static void tb_htable_init(void)
 unsigned int mode = QHT_MODE_AUTO_RESIZE;
 
 qht_init(_ctx.htable, tb_cmp, CODE_GEN_HTABLE_SIZE, mode);
+init_tb_stats_htable_if_not();
 }
 
 /* Must be called before using the QEMU cpus. 'tb_size' is the size
@@ -1666,6 +1684,34 @@ tb_link_page(TranslationBlock *tb, tb_page_addr_t 
phys_pc,
 return tb;
 }
 
+static TBStatistics *tb_get_stats(tb_page_addr_t phys_pc, target_ulong pc,
+  

[Qemu-devel] [PATCH v8 00/11] Measure Tiny Code Generation Quality

[vandersonmr]

This patch is part of Google Summer of Code (GSoC) 2019.
More about the project can be found in:
https://wiki.qemu.org/Internships/ProjectIdeas/TCGCodeQuality

The goal of this patch is to add infrastructure to collect
execution and JIT statistics during the emulation with accel/TCG.
The statistics are stored in TBStatistic structures (TBStats)
with each TB having its respective TBStats.

We added -d tb_stats and HMP tb_stats commands to allow the control
of this statistics collection. And info tb, tbs, and coverset commands
were also added to allow dumping and exploring all this information
while emulating.

Collecting these statistics and information is useful to understand
qemu performance and to help to add the support for traces to QEMU.

v8:
 - fixing missing commit from v7
v7:
 - rebase
 - adding license to new files
 - applying comments from v6
v6:
 - applying comments from V5.
 - change info tbs to info tb-list
 - fix crash when dumping tb's targets
 - fix "liveness/code time" calculation
v5:
 - full replacement of CONFIG_PROFILER
 - several fixes
 - adds "info cfg"
 - adds TB's targets to dump

vandersonmr (11):
  accel: introducing TBStatistics structure
  accel: collecting TB execution count
  accel: collecting JIT statistics
  accel: replacing part of CONFIG_PROFILER with TBStats
  accel: adding TB_JIT_TIME and full replacing CONFIG_PROFILER
  Adding -d tb_stats to control TBStatistics collection:
  monitor: adding tb_stats hmp command
  Adding tb_stats [start|pause|stop|filter] command to hmp.
  Adding info [tb-list|tb|coverset] commands to HMP.
  monitor: adding new info cfg command
  linux-user: dumping hot TBs at the end of the execution

 accel/tcg/Makefile.objs   |   2 +-
 accel/tcg/cpu-exec.c  |   4 +
 accel/tcg/tb-stats.c  | 894 ++
 accel/tcg/tcg-runtime.c   |   7 +
 accel/tcg/tcg-runtime.h   |   2 +
 accel/tcg/translate-all.c | 132 +++--
 accel/tcg/translator.c|   5 +
 configure |   3 -
 cpus.c|  14 +-
 disas.c   |  31 +-
 hmp-commands-info.hx  |  31 ++
 hmp-commands.hx   |  17 +
 include/exec/exec-all.h   |  15 +-
 include/exec/gen-icount.h |  10 +
 include/exec/tb-context.h |  12 +
 include/exec/tb-hash.h|   7 +
 include/exec/tb-stats-flags.h |  43 ++
 include/exec/tb-stats.h   | 170 +++
 include/qemu/log-for-trace.h  |   4 +
 include/qemu/log.h|   3 +
 include/qemu/timer.h  |   5 +-
 linux-user/exit.c |   4 +
 monitor/misc.c| 171 ++-
 tcg/tcg.c | 230 +++--
 tcg/tcg.h |  22 +-
 util/log.c|  90 +++-
 vl.c  |   8 +-
 27 files changed, 1652 insertions(+), 284 deletions(-)
 create mode 100644 accel/tcg/tb-stats.c
 create mode 100644 include/exec/tb-stats-flags.h
 create mode 100644 include/exec/tb-stats.h

-- 
2.22.0

Re: [Qemu-devel] [PATCH v6 1/6] iotests: allow Valgrind checking all QEMU processes

[John Snow]

On 8/29/19 6:50 AM, Andrey Shinkevich wrote:
> 
> 
> On 29/08/2019 03:30, Eric Blake wrote:
>> On 8/28/19 5:58 PM, John Snow wrote:
>>
 +++ b/tests/qemu-iotests/common.rc
 @@ -60,61 +60,132 @@ if ! . ./common.config
   exit 1
   fi
   
 +# Unset the variables to turn Valgrind off for specific processes, e.g.
>>
>> That's not unsetting, that's setting to the empty string.
>>
> 
> Thanks Eric, I will make the correction of the comment. Any string other 
> than "y", including the empty one, fits.
> 
 +# $ VALGRIND_QEMU_IO= ./check -qcow2 -valgrind 015
 +
 +: ${VALGRIND_QEMU_VM='y'}
 +: ${VALGRIND_QEMU_IMG='y'}
 +: ${VALGRIND_QEMU_IO='y'}
 +: ${VALGRIND_QEMU_NBD='y'}
 +: ${VALGRIND_QEMU_VXHS='y'}
 +
>>>
> 
> I am going to make the change:
> 
> : ${VALGRIND_QEMU_VM=$VALGRIND_QEMU}
> : ${VALGRIND_QEMU_IMG=$VALGRIND_QEMU}
> : ${VALGRIND_QEMU_IO=$VALGRIND_QEMU}
> : ${VALGRIND_QEMU_NBD=$VALGRIND_QEMU}
> : ${VALGRIND_QEMU_VXHS=$VALGRIND_QEMU}
> 
> and get rid of the local VALGRIND_ON="${VALGRIND_QEMU}"
> 
> so that the code will be optimized.
> 

Seems good!

>>> I have to admit to you that I'm not familiar with this trick. I'm
>>> looking it up and I see := documented, but not = alone.
>>
>> It's been a repeated complaint to the bash developer that the manual is
>> doing a disservice to its users by not documenting ${var=val} in an
>> easily searchable form.  It IS documented, but only by virtue of
>> ${var:=val} occurring under a section header that states:
>>
>> When not performing substring expansion,  using  the  forms
>> documented
>> below  (e.g.,  :-),  bash  tests for a parameter that is unset or
>> null.
>> Omitting the colon results in a test  only  for  a  parameter
>> that  is
>> unset.
>>
>> So the choice is whether you want to special case a variable set to an
>> empty string the same as an unset variable, or the same as a variable
>> with a non-empty value.
>>
> 
> Thank you all for your reviews and comments. The purpose why I omitted 
> the colon is to allow a user writing the shorter command syntax like
> $ VALGRIND_QEMU_IO= ./check -valgrind 
> rather than
> $ VALGRIND_QEMU_IO=" 'no' or 'off' or else anything other than 'y' " 
> ./check -valgrind 
> so, no need to strike the Shift key twice and guess at what else is 
> acceptable to type )))
> 
> The variable default value 'y' looks good to me to implement the new 
> functionality that is compatible with the existing one when we just set 
> the '-valgrind' switch. The general idea behind using the Valgrind is to 
> make a careful search for memory issues. Once found, a user can tune the 
> particular test with extra variables to save their development/testing 
> time as John suggested. Also, no need to specify all the five long name 
> variables each time a user writes the command if default values aren't set.
> 
> I am flexible to make a change that is good for all. So, what solution 
> will we come to?
> 

I don't actually really have a preference here; it's development and
testing infrastructure. As long as it is POSIX portable, I'm happy. If
we goof it up, we'll find out eventually. If we don't, well. Just more
evidence we need more non-Linux contributors.

--js

Re: [Qemu-devel] [Qemu-block] [PATCH] qemu-doc: Do not hard-code the name of the QEMU binary

[John Snow]

On 8/29/19 1:41 AM, Thomas Huth wrote:
> On 28/08/2019 21.18, John Snow wrote:
>>
>>
>> On 8/28/19 5:34 AM, Thomas Huth wrote:
>>> In our documentation, we use a mix of "$QEMU", "qemu-system-i386" and
>>> "qemu-system-x86_64" when we give examples to the users how to run
>>> QEMU. Some more consistency would be good here. Also some distributions
>>> use different names for the QEMU binary (e.g. "qemu-kvm" in RHEL), so
>>> providing more flexibility here would also be good. Thus let's define
>>> some variables for the names of the QEMU command and use those in the
>>> documentation instead: @value{qemu_system} for generic examples, and
>>> @value{qemu_system_x86} for examples that only work with the x86
>>> binaries.
>>>
>>> Signed-off-by: Thomas Huth 
>>
>> Makes sense to me, but do we want a definitions.texi or similar that can
>> be used globally (and is easy to find and edit by e.g. distro
>> packagers), or is it better to re-define them per-each file as you've done?
> 
> Hmm, as long as it's just one or two variables, it seems a little bit
> excessive to me, but if we'd have more config knobs, that would
> certainly be the right way to go ... but currently we do not have any
> more variables, do we?
> 

Not that I'm aware of. We might find more as we embark on the
ReSTification of our docs.

I only bring it up because it might not be clear in which documents and
in how many places this definition needs to be changed by a package
maintainer; if our end goal is one big unified manual then I think we
need a central configuration for it, too. Maybe that only shows up for
the Sphinx manual.

(Maybe a yaml file that conf.py can consume and uses to generate ReST
definitions that can be used throughout the rest of the docs would be an
appropriate thing to do.)

I won't insist, because creating new infrastructure for texi docs seems
lateral. Your patch doesn't make anything I'm pointing out worse than it
already was, though, so:

Reviewed-by: John Snow 

--js

Re: [Qemu-devel] [PATCH 0/3] Acceptance test x86_cpu_model_versions: fixes and minor improvements

[Eduardo Habkost]

On Wed, Aug 28, 2019 at 03:36:25PM -0400, Cleber Rosa wrote:
> This corrects some of the messages given on failures (aligning them to
> the test code), and splits a larger test into smaller and more
> manageable pieces.

Thanks!

I'm queueing this on python-next.

-- 
Eduardo

Re: [Qemu-devel] [PATCH v2 2/8] exec: Factor out core logic of check_watchpoint()

[Philippe Mathieu-Daudé]

Hi Richard, David,

On 8/29/19 1:16 AM, Richard Henderson wrote:
> From: David Hildenbrand 
> 
> We want to perform the same checks in probe_write() to trigger a cpu
> exit before doing any modifications. We'll have to pass a PC.
> 
> Signed-off-by: David Hildenbrand 
> Reviewed-by: Richard Henderson 
> Message-Id: <20190823100741.9621-9-da...@redhat.com>
> [rth: Use vaddr for len, like other watchpoint functions;
> Move user-only stub to static inline.]
> Signed-off-by: Richard Henderson 
> ---
>  include/hw/core/cpu.h |  7 +++
>  exec.c| 26 ++
>  2 files changed, 25 insertions(+), 8 deletions(-)
> 
> diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
> index 6de688059d..7bd8bed5b2 100644
> --- a/include/hw/core/cpu.h
> +++ b/include/hw/core/cpu.h
> @@ -1091,6 +1091,11 @@ static inline void 
> cpu_watchpoint_remove_by_ref(CPUState *cpu,
>  static inline void cpu_watchpoint_remove_all(CPUState *cpu, int mask)
>  {
>  }
> +
> +static inline void cpu_check_watchpoint(CPUState *cpu, vaddr addr, vaddr len,
> +MemTxAttrs atr, int fl, uintptr_t ra)
> +{
> +}
>  #else
>  int cpu_watchpoint_insert(CPUState *cpu, vaddr addr, vaddr len,
>int flags, CPUWatchpoint **watchpoint);
> @@ -1098,6 +1103,8 @@ int cpu_watchpoint_remove(CPUState *cpu, vaddr addr,
>vaddr len, int flags);
>  void cpu_watchpoint_remove_by_ref(CPUState *cpu, CPUWatchpoint *watchpoint);
>  void cpu_watchpoint_remove_all(CPUState *cpu, int mask);
> +void cpu_check_watchpoint(CPUState *cpu, vaddr addr, vaddr len,
> +  MemTxAttrs attrs, int flags, uintptr_t ra);
>  #endif
>  
>  /**
> diff --git a/exec.c b/exec.c
> index 31fb75901f..cb6f5763dc 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -2789,11 +2789,10 @@ static const MemoryRegionOps notdirty_mem_ops = {
>  };
>  
>  /* Generate a debug exception if a watchpoint has been hit.  */
> -static void check_watchpoint(int offset, int len, MemTxAttrs attrs, int 
> flags)
> +void cpu_check_watchpoint(CPUState *cpu, vaddr addr, vaddr len,
> +  MemTxAttrs attrs, int flags, uintptr_t ra)
>  {
> -CPUState *cpu = current_cpu;
>  CPUClass *cc = CPU_GET_CLASS(cpu);
> -target_ulong vaddr;
>  CPUWatchpoint *wp;
>  
>  assert(tcg_enabled());
> @@ -2804,17 +2803,17 @@ static void check_watchpoint(int offset, int len, 
> MemTxAttrs attrs, int flags)
>  cpu_interrupt(cpu, CPU_INTERRUPT_DEBUG);
>  return;
>  }
> -vaddr = (cpu->mem_io_vaddr & TARGET_PAGE_MASK) + offset;
> -vaddr = cc->adjust_watchpoint_address(cpu, vaddr, len);
> +
> +addr = cc->adjust_watchpoint_address(cpu, addr, len);
>  QTAILQ_FOREACH(wp, >watchpoints, entry) {
> -if (cpu_watchpoint_address_matches(wp, vaddr, len)
> +if (cpu_watchpoint_address_matches(wp, addr, len)
>  && (wp->flags & flags)) {
>  if (flags == BP_MEM_READ) {
>  wp->flags |= BP_WATCHPOINT_HIT_READ;
>  } else {
>  wp->flags |= BP_WATCHPOINT_HIT_WRITE;
>  }
> -wp->hitaddr = vaddr;
> +wp->hitaddr = MAX(addr, wp->vaddr);

When is addr > wp->vaddr?

>  wp->hitattrs = attrs;
>  if (!cpu->watchpoint_hit) {
>  if (wp->flags & BP_CPU &&
> @@ -2829,11 +2828,14 @@ static void check_watchpoint(int offset, int len, 
> MemTxAttrs attrs, int flags)
>  if (wp->flags & BP_STOP_BEFORE_ACCESS) {
>  cpu->exception_index = EXCP_DEBUG;
>  mmap_unlock();
> -cpu_loop_exit(cpu);
> +cpu_loop_exit_restore(cpu, ra);
>  } else {
>  /* Force execution of one insn next time.  */
>  cpu->cflags_next_tb = 1 | curr_cflags();
>  mmap_unlock();
> +if (ra) {
> +cpu_restore_state(cpu, ra, true);
> +}
>  cpu_loop_exit_noexc(cpu);
>  }
>  }
> @@ -2843,6 +2845,14 @@ static void check_watchpoint(int offset, int len, 
> MemTxAttrs attrs, int flags)
>  }
>  }
>  
> +static void check_watchpoint(int offset, int len, MemTxAttrs attrs, int 
> flags)
> +{
> +CPUState *cpu = current_cpu;
> +vaddr addr = (cpu->mem_io_vaddr & TARGET_PAGE_MASK) + offset;
> +
> +cpu_check_watchpoint(cpu, addr, len, attrs, flags, 0);
> +}
> +
>  /* Watchpoint access routines.  Watchpoints are inserted using TLB tricks,
> so these check for a hit then pass through to the normal out-of-line
> phys routines.  */
> 

Re: [Qemu-devel] [PATCH v2 4/8] exec: Factor out cpu_watchpoint_address_matches

[Philippe Mathieu-Daudé]

On 8/29/19 1:16 AM, Richard Henderson wrote:
> We want to move the check for watchpoints from
> memory_region_section_get_iotlb to tlb_set_page_with_attrs.
> Isolate the loop over watchpoints to an exported function.
> 
> Rename the existing cpu_watchpoint_address_matches to
> watchpoint_address_matches, since it doesn't actually
> have a cpu argument.
> 
> Reviewed-by: David Hildenbrand 
> Signed-off-by: Richard Henderson 
> ---
>  include/hw/core/cpu.h |  7 +++
>  exec.c| 45 ---
>  2 files changed, 36 insertions(+), 16 deletions(-)
> 
> diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
> index 7bd8bed5b2..c7cda65c66 100644
> --- a/include/hw/core/cpu.h
> +++ b/include/hw/core/cpu.h
> @@ -1096,6 +1096,12 @@ static inline void cpu_check_watchpoint(CPUState *cpu, 
> vaddr addr, vaddr len,
>  MemTxAttrs atr, int fl, uintptr_t ra)
>  {
>  }
> +
> +static inline int cpu_watchpoint_address_matches(CPUState *cpu,
> + vaddr addr, vaddr len)
> +{
> +return 0;
> +}
>  #else
>  int cpu_watchpoint_insert(CPUState *cpu, vaddr addr, vaddr len,
>int flags, CPUWatchpoint **watchpoint);
> @@ -1105,6 +,7 @@ void cpu_watchpoint_remove_by_ref(CPUState *cpu, 
> CPUWatchpoint *watchpoint);
>  void cpu_watchpoint_remove_all(CPUState *cpu, int mask);
>  void cpu_check_watchpoint(CPUState *cpu, vaddr addr, vaddr len,
>MemTxAttrs attrs, int flags, uintptr_t ra);
> +int cpu_watchpoint_address_matches(CPUState *cpu, vaddr addr, vaddr len);
>  #endif
>  
>  /**
> diff --git a/exec.c b/exec.c
> index cb6f5763dc..8575ce51ad 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -1138,9 +1138,8 @@ void cpu_watchpoint_remove_all(CPUState *cpu, int mask)
>   * partially or completely with the address range covered by the
>   * access).
>   */
> -static inline bool cpu_watchpoint_address_matches(CPUWatchpoint *wp,
> -  vaddr addr,
> -  vaddr len)
> +static inline bool watchpoint_address_matches(CPUWatchpoint *wp,
> +  vaddr addr, vaddr len)
>  {
>  /* We know the lengths are non-zero, but a little caution is
>   * required to avoid errors in the case where the range ends
> @@ -1152,6 +1151,20 @@ static inline bool 
> cpu_watchpoint_address_matches(CPUWatchpoint *wp,
>  
>  return !(addr > wpend || wp->vaddr > addrend);
>  }
> +
> +/* Return flags for watchpoints that match addr + prot.  */
> +int cpu_watchpoint_address_matches(CPUState *cpu, vaddr addr, vaddr len)
> +{
> +CPUWatchpoint *wp;
> +int ret = 0;
> +
> +QTAILQ_FOREACH(wp, >watchpoints, entry) {
> +if (watchpoint_address_matches(wp, addr, TARGET_PAGE_SIZE)) {
> +ret |= wp->flags;
> +}
> +}
> +return ret;
> +}
>  #endif /* !CONFIG_USER_ONLY */
>  
>  /* Add a breakpoint.  */
> @@ -1459,7 +1472,7 @@ hwaddr memory_region_section_get_iotlb(CPUState *cpu,
> target_ulong *address)
>  {
>  hwaddr iotlb;
> -CPUWatchpoint *wp;
> +int flags, match;
>  
>  if (memory_region_is_ram(section->mr)) {
>  /* Normal RAM.  */
> @@ -1477,17 +1490,17 @@ hwaddr memory_region_section_get_iotlb(CPUState *cpu,
>  iotlb += xlat;
>  }
>  
> -/* Make accesses to pages with watchpoints go via the
> -   watchpoint trap routines.  */
> -QTAILQ_FOREACH(wp, >watchpoints, entry) {
> -if (cpu_watchpoint_address_matches(wp, vaddr, TARGET_PAGE_SIZE)) {
> -/* Avoid trapping reads of pages with a write breakpoint. */
> -if ((prot & PAGE_WRITE) || (wp->flags & BP_MEM_READ)) {
> -iotlb = PHYS_SECTION_WATCH + paddr;
> -*address |= TLB_MMIO;
> -break;
> -}
> -}
> +/* Avoid trapping reads of pages with a write breakpoint. */
> +match = (prot & PAGE_READ ? BP_MEM_READ : 0)
> +  | (prot & PAGE_WRITE ? BP_MEM_WRITE : 0);

Isn't it cheaper to do here:

   if (!match) {
   return iotlb;
   }

or

   if (match) {

> +flags = cpu_watchpoint_address_matches(cpu, vaddr, TARGET_PAGE_SIZE);
> +if (flags & match) {
> +/*
> + * Make accesses to pages with watchpoints go via the
> + * watchpoint trap routines.
> + */
> +iotlb = PHYS_SECTION_WATCH + paddr;
> +*address |= TLB_MMIO;
>  }

}

Regardless:
Reviewed-by: Philippe Mathieu-Daudé 

>  return iotlb;
> @@ -2806,7 +2819,7 @@ void cpu_check_watchpoint(CPUState *cpu, vaddr addr, 
> vaddr len,
>  
>  addr = cc->adjust_watchpoint_address(cpu, addr, len);
>  QTAILQ_FOREACH(wp, >watchpoints, entry) {
> -if (cpu_watchpoint_address_matches(wp, addr, len)
> +if 

Re: [Qemu-devel] [Qemu-block] [PATCH 2/2] qapi: deprecate implicit filters

[John Snow]

On 8/29/19 11:59 AM, Christophe de Dinechin wrote:
> 
> John Snow writes:
> [...]
>>
>> This might be OK to do right away, though.
>>
>> I asked Markus this not too long ago; do we want to amend the QAPI
>> schema specification to allow commands to return with "Warning" strings,
>> or "Deprecated" stings to allow in-band deprecation notices for cases
>> like these?
>>
>> example:
>>
>> { "return": {},
>>   "deprecated": True,
>>   "warning": "Omitting filter-node-name parameter is deprecated, it will
>> be required in the future"
>> }
>>
>> There's no "error" key, so this should be recognized as success by
>> compatible clients, but they'll definitely see the extra information.
>>
>> Part of my motivation is to facilitate a more aggressive deprecation of
>> legacy features by ensuring that we are able to rigorously notify users
>> through any means that they need to adjust their scripts.
> 
> I like this approach even if there is no consumer today. It does not
> hurt, and it is indeed a motivation to develop consumers that care.
> 
> I personally find this much easier to swallow than any kind of crash on
> deprecation, which already at the BoF seemed like a really big hammer to
> kill a fly.
> 
> CC'ing Andrea as well, because we discussed recently about how to deal
> with error checking in general, and if a new error checking framework is
> being put in place, adding deprecation to the thinking could be a good
> idea.

The most convincing argument against deprecation notices like this is
not that they won't be consumed, but that they are difficult to plumb
through the C infrastructure.

Sadly, I think I have to agree there -- we can't even really model it
like hints, because these are cases where there was no /error/ but
instead a success -- but our error propagation doesn't work on those
terms generally and we'd need a rather extensive audit to allow warnings.

We could always fudge it with a kind of global warning log: clear the
log at the beginning of a QMP interaction and if the log is non-empty
when we return, amend the return with that information.

That's not really the nicest thing to do in a multi-process,
multi-threaded, multi-stacked application, though, so...

--js

Re: [Qemu-devel] [PATCH v2 7/8] cputlb: Handle watchpoints via TLB_WATCHPOINT

[Philippe Mathieu-Daudé]

On 8/29/19 1:16 AM, Richard Henderson wrote:
> The raising of exceptions from check_watchpoint, buried inside
> of the I/O subsystem, is fundamentally broken.  We do not have
> the helper return address with which we can unwind guest state.
> 
> Replace PHYS_SECTION_WATCH and io_mem_watch with TLB_WATCHPOINT.
> Move the call to cpu_check_watchpoint into the cputlb helpers
> where we do have the helper return address.
> 
> This also allows us to handle watchpoints on RAM to bypass the
> full i/o access path.

Yay!

Reviewed-by: Philippe Mathieu-Daudé 

> Signed-off-by: Richard Henderson 
> ---
>  include/exec/cpu-all.h |   5 +-
>  accel/tcg/cputlb.c |  89 
>  exec.c | 114 +++--
>  3 files changed, 90 insertions(+), 118 deletions(-)
> 
> diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
> index 8d07ae23a5..d2d443c4f9 100644
> --- a/include/exec/cpu-all.h
> +++ b/include/exec/cpu-all.h
> @@ -329,11 +329,14 @@ CPUArchState *cpu_copy(CPUArchState *env);
>  #define TLB_NOTDIRTY(1 << (TARGET_PAGE_BITS - 2))
>  /* Set if TLB entry is an IO callback.  */
>  #define TLB_MMIO(1 << (TARGET_PAGE_BITS - 3))
> +/* Set if TLB entry contains a watchpoint.  */
> +#define TLB_WATCHPOINT  (1 << (TARGET_PAGE_BITS - 4))
>  
>  /* Use this mask to check interception with an alignment mask
>   * in a TCG backend.
>   */
> -#define TLB_FLAGS_MASK  (TLB_INVALID_MASK | TLB_NOTDIRTY | TLB_MMIO)
> +#define TLB_FLAGS_MASK \
> +(TLB_INVALID_MASK | TLB_NOTDIRTY | TLB_MMIO | TLB_WATCHPOINT)
>  
>  /**
>   * tlb_hit_page: return true if page aligned @addr is a hit against the
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index d0f8db33a2..9a9a626938 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -710,6 +710,7 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong 
> vaddr,
>  hwaddr iotlb, xlat, sz, paddr_page;
>  target_ulong vaddr_page;
>  int asidx = cpu_asidx_from_attrs(cpu, attrs);
> +int wp_flags;
>  
>  assert_cpu_is_self(cpu);
>  
> @@ -752,6 +753,8 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong 
> vaddr,
>  code_address = address;
>  iotlb = memory_region_section_get_iotlb(cpu, section, vaddr_page,
>  paddr_page, xlat, prot, 
> );
> +wp_flags = cpu_watchpoint_address_matches(cpu, vaddr_page,
> +  TARGET_PAGE_SIZE);
>  
>  index = tlb_index(env, mmu_idx, vaddr_page);
>  te = tlb_entry(env, mmu_idx, vaddr_page);
> @@ -805,6 +808,9 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong 
> vaddr,
>  tn.addend = addend - vaddr_page;
>  if (prot & PAGE_READ) {
>  tn.addr_read = address;
> +if (wp_flags & BP_MEM_READ) {
> +tn.addr_read |= TLB_WATCHPOINT;
> +}
>  } else {
>  tn.addr_read = -1;
>  }
> @@ -831,6 +837,9 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong 
> vaddr,
>  if (prot & PAGE_WRITE_INV) {
>  tn.addr_write |= TLB_INVALID_MASK;
>  }
> +if (wp_flags & BP_MEM_WRITE) {
> +tn.addr_write |= TLB_WATCHPOINT;
> +}
>  }
>  
>  copy_tlb_helper_locked(te, );
> @@ -1264,13 +1273,33 @@ load_helper(CPUArchState *env, target_ulong addr, 
> TCGMemOpIdx oi,
>  tlb_addr &= ~TLB_INVALID_MASK;
>  }
>  
> -/* Handle an IO access.  */
> +/* Handle anything that isn't just a straight memory access.  */
>  if (unlikely(tlb_addr & ~TARGET_PAGE_MASK)) {
> +CPUIOTLBEntry *iotlbentry;
> +
> +/* For anything that is unaligned, recurse through full_load.  */
>  if ((addr & (size - 1)) != 0) {
>  goto do_unaligned_access;
>  }
> -return io_readx(env, _tlb(env)->d[mmu_idx].iotlb[index],
> -mmu_idx, addr, retaddr, access_type, op);
> +
> +iotlbentry = _tlb(env)->d[mmu_idx].iotlb[index];
> +
> +/* Handle watchpoints.  */
> +if (unlikely(tlb_addr & TLB_WATCHPOINT)) {
> +/* On watchpoint hit, this will longjmp out.  */
> +cpu_check_watchpoint(env_cpu(env), addr, size,
> + iotlbentry->attrs, BP_MEM_READ, retaddr);
> +
> +/* The backing page may or may not require I/O.  */
> +tlb_addr &= ~TLB_WATCHPOINT;
> +if ((tlb_addr & ~TARGET_PAGE_MASK) == 0) {
> +goto do_aligned_access;
> +}
> +}
> +
> +/* Handle I/O access.  */
> +return io_readx(env, iotlbentry, mmu_idx, addr,
> +retaddr, access_type, op);
>  }
>  
>  /* Handle slow unaligned access (it spans two pages or IO).  */
> @@ -1297,6 +1326,7 @@ load_helper(CPUArchState *env, target_ulong addr, 
> TCGMemOpIdx oi,
>  return res & MAKE_64BIT_MASK(0, size * 8);
>  }
>  
> 

Re: [Qemu-devel] [PATCH v2 0/4] docs: add docs about use of automatic cleanup functions

[no-reply]

Patchew URL: https://patchew.org/QEMU/20190829160710.8792-1-berra...@redhat.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Subject: [Qemu-devel] [PATCH v2 0/4] docs: add docs about use of automatic 
cleanup functions
Message-id: 20190829160710.8792-1-berra...@redhat.com

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
d237cf1 docs: split the CODING_STYLE doc into distinct groups
e40d52d docs: document use of automatic cleanup functions in glib
3988984 docs: merge HACKING.rst contents into CODING_STYLE.rst
d1e2be4 docs: convert README, CODING_STYLE and HACKING to RST syntax

=== OUTPUT BEGIN ===
Must be run from the top-level dir. of a kernel tree
=== OUTPUT END ===

Test command exited with code: 2


The full log is available at
http://patchew.org/logs/20190829160710.8792-1-berra...@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

Re: [Qemu-devel] [PATCH v2 3/4] docs: document use of automatic cleanup functions in glib

[Alex Bennée]

Daniel P. Berrangé  writes:

> Document the use of g_autofree and g_autoptr in glib for automatic
> freeing of memory.
>
> Signed-off-by: Daniel P. Berrangé 

Reviewed-by: Alex Bennée 

> ---
>  CODING_STYLE.rst | 85 
>  1 file changed, 85 insertions(+)
>
> diff --git a/CODING_STYLE.rst b/CODING_STYLE.rst
> index 4501d87352..39397f0f6f 100644
> --- a/CODING_STYLE.rst
> +++ b/CODING_STYLE.rst
> @@ -441,6 +441,91 @@ In addition, QEMU assumes that the compiler does not use 
> the latitude
>  given in C99 and C11 to treat aspects of signed '<<' as undefined, as
>  documented in the GNU Compiler Collection manual starting at version 4.0.
>
> +Automatic memory deallocation
> +=
> +
> +QEMU has a mandatory dependency either the GCC or CLang compiler. As
> +such it has the freedom to make use of a C language extension for
> +automatically running a cleanup function when a stack variable goes
> +out of scope. This can be used to simplify function cleanup paths,
> +often allowing many goto jumps to be eliminated, through automatic
> +free'ing of memory.
> +
> +The GLib2 library provides a number of functions/macros for enabling
> +automatic cleanup:
> +
> +  ``_
> +
> +Most notably:
> +
> +* g_autofree - will invoke g_free() on the variable going out of scope
> +
> +* g_autoptr - for structs / objects, will invoke the cleanup func created
> +  by a previous use of G_DEFINE_AUTOPTR_CLEANUP_FUNC. This is
> +  supported for most GLib data types and GObjects
> +
> +For example, instead of
> +
> +.. code-block:: c
> +
> +int somefunc(void) {
> +int ret = -1;
> +char *foo = g_strdup_printf("foo%", "wibble");
> +GList *bar = .
> +
> +if (eek) {
> +   goto cleanup;
> +}
> +
> +ret = 0;
> +
> +  cleanup:
> +g_free(foo);
> +g_list_free(bar);
> +return ret;
> +}
> +
> +Using g_autofree/g_autoptr enables the code to be written as:
> +
> +.. code-block:: c
> +
> +int somefunc(void) {
> +g_autofree char *foo = g_strdup_printf("foo%", "wibble");
> +g_autoptr (GList) bar = .
> +
> +if (eek) {
> +   return -1;
> +}
> +
> +return 0;
> +}
> +
> +While this generally results in simpler, less leak-prone code, there
> +are still some caveats to beware of
> +
> +* Variables declared with g_auto* MUST always be initialized,
> +  otherwise the cleanup function will use uninitialized stack memory
> +
> +* If a variable declared with g_auto* holds a value which must
> +  live beyond the life of the function, that value must be saved
> +  and the original variable NULL'd out. This can be simpler using
> +  g_steal_pointer
> +
> +
> +.. code-block:: c
> +
> +char *somefunc(void) {
> +g_autofree char *foo = g_strdup_printf("foo%", "wibble");
> +g_autoptr (GList) bar = .
> +
> +if (eek) {
> +   return NULL;
> +}
> +
> +return g_steal_pointer();
> +}
> +
> +
>  Error handling and reporting
>  


--
Alex Bennée

Re: [Qemu-devel] [PATCH v6 0/4] 9p: Fix file ID collisions

[Greg Kurz]

On Thu, 22 Aug 2019 15:18:54 -0700 (PDT)
no-re...@patchew.org wrote:

> Patchew URL: 
> https://patchew.org/QEMU/cover.1566503584.git.qemu_...@crudebyte.com/
> 
> 
> 
> Hi,
> 
> This series seems to have some coding style problems. See output below for
> more information:
> 
> Type: series
> Subject: [Qemu-devel] [PATCH v6 0/4] 9p: Fix file ID collisions
> Message-id: cover.1566503584.git.qemu_...@crudebyte.com
> 
> === TEST SCRIPT BEGIN ===
> #!/bin/bash
> git rev-parse base > /dev/null || exit 0
> git config --local diff.renamelimit 0
> git config --local diff.renames True
> git config --local diff.algorithm histogram
> ./scripts/checkpatch.pl --mailback base..
> === TEST SCRIPT END ===
> 
> Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
> From https://github.com/patchew-project/qemu
>  * [new tag] patchew/cover.1566503584.git.qemu_...@crudebyte.com -> 
> patchew/cover.1566503584.git.qemu_...@crudebyte.com
> Submodule 'capstone' (https://git.qemu.org/git/capstone.git) registered for 
> path 'capstone'
> Submodule 'dtc' (https://git.qemu.org/git/dtc.git) registered for path 'dtc'
> Submodule 'roms/QemuMacDrivers' (https://git.qemu.org/git/QemuMacDrivers.git) 
> registered for path 'roms/QemuMacDrivers'
> Submodule 'roms/SLOF' (https://git.qemu.org/git/SLOF.git) registered for path 
> 'roms/SLOF'
> Submodule 'roms/edk2' (https://git.qemu.org/git/edk2.git) registered for path 
> 'roms/edk2'
> Submodule 'roms/ipxe' (https://git.qemu.org/git/ipxe.git) registered for path 
> 'roms/ipxe'
> Submodule 'roms/openbios' (https://git.qemu.org/git/openbios.git) registered 
> for path 'roms/openbios'
> Submodule 'roms/openhackware' (https://git.qemu.org/git/openhackware.git) 
> registered for path 'roms/openhackware'
> Submodule 'roms/opensbi' (https://git.qemu.org/git/opensbi.git) registered 
> for path 'roms/opensbi'
> Submodule 'roms/qemu-palcode' (https://git.qemu.org/git/qemu-palcode.git) 
> registered for path 'roms/qemu-palcode'
> Submodule 'roms/seabios' (https://git.qemu.org/git/seabios.git/) registered 
> for path 'roms/seabios'
> Submodule 'roms/seabios-hppa' (https://git.qemu.org/git/seabios-hppa.git) 
> registered for path 'roms/seabios-hppa'
> Submodule 'roms/sgabios' (https://git.qemu.org/git/sgabios.git) registered 
> for path 'roms/sgabios'
> Submodule 'roms/skiboot' (https://git.qemu.org/git/skiboot.git) registered 
> for path 'roms/skiboot'
> Submodule 'roms/u-boot' (https://git.qemu.org/git/u-boot.git) registered for 
> path 'roms/u-boot'
> Submodule 'roms/u-boot-sam460ex' 
> (https://git.qemu.org/git/u-boot-sam460ex.git) registered for path 
> 'roms/u-boot-sam460ex'
> Submodule 'slirp' (https://git.qemu.org/git/libslirp.git) registered for path 
> 'slirp'
> Submodule 'tests/fp/berkeley-softfloat-3' 
> (https://git.qemu.org/git/berkeley-softfloat-3.git) registered for path 
> 'tests/fp/berkeley-softfloat-3'
> Submodule 'tests/fp/berkeley-testfloat-3' 
> (https://git.qemu.org/git/berkeley-testfloat-3.git) registered for path 
> 'tests/fp/berkeley-testfloat-3'
> Submodule 'ui/keycodemapdb' (https://git.qemu.org/git/keycodemapdb.git) 
> registered for path 'ui/keycodemapdb'
> Cloning into 'capstone'...
> Submodule path 'capstone': checked out 
> '22ead3e0bfdb87516656453336160e0a37b066bf'
> Cloning into 'dtc'...
> Submodule path 'dtc': checked out '88f18909db731a627456f26d779445f84e449536'
> Cloning into 'roms/QemuMacDrivers'...
> Submodule path 'roms/QemuMacDrivers': checked out 
> '90c488d5f4a407342247b9ea869df1c2d9c8e266'
> Cloning into 'roms/SLOF'...
> Submodule path 'roms/SLOF': checked out 
> '7bfe584e321946771692711ff83ad2b5850daca7'
> Cloning into 'roms/edk2'...
> Submodule path 'roms/edk2': checked out 
> '20d2e5a125e34fc8501026613a71549b2a1a3e54'
> Submodule 'SoftFloat' (https://github.com/ucb-bar/berkeley-softfloat-3.git) 
> registered for path 'ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3'
> Submodule 'CryptoPkg/Library/OpensslLib/openssl' 
> (https://github.com/openssl/openssl) registered for path 
> 'CryptoPkg/Library/OpensslLib/openssl'
> Cloning into 'ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3'...
> Submodule path 
> 'roms/edk2/ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3': checked out 
> 'b64af41c3276f97f0e181920400ee056b9c88037'
> Cloning into 'CryptoPkg/Library/OpensslLib/openssl'...
> Submodule path 'roms/edk2/CryptoPkg/Library/OpensslLib/openssl': checked out 
> '50eaac9f3337667259de725451f201e784599687'
> Submodule 'boringssl' (https://boringssl.googlesource.com/boringssl) 
> registered for path 'boringssl'
> Submodule 'krb5' (https://github.com/krb5/krb5) registered for path 'krb5'
> Submodule 'pyca.cryptography' (https://github.com/pyca/cryptography.git) 
> registered for path 'pyca-cryptography'
> Cloning into 'boringssl'...
> Submodule path 'roms/edk2/CryptoPkg/Library/OpensslLib/openssl/boringssl': 
> checked out '2070f8ad9151dc8f3a73bffaa146b5e6937a583f'
> Cloning into 'krb5'...
> Submodule path 

Re: [Qemu-devel] [edk2-rfc] [edk2-devel] CPU hotplug using SMM with QEMU+OVMF

[Laszlo Ersek]

On 08/27/19 20:31, Igor Mammedov wrote:
> On Sat, 24 Aug 2019 01:48:09 +
> "Yao, Jiewen"  wrote:

>> (05) Host CPU: (OS) Port 0xB2 write, all CPUs enter SMM (NOTE: New CPU
>>  will not enter CPU because SMI is disabled)
> I think only CPU that does the write will enter SMM

That used to be the case (and it is still the default QEMU behavior, if
broadcast SMI is not negotiated). However, OVMF does negotiate broadcast
SMI whenever QEMU offers the feature. Broadcast SMI is important for the
stability of the edk2 SMM infrastructure on QEMU/KVM, we've found.

https://bugzilla.redhat.com/show_bug.cgi?id=1412313
https://bugzilla.redhat.com/show_bug.cgi?id=1412327

> and we might not need to pull in all already initialized CPUs into SMM.

That, on the other hand, could be a valid idea. But then the CPU should
use a different method for raising a synchronous SMI for itself (not a
write to IO port 0xB2). Is a "directed SMI for self" possible?

> [...]

I've tried to read through the procedure with your suggested changes,
but I'm failing at composing a coherent mental image, in this email
response format.

If you have the time, can you write up the suggested list of steps in a
"flat" format? (I believe you are suggesting to eliminate some steps
completely.)

... jumping to another point:

>> 2) Let trusted software (SMM and init code) guarantee SMREBASE one by one 
>> (include any code runs before SMREBASE)
> that would mean pulling all present CPUs into SMM mode so no attack
> code could be executing before doing hotplug. With a lot of present CPUs
> it could be quite expensive and unlike physical hardware, guest's CPUs
> could be preempted arbitrarily long causing long delays.

I agree with your analysis, but I slightly disagree about the impact:

- CPU hotplug is not a frequent administrative action, so the CPU load
should be temporary (it should be a spike). I don't worry that it would
trip up OS kernel code. (SMI handling is known to take long on physical
platforms oo.) In practice, all "normal" SMIs are broadcast already (for
example when calling the runtime UEFI variable services from the OS kernel).

- The fact that QEMU/KVM introduces some jitter into the execution of
multi-core code (including SMM code) has proved useful in the past, for
catching edk2 regressions.

Again, this is not a strong disagreement from my side. I'm open to
better ways for synching CPUs during muti-CPU-hotplug.

(Digression:

I expect someone could be curious why (a) I find it acceptable (even
beneficial) that "some jitter" injected by the QEMU/KVM scheduling
exposes multi-core regressions in edk2, but at the same time (b) I found
it really important to add broadcast SMI to QEMU and OVMF. After all,
both "jitter" and "unicast SMIs" are QEMU/KVM platform specifics, so why
the different treatment?

The reason is that the "jitter" does not interfere with normal
operation, and it has been good for catching *regressions*. IOW, there
is a working edk2 state, someone posts a patch, works on physical
hardware, but breaks on QEMU/KVM --> then we can still reject or rework
or revert the patch. And we're back to a working state again (in the
best case, with a fixed feature patch).

With the unicast SMIs however, it was impossible to enable the SMM stack
reliably in the first place. There was no functional state to return to.

Digression ends.)

> lets first see if if we can ignore race

Makes me uncomfortable, but if this is the consensus, I'll go along.

> and if it's not then
> we probably end up with implementing some form of #1

OK.

Thanks!
Laszlo

Re: [Qemu-devel] [PATCH v2 5/8] cputlb: Fix size operand for tlb_fill on unaligned store

[Philippe Mathieu-Daudé]

On 8/29/19 1:16 AM, Richard Henderson wrote:
> We are currently passing the size of the full write to
> the tlb_fill for the second page.  Instead pass the real
> size of the write to that page.
> 
> This argument is unused within all tlb_fill, except to be
> logged via tracing, so in practice this makes no difference.
> 
> But in a moment we'll need the value of size2 for watchpoints,
> and if we've computed the value we might as well use it.
> 
> Signed-off-by: Richard Henderson 

Reviewed-by: Philippe Mathieu-Daudé 

> ---
>  accel/tcg/cputlb.c | 5 -
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index c9576bebcf..7fb67d2f05 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -1504,6 +1504,8 @@ store_helper(CPUArchState *env, target_ulong addr, 
> uint64_t val,
>  uintptr_t index2;
>  CPUTLBEntry *entry2;
>  target_ulong page2, tlb_addr2;
> +size_t size2;
> +
>  do_unaligned_access:
>  /*
>   * Ensure the second page is in the TLB.  Note that the first page
> @@ -1511,13 +1513,14 @@ store_helper(CPUArchState *env, target_ulong addr, 
> uint64_t val,
>   * cannot evict the first.
>   */
>  page2 = (addr + size) & TARGET_PAGE_MASK;
> +size2 = (addr + size) & ~TARGET_PAGE_MASK;
>  index2 = tlb_index(env, mmu_idx, page2);
>  entry2 = tlb_entry(env, mmu_idx, page2);
>  tlb_addr2 = tlb_addr_write(entry2);
>  if (!tlb_hit_page(tlb_addr2, page2)
>  && !victim_tlb_hit(env, mmu_idx, index2, tlb_off,
> page2 & TARGET_PAGE_MASK)) {
> -tlb_fill(env_cpu(env), page2, size, MMU_DATA_STORE,
> +tlb_fill(env_cpu(env), page2, size2, MMU_DATA_STORE,
>   mmu_idx, retaddr);
>  }
>  
> 

Re: [Qemu-devel] [PATCH v6 2/4] 9p: Added virtfs option 'multidevs=remap|forbid|warn'

[Greg Kurz]

On Thu, 22 Aug 2019 21:33:37 +0200
Christian Schoenebeck  wrote:

> 'warn' (default): Only log an error message (once) on host if more than one
> device is shared by same export, except of that just ignore this config
> error though. This is the default behaviour for not breaking existing
> installations implying that they really know what they are doing.
> 
> 'forbid': Like 'warn', but except of just logging an error this
> also denies access of guest to additional devices.
> 
> 'remap': Allows to share more than one device per export by remapping
> inodes from host to guest appropriately. To support multiple devices on the
> 9p share, and avoid qid path collisions we take the device id as input to
> generate a unique QID path. The lowest 48 bits of the path will be set
> equal to the file inode, and the top bits will be uniquely assigned based
> on the top 16 bits of the inode and the device id.
> 
> Signed-off-by: Antonios Motakis 
> [CS: - Rebased to https://github.com/gkurz/qemu/commits/9p-next
>(SHA1 177fd3b6a8).
>  - Updated hash calls to new xxhash API.
>  - Added virtfs option 'multidevs', original patch simply did the inode
>remapping without being asked.
>  - Updated docs for new option 'multidevs'.
>  - Capture root_ino in v9fs_device_realize_common() as well, not just
>the device id.
>  - Fixed v9fs_do_readdir() not having remapped inodes.
>  - Log error message when running out of prefixes in
>qid_path_prefixmap().
>  - Fixed definition of QPATH_INO_MASK.
>  - Dropped unnecessary parantheses in qpp_lookup_func().
>  - Dropped unnecessary g_malloc0() result checks. ]
> Signed-off-by: Christian Schoenebeck 
> ---

Looks good but needs some more polishing. See below.

>  fsdev/file-op-9p.h  |   5 ++
>  fsdev/qemu-fsdev-opts.c |   7 +-
>  fsdev/qemu-fsdev.c  |  11 +++
>  hw/9pfs/9p.c| 182 
> ++--
>  hw/9pfs/9p.h|  13 
>  qemu-options.hx |  33 +++--
>  vl.c|   6 +-
>  7 files changed, 229 insertions(+), 28 deletions(-)
> 
> diff --git a/fsdev/file-op-9p.h b/fsdev/file-op-9p.h
> index c757c8099f..f2f7772c86 100644
> --- a/fsdev/file-op-9p.h
> +++ b/fsdev/file-op-9p.h
> @@ -59,6 +59,11 @@ typedef struct ExtendedOps {
>  #define V9FS_RDONLY 0x0040
>  #define V9FS_PROXY_SOCK_FD  0x0080
>  #define V9FS_PROXY_SOCK_NAME0x0100
> +/*
> + * multidevs option (either one of the two applies exclusively)
> + */
> +#define V9FS_REMAP_INODES   0x0200
> +#define V9FS_FORBID_MULTIDEVS   0x0400
>  
>  #define V9FS_SEC_MASK   0x003C
>  
> diff --git a/fsdev/qemu-fsdev-opts.c b/fsdev/qemu-fsdev-opts.c
> index 7c31af..07a18c6e48 100644
> --- a/fsdev/qemu-fsdev-opts.c
> +++ b/fsdev/qemu-fsdev-opts.c
> @@ -31,7 +31,9 @@ static QemuOptsList qemu_fsdev_opts = {
>  }, {
>  .name = "readonly",
>  .type = QEMU_OPT_BOOL,
> -
> +}, {
> +.name = "multidevs",
> +.type = QEMU_OPT_STRING,
>  }, {
>  .name = "socket",
>  .type = QEMU_OPT_STRING,
> @@ -76,6 +78,9 @@ static QemuOptsList qemu_virtfs_opts = {
>  .name = "readonly",
>  .type = QEMU_OPT_BOOL,
>  }, {
> +.name = "multidevs",
> +.type = QEMU_OPT_STRING,
> +}, {
>  .name = "socket",
>  .type = QEMU_OPT_STRING,
>  }, {
> diff --git a/fsdev/qemu-fsdev.c b/fsdev/qemu-fsdev.c
> index 077a8c4e2b..ed03d559a9 100644
> --- a/fsdev/qemu-fsdev.c
> +++ b/fsdev/qemu-fsdev.c
> @@ -58,6 +58,7 @@ static FsDriverTable FsDrivers[] = {
>  "writeout",
>  "fmode",
>  "dmode",
> +"multidevs",

So we only allow this for the "local" backend. Any reason not to
add this to "proxy" as well ?

I didn't do it for the "throttling" options because it is a
feature I didn't care to support much, but "multidevs" is more
a fix than a fancy feature.

>  "throttling.bps-total",
>  "throttling.bps-read",
>  "throttling.bps-write",
> @@ -121,6 +122,7 @@ int qemu_fsdev_add(QemuOpts *opts, Error **errp)
>  const char *fsdev_id = qemu_opts_id(opts);
>  const char *fsdriver = qemu_opt_get(opts, "fsdriver");
>  const char *writeout = qemu_opt_get(opts, "writeout");
> +const char *multidevs = qemu_opt_get(opts, "multidevs");
>  bool ro = qemu_opt_get_bool(opts, "readonly", 0);
>  
>  if (!fsdev_id) {
> @@ -161,6 +163,15 @@ int qemu_fsdev_add(QemuOpts *opts, Error **errp)
>  } else {
>  fsle->fse.export_flags &= ~V9FS_RDONLY;
>  }
> +if (multidevs) {
> +if (!strcmp(multidevs, "remap")) {
> +fsle->fse.export_flags &= ~V9FS_FORBID_MULTIDEVS;
> +fsle->fse.export_flags |= V9FS_REMAP_INODES;
> +} else if 

Re: [Qemu-devel] [PATCH v2 6/8] cputlb: Remove double-alignment in store_helper

[Philippe Mathieu-Daudé]

On 8/29/19 1:16 AM, Richard Henderson wrote:
> We have already aligned page2 to the start of the next page.
> There is no reason to do that a second time.
> 
> Signed-off-by: Richard Henderson 

Reviewed-by: Philippe Mathieu-Daudé 

> ---
>  accel/tcg/cputlb.c | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
> index 7fb67d2f05..d0f8db33a2 100644
> --- a/accel/tcg/cputlb.c
> +++ b/accel/tcg/cputlb.c
> @@ -1518,8 +1518,7 @@ store_helper(CPUArchState *env, target_ulong addr, 
> uint64_t val,
>  entry2 = tlb_entry(env, mmu_idx, page2);
>  tlb_addr2 = tlb_addr_write(entry2);
>  if (!tlb_hit_page(tlb_addr2, page2)
> -&& !victim_tlb_hit(env, mmu_idx, index2, tlb_off,
> -   page2 & TARGET_PAGE_MASK)) {
> +&& !victim_tlb_hit(env, mmu_idx, index2, tlb_off, page2)) {
>  tlb_fill(env_cpu(env), page2, size2, MMU_DATA_STORE,
>   mmu_idx, retaddr);
>  }
> 

Re: [Qemu-devel] [PATCH v2 1/8] exec: Move user-only watchpoint stubs inline

[Philippe Mathieu-Daudé]

On 8/29/19 1:16 AM, Richard Henderson wrote:
> Let the user-only watchpoint stubs resolve to empty inline functions.
> 
> Reviewed-by: David Hildenbrand 
> Signed-off-by: Richard Henderson 

Reviewed-by: Philippe Mathieu-Daudé 

> ---
>  include/hw/core/cpu.h | 23 +++
>  exec.c| 26 ++
>  2 files changed, 25 insertions(+), 24 deletions(-)
> 
> diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
> index 77fca95a40..6de688059d 100644
> --- a/include/hw/core/cpu.h
> +++ b/include/hw/core/cpu.h
> @@ -1070,12 +1070,35 @@ static inline bool cpu_breakpoint_test(CPUState *cpu, 
> vaddr pc, int mask)
>  return false;
>  }
>  
> +#ifdef CONFIG_USER_ONLY
> +static inline int cpu_watchpoint_insert(CPUState *cpu, vaddr addr, vaddr len,
> +int flags, CPUWatchpoint 
> **watchpoint)
> +{
> +return -ENOSYS;
> +}
> +
> +static inline int cpu_watchpoint_remove(CPUState *cpu, vaddr addr,
> +vaddr len, int flags)
> +{
> +return -ENOSYS;
> +}
> +
> +static inline void cpu_watchpoint_remove_by_ref(CPUState *cpu,
> +CPUWatchpoint *wp)
> +{
> +}
> +
> +static inline void cpu_watchpoint_remove_all(CPUState *cpu, int mask)
> +{
> +}
> +#else
>  int cpu_watchpoint_insert(CPUState *cpu, vaddr addr, vaddr len,
>int flags, CPUWatchpoint **watchpoint);
>  int cpu_watchpoint_remove(CPUState *cpu, vaddr addr,
>vaddr len, int flags);
>  void cpu_watchpoint_remove_by_ref(CPUState *cpu, CPUWatchpoint *watchpoint);
>  void cpu_watchpoint_remove_all(CPUState *cpu, int mask);
> +#endif
>  
>  /**
>   * cpu_get_address_space:
> diff --git a/exec.c b/exec.c
> index 53a15b7ad7..31fb75901f 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -1062,28 +1062,7 @@ static void breakpoint_invalidate(CPUState *cpu, 
> target_ulong pc)
>  }
>  #endif
>  
> -#if defined(CONFIG_USER_ONLY)
> -void cpu_watchpoint_remove_all(CPUState *cpu, int mask)
> -
> -{
> -}
> -
> -int cpu_watchpoint_remove(CPUState *cpu, vaddr addr, vaddr len,
> -  int flags)
> -{
> -return -ENOSYS;
> -}
> -
> -void cpu_watchpoint_remove_by_ref(CPUState *cpu, CPUWatchpoint *watchpoint)
> -{
> -}
> -
> -int cpu_watchpoint_insert(CPUState *cpu, vaddr addr, vaddr len,
> -  int flags, CPUWatchpoint **watchpoint)
> -{
> -return -ENOSYS;
> -}
> -#else
> +#ifndef CONFIG_USER_ONLY
>  /* Add a watchpoint.  */
>  int cpu_watchpoint_insert(CPUState *cpu, vaddr addr, vaddr len,
>int flags, CPUWatchpoint **watchpoint)
> @@ -1173,8 +1152,7 @@ static inline bool 
> cpu_watchpoint_address_matches(CPUWatchpoint *wp,
>  
>  return !(addr > wpend || wp->vaddr > addrend);
>  }
> -
> -#endif
> +#endif /* !CONFIG_USER_ONLY */
>  
>  /* Add a breakpoint.  */
>  int cpu_breakpoint_insert(CPUState *cpu, vaddr pc, int flags,
> 

[Qemu-devel] [PATCH v2] virtio-mmio: implement modern (v2) personality (virtio-1)

[Sergio Lopez]

Implement the modern (v2) personality, according to the VirtIO 1.0
specification.

Support for v2 among guests is not as widespread as it'd be
desirable. While the Linux driver has had it for a while, support is
missing, at least, from Tianocore EDK II, NetBSD and FreeBSD.

For this reason, the v2 personality is disabled, keeping the legacy
behavior as default. Machine types willing to use v2, can enable it
using MachineClass's compat_props.

Signed-off-by: Sergio Lopez 
---
Changelog:

v2:
 - Switch from RFC to PATCH.
 - Avoid the modern vs. legacy dichotomy. Use legacy or non-legacy
   instead. (Andrea Bolognani, Cornelia Huck)
 - Include the register offset in the warning messages. (Stefan
   Hajnoczi)
 - Fix device endianness for the non-legacy mode. (Michael S. Tsirkin)
 - Honor the specs in VIRTIO_MMIO_QUEUE_READY. (Michael S. Tsirkin)
---
 hw/virtio/virtio-mmio.c | 296 +---
 1 file changed, 279 insertions(+), 17 deletions(-)

diff --git a/hw/virtio/virtio-mmio.c b/hw/virtio/virtio-mmio.c
index 97b7f35496..1e47fbafe5 100644
--- a/hw/virtio/virtio-mmio.c
+++ b/hw/virtio/virtio-mmio.c
@@ -47,14 +47,24 @@
 OBJECT_CHECK(VirtIOMMIOProxy, (obj), TYPE_VIRTIO_MMIO)
 
 #define VIRT_MAGIC 0x74726976 /* 'virt' */
-#define VIRT_VERSION 1
+#define VIRT_VERSION 2
+#define VIRT_VERSION_LEGACY 1
 #define VIRT_VENDOR 0x554D4551 /* 'QEMU' */
 
+typedef struct VirtIOMMIOQueue {
+uint16_t num;
+bool enabled;
+uint32_t desc[2];
+uint32_t avail[2];
+uint32_t used[2];
+} VirtIOMMIOQueue;
+
 typedef struct {
 /* Generic */
 SysBusDevice parent_obj;
 MemoryRegion iomem;
 qemu_irq irq;
+bool legacy;
 /* Guest accessible state needing migration and reset */
 uint32_t host_features_sel;
 uint32_t guest_features_sel;
@@ -62,6 +72,9 @@ typedef struct {
 /* virtio-bus */
 VirtioBusState bus;
 bool format_transport_address;
+/* Fields only used for non-legacy (v2) devices */
+uint32_t guest_features[2];
+VirtIOMMIOQueue vqs[VIRTIO_QUEUE_MAX];
 } VirtIOMMIOProxy;
 
 static bool virtio_mmio_ioeventfd_enabled(DeviceState *d)
@@ -115,7 +128,11 @@ static uint64_t virtio_mmio_read(void *opaque, hwaddr 
offset, unsigned size)
 case VIRTIO_MMIO_MAGIC_VALUE:
 return VIRT_MAGIC;
 case VIRTIO_MMIO_VERSION:
-return VIRT_VERSION;
+if (proxy->legacy) {
+return VIRT_VERSION_LEGACY;
+} else {
+return VIRT_VERSION;
+}
 case VIRTIO_MMIO_VENDOR_ID:
 return VIRT_VENDOR;
 default:
@@ -146,28 +163,51 @@ static uint64_t virtio_mmio_read(void *opaque, hwaddr 
offset, unsigned size)
 case VIRTIO_MMIO_MAGIC_VALUE:
 return VIRT_MAGIC;
 case VIRTIO_MMIO_VERSION:
-return VIRT_VERSION;
+if (proxy->legacy) {
+return VIRT_VERSION_LEGACY;
+} else {
+return VIRT_VERSION;
+}
 case VIRTIO_MMIO_DEVICE_ID:
 return vdev->device_id;
 case VIRTIO_MMIO_VENDOR_ID:
 return VIRT_VENDOR;
 case VIRTIO_MMIO_DEVICE_FEATURES:
-if (proxy->host_features_sel) {
-return 0;
-}
-return vdev->host_features;
+return vdev->host_features >> (32 * proxy->host_features_sel);
 case VIRTIO_MMIO_QUEUE_NUM_MAX:
 if (!virtio_queue_get_num(vdev, vdev->queue_sel)) {
 return 0;
 }
 return VIRTQUEUE_MAX_SIZE;
 case VIRTIO_MMIO_QUEUE_PFN:
+if (!proxy->legacy) {
+qemu_log_mask(LOG_GUEST_ERROR,
+"%s: read from legacy register (0x%lx) in non-legacy mode\n",
+__func__, offset);
+return 0;
+}
 return virtio_queue_get_addr(vdev, vdev->queue_sel)
 >> proxy->guest_page_shift;
+case VIRTIO_MMIO_QUEUE_READY:
+if (proxy->legacy) {
+qemu_log_mask(LOG_GUEST_ERROR,
+"%s: read from non-legacy register (0x%lx) in legacy mode\n",
+__func__, offset);
+return 0;
+}
+return proxy->vqs[vdev->queue_sel].enabled;
 case VIRTIO_MMIO_INTERRUPT_STATUS:
 return atomic_read(>isr);
 case VIRTIO_MMIO_STATUS:
 return vdev->status;
+case VIRTIO_MMIO_CONFIG_GENERATION:
+if (proxy->legacy) {
+qemu_log_mask(LOG_GUEST_ERROR,
+"%s: read from non-legacy register (0x%lx) in legacy mode\n",
+__func__, offset);
+return 0;
+}
+return vdev->generation;
 case VIRTIO_MMIO_DEVICE_FEATURES_SEL:
 case VIRTIO_MMIO_DRIVER_FEATURES:
 case VIRTIO_MMIO_DRIVER_FEATURES_SEL:
@@ -177,6 +217,12 @@ static uint64_t virtio_mmio_read(void *opaque, hwaddr 
offset, unsigned size)
 case VIRTIO_MMIO_QUEUE_ALIGN:
 case VIRTIO_MMIO_QUEUE_NOTIFY:
 case VIRTIO_MMIO_INTERRUPT_ACK:
+case VIRTIO_MMIO_QUEUE_DESC_LOW:
+case 

[Qemu-devel] [PATCH v3 0/4] docs: add docs about use of automatic cleanup functions

[Daniel P . Berrangé]

This is ostensibly about adding docs for the g_autofree/g_autoptr
macros. As part of doing that, however, the existing HACKING doc
is merged into the CODING_STYLE doc and the text is converted to
rst with a table of contents.

Changed in v3:

 - Make checkpatch.pl aware of README.rst rename

Changed in v2:

 - Use RST instead of markdown
 - Also convert README file
 - Group sections better
 - Remove GMutexLocker example

Daniel P. Berrangé (4):
  docs: convert README, CODING_STYLE and HACKING to RST syntax
  docs: merge HACKING.rst contents into CODING_STYLE.rst
  docs: document use of automatic cleanup functions in glib
  docs: split the CODING_STYLE doc into distinct groups

 CODING_STYLE  | 216 --
 CODING_STYLE.rst  | 641 ++
 HACKING   | 257 -
 README => README.rst  |  47 ++--
 scripts/checkpatch.pl |   2 +-
 5 files changed, 671 insertions(+), 492 deletions(-)
 delete mode 100644 CODING_STYLE
 create mode 100644 CODING_STYLE.rst
 delete mode 100644 HACKING
 rename README => README.rst (84%)

-- 
2.21.0

Re: [Qemu-devel] [PATCH v2 2/4] docs: merge HACKING.rst contents into CODING_STYLE.rst

[Alex Bennée]

Daniel P. Berrangé  writes:

> The split of information between the two docs is rather arbitary and
> unclear. It is simpler for contributors if all the information is in
> one file.
>
> Signed-off-by: Daniel P. Berrangé 

Reviewed-by: Alex Bennée 

> ---
>  CODING_STYLE.rst | 296 ++
>  HACKING.rst  | 300 ---
>  README.rst   |   2 +-
>  3 files changed, 297 insertions(+), 301 deletions(-)
>  delete mode 100644 HACKING.rst
>
> diff --git a/CODING_STYLE.rst b/CODING_STYLE.rst
> index 713357cb80..4501d87352 100644
> --- a/CODING_STYLE.rst
> +++ b/CODING_STYLE.rst
> @@ -205,6 +205,302 @@ comment anyway.)
>  Rationale: Consistency, and ease of visually picking out a multiline
>  comment from the surrounding code.
>
> +Preprocessor
> +
> +
> +Variadic macros
> +---
> +
> +For variadic macros, stick with this C99-like syntax:
> +
> +.. code-block:: c
> +
> +#define DPRINTF(fmt, ...)   \
> +do { printf("IRQ: " fmt, ## __VA_ARGS__); } while (0)
> +
> +Include directives
> +--
> +
> +Order include directives as follows:
> +
> +.. code-block:: c
> +
> +#include "qemu/osdep.h"  /* Always first... */
> +#include <...>   /* then system headers... */
> +#include "..."   /* and finally QEMU headers. */
> +
> +The "qemu/osdep.h" header contains preprocessor macros that affect the 
> behavior
> +of core system headers like .  It must be the first include so that
> +core system headers included by external libraries get the preprocessor 
> macros
> +that QEMU depends on.
> +
> +Do not include "qemu/osdep.h" from header files since the .c file will have
> +already included it.
> +
> +C types
> +===
> +
> +It should be common sense to use the right type, but we have collected
> +a few useful guidelines here.
> +
> +Scalars
> +---
> +
> +If you're using "int" or "long", odds are good that there's a better type.
> +If a variable is counting something, it should be declared with an
> +unsigned type.
> +
> +If it's host memory-size related, size_t should be a good choice (use
> +ssize_t only if required). Guest RAM memory offsets must use ram_addr_t,
> +but only for RAM, it may not cover whole guest address space.
> +
> +If it's file-size related, use off_t.
> +If it's file-offset related (i.e., signed), use off_t.
> +If it's just counting small numbers use "unsigned int";
> +(on all but oddball embedded systems, you can assume that that
> +type is at least four bytes wide).
> +
> +In the event that you require a specific width, use a standard type
> +like int32_t, uint32_t, uint64_t, etc.  The specific types are
> +mandatory for VMState fields.
> +
> +Don't use Linux kernel internal types like u32, __u32 or __le32.
> +
> +Use hwaddr for guest physical addresses except pcibus_t
> +for PCI addresses.  In addition, ram_addr_t is a QEMU internal address
> +space that maps guest RAM physical addresses into an intermediate
> +address space that can map to host virtual address spaces.  Generally
> +speaking, the size of guest memory can always fit into ram_addr_t but
> +it would not be correct to store an actual guest physical address in a
> +ram_addr_t.
> +
> +For CPU virtual addresses there are several possible types.
> +vaddr is the best type to use to hold a CPU virtual address in
> +target-independent code. It is guaranteed to be large enough to hold a
> +virtual address for any target, and it does not change size from target
> +to target. It is always unsigned.
> +target_ulong is a type the size of a virtual address on the CPU; this means
> +it may be 32 or 64 bits depending on which target is being built. It should
> +therefore be used only in target-specific code, and in some
> +performance-critical built-per-target core code such as the TLB code.
> +There is also a signed version, target_long.
> +abi_ulong is for the ``*``-user targets, and represents a type the size of
> +'void ``*``' in that target's ABI. (This may not be the same as the size of a
> +full CPU virtual address in the case of target ABIs which use 32 bit pointers
> +on 64 bit CPUs, like sparc32plus.) Definitions of structures that must match
> +the target's ABI must use this type for anything that on the target is 
> defined
> +to be an 'unsigned long' or a pointer type.
> +There is also a signed version, abi_long.
> +
> +Of course, take all of the above with a grain of salt.  If you're about
> +to use some system interface that requires a type like size_t, pid_t or
> +off_t, use matching types for any corresponding variables.
> +
> +Also, if you try to use e.g., "unsigned int" as a type, and that
> +conflicts with the signedness of a related variable, sometimes
> +it's best just to use the *wrong* type, if "pulling the thread"
> +and fixing all related variables would be too invasive.
> +
> +Finally, while using descriptive types is 

[Qemu-devel] [PATCH v3 2/4] docs: merge HACKING.rst contents into CODING_STYLE.rst

[Daniel P . Berrangé]

The split of information between the two docs is rather arbitary and
unclear. It is simpler for contributors if all the information is in
one file.

Signed-off-by: Daniel P. Berrangé 
---
 CODING_STYLE.rst | 296 ++
 HACKING.rst  | 300 ---
 README.rst   |   2 +-
 3 files changed, 297 insertions(+), 301 deletions(-)
 delete mode 100644 HACKING.rst

diff --git a/CODING_STYLE.rst b/CODING_STYLE.rst
index 713357cb80..4501d87352 100644
--- a/CODING_STYLE.rst
+++ b/CODING_STYLE.rst
@@ -205,6 +205,302 @@ comment anyway.)
 Rationale: Consistency, and ease of visually picking out a multiline
 comment from the surrounding code.
 
+Preprocessor
+
+
+Variadic macros
+---
+
+For variadic macros, stick with this C99-like syntax:
+
+.. code-block:: c
+
+#define DPRINTF(fmt, ...)   \
+do { printf("IRQ: " fmt, ## __VA_ARGS__); } while (0)
+
+Include directives
+--
+
+Order include directives as follows:
+
+.. code-block:: c
+
+#include "qemu/osdep.h"  /* Always first... */
+#include <...>   /* then system headers... */
+#include "..."   /* and finally QEMU headers. */
+
+The "qemu/osdep.h" header contains preprocessor macros that affect the behavior
+of core system headers like .  It must be the first include so that
+core system headers included by external libraries get the preprocessor macros
+that QEMU depends on.
+
+Do not include "qemu/osdep.h" from header files since the .c file will have
+already included it.
+
+C types
+===
+
+It should be common sense to use the right type, but we have collected
+a few useful guidelines here.
+
+Scalars
+---
+
+If you're using "int" or "long", odds are good that there's a better type.
+If a variable is counting something, it should be declared with an
+unsigned type.
+
+If it's host memory-size related, size_t should be a good choice (use
+ssize_t only if required). Guest RAM memory offsets must use ram_addr_t,
+but only for RAM, it may not cover whole guest address space.
+
+If it's file-size related, use off_t.
+If it's file-offset related (i.e., signed), use off_t.
+If it's just counting small numbers use "unsigned int";
+(on all but oddball embedded systems, you can assume that that
+type is at least four bytes wide).
+
+In the event that you require a specific width, use a standard type
+like int32_t, uint32_t, uint64_t, etc.  The specific types are
+mandatory for VMState fields.
+
+Don't use Linux kernel internal types like u32, __u32 or __le32.
+
+Use hwaddr for guest physical addresses except pcibus_t
+for PCI addresses.  In addition, ram_addr_t is a QEMU internal address
+space that maps guest RAM physical addresses into an intermediate
+address space that can map to host virtual address spaces.  Generally
+speaking, the size of guest memory can always fit into ram_addr_t but
+it would not be correct to store an actual guest physical address in a
+ram_addr_t.
+
+For CPU virtual addresses there are several possible types.
+vaddr is the best type to use to hold a CPU virtual address in
+target-independent code. It is guaranteed to be large enough to hold a
+virtual address for any target, and it does not change size from target
+to target. It is always unsigned.
+target_ulong is a type the size of a virtual address on the CPU; this means
+it may be 32 or 64 bits depending on which target is being built. It should
+therefore be used only in target-specific code, and in some
+performance-critical built-per-target core code such as the TLB code.
+There is also a signed version, target_long.
+abi_ulong is for the ``*``-user targets, and represents a type the size of
+'void ``*``' in that target's ABI. (This may not be the same as the size of a
+full CPU virtual address in the case of target ABIs which use 32 bit pointers
+on 64 bit CPUs, like sparc32plus.) Definitions of structures that must match
+the target's ABI must use this type for anything that on the target is defined
+to be an 'unsigned long' or a pointer type.
+There is also a signed version, abi_long.
+
+Of course, take all of the above with a grain of salt.  If you're about
+to use some system interface that requires a type like size_t, pid_t or
+off_t, use matching types for any corresponding variables.
+
+Also, if you try to use e.g., "unsigned int" as a type, and that
+conflicts with the signedness of a related variable, sometimes
+it's best just to use the *wrong* type, if "pulling the thread"
+and fixing all related variables would be too invasive.
+
+Finally, while using descriptive types is important, be careful not to
+go overboard.  If whatever you're doing causes warnings, or requires
+casts, then reconsider or ask for help.
+
+Pointers
+
+
+Ensure that all of your pointers are "const-correct".
+Unless a pointer is used to modify the pointed-to storage,
+give it the "const" 

Re: [Qemu-devel] [PATCH v1 0/9] arm semih-hosting cleanups and other misc cleanups

[Philippe Mathieu-Daudé]

On 8/28/19 6:52 PM, Alex Bennée wrote:
> Hi Peter,
> 
> Here are the ARM semi-hosting cleanup patches and a small selection of
> miscellaneous TCG clean-ups. If your happy taking them all through
> your tree please do otherwise I'll poke Richard. I'm just trying to
> reduce the delta for my next iteration of the plugin series.
> 
> The following patches need review
>04 - target arm remove run time semihosting checks
>05 - includes remove stale smp max _cpus externs
> 
> The first is the result of review comments, moving CONFIG_TCG ifdefs
> around and the second is pretty trivial.
> 
> Alex Bennée (7):
>   target/arm: handle M-profile semihosting at translate time
>   target/arm: handle A-profile T32 semihosting at translate time
>   target/arm: handle A-profile A32 semihosting at translate time
>   target/arm: remove run time semihosting checks
>   includes: remove stale [smp|max]_cpus externs
>   accel/stubs: reduce headers from tcg-stub
>   include/exec/cpu-defs.h: fix typo
> 
> Emilio G. Cota (2):
>   tcg/README: fix typo s/afterwise/afterwards/
>   atomic_template: fix indentation in GEN_ATOMIC_HELPER

Thanks for cleaning this!

Series except patch #8 which you dropped:
Reviewed-by: Philippe Mathieu-Daudé 

[Qemu-devel] [PATCH v7 10/10] linux-user: dumping hot TBs at the end of the execution

[vandersonmr]

dumps, in linux-user mode, the hottest TBs if -d tb_stats is used.

Example of output for the 3 hottest TBs:

TB id:1 | phys:0x34d54 virt:0x00034d54 flags:0xf0
| exec:4828932/0 guest inst cov:16.38%
| trans:1 ints: g:3 op:82 op_opt:34 spills:3
| h/g (host bytes / guest insts): 90.64
| time to gen at 2.4GHz => code:3150.83(ns) IR:712.08(ns)
| targets: 0x00034d5e (id:11), 0x00034d0d (id:2)

TB id:2 | phys:0x34d0d virt:0x00034d0d flags:0xf0
| exec:4825842/0 guest inst cov:21.82%
| trans:1 ints: g:4 op:80 op_opt:38 spills:2
| h/g (host bytes / guest insts): 84.00
| time to gen at 2.4GHz => code:3362.92(ns) IR:793.75(ns)
| targets: 0x00034d19 (id:12), 0x00034d54 (id:1)

TB id:3 | phys:0xec1c1 virt:0x000ec1c1 flags:0xb0
| exec:872032/0 guest inst cov:1.97%
| trans:1 ints: g:2 op:56 op_opt:26 spills:1
| h/g (host bytes / guest insts): 68.00
| time to gen at 2.4GHz => code:1692.08(ns) IR:473.75(ns)
| targets: 0x000ec1c5 (id:4), 0x000ec1cb (id:13)

Signed-off-by: Vanderson M. do Rosario 
---
 linux-user/exit.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/linux-user/exit.c b/linux-user/exit.c
index bdda720553..7226104959 100644
--- a/linux-user/exit.c
+++ b/linux-user/exit.c
@@ -28,6 +28,10 @@ extern void __gcov_dump(void);
 
 void preexit_cleanup(CPUArchState *env, int code)
 {
+if (tb_stats_collection_enabled()) {
+dump_tbs_info(max_num_hot_tbs_to_dump, SORT_BY_HOTNESS, false);
+}
+
 #ifdef TARGET_GPROF
 _mcleanup();
 #endif
-- 
2.22.0

[Qemu-devel] [PATCH v7 08/10] Adding info [tb-list|tb|coverset] commands to HMP.

[vandersonmr]

These commands allow the exploration of TBs
generated by the TCG. Understand which one
hotter, with more guest/host instructions...
and examine their guest, host and IR code.

The goal of this command is to allow the dynamic exploration
of TCG behavior and code quality. Therefore, for now, a
corresponding QMP command is not worthwhile.

Example of output:

TB id:1 | phys:0x34d54 virt:0x00034d54 flags:0xf0
| exec:4828932/0 guest inst cov:16.38%
| trans:1 ints: g:3 op:82 op_opt:34 spills:3
| h/g (host bytes / guest insts): 90.64
| time to gen at 2.4GHz => code:3150.83(ns) IR:712.08(ns)
| targets: 0x00034d5e (id:11), 0x00034d0d (id:2)

TB id:2 | phys:0x34d0d virt:0x00034d0d flags:0xf0
| exec:4825842/0 guest inst cov:21.82%
| trans:1 ints: g:4 op:80 op_opt:38 spills:2
| h/g (host bytes / guest insts): 84.00
| time to gen at 2.4GHz => code:3362.92(ns) IR:793.75(ns)
| targets: 0x00034d19 (id:12), 0x00034d54 (id:1)

TB id:2 | phys:0x34d0d virt:0x00034d0d flags:0xf0
| exec:6956495/0  guest inst cov:21.82%
| trans:2 ints: g:2 op:40 op_opt:19 spills:1
| h/g (host bytes / guest insts): 84.00
| time to gen at 2.4GHz => code:3130.83(ns) IR:722.50(ns)
| targets: 0x00034d19 (id:12), 0x00034d54 (id:1)


IN:
0x00034d0d:  89 demovl %ebx, %esi
0x00034d0f:  26 8b 0e movl %es:(%esi), %ecx
0x00034d12:  26 f6 46 08 80   testb$0x80, %es:8(%esi)
0x00034d17:  75 3bjne  0x34d54

--

TB id:1 | phys:0x34d54 virt:0x00034d54 flags:0xf0
| exec:5202686/0 guest inst cov:11.28%
| trans:1 ints: g:3 op:82 op_opt:34 spills:3
| h/g (host bytes / guest insts): 90.64
| time to gen at 2.4GHz => code:2793.75(ns) IR:614.58(ns)
| targets: 0x00034d5e (id:3), 0x00034d0d (id:2)

TB id:2 | phys:0x34d0d virt:0x00034d0d flags:0xf0
| exec:5199468/0 guest inst cov:15.03%
| trans:1 ints: g:4 op:80 op_opt:38 spills:2
| h/g (host bytes / guest insts): 84.00
| time to gen at 2.4GHz => code:2958.75(ns) IR:719.58(ns)
| targets: 0x00034d19 (id:4), 0x00034d54 (id:1)

--
2 TBs to reach 25% of guest inst exec coverage
Total of guest insts exec: 138346727

--

Acked-by: Dr. David Alan Gilbert 
Signed-off-by: Vanderson M. do Rosario 
---
 accel/tcg/tb-stats.c | 421 ++-
 accel/tcg/translate-all.c|   2 +-
 disas.c  |  31 ++-
 hmp-commands-info.hx |  24 ++
 include/exec/tb-stats.h  |  45 +++-
 include/qemu/log-for-trace.h |   4 +
 include/qemu/log.h   |   2 +
 monitor/misc.c   |  74 ++
 util/log.c   |  52 -
 9 files changed, 634 insertions(+), 21 deletions(-)

diff --git a/accel/tcg/tb-stats.c b/accel/tcg/tb-stats.c
index 9959477fbb..d588c551c9 100644
--- a/accel/tcg/tb-stats.c
+++ b/accel/tcg/tb-stats.c
@@ -34,9 +34,35 @@
 
 /* only accessed in safe work */
 static GList *last_search;
-
+int id = 1; /* display_id increment counter */
 uint64_t dev_time;
 
+static TBStatistics *get_tbstats_by_id(int id)
+{
+GList *iter;
+
+for (iter = last_search; iter; iter = g_list_next(iter)) {
+TBStatistics *tbs = iter->data;
+if (tbs && tbs->display_id == id) {
+return tbs;
+break;
+}
+}
+return NULL;
+}
+
+static TBStatistics *get_tbstats_by_addr(target_ulong pc)
+{
+GList *iter;
+for (iter = last_search; iter; iter = g_list_next(iter)) {
+TBStatistics *tbs = iter->data;
+if (tbs && tbs->pc == pc) {
+return tbs;
+}
+}
+return NULL;
+}
+
 struct jit_profile_info {
 uint64_t translations;
 uint64_t aborted;
@@ -175,6 +201,7 @@ static void clean_tbstats(void)
 qht_destroy(_ctx.tb_stats);
 }
 
+
 void do_hmp_tbstats_safe(CPUState *cpu, run_on_cpu_data icmd)
 {
 struct TbstatsCommand *cmdinfo = icmd.host_ptr;
@@ -261,6 +288,398 @@ void init_tb_stats_htable_if_not(void)
 }
 }
 
+static void collect_tb_stats(void *p, uint32_t hash, void *userp)
+{
+last_search = g_list_prepend(last_search, p);
+}
+
+static void dump_tb_targets(TBStatistics *tbs)
+{
+if (tbs && tbs->tb) {
+TBStatistics *valid_target_tbstats[2] = {NULL, NULL};
+
+/*
+ * Check and ensure that targets' tbstats have a valid display_id and
+ * are in last_search list
+ */
+for (int jmp_id = 0; jmp_id < 2; jmp_id++) {
+qemu_spin_lock(>tb->jmp_lock);
+TranslationBlock *tb_dst =
+(TranslationBlock *) (atomic_read(>tb->jmp_dest[jmp_id]) 
& ~1);
+

[Qemu-devel] [PATCH v3 4/4] docs: split the CODING_STYLE doc into distinct groups

[Daniel P . Berrangé]

Signed-off-by: Daniel P. Berrangé 
---
 CODING_STYLE.rst | 9 +
 1 file changed, 9 insertions(+)

diff --git a/CODING_STYLE.rst b/CODING_STYLE.rst
index 39397f0f6f..427699e0e4 100644
--- a/CODING_STYLE.rst
+++ b/CODING_STYLE.rst
@@ -7,6 +7,9 @@ QEMU Coding Style
 Please use the script checkpatch.pl in the scripts directory to check
 patches before submitting.
 
+Formatting and style
+
+
 Whitespace
 ==
 
@@ -205,6 +208,9 @@ comment anyway.)
 Rationale: Consistency, and ease of visually picking out a multiline
 comment from the surrounding code.
 
+Language usage
+**
+
 Preprocessor
 
 
@@ -526,6 +532,9 @@ are still some caveats to beware of
 }
 
 
+QEMU Specific Idioms
+
+
 Error handling and reporting
 
 
-- 
2.21.0

Re: [Qemu-devel] [PATCH v2 1/4] docs: convert README, CODING_STYLE and HACKING to RST syntax

[Alex Bennée]

Daniel P. Berrangé  writes:

> Signed-off-by: Daniel P. Berrangé 

Reviewed-by: Alex Bennée 

> ---
>  CODING_STYLE => CODING_STYLE.rst | 121 +++---
>  HACKING => HACKING.rst   | 123 +--
>  README => README.rst |  47 +++-
>  3 files changed, 190 insertions(+), 101 deletions(-)
>  rename CODING_STYLE => CODING_STYLE.rst (72%)
>  rename HACKING => HACKING.rst (79%)
>  rename README => README.rst (84%)
>
> diff --git a/CODING_STYLE b/CODING_STYLE.rst
> similarity index 72%
> rename from CODING_STYLE
> rename to CODING_STYLE.rst
> index cb8edcbb36..713357cb80 100644
> --- a/CODING_STYLE
> +++ b/CODING_STYLE.rst
> @@ -1,10 +1,14 @@
> +=
>  QEMU Coding Style
>  =
>
> +.. contents:: Table of Contents
> +
>  Please use the script checkpatch.pl in the scripts directory to check
>  patches before submitting.
>
> -1. Whitespace
> +Whitespace
> +==
>
>  Of course, the most important aspect in any coding style is whitespace.
>  Crusty old coders who have trouble spotting the glasses on their noses
> @@ -16,26 +20,27 @@ QEMU indents are four spaces.  Tabs are never used, 
> except in Makefiles
>  where they have been irreversibly coded into the syntax.
>  Spaces of course are superior to tabs because:
>
> - - You have just one way to specify whitespace, not two.  Ambiguity breeds
> -   mistakes.
> - - The confusion surrounding 'use tabs to indent, spaces to justify' is gone.
> - - Tab indents push your code to the right, making your screen seriously
> -   unbalanced.
> - - Tabs will be rendered incorrectly on editors who are misconfigured not
> -   to use tab stops of eight positions.
> - - Tabs are rendered badly in patches, causing off-by-one errors in almost
> -   every line.
> - - It is the QEMU coding style.
> +* You have just one way to specify whitespace, not two.  Ambiguity breeds
> +  mistakes.
> +* The confusion surrounding 'use tabs to indent, spaces to justify' is gone.
> +* Tab indents push your code to the right, making your screen seriously
> +  unbalanced.
> +* Tabs will be rendered incorrectly on editors who are misconfigured not
> +  to use tab stops of eight positions.
> +* Tabs are rendered badly in patches, causing off-by-one errors in almost
> +  every line.
> +* It is the QEMU coding style.
>
>  Do not leave whitespace dangling off the ends of lines.
>
> -1.1 Multiline Indent
> +Multiline Indent
> +
>
>  There are several places where indent is necessary:
>
> - - if/else
> - - while/for
> - - function definition & call
> +* if/else
> +* while/for
> +* function definition & call
>
>  When breaking up a long line to fit within line width, we need a proper 
> indent
>  for the following lines.
> @@ -45,6 +50,8 @@ opening parenthesis of the first.
>
>  For example:
>
> +.. code-block:: c
> +
>  if (a == 1 &&
>  b == 2) {
>
> @@ -53,12 +60,13 @@ For example:
>
>  In case of function, there are several variants:
>
> -* 4 spaces indent from the beginning
> -* align the secondary lines just after the opening parenthesis of the
> -  first
> +* 4 spaces indent from the beginning
> +* align the secondary lines just after the opening parenthesis of the first
>
>  For example:
>
> +.. code-block:: c
> +
>  do_something(x, y,
>  z);
>
> @@ -68,7 +76,8 @@ For example:
>  do_something(x, do_another(y,
> z));
>
> -2. Line width
> +Line width
> +==
>
>  Lines should be 80 characters; try not to make them longer.
>
> @@ -77,16 +86,18 @@ that use long function or symbol names.  Even in that 
> case, do not make
>  lines much longer than 80 characters.
>
>  Rationale:
> - - Some people like to tile their 24" screens with a 6x4 matrix of 80x24
> -   xterms and use vi in all of them.  The best way to punish them is to
> -   let them keep doing it.
> - - Code and especially patches is much more readable if limited to a sane
> -   line length.  Eighty is traditional.
> - - The four-space indentation makes the most common excuse ("But look
> -   at all that white space on the left!") moot.
> - - It is the QEMU coding style.
>
> -3. Naming
> +* Some people like to tile their 24" screens with a 6x4 matrix of 80x24
> +  xterms and use vi in all of them.  The best way to punish them is to
> +  let them keep doing it.
> +* Code and especially patches is much more readable if limited to a sane
> +  line length.  Eighty is traditional.
> +* The four-space indentation makes the most common excuse ("But look
> +  at all that white space on the left!") moot.
> +* It is the QEMU coding style.
> +
> +Naming
> +==
>
>  Variables are lower_case_with_underscores; easy to type and read.  Structured
>  type names are in CamelCase; harder to type but standing out.  Enum type
> @@ -95,10 +106,11 @@ names are lower_case_with_underscores_ending_with_a_t, 
> like the POSIX
>  uint64_t and family.  Note that this last 

[Qemu-devel] [PATCH v7 09/10] monitor: adding new info cfg command

[vandersonmr]

Adding "info cfg id depth" commands to HMP.
This command allow the exploration a TB
neighbors by dumping [and opening] a .dot
file with the TB CFG neighbors colorized
by their hotness.

The goal of this command is to allow the dynamic exploration
of TCG behavior and code quality. Therefore, for now, a
corresponding QMP command is not worthwhile.

Signed-off-by: Vanderson M. do Rosario 
---
 accel/tcg/tb-stats.c| 164 
 hmp-commands-info.hx|   7 ++
 include/exec/tb-stats.h |   1 +
 monitor/misc.c  |  22 ++
 4 files changed, 194 insertions(+)

diff --git a/accel/tcg/tb-stats.c b/accel/tcg/tb-stats.c
index d588c551c9..fd2344c5d1 100644
--- a/accel/tcg/tb-stats.c
+++ b/accel/tcg/tb-stats.c
@@ -679,6 +679,170 @@ void dump_tb_info(int id, int log_mask, bool use_monitor)
 /* tbdi free'd by do_dump_tb_info_safe */
 }
 
+/* TB CFG xdot/dot dump implementation */
+#define MAX_CFG_NUM_NODES 1000
+static int cfg_tb_id;
+static GHashTable *cfg_nodes;
+static uint64_t root_count;
+
+static void fputs_jump(TBStatistics *from, TBStatistics *to, FILE *dot)
+{
+if (!from || !to) {
+return;
+}
+
+int *from_id = (int *) g_hash_table_lookup(cfg_nodes, from);
+int *to_id   = (int *) g_hash_table_lookup(cfg_nodes, to);
+
+if (!from_id || !to_id) {
+return;
+}
+
+fprintf(dot, "   node_%d -> node_%d;\n", *from_id, *to_id);
+}
+
+/* Hotness colors used in the CFG */
+#define HOT_RED1  0xFF000 /* RGB(255,0,0) */
+#define HOT_RED2  0xFF333 /* RGB(255,51,48)   */
+#define MILD_RED  0xFF666 /* RGB(255,102,96)  */
+#define WEAK_RED1 0xFF999 /* RGB(255,153,144) */
+#define WEAK_RED2 0xFFCCC /* RGB(255,204,192) */
+
+static void fputs_tbstats(TBStatistics *tbs, FILE *dot, int log_flags)
+{
+if (!tbs) {
+return;
+}
+
+uint32_t color = MILD_RED;
+uint64_t count = tbs->executions.normal;
+if (count > 1.6 * root_count) {
+color = HOT_RED1;
+} else if (count > 1.2 * root_count) {
+color = HOT_RED2;
+} else if (count < 0.4 * root_count) {
+color = WEAK_RED2;
+} else if (count < 0.8 * root_count) {
+color = WEAK_RED1;
+}
+
+GString *code_s = get_code_string(tbs, log_flags);
+
+for (int i = 0; i < code_s->len; i++) {
+if (code_s->str[i] == '\n') {
+code_s->str[i] = ' ';
+code_s = g_string_insert(code_s, i, "\\l");
+i += 2;
+}
+}
+
+fprintf(dot,
+"   node_%d [fillcolor=\"#%xFF\" shape=\"record\" "
+"label=\"TB %d\\l"
+"-\\l"
+"PC:\t0x"TARGET_FMT_lx"\\l"
+"exec count:\t%lu\\l"
+"\\l %s\"];\n",
+cfg_tb_id, color, cfg_tb_id, tbs->pc,
+tbs->executions.normal, code_s->str);
+
+int *id = g_new(int, 1);
+*id = cfg_tb_id;
+g_hash_table_insert(cfg_nodes, tbs, id);
+
+cfg_tb_id++;
+
+g_string_free(code_s, true);
+}
+
+static void fputs_preorder_walk(TBStatistics *tbs, int depth, FILE *dot, int 
log_flags)
+{
+if (tbs && depth > 0
+&& cfg_tb_id < MAX_CFG_NUM_NODES
+&& !g_hash_table_contains(cfg_nodes, tbs)) {
+
+fputs_tbstats(tbs, dot, log_flags);
+
+if (tbs->tb) {
+TranslationBlock *left_tb  = NULL;
+TranslationBlock *right_tb = NULL;
+if (tbs->tb->jmp_dest[0]) {
+left_tb = (TranslationBlock *) atomic_read(tbs->tb->jmp_dest);
+}
+if (tbs->tb->jmp_dest[1]) {
+right_tb = (TranslationBlock *) atomic_read(tbs->tb->jmp_dest 
+ 1);
+}
+
+if (left_tb) {
+fputs_preorder_walk(left_tb->tb_stats, depth - 1, dot, 
log_flags);
+fputs_jump(tbs, left_tb->tb_stats, dot);
+}
+if (right_tb) {
+fputs_preorder_walk(right_tb->tb_stats, depth - 1, dot, 
log_flags);
+fputs_jump(tbs, right_tb->tb_stats, dot);
+}
+}
+}
+}
+
+struct PreorderInfo {
+TBStatistics *tbs;
+int depth;
+int log_flags;
+};
+
+static void fputs_preorder_walk_safe(CPUState *cpu, run_on_cpu_data icmd)
+{
+struct PreorderInfo *info = icmd.host_ptr;
+
+GString *file_name = g_string_new(NULL);;
+g_string_printf(file_name, "/tmp/qemu-cfg-tb-%d-%d.dot", id, info->depth);
+FILE *dot = fopen(file_name->str, "w+");
+
+fputs(
+"digraph G {\n"
+"   mclimit=1.5;\n"
+"   rankdir=TD; ordering=out;\n"
+"   graph[fontsize=10 fontname=\"Verdana\"];\n"
+"   color=\"#efefef\";\n"
+"   node[shape=box style=filled fontsize=8 fontname=\"Verdana\" 
fillcolor=\"#efefef\"];\n"
+"   edge[fontsize=8 fontname=\"Verdana\"];\n"
+ , dot);
+
+cfg_nodes = g_hash_table_new(NULL, NULL);
+fputs_preorder_walk(info->tbs, info->depth, dot, info->log_flags);
+

[Qemu-devel] [PATCH v3 3/4] docs: document use of automatic cleanup functions in glib

[Daniel P . Berrangé]

Document the use of g_autofree and g_autoptr in glib for automatic
freeing of memory.

Signed-off-by: Daniel P. Berrangé 
---
 CODING_STYLE.rst | 85 
 1 file changed, 85 insertions(+)

diff --git a/CODING_STYLE.rst b/CODING_STYLE.rst
index 4501d87352..39397f0f6f 100644
--- a/CODING_STYLE.rst
+++ b/CODING_STYLE.rst
@@ -441,6 +441,91 @@ In addition, QEMU assumes that the compiler does not use 
the latitude
 given in C99 and C11 to treat aspects of signed '<<' as undefined, as
 documented in the GNU Compiler Collection manual starting at version 4.0.
 
+Automatic memory deallocation
+=
+
+QEMU has a mandatory dependency either the GCC or CLang compiler. As
+such it has the freedom to make use of a C language extension for
+automatically running a cleanup function when a stack variable goes
+out of scope. This can be used to simplify function cleanup paths,
+often allowing many goto jumps to be eliminated, through automatic
+free'ing of memory.
+
+The GLib2 library provides a number of functions/macros for enabling
+automatic cleanup:
+
+  ``_
+
+Most notably:
+
+* g_autofree - will invoke g_free() on the variable going out of scope
+
+* g_autoptr - for structs / objects, will invoke the cleanup func created
+  by a previous use of G_DEFINE_AUTOPTR_CLEANUP_FUNC. This is
+  supported for most GLib data types and GObjects
+
+For example, instead of
+
+.. code-block:: c
+
+int somefunc(void) {
+int ret = -1;
+char *foo = g_strdup_printf("foo%", "wibble");
+GList *bar = .
+
+if (eek) {
+   goto cleanup;
+}
+
+ret = 0;
+
+  cleanup:
+g_free(foo);
+g_list_free(bar);
+return ret;
+}
+
+Using g_autofree/g_autoptr enables the code to be written as:
+
+.. code-block:: c
+
+int somefunc(void) {
+g_autofree char *foo = g_strdup_printf("foo%", "wibble");
+g_autoptr (GList) bar = .
+
+if (eek) {
+   return -1;
+}
+
+return 0;
+}
+
+While this generally results in simpler, less leak-prone code, there
+are still some caveats to beware of
+
+* Variables declared with g_auto* MUST always be initialized,
+  otherwise the cleanup function will use uninitialized stack memory
+
+* If a variable declared with g_auto* holds a value which must
+  live beyond the life of the function, that value must be saved
+  and the original variable NULL'd out. This can be simpler using
+  g_steal_pointer
+
+
+.. code-block:: c
+
+char *somefunc(void) {
+g_autofree char *foo = g_strdup_printf("foo%", "wibble");
+g_autoptr (GList) bar = .
+
+if (eek) {
+   return NULL;
+}
+
+return g_steal_pointer();
+}
+
+
 Error handling and reporting
 
 
-- 
2.21.0

[Qemu-devel] [PATCH v7 06/10] monitor: adding tb_stats hmp command

[vandersonmr]

Adding tb_stats [start|pause|stop|filter] command to hmp.
This allows controlling the collection of statistics.
It is also possible to set the level of collection:
all, jit, or exec.

tb_stats filter allow to only collect statistics for the TB
in the last_search list.

The goal of this command is to allow the dynamic exploration
of the TCG behavior and quality. Therefore, for now, a
corresponding QMP command is not worthwhile.

Acked-by: Dr. David Alan Gilbert 
Signed-off-by: Vanderson M. do Rosario 
---
 accel/tcg/tb-stats.c  | 110 ++
 hmp-commands.hx   |  17 ++
 include/exec/tb-stats-flags.h |   1 +
 include/exec/tb-stats.h   |  11 
 monitor/misc.c|  47 +++
 vl.c  |   6 ++
 6 files changed, 192 insertions(+)

diff --git a/accel/tcg/tb-stats.c b/accel/tcg/tb-stats.c
index 66abc97ad4..9959477fbb 100644
--- a/accel/tcg/tb-stats.c
+++ b/accel/tcg/tb-stats.c
@@ -32,6 +32,9 @@
 
 #include "exec/tb-stats.h"
 
+/* only accessed in safe work */
+static GList *last_search;
+
 uint64_t dev_time;
 
 struct jit_profile_info {
@@ -160,6 +163,95 @@ void dump_jit_profile_info(TCGProfile *s)
 g_free(jpi);
 }
 
+static void free_tbstats(void *p, uint32_t hash, void *userp)
+{
+g_free(p);
+}
+
+static void clean_tbstats(void)
+{
+/* remove all tb_stats */
+qht_iter(_ctx.tb_stats, free_tbstats, NULL);
+qht_destroy(_ctx.tb_stats);
+}
+
+void do_hmp_tbstats_safe(CPUState *cpu, run_on_cpu_data icmd)
+{
+struct TbstatsCommand *cmdinfo = icmd.host_ptr;
+int cmd = cmdinfo->cmd;
+uint32_t level = cmdinfo->level;
+
+switch (cmd) {
+case START:
+if (tb_stats_collection_paused()) {
+set_tbstats_flags(level);
+} else {
+if (tb_stats_collection_enabled()) {
+qemu_printf("TB information already being recorded");
+return;
+}
+qht_init(_ctx.tb_stats, tb_stats_cmp, CODE_GEN_HTABLE_SIZE,
+QHT_MODE_AUTO_RESIZE);
+}
+
+set_default_tbstats_flag(level);
+enable_collect_tb_stats();
+tb_flush(cpu);
+break;
+case PAUSE:
+if (!tb_stats_collection_enabled()) {
+qemu_printf("TB information not being recorded");
+return;
+}
+
+/* Continue to create TBStatistic structures but stop collecting 
statistics */
+pause_collect_tb_stats();
+set_default_tbstats_flag(TB_NOTHING);
+set_tbstats_flags(TB_PAUSED);
+tb_flush(cpu);
+break;
+case STOP:
+if (!tb_stats_collection_enabled()) {
+qemu_printf("TB information not being recorded");
+return;
+}
+
+/* Dissalloc all TBStatistics structures and stop creating new ones */
+disable_collect_tb_stats();
+clean_tbstats();
+tb_flush(cpu);
+break;
+case FILTER:
+if (!tb_stats_collection_enabled()) {
+qemu_printf("TB information not being recorded");
+return;
+}
+if (!last_search) {
+qemu_printf("no search on record! execute info tbs before 
filtering!");
+return;
+}
+
+set_default_tbstats_flag(TB_NOTHING);
+
+/* Set all tbstats as paused, then return only the ones from 
last_search */
+pause_collect_tb_stats();
+set_tbstats_flags(TB_PAUSED);
+
+for (GList *iter = last_search; iter; iter = g_list_next(iter)) {
+TBStatistics *tbs = iter->data;
+tbs->stats_enabled = level;
+}
+
+tb_flush(cpu);
+
+break;
+default: /* INVALID */
+g_assert_not_reached();
+break;
+}
+
+g_free(cmdinfo);
+}
 
 void init_tb_stats_htable_if_not(void)
 {
@@ -195,6 +287,24 @@ bool tb_stats_collection_paused(void)
 return tcg_collect_tb_stats == TB_STATS_PAUSED;
 }
 
+static void reset_tbstats_flag(void *p, uint32_t hash, void *userp)
+{
+uint32_t flag = *((int *)userp);
+TBStatistics *tbs = p;
+tbs->stats_enabled = flag;
+}
+
+void set_default_tbstats_flag(uint32_t flag)
+{
+default_tbstats_flag = flag;
+}
+
+void set_tbstats_flags(uint32_t flag)
+{
+/* iterate over tbstats setting their flag as TB_NOTHING */
+qht_iter(_ctx.tb_stats, reset_tbstats_flag, );
+}
+
 uint32_t get_default_tbstats_flag(void)
 {
 return default_tbstats_flag;
diff --git a/hmp-commands.hx b/hmp-commands.hx
index cfcc044ce4..6cd2800378 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1886,6 +1886,23 @@ STEXI
 @findex qemu-io
 Executes a qemu-io command on the given block device.
 
+ETEXI
+#if defined(CONFIG_TCG)
+{
+.name   = "tb_stats",
+.args_type  = "command:s,level:s?",
+.params = "command [stats_level]",
+.help   = "Control tb statistics collection:"
+"tb_stats 

[Qemu-devel] [PATCH v3 1/4] docs: convert README, CODING_STYLE and HACKING to RST syntax

[Daniel P . Berrangé]

Signed-off-by: Daniel P. Berrangé 
---
 CODING_STYLE => CODING_STYLE.rst | 121 +++---
 HACKING => HACKING.rst   | 123 +--
 README => README.rst |  47 +++-
 scripts/checkpatch.pl|   2 +-
 4 files changed, 191 insertions(+), 102 deletions(-)
 rename CODING_STYLE => CODING_STYLE.rst (72%)
 rename HACKING => HACKING.rst (79%)
 rename README => README.rst (84%)

diff --git a/CODING_STYLE b/CODING_STYLE.rst
similarity index 72%
rename from CODING_STYLE
rename to CODING_STYLE.rst
index cb8edcbb36..713357cb80 100644
--- a/CODING_STYLE
+++ b/CODING_STYLE.rst
@@ -1,10 +1,14 @@
+=
 QEMU Coding Style
 =
 
+.. contents:: Table of Contents
+
 Please use the script checkpatch.pl in the scripts directory to check
 patches before submitting.
 
-1. Whitespace
+Whitespace
+==
 
 Of course, the most important aspect in any coding style is whitespace.
 Crusty old coders who have trouble spotting the glasses on their noses
@@ -16,26 +20,27 @@ QEMU indents are four spaces.  Tabs are never used, except 
in Makefiles
 where they have been irreversibly coded into the syntax.
 Spaces of course are superior to tabs because:
 
- - You have just one way to specify whitespace, not two.  Ambiguity breeds
-   mistakes.
- - The confusion surrounding 'use tabs to indent, spaces to justify' is gone.
- - Tab indents push your code to the right, making your screen seriously
-   unbalanced.
- - Tabs will be rendered incorrectly on editors who are misconfigured not
-   to use tab stops of eight positions.
- - Tabs are rendered badly in patches, causing off-by-one errors in almost
-   every line.
- - It is the QEMU coding style.
+* You have just one way to specify whitespace, not two.  Ambiguity breeds
+  mistakes.
+* The confusion surrounding 'use tabs to indent, spaces to justify' is gone.
+* Tab indents push your code to the right, making your screen seriously
+  unbalanced.
+* Tabs will be rendered incorrectly on editors who are misconfigured not
+  to use tab stops of eight positions.
+* Tabs are rendered badly in patches, causing off-by-one errors in almost
+  every line.
+* It is the QEMU coding style.
 
 Do not leave whitespace dangling off the ends of lines.
 
-1.1 Multiline Indent
+Multiline Indent
+
 
 There are several places where indent is necessary:
 
- - if/else
- - while/for
- - function definition & call
+* if/else
+* while/for
+* function definition & call
 
 When breaking up a long line to fit within line width, we need a proper indent
 for the following lines.
@@ -45,6 +50,8 @@ opening parenthesis of the first.
 
 For example:
 
+.. code-block:: c
+
 if (a == 1 &&
 b == 2) {
 
@@ -53,12 +60,13 @@ For example:
 
 In case of function, there are several variants:
 
-* 4 spaces indent from the beginning
-* align the secondary lines just after the opening parenthesis of the
-  first
+* 4 spaces indent from the beginning
+* align the secondary lines just after the opening parenthesis of the first
 
 For example:
 
+.. code-block:: c
+
 do_something(x, y,
 z);
 
@@ -68,7 +76,8 @@ For example:
 do_something(x, do_another(y,
z));
 
-2. Line width
+Line width
+==
 
 Lines should be 80 characters; try not to make them longer.
 
@@ -77,16 +86,18 @@ that use long function or symbol names.  Even in that case, 
do not make
 lines much longer than 80 characters.
 
 Rationale:
- - Some people like to tile their 24" screens with a 6x4 matrix of 80x24
-   xterms and use vi in all of them.  The best way to punish them is to
-   let them keep doing it.
- - Code and especially patches is much more readable if limited to a sane
-   line length.  Eighty is traditional.
- - The four-space indentation makes the most common excuse ("But look
-   at all that white space on the left!") moot.
- - It is the QEMU coding style.
 
-3. Naming
+* Some people like to tile their 24" screens with a 6x4 matrix of 80x24
+  xterms and use vi in all of them.  The best way to punish them is to
+  let them keep doing it.
+* Code and especially patches is much more readable if limited to a sane
+  line length.  Eighty is traditional.
+* The four-space indentation makes the most common excuse ("But look
+  at all that white space on the left!") moot.
+* It is the QEMU coding style.
+
+Naming
+==
 
 Variables are lower_case_with_underscores; easy to type and read.  Structured
 type names are in CamelCase; harder to type but standing out.  Enum type
@@ -95,10 +106,11 @@ names are lower_case_with_underscores_ending_with_a_t, 
like the POSIX
 uint64_t and family.  Note that this last convention contradicts POSIX
 and is therefore likely to be changed.
 
-When wrapping standard library functions, use the prefix qemu_ to alert
+When wrapping standard library functions, use the prefix ``qemu_`` to alert
 readers that they are seeing a wrapped 

[Qemu-devel] [PATCH v7 03/10] accel: replacing part of CONFIG_PROFILER with TBStats

[vandersonmr]

We add some of the statistics collected in the TCGProfiler
into the TBStats, having the statistics not only for the whole
emulation but for each TB. Then, we removed these stats
from TCGProfiler and reconstruct the information for the
"info jit" using the sum of all TBStats statistics.

The goal is to have one unique and better way of collecting
emulation statistics. Moreover, checking dynamiclly if the
profiling is enabled showed to have an insignificant impact
on the performance:
https://wiki.qemu.org/Internships/ProjectIdeas/TCGCodeQuality#Overheads.

Reviewed-by: Alex Bennée 
Signed-off-by: Vanderson M. do Rosario 
---
 accel/tcg/tb-stats.c  | 96 +++
 accel/tcg/translate-all.c |  8 +---
 include/exec/tb-stats.h   | 11 +
 tcg/tcg.c | 93 -
 tcg/tcg.h | 10 
 5 files changed, 119 insertions(+), 99 deletions(-)

diff --git a/accel/tcg/tb-stats.c b/accel/tcg/tb-stats.c
index 1db81d83e7..176da60e13 100644
--- a/accel/tcg/tb-stats.c
+++ b/accel/tcg/tb-stats.c
@@ -25,9 +25,105 @@
 #include "qemu/osdep.h"
 
 #include "disas/disas.h"
+#include "exec/exec-all.h"
+#include "tcg.h"
+
+#include "qemu/qemu-print.h"
 
 #include "exec/tb-stats.h"
 
+struct jit_profile_info {
+uint64_t translations;
+uint64_t aborted;
+uint64_t ops;
+unsigned ops_max;
+uint64_t del_ops;
+uint64_t temps;
+unsigned temps_max;
+uint64_t host;
+uint64_t guest;
+uint64_t search_data;
+};
+
+/* accumulate the statistics from all TBs */
+static void collect_jit_profile_info(void *p, uint32_t hash, void *userp)
+{
+struct jit_profile_info *jpi = userp;
+TBStatistics *tbs = p;
+
+jpi->translations += tbs->translations.total;
+jpi->ops += tbs->code.num_tcg_ops;
+if (stat_per_translation(tbs, code.num_tcg_ops) > jpi->ops_max) {
+jpi->ops_max = stat_per_translation(tbs, code.num_tcg_ops);
+}
+jpi->del_ops += tbs->code.deleted_ops;
+jpi->temps += tbs->code.temps;
+if (stat_per_translation(tbs, code.temps) > jpi->temps_max) {
+jpi->temps_max = stat_per_translation(tbs, code.temps);
+}
+jpi->host += tbs->code.out_len;
+jpi->guest += tbs->code.in_len;
+jpi->search_data += tbs->code.search_out_len;
+}
+
+/* dump JIT statisticis using TCGProfile and TBStats */
+void dump_jit_profile_info(TCGProfile *s)
+{
+if (!tb_stats_collection_enabled()) {
+return;
+}
+
+struct jit_profile_info *jpi = g_new0(struct jit_profile_info, 1);
+
+qht_iter(_ctx.tb_stats, collect_jit_profile_info, jpi);
+
+if (jpi->translations) {
+qemu_printf("translated TBs  %" PRId64 "\n", jpi->translations);
+qemu_printf("avg ops/TB  %0.1f max=%d\n",
+jpi->ops / (double) jpi->translations, jpi->ops_max);
+qemu_printf("deleted ops/TB  %0.2f\n",
+jpi->del_ops / (double) jpi->translations);
+qemu_printf("avg temps/TB%0.2f max=%d\n",
+jpi->temps / (double) jpi->translations, jpi->temps_max);
+qemu_printf("avg host code/TB%0.1f\n",
+jpi->host / (double) jpi->translations);
+qemu_printf("avg search data/TB  %0.1f\n",
+jpi->search_data / (double) jpi->translations);
+
+if (s) {
+int64_t tot = s->interm_time + s->code_time;
+qemu_printf("JIT cycles  %" PRId64 " (%0.3f s at 2.4 
GHz)\n",
+tot, tot / 2.4e9);
+qemu_printf("cycles/op   %0.1f\n",
+jpi->ops ? (double)tot / jpi->ops : 0);
+qemu_printf("cycles/in byte  %0.1f\n",
+jpi->guest ? (double)tot / jpi->guest : 0);
+qemu_printf("cycles/out byte %0.1f\n",
+jpi->host ? (double)tot / jpi->host : 0);
+qemu_printf("cycles/search byte %0.1f\n",
+jpi->search_data ? (double)tot / jpi->search_data : 0);
+if (tot == 0) {
+tot = 1;
+}
+qemu_printf("  gen_interm time   %0.1f%%\n",
+(double)s->interm_time / tot * 100.0);
+qemu_printf("  gen_code time %0.1f%%\n",
+(double)s->code_time / tot * 100.0);
+qemu_printf("optim./code time%0.1f%%\n",
+(double)s->opt_time / (s->code_time ? s->code_time : 1)
+* 100.0);
+qemu_printf("liveness/code time  %0.1f%%\n",
+(double)s->la_time / (s->code_time ? s->code_time : 1) * 
100.0);
+qemu_printf("cpu_restore count   %" PRId64 "\n",
+s->restore_count);
+qemu_printf("  avg cycles%0.1f\n",
+s->restore_count ? (double)s->restore_time / 
s->restore_count : 0);
+}
+}
+g_free(jpi);
+}
+
+
 void 

[Qemu-devel] [PATCH v7 07/10] Adding tb_stats [start|pause|stop|filter] command to hmp.

[vandersonmr]

This allows controlling the collection of statistics.
It is also possible to set the level of collection:
all, jit, or exec.

tb_stats filter allow to only collect statistics for the TB
in the last_search list.

The goal of this command is to allow the dynamic exploration
of the TCG behavior and quality. Therefore, for now, a
corresponding QMP command is not worthwhile.

Acked-by: Dr. David Alan Gilbert 
Signed-off-by: Vanderson M. do Rosario 
---
 monitor/misc.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/monitor/misc.c b/monitor/misc.c
index b389ca09a1..218263d29a 100644
--- a/monitor/misc.c
+++ b/monitor/misc.c
@@ -74,6 +74,8 @@
 #include "sysemu/cpus.h"
 #include "qemu/cutils.h"
 #include "tcg/tcg.h"
+#include "exec/tb-stats.h"
+#include "qemu-common.h"
 
 #if defined(TARGET_S390X)
 #include "hw/s390x/storage-keys.h"
-- 
2.22.0

[Qemu-devel] [PATCH v7 02/10] accel: collecting JIT statistics

[vandersonmr]

If a TB has a TBS (TBStatistics) with the TB_JIT_STATS
enabled then we collect statistics of its translation
processes and code translation.

Collecting the number of host instructions seems to be
not simple as it would imply in having to modify several
target source files. So, for now, we are only collecting
the size of the host gen code.

Reviewed-by: Alex Bennée 
Signed-off-by: Vanderson M. do Rosario 
---
 accel/tcg/translate-all.c | 15 ++-
 accel/tcg/translator.c|  4 
 include/exec/tb-stats.h   | 15 +++
 tcg/tcg.c | 23 +++
 tcg/tcg.h |  2 ++
 5 files changed, 58 insertions(+), 1 deletion(-)

diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index e72aeba682..fb2fe0fa1f 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -1705,6 +1705,7 @@ static TBStatistics *tb_get_stats(tb_page_addr_t phys_pc, 
target_ulong pc,
  * then just make the new TB point to the older TBStatistic
  */
 g_free(new_stats);
+((TBStatistics *) existing_stats)->tb = current_tb;
 return existing_stats;
 } else {
 return new_stats;
@@ -1785,13 +1786,18 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
  */
 if (tb_stats_collection_enabled()) {
 tb->tb_stats = tb_get_stats(phys_pc, pc, cs_base, flags, tb);
+uint32_t flag = get_default_tbstats_flag();
 
 if (qemu_log_in_addr_range(tb->pc)) {
-uint32_t flag = get_default_tbstats_flag();
 if (flag & TB_EXEC_STATS) {
 tb->tb_stats->stats_enabled |= TB_EXEC_STATS;
 }
 }
+
+if (flag & TB_JIT_STATS) {
+tb->tb_stats->stats_enabled |= TB_JIT_STATS;
+atomic_inc(>tb_stats->translations.total);
+}
 } else {
 tb->tb_stats = NULL;
 }
@@ -1869,6 +1875,10 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
 atomic_set(>search_out_len, prof->search_out_len + search_size);
 #endif
 
+if (tb_stats_enabled(tb, TB_JIT_STATS)) {
+atomic_add(>tb_stats->code.out_len, gen_code_size);
+}
+
 #ifdef DEBUG_DISAS
 if (qemu_loglevel_mask(CPU_LOG_TB_OUT_ASM) &&
 qemu_log_in_addr_range(tb->pc)) {
@@ -1926,6 +1936,9 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
 phys_page2 = -1;
 if ((pc & TARGET_PAGE_MASK) != virt_page2) {
 phys_page2 = get_page_addr_code(env, virt_page2);
+if (tb_stats_enabled(tb, TB_JIT_STATS)) {
+atomic_inc(>tb_stats->translations.spanning);
+}
 }
 /*
  * No explicit memory barrier is required -- tb_link_page() makes the
diff --git a/accel/tcg/translator.c b/accel/tcg/translator.c
index ec6bd829a0..9b2e248b09 100644
--- a/accel/tcg/translator.c
+++ b/accel/tcg/translator.c
@@ -116,6 +116,10 @@ void translator_loop(const TranslatorOps *ops, 
DisasContextBase *db,
 db->tb->size = db->pc_next - db->pc_first;
 db->tb->icount = db->num_insns;
 
+if (tb_stats_enabled(tb, TB_JIT_STATS)) {
+atomic_add(>tb->tb_stats->code.num_guest_inst, db->num_insns);
+}
+
 #ifdef DEBUG_DISAS
 if (qemu_loglevel_mask(CPU_LOG_TB_IN_ASM)
 && qemu_log_in_addr_range(db->pc_first)) {
diff --git a/include/exec/tb-stats.h b/include/exec/tb-stats.h
index c4a8715400..b68edd5d24 100644
--- a/include/exec/tb-stats.h
+++ b/include/exec/tb-stats.h
@@ -58,6 +58,20 @@ struct TBStatistics {
 unsigned long atomic;
 } executions;
 
+struct {
+unsigned num_guest_inst;
+unsigned num_tcg_ops;
+unsigned num_tcg_ops_opt;
+unsigned spills;
+unsigned out_len;
+} code;
+
+struct {
+unsigned long total;
+unsigned long uncached;
+unsigned long spanning;
+} translations;
+
 /* current TB linked to this TBStatistics */
 TranslationBlock *tb;
 };
@@ -71,6 +85,7 @@ enum TBStatsStatus { TB_STATS_RUNNING, TB_STATS_PAUSED, 
TB_STATS_STOPPED };
 
 #define TB_NOTHING(1 << 0)
 #define TB_EXEC_STATS (1 << 1)
+#define TB_JIT_STATS  (1 << 2)
 
 extern int tcg_collect_tb_stats;
 extern uint32_t default_tbstats_flag;
diff --git a/tcg/tcg.c b/tcg/tcg.c
index 0458eaec57..ae3e7a2217 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -3125,6 +3125,11 @@ static void temp_sync(TCGContext *s, TCGTemp *ts, 
TCGRegSet allocated_regs,
 case TEMP_VAL_REG:
 tcg_out_st(s, ts->type, ts->reg,
ts->mem_base->reg, ts->mem_offset);
+
+/* Count number of spills */
+if (tb_stats_enabled(s->current_tb, TB_JIT_STATS)) {
+atomic_inc(>current_tb->tb_stats->code.spills);
+}
 break;
 
 case TEMP_VAL_MEM:
@@ -3996,6 +4001,8 @@ int tcg_gen_code(TCGContext *s, TranslationBlock *tb)
 int i, num_insns;
 TCGOp *op;
 
+s->current_tb = tb;
+
 #ifdef CONFIG_PROFILER
 {
 int n = 0;
@@ -4027,6 +4034,14 @@ int tcg_gen_code(TCGContext 

[Qemu-devel] [PATCH v7 05/10] Adding -d tb_stats to control TBStatistics collection:

[vandersonmr]

 -d tb_stats[[,level=(+all+jit+exec+time)][,dump_limit=]]

"dump_limit" is used to limit the number of dumped TBStats in
linux-user mode.

[all+jit+exec+time] control the profilling level used
by the TBStats. Can be used as follow:

-d tb_stats
-d tb_stats,level=jit+time
-d tb_stats,dump_limit=15
...

Signed-off-by: Vanderson M. do Rosario 
---
 include/exec/gen-icount.h |  1 +
 include/exec/tb-stats-flags.h | 42 +++
 include/exec/tb-stats.h   | 18 +++
 include/qemu/log.h|  1 +
 util/log.c| 35 +
 5 files changed, 82 insertions(+), 15 deletions(-)
 create mode 100644 include/exec/tb-stats-flags.h

diff --git a/include/exec/gen-icount.h b/include/exec/gen-icount.h
index be006383b9..3987adfb0e 100644
--- a/include/exec/gen-icount.h
+++ b/include/exec/gen-icount.h
@@ -2,6 +2,7 @@
 #define GEN_ICOUNT_H
 
 #include "qemu/timer.h"
+#include "tb-stats-flags.h"
 
 /* Helpers for instruction counting code generation.  */
 
diff --git a/include/exec/tb-stats-flags.h b/include/exec/tb-stats-flags.h
new file mode 100644
index 00..c936ac1084
--- /dev/null
+++ b/include/exec/tb-stats-flags.h
@@ -0,0 +1,42 @@
+/*
+ * QEMU System Emulator, Code Quality Monitor System
+ *
+ * Copyright (c) 2019 Vanderson M. do Rosario 
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to 
deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+#ifndef TB_STATS_FLAGS
+#define TB_STATS_FLAGS
+
+enum TBStatsStatus {
+TB_STATS_DISABLED = 0,
+TB_STATS_RUNNING,
+TB_STATS_PAUSED,
+TB_STATS_STOPPED
+};
+
+#define TB_NOTHING(1 << 0)
+#define TB_EXEC_STATS (1 << 1)
+#define TB_JIT_STATS  (1 << 2)
+#define TB_JIT_TIME   (1 << 3)
+
+extern int tcg_collect_tb_stats;
+extern uint32_t default_tbstats_flag;
+
+#endif
diff --git a/include/exec/tb-stats.h b/include/exec/tb-stats.h
index 0b9a6e2f72..6a53bef31b 100644
--- a/include/exec/tb-stats.h
+++ b/include/exec/tb-stats.h
@@ -30,6 +30,8 @@
 #include "exec/tb-context.h"
 #include "tcg.h"
 
+#include "exec/tb-stats-flags.h"
+
 #define tb_stats_enabled(tb, JIT_STATS) \
 (tb && tb->tb_stats && (tb->tb_stats->stats_enabled & JIT_STATS))
 
@@ -98,26 +100,12 @@ bool tb_stats_cmp(const void *ap, const void *bp);
 
 void dump_jit_exec_time_info(uint64_t dev_time);
 
+void set_tbstats_flags(uint32_t flags);
 void init_tb_stats_htable_if_not(void);
 
 void dump_jit_profile_info(TCGProfile *s);
 
 /* TBStatistic collection controls */
-enum TBStatsStatus {
-TB_STATS_DISABLED = 0,
-TB_STATS_RUNNING,
-TB_STATS_PAUSED,
-TB_STATS_STOPPED
-};
-
-#define TB_NOTHING(1 << 0)
-#define TB_EXEC_STATS (1 << 1)
-#define TB_JIT_STATS  (1 << 2)
-#define TB_JIT_TIME   (1 << 3)
-
-extern int tcg_collect_tb_stats;
-extern uint32_t default_tbstats_flag;
-
 void enable_collect_tb_stats(void);
 void disable_collect_tb_stats(void);
 void pause_collect_tb_stats(void);
diff --git a/include/qemu/log.h b/include/qemu/log.h
index b097a6cae1..a8d1997cde 100644
--- a/include/qemu/log.h
+++ b/include/qemu/log.h
@@ -45,6 +45,7 @@ static inline bool qemu_log_separate(void)
 /* LOG_TRACE (1 << 15) is defined in log-for-trace.h */
 #define CPU_LOG_TB_OP_IND  (1 << 16)
 #define CPU_LOG_TB_FPU (1 << 17)
+#define CPU_LOG_TB_STATS   (1 << 18)
 
 /* Lock output for a series of related logs.  Since this is not needed
  * for a single qemu_log / qemu_log_mask / qemu_log_mask_and_addr, we
diff --git a/util/log.c b/util/log.c
index 29021a4584..c3805b331b 100644
--- a/util/log.c
+++ b/util/log.c
@@ -19,17 +19,20 @@
 
 #include "qemu/osdep.h"
 #include "qemu/log.h"
+#include "qemu/qemu-print.h"
 #include "qemu/range.h"
 #include "qemu/error-report.h"
 #include "qapi/error.h"
 #include "qemu/cutils.h"
 #include "trace/control.h"
+#include "exec/tb-stats-flags.h"
 
 static char *logfilename;
 FILE *qemu_logfile;
 int qemu_loglevel;
 static int log_append = 0;
 static GArray *debug_regions;
+int32_t max_num_hot_tbs_to_dump;
 
 

[Qemu-devel] [PATCH v7 00/10] Measure Tiny Code Generation Quality

[vandersonmr]

This patch is part of Google Summer of Code (GSoC) 2019.
More about the project can be found in:
https://wiki.qemu.org/Internships/ProjectIdeas/TCGCodeQuality

The goal of this patch is to add infrastructure to collect
execution and JIT statistics during the emulation with accel/TCG.
The statistics are stored in TBStatistic structures (TBStats)
with each TB having its respective TBStats.

We added -d tb_stats and HMP tb_stats commands to allow the control
of this statistics collection. And info tb, tbs, and coverset commands
were also added to allow dumping and exploring all this information
while emulating.

Collecting these statistics and information is useful to understand
qemu performance and to help to add the support for traces to QEMU.

v7:
 - rebase
 - adding license to new files
 - applying comments from v6
v6:
 - applying comments from V5.
 - change info tbs to info tb-list
 - fix crash when dumping tb's targets
 - fix "liveness/code time" calculation
v5:
 - full replacement of CONFIG_PROFILER
 - several fixes
 - adds "info cfg"
 - adds TB's targets to dump

vandersonmr (10):
  accel: collecting TB execution count
  accel: collecting JIT statistics
  accel: replacing part of CONFIG_PROFILER with TBStats
  accel: adding TB_JIT_TIME and full replacing CONFIG_PROFILER
  Adding -d tb_stats to control TBStatistics collection:
  monitor: adding tb_stats hmp command
  Adding tb_stats [start|pause|stop|filter] command to hmp.
  Adding info [tb-list|tb|coverset] commands to HMP.
  monitor: adding new info cfg command
  linux-user: dumping hot TBs at the end of the execution

 accel/tcg/cpu-exec.c  |   4 +
 accel/tcg/tb-stats.c  | 831 ++
 accel/tcg/tcg-runtime.c   |   7 +
 accel/tcg/tcg-runtime.h   |   2 +
 accel/tcg/translate-all.c |  77 ++--
 accel/tcg/translator.c|   5 +
 configure |   3 -
 cpus.c|  14 +-
 disas.c   |  31 +-
 hmp-commands-info.hx  |  31 ++
 hmp-commands.hx   |  17 +
 include/exec/gen-icount.h |  10 +
 include/exec/tb-stats-flags.h |  43 ++
 include/exec/tb-stats.h   | 109 -
 include/qemu/log-for-trace.h  |   4 +
 include/qemu/log.h|   3 +
 include/qemu/timer.h  |   5 +-
 linux-user/exit.c |   4 +
 monitor/misc.c| 171 ++-
 tcg/tcg.c | 230 +++---
 tcg/tcg.h |  22 +-
 util/log.c|  88 +++-
 vl.c  |   8 +-
 23 files changed, 1443 insertions(+), 276 deletions(-)
 create mode 100644 include/exec/tb-stats-flags.h

-- 
2.22.0

[Qemu-devel] [PATCH v7 04/10] accel: adding TB_JIT_TIME and full replacing CONFIG_PROFILER

[vandersonmr]

Replace all others CONFIG_PROFILER statistics and migrate it to
TBStatistics system. However, TCGProfiler still exists and can
be use to store global statistics and times. All TB related
statistics goes to TBStatistics.

Signed-off-by: Vanderson M. do Rosario 
---
 accel/tcg/tb-stats.c  |  91 +++-
 accel/tcg/translate-all.c |  47 ---
 configure |   3 -
 cpus.c|  14 ++---
 include/exec/tb-stats.h   |  19 +-
 include/qemu/timer.h  |   5 +-
 monitor/misc.c|  28 ++---
 tcg/tcg.c | 124 +++---
 tcg/tcg.h |  10 +--
 vl.c  |   8 +--
 10 files changed, 156 insertions(+), 193 deletions(-)

diff --git a/accel/tcg/tb-stats.c b/accel/tcg/tb-stats.c
index 176da60e13..66abc97ad4 100644
--- a/accel/tcg/tb-stats.c
+++ b/accel/tcg/tb-stats.c
@@ -32,6 +32,8 @@
 
 #include "exec/tb-stats.h"
 
+uint64_t dev_time;
+
 struct jit_profile_info {
 uint64_t translations;
 uint64_t aborted;
@@ -43,6 +45,13 @@ struct jit_profile_info {
 uint64_t host;
 uint64_t guest;
 uint64_t search_data;
+
+uint64_t interm_time;
+uint64_t code_time;
+uint64_t restore_count;
+uint64_t restore_time;
+uint64_t opt_time;
+uint64_t la_time;
 };
 
 /* accumulate the statistics from all TBs */
@@ -64,6 +73,29 @@ static void collect_jit_profile_info(void *p, uint32_t hash, 
void *userp)
 jpi->host += tbs->code.out_len;
 jpi->guest += tbs->code.in_len;
 jpi->search_data += tbs->code.search_out_len;
+
+jpi->interm_time += stat_per_translation(tbs, time.interm);
+jpi->code_time += stat_per_translation(tbs, time.code);
+jpi->opt_time += stat_per_translation(tbs, time.opt);
+jpi->la_time += stat_per_translation(tbs, time.la);
+jpi->restore_time += tbs->time.restore;
+jpi->restore_count += tbs->time.restore_count;
+}
+
+void dump_jit_exec_time_info(uint64_t dev_time)
+{
+static uint64_t last_cpu_exec_time;
+uint64_t cpu_exec_time;
+uint64_t delta;
+
+cpu_exec_time = tcg_cpu_exec_time();
+delta = cpu_exec_time - last_cpu_exec_time;
+
+qemu_printf("async time  %" PRId64 " (%0.3f)\n",
+   dev_time, dev_time / (double) NANOSECONDS_PER_SECOND);
+qemu_printf("qemu time   %" PRId64 " (%0.3f)\n",
+   delta, delta / (double) NANOSECONDS_PER_SECOND);
+last_cpu_exec_time = cpu_exec_time;
 }
 
 /* dump JIT statisticis using TCGProfile and TBStats */
@@ -90,34 +122,39 @@ void dump_jit_profile_info(TCGProfile *s)
 qemu_printf("avg search data/TB  %0.1f\n",
 jpi->search_data / (double) jpi->translations);
 
+uint64_t tot = jpi->interm_time + jpi->code_time;
+
+qemu_printf("JIT cycles  %" PRId64 " (%0.3fs at 2.4 GHz)\n",
+tot, tot / 2.4e9);
+qemu_printf("  cycles/op   %0.1f\n",
+jpi->ops ? (double)tot / jpi->ops : 0);
+qemu_printf("  cycles/in byte  %0.1f\n",
+jpi->guest ? (double)tot / jpi->guest : 0);
+qemu_printf("  cycles/out byte %0.1f\n",
+jpi->host ? (double)tot / jpi->host : 0);
+qemu_printf("  cycles/search byte  %0.1f\n",
+jpi->search_data ? (double)tot / jpi->search_data : 0);
+if (tot == 0) {
+tot = 1;
+}
+
+qemu_printf("  gen_interm time %0.1f%%\n",
+(double)jpi->interm_time / tot * 100.0);
+qemu_printf("  gen_code time   %0.1f%%\n",
+(double)jpi->code_time / tot * 100.0);
+
+qemu_printf("optim./code time%0.1f%%\n",
+(double)jpi->opt_time / (jpi->code_time ? jpi->code_time : 1) 
* 100.0);
+qemu_printf("liveness/code time  %0.1f%%\n",
+(double)jpi->la_time / (jpi->code_time ? jpi->code_time : 1) * 
100.0);
+
+qemu_printf("cpu_restore count   %" PRId64 "\n", jpi->restore_count);
+qemu_printf("  avg cycles%0.1f\n",
+jpi->restore_count ? (double)jpi->restore_time / 
jpi->restore_count : 0);
+
 if (s) {
-int64_t tot = s->interm_time + s->code_time;
-qemu_printf("JIT cycles  %" PRId64 " (%0.3f s at 2.4 
GHz)\n",
-tot, tot / 2.4e9);
-qemu_printf("cycles/op   %0.1f\n",
-jpi->ops ? (double)tot / jpi->ops : 0);
-qemu_printf("cycles/in byte  %0.1f\n",
-jpi->guest ? (double)tot / jpi->guest : 0);
-qemu_printf("cycles/out byte %0.1f\n",
-jpi->host ? (double)tot / jpi->host : 0);
-qemu_printf("cycles/search byte %0.1f\n",
-jpi->search_data ? (double)tot / jpi->search_data : 0);
-if (tot == 0) {
-tot = 1;
-}
-qemu_printf("  

[Qemu-devel] [PATCH v7 01/10] accel: collecting TB execution count

[vandersonmr]

If a TB has a TBS (TBStatistics) with the TB_EXEC_STATS
enabled, then we instrument the start code of this TB
to atomically count the number of times it is executed.
We count both the number of "normal" executions and atomic
executions of a TB.

The execution count of the TB is stored in its respective
TBS.

All TBStatistics are created by default with the flags from
default_tbstats_flag.

Signed-off-by: Vanderson M. do Rosario 
---
 accel/tcg/cpu-exec.c  |  4 
 accel/tcg/tb-stats.c  |  5 +
 accel/tcg/tcg-runtime.c   |  7 +++
 accel/tcg/tcg-runtime.h   |  2 ++
 accel/tcg/translate-all.c |  7 +++
 accel/tcg/translator.c|  1 +
 include/exec/gen-icount.h |  9 +
 include/exec/tb-stats.h   | 19 +++
 util/log.c|  1 +
 9 files changed, 55 insertions(+)

diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index 48272c781b..9b2b7bff80 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -251,6 +251,10 @@ void cpu_exec_step_atomic(CPUState *cpu)
 
 start_exclusive();
 
+if (tb_stats_enabled(tb, TB_EXEC_STATS)) {
+tb->tb_stats->executions.atomic++;
+}
+
 /* Since we got here, we know that parallel_cpus must be true.  */
 parallel_cpus = false;
 in_exclusive_region = true;
diff --git a/accel/tcg/tb-stats.c b/accel/tcg/tb-stats.c
index 948b107e68..1db81d83e7 100644
--- a/accel/tcg/tb-stats.c
+++ b/accel/tcg/tb-stats.c
@@ -61,3 +61,8 @@ bool tb_stats_collection_paused(void)
 {
 return tcg_collect_tb_stats == TB_STATS_PAUSED;
 }
+
+uint32_t get_default_tbstats_flag(void)
+{
+return default_tbstats_flag;
+}
diff --git a/accel/tcg/tcg-runtime.c b/accel/tcg/tcg-runtime.c
index 8a1e408e31..6f4aafba11 100644
--- a/accel/tcg/tcg-runtime.c
+++ b/accel/tcg/tcg-runtime.c
@@ -167,3 +167,10 @@ void HELPER(exit_atomic)(CPUArchState *env)
 {
 cpu_loop_exit_atomic(env_cpu(env), GETPC());
 }
+
+void HELPER(inc_exec_freq)(void *ptr)
+{
+TBStatistics *stats = (TBStatistics *) ptr;
+g_assert(stats);
+atomic_inc(>executions.normal);
+}
diff --git a/accel/tcg/tcg-runtime.h b/accel/tcg/tcg-runtime.h
index 4fa61b49b4..bf0b75dbe8 100644
--- a/accel/tcg/tcg-runtime.h
+++ b/accel/tcg/tcg-runtime.h
@@ -28,6 +28,8 @@ DEF_HELPER_FLAGS_1(lookup_tb_ptr, TCG_CALL_NO_WG_SE, ptr, env)
 
 DEF_HELPER_FLAGS_1(exit_atomic, TCG_CALL_NO_WG, noreturn, env)
 
+DEF_HELPER_FLAGS_1(inc_exec_freq, TCG_CALL_NO_RWG, void, ptr)
+
 #ifdef CONFIG_SOFTMMU
 
 DEF_HELPER_FLAGS_5(atomic_cmpxchgb, TCG_CALL_NO_WG,
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index b7bccacd3b..e72aeba682 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -1785,6 +1785,13 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
  */
 if (tb_stats_collection_enabled()) {
 tb->tb_stats = tb_get_stats(phys_pc, pc, cs_base, flags, tb);
+
+if (qemu_log_in_addr_range(tb->pc)) {
+uint32_t flag = get_default_tbstats_flag();
+if (flag & TB_EXEC_STATS) {
+tb->tb_stats->stats_enabled |= TB_EXEC_STATS;
+}
+}
 } else {
 tb->tb_stats = NULL;
 }
diff --git a/accel/tcg/translator.c b/accel/tcg/translator.c
index 70c66c538c..ec6bd829a0 100644
--- a/accel/tcg/translator.c
+++ b/accel/tcg/translator.c
@@ -46,6 +46,7 @@ void translator_loop(const TranslatorOps *ops, 
DisasContextBase *db,
 
 ops->init_disas_context(db, cpu);
 tcg_debug_assert(db->is_jmp == DISAS_NEXT);  /* no early exit */
+gen_tb_exec_count(tb);
 
 /* Reset the temp count so that we can identify leaks */
 tcg_clear_temp_count();
diff --git a/include/exec/gen-icount.h b/include/exec/gen-icount.h
index 822c43cfd3..be006383b9 100644
--- a/include/exec/gen-icount.h
+++ b/include/exec/gen-icount.h
@@ -32,6 +32,15 @@ static inline void gen_io_end(void)
 tcg_temp_free_i32(tmp);
 }
 
+static inline void gen_tb_exec_count(TranslationBlock *tb)
+{
+if (tb_stats_enabled(tb, TB_EXEC_STATS)) {
+TCGv_ptr ptr = tcg_const_ptr(tb->tb_stats);
+gen_helper_inc_exec_freq(ptr);
+tcg_temp_free_ptr(ptr);
+}
+}
+
 static inline void gen_tb_start(TranslationBlock *tb)
 {
 TCGv_i32 count, imm;
diff --git a/include/exec/tb-stats.h b/include/exec/tb-stats.h
index 898e05a36f..c4a8715400 100644
--- a/include/exec/tb-stats.h
+++ b/include/exec/tb-stats.h
@@ -30,6 +30,9 @@
 #include "exec/tb-context.h"
 #include "tcg.h"
 
+#define tb_stats_enabled(tb, JIT_STATS) \
+(tb && tb->tb_stats && (tb->tb_stats->stats_enabled & JIT_STATS))
+
 typedef struct TBStatistics TBStatistics;
 
 /*
@@ -46,6 +49,15 @@ struct TBStatistics {
 uint32_t flags;
 /* cs_base isn't included in the hash but we do check for matches */
 target_ulong cs_base;
+
+uint32_t stats_enabled;
+
+/* Execution stats */
+struct {
+unsigned long normal;
+unsigned long atomic;
+} executions;
+
 /* current TB 

Re: [Qemu-devel] [PATCH v3 56/69] target/arm: Convert T16, extract

[Philippe Mathieu-Daudé]

On 8/28/19 9:04 PM, Richard Henderson wrote:
> Reviewed-by: Peter Maydell 
> Signed-off-by: Richard Henderson 

Reviewed-by: Philippe Mathieu-Daudé 

> ---
>  target/arm/translate.c | 14 +-
>  target/arm/t16.decode  | 10 ++
>  2 files changed, 11 insertions(+), 13 deletions(-)
> 
> diff --git a/target/arm/translate.c b/target/arm/translate.c
> index b7e2c72f35..d06ec48ab9 100644
> --- a/target/arm/translate.c
> +++ b/target/arm/translate.c
> @@ -10743,21 +10743,9 @@ static void disas_thumb_insn(DisasContext *s, 
> uint32_t insn)
>  op = (insn >> 8) & 0xf;
>  switch (op) {
>  case 0: /* add/sub (sp, immediate), in decodetree */
> +case 2: /* sign/zero extend, in decodetree */
>  goto illegal_op;
>  
> -case 2: /* sign/zero extend.  */
> -ARCH(6);
> -rd = insn & 7;
> -rm = (insn >> 3) & 7;
> -tmp = load_reg(s, rm);
> -switch ((insn >> 6) & 3) {
> -case 0: gen_sxth(tmp); break;
> -case 1: gen_sxtb(tmp); break;
> -case 2: gen_uxth(tmp); break;
> -case 3: gen_uxtb(tmp); break;
> -}
> -store_reg(s, rd, tmp);
> -break;
>  case 4: case 5: case 0xc: case 0xd:
>  /*
>   * 0b1011_x10x__
> diff --git a/target/arm/t16.decode b/target/arm/t16.decode
> index b425b86795..b5b5086e8a 100644
> --- a/target/arm/t16.decode
> +++ b/target/arm/t16.decode
> @@ -23,6 +23,7 @@
>  _rrr_shr   !extern s rn rd rm rs shty
>  _rri_rot   !extern s rn rd imm rot
>  _  !extern s rd rn rm ra
> +_rot !extern rd rn rm rot
>!extern rd imm
> !extern rm
>  _rr !extern p w u rn rt rm shimm shtype
> @@ -173,3 +174,12 @@ BX  0100 0111 0  000@branchr
>  BLX_r   0100 0111 1  000@branchr
>  BXNS0100 0111 0  100@branchr
>  BLXNS   0100 0111 1  100@branchr
> +
> +# Extend
> +
> +@extend   .. rm:3 rd:3  _rot rn=15 rot=0
> +
> +SXTAH   1011 0010 00 ... ...@extend
> +SXTAB   1011 0010 01 ... ...@extend
> +UXTAH   1011 0010 10 ... ...@extend
> +UXTAB   1011 0010 11 ... ...@extend
> 

Re: [Qemu-devel] [libvirt] [PATCH 2/2] qapi: deprecate implicit filters

[Christophe de Dinechin]

Markus Armbruster writes:

> Peter Krempa  writes:
>
[...]
>> From my experience users report non-fatal messages mostly only if it is
>> spamming the system log. One of instances are very unlikely to be
>> noticed.
>>
>> In my experience it's better to notify us in libvirt of such change and
>> we will try our best to fix it.
>
> How to best alert the layers above QEMU was one of the topic of the KVM
> Forum 2018 BoF on deprecating stuff.  Minutes:
>
> Message-ID: <87mur0ls8o@dusky.pond.sub.org>
> https://lists.nongnu.org/archive/html/qemu-devel/2018-10/msg05828.html
>
> Relevant part:
>
> * We need to communicate "you're using something that is deprecated".
>   How?  Right now, we print a deprecation message.  Okay when humans use
>   QEMU directly in a shell.  However, when QEMU sits at the bottom of a
>   software stack, the message will likely end up in a log file that is
>   effectively write-only.
>
>   - The one way to get people read log files is crashing their
> application.  A command line option --future could make QEMU crash
> right after printing a deprecation message.  This could help with
> finding use of deprecated features in a testing environment.
>
>   - A less destructive way to grab people's attention is to make things
> run really, really slow: have QEMU go to sleep for a while after
> printing a deprecation message.
>
>   - We can also pass the buck to the next layer up: emit a QMP event.
>
> Sadly, by the time the next layer connects to QMP, plenty of stuff
> already happened.  We'd have to buffer deprecation events somehow.
>
> What would libvirt do with such an event?  Log it, taint the domain,
> emit a (libvirt) event to pass it on to the next layer up.
>
>   - A completely different idea is to have a configuratin linter.  To
> support doing this at the libvirt level, QEMU could expose "is
> deprecated" in interface introspection.  Feels feasible for QMP,
> where we already have sufficiently expressive introspection.  For
> CLI, we'd first have to provide that (but we want that anyway).
>
>   - We might also want to dispay deprecation messages in QEMU's GUI
> somehow, or on serial consoles.

Sorry for catching up late, this mail thread happened during my PTO.

I remember bringing up at the time [1] that the correct solution needs
to take into account usage models that vary from

- a workstation case, where displaying an error box is easy and
  convenient,

- to local headless VMs where system-level notification would do the job
  better, allowing us to leverage things like system-wide email notifications

- to large-scale collections of VMs managed by some layered product,
  where the correct reporting would be through something like Insights,
  i.e. you don't scan individual logs, you want something like "913 VMs
  are using deprecated X"

To me, that implies that we need to have a clear division of roles, with
a standard way to

a) produce the errors,
b) propagate them,
c) consume them (at least up to libvirt)

Notice that this work has already been done for "real" errors,
i.e. there is a real QAPI notion of "errors". AFAICT, warn_report does
not connect to it, though, it goes through error_vprintf which is really
just basic logging.

So would it make sense to:

1. Add a deprecation_report() alongside warn_report()?

2. Connect warn_report() and all the error_vprintf output to QAPI,
   e.g. using John's suggestion of adding the messages using some
   "warning" or "deprecated" tag?

3. Teach libvirt how to consume that new tag and pass it along?


[1] https://lists.nongnu.org/archive/html/qemu-devel/2018-10/msg06131.html


--
Cheers,
Christophe de Dinechin (IRC c3d)

Re: [Qemu-devel] [PATCH v3 64/69] target/arm: Convert T16, shift immediate

[Philippe Mathieu-Daudé]

On 8/28/19 9:04 PM, Richard Henderson wrote:
> Reviewed-by: Peter Maydell 
> Signed-off-by: Richard Henderson 

Reviewed-by: Philippe Mathieu-Daudé 

> ---
>  target/arm/translate.c | 26 ++
>  target/arm/t16.decode  |  8 
>  2 files changed, 10 insertions(+), 24 deletions(-)
> 
> diff --git a/target/arm/translate.c b/target/arm/translate.c
> index 5fb0e2066b..dd292b3042 100644
> --- a/target/arm/translate.c
> +++ b/target/arm/translate.c
> @@ -10731,7 +10731,7 @@ static void disas_thumb2_insn(DisasContext *s, 
> uint32_t insn)
>  
>  static void disas_thumb_insn(DisasContext *s, uint32_t insn)
>  {
> -uint32_t val, op, rm, rd, shift;
> +uint32_t val, rd;
>  int32_t offset;
>  TCGv_i32 tmp;
>  TCGv_i32 tmp2;
> @@ -10743,29 +10743,7 @@ static void disas_thumb_insn(DisasContext *s, 
> uint32_t insn)
>  /* fall back to legacy decoder */
>  
>  switch (insn >> 12) {
> -case 0: case 1:
> -
> -rd = insn & 7;
> -op = (insn >> 11) & 3;
> -if (op == 3) {
> -/*
> - * 0b0001_1xxx__
> - *  - Add, subtract (three low registers)
> - *  - Add, subtract (two low registers and immediate)
> - * In decodetree.
> - */
> -goto illegal_op;
> -} else {
> -/* shift immediate */
> -rm = (insn >> 3) & 7;
> -shift = (insn >> 6) & 0x1f;
> -tmp = load_reg(s, rm);
> -gen_arm_shift_im(tmp, op, shift, s->condexec_mask == 0);
> -if (!s->condexec_mask)
> -gen_logic_CC(tmp);
> -store_reg(s, rd, tmp);
> -}
> -break;
> +case 0: case 1: /* add/sub (3reg, 2reg imm), shift imm; in decodetree */
>  case 2: case 3: /* add, sub, cmp, mov (reg, imm), in decodetree */
>  goto illegal_op;
>  case 4:
> diff --git a/target/arm/t16.decode b/target/arm/t16.decode
> index f128110dee..79a1d66d6c 100644
> --- a/target/arm/t16.decode
> +++ b/target/arm/t16.decode
> @@ -126,6 +126,14 @@ ADD_rri 10101 rd:3  \
>  STM 11000 ...   @ldstm
>  LDM_t16 11001 ...   @ldstm
>  
> +# Shift (immediate)
> +
> +@shift_i. shim:5 rm:3 rd:3  _rrr_shi %s rn=%reg_0
> +
> +MOV_rxri000 00 . ... ...@shift_i shty=0  # LSL
> +MOV_rxri000 01 . ... ...@shift_i shty=1  # LSR
> +MOV_rxri000 10 . ... ...@shift_i shty=2  # ASR
> +
>  # Add/subtract (three low registers)
>  
>  @addsub_3   ... rm:3 rn:3 rd:3 \
> 

Re: [Qemu-devel] [PATCH v2 0/4] docs: add docs about use of automatic cleanup functions

[no-reply]

Patchew URL: https://patchew.org/QEMU/20190829160710.8792-1-berra...@redhat.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Message-id: 20190829160710.8792-1-berra...@redhat.com
Type: series
Subject: [Qemu-devel] [PATCH v2 0/4] docs: add docs about use of automatic 
cleanup functions

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
0eae52d docs: split the CODING_STYLE doc into distinct groups
68d2b3f docs: document use of automatic cleanup functions in glib
6316ff5 docs: merge HACKING.rst contents into CODING_STYLE.rst
89d6687 docs: convert README, CODING_STYLE and HACKING to RST syntax

=== OUTPUT BEGIN ===
Must be run from the top-level dir. of a kernel tree
=== OUTPUT END ===

Test command exited with code: 2


The full log is available at
http://patchew.org/logs/20190829160710.8792-1-berra...@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

Re: [Qemu-devel] [PATCH v3 35/69] target/arm: Convert Clear-Exclusive, Barriers

[Philippe Mathieu-Daudé]

On 8/28/19 9:04 PM, Richard Henderson wrote:
> Signed-off-by: Richard Henderson 
> ---
> v3: Check m-profile for all; fix missing isa check for ISB;
> fix thumb isa check for CLREX and DSB.
> ---
>  target/arm/translate.c   | 127 ---
>  target/arm/a32-uncond.decode |  10 +++
>  target/arm/t32.decode|  10 +++
>  3 files changed, 78 insertions(+), 69 deletions(-)
> 
> diff --git a/target/arm/translate.c b/target/arm/translate.c
> index 46e3f946d5..003b8ac414 100644
> --- a/target/arm/translate.c
> +++ b/target/arm/translate.c
> @@ -10122,6 +10122,63 @@ static bool trans_SRS(DisasContext *s, arg_SRS *a)
>  return true;
>  }
>  
> +/*
> + * Clear-Exclusive, Barriers
> + */
> +
> +static bool trans_CLREX(DisasContext *s, arg_CLREX *a)
> +{
> +if (s->thumb
> +? !ENABLE_ARCH_7 && !arm_dc_feature(s, ARM_FEATURE_M)
> +: !ENABLE_ARCH_6K) {
> +return false;
> +}
> +gen_clrex(s);
> +return true;
> +}
> +
> +static bool trans_DSB(DisasContext *s, arg_DSB *a)
> +{
> +if (!ENABLE_ARCH_7 && !arm_dc_feature(s, ARM_FEATURE_M)) {
> +return false;
> +}
> +tcg_gen_mb(TCG_MO_ALL | TCG_BAR_SC);
> +return true;
> +}
> +
> +static bool trans_DMB(DisasContext *s, arg_DMB *a)
> +{
> +return trans_DSB(s, NULL);
> +}
> +
> +static bool trans_ISB(DisasContext *s, arg_ISB *a)
> +{
> +if (!ENABLE_ARCH_7 && !arm_dc_feature(s, ARM_FEATURE_M)) {
> +return false;
> +}
> +/*
> + * We need to break the TB after this insn to execute
> + * self-modifying code correctly and also to take
> + * any pending interrupts immediately.
> + */
> +gen_goto_tb(s, 0, s->base.pc_next);
> +return true;
> +}
> +
> +static bool trans_SB(DisasContext *s, arg_SB *a)
> +{
> +if (!dc_isar_feature(aa32_sb, s)) {
> +return false;
> +}
> +/*
> + * TODO: There is no speculation barrier opcode
> + * for TCG; MB and end the TB instead.
> + */
> +tcg_gen_mb(TCG_MO_ALL | TCG_BAR_SC);
> +gen_goto_tb(s, 0, s->base.pc_next);
> +return true;
> +}
> +
>  /*
>   * Legacy decoder.
>   */
> @@ -10215,38 +10272,6 @@ static void disas_arm_insn(DisasContext *s, unsigned 
> int insn)
>  s->base.is_jmp = DISAS_UPDATE;
>  }
>  return;
> -} else if ((insn & 0x0f00) == 0x057ff000) {
> -switch ((insn >> 4) & 0xf) {
> -case 1: /* clrex */
> -ARCH(6K);
> -gen_clrex(s);
> -return;
> -case 4: /* dsb */
> -case 5: /* dmb */
> -ARCH(7);
> -tcg_gen_mb(TCG_MO_ALL | TCG_BAR_SC);
> -return;
> -case 6: /* isb */
> -/* We need to break the TB after this insn to execute
> - * self-modifying code correctly and also to take
> - * any pending interrupts immediately.
> - */
> -gen_goto_tb(s, 0, s->base.pc_next);
> -return;
> -case 7: /* sb */
> -if ((insn & 0xf) || !dc_isar_feature(aa32_sb, s)) {
> -goto illegal_op;
> -}
> -/*
> - * TODO: There is no speculation barrier opcode
> - * for TCG; MB and end the TB instead.
> - */
> -tcg_gen_mb(TCG_MO_ALL | TCG_BAR_SC);
> -gen_goto_tb(s, 0, s->base.pc_next);
> -return;
> -default:
> -goto illegal_op;
> -}
>  } else if ((insn & 0x0e000f00) == 0x0c000100) {
>  if (arm_dc_feature(s, ARM_FEATURE_IWMMXT)) {
>  /* iWMMXt register transfer.  */
> @@ -10707,43 +10732,7 @@ static void disas_thumb2_insn(DisasContext *s, 
> uint32_t insn)
>  gen_set_psr_im(s, offset, 0, imm);
>  }
>  break;
> -case 3: /* Special control operations.  */
> -if (!arm_dc_feature(s, ARM_FEATURE_V7) &&
> -!arm_dc_feature(s, ARM_FEATURE_M)) {
> -goto illegal_op;
> -}
> -op = (insn >> 4) & 0xf;
> -switch (op) {
> -case 2: /* clrex */
> -gen_clrex(s);
> -break;
> -case 4: /* dsb */
> -case 5: /* dmb */
> -tcg_gen_mb(TCG_MO_ALL | TCG_BAR_SC);
> -break;
> -case 6: /* isb */
> -/* We need to break the TB after this insn
> - * to execute self-modifying code correctly
> - * and also to take any pending interrupts
> - 

Re: [Qemu-devel] [PATCH v6 1/4] 9p: Treat multiple devices on one export as an error

[Greg Kurz]

On Thu, 22 Aug 2019 21:28:19 +0200
Christian Schoenebeck via Qemu-devel  wrote:

> The QID path should uniquely identify a file. However, the
> inode of a file is currently used as the QID path, which
> on its own only uniquely identifies files within a device.
> Here we track the device hosting the 9pfs share, in order
> to prevent security issues with QID path collisions from
> other devices.
> 
> Signed-off-by: Antonios Motakis 
> [CS: - Assign dev_id to export root's device already in
>v9fs_device_realize_common(), not postponed in
>stat_to_qid().
>  - error_report_once() if more than one device was
>shared by export.
>  - Return -ENODEV instead of -ENOSYS in stat_to_qid().
>  - Fixed typo in log comment. ]
> Signed-off-by: Christian Schoenebeck 
> ---
>  hw/9pfs/9p.c | 69 
> 
>  hw/9pfs/9p.h |  1 +
>  2 files changed, 56 insertions(+), 14 deletions(-)
> 
> diff --git a/hw/9pfs/9p.c b/hw/9pfs/9p.c
> index 586a6dccba..8cc65c2c67 100644
> --- a/hw/9pfs/9p.c
> +++ b/hw/9pfs/9p.c
> @@ -572,10 +572,18 @@ static void coroutine_fn virtfs_reset(V9fsPDU *pdu)
>  P9_STAT_MODE_SOCKET)
>  
>  /* This is the algorithm from ufs in spfs */
> -static void stat_to_qid(const struct stat *stbuf, V9fsQID *qidp)
> +static int stat_to_qid(V9fsPDU *pdu, const struct stat *stbuf, V9fsQID *qidp)
>  {
>  size_t size;
>  
> +if (pdu->s->dev_id != stbuf->st_dev) {
> +error_report_once(
> +"9p: Multiple devices detected in same VirtFS export. "
> +"You must use a separate export for each device."
> +);
> +return -ENODEV;

As explained in the v5 review, we don't necessarily want to break existing
cross-device setups that just happen to work. Moreover, the next patch
re-allows them since remap_inodes=ignore is the default. I would thus
only do:

warn_report_once(
"9p: Multiple devices detected in same VirtFS export, "
"which might lead to file ID collisions and severe "
"misbehaviours on guest! You should use a separate "
"export for each device shared from host."
);

So I've just changed that and applied to 9p-next since it is
a valuable improvement. Note that I've kept the signature change
of stat_to_qid() for simplicity even if it isn't needed before
the next patch.

> +}
> +
>  memset(>path, 0, sizeof(qidp->path));
>  size = MIN(sizeof(stbuf->st_ino), sizeof(qidp->path));
>  memcpy(>path, >st_ino, size);
> @@ -587,6 +595,8 @@ static void stat_to_qid(const struct stat *stbuf, V9fsQID 
> *qidp)
>  if (S_ISLNK(stbuf->st_mode)) {
>  qidp->type |= P9_QID_TYPE_SYMLINK;
>  }
> +
> +return 0;
>  }
>  
>  static int coroutine_fn fid_to_qid(V9fsPDU *pdu, V9fsFidState *fidp,
> @@ -599,7 +609,10 @@ static int coroutine_fn fid_to_qid(V9fsPDU *pdu, 
> V9fsFidState *fidp,
>  if (err < 0) {
>  return err;
>  }
> -stat_to_qid(, qidp);
> +err = stat_to_qid(pdu, , qidp);
> +if (err < 0) {
> +return err;
> +}
>  return 0;
>  }
>  
> @@ -830,7 +843,10 @@ static int coroutine_fn stat_to_v9stat(V9fsPDU *pdu, 
> V9fsPath *path,
>  
>  memset(v9stat, 0, sizeof(*v9stat));
>  
> -stat_to_qid(stbuf, >qid);
> +err = stat_to_qid(pdu, stbuf, >qid);
> +if (err < 0) {
> +return err;
> +}
>  v9stat->mode = stat_to_v9mode(stbuf);
>  v9stat->atime = stbuf->st_atime;
>  v9stat->mtime = stbuf->st_mtime;
> @@ -891,7 +907,7 @@ static int coroutine_fn stat_to_v9stat(V9fsPDU *pdu, 
> V9fsPath *path,
>  #define P9_STATS_ALL   0x3fffULL /* Mask for All fields above */
>  
>  
> -static void stat_to_v9stat_dotl(V9fsState *s, const struct stat *stbuf,
> +static int stat_to_v9stat_dotl(V9fsPDU *pdu, const struct stat *stbuf,
>  V9fsStatDotl *v9lstat)
>  {
>  memset(v9lstat, 0, sizeof(*v9lstat));
> @@ -913,7 +929,7 @@ static void stat_to_v9stat_dotl(V9fsState *s, const 
> struct stat *stbuf,
>  /* Currently we only support BASIC fields in stat */
>  v9lstat->st_result_mask = P9_STATS_BASIC;
>  
> -stat_to_qid(stbuf, >qid);
> +return stat_to_qid(pdu, stbuf, >qid);
>  }
>  
>  static void print_sg(struct iovec *sg, int cnt)
> @@ -1115,7 +1131,6 @@ static void coroutine_fn v9fs_getattr(void *opaque)
>  uint64_t request_mask;
>  V9fsStatDotl v9stat_dotl;
>  V9fsPDU *pdu = opaque;
> -V9fsState *s = pdu->s;
>  
>  retval = pdu_unmarshal(pdu, offset, "dq", , _mask);
>  if (retval < 0) {
> @@ -1136,7 +1151,10 @@ static void coroutine_fn v9fs_getattr(void *opaque)
>  if (retval < 0) {
>  goto out;
>  }
> -stat_to_v9stat_dotl(s, , _dotl);
> +retval = stat_to_v9stat_dotl(pdu, , _dotl);
> +if (retval < 0) {
> +goto out;
> +}
>  
>  /*  fill st_gen if requested and 

Re: [Qemu-devel] [edk2-rfc] [edk2-devel] CPU hotplug using SMM with QEMU+OVMF

[Laszlo Ersek]

On 08/28/19 14:01, Igor Mammedov wrote:
> On Tue, 27 Aug 2019 22:11:15 +0200
> Laszlo Ersek  wrote:
> 
>> On 08/27/19 18:23, Igor Mammedov wrote:
>>> On Mon, 26 Aug 2019 17:30:43 +0200
>>> Laszlo Ersek  wrote:
>>>
 On 08/23/19 17:25, Kinney, Michael D wrote:
> Hi Jiewen,
>
> If a hot add CPU needs to run any code before the
> first SMI, I would recommend is only executes code
> from a write protected FLASH range without a stack
> and then wait for the first SMI.  

 "without a stack" looks very risky to me. Even if we manage to implement
 the guest code initially, we'll be trapped without a stack, should we
 ever need to add more complex stuff there.
>>>
>>> Do we need anything complex in relocation handler, though?
>>> From what I'd imagine, minimum handler should
>>>   1: get address of TSEG, possibly read it from chipset
>>
>> The TSEG base calculation is not trivial in this environment. The 32-bit
>> RAM size needs to be read from the CMOS (IO port accesses). Then the
>> extended TSEG size (if any) needs to be detected from PCI config space
>> (IO port accesses). Both CMOS and PCI config space requires IO port
>> writes too (not just reads). Even if there are enough registers for the
>> calculations, can we rely on these unprotected IO ports?
>>
>> Also, can we switch to 32-bit mode without a stack? I assume it would be
>> necessary to switch to 32-bit mode for 32-bit arithmetic.
> from SDM vol 3:
> "
> 34.5.1 Initial SMM Execution Environment
> After saving the current context of the processor, the processor initializes 
> its core registers to the values shown in Table 34-4. Upon entering SMM, the 
> PE and PG flags in control register CR0 are cleared, which places the 
> processor in an environment similar to real-address mode. The differences 
> between the SMM execution environment and the real-address mode execution 
> environment are as follows:
> • The addressable address space ranges from 0 to H (4 GBytes).
> • The normal 64-KByte segment limit for real-address mode is increased to 4 
> GBytes.
> • The default operand and address sizes are set to 16 bits, which restricts 
> the addressable SMRAM address space to the 1-MByte real-address mode limit 
> for native real-address-mode code. However, operand-size and address-size 
> override prefixes can be used to access the address space beyond
>  
> the 1-MByte.
> "

That helps. Thanks for the quote!

>> Getting the initial APIC ID needs some CPUID instructions IIUC, which
>> clobber EAX through EDX, if I understand correctly. Given the register
>> pressure, CPUID might have to be one of the first instructions to call.
> 
> we could map at 3 not 64K required for save area but 128K and use
> 2nd half as secure RAM for stack and intermediate data.
> 
> Firmware could put there pre-calculated pointer to TSEG after it's configured 
> and locked down,
> this way relocation handler won't have to figure out TSEG address on its own.

Sounds like a great idea.

>>>   2: calculate its new SMBASE offset based on its APIC ID
>>>   3: save new SMBASE
>>>
> For this OVMF use case, is any CPU init required
> before the first SMI?  

 I expressed a preference for that too: "I wish we could simply wake the
 new CPU [...] with an SMI".

 http://mid.mail-archive.com/398b3327-0820-95af-a34d-1a4a1d50cf35@redhat.com


> From Paolo's list of steps are steps (8a) and (8b) 
> really required?  
>>>
>>> 07b - implies 08b
>>
>> I agree about that implication, yes. *If* we send an INIT/SIPI/SIPI to
>> the new CPU, then the new CPU needs a HLT loop, I think.
> It also could execute INIT reset, which leaves initialized SMM untouched
> but otherwise CPU would be inactive.
>  
>>
>>>8b could be trivial hlt loop and we most likely could skip 08a and 
>>> signaling host CPU steps
>>>but we need INIT/SIPI/SIPI sequence to wake up AP so it could handle 
>>> pending SMI
>>>before handling SIPI (so behavior would follow SDM).
>>>
>>>
 See again my message linked above -- just after the quoted sentence, I
 wrote, "IOW, if we could excise steps 07b, 08a, 08b".

 But, I obviously defer to Paolo and Igor on that.

 (I do believe we have a dilemma here. In QEMU, we probably prefer to
 emulate physical hardware as faithfully as possible. However, we do not
 have Cache-As-RAM (nor do we intend to, IIUC). Does that justify other
 divergences from physical hardware too, such as waking just by virtue of
 an SMI?)
>>> So far we should be able to implement it per spec (at least SDM one),
>>> but we would still need to invent chipset hardware
>>> i.e. like adding to Q35 non exiting SMRAM and means to map/unmap it
>>> to non-SMM address space.
>>> (and I hope we could avoid adding "parked CPU" thingy)
>>
>> I think we'll need a separate QEMU tree for this. I'm quite in the dark
>> -- I can't tell if 

Re: [Qemu-devel] [qemu-s390x] [PATCH v2 0/7] tcg: probe_write() refactorings

[Richard Henderson]

> On 8/27/19 9:09 AM, David Hildenbrand wrote:
>> Richard, in case there is no more feedback, will you take these patches
>> via your tree?

Queued to tcg-next.


r~

Re: [Qemu-devel] [PATCH] qga: add command guest-get-devices for reporting VirtIO devices

[Eric Blake]

On 8/29/19 11:03 AM, Tomáš Golembiovský wrote:
> Add command for reporting devices on Windows guest. The intent is not so
> much to report the devices but more importantly the driver (and its
> version) that is assigned to the device.
> 
> Signed-off-by: Tomáš Golembiovský 

> +++ b/qga/qapi-schema.json
> @@ -1242,3 +1242,35 @@
>  ##
>  { 'command': 'guest-get-osinfo',
>'returns': 'GuestOSInfo' }
> +
> +##
> +# @GuestDeviceInfo:
> +#
> +# @vendor-id: vendor ID as hexadecimal string in uper case without 0x prefix
> +# @device-id: device ID as hexadecimal string in uper case without 0x prefix

s/uper/upper/ twice

Should these be ints instead of strings (yes, it means they would be
decimal over the wire, which is not the typical representation)?

> +# @driver-name: name of the associated driver
> +# @driver-date: driver release date in format MM/DD/YY

Why US-centric?  Better would be something like ISO, -MM-DD

> +# @driver-version: driver version
> +#
> +# Since: 4.1.1

4.2.  We don't tend to add features on stable backport branches (as this
missed 4.1.0, we're unlikely to add it for 4.1.1).

> +##
> +{ 'struct': 'GuestDeviceInfo',
> +  'data': {
> +  'vendor-id': 'str',
> +  'device-id': 'str',
> +  'driver-name': 'str',
> +  'driver-date': 'str',
> +  'driver-version': 'str'
> +  } }
> +
> +##
> +# @guest-get-devices:
> +#
> +# Retrieve information about device drivers in Windows guest
> +#
> +# Returns: @GuestOSInfo
> +#
> +# Since: 4.1.1

again, 4.2

> +##
> +{ 'command': 'guest-get-devices',
> +  'returns': ['GuestDeviceInfo'] }
> 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] git.qemu.org gitweb misrenders on git/ URLs

[Andrea Bolognani]

On Thu, 2019-08-29 at 16:48 +0100, Stefan Hajnoczi wrote:
> Hi Jeff,
> Philippe noticed that the git HTTPS clone URL
> https://git.qemu.org/git/libslirp.git renders a gitweb page that looks
> right but has broken links.  The correct gitweb URL listed on
> https://git.qemu.org/ is https://git.qemu.org/?p=libslirp.git;a=summary,
> but there's a chance that people will open the HTTPS clone URL in their
> browser and expect to see gitweb working.
> 
> Is it possible to tweak the Apache configuration so that
> https://git.qemu.org/git/libslirp.git[/] redirects to the working gitweb
> URL?
> 
> The tricky part is not breaking HTTPS git clone, which accesses URLs
> below https://git.qemu.org/git/libslirp.git/ :).

I know that's not quite the answer to your question, but if you look
for example at

  https://git.zx2c4.com/cgit

you'll see that the same URL can be used both for viewing with a
browser *and* cloning.

Basically with cgit all requests go through the CGI script, and an
advantage of that is that you don't even need to call

  git update-server-info

to make the repository accessible via HTTPs. It's also pretty fast
and extremely easy to setup. Maybe consider switching from gitweb
to it?

-- 
Andrea Bolognani / Red Hat / Virtualization

[Qemu-devel] [PATCH v2 4/4] docs: split the CODING_STYLE doc into distinct groups

[Daniel P . Berrangé]

Signed-off-by: Daniel P. Berrangé 
---
 CODING_STYLE.rst | 9 +
 1 file changed, 9 insertions(+)

diff --git a/CODING_STYLE.rst b/CODING_STYLE.rst
index 39397f0f6f..427699e0e4 100644
--- a/CODING_STYLE.rst
+++ b/CODING_STYLE.rst
@@ -7,6 +7,9 @@ QEMU Coding Style
 Please use the script checkpatch.pl in the scripts directory to check
 patches before submitting.
 
+Formatting and style
+
+
 Whitespace
 ==
 
@@ -205,6 +208,9 @@ comment anyway.)
 Rationale: Consistency, and ease of visually picking out a multiline
 comment from the surrounding code.
 
+Language usage
+**
+
 Preprocessor
 
 
@@ -526,6 +532,9 @@ are still some caveats to beware of
 }
 
 
+QEMU Specific Idioms
+
+
 Error handling and reporting
 
 
-- 
2.21.0

[Qemu-devel] [PATCH v2 3/4] docs: document use of automatic cleanup functions in glib

[Daniel P . Berrangé]

Document the use of g_autofree and g_autoptr in glib for automatic
freeing of memory.

Signed-off-by: Daniel P. Berrangé 
---
 CODING_STYLE.rst | 85 
 1 file changed, 85 insertions(+)

diff --git a/CODING_STYLE.rst b/CODING_STYLE.rst
index 4501d87352..39397f0f6f 100644
--- a/CODING_STYLE.rst
+++ b/CODING_STYLE.rst
@@ -441,6 +441,91 @@ In addition, QEMU assumes that the compiler does not use 
the latitude
 given in C99 and C11 to treat aspects of signed '<<' as undefined, as
 documented in the GNU Compiler Collection manual starting at version 4.0.
 
+Automatic memory deallocation
+=
+
+QEMU has a mandatory dependency either the GCC or CLang compiler. As
+such it has the freedom to make use of a C language extension for
+automatically running a cleanup function when a stack variable goes
+out of scope. This can be used to simplify function cleanup paths,
+often allowing many goto jumps to be eliminated, through automatic
+free'ing of memory.
+
+The GLib2 library provides a number of functions/macros for enabling
+automatic cleanup:
+
+  ``_
+
+Most notably:
+
+* g_autofree - will invoke g_free() on the variable going out of scope
+
+* g_autoptr - for structs / objects, will invoke the cleanup func created
+  by a previous use of G_DEFINE_AUTOPTR_CLEANUP_FUNC. This is
+  supported for most GLib data types and GObjects
+
+For example, instead of
+
+.. code-block:: c
+
+int somefunc(void) {
+int ret = -1;
+char *foo = g_strdup_printf("foo%", "wibble");
+GList *bar = .
+
+if (eek) {
+   goto cleanup;
+}
+
+ret = 0;
+
+  cleanup:
+g_free(foo);
+g_list_free(bar);
+return ret;
+}
+
+Using g_autofree/g_autoptr enables the code to be written as:
+
+.. code-block:: c
+
+int somefunc(void) {
+g_autofree char *foo = g_strdup_printf("foo%", "wibble");
+g_autoptr (GList) bar = .
+
+if (eek) {
+   return -1;
+}
+
+return 0;
+}
+
+While this generally results in simpler, less leak-prone code, there
+are still some caveats to beware of
+
+* Variables declared with g_auto* MUST always be initialized,
+  otherwise the cleanup function will use uninitialized stack memory
+
+* If a variable declared with g_auto* holds a value which must
+  live beyond the life of the function, that value must be saved
+  and the original variable NULL'd out. This can be simpler using
+  g_steal_pointer
+
+
+.. code-block:: c
+
+char *somefunc(void) {
+g_autofree char *foo = g_strdup_printf("foo%", "wibble");
+g_autoptr (GList) bar = .
+
+if (eek) {
+   return NULL;
+}
+
+return g_steal_pointer();
+}
+
+
 Error handling and reporting
 
 
-- 
2.21.0

[Qemu-devel] [PATCH v2 2/4] docs: merge HACKING.rst contents into CODING_STYLE.rst

[Daniel P . Berrangé]

The split of information between the two docs is rather arbitary and
unclear. It is simpler for contributors if all the information is in
one file.

Signed-off-by: Daniel P. Berrangé 
---
 CODING_STYLE.rst | 296 ++
 HACKING.rst  | 300 ---
 README.rst   |   2 +-
 3 files changed, 297 insertions(+), 301 deletions(-)
 delete mode 100644 HACKING.rst

diff --git a/CODING_STYLE.rst b/CODING_STYLE.rst
index 713357cb80..4501d87352 100644
--- a/CODING_STYLE.rst
+++ b/CODING_STYLE.rst
@@ -205,6 +205,302 @@ comment anyway.)
 Rationale: Consistency, and ease of visually picking out a multiline
 comment from the surrounding code.
 
+Preprocessor
+
+
+Variadic macros
+---
+
+For variadic macros, stick with this C99-like syntax:
+
+.. code-block:: c
+
+#define DPRINTF(fmt, ...)   \
+do { printf("IRQ: " fmt, ## __VA_ARGS__); } while (0)
+
+Include directives
+--
+
+Order include directives as follows:
+
+.. code-block:: c
+
+#include "qemu/osdep.h"  /* Always first... */
+#include <...>   /* then system headers... */
+#include "..."   /* and finally QEMU headers. */
+
+The "qemu/osdep.h" header contains preprocessor macros that affect the behavior
+of core system headers like .  It must be the first include so that
+core system headers included by external libraries get the preprocessor macros
+that QEMU depends on.
+
+Do not include "qemu/osdep.h" from header files since the .c file will have
+already included it.
+
+C types
+===
+
+It should be common sense to use the right type, but we have collected
+a few useful guidelines here.
+
+Scalars
+---
+
+If you're using "int" or "long", odds are good that there's a better type.
+If a variable is counting something, it should be declared with an
+unsigned type.
+
+If it's host memory-size related, size_t should be a good choice (use
+ssize_t only if required). Guest RAM memory offsets must use ram_addr_t,
+but only for RAM, it may not cover whole guest address space.
+
+If it's file-size related, use off_t.
+If it's file-offset related (i.e., signed), use off_t.
+If it's just counting small numbers use "unsigned int";
+(on all but oddball embedded systems, you can assume that that
+type is at least four bytes wide).
+
+In the event that you require a specific width, use a standard type
+like int32_t, uint32_t, uint64_t, etc.  The specific types are
+mandatory for VMState fields.
+
+Don't use Linux kernel internal types like u32, __u32 or __le32.
+
+Use hwaddr for guest physical addresses except pcibus_t
+for PCI addresses.  In addition, ram_addr_t is a QEMU internal address
+space that maps guest RAM physical addresses into an intermediate
+address space that can map to host virtual address spaces.  Generally
+speaking, the size of guest memory can always fit into ram_addr_t but
+it would not be correct to store an actual guest physical address in a
+ram_addr_t.
+
+For CPU virtual addresses there are several possible types.
+vaddr is the best type to use to hold a CPU virtual address in
+target-independent code. It is guaranteed to be large enough to hold a
+virtual address for any target, and it does not change size from target
+to target. It is always unsigned.
+target_ulong is a type the size of a virtual address on the CPU; this means
+it may be 32 or 64 bits depending on which target is being built. It should
+therefore be used only in target-specific code, and in some
+performance-critical built-per-target core code such as the TLB code.
+There is also a signed version, target_long.
+abi_ulong is for the ``*``-user targets, and represents a type the size of
+'void ``*``' in that target's ABI. (This may not be the same as the size of a
+full CPU virtual address in the case of target ABIs which use 32 bit pointers
+on 64 bit CPUs, like sparc32plus.) Definitions of structures that must match
+the target's ABI must use this type for anything that on the target is defined
+to be an 'unsigned long' or a pointer type.
+There is also a signed version, abi_long.
+
+Of course, take all of the above with a grain of salt.  If you're about
+to use some system interface that requires a type like size_t, pid_t or
+off_t, use matching types for any corresponding variables.
+
+Also, if you try to use e.g., "unsigned int" as a type, and that
+conflicts with the signedness of a related variable, sometimes
+it's best just to use the *wrong* type, if "pulling the thread"
+and fixing all related variables would be too invasive.
+
+Finally, while using descriptive types is important, be careful not to
+go overboard.  If whatever you're doing causes warnings, or requires
+casts, then reconsider or ask for help.
+
+Pointers
+
+
+Ensure that all of your pointers are "const-correct".
+Unless a pointer is used to modify the pointed-to storage,
+give it the "const" 

Re: [Qemu-devel] [PATCH 0/2] git.orderfile: Order Python/shell scripts before unordered files

[Philippe Mathieu-Daudé]

Oops Cc'ing qemu-triv...@nongnu.org

On 8/29/19 12:05 PM, Philippe Mathieu-Daudé wrote:
> This series update the git.orderfile to order Python and shell
> scripts before unordered files.
> This is particularly useful for changes in tests/qemu-iotests.
> 
> Regards,
> 
> Phil.
> 
> Philippe Mathieu-Daudé (2):
>   scripts/git.orderfile: Order Python files before unordered ones
>   scripts/git.orderfile: Order shell scripts before unordered files
> 
>  scripts/git.orderfile | 5 +
>  1 file changed, 5 insertions(+)

[Qemu-devel] [PATCH v2 0/4] docs: add docs about use of automatic cleanup functions

[Daniel P . Berrangé]

This is ostensibly about adding docs for the g_autofree/g_autoptr
macros. As part of doing that, however, the existing HACKING doc
is merged into the CODING_STYLE doc and the text is converted to
rst with a table of contents.

Changed in v2:

 - Use RST instead of markdown
 - Also convert README file
 - Group sections better
 - Remove GMutexLocker example

Daniel P. Berrangé (4):
  docs: convert README, CODING_STYLE and HACKING to RST syntax
  docs: merge HACKING.rst contents into CODING_STYLE.rst
  docs: document use of automatic cleanup functions in glib
  docs: split the CODING_STYLE doc into distinct groups

 CODING_STYLE | 216 ---
 CODING_STYLE.rst | 641 +++
 HACKING  | 257 -
 README => README.rst |  47 ++--
 4 files changed, 670 insertions(+), 491 deletions(-)
 delete mode 100644 CODING_STYLE
 create mode 100644 CODING_STYLE.rst
 delete mode 100644 HACKING
 rename README => README.rst (84%)

-- 
2.21.0

[Qemu-devel] [PATCH v2 1/4] docs: convert README, CODING_STYLE and HACKING to RST syntax

[Daniel P . Berrangé]

Signed-off-by: Daniel P. Berrangé 
---
 CODING_STYLE => CODING_STYLE.rst | 121 +++---
 HACKING => HACKING.rst   | 123 +--
 README => README.rst |  47 +++-
 3 files changed, 190 insertions(+), 101 deletions(-)
 rename CODING_STYLE => CODING_STYLE.rst (72%)
 rename HACKING => HACKING.rst (79%)
 rename README => README.rst (84%)

diff --git a/CODING_STYLE b/CODING_STYLE.rst
similarity index 72%
rename from CODING_STYLE
rename to CODING_STYLE.rst
index cb8edcbb36..713357cb80 100644
--- a/CODING_STYLE
+++ b/CODING_STYLE.rst
@@ -1,10 +1,14 @@
+=
 QEMU Coding Style
 =
 
+.. contents:: Table of Contents
+
 Please use the script checkpatch.pl in the scripts directory to check
 patches before submitting.
 
-1. Whitespace
+Whitespace
+==
 
 Of course, the most important aspect in any coding style is whitespace.
 Crusty old coders who have trouble spotting the glasses on their noses
@@ -16,26 +20,27 @@ QEMU indents are four spaces.  Tabs are never used, except 
in Makefiles
 where they have been irreversibly coded into the syntax.
 Spaces of course are superior to tabs because:
 
- - You have just one way to specify whitespace, not two.  Ambiguity breeds
-   mistakes.
- - The confusion surrounding 'use tabs to indent, spaces to justify' is gone.
- - Tab indents push your code to the right, making your screen seriously
-   unbalanced.
- - Tabs will be rendered incorrectly on editors who are misconfigured not
-   to use tab stops of eight positions.
- - Tabs are rendered badly in patches, causing off-by-one errors in almost
-   every line.
- - It is the QEMU coding style.
+* You have just one way to specify whitespace, not two.  Ambiguity breeds
+  mistakes.
+* The confusion surrounding 'use tabs to indent, spaces to justify' is gone.
+* Tab indents push your code to the right, making your screen seriously
+  unbalanced.
+* Tabs will be rendered incorrectly on editors who are misconfigured not
+  to use tab stops of eight positions.
+* Tabs are rendered badly in patches, causing off-by-one errors in almost
+  every line.
+* It is the QEMU coding style.
 
 Do not leave whitespace dangling off the ends of lines.
 
-1.1 Multiline Indent
+Multiline Indent
+
 
 There are several places where indent is necessary:
 
- - if/else
- - while/for
- - function definition & call
+* if/else
+* while/for
+* function definition & call
 
 When breaking up a long line to fit within line width, we need a proper indent
 for the following lines.
@@ -45,6 +50,8 @@ opening parenthesis of the first.
 
 For example:
 
+.. code-block:: c
+
 if (a == 1 &&
 b == 2) {
 
@@ -53,12 +60,13 @@ For example:
 
 In case of function, there are several variants:
 
-* 4 spaces indent from the beginning
-* align the secondary lines just after the opening parenthesis of the
-  first
+* 4 spaces indent from the beginning
+* align the secondary lines just after the opening parenthesis of the first
 
 For example:
 
+.. code-block:: c
+
 do_something(x, y,
 z);
 
@@ -68,7 +76,8 @@ For example:
 do_something(x, do_another(y,
z));
 
-2. Line width
+Line width
+==
 
 Lines should be 80 characters; try not to make them longer.
 
@@ -77,16 +86,18 @@ that use long function or symbol names.  Even in that case, 
do not make
 lines much longer than 80 characters.
 
 Rationale:
- - Some people like to tile their 24" screens with a 6x4 matrix of 80x24
-   xterms and use vi in all of them.  The best way to punish them is to
-   let them keep doing it.
- - Code and especially patches is much more readable if limited to a sane
-   line length.  Eighty is traditional.
- - The four-space indentation makes the most common excuse ("But look
-   at all that white space on the left!") moot.
- - It is the QEMU coding style.
 
-3. Naming
+* Some people like to tile their 24" screens with a 6x4 matrix of 80x24
+  xterms and use vi in all of them.  The best way to punish them is to
+  let them keep doing it.
+* Code and especially patches is much more readable if limited to a sane
+  line length.  Eighty is traditional.
+* The four-space indentation makes the most common excuse ("But look
+  at all that white space on the left!") moot.
+* It is the QEMU coding style.
+
+Naming
+==
 
 Variables are lower_case_with_underscores; easy to type and read.  Structured
 type names are in CamelCase; harder to type but standing out.  Enum type
@@ -95,10 +106,11 @@ names are lower_case_with_underscores_ending_with_a_t, 
like the POSIX
 uint64_t and family.  Note that this last convention contradicts POSIX
 and is therefore likely to be changed.
 
-When wrapping standard library functions, use the prefix qemu_ to alert
+When wrapping standard library functions, use the prefix ``qemu_`` to alert
 readers that they are seeing a wrapped version; otherwise avoid this prefix.
 
-4. Block 

[Qemu-devel] [PATCH] qga: add command guest-get-devices for reporting VirtIO devices

[Tomáš Golembiovský]

Add command for reporting devices on Windows guest. The intent is not so
much to report the devices but more importantly the driver (and its
version) that is assigned to the device.

Signed-off-by: Tomáš Golembiovský 
---
 qga/commands-posix.c |  11 +++
 qga/commands-win32.c | 195 ++-
 qga/qapi-schema.json |  32 +++
 3 files changed, 237 insertions(+), 1 deletion(-)

diff --git a/qga/commands-posix.c b/qga/commands-posix.c
index dfc05f5b8a..9adf8bb520 100644
--- a/qga/commands-posix.c
+++ b/qga/commands-posix.c
@@ -2709,6 +2709,7 @@ int64_t qmp_guest_fsfreeze_thaw(Error **errp)
 
 return 0;
 }
+
 #endif /* CONFIG_FSFREEZE */
 
 #if !defined(CONFIG_FSTRIM)
@@ -2757,6 +2758,8 @@ GList *ga_command_blacklist_init(GList *blacklist)
 blacklist = g_list_append(blacklist, g_strdup("guest-fstrim"));
 #endif
 
+blacklist = g_list_append(blacklist, g_strdup("guest-get-devices"));
+
 return blacklist;
 }
 
@@ -2977,3 +2980,11 @@ GuestOSInfo *qmp_guest_get_osinfo(Error **errp)
 
 return info;
 }
+
+GuestDeviceInfoList *qmp_guest_get_devices(Error **errp)
+{
+error_setg(errp, QERR_UNSUPPORTED);
+
+return NULL;
+}
+
diff --git a/qga/commands-win32.c b/qga/commands-win32.c
index 6b67f16faf..0bb93422c7 100644
--- a/qga/commands-win32.c
+++ b/qga/commands-win32.c
@@ -21,10 +21,11 @@
 #ifdef CONFIG_QGA_NTDDSCSI
 #include 
 #include 
+#endif
 #include 
 #include 
 #include 
-#endif
+#include 
 #include 
 #include 
 #include 
@@ -38,6 +39,36 @@
 #include "qemu/host-utils.h"
 #include "qemu/base64.h"
 
+
+/* The following should be in devpkey.h, but it isn't */
+DEFINE_DEVPROPKEY(DEVPKEY_NAME, 0xb725f130, 0x47ef, 0x101a, 0xa5, 0xf1, 0x02,
+0x60, 0x8c, 0x9e, 0xeb, 0xac, 10);  /* DEVPROP_TYPE_STRING */
+DEFINE_DEVPROPKEY(DEVPKEY_Device_HardwareIds, 0xa45c254e, 0xdf1c, 0x4efd, 0x80,
+0x20, 0x67, 0xd1, 0x46, 0xa8, 0x50, 0xe0, 3);
+/* DEVPROP_TYPE_STRING_LIST */
+DEFINE_DEVPROPKEY(DEVPKEY_Device_DriverDate, 0xa8b865dd, 0x2e3d, 0x4094, 0xad,
+0x97, 0xe5, 0x93, 0xa7, 0xc, 0x75, 0xd6, 2);  /* DEVPROP_TYPE_FILETIME */
+DEFINE_DEVPROPKEY(DEVPKEY_Device_DriverVersion, 0xa8b865dd, 0x2e3d, 0x4094,
+0xad, 0x97, 0xe5, 0x93, 0xa7, 0xc, 0x75, 0xd6, 3);
+/* DEVPROP_TYPE_STRING */
+/* The following should be in sal.h, but it isn't */
+#ifndef _Out_writes_bytes_opt_
+#define _Out_writes_bytes_opt_(s)
+#endif
+/* The following shoud be in cfgmgr32.h, but it isn't */
+#ifndef CM_Get_DevNode_Property
+CMAPI CONFIGRET WINAPI CM_Get_DevNode_PropertyW(
+_In_  DEVINST   dnDevInst,
+_In_  CONST DEVPROPKEY* PropertyKey,
+_Out_ DEVPROPTYPE * PropertyType,
+_Out_writes_bytes_opt_(*PropertyBufferSize) PBYTE PropertyBuffer,
+_Inout_ PULONG  PropertyBufferSize,
+_In_  ULONG ulFlags
+);
+#define CM_Get_DevNode_Property  CM_Get_DevNode_PropertyW
+#endif
+
+
 #ifndef SHTDN_REASON_FLAG_PLANNED
 #define SHTDN_REASON_FLAG_PLANNED 0x8000
 #endif
@@ -2234,3 +2265,165 @@ GuestOSInfo *qmp_guest_get_osinfo(Error **errp)
 
 return info;
 }
+
+/*
+ * Safely get device property. Returned strings are using wide characters.
+ * Caller is responsible for freeing the buffer.
+ */
+static LPBYTE cm_get_property(DEVINST devInst, const DEVPROPKEY *propName,
+PDEVPROPTYPE propType)
+{
+CONFIGRET cr;
+LPBYTE buffer = NULL;
+ULONG bufferLen = 0;
+
+/* First query for needed space */
+cr = CM_Get_DevNode_Property(devInst, propName, propType,
+buffer, , 0);
+if ((cr != CR_SUCCESS) && (cr != CR_BUFFER_SMALL)) {
+
+g_debug(
+"failed to get size of device property, device error code=%lx",
+cr);
+return NULL;
+}
+buffer = (LPBYTE)g_malloc(bufferLen);
+cr = CM_Get_DevNode_Property(devInst, propName, propType,
+buffer, , 0);
+if (cr != CR_SUCCESS) {
+g_free(buffer);
+g_debug(
+"failed to get device property, device error code=%lx", cr);
+return NULL;
+}
+return buffer;
+}
+
+/*
+ * 
https://docs.microsoft.com/en-us/windows-hardware/drivers/install/identifiers-for-pci-devices
+ */
+#define DEVICE_PCI_RE "PCIVEN_(1AF4|1B36)_([0-9A-B]{4})(&|$)"
+
+GuestDeviceInfoList *qmp_guest_get_devices(Error **errp)
+{
+GuestDeviceInfoList *head = NULL, *cur_item = NULL, *item = NULL;
+HDEVINFO dev_info = INVALID_HANDLE_VALUE;
+SP_DEVINFO_DATA dev_info_data;
+int i;
+GError *gerr = NULL;
+GRegex *device_pci_re = NULL;
+
+device_pci_re = g_regex_new(DEVICE_PCI_RE,
+G_REGEX_ANCHORED | G_REGEX_OPTIMIZE, 0,
+);
+
+if (gerr) {
+error_setg(errp, QERR_QGA_COMMAND_FAILED, gerr->message);
+g_error_free(gerr);
+goto out;
+}
+
+dev_info_data.cbSize = sizeof(SP_DEVINFO_DATA);
+dev_info = SetupDiGetClassDevs(0, 0, 0, DIGCF_PRESENT | DIGCF_ALLCLASSES);
+if (dev_info == INVALID_HANDLE_VALUE) {