[PATCH v9 28/74] xtensa: convert to cpu_halted

2020-05-21 Thread Robert Foley
From: "Emilio G. Cota" Cc: Max Filippov Reviewed-by: Richard Henderson Reviewed-by: Alex Bennée Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- target/xtensa/cpu.c| 2 +- target/xtensa/exc_helper.c | 2 +- target/xtensa/helper.c | 2 +- 3 files

[PATCH v9 38/74] openrisc: use cpu_reset_interrupt

2020-05-21 Thread Robert Foley
From: Paolo Bonzini Cc: Stafford Horne Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Richard Henderson Reviewed-by: Alex Bennée Signed-off-by: Paolo Bonzini Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- target/openrisc/sys_helper.c | 2 +- 1 file changed, 1 insertion

[PATCH v9 71/74] cpu: protect CPU state with cpu->lock instead of the BQL

2020-05-21 Thread Robert Foley
up. Reviewed-by: Alex Bennée Reviewed-by: Richard Henderson Signed-off-by: Emilio G. Cota [RF: Adjusted assert and changed to use timed_wait] Signed-off-by: Robert Foley --- cpus.c| 288 +++--- hw/core/cpu.c | 29 + include/hw/core/cpu.h

[PATCH v9 51/74] mips: convert to cpu_interrupt_request

2020-05-21 Thread Robert Foley
From: "Emilio G. Cota" Cc: Aurelien Jarno Cc: Aleksandar Markovic Reviewed-by: Richard Henderson Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- target/mips/cpu.c | 7 --- target/mips/kvm.c | 2 +- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/t

[PATCH v9 53/74] s390x: convert to cpu_interrupt_request

2020-05-21 Thread Robert Foley
From: "Emilio G. Cota" Cc: Christian Borntraeger Cc: David Hildenbrand Cc: qemu-s3...@nongnu.org Reviewed-by: Richard Henderson Reviewed-by: Alex Bennée Reviewed-by: Cornelia Huck Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- hw/intc/s390_flic.c | 2 +- target/s

[PATCH v9 35/74] exec: use cpu_reset_interrupt

2020-05-21 Thread Robert Foley
From: "Emilio G. Cota" Reviewed-by: Richard Henderson Reviewed-by: Alex Bennée Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- exec.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/exec.c b/exec.c index 5162f0d12f..37e039de06 100644 --- a/ex

[PATCH v9 08/74] tcg-runtime: define helper_cpu_halted_set

2020-05-21 Thread Robert Foley
From: "Emilio G. Cota" Reviewed-by: Richard Henderson Reviewed-by: Alex Bennée Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- accel/tcg/tcg-runtime.c | 7 +++ accel/tcg/tcg-runtime.h | 2 ++ 2 files changed, 9 insertions(+) diff --git a/accel/tcg/tcg-runtime

[PATCH v9 10/74] cris: convert to helper_cpu_halted_set

2020-05-21 Thread Robert Foley
From: "Emilio G. Cota" And fix the temp leak along the way. Cc: "Edgar E. Iglesias" Reviewed-by: Richard Henderson Reviewed-by: Alex Bennée Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- target/cris/translate.c | 5 +++-- 1 file changed, 3 insert

[PATCH v9 16/74] tcg-runtime: convert to cpu_halted_set

2020-05-21 Thread Robert Foley
From: "Emilio G. Cota" Reviewed-by: Richard Henderson Reviewed-by: Alex Bennée Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- accel/tcg/tcg-runtime.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/accel/tcg/tcg-runtime.c b/accel/tcg/tcg-runti

[PATCH v9 11/74] hppa: convert to helper_cpu_halted_set

2020-05-21 Thread Robert Foley
From: "Emilio G. Cota" Reviewed-by: Richard Henderson Reviewed-by: Alex Bennée Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- target/hppa/translate.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/target/hppa/translate.c b/target/hppa/transla

[PATCH v9 31/74] cpu-exec: convert to cpu_halted

2020-05-21 Thread Robert Foley
From: "Emilio G. Cota" Reviewed-by: Richard Henderson Reviewed-by: Alex Bennée Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- accel/tcg/cpu-exec.c | 25 + 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/accel/tcg/cpu-exec.c b

[PATCH v9 22/74] lm32: convert to cpu_halted

2020-05-21 Thread Robert Foley
From: "Emilio G. Cota" Cc: Michael Walle Reviewed-by: Richard Henderson Reviewed-by: Alex Bennée Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- target/lm32/op_helper.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/target/lm32/op_helper.

[PATCH v9 70/74] cpu: rename all_cpu_threads_idle to qemu_tcg_rr_all_cpu_threads_idle

2020-05-21 Thread Robert Foley
From: "Emilio G. Cota" This function is only called from TCG rr mode, so add a prefix to mark this as well as an assertion. Reviewed-by: Alex Bennée Reviewed-by: Richard Henderson Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- cpus.c | 10 ++ 1 file

[PATCH v9 50/74] m68k: convert to cpu_interrupt_request

2020-05-21 Thread Robert Foley
From: "Emilio G. Cota" Reviewed-by: Richard Henderson Reviewed-by: Laurent Vivier Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- target/m68k/cpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/target/m68k/cpu.c b/target/m68k/cpu.c index

[PATCH v9 56/74] sparc: convert to cpu_interrupt_request

2020-05-21 Thread Robert Foley
From: "Emilio G. Cota" Cc: Artyom Tarasenko Reviewed-by: Richard Henderson Reviewed-by: Alex Bennée Reviewed-by: Mark Cave-Ayland Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- hw/sparc64/sparc64.c | 4 ++-- target/sparc/cpu.c | 2 +- 2 files changed, 3 insert

[PATCH v9 57/74] openrisc: convert to cpu_interrupt_request

2020-05-21 Thread Robert Foley
From: "Emilio G. Cota" Cc: Stafford Horne Reviewed-by: Richard Henderson Reviewed-by: Alex Bennée Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- hw/openrisc/cputimer.c | 2 +- target/openrisc/cpu.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff

[PATCH v9 62/74] cpu: call .cpu_has_work with the CPU lock held

2020-05-21 Thread Robert Foley
From: "Emilio G. Cota" Reviewed-by: Richard Henderson Reviewed-by: Alex Bennée Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- include/hw/core/cpu.h | 12 ++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/include/hw/core/cpu.h b/include/hw/

[PATCH v9 67/74] riscv: convert to cpu_has_work_with_iothread_lock

2020-05-21 Thread Robert Foley
From: "Emilio G. Cota" Soon we will call cpu_has_work without the BQL. Cc: Sagar Karandikar Cc: Bastian Koppelmann Reviewed-by: Palmer Dabbelt Reviewed-by: Richard Henderson Reviewed-by: Alistair Francis Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- target/r

[PATCH v9 00/74] per-CPU locks

2020-05-21 Thread Robert Foley
t_interrupt s390x: use cpu_reset_interrupt openrisc: use cpu_reset_interrupt Robert Foley (1): hw/semihosting: convert to cpu_halted_set accel/tcg/cpu-exec.c| 40 ++- accel/tcg/cputlb.c | 10 +- accel/tcg/tcg-all.c | 12 +- accel/tcg/tcg-runtime.c

[PATCH v9 02/74] cpu: rename cpu->work_mutex to cpu->lock

2020-05-21 Thread Robert Foley
From: "Emilio G. Cota" This lock will soon protect more fields of the struct. Give it a more appropriate name. Reviewed-by: Richard Henderson Reviewed-by: Alex Bennée Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- cpus-common.c | 14 +++-

[PATCH v9 01/74] cpu: convert queued work to a QSIMPLEQ

2020-05-21 Thread Robert Foley
From: "Emilio G. Cota" Instead of open-coding it. While at it, make sure that all accesses to the list are performed while holding the list's lock. Reviewed-by: Richard Henderson Reviewed-by: Alex Bennée Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- cpu

[PATCH v9 06/74] cpu: introduce process_queued_cpu_work_locked

2020-05-21 Thread Robert Foley
From: "Emilio G. Cota" This completes the conversion to cpu_mutex_lock/unlock in the file. Reviewed-by: Richard Henderson Reviewed-by: Alex Bennée Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- cpus-common.c | 17 +++-- 1 file changed, 11 insert

[PATCH v9 37/74] s390x: use cpu_reset_interrupt

2020-05-21 Thread Robert Foley
From: Paolo Bonzini Cc: qemu-s3...@nongnu.org Reviewed-by: David Hildenbrand Reviewed-by: Richard Henderson Reviewed-by: Cornelia Huck Reviewed-by: Alex Bennée Signed-off-by: Paolo Bonzini Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- target/s390x/excp_helper.c | 2 +- 1

[PATCH v9 59/74] microblaze: convert to cpu_interrupt_request

2020-05-21 Thread Robert Foley
From: "Emilio G. Cota" Cc: "Edgar E. Iglesias" Reviewed-by: Richard Henderson Reviewed-by: Alex Bennée Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- target/microblaze/cpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/target/micro

[PATCH v9 40/74] i386: convert to cpu_interrupt_request

2020-05-21 Thread Robert Foley
From: "Emilio G. Cota" Reviewed-by: Richard Henderson Reviewed-by: Alex Bennée Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- target/i386/cpu.c| 2 +- target/i386/helper.c | 4 ++-- target/i386/svm_helper.c | 4 ++-- 3 files changed, 5 insertions(+), 5

[PATCH v9 39/74] arm: convert to cpu_interrupt_request

2020-05-21 Thread Robert Foley
From: "Emilio G. Cota" Cc: Peter Maydell Cc: qemu-...@nongnu.org Reviewed-by: Richard Henderson Reviewed-by: Alex Bennée Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- target/arm/cpu.c | 6 +++--- target/arm/helper.c | 16 +++- target/arm/mach

Re: [PATCH-for-5.2] exec: Remove MemoryRegion::global_locking field

2020-08-23 Thread Robert Foley
On Sat, 22 Aug 2020 at 16:13, Philippe Mathieu-Daudé wrote: > > +Robert > > On 8/7/20 12:16 PM, Paolo Bonzini wrote: > > On 07/08/20 12:02, Stefan Hajnoczi wrote: > >> On Thu, Aug 06, 2020 at 05:07:26PM +0200, Philippe Mathieu-Daudé wrote: > >>> Last uses of memory_region_clear_global_locking()

[PATCH v2 4/7] target: Push BQL on ->do_interrupt down into per-arch implementation

2020-08-19 Thread Robert Foley
u-devel/2020-08/msg00044.html Signed-off-by: Robert Foley --- accel/tcg/cpu-exec.c| 2 -- target/alpha/cpu.c | 2 +- target/alpha/cpu.h | 2 +- target/alpha/helper.c | 9 - target/arm/cpu.c| 2 +- target/arm/cpu.h

[PATCH v2 7/7] target: Push BQL on ->cpu_exec_interrupt down into per-arch implementation

2020-08-19 Thread Robert Foley
g/archive/html/qemu-devel/2020-08/msg00784.html https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg01517.html https://lists.gnu.org/archive/html/qemu-devel/2020-07/msg08731.html https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg00044.html Signed-off-by: Robert Foley --- accel/tcg/c

[PATCH v2 2/7] target/arm: add ARMCPUClass->do_interrupt_locked

2020-08-19 Thread Robert Foley
his approach. https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg00784.html https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg01517.html https://lists.gnu.org/archive/html/qemu-devel/2020-07/msg08731.html https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg00044.html Signed-off-by:

[PATCH v2 1/7] target: rename all *_do_interupt functions to _do_interrupt_locked

2020-08-19 Thread Robert Foley
://lists.gnu.org/archive/html/qemu-devel/2020-08/msg01517.html https://lists.gnu.org/archive/html/qemu-devel/2020-07/msg08731.html https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg00044.html Signed-off-by: Robert Foley --- hw/ppc/spapr_events.c | 2 +- target/alpha/cpu.c

[PATCH v2 6/7] target: rename all *_cpu_exec_interrupt functions to *_cpu_exec_interrupt_locked

2020-08-19 Thread Robert Foley
-devel/2020-08/msg00784.html https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg01517.html https://lists.gnu.org/archive/html/qemu-devel/2020-07/msg08731.html https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg00044.html Signed-off-by: Robert Foley --- target/alpha/cpu.c | 2

[PATCH v2 5/7] accel/tcg: Change BQL critical section in cpu_handle_interrupt

2020-08-19 Thread Robert Foley
e need to consider? Are we missing anything here? Signed-off-by: Robert Foley --- accel/tcg/cpu-exec.c | 8 +++- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c index e661635f06..499a8bdc5e 100644 --- a/accel/tcg/cpu-exec.c +++ b/accel/t

[PATCH v2 3/7] target/cris: add CRISCPUClass->do_interrupt_locked

2020-08-19 Thread Robert Foley
his approach. https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg00784.html https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg01517.html https://lists.gnu.org/archive/html/qemu-devel/2020-07/msg08731.html https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg00044.html Signed-off-by:

[PATCH v2 0/7] accel/tcg: remove implied BQL from cpu_handle_interrupt/exception path

2020-08-19 Thread Robert Foley
evel/2020-07/msg08731.html https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg00044.html This patch series is based on the per-CPU locks patch: https://lists.gnu.org/archive/html/qemu-devel/2020-06/msg05314.html Our most recent WIP tree is here: https://github.com/rf972/qemu/tree/interrupts_

Re: [PATCH v2 0/7] accel/tcg: remove implied BQL from cpu_handle_interrupt/exception path

2020-08-27 Thread Robert Foley
On Fri, 21 Aug 2020 at 06:56, Cornelia Huck wrote: > > > > As the first step in removing the BQL from this path, we will make > > changes to the core/common functions of cpu_handle_interrupt/exception > > to drop the holding of the BQL. The holding of the BQL is pushed down > > to the per-arch

Re: [PATCH 13/19] accel/tcg: Fixed tsan warnings.

2020-05-26 Thread Robert Foley
On Sat, 23 May 2020 at 16:06, Emilio G. Cota wrote: > > On Fri, May 22, 2020 at 12:07:49 -0400, Robert Foley wrote: > > For example: > > WARNING: ThreadSanitizer: data race (pid=35425) > > Write of size 4 at 0x7bbc00ac by main thread (mutexes: write M875): > >

Re: [PATCH 02/19] cpu: convert queued work to a QSIMPLEQ

2020-05-26 Thread Robert Foley
On Sun, 24 May 2020 at 06:21, Philippe Mathieu-Daudé wrote: > > On 5/22/20 6:07 PM, Robert Foley wrote: > > From: "Emilio G. Cota" > > > > Instead of open-coding it. > > Please use a full sentence (repeating the patch subject): > > "Conve

Re: [PATCH 18/19] target/arm: Fix tsan warning in cpu.c

2020-05-26 Thread Robert Foley
On Sat, 23 May 2020 at 13:18, Emilio G. Cota wrote: > > On Fri, May 22, 2020 at 23:36:18 +0100, Peter Maydell wrote: > > So is this: > > (a) a TSan false positive, because we've analysed the use > > of this struct field and know it's not a race because > > [details], but which we're

Re: [PATCH 00/19] Add Thread Sanitizer support to QEMU

2020-05-26 Thread Robert Foley
On Sat, 23 May 2020 at 17:36, Emilio G. Cota wrote: > > On Fri, May 22, 2020 at 12:07:36 -0400, Robert Foley wrote: > > This patch series continues the work done by Emilio Cota and others to add > > Thread Sanitizer (TSan) support to QEMU. > > > > The starting po

Re: [PATCH 14/19] util/async: Fixed tsan warnings

2020-05-26 Thread Robert Foley
On Sat, 23 May 2020 at 16:12, Emilio G. Cota wrote: > > On Fri, May 22, 2020 at 12:07:50 -0400, Robert Foley wrote: > > > QSIMPLEQ_INSERT_TAIL(>bh_slice_list, , next); > > > > while ((s = QSIMPLEQ_FIRST(>bh_slice_list))) { > > @@ -280,14 +

Re: [PATCH 14/19] util/async: Fixed tsan warnings

2020-05-26 Thread Robert Foley
Hi Stefan, On Tue, 26 May 2020 at 06:32, Stefan Hajnoczi wrote: > > On Fri, May 22, 2020 at 12:07:50PM -0400, Robert Foley wrote: > > For example: > > Atomic write of size 8 at 0x7b4800113c28 by main thread (mutexes: write > > M30): > > #0 __tsan_atomic64_

Re: [PATCH 01/19] configure: add --enable-tsan flag + fiber annotations for coroutine-ucontext

2020-05-26 Thread Robert Foley
On Sat, 23 May 2020 at 12:55, Philippe Mathieu-Daudé wrote: > > Hi Robert, > > On 5/22/20 6:07 PM, Robert Foley wrote: > > From: Lingfeng Yang > > > > We tried running QEMU under tsan in 2016, but tsan's lack of support for > > longjmp-based fibers was

Re: [PATCH 12/19] configure: added tsan support for blacklist.

2020-05-26 Thread Robert Foley
On Sat, 23 May 2020 at 13:27, Emilio G. Cota wrote: > > On Fri, May 22, 2020 at 12:07:48 -0400, Robert Foley wrote: > > Initially put several files into blacklist that were > > causing the most problems, namely bitops.c and bitmap.c. > > > > Signed-off-by: Robert

[PATCH v1 08/12] thread: add tsan annotations to QemuSpin

2020-05-29 Thread Robert Foley
From: "Emilio G. Cota" Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- include/qemu/thread.h | 39 --- 1 file changed, 36 insertions(+), 3 deletions(-) diff --git a/include/qemu/thread.h b/include/qemu/thread.h index e50a073889..

[PATCH v1 07/12] translate-all: call qemu_spin_destroy for PageDesc

2020-05-29 Thread Robert Foley
From: "Emilio G. Cota" The radix tree is append-only, but we can fail to insert a PageDesc if the insertion races with another thread. Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- accel/tcg/translate-all.c | 9 + 1 file changed, 9 insertions(+) diff --g

[PATCH v1 05/12] qht: call qemu_spin_destroy for head buckets

2020-05-29 Thread Robert Foley
From: "Emilio G. Cota" Signed-off-by: Robert Foley --- util/qht.c | 1 + 1 file changed, 1 insertion(+) diff --git a/util/qht.c b/util/qht.c index aa51be3c52..67e5d5b916 100644 --- a/util/qht.c +++ b/util/qht.c @@ -348,6 +348,7 @@ static inline void qht_chain_destroy(const struct

[PATCH v1 00/12] Add Thread Sanitizer support to QEMU

2020-05-29 Thread Robert Foley
tcg: call qemu_spin_destroy for tb->jmp_lock translate-all: call qemu_spin_destroy for PageDesc thread: add tsan annotations to QemuSpin Lingfeng Yang (1): configure: add --enable-tsan flag + fiber annotations for coroutine-ucontext Robert Foley (4): tests/docker: Added docker bu

[PATCH v1 02/12] cpu: convert queued work to a QSIMPLEQ

2020-05-29 Thread Robert Foley
Signed-off-by: Robert Foley --- cpus-common.c | 25 - cpus.c| 14 -- hw/core/cpu.c | 1 + include/hw/core/cpu.h | 6 +++--- 4 files changed, 24 insertions(+), 22 deletions(-) diff --git a/cpus-common.c b/cpus-common.c index 55d5df8923..

[PATCH v1 04/12] cputlb: destroy CPUTLB with tlb_destroy

2020-05-29 Thread Robert Foley
From: "Emilio G. Cota" I was after adding qemu_spin_destroy calls, but while at it I noticed that we are leaking some memory. Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- accel/tcg/cputlb.c | 15 +++ exec.c | 1 + include/exec/

[PATCH v1 10/12] include/qemu: Added tsan.h for annotations.

2020-05-29 Thread Robert Foley
These annotations will allow us to give tsan additional hints. For example, we can inform tsan about reads/writes to ignore to silence certain classes of warnings. We can also annotate threads so that the proper thread naming shows up in tsan warning results. Signed-off-by: Robert Foley

[PATCH v1 01/12] configure: add --enable-tsan flag + fiber annotations for coroutine-ucontext

2020-05-29 Thread Robert Foley
' configure flags. Signed-off-by: Lingfeng Yang Signed-off-by: Emilio G. Cota [cota: minor modifications + configure changes] Signed-off-by: Robert Foley [RF: Error out in configure if tsan not available, fix checkpatch warnings] --- configure | 41 + util/coroutine

[PATCH v1 11/12] util: Added tsan annotate for thread name.

2020-05-29 Thread Robert Foley
This allows us to see the name of the thread in tsan warning reports such as this: Thread T7 'CPU 1/TCG' (tid=24317, running) created by main thread at: Signed-off-by: Robert Foley --- util/qemu-thread-posix.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/util/qemu-thread-posix.c b

Re: [PATCH v1 01/12] configure: add --enable-tsan flag + fiber annotations for coroutine-ucontext

2020-05-29 Thread Robert Foley
On Fri, 29 May 2020 at 09:51, Eric Blake wrote: > > On 5/29/20 8:23 AM, Robert Foley wrote: > > From: Lingfeng Yang > > > > We tried running QEMU under tsan in 2016, but tsan's lack of support for > > longjmp-based fibers was a blocker: > >https://gr

[PATCH v1 09/12] tests/docker: Added docker build support for TSan.

2020-05-29 Thread Robert Foley
the TSAN suppresion file to disable certain cases of TSAN warnings. Cc: Fam Zheng Cc: Philippe Mathieu-Daudé Signed-off-by: Robert Foley --- tests/docker/Makefile.include | 2 + tests/docker/common.rc | 19 +++ tests/docker/dockerfiles/ubuntu2004.docker | 65

[PATCH v1 06/12] tcg: call qemu_spin_destroy for tb->jmp_lock

2020-05-29 Thread Robert Foley
From: "Emilio G. Cota" Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley [RF: Minor changes to fix some checkpatch errors] --- accel/tcg/translate-all.c | 10 +- include/tcg/tcg.h | 3 ++- tcg/tcg.c | 19 --- 3 files c

[PATCH v1 03/12] thread: add qemu_spin_destroy

2020-05-29 Thread Robert Foley
From: "Emilio G. Cota" It will be used for TSAN annotations. Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- include/qemu/thread.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/qemu/thread.h b/include/qemu/thread.h index d22848138e..e50a073

[PATCH v1 12/12] docs: Added details on TSan to testing.rst

2020-05-29 Thread Robert Foley
Adds TSan details to testing.rst. This includes background and reference details on TSan, and details on how to build and test with TSan both with and without docker. Signed-off-by: Robert Foley Reviewed-by: Emilio G. Cota --- docs/devel/testing.rst | 113

[PATCH v8 04/12] tests/vm: Pass --debug through for vm-boot-ssh.

2020-05-29 Thread Robert Foley
This helps debug issues that occur during the boot sequence. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Reviewed-by: Alex Bennée --- tests/vm/Makefile.include | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include index d9b34eae63

[PATCH v8 05/12] tests/vm: Add ability to select QEMU from current build.

2020-05-29 Thread Robert Foley
Added a new special variable QEMU_LOCAL=1, which will indicate to take the QEMU binary from the current build. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Reviewed-by: Alex Bennée --- tests/vm/Makefile.include | 4 tests/vm/basevm.py| 23 --- 2 files

[PATCH v8 06/12] tests/vm: allow wait_ssh() to specify command

2020-05-29 Thread Robert Foley
This allows for waiting for completion of arbitrary commands. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Reviewed-by: Alex Bennée --- tests/vm/basevm.py | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py index

[PATCH v8 09/12] tests/vm: Added a new script for centos.aarch64.

2020-05-29 Thread Robert Foley
centos.aarch64 creates a CentOS 8 image. Also added a new kickstart script used to build the centos.aarch64 image. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Reviewed-by: Alex Bennée --- tests/vm/Makefile.include| 3 +- tests/vm/centos-8-aarch64.ks | 51 tests/vm

[PATCH v8 08/12] tests/vm: Added a new script for ubuntu.aarch64.

2020-05-29 Thread Robert Foley
ubuntu.aarch64 provides a script to create an Ubuntu 18.04 VM. Another new file is also added aarch64vm.py, which is a module with common methods used by aarch64 VMs, such as how to create the flash images. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov --- configure

[PATCH v8 10/12] tests/vm: change scripts to use self._config

2020-05-29 Thread Robert Foley
This change converts existing scripts to using for example self.ROOT_PASS, to self._config['root_pass']. We made similar changes for GUEST_USER, and GUEST_PASS. This allows us also to remove the change in basevm.py, which adds __getattr__ for backwards compatibility. Signed-off-by: Robert Foley

[PATCH v8 00/12] tests/vm: Add support for aarch64 VMs

2020-05-29 Thread Robert Foley
of the --build-path argument instead. Robert Foley (12): tests/vm: pass args through to BaseVM's __init__ tests/vm: Add configuration to basevm.py tests/vm: Added configuration file support tests/vm: Pass --debug through for vm-boot-ssh. tests/vm: Add ability to select QEMU from current build

[PATCH v8 01/12] tests/vm: pass args through to BaseVM's __init__

2020-05-29 Thread Robert Foley
Adding the args parameter to BaseVM's __init__. We will shortly need to pass more parameters to the class so let's just pass args rather than growing the parameter list. Signed-off-by: Robert Foley Reviewed-by: Alex Bennée --- tests/vm/basevm.py | 13 ++--- 1 file changed, 6 insertions

[PATCH v8 07/12] tests/vm: Add common Ubuntu python module

2020-05-29 Thread Robert Foley
Add a common Ubuntu python module and make use of it with the ubuntu.i386 script. This is preparation for adding an Ubuntu script ubuntu.aarch64. Splitting out the common logic such as build_image() will reduce duplication. Signed-off-by: Robert Foley --- tests/vm/ubuntu.i386 | 46

[PATCH v8 12/12] tests/vm: Add workaround to consume console

2020-05-29 Thread Robert Foley
characters to be consumed. We also added the option of logging the console to a file. LOG_CONSOLE=1 will now log the output to a file. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Acked-by: Alex Bennée --- tests/vm/Makefile.include | 4 tests/vm/basevm.py| 17 +++-- 2

[PATCH v8 11/12] python/qemu: Add ConsoleSocket for optional use in QEMUMachine

2020-05-29 Thread Robert Foley
for basevm.py since there is a known issue where QEMU will hang waiting for console characters to be consumed. Cc: Eduardo Habkost Cc: Cleber Rosa Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Acked-by: Alex Bennée --- python/qemu/console_socket.py | 118 ++ python

[PATCH v8 03/12] tests/vm: Added configuration file support

2020-05-29 Thread Robert Foley
.yml. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Reviewed-by: Alex Bennée --- configure | 9 ++ tests/vm/Makefile.include | 6 tests/vm/basevm.py| 40 +++- tests/vm/conf_example_aarch64.yml | 51

[PATCH v8 02/12] tests/vm: Add configuration to basevm.py

2020-05-29 Thread Robert Foley
to vary on the VM being created or launched. This will for example allow for creating an aarch64 vm. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Reviewed-by: Alex Bennée --- tests/vm/basevm.py | 172 +++-- 1 file changed, 133 insertions(+), 39

Re: [PATCH v8 07/12] tests/vm: Add common Ubuntu python module

2020-06-01 Thread Robert Foley
On Sun, 31 May 2020 at 06:37, Philippe Mathieu-Daudé wrote: > > On 5/29/20 10:34 PM, Robert Foley wrote: > > Add a common Ubuntu python module and make use of > > it with the ubuntu.i386 script. > > This is preparation for adding an Ubuntu script > > ubuntu.aarc

Re: [PATCH v8 11/12] python/qemu: Add ConsoleSocket for optional use in QEMUMachine

2020-06-01 Thread Robert Foley
Hi Philippe, On Sun, 31 May 2020 at 07:22, Philippe Mathieu-Daudé wrote: > > On 5/29/20 10:34 PM, Robert Foley wrote: > > We add the ConsoleSocket object, which has a socket interface > > and which will consume all arriving characters on the > > socket, placing them

Re: [PATCH v8 08/12] tests/vm: Added a new script for ubuntu.aarch64.

2020-06-01 Thread Robert Foley
On Sun, 31 May 2020 at 06:54, Alex Bennée wrote: > > > Philippe Mathieu-Daudé writes: > > > Hi Robert, > > > > On 5/29/20 10:34 PM, Robert Foley wrote: > > >> diff --git a/configure b/configure > >> index d82de47fdd..8a286e75a5 100755 >

Re: [PATCH v8 12/12] tests/vm: Add workaround to consume console

2020-06-01 Thread Robert Foley
On Sun, 31 May 2020 at 07:27, Philippe Mathieu-Daudé wrote: > > On 5/29/20 10:34 PM, Robert Foley wrote: > > @@ -271,7 +276,9 @@ class BaseVM(object): > > args += self._data_args + extra_args + self._config['extra_args'] > > logging.debug("

[PATCH v9 7/9] tests/vm: change scripts to use self._config

2020-06-01 Thread Robert Foley
This change converts existing scripts to using for example self.ROOT_PASS, to self._config['root_pass']. We made similar changes for GUEST_USER, and GUEST_PASS. This allows us also to remove the change in basevm.py, which adds __getattr__ for backwards compatibility. Signed-off-by: Robert Foley

[PATCH v9 4/9] tests/vm: Add common Ubuntu python module

2020-06-01 Thread Robert Foley
Add a common Ubuntu python module and make use of it with the ubuntu.i386 script. This is preparation for adding an Ubuntu script ubuntu.aarch64. Splitting out the common logic such as build_image() will reduce duplication. Signed-off-by: Robert Foley Tested-by: Philippe Mathieu-Daudé

[PATCH v9 3/9] tests/vm: Added configuration file support

2020-06-01 Thread Robert Foley
.yml. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Reviewed-by: Alex Bennée --- configure | 9 ++ tests/vm/Makefile.include | 6 tests/vm/basevm.py| 40 +++- tests/vm/conf_example_aarch64.yml | 51

[PATCH v9 1/9] tests/vm: pass args through to BaseVM's __init__

2020-06-01 Thread Robert Foley
Adding the args parameter to BaseVM's __init__. We will shortly need to pass more parameters to the class so let's just pass args rather than growing the parameter list. Signed-off-by: Robert Foley Reviewed-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé

[PATCH v9 0/9] tests/vm: Add support for aarch64 VMs

2020-06-01 Thread Robert Foley
indentation of efi-aarch64 code in configure. - Updated file header for ubuntuvm.py - Few minor changes in ConsoleSocket - Updated some comments in basevm.py around the call to QEMUMachine regarding drain_console=True. Robert Foley (9): tests/vm: pass args through to BaseVM's __init__ tests

[PATCH v9 5/9] tests/vm: Added a new script for ubuntu.aarch64.

2020-06-01 Thread Robert Foley
ubuntu.aarch64 provides a script to create an Ubuntu 18.04 VM. Another new file is also added aarch64vm.py, which is a module with common methods used by aarch64 VMs, such as how to create the flash images. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov --- configure

[PATCH v9 2/9] tests/vm: Add configuration to basevm.py

2020-06-01 Thread Robert Foley
to vary on the VM being created or launched. This will for example allow for creating an aarch64 vm. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Reviewed-by: Alex Bennée --- tests/vm/basevm.py | 172 +++-- 1 file changed, 133 insertions(+), 39

[PATCH v9 9/9] tests/vm: Add workaround to consume console

2020-06-01 Thread Robert Foley
characters to be consumed. We also added the option of logging the console to a file. LOG_CONSOLE=1 will now log the output to a file. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Acked-by: Alex Bennée --- tests/vm/Makefile.include | 4 tests/vm/basevm.py| 21

[PATCH v9 6/9] tests/vm: Added a new script for centos.aarch64.

2020-06-01 Thread Robert Foley
centos.aarch64 creates a CentOS 8 image. Also added a new kickstart script used to build the centos.aarch64 image. Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Reviewed-by: Alex Bennée --- tests/vm/Makefile.include| 3 +- tests/vm/centos-8-aarch64.ks | 51 tests/vm

[PATCH v9 8/9] python/qemu: Add ConsoleSocket for optional use in QEMUMachine

2020-06-01 Thread Robert Foley
for basevm.py since there is a known issue where QEMU will hang waiting for console characters to be consumed. Cc: Eduardo Habkost Cc: Cleber Rosa Signed-off-by: Robert Foley Reviewed-by: Peter Puhov Acked-by: Alex Bennée Tested-by: Philippe Mathieu-Daudé --- python/qemu/console_socket.py | 110

[PATCH 11/19] accel/tcg: Fixed tsan warnings related to parallel_cpus

2020-05-22 Thread Robert Foley
(qemu-system-aarch64+0x01fb3e28) Cc: Richard Henderson Cc: Paolo Bonzini Signed-off-by: Robert Foley --- accel/tcg/cpu-exec.c| 4 ++-- cpus.c | 2 +- include/exec/exec-all.h | 2 +- linux-user/syscall.c| 4 ++-- 4 files changed, 6 insertions(+), 6 deletions(-) diff

[PATCH 10/19] include/qemu: Added tsan.h for annotations.

2020-05-22 Thread Robert Foley
These annotations will allow us to give tsan additional hints. For example, we can inform tsan about reads/writes to ignore to silence certain classes of warnings. We can also annotate threads so that the proper thread naming shows up in tsan warning results. Signed-off-by: Robert Foley

[PATCH 14/19] util/async: Fixed tsan warnings

2020-05-22 Thread Robert Foley
util/qemu-thread-posix.c:519:9 (qemu-system-aarch64+0xcde280) Cc: Stefan Hajnoczi Cc: Fam Zheng Signed-off-by: Robert Foley --- util/async.c | 11 +-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/util/async.c b/util/async.c index 1319eee3bc..51e306bf0c 100644 --- a/util

[PATCH 18/19] target/arm: Fix tsan warning in cpu.c

2020-05-22 Thread Robert Foley
: Richard Henderson Signed-off-by: Robert Foley --- target/arm/cpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/target/arm/cpu.c b/target/arm/cpu.c index 32bec156f2..cdb90582ee 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -75,7 +75,7 @@ static bool arm_cpu_has_work

[PATCH 09/19] tests/docker: Added docker build support for TSan.

2020-05-22 Thread Robert Foley
the TSAN suppresion file to disable certain cases of TSAN warnings. Cc: Fam Zheng Cc: Philippe Mathieu-Daudé Signed-off-by: Robert Foley --- tests/docker/Makefile.include | 2 + tests/docker/common.rc | 19 +++ tests/docker/dockerfiles/ubuntu2004.docker | 65

[PATCH 02/19] cpu: convert queued work to a QSIMPLEQ

2020-05-22 Thread Robert Foley
From: "Emilio G. Cota" Instead of open-coding it. While at it, make sure that all accesses to the list are performed while holding the list's lock. Reviewed-by: Richard Henderson Reviewed-by: Alex Bennée Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- cpu

[PATCH 06/19] tcg: call qemu_spin_destroy for tb->jmp_lock

2020-05-22 Thread Robert Foley
From: "Emilio G. Cota" Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley [RF: Minor changes to fix some checkpatch errors] --- accel/tcg/translate-all.c | 10 +- include/tcg/tcg.h | 3 ++- tcg/tcg.c | 19 --- 3 files c

[PATCH 08/19] thread: add tsan annotations to QemuSpin

2020-05-22 Thread Robert Foley
From: "Emilio G. Cota" Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- include/qemu/thread.h | 39 --- 1 file changed, 36 insertions(+), 3 deletions(-) diff --git a/include/qemu/thread.h b/include/qemu/thread.h index e50a073889..

[PATCH 07/19] translate-all: call qemu_spin_destroy for PageDesc

2020-05-22 Thread Robert Foley
From: "Emilio G. Cota" The radix tree is append-only, but we can fail to insert a PageDesc if the insertion races with another thread. Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- accel/tcg/translate-all.c | 9 + 1 file changed, 9 insertions(+) diff --g

[PATCH 17/19] util: Added tsan annotate for thread name.

2020-05-22 Thread Robert Foley
This allows us to see the name of the thread in tsan warning reports such as this: Thread T7 'CPU 1/TCG' (tid=24317, running) created by main thread at: Signed-off-by: Robert Foley --- util/qemu-thread-posix.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/util/qemu-thread-posix.c b

[PATCH 19/19] docs: Added details on TSan to testing.rst

2020-05-22 Thread Robert Foley
This includes details on how to build and test with TSan both inside a docker and outside. Signed-off-by: Robert Foley --- docs/devel/testing.rst | 72 ++ 1 file changed, 72 insertions(+) diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index

[PATCH 04/19] cputlb: destroy CPUTLB with tlb_destroy

2020-05-22 Thread Robert Foley
From: "Emilio G. Cota" I was after adding qemu_spin_destroy calls, but while at it I noticed that we are leaking some memory. Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- accel/tcg/cputlb.c | 15 +++ exec.c | 1 + include/exec/

[PATCH 01/19] configure: add --enable-tsan flag + fiber annotations for coroutine-ucontext

2020-05-22 Thread Robert Foley
' configure flags. Signed-off-by: Lingfeng Yang Signed-off-by: Emilio G. Cota [cota: minor modifications + configure changes] Signed-off-by: Robert Foley [RF: minor changes to clean up checkpatch warnings/errors] --- configure | 39 util/coroutine-ucontext.c | 97

[PATCH 00/19] Add Thread Sanitizer support to QEMU

2020-05-22 Thread Robert Foley
translate-all: call qemu_spin_destroy for PageDesc thread: add tsan annotations to QemuSpin Lingfeng Yang (1): configure: add --enable-tsan flag + fiber annotations for coroutine-ucontext Robert Foley (11): tests/docker: Added docker build support for TSan. include/qemu: Added tsan.h fo

[PATCH 03/19] thread: add qemu_spin_destroy

2020-05-22 Thread Robert Foley
From: "Emilio G. Cota" It will be used for TSAN annotations. Signed-off-by: Emilio G. Cota Signed-off-by: Robert Foley --- include/qemu/thread.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/qemu/thread.h b/include/qemu/thread.h index d22848138e..e50a073

<    1   2   3   4   5   6   >