Re: [PATCH v2 0/2] fix for #285

2023-03-19 Thread Emilio Cota
Ping. Any feedback on these two patches? https://patchew.org/QEMU/20230205163758.416992-1-c...@braap.org/ https://lore.kernel.org/qemu-devel/20230205163758.416992-1-c...@braap.org/ Happy to resend if needed. Thanks, Emilio On Fri, Feb 17, 2023 at 07:44:38 -0500, Emilio Cota

Re: [PATCH v2 0/7] plugin: fix clearing of plugin_mem_cbs on TB exit

2023-03-19 Thread Emilio Cota
; *: Add missing includes of qemu/error-report.h > *: Add missing includes of qemu/plugin.h > include/qemu: Split out plugin-event.h > include/qemu/plugin: Inline qemu_plugin_disable_mem_helpers Reviewed-by: Emilio Cota Thanks, Richard! Alex: is it too late to ad

Re: [PATCH 00/27] tcg: Simplify temporary usage

2023-03-01 Thread Emilio Cota
On Wed, Feb 15, 2023 at 20:15:37 -1000, Richard Henderson wrote: > On 2/10/23 02:35, Emilio Cota wrote: > > I ran yesterday linux-user SPEC06 benchmarks from your tcg-life branch. > > I do see perf regressions for two workloads (sjeng and xalancbmk). > > With perf(1

Re: [PATCH 1/2] tcg: Clear plugin_mem_cbs on TB exit

2023-03-01 Thread Emilio Cota
As I mentioned in the patch that is being superseded here I like this approach -- it is simpler and generates less code. I'd also like to see the plugin_gen_disable_mem_helpers function go away, and a mention somewhere that now we are intentionally not clearing cpu->plugin_mem_cbs until TB exit

Re: [PATCH] plugin: fix clearing of plugin_mem_cbs before TB exit

2023-03-01 Thread Emilio Cota
On Tue, Feb 28, 2023 at 10:50:26 -1000, Richard Henderson wrote: > On 2/21/23 18:32, Emilio Cota wrote: > > Currently we are wrongly accessing plugin_tb->mem_helper at > > translation time from plugin_gen_disable_mem_helpers, which is > > called before generating a TB

[PATCH] plugin: fix clearing of plugin_mem_cbs before TB exit

2023-02-21 Thread Emilio Cota
lt;-- after_insn clearing (dead) set_label $L0 mov_i64 tmp2,$0x0 st_i64 tmp2,env,$0xef68 <-- before_exit clearing (doesn't matter due to $L0) exit_tb $0x7f38c843 [...] Fixes: #1381 Signed-off-by: Emilio Cota --- accel/tcg/plugin-gen.c| 44 --

Re: [PATCH v2 0/2] fix for #285

2023-02-17 Thread Emilio Cota
Ping. This fixes a bug (admittedly with a big hammer) that affects users with heavily multi-threaded user-mode workloads. Thanks, Emilio On Sun, Feb 05, 2023 at 11:37:56 -0500, Emilio Cota wrote: > Changes since v1: > > - Add configure check to only use QTree if G

Re: [PATCH 00/27] tcg: Simplify temporary usage

2023-02-10 Thread Emilio Cota
Hi Richard, On Mon, Jan 30, 2023 at 10:59:07 -1000, Richard Henderson wrote: (snip) > With this, and by not recycling TEMP_LOCAL, we can get identical code > out of the backend even when changing the front end translators are > adjusted to use TEMP_LOCAL for everything. > > Benchmarking one test

Re: [PATCH 2/2] tcg: use QTree instead of GTree

2023-02-05 Thread Emilio Cota
On Mon, Jan 30, 2023 at 09:09:47 -1000, Richard Henderson wrote: > On 1/29/23 23:27, Daniel P. Berrangé wrote: > > On Sun, Jan 29, 2023 at 05:38:08PM -0500, Emilio Cota wrote: > > > Since this is a correctness issue, I think we should ship with qtree > > > and use it

[PATCH v2 1/2] util: import GTree as QTree

2023-02-05 Thread Emilio Cota
.68 276.05 266.75 251.65 104.93 QTree Traverse 278.31 (1.00x) 275.78 (1.00x) 266.42 (1.00x) 247.89 (0.99x) 104.58 (1.00x) Related: #285 Signed-off-by: Emilio C

[PATCH v2 0/2] fix for #285

2023-02-05 Thread Emilio Cota
Changes since v1: - Add configure check to only use QTree if Glib still implements gslice. If Glib doesn't, then we call Glib directly with inline functions. - Add TODO's so that in the future (i.e. when the minimum version of Glib that we use doesn't implement gslice) we remove QTree. - Add

[PATCH v2 2/2] tcg: use QTree instead of GTree

2023-02-05 Thread Emilio Cota
wstatus; waitpid(pid, , 0); } } void supragarble(unsigned depth) { if (depth == 0) return ; std::thread a(supragarble, depth-1); std::thread b(supragarble, depth-1); garble(); a.join(); b.join(); } int main() { supragarble(10); } ``` Fixes: #285 Signed-off-by: Emilio Cota

Re: [PATCH 1/2] util: import GTree as QTree

2023-01-29 Thread Emilio Cota
On Wed, Jan 11, 2023 at 12:08:26 +, Daniel P. Berrangé wrote: > First, I find the test to be a little unreliable the first few > times it is ran. I ran it in a loop 20 times and it got more > stable results. Looking at just the QTree lines I get something > typically like: Agreed, this is a

Re: [PATCH 2/2] tcg: use QTree instead of GTree

2023-01-29 Thread Emilio Cota
On Wed, Jan 25, 2023 at 15:58:25 +, Daniel P. Berrangé wrote: > On Wed, Jan 11, 2023 at 12:34:29PM +, Daniel P. Berrangé wrote: > > On Tue, Jan 10, 2023 at 10:55:36PM -0500, Emilio Cota wrote: > > > qemu-user can hang in a multi-threaded fork. One common > > > r

Re: [PATCH 2/2] tcg: use QTree instead of GTree

2023-01-29 Thread Emilio Cota
On Wed, Jan 11, 2023 at 12:10:53 +, Daniel P. Berrangé wrote: > On Tue, Jan 10, 2023 at 10:55:36PM -0500, Emilio Cota wrote: > > Performance impact on linux-user: > > - ~2% slowdown in spec06 > > - 1.05% slowdown in Nbench-int > > - 4.51% slowdown in Nbench-fp &

Re: [PATCH] linux-user,bsd-user: re-exec with G_SLICE=always-malloc

2023-01-29 Thread Emilio Cota
On Thu, Dec 01, 2022 at 10:49:27 +, Alex Bennée wrote: > Emilio Cota writes: > > On Tue, Oct 04, 2022 at 13:00:47 +0100, Daniel P. Berrangé wrote: > > (snip) > >> Can't say I especially like this but I'm out of other ideas for how > >> to guarantee a solutio

[PATCH v3 5/5] plugins: make qemu_plugin_user_exit's locking order consistent with fork_start's

2023-01-11 Thread Emilio Cota
To fix potential deadlocks as reported by tsan. Reviewed-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Emilio Cota --- plugins/core.c | 16 +++- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/plugins/core.c b/plugins/core.c index

[PATCH v3 0/5] tsan fixes

2023-01-11 Thread Emilio Cota
Changes since v2: - Add R-b's - patch 4/5: Fix incompatible pointer type errors - patch 4/5: Remove leftover helper Thanks, Emilio

[PATCH v3 4/5] util/qht: use striped locks under TSAN

2023-01-11 Thread Emilio Cota
ibc-start.c:392 (libc.so.6+0x29e3f) #24 _start (test-qht+0xdb44) Signed-off-by: Emilio Cota --- util/qht.c | 95 ++ 1 file changed, 81 insertions(+), 14 deletions(-) diff --git a/util/qht.c b/util/qht.c index 15866299e6..92c6b78759 100644

[PATCH v3 2/5] util/qht: add missing atomic_set(hashes[i])

2023-01-11 Thread Emilio Cota
We forgot to add this one in "a890643958 util/qht: atomically set b->hashes". Detected with tsan. Reviewed-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Alex Bennée Signed-off-by: Emilio Cota --- util/qht.c | 2 +- 1 file changed, 1 insertion(+), 1 dele

[PATCH v3 3/5] thread: de-const qemu_spin_destroy

2023-01-11 Thread Emilio Cota
Reviewed-by: Alex Bennée Signed-off-by: Emilio Cota --- include/qemu/thread.h | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/include/qemu/thread.h b/include/qemu/thread.h index 7c6703bce3..7841084199 100644 --- a/include/qemu/thread.h +++ b/include/qemu/thread.h

[PATCH v3 1/5] cpu: free cpu->tb_jmp_cache with RCU

2023-01-11 Thread Emilio Cota
T148 here: #0 0x7f49627b6460 in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:52 #1 0x5593da5ac057 in cpu_exec_unrealizefn ../cpu.c:180 #2 0x5593da81f851 (/home/cota/src/qemu/build/qemu-x86_64+0x484851) Signed-off-by: Emilio Cota --- accel/tcg/cpu-exec.c

[PATCH 2/2] tcg: use QTree instead of GTree

2023-01-10 Thread Emilio Cota
+-| | + ***| 13 ++ master-gcc12 qtree-gcc12 QEMU version Signed-off-by: Emilio Cota --- accel/tcg/tb-maint.c | 17

[PATCH 1/2] util: import GTree as QTree

2023-01-10 Thread Emilio Cota
) 62.25 (0.68x) Signed-off-by: Emilio Cota --- include/qemu/qtree.h | 119 +++ tests/bench/meson.build |4 + tests/bench/qtree-bench.c | 273 ++ tests/unit/meson.build|1

[PATCH 0/2] fix for #285

2023-01-10 Thread Emilio Cota
Context: https://gitlab.com/qemu-project/qemu/-/issues/285 So far the only fix that we have had posted on the list is https://lists.gnu.org/archive/html/qemu-devel/2022-10/msg00391.html by Daniel. The approach that I'm following here should have the same outcome, except that it doesn't change

Re: [PATCH v2 4/5] util/qht: use striped locks under TSAN

2023-01-10 Thread Emilio Cota
On Tue, Jan 10, 2023 at 20:58:01 +, Alex Bennée wrote: > Emilio Cota writes: (snip) > > +static inline void qht_do_if_first_in_stripe(const struct qht_map *map, > > + struct qht_bucket *b, > > +

Re: [PATCH 4/4] cpu-exec: assert that plugin_mem_cbs is NULL after execution

2023-01-09 Thread Emilio Cota
On Mon, Jan 09, 2023 at 13:52:36 +, Alex Bennée wrote: > Emilio Cota writes: > > --- a/accel/tcg/cpu-exec.c > > +++ b/accel/tcg/cpu-exec.c > > @@ -504,6 +504,7 @@ static void cpu_exec_exit(CPUState *cpu) > > if (cc->tcg_ops->cpu_exec_exit) { > >

[PATCH v2 0/5] tsan fixes

2023-01-09 Thread Emilio Cota
Changes since v1: - call g_free_rcu on tb_jmp_cache directly, and call tcg_exec_unrealizefn after calling cpu_list_remove(cpu) - add patch to de-const qemu_spin_destroy - remove wrappers for qht_do_if_first_in_stripe Thanks, Emilio

[PATCH v2 4/5] util/qht: use striped locks under TSAN

2023-01-09 Thread Emilio Cota
ibc-start.c:392 (libc.so.6+0x29e3f) #24 _start (test-qht+0xdb44) Signed-off-by: Emilio Cota --- util/qht.c | 101 + 1 file changed, 87 insertions(+), 14 deletions(-) diff --git a/util/qht.c b/util/qht.c index 15866299e6..70cc733d5d 100644

[PATCH v2 5/5] plugins: make qemu_plugin_user_exit's locking order consistent with fork_start's

2023-01-09 Thread Emilio Cota
To fix potential deadlocks as reported by tsan. Reviewed-by: Richard Henderson Signed-off-by: Emilio Cota --- plugins/core.c | 16 +++- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/plugins/core.c b/plugins/core.c index ccb770a485..728bacef95 100644 --- a/plugins

[PATCH v2 3/5] thread: de-const qemu_spin_destroy

2023-01-09 Thread Emilio Cota
Signed-off-by: Emilio Cota --- include/qemu/thread.h | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/include/qemu/thread.h b/include/qemu/thread.h index 7c6703bce3..7841084199 100644 --- a/include/qemu/thread.h +++ b/include/qemu/thread.h @@ -237,11 +237,10 @@ static

[PATCH v2 2/5] util/qht: add missing atomic_set(hashes[i])

2023-01-09 Thread Emilio Cota
We forgot to add this one in "a890643958 util/qht: atomically set b->hashes". Detected with tsan. Reviewed-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Emilio Cota --- util/qht.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/util

[PATCH v2 1/5] cpu: free cpu->tb_jmp_cache with RCU

2023-01-09 Thread Emilio Cota
T148 here: #0 0x7f49627b6460 in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:52 #1 0x5593da5ac057 in cpu_exec_unrealizefn ../cpu.c:180 #2 0x5593da81f851 (/home/cota/src/qemu/build/qemu-x86_64+0x484851) Signed-off-by: Emilio Cota --- accel/tcg/cpu-exec.c

Re: [PATCH 1/4] cpu: free cpu->tb_jmp_cache with RCU

2023-01-09 Thread Emilio Cota
On Sun, Jan 08, 2023 at 11:19:53 -0800, Richard Henderson wrote: > On 1/8/23 08:39, Emilio Cota wrote: (snip) > > diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c > > index 356fe348de..ca95d21528 100644 > > --- a/accel/tcg/cpu-exec.c > > +++ b/accel/tcg/cpu-ex

Re: [PATCH 3/4] util/qht: use striped locks under TSAN

2023-01-09 Thread Emilio Cota
On Sun, Jan 08, 2023 at 11:51:44 -0800, Richard Henderson wrote: > On 1/8/23 08:39, Emilio Cota wrote: > > +static inline void qht_bucket_lock_init(const struct qht_map *map, > > +struct qht_bucket *b) > > +{ > > +qht_d

[PATCH 4/4] cpu-exec: assert that plugin_mem_cbs is NULL after execution

2023-01-08 Thread Emilio Cota
Fixes: #1381 Signed-off-by: Emilio Cota --- accel/tcg/cpu-exec.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c index 356fe348de..de4ba6e23c 100644 --- a/accel/tcg/cpu-exec.c +++ b/accel/tcg/cpu-exec.c @@ -504,6 +504,7 @@ static void

[PATCH 2/4] translator: always pair plugin_gen_insn_{start, end} calls

2023-01-08 Thread Emilio Cota
Related: #1381 Signed-off-by: Emilio Cota --- accel/tcg/translator.c | 15 ++- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/accel/tcg/translator.c b/accel/tcg/translator.c index 061519691f..ef5193c67e 100644 --- a/accel/tcg/translator.c +++ b/accel/tcg/translator.c

[PATCH 3/4] tcg: exclude lookup_tb_ptr from helper instrumentation

2023-01-08 Thread Emilio Cota
It is internal to TCG and therefore we know it does not access guest memory. Related: #1381 Signed-off-by: Emilio Cota --- tcg/tcg.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/tcg/tcg.c b/tcg/tcg.c index da91779890..ee67eefc0c 100644 --- a/tcg/tcg.c +++ b/tcg

[PATCH 1/4] plugins: fix optimization in plugin_gen_disable_mem_helpers

2023-01-08 Thread Emilio Cota
earing cpu->plugin_mem_cbs) way too often, since it's not rare that the last instruction in the TB doesn't use helpers. Fix it by tracking a per-TB canary. While at it, expand documentation. Related: #1381 Signed-off-by: Emilio Cota --- accel/tcg/plugin-gen.c | 26 ++

[PATCH 0/4] plugin patches to fix #1381

2023-01-08 Thread Emilio Cota
Hi, These are the plugin fixes that I mentioned here: https://lists.gnu.org/archive/html/qemu-devel/2022-12/msg02865.html They should fix https://gitlab.com/qemu-project/qemu/-/issues/1381 Thanks, Emilio

[PATCH 3/4] util/qht: use striped locks under TSAN

2023-01-08 Thread Emilio Cota
ibc-start.c:392 (libc.so.6+0x29e3f) #24 _start (test-qht+0xdb44) Signed-off-by: Emilio Cota --- util/qht.c | 107 ++--- 1 file changed, 93 insertions(+), 14 deletions(-) diff --git a/util/qht.c b/util/qht.c index 15866299e6..6174533f10 100644

[PATCH 2/4] util/qht: add missing atomic_set(hashes[i])

2023-01-08 Thread Emilio Cota
We forgot to add this one in "a890643958 util/qht: atomically set b->hashes". Detected with tsan. Signed-off-by: Emilio Cota --- util/qht.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/util/qht.c b/util/qht.c index 065fc501f4..15866299e6 100644 --- a/uti

[PATCH 4/4] plugins: make qemu_plugin_user_exit's locking order consistent with fork_start's

2023-01-08 Thread Emilio Cota
To fix potential deadlocks as reported by tsan. Signed-off-by: Emilio Cota --- plugins/core.c | 16 +++- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/plugins/core.c b/plugins/core.c index ccb770a485..728bacef95 100644 --- a/plugins/core.c +++ b/plugins/core.c

[PATCH 0/4] tsan fixes

2023-01-08 Thread Emilio Cota
Hi, Here are some fixes for tsan issues that I've encountered. The most important patch is 3/4, which allows us to run tsan for non-trivial workloads. Thanks, Emilio

[PATCH 1/4] cpu: free cpu->tb_jmp_cache with RCU

2023-01-08 Thread Emilio Cota
T148 here: #0 0x7f49627b6460 in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:52 #1 0x5593da5ac057 in cpu_exec_unrealizefn ../cpu.c:180 #2 0x5593da81f851 (/home/cota/src/qemu/build/qemu-x86_64+0x484851) Signed-off-by: Emilio Cota --- accel/tcg/cpu-exec.c

Re: Plugin Memory Callback Debugging

2023-01-06 Thread Emilio Cota
On Fri, Jan 6, 2023, 5:31 AM Alex Bennée wrote: > Are you going to be able to post the patches soon? I'd like to get the > fixes in as early in the cycle as possible. > I intend to post this series on Sunday. Thanks, Emilio

Re: Plugin Memory Callback Debugging

2022-12-17 Thread Emilio Cota
On Tue, Nov 29, 2022 at 15:37:51 -0500, Aaron Lindsay wrote: (snip) > > Does this hint that there are cases where reset cpu->plugin_mem_cbs to NULL > > is > > getting optimized away, but not the code to set it in the first place? > > Is there anyone who could help take a look at this from the

Re: [PATCH] linux-user,bsd-user: re-exec with G_SLICE=always-malloc

2022-11-30 Thread Emilio Cota
On Tue, Oct 04, 2022 at 13:00:47 +0100, Daniel P. Berrangé wrote: (snip) > Can't say I especially like this but I'm out of other ideas for how > to guarantee a solution. Users can't set env vars prior to launching > QEMU user emulators when using binfmt. An alternative is to not use GSlice

Re: Plugin Memory Callback Debugging

2022-11-15 Thread Emilio Cota
On Tue, Nov 15, 2022 at 22:36:07 +, Alex Bennée wrote: > This is exactly the sort of thing rr is great for. Can you trigger it in > that? > > https://rr-project.org/ The sanitizers should also help. For TLB flush tracing, defining DEBUG_TLB at the top of cputlb.c might be useful.

Re: [PATCH v3 23/26] tests/plugin: allow libinsn.so per-CPU counts

2022-02-08 Thread Emilio Cota
(Sorry if this comes out garbled, I'm on a web editor not a proper email client) On Fri, Feb 4, 2022 at 3:49 PM Alex Bennée wrote: > +typedef struct { > +uint64_t last_pc; > +uint64_t insn_count; > +} InstructionCount; This will need padding to take up a cache line. > +static

Re: Suggestions for TCG performance improvements

2021-12-02 Thread Emilio Cota
On Thu, Dec 2, 2021 at 4:47 AM Vasilev Oleg wrote: > The mentioned paper[4] also describes other possible improvements. > Some of those are already implemented (such as victim TLB and dynamic > size for TLB), but others are not (e.g. TLB lookup uninlining and > set-associative TLB layer). Do you