On Fri, 2026-10-02 at 06:47 -0700, Rick Richard wrote:
> With Ztso, plain loads and stores follow RVTSO, but the barriers
> around them use TCG_MO_ALL. That also orders a store before a later
> load, which TSO doesn't require, and is expensive: on x86 hosts it
> adds a locked instruction around every load and store in a
> multi-threaded guest.
>
> The barrier after a load now only keeps later loads and stores
> behind it, and the barrier before a store only keeps earlier loads
> and stores ahead of it. That is all TSO needs.
>
> Signed-off-by: Rick Richard <[email protected]>
Thanks!
Applied to riscv-to-apply.next
Alistair
> ---
> I found this while looking into why riscv64 guests were so much
> slower than aarch64 ones under bsd-user on FreeBSD. The default -cpu
> max enables Ztso, and turning it off with ztso=false accounted for
> most of the difference.
>
> On an x86-64 host the patch made xz almost four times faster and cut
> the time of a clang compile by about 30%, the same as running with
> ztso=false. The output didn't change.
>
> I also tested it on a Raspberry Pi 3B+, which is arm64 and doesn't
> give TSO ordering in hardware, to make sure the weaker barriers still
> hold up there. Everything ran correctly, and xz was about 12%
> faster.
>
> target/riscv/tcg/insn_trans/trans_rvi.c.inc | 4 ++--
> target/riscv/tcg/insn_trans/trans_rvv.c.inc | 4 ++--
> 2 files changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/target/riscv/tcg/insn_trans/trans_rvi.c.inc
> b/target/riscv/tcg/insn_trans/trans_rvi.c.inc
> index 40b4c9aa..d31e4295 100644
> --- a/target/riscv/tcg/insn_trans/trans_rvi.c.inc
> +++ b/target/riscv/tcg/insn_trans/trans_rvi.c.inc
> @@ -424,7 +424,7 @@ static bool gen_load(DisasContext *ctx, arg_lb
> *a, MemOp memop)
> }
>
> if (ctx->ztso) {
> - tcg_gen_mb(TCG_MO_ALL | TCG_BAR_LDAQ);
> + tcg_gen_mb(TCG_MO_LD_LD | TCG_MO_LD_ST | TCG_BAR_LDAQ);
> }
>
> return out;
> @@ -485,7 +485,7 @@ static bool gen_store_tl(DisasContext *ctx,
> arg_sb *a, MemOp memop)
> TCGv data = get_gpr(ctx, a->rs2, EXT_NONE);
>
> if (ctx->ztso) {
> - tcg_gen_mb(TCG_MO_ALL | TCG_BAR_STRL);
> + tcg_gen_mb(TCG_MO_LD_ST | TCG_MO_ST_ST | TCG_BAR_STRL);
> }
>
> tcg_gen_qemu_st_tl(data, addr, ctx->mem_idx, memop);
> diff --git a/target/riscv/tcg/insn_trans/trans_rvv.c.inc
> b/target/riscv/tcg/insn_trans/trans_rvv.c.inc
> index a22e2cae..8c21ab87 100644
> --- a/target/riscv/tcg/insn_trans/trans_rvv.c.inc
> +++ b/target/riscv/tcg/insn_trans/trans_rvv.c.inc
> @@ -731,7 +731,7 @@ static bool ldst_us_trans(uint32_t vd, uint32_t
> rs1, uint32_t data,
> * around the loop.
> */
> if (is_store && s->ztso) {
> - tcg_gen_mb(TCG_MO_ALL | TCG_BAR_STRL);
> + tcg_gen_mb(TCG_MO_LD_ST | TCG_MO_ST_ST | TCG_BAR_STRL);
> }
>
> mark_vs_dirty(s);
> @@ -739,7 +739,7 @@ static bool ldst_us_trans(uint32_t vd, uint32_t
> rs1, uint32_t data,
> fn(dest, mask, base, tcg_env, desc);
>
> if (!is_store && s->ztso) {
> - tcg_gen_mb(TCG_MO_ALL | TCG_BAR_LDAQ);
> + tcg_gen_mb(TCG_MO_LD_LD | TCG_MO_LD_ST | TCG_BAR_LDAQ);
> }
>
> finalize_rvv_inst(s);
