On Wed, Oct 14, 2026 at 05:58:42PM +0200, Roman Storozhenko wrote: > Hi Jisheng, > > Thanks for the clarification. > > Since Radim's tracepoint code-generation results are now part of this > discussion, I've added him to Cc in case I misinterpret any aspect of > that issue. > > I agree with you on one point regarding the compiler overhead. Since > the kernel supports older GCC versions, we have to take the code > generated by those supported compilers into account today. So > describing the issue simply as temporary was too strong. > > However, I think there are two separate questions here: whether the > event exposes useful information, and whether the disabled-path cost > of its current implementation is acceptable. > > Regarding kprobes, most of the raw fields in this event are indeed > already arguments of `__flush_tlb_range()`. My point is not that these > values are impossible to obtain dynamically. > > Kprobes can be used to observe a very large part of the kernel. If the > ability to reconstruct an event with kprobes were by itself sufficient > reason not to add a tracepoint, the same reasoning would apply to many > existing tracepoints. > > The distinction I see here is between observing implementation details > and exposing the semantic event directly. > > For this case, the interesting information is not only `start`, > `size`, `stride`, `cmask`, etc., but their correlation with the path > Linux selected for that particular request: > > local > SBI RFENCE > cross-CPU call > > Calculating the local case from `cmask` may be simple with the current > implementation, but the tracing program still has to reproduce > knowledge of the selection logic. Likewise, observing downstream > functions ties the tracing program to the current call graph. > > I also don't think `/proc/interrupts` provides equivalent information here. > > The RISC-V IPI statistics are aggregate counters. In particular, Linux > accounts `IPI_CALL_FUNC` as "Function call interrupts", while SBI > RFENCE is handled below Linux by firmware; for example, OpenSBI > implements the remote TLB operation using its own `IPI_TLB` event.
I think you misunderstood something. A riscv platform uses either SBI RFENCE or IPI for tlb shootdown, and won't change. So you can know whether the system is SBI RFENCE powered or IPI powered by cat /proc/interrupts. It's much simpler. > > So checking `/proc/interrupts` can certainly be useful as a sanity > check in a controlled workload: an increase in function-call IPIs can > indicate use of the Linux cross-CPU path. > > But it cannot associate that IPI activity with a particular TLB > invalidation request, and it cannot correlate it with the range, ASID, > stride and target cpumask. Other users of the call-function IPI also > contribute to the same counters. In a concurrent workload this becomes > an inference rather than an observation of the TLB path decision. > > That per-request correlation is what I am trying to expose with the > tracepoint. > > Regarding the performance argument, I agree that the disabled path > must be cheap and that the current RISC-V tracepoint code-generation > issue is relevant when evaluating the patch. > > I would not claim that all tracepoints have exactly the same overhead. > My narrower point is that the issue referenced in Radim's thread is a > general interaction between RISC-V tracepoints and compiler code > generation rather than something specific to the semantics of this > event. > > If the generated disabled path is considered too expensive, that is a > valid implementation concern and may require improvement or > mitigation. But I don't think it shows that the TLB path-selection > event itself is redundant because the information can be reconstructed > with kprobes. > > There is also no contradiction, in my view, between using the event > for performance analysis and the event having overhead when enabled. > Tracing normally perturbs the workload when it is enabled. The > intended use here is primarily to understand things such as the > distribution of local/SBI/IPI decisions and the request context > associated with those decisions, rather than to use the tracepoint > itself for cycle-accurate measurement of the cost of a TLB flush. > > The important requirement is therefore that the disabled path be > sufficiently cheap. > > I think the PowerPC TLB tracepoints also remain a useful precedent > here: they show that permanent architecture-specific TLB observability > can be worthwhile when the architecture-specific information is > useful. > > So I agree that convenience, maintenance cost and disabled-path > overhead all need to be considered. I just don't think that kprobes or > `/proc/interrupts` provide the same abstraction as recording the > path-selection decision directly. > > Thanks, > Roman > > On Wed, Oct 14, 2026 at 7:25 AM Jisheng Zhang <[email protected]> wrote: > > > > On Sun, Oct 11, 2026 at 02:43:05PM +0200, Roman Storozhenko wrote: > > > On Sat, Oct 10, 2026 at 6:44 AM Jisheng Zhang <[email protected]> wrote: > > > > > > > > On Sat, Oct 03, 2026 at 10:54:44AM +0200, Roman Storozhenko wrote: > > > > > On Fri, Oct 2, 2026 at 6:04 PM Jisheng Zhang <[email protected]> > > > > > wrote: > > > > > > > > > > > > On Sun, Aug 30, 2026 at 04:36:37PM +0200, Roman 'Hedin' Storozhenko > > > > > > wrote: > > > > > > > Make RISC-V TLB flush path selection observable. Record whether > > > > > > > Linux > > > > > > > handles an invalidation locally, delegates it to SBI RFENCE, or > > > > > > > executes > > > > > > > it through a cross-CPU call, so MM activity can be correlated > > > > > > > with the > > > > > > > RISC-V, firmware, or Linux cross-CPU path carrying the request. > > > > > > > > > > > > I didn't see too much usefullness of the trace here. why not use > > > > > > kprobe > > > > > > instead? can you plz show a real usage example of the trace? Which > > > > > > bug/performance problem can be solved conveniently with this trace > > > > > > only? > > > > > > > > > > > > From another side, except x86, other architectures don't have the > > > > > > trace > > > > > > in the TLB path. > > > > > > > > > > > > > > > > Hi Jisheng, > > > > > > > > > > Thanks for the review. I agree that the current commit message > > > > > describes > > > > > the intended use rather abstractly. > > > > > > > > > > A kprobe can certainly be used to investigate this. My motivation > > > > > for a tracepoint is not that dynamic tracing cannot obtain the > > > > > information, but that doing so requires knowledge of the current > > > > > implementation and reconstruction of a semantic decision made inside > > > > > __flush_tlb_range() / flush_tlb_all(). > > > > > > > > I prefer aligning with other architectures(except x86), I.E not > > > > add this kind of trace. However, I would leave the decision to riscv > > > > maintainers. > > > > > > > > FYI, you may notice that the riscv trace overhead is not trivial per > > > > https://lore.kernel.org/linux-riscv/[email protected]/ > > > > > > Hi Jisheng, > > > > > > Thanks for the feedback. > > > > > > Regarding the comparison with other architectures, thanks to Michael > > > Ellerman for pointing me to the PowerPC TLB tracing when we happened > > > to discuss this at LPC 2026. > > > > > > PowerPC defines the powerpc:tlbie and powerpc:tlbia trace events in > > > arch/powerpc/include/asm/trace.h, and the corresponding trace calls > > > are used in the PowerPC TLB invalidation paths. > > > > > > These events are not identical to the proposed RISC-V event: the > > > PowerPC events primarily describe architecture invalidation > > > operations, whereas this patch records the path selected by Linux > > > before dispatch. However, I think they provide useful precedent for > > > architecture-specific TLB observability beyond x86. > > > > > > Regarding kprobes, I agree that dynamic tracing can be used to > > > investigate this. > > > > > > My concern is that obtaining the same information requires knowledge > > > of the current implementation and reconstruction of the decision made > > > inside __flush_tlb_range() / flush_tlb_all(). > > > > > > For example, probing __flush_tlb_range() gives us the incoming > > > request, but does not directly tell us whether Linux selected the > > > local flush, SBI RFENCE, or cross-CPU-call path. A tracing tool has to > > > probe and correlate implementation-specific downstream functions, or > > > reproduce the selection logic itself. > > > > Hmm, let's look at where the tracepoints added: > > void __flush_tlb_range(struct mm_struct *mm, > > const struct cpumask *cmask, > > unsigned long start, unsigned long size, > > unsigned long stride) > > void flush_tlb_all(void) > > > > most of the information you exported with the tracepoints are just > > parameters of flush_tlb_range. > > So kprobe or function trace can also get most of them, while whether > > local or not needs to calcuate the cmask, but getting this info is > > also easy via. adding one LoC to the kprobe routine. > > > > The tlb invalidation is frequently executed, so we need to carefully > > evaluate the convience VS maitanince offort and performance overhead when > > adding permanent tracepoints, given current riscv tracepoint overhead > > isn't trivial. > > > > BTW: whether SBI RFENCE or cross-cpu-call is much easier. Just search > > IPI in /proc/interrupts. No need to kprobe/function trace/tracepoint > > > > > > > > The proposed event records this semantic decision directly at the > > > point where Linux makes it. Users therefore do not need to depend on > > > the current helper names, call graph, or exact implementation of the > > > selection logic. > > > > > > Regarding the tracepoint overhead issue, I agree that disabled-path > > > overhead is important, especially in the TLB flush path. > > > > > > However, I see the referenced SBI tracepoint case primarily as a > > > RISC-V compiler/code-generation issue rather than as an argument > > > against providing this kind of observability. > > > > > > The referenced discussion itself shows that the generated code depends > > > significantly on compiler behavior, and that this is an area being > > > improved. The undesirable overhead is not an inherent semantic > > > requirement of the tracepoint. > > > > > > Even if the same code-generation problem currently affects this > > > tracepoint, I don't think a temporary compiler limitation should by > > > itself determine whether the tracepoint is useful or whether this > > > observability belongs in the kernel. > > > > > > If the compiler currently generates unacceptable disabled-path > > > overhead, that is certainly something that should be improved or > > > mitigated in the implementation. But I would separate that issue from > > > the question of whether exposing the RISC-V TLB path-selection > > > decision is useful. > > > > > > > But we need to live with it. Per the Radim email, gcc14 can still > > see the issue, given the minimum gcc version of kernel building > > requirement is gcc8 now, gcc14 has a long time to bailout. > > > > And another tricky point here is > > one of your motivations of adding the tracepoint is to profile then > > improve the tlb related code path performance, but adding the > > tracepoint brings non-trivial overhead to tlb code path. > > > > -- > Kind regards, > Roman 'Hedin' Storozhenko
