Hi folks, So I've seen a few times now reports of latency spikes caused by IPIs, usually because of isolation misconfiguration, but only detected at the tail of end e.g. a 24h timerlat run.
It's not because those IPIs are rare, but rather that they don't by themselves cause a monitered CPU to reach the latency threshold, it's usually a combined interference that gets us there. I'd like to make it easier to detect such misconfigurations and thus IPIs hitting supposedly-isolated CPUs. I initially kludged a timerlat option to stop tracing as soon as an IPI was sent to a monitored CPU, regardless of the latency threshold. It sort of did the trick, but Tomáš convinced me timerlat wasn't really the place for that. Patches are available at https://gitlab.com/vschneid/linux.git -b mainline/tracing/osnoise-ipi-userspace/v4 Cheers, Valentin Revisions ========= v3 -> v4 ++++++++ o Rebased onto v7.2-rc4 o Fixed changelog typo v2 -> v3 ++++++++ o Dropped the short-form -i option to leave it free o Re-arranged top header printing o Fixed tracefs_event_file_write() return value handling o Changed IPI filtering to allow it to gracefully fail on older kernels o Added filter clearing for -e events to ensure a known state v1 -> v2 ++++++++ o Dropped the in-kernel osnoise_sample changes and made it all userspace Valentin Schneider (6): rtla/osnoise: Add IPI tracking cmdline option rtla/osnoise: Record IPI count in osnoise top rtla/osnoise: Leverage IPI event filters when tracing a subset of CPUs rtla/osnoise: Allow IPI filters to gracefully fail rtla: Unconditionally clean any pre-existing filters for user-provided events rtla/osnoise: Trace IPI events when recording a trace file Documentation/tools/rtla/rtla-osnoise-top.rst | 4 + tools/tracing/rtla/src/cli.c | 1 + tools/tracing/rtla/src/cli_p.h | 3 + tools/tracing/rtla/src/common.c | 2 +- tools/tracing/rtla/src/common.h | 3 +- tools/tracing/rtla/src/osnoise.c | 72 +++++++- tools/tracing/rtla/src/osnoise.h | 4 + tools/tracing/rtla/src/osnoise_top.c | 155 +++++++++++++++++- tools/tracing/rtla/src/trace.c | 4 + 9 files changed, 244 insertions(+), 4 deletions(-) -- 2.55.0
