Hi all, This RFC introduces an in-kernel PSI auto monitor aimed at improving root-cause visibility for resource pressure events in Linux systems.
Motivation: PSI already provides an excellent mechanism to detect CPU, memory and I/O pressure and includes trigger-based notifications via pollable interfaces. However, it deliberately avoids attributing pressure to individual tasks. In real-world systems, this creates a gap: when a PSI trigger fires, users still need to determine *which tasks caused the stall* by combining multiple tools (top, meminfo, vmstat, perf, tracing, etc.), often after the event has already passed. This process becomes particularly difficult during: - transient bursts of pressure - system boot or early initialization before user space - PREEMPT_RT or latency-sensitive workloads - heavily loaded embedded systems where user space is delayed - small resource-constraints minimal system - production system where most debugging interface are disabled Proposal: This patch introduces an optional in-kernel PSI auto monitor that: - periodically samples PSI signals - detects threshold breaches - captures top contributing tasks at that moment - emits trace events and kernel logs for analysis The design goal is **low-latency attribution at the source of truth**, without relying on user-space daemons or polling loops. Why in-kernel? While similar logic can be implemented in user space, there are inherent limitations: - scheduling delays under high pressure - risk of missing short-lived spikes - dependency on continuous polling or daemons - difficulty deploying in early boot or minimal environments In contrast, the in-kernel approach: - observes PSI signals without scheduling latency - captures contributors exactly at threshold breach - works during early boot and degraded system states - avoids duplicating logic across multiple user-space tools - easy configurable even in runtime - captures all sorts of information during same timestamp Design Highlights: - Does not modify PSI fast paths - Optional (CONFIG_PSI_AUTO_MONITOR) - Runtime configurable thresholds and interval - Uses existing kernel accounting (task runtime, RSS, I/O stats) - Provides structured tracepoints for post-processing - Lightweight and intended for diagnostic use - Idea is similar to, when OOM occurs dump contending tasks Reviews and Assistance: The core idea is mine. However, I have taken few assistance from AI for review and enhancement. I have done extensive review and suggestion using ChatGPT and Copilot. The commit message and this cover letter were also prepared by Copilot. I have done self-review and corrective actions accordingly. Experimental Validation: The feature has been evaluated on multiple ARM64 platforms (Cortex-A53, A55) across different kernels and storage setups. Extensive experiments has been carried out with multiple workloads. Some tools and logs are shared here: https://github.com/pintuk/KERNEL/tree/master/PSI_WORK Test scenarios include: - CPU/memory/IO stress workloads both on eMMC and NAND - system boot tracing (no external tools) - mixed workloads (stress-ng, workqueues, user/kernel threads, processes) - PREEMPT_RT cyclictest correlation with real workloads Results show: - consistent identification of top resource contributors - improved root-cause visibility compared to user-space-only methods - ability to capture transient hotspots during boot and runtime - correlation of latency spikes with system pressure Papers and Reference: The paper is presented in Open Source Summit India - 2026: https://ossindia2026.sched.com/event/2KNI4/introducing-in-kernel-psi-auto-monitor-feature-pintu-kumar-agarwal-qualcomm?iframe=yes&w=100%&sidebar=yes&bg=no https://hosted-files.sched.co/ossindia2026/19/OSS-IND-26-PSI-Auto-Monitor.pdf The initial idea was also presented in LPC-2024: https://lpc.events/event/18/contributions/1884/attachments/1439/3069/LPC2024_PIntu_PSI.pdf Open Questions (RFC): Feedback is especially appreciated on: - whether this functionality belongs in-kernel vs user-space - interface choice (sysfs vs tracefs/debugfs alternatives) - scoring heuristic (CPU/RSS/IO weighting) - potential reuse or extension of existing PSI interfaces - cgroup-aware extensions for future work Future Work: - finer-grained PSI window integration - IRQ pressure support - cgroup-based attribution - improved tracing/export interfaces - optional integration with user-space analysis tools Thanks for your time, and I’d really appreciate feedback. Regards, Pintu Kumar Agarwal Pintu Kumar Agarwal (1): psi: Introduce in-kernel PSI auto monitor feature include/trace/events/psi_monitor.h | 53 +++++ init/Kconfig | 16 ++ kernel/sched/build_utility.c | 4 + kernel/sched/psi_monitor.c | 307 +++++++++++++++++++++++++++++ 4 files changed, 380 insertions(+) create mode 100644 include/trace/events/psi_monitor.h create mode 100644 kernel/sched/psi_monitor.c -- 2.34.1
