On Sat, 07 Nov 2020 13:37:01 +0100 "Thomas Rosenstein" <[email protected]> wrote:
> On 6 Nov 2020, at 21:19, Jesper Dangaard Brouer wrote: > > > On Fri, 06 Nov 2020 18:04:49 +0100 > > "Thomas Rosenstein" <[email protected]> wrote: > > > >> On 6 Nov 2020, at 15:13, Jesper Dangaard Brouer wrote: > >> > >> [...] > > > >>> Have you tried to use 'perf record' to observe that is happening on > >>> the system while these latency incidents happen? (let me know if > >>> you want some cmdline hints) > >> > >> Haven't tried this yet. If you have some hints what events to monitor > >> I'll take them! > > > > Okay to record everything (-a) on the system and save call-graph (-g), > > and run for 5 seconds (via profiling the sleep function). > > > > # perf record -g -a sleep 5 > > > > To view the result the simply use the 'perf report', but likely you > > want to use option --no-children as you are profiling the kernel (and > > not a userspace program you want to have grouped 'children' by). I > > also include the CPU column via '--sort cpu,comm,dso,symbol' and you > > can select/zoom-in-on a specific CPU via '-C zero-indexed-cpu-num'. > > > > # perf report --sort cpu,comm,dso,symbol --no-children > > > > When we ask you to provide the output, you can use the --stdio option, > > and provide txt-info via a pastebin link as it is very long. > > Here is the output from kernel 3.10_1127 (I updated to the really newest > in that branch): https://pastebin.com/5mxirXPw > Here is the output from kernel 5.9.4: https://pastebin.com/KDZ2Ei2F Did the latency issue happen during this this perf recording? (it is obviously important that you record during the issue) Finding a single latency event/issue in perf.data is like looking in for needle in a haystack. It does give hints and e.g. tell me details like what part of the kernel are used/activated. I can see that you have likely configured XPS (Transmit Packet Steering). Is this on purpose? -- Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer _______________________________________________ Bloat mailing list [email protected] https://lists.bufferbloat.net/listinfo/bloat
