Rafael.Vanoni at Sun.COM wrote: > Li, Aubrey wrote: >> Rafael.Vanoni wrote: >> >>> On 12/08/08 08:34, Rafael Vanoni wrote: >>>> Li, Aubrey wrote: >>>>> Li, Aubrey wrote: >>>>> >>>>>> Hi Rafael, >>>>>> >>>>>> Rafael.Vanoni wrote: >>>>>> >>>>>>> Here's a patch to identify the function causing a xcall. Here's >>>>>>> an example output of the change this introduces: >>>>>>> >>>>>>> from >>>>>>> >>>>>>> sched : <cross calls> >>>>>>> >>>>>>> to >>>>>>> >>>>>>> sched : <cross call> unix`dtrace_xcall_func >>>>>>> >>>>>>> >>>>>>> I'm not sure whether this should be the default option or only >>>>>>> for -v. So please let me know what you think. >>>>>>> >>>>>>> Thanks, >>>>>>> Rafael >>>>>> I saw this patch in the powertop repo. >>>>>> Previously, the cross call number on my idle system is: >>>>>> ============= Wakeups-from-idle per second: 1621.6 interval: >>>>>> 5.0s >>>>>> Top causes for wakeups: >>>>>> 76.1% (1234.3) sched :<cross calls> >>>>>> >>>>>> --------snip-------- >>>>>> 0.9% ( 14.0) dtrace :<cross calls> >>>>>> ============ >>>>>> >>>>>> with this patch, the number decreased too much, see below: >>>>>> ============== Wakeups-from-idle per second: 1592.7 interval: >>>>>> 5.0s Top causes for wakeups: >>>>>> 11.2% (178.8) sched :<cross call> >>>>>> unix`dtrace_xcall_func >>>>>> ----snip---- >>>>>> 1.0% ( 16.0) dtrace :<cross call> >>>>>> unix`dtrace_xcall_func ============== >>>>>> >>>>>> I believe the actual cross call shouldn't vary too much in idle. >>>>>> Any thoughts? >>>>>> >>>>> hmm..., on x86, the patch missed xc_capture_cpus() function. on >>>>> sparc, it missed too much. It looks like "sysinfo:::xcalls" is far >>>>> more than "fbt::xc_common:entry". >>>>> >>>>> If we want to implement 6781149, we probably need to improve the >>>>> probe. >>>>> >>>>> Thanks, >>>>> -Aubrey >>> Hi Aubrey >>> >>> Could you test this patch on your system? I've added firings of >>> fbt::send_dirint that were not accounted for by fbt::xc_common. >>> Unfortunately, I don't see a way of getting the function causing >>> the xcalls in this situation. >>> >>> I chose to use send_dirint because it relates to a patch I'm sending >>> in a few minutes to run PowerTOP on a specific cpu. >>> >> >> I don't think the patch does the right way to fix this problem. >> Previously, we are using "sysinfo:::xcalls", this probe comes from >> CPU_STATS_ADDQ(CPU, sys, xcall, xxx) >> >> Let's take a look at where has this call: >> ======================================================== >> $$work/pad-gate/usr/src/uts$ find . -name *.c | xargs grep >> CPU_STATS_ADDQ | grep xcalls >> >> ./sun4v/os/mach_cpu_states.c: CPU_STATS_ADDQ(CPU, sys, xcalls, 1); >> ./sun4v/os/mach_cpu_states.c: CPU_STATS_ADDQ(CPU, sys, xcalls, >> shipped); ./sun4u/cpu/us3_cheetah.c: CPU_STATS_ADDQ(CPU, sys, >> xcalls, shipped); ./sun4u/cpu/us3_cheetah.c: >> CPU_STATS_ADDQ(CPU, sys, xcalls, ncpuids); >> ./sun4u/cpu/us3_jalapeno.c: CPU_STATS_ADDQ(CPU, sys, xcalls, >> shipped); ./sun4u/cpu/opl_olympus.c: CPU_STATS_ADDQ(CPU, sys, >> xcalls, shipped); ./sun4u/cpu/opl_olympus.c: >> CPU_STATS_ADDQ(CPU, sys, xcalls, ncpuids); >> ./sun4u/cpu/opl_olympus.c: CPU_STATS_ADDQ(CPU, sys, xcalls, 1); >> ./sun4u/cpu/us3_cheetahplus.c: CPU_STATS_ADDQ(CPU, sys, xcalls, >> shipped); ./sun4u/cpu/us3_cheetahplus.c: CPU_STATS_ADDQ(CPU, sys, >> xcalls, ncpuids); ./sun4u/cpu/spitfire.c: CPU_STATS_ADDQ(CPU, sys, >> xcalls, 1); ./sun4u/cpu/us3_common.c: CPU_STATS_ADDQ(CPU, sys, >> xcalls, 1); >> >> ./i86pc/os/x_call.c: CPU_STATS_ADDQ(CPU, sys, xcalls, 1); >> ./i86pc/os/x_call.c: CPU_STATS_ADDQ(CPU, sys, >> xcalls, 1); > > Thanks for catching this. For xc_capture_cpus() on x86, I think > send_dirint() is enough. It's actually more than what sysinfo:::xcalls > reports, since send_dirint() is called for each CPU that gets > xcall'ed, and sysinfo:::xcalls only once early in xc_capture_cpus() - > which looks > like a bug. > > For SPARC systems, we currently don't offer a sun4u build of PowerTOP > (which we might want to change (?)). So, for now, we can leave > that aside. > > For sun4v, all xcalls are init'ed with init_mondo(), which > specifies the > function and arguments. So I think there are two options: > > (a) tie fbt::xc_common firings to fbt::send_dirint on x86 and > fbt::init_mondo calls to fbt::shipit on sun4v. > > or > > (b) tie fbt::xc_common and fbt::init_mondo to sysinfo:::xcalls > > On both cases we'll need to catch free/untied firings of > sysinfo:::xcalls for specific situations. > > The patch I sent for the --cpu option needs fbt::send_dirint (and > fbt::shipt) to catch incoming xcalls. So I'd suggest using (a) above. > > What do you think ? >
a) sounds good. But can we still get the xcall function name if we use send_dirint? I have a quick try and the xcalls number still quite different from the current implementation. Can you confirm it on a 8-core platform? As for SPARC, I didn't have a platform to try, you have to be on your own, :) Thanks, -Aubrey
