Li, Aubrey wrote: > Rafael.Vanoni wrote: > >> On 12/08/08 08:34, Rafael Vanoni wrote: >>> Li, Aubrey wrote: >>>> Li, Aubrey wrote: >>>> >>>>> Hi Rafael, >>>>> >>>>> Rafael.Vanoni wrote: >>>>> >>>>>> Here's a patch to identify the function causing a xcall. Here's an >>>>>> example output of the change this introduces: >>>>>> >>>>>> from >>>>>> >>>>>> sched : <cross calls> >>>>>> >>>>>> to >>>>>> >>>>>> sched : <cross call> unix`dtrace_xcall_func >>>>>> >>>>>> >>>>>> I'm not sure whether this should be the default option or only for >>>>>> -v. So please let me know what you think. >>>>>> >>>>>> Thanks, >>>>>> Rafael >>>>> I saw this patch in the powertop repo. >>>>> Previously, the cross call number on my idle system is: >>>>> ============= Wakeups-from-idle per second: 1621.6 interval: >>>>> 5.0s >>>>> Top causes for wakeups: >>>>> 76.1% (1234.3) sched :<cross calls> >>>>> >>>>> --------snip-------- >>>>> 0.9% ( 14.0) dtrace :<cross calls> >>>>> ============ >>>>> >>>>> with this patch, the number decreased too much, see below: >>>>> ============== Wakeups-from-idle per second: 1592.7 interval: 5.0s >>>>> Top causes for wakeups: >>>>> 11.2% (178.8) sched :<cross call> >>>>> unix`dtrace_xcall_func >>>>> ----snip---- >>>>> 1.0% ( 16.0) dtrace :<cross call> >>>>> unix`dtrace_xcall_func ============== >>>>> >>>>> I believe the actual cross call shouldn't vary too much in idle. >>>>> Any thoughts? >>>>> >>>> hmm..., on x86, the patch missed xc_capture_cpus() function. on >>>> sparc, it missed too much. It looks like "sysinfo:::xcalls" is far >>>> more than "fbt::xc_common:entry". >>>> >>>> If we want to implement 6781149, we probably need to improve the >>>> probe. >>>> >>>> Thanks, >>>> -Aubrey >> Hi Aubrey >> >> Could you test this patch on your system? I've added firings of >> fbt::send_dirint that were not accounted for by fbt::xc_common. >> Unfortunately, I don't see a way of getting the function causing the >> xcalls in this situation. >> >> I chose to use send_dirint because it relates to a patch I'm sending >> in a few minutes to run PowerTOP on a specific cpu. >> > > I don't think the patch does the right way to fix this problem. > Previously, we are using "sysinfo:::xcalls", this probe comes from > CPU_STATS_ADDQ(CPU, sys, xcall, xxx) > > Let's take a look at where has this call: > ======================================================== > $$work/pad-gate/usr/src/uts$ find . -name *.c | xargs grep CPU_STATS_ADDQ | > grep xcalls > > ./sun4v/os/mach_cpu_states.c: CPU_STATS_ADDQ(CPU, sys, xcalls, 1); > ./sun4v/os/mach_cpu_states.c: CPU_STATS_ADDQ(CPU, sys, xcalls, shipped); > ./sun4u/cpu/us3_cheetah.c: CPU_STATS_ADDQ(CPU, sys, xcalls, shipped); > ./sun4u/cpu/us3_cheetah.c: CPU_STATS_ADDQ(CPU, sys, xcalls, ncpuids); > ./sun4u/cpu/us3_jalapeno.c: CPU_STATS_ADDQ(CPU, sys, xcalls, shipped); > ./sun4u/cpu/opl_olympus.c: CPU_STATS_ADDQ(CPU, sys, xcalls, shipped); > ./sun4u/cpu/opl_olympus.c: CPU_STATS_ADDQ(CPU, sys, xcalls, ncpuids); > ./sun4u/cpu/opl_olympus.c: CPU_STATS_ADDQ(CPU, sys, xcalls, 1); > ./sun4u/cpu/us3_cheetahplus.c: CPU_STATS_ADDQ(CPU, sys, xcalls, shipped); > ./sun4u/cpu/us3_cheetahplus.c: CPU_STATS_ADDQ(CPU, sys, xcalls, ncpuids); > ./sun4u/cpu/spitfire.c: CPU_STATS_ADDQ(CPU, sys, xcalls, 1); > ./sun4u/cpu/us3_common.c: CPU_STATS_ADDQ(CPU, sys, xcalls, 1); > > ./i86pc/os/x_call.c: CPU_STATS_ADDQ(CPU, sys, xcalls, 1); > ./i86pc/os/x_call.c: CPU_STATS_ADDQ(CPU, sys, xcalls, 1);
Thanks for catching this. For xc_capture_cpus() on x86, I think send_dirint() is enough. It's actually more than what sysinfo:::xcalls reports, since send_dirint() is called for each CPU that gets xcall'ed, and sysinfo:::xcalls only once early in xc_capture_cpus() - which looks like a bug. For SPARC systems, we currently don't offer a sun4u build of PowerTOP (which we might want to change (?)). So, for now, we can leave that aside. For sun4v, all xcalls are init'ed with init_mondo(), which specifies the function and arguments. So I think there are two options: (a) tie fbt::xc_common firings to fbt::send_dirint on x86 and fbt::init_mondo calls to fbt::shipit on sun4v. or (b) tie fbt::xc_common and fbt::init_mondo to sysinfo:::xcalls On both cases we'll need to catch free/untied firings of sysinfo:::xcalls for specific situations. The patch I sent for the --cpu option needs fbt::send_dirint (and fbt::shipt) to catch incoming xcalls. So I'd suggest using (a) above. What do you think ? Thanks, Rafael
