Hi Bill, Bill Holler wrote: > Hi, > > Solaris programs the HPET at boot time to use an I/O APIC > interrupt targeting CPU 0. When CPUs go into deep C-state, > they schedule their next cbe interrupt on the HPET because > their local APIC will stall in deep C-States. Currently the HPET > always interrupts CPU 0 which in turn sends an IPI to the > CPU that needs to wakeup next. > > > I have not looked into re-programming the I/O APIC to > target different CPUs such as the next CPU to wakeup. > > Here are some thoughts on this: > > 1. Always program the I/O APIC to target the HPET's interrupt > to the next CPU that should wake up. > Advantage: only one cpu ever has to wake up. > Disadvantage: Programming the I/O APIC whenever the > HPET's timer is programmed may be quite expensive. > For example removing a few HPET reads from its ISR made > a huge performance difference. > > 2. Program the I/O APIC to target the HPET interrupt to a CPU > on the socket the power-aware scheduler will attempt to make > idle first. > Advantage: HPET is least likely to interrupt "busy" cpus. > Advantage: The next deep C-state cpu to wake up is likely > to be on the same "package" aka "socket". Much of > the benefit of deep C-states is only realized when all CPUs > on a package enter deep C-states, so waking up 2 core on > a socket is not mush worse power-wise than waking just 1. > Advantage: minimizes I/O APIC re-programming. > > 3. Leave the I/O APIC targetting the HPET interrupt on CPU 0. > Advantage: ? No reliance on PAD for prototyping? > Advantage: least I/O APIC reprogramming in ISR. > Disadvantage: CPU 0 is usually already quite busy. > > > > Option #2 sounds interesting to me. Any thoughts? > I have not yet experimented with I/O APIC re-programming. > I am guessing #1 will be too expensive for the interrupt service > routine. > How does all this tie in with c-state dependency domains? I'm assuming that unless (and until) every CPU in the domain has successfully transitioned into the deeper c-state, that the APIC timer for the other CPUS in the domain that already requested the deeper state might be ok? I'm just wondering if we can save some overhead knowing this...
Thanks, -Eric
