Bill.Holler wrote: > Bill Holler wrote: >> Eric Saxe wrote: >> >>> Hi Bill, >>> >>> Bill Holler wrote: >>> >>> >>>> Hi, >>>> >>>> Solaris programs the HPET at boot time to use an I/O APIC >>>> interrupt targeting CPU 0. When CPUs go into deep C-state, >>>> they schedule their next cbe interrupt on the HPET because >>>> their local APIC will stall in deep C-States. Currently the HPET >>>> always interrupts CPU 0 which in turn sends an IPI to the >>>> CPU that needs to wakeup next. >>>> >>>> >>>> I have not looked into re-programming the I/O APIC to >>>> target different CPUs such as the next CPU to wakeup. >>>> >>>> Here are some thoughts on this: >>>> >>>> 1. Always program the I/O APIC to target the HPET's interrupt >>>> to the next CPU that should wake up. >>>> Advantage: only one cpu ever has to wake up. >>>> Disadvantage: Programming the I/O APIC whenever the >>>> HPET's timer is programmed may be quite expensive. >>>> For example removing a few HPET reads from its ISR made >>>> a huge performance difference. >>>> >>>> 2. Program the I/O APIC to target the HPET interrupt to a CPU >>>> on the socket the power-aware scheduler will attempt to make >>>> idle first. Advantage: HPET is least likely to interrupt >>>> "busy" cpus. Advantage: The next deep C-state cpu to wake up >>>> is likely to be on the same "package" aka "socket". Much of >>>> the benefit of deep C-states is only realized when all CPUs >>>> on a package enter deep C-states, so waking up 2 core on >>>> a socket is not mush worse power-wise than waking just 1. >>>> Advantage: minimizes I/O APIC re-programming. >>>> >>>> 3. Leave the I/O APIC targetting the HPET interrupt on CPU 0. >>>> Advantage: ? No reliance on PAD for prototyping? >>>> Advantage: least I/O APIC reprogramming in ISR. >>>> Disadvantage: CPU 0 is usually already quite busy. >>>> >>>> >>>> >>>> Option #2 sounds interesting to me. Any thoughts? >>>> I have not yet experimented with I/O APIC re-programming. >>>> I am guessing #1 will be too expensive for the interrupt service >>>> routine. >>>> >>>> >>>> >>> How does all this tie in with c-state dependency domains? I'm >>> assuming that unless (and until) every CPU in the domain has >>> successfully transitioned into the deeper c-state, that the APIC >>> timer for the other CPUS in the domain that already requested the >>> deeper state might be ok? I'm just wondering if we can save some >>> overhead knowing this... >>> >>> >> >> >> I think you are suggesting; do not program the HPET for any >> cpu in a c-state dependency domain until the last cpu in that c-state >> dependency domain attempts to enter a deep c-state? >> This would reduce a large amount of HPET programming (probably 2x to >> 8x). >> >> I do not know if the hardware guaranties local APIC timers are >> OK until all cpus in a dependency domain attempt to enter a >> deep c-state? >> > > If not, other local APICs in the c-state dependency domain > could be used to generate interrupts for CPUs in deep c-state > until all CPUs in the dependency domain enter deep c-state. > Programming local APICs should be less expensive than > the HPET. > > Bill >
No, Once a logic core enter deep c-state, its local APIC time freezes. this doesn't depend on the c-state domain. Thanks, -Aubrey
