Eric Saxe wrote: > Hi Bill, > > Bill Holler wrote: > >> Hi, >> >> Solaris programs the HPET at boot time to use an I/O APIC >> interrupt targeting CPU 0. When CPUs go into deep C-state, >> they schedule their next cbe interrupt on the HPET because >> their local APIC will stall in deep C-States. Currently the HPET >> always interrupts CPU 0 which in turn sends an IPI to the >> CPU that needs to wakeup next. >> >> >> I have not looked into re-programming the I/O APIC to >> target different CPUs such as the next CPU to wakeup. >> >> Here are some thoughts on this: >> >> 1. Always program the I/O APIC to target the HPET's interrupt >> to the next CPU that should wake up. >> Advantage: only one cpu ever has to wake up. >> Disadvantage: Programming the I/O APIC whenever the >> HPET's timer is programmed may be quite expensive. >> For example removing a few HPET reads from its ISR made >> a huge performance difference. >> >> 2. Program the I/O APIC to target the HPET interrupt to a CPU >> on the socket the power-aware scheduler will attempt to make >> idle first. >> Advantage: HPET is least likely to interrupt "busy" cpus. >> Advantage: The next deep C-state cpu to wake up is likely >> to be on the same "package" aka "socket". Much of >> the benefit of deep C-states is only realized when all CPUs >> on a package enter deep C-states, so waking up 2 core on >> a socket is not mush worse power-wise than waking just 1. >> Advantage: minimizes I/O APIC re-programming. >> >> 3. Leave the I/O APIC targetting the HPET interrupt on CPU 0. >> Advantage: ? No reliance on PAD for prototyping? >> Advantage: least I/O APIC reprogramming in ISR. >> Disadvantage: CPU 0 is usually already quite busy. >> >> >> >> Option #2 sounds interesting to me. Any thoughts? >> I have not yet experimented with I/O APIC re-programming. >> I am guessing #1 will be too expensive for the interrupt service >> routine. >> >> > How does all this tie in with c-state dependency domains? I'm assuming > that unless (and until) every CPU in the domain has successfully > transitioned into the deeper c-state, that the APIC timer for the other > CPUS in the domain that already requested the deeper state might be ok? > I'm just wondering if we can save some overhead knowing this... >
I think you are suggesting; do not program the HPET for any cpu in a c-state dependency domain until the last cpu in that c-state dependency domain attempts to enter a deep c-state? This would reduce a large amount of HPET programming (probably 2x to 8x). I do not know if the hardware guaranties local APIC timers are OK until all cpus in a dependency domain attempt to enter a deep c-state? Thank you, Bill > Thanks, > -Eric > _______________________________________________ > tesla-dev mailing list > tesla-dev at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/tesla-dev >
