Re: RFC: New event timers infrastructure
On Mon, Jun 21, 2010 at 12:33 AM, Alexander Motin wrote: > Brandon Gooch wrote: >> On Sun, Jun 20, 2010 at 3:35 PM, Doug Barton wrote: >>> On 06/20/10 08:47, Alexander Motin wrote: While this can be done in sysctl.conf, it would be better to do it in loader.conf to make it applied from the beginning, without on-the-fly timers change. >>> You're probably right that for something this fundamental it's better to do >>> it in loader.conf, however I wanted to mention that I recently changed the >>> rcorder for the "early" boot scripts to run sysctl first for very similar >>> reasons. >> >> This is good information, so sorry if I'm being dense: does this mean >> that it should have worked by applying the changes via >> /etc/sysctl.conf? > > "You are too old. You are already born." (c) Sergey Lukyanenko. > stathz set much earlier, just before "Starting kernel event timers" > logged first time. It can only be affected from loader.conf. OK. I understand everything[*] now... >> No worries though, I've set it in /boot/loader.conf to avoid any >> possible ambiguity or anomalous behavior, and it's working very well! > > Nice. Very nice :) > BTW, I've successfully tested suspend/resume on my amd64 laptop with all > timers. For me, suspend/resume trials will be this evening, U.S. Central Standard Time... Looking forward to it! -Brandon [*] Well, almost everything, except for the Russian Sci-Fi quote ;) ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: RFC: New event timers infrastructure
Brandon Gooch wrote: > On Sun, Jun 20, 2010 at 3:35 PM, Doug Barton wrote: >> On 06/20/10 08:47, Alexander Motin wrote: >>> While this can be done in sysctl.conf, it would be better to do it in >>> loader.conf to make it applied from the beginning, without on-the-fly >>> timers change. >> You're probably right that for something this fundamental it's better to do >> it in loader.conf, however I wanted to mention that I recently changed the >> rcorder for the "early" boot scripts to run sysctl first for very similar >> reasons. > > This is good information, so sorry if I'm being dense: does this mean > that it should have worked by applying the changes via > /etc/sysctl.conf? "You are too old. You are already born." (c) Sergey Lukyanenko. stathz set much earlier, just before "Starting kernel event timers" logged first time. It can only be affected from loader.conf. > No worries though, I've set it in /boot/loader.conf to avoid any > possible ambiguity or anomalous behavior, and it's working very well! Nice. BTW, I've successfully tested suspend/resume on my amd64 laptop with all timers. -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: RFC: New event timers infrastructure
On Sun, Jun 20, 2010 at 3:35 PM, Doug Barton wrote: > On 06/20/10 08:47, Alexander Motin wrote: >> >> While this can be done in sysctl.conf, it would be better to do it in >> loader.conf to make it applied from the beginning, without on-the-fly >> timers change. > > You're probably right that for something this fundamental it's better to do > it in loader.conf, however I wanted to mention that I recently changed the > rcorder for the "early" boot scripts to run sysctl first for very similar > reasons. This is good information, so sorry if I'm being dense: does this mean that it should have worked by applying the changes via /etc/sysctl.conf? No worries though, I've set it in /boot/loader.conf to avoid any possible ambiguity or anomalous behavior, and it's working very well! Alexander, if I haven't said it enough already, thanks! I see that you committed this work, looking forward to wider-spread testing now... -Brandon ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: RFC: New event timers infrastructure
On 06/20/10 08:47, Alexander Motin wrote: While this can be done in sysctl.conf, it would be better to do it in loader.conf to make it applied from the beginning, without on-the-fly timers change. You're probably right that for something this fundamental it's better to do it in loader.conf, however I wanted to mention that I recently changed the rcorder for the "early" boot scripts to run sysctl first for very similar reasons. Doug -- ... and that's just a little bit of history repeating. -- Propellerheads Improve the effectiveness of your Internet presence with a domain name makeover!http://SupersetSolutions.com/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: RFC: New event timers infrastructure
Brandon Gooch wrote: > On Sun, Jun 20, 2010 at 1:17 AM, Alexander Motin wrote: > In by /boot/loader.conf, I now have: > > # Power Saving > kern.hz="100" > #hint.apic.0.clock="0" > #hint.atrtc.0.clock="0" > hint.p4tcc.0.disabled="1" > hint.p4tcc.1.disabled="1" > hint.acpi_throttle.0.disabled="1" > hint.acpi_throttle.1.disabled="1" > hw.pci.do_power_nodriver="3" > > In /etc/sysctl.conf, I have as suggested: > > kern.eventtimer.timer1=HPET > kern.eventtimer.timer2=NONE > kern.eventtimer.singlemul=1 While this can be done in sysctl.conf, it would be better to do it in loader.conf to make it applied from the beginning, without on-the-fly timers change. > But in my dmesg, I see: > > # dmesg | grep Starting > Starting kernel event timers: LAPIC @ 100Hz, HPET @ 128Hz > Starting kernel event timers: LAPIC @ 400Hz, NONE @ 0Hz > Starting kernel event timers: HPET @ 200Hz, NONE @ 0Hz This is result of changing timers during late boot by sysctl.conf. > So it seems as if the HPET doesn't honor kern.hz setting? Maybe I > still need to explicitly disable the apic timers... All timers now equally honor hz. But except hz, stathz also should be honored, which set once during startup. In your case, as second timer was initially present, system was free to choose stathz frequency, and it chosen 128Hz. When you later disabled second timer, system had to rise first timer frequency to emulate declared stathz. If you move timers selection options from sysctl.conf to loader.conf you will get what you want. > Would you suggest using LAPIC as opposed to HPET? I have seen power > savings being able to use C3 state, but if tickless kernel eventually > buys those savings back, perhaps LAPIC would be better... Tickless operation usually effective only together with C-states. It increases their effectiveness (or even applicability). The only case benefit from tickless operation without C3 - is a virtual machines. So if your LAPIC dies on C3 (it seems not to on Core i5 any more) you have no much other options then avoid it. HPET same time never have problems with C-states, as it located in chipset. But not every HPET is equally useful. Except AMD since at least SB700) and latest Intel chipsets, HPET uses regular IRQs, that are often shared with PCI devices and so hardly could be bound to CPU cores to provide separate events for every core. While tickless operation is still possible in that case, it is somewhat limited, as one core will have wake up every time when any other need event. Mentioned chipset's HPETs same time support FSB interrupts (alike to PCI MSI-X), that are never shared and so timers could be dedicated to CPU cores. >> As result, you will have single timer, running at HZ rate. Instead of >> HPET there you may choose any timer: >> LAPIC - it is per-CPU, so saves on IPI interrupts, supports one-shot >> mode and so suitable for further tickless kernel, but it doesn't work in >> C3 state; > > So if LAPIC is disabled in C3 state, is there a possibility that the > "on-the-fly" method of changing clocks could kick in when the CPU > enters C3 state, switching to the HPET when it happens? On-the-fly timers change is possible, as you have already tried with sysctl.conf. But the switch is not so easy because of LAPIC specifics, and it is meaningless to use LAPIC in that case. If you need to run C3 - turn LAPIC off. -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: RFC: New event timers infrastructure
On Sun, Jun 20, 2010 at 1:17 AM, Alexander Motin wrote:
> Brandon Gooch wrote:
>> I've been testing these patches since the first iteration
>> (et.20100606), and I haven't discovered any related issues.
>
> Thank you!
>
>> I am unclear about the number of interrupts I should expect from the
>> hpet0 device (compared to the 99 from the rtc at 100Hz), so here is
>> the output of vmstat -i with and without the "et" patches:
>>
>> With "et" patches:
>>
>> interrupt total rate
>> irq1: atkbd0 369 3
>> irq9: acpi0 961 8
>> irq12: psm0 1002 9
>> irq18: uhci5 140 1
>> irq19: uhci2 ehci0* 4823 45
>> irq20: hpet0 23893223
>> irq23: uhci3 ehci111 0
>> irq256: vgapci0 1031 9
>> irq257: hdac0 14 0
>> irq258: iwn04258 39
>> irq259: bge0 1 0
>> Total 36503341
>>
>> Without "et" patches:
>>
>> interrupt total rate
>> irq1: atkbd0 449 2
>> irq0: clk 17334 99
>> irq9: acpi0 1701 9
>> irq12: psm0 8784 50
>> irq18: uhci5 188 1
>> irq19: uhci2 ehci0* 5828 33
>> irq23: uhci3 ehci111 0
>> irq256: vgapci0 1896 10
>> irq257: hdac0 14 0
>> irq258: iwn0 29571169
>> irq259: bge0 1 0
>> Total 65777378
>>
>> And lastly, the values of the kern.eventtimer sysctls:
>>
>> $ sysctl kern.eventtimer
>> kern.eventtimer.choice: HPET(450) HPET1(440) HPET2(440) HPET3(440) i8254(100)
>> kern.eventtimer.timer2: HPET1
>> kern.eventtimer.timer1: HPET
>> kern.eventtimer.singlemul: 4
>
> I don't see there neither LAPIC, nor RTC timer. I suppose you have
> disabled them via device hints. I also suppose you've done it to left
> with only 100Hz IRQs of i8254 timer. Now, with the patch, these two are
> still disabled, but system got 4 more HPET timers. As they have higher
> precedence, two of them were taken (HPET and HPET1). Number if
> interrupts can be explained by the line:
>
> Starting kernel event timers: HPET @ 100Hz, HPET1 @ 128Hz
Yes, I had the LAPIC and RTC timers disabled via my /boot/loader.conf,
as per your suggestions here:
http://wiki.freebsd.org/TuningPowerConsumption
> As result, you've got 228Hz IRQs (which then redistributed to every
> CPU). As your HPET timer doesn't support FSB interrupts, all it's timers
> share same IRQ vector, seen as "hpet0".
Understood.
> If you wish to get back to 100Hz IRQs as before, you may remove your
> previous clock-disabling hints and add instead:
>
> kern.eventtimer.timer1=HPET
> kern.eventtimer.timer2=NONE
> kern.eventtimer.singlemul=1
In by /boot/loader.conf, I now have:
# Power Saving
kern.hz="100"
#hint.apic.0.clock="0"
#hint.atrtc.0.clock="0"
hint.p4tcc.0.disabled="1"
hint.p4tcc.1.disabled="1"
hint.acpi_throttle.0.disabled="1"
hint.acpi_throttle.1.disabled="1"
hw.pci.do_power_nodriver="3"
In /etc/sysctl.conf, I have as suggested:
kern.eventtimer.timer1=HPET
kern.eventtimer.timer2=NONE
kern.eventtimer.singlemul=1
In my /etc/rc.conf, I'm still using:
performance_cpu_freq="NONE"
economy_cpu_freq="NONE"
performance_cx_lowest="C3"
economy_cx_lowest="C3"
I'm running powerd:
powerd_enable="YES"
powerd_flags="-a adaptive -b adaptive -n adaptive"
But in my dmesg, I see:
# dmesg | grep Starting
Starting kernel event timers: LAPIC @ 100Hz, HPET @ 128Hz
Starting kernel event timers: LAPIC @ 400Hz, NONE @ 0Hz
Starting kernel event timers: HPET @ 200Hz, NONE @ 0Hz
So it seems as if the HPET doesn't honor kern.hz setting? Maybe I
still need to explicitly disable the apic timers...
Would you suggest using LAPIC as opposed to HPET? I have seen power
savings being able to use C3 state, but if tickless kernel eventually
buys those savings back, perhaps LAPIC would be better...
> As result, you will have single timer, running at HZ rate. Instead of
> HPET there you may choose any timer:
> LAPIC - it is per-CPU, so saves on IPI interrupts, supports one-shot
> mode and so suitable for further tickless kernel, but it doesn't work in
> C3 state;
So if LAPIC is disabled in C3 state, is there a possibility that the
"on-the-fly" method of changing clocks could kick in when the CPU
enters C3 state, switching to the HPET when it happens?
> HPET{x} - on this hardware it can't be used as per-CPU, it supports
> one-shot mode, but less suitable for further tickless kernel, as CPUs
> can't run independe
Re: RFC: New event timers infrastructure
Brandon Gooch wrote:
> I've been testing these patches since the first iteration
> (et.20100606), and I haven't discovered any related issues.
Thank you!
> I am unclear about the number of interrupts I should expect from the
> hpet0 device (compared to the 99 from the rtc at 100Hz), so here is
> the output of vmstat -i with and without the "et" patches:
>
> With "et" patches:
>
> interrupt total rate
> irq1: atkbd0 369 3
> irq9: acpi0 961 8
> irq12: psm0 1002 9
> irq18: uhci5 140 1
> irq19: uhci2 ehci0* 4823 45
> irq20: hpet0 23893223
> irq23: uhci3 ehci111 0
> irq256: vgapci0 1031 9
> irq257: hdac0 14 0
> irq258: iwn04258 39
> irq259: bge0 1 0
> Total 36503341
>
> Without "et" patches:
>
> interrupt total rate
> irq1: atkbd0 449 2
> irq0: clk 17334 99
> irq9: acpi0 1701 9
> irq12: psm0 8784 50
> irq18: uhci5 188 1
> irq19: uhci2 ehci0* 5828 33
> irq23: uhci3 ehci111 0
> irq256: vgapci0 1896 10
> irq257: hdac0 14 0
> irq258: iwn0 29571169
> irq259: bge0 1 0
> Total 65777378
>
> And lastly, the values of the kern.eventtimer sysctls:
>
> $ sysctl kern.eventtimer
> kern.eventtimer.choice: HPET(450) HPET1(440) HPET2(440) HPET3(440) i8254(100)
> kern.eventtimer.timer2: HPET1
> kern.eventtimer.timer1: HPET
> kern.eventtimer.singlemul: 4
I don't see there neither LAPIC, nor RTC timer. I suppose you have
disabled them via device hints. I also suppose you've done it to left
with only 100Hz IRQs of i8254 timer. Now, with the patch, these two are
still disabled, but system got 4 more HPET timers. As they have higher
precedence, two of them were taken (HPET and HPET1). Number if
interrupts can be explained by the line:
Starting kernel event timers: HPET @ 100Hz, HPET1 @ 128Hz
As result, you've got 228Hz IRQs (which then redistributed to every
CPU). As your HPET timer doesn't support FSB interrupts, all it's timers
share same IRQ vector, seen as "hpet0".
If you wish to get back to 100Hz IRQs as before, you may remove your
previous clock-disabling hints and add instead:
kern.eventtimer.timer1=HPET
kern.eventtimer.timer2=NONE
kern.eventtimer.singlemul=1
As result, you will have single timer, running at HZ rate. Instead of
HPET there you may choose any timer:
LAPIC - it is per-CPU, so saves on IPI interrupts, supports one-shot
mode and so suitable for further tickless kernel, but it doesn't work in
C3 state;
HPET{x} - on this hardware it can't be used as per-CPU, it supports
one-shot mode, but less suitable for further tickless kernel, as CPUs
can't run independently;
i8254 - somewhat faster, as it doesn't needs status reading, but due to
being also a timecounter it can't be used in one-shot mode;
RTC - somewhat slower, has limited set of supported frequencies (powers
of 2), only periodic mode.
--
Alexander Motin
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: RFC: New event timers infrastructure
In message: <4c0e5646.1060...@freebsd.org> Alexander Motin writes: : Brandon Gooch wrote: : > Alexander, do you feel that the code is at a stage where meaningful : > user testing can occur? : : I think yes. I've touched a lot of legacy code, so it would be nice to : know what I may have broken. For example, i8254 and RTC drivers now more : dependent on attaching to PnP/ACPI reported hardware. I am not sure if : there still any legacy system which not doing that. The oldest system I : have tested was P-III Celeron. Unluckily my small development SSD is too : "large" for my Pentium board's BIOS. :) The only issue I'm aware of is that on older systems PnP and ACPI can sometimes interfere. But I thought we'd fixed those issues... I can test on my older pentium too... Warner : There is still work planned, but I hope it won't require major changes : in already written code. : : -- : Alexander Motin : ___ : freebsd-current@freebsd.org mailing list : http://lists.freebsd.org/mailman/listinfo/freebsd-current : To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" : ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: RFC: New event timers infrastructure
Brandon Gooch wrote: > I'm giving http://people.freebsd.org/~mav/et.20100607.patch a go right now... > > After patching and recompiling the kernel, I'm up and running. > > What information/feedback would you like to see from us users? As always: what is working and especially what is not. What timers detected on system, what are used, are all of them running, and if not, does system automatically falls back to different ones, is there any problems with choosing timers manually in different combinations, does suspend/resume still working... -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: RFC: New event timers infrastructure
On Tue, Jun 8, 2010 at 9:40 AM, Alexander Motin wrote: > Brandon Gooch wrote: >> Alexander, do you feel that the code is at a stage where meaningful >> user testing can occur? > > I think yes. I've touched a lot of legacy code, so it would be nice to > know what I may have broken. For example, i8254 and RTC drivers now more > dependent on attaching to PnP/ACPI reported hardware. I am not sure if > there still any legacy system which not doing that. The oldest system I > have tested was P-III Celeron. Unluckily my small development SSD is too > "large" for my Pentium board's BIOS. :) > > There is still work planned, but I hope it won't require major changes > in already written code. > > -- > Alexander Motin > I'm giving http://people.freebsd.org/~mav/et.20100607.patch a go right now... After patching and recompiling the kernel, I'm up and running. What information/feedback would you like to see from us users? -Brandon ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: RFC: New event timers infrastructure
Brandon Gooch wrote: > Alexander, do you feel that the code is at a stage where meaningful > user testing can occur? I think yes. I've touched a lot of legacy code, so it would be nice to know what I may have broken. For example, i8254 and RTC drivers now more dependent on attaching to PnP/ACPI reported hardware. I am not sure if there still any legacy system which not doing that. The oldest system I have tested was P-III Celeron. Unluckily my small development SSD is too "large" for my Pentium board's BIOS. :) There is still work planned, but I hope it won't require major changes in already written code. -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: RFC: New event timers infrastructure
On Tue, Jun 8, 2010 at 12:24 AM, Tsuyoshi Ozawa wrote: > 2010/6/6, Alexander Motin : >> Hi. >> >> Most of x86 systems now has at least 4 types of event timers: i8254, >> RTC, LAPIC and HPET. Respective code in kernel is very tangled, heavily >> hardcoded and absolutely not scalable. I have reimplemented it, trying >> to solve these issues. >> >> I did such things: >> - created unified timer driver's API (sys/timeet.h, kernel/kern_et.c). >> It supports global and per-CPU timers, periodic and one-shot. Provides >> driver and consumer interfaces for choosing timers and operating them; >> - cleaned existing x86 event timer driver's code and modified it for >> new API (x86/isa/atrtc.c, x86/isa/clock.c, x86/x86/local_apic.c). LAPIC >> timer is now per-CPU and supports both periodic and one-shot modes; >> - extended HPET driver to support it's event timers in periodic and >> one-shot mode (dev/acpica/acpi_hpet.c). Support for per-CPU operation >> and FSB interrupts planned for later; >> - written mostly machine-independent mid-layer for managing any present >> timers to provide clocks needed for kernel (x86/x86/timeevents.c). It >> supports both global and per-CPU timers. Now it supports only periodic >> mode, but one-shot mode support planned for later. >> >> All this stuff deeply configurable via both loader tunables on boot and >> sysctls in real time: >> >> %sysctl kern.eventtimer >> kern.eventtimer.choice: LAPIC(500) HPET(400) HPET1(390) HPET2(390) >> i8254(100) RTC(0) >> kern.eventtimer.et.LAPIC.flags: 7 >> kern.eventtimer.et.LAPIC.frequency: 99752386 >> kern.eventtimer.et.LAPIC.quality: 500 >> kern.eventtimer.et.HPET.flags: 3 >> kern.eventtimer.et.HPET.frequency: 14318180 >> kern.eventtimer.et.HPET.quality: 400 >> kern.eventtimer.et.HPET1.flags: 3 >> kern.eventtimer.et.HPET1.frequency: 14318180 >> kern.eventtimer.et.HPET1.quality: 390 >> kern.eventtimer.et.HPET2.flags: 3 >> kern.eventtimer.et.HPET2.frequency: 14318180 >> kern.eventtimer.et.HPET2.quality: 390 >> kern.eventtimer.et.RTC.flags: 1 >> kern.eventtimer.et.RTC.frequency: 32768 >> kern.eventtimer.et.RTC.quality: 0 >> kern.eventtimer.et.i8254.flags: 1 >> kern.eventtimer.et.i8254.frequency: 1193182 >> kern.eventtimer.et.i8254.quality: 100 >> kern.eventtimer.timer2: NONE >> kern.eventtimer.timer1: i8254 >> kern.eventtimer.singlemul: 2 >> >> By default system chooses two timers with highest "quality" for >> hardclock and statclock/profclock. User may affect that choice via >> disabling unwanted drivers and/or via direct specification of wanted >> ones. It is possible to change timers on-flight via sysctls: >> >> %sysctl kern.eventtimer.timer1=hpet >> kern.eventtimer.timer1: i8254 -> HPET >> %sysctl kern.eventtimer.timer2=hpet1 >> kern.eventtimer.timer2: NONE -> HPET1 >> >> After every timer change, if two timers available, mid-layer >> cross-checks them, and if one of them is not functional - replaces it. >> >> If there is no second timer available, or user specified to not use it - >> mid-layer automatically increases rate of the first timer and divide >> it's frequency to satisfy system needs as good as possible. User may >> specify how fast he wish to run fist timer relative to hz by setting >> kern.eventtimer.singlemul tunable/sysctl. >> >> When profiling is active, mid-layer automatically rises respective timer >> frequency to about 8KHz (was 1KHz previously) and decreases it back on >> profiling end. >> >> All above was tested on i386 and amd64. XEN was not affected and builds >> fine. pc98 was slightly touched. It wasn't tested, but builds fine. It's >> pc98/cbus/clock.c needs respective rewrite to use new features. Other >> architectures are untouched, but if any of them may benefit from this >> functionality - it should be possible to share most of the code. >> >> Latest patches can be found here: >> http://people.freebsd.org/~mav/et.20100606.patch >> >> Known issues: >> - i8254 timer generates 18Hz interrupt rate when not used and not >> disabled. I haven't found a way to disable it's interrupt source while >> holding spinlock. >> - timer drivers code will need some more cleaning after interrupt >> handler will be able to return both argument and frame same time. >> >> Feedback is very appreciated. >> >> -- >> Alexander Motin >> > > This is excellent! > I'll try this to apply this patch and rewrite my old dynamic ticks code to > fit this event timer API . > > Thank you, Alexander ! > -- > Tsuyoshi Ozawa > > > 2010/06/07 7:03 "Alexander Motin" : > > Hi. > > Most of x86 systems now has at least 4 types of event timers: i8254, > RTC, LAPIC and HPET. Respective code in kernel is very tangled, heavily > hardcoded and absolutely not scalable. I have reimplemented it, trying > to solve these issues. > > I did such things: > - created unified timer driver's API (sys/timeet.h, kernel/kern_et.c). > It supports global and per-CPU timers, periodic and one-shot. Provides > driver and consumer interfaces for choosing timers and operating them; >
Re: RFC: New event timers infrastructure
2010/6/6, Alexander Motin : > Hi. > > Most of x86 systems now has at least 4 types of event timers: i8254, > RTC, LAPIC and HPET. Respective code in kernel is very tangled, heavily > hardcoded and absolutely not scalable. I have reimplemented it, trying > to solve these issues. > > I did such things: > - created unified timer driver's API (sys/timeet.h, kernel/kern_et.c). > It supports global and per-CPU timers, periodic and one-shot. Provides > driver and consumer interfaces for choosing timers and operating them; > - cleaned existing x86 event timer driver's code and modified it for > new API (x86/isa/atrtc.c, x86/isa/clock.c, x86/x86/local_apic.c). LAPIC > timer is now per-CPU and supports both periodic and one-shot modes; > - extended HPET driver to support it's event timers in periodic and > one-shot mode (dev/acpica/acpi_hpet.c). Support for per-CPU operation > and FSB interrupts planned for later; > - written mostly machine-independent mid-layer for managing any present > timers to provide clocks needed for kernel (x86/x86/timeevents.c). It > supports both global and per-CPU timers. Now it supports only periodic > mode, but one-shot mode support planned for later. > > All this stuff deeply configurable via both loader tunables on boot and > sysctls in real time: > > %sysctl kern.eventtimer > kern.eventtimer.choice: LAPIC(500) HPET(400) HPET1(390) HPET2(390) > i8254(100) RTC(0) > kern.eventtimer.et.LAPIC.flags: 7 > kern.eventtimer.et.LAPIC.frequency: 99752386 > kern.eventtimer.et.LAPIC.quality: 500 > kern.eventtimer.et.HPET.flags: 3 > kern.eventtimer.et.HPET.frequency: 14318180 > kern.eventtimer.et.HPET.quality: 400 > kern.eventtimer.et.HPET1.flags: 3 > kern.eventtimer.et.HPET1.frequency: 14318180 > kern.eventtimer.et.HPET1.quality: 390 > kern.eventtimer.et.HPET2.flags: 3 > kern.eventtimer.et.HPET2.frequency: 14318180 > kern.eventtimer.et.HPET2.quality: 390 > kern.eventtimer.et.RTC.flags: 1 > kern.eventtimer.et.RTC.frequency: 32768 > kern.eventtimer.et.RTC.quality: 0 > kern.eventtimer.et.i8254.flags: 1 > kern.eventtimer.et.i8254.frequency: 1193182 > kern.eventtimer.et.i8254.quality: 100 > kern.eventtimer.timer2: NONE > kern.eventtimer.timer1: i8254 > kern.eventtimer.singlemul: 2 > > By default system chooses two timers with highest "quality" for > hardclock and statclock/profclock. User may affect that choice via > disabling unwanted drivers and/or via direct specification of wanted > ones. It is possible to change timers on-flight via sysctls: > > %sysctl kern.eventtimer.timer1=hpet > kern.eventtimer.timer1: i8254 -> HPET > %sysctl kern.eventtimer.timer2=hpet1 > kern.eventtimer.timer2: NONE -> HPET1 > > After every timer change, if two timers available, mid-layer > cross-checks them, and if one of them is not functional - replaces it. > > If there is no second timer available, or user specified to not use it - > mid-layer automatically increases rate of the first timer and divide > it's frequency to satisfy system needs as good as possible. User may > specify how fast he wish to run fist timer relative to hz by setting > kern.eventtimer.singlemul tunable/sysctl. > > When profiling is active, mid-layer automatically rises respective timer > frequency to about 8KHz (was 1KHz previously) and decreases it back on > profiling end. > > All above was tested on i386 and amd64. XEN was not affected and builds > fine. pc98 was slightly touched. It wasn't tested, but builds fine. It's > pc98/cbus/clock.c needs respective rewrite to use new features. Other > architectures are untouched, but if any of them may benefit from this > functionality - it should be possible to share most of the code. > > Latest patches can be found here: > http://people.freebsd.org/~mav/et.20100606.patch > > Known issues: > - i8254 timer generates 18Hz interrupt rate when not used and not > disabled. I haven't found a way to disable it's interrupt source while > holding spinlock. > - timer drivers code will need some more cleaning after interrupt > handler will be able to return both argument and frame same time. > > Feedback is very appreciated. > > -- > Alexander Motin > This is excellent! I'll try this to apply this patch and rewrite my old dynamic ticks code to fit this event timer API . Thank you, Alexander ! -- Tsuyoshi Ozawa 2010/06/07 7:03 "Alexander Motin" : Hi. Most of x86 systems now has at least 4 types of event timers: i8254, RTC, LAPIC and HPET. Respective code in kernel is very tangled, heavily hardcoded and absolutely not scalable. I have reimplemented it, trying to solve these issues. I did such things: - created unified timer driver's API (sys/timeet.h, kernel/kern_et.c). It supports global and per-CPU timers, periodic and one-shot. Provides driver and consumer interfaces for choosing timers and operating them; - cleaned existing x86 event timer driver's code and modified it for new API (x86/isa/atrtc.c, x86/isa/clock.c, x86/x86/local_apic.c). LAPIC timer is now per-CPU and supports both periodic and
Re: RFC: New event timers infrastructure
Hi Alexander, * Alexander Motin wrote: > Ed Schouten wrote: > > Just out of curiosity, how does this work relate to things like having a > > tickless kernel? > > It is almost mandatory prerequisite. We can't do any fancy timer stuff > without unified timer API. Tsuyoshi Ozawa in his Dynamic Ticks work also > got to the same conclusion, but he was much less aggressive in rewriting > legacy code. That's just what I wanted to hear. Thanks! -- Ed Schouten WWW: http://80386.nl/ pgpT48NQnkniA.pgp Description: PGP signature
Re: RFC: New event timers infrastructure
Ed Schouten wrote: > * Alexander Motin wrote: >> Most of x86 systems now has at least 4 types of event timers: i8254, >> RTC, LAPIC and HPET. Respective code in kernel is very tangled, heavily >> hardcoded and absolutely not scalable. I have reimplemented it, trying >> to solve these issues. > > Just out of curiosity, how does this work relate to things like having a > tickless kernel? It is almost mandatory prerequisite. We can't do any fancy timer stuff without unified timer API. Tsuyoshi Ozawa in his Dynamic Ticks work also got to the same conclusion, but he was much less aggressive in rewriting legacy code. -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: RFC: New event timers infrastructure
Hi Alexander, * Alexander Motin wrote: > Most of x86 systems now has at least 4 types of event timers: i8254, > RTC, LAPIC and HPET. Respective code in kernel is very tangled, heavily > hardcoded and absolutely not scalable. I have reimplemented it, trying > to solve these issues. Just out of curiosity, how does this work relate to things like having a tickless kernel? -- Ed Schouten WWW: http://80386.nl/ pgpM98YsnquDQ.pgp Description: PGP signature
