On Mon, Jan 16, 2017 at 8:00 PM, Konstantin Belousov <kostik...@gmail.com> wrote:
> On Mon, Jan 16, 2017 at 12:28:54PM +0800, Jia-Shiun Li wrote: > > BTW please see my other mail of this thread. It seems to be related to > > EARLY_AP_STARTUP option. > Yes, I noted, I might have an idea, but the report that changing the > timecounter makes the lags go away still does not fit into my understanding > of the code. > > Most likely this is an interaction between the EARLY_AP_STARTUP and the > fact that HPET interrupt is global, while most modern systems use LAPIC > event timer, which is per-cpu, and the testing of the option was done > on them. There are some differences in handling the configurations, see > sys/kern/kern_clocksource.c, the option and ET_FLAG_PERCPU. > > And, changing the _timecounter_ fixes the issue ? Can you double-check > this ? > Yes. I noticed this because systat refreshes looked slower, and keystroke did not repeat smoothly for 30/s. I have system clock shown on tmux status line. On c2d it drifted away. Setting timecounter brings it back to normal. See also eventtimer & timecounter tests below. > > With the settings above, i.e. HPET for both eventtimer and timecounter, > please show vmstat -ia output for two times with the interval of 2 secs. > Attached. > > What if you change _eventtimer_ to APIC and then immediately back to HPET, > does the problem go away ? > > It doesn't look so. But keeping LAPIC as eventtimer helps. jsli@jsli-bsd:/home/jsli # sysctl kern.eventtimer.timer=LAPIC && sysctl kern.eventtimer.timer=HPET && ntpdate tw.pool.ntp.org && sleep 30 && ntpdate tw.pool.ntp.org kern.eventtimer.timer: HPET -> LAPIC kern.eventtimer.timer: LAPIC -> HPET 16 Jan 22:00:21 ntpdate[8472]: step time server 203.71.244.7 offset 18.980716 sec 16 Jan 22:01:56 ntpdate[8601]: step time server 103.226.213.30 offset 58.813079 sec jsli@jsli-bsd:/home/jsli # sysctl kern.eventtimer.timer=LAPIC && ntpdate tw.pool.ntp.org && sleep 30 && ntpdate tw.pool.ntp.org kern.eventtimer.timer: HPET -> LAPIC 16 Jan 22:02:36 ntpdate[8666]: step time server 103.226.213.30 offset 19.773086 sec 16 Jan 22:03:13 ntpdate[8776]: adjust time server 103.226.213.30 offset 0.000455 sec jsli@jsli-bsd:/home/jsli # sysctl kern.eventtimer.timer=HPET && ntpdate tw.pool.ntp.org && sleep 30 && ntpdate tw.pool.ntp.org kern.eventtimer.timer: LAPIC -> HPET 16 Jan 22:03:47 ntpdate[8853]: step time server 103.226.213.30 offset 6.344004 sec 16 Jan 22:05:18 ntpdate[8975]: step time server 61.216.153.105 offset 54.908872 sec jsli@jsli-bsd:/home/jsli # sysctl kern.timecounter.hardware=HPET && ntpdate tw.pool.ntp.org && sleep 30 && ntpdate tw.pool.ntp.org kern.timecounter.hardware: TSC-low -> HPET 16 Jan 22:06:29 ntpdate[9073]: step time server 59.124.29.241 offset 39.211691 sec 16 Jan 22:07:05 ntpdate[9185]: adjust time server 61.216.153.105 offset 0.001015 sec jsli@jsli-bsd:/home/jsli # sysctl kern.timecounter.hardware=TSC-low && ntpdate tw.pool.ntp.org && sleep 30 && ntpdate tw.pool.ntp.org kern.timecounter.hardware: HPET -> TSC-low 16 Jan 22:07:28 ntpdate[9244]: step time server 61.216.153.105 offset 3.122954 sec 16 Jan 22:08:58 ntpdate[9357]: step time server 61.216.153.104 offset 53.758451 sec jsli@jsli-bsd:/home/jsli # > Also, if you set the loader tunable kern.eventtimer.timer to LAPIC, > and do not enable C2+, does the system boot into the usable state ? > > Not sure if you mean disabling C2 from BIOS. Right now I don't have BIOS access to the c2d, and my notebook only has minimal BIOS settings to play with. Anyway on notebook set kern.eventtimer.timer=LAPIC in loader prompt to boot , and the system still lags. But after setting sysctl dev.cpu.0.cx_lowest=C1 && sysctl dev.cpu.1.cx_lowest=C1 system clock returns to normal speed. -Jia-Shiun.
jsli@jsli-bsd:~ % vmstat -ia 2 2 interrupt total rate ??? 0 0 irq1: atkbd0 2 0 stray irq1 0 0 irq0: attimer0 0 0 stray irq0 0 0 irq3: 0 0 stray irq3 0 0 irq4: uart0 0 0 stray irq4 0 0 irq5: 0 0 stray irq5 0 0 irq6: 0 0 stray irq6 0 0 irq7: 0 0 stray irq7 0 0 irq8: atrtc0 0 0 stray irq8 0 0 irq9: acpi0 0 0 stray irq9 0 0 irq10: 0 0 stray irq10 0 0 irq11: 0 0 stray irq11 0 0 irq12: 0 0 stray irq12 0 0 irq13: 0 0 stray irq13 0 0 irq14: 0 0 stray irq14 0 0 irq15: 0 0 stray irq15 0 0 irq16: em0:irq0++ 19 0 stray irq16 0 0 irq17: 0 0 stray irq17 0 0 irq18: uhci2 ehci0+ 18 0 stray irq18 0 0 irq19: uhci4 0 0 stray irq19 0 0 irq20: hpet0 13592 192 stray irq20 0 0 irq21: uhci1 0 0 stray irq21 0 0 irq22: 0 0 stray irq22 0 0 irq23: uhci3 ehci1 0 0 stray irq23 0 0 cpu0:timer 0 0 cpu1:timer 0 0 irq256: hdac0 95 1 stray irq256 0 0 irq257: pcib1 0 0 stray irq257 0 0 irq258: pcib2 0 0 stray irq258 0 0 irq259: pcib3 0 0 stray irq259 0 0 irq260: re0 16191 229 stray irq260 0 0 irq261: ahci0:ch0 4210 60 stray irq261 0 0 irq262: ahci0:ch1 0 0 stray irq262 0 0 irq263: ahci0:ch2 0 0 stray irq263 0 0 irq264: ahci0:ch3 0 0 stray irq264 0 0 irq265: ahci0:ch4 0 0 stray irq265 0 0 irq266: ahci0:ch5 0 0 stray irq266 0 0 irq267: ahci0:6 0 0 stray irq267 0 0 irq268: ahci0:7 0 0 stray irq268 0 0 irq269: ahci0:8 0 0 stray irq269 0 0 irq270: ahci0:9 0 0 stray irq270 0 0 irq271: ahci0:10 0 0 stray irq271 0 0 irq272: ahci0:11 0 0 stray irq272 0 0 irq273: ahci0:12 0 0 stray irq273 0 0 irq274: ahci0:13 0 0 stray irq274 0 0 irq275: ahci0:14 0 0 stray irq275 0 0 irq276: ahci0:15 0 0 stray irq276 0 0 Total 34127 483 ??? 0 0 irq1: atkbd0 0 0 stray irq1 0 0 irq0: attimer0 0 0 stray irq0 0 0 irq3: 0 0 stray irq3 0 0 irq4: uart0 0 0 stray irq4 0 0 irq5: 0 0 stray irq5 0 0 irq6: 0 0 stray irq6 0 0 irq7: 0 0 stray irq7 0 0 irq8: atrtc0 0 0 stray irq8 0 0 irq9: acpi0 0 0 stray irq9 0 0 irq10: 0 0 stray irq10 0 0 irq11: 0 0 stray irq11 0 0 irq12: 0 0 stray irq12 0 0 irq13: 0 0 stray irq13 0 0 irq14: 0 0 stray irq14 0 0 irq15: 0 0 stray irq15 0 0 irq16: em0:irq0++ 0 0 stray irq16 0 0 irq17: 0 0 stray irq17 0 0 irq18: uhci2 ehci0+ 0 0 stray irq18 0 0 irq19: uhci4 0 0 stray irq19 0 0 irq20: hpet0 81 40 stray irq20 0 0 irq21: uhci1 0 0 stray irq21 0 0 irq22: 0 0 stray irq22 0 0 irq23: uhci3 ehci1 0 0 stray irq23 0 0 cpu0:timer 0 0 cpu1:timer 0 0 irq256: hdac0 0 0 stray irq256 0 0 irq257: pcib1 0 0 stray irq257 0 0 irq258: pcib2 0 0 stray irq258 0 0 irq259: pcib3 0 0 stray irq259 0 0 irq260: re0 580 289 stray irq260 0 0 irq261: ahci0:ch0 0 0 stray irq261 0 0 irq262: ahci0:ch1 0 0 stray irq262 0 0 irq263: ahci0:ch2 0 0 stray irq263 0 0 irq264: ahci0:ch3 0 0 stray irq264 0 0 irq265: ahci0:ch4 0 0 stray irq265 0 0 irq266: ahci0:ch5 0 0 stray irq266 0 0 irq267: ahci0:6 0 0 stray irq267 0 0 irq268: ahci0:7 0 0 stray irq268 0 0 irq269: ahci0:8 0 0 stray irq269 0 0 irq270: ahci0:9 0 0 stray irq270 0 0 irq271: ahci0:10 0 0 stray irq271 0 0 irq272: ahci0:11 0 0 stray irq272 0 0 irq273: ahci0:12 0 0 stray irq273 0 0 irq274: ahci0:13 0 0 stray irq274 0 0 irq275: ahci0:14 0 0 stray irq275 0 0 irq276: ahci0:15 0 0 stray irq276 0 0 Total 661 330 jsli@jsli-bsd:~ % su Password: jsli@jsli-bsd:/home/jsli # sysctl kern.timecounter.hardware=HPET kern.timecounter.hardware: TSC-low -> HPET jsli@jsli-bsd:/home/jsli # vmstat -ia 2 2 interrupt total rate ??? 0 0 irq1: atkbd0 2 0 stray irq1 0 0 irq0: attimer0 0 0 stray irq0 0 0 irq3: 0 0 stray irq3 0 0 irq4: uart0 0 0 stray irq4 0 0 irq5: 0 0 stray irq5 0 0 irq6: 0 0 stray irq6 0 0 irq7: 0 0 stray irq7 0 0 irq8: atrtc0 0 0 stray irq8 0 0 irq9: acpi0 0 0 stray irq9 0 0 irq10: 0 0 stray irq10 0 0 irq11: 0 0 stray irq11 0 0 irq12: 0 0 stray irq12 0 0 irq13: 0 0 stray irq13 0 0 irq14: 0 0 stray irq14 0 0 irq15: 0 0 stray irq15 0 0 irq16: em0:irq0++ 19 0 stray irq16 0 0 irq17: 0 0 stray irq17 0 0 irq18: uhci2 ehci0+ 18 0 stray irq18 0 0 irq19: uhci4 0 0 stray irq19 0 0 irq20: hpet0 14679 147 stray irq20 0 0 irq21: uhci1 0 0 stray irq21 0 0 irq22: 0 0 stray irq22 0 0 irq23: uhci3 ehci1 0 0 stray irq23 0 0 cpu0:timer 0 0 cpu1:timer 0 0 irq256: hdac0 95 1 stray irq256 0 0 irq257: pcib1 0 0 stray irq257 0 0 irq258: pcib2 0 0 stray irq258 0 0 irq259: pcib3 0 0 stray irq259 0 0 irq260: re0 23596 237 stray irq260 0 0 irq261: ahci0:ch0 4331 43 stray irq261 0 0 irq262: ahci0:ch1 0 0 stray irq262 0 0 irq263: ahci0:ch2 0 0 stray irq263 0 0 irq264: ahci0:ch3 0 0 stray irq264 0 0 irq265: ahci0:ch4 0 0 stray irq265 0 0 irq266: ahci0:ch5 0 0 stray irq266 0 0 irq267: ahci0:6 0 0 stray irq267 0 0 irq268: ahci0:7 0 0 stray irq268 0 0 irq269: ahci0:8 0 0 stray irq269 0 0 irq270: ahci0:9 0 0 stray irq270 0 0 irq271: ahci0:10 0 0 stray irq271 0 0 irq272: ahci0:11 0 0 stray irq272 0 0 irq273: ahci0:12 0 0 stray irq273 0 0 irq274: ahci0:13 0 0 stray irq274 0 0 irq275: ahci0:14 0 0 stray irq275 0 0 irq276: ahci0:15 0 0 stray irq276 0 0 Total 42740 429 ??? 0 0 irq1: atkbd0 0 0 stray irq1 0 0 irq0: attimer0 0 0 stray irq0 0 0 irq3: 0 0 stray irq3 0 0 irq4: uart0 0 0 stray irq4 0 0 irq5: 0 0 stray irq5 0 0 irq6: 0 0 stray irq6 0 0 irq7: 0 0 stray irq7 0 0 irq8: atrtc0 0 0 stray irq8 0 0 irq9: acpi0 0 0 stray irq9 0 0 irq10: 0 0 stray irq10 0 0 irq11: 0 0 stray irq11 0 0 irq12: 0 0 stray irq12 0 0 irq13: 0 0 stray irq13 0 0 irq14: 0 0 stray irq14 0 0 irq15: 0 0 stray irq15 0 0 irq16: em0:irq0++ 0 0 stray irq16 0 0 irq17: 0 0 stray irq17 0 0 irq18: uhci2 ehci0+ 0 0 stray irq18 0 0 irq19: uhci4 0 0 stray irq19 0 0 irq20: hpet0 42 21 stray irq20 0 0 irq21: uhci1 0 0 stray irq21 0 0 irq22: 0 0 stray irq22 0 0 irq23: uhci3 ehci1 0 0 stray irq23 0 0 cpu0:timer 0 0 cpu1:timer 0 0 irq256: hdac0 0 0 stray irq256 0 0 irq257: pcib1 0 0 stray irq257 0 0 irq258: pcib2 0 0 stray irq258 0 0 irq259: pcib3 0 0 stray irq259 0 0 irq260: re0 219 108 stray irq260 0 0 irq261: ahci0:ch0 0 0 stray irq261 0 0 irq262: ahci0:ch1 0 0 stray irq262 0 0 irq263: ahci0:ch2 0 0 stray irq263 0 0 irq264: ahci0:ch3 0 0 stray irq264 0 0 irq265: ahci0:ch4 0 0 stray irq265 0 0 irq266: ahci0:ch5 0 0 stray irq266 0 0 irq267: ahci0:6 0 0 stray irq267 0 0 irq268: ahci0:7 0 0 stray irq268 0 0 irq269: ahci0:8 0 0 stray irq269 0 0 irq270: ahci0:9 0 0 stray irq270 0 0 irq271: ahci0:10 0 0 stray irq271 0 0 irq272: ahci0:11 0 0 stray irq272 0 0 irq273: ahci0:12 0 0 stray irq273 0 0 irq274: ahci0:13 0 0 stray irq274 0 0 irq275: ahci0:14 0 0 stray irq275 0 0 irq276: ahci0:15 0 0 stray irq276 0 0 Total 261 129 jsli@jsli-bsd:/home/jsli #
_______________________________________________ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
