Scott Cheloha <scottchel...@gmail.com> [2022-07-28, 20:34 -0500]: > On Thu, Jul 28, 2022 at 04:57:41PM -0400, Dave Voutila wrote: > >> >> Stuart Henderson <s...@spacehopper.org> writes: >> >> > On 2022/07/28 12:57, Scott Cheloha wrote: >> >> On Thu, Jul 28, 2022 at 07:55:40AM -0400, Dave Voutila wrote: >> >> > >> >> > This is breaking timecounter selection on my x13 Ryzen 5 Pro laptop >> >> > running the latest kernel from snaps. >> >> >> >> Define "breaking". >> > >> > That's clear from the output: >> > >> > : On 2022/07/28 07:55, Dave Voutila wrote: >> > : > $ sysctl -a | grep tsc >> > : > kern.timecounter.choice=i8254(0) tsc(-1000) acpihpet0(1000) >> > : > acpitimer0(1000) >> > : > machdep.tscfreq=2096064730 >> > : > machdep.invarianttsc=1 >> > : > >> > : > $ sysctl kern.timecounter >> > : > kern.timecounter.tick=1 >> > : > kern.timecounter.timestepwarnings=0 >> > : > kern.timecounter.hardware=i8254 >> > : > kern.timecounter.choice=i8254(0) tsc(-1000) acpihpet0(1000) >> > : > acpitimer0(1000) >> > >> >> The code detects TSC desync and marks the timecounter non-monotonic. >> > >> > That's good (and I think as would have happened before) >> > >> >> So it uses the i8254 instead. >> > >> > But that's not so good, there are higher prio timecounters available, >> > acpihpet0 and acpitimer0, which would be better choices than i8254. >> >> Exactly my point. Thanks Stuart. > > Okay, please try this patch on the machine in question. > > It adds a tc_detach() function to kern_tc.c. The first time we fail > the sync test, the BP calls tc_detach(), changes the TSC's tc_quality > to a negative value to tell everyone "this is not monotonic", then > reinstalls the TSC timecounter again with tc_init(). > > Because we are making this call *once*, from one place, I do not think > the O(n) removal time matters, so I have not switched the tc_list from > SLIST to TAILQ. > > It is possible for a thread to be asleep in sysctl_tc_hardware() > during resume, but the thread would be done iterating through the list > if it had reached rw_enter_write(), so removing/adding tsc_timecounter > to the list during resume cannot break list traversal. > > Switching the active timecounter during resume is also fine. The only > race is with tc_adjfreq(). If a thread is asleep in adjfreq(2) when > the system suspends, and we change the active timecounter during > resume, the frequency change may be applied to the "wrong" timecounter. > > ... but this is always a race, because adjfreq(2) only operates on the > active timecounter, and root can change it at any time via sysctl(2). > So it's not a new problem. > > ... > > It might be simpler to just change tc_lock from a rwlock to a mutex. > Then the MP analysis is much simpler across a suspend/resume. > > Index: sys/arch/amd64/amd64/tsc.c > =================================================================== > RCS file: /cvs/src/sys/arch/amd64/amd64/tsc.c,v > retrieving revision 1.24 > diff -u -p -r1.24 tsc.c > --- sys/arch/amd64/amd64/tsc.c 31 Aug 2021 15:11:54 -0000 1.24 > +++ sys/arch/amd64/amd64/tsc.c 29 Jul 2022 01:06:17 -0000 > @@ -36,13 +36,6 @@ int tsc_recalibrate; > uint64_t tsc_frequency; > int tsc_is_invariant; > > -#define TSC_DRIFT_MAX 250 > -#define TSC_SKEW_MAX 100 > -int64_t tsc_drift_observed; > - > -volatile int64_t tsc_sync_val; > -volatile struct cpu_info *tsc_sync_cpu; > - > u_int tsc_get_timecount(struct timecounter *tc); > void tsc_delay(int usecs); > > @@ -236,22 +229,12 @@ cpu_recalibrate_tsc(struct timecounter * > u_int > tsc_get_timecount(struct timecounter *tc) > { > - return rdtsc_lfence() + curcpu()->ci_tsc_skew; > + return rdtsc_lfence(); > } > > void > tsc_timecounter_init(struct cpu_info *ci, uint64_t cpufreq) > { > -#ifdef TSC_DEBUG > - printf("%s: TSC skew=%lld observed drift=%lld\n", ci->ci_dev->dv_xname, > - (long long)ci->ci_tsc_skew, (long long)tsc_drift_observed); > -#endif > - if (ci->ci_tsc_skew < -TSC_SKEW_MAX || ci->ci_tsc_skew > TSC_SKEW_MAX) { > - printf("%s: disabling user TSC (skew=%lld)\n", > - ci->ci_dev->dv_xname, (long long)ci->ci_tsc_skew); > - tsc_timecounter.tc_user = 0; > - } > - > if (!(ci->ci_flags & CPUF_PRIMARY) || > !(ci->ci_flags & CPUF_CONST_TSC) || > !(ci->ci_flags & CPUF_INVAR_TSC)) > @@ -268,111 +251,276 @@ tsc_timecounter_init(struct cpu_info *ci > calibrate_tsc_freq(); > } > > - if (tsc_drift_observed > TSC_DRIFT_MAX) { > - printf("ERROR: %lld cycle TSC drift observed\n", > - (long long)tsc_drift_observed); > - tsc_timecounter.tc_quality = -1000; > - tsc_timecounter.tc_user = 0; > - tsc_is_invariant = 0; > - } > - > tc_init(&tsc_timecounter); > } > > -/* > - * Record drift (in clock cycles). Called during AP startup. > - */ > void > -tsc_sync_drift(int64_t drift) > +tsc_delay(int usecs) > { > - if (drift < 0) > - drift = -drift; > - if (drift > tsc_drift_observed) > - tsc_drift_observed = drift; > + uint64_t interval, start; > + > + interval = (uint64_t)usecs * tsc_frequency / 1000000; > + start = rdtsc_lfence(); > + while (rdtsc_lfence() - start < interval) > + CPU_BUSY_CYCLE(); > } > > +#ifdef MULTIPROCESSOR > + > +#define TSC_DEBUG 1 > + > +/* > + * Protections for global variables in this code: > + * > + * a Modified atomically > + * b Protected by a barrier > + * p Only modified by the primary CPU > + */ > + > +#define TSC_TEST_MS 1 /* Test round duration */ > +#define TSC_TEST_ROUNDS 2 /* Number of test rounds */ > + > /* > - * Called during startup of APs, by the boot processor. Interrupts > - * are disabled on entry. > + * tsc_test_status.val is isolated to its own cache line to limit > + * false sharing and reduce the test's margin of error. > */ > +struct tsc_test_status { > + volatile uint64_t val; /* [b] Latest RDTSC value */ > + uint64_t pad1[7]; > + uint64_t lag_count; /* [b] Number of lags seen by CPU */ > + uint64_t lag_max; /* [b] Biggest lag seen by CPU */ > + int64_t adj; /* [b] Initial IA32_TSC_ADJUST value */ > + uint64_t pad2[5]; > +} __aligned(64); > +struct tsc_test_status tsc_ap_status; /* [b] Test results from AP */ > +struct tsc_test_status tsc_bp_status; /* [p] Test results from BP */ > +uint64_t tsc_test_cycles; /* [p] TSC cycles per test round */ > +const char *tsc_ap_name; /* [b] Name of AP running test */ > +volatile u_int tsc_egress_barrier; /* [a] Test end barrier */ > +volatile u_int tsc_ingress_barrier; /* [a] Test start barrier */ > +volatile u_int tsc_test_rounds; /* [p] Remaining test rounds */ > +int tsc_is_synchronized = 1; /* [p] TSC sync'd across all CPUs? */ > + > +void tsc_report_test_results(void); > +void tsc_reset_adjust(struct tsc_test_status *); > +void tsc_test_ap(void); > +void tsc_test_bp(void); > + > void > -tsc_read_bp(struct cpu_info *ci, uint64_t *bptscp, uint64_t *aptscp) > +tsc_test_sync_bp(struct cpu_info *ci) > { > - uint64_t bptsc; > - > - if (atomic_swap_ptr(&tsc_sync_cpu, ci) != NULL) > - panic("tsc_sync_bp: 1"); > + if (!tsc_is_invariant) > + return; > +#ifndef TSC_DEBUG > + /* No point in testing again if we already failed. */ > + if (!tsc_is_synchronized) > + return; > +#endif > + /* Reset IA32_TSC_ADJUST if it exists. */ > + tsc_reset_adjust(&tsc_bp_status); > > - /* Flag it and read our TSC. */ > - atomic_setbits_int(&ci->ci_flags, CPUF_SYNCTSC); > - bptsc = (rdtsc_lfence() >> 1); > + /* Reset the test cycle limit and round count. */ > + tsc_test_cycles = TSC_TEST_MS * tsc_frequency / 1000; > + tsc_test_rounds = TSC_TEST_ROUNDS; > + > + do { > + /* > + * Pass through the ingress barrier, run the test, > + * then wait for the AP to reach the egress barrier. > + */ > + atomic_inc_int(&tsc_ingress_barrier); > + while (tsc_ingress_barrier != 2) > + CPU_BUSY_CYCLE(); > + tsc_test_bp(); > + while (tsc_egress_barrier != 1) > + CPU_BUSY_CYCLE(); > + > + /* > + * Report what happened. Adjust the TSC's tc_quality > + * if this is the first time we've failed the test. > + */ > + tsc_report_test_results(); > + if (tsc_ap_status.lag_count || tsc_bp_status.lag_count) { > + if (tsc_is_synchronized) { > + tsc_is_synchronized = 0; > + tc_detach(&tsc_timecounter); > + tsc_timecounter.tc_quality = -1000; > + tc_init(&tsc_timecounter); > + } > + tsc_test_rounds = 0; > + } else > + tsc_test_rounds--; > + > + /* > + * Clean up for the next round. It is safe to reset the > + * ingress barrier because at this point we know the AP > + * has reached the egress barrier. > + */ > + memset(&tsc_ap_status, 0, sizeof tsc_ap_status); > + memset(&tsc_bp_status, 0, sizeof tsc_bp_status); > + tsc_ingress_barrier = 0; > + if (tsc_test_rounds == 0) > + tsc_ap_name = NULL; > + > + /* > + * Pass through the egress barrier and release the AP. > + * The AP is responsible for resetting the egress barrier. > + */ > + if (atomic_inc_int_nv(&tsc_egress_barrier) != 2) > + panic("%s: unexpected egress count", __func__); > + } while (tsc_test_rounds > 0); > +} > > - /* Wait for remote to complete, and read ours again. */ > - while ((ci->ci_flags & CPUF_SYNCTSC) != 0) > - membar_consumer(); > - bptsc += (rdtsc_lfence() >> 1); > +void > +tsc_test_sync_ap(struct cpu_info *ci) > +{ > + if (!tsc_is_invariant) > + return; > +#ifndef TSC_DEBUG > + if (!tsc_is_synchronized) > + return; > +#endif > + /* The BP needs our name in order to report any problems. */ > + if (atomic_cas_ptr(&tsc_ap_name, NULL, ci->ci_dev->dv_xname) != NULL) { > + panic("%s: %s: tsc_ap_name is not NULL: %s", > + __func__, ci->ci_dev->dv_xname, tsc_ap_name); > + } > > - /* Wait for the results to come in. */ > - while (tsc_sync_cpu == ci) > - CPU_BUSY_CYCLE(); > - if (tsc_sync_cpu != NULL) > - panic("tsc_sync_bp: 2"); > + tsc_reset_adjust(&tsc_ap_status); > > - *bptscp = bptsc; > - *aptscp = tsc_sync_val; > + /* > + * The AP is only responsible for running the test and > + * resetting the egress barrier. The BP handles everything > + * else. > + */ > + do { > + atomic_inc_int(&tsc_ingress_barrier); > + while (tsc_ingress_barrier != 2) > + CPU_BUSY_CYCLE(); > + tsc_test_ap(); > + atomic_inc_int(&tsc_egress_barrier); > + while (atomic_cas_uint(&tsc_egress_barrier, 2, 0) != 2) > + CPU_BUSY_CYCLE(); > + } while (tsc_test_rounds > 0); > } > > void > -tsc_sync_bp(struct cpu_info *ci) > +tsc_report_test_results(void) > { > - uint64_t bptsc, aptsc; > + u_int round = TSC_TEST_ROUNDS - tsc_test_rounds + 1; > > - tsc_read_bp(ci, &bptsc, &aptsc); /* discarded - cache effects */ > - tsc_read_bp(ci, &bptsc, &aptsc); > - > - /* Compute final value to adjust for skew. */ > - ci->ci_tsc_skew = bptsc - aptsc; > + if (tsc_bp_status.adj != 0) { > + printf("tsc: cpu0: IA32_TSC_ADJUST: %lld -> 0\n", > + tsc_bp_status.adj); > + } > + if (tsc_ap_status.adj != 0) { > + printf("tsc: %s: IA32_TSC_ADJUST: %lld -> 0\n", > + tsc_ap_name, tsc_ap_status.adj); > + } > + if (tsc_ap_status.lag_count > 0 || tsc_bp_status.lag_count > 0) { > + printf("tsc: cpu0/%s: sync test round %u/%u failed\n", > + tsc_ap_name, round, TSC_TEST_ROUNDS); > + } > + if (tsc_bp_status.lag_count > 0) { > + printf("tsc: cpu0/%s: cpu0: %llu lags %llu cycles\n", > + tsc_ap_name, tsc_bp_status.lag_count, > + tsc_bp_status.lag_max); > + } > + if (tsc_ap_status.lag_count > 0) { > + printf("tsc: cpu0/%s: %s: %llu lags %llu cycles\n", > + tsc_ap_name, tsc_ap_name, tsc_ap_status.lag_count, > + tsc_ap_status.lag_max); > + } > } > > /* > - * Called during startup of AP, by the AP itself. Interrupts are > - * disabled on entry. > + * Reset IA32_TSC_ADJUST if we have it. > + * > + * XXX We should rearrange cpu_hatch() so that the feature > + * flags are already set before we get here. Check CPUID > + * by hand until then. > */ > void > -tsc_post_ap(struct cpu_info *ci) > +tsc_reset_adjust(struct tsc_test_status *tts) > { > - uint64_t tsc; > - > - /* Wait for go-ahead from primary. */ > - while ((ci->ci_flags & CPUF_SYNCTSC) == 0) > - membar_consumer(); > - tsc = (rdtsc_lfence() >> 1); > - > - /* Instruct primary to read its counter. */ > - atomic_clearbits_int(&ci->ci_flags, CPUF_SYNCTSC); > - tsc += (rdtsc_lfence() >> 1); > - > - /* Post result. Ensure the whole value goes out atomically. */ > - (void)atomic_swap_64(&tsc_sync_val, tsc); > + uint32_t eax, ebx, ecx, edx; > > - if (atomic_swap_ptr(&tsc_sync_cpu, NULL) != ci) > - panic("tsc_sync_ap"); > + CPUID(0, eax, ebx, ecx, edx); > + if (eax >= 7) { > + CPUID_LEAF(7, 0, eax, ebx, ecx, edx); > + if (ISSET(ebx, SEFF0EBX_TSC_ADJUST)) { > + tts->adj = rdmsr(MSR_TSC_ADJUST); > + if (tts->adj != 0) > + wrmsr(MSR_TSC_ADJUST, 0); > + } > + } > } > > void > -tsc_sync_ap(struct cpu_info *ci) > +tsc_test_ap(void) > { > - tsc_post_ap(ci); > - tsc_post_ap(ci); > + uint64_t ap_val, bp_val, end, lag; > + u_int i = 0; > + > + ap_val = rdtsc_lfence(); > + end = ap_val + tsc_test_cycles; > + while (ap_val < end) { > + /* > + * The LFENCE ensures bp_val predates ap_val. If ap_val > + * is smaller than bp_val then the AP's TSC must lag that > + * of the BP and the counters cannot be synchronized. > + */ > + bp_val = tsc_bp_status.val; > + ap_val = rdtsc_lfence(); > + tsc_ap_status.val = ap_val; > + > + /* > + * Ensure the other CPU has a chance to run. This is > + * crucial if the other CPU is our SMT sibling. SMT > + * starvation can prevent this test from detecting > + * relatively large lags. Eight is an arbitrary value, > + * but it seems to work in practice without impacting > + * the sensitivity of the test for non-sibling threads. > + */ > + if (++i % 8 == 0) > + CPU_BUSY_CYCLE(); > + > + /* > + * Record the magnitude of the problem if our TSC lags > + * the other. > + */ > + if (ap_val < bp_val) { > + tsc_ap_status.lag_count++; > + lag = bp_val - ap_val; > + if (tsc_ap_status.lag_max < lag) > + tsc_ap_status.lag_max = lag; > + } > + } > } > > void > -tsc_delay(int usecs) > +tsc_test_bp(void) > { > - uint64_t interval, start; > + uint64_t ap_val, bp_val, end, lag; > + u_int i = 0; > > - interval = (uint64_t)usecs * tsc_frequency / 1000000; > - start = rdtsc_lfence(); > - while (rdtsc_lfence() - start < interval) > - CPU_BUSY_CYCLE(); > + bp_val = rdtsc_lfence(); > + end = bp_val + tsc_test_cycles; > + while (bp_val < end) { > + ap_val = tsc_ap_status.val; > + bp_val = rdtsc_lfence(); > + tsc_bp_status.val = bp_val; > + > + if (++i % 8 == 0) > + CPU_BUSY_CYCLE(); > + > + if (bp_val < ap_val) { > + tsc_bp_status.lag_count++; > + lag = ap_val - bp_val; > + if (tsc_bp_status.lag_max < lag) > + tsc_bp_status.lag_max = lag; > + } > + } > } > + > +#endif /* MULTIPROCESSOR */ > Index: sys/arch/amd64/amd64/cpu.c > =================================================================== > RCS file: /cvs/src/sys/arch/amd64/amd64/cpu.c,v > retrieving revision 1.156 > diff -u -p -r1.156 cpu.c > --- sys/arch/amd64/amd64/cpu.c 26 Apr 2022 08:35:30 -0000 1.156 > +++ sys/arch/amd64/amd64/cpu.c 29 Jul 2022 01:06:17 -0000 > @@ -772,9 +772,9 @@ cpu_init(struct cpu_info *ci) > lcr4(cr4 & ~CR4_PGE); > lcr4(cr4); > > - /* Synchronize TSC */ > + /* Check if TSC is synchronized. */ > if (cold && !CPU_IS_PRIMARY(ci)) > - tsc_sync_ap(ci); > + tsc_test_sync_ap(ci); > #endif > } > > @@ -854,18 +854,14 @@ cpu_start_secondary(struct cpu_info *ci) > #endif > } else { > /* > - * Synchronize time stamp counters. Invalidate cache and > - * synchronize twice (in tsc_sync_bp) to minimize possible > - * cache effects. Disable interrupts to try and rule out any > - * external interference. > + * Test if TSCs are synchronized. Invalidate cache to > + * minimize possible cache effects. Disable interrupts to > + * try to rule out external interference. > */ > s = intr_disable(); > wbinvd(); > - tsc_sync_bp(ci); > + tsc_test_sync_bp(ci); > intr_restore(s); > -#ifdef TSC_DEBUG > - printf("TSC skew=%lld\n", (long long)ci->ci_tsc_skew); > -#endif > } > > if ((ci->ci_flags & CPUF_IDENTIFIED) == 0) { > @@ -890,7 +886,6 @@ void > cpu_boot_secondary(struct cpu_info *ci) > { > int i; > - int64_t drift; > u_long s; > > atomic_setbits_int(&ci->ci_flags, CPUF_GO); > @@ -905,18 +900,11 @@ cpu_boot_secondary(struct cpu_info *ci) > db_enter(); > #endif > } else if (cold) { > - /* Synchronize TSC again, check for drift. */ > - drift = ci->ci_tsc_skew; > + /* Test if TSCs are synchronized again. */ > s = intr_disable(); > wbinvd(); > - tsc_sync_bp(ci); > + tsc_test_sync_bp(ci); > intr_restore(s); > - drift -= ci->ci_tsc_skew; > -#ifdef TSC_DEBUG > - printf("TSC skew=%lld drift=%lld\n", > - (long long)ci->ci_tsc_skew, (long long)drift); > -#endif > - tsc_sync_drift(drift); > } > } > > @@ -942,13 +930,12 @@ cpu_hatch(void *v) > #endif > > /* > - * Synchronize the TSC for the first time. Note that interrupts are > - * off at this point. > + * Test if our TSC is synchronized for the first time. > + * Note that interrupts are off at this point. > */ > wbinvd(); > ci->ci_flags |= CPUF_PRESENT; > - ci->ci_tsc_skew = 0; /* reset on resume */ > - tsc_sync_ap(ci); > + tsc_test_sync_ap(ci); > > lapic_enable(); > lapic_startclock(); > Index: sys/arch/amd64/include/cpu.h > =================================================================== > RCS file: /cvs/src/sys/arch/amd64/include/cpu.h,v > retrieving revision 1.145 > diff -u -p -r1.145 cpu.h > --- sys/arch/amd64/include/cpu.h 12 Jul 2022 04:46:00 -0000 1.145 > +++ sys/arch/amd64/include/cpu.h 29 Jul 2022 01:06:18 -0000 > @@ -207,8 +207,6 @@ struct cpu_info { > paddr_t ci_vmxon_region_pa; > struct vmxon_region *ci_vmxon_region; > > - int64_t ci_tsc_skew; /* counter skew vs cpu0 */ > - > char ci_panicbuf[512]; > > paddr_t ci_vmcs_pa; > @@ -228,7 +226,6 @@ struct cpu_info { > #define CPUF_INVAR_TSC 0x0100 /* CPU has invariant TSC */ > #define CPUF_USERXSTATE 0x0200 /* CPU has curproc's xsave > state */ > > -#define CPUF_SYNCTSC 0x0800 /* Synchronize TSC */ > #define CPUF_PRESENT 0x1000 /* CPU is present */ > #define CPUF_RUNNING 0x2000 /* CPU is running */ > #define CPUF_PAUSE 0x4000 /* CPU is paused in DDB */ > Index: sys/arch/amd64/include/cpuvar.h > =================================================================== > RCS file: /cvs/src/sys/arch/amd64/include/cpuvar.h,v > retrieving revision 1.11 > diff -u -p -r1.11 cpuvar.h > --- sys/arch/amd64/include/cpuvar.h 16 May 2021 04:33:05 -0000 1.11 > +++ sys/arch/amd64/include/cpuvar.h 29 Jul 2022 01:06:18 -0000 > @@ -97,8 +97,7 @@ void identifycpu(struct cpu_info *); > void cpu_init(struct cpu_info *); > void cpu_init_first(void); > > -void tsc_sync_drift(int64_t); > -void tsc_sync_bp(struct cpu_info *); > -void tsc_sync_ap(struct cpu_info *); > +void tsc_test_sync_bp(struct cpu_info *); > +void tsc_test_sync_ap(struct cpu_info *); > > #endif > Index: sys/sys/timetc.h > =================================================================== > RCS file: /cvs/src/sys/sys/timetc.h,v > retrieving revision 1.12 > diff -u -p -r1.12 timetc.h > --- sys/sys/timetc.h 6 Jul 2020 13:33:09 -0000 1.12 > +++ sys/sys/timetc.h 29 Jul 2022 01:06:18 -0000 > @@ -120,6 +120,7 @@ extern struct timekeep *timekeep; > u_int64_t tc_getfrequency(void); > u_int64_t tc_getprecision(void); > void tc_init(struct timecounter *tc); > +void tc_detach(struct timecounter *); > void tc_setclock(const struct timespec *ts); > void tc_setrealtimeclock(const struct timespec *ts); > void tc_ticktock(void); > Index: sys/kern/kern_tc.c > =================================================================== > RCS file: /cvs/src/sys/kern/kern_tc.c,v > retrieving revision 1.76 > diff -u -p -r1.76 kern_tc.c > --- sys/kern/kern_tc.c 23 Jul 2022 22:58:51 -0000 1.76 > +++ sys/kern/kern_tc.c 29 Jul 2022 01:06:18 -0000 > @@ -458,6 +458,36 @@ tc_init(struct timecounter *tc) > timecounter = tc; > } > > +/* > + * Remove the given timecounter from the list of counters. > + * Activate the best remaining counter if it was the active > + * counter. > + */ > +void > +tc_detach(struct timecounter *tc) > +{ > + struct timecounter *best = &dummy_timecounter, *tmp; > + > + if (tc == &dummy_timecounter) > + panic("%s: cannot detach dummy counter", __func__); > + > + SLIST_REMOVE(&tc_list, tc, timecounter, tc_next); > + if (timecounter == tc) { > + SLIST_FOREACH(tmp, &tc_list, tc_next) { > + if (tmp->tc_quality < 0) > + continue; > + if (tmp->tc_quality < best->tc_quality) > + continue; > + if (tmp->tc_quality == best->tc_quality && > + tmp->tc_frequency < best->tc_frequency) > + continue; > + best = tmp; > + } > + enqueue_randomness(best->tc_get_timecount(best)); > + timecounter = best; > + } > +} > + > /* Report the frequency of the current timecounter. */ > u_int64_t > tc_getfrequency(void)
Hi, hijacking this a bit but I have same issue on my Thinkpad E485. This patch seems to fix the selection of timecounter as shown below: before patch: $ sysctl kern.timecounter kern.timecounter.tick=1 kern.timecounter.timestepwarnings=0 kern.timecounter.hardware=i8254 kern.timecounter.choice=i8254(0) tsc(-1000) acpihpet0(1000) acpitimer0(1000) --- after patch: $ sysctl kern.timecounter kern.timecounter.tick=1 kern.timecounter.timestepwarnings=0 kern.timecounter.hardware=acpihpet0 kern.timecounter.choice=i8254(0) tsc(-1000) acpihpet0(1000) acpitimer0(1000) Heres my dmesg: OpenBSD 7.2-beta (GENERIC.MP) #18: Fri Jul 29 08:16:21 EEST 2022 t...@asteroid.bittivirhe.fi:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 16733204480 (15958MB) avail mem = 16208703488 (15457MB) random: good seed from bootblocks mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 3.1 @ 0x986eb000 (62 entries) bios0: vendor LENOVO version "R0UET81W (1.61 )" date 03/17/2022 bios0: LENOVO 20KUCTO1WW acpi0 at bios0: ACPI 5.0 acpi0: sleep states S0 S3 S4 S5 acpi0: tables DSDT FACP SSDT SSDT CRAT CDIT SSDT TPM2 UEFI MSDM BATB HPET APIC MCFG SBST WSMT VFCT IVRS FPDT SSDT SSDT SSDT BGRT UEFI SSDT acpi0: wakeup devices GPP0(S3) GPP1(S3) GPP2(S3) GPP3(S3) GPP4(S3) GPP5(S3) GPP6(S3) GP17(S3) XHC0(S3) XHC1(S3) GP18(S3) LID_(S3) SLPB(S3) acpitimer0 at acpi0: 3579545 Hz, 32 bits acpihpet0 at acpi0: 14318180 Hz acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: AMD Ryzen 5 2500U with Radeon Vega Mobile Gfx, 1996.62 MHz, 17-11-00 cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,IBPB,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu0: 32KB 64b/line 8-way D-cache, 64KB 64b/line 4-way I-cache, 512KB 64b/line 8-way L2 cache, 4MB 64b/line 16-way L3 cache cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges cpu0: apic clock running at 24MHz cpu0: mwait min=64, max=64, C-substates=1.1, IBE cpu1 at mainbus0: apid 1 (application processor) tsc: cpu0/cpu1: sync test round 1/2 failed tsc: cpu0/cpu1: cpu0: 35234 lags 7193398152 cycles cpu1: AMD Ryzen 5 2500U with Radeon Vega Mobile Gfx, 1996.23 MHz, 17-11-00 cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,IBPB,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu1: 32KB 64b/line 8-way D-cache, 64KB 64b/line 4-way I-cache, 512KB 64b/line 8-way L2 cache, 4MB 64b/line 16-way L3 cache cpu1: smt 1, core 0, package 0 cpu2 at mainbus0: apid 2 (application processor) tsc: cpu0/cpu2: sync test round 1/2 failed tsc: cpu0/cpu2: cpu0: 19367 lags 7193398152 cycles cpu2: AMD Ryzen 5 2500U with Radeon Vega Mobile Gfx, 1996.23 MHz, 17-11-00 cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,IBPB,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu2: 32KB 64b/line 8-way D-cache, 64KB 64b/line 4-way I-cache, 512KB 64b/line 8-way L2 cache, 4MB 64b/line 16-way L3 cache cpu2: smt 0, core 1, package 0 cpu3 at mainbus0: apid 3 (application processor) tsc: cpu0/cpu3: sync test round 1/2 failed tsc: cpu0/cpu3: cpu0: 21199 lags 7193398152 cycles cpu3: AMD Ryzen 5 2500U with Radeon Vega Mobile Gfx, 1996.23 MHz, 17-11-00 cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,IBPB,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu3: 32KB 64b/line 8-way D-cache, 64KB 64b/line 4-way I-cache, 512KB 64b/line 8-way L2 cache, 4MB 64b/line 16-way L3 cache cpu3: smt 1, core 1, package 0 cpu4 at mainbus0: apid 4 (application processor) tsc: cpu0/cpu4: sync test round 1/2 failed tsc: cpu0/cpu4: cpu0: 19329 lags 7193398132 cycles cpu4: AMD Ryzen 5 2500U with Radeon Vega Mobile Gfx, 1996.23 MHz, 17-11-00 cpu4: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,IBPB,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu4: 32KB 64b/line 8-way D-cache, 64KB 64b/line 4-way I-cache, 512KB 64b/line 8-way L2 cache, 4MB 64b/line 16-way L3 cache cpu4: smt 0, core 2, package 0 cpu5 at mainbus0: apid 5 (application processor) tsc: cpu0/cpu5: sync test round 1/2 failed tsc: cpu0/cpu5: cpu0: 20717 lags 7193398132 cycles cpu5: AMD Ryzen 5 2500U with Radeon Vega Mobile Gfx, 1996.23 MHz, 17-11-00 cpu5: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,IBPB,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu5: 32KB 64b/line 8-way D-cache, 64KB 64b/line 4-way I-cache, 512KB 64b/line 8-way L2 cache, 4MB 64b/line 16-way L3 cache cpu5: smt 1, core 2, package 0 cpu6 at mainbus0: apid 6 (application processor) tsc: cpu0/cpu6: sync test round 1/2 failed tsc: cpu0/cpu6: cpu0: 18741 lags 7193398132 cycles cpu6: AMD Ryzen 5 2500U with Radeon Vega Mobile Gfx, 1996.23 MHz, 17-11-00 cpu6: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,IBPB,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu6: 32KB 64b/line 8-way D-cache, 64KB 64b/line 4-way I-cache, 512KB 64b/line 8-way L2 cache, 4MB 64b/line 16-way L3 cache cpu6: smt 0, core 3, package 0 cpu7 at mainbus0: apid 7 (application processor) tsc: cpu0/cpu7: sync test round 1/2 failed tsc: cpu0/cpu7: cpu0: 20352 lags 7193398132 cycles cpu7: AMD Ryzen 5 2500U with Radeon Vega Mobile Gfx, 1996.23 MHz, 17-11-00 cpu7: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,IBPB,XSAVEOPT,XSAVEC,XGETBV1,XSAVES cpu7: 32KB 64b/line 8-way D-cache, 64KB 64b/line 4-way I-cache, 512KB 64b/line 8-way L2 cache, 4MB 64b/line 16-way L3 cache cpu7: smt 1, core 3, package 0 ioapic0 at mainbus0: apid 32 pa 0xfec00000, version 21, 24 pins, can't remap ioapic1 at mainbus0: apid 33 pa 0xfec01000, version 21, 32 pins, can't remap acpimcfg0 at acpi0 acpimcfg0: addr 0xf8000000, bus 0-63 acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 1 (GPP0) acpiprt2 at acpi0: bus 2 (GPP1) acpiprt3 at acpi0: bus 3 (GPP2) acpiprt4 at acpi0: bus -1 (GPP3) acpiprt5 at acpi0: bus -1 (GPP4) acpiprt6 at acpi0: bus 4 (GPP5) acpiprt7 at acpi0: bus -1 (GPP6) acpiprt8 at acpi0: bus 5 (GP17) acpiprt9 at acpi0: bus 6 (GP18) acpiec0 at acpi0 acpibtn0 at acpi0: PWRB acpipci0 at acpi0 PCI0: 0x00000010 0x00000011 0x00000000 acpicmos0 at acpi0 acpibat0 at acpi0: BAT0 model "01AV445" serial 2591 type LiP oem "LGC" acpiac0 at acpi0: AC unit online acpithinkpad0 at acpi0: version 2.0 "SMB0001" at acpi0 not configured acpibtn1 at acpi0: LID_ acpibtn2 at acpi0: SLPB "PNP0C14" at acpi0 not configured "PNP0C14" at acpi0 not configured "PNP0C14" at acpi0 not configured tpm0 at acpi0 TPM_ 2.0 (TIS) addr 0xfed40000/0x5000, device 0x0000104a rev 0x4e "USBC000" at acpi0 not configured acpicpu0 at acpi0: C2(0@400 io@0x414), C1(0@1 mwait), PSS acpicpu1 at acpi0: C2(0@400 io@0x414), C1(0@1 mwait), PSS acpicpu2 at acpi0: C2(0@400 io@0x414), C1(0@1 mwait), PSS acpicpu3 at acpi0: C2(0@400 io@0x414), C1(0@1 mwait), PSS acpicpu4 at acpi0: C2(0@400 io@0x414), C1(0@1 mwait), PSS acpicpu5 at acpi0: C2(0@400 io@0x414), C1(0@1 mwait), PSS acpicpu6 at acpi0: C2(0@400 io@0x414), C1(0@1 mwait), PSS acpicpu7 at acpi0: C2(0@400 io@0x414), C1(0@1 mwait), PSS acpipwrres0 at acpi0: P0ST, resource for SATA acpipwrres1 at acpi0: P3ST, resource for SATA acpivideo0 at acpi0: VGA_ acpivout0 at acpivideo0: LCD_ cpu0: 1996 MHz: speeds: 2000 1700 1600 MHz pci0 at mainbus0 bus 0 ksmn0 at pci0 dev 0 function 0 "AMD 17h/1xh Root Complex" rev 0x00 "AMD 17h/1xh IOMMU" rev 0x00 at pci0 dev 0 function 2 not configured pchb0 at pci0 dev 1 function 0 "AMD 17h PCIE" rev 0x00 ppb0 at pci0 dev 1 function 1 "AMD 17h/1xh PCIE" rev 0x00: msi pci1 at ppb0 bus 1 nvme0 at pci1 dev 0 function 0 "Lenovo NVMe" rev 0x00: msix, NVMe 1.2 nvme0: LENSE20256GMSP34MEAT2TA, firmware 2.6.8341, serial 1227066205564 scsibus1 at nvme0: 2 targets, initiator 0 sd0 at scsibus1 targ 1 lun 0: <NVMe, LENSE20256GMSP34, 2.6.> sd0: 244198MB, 512 bytes/sector, 500118192 sectors ppb1 at pci0 dev 1 function 2 "AMD 17h/1xh PCIE" rev 0x00: msi pci2 at ppb1 bus 2 re0 at pci2 dev 0 function 0 "Realtek 8168" rev 0x10: RTL8168GU/8111GU (0x5080), msi, address e8:6a:64:33:83:cc rgephy0 at re0 phy 7: RTL8251 PHY, rev. 0 ppb2 at pci0 dev 1 function 3 "AMD 17h/1xh PCIE" rev 0x00: msi pci3 at ppb2 bus 3 sdhc0 at pci3 dev 0 function 0 "O2 Micro 0Z8621 SD/MMC" rev 0x01: apic 33 int 8 sdhc0: SDHC 4.0, 50 MHz base clock sdmmc0 at sdhc0: 4-bit, sd high-speed, mmc high-speed, ddr52, dma ppb3 at pci0 dev 1 function 6 "AMD 17h/1xh PCIE" rev 0x00: msi pci4 at ppb3 bus 4 iwm0 at pci4 dev 0 function 0 "Intel Dual Band Wireless-AC 8265" rev 0x78, msi pchb1 at pci0 dev 8 function 0 "AMD 17h PCIE" rev 0x00 ppb4 at pci0 dev 8 function 1 "AMD 17h/1xh PCIE" rev 0x00 pci5 at ppb4 bus 5 amdgpu0 at pci5 dev 0 function 0 "ATI Radeon Vega" rev 0xc4 drm0 at amdgpu0 amdgpu0: msi azalia0 at pci5 dev 0 function 1 "ATI Radeon Vega HD Audio" rev 0x00: msi azalia0: no supported codecs ccp0 at pci5 dev 0 function 2 "AMD 17h/1xh Crypto" rev 0x00 xhci0 at pci5 dev 0 function 3 "AMD 17h/1xh xHCI" rev 0x00: msi, xHCI 1.10 usb0 at xhci0: USB revision 3.0 uhub0 at usb0 configuration 1 interface 0 "AMD xHCI root hub" rev 3.00/1.00 addr 1 xhci1 at pci5 dev 0 function 4 "AMD 17h/1xh xHCI" rev 0x00: msi, xHCI 1.10 usb1 at xhci1: USB revision 3.0 uhub1 at usb1 configuration 1 interface 0 "AMD xHCI root hub" rev 3.00/1.00 addr 1 azalia1 at pci5 dev 0 function 6 "AMD 17h/1xh HD Audio" rev 0x00: apic 33 int 30 azalia1: codecs: Conexant/0x5111 audio0 at azalia1 ppb5 at pci0 dev 8 function 2 "AMD 17h/1xh PCIE" rev 0x00 pci6 at ppb5 bus 6 ahci0 at pci6 dev 0 function 0 "AMD FCH AHCI" rev 0x61: msi, AHCI 1.3.1 ahci0: port 0: 6.0Gb/s scsibus2 at ahci0: 32 targets sd1 at scsibus2 targ 0 lun 0: <ATA, Samsung SSD 850, EXM0> naa.50025388400c34c6 sd1: 488386MB, 512 bytes/sector, 1000215216 sectors, thin piixpm0 at pci0 dev 20 function 0 "AMD FCH SMBus" rev 0x61: SMI iic0 at piixpm0 admtemp0 at iic0 addr 0x18: Xeon admtemp1 at iic0 addr 0x1a: Xeon spdmem0 at iic0 addr 0x50: 8GB DDR4 SDRAM PC4-19200 SO-DIMM with thermal sensor spdmem1 at iic0 addr 0x52: 8GB DDR4 SDRAM PC4-19200 SO-DIMM with thermal sensor iic1 at piixpm0 pcib0 at pci0 dev 20 function 3 "AMD FCH LPC" rev 0x51 pchb2 at pci0 dev 24 function 0 "AMD 17h/1xh Data Fabric" rev 0x00 pchb3 at pci0 dev 24 function 1 "AMD 17h/1xh Data Fabric" rev 0x00 pchb4 at pci0 dev 24 function 2 "AMD 17h/1xh Data Fabric" rev 0x00 pchb5 at pci0 dev 24 function 3 "AMD 17h/1xh Data Fabric" rev 0x00 pchb6 at pci0 dev 24 function 4 "AMD 17h/1xh Data Fabric" rev 0x00 pchb7 at pci0 dev 24 function 5 "AMD 17h/1xh Data Fabric" rev 0x00 pchb8 at pci0 dev 24 function 6 "AMD 17h/1xh Data Fabric" rev 0x00 pchb9 at pci0 dev 24 function 7 "AMD 17h/1xh Data Fabric" rev 0x00 isa0 at pcib0 isadma0 at isa0 pckbc0 at isa0 port 0x60/5 irq 1 irq 12 pckbd0 at pckbc0 (kbd slot) wskbd0 at pckbd0: console keyboard pms0 at pckbc0 (aux slot) wsmouse0 at pms0 mux 0 wsmouse1 at pms0 mux 0 pms0: Synaptics clickpad, firmware 8.16, 0x1e2b1 0x940300 0x373740 0xf016a3 0x12e800 pcppi0 at isa0 port 0x61 spkr0 at pcppi0 vmm0 at mainbus0: SVM/RVI efifb at mainbus0 not configured vscsi0 at root scsibus3 at vscsi0: 256 targets softraid0 at root scsibus4 at softraid0: 256 targets sd2 at scsibus4 targ 1 lun 0: <OPENBSD, SR CRYPTO, 006> sd2: 227678MB, 512 bytes/sector, 466284711 sectors root on sd2a (88532b67c09ce3ee.a) swap on sd2b dump on sd2b iwm0: hw rev 0x230, fw ver 36.ca7b901d.0, address 68:ec:c5:ad:9a:cb amdgpu0: RAVEN 8 CU rev 0x01 amdgpu0: 1920x1080, 32bpp wsdisplay0 at amdgpu0 mux 1: console (std, vt100 emulation), using wskbd0 wsdisplay0: screen 1-5 added (std, vt100 emulation)