Re: PSA: Clock drift and pkgin
I believe you. But aren't we now getting into pretty realtime stuff? Not sure NetBSD is at all suitable for such applications/environments. As you say - if this only behaves acceptably if the system is not under load, then it's not a solution I would go for. But again - I guess we're talking very personal opinions/experiences/tradeoffs here. I still don't really believe we need very high resolution scheduling. And tickless is sortof a separate topic to this. Johnny On 2023-12-31 05:20, Konrad Schroder wrote: On 12/30/2023 3:42 PM, Johnny Billquist wrote: On 2023-12-31 00:11, Michael van Elst wrote: Better than 100Hz is possible and still precise. Something around 1000Hz is necessary for human interaction. Modern hardware could easily do 100kHz. ? If I remember right, anything less than 200ms is immediate response for a human brain. Which means you can get away with much coarser than even 100Hz. And there are certainly lots of examples of older computers with clocks running in the 10s of ms, where human interaction feels perfect. I'm not sure about visual and auditory sensation, but haptic VR requires position updates >= 1000Hz to get texture right. The timing of two impulses that close together may not be felt as two separate events, but the frequency of vibrations within the skin when it interacts with a surface (even through a tool, such as a stylus) is encoded by the nerve endings in the skin itself. We used to use PHANTOM haptic arms at $WORK, driven by an Indigo2. If the control loop operated at less than 1000Hz---for example, if the Indigo2 was under load--- it introduced noticeable differences in the sensation of running the pen over a virtual object. The simulation was much more sensitive to that than it was to the timing of the video output, for which anything greater than 72Hz was wasted. Take care, -Konrad -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
re: Perceivable time differences [was Re: PSA: Clock drift and pkgin]
Johnny Billquist writes: > Ok. I oversimplified. > > If I remember right, the point was that something sub 200ms is perceived > by the brain as being "instananeous" response. It don't mean that one > cannot discern shorter times, just that from an action-reaction point of > view, anything below 200ms is "good enough". > > My point was merely that I don't believe you need to have something down > to ms resolution when it comes to human interaction, which was the claim > I reacted to. mouse's example is actually not the limits. humans can tell audio down to about 1ms or less timing in the best cases. reaction time has nothing to do with expected time when you're doing music things. you can't react to the beat, you have to be ready to go at the same time, *MUCH* closer than 200ms, so that all the musicians in a band are in sync. what one needs from their computer is different for each of us and while most tasks are fine with our current tickless setup, there are plenty of cases we can do better with it. note that tickless and hi-res timers are not really the same thing even if we can achieve one by implementing the other, we *could* introduce hi-res timers on machines with it, but it would be easier with a tickless framework to use. .mrg.
Re: PSA: Clock drift and pkgin
On Saturday, December 30, 2023 at 10:43:34 AM PST, Martin Husemann wrote: > Kernels on that machines just would not run fully tickless. That makes sense. I personallly would call that "tickless where possible", not "fullly tickless".
Re: PSA: Clock drift and pkgin
On 12/30/2023 3:42 PM, Johnny Billquist wrote: On 2023-12-31 00:11, Michael van Elst wrote: Better than 100Hz is possible and still precise. Something around 1000Hz is necessary for human interaction. Modern hardware could easily do 100kHz. ? If I remember right, anything less than 200ms is immediate response for a human brain. Which means you can get away with much coarser than even 100Hz. And there are certainly lots of examples of older computers with clocks running in the 10s of ms, where human interaction feels perfect. I'm not sure about visual and auditory sensation, but haptic VR requires position updates >= 1000Hz to get texture right. The timing of two impulses that close together may not be felt as two separate events, but the frequency of vibrations within the skin when it interacts with a surface (even through a tool, such as a stylus) is encoded by the nerve endings in the skin itself. We used to use PHANTOM haptic arms at $WORK, driven by an Indigo2. If the control loop operated at less than 1000Hz---for example, if the Indigo2 was under load--- it introduced noticeable differences in the sensation of running the pen over a virtual object. The simulation was much more sensitive to that than it was to the timing of the video output, for which anything greater than 72Hz was wasted. Take care, -Konrad
Re: Perceivable time differences [was Re: PSA: Clock drift and pkgin]
On Sun, Dec 31, 2023 at 02:54:50AM +0100, Johnny Billquist wrote: > Ok. I oversimplified. > > If I remember right, the point was that something sub 200ms is perceived by > the brain as being "instananeous" response. It don't mean that one cannot > discern shorter times, just that from an action-reaction point of view, > anything below 200ms is "good enough". The usual figure cited is 100 ms, not 200, but yeah. it is instructive to look at the stopwatch function on a digital watch; you can easily see the tenths counting but not the 100ths. -- David A. Holland dholl...@netbsd.org
Re: Perceivable time differences [was Re: PSA: Clock drift and pkgin]
Ok. I oversimplified. If I remember right, the point was that something sub 200ms is perceived by the brain as being "instananeous" response. It don't mean that one cannot discern shorter times, just that from an action-reaction point of view, anything below 200ms is "good enough". My point was merely that I don't believe you need to have something down to ms resolution when it comes to human interaction, which was the claim I reacted to. Johnny On 2023-12-31 02:47, Mouse wrote: ? If I remember right, anything less than 200ms is immediate response for a human brain. "Response"? For some purposes, it is. But under the right conditions humans can easily discern time deltas in the sub-200ms range. I just did a little psychoacoustics experiment on myself. First, I generated (44.1kHz) soundfiles containing two single-sample ticks separated by N samples for N being 1, 101, 201, 401, 801, and going up by 800 from there to 6401, with a second of silence before and after (see notes below for the commands used): for d in 0 100 200 400 800 1600 2400 3200 4000 4800 5600 6400 do (count from 0 to 44100 | sed -e "s/.*/0 0 0 0/" echo 0 128 0 128 count from 0 to $d | sed -e "s/.*/0 0 0 0/" echo 0 128 0 128 count from 0 to 44100 | sed -e "s/.*/0 0 0 0/" ) | code-to-char > zz.$d done I don't know stock NetBSD analogs for count and code-to-char. count, as used here, just counts as the command line indicates; given what count's output is piped into, the details don't matter much. code-to-char converts numbers 0..255 into single bytes with the same values, with non-digits ignored except that they serve to separate numbers. (The time delta between the beginnings of the two ticks is of course one more than the number of samples between the two ticks.) After listening to them, I picked the 800 and 1600 files and did the test. I grabbed 128 bits from /dev/urandom and used them to play, randomly, either one file or the other, letting me guess which one it was in each case: dd if=/dev/urandom bs=1 count=16 | char-to-code | cvtbase -m8 d b | sed -e 's/./& /g' -e 's/ $//' -e 's/0/800/g' -e 's/1/1600/g' | tr \ \\n | ( exec 3>zz.list 4>zz.guess 5&3 audioplay -f -c 2 -e slinear_le -P 16 -s 44100 < zz.$n skipcat 0 1 0<&5 1>&4 done ) char-to-code is the inverse of code-to-char: for each byte of input, it produces one line of output containing the ASCII decimal for that byte's value, 0..255. cvtbase -m8 d b converts decimal to binary, generating a minimum of 8 "digits" (bits) of output for each input number. skipcat, as used here, has the I/O behaviour of "dd bs=1 count=1" but without the blather on stderr: it skips no bytes and copies one byte, then exits. (The use of /dev/urandom is to ensure that I have no a priori hint which file is being played which time.) I then typed "s" when I thought it was a short-gap file and "l" when I thought it was a long-gap file. I got tired of it after 83 data samples and killed it. I then postprocessed zz.guess and compared it to zz.list: < zz.guess sed -e 's/s/800 /g' -e 's/l/1600 /g' | tr \ \\n | diff -u zz.list - I got exactly two wrong out of 83 (and the stats are about evenly balanced, 39 short files played and 44 long). So I think it's fair to say that, in the right context (an important caveat!), a time difference as short as (1602-802)/44.1=18.14+ milliseconds is clearly discernible to me. This is, of course, a situation designed to perceive a very small difference. I'm sure there are plenty of contexts in which I would fail to notice even 200ms of delay. /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTML mo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Perceivable time differences [was Re: PSA: Clock drift and pkgin]
> ? If I remember right, anything less than 200ms is immediate response > for a human brain. "Response"? For some purposes, it is. But under the right conditions humans can easily discern time deltas in the sub-200ms range. I just did a little psychoacoustics experiment on myself. First, I generated (44.1kHz) soundfiles containing two single-sample ticks separated by N samples for N being 1, 101, 201, 401, 801, and going up by 800 from there to 6401, with a second of silence before and after (see notes below for the commands used): for d in 0 100 200 400 800 1600 2400 3200 4000 4800 5600 6400 do (count from 0 to 44100 | sed -e "s/.*/0 0 0 0/" echo 0 128 0 128 count from 0 to $d | sed -e "s/.*/0 0 0 0/" echo 0 128 0 128 count from 0 to 44100 | sed -e "s/.*/0 0 0 0/" ) | code-to-char > zz.$d done I don't know stock NetBSD analogs for count and code-to-char. count, as used here, just counts as the command line indicates; given what count's output is piped into, the details don't matter much. code-to-char converts numbers 0..255 into single bytes with the same values, with non-digits ignored except that they serve to separate numbers. (The time delta between the beginnings of the two ticks is of course one more than the number of samples between the two ticks.) After listening to them, I picked the 800 and 1600 files and did the test. I grabbed 128 bits from /dev/urandom and used them to play, randomly, either one file or the other, letting me guess which one it was in each case: dd if=/dev/urandom bs=1 count=16 | char-to-code | cvtbase -m8 d b | sed -e 's/./& /g' -e 's/ $//' -e 's/0/800/g' -e 's/1/1600/g' | tr \ \\n | ( exec 3>zz.list 4>zz.guess 5&3 audioplay -f -c 2 -e slinear_le -P 16 -s 44100 < zz.$n skipcat 0 1 0<&5 1>&4 done ) char-to-code is the inverse of code-to-char: for each byte of input, it produces one line of output containing the ASCII decimal for that byte's value, 0..255. cvtbase -m8 d b converts decimal to binary, generating a minimum of 8 "digits" (bits) of output for each input number. skipcat, as used here, has the I/O behaviour of "dd bs=1 count=1" but without the blather on stderr: it skips no bytes and copies one byte, then exits. (The use of /dev/urandom is to ensure that I have no a priori hint which file is being played which time.) I then typed "s" when I thought it was a short-gap file and "l" when I thought it was a long-gap file. I got tired of it after 83 data samples and killed it. I then postprocessed zz.guess and compared it to zz.list: < zz.guess sed -e 's/s/800 /g' -e 's/l/1600 /g' | tr \ \\n | diff -u zz.list - I got exactly two wrong out of 83 (and the stats are about evenly balanced, 39 short files played and 44 long). So I think it's fair to say that, in the right context (an important caveat!), a time difference as short as (1602-802)/44.1=18.14+ milliseconds is clearly discernible to me. This is, of course, a situation designed to perceive a very small difference. I'm sure there are plenty of contexts in which I would fail to notice even 200ms of delay. /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: PSA: Clock drift and pkgin
mo...@rodents-montreal.org (Mouse) writes: >> Modern hardware could easily do 100kHz. >Not with curren^Wat least one moderately recent NetBSD version! >At work, I had occasion to run 9.1/amd64 with HZ=8000. This was to get >8-bit data pushed out a parallel port at 8kHz; I added special-case >hooks between the relevant driver and the clock (I forget whether >softclock or hardclock). It worked for its intended use fairly >nicely...but when I tried one of my SIGALRM testers on it, instead of >the 100Hz it asked for, I got signals at, IIRC, about 77Hz. Scheduling and switching userland processes is heavy. For a test try to schedule kernel callouts with high HZ values. That still generates lots of overhead with the current design but you should be able to go faster than 8kHz.
Re: PSA: Clock drift and pkgin
On 2023-12-31 00:11, Michael van Elst wrote: On Sat, Dec 30, 2023 at 10:48:26PM +0100, Johnny Billquist wrote: Right. But if you expect high precision on delays and scheduling, then you start also having issues with just random unpredictable delays because of other interrupts, paging, and whatnot. So in the end, your high precision delays and scheduling becomes very imprecise again. So, is there really that much value in that higher resolution? Better than 100Hz is possible and still precise. Something around 1000Hz is necessary for human interaction. Modern hardware could easily do 100kHz. ? If I remember right, anything less than 200ms is immediate response for a human brain. Which means you can get away with much coarser than even 100Hz. And there are certainly lots of examples of older computers with clocks running in the 10s of ms, where human interaction feels perfect. Another advantage is that you can use independent timing (that's what bites in the emulator case where guest and host clocks run at the same rate). I think that is a separate question/problem/issue. That we fail when guest and host run at the same rate is something I consider a flaw in the system. It's technically perfectly possible to run such a combo good, and the fact that we didn't (don't) is just sad (in my opinion). Not sure what you mean by independent timing here. For me, that would be if you had two different clock sources independent of each other. Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: PSA: Clock drift and pkgin
> Better than 100Hz is possible and still precise. Something around > 1000Hz is necessary for human interaction. That doesn't sound right. I've had good HCI experiences with HZ=100. Why do you see a higher HZ as necessary for human interaction? > Modern hardware could easily do 100kHz. Not with curren^Wat least one moderately recent NetBSD version! At work, I had occasion to run 9.1/amd64 with HZ=8000. This was to get 8-bit data pushed out a parallel port at 8kHz; I added special-case hooks between the relevant driver and the clock (I forget whether softclock or hardclock). It worked for its intended use fairly nicely...but when I tried one of my SIGALRM testers on it, instead of the 100Hz it asked for, I got signals at, IIRC, about 77Hz. I never investigated. I think I still have access to the work machine in question if anyone wants me to try any other quick tests, but trying to figure out an issue on a version I don't use except at work is something I am unmotivated to do on my own time, and using work time to dig after an issue that doesn't affect work's use case isn't an appropriate use of work resources. /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: PSA: Clock drift and pkgin
On Sat, Dec 30, 2023 at 10:48:26PM +0100, Johnny Billquist wrote: > > Right. But if you expect high precision on delays and scheduling, then you > start also having issues with just random unpredictable delays because of > other interrupts, paging, and whatnot. So in the end, your high precision > delays and scheduling becomes very imprecise again. So, is there really that > much value in that higher resolution? Better than 100Hz is possible and still precise. Something around 1000Hz is necessary for human interaction. Modern hardware could easily do 100kHz. Another advantage is that you can use independent timing (that's what bites in the emulator case where guest and host clocks run at the same rate). -- Michael van Elst Internet: mlel...@serpens.de "A potential Snark may lurk in every tree."
Re: PSA: Clock drift and pkgin
On 2023-12-30 22:10, Michael van Elst wrote: b...@softjar.se (Johnny Billquist) writes: Being able to measure time with high precision is desierable, but we can already do that without being tickless. We cannot delay with high precision. You can increase HZ to some degree, but that comes at a price. Right. But if you expect high precision on delays and scheduling, then you start also having issues with just random unpredictable delays because of other interrupts, paging, and whatnot. So in the end, your high precision delays and scheduling becomes very imprecise again. So, is there really that much value in that higher resolution? But of course, this all becomes a question of tradeoffs, preferences and desires. Not sure if we need to have an argument about it. I don't know if anyone is working on a tickless design, or how far it has come. I will certainly not complain if someone does it. But I'm personally not feeling much of a lack that we don't have it. Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: PSA: Clock drift and pkgin
b...@softjar.se (Johnny Billquist) writes: >Being able to measure time with high precision is desierable, but we can >already do that without being tickless. We cannot delay with high precision. You can increase HZ to some degree, but that comes at a price.
Re: PSA: Clock drift and pkgin
On 2023-12-30 19:43, Martin Husemann wrote: On Sat, Dec 30, 2023 at 06:25:29PM +, Jonathan Stone wrote: You can only do tickless if you can track how much time is elapsing when no ticks fire, or none are pending. I don't see how to do that without a high-res timer like a CPU cycle counter, or I/O bus cycle counter, or what-have-you. Gong fully tickless would therefore end support for machines without such a timer. Is NetBSD ready to do that? Kernels on that machines just would not run fully tickless. Right. There is no reason to assume that all platforms would have to go tickless just because it becomes a possibility. However, I also am not sure how much value tickless adds here. The main reason I know of for tickless systems is power consumption. Not having to wake up just to count time can make a big difference. Sure, you can get higher precision for some scheduling with tickless, but I'm not sure it generally makes any actual significant difference. Being able to measure time with high precision is desierable, but we can already do that without being tickless. Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: PSA: Clock drift and pkgin
On Sat, Dec 30, 2023 at 06:25:29PM +, Jonathan Stone wrote: > You can only do tickless if you can track how much time is elapsing when no > ticks fire, or none are pending. > I don't see how to do that without a high-res timer like a CPU cycle counter, > or I/O bus cycle counter, > or what-have-you. Gong fully tickless would therefore end support for > machines without such a timer. > Is NetBSD ready to do that? Kernels on that machines just would not run fully tickless. Martin
Re: PSA: Clock drift and pkgin
On Saturday, December 23, 2023 at 10:19:53 PM PST, Simon Burge wrote: > I have a grotty hack that attempted to spin if the requested timeout > was less than a tick based on what DragonflyBSD does. It mostly > worked for simple tests but I haven't tested it seriously. It's at > https://www.NetBSD.org/~simonb/pollfixhack.diff . is that really viable on uniprocessor machines? > This is potentially >another direction until we get a pure tickless kernel... You can only do tickless if you can track how much time is elapsing when no ticks fire, or none are pending. I don't see how to do that without a high-res timer like a CPU cycle counter, or I/O bus cycle counter, or what-have-you. Gong fully tickless would therefore end support for machines without such a timer. Is NetBSD ready to do that?
Re: PSA: Clock drift and pkgin
On 2023-12-25 02:17, Robert Elz wrote: Date:Sun, 24 Dec 2023 13:49:53 +0100 From:Johnny Billquist Message-ID: | In my opinion, all of these POSIX calls that take a time argument should | really have been done the same as clock_gettime(), in that you specify | what clock it should be based on. The next version of POSIX will contain pthread_cond_clockwait() which is just like pthread_cond_timedwait() but has a clock_id parameter. | As it is now, it is (or should be according to POSIX) unconditionally | CLOCK_REALTIME. Not sure about the current released standard, and too lazy to look ... but in the coming one that's not true either: The pthread_cond_timedwait() function shall be equivalent to pthread_cond_clockwait(), except that it lacks the clock_id argument. The clock to measure abstime against shall instead come from the condition variable's clock attribute which can be set by pthread_condattr_setclock() prior to the condition variable's creation. If no clock attribute has been set, the default shall be CLOCK_REALTIME. Happy to see that this is finally getting fixed. Working on embedded systems where sometimes you have no clue what the absolute time is, current POSIX was just close to unusable, and we've had to do some pretty horrible workarounds. Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: PSA: Clock drift and pkgin
On Mon, 2023-12-25 00:26:34 +0100, Johnny Billquist wrote: > But I think the suggestion that the time adjustment might actually be a > source of the problem is interesting, and should be investigated. It just > takes so bloody long to do a full build these days. I still haven't > finished, and can't start chasing this quite yet. The /60 finished installation; upon its first boot from local PiSCSI "disk", I found: [ 1.0672720] WARNING: lwp 30 (system rt_timer) flags 0x2000: timecounter went backwards from (1 + 0x36821f2990edf767/2^6d (cut off as I didn't have linewrap or logging active in `minicom`.) Didn't notice that with the slightly older image, but maybe it's only happening rarely. Waiting for it to fully come up (-> activated `ssh`, so it's generating host keys...) Will then give it a time and let it run idle over the day and loaded over night. (And maybe give that older image a few boots to see whether or not I can the a "timecounter went backwards" message from it...) MfG, JBG -- signature.asc Description: PGP signature
Re: PSA: Clock drift and pkgin
On Sunday, December 24, 2023 at 02:43:55 AM PST, Johnny Billquist wrote: > Oh? So we are actually not POSIX compliant on that one? Interesting. > (POSIX explicitly says that the timeout should be for an absolute time, > which means that if you for example update the clock, moving it > backwards, the timeout should still only happen when that time arrives, > and not after some precomputed number of ticks.) one could keep track, for every timeout, whether it's relative or absolute; and when the time is changed, walk the list of a-yet-unfired timeouts, updating all the "absolute" timeouts by the clock-change delta. Anyway .. I wonder if the "clock drift" is related to the clock drift I've heard about, on machines which don't have a hardware cycle-counter-style clock, and rely on clock-tick interrupts to track time. (for example, pmax 2100/3100; decstation 5000/200; (most) vax). I'd really like to help out with clock-drift', if I can do anything to help.
Re: PSA: Clock drift and pkgin
Date:Sun, 24 Dec 2023 13:49:53 +0100 From:Johnny Billquist Message-ID: | In my opinion, all of these POSIX calls that take a time argument should | really have been done the same as clock_gettime(), in that you specify | what clock it should be based on. The next version of POSIX will contain pthread_cond_clockwait() which is just like pthread_cond_timedwait() but has a clock_id parameter. | As it is now, it is (or should be according to POSIX) unconditionally | CLOCK_REALTIME. Not sure about the current released standard, and too lazy to look ... but in the coming one that's not true either: The pthread_cond_timedwait() function shall be equivalent to pthread_cond_clockwait(), except that it lacks the clock_id argument. The clock to measure abstime against shall instead come from the condition variable's clock attribute which can be set by pthread_condattr_setclock() prior to the condition variable's creation. If no clock attribute has been set, the default shall be CLOCK_REALTIME. kre
Re: PSA: Clock drift and pkgin
On 2023-12-24 20:58, Jonathan Stone wrote: On Sunday, December 24, 2023 at 02:43:55 AM PST, Johnny Billquist wrote: > Oh? So we are actually not POSIX compliant on that one? Interesting. > (POSIX explicitly says that the timeout should be for an absolute time, > which means that if you for example update the clock, moving it > backwards, the timeout should still only happen when that time arrives, > and not after some precomputed number of ticks.) one could keep track, for every timeout, whether it's relative or absolute; and when the time is changed, walk the list of a-yet-unfired timeouts, updating all the "absolute" timeouts by the clock-change delta. One could, indeed. And then it would be compliant. (I'd dislike it, but that's a very personal opinion. :-) ) Anyway .. I wonder if the "clock drift" is related to the clock drift I've heard about, on machines which don't have a hardware cycle-counter-style clock, and rely on clock-tick interrupts to track time. (for example, pmax 2100/3100; decstation 5000/200; (most) vax). I'd really like to help out with clock-drift', if I can do anything to help. I am fairly sure all systems use the clock tick interrupt to track time in the end. No NetBSD port, as far as I know, is running a tickless implementation. But I think the suggestion that the time adjustment might actually be a source of the problem is interesting, and should be investigated. It just takes so bloody long to do a full build these days. I still haven't finished, and can't start chasing this quite yet. Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: PSA: Clock drift and pkgin
On 2023-12-24 11:43, Johnny Billquist wrote: On 2023-12-24 09:26, Michael van Elst wrote: sim...@netbsd.org (Simon Burge) writes: qemu uses ppoll() which is implemented with pollts() to do emulated timers, so that doesn't help here. I don't know what simh uses,nor any of the other emulators. simh uses pthread_cond_timedwait(). This actually waits using TIMER_ABSTIME for a deadline, but which is converted to a timeout with ts2timo() and passed to sleepq_block() as a number of ticks to wait for. Oh? So we are actually not POSIX compliant on that one? Interesting. (POSIX explicitly says that the timeout should be for an absolute time, which means that if you for example update the clock, moving it backwards, the timeout should still only happen when that time arrives, and not after some precomputed number of ticks.) By the way - I should point out that I am not advocating that we change this to be POSIX compliant. I do think POSIX is broken here. In my opinion, all of these POSIX calls that take a time argument should really have been done the same as clock_gettime(), in that you specify what clock it should be based on. As it is now, it is (or should be according to POSIX) unconditionally CLOCK_REALTIME. But depending on what you are doing, CLOCK_MONOTONIC might be what you really wished you could use. I think Linux have some private extension to the whole thing, which makes it possible to pick other clocks, but I've forgotten the details. Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: PSA: Clock drift and pkgin
On 2023-12-24 09:26, Michael van Elst wrote: sim...@netbsd.org (Simon Burge) writes: qemu uses ppoll() which is implemented with pollts() to do emulated timers, so that doesn't help here. I don't know what simh uses, nor any of the other emulators. simh uses pthread_cond_timedwait(). This actually waits using TIMER_ABSTIME for a deadline, but which is converted to a timeout with ts2timo() and passed to sleepq_block() as a number of ticks to wait for. Oh? So we are actually not POSIX compliant on that one? Interesting. (POSIX explicitly says that the timeout should be for an absolute time, which means that if you for example update the clock, moving it backwards, the timeout should still only happen when that time arrives, and not after some precomputed number of ticks.) Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: PSA: Clock drift and pkgin
sim...@netbsd.org (Simon Burge) writes: >qemu uses ppoll() which is implemented with pollts() to do emulated >timers, so that doesn't help here. I don't know what simh uses, nor >any of the other emulators. simh uses pthread_cond_timedwait(). This actually waits using TIMER_ABSTIME for a deadline, but which is converted to a timeout with ts2timo() and passed to sleepq_block() as a number of ticks to wait for. ts2timo() uses tvothz() which rounds up...
Re: PSA: Clock drift and pkgin
Mouse wrote: > Agreed. ITIMER_REAL in the form I've been arguing for is of little > help to a process that wants timer ticks separated by a hard minimum > interval as seen by the signal handler. At least when using > it_interval to get repeated signals. > > But then, so is every other ITIMER_REAL I've ever used. Maybe a possible answer/hack is a new timer like ITIMER_MOSTLY_REAL or ITIMER_TICK or some variation thereof, which doesn't change existing semantics for the current timers? The original discussion that lead to this was timekeeping on a simh vax running on amd64 where both had HZ=100. We also have archs like pmax that use HZ=256 and alpha that use HZ=1024. Timekeeping isn't going to work well on those where the requested HZ is greater than the host HZ. Similarly, the discussions on this thread aren't going to help a 100 HZ simh vax keep time any better if run on a shark with HZ=64(!). qemu uses ppoll() which is implemented with pollts() to do emulated timers, so that doesn't help here. I don't know what simh uses, nor any of the other emulators. I have a grotty hack that attempted to spin if the requested timeout was less than a tick based on what DragonflyBSD does. It mostly worked for simple tests but I haven't tested it seriously. It's at https://www.NetBSD.org/~simonb/pollfixhack.diff . This is potentially another direction until we get a pure tickless kernel... Cheers, Simon.
Re: PSA: Clock drift and pkgin
> So even though we added one tick, you can still get two timer events > in much closer proximity than a single tick as far as the process is > concerned. Certainly. I think that's unavoidable without resetting the timer inside the signal handler, or hard realtime guarantees (which are Not Easy). > And we probably do need to talk about the timer expiration and > rearming as separate from signal deliver and process scheduling. There are plenty of reasons user code running the signal handler may be delayed from the time the timer is supposed to tick. But without the timer ticking as requested, I don't think the rest matters nearly as much. When even an _unloaded_ machine can't get the ticks it asks for, something is wrong. A machine which gets that overloaded just from delivering 100 signals to a mostly-trivial signal handler per second, well, I doubt NetBSD runs on anything that weak. > And from a program point of view, that is what really matters in the > end. If the program really wants a minimum amount of time before the > next timeout, it needs to do the request for the next time event at > the processing point, not something kernel internal which happend > very disconnected from the process. Agreed. ITIMER_REAL in the form I've been arguing for is of little help to a process that wants timer ticks separated by a hard minimum interval as seen by the signal handler. At least when using it_interval to get repeated signals. But then, so is every other ITIMER_REAL I've ever used. /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: PSA: Clock drift and pkgin
By the way, I should point out that adding 1 tick to the reload of the interval timer in no way gets you away from the possibility that you'll get two timer signals with almost 0 time between them. Because the simple truth is that it is completely unknown when the program will actually get the signal for the first timeout. Basically, using a similar notation: Time 0*T: timer_settime(.it_value = T, .it_interval = T), arms timer at 2*T Time 2*T: timer expires, rearms timer at 4*T Time 2*T + 1.9*T: process scheduled and signal delivered Time 4*T: timer expires, rearms timer at 6*T Time 4*T + 0.1*T: process scheduled and signal delivered So even though we added one tick, you can still get two timer events in much closer proximity than a single tick as far as the process is concerned. So the current code actually does nothing to avoid the situation, but prevents running a timer at the same frequency as the system tick. And we probably do need to talk about the timer expiration and rearming as separate from signal deliver and process scheduling. Timer expiration and rearming is happening fairly reliably always close to correct time. However, signal delivery and scheduling can be way off. And from a program point of view, that is what really matters in the end. If the program really wants a minimum amount of time before the next timeout, it needs to do the request for the next time event at the processing point, not something kernel internal which happend very disconnected from the process. Johnny On 2023-12-24 02:22, Johnny Billquist wrote: On 2023-12-23 23:05, Taylor R Campbell wrote: The attached (untested) patch reverts to the old algorithm specifically for the case of rearming a periodic timer, leaving the new algorithm with +1 in place for all other uses. Now, it's unclear to me whether this is correct, because it can have the following effect. Suppose ticks happen on intervals of time T. Then, owing to unpredictable and uncontrollable scheduling delays, the following sequence of events may happen: Time 0*T: timer_settime(.it_value = T, .it_interval = T), arms timer at 1*T Time 1*T + 0.9*T: timer expires, rearms timer at 2*T Time 2*T + 0.1*T: timer expires, rearms timer at 3*T The duration between these consecutive expirations of the timer is 0.2*T, even though we asked for an interval of T. Of course, the _average_ duration will be around T, but the _minimum_ duration is essentially zero. POSIX clearly forbids a _one-shot_ timer, which is scheduled to expire after time T, to actually expire after only time 0.2*T. But the language in POSIX is unclear to me on whether this is allowed for _periodic_ timers: I would argue that the first timeout should not happen with less than T time, so the rounding up of that one is correct. The rearming should be with T. The fact that the user level program might the events with a 0.2*T interval (could even be infinitely close to 0 actually) is an effect of how systems work. You can always have higher priority stuff going on, which delays the scheduling of a process for an unknown amount. Even in soft realtime systems, that is the case. Hard realtime is pretty tricky actually, and a lot of the time, people trying to do realtime don't understand the problems around this. Anyway, this, in the end, goes to the point of what the purpose/usecase/reason for the itimer is. I would like/prefer that they try to give that average interval. Because if not, then it have no difference to the process just doing the call in the timer signal handler itself, and the interval parameter becomes just a convenience to not have to do the call youself, and there is no way to get something that essentially gives a higher chance of getting the average rate to what you ask for. Which seems poorer than having the ability to have both the average, and a way of having a minimum interval. But when it comes to times and timeouts, POSIX have some serious flaws already, and POSIX is close to unusable for anything realtime related anyway. (Try to do a pthread_cond_timedwait() for a specific number of seconds, and consider a time change in between and you'll see the problem.) So it's questionable if POSIX is useful/helpful here anyway. But maybe (as Mouse suggested) we should add/have a second interface which actually provides what is useful, if POSIX insists on an always minimum interval interpretation. Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: PSA: Clock drift and pkgin
On 2023-12-23 23:05, Taylor R Campbell wrote: The attached (untested) patch reverts to the old algorithm specifically for the case of rearming a periodic timer, leaving the new algorithm with +1 in place for all other uses. Now, it's unclear to me whether this is correct, because it can have the following effect. Suppose ticks happen on intervals of time T. Then, owing to unpredictable and uncontrollable scheduling delays, the following sequence of events may happen: Time 0*T: timer_settime(.it_value = T, .it_interval = T), arms timer at 1*T Time 1*T + 0.9*T: timer expires, rearms timer at 2*T Time 2*T + 0.1*T: timer expires, rearms timer at 3*T The duration between these consecutive expirations of the timer is 0.2*T, even though we asked for an interval of T. Of course, the _average_ duration will be around T, but the _minimum_ duration is essentially zero. POSIX clearly forbids a _one-shot_ timer, which is scheduled to expire after time T, to actually expire after only time 0.2*T. But the language in POSIX is unclear to me on whether this is allowed for _periodic_ timers: I would argue that the first timeout should not happen with less than T time, so the rounding up of that one is correct. The rearming should be with T. The fact that the user level program might the events with a 0.2*T interval (could even be infinitely close to 0 actually) is an effect of how systems work. You can always have higher priority stuff going on, which delays the scheduling of a process for an unknown amount. Even in soft realtime systems, that is the case. Hard realtime is pretty tricky actually, and a lot of the time, people trying to do realtime don't understand the problems around this. Anyway, this, in the end, goes to the point of what the purpose/usecase/reason for the itimer is. I would like/prefer that they try to give that average interval. Because if not, then it have no difference to the process just doing the call in the timer signal handler itself, and the interval parameter becomes just a convenience to not have to do the call youself, and there is no way to get something that essentially gives a higher chance of getting the average rate to what you ask for. Which seems poorer than having the ability to have both the average, and a way of having a minimum interval. But when it comes to times and timeouts, POSIX have some serious flaws already, and POSIX is close to unusable for anything realtime related anyway. (Try to do a pthread_cond_timedwait() for a specific number of seconds, and consider a time change in between and you'll see the problem.) So it's questionable if POSIX is useful/helpful here anyway. But maybe (as Mouse suggested) we should add/have a second interface which actually provides what is useful, if POSIX insists on an always minimum interval interpretation. Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: PSA: Clock drift and pkgin
> The attached (untested) patch reverts to the old algorithm > specifically for the case of rearming a periodic timer, leaving the > new algorithm with +1 in place for all other uses. > Now, it's unclear to me whether this is correct, because it can have > the following effect. Suppose ticks happen on intervals of time T. > Then, owing to unpredictable and uncontrollable scheduling delays, > the following sequence of events may happen: > Time 0*T: timer_settime(.it_value =3D T, .it_interval =3D T), arms timer at > 1*T > Time 1*T + 0.9*T: timer expires, rearms timer at 2*T > Time 2*T + 0.1*T: timer expires, rearms timer at 3*T > The duration between these consecutive expirations of the timer is > 0.2*T, even though we asked for an interval of T. True. In my opinion that is the correct behaviour; userland requested timer ticks at multiples of T, so there is a conceptually infinite stream of (conceptual) ticks generated at those times. Those then get turned into real events when the kernel can manage it. But a delay for one of them should not affect any other, except for the case where one is delayed long enough to occur after another's ideal time, in which case I would consider it acceptable (though not required) to drop one of the two. > [...POSIX...] IMO if POSIX forbids the above, POSIX is broken and should, in this respect, be ignored. One reason for using facilities taking structs itimerval is for ticks to _not_ be delayed by delay of previous ticks. If POSIX cannot be ignored for whatever reason, I would argue that a new facility that _does_ provide undelayed ticks should be provided. (I'm partial to timer sockets, but I am hardly unbiased. :-) > On the other hand, if it's not correct to do that, I'm not sure > correct POSIX periodic timers can attain a target _average_ interval > between expirations [...] I would argue that it's misleading, to the point I would call it incorrect, to call such a thing "periodic". /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: PSA: Clock drift and pkgin
> Date: Sat, 23 Dec 2023 12:24:08 -0500 (EST)
> From: Mouse
>
> >> } else if (sec <= (LONG_MAX / 100))
> >> ticks = (((sec * 100) + (unsigned long)usec + (tick -
> >> 1))
> >> / tick) + 1;
>
> > Whether this is a bug depends on whether: [...]
>
> I think that code is not a bug per se; for sleeps, that codepath
> is...well, "reasonable" at the very least. The bug is that it is
> broken for timer reloads, but timer reloads are using it anyway;
> whether you think of this as a bug in timer reloads or a bug in tvtohz
> is a question of which way you prefer to squint your mind.
This was introduced in kern_clock.c 1.62 in July 2000 by thorpej,
specifically to avoid waking up sleeps early by up to one tick's worth
of time:
https://mail-index.netbsd.org/source-changes/2000/07/13/msg081191.html
Author: thorpej
Date: Thu Jul 13 17:06:15 2000 +
New hzto() function from FreeBSD and Artur Grabowski .
Stops sleeps from returning early (by up to a clock tick), and return 0
ticks for timeouts that should happen now or in the past.
Returning 0 is different from the legacy hzto() interface, and callers
need to check for it.
The attached (untested) patch reverts to the old algorithm
specifically for the case of rearming a periodic timer, leaving the
new algorithm with +1 in place for all other uses.
Now, it's unclear to me whether this is correct, because it can have
the following effect. Suppose ticks happen on intervals of time T.
Then, owing to unpredictable and uncontrollable scheduling delays, the
following sequence of events may happen:
Time 0*T: timer_settime(.it_value = T, .it_interval = T), arms timer at 1*T
Time 1*T + 0.9*T: timer expires, rearms timer at 2*T
Time 2*T + 0.1*T: timer expires, rearms timer at 3*T
The duration between these consecutive expirations of the timer is
0.2*T, even though we asked for an interval of T.
Of course, the _average_ duration will be around T, but the _minimum_
duration is essentially zero. POSIX clearly forbids a _one-shot_
timer, which is scheduled to expire after time T, to actually expire
after only time 0.2*T. But the language in POSIX is unclear to me on
whether this is allowed for _periodic_ timers:
https://pubs.opengroup.org/onlinepubs/9699919799/functions/timer_settime.html
The reload value of the timer shall be set to the value specified
by the it_interval member of value. When a timer is armed with a
non-zero it_interval, a periodic (or repetitive) timer is
specified.
Time values that are between two consecutive non-negative integer
multiples of the resolution of the specified timer shall be rounded
up to the larger multiple of the resolution. Quantization error
shall not cause the timer to expire earlier than the rounded time
value.
If the argument ovalue is not NULL, the timer_settime() function
shall store, in the location referenced by ovalue, a value
representing the previous amount of time before the timer would
have expired, or zero if the timer was disarmed, together with the
previous timer reload value. Timers shall not expire before their
scheduled time.
So on the one hand, I'm not sure it's correct to implement the timer
APIs (setitimer, timer_settime) with this patch.
On the other hand, if it's not correct to do that, I'm not sure
correct POSIX periodic timers can attain a target _average_ interval
between expirations -- only a target _minimum_, with an average that
is higher by necessity.
>From bca90989210e59c45efee9b44d7575e91f053a07 Mon Sep 17 00:00:00 2001
From: Taylor R Campbell
Date: Sat, 23 Dec 2023 21:03:13 +
Subject: [PATCH] time(9): Avoid adding partial tick to periodic interval
timers.
This way, configuring a periodic timer with period <=1/hz sec will
delay ~1/hz sec between consecutive firings.
This is different from one-shot sleeps of <=1/hz sec, which are still
scheduled two ticks away, giving a delay of [1/hz sec, 2/hz sec],
since there's no way to reliably sleep for at least a shorter
duration than 1/hz sec starting at an arbitrary time between ticks.
PR kern/43997
---
sys/kern/kern_time.c | 24 ++-
sys/kern/subr_time.c | 72 +---
sys/sys/timevar.h| 2 ++
3 files changed, 92 insertions(+), 6 deletions(-)
diff --git a/sys/kern/kern_time.c b/sys/kern/kern_time.c
index 7bc39324e32e..5130ddd390f5 100644
--- a/sys/kern/kern_time.c
+++ b/sys/kern/kern_time.c
@@ -824,6 +824,28 @@ itimer_arm_real(struct itimer * const it)
: tshzto(>it_time.it_value)));
}
+/*
+ * itimer_rearm_real:
+ *
+ * Rearm a non-virtual timer.
+ */
+static void
+itimer_rearm_real(struct itimer * const it)
+{
+ int value_ticks, interval_ticks;
+
+ KASSERT(!it->it_dying);
+ KASSERT(!CLOCK_VIRTUAL_P(it->it_clockid));
+ KASSERT(!callout_pending(>it_ch));
+
+ value_ticks = (it->it_clockid == CLOCK_MONOTONIC
+
Re: PSA: Clock drift and pkgin
>> Specifically, under a kernel built with HZ=100, requesting signals >> at 100Hz actually delivers them at 50Hz. [...] > This is the well-known problem that we don't have timers with > sub-tick resolution, PR kern/43997: https://gnats.netbsd.org/43997 It doesn't need sub-tick resolution; one-tick resolution would fix the problem. The problem appears to be that an ITIMER_REAL timer can't deliver signals more often than every second tick. > In particular, there is no way to reliably sleep for a duration below > 1/hz sec. Nor does there need to be to fix this. 1.4T/sparc and /i386 get it right, even when running with HZ=100 and requesting 100Hz SIGALRMs. (My timer sockets get it right too, but their codepath is completely different, depending on only timeout(...,0,1) (1.4T) or callout_schedule(...,1) (4.0.1 and 5.2). > Fixing this requires adopting an MI API and MD clock driver support > for wakeups with sub-tick resolution, You must be talking about something different from what I'm talking about. What I want fixed does not involve sub-tick-resolution timers at any level. If using setitimer(ITIMER_REAL,...) to request SIGALRMs every tick actually delivered a SIGALRM every tick, I'd be fine. But, instead, doing that delivers a SIGALRM every second tick. > which nobody's done yet -- Nobody's done the sub-tick resolution you're talking about, maybe. But 1.4T long ago did what I'm looking for. Something between 1.4T and 4.0.1 broke it, and it's stayed broken until at least 9.1, probably 9.3 based on someone else's report on port-vax. (Okay, strictly, I don't know that it's stayed broken. It could have been fixed and then re-broken.) >> } else if (sec <= (LONG_MAX / 100)) >> ticks = (((sec * 100) + (unsigned long)usec + (tick - 1)) >> / tick) + 1; > Whether this is a bug depends on whether: [...] I think that code is not a bug per se; for sleeps, that codepath is...well, "reasonable" at the very least. The bug is that it is broken for timer reloads, but timer reloads are using it anyway; whether you think of this as a bug in timer reloads or a bug in tvtohz is a question of which way you prefer to squint your mind. Always adding an extra tick may be fine for sleeps (though that's arguable for short sleeps on a system with a high-res wallclock), but not for timer reloads. /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: PSA: Clock drift and pkgin
On 2023-12-23 17:39, Taylor R Campbell wrote: Date: Fri, 22 Dec 2023 23:41:47 -0500 (EST) From: Mouse Specifically, under a kernel built with HZ=100, requesting signals at 100Hz actually delivers them at 50Hz. This is behind the clock running at half speed on my VAX emulator, and quite likely behind similar behaviour from simh (which emulates VAXen, among other things) on 9.3. I suspect it will happen on any port when requesting signals one timer tick apart (ie, at HZ Hz). This is the well-known problem that we don't have timers with sub-tick resolution, PR kern/43997: https://gnats.netbsd.org/43997 You are somewhat missing the point. What Mouse is asking for is not at all about sub-tick resolution. What he asks for is exactly tick resolution. And the important second half of this point is that with an interval timer, it automatically is reset every time it has fired, which means it happens exactly at the tick, and then you want a new trigger at the next tick. And currently, that is not possible. Because the reset of the interval timer rounds one tick up to two ticks (it basically always add one tick, no matter what interval you ask for, which implies that if you ask for 1s, it will actually be after 101 ticks (if we have a 10ms tick)). So nothing is sub-tick about this. Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: PSA: Clock drift and pkgin
> Date: Fri, 22 Dec 2023 23:41:47 -0500 (EST) > From: Mouse > > Specifically, under a kernel built with HZ=100, requesting signals at > 100Hz actually delivers them at 50Hz. This is behind the clock running > at half speed on my VAX emulator, and quite likely behind similar > behaviour from simh (which emulates VAXen, among other things) on 9.3. > I suspect it will happen on any port when requesting signals one timer > tick apart (ie, at HZ Hz). This is the well-known problem that we don't have timers with sub-tick resolution, PR kern/43997: https://gnats.netbsd.org/43997 In particular, there is no way to reliably sleep for a duration d below 1/hz sec. In principle with the current timer design it may be possible to sleep for shorter durations, but not to sleep for _at least_ any shorter duration (because the next tick wakeup might be arbitrarily soon); the shortest duration that a process can reliably sleep is 1/hz sec, but it may be up to 2/hz sec owing to scheduling delays. Fixing this requires adopting an MI API and MD clock driver support for wakeups with sub-tick resolution, which nobody's done yet -- it's not trivial and there's a lot of design choices to make. > I don't _know_ what's behind it. But today I went looking, and, in > 5.2, there is code which looks suspicious. I don't know where the > analogous code is in 9.x, but presumably plenty of people here do. > Speaking of 5.2, then: in kern/subr_time.c, there is tvtohz(), which > has code > > } else if (sec <= (LONG_MAX / 100)) > ticks = (((sec * 100) + (unsigned long)usec + (tick - 1)) > / tick) + 1; > > which looks suspicious. If sec is zero and usec is tick, that > expression will return 2 instead of the 1 I suspect it needs to return. Whether this is a bug depends on whether: (a) callout_schedule(ch, 1) triggers ch on the _next_ tick, meaning its delay may be arbitrarily small, or (b) callout_schedule(ch, 1) triggers ch on the next tick after that, meaning it may be delayed by [1/hz sec, 2/hz sec]. I don't know offhand which of these it is but it shouldn't be too hard to determine.
Re: PSA: Clock drift and pkgin
On 2023-12-23 16:53, Mouse wrote: [...], but we are in fact rounding it up to the double amount of time between alarms/interrupts. Not what I think anyone would have expected. Quite so. Whatever the internals behind it, the overall effect is "ask for 100Hz, get 50Hz", which - at least for me - violates POLA hard. It is, when the system clock is running at 100Hz. If the system clock is running at 100Hz, you could expect it should be possible to have an interval timer that runs at 100Hz. But it seems at the moment, we basically have that interval timers can run at most at HZ/2. Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: PSA: Clock drift and pkgin
> [...], but we are in fact rounding it up to the double amount of time > between alarms/interrupts. Not what I think anyone would have > expected. Quite so. Whatever the internals behind it, the overall effect is "ask for 100Hz, get 50Hz", which - at least for me - violates POLA hard. /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: PSA: Clock drift and pkgin
On 2023-12-23 14:35, Mouse wrote: } else if (sec <= (LONG_MAX / 100)) ticks = (((sec * 100) + (unsigned long)usec + (tick - 1)) / tick) + 1; The delay is always rounded up to the resolution of the clock, so waiting for 1 microsecond waits at least 10ms. But it is increased by 1 tick when it is an exact multiple of the clock resolution, too. For sleeps, that makes some sense. For timer reloads, it doesn't. I would probably agree that for timer reloads it should not do that rounding up when the interval is evenly divisible. It is a very different case than a sleep. For the reload we know that this happens at a specific time. I could of course be wrong about that code being responsible, but reading realtimerexpire() makes me think not; it uses tshzto, which calls tstohz, which calls tvtohz, which is where the code quoted above comes from. Maybe realtimerexpire should be using other code? Agreed. Two options are to increase HZ on the host as suggested, or halve HZ on the guest. I suppose actually fixing the bug isn't an option? I don't know whether that would mean using different code for timer reloads and sleeps or what. But 1.4T is demonstration-by-example that it is entirely possible to get this right, even in a tickful system. (I don't know whether 1.4T sleeps may be slightly too short; I haven't tested that. But, even if so, fixing that should not involve breaking timer reloads.) A tickless system do not fundamentally change anything either. You can't go below the resolution of a timer, and sleeps are supposed to be sleeping for *at least* the given time, but it could be more. But in this case, we end up where the expected, and reasonable behaviour would be to get alarms/interrupts at the specified frequency, because it is the resolution of the clock, but we are in fact rounding it up to the double amount of time between alarms/interrupts. Not what I think anyone would have expected. Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: b...@softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol
Re: PSA: Clock drift and pkgin
>>>} else if (sec <= (LONG_MAX / 100)) >>>ticks = (((sec * 100) + (unsigned long)usec + (tick - 1)) >>>/ tick) + 1; >> The delay is always rounded up to the resolution of the clock, so >> waiting for 1 microsecond waits at least 10ms. But it is increased by 1 tick when it is an exact multiple of the clock resolution, too. For sleeps, that makes some sense. For timer reloads, it doesn't. I could of course be wrong about that code being responsible, but reading realtimerexpire() makes me think not; it uses tshzto, which calls tstohz, which calls tvtohz, which is where the code quoted above comes from. Maybe realtimerexpire should be using other code? > Look at the wording sleep(3), nanosleep(2), etc. They all use > wording like "... the number of time units have elapsed ..." True. And, if the misbehaviour involved sleep, nanosleep, etc, that would be relevant. The symptom I'm seeing has nothing to do with them (except that both are related to time); what I'm talking about is the timing of SIGALRMs generated by setitimer(ITIMER_REAL,...) when it_interval is set to 1/HZ (which in my test cases is exact). setitimer(2) does say that "[t]ime values smaller than the resolution of the system clock are rounded up to this resolution (typically 10 milliseconds)", but it does _not_ have language similar to what you quote for sleep() and relatives. Nor, IMO, should it. The signals should be delivered on schedule, though of course process scheduling means the target process may not run the handler on schedule. Under interrupt load sufficient that softclock isn't running when it should, I'd consider this excusable. That does not describe my test systems. 1.4T does not have this bug. As I mentioned, it works fine on sparc. Even on i386, I see: $ date; test-alrm > test-alrm.out; date Sat Dec 23 07:57:45 EST 2023 Sat Dec 23 07:58:46 EST 2023 $ sed -n -e 1p -e \$p < test-alrm.out 1703336265.921251 1703336325.916413 $ Linux, at least on x86_64, gets this right too. On a work machine: $ date; ./test-alrm > test-alrm.out; date Sat Dec 23 08:18:15 EST 2023 Sat Dec 23 08:19:15 EST 2023 $ sed -n -e 1p -e \$p < test-alrm.out 1703337495.219734 1703337555.209737 $ uname -a Linux mouchine 5.15.0-86-generic #96-Ubuntu SMP Wed Sep 20 08:23:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux $ > Two options are to increase HZ on the host as suggested, or halve HZ > on the guest. I suppose actually fixing the bug isn't an option? I don't know whether that would mean using different code for timer reloads and sleeps or what. But 1.4T is demonstration-by-example that it is entirely possible to get this right, even in a tickful system. (I don't know whether 1.4T sleeps may be slightly too short; I haven't tested that. But, even if so, fixing that should not involve breaking timer reloads.) /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: PSA: Clock drift and pkgin
Michael van Elst wrote: > mo...@rodents-montreal.org (Mouse) writes: > > >} else if (sec <= (LONG_MAX / 100)) > >ticks = (((sec * 100) + (unsigned long)usec + (tick - 1)) > >/ tick) + 1; > > >which looks suspicious. If sec is zero and usec is tick, that > >expression will return 2 instead of the 1 I suspect it needs to return. > > The delay is always rounded up to the resolution of the clock, > so waiting for 1 microsecond waits at least 10ms. > > The interval to the next tick can be arbitrarily short. Waiting > for at least 10ms therefore means to wait for the second next > tick. Look at the wording sleep(3), nanosleep(2), etc. They all use wording like "... the number of time units have elapsed ..." and none of them make any guarantee that they'll return within any specific time frame after the time period has elapsed. In the same thread on port-vax Mouse references, Matt previously said: > unfortunately, we have a known issue hosting VMs on machines > with the same or near HZ values. i've built my VM hosting > systems with HZ=256 for the last year or so and it helps a lot. What Mouse is describing the Michael expanded on is is exactly the issue that Matt is referring to. Two options are to increase HZ on the host as suggested, or halve HZ on the guest. > In a tickless system, such a problem doesn't exist. Well, this is a third option. A bit harder though :) Cheers, Simon.
Re: PSA: Clock drift and pkgin
mo...@rodents-montreal.org (Mouse) writes: >} else if (sec <= (LONG_MAX / 100)) >ticks = (((sec * 100) + (unsigned long)usec + (tick - 1)) >/ tick) + 1; >which looks suspicious. If sec is zero and usec is tick, that >expression will return 2 instead of the 1 I suspect it needs to return. The delay is always rounded up to the resolution of the clock, so waiting for 1 microsecond waits at least 10ms. The interval to the next tick can be arbitrarily short. Waiting for at least 10ms therefore means to wait for the second next tick. In a tickless system, such a problem doesn't exist.
Re: PSA: Clock drift and pkgin
In a discussion of timekeeping on emulated VAXen, over on port-vax@, I
mentioned that I've found that, on 4.0.1, 5.2, and 9.1, and, based on a
report on port-vax@, apparently 9.3 as well, there's a bug in
ITIMER_REAL signals (possibly on only some hardware - I've seen it on
amd64 and i386, and, if my guess below is correct, it should manifest
on various others as well).
Specifically, under a kernel built with HZ=100, requesting signals at
100Hz actually delivers them at 50Hz. This is behind the clock running
at half speed on my VAX emulator, and quite likely behind similar
behaviour from simh (which emulates VAXen, among other things) on 9.3.
I suspect it will happen on any port when requesting signals one timer
tick apart (ie, at HZ Hz).
In case anyone wants it, I wrote a small test program. It requests
100Hz signals, then, in the signal handler, takes a gettimeofday()
timestamp. After taking 6000 timestamps (which ideally should take
60.00 seconds), it then prints out all the timestamps, thus indicating
the actual rate signals were delivered at. It's on
ftp.rodents-montreal.org (which supports HTTP fetches as well as FTP)
in /mouse/misc/test-alrm.c for anyone interested. On machines with the
half-speed bug, it takes two minutes rather that one, and the
timestamps average about 20ms apart, instead of 10ms. ("About" because
in most of my tests there is usually at least one interval that is
slightly longer than it should be.)
I don't _know_ what's behind it. But today I went looking, and, in
5.2, there is code which looks suspicious. I don't know where the
analogous code is in 9.x, but presumably plenty of people here do.
Speaking of 5.2, then: in kern/subr_time.c, there is tvtohz(), which
has code
} else if (sec <= (LONG_MAX / 100))
ticks = (((sec * 100) + (unsigned long)usec + (tick - 1))
/ tick) + 1;
which looks suspicious. If sec is zero and usec is tick, that
expression will return 2 instead of the 1 I suspect it needs to return.
I haven't yet actually tried changing that. Holiday preparations and
observations are likely to occupy much of my time for the next week or
so, but I'll try to fit in the time to change that and see if it helps
any.
/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTMLmo...@rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
