Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
In any case, NTP is not the only thing that adjusts the clock, e.g. the operating system will adjust the time for daylight savings. Daylight savings time is not a clock adjustment, at least not in the sense this thread has mostly been talking about the word clock. It doesn't affect the seconds from epoch measurement, it affects the way in which the clock is formatted to the user. Ah yes, you're right. The system clock uses the UTC time zone on Linux and Windows, and it is not affected by DST. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On 4/7/2012 5:49 AM, Victor Stinner wrote: 2012/4/7 Janzertjanz...@janzert.com: On 4/5/2012 6:32 AM, Victor Stinner wrote: I prefer to use CLOCK_MONOTONIC, not because it is also available for older Linux kernels, but because it is more reliable. Even if the underlying clock source is unstable (unstable frequency), a delta of two reads of the CLOCK_MONOTONIC clock is a result in *seconds*, whereas CLOCK_MONOTONIC_RAW may use an unit a little bit bigger or smaller than a second. time.monotonic() unit is the second, as written in its documentation. I believe the above is only true for sufficiently large time deltas. One of the major purposes of NTP slewing is to give up some short term accuracy in order to achieve long term accuracy (e.g. whenever the clock is found to be ahead of real time it is purposefully ticked slower than real time). I don't think that NTP works like that. NTP only uses very smooth adjustements: slewing: change the clock frequency to be slightly faster or slower (which is done with adjtime()). Since the slew rate is limited to 0.5 ms/s, each second of adjustment requires an amortization interval of 2000 s. Thus, an adjustment of many seconds can take hours or days to amortize. http://www.python.org/dev/peps/pep-0418/#ntp-adjustment Right, the description in that paragraph is exactly what I was referring to above. :) It is unfortunate that a clock with a resolution of 1ns may be purposefully thrown off by 500,000ns per second in the short term. In practice you are probably correct that it is better to take the slewed clock even though it may have purposeful short term inaccuracy thrown in than it is to use the completely unadjusted one. So for benchmarking it would not be surprising to be better off with the non-adjusted clock. Ideally there would be a clock that was slewed just enough to try and achieve short term accuracy, but I don't know of anything providing that. time.monotonic() is not written for benchmarks. It does not have the highest frequecency, it's primary property is that is monotonic. A side effect is that it is usually the steadiest clock. For example, on Windows time.monotonic() has only an accuracy of 15 ms (15 milliseconds not 15 microseconds). Hmm, I just realized an unfortunate result of that. Since it means time.monotonic() will be too coarse to be useful for frame rate level timing on windows. Janzert ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On Sat, Apr 7, 2012 at 4:56 PM, Glyph Lefkowitz gl...@twistedmatrix.comwrote: On Apr 7, 2012, at 3:40 AM, Steven D'Aprano wrote: In any case, NTP is not the only thing that adjusts the clock, e.g. the operating system will adjust the time for daylight savings. Daylight savings time is not a clock adjustment, at least not in the sense this thread has mostly been talking about the word clock. It doesn't affect the seconds from epoch measurement, it affects the way in which the clock is formatted to the user. -glyph even on windows where the system hardware clock is maintained in local time? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
Is it still? I thought they fixed that ages ago? On Mon, Apr 9, 2012 at 4:42 PM, Gregory P. Smith g...@krypto.org wrote: On Sat, Apr 7, 2012 at 4:56 PM, Glyph Lefkowitz gl...@twistedmatrix.com wrote: On Apr 7, 2012, at 3:40 AM, Steven D'Aprano wrote: In any case, NTP is not the only thing that adjusts the clock, e.g. the operating system will adjust the time for daylight savings. Daylight savings time is not a clock adjustment, at least not in the sense this thread has mostly been talking about the word clock. It doesn't affect the seconds from epoch measurement, it affects the way in which the clock is formatted to the user. -glyph even on windows where the system hardware clock is maintained in local time? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On Mon, Apr 9, 2012 at 4:46 PM, Guido van Rossum gu...@python.org wrote: Is it still? I thought they fixed that ages ago? sadly, no. http://www.cl.cam.ac.uk/~mgk25/mswish/ut-rtc.html On Mon, Apr 9, 2012 at 4:42 PM, Gregory P. Smith g...@krypto.org wrote: On Sat, Apr 7, 2012 at 4:56 PM, Glyph Lefkowitz gl...@twistedmatrix.com wrote: On Apr 7, 2012, at 3:40 AM, Steven D'Aprano wrote: In any case, NTP is not the only thing that adjusts the clock, e.g. the operating system will adjust the time for daylight savings. Daylight savings time is not a clock adjustment, at least not in the sense this thread has mostly been talking about the word clock. It doesn't affect the seconds from epoch measurement, it affects the way in which the clock is formatted to the user. -glyph even on windows where the system hardware clock is maintained in local time? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
2012/4/7 Janzert janz...@janzert.com: On 4/5/2012 6:32 AM, Victor Stinner wrote: I prefer to use CLOCK_MONOTONIC, not because it is also available for older Linux kernels, but because it is more reliable. Even if the underlying clock source is unstable (unstable frequency), a delta of two reads of the CLOCK_MONOTONIC clock is a result in *seconds*, whereas CLOCK_MONOTONIC_RAW may use an unit a little bit bigger or smaller than a second. time.monotonic() unit is the second, as written in its documentation. I believe the above is only true for sufficiently large time deltas. One of the major purposes of NTP slewing is to give up some short term accuracy in order to achieve long term accuracy (e.g. whenever the clock is found to be ahead of real time it is purposefully ticked slower than real time). I don't think that NTP works like that. NTP only uses very smooth adjustements: slewing: change the clock frequency to be slightly faster or slower (which is done with adjtime()). Since the slew rate is limited to 0.5 ms/s, each second of adjustment requires an amortization interval of 2000 s. Thus, an adjustment of many seconds can take hours or days to amortize. http://www.python.org/dev/peps/pep-0418/#ntp-adjustment So for benchmarking it would not be surprising to be better off with the non-adjusted clock. Ideally there would be a clock that was slewed just enough to try and achieve short term accuracy, but I don't know of anything providing that. time.monotonic() is not written for benchmarks. It does not have the highest frequecency, it's primary property is that is monotonic. A side effect is that it is usually the steadiest clock. For example, on Windows time.monotonic() has only an accuracy of 15 ms (15 milliseconds not 15 microseconds). If you consider that the PEP should also solve the issue of benchmarking clock, we should continue the work on this section: http://www.python.org/dev/peps/pep-0418/#deferred-api-time-perf-counter Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
Victor Stinner wrote: 2012/4/7 Janzert janz...@janzert.com: On 4/5/2012 6:32 AM, Victor Stinner wrote: I prefer to use CLOCK_MONOTONIC, not because it is also available for older Linux kernels, but because it is more reliable. Even if the underlying clock source is unstable (unstable frequency), a delta of two reads of the CLOCK_MONOTONIC clock is a result in *seconds*, whereas CLOCK_MONOTONIC_RAW may use an unit a little bit bigger or smaller than a second. time.monotonic() unit is the second, as written in its documentation. I believe the above is only true for sufficiently large time deltas. One of the major purposes of NTP slewing is to give up some short term accuracy in order to achieve long term accuracy (e.g. whenever the clock is found to be ahead of real time it is purposefully ticked slower than real time). I don't think that NTP works like that. NTP only uses very smooth adjustements: slewing: change the clock frequency to be slightly faster or slower (which is done with adjtime()). Since the slew rate is limited to 0.5 ms/s, each second of adjustment requires an amortization interval of 2000 s. Thus, an adjustment of many seconds can take hours or days to amortize. http://www.python.org/dev/peps/pep-0418/#ntp-adjustment That is incorrect. NTP by default will only slew the clock for small discrepancies. For large discrepancies, it will step the clock, causing the time to jump. By default, large here means more than 128 milliseconds. Yes, milliseconds. http://www.ntp.org/ntpfaq/NTP-s-config-tricks.htm#AEN4249 In any case, NTP is not the only thing that adjusts the clock, e.g. the operating system will adjust the time for daylight savings. -- Steven ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
2012/4/7 Steven D'Aprano st...@pearwood.info: Victor Stinner wrote: 2012/4/7 Janzert janz...@janzert.com: On 4/5/2012 6:32 AM, Victor Stinner wrote: I prefer to use CLOCK_MONOTONIC, not because it is also available for older Linux kernels, but because it is more reliable. Even if the underlying clock source is unstable (unstable frequency), a delta of two reads of the CLOCK_MONOTONIC clock is a result in *seconds*, whereas CLOCK_MONOTONIC_RAW may use an unit a little bit bigger or smaller than a second. time.monotonic() unit is the second, as written in its documentation. I believe the above is only true for sufficiently large time deltas. One of the major purposes of NTP slewing is to give up some short term accuracy in order to achieve long term accuracy (e.g. whenever the clock is found to be ahead of real time it is purposefully ticked slower than real time). I don't think that NTP works like that. NTP only uses very smooth adjustements: slewing: change the clock frequency to be slightly faster or slower (which is done with adjtime()). Since the slew rate is limited to 0.5 ms/s, each second of adjustment requires an amortization interval of 2000 s. Thus, an adjustment of many seconds can take hours or days to amortize. http://www.python.org/dev/peps/pep-0418/#ntp-adjustment That is incorrect. NTP by default will only slew the clock for small discrepancies. For large discrepancies, it will step the clock, causing the time to jump. By default, large here means more than 128 milliseconds. Yes, milliseconds. http://www.ntp.org/ntpfaq/NTP-s-config-tricks.htm#AEN4249 We are talking about CLOCK_MONOTONIC. Steping is disabled on this clock. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On 07Apr2012 20:40, Steven D'Aprano st...@pearwood.info wrote: | Victor Stinner wrote: | I don't think that NTP works like that. NTP only uses very smooth adjustements: [...] | http://www.python.org/dev/peps/pep-0418/#ntp-adjustment | | That is incorrect. NTP by default will only slew the clock for small | discrepancies. For large discrepancies, it will step the clock, causing the | time to jump. By default, large here means more than 128 milliseconds. | Yes, milliseconds. | http://www.ntp.org/ntpfaq/NTP-s-config-tricks.htm#AEN4249 | | In any case, NTP is not the only thing that adjusts the clock, e.g. the | operating system will adjust the time for daylight savings. Ignoring the discussion of NTP, daylight saving is a _presentation_ issue. It is _display_. The OS clock does not change for daylight saving! Think: seconds since the epoch. This is a continuous function. Daylight saving presentation occurs turning a seconds-since-epoch into a human decomposed time (hours, etc). Now, AFAIR, Windows used to run its system clock in local time i.e. it did have to jump its clock for daylight saving. Hopefully that is long gone. UNIX never did this. It ran in seconds since the epoch (in its case, start of 01jan1970 GMT). Printing dates and timestamps to humans needed daylight saving knowledge etc. Cheers, -- Cameron Simpson c...@zip.com.au DoD#743 http://www.cskk.ezoshosting.com/cs/ Stan Malyshev s...@netcom.com wrote: | You're dragging a peg in a blind hairpin when you see a patch of coolant | up ahead, and you hear the airbrakes of an oncoming 18-wheeler. | What do you do? WHAT DO YOU DO? (Speed, anyone?) Shoot my pillion? - Vociferous Mole stev...@starbase.neosoft.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On Apr 7, 2012, at 3:40 AM, Steven D'Aprano wrote: In any case, NTP is not the only thing that adjusts the clock, e.g. the operating system will adjust the time for daylight savings. Daylight savings time is not a clock adjustment, at least not in the sense this thread has mostly been talking about the word clock. It doesn't affect the seconds from epoch measurement, it affects the way in which the clock is formatted to the user. -glyph___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
Cameron Simpson wrote: I maintain that monotonic still means what I said, and that it is the combination of the word with clock that brings in your other criteria. I'm not trying to redefine the word monotonic in general. All I'm saying is that *if* the PEP is going to talk about a monotonic clock or monotonic time, it should clearly define what this means, because it's not obvious to everyone that it implies something more than the mathematical meaning. Alternatively, don't use the word monotonic at all, and find a better term. CLOCK_MONOTONIC Clock that cannot be set and represents monotonic time since some unspecified starting point. Which doesn't help very much, because it talks about monotonic time without saying what that means. Googling for that phrase doesn't seem turn up anything very useful. Apparently we're supposed to just know. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
Steven D'Aprano wrote: Greg Ewing wrote: the important thing about a clock that it *keeps going forward* That would be a *strictly* monotonic clock, as opposed to merely monotonic. Well, yes, but even that's not enough -- it needs to go forward at a reasonably constant rate, otherwise it's just as useless. If it had enough resolution, it could go forward by one femtosecond every hour for a while and still call itself strictly monotonic... -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On 6 April 2012 02:50, Cameron Simpson c...@zip.com.au wrote: (Quoted from the Linux manpage) All implementations support the system-wide real-time clock, which is identified by CLOCK_REALTIME. Its time represents seconds and nanoseconds since the Epoch. When its time is changed, timers for a relative interval are unaffected, but timers for an absolute point in time are affected. This made me think. They make a distinction between timers for a relative interval and timers for an absolute point in time. But that's not right - *all* of the clock calls we are talking about here return a *single* number. Interpreting that as an absolute time needs an epoch. Or to put it another way, clock values are always meaningless without context - whereas clock *differences* are what actually carry meaning (in terms of a duration). On that basis, I'd say that - A clock that doesn't get adjusted or slewed is a tick counter (which technically doesn't have any relationship to time, although the tick frequency can be used to convert to seconds, but see the next entry) - A clock that gets slewed but not adjusted is a seconds counter (or at least, the nearest approximation the system can provide - presumably better than a tick counter) - A clock that gets adjusted is not an interval timer at all, but an absolute timer (which therefore shouldn't really be used for benchmarking or timeouts) It seems to me that what *I* would most often need are the second two of these (to at least as high a precision as my app needs, which may vary but to the highest precision possible would do :-)) I'd be happy for a seconds counter to fallback to a tick counter converted to seconds using its frequency - slewing is simply an accuracy improvement process, as far as I can see. It seems to me that the current time.time() and time.wallclock() are the right names for my absolute timer and seconds timer above. Whether their implementations match my definitions I'm not sure, but that's what I'd hope. One thing I would expect is that time.wallclock() would never go backwards (so differences are always positive). The various other debates about monotonic, steady, etc, seem to me to be only relevant for specialist uses that I don't care about. As regards suspension, if I'm timing intervals and the system suspends, I'd be happy to say all bets are off. Similarly with timeouts. If I cared, I'd simply make sure the system didn't suspend :-) As far as comparability between different threads or processes are concerned, I would expect absolute time (time.time) to be the same across threads or processes (but wouldn't generally write apps that were affected if it weren't - at least by small amounts), but I wouldn't expect time.wallclock values obtained in different threads or processes to be comparable (mostly because I can't think of a case where I'd compare them). Where VMs or multiple machines are involved, I wouldn't even expect absolute time to match (but that's the job of NTP, and if time.time follows NTP, there's no reason why there would be an issue even there). Summary: I'm happy with time.time and time.wallclock. The rest of this debate doesn't matter for my usecases (and I suspect many other people's in practice). [Update, after I downloaded and installed 3.3a2] Bah, looks like time.wallclock is gone. (Actually, looks like it was documented but not implemented in 3.3a1!). Actually, the docs and the implementation don't match - clock_gettime is documented as available, but it's not (at least on Windows). I still prefer time.wallclock() as described above and in the 3.3a1 documentation. I thought I knew what was going on, but now I'm confused. My comments above still stand, though. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On Thu, Apr 5, 2012 at 12:32, Victor Stinner victor.stin...@gmail.com wrote: I prefer to use CLOCK_MONOTONIC, not because it is also available for older Linux kernels, but because it is more reliable. Even if the underlying clock source is unstable (unstable frequency), a delta of two reads of the CLOCK_MONOTONIC clock is a result in *seconds*, whereas CLOCK_MONOTONIC_RAW may use an unit a little bit bigger or smaller than a second. Aha. OK, CLOCK_MONOTONIC it is then. //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On 4 April 2012 10:33, Steven D'Aprano st...@pearwood.info wrote: try: from os import bestclock as _bestclock except ImportError: _bestclock = time My problem here is that best clock means different things to different people (as the number of emails shows). I think exposing specific clocks is also useful (sometimes people may need a steady clock, and early failure is better than clock skew). However, I propose a loosely classified set of clocks built on top of the specific clocks, all of which can fall back to the lowest precision/non-monotonic clock if needed. 1. The steadiest clock on the system. Ideally this would be a steady clock, but may not be. 2. The most precise clock on the system. This would have the finest-grain tick available on the system. 3. The highest performance (or maybe lowest latency) clock. This would be whichever clock on the system returned its results fastest. I'm not sure if there are more that would be needed (most accurate comes to mind, but feels like it's arbitrarily choosing between steadiest and most precise, so I don't think it's valid). By choosing relative terms, it caters to people's desire to have the best clock, but doesn't set the expectation that the behaviour is guaranteed. Tim Delaney ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
1. The steadiest clock on the system. Ideally this would be a steady clock, but may not be. time.monotonic() as proposed in the PEP 418 *is* the steadiest available clock, but we cannot say that it is steady :-) 2. The most precise clock on the system. This would have the finest-grain tick available on the system. It's discussed in the Deferred API: time.perf_counter() section. It would be nice to provide such clock, but I don't feel able right now to propose ab API for such requirement. It's unclear to me if it must be monotonic, steady, count elapsed time during a sleep or not, etc. It is already very hard to propose one single time function (time.monotonic), so I chose to simplify the PEP and not propose two functions but only one :-) 3. The highest performance (or maybe lowest latency) clock. This would be whichever clock on the system returned its results fastest. Linux provides CLOCK_REALTIME_COARSE and CLOCK_MONOTONIC_COARSE clocks, reading the ACPI Power Management clock is known to be slow. But should the clock be monotonic or not? Return seconds or CPU ticks? If the clock is not well defined, it's useless or at least, not portable. Exposing CLOCK_REALTIME_COARSE and CLOCK_MONOTONIC_COARSE constants should be enough. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On 4/5/2012 6:32 AM, Victor Stinner wrote: Since the only monotonic clock that can be adjusted by NTP is Linux' CLOCK_MONOTONIC, if we avoid it, then time.monotonic() would always give a clock that isn't adjusted by NTP. I thought we decided that NTP adjustment isn't an issue, because it's always gradual. Well, in timings it is an issue, but perhaps not a big one. :-) In any case, which one we use will not change the API, so if it is decided it is an issue, we can always more to CLOCK_MONOTONIC_RAW in the future, once Linux 2.6.26 (or whatever it was) is deemed unsupported. I prefer to use CLOCK_MONOTONIC, not because it is also available for older Linux kernels, but because it is more reliable. Even if the underlying clock source is unstable (unstable frequency), a delta of two reads of the CLOCK_MONOTONIC clock is a result in *seconds*, whereas CLOCK_MONOTONIC_RAW may use an unit a little bit bigger or smaller than a second. time.monotonic() unit is the second, as written in its documentation. I believe the above is only true for sufficiently large time deltas. One of the major purposes of NTP slewing is to give up some short term accuracy in order to achieve long term accuracy (e.g. whenever the clock is found to be ahead of real time it is purposefully ticked slower than real time). So for benchmarking it would not be surprising to be better off with the non-adjusted clock. Ideally there would be a clock that was slewed just enough to try and achieve short term accuracy, but I don't know of anything providing that. Janzert ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On Thu, Apr 5, 2012 at 00:45, Greg Ewing greg.ew...@canterbury.ac.nz wrote: Lennart Regebro wrote: Since the only monotonic clock that can be adjusted by NTP is Linux' CLOCK_MONOTONIC, if we avoid it, then time.monotonic() would always give a clock that isn't adjusted by NTP. I thought we decided that NTP adjustment isn't an issue, because it's always gradual. Well, in timings it is an issue, but perhaps not a big one. :-) In any case, which one we use will not change the API, so if it is decided it is an issue, we can always more to CLOCK_MONOTONIC_RAW in the future, once Linux 2.6.26 (or whatever it was) is deemed unsupported. //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
Since the only monotonic clock that can be adjusted by NTP is Linux' CLOCK_MONOTONIC, if we avoid it, then time.monotonic() would always give a clock that isn't adjusted by NTP. I thought we decided that NTP adjustment isn't an issue, because it's always gradual. Well, in timings it is an issue, but perhaps not a big one. :-) In any case, which one we use will not change the API, so if it is decided it is an issue, we can always more to CLOCK_MONOTONIC_RAW in the future, once Linux 2.6.26 (or whatever it was) is deemed unsupported. I prefer to use CLOCK_MONOTONIC, not because it is also available for older Linux kernels, but because it is more reliable. Even if the underlying clock source is unstable (unstable frequency), a delta of two reads of the CLOCK_MONOTONIC clock is a result in *seconds*, whereas CLOCK_MONOTONIC_RAW may use an unit a little bit bigger or smaller than a second. time.monotonic() unit is the second, as written in its documentation. Linux is the OS providing the most reliable monotonic clock, why should you use a not reliable monotonic clock instead? NTP doesn't step CLOCK_MONOTONIC, it only slew it. http://www.python.org/dev/peps/pep-0418/#ntp-adjustment Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
2012/4/5 PJ Eby p...@telecommunity.com: More details why it's hard to define such function and why I dropped it from the PEP. If someone wants to propose again such function (monotonic or fallback to system clock), two issues should be solved: - name of the function - description of the function Maybe I missed it, but did anyone ever give a reason why the fallback couldn't be to Steven D'Aprano's monotonic wrapper algorithm over the system clock? (Given a suitable minimum delta.) That function appeared to me to provide a sufficiently monotonic clock for timeout purposes, if nothing else. Did you read the following section of the PEP? http://www.python.org/dev/peps/pep-0418/#working-around-operating-system-bugs Did I miss something? If yes, could you write a patch for the PEP please? Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
Cameron Simpson wrote: On 04Apr2012 22:23, PJ Eby p...@telecommunity.com wrote: | On Apr 4, 2012 7:28 PM, Victor Stinner victor.stin...@gmail.com wrote: | More details why it's hard to define such function and why I dropped | it from the PEP. | | If someone wants to propose again such function (monotonic or | fallback to system clock), two issues should be solved: | | - name of the function | - description of the function | | Maybe I missed it, but did anyone ever give a reason why the fallback | couldn't be to Steven D'Aprano's monotonic wrapper algorithm over the | system clock? (Given a suitable minimum delta.) That function appeared to | me to provide a sufficiently monotonic clock for timeout purposes, if | nothing else. It was pointed out (by Nick Coglan I think?) that if the system clock stepped backwards then a timeout would be extended by at least that long. For example, code that waited (by polling the synthetic clock) for 1s could easily wait an hour if the system clock stepped back that far. Probaby undesirable. Steven D'Aprano's synthetic clock is able to partially avoid that situation -- worst case is a timeout of double what you asked for -- so 10 seconds instead of 5 (which is much better than 3600!). ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On Thu, 05 Apr 2012 08:32:22 -0700 Ethan Furman et...@stoneleaf.us wrote: Steven D'Aprano's synthetic clock is able to partially avoid that situation -- worst case is a timeout of double what you asked for -- so 10 seconds instead of 5 (which is much better than 3600!). The remaining issue is that the clock is not system-wide, it's interpreter-specific. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On Wed, Apr 4, 2012 at 11:41 PM, Cameron Simpson c...@zip.com.au wrote: On 04Apr2012 22:23, PJ Eby p...@telecommunity.com wrote: | On Apr 4, 2012 7:28 PM, Victor Stinner victor.stin...@gmail.com wrote: | More details why it's hard to define such function and why I dropped | it from the PEP. | | If someone wants to propose again such function (monotonic or | fallback to system clock), two issues should be solved: | | - name of the function | - description of the function | | Maybe I missed it, but did anyone ever give a reason why the fallback | couldn't be to Steven D'Aprano's monotonic wrapper algorithm over the | system clock? (Given a suitable minimum delta.) That function appeared to | me to provide a sufficiently monotonic clock for timeout purposes, if | nothing else. It was pointed out (by Nick Coglan I think?) that if the system clock stepped backwards then a timeout would be extended by at least that long. For example, code that waited (by polling the synthetic clock) for 1s could easily wait an hour if the system clock stepped back that far. Probaby undesirable. Steven D'Aprano's algorithm doesn't do that. If the system clock steps backwards, it still stepped forward by a specified minimum delta. The amount of time that a timeout was extended would be a function of the polling frequency, not the presence of absence of backward steps in the underlying clock. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On Thu, Apr 5, 2012 at 6:34 AM, Victor Stinner victor.stin...@gmail.comwrote: 2012/4/5 PJ Eby p...@telecommunity.com: More details why it's hard to define such function and why I dropped it from the PEP. If someone wants to propose again such function (monotonic or fallback to system clock), two issues should be solved: - name of the function - description of the function Maybe I missed it, but did anyone ever give a reason why the fallback couldn't be to Steven D'Aprano's monotonic wrapper algorithm over the system clock? (Given a suitable minimum delta.) That function appeared to me to provide a sufficiently monotonic clock for timeout purposes, if nothing else. Did you read the following section of the PEP? http://www.python.org/dev/peps/pep-0418/#working-around-operating-system-bugs Did I miss something? If yes, could you write a patch for the PEP please? What's missing is that if you're using a monotonic clock for timeouts, then a monotonically-adjusted system clock can do that, subject to the polling frequency -- it does not break just because the system clock is set backwards; it simply loses time proportional to the frequency with which it is polled. For timeout purposes in a single process, such a clock is useful. It just isn't suitable for benchmarks, or for interprocess coordination. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On Thu, Apr 5, 2012 at 9:48 AM, PJ Eby p...@telecommunity.com wrote: On Thu, Apr 5, 2012 at 6:34 AM, Victor Stinner victor.stin...@gmail.com wrote: 2012/4/5 PJ Eby p...@telecommunity.com: More details why it's hard to define such function and why I dropped it from the PEP. If someone wants to propose again such function (monotonic or fallback to system clock), two issues should be solved: - name of the function - description of the function Maybe I missed it, but did anyone ever give a reason why the fallback couldn't be to Steven D'Aprano's monotonic wrapper algorithm over the system clock? (Given a suitable minimum delta.) That function appeared to me to provide a sufficiently monotonic clock for timeout purposes, if nothing else. Did you read the following section of the PEP? http://www.python.org/dev/peps/pep-0418/#working-around-operating-system-bugs Did I miss something? If yes, could you write a patch for the PEP please? What's missing is that if you're using a monotonic clock for timeouts, then a monotonically-adjusted system clock can do that, subject to the polling frequency -- it does not break just because the system clock is set backwards; it simply loses time proportional to the frequency with which it is polled. Depending on the polling frequency sounds like a bad idea, since you can't know that you're the only user of the clock. Also depending on the use case, too short a timeout may be worse than too long a timeout. E.g. imagine hitting a website that usually takes 2 seconds to respond, and setting a timeout to e.g. 4 seconds to bail. If the timeout somehow gets reduced to 1 second it will appear as if the website died, whereas if the timeout was increased to 1 hour, nothing bad would happen unless the website *actually* started having truly bad response times. For timeout purposes in a single process, such a clock is useful. It just isn't suitable for benchmarks, or for interprocess coordination. I think it would be better if the proposed algorithm (or whatever algorithm to fix timeouts) was implemented by the application/library code using the timeout (or provided as a separate library function), rather than by the clock, since the clock can't know what fallback behavior the app/lib needs. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On Thu, 5 Apr 2012 09:56:19 -0700 Guido van Rossum gu...@python.org wrote: For timeout purposes in a single process, such a clock is useful. It just isn't suitable for benchmarks, or for interprocess coordination. I think it would be better if the proposed algorithm (or whatever algorithm to fix timeouts) was implemented by the application/library code using the timeout (or provided as a separate library function), rather than by the clock, since the clock can't know what fallback behavior the app/lib needs. Agreed with providing it as a separate library function. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
For timeout purposes in a single process, such a clock is useful. It just isn't suitable for benchmarks, or for interprocess coordination. I think it would be better if the proposed algorithm (or whatever algorithm to fix timeouts) was implemented by the application/library code using the timeout (or provided as a separate library function), rather than by the clock, since the clock can't know what fallback behavior the app/lib needs. Agreed with providing it as a separate library function. I changed time.monotonic() to not fallback to the system clock exactly for this reason: Python cannot guess what the developer expects, or how the developer will use the clock. Instead of implementing your own clock in your application, it's maybe easier to patch your OS? I suppose that you are running on GNU/Hurd, because I didn't find yet other OS not providing a monotonic clock :-) If you are using an OS that doesn't provide a monotonic clock, do you really need to implement your own in your application? Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
Victor Stinner wrote: I changed time.monotonic() to not fallback to the system clock exactly for this reason: Python cannot guess what the developer expects, or how the developer will use the clock. Which is exactly why I like Cameron Simpson's approach to selecting a clock -- let the developer/user decide what kind of clock they need, and ask for one that matches their criteria. ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On Thu, Apr 5, 2012 at 12:56 PM, Guido van Rossum gu...@python.org wrote: Depending on the polling frequency sounds like a bad idea, since you can't know that you're the only user of the clock. Also depending on the use case, too short a timeout may be worse than too long a timeout. Given a small enough delta, the timeout won't be too short. (Steven's original code sample I believed used either 0 or 1 as a delta, but it could be as small a fraction as will add correctly in the datatype used.) And the worst case polling frequency is the length of the timeout, meaning you can't end up with more than double your intended timeout. In the opposite scenario, where the time is polled in a tight loop, then as long as Python doesn't sample the raw clock so often that the summed fractional deltas exceeds the real clock speed, the timeout won't be shortened by any appreciable amount. In fact, this can be guaranteed by measuring time as a (raw, increment) tuple, where the increment can be an arbitrarily-large integer. Each new time value is greater than the one before, yet the real component remains untouched. With this approach, the timeout can only be delayed for however long the system clock *stops*, and the timeout can only be shortened by the system clock skipping ahead. Okay, having thought that out, I now agree that there are too many fine points to make this cover enough of the use cases without needing parameters. Or more to the point, If the implementation is hard to explain, it's a bad idea. ;-) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
Folks: Good job, Victor Stinner on baking the accumulated knowledge of this thread into PEP 418. Even though I'm very interested in the topic, I haven't been able to digest the whole thread(s) on the list and understand what the current collective understanding is. The detailed PEP document helps a lot. I think there are still some mistakes, either in our collective understanding as reflected by the PEP, or in my own head. For starters, I still don't understand the first, most basic thing: what do people mean when they say monotonic clock? I don't understand the current text of PEP 418 with regard to the definition of that word. Allow me to resort to an analogy. There is an infinitely long, perfectly straight and flat racetrack. There is a flag that gets dragged along it at a constant rate, with the label REAL TIME on the flag. There are some runners, each with a different label on their chest: Runner A: a helicopter hovers over Runner A. Occasionally it picks him up and plops him down right next to the flag. Also, he wears a headset and listens to instructions from his coach to run a little faster or slower, as necessary, to remain abreast of the flag. Runner B: a helicopter hovers over Runner B. If he is behind the flag, it will pick him up and plop him down right next to the flag. However, if he is ahead of the flag it will not pick him up. Runner C: no helicopter ever picks up Runner C, but he does wear a headset and listens to instructions from his coach to run a little faster or a little slower. His coach tells him to run a little faster if he is behind the flag or run a little slower if he is in front of the flag, with the goal of eventually having him right next to the flag. Runner D: like Runner C, he never gets picked up, but he listens to instructions to run a little faster or a little slower. However, instead of telling him to run faster in order to catch up to the flag, or to run slower in order to catch down to the flag, his coach instead tells him to run a little faster if he is moving slower than the flag is moving, and to run a little slower if he is moving faster than the flag is moving. Note that this is very different from Runner C, in that it is not intended to cause him to eventually be right next to the flag, and indeed if it is done right it guarantees that he will *never* be right next to the flag, although he will be moving just as fast as the flag is moving. Runner E: no helicopter, no headset. He just proceeds at his own pace, blissfully unaware of the exhortations of others. Now: which ones of these five runners do you call monotonic? Which ones do you call steady? Regards, Zooko ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On 05Apr2012 09:56, Guido van Rossum gu...@python.org wrote: | On Thu, Apr 5, 2012 at 9:48 AM, PJ Eby p...@telecommunity.com wrote: | What's missing is that if you're using a monotonic clock for timeouts, then | a monotonically-adjusted system clock can do that, subject to the polling | frequency -- it does not break just because the system clock is set | backwards; it simply loses time proportional to the frequency with which it | is polled. | | Depending on the polling frequency sounds like a bad idea, since you | can't know that you're the only user of the clock. You can if you're handed a shiny new clock object in some way, with a not-a-singleton guarrentee. Of course, such a clock is immediately _less_ reliable to synchornisation with other clock users:-) | Also depending on | the use case, too short a timeout may be worse than too long a | timeout. [...] | | For timeout purposes in a single process, such a clock is useful. It just | isn't suitable for benchmarks, or for interprocess coordination. | | I think it would be better if the proposed algorithm (or whatever | algorithm to fix timeouts) was implemented by the | application/library code using the timeout (or provided as a separate | library function), rather than by the clock, since the clock can't | know what fallback behavior the app/lib needs. Absolutely. I for one would be happy with a clocktools module or something offering a bunch of synthetic clocks. Especially if they were compatible in API with whatever clock objects the core time module clocks used, so that a user _could_ add them into the pick-a-clock decision easily. Cheers, -- Cameron Simpson c...@zip.com.au DoD#743 http://www.cskk.ezoshosting.com/cs/ I'd be careful who you call smart or not smart. Smart isn't knowing how to save six bytes. Smart is knowing WHEN. - Peter Cherna, Amiga O.S. Development ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On 05Apr2012 13:39, Zooko Wilcox-O'Hearn zo...@zooko.com wrote: | Good job, Victor Stinner on baking the accumulated knowledge of this | thread into PEP 418. Even though I'm very interested in the topic, I | haven't been able to digest the whole thread(s) on the list and | understand what the current collective understanding is. There isn't a collective understanding :-) That's why all the noise! | The detailed | PEP document helps a lot. Yes indeed, though like all of us I think it could (always) use more detail on my pet concerns. | I think there are still some mistakes, either in our collective | understanding as reflected by the PEP, or in my own head. | | For starters, I still don't understand the first, most basic thing: | what do people mean when they say monotonic clock? I don't | understand the current text of PEP 418 with regard to the definition | of that word. A monotonic clock never returns t0 t1 for t0, t1 being two adjacent polls of the clock. On its own it says nothing about steadiness or correlation with real world time. _Quality of implementation_ says that the montonic() call should try to return a close that is monotonic and _also_ steady and preferably precise. How these things are balancer is a matter of policy. | Allow me to resort to an analogy. There is an infinitely long, | perfectly straight and flat racetrack. There is a flag that gets | dragged along it at a constant rate, with the label REAL TIME on the | flag. There are some runners, each with a different label on their | chest: | | Runner A: a helicopter hovers over Runner A. Occasionally it picks him | up and plops him down right next to the flag. Also, he wears a headset | and listens to instructions from his coach to run a little faster or | slower, as necessary, to remain abreast of the flag. If he always runs forwards, it is montonic. Not very steady when the helicopter comes to play. | Runner B: a helicopter hovers over Runner B. If he is behind the flag, | it will pick him up and plop him down right next to the flag. However, | if he is ahead of the flag it will not pick him up. Seems like runner A without instruction. Monotonic. Not very steady. | Runner C: no helicopter ever picks up Runner C, but he does wear a | headset and listens to instructions from his coach to run a little | faster or a little slower. His coach tells him to run a little faster | if he is behind the flag or run a little slower if he is in front of | the flag, with the goal of eventually having him right next to the | flag. If he always runs forward, monotonic. And steady. | Runner D: like Runner C, he never gets picked up, but he listens to | instructions to run a little faster or a little slower. However, | instead of telling him to run faster in order to catch up to the flag, | or to run slower in order to catch down to the flag, his coach | instead tells him to run a little faster if he is moving slower than | the flag is moving, and to run a little slower if he is moving faster | than the flag is moving. Note that this is very different from Runner | C, in that it is not intended to cause him to eventually be right next | to the flag, and indeed if it is done right it guarantees that he will | *never* be right next to the flag, although he will be moving just as | fast as the flag is moving. | | Runner E: no helicopter, no headset. He just proceeds at his own pace, | blissfully unaware of the exhortations of others. | | Now: which ones of these five runners do you call monotonic? Which | ones do you call steady? If they all run forwards, they're all monotonic. If their coach or helicopter can move then _backwards_ they're not monotonic. If the helicopter can move them an arbitrary (but matching the game plan) distance, they're not steady. Otherwise they are steady, if the runner's speed is always sufficiently close to the flag speed (this threshold and the criteria for measuring it as subject to debate, forming policy). And high resolution has its own flavours, though generally less contentious. Cheers, -- Cameron Simpson c...@zip.com.au DoD#743 http://www.cskk.ezoshosting.com/cs/ If you don't shoot the fish in your barrel, your barrel will soon be full of fish. - Tim Mefford ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On Apr 5, 2012 11:01 AM, Guido van Rossum gu...@python.org wrote: I think it would be better if the proposed algorithm (or whatever algorithm to fix timeouts) was implemented by the application/library code using the timeout (or provided as a separate library function), rather than by the clock, since the clock can't know what fallback behavior the app/lib needs. +1 -eric ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
Cameron Simpson wrote: A monotonic clock never returns t0 t1 for t0, t1 being two adjacent polls of the clock. On its own it says nothing about steadiness or correlation with real world time. No, no, no. This is the strict mathematical meaning of the word monotonic, but the way it's used in relation to OS clocks, it seems to mean rather more than that. A clock whose only guarantee is that it never goes backwards is next to useless. For things like benchmarks and timeouts, the important thing about a clock that it *keeps going forward* at a reasonably constant rate. On the other hand, it can have an arbitrary starting point and doesn't have to be related to any external time standard. I'm assuming this is what Linux et al mean when they talk about a monotonic clock, because anything else doesn't make sense. So if we're going to use the term monotonic at all, I think we should explicitly define it as having this meaning, i.e. both mathematically monotonic and steady. Failure to be clear about this has caused a huge amount of confusion in this thead so far. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On 06Apr2012 13:14, Greg Ewing greg.ew...@canterbury.ac.nz wrote: | Cameron Simpson wrote: | A monotonic clock never returns t0 t1 for t0, t1 being two adjacent | polls of the clock. On its own it says nothing about steadiness or | correlation with real world time. | | No, no, no. | This is the strict mathematical meaning of the word monotonic, | but the way it's used in relation to OS clocks, it seems to | mean rather more than that. It's like my next paragraph didn't exist... _Quality of implementation_ says that the montonic() call should try to return a close that is monotonic and _also_ steady and preferably precise. How these things are balancer is a matter of policy. To forstall things, right at the bottom of this I'm going to say i agree with you about objectives, but not about terminology. I maintain that monotonic still means what I said, and that it is the combination of the word with clock that brings in your other criteria. | A clock whose only guarantee is that it never goes backwards | is next to useless. For things like benchmarks and timeouts, | the important thing about a clock that it *keeps going forward* | at a reasonably constant rate. [...] | I'm assuming this is what Linux et al mean when they talk | about a monotonic clock, because anything else doesn't make | sense. This is why I say there's no global understanding. Why assume? On a Linux box, man 3 clock_getres says (sorry, this is a bit wordy): All implementations support the system-wide real-time clock, which is identified by CLOCK_REALTIME. Its time represents seconds and nanoseconds since the Epoch. When its time is changed, timers for a relative interval are unaffected, but timers for an absolute point in time are affected. More clocks may be implemented. The interpretation of the corresponding time values and the effect on timers is unspecified. Sufficiently recent versions of glibc and the Linux kernel support the following clocks: CLOCK_REALTIME System-wide real-time clock. Setting this clock requires appropriate privileges. CLOCK_MONOTONIC Clock that cannot be set and represents monotonic time since some unspecified starting point. CLOCK_MONOTONIC_RAW (since Linux 2.6.28; Linux-specific) Similar to CLOCK_MONOTONIC, but provides access to a raw hard- ware-based time that is not subject to NTP adjustments. CLOCK_PROCESS_CPUTIME_ID High-resolution per-process timer from the CPU. CLOCK_THREAD_CPUTIME_ID Thread-specific CPU-time clock. The first paragraph is very clear about real time (wall clock, and what time.time() does, being seconds since the epoch). The CLOCK_MONOTONIC* modes clearly imply steadiness. man 3p clock_getres (POSIX clock_getres) is even more verbose and general. | So if we're going to use the term monotonic at all, I think we | should explicitly define it as having this meaning, i.e. | both mathematically monotonic and steady. I think if it is too unsteady then it is not time. So I actually side with you in the requirement for a clock, but monotonic alone does not mean that. Quality of implementation _may_ mean we don't offer something abjectly erratic. | Failure to be clear | about this has caused a huge amount of confusion in this thead | so far. And burdening the word monotonic itself with exciting new meanings doesn't help. monotonic clock, sure, _that_ has additional connotations, but not just the word monotonic. Cheers, -- Cameron Simpson c...@zip.com.au DoD#743 http://www.cskk.ezoshosting.com/cs/ We had the experience, but missed the meaning. - T.S. Eliot ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
Greg Ewing wrote: Cameron Simpson wrote: A monotonic clock never returns t0 t1 for t0, t1 being two adjacent polls of the clock. On its own it says nothing about steadiness or correlation with real world time. No, no, no. This is the strict mathematical meaning of the word monotonic, but the way it's used in relation to OS clocks, it seems to mean rather more than that. A clock whose only guarantee is that it never goes backwards is next to useless. For things like benchmarks and timeouts, the important thing about a clock that it *keeps going forward* That would be a *strictly* monotonic clock, as opposed to merely monotonic. And yes, a merely monotonic clock could just return a constant value, forever: 9, 9, 9, 9, 9, ... and yes, such a thing would be useless. Various people have suggested caching the last value of time() and re-using it if the new value is in the past. This will give a monotonic clock, but since it can give constant timestamps for an indefinite period, it's usefulness is limited. I earlier put forward an alternate implementation which gives no more than one such constant tick in a row. If you know the hardware resolution of the clock, you can even avoid that single constant tick by always advancing the timestamp by that minimum resolution: _prev = _prev_raw = 0 _res = 1e-9 # nanosecond resolution def monotonic(): global _prev, _prev_raw raw = time() delta = max(_res, raw - _prev_raw) _prev_raw = raw _prev += delta return _prev Even if time() jumps backwards, or stays constant, monotonic() here will be strictly monotonic. -- Steven ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On Wed, Apr 04, 2012 at 03:28:34AM +0200, Victor Stinner wrote: Le 04/04/2012 02:33, Steven D'Aprano a écrit : Judging by the hundreds of emails regarding PEP 418, the disagreements about APIs, namings, and even what characteristics clocks should have, I believe that the suggestion is too divisive (and confusing!) to be accepted or rejected at this time. Oh, I just rewrote the PEP before reading your email. Sorry for the noise with this PEP :-) I just read again all emails related to this PEP to complete the PEP. The PEP should now list all proposed API designs. I hope that I did not forget anything. I think the PEP is a good, important PEP, and thank you for being the PEP's champion. But in my opinion, this is too big to rush it and risk locking in a sub-standard API for the next decade or two. I propose for Python 3.3: 1) the os module should expose lightweight wrappers around whatever clocks the operating system provides; Python 3.3 has already time.clock_gettime() and time.clock_getres() with CLOCK_REALTIME, CLOCK_MONOTONIC, CLOCK_MONOTONIC_RAW, CLOCK_HIGHRES. Why does it already have these things when the PEP is not accepted? (This is not a rhetorical question, perhaps there is a good reason why these have been added independently of the PEP.) If I remember correctly, Guido earlier mentioned that he only wanted to see one or two (I forget which) new clocks, and I see in 3.3.0a1 there are already at least five new clocks: monotonic or clock_gettime(CLOCK_MONOTONIC) # Are these the same thing? wallclock clock_gettime(CLOCK_PROCESS_CPUTIME_ID) clock_gettime(CLOCK_REALTIME) clock_gettime(CLOCK_THREAD_CPUTIME_ID) plus the old ways, time.time and time.clock. (Neither of which seems to have a clock-id.) mach_absolute_time() and GetTickCount/GetTick64 are not available yet. That will make potentially 10 different clocks in the time module. It may be that, eventually, Python should support all these ten different clocks. (Personally, I doubt that the average Python programmer cares about the difference between time() and clock(), let alone another eight more.) But there's no rush. I think we should start by supporting OS-specific clocks in the os module, and then once we have some best-practice idioms, we can promote some of them to the time module. -- Steven ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On Wed, 4 Apr 2012 18:09:40 +1000 Steven D'Aprano st...@pearwood.info wrote: Python 3.3 has already time.clock_gettime() and time.clock_getres() with CLOCK_REALTIME, CLOCK_MONOTONIC, CLOCK_MONOTONIC_RAW, CLOCK_HIGHRES. Why does it already have these things when the PEP is not accepted? (This is not a rhetorical question, perhaps there is a good reason why these have been added independently of the PEP.) Because these are thin (low-level) wrappers around the corresponding POSIX APIs, so there is no reason not to add them. I know you were asking for such wrappers to be in the os module, but my understanding is that time-related functions should preferably go into the time module. os is already full of very diverse stuff, and documentation-wise it is better if time-related functions end up in a time-related module. Otherwise we'll end up having to cross-link manually, which is always cumbersome (for us) and less practical (for the reader). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
Why does it already have these things when the PEP is not accepted? ... (This is not a rhetorical question, perhaps there is a good reason why these have been added independently of the PEP.) time.clock_gettime() friends were added by the issue #10278. The function was added before someone asked (me) to write a PEP. The need of a PEP came later, when time.wallclock() and time.monotonic() functions were added. Because these are thin (low-level) wrappers around the corresponding POSIX APIs, so there is no reason not to add them. time.clock_gettime() can be used for other purpose than a monotonic clock. For example, CLOCK_THREAD_CPUTIME_ID is the only available function to get the Thread-specific CPU-time clock. It also gives access to CLOCK_MONOTONIC_RAW which is not used by the time.monotonic() function proposed in the PEP. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
I failed to propose a consistent and clear API because I (and Guido!) wanted to fallback to the system clock. Falling back to the system clock is a problem when you have to define the function in the documentation or if you don't want to use the system clock (but do something else) if no monotonic clock is available. Well, it was not only Guido and me. Nick Coghlan wrote: However, I think Victor's right to point out that the *standard library* needs to have a fallback to maintain backwards compatibility if time.monotonic() isn't available, and it seems silly to implement the same fallback logic in every module where we manipulate timeouts. and Since duplicating that logic in every module that handles timeouts would be silly, it makes sense to provide an obvious way to do it in the time module. Michael Foord wrote: It is this always-having-to-manually-fallback-depending-on-os that I was hoping your new functionality would avoid. Is time.try_monotonic() suitable for this usecase? The following functions / libraries fall back to the system clock if no monotonic clock is available: - QElapsedTimer class of the Qt library - g_get_monotonic_time() of the glib library - monotonic_clock library - AbsoluteTime.now (third-party Ruby library), AbsoluteTime.monotonic? tells if AbsoluteTime.now is monotonic Extract of the glib doc: Otherwise, we make a best effort that probably involves returning the wall clock time (with at least microsecond accuracy, subject to the limitations of the OS kernel). -- Only the python-monotonic-time fails with an OSError if no monotonic clock is available. System.nanoTime() of Java has few garantee: Returns the current value of the most precise available system timer, in nanoseconds. This method can only be used to measure elapsed time and is not related to any other notion of system or wall-clock time. The value returned represents nanoseconds since some fixed but arbitrary time (perhaps in the future, so values may be negative). I don't even know if it is monotonic, steady or has an high resolution. Note: Boost.Chrono.high_resolution_clock falls back to the system clock if no steady clock is available. (But the high-resolution clock idea was deferred, it's something different than a monotonic or steady clock.) Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
I am fine with the PEP as it is now (2012-04-04 15:34 GMT). A question: Since the only monotonic clock that can be adjusted by NTP is Linux' CLOCK_MONOTONIC, if we avoid it, then time.monotonic() would always give a clock that isn't adjusted by NTP. That would however mean we wouldn't support monotonic clocks on systems that run a Linux that is older than mid-2008. Is this generally seen as a problem? //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On 2012-04-03, at 9:28 PM, Victor Stinner wrote: In the last version of my PEP, time.monotonic() is simply defined as a monotonic clock (cannot go backward). There is no more ... best ... in its definition. I like the last version of the PEP ;) - Yury ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
I failed to propose a consistent and clear API because I (and Guido!) wanted to fallback to the system clock. Falling back to the system clock is a problem when you have to define the function in the documentation or if you don't want to use the system clock (but do something else) if no monotonic clock is available. More details why it's hard to define such function and why I dropped it from the PEP. If someone wants to propose again such function (monotonic or fallback to system clock), two issues should be solved: - name of the function - description of the function At least, monotonic and steady names are not acceptable names for such function, even if the function has an optional strict=False or fallback=True parameter. By the way, someone complained that having a boolean parameter requires to create a new function if you want to call it without an argument (use a lambda function, functools.partial, or anything else). Example: get_time = lambda: try_monotonic(fallback=True) t = get_time() The description should give the least guarantees. If the function doesn't promise anything (or only a few weak properties), it is harder to decide which clock do you need for your use case: time.clock(), time.time(), time.monotonic(), time.name of the monotonic-of-fallback function, ... Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
Lennart Regebro wrote: Since the only monotonic clock that can be adjusted by NTP is Linux' CLOCK_MONOTONIC, if we avoid it, then time.monotonic() would always give a clock that isn't adjusted by NTP. I thought we decided that NTP adjustment isn't an issue, because it's always gradual. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On Apr 4, 2012 7:28 PM, Victor Stinner victor.stin...@gmail.com wrote: More details why it's hard to define such function and why I dropped it from the PEP. If someone wants to propose again such function (monotonic or fallback to system clock), two issues should be solved: - name of the function - description of the function Maybe I missed it, but did anyone ever give a reason why the fallback couldn't be to Steven D'Aprano's monotonic wrapper algorithm over the system clock? (Given a suitable minimum delta.) That function appeared to me to provide a sufficiently monotonic clock for timeout purposes, if nothing else. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On 04Apr2012 22:23, PJ Eby p...@telecommunity.com wrote: | On Apr 4, 2012 7:28 PM, Victor Stinner victor.stin...@gmail.com wrote: | More details why it's hard to define such function and why I dropped | it from the PEP. | | If someone wants to propose again such function (monotonic or | fallback to system clock), two issues should be solved: | | - name of the function | - description of the function | | Maybe I missed it, but did anyone ever give a reason why the fallback | couldn't be to Steven D'Aprano's monotonic wrapper algorithm over the | system clock? (Given a suitable minimum delta.) That function appeared to | me to provide a sufficiently monotonic clock for timeout purposes, if | nothing else. It was pointed out (by Nick Coglan I think?) that if the system clock stepped backwards then a timeout would be extended by at least that long. For example, code that waited (by polling the synthetic clock) for 1s could easily wait an hour if the system clock stepped back that far. Probaby undesirable. I think synthetic clocks are an extra task; they will all have side effects of one kind of another. A system monotonic clock, by contrast, may have access to some clock hardware that doesn't step when the main system clock gets adjusted, and can stay monotonic. A synthetic clock without such access can't behave as nicely. If synthetic clocks get handed out as fallback there should be some way for the user to know, or a big glaring negative guarrentee in the docs and on platforms without a system monotonic clock you might get a clock with weird (but monotonic!) behaviours if you use the fallback. Cheers, -- Cameron Simpson c...@zip.com.au DoD#743 http://www.cskk.ezoshosting.com/cs/ Tachyon: A gluon that's not completely dry. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On Thu, Apr 5, 2012 at 1:41 PM, Cameron Simpson c...@zip.com.au wrote: It was pointed out (by Nick Coglan I think?) that if the system clock stepped backwards then a timeout would be extended by at least that long. Guido pointed it out (it was in a reply to me, though). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 418 is too divisive and confusing and should be postponed
Judging by the hundreds of emails regarding PEP 418, the disagreements about APIs, namings, and even what characteristics clocks should have, I believe that the suggestion is too divisive (and confusing!) to be accepted or rejected at this time. Everyone has a different opinion, everyone believes their opinion holds for the majority, and it isn't practical for anyone to read the entire discussion. I propose for Python 3.3: 1) the os module should expose lightweight wrappers around whatever clocks the operating system provides; 2) the time module should NOT provide any further clocks other than the existing time() and clock() functions (but see point 4 below); 3) we postpone PEP 418 until there is some real-world experience with using the os clocks from Python and we can develop a consensus of what is actually needed rather than what people think we need (i.e. probably in 3.4); 4) if the standard library has need for a use the best clock available, for some definition of best, and fall back to time() if not clock, then the time module should do the simplest thing that could possible work, flagged as a private function: try: from os import bestclock as _bestclock except ImportError: _bestclock = time This can always be promoted to a public function later, if necessary. Python has worked pretty well without high res and monotonic clocks for 20 years. Let's not rush into a suboptimal design based on who can outlast everyone else in this discussion. -- Steven ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On Wed, Apr 4, 2012 at 10:33 AM, Steven D'Aprano st...@pearwood.info wrote: Python has worked pretty well without high res and monotonic clocks for 20 years. Let's not rush into a suboptimal design based on who can outlast everyone else in this discussion. +1 FWIW, I'd be fine with underscore prefixes on *any* additions to the relevant module APIs for 3.3. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
Le 04/04/2012 02:33, Steven D'Aprano a écrit : Judging by the hundreds of emails regarding PEP 418, the disagreements about APIs, namings, and even what characteristics clocks should have, I believe that the suggestion is too divisive (and confusing!) to be accepted or rejected at this time. Oh, I just rewrote the PEP before reading your email. Sorry for the noise with this PEP :-) I just read again all emails related to this PEP to complete the PEP. The PEP should now list all proposed API designs. I hope that I did not forget anything. I failed to propose a consistent and clear API because I (and Guido!) wanted to fallback to the system clock. Falling back to the system clock is a problem when you have to define the function in the documentation or if you don't want to use the system clock (but do something else) if no monotonic clock is available. So I rewrote the PEP to simplify it: * Don't fallback to system clock: time.monotonic() is always monotonic (cannot go backward), but it is not always available. You have to write a classic try/except ImportError which has a nice advantage: your program will work on Python older than 3.3 ;-) * Remove the time.perf_counter() function (it was called time.highres() before). highres notion was confusing. I only wrote the function to expose QueryPerformanceCounter (while it was already accessible using os.clock()). The function was not well defined. Another PEP should be written or at least the subject should be discussed after the PEP 418 (monotonic clock). * Rename time.steady() to time.monotonic(), again :-) Everyone has a different opinion, everyone believes their opinion holds for the majority, and it isn't practical for anyone to read the entire discussion. I read most emails and I can say that: * There is a need for a monotonic clock * Most people prefer to handle explicitly the fallback if no monotonic clock is available * Most people don't want to call the new function steady because it stands for something different I propose for Python 3.3: 1) the os module should expose lightweight wrappers around whatever clocks the operating system provides; Python 3.3 has already time.clock_gettime() and time.clock_getres() with CLOCK_REALTIME, CLOCK_MONOTONIC, CLOCK_MONOTONIC_RAW, CLOCK_HIGHRES. mach_absolute_time() and GetTickCount/GetTick64 are not available yet. 3) we postpone PEP 418 until there is some real-world experience with using the os clocks from Python and we can develop a consensus of what is actually needed rather than what people think we need (i.e. probably in 3.4); Many applications already implement their own monotonic clock. Some libraries provide also such clock for Python. On UNIX, it's always using clock_gettime(MONOTONIC). On Windows, it's sometimes GetTickCount, sometimes QueryPerformanceCounter. On Mac OS X, it's always mach_absolute_time(). I didn't find a library supporting Solaris. 4) if the standard library has need for a use the best clock available, for some definition of best, and fall back to time() if not clock, then the time module should do the simplest thing that could possible work, flagged as a private function: In the last version of my PEP, time.monotonic() is simply defined as a monotonic clock (cannot go backward). There is no more ... best ... in its definition. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
On 2012-04-03, at 8:33 PM, Steven D'Aprano wrote: 1) the os module should expose lightweight wrappers around whatever clocks the operating system provides; +1. That should make it flexible enough to those who really need it. 2) the time module should NOT provide any further clocks other than the existing time() and clock() functions (but see point 4 below); 3) we postpone PEP 418 until there is some real-world experience with using the os clocks from Python and we can develop a consensus of what is actually needed rather than what people think we need (i.e. probably in 3.4); 4) if the standard library has need for a use the best clock available, for some definition of best, and fall back to time() if not clock, then the time module should do the simplest thing that could possible work, flagged as a private function: +1 on overall idea too. - Yury ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
Finally! We've come full circle. +1 for monotonic as just described by Victor. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 is too divisive and confusing and should be postponed
Lock it in before the paint dries. On Apr 4, 2012 10:05 AM, Matt Joiner anacro...@gmail.com wrote: Finally! We've come full circle. +1 for monotonic as just described by Victor. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
