Re: UT1 confidence

2007-01-18 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Zefram [EMAIL PROTECTED] writes:
: Steve Allen wrote:
: http://www.ucolick.org/~sla/leapsecs/torino/arias_3.pdf
:
: This is the really interesting one.  They present the accuracy of
: simulated predictions of UT1, and that accuracy is much poorer than
: the figures we've been discussing so far.  But they make it clear that
: the prediction is by a naive algorithm, not even applying the well-known
: periodic terms that are used in UT2.  This is obviously not the algorithm
: used by IERS.

The good news is that even an incredibly stupid (in terms of knowledge
used) algorithm can predict almost 3 years out

Warner


Re: UT1 confidence

2007-01-17 Thread M. Warner Losh
Steve,

thank you for this enlightening report.

In message: [EMAIL PROTECTED]
Steve Allen [EMAIL PROTECTED] writes:
: On Wed 2007-01-17T12:31:14 -0700, Warner Losh hath writ:
:  It has been remarked that the current state of the art is that 100ms
:  accuracy can be predicted about a year in advance only and that the
:  models are constantly undergoing refinement.  It has been estimated
:  that IERS could issue leap seconds, with today's technology, about 3-5
:  years out and still be in a 95% or 99% band of certainty that the 0.9s
:  margin is maintained.  However, I can't find papers that show these
:  models or point to any better data than hearsay...
:
: The best that I know of were the ones presented at the Colloquium that
: the WP7A SRG held in Torino in May 2003.  There was a time when the
: host institution (IEN) was providing the proceedings online, but the
: contents of that URL went away sometime around a year ago.  (I wonder
: if they may not have liked the conclusion that was reached.)
:
: In the spirit of promulgation I provide what they once did at
: http://www.ucolick.org/~sla/leapsecs/torino/ITU.shtml
:
: The conclusion was originally a powerpoint drafted in real time, it is
: http://www.ucolick.org/~sla/leapsecs/torino/closure.pdf
:
: The indications of how well predictions of UT1 might be done are found
: in three presentations to which Felicitas Arias contributed.
: There are two which were powerpoint
: http://www.ucolick.org/~sla/leapsecs/torino/guinot.pdf
: http://www.ucolick.org/~sla/leapsecs/torino/arias_2.pdf
: and one which is a more verbose writeup of one of the powerpoints
: http://www.ucolick.org/~sla/leapsecs/torino/arias_3.pdf

I like figure 8, that shows that 20ms steps lead to 50ms steps lead to
100ms steps lead to 1s steps. :-)

I also like the quotes:

Dating in UTC is ambiguous when a positive leap second occurs in
systems other than those using hours, minutes and seconds; in this
latter system the use of second 60 may a cause of difficulty.

and

Thus UT1 is not, strictly speaking, a form of solar time

Also, did I miss figure 9 in arias_3?

Proposal II has gotten much press here (the leap hour one), but
Proposal I sounds a lot like what I've suggested: Use TAI time and let
countries move the time zones when they feel like they no longer are
close enough, but it kinda omits that last part...

Warner


Re: Introduction of long term scheduling

2007-01-08 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Tony Finch [EMAIL PROTECTED] writes:
: On Sat, 6 Jan 2007, M. Warner Losh wrote:
: 
:  Unfortunately, the kernel has to have a notion of time stepping around
:  a leap-second if it implements ntp.
:
: Surely ntpd could be altered to isolate the kernel from ntp's broken
: timescale (assuming the kernel has an atomic seconds count timescale)

ntpd is the one that mandates it.

One could use an atomic scale in the kernel, but nobody that I'm aware
of does.

Warner


Re: Introduction of long term scheduling

2007-01-07 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Tony Finch [EMAIL PROTECTED] writes:
: On Sat, 6 Jan 2007, M. Warner Losh wrote:
: 
:  Most filesystems store time as UTC anyway...
:
: POSIX time is not UTC :-)

True.  It is designed to be UTC, but fails to properly implement UTC's
leap seconds and intervals around leapseconds.

Warner


Re: Introduction of long term scheduling

2007-01-07 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Rob Seaman [EMAIL PROTECTED] writes:
:  If by some limp attempt you mean returns the correct time then you
:  are correct.
:
: It's not the correct time under the current standard if the
: timekeeping model doesn't implement leap seconds correctly.  I don't
: think this is an impossible expectation, see http://
: www.eecis.udel.edu/~mills/exec.html, starting with the Levine and
: Mills PTTI paper.

It implements exactly what ntpd wants.  I asked Judah Levine when
determining what was pedantically correct during the leap second.  I
also consulted with the many different resources avaialable to
deterimine what the right thing is.  Of course, there are different
explaintions about what the leap second should look like depending on
if you listen to Dr. Levine or Dr Mills.  Dr. Mills web site says
'redo the first second of the next day' while Dr. Levine's
leapsecond.dat file says 'repeat the last second of the day.'
Actually, both of them hedge and say 'most systems implement...'  or
some variation on that theme.

It is possible to determine when you are in a leap second using ntp
extensions with their model.  Just not with POSIX interfaces.  The
POSIX interfaces are kludged, while the ntpd ones give you enough info
to know to print :59 or :60, but POSIX time_t is insufficiently
expressive, by itself, to know.  But ntp_gettime returns a timespec
for the time, as well as a time_state for the current time status,
which includes TIME_INS and TIME_DEL for psotive and negative leap
second 'warning' for end of the day so you know there will be a leap
today, and TIME_WAIT for the actual positive leap second itself
(there's nothing for a negative leapsecond, obviously).

So I stand by my returns the correct time statement.

Warner


Re: Introduction of long term scheduling

2007-01-06 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Tony Finch [EMAIL PROTECTED] writes:
: On Sat, 6 Jan 2007, M. Warner Losh wrote:
: 
:  OSes usually deal with timestamps all the time for various things.  To
:  find out how much CPU to bill a process, to more mondane things.
:  Having to do all these gymnastics is going to hurt performance.
:
: That's why leap second handling should be done in userland as part of the
: conversion from clock (scalar) time to civil (broken-down) time.

Right.  And that's what makes things hard because the kernel time
clock needs to be monotonic, and leap seconds break that rule if one
does things in UTC such that the naive math just works (aka POSIX
time_t).  Some systems punt on keeping posix time internally, but have
complications for getting leapseconds right for times they return to
userland

Warner


Re: Introduction of long term scheduling

2007-01-06 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Rob Seaman [EMAIL PROTECTED] writes:
: Warner Losh wrote:
:
:  leap seconds break that rule if one does things in UTC such that
:  the naive math just works
:
: All civil timekeeping, and most precision timekeeping, requires only
: pretty naive math.  Whatever the problem is - or is not - with leap
: seconds, it isn't the arithmetic involved.  Take a look a [EMAIL PROTECTED]
: and other BOINC projects.  Modern computers have firepower to burn in
: fluff like live 3-D screensavers.  POSIX time handling just sucks for
: no good reason.  Other system interfaces successfully implement
: significantly more stringent facilities.

But modern servers and routers don't.  Anything that makes the math
harder (more computationally expensive) can have huge effects on
performance in these areas.  That's because the math is done so often
that any little change causes big headaches.

: Expecting to be able to naively subtract timestamps to compute an
: accurate interval reminds me of expecting to be able to naively stuff
: pointers into integer datatypes and have nothing ever go wrong.

Well, the kernel doesn't expect to be able to do that.  Internally,
all the FreeBSD kernel does is time based on a monotonically
increasing second count since boot.  When time is returned, it is
adjusted to the right wall time.  The kernel only worries about leap
seconds when time is incremented, since the ntpd portion in the kernel
needs to return special things during the leap second.  If there were
no leapseconds, then even that computation could be eliminated.  One
might think that one could 'defer' this work to gettimeofday and
friends, but that turns out to not be possible (or at least it is much
more inefficient to do it there).

Since the interface to the kernel is time_t, there's really no chance
for the kernel to do anything smarter with leapseconds.  gettimeofday,
time and clock_gettime all return a time_t in different flavors.

In short, you are taking things out of context and drawing the wrong
conclusion about what is done.  It is these complications, which I've
had to deal with over the past 7 years, that have lead me to the
understanding of the complications.  Espeically the 'non-uniform radix
crap' that's in UTC.  It really does complicate things in a number of
places that you wouldn't think.  To dimissively suggest it is only a
problem when subtracting two numbers to get an interval time is to
completely misunderstand the complications that leapseconds introduce
into systems and the unexpected places where they pop up.  Really, it
is a lot more complicated than just the 'simple' case you've latched
onto.

: A
: strongly typed language might even overload the subtraction of UTC
: typed variables with the correct time-of-day to interval
: calculations.

Kernels aren't written in these languages.  To base one's arugments
about what the right type for time is that is predicated on these
langauges is a non-starter.

: But then, what should one expect the subtraction of
: Earth orientation values to return but some sort of angle, not an
: interval?

These are a specialized thing that kernels don't care about.

Warner


Re: Introduction of long term scheduling

2007-01-06 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Ashley Yakeley [EMAIL PROTECTED] writes:
: On Jan 6, 2007, at 08:35, M. Warner Losh wrote:
:
:  So for the foreseeable future,
:  timestamps in OSes will be a count of seconds and a fractional second
:  part.  That's not going to change anytime soon even with faster
:  machines, more memory, etc.  Too many transaction processing
:  applications demand maximum speed.
:
: That's sensible for a simple timestamp, but trying to squeeze in a
: leap-second table probably isn't such a good idea.

Unfortunately, the kernel has to have a notion of time stepping around
a leap-second if it implements ntp.  There's no way around that that
isn't horribly expensive or difficult to code.  The reasons for the
kernel's need to know have been enumerated elsewhere...

Warner


Re: Introduction of long term scheduling

2007-01-06 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Ashley Yakeley [EMAIL PROTECTED] writes:
: On Jan 6, 2007, at 16:18, M. Warner Losh wrote:
:
:  Unfortunately, the kernel has to have a notion of time stepping around
:  a leap-second if it implements ntp.  There's no way around that that
:  isn't horribly expensive or difficult to code.  The reasons for the
:  kernel's need to know have been enumerated elsewhere...
:
: Presumably it only needs to know the next leap-second to do this, not
: the whole known table?

Yes.  ntpd or another agent tells it when leap seconds are coming.  It
doesn't need a table.  Then again, none of the broadcast time services
provide a table...

Warner


Re: Introduction of long term scheduling

2007-01-06 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Rob Seaman [EMAIL PROTECTED] writes:
: Warner Losh wrote:
:  Anything that makes the math
:  harder (more computationally expensive) can have huge effects on
:  performance in these areas.  That's because the math is done so often
:  that any little change causes big headaches.
:
: Every IP packet has a 1's complement checksum.  (That not all
: switches handle these properly is a different issue.)

Actually, every IP does not have a 1's complement checksum.  Sure,
there is a trivial one that covers the 20 bytes of header, but that's
it.  Most hardware these days off loads checksumming to the hardware
anyway to increase the throughput.  Maybe you are thinking of TCP or
UDP :-).  Often, the packets are copied and therefore in the cache, so
the addition operations are very cheap.

: Calculating a
: checksum is about as expensive (or more so) than subtracting
: timestamps the right way.  I have a hard time believing that epoch-
:  interval conversions have to be performed more often than IP
: packets are assembled.

Benchmarks do not lie.  Also, you are misunderstanding the purpose of
timestamps in the kernel.  Adding or subtracting two of them is
relatively easy.  Converting to a broken down format or doing math
with the complicated forms is much more code intensive.  Dealing with
broken down forms, and all the special cases usually involves
multiplcation and division, when tend to be more computationally
expensive than the checksum.

: One imagines (would love to be pointed to
: actual literature regarding such issues) that most computer time
: handling devolves to requirements for relative intervals and epochs,
: not to stepping outside to any external clock at all.  Certainly the
: hardware clocking of signals is an issue entirely separate from what
: we've been discussing as timekeeping and traceability.  (And note
: that astronomers face much more rigorous requirements in a number of
: ways when clocking out their CCDs.)

Having actually participated in the benchmarks that showed the effects
of inefficient timekeeping, I can say that they have a measurable
effect.  I'll try to find references that the benchmarks generated.

:  Well, the kernel doesn't expect to be able to do that.  Internally,
:  all the FreeBSD kernel does is time based on a monotonically
:  increasing second count since boot.  When time is returned, it is
:  adjusted to the right wall time.
:
: Well, no - the point is that only some limp attempt is made to adjust
: to the right time.

If by some limp attempt you mean returns the correct time then you
are correct.

:  The kernel only worries about leap
:  seconds when time is incremented, since the ntpd portion in the kernel
:  needs to return special things during the leap second.  If there were
:  no leapseconds, then even that computation could be eliminated.  One
:  might think that one could 'defer' this work to gettimeofday and
:  friends, but that turns out to not be possible (or at least it is much
:  more inefficient to do it there).
:
: One might imagine that an interface could be devised that would only
: carry the burden for a leap second when a leap second is actually
: pending.  Then it could be handled like any other rare phenomenon
: that has to be dealt with correctly - like context switching or
: swapping.

You'd think that, but you have to test to see if something was
pending.  And the code actually does that.

:  Really, it is a lot more complicated than just the 'simple' case
:  you've latched onto.
:
: Ditto for Earth orientation and its relation to civil timekeeping.
: I'm happy to admit that getting it right at the CPU level is
: complex.  Shouldn't we be focusing on that, rather than on
: eviscerating mean solar time?

Did I say anything about eviscerating mean solar time?

: A proposal to actually address the intrinsic complications of
: timekeeping is more likely to be received warmly than is a kludge or
: partial workaround.  I suspect it would be a lot more fun, too.

I'm just suggesting that some of the suggested ideas have real
performance issues that means they wouldn't even be considered as
viable options.

:  Kernels aren't written in these languages.  To base one's arugments
:  about what the right type for time is that is predicated on these
:  langauges is a non-starter.
:
: No, but the kernels can implement support for these types and the
: applications can code to them in whatever language.  Again - there is
: a hell of a lot more complicated stuff going on under the hood than
: what would be required to implement a proper model of timekeeping.

True, but timekeeping is one of those areas of the kernel that extra
overhead is called so many times that making it more complex hurts a
lot more than you'd naively think.

Warner


Re: Introduction of long term scheduling

2007-01-02 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
John Cowan [EMAIL PROTECTED] writes:
: Warner Losh scripsit:
:
:  There's no provision for emergency leapseconds.  They just have to be
:  at the end of the month, and annoucned 8 weeks in advance.  IERS has
:  actually exceeded this mandate by announcing them ~24 weeks in advance
:  in recent history.
:
: So much the worse.  That means that if the Earth hiccups on March 7, the
: value of |DUT1| will not return to normal until May 31.

Yes.  But it would take a change in angular momementum would likely
mean that |DUT1| being a little too large would be the least of our
worries.

The earthquake that hit Indonesia last year changed the time of day by
microseconds.  What would cause a sudden jump of hundreds of
milliseconds hurts my brain to contemplate.

Warner


Re: A lurker surfaces

2007-01-02 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Daniel R. Tobias [EMAIL PROTECTED] writes:
: On 2 Jan 2007 at 11:56, Ashley Yakeley wrote:
:
:  GPS is TAI. I'm not proposing abandoning TAI for those applications
:  that need it.
:
: It's a few seconds off from TAI, isn't it?  It was synchronized to
: UTC in 1980 (I think), but without subsequent leap seconds, so it's
: now different from both TAI and UTC.  They probably should just have
: used TAI if they wanted a time scale without leap seconds, rather
: than ending up creating a different one.

Yes.  There's a fixed offset between seconds in TAI and seconds in
GPS.  The GPS timescale is tied to seconds in UTC(NRO).  TAI is a
paper clock, computed after the fact, so GPS can't ever be TAI time.
However, the differences there are down in the nanosecond or so
range.

There's a big philosophical opposition to using the paper clock that
is TAI in a real-time operational timescale that GPS uses.  The
European version of GPS originally specified TAI time, but this was
changed in later revisions to be the same as GPS time.  There's an
extreme reluctance in the time community to call something without
leap seconds TAI or TAI + fixed offset.  TAI means something very
specific.  That's the other problem with just using TAI, btw, but
explaining that point is very hard...

The principle time scientist made me change the description of the
output of measurment time for a product I did.  I described it as TAI
time seconds since 1970.  Instead, he descriped it as Number of PPS
ticks in the UTC time scale since 1972 + 63072000 + 10.
Mathematically, they work out to be the same thing, but he was
extremely resistant to calling it TAI time or using the TAI moniker
for it at all.  When I asked him about this, he said that TAI time
isn't realized in real time, but UTC is.  UTC is what one can measure
against.  Producing a number that corresponded to TAI time was OK, and
likely the least confusing thing to do (we give a second number and
UTC time in '-MM-DD HH:MM:SS Z' as well as the channel and the
measurement for that time in out output), but actually calling it TAI
would 'confuse' the really smart time geeks out in the world.  I asked
him for a reference where I could read up on this, and he shrugged and
said he just knew it and didn't know of any good write up.

Warner


Re: A lurker surfaces

2007-01-01 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Ashley Yakeley [EMAIL PROTECTED] writes:
: Software should serve human needs, not the other
: way around. Anyone needing fixed seconds should use TAI.

I think this idea would be harder to implement than the current
leapseconds.

There are many systems that need to display UTC externally, but need
to operate on a tai-like timescale internally.  Having there being a
sliding delta between them would be a nightmare.

Warner


Re: A lurker surfaces

2007-01-01 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
[EMAIL PROTECTED] (Michael Sokolov) writes:
: The people who complain about leap seconds screwing up their interval
: time computations are usually told to use TAI.  They retort that they
: need interval time *between civil timestamps*.

Actaully, interval deltas are needed, but it is in TAI time.  There
needs to be, in the systems I've done, a timescale without leapseconds
to keep all the measurements sane.  In addition, many of the systems
I've done have other functions they do also, some of which need to
present UTC to the user.  As much of a pita that leapseconds are
today, having the converion between TAI and UTC (or put another way:
between GPS and UTC or LORAN and UTC) would make these systems
signficantly more difficult to get correct.

There's also times when our systems take in events that are
timestamped using UTC time, so we need to back correlate them to TAI
or whatever internal timescale we're using.  Some of that is because
UTC is the standard for exchanging time, part of that is because the
events in question are measured in whatever timescale is present on an
NTP server.

: To me that seems like
: what they are really measuring as interval time is not physical
: interval time, but how much time has elapsed *in civil society*.  Hence
: my idea of civil interval time that's completely decoupled from physical
: time and is instead defined as the turning angle of the clock on the
: Kremlin tower.

Actually, you are wrong.  The intervals is in the number of pps that
have happened, or fractions thereof.  Civil time does intrude because
that's what people use right now to know time fo day.  In the systems
I've done, you need to know both.

The requirements, as you may have noticed, for my needs aren't that
the intervals be done in TAI (we use a variant of LORAN time due to
historical accident, but with a 1970 epoch), but rather that UTC and
these time scales be convertable between one another.

Like I've said a number of times, saying 'just use TAI' isn't viable
because of the conversion issue.  Using TAI (or something with the no
leapsecond regualrity that TAI gives) is necessary for the algorithms
to work.  However, external pressures also require that some things be
done in UTC time and that some of the external sources of data use UTC
time and that needs to be back correlated to the internal timescale
that we're using.

The algorithms, btw, basically integrate frequencies of different
clocks, over time, to predict phase difference.  In this case, you
definitely want to use an interval, and not whatever weirdnesses civil
time happened to do in that interval.  Using an civil time interval
would introduce errors in algorithms.  These algorithms are used to
estimate how well the clocks are doing, but other parts of the system
need to play our UTC for things like NTPD and IRIG.

Warner


Re: A lurker surfaces

2006-12-31 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
John Cowan [EMAIL PROTECTED] writes:
: Tony Finch scripsit:
:  On Sun, 31 Dec 2006, John Cowan wrote:
:  
:   However, it's clear that UTC does not contain the sort of jumps
:   that LCT does, where a single broken-down time may represent
:   two different UTC seconds.
: 
:  Not if you include the timezone offset in the representation.
:
: Quite so.  Or alternatively a standard/daylight flag.  The point is,
: people usually don't.

This is the leapsecond problem in a nutshell:  While it is possible to
keep enough information around in the representation of time, most
people opt not to do so.  It complicates the representation of time,
and is one (or more) pieces of information that need to be carried
around, mostly just to handle weird edge cases.  Sure, you can do it,
and some people try.  However, there's no standardized way to do so,
and people re-invent it wrong over and over again.

Another poster I think was right: ntpd (and the underlying OS) hides a
lot of these sins.  Either by just twitterpating at the leapsecond in
some way (time stands still, time jumps back a little, etc), or for a
period of time around the leapsecond (by sticking its fingers in its
ears, singing lalala and changing the size of a second to get through
it a millisecond at a time).

Warner


Re: A lurker surfaces

2006-12-31 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Rob Seaman [EMAIL PROTECTED] writes:
: Poul-Henning Kamp wrote:
:
:  Rob, If you feel uncomfortable with calling leapseconds
:  discontinuities, then we can use the term arrhythmia instead.
:
: Which raises the question of why projects requiring an interval time
: scale lacking in such arrhythmias would have selected UTC in the
: first place.

The real world often times requires both kinds of time keeping, with
correlation back and forth between all the timescales.  The customers
want to deal with times in UTC.  To get an effective, high precision
time system, you need to count seconds in a sane way (UTC isn't a sane
way for this purpose) so some conversion between this count and UTC is
needed.

The problem is that most interesting systems have a need to do things
based on UTC time of day, as well as in a time scale that has none of
the complications of the unpredictable UTC implementation we have
today.  So you can't just 'pick one' because real-world systems need
to use different ones for different things, and they also need to
translates events from one timescale to events in the other
timescale.

Let me give you a concrete example.  Let us say we are measuring a
number of atomic clocks against one another.  We get the measurements
and those measurements are in a monotonic timescale that is defined as
PPS's from some epoch (or number of 5MHz or 10MHz ticks from some
arbitrary epoch).  We get data from GPS, which has a similar
timescale.  We compute the clock states and deside that we need to
steer one of the clocks.  These algoritmns are very much elapsed time
sorts of deals.  So we schedule a time to steer it, and then note the
time when we are done steering it.  Since we cannot easily get the GPS
or timer times without interfereing with the operations of those
subsystems (maybe because the steering system and GPS system are on
dufferent cpus), we get the system time when the steer starts and when
the clock tells us the steer is complete (since a steer isn't an
instantaneous event).  Since this system also is an ntpd server, that
system time is in UTC.  Now we have to convert UTC into the internal
timescale so that the algorithms know when the steer happened and can
take that into account for their next round of measurments and
decision making.

To make this all work out well, one has to have perfect leap second
knowledge and all the special case code you have for dealing with the
arrhythmia of a leap second has to work perfectly.  Anything that can
be dobe to make things more perdictable helps out quite a bit...

Warner


Re: A lurker surfaces

2006-12-30 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Rob Seaman [EMAIL PROTECTED] writes:
: Just a reminder that UTC has no - none - nada - discontinuities.

At the very least, the TAI-UTC difference is discontinuous with jumps
at the leap seconds.  One could easily suggest that 'UTC has
discontinuities' really is a shorthand way of saying this.  One could
also call them irregularities, like the time scale is constipated, as
well.

And the variable radix seems to be to be a crude way to define away
the problem.  You never know, unless you look it up in a table, when
to do the variation in the radix.

: Various computer mis-implementations may, but the standard is very
: carefully constructed to avoid spring-forward or fall-back gaps or do-
: overs.

Well, if you count the variable radix notation as being 'continuous'
then maybe you are right.  However, you never know when the radix is
variable, hence the assertion on many people's part that UTC is
discontinuous.

Warner


Re: Design - a Tufte decision

2006-12-28 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Rob Seaman [EMAIL PROTECTED] writes:
: On Dec 27, 2006, at 12:02 AM, M. Warner Losh wrote:
:
:  Calculating time intervals for times 6+ months in the future can be
:  the least of one's worries when one tries to code up a library to deal
:  gracefully with these different failure modes.  The trivial case where
:  one has perfect knowledge of TAI-UTC and one can keep that knowledge
:  current is very simple in comparison.  Dealing with this case is very
:  simple, and is the way most people think about leap seconds.  But
:  dealing with the edge cases can be difficult because there are so
:  many, and so many that people forget to test or conceive of until the
:  call from the field comes in with a failure...
:
: A lot of these edge cases are really firmly centered in issues of
: real-time programming.  Few versions of Unix are equipped to deal
: with real-time issues in even a rudimentary fashion.  In any event,
: these cases have very little to do with leap seconds or any other
: aspects of the representation of time quantities.

Actually, they can have a great deal to do with it.  The absolute need
to deal properly with these edge cases can, at times, lead to design
decisions that aren't the 'simplest'.  I also disagree that Unix is
ill-equipted to deal with real-time issues.  While one does need to
take care, I have successfully deployed over a dozen different types
of timing systems over the past 7 years on FreeBSD.  My familiarity
with these edge cases, the customers that expect them to work and the
different sources of time has been learned through these experiences.
The nature of these customers also means that they are less well
connected to networks than one would like, which also complicates
matters.

The salient point from my ramblings is that different time scales may
become available at different points in time because data about them
is available at different points in time to the application/os.  If
there's only one time scale, then once you know the time, you are
done.  If there's more than one, then you need to either discover both
times, or you need to discover one time and the transforms to that
time to know the others.

Warner


Re: Mechanism to provide tai-utc.dat locally

2006-12-28 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Poul-Henning Kamp [EMAIL PROTECTED] writes:
: We can get that only by increasing the DUT tolerance.

Yes.  Letting DUT be bounded by +- 10s rather than +- 0.9s is going to
affect a tiny number of users.  Having leapseconds only known 6 months
in advance affects a lot more users...

: As Warner, I and others have repeatedly emphasized:  It is not the
: step size that is the problem, it is the 6 month warning.
:
: I don't care if you want to implement leap-milliseconds, as long
: as you tell me 10 years in advance when they happen.

While my preference would be to never see another leap second, I know
that's likely not an option.

For many practical reasons, scheduling them out 10 years would help
ameliorate the costs of leap seconds and allow for better long term
planning.

Warner


Re: Mechanism to provide tai-utc.dat locally

2006-12-28 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
John Cowan [EMAIL PROTECTED] writes:
: Rob Seaman scripsit:
:
:  I don't care if you want to implement leap-milliseconds, as long
:  as you tell me 10 years in advance when they happen.
: 
:  Again - with no intent to minimize the issues - what supports this
:  assertion?  Is there any reason to believe that 10 years advance notice
:  would encourage projects and vendors to do anything other than ignore
:  the requirement entirely?  A statement that 10 years, or 600 years,
:  notice is all that is needed to resolve all the problems, smooth over
:  all the complications, is entirely too glib.
:
: You are confusing the question of fixity (which is what notice is
: about) with granularity.  It's true that probably no one would bother
: to implement the ALHP.  However, if computer technologists were handed a
: list of leap seconds from now until 2015, and told Implement these, it
: wouldn't matter how many or how few leap seconds there were.  But since
: you astronomers insist on a fixed maximum for |DUT1|, no such table
: can exist.
:
: The proposal is this:  look at the trends, take your best shot at
: working out a leap-year schedule for 10 years in the future, and then
: live with it.

Let's turn the question around.  What would the harm be if |DUT1| were
1.1s?  1.5s?  2.0s?  Contrast this with the harm and difficulty that
the current 6 month scheduling window affords.

Warner


Re: Mechanism to provide tai-utc.dat locally

2006-12-28 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Rob Seaman [EMAIL PROTECTED] writes:
: M. Warner Losh wrote:
:
:  Let's turn the question around.  What would the harm be if |DUT1| were
:  1.1s?  1.5s?  2.0s?  Contrast this with the harm and difficulty that
:  the current 6 month scheduling window affords.
:
: Indeed.  Go for it.  I look forward to reading your report.  Who and
: what interests are adversely affected in each case?  How are these
: effects mitigated as a function of the limit on DUT1?  Also, contrast
: what benefits accrue.  One would think that the responsibility for
: quantifying the implications of a change to a standard would fall on
: the parties proposing said change.

I know the benefits, but nobody has yet produced a study on why 0.9s
was chosen.  Some vague rumblings about astronomical software needing
to be rewritten, and some vague notions about 'hearty souls' that do
celestial navigation with atomic clocks for high accuracy have been
offered, but who really would be hurt by this change?  Since you are
an astronomer, I thought you might be able to give some insight into
at least one of these users of time.  Daylight savings time and time
zones prove that society at large has a very high tolerance for
variations between the mean solar time at an arbitrary location, maybe
hundreds of miles away, and the local time.  Only specialized users of
time would be affected.  Who are they and how do we find out the cost
of change?

The world has changed from having maybe a few dozen or tens of dozen
atomic clocks when leap seconds started to having GPS with cheap,
disciplined oscillators that number in the hundreds of thousands.
These little devices have obviated the need, in many cases, for
celestial navigation.  Given that change, the cost benefit analysis
that was done in 1972 likely needs updating.

Warner


Re: Mechanism to provide tai-utc.dat locally

2006-12-25 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Zefram [EMAIL PROTECTED] writes:
: Keith Winstein wrote:
:   what should a library do when a program asks to
: convert between UTC and TAI for some instant further than six months in
: the future?
:
: Refuse, of course.  The correct answer is I don't know.  You might
: know if asked later.

Actually, the real answer is domain specific.  There are many things
that don't really care (when do I need to make the next 20 house
payments, off by one second just doesn't matter at all since
transcations are batched on a daily basis a few days early).  There
are other things that do care (when do I fire the lasers at the
target).  For those things, however, it is rare to be computing time
into the future that far.

Writing a library that copes with both types of users can be
problematical at best...

Warner


Re: Mechanism to provide tai-utc.dat locally

2006-12-25 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Zefram [EMAIL PROTECTED] writes:
: Steve Allen wrote:
:  and if I continue that practice
: I can later give you an estimate of how wrong I was when I told you.
:
: This is something that's missing from current chronological APIs.
: It needs to be formalised.

time_t is so totally broken, it isn't funny.  That's the closest thing
to a standardized API there is for time.  All others are stuff folks
have done here or there, but they aren't universal enough to be
considered.

Too bad the problems with time_t are well known, well discussed and
well enumerated.  Or rather I should say too bad POSIX doesn't care
enough to change it since the cost of changing time_t is huge...

Also, things in TAI time don't care either. 1 day, 1 year, 100 years
don't matter to TAI.  Periodic things that happen at a given time of
day (UTC) are the only things that do care.

Warner


Re: Mechanism to provide tai-utc.dat locally

2006-12-24 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Ashley Yakeley [EMAIL PROTECTED] writes:
: Hello, I just joined the leap seconds list. I wrote the time
: package for the Haskell programming language.
: http://semantic.org/TimeLib/
:
: I include code for making conversions between TAI and UTC, given a
: leap-second table. I also include code for parsing a tai-utc.dat file
: into a leap-second table. I do not, however, simply include a leap-
: second table as any program compiled with one would be out of date
: after six months.
:
: This has led me to consider run-time methods of obtaining leap-second
: information, and how that might be standardised for use by software
: authors. For instance, I imagine a software package that established
: a well-known place in the directory hierarchy to find tai-utc.dat and
: perhaps other earth data files, and was responsible for keeping
: them up to date. Other software could then make use of the package
: without each having to implement their own mechanism.
:
: Does this strike people as worthwhile? Does anyone know of an
: existing effort along these lines?

One can find the leap data from NIST that's up to date:
ftp://time.nist.org/leap-seconds.list
I believe that there are other places for this information as well.

The problem is that often times one has systems that aren't guarnateed
to be connected to the internet, but are connected to GPS so that this
number can be obtained.  Other problems arise from systems that are
intermittently operational (like in sparing situations) that may be
off over many leap seconds...

Warner


Re: what time is it, legally?

2006-12-12 Thread M. Warner Losh
I view the same data differently.  I see it as a progression:

Local Solar time - mean local solar time -
  timezone as mean local time at one point used for many - UTC - ???

Clearly, we're moving away from solar time and towards something else.
Our ability to tell time has exceeded the earth's ability to be
perfectly periodic.  The time will come when we need to change
something as the current system is doomed to failure, sooner or
later.  I feel that attempts to tie time to the sun and to also have a
precice notion of a second are fundamentally at odds with one another
and the current system tries very hard to paper over the differences.
No doubt they can be papered over for years to come, but the basic
physics of platentary rotation with a large satellite stand against
it.  I'm unsure what that something else should be, and I'm not sure
that we're in a good position to know with certainty what would be a
good solution.  I just know that invoking variable radix counting
schemes has its limitations and difficulties...

Anyway, I know others view it differently, but if it was easy and
obvious there wouldn't be such a divergence of opinion, eh?

Warner


Re: independence day

2006-07-04 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Poul-Henning Kamp [EMAIL PROTECTED] writes:
: In message [EMAIL PROTECTED], Steve Allen writes:
:
: In the middle of May some text about legal time in the US was
: introduced into a US Senate bill regarding funding for NSF and NIST.
: See section 508 of S.2802 introduced 2006-05-15, e.g.
: http://thomas.loc.gov/cgi-bin/query/z?c109:S.2802:
:
: `(b) COORDINATED UNIVERSAL TIME DEFINED- In this section,
: the term `Coordinated Universal Time' means the time scale
: maintained through the General Conference of Weights and
: Measures and interpreted or modified for the United States
: by the Secretary of Commerce.'.
:
: That could sound like the drilling of a loophole :-)

As has been pointed out in the past, the Secretary of Commerce has had
the ability to define mean solar time to mean UTC (or something else,
if they felt the urge)...  I think this is just another attempt to
keep their options open, like they have now...

Warner


Re: building consensus

2006-06-05 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
John Cowan [EMAIL PROTECTED] writes:
: M. Warner Losh scripsit:
:
:  : The designers of Posix time thought it was more important to preserve
:  : the property that dividing the difference between two time_t values
:  : by 60, 3600, 86400 would give minutes, hours, days.
: 
:  That's the one property that Posix time_t does not have.  The
:  difference between time_t's that cross a leap second are off by one
:  second, and therefore do not start with the right answer to do the
:  division...
:
: I expressed myself badly.  My point is that if you have a Posix time_t
: representing 11:22:33 UTC on a certain day, and you add 86400 to that
: time_t, you will get the Posix representation of 11:22:33 UTC on the
: following day, whether a leap second intervenes or not.  This is a valuable
: property, many existing programs depended on it, and the authors of the
: Posix spec preserved it at the expense of having a distinct representation
: for each UTC second.

Yes.  To find a second absolute time that represents the same wall
time a minute/hour/day later is why posix time_t has this property.
This is abstime + delta, which is a little different than the
difference between two time_t.

: You may call this position wrong (and I have done so), but it is
: unquestionably defensible.

Differences in time_t are adjusted by leapseconds.  This makes naive
math work for the ABSTIME+delta case, but breaks the difference case
when you want an actual elapased time.  It is an engineering tradeoff,
but an inconvenient one for the problem domains that I tend to have to
solve.

Warner


Re: building consensus

2006-06-04 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
John Cowan [EMAIL PROTECTED] writes:
: Zefram scripsit:
:
:  If this means that leap secounds and leap days are analogous, then I
:  suppose so.  If it means something else, I don't understand it.
: 
:  That's what I meant.  Can you suggest a clearer wording?
:
: Leap seconds (after 1972) are closely analogous to leap days.

Leap seconds (after 1972) are closely analogous to irregularlly
 happening leap days.

might be better, since leap days have a rule, while leap seconds are
scheduled.

:  Being ambiguous between adjacent seconds seems inherently faulty to me.
:
: The designers of Posix time thought it was more important to preserve
: the property that dividing the difference between two time_t values
: by 60, 3600, 86400 would give minutes, hours, days.

That's the one property that Posix time_t does not have.  The
difference between time_t's that cross a leap second are off by one
second, and therefore do not start with the right answer to do the
division...

POSIX time_t definition effecitvely omits leap seconds from the time
scale (by playing them twice, or pretending the second happened), the
difference between two time_t's always gives a duration adjusted for
the leap seconds that happen, rather than the actual duration.

:  Are you thinking of linear counts such as POSIX time, where the
:  representation is ambiguous?  I was implicitly excluding those, on the
:  grounds that they don't count as a representation.  It's also not
:  linear.
:
: No, it isn't.  But that doesn't mean you *can't* construct a numerical
: representation of UTC time: say, the number of UTC seconds since
: 1972-01-01T00:00:00Z.

It would be better to say the number of SI seconds since 1972 rather
than UTC seconds, I think.  I use a timescale like this at work to
ensure that duration calcuations over leap secondsd are correct.

Warner


Re: building consensus

2006-06-01 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Rob Seaman [EMAIL PROTECTED] writes:
: Actually, this list is not a discussion per se.  If we simplify the
: positions - just for the sake of argument here - to leap second yes
: and leap second no, the reality is that the folks pushing the leap
: second no position have never engaged with this list.  There are
: several doughty people here who happen to have that opinion, but they
: abide with us mortals outside the time lords' hushed inner sanctum.

What an amaizingly unhelpful and offsensive statement.  I have spent
much time explaining why leap seconds cause real problems in real
applications, only to be insulted like this.

Warner


Re: building consensus

2006-06-01 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
John Cowan [EMAIL PROTECTED] writes:
: M. Warner Losh scripsit:
:
:  In message: [EMAIL PROTECTED]
:  Rob Seaman [EMAIL PROTECTED] writes:
:  : Actually, this list is not a discussion per se.  If we simplify the
:  : positions - just for the sake of argument here - to leap second yes
:  : and leap second no, the reality is that the folks pushing the leap
:  : second no position have never engaged with this list.  There are
:  : several doughty people here who happen to have that opinion, but they
:  : abide with us mortals outside the time lords' hushed inner sanctum.
: 
:  What an amaizingly unhelpful and offsensive statement.  I have spent
:  much time explaining why leap seconds cause real problems in real
:  applications, only to be insulted like this.
:
: I believe you have misread Rob's remark, though I concede that it was
: easy to misread.  I believe Rob meant that the people who are pushing
: leap seconds no in *official* channels are not to be found on this list.
: That being so, the leap seconds yes folks are unable to challenge them
: or persuade them otherwise.
:
: You and I, on the other hand, fall into the doughty people here group.

Maybe I did misread them.  I've been sick the past three days, so
maybe we can chalk it up to that and I'll offer my oppologies for
having such a thin skin.

Wanrer


Re: building consensus

2006-06-01 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Rob Seaman [EMAIL PROTECTED] writes:
: Warner Losh objects:
:
:  There are several doughty people here who happen to have that
:  opinion, but they abide with us mortals outside the time lords'
:  hushed inner sanctum.
: 
:  I have spent much time explaining why leap seconds cause real
:  problems in real applications, only to be insulted like this.
:
: Sincere apologies for my awkward statement.  Dictionary.com defines
: doughty as marked by stouthearted courage; brave.  I wasn't
: questioning the knowledge or passion of folks holding views that
: differ from my own.  Rather I was attempting to question whether
: anybody actively participating on this list - holding whatever view -
: is also participating in ITU discussions.
:
: I see that Mr. Cowan has also parsed my admittedly opaque remarks.

Yes.  I'm sorry I was so easily offended.  Please accept my appologies
for my hasty words.

Warner


Re: ideas for new UTC rules

2006-04-14 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Rob Seaman [EMAIL PROTECTED] writes:
: Simply allow the IERS to announce any number of leap seconds
: in advance extending over any time horizon - and yes - occurring
: at the end of any month.  If predictability is the goal, relaxing
: unnecessary constraints is the solution.

There's a lot of timing gear that will fail if leap second is inserted
not at the end of June or December, not least ntp.

Warner


Re: An immodest proposal

2006-02-14 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Neal McBurnett [EMAIL PROTECTED] writes:
: On Mon, Feb 06, 2006 at 10:37:37AM -0700, Rob Seaman wrote:
: 1)  TAI can be recovered from UTC given a table of DTAI.
: 2)  NTP can convey TAI as simply as UTC.
: 3)  Deploy a small network of NTP servers to keep TAI, not UTC.
: 4)  NTP client machines could therefore trivially select between TAI and
: UTC by subscribing to different servers.
: 5)  This would provide an unbiased experimental sandbox for civil
: timekeeping issues.
:
: Since TAI can be recovered from UTC, and hosts that use NTP know UTC,
: those hosts can serve any time they want to clients on that host.  I
: see little reason to change the particular timestamps that are used in
: the NTP protocol, an less reason to have two different timestamps
: within NTP, or to require users on ntp-using hosts to switch their
: ntp servers just to get a different timescale.

ntp time stampes are ambiguous at leap seconds (the leap indicator
bits aren't part of the time stamp, exactly), which is a good reason
to change them.  However, the cost to change them could be quite high
if done in an incompatible manner.

: That's why I support the Fixing POSIX time proposal from
: Poul-Henning Kamp.  This can be implemented with the latest NTP
: daemons.  The operating systems would seem to just need the right sort
: of CLOCK_NTP definition for /usr/include/linux/time.h or equivalent,
: and the appropriate conversion routines as he outlined.

Agreed.

: I'd like to see a standard for NTP that includes the TAI information
: that is now passed in non-standard formats, but that is a separable
: issue.

I believe that the new draft standard of ntp uses TAI time for its
time exchange, and explicitly includes DTAI information in a standard
format.  Since the protocol version changes, there's no ambiguity and
that's a good time to change things like this.  However, I've not read
the latest ntpng drafts to confirm this.

Warner


Re: An immodest proposal

2006-02-14 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Rob Seaman [EMAIL PROTECTED] writes:
: On Feb 14, 2006, at 12:50 PM, Markus Kuhn wrote:
:
:  You can, of course, define, publish, implement, and promote a new
:  version (4?) of NTP that can also diseminate TAI, EOPs, leap-second
:  tables, and other good things. I'm all for it.
:
: But why are you for it?  Before investing large amounts of time and
: money in developing and deploying a large new timekeeping system,
: wouldn't one want to invest smaller amounts in exploring the issues
: and options?  Heck - one has to imagine that a number of successful
: grant applications are lurking around here somewhere.  Time is an
: issue that cuts across every funding agency out there.

UTC time stamps in NTP are ambiguous.  TAI ones are not.  UTC time
stamps do not convey enough information to properly implement things
like intervals, while TAI ones do.  The NTPNG stuff that I've seen
appears to consider these problems as worthy of needing a solution and
they plan on solving them.  It isn't rocket science, but one has to
divorce ones self from the chauvinistic view that UTC is always best.
For time exchange, it is not the best, and has many problems around
the edges.

Doing NTP with TAI (and the implied requirement for DTAI) doesn't
change what time is displayed for users.  It does make it *MUCH*
easier to get leap seconds right for those users that need them.
Anything else is madness.  UTC is a display convention, not a sane[*]
counting convention.

Warner

[*] All variable radix counting conventions are insane by (my humble)
definition :-).


Re: An immodest proposal

2006-02-14 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Neal McBurnett [EMAIL PROTECTED] writes:
:  UTC time stamps in NTP are ambiguous.  TAI ones are not.  UTC time
:  stamps do not convey enough information to properly implement things
:  like intervals, while TAI ones do.  The NTPNG stuff that I've seen
:  appears to consider these problems as worthy of needing a solution and
:  they plan on solving them.  It isn't rocket science, but one has to
:  divorce ones self from the chauvinistic view that UTC is always best.
:  For time exchange, it is not the best, and has many problems around
:  the edges.
:
: Yes it is messy, but the tradeoffs are complex.  I don't think the
: latest drafts specify NTP timestamps.  I suspect they still rely on
: the leap second bit to disambiguate the timestamp on a leap second,
: but I haven't checked recently.  They are all linked to from the
: charter page I noted above.

But ntp time stamps to not have a leap second bit.  To be precice, it
is a leap indicator that's in the packet.  This makes it impossible to
represent times that straddle the leap event, since there are 4
different time stamps involved...  I guess in those cases, the packets
should be discarded and tried again later...

But you are right.  I did misremeber that it used TAI time.  The
drafts still specify time in UTC.  I could have sworn I saw something
that said otherwise, but I can't seem to find it now.

:  Doing NTP with TAI (and the implied requirement for DTAI) doesn't
:  change what time is displayed for users.  It does make it *MUCH*
:  easier to get leap seconds right for those users that need them.
:  Anything else is madness.  UTC is a display convention, not a sane[*]
:  counting convention.
:
: I think that changing to a different over-the-wire timestamp epoch
: would just add to the confusion, not make things simpler in practice.

TAI is the same epoch as UTC.  It is just not adjusted for leap
events.  However, your point that the second numbering would be
different and might cause confusion is still valid.

: People still need to know UTC, and transmitting the leap second table
: info, especially via autokey, is much more complex than the basic
: protocol.  But at least this is a standards process conducted in the
: open, where you can just get involved directly if you have something
: to add.

Autokey is interesting, but it is way too complex for something as
simple as leap second propigation

Warner


Re: Comparing Time Scales

2006-02-04 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
M. Warner Losh [EMAIL PROTECTED] writes:
: In message: [EMAIL PROTECTED]
: James Maynard [EMAIL PROTECTED] writes:
: : M. Warner Losh wrote:
: :  In message: [EMAIL PROTECTED]
: :  James Maynard [EMAIL PROTECTED] writes:
: :  : Thanks, guys, for your feedback. Here's another iteration.
: :  :
: :  : The numbering of NTP seconds in the vicinity of a leap second seems to
: :  : differ from one document to another.  Here I follow the NTP (version 3)
: :  : specification, RFC 1305, in which the Leap Indicator (sys.leap,
: :  : peer.leap, pkt.leap) is 01 if a positive leap second is to occur at the
: :  : end of the current UTC day, and 00 if no leap second is pending.
: :  :
: :  : UTC = 2005-12-31 23:59:59, NTP seconds = 3 345 062 399, LS_pending = 01
: :  : UTC = 2005-12-31 23:59:60, NTP seconds = 3 345 062 400, LS_pending = 01
: :  : UTC = 2006-01-01 00:00:00, NTP seconds = 3 345 062 400, LS_pending = 00
: : 
: : 
: : 
: : In the NTP time scale, it is the last second of the UTC day, NTP second
: : number 3 124 137 600, that is repeated. The first instance of this NTP
: : second number is the leap second, during which (TAI - UTC) = 31 and the
: : NTP Leap Indicator is 01.  The second instance of this NTP second number
: : is the the first second of the new UTC day, during which (TAI - UTC) =
: : 32 and the NTP leap indicator is 00.  You can distinguish the leap
: : second (while it is occuring) by the fact that it is the last second of
: : the UTC day (time of UTC day = 23:59:60) and that the NTP Leap Indicator
: : is 01. You can distinguish the first second of the new UTC day by the
: : fact that, of the two seconds with the same NTP second number, it is the
: : one that has the NTP leap indicator set back to 00.
: :
: : Note that according to the write-up to which you referred me, it is
: : NTP second number 3,124,137,600 (not 3,124,137,599) that is repeated.
: : NTP second number 3,124,137,599 is the *next-to-last* second of the day,
: : not the *last* second of the day.)
:
: No.  The article specifically says that after it the system time gets
: to ,600, it is decremented by one and there is specific code in the
: code that returns the system time to applications that makes it stand
: still.  The second isn't *NOT* repeated.  Repeat: The second is *NOT*
: repeated in what they said.  Time stands still outside of the kernel,
: while inside the kernel the last second of the day *IS* repeated,
: hence the need for the limiter that the article talks about.
:
: Not all kernels keep time standing nearly still during the leap second
: (since that has other bad effects).  Some expose this decrement to the
: users.  The highlighed part of what I quoted said exactly this.
:
: I've actually implemented this for FreeBSD.  You are arguing theory,
: and I'm arguing the fine points of an actual, real implementation.

More specifically, read the text I highlighted:

: : The obvious question raised by this scenario is what happens during
: : the leap second itself when NTP time stops and the clock remains
: : unchanged. If the precision time kernel modifications have been
: : implemented, the kernel includes a state machine that implements the
: : actions required by the scenario. [[ At the first occurrence of second
-
: : 3,124,137,600, the system clock is stepped backward one
   ---
: : second.]] However, the routine that actually reads the clock is
   ---
: : constrained never to step backwards, unless the step is significantly
: : larger than one second, which might occur due to explicit operator
: : direction. In this design time stands still during the leap second,
: : but is correct commencing with the next second.

Notice the [[ ]] text.  When it becomes 400, the system time is
decremented internally, which is why I wrote the 399.5 example that I
did.  Also, the picture on the page shows how time stops/slows way
down during the leap second in Dave Mills' view of how time should be
returned to the user during the leap second.  Text I didn't quote says
that the system time must be monotonically increasing, so it is
incremented by 1ns each call.

So internally to the kernel, the time does repeat the last second.
Externally, in a Mills' perfect world, time stands still during the
leap second.  The second doesn't repeat, per se, but it freezes for a
second.

In FreeBSD, the actual internal time is exposed to the user.  If we
froze time as returned from the system, then threaded applications'
threads would potentially stop during the leap second due to the way
certain timeouts are handled.  This is a bigger problem, so we keep
time running by exposing the internal state.

These sorts of details, btw, are why I hate leapseconds.  They cause
needless arguments over which part of the system you are talking
about

Re: Comparing Time Scales

2006-02-04 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Tim Shepard [EMAIL PROTECTED] writes:
: But there's a difference between NTP timestamps, and the details of
: the implementation of a system which uses NTP for synchronization.

Ah, I was getting the two confused and didn't quite realize it until
your clarification.  Thanks!

Warner


Re: Comparing Time Scales

2006-02-03 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
James Maynard [EMAIL PROTECTED] writes:
: Thanks, guys, for your feedback. Here's another iteration.
:
: The numbering of NTP seconds in the vicinity of a leap second seems to
: differ from one document to another.  Here I follow the NTP (version 3)
: specification, RFC 1305, in which the Leap Indicator (sys.leap,
: peer.leap, pkt.leap) is 01 if a positive leap second is to occur at the
: end of the current UTC day, and 00 if no leap second is pending.
:
: UTC = 2005-12-31 23:59:59, NTP seconds = 3 345 062 399, LS_pending = 01
: UTC = 2005-12-31 23:59:60, NTP seconds = 3 345 062 400, LS_pending = 01
: UTC = 2006-01-01 00:00:00, NTP seconds = 3 345 062 400, LS_pending = 00


 The obvious question raised by this scenario is what happens during
 the leap second itself when NTP time stops and the clock remains
 unchanged. If the precision time kernel modifications have been
 implemented, the kernel includes a state machine that implements the
 actions required by the scenario. [[ At the first occurrence of second
 3,124,137,600, the system clock is stepped backward one
 second.]] However, the routine that actually reads the clock is
 constrained never to step backwards, unless the step is significantly
 larger than one second, which might occur due to explicit operator
 direction. In this design time stands still during the leap second,
 but is correct commencing with the next second.

So internally, once we hit the leap second time, we step backwards.
Some systems contrain the values returned from gettimeofday to not
notice the step, while others do not.  This talks about incrementing
the time by 1 unit (usually a nanosecond or microsecond) while time is
catching up.

http://www.eecis.udel.edu/~mills/leap.html

So it very much depends on if you are looking inside the kernel or
outside the kernel for what the correct values are.  If you freeze
time, then doing a time exchange during a leap second is going to give
bogus results.  If you step it back, then it will give good results,
but there are other bad effects.

Basically, no matter what you do, it is broken (unless you tick in TAI
time).

Warner


Re: Comparing Time Scales

2006-02-03 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
James Maynard [EMAIL PROTECTED] writes:
: M. Warner Losh wrote:
:  In message: [EMAIL PROTECTED]
:  James Maynard [EMAIL PROTECTED] writes:
:  : Thanks, guys, for your feedback. Here's another iteration.
:  :
:  : The numbering of NTP seconds in the vicinity of a leap second seems to
:  : differ from one document to another.  Here I follow the NTP (version 3)
:  : specification, RFC 1305, in which the Leap Indicator (sys.leap,
:  : peer.leap, pkt.leap) is 01 if a positive leap second is to occur at the
:  : end of the current UTC day, and 00 if no leap second is pending.
:  :
:  : UTC = 2005-12-31 23:59:59, NTP seconds = 3 345 062 399, LS_pending = 01
:  : UTC = 2005-12-31 23:59:60, NTP seconds = 3 345 062 400, LS_pending = 01
:  : UTC = 2006-01-01 00:00:00, NTP seconds = 3 345 062 400, LS_pending = 00
: 
: 
: 
: The obvious question raised by this scenario is what happens during
: the leap second itself when NTP time stops and the clock remains
: unchanged. If the precision time kernel modifications have been
: implemented, the kernel includes a state machine that implements the
: actions required by the scenario. [[ At the first occurrence of second
: 3,124,137,600, the system clock is stepped backward one
: second.]] However, the routine that actually reads the clock is
: constrained never to step backwards, unless the step is significantly
: larger than one second, which might occur due to explicit operator
: direction. In this design time stands still during the leap second,
: but is correct commencing with the next second.
: 
: 
:  So internally, once we hit the leap second time, we step backwards.
:  Some systems contrain the values returned from gettimeofday to not
:  notice the step, while others do not.  This talks about incrementing
:  the time by 1 unit (usually a nanosecond or microsecond) while time is
:  catching up.
: 
:  http://www.eecis.udel.edu/~mills/leap.html
: 
:  So it very much depends on if you are looking inside the kernel or
:  outside the kernel for what the correct values are.  If you freeze
:  time, then doing a time exchange during a leap second is going to give
:  bogus results.  If you step it back, then it will give good results,
:  but there are other bad effects.
: 
:  Basically, no matter what you do, it is broken (unless you tick in TAI
:  time).
: 
:  Warner
: 
: 
: 
: Thank you. You are trying hard to educate me on this matter, and I
: appreciate your efforts.
:
: Following the link you gave me in this message, , I found a discussion
: of how the leap second is to be handled in the NTP time scale. That
: discussion includes the following table to illustrate the leap second
: transition of 1998-12-31 23:59:60 UTC:
:
: +---+--++--+---+
: |   Date|   Time   | TAI Offset | NTP Leap |  NTP Seconds  |
: +---+--++--+---+
: | 31 Dec 98 | 23:59:59 | 31 |01| 3,124,137,599 |
: +---+--++--+---+
: |   | 23:59:60 | 31 |01| 3,124,137,600 |
: +---+--++--+---+
: |  1 Jan 99 | 00:00:00 | 32 |00| 3,124,137,600 |
: +---+--++--+---+
: |   | 00:00:01 | 32 |00| 3,124,137,601 |
: +---+--++--+---+
:
: Funny, that's exactly what I wrote in my last iteration of my table!
:
: In the NTP time scale, it is the last second of the UTC day, NTP second
: number 3 124 137 600, that is repeated. The first instance of this NTP
: second number is the leap second, during which (TAI - UTC) = 31 and the
: NTP Leap Indicator is 01.  The second instance of this NTP second number
: is the the first second of the new UTC day, during which (TAI - UTC) =
: 32 and the NTP leap indicator is 00.  You can distinguish the leap
: second (while it is occuring) by the fact that it is the last second of
: the UTC day (time of UTC day = 23:59:60) and that the NTP Leap Indicator
: is 01. You can distinguish the first second of the new UTC day by the
: fact that, of the two seconds with the same NTP second number, it is the
: one that has the NTP leap indicator set back to 00.
:
: Note that according to the write-up to which you referred me, it is
: NTP second number 3,124,137,600 (not 3,124,137,599) that is repeated.
: NTP second number 3,124,137,599 is the *next-to-last* second of the day,
: not the *last* second of the day.)

No.  The article specifically says that after it the system time gets
to ,600, it is decremented by one and there is specific code in the
code that returns the system time to applications that makes it stand
still.  The second isn't *NOT* repeated.  Repeat: The second is *NOT*
repeated in what they said.  Time stands still outside of the kernel,
while inside the kernel the last

Re: wikipedia Leap Seconds collaboration

2006-01-23 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Neal McBurnett [EMAIL PROTECTED] writes:
:  Rob Seaman wrote:
:  I hope we can all continue this discussion in a more positive manner.
:
: It is the nature of email lists to be good at stimulating discussion,
: and bad at generating clear resolutions.  Thus was the FAQ born.  But
: we have a more modern technology than FAQs, the wiki, which can more
: effectively funnel passionate energy from groups of people with
: diverging ideas into coherent descriptions of a variety of viewpoints,
: suitable for enlightening the world.  Imperfectly, to be sure, but
: better than a mail list
:
: I think the thing we need to do is build on what clarity we can find
: in the moment, and document it at wikipedia.  If the folks discussing
: the Jesus article can arrive at a relatively stable set of positions
: (and last time I looked, they had done remarkably well, considering),
: surely we can also.
:
: Note the relatively successful policy of presenting things from a
: Neutral Pointof View (NPOV):
:
:   http://en.wikipedia.org/wiki/Wikipedia:Neutral_point_of_view
:
: So would folks be willing to collaborate at
:
:  http://en.wikipedia.org/wiki/Leap_second
:
: and related pages?

I've contributed to the unix time over leap seconds pages, and would
be happy to help.  I feel I can write a good argument for both sides,
even though I have my preferences.

Warner


Re: the tail wags the dog

2006-01-23 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Steve Allen [EMAIL PROTECTED] writes:
: The legal time of the US is (in many more words) GMT.
: The officials who are charged by congress with the task of providing
: time provide UTC.

The legal time in the US is the mean solar time at a given meridian,
as determined by the secretary of commerce (the actual law is a little
more verbose than this, but this is an accurate boil down) plus some
weird options for 'border states' which timezone they are in.  This is
why NIST provides UTC and leap seconds happen on the UTC schedule
rather than some other schedule that would produce the same results.
It is also why there are leap seconds and not the old-style frequency
adjustments + tiny steps.  Both of these schemes fit the law, as it is
rather vague in the words it uses in a legal sense (the term mean
solar time isn't legally defined, but does have an accepted scientific
meaning).  Other schemes could also fit the law that aren't UTC today
since there's no what we would call 'DUT1 tolerance' written into the
law...

Warner


Re: NOT A cruel fraud!

2006-01-22 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Ed Davies [EMAIL PROTECTED] writes:
: M. Warner Losh wrote:
:  :   TAI is specifically contraindicated as a time
:  :  scale.
:  :
:  :   TAI is not currently recommended by its creators as a viable time
:  :   scale.
:  :  
:  :
:  : These claims are intellectually fraudulent. The archives in fact support
:  : the opposite of what Mr. Losh contends.
: 
:  Actually, it isn't quite that cut and dried.
:
: OK, so why is TAI contraindicated then?
:
: I've been on this list since 2002-07 and not yet seen a good
: argument against it.

The short answer is that you cannot get a time feed of TAI, so the
high precision folks don't like people calling something that's
derived indirectly TAI.  TAI(UTC(foo)) can be derived, however, if
you know the current leap offset.  This isn't broadcast in many time
formats.

Having said that, our company uses TAI internally for all its time
code and translates on output.  Doing that doesn't free you from
knowing about leap seconds, however, since you have to know about them
to convert to/from UTC which is universally broadcast or available.
GPS makes things a little easier, since there's a fixed offset from
TAI, but even it present difficulties because you need the current
leap offset to display UTC times.  However, some GPS receivers insist
on doing silly things when you try to get the raw time without
leapseconds.  Our systems need to have unambiguous timestamps, as well
as a requirement to display UTC.

Warner


Re: NOT A cruel fraud!

2006-01-22 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Ed Davies [EMAIL PROTECTED] writes:
: Earlier, I wrote:
:  We all know that it (and any other world-wide timescale) is
:  postal at the level of the time it takes light to cross a
:  moderately small room but for microsecond precision and looser
:  this is not an issue.
:
: I ought to qualify that.  There are, of course, time scales which
: are synchronized across the world with much better than microsecond
: precision (e.g., GPS time) but it's my understanding that they are
: not anything like as uniform as TAI.  Is this right?  Can anybody
: quote accuracy information for GPS time?  At what level of precision
: do time transfers using GPS time need out-of-band (postal)
: correction for uniformity?

GPS is synchronized to UTC(USNO) which agrees with UTC(NIST) to within
a few nanoseconds.  GPS receivers can recover UTC(USNO) to within
about a few nanoseconds (as measured by a direct two way time exchange
with NIST), although not all of them are that quality.  The rms of a
good timing receiver is on the order of nanoseconds.  NIST and USNO
coordinate with each other to make sure there's close agreement in the
two minor variations of UTC.  They also participate in the computation
of TAI.

TAI is computed after the fact based on the observed time of different
frequency standards.  Phase offsets between different time sources are
measured, the data is decimated and placed into a particular format
and sent to BIPM.  These measurements are sent in batches and used to
measure interesting effects.

Here are some reports on how the various labs measure time relative to
a known source:

http://www.bipm.fr/cc/CCTF/Allowed/16/cctf04-18.pdf
http://www.bipm.org/en/publications/scientif/tai.html

Warner


Re: NOT A cruel fraud!

2006-01-22 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Peter Bunclark [EMAIL PROTECTED] writes:
: On Sun, 22 Jan 2006, M. Warner Losh wrote:
:  The short answer is that you cannot get a time feed of TAI, so the
: 
: So isn't this one of the things we want to fix in the brave new world of
: joined-up timekeeping? Distribute (very close to) TAI, keep the kernel
: PLLs sweet, move leap second handling to user-space and thus make
: debugging very easy, then everone can get their timescale of choice as
: a f(TAI)?

Yes.  That's a fair summary.

Warner


Re: NOT A cruel fraud!

2006-01-22 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
James Maynard [EMAIL PROTECTED] writes:
: Mr. Losh and I have apologized to each other, off list.  I think we
: should now retire the cruel fraud subject line.

Yes.  I was too hard on Mr. Maynard and I appologize for my rough
treatment of him here.  It was really over the line, and I'm sorry for
my ill considered words.

Warner


Re: the tail wags the dog

2006-01-22 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
[EMAIL PROTECTED] (Michael Sokolov) writes:
: Steve Allen [EMAIL PROTECTED] wrote:
:
:  The CGPM recommendation on the timescale everyone should use says UTC.
: 
:  UTC(insert your national time service here) is available in real time.
: 
:  TAI is the mathematical (really the political or diplomatic) entity
:  upon which UTC is ostensibly based, but the practical and legal
:  reality is the other way around.
:
: Has it occurred to any of you that *THIS* is the very root of the problem,
: that *THIS* is what we need to change?

I believe so.

: Also, isn't TAI readily available in real time from GPS?  (How difficult
: can it be for the routine parsing the data stream from the GPS receiver
: to add 19?)  OK, OK, it'll be TAI(GPS) rather than true TAI, but then
: your UTC is really UTC(NIST) or UTC(USNO) or UTC(PTB) or whatever rather
: than true UTC.

You can get an excellent approximation of TAI from GPS, assuming that
you use a sane receiver...  Some firmware doesn't deal well with
reporting the raw, uncorrected GPS time, but many do it well.
Depending on your needs, you may have to put the GPS receiver into a
mode that only reports URC...  Well, you get the idea.  It can be done
to a degree of accuracy that most people will accept, but the
post-processed time scale calculations are a little better than what
happens in real time.

Warner


Re: Risks of change to UTC

2006-01-21 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
James Maynard [EMAIL PROTECTED] writes:
: M. Warner Losh wrote:
:  UTC works for navigation, but leap seconds pose problems for other
:  users of time.  Stating absolutely that UTC is not broken ignores
:  these other users.
:
: Those other uses, for whom leap seconds pose a problem, should be
: using a time scale that does not have leap seconds. They would be better
: served, for example, by TAI.

You really should read the archives of this list.  We've been over
this in great detail.  TAI is specifically contraindicated as a time
scale.  This also ignores the issues surrounding the proper
computation of elapsed time using the UTC time scale when represnted
by a number.  It ignores the fact that you must have a table of leap
seconds to do this effectively.  It ignores that one must have a table
of leap seconds to get TAI time, because most providers of time
provide it in UTC.

: For civil use, a calendar that counts days reaonably accurately is
: appropriate.  The Gregorian (New Style calendar) that the vast
: majority of the planetary population uses does this. UTC copes with
: the variable length of the mean solar day by inserting leap seconds
: as needed.  The role of the IERS in decreeing when leap seconds are
: needed is similar to that of the Roman College of Pontifices who managed
: the old Roman Republican calendar (before Julius Caesar's reform) by
: decreeing, as needed, when to shorten the month of February and insert
: the intercalary month of Mercurius.

I maintain that for human activity, there's no need for leap seconds
at all.  In each person's lifetime, the accumulated error is on the
order of a few minutes.  Over generations, the problems with noon
drifting to 1pm can trivially be solved by moving the timezones that
civilian time uses.

Keeping universal time synchronized to an arbitrary meridian is
already arbitrary.  Adding leap seconds is an arbitrary decision to
implement that arbitrary requirement.  It is really only important for
those uses of time that care what the local solar time is to within a
second (as opposed to within an hour which everyone in civil society
is used to since the late 19th century and time zones).

: If your primary need is for a time scale ithat counts SI seconds, with
: no leap seconds to confuse the matter, then don't use UTC. Use a time
: scale that counts SI seconds, such as TAI or GPS time. There's no point
: to applying the mised radix Gregorian calendar system to such a time
: scale, although you can do so if you wish.  Count days of 86 400 SI
: seconds each, or GPS weeks of 604 800 SI seconds, or just count SI seconds.
:
: If, on the other hand, you need to count solar days, or mean solar days,
: use a calendar and time scale that does so.  In order to know which way
: the earth is pointing, use UT1, or a compromise scale such as UTC that
:   is kept reasonably close to UT1.  For the vast majority of the
: population of the planet, including celestial navigators, UTC is good
: enough. If you want to know the direction the earth is pointing with
: more precision, apply DUT1 corrections, or use other IERS products such
: as Bulletin A.

That is one solution to the problem, but it is not the only solution
to the problem.  Quoting the old, tired saws about using TAI or GPS
time scales to count SI seconds doesn't promote good dialog.  Instead,
it drags this discussion down to the level of Dogma.

There are many users that need to both count SI seconds, as well as
synchronize their operations to civilian time.  Leap seconds cause
these users real difficulties.  Implementing leap seconds in software
is hard to get pedantically right (I have a pile of bugs in code that
was written by smart people that get leap seconds wrong, and over 200
hours of time spent in the past few years on implement, testing and
debugging leap seconds).  Even many ntp servers on the net got the
leap second wrong, as did many of the time stations.  There's much
evidence that this solution to the problem has shifted the cost of
keeping astronomers, celestial navigators, and others with a real need
to the solar time happy to a large number of other users who have no
such real need.  As such, it seems only fair in today's economy to
place the costs of the needs of a community on those communities that
have a real need for those goods or services.

: There is no need to fix the time scale, UTC, that is used by the vast
: majority of the planet's population to accommodate the very small
: minority of precision time users who desire a time scale that has no
: leap seconds. Let that minority use a time scale, such as TAI, that does
: not have those messy leap seconds.

TAI is not currently recommended by its creators as a viable time
scale.

In my day job, we do use TAI internally.  That doesn't solve the
problems of leap seconds.  It doesn't solve the problem of products
that need to participate in leap seconds (say irig generators or ntp
servers

Re: Risks of change to UTC

2006-01-21 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Daniel R. Tobias [EMAIL PROTECTED] writes:
: On 21 Jan 2006 at 10:11, M. Warner Losh wrote:
:
:  I maintain that for human activity, there's no need for leap seconds
:  at all.  In each person's lifetime, the accumulated error is on the
:  order of a few minutes.  Over generations, the problems with noon
:  drifting to 1pm can trivially be solved by moving the timezones that
:  civilian time uses.
:
: What about when that accumulated difference is over 24 hours, so the
: offset between solar-based time and atomic time is actually on the
: order of days?

From http://www.ucolick.org/~sla/leapsecs/dutc.html we know that the
rate of change of the day is somewhere between 25.6 s/century^2 and 42
s/century^2.  At those rates, it will be the year 6360-7633 when
enough time has accumulated for there to be a day difference.  Before
then, however, we go through a number of events.  Somewhere between
2058 and 2211, enter the realm where there's more than 2 leap seconds
per year.  This will be the first great UTC breakage because many
devices today (ntp included) KNOW that leap seconds happen twice
yearly.  The next break will happen sometime between 2300 and 2600
when we'll need more than 4 leap seconds a year.  The current ITU-R
TG.460 standard for leap seconds defines only primary and secondary
leap second times.  It does not define tertiary, so no one knows when
the leap seconds will happen if you need more than 4 per year, but the
ITU recommendation is that they happen at the end of a month.
Somewhere between 3250 and 4200 there will be more than 1 leap second
a month needed.  At this point the ITU scheme of having the leap
second at the end of the month will need to be modified.

That's at least 2000 years before 24 hours of delta have accumulated.

For some perspective, we've been using UTC for only ~50 years and the
gregorian calendar for only ~1500 years.  I'd anticipate that
something would need to be done about the slowing of the day well
before 4300 years have passed.

Somewhere around betwee 45,000-80,000 you'll need more than one leap
second a day.

: Will people be able to deal with a civil time
: standard that is based on an offset from a UTC that says it's
: Monday when all actual points on Earth have the local date at
: Saturday or Sunday?

Since that's 4k or more years into the future, who alive today will
know enough about what the future is like to impose a scheme that is
guranteed to work?

Clearly some scheme better than leap seconds will need to be invented
well in advance of these events.  A new scheme will be needed well in
advance of the Tuesday is really Wednesday problem.

: Many Web sites (including Wikipedia) use UTC as
: the standard for date/timestamps; will this be a reasonable thing
: when this causes the date of postings to be far off from what is
: being used locally?  And when, at some future point, the Gregorian
: calendar itself needs adjustment to handle the fact that it doesn't
: get the length of the year precisely correctly (and the length of the
: year in terms of solar days is changing due to the lengthening of the
: day, anyway), will this adjustment be done to the UTC standard (why,
: when it doesn't follow astronomy anyway?), or as an additional offset
: to local times (which could result in different countries having
: different dates as in the Julian/Gregorian transition period)?

The length of the gregorian calendar is off by 23s per year.  In year
5500 or so we'll have accumulated a day of error in it and we'll need to
skip a leap year to correct for that problem.  This is a good 2000
years before we'll have accumulated a day of DUT.

As you can see from the above, leap seconds won't save you.  They will
run out of steap in about 1500 to 2500 years.  At that point the
accumulated difference will be only about 2 hours.  If leap seconds
are totally abolished, time zone transitions could easily continue for
about 4000 years.  Either way, you have a problem.  The length of the
SI second is fixed, and the length of the day is getting shorter.

1500 years ago, no one spoke English.  Chances are the people that
deal with this problem in 1000 or 2000 years won't speak any language
recognizable to anybody alive today.

Warner


Re: Risks of change to UTC

2006-01-20 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
James Maynard [EMAIL PROTECTED] writes:
: ones position using sight reduction tables.  Today a mechanical watch or
: chronometer, or a battery-powered wristwatch, can be set to UTC using
: radio time signals. Then when power fails, the sailor still has a
: reasonably accurate spprodximation to UT1 available.

If DUT1 is broadcast, then one can set the time keeping device to UT1
by a means similar to setting it to UTC, even if DUT1 exceeds 0.9s
with a similar accuacy (or better).  There's nothing that says a watch
has to display UTC to be set correctly.

Warner


Re: Risks of change to UTC

2006-01-20 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
James Maynard [EMAIL PROTECTED] writes:
: M. Warner Losh wrote:
:
:  :  If DUT1 is broadcast, then one can set the time keeping device to UT1
:  :  by a means similar to setting it to UTC, even if DUT1 exceeds 0.9s
:  :  with a similar accuacy (or better).  There's nothing that says a watch
:  :  has to display UTC to be set correctly.

WWV and most of the world's time stations broadcast DUT1.  I should
have added in my last message that some change in the signal format
would be necessary if the range of DUT1 exceeds 0.9s.  Given a
sufficently long time horizon, this becomes a non-issue.

: You seem to expect celestial navigators to become more sophisticated in
: these matters of time scales than is presently the case. It would be far
: better, in my option, not to tinker with the definition of UTC and of
: civil time at all, but leave civil time zones tied approximately to UT1.

Maybe that is a reasonable expectation.  Maybe it is not.

I will note that the profile of high precision time users has changed
since 1972 when UTC was invented.  In 1972 celestial navigators were a
big part of the users of time.  These days, they represent a much
smaller portion of that audience.  If there are changes to how time is
provided, then all users of precision time should be considered.
Should we continue to tie our time up in knots because of a tiny
minority of users?

Maybe the answer that question is yes.  Maybe it is no.  However,
stating absoultely, unquestionably that it is one or the other is what
is wrong and what takes this discussion off into the weeds.

How many celestial navigators are there today?

: There's nothing that says a watch has to display UTC to be set
: correctly.  But then the user would have to carry two watches, one set
: to an approximation of UT1 (e.g., UTC + DUT1) and another set to civil
: time (TI, or local zone time, or whatever). For simpler, it would be,
: with far less impact on ordinary users of time, not to let UTC (or its
: replacement) differ from UT1 by more than the present +/- 0.9 s.

Over the next 50 years, these two watches will be well within the
tolerance of most normal watches.  The approximation of civil time
will be less than one minute off during that time (if most of the
projections are to be believed).

If this is a real issue, the market will take over and produce watches
that have 'navigation time' and 'civilian time' at the touch of a
button.  If you still desire to use older, purely mechanical watches,
then you must take extra steps, like knowing the current offset
between civil time and navigation time.  Of course, purely mechanical
watches have other issues when used on a boat.  Electro-mechanical
ones don't suffer those problems, but require some power source.

The foregoing assumes, for the sake of argument, that DUT1 becomes
unbounded.

: UTC is not broken. There is no need to fix it.

UTC works for navigation, but leap seconds pose problems for other
users of time.  Stating absolutely that UTC is not broken ignores
these other users.

:  I'd also add that GPS receivers today already do exactly this sort of
:  correction when they decode the GPS time, but display the UTC time.
:
: True. But GPS receivers are electrically powered, and are not a fallback
: for the case when a boat's electrical systems fail.  Nor are they a
: backup system for the case when the the boater is in proximity to a
: war zone where the U.S. DoD is jamming or degrading the civil SPS GPS
: service.

GLASNOS is a backup system to GPS that is not subject to DoD's
selective denial of signal.  Soon, Galileo will provide additional
backup.  Also, there's about a dozen time stations worldwide that
provide time signals today which can act as backup to GPS during a GPS
outage.  I pointed out GPS more as an example of a 'clock that gets it
times in one timescale, but displays it in another' rather than as a
backup time source for your example mariner.

Warner


Re: Internet-Draft on UTC-SLS

2006-01-19 Thread M. Warner Losh
The biggest objection that I have to it is that NTP servers will be at
least .5s off, which is far outside the normal range that NTP likes to
operate.  Unless the prceice interval is defined, you'll wind up with
the stratum 1 servers putting out different times, which ntpd doesn't
react well to.

ntpd runs at a 1024s interval, typically.  Depending on when the leap
second is announced, and when it polls, ntpd would see a 1s step
(since the rubber seconds happen over 1000s).  It wouldn't believe it
(since it has popcorn outlier rejection).  It would then believe the
step after 2048s, but since the step is so large, it would just exit
(depending on the flags given to ntpd).  This will lead to *MORE*
clock skew, not less.

I'm also concerned about the fact that many radio time codes do not
announce the leap second pending until the last hour or less.  This
makes it hard to propigate out to the non-stratum 1 servers.

It is a horrible idea.  It is a horrible kludge and I'm not mollified
by the weak justifications that are given in the faqcrit.  My
concerns are real and valid and this will cause more problems than it
kludges over.  It is a terrible idea.

Warner


Re: Fixing POSIX time

2006-01-19 Thread M. Warner Losh
I like this idea as well...

Poul, maybe we should implement this on FreeBSD.

I'd suggest working_time_t or correct_time_t as the name of the
type to replace time_t which would be deprecated. :-)

Warner


Re: Internet-Draft on UTC-SLS

2006-01-19 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Markus Kuhn [EMAIL PROTECTED] writes:
: M. Warner Losh wrote on 2006-01-19 16:58 UTC:
:   http://www.ietf.org/internet-drafts/draft-kuhn-leapsecond-00.txt
:  The biggest objection that I have to it is that NTP servers will be at
:  least .5s off, which is far outside the normal range that NTP likes to
:  operate.  Unless the prceice interval is defined, you'll wind up with
:  the stratum 1 servers putting out different times, which ntpd doesn't
:  react well to.
:
: Please read the proposal carefully (Section 2):
:
:UTC-SLS is not intended to be used as a drop-in replacement in
:specifications that are themselves concerned with the accurate
:synchronization of clocks and that have already an exactly specified
:behavior near UTC leap seconds (e.g., NTP [RFC1305], PTP [IEC1588]).

I saw that, but it does present some interesting implementation issues
for an ntp server.

: What this means is:
:
:   - NTP still uses UTC on the wire, exactly in the same way in which it
: does so far, *independent* of whether any of the NTP servers or clients
: involved supply UTC-SLS to their applications.

This is the hard part.  Right now, NTP steers the system time to UTC
as best it can.  It steps the time back by one second at the leap
second since there's no way in POSIX's time_t to represent leap
seconds, and all the time interfaces to the kernel are in terms of
time_t.

During the recent leap second, I observed three different clients of
one of our GPS receivers do three different things with the leap
second:  One inserted it early with the kind of hard frequency steer
that UTC-SLS does.  The second did it centered around the leap second,
and one didn't do anything until after the leap second and a couple of
polling intervals, then steered it out.  It is based on these systems
that implement the UTC-SLS-like steers that I based my reaction on.

:   - NTP implementations (by this I mean the combined user-space and
: kernel-space segments) should convert NTP timestamps that have been
: received over the wire through the UTC - UTC-SLS mapping, and steer
: after that what gettimeofday() provides to users .
:
:   - NTP implementations should equally also convert any timestamp received
: from gettimeofday through the UTC-SLS - UTC mapping before it goes
: out the wire.

This is the difficult part.  Right now, ntp daemons use the system
time as the time they do the time exchange with.  This is even if they
have a fancy gps receiver connected to them that isn't yet fully
synchronized.  Since they use system time, as you observe there would
need to be some mapping.  This mapping would necessarily have to be in
the kernel (since only it has the knowledge of the actual UTC UTC-SLS
offset) and would need to be an additional system call.  Such system
calls are not yet standardized.  This adds additional overhead to the
time exchange, which increases the opportunity for latency to creap
in.

Effectively, you'd have to have two time scales in the kernel.  UTC
and UTC-SLS.  You make it sound simple, but the hair in doing this may
be quite difficult.  There's more book for the kernel to keep, and it
would have to expose the bookkeeping to userland for ntp to work.
What makes this hard is that ntpd may introduce steers into the normal
system time at normal times which it doesn't want to confuse with the
steers in frequency that are used during a UTC-SLS operation.

: In other words, *no* incompatible changes are made to the NTP protocol.
: In a correct UTC-SLS implementation, you should *not* be able to
: distinguish remotely, whether some NTP server synchronizes its kernel
: clock to UTC or UTC-SLS, because this must not influence its NTP
: interface in any way.

In the systems that I observed over the leap second, I can say that
those systems that did implement a UTC-SLS-like approach to steering
out the extra second were definitely observable while they did this.
Given the variance in approach to this steering, and given it was
observable to the outside world, I concluded that N independent ntp
servers doing this would only confuse ntp clients...

: I hope this answers your concerns.

Well, it raises a whole different set of issues than I had originally
feared.  ntpd's use of system time to do its time exchanges has caused
me a significant amount of trouble in the past.

:  I'm also concerned about the fact that many radio time codes do not
:  announce the leap second pending until the last hour or less.  This
:  makes it hard to propigate out to the non-stratum 1 servers.
:
: I fully agree that leap seconds should be announced as early as
: possible, and I think that anything less than a month is undesireable.
: GPS sends out announcements within days after IERS does, which is
: excellent service. NIST sends out announcements a month in advance on
: their WW* stations, which is also pretty good. DCF77/HBG sadly do so
: only 59 minutes in advance

Re: the GPS impending leap second bit

2006-01-19 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Steve Allen [EMAIL PROTECTED] writes:
: On Thu 2006-01-19T18:30:02 +, Markus Kuhn hath writ:
:  GPS sends out announcements within days after IERS does, which is
:  excellent service.

IERS announced the leap second on July 4th, about 6 weeks before it
was encoded into GPS stream.  This didn't happen until mid to late
August.

The Canadian time service was announcing the leap second months before
July (I have a friend who has been writing software to decode the
world's time broadcast stations, and he was amused by this).

: Yes, and no, for some notes from the NTP newsgroup imply that the
: setting of the GPS impending leap bit caused some GPS units to insert
: leap seconds every day at 0h UT.

We noted that the GPS system started to advertise leap seconds in the
middle of August.  This causes the HP SmartClocks to hickup at the
alternate leap second time...

: Furthermore, in
: http://www.uscg.mil/hq/g-m/moa/docs/Saab505.pdf
: we see that the Saab AIS units introduced the leap second immediately
: and needed a firmware update to fix the condition.

Cute.  Older Motorola Oncore receivers had a firmware bug that caused
a three second blimp in their time code when it had been 1024 weeks
since the last leap second as well.

Warner


Re: Internet-Draft on UTC-SLS

2006-01-19 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Markus Kuhn [EMAIL PROTECTED] writes:
: M. Warner Losh wrote on 2006-01-19 19:20 UTC:
:  : In other words, *no* incompatible changes are made to the NTP protocol.
:  : In a correct UTC-SLS implementation, you should *not* be able to
:  : distinguish remotely, whether some NTP server synchronizes its kernel
:  : clock to UTC or UTC-SLS, because this must not influence its NTP
:  : interface in any way.
: 
:  In the systems that I observed over the leap second, I can say that
:  those systems that did implement a UTC-SLS-like approach to steering
:  out the extra second were definitely observable while they did this.
:
: Yes, of course! If you connect a GPS receiver that delivers an
: undocumented UTC-SLS-alike timescale on its output to an NTP server that
: in turn knows nothing of this UTC-SLS-alike timescale and is not
: implemented accordingly, you must get a mess. What did you expect? What
: you observed is *not* an implementation of UTC-SLS, but just a
: connection of incompatible components!

No.  You misunderstand me.  There were no GPS receivers that did
UTC-SLS-alike timestamps.  This was what CLIENTS of the GPS receiver
did.  The GPS receiver did exactly the right thing, ntp did the right
thing, our products did the right thing.  What I observed was in the
CLIENTS how they dealt with the leap seconds.

GPS-RECEIVER --- GPSNTPD --- NTP CLIENT1[*] windows NT box
 +-- NTP CLIENT2[*] Linux box
 +-- NTP CLIENT3[*] FreeBSD box
 +-- NTP CLIENT4[*] FreeBSD box

The items marked with a [*] are the ones I observed.  client1 and
client2 were the ones that I observed implementing UTC-SLS like time
and such time was visible to the outside world.  I was clearly able to
see both on the GPSNTPD clients 3 and 4 the skews of clients 1 and 2
during the leap second, where I observed no such skews relative to
GSPNTPD.

I'm saying that certain OSes have implemented the UTC-SLS-like
systems.  I'm saying that ntp daemons running on those systems use the
actual system time for time exchanges, rather than real UTC.  I'm
saying that this can cause confusion to a downstream ntp client and
that's my basis for thinking this idea is not good.  I'm further
saying that for the OS to support ntpd, it would need to also support
UTC and provide an interface to get the raw UTC time.  It would need
to keep track of steers for the purpose of implementing UTC-SLS and
steers needed to compensate for the changing base clock frequency
separate so that it could give the right time when asked for UTC
time.  It would also need to give some kind of feedback to ntpd that
it didn't do the small steer it wanted because UTC-SLS is going on at
that time (since otherwise ntp would think that there's some frequency
step going on if it just ignored it, and doing both steers
concurrently would be very hairy).

It is a seductive idea that leaves one with a big hangover in the
morning.

:  : However, MSF has no leap warning at all, nor do some time codes used in
:  : the power or military industry. And recent extensions to the latter
:  : added only a leap second warning that arrives a few seconds before the
:  : leap. I consider the leap-second handling of these latter time codes
:  : pretty useless.
: 
:  IRIG-B (all IRIG?) has no leap second announcement that I could find.
:  The Irig standard I could find (IRIG STANDARD 200-04, issued September
:  2004, at http://www.jcte.jcs.mil/RCC/PUBS/pubs.htm) makes no mention
:  of the format of the leap seconds at all (the leap second convention
:  appendix says that leap seconds happen).  Talking to engineers here
:  who have implemented IRIG at various places reveals that there are
:  multiple conventions for dealing with leap seconds in irig (the new
:  standard being to do 59, 60, the old standard was to do 59 59).
:  There's no provision for leap second announcement, although control
:  bits do remain unassigned...
:
: The IRIG world is clearly a huge mess, with more than two dozen
: formats in use. A useful beginner's overview is at
:
:   http://www.symmttm.com/pdf/Gps/an_Time_and_Time_Code_Ref.pdf

Excuse me, but isn't it a bit patronizing to point someone at a
beginner's guide?  I don't need beginner's overviews.  I have plenty.
I've been dealing with Irig for several years now, as well as many
other time codes and timing issues (including leap seconds).  What I
need is references to standards and copies of standards.  I want to
understand all the variations of IRIG so I know what the
sales/marketing folk are talking about when they ask me if thus and
such standard is easy or hard.

: which outlines on page 23 the 27-bit IEEE 1355 power-industry extension
: to IRG-B that has a Leap Second Pending (LSP) bit that becomes 1 up to
: 59 sec before leap second insertion. That was the one I referred to. If
: you have a good and very reliable signal, it may be just good enough to
: insert 23:59:60

Re: Internet-Draft on UTC-SLS

2006-01-19 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Markus Kuhn [EMAIL PROTECTED] writes:
: M. Warner Losh wrote on 2006-01-19 19:20 UTC:
:  Effectively, you'd have to have two time scales in the kernel.  UTC
:  and UTC-SLS.  You make it sound simple, but the hair in doing this may
:  be quite difficult.  There's more book for the kernel to keep, and it
:  would have to expose the bookkeeping to userland for ntp to work.
:  What makes this hard is that ntpd may introduce steers into the normal
:  system time at normal times which it doesn't want to confuse with the
:  steers in frequency that are used during a UTC-SLS operation.
:
: You correctly point out some of the design considerations that have to
: go into such code. You describe roughly one of the (several) different
: ways of implementing all this that I have in mind. In comparison to how
: complicated the Mills kernel PLL is already today, that does not
: actually sound like an overwhelming additional complexity. Actually, it
: sounds quite doable when you think through it a bit. Not trivial, but
: carefully doable without performance penalty.

Anything that makes the Mills' kernel PLL more complicated is unlikely
to be implemented correctly.  There have been several subtle bugs in
Mills' implementation that have taken a long time to sort out.  In
fact, members of this list have found some of them and have a lot of
experience with Mills' kernel PLL.

Many people in the timing community are unhappy with Mills'
implementation.  They believe it to be way too complex and have been
trying to implement a much simpler implementation that gives more
control to the measurement process to apportion the errors that it
measures by doing simpler steers.  It moves the kernel time control in
the wrong direction.

: The important thing is: All this has to be done only *once* for each
: operating system and *once* for each each NTP client/server used on it,
: by someone who understands the issue, and then the problem is hopefully
: solved. We've handled far more complicated things in OS kernels, right?

No.  It has to be done continuously.  By that I mean that you can
clearly write an implementation once, but over time you will find
flaws with that implementation (I can 100% guarnatee that).  As you
find the flaws, you'll have to investigate them and fix them.  In
addition, other parts of the kernel change, and some of those changes,
though innocent looking, can have a big impact on how you implemented
the dual UTC UTC-SLS offset.

By way of illustration, in FreeBSD 2.2, the timing implementation was
changed from keeping the current time, to keeping an uptime plus a
boot time.  Adjustments to the boot time were used to accomplish phase
shifts in the system time.  Unbeknownst to the person implementing
this, despite him having integrated the Mills kernel NTP portion into
a prior version of FreeBSD, this broke the exact semantics of system
time around the leap second.  It took years for someone (me) to notice
and fix this problem.

: I also fully appreciate that the existing *interface* between
: kernel-level clock drivers and user-level NTP clients and servers is by
: far not as well specified as it could be. With all due respect to its
: author, I do not hope that RFC 1589 will not remain the last word on
: clock-driver interfaces. A detailed proposal for a new standard on that
: area should help to smooth the process of getting the above right. I
: think a serious clock-controlling API this is what some people have been
: asking for for a long time from POSIX.

In order to implement your UTC-SLS proposal, it must change.

: I believe it is good engineering practice to specify the functionality
: before the interface and implementation, therefore I would like to
: discuss aspects of desired functionality (including UTC-SLS) before
: starting to work on a proposed successor to adjtimex(), ntp_adjtime(),
: etc.

While I agree in principle, I can tell you as one who has bloodied his
knuckles on implementing timekeeping that what you are saying sounds
hard to get pedantically right.

: I am not claiming that the spec-writing job is finished, or even half
: finished yet. I see a replacement for RFC 1589 etc. as the next job on
: the agenda before UTC-SLS (and lots of other time API ideas floating
: around here and elsewhere) can start to fly.

Unfortunately, it makes it hard to evaluate your proposal in the
absense of a concrete spec.  What is there now sounds nice, but it
still strikes me as difficult to implement.

Warner


Re: Internet-Draft on UTC-SLS

2006-01-19 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Tim Shepard [EMAIL PROTECTED] writes:
:  The serious timekeeping people gave up on rubberseconds in 1972 and
:  I will object with all that I can muster against reinventing them
:  to paste over a problem that has a multitude of correct solutions.
:
: As I learned from a recent posting to this mailing list, it seems that
: even TAI has rubber seconds (adjustments to the rate is made from time
: to time to compensate for errors that have been accumulating, making
: TAI a better (more useful) approximation time).
:
: Do you object to those adjustments (rubber seconds) to TAI as well?

TAI changed the definition of the second in 1977 by one part in 10^12.
Those weren't rubber seconds.  It again slowly changed the definition
over 3 years of the second by 2 parts in 10^14 ending in 1998.  They
were made not to correct the phase of TAI, but rather to correct the
length of the second to more closely match the SI second definition.
No phase adjustments were made to TAI.

Rubber seconds, on the other hand, are a steering device designed to
keep time in sync with another clock, in this case the earth.  They
aren't changing the definition of the second, but rather are inserting
phase adjustments to the timescale.  The goofy scheme that existed
prior to 1972 fell into this latter category.

Warner


Re: The real problem with leap seconds

2006-01-18 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Francois Meyer [EMAIL PROTECTED] writes:
: On Mon, 16 Jan 2006, Mark Calabretta wrote:
:
:  On Fri 2006/01/13 11:17:52 -, Michael Deckers wrote
:  in a message to: LEAPSECS@ROM.USNO.NAVY.MIL
: 
: I must get TAI, up to an integration constant. This is correct.
: The integral of d( UTC ) is TAI (up to an integration constant),
: but this integral is UTC only where UTC is a continuous function
: of TAI.
: 
:  You're still not getting the point that UTC is just a representation
:  of TAI.
:
: Maybe it should be,  but  this  is  far  from  being
: obvious from its current definition.
:
: The actual situation corresponds to :
:
: 1. UTC and TAI  share  the  same  rate,  the  same
:origin, the same second. And therefore :
:
: UTC - TAI = 0

The history of UTC and TAI is complicated and tricky.  You can only
say that on Jan 1, 1972, the TAI - UTC offset was fixed to be 10s.
UTC and TAI prior to 1972 did not evolve at the same rate.  The UTC
seconds differed in length from the SI seconds.  I'll ignore the
nomenclature differences over time as well.

: 2. UTC only differs from TAI by  its  definitions of
:the minute, hour and day.

This is not true prior to 1972.  There were a number of frequency
offsets in UTC that weren't present in TAI.  Some leap second charts
have these included in them.

: 4. the UTC minute is defined  to  ensure  that  dhms
:expressions of UTC match UT1 at .9 s; it  can  be
:either  59,  60  or  61  SI  seconds  long.  This
:definition of the minute is realized
:by  (positive  or  negative)  leap  seconds   and
:ensures that the mean UTC day is the  mean  solar
:day in the long term. The UTC  hour  has  60  UTC
:minutes, the UTC day has 24 UTC hours.

Again, post 1972...  I'm not sure what I think of this definition.


: From that point of view, the sentence from the ITU460-6 :
:
: [UTC] ...differs from it [TAI] from an integer of seconds
:
: should read :
:
: representations of UTC  involving  minutes,  hours,
: days differ from equivalent representations  of  TAI
: by an integral number of seconds

After 1972, and ignoring minor variations in the realization of UTC
and TAI in any given location, this is basically true.  The only ideal
difference between TAI(ideal) and UTC(ideal) is in the labeling of the
pulses that both time scales have experienced.  If one were to treat
them both as fixed radix, you get the difference of 33s.  Viewed
topologically as a 1-1 mapping, one could easily define subtraction so
that the difference is zero since UTC has a variable radix...

To further complicate things, TAI isn't created in realtime, but is a
paper clock that's steered to the correct time of the clocks that feed
it data.  There are clocks that are steered to this paper clock, but
it is all done by hand (if the various web pages I've read are still
accurate).  Different facilities realization of the TAI and UTC time
scales may differ by several nanoseconds (or more depending on a lot
of factors).

However, leaving aside those complications...

Given that UTC is a variable radix notation for labeling the pulses
that have elapsed since the epoch.  TAI is a fixed radix notation for
labeling pulses that have elapsed since the epoch.

Warner


Re: The real problem with leap seconds

2006-01-18 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Mark Calabretta [EMAIL PROTECTED] writes:
: On Wed 2006/01/18 08:17:54 -, Francois Meyer wrote
: in a message to: LEAPSECS@ROM.USNO.NAVY.MIL
:
: Maybe it should be,  but  this  is  far  from  being
: obvious from its current definition.
:
: I agree that the current definitions leave a lot to be desired in terms
: of clarity and rigour - an uncharitable person might even describe the
: extract of ITU-R TF.460-6 cited the other day by Michael Deckers as
: inconsistent.  However, you have to consider how UTC is actually used in
: practice and this is what my comments are based on.
:
: 1. UTC and TAI  share  the  same  rate,  the  same
:origin, the same second. And therefore :
: 
: UTC - TAI = 0
:
: They both count SI seconds but the question of the origin is a bit
: muddy.
:
: You could argue that there is a fixed 10s offset between UTC and TAI
: because UTC post-1972 (everything I've said about UTC only applies
: post-1972) started with 10 leap seconds, and before 1972 UTC wasn't
: simply a representation of TAI.  There's no simple way of fudging
: radixes that I can think of to make them match up, but if this worries
: you then simply think in terms of proleptic UTC (post-1972),
: see http://en.wikipedia.org/wiki/Proleptic.

UTC (and its predecessors) prior to 1972 did not tick off in SI
seconds.  It used a fixed radix like TAI currently does.  The amount
of time in UTC seconds varied a little.  Here's the table from
ftp://maia.usno.navy.mil/ser7/tai-utc.dat

1961 JAN  1 =JD 2437300.5  TAI-UTC=   1.4228180 S + (MJD - 37300.) X 0.001296 S
1961 AUG  1 =JD 2437512.5  TAI-UTC=   1.3728180 S + (MJD - 37300.) X 0.001296 S
1962 JAN  1 =JD 2437665.5  TAI-UTC=   1.8458580 S + (MJD - 37665.) X 0.0011232S
1963 NOV  1 =JD 2438334.5  TAI-UTC=   1.9458580 S + (MJD - 37665.) X 0.0011232S
1964 JAN  1 =JD 2438395.5  TAI-UTC=   3.2401300 S + (MJD - 38761.) X 0.001296 S
1964 APR  1 =JD 2438486.5  TAI-UTC=   3.3401300 S + (MJD - 38761.) X 0.001296 S
1964 SEP  1 =JD 2438639.5  TAI-UTC=   3.4401300 S + (MJD - 38761.) X 0.001296 S
1965 JAN  1 =JD 2438761.5  TAI-UTC=   3.5401300 S + (MJD - 38761.) X 0.001296 S
1965 MAR  1 =JD 2438820.5  TAI-UTC=   3.6401300 S + (MJD - 38761.) X 0.001296 S
1965 JUL  1 =JD 2438942.5  TAI-UTC=   3.7401300 S + (MJD - 38761.) X 0.001296 S
1965 SEP  1 =JD 2439004.5  TAI-UTC=   3.8401300 S + (MJD - 38761.) X 0.001296 S
1966 JAN  1 =JD 2439126.5  TAI-UTC=   4.3131700 S + (MJD - 39126.) X 0.002592 S
1968 FEB  1 =JD 2439887.5  TAI-UTC=   4.2131700 S + (MJD - 39126.) X 0.002592 S
1972 JAN  1 =JD 2441317.5  TAI-UTC=  10.0   S + (MJD - 41317.) X 0.0  S
1972 JUL  1 =JD 2441499.5  TAI-UTC=  11.0   S + (MJD - 41317.) X 0.0  S
1973 JAN  1 =JD 2441683.5  TAI-UTC=  12.0   S + (MJD - 41317.) X 0.0  S
1974 JAN  1 =JD 2442048.5  TAI-UTC=  13.0   S + (MJD - 41317.) X 0.0  S
1975 JAN  1 =JD 2442413.5  TAI-UTC=  14.0   S + (MJD - 41317.) X 0.0  S
1976 JAN  1 =JD 2442778.5  TAI-UTC=  15.0   S + (MJD - 41317.) X 0.0  S
1977 JAN  1 =JD 2443144.5  TAI-UTC=  16.0   S + (MJD - 41317.) X 0.0  S
1978 JAN  1 =JD 2443509.5  TAI-UTC=  17.0   S + (MJD - 41317.) X 0.0  S
1979 JAN  1 =JD 2443874.5  TAI-UTC=  18.0   S + (MJD - 41317.) X 0.0  S
1980 JAN  1 =JD 2444239.5  TAI-UTC=  19.0   S + (MJD - 41317.) X 0.0  S
1981 JUL  1 =JD 2444786.5  TAI-UTC=  20.0   S + (MJD - 41317.) X 0.0  S
1982 JUL  1 =JD 2445151.5  TAI-UTC=  21.0   S + (MJD - 41317.) X 0.0  S
1983 JUL  1 =JD 2445516.5  TAI-UTC=  22.0   S + (MJD - 41317.) X 0.0  S
1985 JUL  1 =JD 2446247.5  TAI-UTC=  23.0   S + (MJD - 41317.) X 0.0  S
1988 JAN  1 =JD 2447161.5  TAI-UTC=  24.0   S + (MJD - 41317.) X 0.0  S
1990 JAN  1 =JD 2447892.5  TAI-UTC=  25.0   S + (MJD - 41317.) X 0.0  S
1991 JAN  1 =JD 2448257.5  TAI-UTC=  26.0   S + (MJD - 41317.) X 0.0  S
1992 JUL  1 =JD 2448804.5  TAI-UTC=  27.0   S + (MJD - 41317.) X 0.0  S
1993 JUL  1 =JD 2449169.5  TAI-UTC=  28.0   S + (MJD - 41317.) X 0.0  S
1994 JUL  1 =JD 2449534.5  TAI-UTC=  29.0   S + (MJD - 41317.) X 0.0  S
1996 JAN  1 =JD 2450083.5  TAI-UTC=  30.0   S + (MJD - 41317.) X 0.0  S
1997 JUL  1 =JD 2450630.5  TAI-UTC=  31.0   S + (MJD - 41317.) X 0.0  S
1999 JAN  1 =JD 2451179.5  TAI-UTC=  32.0   S + (MJD - 41317.) X 0.0  S
2006 JAN  1 =JD 2453736.5  TAI-UTC=  33.0   S + (MJD - 41317.) X 0.0  S

(or the yet to be updated http://www.iers.org/MainDisp.csl?pid=95-106)

Here's slides from a presentation that is actually fairly well
balanced:
http://www.ien.it/luc/cesio/itu/mc_carthy.pdf
It has history there as well.  There's a nice graph of the above on
page 20.

http://www.ucolick.org/~sla/leapsecs/timescales.html also contains a
nice summary of UTC which is too long to include here but in outline
form is:

UTC during 1960
UTC from 1961 through 1971
UTC in 1963
UTC in 1965
UTC 

Re: Report of Leap Second Problem with GPS Data

2006-01-14 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Poul-Henning Kamp [EMAIL PROTECTED] writes:
: In message [EMAIL PROTECTED], Rob Seaman writes:
: On Jan 13, 2006, at 6:26 AM, Richard Langley wrote:
:
: I won't claim to know the intrinsic importance attached to this.
: Critical systems may depend on the information.  But is it fair to
: sum up the situation by saying that a leap second triggered a couple
: of bugs (or perhaps one common bug), they were detected, have been
: fixed, and affected data products have been remediated?  Also, it
: appears that some other data products were unaffected?
: 
: So, the issue has been resolved - would likely have been resolved
: sooner if a leap second had occurred earlier - and is no longer
: directly pertinent to a discussion of future leap seconds?
:
: Yeah, right
:
: This goes counter to my claims so it is of no importance.
:
: Sorry, things don't work that way Rob.

This time, there were no reports of death with the leap second,
therefore they can't be too bad... :-)

Warner


Re: The real problem with leap seconds

2006-01-11 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Mark Calabretta [EMAIL PROTECTED] writes:
: On Wed 2006/01/11 10:47:25 -, Michael Deckers wrote
: in a message to: LEAPSECS@ROM.USNO.NAVY.MIL
:
:At some instant when TAI took a value in the positive leap second between
:2006-01-01 + 00 h + 00 min + 32 s and 2006-01-01 + 00 h + 00 min + 33 s
:(the exact instant is not clear from [ITU-R TF.460-6 2002]), DTAI jumped
:from 32 s to 33 s; thus, UTC is not a monotone increasing function of
:TAI either.
:
: Here in a topology-free way is what the axis labels of my graph look
: like during the said leap second insertion:
:
: UTC axisTAI axis DTAI
:2005/12/31 23:59:58 2006/01/01 00:00:3032
:2005/12/31 23:59:59 2006/01/01 00:00:3132
:2005/12/31 23:59:60 2006/01/01 00:00:3232
: 60.932.9  32
: 60.99   32.99 32
: 60.999...   32.999... 32
:2006/01/01 00:00:00 2006/01/01 00:00:3333
:2006/01/01 00:00:01 2006/01/01 00:00:3433
:
: The seconds keep step and the graph has no gaps, jumps or kinks.

Shouldn't the DTAI increment for the leap second?  That's the point of
the leap second...  Or is that only needed when you do the POSIX
time_t thing of repeating 59?

In computer science, regular things are easy.  Irregular ones are hard
and prone to errors.  The :60 kludge makes a regular 1-1 mapping
function for time into a table driven nightmare :-(.

: The only difference between UTC and TAI is one of representation, like
: the current year which may be represented as 2006 or MMVI but it's still
: the same year.

The point isn't that one can construct a 'second labelling' that is
unambiguous, monotonic and increasing, but rather that when one
distills the UTC time down to a single number, you necessarily have
ambiguities and discontinuties in that function.  Of course there's an
implicit assumption that the conversion function is the same as
POSIX's time_t, otherwise you would be using TAI or a count of actual
seconds.  We call it PPSes or pulses at work to distinguish it.  We
label UTC time as having leap seconds swizzled in, and TAI time as
unswizzled.

Warner


Re: The real problem with leap seconds

2006-01-08 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Poul-Henning Kamp [EMAIL PROTECTED] writes:
: In message [EMAIL PROTECTED], Ed Davies writes:
: Wow, things have got really stirred up around here.  Lots of interesting
: points but I'll just concentrate on one...
: 
: Poul-Henning Kamp wrote:
:  Well, the BIPM doesn't really want anybody to use TAI, their director
:  said as much last year, and I can see where he is coming from on that
:  one.
: 
: Since the usual response of the pro-leap second lobby to people
: who want a uniform timescale is use TAI this is significant.
: Do you have any information or references on why the BIPM director
: said this?
:
: As I understood it, it was mainly that TAI is a post-factum postal
: timescale.

How is it that UTC can be realized in realtime, but TAI isn't.  I
thought the difference between the two was an integral number of
seconds, by definition.  Is that understanding flawed?

Wanrer


Re: interoperability

2006-01-08 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
John Cowan [EMAIL PROTECTED] writes:
:  But how in practice is it envisaged that a scheme
:  for migrating time zones versus TAI would work, precisely?
:
: Straightforwardly.  Each locality decides when and how to adjust both
: its offset from TAI and its seasonal transition function (if any),
: just as it does today.  What we abandon is a universal time tightly
: synchronized to Earth rotation in favor of a universal time
: independent of earth rotation plus 400+ local civil times roughly
: synchronized to Earth rotation containing various glitches.

No matter what we do with leapseconds, there are still all those time
zones.

The problem with stopping leap seconds altogether is that the legal
definitions of time, although quite varied, are all about the same as
UTC as it exists today.  They are close enough that most countries
have adopted UTC bureaucratically rather than legislatively.  The
official time for the US, as published by the folks at NIST, is UTC.
The US law says mean solar time, as determined by the Secretary of
Commerce, who has delegated it to the Time and Frequency division of
NIST, who in turn use UTC.  NIST could easily use a different schedule
for leap second insertion (it could have inserted the leap second in
civilian time at the end of any day it wanted to and still maintained
the mean solar time legal requirement).  However, since UTC is a
recognized, international standard, the US went along and did its leap
second according to that standard.  This is a explicit choice that
someone, somewhere had to make, even though it is arguably the best
choice to make (wouldn't want to be the odd man out in civil time,
think of the impact on business).

The combination of UTC approximating the legal time is so man nations,
as well as the need for international consensus among lots of parties
with divergent views for any changes to the current system is why
we'll likely not see significant changes any time soon.  The best we
can hope for is that something will be done to change their
unpredictable nature given that we have good forcasting tools at our
disposal.

Warner


Re: The real problem with leap seconds

2006-01-07 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Daniel R. Tobias [EMAIL PROTECTED] writes:
: On 7 Jan 2006 at 16:02, Poul-Henning Kamp wrote:
:
:  Civil time is in the hands of individual governments, and they
:  tend to expect their computers to use the same time as the
:  rest of their country.
:
: And, in many countries (including the United States), the legally-
: defined civil time is the mean solar time at some particular spot, or
: a fixed or seasonally-variable offset from it.  Any use of UTC-based
: time scales for determining civil time in such places is merely an
: approximation, currently to within a second, but perhaps varying by
: greater amounts if some new timekeeping plan is adopted.  Once UTC
: stopped being a sufficiently-close approximation to the solar mean
: time at the Prime Meridian (with sufficiently close possibly being
: of differing values depending on the particular purpose), it would be
: necessary to either stop using UTC for determining civil time in such
: countries, or to change the law to base the civil time on UTC instead
: of a solar standard.

The US's legislation leaves it to the commerce department to define
what the mean solar time is (or rather states that it is the mean
solar time, as determined by the secretary of commerce).  This is what
presumably gives us leap seconds and the like in our civil time and
when we insert them.  This is a political function: Up to a second of
tolerance is allowed and leap seconds are inserted whenever everybody
else does.  The same political decision could be made to say something
else, since it is the folks in DC could say that a minute is close
enough to solar mean and that's what we're going to do.

Warner


Re: predicting leap seconds

2006-01-07 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Neal McBurnett [EMAIL PROTECTED] writes:
: I still haven't seen any good data on predictions for periods of
: longer than 9 years.

Neal,

thanks for the excellent summary of the current state of the art in
prediction.

I think this shows that a 20 year time line is unrealistic at this
point, but 5-10 years would keep things fairly close, and 4 years
should be able to keep the current tolerances.  It might be worth an
experiment where over the next 5 years IERS publish 12 new months of
data every 6 months.  (eg Jan 2006 publish both the June 2006 and Dec
2006 correct, July 2006 publish the June 2007 and Dec 2007 correction,
Jan 2007 publish Jun 2008 and Dec 2008).  We'd hit 4 years in advance
in Jan 2009.  This would phase in the predictive timeline for leap
second insertions, and would also give the IERS control to end the
experiment if the time horizons exceeded their ability to predict with
confidence.  This would be an evolutionary step, rather than a
revolutionary one.  Of course this would make them even more
entrenched than they already are, because to kill them would require
waiting many years...

Warner


Re: Longer leap second notice

2006-01-04 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Rob Seaman [EMAIL PROTECTED] writes:
: On Jan 3, 2006, at 5:46 PM, Warner Losh wrote:
:
:  As someone who has fought the battles, I can tell you that a simple
:  table is 10x or 100x easier to implement than dealing with parsing
:  the data from N streams.  Sure, it limits the lifetime of the
:  device, but a 20 year limit is very reasonable.
:
: Simpler is indeed better - if it satisfies the requirements.  While
: we're at it - how about a table to describe worldwide daylight saving
: rules?  Oh right - we already have that :-)  What we don't have is a
: mechanism to force the U.S. Congress not to change the rules out from
: under us.  Retaining the flexibility to easily change the rules is
: one of our requirements.

You are comparing two dissimilar things.  A more apt comparison would
be to the leap year rules that we have.  We know the rules going
forward a thousand years or so.  These too represent the fact that the
oribit arount the earth is not exactly 365.25 days.  We add leap
seconds because the rotation of the earth isn't exactly 86400s.  The
difference between them is that the earth's rotation around the sun is
more stable than its rotation on its axis.

Daylight savings time is something else entirely.  It is a political
decision that sunlight is better used in the evening than the morning.

: Twenty years does seem reasonable.  Would suggest this might be
: marketed as an extended cadence maintenance requirement, rather than
: as an expiration date - suspect astronomers aren't the only ones to
: rely on 30 year old computers on occasion.  I would heartily agree
: with the notion that a twenty year horizon is about appropriate for
: expecting to reach any decision on the future of UTC.  We'd be a lot
: further ahead on this if a closed door decision hadn't been rushed
: for the imagined benefit of the few.  In the mean time, there are
: many members of the astronomical software community who would be
: happy to contribute to an effort to improve time handling
: infrastructure and standards, rather than spending their own precious
: time fending off ill conceived political machinations.

Twenty years is an example number.  Ideally, as predictive science
gets better, we can do it for longer periods of time.  One would hope
to eventually have a schedule that's published 50 or 100 years in
advance.  Leap years have been published far in advanced for thousands
of years now, with only one correction to the rule about 700 years ago
as the measurement of the year was refined.  As we've refined it
further in the last few hundred years, we know, with a very high
degree of confidence, that the rule is good for at least a thousand
years.  The rule isn't perfect, as almost a full day of error can
accumulate in 400 years (requiring an extra leap day), but it does
bound the error nicely.

If we can extend the horizon from 6 months, then that would lead to
better predictibilty of leap seconds, and also allow for better
testing.

Of course, a rule that eliminates them entirely would also fit these
needs, but appears to have little support...

:  If we could have a table for the next 20 years, there'd be no need
:  to even write the code to get from the GPS stream :-)
:
: And if latitude and longitude were engraved on every street corner,
: there would be no need for GPS at all :-)  Transport of time signals
: to remote locations is the whole point.

I think it would have been better if I'd stated my point more
directly: Multiple sources of information about leap seconds leads to
a more robust system.  GPS can tell us about it, ntpd can tell us
about it, and having a table of known leap seconds can inform us.
These redundant sources of information act as a sanity check.  In the
development box that did do the leap second correctly, a backup source
of infomration allowed it to perform correctly over the leap second
despite its development not being complete.

: But should any of us be open to persuasion by a political tool to
: make the proposal go through without commiting anybody to anything
: for the next couple of hundred of years?

There are a number of politically expedient quick fixes.  Foisting it
off on future generations is a time honored political solution :-(.

:  I find your dismissive attitude towards software professionals that
:  have implemented a complete leap second handling infrastructure,
:  with pluggable sources for leap second rather annoying :-(
:
: Indeed, I'm sure I've exhausted my scant store of good will again.

My statement was born of frustration.  It was a little uncalled for.
I regret sending it as it was a cheap shot.

: Would be delighted to hear more about your leap second infrastructure.

In brief, we have a leap second table.  This table can be populated
from a number of different sources, usually via a table we've hard
coded into our products.  As the products run and discover new leap
seconds, these are added to the table and the 

Re: December 2005 leap second on MSF, Rugby, England

2006-01-01 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Joseph S. Myers [EMAIL PROTECTED] writes:
: The Linux kernel (with NTP synchronisation) duly syslogged
:
: Dec 31 23:59:59 digraph kernel: Clock: inserting leap second 23:59:60 UTC
:
: and Markus's program showed a transition from 1136073600.005623 to
: 1136073599.013130, i.e. 23:59:59 was repeated (whereas a POSIX clock in
: perfect synchronisation with UTC should appear to repeat 00:00:00, and a
: kernel source code comment
:
: /* The timer interpolator will make time change gradually instead
:  * of an immediate jump by one second.
:  */
:
: describes what would practically be a safer approach, if the code followed
: the comment).

ntp synced computers repeat the last second of the day.  POSIX is just
flat wrong here.  Actually, POSIX is completely silent on what is to
happen at the leap second to system time.  Once can infer what might
be the right behavior by working backwards from what mktime does for
23:59:60, but that's a weak inference for what the 'right' posix
behavior is for the system time accross a leap second.  POSIX simply
does not have leap seconds in any meaningful way.  It is as if they do
not exist at all.  There's no clarifications to the POSIX standard
that say that 0:00:00 should be repeated at a leap second, at least
that I've been able to find.  If there is such an explicit
clarification, I'd like to know about it.

FreeBSD does the same thing (as do all kernels that use the ntp kernel
stuff supplied with udel ntpd and successors).  FreeBSD doesn't
actually log the leap second via syslog, but maybe it should.

Ideally, we'd switch to running computers with TAI and doing all the
conversion on input/output of time.  However, *THAT*'s definitely not
POSIX.

Warner


Re: leap seconds and Linux/Unix, timezones, and zdump

2005-12-31 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Neal McBurnett [EMAIL PROTECTED] writes:
: Here are some notes on facilities in some Unix systems to show
: evidence of leap seconds.  Some recent distributions are out of date
: as noted below.  Can folks check others (Solaris, *BSD, etc?).

I'll comment for FreeBSD.

: First, this should work for everyone who cares about time and runs
: ntp.  Make sure you're in NTP sync with a host that knows a leap
: second is coming.  Look for leap=01 in the output of this:
:
:  $ ntpq -c rv
:  assID=0 status=46f4 leap_add_sec, sync_ntp, 15 events, event_peer/strat_chg,
:  version=ntpd [EMAIL PROTECTED]:4.2.0a-11-r Mon Mar 14 12:39:28 GMT 2005 
(1)?,
:  processor=i686, system=Linux/2.6.10-6-386, leap=01, stratum=3,
:  precision=-19, rootdelay=158.366, rootdispersion=77.103, peer=17708,
:  refid=82.211.81.145,
:  reftime=c76127cd.19f9acff  Sat, Dec 31 2005  8:52:45.101, poll=7,
:  clock=0xc7612b1b.74322291, state=4, offset=-0.430, frequency=37.132,
:  error=0.572, jitter=1.795, stability=64.092

FreeBSD does this in a similar way, since it is all based on ntp.

: Next check to see if your time zone files are up-to-date.  I'm no
: expert (help!) but it appears that on many Unix systems, the Olson
: notion of right/ timezones allows the system to understand leap
: seconds.

The so-called right files aren't installed on FreeBSD by default.
Instead, at zone compilation time, the leap seconds are added or not.
The default is for them not to be added, likely to comply with the
lame and broken requirements of POSIX's time_t.

Warner


Re: Schreiver AFB warns about leapsec

2005-12-20 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Francois Meyer [EMAIL PROTECTED] writes:
: On Tue, 20 Dec 2005, Rob Seaman wrote:
:
:  On Dec 20, 2005, at 1:30 AM, Poul-Henning Kamp wrote:
: 
:   There is an interesting PowerPoint (sigh...) at Schriever AFB's GPS
:   support center:
:  
:   https://www.schriever.af.mil/GpsSupportCenter/archive/advisory/
:   Leap_Second_Event.ppt
: 
:  Agreed.  Very interesting.
:  ...
:  An interesting observation:
: 
:  - Leap second occurs at an awkward time - New Years Eve
: 
:  Maybe obscurity in scheduling and implementation is not a desirable
:  characteristic after all.
:
: The same paradigm suggests a new definition of  UTC,
: strengthening its link to UT1  down  to  0.09s,  and
: switching from leap seconds  to  leap  tenths  of  a
: second. This aims at making leap  intervals  a  rule
: and not an exception. Tens of a second are  as  easy
: (or as difficult)  to  implement  as  leap  seconds,
: their instantaneous impact is  10  times  lower  and
: since only automated systems  are  really  affected,
: the increased frequency of their occurrence  is  not
: an issue.

I'd take issue that .1th of seconds are easier to implement,
especially given that most of the timing gear today would have to be
discarded because it isn't compliant.  All the time signals would need
to be redesigned.  A lot of software would need to be rewritten to
cope with the smaller offsets (not least ntp).  There's a huge
deployed base of gadgets that grok leap seconds as they are today and
that would become obsolete.

Not to mention that .1s jumps would make it harder to correlate data
back to TAI (the tables you'd need are more complicated).

.1s jumps would happen more often, so would be more unpredictable than
the leap seconds we have today.  My main beef with them is that we
have no way of knowing today, with certainty, if there will be a leap
second on June 30, 2006.  The schedule is essentially random, which is
a problem for devices that need to know utc-tai offset given an
approximate time.

I don't see how that would ever work.

Warner


Re: a system that fails spectacularly

2005-12-07 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Poul-Henning Kamp [EMAIL PROTECTED] writes:
: ISO9000 certification only means that you have documented your
: quality assurance process.
:
: There is no requirement that your documentation pertains to
: or results in a quality product.
:
: One of the Danish ISO9K consultants used to bring a ISO9000
: certification case along to explain this to companies:  It was
: basically the entire ISO9000 process for a small company written
: on one page of paper and the essence was We don't gove a hoot about
: quality.
:
: The information that company X is iso9000 certified only conveys
: one bit of information:  The company has a quality policy.
:
: You still need to read their quality policy to know what it is,
: and on average, the ISO9000 certified ones contain less usable
: or even readable information, than the other kind.

ISO 9000 only requires like 3 or 5 documents.  Small companies can
comply with just a notebook that contains these documents, assuming
that the quality policy doesn't require more...

Warner


Re: The Truth is Out There

2005-11-08 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Steve Allen [EMAIL PROTECTED] writes:
: After all, it's not like the world is going to come to an end.

If the end of the world is what it takes to abolish leap seconds, I'm
all for it :-)

Warner


Re: RAS hits the news

2005-09-26 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Rob Seaman [EMAIL PROTECTED] writes:
: The question is whether at least 20 minutes (presuming this to be
: accurate) is intrinsic to the system design or is rather a result of
: poor implementation by some receiver manufacturers.

A cold GPS receiver takes about 20 minutes to get the almanac data
from the GPS constellation.  It is intrinsic to GPS that this is the
case.  You cannot get around this.

As I stated before, various recievers have differing definitions of
'cold'.  Some are 'cold' the instant you turn them off.  Others are
cold only if they have been off so long that they have no way of
knowing the current correct leap count.  Given that BIPM only
publishes 6 months in advance, this means that the longest a receiver
can be off is about 6 months.  Many modern receivers do a good job of
caching the data over short to medium periods of being off.  This is
why others have reported that you get the right time from a Garman
within a few seconds of power on.

Also, a GPS receiver that has cached the almanac can acquire
satellites much more quickly than one who has to wait for the almanac
to be downloaded.  Many receivers, when acquiring satellites, take a
while to do this when they have no almanac.  This adds a little time
to recovery.  I believe the almanac is retransmitted every 18:mumble
seconds, but I've been saying 20 because it takes a little time (a
couple of minutes) to find the first satellite to start the process.

: A typical design pattern for conveying crucial metadata that is only
: rarely updated is to also convey a timestamp or expiration date.
: Either the eggs are expired or they aren't.  There certainly are ways
: for a GPS receiver to store metadata - even over a period of a year.
: Having cached the leap second table, the only question is whether it
: is expired for which a 4 or 8 byte timestamp would be quite sufficient.

The GPS format already does this, even more efficiently than you might
think.  There's two 8 bit quantities, and two 10 bit quantities that
represent the current and future leap second data.  The 8 bit fields,
as others have pointed out, are due to overflow in the next century or
so.  The week number also overflows in GPS.  Many receivers 'cheat'
and use a prediction algorithm to know which 1024 week epoch we're in
by looking at the number of leap seconds.  So when that field
overflows (be it at 7 bit or 8 bit (it is defined to be a signed
number, but that definition could be change)), better algorithms will
be needed.

Warner


Re: RAS hits the news

2005-09-26 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Tom Van Baak [EMAIL PROTECTED] writes:
:  In addition, since GPS time is TAI - 19s, the GPS-UTC difference will
:  eventually overflow any fixed-sized transmission packet (if transmitted
:  as a delta or as a table, it makes no difference in the end).
:
: True, but the GPS signal format has a number of
: fixed length fields and they do not cause a roadblock
: for receiver firmware engineers. There are fixed-sized
: 256 and 1024 week number fields that overflow,
: for example, and all modern GPS receivers get them
: right. It is trivial for a receiver to handle the equivalent
: 256 leap second rollover should one occur in the next
: hundred years.

Many of the week number roll over algorithms use the leap second count
as a good first guess which 1024 week epoch you are in.  If that field
rolls over, then these algorithms will need adjustment.  So it is
likely doable, but it might not be completely trivial due to the
irregularity of leap second insertions.

: On the question of UTC updates; it's true that a cold
: GPS receiver has to wait up to 12 minutes for the
: correct GPS/TAI/UTC delta. I am wondering, though,
: if anyone knows of an example of a GPS receiver
: that caches the delta value from the last power-up?
: It seems to me this would take care of the delay in
: all but the most extreme cases.

Most receivers cache some value.  That is why I've been careful about
saying cold since there are different definitions of cold between
receivers.  Having a cached almanac greatly speeds satellite
acquisition time.  This is why many GPS receivers do well when off for
up to about a week, and then have a much longer acquisition time when
they are turned on after a longer haitus.  They have to 'guess' at
what satellites are in the sky and rotate through their guesses until
they happen to hit on one that is in the sky and can download more
accurate almanac information.  Fortunately, that almanac data is
transmitted more frequently so once you have one, it goes pretty
fast to acquire the rest.

Warner


Re: RAS hits the news

2005-09-26 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Tom Van Baak [EMAIL PROTECTED] writes:
: Too much is made of the overflow. Fields rollover all
: the time in real life and it's often a simple engineering
: matter to take this into account. Not sure I would call
: it cheating. We get by fine with just 7 names for days
: of the week; calendars that rollover every 12 months,
: wristwatches that overflow every 12 hours...

These instances of overflow come from remainders of division
operations overflowing.  They all can be derived from a single base
number (say number of seconds since 1970, MJD, etc).  However, when
you are deriving that single base number, it can be much harder.

Could you tell me what year it was if I told you it was Monday,
October 15th?  No, you couldn't.  You could tell me that it might be
2001, but it could also be 2007 or 1990.  You need more information to
resolve the ambiguity.  If I told you that it was Monday, October 15th
and that TAI-UTC=32, you'd know it was 2001.  If I told you
TAI-UTC=100, you might guess that it is 2063, or maybe even 2068 or
2057 or maybe other years earlier or later depending on your leap
second model, utc-ut1 tolerance parameters, and other factors
unknowable today.

The GPS 1024 week overflow is easier to deal with, since it is a 20
year ambiguity, not a 5 year one.  You can make a better guess than in
my example, I'll not argue.  The better the guess you make based on
today's understanding, the more external factors that might cause you
to be wrong.  Eg, leap second rules changes, lifetime of GPS signals,
etc.

I guess I agree with you that these things are doable.  Working out
the details, however, makes them non-trivial.

Warner


Re: RAS hits the news

2005-09-21 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Steve Allen [EMAIL PROTECTED] writes:
: In my understanding the GPS system
: itself handles leap seconds pretty well, almost optimally.

One could say that GPS handles them perfectly, in that they do not
exist at all in the GPS time scale.  However, GPS' propigation of the
GPS UTC offset leaves much to be desired.  That data is sent in the
alminac, which takes at least 20 minutes to down when a reciever is
started cold[*].

Although you know the GPS time to within a few tens of nanos as soon
as you have 4 satellites, you have to wait another 20 minutes after
that to know UTC time if you are coming up cold.

One can debate the meaning of 'almost optimally' til the cows come
home, but my views lean away from such a characterization...

Warner

[*] The definition of cold varies from receiver to receiver, but all
of them necessarily have a cold state (turn it off for a year, and it
is guaranteed to be cold in that it can't possibly know the leap
second values).


Re: Consensus rather than compromise

2005-08-31 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Steve Allen [EMAIL PROTECTED] writes:
: If POSIX were to fix the definitions of time_t and epoch, why would
: this not imply that handling leap seconds with Unix would become
: trivial?

Because leap seconds are not trivial.  If you make time_t a TAI-like
thing, then you break all programs that do math to calculate a date
and time since the usual naive math no longer works.  You could fix
these programs to use new APIs to do the math instead.  However, you
also enforce upon all systems a requirement to keep a leap second
table up to date.  While not so bad for systems that are constantly
running, this can cause problems for people that maintain a stockpile
of spare parts.  These spares generally aren't on when leap updates
occur and won't have them for some period of time after being
deployed.  Also, many systems just aren't connected to a public
network, or aren't connected to a network at all, but still have a
need to have knowledge of leap seconds.

The class of functions that is defined as being continuous at all
points, except where it isn't, while easy to understand can and is
hard to implement correctly in all cases.

Warner


Re: Consensus rather than compromise

2005-08-31 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Ed Davies [EMAIL PROTECTED] writes:
: M. Warner Losh wrote:
:    Also, many systems just aren't connected to a public
:  network, or aren't connected to a network at all, but still have a
:  need to have knowledge of leap seconds.
:  
:
: Can you give any examples of systems which need to follow
: UTC to sub-second accuracy ... have a clock stable enough to
: do so and yet are not connected by any mechanism which could
: potentially provide leap-second information?

First, your question is a bit of a red herring.  More systems than
just those are affected.  If you have system time in TAI, and you want
to convert to/from UTC, you must necessarily know all the leap seconds
that have ever happend, or you can't do it.  The classic example of a
system that's not connected to any source of leap seconds is the spare
sitting off in the corner.  It has no knowledge of leap seconds that
happened  6 months after it was made, and has to acquire that
knowledge somehow after it is first powered on.

However, to answer your hypothetical question directly:

Consider a system that has GPS steering a stable time source
(say a cesium clock or rubidium standard).  GPS goes away for
some reason.  You have a stable time source for a long period
of time, yet no further knowledge of leap seconds.  If the
system is installed down the block, it is a simple matter to
go out and fix the GPS antenna.  If the system is in the
middle of nowhere in alaska controlling timing signal
automatically, it can take a while to get a crew out to fix
it.  Yet, it is highly desirable that it continue to work for
as long as possible.  Leap seconds artificially limit how long
ntp will work on such a system, for example, because you can't
flywheel more than about 6 months since after a leap second
opportunity, you don't know if there was one or not.

Of course, certain manual proceedures can be put into place for the
above example.  However, they are added complications that can't be
dismissed out of hand.  You have to design them well, or you get all
kinds of weird edge case behavior.

: (running to their own little timezone not being good enough),

Might I suggest that digs like this make rational discussions difficult?

Warner


Re: Consensus rather than compromise

2005-08-30 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Poul-Henning Kamp [EMAIL PROTECTED] writes:
: In message [EMAIL PROTECTED], Peter Bunclark writes:
:
: I would have thought that part of the answer to the difficulty in
: implementation and testing would be to use an open-source library of tried
: and tested algorithms.  I don't quite understand why software engineers
: seem to feel the need to write new leap-second handling code every time
: they invent a new gadget.
:
: The vast majority of software engineers do use standard code, they
: use their operating systems libraries, this makes them seemingly
: leap second compliant.
:
: Seemingly here covers that they are only compliant in all the
: seconds that are not leapseconds or seconds right before leap
: seconds.
:
: The POSIX definition makes it impossible to correctly handle leap
: seconds with any complying implementation of the standard, and
: therefore applications which needs to be *truly* leapsecond compliant,
: cannot use the standard libraries.

Not to mention devices that handle leap seconds almost, but not
entirely, correctly.  Much of the fancy footwork that I've had to do
is because some devices are better than others at their pedantic
compliance.  If one relies on just one detail that's gotten wrong,
then one's downstream data will be wrong.  Knowing what one can trust
and not trust, as well as building in the cross checks is also very
device specific.  This motorola receiver gets this datum wrong, but
that datum right, while this other motorola receiver gets it the other
way round.  This other GPS receiver supplies data that sounds like it
should be the same as the reliable motorola data, but in fact is
something subtly different.

The problems generally aren't in the leap second ticking code (which
is in the kernel and has been proven correct through repeated
testing).  The problems are in getting experience with the actual
details of how a specific device (and sometimes the specific firmware)
operates.  The problems are in what optimizations one can make to
recover time faster from a cold start than waiting for the leap info
to arrive at some random time in the next 1/2 hour.  The problems are
in what the power off behaviors of devices are.  The problems can even
be in how one tests the leap seconds in a simulator because some
devices have a strong notion that time flows forward only and produce
bad data for a while when that notion is violated.

All these details are hard to enshrine in a write once, reuse forever
open source (or even closed source) library.

Warner


Re: vive le BIH!

2005-08-26 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Steve Allen [EMAIL PROTECTED] writes:
: http://www.guardian.co.uk/australia/story/0,12070,1556764,00.html

The second is now defined not as the a fraction of a day, and
therefore an even smaller fraction of a year. It is 9,192,631,770
periods of radiation from a caesium atom in its ground state.

Looks like this reporter's idea of the current definition of the
second is a few years old too...

Warner


Re: decision tree for civil time

2005-08-13 Thread M. Warner Losh
In message: [EMAIL PROTECTED]
Greg Hennessy [EMAIL PROTECTED] writes:
:  Look what predictability did for leap days...
:
: Well, even though they are predictable, lots of computer programs still
: got them wrong.

Lots?  Do you have evidence to support this?

I'll grant you that some did, but the vast majority of the programs
I've encountered got it right.

Warner