Re: UT1 confidence

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Zefram <[EMAIL PROTECTED]> writes:
: Steve Allen wrote:
: >http://www.ucolick.org/~sla/leapsecs/torino/arias_3.pdf
:
: This is the really interesting one.  They present the accuracy of
: simulated predictions of UT1, and that accuracy is much poorer than
: the figures we've been discussing so far.  But they make it clear that
: the prediction is by a naive algorithm, not even applying the well-known
: periodic terms that are used in UT2.  This is obviously not the algorithm
: used by IERS.

The good news is that even an incredibly stupid (in terms of knowledge
used) algorithm can predict almost 3 years out

Warner

Re: UT1 confidence

[M. Warner Losh]

Steve,

thank you for this enlightening report.

In message: <[EMAIL PROTECTED]>
Steve Allen <[EMAIL PROTECTED]> writes:
: On Wed 2007-01-17T12:31:14 -0700, Warner Losh hath writ:
: > It has been remarked that the current state of the art is that 100ms
: > accuracy can be predicted about a year in advance only and that the
: > models are constantly undergoing refinement.  It has been estimated
: > that IERS could issue leap seconds, with today's technology, about 3-5
: > years out and still be in a 95% or 99% band of certainty that the 0.9s
: > margin is maintained.  However, I can't find papers that show these
: > models or point to any better data than hearsay...
:
: The best that I know of were the ones presented at the Colloquium that
: the WP7A SRG held in Torino in May 2003.  There was a time when the
: host institution (IEN) was providing the proceedings online, but the
: contents of that URL went away sometime around a year ago.  (I wonder
: if they may not have liked the conclusion that was reached.)
:
: In the spirit of promulgation I provide what they once did at
: http://www.ucolick.org/~sla/leapsecs/torino/ITU.shtml
:
: The conclusion was originally a powerpoint drafted in real time, it is
: http://www.ucolick.org/~sla/leapsecs/torino/closure.pdf
:
: The indications of how well predictions of UT1 might be done are found
: in three presentations to which Felicitas Arias contributed.
: There are two which were powerpoint
: http://www.ucolick.org/~sla/leapsecs/torino/guinot.pdf
: http://www.ucolick.org/~sla/leapsecs/torino/arias_2.pdf
: and one which is a more verbose writeup of one of the powerpoints
: http://www.ucolick.org/~sla/leapsecs/torino/arias_3.pdf

I like figure 8, that shows that 20ms steps lead to 50ms steps lead to
100ms steps lead to 1s steps. :-)

I also like the quotes:

"Dating in UTC is ambiguous when a positive leap second occurs in
systems other than those using hours, minutes and seconds; in this
latter system the use of second "60" may a cause of difficulty."

and

"Thus UT1 is not, strictly speaking, a form of solar time"

Also, did I miss figure 9 in arias_3?

Proposal II has gotten much press here (the leap hour one), but
Proposal I sounds a lot like what I've suggested: Use TAI time and let
countries move the time zones when they feel like they no longer are
close enough, but it kinda omits that last part...

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Tony Finch <[EMAIL PROTECTED]> writes:
: On Sat, 6 Jan 2007, M. Warner Losh wrote:
: >
: > Unfortunately, the kernel has to have a notion of time stepping around
: > a leap-second if it implements ntp.
:
: Surely ntpd could be altered to isolate the kernel from ntp's broken
: timescale (assuming the kernel has an atomic seconds count timescale)

ntpd is the one that mandates it.

One could use an atomic scale in the kernel, but nobody that I'm aware
of does.

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: What is correct is to have a 61 second minute occasionally, neither
: to redo the first second of the next day, nor to repeat the last
: second of the current day.

Unfortunately, that's not POSIX time_t.  And when you are implementing
unix kernel APIs, that is the only game in town, unless you invent
your own.  ntp_gettime comes close to implementing things right (on
FreeBSD the time state is synchronous to top of second), but even that
is a mess.

You can't do both 61 second minutes and also have the 'naive math'
that time_t requires.

I agree with others that have said that ntp should be TAI or GPS based
(eg a linear count of seconds in a timescale that doesn't have leap
seconds), and that leap second corrections should happen in userland
like is done for timezones.  That's really the only sane way to do
things.  Having tried it, there are a lot of little 33 second
anomolies in many applications :-(.  I've toyed with the idea of
running the kernel in TAI and having 'smart' processes tell libc they
want no UTC translation and having all the TAI<->UTC translation
happen in libc (also hacking those FS that want UTC time to be able to
get it).  Maybe I'll try it and see what breaks since the line between
applications and libraries can make a per-process flag hard to cope
with if your application links with a lot of libraries.

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: > If by "some limp attempt" you mean "returns the correct time" then you
: > are correct.
:
: It's not the correct time under the current standard if the
: timekeeping model doesn't implement leap seconds correctly.  I don't
: think this is an impossible expectation, see http://
: www.eecis.udel.edu/~mills/exec.html, starting with the Levine and
: Mills PTTI paper.

It implements exactly what ntpd wants.  I asked Judah Levine when
determining what was pedantically correct during the leap second.  I
also consulted with the many different resources avaialable to
deterimine what the right thing is.  Of course, there are different
explaintions about what the leap second should look like depending on
if you listen to Dr. Levine or Dr Mills.  Dr. Mills web site says
'redo the first second of the next day' while Dr. Levine's
leapsecond.dat file says 'repeat the last second of the day.'
Actually, both of them hedge and say 'most systems implement...'  or
some variation on that theme.

It is possible to determine when you are in a leap second using ntp
extensions with their model.  Just not with POSIX interfaces.  The
POSIX interfaces are kludged, while the ntpd ones give you enough info
to know to print :59 or :60, but POSIX time_t is insufficiently
expressive, by itself, to know.  But ntp_gettime returns a timespec
for the time, as well as a time_state for the current time status,
which includes TIME_INS and TIME_DEL for psotive and negative leap
second 'warning' for end of the day so you know there will be a leap
today, and TIME_WAIT for the actual positive leap second itself
(there's nothing for a negative leapsecond, obviously).

So I stand by my "returns the correct time" statement.

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Tony Finch <[EMAIL PROTECTED]> writes:
: On Sat, 6 Jan 2007, M. Warner Losh wrote:
: >
: > Most filesystems store time as UTC anyway...
:
: POSIX time is not UTC :-)

True.  It is designed to be UTC, but fails to properly implement UTC's
leap seconds and intervals around leapseconds.

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: Warner Losh wrote:
: > Anything that makes the math
: > harder (more computationally expensive) can have huge effects on
: > performance in these areas.  That's because the math is done so often
: > that any little change causes big headaches.
:
: Every IP packet has a 1's complement checksum.  (That not all
: switches handle these properly is a different issue.)

Actually, every IP does not have a 1's complement checksum.  Sure,
there is a trivial one that covers the 20 bytes of header, but that's
it.  Most hardware these days off loads checksumming to the hardware
anyway to increase the throughput.  Maybe you are thinking of TCP or
UDP :-).  Often, the packets are copied and therefore in the cache, so
the addition operations are very cheap.

: Calculating a
: checksum is about as expensive (or more so) than subtracting
: timestamps the right way.  I have a hard time believing that epoch<-
:  >interval conversions have to be performed more often than IP
: packets are assembled.

Benchmarks do not lie.  Also, you are misunderstanding the purpose of
timestamps in the kernel.  Adding or subtracting two of them is
relatively easy.  Converting to a broken down format or doing math
with the complicated forms is much more code intensive.  Dealing with
broken down forms, and all the special cases usually involves
multiplcation and division, when tend to be more computationally
expensive than the checksum.

: One imagines (would love to be pointed to
: actual literature regarding such issues) that most computer time
: handling devolves to requirements for relative intervals and epochs,
: not to stepping outside to any external clock at all.  Certainly the
: hardware clocking of signals is an issue entirely separate from what
: we've been discussing as "timekeeping" and "traceability".  (And note
: that astronomers face much more rigorous requirements in a number of
: ways when clocking out their CCDs.)

Having actually participated in the benchmarks that showed the effects
of inefficient timekeeping, I can say that they have a measurable
effect.  I'll try to find references that the benchmarks generated.

: > Well, the kernel doesn't expect to be able to do that.  Internally,
: > all the FreeBSD kernel does is time based on a monotonically
: > increasing second count since boot.  When time is returned, it is
: > adjusted to the right wall time.
:
: Well, no - the point is that only some limp attempt is made to adjust
: to the right time.

If by "some limp attempt" you mean "returns the correct time" then you
are correct.

: > The kernel only worries about leap
: > seconds when time is incremented, since the ntpd portion in the kernel
: > needs to return special things during the leap second.  If there were
: > no leapseconds, then even that computation could be eliminated.  One
: > might think that one could 'defer' this work to gettimeofday and
: > friends, but that turns out to not be possible (or at least it is much
: > more inefficient to do it there).
:
: One might imagine that an interface could be devised that would only
: carry the burden for a leap second when a leap second is actually
: pending.  Then it could be handled like any other rare phenomenon
: that has to be dealt with correctly - like context switching or
: swapping.

You'd think that, but you have to test to see if something was
pending.  And the code actually does that.

: > Really, it is a lot more complicated than just the 'simple' case
: > you've latched onto.
:
: Ditto for Earth orientation and its relation to civil timekeeping.
: I'm happy to admit that getting it right at the CPU level is
: complex.  Shouldn't we be focusing on that, rather than on
: eviscerating mean solar time?

Did I say anything about eviscerating mean solar time?

: A proposal to actually address the intrinsic complications of
: timekeeping is more likely to be received warmly than is a kludge or
: partial workaround.  I suspect it would be a lot more fun, too.

I'm just suggesting that some of the suggested ideas have real
performance issues that means they wouldn't even be considered as
viable options.

: > Kernels aren't written in these languages.  To base one's arugments
: > about what the right type for time is that is predicated on these
: > langauges is a non-starter.
:
: No, but the kernels can implement support for these types and the
: applications can code to them in whatever language.  Again - there is
: a hell of a lot more complicated stuff going on under the hood than
: what would be required to implement a proper model of timekeeping.

True, but timekeeping is one of those areas of the kernel that extra
overhead is called so many times that making it more complex hurts a
lot more than you'd naively think.

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Tony Finch <[EMAIL PROTECTED]> writes:
: On Sat, 6 Jan 2007, Ashley Yakeley wrote:
: >
: > Presumably it only needs to know the next leap-second to do this, not
: > the whole known table?
:
: Kernels sometimes need to deal with historical timestamps (principally
: from the filesystem) so it'll need a full table to be able to convert
: between POSIX time and atomic time for compatibility purposes.

Most filesystems store time as UTC anyway...

And that's one reason that kernels do a lot of timestamp operations:
whenever a file is touched, one has to update the time it was last
touched.  Then, when the file is statted, that time must be returned.
That makes it very hard to do all in libc because of different boot
times or nfs, etc.  While one could do the degenerate case of
gettimeofday in libc, these other cases are much harder.

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Ashley Yakeley <[EMAIL PROTECTED]> writes:
: On Jan 6, 2007, at 16:18, M. Warner Losh wrote:
:
: > Unfortunately, the kernel has to have a notion of time stepping around
: > a leap-second if it implements ntp.  There's no way around that that
: > isn't horribly expensive or difficult to code.  The reasons for the
: > kernel's need to know have been enumerated elsewhere...
:
: Presumably it only needs to know the next leap-second to do this, not
: the whole known table?

Yes.  ntpd or another agent tells it when leap seconds are coming.  It
doesn't need a table.  Then again, none of the broadcast time services
provide a table...

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Ashley Yakeley <[EMAIL PROTECTED]> writes:
: On Jan 6, 2007, at 08:35, M. Warner Losh wrote:
:
: > So for the foreseeable future,
: > timestamps in OSes will be a count of seconds and a fractional second
: > part.  That's not going to change anytime soon even with faster
: > machines, more memory, etc.  Too many transaction processing
: > applications demand maximum speed.
:
: That's sensible for a simple timestamp, but trying to squeeze in a
: leap-second table probably isn't such a good idea.

Unfortunately, the kernel has to have a notion of time stepping around
a leap-second if it implements ntp.  There's no way around that that
isn't horribly expensive or difficult to code.  The reasons for the
kernel's need to know have been enumerated elsewhere...

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: Warner Losh wrote:
:
: > leap seconds break that rule if one does things in UTC such that
: > the naive math just works
:
: All civil timekeeping, and most precision timekeeping, requires only
: pretty naive math.  Whatever the problem is - or is not - with leap
: seconds, it isn't the arithmetic involved.  Take a look a [EMAIL PROTECTED]
: and other BOINC projects.  Modern computers have firepower to burn in
: fluff like live 3-D screensavers.  POSIX time handling just sucks for
: no good reason.  Other system interfaces successfully implement
: significantly more stringent facilities.

But modern servers and routers don't.  Anything that makes the math
harder (more computationally expensive) can have huge effects on
performance in these areas.  That's because the math is done so often
that any little change causes big headaches.

: Expecting to be able to "naively" subtract timestamps to compute an
: accurate interval reminds me of expecting to be able to naively stuff
: pointers into integer datatypes and have nothing ever go wrong.

Well, the kernel doesn't expect to be able to do that.  Internally,
all the FreeBSD kernel does is time based on a monotonically
increasing second count since boot.  When time is returned, it is
adjusted to the right wall time.  The kernel only worries about leap
seconds when time is incremented, since the ntpd portion in the kernel
needs to return special things during the leap second.  If there were
no leapseconds, then even that computation could be eliminated.  One
might think that one could 'defer' this work to gettimeofday and
friends, but that turns out to not be possible (or at least it is much
more inefficient to do it there).

Since the interface to the kernel is time_t, there's really no chance
for the kernel to do anything smarter with leapseconds.  gettimeofday,
time and clock_gettime all return a time_t in different flavors.

In short, you are taking things out of context and drawing the wrong
conclusion about what is done.  It is these complications, which I've
had to deal with over the past 7 years, that have lead me to the
understanding of the complications.  Espeically the 'non-uniform radix
crap' that's in UTC.  It really does complicate things in a number of
places that you wouldn't think.  To dimissively suggest it is only a
problem when subtracting two numbers to get an interval time is to
completely misunderstand the complications that leapseconds introduce
into systems and the unexpected places where they pop up.  Really, it
is a lot more complicated than just the 'simple' case you've latched
onto.

: A
: strongly typed language might even overload the subtraction of UTC
: typed variables with the correct time-of-day to interval
: calculations.

Kernels aren't written in these languages.  To base one's arugments
about what the right type for time is that is predicated on these
langauges is a non-starter.

: But then, what should one expect the subtraction of
: Earth orientation values to return but some sort of angle, not an
: interval?

These are a specialized thing that kernels don't care about.

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Tony Finch <[EMAIL PROTECTED]> writes:
: On Sat, 6 Jan 2007, M. Warner Losh wrote:
: >
: > OSes usually deal with timestamps all the time for various things.  To
: > find out how much CPU to bill a process, to more mondane things.
: > Having to do all these gymnastics is going to hurt performance.
:
: That's why leap second handling should be done in userland as part of the
: conversion from clock (scalar) time to civil (broken-down) time.

Right.  And that's what makes things hard because the kernel time
clock needs to be monotonic, and leap seconds break that rule if one
does things in UTC such that the naive math just works (aka POSIX
time_t).  Some systems punt on keeping posix time internally, but have
complications for getting leapseconds right for times they return to
userland

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Ashley Yakeley <[EMAIL PROTECTED]> writes:
: On Jan 5, 2007, at 20:14, Rob Seaman wrote:
:
: > An ISO string is really overkill, MJD can fit into
: > an unsigned short for the next few decades
:
: This isn't really a good idea. Most data formats have been moving
: away from the compact towards more verbose, from binary to text to
: XML. There are good reliability and extensibility reasons for this,
: such as avoiding bit-significance order issues and the ability to
: sanity-check it just by looking at it textually.

While all these date formats are mildly interesting, they are far too
inefficient for implementation by an OS.

OSes usually deal with timestamps all the time for various things.  To
find out how much CPU to bill a process, to more mondane things.
Having to do all these gymnastics is going to hurt performance.  One
might scoff at this statement, but research into performance problems
and issues has found time and again timekeeping and timestamps to have
a surprisingly large impact.  So for the foreseeable future,
timestamps in OSes will be a count of seconds and a fractional second
part.  That's not going to change anytime soon even with faster
machines, more memory, etc.  Too many transaction processing
applications demand maximum speed.

: As the author of a library that consumes leap-second tables, my ideal
: format would look something like this: a text file with first line
: for MJD of expiration date, and each subsequent line with the MJD of
: the start of the offset period, a tab, and then the UTC-TAI seconds
: difference.

This sounds like a trivial variation on the NIST format for leapsecond
data.

Warner

Re: A lurker surfaces

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
"Daniel R. Tobias" <[EMAIL PROTECTED]> writes:
: On 2 Jan 2007 at 11:56, Ashley Yakeley wrote:
:
: > GPS is TAI. I'm not proposing abandoning TAI for those applications
: > that need it.
:
: It's a few seconds off from TAI, isn't it?  It was synchronized to
: UTC in 1980 (I think), but without subsequent leap seconds, so it's
: now different from both TAI and UTC.  They probably should just have
: used TAI if they wanted a time scale without leap seconds, rather
: than ending up creating a different one.

Yes.  There's a fixed offset between seconds in TAI and seconds in
GPS.  The GPS timescale is tied to seconds in UTC(NRO).  TAI is a
paper clock, computed after the fact, so GPS can't ever be TAI time.
However, the differences there are down in the nanosecond or so
range.

There's a big philosophical opposition to using the paper clock that
is TAI in a real-time operational timescale that GPS uses.  The
European version of GPS originally specified TAI time, but this was
changed in later revisions to be the same as GPS time.  There's an
extreme reluctance in the time community to call something without
leap seconds "TAI" or "TAI + fixed offset".  TAI means something very
specific.  That's the other problem with just using TAI, btw, but
explaining that point is very hard...

The principle time scientist made me change the description of the
output of measurment time for a product I did.  I described it as "TAI
time seconds since 1970."  Instead, he descriped it as "Number of PPS
ticks in the UTC time scale since 1972 + 63072000 + 10."
Mathematically, they work out to be the same thing, but he was
extremely resistant to calling it TAI time or using the TAI moniker
for it at all.  When I asked him about this, he said that TAI time
isn't realized in real time, but UTC is.  UTC is what one can measure
against.  Producing a number that corresponded to TAI time was OK, and
likely the least confusing thing to do (we give a second number and
UTC time in '-MM-DD HH:MM:SS Z' as well as the channel and the
measurement for that time in out output), but actually calling it TAI
would 'confuse' the really smart time geeks out in the world.  I asked
him for a reference where I could read up on this, and he shrugged and
said he just knew it and didn't know of any good write up.

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
John Cowan <[EMAIL PROTECTED]> writes:
: Warner Losh scripsit:
:
: > There's no provision for emergency leapseconds.  They just have to be
: > at the end of the month, and annoucned 8 weeks in advance.  IERS has
: > actually exceeded this mandate by announcing them ~24 weeks in advance
: > in recent history.
:
: So much the worse.  That means that if the Earth hiccups on March 7, the
: value of |DUT1| will not return to normal until May 31.

Yes.  But it would take a change in angular momementum would likely
mean that |DUT1| being a little too large would be the least of our
worries.

The earthquake that hit Indonesia last year changed the time of day by
microseconds.  What would cause a sudden jump of hundreds of
milliseconds hurts my brain to contemplate.

Warner

Re: A lurker surfaces

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Ashley Yakeley <[EMAIL PROTECTED]> writes:
: On Jan 2, 2007, at 11:40, Warner Losh wrote:
:
: > The second technical problem is that the length of a second is
: > implicitly encoded in the carrier for many of the longwave time
: > distribution stations.  10MHz is at SI seconds.  For rubber seconds,
: > the broadcast would drift into adjacent bands reserved for other
: > things.
:
: At 1000ns, the carrier would drift by 10Hz. Surely the bandwidth is
: big enough for that?

Likely.  I hasn't done the math.  I didn't know if the rubber seconds
idea was only on leapday, or for the whole year.

: > Also, GPS would have to remain in SI seconds.  The error in GPS time
: > translates directly to an error in position.  Approximately 1m/ns of
: > error (give of take a factor of 3).  Rubber seconds would require that
: > the rubber timescale be off by as much as .5s.  So GPS has to remain
: > in GPS time (UTC w/o leap seconds, basically).  That means that the
: > rubberness of the seconds would need to be broadcast in the
: > datastream.
:
: GPS is TAI. I'm not proposing abandoning TAI for those applications
: that need it.

GPS is also used for UTC today.  Many ntpd's are stratum 1 tied to a
GPS receiver.  ntpd is UTC, by definition.

The gymnastic necessary to produce UTC timing signals from GPS
(non-rubber) timing signals with rubber UTC secods would be, ummm,
very interesting.

Warner

Re: A lurker surfaces

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
[EMAIL PROTECTED] (Michael Sokolov) writes:
: The people who complain about leap seconds screwing up their interval
: time computations are usually told to use TAI.  They retort that they
: need interval time *between civil timestamps*.

Actaully, interval deltas are needed, but it is in TAI time.  There
needs to be, in the systems I've done, a timescale without leapseconds
to keep all the measurements sane.  In addition, many of the systems
I've done have other functions they do also, some of which need to
present UTC to the user.  As much of a pita that leapseconds are
today, having the converion between TAI and UTC (or put another way:
between GPS and UTC or LORAN and UTC) would make these systems
signficantly more difficult to get correct.

There's also times when our systems take in events that are
timestamped using UTC time, so we need to back correlate them to TAI
or whatever internal timescale we're using.  Some of that is because
UTC is the standard for exchanging time, part of that is because the
events in question are measured in whatever timescale is present on an
NTP server.

: To me that seems like
: what they are really measuring as "interval time" is not physical
: interval time, but how much time has elapsed *in civil society*.  Hence
: my idea of civil interval time that's completely decoupled from physical
: time and is instead defined as the turning angle of the clock on the
: Kremlin tower.

Actually, you are wrong.  The intervals is in the number of pps that
have happened, or fractions thereof.  Civil time does intrude because
that's what people use right now to know time fo day.  In the systems
I've done, you need to know both.

The requirements, as you may have noticed, for my needs aren't that
the intervals be done in TAI (we use a variant of LORAN time due to
historical accident, but with a 1970 epoch), but rather that UTC and
these time scales be convertable between one another.

Like I've said a number of times, saying 'just use TAI' isn't viable
because of the conversion issue.  Using TAI (or something with the no
leapsecond regualrity that TAI gives) is necessary for the algorithms
to work.  However, external pressures also require that some things be
done in UTC time and that some of the external sources of data use UTC
time and that needs to be back correlated to the internal timescale
that we're using.

The algorithms, btw, basically integrate frequencies of different
clocks, over time, to predict phase difference.  In this case, you
definitely want to use an interval, and not whatever weirdnesses civil
time happened to do in that interval.  Using an civil time interval
would introduce errors in algorithms.  These algorithms are used to
estimate how well the clocks are doing, but other parts of the system
need to play our UTC for things like NTPD and IRIG.

Warner

Re: A lurker surfaces

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Ashley Yakeley <[EMAIL PROTECTED]> writes:
: Software should serve human needs, not the other
: way around. Anyone needing fixed seconds should use TAI.

I think this idea would be harder to implement than the current
leapseconds.

There are many systems that need to display UTC externally, but need
to operate on a tai-like timescale internally.  Having there being a
sliding delta between them would be a nightmare.

Warner

Re: A lurker surfaces

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: Poul-Henning Kamp wrote:
:
: > Rob, If you feel uncomfortable with calling leapseconds
: > discontinuities, then we can use the term arrhythmia instead.
:
: Which raises the question of why projects requiring an interval time
: scale lacking in such arrhythmias would have selected UTC in the
: first place.

The real world often times requires both kinds of time keeping, with
correlation back and forth between all the timescales.  The customers
want to deal with times in UTC.  To get an effective, high precision
time system, you need to count seconds in a sane way (UTC isn't a sane
way for this purpose) so some conversion between this count and UTC is
needed.

The problem is that most interesting systems have a need to do things
based on UTC time of day, as well as in a time scale that has none of
the complications of the unpredictable UTC implementation we have
today.  So you can't just 'pick one' because real-world systems need
to use different ones for different things, and they also need to
translates events from one timescale to events in the other
timescale.

Let me give you a concrete example.  Let us say we are measuring a
number of atomic clocks against one another.  We get the measurements
and those measurements are in a monotonic timescale that is defined as
PPS's from some epoch (or number of 5MHz or 10MHz ticks from some
arbitrary epoch).  We get data from GPS, which has a similar
timescale.  We compute the clock states and deside that we need to
steer one of the clocks.  These algoritmns are very much elapsed time
sorts of deals.  So we schedule a time to steer it, and then note the
time when we are done steering it.  Since we cannot easily get the GPS
or timer times without interfereing with the operations of those
subsystems (maybe because the steering system and GPS system are on
dufferent cpus), we get the system time when the steer starts and when
the clock tells us the steer is complete (since a steer isn't an
instantaneous event).  Since this system also is an ntpd server, that
system time is in UTC.  Now we have to convert UTC into the internal
timescale so that the algorithms know when the steer happened and can
take that into account for their next round of measurments and
decision making.

To make this all work out well, one has to have perfect leap second
knowledge and all the special case code you have for dealing with the
arrhythmia of a leap second has to work perfectly.  Anything that can
be dobe to make things more perdictable helps out quite a bit...

Warner

Re: A lurker surfaces

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
John Cowan <[EMAIL PROTECTED]> writes:
: Tony Finch scripsit:
: > On Sun, 31 Dec 2006, John Cowan wrote:
: > >
: > > However, it's clear that UTC does not contain the sort of jumps
: > > that LCT does, where a single broken-down time may represent
: > > two different UTC seconds.
: >
: > Not if you include the timezone offset in the representation.
:
: Quite so.  Or alternatively a standard/daylight flag.  The point is,
: people usually don't.

This is the leapsecond problem in a nutshell:  While it is possible to
keep enough information around in the representation of time, most
people opt not to do so.  It complicates the representation of time,
and is one (or more) pieces of information that need to be carried
around, mostly just to handle weird edge cases.  Sure, you can do it,
and some people try.  However, there's no standardized way to do so,
and people re-invent it wrong over and over again.

Another poster I think was right: ntpd (and the underlying OS) hides a
lot of these sins.  Either by just twitterpating at the leapsecond in
some way (time stands still, time jumps back a little, etc), or for a
period of time around the leapsecond (by sticking its fingers in its
ears, singing lalala and changing the size of a second to get through
it a millisecond at a time).

Warner

Re: A lurker surfaces

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: Just a reminder that UTC has no - none - nada - discontinuities.

At the very least, the TAI-UTC difference is discontinuous with jumps
at the leap seconds.  One could easily suggest that 'UTC has
discontinuities' really is a shorthand way of saying this.  One could
also call them irregularities, like the time scale is constipated, as
well.

And the variable radix seems to be to be a crude way to define away
the problem.  You never know, unless you look it up in a table, when
to do the variation in the radix.

: Various computer mis-implementations may, but the standard is very
: carefully constructed to avoid spring-forward or fall-back gaps or do-
: overs.

Well, if you count the variable radix notation as being 'continuous'
then maybe you are right.  However, you never know when the radix is
variable, hence the assertion on many people's part that UTC is
discontinuous.

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
John Cowan <[EMAIL PROTECTED]> writes:
: In this case there are really two questions:  how much it would
: cost to loosen DUT1 but leave it bounded, and how much it would
: cost if it were only statistically, not absolutely, bounded.

We've accepted a statistical solution for the leap-day problem now for
about 500 years.  There are minor variations, and over the next 10's
of thousands of years this simple solution may not prove workable (it
will be off by more than a day on the average in about 8,000 years if
nothing is done).  Right now winter solstice can be off by over a day
from the mean either direction, yet over the long haul things average
out.  There's no call for there to be leap days scheduled at the whim
of the astronomers anymore...  Seems like a logical leap for leap
seconds to follow, if the costs aren't prohibitive.  Chances are that
one person knows all the users of time that still need DUT1 to be less
than 1s.

One big area of change would be the time and frequency stations, like
WWV, since they currently broadcast DUT1, or variations of it.  But
the correction they broadcast is good only to 100ms typically
(time-nuts@ have documented cases where it was worse than 100ms when
some national time lab didn't update the data transmitted when they
were supposed to), so maybe some other means of distribution is
necessary...  And is 100ms really good enough?

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: M. Warner Losh wrote:
:
: > Let's turn the question around.  What would the harm be if |DUT1| were
: > 1.1s?  1.5s?  2.0s?  Contrast this with the harm and difficulty that
: > the current 6 month scheduling window affords.
:
: Indeed.  Go for it.  I look forward to reading your report.  Who and
: what interests are adversely affected in each case?  How are these
: effects mitigated as a function of the limit on DUT1?  Also, contrast
: what benefits accrue.  One would think that the responsibility for
: quantifying the implications of a change to a standard would fall on
: the parties proposing said change.

I know the benefits, but nobody has yet produced a study on why 0.9s
was chosen.  Some vague rumblings about astronomical software needing
to be rewritten, and some vague notions about 'hearty souls' that do
celestial navigation with atomic clocks for high accuracy have been
offered, but who really would be hurt by this change?  Since you are
an astronomer, I thought you might be able to give some insight into
at least one of these users of time.  Daylight savings time and time
zones prove that society at large has a very high tolerance for
variations between the mean solar time at an arbitrary location, maybe
hundreds of miles away, and the local time.  Only specialized users of
time would be affected.  Who are they and how do we find out the cost
of change?

The world has changed from having maybe a few dozen or tens of dozen
atomic clocks when leap seconds started to having GPS with cheap,
disciplined oscillators that number in the hundreds of thousands.
These little devices have obviated the need, in many cases, for
celestial navigation.  Given that change, the cost benefit analysis
that was done in 1972 likely needs updating.

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
John Cowan <[EMAIL PROTECTED]> writes:
: Rob Seaman scripsit:
:
: > >I don't care if you want to implement leap-milliseconds, as long
: > >as you tell me 10 years in advance when they happen.
: >
: > Again - with no intent to minimize the issues - what supports this
: > assertion?  Is there any reason to believe that 10 years advance notice
: > would encourage projects and vendors to do anything other than ignore
: > the requirement entirely?  A statement that 10 years, or 600 years,
: > notice is all that is needed to resolve all the problems, smooth over
: > all the complications, is entirely too glib.
:
: You are confusing the question of fixity (which is what notice is
: about) with granularity.  It's true that probably no one would bother
: to implement the ALHP.  However, if computer technologists were handed a
: list of leap seconds from now until 2015, and told "Implement these," it
: wouldn't matter how many or how few leap seconds there were.  But since
: you astronomers insist on a fixed maximum for |DUT1|, no such table
: can exist.
:
: The proposal is this:  look at the trends, take your best shot at
: working out a leap-year schedule for 10 years in the future, and then
: live with it.

Let's turn the question around.  What would the harm be if |DUT1| were
1.1s?  1.5s?  2.0s?  Contrast this with the harm and difficulty that
the current 6 month scheduling window affords.

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
"Poul-Henning Kamp" <[EMAIL PROTECTED]> writes:
: We can get that only by increasing the DUT tolerance.

Yes.  Letting DUT be bounded by +- 10s rather than +- 0.9s is going to
affect a tiny number of users.  Having leapseconds only known 6 months
in advance affects a lot more users...

: As Warner, I and others have repeatedly emphasized:  It is not the
: step size that is the problem, it is the 6 month warning.
:
: I don't care if you want to implement leap-milliseconds, as long
: as you tell me 10 years in advance when they happen.

While my preference would be to never see another leap second, I know
that's likely not an option.

For many practical reasons, scheduling them out 10 years would help
ameliorate the costs of leap seconds and allow for better long term
planning.

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: John Cowan wrote:
:
: > I assume you mean 23-hour or 25-hour LCT days?  True.  It does work
: > against UCT days, though, since they are uniformly 1440 minutes long.
:
: Not should leap hours replace leap seconds.

And avoiding the ugly 61 or 59 second minutes to define away the
problem...

Warner

Re: Design - a Tufte decision

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: On Dec 27, 2006, at 12:02 AM, M. Warner Losh wrote:
:
: > Calculating time intervals for times 6+ months in the future can be
: > the least of one's worries when one tries to code up a library to deal
: > gracefully with these different failure modes.  The trivial case where
: > one has perfect knowledge of TAI-UTC and one can keep that knowledge
: > current is very simple in comparison.  Dealing with this case is very
: > simple, and is the way most people think about leap seconds.  But
: > dealing with the edge cases can be difficult because there are so
: > many, and so many that people forget to test or conceive of until the
: > call from the field comes in with a failure...
:
: A lot of these "edge" cases are really firmly centered in issues of
: real-time programming.  Few versions of Unix are equipped to deal
: with real-time issues in even a rudimentary fashion.  In any event,
: these cases have very little to do with leap seconds or any other
: aspects of the representation of time quantities.

Actually, they can have a great deal to do with it.  The absolute need
to deal properly with these edge cases can, at times, lead to design
decisions that aren't the 'simplest'.  I also disagree that Unix is
ill-equipted to deal with real-time issues.  While one does need to
take care, I have successfully deployed over a dozen different types
of timing systems over the past 7 years on FreeBSD.  My familiarity
with these edge cases, the customers that expect them to work and the
different sources of time has been learned through these experiences.
The nature of these customers also means that they are less well
connected to networks than one would like, which also complicates
matters.

The salient point from my ramblings is that different time scales may
become available at different points in time because data about them
is available at different points in time to the application/os.  If
there's only one time scale, then once you know the time, you are
done.  If there's more than one, then you need to either discover both
times, or you need to discover one time and the transforms to that
time to know the others.

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Ashley Yakeley <[EMAIL PROTECTED]> writes:
: On Dec 27, 2006, at 06:29, Poul-Henning Kamp wrote:
:
: > That's a pretty bad format.  Computers are binary and having
: > pseudo-decimal fields like tv_usec in timeval, tv_nsec in timespec
: > and picoseconds in Haskell is both inefficient and stupid.
: >
: > The fractional part should be a binary field, so that the width
: > can be adjusted to whatever precision and wordsize is relevant.
:
: It's impossible to accurately represent a millisecond using binary
: fractions. That would be unacceptable for most sub-second use.

Ummm, it is possible to represent sub-picoseconds accurately using
binary fractions.  what are you talking about?

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Tony Finch <[EMAIL PROTECTED]> writes:
: > The nature of the uncertainty is very different.  The uncertainty of
: > future UTC can be managed, but for timezones the only sane path is to
: > eschew their use entirely.
:
: That isn't possible for applications like appointment books and job
: schedulers. As Warner suggests, you need to calculate future times
: provisionally, and in a way that insulates you from discontinuities.
: For example, "same time tomorrow" instead of "in 86400 seconds".

And 'same time tomorrow' might not exist depending on the timescale
that's used.  Cron runs into this problem (and has solved it) because
it runs in localtime which has hours repeated or omitted twice a
year.  Leap seconds are handled in an ad-hoc way as well, typically
the last second of the day is repeated for a positive leap-second so
cron jobs on fast machines that run at midnight UTC don't run twice.

There is a more subtle point that I should mention explicitly.  The
folks that say "do everything internally as TAI time" tend to forget
that to do that one must restrict one's self in a number of important
ways.  Because certain things are almost always in UTC time (IRIG
timecode, measurment of atomic clocks for BIPM UTC calculations, etc),
one must take care to know that when you convert a time into TAI from
UTC that you can then make the inverse computation and arrive back at
the same time.  One must also know the appropriate TAI-UTC offset for
that time.  Typically, one doesn't schedule things in these kinds of
systems too far in the future, and one's historical data processing is
handled offline where perfect historical knowledge of leap is
available for dealing with long or abnormal intervals.

Of course, needing to know TAI-UTC offsets leads one to interesting
situations.  What does one do if one has TAI time, but not UTC and a
conversion is asked for?  Is it right to pause that thread until it is
available, or is an exception right?  Pausing the thread means that
less code has to deal with this ugliness and makes it so the
programmer doesn't have to know that time1 - time2 might fail
randomly.  However, pausing can lead to deadlocks if the thread that
acquires this knowledge pauses.  Throws in an exception avoids the
deadlock case, but makes every single time expression prone to having
to deal with exceptions, which can be tedious and time consuming to
code every single time.  In C++ this can be really bad because
operators are often overloaded for time libraries to make the time
types look like native.

There are a number of sources of phase (rtc clock, UTC time, TAI time,
GPS time, NTP servers) and a number of sources of TAI-UTC (internet
ftp, GPS almanac, Loran Data Channel, local data file).  The arrival
of the data from these sources may be asynchronous, and the cached
value may be too old to be useful (for spares that have been sitting
on the shelf for a few years).  The data sources might be unavailable
because the device is cold starting, or because the antenna cable is
damanged.  These different scenarios have different "right" things to
do.  The data can also expire because that GPS antenna has been
damanged for too long and you don't know the next leap second is
coming...

Calculating time intervals for times 6+ months in the future can be
the least of one's worries when one tries to code up a library to deal
gracefully with these different failure modes.  The trivial case where
one has perfect knowledge of TAI-UTC and one can keep that knowledge
current is very simple in comparison.  Dealing with this case is very
simple, and is the way most people think about leap seconds.  But
dealing with the edge cases can be difficult because there are so
many, and so many that people forget to test or conceive of until the
call from the field comes in with a failure...

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
John Cowan <[EMAIL PROTECTED]> writes:
: M. Warner Losh scripsit:
:
: > If I was to compute the number of seconds between Jan 1, 2007 0:0:0 and
: > Dec 31, 2008 23:59:50, the answer is 63071990 or 63071991 or 63071992.
: > We have no way of knowing today how many seconds are in that interval.
: > We do know the answer is one of the above.
:
: Technically 63071999 and 63071998 are also possibilities, though
: I admit they are unlikely.

you are correct...  this simple stuff sure is hard...

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Zefram <[EMAIL PROTECTED]> writes:
: M. Warner Losh wrote:
: >Actually, the real answer is domain specific.  There are many things
: >that don't really care (when do I need to make the next 20 house
: >payments, off by one second just doesn't matter at all since
: >transcations are batched on a daily basis a few days early).
:
: In that case you don't really want a TAI<->UTC conversion.  You want
: an estimate of TAI<->UT1.  Possibly you'd prefer an exact TAI<->UTC
: conversion but want to fall back on the planetary rotation estimate in
: the event that UTC isn't defined that far yet.  The point is that you're
: asking a different question from the one that elicits "I don't know".

Well, if I compute an interval, using UTC times, then I can go back
several decates before the results start to be suspect.  Going
forward, you can get up to almost 12 months error free at some times
of the year, and a little over 6 months error free (assuming
instantaneous distribution of circular C).  But going out a year
you'll have at least one leap second opportunity.  Going out 2 years
gives you 3, etc.  This allows one to compute these long intervals
with only a small error.  If I was to compute the number of seconds
between Jan 1, 2007 0:0:0 and Dec 31, 2008 23:59:50, the answer is
63071990 or 63071991 or 63071992.  We have no way of knowing today how
many seconds are in that interval.  We do know the answer is one of
the above.  The error in not knowing is tiny, about a 6 parts in 10^7.
Large enough to matter for some things, not large enough to matter for
others.

There's been little work on having variable precision math of this
nature where the computation depends on the numbers involved and a set
of rules.  The precision work for floating point does come closest,
because there has been a lot of effort paid to analyzing the different
sources of error, but there's little work that's been done on
computing the error in parallel to the actual computation, wrapping
that up into convenient data structures and allowing the programmer to
test the results for an acceptle error (or rather in this case
uncertainty).

In all the high precision time work I've done, the biggest, most
overriding problem has been: What's the leap second value for the
immediate past (< 1 day) or near future (< 1 month).  Very little
matters outside of those parameters.  UTC time is important for IRIG
(and other time codes), UI display, and atomic clock coorinated
measurements (more generally, any coordinated event), but little
else.  Some time scale that one can use to pace the system, or to make
measurements against is needed, but TAI or TAI-like timescales will
fit the bill.  The hard part is when you need to convert from one to
the other of the time scales and a realization of the limits to doing
that.

So the answer to my above question isn't "I don't know" but rather
63071991 +- 1 given the current leap second practices.  It would take
a lot longer than a year to notice different behavior in rotations and
change the rules, but factoring that in would be a crap shoot at best.

The important thing to realize also in constructing time libraries is
that I can have a TAI time in the future that doesn't have a
corresponding UTC time now, but will have one the closer we get to it.
And vice versa.  I can have a UTC time (say 23:59:50 Dec 31, 2008)
that has no TAI time associated with it yet, but will at some point in
the future before that time.

: >Writing a library that copes with both types of users can be
: >problematical at best...
:
: I think a different library is required.  One is about atomic timekeeping,
: the other is about astronomy.

Well, one is about atomic timekeeping.  The other is the civil
approximation of astronomy, since a purely astronomical answer depends
on exactly where you are, seasonal variations, etc.

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Zefram <[EMAIL PROTECTED]> writes:
: Steve Allen wrote:
: > and if I continue that practice
: >I can later give you an estimate of how wrong I was when I told you.
:
: This is something that's missing from current chronological APIs.
: It needs to be formalised.

time_t is so totally broken, it isn't funny.  That's the closest thing
to a standardized API there is for time.  All others are stuff folks
have done here or there, but they aren't universal enough to be
considered.

Too bad the problems with time_t are well known, well discussed and
well enumerated.  Or rather I should say "too bad POSIX doesn't care
enough to change it" since the cost of changing time_t is huge...

Also, things in TAI time don't care either. 1 day, 1 year, 100 years
don't matter to TAI.  Periodic things that happen at a given time of
day (UTC) are the only things that do care.

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Zefram <[EMAIL PROTECTED]> writes:
: Keith Winstein wrote:
: >  what should a library do when a program asks to
: >convert between UTC and TAI for some instant further than six months in
: >the future?
:
: Refuse, of course.  The correct answer is "I don't know".  You might
: know if asked later.

Actually, the real answer is domain specific.  There are many things
that don't really care (when do I need to make the next 20 house
payments, off by one second just doesn't matter at all since
transcations are batched on a daily basis a few days early).  There
are other things that do care (when do I fire the lasers at the
target).  For those things, however, it is rare to be computing time
into the future that far.

Writing a library that copes with both types of users can be
problematical at best...

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Keith Winstein <[EMAIL PROTECTED]> writes:
: The GNU C library includes a leap-second table in its timezone/leapseconds
: file in the source package. This gets compiled in to the "right" (that is,
: non-POSIX) zoneinfo files in, e.g., /usr/share/zoneinfo/right/... .
:
: If your users have an up-to-date GNU libc, they can get the value of
: TAI-UTC at any point since 1970 using C library functions. Here's a sample
: C program that shows how to do it.

The problem with this approach is that it fails to work for long
running programs that need to know about leapseconds.  The zonefiles
aren't re-read when they change, and even if they were, updating the
zonefiles to keep them up to date presents some interesting
challanges.

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Ashley Yakeley <[EMAIL PROTECTED]> writes:
: Hello, I just joined the leap seconds list. I wrote the "time"
: package for the Haskell programming language.
: 
:
: I include code for making conversions between TAI and UTC, given a
: leap-second table. I also include code for parsing a tai-utc.dat file
: into a leap-second table. I do not, however, simply include a leap-
: second table as any program compiled with one would be out of date
: after six months.
:
: This has led me to consider run-time methods of obtaining leap-second
: information, and how that might be standardised for use by software
: authors. For instance, I imagine a software package that established
: a well-known place in the directory hierarchy to find tai-utc.dat and
: perhaps other "earth data" files, and was responsible for keeping
: them up to date. Other software could then make use of the package
: without each having to implement their own mechanism.
:
: Does this strike people as worthwhile? Does anyone know of an
: existing effort along these lines?

One can find the leap data from NIST that's up to date:
ftp://time.nist.org/leap-seconds.list
I believe that there are other places for this information as well.

The problem is that often times one has systems that aren't guarnateed
to be connected to the internet, but are connected to GPS so that this
number can be obtained.  Other problems arise from systems that are
intermittently operational (like in sparing situations) that may be
off over many leap seconds...

Warner

Re: Equitable estoppel

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
John E Hein <[EMAIL PROTECTED]> writes:
: Peter Vince wrote at 20:15 + on Dec 18, 2006:
:  > >For the moment, if leap seconds is to be abbandoned, I would favour the 
leap
:  > >minute instead.
:  >
:  > Is that not sitting on the fence, and ending up with the worst of
:  > both worlds? It is neither as precise as leap (micro, milli, or
:  > whole) seconds, nor as long term as leap hours. It would put off the
:  > updates for only a few tens of years, by which time experience of
:  > them would have been lost, and our descendants would end up with Y2K
:  > type problems.
:  >
:  > Peter Vince
:
: One of the problems described on this list in the past is that of leap
: tables in devices designed to operate over 10-20 years without
: availability of leap second updates.  Being able to rely on a table
: that is good for that long makes the job of the designer much easier.

As would a schedule for the next 10 years, say something like:

Dec 2007
Jun 2009
Jun 2010
Dec 2012
Dec 2013
Jun 2015
Jun 2016
Dec 2018
...

The state of the art is such that we have a high confidence of being
able to predict things out at least 18 if not 36 months and still
keeping within the 0.9s performance goal.  If that were loosened a
little, then we could schedule things out 10 years or so.  It could
even be phased in so that the next two leap second opportunities could
be scheduled after the first of the year, and the two following that
after June, etc until.  While annoying, at least it would be known far
in advance and start to feel more like a leap day rather than a total
pita.

: In this respect, leap minutes would be "better" than leap seconds.

Then a leap hour would be 60 times better[*]  Of course, a leap minute
would happen once or twice a lifetime.  A leap hour happens once every
100 generations or so...

: Do not mistake this comment as an advocation of leap minutes.

Ditto for me...

Warner

[*] And 3600 times more disruptive and less likely to get right...

Re: Equitable estoppel

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Steve Allen <[EMAIL PROTECTED]> writes:
: ITU rules which say their documents must be sold, not given away.  All

A simple web search shows that at least the UTC/leap second
recommendations are readily available in electronic form...

Warner

Re: what time is it, legally?

[M. Warner Losh]

I view the same data differently.  I see it as a progression:

Local Solar time -> mean local solar time ->
  timezone as mean local time at one point used for many -> UTC -> ???

Clearly, we're moving away from solar time and towards something else.
Our ability to tell time has exceeded the earth's ability to be
perfectly periodic.  The time will come when we need to change
something as the current system is doomed to failure, sooner or
later.  I feel that attempts to tie time to the sun and to also have a
precice notion of a second are fundamentally at odds with one another
and the current system tries very hard to paper over the differences.
No doubt they can be papered over for years to come, but the basic
physics of platentary rotation with a large satellite stand against
it.  I'm unsure what that "something else" should be, and I'm not sure
that we're in a good position to know with certainty what would be a
good solution.  I just know that invoking variable radix counting
schemes has its limitations and difficulties...

Anyway, I know others view it differently, but if it was easy and
obvious there wouldn't be such a divergence of opinion, eh?

Warner

Re: independence day

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
"Poul-Henning Kamp" <[EMAIL PROTECTED]> writes:
: In message <[EMAIL PROTECTED]>, Steve Allen writes:
:
: >In the middle of May some text about legal time in the US was
: >introduced into a US Senate bill regarding funding for NSF and NIST.
: >See section 508 of S.2802 introduced 2006-05-15, e.g.
: >http://thomas.loc.gov/cgi-bin/query/z?c109:S.2802:
:
: `(b) COORDINATED UNIVERSAL TIME DEFINED- In this section,
: the term `Coordinated Universal Time' means the time scale
: maintained through the General Conference of Weights and
: Measures and interpreted or modified for the United States
: by the Secretary of Commerce.'.
:
: That could sound like the drilling of a loophole :-)

As has been pointed out in the past, the Secretary of Commerce has had
the ability to define mean solar time to mean UTC (or something else,
if they felt the urge)...  I think this is just another attempt to
keep their options open, like they have now...

Warner

Re: building consensus

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: This does not express a complete algorithm, of course, since it is
: not tied to the days of the week.  One presume the zero point of 1875
: was selected as a practical compromise with historical realities.
: One questions whether the Gregorian calendar can be used to express
: dates before 1582-10-15, and it is ambiguous for large parts of the
: world until the mid-eighteenth century.  Still in 1875, it had not
: reached all locales.  Etc.

This was answered earlier in the thread.  One can project the
Gregorian calendar back in time from the above date and use that for
dating (there's a special name for when you do this that can be found
in the archives).  However, most historians use Julian dates back from
there (with the same backward projected dates when time is before the
Julian calendar was established).

Warner

Re: building consensus

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
John Cowan <[EMAIL PROTECTED]> writes:
: M. Warner Losh scripsit:
:
: > : The designers of Posix time thought it was more important to preserve
: > : the property that dividing the difference between two time_t values
: > : by 60, 3600, 86400 would give minutes, hours, days.
: >
: > That's the one property that Posix time_t does not have.  The
: > difference between time_t's that cross a leap second are off by one
: > second, and therefore do not start with the right answer to do the
: > division...
:
: I expressed myself badly.  My point is that if you have a Posix time_t
: representing 11:22:33 UTC on a certain day, and you add 86400 to that
: time_t, you will get the Posix representation of 11:22:33 UTC on the
: following day, whether a leap second intervenes or not.  This is a valuable
: property, many existing programs depended on it, and the authors of the
: Posix spec preserved it at the expense of having a distinct representation
: for each UTC second.

Yes.  To find a second absolute time that represents the same wall
time a minute/hour/day later is why posix time_t has this property.
This is abstime + delta, which is a little different than the
difference between two time_t.

: You may call this position wrong (and I have done so), but it is
: unquestionably defensible.

Differences in time_t are adjusted by leapseconds.  This makes naive
math work for the ABSTIME+delta case, but breaks the difference case
when you want an actual elapased time.  It is an engineering tradeoff,
but an inconvenient one for the problem domains that I tend to have to
solve.

Warner

Re: building consensus

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
John Cowan <[EMAIL PROTECTED]> writes:
: Zefram scripsit:
:
: > >If this means that leap secounds and leap days are analogous, then I
: > >suppose so.  If it means something else, I don't understand it.
: >
: > That's what I meant.  Can you suggest a clearer wording?
:
: "Leap seconds (after 1972) are closely analogous to leap days."

"Leap seconds (after 1972) are closely analogous to irregularlly
 happening leap days."

might be better, since leap days have a rule, while leap seconds are
scheduled.

: > Being ambiguous between adjacent seconds seems inherently faulty to me.
:
: The designers of Posix time thought it was more important to preserve
: the property that dividing the difference between two time_t values
: by 60, 3600, 86400 would give minutes, hours, days.

That's the one property that Posix time_t does not have.  The
difference between time_t's that cross a leap second are off by one
second, and therefore do not start with the right answer to do the
division...

POSIX time_t definition effecitvely omits leap seconds from the time
scale (by playing them twice, or pretending the second happened), the
difference between two time_t's always gives a duration adjusted for
the leap seconds that happen, rather than the actual duration.

: > Are you thinking of linear counts such as POSIX time, where the
: > representation is ambiguous?  I was implicitly excluding those, on the
: > grounds that they don't count as a "representation".  It's also not
: > "linear".
:
: No, it isn't.  But that doesn't mean you *can't* construct a numerical
: representation of UTC time: say, the number of UTC seconds since
: 1972-01-01T00:00:00Z.

It would be better to say the number of SI seconds since 1972 rather
than UTC seconds, I think.  I use a timescale like this at work to
ensure that duration calcuations over leap secondsd are correct.

Warner

Re: building consensus

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: Warner Losh objects:
:
: >> There are several doughty people here who happen to have that
: >> opinion, but they abide with us mortals outside the time lords'
: >> hushed inner sanctum.
: >
: > I have spent much time explaining why leap seconds cause real
: > problems in real applications, only to be insulted like this.
:
: Sincere apologies for my awkward statement.  Dictionary.com defines
: "doughty" as "marked by stouthearted courage; brave".  I wasn't
: questioning the knowledge or passion of folks holding views that
: differ from my own.  Rather I was attempting to question whether
: anybody actively participating on this list - holding whatever view -
: is also participating in ITU discussions.
:
: I see that Mr. Cowan has also parsed my admittedly opaque remarks.

Yes.  I'm sorry I was so easily offended.  Please accept my appologies
for my hasty words.

Warner

Re: building consensus

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
John Cowan <[EMAIL PROTECTED]> writes:
: M. Warner Losh scripsit:
:
: > In message: <[EMAIL PROTECTED]>
: > Rob Seaman <[EMAIL PROTECTED]> writes:
: > : Actually, this list is not a "discussion" per se.  If we simplify the
: > : positions - just for the sake of argument here - to "leap second yes"
: > : and "leap second no", the reality is that the folks pushing the "leap
: > : second no" position have never engaged with this list.  There are
: > : several doughty people here who happen to have that opinion, but they
: > : abide with us mortals outside the time lords' hushed inner sanctum.
: >
: > What an amaizingly unhelpful and offsensive statement.  I have spent
: > much time explaining why leap seconds cause real problems in real
: > applications, only to be insulted like this.
:
: I believe you have misread Rob's remark, though I concede that it was
: easy to misread.  I believe Rob meant that the people who are pushing
: "leap seconds no" in *official* channels are not to be found on this list.
: That being so, the "leap seconds yes" folks are unable to challenge them
: or persuade them otherwise.
:
: You and I, on the other hand, fall into the "doughty people here" group.

Maybe I did misread them.  I've been sick the past three days, so
maybe we can chalk it up to that and I'll offer my oppologies for
having such a thin skin.

Wanrer

Re: building consensus

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: Actually, this list is not a "discussion" per se.  If we simplify the
: positions - just for the sake of argument here - to "leap second yes"
: and "leap second no", the reality is that the folks pushing the "leap
: second no" position have never engaged with this list.  There are
: several doughty people here who happen to have that opinion, but they
: abide with us mortals outside the time lords' hushed inner sanctum.

What an amaizingly unhelpful and offsensive statement.  I have spent
much time explaining why leap seconds cause real problems in real
applications, only to be insulted like this.

Warner

Re: ideas for new UTC rules

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: Simply allow the IERS to announce any number of leap seconds
: in advance extending over any time horizon - and yes - occurring
: at the end of any month.  If predictability is the goal, relaxing
: unnecessary constraints is the solution.

There's a lot of timing gear that will fail if leap second is inserted
not at the end of June or December, not least ntp.

Warner

Re: An immodest proposal

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Neal McBurnett <[EMAIL PROTECTED]> writes:
: > UTC time stamps in NTP are ambiguous.  TAI ones are not.  UTC time
: > stamps do not convey enough information to properly implement things
: > like intervals, while TAI ones do.  The NTPNG stuff that I've seen
: > appears to consider these problems as worthy of needing a solution and
: > they plan on solving them.  It isn't rocket science, but one has to
: > divorce ones self from the chauvinistic view that UTC is always best.
: > For time exchange, it is not the best, and has many problems around
: > the edges.
:
: Yes it is messy, but the tradeoffs are complex.  I don't think the
: latest drafts specify NTP timestamps.  I suspect they still rely on
: the leap second bit to disambiguate the timestamp on a leap second,
: but I haven't checked recently.  They are all linked to from the
: charter page I noted above.

But ntp time stamps to not have a leap second bit.  To be precice, it
is a leap indicator that's in the packet.  This makes it impossible to
represent times that straddle the leap event, since there are 4
different time stamps involved...  I guess in those cases, the packets
should be discarded and tried again later...

But you are right.  I did misremeber that it used TAI time.  The
drafts still specify time in UTC.  I could have sworn I saw something
that said otherwise, but I can't seem to find it now.

: > Doing NTP with TAI (and the implied requirement for DTAI) doesn't
: > change what time is displayed for users.  It does make it *MUCH*
: > easier to get leap seconds right for those users that need them.
: > Anything else is madness.  UTC is a display convention, not a sane[*]
: > counting convention.
:
: I think that changing to a different over-the-wire timestamp epoch
: would just add to the confusion, not make things simpler in practice.

TAI is the same epoch as UTC.  It is just not adjusted for leap
events.  However, your point that the second numbering would be
different and might cause confusion is still valid.

: People still need to know UTC, and transmitting the leap second table
: info, especially via autokey, is much more complex than the basic
: protocol.  But at least this is a standards process conducted in the
: open, where you can just get involved directly if you have something
: to add.

Autokey is interesting, but it is way too complex for something as
simple as leap second propigation

Warner

Re: An immodest proposal

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: On Feb 14, 2006, at 12:50 PM, Markus Kuhn wrote:
:
: > You can, of course, define, publish, implement, and promote a new
: > version (4?) of NTP that can also diseminate TAI, EOPs, leap-second
: > tables, and other good things. I'm all for it.
:
: But why are you for it?  Before investing large amounts of time and
: money in developing and deploying a large new timekeeping system,
: wouldn't one want to invest smaller amounts in exploring the issues
: and options?  Heck - one has to imagine that a number of successful
: grant applications are lurking around here somewhere.  Time is an
: issue that cuts across every funding agency out there.

UTC time stamps in NTP are ambiguous.  TAI ones are not.  UTC time
stamps do not convey enough information to properly implement things
like intervals, while TAI ones do.  The NTPNG stuff that I've seen
appears to consider these problems as worthy of needing a solution and
they plan on solving them.  It isn't rocket science, but one has to
divorce ones self from the chauvinistic view that UTC is always best.
For time exchange, it is not the best, and has many problems around
the edges.

Doing NTP with TAI (and the implied requirement for DTAI) doesn't
change what time is displayed for users.  It does make it *MUCH*
easier to get leap seconds right for those users that need them.
Anything else is madness.  UTC is a display convention, not a sane[*]
counting convention.

Warner

[*] All variable radix counting conventions are insane by (my humble)
definition :-).

Re: An immodest proposal

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Neal McBurnett <[EMAIL PROTECTED]> writes:
: On Mon, Feb 06, 2006 at 10:37:37AM -0700, Rob Seaman wrote:
: >1)  TAI can be recovered from UTC given a table of DTAI.
: >2)  NTP can convey TAI as simply as UTC.
: >3)  Deploy a small network of NTP servers to keep TAI, not UTC.
: >4)  NTP client machines could therefore trivially select between TAI and
: >UTC by subscribing to different servers.
: >5)  This would provide an unbiased experimental sandbox for civil
: >timekeeping issues.
:
: Since TAI can be recovered from UTC, and hosts that use NTP know UTC,
: those hosts can serve any time they want to clients on that host.  I
: see little reason to change the particular timestamps that are used in
: the NTP protocol, an less reason to have two different timestamps
: within NTP, or to require users on ntp-using hosts to switch their
: ntp servers just to get a different timescale.

ntp time stampes are ambiguous at leap seconds (the leap indicator
bits aren't part of the time stamp, exactly), which is a good reason
to change them.  However, the cost to change them could be quite high
if done in an incompatible manner.

: That's why I support the "Fixing POSIX time" proposal from
: Poul-Henning Kamp.  This can be implemented with the latest NTP
: daemons.  The operating systems would seem to just need the right sort
: of CLOCK_NTP definition for /usr/include/linux/time.h or equivalent,
: and the appropriate conversion routines as he outlined.

Agreed.

: I'd like to see a standard for NTP that includes the TAI information
: that is now passed in non-standard formats, but that is a separable
: issue.

I believe that the new draft standard of ntp uses TAI time for its
time exchange, and explicitly includes DTAI information in a standard
format.  Since the protocol version changes, there's no ambiguity and
that's a good time to change things like this.  However, I've not read
the latest ntpng drafts to confirm this.

Warner

Re: Comparing Time Scales

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Tim Shepard <[EMAIL PROTECTED]> writes:
: But there's a difference between NTP timestamps, and the details of
: the implementation of a system which uses NTP for synchronization.

Ah, I was getting the two confused and didn't quite realize it until
your clarification.  Thanks!

Warner

Re: Comparing Time Scales

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
        M. Warner Losh <[EMAIL PROTECTED]> writes:
: In message: <[EMAIL PROTECTED]>
: James Maynard <[EMAIL PROTECTED]> writes:
: : M. Warner Losh wrote:
: : > In message: <[EMAIL PROTECTED]>
: : > James Maynard <[EMAIL PROTECTED]> writes:
: : > : Thanks, guys, for your feedback. Here's another iteration.
: : > :
: : > : The numbering of NTP seconds in the vicinity of a leap second seems to
: : > : differ from one document to another.  Here I follow the NTP (version 3)
: : > : specification, RFC 1305, in which the Leap Indicator (sys.leap,
: : > : peer.leap, pkt.leap) is 01 if a positive leap second is to occur at the
: : > : end of the current UTC day, and 00 if no leap second is pending.
: : > :
: : > : UTC = 2005-12-31 23:59:59, NTP seconds = 3 345 062 399, LS_pending = 01
: : > : UTC = 2005-12-31 23:59:60, NTP seconds = 3 345 062 400, LS_pending = 01
: : > : UTC = 2006-01-01 00:00:00, NTP seconds = 3 345 062 400, LS_pending = 00
: : >
: : >
: : >
: : In the NTP time scale, it is the last second of the UTC day, NTP second
: : number 3 124 137 600, that is repeated. The first instance of this NTP
: : second number is the leap second, during which (TAI - UTC) = 31 and the
: : NTP Leap Indicator is 01.  The second instance of this NTP second number
: : is the the first second of the new UTC day, during which (TAI - UTC) =
: : 32 and the NTP leap indicator is 00.  You can distinguish the leap
: : second (while it is occuring) by the fact that it is the last second of
: : the UTC day (time of UTC day = 23:59:60) and that the NTP Leap Indicator
: : is 01. You can distinguish the first second of the new UTC day by the
: : fact that, of the two seconds with the same NTP second number, it is the
: : one that has the NTP leap indicator set back to 00.
: :
: : Note that according to the write-up to which you referred me, it is
: : NTP second number 3,124,137,600 (not 3,124,137,599) that is repeated.
: : NTP second number 3,124,137,599 is the *next-to-last* second of the day,
: : not the *last* second of the day.)
:
: No.  The article specifically says that after it the system time gets
: to ,600, it is decremented by one and there is specific code in the
: code that returns the system time to applications that makes it stand
: still.  The second isn't *NOT* repeated.  Repeat: The second is *NOT*
: repeated in what they said.  Time stands still outside of the kernel,
: while inside the kernel the last second of the day *IS* repeated,
: hence the need for the limiter that the article talks about.
:
: Not all kernels keep time standing nearly still during the leap second
: (since that has other bad effects).  Some expose this decrement to the
: users.  The highlighed part of what I quoted said exactly this.
:
: I've actually implemented this for FreeBSD.  You are arguing theory,
: and I'm arguing the fine points of an actual, real implementation.

More specifically, read the text I highlighted:

: : >>>The obvious question raised by this scenario is what happens during
: : >>>the leap second itself when NTP time stops and the clock remains
: : >>>unchanged. If the precision time kernel modifications have been
: : >>>implemented, the kernel includes a state machine that implements the
: : >>>actions required by the scenario. [[ At the first occurrence of second
-
: : >>>3,124,137,600, the system clock is stepped backward one
   ---
: : >>>second.]] However, the routine that actually reads the clock is
   ---
: : >>>constrained never to step backwards, unless the step is significantly
: : >>>larger than one second, which might occur due to explicit operator
: : >>>direction. In this design time stands still during the leap second,
: : >>>but is correct commencing with the next second.

Notice the [[ ]] text.  When it becomes 400, the system time is
decremented internally, which is why I wrote the 399.5 example that I
did.  Also, the picture on the page shows how time stops/slows way
down during the leap second in Dave Mills' view of how time should be
returned to the user during the leap second.  Text I didn't quote says
that the system time must be monotonically increasing, so it is
incremented by 1ns each call.

So internally to the kernel, the time does repeat the last second.
Externally, in a Mills' perfect world, time stands still during the
leap second.  The second doesn't repeat, per se, but it freezes for a
second.

In FreeBSD, the actual internal time is exposed to the user.  If we
froze time as returned from the system, then threaded applications'
threads would potentially stop d

Re: Comparing Time Scales

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
James Maynard <[EMAIL PROTECTED]> writes:
: M. Warner Losh wrote:
: > In message: <[EMAIL PROTECTED]>
: > James Maynard <[EMAIL PROTECTED]> writes:
: > : Thanks, guys, for your feedback. Here's another iteration.
: > :
: > : The numbering of NTP seconds in the vicinity of a leap second seems to
: > : differ from one document to another.  Here I follow the NTP (version 3)
: > : specification, RFC 1305, in which the Leap Indicator (sys.leap,
: > : peer.leap, pkt.leap) is 01 if a positive leap second is to occur at the
: > : end of the current UTC day, and 00 if no leap second is pending.
: > :
: > : UTC = 2005-12-31 23:59:59, NTP seconds = 3 345 062 399, LS_pending = 01
: > : UTC = 2005-12-31 23:59:60, NTP seconds = 3 345 062 400, LS_pending = 01
: > : UTC = 2006-01-01 00:00:00, NTP seconds = 3 345 062 400, LS_pending = 00
: >
: >
: >
: >>>The obvious question raised by this scenario is what happens during
: >>>the leap second itself when NTP time stops and the clock remains
: >>>unchanged. If the precision time kernel modifications have been
: >>>implemented, the kernel includes a state machine that implements the
: >>>actions required by the scenario. [[ At the first occurrence of second
: >>>3,124,137,600, the system clock is stepped backward one
: >>>second.]] However, the routine that actually reads the clock is
: >>>constrained never to step backwards, unless the step is significantly
: >>>larger than one second, which might occur due to explicit operator
: >>>direction. In this design time stands still during the leap second,
: >>>but is correct commencing with the next second.
: >
: >
: > So internally, once we hit the leap second time, we step backwards.
: > Some systems contrain the values returned from gettimeofday to not
: > notice the step, while others do not.  This talks about incrementing
: > the time by 1 unit (usually a nanosecond or microsecond) while time is
: > catching up.
: >
: > http://www.eecis.udel.edu/~mills/leap.html
: >
: > So it very much depends on if you are looking inside the kernel or
: > outside the kernel for what the correct values are.  If you freeze
: > time, then doing a time exchange during a leap second is going to give
: > bogus results.  If you step it back, then it will give good results,
: > but there are other bad effects.
: >
: > Basically, no matter what you do, it is broken (unless you tick in TAI
: > time).
: >
: > Warner
: >
: >
: >
: Thank you. You are trying hard to educate me on this matter, and I
: appreciate your efforts.
:
: Following the link you gave me in this message, <>, I found a discussion
: of how the leap second is to be handled in the NTP time scale. That
: discussion includes the following table to illustrate the leap second
: transition of 1998-12-31 23:59:60 UTC:
:
: +---+--++--+---+
: |   Date|   Time   | TAI Offset | NTP Leap |  NTP Seconds  |
: +---+--++--+---+
: | 31 Dec 98 | 23:59:59 | 31 |01| 3,124,137,599 |
: +---+--++--+---+
: |   | 23:59:60 | 31 |01| 3,124,137,600 |
: +---+--++--+---+
: |  1 Jan 99 | 00:00:00 | 32 |00| 3,124,137,600 |
: +---+--++--+---+
: |   | 00:00:01 | 32 |00| 3,124,137,601 |
: +---+--++--+---+
:
: Funny, that's exactly what I wrote in my last iteration of my table!
:
: In the NTP time scale, it is the last second of the UTC day, NTP second
: number 3 124 137 600, that is repeated. The first instance of this NTP
: second number is the leap second, during which (TAI - UTC) = 31 and the
: NTP Leap Indicator is 01.  The second instance of this NTP second number
: is the the first second of the new UTC day, during which (TAI - UTC) =
: 32 and the NTP leap indicator is 00.  You can distinguish the leap
: second (while it is occuring) by the fact that it is the last second of
: the UTC day (time of UTC day = 23:59:60) and that the NTP Leap Indicator
: is 01. You can distinguish the first second of the new UTC day by the
: fact that, of the two seconds with the same NTP second number, it is the
: one that has the NTP leap indicator set back to 00.
:
: Note that according to the write-up to which you referred me, it is
: NTP second number 3,124,137,600 (not 3,124,137,599) that is repeated.
: NTP second number 3,124,137,599 is the *next-to-last* second of the day,
: not the *last* second of the day.)

No.  The article specifically says that aft

Re: Comparing Time Scales

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
James Maynard <[EMAIL PROTECTED]> writes:
: Thanks, guys, for your feedback. Here's another iteration.
:
: The numbering of NTP seconds in the vicinity of a leap second seems to
: differ from one document to another.  Here I follow the NTP (version 3)
: specification, RFC 1305, in which the Leap Indicator (sys.leap,
: peer.leap, pkt.leap) is 01 if a positive leap second is to occur at the
: end of the current UTC day, and 00 if no leap second is pending.
:
: UTC = 2005-12-31 23:59:59, NTP seconds = 3 345 062 399, LS_pending = 01
: UTC = 2005-12-31 23:59:60, NTP seconds = 3 345 062 400, LS_pending = 01
: UTC = 2006-01-01 00:00:00, NTP seconds = 3 345 062 400, LS_pending = 00


>> The obvious question raised by this scenario is what happens during
>> the leap second itself when NTP time stops and the clock remains
>> unchanged. If the precision time kernel modifications have been
>> implemented, the kernel includes a state machine that implements the
>> actions required by the scenario. [[ At the first occurrence of second
>> 3,124,137,600, the system clock is stepped backward one
>> second.]] However, the routine that actually reads the clock is
>> constrained never to step backwards, unless the step is significantly
>> larger than one second, which might occur due to explicit operator
>> direction. In this design time stands still during the leap second,
>> but is correct commencing with the next second.

So internally, once we hit the leap second time, we step backwards.
Some systems contrain the values returned from gettimeofday to not
notice the step, while others do not.  This talks about incrementing
the time by 1 unit (usually a nanosecond or microsecond) while time is
catching up.

http://www.eecis.udel.edu/~mills/leap.html

So it very much depends on if you are looking inside the kernel or
outside the kernel for what the correct values are.  If you freeze
time, then doing a time exchange during a leap second is going to give
bogus results.  If you step it back, then it will give good results,
but there are other bad effects.

Basically, no matter what you do, it is broken (unless you tick in TAI
time).

Warner

Re: the tail wags the dog

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Steve Allen <[EMAIL PROTECTED]> writes:
: The legal time of the US is (in many more words) GMT.
: The officials who are charged by congress with the task of providing
: time provide UTC.

The legal time in the US is the mean solar time at a given meridian,
as determined by the secretary of commerce (the actual law is a little
more verbose than this, but this is an accurate boil down) plus some
weird options for 'border states' which timezone they are in.  This is
why NIST provides UTC and leap seconds happen on the UTC schedule
rather than some other schedule that would produce the same results.
It is also why there are leap seconds and not the old-style frequency
adjustments + tiny steps.  Both of these schemes fit the law, as it is
rather vague in the words it uses in a legal sense (the term mean
solar time isn't legally defined, but does have an accepted scientific
meaning).  Other schemes could also fit the law that aren't UTC today
since there's no what we would call 'DUT1 tolerance' written into the
law...

Warner

Re: wikipedia "Leap Seconds" collaboration

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Neal McBurnett <[EMAIL PROTECTED]> writes:
: > Rob Seaman wrote:
: > >>I hope we can all continue this discussion in a more positive manner.
:
: It is the nature of email lists to be good at stimulating discussion,
: and bad at generating clear resolutions.  Thus was the FAQ born.  But
: we have a more modern technology than FAQs, the wiki, which can more
: effectively funnel passionate energy from groups of people with
: diverging ideas into coherent descriptions of a variety of viewpoints,
: suitable for enlightening the world.  Imperfectly, to be sure, but
: better than a mail list
:
: I think the thing we need to do is build on what clarity we can find
: in the moment, and document it at wikipedia.  If the folks discussing
: the "Jesus" article can arrive at a relatively stable set of positions
: (and last time I looked, they had done remarkably well, considering),
: surely we can also.
:
: Note the relatively successful policy of presenting things from a
: "Neutral Pointof View" (NPOV):
:
:   http://en.wikipedia.org/wiki/Wikipedia:Neutral_point_of_view
:
: So would folks be willing to collaborate at
:
:  http://en.wikipedia.org/wiki/Leap_second
:
: and related pages?

I've contributed to the unix time over leap seconds pages, and would
be happy to help.  I feel I can write a good argument for both sides,
even though I have my preferences.

Warner

Re: Risks of change to UTC

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Mark Calabretta <[EMAIL PROTECTED]> writes:
: On Sat 2006/01/21 10:11:04 PDT, "M. Warner Losh" wrote
: in a message to: LEAPSECS@ROM.USNO.NAVY.MIL
:
: >You really should read the archives of this list.  We've been over
: >this in great detail.  TAI is specifically contraindicated as a time
:
: I don't think new contributors (or even old ones) should have to read
: the mountain of email, with its many irrelevant diversions, that has
: accumulated over the last 6 years - somewhat over 9MB by my count.
:
: Instead, I (re-)suggest that you (someone) write a position paper, or
: at least a FAQ, summarising your points.

I agree.  I was unreasonable in my demands.  You are quite right.

Warner

Re: the tail wags the dog

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
[EMAIL PROTECTED] (Michael Sokolov) writes:
: Steve Allen <[EMAIL PROTECTED]> wrote:
:
: > The CGPM recommendation on the timescale everyone should use says UTC.
: >
: > UTC(insert your national time service here) is available in real time.
: >
: > TAI is the mathematical (really the political or diplomatic) entity
: > upon which UTC is ostensibly based, but the practical and legal
: > reality is the other way around.
:
: Has it occurred to any of you that *THIS* is the very root of the problem,
: that *THIS* is what we need to change?

I believe so.

: Also, isn't TAI readily available in real time from GPS?  (How difficult
: can it be for the routine parsing the data stream from the GPS receiver
: to add 19?)  OK, OK, it'll be TAI(GPS) rather than "true TAI", but then
: your UTC is really UTC(NIST) or UTC(USNO) or UTC(PTB) or whatever rather
: than "true UTC".

You can get an excellent approximation of TAI from GPS, assuming that
you use a sane receiver...  Some firmware doesn't deal well with
reporting the raw, uncorrected GPS time, but many do it well.
Depending on your needs, you may have to put the GPS receiver into a
mode that only reports URC...  Well, you get the idea.  It can be done
to a degree of accuracy that most people will accept, but the
post-processed time scale calculations are a little better than what
happens in real time.

Warner

Re: NOT A cruel fraud!

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
James Maynard <[EMAIL PROTECTED]> writes:
: Mr. Losh and I have apologized to each other, off list.  I think we
: should now retire the "cruel fraud" subject line.

Yes.  I was too hard on Mr. Maynard and I appologize for my rough
treatment of him here.  It was really over the line, and I'm sorry for
my ill considered words.

Warner

Re: NOT A cruel fraud!

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Peter Bunclark <[EMAIL PROTECTED]> writes:
: On Sun, 22 Jan 2006, M. Warner Losh wrote:
: > The short answer is that you cannot get a time feed of TAI, so the
: >
: So isn't this one of the things we want to fix in the brave new world of
: joined-up timekeeping? Distribute (very close to) TAI, keep the kernel
: PLLs sweet, move leap second handling to user-space and thus make
: debugging very easy, then everone can get their timescale of choice as
: a f(TAI)?

Yes.  That's a fair summary.

Warner

Re: NOT A cruel fraud!

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Ed Davies <[EMAIL PROTECTED]> writes:
: Earlier, I wrote:
: > We all know that it (and any other world-wide timescale) is
: > "postal" at the level of the time it takes light to cross a
: > moderately small room but for microsecond precision and looser
: > this is not an issue.
:
: I ought to qualify that.  There are, of course, time scales which
: are synchronized across the world with much better than microsecond
: precision (e.g., GPS time) but it's my understanding that they are
: not anything like as uniform as TAI.  Is this right?  Can anybody
: quote accuracy information for GPS time?  At what level of precision
: do time transfers using GPS time need out-of-band ("postal")
: correction for uniformity?

GPS is synchronized to UTC(USNO) which agrees with UTC(NIST) to within
a few nanoseconds.  GPS receivers can recover UTC(USNO) to within
about a few nanoseconds (as measured by a direct two way time exchange
with NIST), although not all of them are that quality.  The rms of a
good timing receiver is on the order of nanoseconds.  NIST and USNO
coordinate with each other to make sure there's close agreement in the
two minor variations of UTC.  They also participate in the computation
of TAI.

TAI is computed after the fact based on the observed time of different
frequency standards.  Phase offsets between different time sources are
measured, the data is decimated and placed into a particular format
and sent to BIPM.  These measurements are sent in batches and used to
measure interesting effects.

Here are some reports on how the various labs measure time relative to
a known source:

http://www.bipm.fr/cc/CCTF/Allowed/16/cctf04-18.pdf
http://www.bipm.org/en/publications/scientif/tai.html

Warner

Re: NOT A cruel fraud!

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Ed Davies <[EMAIL PROTECTED]> writes:
: M. Warner Losh wrote:
: > : >  TAI is specifically contraindicated as a time
: > : > scale.
: > :
: > :  > TAI is not currently recommended by its creators as a viable time
: > :  > scale.
: > :  >
: > :
: > : These claims are intellectually fraudulent. The archives in fact support
: > : the opposite of what Mr. Losh contends.
: >
: > Actually, it isn't quite that cut and dried.
:
: OK, so why is TAI contraindicated then?
:
: I've been on this list since 2002-07 and not yet seen a good
: argument against it.

The short answer is that you cannot get a time feed of TAI, so the
high precision folks don't like people calling something that's
derived indirectly TAI.  TAI(UTC(foo)) can be derived, however, if
you know the current leap offset.  This isn't broadcast in many time
formats.

Having said that, our company uses TAI internally for all its time
code and translates on output.  Doing that doesn't free you from
knowing about leap seconds, however, since you have to know about them
to convert to/from UTC which is universally broadcast or available.
GPS makes things a little easier, since there's a fixed offset from
TAI, but even it present difficulties because you need the current
leap offset to display UTC times.  However, some GPS receivers insist
on doing silly things when you try to get the raw time without
leapseconds.  Our systems need to have unambiguous timestamps, as well
as a requirement to display UTC.

Warner

Re: NOT A cruel fraud!

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
James Maynard <[EMAIL PROTECTED]> writes:
: If "we've been over this in great detail," I would like a more specific
: reference to the postings that did so.  Also, "we've been over this in
: great detail" seems not to have settled the issue.

We have been over these points in great detail.  Both here, and over
on the time-nuts mailing list.  Many of the same posters post to both
lists.

: It's cruel to insult a newbie so, and cause him to spend hours and hours
: perusing the archives, to no avail.

You learned a great deal.  You can learn more by reading the time nuts
archive as well.  http://www.febo.com/pipermail/time-nuts/

: >  TAI is specifically contraindicated as a time
: > scale.
:
:  > TAI is not currently recommended by its creators as a viable time
:  > scale.
:  >
:
: These claims are intellectually fraudulent. The archives in fact support
: the opposite of what Mr. Losh contends.

Actually, it isn't quite that cut and dried.

Warner

Re: Risks of change to UTC

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
"Daniel R. Tobias" <[EMAIL PROTECTED]> writes:
: On 21 Jan 2006 at 10:11, M. Warner Losh wrote:
:
: > I maintain that for human activity, there's no need for leap seconds
: > at all.  In each person's lifetime, the accumulated error is on the
: > order of a few minutes.  Over generations, the problems with noon
: > drifting to 1pm can trivially be solved by moving the timezones that
: > civilian time uses.
:
: What about when that accumulated difference is over 24 hours, so the
: offset between solar-based time and atomic time is actually on the
: order of days?

>From http://www.ucolick.org/~sla/leapsecs/dutc.html we know that the
rate of change of the day is somewhere between 25.6 s/century^2 and 42
s/century^2.  At those rates, it will be the year 6360-7633 when
enough time has accumulated for there to be a day difference.  Before
then, however, we go through a number of events.  Somewhere between
2058 and 2211, enter the realm where there's more than 2 leap seconds
per year.  This will be the first great UTC breakage because many
devices today (ntp included) KNOW that leap seconds happen twice
yearly.  The next break will happen sometime between 2300 and 2600
when we'll need more than 4 leap seconds a year.  The current ITU-R
TG.460 standard for leap seconds defines only primary and secondary
leap second times.  It does not define tertiary, so no one knows when
the leap seconds will happen if you need more than 4 per year, but the
ITU recommendation is that they happen at the end of a month.
Somewhere between 3250 and 4200 there will be more than 1 leap second
a month needed.  At this point the ITU scheme of having the leap
second at the end of the month will need to be modified.

That's at least 2000 years before 24 hours of delta have accumulated.

For some perspective, we've been using UTC for only ~50 years and the
gregorian calendar for only ~1500 years.  I'd anticipate that
something would need to be done about the slowing of the day well
before 4300 years have passed.

Somewhere around betwee 45,000-80,000 you'll need more than one leap
second a day.

: Will people be able to deal with a civil time
: standard that is based on an offset from a "UTC" that says it's
: Monday when all actual points on Earth have the local date at
: Saturday or Sunday?

Since that's 4k or more years into the future, who alive today will
know enough about what the future is like to impose a scheme that is
guranteed to work?

Clearly some scheme better than leap seconds will need to be invented
well in advance of these events.  A new scheme will be needed well in
advance of the Tuesday is really Wednesday problem.

: Many Web sites (including Wikipedia) use UTC as
: the standard for date/timestamps; will this be a reasonable thing
: when this causes the date of postings to be far off from what is
: being used locally?  And when, at some future point, the Gregorian
: calendar itself needs adjustment to handle the fact that it doesn't
: get the length of the year precisely correctly (and the length of the
: year in terms of solar days is changing due to the lengthening of the
: day, anyway), will this adjustment be done to the UTC standard (why,
: when it doesn't follow astronomy anyway?), or as an additional offset
: to local times (which could result in different countries having
: different dates as in the Julian/Gregorian transition period)?

The length of the gregorian calendar is off by 23s per year.  In year
5500 or so we'll have accumulated a day of error in it and we'll need to
skip a leap year to correct for that problem.  This is a good 2000
years before we'll have accumulated a day of DUT.

As you can see from the above, leap seconds won't save you.  They will
run out of steap in about 1500 to 2500 years.  At that point the
accumulated difference will be only about 2 hours.  If leap seconds
are totally abolished, time zone transitions could easily continue for
about 4000 years.  Either way, you have a problem.  The length of the
SI second is fixed, and the length of the day is getting shorter.

1500 years ago, no one spoke English.  Chances are the people that
deal with this problem in 1000 or 2000 years won't speak any language
recognizable to anybody alive today.

Warner

Re: Risks of change to UTC

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
James Maynard <[EMAIL PROTECTED]> writes:
: M. Warner Losh wrote:
: > UTC works for navigation, but leap seconds pose problems for other
: > users of time.  Stating absolutely that UTC is not broken ignores
: > these other users.
:
: Those "other uses," for whom leap seconds pose a problem, should be
: using a time scale that does not have leap seconds. They would be better
: served, for example, by TAI.

You really should read the archives of this list.  We've been over
this in great detail.  TAI is specifically contraindicated as a time
scale.  This also ignores the issues surrounding the proper
computation of elapsed time using the UTC time scale when represnted
by a number.  It ignores the fact that you must have a table of leap
seconds to do this effectively.  It ignores that one must have a table
of leap seconds to get TAI time, because most providers of time
provide it in UTC.

: For civil use, a calendar that counts days reaonably accurately is
: appropriate.  The Gregorian (New Style calendar) that the vast
: majority of the planetary population uses does this. UTC copes with
: the variable length of the mean solar day by inserting leap seconds
: as needed.  The role of the IERS in decreeing when leap seconds are
: needed is similar to that of the Roman College of Pontifices who managed
: the old Roman Republican calendar (before Julius Caesar's reform) by
: decreeing, as needed, when to shorten the month of February and insert
: the intercalary month of Mercurius.

I maintain that for human activity, there's no need for leap seconds
at all.  In each person's lifetime, the accumulated error is on the
order of a few minutes.  Over generations, the problems with noon
drifting to 1pm can trivially be solved by moving the timezones that
civilian time uses.

Keeping universal time synchronized to an arbitrary meridian is
already arbitrary.  Adding leap seconds is an arbitrary decision to
implement that arbitrary requirement.  It is really only important for
those uses of time that care what the local solar time is to within a
second (as opposed to within an hour which everyone in civil society
is used to since the late 19th century and time zones).

: If your primary need is for a time scale ithat counts SI seconds, with
: no leap seconds to confuse the matter, then don't use UTC. Use a time
: scale that counts SI seconds, such as TAI or GPS time. There's no point
: to applying the mised radix Gregorian calendar system to such a time
: scale, although you can do so if you wish.  Count days of 86 400 SI
: seconds each, or GPS weeks of 604 800 SI seconds, or just count SI seconds.
:
: If, on the other hand, you need to count solar days, or mean solar days,
: use a calendar and time scale that does so.  In order to know which way
: the earth is pointing, use UT1, or a compromise scale such as UTC that
:   is kept reasonably close to UT1.  For the vast majority of the
: population of the planet, including celestial navigators, UTC is good
: enough. If you want to know the direction the earth is pointing with
: more precision, apply DUT1 corrections, or use other IERS products such
: as Bulletin A.

That is one solution to the problem, but it is not the only solution
to the problem.  Quoting the old, tired saws about using TAI or GPS
time scales to count SI seconds doesn't promote good dialog.  Instead,
it drags this discussion down to the level of Dogma.

There are many users that need to both count SI seconds, as well as
synchronize their operations to civilian time.  Leap seconds cause
these users real difficulties.  Implementing leap seconds in software
is hard to get pedantically right (I have a pile of bugs in code that
was written by smart people that get leap seconds wrong, and over 200
hours of time spent in the past few years on implement, testing and
debugging leap seconds).  Even many ntp servers on the net got the
leap second wrong, as did many of the time stations.  There's much
evidence that this "solution" to the problem has shifted the cost of
keeping astronomers, celestial navigators, and others with a real need
to the solar time happy to a large number of other users who have no
such real need.  As such, it seems only fair in today's economy to
place the costs of the needs of a community on those communities that
have a real need for those goods or services.

: There is no need to "fix" the time scale, UTC, that is used by the vast
: majority of the planet's population to accommodate the very small
: minority of precision time users who desire a time scale that has no
: leap seconds. Let that minority use a time scale, such as TAI, that does
: not have those messy leap seconds.

TAI is not currently recommended by its creators as a viable time
scale.

In my day job, we do use TAI internally.  That doesn't solve the
problems of leap second

Re: Risks of change to UTC

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
"Brian Garrett" <[EMAIL PROTECTED]> writes:
: > But the protocol for broadcasting DUT1 in, e.g., WWV, does not provide
: > for DUT1 values of more than plus or minus 0.9 s.  The value of DUT1
: > could be announced by voice message -- but that would not be
: > language-independent.  If I travel to asia in my boat, I will not be
: > able to benefit from DUT1 announcemnts in Japanese from JJY or in
: > Chinese from BPM (or whatever their standard time and frequency station
: > is).  An longwave broadcasts such as those from WWVB do not have voice
: > modulation at all!
: >
: WWVB broadcasts the DUT1 correction in binary-coded decimal form, and is
: capable of representing DUT1 values up to +/- 1.5 seconds with their current
: format.  I would think other longwave stations do the same, but not knowing
: their particular BCD format I couldn't say what DUT1 values they are capable
: of representing.

Taking a long term view, some changes in the broadcast format or
interpratation would be necessary.  It is quite possible to multiplex
these bits so that successive opportunities to transmit the DUT1
encode additional digits as well.  This would mean it would take a
while to transmit DUT1, but such operational issues have been deemed
acceptable in systems such as GPS which transmit leap second
information only a few times an hour.

There appears to be about 13 unused bits in the WWV stream
http://tf.nist.gov/timefreq/stations/wwvtimecode.htm at second 8, 14,
24, 27, 28, 34, 42, 43, 44, 45, 46, 47, and 48.  Granted, some of
these are spacer bits between BCD digits, but they could be reclaimed.
The WWVB timecode
http://tf.nist.gov/timefreq/stations/wwvbtimecode.htm has fewer free
bits at 9: 5, 11, 14, 21, 24, 34, 35, 44, and 54.  10 if you count the
redundant sign bit.  With enough advance notice, these bits could be
used to encode larger DUT1 values.

If you travel to asia, you will no doubt know what DUT1 is to within a
second before you leave the US.  The rate of DUT1 change is on the
order of 1s per year.  This should obviate the need to be told the
current DUT1 value every minute.  Even the DUT1 values broadcast to
the .1s level only change monthly or so (although there are provisions
to change them with only 2 weeks notice).

I have no delusions that changing the meaning of the bits in the time
codes braodcast from WWV and WWVB will happen in less than a decade...

Warner

Re: Risks of change to UTC

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
James Maynard <[EMAIL PROTECTED]> writes:
: M. Warner Losh wrote:
:
: > : > If DUT1 is broadcast, then one can set the time keeping device to UT1
: > : > by a means similar to setting it to UTC, even if DUT1 exceeds 0.9s
: > : > with a similar accuacy (or better).  There's nothing that says a watch
: > : > has to display UTC to be set correctly.

WWV and most of the world's time stations broadcast DUT1.  I should
have added in my last message that some change in the signal format
would be necessary if the range of DUT1 exceeds 0.9s.  Given a
sufficently long time horizon, this becomes a non-issue.

: You seem to expect celestial navigators to become more sophisticated in
: these matters of time scales than is presently the case. It would be far
: better, in my option, not to tinker with the definition of UTC and of
: civil time at all, but leave civil time zones tied approximately to UT1.

Maybe that is a reasonable expectation.  Maybe it is not.

I will note that the profile of high precision time users has changed
since 1972 when UTC was invented.  In 1972 celestial navigators were a
big part of the users of time.  These days, they represent a much
smaller portion of that audience.  If there are changes to how time is
provided, then all users of precision time should be considered.
Should we continue to tie our time up in knots because of a tiny
minority of users?

Maybe the answer that question is yes.  Maybe it is no.  However,
stating absoultely, unquestionably that it is one or the other is what
is wrong and what takes this discussion off into the weeds.

How many celestial navigators are there today?

: "There's nothing that says a watch has to display UTC to be set
: correctly."  But then the user would have to carry two watches, one set
: to an approximation of UT1 (e.g., UTC + DUT1) and another set to civil
: time (TI, or local zone time, or whatever). For simpler, it would be,
: with far less impact on ordinary users of time, not to let UTC (or its
: replacement) differ from UT1 by more than the present +/- 0.9 s.

Over the next 50 years, these two watches will be well within the
tolerance of most normal watches.  The approximation of civil time
will be less than one minute off during that time (if most of the
projections are to be believed).

If this is a real issue, the market will take over and produce watches
that have 'navigation time' and 'civilian time' at the touch of a
button.  If you still desire to use older, purely mechanical watches,
then you must take extra steps, like knowing the current offset
between civil time and navigation time.  Of course, purely mechanical
watches have other issues when used on a boat.  Electro-mechanical
ones don't suffer those problems, but require some power source.

The foregoing assumes, for the sake of argument, that DUT1 becomes
unbounded.

: UTC is not broken. There is no need to "fix" it.

UTC works for navigation, but leap seconds pose problems for other
users of time.  Stating absolutely that UTC is not broken ignores
these other users.

: > I'd also add that GPS receivers today already do exactly this sort of
: > correction when they decode the GPS time, but display the UTC time.
:
: True. But GPS receivers are electrically powered, and are not a fallback
: for the case when a boat's electrical systems fail.  Nor are they a
: backup system for the case when the the boater is in proximity to a
: war zone where the U.S. DoD is jamming or degrading the civil SPS GPS
: service.

GLASNOS is a backup system to GPS that is not subject to DoD's
selective denial of signal.  Soon, Galileo will provide additional
backup.  Also, there's about a dozen time stations worldwide that
provide time signals today which can act as backup to GPS during a GPS
outage.  I pointed out GPS more as an example of a 'clock that gets it
times in one timescale, but displays it in another' rather than as a
backup time source for your example mariner.

Warner

Re: Risks of change to UTC

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
James Maynard <[EMAIL PROTECTED]> writes:
: M. Warner Losh wrote:
: > In message: <[EMAIL PROTECTED]>
: > James Maynard <[EMAIL PROTECTED]> writes:
: > : ones position using sight reduction tables.  Today a mechanical watch or
: > : chronometer, or a battery-powered wristwatch, can be set to UTC using
: > : radio time signals. Then when power fails, the sailor still has a
: > : reasonably accurate spprodximation to UT1 available.
: >
: > If DUT1 is broadcast, then one can set the time keeping device to UT1
: > by a means similar to setting it to UTC, even if DUT1 exceeds 0.9s
: > with a similar accuacy (or better).  There's nothing that says a watch
: > has to display UTC to be set correctly.
: >
: > Warner
: >
: > .
: >
: And how is DUT1 to be broadcast in a language-independent manner?  That
: protocol needs to be established well in advance.

I should add that I read your 'can be set to utc using radio time
signals' to mean something like WWV or other radio time service.
Those sevices do already broadcast DUT1.  Any watch that is smart
enough to decode those signals would be smart enough to add this minor
correction as well.

The mechanical watch might be a bit of a problem, but DUT1 doesn't
change enough to introduce navigation errors similar to what we have
today over the course of a year and can easily be looked up like
someone would lookup what the weather was going to be like.

I'd also add that GPS receivers today already do exactly this sort of
correction when they decode the GPS time, but display the UTC time.

Warner

Re: Risks of change to UTC

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
James Maynard <[EMAIL PROTECTED]> writes:
: M. Warner Losh wrote:
: > In message: <[EMAIL PROTECTED]>
: > James Maynard <[EMAIL PROTECTED]> writes:
: > : ones position using sight reduction tables.  Today a mechanical watch or
: > : chronometer, or a battery-powered wristwatch, can be set to UTC using
: > : radio time signals. Then when power fails, the sailor still has a
: > : reasonably accurate spprodximation to UT1 available.
: >
: > If DUT1 is broadcast, then one can set the time keeping device to UT1
: > by a means similar to setting it to UTC, even if DUT1 exceeds 0.9s
: > with a similar accuacy (or better).  There's nothing that says a watch
: > has to display UTC to be set correctly.
: >
: > Warner
: >
: > .
: >
: And how is DUT1 to be broadcast in a language-independent manner?  That
: protocol needs to be established well in advance.

It already is being broadcast in, eg, WWV.

Warner

Re: Risks of change to UTC

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
James Maynard <[EMAIL PROTECTED]> writes:
: ones position using sight reduction tables.  Today a mechanical watch or
: chronometer, or a battery-powered wristwatch, can be set to UTC using
: radio time signals. Then when power fails, the sailor still has a
: reasonably accurate spprodximation to UT1 available.

If DUT1 is broadcast, then one can set the time keeping device to UT1
by a means similar to setting it to UTC, even if DUT1 exceeds 0.9s
with a similar accuacy (or better).  There's nothing that says a watch
has to display UTC to be set correctly.

Warner

Re: Risks of change to UTC

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Neal McBurnett <[EMAIL PROTECTED]> writes:
: These astronomical and navigation systems currently get inputs of time
: from all sorts of other systems and people: other instruments, other
: computers, sub-contractors, etc.  Changing the meaning of UTC would
: lead to a cascade of systems that changed their own timescales and
: I/O formats, meaning that auditing the flow of time data between them
: would get more and more complex for a long time.

The changing formats should indeed be phased in.  First, changes to
the data streams so that they can represent DUT1 > 0.9s.  Next, fix
everything that breaks...  Repeat.  You're likely right, this will
take a long long time :-(.

Warner

Re: Internet-Draft on UTC-SLS

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Tim Shepard <[EMAIL PROTECTED]> writes:
: > The serious timekeeping people gave up on rubberseconds in 1972 and
: > I will object with all that I can muster against reinventing them
: > to paste over a problem that has a multitude of correct solutions.
:
: As I learned from a recent posting to this mailing list, it seems that
: even TAI has rubber seconds (adjustments to the rate is made from time
: to time to compensate for errors that have been accumulating, making
: TAI a better (more useful) approximation time).
:
: Do you object to those adjustments (rubber seconds) to TAI as well?

TAI changed the definition of the second in 1977 by one part in 10^12.
Those weren't rubber seconds.  It again slowly changed the definition
over 3 years of the second by 2 parts in 10^14 ending in 1998.  They
were made not to correct the phase of TAI, but rather to correct the
length of the second to more closely match the SI second definition.
No phase adjustments were made to TAI.

Rubber seconds, on the other hand, are a steering device designed to
keep time in sync with another clock, in this case the earth.  They
aren't changing the definition of the second, but rather are inserting
phase adjustments to the timescale.  The goofy scheme that existed
prior to 1972 fell into this latter category.

Warner

Re: Internet-Draft on UTC-SLS

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Markus Kuhn <[EMAIL PROTECTED]> writes:
: "M. Warner Losh" wrote on 2006-01-19 19:20 UTC:
: > Effectively, you'd have to have two time scales in the kernel.  UTC
: > and UTC-SLS.  You make it sound simple, but the hair in doing this may
: > be quite difficult.  There's more book for the kernel to keep, and it
: > would have to expose the bookkeeping to userland for ntp to work.
: > What makes this hard is that ntpd may introduce steers into the normal
: > system time at normal times which it doesn't want to confuse with the
: > steers in frequency that are used during a UTC-SLS operation.
:
: You correctly point out some of the design considerations that have to
: go into such code. You describe roughly one of the (several) different
: ways of implementing all this that I have in mind. In comparison to how
: complicated the Mills kernel PLL is already today, that does not
: actually sound like an overwhelming additional complexity. Actually, it
: sounds quite doable when you think through it a bit. Not trivial, but
: carefully doable without performance penalty.

Anything that makes the Mills' kernel PLL more complicated is unlikely
to be implemented correctly.  There have been several subtle bugs in
Mills' implementation that have taken a long time to sort out.  In
fact, members of this list have found some of them and have a lot of
experience with Mills' kernel PLL.

Many people in the timing community are unhappy with Mills'
implementation.  They believe it to be way too complex and have been
trying to implement a much simpler implementation that gives more
control to the measurement process to apportion the errors that it
measures by doing simpler steers.  It moves the kernel time control in
the wrong direction.

: The important thing is: All this has to be done only *once* for each
: operating system and *once* for each each NTP client/server used on it,
: by someone who understands the issue, and then the problem is hopefully
: solved. We've handled far more complicated things in OS kernels, right?

No.  It has to be done continuously.  By that I mean that you can
clearly write an implementation once, but over time you will find
flaws with that implementation (I can 100% guarnatee that).  As you
find the flaws, you'll have to investigate them and fix them.  In
addition, other parts of the kernel change, and some of those changes,
though innocent looking, can have a big impact on how you implemented
the dual UTC UTC-SLS offset.

By way of illustration, in FreeBSD 2.2, the timing implementation was
changed from keeping the current time, to keeping an uptime plus a
boot time.  Adjustments to the boot time were used to accomplish phase
shifts in the system time.  Unbeknownst to the person implementing
this, despite him having integrated the Mills kernel NTP portion into
a prior version of FreeBSD, this broke the exact semantics of system
time around the leap second.  It took years for someone (me) to notice
and fix this problem.

: I also fully appreciate that the existing *interface* between
: kernel-level clock drivers and user-level NTP clients and servers is by
: far not as well specified as it could be. With all due respect to its
: author, I do not hope that RFC 1589 will not remain the last word on
: clock-driver interfaces. A detailed proposal for a new standard on that
: area should help to smooth the process of getting the above right. I
: think a serious clock-controlling API this is what some people have been
: asking for for a long time from POSIX.

In order to implement your UTC-SLS proposal, it must change.

: I believe it is good engineering practice to specify the functionality
: before the interface and implementation, therefore I would like to
: discuss aspects of desired functionality (including UTC-SLS) before
: starting to work on a proposed successor to adjtimex(), ntp_adjtime(),
: etc.

While I agree in principle, I can tell you as one who has bloodied his
knuckles on implementing timekeeping that what you are saying sounds
hard to get pedantically right.

: I am not claiming that the spec-writing job is finished, or even half
: finished yet. I see a replacement for RFC 1589 etc. as the next job on
: the agenda before UTC-SLS (and lots of other time API ideas floating
: around here and elsewhere) can start to fly.

Unfortunately, it makes it hard to evaluate your proposal in the
absense of a concrete spec.  What is there now sounds nice, but it
still strikes me as difficult to implement.

Warner

Re: Internet-Draft on UTC-SLS

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Markus Kuhn <[EMAIL PROTECTED]> writes:
: "M. Warner Losh" wrote on 2006-01-19 19:20 UTC:
: > : In other words, *no* incompatible changes are made to the NTP protocol.
: > : In a correct UTC-SLS implementation, you should *not* be able to
: > : distinguish remotely, whether some NTP server synchronizes its kernel
: > : clock to UTC or UTC-SLS, because this must not influence its NTP
: > : interface in any way.
: >
: > In the systems that I observed over the leap second, I can say that
: > those systems that did implement a UTC-SLS-like approach to steering
: > out the extra second were definitely observable while they did this.
:
: Yes, of course! If you connect a GPS receiver that delivers an
: undocumented UTC-SLS-alike timescale on its output to an NTP server that
: in turn knows nothing of this UTC-SLS-alike timescale and is not
: implemented accordingly, you must get a mess. What did you expect? What
: you observed is *not* an implementation of UTC-SLS, but just a
: connection of incompatible components!

No.  You misunderstand me.  There were no GPS receivers that did
UTC-SLS-alike timestamps.  This was what CLIENTS of the GPS receiver
did.  The GPS receiver did exactly the right thing, ntp did the right
thing, our products did the right thing.  What I observed was in the
CLIENTS how they dealt with the leap seconds.

GPS-RECEIVER --- GPSNTPD --- NTP CLIENT1[*] windows NT box
 +-- NTP CLIENT2[*] Linux box
 +-- NTP CLIENT3[*] FreeBSD box
 +-- NTP CLIENT4[*] FreeBSD box

The items marked with a [*] are the ones I observed.  client1 and
client2 were the ones that I observed implementing UTC-SLS like time
and such time was visible to the outside world.  I was clearly able to
see both on the GPSNTPD clients 3 and 4 the skews of clients 1 and 2
during the leap second, where I observed no such skews relative to
GSPNTPD.

I'm saying that certain OSes have implemented the UTC-SLS-like
systems.  I'm saying that ntp daemons running on those systems use the
actual system time for time exchanges, rather than real UTC.  I'm
saying that this can cause confusion to a downstream ntp client and
that's my basis for thinking this idea is not good.  I'm further
saying that for the OS to support ntpd, it would need to also support
UTC and provide an interface to get the raw UTC time.  It would need
to keep track of steers for the purpose of implementing UTC-SLS and
steers needed to compensate for the changing base clock frequency
separate so that it could give the right time when asked for UTC
time.  It would also need to give some kind of feedback to ntpd that
it didn't do the small steer it wanted because UTC-SLS is going on at
that time (since otherwise ntp would think that there's some frequency
step going on if it just ignored it, and doing both steers
concurrently would be very hairy).

It is a seductive idea that leaves one with a big hangover in the
morning.

: > : However, MSF has no leap warning at all, nor do some time codes used in
: > : the power or military industry. And recent extensions to the latter
: > : added only a leap second warning that arrives a few seconds before the
: > : leap. I consider the leap-second handling of these latter time codes
: > : pretty useless.
: >
: > IRIG-B (all IRIG?) has no leap second announcement that I could find.
: > The Irig standard I could find (IRIG STANDARD 200-04, issued September
: > 2004, at http://www.jcte.jcs.mil/RCC/PUBS/pubs.htm) makes no mention
: > of the format of the leap seconds at all (the leap second convention
: > appendix says that leap seconds happen).  Talking to engineers here
: > who have implemented IRIG at various places reveals that there are
: > multiple conventions for dealing with leap seconds in irig (the new
: > standard being to do 59, 60, the old standard was to do 59 59).
: > There's no provision for leap second announcement, although control
: > bits do remain unassigned...
:
: The IRIG world is clearly a huge mess, with more than two dozen
: formats in use. A useful beginner's overview is at
:
:   http://www.symmttm.com/pdf/Gps/an_Time_and_Time_Code_Ref.pdf

Excuse me, but isn't it a bit patronizing to point someone at a
beginner's guide?  I don't need beginner's overviews.  I have plenty.
I've been dealing with Irig for several years now, as well as many
other time codes and timing issues (including leap seconds).  What I
need is references to standards and copies of standards.  I want to
understand all the variations of IRIG so I know what the
sales/marketing folk are talking about when they ask me if thus and
such standard is easy or hard.

: which outlines on page 23 the 27-bit IEEE 1355 power-industry extension
: to IRG-B that has a Leap Second Pendin

Re: Fixing POSIX time

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Markus Kuhn <[EMAIL PROTECTED]> writes:
: Poul-Henning Kamp wrote on 2006-01-19 17:56 UTC:
: > >For TAI I'd suggest 1958-01-01, when TAI and UT were set nearly together.
: >
: > I chose the time when TAI became constant rate so that
: > all the rubber seconds are confined to negative values.
:
: Please remember that the TAI second differed noticeably from the SI
: second until about 1998, because black-body radiation shift was not
: taken into account in the definition of TAI before then. Also caesium
: fountains have improved quite a lot shortly before 2000.

According to the web site that I posted
(http://www.ucolick.org/~sla/leapsecs/timescales.html), this feature
was in both TAI and UTC.  From the url:

UTC on 1977-01-01
 Because the rate of TAI was reduced by one part in 1012, the rate
 of UTC was reduced by the same amount. Therefore, before this date UTC
 seconds were shorter than they currently are.
UTC from 1995 through 1998
 In 1995 a CCTF working group determined that the length of TAI
 seconds was longer than the SI second because the clocks contributing
 to TAI were not corrected for the effects of blackbody
 radiation. Over the next three years the frequency of TAI was steered
 to reduce the length of its seconds by about 2 parts in
 1014. Therefore the length of UTC seconds was also reduced. This
 change is evident as the final kink in the plot of TT(BIPM04).

[EMAIL PROTECTED] is a frequent contributer here, and this looks like
his web site.

I'd be interested in a source that describes these events differently...

: Therefore, if people ask me for my favourite epoch for a new time scale,
: then it is
:
:   2000-03-01 00:00:00 (preferably UTC, but I would not mind much
:if it were TAI, or even "GPS time")
:
: This epoch has the following advantages:
:
:   a) It is well after TAI rubber seconds were fixed in ~1998,
:  so we know the time of *that* epoch with much greater accuracy than
:  any before 1998.

TAI and UTC have ticked at the same rate since 1972.  While this rate
has changed twice (by very small amounts, first by 1 part in 10^12 and
then later by 2 parts in 10^14), they have been the same.  Prior to
1972 we had both steps in time (on the order of 50ms to 100ms) as well
as TAI and UTC having different notions of the second.

Warner

Re: the GPS impending leap second bit

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Steve Allen <[EMAIL PROTECTED]> writes:
: On Thu 2006-01-19T18:30:02 +, Markus Kuhn hath writ:
: > GPS sends out announcements within days after IERS does, which is
: > excellent service.

IERS announced the leap second on July 4th, about 6 weeks before it
was encoded into GPS stream.  This didn't happen until mid to late
August.

The Canadian time service was announcing the leap second months before
July (I have a friend who has been writing software to decode the
world's time broadcast stations, and he was amused by this).

: Yes, and no, for some notes from the NTP newsgroup imply that the
: setting of the GPS impending leap bit caused some GPS units to insert
: leap seconds every day at 0h UT.

We noted that the GPS system started to advertise leap seconds in the
middle of August.  This causes the HP SmartClocks to hickup at the
alternate leap second time...

: Furthermore, in
: http://www.uscg.mil/hq/g-m/moa/docs/Saab505.pdf
: we see that the Saab AIS units introduced the leap second immediately
: and needed a firmware update to fix the condition.

Cute.  Older Motorola Oncore receivers had a firmware bug that caused
a three second blimp in their time code when it had been 1024 weeks
since the last leap second as well.

Warner

Re: Internet-Draft on UTC-SLS

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Markus Kuhn <[EMAIL PROTECTED]> writes:
: "M. Warner Losh" wrote on 2006-01-19 16:58 UTC:
: > > http://www.ietf.org/internet-drafts/draft-kuhn-leapsecond-00.txt
: > The biggest objection that I have to it is that NTP servers will be at
: > least .5s off, which is far outside the normal range that NTP likes to
: > operate.  Unless the prceice interval is defined, you'll wind up with
: > the stratum 1 servers putting out different times, which ntpd doesn't
: > react well to.
:
: Please read the proposal carefully (Section 2):
:
:UTC-SLS is not intended to be used as a drop-in replacement in
:specifications that are themselves concerned with the accurate
:synchronization of clocks and that have already an exactly specified
:behavior near UTC leap seconds (e.g., NTP [RFC1305], PTP [IEC1588]).

I saw that, but it does present some interesting implementation issues
for an ntp server.

: What this means is:
:
:   - NTP still uses UTC on the wire, exactly in the same way in which it
: does so far, *independent* of whether any of the NTP servers or clients
: involved supply UTC-SLS to their applications.

This is the hard part.  Right now, NTP steers the system time to UTC
as best it can.  It steps the time back by one second at the leap
second since there's no way in POSIX's time_t to represent leap
seconds, and all the time interfaces to the kernel are in terms of
time_t.

During the recent leap second, I observed three different clients of
one of our GPS receivers do three different things with the leap
second:  One inserted it early with the kind of hard frequency steer
that UTC-SLS does.  The second did it centered around the leap second,
and one didn't do anything until after the leap second and a couple of
polling intervals, then steered it out.  It is based on these systems
that implement the UTC-SLS-like steers that I based my reaction on.

:   - NTP implementations (by this I mean the combined user-space and
: kernel-space segments) should convert NTP timestamps that have been
: received over the wire through the UTC -> UTC-SLS mapping, and steer
: after that what gettimeofday() provides to users .
:
:   - NTP implementations should equally also convert any timestamp received
: from gettimeofday through the UTC-SLS -> UTC mapping before it goes
: out the wire.

This is the difficult part.  Right now, ntp daemons use the system
time as the time they do the time exchange with.  This is even if they
have a fancy gps receiver connected to them that isn't yet fully
synchronized.  Since they use system time, as you observe there would
need to be some mapping.  This mapping would necessarily have to be in
the kernel (since only it has the knowledge of the actual UTC UTC-SLS
offset) and would need to be an additional system call.  Such system
calls are not yet standardized.  This adds additional overhead to the
time exchange, which increases the opportunity for latency to creap
in.

Effectively, you'd have to have two time scales in the kernel.  UTC
and UTC-SLS.  You make it sound simple, but the hair in doing this may
be quite difficult.  There's more book for the kernel to keep, and it
would have to expose the bookkeeping to userland for ntp to work.
What makes this hard is that ntpd may introduce steers into the normal
system time at normal times which it doesn't want to confuse with the
steers in frequency that are used during a UTC-SLS operation.

: In other words, *no* incompatible changes are made to the NTP protocol.
: In a correct UTC-SLS implementation, you should *not* be able to
: distinguish remotely, whether some NTP server synchronizes its kernel
: clock to UTC or UTC-SLS, because this must not influence its NTP
: interface in any way.

In the systems that I observed over the leap second, I can say that
those systems that did implement a UTC-SLS-like approach to steering
out the extra second were definitely observable while they did this.
Given the variance in approach to this steering, and given it was
observable to the outside world, I concluded that N independent ntp
servers doing this would only confuse ntp clients...

: I hope this answers your concerns.

Well, it raises a whole different set of issues than I had originally
feared.  ntpd's use of system time to do its time exchanges has caused
me a significant amount of trouble in the past.

: > I'm also concerned about the fact that many radio time codes do not
: > announce the leap second pending until the last hour or less.  This
: > makes it hard to propigate out to the non-stratum 1 servers.
:
: I fully agree that leap seconds should be announced as early as
: possible, and I think that anything less than a month is undesireable.
: GPS sends out announcements within days after IERS does, which is
: excellent service. NIST sends out ann

Re: Fixing POSIX time

[M. Warner Losh]

I like this idea as well...

Poul, maybe we should implement this on FreeBSD.

I'd suggest "working_time_t" or "correct_time_t" as the name of the
type to replace "time_t" which would be deprecated. :-)

Warner

Re: Internet-Draft on UTC-SLS

[M. Warner Losh]

The biggest objection that I have to it is that NTP servers will be at
least .5s off, which is far outside the normal range that NTP likes to
operate.  Unless the prceice interval is defined, you'll wind up with
the stratum 1 servers putting out different times, which ntpd doesn't
react well to.

ntpd runs at a 1024s interval, typically.  Depending on when the leap
second is announced, and when it polls, ntpd would see a 1s step
(since the rubber seconds happen over 1000s).  It wouldn't believe it
(since it has popcorn outlier rejection).  It would then believe the
step after 2048s, but since the step is so large, it would just exit
(depending on the flags given to ntpd).  This will lead to *MORE*
clock skew, not less.

I'm also concerned about the fact that many radio time codes do not
announce the leap second pending until the last hour or less.  This
makes it hard to propigate out to the non-stratum 1 servers.

It is a horrible idea.  It is a horrible kludge and I'm not mollified
by the weak justifications that are given in the "faqcrit."  My
concerns are real and valid and this will cause more problems than it
kludges over.  It is a terrible idea.

Warner

Re: The real problem with leap seconds

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Mark Calabretta <[EMAIL PROTECTED]> writes:
: On Wed 2006/01/18 08:17:54 -, Francois Meyer wrote
: in a message to: LEAPSECS@ROM.USNO.NAVY.MIL
:
: >Maybe it should be,  but  this  is  far  from  being
: >obvious from its current definition.
:
: I agree that the current definitions leave a lot to be desired in terms
: of clarity and rigour - an uncharitable person might even describe the
: extract of ITU-R TF.460-6 cited the other day by Michael Deckers as
: inconsistent.  However, you have to consider how UTC is actually used in
: practice and this is what my comments are based on.
:
: >1. UTC and TAI  share  the  same  rate,  the  same
: >   origin, the same second. And therefore :
: >
: >UTC - TAI = 0
:
: They both count SI seconds but the question of the origin is a bit
: muddy.
:
: You could argue that there is a fixed 10s offset between UTC and TAI
: because UTC post-1972 (everything I've said about UTC only applies
: post-1972) started with 10 leap seconds, and before 1972 UTC wasn't
: simply a representation of TAI.  There's no simple way of fudging
: radixes that I can think of to make them match up, but if this worries
: you then simply think in terms of proleptic UTC (post-1972),
: see http://en.wikipedia.org/wiki/Proleptic.

UTC (and its predecessors) prior to 1972 did not tick off in SI
seconds.  It used a fixed radix like TAI currently does.  The amount
of time in UTC seconds varied a little.  Here's the table from
ftp://maia.usno.navy.mil/ser7/tai-utc.dat

1961 JAN  1 =JD 2437300.5  TAI-UTC=   1.4228180 S + (MJD - 37300.) X 0.001296 S
1961 AUG  1 =JD 2437512.5  TAI-UTC=   1.3728180 S + (MJD - 37300.) X 0.001296 S
1962 JAN  1 =JD 2437665.5  TAI-UTC=   1.8458580 S + (MJD - 37665.) X 0.0011232S
1963 NOV  1 =JD 2438334.5  TAI-UTC=   1.9458580 S + (MJD - 37665.) X 0.0011232S
1964 JAN  1 =JD 2438395.5  TAI-UTC=   3.2401300 S + (MJD - 38761.) X 0.001296 S
1964 APR  1 =JD 2438486.5  TAI-UTC=   3.3401300 S + (MJD - 38761.) X 0.001296 S
1964 SEP  1 =JD 2438639.5  TAI-UTC=   3.4401300 S + (MJD - 38761.) X 0.001296 S
1965 JAN  1 =JD 2438761.5  TAI-UTC=   3.5401300 S + (MJD - 38761.) X 0.001296 S
1965 MAR  1 =JD 2438820.5  TAI-UTC=   3.6401300 S + (MJD - 38761.) X 0.001296 S
1965 JUL  1 =JD 2438942.5  TAI-UTC=   3.7401300 S + (MJD - 38761.) X 0.001296 S
1965 SEP  1 =JD 2439004.5  TAI-UTC=   3.8401300 S + (MJD - 38761.) X 0.001296 S
1966 JAN  1 =JD 2439126.5  TAI-UTC=   4.3131700 S + (MJD - 39126.) X 0.002592 S
1968 FEB  1 =JD 2439887.5  TAI-UTC=   4.2131700 S + (MJD - 39126.) X 0.002592 S
1972 JAN  1 =JD 2441317.5  TAI-UTC=  10.0   S + (MJD - 41317.) X 0.0  S
1972 JUL  1 =JD 2441499.5  TAI-UTC=  11.0   S + (MJD - 41317.) X 0.0  S
1973 JAN  1 =JD 2441683.5  TAI-UTC=  12.0   S + (MJD - 41317.) X 0.0  S
1974 JAN  1 =JD 2442048.5  TAI-UTC=  13.0   S + (MJD - 41317.) X 0.0  S
1975 JAN  1 =JD 2442413.5  TAI-UTC=  14.0   S + (MJD - 41317.) X 0.0  S
1976 JAN  1 =JD 2442778.5  TAI-UTC=  15.0   S + (MJD - 41317.) X 0.0  S
1977 JAN  1 =JD 2443144.5  TAI-UTC=  16.0   S + (MJD - 41317.) X 0.0  S
1978 JAN  1 =JD 2443509.5  TAI-UTC=  17.0   S + (MJD - 41317.) X 0.0  S
1979 JAN  1 =JD 2443874.5  TAI-UTC=  18.0   S + (MJD - 41317.) X 0.0  S
1980 JAN  1 =JD 2444239.5  TAI-UTC=  19.0   S + (MJD - 41317.) X 0.0  S
1981 JUL  1 =JD 2444786.5  TAI-UTC=  20.0   S + (MJD - 41317.) X 0.0  S
1982 JUL  1 =JD 2445151.5  TAI-UTC=  21.0   S + (MJD - 41317.) X 0.0  S
1983 JUL  1 =JD 2445516.5  TAI-UTC=  22.0   S + (MJD - 41317.) X 0.0  S
1985 JUL  1 =JD 2446247.5  TAI-UTC=  23.0   S + (MJD - 41317.) X 0.0  S
1988 JAN  1 =JD 2447161.5  TAI-UTC=  24.0   S + (MJD - 41317.) X 0.0  S
1990 JAN  1 =JD 2447892.5  TAI-UTC=  25.0   S + (MJD - 41317.) X 0.0  S
1991 JAN  1 =JD 2448257.5  TAI-UTC=  26.0   S + (MJD - 41317.) X 0.0  S
1992 JUL  1 =JD 2448804.5  TAI-UTC=  27.0   S + (MJD - 41317.) X 0.0  S
1993 JUL  1 =JD 2449169.5  TAI-UTC=  28.0   S + (MJD - 41317.) X 0.0  S
1994 JUL  1 =JD 2449534.5  TAI-UTC=  29.0   S + (MJD - 41317.) X 0.0  S
1996 JAN  1 =JD 2450083.5  TAI-UTC=  30.0   S + (MJD - 41317.) X 0.0  S
1997 JUL  1 =JD 2450630.5  TAI-UTC=  31.0   S + (MJD - 41317.) X 0.0  S
1999 JAN  1 =JD 2451179.5  TAI-UTC=  32.0   S + (MJD - 41317.) X 0.0  S
2006 JAN  1 =JD 2453736.5  TAI-UTC=  33.0   S + (MJD - 41317.) X 0.0  S

(or the yet to be updated http://www.iers.org/MainDisp.csl?pid=95-106)

Here's slides from a presentation that is actually fairly well
balanced:
http://www.ien.it/luc/cesio/itu/mc_carthy.pdf
It has history there as well.  There's a nice graph of the above on
page 20.

http://www.ucolick.org/~sla/leapsecs/timescales.html also contains a
nice summary of UTC which is too long to include here but in outline
form is:

UTC during 1960
UTC from 1961 through 1971
UTC in 1963
UTC in 

Re: The real problem with leap seconds

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Francois Meyer <[EMAIL PROTECTED]> writes:
: On Mon, 16 Jan 2006, Mark Calabretta wrote:
:
: > On Fri 2006/01/13 11:17:52 -, Michael Deckers wrote
: > in a message to: LEAPSECS@ROM.USNO.NAVY.MIL
: >
: > >   I must get TAI, up to an integration constant. This is correct.
: > >   The integral of d( UTC ) is TAI (up to an integration constant),
: > >   but this integral is UTC only where UTC is a continuous function
: > >   of TAI.
: >
: > You're still not getting the point that UTC is just a representation
: > of TAI.
:
: Maybe it should be,  but  this  is  far  from  being
: obvious from its current definition.
:
: The actual situation corresponds to :
:
: 1. UTC and TAI  share  the  same  rate,  the  same
:origin, the same second. And therefore :
:
: UTC - TAI = 0

The history of UTC and TAI is complicated and tricky.  You can only
say that on Jan 1, 1972, the TAI - UTC offset was fixed to be 10s.
UTC and TAI prior to 1972 did not evolve at the same rate.  The UTC
seconds differed in length from the SI seconds.  I'll ignore the
nomenclature differences over time as well.

: 2. UTC only differs from TAI by  its  definitions of
:the minute, hour and day.

This is not true prior to 1972.  There were a number of frequency
offsets in UTC that weren't present in TAI.  Some leap second charts
have these included in them.

: 4. the UTC minute is defined  to  ensure  that  dhms
:expressions of UTC match UT1 at .9 s; it  can  be
:either  59,  60  or  61  SI  seconds  long.  This
:definition of the minute is realized
:by  (positive  or  negative)  leap  seconds   and
:ensures that the mean UTC day is the  mean  solar
:day in the long term. The UTC  hour  has  60  UTC
:minutes, the UTC day has 24 UTC hours.

Again, post 1972...  I'm not sure what I think of this definition.


: >From that point of view, the sentence from the ITU460-6 :
:
: "[UTC] ...differs from it [TAI] from an integer of seconds"
:
: should read :
:
: "representations of UTC  involving  minutes,  hours,
: days differ from equivalent representations  of  TAI
: by an integral number of seconds"

After 1972, and ignoring minor variations in the realization of UTC
and TAI in any given location, this is basically true.  The only ideal
difference between TAI(ideal) and UTC(ideal) is in the labeling of the
pulses that both time scales have experienced.  If one were to treat
them both as fixed radix, you get the difference of 33s.  Viewed
topologically as a 1-1 mapping, one could easily define subtraction so
that the difference is zero since UTC has a variable radix...

To further complicate things, TAI isn't created in realtime, but is a
paper clock that's steered to the correct time of the clocks that feed
it data.  There are clocks that are steered to this paper clock, but
it is all done by hand (if the various web pages I've read are still
accurate).  Different facilities realization of the TAI and UTC time
scales may differ by several nanoseconds (or more depending on a lot
of factors).

However, leaving aside those complications...

Given that UTC is a variable radix notation for labeling the pulses
that have elapsed since the epoch.  TAI is a fixed radix notation for
labeling pulses that have elapsed since the epoch.

Warner

Re: Report of Leap Second Problem with GPS Data

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
"Poul-Henning Kamp" <[EMAIL PROTECTED]> writes:
: In message <[EMAIL PROTECTED]>, Rob Seaman writes:
: >On Jan 13, 2006, at 6:26 AM, Richard Langley wrote:
:
: >I won't claim to know the intrinsic importance attached to this.
: >Critical systems may depend on the information.  But is it fair to
: >sum up the situation by saying that a leap second triggered a couple
: >of bugs (or perhaps one common bug), they were detected, have been
: >fixed, and affected data products have been remediated?  Also, it
: >appears that some other data products were unaffected?
: >
: >So, the issue has been resolved - would likely have been resolved
: >sooner if a leap second had occurred earlier - and is no longer
: >directly pertinent to a discussion of future leap seconds?
:
: Yeah, right
:
: "This goes counter to my claims so it is of no importance".
:
: Sorry, things don't work that way Rob.

This time, there were no reports of death with the leap second,
therefore they can't be too bad... :-)

Warner

Re: The real problem with leap seconds

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Mark Calabretta <[EMAIL PROTECTED]> writes:
: On Wed 2006/01/11 10:47:25 -, Michael Deckers wrote
: in a message to: LEAPSECS@ROM.USNO.NAVY.MIL
:
: >   At some instant when TAI took a value in the positive leap second between
: >   2006-01-01 + 00 h + 00 min + 32 s and 2006-01-01 + 00 h + 00 min + 33 s
: >   (the exact instant is not clear from [ITU-R TF.460-6 2002]), DTAI jumped
: >   from 32 s to 33 s; thus, UTC is not a monotone increasing function of
: >   TAI either.
:
: Here in a topology-free way is what the axis labels of my graph look
: like during the said leap second insertion:
:
: UTC axisTAI axis DTAI
:2005/12/31 23:59:58 2006/01/01 00:00:3032
:2005/12/31 23:59:59 2006/01/01 00:00:3132
:2005/12/31 23:59:60 2006/01/01 00:00:3232
: 60.932.9  32
: 60.99   32.99 32
: 60.999...   32.999... 32
:2006/01/01 00:00:00 2006/01/01 00:00:3333
:2006/01/01 00:00:01 2006/01/01 00:00:3433
:
: The seconds keep step and the graph has no gaps, jumps or kinks.

Shouldn't the DTAI increment for the leap second?  That's the point of
the leap second...  Or is that only needed when you do the POSIX
time_t thing of repeating 59?

In computer science, regular things are easy.  Irregular ones are hard
and prone to errors.  The :60 kludge makes a regular 1<->1 mapping
function for time into a table driven nightmare :-(.

: The only difference between UTC and TAI is one of representation, like
: the current year which may be represented as 2006 or MMVI but it's still
: the same year.

The point isn't that one can construct a 'second labelling' that is
unambiguous, monotonic and increasing, but rather that when one
distills the UTC time down to a single number, you necessarily have
ambiguities and discontinuties in that function.  Of course there's an
implicit assumption that the conversion function is the same as
POSIX's time_t, otherwise you would be using TAI or a count of actual
seconds.  We call it PPSes or pulses at work to distinguish it.  We
label UTC time as having leap seconds swizzled in, and TAI time as
unswizzled.

Warner

Re: The real problem with leap seconds

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Peter Bunclark <[EMAIL PROTECTED]> writes:
: On Mon, 9 Jan 2006, Poul-Henning Kamp wrote:
: >
: > I don't think anybody dare even think about redefining POSIX time_t
:
: I wish people would stop making positive assertions about what other
: people are bound to think.  What you mean in is, YOU are against
: suggesting changing (or augmenting) POSIX time.
:
: I't be interesting to do an FFT on this list, and see if some of the
: contributers actually ever sleep, or do any other work...

There's been many many many people that have tried to fix POSIX
time_t.  They have all met with great resistance from the standards
committee.  I'm guessing that these well documented cases are what phk
is familiar with when he makes statements like the above.

Warner

Re: interoperability

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
John Cowan <[EMAIL PROTECTED]> writes:
: > But how in practice is it envisaged that a scheme
: > for migrating time zones versus TAI would work, precisely?
:
: Straightforwardly.  Each locality decides when and how to adjust both
: its offset from TAI and its seasonal transition function (if any),
: just as it does today.  What we abandon is a universal time tightly
: synchronized to Earth rotation in favor of a universal time
: independent of earth rotation plus 400+ local civil times roughly
: synchronized to Earth rotation containing various glitches.

No matter what we do with leapseconds, there are still all those time
zones.

The problem with stopping leap seconds altogether is that the legal
definitions of time, although quite varied, are all about the same as
UTC as it exists today.  They are close enough that most countries
have adopted UTC bureaucratically rather than legislatively.  The
official time for the US, as published by the folks at NIST, is UTC.
The US law says mean solar time, as determined by the Secretary of
Commerce, who has delegated it to the Time and Frequency division of
NIST, who in turn use UTC.  NIST could easily use a different schedule
for leap second insertion (it could have inserted the leap second in
civilian time at the end of any day it wanted to and still maintained
the mean solar time legal requirement).  However, since UTC is a
recognized, international standard, the US went along and did its leap
second according to that standard.  This is a explicit choice that
someone, somewhere had to make, even though it is arguably the best
choice to make (wouldn't want to be the odd man out in civil time,
think of the impact on business).

The combination of UTC approximating the legal time is so man nations,
as well as the need for international consensus among lots of parties
with divergent views for any changes to the current system is why
we'll likely not see significant changes any time soon.  The best we
can hope for is that something will be done to change their
unpredictable nature given that we have good forcasting tools at our
disposal.

Warner

Re: The real problem with leap seconds

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
"Poul-Henning Kamp" <[EMAIL PROTECTED]> writes:
: In message <[EMAIL PROTECTED]>, Ed Davies writes:
: >Wow, things have got really stirred up around here.  Lots of interesting
: >points but I'll just concentrate on one...
: >
: >Poul-Henning Kamp wrote:
: >> Well, the BIPM doesn't really want anybody to use TAI, their director
: >> said as much last year, and I can see where he is coming from on that
: >> one.
: >
: >Since the usual response of the pro-leap second lobby to people
: >who want a uniform timescale is "use TAI" this is significant.
: >Do you have any information or references on why the BIPM director
: >said this?
:
: As I understood it, it was mainly that TAI is a post-factum "postal"
: timescale.

How is it that UTC can be realized in realtime, but TAI isn't.  I
thought the difference between the two was an integral number of
seconds, by definition.  Is that understanding flawed?

Wanrer

Re: predicting leap seconds

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Neal McBurnett <[EMAIL PROTECTED]> writes:
: I still haven't seen any good data on predictions for periods of
: longer than 9 years.

Neal,

thanks for the excellent summary of the current state of the art in
prediction.

I think this shows that a 20 year time line is unrealistic at this
point, but 5-10 years would keep things fairly close, and 4 years
should be able to keep the current tolerances.  It might be worth an
experiment where over the next 5 years IERS publish 12 new months of
data every 6 months.  (eg Jan 2006 publish both the June 2006 and Dec
2006 correct, July 2006 publish the June 2007 and Dec 2007 correction,
Jan 2007 publish Jun 2008 and Dec 2008).  We'd hit 4 years in advance
in Jan 2009.  This would phase in the predictive timeline for leap
second insertions, and would also give the IERS control to end the
experiment if the time horizons exceeded their ability to predict with
confidence.  This would be an evolutionary step, rather than a
revolutionary one.  Of course this would make them even more
entrenched than they already are, because to kill them would require
waiting many years...

Warner

Re: The real problem with leap seconds

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
"Daniel R. Tobias" <[EMAIL PROTECTED]> writes:
: On 7 Jan 2006 at 16:02, Poul-Henning Kamp wrote:
:
: > Civil time is in the hands of individual governments, and they
: > tend to expect their computers to use the same time as the
: > rest of their country.
:
: And, in many countries (including the United States), the legally-
: defined civil time is the mean solar time at some particular spot, or
: a fixed or seasonally-variable offset from it.  Any use of UTC-based
: time scales for determining civil time in such places is merely an
: approximation, currently to within a second, but perhaps varying by
: greater amounts if some new timekeeping plan is adopted.  Once UTC
: stopped being a sufficiently-close approximation to the solar mean
: time at the Prime Meridian (with "sufficiently close" possibly being
: of differing values depending on the particular purpose), it would be
: necessary to either stop using UTC for determining civil time in such
: countries, or to change the law to base the civil time on UTC instead
: of a solar standard.

The US's legislation leaves it to the commerce department to define
what the mean solar time is (or rather states that it is the mean
solar time, as determined by the secretary of commerce).  This is what
presumably gives us leap seconds and the like in our civil time and
when we insert them.  This is a political function: Up to a second of
tolerance is allowed and leap seconds are inserted whenever everybody
else does.  The same political decision could be made to say something
else, since it is the folks in DC could say that a minute is close
enough to solar mean and that's what we're going to do.

Warner

Re: Double 59; 60; or double 00?

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
"Tom Van Baak" <[EMAIL PROTECTED]> writes:
: With the surge of leap second captures this time
: around, are there any concerns over the growing(?)
: use of double :59 second or double :00 second
: instead of :59:60 for a positive leap second?
:
: Although not technically correct, they do seem a
: practical, perhaps even clever, alternative -- in some
: cases -- to the upstream parsing trouble that the
: exceptional :59:60 causes.
:
: Given a choice I would vote for double :59 over
: double :00 since :59 is still in the previous day
: (where the leap second occurs). Also, :59 offers
: more symmetry between positive and negative
: leap seconds (58 and 00 are constant; 59 is
: double or nothing depending on leap type).

ntp implements the double 59, and that works well when one can't
represent the number as :60 (which is impossible to do in POSIX
time_t).

POSIX, interestingly enough, doesn't define what happens at the
leapsecond very well.  On the one hand, they officially don't exist.
On the other, if you convert 23:59:60 to a time_t using mktime, you
get 0:00:00 due to normalization that happens in mktime (a more or
less accidental effect).  This leads some to conclude that the proper
POSIX sequence is with two 0's in a row.  While this is a conclusion
from reading POSIX, it isn't the only conclusion.

Warner

Re: Longer leap second notice

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: On Jan 3, 2006, at 5:46 PM, Warner Losh wrote:
:
: > As someone who has fought the battles, I can tell you that a simple
: > table is 10x or 100x easier to implement than dealing with parsing
: > the data from N streams.  Sure, it limits the lifetime of the
: > device, but a 20 year limit is very reasonable.
:
: Simpler is indeed better - if it satisfies the requirements.  While
: we're at it - how about a table to describe worldwide daylight saving
: rules?  Oh right - we already have that :-)  What we don't have is a
: mechanism to force the U.S. Congress not to change the rules out from
: under us.  Retaining the flexibility to easily change the rules is
: one of our requirements.

You are comparing two dissimilar things.  A more apt comparison would
be to the leap year rules that we have.  We know the rules going
forward a thousand years or so.  These too represent the fact that the
oribit arount the earth is not exactly 365.25 days.  We add leap
seconds because the rotation of the earth isn't exactly 86400s.  The
difference between them is that the earth's rotation around the sun is
more stable than its rotation on its axis.

Daylight savings time is something else entirely.  It is a political
decision that sunlight is better used in the evening than the morning.

: Twenty years does seem reasonable.  Would suggest this might be
: marketed as an extended cadence maintenance requirement, rather than
: as an expiration date - suspect astronomers aren't the only ones to
: rely on 30 year old computers on occasion.  I would heartily agree
: with the notion that a twenty year horizon is about appropriate for
: expecting to reach any decision on the future of UTC.  We'd be a lot
: further ahead on this if a closed door decision hadn't been rushed
: for the imagined benefit of the few.  In the mean time, there are
: many members of the astronomical software community who would be
: happy to contribute to an effort to improve time handling
: infrastructure and standards, rather than spending their own precious
: time fending off ill conceived political machinations.

Twenty years is an example number.  Ideally, as predictive science
gets better, we can do it for longer periods of time.  One would hope
to eventually have a schedule that's published 50 or 100 years in
advance.  Leap years have been published far in advanced for thousands
of years now, with only one correction to the rule about 700 years ago
as the measurement of the year was refined.  As we've refined it
further in the last few hundred years, we know, with a very high
degree of confidence, that the rule is good for at least a thousand
years.  The rule isn't perfect, as almost a full day of error can
accumulate in 400 years (requiring an extra leap day), but it does
bound the error nicely.

If we can extend the horizon from 6 months, then that would lead to
better predictibilty of leap seconds, and also allow for better
testing.

Of course, a rule that eliminates them entirely would also fit these
needs, but appears to have little support...

: > If we could have a table for the next 20 years, there'd be no need
: > to even write the code to get from the GPS stream :-)
:
: And if latitude and longitude were engraved on every street corner,
: there would be no need for GPS at all :-)  Transport of time signals
: to remote locations is the whole point.

I think it would have been better if I'd stated my point more
directly: Multiple sources of information about leap seconds leads to
a more robust system.  GPS can tell us about it, ntpd can tell us
about it, and having a table of known leap seconds can inform us.
These redundant sources of information act as a sanity check.  In the
development box that did do the leap second correctly, a backup source
of infomration allowed it to perform correctly over the leap second
despite its development not being complete.

: But should any of us be open to persuasion by a "political tool to
: make the proposal go through without commiting anybody to anything
: for the next couple of hundred of years"?

There are a number of politically expedient quick fixes.  Foisting it
off on future generations is a time honored political solution :-(.

: > I find your dismissive attitude towards software professionals that
: > have implemented a complete leap second handling infrastructure,
: > with pluggable sources for leap second rather annoying :-(
:
: Indeed, I'm sure I've exhausted my scant store of good will again.

My statement was born of frustration.  It was a little uncalled for.
I regret sending it as it was a cheap shot.

: Would be delighted to hear more about your leap second infrastructure.

In brief, we have a leap second table.  This table can be populated
from a number of different sources, usually via a table we've hard
coded into our products.  As the products run and discover new leap
seconds, these are added to the t

Re: text book example why Leapseconds are bad

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Peter Bunclark <[EMAIL PROTECTED]> writes:
: And these "Rocket Scientists" can't even spell. Perhaps they can't read,
: either. Surely 5 decades into the space age there exists a "How to build a
: Spacecraft" text book with a chapter on timescales?

I don't have high hopes.  It took me almost 3 years of working for a
company whose business was high precision time and frequency systems
to learn about TAI.  This despite asking our time scientists if there
was a universally accepted time scale without leap seconds.  When I
asked them about TAI later, they told me they knew about it when I
asked, but its use was generally discouraged and none of our customers
ever wanted it.

It took another years of piecing things together from different
sources to understand where the canonical information about leap
seconds, frequency adjustments (pre-leapseconds) and the like was to
be found.

The information is out there, but it is so dispersed that it can be
hard for someone to find.  Maybe the latest leap second will cause
this situation to change...

Warner

Re: December 2005 leap second on MSF, Rugby, England

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
"Joseph S. Myers" <[EMAIL PROTECTED]> writes:
: The Linux kernel (with NTP synchronisation) duly syslogged
:
: Dec 31 23:59:59 digraph kernel: Clock: inserting leap second 23:59:60 UTC
:
: and Markus's program showed a transition from 1136073600.005623 to
: 1136073599.013130, i.e. 23:59:59 was repeated (whereas a POSIX clock in
: perfect synchronisation with UTC should appear to repeat 00:00:00, and a
: kernel source code comment
:
: /* The timer interpolator will make time change gradually instead
:  * of an immediate jump by one second.
:  */
:
: describes what would practically be a safer approach, if the code followed
: the comment).

ntp synced computers repeat the last second of the day.  POSIX is just
flat wrong here.  Actually, POSIX is completely silent on what is to
happen at the leap second to system time.  Once can infer what might
be the right behavior by working backwards from what mktime does for
23:59:60, but that's a weak inference for what the 'right' posix
behavior is for the system time accross a leap second.  POSIX simply
does not have leap seconds in any meaningful way.  It is as if they do
not exist at all.  There's no clarifications to the POSIX standard
that say that 0:00:00 should be repeated at a leap second, at least
that I've been able to find.  If there is such an explicit
clarification, I'd like to know about it.

FreeBSD does the same thing (as do all kernels that use the ntp kernel
stuff supplied with udel ntpd and successors).  FreeBSD doesn't
actually log the leap second via syslog, but maybe it should.

Ideally, we'd switch to running computers with TAI and doing all the
conversion on input/output of time.  However, *THAT*'s definitely not
POSIX.

Warner

Re: leap seconds and Linux/Unix, timezones, and zdump

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Neal McBurnett <[EMAIL PROTECTED]> writes:
: Here are some notes on facilities in some Unix systems to show
: evidence of leap seconds.  Some recent distributions are out of date
: as noted below.  Can folks check others (Solaris, *BSD, etc?).

I'll comment for FreeBSD.

: First, this should work for everyone who cares about time and runs
: ntp.  Make sure you're in NTP sync with a host that knows a leap
: second is coming.  Look for "leap=01" in the output of this:
:
:  $ ntpq -c rv
:  assID=0 status=46f4 leap_add_sec, sync_ntp, 15 events, event_peer/strat_chg,
:  version="ntpd [EMAIL PROTECTED]:4.2.0a-11-r Mon Mar 14 12:39:28 GMT 2005 
(1)"?,
:  processor="i686", system="Linux/2.6.10-6-386", leap=01, stratum=3,
:  precision=-19, rootdelay=158.366, rootdispersion=77.103, peer=17708,
:  refid=82.211.81.145,
:  reftime=c76127cd.19f9acff  Sat, Dec 31 2005  8:52:45.101, poll=7,
:  clock=0xc7612b1b.74322291, state=4, offset=-0.430, frequency=37.132,
:  error=0.572, jitter=1.795, stability=64.092

FreeBSD does this in a similar way, since it is all based on ntp.

: Next check to see if your time zone files are up-to-date.  I'm no
: expert (help!) but it appears that on many Unix systems, the "Olson"
: notion of "right/" timezones allows the system to understand leap
: seconds.

The so-called "right" files aren't installed on FreeBSD by default.
Instead, at zone compilation time, the leap seconds are added or not.
The default is for them not to be added, likely to comply with the
lame and broken requirements of POSIX's time_t.

Warner

Re: Schreiver AFB warns about leapsec

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Francois Meyer <[EMAIL PROTECTED]> writes:
: On Tue, 20 Dec 2005, Rob Seaman wrote:
:
: > On Dec 20, 2005, at 1:30 AM, Poul-Henning Kamp wrote:
: >
: > > There is an interesting PowerPoint (sigh...) at Schriever AFB's GPS
: > > support center:
: > >
: > > https://www.schriever.af.mil/GpsSupportCenter/archive/advisory/
: > > Leap_Second_Event.ppt
: >
: > Agreed.  Very interesting.
: > ...
: > An interesting observation:
: >
: > - Leap second occurs at an awkward time - New Years Eve
: >
: > Maybe obscurity in scheduling and implementation is not a desirable
: > characteristic after all.
:
: The same paradigm suggests a new definition of  UTC,
: strengthening its link to UT1  down  to  0.09s,  and
: switching from leap seconds  to  leap  tenths  of  a
: second. This aims at making leap  intervals  a  rule
: and not an exception. Tens of a second are  as  easy
: (or as difficult)  to  implement  as  leap  seconds,
: their instantaneous impact is  10  times  lower  and
: since only automated systems  are  really  affected,
: the increased frequency of their occurrence  is  not
: an issue.

I'd take issue that .1th of seconds are easier to implement,
especially given that most of the timing gear today would have to be
discarded because it isn't compliant.  All the time signals would need
to be redesigned.  A lot of software would need to be rewritten to
cope with the smaller offsets (not least ntp).  There's a huge
deployed base of gadgets that grok leap seconds as they are today and
that would become obsolete.

Not to mention that .1s jumps would make it harder to correlate data
back to TAI (the tables you'd need are more complicated).

.1s jumps would happen more often, so would be more unpredictable than
the leap seconds we have today.  My main beef with them is that we
have no way of knowing today, with certainty, if there will be a leap
second on June 30, 2006.  The schedule is essentially random, which is
a problem for devices that need to know utc-tai offset given an
approximate time.

I don't see how that would ever work.

Warner

Re: a system that fails spectacularly

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
"Clive D.W. Feather" <[EMAIL PROTECTED]> writes:
: M. Warner Losh said:
: > >> * A second is represented by an integer from 0 to 61;
: [...]
: > >> but this specification
: > >> follows the date and time conventions for ISO C.
:
: Of course, ISO C fixed this misunderstanding many years ago.

True, but the Java spec that was referenced didn't...

Warner

Re: a system that fails spectacularly

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Ed Davies <[EMAIL PROTECTED]> writes:
: The Java Date class documentation does, at least, show reasonable
: awareness of leap seconds:
:
:http://java.sun.com/j2se/1.5.0/docs/api/java/util/Date.html
:
: Pity those of us who've been reading this list a little too long
: can pick a few minor holes but it's basically reasonable stuff -
: much better than ignoring leap seconds.


>> * A second is represented by an integer from 0 to 61; the
>> values 60 and 61 occur only for leap seconds and even then only
>> in Java implementations that actually track leap seconds
>> correctly. Because of the manner in which leap seconds are
>> currently introduced, it is extremely unlikely that two leap
>> seconds will occur in the same minute, but this specification
>> follows the date and time conventions for ISO C.
-^

Followed by:

>> The values 60 and 61 can only occur on those Java Virtual
>> Machines that take leap seconds into account.

So they adopted the broken ISO C definitions, but made them optional :-(.

Warner

Re: a system that fails spectacularly

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
"Poul-Henning Kamp" <[EMAIL PROTECTED]> writes:
: ISO9000 certification only means that you have documented your
: quality assurance process.
:
: There is no requirement that your documentation pertains to
: or results in a quality product.
:
: One of the Danish ISO9K consultants used to bring a ISO9000
: certification case along to explain this to companies:  It was
: basically the entire ISO9000 process for a small company written
: on one page of paper and the essence was "We don't gove a hoot about
: quality".
:
: The information that company X is "iso9000 certified" only conveys
: one bit of information:  The company has a quality policy.
:
: You still need to read their quality policy to know what it is,
: and on average, the ISO9000 certified ones contain less usable
: or even readable information, than the other kind.

ISO 9000 only requires like 3 or 5 documents.  Small companies can
comply with just a notebook that contains these documents, assuming
that the quality policy doesn't require more...

Warner

Re: The Truth is Out There

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Steve Allen <[EMAIL PROTECTED]> writes:
: After all, it's not like the world is going to come to an end.

If the end of the world is what it takes to abolish leap seconds, I'm
all for it :-)

Warner

Re: The Trouble with POSIX

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Peter Bunclark <[EMAIL PROTECTED]> writes:
: John Cowan
: adds:
...
: The upside of Posix is that time arithmetic is simple.  The downside is
: that Posix sometimes lies about elapsed time and labels both a leap second
: and the preceding second with the same time_t.

POSIX lies, but it tells a different lie.  It labels the leap second
as the following second, and then does the following second again.

NTP lies in the manner that John describes, which is inconsistant with
POSIX.  Most systems disciplined by NTP have their system time follow
this convention, so you get weird results with the POSIX time routines
near a leap second.

In summary:

UTC Label   Posix   NTP
23:59:59799 799
23:59:60800 799
00:00:00800 800
00:00:01801 801

Warner

Re: RAS hits the news

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
"Tom Van Baak" <[EMAIL PROTECTED]> writes:
: Too much is made of the "overflow". Fields rollover all
: the time in real life and it's often a simple engineering
: matter to take this into account. Not sure I would call
: it "cheating". We get by fine with just 7 names for days
: of the week; calendars that rollover every 12 months,
: wristwatches that overflow every 12 hours...

These instances of overflow come from remainders of division
operations overflowing.  They all can be derived from a single base
number (say number of seconds since 1970, MJD, etc).  However, when
you are deriving that single base number, it can be much harder.

Could you tell me what year it was if I told you it was Monday,
October 15th?  No, you couldn't.  You could tell me that it might be
2001, but it could also be 2007 or 1990.  You need more information to
resolve the ambiguity.  If I told you that it was Monday, October 15th
and that TAI-UTC=32, you'd know it was 2001.  If I told you
TAI-UTC=100, you might guess that it is 2063, or maybe even 2068 or
2057 or maybe other years earlier or later depending on your leap
second model, utc-ut1 tolerance parameters, and other factors
unknowable today.

The GPS 1024 week overflow is easier to deal with, since it is a 20
year ambiguity, not a 5 year one.  You can make a better guess than in
my example, I'll not argue.  The better the guess you make based on
today's understanding, the more external factors that might cause you
to be wrong.  Eg, leap second rules changes, lifetime of GPS signals,
etc.

I guess I agree with you that these things are doable.  Working out
the details, however, makes them non-trivial.

Warner

Re: RAS hits the news

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
"Poul-Henning Kamp" <[EMAIL PROTECTED]> writes:
: At least the Oncore receives will happily use a 2 year old alamanc
: to aid in getting first fix.

I've seen some receivers that don't do this, and consequently have
trouble getting first fix.  These may be the Oncore VP that we stopped
deploying years ago, but are in our leap second testing matrix because
important customers still have them deployed.  I can watch our status
screen as they happily cycle through different sets of satellites a
couple a minute until they get one good one...  Of course the VPs we
used loose all knowledge when power goes away...

: But they will not use anything but the current almanac to report
: leap seconds.

I think that it depends on the model of oncore receiver. The M12+
appears to cache the almanac wrt leap seconds for a period of time
after power is removed from them (I'm sure it does this if the power
is off for minutes, I'm sure it doesn't if it is off all weekend, but
don't know where the cutoff point is).  Other GPS recievers, from
other manufacturers, that we've tested will report the alamanac data
even after being off all weekend.  The older VP will not cache the
almanac data at all beyond the charge in the capacitors in the power
supply when power is removed.

Warner