Re: UT1 confidence

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Zefram <[EMAIL PROTECTED]> writes:
: Steve Allen wrote:
: >http://www.ucolick.org/~sla/leapsecs/torino/arias_3.pdf
:
: This is the really interesting one.  They present the accuracy of
: simulated predictions of UT1, and that accuracy is much poorer than
: the figures we've been discussing so far.  But they make it clear that
: the prediction is by a naive algorithm, not even applying the well-known
: periodic terms that are used in UT2.  This is obviously not the algorithm
: used by IERS.

The good news is that even an incredibly stupid (in terms of knowledge
used) algorithm can predict almost 3 years out

Warner

Re: UT1 confidence

[M. Warner Losh]

Steve,

thank you for this enlightening report.

In message: <[EMAIL PROTECTED]>
Steve Allen <[EMAIL PROTECTED]> writes:
: On Wed 2007-01-17T12:31:14 -0700, Warner Losh hath writ:
: > It has been remarked that the current state of the art is that 100ms
: > accuracy can be predicted about a year in advance only and that the
: > models are constantly undergoing refinement.  It has been estimated
: > that IERS could issue leap seconds, with today's technology, about 3-5
: > years out and still be in a 95% or 99% band of certainty that the 0.9s
: > margin is maintained.  However, I can't find papers that show these
: > models or point to any better data than hearsay...
:
: The best that I know of were the ones presented at the Colloquium that
: the WP7A SRG held in Torino in May 2003.  There was a time when the
: host institution (IEN) was providing the proceedings online, but the
: contents of that URL went away sometime around a year ago.  (I wonder
: if they may not have liked the conclusion that was reached.)
:
: In the spirit of promulgation I provide what they once did at
: http://www.ucolick.org/~sla/leapsecs/torino/ITU.shtml
:
: The conclusion was originally a powerpoint drafted in real time, it is
: http://www.ucolick.org/~sla/leapsecs/torino/closure.pdf
:
: The indications of how well predictions of UT1 might be done are found
: in three presentations to which Felicitas Arias contributed.
: There are two which were powerpoint
: http://www.ucolick.org/~sla/leapsecs/torino/guinot.pdf
: http://www.ucolick.org/~sla/leapsecs/torino/arias_2.pdf
: and one which is a more verbose writeup of one of the powerpoints
: http://www.ucolick.org/~sla/leapsecs/torino/arias_3.pdf

I like figure 8, that shows that 20ms steps lead to 50ms steps lead to
100ms steps lead to 1s steps. :-)

I also like the quotes:

"Dating in UTC is ambiguous when a positive leap second occurs in
systems other than those using hours, minutes and seconds; in this
latter system the use of second "60" may a cause of difficulty."

and

"Thus UT1 is not, strictly speaking, a form of solar time"

Also, did I miss figure 9 in arias_3?

Proposal II has gotten much press here (the leap hour one), but
Proposal I sounds a lot like what I've suggested: Use TAI time and let
countries move the time zones when they feel like they no longer are
close enough, but it kinda omits that last part...

Warner

Re: UT1 confidence

[Warner Losh]

> In message <[EMAIL PROTECTED]>, Zefram writes:
> >IERS Bulletin A gives an expression for the uncertainty of its UT1-UTC
> >data predictions:
> >
> >S t = 0.00025 (MJD-today)**0.75
> >
> >where "today" is the MJD of the bulletin's publication.  The Bulletin
> >only predicts a year ahead.  Applying that formula gives an uncertainty
> >a year ahead of 21 ms.
>
> The question is what domain of validity the above formula has ?
>
> In the builletin they only apply it up to 40d.

In addition, since the drift is a higher order polynomial (13th order
I recall hearing), a simple linear formula (or close approximation) is
likely only going to fit the curve near the prediction date with wider
variances the further one gets from today.

It has been remarked that the current state of the art is that 100ms
accuracy can be predicted about a year in advance only and that the
models are constantly undergoing refinement.  It has been estimated
that IERS could issue leap seconds, with today's technology, about 3-5
years out and still be in a 95% or 99% band of certainty that the 0.9s
margin is maintained.  However, I can't find papers that show these
models or point to any better data than hearsay...

Warner

Timekeeping data

[Warner Losh]

In the last about time stamps and time representations, I took the
position that kernel timestamps had to be as simple as possible and
time efficient as possible to not impact overall system performance.
I was asked for data about timekeeping overhad causing problems to
backup my claims.  I've spent a little time looking around, and I
cannot point to a paper on the topic.  However, I have found raw data
to support that small changes in timekeeping can have mesruable
effects on macro benchmarks:

http://files.gslin.org/Misc/mysql5-freebsd6-bench.txt

Comparing apples to apples, changing only from TSC to ACPI-Fast, we
see the following results for one of the elements in the testing
matrix (others are similar):

ACPI-Fast
select_index2   0   0   14097.47
select_index2   0   0   13741.43
select_index2   1   0   13704.01
select_index2   0   0   13626.05
select_index2   0   0   13769.32

vs

TSC
select_index2   1   0   13638.09
select_index2   1   0   15204.89
select_index2   0   0   15126.16
select_index2   1   0   15199.22
select_index2   1   0   15111.09

The far right column is transactions per second.  Even the 'eyeball'
test suggests that there's a signficant different.  Eliminating the
one outlier from each dataset shows the difference here is big:

N   Min   MaxMedian   Avg Stddev
x   4  13626.05  13769.32  13722.72 13710.203  62.155788
+   4  15111.09  15204.89  15162.69  15160.34  48.614784
Difference at 95.0% confidence
1450.14 +/- 96.546
10.5771% +/- 0.704191%
(Student's t, pooled s = 55.7976)

This pattern is repeated again and again the different data sets,
although I won't bore the list with repetition (I just quickly
eyeballed them).  The only difference between the two runs above is
that the 'TSC' time counter takes on the order of a 2ns, while
ACPI-Fast is closer to 60ns (plus overhead of the time keeping system,
which is the same for each timecounter).  This small difference
results in a 10% tps difference for these tests.

While this is not a scientific study, and hasn't been put through the
riggors of peer review, it is a pattern than has repeated itself in
many of the other benchmarks that were done at the same time.  It
certainly is suggestive of a correlation between the time it takes to
do timekeeping and overall system performance.

It certainly would be worth testing other timestamp/timekeeping
operations to see if they also produce a similar effect, but I've not
had the time to implement something to test that hypothesis.  It is a
reasonable inference that overhead in timestamp math would have
similar effects to the timekeeping overhead, perhaps more so, as
timestamp math is done more often than raw timekeeping generation of
timestamps.

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Tony Finch <[EMAIL PROTECTED]> writes:
: On Sat, 6 Jan 2007, M. Warner Losh wrote:
: >
: > Unfortunately, the kernel has to have a notion of time stepping around
: > a leap-second if it implements ntp.
:
: Surely ntpd could be altered to isolate the kernel from ntp's broken
: timescale (assuming the kernel has an atomic seconds count timescale)

ntpd is the one that mandates it.

One could use an atomic scale in the kernel, but nobody that I'm aware
of does.

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: What is correct is to have a 61 second minute occasionally, neither
: to redo the first second of the next day, nor to repeat the last
: second of the current day.

Unfortunately, that's not POSIX time_t.  And when you are implementing
unix kernel APIs, that is the only game in town, unless you invent
your own.  ntp_gettime comes close to implementing things right (on
FreeBSD the time state is synchronous to top of second), but even that
is a mess.

You can't do both 61 second minutes and also have the 'naive math'
that time_t requires.

I agree with others that have said that ntp should be TAI or GPS based
(eg a linear count of seconds in a timescale that doesn't have leap
seconds), and that leap second corrections should happen in userland
like is done for timezones.  That's really the only sane way to do
things.  Having tried it, there are a lot of little 33 second
anomolies in many applications :-(.  I've toyed with the idea of
running the kernel in TAI and having 'smart' processes tell libc they
want no UTC translation and having all the TAI<->UTC translation
happen in libc (also hacking those FS that want UTC time to be able to
get it).  Maybe I'll try it and see what breaks since the line between
applications and libraries can make a per-process flag hard to cope
with if your application links with a lot of libraries.

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: > If by "some limp attempt" you mean "returns the correct time" then you
: > are correct.
:
: It's not the correct time under the current standard if the
: timekeeping model doesn't implement leap seconds correctly.  I don't
: think this is an impossible expectation, see http://
: www.eecis.udel.edu/~mills/exec.html, starting with the Levine and
: Mills PTTI paper.

It implements exactly what ntpd wants.  I asked Judah Levine when
determining what was pedantically correct during the leap second.  I
also consulted with the many different resources avaialable to
deterimine what the right thing is.  Of course, there are different
explaintions about what the leap second should look like depending on
if you listen to Dr. Levine or Dr Mills.  Dr. Mills web site says
'redo the first second of the next day' while Dr. Levine's
leapsecond.dat file says 'repeat the last second of the day.'
Actually, both of them hedge and say 'most systems implement...'  or
some variation on that theme.

It is possible to determine when you are in a leap second using ntp
extensions with their model.  Just not with POSIX interfaces.  The
POSIX interfaces are kludged, while the ntpd ones give you enough info
to know to print :59 or :60, but POSIX time_t is insufficiently
expressive, by itself, to know.  But ntp_gettime returns a timespec
for the time, as well as a time_state for the current time status,
which includes TIME_INS and TIME_DEL for psotive and negative leap
second 'warning' for end of the day so you know there will be a leap
today, and TIME_WAIT for the actual positive leap second itself
(there's nothing for a negative leapsecond, obviously).

So I stand by my "returns the correct time" statement.

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Tony Finch <[EMAIL PROTECTED]> writes:
: On Sat, 6 Jan 2007, M. Warner Losh wrote:
: >
: > Most filesystems store time as UTC anyway...
:
: POSIX time is not UTC :-)

True.  It is designed to be UTC, but fails to properly implement UTC's
leap seconds and intervals around leapseconds.

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: Warner Losh wrote:
: > Anything that makes the math
: > harder (more computationally expensive) can have huge effects on
: > performance in these areas.  That's because the math is done so often
: > that any little change causes big headaches.
:
: Every IP packet has a 1's complement checksum.  (That not all
: switches handle these properly is a different issue.)

Actually, every IP does not have a 1's complement checksum.  Sure,
there is a trivial one that covers the 20 bytes of header, but that's
it.  Most hardware these days off loads checksumming to the hardware
anyway to increase the throughput.  Maybe you are thinking of TCP or
UDP :-).  Often, the packets are copied and therefore in the cache, so
the addition operations are very cheap.

: Calculating a
: checksum is about as expensive (or more so) than subtracting
: timestamps the right way.  I have a hard time believing that epoch<-
:  >interval conversions have to be performed more often than IP
: packets are assembled.

Benchmarks do not lie.  Also, you are misunderstanding the purpose of
timestamps in the kernel.  Adding or subtracting two of them is
relatively easy.  Converting to a broken down format or doing math
with the complicated forms is much more code intensive.  Dealing with
broken down forms, and all the special cases usually involves
multiplcation and division, when tend to be more computationally
expensive than the checksum.

: One imagines (would love to be pointed to
: actual literature regarding such issues) that most computer time
: handling devolves to requirements for relative intervals and epochs,
: not to stepping outside to any external clock at all.  Certainly the
: hardware clocking of signals is an issue entirely separate from what
: we've been discussing as "timekeeping" and "traceability".  (And note
: that astronomers face much more rigorous requirements in a number of
: ways when clocking out their CCDs.)

Having actually participated in the benchmarks that showed the effects
of inefficient timekeeping, I can say that they have a measurable
effect.  I'll try to find references that the benchmarks generated.

: > Well, the kernel doesn't expect to be able to do that.  Internally,
: > all the FreeBSD kernel does is time based on a monotonically
: > increasing second count since boot.  When time is returned, it is
: > adjusted to the right wall time.
:
: Well, no - the point is that only some limp attempt is made to adjust
: to the right time.

If by "some limp attempt" you mean "returns the correct time" then you
are correct.

: > The kernel only worries about leap
: > seconds when time is incremented, since the ntpd portion in the kernel
: > needs to return special things during the leap second.  If there were
: > no leapseconds, then even that computation could be eliminated.  One
: > might think that one could 'defer' this work to gettimeofday and
: > friends, but that turns out to not be possible (or at least it is much
: > more inefficient to do it there).
:
: One might imagine that an interface could be devised that would only
: carry the burden for a leap second when a leap second is actually
: pending.  Then it could be handled like any other rare phenomenon
: that has to be dealt with correctly - like context switching or
: swapping.

You'd think that, but you have to test to see if something was
pending.  And the code actually does that.

: > Really, it is a lot more complicated than just the 'simple' case
: > you've latched onto.
:
: Ditto for Earth orientation and its relation to civil timekeeping.
: I'm happy to admit that getting it right at the CPU level is
: complex.  Shouldn't we be focusing on that, rather than on
: eviscerating mean solar time?

Did I say anything about eviscerating mean solar time?

: A proposal to actually address the intrinsic complications of
: timekeeping is more likely to be received warmly than is a kludge or
: partial workaround.  I suspect it would be a lot more fun, too.

I'm just suggesting that some of the suggested ideas have real
performance issues that means they wouldn't even be considered as
viable options.

: > Kernels aren't written in these languages.  To base one's arugments
: > about what the right type for time is that is predicated on these
: > langauges is a non-starter.
:
: No, but the kernels can implement support for these types and the
: applications can code to them in whatever language.  Again - there is
: a hell of a lot more complicated stuff going on under the hood than
: what would be required to implement a proper model of timekeeping.

True, but timekeeping is one of those areas of the kernel that extra
overhead is called so many times that making it more complex hurts a
lot more than you'd naively think.

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Tony Finch <[EMAIL PROTECTED]> writes:
: On Sat, 6 Jan 2007, Ashley Yakeley wrote:
: >
: > Presumably it only needs to know the next leap-second to do this, not
: > the whole known table?
:
: Kernels sometimes need to deal with historical timestamps (principally
: from the filesystem) so it'll need a full table to be able to convert
: between POSIX time and atomic time for compatibility purposes.

Most filesystems store time as UTC anyway...

And that's one reason that kernels do a lot of timestamp operations:
whenever a file is touched, one has to update the time it was last
touched.  Then, when the file is statted, that time must be returned.
That makes it very hard to do all in libc because of different boot
times or nfs, etc.  While one could do the degenerate case of
gettimeofday in libc, these other cases are much harder.

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Ashley Yakeley <[EMAIL PROTECTED]> writes:
: On Jan 6, 2007, at 16:18, M. Warner Losh wrote:
:
: > Unfortunately, the kernel has to have a notion of time stepping around
: > a leap-second if it implements ntp.  There's no way around that that
: > isn't horribly expensive or difficult to code.  The reasons for the
: > kernel's need to know have been enumerated elsewhere...
:
: Presumably it only needs to know the next leap-second to do this, not
: the whole known table?

Yes.  ntpd or another agent tells it when leap seconds are coming.  It
doesn't need a table.  Then again, none of the broadcast time services
provide a table...

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Ashley Yakeley <[EMAIL PROTECTED]> writes:
: On Jan 6, 2007, at 08:35, M. Warner Losh wrote:
:
: > So for the foreseeable future,
: > timestamps in OSes will be a count of seconds and a fractional second
: > part.  That's not going to change anytime soon even with faster
: > machines, more memory, etc.  Too many transaction processing
: > applications demand maximum speed.
:
: That's sensible for a simple timestamp, but trying to squeeze in a
: leap-second table probably isn't such a good idea.

Unfortunately, the kernel has to have a notion of time stepping around
a leap-second if it implements ntp.  There's no way around that that
isn't horribly expensive or difficult to code.  The reasons for the
kernel's need to know have been enumerated elsewhere...

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: Warner Losh wrote:
:
: > leap seconds break that rule if one does things in UTC such that
: > the naive math just works
:
: All civil timekeeping, and most precision timekeeping, requires only
: pretty naive math.  Whatever the problem is - or is not - with leap
: seconds, it isn't the arithmetic involved.  Take a look a [EMAIL PROTECTED]
: and other BOINC projects.  Modern computers have firepower to burn in
: fluff like live 3-D screensavers.  POSIX time handling just sucks for
: no good reason.  Other system interfaces successfully implement
: significantly more stringent facilities.

But modern servers and routers don't.  Anything that makes the math
harder (more computationally expensive) can have huge effects on
performance in these areas.  That's because the math is done so often
that any little change causes big headaches.

: Expecting to be able to "naively" subtract timestamps to compute an
: accurate interval reminds me of expecting to be able to naively stuff
: pointers into integer datatypes and have nothing ever go wrong.

Well, the kernel doesn't expect to be able to do that.  Internally,
all the FreeBSD kernel does is time based on a monotonically
increasing second count since boot.  When time is returned, it is
adjusted to the right wall time.  The kernel only worries about leap
seconds when time is incremented, since the ntpd portion in the kernel
needs to return special things during the leap second.  If there were
no leapseconds, then even that computation could be eliminated.  One
might think that one could 'defer' this work to gettimeofday and
friends, but that turns out to not be possible (or at least it is much
more inefficient to do it there).

Since the interface to the kernel is time_t, there's really no chance
for the kernel to do anything smarter with leapseconds.  gettimeofday,
time and clock_gettime all return a time_t in different flavors.

In short, you are taking things out of context and drawing the wrong
conclusion about what is done.  It is these complications, which I've
had to deal with over the past 7 years, that have lead me to the
understanding of the complications.  Espeically the 'non-uniform radix
crap' that's in UTC.  It really does complicate things in a number of
places that you wouldn't think.  To dimissively suggest it is only a
problem when subtracting two numbers to get an interval time is to
completely misunderstand the complications that leapseconds introduce
into systems and the unexpected places where they pop up.  Really, it
is a lot more complicated than just the 'simple' case you've latched
onto.

: A
: strongly typed language might even overload the subtraction of UTC
: typed variables with the correct time-of-day to interval
: calculations.

Kernels aren't written in these languages.  To base one's arugments
about what the right type for time is that is predicated on these
langauges is a non-starter.

: But then, what should one expect the subtraction of
: Earth orientation values to return but some sort of angle, not an
: interval?

These are a specialized thing that kernels don't care about.

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Tony Finch <[EMAIL PROTECTED]> writes:
: On Sat, 6 Jan 2007, M. Warner Losh wrote:
: >
: > OSes usually deal with timestamps all the time for various things.  To
: > find out how much CPU to bill a process, to more mondane things.
: > Having to do all these gymnastics is going to hurt performance.
:
: That's why leap second handling should be done in userland as part of the
: conversion from clock (scalar) time to civil (broken-down) time.

Right.  And that's what makes things hard because the kernel time
clock needs to be monotonic, and leap seconds break that rule if one
does things in UTC such that the naive math just works (aka POSIX
time_t).  Some systems punt on keeping posix time internally, but have
complications for getting leapseconds right for times they return to
userland

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Ashley Yakeley <[EMAIL PROTECTED]> writes:
: On Jan 5, 2007, at 20:14, Rob Seaman wrote:
:
: > An ISO string is really overkill, MJD can fit into
: > an unsigned short for the next few decades
:
: This isn't really a good idea. Most data formats have been moving
: away from the compact towards more verbose, from binary to text to
: XML. There are good reliability and extensibility reasons for this,
: such as avoiding bit-significance order issues and the ability to
: sanity-check it just by looking at it textually.

While all these date formats are mildly interesting, they are far too
inefficient for implementation by an OS.

OSes usually deal with timestamps all the time for various things.  To
find out how much CPU to bill a process, to more mondane things.
Having to do all these gymnastics is going to hurt performance.  One
might scoff at this statement, but research into performance problems
and issues has found time and again timekeeping and timestamps to have
a surprisingly large impact.  So for the foreseeable future,
timestamps in OSes will be a count of seconds and a fractional second
part.  That's not going to change anytime soon even with faster
machines, more memory, etc.  Too many transaction processing
applications demand maximum speed.

: As the author of a library that consumes leap-second tables, my ideal
: format would look something like this: a text file with first line
: for MJD of expiration date, and each subsequent line with the MJD of
: the start of the offset period, a tab, and then the UTC-TAI seconds
: difference.

This sounds like a trivial variation on the NIST format for leapsecond
data.

Warner

Re: A lurker surfaces

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
"Daniel R. Tobias" <[EMAIL PROTECTED]> writes:
: On 2 Jan 2007 at 11:56, Ashley Yakeley wrote:
:
: > GPS is TAI. I'm not proposing abandoning TAI for those applications
: > that need it.
:
: It's a few seconds off from TAI, isn't it?  It was synchronized to
: UTC in 1980 (I think), but without subsequent leap seconds, so it's
: now different from both TAI and UTC.  They probably should just have
: used TAI if they wanted a time scale without leap seconds, rather
: than ending up creating a different one.

Yes.  There's a fixed offset between seconds in TAI and seconds in
GPS.  The GPS timescale is tied to seconds in UTC(NRO).  TAI is a
paper clock, computed after the fact, so GPS can't ever be TAI time.
However, the differences there are down in the nanosecond or so
range.

There's a big philosophical opposition to using the paper clock that
is TAI in a real-time operational timescale that GPS uses.  The
European version of GPS originally specified TAI time, but this was
changed in later revisions to be the same as GPS time.  There's an
extreme reluctance in the time community to call something without
leap seconds "TAI" or "TAI + fixed offset".  TAI means something very
specific.  That's the other problem with just using TAI, btw, but
explaining that point is very hard...

The principle time scientist made me change the description of the
output of measurment time for a product I did.  I described it as "TAI
time seconds since 1970."  Instead, he descriped it as "Number of PPS
ticks in the UTC time scale since 1972 + 63072000 + 10."
Mathematically, they work out to be the same thing, but he was
extremely resistant to calling it TAI time or using the TAI moniker
for it at all.  When I asked him about this, he said that TAI time
isn't realized in real time, but UTC is.  UTC is what one can measure
against.  Producing a number that corresponded to TAI time was OK, and
likely the least confusing thing to do (we give a second number and
UTC time in '-MM-DD HH:MM:SS Z' as well as the channel and the
measurement for that time in out output), but actually calling it TAI
would 'confuse' the really smart time geeks out in the world.  I asked
him for a reference where I could read up on this, and he shrugged and
said he just knew it and didn't know of any good write up.

Warner

Re: Introduction of long term scheduling

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
John Cowan <[EMAIL PROTECTED]> writes:
: Warner Losh scripsit:
:
: > There's no provision for emergency leapseconds.  They just have to be
: > at the end of the month, and annoucned 8 weeks in advance.  IERS has
: > actually exceeded this mandate by announcing them ~24 weeks in advance
: > in recent history.
:
: So much the worse.  That means that if the Earth hiccups on March 7, the
: value of |DUT1| will not return to normal until May 31.

Yes.  But it would take a change in angular momementum would likely
mean that |DUT1| being a little too large would be the least of our
worries.

The earthquake that hit Indonesia last year changed the time of day by
microseconds.  What would cause a sudden jump of hundreds of
milliseconds hurts my brain to contemplate.

Warner

Re: A lurker surfaces

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Ashley Yakeley <[EMAIL PROTECTED]> writes:
: On Jan 2, 2007, at 11:40, Warner Losh wrote:
:
: > The second technical problem is that the length of a second is
: > implicitly encoded in the carrier for many of the longwave time
: > distribution stations.  10MHz is at SI seconds.  For rubber seconds,
: > the broadcast would drift into adjacent bands reserved for other
: > things.
:
: At 1000ns, the carrier would drift by 10Hz. Surely the bandwidth is
: big enough for that?

Likely.  I hasn't done the math.  I didn't know if the rubber seconds
idea was only on leapday, or for the whole year.

: > Also, GPS would have to remain in SI seconds.  The error in GPS time
: > translates directly to an error in position.  Approximately 1m/ns of
: > error (give of take a factor of 3).  Rubber seconds would require that
: > the rubber timescale be off by as much as .5s.  So GPS has to remain
: > in GPS time (UTC w/o leap seconds, basically).  That means that the
: > rubberness of the seconds would need to be broadcast in the
: > datastream.
:
: GPS is TAI. I'm not proposing abandoning TAI for those applications
: that need it.

GPS is also used for UTC today.  Many ntpd's are stratum 1 tied to a
GPS receiver.  ntpd is UTC, by definition.

The gymnastic necessary to produce UTC timing signals from GPS
(non-rubber) timing signals with rubber UTC secods would be, ummm,
very interesting.

Warner

Re: Introduction of long term scheduling

[Warner Losh]

> Warner Losh scripsit:
>
> > There's an exception for IERS to
> > step in two weeks in advance if the earth's rotation rate hickups.
>
> So if I understand this correctly, there could be as many as 14
> consecutive days during which |DUT1| > 0.9s before the emergency leap
> second can be implemented; consequently, the current guarantee is only
> statistical, not absolute.

I think I understand differently.  BIH says on Jan 1 that the
Februrary value of DUT1 is 0.2ms.  If the earth hickups, IERS can step
in by Jan 15th and say, no, the real correct value is 0.3ms.

There's no provision for emergency leapseconds.  They just have to be
at the end of the month, and annoucned 8 weeks in advance.  IERS has
actually exceeded this mandate by announcing them ~24 weeks in advance
in recent history.

The IERS bulletin C is a little different than the ITU TF.460:

>>Leap seconds can be introduced in UTC at the end of the months of  December
>>or June,  depending on the evolution of UT1-TAI. Bulletin C is mailed every
>>six months, either to announce a time step in UTC, or to confirm that there
>>will be no time step at the next possible date.

IERS is issuing Bulletin B as needed.  The latest one can be found at
ftp://hpiers.obspm.fr/iers/bul/buld/bulletind.dat .  Right now DUT1 is
+0.0s until further notice.  From the last few B's, it looks like this
is decreasing at about 300ms per year.  This suggests that the next
leap second will be end of 2008.

Warner

Re: A lurker surfaces

[Warner Losh]

> > Then let's improve the infrastructure for communicating the best
> > estimation of earth orientation parameters.  Then in a world of
> > ubiquitous computing anyone who wants to estimate the current
> > rubber-second-time is free to evaluate the splines or polynomials
> > (or whatever is used) and come up with output devices to display that.
>
> This is fine, but leaves open the question of when 9:00am is here in
> Seattle. And why not transmit rubber-second-time as well, where
> technically feasible (such as over the internet)?

The technical problem is that many timing systems aren't connected to
the internet.  They are at secure installations and they only have GPS
almanac data at their disposal.  And, no, they aren't likely to ever
be on the internet.  Any changes would have to be announced a long
time in advance, and GPS almanac would have to be updated to include
more information.

The second technical problem is that the length of a second is
implicitly encoded in the carrier for many of the longwave time
distribution stations.  10MHz is at SI seconds.  For rubber seconds,
the broadcast would drift into adjacent bands reserved for other
things.

Also, GPS would have to remain in SI seconds.  The error in GPS time
translates directly to an error in position.  Approximately 1m/ns of
error (give of take a factor of 3).  Rubber seconds would require that
the rubber timescale be off by as much as .5s.  So GPS has to remain
in GPS time (UTC w/o leap seconds, basically).  That means that the
rubberness of the seconds would need to be broadcast in the
datastream.

Many GPS receivers put out a PPS.  This needs to be in SI seconds, or
a number of other applications break.  However, if one defined UTC in
terms of these rubber seconds, then the top of UTC second would be out
of phase with this PPS.  That breaks a lot of assumptiosn that rightly
assume that PPS is phase aligned to top of second.

The interval math in UTC that's hard today would be significantly
harder with rubber seconds.  But it is just software, eh?

In short, it is an interestingly naive idea that was tried in the
1960's and failed when there were only dozens of high precision time
users rather than the hundreds of thousands there are today.

The earth does not define the second any more.  At most, it defines
the day and the year.

> What is a good source of earth orientation parameters, btw?

usno publishes several.

Warner

Re: Introduction of long term scheduling

[Warner Losh]

> Still, it's a strange assumption, given that TF.640 allows, I
> understand, leaps at the end of any month.  Unofficially, the
> wording seems to be:
>
> > A positive or negative leap-second should be the last second
> > of a UTC month, but first preference should be given to the end
> > of December and June, and second preference to the end of March
> > and September.
>
> Anybody got access to a proper copy and can say whether that's
> right or not?  If it is right then the Wikipedia article on leap
> seconds needs fixing.

The above is a direct quote from ITU-R-TF.460-4, annex I, section
D.2.1.

Secont D.2.3 is the one that many people here would like to change in
some way, usually the time period:

"The IERS should decide upon and annoucne the introduction of a
leap-second, such an announcement to be made at least eight weeks in
advance."

Which many people would like see extended to a number of years.  Heck,
a simple step would be to announce every 6 months the leap seconds for
a year, but I'll bet even that step would be politically difficult.

Curiously, BIH is currently, at least in the document I have, expected
to predict what the value of DUT1 is to .1 second at least a month in
advance so that frequency standard broadcasts can prepare for changes
of this value a month in advance.  There's an exception for IERS to
step in two weeks in advance if the earth's rotation rate hickups.

Warner

Re: A lurker surfaces

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
[EMAIL PROTECTED] (Michael Sokolov) writes:
: The people who complain about leap seconds screwing up their interval
: time computations are usually told to use TAI.  They retort that they
: need interval time *between civil timestamps*.

Actaully, interval deltas are needed, but it is in TAI time.  There
needs to be, in the systems I've done, a timescale without leapseconds
to keep all the measurements sane.  In addition, many of the systems
I've done have other functions they do also, some of which need to
present UTC to the user.  As much of a pita that leapseconds are
today, having the converion between TAI and UTC (or put another way:
between GPS and UTC or LORAN and UTC) would make these systems
signficantly more difficult to get correct.

There's also times when our systems take in events that are
timestamped using UTC time, so we need to back correlate them to TAI
or whatever internal timescale we're using.  Some of that is because
UTC is the standard for exchanging time, part of that is because the
events in question are measured in whatever timescale is present on an
NTP server.

: To me that seems like
: what they are really measuring as "interval time" is not physical
: interval time, but how much time has elapsed *in civil society*.  Hence
: my idea of civil interval time that's completely decoupled from physical
: time and is instead defined as the turning angle of the clock on the
: Kremlin tower.

Actually, you are wrong.  The intervals is in the number of pps that
have happened, or fractions thereof.  Civil time does intrude because
that's what people use right now to know time fo day.  In the systems
I've done, you need to know both.

The requirements, as you may have noticed, for my needs aren't that
the intervals be done in TAI (we use a variant of LORAN time due to
historical accident, but with a 1970 epoch), but rather that UTC and
these time scales be convertable between one another.

Like I've said a number of times, saying 'just use TAI' isn't viable
because of the conversion issue.  Using TAI (or something with the no
leapsecond regualrity that TAI gives) is necessary for the algorithms
to work.  However, external pressures also require that some things be
done in UTC time and that some of the external sources of data use UTC
time and that needs to be back correlated to the internal timescale
that we're using.

The algorithms, btw, basically integrate frequencies of different
clocks, over time, to predict phase difference.  In this case, you
definitely want to use an interval, and not whatever weirdnesses civil
time happened to do in that interval.  Using an civil time interval
would introduce errors in algorithms.  These algorithms are used to
estimate how well the clocks are doing, but other parts of the system
need to play our UTC for things like NTPD and IRIG.

Warner

Re: A lurker surfaces

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Ashley Yakeley <[EMAIL PROTECTED]> writes:
: Software should serve human needs, not the other
: way around. Anyone needing fixed seconds should use TAI.

I think this idea would be harder to implement than the current
leapseconds.

There are many systems that need to display UTC externally, but need
to operate on a tai-like timescale internally.  Having there being a
sliding delta between them would be a nightmare.

Warner

Re: A lurker surfaces

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: Poul-Henning Kamp wrote:
:
: > Rob, If you feel uncomfortable with calling leapseconds
: > discontinuities, then we can use the term arrhythmia instead.
:
: Which raises the question of why projects requiring an interval time
: scale lacking in such arrhythmias would have selected UTC in the
: first place.

The real world often times requires both kinds of time keeping, with
correlation back and forth between all the timescales.  The customers
want to deal with times in UTC.  To get an effective, high precision
time system, you need to count seconds in a sane way (UTC isn't a sane
way for this purpose) so some conversion between this count and UTC is
needed.

The problem is that most interesting systems have a need to do things
based on UTC time of day, as well as in a time scale that has none of
the complications of the unpredictable UTC implementation we have
today.  So you can't just 'pick one' because real-world systems need
to use different ones for different things, and they also need to
translates events from one timescale to events in the other
timescale.

Let me give you a concrete example.  Let us say we are measuring a
number of atomic clocks against one another.  We get the measurements
and those measurements are in a monotonic timescale that is defined as
PPS's from some epoch (or number of 5MHz or 10MHz ticks from some
arbitrary epoch).  We get data from GPS, which has a similar
timescale.  We compute the clock states and deside that we need to
steer one of the clocks.  These algoritmns are very much elapsed time
sorts of deals.  So we schedule a time to steer it, and then note the
time when we are done steering it.  Since we cannot easily get the GPS
or timer times without interfereing with the operations of those
subsystems (maybe because the steering system and GPS system are on
dufferent cpus), we get the system time when the steer starts and when
the clock tells us the steer is complete (since a steer isn't an
instantaneous event).  Since this system also is an ntpd server, that
system time is in UTC.  Now we have to convert UTC into the internal
timescale so that the algorithms know when the steer happened and can
take that into account for their next round of measurments and
decision making.

To make this all work out well, one has to have perfect leap second
knowledge and all the special case code you have for dealing with the
arrhythmia of a leap second has to work perfectly.  Anything that can
be dobe to make things more perdictable helps out quite a bit...

Warner

Re: A lurker surfaces

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
John Cowan <[EMAIL PROTECTED]> writes:
: Tony Finch scripsit:
: > On Sun, 31 Dec 2006, John Cowan wrote:
: > >
: > > However, it's clear that UTC does not contain the sort of jumps
: > > that LCT does, where a single broken-down time may represent
: > > two different UTC seconds.
: >
: > Not if you include the timezone offset in the representation.
:
: Quite so.  Or alternatively a standard/daylight flag.  The point is,
: people usually don't.

This is the leapsecond problem in a nutshell:  While it is possible to
keep enough information around in the representation of time, most
people opt not to do so.  It complicates the representation of time,
and is one (or more) pieces of information that need to be carried
around, mostly just to handle weird edge cases.  Sure, you can do it,
and some people try.  However, there's no standardized way to do so,
and people re-invent it wrong over and over again.

Another poster I think was right: ntpd (and the underlying OS) hides a
lot of these sins.  Either by just twitterpating at the leapsecond in
some way (time stands still, time jumps back a little, etc), or for a
period of time around the leapsecond (by sticking its fingers in its
ears, singing lalala and changing the size of a second to get through
it a millisecond at a time).

Warner

Re: A lurker surfaces

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: Just a reminder that UTC has no - none - nada - discontinuities.

At the very least, the TAI-UTC difference is discontinuous with jumps
at the leap seconds.  One could easily suggest that 'UTC has
discontinuities' really is a shorthand way of saying this.  One could
also call them irregularities, like the time scale is constipated, as
well.

And the variable radix seems to be to be a crude way to define away
the problem.  You never know, unless you look it up in a table, when
to do the variation in the radix.

: Various computer mis-implementations may, but the standard is very
: carefully constructed to avoid spring-forward or fall-back gaps or do-
: overs.

Well, if you count the variable radix notation as being 'continuous'
then maybe you are right.  However, you never know when the radix is
variable, hence the assertion on many people's part that UTC is
discontinuous.

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
John Cowan <[EMAIL PROTECTED]> writes:
: In this case there are really two questions:  how much it would
: cost to loosen DUT1 but leave it bounded, and how much it would
: cost if it were only statistically, not absolutely, bounded.

We've accepted a statistical solution for the leap-day problem now for
about 500 years.  There are minor variations, and over the next 10's
of thousands of years this simple solution may not prove workable (it
will be off by more than a day on the average in about 8,000 years if
nothing is done).  Right now winter solstice can be off by over a day
from the mean either direction, yet over the long haul things average
out.  There's no call for there to be leap days scheduled at the whim
of the astronomers anymore...  Seems like a logical leap for leap
seconds to follow, if the costs aren't prohibitive.  Chances are that
one person knows all the users of time that still need DUT1 to be less
than 1s.

One big area of change would be the time and frequency stations, like
WWV, since they currently broadcast DUT1, or variations of it.  But
the correction they broadcast is good only to 100ms typically
(time-nuts@ have documented cases where it was worse than 100ms when
some national time lab didn't update the data transmitted when they
were supposed to), so maybe some other means of distribution is
necessary...  And is 100ms really good enough?

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: M. Warner Losh wrote:
:
: > Let's turn the question around.  What would the harm be if |DUT1| were
: > 1.1s?  1.5s?  2.0s?  Contrast this with the harm and difficulty that
: > the current 6 month scheduling window affords.
:
: Indeed.  Go for it.  I look forward to reading your report.  Who and
: what interests are adversely affected in each case?  How are these
: effects mitigated as a function of the limit on DUT1?  Also, contrast
: what benefits accrue.  One would think that the responsibility for
: quantifying the implications of a change to a standard would fall on
: the parties proposing said change.

I know the benefits, but nobody has yet produced a study on why 0.9s
was chosen.  Some vague rumblings about astronomical software needing
to be rewritten, and some vague notions about 'hearty souls' that do
celestial navigation with atomic clocks for high accuracy have been
offered, but who really would be hurt by this change?  Since you are
an astronomer, I thought you might be able to give some insight into
at least one of these users of time.  Daylight savings time and time
zones prove that society at large has a very high tolerance for
variations between the mean solar time at an arbitrary location, maybe
hundreds of miles away, and the local time.  Only specialized users of
time would be affected.  Who are they and how do we find out the cost
of change?

The world has changed from having maybe a few dozen or tens of dozen
atomic clocks when leap seconds started to having GPS with cheap,
disciplined oscillators that number in the hundreds of thousands.
These little devices have obviated the need, in many cases, for
celestial navigation.  Given that change, the cost benefit analysis
that was done in 1972 likely needs updating.

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
John Cowan <[EMAIL PROTECTED]> writes:
: Rob Seaman scripsit:
:
: > >I don't care if you want to implement leap-milliseconds, as long
: > >as you tell me 10 years in advance when they happen.
: >
: > Again - with no intent to minimize the issues - what supports this
: > assertion?  Is there any reason to believe that 10 years advance notice
: > would encourage projects and vendors to do anything other than ignore
: > the requirement entirely?  A statement that 10 years, or 600 years,
: > notice is all that is needed to resolve all the problems, smooth over
: > all the complications, is entirely too glib.
:
: You are confusing the question of fixity (which is what notice is
: about) with granularity.  It's true that probably no one would bother
: to implement the ALHP.  However, if computer technologists were handed a
: list of leap seconds from now until 2015, and told "Implement these," it
: wouldn't matter how many or how few leap seconds there were.  But since
: you astronomers insist on a fixed maximum for |DUT1|, no such table
: can exist.
:
: The proposal is this:  look at the trends, take your best shot at
: working out a leap-year schedule for 10 years in the future, and then
: live with it.

Let's turn the question around.  What would the harm be if |DUT1| were
1.1s?  1.5s?  2.0s?  Contrast this with the harm and difficulty that
the current 6 month scheduling window affords.

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
"Poul-Henning Kamp" <[EMAIL PROTECTED]> writes:
: We can get that only by increasing the DUT tolerance.

Yes.  Letting DUT be bounded by +- 10s rather than +- 0.9s is going to
affect a tiny number of users.  Having leapseconds only known 6 months
in advance affects a lot more users...

: As Warner, I and others have repeatedly emphasized:  It is not the
: step size that is the problem, it is the 6 month warning.
:
: I don't care if you want to implement leap-milliseconds, as long
: as you tell me 10 years in advance when they happen.

While my preference would be to never see another leap second, I know
that's likely not an option.

For many practical reasons, scheduling them out 10 years would help
ameliorate the costs of leap seconds and allow for better long term
planning.

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: John Cowan wrote:
:
: > I assume you mean 23-hour or 25-hour LCT days?  True.  It does work
: > against UCT days, though, since they are uniformly 1440 minutes long.
:
: Not should leap hours replace leap seconds.

And avoiding the ugly 61 or 59 second minutes to define away the
problem...

Warner

Re: Design - a Tufte decision

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: On Dec 27, 2006, at 12:02 AM, M. Warner Losh wrote:
:
: > Calculating time intervals for times 6+ months in the future can be
: > the least of one's worries when one tries to code up a library to deal
: > gracefully with these different failure modes.  The trivial case where
: > one has perfect knowledge of TAI-UTC and one can keep that knowledge
: > current is very simple in comparison.  Dealing with this case is very
: > simple, and is the way most people think about leap seconds.  But
: > dealing with the edge cases can be difficult because there are so
: > many, and so many that people forget to test or conceive of until the
: > call from the field comes in with a failure...
:
: A lot of these "edge" cases are really firmly centered in issues of
: real-time programming.  Few versions of Unix are equipped to deal
: with real-time issues in even a rudimentary fashion.  In any event,
: these cases have very little to do with leap seconds or any other
: aspects of the representation of time quantities.

Actually, they can have a great deal to do with it.  The absolute need
to deal properly with these edge cases can, at times, lead to design
decisions that aren't the 'simplest'.  I also disagree that Unix is
ill-equipted to deal with real-time issues.  While one does need to
take care, I have successfully deployed over a dozen different types
of timing systems over the past 7 years on FreeBSD.  My familiarity
with these edge cases, the customers that expect them to work and the
different sources of time has been learned through these experiences.
The nature of these customers also means that they are less well
connected to networks than one would like, which also complicates
matters.

The salient point from my ramblings is that different time scales may
become available at different points in time because data about them
is available at different points in time to the application/os.  If
there's only one time scale, then once you know the time, you are
done.  If there's more than one, then you need to either discover both
times, or you need to discover one time and the transforms to that
time to know the others.

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Ashley Yakeley <[EMAIL PROTECTED]> writes:
: On Dec 27, 2006, at 06:29, Poul-Henning Kamp wrote:
:
: > That's a pretty bad format.  Computers are binary and having
: > pseudo-decimal fields like tv_usec in timeval, tv_nsec in timespec
: > and picoseconds in Haskell is both inefficient and stupid.
: >
: > The fractional part should be a binary field, so that the width
: > can be adjusted to whatever precision and wordsize is relevant.
:
: It's impossible to accurately represent a millisecond using binary
: fractions. That would be unacceptable for most sub-second use.

Ummm, it is possible to represent sub-picoseconds accurately using
binary fractions.  what are you talking about?

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Tony Finch <[EMAIL PROTECTED]> writes:
: > The nature of the uncertainty is very different.  The uncertainty of
: > future UTC can be managed, but for timezones the only sane path is to
: > eschew their use entirely.
:
: That isn't possible for applications like appointment books and job
: schedulers. As Warner suggests, you need to calculate future times
: provisionally, and in a way that insulates you from discontinuities.
: For example, "same time tomorrow" instead of "in 86400 seconds".

And 'same time tomorrow' might not exist depending on the timescale
that's used.  Cron runs into this problem (and has solved it) because
it runs in localtime which has hours repeated or omitted twice a
year.  Leap seconds are handled in an ad-hoc way as well, typically
the last second of the day is repeated for a positive leap-second so
cron jobs on fast machines that run at midnight UTC don't run twice.

There is a more subtle point that I should mention explicitly.  The
folks that say "do everything internally as TAI time" tend to forget
that to do that one must restrict one's self in a number of important
ways.  Because certain things are almost always in UTC time (IRIG
timecode, measurment of atomic clocks for BIPM UTC calculations, etc),
one must take care to know that when you convert a time into TAI from
UTC that you can then make the inverse computation and arrive back at
the same time.  One must also know the appropriate TAI-UTC offset for
that time.  Typically, one doesn't schedule things in these kinds of
systems too far in the future, and one's historical data processing is
handled offline where perfect historical knowledge of leap is
available for dealing with long or abnormal intervals.

Of course, needing to know TAI-UTC offsets leads one to interesting
situations.  What does one do if one has TAI time, but not UTC and a
conversion is asked for?  Is it right to pause that thread until it is
available, or is an exception right?  Pausing the thread means that
less code has to deal with this ugliness and makes it so the
programmer doesn't have to know that time1 - time2 might fail
randomly.  However, pausing can lead to deadlocks if the thread that
acquires this knowledge pauses.  Throws in an exception avoids the
deadlock case, but makes every single time expression prone to having
to deal with exceptions, which can be tedious and time consuming to
code every single time.  In C++ this can be really bad because
operators are often overloaded for time libraries to make the time
types look like native.

There are a number of sources of phase (rtc clock, UTC time, TAI time,
GPS time, NTP servers) and a number of sources of TAI-UTC (internet
ftp, GPS almanac, Loran Data Channel, local data file).  The arrival
of the data from these sources may be asynchronous, and the cached
value may be too old to be useful (for spares that have been sitting
on the shelf for a few years).  The data sources might be unavailable
because the device is cold starting, or because the antenna cable is
damanged.  These different scenarios have different "right" things to
do.  The data can also expire because that GPS antenna has been
damanged for too long and you don't know the next leap second is
coming...

Calculating time intervals for times 6+ months in the future can be
the least of one's worries when one tries to code up a library to deal
gracefully with these different failure modes.  The trivial case where
one has perfect knowledge of TAI-UTC and one can keep that knowledge
current is very simple in comparison.  Dealing with this case is very
simple, and is the way most people think about leap seconds.  But
dealing with the edge cases can be difficult because there are so
many, and so many that people forget to test or conceive of until the
call from the field comes in with a failure...

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
John Cowan <[EMAIL PROTECTED]> writes:
: M. Warner Losh scripsit:
:
: > If I was to compute the number of seconds between Jan 1, 2007 0:0:0 and
: > Dec 31, 2008 23:59:50, the answer is 63071990 or 63071991 or 63071992.
: > We have no way of knowing today how many seconds are in that interval.
: > We do know the answer is one of the above.
:
: Technically 63071999 and 63071998 are also possibilities, though
: I admit they are unlikely.

you are correct...  this simple stuff sure is hard...

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Zefram <[EMAIL PROTECTED]> writes:
: M. Warner Losh wrote:
: >Actually, the real answer is domain specific.  There are many things
: >that don't really care (when do I need to make the next 20 house
: >payments, off by one second just doesn't matter at all since
: >transcations are batched on a daily basis a few days early).
:
: In that case you don't really want a TAI<->UTC conversion.  You want
: an estimate of TAI<->UT1.  Possibly you'd prefer an exact TAI<->UTC
: conversion but want to fall back on the planetary rotation estimate in
: the event that UTC isn't defined that far yet.  The point is that you're
: asking a different question from the one that elicits "I don't know".

Well, if I compute an interval, using UTC times, then I can go back
several decates before the results start to be suspect.  Going
forward, you can get up to almost 12 months error free at some times
of the year, and a little over 6 months error free (assuming
instantaneous distribution of circular C).  But going out a year
you'll have at least one leap second opportunity.  Going out 2 years
gives you 3, etc.  This allows one to compute these long intervals
with only a small error.  If I was to compute the number of seconds
between Jan 1, 2007 0:0:0 and Dec 31, 2008 23:59:50, the answer is
63071990 or 63071991 or 63071992.  We have no way of knowing today how
many seconds are in that interval.  We do know the answer is one of
the above.  The error in not knowing is tiny, about a 6 parts in 10^7.
Large enough to matter for some things, not large enough to matter for
others.

There's been little work on having variable precision math of this
nature where the computation depends on the numbers involved and a set
of rules.  The precision work for floating point does come closest,
because there has been a lot of effort paid to analyzing the different
sources of error, but there's little work that's been done on
computing the error in parallel to the actual computation, wrapping
that up into convenient data structures and allowing the programmer to
test the results for an acceptle error (or rather in this case
uncertainty).

In all the high precision time work I've done, the biggest, most
overriding problem has been: What's the leap second value for the
immediate past (< 1 day) or near future (< 1 month).  Very little
matters outside of those parameters.  UTC time is important for IRIG
(and other time codes), UI display, and atomic clock coorinated
measurements (more generally, any coordinated event), but little
else.  Some time scale that one can use to pace the system, or to make
measurements against is needed, but TAI or TAI-like timescales will
fit the bill.  The hard part is when you need to convert from one to
the other of the time scales and a realization of the limits to doing
that.

So the answer to my above question isn't "I don't know" but rather
63071991 +- 1 given the current leap second practices.  It would take
a lot longer than a year to notice different behavior in rotations and
change the rules, but factoring that in would be a crap shoot at best.

The important thing to realize also in constructing time libraries is
that I can have a TAI time in the future that doesn't have a
corresponding UTC time now, but will have one the closer we get to it.
And vice versa.  I can have a UTC time (say 23:59:50 Dec 31, 2008)
that has no TAI time associated with it yet, but will at some point in
the future before that time.

: >Writing a library that copes with both types of users can be
: >problematical at best...
:
: I think a different library is required.  One is about atomic timekeeping,
: the other is about astronomy.

Well, one is about atomic timekeeping.  The other is the civil
approximation of astronomy, since a purely astronomical answer depends
on exactly where you are, seasonal variations, etc.

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Zefram <[EMAIL PROTECTED]> writes:
: Steve Allen wrote:
: > and if I continue that practice
: >I can later give you an estimate of how wrong I was when I told you.
:
: This is something that's missing from current chronological APIs.
: It needs to be formalised.

time_t is so totally broken, it isn't funny.  That's the closest thing
to a standardized API there is for time.  All others are stuff folks
have done here or there, but they aren't universal enough to be
considered.

Too bad the problems with time_t are well known, well discussed and
well enumerated.  Or rather I should say "too bad POSIX doesn't care
enough to change it" since the cost of changing time_t is huge...

Also, things in TAI time don't care either. 1 day, 1 year, 100 years
don't matter to TAI.  Periodic things that happen at a given time of
day (UTC) are the only things that do care.

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Zefram <[EMAIL PROTECTED]> writes:
: Keith Winstein wrote:
: >  what should a library do when a program asks to
: >convert between UTC and TAI for some instant further than six months in
: >the future?
:
: Refuse, of course.  The correct answer is "I don't know".  You might
: know if asked later.

Actually, the real answer is domain specific.  There are many things
that don't really care (when do I need to make the next 20 house
payments, off by one second just doesn't matter at all since
transcations are batched on a daily basis a few days early).  There
are other things that do care (when do I fire the lasers at the
target).  For those things, however, it is rare to be computing time
into the future that far.

Writing a library that copes with both types of users can be
problematical at best...

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Keith Winstein <[EMAIL PROTECTED]> writes:
: The GNU C library includes a leap-second table in its timezone/leapseconds
: file in the source package. This gets compiled in to the "right" (that is,
: non-POSIX) zoneinfo files in, e.g., /usr/share/zoneinfo/right/... .
:
: If your users have an up-to-date GNU libc, they can get the value of
: TAI-UTC at any point since 1970 using C library functions. Here's a sample
: C program that shows how to do it.

The problem with this approach is that it fails to work for long
running programs that need to know about leapseconds.  The zonefiles
aren't re-read when they change, and even if they were, updating the
zonefiles to keep them up to date presents some interesting
challanges.

Warner

Re: Mechanism to provide tai-utc.dat locally

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Ashley Yakeley <[EMAIL PROTECTED]> writes:
: Hello, I just joined the leap seconds list. I wrote the "time"
: package for the Haskell programming language.
: 
:
: I include code for making conversions between TAI and UTC, given a
: leap-second table. I also include code for parsing a tai-utc.dat file
: into a leap-second table. I do not, however, simply include a leap-
: second table as any program compiled with one would be out of date
: after six months.
:
: This has led me to consider run-time methods of obtaining leap-second
: information, and how that might be standardised for use by software
: authors. For instance, I imagine a software package that established
: a well-known place in the directory hierarchy to find tai-utc.dat and
: perhaps other "earth data" files, and was responsible for keeping
: them up to date. Other software could then make use of the package
: without each having to implement their own mechanism.
:
: Does this strike people as worthwhile? Does anyone know of an
: existing effort along these lines?

One can find the leap data from NIST that's up to date:
ftp://time.nist.org/leap-seconds.list
I believe that there are other places for this information as well.

The problem is that often times one has systems that aren't guarnateed
to be connected to the internet, but are connected to GPS so that this
number can be obtained.  Other problems arise from systems that are
intermittently operational (like in sparing situations) that may be
off over many leap seconds...

Warner

Re: Equitable estoppel

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
John E Hein <[EMAIL PROTECTED]> writes:
: Peter Vince wrote at 20:15 + on Dec 18, 2006:
:  > >For the moment, if leap seconds is to be abbandoned, I would favour the 
leap
:  > >minute instead.
:  >
:  > Is that not sitting on the fence, and ending up with the worst of
:  > both worlds? It is neither as precise as leap (micro, milli, or
:  > whole) seconds, nor as long term as leap hours. It would put off the
:  > updates for only a few tens of years, by which time experience of
:  > them would have been lost, and our descendants would end up with Y2K
:  > type problems.
:  >
:  > Peter Vince
:
: One of the problems described on this list in the past is that of leap
: tables in devices designed to operate over 10-20 years without
: availability of leap second updates.  Being able to rely on a table
: that is good for that long makes the job of the designer much easier.

As would a schedule for the next 10 years, say something like:

Dec 2007
Jun 2009
Jun 2010
Dec 2012
Dec 2013
Jun 2015
Jun 2016
Dec 2018
...

The state of the art is such that we have a high confidence of being
able to predict things out at least 18 if not 36 months and still
keeping within the 0.9s performance goal.  If that were loosened a
little, then we could schedule things out 10 years or so.  It could
even be phased in so that the next two leap second opportunities could
be scheduled after the first of the year, and the two following that
after June, etc until.  While annoying, at least it would be known far
in advance and start to feel more like a leap day rather than a total
pita.

: In this respect, leap minutes would be "better" than leap seconds.

Then a leap hour would be 60 times better[*]  Of course, a leap minute
would happen once or twice a lifetime.  A leap hour happens once every
100 generations or so...

: Do not mistake this comment as an advocation of leap minutes.

Ditto for me...

Warner

[*] And 3600 times more disruptive and less likely to get right...

Re: Equitable estoppel

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Steve Allen <[EMAIL PROTECTED]> writes:
: ITU rules which say their documents must be sold, not given away.  All

A simple web search shows that at least the UTC/leap second
recommendations are readily available in electronic form...

Warner

Re: what time is it, legally?

[M. Warner Losh]

I view the same data differently.  I see it as a progression:

Local Solar time -> mean local solar time ->
  timezone as mean local time at one point used for many -> UTC -> ???

Clearly, we're moving away from solar time and towards something else.
Our ability to tell time has exceeded the earth's ability to be
perfectly periodic.  The time will come when we need to change
something as the current system is doomed to failure, sooner or
later.  I feel that attempts to tie time to the sun and to also have a
precice notion of a second are fundamentally at odds with one another
and the current system tries very hard to paper over the differences.
No doubt they can be papered over for years to come, but the basic
physics of platentary rotation with a large satellite stand against
it.  I'm unsure what that "something else" should be, and I'm not sure
that we're in a good position to know with certainty what would be a
good solution.  I just know that invoking variable radix counting
schemes has its limitations and difficulties...

Anyway, I know others view it differently, but if it was easy and
obvious there wouldn't be such a divergence of opinion, eh?

Warner

Re: independence day

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
"Poul-Henning Kamp" <[EMAIL PROTECTED]> writes:
: In message <[EMAIL PROTECTED]>, Steve Allen writes:
:
: >In the middle of May some text about legal time in the US was
: >introduced into a US Senate bill regarding funding for NSF and NIST.
: >See section 508 of S.2802 introduced 2006-05-15, e.g.
: >http://thomas.loc.gov/cgi-bin/query/z?c109:S.2802:
:
: `(b) COORDINATED UNIVERSAL TIME DEFINED- In this section,
: the term `Coordinated Universal Time' means the time scale
: maintained through the General Conference of Weights and
: Measures and interpreted or modified for the United States
: by the Secretary of Commerce.'.
:
: That could sound like the drilling of a loophole :-)

As has been pointed out in the past, the Secretary of Commerce has had
the ability to define mean solar time to mean UTC (or something else,
if they felt the urge)...  I think this is just another attempt to
keep their options open, like they have now...

Warner

Re: building consensus

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: This does not express a complete algorithm, of course, since it is
: not tied to the days of the week.  One presume the zero point of 1875
: was selected as a practical compromise with historical realities.
: One questions whether the Gregorian calendar can be used to express
: dates before 1582-10-15, and it is ambiguous for large parts of the
: world until the mid-eighteenth century.  Still in 1875, it had not
: reached all locales.  Etc.

This was answered earlier in the thread.  One can project the
Gregorian calendar back in time from the above date and use that for
dating (there's a special name for when you do this that can be found
in the archives).  However, most historians use Julian dates back from
there (with the same backward projected dates when time is before the
Julian calendar was established).

Warner

Re: building consensus

[Warner Losh]

> Warner Losh wrote:
> >A rule implies that it is long term, I guess.  Maybe there's a better
> >word for that implication.
>
> In the realm of calendars the terminology is "arithmetic" versus
> "observational".  That's one of the things I included at the start of
> this thread.  I'd also like to throw in the word "deterministic".

I missed that terminology, and I like it a lot better than the
terminology I've been using.  Thank you!  UTC is an observationally
based time scale...  I like how that sounds...

> There is the alternate point of view that the calendar in actual civil use
> in a particular locality, changing between different arithmetic calendars
> at different times, constitutes an unpredictable observational calendar.
> Perhaps we need a concept of "calendar zone" analogous to time zone,
> with a calendar zone database to match.

The Theory file in the current time zone files has some very
interesting information about this.  There's a 'Calendrical Issues'
section that talks about these issues.  There's a reference to a book
http://emr.cs.uiuc.edu/home/reingold/calendar-book/index.shtml
which is good.  The problem is that many authoritative sources on
these matters often disagree what happened.  I'm sure that someone has
taken this as the basis for starting a more comprehensive database.

Here's a few of my favorite entries:

>> In 1700, Denmark made the transition from Julian to Gregorian.  Sweden
>> decided to *start* a transition in 1700 as well, but rather than have one of
>> those unsightly calendar gaps :-), they simply decreed that the next leap
>> year after 1696 would be in 1744 -- putting the whole country on a calendar
>> different from both Julian and Gregorian for a period of 40 years.
>>
>> However, in 1704 something went wrong and the plan was not carried through;
>> they did, after all, have a leap year that year.  And one in 1708.  In 1712
>> they gave it up and went back to Julian, putting 30 days in February that
>> year!...

>> Russia
>>
>> From Chris Carrier <[EMAIL PROTECTED]> (1996-12-02):
>> On 1929-10-01 the Soviet Union instituted an ``Eternal Calendar''
>> with 30-day months plus 5 holidays, with a 5-day week.
>> On 1931-12-01 it changed to a 6-day week; in 1934 it reverted to the
>> Gregorian calendar while retaining the 6-day week; on 1940-06-27 it
>> reverted to the 7-day week.  With the 6-day week the usual days
>> off were the 6th, 12th, 18th, 24th and 30th of the month.
>>If your source is correct, how come documents between 1929 -- 1940 were
>>still dated using the conventional, Gregorian calendar?
>>I can post a scan of a document dated December 1, 1934, signed by
>>Yenukidze, the secretary, on behalf of Kalinin, the President of the
>>Executive Committee of the Supreme Soviet, if you like.

Re: building consensus

[Warner Losh]

From: Rob Seaman <[EMAIL PROTECTED]>
Subject: Re: [LEAPSECS] building consensus
Date: Mon, 5 Jun 2006 08:35:39 -0700

> On Jun 4, 2006, at 9:57 PM, M. Warner Losh wrote:
>
> > leap days have a rule, while leap seconds are scheduled.
>
> A schedule and a rule are the same thing, just regarded from
> different historical perspectives.

Leap days have an iron-clad rule that generates the schedule on which
they happen.  Leap seconds have a committee that generates the
schedule on which they happen.

A rule implies that it is long term, I guess.  Maybe there's a better
word for that implication.

> The "leap day rule" will most
> certainly have to accommodate scheduling changes over the millennia.

True.

> On the other hand, I am sure we haven't exhaustively discussed
> possible refinements to the leap second "scheduling algorithm".  (And
> ain't that a rule?)  If we have to spend all our time fending off
> this silly leap hour proposal, we'll never have the opportunity to
> focus on rules and algorithms (not to mention technology and
> infrastructure).

We have discussed having some kind of rule for when leap seconds are
inserted.  So far, none of these 'rules' are that long term.  "follow
this table for the next 10 years" or "for the next 10 years, we'll
have one every 18 months" are simplified versions of the proposals
I've seen.  These have a very limited time horizon.

> The biggest difference between leap days and leap seconds is that
> days are quantized.

I'm afraid I don't understand this statement.  Care to explain?

Warner

Re: building consensus

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
John Cowan <[EMAIL PROTECTED]> writes:
: M. Warner Losh scripsit:
:
: > : The designers of Posix time thought it was more important to preserve
: > : the property that dividing the difference between two time_t values
: > : by 60, 3600, 86400 would give minutes, hours, days.
: >
: > That's the one property that Posix time_t does not have.  The
: > difference between time_t's that cross a leap second are off by one
: > second, and therefore do not start with the right answer to do the
: > division...
:
: I expressed myself badly.  My point is that if you have a Posix time_t
: representing 11:22:33 UTC on a certain day, and you add 86400 to that
: time_t, you will get the Posix representation of 11:22:33 UTC on the
: following day, whether a leap second intervenes or not.  This is a valuable
: property, many existing programs depended on it, and the authors of the
: Posix spec preserved it at the expense of having a distinct representation
: for each UTC second.

Yes.  To find a second absolute time that represents the same wall
time a minute/hour/day later is why posix time_t has this property.
This is abstime + delta, which is a little different than the
difference between two time_t.

: You may call this position wrong (and I have done so), but it is
: unquestionably defensible.

Differences in time_t are adjusted by leapseconds.  This makes naive
math work for the ABSTIME+delta case, but breaks the difference case
when you want an actual elapased time.  It is an engineering tradeoff,
but an inconvenient one for the problem domains that I tend to have to
solve.

Warner

Re: building consensus

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
John Cowan <[EMAIL PROTECTED]> writes:
: Zefram scripsit:
:
: > >If this means that leap secounds and leap days are analogous, then I
: > >suppose so.  If it means something else, I don't understand it.
: >
: > That's what I meant.  Can you suggest a clearer wording?
:
: "Leap seconds (after 1972) are closely analogous to leap days."

"Leap seconds (after 1972) are closely analogous to irregularlly
 happening leap days."

might be better, since leap days have a rule, while leap seconds are
scheduled.

: > Being ambiguous between adjacent seconds seems inherently faulty to me.
:
: The designers of Posix time thought it was more important to preserve
: the property that dividing the difference between two time_t values
: by 60, 3600, 86400 would give minutes, hours, days.

That's the one property that Posix time_t does not have.  The
difference between time_t's that cross a leap second are off by one
second, and therefore do not start with the right answer to do the
division...

POSIX time_t definition effecitvely omits leap seconds from the time
scale (by playing them twice, or pretending the second happened), the
difference between two time_t's always gives a duration adjusted for
the leap seconds that happen, rather than the actual duration.

: > Are you thinking of linear counts such as POSIX time, where the
: > representation is ambiguous?  I was implicitly excluding those, on the
: > grounds that they don't count as a "representation".  It's also not
: > "linear".
:
: No, it isn't.  But that doesn't mean you *can't* construct a numerical
: representation of UTC time: say, the number of UTC seconds since
: 1972-01-01T00:00:00Z.

It would be better to say the number of SI seconds since 1972 rather
than UTC seconds, I think.  I use a timescale like this at work to
ensure that duration calcuations over leap secondsd are correct.

Warner

Re: building consensus

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: Warner Losh objects:
:
: >> There are several doughty people here who happen to have that
: >> opinion, but they abide with us mortals outside the time lords'
: >> hushed inner sanctum.
: >
: > I have spent much time explaining why leap seconds cause real
: > problems in real applications, only to be insulted like this.
:
: Sincere apologies for my awkward statement.  Dictionary.com defines
: "doughty" as "marked by stouthearted courage; brave".  I wasn't
: questioning the knowledge or passion of folks holding views that
: differ from my own.  Rather I was attempting to question whether
: anybody actively participating on this list - holding whatever view -
: is also participating in ITU discussions.
:
: I see that Mr. Cowan has also parsed my admittedly opaque remarks.

Yes.  I'm sorry I was so easily offended.  Please accept my appologies
for my hasty words.

Warner

Re: building consensus

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
John Cowan <[EMAIL PROTECTED]> writes:
: M. Warner Losh scripsit:
:
: > In message: <[EMAIL PROTECTED]>
: > Rob Seaman <[EMAIL PROTECTED]> writes:
: > : Actually, this list is not a "discussion" per se.  If we simplify the
: > : positions - just for the sake of argument here - to "leap second yes"
: > : and "leap second no", the reality is that the folks pushing the "leap
: > : second no" position have never engaged with this list.  There are
: > : several doughty people here who happen to have that opinion, but they
: > : abide with us mortals outside the time lords' hushed inner sanctum.
: >
: > What an amaizingly unhelpful and offsensive statement.  I have spent
: > much time explaining why leap seconds cause real problems in real
: > applications, only to be insulted like this.
:
: I believe you have misread Rob's remark, though I concede that it was
: easy to misread.  I believe Rob meant that the people who are pushing
: "leap seconds no" in *official* channels are not to be found on this list.
: That being so, the "leap seconds yes" folks are unable to challenge them
: or persuade them otherwise.
:
: You and I, on the other hand, fall into the "doughty people here" group.

Maybe I did misread them.  I've been sick the past three days, so
maybe we can chalk it up to that and I'll offer my oppologies for
having such a thin skin.

Wanrer

Re: building consensus

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: Actually, this list is not a "discussion" per se.  If we simplify the
: positions - just for the sake of argument here - to "leap second yes"
: and "leap second no", the reality is that the folks pushing the "leap
: second no" position have never engaged with this list.  There are
: several doughty people here who happen to have that opinion, but they
: abide with us mortals outside the time lords' hushed inner sanctum.

What an amaizingly unhelpful and offsensive statement.  I have spent
much time explaining why leap seconds cause real problems in real
applications, only to be insulted like this.

Warner

Re: ideas for new UTC rules

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: Simply allow the IERS to announce any number of leap seconds
: in advance extending over any time horizon - and yes - occurring
: at the end of any month.  If predictability is the goal, relaxing
: unnecessary constraints is the solution.

There's a lot of timing gear that will fail if leap second is inserted
not at the end of June or December, not least ntp.

Warner

Re: Ambiguous NTP timestamps near leap second

[Warner Losh]

From: Rob Seaman <[EMAIL PROTECTED]>
Subject: Re: [LEAPSECS] Ambiguous NTP timestamps near leap second
Date: Thu, 16 Feb 2006 16:30:37 -0700

> On Feb 16, 2006, at 2:06 PM, Markus Kuhn wrote:
>
> > While there is a 24:00:00, there is certainly *no*
> > 24:00:00.0001.
> > That would be 00:00:00.0001 instead.
>
> Says who?  Didn't we just burn a lot of calories discussing whether
> UTC was a real number or a continuous function?  Time does not end at
> midnight, rather, a time without a date or a date without a time are
> incomplete.  Are we to believe that there is something evil about
> expressing negative time values, too?  Just depends on the epoch.
>
> Put it in angle notation.  12 hours UT = 180 degrees = pi radians,
> but also = -180 degrees or -pi radians.  So what?
>
> Would think the usual tactics apply:  interpret rules as loosely as
> possible on input and as strictly as possible on output.

UTC rules state that the time sequence should be

23:59:59.75
23:59:60.0
23:59:60.25
23:59:60.50
23:59:60.75
00:00:00.00
00:00:00.25

The problem is for time exchanges that cross the 0:00:00.00 boundary.
Inventing a new notation that is not described in the ITU UTC
definition is not helpful, even if lots of other people use it
informally.  It will create confusion because it has no precise
definition in the UTC standard (note: NTP specifically states UTC, and
no other standard).

A better wording for the paragraph would be:

Time exchanges initiated prior to or during a positive leap
second must complete before the end of that leap second, or be
discarded.

Or

Time exchanges may not span the end of a positive leap
second.  If an implementation detects a time exchange that
would span this time, it must discard it.

Warner

Re: 24:00 versus 00:00

[Warner Losh]

From: Markus Kuhn <[EMAIL PROTECTED]>
Subject: Re: 24:00 versus 00:00
Date: Thu, 16 Feb 2006 19:53:22 +

> Steve Allen wrote on 2006-02-16 19:25 UTC:
> > >   No reply from an NTP server shall ever represent any point in time
> > >   between 23:59:60 and 24:00:00 of a UTC day.
> >
> > Minor point, I think it has to read more like this
> >
> > between 23:59:60 of a UTC day that ends with a positive leap
> > second and 00:00:00 of the subsequent UTC day.
>
> I disagree.
>
> With the 24-h notation, it is a very useful and well-established

But that's not what UTC defines.  It doesn't matter what the well
established traditions are.

Warner

Re: Ambiguous NTP timestamps near leap second

[Warner Losh]

> "M. Warner Losh" wrote on 2006-02-14 21:18 UTC:
> > ntp time stampes are ambiguous at leap seconds (the leap indicator
> > bits aren't part of the time stamp, exactly), which is a good reason
> > to change them.  However, the cost to change them could be quite high
> > if done in an incompatible manner.
>
> No, this ambiguity alone is surely not a good enough reason to change.
> It can be fixed trivially in a backwards compatible way, without
> introducing TAI or leap-second status bits. Simply add to the NTP
> specification the paragraph:

The leap status bits are already in the time echange packet, but not
part of the timestamps themselves.  You need the whole packet to know
which second the timestamps are from.  The problem comes when the
timestamps strattle a leap event, or when the timestamps are
propigated to parts of the code taht don't have the leap status
bits...

>   No reply from an NTP server shall ever represent any point in time
>   between 23:59:60 and 24:00:00 of a UTC day. If a client requests an
>   NTP timestamp and the response would represent a point in time during
>   an inserted leap second, then this request shall be ignored.

The problem isn't the inserted leap second, since the status bits are
the same for it as for the previous second.  The problem is the
following second.  However, the problem could be kludged around in
that way.  It isn't a perfect solution, since it opens up the
possibility of a reduced amount of data (TAI wouldn't suffer from
that).  However, it is a good enough solution in that this loss of
data is typically non-critical and the timing window is small.

> Rationale: NTP runs over UDP, and therefore must be (and is perfectly)
> able to cope with occasionally dropped packets. Dropping NTP server
> reply packets for one second every few years will hardly affect any
> application, because NTP clients already implement robust retry
> and filtering mechanism today.
>
> NTP is actually one of the simple cases, where no harm is done by simply
> going offline very briefly near a leap second. In other applications,
> e.g. streaming media, things are not that easy, and other workarounds
> are needed if one wants to remain backwards compatible with existing UTC
> practice while minimizing the chance of leap-second induced
> malfunctions. UTC-SLS is what I'd recommend for most of these cases (but
> not for the NTP protocol itself).

But won't stretching time in one part of the system result in
anomalous resutls in these multi-media applications?  They play out
their data in TAI seconds not UTC-SLS due to hardware oscillators that
pace the output of the data.  Long, large slews in system time
evolution would cause the system time to elapse at a different rate
than the media is played out, even when it isn't playing out over the
leap second.  Real multi-media applications will still need to run in
UTC time so that the external timing requirements would still be met
if this media were destined for down-stream consumers (eg, TV stations
couldn't use UTC-SLS since they have to broadcast at a fixed field
rate that can't be stretched or shrunk by a part per thousand).

That's the problem with UTC-SLS.  It is good enough for the wrist
watch crowd, but there are many applications that it is ill suited for
where one must use TAI or UTC, and many of these applications may be
co-located on the same box.

Warner

Re: An immodest proposal

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Neal McBurnett <[EMAIL PROTECTED]> writes:
: > UTC time stamps in NTP are ambiguous.  TAI ones are not.  UTC time
: > stamps do not convey enough information to properly implement things
: > like intervals, while TAI ones do.  The NTPNG stuff that I've seen
: > appears to consider these problems as worthy of needing a solution and
: > they plan on solving them.  It isn't rocket science, but one has to
: > divorce ones self from the chauvinistic view that UTC is always best.
: > For time exchange, it is not the best, and has many problems around
: > the edges.
:
: Yes it is messy, but the tradeoffs are complex.  I don't think the
: latest drafts specify NTP timestamps.  I suspect they still rely on
: the leap second bit to disambiguate the timestamp on a leap second,
: but I haven't checked recently.  They are all linked to from the
: charter page I noted above.

But ntp time stamps to not have a leap second bit.  To be precice, it
is a leap indicator that's in the packet.  This makes it impossible to
represent times that straddle the leap event, since there are 4
different time stamps involved...  I guess in those cases, the packets
should be discarded and tried again later...

But you are right.  I did misremeber that it used TAI time.  The
drafts still specify time in UTC.  I could have sworn I saw something
that said otherwise, but I can't seem to find it now.

: > Doing NTP with TAI (and the implied requirement for DTAI) doesn't
: > change what time is displayed for users.  It does make it *MUCH*
: > easier to get leap seconds right for those users that need them.
: > Anything else is madness.  UTC is a display convention, not a sane[*]
: > counting convention.
:
: I think that changing to a different over-the-wire timestamp epoch
: would just add to the confusion, not make things simpler in practice.

TAI is the same epoch as UTC.  It is just not adjusted for leap
events.  However, your point that the second numbering would be
different and might cause confusion is still valid.

: People still need to know UTC, and transmitting the leap second table
: info, especially via autokey, is much more complex than the basic
: protocol.  But at least this is a standards process conducted in the
: open, where you can just get involved directly if you have something
: to add.

Autokey is interesting, but it is way too complex for something as
simple as leap second propigation

Warner

Re: An immodest proposal

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Rob Seaman <[EMAIL PROTECTED]> writes:
: On Feb 14, 2006, at 12:50 PM, Markus Kuhn wrote:
:
: > You can, of course, define, publish, implement, and promote a new
: > version (4?) of NTP that can also diseminate TAI, EOPs, leap-second
: > tables, and other good things. I'm all for it.
:
: But why are you for it?  Before investing large amounts of time and
: money in developing and deploying a large new timekeeping system,
: wouldn't one want to invest smaller amounts in exploring the issues
: and options?  Heck - one has to imagine that a number of successful
: grant applications are lurking around here somewhere.  Time is an
: issue that cuts across every funding agency out there.

UTC time stamps in NTP are ambiguous.  TAI ones are not.  UTC time
stamps do not convey enough information to properly implement things
like intervals, while TAI ones do.  The NTPNG stuff that I've seen
appears to consider these problems as worthy of needing a solution and
they plan on solving them.  It isn't rocket science, but one has to
divorce ones self from the chauvinistic view that UTC is always best.
For time exchange, it is not the best, and has many problems around
the edges.

Doing NTP with TAI (and the implied requirement for DTAI) doesn't
change what time is displayed for users.  It does make it *MUCH*
easier to get leap seconds right for those users that need them.
Anything else is madness.  UTC is a display convention, not a sane[*]
counting convention.

Warner

[*] All variable radix counting conventions are insane by (my humble)
definition :-).

Re: An immodest proposal

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Neal McBurnett <[EMAIL PROTECTED]> writes:
: On Mon, Feb 06, 2006 at 10:37:37AM -0700, Rob Seaman wrote:
: >1)  TAI can be recovered from UTC given a table of DTAI.
: >2)  NTP can convey TAI as simply as UTC.
: >3)  Deploy a small network of NTP servers to keep TAI, not UTC.
: >4)  NTP client machines could therefore trivially select between TAI and
: >UTC by subscribing to different servers.
: >5)  This would provide an unbiased experimental sandbox for civil
: >timekeeping issues.
:
: Since TAI can be recovered from UTC, and hosts that use NTP know UTC,
: those hosts can serve any time they want to clients on that host.  I
: see little reason to change the particular timestamps that are used in
: the NTP protocol, an less reason to have two different timestamps
: within NTP, or to require users on ntp-using hosts to switch their
: ntp servers just to get a different timescale.

ntp time stampes are ambiguous at leap seconds (the leap indicator
bits aren't part of the time stamp, exactly), which is a good reason
to change them.  However, the cost to change them could be quite high
if done in an incompatible manner.

: That's why I support the "Fixing POSIX time" proposal from
: Poul-Henning Kamp.  This can be implemented with the latest NTP
: daemons.  The operating systems would seem to just need the right sort
: of CLOCK_NTP definition for /usr/include/linux/time.h or equivalent,
: and the appropriate conversion routines as he outlined.

Agreed.

: I'd like to see a standard for NTP that includes the TAI information
: that is now passed in non-standard formats, but that is a separable
: issue.

I believe that the new draft standard of ntp uses TAI time for its
time exchange, and explicitly includes DTAI information in a standard
format.  Since the protocol version changes, there's no ambiguity and
that's a good time to change things like this.  However, I've not read
the latest ntpng drafts to confirm this.

Warner

Re: Comparing Time Scales

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Tim Shepard <[EMAIL PROTECTED]> writes:
: But there's a difference between NTP timestamps, and the details of
: the implementation of a system which uses NTP for synchronization.

Ah, I was getting the two confused and didn't quite realize it until
your clarification.  Thanks!

Warner

Re: Comparing Time Scales

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
    M. Warner Losh <[EMAIL PROTECTED]> writes:
: In message: <[EMAIL PROTECTED]>
: James Maynard <[EMAIL PROTECTED]> writes:
: : M. Warner Losh wrote:
: : > In message: <[EMAIL PROTECTED]>
: : > James Maynard <[EMAIL PROTECTED]> writes:
: : > : Thanks, guys, for your feedback. Here's another iteration.
: : > :
: : > : The numbering of NTP seconds in the vicinity of a leap second seems to
: : > : differ from one document to another.  Here I follow the NTP (version 3)
: : > : specification, RFC 1305, in which the Leap Indicator (sys.leap,
: : > : peer.leap, pkt.leap) is 01 if a positive leap second is to occur at the
: : > : end of the current UTC day, and 00 if no leap second is pending.
: : > :
: : > : UTC = 2005-12-31 23:59:59, NTP seconds = 3 345 062 399, LS_pending = 01
: : > : UTC = 2005-12-31 23:59:60, NTP seconds = 3 345 062 400, LS_pending = 01
: : > : UTC = 2006-01-01 00:00:00, NTP seconds = 3 345 062 400, LS_pending = 00
: : >
: : >
: : >
: : In the NTP time scale, it is the last second of the UTC day, NTP second
: : number 3 124 137 600, that is repeated. The first instance of this NTP
: : second number is the leap second, during which (TAI - UTC) = 31 and the
: : NTP Leap Indicator is 01.  The second instance of this NTP second number
: : is the the first second of the new UTC day, during which (TAI - UTC) =
: : 32 and the NTP leap indicator is 00.  You can distinguish the leap
: : second (while it is occuring) by the fact that it is the last second of
: : the UTC day (time of UTC day = 23:59:60) and that the NTP Leap Indicator
: : is 01. You can distinguish the first second of the new UTC day by the
: : fact that, of the two seconds with the same NTP second number, it is the
: : one that has the NTP leap indicator set back to 00.
: :
: : Note that according to the write-up to which you referred me, it is
: : NTP second number 3,124,137,600 (not 3,124,137,599) that is repeated.
: : NTP second number 3,124,137,599 is the *next-to-last* second of the day,
: : not the *last* second of the day.)
:
: No.  The article specifically says that after it the system time gets
: to ,600, it is decremented by one and there is specific code in the
: code that returns the system time to applications that makes it stand
: still.  The second isn't *NOT* repeated.  Repeat: The second is *NOT*
: repeated in what they said.  Time stands still outside of the kernel,
: while inside the kernel the last second of the day *IS* repeated,
: hence the need for the limiter that the article talks about.
:
: Not all kernels keep time standing nearly still during the leap second
: (since that has other bad effects).  Some expose this decrement to the
: users.  The highlighed part of what I quoted said exactly this.
:
: I've actually implemented this for FreeBSD.  You are arguing theory,
: and I'm arguing the fine points of an actual, real implementation.

More specifically, read the text I highlighted:

: : >>>The obvious question raised by this scenario is what happens during
: : >>>the leap second itself when NTP time stops and the clock remains
: : >>>unchanged. If the precision time kernel modifications have been
: : >>>implemented, the kernel includes a state machine that implements the
: : >>>actions required by the scenario. [[ At the first occurrence of second
-
: : >>>3,124,137,600, the system clock is stepped backward one
   ---
: : >>>second.]] However, the routine that actually reads the clock is
   ---
: : >>>constrained never to step backwards, unless the step is significantly
: : >>>larger than one second, which might occur due to explicit operator
: : >>>direction. In this design time stands still during the leap second,
: : >>>but is correct commencing with the next second.

Notice the [[ ]] text.  When it becomes 400, the system time is
decremented internally, which is why I wrote the 399.5 example that I
did.  Also, the picture on the page shows how time stops/slows way
down during the leap second in Dave Mills' view of how time should be
returned to the user during the leap second.  Text I didn't quote says
that the system time must be monotonically increasing, so it is
incremented by 1ns each call.

So internally to the kernel, the time does repeat the last second.
Externally, in a Mills' perfect world, time stands still during the
leap second.  The second doesn't repeat, per se, but it freezes for a
second.

In FreeBSD, the actual internal time is exposed to the user.  If we
froze time as returned from the system, then threaded applications'
threads would potentially stop d

Re: Comparing Time Scales

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
James Maynard <[EMAIL PROTECTED]> writes:
: M. Warner Losh wrote:
: > In message: <[EMAIL PROTECTED]>
: > James Maynard <[EMAIL PROTECTED]> writes:
: > : Thanks, guys, for your feedback. Here's another iteration.
: > :
: > : The numbering of NTP seconds in the vicinity of a leap second seems to
: > : differ from one document to another.  Here I follow the NTP (version 3)
: > : specification, RFC 1305, in which the Leap Indicator (sys.leap,
: > : peer.leap, pkt.leap) is 01 if a positive leap second is to occur at the
: > : end of the current UTC day, and 00 if no leap second is pending.
: > :
: > : UTC = 2005-12-31 23:59:59, NTP seconds = 3 345 062 399, LS_pending = 01
: > : UTC = 2005-12-31 23:59:60, NTP seconds = 3 345 062 400, LS_pending = 01
: > : UTC = 2006-01-01 00:00:00, NTP seconds = 3 345 062 400, LS_pending = 00
: >
: >
: >
: >>>The obvious question raised by this scenario is what happens during
: >>>the leap second itself when NTP time stops and the clock remains
: >>>unchanged. If the precision time kernel modifications have been
: >>>implemented, the kernel includes a state machine that implements the
: >>>actions required by the scenario. [[ At the first occurrence of second
: >>>3,124,137,600, the system clock is stepped backward one
: >>>second.]] However, the routine that actually reads the clock is
: >>>constrained never to step backwards, unless the step is significantly
: >>>larger than one second, which might occur due to explicit operator
: >>>direction. In this design time stands still during the leap second,
: >>>but is correct commencing with the next second.
: >
: >
: > So internally, once we hit the leap second time, we step backwards.
: > Some systems contrain the values returned from gettimeofday to not
: > notice the step, while others do not.  This talks about incrementing
: > the time by 1 unit (usually a nanosecond or microsecond) while time is
: > catching up.
: >
: > http://www.eecis.udel.edu/~mills/leap.html
: >
: > So it very much depends on if you are looking inside the kernel or
: > outside the kernel for what the correct values are.  If you freeze
: > time, then doing a time exchange during a leap second is going to give
: > bogus results.  If you step it back, then it will give good results,
: > but there are other bad effects.
: >
: > Basically, no matter what you do, it is broken (unless you tick in TAI
: > time).
: >
: > Warner
: >
: >
: >
: Thank you. You are trying hard to educate me on this matter, and I
: appreciate your efforts.
:
: Following the link you gave me in this message, <>, I found a discussion
: of how the leap second is to be handled in the NTP time scale. That
: discussion includes the following table to illustrate the leap second
: transition of 1998-12-31 23:59:60 UTC:
:
: +---+--++--+---+
: |   Date|   Time   | TAI Offset | NTP Leap |  NTP Seconds  |
: +---+--++--+---+
: | 31 Dec 98 | 23:59:59 | 31 |01| 3,124,137,599 |
: +---+--++--+---+
: |   | 23:59:60 | 31 |01| 3,124,137,600 |
: +---+--++--+---+
: |  1 Jan 99 | 00:00:00 | 32 |00| 3,124,137,600 |
: +---+--++--+---+
: |   | 00:00:01 | 32 |00| 3,124,137,601 |
: +---+--++--+---+
:
: Funny, that's exactly what I wrote in my last iteration of my table!
:
: In the NTP time scale, it is the last second of the UTC day, NTP second
: number 3 124 137 600, that is repeated. The first instance of this NTP
: second number is the leap second, during which (TAI - UTC) = 31 and the
: NTP Leap Indicator is 01.  The second instance of this NTP second number
: is the the first second of the new UTC day, during which (TAI - UTC) =
: 32 and the NTP leap indicator is 00.  You can distinguish the leap
: second (while it is occuring) by the fact that it is the last second of
: the UTC day (time of UTC day = 23:59:60) and that the NTP Leap Indicator
: is 01. You can distinguish the first second of the new UTC day by the
: fact that, of the two seconds with the same NTP second number, it is the
: one that has the NTP leap indicator set back to 00.
:
: Note that according to the write-up to which you referred me, it is
: NTP second number 3,124,137,600 (not 3,124,137,599) that is repeated.
: NTP second number 3,124,137,599 is the *next-to-last* second of the day,
: not the *last* second of the day.)

No.  The article specifically says that aft

Re: Comparing Time Scales

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
James Maynard <[EMAIL PROTECTED]> writes:
: Thanks, guys, for your feedback. Here's another iteration.
:
: The numbering of NTP seconds in the vicinity of a leap second seems to
: differ from one document to another.  Here I follow the NTP (version 3)
: specification, RFC 1305, in which the Leap Indicator (sys.leap,
: peer.leap, pkt.leap) is 01 if a positive leap second is to occur at the
: end of the current UTC day, and 00 if no leap second is pending.
:
: UTC = 2005-12-31 23:59:59, NTP seconds = 3 345 062 399, LS_pending = 01
: UTC = 2005-12-31 23:59:60, NTP seconds = 3 345 062 400, LS_pending = 01
: UTC = 2006-01-01 00:00:00, NTP seconds = 3 345 062 400, LS_pending = 00


>> The obvious question raised by this scenario is what happens during
>> the leap second itself when NTP time stops and the clock remains
>> unchanged. If the precision time kernel modifications have been
>> implemented, the kernel includes a state machine that implements the
>> actions required by the scenario. [[ At the first occurrence of second
>> 3,124,137,600, the system clock is stepped backward one
>> second.]] However, the routine that actually reads the clock is
>> constrained never to step backwards, unless the step is significantly
>> larger than one second, which might occur due to explicit operator
>> direction. In this design time stands still during the leap second,
>> but is correct commencing with the next second.

So internally, once we hit the leap second time, we step backwards.
Some systems contrain the values returned from gettimeofday to not
notice the step, while others do not.  This talks about incrementing
the time by 1 unit (usually a nanosecond or microsecond) while time is
catching up.

http://www.eecis.udel.edu/~mills/leap.html

So it very much depends on if you are looking inside the kernel or
outside the kernel for what the correct values are.  If you freeze
time, then doing a time exchange during a leap second is going to give
bogus results.  If you step it back, then it will give good results,
but there are other bad effects.

Basically, no matter what you do, it is broken (unless you tick in TAI
time).

Warner

Re: Comparing Time Scales

[Warner Losh]

From: "Daniel R. Tobias" <[EMAIL PROTECTED]>
Subject: Re: [LEAPSECS] Comparing Time Scales
Date: Fri, 03 Feb 2006 21:27:54 -0500

> On 3 Feb 2006 at 15:37, James Maynard wrote:
>
> > Thanks, guys, for your feedback. Here's another iteration.
>
> It would seem that PTP is actually counting from 1970-01-01 00:00:00
> in TAI rather than UTC; the subsequent values of that scale reflect
> the full TAI-UTC offset, not just the part of it that occurred after
> 1970.  For instance, the PTP value for 1972-01-01 00:00:00 UTC
> consists of the number of seconds in the last two (non-leap) years
> plus 10 seconds.  The actual UTC value at 1970-01-01 reflected the
> variable-length "rubber seconds" of the time, so the TAI-UTC offset
> then was some fractional value that was less than 10 but greater than
> 0.

My company produces measurements that are tagged with UTC time and TAI
time.  The UTC time is the broken down time, while the TAI is the
number of seconds since 1972 plus a contant.  This gives the same
effect as the number of PPS pulses since 1970 without having to worry
about rubber seconds between 1970 and 1972 when just under 2 seconds
accumulated in the TAI-UTC offset.  It works out so that if you take a
time today, and add the TAI-UTC offset (or is that subtract), you get
a number that you can convert using the POSIX routines into a UTC time
for all seconds execpt leap seconds.

Warner

Re: Comparing Time Scales

[Warner Losh]

From: James Maynard <[EMAIL PROTECTED]>
Subject: Re: [LEAPSECS] Comparing Time Scales
Date: Fri, 03 Feb 2006 17:31:46 -0800

> Warner Losh wrote:
> > From: James Maynard <[EMAIL PROTECTED]>
> > Subject: Re: [LEAPSECS]Comparing Time Scales
> > Date: Fri, 03 Feb 2006 15:37:40 -0800
> >
> >
> >>Thanks, guys, for your feedback. Here's another iteration.
> >>
> >>The numbering of NTP seconds in the vicinity of a leap second seems to
> >>differ from one document to another.  Here I follow the NTP (version 3)
> >>specification, RFC 1305, in which the Leap Indicator (sys.leap,
> >>peer.leap, pkt.leap) is 01 if a positive leap second is to occur at the
> >>end of the current UTC day, and 00 if no leap second is pending.
> >>
> >>UTC = 2005-12-31 23:59:59, NTP seconds = 3 345 062 399, LS_pending = 01
> >>UTC = 2005-12-31 23:59:60, NTP seconds = 3 345 062 400, LS_pending = 01
> >>UTC = 2006-01-01 00:00:00, NTP seconds = 3 345 062 400, LS_pending = 00
> >
> >
> > If you read different documents carefully, you'll see this sequence:
> >
> >  UTC = 2005-12-31 23:59:60.0, NTP seconds = 3 345 062 400.0, LS_pending = 01
> >  UTC = 2005-12-31 23:59:60.5, NTP seconds = 3 345 062 399.5, LS_pending = 01
> >  UTC = 2006-01-01 00:00:00.0, NTP seconds = 3 345 062 400.0, LS_pending = 00
> >
> > Where the 399 second repeats.  The documents say that just after time
> > is incremented to 400, the last second of the day is repeated
> >
> > Warner
> >
>
> This looks like something that needs to be resolved in a standards
> committee, such as David Mills' working group on Version 4 of the NTP
> protocol.  Meanwhile, you have to cope with different implementations
> that do it different ways!

I know that various ntp implementations behave exactly as I describe.
They repeat the last second of the day.  I believe this is the correct
behavior, and have made sure that FreeBSD behaves this way.

> If a device has leap second information available to it at all, it
> should know at least an hour ahead of time that a positive leap second
> is pending.  Given that information, it should be able to arm itself for
> the upcoming leap second. Once in "armed for leap second state," each
> NTP server and client would count the one-second epochs in the last
> 61-second minute of the day, assigning NTP second numbers "correctly"
> and displaying (if it has a display) the UTC time correctly. (Of course,
> I think the method shown in my table should be the standard way of
> assigning NTP second numbers. But that would be a matter for the
> standards committee to resolve.)
>
> So I *think* my table shows how the leap second *should* be handled on
> the various time scales listed in the table. There remains the standards
> effort -- electropolitical engineering -- of reaching agreement on an
> unambiguous standard to describe how leap seconds should be handled.

The problem is that your results give a second that's in the wrong
day.  The leap second is added to the end of the day, not the
beginning of the day.  You don't repeat the first second of the day
twice.  That poses real problems for things like cron that might start
things twice at midnight.

Also, most ntp implemenations in the host have more than just
pending/not-pending.  They have a stat that's 'leap second executing'
which is part of the standard ntp_adjtime info returned.

Warner

Re: Comparing Time Scales

[Warner Losh]

From: James Maynard <[EMAIL PROTECTED]>
Subject: Re: [LEAPSECS]Comparing Time Scales
Date: Fri, 03 Feb 2006 15:37:40 -0800

> Thanks, guys, for your feedback. Here's another iteration.
>
> The numbering of NTP seconds in the vicinity of a leap second seems to
> differ from one document to another.  Here I follow the NTP (version 3)
> specification, RFC 1305, in which the Leap Indicator (sys.leap,
> peer.leap, pkt.leap) is 01 if a positive leap second is to occur at the
> end of the current UTC day, and 00 if no leap second is pending.
>
> UTC = 2005-12-31 23:59:59, NTP seconds = 3 345 062 399, LS_pending = 01
> UTC = 2005-12-31 23:59:60, NTP seconds = 3 345 062 400, LS_pending = 01
> UTC = 2006-01-01 00:00:00, NTP seconds = 3 345 062 400, LS_pending = 00

If you read different documents carefully, you'll see this sequence:

 UTC = 2005-12-31 23:59:60.0, NTP seconds = 3 345 062 400.0, LS_pending = 01
 UTC = 2005-12-31 23:59:60.5, NTP seconds = 3 345 062 399.5, LS_pending = 01
 UTC = 2006-01-01 00:00:00.0, NTP seconds = 3 345 062 400.0, LS_pending = 00

Where the 399 second repeats.  The documents say that just after time
is incremented to 400, the last second of the day is repeated

Warner

Re: The nature of risk

[Warner Losh]

> First risk - do all airlines and shipping lines automatically hew to
> NT for all purposes?  Or will UT time signals persist for some some
> subsystems of some planes and ships from some countries under some
> circumstances?  A lack of imagination of how this might occur is no
> protection - only an inventory similar to, but perhaps even more
> invasive than, Y2K will suffice.

I'll point out here that Loran-C broadcasts signals based on a 1958
epoch that have no leap seconds.  Loran time has diverged from UTC by
23 seconds, since it followed the 'rubber seconds' until 1972, but
used pure atomic time since then that when leap seconds were
introduced.  LORAN-C, for those that don't know, is the terrestial
nagivation system used these days as a backup to GPS.

Remember, navigation depends on the solar time only when celestial
sightings are used.  Otherwise, it depends on whatever beacons or
signals that are used.

> Second risk - a later change to fundamental assumptions made during
> the design of a complicated system often reveals contingent errors in
> components that depend on those assumptions.  We have the issue of
> DUT1 exceeding 0.9s, of course, but other algorithmic assumptions may
> have been made.  For instance, use of UTC for calculating intervals
> requires a table of leap seconds.  We're given to understand that
> this is subject to error.  Those errors are as likely to be revealed
> by the absence of leap seconds as by their continued presence.
> Simply introducing interval time doesn't guarantee that interval
> calculations are bug free.

Given the actual, real implementation experiences of those in the
group that have deal with iterval time, we can be pretty sure that the
number of interval calculation errors will decrease.

> Third risk - UT or GMT permit the use of simple closed form
> algorithms for converting between local and standard, mean and
> apparent time (for example).  This is precisely the area of
> remediation that will cause astronomers to incur significant costs -
> we would have to add DUT1 corrections to our many assorted systems.
> Navigation applications face similar remediation - even if only for
> backup systems in the case of a GPS outage.  Will all airlines, all
> shipping companies, all air and seaports, all national and
> international air and sea traffic control systems, all communications
> channels carry out this remediation - and carry it out perfectly?
> Won't the DUT1 term be added in some instances where it should have
> been subtracted?  Such an error might only be revealed years later
> when DUT1 passes some threshold.  The phrase "ticking time bomb"
> comes to mind.

In the case of a GPS outage, LORAN-C is used.  It is specifically
still maintained, at least in the US, as a backup in case something
bad happened to GPS system (most likely the ground station).  There's
a new data channel for LORAN-C which just went operational that
propigates backup information as well.  If you need to go to a
tertiary backup, assuming that the new Galileo system also goes out,
you're unlikely to have the necessary charts at hand anyway.  They are
bulky and take up a lot of space, relatively speaking.  And they tend
to be specific for a given location (which is what makes them bulky).

> Fourth risk - sabotage, or simply stale data structures.  NT will
> often need to be corrected using DUT1.  Values for DUT1 change and
> must be maintained in some tabular data structure.  That table must
> be loaded with some cadence using some strategy - perhaps the values
> are downloaded when needed from the internet - perhaps they are
> preloaded into firmware.  In either case, the values may be incorrect
> due to either intentional or unintentional mischief.  The behavior of
> two otherwise similar systems may diverge simply because their data
> structures or strategies for updating these differ.

This is no different than the leap second tables today.  If you don't
know the number of leap seconds, you can't recover time from GPS or
LORAN-C signals.  However, you do not need to know DUT1 values to get
highly accurate results from both of these navigation systems.

> Ignore all of the many other misadventures that could result from
> clock errors - do I really need to spell out the possibility of
> midair or sea collisions from any of these risks?  A primary, well
> tested, system goes out for some obscure radio beacon, for instance.
> The secondary system has a bug whose genesis is risk 1, 2, 3 or 4.  A
> plane from one airline uses that beacon - another airline's aircraft
> is flying a parallel track in the opposite direction (as I am assured
> they do to high precision) but is guided by a different navigation
> system without the same bug.  Their intended paths never cross, their
> actual paths do.  Or an oil tanker runs aground, or an owner piloted
> sailboat misses landfall in the middle of the Pacific, or...

Yes.  There are risks.

> Even the most well-conceived and carefully 

Re: Accommodating both camps

[Warner Losh]

From: Michael Deckers <[EMAIL PROTECTED]>
Subject: Re: [LEAPSECS] Accommodating both camps
Date: Wed, 25 Jan 2006 16:50:59 +

>Poul-Henning Kamp wrote on 2006-01-25:
>
> >  If we abandon leapseconds today to avoid getting computer problems,
> >  we still have several hundred years of time to decide how to
> >  deal with any long term effects.
>
>I do not think so. When civil time is no longer connected to solar
>time (which is more than abandoning leap seconds!),

I don't understand your statement here.  Can you elaberate?

>then this affects
>legal and cultural issues all over the world, and a difference of a
>minute or so may well be relevant. Such a difference can accrue
>in 60 or 80 years.

I have a hard time believing that 1 minute would be significant from a
cultural point of view, when 1 hour of daylight saving time isn't
significant at all.  I have a hard time believing that 1 minute would
make a difference when the deviation from mean solar time is +- 1/2
hour (or more) in most time zones.  Since 1960 (call it 45 years),
we've had a divergence of 33 seconds between TAI time and UTC time, so
the 60-80 year time frame seems reasonable.

This is the biggest misunderstanding that I have with people that make
a big deal out of no longer being on solar time.  We already have a
huge tolerance for the difference between local solar time and legal
time, given the 15 degree time zones.  I have yet to meet anybody in
Denver that cares that our solar noon is off a bit (being at ~105
degrees west, it is pretty close), or that there appears to be a
seasonal variation on the order of 10 minutes in the observed time of
solar noon, and the mean solar noon time at the extremes of the year.
In addition, while Denver is on Mountain Time, most of Mexico, which
is directly south of Denver, is on Central Time, having seemly chosen
to be an hour off of solar time.

See http://www.maa.mhn.de/Scholar/times.html for information on solar
time vs mean solar time.  See
http://webexhibits.org/daylightsaving/ref/CurranTaylor.html for some
background om time zones and daylight saving time.

>I am not just thinking of lawyers trying to exploit the difference
>between legal time in Britain (which is GMT) and the time scale
>disseminated by the NPL. What I find much more disquieting is first,
>that some problems may become apparent only very late, when the difference
>between TI and UT is large enough, and, secondly, that it appears to
>be very difficult to say which areas of daily life will be affected.

We already have a > 30 second difference between TAI and UTC time.

>On the other hand, I believe that most if not all of these problems
>will not be life-threatening -- whereas many of the problems with
>leap seconds and computers may well be. So this, rather than other
>technical merit may finally decide the question. But again, giving up
>leap seconds in UTC is not the same as accepting atomic time as civil
>time.

Again, I don't quite understand this statement.  Can you elaberate a
bit on the difference?

Warner

Re: Accommodating both camps

[Warner Losh]

> It seems clear that we have two camps, or schools of thought, on this
> mailing list:
>
> 1) Those who favour retaining the status quo ante, in which civil time
> is based on UTC and the standard time and frequency stations broadcast
> UTC; and
>
> 2) Those who find it difficult to cope with UTC's leap seconds mechanism
> and which to abolish leap seconds. These people propose abolishing leap
> seconds, thereby causing the offset from UTC (or its renamed
> replacement, TI) to be constant.
>
> I belong to the former camp. I want UTC, leap seconds and all, to
> continue to be broadcast on the standard time and frequency stations,
> and to continue to be used as the reference from which the various civil
> time zones are offset.
>
> I wonder, though, whether those in the other camp would find it
> acceptable to have the standard time and frequency stations not only
> broadcast UTC and DUT1 (= UT1 - UTC, to 0.1 s resolution), but also to
> broadcast DTAI (= TAI - UTC, 1 s resolution)?
>
> DTAI changes infrequently, so it could be broadcast at a rather low data
> rate. Currently all the various one-bit-per-second digital time codes
> from the various standard time and frequency stations have at least one
> or two unassigned (reserved) bits. So one way to broadcast DTAI would to
> be to change that bit or bits once per minute, to broadcast DTAI at a
> one-bit-per-minute rate.
>
> This would provide a backward-compatible way to accommodate all users.

This might be necessary, but it isn't sufficient and wouldn't
accomodate all users.  It would be a good start.

Here's why I'm in camp #2:

(1) Nobody counts time in the variable radix that UTC counts time in.
Instead, they rely on 'naive math' and count time based on the
number of seconds since some epoch.  As such, leap seconds
represent an irregularity in this scheme.  This is the POSIX
time_t problem, and cannot be solved by simply using TAI (posix
time_t is explicitly UTC).

(2) Broadcasting the DTAI as you call it is insufficient.  Many
systems aren't connected to such broadcasts (IRIG doesn't provide
it, ntp doesn't really provide it (although that can change),
etc).  In addition, even those systems that are attached cannot
run in TAI until they know the current value.  In the case of
spares, they can be powered off for months at a time.  This
reduces the performance at startup.  GPS receivers have this same
problem: if they have been off a while, they don't know the leap
second count so can't recover UTC until the alminac is broadcast.
Waiting several minutes for the number of leap seconds to be
broadcast is not something that users of time systems like.  See
#5 below, as a longer lead time for leap seconds would help this
problem.

(3) There appears to be no provision for broadcasting the history of
leap seconds in your proposal.  Systems that deal with time need
this information to correctly process times in the past, or to
correctly compute intervals.  While the errors from not having it
are small, they can be meaningful to some applications.

(4) The variable radix of utc makes it hard to convert to TAI without
a full leap second table (in the general case), although 'nowish'
times can be computed.

(5) There's no way to know when the next leap second will happen,
unless you are connected to a source of this information.  People
well connected to sources of time get this information in enough
time to do something sane.  People connected via IRIG get at most
1 minute of warning of an impending leap second (which means that
you can't feed a NTP server from irig w/o a backchannel for leap
second information).  Leap seconds are announced so close to the
event that this presents issues for those sites that wish to have
spares.  Matters can be complicated by limited internet
connectivity for some installations.  This presents operational
problems for those wishing to deploy systems with a life of
multiple years.

(6) Bullitin C, in a machine parsable form along with history, should
be available via the radio broadcasts and the internet in a well
known place that the time keeping authorities publicly and
frequently acknowledge as being canonical.  One can find sources
for this information on the net, but it is hard to know of the
long-term commitment to keeping these files in those exact URLs.

(7) Leap seconds are hard to implement.  I can tell you from direct,
first-hand implementation experience that even good, smart
programmers get something wrong.  It is hard to test real leap
second events in many systems (unless you have a GPS simulator),
so many of the details are gotten wrong despite programmer's best
efforts.  The biggest reason for this that I've seen is that many
of the data streams are vaguely documented, or the documentation
assumes that the programmer has deep kno

Re: the tail wags the dog

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Steve Allen <[EMAIL PROTECTED]> writes:
: The legal time of the US is (in many more words) GMT.
: The officials who are charged by congress with the task of providing
: time provide UTC.

The legal time in the US is the mean solar time at a given meridian,
as determined by the secretary of commerce (the actual law is a little
more verbose than this, but this is an accurate boil down) plus some
weird options for 'border states' which timezone they are in.  This is
why NIST provides UTC and leap seconds happen on the UTC schedule
rather than some other schedule that would produce the same results.
It is also why there are leap seconds and not the old-style frequency
adjustments + tiny steps.  Both of these schemes fit the law, as it is
rather vague in the words it uses in a legal sense (the term mean
solar time isn't legally defined, but does have an accepted scientific
meaning).  Other schemes could also fit the law that aren't UTC today
since there's no what we would call 'DUT1 tolerance' written into the
law...

Warner

Re: wikipedia "Leap Seconds" collaboration

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Neal McBurnett <[EMAIL PROTECTED]> writes:
: > Rob Seaman wrote:
: > >>I hope we can all continue this discussion in a more positive manner.
:
: It is the nature of email lists to be good at stimulating discussion,
: and bad at generating clear resolutions.  Thus was the FAQ born.  But
: we have a more modern technology than FAQs, the wiki, which can more
: effectively funnel passionate energy from groups of people with
: diverging ideas into coherent descriptions of a variety of viewpoints,
: suitable for enlightening the world.  Imperfectly, to be sure, but
: better than a mail list
:
: I think the thing we need to do is build on what clarity we can find
: in the moment, and document it at wikipedia.  If the folks discussing
: the "Jesus" article can arrive at a relatively stable set of positions
: (and last time I looked, they had done remarkably well, considering),
: surely we can also.
:
: Note the relatively successful policy of presenting things from a
: "Neutral Pointof View" (NPOV):
:
:   http://en.wikipedia.org/wiki/Wikipedia:Neutral_point_of_view
:
: So would folks be willing to collaborate at
:
:  http://en.wikipedia.org/wiki/Leap_second
:
: and related pages?

I've contributed to the unix time over leap seconds pages, and would
be happy to help.  I feel I can write a good argument for both sides,
even though I have my preferences.

Warner

Re: Risks of change to UTC

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Mark Calabretta <[EMAIL PROTECTED]> writes:
: On Sat 2006/01/21 10:11:04 PDT, "M. Warner Losh" wrote
: in a message to: LEAPSECS@ROM.USNO.NAVY.MIL
:
: >You really should read the archives of this list.  We've been over
: >this in great detail.  TAI is specifically contraindicated as a time
:
: I don't think new contributors (or even old ones) should have to read
: the mountain of email, with its many irrelevant diversions, that has
: accumulated over the last 6 years - somewhat over 9MB by my count.
:
: Instead, I (re-)suggest that you (someone) write a position paper, or
: at least a FAQ, summarising your points.

I agree.  I was unreasonable in my demands.  You are quite right.

Warner

Re: the tail wags the dog

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
[EMAIL PROTECTED] (Michael Sokolov) writes:
: Steve Allen <[EMAIL PROTECTED]> wrote:
:
: > The CGPM recommendation on the timescale everyone should use says UTC.
: >
: > UTC(insert your national time service here) is available in real time.
: >
: > TAI is the mathematical (really the political or diplomatic) entity
: > upon which UTC is ostensibly based, but the practical and legal
: > reality is the other way around.
:
: Has it occurred to any of you that *THIS* is the very root of the problem,
: that *THIS* is what we need to change?

I believe so.

: Also, isn't TAI readily available in real time from GPS?  (How difficult
: can it be for the routine parsing the data stream from the GPS receiver
: to add 19?)  OK, OK, it'll be TAI(GPS) rather than "true TAI", but then
: your UTC is really UTC(NIST) or UTC(USNO) or UTC(PTB) or whatever rather
: than "true UTC".

You can get an excellent approximation of TAI from GPS, assuming that
you use a sane receiver...  Some firmware doesn't deal well with
reporting the raw, uncorrected GPS time, but many do it well.
Depending on your needs, you may have to put the GPS receiver into a
mode that only reports URC...  Well, you get the idea.  It can be done
to a degree of accuracy that most people will accept, but the
post-processed time scale calculations are a little better than what
happens in real time.

Warner

Re: NOT A cruel fraud!

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
James Maynard <[EMAIL PROTECTED]> writes:
: Mr. Losh and I have apologized to each other, off list.  I think we
: should now retire the "cruel fraud" subject line.

Yes.  I was too hard on Mr. Maynard and I appologize for my rough
treatment of him here.  It was really over the line, and I'm sorry for
my ill considered words.

Warner

Re: NOT A cruel fraud!

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Peter Bunclark <[EMAIL PROTECTED]> writes:
: On Sun, 22 Jan 2006, M. Warner Losh wrote:
: > The short answer is that you cannot get a time feed of TAI, so the
: >
: So isn't this one of the things we want to fix in the brave new world of
: joined-up timekeeping? Distribute (very close to) TAI, keep the kernel
: PLLs sweet, move leap second handling to user-space and thus make
: debugging very easy, then everone can get their timescale of choice as
: a f(TAI)?

Yes.  That's a fair summary.

Warner

Re: NOT A cruel fraud!

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Ed Davies <[EMAIL PROTECTED]> writes:
: Earlier, I wrote:
: > We all know that it (and any other world-wide timescale) is
: > "postal" at the level of the time it takes light to cross a
: > moderately small room but for microsecond precision and looser
: > this is not an issue.
:
: I ought to qualify that.  There are, of course, time scales which
: are synchronized across the world with much better than microsecond
: precision (e.g., GPS time) but it's my understanding that they are
: not anything like as uniform as TAI.  Is this right?  Can anybody
: quote accuracy information for GPS time?  At what level of precision
: do time transfers using GPS time need out-of-band ("postal")
: correction for uniformity?

GPS is synchronized to UTC(USNO) which agrees with UTC(NIST) to within
a few nanoseconds.  GPS receivers can recover UTC(USNO) to within
about a few nanoseconds (as measured by a direct two way time exchange
with NIST), although not all of them are that quality.  The rms of a
good timing receiver is on the order of nanoseconds.  NIST and USNO
coordinate with each other to make sure there's close agreement in the
two minor variations of UTC.  They also participate in the computation
of TAI.

TAI is computed after the fact based on the observed time of different
frequency standards.  Phase offsets between different time sources are
measured, the data is decimated and placed into a particular format
and sent to BIPM.  These measurements are sent in batches and used to
measure interesting effects.

Here are some reports on how the various labs measure time relative to
a known source:

http://www.bipm.fr/cc/CCTF/Allowed/16/cctf04-18.pdf
http://www.bipm.org/en/publications/scientif/tai.html

Warner

Re: NOT A cruel fraud!

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Ed Davies <[EMAIL PROTECTED]> writes:
: M. Warner Losh wrote:
: > : >  TAI is specifically contraindicated as a time
: > : > scale.
: > :
: > :  > TAI is not currently recommended by its creators as a viable time
: > :  > scale.
: > :  >
: > :
: > : These claims are intellectually fraudulent. The archives in fact support
: > : the opposite of what Mr. Losh contends.
: >
: > Actually, it isn't quite that cut and dried.
:
: OK, so why is TAI contraindicated then?
:
: I've been on this list since 2002-07 and not yet seen a good
: argument against it.

The short answer is that you cannot get a time feed of TAI, so the
high precision folks don't like people calling something that's
derived indirectly TAI.  TAI(UTC(foo)) can be derived, however, if
you know the current leap offset.  This isn't broadcast in many time
formats.

Having said that, our company uses TAI internally for all its time
code and translates on output.  Doing that doesn't free you from
knowing about leap seconds, however, since you have to know about them
to convert to/from UTC which is universally broadcast or available.
GPS makes things a little easier, since there's a fixed offset from
TAI, but even it present difficulties because you need the current
leap offset to display UTC times.  However, some GPS receivers insist
on doing silly things when you try to get the raw time without
leapseconds.  Our systems need to have unambiguous timestamps, as well
as a requirement to display UTC.

Warner

Re: NOT A cruel fraud!

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
James Maynard <[EMAIL PROTECTED]> writes:
: If "we've been over this in great detail," I would like a more specific
: reference to the postings that did so.  Also, "we've been over this in
: great detail" seems not to have settled the issue.

We have been over these points in great detail.  Both here, and over
on the time-nuts mailing list.  Many of the same posters post to both
lists.

: It's cruel to insult a newbie so, and cause him to spend hours and hours
: perusing the archives, to no avail.

You learned a great deal.  You can learn more by reading the time nuts
archive as well.  http://www.febo.com/pipermail/time-nuts/

: >  TAI is specifically contraindicated as a time
: > scale.
:
:  > TAI is not currently recommended by its creators as a viable time
:  > scale.
:  >
:
: These claims are intellectually fraudulent. The archives in fact support
: the opposite of what Mr. Losh contends.

Actually, it isn't quite that cut and dried.

Warner

Re: Risks of change to UTC

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
"Daniel R. Tobias" <[EMAIL PROTECTED]> writes:
: On 21 Jan 2006 at 10:11, M. Warner Losh wrote:
:
: > I maintain that for human activity, there's no need for leap seconds
: > at all.  In each person's lifetime, the accumulated error is on the
: > order of a few minutes.  Over generations, the problems with noon
: > drifting to 1pm can trivially be solved by moving the timezones that
: > civilian time uses.
:
: What about when that accumulated difference is over 24 hours, so the
: offset between solar-based time and atomic time is actually on the
: order of days?

>From http://www.ucolick.org/~sla/leapsecs/dutc.html we know that the
rate of change of the day is somewhere between 25.6 s/century^2 and 42
s/century^2.  At those rates, it will be the year 6360-7633 when
enough time has accumulated for there to be a day difference.  Before
then, however, we go through a number of events.  Somewhere between
2058 and 2211, enter the realm where there's more than 2 leap seconds
per year.  This will be the first great UTC breakage because many
devices today (ntp included) KNOW that leap seconds happen twice
yearly.  The next break will happen sometime between 2300 and 2600
when we'll need more than 4 leap seconds a year.  The current ITU-R
TG.460 standard for leap seconds defines only primary and secondary
leap second times.  It does not define tertiary, so no one knows when
the leap seconds will happen if you need more than 4 per year, but the
ITU recommendation is that they happen at the end of a month.
Somewhere between 3250 and 4200 there will be more than 1 leap second
a month needed.  At this point the ITU scheme of having the leap
second at the end of the month will need to be modified.

That's at least 2000 years before 24 hours of delta have accumulated.

For some perspective, we've been using UTC for only ~50 years and the
gregorian calendar for only ~1500 years.  I'd anticipate that
something would need to be done about the slowing of the day well
before 4300 years have passed.

Somewhere around betwee 45,000-80,000 you'll need more than one leap
second a day.

: Will people be able to deal with a civil time
: standard that is based on an offset from a "UTC" that says it's
: Monday when all actual points on Earth have the local date at
: Saturday or Sunday?

Since that's 4k or more years into the future, who alive today will
know enough about what the future is like to impose a scheme that is
guranteed to work?

Clearly some scheme better than leap seconds will need to be invented
well in advance of these events.  A new scheme will be needed well in
advance of the Tuesday is really Wednesday problem.

: Many Web sites (including Wikipedia) use UTC as
: the standard for date/timestamps; will this be a reasonable thing
: when this causes the date of postings to be far off from what is
: being used locally?  And when, at some future point, the Gregorian
: calendar itself needs adjustment to handle the fact that it doesn't
: get the length of the year precisely correctly (and the length of the
: year in terms of solar days is changing due to the lengthening of the
: day, anyway), will this adjustment be done to the UTC standard (why,
: when it doesn't follow astronomy anyway?), or as an additional offset
: to local times (which could result in different countries having
: different dates as in the Julian/Gregorian transition period)?

The length of the gregorian calendar is off by 23s per year.  In year
5500 or so we'll have accumulated a day of error in it and we'll need to
skip a leap year to correct for that problem.  This is a good 2000
years before we'll have accumulated a day of DUT.

As you can see from the above, leap seconds won't save you.  They will
run out of steap in about 1500 to 2500 years.  At that point the
accumulated difference will be only about 2 hours.  If leap seconds
are totally abolished, time zone transitions could easily continue for
about 4000 years.  Either way, you have a problem.  The length of the
SI second is fixed, and the length of the day is getting shorter.

1500 years ago, no one spoke English.  Chances are the people that
deal with this problem in 1000 or 2000 years won't speak any language
recognizable to anybody alive today.

Warner

Re: Risks of change to UTC

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
James Maynard <[EMAIL PROTECTED]> writes:
: M. Warner Losh wrote:
: > UTC works for navigation, but leap seconds pose problems for other
: > users of time.  Stating absolutely that UTC is not broken ignores
: > these other users.
:
: Those "other uses," for whom leap seconds pose a problem, should be
: using a time scale that does not have leap seconds. They would be better
: served, for example, by TAI.

You really should read the archives of this list.  We've been over
this in great detail.  TAI is specifically contraindicated as a time
scale.  This also ignores the issues surrounding the proper
computation of elapsed time using the UTC time scale when represnted
by a number.  It ignores the fact that you must have a table of leap
seconds to do this effectively.  It ignores that one must have a table
of leap seconds to get TAI time, because most providers of time
provide it in UTC.

: For civil use, a calendar that counts days reaonably accurately is
: appropriate.  The Gregorian (New Style calendar) that the vast
: majority of the planetary population uses does this. UTC copes with
: the variable length of the mean solar day by inserting leap seconds
: as needed.  The role of the IERS in decreeing when leap seconds are
: needed is similar to that of the Roman College of Pontifices who managed
: the old Roman Republican calendar (before Julius Caesar's reform) by
: decreeing, as needed, when to shorten the month of February and insert
: the intercalary month of Mercurius.

I maintain that for human activity, there's no need for leap seconds
at all.  In each person's lifetime, the accumulated error is on the
order of a few minutes.  Over generations, the problems with noon
drifting to 1pm can trivially be solved by moving the timezones that
civilian time uses.

Keeping universal time synchronized to an arbitrary meridian is
already arbitrary.  Adding leap seconds is an arbitrary decision to
implement that arbitrary requirement.  It is really only important for
those uses of time that care what the local solar time is to within a
second (as opposed to within an hour which everyone in civil society
is used to since the late 19th century and time zones).

: If your primary need is for a time scale ithat counts SI seconds, with
: no leap seconds to confuse the matter, then don't use UTC. Use a time
: scale that counts SI seconds, such as TAI or GPS time. There's no point
: to applying the mised radix Gregorian calendar system to such a time
: scale, although you can do so if you wish.  Count days of 86 400 SI
: seconds each, or GPS weeks of 604 800 SI seconds, or just count SI seconds.
:
: If, on the other hand, you need to count solar days, or mean solar days,
: use a calendar and time scale that does so.  In order to know which way
: the earth is pointing, use UT1, or a compromise scale such as UTC that
:   is kept reasonably close to UT1.  For the vast majority of the
: population of the planet, including celestial navigators, UTC is good
: enough. If you want to know the direction the earth is pointing with
: more precision, apply DUT1 corrections, or use other IERS products such
: as Bulletin A.

That is one solution to the problem, but it is not the only solution
to the problem.  Quoting the old, tired saws about using TAI or GPS
time scales to count SI seconds doesn't promote good dialog.  Instead,
it drags this discussion down to the level of Dogma.

There are many users that need to both count SI seconds, as well as
synchronize their operations to civilian time.  Leap seconds cause
these users real difficulties.  Implementing leap seconds in software
is hard to get pedantically right (I have a pile of bugs in code that
was written by smart people that get leap seconds wrong, and over 200
hours of time spent in the past few years on implement, testing and
debugging leap seconds).  Even many ntp servers on the net got the
leap second wrong, as did many of the time stations.  There's much
evidence that this "solution" to the problem has shifted the cost of
keeping astronomers, celestial navigators, and others with a real need
to the solar time happy to a large number of other users who have no
such real need.  As such, it seems only fair in today's economy to
place the costs of the needs of a community on those communities that
have a real need for those goods or services.

: There is no need to "fix" the time scale, UTC, that is used by the vast
: majority of the planet's population to accommodate the very small
: minority of precision time users who desire a time scale that has no
: leap seconds. Let that minority use a time scale, such as TAI, that does
: not have those messy leap seconds.

TAI is not currently recommended by its creators as a viable time
scale.

In my day job, we do use TAI internally.  That doesn't solve the
problems of leap second

Re: Risks of change to UTC

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
"Brian Garrett" <[EMAIL PROTECTED]> writes:
: > But the protocol for broadcasting DUT1 in, e.g., WWV, does not provide
: > for DUT1 values of more than plus or minus 0.9 s.  The value of DUT1
: > could be announced by voice message -- but that would not be
: > language-independent.  If I travel to asia in my boat, I will not be
: > able to benefit from DUT1 announcemnts in Japanese from JJY or in
: > Chinese from BPM (or whatever their standard time and frequency station
: > is).  An longwave broadcasts such as those from WWVB do not have voice
: > modulation at all!
: >
: WWVB broadcasts the DUT1 correction in binary-coded decimal form, and is
: capable of representing DUT1 values up to +/- 1.5 seconds with their current
: format.  I would think other longwave stations do the same, but not knowing
: their particular BCD format I couldn't say what DUT1 values they are capable
: of representing.

Taking a long term view, some changes in the broadcast format or
interpratation would be necessary.  It is quite possible to multiplex
these bits so that successive opportunities to transmit the DUT1
encode additional digits as well.  This would mean it would take a
while to transmit DUT1, but such operational issues have been deemed
acceptable in systems such as GPS which transmit leap second
information only a few times an hour.

There appears to be about 13 unused bits in the WWV stream
http://tf.nist.gov/timefreq/stations/wwvtimecode.htm at second 8, 14,
24, 27, 28, 34, 42, 43, 44, 45, 46, 47, and 48.  Granted, some of
these are spacer bits between BCD digits, but they could be reclaimed.
The WWVB timecode
http://tf.nist.gov/timefreq/stations/wwvbtimecode.htm has fewer free
bits at 9: 5, 11, 14, 21, 24, 34, 35, 44, and 54.  10 if you count the
redundant sign bit.  With enough advance notice, these bits could be
used to encode larger DUT1 values.

If you travel to asia, you will no doubt know what DUT1 is to within a
second before you leave the US.  The rate of DUT1 change is on the
order of 1s per year.  This should obviate the need to be told the
current DUT1 value every minute.  Even the DUT1 values broadcast to
the .1s level only change monthly or so (although there are provisions
to change them with only 2 weeks notice).

I have no delusions that changing the meaning of the bits in the time
codes braodcast from WWV and WWVB will happen in less than a decade...

Warner

Re: Risks of change to UTC

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
James Maynard <[EMAIL PROTECTED]> writes:
: M. Warner Losh wrote:
:
: > : > If DUT1 is broadcast, then one can set the time keeping device to UT1
: > : > by a means similar to setting it to UTC, even if DUT1 exceeds 0.9s
: > : > with a similar accuacy (or better).  There's nothing that says a watch
: > : > has to display UTC to be set correctly.

WWV and most of the world's time stations broadcast DUT1.  I should
have added in my last message that some change in the signal format
would be necessary if the range of DUT1 exceeds 0.9s.  Given a
sufficently long time horizon, this becomes a non-issue.

: You seem to expect celestial navigators to become more sophisticated in
: these matters of time scales than is presently the case. It would be far
: better, in my option, not to tinker with the definition of UTC and of
: civil time at all, but leave civil time zones tied approximately to UT1.

Maybe that is a reasonable expectation.  Maybe it is not.

I will note that the profile of high precision time users has changed
since 1972 when UTC was invented.  In 1972 celestial navigators were a
big part of the users of time.  These days, they represent a much
smaller portion of that audience.  If there are changes to how time is
provided, then all users of precision time should be considered.
Should we continue to tie our time up in knots because of a tiny
minority of users?

Maybe the answer that question is yes.  Maybe it is no.  However,
stating absoultely, unquestionably that it is one or the other is what
is wrong and what takes this discussion off into the weeds.

How many celestial navigators are there today?

: "There's nothing that says a watch has to display UTC to be set
: correctly."  But then the user would have to carry two watches, one set
: to an approximation of UT1 (e.g., UTC + DUT1) and another set to civil
: time (TI, or local zone time, or whatever). For simpler, it would be,
: with far less impact on ordinary users of time, not to let UTC (or its
: replacement) differ from UT1 by more than the present +/- 0.9 s.

Over the next 50 years, these two watches will be well within the
tolerance of most normal watches.  The approximation of civil time
will be less than one minute off during that time (if most of the
projections are to be believed).

If this is a real issue, the market will take over and produce watches
that have 'navigation time' and 'civilian time' at the touch of a
button.  If you still desire to use older, purely mechanical watches,
then you must take extra steps, like knowing the current offset
between civil time and navigation time.  Of course, purely mechanical
watches have other issues when used on a boat.  Electro-mechanical
ones don't suffer those problems, but require some power source.

The foregoing assumes, for the sake of argument, that DUT1 becomes
unbounded.

: UTC is not broken. There is no need to "fix" it.

UTC works for navigation, but leap seconds pose problems for other
users of time.  Stating absolutely that UTC is not broken ignores
these other users.

: > I'd also add that GPS receivers today already do exactly this sort of
: > correction when they decode the GPS time, but display the UTC time.
:
: True. But GPS receivers are electrically powered, and are not a fallback
: for the case when a boat's electrical systems fail.  Nor are they a
: backup system for the case when the the boater is in proximity to a
: war zone where the U.S. DoD is jamming or degrading the civil SPS GPS
: service.

GLASNOS is a backup system to GPS that is not subject to DoD's
selective denial of signal.  Soon, Galileo will provide additional
backup.  Also, there's about a dozen time stations worldwide that
provide time signals today which can act as backup to GPS during a GPS
outage.  I pointed out GPS more as an example of a 'clock that gets it
times in one timescale, but displays it in another' rather than as a
backup time source for your example mariner.

Warner

Re: Risks of change to UTC

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
James Maynard <[EMAIL PROTECTED]> writes:
: M. Warner Losh wrote:
: > In message: <[EMAIL PROTECTED]>
: > James Maynard <[EMAIL PROTECTED]> writes:
: > : ones position using sight reduction tables.  Today a mechanical watch or
: > : chronometer, or a battery-powered wristwatch, can be set to UTC using
: > : radio time signals. Then when power fails, the sailor still has a
: > : reasonably accurate spprodximation to UT1 available.
: >
: > If DUT1 is broadcast, then one can set the time keeping device to UT1
: > by a means similar to setting it to UTC, even if DUT1 exceeds 0.9s
: > with a similar accuacy (or better).  There's nothing that says a watch
: > has to display UTC to be set correctly.
: >
: > Warner
: >
: > .
: >
: And how is DUT1 to be broadcast in a language-independent manner?  That
: protocol needs to be established well in advance.

I should add that I read your 'can be set to utc using radio time
signals' to mean something like WWV or other radio time service.
Those sevices do already broadcast DUT1.  Any watch that is smart
enough to decode those signals would be smart enough to add this minor
correction as well.

The mechanical watch might be a bit of a problem, but DUT1 doesn't
change enough to introduce navigation errors similar to what we have
today over the course of a year and can easily be looked up like
someone would lookup what the weather was going to be like.

I'd also add that GPS receivers today already do exactly this sort of
correction when they decode the GPS time, but display the UTC time.

Warner

Re: Risks of change to UTC

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
James Maynard <[EMAIL PROTECTED]> writes:
: M. Warner Losh wrote:
: > In message: <[EMAIL PROTECTED]>
: > James Maynard <[EMAIL PROTECTED]> writes:
: > : ones position using sight reduction tables.  Today a mechanical watch or
: > : chronometer, or a battery-powered wristwatch, can be set to UTC using
: > : radio time signals. Then when power fails, the sailor still has a
: > : reasonably accurate spprodximation to UT1 available.
: >
: > If DUT1 is broadcast, then one can set the time keeping device to UT1
: > by a means similar to setting it to UTC, even if DUT1 exceeds 0.9s
: > with a similar accuacy (or better).  There's nothing that says a watch
: > has to display UTC to be set correctly.
: >
: > Warner
: >
: > .
: >
: And how is DUT1 to be broadcast in a language-independent manner?  That
: protocol needs to be established well in advance.

It already is being broadcast in, eg, WWV.

Warner

Re: Risks of change to UTC

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
James Maynard <[EMAIL PROTECTED]> writes:
: ones position using sight reduction tables.  Today a mechanical watch or
: chronometer, or a battery-powered wristwatch, can be set to UTC using
: radio time signals. Then when power fails, the sailor still has a
: reasonably accurate spprodximation to UT1 available.

If DUT1 is broadcast, then one can set the time keeping device to UT1
by a means similar to setting it to UTC, even if DUT1 exceeds 0.9s
with a similar accuacy (or better).  There's nothing that says a watch
has to display UTC to be set correctly.

Warner

Re: Risks of change to UTC

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Neal McBurnett <[EMAIL PROTECTED]> writes:
: These astronomical and navigation systems currently get inputs of time
: from all sorts of other systems and people: other instruments, other
: computers, sub-contractors, etc.  Changing the meaning of UTC would
: lead to a cascade of systems that changed their own timescales and
: I/O formats, meaning that auditing the flow of time data between them
: would get more and more complex for a long time.

The changing formats should indeed be phased in.  First, changes to
the data streams so that they can represent DUT1 > 0.9s.  Next, fix
everything that breaks...  Repeat.  You're likely right, this will
take a long long time :-(.

Warner

Re: Internet-Draft on UTC-SLS

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Tim Shepard <[EMAIL PROTECTED]> writes:
: > The serious timekeeping people gave up on rubberseconds in 1972 and
: > I will object with all that I can muster against reinventing them
: > to paste over a problem that has a multitude of correct solutions.
:
: As I learned from a recent posting to this mailing list, it seems that
: even TAI has rubber seconds (adjustments to the rate is made from time
: to time to compensate for errors that have been accumulating, making
: TAI a better (more useful) approximation time).
:
: Do you object to those adjustments (rubber seconds) to TAI as well?

TAI changed the definition of the second in 1977 by one part in 10^12.
Those weren't rubber seconds.  It again slowly changed the definition
over 3 years of the second by 2 parts in 10^14 ending in 1998.  They
were made not to correct the phase of TAI, but rather to correct the
length of the second to more closely match the SI second definition.
No phase adjustments were made to TAI.

Rubber seconds, on the other hand, are a steering device designed to
keep time in sync with another clock, in this case the earth.  They
aren't changing the definition of the second, but rather are inserting
phase adjustments to the timescale.  The goofy scheme that existed
prior to 1972 fell into this latter category.

Warner

Re: Internet-Draft on UTC-SLS

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Markus Kuhn <[EMAIL PROTECTED]> writes:
: "M. Warner Losh" wrote on 2006-01-19 19:20 UTC:
: > Effectively, you'd have to have two time scales in the kernel.  UTC
: > and UTC-SLS.  You make it sound simple, but the hair in doing this may
: > be quite difficult.  There's more book for the kernel to keep, and it
: > would have to expose the bookkeeping to userland for ntp to work.
: > What makes this hard is that ntpd may introduce steers into the normal
: > system time at normal times which it doesn't want to confuse with the
: > steers in frequency that are used during a UTC-SLS operation.
:
: You correctly point out some of the design considerations that have to
: go into such code. You describe roughly one of the (several) different
: ways of implementing all this that I have in mind. In comparison to how
: complicated the Mills kernel PLL is already today, that does not
: actually sound like an overwhelming additional complexity. Actually, it
: sounds quite doable when you think through it a bit. Not trivial, but
: carefully doable without performance penalty.

Anything that makes the Mills' kernel PLL more complicated is unlikely
to be implemented correctly.  There have been several subtle bugs in
Mills' implementation that have taken a long time to sort out.  In
fact, members of this list have found some of them and have a lot of
experience with Mills' kernel PLL.

Many people in the timing community are unhappy with Mills'
implementation.  They believe it to be way too complex and have been
trying to implement a much simpler implementation that gives more
control to the measurement process to apportion the errors that it
measures by doing simpler steers.  It moves the kernel time control in
the wrong direction.

: The important thing is: All this has to be done only *once* for each
: operating system and *once* for each each NTP client/server used on it,
: by someone who understands the issue, and then the problem is hopefully
: solved. We've handled far more complicated things in OS kernels, right?

No.  It has to be done continuously.  By that I mean that you can
clearly write an implementation once, but over time you will find
flaws with that implementation (I can 100% guarnatee that).  As you
find the flaws, you'll have to investigate them and fix them.  In
addition, other parts of the kernel change, and some of those changes,
though innocent looking, can have a big impact on how you implemented
the dual UTC UTC-SLS offset.

By way of illustration, in FreeBSD 2.2, the timing implementation was
changed from keeping the current time, to keeping an uptime plus a
boot time.  Adjustments to the boot time were used to accomplish phase
shifts in the system time.  Unbeknownst to the person implementing
this, despite him having integrated the Mills kernel NTP portion into
a prior version of FreeBSD, this broke the exact semantics of system
time around the leap second.  It took years for someone (me) to notice
and fix this problem.

: I also fully appreciate that the existing *interface* between
: kernel-level clock drivers and user-level NTP clients and servers is by
: far not as well specified as it could be. With all due respect to its
: author, I do not hope that RFC 1589 will not remain the last word on
: clock-driver interfaces. A detailed proposal for a new standard on that
: area should help to smooth the process of getting the above right. I
: think a serious clock-controlling API this is what some people have been
: asking for for a long time from POSIX.

In order to implement your UTC-SLS proposal, it must change.

: I believe it is good engineering practice to specify the functionality
: before the interface and implementation, therefore I would like to
: discuss aspects of desired functionality (including UTC-SLS) before
: starting to work on a proposed successor to adjtimex(), ntp_adjtime(),
: etc.

While I agree in principle, I can tell you as one who has bloodied his
knuckles on implementing timekeeping that what you are saying sounds
hard to get pedantically right.

: I am not claiming that the spec-writing job is finished, or even half
: finished yet. I see a replacement for RFC 1589 etc. as the next job on
: the agenda before UTC-SLS (and lots of other time API ideas floating
: around here and elsewhere) can start to fly.

Unfortunately, it makes it hard to evaluate your proposal in the
absense of a concrete spec.  What is there now sounds nice, but it
still strikes me as difficult to implement.

Warner

Re: Internet-Draft on UTC-SLS

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Markus Kuhn <[EMAIL PROTECTED]> writes:
: "M. Warner Losh" wrote on 2006-01-19 19:20 UTC:
: > : In other words, *no* incompatible changes are made to the NTP protocol.
: > : In a correct UTC-SLS implementation, you should *not* be able to
: > : distinguish remotely, whether some NTP server synchronizes its kernel
: > : clock to UTC or UTC-SLS, because this must not influence its NTP
: > : interface in any way.
: >
: > In the systems that I observed over the leap second, I can say that
: > those systems that did implement a UTC-SLS-like approach to steering
: > out the extra second were definitely observable while they did this.
:
: Yes, of course! If you connect a GPS receiver that delivers an
: undocumented UTC-SLS-alike timescale on its output to an NTP server that
: in turn knows nothing of this UTC-SLS-alike timescale and is not
: implemented accordingly, you must get a mess. What did you expect? What
: you observed is *not* an implementation of UTC-SLS, but just a
: connection of incompatible components!

No.  You misunderstand me.  There were no GPS receivers that did
UTC-SLS-alike timestamps.  This was what CLIENTS of the GPS receiver
did.  The GPS receiver did exactly the right thing, ntp did the right
thing, our products did the right thing.  What I observed was in the
CLIENTS how they dealt with the leap seconds.

GPS-RECEIVER --- GPSNTPD --- NTP CLIENT1[*] windows NT box
 +-- NTP CLIENT2[*] Linux box
 +-- NTP CLIENT3[*] FreeBSD box
 +-- NTP CLIENT4[*] FreeBSD box

The items marked with a [*] are the ones I observed.  client1 and
client2 were the ones that I observed implementing UTC-SLS like time
and such time was visible to the outside world.  I was clearly able to
see both on the GPSNTPD clients 3 and 4 the skews of clients 1 and 2
during the leap second, where I observed no such skews relative to
GSPNTPD.

I'm saying that certain OSes have implemented the UTC-SLS-like
systems.  I'm saying that ntp daemons running on those systems use the
actual system time for time exchanges, rather than real UTC.  I'm
saying that this can cause confusion to a downstream ntp client and
that's my basis for thinking this idea is not good.  I'm further
saying that for the OS to support ntpd, it would need to also support
UTC and provide an interface to get the raw UTC time.  It would need
to keep track of steers for the purpose of implementing UTC-SLS and
steers needed to compensate for the changing base clock frequency
separate so that it could give the right time when asked for UTC
time.  It would also need to give some kind of feedback to ntpd that
it didn't do the small steer it wanted because UTC-SLS is going on at
that time (since otherwise ntp would think that there's some frequency
step going on if it just ignored it, and doing both steers
concurrently would be very hairy).

It is a seductive idea that leaves one with a big hangover in the
morning.

: > : However, MSF has no leap warning at all, nor do some time codes used in
: > : the power or military industry. And recent extensions to the latter
: > : added only a leap second warning that arrives a few seconds before the
: > : leap. I consider the leap-second handling of these latter time codes
: > : pretty useless.
: >
: > IRIG-B (all IRIG?) has no leap second announcement that I could find.
: > The Irig standard I could find (IRIG STANDARD 200-04, issued September
: > 2004, at http://www.jcte.jcs.mil/RCC/PUBS/pubs.htm) makes no mention
: > of the format of the leap seconds at all (the leap second convention
: > appendix says that leap seconds happen).  Talking to engineers here
: > who have implemented IRIG at various places reveals that there are
: > multiple conventions for dealing with leap seconds in irig (the new
: > standard being to do 59, 60, the old standard was to do 59 59).
: > There's no provision for leap second announcement, although control
: > bits do remain unassigned...
:
: The IRIG world is clearly a huge mess, with more than two dozen
: formats in use. A useful beginner's overview is at
:
:   http://www.symmttm.com/pdf/Gps/an_Time_and_Time_Code_Ref.pdf

Excuse me, but isn't it a bit patronizing to point someone at a
beginner's guide?  I don't need beginner's overviews.  I have plenty.
I've been dealing with Irig for several years now, as well as many
other time codes and timing issues (including leap seconds).  What I
need is references to standards and copies of standards.  I want to
understand all the variations of IRIG so I know what the
sales/marketing folk are talking about when they ask me if thus and
such standard is easy or hard.

: which outlines on page 23 the 27-bit IEEE 1355 power-industry extension
: to IRG-B that has a Leap Second Pendin

Re: Fixing POSIX time

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Markus Kuhn <[EMAIL PROTECTED]> writes:
: Poul-Henning Kamp wrote on 2006-01-19 17:56 UTC:
: > >For TAI I'd suggest 1958-01-01, when TAI and UT were set nearly together.
: >
: > I chose the time when TAI became constant rate so that
: > all the rubber seconds are confined to negative values.
:
: Please remember that the TAI second differed noticeably from the SI
: second until about 1998, because black-body radiation shift was not
: taken into account in the definition of TAI before then. Also caesium
: fountains have improved quite a lot shortly before 2000.

According to the web site that I posted
(http://www.ucolick.org/~sla/leapsecs/timescales.html), this feature
was in both TAI and UTC.  From the url:

UTC on 1977-01-01
 Because the rate of TAI was reduced by one part in 1012, the rate
 of UTC was reduced by the same amount. Therefore, before this date UTC
 seconds were shorter than they currently are.
UTC from 1995 through 1998
 In 1995 a CCTF working group determined that the length of TAI
 seconds was longer than the SI second because the clocks contributing
 to TAI were not corrected for the effects of blackbody
 radiation. Over the next three years the frequency of TAI was steered
 to reduce the length of its seconds by about 2 parts in
 1014. Therefore the length of UTC seconds was also reduced. This
 change is evident as the final kink in the plot of TT(BIPM04).

[EMAIL PROTECTED] is a frequent contributer here, and this looks like
his web site.

I'd be interested in a source that describes these events differently...

: Therefore, if people ask me for my favourite epoch for a new time scale,
: then it is
:
:   2000-03-01 00:00:00 (preferably UTC, but I would not mind much
:if it were TAI, or even "GPS time")
:
: This epoch has the following advantages:
:
:   a) It is well after TAI rubber seconds were fixed in ~1998,
:  so we know the time of *that* epoch with much greater accuracy than
:  any before 1998.

TAI and UTC have ticked at the same rate since 1972.  While this rate
has changed twice (by very small amounts, first by 1 part in 10^12 and
then later by 2 parts in 10^14), they have been the same.  Prior to
1972 we had both steps in time (on the order of 50ms to 100ms) as well
as TAI and UTC having different notions of the second.

Warner

Re: the GPS impending leap second bit

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Steve Allen <[EMAIL PROTECTED]> writes:
: On Thu 2006-01-19T18:30:02 +, Markus Kuhn hath writ:
: > GPS sends out announcements within days after IERS does, which is
: > excellent service.

IERS announced the leap second on July 4th, about 6 weeks before it
was encoded into GPS stream.  This didn't happen until mid to late
August.

The Canadian time service was announcing the leap second months before
July (I have a friend who has been writing software to decode the
world's time broadcast stations, and he was amused by this).

: Yes, and no, for some notes from the NTP newsgroup imply that the
: setting of the GPS impending leap bit caused some GPS units to insert
: leap seconds every day at 0h UT.

We noted that the GPS system started to advertise leap seconds in the
middle of August.  This causes the HP SmartClocks to hickup at the
alternate leap second time...

: Furthermore, in
: http://www.uscg.mil/hq/g-m/moa/docs/Saab505.pdf
: we see that the Saab AIS units introduced the leap second immediately
: and needed a firmware update to fix the condition.

Cute.  Older Motorola Oncore receivers had a firmware bug that caused
a three second blimp in their time code when it had been 1024 weeks
since the last leap second as well.

Warner

Re: Internet-Draft on UTC-SLS

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Markus Kuhn <[EMAIL PROTECTED]> writes:
: "M. Warner Losh" wrote on 2006-01-19 16:58 UTC:
: > > http://www.ietf.org/internet-drafts/draft-kuhn-leapsecond-00.txt
: > The biggest objection that I have to it is that NTP servers will be at
: > least .5s off, which is far outside the normal range that NTP likes to
: > operate.  Unless the prceice interval is defined, you'll wind up with
: > the stratum 1 servers putting out different times, which ntpd doesn't
: > react well to.
:
: Please read the proposal carefully (Section 2):
:
:UTC-SLS is not intended to be used as a drop-in replacement in
:specifications that are themselves concerned with the accurate
:synchronization of clocks and that have already an exactly specified
:behavior near UTC leap seconds (e.g., NTP [RFC1305], PTP [IEC1588]).

I saw that, but it does present some interesting implementation issues
for an ntp server.

: What this means is:
:
:   - NTP still uses UTC on the wire, exactly in the same way in which it
: does so far, *independent* of whether any of the NTP servers or clients
: involved supply UTC-SLS to their applications.

This is the hard part.  Right now, NTP steers the system time to UTC
as best it can.  It steps the time back by one second at the leap
second since there's no way in POSIX's time_t to represent leap
seconds, and all the time interfaces to the kernel are in terms of
time_t.

During the recent leap second, I observed three different clients of
one of our GPS receivers do three different things with the leap
second:  One inserted it early with the kind of hard frequency steer
that UTC-SLS does.  The second did it centered around the leap second,
and one didn't do anything until after the leap second and a couple of
polling intervals, then steered it out.  It is based on these systems
that implement the UTC-SLS-like steers that I based my reaction on.

:   - NTP implementations (by this I mean the combined user-space and
: kernel-space segments) should convert NTP timestamps that have been
: received over the wire through the UTC -> UTC-SLS mapping, and steer
: after that what gettimeofday() provides to users .
:
:   - NTP implementations should equally also convert any timestamp received
: from gettimeofday through the UTC-SLS -> UTC mapping before it goes
: out the wire.

This is the difficult part.  Right now, ntp daemons use the system
time as the time they do the time exchange with.  This is even if they
have a fancy gps receiver connected to them that isn't yet fully
synchronized.  Since they use system time, as you observe there would
need to be some mapping.  This mapping would necessarily have to be in
the kernel (since only it has the knowledge of the actual UTC UTC-SLS
offset) and would need to be an additional system call.  Such system
calls are not yet standardized.  This adds additional overhead to the
time exchange, which increases the opportunity for latency to creap
in.

Effectively, you'd have to have two time scales in the kernel.  UTC
and UTC-SLS.  You make it sound simple, but the hair in doing this may
be quite difficult.  There's more book for the kernel to keep, and it
would have to expose the bookkeeping to userland for ntp to work.
What makes this hard is that ntpd may introduce steers into the normal
system time at normal times which it doesn't want to confuse with the
steers in frequency that are used during a UTC-SLS operation.

: In other words, *no* incompatible changes are made to the NTP protocol.
: In a correct UTC-SLS implementation, you should *not* be able to
: distinguish remotely, whether some NTP server synchronizes its kernel
: clock to UTC or UTC-SLS, because this must not influence its NTP
: interface in any way.

In the systems that I observed over the leap second, I can say that
those systems that did implement a UTC-SLS-like approach to steering
out the extra second were definitely observable while they did this.
Given the variance in approach to this steering, and given it was
observable to the outside world, I concluded that N independent ntp
servers doing this would only confuse ntp clients...

: I hope this answers your concerns.

Well, it raises a whole different set of issues than I had originally
feared.  ntpd's use of system time to do its time exchanges has caused
me a significant amount of trouble in the past.

: > I'm also concerned about the fact that many radio time codes do not
: > announce the leap second pending until the last hour or less.  This
: > makes it hard to propigate out to the non-stratum 1 servers.
:
: I fully agree that leap seconds should be announced as early as
: possible, and I think that anything less than a month is undesireable.
: GPS sends out announcements within days after IERS does, which is
: excellent service. NIST sends out ann

Re: Fixing POSIX time

[M. Warner Losh]

I like this idea as well...

Poul, maybe we should implement this on FreeBSD.

I'd suggest "working_time_t" or "correct_time_t" as the name of the
type to replace "time_t" which would be deprecated. :-)

Warner

Re: Internet-Draft on UTC-SLS

[M. Warner Losh]

The biggest objection that I have to it is that NTP servers will be at
least .5s off, which is far outside the normal range that NTP likes to
operate.  Unless the prceice interval is defined, you'll wind up with
the stratum 1 servers putting out different times, which ntpd doesn't
react well to.

ntpd runs at a 1024s interval, typically.  Depending on when the leap
second is announced, and when it polls, ntpd would see a 1s step
(since the rubber seconds happen over 1000s).  It wouldn't believe it
(since it has popcorn outlier rejection).  It would then believe the
step after 2048s, but since the step is so large, it would just exit
(depending on the flags given to ntpd).  This will lead to *MORE*
clock skew, not less.

I'm also concerned about the fact that many radio time codes do not
announce the leap second pending until the last hour or less.  This
makes it hard to propigate out to the non-stratum 1 servers.

It is a horrible idea.  It is a horrible kludge and I'm not mollified
by the weak justifications that are given in the "faqcrit."  My
concerns are real and valid and this will cause more problems than it
kludges over.  It is a terrible idea.

Warner

Re: The real problem with leap seconds

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Mark Calabretta <[EMAIL PROTECTED]> writes:
: On Wed 2006/01/18 08:17:54 -, Francois Meyer wrote
: in a message to: LEAPSECS@ROM.USNO.NAVY.MIL
:
: >Maybe it should be,  but  this  is  far  from  being
: >obvious from its current definition.
:
: I agree that the current definitions leave a lot to be desired in terms
: of clarity and rigour - an uncharitable person might even describe the
: extract of ITU-R TF.460-6 cited the other day by Michael Deckers as
: inconsistent.  However, you have to consider how UTC is actually used in
: practice and this is what my comments are based on.
:
: >1. UTC and TAI  share  the  same  rate,  the  same
: >   origin, the same second. And therefore :
: >
: >UTC - TAI = 0
:
: They both count SI seconds but the question of the origin is a bit
: muddy.
:
: You could argue that there is a fixed 10s offset between UTC and TAI
: because UTC post-1972 (everything I've said about UTC only applies
: post-1972) started with 10 leap seconds, and before 1972 UTC wasn't
: simply a representation of TAI.  There's no simple way of fudging
: radixes that I can think of to make them match up, but if this worries
: you then simply think in terms of proleptic UTC (post-1972),
: see http://en.wikipedia.org/wiki/Proleptic.

UTC (and its predecessors) prior to 1972 did not tick off in SI
seconds.  It used a fixed radix like TAI currently does.  The amount
of time in UTC seconds varied a little.  Here's the table from
ftp://maia.usno.navy.mil/ser7/tai-utc.dat

1961 JAN  1 =JD 2437300.5  TAI-UTC=   1.4228180 S + (MJD - 37300.) X 0.001296 S
1961 AUG  1 =JD 2437512.5  TAI-UTC=   1.3728180 S + (MJD - 37300.) X 0.001296 S
1962 JAN  1 =JD 2437665.5  TAI-UTC=   1.8458580 S + (MJD - 37665.) X 0.0011232S
1963 NOV  1 =JD 2438334.5  TAI-UTC=   1.9458580 S + (MJD - 37665.) X 0.0011232S
1964 JAN  1 =JD 2438395.5  TAI-UTC=   3.2401300 S + (MJD - 38761.) X 0.001296 S
1964 APR  1 =JD 2438486.5  TAI-UTC=   3.3401300 S + (MJD - 38761.) X 0.001296 S
1964 SEP  1 =JD 2438639.5  TAI-UTC=   3.4401300 S + (MJD - 38761.) X 0.001296 S
1965 JAN  1 =JD 2438761.5  TAI-UTC=   3.5401300 S + (MJD - 38761.) X 0.001296 S
1965 MAR  1 =JD 2438820.5  TAI-UTC=   3.6401300 S + (MJD - 38761.) X 0.001296 S
1965 JUL  1 =JD 2438942.5  TAI-UTC=   3.7401300 S + (MJD - 38761.) X 0.001296 S
1965 SEP  1 =JD 2439004.5  TAI-UTC=   3.8401300 S + (MJD - 38761.) X 0.001296 S
1966 JAN  1 =JD 2439126.5  TAI-UTC=   4.3131700 S + (MJD - 39126.) X 0.002592 S
1968 FEB  1 =JD 2439887.5  TAI-UTC=   4.2131700 S + (MJD - 39126.) X 0.002592 S
1972 JAN  1 =JD 2441317.5  TAI-UTC=  10.0   S + (MJD - 41317.) X 0.0  S
1972 JUL  1 =JD 2441499.5  TAI-UTC=  11.0   S + (MJD - 41317.) X 0.0  S
1973 JAN  1 =JD 2441683.5  TAI-UTC=  12.0   S + (MJD - 41317.) X 0.0  S
1974 JAN  1 =JD 2442048.5  TAI-UTC=  13.0   S + (MJD - 41317.) X 0.0  S
1975 JAN  1 =JD 2442413.5  TAI-UTC=  14.0   S + (MJD - 41317.) X 0.0  S
1976 JAN  1 =JD 2442778.5  TAI-UTC=  15.0   S + (MJD - 41317.) X 0.0  S
1977 JAN  1 =JD 2443144.5  TAI-UTC=  16.0   S + (MJD - 41317.) X 0.0  S
1978 JAN  1 =JD 2443509.5  TAI-UTC=  17.0   S + (MJD - 41317.) X 0.0  S
1979 JAN  1 =JD 2443874.5  TAI-UTC=  18.0   S + (MJD - 41317.) X 0.0  S
1980 JAN  1 =JD 2444239.5  TAI-UTC=  19.0   S + (MJD - 41317.) X 0.0  S
1981 JUL  1 =JD 2444786.5  TAI-UTC=  20.0   S + (MJD - 41317.) X 0.0  S
1982 JUL  1 =JD 2445151.5  TAI-UTC=  21.0   S + (MJD - 41317.) X 0.0  S
1983 JUL  1 =JD 2445516.5  TAI-UTC=  22.0   S + (MJD - 41317.) X 0.0  S
1985 JUL  1 =JD 2446247.5  TAI-UTC=  23.0   S + (MJD - 41317.) X 0.0  S
1988 JAN  1 =JD 2447161.5  TAI-UTC=  24.0   S + (MJD - 41317.) X 0.0  S
1990 JAN  1 =JD 2447892.5  TAI-UTC=  25.0   S + (MJD - 41317.) X 0.0  S
1991 JAN  1 =JD 2448257.5  TAI-UTC=  26.0   S + (MJD - 41317.) X 0.0  S
1992 JUL  1 =JD 2448804.5  TAI-UTC=  27.0   S + (MJD - 41317.) X 0.0  S
1993 JUL  1 =JD 2449169.5  TAI-UTC=  28.0   S + (MJD - 41317.) X 0.0  S
1994 JUL  1 =JD 2449534.5  TAI-UTC=  29.0   S + (MJD - 41317.) X 0.0  S
1996 JAN  1 =JD 2450083.5  TAI-UTC=  30.0   S + (MJD - 41317.) X 0.0  S
1997 JUL  1 =JD 2450630.5  TAI-UTC=  31.0   S + (MJD - 41317.) X 0.0  S
1999 JAN  1 =JD 2451179.5  TAI-UTC=  32.0   S + (MJD - 41317.) X 0.0  S
2006 JAN  1 =JD 2453736.5  TAI-UTC=  33.0   S + (MJD - 41317.) X 0.0  S

(or the yet to be updated http://www.iers.org/MainDisp.csl?pid=95-106)

Here's slides from a presentation that is actually fairly well
balanced:
http://www.ien.it/luc/cesio/itu/mc_carthy.pdf
It has history there as well.  There's a nice graph of the above on
page 20.

http://www.ucolick.org/~sla/leapsecs/timescales.html also contains a
nice summary of UTC which is too long to include here but in outline
form is:

UTC during 1960
UTC from 1961 through 1971
UTC in 1963
UTC in 

Re: The real problem with leap seconds

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Francois Meyer <[EMAIL PROTECTED]> writes:
: On Mon, 16 Jan 2006, Mark Calabretta wrote:
:
: > On Fri 2006/01/13 11:17:52 -, Michael Deckers wrote
: > in a message to: LEAPSECS@ROM.USNO.NAVY.MIL
: >
: > >   I must get TAI, up to an integration constant. This is correct.
: > >   The integral of d( UTC ) is TAI (up to an integration constant),
: > >   but this integral is UTC only where UTC is a continuous function
: > >   of TAI.
: >
: > You're still not getting the point that UTC is just a representation
: > of TAI.
:
: Maybe it should be,  but  this  is  far  from  being
: obvious from its current definition.
:
: The actual situation corresponds to :
:
: 1. UTC and TAI  share  the  same  rate,  the  same
:origin, the same second. And therefore :
:
: UTC - TAI = 0

The history of UTC and TAI is complicated and tricky.  You can only
say that on Jan 1, 1972, the TAI - UTC offset was fixed to be 10s.
UTC and TAI prior to 1972 did not evolve at the same rate.  The UTC
seconds differed in length from the SI seconds.  I'll ignore the
nomenclature differences over time as well.

: 2. UTC only differs from TAI by  its  definitions of
:the minute, hour and day.

This is not true prior to 1972.  There were a number of frequency
offsets in UTC that weren't present in TAI.  Some leap second charts
have these included in them.

: 4. the UTC minute is defined  to  ensure  that  dhms
:expressions of UTC match UT1 at .9 s; it  can  be
:either  59,  60  or  61  SI  seconds  long.  This
:definition of the minute is realized
:by  (positive  or  negative)  leap  seconds   and
:ensures that the mean UTC day is the  mean  solar
:day in the long term. The UTC  hour  has  60  UTC
:minutes, the UTC day has 24 UTC hours.

Again, post 1972...  I'm not sure what I think of this definition.


: >From that point of view, the sentence from the ITU460-6 :
:
: "[UTC] ...differs from it [TAI] from an integer of seconds"
:
: should read :
:
: "representations of UTC  involving  minutes,  hours,
: days differ from equivalent representations  of  TAI
: by an integral number of seconds"

After 1972, and ignoring minor variations in the realization of UTC
and TAI in any given location, this is basically true.  The only ideal
difference between TAI(ideal) and UTC(ideal) is in the labeling of the
pulses that both time scales have experienced.  If one were to treat
them both as fixed radix, you get the difference of 33s.  Viewed
topologically as a 1-1 mapping, one could easily define subtraction so
that the difference is zero since UTC has a variable radix...

To further complicate things, TAI isn't created in realtime, but is a
paper clock that's steered to the correct time of the clocks that feed
it data.  There are clocks that are steered to this paper clock, but
it is all done by hand (if the various web pages I've read are still
accurate).  Different facilities realization of the TAI and UTC time
scales may differ by several nanoseconds (or more depending on a lot
of factors).

However, leaving aside those complications...

Given that UTC is a variable radix notation for labeling the pulses
that have elapsed since the epoch.  TAI is a fixed radix notation for
labeling pulses that have elapsed since the epoch.

Warner

Re: The real problem with leap seconds

[Warner Losh]

From: Mark Calabretta <[EMAIL PROTECTED]>
Subject: Re: [LEAPSECS] The real problem with leap seconds
Date: Tue, 17 Jan 2006 11:15:09 +1100

> On Fri 2006/01/13 12:32:27 +1100, Mark Calabretta wrote
> in a message to: Leap Seconds Issues 
>
> >So if asked for a definition I would say that "UTC (post 1972) is a
> >representation of TAI such that ... (you know the rest)".
>
> .. and I should have said that prior to 1972, when UTC had "rubber
> seconds", it was not a representation of TAI but something else again.

Yes.  TAI was recoverable from the UTC that existed prior to 1972, but
the math wasn't a simple addition...

Warner

Re: Report of Leap Second Problem with GPS Data

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
"Poul-Henning Kamp" <[EMAIL PROTECTED]> writes:
: In message <[EMAIL PROTECTED]>, Rob Seaman writes:
: >On Jan 13, 2006, at 6:26 AM, Richard Langley wrote:
:
: >I won't claim to know the intrinsic importance attached to this.
: >Critical systems may depend on the information.  But is it fair to
: >sum up the situation by saying that a leap second triggered a couple
: >of bugs (or perhaps one common bug), they were detected, have been
: >fixed, and affected data products have been remediated?  Also, it
: >appears that some other data products were unaffected?
: >
: >So, the issue has been resolved - would likely have been resolved
: >sooner if a leap second had occurred earlier - and is no longer
: >directly pertinent to a discussion of future leap seconds?
:
: Yeah, right
:
: "This goes counter to my claims so it is of no importance".
:
: Sorry, things don't work that way Rob.

This time, there were no reports of death with the leap second,
therefore they can't be too bad... :-)

Warner

Re: The real problem with leap seconds

[Warner Losh]

From: Ed Davies <[EMAIL PROTECTED]>
> Markus Kuhn wrote:
> > Ed Davies wrote on 2006-01-13 11:45 UTC:
> >
> >>The use of the 23:59:60 notation is described in ISO 8601.
> >>Is it also specified in TF.460?
> >
> >
> > It originally comes from ITU-R TF.460, which is a standard for radio
> > time signals.
>
> OK, thanks.

Has anybody compiled a canonical list of the standards in this area?
This is the first, I think I've seen ISO 8601 mentioned.

TF.460 doesn't talk about days at all, really, or MJD.  It doesn't
talk about rendering a time a floating point number, only as the
traditional sexagesimal fractional time, with the 'execption' during
the positive leap second.

If we explore the orgins of the time
12:34:56 (just after noon)
we note that it is in the 13th 1/24th division of a day (since the 0th
(aka 12) hour is first), the 35th 1/60th division of the 1/24th
division (since the first one is 0) and the 56th 1/60th division of
the 1/60th division of the 1/24th division a day.  Labeling the
positive leap second as
23:59:60
leads to some trouble if we try to work backwards through the above
derivation.  It creates an exception to the nice, orderly rules of
time.  But I'm digressing...

The ITU-R TF.460 states:

2.2 A positive leap-second begins at 23h 59m 60s and ends at 0h 0m 0s
of the first day of the following month. In the case of a negative
leap-second, 23h 59m 58s will be followed one second later by 0h 0m 0s
of the first day of the following month (see Annex III).

2.3 The IERS should decide upon and announce the introduction of a
leap-second, such an announcement to be made at least eight weeks in
advance.

In Annex III, it talks about the dating of events during the positive
leap second.  If something were to happen .6s into that second, it
would be denoted (assuming a june leap) as:

30 June, 23h 59m 60.6s UTC

(the document has h, m and s superscripted, and the European (?) style
centered decimal point)

Warner

Re: The real problem with leap seconds

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Mark Calabretta <[EMAIL PROTECTED]> writes:
: On Wed 2006/01/11 10:47:25 -, Michael Deckers wrote
: in a message to: LEAPSECS@ROM.USNO.NAVY.MIL
:
: >   At some instant when TAI took a value in the positive leap second between
: >   2006-01-01 + 00 h + 00 min + 32 s and 2006-01-01 + 00 h + 00 min + 33 s
: >   (the exact instant is not clear from [ITU-R TF.460-6 2002]), DTAI jumped
: >   from 32 s to 33 s; thus, UTC is not a monotone increasing function of
: >   TAI either.
:
: Here in a topology-free way is what the axis labels of my graph look
: like during the said leap second insertion:
:
: UTC axisTAI axis DTAI
:2005/12/31 23:59:58 2006/01/01 00:00:3032
:2005/12/31 23:59:59 2006/01/01 00:00:3132
:2005/12/31 23:59:60 2006/01/01 00:00:3232
: 60.932.9  32
: 60.99   32.99 32
: 60.999...   32.999... 32
:2006/01/01 00:00:00 2006/01/01 00:00:3333
:2006/01/01 00:00:01 2006/01/01 00:00:3433
:
: The seconds keep step and the graph has no gaps, jumps or kinks.

Shouldn't the DTAI increment for the leap second?  That's the point of
the leap second...  Or is that only needed when you do the POSIX
time_t thing of repeating 59?

In computer science, regular things are easy.  Irregular ones are hard
and prone to errors.  The :60 kludge makes a regular 1<->1 mapping
function for time into a table driven nightmare :-(.

: The only difference between UTC and TAI is one of representation, like
: the current year which may be represented as 2006 or MMVI but it's still
: the same year.

The point isn't that one can construct a 'second labelling' that is
unambiguous, monotonic and increasing, but rather that when one
distills the UTC time down to a single number, you necessarily have
ambiguities and discontinuties in that function.  Of course there's an
implicit assumption that the conversion function is the same as
POSIX's time_t, otherwise you would be using TAI or a count of actual
seconds.  We call it PPSes or pulses at work to distinguish it.  We
label UTC time as having leap seconds swizzled in, and TAI time as
unswizzled.

Warner