Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Brooks Harris]

On 2020-02-06 7:28 AM, Seaman, Robert Lewis - (rseaman) wrote:


Hello all,

The fundamental answer / constraint to all questions of engineering, 
including temporal engineering, is funding. No bucks, no Buck Rogers. 
"Time" is a vast topic, pretty much as big as "space". Precision 
timekeeping topics are only somewhat smaller in practical terms since 
issues of anthropology and philosophy, etc., that may be far afield in 
other kinds of engineering, remain remarkably pertinent.


Funding falls into per-project and per-community responsibilities. 
Per-project, the engineering requirements related to timekeeping are 
remarkably diverse. You can read about the resulting timekeeping 
solution we implemented in support of our Near-Earth Asteroid (NEA) 
survey (https://arxiv.org/abs/1807.01370), but the precise mix of 
ingredients varies even between astronomical observatories. The short 
answer, per-project, is that organizations should plan and budget for 
timekeeping just as with any other critical infrastructure.



Rob,

I think that's a terrific and interesting article. Thanks.

-Brooks
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Warner Losh]

On Thu, Feb 6, 2020 at 7:17 AM Brooks Harris  wrote:

> On 2020-02-06 9:08 AM, Warner Losh wrote:
>
>
>
> On Thu, Feb 6, 2020, 3:22 AM Tom Van Baak  wrote:
>
>> Hi Hal,
>>
>> It's 2020. How on earth can NTP still not implement UTC correctly, in all
>> cases? Or is it a fundamental NTP design flaw?
>>
>
> Design flaw. NTP time stamps can't encode a leap second. It can therefore
> never really work in all cases. We are left with what hack do you want to
> do today.
>
> You can't fit 86401 pegs in 86400 holes. Something's got to go. No
> agreement on what goes.
>

Exactly. We have a number of different hacks that we use to keep the
problems under control, to discard potentially bad data, to filter bad data
that we can't discard, etc. It's a series of hacks that usually work well.
Not because it's well specified, but because different implementations have
programmed defensively to ensure that the current spec + unwritten policy
rules + some known past bugs are filtered so ntpd doesn't react when it's
likely a bad time. This will fail, of course, if we ever have a leap second
in March or September. It's working for the conditions we have today, which
is great, but it's not robust or resilient.

> The Z3801A issue doesn't sound like an NTP problem. This is a known legacy
>> Z3801A f/w or Motorola Oncore problem, yes? Maybe also affected by one or
>> even two GPS WNRO problems buy now?
>>
> Known past issues are likely future problems. 40 years in software has
> taught me that we do not always learn and apply the lessons of the past.
> Every 5-10 years we swap out the coders that learned them for newer,
> cheaper labor. Or there are new players in a niche that have more hubris
> than tribal knowledge. This guarantees bugs will repeat.
>
> Especially in the absence of clear specifications.
>

Yes. That's exactly what I meant by tribal knowledge. We could all tell
long stories about how we learned all the details because of that absence...

Warner


> Warner
>
>
> /tvb
>>
>> On 2/6/2020 1:41 AM, Hal Murray wrote:
>>
>> tvb said:
>>
>> There's no ambiguity. Those are just bugs. No software should depend on  more
>> than 1 month notice of a leap second and no software should be  fooled if the
>> notice is months or even years in advance.
>>
>> There are plenty of quirks in ntp code along that line.  The APIs don't have
>> an explicit when.  The NTP-Kernal API for leap-pending is leap-tonight.  You
>> have most of the next day to turn it off.  The leap-pending on the wire is
>> leap-at-the-end-of-this-month.
>>
>> I fixed a bug in the Z3801 driver by ignoring a leap pending unless it was
>> June or December.  It's a hack, but it gets the job done and the code wasn't
>> setup to ask it when the leap would happen.
>>
>>
>> tvb said:
>>
>> If you're writing a FAQ or best practices guide stay in touch. I have a
>> semi-technical leap second report in the works. UTC is actually very  simple;
>> but it's been complicated by untold levels of bad assumptions,  bad
>> execution, and bad prose.
>>
>> Are you going to say anything about POSIX?
>>
>>
>>
>>
>> ___
>> LEAPSECS mailing list
>> LEAPSECS@leapsecond.com
>> https://pairlist6.pair.net/mailman/listinfo/leapsecs
>>
>
>
> ___
> LEAPSECS mailing 
> listLEAPSECS@leapsecond.comhttps://pairlist6.pair.net/mailman/listinfo/leapsecs
>
>
> ___
> LEAPSECS mailing list
> LEAPSECS@leapsecond.com
> https://pairlist6.pair.net/mailman/listinfo/leapsecs
>
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Brooks Harris]

On 2020-02-06 9:08 AM, Warner Losh wrote:



On Thu, Feb 6, 2020, 3:22 AM Tom Van Baak > wrote:


Hi Hal,

It's 2020. How on earth can NTP still not implement UTC correctly,
in all cases? Or is it a fundamental NTP design flaw?


Design flaw. NTP time stamps can't encode a leap second. It can 
therefore never really work in all cases. We are left with what hack 
do you want to do today.
You can't fit 86401 pegs in 86400 holes. Something's got to go. No 
agreement on what goes.


The Z3801A issue doesn't sound like an NTP problem. This is a
known legacy Z3801A f/w or Motorola Oncore problem, yes? Maybe
also affected by one or even two GPS WNRO problems buy now?

Known past issues are likely future problems. 40 years in software has 
taught me that we do not always learn and apply the lessons of the 
past. Every 5-10 years we swap out the coders that learned them for 
newer, cheaper labor. Or there are new players in a niche that have 
more hubris than tribal knowledge. This guarantees bugs will repeat.

Especially in the absence of clear specifications.
-Brooks


Warner


/tvb


On 2/6/2020 1:41 AM, Hal Murray wrote:

tvb said:

There's no ambiguity. Those are just bugs. No software should depend on  
more
than 1 month notice of a leap second and no software should be  fooled if 
the
notice is months or even years in advance.

There are plenty of quirks in ntp code along that line.  The APIs don't have
an explicit when.  The NTP-Kernal API for leap-pending is leap-tonight.  You
have most of the next day to turn it off.  The leap-pending on the wire is
leap-at-the-end-of-this-month.

I fixed a bug in the Z3801 driver by ignoring a leap pending unless it was
June or December.  It's a hack, but it gets the job done and the code wasn't
setup to ask it when the leap would happen.


tvb said:

If you're writing a FAQ or best practices guide stay in touch. I have a
semi-technical leap second report in the works. UTC is actually very  
simple;
but it's been complicated by untold levels of bad assumptions,  bad
execution, and bad prose.

Are you going to say anything about POSIX?




___
LEAPSECS mailing list
LEAPSECS@leapsecond.com 
https://pairlist6.pair.net/mailman/listinfo/leapsecs



___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs


___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Warner Losh]

On Thu, Feb 6, 2020, 3:22 AM Tom Van Baak  wrote:

> Hi Hal,
>
> It's 2020. How on earth can NTP still not implement UTC correctly, in all
> cases? Or is it a fundamental NTP design flaw?
>

Design flaw. NTP time stamps can't encode a leap second. It can therefore
never really work in all cases. We are left with what hack do you want to
do today.

> The Z3801A issue doesn't sound like an NTP problem. This is a known legacy
> Z3801A f/w or Motorola Oncore problem, yes? Maybe also affected by one or
> even two GPS WNRO problems buy now?
>
Known past issues are likely future problems. 40 years in software has
taught me that we do not always learn and apply the lessons of the past.
Every 5-10 years we swap out the coders that learned them for newer,
cheaper labor. Or there are new players in a niche that have more hubris
than tribal knowledge. This guarantees bugs will repeat.

Warner


/tvb
>
> On 2/6/2020 1:41 AM, Hal Murray wrote:
>
> tvb said:
>
> There's no ambiguity. Those are just bugs. No software should depend on  more
> than 1 month notice of a leap second and no software should be  fooled if the
> notice is months or even years in advance.
>
> There are plenty of quirks in ntp code along that line.  The APIs don't have
> an explicit when.  The NTP-Kernal API for leap-pending is leap-tonight.  You
> have most of the next day to turn it off.  The leap-pending on the wire is
> leap-at-the-end-of-this-month.
>
> I fixed a bug in the Z3801 driver by ignoring a leap pending unless it was
> June or December.  It's a hack, but it gets the job done and the code wasn't
> setup to ask it when the leap would happen.
>
>
> tvb said:
>
> If you're writing a FAQ or best practices guide stay in touch. I have a
> semi-technical leap second report in the works. UTC is actually very  simple;
> but it's been complicated by untold levels of bad assumptions,  bad
> execution, and bad prose.
>
> Are you going to say anything about POSIX?
>
>
>
>
> ___
> LEAPSECS mailing list
> LEAPSECS@leapsecond.com
> https://pairlist6.pair.net/mailman/listinfo/leapsecs
>
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Martin Burnicki]

Seaman, Robert Lewis - (rseaman) wrote:
[...]
> We are extremely happy with the quality and support we have received
> from Meinberg, ...
Thanks, Rob, I'm very happy to hear this.

[...]
> (I have no financial interests in Meinberg or NTF.)

Disclaimer: ;-)

Even though I'm working for Meinberg, I'd like to point out that what I
usually write in this and similar mailing lists is also not due to
financial interests of any kind.

It's just my personal opinion, from my technical and practical point of
view.


Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Martin Burnicki]

Hal Murray wrote:
> 
> tvb said:
>> There's no ambiguity. Those are just bugs. No software should depend on  more
>> than 1 month notice of a leap second and no software should be  fooled if the
>> notice is months or even years in advance. 

Please keep in mind that e.g. GPS sends out leap second announcements
about 6 months in advance. This is not a problem if the receiver
firmware follows the specs, since the week number and day number are
specified. But, as we have seen, this can be a problem due to bugs in
the receiver firmware.

> There are plenty of quirks in ntp code along that line.  The APIs don't have 
> an explicit when.  The NTP-Kernal API for leap-pending is leap-tonight.  You 
> have most of the next day to turn it off.  The leap-pending on the wire is 
> leap-at-the-end-of-this-month.

I think this is some "best practice" that has evolved over many years.
For the kernel, the "end of this day" announcement is sufficient.

For the protocol, roughly 1 month is sufficient but short enough to
avoid ambiguous dates, i.e. at the end of which month.

If I remember correctly, the PTP specs even specify an announcement
interval of 12 hours only, the German DCF77 transmitter sends the
announcement only 1 hour in advance, and IRIG time codes (if they
support it at all, like IEEE1344 / C37.118) only 10 seconds or so in
advance.

So I wonder which spec should have been relevant for NTP/ntpd?

Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Martin Burnicki]

Tom Van Baak wrote:
> Hi Hal,
> 
> It's 2020. How on earth can NTP still not implement UTC correctly, in
> all cases? Or is it a fundamental NTP design flaw?

A program like ntpd (or ptpd, FWIW) has to rely on available sources of
information, and if those sources provide wrong information, this can
produce wrong results.

Of course there can be (and have been) bugs in the way ntpd handled leap
seconds. However, I'm not aware of any bufg in the current ntpd code.

Originally ntpd accepted a leap second from any upstream source that
announced it, and forwarded the announcement to its clients.

However, after it had happened that some GPS receiver models had sent a
faulty leap second announcement which the server forwarded to its
clients, the code in ntpd was modified in a way that a client accepts a
leap second announcement only if a majority of its upstream servers
provides the announcement.

If you implement a plausibility check that accepts leap second only at
the end of June or December, you avoid (at the first glance) that your
server's time is messed up if a faulty device sends an announcement at
the end of the wrong month.

But what if your GPS receiver sends a wrong leap second announcement at
the end of December, when there is no real leap second, or does *not*
send an announcement if a leap second has been scheduled?

If the GPS receiver which sends a wrong leap second announcement also
applies the leap second to its internal time, then the time sent by the
receiver is off by 1 s after the false leap second event.

What should a program do that has the GPS receiver as its sole time
source? Stop synchronizing to the receiver, or follow the time step
after some time if the 1 s offset persists?

What if the GPS satellites start sending data that cause the UTC
correction by GPS receivers to be 13 microseconds off? Is NTP to blame
if it follows the GPS receiver's time?

Which is the offset range that should be accepted if an "authoritative"
time source like a GPS receiver starts sending a different time?

So I wonder why NTP (the protocol) or ntpd (the implementation) are
blamed here. There's a lot of stuff in there which tries to sort out
errors coming from external time sources, but I doubt there can be an
implementation which is in no way affected by any potential error from
outside.


Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Seaman, Robert Lewis - (rseaman)]

Hello all,

The fundamental answer / constraint to all questions of engineering, including 
temporal engineering, is funding. No bucks, no Buck Rogers. “Time” is a vast 
topic, pretty much as big as “space”. Precision timekeeping topics are only 
somewhat smaller in practical terms since issues of anthropology and 
philosophy, etc., that may be far afield in other kinds of engineering, remain 
remarkably pertinent.

Funding falls into per-project and per-community responsibilities. Per-project, 
the engineering requirements related to timekeeping are remarkably diverse. You 
can read about the resulting timekeeping solution we implemented in support of 
our Near-Earth Asteroid (NEA) survey (https://arxiv.org/abs/1807.01370), but 
the precise mix of ingredients varies even between astronomical observatories. 
The short answer, per-project, is that organizations should plan and budget for 
timekeeping just as with any other critical infrastructure.

For example, we operate facilities in 6 buildings in 3 diverse locations, 
including two remote mountaintops. So we bought 3 commercial GNSS clocks and 
ran fiber between the pairs of buildings. We are extremely happy with the 
quality and support we have received from Meinberg, and I am happy to leave the 
NTP server-side updates to them, rolled into their occasional firmware updates. 
These updates have included significant feature improvements, including a nifty 
time synchronization monitor tool. My boss might call it the world’s most 
boring video game, but really that’s kind of the point. Many of the figures in 
the preprint above came directly from our Meinberg clocks. The cost of these 
clocks and related equipment is in the ballpark (baseball for “similar to”) for 
other classes of computing infrastructure. We don’t expect commodity computers 
to do hardware time-capture out of the box.

At the moment we are commissioning operations on another telescope on yet 
another mountaintop and will probably buy another GNSS clock to provide both 
hardware time capture and NTP services. We are also planning a project-wide OS 
upgrade for which chronyd is the default NTP option and are evaluating its 
performance, which is to say that project timekeeping operations costs continue.

Per-community, NTP is just one tool in a larger toolkit, but even so the 
panoply of NTP engineering requirements across many hundreds or thousands of 
projects is more diverse than POSIX or IETF have ever even attempted to 
capture, and funding has to compete with many other host-level and 
network-level standards. The Network Time Foundation (https://www.nwtime.org) 
supports the core NTP project, but NTF’s responsibilities are themselves 
broader than NTP. How many organizations support (financially) all the 
organizations like NTF that are responsible for standards, APIs, and protocols 
they rely on? Heck, how many of us read and respond to this 
professionally-relevant mailing list off-the-clock (as it were), as I am doing 
right now, rather than during work hours?

Many people reading this are their organization’s expert on timekeeping issues, 
and time is likely of significant importance to your organization if you are 
still reading to the end of this message. Ask yourself if your project, 
company, or community budgets for timekeeping infrastructure, operations, and 
standards proportional to their impact and risks, positive and negative, to 
your mission? I believe one reason Meinberg added its very useful syncmon tool 
was to address customers’ precise-timekeeping reporting requirements, 
especially for financial institutions, who attach equally precise estimates of 
its value in units of currency.

What’s time worth to you?

Rob Seaman
Catalina Sky Survey
Lunar and Planetary Laboratory

(I have no financial interests in Meinberg or NTF.)
--

Hi Hal,

It's 2020. How on earth can NTP still not implement UTC correctly, in all 
cases? Or is it a fundamental NTP design flaw?

The Z3801A issue doesn't sound like an NTP problem. This is a known legacy 
Z3801A f/w or Motorola Oncore problem, yes? Maybe also affected by one or even 
two GPS WNRO problems buy now?

/tvb

On 2/6/2020 1:41 AM, Hal Murray wrote:

tvb said:

There's no ambiguity. Those are just bugs. No software should depend on  more

than 1 month notice of a leap second and no software should be  fooled if the

notice is months or even years in advance.

There are plenty of quirks in ntp code along that line.  The APIs don't have

an explicit when.  The NTP-Kernal API for leap-pending is leap-tonight.  You

have most of the next day to turn it off.  The leap-pending on the wire is

leap-at-the-end-of-this-month.



I fixed a bug in the Z3801 driver by ignoring a leap pending unless it was

June or December.  It's a hack, but it gets the job done and the code wasn't

setup to ask it when the leap would happen.





tvb said:

If you're writing a FAQ or best practices guide stay in touch. I have a

semi-technical leap 

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Brooks Harris]

On 2020-02-06 5:22 AM, Tom Van Baak wrote:


Hi Hal,

It's 2020. How on earth can NTP still not implement UTC correctly, in 
all cases? Or is it a fundamental NTP design flaw?


The Z3801A issue doesn't sound like an NTP problem. This is a known 
legacy Z3801A f/w or Motorola Oncore problem, yes? Maybe also affected 
by one or even two GPS WNRO problems buy now?


If I understand this correctly Google Smear (and others, AWS, Bloomberg, 
etc) is a work-around to prevent possible system failure on 
leap-seconds. This is their primary concern, not leap-second accuracy. 
There are many potential ways a system or application might fail on 
leap-second depending on the implementations. At least one failure that 
was documented was what I'd call a 'stupid' bug where the code driving 
the output to the logging mechanism hung the kernel, setting off race 
conditions that took the whole data center down. It wasn't a bug in the 
handing of the leap-second itself but a bug in the reporting of the 
leap-second. Any given version of any OS might have some sort of bug in 
their leap-second handling or some related service.


Those data centers have millions of OS instances running on many 
different cpus (Intel, AMD, etc) on many different platforms (blades, 
etc), probably including various versions of Linux and Unix, Windows, 
Macintosh, and possibly main frame OSs. It is infeasible to upgrade all 
these at once only for leap-second or NTP updates because any OS version 
may have side effects that potentially upset the many critical 
applications running of each OS and it's subsystems. Potential side 
effects of the application behavior is a bigger risk than any 
leap-second behavior. The updates and upgrades must proceed very 
carefully in stages. Who knows how old some of these tiers might be. 
We've also heard complaints that many systems have not upgraded their 
NTP, with old systems still deployed. Its a very complex maintenance issue.


The potential practical problems related to leap-seconds implementation 
far outweigh the concern of accurate timekeeping. It will be many years 
before A) all OSs have identical and correct leap-second support 
(especially since the specs and common practices are vague and Windows 
is intentionally diverging from POSIX compliance), and B) the time-link 
systems, NTP, PTP, etc, also have identical behavior consistent with the 
OS's treatment.


It looks to me the smears will be with us for a very long time much to 
the frustration to those who wish computer timekeeping could be 
accurate. I don't see how consistency comes about without significant 
investment and this seems unlikely as the timing community debates the 
fate of the leap-second and no efforts are made to clarify the specs. 
The leap-second is evil but it looks like its with us for a very long 
time to come.


-Brooks


/tvb


On 2/6/2020 1:41 AM, Hal Murray wrote:

tvb said:

There's no ambiguity. Those are just bugs. No software should depend on  more
than 1 month notice of a leap second and no software should be  fooled if the
notice is months or even years in advance.

There are plenty of quirks in ntp code along that line.  The APIs don't have
an explicit when.  The NTP-Kernal API for leap-pending is leap-tonight.  You
have most of the next day to turn it off.  The leap-pending on the wire is
leap-at-the-end-of-this-month.

I fixed a bug in the Z3801 driver by ignoring a leap pending unless it was
June or December.  It's a hack, but it gets the job done and the code wasn't
setup to ask it when the leap would happen.


tvb said:

If you're writing a FAQ or best practices guide stay in touch. I have a
semi-technical leap second report in the works. UTC is actually very  simple;
but it's been complicated by untold levels of bad assumptions,  bad
execution, and bad prose.

Are you going to say anything about POSIX?






___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs


___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Tom Van Baak]

  
  
Hi Hal,
It's 2020. How on earth can NTP still not implement UTC
  correctly, in all cases? Or is it a fundamental NTP design flaw?

The Z3801A issue doesn't sound like an NTP problem. This is a
  known legacy Z3801A f/w or Motorola Oncore problem, yes? Maybe
  also affected by one or even two GPS WNRO problems buy now?

/tvb

On 2/6/2020 1:41 AM, Hal Murray wrote:


  
tvb said:

  
There's no ambiguity. Those are just bugs. No software should depend on  more
than 1 month notice of a leap second and no software should be  fooled if the
notice is months or even years in advance. 

  
  
There are plenty of quirks in ntp code along that line.  The APIs don't have 
an explicit when.  The NTP-Kernal API for leap-pending is leap-tonight.  You 
have most of the next day to turn it off.  The leap-pending on the wire is 
leap-at-the-end-of-this-month.

I fixed a bug in the Z3801 driver by ignoring a leap pending unless it was 
June or December.  It's a hack, but it gets the job done and the code wasn't 
setup to ask it when the leap would happen.


tvb said:

  
If you're writing a FAQ or best practices guide stay in touch. I have a
semi-technical leap second report in the works. UTC is actually very  simple;
but it's been complicated by untold levels of bad assumptions,  bad
execution, and bad prose. 

  
  
Are you going to say anything about POSIX?





  

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Hal Murray]

tvb said:
> There's no ambiguity. Those are just bugs. No software should depend on  more
> than 1 month notice of a leap second and no software should be  fooled if the
> notice is months or even years in advance. 

There are plenty of quirks in ntp code along that line.  The APIs don't have 
an explicit when.  The NTP-Kernal API for leap-pending is leap-tonight.  You 
have most of the next day to turn it off.  The leap-pending on the wire is 
leap-at-the-end-of-this-month.

I fixed a bug in the Z3801 driver by ignoring a leap pending unless it was 
June or December.  It's a hack, but it gets the job done and the code wasn't 
setup to ask it when the leap would happen.


tvb said:
> If you're writing a FAQ or best practices guide stay in touch. I have a
> semi-technical leap second report in the works. UTC is actually very  simple;
> but it's been complicated by untold levels of bad assumptions,  bad
> execution, and bad prose. 

Are you going to say anything about POSIX?


-- 
These are my opinions.  I hate spam.



___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Miroslav Lichvar]

On Wed, Feb 05, 2020 at 03:32:54PM -0800, Tom Van Baak wrote:
> Code should allow for a leap second event at the end of any month. The June
> / December thing is one of several guidelines for BIPM; it's not a rule that
> anyone writing UTC code should assume or depend on. Nor should code ever
> assume leap seconds are positive.

If the source can be trusted, yes, that would make sense. But in
reality, when working with not completely reliable information (due to
bugs in specifications, software, firmware, attacks, etc.), I think it
might be better to have the current June/December practice hardcoded.

For example, when a bug in an NTP implementation allowing leap seconds
to be inserted at any month caused them to sometimes get stuck and
repeat every month, the bogus leap seconds were spreading among
servers that didn't check what month it is.

How much energy is needed to slow down or speed up Earth enough for
UTC to require more than two leap seconds per year and how quickly can
it be released without destroying those that care about UTC following
UT1?

-- 
Miroslav Lichvar

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Steve Allen]

In the late 1960s German radio station DCF77 did not broadcast time
signals 24 hours a day.  Different agencies controlled different
signals at different times of day.

Deutsches Hydrographisches Institut broadcast signals based on
old UTC using their measurements of UT2 and the BIH offsets.

PTB broadcast signals of stepped atomic time based on cesium
seconds that became the SI second.

On Wed 2020-02-05T17:59:31+ Michael Deckers hath writ:
>The law on legal units in West Germany, from 1969-07-02, lists under
>the title "tasks of the Physikalisch-Technische Bundesanstalt"
>that the PTB has to publicize the procedures by which units without
>material prototype are realized, including the units of time and
>time scales, as well as the temperature unit and temperature scales.
>  hat die Verfahren bekanntzumachen, nach denen nicht verkörperte
>  Einheiten, einschließlich der Zeiteinheiten und der Zeitskalen
>  sowie der Temperatureinheit und Temperaturskalen, dargestellt
>  werden,")
>This can be taken to imply the task to disseminate a standard
>frequency (which they already did). But in my opinion it does not
>imply that UTC must have the same rate as the atomic time scales
>at the time -- the law even allows for several time scales.

What this means is that DHI was out of the legal time business, and
only PTB got to say what legal time was in Germany.  The head of PTB
said only SI seconds.

--
Steve Allen  WGS-84 (GPS)
UCO/Lick Observatory--ISB 260  Natural Sciences II, Room 165  Lat  +36.99855
1156 High Street   Voice: +1 831 459 3046 Lng -122.06015
Santa Cruz, CA 95064   https://www.ucolick.org/~sla/  Hgt +250 m
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Steve Allen]

On Wed 2020-02-05T15:32:54-0800 Tom Van Baak hath writ:
> I'm not sure it's fundamental to TAI that one must always, or only, use
> 24x60x60 radix notation. That's a useful convention in many cases, but at
> the h/w counter or s/w binary integer level radix notation is not required.

There were discussions about this in the early 1970s among members of
the CCDS and CIPM.  The USNO had been preferring a decimal count of
seconds, and this notion is pretty much what ended up in GPS.
Other members had other preferences, but in the end they made
no decision.

> Code should allow for a leap second event at the end of any month. The June
> / December thing is one of several guidelines for BIPM; it's not a rule that
> anyone writing UTC code should assume or depend on. Nor should code ever
> assume leap seconds are positive.

But the Soviets very much wanted not just any month, and they had
previously been using their own form of non-BIH UT2 for their own
different non-BIH rubber second UTC before 1972, and that looks like
why the CCIR directed BIH to issue the leap second 1972-12-31T23:59:60
early and exceed the 0.7 s limit.  Clearly the Soviets are no longer
in a position to insist again, but this whole arena seems well
described by Hector Barbossa telling Elizabeth Swan that the Pirate
Code is more like guidelines than actual rules.  Welcome abord the
Black Pearl.

--
Steve Allen  WGS-84 (GPS)
UCO/Lick Observatory--ISB 260  Natural Sciences II, Room 165  Lat  +36.99855
1156 High Street   Voice: +1 831 459 3046 Lng -122.06015
Santa Cruz, CA 95064   https://www.ucolick.org/~sla/  Hgt +250 m
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Tom Van Baak]

Warner,

> Yes. I wish it were clearer that TAI time is a regular radio 
expression of time.
> Here regular radix means that it every day has 24 hours and every 
hour 60 minutes

> and every minute has 60 seconds.

I'm not sure it's fundamental to TAI that one must always, or only, use 
24x60x60 radix notation. That's a useful convention in many cases, but 
at the h/w counter or s/w binary integer level radix notation is not 
required.


The key features of that timescale are its rate, its epoch, its 
continuity, and its lack of physical, astronomical, or political 
interference. A side effect is that this is a timescale that can be 
converted to/from broken-down date/time format in perpetuity using 
simple math and without using external tables.



> At the moment there are only two opportunities to consider, though the
> standard allows up to end of every month.

Code should allow for a leap second event at the end of any month. The 
June / December thing is one of several guidelines for BIPM; it's not a 
rule that anyone writing UTC code should assume or depend on. Nor should 
code ever assume leap seconds are positive.


> This ambiguity has lead to bugs when leaps were announced more than 3 
months
> in advance. I'd feel better if there were a commitment to these 
parameters for some

> number of years. But it's all just convention today.

There's no ambiguity. Those are just bugs. No software should depend on 
more than 1 month notice of a leap second and no software should be 
fooled if the notice is months or even years in advance.



> Have I forgotten any of the other details of leap seconds that are more
> tribal knowledge than rigorously specified?

If you're writing a FAQ or best practices guide stay in touch. I have a 
semi-technical leap second report in the works. UTC is actually very 
simple; but it's been complicated by untold levels of bad assumptions, 
bad execution, and bad prose.


/tvb

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Michael Deckers via LEAPSECS]

   On 2020-02-04 21:16, Steve Allen wrote:



The first time that the 4th meeting of the CCDS happened was in 1966,
but that meeting is not found in any official record.  The meeting
ended with a vote to recommend that the CGPM should adopt an SI second
based on cesium, but the circumstances of that vote were deemed so
abusive that the entire meeting was nullified.  That did not stop the
rush for an atomic second.  During the next year subsets of the CCDS
members gathered for discussions at other meetings.  When the second
4th meeting of the CCDS was held in 1967 they did recommend the cesium
second to the CGPM.



   From the standpoint of a physicist, the 1960 definition of the SI second
   (based on ET and Newcomb's tables for the Sun) was extremely 
impractical.

   With the wide availability of Cs clocks, the atomic second was much
   easier and more precisely reproducible than the SI second, so a
   redefinition of the second (or at least a practical unit, as with
   the recently abolished practical volt and ohm) was urgent.

   On the other hand, you are certainly right that the actions of the
   CCDS in 1967 appear strange: they propose to redefine the SI second,
   and then go on to propose that the BIH, IAU, IUGG, URSI, and CCIR
   study the problems arising from the new definition ("étudier les
   problèmes soulevés par l'application des décisions prises
   concernants la nouvelle définition de l'unité du temps").
   Apparently, it did not even occur to them that this is bad
   engineering.

   The proposal of the IAU GA13 in 1967 to introduce an (unsteered)
   international atomic time scale would have allowed to study the
   possible problems of a redefinition of the SI second before
   applying it.




Folks at the PTB took a different aim by introducing draft legislation
that the German government passed in 1969.  The law made it illegal
for the German government to broadcast anything other than SI seconds,
and it would become effective in 1970.  This seems to have pulled the
trigger on the CCIR process, for without some kind of quick action a
major nation would be broadcasting time signals using a different
scale than other nations.


   The law on legal units in West Germany, from 1969-07-02, lists under
   the title "tasks of the Physikalisch-Technische Bundesanstalt"
   that the PTB has to publicize the procedures by which units without
   material prototype are realized, including the units of time and
   time scales, as well as the temperature unit and temperature scales.
 (" hat die Verfahren bekanntzumachen, nach denen nicht verkörperte
    Einheiten, einschließlich der Zeiteinheiten und der Zeitskalen
    sowie der Temperatureinheit und Temperaturskalen, dargestellt
    werden,")
   This can be taken to imply the task to disseminate a standard
   frequency (which they already did). But in my opinion it does not
   imply that UTC must have the same rate as the atomic time scales
   at the time -- the law even allows for several time scales.

   I'll try to find out how the PTB was involved in this legislation
   as far as time in concerned. In Germany, federal law can only
   be proposed by members of parliament, and by federal and state
   governments; but the PTB was certainly heard during the legislative
   process.




In my home state of California the process that led to UTC with leap
seconds would have been illegal under the Brown Act that requires
public access to meetings.  But in the full context that is not the
most criminal aspect of the process that led to the 1970 CCIR
decision.



   Yes, and the ITU-R deliberations before the WRC in 2015 were not
   transparent either. Nevertheless, past decisions of the CCIR and
   the IAU have become accessible nowadays.

   Let's hope that the CIPM treats any future discussions about
   a redefinition of UTC in an open manner, and that it adheres
   to rational design and decision processes. The recent revision
   of the SI has been largely transparent and guided by good practices.

   Michael Deckers.

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Warner Losh]

On Tue, Feb 4, 2020 at 7:31 AM Brooks Harris  wrote:

> On 2020-02-04 2:54 AM, Warner Losh wrote:
> >
> > Have I forgotten any of the other details of leap seconds that are
> > more tribal knowledge than rigorously specified?
> >
> > Warner
> >
> >
> I think another unclear topic is when the value of DTAI ("The value of
> the difference TAI – UTC") is updated. This is not clearly stated
> anywhere and I've heard many discussions about it, including here on
> this list. The question is if DTAI is updated simultaneous with the
> leap-second of after the leap-second. Rec 460 does not state this. The
> answer is evident only by *implication* in Bulletin C and other
> publications. Bulletin C 52, for example, says " UTC TIME STEP on the
> 1st of January 2017", and shows:
> from 2015 July 1, 0h UTC, to 2017 January 1 0h UTC : UTC-TAI = - 36s
> from 2017 January 1, 0h UTC, until further notice : UTC-TAI = - 37s
>

That's a good point. Lord knows I've thought, in the past, that it
accumulated over the leap second. However, that's not entirely true because
the rules for the irregular radix that embodies UTC are weird and require
much careful thought that often runs against mathematical intuition. I even
had worked out the math to prove his once, only to discover that my 'proof'
was fatally flawed and the only proper way to increase the delta is by
1.0s, exactly, at the end of the leap second. This is all tricky stuff that
should be explicitly specified for a rigorous standard.


> Bulletin C does not say or indicate the value will change with the
> leap-second, but immediately after the leap-second, simultaneous with
> the following midnight. Other publications, such as
> Leap_Second_History.dat show this same relationship. But no prose says
> "the value *shall* update *after* the leap-second".
>

Yes. That's the only way I found to make the math work properly.


> Curiously Bulletin C does not call this "DTAI" as Rec 460 does, and
> gives the value in the opposite sign (Rec 460 calls it "TAI – UTC" and
> Bulletin C calls it "UTC-TAI"). This is an example of the inconsistency
> in the use of terms and nomenclature in the relevant documents at ITU-R,
> BIPM, and IERS, which does not help clarity.
>

That inconsistency doesn't help. There's also the issue that what time
geeks call "T1-T2" is computed by subtracting T2 from T1 because that's
what time counters do and that's a convention that carries over from the
metrological laboratory side of the time keeping business. I suspect that
some of this sloppiness is due to different groups having different
conventions in what they mean, and this disconnect wasn't noticed in the
drafting process.


> Find is a copy of Rec 460 I've modified at:
>
> http://edlmax.com/Common_Calendar/R-REC-TF.460-6-200202-I!!MSW-E_BEH_V1_2019-11_15.doc
>
> I've added minimal prose changes (red) and an addition to ANNEX 3,
> FIGURE 3 to clarify these two issues, 1) that TAI and UTC are to be in
> the YMDhms form, and 2) that DTAI updates after the leap-second. I feel
> if Rec 460 had these small changes it would make the use of leap-seconds
> much more clear and save many hours of discussion and angst. It took me
> a long time to have confidence in my understanding when I first
> encountered the leap-second some years ago.
>

I'll have to read through it. It sounds useful.

One bone to pick, though, is that TAI and UTC are both continuous time
scales, so applying the continuous bit to TAI  seems to imply UTC isn't
continuous. But I think it means continuous in the sense that TAI has been
maintained since that time, without interruption, and that no inference
about UTC should be made. UTC has had small jumps back and forth in the 60s
in the rubber leap second era (which were discontinuities), so perhaps the
wording is a nod to that, but an explicit nod would be better. So there's
two ways to view this and ambiguity is rarely a good thing...

Warner
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Steve Allen]

I have a growing bibliography with references that show the rush
toward atomic time was already underway at the 3rd CCDS meeting in
1963.  At that meeting the delegates restricted themselves merely to
ask CGPM12 to authorize an atomic SI second before CGPM13.

The first time that the 4th meeting of the CCDS happened was in 1966,
but that meeting is not found in any official record.  The meeting
ended with a vote to recommend that the CGPM should adopt an SI second
based on cesium, but the circumstances of that vote were deemed so
abusive that the entire meeting was nullified.  That did not stop the
rush for an atomic second.  During the next year subsets of the CCDS
members gathered for discussions at other meetings.  When the second
4th meeting of the CCDS was held in 1967 they did recommend the cesium
second to the CGPM.

With that the push had turned to getting all radio broadcast time
signals to use the cesium second.  Proponents of the cesium second
made demonstrably unrealistic presentations that minimized the
difference that would accumulate between Atomic Time and Universal
Time.  When the issues were presented to agencies they recognized the
tension between the need to broadcast atomically-regulated seconds and
the need to provide earth rotation time used by navigators.  The
recommendations from several agencies were to involve other agencies
in gathering together to study the problem.

In 1968 delegates heading toward the next meeting of CCIR Working
Party 7 gathered beforehand during a different meeting.  There could
be no proceedings from that gathering, but an offhand remark in a
later report indicates they agreed that constructing a calendar out of
SI seconds would require forging an international agreement.

Altogether the meeting results were painting a picture that involved
many agencies comprised of many people and many countries, and also
much time to figure out a reasonable way to broadcast time using SI
seconds.

Folks at the PTB took a different aim by introducing draft legislation
that the German government passed in 1969.  The law made it illegal
for the German government to broadcast anything other than SI seconds,
and it would become effective in 1970.  This seems to have pulled the
trigger on the CCIR process, for without some kind of quick action a
major nation would be broadcasting time signals using a different
scale than other nations.

Contributions and responses to questions from around 1969 make it
clear that some technical agencies were unaware of the political
constraints, and some bureaucratic agencies were unaware of the
technical constraints.  Several memoirs make it likely that H.M.
Smith performed Herculean efforts to contain this dumpster fire of
spreading incomplete information and forge a final agreement which was
minimally objectionable to all parties.

In my home state of California the process that led to UTC with leap
seconds would have been illegal under the Brown Act that requires
public access to meetings.  But in the full context that is not the
most criminal aspect of the process that led to the 1970 CCIR
decision.

After WW2 ended the ITU held a months-long meeting at Atlantic City in
1947 to discuss how recent advancements in radio technology would
affect international agreements for use of radio broadcasts.  The 1947
proceedings include reports from many national agencies on the history
and technology of time determination and dissemination.  The result is
an encyclopedic treatise on how they determined time, how they
broadcast time, what were the technological limitations, and what they
thought was important.  It also produced a synopsis of all those
contributions which compared and contrasted the different methods,
indicated what were the goals of the CCIR, and described why the CCIR
recommendations were choosing particular methods already in use which
were generally deemed to be best practices.

The CCIR process that led to the 1970 Rec 460 produced nothing like
those 1947 proceedings.  That is the criminal difference between CCIR
1947 proceedings and recommendations for broadcast time signals and
the 1970 recommendations.  The process that gave us leap seconds was
not merely based in fear, it was also based on maintaining ignorance.

The closest thing to an admission that there might be reasons not to
use UTC as specified by CCIR was Recommendation 485.  That allowed for
dates to be specified using either UTC or TAI, but ITU-R suppressed
TF.485 in 1997, just before the push to remove leaps from UTC began.

Based on all the history my impression is that it is not possible to
obtain any official improvements in the documentation for the rules
governing UTC nor for the way that authoritative information about
leap seconds and DUT1 is disseminated.  Any such improvements risk
discussions that might delve into the origins and purpose of UTC.
Fear means that ignorance must be maintained for the sake of hegemony.

--
Steve Allen  

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Martin Burnicki]

Tony Finch wrote:
> The IERS Bulletins C state a value of UTC-TAI "until further notice".

As I understand this, it just means that yet it's not known if and when
another leap second will have to be scheduled.

> However the machine-readable files from IERS and NIST give an expiry date
> of a few days less than 6 months after the announced (lack of) leap
> second, or a bit more than 11 months after the latest Bulletin C.
> Is this expiry date reliable or just advisory? History suggests it's
> reliable, but the standards do not.

In my opinion this expiration date is very important for those who
evaluate the file, e.g. ntpd.

If ntpd had a leap second file which doesn't contain a leap second after
2016, it didn't know it there really was no other leap second, or if
only the file was too old and hadn't been updated.

In the latter case ntpd wouldn't even know if the TAI/UTC offset derived
from the available leap second file is still valid, or if another leap
second has been inserted in the mean time, and the TAI/UTC offset has
changed.

With the expiration date ntpd can be sure there was no other leap
second, and the TAI/UTC offset is still valid, as long as the current
time is before the expiration date, and the file can be ignored after it
has expired.

If the bulletin C is published once every 6 months, and you have the
bulletin from January, you know for sure if there will be a leap second
at the end of June, or not.

However, only the next bulletin C from next July will tell you if there
will be leap second at the end of December. So in January you can be
sure the information you have won't change until shortly before the end
of December, which is about 11 months.

Even after the next bulletin C and leap second file have been published
in July, things won't change at least up to the expiration date of the
previous file. So you have still about 5 months to deploy the leap
second file from July to your ntpd instances, before anything expire.

One way to propagate a new leap second file is to

- add the new file to the TZ DB some time after tit has been published

- the new file appears in the TZ DB when a new TZ DB versio is released

- after this it appears at the IANA web page, and eventually in software
update packages for your favorite operating system

In summary, it can take weeks or even months until the file is available
everywhere.

So IMO it's very practical that the expiration date is there, and that
you have about 5 months to deploy a new version.


Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Michael Deckers via LEAPSECS]

   On 2020-02-04 13:44, Tony Finch wrote:



The IERS Bulletins C state a value of UTC-TAI "until further notice".

However the machine-readable files from IERS and NIST give an expiry date
of a few days less than 6 months after the announced (lack of) leap
second, or a bit more than 11 months after the latest Bulletin C.
Is this expiry date reliable or just advisory? History suggests it's
reliable, but the standards do not.

It's unclear to me what governs the frequency of announcements or their
validity period, i.e. where are current practices documented? what is the
process for changing them? how will we know if a change is planned? and so
on. This is all about how much we can assume that the IERS will continue
to operate leap seconds as they have for nearly 50 years, or whether they
will make use of the much weaker guarantees given by TF.460, or (wishful
thinking) whether they can schedule leap seconds further in the future.




   The IERS (and BIH) policy to use only the primary
   choices for the insertions of leap seconds is only
   guaranteed in the text of Bulletin C -- if LOD
   increases sufficiently, that text will have to
   change.

   There is a similar situation for Bulletins D -- each
   of them announces when the next one is expected to
   be issued. But even nowadays these predictions of
   UT1 - UTC are not very reliable, and they often err
   on the "wrong" side (DUT1 changes earlier than
   predicted).

   For instance, Bulletins D139, D134, and D129 each
   came earlier than predicted by the preceding Bulletin D;
   Bulletin D129 (of 2016-04-15) was even significantly earlier
   (45 d) than predicted by Bulletin D128 (of 2016-02-19).

   Michael Deckers.

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Brooks Harris]

On 2020-02-04 2:54 AM, Warner Losh wrote:


Have I forgotten any of the other details of leap seconds that are 
more tribal knowledge than rigorously specified?


Warner


I think another unclear topic is when the value of DTAI ("The value of 
the difference TAI – UTC") is updated. This is not clearly stated 
anywhere and I've heard many discussions about it, including here on 
this list. The question is if DTAI is updated simultaneous with the 
leap-second of after the leap-second. Rec 460 does not state this. The 
answer is evident only by *implication* in Bulletin C and other 
publications. Bulletin C 52, for example, says " UTC TIME STEP on the 
1st of January 2017", and shows:

from 2015 July 1, 0h UTC, to 2017 January 1 0h UTC : UTC-TAI = - 36s
from 2017 January 1, 0h UTC, until further notice : UTC-TAI = - 37s

Bulletin C does not say or indicate the value will change with the 
leap-second, but immediately after the leap-second, simultaneous with 
the following midnight. Other publications, such as 
Leap_Second_History.dat show this same relationship. But no prose says 
"the value *shall* update *after* the leap-second".


Curiously Bulletin C does not call this "DTAI" as Rec 460 does, and 
gives the value in the opposite sign (Rec 460 calls it "TAI – UTC" and 
Bulletin C calls it "UTC-TAI"). This is an example of the inconsistency 
in the use of terms and nomenclature in the relevant documents at ITU-R, 
BIPM, and IERS, which does not help clarity.


Find is a copy of Rec 460 I've modified at:
http://edlmax.com/Common_Calendar/R-REC-TF.460-6-200202-I!!MSW-E_BEH_V1_2019-11_15.doc

I've added minimal prose changes (red) and an addition to ANNEX 3, 
FIGURE 3 to clarify these two issues, 1) that TAI and UTC are to be in 
the YMDhms form, and 2) that DTAI updates after the leap-second. I feel 
if Rec 460 had these small changes it would make the use of leap-seconds 
much more clear and save many hours of discussion and angst. It took me 
a long time to have confidence in my understanding when I first 
encountered the leap-second some years ago.


-Brooks
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Tony Finch]

Warner Losh  wrote:
>
> Have I forgotten any of the other details of leap seconds that are more
> tribal knowledge than rigorously specified?

The IERS Bulletins C state a value of UTC-TAI "until further notice".

However the machine-readable files from IERS and NIST give an expiry date
of a few days less than 6 months after the announced (lack of) leap
second, or a bit more than 11 months after the latest Bulletin C.
Is this expiry date reliable or just advisory? History suggests it's
reliable, but the standards do not.

It's unclear to me what governs the frequency of announcements or their
validity period, i.e. where are current practices documented? what is the
process for changing them? how will we know if a change is planned? and so
on. This is all about how much we can assume that the IERS will continue
to operate leap seconds as they have for nearly 50 years, or whether they
will make use of the much weaker guarantees given by TF.460, or (wishful
thinking) whether they can schedule leap seconds further in the future.

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
Cape Wrath to Rattray Head including Orkney: Northwest 3 to 5, occasionally 6
at first near Rattray Head, then backing southwest 4 to 6. Moderate or rough,
becoming slight in Moray Firth. Showers later in north. Good.
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Warner Losh]

On Mon, Feb 3, 2020, 6:01 PM Brooks Harris  wrote:

> On 2020-02-03 10:37 AM, Michael Deckers via LEAPSECS wrote:
> >
> >
> >The 1970 report also contains the proposal that the CIPM should be
> >responsible for the definition of UTC, and 49 years later, the CGPM
> >in 2019 seems to have taken on that task with the resolution
> >[https://www.bipm.org/en/CGPM/db/26/2/] which notably has no
> >requirement that |UTC - UT1| be bounded.
> >
> >
> I'd like to point out something about the language used in this CGPM
> reference I feel contributes to confusion in some forums.
>
> It says "
> - International Atomic Time (TAI) is a continuous time scale produced by
> the BIPM based on the best realizations of the SI second. ...
> - Coordinated Universal Time (UTC) is a time scale produced by the BIPM
> with the same rate as TAI, but differing from TAI only by an integral
> number of seconds..."
>
> This seems to imply that TAI is an uninterrupted incrementing count of
> SI seconds, and that there is some corresponding form of UTC also
> delineated in uninterrupted incrementing count of SI seconds. This is
> misleading.
>
> ITU-R Recommendation 460 says:
> "TAI ... It is in the form of a continuous scale, e.g. in days, hours,
> minutes and seconds from the origin 1 January 1958"
> and
> "UTC ... corresponds exactly in rate with TAI but differs from it by an
> integer number of seconds."
>
> What is meant here, as most on this list will understand, is that both
> TAI and UTC are to be represented in YMDhms form and that UTC "differs"
> from TAI "by an integer number of seconds" *as represented in YMDhms
> form*. There is no meaningful UTC as an uninterrupted incrementing count
> of SI seconds.
>
> Recommendation 460 itself is slightly misleading in this regard. I think
> it would be better if the UTC statement made it clear that UTC is also
> in the form of YMDhms with its "23:59:60" leap-seconds. This can be
> surmised from other parts of the document, but I don't think its clear
> without careful study.
>
> I've heard many discussions where "uninterrupted incrementing count of
> SI seconds" is confused with the YMDhms representations. I wish the
> BIPM, IERS, CGPM, and ITU-R documents were more clear and used more
> consistent language and descriptions.
>

Yes. I wish it were clearer that TAI time is a regular radio expression of
time. Here regular radix means that it every day has 24 hours and every
hour 60 minutes and every minute has 60 seconds.

UTC has an irregular radix where minutes may have 59 or 61 seconds that are
published in table form or as new rows in the table. All past times have a
set and knowable table. Future times cannot be know past 1 second before
the leap second opportunity after the next announced leap second
opportunity.

At the moment there are only two opportunities to consider, though the
standard allows up to end of every month. This ambiguity has lead to bugs
when leaps were announced more than 3 months in advance. I'd feel better if
there were a commitment to these parameters for some number of years. But
it's all just convention today.

Have I forgotten any of the other details of leap seconds that are more
tribal knowledge than rigorously specified?

Warner


-Brooks
>
>
>
> ___
> LEAPSECS mailing list
> LEAPSECS@leapsecond.com
> https://pairlist6.pair.net/mailman/listinfo/leapsecs
>
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Brooks Harris]

On 2020-02-03 10:37 AM, Michael Deckers via LEAPSECS wrote:



   The 1970 report also contains the proposal that the CIPM should be
   responsible for the definition of UTC, and 49 years later, the CGPM
   in 2019 seems to have taken on that task with the resolution
   [https://www.bipm.org/en/CGPM/db/26/2/] which notably has no
   requirement that |UTC - UT1| be bounded.


I'd like to point out something about the language used in this CGPM 
reference I feel contributes to confusion in some forums.


It says "
- International Atomic Time (TAI) is a continuous time scale produced by 
the BIPM based on the best realizations of the SI second. ...
- Coordinated Universal Time (UTC) is a time scale produced by the BIPM 
with the same rate as TAI, but differing from TAI only by an integral 
number of seconds..."


This seems to imply that TAI is an uninterrupted incrementing count of 
SI seconds, and that there is some corresponding form of UTC also 
delineated in uninterrupted incrementing count of SI seconds. This is 
misleading.


ITU-R Recommendation 460 says:
"TAI ... It is in the form of a continuous scale, e.g. in days, hours, 
minutes and seconds from the origin 1 January 1958"

and
"UTC ... corresponds exactly in rate with TAI but differs from it by an 
integer number of seconds."


What is meant here, as most on this list will understand, is that both 
TAI and UTC are to be represented in YMDhms form and that UTC "differs" 
from TAI "by an integer number of seconds" *as represented in YMDhms 
form*. There is no meaningful UTC as an uninterrupted incrementing count 
of SI seconds.


Recommendation 460 itself is slightly misleading in this regard. I think 
it would be better if the UTC statement made it clear that UTC is also 
in the form of YMDhms with its "23:59:60" leap-seconds. This can be 
surmised from other parts of the document, but I don't think its clear 
without careful study.


I've heard many discussions where "uninterrupted incrementing count of 
SI seconds" is confused with the YMDhms representations. I wish the 
BIPM, IERS, CGPM, and ITU-R documents were more clear and used more 
consistent language and descriptions.


-Brooks



___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Michael Deckers via LEAPSECS]

On 2020-02-02 22:30, Steve Allen wrote:

On Sun 2020-02-02T17:59:20+ Michael Deckers hath writ:

The maximum deviation |UTC - UT1| <= 0.9 s as stipulated in
1974 by CCIR Rec. 460-1 has never been violated until now.

That violates the agreement that the difference between
UTC and UT1 would be encoded as part of the time broadcasts.



   Actually, the difference |UTC - UT1| has always been < 0.8 s
   except around 1973-01-01.

   And DUT1 has assumed the value 0.8 s only once, for a few days
   on and after 1994-07-01, as specified in Bulletin D46 (online at
   [https://datacenter.iers.org/data/17/bulletind-046.txt]).

   That Bulletin is remarkable in several respects: it describes
   two switches of DUT1, not just one, and it was issued on 1994-06-21,
   only 10 days before the first switch (rather than a month before
   it, as was requested -- from the BIH -- in CCIR Rec 460-4 of 1986).




In one case it was broken specifically because a high official at CCIR
conceded to a high official from USSR and directed the BIH to violate
the wording of the existing agreement.

Do you mean the only violation of applicable CCIR rules, the
introduction of a leap second into UTC at 1973-01-01?

Right.  Sadler covers this in his memoir and in several contemporary
publications.

Delving into this reveals more of the fear in the process.

Several memoirs show that the principals involved with the creation of
UTC with leaps were very concerned that the change of broadcast time
signals might cause havoc with ships using celestial navigation.
Reading through those shows palpable relief when they managed to evoke
from the Maritime Safety Committee of the IMCO a statement that Rec.
460 would not cause difficulties with navigation predicated on the
expectation that governments whose radio broadcasts used new UTC would
issue notices about the change of their broadcasts.  That meant that
the Time Lords did not have their arses on the line if a ships might
collide as a result of the new system.  With the maximum difference of
0.7 s that could be encoded in the radio broadcasts not being able to
handle the 0.9 s difference that put their arses back on the line.

Other concern was expressed that exceeding the 0.7 limit might be
blamed on the BIH and might trigger governmental review of the
operation and funding of the BIH.  At that time about 80% of the funds
for BIH were coming from Observatoire de Paris as slush from their
allotment from the French government.  That was hardly an
"international" arrangement, but BIH had only just been handed the
responsibility for maintaining TAI specifically because any other
arrangement would have required effectively duplicating the
expertise and hardware of the BIH and finding a way to fund that.

Prompting governments or journalists to open an investigation into the
process of writing an international "technical" specification that was
violated in less than two years was not a welcome notion.




   Very interesting, thanks for these details!

   Concerning the technical expertise of the CCIR with time scales: one
   of the early proposals of the CCIR has been a "stepped atomic time"
   with steps of 1 s and maximal difference of 0.5 s from UT2 (as mentioned
   in the 1970 report of commission 31 available via your web site on
[https://ui.adsabs.harvard.edu/link_gateway/1970IAUTA..14..343Z/ADS_PDF])
   -- apparently they had not consulted any astronomer, even though they
   used to "request" many actions from the BIH in their specifications of
   time scales.

   The 1970 report also contains the proposal that the CIPM should be
   responsible for the definition of UTC, and 49 years later, the CGPM
   in 2019 seems to have taken on that task with the resolution
   [https://www.bipm.org/en/CGPM/db/26/2/] which notably has no
   requirement that |UTC - UT1| be bounded.

   Michael Deckers.


___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Steve Allen]

On Sun 2020-02-02T17:59:20+ Michael Deckers hath writ:
> The maximum deviation |UTC - UT1| <= 0.9 s as stipulated in
> 1974 by CCIR Rec. 460-1 has never been violated until now.

That violates the agreement that the difference between
UTC and UT1 would be encoded as part of the time broadcasts.

> > In one case it was broken specifically because a high official at CCIR
> > conceded to a high official from USSR and directed the BIH to violate
> > the wording of the existing agreement.
>
> Do you mean the only violation of applicable CCIR rules, the
> introduction of a leap second into UTC at 1973-01-01?

Right.  Sadler covers this in his memoir and in several contemporary
publications.

Delving into this reveals more of the fear in the process.

Several memoirs show that the principals involved with the creation of
UTC with leaps were very concerned that the change of broadcast time
signals might cause havoc with ships using celestial navigation.
Reading through those shows palpable relief when they managed to evoke
from the Maritime Safety Committee of the IMCO a statement that Rec.
460 would not cause difficulties with navigation predicated on the
expectation that governments whose radio broadcasts used new UTC would
issue notices about the change of their broadcasts.  That meant that
the Time Lords did not have their arses on the line if a ships might
collide as a result of the new system.  With the maximum difference of
0.7 s that could be encoded in the radio broadcasts not being able to
handle the 0.9 s difference that put their arses back on the line.

Other concern was expressed that exceeding the 0.7 limit might be
blamed on the BIH and might trigger governmental review of the
operation and funding of the BIH.  At that time about 80% of the funds
for BIH were coming from Observatoire de Paris as slush from their
allotment from the French government.  That was hardly an
"international" arrangement, but BIH had only just been handed the
responsibility for maintaining TAI specifically because any other
arrangement would have required effectively duplicating the
expertise and hardware of the BIH and finding a way to fund that.

Prompting governments or journalists to open an investigation into the
process of writing an international "technical" specification that was
violated in less than two years was not a welcome notion.

--
Steve Allen  WGS-84 (GPS)
UCO/Lick Observatory--ISB 260  Natural Sciences II, Room 165  Lat  +36.99855
1156 High Street   Voice: +1 831 459 3046 Lng -122.06015
Santa Cruz, CA 95064   https://www.ucolick.org/~sla/  Hgt +250 m
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Michael Deckers via LEAPSECS]

   On 2020-02-01 23:59, Steve Allen wrote:


In every instance where a document
specified a maximum deviation that agreement was later violated.



   The maximum deviation |UTC - UT1| <= 0.9 s as stipulated in
   1974 by CCIR Rec. 460-1 has never been violated until now.


In one case it was broken specifically because a high official at CCIR
conceded to a high official from USSR and directed the BIH to violate
fthe wording of the existing agreement.


   Do you mean the only violation of applicable CCIR rules, the
   introduction of a leap second into UTC at 1973-01-01?

   If so -- this was the choice of using either the date 1973-01-01
   for the insertion of the leap second, or a later date before
   1973-07-01.
  This is evident because at the time, the mean excess length
  of day LOD = d(TAI - UT1)/d(UT1) was observed to be >= 3 ms/d,
  which is more than 0.5 s per 6 months.

   Hence the choice was to either stick with the bound 0.7 s for
   |UT1 - UTC| as required by CCIR Report 517 of 1971, or else stick
   with the primary choices for the possible dates of the insertion
   of leap seconds.

   Apparently, the "high official from USSR" must have preferred
   the latter.

   Michael Deckers.

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Steve Allen]

On Sun 2020-02-02T00:33:08+0100 Warner Losh hath writ:
>  It's the fact that things like filesystems
> specify an elapsed time since an epoch in a time scale without leap
> seconds. Every FAT or NTFS disk around has a time like this.

Beginning 2018-06-01 the value of Microsoft Windows FILETIME ceases to
be seconds of UT since 1600 and begins to be (TAI - 37 s).
Microsoft has decided to become the authoritative point of
distribution for the leap second information that no international
agency has ever been tasked to be.

> Replacing steering systems
> for telescopes is likely somewhat less than that.

I have already written up how it is not an insurmountable problem for
telescope systems, but digging deep into the literature from around
1970 even further invalidates any notion that leap seconds are for
astronomers.

In the many meetings and recommendations there were several instances
where the participants and wording recognized the requests from
astronomers to preserve UT.  In every instance where a document
specified a maximum deviation that agreement was later violated.  In
one case it was broken specifically because a high official at CCIR
conceded to a high official from USSR and directed the BIH to violate
the wording of the existing agreement.

The leap seconds were included in the CCIR recommendation not because
of anything any astronomer said, but because a few of the participants
in the CCIR process understood that they did not have the legal
authority to cause all nations to change calendar days from mean solar
days to atomic days.

--
Steve Allen  WGS-84 (GPS)
UCO/Lick Observatory--ISB 260  Natural Sciences II, Room 165  Lat  +36.99855
1156 High Street   Voice: +1 831 459 3046 Lng -122.06015
Santa Cruz, CA 95064   https://www.ucolick.org/~sla/  Hgt +250 m
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Warner Losh]

On Sat, Feb 1, 2020 at 10:57 PM Seaman, Robert Lewis - (rseaman) <
rsea...@email.arizona.edu> wrote:

> Tried to send this a few days ago, but it never showed up on the list.
> Steve has provided gritty details since.
>
>
>
> Since roughly the second world war, the distinction between time-of-day
> and interval-time has become increasingly clear. But the history of this
> distinction goes back at least as far as Galileo. UTC was an attempt to
> serve both Universal Time (time-of-day) and Atomic Time (interval timing)
> using a single standard.
>

UTC is one solution, but it's :60 second is a wart it invented in 1972 with
no consultation with the rest of the world.  To say it's universal really
misses the point that it's just a good approximation of a Universal time
that's handy for some group of people, but difficult for others. It was
changed in the lifetime of many on this list. It could change again to be a
better fit with changing times. There should be nothing sacrosanct about it
as it was invented out of whole cloth 48  years ago. It was unprecedented
at the time, and for the first 35 or so years the difficulties of
implementing it for computers were unknown because computers didn't keep
time well enough for it to matter. Now they do.


> Folks in this thread are focusing on the difficulties of this scheme, but
> UTC has had significant success as well. Nobody would be trying to build on
> top of it if this were not true. One reason UTC has succeeded is because
> of  its design implementing Universal Time, not in spite of it.
>

I'd argue one reason it's been successful is that it papers over the issues
with seconds changing length to get universal time. I think the important
part of UTC is thee C part not the UT part. So long as everybody agrees on
what time it is, and that time is close to the Solar time, we're good. We
have lots of timezones, especially in Russia, that are more than an hour
different than mean solar time. This suggests that time of day relative to
the sun has a tolerance for almost everybody of around an hour, which
suggests a UTC++ plus timezones that can keep things within an hour is
fine, which leaves a lot of wiggle room, though there will be winners and
losers to any change.


> It seems bizarre to have to state that it would be wise to fully analyze
> civil timekeeping engineering requirements before making any changes. Some
> here have participated in a variety of meetings and discussions with this
> goal, but I am unaware of any external funding for such activities. Other
> communities have invested much greater time (so to speak) and money
> regarding similarly ubiquitous, yet esoteric, standards and protocols.
>

Part of that analysis should necessarily include the impact on computing as
well

If the precision timekeeping community had adopted the proposal from the
> 2003 Torino symposium to define the new time scale called TI, we would have
> had 17 years to cover the Earth like locusts. Instead the intervening
> period has been squandered trying unnecessarily to undermine UTC. Recipe
> for success:
>

>
>1. Define a new time scale and leave UTC alone for future
>compatibility. Your own systems engineering requirements likely include
>time-of-day.
>
> Except nobody will use this. We already have a dozen different time scales
without leap seconds. Nobody uses them outside a niche. Everybody inside
that niche translates the niche time scale to UTC for displaying to users.


And people have tried running kernel time in TAI time.  It didn't work.
Even when  you try  to be compatible, there's dozens of problems that make
it hard because it's not just POSIX time_t that's at issue (though within
the kernel it's bad enough). It's the fact that things like filesystems
specify an elapsed time since an epoch in a time scale without leap
seconds. Every FAT or NTFS disk around has a time like this. So conversion
of kernel TAI times to UTC or these other time scales requires perfect
knowledge of leap seconds, and conversions more than 6 months in the future
are ambiguous at best.


>
>1.
>2. There are two kinds of time and timekeeping. All successful systems
>engineering will start from this simple fact.
>
>
> Whatever POSIX does with leap seconds should be in a larger and more
> coherent global concept of timekeeping.
>

POSIX can't do leap seconds. That ship has already sailed. It's more than
just POSIX as it is baked into almost all storage formats today. Even if we
could fix that issue, there are significant operational issues with ticking
in TAI since a full leap table is required to convert TAI times to UTC.

If it were just what is returned by the kernel during a leap second, that
might be solvable (smearing NTP shows that's possible). But given the
mutually exclusive demands on timestamps and their arithmetic properties,
it's impossible to fix without significant breakage. A cost/benefit
analysis likely would conclude the cost to, say, eliminate leap