Re: [LEAPSECS] Leap second to be introduced at midnight UTC December 31 this year

[Steve Summit]

Hal Murray:
>> Has anybody done any serious thinking about fixing the POSIX problem?

I would love to, love to, love to fix it.  But the solution won't
be to redefine Posix time_t, or abandon it.

Stephen Colebourne:
> The basic observation that I have had over many years is that beyond a
> few experts, very few care about leap seconds.

Right.  Which is why Posix time_t, despite its (to us, anyway)
gaping flaw, is still so ubiquitous.

I think about Posix time_t in three ways:

1. It's utterly broken, in that it claims to be UTC, but fails to
   handle the one facet that distinguishes UTC from other timescales.

2. It's wildly useful, because it's perfectly continuous, and
   allows you to seamlessly integrate clock and calendar handling
   at the second, day, and year scales.

3. Its definition can't be changed, because its published
   definition absolutely allows (and even encourages) any random
   scrap of user code out there to use its own handcrafted logic
   to convert back and forth between time_t and calendar time,
   meaning that we can't just "fix" the Posix leap-second problem
   by redefining time_t and then rewriting library functions like
   localtime() to use our new, improved definition.  Posix time_t
   is here to stay.

Stephen Colebourne:
> My hope when I defined the Java time-scale was that eventually the
> OS would provide it, or something very similar.

Indeed.  I've been exploring a reasonably straightforward
scheme to have the OS kernel keep true UTC internally, but
report smeared Posix time_t by default, but with new way(s)
to report and use true UTC (that is, including an unambiguous
representation of 23:59:60Z) for those few applications that
care.  This is essentially the scheme that Markus Kuhn described
at https://www.cl.cam.ac.uk/~mgk25/posix-clocks.html almost 20
years ago.

Hal Murray again:
> I assume the first step would be to have the kernel keep time in a non 
> leaping time scale.  Is UT1 the obvious choice?

I don't think so.  Civil time is everywhere, even (typically) in
the OS kernel.  UTC and Posix time_t are everywhere, too.  So my
approach is to bite the bullet and implement a "leaping" time
scale in the kernel, i.e. true UTC.  One way to do so is to count
*days* since 1970-01-01 and then seconds -- either 86399, 86400,
or 86401 of them -- within each day.  This is both simple to
implement and simple to convert back and forth between other
representations.

> Would we need a version of ntp that used that time scale?

NTP is well-behaved across leap seconds and can be thought of
as keeping true UTC already, in that its explicit 'insert' and
'delete' bits let you unambiguously interpret its otherwise
"continuous" (Posix-like) time scale.  So while improvements to
NTP are possible, such as having it be a distribution channel
for TAI-UTC data (current and possibly also historical), I don't
think it *has* to be changed.  (Indeed one of my goals is having
a leapsecond-aware OS that works seamlessly with stock NTP feeds.)

> I assume the file dates could be fixed with a flag day and one
> pass over the whole file system.
>
> I think the hard part would be time-stamps in network protocols
> or data bases.  Consider tar files or rsync.

Trying to rewrite every time_t-using protocol and file format,
and adjusting the timestamp on every file in every filesystem
everywhere, isn't just hard, it's impossible!

And in any case, most of them don't need fixing.  For example,
I believe file timestamps can continue to use Posix time_t.
There's a consequence that on leap-second days they'd be up to
one second off with respect to true UTC -- but since they have
only one-second granularity, they're always up to one second off
anyway.

As for network protocols, most of those don't need to (and won't)
change, either, can continue to use the Posix timestamps they use
today, either explicitly finessing leapsecond discontinuities
(if necessary) as suggested by RFC 7164, or simply not noticing
them (and having their performance improved, and their glitches
removed, by running on a true-UTC kernel that delivers smeared,
non-discontinuous Posix timestamps across leap seconds).
Those -- likely few -- protocols that truly care and that want to
introduce a true-UTC timestamp can begin to do so, once there's a
defined way to fetch true UTC from the kernel.

I think the most important thing to fix is the ill-defined timing
anomalies we're putting up with once every 18 months or so when
pure-Posix clocks meet true-UTC leap seconds in an unignorable
way.  The right compromise, I think, is to smear them in a
decently-defined and ignorable way.  Google and Amazon have
supposedly implemented this already in their data centers, by
modifying NTP and leaving the OSes alone.  But I think it's well
worth exploring the opposite approach, too.

Steve Summit
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

[LEAPSECS] NIST UT1 NTP server results

[mike]

Hi,

   This was originally directed to the time-nuts list as it is more time 
scale oriented. However there has been quite a bit of leakage between 
the two lists following the Dec16 leap announcement, so I will post to 
both, but the discussion should stay on one list of possible, so tvb and 
I suggest it be on the leapsecs list. Sorry to those who ahve already 
received the text and no image. I have added a link here so that you can 
see the graph.


I am a rubbery seconds supporter myself. It is about time we realized 
that humans are not machines and like the idea of 86400 second days from 
here to the end of time.
There is of course a need for precise SI time intervals and a time scale 
to go with, but that can be distributed alongside an 86400sec day UTC. 
The techno exists, we just need the will to say that we humans take 
precedence. UT1 rules.


I'll jump down from my drum and share some data which I have not seen 
here before.


As most of you will already be aware, one of the results of the 
never-ending arguments about what to do with leap seconds, was that the 
IERS agreed to make available electronically UT1-UTC deltas with much 
greater precision than the GPS stream does (0.1 sec resolution). AFIK we 
don't have that yet, but at the beginning of June 2015, Judah Levine at 
NIST announced that NIST would be distributing high resolution UT1 in 
NTP frames .

See < http://www.nist.gov/pml/div688/grp40/ut1_ntp_description.cfm>.

As you can see from the document, the service was available to 
registered users with static IP addresses. My ISP only hands these out 
for $$$s so I registered with some of the cheaper VPN providers ones to 
test out the service over VPN links. Unfortunately there were such 
severe latency and jitter issues with all of those that I tried, that I 
abandoned my tests in August 2015. I also think I unfortunately pissed 
off Judah with my repeated requests for IP address registration as he 
stopped responding to mails. Sorry for that Judah if you are looking in.


Anyway I forgot all about it until the other day when I was looking at 
the peerstats data of the server I was using for the tests and 
discovered that the UT1 server was alive and responding over my 
unregistered IP with half the latency and usec level jitter. Luckily I 
had left the address in place in my ntp.conf with noselect option.

Here is the ntpq -pn data.
mike@cubieez2:~/NIST_UT1_server_data$ ntpq -pn
remote refid   st  t when poll reach delay   offset 
jitter

==
+192.168.1.23  .GPS.   1   u   61   64   377 0.173   -0.014  
0.024
 128.138.140.50.NIST.  1   u   41   64   377   130.670 -225.01   
0.102


You will also note from the NIST document and the NIST time server 
address links, that the UT1 NTP service will not respond to unregistered 
requests.
NIST may or may not have opened the box deliberately. I don't know, but 
if you wish to use the service please at least contact Judah before 
doing so. It would be a shame to have it going deaf.


Anyway, here are the results from the data I collected.
I have graphed the UT1 server offsets reported by the NTP peerstats data 
over the last 20 days and also the observed UT1-UTC deltas from IERS 
Bulletin A and the predicted UT1-UTC deltas for the same period from 
Bulletin A.


The plot is at < 
http://stratum1.ddns.net:8080/timenuts-data/peerstats17677_128.138.140.50.png 
>


As you can see, there is a systematic offset from the observed values 
reported in Bulletin A but the served value appears to track the 
predictions rather than the observed values. The resolution is much 
better than the 0.1s available via GPS but as the UT1 time is constant 
over the 24h day, it is not good enough to make a rubbery seconds clock. 
We need some interpolation.


The 13/14th of July something strange was going on. I was not monitoring 
this system at the time and have no idea what it was.


Mike

"The power of accurate observation is commonly called cynicism by those 
who have not got it. »

George Bernard Shaw
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap second to be introduced at midnight UTC December 31 this year

[Tom Van Baak]

> The data format only has a "leap second pending flag", which means a
> leap second is to be inserted.

Hi Martin,

Ouch. Well that's a problem. LSEM aside, what are you going to do if the earth 
continues to gradually speed up as it has the past couple of decades?

If you look at the WWVB spec, it also has a single bit for leap second pending. 
So at first glance it would appear to have the same problem as DCF77. But WWVB 
has bits for DUT1; the sign bit of DUT1 effectively tells you if the pending 
leap second is insert or delete.

/tvb

- Original Message - 
From: "Martin Burnicki" 
To: "Tom Van Baak" ; "Leap Second Discussion List" 
; "Discussion of precise time and frequency 
measurement" 
Sent: Friday, July 22, 2016 1:44 AM
Subject: Re: [LEAPSECS] [time-nuts] Leap second to be introduced at midnight 
UTC December 31 this year


> Tom Van Baak wrote:
>> Time to mention this again...
>> 
>> If we adopted the LSEM (Leap Second Every Month) model then none of
>> this would be a problem. The idea is not to decide *if* there will be
>> leap second, but to force every month to have a leap second. The IERS
>> decision is then what the *sign* of the leap second should be this
>> month.
> 
> Although this approach sound good, it would cause major problems for
> users of the German longwave transmitter DCF-77. The data format only
> has a "leap second pending flag", which means a leap second is to be
> inserted. AFAIK there is no spec to announce a negative leap second via
> DCF-77.
> 
> Martin
>
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] [time-nuts] Leap second to be introduced at midnight UTC December 31 this year

[Tony Finch]

Tom Van Baak  wrote:
>
> > Does your proposal allow for a Zero leap second
>
> Nope, LSEM avoids the zero leap second situation. That's the idea: to
> always have a leap second. Either an add or a delete, at the end of
> every month. The beauty is that it wouldn't violate how UTC is already
> defined.

Isn't the limit on DUTC 0.9s? So you can't have a leap second at the end
of the month when | UT1 - UTC | < 0.1

Tony.
-- 
f.anthony.n.finch    http://dotat.at/  -  I xn--zr8h punycode
Fisher, German Bight: Variable, mainly north, 3 or 4. Slight. Thundery
showers, fog patches in east. Moderate or good, occasionally very poor in
east.
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] [time-nuts] Leap second to be introduced at midnight UTC December 31 this year

[Martin Burnicki]

Hal Murray wrote:
> 
> s...@ucolick.org said:
>> This idea pushes extra complexity into every implementation of low level
>> kernel-space software, firmware, and hardware.  That's nice as a policy for
>> full employment of programmers, but it's hard to justify by any other
>> metric.  Instead those low level places should be as simple as possible, and
>> that means making the underlying precision time scale, and thus any
>> broadcast distributions of a precision time scale, as simple as possible. 
> 
> Has anybody done any serious thinking about fixing the POSIX problem?
> 
> I assume the first step would be to have the kernel keep time in a non 
> leaping time scale.  Is UT1 the obvious choice?

Wouldn't that basically mean to get back rubber band seconds? Also, how
would you distribute UT1? Via a table with a correction value to TAI?

IMO a better approach would be to let the system time run on TAI, and do
the conversion to UTC, and local time in the next step, by runtime
libraries. The tzdist protocol could be used to update both the TZ DB
and the leap second table.

This could solve the ambiguity of duplicate time stamps which you also
have at the end of a DST interval, unless you know the DST status. I
mean at least the conversion from TAI to UTC.

The conversion from UTC back to TAI can only be unambiguous if you have
some status information with the time stamp. For example, if you want to
derive UTC from local time then you need to know the basic local time
offset *and* the DST status. Otherwise you get ambiguous conversion
results at least at the end of a DST period where 1 hour is repeated.

This is similar with UTC. If the timestamp APIs always returned a
timestamp+status you could deal with that unambiguously. However,
implementation of the API would be expensive with regard to computation
time since it has to make sure that the time stamp and status
information are always returned consistently. This is bad for
applications which want to read time stamps as fast as possible.

> Would we need a version of ntp that used that time scale?  (or some 
> non-leaping time scale)

I think a compatible approach would be to let existing API calls behave
like they do now, and provide new enhanced API calls where the
application can determine if the kernel runs on TAI, and then use
different calls to return a TAI time stamp only, or a raw UTC time stamp
derived from TAI, or a time stamp plus status to resolve the ambiguity.

For example, Meinberg PCI cards provide API calls which return a 64 bit
time stamp very fast for applications which need it, and there are other
API calls which return a UTC time stamp, plus local time offset, plus a
status field indicating if the device is synchronized, if a leap second
is pending, if the current time stamp is in the middle of a leap second,
etc.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] [time-nuts] Leap second to be introduced at midnight UTC December 31 this year

[Martin Burnicki]

Tom Van Baak wrote:
> Time to mention this again...
> 
> If we adopted the LSEM (Leap Second Every Month) model then none of
> this would be a problem. The idea is not to decide *if* there will be
> leap second, but to force every month to have a leap second. The IERS
> decision is then what the *sign* of the leap second should be this
> month.

Although this approach sound good, it would cause major problems for
users of the German longwave transmitter DCF-77. The data format only
has a "leap second pending flag", which means a leap second is to be
inserted. AFAIK there is no spec to announce a negative leap second via
DCF-77.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs