Re: [LEAPSECS] 256-week / leap seconds / in the news

[Martin Burnicki via LEAPSECS]

Hi again,

one of my colleagues has a u-blox LEA M8T with firmware 3.01 available, 
and this morning, the device has output the following fields in the 
UBX-NAV_TIMELS message:



iTOW118812000Time Of Week [ms]
srcOfCurrLs 2Source of Leap Second Info
 (2 -> GPS)
currLs  18   Current GPS/UTC offset [s]
srcOfLsChange   2Source of Leap Second Change Info
 (2 -> GPS)
lsChange0Future Leap Second, if scheduled
 (0 -> none scheduled)
timetoLsEvent   767606   Time To Next Leap Second Event [s]
 (counting down in subsequent messages)
dateOfLsGpsWn   2185 [extended] GPS week number of the nearest
 leap second
dateOfLsGpsDn   7GPS day of week number for the nearest
 leap second
valid   0x03 Validity flags (0x03 -> both currLs and
 timetoLsEvent are valid)


GPS LS week number 2185 and day number 7 would indicate a leap second 
scheduled for the end of Sat, 2021-11-27. This must be wrong because 
2021-11-27 is not the end of June or December, but the wrong expanded 
week number is probably just due to the ambiguity of the truncated 8 bit 
week number in the UTC parameter set.


The current GPS time when this message was received was WN 2168 and TOW 
118812.

According to timetoLsEvent, the next LS would be 767606 s in the future,
which would be around WN 2168 TOW 886418,
which would be normalized to WN 2169 and TOW 281618,
which in turn means around 6 AM on 2021-08-04,
which is obviously not plausible, either.

And it differs from the other wrong date indicated by WN 2185 DN 7.

So the output is not consistent, specifically since timetoLsEvent is 
continuously counting down.


On the other hand, the field lsChange is 0, which means there is no leap 
second pending at all, and thus there's no need to evaluate the other 
fields to generate a leap second warning to other applications.



Martin
--
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg, 
Andre Hartmann, Heiko Gerstung

Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy



OpenPGP_signature
Description: OpenPGP digital signature
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] 256-week / leap seconds / in the news

[Martin Burnicki]

Tom,

Tom Van Baak wrote:

Steve,

I'm thinking the problem is not with GPS or with modulo arithmetic.


I fully agree.

I'm 
in contact with Leo and others for the root cause or the scope of the 
problem as reported on twitter. I'll say more later.


Per the GPS ICD, when dt_LS == dt_LSF there is no leap second to speak 
of; not in recent past; not in near future. The 8-bit WN and DN fields 
are not applicable to a non event.


That's exactly what the firmware has to be taken into account.

Some old firmware of the first Meinberg GPS receivers back in the 1990es 
had a similar problem.


At that time there were leap seconds inserted once every 12 or 18 
months. Also, at that time the expectation was then, that the interval 
between leap seconds would become shorter, and not longer, and I guess 
the folks who designed the data format for the GPS satellites assumed 
that the offset +/- 127 week from "now" was sufficient to handle that.


Meinberg GPS receivers use a 16 bit week number internally, and it was 
derived from the 10 bit week number of the "current" GPS time.


The 8 bit week number included in the UTC offset parameter set had to be 
expanded to that 16 bit week number.


So the very old Meinberg firmware also tried to expand the 8 bit week 
number to determine when the next leap second event would be and *then* 
looked at the 2 UTC offset parameters before and after that even.


Happily, it soon became clear that this was a wrong approach, and the 
handling was changed.


When dt_LS != dt_LSF you know there is a nearby leap second event. You 
then look at WN and DN to determine when. It could have been in the 
recent past, it could be in the near future, or even in progress. By 
"recent past" and "near future" I mean ±127 weeks (about ±2½ years).


Exactly. That's what the Meinberg GPS firmware now does when it 
evaluates the UTC parameter set from the satellites.


Of course, even if no leap second is scheduled, the truncated week 
number and day number of the latest leap second event are still 
transmitted, and in fact you can show the date derived from the expanded 
week number, but the date alone is just informational, and not 
considered as upcoming leap second event.


In fact, also the Meinberg GPS receiver firmware derives an extended 
week number 2185 from the satellite data. The "mbgstatus" program from 
an older version of the Meinberg driver software package for GPS PCI 
cards now shows the same wrong week number which refers to a leap second 
event in November 2021:



UTC correction parameters:
  CSUM: 1550, valid: 0001
  t0t: 2168|147456.000, A0: 0 A1: -1.77636e-15
  WNlsf: 2185, DN: 7, offs: 18/18
Last leap second eventually at UTC midnight at the end of Sat, 2021-11-27


But, as said earlier, this is only informational. Because the 2 UTC 
offset values are identical ("18/18"), the PCI card and API won't 
generate a leap second warning in November.


Some time ago, you (Tom) published an idea that could help to determine 
the real week number from the truncated on, and solve the +/- 127 week 
ambiguity. The point was to not only check the week number from the UTC 
parameter set, but also the day number, and the combination of week 
number and day number should expand to the end of June or December.


I've implemented that in our library, and the "mbgstatus" program from 
the current driver package shows (in verbose mode) the true date of the 
latest leap second event:



UTC correction parameters:
  CSUM: 15DA, valid: 0001
  t0t: 2168|233472.000, A0: 0 A1: -1.77636e-15
  WNlsf: 2185 (true: 1929), DN: 7, offs: 18/18
Current GPS/UTC offset: 18 s, no leap second announced.
Nearest leap second just before UTC midnight on Sun, 2017-01-01.


Clearly this design means GPS cannot give indefinite past history of a 
previous leap second(s), nor indefinite future notice of a pending leap 
second(s). This is not a problem given how UTC is currently defined and 
managed. BIPM has a good track record of 6 months (~26 weeks) of notice. 
The official UTC spec is 1 month (~4 weeks) of notice so a 127 week GPS 
limit is more than adequate.


Yes, but as you can see above, your idea to solve the ambiguity was 
really helpful. Thanks for that.



On 7/25/2021 8:54 AM, Steve Allen wrote:

On Sat 2021-07-24T18:50:50-0700 Tom Van Baak hath writ:

In the news:

"GPS will broadcast a 0 second leap second in 128 days"
https://news.ycombinator.com/item?id=27944776


I think the title of this article is just misleading.

As far as I can see, the GPS satellites are transmitting data according 
to the specs (ICD), and the expansion of the truncated week number is 
not ambiguous *if* a leap second is scheduled, and the leap second date 
is within 

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Martin Burnicki]

Seaman, Robert Lewis - (rseaman) wrote:
[...]
> We are extremely happy with the quality and support we have received
> from Meinberg, ...
Thanks, Rob, I'm very happy to hear this.

[...]
> (I have no financial interests in Meinberg or NTF.)

Disclaimer: ;-)

Even though I'm working for Meinberg, I'd like to point out that what I
usually write in this and similar mailing lists is also not due to
financial interests of any kind.

It's just my personal opinion, from my technical and practical point of
view.


Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Martin Burnicki]

Hal Murray wrote:
> 
> tvb said:
>> There's no ambiguity. Those are just bugs. No software should depend on  more
>> than 1 month notice of a leap second and no software should be  fooled if the
>> notice is months or even years in advance. 

Please keep in mind that e.g. GPS sends out leap second announcements
about 6 months in advance. This is not a problem if the receiver
firmware follows the specs, since the week number and day number are
specified. But, as we have seen, this can be a problem due to bugs in
the receiver firmware.

> There are plenty of quirks in ntp code along that line.  The APIs don't have 
> an explicit when.  The NTP-Kernal API for leap-pending is leap-tonight.  You 
> have most of the next day to turn it off.  The leap-pending on the wire is 
> leap-at-the-end-of-this-month.

I think this is some "best practice" that has evolved over many years.
For the kernel, the "end of this day" announcement is sufficient.

For the protocol, roughly 1 month is sufficient but short enough to
avoid ambiguous dates, i.e. at the end of which month.

If I remember correctly, the PTP specs even specify an announcement
interval of 12 hours only, the German DCF77 transmitter sends the
announcement only 1 hour in advance, and IRIG time codes (if they
support it at all, like IEEE1344 / C37.118) only 10 seconds or so in
advance.

So I wonder which spec should have been relevant for NTP/ntpd?

Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Martin Burnicki]

Tom Van Baak wrote:
> Hi Hal,
> 
> It's 2020. How on earth can NTP still not implement UTC correctly, in
> all cases? Or is it a fundamental NTP design flaw?

A program like ntpd (or ptpd, FWIW) has to rely on available sources of
information, and if those sources provide wrong information, this can
produce wrong results.

Of course there can be (and have been) bugs in the way ntpd handled leap
seconds. However, I'm not aware of any bufg in the current ntpd code.

Originally ntpd accepted a leap second from any upstream source that
announced it, and forwarded the announcement to its clients.

However, after it had happened that some GPS receiver models had sent a
faulty leap second announcement which the server forwarded to its
clients, the code in ntpd was modified in a way that a client accepts a
leap second announcement only if a majority of its upstream servers
provides the announcement.

If you implement a plausibility check that accepts leap second only at
the end of June or December, you avoid (at the first glance) that your
server's time is messed up if a faulty device sends an announcement at
the end of the wrong month.

But what if your GPS receiver sends a wrong leap second announcement at
the end of December, when there is no real leap second, or does *not*
send an announcement if a leap second has been scheduled?

If the GPS receiver which sends a wrong leap second announcement also
applies the leap second to its internal time, then the time sent by the
receiver is off by 1 s after the false leap second event.

What should a program do that has the GPS receiver as its sole time
source? Stop synchronizing to the receiver, or follow the time step
after some time if the 1 s offset persists?

What if the GPS satellites start sending data that cause the UTC
correction by GPS receivers to be 13 microseconds off? Is NTP to blame
if it follows the GPS receiver's time?

Which is the offset range that should be accepted if an "authoritative"
time source like a GPS receiver starts sending a different time?

So I wonder why NTP (the protocol) or ntpd (the implementation) are
blamed here. There's a lot of stuff in there which tries to sort out
errors coming from external time sources, but I doubt there can be an
implementation which is in no way affected by any potential error from
outside.


Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds have a larger context than POSIX

[Martin Burnicki]

Tony Finch wrote:
> The IERS Bulletins C state a value of UTC-TAI "until further notice".

As I understand this, it just means that yet it's not known if and when
another leap second will have to be scheduled.

> However the machine-readable files from IERS and NIST give an expiry date
> of a few days less than 6 months after the announced (lack of) leap
> second, or a bit more than 11 months after the latest Bulletin C.
> Is this expiry date reliable or just advisory? History suggests it's
> reliable, but the standards do not.

In my opinion this expiration date is very important for those who
evaluate the file, e.g. ntpd.

If ntpd had a leap second file which doesn't contain a leap second after
2016, it didn't know it there really was no other leap second, or if
only the file was too old and hadn't been updated.

In the latter case ntpd wouldn't even know if the TAI/UTC offset derived
from the available leap second file is still valid, or if another leap
second has been inserted in the mean time, and the TAI/UTC offset has
changed.

With the expiration date ntpd can be sure there was no other leap
second, and the TAI/UTC offset is still valid, as long as the current
time is before the expiration date, and the file can be ignored after it
has expired.

If the bulletin C is published once every 6 months, and you have the
bulletin from January, you know for sure if there will be a leap second
at the end of June, or not.

However, only the next bulletin C from next July will tell you if there
will be leap second at the end of December. So in January you can be
sure the information you have won't change until shortly before the end
of December, which is about 11 months.

Even after the next bulletin C and leap second file have been published
in July, things won't change at least up to the expiration date of the
previous file. So you have still about 5 months to deploy the leap
second file from July to your ntpd instances, before anything expire.

One way to propagate a new leap second file is to

- add the new file to the TZ DB some time after tit has been published

- the new file appears in the TZ DB when a new TZ DB versio is released

- after this it appears at the IANA web page, and eventually in software
update packages for your favorite operating system

In summary, it can take weeks or even months until the file is available
everywhere.

So IMO it's very practical that the expiration date is there, and that
you have about 5 months to deploy a new version.


Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Martin Burnicki]

Brooks Harris wrote:
> Evolution of Timekeeping in Windows
> https://techcommunity.microsoft.com/t5/networking-blog/evolution-of-timekeeping-in-windows/ba-p/778020

Interesting that they call it "the (r)evolutionary changes to time
keeping in the Windows operating system" when they finally managed to
achieve a precision that other operating systems are already providing
since decades. ;-)

Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Martin Burnicki]

Hal Murray wrote:
> 
> Does anybody know of a good writeup of how to fix POSIX to know about leap 
> seconds and/or why POSIX hasn't done anything about it yet?

I've made a number of presentations and whitepapers about leap seconds
and problems related to them. However I'm not aware of an easy, good
solution.

The basic problem is that common API calls used to retrieve the system
time stamp from an OS don't provide a status that could be used to
distinguish between a normal second and an ongoing, inserted leap second.

Without such status a function that converts a timestamp to calendar
date and time doesn't know if the timestamp associated with an inserted
leap second should yield a second with number '60' of the current day,
or a second '00' of the next day.

I think an API that provides a timestamp and an associated status in a
*consistent* way would be too "expensive" with regard to execution time
because some locking mechanism needed to be implemented to avoid that
inconsistent timestamp and status information could be returned.

On the other hand, if an application already has a broken down date and
time (e.g. from an external time source, serial string or whatever), it
knows that the time is the leap second if the second number is '60'. So
the '60' is the required status information.

However, if you "normalize" the time of a leap second, e.g. 23:59:60,
then 60 seconds carry over to one minute, 60 minutes to 1 hour, and 24
hours to the next day. So when computing an associated timestamp, the
effective timestamp is the same as for 00:00:00 the next day, and the
information that this second is a leap second is simply lost unless
otherwise preserved in some way.

On most Unix-like systems the system time is kept as POSIX time, and
when a leap second is inserted, the kernel just steps the system time
back by 1 second.

*This* is what confuses applications that don't expect that the system
timestamp ever goes backward.

Many years ago Dave Mills has proposed a way to avoid this problem by
stopping the system clock for 1 s, and doing a system time increment by
the smallest unit whenever an application retrieves a system time stamp
while the clock is stopped.

This would be a good workaround since the time returned from the kernel
never steps back, and there are no duplicate timestamps because even
during the leap second the timestamps increment by a small amount.

Quite some time ago I asked one of the Linux kernel maintainers why they
don't implement the leap second handling this way, and the answer was
just "because it's too expensive". Whenever an application queried the
current system time, the kernel had to check if a leap second was just
being inserted, and if there was, some small amount of time had to be
added to the stopped system time.

And all this just to get it "right" for 1 second in several years. Just
stepping the time back by 1 second at a certain point in time is much
faster, and much easier to implement.

> I assume the basic idea would be something like switch the kernel to use TAI 
> rather than UTC and implement conversion in some library routines.

I think this could be a good approach, but this requires that not only
leap second announcements but also the current TAI offset is supplied to
the OS.

Runtime libraries require the correct TAI offset to be able to convert
the TAI system time to UTC. E.g., the kernel could return a TAI
timestamp, and the runtime library function like gmtime() needs to know
the current TAI offset and leap second date to be able to return a time
with a 60 in the seconds field.

I know there are leap second files from IERS or NIST, and also current
versions of the TZ database contain a copy of such file, but this
requires that this information is continuously updated, i.e. new
versions of the leap second files or the TZ data base are supplied.

This should work (I haven't tried it) if you configure the system with
one of the "right" timezones, which gets the TAI information from a
table of the TZ DB.

This may work with current versions of operating systems, where the OS
maintainers or the admin provides the required information, but may not
work reliably for systems that are out of support, or embedded systems
that never get any update. Thes would use a wrong UTC time after the
next leap second.

In the past, ntpd could provide its clients also with the TAI offset if
autokey was configured. Since autokey is now obsolete and replaced by
NTS, an extension field to the NTP packet could provide the TAI offset.

PTP/IEEE 1588 works based on TAI, and the protocol provides the TAI to
UTC offset, so either PTP or NTP with TAI offset extension field could
be used to adjust the kernel time to TAI.

Both ntpd and PTP clients should be able to write the current TAI offset
and leap second announcments to the kernel. However, AFAIK time
conversion functions only retrieve the TAI information from the TZ DB,
not from the kernel, so the TAI capabilities of NTP and PTP 

Re: [LEAPSECS] aircraft GPS receivers hit by leap second bug

[Martin Burnicki]

Tom,

Tom Van Baak wrote:
>> What I meant is that if you try to derive the date of the last recent
>> leap second from WNlsf if the 2 offsets *are* the same, the result is
>> ambiguous since you don't know if you are in a +/- 128 weeks interval,
>> or if another 256 weeks interval has passed. That's exactly what we are
>> observing right now.
> 
> Let me explain a cute trick. The ± 128 or modulo-256 week ambiguity that
> you mention is certainly true, as well as the 19.x year 1024 week GPS WNRO
> that we all know.
> 
> But look one step deeper. Each 8-bit week number and 3-bit day number used
> to describe the most recent or pending leap second must necessarily be the
> last day of a calendar month, per UTC rules, yes? It turns out this fact
> can be used to resolve the ambiguity that you speak of.
[...]

This is really cute! If you permit, I'll see if we can add this to our
driver software packages.

Thanks,

Martin
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] aircraft GPS receivers hit by leap second bug

[Martin Burnicki]

Tom Van Baak wrote:
>> However, the current GPS/UTC offset numbers before and after the nearest
>> leap seconds are still 18/18, so there is no current leap second
>> announcement, and WNlsf may be ambiguous.
> 
> Perhaps call it "immaterial" rather than "ambiguous"? The fact that it's
> 18/18 means no positive or negative leap second is pending. Period.

Yes, the WNlsf number is immaterial for timing computations as long as
the 2 offset values don't differ.

What I meant is that if you try to derive the date of the last recent
leap second from WNlsf if the 2 offsets *are* the same, the result is
ambiguous since you don't know if you are in a +/- 128 weeks interval,
or if another 256 weeks interval has passed. That's exactly what we are
observing right now.

> In other words, the value of WNlsf doesn't matter in this case. It's an
> 8-bit value so obviously it must always be something between 0x00 and
> 0xFF, or -128 to +127. Maybe it's a recent old value, maybe it's zero,
> maybe a future new value, or maybe random. It doesn't matter. What
> matters first are the tLS and tFLS values, which are currently 18/18 --
> which means no leap second. Period.

Yes, and as I wrote in my previous email the Meinberg GPS firmware
doesn't evaluate WNlsf for timing in this case.

Only some of our utility programs try to find out the date of the last
recent leap second, and display it just for informational reasons.

As mentioned before we used the word "eventually" ("probably" in the
next software release) to say that due to the ambiguity this is not
necessarily a distinct date.

> A similar issue arose in some GPS receivers during the 7 year "leap
> second drought" of 1998 to 2005. [1]

Yes.

> Here's the math:
> 
>   * GPS start date: 1980-01-06 (MJD 44244)
>   * Most recent leap second: end of the day 2016-12-31 (MJD 57753)
>   * Most recent leap second: prior to the day 2017-01-01 (MJD 57754)
>   * Date of first Collins / GPS / ADS-B anomaly: 2019-06-09 (MJD 58643)
>   * Date that Collins says their systems will start working again:
> 2019-06-16 (MJD 58650)
> 
> Note that (58650 - 57754) / 7 = 128.000
> 
> So it's a big Rockwell-Collins oops. Both the GPS receiver firmware
> engineer who wrote the embedded code and the tester using a fancy GPS
> simulator dropped the ball.

Exactly.

Martin
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] aircraft GPS receivers hit by leap second bug

[Martin Burnicki]

Paul Hirose wrote:
> Thinking I had missed a pending leap second, I checked the IERS site,
> but Bulletin C says the offset is still 37 seconds and nothing is
> scheduled. ???

The GPS satellites transmit the week number of the nearest leap second
(WNlsf) as 8 bit value only, giving a valid range of ~+/- 128 weeks from
"now".

The last leap second was on 2016-12-31, and counting the number of weeks
in 2017, 2018, and 2019 until now, this sounds like close to 128. I
haven't computed the exact number, but I'm assuming that the range was
just recently exceeded. I'm currently out of the office, but I've heard
from my colleagues that the currently sent WNlsf number is 0x89.

However, the current GPS/UTC offset numbers before and after the nearest
leap seconds are still 18/18, so there is no current leap second
announcement, and WNlsf may be ambiguous. When the next announcement
starts the offsets will change to 18/19, and WNlsf will also be updated
to be in range.

The "mbgstatus -vvv" program for the Meinberg GPS PCI cards now shows a
wrong leap second date for 2021, too:

UTC correction parameters:
  t0t: 2057|405504.000, A0: -9.31323e-10 A1: -2.66454e-15
  WNlsf: 2185, DN: 7, offs: 18/18
Last leap second eventually at UTC midnight at the end of Sat, 2021-11-27

while just recently the displayed leap second date was (correctly)
2016-12-31.

However, Meinberg GPS receivers evaluate WNlsf *only* if the 2 offset
values differ, i.e. there is really an announcement of a leap second,
and WNlsf is in a valid range, so these receivers don't have a problem
with a wrong leap second.

If no leap second is announced then extended week number and thus the
date *can* be wrong, but it's just informational, and the word
"eventually" is used to point out the ambiguity.

Martin
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] is leap smear legal in Germany?

[Martin Burnicki]

Steve,

Steve Allen wrote:
> The story about the German time broadcasts of DCF77 is in Bulletin
> Horaire.
[...]
> But now I am trolling and asking:
> Given that rubber seconds are illegal in Germany, is it legal to
> use Google/Amazon NTP servers that provide smeared leap seconds?

Thanks for the historic facts on DCF77, that's very interesting.

I don't know if it's legal or not, but I know some of our customers here
in Germany have explicitly configured their Meinberg LANTIME NTP servers
to do leap second smearing.

It's not because they like smearing seconds so much, it's just because
it's a reliable way to avoid problems caused by OS kernels which simply
step the system time back to insert a leap second.

Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Running on TAI

[Martin Burnicki]

Steve Allen wrote:
> On Thu 2019-01-17T18:12:25+0100 Martin Burnicki hath writ:
>> Hm, maybe that was originally the case. I wonder whether the folks who
>> wrote the text just had UTC in mind when they "invented" time_t.
> 
> The best insight into the POSIX committee was posted on LEAPSECS in
> 2003
> https://www.mail-archive.com/leapsecs@rom.usno.navy.mil/msg00109.html

Very interesting. Thanks for the pointer!

> Not clear in that posting was that Arthur David Olson's timezone code
> for Unix systems already contained the code written by Bradley White
> which allows time_t to count every second in the radio broadcast time
> scale and handle leap seconds.  POSIX committee members knew that and
> decided to disregard it with words that actually prevented system
> designers from adopting a more correct scheme.
> 
>> So IMO this clearly says that time_t has to be interpreted depending on
>> the context, and must not necessarily hold exclusively UTC numbers of
>> second.
> 
> The tricky part comes when software on one system exchanges values of
> time_t with another system that believes time_t has a different value.

Yes, but that's basically what I said: It depends on the context (or
maybe a different word would be more appropriate than "context"), but
not on the data type.

It's even the same if you look at different struct timespecs taken on
the same system, but with different clock IDs. From the data type alone
you don't know how to interpret this correctly.

>> I think a good solution would be to have a structure that contains a
>> time_t value plus at least one flag that indicates the leap second
>> status, or a field with TAI index. It would be even better to have such
>> additional information available with struct timespec, so an application
>> can tell if the time stamp is inside a leap second, or not.
> 
> I think that the entire infrastructure should recognize the notion of
> needing two kinds of time:  precision seconds and civil days.  That is
> what the astronomy community had understood and promoted since before
> 1950.

I fully agree. From what I've read there are some programming
environments (Java, Python, ...) that provide individual workarounds for
the limitations of the OS, but I think you run into trouble again if you
have to exchange timestamped data between applications that have been
developed using different build environments.

> That is the hard part.  It is the part that technical folks punted on
> doing during the 1960s because it would have required explaining the
> need for two kinds of time to bureaucrats and legislators and waiting
> for them to change laws and regulations.  Instead they chose to act
> quickly and implement something controversial and technically barren.
> 
> The rest of the technical parts needed for the notion of two kinds of
> time have been discussed here before.  The underlying time scale used
> for radio broadcasts and operations by any kind of machine should be
> purely atomic.  The counting of calendar days and setting of civil
> clocks should be calculated using an offset from the underlying atomic
> time scale.  The conversions from that underlying atomic time scale
> should be widely distributed by a robust scheme like the Domain Name
> System and financial transactions, and those conversions should not
> happen in critical places like the kernel but in other less-critical
> places like libraries for locale and internationalization.

I fully agree.


Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Running on TAI

[Martin Burnicki]

Steve Summit wrote:
> Martin Burnicki wrote:
>> Just a few general thoughts based on an internal note...
> 
> And some very good points they were.
> 
>> ...The same applies to the time_t type, IMO. If you let the system kernel
>> run on TAI or whatever then the time() function still returns a time_t
>> data type, but its value represents TAI or whatever has been configured.
> 
> Right... but you may or may not be skating on thin ice!
> 
> I think the question of "Must time_t represent UTC?" can be
> a pretty illuminating one, and it cuts across several of the
> fundamental issues here.  Strictly speaking, the Posix definition
> of time_t is, as Warner Losh has argued, very specifically UTC.
> (And of course it's also a very specifically non-UTC form of UTC.)

Hm, maybe that was originally the case. I wonder whether the folks who
wrote the text just had UTC in mind when they "invented" time_t.

As I've mentioned earlier, also "struct time_spec" has a time_t field,
and the page
http://pubs.opengroup.org/onlinepubs/7908799/xsh/clock_settime.html

says:

"All implementations support a clock_id of CLOCK_REALTIME defined in
. This clock represents the realtime clock for the system. For
this clock, the values returned by clock_gettime() and specified by
clock_settime() represent the amount of time (in seconds and
nanoseconds) since the Epoch. An implementation may also support
additional clocks. The interpretation of time values for these clocks is
unspecified."

So IMO this clearly says that time_t has to be interpreted depending on
the context, and must not necessarily hold exclusively UTC numbers of
second.

> But it's equally true, as Gary Miller and Martin have argued,
> that in practice people use time_t for all sorts of other things...
> but I think we can also agree that (some of the time, at least)
> this can lead to additional real confusion, precisely because it
> breaks the spec, precisely because doing so may clash with someone
> else's spec-based assumptions.

I don't think this breaks the spec, see above. IMO, time_t is just the
type of parameters used with time(), gmtime() and friends, its bit width
is implementation defined, but if you use it you are always safe to use
an appropriate data type with those functions.

> But the real tragedy is that UTC is the timescale that time_t is
> the *worst* at representing, precisely because time_t is a uniform
> count of seconds since an epoch, and UTC is nonuniform.  Anyone
> who tries to "fix" some time_t-using software to do UTC "properly"
> is doomed to failure.  At best they'll end up reinventing TAI.
> 
> To my mind, a pretty good litmus test for how good an alleged
> scheme for properly handling leap seconds actually is is to look
> at its data structure.  Can it represent :60 explicitly and
> unambiguously?  If not, it's probably a kludge that's never going
> to work perfectly.  (But, indeed, the near universality of time_t
> and related representations is one of the factors that makes
> "solving" the leap second problem almost impossible, and will
> likely help lead to the abandonment of leap seconds.)

I think a good solution would be to have a structure that contains a
time_t value plus at least one flag that indicates the leap second
status, or a field with TAI index. It would be even better to have such
additional information available with struct timespec, so an application
can tell if the time stamp is inside a leap second, or not.

> This message will get too long if I start discussing the data
> structures that *can* represent :60 explicitly and unambiguously.
> Some of them are:
> * struct timespec (using the nonnormalized trick described under
>   "Additional clocks" at https://www.cl.cam.ac.uk/~mgk25/posix-clocks.html)
> * struct tm
> * the pair (days-since-epoch, seconds-within-day)
> * a time_t *plus* a well-defined bit disambiguating the two
>   instances of a repeated second across a leap second

If a timestamp has an associated status information available then it is
no problem for a library function like gmtime() or localtime() to set
the seconds field to 60 during a leap second.

We use a similar approach in the library functions we use for the
firmware of our PCI cards, etc.

On the other hand, if you normalize "60" seconds you get one more minute
which lets 59 increase to 60, and so on, so you will also get the
effective date/time twice. It's just a question how to interpret the
numbers.


Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Direc

Re: [LEAPSECS] Running on TAI

[Martin Burnicki]

Hi all,

Rob Seaman wrote:
> Our Meinberg ref clocks can deliver GPS, UTC, or TAI (and thus other
> derived timescales).

Yes, and the same data formats are used for each of these time scales.


Just a few general thoughts based on an internal note, instead of
replies to individual posts here on the list:

IMO you have to generally distinguish between specific data formats, and
the type of information a particular data type can contains.

For example, the 32:32 bit timestamp type used in NTP packets has been
specified to provide UTC. But of course, if you configure your
timeserver for a different time scale than UTC then it just transports
"a time" according to the configured scale.

I'm sure that even the UT1 NTP server operated by NIST puts a UTC
timestamp into those fields.

The same applies to the time_t type, IMO. If you let the system kernel
run on TAI or whatever then the time() function still returns a time_t
data type, but its value represents TAI or whatever has been configured.

Even if you have separate API calls that either return TAI, UTC, or
whatever from the kernel you can use the same data format for the
returned timestamps.

In fact, clock_gettime() always returns the same data type (struct
timespec, which includes a time_t field), but the type of information
differs depending depending on the specified clock ID, and may be a UTC
time, a number of seconds and nanoseconds since boot, or whatever.


IMO the basic problem is in many cases that some extended information is
missing. For example, if you receive an NTP packet you are basically
unable to distinguish if the time in the packet is really UTC, or
something else. However, if you know that the packet originates from a
UT1 server you can assume that it transports UT1 rather than UTC.

Similar with time_t. If you read timestamps in a fast loop across an
inserted leap second you can only distinguish times if you have a leap
second status flag associated with the time, so when kernels simply step
the time back to insert a leap second you don't know if the "UTC
midnight" timestamp is represents the original second, or the repeated
second, which is in fact the leap second.

This is similar to the end of DST, where a pure hh:mm:ss time doesn't
tell if it belongs to the first hour, or to the repeated hour. You know
this only if you can retrieve the DST status with the time in a
consistent way, so the first time has DST enabled and the repeated time
has it disabled.

Even if a DST flag is available (e.g. in struct tm), you don't know from
the information from the structure alone if the time the structure
contains is UTC, or local a time according to whichever time zone has
been configured.


Generally it makes a difference whether you simply need the current time
which steps when a leap second is applied to UTC, or when DST changes
for local time, and continues ticking normally after that event, or if
you have to deal with time *intervals*.

If your application deals with time *intervals* then it has to be aware
that the day is one second longer than usual if a leap second has been
inserted, and in local time even one hour longer at the day when DST
ends. I think developers more often care about DST steps than leap
second steps because DST steps simply occur more often, in shorter
intervals.


If you need to convert GPS time or TAI to UTC, or vice versa, then of
course you need some source of information that tells the current
UTC/TAI or UTC/GPS offset. This is also the case if your kernel runs on
TAI. There are different possible sources which provide different amount
of information.

For "current" times the information that can be provided by the NTP or
PTP protocol, or from a GPS receiver, is sufficient.

However, if you need to deal with historic time intervals then a leap
second table is required which explicitly tells at which points in time
leap seconds have been inserted. This is just similar to dealing with
historic local times, where the TZ DB provides information if and when a
DST changeover occurred in the past.

Neither GPS, nor PTP or NTP provide this historic information, but e.g.
the TZ DB also includes a leap second table, or you can get the leap
second file directly from the sources (IERS, NIST, USNO, ...).

The problem here is that a leap second table as well as the TZ DB has to
be kept up-to-date by some means, since upcoming changes can't be
predicted automatically.

This requirement is not only for TAI/GPS to UTC conversion. Also if you
need to derive e.g. UT1 from TAI, GPS or UTC you have to provide some
information (e.g. a data file) how the UT1 offset develops over time,
and this information has to be updated in regular intervals.


Sorry for the long post which just tries to summerize a couple of things.


Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linke

Re: [LEAPSECS] Windows Server 2019

[Martin Burnicki]

Nero Imhard wrote:
> Martin Burnicki schreef op 2018-07-20 12:52:
> 
>> Nero Imhard wrote:
>>> If that translates to "don't smear ntp" I could not agree more.
>>> Smearing is catering to those who won't clean up their act,
>>> causing trouble for those who try to do the right thing.
>>
>> Please don't blame ntpd for smearing leap seconds.
>  
> I don't. ntpd would be the victim here.

;-)
  
>> Leap second smearing by an NTP server is just an optional way to avoid
>> these problems because there's no better way to get around it.
>  
> Leap second smearing by an NTP server effectively means that it uses
> a different time. Without a way to indicate, within the protocol,
> what time a server uses (and there isn't because UTC is implicit) this
> seems a really bad idea.

Basically I agree with you, but ...

> On another level, I regard it as some sort of "technical betrayal"
> which erodes trust in protocols and standards, rendering them less
> useful than they could be.

Yes, the NTP protocol has been designed to work with UTC time stamps.

I know there are also folks who let their own NTP server transmit pure
TAI or raw GPS time instead of UTC, since some devices on their network
need this.

Also, even the NIST operates an NTP server that provides UT1 rather than
UTC.

So for specific cases, in a closed environment, it's OK IMO.
  
> Isolate your workarounds and don't let them pollute your environment. 

Especially, there shouldn't be any public/pool servers that smear leap
seconds.

If you are administrator for a huge company network you are eventually
very happy if you can just let the company's NTP server smear a leap
second, and none of the clients or applications encounter any problem.

If you don't want it, don't use it. ;-)

The problem is if there are folks that don't know (exactly) what they
are doing, e.g. use as reference one of Google's public servers that are
known to smear leap seconds.

Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Windows Server 2019

[Martin Burnicki]

Nero Imhard wrote:
> scs...@eskimo.com schreef op 2018-07-20 11:35:
> 
>> The question of what happens if you try to run a leapsec-aware
>> kernel downstream of a smearing NTP server is an interesting one.
>> My preferred answer is "Don't do that."
>  
> If that translates to "don't smear ntp" I could not agree more.
> Smearing is catering to those who won't clean up their act,
> causing trouble for those who try to do the right thing.

Please don't blame ntpd for smearing leap seconds.

The leap second is inserted/handled by the kernel, and that's basically
the way it should be.

However, this is often done in a way that applications may become
confused if the system time is simply stepped back by 1 s.

If you used any other time synchronization software which just passes a
leap second announcement to the kernel then the applications will have
the same problems.

Leap second smearing by an NTP server is just an optional way to avoid
these problems because there's no better way to get around it.


Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Windows Server 2019

[Martin Burnicki]

Steve Summit wrote:
> Steve Allen wrote:
>>> as seen on slashdot and more, Windows Server 2019 will support
>>> leap seconds.
> 
> I don't often say this, but: Good for Microsoft.
> 
> Despite the several, well known, significant difficulties, I have
> been on a quixotic on-and-off quest to implement "proper" leap
> second handling for Linux.  (As some will recall, on 2016-12-31
> I managed to post a message to this list at 23:59:60 UTC, with a
> Date: line to prove it.)  If anyone feels shamed into action by
> Microsoft's announcement, and wants to try things out under Linux,
> please feel free to pester me into finally publishing that work
> publicly. :-)
> 
> Stephen Colebourne wrote:
>> ...it needs to be possible for any of these setups to be valid
>> (and maybe more options too):
>>
>> - network clock returns UTC, OS handles UTC, application/framework handles 
>> UTC
>> - network clock returns UTC, OS handles UTC, application/framework smears
>> - network clock returns UTC, OS smears, application/framework unaware
> 
> The modified Linux kernel that I have implements these sorts of
> choices via clock_gettime().  The new clkid value CLOCK_UTC gives
> you true UTC, the new clkid value CLOCK_SMEARED gives you smeared
> time, and the existing clkid value CLOCK_REALTIME currently gives
> you whatever we used to put up with (basically a repeated second),
> but could/would be changed to return the same as CLOCK_SMEARED
> once it was confidently proved to be working.

This sounds like a real step forward to flexible leap second handling by
the kernel, and IMO it would be good if this could make it into the kernel.

> The question of what happens if you try to run a leapsec-aware
> kernel downstream of a smearing NTP server is an interesting one.
> My preferred answer is "Don't do that."  The failure modes if you
> did wouldn't be *too* bad, but "proper" handling of CLOCK_UTC in
> that case would of course require an unsmearing algorithm, which
> I for one am not interested in writing.

I think the admins had to take care that they use a reliable clock
source. If you install a GPS receiver which outputs faulty time (e.g. to
GPS week number rollovers) or output faulty leap second warnings then
you may end up with a wrong system time. Similarly, if you chose a
smearing NTP server even though you don't want the incoming time to be
smeared.

>> Getting the smear off the network and into the OS will undoubtably
>> be useful in some use cases, but they are not the only use cases.
> 
> Indeed.

However, this had to be made public so that software developers become
aware of these features and what the differences are, so they can call
clock_gettime() with the clock ID that is best suited for their
application.

If they are not aware of the differences in the clock models then their
application will probably not work as expected anyway.


Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Windows Server 2019

[Martin Burnicki]

Stephen Colebourne wrote:
> On 19 July 2018 at 11:24, Martin Burnicki  wrote:
>> As far as I know, UTC-SLS is done locally on a client, and since it's
>> implemented in a Java runtime it is only "seen" by Java applications.
>>
>> This means if you get timestamps in Java and non-Java applications on
>> the same machine then they may be off by up to 1000 s at the end of the
>> UTC-SLS smear interval, isn't it?
>>
>> If you have a smearing NTP server then all clients of the same server
>> will have the same smeared time, which is of course off UTC during the
>> smear interval, but at least all Java and non-Java applications on a
>> particular machine will see the same system time.
>>
>> I think it depends on the requirements of the applications which way is
>> to be preferred.
> 
> True, but the point is that we have already, and are likely to have
> more in the future, boundaries that want different representations.
> Specifically, cases where a low level system understands and models
> leap seconds, but a high level system does not want to (for various
> perfectly good reasons). Once it is accepted that there are two
> representations, and both are valid, there is a need to convert
> between the two.
> 
> For the plan to work fully it needs to be possible for any of these
> setups to be valid (and maybe more options too):
> 
> - network clock returns UTC, OS handles UTC, application/framework handles UTC
> - network clock returns UTC, OS handles UTC, application/framework smears
> - network clock returns UTC, OS smears, application/framework unaware
> - network clock smears, OS unaware, application/framework unaware
> 
> Getting the smear off the network and into the OS will undoubtably be
> useful in some use cases, but they are not the only use cases.

I fully agree, and that's what I meant by "it depends on the
requirements of the applications". ;-)

Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Windows Server 2019

[Martin Burnicki]

Stephen Colebourne wrote:
> On 19 July 2018 at 09:44, Hal Murray  wrote:
>>> As a IT professional, and author of date/time libraries, I cannot stress
>>> enough how much a standard is needed here. We are going to have both UTC
>>> (with leap seconds) and systems that smear ("UT-Smear") and there is
>>> currently no agreed way to define the latter or move from one to the other. 
>>> I
>>> strongly suspect that Microsoft is going to have to define a smear in order
>>> to meet old APIs, but this really should be something well defined by a
>>> standard, not invented by a company.
>>
>> I think there is a semi-standard emerging for NTP.
>>
>> There are several big companies running smearing servers.  I think they have
>> all agreed to use the same parameters.  I think that's linear over 24 hours,
>> 12 before the leap and 12 after.  I'll dig deeper if it matters.
> 
> What I'm hoping will appear is a document that can be treated as a
> standard (preferably at a formal standards body of some kind).
> 
> With the changes to Java, I used UTC-SLS (linear over 1000secs). But I
> deliberately left open the possibility for Java to adopt any future
> standard for smearing that emerged.
> https://docs.oracle.com/javase/10/docs/api/java/time/Instant.html

As far as I know, UTC-SLS is done locally on a client, and since it's
implemented in a Java runtime it is only "seen" by Java applications.

This means if you get timestamps in Java and non-Java applications on
the same machine then they may be off by up to 1000 s at the end of the
UTC-SLS smear interval, isn't it?

If you have a smearing NTP server then all clients of the same server
will have the same smeared time, which is of course off UTC during the
smear interval, but at least all Java and non-Java applications on a
particular machine will see the same system time.

I think it depends on the requirements of the applications which way is
to be preferred.

Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Windows Server 2019

[Martin Burnicki]

Steve Allen wrote:
> On Thu 2018-07-19T04:14:12+0100 Stephen Colebourne hath writ:
>> "In addition, there is no standard method for applying this frequency
>> adjustment, so that different implementations may disagree among
>> themselves in addition to the time error with respect to UTC."
> 
> That quote is not from Microsoft, but rather from the
> ION paper by Matsakis, Levine, and Lombardi
> That can be got by ION members, or from USNO if their
> webserver recovers, or from researchgate if you can
> tolerate the javascript that will try to execute
> https://www.researchgate.net/publication/323600621_Metrological_and_legal_traceability_of_time_signals

Thanks for the pointers.

> Microsoft seems to be documenting a new mode of OS configuration which
> actually does count the second named "60", and specifically in order
> that Microsoft systems can say they conform to MiFID II and other
> regulatory agencies that demand conformance with UTC as defined.
> Otherwise the old default configuration will have a second that lasts
> 2000 millis.

I wonder what you mean by "old configuration". The latest leap second
tests I made with Windows and w32time running showed that there was no
leap second handling at all.

This means whenever w32time polled an NTP server the next time after a
leap second (which could be up to 1 week, with a 1 week polling interval
for machines that are not members oof an Active Directory)) it simply
observed a time offset > 1 s, and adjusted the Windows time accordingly.

Slewing the Windows system time over 2 seconds is a feature of the ntpd
port for Windows (and the Meinberg time adjustment service for their PCI
cards), which implements this slewing as a workaround for windows.

If Windows starts to provide an API to handle leap seconds then these
programs should use the API, if it is available, and skip the workaround.

Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] UT1 via NTP

[Martin Burnicki]

Hi Gary,

> Yo Warner!
> 
> On Mon, 19 Mar 2018 14:12:40 -0600 Warner Losh 
> wrote:
> 
>>> Looks to me about a kinda sort maybe 1 micro second per day
>>> shift.
>>> 
>> 
>> Should be closer to 1ms/day. And it is: the above is 4.6ms in 6
>> days or 770us/day.
> 
> Yup, I read it wrong, you read it right.
> 
> Bulletin D, what is broadcast over WWV, is here:
> 
> https://datacenter.iers.org/eop/-/somos/5Rgv/latest/17

If I understand this correctly then the UT1-UTC number in Bulletin A
should better be used, which provides a prediction for the near future:
https://datacenter.iers.org/eop/-/somos/5Rgv/latest/6

Martin
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] UT1 via NTP

[Martin Burnicki]

Rob Seaman wrote:
> One uses the tools as designed for diverse purposes. NTP in a pool
> environment is accurate to a few milliseconds. With our local GNSS
> references something better than a millisecond. With an IRIG shared
> memory reference maybe an order of magnitude better. Hardware time
> capture for our Meinberg IRIG PCIe cards is spec'ed and measured to 5
> microseconds. The PTP versions of those cards quotes something better
> than 1 microsecond. Hardware time capture to OCXO in the Meinberg M1000
> should be precise / accurate to 100 nanoseconds. All of these are
> commercially available with only ordinary attention to metrology, e.g.,
> like understanding the calibration of a precision balance in a lab.
> Meinberg has an excellent monitoring tool built in to their reference
> clocks.
> 
> "Customers" (internal or external) for astronomical timekeeping (but
> should be applicable to other fields) may require TAI, GPS, UTC, UT1, or
> more esoteric things like TT. I can set the Meinberg references to
> deliver all of these except UT1. All the rest would follow. I'd be happy
> to hear about support from other vendors. Needless to say, commercial
> timekeeping vendors should also be expected to implement conforming leap
> second support for their internet attached devices.

ntpd has already built-in support for leap second smearing. This means,
it anyway reads a leap second file as provided by NIST or IERS, checks
automatically if a new file version is available, and then optionally
applies a smearing offset to the time sent to its clients, if configured
to do so. The internal system time would still be UTC, though.

Similarly, it shouldn't be too hard to add some support to read a DUT1
file, and apply the DUT1 offset to the time sent to clients. So you
could easily set up your own UT1 server.

However, similar to the leapsecond file, it has to be made sure that the
DUT1 file is updated as necessary. This could be done automatically by
scripts / wget / curl, similar as it's done with the leap second file.

Of course it would be possible to set up an extra DUT1 server, as Hal
has suggested, but then each client had to get UTC from one source, the
DUT1 offset from another source, and put things together.

A manual or automatic feed of the DUT1 numbers would be required in both
cases, so which approach is to be preferred depends on the accuracy and
other requirements of the tasks that have to be solved.

Martin
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] current / future state of UT1 access?

[Martin Burnicki]

Steven Sommars wrote:
> I regularly monitor the NIST public NTP servers including the UT1 server.
> ut1-time.colorado.edu <http://ut1-time.colorado.edu> reachability was
> good for the past 10 weeks, though the server was briefly in alarm on
> January 22 and March 10.   I can supply details off-list.
> 
> NTP traffic is subject to Internet delay and loss that depends on the
> endpoint IP addresses and often on the client UDP port.  If the client
> NTP daemon (ntpd, etc.) can't contact a server, try to manually poll
> using ntpdate.  

I've also added ut1-time.colorado.edu as "noselect" peer to a local, GPS
controlled ntpd. Even with a 1024 s poll interval my ntpd doesn't
receive a reply for each request, so the "reach" column in the output of
"ntpq -p" is not "377", which is the same for other NIST servers:

~ # ntpq -p
 remote   refid  st t when poll reach   delay   offset  jitter
==
*SHM(0)  .shm0.   0 l58  3770.0000.000   0.000
 ptbtime1.ptb.de .PTB.1 u   79  128  377   28.7871.397   7.650
 ptbtime2.ptb.de .PTB.1 u  129  128  277   30.6081.212   5.261
 ptbtime3.ptb.de .PTB.1 u   75  128  377   29.5362.891   2.165
 resolver2.skyfi 216.xxx  2 u5   64  377  172.3854.483   2.485
 hadb2.smatwebde 192.xxx  3 u   46   64  377  137.7721.627   2.066
 time-a-b.nist.g .NIST.   1 u   95 10245  162.529   -8.114   1.445
 time-b-b.nist.g .NIST.   1 u 1302 1024   42  159.903  -10.609   2.652
 utcnist2.colora .NIST.   1 u  37m 1024  334  161.740   -9.495   3.151
 ut1-time.colora .Nut1.   1 u  675 1024  143  162.015  130.909   1.048

This is not a basic network problem at my side. The servers "resolver2"
and "hadb2" which are public pool servers located in the US have both
reach "377", so these ones reply reliably.

You can simply test this if you run "ntpdate -d -p1 "
repeatedly. When I try this for a NIST server here from Germany I only
receive a reply occasionally, and in most cases I don't. On the other
hand, if I run the same ntpdate command against a pool server located in
the US, this works most of the time, and only very occasionally there is
no reply.

So there seem to be quite strict restrict rules at NIST. The NIST
servers seem to simply not send a reply if they see queries from the
same IP address in short intervals, whatever "short" means. I'm assuming
this is to reduce the high load they probably have to handle.

Please note you need to take care if you have several nodes behind a NAT
router that poll the same server. From the server's point of view it
looks like all the requests from the nodes behind the router seem to
come from the same (public) IP address, so a particular node on the NAT
subnet may receive even less replies.


Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap smear at the IETF

[Martin Burnicki]

Tony Finch wrote:
> There's a discussion about standardizing leap smear at the IETF. Carsten
> Bormann suggested the acronym “Smeared Leapsecond over twentyfour hours" —
> SLOTH.
> 
> https://mailarchive.ietf.org/arch/msg/art/4LSfOGjp_Tb6yQdnhhvK5w3jucU

Thanks for the pointer!

Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds ain't broken, but most implementations are broken

[Martin Burnicki]

Zefram wrote:
> To be clear: obviously knowledge of the upcoming leap second must come
> from some external source.  A system with such advance knowledge is then
> able to apply that knowledge at the proper time, maintaining a correct
> clock through the leap second, without any further external contact.
> It doesn't have to be told that the leap second has arrived.  This kind
> of knowledge is exhibited by some Unices, but not by Windows.

That's exactly the point, IMO.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds ain't broken, but most implementations are broken

[Martin Burnicki]

Brooks Harris wrote:
> On 2017-01-06 11:52 AM, Martin Burnicki wrote:
>>
>>>
>>> - OS kernels with different features (Windows doesn't even know leap
>>> seconds, AFAIK)
>>>
>>>
> 
> It is often repeated on LEAPSECS that "Windows doesn't even know leap
> seconds". That's just not true. It knows very well about Leap Seconds in
> the same way other systems do but it (typical desktop versions) is lazy
> about when it updates - it doesn't attempt to update for Leap Seconds
> until the particular system gets round to syncing to some NTP server
> (defaulting to time.windows.com, but works perfectly well if set to
> time.nist.gov, for example).

Many years ago I wrote a DOS TSR which could read the time from Meinberg
PCI cards (and ISA cards at that time), and adjusted the DOS time if
there was a significant difference.

So of course the DOS time was also adjusted after the plug-in card had
stepped its time due to a leap second.

So according to your argumentation even plain old DOS was aware of leap
seconds, wasn't it?

> In Windows, FILETIME (an unsigned 64-bit value) is directly analogous to
> POSIX time_t except it has a 1601-01-01 00:00:00 epoch. struct
> SYSTEMTIME is analogous to POSIX struct tm ("broken down time" or
> "YMDhms" representation). FileTimeToSystemTime() is analogous to POSIX
> gmtime();
> 
> Like time_t, FILETIME is "bumped" when the system syncs to a NTP server.

Yes because the time adjustment software slews or steps the time.

Which Windows API call can be used to notify Windows that a leap second
is pending, so the Windows system time can account for it at the right time?

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds ain't broken, but most implementations are broken

[Martin Burnicki]

Steffen Nurpmeso wrote:
> Martin Burnicki <martin.burni...@burnicki.net> wrote:
>  |Yes, and there needs to be an interface to timekeeping software in
>  |general, i.e. provide glibc and friends with updated tz rules, and make
>  |the leapsecond table available in a form that can be used by ntpd or ptpd.
>  |
>  |In case of ntpd this means that either a leapsecond file has to been
>  |written so it can be scanned by ntpd, or ntpd needs to support a
>  |different different (binary?) format provided by a tzdist client.
>  |
>  |Anyway, some basic infrastructure is available, but there's still some
>  |work to do.
> 
> Yes, but no in sofar as that "some work" is definetely not enough
> from my point of view.  It may very well be so from your Meinberg
> NTP+ timekeeping point-of-view,

Hm, you but you saw above that I mentioned "timekeeping software in
general" ? ;-)

ntpd is just an example. PTP daemons need this information, too. As I've
recently read, openntpd does not even handle leap seconds properly.

> ... but as a C(/C++) application
> developer you are, i repeat a phrase from a message i have written
> just a minute ago, in blind flight.  Well that brought us to where
> we stand now, but of course this lets aside all the other
> languages with a more complete time interface, and all the special
> options that do exist and which people with special needs have to
> use, e.g., a GPS receiver plus driver, for TAI time.

Yes, I absolutely agree. We have several parts of a puzzle:

- Time synchronization software with different features

- Old protocols which support leap seconds and/or TAI, or not,
eventually in a strange way (NTP's autokey used to provide TAI)

- OS kernels with different features (Windows doesn't even know leap
seconds, AFAIK)

- New protocols like tzdist which can provide the information, but
there's no standard way, AFAIK, how to pass the information to
applications like time synchronization software, time zone libraries, or
directly to the kernel without interfering with other software like ntpd.

So with "some work to do" I meant discussions like currently on this
list, to find proper solutions how to put things together, and of course
have API calls to submit and retrieve information.

I don't think I only have Meinberg and NTP in mind. ;-)

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds ain't broken, but most implementations are broken

[Martin Burnicki]

Brooks Harris wrote:
> On 2017-01-05 05:56 AM, Tony Finch wrote:
>> Martin Burnicki <martin.burni...@burnicki.net> wrote:
>>> Please note that NTP servers not necessarily need to be providers for
>>> leap second files. There are some well known sites which provide this
>>> file, and the NTP software package from ntp.org comes with a script
>>> which can be used to update the file automatically.
>> I was thinking more that an NTP client or server would use its
>> leapseconds
>> file for validating LI bits from servers and for determining when they
>> should leap.
>>
>> My thinking is that routine software patching and security updates happen
>> often enough that maybe NTP can get leap second more reliably out-of-band
>> instead of using in-band leap indicators from upstream servers.
>> Lower-stratum devices could use their own leap second information to
>> correct for operational or implementation errors upstream.
>>
>>> The potential approach with tzdist or special DNS allowed for a
>>> distributed system, where the special DNS can only provide leap second
>>> warning and the current TAI offset, while tzdist also provides the leap
>>> second history, and a way to update time zone rules, so it could be
>>> generally used to keep also conversion to local time correct.
>> Oh, I forgot about the DNS publication scheme. That would also help a lot
>> if it were implemented. And maybe better than relying on sufficiently
>> frequent software updates.
> 
> So, thing is, since 1972, no common and official way to automatically
> obtain the Leap Seconds information has been adopted. Its an obvious
> missing link, missing for 4 decades! I'd find this just incredible
> except I've now come accept this sort of frustration where timekeeping
> is concerned. This has been discussed many times on LEAPSECS.

Hm, I think there was no such requirement when NTP was introduced. Folks
were happy that they could at all forward leap second announcements,
which was sufficient then.

The leap second file was introduced by Judah Levine in 2000, AFAIK, see:
http://tycho.usno.navy.mil/ptti/2000papers/paper34.pdf

This was probably because it became more important to know the history
of leap seconds, and thus the TAI offset. Ntpd's "autokey" feature was
(mis-)used to be able to pass the TAI offset down to clients.

As I wrote in another email, when autokey is obsoleted by NTS then
eventually another new NTP extension field is required to be able to do
this via the NTP protocol.

Anyway, it's quite a number of years.

> Ideally there would be one, and only one, TAI-UTC table, in some very
> well specified form, residing somewhere in cyberspace, administered and
> maintained by official rules and regulations by the IERS. There then
> could be many API's via many technologies to access the information.

Agreed.

> Today, there are many ways to get it, but they are all in different
> forms, not always so well specified, and all require some human
> intervention or oversight somewhere between the IERS announcements and
> the distribution. None of them are particularly "fast", imposing too
> much overhead on the receiver in some circumstances. And "many ways to
> get it" does not inspire confidence that each implementation will get
> everything the same.
> 
> I like the DNS publication scheme because you could imagine that IANA
> could take responsibility for maintaining it, especially if there were
> an official way to keep it automatically updated from a hypothetical
> official IERS source.

Yes, but if you have a greater view of the system then this fixes only
the leap table issue, which may be sufficient for many purposes.

The tzdist protocol, however, has the advantage that it can also
distribute updated time zone descriptions, so you can update everything
from the kernel/TAI up to user spaces applications dealing with local
time in a single service.

You could build up a tzdist server hierarchy similar to NTP stratums or
DNS root and secondary servers to distribute the information, and the load.

> One could imagine, and it would be straight forward, to have NTP servers
> provide the TAI-UTC table, announcements, and expiration via the same
> IPC transport mechanisms used by NTP. Again, hopefully, updated from a
> hypothetical official IERS source.
> 
> I've had the thought that Block-chain would be a good way to do it. It
> would have all the purported anonymity, security, and persistence
> qualities of Block-chain. In such a scheme, only IERS could make updates
> to it and everybody else could read it.
> 
> It makes sense that the Leap Second Table be combined with time zones,
> as it is in Tz Database, because you really n

Re: [LEAPSECS] Time math libraries, UTC to TAI

[Martin Burnicki]

Steve Summit wrote (offlist):
> In some ways I agree that running the kernel on TAI would be far
> preferable.  I've shied away from it so as to be able to answer
> the critics who are always worrying about disconnected systems
> that might not have a source of TAI-UTC at boot time, or at all.

If your system time is disciplined via NTP then currently the TAI/UTC
offset may not be available since the default NTP protocol doesn't
provide it.

As said earlier, you can use tzdist, or DNS, or a future NTP extension
field to pass down the UTC/TAI offset to clients.

If your system time is adjusted via PTP then the TAI offset is available
anyway via the protocol, if the grandmaster has been configured correctly.

You can use the adjtimex() call to retrieve the kernel's TAI/UTC offset,
so applications can determine this.

I'd assume that for disconnected systems this doesn't even matter.

If the kernel runs on TAI, but the TAI time isn't synchronized anyway,
and the TAI offset is 0, then the UTC time stored in inodes matches TAI
time on this system, but who cares?

BTW, AFAICS current Linux kernels increment the TAI offset when they
insert a leap second. I wonder if that is appropriate.

Initially, if the kernel's TAI offset has not been set then it is 0, so
callers of adjtimex() can see that the offset has not yet been set.

However, after a leap second the kernel increases the TAI offset to 1,
which *might* suggest to the unexperienced user that the TAI offset is
valid. Some tests showed that you can *not* set the kernel's TAI offset
to 0 via the adjtimex() call. So I think the kernel should increment the
TAI offset value only if it has been set before, i.e. if it isn't 0, to
avoid confusion.
See also the discussion in:
http://bugs.ntp.org/show_bug.cgi?id=2869

Thoughts?

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds ain't broken, but most implementations are broken

[Martin Burnicki]

Tony Finch wrote:
> Martin Burnicki <martin.burni...@burnicki.net> wrote:
>>
>> Please note that NTP servers not necessarily need to be providers for
>> leap second files. There are some well known sites which provide this
>> file, and the NTP software package from ntp.org comes with a script
>> which can be used to update the file automatically.
> 
> I was thinking more that an NTP client or server would use its leapseconds
> file for validating LI bits from servers and for determining when they
> should leap.

In fact, the current implementation of ntpd uses exclusively the leap
second file if it has been configured, is available and not expired.

However, as I've pointed out earlier, this helps *only* to get the leap
second itself right.

If upstream servers aren't aware of the leap second and don't insert it,
then our server will see that its time is off by 1 sec after it has
(correctly) inserted the leap second. So a few minutes after the leap
second our server will step its time to match the time of its upstream
servers, and thus effectively undo the leap second it has inserted.

That's why a leap second file is only of limited help in cases where
other time sources don't get the leap second right.

> My thinking is that routine software patching and security updates happen
> often enough that maybe NTP can get leap second more reliably out-of-band
> instead of using in-band leap indicators from upstream servers.
> Lower-stratum devices could use their own leap second information to
> correct for operational or implementation errors upstream.
> 
>> The potential approach with tzdist or special DNS allowed for a
>> distributed system, where the special DNS can only provide leap second
>> warning and the current TAI offset, while tzdist also provides the leap
>> second history, and a way to update time zone rules, so it could be
>> generally used to keep also conversion to local time correct.
> 
> Oh, I forgot about the DNS publication scheme. That would also help a lot
> if it were implemented. And maybe better than relying on sufficiently
> frequent software updates.

Yes. Once a time client software has been deployed which can use the
specific DNS (or tzdist), this continues to work as long as the servers
stay reachable, without having to apply software updates (think of
embedded devices).

>> Comparing to your example with DNS: If a root server has a software bug
>> which lets it deliver a wrong IP address, how should your local DNS
>> resolver detect this?
> 
> My analogy was more along the lines of, when a root server IP address
> changes, the DNS server notices and logs that it has out of date hints.
> It's not a great analogy though, because if the DNS server has the wrong
> data about one root server, it can recover using the other 12, but if an
> NTP server has wrong data about the next leap second, it's screwed.

Again, I find this comparison not quite appropriate, you write:

If a DNS server has the wrong data about one root server, it can recover
using the other 12

Similarly, you can configure an NTP server to use several upstream NTP
servers, so if one of them fails, our server can discard it and use the
remaining ones.

IMO a stratum 1 NTP servers are more comparable to a root DNS server: if
it fails because its refclock fails then this server is wrong, but
secondary servers can detect this and ignore it, if there's sufficient
redundance as with the 13 root DNS servers.

>>> I wonder if it would be better to set the leap indicator bits to NOSYNC if
>>> the configured leap seconds file has expired.
>>
>> Sounds good at the first glance, but I think this would cause much bad
>> surprise if you have a company network and suddenly all NTP clients stop
>> to be synchronized.
> 
> Well, at that point (end of June or December) they don't know if there
> should be a leap second or not, so they can't reliably tell the right
> time.

If those servers stop providing time then the clients don't have the
right time, either. Instead, the clients would in addition start to
drift apart. That's not what you want.

> Maybe they should fall back to relying on leap indicators from upstream,
> but they need some way to make it obvious they might fail.

They do this if the leap second file is expired, at least since ntpd 4.8.2.

>> The basic problem is more with a stratum-1 server which in many cases
>> gets its time only from a GPS receiver. If the GPS receiver provides
>> faulty leap second information then the NTP server can hardly detect
>> this. Even if it has a current leap second file this wouldn't work.
> 
> I don't understand. If it has a current leap second file, it can use that
> to detect that its GPS receiver screwed up the leap second. It should then
> 

Re: [LEAPSECS] Leap second smearing test results

[Martin Burnicki]

Hal Murray wrote:
> 
>>> I don't think the client pays any attention to that field in the packet.
>> Hm, when Harlan made the proposal then he said the client did.
> 
> You are right.  There is some code in there.  Thanks.
> 
> It might be interesting to see what the RFC or book says about that area.  
> I'd like to understand what  it does in the non-leap case and/or why.

Dito. Unfortunately I haven't had the time to dig into it, yet.

I also wasn't even aware that the server can suggest a poll interval to
the client before Harlan mentioned this in his proposal.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds ain't broken

[Martin Burnicki]

Michael Shields via LEAPSECS wrote:
> On Wed, Jan 4, 2017 at 1:46 AM, Martin Burnicki
> <martin.burni...@burnicki.net> wrote:
>> I agree it would be good to implement this in an extension field, but
>> generally care should be taken not to break compatibility. I mean, how
>> do existing clients act if they receive a packet with unknown extension
>> field? Do they just ignore it, or do they just drop and ignore the whole
>> packet?
> 
> Another problem is that many ISPs are now filtering port 123 to
> packets with a specific, limited range of packet sizes.  Adding any
> extension fields can cause the NTP replies to be blocked entirely.

Ouch. Never heard about that.

But this may also mean that today's NTP service wouldn't work correctly
if you tried to use authentication. So this sounds like a broken
configuration at the ISP side.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds ain't broken, but most implementations are broken

[Martin Burnicki]

Tony Finch wrote:
> Martin Burnicki <martin.burni...@burnicki.net> wrote:
>> Tony Finch wrote:
>>>
>>> Even though NTP can represent current UTC correctly, it often gets leap
>>> seconds wrong. It does not give confidence that we will be able to
>>> reduce bugs by teaching more code about leap seconds, when NTP cares
>>> about time and gets it wrong, and most code cares much less.
>>
>> I think this you statement isn't quite fair.
> 
> Probably :-) But I think there are ways to make software more easy to
> operate correctly.

I think e.g. ntpd already implements a lot to make operation as safe as
possible.

> For example, it used to be the case that you had to explicitly configure
> DNS software with the addresses of the root name servers. Nowadays, the
> root hints are compiled in and maintained as part of the usual software
> update mechanism, so the operator has one less thing to worry about.

Yes, and whenever someone tries to ship some time synchronization
software with hardcoded of NTP server addresses then this is considered
abusive, and it's exactly that. ;-)

I think you have to distinguish here. For DNS resolvers (at least at the
top level, ISP, etc.) it is *required* that they know the addresses of
some root server, otherwise DNS might not work at all.

> Maybe a similar scheme would work for leap second files, except that NTP
> servers tend to need much less care and feeding than DNS servers.

Please note that NTP servers not necessarily need to be providers for
leap second files. There are some well known sites which provide this
file, and the NTP software package from ntp.org comes with a script
which can be used to update the file automatically.

The bad thing here is that it probably causes quite some load on these
servers if every single NTP node would regularly check for a new leap
second file.

The potential approach with tzdist or special DNS allowed for a
distributed system, where the special DNS can only provide leap second
warning and the current TAI offset, while tzdist also provides the leap
second history, and a way to update time zone rules, so it could be
generally used to keep also conversion to local time correct.

>> IMO this is similar to ntpd. If it's not provided with an updated leap
>> second file then it may have no idea that a leap second is approaching.
>> If a faulty GPS receiver passes a leap second warning to ntpd, should
>> ntpd not trust the GPS receiver since it knows there are some broken
>> receivers out there?
> 
> At this point you can insert another copy of my unfair complaint, but with
> GPS instead of NTP :-)

Agreed. :-)

Some of the 3rd party GPS firmware bugs we have seen indicate that the
software developers haven't worked carefully enough. But again, a
program like ntpd which has to rely on information e.g. from GPS
receivers can't be made absolutely safe against GPS receiver bugs.

Comparing to your example with DNS: If a root server has a software bug
which lets it deliver a wrong IP address, how should your local DNS
resolver detect this?

>> Current versions of ntpd accept a leap second file if it has been
>> configured, and the file hasn't expired. If no leap second file can be
>> used then a leap second announcement from a refclock is used.
> 
> I wonder if it would be better to set the leap indicator bits to NOSYNC if
> the configured leap seconds file has expired.

Sounds good at the first glance, but I think this would cause much bad
surprise if you have a company network and suddenly all NTP clients stop
to be synchronized.

Instead, current versions of ntpd emit a log message when the leap
second file expires, or is going to expire.

>> This tries to make operation as safe as possible, but this doesn't even
>> help in any case. Imagine your NTP daemon has a valid leap second file,
>> handles the leap second correctly, and also passes the leap second
>> warning to its clients.
>>
>> If this daemon's time sources (GPS receiver, or upstream NTP server(s))
>> don't insert the leap second at the same time then our daemon will
>> observe a sudden 1 s offset after the leap second, and even though it
>> has itself handled the leap second correctly, it will step the time a
>> few minutes later to the wrong time provided by its reference time
>> source(s).
> 
> Shouldn't it treat incorrect upstreams as false tickers? Can't it drop
> them from its list of candidate servers when it realises they got the leap
> second wrong or they will get it wrong?

The basic problem is more with a stratum-1 server which in many cases
gets its time only from a GPS receiver. If the GPS receiver provides
faulty leap second information then the NTP server can hardly detect
this. Even if it has a current leap second file this would

Re: [LEAPSECS] Leap seconds ain't broken, but most implementations are broken

[Martin Burnicki]

Steffen Nurpmeso wrote:
> Martin Burnicki <martin.burni...@burnicki.net> wrote:
>  |Steve Summit wrote:
>  |> Tony Finch wrote:
>  |>> Even though NTP can represent current UTC correctly, it often gets leap
>  |>> seconds wrong. It does not give confidence that we will be able to
>  |>> reduce bugs by teaching more code about leap seconds, when NTP cares
>  |>> about time and gets it wrong, and most code cares much less.
>  |> 
>  |> I feel a little bit like I'm rearranging deck chairs on the
>  |> Titanic with this kind of suggestion (and I know this isn't the
>  |> NTP list), but the appalling number of NTP servers that got it
>  |> wrong this time around made me think that another extension NTP
>  |> needs is a way to announce an upcoming leap second (i.e. one it
>  |> knows about) more than 24 hours in advance, so that those of us
>  |> who care can confirm that the servers we're using are going to
>  |> get it right.
>  ..
>  |I don't believe this is necessary. If the existing ways to announce a
>  |leap second are used correctly then ntpd gets it right, in that it
>  |notices the kernel of the upcoming leap second, and also forwards the
>  |announcement out to its clients.
>  ...
>  |There is the new tzdist protocol which could be used to update the leap
>  |second table automatically, and there is a proposal by Poul-Henning Kamp
>  |how this could be done via specific DNS queries:
>  |http://phk.freebsd.dk/time/20151122.html
> 
> And especially that the information of a leapsecond is permanent
> is notable; tzdist added leapseconds, but it requires some
> processing power (xml, json), unless it adds an easy standardized
> binary access protocol, like e.g., CBOR (this becoming popular
> across IETF, so it may only be a matter of time).

Yes, and there needs to be an interface to timekeeping software in
general, i.e. provide glibc and friends with updated tz rules, and make
the leapsecond table available in a form that can be used by ntpd or ptpd.

In case of ntpd this means that either a leapsecond file has to been
written so it can be scanned by ntpd, or ntpd needs to support a
different different (binary?) format provided by a tzdist client.

Anyway, some basic infrastructure is available, but there's still some
work to do.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap second smearing test results

[Martin Burnicki]

Warner Losh wrote:
> I'd be curious what a longer smear time would do? Longer smears would
> give a smaller frequency error, which might be easier to digest. It
> also copes better with long-polling intervals in clients at the
> expense of a longer phase error in the clients.

I'd expect that the shape matters less if the smear interval is
sufficiently longer than the poll interval(s), simply because the time
offset changes from one poll to the next are smaller, as could be seen
in the initial tests with the fixed poll intervals.

> I'd have to say that any hope of recovering actual UTC on the clients
> that are smearing is likely in vain. Too many steering loops involved
> to get a good, stable, reliable, predictable answer at any given
> moment, even if you on the average get decent behavior. If you need
> UTC, you must count in UTC and lie to the clients on your machine
> directly if you are smearing for their sake.

Agreed. But as I've said earlier, the current way of smearing is just a
hack to work around other potential problems.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap second smearing test results

[Martin Burnicki]

Zefram wrote:
> Martin Burnicki wrote:
>> I've run some more tests with smearing of leap seconds,
> 
> These new ones, with variable polling interval on the client, are much
> more useful than the former ones with fixed polling interval.  It seems
> to me that these measurements should concentrate on clients with default
> settings, because if one is able to configure the client to better follow
> the smear then one could easily do an even better job by configuring
> the client to follow an honest server and to introduce the smear itself.

Yes, that's basically what UTC-SLS does, and what has been discussed
here for modifications in the kernel. However, this requires fiddling
with *each* individual client right now, where no out-of-the-box
solution is available for the client side.

> Only the case where the client can't be specially configured is really
> relevant to the concept of introducing the smear upstream.

That's the intention. It's just a workaround for now.

> I'd like to see a run of this client type with the 24 hour smears
> that you used earlier, or of a fixed-polling-interval client with the
> new 10 hour smears, so that variable polling interval can be directly
> compared against fixed polling interval.  I'd interpreted your earlier
> tests generally based on the poll=10 client, in the expectation that
> the default variable polling interval would behave similarly, but it's
> not clear whether the reductions of polling interval that you see would
> result in a significant difference in performance.
> 
> It is strange that the polling interval varies so much more with the
> linear smear than with the sinusoidal smear.  I would have expected them
> to remain stable at poll=10 for the bulk of the smear, after they've
> recovered from the initial frequency change.  There is a point 2.5
> hours into the smear where all the clients have returned to poll=10;
> can you explain why they reduce it again after that?

Unfortunately not. More tests to be done. Especially the cases where the
server sends "poll 4" to its clients during the smear interval. On the
one hand this seems to have an affect on the client, in that the
client's poll interval decreases more than without it. On the other
hand, why does the client poll only decrease to 6, even though the
server suggests 4, and why is the client increasing its poll interval
when the frequency seems to become "stable", even though the server
still keeps suggesting "poll 4"?

> Also strange that the three clients had such different reactions to the
> end of the linear smear, having been so similar in their reactions to
> the start of it, and similar in all parts of the sinusoidal smear.

I think a problem with the linear smear is that the frequency does *not*
change during smear interval, so the clients assume everything is fine
again and ramp up their poll interval more or less. Then suddenly
smearing stops, the frequency suddenly changes to its original value,
and it depends on the time until the next poll how the client reacts.

If the client has a long poll interval and/or polls only lately next
time after smearing has stopped the time offset due to the changed
frequency has increased much more than with a client that polls next
time shortly after smearing has stopped.

> It is interesting that when the server suggests a smaller poll interval
> the shape of the smear makes a big difference to the tracking accuracy.
> This is much like the difference that we intuitively expected the shape
> to make in the first place.

Not quite. The test with the fixed polling intervals have shown that the
shape matters less if the smear interval is sufficiently large compared
to the poll interval.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap second smearing test results

[Martin Burnicki]

Again, sorry for replying so late. I've been mostly offline over the
holidays.

Zefram schrieb:
> Martin Burnicki wrote:
>> https://www.meinberg.de/download/burnicki/ntp_leap_smearing_test_results.pdf
> 
> This is interesting.  The smear actually achieved on the downstream ntp
> is a complicated function of the smear introduced upstream.  Your graphs
> nicely illustrate the lesson that Google has imparted recently, that
> there's little advantage in a sinusoidal shape for a slow smear.  It also
> shows that the smear isn't really as successful as one might expect:
> the smeared clock isn't being tracked without significant error.

Indeed. As expected this depends on the server's smear interval and
smear shape, on the client's poll interval, if the server starts
smearing just after the client has polled, or shortly before it polls
next time, etc.

However, my initial motivation to run these time-consuming tests was
that I read that some folks thought the could just smear the leap second
over 2000 seconds or so, and I didn't believe this works properly with
standard NTP clients.

> An immediate implication is that, if one needs to recover real UTC
> from a timestamp generated on a system downstream of such a smear, then
> for precision finer than 100 ms it is not sufficient to just apply the
> inverse of the smear that was introduced.

As I've already mentioned in another reply, I think this kind of leap
smearing is just a hack, an ugly workaround for clients which are
seriously affected by leap seconds.

For example, we have a customer running an old Linux system with the
kernel bug where it just hangs when the leap second is inserted. Even
though they know there is a patched kernel available there are some
specific reasons why they can't apply any update. So letting the server
smear the leap second avoids problems with that machine.

I don't think clients of a smearing server should try to undo the
smearing. Rather, the time distribution should be set up such that most
of the stuff works as usually, without smearing, and smeared time should
only be supplied to machines which can't handle leap seconds for some
reason.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds ain't broken, but most implementations are broken

[Martin Burnicki]

Michael.Deckers. via LEAPSECS wrote:
>Why doesn't the NTP message include the TAI - UTC offset used for
>the UTC timestamp in the message?  Even a faultily configured server
>knows when it changes this offset, and it could help avoid the
>interpretation of incorrect warning bits.

I think nobody thought they had to deal with TAI when the packet format
was designed.

Support for TAI offset was introduced much later, in NTPv4, when autokey
and support for the NIST leap second file was introduced.

BTW, it's also an advantage that the packet format hasn't changed. With
PTP/IEEE1588, where not even PTPv2 is compatible with PTPv1, even though
both standards are new, compared to NTP, NTP packets have been
compatible across several protocol versions, for many many years.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] JD & MJD, UT1 & UTC

[Martin Burnicki]

John Sauter wrote:
> On Wed, 2017-01-04 at 15:22 +0100, Martin Burnicki wrote:
>> John Sauter wrote:
>>> I did some experimenting with this on Fedora 25, Linux kernel 
>>> 4.8.15- 300.fc25.x86_64.  I found that adjtimex sets STA_NANO
>>> and returns nanoseconds only when NTP has been running for a
>>> while. John Sauter (john_sau...@systemeyescomputerstore.com)
>> 
>> Ah, interesting.
>> 
>> Did you also have a chance to see if the kernel returns
>> microseconds rather then nanoseconds if ntpd has *not* set this
>> flag?
>> 
>> Thanks, Martin
> 
> Yes.  I determined that if NTPD has been running for only a short
> time the value returned is microseconds and the STA_NANO flag is
> not set.

Ah, OK, so the kernel handles this indeed dynamically.

> The code looks like this:
> 
> /* Subroutine to return the current UTC time in a tm structure *
> and the number of nanoseconds since the start of the last second.
>  */ int time_current_tm_nano (struct tm *current_tm, int
> *nanoseconds) { struct timex current_timex; int adjtimex_result;
> 
> /* Fetch time information from the kernel.  */ current_timex.status
> = 0; current_timex.modes = 0; adjtimex_result = adjtimex
> (_timex);
> 
> /* Format that information into a tm structure.  */ gmtime_r
> (_timex.time.tv_sec, current_tm);
> 
> /* If the kernel told us we are in a leap second, increment * the
> seconds value.  This will change it from 59 to 60.  */ if
> (adjtimex_result == TIME_OOP) { current_tm->tm_sec =
> current_tm->tm_sec + 1; }
> 
> /* Return the number of nanoseconds since the start of the last *
> second.  If the kernel's clock is not being controlled by * NTP it
> will return microseconds instead of nanoseconds.  */ if
> (current_timex.status & STA_NANO) *nanoseconds =
> current_timex.time.tv_usec; else *nanoseconds =
> current_timex.time.tv_usec * 1e3;
> 
> return (0); }

Thanks. That's basically just how I would implement it.

> During the recent leap second I also verified that adjtimex returns
> TIME_OOP.

Yes, I've observed that, too. I just wasn't sure how STA_NANO is
handled by the kernel.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] alternative to smearing

[Martin Burnicki]

Michael Rothwell wrote:
> The Gregorian calendar doesn't mess up how computers keep track of time,
> like leap seconds do. Neither do time zones. Leap seconds are different
> -- a special kind of awful.

I think this depends on what your basic time is, and what time you
derive from it.

If the system time runs on UTC, and has current tz rules available, then
you can easily derive local time from UTC. However, without a DST status
you can't distinguish a timestamp like 2:30h in the repeated hour from
the same time 2:30h in the original hour when DST ends.

If your computer ran on local time (like legacy DOS) without a more
monotonic basic time like UTC then you had the same kind of problems
with the end of DST as with a leap second because the basic time is just
stepped back.

On the other hand, if your computer's system time was TAI, and you had a
table of leap seconds ("leap second rules") available, you could do the
conversion to UTC as unambiguously as you can derive a local time from
UTC right now.

So it depends on the implementation.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] JD & MJD, UT1 & UTC

[Martin Burnicki]

John Sauter wrote:
> I did some experimenting with this on Fedora 25, Linux kernel
> 4.8.15- 300.fc25.x86_64.  I found that adjtimex sets STA_NANO and
> returns nanoseconds only when NTP has been running for a while. 
> John Sauter (john_sau...@systemeyescomputerstore.com)

Ah, interesting.

Did you also have a chance to see if the kernel returns microseconds
rather then nanoseconds if ntpd has *not* set this flag?

Thanks,
Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Time math libraries, UTC to TAI

[Martin Burnicki]

Steve Summit wrote:
> Martin Burnicki wrote:
>> IMO another clock type to be used with clock_gettime() would help, as
>> already proposed by Markus Kuhn (there was a link to this in a recent
>> post here), which could be used by new or actively maintained software.
> 
> Sure.  (But are you talking about CLOCK_UTC, or yet another new id?)
> At any rate, yes, ntpd is certainly a fine candidate for being
> modified to use clock_gettime and special clockids.

I have no particular ID in mind.

Generally there are 2 ways:

1.) You could leave the existing API with CLOCK_REALTIME unchanged, and
introduce a new clock ID to return "smeared" time, or ...

2.) You could change the existing API to let CLOCK_REALTIME return
"smeared" time, and introduce a new clock ID (e.g. "CLOCK_UTC") to
return "true" UTC time (whatever "true" means for timestamps taken
during a leap second).

Both approaches will probably cause unexpected behavior for certain
applications, but my feeling is that if you implement the 2nd approach
then at first the time synchronization software needs to be modified to
use the new API, while most existing applications wouldn't even notice
that CLOCK_REALTIME behaves differently, except that they don't observe
a timestep back when a leap second is inserted.

If we don't look only at the kernel and ntpd, but also consider PTP,
then there's still the question if if wouldn't be better to let the
kernel time run on TAI, and derive true and/or smeared UTC from it.

Currently the TAI/UTC offset is transmitted by the PTP protocol, ntpd
passes the TAI offset to the kernel if the offset is known, e.g. from a
leap second file, or from TAI information transferred by NTP's autokey
extensions.

BTW, autokey is going to be obsoleted and replaced by the new Network
Time Security (NTS) protocol extension, so there should be a new
extension field defined for NTP to be able to transmit the TAI offset.

> (I went down the road I did because one of my secondary goals was
> to rig things up so that you could successfully use a modified --
> fully UTC-aware -- kernel along with an unmodified ntpd.  At
> first I was thinking of having the smearing scheme be adjustable
> on a per-process basis, with ntpd's process getting 'unsmeared'
> even for CLOCK_REALTIME.  But then I realized that since ntp --
> I thought -- used adjtime for everything, I could use the trick
> I described.  Oh, well.)

Yes, see my other reply to Zefram, and my thoughts on SO_TIMESTAMP and
friends.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Time math libraries, UTC to TAI

[Martin Burnicki]

Zefram wrote:
> Martin Burnicki wrote:
>> Of course it would be great if there was an API call which executes fast
>> to yield a high performance, and returns a timestamp plus an associated
>> status code consistently, in an atomic operation.
> 
> A read-only adjtimex() does return all of this information, at least on
> systems where struct timex includes the current time.  (Where it doesn't,
> there's no atomic way to get this information.)  Obviously it's more
> expensive than a mere clock_gettime(), but by how much?  One would think
> that just reading the state would be relatively cheap; nothing needs to
> be recomputed.

Agreed. However, I think the blocking mechanisms required to implement
this atomically may require lots of the execution time.

I have never tested how long the calls take to execute, and which
latencies can arise if you either of these functions in the middle of a
status update or so.

Also I'm not too familiar with the way clock_gettime() has been
implemented, but if I remember correctly it uses some tricks to make
access also from user space it really fast.

Today there are many users today which require a high number of request
per second to be serviced, with very high accuracy and resolution, so
that there are also hardware implementations of the NTP service.

Software implementations like ntpd also try to use SO_TIMESTAMP and
friends to eliminate the processing time in the network stack, etc. If
adjtimex() should be used by ntpd to get raw kernel time then either
SO_TIMESTAMP implementations also had to support this (eventually
alternatively, using a new SO_TIMESTAMP_XX code), or the SO_TIMESTAMP
feature could not be used to reduce latency, which resulted in increased
time error and jitter.

BTW, all of this not only applies to ntpd and friends, but also to PTP
daemon implementations.

> I think the "timestamp plus associated status code" call that should
> exist is actually clock_gettime(CLOCK_UTC).  Once that exists, I'm
> dubious about the utility of any proposed syscall that would just return
> a subset of struct timex.  The whole structure is quite useful for clock
> readers because it lets one construct an uncertainty bound, which is
> not available through any of the purpose-specific clock reading syscalls.

Agreed, by see my thoughts above regarding SO_TIMESTAMP.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] JD & MJD, UT1 & UTC

[Martin Burnicki]

Zefram wrote:
> Steve Summit wrote:
>> And this is eerily similar to the idea of using a struct
>> timespec with a nonnormalized tv_nsec field.
> 
> For that matter, tm_sec==60 is the same class of trick too.  We're just
> varying at which digit position we stuff in an out-of-radix value.

Yes, but the advantage of a nonnormalized tv_nsec field is that it can
be used with "legacy" structures which have "seconds since the epoch" in
the tv_sec field.

On the other hand, this may break existing applications which aren't
aware that the number in tv_nsec can exceed 10, so those might
display strange timestamps. ;-)

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds ain't broken, but most implementations are broken

[Martin Burnicki]

Tony Finch wrote:
> It's pretty embarrassing that one of our main time distribution 
> systems routinely screws up leap seconds.

And it's even more embarrassing that the maintainers of such an
important piece of network infrastructure don't get more support from
vendors, even big ones which have huge benefits of it.

http://networktimefoundation.org/

Martin
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds ain't broken, but most implementations are broken

[Martin Burnicki]

Steve Summit wrote:
> Tony Finch wrote:
>> Even though NTP can represent current UTC correctly, it often gets leap
>> seconds wrong. It does not give confidence that we will be able to
>> reduce bugs by teaching more code about leap seconds, when NTP cares
>> about time and gets it wrong, and most code cares much less.
> 
> I feel a little bit like I'm rearranging deck chairs on the
> Titanic with this kind of suggestion (and I know this isn't the
> NTP list), but the appalling number of NTP servers that got it
> wrong this time around made me think that another extension NTP
> needs is a way to announce an upcoming leap second (i.e. one it
> knows about) more than 24 hours in advance, so that those of us
> who care can confirm that the servers we're using are going to
> get it right.

I don't believe this is necessary. If the existing ways to announce a
leap second are used correctly then ntpd gets it right, in that it
notices the kernel of the upcoming leap second, and also forwards the
announcement out to its clients.

However, as mentioned in my other post, if the information passed to
ntpd is not correct then it can't act correctly.

If you introduce another extension to provide leap second information
then this even becomes more confusing: what if the extension field
announces a leap second, and the upstream server(s) / refclock(s) / leap
second file doesn't? Or any variation of the above?

If you provide your NTP daemon with a *current* leap second file then
your ntpd knows about 6 months in advance if a leap second has been
scheduled, and it also knows the TAI offset it can pass down to the kernel.

There is the new tzdist protocol which could be used to update the leap
second table automatically, and there is a proposal by Poul-Henning Kamp
how this could be done via specific DNS queries:
http://phk.freebsd.dk/time/20151122.html

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds ain't broken, but most implementations are broken

[Martin Burnicki]

Ask Bjørn Hansen wrote:
> [1] As much as I dislike the leap seconds; smearing is only appropriate
> if you specifically have chosen it so they’re not supposed to be in the
> pool.

Agreed. Smearing in the way it is currently done by some Google servers
(and ntpd, if configured accordingly) is just a hack to workaround
problems with applications that are unable to account for leap seconds
correctly, and should only be used in closed environments.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap seconds ain't broken, but most implementations are broken

[Martin Burnicki]

Tony Finch wrote:
> Warner Losh  wrote:
>> We have a
>> specific legacy standard called POSIX that's causing all kinds of
>> issues that pop up when you least expect it
> 
> I haven't mentioned the usual litany of NTP servers getting it wrong,
> including servers run by national time labs. It's pretty embarrassing
> that one of our main time distribution systems routinely screws up leap
> seconds.
> 
> So the blame isn't just due to POSIX, or even mostly. Remember, NTP time
> stamps are equal to POSIX time stamps with a constant offset regardless
> of the number of leap seconds. The difference is that NTP actually
> specifies how to handle leap seconds, and carries leap indicator bits
> alongside the timestamp. 
> 
> Even though NTP can represent current UTC correctly, it often gets leap
> seconds wrong. It does not give confidence that we will be able to
> reduce bugs by teaching more code about leap seconds, when NTP cares
> about time and gets it wrong, and most code cares much less.

I think this you statement isn't quite fair.

If a web server delivered a page with broken HTML code you wouldn't
blame the web server daemon, e.g. apache, would you? It's the task of
the web server admin to configure the server correctly and make sure the
original PHP or HTML code is such that the delivered page isn't broken.

IMO this is similar to ntpd. If it's not provided with an updated leap
second file then it may have no idea that a leap second is approaching.
If a faulty GPS receiver passes a leap second warning to ntpd, should
ntpd not trust the GPS receiver since it knows there are some broken
receivers out there?

Current versions of ntpd accept a leap second file if it has been
configured, and the file hasn't expired. If no leap second file can be
used then a leap second announcement from a refclock is used.

For client nodes without own refclock either the leap second file needs
to be provided, or a majority of the configured upstream NTP servers
need to send a leap second warning flag to let ntpd accept the announcement.

This tries to make operation as safe as possible, but this doesn't even
help in any case. Imagine your NTP daemon has a valid leap second file,
handles the leap second correctly, and also passes the leap second
warning to its clients.

If this daemon's time sources (GPS receiver, or upstream NTP server(s))
don't insert the leap second at the same time then our daemon will
observe a sudden 1 s offset after the leap second, and even though it
has itself handled the leap second correctly, it will step the time a
few minutes later to the wrong time provided by its reference time
source(s).

IMO a basic problem here is that there is no general rule when another
leap second has to be inserted. Instead, it depends on human decisions,
and time servers need to be provided with the new information manually.

BTW, this is similar with time zones: If the time zone files aren't
updated after some rules have changed, the DST conversion for certain
zones may not be correct. Would you blame glibc and friends for this?

I had expected that at least the folks at the time labs would take care
that they have set up their servers correctly, and I agree that it's
very disappointing that this isn't always the case.

Martin
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Time math libraries, UTC to TAI

[Martin Burnicki]

Sanjeev Gupta wrote:
> If I have understood you correctly, this is publicly available, based on
> the IERS Bulletin C:
> 
> https://www.ietf.org/timezones/data/leap-seconds.list

Not to forget the new tzdist protocol which can be used to update the
leap second table dynamically across the network.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Time math libraries, UTC to TAI

[Martin Burnicki]

Steve Summit wrote:
> The answer is that on a system with the mods installed, #1
> happens all the time for the system calls time(), gettimeofday(),
> and clock_gettime(CLOCK_REALTIME).  It does not happen for
> adjtimex, on the assumption that that's what ntpd is using.
> #2 happens only when you call clock_gettime(CLOCK_UTC).

Unfortunately the assumption that ntpd uses adjtimex() to compute the
current time offset is wrong, as far as I have seen. See my other post.

So if the system calls gettimeofday() and clock_gettime(CLOCK_REALTIME)
return smeared time then ntpd also sees smeared time.

IMO another clock type to be used with clock_gettime() would help, as
already proposed by Markus Kuhn (there was a link to this in a recent
post here), which could be used by new or actively maintained software.

I think this is similar to clock_gettime() being used instead of the old
gettimeofday() call which is still available.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Time math libraries, UTC to TAI

[Martin Burnicki]

Sorry for jumping in so late.

Zefram wrote:
> Warner Losh wrote:
>> Except that it does advance... It plays the second twice, once with
>> the pending leap set, and once with it cleared I though.

The current implementation of ntpd just calls clock_gettime(
CLOCK_REALTIME ) to retrieve receive and transmit timestamps, at least
if kernel discipline is enabled.

So which time stamps are seen across a leap second depends on the way
the kernel handles leap seconds. If the kernel just steps the time back
and replays the last second of the day then this is also what NTP
clients get from this server.

ntpd calls adjtimex() AKA ntp_adjtime() only to pass time or frequency
adjustments to the kernel, as well as flags like a leap second warning,
etc., or the TAI offset, if it is available.

The kernel then handles the leap second automatically at the right time,
depending on the implementation in the kernel.

Only the return code or kernel status returned by adjtimex() these calls
tells ntpd if the kernel has already inserted the leap second, or not.

> In practice the flags are not that well behaved.  I see the second played
> twice, but the flag change is not synchronised with the jump of the
> scalar time value.  In the logs I have, the flag changed about halfway
> through the second iteration of the second.  So empirically, the two
> seconds cannot be distinguished purely by means of the packet on the wire.

The problem here is that ntpd has to call adjtimex() in periodic
intervals. On the one hand it must not do this too often (keywords:
tickless kernels, avoid waking up the CPU to save power), but on the
other hand ntpd can only see after the next adjtimex() call that the
leap second has passed.

Of course it would be great if there was an API call which executes fast
to yield a high performance, and returns a timestamp plus an associated
status code consistently, in an atomic operation.

On the other hand this probably requires some locking mechanism in the
kernel, which is expensive with regard to the execution time of the call.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap second smearing test results

[Martin Burnicki]

Hi all,

I've run some more tests with smearing of leap seconds, and have updated
the PDF with my results:
https://www.meinberg.de/download/burnicki/ntp_leap_smearing_test_results.pdf

It's late here right now and I've just finished the PDF, so I'm going to
reply to some other emails in this thread tomorrow. ;-)

Martin
-- 
Martin Burnicki

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Web: http://www.meinberg.de
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

[LEAPSECS] Leap second smearing test results

[Martin Burnicki]

Folks,

I've run some tests with smearing of leap seconds. If you're interested,
you can find the results here:
https://www.meinberg.de/download/burnicki/ntp_leap_smearing_test_results.pdf

Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Web: http://www.meinberg.de
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Time Synchronization in Financial markets

[Martin Burnicki]

Hal Murray wrote:
> 
> i...@bsdimp.com said:
>> Except, last I checked, NTP doesn't use this. And applications don't re-read
>> the zone file info when they are updated, so long running applications will
>> use the old data. Also, that file doesn't have then pending leap second this
>> December in it since the date is too old. 
> 
> ntp has code to read that file.  It takes a
>   leapfile 
> in your ntp.conf
> 
> 
> If it's time for a new one, ntpd checks for a new version every 24 hours.

Yes, but only ntpd 4.2.8 does so, and re-reads an updated file
automatically.

Earlier versions didn't re-read the file. You had to restart ntpd. Also,
ntpd 4.2.6 accepts a leap second file even if it is expired. This was
intentional by Dave Mills. As a consequence this means that ntpd 4.2.6
will *not* handle a leap second even though an upstream NTP server or a
GPS refclock provides a valid announcement, if ntpd has been provided
with an outdated leap second file.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Time Synchronization in Financial markets

[Martin Burnicki]

John Sauter wrote:
> I took the lack of mention of leap seconds to mean that leap seconds
> ere not a problem.  The output of the NISTDC units is an astonishingly
>  accurate 1 pulse per second.  That feeds NTP, which handles leap
> seconds using a table.  As long as the table is kept up to date,
> everyone agrees on each second's name.

I don't think this is true.

On *ix systems ntpd uses the leap second table *only* as one possible
source to determine that a leap second has been scheduled. As a
consequence a leap second announcement flag is included in the replies
to the NTP clients, and is also passed to the OS kernel.

It's the kernel who actually inserts the leap second by stepping the
binary system time back, and as already mentioned in my other reply most
library functions are unable to distinguish the leap second from the
"normal" second, and thus yield the same time 23:59:59 twice.

This is similar to the end of DST, where a particular hour is repeated,
and you don't know if you refer to the first or second occurrence if you
don't take the DST status into account.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Bloomberg announced its smear

[Martin Burnicki]

Tom Van Baak wrote:
>> Actually, cosine with a much higher frequency might be the way to
>> beat the median filter.

Yep.

> If the only application of leap smear is to placate NTP, and if all
> NTP clients use the same hardcoded filter parameters, then, yes, by
> all means, find a higher, optimal frequency.
> 
> But I would worry about defining a future leap smear specification
> based on legacy NTP parameters.

Agreed. No one said that the way smearing is implemented in ntpd is the
optimal solution which should be standardized.

As mentioned in my other reply it is just a hack to help users avoiding
problems wthh leap seconds.

> And it begs the question -- if the
> choice of parameters for a proposed new universal smear standard are
> so rooted in NTP, why not just fix NTP in the first place?

As has been mentioned in a different post ntpd has *no* problems with
leap seconds. It just propagates the information of an upcoming leap
second to its kernel and to its clients.

One of the earlier "leap second problems with ntpd" was that it
announced a leap second at a wrong time, e.g. for the end of September.
However, this was not a problem with ntpd but with faulty GPS receivers
which output a leap second announcement at the wrong time.

You know the discussion how many plausibility checks ntpd should make:
accept a leap second announcement only at the end of June or December,
just because all previous leap seconds happened in these months, or
accept a leap second at the end of *any* month, since basically leap
seconds could be scheduled for any month.

You also know some GPS receivers already added the leap second as soon
as the GPS satellites started announcing it, and thus the time output by
these receivers was off UTC by one second. ntpd is unable to detect
this, and you can't blame ntpd if it starts providing a wrong time just
because its reference time source, the GPS receiver, outputs a wrong time.

BTW, even if ntpd would be able to detect that a leap second
announcement at the end of September was invalid and would ignore it,
this would just delay the problem with the faulty GPS receiver. At the
beginning of October the leap second announcement has been cleared, the
GPS receiver has probably inserted the leap second and thus is off UTC
by 1 second, so ntpd would anyway accept the new, faulty time after a
few minutes. The only real way to fix this is to fix the GPS receiver,
not ntpd.

ntpd includes quite a number of workarounds for limitations in operating
systems, especially in Windows. E.g. Windows knows nothing about leap
seconds at all, so ntpd takes care to slew the system time over 2
seconds when a leap second is inserted. Similarly the leap smear feature
is just a workaround to avoid problems with the OS.

If you'd run a PTP daemon instead of ntpd, and the PTP daemon just
passes the leap second warning down to the kernel the you had the same
problems with applications when the kernel steps the time back, or when
particular versions of the Linux kernel lock up due to the leap second.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Bloomberg announced its smear

[Martin Burnicki]

Harlan,

Harlan Stenn wrote:
> Martin,
> 
> Cosine smearing might need to be a choice.  It's harder to track the
> leap second if you get a sample during when both phase and frequency are
> changing.

Indeed it could be just a choice, but I think it's a very good one.

Leap smearing in general is just a hack to workaround the limitations in
POSIX time which can severely affect client applications, e.g. problems
with databases when the client kernel just steps the time back by 1 s
and thus duplicate time stamps occur.

I agree that it would be better to have a proper solution for this
problem, and I believe that Dave Mills' idea to stop the time during an
inserted leap second, and increment it only by 1 LSB whenever an
application queries the time would solve most of the problems we see.

However, quite some time ago one of the Linux kernel developers said
it's just much more complex (and thus more expensive, with regard to
execution time) to implement this in the kernel.

If all servers used by a client do the smearing in the same way then the
NTP clients will will happily move across the leap second.

We (Meinberg) have quite a number of customers who asked for a solution
for problems with leap seconds, and this works very well if their
corporate NTP servers (e.g. Meinberg LANTIME NTP servers) smear the leap
second. Of course this is only an option if the customers applications
can tolerate that the time is temporarily off UTC, but this is often the
least evil.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Bloomberg announced its smear

[Martin Burnicki]

Tom,

Tom Van Baak wrote:
>> Hm, IMO the advantage of the initial smear approach (24 hours before the
>> leap second) is that smearing is finished as soon as the leap second has
>> occurred, so the beginning of the next UTC day /hour / minute is accurate.
> 
> Martin,
> 
> I suspect your idea only works in cases when the time server is
> instantaneously jamming time to a stateless client. But there is
> often some level of filtering and so the client will necessarily lag
> the server. In this case the client will not have accurate UTC until
> many time constants after midnight.

First please note I've implemented this in ntpd, but it's not my idea.
It's what Google did when they first smeared time.

IMO the advantage of the cosine approach is that smearing starts slowly,
then increases, and then fades out quite some time before it ends, so
the last samples during the smear interval should pretty much be in
coincidence with real UTC after the leap second. See the measurement
graphs in my PDF.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Bloomberg announced its smear

[Martin Burnicki]

Christopher Hoover wrote:
> This I believe is the most recent public statement on the topic:
> 
> https://cloudplatform.googleblog.com/2015/05/Got-a-second-A-leap-second-that-is-Be-ready-for-June-30th.html
> 
> One could imagine having a public ntp pool on the interwebs that
> implemented smearing.  (I know this will be heresy to some.)

The current version of ntpd supports smearing, if support is explicitly
compiled in and then enabled in the ntp.conf file.

See:
http://bugs.ntp.org/show_bug.cgi?id=2855

This implements leap smearing using a cosine approach, like Google did
it initially, and the smear interval is such that smearing is finished
at the end of the leap second, so the beginning of the next day / hour /
minute is accurate.

And this should *not* be used with pool servers or other public servers.

A summary can be found here:
https://www.meinbergglobal.com/download/burnicki/Leap%20Second%20Smearing%20With%20NTP.pdf

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Leap second to be introduced at midnight UTC December 31 this year

[Martin Burnicki]

Tom Van Baak wrote:
>> The data format only has a "leap second pending flag", which means a
>> leap second is to be inserted.
> 
> Hi Martin,
> 
> Ouch. Well that's a problem. LSEM aside, what are you going to do if the 
> earth continues to gradually speed up as it has the past couple of decades?

I don't know. This must have been overseen by the folks who made the
specs for the DCF-77 data format. This is bad especially since there
were a number of bits which were unused in the original spec but are
nowadays used to transmit some information which is not directly related
to the current date and time.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] [time-nuts] Leap second to be introduced at midnight UTC December 31 this year

[Martin Burnicki]

Tony Finch wrote:
> Martin Burnicki <martin.burni...@burnicki.net> wrote:
>>
>> IMO a better approach would be to let the system time run on TAI, and do
>> the conversion to UTC, and local time in the next step, by runtime
>> libraries.
> 
> The kernel needs to know UTC for things like filesystem timestamps.

I know. This is mainly to avoid ambiguities you'd have with time stamps
in local time, as under plain old DOS.

Using TAI for the file time stamps should be fine as well, it would even
avoid ambiguities for files which are written during a leap second. Of
course, if the kernel could *tell* the time stamps are TAI instead UTC
then tools like 'ls' could display the TAI timestamps in UTC, if required.

If you insist that file times should be UTC then the kernel could
convert the TAI time stamps to UTC before writing the file time stamps.

For example, ntpd already passes the TAI offset down to the kernel if it
knows this number, e.g. from a leap second file, or via autokey from an
NTP server.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] [time-nuts] Leap second to be introduced at midnight UTC December 31 this year

[Martin Burnicki]

Hal Murray wrote:
> 
> s...@ucolick.org said:
>> This idea pushes extra complexity into every implementation of low level
>> kernel-space software, firmware, and hardware.  That's nice as a policy for
>> full employment of programmers, but it's hard to justify by any other
>> metric.  Instead those low level places should be as simple as possible, and
>> that means making the underlying precision time scale, and thus any
>> broadcast distributions of a precision time scale, as simple as possible. 
> 
> Has anybody done any serious thinking about fixing the POSIX problem?
> 
> I assume the first step would be to have the kernel keep time in a non 
> leaping time scale.  Is UT1 the obvious choice?

Wouldn't that basically mean to get back rubber band seconds? Also, how
would you distribute UT1? Via a table with a correction value to TAI?

IMO a better approach would be to let the system time run on TAI, and do
the conversion to UTC, and local time in the next step, by runtime
libraries. The tzdist protocol could be used to update both the TZ DB
and the leap second table.

This could solve the ambiguity of duplicate time stamps which you also
have at the end of a DST interval, unless you know the DST status. I
mean at least the conversion from TAI to UTC.

The conversion from UTC back to TAI can only be unambiguous if you have
some status information with the time stamp. For example, if you want to
derive UTC from local time then you need to know the basic local time
offset *and* the DST status. Otherwise you get ambiguous conversion
results at least at the end of a DST period where 1 hour is repeated.

This is similar with UTC. If the timestamp APIs always returned a
timestamp+status you could deal with that unambiguously. However,
implementation of the API would be expensive with regard to computation
time since it has to make sure that the time stamp and status
information are always returned consistently. This is bad for
applications which want to read time stamps as fast as possible.

> Would we need a version of ntp that used that time scale?  (or some 
> non-leaping time scale)

I think a compatible approach would be to let existing API calls behave
like they do now, and provide new enhanced API calls where the
application can determine if the kernel runs on TAI, and then use
different calls to return a TAI time stamp only, or a raw UTC time stamp
derived from TAI, or a time stamp plus status to resolve the ambiguity.

For example, Meinberg PCI cards provide API calls which return a 64 bit
time stamp very fast for applications which need it, and there are other
API calls which return a UTC time stamp, plus local time offset, plus a
status field indicating if the device is synchronized, if a leap second
is pending, if the current time stamp is in the middle of a leap second,
etc.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] [time-nuts] Leap second to be introduced at midnight UTC December 31 this year

[Martin Burnicki]

Tom Van Baak wrote:
> Time to mention this again...
> 
> If we adopted the LSEM (Leap Second Every Month) model then none of
> this would be a problem. The idea is not to decide *if* there will be
> leap second, but to force every month to have a leap second. The IERS
> decision is then what the *sign* of the leap second should be this
> month.

Although this approach sound good, it would cause major problems for
users of the German longwave transmitter DCF-77. The data format only
has a "leap second pending flag", which means a leap second is to be
inserted. AFAIK there is no spec to announce a negative leap second via
DCF-77.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] [time-nuts] Leap second to be introduced at midnight UTC December 31 this year

[Martin Burnicki]

Warner Losh wrote:
> On Wed, Jul 20, 2016 at 9:27 AM, Martin Burnicki
> <martin.burni...@burnicki.net> wrote:
>> Each bulletin C from IERS says:
>>
>> "Leap seconds can be introduced in UTC at the end of the months of
>> December  or June, depending on the evolution of UT1-TAI. Bulletin C is
>> mailed every six months, either to announce a time step in UTC or to
>> confirm that there will be no time step at the next possible date."
> 
> That's not an official standards document. That's IERS telling us how often
> they will publish their Bulletin C with not a lot of precision. Sometimes it's
> sent out on July 1, other times later, for example.

Agreed, but it clearly states what will happen in about 6 months, and
even if no leap second is currently scheduled there will be no leap
second before the next bulletin C is published.

I think the problems we've observed with some leap seconds is not due to
this practice.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] [time-nuts] Leap second to be introduced at midnight UTC December 31 this year

[Martin Burnicki]

Brooks Harris wrote:
> On 2016-07-20 01:08 AM, Tom Van Baak wrote:
>> I recall this is a known problem in the Z3801A status reporting, and
>> possibly other GPS receivers of that era as well. It stems indirectly
>> from a change years ago in how far in advance IERS and DoD were able
>> to update the leap second info into the GPS constellation. Nowadays
>> it's common to get 6 months notice; it wasn't always that much.
>
> TF.460-6 says:
> "2.3The IERS should decide upon and announce the introduction of a
> leap-second, such an announcement to be made at least eight weeks in
> advance."
> 
> Is there a statement in some document from BIPM or IERS that states
> their current announcement policy? How, when, and why is it different
> from 460? I mean, more lead time is a good thing, but what, exactly, is
> an implementer to expect and what standard would would they look to
> learn and confirm that?

Each bulletin C from IERS says:

"Leap seconds can be introduced in UTC at the end of the months of
December  or June, depending on the evolution of UT1-TAI. Bulletin C is
mailed every six months, either to announce a time step in UTC or to
confirm that there will be no time step at the next possible date."

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] GPS is getting the leap second

[Martin Burnicki]

Steve Allen wrote:
> According to
> http://cruisersnet.net/civil-global-positioning-system-service-interface-committee-future-leap-second-scheduled/
> yesterday the USAF started telling the GPS constellation about the
> upcoming leap second, so receivers are beginning to demonstrate
> reactions to that information.

Yes, the mbgstatus program for Meinberg GPS PCI cards now reports for
the UTC parameters:

--
WNlsf: 1929, DN: 7, offs: 17/18
UTC offset transition from 17s to 18s due to leap second
insertion at UTC midnight at the end of 2016-12-31.
--

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

[LEAPSECS] NTP installer 4.2.8p7 for Windows is available

[Martin Burnicki]

Hi all,

just want to let you know that version 4.2.8p7 of NTP installer for
Windows is now available:
https://www.meinbergglobal.com/english/sw/ntp.htm#ntp_stable

NTP v4.2.8p7 has been published 2016-04-26. It provides some security
patches and some other enhancements. See:
http://nwtime.org/ntf-releases-ntp-4-2-8p7-security-patch/

The installer for Windows also provides openSSL DLL v1.0.1s, which
contains some security fixes as well.

Martin
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] [updated] Bulletin C 51: No leap second in June 2016, but new leap second file

[Martin Burnicki]

Martin Burnicki wrote:
> The International Earth rotation Service (IERS) has just published
> bulletin C 51, announcing that *no* leap second will be introduced at
> the end of June 2016:
> ftp://hpiers.obspm.fr/iers/bul/bulc/bulletinc.dat
> 
> The IERS now also makes a leap second file available for use with NTP:
> https://hpiers.obspm.fr/iers/bul/bulc/ntp/leap-seconds.list
> 
> This new file has the expiration date set to 28 December 2016, so users
> who have configured ntpd to use the leap second file should upgrade
> their local copy of the file.

Unfortunately it turned out that the original version of the IERS leap
second file only had the human readable expiration date set to 28
December 2016, while the machine readable time stamp which is evaluated
by ntpd was 30 September 2016 only.

This has been fixed now, and and a fixed version where both dates are
consistently 28 December 2016 has been published:
https://hpiers.obspm.fr/iers/bul/bulc/ntp/leap-seconds.3661632000
https://hpiers.obspm.fr/iers/bul/bulc/ntp/leap-seconds.list

where the latter is just a symbolic link to the first file, in order to
be able to access the file always under the same name.

> The leap second file does not only announce leap seconds, it also
> provides ntpd and thus the OS kernel with the current TAI/UTC offset,
> which is helpful for applications working with TAI.

Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Web: http://www.meinberg.de
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] embedded data loggers?

[Martin Burnicki]

Rob Seaman wrote:
> Don’t know if this got through using my new email address.
> 
> Have been rummaging around looking at Arduino and/or Raspberry Pi hacks,
> but the former seems under-powered to run a full NTP instance and the
> latter is not real-time.

If you want a cheap solution then maybe you should have a look at the
Banana Pi and/or Beaglebone Black.

The Raspberry Pi's NIC chip is internally connected via USB, which
causes latencies and asymmetries for network packets, which in turn can
decrease the accuracy you can yield via NTP.

The Beaglebone Black and AFAIK also the Banana Pi have NIC chips which
are connected directly to the CPU, not via USB, which avoids the
latencies introduced by USB.

Unlike the Beaglebone Black and the Raspberry Pi the Banana Pi also
supports native 1000 MBit link speed. Eventually you don't have to
transfer much data and 100 MBit sounds enough, but if your switch and
your LAN run on 1 GBit and you connect a single 100 MBit device to one
switch port then all packets from or to the slow port need speed
conversion, which often means queuing by the switch, which in turn also
inserts latencies affecting the resulting accuracy.

In any case you need to spend some of your time to configure and test
this properly, so as Warner has already mentioned in his reply, if you
only need one or a few instances of those loggers it may be easier to
buy a solution which is more expensive, but doesn't require lots of your
time.

So just for completeness I'd like to mention that most Meinberg GPS
receivers provide 2 "time capture" inputs which directly capture the
on-board UTC time derived from the stable on-board oscillator which is
directly disciplined by the GPS input signal.

You can apply 2 different hardware trigger signals. The captured time
stamps are saved in an on-board FIFO buffer and can be retrieved via a
serial port, or in case of a GPS PCI cards, directly via the PCI bus.

Take care, I'm biased. ;-)

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

[LEAPSECS] Bulletin C 51: No leap second in June 2016, but new leap second file

[Martin Burnicki]

The International Earth rotation Service (IERS) has just published
bulletin C 51, announcing that *no* leap second will be introduced at
the end of June 2016:
ftp://hpiers.obspm.fr/iers/bul/bulc/bulletinc.dat

The IERS now also makes a leap second file available for use with NTP:
https://hpiers.obspm.fr/iers/bul/bulc/ntp/leap-seconds.list

This new file has the expiration date set to 28 December 2016, so users
who have configured ntpd to use the leap second file should upgrade
their local copy of the file.

The leap second file does not only announce leap seconds, it also
provides ntpd and thus the OS kernel with the current TAI/UTC offset,
which is helpful for applications working with TAI.

Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Web: http://www.meinberg.de
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

[LEAPSECS] Admins should check expiration date of leap second file

[Martin Burnicki]

Folks,

I've just checked some of the commonly known sources for the leap second
file to be used with NTP and found that some sites on the internet only
provide outdated versions of the leap second file. Some of the files
will expire on 28 December 2015.

So if you have an NTP daemon running and a leap second file specified in
the ntp.conf file you should check if you have a current copy of the
file installed, or otherwise update the leap second file.

Unlike earlier version of ntpd (up to 4.2.6*), current versions (4.2.8*)
of the NTP daemon really evaluate the expiration date of the file, and
don't accept outdated an outdated leap second file.

For many NTP installations this doesn't really matter, but

- if an applications expects a current TAI offset set in the OS kernel
then it won't get the correct information without a valid leap second file

- if a valid leap second file is available then the leap second
information takes precedence over leap second announcements from
upstream servers, etc., so a valid leap second files provides some
protection against wrong leap second announcements from faulty upstream
servers.


The original leap second file from NIST is available via this link:
ftp://time.nist.gov/pub/leap-seconds.3629404800 (current)
ftp://time.nist.gov/pub/leap-seconds.list (link to current)

and the current version expires 28 June 2016. However, unfortunately the
NIST server is often not reachable.


The IERS which decides whether a leap second is to be scheduled or not,
is now also providing an own leap second file:

https://hpiers.obspm.fr/iers/bul/bulc/ntp/leap-seconds.3651350400 (current)
https://hpiers.obspm.fr/iers/bul/bulc/ntp/leap-seconds.list (link to
current)

However, this file already expires on 28 January 2016, which is pretty
early. I've already contacted the folks at IERS and asked them to set
the expiration date to shortly before the next leap second event. I
guess they will provide a new leap second file shortly after the next
bulletin C has been published in January 2016.


The USNO formerly also provided an own version of a leap second file via
ftp://tycho.usno.navy.mil/pub/ntp/

However, on this web page there is only an outdated copy of the IERS
file now, which expires on 28 December 2015. I've already tried to
contact the folks at USNO and asked them to update their server, but
haven't received a reply, yet.


A copy of the current leap second file can also be found in the current
Time Zone data package provided by the IANA. Since the IETF web site
provides an unpacked copy of the tzdata archive the NIST file is also
available here:
http://www.ietf.org/timezones/data/
http://www.ietf.org/timezones/data/leap-seconds.list (shipped with
tzdata, copy of a NIST file)

Since this is a copy of the current NIST file this file is valid until
28 June 2016.

Another copy of the current NIST leap second file is available via the
Meinberg web site:
http://www.meinberg.de/download/ntp/leap-seconds.list


Hope this helps.

Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Web: http://www.meinberg.de
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap second festivities?

[Martin Burnicki]

Poul-Henning Kamp wrote:


In message 5593eb55.6090...@meinberg.de, Martin Burnicki writes:


The maximum offset with the linear smear was almost three times larger
than with the cosine smear. I think there is a different reason why
they switched to the linear smear.


I fear that test has sufficient power to reveal the full picture.

The NTPD PLL is quite temperamental, and when combined with the
median filter and a poll rate of 2048 seconds, things can get quite
interesting.

I'm pretty sure that the linear smear interacts better with the
median filter at high poll intervals.


I'm going to run a test with a linear approach, too.


Please run it with minpoll N maxpoll N for N in (4...11)


Will do. However, for a general picture we'll need results for different 
smear intervals, linear or cosine smearing for each interval, as well as 
several polling intervals on the clients for the combination of each of 
the above.


This will take some time.

Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Did John Oliver just save the leap-second ?

[Martin Burnicki]

Poul-Henning Kamp wrote:

I'm not quite sure about the consequence of this one:

https://github.com/systemd/systemd/issues/437

But it seems possible that large fraction of Linux systems have
been following Googles leap-smear without knowing it because of
systemd.


One of my colleagues has also monitored time{1,2,3,4}.google.com, and we 
saw that those servers were obviously smearing in the way explained in

http://googlecloudplatform.blogspot.de/2015/05/Got-a-second-A-leap-second-that-is-Be-ready-for-June-30th.html

In fact I was pretty surprised to see this since one of the last 
paragraphs on the Google states:


Google does not offer an external NTP service that advertises smeared 
leap seconds.


So I had assumed smearing Google servers were only visible from within 
their cloud services.


Martin
--
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH  Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg, 
Andre Hartmann, Heiko Gerstung

Web: http://www.meinberg.de
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap second festivities?

[Martin Burnicki]

m...@lumieresimaginaire.com wrote:

After waking this morning and reviewing the data I discovered that the
Azure VMs did in fact leap. The data below was taken from a system that
was set up on purpose to NOT leap. So I had no issues with the NTP
server/clients as far a time goes. As indicated below some systems were
reporting tai=1 in ntpq -c rv output after the leap. This appears to
be due to the fact that they no leap second history (no leapsecond file)
and no ref clock. It seems that the current value is not transmitted in
the NTP packets.


Correct. The normal packets only transport a leap second warning flag, 
not a TAI offset. The TAI offset can be determined locally from a leap 
second file, or remotely from an NTP server via autokey. However, for 
the latter there was recently an issue. See NTP bug 2830:


ntpd doesn't always transfer the correct TAI offset via autokey
http://bugs.ntp.org/show_bug.cgi?id=2830

Martin
--
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH  Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg, 
Andre Hartmann, Heiko Gerstung

Web: http://www.meinberg.de
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap second festivities?

[Martin Burnicki]

Miroslav Lichvar wrote:

On Tue, Jun 30, 2015 at 09:42:50PM +0100, Rob Seaman wrote:

Any thoughts on watching Google’s (or anybody else’s) smear in action?  Kind of 
like watching paint dry, but still…


It seems the Google leap smear has finished. Here is a plot of offsets
of some Google servers I included in my leap second monitoring.

https://mlichvar.fedorapeople.org/leap2015/google_smear.png


Similar results here, recorded on a system which did the leap second 
step, so you can see the +/- 0.5 s maximum error:

http://people.ntp.org/burnicki/leap-second-2015-06-30/time1.google.com-216.239.32.15.png

We have also monitored time.windows.com, and the result is really bad:
http://people.ntp.org/burnicki/leap-second-2015-06-30/time.windows.com-23.99.222.162.png

Martin
--
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH  Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg, 
Andre Hartmann, Heiko Gerstung

Web: http://www.meinberg.de
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] 2-hour leap smear in Japan

[Martin Burnicki]

Hi Kevin,

Kevin Birth wrote:

For anyone curious about how Japan is handling the leap second, since it
is occurring at 8:59:59 in the morning on a business day, the following
two articles refer to the solution that Japanese financial exchanges are
using a mini leap smear spread over two hours before the opening bell.


For what it's worth, I've just submitted a patch for ntpd which 
optionally supports Google leap smear:


Bug 2855 - leap smear support for ntpd
http://bugs.ntp.org/show_bug.cgi?id=2855

I really doubt that real NTP clients can follow a server smearing a leap 
second over 2 hours only.


Martin
--
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH  Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg, 
Andre Hartmann, Heiko Gerstung

Web: http://www.meinberg.de
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] W1K GPS rollover for some time servers

[Martin Burnicki]

Hi Tom,

Tom Van Baak wrote:

As seen at
http://lists.ntp.org/pipermail/hackers/2015-May/006866.html
and also as experienced at Keck Observatory last night, some models
of GPS time servers just did their firmware's W1K rollover, so those
are saying the date is 1995-09-17.

But the leap second is, inappropriately, getting the blame!


Steve,

No need to take it personally. At some level it doesn't matter if
it's leap seconds, or the GPS spec, or one GPS manufacturer, or one
particular GPS receiver model, or one OS, or one line of code that
doesn't handle rollover correctly. To the person who wants and
expects time to work right, it's all the same. And any of us who work
with precise time are part of the problem and share the blame.


Hm, in my opinion this is somewhat similar to when a good product sold 
via an online shop gets a bad rating even though the product itself is 
very good, but it took very long for the vendor to ship the product.


Of course leap seconds can cause lots of trouble, but nevertheless it's 
not OK if the leap seconds are blamed for every firmware bug.


I've recently seen a 3rd party GPS puck which applied the UTC correction 
parameters wrong, and thus the generated 1 PPS signal was off by more 
than 1 microsecond. After a discussion with the manufacturer it turned 
out that they had computed the time (t - t0t) from the UTC correction 
parameters wrong during a certain week number and thus the polynomial 
which computes the time correction returned a wrong value.


Even though these correction parameters are in the same parameter set 
the UTC seconds offset, this is not related to leap seconds, just to 
incorrect handling of the week numbers.


So eventually week numbers should also be avoided since some developers 
are unable to work with them correctly. ;-))



What you can do is represent the astronomical community and do what
you can in your professional career to promote clean, robust, and
redundant timekeeping. That can either be passive education or active
steering the future away from sextants, s/w radio, and other outdated
methods of astronomical timekeeping. That doesn't make problems
vanish right away, it helps reduce risk in the future.

As you know the GPS folks enlarged the 10-bit week number in the next
signal spec so receiver manufacturers have less rope to hang
themselves. One step at a time, but at least it's a step.

The shame in today's example rests on makers of TymServe 2100, who
either didn't test their firmware, or who knowingly allowed a product
to have this bug. And worse yet, are now refusing to support it,
because it's out of warranty. Hopefully a two-line fix to NTP can
be used to get around the bug. OTOH, I can't believe the Keck guys
are reliant on a single GPS receiver for their time? May I bring them
another clock or two for them to use?


Isn't the TymServe 2100 just using a GPS puck from a different 
manufacturer? Then the manufacturer of the TymServe has to rely on the 
GPS puck to work correctly, and the manufacturer of the GPS puck is to 
blame, and the TymServe guys don't even know if the GPS puck will show 
up with some other bug at some time.


Of course it's also a shame that the TymServe manufacturer just says the 
device is out of support.


Martin
--
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH  Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg, 
Andre Hartmann, Heiko Gerstung

Web: http://www.meinberg.de
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] My FOSDEM slides

[Martin Burnicki]

michael.deckers via LEAPSECS wrote:


On 2015-03-03 21:05, Martin Burnicki wrote about
negative leap seconds:


 In the 7 year interval where no leap second was required/scheduled I
heard
 several people saying we might have needed a negative leap second.


Fortunately, this is not a matter of speculation.


I know.


An easy way to
see the trend of UT1 - UTC is to look at DUT1 (published in
Bulletin D). DUT1 is an approximation to UT1 - UTC and has
always stepped down (except, of course, at positive leap seconds),
ever since the earliest Bulletin D available on the web (1991-06-20).


Yes, but if you observe that DUT1 steps down slower and slower you can 
speculate whether it would start to step up over some years.



Before a negative leap seconds would be scheduled, we would see
DUT1 stepping up several times in a row, so there _is_ some
advance warning.


Agreed.


Martin
--
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH  Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg, 
Andre Hartmann, Heiko Gerstung

Web: http://www.meinberg.de
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] My FOSDEM slides

[Martin Burnicki]

Joseph Gwinn wrote:

On Tue, 03 Mar 2015 21:56:42 +0100, Martin Burnicki wrote:

Now in 4.2.8 the leap second code has been reworked and cleaned up,
and a very quick look at the source code seems to indicate that this
might work again. At least there's code which passes the announcement
flag to the kernel, if kernel support is enabled.

I think I'll give it a try soon. I'd expect that a negative leap
second might work if an appropriate announcement is received from a
refclock or upstream NTP server, but it will be interesting to find
out if this works with a NIST-style leap second file where the TAI
offset decreases at a given date.


Hell - lots of code can't handle a positive leap second, so what hope
is there?


Should we abandon everything which can't easily be handled, or where 
developers don't work thoroughly?



Right now, GPS System Time is the best solution.  In ten years, I bet
the answer will be TAI.


Agreed.

And we might already start to think about how to get this right in mixed 
environments, e.g. using the NTF's General timestamp API, or find a way 
to determine if the kernel time is UTC, or TAI.



Martin
--
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH  Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg, 
Andre Hartmann, Heiko Gerstung

Web: http://www.meinberg.de
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] My FOSDEM slides

[Martin Burnicki]

Tom,

Tom Van Baak wrote:

GPS can deal with it, even IEEE 1344 and C37.118 time codes can, but I'm
not sure if WWVB can. Anyway, I know the German DCF-77 transmitter has
no flag defined to announce a negative leap second, so there would be
major problems if one had to be inserted.


Both WWVB and DCF-77 have a single leap second warning bit. With WWVB you look 
at the sign of DUT1 to know if the leap is insert or delete. So WWVB is fine.


Thanks for the hint.


DCF-77 has no DUT1 or leap gender bit so, yes, there will be trouble. I'm sure 
PTB will figure something out if it ever comes to that.


Hm, the question is if existing receivers could handle what they figure 
out, and I wonder if existing WWVB receivers check the sign of DUT1 to 
determine if an upcoming leap second is +1 or -1.


Martin
--
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH  Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg, 
Andre Hartmann, Heiko Gerstung

Web: http://www.meinberg.de
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

[LEAPSECS] FW: Rescheduled: Leap seconds webinar - 10 March 2015

[Martin Burnicki]

Title: Rescheduled: Leap seconds webinar - 10 March 2015


The Sciencewise webinar on ‘Leap seconds’ with Paul Broadbent, which had been originally scheduled for Wednesday 5 November 2014, has now been rescheduled.Martin Burnicki-

  

  



  

  

  If you’re having problems viewing this message, please click here.

  
			
 
			







 Rescheduled: Leap seconds webinar - 10 March 2015 







  

  


Register for the re-scheduled Sciencewise webinar on Leap Seconds on Tuesday 10 March, 1:00PM - 2:00PM (GMT). The webinar will explore the lessons learnt from the National Measurement Office's (NMO) dialogue project to inform ministerial decision on how the UK should vote on maintaining the addition of leap seconds. The aim of the dialogue was to determine the importance to the public of a link between time and the Earth’s movement. Register for webinar The webinar will feature a talk from Paul Broadbent from the NMO on why the organisation commissioned the dialogue, how it sought to engage the public and how the findings from the project will be used to feed into UK policy on this internationally controversial issue. The webinar will cover:• The value of the public dialogue• How it helped/will help to create a better policy• What, if any, surprises the dialogue uncovered• How well it was received internally – before, during and after • Any key learnings or top tips Who should attend?The session is designed to provide guidance to policy makers who are curious about/or actively looking to engage the public on policy issues involving science and technology.




  






 Future events 







  

  
  





			
 
			





 Introduction to public dialogues - engaging the public in complex and controversial policy issuesJoin Sciencewise on Tuesday 17 March, 1:00 PM - 2:00 PM (GMT), for a webinar on the essentials that policy professionals need to know about public dialogue, including an overview of what public dialogue is, what it means in practice and the impact it can deliver. Participants will be able to explore a range of real-life case studies, consider where and when it might be relevant for their own work, and understand the financial support and services that Sciencewise can offer policy makers on public dialogue. Register for webinar.As part of a wider programme of open policy-making training in partnership with government departments, Sciencewise will be hosting a number training sessions in the coming weeks. For more information on dates and training, click here.



  


  
  


  
  



Follow us on Facebook 







Follow us on Twitter 



  

  


  


SciencewiseSciencewise helps the Government engage with the public 

Re: [LEAPSECS] My FOSDEM slides

[Martin Burnicki]

Joseph Gwinn wrote:

On Thu, 26 Feb 2015 15:09:01 +0100, Martin Burnicki wrote:

Hi folks,

I've been asked off list to make the slides of my presentation at
FOSDEM 2015 in Brussels available and post the download link on this
list.

So here it is:


https://fosdem.org/2015/schedule/event/technical_aspects_of_leap_second_propagation_and_evaluation/


See the Attachment link.


Very interesting; thanks for posting this.

I have a few questions and comments:

1.  Slide titled Known Bugs (2): Has support for negative leap
seconds been restored in NTP v4?  It wasn't clear.


I have to admit that I wrote this before 4.2.8 had been released. 
Support for negative leap second has been in older NTP versions, but had 
been removed in 4.2.6.


Now in 4.2.8 the leap second code has been reworked and cleaned up, and 
a very quick look at the source code seems to indicate that this might 
work again. At least there's code which passes the announcement flag to 
the kernel, if kernel support is enabled.


I think I'll give it a try soon. I'd expect that a negative leap second 
might work if an appropriate announcement is received from a refclock or 
upstream NTP server, but it will be interesting to find out if this 
works with a NIST-style leap second file where the TAI offset decreases 
at a given date.



2.  Slide titled Possibilities for Future Improvements (2):  In the
wish list for a kernel call to ask if the kernel runs UTC or TAI, a
couple of issues come to mind.  First, many systems set the GPS
receiver to emit GPS System Time via NTP (and IRIG), so a GPS System
Time option may be needed.


Yes.

Though I would prefer using TAI instead of raw GPS time if a linear time 
scale is required. What if you use a different GNSS receiver, e.g. for 
Galileo, or the Chinese Beidou?


GPS time (or whatever) is fine in closed projects/environments, but IMO 
a UTC and TAI are the global time scales, while GPS is specific to the 
U.S.



Second, we often have the GPS (or PTP 1588)
receiver to emit GPS System Time, but never share this with downstream
servers, who are configured for UTC (but strangely the leap seconds
never come).  The difference between UTC and GPS System Time is handled
in applications code.  The reason for this approach is so that the bulk
of the system is free from step discontinuities, and only the
interfaces need deal with UTC.


I agree, but as I've tried to point out above I think a better project 
design would have been to use TAI instead of GPS time. PTP works 
natively with TAI, and you can easily convert between he two scales.


Of course it's easy to convert GPS to TAI, and vice versa, but you have 
to take care that more types of conversions are required and implemented 
correctly.


Thanks for your feedback!


Martin
--
Martin Burnicki

MEINBERG Funkuhren GmbH  Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg, 
Andre Hartmann, Heiko Gerstung

Web: http://www.meinberg.de
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] My FOSDEM slides

[Martin Burnicki]

Joseph Gwinn wrote:

Harlan,

On Sun, 01 Mar 2015 20:35:20 +, Harlan Stenn wrote:

Joseph Gwinn writes:

1.  Slide titled Known Bugs (2): Has support for negative leap
seconds been restored in NTP v4?  It wasn't clear.


Not yet.  It's a tradeoff.

We've never needed to delete a leapsecond and depending on who you ask
it will probably never happen.


So long as UTC can do negative leaps, and the Earth is a wobbly clock,
the possibility is always with us.


I absolutely agree. It would have been better to fix this (in case it 
didn't work) than to remove the code which supported negative leap seconds.


In the 7 year interval where no leap second was required/scheduled I 
heard several people saying we might have needed a negative leap second.


Fortunately we didn't, but I still think it's better to prepare for it, 
if possible than just ignore it and even remove existing support.


GPS can deal with it, even IEEE 1344 and C37.118 time codes can, but I'm 
not sure if WWVB can. Anyway, I know the German DCF-77 transmitter has 
no flag defined to announce a negative leap second, so there would be 
major problems if one had to be inserted.



If we add the code to handle it, we increase complexity, potentially add
in a (pretty small) abuse vector (a bad actor could find a way to tell
your system to delete a second), and make some small demands on the test
framework that we want to have.


The abuse vector argument is pretty weak -- the attacker could just as
well add a leap second.  In both cases, one is off by a second.  So, I
would submit that it's support for leap seconds that provides the
attack surface, and the area of that surface is not reduced by
elimination of negative leap seconds.


Again, I strongly agree.


If we don't have it and we end up needing it, that causes different
problems.

There is a parallel issue about folks who cannot or do not upgrade their
software.  Over 1100 issues were addressed between 4.2.6 and 4.2.8 and
people still think 4.2.6 should be good enough for them.


Certainly in my world, changing software is a big deal, because one
needs to rerun all the regression tests.  Changing NTP isn't as big a
deal as changing the OS or the C++ compiler and/or libraries, but still
people are wary.



We've probably fixed about 3000 issues since 4.2.0 and people still run
that.

We don't have numbers for the number of bugs fixed between xntp3.5f,
xntp3-5.86.5, ntp-4.0, ntp-4.1.{0,1,2}, and ntp-4.2.0.

These older releases are still running out there, too.


And don't forget NTPv3 - bet lots of those still run.

Once people get a system to work, they don't tend to go fixing things
that ain't broke.


Yep.

Martin
--
Martin Burnicki

MEINBERG Funkuhren GmbH  Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg, 
Andre Hartmann, Heiko Gerstung

Web: http://www.meinberg.de
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

[LEAPSECS] My FOSDEM slides

[Martin Burnicki]

Hi folks,

I've been asked off list to make the slides of my presentation at FOSDEM 
2015 in Brussels available and post the download link on this list.


So here it is:
https://fosdem.org/2015/schedule/event/technical_aspects_of_leap_second_propagation_and_evaluation/

See the Attachment link.

Martin
--
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH  Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg, 
Andre Hartmann, Heiko Gerstung

Web: http://www.meinberg.de
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] The leap second, deep space and how we keep time -Brooks

[Martin Burnicki]

Warner Losh schrieb:



On Jan 27, 2015, at 7:18 PM, Steve Allen s...@ucolick.org wrote:

On Tue 2015-01-27T21:41:17 +, Matsakis, Demetrios hath writ:

Equally unfortunate is that 30 servers in the NTP pool inserted a
leap second last Dec 31.


There is no action that the ITU-R can take which will change this kind
of misbehavior in these already-deployed systems.

Therefore this is irrelevant to the activities which will happen at
ITU-R WRC-15, and I don't think that misdirecting attention will help
the ITU-R decide what to do.


One of the arguments against leap seconds  are that they are hard to get right.
How many machines inserted an April 31st by mistake? Has an error like that
ever happened? No. Or more likely, has anybody thought it was Feb 29th in
a non-leap year? No. There have been some applications that mistakenly though
Y2K wasn’t a leap year. And there were some issues with the Zune music player.
And there was that unfortunate Feb 30th incident, but that was in a transition
to Gregorian… :)

By comparison, the list of mistaken leap second issues is legion… Isn’t that
relevant to the practicability of any standard under discussion? Or am I missing
your point?


So similarly, if there are faulty DNS or DHCP servers around, should 
these services be abolished instead of fixing the source of the problem? ;-)


Martin
--
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH  Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg, 
Andre Hartmann, Heiko Gerstung

Web: http://www.meinberg.de
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Bulletin C and all that

[Martin Burnicki]

Harlan,

Harlan Stenn wrote:

Yes, there have been bugs (and shortcomings, or at least trade-offs)
with leapsecond propagation in the NTP codebase over the decades.

Upgrading software is a good thing.

That can be ancient release + patches or current supported software
that has patches.

Please see http://queue.acm.org/detail.cfm?id=2717320 for more
information.


Thanks, this is a very nice overview on the history of NTP.

Anyway, we need to distinguish if there's a bug in the NTP code, or if 
ntpd just receives wrong data from faulty refclocks.


Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Bulletin C and all that

[Martin Burnicki]

Tom Van Baak wrote:

The truncated week numbers are a good source for potential errors


I agree, but...


If the current week number is off by more than +-127 then this is ambiguous.


No, it's not ambiguous, it was just a Motorola bug. The wrap is in the spec and 
there's no problem with that. In fact, in 2003 everyone got it right except for 
one(?) model of Motorola GPS timing receiver. So the spec is fine, GPS is fine, 
wrapping is fine. But yes, it is a source for errors, and one engineer at 
Motorola fell for it.


What I mean with ambiguous is that you can't even *display* the 
correct leap second date if there has been no leap second for more than 
the number of weeks covered by the 8 bit week number.


For example, IIRC the GPS satellites sent the same 8 bit WNlsf number of 
the last leap date for seven years when no leap second was scheduled for 
that long time, and after some years you had no chance to expand these 8 
bits unambiguously to 10 or more bits to convert this to the real date 
of the last leap second event.


Of course, if you only evaluate WNlsf and DNt if tls and tlsf differ 
(i.e. a leap second is actually scheduled) then everything is fine since 
the numbers have been updated accordingly. Of course this can still fail 
if you evaluate this in a wrong way, which is probably the problem with 
the GPS receivers in fact doing it wrong.



The only limitation of using an 7- or 8-bit WN field like this is that it 
prevents GPS from announcing a leap second more than 2.5 or 5 years in advance. 
That's not much of a limitation.


Or display the correct date of a leap second more than 2.5 or 5 years in 
the past. ;-)



Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Bulletin C and all that

[Martin Burnicki]

Tim Shepard wrote:




While you're at it, make sure the words June and December are also never written into the standard. 
Only IERS gets to pick the month, and everyone one else follows what they tell us. That goes for 
code, tables, scripts, standards, man pages, etc. We can assume with near-term modest earth 
rotation rate offsets that IERS will pick June and December, but as the offset grows then March and 
September, and then other 8 months are fair game as well. So if you want a standard to be robust, 
please avoid any hard-coded reference to preferred months. Note that the current 
6 month advance warning is also not set in stone so no one standard should ever depend 
on that either.



And also we might someday get more than one leap second scheduled in
the same Bulletin C.  For example, maybe someday a bulletin C might be
issued that says no leap second at the end of June, there will be a
leap second a the end of September, there will be a leap second at the
end of December, no leap second at the end of March, and there will be
a leap second at then end of *next* June.'  Maybe something like all
that in the same bulletin C.  And maybe the next bulletin C shortly
after the beginning of June would reiterate the schedule through next
June and also announce what will happen at the end of September.
(This is all speculation, but it would be nice if code that is written
this year and deployed later doesn't become broken if changes like
that happen some day.)


At least the new leap second code in ntpd 4.2.8 should be able to handle 
this correctly, if I have understood Juergen Perlinger correctly who 
wrote this code.


Martin
--
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH  Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg, 
Andre Hartmann, Heiko Gerstung

Web: http://www.meinberg.de
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Bulletin C and all that

[Martin Burnicki]

Tom Van Baak wrote:

But there have been real bugs due to leap indicators remaining set too
long, leading to bogus leaps at the end of July. So in practice there is
less risk in allowing leaps only in June and December.


Those real bugs are better fixed at their source than worked around in this 
manner. Ok, easy to say and hard to do, I know.

Perhaps leap indicators should not be booleans but small wrapping integers. For 
example GPS makes use the low order 8 bits of the week number for almanac 
version checks. Perhaps the same could be done for leaps, especially if they 
are encoded as a TAI offset integer instead of +/- yes/no booleans.


The truncated week numbers are a good source for potential errors.

Especially the 8 bit WNls week number in the GPS UTC parameter set has 
to be untruncated to 10 or more bits by the firmware to compute the real 
point in time for a leap second.


If the current week number is off by more than +-127 then this is 
ambiguous. This has rolled over several time in the period where no leap 
second had been scheduled for 7 years, and all the time the 8 bit week 
number of the last recent leap second was broadcast.


Martin
--
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH  Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg, 
Andre Hartmann, Heiko Gerstung

Web: http://www.meinberg.de
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] stale leap second information

[Martin Burnicki]

Brooks Harris wrote:


IERS publishes this - Its up to date (includes 2014-07-01) as of today
as I access it (2015-01-12).
http://hpiers.obspm.fr/eoppc/bul/bulc/Leap_Second_History.dat

I'm not sure when it was updated, maybe with their Bullitin C announcement.
ftp://hpiers.obspm.fr/iers/bul/bulc/bulletinc.dat

If would really be good if there was one authoritative soure for this,
and that there was a uniform format.


I've suggested at various occasions that the IERS should be the 
authoritative source for a leap second file.


One reason is that the IERS is the institution which schedules leap 
seconds, and from a global (Earth/countries) point of view, another 
strong reason is that the IERS is an international institution which may 
easier be accepted by some foreign countries than the NIST, which is a 
U.S. institution.


About one year ago I contacted the folks at IERS and asked them to put 
an expiration date into their leap second file since the existing one 
didn't have one.


In the directory listing at http://hpiers.obspm.fr/iers/bul/bulc/ if you 
look at the previous version (Leap_Second_History.datsave from January 
2014) then there was no expiration date, but the current one 
(Leap_Second_History.dat from January 2015) now has it.


Actually there is some discussion with the IERS folks to provide in 
addition a file in NIST format which can directly be used by NTP or 
TZDB, and later by the tzdist servers. Once this has been done there is 
an authoritative source for this file.


Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] stale leap second information

[Martin Burnicki]

Tom Van Baak wrote:

If would really be good if there was one authoritative soure for this,
and that there was a uniform format. Ideally there would be multiple
ways to access it, via text and binary for different architectures. The
might be thought of as a UTC Metadata API, from which various UTC
Metadata Servers could be implemented.

-Brooks


One authoritative source or one uniform format is asking for trouble.


There *is* only one authoritative source, namely the IERS, and there are 
different ways to spread the information:


By bulletin C directly from NIST in the first place.

Based on this, GPS operators feed this information into the satellite 
data to be broadcasted.


The folks at IERS and NIST update their leap second files which are then 
fed into NTP servers, the TZDB, etc.



An NTP pool style list of leap second servers would be a better idea.


That's what the tzdist servers can be. But also these servers need to be 
updated when new information from IERS becomes available. Even if no 
leap second is scheduled, the expiration date of the information needs 
to be updated.



You don't want the same format for an OS as you do for an Arduino, or for a 
compilable piece of software vs. a running binary.


Right, there are conversion routines, but basically all the information 
on leap seconds comes from the same source, IMO. ;-)


Martin
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] stale leap second information

[Martin Burnicki]

Tom Van Baak wrote:

Or use this 40-line text file instead:
 http://hpiers.obspm.fr/iers/bul/bulc/Leap_Second.dat
The one is nice because it includes a File expires on 31 December 2015 notice.


This in not an optimal solution, yet, since parsing a string like this 
can be error prone. The approach used by the NIST format is better since 
it contains the time stamp in a machine readable format and in addition 
in a human readable format. Just my 2 cents.


And, applications evaluating the expiration date should emit a warning 
quite some time *before* the expiration date becomes current.


For example, the current versions of the files contain the leap second 
date at end of June, and the files expire at the end of December, when 
(at least in theory) another leap second might be scheduled.


If a warning for the expired file is only emitted at December 28 or 30 
then admins don't have much time to care about an update of this file. 
If they are warned earlier then they can carefully look at this, and an 
updated version of the files should be available roughly 6 months before 
the expiration of the previous one.


Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Improvements since 2012

[Martin Burnicki]

Folks, originally I wrote this email about 9 hours ago, but sent it 
using a different email address which is not subscribed to the list. So 
apparingly the first email was blocked, and I'm sending it once more.


Ask Bjørn Hansen wrote:

Steve Allen s...@ucolick.org wrote:


[...]

NTP and the NTP pool are more robust too, so that will be better.


What's better about NTP and the NTP Pool since 2012?

I try not to mean that as a retorical question; I'm interested -- it
would be good news. I don't know what's better about them and in the
case of the pool I should probably know. :-)


The main problem with the last leap seconds was due to implementation 
faults in the Linux kernel, when the leap second was handled.


At the leap second in June 2012 this could cause a load spike (and thus 
increased power consumption). See:

https://lkml.org/lkml/2012/7/1/27

At the leap second at the end of 2008 it was even worse and could cause 
a deadlock of the Linux kernel. See:

https://lkml.org/lkml/2009/1/2/373

I don't know exactly from memory in which versions these bugs have been 
fixed, but I'll have a closer look at this the next days.


If I remember correctly the Linux kernel maintainers have very carefully 
checked the leap second handling code, and provided some testing 
routines for it.


So unless a new bug has been introduced I'd expect this works now in 
current kernel versions.



In ntpd there has been a bug which could cause a leap second loop when 
a leap second warning was propagated in a loop and could cause another 
leap second warning for the end of the next month after a real leap second.


If I remember correctly then this happened only under certain 
conditions, i.e. with certain configurations, and as far as I know there 
has been a patch in 4.2.8 which avoids this.


Anyway, if the pool's NTP server monitoring software checks the leap 
second warning bit and detects a leap second warning is still lit after 
the leap second has already occurred you have 1 month time to fix this, 
which means you have just to restart ntpd on the affected machine.


As always, this depends on the versions of the kernel and ntpd you are 
running.


Martin

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] the big artillery

[Martin Burnicki]

Warner,

Warner Losh wrote:


On Oct 31, 2014, at 4:17 AM, Martin Burnicki martin.burni...@meinberg.de 
wrote:


Magnus Danielson wrote:

On 10/31/2014 02:49 AM, Sanjeev Gupta wrote:

Give it a new name, please.  Independent of what the fundamental
unit is.


TAI and UTC already exists, but the use of TAI has been actively
discouraged.


Huuh?

Just recently PTP/IEEE1588 has been specified to use TAI timestamps by 
default, and provide a UTC offset as parameter.

As far as I can see it's easy to derive legacy UTC from TAI unambiguously if 
you know the current offset, and if you have a leap second table available this also 
works for timestamps from the past, at least after 1970. So what could be the reason 
*not* to use TAI?


BIPM (or their successors, I can never keep all the reorgs straight), the 
owners of TAI, have discouraged it. The reasons are that while it is similar to 
UTC, it differs in some technical ways. TAI and UTC have a fixed offset 
relationship, it is true. However, UTC is computed in real time (with several 
varieties to choose from if you care about the nano-seconds), but TAI is a 
retrospective timescale that’s not computed until after the fact. I get the 
feeling that the BIPM want TAI to be their baby, free from “production” 
concerns that UTC  has to deal with

IEEE isn’t part of BIPM, so they are free to do what they want, and they make a 
contrary recommendation. But if you look closely, they aren’t recommending 
using TAI, as BIPM defines it, they are using the TAI second labeling for this 
real-time realized timescale. So this is a real-time realization of a timescale 
whose seconds are numbered like TAI rather than like UTC. It isn’t a TAI 
timestamp, since technically those have to be compute after the fact from the 
raw data rather than done in real time. But it is a timestamp using the TAI 
conventions for labeling of seconds. The difference is subtle, and for PTP 
makes no difference at all, but does exist.


Thanks for this information. I didn't know these details before.


The trouble is that those that wants a TAI-like time-scale
sometimes needs to comply to UTC needs, and for a number of reasons they
have difficulty in using it, so they want to make UTC a TAI-timescale.


The naming of a possible future UTC-like time scale without leap seconds is a 
different topic, though, and I fully agree with Harlan's and Sanjeev's recent 
postings.


Rules change all the time as do the details (UTC pre 1972 is significantly 
different than post 1972 for everything except the tracking UT1 attribute), 
sometimes the name changes, other times no. Sometimes the change matters to a 
lot of people, other times not so many (like the black body correction 
introduced in the 1990’s). But that’s a different set of posts, eh?


Agreed.

Martin
--
Martin Burnicki

MEINBERG Funkuhren GmbH  Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg, 
Andre Hartmann, Heiko Gerstung

Web: http://www.meinberg.de
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

[LEAPSECS] Next leap second approaching ... ;-)

[Martin Burnicki]

Next leap second approaching ... ;-)
http://xkcd.com/162/

Martin
--
Martin Burnicki

MEINBERG Funkuhren GmbH  Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 (0)5281 9309-14
Fax: +49 (0)5281 9309-30

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg, 
Andre Hartmann, Heiko Gerstung

Web: http://www.meinberg.de
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs