Re: [LEAPSECS] Windows Server 2019

[Steve Allen]

On Fri 2018-07-20T14:23:38+ Nero Imhard hath writ:
> Leap second smearing by an NTP server effectively means that it uses
> a different time. Without a way to indicate, within the protocol,
> what time a server uses (and there isn't because UTC is implicit) this
> seems a really bad idea.
>
> On another level, I regard it as some sort of "technical betrayal"
> which erodes trust in protocols and standards, rendering them less
> useful than they could be.
>
> Isolate your workarounds and don't let them pollute your environment.

In the arena of leap seconds, technical betrayal and dilution of the
protocols and standards began before the inception date.

Gernot Winkler of USNO is credited as one of the people who suggested
that the leap second could be the solution to the problem.  Gernot
Winkler reported that radio broadcast time signals operated by the
USNO would follow TAI and not implement leap seconds, while at the
same time the USNO Nautical Almanac tabulated events in Ephemeris Time
and UT1.  This is clearly the USNO making use of the "two time scales"
scheme that since 1948 astronomers had been saying would be necessary,
and not following international protocols and standards.
https://www.ucolick.org/~sla/leapsecs/leapincept.html
Gernot Winkler was co-author of the 1970 report to the IAU that said
leap seconds would cause problems for automated navigation systems.
http://adsabs.harvard.edu/abs/1970IAUTA..14..343Z
At the 1970 IAU Comm 31 meeting Gernot Winkler was acting president
where in the transcript it is clear that at least one paragraph has
been omitted about the response in the IAU meeting when the CCIR
decision to implement leap seconds was announced.
http://adsabs.harvard.edu/abs/1971IAUTB..14..193W
At that same meeting IAU Comm 31 was led to yield that they had no
influence over the leap seconds that the CCIR had instituted, and IAU
Comm 31 was pressed to produce a statement declaring that leap seconds
were "the optimum solution."
http://adsabs.harvard.edu/abs/1971IAUTB..14..198W

All of the above strike me as "something is seriously wrong here."

Looking deeper into the history and memoirs by folks who were involved
it becomes clear that the inception of leap seconds was the
culmination of a 20 year game of international regulatory and
scientific agency pinball.  After the CCIR introduced them that game
continued for another 10 years as other agencies and governments were
led to approve the notion of UTC with leap seconds using words like
"parfaitment recommandable".

The game started when the IAU produced a recommendation for radio
broadcast time signals that was not implementable within available
resources.  Another agency produced a recommendation based on criteria
which were relevant to its particular mission.  At subsequent meetings
of other agencies the agreements made by previous agencies were
represented as the basis for further recommendations.  One of the
agreements used in this game was achieved not at an international
meeting, but in the living room of one of the delegates.  This process
continued until CCIR was led to approve leap seconds without any
accompanying documentation about the technical details of how they
would be implemented.

Several documents indicate that the preparatory meetings prior to the
international assemblies discussed that changes to national laws would
be required in order to broadcast a purely atomic time scale.
I have found nothing that directly explains why it was repeatedly
deemed impossible for any of these agencies to explain and recommend
the existence of two kinds of time scales, but it seems clear that
the legal considerations led toward the notion of a compromise.
I surmise that the folks who wanted to broadcast atomic frequency were
in too much of a hurry to wait for a critical mass of nations to
legislate and allow their national metrology agencies to recognize and
broadcast purely atomic time, so they took a shortcut.
Taking that shortcut meant going to the CCIR with a proposal that did
not explain the consequences and mechanisms, neither to the delegates
who voted nor to any implementors of systems using UTC.

I am not sure whether these kinds of interaction among international
regulatory and scientific agencies are perfectly normal business as
usual, or an example of systematic abuse spanning over decades.  I am
sure that the result was dysfunctional operational systems (which
still lack solid technical underpinnings) and a trail of technical
folks who had withdrawn from the arena to have their agencies continue
using time scales that were not UTC with leap seconds.  I am also sure
that after an ensemble of international agencies have been persuaded
to approve something it becomes very difficult to unapprove it even if
its initial approval was achieved under dubious circumstances with
serious unaddressed technical questions.

So we have betrayal, eroded trust, and reduced usefulness because some
folks wanted to take what looked like 

Re: [LEAPSECS] Windows Server 2019

[Stephen Scott]

On 2018-07-20 14:16, Warner Losh wrote:



On Fri, Jul 20, 2018, 12:05 PM Stephen Scott 
mailto:stephensc...@videotron.ca>> wrote:


I would agree and I am not a fan of leap second smearing.
A.)  For an application that depends on an accurate frequency
reference smearing over 24 hours represents about a 11 ppm
frequency shift. This exceeds the tolerance specified for some
audio/video applications.
B.) It is not standardized, so maybe it is a smear of maybe its not.

There is a lack of standards and that is probably the biggest
source of most leap second problems.

I have been searching for standards, documentation on how leap
seconds must be applied in local jurisdictions.

I find it strange that for time in local time zones the time
points are shifted by the UTC offset for all 86400 seconds of the day.
But that the one leap second is treated differently and is not
shifted by the UTC offset.
The consequence is that the leap second is being incorporated at
critical times of the day in many time zones.

While there is no perfect answer, it seems that Microsoft Azure
servers got it right for the last one, incorporating the leap
second just before midnight local time.


Unless you are at UTC+0, I don't see how this can be right... Leap 
seconds happen during the day for most time zones...

What I am asking is WHY.
Where is the standard for that?
Or at least some document that specifies that?

Stephen


Warner

Control, on Time and in Sync
Stephen Scott



On 2018-07-20 08:03, Nero Imhard wrote:


scs...@eskimo.com  schreef op
2018-07-20 11:35:


The question of what happens if you try to run a leapsec-aware
kernel downstream of a smearing NTP server is an interesting one.
My preferred answer is "Don't do that."

If that translates to "don't smear ntp" I could not agree more.
Smearing is catering to those who won't clean up their act,
causing trouble for those who try to do the right thing.
N


___
LEAPSECS mailing list
LEAPSECS@leapsecond.com 
https://pairlist6.pair.net/mailman/listinfo/leapsecs





Virus-free. www.avast.com




<#m_-5058450252262385288_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com 
https://pairlist6.pair.net/mailman/listinfo/leapsecs





---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Windows Server 2019

[Warner Losh]

On Fri, Jul 20, 2018, 12:05 PM Stephen Scott 
wrote:

> I would agree and I am not a fan of leap second smearing.
> A.)  For an application that depends on an accurate frequency reference
> smearing over 24 hours represents about a 11 ppm frequency shift. This
> exceeds the tolerance specified for some audio/video applications.
> B.) It is not standardized, so maybe it is a smear of maybe its not.
>
> There is a lack of standards and that is probably the biggest source of
> most leap second problems.
>
> I have been searching for standards, documentation on how leap seconds
> must be applied in local jurisdictions.
>
> I find it strange that for time in local time zones the time points are
> shifted by the UTC offset for all 86400 seconds of the day.
> But that the one leap second is treated differently and is not shifted by
> the UTC offset.
> The consequence is that the leap second is being incorporated at critical
> times of the day in many time zones.
>
> While there is no perfect answer, it seems that Microsoft Azure servers
> got it right for the last one, incorporating the leap second just before
> midnight local time.
>

Unless you are at UTC+0, I don't see how this can be right... Leap seconds
happen during the day for most time zones...

Warner

> Control, on Time and in Sync
> Stephen Scott
>
>
>
> On 2018-07-20 08:03, Nero Imhard wrote:
>
> scs...@eskimo.com schreef op 2018-07-20 11:35:
>
> The question of what happens if you try to run a leapsec-aware
> kernel downstream of a smearing NTP server is an interesting one.
> My preferred answer is "Don't do that."
>
>
> If that translates to "don't smear ntp" I could not agree more.
> Smearing is catering to those who won't clean up their act,
> causing trouble for those who try to do the right thing.
>
> N
>
>
> ___
> LEAPSECS mailing 
> listLEAPSECS@leapsecond.comhttps://pairlist6.pair.net/mailman/listinfo/leapsecs
>
>
>
>
> 
>  Virus-free.
> www.avast.com
> 
> <#m_-5058450252262385288_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
> ___
> LEAPSECS mailing list
> LEAPSECS@leapsecond.com
> https://pairlist6.pair.net/mailman/listinfo/leapsecs
>
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Windows Server 2019

[Stephen Scott]

I would agree and I am not a fan of leap second smearing.
A.)  For an application that depends on an accurate frequency reference 
smearing over 24 hours represents about a 11 ppm frequency shift. This 
exceeds the tolerance specified for some audio/video applications.

B.) It is not standardized, so maybe it is a smear of maybe its not.

There is a lack of standards and that is probably the biggest source of 
most leap second problems.


I have been searching for standards, documentation on how leap seconds 
must be applied in local jurisdictions.


I find it strange that for time in local time zones the time points are 
shifted by the UTC offset for all 86400 seconds of the day.
But that the one leap second is treated differently and is not shifted 
by the UTC offset.
The consequence is that the leap second is being incorporated at 
critical times of the day in many time zones.


While there is no perfect answer, it seems that Microsoft Azure servers 
got it right for the last one, incorporating the leap second just before 
midnight local time.


Control, on Time and in Sync
Stephen Scott



On 2018-07-20 08:03, Nero Imhard wrote:


scs...@eskimo.com schreef op 2018-07-20 11:35:


The question of what happens if you try to run a leapsec-aware
kernel downstream of a smearing NTP server is an interesting one.
My preferred answer is "Don't do that."

If that translates to "don't smear ntp" I could not agree more.
Smearing is catering to those who won't clean up their act,
causing trouble for those who try to do the right thing.
N


___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs




---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Windows Server 2019

[Martin Burnicki]

Nero Imhard wrote:
> Martin Burnicki schreef op 2018-07-20 12:52:
> 
>> Nero Imhard wrote:
>>> If that translates to "don't smear ntp" I could not agree more.
>>> Smearing is catering to those who won't clean up their act,
>>> causing trouble for those who try to do the right thing.
>>
>> Please don't blame ntpd for smearing leap seconds.
>  
> I don't. ntpd would be the victim here.

;-)
  
>> Leap second smearing by an NTP server is just an optional way to avoid
>> these problems because there's no better way to get around it.
>  
> Leap second smearing by an NTP server effectively means that it uses
> a different time. Without a way to indicate, within the protocol,
> what time a server uses (and there isn't because UTC is implicit) this
> seems a really bad idea.

Basically I agree with you, but ...

> On another level, I regard it as some sort of "technical betrayal"
> which erodes trust in protocols and standards, rendering them less
> useful than they could be.

Yes, the NTP protocol has been designed to work with UTC time stamps.

I know there are also folks who let their own NTP server transmit pure
TAI or raw GPS time instead of UTC, since some devices on their network
need this.

Also, even the NIST operates an NTP server that provides UT1 rather than
UTC.

So for specific cases, in a closed environment, it's OK IMO.
  
> Isolate your workarounds and don't let them pollute your environment. 

Especially, there shouldn't be any public/pool servers that smear leap
seconds.

If you are administrator for a huge company network you are eventually
very happy if you can just let the company's NTP server smear a leap
second, and none of the clients or applications encounter any problem.

If you don't want it, don't use it. ;-)

The problem is if there are folks that don't know (exactly) what they
are doing, e.g. use as reference one of Google's public servers that are
known to smear leap seconds.

Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Windows Server 2019

[Nero Imhard]

Martin Burnicki schreef op 2018-07-20 12:52:

> Nero Imhard wrote: 
> 
>> If that translates to "don't smear ntp" I could not agree more.
>> Smearing is catering to those who won't clean up their act,
>> causing trouble for those who try to do the right thing.
> 
> Please don't blame ntpd for smearing leap seconds.

I don't. ntpd would be the victim here. 

> Leap second smearing by an NTP server is just an optional way to avoid 
> these problems because there's no better way to get around it.

Leap second smearing by an NTP server effectively means that it uses 
a different time. Without a way to indicate, within the protocol, 
what time a server uses (and there isn't because UTC is implicit) this 
seems a really bad idea. 

On another level, I regard it as some sort of "technical betrayal" 
which erodes trust in protocols and standards, rendering them less 
useful than they could be. 

Isolate your workarounds and don't let them pollute your environment.  

N___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Windows Server 2019

[Warner Losh]

On Fri, Jul 20, 2018 at 6:03 AM, Nero Imhard  wrote:

> scs...@eskimo.com schreef op 2018-07-20 11:35:
>
> The question of what happens if you try to run a leapsec-aware
> kernel downstream of a smearing NTP server is an interesting one.
> My preferred answer is "Don't do that."
>
>
> If that translates to "don't smear ntp" I could not agree more.
> Smearing is catering to those who won't clean up their act,
> causing trouble for those who try to do the right thing.
>

I view smeared leap seconds as a wholesale repudiation of the whole concept
of leap seconds. They are too hard to get right, despite decades of trying,
so screw those time guys, we'll paper over their horrible design...

Of course, I came to this conclusion before smeared leapseconds trying to
get systems to behave perfectly. The old "but it's only a second" refrain
when I filed bug reports and talked to management showed nobody cared.

Warner
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Windows Server 2019

[Martin Burnicki]

Nero Imhard wrote:
> scs...@eskimo.com schreef op 2018-07-20 11:35:
> 
>> The question of what happens if you try to run a leapsec-aware
>> kernel downstream of a smearing NTP server is an interesting one.
>> My preferred answer is "Don't do that."
>  
> If that translates to "don't smear ntp" I could not agree more.
> Smearing is catering to those who won't clean up their act,
> causing trouble for those who try to do the right thing.

Please don't blame ntpd for smearing leap seconds.

The leap second is inserted/handled by the kernel, and that's basically
the way it should be.

However, this is often done in a way that applications may become
confused if the system time is simply stepped back by 1 s.

If you used any other time synchronization software which just passes a
leap second announcement to the kernel then the applications will have
the same problems.

Leap second smearing by an NTP server is just an optional way to avoid
these problems because there's no better way to get around it.


Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Windows Server 2019

[Martin Burnicki]

Steve Summit wrote:
> Steve Allen wrote:
>>> as seen on slashdot and more, Windows Server 2019 will support
>>> leap seconds.
> 
> I don't often say this, but: Good for Microsoft.
> 
> Despite the several, well known, significant difficulties, I have
> been on a quixotic on-and-off quest to implement "proper" leap
> second handling for Linux.  (As some will recall, on 2016-12-31
> I managed to post a message to this list at 23:59:60 UTC, with a
> Date: line to prove it.)  If anyone feels shamed into action by
> Microsoft's announcement, and wants to try things out under Linux,
> please feel free to pester me into finally publishing that work
> publicly. :-)
> 
> Stephen Colebourne wrote:
>> ...it needs to be possible for any of these setups to be valid
>> (and maybe more options too):
>>
>> - network clock returns UTC, OS handles UTC, application/framework handles 
>> UTC
>> - network clock returns UTC, OS handles UTC, application/framework smears
>> - network clock returns UTC, OS smears, application/framework unaware
> 
> The modified Linux kernel that I have implements these sorts of
> choices via clock_gettime().  The new clkid value CLOCK_UTC gives
> you true UTC, the new clkid value CLOCK_SMEARED gives you smeared
> time, and the existing clkid value CLOCK_REALTIME currently gives
> you whatever we used to put up with (basically a repeated second),
> but could/would be changed to return the same as CLOCK_SMEARED
> once it was confidently proved to be working.

This sounds like a real step forward to flexible leap second handling by
the kernel, and IMO it would be good if this could make it into the kernel.

> The question of what happens if you try to run a leapsec-aware
> kernel downstream of a smearing NTP server is an interesting one.
> My preferred answer is "Don't do that."  The failure modes if you
> did wouldn't be *too* bad, but "proper" handling of CLOCK_UTC in
> that case would of course require an unsmearing algorithm, which
> I for one am not interested in writing.

I think the admins had to take care that they use a reliable clock
source. If you install a GPS receiver which outputs faulty time (e.g. to
GPS week number rollovers) or output faulty leap second warnings then
you may end up with a wrong system time. Similarly, if you chose a
smearing NTP server even though you don't want the incoming time to be
smeared.

>> Getting the smear off the network and into the OS will undoubtably
>> be useful in some use cases, but they are not the only use cases.
> 
> Indeed.

However, this had to be made public so that software developers become
aware of these features and what the differences are, so they can call
clock_gettime() with the clock ID that is best suited for their
application.

If they are not aware of the differences in the clock models then their
application will probably not work as expected anyway.


Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Windows Server 2019

[Nero Imhard]

scs...@eskimo.com schreef op 2018-07-20 11:35:

> The question of what happens if you try to run a leapsec-aware
> kernel downstream of a smearing NTP server is an interesting one.
> My preferred answer is "Don't do that."

If that translates to "don't smear ntp" I could not agree more. 
Smearing is catering to those who won't clean up their act, 
causing trouble for those who try to do the right thing. 

N___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] Windows Server 2019

[Steve Summit]

Steve Allen wrote:
>> as seen on slashdot and more, Windows Server 2019 will support
>> leap seconds.

I don't often say this, but: Good for Microsoft.

Despite the several, well known, significant difficulties, I have
been on a quixotic on-and-off quest to implement "proper" leap
second handling for Linux.  (As some will recall, on 2016-12-31
I managed to post a message to this list at 23:59:60 UTC, with a
Date: line to prove it.)  If anyone feels shamed into action by
Microsoft's announcement, and wants to try things out under Linux,
please feel free to pester me into finally publishing that work
publicly. :-)

Stephen Colebourne wrote:
> ...it needs to be possible for any of these setups to be valid
> (and maybe more options too):
>
> - network clock returns UTC, OS handles UTC, application/framework handles UTC
> - network clock returns UTC, OS handles UTC, application/framework smears
> - network clock returns UTC, OS smears, application/framework unaware

The modified Linux kernel that I have implements these sorts of
choices via clock_gettime().  The new clkid value CLOCK_UTC gives
you true UTC, the new clkid value CLOCK_SMEARED gives you smeared
time, and the existing clkid value CLOCK_REALTIME currently gives
you whatever we used to put up with (basically a repeated second),
but could/would be changed to return the same as CLOCK_SMEARED
once it was confidently proved to be working.

The question of what happens if you try to run a leapsec-aware
kernel downstream of a smearing NTP server is an interesting one.
My preferred answer is "Don't do that."  The failure modes if you
did wouldn't be *too* bad, but "proper" handling of CLOCK_UTC in
that case would of course require an unsmearing algorithm, which
I for one am not interested in writing.

> Getting the smear off the network and into the OS will undoubtably
> be useful in some use cases, but they are not the only use cases.

Indeed.
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs